2024-03-27T10:29:07.246 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-27T10:29:07.246 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_rishabh@teuthology --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/suites/iogen}} --name rishabh-2024-03-27_05:27:11-fs-wip-rishabh-testing-20240326.131558-testing-default-smithi --archive /home/teuthworker/archive/rishabh-2024-03-27_05:27:11-fs-wip-rishabh-testing-20240326.131558-testing-default-smithi/7625781 --verbose -- /home/teuthworker/archive/rishabh-2024-03-27_05:27:11-fs-wip-rishabh-testing-20240326.131558-testing-default-smithi/7625781/orig.config.yaml 2024-03-27T10:29:07.304 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-27T10:29:07.396 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-03-27_05:27:11-fs-wip-rishabh-testing-20240326.131558-testing-default-smithi/7625781 branch: wip-rishabh-testing-20240326.131558 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/suites/iogen}} email: null first_in_suite: false job_id: '7625781' kernel: flavor: default kdb: true sha1: 1df0c7e8a2354c42d738e5454ba83ea7332deac2 last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: rishabh-2024-03-27_05:27:11-fs-wip-rishabh-testing-20240326.131558-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-rishabh-testing-20240326.131558 ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - slow metadata IO - SLOW_OPS - slow request sha1: 155268c4e432a12433aa833f174f9fe3b1016ae0 subvols: create: 2 subvol_options: --namespace-isolated ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt dry_run: true install: ceph: flavor: default sha1: 155268c4e432a12433aa833f174f9fe3b1016ae0 kclient: client.0: mount_subvol_num: 1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-testing-20240326.131558 sha1: 155268c4e432a12433aa833f174f9fe3b1016ae0 owner: scheduled_rishabh@teuthology priority: 75 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 6931 sha1: 155268c4e432a12433aa833f174f9fe3b1016ae0 sleep_before_teardown: 0 subset: 80/512 suite: fs suite_branch: wip-rishabh-testing-20240326.131558 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_155268c4e432a12433aa833f174f9fe3b1016ae0/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 155268c4e432a12433aa833f174f9fe3b1016ae0 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi097.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNlLz08IArfvIjetErgvQ+5YKaLSe8NPzl4LVgM6pSqZOod3t3W3hh5h6dQ+in5EMm6YNDwtMDLymdBFBgPG6uA= smithi114.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKCZcHiv9DXJnWWjv/t9fR+w9U+PfTdXPwrECaQtupN73Qxfc/JcEfWcTUNP8swunI9TpqdmVDfG7D3A2/EP31M= smithi177.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJwXtJQVOIeDN2PKjI9hI9Oa4+xqJ7yG+AvSuY1u9Xl/rYTP4QQUptvjx9OC2DqSRqOojIU4egKICDfXbjLAC6A= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - ceph-fuse: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - suites/iogen.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-27_05:27:11 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-03-27T10:29:07.396 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_155268c4e432a12433aa833f174f9fe3b1016ae0/qa; will attempt to use it 2024-03-27T10:29:07.397 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_155268c4e432a12433aa833f174f9fe3b1016ae0/qa/tasks 2024-03-27T10:29:07.397 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-27T10:29:07.398 INFO:teuthology.task.internal:Checking packages... 2024-03-27T10:29:07.424 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '155268c4e432a12433aa833f174f9fe3b1016ae0' 2024-03-27T10:29:07.424 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-27T10:29:07.424 INFO:teuthology.packaging:ref: None 2024-03-27T10:29:07.424 INFO:teuthology.packaging:tag: None 2024-03-27T10:29:07.424 INFO:teuthology.packaging:branch: wip-rishabh-testing-20240326.131558 2024-03-27T10:29:07.424 INFO:teuthology.packaging:sha1: 155268c4e432a12433aa833f174f9fe3b1016ae0 2024-03-27T10:29:07.424 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-rishabh-testing-20240326.131558 2024-03-27T10:29:07.658 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2478.g155268c4 2024-03-27T10:29:07.660 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-27T10:29:07.668 INFO:teuthology.task.internal:no buildpackages task found 2024-03-27T10:29:07.668 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-27T10:29:07.690 INFO:teuthology.task.internal:Saving configuration 2024-03-27T10:29:07.710 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-27T10:29:07.718 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-27T10:29:07.745 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi097.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-03-27_05:27:11-fs-wip-rishabh-testing-20240326.131558-testing-default-smithi/7625781', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-27 10:24:05.053643', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNlLz08IArfvIjetErgvQ+5YKaLSe8NPzl4LVgM6pSqZOod3t3W3hh5h6dQ+in5EMm6YNDwtMDLymdBFBgPG6uA='} 2024-03-27T10:29:07.768 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi114.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-03-27_05:27:11-fs-wip-rishabh-testing-20240326.131558-testing-default-smithi/7625781', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-27 10:24:05.050943', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKCZcHiv9DXJnWWjv/t9fR+w9U+PfTdXPwrECaQtupN73Qxfc/JcEfWcTUNP8swunI9TpqdmVDfG7D3A2/EP31M='} 2024-03-27T10:29:07.791 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi177.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-03-27_05:27:11-fs-wip-rishabh-testing-20240326.131558-testing-default-smithi/7625781', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-27 10:24:05.052479', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJwXtJQVOIeDN2PKjI9hI9Oa4+xqJ7yG+AvSuY1u9Xl/rYTP4QQUptvjx9OC2DqSRqOojIU4egKICDfXbjLAC6A='} 2024-03-27T10:29:07.791 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-27T10:29:07.799 INFO:teuthology.task.internal:roles: ubuntu@smithi097.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-03-27T10:29:07.799 INFO:teuthology.task.internal:roles: ubuntu@smithi114.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-03-27T10:29:07.799 INFO:teuthology.task.internal:roles: ubuntu@smithi177.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-03-27T10:29:07.799 INFO:teuthology.run_tasks:Running task console_log... 2024-03-27T10:29:07.951 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fc058be9160>, signals=[15]) 2024-03-27T10:29:07.952 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-27T10:29:07.959 INFO:teuthology.task.internal:Opening connections... 2024-03-27T10:29:07.959 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi097.front.sepia.ceph.com 2024-03-27T10:29:07.961 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-27T10:29:08.056 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi114.front.sepia.ceph.com 2024-03-27T10:29:08.057 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-27T10:29:08.147 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi177.front.sepia.ceph.com 2024-03-27T10:29:08.148 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi177.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-27T10:29:08.235 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-27T10:29:08.243 DEBUG:teuthology.orchestra.run.smithi097:> uname -m 2024-03-27T10:29:08.261 INFO:teuthology.orchestra.run.smithi097.stdout:x86_64 2024-03-27T10:29:08.261 DEBUG:teuthology.orchestra.run.smithi097:> cat /etc/os-release 2024-03-27T10:29:08.318 INFO:teuthology.orchestra.run.smithi097.stdout:NAME="CentOS Stream" 2024-03-27T10:29:08.318 INFO:teuthology.orchestra.run.smithi097.stdout:VERSION="9" 2024-03-27T10:29:08.318 INFO:teuthology.orchestra.run.smithi097.stdout:ID="centos" 2024-03-27T10:29:08.318 INFO:teuthology.orchestra.run.smithi097.stdout:ID_LIKE="rhel fedora" 2024-03-27T10:29:08.318 INFO:teuthology.orchestra.run.smithi097.stdout:VERSION_ID="9" 2024-03-27T10:29:08.318 INFO:teuthology.orchestra.run.smithi097.stdout:PLATFORM_ID="platform:el9" 2024-03-27T10:29:08.318 INFO:teuthology.orchestra.run.smithi097.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-27T10:29:08.318 INFO:teuthology.orchestra.run.smithi097.stdout:ANSI_COLOR="0;31" 2024-03-27T10:29:08.319 INFO:teuthology.orchestra.run.smithi097.stdout:LOGO="fedora-logo-icon" 2024-03-27T10:29:08.319 INFO:teuthology.orchestra.run.smithi097.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-27T10:29:08.319 INFO:teuthology.orchestra.run.smithi097.stdout:HOME_URL="https://centos.org/" 2024-03-27T10:29:08.319 INFO:teuthology.orchestra.run.smithi097.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-27T10:29:08.319 INFO:teuthology.orchestra.run.smithi097.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-27T10:29:08.319 INFO:teuthology.orchestra.run.smithi097.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-27T10:29:08.319 INFO:teuthology.lock.ops:Updating smithi097.front.sepia.ceph.com on lock server 2024-03-27T10:29:08.345 DEBUG:teuthology.orchestra.run.smithi114:> uname -m 2024-03-27T10:29:08.362 INFO:teuthology.orchestra.run.smithi114.stdout:x86_64 2024-03-27T10:29:08.363 DEBUG:teuthology.orchestra.run.smithi114:> cat /etc/os-release 2024-03-27T10:29:08.419 INFO:teuthology.orchestra.run.smithi114.stdout:NAME="CentOS Stream" 2024-03-27T10:29:08.420 INFO:teuthology.orchestra.run.smithi114.stdout:VERSION="9" 2024-03-27T10:29:08.420 INFO:teuthology.orchestra.run.smithi114.stdout:ID="centos" 2024-03-27T10:29:08.420 INFO:teuthology.orchestra.run.smithi114.stdout:ID_LIKE="rhel fedora" 2024-03-27T10:29:08.420 INFO:teuthology.orchestra.run.smithi114.stdout:VERSION_ID="9" 2024-03-27T10:29:08.420 INFO:teuthology.orchestra.run.smithi114.stdout:PLATFORM_ID="platform:el9" 2024-03-27T10:29:08.420 INFO:teuthology.orchestra.run.smithi114.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-27T10:29:08.420 INFO:teuthology.orchestra.run.smithi114.stdout:ANSI_COLOR="0;31" 2024-03-27T10:29:08.420 INFO:teuthology.orchestra.run.smithi114.stdout:LOGO="fedora-logo-icon" 2024-03-27T10:29:08.420 INFO:teuthology.orchestra.run.smithi114.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-27T10:29:08.421 INFO:teuthology.orchestra.run.smithi114.stdout:HOME_URL="https://centos.org/" 2024-03-27T10:29:08.421 INFO:teuthology.orchestra.run.smithi114.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-27T10:29:08.421 INFO:teuthology.orchestra.run.smithi114.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-27T10:29:08.421 INFO:teuthology.orchestra.run.smithi114.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-27T10:29:08.421 INFO:teuthology.lock.ops:Updating smithi114.front.sepia.ceph.com on lock server 2024-03-27T10:29:08.445 DEBUG:teuthology.orchestra.run.smithi177:> uname -m 2024-03-27T10:29:08.463 INFO:teuthology.orchestra.run.smithi177.stdout:x86_64 2024-03-27T10:29:08.463 DEBUG:teuthology.orchestra.run.smithi177:> cat /etc/os-release 2024-03-27T10:29:08.523 INFO:teuthology.orchestra.run.smithi177.stdout:NAME="CentOS Stream" 2024-03-27T10:29:08.524 INFO:teuthology.orchestra.run.smithi177.stdout:VERSION="9" 2024-03-27T10:29:08.524 INFO:teuthology.orchestra.run.smithi177.stdout:ID="centos" 2024-03-27T10:29:08.524 INFO:teuthology.orchestra.run.smithi177.stdout:ID_LIKE="rhel fedora" 2024-03-27T10:29:08.524 INFO:teuthology.orchestra.run.smithi177.stdout:VERSION_ID="9" 2024-03-27T10:29:08.524 INFO:teuthology.orchestra.run.smithi177.stdout:PLATFORM_ID="platform:el9" 2024-03-27T10:29:08.524 INFO:teuthology.orchestra.run.smithi177.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-27T10:29:08.524 INFO:teuthology.orchestra.run.smithi177.stdout:ANSI_COLOR="0;31" 2024-03-27T10:29:08.524 INFO:teuthology.orchestra.run.smithi177.stdout:LOGO="fedora-logo-icon" 2024-03-27T10:29:08.524 INFO:teuthology.orchestra.run.smithi177.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-27T10:29:08.524 INFO:teuthology.orchestra.run.smithi177.stdout:HOME_URL="https://centos.org/" 2024-03-27T10:29:08.524 INFO:teuthology.orchestra.run.smithi177.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-27T10:29:08.525 INFO:teuthology.orchestra.run.smithi177.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-27T10:29:08.525 INFO:teuthology.orchestra.run.smithi177.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-27T10:29:08.525 INFO:teuthology.lock.ops:Updating smithi177.front.sepia.ceph.com on lock server 2024-03-27T10:29:08.545 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-27T10:29:08.555 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-27T10:29:08.563 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-27T10:29:08.563 DEBUG:teuthology.orchestra.run.smithi097:> test '!' -e /home/ubuntu/cephtest 2024-03-27T10:29:08.565 DEBUG:teuthology.orchestra.run.smithi114:> test '!' -e /home/ubuntu/cephtest 2024-03-27T10:29:08.568 DEBUG:teuthology.orchestra.run.smithi177:> test '!' -e /home/ubuntu/cephtest 2024-03-27T10:29:08.583 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-27T10:29:08.590 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-27T10:29:08.590 DEBUG:teuthology.orchestra.run.smithi097:> test -z $(ls -A /var/lib/ceph) 2024-03-27T10:29:08.622 DEBUG:teuthology.orchestra.run.smithi114:> test -z $(ls -A /var/lib/ceph) 2024-03-27T10:29:08.624 DEBUG:teuthology.orchestra.run.smithi177:> test -z $(ls -A /var/lib/ceph) 2024-03-27T10:29:08.713 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-27T10:29:08.789 INFO:teuthology.run_tasks:Running task kernel... 2024-03-27T10:29:08.804 INFO:teuthology.task.kernel:normalize config orig: {'flavor': 'default', 'kdb': True, 'sha1': '1df0c7e8a2354c42d738e5454ba83ea7332deac2'} 2024-03-27T10:29:08.804 INFO:teuthology.task.kernel:config {'mon.a': {'flavor': 'default', 'kdb': True, 'sha1': '1df0c7e8a2354c42d738e5454ba83ea7332deac2'}, 'mon.b': {'flavor': 'default', 'kdb': True, 'sha1': '1df0c7e8a2354c42d738e5454ba83ea7332deac2'}, 'mon.c': {'flavor': 'default', 'kdb': True, 'sha1': '1df0c7e8a2354c42d738e5454ba83ea7332deac2'}}, timeout 300 2024-03-27T10:29:08.804 DEBUG:teuthology.orchestra.run.smithi097:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-27T10:29:08.805 DEBUG:teuthology.orchestra.run.smithi114:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-27T10:29:08.805 DEBUG:teuthology.orchestra.run.smithi177:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-27T10:29:08.821 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-27T10:29:08.821 DEBUG:teuthology.task.kernel:sha1 for mon.c is 1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-27T10:29:08.822 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "1df0c7e8a2354c42d738e5454ba83ea7332deac2"... 2024-03-27T10:29:08.822 DEBUG:teuthology.orchestra.run.smithi177:> uname -r 2024-03-27T10:29:08.823 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-27T10:29:08.823 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-27T10:29:08.824 DEBUG:teuthology.task.kernel:sha1 for mon.a is 1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-27T10:29:08.824 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "1df0c7e8a2354c42d738e5454ba83ea7332deac2"... 2024-03-27T10:29:08.824 DEBUG:teuthology.orchestra.run.smithi097:> uname -r 2024-03-27T10:29:08.824 DEBUG:teuthology.task.kernel:sha1 for mon.b is 1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-27T10:29:08.824 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "1df0c7e8a2354c42d738e5454ba83ea7332deac2"... 2024-03-27T10:29:08.824 DEBUG:teuthology.orchestra.run.smithi114:> uname -r 2024-03-27T10:29:08.881 INFO:teuthology.orchestra.run.smithi097.stdout:5.14.0-361.el9.x86_64 2024-03-27T10:29:08.881 INFO:teuthology.orchestra.run.smithi114.stdout:5.14.0-361.el9.x86_64 2024-03-27T10:29:08.881 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-27T10:29:08.882 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-03-27T10:29:08.882 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-27T10:29:08.883 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-27T10:29:08.883 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-03-27T10:29:08.883 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-27T10:29:08.888 INFO:teuthology.orchestra.run.smithi177.stdout:5.14.0-361.el9.x86_64 2024-03-27T10:29:08.888 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-27T10:29:08.888 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-03-27T10:29:08.888 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-27T10:29:09.031 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum upgrade -y linux-firmware 2024-03-27T10:29:09.063 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum upgrade -y linux-firmware 2024-03-27T10:29:09.124 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum upgrade -y linux-firmware 2024-03-27T10:29:11.056 INFO:teuthology.orchestra.run.smithi177.stdout:CentOS-9 - AppStream 17 MB/s | 6.7 MB 00:00 2024-03-27T10:29:11.215 INFO:teuthology.orchestra.run.smithi114.stdout:CentOS-9 - AppStream 12 MB/s | 6.7 MB 00:00 2024-03-27T10:29:11.248 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS-9 - AppStream 11 MB/s | 6.7 MB 00:00 2024-03-27T10:29:12.873 INFO:teuthology.orchestra.run.smithi114.stdout:CentOS-9 - BaseOS 3.5 MB/s | 2.0 MB 00:00 2024-03-27T10:29:12.881 INFO:teuthology.orchestra.run.smithi177.stdout:CentOS-9 - BaseOS 2.7 MB/s | 2.0 MB 00:00 2024-03-27T10:29:12.981 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS-9 - BaseOS 3.6 MB/s | 2.0 MB 00:00 2024-03-27T10:29:14.264 INFO:teuthology.orchestra.run.smithi114.stdout:CentOS Stream 9 - BaseOS 7.6 MB/s | 8.0 MB 00:01 2024-03-27T10:29:14.463 INFO:teuthology.orchestra.run.smithi177.stdout:CentOS Stream 9 - BaseOS 6.4 MB/s | 8.0 MB 00:01 2024-03-27T10:29:14.855 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - BaseOS 5.3 MB/s | 8.0 MB 00:01 2024-03-27T10:29:18.165 INFO:teuthology.orchestra.run.smithi114.stdout:CentOS Stream 9 - AppStream 6.6 MB/s | 19 MB 00:02 2024-03-27T10:29:21.256 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - AppStream 3.5 MB/s | 19 MB 00:05 2024-03-27T10:29:22.107 INFO:teuthology.orchestra.run.smithi114.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.1 MB 00:00 2024-03-27T10:29:23.566 INFO:teuthology.orchestra.run.smithi114.stdout:CentOS Stream 9 - Extras packages 47 kB/s | 16 kB 00:00 2024-03-27T10:29:24.653 INFO:teuthology.orchestra.run.smithi114.stdout:Extra Packages for Enterprise Linux 21 MB/s | 21 MB 00:00 2024-03-27T10:29:26.123 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - CRB 4.4 MB/s | 6.1 MB 00:01 2024-03-27T10:29:27.591 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - Extras packages 49 kB/s | 16 kB 00:00 2024-03-27T10:29:29.484 INFO:teuthology.orchestra.run.smithi114.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-03-27T10:29:29.725 INFO:teuthology.orchestra.run.smithi097.stdout:Extra Packages for Enterprise Linux 10 MB/s | 21 MB 00:02 2024-03-27T10:29:31.747 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-03-27T10:29:31.758 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-03-27T10:29:31.758 INFO:teuthology.orchestra.run.smithi114.stdout: Package Arch Version Repository Size 2024-03-27T10:29:31.758 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-03-27T10:29:31.758 INFO:teuthology.orchestra.run.smithi114.stdout:Upgrading: 2024-03-27T10:29:31.758 INFO:teuthology.orchestra.run.smithi114.stdout: linux-firmware noarch 20240219-143.el9 CentOS-BaseOS 375 M 2024-03-27T10:29:31.758 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-03-27T10:29:31.758 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction Summary 2024-03-27T10:29:31.758 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-03-27T10:29:31.758 INFO:teuthology.orchestra.run.smithi114.stdout:Upgrade 1 Package 2024-03-27T10:29:31.759 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-03-27T10:29:31.759 INFO:teuthology.orchestra.run.smithi114.stdout:Total download size: 375 M 2024-03-27T10:29:31.759 INFO:teuthology.orchestra.run.smithi114.stdout:Downloading Packages: 2024-03-27T10:29:34.642 INFO:teuthology.orchestra.run.smithi097.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-03-27T10:29:36.908 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2024-03-27T10:29:36.918 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-03-27T10:29:36.918 INFO:teuthology.orchestra.run.smithi097.stdout: Package Arch Version Repository Size 2024-03-27T10:29:36.919 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-03-27T10:29:36.919 INFO:teuthology.orchestra.run.smithi097.stdout:Upgrading: 2024-03-27T10:29:36.919 INFO:teuthology.orchestra.run.smithi097.stdout: linux-firmware noarch 20240219-143.el9 CentOS-BaseOS 375 M 2024-03-27T10:29:36.919 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T10:29:36.919 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction Summary 2024-03-27T10:29:36.919 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-03-27T10:29:36.919 INFO:teuthology.orchestra.run.smithi097.stdout:Upgrade 1 Package 2024-03-27T10:29:36.919 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T10:29:36.919 INFO:teuthology.orchestra.run.smithi097.stdout:Total download size: 375 M 2024-03-27T10:29:36.920 INFO:teuthology.orchestra.run.smithi097.stdout:Downloading Packages: 2024-03-27T10:29:40.260 INFO:teuthology.orchestra.run.smithi177.stdout:CentOS Stream 9 - AppStream 782 kB/s | 19 MB 00:24 2024-03-27T10:29:45.511 INFO:teuthology.orchestra.run.smithi177.stdout:CentOS Stream 9 - CRB 3.6 MB/s | 6.1 MB 00:01 2024-03-27T10:29:46.929 INFO:teuthology.orchestra.run.smithi177.stdout:CentOS Stream 9 - Extras packages 60 kB/s | 16 kB 00:00 2024-03-27T10:29:49.017 INFO:teuthology.orchestra.run.smithi097.stdout:linux-firmware-20240219-143.el9.noarch.rpm 31 MB/s | 375 MB 00:12 2024-03-27T10:29:49.018 INFO:teuthology.orchestra.run.smithi097.stdout:-------------------------------------------------------------------------------- 2024-03-27T10:29:49.018 INFO:teuthology.orchestra.run.smithi097.stdout:Total 31 MB/s | 375 MB 00:12 2024-03-27T10:29:49.019 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction check 2024-03-27T10:29:49.110 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction check succeeded. 2024-03-27T10:29:49.111 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction test 2024-03-27T10:29:49.290 INFO:teuthology.orchestra.run.smithi177.stdout:Extra Packages for Enterprise Linux 9.2 MB/s | 21 MB 00:02 2024-03-27T10:29:49.742 INFO:teuthology.orchestra.run.smithi114.stdout:linux-firmware-20240219-143.el9.noarch.rpm 21 MB/s | 375 MB 00:17 2024-03-27T10:29:49.743 INFO:teuthology.orchestra.run.smithi114.stdout:-------------------------------------------------------------------------------- 2024-03-27T10:29:49.744 INFO:teuthology.orchestra.run.smithi114.stdout:Total 21 MB/s | 375 MB 00:17 2024-03-27T10:29:49.744 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction check 2024-03-27T10:29:49.836 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction check succeeded. 2024-03-27T10:29:49.836 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction test 2024-03-27T10:29:51.560 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction test succeeded. 2024-03-27T10:29:51.560 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction 2024-03-27T10:29:52.266 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction test succeeded. 2024-03-27T10:29:52.266 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction 2024-03-27T10:29:54.304 INFO:teuthology.orchestra.run.smithi177.stdout:lab-extras 12 kB/s | 1.7 kB 00:00 2024-03-27T10:29:56.618 INFO:teuthology.orchestra.run.smithi177.stdout:Dependencies resolved. 2024-03-27T10:29:56.629 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-03-27T10:29:56.629 INFO:teuthology.orchestra.run.smithi177.stdout: Package Arch Version Repository Size 2024-03-27T10:29:56.629 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-03-27T10:29:56.629 INFO:teuthology.orchestra.run.smithi177.stdout:Upgrading: 2024-03-27T10:29:56.629 INFO:teuthology.orchestra.run.smithi177.stdout: linux-firmware noarch 20240219-143.el9 CentOS-BaseOS 375 M 2024-03-27T10:29:56.629 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T10:29:56.629 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction Summary 2024-03-27T10:29:56.629 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-03-27T10:29:56.629 INFO:teuthology.orchestra.run.smithi177.stdout:Upgrade 1 Package 2024-03-27T10:29:56.629 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T10:29:56.630 INFO:teuthology.orchestra.run.smithi177.stdout:Total download size: 375 M 2024-03-27T10:29:56.630 INFO:teuthology.orchestra.run.smithi177.stdout:Downloading Packages: 2024-03-27T10:29:58.015 INFO:teuthology.orchestra.run.smithi097.stdout: Preparing : 1/1 2024-03-27T10:29:58.293 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-27T10:29:58.391 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-27T10:29:58.576 INFO:teuthology.orchestra.run.smithi114.stdout: Preparing : 1/1 2024-03-27T10:29:58.818 INFO:teuthology.orchestra.run.smithi114.stdout: Upgrading : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-27T10:29:58.908 INFO:teuthology.orchestra.run.smithi114.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-27T10:29:59.696 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2024-03-27T10:29:59.696 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-27T10:30:00.059 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2024-03-27T10:30:00.059 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-27T10:30:00.090 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-27T10:30:00.090 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T10:30:00.090 INFO:teuthology.orchestra.run.smithi097.stdout:Upgraded: 2024-03-27T10:30:00.090 INFO:teuthology.orchestra.run.smithi097.stdout: linux-firmware-20240219-143.el9.noarch 2024-03-27T10:30:00.090 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T10:30:00.090 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2024-03-27T10:30:00.381 INFO:teuthology.task.kernel:Downloading kernel 1df0c7e8a2354c42d738e5454ba83ea7332deac2 on mon.a... 2024-03-27T10:30:00.381 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-27T10:30:00.448 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-27T10:30:00.449 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-03-27T10:30:00.449 INFO:teuthology.orchestra.run.smithi114.stdout:Upgraded: 2024-03-27T10:30:00.449 INFO:teuthology.orchestra.run.smithi114.stdout: linux-firmware-20240219-143.el9.noarch 2024-03-27T10:30:00.449 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-03-27T10:30:00.449 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-03-27T10:30:00.692 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/centos/9/flavors/default/x86_64/ 2024-03-27T10:30:00.692 DEBUG:teuthology.orchestra.run.smithi097:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.8.0_g1df0c7e8a235-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/centos/9/flavors/default/x86_64/ --input-file=- 2024-03-27T10:30:00.696 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi097.front.sepia.ceph.com to complete... 2024-03-27T10:30:00.758 INFO:teuthology.task.kernel:Downloading kernel 1df0c7e8a2354c42d738e5454ba83ea7332deac2 on mon.b... 2024-03-27T10:30:00.758 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-27T10:30:00.898 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/centos/9/flavors/default/x86_64/ 2024-03-27T10:30:00.898 DEBUG:teuthology.orchestra.run.smithi114:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.8.0_g1df0c7e8a235-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/centos/9/flavors/default/x86_64/ --input-file=- 2024-03-27T10:30:00.901 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi114.front.sepia.ceph.com to complete... 2024-03-27T10:30:05.663 INFO:teuthology.orchestra.run.smithi177.stdout:linux-firmware-20240219-143.el9.noarch.rpm 42 MB/s | 375 MB 00:09 2024-03-27T10:30:05.664 INFO:teuthology.orchestra.run.smithi177.stdout:-------------------------------------------------------------------------------- 2024-03-27T10:30:05.664 INFO:teuthology.orchestra.run.smithi177.stdout:Total 41 MB/s | 375 MB 00:09 2024-03-27T10:30:05.664 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction check 2024-03-27T10:30:05.761 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction check succeeded. 2024-03-27T10:30:05.761 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction test 2024-03-27T10:30:08.141 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction test succeeded. 2024-03-27T10:30:08.142 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction 2024-03-27T10:30:13.221 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27 10:30:13 URL:https://3.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/centos/9/flavors/default/x86_64/kernel-6.8.0_g1df0c7e8a235-1.x86_64.rpm [336317861/336317861] -> "/tmp/kernel.x86_64.rpm" [1] 2024-03-27T10:30:13.221 INFO:teuthology.orchestra.run.smithi114.stderr:FINISHED --2024-03-27 10:30:13-- 2024-03-27T10:30:13.221 INFO:teuthology.orchestra.run.smithi114.stderr:Total wall clock time: 12s 2024-03-27T10:30:13.221 INFO:teuthology.orchestra.run.smithi114.stderr:Downloaded: 1 files, 321M in 12s (26.7 MB/s) 2024-03-27T10:30:13.224 INFO:teuthology.task.kernel:Installing kernel 1df0c7e8a2354c42d738e5454ba83ea7332deac2 on mon.b... 2024-03-27T10:30:13.224 DEBUG:teuthology.orchestra.run.smithi114:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-03-27T10:30:14.740 INFO:teuthology.orchestra.run.smithi177.stdout: Preparing : 1/1 2024-03-27T10:30:14.988 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-27T10:30:15.071 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-27T10:30:15.155 INFO:teuthology.orchestra.run.smithi114.stdout:Verifying... ######################################## 2024-03-27T10:30:15.211 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing... ######################################## 2024-03-27T10:30:15.290 INFO:teuthology.orchestra.run.smithi114.stdout:Updating / installing... 2024-03-27T10:30:16.551 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2024-03-27T10:30:16.551 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-27T10:30:16.943 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-27T10:30:16.943 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T10:30:16.943 INFO:teuthology.orchestra.run.smithi177.stdout:Upgraded: 2024-03-27T10:30:16.943 INFO:teuthology.orchestra.run.smithi177.stdout: linux-firmware-20240219-143.el9.noarch 2024-03-27T10:30:16.943 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T10:30:16.943 INFO:teuthology.orchestra.run.smithi177.stdout:Complete! 2024-03-27T10:30:17.235 INFO:teuthology.task.kernel:Downloading kernel 1df0c7e8a2354c42d738e5454ba83ea7332deac2 on mon.c... 2024-03-27T10:30:17.236 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-27T10:30:17.397 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/centos/9/flavors/default/x86_64/ 2024-03-27T10:30:17.397 DEBUG:teuthology.orchestra.run.smithi177:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.8.0_g1df0c7e8a235-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/centos/9/flavors/default/x86_64/ --input-file=- 2024-03-27T10:30:17.400 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi177.front.sepia.ceph.com to complete... 2024-03-27T10:30:22.759 INFO:teuthology.orchestra.run.smithi114.stdout:kernel-6.8.0_g1df0c7e8a235-1 ######################################## 2024-03-27T10:30:31.559 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27 10:30:31 URL:https://3.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/centos/9/flavors/default/x86_64/kernel-6.8.0_g1df0c7e8a235-1.x86_64.rpm [336317861/336317861] -> "/tmp/kernel.x86_64.rpm" [1] 2024-03-27T10:30:31.559 INFO:teuthology.orchestra.run.smithi177.stderr:FINISHED --2024-03-27 10:30:31-- 2024-03-27T10:30:31.559 INFO:teuthology.orchestra.run.smithi177.stderr:Total wall clock time: 14s 2024-03-27T10:30:31.559 INFO:teuthology.orchestra.run.smithi177.stderr:Downloaded: 1 files, 321M in 14s (23.1 MB/s) 2024-03-27T10:30:31.563 INFO:teuthology.task.kernel:Installing kernel 1df0c7e8a2354c42d738e5454ba83ea7332deac2 on mon.c... 2024-03-27T10:30:31.563 DEBUG:teuthology.orchestra.run.smithi177:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-03-27T10:30:33.543 INFO:teuthology.orchestra.run.smithi177.stdout:Verifying... ######################################## 2024-03-27T10:30:33.887 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing... ######################################## 2024-03-27T10:30:33.967 INFO:teuthology.orchestra.run.smithi177.stdout:Updating / installing... 2024-03-27T10:30:37.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27 10:30:37 URL:https://3.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/centos/9/flavors/default/x86_64/kernel-6.8.0_g1df0c7e8a235-1.x86_64.rpm [336317861/336317861] -> "/tmp/kernel.x86_64.rpm" [1] 2024-03-27T10:30:37.465 INFO:teuthology.orchestra.run.smithi097.stderr:FINISHED --2024-03-27 10:30:37-- 2024-03-27T10:30:37.465 INFO:teuthology.orchestra.run.smithi097.stderr:Total wall clock time: 37s 2024-03-27T10:30:37.465 INFO:teuthology.orchestra.run.smithi097.stderr:Downloaded: 1 files, 321M in 36s (8.84 MB/s) 2024-03-27T10:30:37.468 INFO:teuthology.task.kernel:Installing kernel 1df0c7e8a2354c42d738e5454ba83ea7332deac2 on mon.a... 2024-03-27T10:30:37.468 DEBUG:teuthology.orchestra.run.smithi097:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-03-27T10:30:38.529 INFO:teuthology.orchestra.run.smithi114.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-27T10:30:39.390 INFO:teuthology.orchestra.run.smithi097.stdout:Verifying... ######################################## 2024-03-27T10:30:39.444 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing... ######################################## 2024-03-27T10:30:39.523 INFO:teuthology.orchestra.run.smithi097.stdout:Updating / installing... 2024-03-27T10:30:41.531 INFO:teuthology.orchestra.run.smithi177.stdout:kernel-6.8.0_g1df0c7e8a235-1 ######################################## 2024-03-27T10:30:47.008 INFO:teuthology.orchestra.run.smithi097.stdout:kernel-6.8.0_g1df0c7e8a235-1 ######################################## 2024-03-27T10:30:57.322 INFO:teuthology.orchestra.run.smithi177.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-27T10:31:00.968 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi114.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-03-27T10:31:00.969 DEBUG:teuthology.orchestra.run.smithi114:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-03-27T10:31:01.080 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235 2024-03-27T10:31:01.080 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/System.map 2024-03-27T10:31:01.080 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/config 2024-03-27T10:31:01.080 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel 2024-03-27T10:31:01.081 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch 2024-03-27T10:31:01.081 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86 2024-03-27T10:31:01.081 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto 2024-03-27T10:31:01.081 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-03-27T10:31:01.081 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-03-27T10:31:01.081 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-03-27T10:31:01.081 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-03-27T10:31:01.082 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-03-27T10:31:01.082 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-03-27T10:31:01.082 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-03-27T10:31:01.082 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crc32c-intel.ko 2024-03-27T10:31:01.082 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-03-27T10:31:01.082 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-03-27T10:31:01.083 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-03-27T10:31:01.083 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-03-27T10:31:01.083 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-avx2.ko 2024-03-27T10:31:01.083 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-03-27T10:31:01.083 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-03-27T10:31:01.083 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-03-27T10:31:01.083 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-03-27T10:31:01.083 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-03-27T10:31:01.083 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-03-27T10:31:01.084 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-03-27T10:31:01.084 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel 2024-03-27T10:31:01.084 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu 2024-03-27T10:31:01.084 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu/mce 2024-03-27T10:31:01.084 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-03-27T10:31:01.084 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm 2024-03-27T10:31:01.085 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm-amd.ko 2024-03-27T10:31:01.085 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm-intel.ko 2024-03-27T10:31:01.085 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm.ko 2024-03-27T10:31:01.085 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto 2024-03-27T10:31:01.085 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/ansi_cprng.ko 2024-03-27T10:31:01.086 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/anubis.ko 2024-03-27T10:31:01.086 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/arc4.ko 2024-03-27T10:31:01.086 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx 2024-03-27T10:31:01.086 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_memcpy.ko 2024-03-27T10:31:01.086 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_pq.ko 2024-03-27T10:31:01.086 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_raid6_recov.ko 2024-03-27T10:31:01.087 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_tx.ko 2024-03-27T10:31:01.087 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_xor.ko 2024-03-27T10:31:01.087 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/raid6test.ko 2024-03-27T10:31:01.087 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/authenc.ko 2024-03-27T10:31:01.087 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/authencesn.ko 2024-03-27T10:31:01.087 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blake2b_generic.ko 2024-03-27T10:31:01.088 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blowfish_common.ko 2024-03-27T10:31:01.088 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blowfish_generic.ko 2024-03-27T10:31:01.088 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/camellia_generic.ko 2024-03-27T10:31:01.088 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast5_generic.ko 2024-03-27T10:31:01.088 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast6_generic.ko 2024-03-27T10:31:01.088 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast_common.ko 2024-03-27T10:31:01.088 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/ccm.ko 2024-03-27T10:31:01.088 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cmac.ko 2024-03-27T10:31:01.089 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crc32_generic.ko 2024-03-27T10:31:01.089 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crypto_engine.ko 2024-03-27T10:31:01.089 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crypto_user.ko 2024-03-27T10:31:01.089 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cts.ko 2024-03-27T10:31:01.089 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/des_generic.ko 2024-03-27T10:31:01.089 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/dh_generic.ko 2024-03-27T10:31:01.089 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/echainiv.ko 2024-03-27T10:31:01.090 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/essiv.ko 2024-03-27T10:31:01.090 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/fcrypt.ko 2024-03-27T10:31:01.090 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/khazad.ko 2024-03-27T10:31:01.090 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/lz4.ko 2024-03-27T10:31:01.090 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/lz4hc.ko 2024-03-27T10:31:01.091 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/md4.ko 2024-03-27T10:31:01.091 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/michael_mic.ko 2024-03-27T10:31:01.091 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/pcbc.ko 2024-03-27T10:31:01.091 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/pcrypt.ko 2024-03-27T10:31:01.091 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/rmd160.ko 2024-03-27T10:31:01.091 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/seed.ko 2024-03-27T10:31:01.091 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/serpent_generic.ko 2024-03-27T10:31:01.091 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/tcrypt.ko 2024-03-27T10:31:01.091 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/tea.ko 2024-03-27T10:31:01.091 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/twofish_common.ko 2024-03-27T10:31:01.092 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/twofish_generic.ko 2024-03-27T10:31:01.092 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/vmac.ko 2024-03-27T10:31:01.092 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/wp512.ko 2024-03-27T10:31:01.092 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xcbc.ko 2024-03-27T10:31:01.092 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xor.ko 2024-03-27T10:31:01.092 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xxhash_generic.ko 2024-03-27T10:31:01.093 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers 2024-03-27T10:31:01.093 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi 2024-03-27T10:31:01.093 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/acpi_ipmi.ko 2024-03-27T10:31:01.093 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/acpi_pad.ko 2024-03-27T10:31:01.093 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/custom_method.ko 2024-03-27T10:31:01.093 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/ec_sys.ko 2024-03-27T10:31:01.093 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/platform_profile.ko 2024-03-27T10:31:01.093 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/sbs.ko 2024-03-27T10:31:01.094 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/sbshc.ko 2024-03-27T10:31:01.094 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/video.ko 2024-03-27T10:31:01.094 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata 2024-03-27T10:31:01.094 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/acard-ahci.ko 2024-03-27T10:31:01.094 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/ahci_platform.ko 2024-03-27T10:31:01.094 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/ata_generic.ko 2024-03-27T10:31:01.094 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/libahci_platform.ko 2024-03-27T10:31:01.094 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_acpi.ko 2024-03-27T10:31:01.095 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ali.ko 2024-03-27T10:31:01.095 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_amd.ko 2024-03-27T10:31:01.095 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_artop.ko 2024-03-27T10:31:01.095 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_atiixp.ko 2024-03-27T10:31:01.095 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_atp867x.ko 2024-03-27T10:31:01.095 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cmd640.ko 2024-03-27T10:31:01.095 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cmd64x.ko 2024-03-27T10:31:01.095 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cypress.ko 2024-03-27T10:31:01.095 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_efar.ko 2024-03-27T10:31:01.095 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt366.ko 2024-03-27T10:31:01.095 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt37x.ko 2024-03-27T10:31:01.096 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt3x2n.ko 2024-03-27T10:31:01.096 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt3x3.ko 2024-03-27T10:31:01.096 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_it8213.ko 2024-03-27T10:31:01.096 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_it821x.ko 2024-03-27T10:31:01.096 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_jmicron.ko 2024-03-27T10:31:01.096 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_marvell.ko 2024-03-27T10:31:01.096 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_mpiix.ko 2024-03-27T10:31:01.096 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_netcell.ko 2024-03-27T10:31:01.097 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ninja32.ko 2024-03-27T10:31:01.097 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ns87410.ko 2024-03-27T10:31:01.097 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ns87415.ko 2024-03-27T10:31:01.097 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_oldpiix.ko 2024-03-27T10:31:01.097 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_opti.ko 2024-03-27T10:31:01.097 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_optidma.ko 2024-03-27T10:31:01.097 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pcmcia.ko 2024-03-27T10:31:01.097 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pdc2027x.ko 2024-03-27T10:31:01.098 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-03-27T10:31:01.098 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_piccolo.ko 2024-03-27T10:31:01.098 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_rdc.ko 2024-03-27T10:31:01.098 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sch.ko 2024-03-27T10:31:01.098 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_serverworks.ko 2024-03-27T10:31:01.098 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sil680.ko 2024-03-27T10:31:01.098 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sis.ko 2024-03-27T10:31:01.099 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sl82c105.ko 2024-03-27T10:31:01.099 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_triflex.ko 2024-03-27T10:31:01.099 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_via.ko 2024-03-27T10:31:01.099 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pdc_adma.ko 2024-03-27T10:31:01.099 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_inic162x.ko 2024-03-27T10:31:01.099 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_mv.ko 2024-03-27T10:31:01.100 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_nv.ko 2024-03-27T10:31:01.100 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_promise.ko 2024-03-27T10:31:01.100 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_qstor.ko 2024-03-27T10:31:01.100 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sil.ko 2024-03-27T10:31:01.100 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sil24.ko 2024-03-27T10:31:01.100 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sis.ko 2024-03-27T10:31:01.101 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_svw.ko 2024-03-27T10:31:01.101 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sx4.ko 2024-03-27T10:31:01.101 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_uli.ko 2024-03-27T10:31:01.101 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_via.ko 2024-03-27T10:31:01.101 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_vsc.ko 2024-03-27T10:31:01.101 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/auxdisplay 2024-03-27T10:31:01.101 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/auxdisplay/ks0108.ko 2024-03-27T10:31:01.102 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base 2024-03-27T10:31:01.102 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base/regmap 2024-03-27T10:31:01.102 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base/regmap/regmap-i2c.ko 2024-03-27T10:31:01.102 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base/regmap/regmap-mmio.ko 2024-03-27T10:31:01.102 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/bcma 2024-03-27T10:31:01.103 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/bcma/bcma.ko 2024-03-27T10:31:01.103 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block 2024-03-27T10:31:01.103 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/aoe 2024-03-27T10:31:01.103 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/aoe/aoe.ko 2024-03-27T10:31:01.103 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/brd.ko 2024-03-27T10:31:01.103 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/drbd 2024-03-27T10:31:01.103 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/drbd/drbd.ko 2024-03-27T10:31:01.103 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/floppy.ko 2024-03-27T10:31:01.104 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/loop.ko 2024-03-27T10:31:01.104 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/mtip32xx 2024-03-27T10:31:01.104 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-03-27T10:31:01.104 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/nbd.ko 2024-03-27T10:31:01.104 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/null_blk 2024-03-27T10:31:01.104 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/null_blk/null_blk.ko 2024-03-27T10:31:01.104 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/pktcdvd.ko 2024-03-27T10:31:01.104 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/rbd.ko 2024-03-27T10:31:01.104 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/virtio_blk.ko 2024-03-27T10:31:01.104 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkback 2024-03-27T10:31:01.105 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-03-27T10:31:01.105 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkfront.ko 2024-03-27T10:31:01.105 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/zram 2024-03-27T10:31:01.105 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/zram/zram.ko 2024-03-27T10:31:01.105 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char 2024-03-27T10:31:01.105 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hangcheck-timer.ko 2024-03-27T10:31:01.105 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random 2024-03-27T10:31:01.105 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/amd-rng.ko 2024-03-27T10:31:01.105 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/intel-rng.ko 2024-03-27T10:31:01.105 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-03-27T10:31:01.106 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/via-rng.ko 2024-03-27T10:31:01.106 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/virtio-rng.ko 2024-03-27T10:31:01.106 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi 2024-03-27T10:31:01.106 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-03-27T10:31:01.106 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-03-27T10:31:01.106 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-03-27T10:31:01.106 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_si.ko 2024-03-27T10:31:01.106 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-03-27T10:31:01.106 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-03-27T10:31:01.107 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/lp.ko 2024-03-27T10:31:01.107 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/mwave 2024-03-27T10:31:01.107 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/mwave/mwave.ko 2024-03-27T10:31:01.107 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ppdev.ko 2024-03-27T10:31:01.107 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tlclk.ko 2024-03-27T10:31:01.107 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm 2024-03-27T10:31:01.107 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm.ko 2024-03-27T10:31:01.107 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_atmel.ko 2024-03-27T10:31:01.108 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_infineon.ko 2024-03-27T10:31:01.108 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_nsc.ko 2024-03-27T10:31:01.108 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_tis.ko 2024-03-27T10:31:01.108 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-03-27T10:31:01.108 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/uv_mmtimer.ko 2024-03-27T10:31:01.108 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/virtio_console.ko 2024-03-27T10:31:01.108 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq 2024-03-27T10:31:01.109 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-03-27T10:31:01.109 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-03-27T10:31:01.109 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/p4-clockmod.ko 2024-03-27T10:31:01.109 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-03-27T10:31:01.109 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/powernow-k8.ko 2024-03-27T10:31:01.109 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/speedstep-lib.ko 2024-03-27T10:31:01.110 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto 2024-03-27T10:31:01.110 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp 2024-03-27T10:31:01.110 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-03-27T10:31:01.110 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp/ccp.ko 2024-03-27T10:31:01.110 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel 2024-03-27T10:31:01.110 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat 2024-03-27T10:31:01.111 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_common 2024-03-27T10:31:01.111 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-03-27T10:31:01.111 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-03-27T10:31:01.111 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-03-27T10:31:01.111 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/padlock-aes.ko 2024-03-27T10:31:01.111 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/padlock-sha.ko 2024-03-27T10:31:01.111 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/virtio 2024-03-27T10:31:01.112 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-03-27T10:31:01.112 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dca 2024-03-27T10:31:01.112 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dca/dca.ko 2024-03-27T10:31:01.112 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/devfreq 2024-03-27T10:31:01.112 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-03-27T10:31:01.112 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma 2024-03-27T10:31:01.112 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/dw 2024-03-27T10:31:01.112 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/dw/dw_dmac.ko 2024-03-27T10:31:01.112 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/ioat 2024-03-27T10:31:01.113 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/ioat/ioatdma.ko 2024-03-27T10:31:01.113 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac 2024-03-27T10:31:01.113 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/amd64_edac.ko 2024-03-27T10:31:01.113 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/e752x_edac.ko 2024-03-27T10:31:01.113 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/edac_mce_amd.ko 2024-03-27T10:31:01.113 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i3000_edac.ko 2024-03-27T10:31:01.113 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i3200_edac.ko 2024-03-27T10:31:01.113 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i5100_edac.ko 2024-03-27T10:31:01.114 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i5400_edac.ko 2024-03-27T10:31:01.114 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i7300_edac.ko 2024-03-27T10:31:01.114 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i7core_edac.ko 2024-03-27T10:31:01.114 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i82975x_edac.ko 2024-03-27T10:31:01.114 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/ie31200_edac.ko 2024-03-27T10:31:01.115 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/sb_edac.ko 2024-03-27T10:31:01.115 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/x38_edac.ko 2024-03-27T10:31:01.115 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware 2024-03-27T10:31:01.115 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware/edd.ko 2024-03-27T10:31:01.115 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware/iscsi_ibft.ko 2024-03-27T10:31:01.115 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio 2024-03-27T10:31:01.115 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-ich.ko 2024-03-27T10:31:01.115 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-viperboard.ko 2024-03-27T10:31:01.116 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid 2024-03-27T10:31:01.116 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-appleir.ko 2024-03-27T10:31:01.116 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-aureal.ko 2024-03-27T10:31:01.116 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-axff.ko 2024-03-27T10:31:01.116 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-dr.ko 2024-03-27T10:31:01.116 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-elecom.ko 2024-03-27T10:31:01.116 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-elo.ko 2024-03-27T10:31:01.116 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-emsff.ko 2024-03-27T10:31:01.116 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gaff.ko 2024-03-27T10:31:01.116 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gt683r.ko 2024-03-27T10:31:01.116 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gyration.ko 2024-03-27T10:31:01.117 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtek-kbd.ko 2024-03-27T10:31:01.117 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtek-mouse.ko 2024-03-27T10:31:01.117 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtekff.ko 2024-03-27T10:31:01.117 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-hyperv.ko 2024-03-27T10:31:01.117 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-icade.ko 2024-03-27T10:31:01.117 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-keytouch.ko 2024-03-27T10:31:01.117 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-kye.ko 2024-03-27T10:31:01.118 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-lcpower.ko 2024-03-27T10:31:01.118 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-led.ko 2024-03-27T10:31:01.118 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-lenovo.ko 2024-03-27T10:31:01.118 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-logitech-dj.ko 2024-03-27T10:31:01.118 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-03-27T10:31:01.119 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-multitouch.ko 2024-03-27T10:31:01.119 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-ortek.ko 2024-03-27T10:31:01.119 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-penmount.ko 2024-03-27T10:31:01.119 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-petalynx.ko 2024-03-27T10:31:01.119 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-picolcd.ko 2024-03-27T10:31:01.119 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-pl.ko 2024-03-27T10:31:01.119 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-plantronics.ko 2024-03-27T10:31:01.119 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-primax.ko 2024-03-27T10:31:01.119 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-rmi.ko 2024-03-27T10:31:01.119 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-arvo.ko 2024-03-27T10:31:01.119 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-common.ko 2024-03-27T10:31:01.120 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-isku.ko 2024-03-27T10:31:01.120 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-kone.ko 2024-03-27T10:31:01.120 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-03-27T10:31:01.120 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-konepure.ko 2024-03-27T10:31:01.120 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-03-27T10:31:01.120 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-lua.ko 2024-03-27T10:31:01.121 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-pyra.ko 2024-03-27T10:31:01.121 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-ryos.ko 2024-03-27T10:31:01.121 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-savu.ko 2024-03-27T10:31:01.121 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat.ko 2024-03-27T10:31:01.121 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-saitek.ko 2024-03-27T10:31:01.121 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-samsung.ko 2024-03-27T10:31:01.122 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sensor-hub.ko 2024-03-27T10:31:01.122 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sjoy.ko 2024-03-27T10:31:01.122 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sony.ko 2024-03-27T10:31:01.122 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-speedlink.ko 2024-03-27T10:31:01.122 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-steelseries.ko 2024-03-27T10:31:01.123 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sunplus.ko 2024-03-27T10:31:01.123 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-thrustmaster.ko 2024-03-27T10:31:01.123 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-tivo.ko 2024-03-27T10:31:01.123 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-tmff.ko 2024-03-27T10:31:01.123 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-topseed.ko 2024-03-27T10:31:01.123 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-twinhan.ko 2024-03-27T10:31:01.123 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-uclogic.ko 2024-03-27T10:31:01.124 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-waltop.ko 2024-03-27T10:31:01.124 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-wiimote.ko 2024-03-27T10:31:01.124 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-xinmo.ko 2024-03-27T10:31:01.124 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-zpff.ko 2024-03-27T10:31:01.124 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-zydacron.ko 2024-03-27T10:31:01.124 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/uhid.ko 2024-03-27T10:31:01.124 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/wacom.ko 2024-03-27T10:31:01.125 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv 2024-03-27T10:31:01.125 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_balloon.ko 2024-03-27T10:31:01.125 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_utils.ko 2024-03-27T10:31:01.125 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_vmbus.ko 2024-03-27T10:31:01.125 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon 2024-03-27T10:31:01.125 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/abituguru.ko 2024-03-27T10:31:01.125 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/abituguru3.ko 2024-03-27T10:31:01.126 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/acpi_power_meter.ko 2024-03-27T10:31:01.126 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ad7414.ko 2024-03-27T10:31:01.126 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ad7418.ko 2024-03-27T10:31:01.126 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adc128d818.ko 2024-03-27T10:31:01.126 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1025.ko 2024-03-27T10:31:01.126 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1026.ko 2024-03-27T10:31:01.126 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1029.ko 2024-03-27T10:31:01.126 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1031.ko 2024-03-27T10:31:01.127 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm9240.ko 2024-03-27T10:31:01.127 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ads7828.ko 2024-03-27T10:31:01.127 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7410.ko 2024-03-27T10:31:01.127 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7411.ko 2024-03-27T10:31:01.127 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7462.ko 2024-03-27T10:31:01.127 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7470.ko 2024-03-27T10:31:01.127 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7475.ko 2024-03-27T10:31:01.128 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7x10.ko 2024-03-27T10:31:01.128 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/amc6821.ko 2024-03-27T10:31:01.128 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/applesmc.ko 2024-03-27T10:31:01.128 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asb100.ko 2024-03-27T10:31:01.128 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asc7621.ko 2024-03-27T10:31:01.128 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asus_atk0110.ko 2024-03-27T10:31:01.128 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/atxp1.ko 2024-03-27T10:31:01.129 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/coretemp.ko 2024-03-27T10:31:01.129 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-03-27T10:31:01.129 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/dme1737.ko 2024-03-27T10:31:01.129 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ds1621.ko 2024-03-27T10:31:01.129 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ds620.ko 2024-03-27T10:31:01.129 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/emc1403.ko 2024-03-27T10:31:01.129 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/emc6w201.ko 2024-03-27T10:31:01.130 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f71805f.ko 2024-03-27T10:31:01.130 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f71882fg.ko 2024-03-27T10:31:01.130 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f75375s.ko 2024-03-27T10:31:01.130 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/fam15h_power.ko 2024-03-27T10:31:01.130 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/fschmd.ko 2024-03-27T10:31:01.130 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/g760a.ko 2024-03-27T10:31:01.130 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/g762.ko 2024-03-27T10:31:01.131 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/gl518sm.ko 2024-03-27T10:31:01.131 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/gl520sm.ko 2024-03-27T10:31:01.131 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/hwmon-vid.ko 2024-03-27T10:31:01.131 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/i5500_temp.ko 2024-03-27T10:31:01.131 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/i5k_amb.ko 2024-03-27T10:31:01.131 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ibmaem.ko 2024-03-27T10:31:01.131 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ibmpex.ko 2024-03-27T10:31:01.131 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ina209.ko 2024-03-27T10:31:01.132 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ina2xx.ko 2024-03-27T10:31:01.132 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/it87.ko 2024-03-27T10:31:01.132 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/k10temp.ko 2024-03-27T10:31:01.132 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/k8temp.ko 2024-03-27T10:31:01.132 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lineage-pem.ko 2024-03-27T10:31:01.132 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm63.ko 2024-03-27T10:31:01.132 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm73.ko 2024-03-27T10:31:01.133 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm75.ko 2024-03-27T10:31:01.133 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm77.ko 2024-03-27T10:31:01.133 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm78.ko 2024-03-27T10:31:01.133 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm80.ko 2024-03-27T10:31:01.133 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm83.ko 2024-03-27T10:31:01.133 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm85.ko 2024-03-27T10:31:01.133 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm87.ko 2024-03-27T10:31:01.134 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm90.ko 2024-03-27T10:31:01.134 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm92.ko 2024-03-27T10:31:01.134 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm93.ko 2024-03-27T10:31:01.134 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95234.ko 2024-03-27T10:31:01.134 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95241.ko 2024-03-27T10:31:01.134 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95245.ko 2024-03-27T10:31:01.135 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc2945.ko 2024-03-27T10:31:01.135 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4151.ko 2024-03-27T10:31:01.135 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4215.ko 2024-03-27T10:31:01.135 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4222.ko 2024-03-27T10:31:01.135 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4245.ko 2024-03-27T10:31:01.135 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4260.ko 2024-03-27T10:31:01.135 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4261.ko 2024-03-27T10:31:01.135 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max16065.ko 2024-03-27T10:31:01.136 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max1619.ko 2024-03-27T10:31:01.136 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max1668.ko 2024-03-27T10:31:01.136 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max197.ko 2024-03-27T10:31:01.136 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6639.ko 2024-03-27T10:31:01.136 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6650.ko 2024-03-27T10:31:01.136 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6697.ko 2024-03-27T10:31:01.136 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/mcp3021.ko 2024-03-27T10:31:01.136 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6683.ko 2024-03-27T10:31:01.136 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6775-core.ko 2024-03-27T10:31:01.137 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6775.ko 2024-03-27T10:31:01.137 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct7802.ko 2024-03-27T10:31:01.137 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pc87360.ko 2024-03-27T10:31:01.137 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pc87427.ko 2024-03-27T10:31:01.137 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pcf8591.ko 2024-03-27T10:31:01.137 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus 2024-03-27T10:31:01.138 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-03-27T10:31:01.138 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-03-27T10:31:01.138 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-03-27T10:31:01.138 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max16064.ko 2024-03-27T10:31:01.138 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max34440.ko 2024-03-27T10:31:01.138 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max8688.ko 2024-03-27T10:31:01.138 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-03-27T10:31:01.139 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-03-27T10:31:01.139 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-03-27T10:31:01.139 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-03-27T10:31:01.139 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-03-27T10:31:01.139 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-03-27T10:31:01.139 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/powr1220.ko 2024-03-27T10:31:01.140 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch5627.ko 2024-03-27T10:31:01.140 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch5636.ko 2024-03-27T10:31:01.140 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch56xx-common.ko 2024-03-27T10:31:01.140 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sht15.ko 2024-03-27T10:31:01.140 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sht21.ko 2024-03-27T10:31:01.141 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/shtc1.ko 2024-03-27T10:31:01.141 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sis5595.ko 2024-03-27T10:31:01.141 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47b397.ko 2024-03-27T10:31:01.141 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47m1.ko 2024-03-27T10:31:01.141 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47m192.ko 2024-03-27T10:31:01.141 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/thmc50.ko 2024-03-27T10:31:01.141 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp102.ko 2024-03-27T10:31:01.141 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp103.ko 2024-03-27T10:31:01.141 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp401.ko 2024-03-27T10:31:01.142 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp421.ko 2024-03-27T10:31:01.142 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/via-cputemp.ko 2024-03-27T10:31:01.142 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/via686a.ko 2024-03-27T10:31:01.142 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/vt1211.ko 2024-03-27T10:31:01.142 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/vt8231.ko 2024-03-27T10:31:01.142 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83627ehf.ko 2024-03-27T10:31:01.142 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83627hf.ko 2024-03-27T10:31:01.142 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83781d.ko 2024-03-27T10:31:01.142 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83791d.ko 2024-03-27T10:31:01.143 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83792d.ko 2024-03-27T10:31:01.143 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83793.ko 2024-03-27T10:31:01.143 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83795.ko 2024-03-27T10:31:01.143 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83l785ts.ko 2024-03-27T10:31:01.143 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83l786ng.ko 2024-03-27T10:31:01.143 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c 2024-03-27T10:31:01.143 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos 2024-03-27T10:31:01.144 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-03-27T10:31:01.144 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-03-27T10:31:01.144 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses 2024-03-27T10:31:01.144 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-03-27T10:31:01.144 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-03-27T10:31:01.144 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-03-27T10:31:01.145 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-03-27T10:31:01.145 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-03-27T10:31:01.145 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-03-27T10:31:01.145 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-03-27T10:31:01.145 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-03-27T10:31:01.145 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-i801.ko 2024-03-27T10:31:01.145 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-isch.ko 2024-03-27T10:31:01.146 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-03-27T10:31:01.146 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-03-27T10:31:01.146 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-03-27T10:31:01.146 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-parport.ko 2024-03-27T10:31:01.146 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-03-27T10:31:01.146 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-03-27T10:31:01.146 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-03-27T10:31:01.146 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-03-27T10:31:01.146 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-03-27T10:31:01.147 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-03-27T10:31:01.147 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-via.ko 2024-03-27T10:31:01.147 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-03-27T10:31:01.147 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-03-27T10:31:01.147 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-dev.ko 2024-03-27T10:31:01.147 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-mux.ko 2024-03-27T10:31:01.148 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-03-27T10:31:01.148 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-smbus.ko 2024-03-27T10:31:01.148 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-stub.ko 2024-03-27T10:31:01.148 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband 2024-03-27T10:31:01.148 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core 2024-03-27T10:31:01.148 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_cm.ko 2024-03-27T10:31:01.148 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_core.ko 2024-03-27T10:31:01.148 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_umad.ko 2024-03-27T10:31:01.149 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-03-27T10:31:01.149 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/iw_cm.ko 2024-03-27T10:31:01.149 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/rdma_cm.ko 2024-03-27T10:31:01.149 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-03-27T10:31:01.149 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw 2024-03-27T10:31:01.149 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/cxgb4 2024-03-27T10:31:01.150 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-03-27T10:31:01.150 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx4 2024-03-27T10:31:01.150 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-03-27T10:31:01.150 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx5 2024-03-27T10:31:01.150 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-03-27T10:31:01.150 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mthca 2024-03-27T10:31:01.150 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-03-27T10:31:01.151 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp 2024-03-27T10:31:01.151 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/ipoib 2024-03-27T10:31:01.151 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-03-27T10:31:01.151 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/iser 2024-03-27T10:31:01.151 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-03-27T10:31:01.151 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/isert 2024-03-27T10:31:01.151 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-03-27T10:31:01.152 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srp 2024-03-27T10:31:01.152 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-03-27T10:31:01.152 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srpt 2024-03-27T10:31:01.152 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-03-27T10:31:01.152 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input 2024-03-27T10:31:01.152 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport 2024-03-27T10:31:01.152 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-03-27T10:31:01.153 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/fm801-gp.ko 2024-03-27T10:31:01.153 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/gameport.ko 2024-03-27T10:31:01.153 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/joydev.ko 2024-03-27T10:31:01.153 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard 2024-03-27T10:31:01.153 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/gpio_keys.ko 2024-03-27T10:31:01.153 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse 2024-03-27T10:31:01.153 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/appletouch.ko 2024-03-27T10:31:01.154 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/bcm5974.ko 2024-03-27T10:31:01.154 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/cyapatp.ko 2024-03-27T10:31:01.154 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/elan_i2c.ko 2024-03-27T10:31:01.154 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/sermouse.ko 2024-03-27T10:31:01.154 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-03-27T10:31:01.155 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/synaptics_usb.ko 2024-03-27T10:31:01.155 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/vsxxxaa.ko 2024-03-27T10:31:01.155 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/rmi4 2024-03-27T10:31:01.155 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/rmi4/rmi_core.ko 2024-03-27T10:31:01.155 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio 2024-03-27T10:31:01.156 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/altera_ps2.ko 2024-03-27T10:31:01.156 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/arc_ps2.ko 2024-03-27T10:31:01.156 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-03-27T10:31:01.156 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/serio_raw.ko 2024-03-27T10:31:01.156 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/sparse-keymap.ko 2024-03-27T10:31:01.156 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds 2024-03-27T10:31:01.156 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-blinkm.ko 2024-03-27T10:31:01.156 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-lm3530.ko 2024-03-27T10:31:01.156 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-lp3944.ko 2024-03-27T10:31:01.157 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-ss4200.ko 2024-03-27T10:31:01.157 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger 2024-03-27T10:31:01.157 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-03-27T10:31:01.157 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-03-27T10:31:01.157 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-03-27T10:31:01.157 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-03-27T10:31:01.157 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-03-27T10:31:01.158 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-03-27T10:31:01.158 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-03-27T10:31:01.158 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-03-27T10:31:01.158 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-03-27T10:31:01.158 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md 2024-03-27T10:31:01.158 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/bcache 2024-03-27T10:31:01.159 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/bcache/bcache.ko 2024-03-27T10:31:01.159 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-bio-prison.ko 2024-03-27T10:31:01.159 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-cache-smq.ko 2024-03-27T10:31:01.159 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-cache.ko 2024-03-27T10:31:01.159 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-crypt.ko 2024-03-27T10:31:01.159 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-delay.ko 2024-03-27T10:31:01.160 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-flakey.ko 2024-03-27T10:31:01.160 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-log-userspace.ko 2024-03-27T10:31:01.160 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-multipath.ko 2024-03-27T10:31:01.160 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-queue-length.ko 2024-03-27T10:31:01.160 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-raid.ko 2024-03-27T10:31:01.160 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-round-robin.ko 2024-03-27T10:31:01.161 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-service-time.ko 2024-03-27T10:31:01.161 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-switch.ko 2024-03-27T10:31:01.161 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-thin-pool.ko 2024-03-27T10:31:01.161 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-verity.ko 2024-03-27T10:31:01.161 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/persistent-data 2024-03-27T10:31:01.161 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-03-27T10:31:01.162 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid0.ko 2024-03-27T10:31:01.162 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid1.ko 2024-03-27T10:31:01.162 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid10.ko 2024-03-27T10:31:01.162 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid456.ko 2024-03-27T10:31:01.162 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message 2024-03-27T10:31:01.163 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion 2024-03-27T10:31:01.163 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptbase.ko 2024-03-27T10:31:01.163 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptctl.ko 2024-03-27T10:31:01.163 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptfc.ko 2024-03-27T10:31:01.163 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptlan.ko 2024-03-27T10:31:01.163 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptsas.ko 2024-03-27T10:31:01.164 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptscsih.ko 2024-03-27T10:31:01.164 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptspi.ko 2024-03-27T10:31:01.164 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd 2024-03-27T10:31:01.164 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/lpc_ich.ko 2024-03-27T10:31:01.164 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/lpc_sch.ko 2024-03-27T10:31:01.165 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/mfd-core.ko 2024-03-27T10:31:01.165 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/sm501.ko 2024-03-27T10:31:01.165 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/viperboard.ko 2024-03-27T10:31:01.165 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/vx855.ko 2024-03-27T10:31:01.165 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/wl1273-core.ko 2024-03-27T10:31:01.165 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc 2024-03-27T10:31:01.166 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/altera-stapl 2024-03-27T10:31:01.166 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-03-27T10:31:01.166 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/apds9802als.ko 2024-03-27T10:31:01.166 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/apds990x.ko 2024-03-27T10:31:01.166 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/bh1770glc.ko 2024-03-27T10:31:01.166 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/cb710 2024-03-27T10:31:01.167 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/cb710/cb710.ko 2024-03-27T10:31:01.167 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom 2024-03-27T10:31:01.167 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/at24.ko 2024-03-27T10:31:01.167 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-03-27T10:31:01.167 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/max6875.ko 2024-03-27T10:31:01.167 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/enclosure.ko 2024-03-27T10:31:01.168 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/hpilo.ko 2024-03-27T10:31:01.168 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/isl29003.ko 2024-03-27T10:31:01.168 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/isl29020.ko 2024-03-27T10:31:01.168 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d 2024-03-27T10:31:01.168 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-03-27T10:31:01.168 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-03-27T10:31:01.169 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei 2024-03-27T10:31:01.169 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei-me.ko 2024-03-27T10:31:01.169 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei-txe.ko 2024-03-27T10:31:01.169 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei.ko 2024-03-27T10:31:01.169 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-gru 2024-03-27T10:31:01.169 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-gru/gru.ko 2024-03-27T10:31:01.169 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp 2024-03-27T10:31:01.169 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp/xp.ko 2024-03-27T10:31:01.170 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp/xpc.ko 2024-03-27T10:31:01.170 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-03-27T10:31:01.170 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tifm_7xx1.ko 2024-03-27T10:31:01.170 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tifm_core.ko 2024-03-27T10:31:01.170 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tsl2550.ko 2024-03-27T10:31:01.170 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_balloon.ko 2024-03-27T10:31:01.171 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_vmci 2024-03-27T10:31:01.171 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-03-27T10:31:01.171 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net 2024-03-27T10:31:01.171 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/bonding 2024-03-27T10:31:01.171 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/bonding/bonding.ko 2024-03-27T10:31:01.171 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa 2024-03-27T10:31:01.171 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/b53 2024-03-27T10:31:01.172 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/b53/b53_common.ko 2024-03-27T10:31:01.172 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/bcm-sf2.ko 2024-03-27T10:31:01.172 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6060.ko 2024-03-27T10:31:01.172 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6xxx 2024-03-27T10:31:01.172 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-03-27T10:31:01.172 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dummy.ko 2024-03-27T10:31:01.173 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/eql.ko 2024-03-27T10:31:01.173 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet 2024-03-27T10:31:01.173 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com 2024-03-27T10:31:01.173 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-03-27T10:31:01.173 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-03-27T10:31:01.174 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-03-27T10:31:01.174 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-03-27T10:31:01.174 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390 2024-03-27T10:31:01.174 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/8390.ko 2024-03-27T10:31:01.174 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-03-27T10:31:01.174 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-03-27T10:31:01.174 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-03-27T10:31:01.175 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/adaptec 2024-03-27T10:31:01.175 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-03-27T10:31:01.175 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/agere 2024-03-27T10:31:01.175 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/agere/et131x.ko 2024-03-27T10:31:01.175 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/alteon 2024-03-27T10:31:01.175 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-03-27T10:31:01.175 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/altera 2024-03-27T10:31:01.175 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-03-27T10:31:01.175 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd 2024-03-27T10:31:01.175 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-03-27T10:31:01.176 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-03-27T10:31:01.176 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-03-27T10:31:01.176 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros 2024-03-27T10:31:01.176 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/alx 2024-03-27T10:31:01.176 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-03-27T10:31:01.176 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1c 2024-03-27T10:31:01.176 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-03-27T10:31:01.176 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1e 2024-03-27T10:31:01.176 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-03-27T10:31:01.176 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx 2024-03-27T10:31:01.176 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-03-27T10:31:01.176 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-03-27T10:31:01.176 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom 2024-03-27T10:31:01.176 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-03-27T10:31:01.177 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-03-27T10:31:01.177 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-03-27T10:31:01.177 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-03-27T10:31:01.177 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-03-27T10:31:01.177 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/genet 2024-03-27T10:31:01.177 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-03-27T10:31:01.177 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-03-27T10:31:01.177 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade 2024-03-27T10:31:01.177 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade/bna 2024-03-27T10:31:01.177 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-03-27T10:31:01.177 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio 2024-03-27T10:31:01.177 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb 2024-03-27T10:31:01.177 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-03-27T10:31:01.177 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-03-27T10:31:01.178 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-03-27T10:31:01.178 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-03-27T10:31:01.178 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-03-27T10:31:01.178 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-03-27T10:31:01.178 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-03-27T10:31:01.178 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-03-27T10:31:01.178 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-03-27T10:31:01.178 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco 2024-03-27T10:31:01.178 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco/enic 2024-03-27T10:31:01.178 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-03-27T10:31:01.178 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec 2024-03-27T10:31:01.178 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip 2024-03-27T10:31:01.179 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-03-27T10:31:01.179 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-03-27T10:31:01.179 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-03-27T10:31:01.179 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-03-27T10:31:01.179 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-03-27T10:31:01.179 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-03-27T10:31:01.179 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink 2024-03-27T10:31:01.179 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-03-27T10:31:01.179 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-03-27T10:31:01.179 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dnet.ko 2024-03-27T10:31:01.179 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex 2024-03-27T10:31:01.179 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex/benet 2024-03-27T10:31:01.180 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-03-27T10:31:01.180 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ethoc.ko 2024-03-27T10:31:01.180 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/fealnx.ko 2024-03-27T10:31:01.180 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel 2024-03-27T10:31:01.180 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e100.ko 2024-03-27T10:31:01.180 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000 2024-03-27T10:31:01.180 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-03-27T10:31:01.180 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000e 2024-03-27T10:31:01.181 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-03-27T10:31:01.181 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/fm10k 2024-03-27T10:31:01.181 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-03-27T10:31:01.181 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/i40e 2024-03-27T10:31:01.181 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-03-27T10:31:01.181 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/iavf 2024-03-27T10:31:01.181 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-03-27T10:31:01.181 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igb 2024-03-27T10:31:01.181 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-03-27T10:31:01.182 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igbvf 2024-03-27T10:31:01.182 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-03-27T10:31:01.182 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbe 2024-03-27T10:31:01.182 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-03-27T10:31:01.182 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbevf 2024-03-27T10:31:01.182 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-03-27T10:31:01.182 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/jme.ko 2024-03-27T10:31:01.182 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell 2024-03-27T10:31:01.183 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-03-27T10:31:01.183 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/skge.ko 2024-03-27T10:31:01.183 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-03-27T10:31:01.183 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox 2024-03-27T10:31:01.183 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4 2024-03-27T10:31:01.183 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-03-27T10:31:01.183 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-03-27T10:31:01.183 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5 2024-03-27T10:31:01.184 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-03-27T10:31:01.184 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-03-27T10:31:01.184 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/micrel 2024-03-27T10:31:01.184 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-03-27T10:31:01.184 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom 2024-03-27T10:31:01.184 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom/myri10ge 2024-03-27T10:31:01.184 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-03-27T10:31:01.184 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi 2024-03-27T10:31:01.184 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-03-27T10:31:01.185 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-03-27T10:31:01.185 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/neterion 2024-03-27T10:31:01.185 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-03-27T10:31:01.185 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/nvidia 2024-03-27T10:31:01.185 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-03-27T10:31:01.185 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines 2024-03-27T10:31:01.185 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-03-27T10:31:01.185 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-03-27T10:31:01.185 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic 2024-03-27T10:31:01.185 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/netxen 2024-03-27T10:31:01.185 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-03-27T10:31:01.185 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-03-27T10:31:01.185 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-03-27T10:31:01.186 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-03-27T10:31:01.186 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rdc 2024-03-27T10:31:01.186 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-03-27T10:31:01.186 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek 2024-03-27T10:31:01.186 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-03-27T10:31:01.186 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-03-27T10:31:01.186 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/atp.ko 2024-03-27T10:31:01.186 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-03-27T10:31:01.186 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rocker 2024-03-27T10:31:01.186 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-03-27T10:31:01.186 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sfc 2024-03-27T10:31:01.186 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-03-27T10:31:01.187 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/silan 2024-03-27T10:31:01.187 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-03-27T10:31:01.187 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis 2024-03-27T10:31:01.187 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis/sis190.ko 2024-03-27T10:31:01.187 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis/sis900.ko 2024-03-27T10:31:01.187 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc 2024-03-27T10:31:01.187 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-03-27T10:31:01.187 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-03-27T10:31:01.187 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-03-27T10:31:01.187 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-03-27T10:31:01.187 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro 2024-03-27T10:31:01.188 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac 2024-03-27T10:31:01.188 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-03-27T10:31:01.188 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-03-27T10:31:01.188 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun 2024-03-27T10:31:01.188 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/cassini.ko 2024-03-27T10:31:01.188 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/niu.ko 2024-03-27T10:31:01.188 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/sungem.ko 2024-03-27T10:31:01.188 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-03-27T10:31:01.188 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/tehuti 2024-03-27T10:31:01.188 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-03-27T10:31:01.188 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ti 2024-03-27T10:31:01.188 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ti/tlan.ko 2024-03-27T10:31:01.188 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via 2024-03-27T10:31:01.188 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-03-27T10:31:01.189 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-03-27T10:31:01.189 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet 2024-03-27T10:31:01.189 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-03-27T10:31:01.189 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-03-27T10:31:01.189 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/xircom 2024-03-27T10:31:01.189 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-03-27T10:31:01.189 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/geneve.ko 2024-03-27T10:31:01.189 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/hyperv 2024-03-27T10:31:01.189 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-03-27T10:31:01.189 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ieee802154 2024-03-27T10:31:01.189 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ieee802154/fakelb.ko 2024-03-27T10:31:01.190 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ifb.ko 2024-03-27T10:31:01.190 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ipvlan 2024-03-27T10:31:01.190 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ipvlan/ipvlan.ko 2024-03-27T10:31:01.190 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/macvlan.ko 2024-03-27T10:31:01.190 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/macvtap.ko 2024-03-27T10:31:01.190 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio 2024-03-27T10:31:01.191 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio.ko 2024-03-27T10:31:01.191 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-03-27T10:31:01.191 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-03-27T10:31:01.191 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-regmap.ko 2024-03-27T10:31:01.191 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mii.ko 2024-03-27T10:31:01.191 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/net_failover.ko 2024-03-27T10:31:01.191 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/netconsole.ko 2024-03-27T10:31:01.191 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/nlmon.ko 2024-03-27T10:31:01.191 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs 2024-03-27T10:31:01.191 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs/pcs-lynx.ko 2024-03-27T10:31:01.191 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-03-27T10:31:01.192 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy 2024-03-27T10:31:01.192 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/amd.ko 2024-03-27T10:31:01.192 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-03-27T10:31:01.192 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-03-27T10:31:01.192 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm7xxx.ko 2024-03-27T10:31:01.192 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm87xx.ko 2024-03-27T10:31:01.192 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/broadcom.ko 2024-03-27T10:31:01.192 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/cicada.ko 2024-03-27T10:31:01.192 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/davicom.ko 2024-03-27T10:31:01.192 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/dp83640.ko 2024-03-27T10:31:01.192 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/et1011c.ko 2024-03-27T10:31:01.193 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/icplus.ko 2024-03-27T10:31:01.193 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/lxt.ko 2024-03-27T10:31:01.193 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/marvell.ko 2024-03-27T10:31:01.193 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/micrel.ko 2024-03-27T10:31:01.193 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/national.ko 2024-03-27T10:31:01.193 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/phylink.ko 2024-03-27T10:31:01.193 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/qsemi.ko 2024-03-27T10:31:01.193 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/realtek.ko 2024-03-27T10:31:01.194 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/smsc.ko 2024-03-27T10:31:01.194 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/ste10Xp.ko 2024-03-27T10:31:01.194 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/vitesse.ko 2024-03-27T10:31:01.194 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp 2024-03-27T10:31:01.194 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/bsd_comp.ko 2024-03-27T10:31:01.194 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_async.ko 2024-03-27T10:31:01.194 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_deflate.ko 2024-03-27T10:31:01.194 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_generic.ko 2024-03-27T10:31:01.194 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_mppe.ko 2024-03-27T10:31:01.195 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_synctty.ko 2024-03-27T10:31:01.195 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pppoe.ko 2024-03-27T10:31:01.195 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pppox.ko 2024-03-27T10:31:01.195 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pptp.ko 2024-03-27T10:31:01.195 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip 2024-03-27T10:31:01.195 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip/slhc.ko 2024-03-27T10:31:01.195 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip/slip.ko 2024-03-27T10:31:01.195 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/sungem_phy.ko 2024-03-27T10:31:01.195 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/tap.ko 2024-03-27T10:31:01.195 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team 2024-03-27T10:31:01.195 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team.ko 2024-03-27T10:31:01.196 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_activebackup.ko 2024-03-27T10:31:01.196 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_broadcast.ko 2024-03-27T10:31:01.196 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-03-27T10:31:01.196 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_random.ko 2024-03-27T10:31:01.196 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-03-27T10:31:01.196 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/tun.ko 2024-03-27T10:31:01.196 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/veth.ko 2024-03-27T10:31:01.196 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/virtio_net.ko 2024-03-27T10:31:01.196 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vmxnet3 2024-03-27T10:31:01.196 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-03-27T10:31:01.196 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vxlan 2024-03-27T10:31:01.196 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vxlan/vxlan.ko 2024-03-27T10:31:01.196 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netback 2024-03-27T10:31:01.197 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netback/xen-netback.ko 2024-03-27T10:31:01.197 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netfront.ko 2024-03-27T10:31:01.197 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ntb 2024-03-27T10:31:01.197 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ntb/ntb.ko 2024-03-27T10:31:01.197 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme 2024-03-27T10:31:01.197 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host 2024-03-27T10:31:01.197 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host/nvme-core.ko 2024-03-27T10:31:01.197 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host/nvme.ko 2024-03-27T10:31:01.197 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport 2024-03-27T10:31:01.197 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport.ko 2024-03-27T10:31:01.197 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_cs.ko 2024-03-27T10:31:01.197 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_pc.ko 2024-03-27T10:31:01.198 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_serial.ko 2024-03-27T10:31:01.198 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci 2024-03-27T10:31:01.198 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/hotplug 2024-03-27T10:31:01.198 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-03-27T10:31:01.198 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/pcie 2024-03-27T10:31:01.198 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/pcie/aer_inject.ko 2024-03-27T10:31:01.198 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/xen-pcifront.ko 2024-03-27T10:31:01.198 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia 2024-03-27T10:31:01.198 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/i82092.ko 2024-03-27T10:31:01.199 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/pd6729.ko 2024-03-27T10:31:01.199 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/yenta_socket.ko 2024-03-27T10:31:01.199 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl 2024-03-27T10:31:01.199 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl/intel 2024-03-27T10:31:01.199 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-03-27T10:31:01.199 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform 2024-03-27T10:31:01.199 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86 2024-03-27T10:31:01.199 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/acer-wmi.ko 2024-03-27T10:31:01.199 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/acerhdf.ko 2024-03-27T10:31:01.200 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/apple-gmux.ko 2024-03-27T10:31:01.200 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-laptop.ko 2024-03-27T10:31:01.200 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-03-27T10:31:01.200 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-wmi.ko 2024-03-27T10:31:01.200 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/classmate-laptop.ko 2024-03-27T10:31:01.200 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-03-27T10:31:01.200 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-03-27T10:31:01.200 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-03-27T10:31:01.200 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-03-27T10:31:01.200 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/hdaps.ko 2024-03-27T10:31:01.200 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel 2024-03-27T10:31:01.200 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-03-27T10:31:01.201 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel_ips.ko 2024-03-27T10:31:01.201 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/msi-wmi.ko 2024-03-27T10:31:01.201 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/mxm-wmi.ko 2024-03-27T10:31:01.201 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-03-27T10:31:01.201 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/samsung-laptop.ko 2024-03-27T10:31:01.201 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/samsung-q10.ko 2024-03-27T10:31:01.201 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/topstar-laptop.ko 2024-03-27T10:31:01.201 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-03-27T10:31:01.201 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/toshiba_haps.ko 2024-03-27T10:31:01.201 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/wmi-bmof.ko 2024-03-27T10:31:01.201 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/wmi.ko 2024-03-27T10:31:01.201 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap 2024-03-27T10:31:01.201 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap/intel_rapl_common.ko 2024-03-27T10:31:01.202 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap/intel_rapl_msr.ko 2024-03-27T10:31:01.202 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps 2024-03-27T10:31:01.202 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients 2024-03-27T10:31:01.202 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps-gpio.ko 2024-03-27T10:31:01.202 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps-ldisc.ko 2024-03-27T10:31:01.202 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps_parport.ko 2024-03-27T10:31:01.202 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/pps_core.ko 2024-03-27T10:31:01.202 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp 2024-03-27T10:31:01.202 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp/ptp.ko 2024-03-27T10:31:01.202 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp/ptp_kvm.ko 2024-03-27T10:31:01.202 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc 2024-03-27T10:31:01.202 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-bq32k.ko 2024-03-27T10:31:01.203 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1286.ko 2024-03-27T10:31:01.203 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1307.ko 2024-03-27T10:31:01.203 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1374.ko 2024-03-27T10:31:01.203 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1511.ko 2024-03-27T10:31:01.203 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1553.ko 2024-03-27T10:31:01.203 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1672.ko 2024-03-27T10:31:01.203 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1742.ko 2024-03-27T10:31:01.203 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds2404.ko 2024-03-27T10:31:01.203 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds3232.ko 2024-03-27T10:31:01.203 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-em3027.ko 2024-03-27T10:31:01.203 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-fm3130.ko 2024-03-27T10:31:01.203 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-isl12022.ko 2024-03-27T10:31:01.203 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-isl1208.ko 2024-03-27T10:31:01.204 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m41t80.ko 2024-03-27T10:31:01.204 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m48t35.ko 2024-03-27T10:31:01.204 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m48t59.ko 2024-03-27T10:31:01.204 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-max6900.ko 2024-03-27T10:31:01.204 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-msm6242.ko 2024-03-27T10:31:01.204 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf2127.ko 2024-03-27T10:31:01.204 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf85063.ko 2024-03-27T10:31:01.204 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8523.ko 2024-03-27T10:31:01.204 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8563.ko 2024-03-27T10:31:01.204 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8583.ko 2024-03-27T10:31:01.204 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rp5c01.ko 2024-03-27T10:31:01.205 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rs5c372.ko 2024-03-27T10:31:01.205 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-03-27T10:31:01.205 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rx8025.ko 2024-03-27T10:31:01.205 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rx8581.ko 2024-03-27T10:31:01.205 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-03-27T10:31:01.205 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-x1205.ko 2024-03-27T10:31:01.205 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi 2024-03-27T10:31:01.205 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-9xxx.ko 2024-03-27T10:31:01.205 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-sas.ko 2024-03-27T10:31:01.205 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-xxxx.ko 2024-03-27T10:31:01.205 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/BusLogic.ko 2024-03-27T10:31:01.205 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/a100u2w.ko 2024-03-27T10:31:01.206 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aacraid 2024-03-27T10:31:01.206 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aacraid/aacraid.ko 2024-03-27T10:31:01.206 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/advansys.ko 2024-03-27T10:31:01.206 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx 2024-03-27T10:31:01.206 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-03-27T10:31:01.206 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-03-27T10:31:01.206 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/am53c974.ko 2024-03-27T10:31:01.206 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/arcmsr 2024-03-27T10:31:01.206 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-03-27T10:31:01.207 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/atp870u.ko 2024-03-27T10:31:01.207 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/be2iscsi 2024-03-27T10:31:01.207 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-03-27T10:31:01.207 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bfa 2024-03-27T10:31:01.207 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bfa/bfa.ko 2024-03-27T10:31:01.207 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2fc 2024-03-27T10:31:01.207 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-03-27T10:31:01.207 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2i 2024-03-27T10:31:01.207 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-03-27T10:31:01.207 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ch.ko 2024-03-27T10:31:01.207 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/csiostor 2024-03-27T10:31:01.208 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/csiostor/csiostor.ko 2024-03-27T10:31:01.208 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi 2024-03-27T10:31:01.208 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb3i 2024-03-27T10:31:01.208 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-03-27T10:31:01.208 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb4i 2024-03-27T10:31:01.208 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-03-27T10:31:01.208 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-03-27T10:31:01.208 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/dc395x.ko 2024-03-27T10:31:01.208 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler 2024-03-27T10:31:01.208 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-03-27T10:31:01.209 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-03-27T10:31:01.209 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-03-27T10:31:01.209 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-03-27T10:31:01.209 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/dmx3191d.ko 2024-03-27T10:31:01.209 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esas2r 2024-03-27T10:31:01.209 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esas2r/esas2r.ko 2024-03-27T10:31:01.209 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esp_scsi.ko 2024-03-27T10:31:01.209 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe 2024-03-27T10:31:01.209 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe/fcoe.ko 2024-03-27T10:31:01.209 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-03-27T10:31:01.209 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fnic 2024-03-27T10:31:01.210 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fnic/fnic.ko 2024-03-27T10:31:01.210 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hpsa.ko 2024-03-27T10:31:01.210 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hptiop.ko 2024-03-27T10:31:01.210 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hv_storvsc.ko 2024-03-27T10:31:01.210 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/initio.ko 2024-03-27T10:31:01.210 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ipr.ko 2024-03-27T10:31:01.210 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ips.ko 2024-03-27T10:31:01.210 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/isci 2024-03-27T10:31:01.210 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/isci/isci.ko 2024-03-27T10:31:01.210 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-03-27T10:31:01.211 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/iscsi_tcp.ko 2024-03-27T10:31:01.211 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libfc 2024-03-27T10:31:01.211 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libfc/libfc.ko 2024-03-27T10:31:01.211 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libiscsi.ko 2024-03-27T10:31:01.211 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libiscsi_tcp.ko 2024-03-27T10:31:01.211 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libsas 2024-03-27T10:31:01.211 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libsas/libsas.ko 2024-03-27T10:31:01.211 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/lpfc 2024-03-27T10:31:01.211 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/lpfc/lpfc.ko 2024-03-27T10:31:01.211 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid 2024-03-27T10:31:01.211 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid.ko 2024-03-27T10:31:01.211 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-03-27T10:31:01.212 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-03-27T10:31:01.212 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-03-27T10:31:01.212 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mpt3sas 2024-03-27T10:31:01.212 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-03-27T10:31:01.212 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvsas 2024-03-27T10:31:01.212 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvsas/mvsas.ko 2024-03-27T10:31:01.212 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvumi.ko 2024-03-27T10:31:01.212 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pm8001 2024-03-27T10:31:01.212 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-03-27T10:31:01.212 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pmcraid.ko 2024-03-27T10:31:01.212 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla1280.ko 2024-03-27T10:31:01.213 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx 2024-03-27T10:31:01.213 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-03-27T10:31:01.213 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-03-27T10:31:01.213 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla4xxx 2024-03-27T10:31:01.213 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-03-27T10:31:01.213 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/raid_class.ko 2024-03-27T10:31:01.213 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_debug.ko 2024-03-27T10:31:01.213 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_fc.ko 2024-03-27T10:31:01.213 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-03-27T10:31:01.213 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_sas.ko 2024-03-27T10:31:01.213 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_spi.ko 2024-03-27T10:31:01.213 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_srp.ko 2024-03-27T10:31:01.214 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ses.ko 2024-03-27T10:31:01.214 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/st.ko 2024-03-27T10:31:01.214 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/stex.ko 2024-03-27T10:31:01.214 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/sym53c8xx_2 2024-03-27T10:31:01.214 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-03-27T10:31:01.214 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/virtio_scsi.ko 2024-03-27T10:31:01.214 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/vmw_pvscsi.ko 2024-03-27T10:31:01.214 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/wd719x.ko 2024-03-27T10:31:01.214 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ssb 2024-03-27T10:31:01.214 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ssb/ssb.ko 2024-03-27T10:31:01.214 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target 2024-03-27T10:31:01.215 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/iscsi 2024-03-27T10:31:01.215 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-03-27T10:31:01.215 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/loopback 2024-03-27T10:31:01.215 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/loopback/tcm_loop.ko 2024-03-27T10:31:01.215 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_file.ko 2024-03-27T10:31:01.215 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_iblock.ko 2024-03-27T10:31:01.215 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_mod.ko 2024-03-27T10:31:01.215 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_pscsi.ko 2024-03-27T10:31:01.215 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_user.ko 2024-03-27T10:31:01.215 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/tcm_fc 2024-03-27T10:31:01.216 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-03-27T10:31:01.216 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal 2024-03-27T10:31:01.216 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel 2024-03-27T10:31:01.216 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal 2024-03-27T10:31:01.216 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-03-27T10:31:01.216 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-03-27T10:31:01.216 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-03-27T10:31:01.216 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-03-27T10:31:01.216 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-03-27T10:31:01.216 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-03-27T10:31:01.216 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-03-27T10:31:01.217 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-03-27T10:31:01.217 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-03-27T10:31:01.217 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-03-27T10:31:01.217 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-03-27T10:31:01.217 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-03-27T10:31:01.217 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-03-27T10:31:01.217 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-03-27T10:31:01.217 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-03-27T10:31:01.217 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-03-27T10:31:01.217 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-03-27T10:31:01.218 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-03-27T10:31:01.218 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty 2024-03-27T10:31:01.218 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/ipwireless 2024-03-27T10:31:01.218 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-03-27T10:31:01.218 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/n_gsm.ko 2024-03-27T10:31:01.218 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/n_hdlc.ko 2024-03-27T10:31:01.218 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/nozomi.ko 2024-03-27T10:31:01.218 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial 2024-03-27T10:31:01.218 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/8250 2024-03-27T10:31:01.218 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-03-27T10:31:01.219 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/arc_uart.ko 2024-03-27T10:31:01.219 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/jsm 2024-03-27T10:31:01.219 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/jsm/jsm.ko 2024-03-27T10:31:01.219 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs 2024-03-27T10:31:01.219 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/core 2024-03-27T10:31:01.219 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/core/ufshcd-core.ko 2024-03-27T10:31:01.219 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/host 2024-03-27T10:31:01.219 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-03-27T10:31:01.219 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio 2024-03-27T10:31:01.219 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio.ko 2024-03-27T10:31:01.219 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_aec.ko 2024-03-27T10:31:01.220 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_cif.ko 2024-03-27T10:31:01.220 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_pci_generic.ko 2024-03-27T10:31:01.220 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_sercos3.ko 2024-03-27T10:31:01.220 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb 2024-03-27T10:31:01.220 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class 2024-03-27T10:31:01.220 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/cdc-acm.ko 2024-03-27T10:31:01.220 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/cdc-wdm.ko 2024-03-27T10:31:01.220 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/usblp.ko 2024-03-27T10:31:01.220 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/usbtmc.ko 2024-03-27T10:31:01.220 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host 2024-03-27T10:31:01.221 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host/sl811-hcd.ko 2024-03-27T10:31:01.221 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image 2024-03-27T10:31:01.221 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image/mdc800.ko 2024-03-27T10:31:01.221 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image/microtek.ko 2024-03-27T10:31:01.221 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc 2024-03-27T10:31:01.221 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/adutux.ko 2024-03-27T10:31:01.221 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/appledisplay.ko 2024-03-27T10:31:01.221 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/emi26.ko 2024-03-27T10:31:01.221 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/emi62.ko 2024-03-27T10:31:01.221 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/ezusb.ko 2024-03-27T10:31:01.221 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/idmouse.ko 2024-03-27T10:31:01.221 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/iowarrior.ko 2024-03-27T10:31:01.221 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/isight_firmware.ko 2024-03-27T10:31:01.222 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/ldusb.ko 2024-03-27T10:31:01.222 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/legousbtower.ko 2024-03-27T10:31:01.222 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/sisusbvga 2024-03-27T10:31:01.222 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-03-27T10:31:01.222 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/trancevibrator.ko 2024-03-27T10:31:01.222 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usb3503.ko 2024-03-27T10:31:01.222 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usblcd.ko 2024-03-27T10:31:01.222 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usbsevseg.ko 2024-03-27T10:31:01.222 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/uss720.ko 2024-03-27T10:31:01.222 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/yurex.ko 2024-03-27T10:31:01.222 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/phy 2024-03-27T10:31:01.223 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/phy/phy-generic.ko 2024-03-27T10:31:01.223 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial 2024-03-27T10:31:01.223 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/aircable.ko 2024-03-27T10:31:01.223 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ark3116.ko 2024-03-27T10:31:01.223 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/belkin_sa.ko 2024-03-27T10:31:01.223 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ch341.ko 2024-03-27T10:31:01.223 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cp210x.ko 2024-03-27T10:31:01.223 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cyberjack.ko 2024-03-27T10:31:01.223 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cypress_m8.ko 2024-03-27T10:31:01.223 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/digi_acceleport.ko 2024-03-27T10:31:01.223 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/empeg.ko 2024-03-27T10:31:01.224 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ftdi_sio.ko 2024-03-27T10:31:01.224 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/garmin_gps.ko 2024-03-27T10:31:01.224 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/io_edgeport.ko 2024-03-27T10:31:01.224 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/io_ti.ko 2024-03-27T10:31:01.224 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ipaq.ko 2024-03-27T10:31:01.224 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ipw.ko 2024-03-27T10:31:01.224 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ir-usb.ko 2024-03-27T10:31:01.224 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-03-27T10:31:01.224 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/keyspan.ko 2024-03-27T10:31:01.224 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/keyspan_pda.ko 2024-03-27T10:31:01.224 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/kl5kusb105.ko 2024-03-27T10:31:01.225 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/kobil_sct.ko 2024-03-27T10:31:01.225 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mct_u232.ko 2024-03-27T10:31:01.225 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mos7720.ko 2024-03-27T10:31:01.225 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mos7840.ko 2024-03-27T10:31:01.225 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/navman.ko 2024-03-27T10:31:01.225 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/omninet.ko 2024-03-27T10:31:01.225 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/opticon.ko 2024-03-27T10:31:01.225 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/option.ko 2024-03-27T10:31:01.225 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/oti6858.ko 2024-03-27T10:31:01.225 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/pl2303.ko 2024-03-27T10:31:01.225 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/qcaux.ko 2024-03-27T10:31:01.226 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/qcserial.ko 2024-03-27T10:31:01.226 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/quatech2.ko 2024-03-27T10:31:01.226 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/safe_serial.ko 2024-03-27T10:31:01.226 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/sierra.ko 2024-03-27T10:31:01.226 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/spcp8x5.ko 2024-03-27T10:31:01.226 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ssu100.ko 2024-03-27T10:31:01.226 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/symbolserial.ko 2024-03-27T10:31:01.226 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-03-27T10:31:01.226 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-03-27T10:31:01.226 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb_debug.ko 2024-03-27T10:31:01.226 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb_wwan.ko 2024-03-27T10:31:01.226 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/visor.ko 2024-03-27T10:31:01.226 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/whiteheat.ko 2024-03-27T10:31:01.227 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/xsens_mt.ko 2024-03-27T10:31:01.227 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage 2024-03-27T10:31:01.227 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/uas.ko 2024-03-27T10:31:01.227 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-alauda.ko 2024-03-27T10:31:01.227 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-cypress.ko 2024-03-27T10:31:01.227 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-datafab.ko 2024-03-27T10:31:01.227 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-03-27T10:31:01.227 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-freecom.ko 2024-03-27T10:31:01.227 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-isd200.ko 2024-03-27T10:31:01.228 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-03-27T10:31:01.228 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-karma.ko 2024-03-27T10:31:01.228 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-onetouch.ko 2024-03-27T10:31:01.228 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-realtek.ko 2024-03-27T10:31:01.228 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-sddr09.ko 2024-03-27T10:31:01.228 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-sddr55.ko 2024-03-27T10:31:01.228 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-usbat.ko 2024-03-27T10:31:01.228 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/usb-storage.ko 2024-03-27T10:31:01.228 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip 2024-03-27T10:31:01.228 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/usbip-core.ko 2024-03-27T10:31:01.229 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/usbip-host.ko 2024-03-27T10:31:01.229 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-03-27T10:31:01.229 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio 2024-03-27T10:31:01.229 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci 2024-03-27T10:31:01.229 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-03-27T10:31:01.229 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci/vfio-pci.ko 2024-03-27T10:31:01.229 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/vfio.ko 2024-03-27T10:31:01.229 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-03-27T10:31:01.229 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost 2024-03-27T10:31:01.229 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost.ko 2024-03-27T10:31:01.229 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_iotlb.ko 2024-03-27T10:31:01.229 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_net.ko 2024-03-27T10:31:01.230 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_scsi.ko 2024-03-27T10:31:01.230 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video 2024-03-27T10:31:01.230 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight 2024-03-27T10:31:01.230 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/apple_bl.ko 2024-03-27T10:31:01.230 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/lcd.ko 2024-03-27T10:31:01.230 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/platform_lcd.ko 2024-03-27T10:31:01.230 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio 2024-03-27T10:31:01.230 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio.ko 2024-03-27T10:31:01.230 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_balloon.ko 2024-03-27T10:31:01.231 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_mmio.ko 2024-03-27T10:31:01.231 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_pci.ko 2024-03-27T10:31:01.231 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-03-27T10:31:01.231 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-03-27T10:31:01.231 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_ring.ko 2024-03-27T10:31:01.231 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1 2024-03-27T10:31:01.231 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters 2024-03-27T10:31:01.231 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters/ds2482.ko 2024-03-27T10:31:01.231 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters/ds2490.ko 2024-03-27T10:31:01.231 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves 2024-03-27T10:31:01.231 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-03-27T10:31:01.231 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-03-27T10:31:01.231 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-03-27T10:31:01.232 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-03-27T10:31:01.232 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-03-27T10:31:01.232 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-03-27T10:31:01.232 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-03-27T10:31:01.232 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-03-27T10:31:01.232 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-03-27T10:31:01.232 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_smem.ko 2024-03-27T10:31:01.232 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_therm.ko 2024-03-27T10:31:01.232 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/wire.ko 2024-03-27T10:31:01.232 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog 2024-03-27T10:31:01.232 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/alim1535_wdt.ko 2024-03-27T10:31:01.232 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/alim7101_wdt.ko 2024-03-27T10:31:01.232 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/f71808e_wdt.ko 2024-03-27T10:31:01.233 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/hpwdt.ko 2024-03-27T10:31:01.233 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/i6300esb.ko 2024-03-27T10:31:01.233 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-03-27T10:31:01.233 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/iTCO_wdt.ko 2024-03-27T10:31:01.233 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ib700wdt.ko 2024-03-27T10:31:01.233 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ibmasr.ko 2024-03-27T10:31:01.233 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-03-27T10:31:01.233 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/it8712f_wdt.ko 2024-03-27T10:31:01.233 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/it87_wdt.ko 2024-03-27T10:31:01.233 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/machzwd.ko 2024-03-27T10:31:01.233 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/nv_tco.ko 2024-03-27T10:31:01.233 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/pcwd_pci.ko 2024-03-27T10:31:01.233 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/pcwd_usb.ko 2024-03-27T10:31:01.234 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-03-27T10:31:01.234 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sch311x_wdt.ko 2024-03-27T10:31:01.234 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/softdog.ko 2024-03-27T10:31:01.234 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sp5100_tco.ko 2024-03-27T10:31:01.234 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/via_wdt.ko 2024-03-27T10:31:01.234 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-03-27T10:31:01.234 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83877f_wdt.ko 2024-03-27T10:31:01.234 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83977f_wdt.ko 2024-03-27T10:31:01.234 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/wdt_pci.ko 2024-03-27T10:31:01.234 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/xen_wdt.ko 2024-03-27T10:31:01.234 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen 2024-03-27T10:31:01.234 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-acpi-processor.ko 2024-03-27T10:31:01.234 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-evtchn.ko 2024-03-27T10:31:01.235 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-gntalloc.ko 2024-03-27T10:31:01.235 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-gntdev.ko 2024-03-27T10:31:01.235 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-pciback 2024-03-27T10:31:01.235 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-03-27T10:31:01.235 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-privcmd.ko 2024-03-27T10:31:01.235 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xenfs 2024-03-27T10:31:01.235 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xenfs/xenfs.ko 2024-03-27T10:31:01.235 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs 2024-03-27T10:31:01.235 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/9p 2024-03-27T10:31:01.235 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/9p/9p.ko 2024-03-27T10:31:01.235 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/affs 2024-03-27T10:31:01.235 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/affs/affs.ko 2024-03-27T10:31:01.236 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/befs 2024-03-27T10:31:01.236 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/befs/befs.ko 2024-03-27T10:31:01.236 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/binfmt_misc.ko 2024-03-27T10:31:01.236 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/btrfs 2024-03-27T10:31:01.236 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/btrfs/btrfs.ko 2024-03-27T10:31:01.236 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cachefiles 2024-03-27T10:31:01.237 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cachefiles/cachefiles.ko 2024-03-27T10:31:01.237 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ceph 2024-03-27T10:31:01.237 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ceph/ceph.ko 2024-03-27T10:31:01.237 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/coda 2024-03-27T10:31:01.237 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/coda/coda.ko 2024-03-27T10:31:01.237 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cramfs 2024-03-27T10:31:01.237 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cramfs/cramfs.ko 2024-03-27T10:31:01.237 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/dlm 2024-03-27T10:31:01.237 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/dlm/dlm.ko 2024-03-27T10:31:01.238 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ecryptfs 2024-03-27T10:31:01.238 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ecryptfs/ecryptfs.ko 2024-03-27T10:31:01.238 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/f2fs 2024-03-27T10:31:01.238 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/f2fs/f2fs.ko 2024-03-27T10:31:01.238 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat 2024-03-27T10:31:01.238 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/fat.ko 2024-03-27T10:31:01.238 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/msdos.ko 2024-03-27T10:31:01.238 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/vfat.ko 2024-03-27T10:31:01.238 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse 2024-03-27T10:31:01.238 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse/cuse.ko 2024-03-27T10:31:01.238 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse/fuse.ko 2024-03-27T10:31:01.238 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/gfs2 2024-03-27T10:31:01.239 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/gfs2/gfs2.ko 2024-03-27T10:31:01.239 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfs 2024-03-27T10:31:01.239 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfs/hfs.ko 2024-03-27T10:31:01.239 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfsplus 2024-03-27T10:31:01.239 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfsplus/hfsplus.ko 2024-03-27T10:31:01.239 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/isofs 2024-03-27T10:31:01.239 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/isofs/isofs.ko 2024-03-27T10:31:01.239 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/jfs 2024-03-27T10:31:01.239 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/jfs/jfs.ko 2024-03-27T10:31:01.239 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/lockd 2024-03-27T10:31:01.239 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/lockd/lockd.ko 2024-03-27T10:31:01.239 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/minix 2024-03-27T10:31:01.240 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/minix/minix.ko 2024-03-27T10:31:01.240 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/netfs 2024-03-27T10:31:01.240 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/netfs/netfs.ko 2024-03-27T10:31:01.240 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs 2024-03-27T10:31:01.240 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/blocklayout 2024-03-27T10:31:01.240 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-03-27T10:31:01.240 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/filelayout 2024-03-27T10:31:01.240 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-03-27T10:31:01.240 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/flexfilelayout 2024-03-27T10:31:01.240 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-03-27T10:31:01.240 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfs.ko 2024-03-27T10:31:01.240 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfsv3.ko 2024-03-27T10:31:01.241 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfsv4.ko 2024-03-27T10:31:01.241 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common 2024-03-27T10:31:01.241 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common/grace.ko 2024-03-27T10:31:01.241 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common/nfs_acl.ko 2024-03-27T10:31:01.241 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfsd 2024-03-27T10:31:01.241 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfsd/nfsd.ko 2024-03-27T10:31:01.241 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nilfs2 2024-03-27T10:31:01.241 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nilfs2/nilfs2.ko 2024-03-27T10:31:01.241 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls 2024-03-27T10:31:01.241 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-celtic.ko 2024-03-27T10:31:01.241 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-centeuro.ko 2024-03-27T10:31:01.241 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-croatian.ko 2024-03-27T10:31:01.241 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-cyrillic.ko 2024-03-27T10:31:01.241 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-gaelic.ko 2024-03-27T10:31:01.242 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-greek.ko 2024-03-27T10:31:01.242 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-iceland.ko 2024-03-27T10:31:01.242 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-inuit.ko 2024-03-27T10:31:01.242 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-roman.ko 2024-03-27T10:31:01.242 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-romanian.ko 2024-03-27T10:31:01.242 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-turkish.ko 2024-03-27T10:31:01.242 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1250.ko 2024-03-27T10:31:01.242 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1251.ko 2024-03-27T10:31:01.242 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1255.ko 2024-03-27T10:31:01.242 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp737.ko 2024-03-27T10:31:01.242 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp775.ko 2024-03-27T10:31:01.242 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp850.ko 2024-03-27T10:31:01.242 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp852.ko 2024-03-27T10:31:01.243 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp855.ko 2024-03-27T10:31:01.243 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp857.ko 2024-03-27T10:31:01.243 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp860.ko 2024-03-27T10:31:01.243 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp861.ko 2024-03-27T10:31:01.243 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp862.ko 2024-03-27T10:31:01.243 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp863.ko 2024-03-27T10:31:01.243 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp864.ko 2024-03-27T10:31:01.243 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp865.ko 2024-03-27T10:31:01.243 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp866.ko 2024-03-27T10:31:01.243 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp869.ko 2024-03-27T10:31:01.243 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp874.ko 2024-03-27T10:31:01.243 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp932.ko 2024-03-27T10:31:01.244 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp936.ko 2024-03-27T10:31:01.244 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp949.ko 2024-03-27T10:31:01.244 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp950.ko 2024-03-27T10:31:01.244 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_euc-jp.ko 2024-03-27T10:31:01.244 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-1.ko 2024-03-27T10:31:01.244 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-13.ko 2024-03-27T10:31:01.244 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-14.ko 2024-03-27T10:31:01.244 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-15.ko 2024-03-27T10:31:01.244 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-2.ko 2024-03-27T10:31:01.245 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-3.ko 2024-03-27T10:31:01.245 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-4.ko 2024-03-27T10:31:01.245 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-5.ko 2024-03-27T10:31:01.245 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-6.ko 2024-03-27T10:31:01.245 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-7.ko 2024-03-27T10:31:01.245 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-9.ko 2024-03-27T10:31:01.245 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-r.ko 2024-03-27T10:31:01.245 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-ru.ko 2024-03-27T10:31:01.245 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-u.ko 2024-03-27T10:31:01.245 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_ucs2_utils.ko 2024-03-27T10:31:01.245 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_utf8.ko 2024-03-27T10:31:01.246 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2 2024-03-27T10:31:01.246 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/cluster 2024-03-27T10:31:01.246 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-03-27T10:31:01.246 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlm 2024-03-27T10:31:01.246 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-03-27T10:31:01.246 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlmfs 2024-03-27T10:31:01.246 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-03-27T10:31:01.246 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2.ko 2024-03-27T10:31:01.246 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-03-27T10:31:01.247 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-03-27T10:31:01.247 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-03-27T10:31:01.247 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/overlayfs 2024-03-27T10:31:01.247 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/overlayfs/overlay.ko 2024-03-27T10:31:01.247 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/pstore 2024-03-27T10:31:01.247 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/pstore/ramoops.ko 2024-03-27T10:31:01.247 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/reiserfs 2024-03-27T10:31:01.247 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/reiserfs/reiserfs.ko 2024-03-27T10:31:01.247 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/romfs 2024-03-27T10:31:01.247 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/romfs/romfs.ko 2024-03-27T10:31:01.248 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb 2024-03-27T10:31:01.248 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/client 2024-03-27T10:31:01.248 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/client/cifs.ko 2024-03-27T10:31:01.248 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common 2024-03-27T10:31:01.248 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common/cifs_arc4.ko 2024-03-27T10:31:01.248 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common/cifs_md4.ko 2024-03-27T10:31:01.248 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/squashfs 2024-03-27T10:31:01.248 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/squashfs/squashfs.ko 2024-03-27T10:31:01.248 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/sysv 2024-03-27T10:31:01.249 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/sysv/sysv.ko 2024-03-27T10:31:01.249 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/udf 2024-03-27T10:31:01.249 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/udf/udf.ko 2024-03-27T10:31:01.249 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ufs 2024-03-27T10:31:01.249 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ufs/ufs.ko 2024-03-27T10:31:01.249 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/xfs 2024-03-27T10:31:01.249 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/xfs/xfs.ko 2024-03-27T10:31:01.249 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel 2024-03-27T10:31:01.250 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/rcu 2024-03-27T10:31:01.250 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/rcu/rcutorture.ko 2024-03-27T10:31:01.250 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/torture.ko 2024-03-27T10:31:01.250 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/trace 2024-03-27T10:31:01.250 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-03-27T10:31:01.250 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib 2024-03-27T10:31:01.250 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/asn1_encoder.ko 2024-03-27T10:31:01.251 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crc-itu-t.ko 2024-03-27T10:31:01.251 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crc8.ko 2024-03-27T10:31:01.251 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto 2024-03-27T10:31:01.251 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto/libarc4.ko 2024-03-27T10:31:01.251 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto/libdes.ko 2024-03-27T10:31:01.251 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/libcrc32c.ko 2024-03-27T10:31:01.252 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lru_cache.ko 2024-03-27T10:31:01.252 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4 2024-03-27T10:31:01.252 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4/lz4_compress.ko 2024-03-27T10:31:01.252 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4/lz4hc_compress.ko 2024-03-27T10:31:01.252 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/math 2024-03-27T10:31:01.252 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/math/cordic.ko 2024-03-27T10:31:01.252 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/raid6 2024-03-27T10:31:01.252 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/raid6/raid6_pq.ko 2024-03-27T10:31:01.252 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/reed_solomon 2024-03-27T10:31:01.252 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/reed_solomon/reed_solomon.ko 2024-03-27T10:31:01.253 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_bm.ko 2024-03-27T10:31:01.253 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_fsm.ko 2024-03-27T10:31:01.253 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_kmp.ko 2024-03-27T10:31:01.253 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/zstd 2024-03-27T10:31:01.253 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/zstd/zstd_compress.ko 2024-03-27T10:31:01.253 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/mm 2024-03-27T10:31:01.253 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/mm/hwpoison-inject.ko 2024-03-27T10:31:01.253 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net 2024-03-27T10:31:01.253 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan 2024-03-27T10:31:01.253 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/6lowpan.ko 2024-03-27T10:31:01.254 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_dest.ko 2024-03-27T10:31:01.254 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_fragment.ko 2024-03-27T10:31:01.254 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_hop.ko 2024-03-27T10:31:01.254 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_ipv6.ko 2024-03-27T10:31:01.254 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_mobility.ko 2024-03-27T10:31:01.254 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_routing.ko 2024-03-27T10:31:01.254 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_udp.ko 2024-03-27T10:31:01.254 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802 2024-03-27T10:31:01.254 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/garp.ko 2024-03-27T10:31:01.255 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/mrp.ko 2024-03-27T10:31:01.255 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/p8022.ko 2024-03-27T10:31:01.255 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/psnap.ko 2024-03-27T10:31:01.255 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/stp.ko 2024-03-27T10:31:01.255 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/8021q 2024-03-27T10:31:01.255 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/8021q/8021q.ko 2024-03-27T10:31:01.255 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p 2024-03-27T10:31:01.255 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet.ko 2024-03-27T10:31:01.256 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_fd.ko 2024-03-27T10:31:01.256 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_rdma.ko 2024-03-27T10:31:01.256 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_virtio.ko 2024-03-27T10:31:01.256 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/batman-adv 2024-03-27T10:31:01.256 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/batman-adv/batman-adv.ko 2024-03-27T10:31:01.256 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge 2024-03-27T10:31:01.256 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/br_netfilter.ko 2024-03-27T10:31:01.256 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/bridge.ko 2024-03-27T10:31:01.256 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter 2024-03-27T10:31:01.257 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-03-27T10:31:01.257 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_among.ko 2024-03-27T10:31:01.257 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_arp.ko 2024-03-27T10:31:01.257 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-03-27T10:31:01.257 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-03-27T10:31:01.257 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_ip.ko 2024-03-27T10:31:01.257 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-03-27T10:31:01.257 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_limit.ko 2024-03-27T10:31:01.257 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_log.ko 2024-03-27T10:31:01.258 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_mark.ko 2024-03-27T10:31:01.258 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-03-27T10:31:01.258 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-03-27T10:31:01.258 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-03-27T10:31:01.258 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-03-27T10:31:01.258 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_snat.ko 2024-03-27T10:31:01.258 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_stp.ko 2024-03-27T10:31:01.258 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-03-27T10:31:01.259 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-03-27T10:31:01.259 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-03-27T10:31:01.259 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-03-27T10:31:01.259 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtables.ko 2024-03-27T10:31:01.259 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-03-27T10:31:01.259 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-03-27T10:31:01.259 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-03-27T10:31:01.260 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ceph 2024-03-27T10:31:01.260 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ceph/libceph.ko 2024-03-27T10:31:01.260 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core 2024-03-27T10:31:01.260 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core/failover.ko 2024-03-27T10:31:01.260 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core/pktgen.ko 2024-03-27T10:31:01.260 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp 2024-03-27T10:31:01.261 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp.ko 2024-03-27T10:31:01.261 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_diag.ko 2024-03-27T10:31:01.261 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_ipv4.ko 2024-03-27T10:31:01.261 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_ipv6.ko 2024-03-27T10:31:01.261 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dns_resolver 2024-03-27T10:31:01.261 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dns_resolver/dns_resolver.ko 2024-03-27T10:31:01.262 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa 2024-03-27T10:31:01.262 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/dsa_core.ko 2024-03-27T10:31:01.262 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_brcm.ko 2024-03-27T10:31:01.262 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_dsa.ko 2024-03-27T10:31:01.262 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_none.ko 2024-03-27T10:31:01.262 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_trailer.ko 2024-03-27T10:31:01.263 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154 2024-03-27T10:31:01.263 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/6lowpan 2024-03-27T10:31:01.263 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-03-27T10:31:01.263 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/ieee802154.ko 2024-03-27T10:31:01.263 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/ieee802154_socket.ko 2024-03-27T10:31:01.263 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4 2024-03-27T10:31:01.263 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ah4.ko 2024-03-27T10:31:01.263 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/esp4.ko 2024-03-27T10:31:01.264 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/fou.ko 2024-03-27T10:31:01.264 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/gre.ko 2024-03-27T10:31:01.264 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/inet_diag.ko 2024-03-27T10:31:01.264 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_gre.ko 2024-03-27T10:31:01.264 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_tunnel.ko 2024-03-27T10:31:01.264 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_vti.ko 2024-03-27T10:31:01.264 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ipcomp.ko 2024-03-27T10:31:01.264 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ipip.ko 2024-03-27T10:31:01.264 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter 2024-03-27T10:31:01.264 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arp_tables.ko 2024-03-27T10:31:01.265 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-03-27T10:31:01.265 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-03-27T10:31:01.265 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-03-27T10:31:01.265 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-03-27T10:31:01.265 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-03-27T10:31:01.265 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-03-27T10:31:01.265 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-03-27T10:31:01.266 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-03-27T10:31:01.266 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-03-27T10:31:01.266 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_security.ko 2024-03-27T10:31:01.266 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-03-27T10:31:01.266 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-03-27T10:31:01.266 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-03-27T10:31:01.266 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-03-27T10:31:01.266 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-03-27T10:31:01.267 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-03-27T10:31:01.267 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-03-27T10:31:01.267 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-03-27T10:31:01.267 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-03-27T10:31:01.267 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-03-27T10:31:01.267 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_bic.ko 2024-03-27T10:31:01.267 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_dctcp.ko 2024-03-27T10:31:01.267 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_diag.ko 2024-03-27T10:31:01.268 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_highspeed.ko 2024-03-27T10:31:01.268 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_htcp.ko 2024-03-27T10:31:01.270 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_hybla.ko 2024-03-27T10:31:01.270 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_illinois.ko 2024-03-27T10:31:01.270 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_lp.ko 2024-03-27T10:31:01.271 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_scalable.ko 2024-03-27T10:31:01.271 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_vegas.ko 2024-03-27T10:31:01.271 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_veno.ko 2024-03-27T10:31:01.271 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_westwood.ko 2024-03-27T10:31:01.271 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_yeah.ko 2024-03-27T10:31:01.271 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tunnel4.ko 2024-03-27T10:31:01.271 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/udp_diag.ko 2024-03-27T10:31:01.272 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/udp_tunnel.ko 2024-03-27T10:31:01.272 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/xfrm4_tunnel.ko 2024-03-27T10:31:01.272 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6 2024-03-27T10:31:01.272 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ah6.ko 2024-03-27T10:31:01.272 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/esp6.ko 2024-03-27T10:31:01.272 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/fou6.ko 2024-03-27T10:31:01.272 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_tunnel.ko 2024-03-27T10:31:01.273 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-03-27T10:31:01.273 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_vti.ko 2024-03-27T10:31:01.273 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ipcomp6.ko 2024-03-27T10:31:01.273 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter 2024-03-27T10:31:01.273 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-03-27T10:31:01.273 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-03-27T10:31:01.273 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-03-27T10:31:01.274 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-03-27T10:31:01.274 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-03-27T10:31:01.274 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-03-27T10:31:01.274 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-03-27T10:31:01.274 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-03-27T10:31:01.274 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-03-27T10:31:01.274 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-03-27T10:31:01.275 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-03-27T10:31:01.275 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-03-27T10:31:01.275 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-03-27T10:31:01.275 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-03-27T10:31:01.275 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-03-27T10:31:01.275 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-03-27T10:31:01.275 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-03-27T10:31:01.276 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-03-27T10:31:01.276 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-03-27T10:31:01.276 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-03-27T10:31:01.276 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-03-27T10:31:01.276 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-03-27T10:31:01.276 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-03-27T10:31:01.276 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-03-27T10:31:01.277 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/sit.ko 2024-03-27T10:31:01.277 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/tunnel6.ko 2024-03-27T10:31:01.277 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/xfrm6_tunnel.ko 2024-03-27T10:31:01.277 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/key 2024-03-27T10:31:01.277 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/key/af_key.ko 2024-03-27T10:31:01.277 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp 2024-03-27T10:31:01.277 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_core.ko 2024-03-27T10:31:01.278 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_debugfs.ko 2024-03-27T10:31:01.278 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_eth.ko 2024-03-27T10:31:01.278 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ip.ko 2024-03-27T10:31:01.278 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ip6.ko 2024-03-27T10:31:01.278 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_netlink.ko 2024-03-27T10:31:01.278 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ppp.ko 2024-03-27T10:31:01.278 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/llc 2024-03-27T10:31:01.279 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/llc/llc.ko 2024-03-27T10:31:01.279 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mac802154 2024-03-27T10:31:01.279 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mac802154/mac802154.ko 2024-03-27T10:31:01.279 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mpls 2024-03-27T10:31:01.279 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mpls/mpls_gso.ko 2024-03-27T10:31:01.279 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter 2024-03-27T10:31:01.279 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset 2024-03-27T10:31:01.280 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set.ko 2024-03-27T10:31:01.280 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-03-27T10:31:01.280 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-03-27T10:31:01.280 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-03-27T10:31:01.280 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-03-27T10:31:01.280 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-03-27T10:31:01.280 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-03-27T10:31:01.280 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-03-27T10:31:01.280 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-03-27T10:31:01.281 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-03-27T10:31:01.281 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-03-27T10:31:01.281 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-03-27T10:31:01.281 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-03-27T10:31:01.281 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-03-27T10:31:01.281 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-03-27T10:31:01.281 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-03-27T10:31:01.281 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs 2024-03-27T10:31:01.282 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs.ko 2024-03-27T10:31:01.282 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-03-27T10:31:01.282 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-03-27T10:31:01.282 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-03-27T10:31:01.282 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-03-27T10:31:01.282 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-03-27T10:31:01.282 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-03-27T10:31:01.282 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-03-27T10:31:01.283 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-03-27T10:31:01.283 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-03-27T10:31:01.283 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-03-27T10:31:01.283 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-03-27T10:31:01.283 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-03-27T10:31:01.283 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-03-27T10:31:01.283 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conncount.ko 2024-03-27T10:31:01.284 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack.ko 2024-03-27T10:31:01.284 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-03-27T10:31:01.284 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-03-27T10:31:01.284 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-03-27T10:31:01.284 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_h323.ko 2024-03-27T10:31:01.284 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_irc.ko 2024-03-27T10:31:01.284 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-03-27T10:31:01.285 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-03-27T10:31:01.285 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-03-27T10:31:01.285 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_sane.ko 2024-03-27T10:31:01.285 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_sip.ko 2024-03-27T10:31:01.285 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-03-27T10:31:01.285 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-03-27T10:31:01.285 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_dup_netdev.ko 2024-03-27T10:31:01.286 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_flow_table.ko 2024-03-27T10:31:01.286 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_flow_table_inet.ko 2024-03-27T10:31:01.286 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_log_syslog.ko 2024-03-27T10:31:01.286 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat.ko 2024-03-27T10:31:01.286 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_amanda.ko 2024-03-27T10:31:01.286 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_ftp.ko 2024-03-27T10:31:01.286 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_irc.ko 2024-03-27T10:31:01.286 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_sip.ko 2024-03-27T10:31:01.286 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_tftp.ko 2024-03-27T10:31:01.287 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_synproxy_core.ko 2024-03-27T10:31:01.287 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_tables.ko 2024-03-27T10:31:01.287 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink.ko 2024-03-27T10:31:01.287 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_acct.ko 2024-03-27T10:31:01.287 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_log.ko 2024-03-27T10:31:01.287 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_osf.ko 2024-03-27T10:31:01.288 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_queue.ko 2024-03-27T10:31:01.288 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_chain_nat.ko 2024-03-27T10:31:01.288 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_compat.ko 2024-03-27T10:31:01.288 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_ct.ko 2024-03-27T10:31:01.288 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_dup_netdev.ko 2024-03-27T10:31:01.288 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_fib.ko 2024-03-27T10:31:01.288 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_fwd_netdev.ko 2024-03-27T10:31:01.288 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_hash.ko 2024-03-27T10:31:01.288 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_limit.ko 2024-03-27T10:31:01.289 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_log.ko 2024-03-27T10:31:01.289 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_masq.ko 2024-03-27T10:31:01.289 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_nat.ko 2024-03-27T10:31:01.289 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_numgen.ko 2024-03-27T10:31:01.289 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_queue.ko 2024-03-27T10:31:01.289 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_quota.ko 2024-03-27T10:31:01.289 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_redir.ko 2024-03-27T10:31:01.289 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_reject.ko 2024-03-27T10:31:01.289 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_reject_inet.ko 2024-03-27T10:31:01.290 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_socket.ko 2024-03-27T10:31:01.290 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_tunnel.ko 2024-03-27T10:31:01.290 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_AUDIT.ko 2024-03-27T10:31:01.290 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CHECKSUM.ko 2024-03-27T10:31:01.290 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CLASSIFY.ko 2024-03-27T10:31:01.290 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-03-27T10:31:01.291 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CT.ko 2024-03-27T10:31:01.291 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_DSCP.ko 2024-03-27T10:31:01.291 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_HL.ko 2024-03-27T10:31:01.291 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_HMARK.ko 2024-03-27T10:31:01.291 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_IDLETIMER.ko 2024-03-27T10:31:01.291 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_LED.ko 2024-03-27T10:31:01.291 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_LOG.ko 2024-03-27T10:31:01.292 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_MASQUERADE.ko 2024-03-27T10:31:01.292 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NETMAP.ko 2024-03-27T10:31:01.292 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NFLOG.ko 2024-03-27T10:31:01.292 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NFQUEUE.ko 2024-03-27T10:31:01.292 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_RATEEST.ko 2024-03-27T10:31:01.292 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_REDIRECT.ko 2024-03-27T10:31:01.292 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_SECMARK.ko 2024-03-27T10:31:01.293 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TCPMSS.ko 2024-03-27T10:31:01.293 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-03-27T10:31:01.293 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TEE.ko 2024-03-27T10:31:01.293 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TPROXY.ko 2024-03-27T10:31:01.293 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TRACE.ko 2024-03-27T10:31:01.293 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_addrtype.ko 2024-03-27T10:31:01.293 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_bpf.ko 2024-03-27T10:31:01.293 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cgroup.ko 2024-03-27T10:31:01.293 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cluster.ko 2024-03-27T10:31:01.294 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_comment.ko 2024-03-27T10:31:01.294 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connbytes.ko 2024-03-27T10:31:01.294 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connlabel.ko 2024-03-27T10:31:01.294 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connlimit.ko 2024-03-27T10:31:01.294 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connmark.ko 2024-03-27T10:31:01.294 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_conntrack.ko 2024-03-27T10:31:01.294 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cpu.ko 2024-03-27T10:31:01.294 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_dccp.ko 2024-03-27T10:31:01.295 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_devgroup.ko 2024-03-27T10:31:01.295 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_dscp.ko 2024-03-27T10:31:01.295 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ecn.ko 2024-03-27T10:31:01.295 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_esp.ko 2024-03-27T10:31:01.295 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_hashlimit.ko 2024-03-27T10:31:01.295 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_helper.ko 2024-03-27T10:31:01.295 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_hl.ko 2024-03-27T10:31:01.295 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ipcomp.ko 2024-03-27T10:31:01.295 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_iprange.ko 2024-03-27T10:31:01.295 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ipvs.ko 2024-03-27T10:31:01.296 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_l2tp.ko 2024-03-27T10:31:01.296 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_length.ko 2024-03-27T10:31:01.296 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_limit.ko 2024-03-27T10:31:01.296 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_mac.ko 2024-03-27T10:31:01.296 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_mark.ko 2024-03-27T10:31:01.296 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_multiport.ko 2024-03-27T10:31:01.296 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_nat.ko 2024-03-27T10:31:01.296 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_nfacct.ko 2024-03-27T10:31:01.296 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_osf.ko 2024-03-27T10:31:01.296 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_owner.ko 2024-03-27T10:31:01.297 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_physdev.ko 2024-03-27T10:31:01.297 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_pkttype.ko 2024-03-27T10:31:01.297 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_policy.ko 2024-03-27T10:31:01.297 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_quota.ko 2024-03-27T10:31:01.297 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_rateest.ko 2024-03-27T10:31:01.297 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_realm.ko 2024-03-27T10:31:01.298 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_recent.ko 2024-03-27T10:31:01.298 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_sctp.ko 2024-03-27T10:31:01.298 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_set.ko 2024-03-27T10:31:01.298 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_socket.ko 2024-03-27T10:31:01.298 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_state.ko 2024-03-27T10:31:01.298 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_statistic.ko 2024-03-27T10:31:01.298 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_string.ko 2024-03-27T10:31:01.298 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_tcpmss.ko 2024-03-27T10:31:01.298 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_time.ko 2024-03-27T10:31:01.298 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_u32.ko 2024-03-27T10:31:01.298 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netlink 2024-03-27T10:31:01.299 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netlink/netlink_diag.ko 2024-03-27T10:31:01.299 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/nsh 2024-03-27T10:31:01.299 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/nsh/nsh.ko 2024-03-27T10:31:01.299 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch 2024-03-27T10:31:01.299 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/openvswitch.ko 2024-03-27T10:31:01.299 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-geneve.ko 2024-03-27T10:31:01.299 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-gre.ko 2024-03-27T10:31:01.299 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-vxlan.ko 2024-03-27T10:31:01.299 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/packet 2024-03-27T10:31:01.300 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/packet/af_packet_diag.ko 2024-03-27T10:31:01.300 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds 2024-03-27T10:31:01.300 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds.ko 2024-03-27T10:31:01.300 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds_rdma.ko 2024-03-27T10:31:01.300 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds_tcp.ko 2024-03-27T10:31:01.300 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched 2024-03-27T10:31:01.300 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_csum.ko 2024-03-27T10:31:01.300 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_gact.ko 2024-03-27T10:31:01.300 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_mirred.ko 2024-03-27T10:31:01.301 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_nat.ko 2024-03-27T10:31:01.301 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_pedit.ko 2024-03-27T10:31:01.301 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_police.ko 2024-03-27T10:31:01.301 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_simple.ko 2024-03-27T10:31:01.301 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_skbedit.ko 2024-03-27T10:31:01.301 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_vlan.ko 2024-03-27T10:31:01.301 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_basic.ko 2024-03-27T10:31:01.302 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_bpf.ko 2024-03-27T10:31:01.302 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_flow.ko 2024-03-27T10:31:01.302 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_fw.ko 2024-03-27T10:31:01.302 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_route.ko 2024-03-27T10:31:01.302 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_u32.ko 2024-03-27T10:31:01.302 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_cmp.ko 2024-03-27T10:31:01.302 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_ipset.ko 2024-03-27T10:31:01.302 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_meta.ko 2024-03-27T10:31:01.302 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_nbyte.ko 2024-03-27T10:31:01.303 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_text.ko 2024-03-27T10:31:01.303 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_u32.ko 2024-03-27T10:31:01.303 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_choke.ko 2024-03-27T10:31:01.303 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_codel.ko 2024-03-27T10:31:01.303 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_drr.ko 2024-03-27T10:31:01.303 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_fq.ko 2024-03-27T10:31:01.304 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_gred.ko 2024-03-27T10:31:01.304 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_hfsc.ko 2024-03-27T10:31:01.304 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_hhf.ko 2024-03-27T10:31:01.304 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_htb.ko 2024-03-27T10:31:01.304 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_ingress.ko 2024-03-27T10:31:01.304 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_mqprio.ko 2024-03-27T10:31:01.305 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_mqprio_lib.ko 2024-03-27T10:31:01.305 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_multiq.ko 2024-03-27T10:31:01.305 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_netem.ko 2024-03-27T10:31:01.305 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_pie.ko 2024-03-27T10:31:01.305 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_plug.ko 2024-03-27T10:31:01.305 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_prio.ko 2024-03-27T10:31:01.306 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_qfq.ko 2024-03-27T10:31:01.306 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_red.ko 2024-03-27T10:31:01.306 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_sfb.ko 2024-03-27T10:31:01.306 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_sfq.ko 2024-03-27T10:31:01.306 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_tbf.ko 2024-03-27T10:31:01.306 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_teql.ko 2024-03-27T10:31:01.307 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp 2024-03-27T10:31:01.307 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp/sctp.ko 2024-03-27T10:31:01.307 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp/sctp_diag.ko 2024-03-27T10:31:01.307 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc 2024-03-27T10:31:01.307 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss 2024-03-27T10:31:01.307 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-03-27T10:31:01.308 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-03-27T10:31:01.308 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/sunrpc.ko 2024-03-27T10:31:01.308 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/xprtrdma 2024-03-27T10:31:01.308 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-03-27T10:31:01.308 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc 2024-03-27T10:31:01.308 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc/diag.ko 2024-03-27T10:31:01.308 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc/tipc.ko 2024-03-27T10:31:01.308 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/unix 2024-03-27T10:31:01.308 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/unix/unix_diag.ko 2024-03-27T10:31:01.309 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock 2024-03-27T10:31:01.309 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-03-27T10:31:01.309 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-03-27T10:31:01.309 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock.ko 2024-03-27T10:31:01.309 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock_diag.ko 2024-03-27T10:31:01.309 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock_loopback.ko 2024-03-27T10:31:01.309 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/xfrm 2024-03-27T10:31:01.309 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/xfrm/xfrm_ipcomp.ko 2024-03-27T10:31:01.310 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security 2024-03-27T10:31:01.310 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys 2024-03-27T10:31:01.310 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/encrypted-keys 2024-03-27T10:31:01.310 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-03-27T10:31:01.310 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/trusted-keys 2024-03-27T10:31:01.310 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/trusted-keys/trusted.ko 2024-03-27T10:31:01.311 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/virt 2024-03-27T10:31:01.311 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/virt/lib 2024-03-27T10:31:01.311 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/virt/lib/irqbypass.ko 2024-03-27T10:31:01.311 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.alias 2024-03-27T10:31:01.311 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.alias.bin 2024-03-27T10:31:01.311 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin 2024-03-27T10:31:01.311 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.alias.bin 2024-03-27T10:31:01.312 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.bin 2024-03-27T10:31:01.312 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.modinfo 2024-03-27T10:31:01.312 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.dep 2024-03-27T10:31:01.312 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.dep.bin 2024-03-27T10:31:01.312 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.devname 2024-03-27T10:31:01.312 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.order 2024-03-27T10:31:01.312 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.softdep 2024-03-27T10:31:01.313 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.symbols 2024-03-27T10:31:01.313 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.symbols.bin 2024-03-27T10:31:01.313 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.8.0-g1df0c7e8a235/vmlinuz 2024-03-27T10:31:01.318 DEBUG:teuthology.task.kernel:get_image_version: 6.8.0-g1df0c7e8a235 2024-03-27T10:31:01.318 DEBUG:teuthology.orchestra.run.smithi114:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-03-27T10:31:01.320 INFO:teuthology.orchestra.run.smithi097.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-27T10:31:01.341 INFO:teuthology.orchestra.run.smithi114.stdout:postinstall scriptlet (using /bin/sh): 2024-03-27T10:31:01.341 INFO:teuthology.orchestra.run.smithi114.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-03-27T10:31:01.341 INFO:teuthology.orchestra.run.smithi114.stdout: /usr/bin/kernel-install add 6.8.0-g1df0c7e8a235 /lib/modules/6.8.0-g1df0c7e8a235/vmlinuz 2024-03-27T10:31:01.341 INFO:teuthology.orchestra.run.smithi114.stdout:fi 2024-03-27T10:31:01.341 INFO:teuthology.orchestra.run.smithi114.stdout:for file in vmlinuz System.map config; do 2024-03-27T10:31:01.342 INFO:teuthology.orchestra.run.smithi114.stdout: if ! cmp --silent "/lib/modules/6.8.0-g1df0c7e8a235/${file}" "/boot/${file}-6.8.0-g1df0c7e8a235"; then 2024-03-27T10:31:01.342 INFO:teuthology.orchestra.run.smithi114.stdout: cp "/lib/modules/6.8.0-g1df0c7e8a235/${file}" "/boot/${file}-6.8.0-g1df0c7e8a235" 2024-03-27T10:31:01.342 INFO:teuthology.orchestra.run.smithi114.stdout: fi 2024-03-27T10:31:01.342 INFO:teuthology.orchestra.run.smithi114.stdout:done 2024-03-27T10:31:01.342 INFO:teuthology.orchestra.run.smithi114.stdout:preuninstall scriptlet (using /bin/sh): 2024-03-27T10:31:01.342 INFO:teuthology.orchestra.run.smithi114.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2024-03-27T10:31:01.342 INFO:teuthology.orchestra.run.smithi114.stdout:new-kernel-pkg --remove 6.8.0-g1df0c7e8a235 --rminitrd --initrdfile=/boot/initramfs-6.8.0-g1df0c7e8a235.img 2024-03-27T10:31:01.342 INFO:teuthology.orchestra.run.smithi114.stdout:elif [ -x /usr/bin/kernel-install ]; then 2024-03-27T10:31:01.342 INFO:teuthology.orchestra.run.smithi114.stdout:kernel-install remove 6.8.0-g1df0c7e8a235 2024-03-27T10:31:01.342 INFO:teuthology.orchestra.run.smithi114.stdout:fi 2024-03-27T10:31:01.342 INFO:teuthology.orchestra.run.smithi114.stdout:postuninstall scriptlet (using /bin/sh): 2024-03-27T10:31:01.343 INFO:teuthology.orchestra.run.smithi114.stdout:if [ -x /sbin/update-bootloader ]; then 2024-03-27T10:31:01.343 INFO:teuthology.orchestra.run.smithi114.stdout:/sbin/update-bootloader --remove 6.8.0-g1df0c7e8a235 2024-03-27T10:31:01.343 INFO:teuthology.orchestra.run.smithi114.stdout:fi 2024-03-27T10:31:01.344 DEBUG:teuthology.orchestra.run.smithi114:> sudo rpm -qi grub2-tools 2024-03-27T10:31:01.420 INFO:teuthology.orchestra.run.smithi114.stdout:Name : grub2-tools 2024-03-27T10:31:01.420 INFO:teuthology.orchestra.run.smithi114.stdout:Epoch : 1 2024-03-27T10:31:01.420 INFO:teuthology.orchestra.run.smithi114.stdout:Version : 2.06 2024-03-27T10:31:01.420 INFO:teuthology.orchestra.run.smithi114.stdout:Release : 68.el9 2024-03-27T10:31:01.420 INFO:teuthology.orchestra.run.smithi114.stdout:Architecture: x86_64 2024-03-27T10:31:01.420 INFO:teuthology.orchestra.run.smithi114.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-27T10:31:01.420 INFO:teuthology.orchestra.run.smithi114.stdout:Group : Unspecified 2024-03-27T10:31:01.420 INFO:teuthology.orchestra.run.smithi114.stdout:Size : 8264254 2024-03-27T10:31:01.420 INFO:teuthology.orchestra.run.smithi114.stdout:License : GPLv3+ 2024-03-27T10:31:01.420 INFO:teuthology.orchestra.run.smithi114.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-27T10:31:01.421 INFO:teuthology.orchestra.run.smithi114.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-27T10:31:01.421 INFO:teuthology.orchestra.run.smithi114.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-27T10:31:01.421 INFO:teuthology.orchestra.run.smithi114.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-27T10:31:01.421 INFO:teuthology.orchestra.run.smithi114.stdout:Packager : builder@centos.org 2024-03-27T10:31:01.421 INFO:teuthology.orchestra.run.smithi114.stdout:Vendor : CentOS 2024-03-27T10:31:01.421 INFO:teuthology.orchestra.run.smithi114.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-27T10:31:01.421 INFO:teuthology.orchestra.run.smithi114.stdout:Summary : Support tools for GRUB. 2024-03-27T10:31:01.421 INFO:teuthology.orchestra.run.smithi114.stdout:Description : 2024-03-27T10:31:01.421 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-03-27T10:31:01.421 INFO:teuthology.orchestra.run.smithi114.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-27T10:31:01.421 INFO:teuthology.orchestra.run.smithi114.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-27T10:31:01.422 INFO:teuthology.orchestra.run.smithi114.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-27T10:31:01.422 INFO:teuthology.orchestra.run.smithi114.stdout:hardware devices. 2024-03-27T10:31:01.422 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-03-27T10:31:01.422 INFO:teuthology.orchestra.run.smithi114.stdout:This subpackage provides tools for support of all platforms. 2024-03-27T10:31:01.423 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-27T10:31:01.423 INFO:teuthology.task.kernel:Updating grub on smithi114 to boot 6.8.0-g1df0c7e8a235 2024-03-27T10:31:01.424 DEBUG:teuthology.orchestra.run.smithi114:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-27T10:31:02.008 INFO:teuthology.orchestra.run.smithi114.stderr:Generating grub configuration file ... 2024-03-27T10:31:03.363 INFO:teuthology.orchestra.run.smithi114.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-27T10:31:03.398 INFO:teuthology.orchestra.run.smithi114.stderr:done 2024-03-27T10:31:03.401 DEBUG:teuthology.orchestra.run.smithi114:> mktemp 2024-03-27T10:31:03.416 INFO:teuthology.orchestra.run.smithi114.stdout:/tmp/tmp.mfPXJfssb8 2024-03-27T10:31:03.416 DEBUG:teuthology.orchestra.run.smithi114:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.mfPXJfssb8 2024-03-27T10:31:03.485 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod 0666 /tmp/tmp.mfPXJfssb8 2024-03-27T10:31:03.632 DEBUG:teuthology.orchestra.remote:smithi114:/tmp/tmp.mfPXJfssb8 is 6KB 2024-03-27T10:31:03.680 DEBUG:teuthology.orchestra.run.smithi114:> rm -fr /tmp/tmp.mfPXJfssb8 2024-03-27T10:31:03.694 DEBUG:teuthology.orchestra.run.smithi114:> sudo /bin/ls /boot/loader/entries || true 2024-03-27T10:31:03.768 INFO:teuthology.orchestra.run.smithi114.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-27T10:31:03.768 INFO:teuthology.orchestra.run.smithi114.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-27T10:31:03.768 INFO:teuthology.orchestra.run.smithi114.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-27T10:31:03.769 INFO:teuthology.orchestra.run.smithi114.stdout:15949e3c88704300882482eddfd7baae-6.8.0-g1df0c7e8a235.conf 2024-03-27T10:31:03.770 DEBUG:teuthology.orchestra.run.smithi114:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.8.0-g1df0c7e8a235 2024-03-27T10:31:03.927 DEBUG:teuthology.orchestra.run.smithi114:> sudo shutdown -r now 2024-03-27T10:31:19.718 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi177.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-03-27T10:31:19.718 DEBUG:teuthology.orchestra.run.smithi177:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-03-27T10:31:19.794 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235 2024-03-27T10:31:19.795 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/System.map 2024-03-27T10:31:19.795 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/config 2024-03-27T10:31:19.795 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel 2024-03-27T10:31:19.795 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch 2024-03-27T10:31:19.795 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86 2024-03-27T10:31:19.795 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto 2024-03-27T10:31:19.795 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-03-27T10:31:19.795 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-03-27T10:31:19.795 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-03-27T10:31:19.795 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-03-27T10:31:19.795 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-03-27T10:31:19.795 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-03-27T10:31:19.796 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-03-27T10:31:19.796 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crc32c-intel.ko 2024-03-27T10:31:19.796 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-03-27T10:31:19.796 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-03-27T10:31:19.796 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-03-27T10:31:19.796 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-03-27T10:31:19.796 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-avx2.ko 2024-03-27T10:31:19.796 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-03-27T10:31:19.796 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-03-27T10:31:19.796 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-03-27T10:31:19.796 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-03-27T10:31:19.796 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-03-27T10:31:19.797 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-03-27T10:31:19.797 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-03-27T10:31:19.797 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel 2024-03-27T10:31:19.797 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu 2024-03-27T10:31:19.797 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu/mce 2024-03-27T10:31:19.797 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-03-27T10:31:19.797 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm 2024-03-27T10:31:19.797 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm-amd.ko 2024-03-27T10:31:19.797 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm-intel.ko 2024-03-27T10:31:19.797 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm.ko 2024-03-27T10:31:19.797 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto 2024-03-27T10:31:19.797 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/ansi_cprng.ko 2024-03-27T10:31:19.797 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/anubis.ko 2024-03-27T10:31:19.798 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/arc4.ko 2024-03-27T10:31:19.798 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx 2024-03-27T10:31:19.798 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_memcpy.ko 2024-03-27T10:31:19.798 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_pq.ko 2024-03-27T10:31:19.798 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_raid6_recov.ko 2024-03-27T10:31:19.798 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_tx.ko 2024-03-27T10:31:19.798 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_xor.ko 2024-03-27T10:31:19.798 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/raid6test.ko 2024-03-27T10:31:19.798 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/authenc.ko 2024-03-27T10:31:19.798 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/authencesn.ko 2024-03-27T10:31:19.798 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blake2b_generic.ko 2024-03-27T10:31:19.798 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blowfish_common.ko 2024-03-27T10:31:19.799 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blowfish_generic.ko 2024-03-27T10:31:19.799 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/camellia_generic.ko 2024-03-27T10:31:19.799 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast5_generic.ko 2024-03-27T10:31:19.799 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast6_generic.ko 2024-03-27T10:31:19.799 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast_common.ko 2024-03-27T10:31:19.799 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/ccm.ko 2024-03-27T10:31:19.799 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cmac.ko 2024-03-27T10:31:19.799 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crc32_generic.ko 2024-03-27T10:31:19.799 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crypto_engine.ko 2024-03-27T10:31:19.799 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crypto_user.ko 2024-03-27T10:31:19.799 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cts.ko 2024-03-27T10:31:19.799 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/des_generic.ko 2024-03-27T10:31:19.799 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/dh_generic.ko 2024-03-27T10:31:19.800 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/echainiv.ko 2024-03-27T10:31:19.800 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/essiv.ko 2024-03-27T10:31:19.800 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/fcrypt.ko 2024-03-27T10:31:19.800 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/khazad.ko 2024-03-27T10:31:19.800 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/lz4.ko 2024-03-27T10:31:19.800 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/lz4hc.ko 2024-03-27T10:31:19.800 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/md4.ko 2024-03-27T10:31:19.800 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/michael_mic.ko 2024-03-27T10:31:19.800 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/pcbc.ko 2024-03-27T10:31:19.800 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/pcrypt.ko 2024-03-27T10:31:19.800 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/rmd160.ko 2024-03-27T10:31:19.800 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/seed.ko 2024-03-27T10:31:19.800 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/serpent_generic.ko 2024-03-27T10:31:19.800 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/tcrypt.ko 2024-03-27T10:31:19.801 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/tea.ko 2024-03-27T10:31:19.801 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/twofish_common.ko 2024-03-27T10:31:19.801 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/twofish_generic.ko 2024-03-27T10:31:19.801 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/vmac.ko 2024-03-27T10:31:19.801 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/wp512.ko 2024-03-27T10:31:19.801 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xcbc.ko 2024-03-27T10:31:19.801 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xor.ko 2024-03-27T10:31:19.801 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xxhash_generic.ko 2024-03-27T10:31:19.801 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers 2024-03-27T10:31:19.801 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi 2024-03-27T10:31:19.801 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/acpi_ipmi.ko 2024-03-27T10:31:19.801 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/acpi_pad.ko 2024-03-27T10:31:19.801 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/custom_method.ko 2024-03-27T10:31:19.802 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/ec_sys.ko 2024-03-27T10:31:19.802 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/platform_profile.ko 2024-03-27T10:31:19.802 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/sbs.ko 2024-03-27T10:31:19.802 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/sbshc.ko 2024-03-27T10:31:19.802 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/video.ko 2024-03-27T10:31:19.802 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata 2024-03-27T10:31:19.802 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/acard-ahci.ko 2024-03-27T10:31:19.802 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/ahci_platform.ko 2024-03-27T10:31:19.802 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/ata_generic.ko 2024-03-27T10:31:19.802 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/libahci_platform.ko 2024-03-27T10:31:19.802 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_acpi.ko 2024-03-27T10:31:19.802 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ali.ko 2024-03-27T10:31:19.802 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_amd.ko 2024-03-27T10:31:19.803 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_artop.ko 2024-03-27T10:31:19.803 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_atiixp.ko 2024-03-27T10:31:19.803 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_atp867x.ko 2024-03-27T10:31:19.803 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cmd640.ko 2024-03-27T10:31:19.803 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cmd64x.ko 2024-03-27T10:31:19.803 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cypress.ko 2024-03-27T10:31:19.803 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_efar.ko 2024-03-27T10:31:19.803 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt366.ko 2024-03-27T10:31:19.803 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt37x.ko 2024-03-27T10:31:19.803 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt3x2n.ko 2024-03-27T10:31:19.803 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt3x3.ko 2024-03-27T10:31:19.803 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_it8213.ko 2024-03-27T10:31:19.803 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_it821x.ko 2024-03-27T10:31:19.803 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_jmicron.ko 2024-03-27T10:31:19.804 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_marvell.ko 2024-03-27T10:31:19.804 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_mpiix.ko 2024-03-27T10:31:19.804 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_netcell.ko 2024-03-27T10:31:19.804 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ninja32.ko 2024-03-27T10:31:19.804 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ns87410.ko 2024-03-27T10:31:19.804 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ns87415.ko 2024-03-27T10:31:19.804 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_oldpiix.ko 2024-03-27T10:31:19.804 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_opti.ko 2024-03-27T10:31:19.804 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_optidma.ko 2024-03-27T10:31:19.804 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pcmcia.ko 2024-03-27T10:31:19.804 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pdc2027x.ko 2024-03-27T10:31:19.804 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-03-27T10:31:19.805 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_piccolo.ko 2024-03-27T10:31:19.805 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_rdc.ko 2024-03-27T10:31:19.805 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sch.ko 2024-03-27T10:31:19.805 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_serverworks.ko 2024-03-27T10:31:19.805 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sil680.ko 2024-03-27T10:31:19.805 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sis.ko 2024-03-27T10:31:19.805 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sl82c105.ko 2024-03-27T10:31:19.805 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_triflex.ko 2024-03-27T10:31:19.805 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_via.ko 2024-03-27T10:31:19.805 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pdc_adma.ko 2024-03-27T10:31:19.805 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_inic162x.ko 2024-03-27T10:31:19.805 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_mv.ko 2024-03-27T10:31:19.805 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_nv.ko 2024-03-27T10:31:19.805 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_promise.ko 2024-03-27T10:31:19.806 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_qstor.ko 2024-03-27T10:31:19.806 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sil.ko 2024-03-27T10:31:19.806 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sil24.ko 2024-03-27T10:31:19.806 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sis.ko 2024-03-27T10:31:19.806 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_svw.ko 2024-03-27T10:31:19.806 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sx4.ko 2024-03-27T10:31:19.806 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_uli.ko 2024-03-27T10:31:19.806 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_via.ko 2024-03-27T10:31:19.806 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_vsc.ko 2024-03-27T10:31:19.806 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/auxdisplay 2024-03-27T10:31:19.806 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/auxdisplay/ks0108.ko 2024-03-27T10:31:19.806 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base 2024-03-27T10:31:19.806 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base/regmap 2024-03-27T10:31:19.807 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base/regmap/regmap-i2c.ko 2024-03-27T10:31:19.807 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base/regmap/regmap-mmio.ko 2024-03-27T10:31:19.807 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/bcma 2024-03-27T10:31:19.807 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/bcma/bcma.ko 2024-03-27T10:31:19.807 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block 2024-03-27T10:31:19.807 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/aoe 2024-03-27T10:31:19.807 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/aoe/aoe.ko 2024-03-27T10:31:19.807 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/brd.ko 2024-03-27T10:31:19.807 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/drbd 2024-03-27T10:31:19.807 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/drbd/drbd.ko 2024-03-27T10:31:19.807 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/floppy.ko 2024-03-27T10:31:19.807 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/loop.ko 2024-03-27T10:31:19.807 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/mtip32xx 2024-03-27T10:31:19.808 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-03-27T10:31:19.808 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/nbd.ko 2024-03-27T10:31:19.808 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/null_blk 2024-03-27T10:31:19.808 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/null_blk/null_blk.ko 2024-03-27T10:31:19.808 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/pktcdvd.ko 2024-03-27T10:31:19.808 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/rbd.ko 2024-03-27T10:31:19.808 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/virtio_blk.ko 2024-03-27T10:31:19.808 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkback 2024-03-27T10:31:19.808 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-03-27T10:31:19.808 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkfront.ko 2024-03-27T10:31:19.808 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/zram 2024-03-27T10:31:19.808 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/zram/zram.ko 2024-03-27T10:31:19.808 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char 2024-03-27T10:31:19.808 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hangcheck-timer.ko 2024-03-27T10:31:19.809 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random 2024-03-27T10:31:19.809 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/amd-rng.ko 2024-03-27T10:31:19.809 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/intel-rng.ko 2024-03-27T10:31:19.809 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-03-27T10:31:19.809 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/via-rng.ko 2024-03-27T10:31:19.809 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/virtio-rng.ko 2024-03-27T10:31:19.809 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi 2024-03-27T10:31:19.809 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-03-27T10:31:19.809 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-03-27T10:31:19.809 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-03-27T10:31:19.809 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_si.ko 2024-03-27T10:31:19.809 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-03-27T10:31:19.809 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-03-27T10:31:19.809 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/lp.ko 2024-03-27T10:31:19.810 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/mwave 2024-03-27T10:31:19.810 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/mwave/mwave.ko 2024-03-27T10:31:19.810 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ppdev.ko 2024-03-27T10:31:19.810 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tlclk.ko 2024-03-27T10:31:19.810 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm 2024-03-27T10:31:19.810 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm.ko 2024-03-27T10:31:19.810 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_atmel.ko 2024-03-27T10:31:19.810 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_infineon.ko 2024-03-27T10:31:19.810 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_nsc.ko 2024-03-27T10:31:19.810 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_tis.ko 2024-03-27T10:31:19.810 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-03-27T10:31:19.810 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/uv_mmtimer.ko 2024-03-27T10:31:19.811 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/virtio_console.ko 2024-03-27T10:31:19.811 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq 2024-03-27T10:31:19.811 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-03-27T10:31:19.811 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-03-27T10:31:19.811 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/p4-clockmod.ko 2024-03-27T10:31:19.811 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-03-27T10:31:19.811 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/powernow-k8.ko 2024-03-27T10:31:19.811 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/speedstep-lib.ko 2024-03-27T10:31:19.811 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto 2024-03-27T10:31:19.811 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp 2024-03-27T10:31:19.811 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-03-27T10:31:19.811 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp/ccp.ko 2024-03-27T10:31:19.811 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel 2024-03-27T10:31:19.811 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat 2024-03-27T10:31:19.812 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_common 2024-03-27T10:31:19.812 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-03-27T10:31:19.812 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-03-27T10:31:19.812 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-03-27T10:31:19.812 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/padlock-aes.ko 2024-03-27T10:31:19.812 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/padlock-sha.ko 2024-03-27T10:31:19.812 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/virtio 2024-03-27T10:31:19.812 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-03-27T10:31:19.812 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dca 2024-03-27T10:31:19.812 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dca/dca.ko 2024-03-27T10:31:19.812 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/devfreq 2024-03-27T10:31:19.812 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-03-27T10:31:19.812 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma 2024-03-27T10:31:19.812 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/dw 2024-03-27T10:31:19.813 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/dw/dw_dmac.ko 2024-03-27T10:31:19.813 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/ioat 2024-03-27T10:31:19.813 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/ioat/ioatdma.ko 2024-03-27T10:31:19.813 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac 2024-03-27T10:31:19.813 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/amd64_edac.ko 2024-03-27T10:31:19.813 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/e752x_edac.ko 2024-03-27T10:31:19.813 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/edac_mce_amd.ko 2024-03-27T10:31:19.813 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i3000_edac.ko 2024-03-27T10:31:19.813 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i3200_edac.ko 2024-03-27T10:31:19.813 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i5100_edac.ko 2024-03-27T10:31:19.813 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i5400_edac.ko 2024-03-27T10:31:19.813 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i7300_edac.ko 2024-03-27T10:31:19.813 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i7core_edac.ko 2024-03-27T10:31:19.813 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i82975x_edac.ko 2024-03-27T10:31:19.814 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/ie31200_edac.ko 2024-03-27T10:31:19.814 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/sb_edac.ko 2024-03-27T10:31:19.814 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/x38_edac.ko 2024-03-27T10:31:19.814 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware 2024-03-27T10:31:19.814 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware/edd.ko 2024-03-27T10:31:19.814 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware/iscsi_ibft.ko 2024-03-27T10:31:19.814 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio 2024-03-27T10:31:19.814 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-ich.ko 2024-03-27T10:31:19.814 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-viperboard.ko 2024-03-27T10:31:19.814 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid 2024-03-27T10:31:19.814 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-appleir.ko 2024-03-27T10:31:19.814 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-aureal.ko 2024-03-27T10:31:19.814 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-axff.ko 2024-03-27T10:31:19.815 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-dr.ko 2024-03-27T10:31:19.815 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-elecom.ko 2024-03-27T10:31:19.815 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-elo.ko 2024-03-27T10:31:19.815 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-emsff.ko 2024-03-27T10:31:19.815 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gaff.ko 2024-03-27T10:31:19.815 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gt683r.ko 2024-03-27T10:31:19.815 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gyration.ko 2024-03-27T10:31:19.815 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtek-kbd.ko 2024-03-27T10:31:19.815 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtek-mouse.ko 2024-03-27T10:31:19.816 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtekff.ko 2024-03-27T10:31:19.816 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-hyperv.ko 2024-03-27T10:31:19.816 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-icade.ko 2024-03-27T10:31:19.816 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-keytouch.ko 2024-03-27T10:31:19.816 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-kye.ko 2024-03-27T10:31:19.816 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-lcpower.ko 2024-03-27T10:31:19.816 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-led.ko 2024-03-27T10:31:19.816 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-lenovo.ko 2024-03-27T10:31:19.816 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-logitech-dj.ko 2024-03-27T10:31:19.816 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-03-27T10:31:19.816 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-multitouch.ko 2024-03-27T10:31:19.816 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-ortek.ko 2024-03-27T10:31:19.816 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-penmount.ko 2024-03-27T10:31:19.817 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-petalynx.ko 2024-03-27T10:31:19.817 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-picolcd.ko 2024-03-27T10:31:19.817 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-pl.ko 2024-03-27T10:31:19.817 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-plantronics.ko 2024-03-27T10:31:19.817 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-primax.ko 2024-03-27T10:31:19.817 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-rmi.ko 2024-03-27T10:31:19.817 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-arvo.ko 2024-03-27T10:31:19.817 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-common.ko 2024-03-27T10:31:19.817 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-isku.ko 2024-03-27T10:31:19.817 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-kone.ko 2024-03-27T10:31:19.817 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-03-27T10:31:19.817 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-konepure.ko 2024-03-27T10:31:19.817 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-03-27T10:31:19.818 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-lua.ko 2024-03-27T10:31:19.818 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-pyra.ko 2024-03-27T10:31:19.818 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-ryos.ko 2024-03-27T10:31:19.818 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-savu.ko 2024-03-27T10:31:19.818 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat.ko 2024-03-27T10:31:19.818 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-saitek.ko 2024-03-27T10:31:19.818 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-samsung.ko 2024-03-27T10:31:19.818 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sensor-hub.ko 2024-03-27T10:31:19.818 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sjoy.ko 2024-03-27T10:31:19.818 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sony.ko 2024-03-27T10:31:19.818 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-speedlink.ko 2024-03-27T10:31:19.818 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-steelseries.ko 2024-03-27T10:31:19.818 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sunplus.ko 2024-03-27T10:31:19.819 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-thrustmaster.ko 2024-03-27T10:31:19.819 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-tivo.ko 2024-03-27T10:31:19.819 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-tmff.ko 2024-03-27T10:31:19.819 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-topseed.ko 2024-03-27T10:31:19.819 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-twinhan.ko 2024-03-27T10:31:19.819 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-uclogic.ko 2024-03-27T10:31:19.819 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-waltop.ko 2024-03-27T10:31:19.820 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-wiimote.ko 2024-03-27T10:31:19.820 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-xinmo.ko 2024-03-27T10:31:19.820 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-zpff.ko 2024-03-27T10:31:19.820 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-zydacron.ko 2024-03-27T10:31:19.820 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/uhid.ko 2024-03-27T10:31:19.820 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/wacom.ko 2024-03-27T10:31:19.820 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv 2024-03-27T10:31:19.820 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_balloon.ko 2024-03-27T10:31:19.820 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_utils.ko 2024-03-27T10:31:19.820 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_vmbus.ko 2024-03-27T10:31:19.820 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon 2024-03-27T10:31:19.820 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/abituguru.ko 2024-03-27T10:31:19.820 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/abituguru3.ko 2024-03-27T10:31:19.820 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/acpi_power_meter.ko 2024-03-27T10:31:19.821 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ad7414.ko 2024-03-27T10:31:19.821 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ad7418.ko 2024-03-27T10:31:19.821 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adc128d818.ko 2024-03-27T10:31:19.821 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1025.ko 2024-03-27T10:31:19.821 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1026.ko 2024-03-27T10:31:19.821 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1029.ko 2024-03-27T10:31:19.821 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1031.ko 2024-03-27T10:31:19.821 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm9240.ko 2024-03-27T10:31:19.821 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ads7828.ko 2024-03-27T10:31:19.821 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7410.ko 2024-03-27T10:31:19.821 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7411.ko 2024-03-27T10:31:19.821 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7462.ko 2024-03-27T10:31:19.821 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7470.ko 2024-03-27T10:31:19.821 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7475.ko 2024-03-27T10:31:19.822 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7x10.ko 2024-03-27T10:31:19.822 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/amc6821.ko 2024-03-27T10:31:19.822 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/applesmc.ko 2024-03-27T10:31:19.822 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asb100.ko 2024-03-27T10:31:19.822 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asc7621.ko 2024-03-27T10:31:19.822 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asus_atk0110.ko 2024-03-27T10:31:19.822 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/atxp1.ko 2024-03-27T10:31:19.822 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/coretemp.ko 2024-03-27T10:31:19.822 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-03-27T10:31:19.822 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/dme1737.ko 2024-03-27T10:31:19.822 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ds1621.ko 2024-03-27T10:31:19.822 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ds620.ko 2024-03-27T10:31:19.822 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/emc1403.ko 2024-03-27T10:31:19.823 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/emc6w201.ko 2024-03-27T10:31:19.823 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f71805f.ko 2024-03-27T10:31:19.823 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f71882fg.ko 2024-03-27T10:31:19.823 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f75375s.ko 2024-03-27T10:31:19.823 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/fam15h_power.ko 2024-03-27T10:31:19.823 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/fschmd.ko 2024-03-27T10:31:19.823 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/g760a.ko 2024-03-27T10:31:19.823 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/g762.ko 2024-03-27T10:31:19.823 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/gl518sm.ko 2024-03-27T10:31:19.823 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/gl520sm.ko 2024-03-27T10:31:19.823 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/hwmon-vid.ko 2024-03-27T10:31:19.823 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/i5500_temp.ko 2024-03-27T10:31:19.823 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/i5k_amb.ko 2024-03-27T10:31:19.823 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ibmaem.ko 2024-03-27T10:31:19.824 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ibmpex.ko 2024-03-27T10:31:19.824 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ina209.ko 2024-03-27T10:31:19.824 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ina2xx.ko 2024-03-27T10:31:19.824 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/it87.ko 2024-03-27T10:31:19.824 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/k10temp.ko 2024-03-27T10:31:19.824 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/k8temp.ko 2024-03-27T10:31:19.824 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lineage-pem.ko 2024-03-27T10:31:19.824 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm63.ko 2024-03-27T10:31:19.824 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm73.ko 2024-03-27T10:31:19.824 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm75.ko 2024-03-27T10:31:19.824 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm77.ko 2024-03-27T10:31:19.824 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm78.ko 2024-03-27T10:31:19.824 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm80.ko 2024-03-27T10:31:19.824 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm83.ko 2024-03-27T10:31:19.825 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm85.ko 2024-03-27T10:31:19.825 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm87.ko 2024-03-27T10:31:19.825 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm90.ko 2024-03-27T10:31:19.825 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm92.ko 2024-03-27T10:31:19.825 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm93.ko 2024-03-27T10:31:19.825 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95234.ko 2024-03-27T10:31:19.825 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95241.ko 2024-03-27T10:31:19.825 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95245.ko 2024-03-27T10:31:19.825 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc2945.ko 2024-03-27T10:31:19.825 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4151.ko 2024-03-27T10:31:19.825 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4215.ko 2024-03-27T10:31:19.825 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4222.ko 2024-03-27T10:31:19.825 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4245.ko 2024-03-27T10:31:19.825 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4260.ko 2024-03-27T10:31:19.826 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4261.ko 2024-03-27T10:31:19.826 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max16065.ko 2024-03-27T10:31:19.826 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max1619.ko 2024-03-27T10:31:19.826 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max1668.ko 2024-03-27T10:31:19.826 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max197.ko 2024-03-27T10:31:19.826 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6639.ko 2024-03-27T10:31:19.826 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6650.ko 2024-03-27T10:31:19.826 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6697.ko 2024-03-27T10:31:19.826 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/mcp3021.ko 2024-03-27T10:31:19.826 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6683.ko 2024-03-27T10:31:19.826 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6775-core.ko 2024-03-27T10:31:19.826 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6775.ko 2024-03-27T10:31:19.826 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct7802.ko 2024-03-27T10:31:19.827 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pc87360.ko 2024-03-27T10:31:19.827 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pc87427.ko 2024-03-27T10:31:19.827 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pcf8591.ko 2024-03-27T10:31:19.827 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus 2024-03-27T10:31:19.827 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-03-27T10:31:19.827 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-03-27T10:31:19.827 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-03-27T10:31:19.827 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max16064.ko 2024-03-27T10:31:19.827 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max34440.ko 2024-03-27T10:31:19.827 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max8688.ko 2024-03-27T10:31:19.827 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-03-27T10:31:19.827 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-03-27T10:31:19.827 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-03-27T10:31:19.827 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-03-27T10:31:19.828 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-03-27T10:31:19.828 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-03-27T10:31:19.828 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/powr1220.ko 2024-03-27T10:31:19.828 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch5627.ko 2024-03-27T10:31:19.828 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch5636.ko 2024-03-27T10:31:19.828 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch56xx-common.ko 2024-03-27T10:31:19.828 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sht15.ko 2024-03-27T10:31:19.828 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sht21.ko 2024-03-27T10:31:19.828 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/shtc1.ko 2024-03-27T10:31:19.828 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sis5595.ko 2024-03-27T10:31:19.828 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47b397.ko 2024-03-27T10:31:19.828 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47m1.ko 2024-03-27T10:31:19.828 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47m192.ko 2024-03-27T10:31:19.829 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/thmc50.ko 2024-03-27T10:31:19.829 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp102.ko 2024-03-27T10:31:19.829 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp103.ko 2024-03-27T10:31:19.829 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp401.ko 2024-03-27T10:31:19.829 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp421.ko 2024-03-27T10:31:19.829 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/via-cputemp.ko 2024-03-27T10:31:19.829 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/via686a.ko 2024-03-27T10:31:19.829 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/vt1211.ko 2024-03-27T10:31:19.829 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/vt8231.ko 2024-03-27T10:31:19.829 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83627ehf.ko 2024-03-27T10:31:19.829 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83627hf.ko 2024-03-27T10:31:19.829 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83781d.ko 2024-03-27T10:31:19.829 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83791d.ko 2024-03-27T10:31:19.829 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83792d.ko 2024-03-27T10:31:19.830 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83793.ko 2024-03-27T10:31:19.830 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83795.ko 2024-03-27T10:31:19.830 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83l785ts.ko 2024-03-27T10:31:19.830 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83l786ng.ko 2024-03-27T10:31:19.830 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c 2024-03-27T10:31:19.830 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos 2024-03-27T10:31:19.830 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-03-27T10:31:19.830 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-03-27T10:31:19.830 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses 2024-03-27T10:31:19.830 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-03-27T10:31:19.830 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-03-27T10:31:19.830 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-03-27T10:31:19.830 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-03-27T10:31:19.831 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-03-27T10:31:19.831 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-03-27T10:31:19.831 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-03-27T10:31:19.831 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-03-27T10:31:19.831 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-i801.ko 2024-03-27T10:31:19.831 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-isch.ko 2024-03-27T10:31:19.831 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-03-27T10:31:19.831 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-03-27T10:31:19.831 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-03-27T10:31:19.831 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-parport.ko 2024-03-27T10:31:19.831 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-03-27T10:31:19.831 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-03-27T10:31:19.831 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-03-27T10:31:19.831 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-03-27T10:31:19.832 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-03-27T10:31:19.832 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-03-27T10:31:19.832 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-via.ko 2024-03-27T10:31:19.832 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-03-27T10:31:19.832 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-03-27T10:31:19.832 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-dev.ko 2024-03-27T10:31:19.832 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-mux.ko 2024-03-27T10:31:19.832 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-03-27T10:31:19.832 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-smbus.ko 2024-03-27T10:31:19.832 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-stub.ko 2024-03-27T10:31:19.832 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband 2024-03-27T10:31:19.832 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core 2024-03-27T10:31:19.832 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_cm.ko 2024-03-27T10:31:19.832 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_core.ko 2024-03-27T10:31:19.833 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_umad.ko 2024-03-27T10:31:19.833 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-03-27T10:31:19.833 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/iw_cm.ko 2024-03-27T10:31:19.833 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/rdma_cm.ko 2024-03-27T10:31:19.833 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-03-27T10:31:19.833 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw 2024-03-27T10:31:19.833 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/cxgb4 2024-03-27T10:31:19.833 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-03-27T10:31:19.833 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx4 2024-03-27T10:31:19.833 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-03-27T10:31:19.833 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx5 2024-03-27T10:31:19.833 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-03-27T10:31:19.833 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mthca 2024-03-27T10:31:19.834 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-03-27T10:31:19.834 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp 2024-03-27T10:31:19.834 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/ipoib 2024-03-27T10:31:19.834 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-03-27T10:31:19.834 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/iser 2024-03-27T10:31:19.834 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-03-27T10:31:19.834 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/isert 2024-03-27T10:31:19.834 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-03-27T10:31:19.834 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srp 2024-03-27T10:31:19.834 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-03-27T10:31:19.834 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srpt 2024-03-27T10:31:19.834 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-03-27T10:31:19.834 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input 2024-03-27T10:31:19.834 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport 2024-03-27T10:31:19.835 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-03-27T10:31:19.835 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/fm801-gp.ko 2024-03-27T10:31:19.835 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/gameport.ko 2024-03-27T10:31:19.835 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/joydev.ko 2024-03-27T10:31:19.835 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard 2024-03-27T10:31:19.835 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/gpio_keys.ko 2024-03-27T10:31:19.835 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse 2024-03-27T10:31:19.835 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/appletouch.ko 2024-03-27T10:31:19.835 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/bcm5974.ko 2024-03-27T10:31:19.835 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/cyapatp.ko 2024-03-27T10:31:19.835 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/elan_i2c.ko 2024-03-27T10:31:19.835 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/sermouse.ko 2024-03-27T10:31:19.835 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-03-27T10:31:19.836 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/synaptics_usb.ko 2024-03-27T10:31:19.836 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/vsxxxaa.ko 2024-03-27T10:31:19.836 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/rmi4 2024-03-27T10:31:19.836 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/rmi4/rmi_core.ko 2024-03-27T10:31:19.836 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio 2024-03-27T10:31:19.836 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/altera_ps2.ko 2024-03-27T10:31:19.836 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/arc_ps2.ko 2024-03-27T10:31:19.836 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-03-27T10:31:19.836 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/serio_raw.ko 2024-03-27T10:31:19.836 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/sparse-keymap.ko 2024-03-27T10:31:19.836 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds 2024-03-27T10:31:19.836 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-blinkm.ko 2024-03-27T10:31:19.836 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-lm3530.ko 2024-03-27T10:31:19.837 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-lp3944.ko 2024-03-27T10:31:19.837 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-ss4200.ko 2024-03-27T10:31:19.837 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger 2024-03-27T10:31:19.837 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-03-27T10:31:19.837 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-03-27T10:31:19.837 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-03-27T10:31:19.837 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-03-27T10:31:19.837 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-03-27T10:31:19.837 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-03-27T10:31:19.837 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-03-27T10:31:19.837 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-03-27T10:31:19.837 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-03-27T10:31:19.837 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md 2024-03-27T10:31:19.837 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/bcache 2024-03-27T10:31:19.838 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/bcache/bcache.ko 2024-03-27T10:31:19.838 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-bio-prison.ko 2024-03-27T10:31:19.838 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-cache-smq.ko 2024-03-27T10:31:19.838 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-cache.ko 2024-03-27T10:31:19.838 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-crypt.ko 2024-03-27T10:31:19.838 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-delay.ko 2024-03-27T10:31:19.838 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-flakey.ko 2024-03-27T10:31:19.838 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-log-userspace.ko 2024-03-27T10:31:19.838 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-multipath.ko 2024-03-27T10:31:19.838 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-queue-length.ko 2024-03-27T10:31:19.838 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-raid.ko 2024-03-27T10:31:19.838 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-round-robin.ko 2024-03-27T10:31:19.838 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-service-time.ko 2024-03-27T10:31:19.839 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-switch.ko 2024-03-27T10:31:19.839 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-thin-pool.ko 2024-03-27T10:31:19.839 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-verity.ko 2024-03-27T10:31:19.839 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/persistent-data 2024-03-27T10:31:19.839 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-03-27T10:31:19.839 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid0.ko 2024-03-27T10:31:19.839 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid1.ko 2024-03-27T10:31:19.839 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid10.ko 2024-03-27T10:31:19.839 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid456.ko 2024-03-27T10:31:19.839 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message 2024-03-27T10:31:19.839 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion 2024-03-27T10:31:19.839 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptbase.ko 2024-03-27T10:31:19.839 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptctl.ko 2024-03-27T10:31:19.840 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptfc.ko 2024-03-27T10:31:19.840 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptlan.ko 2024-03-27T10:31:19.840 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptsas.ko 2024-03-27T10:31:19.840 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptscsih.ko 2024-03-27T10:31:19.840 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptspi.ko 2024-03-27T10:31:19.840 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd 2024-03-27T10:31:19.840 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/lpc_ich.ko 2024-03-27T10:31:19.840 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/lpc_sch.ko 2024-03-27T10:31:19.840 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/mfd-core.ko 2024-03-27T10:31:19.840 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/sm501.ko 2024-03-27T10:31:19.840 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/viperboard.ko 2024-03-27T10:31:19.840 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/vx855.ko 2024-03-27T10:31:19.840 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/wl1273-core.ko 2024-03-27T10:31:19.840 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc 2024-03-27T10:31:19.840 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/altera-stapl 2024-03-27T10:31:19.841 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-03-27T10:31:19.841 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/apds9802als.ko 2024-03-27T10:31:19.841 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/apds990x.ko 2024-03-27T10:31:19.841 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/bh1770glc.ko 2024-03-27T10:31:19.841 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/cb710 2024-03-27T10:31:19.841 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/cb710/cb710.ko 2024-03-27T10:31:19.841 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom 2024-03-27T10:31:19.841 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/at24.ko 2024-03-27T10:31:19.841 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-03-27T10:31:19.841 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/max6875.ko 2024-03-27T10:31:19.841 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/enclosure.ko 2024-03-27T10:31:19.841 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/hpilo.ko 2024-03-27T10:31:19.841 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/isl29003.ko 2024-03-27T10:31:19.842 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/isl29020.ko 2024-03-27T10:31:19.842 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d 2024-03-27T10:31:19.842 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-03-27T10:31:19.842 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-03-27T10:31:19.842 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei 2024-03-27T10:31:19.842 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei-me.ko 2024-03-27T10:31:19.842 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei-txe.ko 2024-03-27T10:31:19.842 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei.ko 2024-03-27T10:31:19.842 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-gru 2024-03-27T10:31:19.842 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-gru/gru.ko 2024-03-27T10:31:19.842 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp 2024-03-27T10:31:19.842 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp/xp.ko 2024-03-27T10:31:19.842 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp/xpc.ko 2024-03-27T10:31:19.843 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-03-27T10:31:19.843 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tifm_7xx1.ko 2024-03-27T10:31:19.843 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tifm_core.ko 2024-03-27T10:31:19.843 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tsl2550.ko 2024-03-27T10:31:19.843 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_balloon.ko 2024-03-27T10:31:19.843 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_vmci 2024-03-27T10:31:19.843 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-03-27T10:31:19.843 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net 2024-03-27T10:31:19.843 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/bonding 2024-03-27T10:31:19.843 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/bonding/bonding.ko 2024-03-27T10:31:19.843 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa 2024-03-27T10:31:19.843 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/b53 2024-03-27T10:31:19.843 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/b53/b53_common.ko 2024-03-27T10:31:19.843 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/bcm-sf2.ko 2024-03-27T10:31:19.844 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6060.ko 2024-03-27T10:31:19.844 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6xxx 2024-03-27T10:31:19.844 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-03-27T10:31:19.844 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dummy.ko 2024-03-27T10:31:19.844 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/eql.ko 2024-03-27T10:31:19.844 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet 2024-03-27T10:31:19.844 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com 2024-03-27T10:31:19.844 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-03-27T10:31:19.844 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-03-27T10:31:19.844 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-03-27T10:31:19.844 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-03-27T10:31:19.844 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390 2024-03-27T10:31:19.844 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/8390.ko 2024-03-27T10:31:19.844 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-03-27T10:31:19.845 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-03-27T10:31:19.845 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-03-27T10:31:19.845 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/adaptec 2024-03-27T10:31:19.845 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-03-27T10:31:19.845 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/agere 2024-03-27T10:31:19.845 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/agere/et131x.ko 2024-03-27T10:31:19.845 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/alteon 2024-03-27T10:31:19.845 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-03-27T10:31:19.845 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/altera 2024-03-27T10:31:19.845 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-03-27T10:31:19.845 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd 2024-03-27T10:31:19.845 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-03-27T10:31:19.846 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-03-27T10:31:19.846 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-03-27T10:31:19.846 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros 2024-03-27T10:31:19.846 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/alx 2024-03-27T10:31:19.846 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-03-27T10:31:19.846 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1c 2024-03-27T10:31:19.846 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-03-27T10:31:19.846 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1e 2024-03-27T10:31:19.846 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-03-27T10:31:19.846 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx 2024-03-27T10:31:19.846 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-03-27T10:31:19.846 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-03-27T10:31:19.846 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom 2024-03-27T10:31:19.846 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-03-27T10:31:19.847 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-03-27T10:31:19.847 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-03-27T10:31:19.847 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-03-27T10:31:19.847 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-03-27T10:31:19.847 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/genet 2024-03-27T10:31:19.847 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-03-27T10:31:19.847 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-03-27T10:31:19.847 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade 2024-03-27T10:31:19.847 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade/bna 2024-03-27T10:31:19.847 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-03-27T10:31:19.847 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio 2024-03-27T10:31:19.847 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb 2024-03-27T10:31:19.847 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-03-27T10:31:19.847 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-03-27T10:31:19.848 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-03-27T10:31:19.848 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-03-27T10:31:19.848 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-03-27T10:31:19.848 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-03-27T10:31:19.848 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-03-27T10:31:19.848 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-03-27T10:31:19.848 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-03-27T10:31:19.848 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco 2024-03-27T10:31:19.848 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco/enic 2024-03-27T10:31:19.848 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-03-27T10:31:19.848 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec 2024-03-27T10:31:19.848 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip 2024-03-27T10:31:19.848 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-03-27T10:31:19.849 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-03-27T10:31:19.849 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-03-27T10:31:19.849 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-03-27T10:31:19.849 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-03-27T10:31:19.849 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-03-27T10:31:19.849 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink 2024-03-27T10:31:19.849 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-03-27T10:31:19.849 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-03-27T10:31:19.849 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dnet.ko 2024-03-27T10:31:19.849 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex 2024-03-27T10:31:19.849 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex/benet 2024-03-27T10:31:19.849 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-03-27T10:31:19.849 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ethoc.ko 2024-03-27T10:31:19.849 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/fealnx.ko 2024-03-27T10:31:19.850 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel 2024-03-27T10:31:19.850 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e100.ko 2024-03-27T10:31:19.850 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000 2024-03-27T10:31:19.850 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-03-27T10:31:19.850 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000e 2024-03-27T10:31:19.850 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-03-27T10:31:19.850 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/fm10k 2024-03-27T10:31:19.850 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-03-27T10:31:19.850 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/i40e 2024-03-27T10:31:19.850 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-03-27T10:31:19.850 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/iavf 2024-03-27T10:31:19.850 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-03-27T10:31:19.850 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igb 2024-03-27T10:31:19.851 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-03-27T10:31:19.851 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igbvf 2024-03-27T10:31:19.851 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-03-27T10:31:19.851 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbe 2024-03-27T10:31:19.851 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-03-27T10:31:19.851 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbevf 2024-03-27T10:31:19.851 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-03-27T10:31:19.851 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/jme.ko 2024-03-27T10:31:19.851 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell 2024-03-27T10:31:19.851 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-03-27T10:31:19.851 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/skge.ko 2024-03-27T10:31:19.851 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-03-27T10:31:19.851 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox 2024-03-27T10:31:19.851 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4 2024-03-27T10:31:19.852 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-03-27T10:31:19.852 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-03-27T10:31:19.852 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5 2024-03-27T10:31:19.852 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-03-27T10:31:19.852 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-03-27T10:31:19.852 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/micrel 2024-03-27T10:31:19.852 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-03-27T10:31:19.852 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom 2024-03-27T10:31:19.852 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom/myri10ge 2024-03-27T10:31:19.852 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-03-27T10:31:19.852 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi 2024-03-27T10:31:19.852 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-03-27T10:31:19.852 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-03-27T10:31:19.853 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/neterion 2024-03-27T10:31:19.853 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-03-27T10:31:19.853 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/nvidia 2024-03-27T10:31:19.853 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-03-27T10:31:19.853 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines 2024-03-27T10:31:19.853 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-03-27T10:31:19.853 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-03-27T10:31:19.853 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic 2024-03-27T10:31:19.853 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/netxen 2024-03-27T10:31:19.853 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-03-27T10:31:19.853 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-03-27T10:31:19.853 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-03-27T10:31:19.853 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-03-27T10:31:19.854 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rdc 2024-03-27T10:31:19.854 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-03-27T10:31:19.854 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek 2024-03-27T10:31:19.854 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-03-27T10:31:19.854 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-03-27T10:31:19.854 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/atp.ko 2024-03-27T10:31:19.854 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-03-27T10:31:19.854 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rocker 2024-03-27T10:31:19.854 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-03-27T10:31:19.854 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sfc 2024-03-27T10:31:19.854 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-03-27T10:31:19.854 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/silan 2024-03-27T10:31:19.854 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-03-27T10:31:19.855 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis 2024-03-27T10:31:19.855 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis/sis190.ko 2024-03-27T10:31:19.855 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis/sis900.ko 2024-03-27T10:31:19.855 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc 2024-03-27T10:31:19.855 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-03-27T10:31:19.855 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-03-27T10:31:19.855 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-03-27T10:31:19.855 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-03-27T10:31:19.855 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro 2024-03-27T10:31:19.855 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac 2024-03-27T10:31:19.855 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-03-27T10:31:19.855 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-03-27T10:31:19.855 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun 2024-03-27T10:31:19.855 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/cassini.ko 2024-03-27T10:31:19.856 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/niu.ko 2024-03-27T10:31:19.856 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/sungem.ko 2024-03-27T10:31:19.856 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-03-27T10:31:19.856 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/tehuti 2024-03-27T10:31:19.856 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-03-27T10:31:19.856 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ti 2024-03-27T10:31:19.856 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ti/tlan.ko 2024-03-27T10:31:19.856 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via 2024-03-27T10:31:19.856 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-03-27T10:31:19.856 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-03-27T10:31:19.856 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet 2024-03-27T10:31:19.856 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-03-27T10:31:19.856 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-03-27T10:31:19.856 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/xircom 2024-03-27T10:31:19.857 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-03-27T10:31:19.857 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/geneve.ko 2024-03-27T10:31:19.857 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/hyperv 2024-03-27T10:31:19.857 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-03-27T10:31:19.857 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ieee802154 2024-03-27T10:31:19.857 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ieee802154/fakelb.ko 2024-03-27T10:31:19.857 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ifb.ko 2024-03-27T10:31:19.857 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ipvlan 2024-03-27T10:31:19.857 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ipvlan/ipvlan.ko 2024-03-27T10:31:19.857 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/macvlan.ko 2024-03-27T10:31:19.857 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/macvtap.ko 2024-03-27T10:31:19.857 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio 2024-03-27T10:31:19.857 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio.ko 2024-03-27T10:31:19.857 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-03-27T10:31:19.858 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-03-27T10:31:19.858 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-regmap.ko 2024-03-27T10:31:19.858 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mii.ko 2024-03-27T10:31:19.858 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/net_failover.ko 2024-03-27T10:31:19.858 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/netconsole.ko 2024-03-27T10:31:19.858 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/nlmon.ko 2024-03-27T10:31:19.858 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs 2024-03-27T10:31:19.858 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs/pcs-lynx.ko 2024-03-27T10:31:19.858 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-03-27T10:31:19.858 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy 2024-03-27T10:31:19.858 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/amd.ko 2024-03-27T10:31:19.858 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-03-27T10:31:19.859 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-03-27T10:31:19.859 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm7xxx.ko 2024-03-27T10:31:19.859 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm87xx.ko 2024-03-27T10:31:19.859 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/broadcom.ko 2024-03-27T10:31:19.859 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/cicada.ko 2024-03-27T10:31:19.859 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/davicom.ko 2024-03-27T10:31:19.859 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/dp83640.ko 2024-03-27T10:31:19.859 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/et1011c.ko 2024-03-27T10:31:19.859 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/icplus.ko 2024-03-27T10:31:19.859 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/lxt.ko 2024-03-27T10:31:19.859 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/marvell.ko 2024-03-27T10:31:19.859 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/micrel.ko 2024-03-27T10:31:19.859 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/national.ko 2024-03-27T10:31:19.859 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/phylink.ko 2024-03-27T10:31:19.860 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/qsemi.ko 2024-03-27T10:31:19.860 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/realtek.ko 2024-03-27T10:31:19.860 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/smsc.ko 2024-03-27T10:31:19.860 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/ste10Xp.ko 2024-03-27T10:31:19.860 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/vitesse.ko 2024-03-27T10:31:19.860 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp 2024-03-27T10:31:19.860 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/bsd_comp.ko 2024-03-27T10:31:19.860 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_async.ko 2024-03-27T10:31:19.860 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_deflate.ko 2024-03-27T10:31:19.860 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_generic.ko 2024-03-27T10:31:19.860 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_mppe.ko 2024-03-27T10:31:19.860 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_synctty.ko 2024-03-27T10:31:19.860 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pppoe.ko 2024-03-27T10:31:19.861 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pppox.ko 2024-03-27T10:31:19.861 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pptp.ko 2024-03-27T10:31:19.861 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip 2024-03-27T10:31:19.861 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip/slhc.ko 2024-03-27T10:31:19.861 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip/slip.ko 2024-03-27T10:31:19.861 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/sungem_phy.ko 2024-03-27T10:31:19.861 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/tap.ko 2024-03-27T10:31:19.861 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team 2024-03-27T10:31:19.861 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team.ko 2024-03-27T10:31:19.861 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_activebackup.ko 2024-03-27T10:31:19.861 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_broadcast.ko 2024-03-27T10:31:19.861 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-03-27T10:31:19.861 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_random.ko 2024-03-27T10:31:19.861 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-03-27T10:31:19.862 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/tun.ko 2024-03-27T10:31:19.862 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/veth.ko 2024-03-27T10:31:19.862 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/virtio_net.ko 2024-03-27T10:31:19.862 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vmxnet3 2024-03-27T10:31:19.862 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-03-27T10:31:19.862 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vxlan 2024-03-27T10:31:19.862 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vxlan/vxlan.ko 2024-03-27T10:31:19.862 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netback 2024-03-27T10:31:19.862 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netback/xen-netback.ko 2024-03-27T10:31:19.862 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netfront.ko 2024-03-27T10:31:19.862 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ntb 2024-03-27T10:31:19.862 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ntb/ntb.ko 2024-03-27T10:31:19.862 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme 2024-03-27T10:31:19.863 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host 2024-03-27T10:31:19.863 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host/nvme-core.ko 2024-03-27T10:31:19.863 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host/nvme.ko 2024-03-27T10:31:19.863 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport 2024-03-27T10:31:19.863 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport.ko 2024-03-27T10:31:19.863 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_cs.ko 2024-03-27T10:31:19.863 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_pc.ko 2024-03-27T10:31:19.863 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_serial.ko 2024-03-27T10:31:19.863 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci 2024-03-27T10:31:19.863 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/hotplug 2024-03-27T10:31:19.863 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-03-27T10:31:19.863 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/pcie 2024-03-27T10:31:19.863 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/pcie/aer_inject.ko 2024-03-27T10:31:19.863 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/xen-pcifront.ko 2024-03-27T10:31:19.864 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia 2024-03-27T10:31:19.864 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/i82092.ko 2024-03-27T10:31:19.864 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/pd6729.ko 2024-03-27T10:31:19.864 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/yenta_socket.ko 2024-03-27T10:31:19.864 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl 2024-03-27T10:31:19.864 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl/intel 2024-03-27T10:31:19.864 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-03-27T10:31:19.864 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform 2024-03-27T10:31:19.864 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86 2024-03-27T10:31:19.864 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/acer-wmi.ko 2024-03-27T10:31:19.864 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/acerhdf.ko 2024-03-27T10:31:19.864 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/apple-gmux.ko 2024-03-27T10:31:19.864 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-laptop.ko 2024-03-27T10:31:19.864 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-03-27T10:31:19.865 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-wmi.ko 2024-03-27T10:31:19.865 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/classmate-laptop.ko 2024-03-27T10:31:19.865 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-03-27T10:31:19.865 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-03-27T10:31:19.865 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-03-27T10:31:19.865 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-03-27T10:31:19.865 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/hdaps.ko 2024-03-27T10:31:19.865 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel 2024-03-27T10:31:19.865 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-03-27T10:31:19.865 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel_ips.ko 2024-03-27T10:31:19.865 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/msi-wmi.ko 2024-03-27T10:31:19.865 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/mxm-wmi.ko 2024-03-27T10:31:19.865 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-03-27T10:31:19.865 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/samsung-laptop.ko 2024-03-27T10:31:19.866 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/samsung-q10.ko 2024-03-27T10:31:19.866 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/topstar-laptop.ko 2024-03-27T10:31:19.866 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-03-27T10:31:19.866 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/toshiba_haps.ko 2024-03-27T10:31:19.866 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/wmi-bmof.ko 2024-03-27T10:31:19.866 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/wmi.ko 2024-03-27T10:31:19.866 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap 2024-03-27T10:31:19.866 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap/intel_rapl_common.ko 2024-03-27T10:31:19.866 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap/intel_rapl_msr.ko 2024-03-27T10:31:19.866 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps 2024-03-27T10:31:19.866 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients 2024-03-27T10:31:19.866 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps-gpio.ko 2024-03-27T10:31:19.866 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps-ldisc.ko 2024-03-27T10:31:19.867 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps_parport.ko 2024-03-27T10:31:19.867 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/pps_core.ko 2024-03-27T10:31:19.867 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp 2024-03-27T10:31:19.867 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp/ptp.ko 2024-03-27T10:31:19.867 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp/ptp_kvm.ko 2024-03-27T10:31:19.867 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc 2024-03-27T10:31:19.867 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-bq32k.ko 2024-03-27T10:31:19.867 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1286.ko 2024-03-27T10:31:19.867 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1307.ko 2024-03-27T10:31:19.867 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1374.ko 2024-03-27T10:31:19.867 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1511.ko 2024-03-27T10:31:19.867 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1553.ko 2024-03-27T10:31:19.867 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1672.ko 2024-03-27T10:31:19.868 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1742.ko 2024-03-27T10:31:19.868 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds2404.ko 2024-03-27T10:31:19.868 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds3232.ko 2024-03-27T10:31:19.868 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-em3027.ko 2024-03-27T10:31:19.868 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-fm3130.ko 2024-03-27T10:31:19.868 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-isl12022.ko 2024-03-27T10:31:19.868 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-isl1208.ko 2024-03-27T10:31:19.868 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m41t80.ko 2024-03-27T10:31:19.868 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m48t35.ko 2024-03-27T10:31:19.868 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m48t59.ko 2024-03-27T10:31:19.868 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-max6900.ko 2024-03-27T10:31:19.868 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-msm6242.ko 2024-03-27T10:31:19.868 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf2127.ko 2024-03-27T10:31:19.868 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf85063.ko 2024-03-27T10:31:19.869 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8523.ko 2024-03-27T10:31:19.869 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8563.ko 2024-03-27T10:31:19.869 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8583.ko 2024-03-27T10:31:19.869 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rp5c01.ko 2024-03-27T10:31:19.869 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rs5c372.ko 2024-03-27T10:31:19.869 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-03-27T10:31:19.869 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rx8025.ko 2024-03-27T10:31:19.869 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rx8581.ko 2024-03-27T10:31:19.869 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-03-27T10:31:19.869 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-x1205.ko 2024-03-27T10:31:19.869 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi 2024-03-27T10:31:19.869 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-9xxx.ko 2024-03-27T10:31:19.869 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-sas.ko 2024-03-27T10:31:19.869 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-xxxx.ko 2024-03-27T10:31:19.870 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/BusLogic.ko 2024-03-27T10:31:19.870 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/a100u2w.ko 2024-03-27T10:31:19.870 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aacraid 2024-03-27T10:31:19.870 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aacraid/aacraid.ko 2024-03-27T10:31:19.870 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/advansys.ko 2024-03-27T10:31:19.870 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx 2024-03-27T10:31:19.870 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-03-27T10:31:19.870 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-03-27T10:31:19.870 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/am53c974.ko 2024-03-27T10:31:19.870 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/arcmsr 2024-03-27T10:31:19.870 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-03-27T10:31:19.870 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/atp870u.ko 2024-03-27T10:31:19.870 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/be2iscsi 2024-03-27T10:31:19.871 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-03-27T10:31:19.871 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bfa 2024-03-27T10:31:19.871 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bfa/bfa.ko 2024-03-27T10:31:19.871 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2fc 2024-03-27T10:31:19.871 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-03-27T10:31:19.871 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2i 2024-03-27T10:31:19.871 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-03-27T10:31:19.871 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ch.ko 2024-03-27T10:31:19.871 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/csiostor 2024-03-27T10:31:19.871 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/csiostor/csiostor.ko 2024-03-27T10:31:19.871 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi 2024-03-27T10:31:19.871 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb3i 2024-03-27T10:31:19.871 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-03-27T10:31:19.871 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb4i 2024-03-27T10:31:19.872 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-03-27T10:31:19.872 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-03-27T10:31:19.872 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/dc395x.ko 2024-03-27T10:31:19.872 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler 2024-03-27T10:31:19.872 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-03-27T10:31:19.872 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-03-27T10:31:19.872 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-03-27T10:31:19.872 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-03-27T10:31:19.872 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/dmx3191d.ko 2024-03-27T10:31:19.872 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esas2r 2024-03-27T10:31:19.872 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esas2r/esas2r.ko 2024-03-27T10:31:19.872 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esp_scsi.ko 2024-03-27T10:31:19.872 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe 2024-03-27T10:31:19.872 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe/fcoe.ko 2024-03-27T10:31:19.873 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-03-27T10:31:19.873 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fnic 2024-03-27T10:31:19.873 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fnic/fnic.ko 2024-03-27T10:31:19.873 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hpsa.ko 2024-03-27T10:31:19.873 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hptiop.ko 2024-03-27T10:31:19.873 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hv_storvsc.ko 2024-03-27T10:31:19.873 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/initio.ko 2024-03-27T10:31:19.873 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ipr.ko 2024-03-27T10:31:19.873 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ips.ko 2024-03-27T10:31:19.873 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/isci 2024-03-27T10:31:19.873 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/isci/isci.ko 2024-03-27T10:31:19.873 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-03-27T10:31:19.873 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/iscsi_tcp.ko 2024-03-27T10:31:19.874 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libfc 2024-03-27T10:31:19.874 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libfc/libfc.ko 2024-03-27T10:31:19.874 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libiscsi.ko 2024-03-27T10:31:19.874 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libiscsi_tcp.ko 2024-03-27T10:31:19.874 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libsas 2024-03-27T10:31:19.874 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libsas/libsas.ko 2024-03-27T10:31:19.874 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/lpfc 2024-03-27T10:31:19.874 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/lpfc/lpfc.ko 2024-03-27T10:31:19.874 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid 2024-03-27T10:31:19.874 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid.ko 2024-03-27T10:31:19.874 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-03-27T10:31:19.874 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-03-27T10:31:19.874 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-03-27T10:31:19.875 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mpt3sas 2024-03-27T10:31:19.875 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-03-27T10:31:19.875 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvsas 2024-03-27T10:31:19.875 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvsas/mvsas.ko 2024-03-27T10:31:19.875 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvumi.ko 2024-03-27T10:31:19.875 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pm8001 2024-03-27T10:31:19.875 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-03-27T10:31:19.875 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pmcraid.ko 2024-03-27T10:31:19.875 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla1280.ko 2024-03-27T10:31:19.875 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx 2024-03-27T10:31:19.875 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-03-27T10:31:19.875 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-03-27T10:31:19.875 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla4xxx 2024-03-27T10:31:19.875 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-03-27T10:31:19.876 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/raid_class.ko 2024-03-27T10:31:19.876 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_debug.ko 2024-03-27T10:31:19.876 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_fc.ko 2024-03-27T10:31:19.876 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-03-27T10:31:19.876 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_sas.ko 2024-03-27T10:31:19.876 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_spi.ko 2024-03-27T10:31:19.876 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_srp.ko 2024-03-27T10:31:19.876 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ses.ko 2024-03-27T10:31:19.876 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/st.ko 2024-03-27T10:31:19.876 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/stex.ko 2024-03-27T10:31:19.876 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/sym53c8xx_2 2024-03-27T10:31:19.876 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-03-27T10:31:19.876 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/virtio_scsi.ko 2024-03-27T10:31:19.876 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/vmw_pvscsi.ko 2024-03-27T10:31:19.877 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/wd719x.ko 2024-03-27T10:31:19.877 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ssb 2024-03-27T10:31:19.877 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ssb/ssb.ko 2024-03-27T10:31:19.877 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target 2024-03-27T10:31:19.877 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/iscsi 2024-03-27T10:31:19.877 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-03-27T10:31:19.877 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/loopback 2024-03-27T10:31:19.877 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/loopback/tcm_loop.ko 2024-03-27T10:31:19.877 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_file.ko 2024-03-27T10:31:19.877 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_iblock.ko 2024-03-27T10:31:19.877 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_mod.ko 2024-03-27T10:31:19.877 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_pscsi.ko 2024-03-27T10:31:19.877 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_user.ko 2024-03-27T10:31:19.878 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/tcm_fc 2024-03-27T10:31:19.878 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-03-27T10:31:19.878 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal 2024-03-27T10:31:19.878 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel 2024-03-27T10:31:19.878 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal 2024-03-27T10:31:19.878 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-03-27T10:31:19.878 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-03-27T10:31:19.878 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-03-27T10:31:19.878 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-03-27T10:31:19.878 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-03-27T10:31:19.878 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-03-27T10:31:19.878 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-03-27T10:31:19.878 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-03-27T10:31:19.878 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-03-27T10:31:19.879 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-03-27T10:31:19.879 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-03-27T10:31:19.879 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-03-27T10:31:19.879 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-03-27T10:31:19.879 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-03-27T10:31:19.879 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-03-27T10:31:19.879 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-03-27T10:31:19.879 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-03-27T10:31:19.879 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-03-27T10:31:19.879 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty 2024-03-27T10:31:19.879 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/ipwireless 2024-03-27T10:31:19.879 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-03-27T10:31:19.879 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/n_gsm.ko 2024-03-27T10:31:19.880 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/n_hdlc.ko 2024-03-27T10:31:19.880 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/nozomi.ko 2024-03-27T10:31:19.880 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial 2024-03-27T10:31:19.880 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/8250 2024-03-27T10:31:19.880 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-03-27T10:31:19.880 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/arc_uart.ko 2024-03-27T10:31:19.880 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/jsm 2024-03-27T10:31:19.880 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/jsm/jsm.ko 2024-03-27T10:31:19.880 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs 2024-03-27T10:31:19.880 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/core 2024-03-27T10:31:19.880 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/core/ufshcd-core.ko 2024-03-27T10:31:19.880 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/host 2024-03-27T10:31:19.880 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-03-27T10:31:19.880 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio 2024-03-27T10:31:19.881 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio.ko 2024-03-27T10:31:19.881 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_aec.ko 2024-03-27T10:31:19.881 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_cif.ko 2024-03-27T10:31:19.881 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_pci_generic.ko 2024-03-27T10:31:19.881 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_sercos3.ko 2024-03-27T10:31:19.881 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb 2024-03-27T10:31:19.881 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class 2024-03-27T10:31:19.881 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/cdc-acm.ko 2024-03-27T10:31:19.881 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/cdc-wdm.ko 2024-03-27T10:31:19.881 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/usblp.ko 2024-03-27T10:31:19.881 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/usbtmc.ko 2024-03-27T10:31:19.881 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host 2024-03-27T10:31:19.881 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host/sl811-hcd.ko 2024-03-27T10:31:19.881 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image 2024-03-27T10:31:19.882 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image/mdc800.ko 2024-03-27T10:31:19.882 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image/microtek.ko 2024-03-27T10:31:19.882 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc 2024-03-27T10:31:19.882 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/adutux.ko 2024-03-27T10:31:19.882 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/appledisplay.ko 2024-03-27T10:31:19.882 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/emi26.ko 2024-03-27T10:31:19.882 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/emi62.ko 2024-03-27T10:31:19.882 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/ezusb.ko 2024-03-27T10:31:19.882 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/idmouse.ko 2024-03-27T10:31:19.882 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/iowarrior.ko 2024-03-27T10:31:19.882 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/isight_firmware.ko 2024-03-27T10:31:19.882 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/ldusb.ko 2024-03-27T10:31:19.882 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/legousbtower.ko 2024-03-27T10:31:19.882 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/sisusbvga 2024-03-27T10:31:19.883 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-03-27T10:31:19.883 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/trancevibrator.ko 2024-03-27T10:31:19.883 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usb3503.ko 2024-03-27T10:31:19.883 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usblcd.ko 2024-03-27T10:31:19.883 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usbsevseg.ko 2024-03-27T10:31:19.883 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/uss720.ko 2024-03-27T10:31:19.883 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/yurex.ko 2024-03-27T10:31:19.883 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/phy 2024-03-27T10:31:19.883 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/phy/phy-generic.ko 2024-03-27T10:31:19.883 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial 2024-03-27T10:31:19.883 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/aircable.ko 2024-03-27T10:31:19.883 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ark3116.ko 2024-03-27T10:31:19.883 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/belkin_sa.ko 2024-03-27T10:31:19.884 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ch341.ko 2024-03-27T10:31:19.884 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cp210x.ko 2024-03-27T10:31:19.884 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cyberjack.ko 2024-03-27T10:31:19.884 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cypress_m8.ko 2024-03-27T10:31:19.884 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/digi_acceleport.ko 2024-03-27T10:31:19.884 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/empeg.ko 2024-03-27T10:31:19.884 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ftdi_sio.ko 2024-03-27T10:31:19.884 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/garmin_gps.ko 2024-03-27T10:31:19.884 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/io_edgeport.ko 2024-03-27T10:31:19.884 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/io_ti.ko 2024-03-27T10:31:19.884 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ipaq.ko 2024-03-27T10:31:19.884 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ipw.ko 2024-03-27T10:31:19.884 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ir-usb.ko 2024-03-27T10:31:19.885 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-03-27T10:31:19.885 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/keyspan.ko 2024-03-27T10:31:19.885 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/keyspan_pda.ko 2024-03-27T10:31:19.885 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/kl5kusb105.ko 2024-03-27T10:31:19.885 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/kobil_sct.ko 2024-03-27T10:31:19.885 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mct_u232.ko 2024-03-27T10:31:19.885 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mos7720.ko 2024-03-27T10:31:19.885 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mos7840.ko 2024-03-27T10:31:19.885 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/navman.ko 2024-03-27T10:31:19.885 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/omninet.ko 2024-03-27T10:31:19.885 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/opticon.ko 2024-03-27T10:31:19.885 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/option.ko 2024-03-27T10:31:19.885 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/oti6858.ko 2024-03-27T10:31:19.885 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/pl2303.ko 2024-03-27T10:31:19.886 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/qcaux.ko 2024-03-27T10:31:19.886 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/qcserial.ko 2024-03-27T10:31:19.886 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/quatech2.ko 2024-03-27T10:31:19.886 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/safe_serial.ko 2024-03-27T10:31:19.886 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/sierra.ko 2024-03-27T10:31:19.886 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/spcp8x5.ko 2024-03-27T10:31:19.886 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ssu100.ko 2024-03-27T10:31:19.886 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/symbolserial.ko 2024-03-27T10:31:19.886 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-03-27T10:31:19.886 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-03-27T10:31:19.886 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb_debug.ko 2024-03-27T10:31:19.886 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb_wwan.ko 2024-03-27T10:31:19.886 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/visor.ko 2024-03-27T10:31:19.886 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/whiteheat.ko 2024-03-27T10:31:19.887 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/xsens_mt.ko 2024-03-27T10:31:19.887 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage 2024-03-27T10:31:19.887 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/uas.ko 2024-03-27T10:31:19.887 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-alauda.ko 2024-03-27T10:31:19.887 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-cypress.ko 2024-03-27T10:31:19.887 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-datafab.ko 2024-03-27T10:31:19.887 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-03-27T10:31:19.887 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-freecom.ko 2024-03-27T10:31:19.887 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-isd200.ko 2024-03-27T10:31:19.887 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-03-27T10:31:19.887 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-karma.ko 2024-03-27T10:31:19.887 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-onetouch.ko 2024-03-27T10:31:19.887 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-realtek.ko 2024-03-27T10:31:19.887 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-sddr09.ko 2024-03-27T10:31:19.888 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-sddr55.ko 2024-03-27T10:31:19.888 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-usbat.ko 2024-03-27T10:31:19.888 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/usb-storage.ko 2024-03-27T10:31:19.888 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip 2024-03-27T10:31:19.888 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/usbip-core.ko 2024-03-27T10:31:19.888 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/usbip-host.ko 2024-03-27T10:31:19.888 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-03-27T10:31:19.888 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio 2024-03-27T10:31:19.888 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci 2024-03-27T10:31:19.888 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-03-27T10:31:19.888 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci/vfio-pci.ko 2024-03-27T10:31:19.888 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/vfio.ko 2024-03-27T10:31:19.888 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-03-27T10:31:19.889 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost 2024-03-27T10:31:19.889 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost.ko 2024-03-27T10:31:19.889 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_iotlb.ko 2024-03-27T10:31:19.889 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_net.ko 2024-03-27T10:31:19.889 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_scsi.ko 2024-03-27T10:31:19.889 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video 2024-03-27T10:31:19.889 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight 2024-03-27T10:31:19.889 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/apple_bl.ko 2024-03-27T10:31:19.889 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/lcd.ko 2024-03-27T10:31:19.889 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/platform_lcd.ko 2024-03-27T10:31:19.889 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio 2024-03-27T10:31:19.889 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio.ko 2024-03-27T10:31:19.889 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_balloon.ko 2024-03-27T10:31:19.890 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_mmio.ko 2024-03-27T10:31:19.890 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_pci.ko 2024-03-27T10:31:19.890 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-03-27T10:31:19.890 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-03-27T10:31:19.890 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_ring.ko 2024-03-27T10:31:19.890 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1 2024-03-27T10:31:19.890 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters 2024-03-27T10:31:19.890 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters/ds2482.ko 2024-03-27T10:31:19.890 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters/ds2490.ko 2024-03-27T10:31:19.890 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves 2024-03-27T10:31:19.890 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-03-27T10:31:19.890 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-03-27T10:31:19.890 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-03-27T10:31:19.891 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-03-27T10:31:19.891 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-03-27T10:31:19.891 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-03-27T10:31:19.891 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-03-27T10:31:19.891 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-03-27T10:31:19.891 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-03-27T10:31:19.891 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_smem.ko 2024-03-27T10:31:19.891 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_therm.ko 2024-03-27T10:31:19.891 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/wire.ko 2024-03-27T10:31:19.891 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog 2024-03-27T10:31:19.891 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/alim1535_wdt.ko 2024-03-27T10:31:19.891 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/alim7101_wdt.ko 2024-03-27T10:31:19.891 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/f71808e_wdt.ko 2024-03-27T10:31:19.891 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/hpwdt.ko 2024-03-27T10:31:19.892 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/i6300esb.ko 2024-03-27T10:31:19.892 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-03-27T10:31:19.892 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/iTCO_wdt.ko 2024-03-27T10:31:19.892 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ib700wdt.ko 2024-03-27T10:31:19.892 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ibmasr.ko 2024-03-27T10:31:19.892 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-03-27T10:31:19.892 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/it8712f_wdt.ko 2024-03-27T10:31:19.892 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/it87_wdt.ko 2024-03-27T10:31:19.892 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/machzwd.ko 2024-03-27T10:31:19.892 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/nv_tco.ko 2024-03-27T10:31:19.892 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/pcwd_pci.ko 2024-03-27T10:31:19.892 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/pcwd_usb.ko 2024-03-27T10:31:19.892 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-03-27T10:31:19.892 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sch311x_wdt.ko 2024-03-27T10:31:19.893 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/softdog.ko 2024-03-27T10:31:19.893 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sp5100_tco.ko 2024-03-27T10:31:19.893 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/via_wdt.ko 2024-03-27T10:31:19.893 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-03-27T10:31:19.893 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83877f_wdt.ko 2024-03-27T10:31:19.893 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83977f_wdt.ko 2024-03-27T10:31:19.893 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/wdt_pci.ko 2024-03-27T10:31:19.893 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/xen_wdt.ko 2024-03-27T10:31:19.893 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen 2024-03-27T10:31:19.893 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-acpi-processor.ko 2024-03-27T10:31:19.893 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-evtchn.ko 2024-03-27T10:31:19.893 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-gntalloc.ko 2024-03-27T10:31:19.893 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-gntdev.ko 2024-03-27T10:31:19.893 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-pciback 2024-03-27T10:31:19.894 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-03-27T10:31:19.894 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-privcmd.ko 2024-03-27T10:31:19.894 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xenfs 2024-03-27T10:31:19.894 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xenfs/xenfs.ko 2024-03-27T10:31:19.894 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs 2024-03-27T10:31:19.894 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/9p 2024-03-27T10:31:19.894 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/9p/9p.ko 2024-03-27T10:31:19.894 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/affs 2024-03-27T10:31:19.894 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/affs/affs.ko 2024-03-27T10:31:19.894 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/befs 2024-03-27T10:31:19.894 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/befs/befs.ko 2024-03-27T10:31:19.894 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/binfmt_misc.ko 2024-03-27T10:31:19.894 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/btrfs 2024-03-27T10:31:19.895 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/btrfs/btrfs.ko 2024-03-27T10:31:19.895 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cachefiles 2024-03-27T10:31:19.895 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cachefiles/cachefiles.ko 2024-03-27T10:31:19.895 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ceph 2024-03-27T10:31:19.895 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ceph/ceph.ko 2024-03-27T10:31:19.895 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/coda 2024-03-27T10:31:19.895 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/coda/coda.ko 2024-03-27T10:31:19.895 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cramfs 2024-03-27T10:31:19.895 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cramfs/cramfs.ko 2024-03-27T10:31:19.895 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/dlm 2024-03-27T10:31:19.895 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/dlm/dlm.ko 2024-03-27T10:31:19.895 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ecryptfs 2024-03-27T10:31:19.895 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ecryptfs/ecryptfs.ko 2024-03-27T10:31:19.896 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/f2fs 2024-03-27T10:31:19.896 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/f2fs/f2fs.ko 2024-03-27T10:31:19.896 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat 2024-03-27T10:31:19.896 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/fat.ko 2024-03-27T10:31:19.896 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/msdos.ko 2024-03-27T10:31:19.896 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/vfat.ko 2024-03-27T10:31:19.896 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse 2024-03-27T10:31:19.896 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse/cuse.ko 2024-03-27T10:31:19.896 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse/fuse.ko 2024-03-27T10:31:19.896 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/gfs2 2024-03-27T10:31:19.896 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/gfs2/gfs2.ko 2024-03-27T10:31:19.896 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfs 2024-03-27T10:31:19.896 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfs/hfs.ko 2024-03-27T10:31:19.897 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfsplus 2024-03-27T10:31:19.897 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfsplus/hfsplus.ko 2024-03-27T10:31:19.897 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/isofs 2024-03-27T10:31:19.897 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/isofs/isofs.ko 2024-03-27T10:31:19.897 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/jfs 2024-03-27T10:31:19.897 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/jfs/jfs.ko 2024-03-27T10:31:19.897 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/lockd 2024-03-27T10:31:19.897 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/lockd/lockd.ko 2024-03-27T10:31:19.897 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/minix 2024-03-27T10:31:19.897 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/minix/minix.ko 2024-03-27T10:31:19.897 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/netfs 2024-03-27T10:31:19.897 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/netfs/netfs.ko 2024-03-27T10:31:19.897 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs 2024-03-27T10:31:19.897 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/blocklayout 2024-03-27T10:31:19.898 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-03-27T10:31:19.898 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/filelayout 2024-03-27T10:31:19.898 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-03-27T10:31:19.898 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/flexfilelayout 2024-03-27T10:31:19.898 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-03-27T10:31:19.898 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfs.ko 2024-03-27T10:31:19.898 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfsv3.ko 2024-03-27T10:31:19.898 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfsv4.ko 2024-03-27T10:31:19.898 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common 2024-03-27T10:31:19.898 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common/grace.ko 2024-03-27T10:31:19.898 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common/nfs_acl.ko 2024-03-27T10:31:19.898 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfsd 2024-03-27T10:31:19.898 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfsd/nfsd.ko 2024-03-27T10:31:19.899 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nilfs2 2024-03-27T10:31:19.899 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nilfs2/nilfs2.ko 2024-03-27T10:31:19.899 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls 2024-03-27T10:31:19.899 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-celtic.ko 2024-03-27T10:31:19.899 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-centeuro.ko 2024-03-27T10:31:19.899 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-croatian.ko 2024-03-27T10:31:19.899 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-cyrillic.ko 2024-03-27T10:31:19.899 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-gaelic.ko 2024-03-27T10:31:19.899 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-greek.ko 2024-03-27T10:31:19.899 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-iceland.ko 2024-03-27T10:31:19.899 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-inuit.ko 2024-03-27T10:31:19.899 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-roman.ko 2024-03-27T10:31:19.899 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-romanian.ko 2024-03-27T10:31:19.899 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-turkish.ko 2024-03-27T10:31:19.900 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1250.ko 2024-03-27T10:31:19.900 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1251.ko 2024-03-27T10:31:19.900 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1255.ko 2024-03-27T10:31:19.900 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp737.ko 2024-03-27T10:31:19.900 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp775.ko 2024-03-27T10:31:19.900 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp850.ko 2024-03-27T10:31:19.900 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp852.ko 2024-03-27T10:31:19.900 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp855.ko 2024-03-27T10:31:19.900 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp857.ko 2024-03-27T10:31:19.900 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp860.ko 2024-03-27T10:31:19.900 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp861.ko 2024-03-27T10:31:19.900 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp862.ko 2024-03-27T10:31:19.900 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp863.ko 2024-03-27T10:31:19.901 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp864.ko 2024-03-27T10:31:19.901 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp865.ko 2024-03-27T10:31:19.901 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp866.ko 2024-03-27T10:31:19.901 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp869.ko 2024-03-27T10:31:19.901 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp874.ko 2024-03-27T10:31:19.901 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp932.ko 2024-03-27T10:31:19.901 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp936.ko 2024-03-27T10:31:19.901 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp949.ko 2024-03-27T10:31:19.901 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp950.ko 2024-03-27T10:31:19.901 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_euc-jp.ko 2024-03-27T10:31:19.901 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-1.ko 2024-03-27T10:31:19.901 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-13.ko 2024-03-27T10:31:19.901 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-14.ko 2024-03-27T10:31:19.901 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-15.ko 2024-03-27T10:31:19.902 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-2.ko 2024-03-27T10:31:19.902 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-3.ko 2024-03-27T10:31:19.902 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-4.ko 2024-03-27T10:31:19.902 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-5.ko 2024-03-27T10:31:19.902 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-6.ko 2024-03-27T10:31:19.902 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-7.ko 2024-03-27T10:31:19.902 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-9.ko 2024-03-27T10:31:19.902 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-r.ko 2024-03-27T10:31:19.902 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-ru.ko 2024-03-27T10:31:19.902 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-u.ko 2024-03-27T10:31:19.902 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_ucs2_utils.ko 2024-03-27T10:31:19.902 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_utf8.ko 2024-03-27T10:31:19.902 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2 2024-03-27T10:31:19.903 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/cluster 2024-03-27T10:31:19.903 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-03-27T10:31:19.903 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlm 2024-03-27T10:31:19.903 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-03-27T10:31:19.903 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlmfs 2024-03-27T10:31:19.903 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-03-27T10:31:19.903 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2.ko 2024-03-27T10:31:19.903 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-03-27T10:31:19.903 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-03-27T10:31:19.903 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-03-27T10:31:19.903 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/overlayfs 2024-03-27T10:31:19.903 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/overlayfs/overlay.ko 2024-03-27T10:31:19.903 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/pstore 2024-03-27T10:31:19.903 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/pstore/ramoops.ko 2024-03-27T10:31:19.904 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/reiserfs 2024-03-27T10:31:19.904 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/reiserfs/reiserfs.ko 2024-03-27T10:31:19.904 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/romfs 2024-03-27T10:31:19.904 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/romfs/romfs.ko 2024-03-27T10:31:19.904 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb 2024-03-27T10:31:19.904 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/client 2024-03-27T10:31:19.904 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/client/cifs.ko 2024-03-27T10:31:19.904 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common 2024-03-27T10:31:19.904 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common/cifs_arc4.ko 2024-03-27T10:31:19.904 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common/cifs_md4.ko 2024-03-27T10:31:19.904 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/squashfs 2024-03-27T10:31:19.904 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/squashfs/squashfs.ko 2024-03-27T10:31:19.904 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/sysv 2024-03-27T10:31:19.905 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/sysv/sysv.ko 2024-03-27T10:31:19.905 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/udf 2024-03-27T10:31:19.905 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/udf/udf.ko 2024-03-27T10:31:19.905 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ufs 2024-03-27T10:31:19.905 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ufs/ufs.ko 2024-03-27T10:31:19.905 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/xfs 2024-03-27T10:31:19.905 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/xfs/xfs.ko 2024-03-27T10:31:19.905 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel 2024-03-27T10:31:19.905 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/rcu 2024-03-27T10:31:19.905 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/rcu/rcutorture.ko 2024-03-27T10:31:19.905 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/torture.ko 2024-03-27T10:31:19.905 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/trace 2024-03-27T10:31:19.905 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-03-27T10:31:19.906 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib 2024-03-27T10:31:19.906 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/asn1_encoder.ko 2024-03-27T10:31:19.906 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crc-itu-t.ko 2024-03-27T10:31:19.906 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crc8.ko 2024-03-27T10:31:19.906 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto 2024-03-27T10:31:19.906 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto/libarc4.ko 2024-03-27T10:31:19.906 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto/libdes.ko 2024-03-27T10:31:19.906 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/libcrc32c.ko 2024-03-27T10:31:19.906 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lru_cache.ko 2024-03-27T10:31:19.906 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4 2024-03-27T10:31:19.906 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4/lz4_compress.ko 2024-03-27T10:31:19.906 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4/lz4hc_compress.ko 2024-03-27T10:31:19.906 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/math 2024-03-27T10:31:19.906 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/math/cordic.ko 2024-03-27T10:31:19.907 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/raid6 2024-03-27T10:31:19.907 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/raid6/raid6_pq.ko 2024-03-27T10:31:19.907 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/reed_solomon 2024-03-27T10:31:19.907 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/reed_solomon/reed_solomon.ko 2024-03-27T10:31:19.907 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_bm.ko 2024-03-27T10:31:19.907 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_fsm.ko 2024-03-27T10:31:19.907 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_kmp.ko 2024-03-27T10:31:19.907 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/zstd 2024-03-27T10:31:19.907 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/zstd/zstd_compress.ko 2024-03-27T10:31:19.907 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/mm 2024-03-27T10:31:19.907 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/mm/hwpoison-inject.ko 2024-03-27T10:31:19.907 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net 2024-03-27T10:31:19.907 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan 2024-03-27T10:31:19.908 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/6lowpan.ko 2024-03-27T10:31:19.908 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_dest.ko 2024-03-27T10:31:19.908 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_fragment.ko 2024-03-27T10:31:19.908 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_hop.ko 2024-03-27T10:31:19.908 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_ipv6.ko 2024-03-27T10:31:19.908 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_mobility.ko 2024-03-27T10:31:19.908 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_routing.ko 2024-03-27T10:31:19.908 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_udp.ko 2024-03-27T10:31:19.908 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802 2024-03-27T10:31:19.908 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/garp.ko 2024-03-27T10:31:19.908 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/mrp.ko 2024-03-27T10:31:19.908 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/p8022.ko 2024-03-27T10:31:19.908 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/psnap.ko 2024-03-27T10:31:19.908 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/stp.ko 2024-03-27T10:31:19.909 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/8021q 2024-03-27T10:31:19.909 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/8021q/8021q.ko 2024-03-27T10:31:19.909 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p 2024-03-27T10:31:19.909 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet.ko 2024-03-27T10:31:19.909 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_fd.ko 2024-03-27T10:31:19.909 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_rdma.ko 2024-03-27T10:31:19.909 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_virtio.ko 2024-03-27T10:31:19.909 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/batman-adv 2024-03-27T10:31:19.909 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/batman-adv/batman-adv.ko 2024-03-27T10:31:19.909 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge 2024-03-27T10:31:19.909 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/br_netfilter.ko 2024-03-27T10:31:19.909 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/bridge.ko 2024-03-27T10:31:19.909 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter 2024-03-27T10:31:19.909 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-03-27T10:31:19.910 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_among.ko 2024-03-27T10:31:19.910 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_arp.ko 2024-03-27T10:31:19.910 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-03-27T10:31:19.910 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-03-27T10:31:19.910 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_ip.ko 2024-03-27T10:31:19.910 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-03-27T10:31:19.910 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_limit.ko 2024-03-27T10:31:19.910 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_log.ko 2024-03-27T10:31:19.910 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_mark.ko 2024-03-27T10:31:19.910 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-03-27T10:31:19.910 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-03-27T10:31:19.910 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-03-27T10:31:19.910 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-03-27T10:31:19.911 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_snat.ko 2024-03-27T10:31:19.911 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_stp.ko 2024-03-27T10:31:19.911 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-03-27T10:31:19.911 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-03-27T10:31:19.911 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-03-27T10:31:19.911 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-03-27T10:31:19.911 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtables.ko 2024-03-27T10:31:19.911 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-03-27T10:31:19.911 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-03-27T10:31:19.911 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-03-27T10:31:19.911 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ceph 2024-03-27T10:31:19.911 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ceph/libceph.ko 2024-03-27T10:31:19.911 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core 2024-03-27T10:31:19.912 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core/failover.ko 2024-03-27T10:31:19.912 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core/pktgen.ko 2024-03-27T10:31:19.912 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp 2024-03-27T10:31:19.912 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp.ko 2024-03-27T10:31:19.912 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_diag.ko 2024-03-27T10:31:19.912 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_ipv4.ko 2024-03-27T10:31:19.912 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_ipv6.ko 2024-03-27T10:31:19.912 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dns_resolver 2024-03-27T10:31:19.912 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dns_resolver/dns_resolver.ko 2024-03-27T10:31:19.912 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa 2024-03-27T10:31:19.912 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/dsa_core.ko 2024-03-27T10:31:19.912 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_brcm.ko 2024-03-27T10:31:19.912 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_dsa.ko 2024-03-27T10:31:19.912 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_none.ko 2024-03-27T10:31:19.913 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_trailer.ko 2024-03-27T10:31:19.913 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154 2024-03-27T10:31:19.913 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/6lowpan 2024-03-27T10:31:19.913 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-03-27T10:31:19.913 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/ieee802154.ko 2024-03-27T10:31:19.913 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/ieee802154_socket.ko 2024-03-27T10:31:19.913 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4 2024-03-27T10:31:19.913 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ah4.ko 2024-03-27T10:31:19.913 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/esp4.ko 2024-03-27T10:31:19.913 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/fou.ko 2024-03-27T10:31:19.913 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/gre.ko 2024-03-27T10:31:19.913 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/inet_diag.ko 2024-03-27T10:31:19.913 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_gre.ko 2024-03-27T10:31:19.914 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_tunnel.ko 2024-03-27T10:31:19.914 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_vti.ko 2024-03-27T10:31:19.914 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ipcomp.ko 2024-03-27T10:31:19.914 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ipip.ko 2024-03-27T10:31:19.914 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter 2024-03-27T10:31:19.914 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arp_tables.ko 2024-03-27T10:31:19.914 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-03-27T10:31:19.914 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-03-27T10:31:19.914 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-03-27T10:31:19.914 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-03-27T10:31:19.914 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-03-27T10:31:19.914 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-03-27T10:31:19.914 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-03-27T10:31:19.914 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-03-27T10:31:19.915 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-03-27T10:31:19.915 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_security.ko 2024-03-27T10:31:19.915 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-03-27T10:31:19.915 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-03-27T10:31:19.915 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-03-27T10:31:19.915 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-03-27T10:31:19.915 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-03-27T10:31:19.915 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-03-27T10:31:19.915 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-03-27T10:31:19.915 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-03-27T10:31:19.915 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-03-27T10:31:19.915 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-03-27T10:31:19.916 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_bic.ko 2024-03-27T10:31:19.916 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_dctcp.ko 2024-03-27T10:31:19.916 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_diag.ko 2024-03-27T10:31:19.916 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_highspeed.ko 2024-03-27T10:31:19.916 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_htcp.ko 2024-03-27T10:31:19.916 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_hybla.ko 2024-03-27T10:31:19.916 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_illinois.ko 2024-03-27T10:31:19.916 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_lp.ko 2024-03-27T10:31:19.916 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_scalable.ko 2024-03-27T10:31:19.917 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_vegas.ko 2024-03-27T10:31:19.917 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_veno.ko 2024-03-27T10:31:19.917 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_westwood.ko 2024-03-27T10:31:19.917 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_yeah.ko 2024-03-27T10:31:19.917 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tunnel4.ko 2024-03-27T10:31:19.917 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/udp_diag.ko 2024-03-27T10:31:19.917 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/udp_tunnel.ko 2024-03-27T10:31:19.917 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/xfrm4_tunnel.ko 2024-03-27T10:31:19.917 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6 2024-03-27T10:31:19.917 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ah6.ko 2024-03-27T10:31:19.917 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/esp6.ko 2024-03-27T10:31:19.917 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/fou6.ko 2024-03-27T10:31:19.917 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_tunnel.ko 2024-03-27T10:31:19.918 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-03-27T10:31:19.918 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_vti.ko 2024-03-27T10:31:19.918 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ipcomp6.ko 2024-03-27T10:31:19.918 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter 2024-03-27T10:31:19.918 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-03-27T10:31:19.918 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-03-27T10:31:19.918 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-03-27T10:31:19.918 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-03-27T10:31:19.918 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-03-27T10:31:19.918 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-03-27T10:31:19.918 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-03-27T10:31:19.918 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-03-27T10:31:19.918 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-03-27T10:31:19.919 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-03-27T10:31:19.919 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-03-27T10:31:19.919 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-03-27T10:31:19.919 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-03-27T10:31:19.919 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-03-27T10:31:19.919 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-03-27T10:31:19.919 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-03-27T10:31:19.919 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-03-27T10:31:19.919 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-03-27T10:31:19.919 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-03-27T10:31:19.919 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-03-27T10:31:19.919 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-03-27T10:31:19.919 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-03-27T10:31:19.919 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-03-27T10:31:19.920 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-03-27T10:31:19.920 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/sit.ko 2024-03-27T10:31:19.920 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/tunnel6.ko 2024-03-27T10:31:19.920 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/xfrm6_tunnel.ko 2024-03-27T10:31:19.920 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/key 2024-03-27T10:31:19.920 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/key/af_key.ko 2024-03-27T10:31:19.920 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp 2024-03-27T10:31:19.920 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_core.ko 2024-03-27T10:31:19.920 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_debugfs.ko 2024-03-27T10:31:19.920 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_eth.ko 2024-03-27T10:31:19.920 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ip.ko 2024-03-27T10:31:19.920 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ip6.ko 2024-03-27T10:31:19.920 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_netlink.ko 2024-03-27T10:31:19.921 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ppp.ko 2024-03-27T10:31:19.921 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/llc 2024-03-27T10:31:19.921 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/llc/llc.ko 2024-03-27T10:31:19.921 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mac802154 2024-03-27T10:31:19.921 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mac802154/mac802154.ko 2024-03-27T10:31:19.921 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mpls 2024-03-27T10:31:19.921 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mpls/mpls_gso.ko 2024-03-27T10:31:19.921 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter 2024-03-27T10:31:19.921 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset 2024-03-27T10:31:19.921 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set.ko 2024-03-27T10:31:19.921 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-03-27T10:31:19.921 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-03-27T10:31:19.921 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-03-27T10:31:19.921 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-03-27T10:31:19.922 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-03-27T10:31:19.922 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-03-27T10:31:19.922 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-03-27T10:31:19.922 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-03-27T10:31:19.922 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-03-27T10:31:19.922 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-03-27T10:31:19.922 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-03-27T10:31:19.922 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-03-27T10:31:19.922 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-03-27T10:31:19.922 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-03-27T10:31:19.922 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-03-27T10:31:19.922 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs 2024-03-27T10:31:19.922 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs.ko 2024-03-27T10:31:19.922 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-03-27T10:31:19.923 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-03-27T10:31:19.923 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-03-27T10:31:19.923 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-03-27T10:31:19.923 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-03-27T10:31:19.923 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-03-27T10:31:19.923 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-03-27T10:31:19.923 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-03-27T10:31:19.923 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-03-27T10:31:19.923 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-03-27T10:31:19.923 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-03-27T10:31:19.923 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-03-27T10:31:19.923 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-03-27T10:31:19.923 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conncount.ko 2024-03-27T10:31:19.924 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack.ko 2024-03-27T10:31:19.924 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-03-27T10:31:19.924 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-03-27T10:31:19.924 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-03-27T10:31:19.924 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_h323.ko 2024-03-27T10:31:19.924 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_irc.ko 2024-03-27T10:31:19.924 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-03-27T10:31:19.924 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-03-27T10:31:19.924 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-03-27T10:31:19.924 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_sane.ko 2024-03-27T10:31:19.924 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_sip.ko 2024-03-27T10:31:19.924 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-03-27T10:31:19.924 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-03-27T10:31:19.924 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_dup_netdev.ko 2024-03-27T10:31:19.925 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_flow_table.ko 2024-03-27T10:31:19.925 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_flow_table_inet.ko 2024-03-27T10:31:19.925 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_log_syslog.ko 2024-03-27T10:31:19.925 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat.ko 2024-03-27T10:31:19.925 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_amanda.ko 2024-03-27T10:31:19.925 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_ftp.ko 2024-03-27T10:31:19.925 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_irc.ko 2024-03-27T10:31:19.925 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_sip.ko 2024-03-27T10:31:19.925 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_tftp.ko 2024-03-27T10:31:19.925 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_synproxy_core.ko 2024-03-27T10:31:19.925 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_tables.ko 2024-03-27T10:31:19.925 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink.ko 2024-03-27T10:31:19.925 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_acct.ko 2024-03-27T10:31:19.925 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_log.ko 2024-03-27T10:31:19.926 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_osf.ko 2024-03-27T10:31:19.926 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_queue.ko 2024-03-27T10:31:19.926 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_chain_nat.ko 2024-03-27T10:31:19.926 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_compat.ko 2024-03-27T10:31:19.926 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_ct.ko 2024-03-27T10:31:19.926 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_dup_netdev.ko 2024-03-27T10:31:19.926 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_fib.ko 2024-03-27T10:31:19.926 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_fwd_netdev.ko 2024-03-27T10:31:19.926 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_hash.ko 2024-03-27T10:31:19.926 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_limit.ko 2024-03-27T10:31:19.926 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_log.ko 2024-03-27T10:31:19.926 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_masq.ko 2024-03-27T10:31:19.927 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_nat.ko 2024-03-27T10:31:19.927 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_numgen.ko 2024-03-27T10:31:19.927 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_queue.ko 2024-03-27T10:31:19.927 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_quota.ko 2024-03-27T10:31:19.927 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_redir.ko 2024-03-27T10:31:19.927 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_reject.ko 2024-03-27T10:31:19.927 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_reject_inet.ko 2024-03-27T10:31:19.927 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_socket.ko 2024-03-27T10:31:19.927 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_tunnel.ko 2024-03-27T10:31:19.927 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_AUDIT.ko 2024-03-27T10:31:19.927 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CHECKSUM.ko 2024-03-27T10:31:19.927 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CLASSIFY.ko 2024-03-27T10:31:19.927 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-03-27T10:31:19.927 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CT.ko 2024-03-27T10:31:19.928 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_DSCP.ko 2024-03-27T10:31:19.928 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_HL.ko 2024-03-27T10:31:19.928 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_HMARK.ko 2024-03-27T10:31:19.928 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_IDLETIMER.ko 2024-03-27T10:31:19.928 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_LED.ko 2024-03-27T10:31:19.928 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_LOG.ko 2024-03-27T10:31:19.928 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_MASQUERADE.ko 2024-03-27T10:31:19.928 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NETMAP.ko 2024-03-27T10:31:19.928 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NFLOG.ko 2024-03-27T10:31:19.928 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NFQUEUE.ko 2024-03-27T10:31:19.928 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_RATEEST.ko 2024-03-27T10:31:19.928 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_REDIRECT.ko 2024-03-27T10:31:19.928 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_SECMARK.ko 2024-03-27T10:31:19.928 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TCPMSS.ko 2024-03-27T10:31:19.929 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-03-27T10:31:19.929 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TEE.ko 2024-03-27T10:31:19.929 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TPROXY.ko 2024-03-27T10:31:19.929 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TRACE.ko 2024-03-27T10:31:19.929 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_addrtype.ko 2024-03-27T10:31:19.929 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_bpf.ko 2024-03-27T10:31:19.929 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cgroup.ko 2024-03-27T10:31:19.929 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cluster.ko 2024-03-27T10:31:19.929 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_comment.ko 2024-03-27T10:31:19.929 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connbytes.ko 2024-03-27T10:31:19.929 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connlabel.ko 2024-03-27T10:31:19.929 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connlimit.ko 2024-03-27T10:31:19.929 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connmark.ko 2024-03-27T10:31:19.929 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_conntrack.ko 2024-03-27T10:31:19.930 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cpu.ko 2024-03-27T10:31:19.930 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_dccp.ko 2024-03-27T10:31:19.930 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_devgroup.ko 2024-03-27T10:31:19.930 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_dscp.ko 2024-03-27T10:31:19.930 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ecn.ko 2024-03-27T10:31:19.930 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_esp.ko 2024-03-27T10:31:19.930 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_hashlimit.ko 2024-03-27T10:31:19.930 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_helper.ko 2024-03-27T10:31:19.930 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_hl.ko 2024-03-27T10:31:19.930 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ipcomp.ko 2024-03-27T10:31:19.930 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_iprange.ko 2024-03-27T10:31:19.930 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ipvs.ko 2024-03-27T10:31:19.930 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_l2tp.ko 2024-03-27T10:31:19.931 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_length.ko 2024-03-27T10:31:19.931 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_limit.ko 2024-03-27T10:31:19.931 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_mac.ko 2024-03-27T10:31:19.931 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_mark.ko 2024-03-27T10:31:19.931 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_multiport.ko 2024-03-27T10:31:19.931 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_nat.ko 2024-03-27T10:31:19.931 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_nfacct.ko 2024-03-27T10:31:19.931 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_osf.ko 2024-03-27T10:31:19.931 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_owner.ko 2024-03-27T10:31:19.931 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_physdev.ko 2024-03-27T10:31:19.931 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_pkttype.ko 2024-03-27T10:31:19.931 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_policy.ko 2024-03-27T10:31:19.931 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_quota.ko 2024-03-27T10:31:19.932 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_rateest.ko 2024-03-27T10:31:19.932 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_realm.ko 2024-03-27T10:31:19.932 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_recent.ko 2024-03-27T10:31:19.932 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_sctp.ko 2024-03-27T10:31:19.932 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_set.ko 2024-03-27T10:31:19.932 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_socket.ko 2024-03-27T10:31:19.932 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_state.ko 2024-03-27T10:31:19.932 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_statistic.ko 2024-03-27T10:31:19.932 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_string.ko 2024-03-27T10:31:19.932 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_tcpmss.ko 2024-03-27T10:31:19.932 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_time.ko 2024-03-27T10:31:19.932 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_u32.ko 2024-03-27T10:31:19.932 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netlink 2024-03-27T10:31:19.933 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netlink/netlink_diag.ko 2024-03-27T10:31:19.933 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/nsh 2024-03-27T10:31:19.933 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/nsh/nsh.ko 2024-03-27T10:31:19.933 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch 2024-03-27T10:31:19.933 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/openvswitch.ko 2024-03-27T10:31:19.933 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-geneve.ko 2024-03-27T10:31:19.933 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-gre.ko 2024-03-27T10:31:19.933 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-vxlan.ko 2024-03-27T10:31:19.933 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/packet 2024-03-27T10:31:19.933 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/packet/af_packet_diag.ko 2024-03-27T10:31:19.933 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds 2024-03-27T10:31:19.933 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds.ko 2024-03-27T10:31:19.933 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds_rdma.ko 2024-03-27T10:31:19.933 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds_tcp.ko 2024-03-27T10:31:19.934 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched 2024-03-27T10:31:19.934 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_csum.ko 2024-03-27T10:31:19.934 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_gact.ko 2024-03-27T10:31:19.934 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_mirred.ko 2024-03-27T10:31:19.934 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_nat.ko 2024-03-27T10:31:19.934 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_pedit.ko 2024-03-27T10:31:19.934 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_police.ko 2024-03-27T10:31:19.934 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_simple.ko 2024-03-27T10:31:19.934 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_skbedit.ko 2024-03-27T10:31:19.934 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_vlan.ko 2024-03-27T10:31:19.934 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_basic.ko 2024-03-27T10:31:19.935 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_bpf.ko 2024-03-27T10:31:19.935 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_flow.ko 2024-03-27T10:31:19.935 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_fw.ko 2024-03-27T10:31:19.935 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_route.ko 2024-03-27T10:31:19.935 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_u32.ko 2024-03-27T10:31:19.935 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_cmp.ko 2024-03-27T10:31:19.935 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_ipset.ko 2024-03-27T10:31:19.935 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_meta.ko 2024-03-27T10:31:19.935 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_nbyte.ko 2024-03-27T10:31:19.935 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_text.ko 2024-03-27T10:31:19.935 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_u32.ko 2024-03-27T10:31:19.935 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_choke.ko 2024-03-27T10:31:19.935 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_codel.ko 2024-03-27T10:31:19.936 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_drr.ko 2024-03-27T10:31:19.936 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_fq.ko 2024-03-27T10:31:19.936 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_gred.ko 2024-03-27T10:31:19.936 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_hfsc.ko 2024-03-27T10:31:19.936 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_hhf.ko 2024-03-27T10:31:19.936 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_htb.ko 2024-03-27T10:31:19.936 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_ingress.ko 2024-03-27T10:31:19.936 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_mqprio.ko 2024-03-27T10:31:19.936 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_mqprio_lib.ko 2024-03-27T10:31:19.936 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_multiq.ko 2024-03-27T10:31:19.936 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_netem.ko 2024-03-27T10:31:19.936 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_pie.ko 2024-03-27T10:31:19.936 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_plug.ko 2024-03-27T10:31:19.937 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_prio.ko 2024-03-27T10:31:19.937 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_qfq.ko 2024-03-27T10:31:19.937 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_red.ko 2024-03-27T10:31:19.937 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_sfb.ko 2024-03-27T10:31:19.937 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_sfq.ko 2024-03-27T10:31:19.937 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_tbf.ko 2024-03-27T10:31:19.937 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_teql.ko 2024-03-27T10:31:19.937 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp 2024-03-27T10:31:19.937 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp/sctp.ko 2024-03-27T10:31:19.937 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp/sctp_diag.ko 2024-03-27T10:31:19.937 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc 2024-03-27T10:31:19.937 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss 2024-03-27T10:31:19.937 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-03-27T10:31:19.937 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-03-27T10:31:19.938 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/sunrpc.ko 2024-03-27T10:31:19.938 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/xprtrdma 2024-03-27T10:31:19.938 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-03-27T10:31:19.938 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc 2024-03-27T10:31:19.938 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc/diag.ko 2024-03-27T10:31:19.938 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc/tipc.ko 2024-03-27T10:31:19.938 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/unix 2024-03-27T10:31:19.938 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/unix/unix_diag.ko 2024-03-27T10:31:19.938 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock 2024-03-27T10:31:19.938 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-03-27T10:31:19.938 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-03-27T10:31:19.938 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock.ko 2024-03-27T10:31:19.938 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock_diag.ko 2024-03-27T10:31:19.939 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock_loopback.ko 2024-03-27T10:31:19.939 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/xfrm 2024-03-27T10:31:19.939 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/xfrm/xfrm_ipcomp.ko 2024-03-27T10:31:19.939 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security 2024-03-27T10:31:19.939 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys 2024-03-27T10:31:19.939 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/encrypted-keys 2024-03-27T10:31:19.939 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-03-27T10:31:19.939 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/trusted-keys 2024-03-27T10:31:19.939 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/trusted-keys/trusted.ko 2024-03-27T10:31:19.939 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/virt 2024-03-27T10:31:19.939 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/virt/lib 2024-03-27T10:31:19.939 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/virt/lib/irqbypass.ko 2024-03-27T10:31:19.939 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.alias 2024-03-27T10:31:19.939 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.alias.bin 2024-03-27T10:31:19.940 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin 2024-03-27T10:31:19.940 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.alias.bin 2024-03-27T10:31:19.940 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.bin 2024-03-27T10:31:19.940 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.modinfo 2024-03-27T10:31:19.940 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.dep 2024-03-27T10:31:19.940 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.dep.bin 2024-03-27T10:31:19.940 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.devname 2024-03-27T10:31:19.940 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.order 2024-03-27T10:31:19.940 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.softdep 2024-03-27T10:31:19.940 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.symbols 2024-03-27T10:31:19.940 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.symbols.bin 2024-03-27T10:31:19.940 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.8.0-g1df0c7e8a235/vmlinuz 2024-03-27T10:31:19.944 DEBUG:teuthology.task.kernel:get_image_version: 6.8.0-g1df0c7e8a235 2024-03-27T10:31:19.944 DEBUG:teuthology.orchestra.run.smithi177:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-03-27T10:31:19.966 INFO:teuthology.orchestra.run.smithi177.stdout:postinstall scriptlet (using /bin/sh): 2024-03-27T10:31:19.966 INFO:teuthology.orchestra.run.smithi177.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-03-27T10:31:19.966 INFO:teuthology.orchestra.run.smithi177.stdout: /usr/bin/kernel-install add 6.8.0-g1df0c7e8a235 /lib/modules/6.8.0-g1df0c7e8a235/vmlinuz 2024-03-27T10:31:19.966 INFO:teuthology.orchestra.run.smithi177.stdout:fi 2024-03-27T10:31:19.966 INFO:teuthology.orchestra.run.smithi177.stdout:for file in vmlinuz System.map config; do 2024-03-27T10:31:19.966 INFO:teuthology.orchestra.run.smithi177.stdout: if ! cmp --silent "/lib/modules/6.8.0-g1df0c7e8a235/${file}" "/boot/${file}-6.8.0-g1df0c7e8a235"; then 2024-03-27T10:31:19.967 INFO:teuthology.orchestra.run.smithi177.stdout: cp "/lib/modules/6.8.0-g1df0c7e8a235/${file}" "/boot/${file}-6.8.0-g1df0c7e8a235" 2024-03-27T10:31:19.967 INFO:teuthology.orchestra.run.smithi177.stdout: fi 2024-03-27T10:31:19.967 INFO:teuthology.orchestra.run.smithi177.stdout:done 2024-03-27T10:31:19.967 INFO:teuthology.orchestra.run.smithi177.stdout:preuninstall scriptlet (using /bin/sh): 2024-03-27T10:31:19.967 INFO:teuthology.orchestra.run.smithi177.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2024-03-27T10:31:19.967 INFO:teuthology.orchestra.run.smithi177.stdout:new-kernel-pkg --remove 6.8.0-g1df0c7e8a235 --rminitrd --initrdfile=/boot/initramfs-6.8.0-g1df0c7e8a235.img 2024-03-27T10:31:19.967 INFO:teuthology.orchestra.run.smithi177.stdout:elif [ -x /usr/bin/kernel-install ]; then 2024-03-27T10:31:19.967 INFO:teuthology.orchestra.run.smithi177.stdout:kernel-install remove 6.8.0-g1df0c7e8a235 2024-03-27T10:31:19.967 INFO:teuthology.orchestra.run.smithi177.stdout:fi 2024-03-27T10:31:19.967 INFO:teuthology.orchestra.run.smithi177.stdout:postuninstall scriptlet (using /bin/sh): 2024-03-27T10:31:19.967 INFO:teuthology.orchestra.run.smithi177.stdout:if [ -x /sbin/update-bootloader ]; then 2024-03-27T10:31:19.967 INFO:teuthology.orchestra.run.smithi177.stdout:/sbin/update-bootloader --remove 6.8.0-g1df0c7e8a235 2024-03-27T10:31:19.967 INFO:teuthology.orchestra.run.smithi177.stdout:fi 2024-03-27T10:31:19.968 DEBUG:teuthology.orchestra.run.smithi177:> sudo rpm -qi grub2-tools 2024-03-27T10:31:20.043 INFO:teuthology.orchestra.run.smithi177.stdout:Name : grub2-tools 2024-03-27T10:31:20.044 INFO:teuthology.orchestra.run.smithi177.stdout:Epoch : 1 2024-03-27T10:31:20.044 INFO:teuthology.orchestra.run.smithi177.stdout:Version : 2.06 2024-03-27T10:31:20.044 INFO:teuthology.orchestra.run.smithi177.stdout:Release : 68.el9 2024-03-27T10:31:20.044 INFO:teuthology.orchestra.run.smithi177.stdout:Architecture: x86_64 2024-03-27T10:31:20.044 INFO:teuthology.orchestra.run.smithi177.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-27T10:31:20.044 INFO:teuthology.orchestra.run.smithi177.stdout:Group : Unspecified 2024-03-27T10:31:20.044 INFO:teuthology.orchestra.run.smithi177.stdout:Size : 8264254 2024-03-27T10:31:20.044 INFO:teuthology.orchestra.run.smithi177.stdout:License : GPLv3+ 2024-03-27T10:31:20.044 INFO:teuthology.orchestra.run.smithi177.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-27T10:31:20.044 INFO:teuthology.orchestra.run.smithi177.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-27T10:31:20.044 INFO:teuthology.orchestra.run.smithi177.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-27T10:31:20.044 INFO:teuthology.orchestra.run.smithi177.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-27T10:31:20.044 INFO:teuthology.orchestra.run.smithi177.stdout:Packager : builder@centos.org 2024-03-27T10:31:20.045 INFO:teuthology.orchestra.run.smithi177.stdout:Vendor : CentOS 2024-03-27T10:31:20.045 INFO:teuthology.orchestra.run.smithi177.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-27T10:31:20.045 INFO:teuthology.orchestra.run.smithi177.stdout:Summary : Support tools for GRUB. 2024-03-27T10:31:20.045 INFO:teuthology.orchestra.run.smithi177.stdout:Description : 2024-03-27T10:31:20.045 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T10:31:20.045 INFO:teuthology.orchestra.run.smithi177.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-27T10:31:20.045 INFO:teuthology.orchestra.run.smithi177.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-27T10:31:20.045 INFO:teuthology.orchestra.run.smithi177.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-27T10:31:20.045 INFO:teuthology.orchestra.run.smithi177.stdout:hardware devices. 2024-03-27T10:31:20.045 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T10:31:20.045 INFO:teuthology.orchestra.run.smithi177.stdout:This subpackage provides tools for support of all platforms. 2024-03-27T10:31:20.046 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-27T10:31:20.046 INFO:teuthology.task.kernel:Updating grub on smithi177 to boot 6.8.0-g1df0c7e8a235 2024-03-27T10:31:20.046 DEBUG:teuthology.orchestra.run.smithi177:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-27T10:31:20.654 INFO:teuthology.orchestra.run.smithi177.stderr:Generating grub configuration file ... 2024-03-27T10:31:22.099 INFO:teuthology.orchestra.run.smithi177.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-27T10:31:22.133 INFO:teuthology.orchestra.run.smithi177.stderr:done 2024-03-27T10:31:22.135 DEBUG:teuthology.orchestra.run.smithi177:> mktemp 2024-03-27T10:31:22.150 INFO:teuthology.orchestra.run.smithi177.stdout:/tmp/tmp.VjbgT0zBcg 2024-03-27T10:31:22.150 DEBUG:teuthology.orchestra.run.smithi177:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.VjbgT0zBcg 2024-03-27T10:31:22.223 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod 0666 /tmp/tmp.VjbgT0zBcg 2024-03-27T10:31:22.362 DEBUG:teuthology.orchestra.remote:smithi177:/tmp/tmp.VjbgT0zBcg is 6KB 2024-03-27T10:31:22.410 DEBUG:teuthology.orchestra.run.smithi177:> rm -fr /tmp/tmp.VjbgT0zBcg 2024-03-27T10:31:22.424 DEBUG:teuthology.orchestra.run.smithi177:> sudo /bin/ls /boot/loader/entries || true 2024-03-27T10:31:22.489 INFO:teuthology.orchestra.run.smithi177.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-27T10:31:22.490 INFO:teuthology.orchestra.run.smithi177.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-27T10:31:22.490 INFO:teuthology.orchestra.run.smithi177.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-27T10:31:22.490 INFO:teuthology.orchestra.run.smithi177.stdout:15949e3c88704300882482eddfd7baae-6.8.0-g1df0c7e8a235.conf 2024-03-27T10:31:22.491 DEBUG:teuthology.orchestra.run.smithi177:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.8.0-g1df0c7e8a235 2024-03-27T10:31:22.644 DEBUG:teuthology.orchestra.run.smithi177:> sudo shutdown -r now 2024-03-27T10:31:23.666 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi097.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-03-27T10:31:23.666 DEBUG:teuthology.orchestra.run.smithi097:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-03-27T10:31:23.750 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235 2024-03-27T10:31:23.750 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/System.map 2024-03-27T10:31:23.750 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/config 2024-03-27T10:31:23.750 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel 2024-03-27T10:31:23.750 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch 2024-03-27T10:31:23.750 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86 2024-03-27T10:31:23.750 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto 2024-03-27T10:31:23.750 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-03-27T10:31:23.751 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-03-27T10:31:23.751 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-03-27T10:31:23.751 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-03-27T10:31:23.751 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-03-27T10:31:23.751 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-03-27T10:31:23.751 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-03-27T10:31:23.751 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crc32c-intel.ko 2024-03-27T10:31:23.751 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-03-27T10:31:23.751 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-03-27T10:31:23.751 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-03-27T10:31:23.751 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-03-27T10:31:23.752 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-avx2.ko 2024-03-27T10:31:23.752 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-03-27T10:31:23.752 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-03-27T10:31:23.752 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-03-27T10:31:23.752 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-03-27T10:31:23.752 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-03-27T10:31:23.752 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-03-27T10:31:23.752 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-03-27T10:31:23.752 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel 2024-03-27T10:31:23.752 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu 2024-03-27T10:31:23.752 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu/mce 2024-03-27T10:31:23.752 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-03-27T10:31:23.752 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm 2024-03-27T10:31:23.753 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm-amd.ko 2024-03-27T10:31:23.753 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm-intel.ko 2024-03-27T10:31:23.753 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm.ko 2024-03-27T10:31:23.753 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto 2024-03-27T10:31:23.753 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/ansi_cprng.ko 2024-03-27T10:31:23.753 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/anubis.ko 2024-03-27T10:31:23.753 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/arc4.ko 2024-03-27T10:31:23.753 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx 2024-03-27T10:31:23.753 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_memcpy.ko 2024-03-27T10:31:23.753 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_pq.ko 2024-03-27T10:31:23.754 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_raid6_recov.ko 2024-03-27T10:31:23.754 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_tx.ko 2024-03-27T10:31:23.754 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_xor.ko 2024-03-27T10:31:23.754 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/raid6test.ko 2024-03-27T10:31:23.754 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/authenc.ko 2024-03-27T10:31:23.754 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/authencesn.ko 2024-03-27T10:31:23.754 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blake2b_generic.ko 2024-03-27T10:31:23.754 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blowfish_common.ko 2024-03-27T10:31:23.754 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blowfish_generic.ko 2024-03-27T10:31:23.754 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/camellia_generic.ko 2024-03-27T10:31:23.754 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast5_generic.ko 2024-03-27T10:31:23.754 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast6_generic.ko 2024-03-27T10:31:23.755 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast_common.ko 2024-03-27T10:31:23.755 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/ccm.ko 2024-03-27T10:31:23.755 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cmac.ko 2024-03-27T10:31:23.755 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crc32_generic.ko 2024-03-27T10:31:23.755 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crypto_engine.ko 2024-03-27T10:31:23.755 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crypto_user.ko 2024-03-27T10:31:23.755 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cts.ko 2024-03-27T10:31:23.755 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/des_generic.ko 2024-03-27T10:31:23.755 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/dh_generic.ko 2024-03-27T10:31:23.755 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/echainiv.ko 2024-03-27T10:31:23.755 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/essiv.ko 2024-03-27T10:31:23.755 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/fcrypt.ko 2024-03-27T10:31:23.756 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/khazad.ko 2024-03-27T10:31:23.756 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/lz4.ko 2024-03-27T10:31:23.756 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/lz4hc.ko 2024-03-27T10:31:23.756 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/md4.ko 2024-03-27T10:31:23.756 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/michael_mic.ko 2024-03-27T10:31:23.756 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/pcbc.ko 2024-03-27T10:31:23.756 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/pcrypt.ko 2024-03-27T10:31:23.756 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/rmd160.ko 2024-03-27T10:31:23.756 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/seed.ko 2024-03-27T10:31:23.756 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/serpent_generic.ko 2024-03-27T10:31:23.756 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/tcrypt.ko 2024-03-27T10:31:23.756 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/tea.ko 2024-03-27T10:31:23.757 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/twofish_common.ko 2024-03-27T10:31:23.757 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/twofish_generic.ko 2024-03-27T10:31:23.757 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/vmac.ko 2024-03-27T10:31:23.757 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/wp512.ko 2024-03-27T10:31:23.757 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xcbc.ko 2024-03-27T10:31:23.757 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xor.ko 2024-03-27T10:31:23.757 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xxhash_generic.ko 2024-03-27T10:31:23.757 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers 2024-03-27T10:31:23.757 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi 2024-03-27T10:31:23.757 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/acpi_ipmi.ko 2024-03-27T10:31:23.757 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/acpi_pad.ko 2024-03-27T10:31:23.757 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/custom_method.ko 2024-03-27T10:31:23.758 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/ec_sys.ko 2024-03-27T10:31:23.758 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/platform_profile.ko 2024-03-27T10:31:23.758 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/sbs.ko 2024-03-27T10:31:23.758 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/sbshc.ko 2024-03-27T10:31:23.758 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/video.ko 2024-03-27T10:31:23.758 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata 2024-03-27T10:31:23.758 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/acard-ahci.ko 2024-03-27T10:31:23.758 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/ahci_platform.ko 2024-03-27T10:31:23.758 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/ata_generic.ko 2024-03-27T10:31:23.758 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/libahci_platform.ko 2024-03-27T10:31:23.758 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_acpi.ko 2024-03-27T10:31:23.758 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ali.ko 2024-03-27T10:31:23.759 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_amd.ko 2024-03-27T10:31:23.759 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_artop.ko 2024-03-27T10:31:23.759 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_atiixp.ko 2024-03-27T10:31:23.759 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_atp867x.ko 2024-03-27T10:31:23.759 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cmd640.ko 2024-03-27T10:31:23.759 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cmd64x.ko 2024-03-27T10:31:23.759 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cypress.ko 2024-03-27T10:31:23.759 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_efar.ko 2024-03-27T10:31:23.759 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt366.ko 2024-03-27T10:31:23.759 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt37x.ko 2024-03-27T10:31:23.759 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt3x2n.ko 2024-03-27T10:31:23.759 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt3x3.ko 2024-03-27T10:31:23.760 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_it8213.ko 2024-03-27T10:31:23.760 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_it821x.ko 2024-03-27T10:31:23.760 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_jmicron.ko 2024-03-27T10:31:23.760 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_marvell.ko 2024-03-27T10:31:23.760 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_mpiix.ko 2024-03-27T10:31:23.760 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_netcell.ko 2024-03-27T10:31:23.760 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ninja32.ko 2024-03-27T10:31:23.760 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ns87410.ko 2024-03-27T10:31:23.760 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ns87415.ko 2024-03-27T10:31:23.760 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_oldpiix.ko 2024-03-27T10:31:23.760 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_opti.ko 2024-03-27T10:31:23.760 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_optidma.ko 2024-03-27T10:31:23.760 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pcmcia.ko 2024-03-27T10:31:23.760 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pdc2027x.ko 2024-03-27T10:31:23.761 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-03-27T10:31:23.761 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_piccolo.ko 2024-03-27T10:31:23.761 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_rdc.ko 2024-03-27T10:31:23.761 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sch.ko 2024-03-27T10:31:23.761 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_serverworks.ko 2024-03-27T10:31:23.761 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sil680.ko 2024-03-27T10:31:23.761 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sis.ko 2024-03-27T10:31:23.761 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sl82c105.ko 2024-03-27T10:31:23.761 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_triflex.ko 2024-03-27T10:31:23.761 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_via.ko 2024-03-27T10:31:23.761 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pdc_adma.ko 2024-03-27T10:31:23.762 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_inic162x.ko 2024-03-27T10:31:23.762 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_mv.ko 2024-03-27T10:31:23.762 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_nv.ko 2024-03-27T10:31:23.762 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_promise.ko 2024-03-27T10:31:23.762 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_qstor.ko 2024-03-27T10:31:23.762 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sil.ko 2024-03-27T10:31:23.762 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sil24.ko 2024-03-27T10:31:23.762 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sis.ko 2024-03-27T10:31:23.762 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_svw.ko 2024-03-27T10:31:23.762 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sx4.ko 2024-03-27T10:31:23.763 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_uli.ko 2024-03-27T10:31:23.763 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_via.ko 2024-03-27T10:31:23.763 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_vsc.ko 2024-03-27T10:31:23.763 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/auxdisplay 2024-03-27T10:31:23.763 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/auxdisplay/ks0108.ko 2024-03-27T10:31:23.763 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base 2024-03-27T10:31:23.763 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base/regmap 2024-03-27T10:31:23.763 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base/regmap/regmap-i2c.ko 2024-03-27T10:31:23.763 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base/regmap/regmap-mmio.ko 2024-03-27T10:31:23.763 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/bcma 2024-03-27T10:31:23.763 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/bcma/bcma.ko 2024-03-27T10:31:23.764 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block 2024-03-27T10:31:23.764 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/aoe 2024-03-27T10:31:23.764 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/aoe/aoe.ko 2024-03-27T10:31:23.764 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/brd.ko 2024-03-27T10:31:23.764 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/drbd 2024-03-27T10:31:23.764 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/drbd/drbd.ko 2024-03-27T10:31:23.764 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/floppy.ko 2024-03-27T10:31:23.764 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/loop.ko 2024-03-27T10:31:23.764 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/mtip32xx 2024-03-27T10:31:23.764 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-03-27T10:31:23.764 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/nbd.ko 2024-03-27T10:31:23.764 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/null_blk 2024-03-27T10:31:23.765 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/null_blk/null_blk.ko 2024-03-27T10:31:23.765 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/pktcdvd.ko 2024-03-27T10:31:23.765 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/rbd.ko 2024-03-27T10:31:23.765 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/virtio_blk.ko 2024-03-27T10:31:23.765 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkback 2024-03-27T10:31:23.765 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-03-27T10:31:23.765 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkfront.ko 2024-03-27T10:31:23.765 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/zram 2024-03-27T10:31:23.765 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/zram/zram.ko 2024-03-27T10:31:23.766 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char 2024-03-27T10:31:23.766 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hangcheck-timer.ko 2024-03-27T10:31:23.766 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random 2024-03-27T10:31:23.766 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/amd-rng.ko 2024-03-27T10:31:23.766 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/intel-rng.ko 2024-03-27T10:31:23.766 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-03-27T10:31:23.766 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/via-rng.ko 2024-03-27T10:31:23.766 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/virtio-rng.ko 2024-03-27T10:31:23.766 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi 2024-03-27T10:31:23.766 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-03-27T10:31:23.766 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-03-27T10:31:23.767 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-03-27T10:31:23.767 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_si.ko 2024-03-27T10:31:23.767 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-03-27T10:31:23.767 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-03-27T10:31:23.767 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/lp.ko 2024-03-27T10:31:23.767 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/mwave 2024-03-27T10:31:23.767 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/mwave/mwave.ko 2024-03-27T10:31:23.767 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ppdev.ko 2024-03-27T10:31:23.767 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tlclk.ko 2024-03-27T10:31:23.767 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm 2024-03-27T10:31:23.767 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm.ko 2024-03-27T10:31:23.768 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_atmel.ko 2024-03-27T10:31:23.768 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_infineon.ko 2024-03-27T10:31:23.768 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_nsc.ko 2024-03-27T10:31:23.768 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_tis.ko 2024-03-27T10:31:23.768 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-03-27T10:31:23.768 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/uv_mmtimer.ko 2024-03-27T10:31:23.768 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/virtio_console.ko 2024-03-27T10:31:23.768 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq 2024-03-27T10:31:23.768 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-03-27T10:31:23.768 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-03-27T10:31:23.768 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/p4-clockmod.ko 2024-03-27T10:31:23.769 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-03-27T10:31:23.769 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/powernow-k8.ko 2024-03-27T10:31:23.769 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/speedstep-lib.ko 2024-03-27T10:31:23.769 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto 2024-03-27T10:31:23.769 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp 2024-03-27T10:31:23.769 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-03-27T10:31:23.769 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp/ccp.ko 2024-03-27T10:31:23.769 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel 2024-03-27T10:31:23.769 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat 2024-03-27T10:31:23.769 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_common 2024-03-27T10:31:23.769 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-03-27T10:31:23.769 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-03-27T10:31:23.770 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-03-27T10:31:23.770 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/padlock-aes.ko 2024-03-27T10:31:23.770 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/padlock-sha.ko 2024-03-27T10:31:23.770 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/virtio 2024-03-27T10:31:23.770 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-03-27T10:31:23.770 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dca 2024-03-27T10:31:23.770 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dca/dca.ko 2024-03-27T10:31:23.770 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/devfreq 2024-03-27T10:31:23.770 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-03-27T10:31:23.770 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma 2024-03-27T10:31:23.770 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/dw 2024-03-27T10:31:23.770 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/dw/dw_dmac.ko 2024-03-27T10:31:23.771 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/ioat 2024-03-27T10:31:23.771 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/ioat/ioatdma.ko 2024-03-27T10:31:23.771 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac 2024-03-27T10:31:23.771 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/amd64_edac.ko 2024-03-27T10:31:23.771 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/e752x_edac.ko 2024-03-27T10:31:23.771 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/edac_mce_amd.ko 2024-03-27T10:31:23.771 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i3000_edac.ko 2024-03-27T10:31:23.771 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i3200_edac.ko 2024-03-27T10:31:23.771 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i5100_edac.ko 2024-03-27T10:31:23.771 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i5400_edac.ko 2024-03-27T10:31:23.771 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i7300_edac.ko 2024-03-27T10:31:23.772 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i7core_edac.ko 2024-03-27T10:31:23.772 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i82975x_edac.ko 2024-03-27T10:31:23.772 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/ie31200_edac.ko 2024-03-27T10:31:23.772 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/sb_edac.ko 2024-03-27T10:31:23.772 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/x38_edac.ko 2024-03-27T10:31:23.772 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware 2024-03-27T10:31:23.772 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware/edd.ko 2024-03-27T10:31:23.772 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware/iscsi_ibft.ko 2024-03-27T10:31:23.772 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio 2024-03-27T10:31:23.772 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-ich.ko 2024-03-27T10:31:23.772 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-viperboard.ko 2024-03-27T10:31:23.773 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid 2024-03-27T10:31:23.773 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-appleir.ko 2024-03-27T10:31:23.773 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-aureal.ko 2024-03-27T10:31:23.773 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-axff.ko 2024-03-27T10:31:23.773 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-dr.ko 2024-03-27T10:31:23.773 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-elecom.ko 2024-03-27T10:31:23.773 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-elo.ko 2024-03-27T10:31:23.773 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-emsff.ko 2024-03-27T10:31:23.773 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gaff.ko 2024-03-27T10:31:23.773 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gt683r.ko 2024-03-27T10:31:23.773 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gyration.ko 2024-03-27T10:31:23.773 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtek-kbd.ko 2024-03-27T10:31:23.774 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtek-mouse.ko 2024-03-27T10:31:23.774 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtekff.ko 2024-03-27T10:31:23.774 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-hyperv.ko 2024-03-27T10:31:23.774 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-icade.ko 2024-03-27T10:31:23.774 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-keytouch.ko 2024-03-27T10:31:23.774 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-kye.ko 2024-03-27T10:31:23.774 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-lcpower.ko 2024-03-27T10:31:23.774 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-led.ko 2024-03-27T10:31:23.774 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-lenovo.ko 2024-03-27T10:31:23.774 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-logitech-dj.ko 2024-03-27T10:31:23.774 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-03-27T10:31:23.775 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-multitouch.ko 2024-03-27T10:31:23.775 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-ortek.ko 2024-03-27T10:31:23.775 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-penmount.ko 2024-03-27T10:31:23.775 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-petalynx.ko 2024-03-27T10:31:23.775 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-picolcd.ko 2024-03-27T10:31:23.775 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-pl.ko 2024-03-27T10:31:23.775 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-plantronics.ko 2024-03-27T10:31:23.775 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-primax.ko 2024-03-27T10:31:23.775 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-rmi.ko 2024-03-27T10:31:23.775 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-arvo.ko 2024-03-27T10:31:23.775 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-common.ko 2024-03-27T10:31:23.775 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-isku.ko 2024-03-27T10:31:23.775 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-kone.ko 2024-03-27T10:31:23.776 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-03-27T10:31:23.776 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-konepure.ko 2024-03-27T10:31:23.776 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-03-27T10:31:23.776 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-lua.ko 2024-03-27T10:31:23.776 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-pyra.ko 2024-03-27T10:31:23.776 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-ryos.ko 2024-03-27T10:31:23.776 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-savu.ko 2024-03-27T10:31:23.776 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat.ko 2024-03-27T10:31:23.776 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-saitek.ko 2024-03-27T10:31:23.776 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-samsung.ko 2024-03-27T10:31:23.776 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sensor-hub.ko 2024-03-27T10:31:23.777 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sjoy.ko 2024-03-27T10:31:23.777 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sony.ko 2024-03-27T10:31:23.777 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-speedlink.ko 2024-03-27T10:31:23.777 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-steelseries.ko 2024-03-27T10:31:23.777 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sunplus.ko 2024-03-27T10:31:23.777 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-thrustmaster.ko 2024-03-27T10:31:23.777 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-tivo.ko 2024-03-27T10:31:23.777 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-tmff.ko 2024-03-27T10:31:23.777 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-topseed.ko 2024-03-27T10:31:23.777 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-twinhan.ko 2024-03-27T10:31:23.777 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-uclogic.ko 2024-03-27T10:31:23.777 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-waltop.ko 2024-03-27T10:31:23.778 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-wiimote.ko 2024-03-27T10:31:23.778 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-xinmo.ko 2024-03-27T10:31:23.778 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-zpff.ko 2024-03-27T10:31:23.778 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-zydacron.ko 2024-03-27T10:31:23.778 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/uhid.ko 2024-03-27T10:31:23.778 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/wacom.ko 2024-03-27T10:31:23.778 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv 2024-03-27T10:31:23.778 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_balloon.ko 2024-03-27T10:31:23.778 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_utils.ko 2024-03-27T10:31:23.778 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_vmbus.ko 2024-03-27T10:31:23.778 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon 2024-03-27T10:31:23.778 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/abituguru.ko 2024-03-27T10:31:23.779 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/abituguru3.ko 2024-03-27T10:31:23.779 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/acpi_power_meter.ko 2024-03-27T10:31:23.779 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ad7414.ko 2024-03-27T10:31:23.779 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ad7418.ko 2024-03-27T10:31:23.779 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adc128d818.ko 2024-03-27T10:31:23.779 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1025.ko 2024-03-27T10:31:23.779 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1026.ko 2024-03-27T10:31:23.779 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1029.ko 2024-03-27T10:31:23.779 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1031.ko 2024-03-27T10:31:23.779 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm9240.ko 2024-03-27T10:31:23.779 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ads7828.ko 2024-03-27T10:31:23.779 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7410.ko 2024-03-27T10:31:23.780 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7411.ko 2024-03-27T10:31:23.780 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7462.ko 2024-03-27T10:31:23.780 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7470.ko 2024-03-27T10:31:23.780 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7475.ko 2024-03-27T10:31:23.780 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7x10.ko 2024-03-27T10:31:23.780 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/amc6821.ko 2024-03-27T10:31:23.780 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/applesmc.ko 2024-03-27T10:31:23.780 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asb100.ko 2024-03-27T10:31:23.780 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asc7621.ko 2024-03-27T10:31:23.780 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asus_atk0110.ko 2024-03-27T10:31:23.780 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/atxp1.ko 2024-03-27T10:31:23.781 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/coretemp.ko 2024-03-27T10:31:23.781 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-03-27T10:31:23.781 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/dme1737.ko 2024-03-27T10:31:23.781 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ds1621.ko 2024-03-27T10:31:23.781 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ds620.ko 2024-03-27T10:31:23.781 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/emc1403.ko 2024-03-27T10:31:23.781 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/emc6w201.ko 2024-03-27T10:31:23.781 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f71805f.ko 2024-03-27T10:31:23.781 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f71882fg.ko 2024-03-27T10:31:23.781 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f75375s.ko 2024-03-27T10:31:23.782 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/fam15h_power.ko 2024-03-27T10:31:23.782 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/fschmd.ko 2024-03-27T10:31:23.782 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/g760a.ko 2024-03-27T10:31:23.782 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/g762.ko 2024-03-27T10:31:23.782 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/gl518sm.ko 2024-03-27T10:31:23.782 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/gl520sm.ko 2024-03-27T10:31:23.782 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/hwmon-vid.ko 2024-03-27T10:31:23.782 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/i5500_temp.ko 2024-03-27T10:31:23.782 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/i5k_amb.ko 2024-03-27T10:31:23.782 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ibmaem.ko 2024-03-27T10:31:23.782 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ibmpex.ko 2024-03-27T10:31:23.783 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ina209.ko 2024-03-27T10:31:23.783 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ina2xx.ko 2024-03-27T10:31:23.783 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/it87.ko 2024-03-27T10:31:23.783 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/k10temp.ko 2024-03-27T10:31:23.783 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/k8temp.ko 2024-03-27T10:31:23.783 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lineage-pem.ko 2024-03-27T10:31:23.783 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm63.ko 2024-03-27T10:31:23.783 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm73.ko 2024-03-27T10:31:23.783 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm75.ko 2024-03-27T10:31:23.783 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm77.ko 2024-03-27T10:31:23.783 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm78.ko 2024-03-27T10:31:23.783 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm80.ko 2024-03-27T10:31:23.783 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm83.ko 2024-03-27T10:31:23.784 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm85.ko 2024-03-27T10:31:23.784 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm87.ko 2024-03-27T10:31:23.784 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm90.ko 2024-03-27T10:31:23.784 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm92.ko 2024-03-27T10:31:23.784 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm93.ko 2024-03-27T10:31:23.784 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95234.ko 2024-03-27T10:31:23.784 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95241.ko 2024-03-27T10:31:23.784 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95245.ko 2024-03-27T10:31:23.784 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc2945.ko 2024-03-27T10:31:23.784 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4151.ko 2024-03-27T10:31:23.784 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4215.ko 2024-03-27T10:31:23.784 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4222.ko 2024-03-27T10:31:23.784 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4245.ko 2024-03-27T10:31:23.785 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4260.ko 2024-03-27T10:31:23.785 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4261.ko 2024-03-27T10:31:23.785 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max16065.ko 2024-03-27T10:31:23.785 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max1619.ko 2024-03-27T10:31:23.785 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max1668.ko 2024-03-27T10:31:23.785 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max197.ko 2024-03-27T10:31:23.785 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6639.ko 2024-03-27T10:31:23.785 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6650.ko 2024-03-27T10:31:23.785 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6697.ko 2024-03-27T10:31:23.785 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/mcp3021.ko 2024-03-27T10:31:23.785 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6683.ko 2024-03-27T10:31:23.785 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6775-core.ko 2024-03-27T10:31:23.786 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6775.ko 2024-03-27T10:31:23.786 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct7802.ko 2024-03-27T10:31:23.786 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pc87360.ko 2024-03-27T10:31:23.786 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pc87427.ko 2024-03-27T10:31:23.786 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pcf8591.ko 2024-03-27T10:31:23.786 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus 2024-03-27T10:31:23.786 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-03-27T10:31:23.786 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-03-27T10:31:23.786 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-03-27T10:31:23.786 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max16064.ko 2024-03-27T10:31:23.786 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max34440.ko 2024-03-27T10:31:23.787 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max8688.ko 2024-03-27T10:31:23.787 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-03-27T10:31:23.787 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-03-27T10:31:23.787 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-03-27T10:31:23.787 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-03-27T10:31:23.787 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-03-27T10:31:23.787 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-03-27T10:31:23.787 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/powr1220.ko 2024-03-27T10:31:23.787 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch5627.ko 2024-03-27T10:31:23.787 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch5636.ko 2024-03-27T10:31:23.787 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch56xx-common.ko 2024-03-27T10:31:23.788 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sht15.ko 2024-03-27T10:31:23.788 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sht21.ko 2024-03-27T10:31:23.788 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/shtc1.ko 2024-03-27T10:31:23.788 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sis5595.ko 2024-03-27T10:31:23.788 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47b397.ko 2024-03-27T10:31:23.788 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47m1.ko 2024-03-27T10:31:23.788 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47m192.ko 2024-03-27T10:31:23.788 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/thmc50.ko 2024-03-27T10:31:23.788 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp102.ko 2024-03-27T10:31:23.788 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp103.ko 2024-03-27T10:31:23.788 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp401.ko 2024-03-27T10:31:23.788 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp421.ko 2024-03-27T10:31:23.788 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/via-cputemp.ko 2024-03-27T10:31:23.789 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/via686a.ko 2024-03-27T10:31:23.789 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/vt1211.ko 2024-03-27T10:31:23.789 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/vt8231.ko 2024-03-27T10:31:23.789 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83627ehf.ko 2024-03-27T10:31:23.789 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83627hf.ko 2024-03-27T10:31:23.789 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83781d.ko 2024-03-27T10:31:23.789 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83791d.ko 2024-03-27T10:31:23.789 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83792d.ko 2024-03-27T10:31:23.789 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83793.ko 2024-03-27T10:31:23.789 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83795.ko 2024-03-27T10:31:23.789 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83l785ts.ko 2024-03-27T10:31:23.789 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83l786ng.ko 2024-03-27T10:31:23.790 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c 2024-03-27T10:31:23.790 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos 2024-03-27T10:31:23.790 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-03-27T10:31:23.790 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-03-27T10:31:23.790 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses 2024-03-27T10:31:23.790 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-03-27T10:31:23.790 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-03-27T10:31:23.790 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-03-27T10:31:23.790 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-03-27T10:31:23.790 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-03-27T10:31:23.790 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-03-27T10:31:23.791 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-03-27T10:31:23.791 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-03-27T10:31:23.791 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-i801.ko 2024-03-27T10:31:23.791 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-isch.ko 2024-03-27T10:31:23.791 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-03-27T10:31:23.791 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-03-27T10:31:23.791 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-03-27T10:31:23.791 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-parport.ko 2024-03-27T10:31:23.791 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-03-27T10:31:23.791 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-03-27T10:31:23.791 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-03-27T10:31:23.791 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-03-27T10:31:23.792 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-03-27T10:31:23.792 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-03-27T10:31:23.792 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-via.ko 2024-03-27T10:31:23.792 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-03-27T10:31:23.792 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-03-27T10:31:23.792 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-dev.ko 2024-03-27T10:31:23.792 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-mux.ko 2024-03-27T10:31:23.792 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-03-27T10:31:23.792 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-smbus.ko 2024-03-27T10:31:23.792 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-stub.ko 2024-03-27T10:31:23.792 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband 2024-03-27T10:31:23.792 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core 2024-03-27T10:31:23.792 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_cm.ko 2024-03-27T10:31:23.793 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_core.ko 2024-03-27T10:31:23.793 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_umad.ko 2024-03-27T10:31:23.793 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-03-27T10:31:23.793 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/iw_cm.ko 2024-03-27T10:31:23.793 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/rdma_cm.ko 2024-03-27T10:31:23.793 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-03-27T10:31:23.793 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw 2024-03-27T10:31:23.793 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/cxgb4 2024-03-27T10:31:23.793 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-03-27T10:31:23.793 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx4 2024-03-27T10:31:23.794 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-03-27T10:31:23.794 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx5 2024-03-27T10:31:23.794 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-03-27T10:31:23.794 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mthca 2024-03-27T10:31:23.794 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-03-27T10:31:23.794 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp 2024-03-27T10:31:23.794 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/ipoib 2024-03-27T10:31:23.794 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-03-27T10:31:23.794 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/iser 2024-03-27T10:31:23.794 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-03-27T10:31:23.795 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/isert 2024-03-27T10:31:23.795 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-03-27T10:31:23.795 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srp 2024-03-27T10:31:23.795 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-03-27T10:31:23.795 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srpt 2024-03-27T10:31:23.795 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-03-27T10:31:23.795 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input 2024-03-27T10:31:23.795 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport 2024-03-27T10:31:23.795 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-03-27T10:31:23.795 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/fm801-gp.ko 2024-03-27T10:31:23.795 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/gameport.ko 2024-03-27T10:31:23.796 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/joydev.ko 2024-03-27T10:31:23.796 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard 2024-03-27T10:31:23.796 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/gpio_keys.ko 2024-03-27T10:31:23.796 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse 2024-03-27T10:31:23.796 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/appletouch.ko 2024-03-27T10:31:23.796 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/bcm5974.ko 2024-03-27T10:31:23.796 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/cyapatp.ko 2024-03-27T10:31:23.796 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/elan_i2c.ko 2024-03-27T10:31:23.796 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/sermouse.ko 2024-03-27T10:31:23.796 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-03-27T10:31:23.796 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/synaptics_usb.ko 2024-03-27T10:31:23.796 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/vsxxxaa.ko 2024-03-27T10:31:23.796 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/rmi4 2024-03-27T10:31:23.797 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/rmi4/rmi_core.ko 2024-03-27T10:31:23.797 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio 2024-03-27T10:31:23.797 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/altera_ps2.ko 2024-03-27T10:31:23.797 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/arc_ps2.ko 2024-03-27T10:31:23.797 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-03-27T10:31:23.797 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/serio_raw.ko 2024-03-27T10:31:23.797 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/sparse-keymap.ko 2024-03-27T10:31:23.797 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds 2024-03-27T10:31:23.797 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-blinkm.ko 2024-03-27T10:31:23.797 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-lm3530.ko 2024-03-27T10:31:23.797 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-lp3944.ko 2024-03-27T10:31:23.797 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-ss4200.ko 2024-03-27T10:31:23.798 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger 2024-03-27T10:31:23.798 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-03-27T10:31:23.798 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-03-27T10:31:23.798 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-03-27T10:31:23.798 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-03-27T10:31:23.798 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-03-27T10:31:23.798 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-03-27T10:31:23.798 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-03-27T10:31:23.798 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-03-27T10:31:23.798 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-03-27T10:31:23.798 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md 2024-03-27T10:31:23.798 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/bcache 2024-03-27T10:31:23.799 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/bcache/bcache.ko 2024-03-27T10:31:23.799 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-bio-prison.ko 2024-03-27T10:31:23.799 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-cache-smq.ko 2024-03-27T10:31:23.799 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-cache.ko 2024-03-27T10:31:23.799 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-crypt.ko 2024-03-27T10:31:23.799 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-delay.ko 2024-03-27T10:31:23.799 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-flakey.ko 2024-03-27T10:31:23.799 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-log-userspace.ko 2024-03-27T10:31:23.799 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-multipath.ko 2024-03-27T10:31:23.799 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-queue-length.ko 2024-03-27T10:31:23.800 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-raid.ko 2024-03-27T10:31:23.800 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-round-robin.ko 2024-03-27T10:31:23.800 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-service-time.ko 2024-03-27T10:31:23.800 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-switch.ko 2024-03-27T10:31:23.800 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-thin-pool.ko 2024-03-27T10:31:23.800 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-verity.ko 2024-03-27T10:31:23.800 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/persistent-data 2024-03-27T10:31:23.800 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-03-27T10:31:23.800 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid0.ko 2024-03-27T10:31:23.800 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid1.ko 2024-03-27T10:31:23.800 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid10.ko 2024-03-27T10:31:23.800 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid456.ko 2024-03-27T10:31:23.801 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message 2024-03-27T10:31:23.801 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion 2024-03-27T10:31:23.801 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptbase.ko 2024-03-27T10:31:23.801 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptctl.ko 2024-03-27T10:31:23.801 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptfc.ko 2024-03-27T10:31:23.801 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptlan.ko 2024-03-27T10:31:23.801 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptsas.ko 2024-03-27T10:31:23.801 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptscsih.ko 2024-03-27T10:31:23.801 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptspi.ko 2024-03-27T10:31:23.801 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd 2024-03-27T10:31:23.801 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/lpc_ich.ko 2024-03-27T10:31:23.801 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/lpc_sch.ko 2024-03-27T10:31:23.802 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/mfd-core.ko 2024-03-27T10:31:23.802 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/sm501.ko 2024-03-27T10:31:23.802 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/viperboard.ko 2024-03-27T10:31:23.802 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/vx855.ko 2024-03-27T10:31:23.802 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/wl1273-core.ko 2024-03-27T10:31:23.802 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc 2024-03-27T10:31:23.802 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/altera-stapl 2024-03-27T10:31:23.802 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-03-27T10:31:23.802 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/apds9802als.ko 2024-03-27T10:31:23.802 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/apds990x.ko 2024-03-27T10:31:23.802 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/bh1770glc.ko 2024-03-27T10:31:23.802 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/cb710 2024-03-27T10:31:23.803 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/cb710/cb710.ko 2024-03-27T10:31:23.803 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom 2024-03-27T10:31:23.803 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/at24.ko 2024-03-27T10:31:23.803 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-03-27T10:31:23.803 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/max6875.ko 2024-03-27T10:31:23.803 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/enclosure.ko 2024-03-27T10:31:23.803 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/hpilo.ko 2024-03-27T10:31:23.803 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/isl29003.ko 2024-03-27T10:31:23.803 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/isl29020.ko 2024-03-27T10:31:23.803 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d 2024-03-27T10:31:23.803 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-03-27T10:31:23.804 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-03-27T10:31:23.804 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei 2024-03-27T10:31:23.804 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei-me.ko 2024-03-27T10:31:23.804 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei-txe.ko 2024-03-27T10:31:23.804 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei.ko 2024-03-27T10:31:23.804 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-gru 2024-03-27T10:31:23.804 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-gru/gru.ko 2024-03-27T10:31:23.804 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp 2024-03-27T10:31:23.804 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp/xp.ko 2024-03-27T10:31:23.804 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp/xpc.ko 2024-03-27T10:31:23.804 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-03-27T10:31:23.804 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tifm_7xx1.ko 2024-03-27T10:31:23.805 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tifm_core.ko 2024-03-27T10:31:23.805 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tsl2550.ko 2024-03-27T10:31:23.805 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_balloon.ko 2024-03-27T10:31:23.805 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_vmci 2024-03-27T10:31:23.805 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-03-27T10:31:23.805 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net 2024-03-27T10:31:23.805 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/bonding 2024-03-27T10:31:23.805 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/bonding/bonding.ko 2024-03-27T10:31:23.805 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa 2024-03-27T10:31:23.805 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/b53 2024-03-27T10:31:23.805 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/b53/b53_common.ko 2024-03-27T10:31:23.806 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/bcm-sf2.ko 2024-03-27T10:31:23.806 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6060.ko 2024-03-27T10:31:23.806 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6xxx 2024-03-27T10:31:23.806 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-03-27T10:31:23.806 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dummy.ko 2024-03-27T10:31:23.806 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/eql.ko 2024-03-27T10:31:23.806 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet 2024-03-27T10:31:23.806 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com 2024-03-27T10:31:23.806 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-03-27T10:31:23.806 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-03-27T10:31:23.806 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-03-27T10:31:23.806 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-03-27T10:31:23.807 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390 2024-03-27T10:31:23.807 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/8390.ko 2024-03-27T10:31:23.807 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-03-27T10:31:23.807 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-03-27T10:31:23.807 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-03-27T10:31:23.807 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/adaptec 2024-03-27T10:31:23.807 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-03-27T10:31:23.807 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/agere 2024-03-27T10:31:23.807 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/agere/et131x.ko 2024-03-27T10:31:23.807 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/alteon 2024-03-27T10:31:23.808 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-03-27T10:31:23.808 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/altera 2024-03-27T10:31:23.808 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-03-27T10:31:23.808 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd 2024-03-27T10:31:23.808 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-03-27T10:31:23.808 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-03-27T10:31:23.808 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-03-27T10:31:23.808 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros 2024-03-27T10:31:23.808 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/alx 2024-03-27T10:31:23.808 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-03-27T10:31:23.808 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1c 2024-03-27T10:31:23.809 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-03-27T10:31:23.809 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1e 2024-03-27T10:31:23.809 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-03-27T10:31:23.809 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx 2024-03-27T10:31:23.809 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-03-27T10:31:23.809 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-03-27T10:31:23.809 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom 2024-03-27T10:31:23.809 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-03-27T10:31:23.809 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-03-27T10:31:23.809 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-03-27T10:31:23.809 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-03-27T10:31:23.809 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-03-27T10:31:23.810 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/genet 2024-03-27T10:31:23.810 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-03-27T10:31:23.810 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-03-27T10:31:23.810 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade 2024-03-27T10:31:23.810 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade/bna 2024-03-27T10:31:23.810 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-03-27T10:31:23.810 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio 2024-03-27T10:31:23.810 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb 2024-03-27T10:31:23.810 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-03-27T10:31:23.810 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-03-27T10:31:23.810 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-03-27T10:31:23.810 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-03-27T10:31:23.811 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-03-27T10:31:23.811 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-03-27T10:31:23.811 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-03-27T10:31:23.811 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-03-27T10:31:23.811 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-03-27T10:31:23.811 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco 2024-03-27T10:31:23.811 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco/enic 2024-03-27T10:31:23.811 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-03-27T10:31:23.811 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec 2024-03-27T10:31:23.811 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip 2024-03-27T10:31:23.811 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-03-27T10:31:23.812 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-03-27T10:31:23.812 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-03-27T10:31:23.812 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-03-27T10:31:23.812 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-03-27T10:31:23.812 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-03-27T10:31:23.812 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink 2024-03-27T10:31:23.812 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-03-27T10:31:23.812 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-03-27T10:31:23.812 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dnet.ko 2024-03-27T10:31:23.813 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex 2024-03-27T10:31:23.813 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex/benet 2024-03-27T10:31:23.813 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-03-27T10:31:23.813 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ethoc.ko 2024-03-27T10:31:23.813 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/fealnx.ko 2024-03-27T10:31:23.813 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel 2024-03-27T10:31:23.813 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e100.ko 2024-03-27T10:31:23.813 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000 2024-03-27T10:31:23.813 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-03-27T10:31:23.813 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000e 2024-03-27T10:31:23.813 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-03-27T10:31:23.814 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/fm10k 2024-03-27T10:31:23.814 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-03-27T10:31:23.814 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/i40e 2024-03-27T10:31:23.814 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-03-27T10:31:23.814 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/iavf 2024-03-27T10:31:23.814 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-03-27T10:31:23.814 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igb 2024-03-27T10:31:23.814 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-03-27T10:31:23.815 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igbvf 2024-03-27T10:31:23.815 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-03-27T10:31:23.815 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbe 2024-03-27T10:31:23.815 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-03-27T10:31:23.815 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbevf 2024-03-27T10:31:23.815 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-03-27T10:31:23.816 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/jme.ko 2024-03-27T10:31:23.816 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell 2024-03-27T10:31:23.816 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-03-27T10:31:23.816 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/skge.ko 2024-03-27T10:31:23.816 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-03-27T10:31:23.816 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox 2024-03-27T10:31:23.816 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4 2024-03-27T10:31:23.816 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-03-27T10:31:23.816 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-03-27T10:31:23.816 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5 2024-03-27T10:31:23.816 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-03-27T10:31:23.816 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-03-27T10:31:23.817 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/micrel 2024-03-27T10:31:23.817 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-03-27T10:31:23.817 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom 2024-03-27T10:31:23.817 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom/myri10ge 2024-03-27T10:31:23.817 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-03-27T10:31:23.817 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi 2024-03-27T10:31:23.817 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-03-27T10:31:23.817 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-03-27T10:31:23.817 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/neterion 2024-03-27T10:31:23.817 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-03-27T10:31:23.817 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/nvidia 2024-03-27T10:31:23.817 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-03-27T10:31:23.818 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines 2024-03-27T10:31:23.818 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-03-27T10:31:23.818 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-03-27T10:31:23.818 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic 2024-03-27T10:31:23.818 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/netxen 2024-03-27T10:31:23.818 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-03-27T10:31:23.818 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-03-27T10:31:23.818 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-03-27T10:31:23.818 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-03-27T10:31:23.819 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rdc 2024-03-27T10:31:23.819 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-03-27T10:31:23.819 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek 2024-03-27T10:31:23.819 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-03-27T10:31:23.819 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-03-27T10:31:23.819 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/atp.ko 2024-03-27T10:31:23.819 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-03-27T10:31:23.819 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rocker 2024-03-27T10:31:23.819 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-03-27T10:31:23.819 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sfc 2024-03-27T10:31:23.819 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-03-27T10:31:23.820 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/silan 2024-03-27T10:31:23.820 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-03-27T10:31:23.820 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis 2024-03-27T10:31:23.820 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis/sis190.ko 2024-03-27T10:31:23.820 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis/sis900.ko 2024-03-27T10:31:23.820 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc 2024-03-27T10:31:23.820 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-03-27T10:31:23.820 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-03-27T10:31:23.820 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-03-27T10:31:23.820 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-03-27T10:31:23.820 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro 2024-03-27T10:31:23.820 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac 2024-03-27T10:31:23.821 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-03-27T10:31:23.821 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-03-27T10:31:23.821 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun 2024-03-27T10:31:23.821 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/cassini.ko 2024-03-27T10:31:23.821 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/niu.ko 2024-03-27T10:31:23.821 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/sungem.ko 2024-03-27T10:31:23.821 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-03-27T10:31:23.821 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/tehuti 2024-03-27T10:31:23.821 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-03-27T10:31:23.821 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ti 2024-03-27T10:31:23.821 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ti/tlan.ko 2024-03-27T10:31:23.822 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via 2024-03-27T10:31:23.822 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-03-27T10:31:23.822 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-03-27T10:31:23.822 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet 2024-03-27T10:31:23.822 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-03-27T10:31:23.822 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-03-27T10:31:23.822 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/xircom 2024-03-27T10:31:23.822 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-03-27T10:31:23.822 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/geneve.ko 2024-03-27T10:31:23.822 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/hyperv 2024-03-27T10:31:23.822 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-03-27T10:31:23.822 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ieee802154 2024-03-27T10:31:23.822 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ieee802154/fakelb.ko 2024-03-27T10:31:23.823 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ifb.ko 2024-03-27T10:31:23.823 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ipvlan 2024-03-27T10:31:23.823 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ipvlan/ipvlan.ko 2024-03-27T10:31:23.823 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/macvlan.ko 2024-03-27T10:31:23.823 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/macvtap.ko 2024-03-27T10:31:23.823 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio 2024-03-27T10:31:23.823 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio.ko 2024-03-27T10:31:23.823 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-03-27T10:31:23.823 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-03-27T10:31:23.823 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-regmap.ko 2024-03-27T10:31:23.823 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mii.ko 2024-03-27T10:31:23.824 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/net_failover.ko 2024-03-27T10:31:23.824 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/netconsole.ko 2024-03-27T10:31:23.824 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/nlmon.ko 2024-03-27T10:31:23.824 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs 2024-03-27T10:31:23.824 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs/pcs-lynx.ko 2024-03-27T10:31:23.824 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-03-27T10:31:23.824 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy 2024-03-27T10:31:23.824 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/amd.ko 2024-03-27T10:31:23.824 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-03-27T10:31:23.824 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-03-27T10:31:23.824 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm7xxx.ko 2024-03-27T10:31:23.825 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm87xx.ko 2024-03-27T10:31:23.825 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/broadcom.ko 2024-03-27T10:31:23.825 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/cicada.ko 2024-03-27T10:31:23.825 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/davicom.ko 2024-03-27T10:31:23.825 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/dp83640.ko 2024-03-27T10:31:23.825 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/et1011c.ko 2024-03-27T10:31:23.825 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/icplus.ko 2024-03-27T10:31:23.825 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/lxt.ko 2024-03-27T10:31:23.825 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/marvell.ko 2024-03-27T10:31:23.825 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/micrel.ko 2024-03-27T10:31:23.825 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/national.ko 2024-03-27T10:31:23.825 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/phylink.ko 2024-03-27T10:31:23.826 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/qsemi.ko 2024-03-27T10:31:23.826 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/realtek.ko 2024-03-27T10:31:23.826 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/smsc.ko 2024-03-27T10:31:23.826 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/ste10Xp.ko 2024-03-27T10:31:23.826 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/vitesse.ko 2024-03-27T10:31:23.826 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp 2024-03-27T10:31:23.826 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/bsd_comp.ko 2024-03-27T10:31:23.826 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_async.ko 2024-03-27T10:31:23.826 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_deflate.ko 2024-03-27T10:31:23.826 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_generic.ko 2024-03-27T10:31:23.826 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_mppe.ko 2024-03-27T10:31:23.827 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_synctty.ko 2024-03-27T10:31:23.827 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pppoe.ko 2024-03-27T10:31:23.827 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pppox.ko 2024-03-27T10:31:23.827 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pptp.ko 2024-03-27T10:31:23.827 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip 2024-03-27T10:31:23.827 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip/slhc.ko 2024-03-27T10:31:23.827 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip/slip.ko 2024-03-27T10:31:23.827 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/sungem_phy.ko 2024-03-27T10:31:23.827 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/tap.ko 2024-03-27T10:31:23.827 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team 2024-03-27T10:31:23.827 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team.ko 2024-03-27T10:31:23.827 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_activebackup.ko 2024-03-27T10:31:23.828 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_broadcast.ko 2024-03-27T10:31:23.828 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-03-27T10:31:23.828 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_random.ko 2024-03-27T10:31:23.828 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-03-27T10:31:23.828 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/tun.ko 2024-03-27T10:31:23.828 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/veth.ko 2024-03-27T10:31:23.828 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/virtio_net.ko 2024-03-27T10:31:23.828 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vmxnet3 2024-03-27T10:31:23.828 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-03-27T10:31:23.828 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vxlan 2024-03-27T10:31:23.828 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vxlan/vxlan.ko 2024-03-27T10:31:23.829 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netback 2024-03-27T10:31:23.829 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netback/xen-netback.ko 2024-03-27T10:31:23.829 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netfront.ko 2024-03-27T10:31:23.829 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ntb 2024-03-27T10:31:23.829 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ntb/ntb.ko 2024-03-27T10:31:23.829 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme 2024-03-27T10:31:23.829 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host 2024-03-27T10:31:23.829 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host/nvme-core.ko 2024-03-27T10:31:23.829 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host/nvme.ko 2024-03-27T10:31:23.829 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport 2024-03-27T10:31:23.830 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport.ko 2024-03-27T10:31:23.830 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_cs.ko 2024-03-27T10:31:23.830 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_pc.ko 2024-03-27T10:31:23.830 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_serial.ko 2024-03-27T10:31:23.830 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci 2024-03-27T10:31:23.830 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/hotplug 2024-03-27T10:31:23.830 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-03-27T10:31:23.830 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/pcie 2024-03-27T10:31:23.830 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/pcie/aer_inject.ko 2024-03-27T10:31:23.830 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/xen-pcifront.ko 2024-03-27T10:31:23.830 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia 2024-03-27T10:31:23.831 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/i82092.ko 2024-03-27T10:31:23.831 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/pd6729.ko 2024-03-27T10:31:23.831 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/yenta_socket.ko 2024-03-27T10:31:23.831 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl 2024-03-27T10:31:23.831 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl/intel 2024-03-27T10:31:23.831 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-03-27T10:31:23.831 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform 2024-03-27T10:31:23.831 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86 2024-03-27T10:31:23.831 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/acer-wmi.ko 2024-03-27T10:31:23.831 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/acerhdf.ko 2024-03-27T10:31:23.832 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/apple-gmux.ko 2024-03-27T10:31:23.832 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-laptop.ko 2024-03-27T10:31:23.832 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-03-27T10:31:23.832 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-wmi.ko 2024-03-27T10:31:23.832 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/classmate-laptop.ko 2024-03-27T10:31:23.832 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-03-27T10:31:23.832 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-03-27T10:31:23.832 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-03-27T10:31:23.832 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-03-27T10:31:23.832 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/hdaps.ko 2024-03-27T10:31:23.832 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel 2024-03-27T10:31:23.833 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-03-27T10:31:23.833 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel_ips.ko 2024-03-27T10:31:23.833 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/msi-wmi.ko 2024-03-27T10:31:23.833 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/mxm-wmi.ko 2024-03-27T10:31:23.833 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-03-27T10:31:23.833 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/samsung-laptop.ko 2024-03-27T10:31:23.833 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/samsung-q10.ko 2024-03-27T10:31:23.833 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/topstar-laptop.ko 2024-03-27T10:31:23.833 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-03-27T10:31:23.833 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/toshiba_haps.ko 2024-03-27T10:31:23.833 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/wmi-bmof.ko 2024-03-27T10:31:23.834 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/wmi.ko 2024-03-27T10:31:23.834 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap 2024-03-27T10:31:23.834 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap/intel_rapl_common.ko 2024-03-27T10:31:23.834 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap/intel_rapl_msr.ko 2024-03-27T10:31:23.834 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps 2024-03-27T10:31:23.834 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients 2024-03-27T10:31:23.834 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps-gpio.ko 2024-03-27T10:31:23.834 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps-ldisc.ko 2024-03-27T10:31:23.834 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps_parport.ko 2024-03-27T10:31:23.834 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/pps_core.ko 2024-03-27T10:31:23.834 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp 2024-03-27T10:31:23.835 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp/ptp.ko 2024-03-27T10:31:23.835 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp/ptp_kvm.ko 2024-03-27T10:31:23.835 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc 2024-03-27T10:31:23.835 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-bq32k.ko 2024-03-27T10:31:23.835 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1286.ko 2024-03-27T10:31:23.835 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1307.ko 2024-03-27T10:31:23.835 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1374.ko 2024-03-27T10:31:23.835 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1511.ko 2024-03-27T10:31:23.835 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1553.ko 2024-03-27T10:31:23.835 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1672.ko 2024-03-27T10:31:23.835 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1742.ko 2024-03-27T10:31:23.835 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds2404.ko 2024-03-27T10:31:23.836 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds3232.ko 2024-03-27T10:31:23.836 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-em3027.ko 2024-03-27T10:31:23.836 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-fm3130.ko 2024-03-27T10:31:23.836 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-isl12022.ko 2024-03-27T10:31:23.836 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-isl1208.ko 2024-03-27T10:31:23.836 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m41t80.ko 2024-03-27T10:31:23.836 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m48t35.ko 2024-03-27T10:31:23.836 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m48t59.ko 2024-03-27T10:31:23.836 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-max6900.ko 2024-03-27T10:31:23.836 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-msm6242.ko 2024-03-27T10:31:23.836 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf2127.ko 2024-03-27T10:31:23.836 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf85063.ko 2024-03-27T10:31:23.836 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8523.ko 2024-03-27T10:31:23.837 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8563.ko 2024-03-27T10:31:23.837 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8583.ko 2024-03-27T10:31:23.837 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rp5c01.ko 2024-03-27T10:31:23.837 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rs5c372.ko 2024-03-27T10:31:23.837 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-03-27T10:31:23.837 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rx8025.ko 2024-03-27T10:31:23.837 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rx8581.ko 2024-03-27T10:31:23.837 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-03-27T10:31:23.837 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-x1205.ko 2024-03-27T10:31:23.837 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi 2024-03-27T10:31:23.837 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-9xxx.ko 2024-03-27T10:31:23.837 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-sas.ko 2024-03-27T10:31:23.837 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-xxxx.ko 2024-03-27T10:31:23.837 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/BusLogic.ko 2024-03-27T10:31:23.838 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/a100u2w.ko 2024-03-27T10:31:23.838 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aacraid 2024-03-27T10:31:23.838 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aacraid/aacraid.ko 2024-03-27T10:31:23.838 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/advansys.ko 2024-03-27T10:31:23.838 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx 2024-03-27T10:31:23.838 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-03-27T10:31:23.838 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-03-27T10:31:23.838 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/am53c974.ko 2024-03-27T10:31:23.838 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/arcmsr 2024-03-27T10:31:23.838 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-03-27T10:31:23.838 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/atp870u.ko 2024-03-27T10:31:23.838 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/be2iscsi 2024-03-27T10:31:23.839 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-03-27T10:31:23.839 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bfa 2024-03-27T10:31:23.839 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bfa/bfa.ko 2024-03-27T10:31:23.839 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2fc 2024-03-27T10:31:23.839 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-03-27T10:31:23.839 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2i 2024-03-27T10:31:23.839 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-03-27T10:31:23.839 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ch.ko 2024-03-27T10:31:23.839 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/csiostor 2024-03-27T10:31:23.839 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/csiostor/csiostor.ko 2024-03-27T10:31:23.839 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi 2024-03-27T10:31:23.839 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb3i 2024-03-27T10:31:23.839 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-03-27T10:31:23.840 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb4i 2024-03-27T10:31:23.840 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-03-27T10:31:23.840 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-03-27T10:31:23.840 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/dc395x.ko 2024-03-27T10:31:23.840 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler 2024-03-27T10:31:23.840 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-03-27T10:31:23.840 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-03-27T10:31:23.840 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-03-27T10:31:23.840 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-03-27T10:31:23.840 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/dmx3191d.ko 2024-03-27T10:31:23.840 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esas2r 2024-03-27T10:31:23.840 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esas2r/esas2r.ko 2024-03-27T10:31:23.840 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esp_scsi.ko 2024-03-27T10:31:23.840 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe 2024-03-27T10:31:23.841 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe/fcoe.ko 2024-03-27T10:31:23.841 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-03-27T10:31:23.841 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fnic 2024-03-27T10:31:23.841 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fnic/fnic.ko 2024-03-27T10:31:23.841 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hpsa.ko 2024-03-27T10:31:23.841 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hptiop.ko 2024-03-27T10:31:23.841 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hv_storvsc.ko 2024-03-27T10:31:23.841 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/initio.ko 2024-03-27T10:31:23.841 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ipr.ko 2024-03-27T10:31:23.841 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ips.ko 2024-03-27T10:31:23.841 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/isci 2024-03-27T10:31:23.841 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/isci/isci.ko 2024-03-27T10:31:23.841 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-03-27T10:31:23.842 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/iscsi_tcp.ko 2024-03-27T10:31:23.842 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libfc 2024-03-27T10:31:23.842 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libfc/libfc.ko 2024-03-27T10:31:23.842 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libiscsi.ko 2024-03-27T10:31:23.842 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libiscsi_tcp.ko 2024-03-27T10:31:23.842 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libsas 2024-03-27T10:31:23.842 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libsas/libsas.ko 2024-03-27T10:31:23.842 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/lpfc 2024-03-27T10:31:23.842 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/lpfc/lpfc.ko 2024-03-27T10:31:23.842 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid 2024-03-27T10:31:23.842 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid.ko 2024-03-27T10:31:23.842 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-03-27T10:31:23.843 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-03-27T10:31:23.843 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-03-27T10:31:23.843 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mpt3sas 2024-03-27T10:31:23.843 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-03-27T10:31:23.843 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvsas 2024-03-27T10:31:23.843 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvsas/mvsas.ko 2024-03-27T10:31:23.843 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvumi.ko 2024-03-27T10:31:23.843 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pm8001 2024-03-27T10:31:23.843 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-03-27T10:31:23.843 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pmcraid.ko 2024-03-27T10:31:23.843 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla1280.ko 2024-03-27T10:31:23.843 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx 2024-03-27T10:31:23.843 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-03-27T10:31:23.844 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-03-27T10:31:23.844 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla4xxx 2024-03-27T10:31:23.844 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-03-27T10:31:23.844 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/raid_class.ko 2024-03-27T10:31:23.844 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_debug.ko 2024-03-27T10:31:23.844 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_fc.ko 2024-03-27T10:31:23.844 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-03-27T10:31:23.844 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_sas.ko 2024-03-27T10:31:23.844 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_spi.ko 2024-03-27T10:31:23.844 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_srp.ko 2024-03-27T10:31:23.844 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ses.ko 2024-03-27T10:31:23.845 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/st.ko 2024-03-27T10:31:23.845 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/stex.ko 2024-03-27T10:31:23.845 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/sym53c8xx_2 2024-03-27T10:31:23.845 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-03-27T10:31:23.845 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/virtio_scsi.ko 2024-03-27T10:31:23.845 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/vmw_pvscsi.ko 2024-03-27T10:31:23.845 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/wd719x.ko 2024-03-27T10:31:23.845 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ssb 2024-03-27T10:31:23.845 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ssb/ssb.ko 2024-03-27T10:31:23.845 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target 2024-03-27T10:31:23.845 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/iscsi 2024-03-27T10:31:23.846 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-03-27T10:31:23.846 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/loopback 2024-03-27T10:31:23.846 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/loopback/tcm_loop.ko 2024-03-27T10:31:23.846 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_file.ko 2024-03-27T10:31:23.846 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_iblock.ko 2024-03-27T10:31:23.846 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_mod.ko 2024-03-27T10:31:23.846 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_pscsi.ko 2024-03-27T10:31:23.846 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_user.ko 2024-03-27T10:31:23.846 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/tcm_fc 2024-03-27T10:31:23.846 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-03-27T10:31:23.847 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal 2024-03-27T10:31:23.847 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel 2024-03-27T10:31:23.847 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal 2024-03-27T10:31:23.847 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-03-27T10:31:23.847 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-03-27T10:31:23.847 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-03-27T10:31:23.847 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-03-27T10:31:23.847 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-03-27T10:31:23.847 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-03-27T10:31:23.847 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-03-27T10:31:23.847 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-03-27T10:31:23.848 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-03-27T10:31:23.848 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-03-27T10:31:23.848 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-03-27T10:31:23.848 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-03-27T10:31:23.848 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-03-27T10:31:23.848 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-03-27T10:31:23.848 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-03-27T10:31:23.848 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-03-27T10:31:23.848 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-03-27T10:31:23.848 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-03-27T10:31:23.848 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty 2024-03-27T10:31:23.848 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/ipwireless 2024-03-27T10:31:23.849 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-03-27T10:31:23.849 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/n_gsm.ko 2024-03-27T10:31:23.849 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/n_hdlc.ko 2024-03-27T10:31:23.849 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/nozomi.ko 2024-03-27T10:31:23.849 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial 2024-03-27T10:31:23.849 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/8250 2024-03-27T10:31:23.849 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-03-27T10:31:23.849 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/arc_uart.ko 2024-03-27T10:31:23.849 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/jsm 2024-03-27T10:31:23.849 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/jsm/jsm.ko 2024-03-27T10:31:23.849 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs 2024-03-27T10:31:23.849 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/core 2024-03-27T10:31:23.849 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/core/ufshcd-core.ko 2024-03-27T10:31:23.849 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/host 2024-03-27T10:31:23.850 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-03-27T10:31:23.850 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio 2024-03-27T10:31:23.850 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio.ko 2024-03-27T10:31:23.850 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_aec.ko 2024-03-27T10:31:23.850 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_cif.ko 2024-03-27T10:31:23.850 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_pci_generic.ko 2024-03-27T10:31:23.850 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_sercos3.ko 2024-03-27T10:31:23.850 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb 2024-03-27T10:31:23.850 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class 2024-03-27T10:31:23.850 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/cdc-acm.ko 2024-03-27T10:31:23.851 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/cdc-wdm.ko 2024-03-27T10:31:23.851 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/usblp.ko 2024-03-27T10:31:23.851 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/usbtmc.ko 2024-03-27T10:31:23.851 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host 2024-03-27T10:31:23.851 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host/sl811-hcd.ko 2024-03-27T10:31:23.851 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image 2024-03-27T10:31:23.851 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image/mdc800.ko 2024-03-27T10:31:23.851 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image/microtek.ko 2024-03-27T10:31:23.851 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc 2024-03-27T10:31:23.851 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/adutux.ko 2024-03-27T10:31:23.851 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/appledisplay.ko 2024-03-27T10:31:23.851 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/emi26.ko 2024-03-27T10:31:23.852 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/emi62.ko 2024-03-27T10:31:23.852 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/ezusb.ko 2024-03-27T10:31:23.852 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/idmouse.ko 2024-03-27T10:31:23.852 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/iowarrior.ko 2024-03-27T10:31:23.852 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/isight_firmware.ko 2024-03-27T10:31:23.852 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/ldusb.ko 2024-03-27T10:31:23.852 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/legousbtower.ko 2024-03-27T10:31:23.852 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/sisusbvga 2024-03-27T10:31:23.852 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-03-27T10:31:23.852 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/trancevibrator.ko 2024-03-27T10:31:23.852 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usb3503.ko 2024-03-27T10:31:23.852 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usblcd.ko 2024-03-27T10:31:23.853 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usbsevseg.ko 2024-03-27T10:31:23.853 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/uss720.ko 2024-03-27T10:31:23.853 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/yurex.ko 2024-03-27T10:31:23.853 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/phy 2024-03-27T10:31:23.853 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/phy/phy-generic.ko 2024-03-27T10:31:23.853 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial 2024-03-27T10:31:23.853 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/aircable.ko 2024-03-27T10:31:23.853 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ark3116.ko 2024-03-27T10:31:23.853 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/belkin_sa.ko 2024-03-27T10:31:23.853 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ch341.ko 2024-03-27T10:31:23.853 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cp210x.ko 2024-03-27T10:31:23.853 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cyberjack.ko 2024-03-27T10:31:23.854 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cypress_m8.ko 2024-03-27T10:31:23.854 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/digi_acceleport.ko 2024-03-27T10:31:23.854 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/empeg.ko 2024-03-27T10:31:23.854 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ftdi_sio.ko 2024-03-27T10:31:23.854 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/garmin_gps.ko 2024-03-27T10:31:23.854 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/io_edgeport.ko 2024-03-27T10:31:23.854 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/io_ti.ko 2024-03-27T10:31:23.854 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ipaq.ko 2024-03-27T10:31:23.854 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ipw.ko 2024-03-27T10:31:23.854 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ir-usb.ko 2024-03-27T10:31:23.854 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-03-27T10:31:23.854 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/keyspan.ko 2024-03-27T10:31:23.855 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/keyspan_pda.ko 2024-03-27T10:31:23.855 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/kl5kusb105.ko 2024-03-27T10:31:23.855 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/kobil_sct.ko 2024-03-27T10:31:23.855 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mct_u232.ko 2024-03-27T10:31:23.855 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mos7720.ko 2024-03-27T10:31:23.855 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mos7840.ko 2024-03-27T10:31:23.855 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/navman.ko 2024-03-27T10:31:23.855 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/omninet.ko 2024-03-27T10:31:23.855 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/opticon.ko 2024-03-27T10:31:23.855 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/option.ko 2024-03-27T10:31:23.855 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/oti6858.ko 2024-03-27T10:31:23.855 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/pl2303.ko 2024-03-27T10:31:23.855 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/qcaux.ko 2024-03-27T10:31:23.856 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/qcserial.ko 2024-03-27T10:31:23.856 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/quatech2.ko 2024-03-27T10:31:23.856 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/safe_serial.ko 2024-03-27T10:31:23.856 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/sierra.ko 2024-03-27T10:31:23.856 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/spcp8x5.ko 2024-03-27T10:31:23.856 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ssu100.ko 2024-03-27T10:31:23.856 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/symbolserial.ko 2024-03-27T10:31:23.856 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-03-27T10:31:23.856 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-03-27T10:31:23.856 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb_debug.ko 2024-03-27T10:31:23.856 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb_wwan.ko 2024-03-27T10:31:23.856 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/visor.ko 2024-03-27T10:31:23.856 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/whiteheat.ko 2024-03-27T10:31:23.857 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/xsens_mt.ko 2024-03-27T10:31:23.857 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage 2024-03-27T10:31:23.857 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/uas.ko 2024-03-27T10:31:23.857 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-alauda.ko 2024-03-27T10:31:23.857 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-cypress.ko 2024-03-27T10:31:23.857 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-datafab.ko 2024-03-27T10:31:23.857 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-03-27T10:31:23.857 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-freecom.ko 2024-03-27T10:31:23.857 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-isd200.ko 2024-03-27T10:31:23.857 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-03-27T10:31:23.857 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-karma.ko 2024-03-27T10:31:23.857 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-onetouch.ko 2024-03-27T10:31:23.858 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-realtek.ko 2024-03-27T10:31:23.858 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-sddr09.ko 2024-03-27T10:31:23.858 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-sddr55.ko 2024-03-27T10:31:23.858 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-usbat.ko 2024-03-27T10:31:23.858 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/usb-storage.ko 2024-03-27T10:31:23.858 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip 2024-03-27T10:31:23.858 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/usbip-core.ko 2024-03-27T10:31:23.858 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/usbip-host.ko 2024-03-27T10:31:23.858 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-03-27T10:31:23.858 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio 2024-03-27T10:31:23.858 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci 2024-03-27T10:31:23.859 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-03-27T10:31:23.859 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci/vfio-pci.ko 2024-03-27T10:31:23.859 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/vfio.ko 2024-03-27T10:31:23.859 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-03-27T10:31:23.859 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost 2024-03-27T10:31:23.859 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost.ko 2024-03-27T10:31:23.859 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_iotlb.ko 2024-03-27T10:31:23.859 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_net.ko 2024-03-27T10:31:23.859 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_scsi.ko 2024-03-27T10:31:23.859 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video 2024-03-27T10:31:23.859 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight 2024-03-27T10:31:23.860 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/apple_bl.ko 2024-03-27T10:31:23.860 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/lcd.ko 2024-03-27T10:31:23.860 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/platform_lcd.ko 2024-03-27T10:31:23.860 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio 2024-03-27T10:31:23.860 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio.ko 2024-03-27T10:31:23.860 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_balloon.ko 2024-03-27T10:31:23.860 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_mmio.ko 2024-03-27T10:31:23.860 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_pci.ko 2024-03-27T10:31:23.860 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-03-27T10:31:23.861 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-03-27T10:31:23.861 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_ring.ko 2024-03-27T10:31:23.861 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1 2024-03-27T10:31:23.861 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters 2024-03-27T10:31:23.861 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters/ds2482.ko 2024-03-27T10:31:23.861 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters/ds2490.ko 2024-03-27T10:31:23.861 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves 2024-03-27T10:31:23.861 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-03-27T10:31:23.861 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-03-27T10:31:23.861 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-03-27T10:31:23.861 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-03-27T10:31:23.862 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-03-27T10:31:23.862 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-03-27T10:31:23.862 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-03-27T10:31:23.862 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-03-27T10:31:23.862 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-03-27T10:31:23.862 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_smem.ko 2024-03-27T10:31:23.862 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_therm.ko 2024-03-27T10:31:23.862 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/wire.ko 2024-03-27T10:31:23.862 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog 2024-03-27T10:31:23.862 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/alim1535_wdt.ko 2024-03-27T10:31:23.862 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/alim7101_wdt.ko 2024-03-27T10:31:23.863 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/f71808e_wdt.ko 2024-03-27T10:31:23.863 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/hpwdt.ko 2024-03-27T10:31:23.863 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/i6300esb.ko 2024-03-27T10:31:23.863 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-03-27T10:31:23.863 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/iTCO_wdt.ko 2024-03-27T10:31:23.863 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ib700wdt.ko 2024-03-27T10:31:23.863 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ibmasr.ko 2024-03-27T10:31:23.863 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-03-27T10:31:23.863 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/it8712f_wdt.ko 2024-03-27T10:31:23.863 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/it87_wdt.ko 2024-03-27T10:31:23.864 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/machzwd.ko 2024-03-27T10:31:23.864 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/nv_tco.ko 2024-03-27T10:31:23.864 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/pcwd_pci.ko 2024-03-27T10:31:23.864 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/pcwd_usb.ko 2024-03-27T10:31:23.864 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-03-27T10:31:23.864 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sch311x_wdt.ko 2024-03-27T10:31:23.864 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/softdog.ko 2024-03-27T10:31:23.865 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sp5100_tco.ko 2024-03-27T10:31:23.865 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/via_wdt.ko 2024-03-27T10:31:23.865 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-03-27T10:31:23.865 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83877f_wdt.ko 2024-03-27T10:31:23.865 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83977f_wdt.ko 2024-03-27T10:31:23.865 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/wdt_pci.ko 2024-03-27T10:31:23.865 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/xen_wdt.ko 2024-03-27T10:31:23.865 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen 2024-03-27T10:31:23.866 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-acpi-processor.ko 2024-03-27T10:31:23.866 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-evtchn.ko 2024-03-27T10:31:23.866 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-gntalloc.ko 2024-03-27T10:31:23.866 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-gntdev.ko 2024-03-27T10:31:23.866 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-pciback 2024-03-27T10:31:23.866 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-03-27T10:31:23.866 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-privcmd.ko 2024-03-27T10:31:23.866 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xenfs 2024-03-27T10:31:23.867 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xenfs/xenfs.ko 2024-03-27T10:31:23.867 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs 2024-03-27T10:31:23.867 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/9p 2024-03-27T10:31:23.867 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/9p/9p.ko 2024-03-27T10:31:23.867 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/affs 2024-03-27T10:31:23.867 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/affs/affs.ko 2024-03-27T10:31:23.867 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/befs 2024-03-27T10:31:23.867 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/befs/befs.ko 2024-03-27T10:31:23.867 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/binfmt_misc.ko 2024-03-27T10:31:23.867 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/btrfs 2024-03-27T10:31:23.867 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/btrfs/btrfs.ko 2024-03-27T10:31:23.868 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cachefiles 2024-03-27T10:31:23.868 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cachefiles/cachefiles.ko 2024-03-27T10:31:23.868 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ceph 2024-03-27T10:31:23.868 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ceph/ceph.ko 2024-03-27T10:31:23.868 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/coda 2024-03-27T10:31:23.868 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/coda/coda.ko 2024-03-27T10:31:23.868 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cramfs 2024-03-27T10:31:23.869 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cramfs/cramfs.ko 2024-03-27T10:31:23.869 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/dlm 2024-03-27T10:31:23.869 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/dlm/dlm.ko 2024-03-27T10:31:23.869 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ecryptfs 2024-03-27T10:31:23.869 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ecryptfs/ecryptfs.ko 2024-03-27T10:31:23.869 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/f2fs 2024-03-27T10:31:23.869 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/f2fs/f2fs.ko 2024-03-27T10:31:23.869 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat 2024-03-27T10:31:23.869 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/fat.ko 2024-03-27T10:31:23.869 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/msdos.ko 2024-03-27T10:31:23.869 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/vfat.ko 2024-03-27T10:31:23.870 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse 2024-03-27T10:31:23.870 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse/cuse.ko 2024-03-27T10:31:23.870 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse/fuse.ko 2024-03-27T10:31:23.870 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/gfs2 2024-03-27T10:31:23.870 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/gfs2/gfs2.ko 2024-03-27T10:31:23.870 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfs 2024-03-27T10:31:23.870 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfs/hfs.ko 2024-03-27T10:31:23.870 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfsplus 2024-03-27T10:31:23.870 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfsplus/hfsplus.ko 2024-03-27T10:31:23.871 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/isofs 2024-03-27T10:31:23.871 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/isofs/isofs.ko 2024-03-27T10:31:23.871 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/jfs 2024-03-27T10:31:23.871 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/jfs/jfs.ko 2024-03-27T10:31:23.871 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/lockd 2024-03-27T10:31:23.871 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/lockd/lockd.ko 2024-03-27T10:31:23.871 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/minix 2024-03-27T10:31:23.871 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/minix/minix.ko 2024-03-27T10:31:23.871 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/netfs 2024-03-27T10:31:23.872 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/netfs/netfs.ko 2024-03-27T10:31:23.872 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs 2024-03-27T10:31:23.872 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/blocklayout 2024-03-27T10:31:23.872 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-03-27T10:31:23.872 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/filelayout 2024-03-27T10:31:23.872 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-03-27T10:31:23.872 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/flexfilelayout 2024-03-27T10:31:23.872 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-03-27T10:31:23.872 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfs.ko 2024-03-27T10:31:23.873 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfsv3.ko 2024-03-27T10:31:23.873 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfsv4.ko 2024-03-27T10:31:23.873 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common 2024-03-27T10:31:23.873 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common/grace.ko 2024-03-27T10:31:23.873 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common/nfs_acl.ko 2024-03-27T10:31:23.873 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfsd 2024-03-27T10:31:23.873 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfsd/nfsd.ko 2024-03-27T10:31:23.873 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nilfs2 2024-03-27T10:31:23.873 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nilfs2/nilfs2.ko 2024-03-27T10:31:23.874 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls 2024-03-27T10:31:23.874 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-celtic.ko 2024-03-27T10:31:23.874 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-centeuro.ko 2024-03-27T10:31:23.874 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-croatian.ko 2024-03-27T10:31:23.874 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-cyrillic.ko 2024-03-27T10:31:23.874 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-gaelic.ko 2024-03-27T10:31:23.874 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-greek.ko 2024-03-27T10:31:23.874 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-iceland.ko 2024-03-27T10:31:23.875 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-inuit.ko 2024-03-27T10:31:23.875 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-roman.ko 2024-03-27T10:31:23.875 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-romanian.ko 2024-03-27T10:31:23.875 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-turkish.ko 2024-03-27T10:31:23.875 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1250.ko 2024-03-27T10:31:23.875 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1251.ko 2024-03-27T10:31:23.875 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1255.ko 2024-03-27T10:31:23.876 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp737.ko 2024-03-27T10:31:23.876 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp775.ko 2024-03-27T10:31:23.876 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp850.ko 2024-03-27T10:31:23.876 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp852.ko 2024-03-27T10:31:23.876 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp855.ko 2024-03-27T10:31:23.876 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp857.ko 2024-03-27T10:31:23.876 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp860.ko 2024-03-27T10:31:23.877 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp861.ko 2024-03-27T10:31:23.877 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp862.ko 2024-03-27T10:31:23.877 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp863.ko 2024-03-27T10:31:23.877 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp864.ko 2024-03-27T10:31:23.877 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp865.ko 2024-03-27T10:31:23.877 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp866.ko 2024-03-27T10:31:23.877 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp869.ko 2024-03-27T10:31:23.877 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp874.ko 2024-03-27T10:31:23.877 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp932.ko 2024-03-27T10:31:23.878 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp936.ko 2024-03-27T10:31:23.878 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp949.ko 2024-03-27T10:31:23.878 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp950.ko 2024-03-27T10:31:23.878 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_euc-jp.ko 2024-03-27T10:31:23.878 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-1.ko 2024-03-27T10:31:23.878 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-13.ko 2024-03-27T10:31:23.878 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-14.ko 2024-03-27T10:31:23.878 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-15.ko 2024-03-27T10:31:23.878 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-2.ko 2024-03-27T10:31:23.879 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-3.ko 2024-03-27T10:31:23.879 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-4.ko 2024-03-27T10:31:23.879 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-5.ko 2024-03-27T10:31:23.879 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-6.ko 2024-03-27T10:31:23.879 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-7.ko 2024-03-27T10:31:23.879 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-9.ko 2024-03-27T10:31:23.879 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-r.ko 2024-03-27T10:31:23.879 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-ru.ko 2024-03-27T10:31:23.880 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-u.ko 2024-03-27T10:31:23.880 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_ucs2_utils.ko 2024-03-27T10:31:23.880 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_utf8.ko 2024-03-27T10:31:23.880 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2 2024-03-27T10:31:23.880 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/cluster 2024-03-27T10:31:23.880 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-03-27T10:31:23.880 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlm 2024-03-27T10:31:23.880 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-03-27T10:31:23.881 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlmfs 2024-03-27T10:31:23.881 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-03-27T10:31:23.881 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2.ko 2024-03-27T10:31:23.881 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-03-27T10:31:23.881 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-03-27T10:31:23.881 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-03-27T10:31:23.881 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/overlayfs 2024-03-27T10:31:23.881 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/overlayfs/overlay.ko 2024-03-27T10:31:23.881 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/pstore 2024-03-27T10:31:23.882 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/pstore/ramoops.ko 2024-03-27T10:31:23.882 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/reiserfs 2024-03-27T10:31:23.882 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/reiserfs/reiserfs.ko 2024-03-27T10:31:23.882 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/romfs 2024-03-27T10:31:23.882 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/romfs/romfs.ko 2024-03-27T10:31:23.882 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb 2024-03-27T10:31:23.882 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/client 2024-03-27T10:31:23.882 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/client/cifs.ko 2024-03-27T10:31:23.883 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common 2024-03-27T10:31:23.883 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common/cifs_arc4.ko 2024-03-27T10:31:23.883 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common/cifs_md4.ko 2024-03-27T10:31:23.883 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/squashfs 2024-03-27T10:31:23.883 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/squashfs/squashfs.ko 2024-03-27T10:31:23.883 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/sysv 2024-03-27T10:31:23.883 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/sysv/sysv.ko 2024-03-27T10:31:23.883 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/udf 2024-03-27T10:31:23.883 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/udf/udf.ko 2024-03-27T10:31:23.884 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ufs 2024-03-27T10:31:23.884 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ufs/ufs.ko 2024-03-27T10:31:23.884 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/xfs 2024-03-27T10:31:23.884 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/xfs/xfs.ko 2024-03-27T10:31:23.884 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel 2024-03-27T10:31:23.884 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/rcu 2024-03-27T10:31:23.884 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/rcu/rcutorture.ko 2024-03-27T10:31:23.884 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/torture.ko 2024-03-27T10:31:23.885 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/trace 2024-03-27T10:31:23.885 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-03-27T10:31:23.885 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib 2024-03-27T10:31:23.885 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/asn1_encoder.ko 2024-03-27T10:31:23.885 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crc-itu-t.ko 2024-03-27T10:31:23.885 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crc8.ko 2024-03-27T10:31:23.885 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto 2024-03-27T10:31:23.885 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto/libarc4.ko 2024-03-27T10:31:23.885 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto/libdes.ko 2024-03-27T10:31:23.885 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/libcrc32c.ko 2024-03-27T10:31:23.886 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lru_cache.ko 2024-03-27T10:31:23.886 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4 2024-03-27T10:31:23.886 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4/lz4_compress.ko 2024-03-27T10:31:23.886 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4/lz4hc_compress.ko 2024-03-27T10:31:23.886 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/math 2024-03-27T10:31:23.886 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/math/cordic.ko 2024-03-27T10:31:23.886 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/raid6 2024-03-27T10:31:23.886 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/raid6/raid6_pq.ko 2024-03-27T10:31:23.886 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/reed_solomon 2024-03-27T10:31:23.887 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/reed_solomon/reed_solomon.ko 2024-03-27T10:31:23.887 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_bm.ko 2024-03-27T10:31:23.887 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_fsm.ko 2024-03-27T10:31:23.887 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_kmp.ko 2024-03-27T10:31:23.887 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/zstd 2024-03-27T10:31:23.887 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/zstd/zstd_compress.ko 2024-03-27T10:31:23.887 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/mm 2024-03-27T10:31:23.888 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/mm/hwpoison-inject.ko 2024-03-27T10:31:23.888 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net 2024-03-27T10:31:23.888 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan 2024-03-27T10:31:23.888 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/6lowpan.ko 2024-03-27T10:31:23.888 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_dest.ko 2024-03-27T10:31:23.888 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_fragment.ko 2024-03-27T10:31:23.888 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_hop.ko 2024-03-27T10:31:23.888 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_ipv6.ko 2024-03-27T10:31:23.888 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_mobility.ko 2024-03-27T10:31:23.889 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_routing.ko 2024-03-27T10:31:23.889 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_udp.ko 2024-03-27T10:31:23.889 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802 2024-03-27T10:31:23.889 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/garp.ko 2024-03-27T10:31:23.889 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/mrp.ko 2024-03-27T10:31:23.889 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/p8022.ko 2024-03-27T10:31:23.889 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/psnap.ko 2024-03-27T10:31:23.889 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/stp.ko 2024-03-27T10:31:23.890 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/8021q 2024-03-27T10:31:23.890 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/8021q/8021q.ko 2024-03-27T10:31:23.890 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p 2024-03-27T10:31:23.890 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet.ko 2024-03-27T10:31:23.890 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_fd.ko 2024-03-27T10:31:23.890 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_rdma.ko 2024-03-27T10:31:23.890 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_virtio.ko 2024-03-27T10:31:23.891 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/batman-adv 2024-03-27T10:31:23.891 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/batman-adv/batman-adv.ko 2024-03-27T10:31:23.891 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge 2024-03-27T10:31:23.891 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/br_netfilter.ko 2024-03-27T10:31:23.891 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/bridge.ko 2024-03-27T10:31:23.891 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter 2024-03-27T10:31:23.891 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-03-27T10:31:23.891 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_among.ko 2024-03-27T10:31:23.891 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_arp.ko 2024-03-27T10:31:23.891 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-03-27T10:31:23.892 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-03-27T10:31:23.892 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_ip.ko 2024-03-27T10:31:23.892 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-03-27T10:31:23.892 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_limit.ko 2024-03-27T10:31:23.892 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_log.ko 2024-03-27T10:31:23.892 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_mark.ko 2024-03-27T10:31:23.892 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-03-27T10:31:23.892 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-03-27T10:31:23.892 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-03-27T10:31:23.893 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-03-27T10:31:23.893 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_snat.ko 2024-03-27T10:31:23.893 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_stp.ko 2024-03-27T10:31:23.893 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-03-27T10:31:23.893 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-03-27T10:31:23.893 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-03-27T10:31:23.893 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-03-27T10:31:23.893 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtables.ko 2024-03-27T10:31:23.893 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-03-27T10:31:23.894 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-03-27T10:31:23.894 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-03-27T10:31:23.894 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ceph 2024-03-27T10:31:23.894 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ceph/libceph.ko 2024-03-27T10:31:23.894 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core 2024-03-27T10:31:23.894 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core/failover.ko 2024-03-27T10:31:23.894 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core/pktgen.ko 2024-03-27T10:31:23.895 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp 2024-03-27T10:31:23.895 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp.ko 2024-03-27T10:31:23.895 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_diag.ko 2024-03-27T10:31:23.895 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_ipv4.ko 2024-03-27T10:31:23.895 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_ipv6.ko 2024-03-27T10:31:23.895 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dns_resolver 2024-03-27T10:31:23.895 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dns_resolver/dns_resolver.ko 2024-03-27T10:31:23.895 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa 2024-03-27T10:31:23.896 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/dsa_core.ko 2024-03-27T10:31:23.896 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_brcm.ko 2024-03-27T10:31:23.896 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_dsa.ko 2024-03-27T10:31:23.896 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_none.ko 2024-03-27T10:31:23.896 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_trailer.ko 2024-03-27T10:31:23.896 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154 2024-03-27T10:31:23.896 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/6lowpan 2024-03-27T10:31:23.896 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-03-27T10:31:23.896 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/ieee802154.ko 2024-03-27T10:31:23.896 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/ieee802154_socket.ko 2024-03-27T10:31:23.897 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4 2024-03-27T10:31:23.897 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ah4.ko 2024-03-27T10:31:23.897 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/esp4.ko 2024-03-27T10:31:23.897 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/fou.ko 2024-03-27T10:31:23.897 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/gre.ko 2024-03-27T10:31:23.897 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/inet_diag.ko 2024-03-27T10:31:23.897 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_gre.ko 2024-03-27T10:31:23.898 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_tunnel.ko 2024-03-27T10:31:23.898 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_vti.ko 2024-03-27T10:31:23.898 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ipcomp.ko 2024-03-27T10:31:23.898 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ipip.ko 2024-03-27T10:31:23.898 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter 2024-03-27T10:31:23.898 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arp_tables.ko 2024-03-27T10:31:23.898 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-03-27T10:31:23.898 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-03-27T10:31:23.898 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-03-27T10:31:23.899 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-03-27T10:31:23.899 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-03-27T10:31:23.899 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-03-27T10:31:23.899 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-03-27T10:31:23.899 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-03-27T10:31:23.899 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-03-27T10:31:23.899 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_security.ko 2024-03-27T10:31:23.899 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-03-27T10:31:23.900 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-03-27T10:31:23.900 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-03-27T10:31:23.900 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-03-27T10:31:23.900 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-03-27T10:31:23.900 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-03-27T10:31:23.900 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-03-27T10:31:23.900 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-03-27T10:31:23.900 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-03-27T10:31:23.900 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-03-27T10:31:23.901 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_bic.ko 2024-03-27T10:31:23.901 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_dctcp.ko 2024-03-27T10:31:23.901 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_diag.ko 2024-03-27T10:31:23.901 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_highspeed.ko 2024-03-27T10:31:23.901 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_htcp.ko 2024-03-27T10:31:23.901 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_hybla.ko 2024-03-27T10:31:23.901 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_illinois.ko 2024-03-27T10:31:23.901 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_lp.ko 2024-03-27T10:31:23.902 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_scalable.ko 2024-03-27T10:31:23.902 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_vegas.ko 2024-03-27T10:31:23.902 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_veno.ko 2024-03-27T10:31:23.902 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_westwood.ko 2024-03-27T10:31:23.902 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_yeah.ko 2024-03-27T10:31:23.902 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tunnel4.ko 2024-03-27T10:31:23.903 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/udp_diag.ko 2024-03-27T10:31:23.903 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/udp_tunnel.ko 2024-03-27T10:31:23.903 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/xfrm4_tunnel.ko 2024-03-27T10:31:23.903 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6 2024-03-27T10:31:23.903 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ah6.ko 2024-03-27T10:31:23.903 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/esp6.ko 2024-03-27T10:31:23.903 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/fou6.ko 2024-03-27T10:31:23.903 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_tunnel.ko 2024-03-27T10:31:23.903 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-03-27T10:31:23.904 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_vti.ko 2024-03-27T10:31:23.904 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ipcomp6.ko 2024-03-27T10:31:23.904 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter 2024-03-27T10:31:23.904 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-03-27T10:31:23.904 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-03-27T10:31:23.904 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-03-27T10:31:23.904 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-03-27T10:31:23.904 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-03-27T10:31:23.905 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-03-27T10:31:23.905 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-03-27T10:31:23.905 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-03-27T10:31:23.905 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-03-27T10:31:23.905 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-03-27T10:31:23.905 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-03-27T10:31:23.905 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-03-27T10:31:23.905 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-03-27T10:31:23.905 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-03-27T10:31:23.906 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-03-27T10:31:23.906 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-03-27T10:31:23.906 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-03-27T10:31:23.906 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-03-27T10:31:23.906 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-03-27T10:31:23.906 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-03-27T10:31:23.906 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-03-27T10:31:23.906 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-03-27T10:31:23.907 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-03-27T10:31:23.907 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-03-27T10:31:23.907 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/sit.ko 2024-03-27T10:31:23.907 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/tunnel6.ko 2024-03-27T10:31:23.907 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/xfrm6_tunnel.ko 2024-03-27T10:31:23.907 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/key 2024-03-27T10:31:23.907 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/key/af_key.ko 2024-03-27T10:31:23.907 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp 2024-03-27T10:31:23.907 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_core.ko 2024-03-27T10:31:23.908 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_debugfs.ko 2024-03-27T10:31:23.908 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_eth.ko 2024-03-27T10:31:23.908 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ip.ko 2024-03-27T10:31:23.908 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ip6.ko 2024-03-27T10:31:23.908 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_netlink.ko 2024-03-27T10:31:23.908 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ppp.ko 2024-03-27T10:31:23.908 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/llc 2024-03-27T10:31:23.908 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/llc/llc.ko 2024-03-27T10:31:23.908 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mac802154 2024-03-27T10:31:23.908 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mac802154/mac802154.ko 2024-03-27T10:31:23.909 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mpls 2024-03-27T10:31:23.909 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mpls/mpls_gso.ko 2024-03-27T10:31:23.909 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter 2024-03-27T10:31:23.909 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset 2024-03-27T10:31:23.909 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set.ko 2024-03-27T10:31:23.909 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-03-27T10:31:23.909 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-03-27T10:31:23.909 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-03-27T10:31:23.910 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-03-27T10:31:23.910 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-03-27T10:31:23.910 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-03-27T10:31:23.910 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-03-27T10:31:23.910 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-03-27T10:31:23.910 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-03-27T10:31:23.910 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-03-27T10:31:23.910 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-03-27T10:31:23.910 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-03-27T10:31:23.911 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-03-27T10:31:23.911 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-03-27T10:31:23.911 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-03-27T10:31:23.911 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs 2024-03-27T10:31:23.911 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs.ko 2024-03-27T10:31:23.911 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-03-27T10:31:23.911 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-03-27T10:31:23.912 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-03-27T10:31:23.912 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-03-27T10:31:23.912 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-03-27T10:31:23.912 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-03-27T10:31:23.912 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-03-27T10:31:23.912 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-03-27T10:31:23.912 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-03-27T10:31:23.912 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-03-27T10:31:23.912 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-03-27T10:31:23.912 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-03-27T10:31:23.913 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-03-27T10:31:23.913 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conncount.ko 2024-03-27T10:31:23.913 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack.ko 2024-03-27T10:31:23.913 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-03-27T10:31:23.913 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-03-27T10:31:23.913 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-03-27T10:31:23.913 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_h323.ko 2024-03-27T10:31:23.913 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_irc.ko 2024-03-27T10:31:23.914 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-03-27T10:31:23.914 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-03-27T10:31:23.914 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-03-27T10:31:23.914 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_sane.ko 2024-03-27T10:31:23.914 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_sip.ko 2024-03-27T10:31:23.914 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-03-27T10:31:23.914 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-03-27T10:31:23.915 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_dup_netdev.ko 2024-03-27T10:31:23.915 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_flow_table.ko 2024-03-27T10:31:23.915 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_flow_table_inet.ko 2024-03-27T10:31:23.915 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_log_syslog.ko 2024-03-27T10:31:23.915 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat.ko 2024-03-27T10:31:23.915 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_amanda.ko 2024-03-27T10:31:23.915 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_ftp.ko 2024-03-27T10:31:23.915 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_irc.ko 2024-03-27T10:31:23.915 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_sip.ko 2024-03-27T10:31:23.915 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_tftp.ko 2024-03-27T10:31:23.916 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_synproxy_core.ko 2024-03-27T10:31:23.916 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_tables.ko 2024-03-27T10:31:23.916 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink.ko 2024-03-27T10:31:23.916 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_acct.ko 2024-03-27T10:31:23.916 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_log.ko 2024-03-27T10:31:23.916 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_osf.ko 2024-03-27T10:31:23.916 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_queue.ko 2024-03-27T10:31:23.917 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_chain_nat.ko 2024-03-27T10:31:23.917 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_compat.ko 2024-03-27T10:31:23.917 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_ct.ko 2024-03-27T10:31:23.917 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_dup_netdev.ko 2024-03-27T10:31:23.917 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_fib.ko 2024-03-27T10:31:23.917 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_fwd_netdev.ko 2024-03-27T10:31:23.918 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_hash.ko 2024-03-27T10:31:23.918 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_limit.ko 2024-03-27T10:31:23.918 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_log.ko 2024-03-27T10:31:23.918 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_masq.ko 2024-03-27T10:31:23.918 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_nat.ko 2024-03-27T10:31:23.918 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_numgen.ko 2024-03-27T10:31:23.918 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_queue.ko 2024-03-27T10:31:23.919 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_quota.ko 2024-03-27T10:31:23.919 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_redir.ko 2024-03-27T10:31:23.919 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_reject.ko 2024-03-27T10:31:23.919 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_reject_inet.ko 2024-03-27T10:31:23.919 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_socket.ko 2024-03-27T10:31:23.919 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_tunnel.ko 2024-03-27T10:31:23.920 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_AUDIT.ko 2024-03-27T10:31:23.920 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CHECKSUM.ko 2024-03-27T10:31:23.920 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CLASSIFY.ko 2024-03-27T10:31:23.920 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-03-27T10:31:23.920 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CT.ko 2024-03-27T10:31:23.920 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_DSCP.ko 2024-03-27T10:31:23.920 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_HL.ko 2024-03-27T10:31:23.920 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_HMARK.ko 2024-03-27T10:31:23.921 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_IDLETIMER.ko 2024-03-27T10:31:23.921 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_LED.ko 2024-03-27T10:31:23.921 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_LOG.ko 2024-03-27T10:31:23.921 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_MASQUERADE.ko 2024-03-27T10:31:23.921 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NETMAP.ko 2024-03-27T10:31:23.921 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NFLOG.ko 2024-03-27T10:31:23.921 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NFQUEUE.ko 2024-03-27T10:31:23.922 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_RATEEST.ko 2024-03-27T10:31:23.922 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_REDIRECT.ko 2024-03-27T10:31:23.922 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_SECMARK.ko 2024-03-27T10:31:23.922 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TCPMSS.ko 2024-03-27T10:31:23.922 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-03-27T10:31:23.922 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TEE.ko 2024-03-27T10:31:23.922 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TPROXY.ko 2024-03-27T10:31:23.922 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TRACE.ko 2024-03-27T10:31:23.923 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_addrtype.ko 2024-03-27T10:31:23.923 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_bpf.ko 2024-03-27T10:31:23.923 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cgroup.ko 2024-03-27T10:31:23.923 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cluster.ko 2024-03-27T10:31:23.923 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_comment.ko 2024-03-27T10:31:23.923 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connbytes.ko 2024-03-27T10:31:23.924 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connlabel.ko 2024-03-27T10:31:23.924 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connlimit.ko 2024-03-27T10:31:23.924 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connmark.ko 2024-03-27T10:31:23.924 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_conntrack.ko 2024-03-27T10:31:23.924 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cpu.ko 2024-03-27T10:31:23.924 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_dccp.ko 2024-03-27T10:31:23.924 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_devgroup.ko 2024-03-27T10:31:23.925 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_dscp.ko 2024-03-27T10:31:23.925 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ecn.ko 2024-03-27T10:31:23.925 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_esp.ko 2024-03-27T10:31:23.925 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_hashlimit.ko 2024-03-27T10:31:23.925 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_helper.ko 2024-03-27T10:31:23.925 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_hl.ko 2024-03-27T10:31:23.925 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ipcomp.ko 2024-03-27T10:31:23.925 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_iprange.ko 2024-03-27T10:31:23.925 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ipvs.ko 2024-03-27T10:31:23.926 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_l2tp.ko 2024-03-27T10:31:23.926 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_length.ko 2024-03-27T10:31:23.926 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_limit.ko 2024-03-27T10:31:23.926 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_mac.ko 2024-03-27T10:31:23.926 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_mark.ko 2024-03-27T10:31:23.926 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_multiport.ko 2024-03-27T10:31:23.926 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_nat.ko 2024-03-27T10:31:23.926 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_nfacct.ko 2024-03-27T10:31:23.927 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_osf.ko 2024-03-27T10:31:23.927 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_owner.ko 2024-03-27T10:31:23.927 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_physdev.ko 2024-03-27T10:31:23.927 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_pkttype.ko 2024-03-27T10:31:23.927 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_policy.ko 2024-03-27T10:31:23.927 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_quota.ko 2024-03-27T10:31:23.928 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_rateest.ko 2024-03-27T10:31:23.928 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_realm.ko 2024-03-27T10:31:23.928 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_recent.ko 2024-03-27T10:31:23.928 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_sctp.ko 2024-03-27T10:31:23.928 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_set.ko 2024-03-27T10:31:23.928 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_socket.ko 2024-03-27T10:31:23.928 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_state.ko 2024-03-27T10:31:23.928 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_statistic.ko 2024-03-27T10:31:23.928 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_string.ko 2024-03-27T10:31:23.929 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_tcpmss.ko 2024-03-27T10:31:23.929 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_time.ko 2024-03-27T10:31:23.929 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_u32.ko 2024-03-27T10:31:23.929 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netlink 2024-03-27T10:31:23.929 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netlink/netlink_diag.ko 2024-03-27T10:31:23.929 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/nsh 2024-03-27T10:31:23.929 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/nsh/nsh.ko 2024-03-27T10:31:23.929 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch 2024-03-27T10:31:23.930 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/openvswitch.ko 2024-03-27T10:31:23.930 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-geneve.ko 2024-03-27T10:31:23.930 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-gre.ko 2024-03-27T10:31:23.930 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-vxlan.ko 2024-03-27T10:31:23.930 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/packet 2024-03-27T10:31:23.930 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/packet/af_packet_diag.ko 2024-03-27T10:31:23.931 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds 2024-03-27T10:31:23.931 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds.ko 2024-03-27T10:31:23.931 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds_rdma.ko 2024-03-27T10:31:23.931 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds_tcp.ko 2024-03-27T10:31:23.931 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched 2024-03-27T10:31:23.931 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_csum.ko 2024-03-27T10:31:23.931 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_gact.ko 2024-03-27T10:31:23.931 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_mirred.ko 2024-03-27T10:31:23.931 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_nat.ko 2024-03-27T10:31:23.932 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_pedit.ko 2024-03-27T10:31:23.932 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_police.ko 2024-03-27T10:31:23.932 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_simple.ko 2024-03-27T10:31:23.932 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_skbedit.ko 2024-03-27T10:31:23.932 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_vlan.ko 2024-03-27T10:31:23.932 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_basic.ko 2024-03-27T10:31:23.932 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_bpf.ko 2024-03-27T10:31:23.932 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_flow.ko 2024-03-27T10:31:23.932 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_fw.ko 2024-03-27T10:31:23.932 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_route.ko 2024-03-27T10:31:23.933 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_u32.ko 2024-03-27T10:31:23.933 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_cmp.ko 2024-03-27T10:31:23.933 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_ipset.ko 2024-03-27T10:31:23.933 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_meta.ko 2024-03-27T10:31:23.933 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_nbyte.ko 2024-03-27T10:31:23.933 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_text.ko 2024-03-27T10:31:23.933 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_u32.ko 2024-03-27T10:31:23.933 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_choke.ko 2024-03-27T10:31:23.934 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_codel.ko 2024-03-27T10:31:23.934 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_drr.ko 2024-03-27T10:31:23.934 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_fq.ko 2024-03-27T10:31:23.934 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_gred.ko 2024-03-27T10:31:23.934 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_hfsc.ko 2024-03-27T10:31:23.934 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_hhf.ko 2024-03-27T10:31:23.934 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_htb.ko 2024-03-27T10:31:23.934 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_ingress.ko 2024-03-27T10:31:23.934 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_mqprio.ko 2024-03-27T10:31:23.935 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_mqprio_lib.ko 2024-03-27T10:31:23.935 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_multiq.ko 2024-03-27T10:31:23.935 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_netem.ko 2024-03-27T10:31:23.935 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_pie.ko 2024-03-27T10:31:23.935 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_plug.ko 2024-03-27T10:31:23.935 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_prio.ko 2024-03-27T10:31:23.935 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_qfq.ko 2024-03-27T10:31:23.936 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_red.ko 2024-03-27T10:31:23.936 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_sfb.ko 2024-03-27T10:31:23.936 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_sfq.ko 2024-03-27T10:31:23.936 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_tbf.ko 2024-03-27T10:31:23.936 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_teql.ko 2024-03-27T10:31:23.936 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp 2024-03-27T10:31:23.936 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp/sctp.ko 2024-03-27T10:31:23.936 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp/sctp_diag.ko 2024-03-27T10:31:23.936 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc 2024-03-27T10:31:23.937 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss 2024-03-27T10:31:23.937 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-03-27T10:31:23.937 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-03-27T10:31:23.937 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/sunrpc.ko 2024-03-27T10:31:23.937 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/xprtrdma 2024-03-27T10:31:23.937 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-03-27T10:31:23.937 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc 2024-03-27T10:31:23.937 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc/diag.ko 2024-03-27T10:31:23.937 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc/tipc.ko 2024-03-27T10:31:23.938 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/unix 2024-03-27T10:31:23.938 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/unix/unix_diag.ko 2024-03-27T10:31:23.938 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock 2024-03-27T10:31:23.938 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-03-27T10:31:23.938 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-03-27T10:31:23.938 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock.ko 2024-03-27T10:31:23.938 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock_diag.ko 2024-03-27T10:31:23.938 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock_loopback.ko 2024-03-27T10:31:23.938 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/xfrm 2024-03-27T10:31:23.938 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/xfrm/xfrm_ipcomp.ko 2024-03-27T10:31:23.939 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security 2024-03-27T10:31:23.939 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys 2024-03-27T10:31:23.939 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/encrypted-keys 2024-03-27T10:31:23.939 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-03-27T10:31:23.939 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/trusted-keys 2024-03-27T10:31:23.939 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/trusted-keys/trusted.ko 2024-03-27T10:31:23.939 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/virt 2024-03-27T10:31:23.939 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/virt/lib 2024-03-27T10:31:23.939 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/virt/lib/irqbypass.ko 2024-03-27T10:31:23.939 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.alias 2024-03-27T10:31:23.939 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.alias.bin 2024-03-27T10:31:23.939 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin 2024-03-27T10:31:23.940 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.alias.bin 2024-03-27T10:31:23.940 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.bin 2024-03-27T10:31:23.940 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.modinfo 2024-03-27T10:31:23.940 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.dep 2024-03-27T10:31:23.940 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.dep.bin 2024-03-27T10:31:23.940 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.devname 2024-03-27T10:31:23.940 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.order 2024-03-27T10:31:23.940 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.softdep 2024-03-27T10:31:23.940 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.symbols 2024-03-27T10:31:23.941 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.symbols.bin 2024-03-27T10:31:23.941 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.8.0-g1df0c7e8a235/vmlinuz 2024-03-27T10:31:23.945 DEBUG:teuthology.task.kernel:get_image_version: 6.8.0-g1df0c7e8a235 2024-03-27T10:31:23.948 DEBUG:teuthology.orchestra.run.smithi097:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-03-27T10:31:23.971 INFO:teuthology.orchestra.run.smithi097.stdout:postinstall scriptlet (using /bin/sh): 2024-03-27T10:31:23.971 INFO:teuthology.orchestra.run.smithi097.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-03-27T10:31:23.971 INFO:teuthology.orchestra.run.smithi097.stdout: /usr/bin/kernel-install add 6.8.0-g1df0c7e8a235 /lib/modules/6.8.0-g1df0c7e8a235/vmlinuz 2024-03-27T10:31:23.971 INFO:teuthology.orchestra.run.smithi097.stdout:fi 2024-03-27T10:31:23.971 INFO:teuthology.orchestra.run.smithi097.stdout:for file in vmlinuz System.map config; do 2024-03-27T10:31:23.971 INFO:teuthology.orchestra.run.smithi097.stdout: if ! cmp --silent "/lib/modules/6.8.0-g1df0c7e8a235/${file}" "/boot/${file}-6.8.0-g1df0c7e8a235"; then 2024-03-27T10:31:23.971 INFO:teuthology.orchestra.run.smithi097.stdout: cp "/lib/modules/6.8.0-g1df0c7e8a235/${file}" "/boot/${file}-6.8.0-g1df0c7e8a235" 2024-03-27T10:31:23.972 INFO:teuthology.orchestra.run.smithi097.stdout: fi 2024-03-27T10:31:23.972 INFO:teuthology.orchestra.run.smithi097.stdout:done 2024-03-27T10:31:23.972 INFO:teuthology.orchestra.run.smithi097.stdout:preuninstall scriptlet (using /bin/sh): 2024-03-27T10:31:23.972 INFO:teuthology.orchestra.run.smithi097.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2024-03-27T10:31:23.972 INFO:teuthology.orchestra.run.smithi097.stdout:new-kernel-pkg --remove 6.8.0-g1df0c7e8a235 --rminitrd --initrdfile=/boot/initramfs-6.8.0-g1df0c7e8a235.img 2024-03-27T10:31:23.972 INFO:teuthology.orchestra.run.smithi097.stdout:elif [ -x /usr/bin/kernel-install ]; then 2024-03-27T10:31:23.972 INFO:teuthology.orchestra.run.smithi097.stdout:kernel-install remove 6.8.0-g1df0c7e8a235 2024-03-27T10:31:23.972 INFO:teuthology.orchestra.run.smithi097.stdout:fi 2024-03-27T10:31:23.972 INFO:teuthology.orchestra.run.smithi097.stdout:postuninstall scriptlet (using /bin/sh): 2024-03-27T10:31:23.972 INFO:teuthology.orchestra.run.smithi097.stdout:if [ -x /sbin/update-bootloader ]; then 2024-03-27T10:31:23.972 INFO:teuthology.orchestra.run.smithi097.stdout:/sbin/update-bootloader --remove 6.8.0-g1df0c7e8a235 2024-03-27T10:31:23.972 INFO:teuthology.orchestra.run.smithi097.stdout:fi 2024-03-27T10:31:23.974 DEBUG:teuthology.orchestra.run.smithi097:> sudo rpm -qi grub2-tools 2024-03-27T10:31:24.049 INFO:teuthology.orchestra.run.smithi097.stdout:Name : grub2-tools 2024-03-27T10:31:24.049 INFO:teuthology.orchestra.run.smithi097.stdout:Epoch : 1 2024-03-27T10:31:24.049 INFO:teuthology.orchestra.run.smithi097.stdout:Version : 2.06 2024-03-27T10:31:24.050 INFO:teuthology.orchestra.run.smithi097.stdout:Release : 68.el9 2024-03-27T10:31:24.050 INFO:teuthology.orchestra.run.smithi097.stdout:Architecture: x86_64 2024-03-27T10:31:24.050 INFO:teuthology.orchestra.run.smithi097.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-27T10:31:24.050 INFO:teuthology.orchestra.run.smithi097.stdout:Group : Unspecified 2024-03-27T10:31:24.050 INFO:teuthology.orchestra.run.smithi097.stdout:Size : 8264254 2024-03-27T10:31:24.050 INFO:teuthology.orchestra.run.smithi097.stdout:License : GPLv3+ 2024-03-27T10:31:24.050 INFO:teuthology.orchestra.run.smithi097.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-27T10:31:24.050 INFO:teuthology.orchestra.run.smithi097.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-27T10:31:24.051 INFO:teuthology.orchestra.run.smithi097.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-27T10:31:24.051 INFO:teuthology.orchestra.run.smithi097.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-27T10:31:24.051 INFO:teuthology.orchestra.run.smithi097.stdout:Packager : builder@centos.org 2024-03-27T10:31:24.051 INFO:teuthology.orchestra.run.smithi097.stdout:Vendor : CentOS 2024-03-27T10:31:24.051 INFO:teuthology.orchestra.run.smithi097.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-27T10:31:24.051 INFO:teuthology.orchestra.run.smithi097.stdout:Summary : Support tools for GRUB. 2024-03-27T10:31:24.051 INFO:teuthology.orchestra.run.smithi097.stdout:Description : 2024-03-27T10:31:24.051 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T10:31:24.051 INFO:teuthology.orchestra.run.smithi097.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-27T10:31:24.051 INFO:teuthology.orchestra.run.smithi097.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-27T10:31:24.052 INFO:teuthology.orchestra.run.smithi097.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-27T10:31:24.052 INFO:teuthology.orchestra.run.smithi097.stdout:hardware devices. 2024-03-27T10:31:24.052 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T10:31:24.052 INFO:teuthology.orchestra.run.smithi097.stdout:This subpackage provides tools for support of all platforms. 2024-03-27T10:31:24.053 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-27T10:31:24.053 INFO:teuthology.task.kernel:Updating grub on smithi097 to boot 6.8.0-g1df0c7e8a235 2024-03-27T10:31:24.053 DEBUG:teuthology.orchestra.run.smithi097:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-27T10:31:24.610 INFO:teuthology.orchestra.run.smithi097.stderr:Generating grub configuration file ... 2024-03-27T10:31:25.982 INFO:teuthology.orchestra.run.smithi097.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-27T10:31:26.012 INFO:teuthology.orchestra.run.smithi097.stderr:done 2024-03-27T10:31:26.015 DEBUG:teuthology.orchestra.run.smithi097:> mktemp 2024-03-27T10:31:26.030 INFO:teuthology.orchestra.run.smithi097.stdout:/tmp/tmp.tPBakzJrf3 2024-03-27T10:31:26.030 DEBUG:teuthology.orchestra.run.smithi097:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.tPBakzJrf3 2024-03-27T10:31:26.107 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 0666 /tmp/tmp.tPBakzJrf3 2024-03-27T10:31:26.244 DEBUG:teuthology.orchestra.remote:smithi097:/tmp/tmp.tPBakzJrf3 is 6KB 2024-03-27T10:31:26.293 DEBUG:teuthology.orchestra.run.smithi097:> rm -fr /tmp/tmp.tPBakzJrf3 2024-03-27T10:31:26.307 DEBUG:teuthology.orchestra.run.smithi097:> sudo /bin/ls /boot/loader/entries || true 2024-03-27T10:31:26.372 INFO:teuthology.orchestra.run.smithi097.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-27T10:31:26.372 INFO:teuthology.orchestra.run.smithi097.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-27T10:31:26.372 INFO:teuthology.orchestra.run.smithi097.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-27T10:31:26.372 INFO:teuthology.orchestra.run.smithi097.stdout:15949e3c88704300882482eddfd7baae-6.8.0-g1df0c7e8a235.conf 2024-03-27T10:31:26.373 DEBUG:teuthology.orchestra.run.smithi097:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.8.0-g1df0c7e8a235 2024-03-27T10:31:26.548 DEBUG:teuthology.orchestra.run.smithi097:> sudo shutdown -r now 2024-03-27T10:31:33.934 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-03-27T10:31:33.935 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi114.front.sepia.ceph.com' 2024-03-27T10:31:33.935 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-27T10:31:52.355 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.114 2024-03-27T10:31:52.648 INFO:teuthology.task.kernel:Checking client mon.c for new kernel version... 2024-03-27T10:31:52.648 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi177.front.sepia.ceph.com' 2024-03-27T10:31:52.649 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi177.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-27T10:31:56.554 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-03-27T10:31:56.555 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi097.front.sepia.ceph.com' 2024-03-27T10:31:56.556 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-27T10:32:01.358 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi114.front.sepia.ceph.com' 2024-03-27T10:32:01.359 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-27T10:32:11.038 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.177 2024-03-27T10:32:20.046 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi177.front.sepia.ceph.com' 2024-03-27T10:32:20.047 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi177.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-27T10:32:23.102 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.177 2024-03-27T10:32:35.114 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi177.front.sepia.ceph.com' 2024-03-27T10:32:35.115 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi177.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-27T10:32:53.534 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.177 2024-03-27T10:32:56.561 DEBUG:teuthology.orchestra.remote:timed out 2024-03-27T10:33:01.364 DEBUG:teuthology.orchestra.remote:timed out 2024-03-27T10:33:05.566 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi097.front.sepia.ceph.com' 2024-03-27T10:33:05.567 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-27T10:33:08.539 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi177.front.sepia.ceph.com' 2024-03-27T10:33:08.539 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi177.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-27T10:33:08.930 DEBUG:teuthology.orchestra.run.smithi177:> true 2024-03-27T10:33:08.996 DEBUG:teuthology.orchestra.run.smithi097:> true 2024-03-27T10:33:09.589 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi177.front.sepia.ceph.com' 2024-03-27T10:33:09.590 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "6.8.0-g1df0c7e8a235"... 2024-03-27T10:33:09.590 DEBUG:teuthology.orchestra.run.smithi177:> uname -r 2024-03-27T10:33:09.643 INFO:teuthology.orchestra.run.smithi177.stdout:6.8.0-g1df0c7e8a235 2024-03-27T10:33:09.643 DEBUG:teuthology.task.kernel:current kernel version is 6.8.0-g1df0c7e8a235 vs 6.8.0-g1df0c7e8a235 2024-03-27T10:33:09.643 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-27T10:33:09.643 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-27T10:33:09.671 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi097.front.sepia.ceph.com' 2024-03-27T10:33:09.671 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "6.8.0-g1df0c7e8a235"... 2024-03-27T10:33:09.671 DEBUG:teuthology.orchestra.run.smithi097:> uname -r 2024-03-27T10:33:09.708 INFO:teuthology.orchestra.run.smithi097.stdout:6.8.0-g1df0c7e8a235 2024-03-27T10:33:09.708 DEBUG:teuthology.task.kernel:current kernel version is 6.8.0-g1df0c7e8a235 vs 6.8.0-g1df0c7e8a235 2024-03-27T10:33:09.708 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-27T10:33:09.708 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-27T10:33:10.644 INFO:teuthology.task.kernel:Enabling kdb on mon.c... 2024-03-27T10:33:10.644 DEBUG:teuthology.orchestra.run.smithi177:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-27T10:33:10.709 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-03-27T10:33:10.709 DEBUG:teuthology.orchestra.run.smithi097:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-27T10:33:10.775 INFO:teuthology.orchestra.run.smithi177.stdout:ttyS1 2024-03-27T10:33:10.821 DEBUG:teuthology.parallel:result is None 2024-03-27T10:33:10.825 INFO:teuthology.orchestra.run.smithi097.stdout:ttyS1 2024-03-27T10:33:10.878 DEBUG:teuthology.parallel:result is None 2024-03-27T10:33:13.365 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi114.front.sepia.ceph.com' 2024-03-27T10:33:13.366 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-27T10:33:13.866 DEBUG:teuthology.orchestra.run.smithi114:> true 2024-03-27T10:33:14.534 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi114.front.sepia.ceph.com' 2024-03-27T10:33:14.534 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "6.8.0-g1df0c7e8a235"... 2024-03-27T10:33:14.535 DEBUG:teuthology.orchestra.run.smithi114:> uname -r 2024-03-27T10:33:14.585 INFO:teuthology.orchestra.run.smithi114.stdout:6.8.0-g1df0c7e8a235 2024-03-27T10:33:14.585 DEBUG:teuthology.task.kernel:current kernel version is 6.8.0-g1df0c7e8a235 vs 6.8.0-g1df0c7e8a235 2024-03-27T10:33:14.585 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-27T10:33:14.586 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-27T10:33:15.586 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-03-27T10:33:15.586 DEBUG:teuthology.orchestra.run.smithi114:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-27T10:33:15.746 INFO:teuthology.orchestra.run.smithi114.stdout:ttyS1 2024-03-27T10:33:15.802 DEBUG:teuthology.parallel:result is None 2024-03-27T10:33:15.803 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-27T10:33:15.811 INFO:teuthology.task.internal:Creating test directory... 2024-03-27T10:33:15.812 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-27T10:33:15.817 DEBUG:teuthology.orchestra.run.smithi114:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-27T10:33:15.825 DEBUG:teuthology.orchestra.run.smithi177:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-27T10:33:15.874 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-27T10:33:15.881 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-27T10:33:15.890 INFO:teuthology.task.internal:Creating archive directory... 2024-03-27T10:33:15.891 DEBUG:teuthology.orchestra.run.smithi097:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-27T10:33:15.897 DEBUG:teuthology.orchestra.run.smithi114:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-27T10:33:15.905 DEBUG:teuthology.orchestra.run.smithi177:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-27T10:33:15.976 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-27T10:33:15.983 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-27T10:33:15.983 DEBUG:teuthology.orchestra.run.smithi097:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-27T10:33:15.987 DEBUG:teuthology.orchestra.run.smithi114:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-27T10:33:15.994 DEBUG:teuthology.orchestra.run.smithi177:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-27T10:33:16.070 INFO:teuthology.orchestra.run.smithi097.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-27T10:33:16.072 INFO:teuthology.orchestra.run.smithi177.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-27T10:33:16.087 INFO:teuthology.orchestra.run.smithi114.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-27T10:33:16.103 INFO:teuthology.orchestra.run.smithi097.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-27T10:33:16.105 INFO:teuthology.orchestra.run.smithi177.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-27T10:33:16.125 INFO:teuthology.orchestra.run.smithi114.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-27T10:33:16.129 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-27T10:33:16.136 INFO:teuthology.task.internal:Configuring sudo... 2024-03-27T10:33:16.136 DEBUG:teuthology.orchestra.run.smithi097:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-27T10:33:16.140 DEBUG:teuthology.orchestra.run.smithi114:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-27T10:33:16.147 DEBUG:teuthology.orchestra.run.smithi177:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-27T10:33:16.234 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-27T10:33:16.243 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-27T10:33:16.243 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-27T10:33:16.247 DEBUG:teuthology.orchestra.run.smithi114:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-27T10:33:16.254 DEBUG:teuthology.orchestra.run.smithi177:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-27T10:33:16.301 DEBUG:teuthology.orchestra.run.smithi097:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-27T10:33:16.380 DEBUG:teuthology.orchestra.run.smithi097:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-27T10:33:16.481 DEBUG:teuthology.orchestra.run.smithi097:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-27T10:33:16.545 DEBUG:teuthology.orchestra.run.smithi097:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-27T10:33:16.634 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-27T10:33:16.634 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-27T10:33:16.703 DEBUG:teuthology.orchestra.run.smithi114:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-27T10:33:16.787 DEBUG:teuthology.orchestra.run.smithi114:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-27T10:33:16.888 DEBUG:teuthology.orchestra.run.smithi114:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-27T10:33:16.972 DEBUG:teuthology.orchestra.run.smithi114:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-27T10:33:17.070 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-03-27T10:33:17.070 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-27T10:33:17.164 DEBUG:teuthology.orchestra.run.smithi177:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-27T10:33:17.234 DEBUG:teuthology.orchestra.run.smithi177:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-27T10:33:17.312 DEBUG:teuthology.orchestra.run.smithi177:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-27T10:33:17.379 DEBUG:teuthology.orchestra.run.smithi177:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-27T10:33:17.467 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-03-27T10:33:17.467 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-27T10:33:17.535 DEBUG:teuthology.orchestra.run.smithi097:> sudo service rsyslog restart 2024-03-27T10:33:17.541 DEBUG:teuthology.orchestra.run.smithi114:> sudo service rsyslog restart 2024-03-27T10:33:17.548 DEBUG:teuthology.orchestra.run.smithi177:> sudo service rsyslog restart 2024-03-27T10:33:17.636 INFO:teuthology.orchestra.run.smithi097.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-27T10:33:17.666 INFO:teuthology.orchestra.run.smithi114.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-27T10:33:17.700 INFO:teuthology.orchestra.run.smithi177.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-27T10:33:18.214 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-27T10:33:18.222 INFO:teuthology.task.internal:Starting timer... 2024-03-27T10:33:18.223 INFO:teuthology.run_tasks:Running task pcp... 2024-03-27T10:33:18.236 INFO:teuthology.run_tasks:Running task selinux... 2024-03-27T10:33:18.244 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-03-27T10:33:18.244 DEBUG:teuthology.orchestra.run.smithi097:> sudo service auditd rotate 2024-03-27T10:33:18.360 INFO:teuthology.orchestra.run.smithi097.stdout:Rotating logs: 2024-03-27T10:33:18.365 DEBUG:teuthology.orchestra.run.smithi114:> sudo service auditd rotate 2024-03-27T10:33:18.494 INFO:teuthology.orchestra.run.smithi114.stdout:Rotating logs: 2024-03-27T10:33:18.499 DEBUG:teuthology.orchestra.run.smithi177:> sudo service auditd rotate 2024-03-27T10:33:18.610 INFO:teuthology.orchestra.run.smithi177.stdout:Rotating logs: 2024-03-27T10:33:18.614 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-27T10:33:18.614 DEBUG:teuthology.orchestra.run.smithi097:> /usr/sbin/getenforce 2024-03-27T10:33:18.666 INFO:teuthology.orchestra.run.smithi097.stdout:Permissive 2024-03-27T10:33:18.667 DEBUG:teuthology.orchestra.run.smithi114:> /usr/sbin/getenforce 2024-03-27T10:33:18.718 INFO:teuthology.orchestra.run.smithi114.stdout:Permissive 2024-03-27T10:33:18.719 DEBUG:teuthology.orchestra.run.smithi177:> /usr/sbin/getenforce 2024-03-27T10:33:18.762 INFO:teuthology.orchestra.run.smithi177.stdout:Permissive 2024-03-27T10:33:18.762 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi097.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi114.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi177.front.sepia.ceph.com': 'permissive'} 2024-03-27T10:33:18.762 DEBUG:teuthology.orchestra.run.smithi097:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-27T10:33:18.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-27T10:33:18.847 DEBUG:teuthology.orchestra.run.smithi114:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-27T10:33:18.935 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-27T10:33:18.935 DEBUG:teuthology.orchestra.run.smithi177:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-27T10:33:19.006 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-27T10:33:19.007 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-27T10:33:19.007 DEBUG:teuthology.orchestra.run.smithi097:> sudo /usr/sbin/setenforce permissive 2024-03-27T10:33:19.088 DEBUG:teuthology.orchestra.run.smithi114:> sudo /usr/sbin/setenforce permissive 2024-03-27T10:33:19.181 DEBUG:teuthology.orchestra.run.smithi177:> sudo /usr/sbin/setenforce permissive 2024-03-27T10:33:19.257 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-27T10:33:19.267 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-03-27T10:33:19.273 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-03-27T10:33:19.395 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-27T10:33:19.410 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-27T10:33:19.412 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi097.front.sepia.ceph.com,smithi114.front.sepia.ceph.com,smithi177.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-27T10:39:49.276 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi097.front.sepia.ceph.com'), Remote(name='ubuntu@smithi114.front.sepia.ceph.com'), Remote(name='ubuntu@smithi177.front.sepia.ceph.com')] 2024-03-27T10:39:49.277 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi097.front.sepia.ceph.com' 2024-03-27T10:39:49.279 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-27T10:39:49.421 DEBUG:teuthology.orchestra.run.smithi097:> true 2024-03-27T10:39:49.579 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi097.front.sepia.ceph.com' 2024-03-27T10:39:49.579 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi114.front.sepia.ceph.com' 2024-03-27T10:39:49.580 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-27T10:39:49.718 DEBUG:teuthology.orchestra.run.smithi114:> true 2024-03-27T10:39:49.873 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi114.front.sepia.ceph.com' 2024-03-27T10:39:49.874 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi177.front.sepia.ceph.com' 2024-03-27T10:39:49.874 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi177.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-27T10:39:50.017 DEBUG:teuthology.orchestra.run.smithi177:> true 2024-03-27T10:39:50.170 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi177.front.sepia.ceph.com' 2024-03-27T10:39:50.172 INFO:teuthology.run_tasks:Running task clock... 2024-03-27T10:39:50.183 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-27T10:39:50.183 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-27T10:39:50.183 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-27T10:39:50.191 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-27T10:39:50.191 DEBUG:teuthology.orchestra.run.smithi114:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-27T10:39:50.198 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-27T10:39:50.198 DEBUG:teuthology.orchestra.run.smithi177:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-27T10:39:50.286 INFO:teuthology.orchestra.run.smithi097.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-27T10:39:50.296 INFO:teuthology.orchestra.run.smithi114.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-27T10:39:50.299 INFO:teuthology.orchestra.run.smithi177.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-27T10:39:50.334 INFO:teuthology.orchestra.run.smithi097.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-27T10:39:50.346 INFO:teuthology.orchestra.run.smithi114.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-27T10:39:50.352 INFO:teuthology.orchestra.run.smithi177.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-27T10:39:50.428 INFO:teuthology.orchestra.run.smithi097.stderr:sudo: ntpd: command not found 2024-03-27T10:39:50.440 INFO:teuthology.orchestra.run.smithi114.stderr:sudo: ntpd: command not found 2024-03-27T10:39:50.446 INFO:teuthology.orchestra.run.smithi177.stderr:sudo: ntpd: command not found 2024-03-27T10:39:50.472 INFO:teuthology.orchestra.run.smithi097.stdout:506 Cannot talk to daemon 2024-03-27T10:39:50.481 INFO:teuthology.orchestra.run.smithi114.stdout:506 Cannot talk to daemon 2024-03-27T10:39:50.487 INFO:teuthology.orchestra.run.smithi177.stdout:506 Cannot talk to daemon 2024-03-27T10:39:50.525 INFO:teuthology.orchestra.run.smithi097.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-27T10:39:50.528 INFO:teuthology.orchestra.run.smithi114.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-27T10:39:50.535 INFO:teuthology.orchestra.run.smithi177.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-27T10:39:50.577 INFO:teuthology.orchestra.run.smithi097.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-27T10:39:50.578 INFO:teuthology.orchestra.run.smithi114.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-27T10:39:50.587 INFO:teuthology.orchestra.run.smithi177.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-27T10:39:50.706 INFO:teuthology.orchestra.run.smithi097.stderr:bash: line 1: ntpq: command not found 2024-03-27T10:39:50.706 INFO:teuthology.orchestra.run.smithi114.stderr:bash: line 1: ntpq: command not found 2024-03-27T10:39:50.713 INFO:teuthology.orchestra.run.smithi097.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-27T10:39:50.713 INFO:teuthology.orchestra.run.smithi097.stdout:=============================================================================== 2024-03-27T10:39:50.713 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-27T10:39:50.713 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-27T10:39:50.713 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-27T10:39:50.713 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-27T10:39:50.714 INFO:teuthology.orchestra.run.smithi114.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-27T10:39:50.714 INFO:teuthology.orchestra.run.smithi114.stdout:=============================================================================== 2024-03-27T10:39:50.714 INFO:teuthology.orchestra.run.smithi114.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-27T10:39:50.714 INFO:teuthology.orchestra.run.smithi114.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-27T10:39:50.714 INFO:teuthology.orchestra.run.smithi114.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-27T10:39:50.714 INFO:teuthology.orchestra.run.smithi114.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-27T10:39:50.721 INFO:teuthology.orchestra.run.smithi177.stderr:bash: line 1: ntpq: command not found 2024-03-27T10:39:50.730 INFO:teuthology.orchestra.run.smithi177.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-27T10:39:50.730 INFO:teuthology.orchestra.run.smithi177.stdout:=============================================================================== 2024-03-27T10:39:50.730 INFO:teuthology.orchestra.run.smithi177.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-27T10:39:50.730 INFO:teuthology.orchestra.run.smithi177.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-27T10:39:50.730 INFO:teuthology.orchestra.run.smithi177.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-27T10:39:50.730 INFO:teuthology.orchestra.run.smithi177.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-27T10:39:50.731 INFO:teuthology.run_tasks:Running task install... 2024-03-27T10:39:50.739 DEBUG:teuthology.task.install:project ceph 2024-03-27T10:39:50.739 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '155268c4e432a12433aa833f174f9fe3b1016ae0'}} 2024-03-27T10:39:50.740 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '155268c4e432a12433aa833f174f9fe3b1016ae0'} 2024-03-27T10:39:50.740 INFO:teuthology.task.install:Using flavor: default 2024-03-27T10:39:50.746 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-27T10:39:50.746 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-03-27T10:39:50.746 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '155268c4e432a12433aa833f174f9fe3b1016ae0', 'tag': None, 'wait_for_package': False} 2024-03-27T10:39:50.746 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=155268c4e432a12433aa833f174f9fe3b1016ae0 2024-03-27T10:39:50.749 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '155268c4e432a12433aa833f174f9fe3b1016ae0', 'tag': None, 'wait_for_package': False} 2024-03-27T10:39:50.749 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=155268c4e432a12433aa833f174f9fe3b1016ae0 2024-03-27T10:39:50.750 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '155268c4e432a12433aa833f174f9fe3b1016ae0', 'tag': None, 'wait_for_package': False} 2024-03-27T10:39:50.750 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=155268c4e432a12433aa833f174f9fe3b1016ae0 2024-03-27T10:39:50.897 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240326.131558/155268c4e432a12433aa833f174f9fe3b1016ae0/centos/9/flavors/default/ 2024-03-27T10:39:50.897 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2478.g155268c4 2024-03-27T10:39:50.967 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240326.131558/155268c4e432a12433aa833f174f9fe3b1016ae0/centos/9/flavors/default/ 2024-03-27T10:39:50.968 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2478.g155268c4 2024-03-27T10:39:50.973 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240326.131558/155268c4e432a12433aa833f174f9fe3b1016ae0/centos/9/flavors/default/ 2024-03-27T10:39:50.973 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2478.g155268c4 2024-03-27T10:39:51.191 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240326.131558/155268c4e432a12433aa833f174f9fe3b1016ae0/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240326.131558/155268c4e432a12433aa833f174f9fe3b1016ae0/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240326.131558/155268c4e432a12433aa833f174f9fe3b1016ae0/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-27T10:39:51.192 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-03-27T10:39:51.192 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-27T10:39:51.248 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240326.131558/155268c4e432a12433aa833f174f9fe3b1016ae0/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240326.131558/155268c4e432a12433aa833f174f9fe3b1016ae0/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240326.131558/155268c4e432a12433aa833f174f9fe3b1016ae0/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-27T10:39:51.248 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-27T10:39:51.248 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-27T10:39:51.273 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240326.131558/155268c4e432a12433aa833f174f9fe3b1016ae0/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240326.131558/155268c4e432a12433aa833f174f9fe3b1016ae0/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240326.131558/155268c4e432a12433aa833f174f9fe3b1016ae0/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-27T10:39:51.273 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-03-27T10:39:51.273 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-27T10:39:51.279 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-27T10:39:51.279 DEBUG:teuthology.orchestra.run.smithi177:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/155268c4e432a12433aa833f174f9fe3b1016ae0/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-27T10:39:51.344 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-27T10:39:51.345 DEBUG:teuthology.orchestra.run.smithi097:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/155268c4e432a12433aa833f174f9fe3b1016ae0/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-27T10:39:51.372 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-27T10:39:51.372 DEBUG:teuthology.orchestra.run.smithi114:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/155268c4e432a12433aa833f174f9fe3b1016ae0/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-27T10:39:51.416 DEBUG:teuthology.orchestra.run.smithi177:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-27T10:39:51.481 DEBUG:teuthology.orchestra.run.smithi097:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-27T10:39:51.511 DEBUG:teuthology.orchestra.run.smithi114:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-27T10:39:51.545 DEBUG:teuthology.orchestra.run.smithi177:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-27T10:39:51.615 DEBUG:teuthology.orchestra.run.smithi097:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-27T10:39:51.635 INFO:teuthology.orchestra.run.smithi177.stdout:check_obsoletes = 1 2024-03-27T10:39:51.640 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum clean all 2024-03-27T10:39:51.643 DEBUG:teuthology.orchestra.run.smithi114:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-27T10:39:51.701 INFO:teuthology.orchestra.run.smithi097.stdout:check_obsoletes = 1 2024-03-27T10:39:51.705 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum clean all 2024-03-27T10:39:51.727 INFO:teuthology.orchestra.run.smithi114.stdout:check_obsoletes = 1 2024-03-27T10:39:51.731 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum clean all 2024-03-27T10:39:51.985 INFO:teuthology.orchestra.run.smithi177.stdout:74 files removed 2024-03-27T10:39:52.024 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-27T10:39:52.062 INFO:teuthology.orchestra.run.smithi097.stdout:74 files removed 2024-03-27T10:39:52.080 INFO:teuthology.orchestra.run.smithi114.stdout:74 files removed 2024-03-27T10:39:52.101 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-27T10:39:52.118 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-27T10:39:53.161 INFO:teuthology.orchestra.run.smithi097.stdout:ceph packages for x86_64 150 kB/s | 83 kB 00:00 2024-03-27T10:39:53.178 INFO:teuthology.orchestra.run.smithi177.stdout:ceph packages for x86_64 128 kB/s | 83 kB 00:00 2024-03-27T10:39:53.241 INFO:teuthology.orchestra.run.smithi114.stdout:ceph packages for x86_64 134 kB/s | 83 kB 00:00 2024-03-27T10:39:53.868 INFO:teuthology.orchestra.run.smithi114.stdout:ceph noarch packages 22 kB/s | 12 kB 00:00 2024-03-27T10:39:53.892 INFO:teuthology.orchestra.run.smithi177.stdout:ceph noarch packages 19 kB/s | 12 kB 00:00 2024-03-27T10:39:53.893 INFO:teuthology.orchestra.run.smithi097.stdout:ceph noarch packages 18 kB/s | 12 kB 00:00 2024-03-27T10:39:54.609 INFO:teuthology.orchestra.run.smithi097.stdout:ceph source packages 3.0 kB/s | 1.9 kB 00:00 2024-03-27T10:39:54.726 INFO:teuthology.orchestra.run.smithi177.stdout:ceph source packages 2.5 kB/s | 1.9 kB 00:00 2024-03-27T10:39:54.743 INFO:teuthology.orchestra.run.smithi114.stdout:ceph source packages 2.4 kB/s | 1.9 kB 00:00 2024-03-27T10:39:55.305 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS-9 - AppStream 10 MB/s | 6.7 MB 00:00 2024-03-27T10:39:55.371 INFO:teuthology.orchestra.run.smithi177.stdout:CentOS-9 - AppStream 12 MB/s | 6.7 MB 00:00 2024-03-27T10:39:55.380 INFO:teuthology.orchestra.run.smithi114.stdout:CentOS-9 - AppStream 12 MB/s | 6.7 MB 00:00 2024-03-27T10:39:56.957 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS-9 - BaseOS 5.8 MB/s | 2.0 MB 00:00 2024-03-27T10:39:57.006 INFO:teuthology.orchestra.run.smithi114.stdout:CentOS-9 - BaseOS 5.7 MB/s | 2.0 MB 00:00 2024-03-27T10:39:57.039 INFO:teuthology.orchestra.run.smithi177.stdout:CentOS-9 - BaseOS 5.3 MB/s | 2.0 MB 00:00 2024-03-27T10:39:57.911 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.0 MB 00:00 2024-03-27T10:40:00.274 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - AppStream 16 MB/s | 19 MB 00:01 2024-03-27T10:40:00.844 INFO:teuthology.orchestra.run.smithi177.stdout:CentOS Stream 9 - BaseOS 2.4 MB/s | 8.0 MB 00:03 2024-03-27T10:40:03.643 INFO:teuthology.orchestra.run.smithi177.stdout:CentOS Stream 9 - AppStream 12 MB/s | 19 MB 00:01 2024-03-27T10:40:04.921 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - CRB 6.9 MB/s | 6.1 MB 00:00 2024-03-27T10:40:06.768 INFO:teuthology.orchestra.run.smithi114.stdout:CentOS Stream 9 - BaseOS 877 kB/s | 8.0 MB 00:09 2024-03-27T10:40:06.956 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - Extras packages 22 kB/s | 16 kB 00:00 2024-03-27T10:40:08.143 INFO:teuthology.orchestra.run.smithi097.stdout:Extra Packages for Enterprise Linux 20 MB/s | 21 MB 00:01 2024-03-27T10:40:08.232 INFO:teuthology.orchestra.run.smithi177.stdout:CentOS Stream 9 - CRB 7.7 MB/s | 6.1 MB 00:00 2024-03-27T10:40:09.894 INFO:teuthology.orchestra.run.smithi177.stdout:CentOS Stream 9 - Extras packages 47 kB/s | 16 kB 00:00 2024-03-27T10:40:11.818 INFO:teuthology.orchestra.run.smithi177.stdout:Extra Packages for Enterprise Linux 12 MB/s | 21 MB 00:01 2024-03-27T10:40:12.146 INFO:teuthology.orchestra.run.smithi114.stdout:CentOS Stream 9 - AppStream 4.5 MB/s | 19 MB 00:04 2024-03-27T10:40:13.882 INFO:teuthology.orchestra.run.smithi097.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-03-27T10:40:16.509 INFO:teuthology.orchestra.run.smithi097.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-27T10:40:16.510 INFO:teuthology.orchestra.run.smithi097.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-27T10:40:16.532 INFO:teuthology.orchestra.run.smithi097.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-27T10:40:16.533 INFO:teuthology.orchestra.run.smithi097.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-27T10:40:16.534 INFO:teuthology.orchestra.run.smithi097.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-27T10:40:16.536 INFO:teuthology.orchestra.run.smithi097.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-27T10:40:16.538 INFO:teuthology.orchestra.run.smithi097.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-27T10:40:16.542 INFO:teuthology.orchestra.run.smithi097.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-27T10:40:16.543 INFO:teuthology.orchestra.run.smithi097.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-27T10:40:16.630 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2024-03-27T10:40:16.669 INFO:teuthology.orchestra.run.smithi097.stdout:============================================================================================ 2024-03-27T10:40:16.669 INFO:teuthology.orchestra.run.smithi097.stdout: Package Arch Version Repository Size 2024-03-27T10:40:16.669 INFO:teuthology.orchestra.run.smithi097.stdout:============================================================================================ 2024-03-27T10:40:16.669 INFO:teuthology.orchestra.run.smithi097.stdout:Installing: 2024-03-27T10:40:16.669 INFO:teuthology.orchestra.run.smithi097.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-27T10:40:16.669 INFO:teuthology.orchestra.run.smithi097.stdout: ceph x86_64 2:19.0.0-2478.g155268c4.el9 ceph 6.6 k 2024-03-27T10:40:16.669 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-base x86_64 2:19.0.0-2478.g155268c4.el9 ceph 5.4 M 2024-03-27T10:40:16.669 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-fuse x86_64 2:19.0.0-2478.g155268c4.el9 ceph 890 k 2024-03-27T10:40:16.669 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2478.g155268c4.el9 ceph 145 k 2024-03-27T10:40:16.669 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr x86_64 2:19.0.0-2478.g155268c4.el9 ceph 1.5 M 2024-03-27T10:40:16.670 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2478.g155268c4.el9 ceph-noarch 141 k 2024-03-27T10:40:16.670 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2478.g155268c4.el9 ceph-noarch 3.5 M 2024-03-27T10:40:16.670 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2478.g155268c4.el9 ceph-noarch 7.4 M 2024-03-27T10:40:16.670 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-rook noarch 2:19.0.0-2478.g155268c4.el9 ceph-noarch 49 k 2024-03-27T10:40:16.670 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-radosgw x86_64 2:19.0.0-2478.g155268c4.el9 ceph 9.8 M 2024-03-27T10:40:16.670 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-test x86_64 2:19.0.0-2478.g155268c4.el9 ceph 48 M 2024-03-27T10:40:16.670 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-volume noarch 2:19.0.0-2478.g155268c4.el9 ceph-noarch 264 k 2024-03-27T10:40:16.670 INFO:teuthology.orchestra.run.smithi097.stdout: cephadm noarch 2:19.0.0-2478.g155268c4.el9 ceph-noarch 762 k 2024-03-27T10:40:16.670 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-mirror x86_64 2:19.0.0-2478.g155268c4.el9 ceph 221 k 2024-03-27T10:40:16.670 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-shell x86_64 2:19.0.0-2478.g155268c4.el9 ceph 27 k 2024-03-27T10:40:16.670 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-top noarch 2:19.0.0-2478.g155268c4.el9 ceph-noarch 18 k 2024-03-27T10:40:16.671 INFO:teuthology.orchestra.run.smithi097.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-27T10:40:16.671 INFO:teuthology.orchestra.run.smithi097.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-27T10:40:16.671 INFO:teuthology.orchestra.run.smithi097.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-27T10:40:16.671 INFO:teuthology.orchestra.run.smithi097.stdout: libcephfs-devel x86_64 2:19.0.0-2478.g155268c4.el9 ceph 32 k 2024-03-27T10:40:16.671 INFO:teuthology.orchestra.run.smithi097.stdout: libcephfs2 x86_64 2:19.0.0-2478.g155268c4.el9 ceph 746 k 2024-03-27T10:40:16.671 INFO:teuthology.orchestra.run.smithi097.stdout: librados-devel x86_64 2:19.0.0-2478.g155268c4.el9 ceph 127 k 2024-03-27T10:40:16.671 INFO:teuthology.orchestra.run.smithi097.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-27T10:40:16.671 INFO:teuthology.orchestra.run.smithi097.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-27T10:40:16.671 INFO:teuthology.orchestra.run.smithi097.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-27T10:40:16.671 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-03-27T10:40:16.671 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-03-27T10:40:16.671 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-03-27T10:40:16.672 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cephfs x86_64 2:19.0.0-2478.g155268c4.el9 ceph 162 k 2024-03-27T10:40:16.672 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rados x86_64 2:19.0.0-2478.g155268c4.el9 ceph 320 k 2024-03-27T10:40:16.672 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rbd x86_64 2:19.0.0-2478.g155268c4.el9 ceph 298 k 2024-03-27T10:40:16.672 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rgw x86_64 2:19.0.0-2478.g155268c4.el9 ceph 99 k 2024-03-27T10:40:16.672 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-fuse x86_64 2:19.0.0-2478.g155268c4.el9 ceph 85 k 2024-03-27T10:40:16.672 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-mirror x86_64 2:19.0.0-2478.g155268c4.el9 ceph 3.1 M 2024-03-27T10:40:16.672 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-nbd x86_64 2:19.0.0-2478.g155268c4.el9 ceph 170 k 2024-03-27T10:40:16.672 INFO:teuthology.orchestra.run.smithi097.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-27T10:40:16.672 INFO:teuthology.orchestra.run.smithi097.stdout:Upgrading: 2024-03-27T10:40:16.672 INFO:teuthology.orchestra.run.smithi097.stdout: NetworkManager x86_64 1:1.47.2-1.el9 baseos 2.3 M 2024-03-27T10:40:16.672 INFO:teuthology.orchestra.run.smithi097.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 baseos 1.8 M 2024-03-27T10:40:16.672 INFO:teuthology.orchestra.run.smithi097.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 baseos 42 k 2024-03-27T10:40:16.672 INFO:teuthology.orchestra.run.smithi097.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 baseos 248 k 2024-03-27T10:40:16.672 INFO:teuthology.orchestra.run.smithi097.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-27T10:40:16.673 INFO:teuthology.orchestra.run.smithi097.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-27T10:40:16.673 INFO:teuthology.orchestra.run.smithi097.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-27T10:40:16.673 INFO:teuthology.orchestra.run.smithi097.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-27T10:40:16.673 INFO:teuthology.orchestra.run.smithi097.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-27T10:40:16.673 INFO:teuthology.orchestra.run.smithi097.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-27T10:40:16.673 INFO:teuthology.orchestra.run.smithi097.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-27T10:40:16.673 INFO:teuthology.orchestra.run.smithi097.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-27T10:40:16.673 INFO:teuthology.orchestra.run.smithi097.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-27T10:40:16.673 INFO:teuthology.orchestra.run.smithi097.stdout: librados2 x86_64 2:19.0.0-2478.g155268c4.el9 ceph 3.4 M 2024-03-27T10:40:16.673 INFO:teuthology.orchestra.run.smithi097.stdout: librbd1 x86_64 2:19.0.0-2478.g155268c4.el9 ceph 3.1 M 2024-03-27T10:40:16.673 INFO:teuthology.orchestra.run.smithi097.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-27T10:40:16.673 INFO:teuthology.orchestra.run.smithi097.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-27T10:40:16.673 INFO:teuthology.orchestra.run.smithi097.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-27T10:40:16.674 INFO:teuthology.orchestra.run.smithi097.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-27T10:40:16.674 INFO:teuthology.orchestra.run.smithi097.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-27T10:40:16.674 INFO:teuthology.orchestra.run.smithi097.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-27T10:40:16.674 INFO:teuthology.orchestra.run.smithi097.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-27T10:40:16.674 INFO:teuthology.orchestra.run.smithi097.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-27T10:40:16.674 INFO:teuthology.orchestra.run.smithi097.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-27T10:40:16.674 INFO:teuthology.orchestra.run.smithi097.stdout:Installing dependencies: 2024-03-27T10:40:16.674 INFO:teuthology.orchestra.run.smithi097.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-27T10:40:16.677 INFO:teuthology.orchestra.run.smithi097.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-27T10:40:16.678 INFO:teuthology.orchestra.run.smithi097.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-27T10:40:16.678 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-common x86_64 2:19.0.0-2478.g155268c4.el9 ceph 21 M 2024-03-27T10:40:16.678 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2478.g155268c4.el9 ceph-noarch 34 k 2024-03-27T10:40:16.678 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mds x86_64 2:19.0.0-2478.g155268c4.el9 ceph 2.4 M 2024-03-27T10:40:16.678 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2478.g155268c4.el9 ceph-noarch 249 k 2024-03-27T10:40:16.678 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mon x86_64 2:19.0.0-2478.g155268c4.el9 ceph 4.7 M 2024-03-27T10:40:16.678 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-osd x86_64 2:19.0.0-2478.g155268c4.el9 ceph 17 M 2024-03-27T10:40:16.678 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2478.g155268c4.el9 ceph-noarch 16 k 2024-03-27T10:40:16.678 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-selinux x86_64 2:19.0.0-2478.g155268c4.el9 ceph 25 k 2024-03-27T10:40:16.678 INFO:teuthology.orchestra.run.smithi097.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-27T10:40:16.678 INFO:teuthology.orchestra.run.smithi097.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-27T10:40:16.679 INFO:teuthology.orchestra.run.smithi097.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-27T10:40:16.679 INFO:teuthology.orchestra.run.smithi097.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-27T10:40:16.679 INFO:teuthology.orchestra.run.smithi097.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-27T10:40:16.679 INFO:teuthology.orchestra.run.smithi097.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-27T10:40:16.679 INFO:teuthology.orchestra.run.smithi097.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-27T10:40:16.679 INFO:teuthology.orchestra.run.smithi097.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-27T10:40:16.679 INFO:teuthology.orchestra.run.smithi097.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-27T10:40:16.679 INFO:teuthology.orchestra.run.smithi097.stdout: libcephsqlite x86_64 2:19.0.0-2478.g155268c4.el9 ceph 163 k 2024-03-27T10:40:16.679 INFO:teuthology.orchestra.run.smithi097.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-27T10:40:16.679 INFO:teuthology.orchestra.run.smithi097.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-27T10:40:16.679 INFO:teuthology.orchestra.run.smithi097.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-27T10:40:16.679 INFO:teuthology.orchestra.run.smithi097.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-27T10:40:16.679 INFO:teuthology.orchestra.run.smithi097.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-27T10:40:16.679 INFO:teuthology.orchestra.run.smithi097.stdout: libradosstriper1 x86_64 2:19.0.0-2478.g155268c4.el9 ceph 501 k 2024-03-27T10:40:16.679 INFO:teuthology.orchestra.run.smithi097.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-27T10:40:16.680 INFO:teuthology.orchestra.run.smithi097.stdout: librgw2 x86_64 2:19.0.0-2478.g155268c4.el9 ceph 5.1 M 2024-03-27T10:40:16.680 INFO:teuthology.orchestra.run.smithi097.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-27T10:40:16.680 INFO:teuthology.orchestra.run.smithi097.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-27T10:40:16.680 INFO:teuthology.orchestra.run.smithi097.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-27T10:40:16.680 INFO:teuthology.orchestra.run.smithi097.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-27T10:40:16.680 INFO:teuthology.orchestra.run.smithi097.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-27T10:40:16.680 INFO:teuthology.orchestra.run.smithi097.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-27T10:40:16.680 INFO:teuthology.orchestra.run.smithi097.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-27T10:40:16.680 INFO:teuthology.orchestra.run.smithi097.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-27T10:40:16.680 INFO:teuthology.orchestra.run.smithi097.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-27T10:40:16.680 INFO:teuthology.orchestra.run.smithi097.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-27T10:40:16.680 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-03-27T10:40:16.680 INFO:teuthology.orchestra.run.smithi097.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-27T10:40:16.680 INFO:teuthology.orchestra.run.smithi097.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-27T10:40:16.680 INFO:teuthology.orchestra.run.smithi097.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-27T10:40:16.681 INFO:teuthology.orchestra.run.smithi097.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-27T10:40:16.681 INFO:teuthology.orchestra.run.smithi097.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-27T10:40:16.681 INFO:teuthology.orchestra.run.smithi097.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-27T10:40:16.681 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-27T10:40:16.681 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ceph-argparse x86_64 2:19.0.0-2478.g155268c4.el9 ceph 45 k 2024-03-27T10:40:16.681 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ceph-common x86_64 2:19.0.0-2478.g155268c4.el9 ceph 134 k 2024-03-27T10:40:16.681 INFO:teuthology.orchestra.run.smithi097.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-27T10:40:16.681 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-27T10:40:16.681 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-27T10:40:16.681 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-27T10:40:16.681 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-27T10:40:16.681 INFO:teuthology.orchestra.run.smithi097.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-27T10:40:16.681 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-27T10:40:16.681 INFO:teuthology.orchestra.run.smithi097.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-27T10:40:16.681 INFO:teuthology.orchestra.run.smithi097.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-27T10:40:16.682 INFO:teuthology.orchestra.run.smithi097.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-27T10:40:16.682 INFO:teuthology.orchestra.run.smithi097.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-27T10:40:16.682 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-27T10:40:16.682 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-27T10:40:16.682 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-27T10:40:16.682 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-27T10:40:16.682 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-27T10:40:16.682 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-27T10:40:16.682 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-27T10:40:16.682 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-27T10:40:16.682 INFO:teuthology.orchestra.run.smithi097.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-27T10:40:16.682 INFO:teuthology.orchestra.run.smithi097.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-27T10:40:16.682 INFO:teuthology.orchestra.run.smithi097.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-27T10:40:16.682 INFO:teuthology.orchestra.run.smithi097.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-27T10:40:16.682 INFO:teuthology.orchestra.run.smithi097.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-27T10:40:16.683 INFO:teuthology.orchestra.run.smithi097.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-27T10:40:16.683 INFO:teuthology.orchestra.run.smithi097.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-27T10:40:16.683 INFO:teuthology.orchestra.run.smithi097.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-27T10:40:16.683 INFO:teuthology.orchestra.run.smithi097.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-27T10:40:16.683 INFO:teuthology.orchestra.run.smithi097.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-27T10:40:16.683 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-27T10:40:16.683 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-27T10:40:16.683 INFO:teuthology.orchestra.run.smithi097.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-27T10:40:16.683 INFO:teuthology.orchestra.run.smithi097.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-27T10:40:16.683 INFO:teuthology.orchestra.run.smithi097.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-27T10:40:16.683 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-27T10:40:16.683 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-27T10:40:16.683 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-27T10:40:16.683 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-27T10:40:16.683 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-27T10:40:16.684 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-27T10:40:16.684 INFO:teuthology.orchestra.run.smithi097.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-27T10:40:16.684 INFO:teuthology.orchestra.run.smithi097.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-27T10:40:16.684 INFO:teuthology.orchestra.run.smithi097.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-27T10:40:16.684 INFO:teuthology.orchestra.run.smithi097.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-27T10:40:16.684 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-27T10:40:16.684 INFO:teuthology.orchestra.run.smithi097.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-27T10:40:16.684 INFO:teuthology.orchestra.run.smithi097.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-27T10:40:16.684 INFO:teuthology.orchestra.run.smithi097.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-27T10:40:16.684 INFO:teuthology.orchestra.run.smithi097.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-27T10:40:16.684 INFO:teuthology.orchestra.run.smithi097.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-27T10:40:16.684 INFO:teuthology.orchestra.run.smithi097.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-27T10:40:16.684 INFO:teuthology.orchestra.run.smithi097.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-27T10:40:16.684 INFO:teuthology.orchestra.run.smithi097.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-27T10:40:16.684 INFO:teuthology.orchestra.run.smithi097.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-27T10:40:16.685 INFO:teuthology.orchestra.run.smithi097.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-27T10:40:16.685 INFO:teuthology.orchestra.run.smithi097.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-03-27T10:40:16.685 INFO:teuthology.orchestra.run.smithi097.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-03-27T10:40:16.685 INFO:teuthology.orchestra.run.smithi097.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-27T10:40:16.685 INFO:teuthology.orchestra.run.smithi097.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-27T10:40:16.685 INFO:teuthology.orchestra.run.smithi097.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-27T10:40:16.685 INFO:teuthology.orchestra.run.smithi097.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-27T10:40:16.685 INFO:teuthology.orchestra.run.smithi097.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-27T10:40:16.685 INFO:teuthology.orchestra.run.smithi097.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-27T10:40:16.685 INFO:teuthology.orchestra.run.smithi097.stdout:Installing weak dependencies: 2024-03-27T10:40:16.685 INFO:teuthology.orchestra.run.smithi097.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-03-27T10:40:16.685 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T10:40:16.685 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction Summary 2024-03-27T10:40:16.685 INFO:teuthology.orchestra.run.smithi097.stdout:============================================================================================ 2024-03-27T10:40:16.685 INFO:teuthology.orchestra.run.smithi097.stdout:Install 150 Packages 2024-03-27T10:40:16.686 INFO:teuthology.orchestra.run.smithi097.stdout:Upgrade 24 Packages 2024-03-27T10:40:16.686 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T10:40:16.686 INFO:teuthology.orchestra.run.smithi097.stdout:Total download size: 227 M 2024-03-27T10:40:16.686 INFO:teuthology.orchestra.run.smithi097.stdout:Downloading Packages: 2024-03-27T10:40:16.846 INFO:teuthology.orchestra.run.smithi114.stdout:CentOS Stream 9 - CRB 6.3 MB/s | 6.1 MB 00:00 2024-03-27T10:40:17.384 INFO:teuthology.orchestra.run.smithi097.stdout:(1/174): ceph-19.0.0-2478.g155268c4.el9.x86_64. 25 kB/s | 6.6 kB 00:00 2024-03-27T10:40:17.603 INFO:teuthology.orchestra.run.smithi177.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-03-27T10:40:17.817 INFO:teuthology.orchestra.run.smithi097.stdout:(2/174): ceph-fuse-19.0.0-2478.g155268c4.el9.x8 2.0 MB/s | 890 kB 00:00 2024-03-27T10:40:17.917 INFO:teuthology.orchestra.run.smithi097.stdout:(3/174): ceph-immutable-object-cache-19.0.0-247 1.4 MB/s | 145 kB 00:00 2024-03-27T10:40:18.322 INFO:teuthology.orchestra.run.smithi114.stdout:CentOS Stream 9 - Extras packages 83 kB/s | 16 kB 00:00 2024-03-27T10:40:18.776 INFO:teuthology.orchestra.run.smithi097.stdout:(4/174): ceph-common-19.0.0-2478.g155268c4.el9. 12 MB/s | 21 MB 00:01 2024-03-27T10:40:18.851 INFO:teuthology.orchestra.run.smithi097.stdout:(5/174): ceph-mds-19.0.0-2478.g155268c4.el9.x86 2.5 MB/s | 2.4 MB 00:00 2024-03-27T10:40:18.976 INFO:teuthology.orchestra.run.smithi097.stdout:(6/174): ceph-base-19.0.0-2478.g155268c4.el9.x8 2.9 MB/s | 5.4 MB 00:01 2024-03-27T10:40:19.185 INFO:teuthology.orchestra.run.smithi097.stdout:(7/174): ceph-mgr-19.0.0-2478.g155268c4.el9.x86 3.7 MB/s | 1.5 MB 00:00 2024-03-27T10:40:19.760 INFO:teuthology.orchestra.run.smithi097.stdout:(8/174): ceph-mon-19.0.0-2478.g155268c4.el9.x86 5.2 MB/s | 4.7 MB 00:00 2024-03-27T10:40:19.861 INFO:teuthology.orchestra.run.smithi097.stdout:(9/174): ceph-selinux-19.0.0-2478.g155268c4.el9 253 kB/s | 25 kB 00:00 2024-03-27T10:40:20.209 INFO:teuthology.orchestra.run.smithi177.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-27T10:40:20.210 INFO:teuthology.orchestra.run.smithi177.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-27T10:40:20.231 INFO:teuthology.orchestra.run.smithi177.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-27T10:40:20.232 INFO:teuthology.orchestra.run.smithi177.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-27T10:40:20.233 INFO:teuthology.orchestra.run.smithi177.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-27T10:40:20.235 INFO:teuthology.orchestra.run.smithi177.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-27T10:40:20.237 INFO:teuthology.orchestra.run.smithi177.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-27T10:40:20.241 INFO:teuthology.orchestra.run.smithi177.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-27T10:40:20.242 INFO:teuthology.orchestra.run.smithi177.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-27T10:40:20.329 INFO:teuthology.orchestra.run.smithi177.stdout:Dependencies resolved. 2024-03-27T10:40:20.352 INFO:teuthology.orchestra.run.smithi177.stdout:============================================================================================ 2024-03-27T10:40:20.352 INFO:teuthology.orchestra.run.smithi177.stdout: Package Arch Version Repository Size 2024-03-27T10:40:20.352 INFO:teuthology.orchestra.run.smithi177.stdout:============================================================================================ 2024-03-27T10:40:20.352 INFO:teuthology.orchestra.run.smithi177.stdout:Installing: 2024-03-27T10:40:20.352 INFO:teuthology.orchestra.run.smithi177.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-27T10:40:20.352 INFO:teuthology.orchestra.run.smithi177.stdout: ceph x86_64 2:19.0.0-2478.g155268c4.el9 ceph 6.6 k 2024-03-27T10:40:20.352 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-base x86_64 2:19.0.0-2478.g155268c4.el9 ceph 5.4 M 2024-03-27T10:40:20.352 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-fuse x86_64 2:19.0.0-2478.g155268c4.el9 ceph 890 k 2024-03-27T10:40:20.352 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2478.g155268c4.el9 ceph 145 k 2024-03-27T10:40:20.352 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr x86_64 2:19.0.0-2478.g155268c4.el9 ceph 1.5 M 2024-03-27T10:40:20.352 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2478.g155268c4.el9 ceph-noarch 141 k 2024-03-27T10:40:20.353 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2478.g155268c4.el9 ceph-noarch 3.5 M 2024-03-27T10:40:20.353 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2478.g155268c4.el9 ceph-noarch 7.4 M 2024-03-27T10:40:20.353 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-rook noarch 2:19.0.0-2478.g155268c4.el9 ceph-noarch 49 k 2024-03-27T10:40:20.353 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-radosgw x86_64 2:19.0.0-2478.g155268c4.el9 ceph 9.8 M 2024-03-27T10:40:20.353 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-test x86_64 2:19.0.0-2478.g155268c4.el9 ceph 48 M 2024-03-27T10:40:20.353 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-volume noarch 2:19.0.0-2478.g155268c4.el9 ceph-noarch 264 k 2024-03-27T10:40:20.353 INFO:teuthology.orchestra.run.smithi177.stdout: cephadm noarch 2:19.0.0-2478.g155268c4.el9 ceph-noarch 762 k 2024-03-27T10:40:20.353 INFO:teuthology.orchestra.run.smithi177.stdout: cephfs-mirror x86_64 2:19.0.0-2478.g155268c4.el9 ceph 221 k 2024-03-27T10:40:20.353 INFO:teuthology.orchestra.run.smithi177.stdout: cephfs-shell x86_64 2:19.0.0-2478.g155268c4.el9 ceph 27 k 2024-03-27T10:40:20.353 INFO:teuthology.orchestra.run.smithi177.stdout: cephfs-top noarch 2:19.0.0-2478.g155268c4.el9 ceph-noarch 18 k 2024-03-27T10:40:20.353 INFO:teuthology.orchestra.run.smithi177.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-27T10:40:20.353 INFO:teuthology.orchestra.run.smithi177.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-27T10:40:20.353 INFO:teuthology.orchestra.run.smithi177.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-27T10:40:20.353 INFO:teuthology.orchestra.run.smithi177.stdout: libcephfs-devel x86_64 2:19.0.0-2478.g155268c4.el9 ceph 32 k 2024-03-27T10:40:20.354 INFO:teuthology.orchestra.run.smithi177.stdout: libcephfs2 x86_64 2:19.0.0-2478.g155268c4.el9 ceph 746 k 2024-03-27T10:40:20.354 INFO:teuthology.orchestra.run.smithi177.stdout: librados-devel x86_64 2:19.0.0-2478.g155268c4.el9 ceph 127 k 2024-03-27T10:40:20.354 INFO:teuthology.orchestra.run.smithi177.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-27T10:40:20.354 INFO:teuthology.orchestra.run.smithi177.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-27T10:40:20.354 INFO:teuthology.orchestra.run.smithi177.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-27T10:40:20.354 INFO:teuthology.orchestra.run.smithi177.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-03-27T10:40:20.354 INFO:teuthology.orchestra.run.smithi177.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-03-27T10:40:20.354 INFO:teuthology.orchestra.run.smithi177.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-03-27T10:40:20.354 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cephfs x86_64 2:19.0.0-2478.g155268c4.el9 ceph 162 k 2024-03-27T10:40:20.354 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rados x86_64 2:19.0.0-2478.g155268c4.el9 ceph 320 k 2024-03-27T10:40:20.354 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rbd x86_64 2:19.0.0-2478.g155268c4.el9 ceph 298 k 2024-03-27T10:40:20.354 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rgw x86_64 2:19.0.0-2478.g155268c4.el9 ceph 99 k 2024-03-27T10:40:20.354 INFO:teuthology.orchestra.run.smithi177.stdout: rbd-fuse x86_64 2:19.0.0-2478.g155268c4.el9 ceph 85 k 2024-03-27T10:40:20.354 INFO:teuthology.orchestra.run.smithi177.stdout: rbd-mirror x86_64 2:19.0.0-2478.g155268c4.el9 ceph 3.1 M 2024-03-27T10:40:20.355 INFO:teuthology.orchestra.run.smithi177.stdout: rbd-nbd x86_64 2:19.0.0-2478.g155268c4.el9 ceph 170 k 2024-03-27T10:40:20.355 INFO:teuthology.orchestra.run.smithi177.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-27T10:40:20.355 INFO:teuthology.orchestra.run.smithi177.stdout:Upgrading: 2024-03-27T10:40:20.355 INFO:teuthology.orchestra.run.smithi177.stdout: NetworkManager x86_64 1:1.47.2-1.el9 baseos 2.3 M 2024-03-27T10:40:20.355 INFO:teuthology.orchestra.run.smithi177.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 baseos 1.8 M 2024-03-27T10:40:20.355 INFO:teuthology.orchestra.run.smithi177.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 baseos 42 k 2024-03-27T10:40:20.355 INFO:teuthology.orchestra.run.smithi177.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 baseos 248 k 2024-03-27T10:40:20.355 INFO:teuthology.orchestra.run.smithi177.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-27T10:40:20.355 INFO:teuthology.orchestra.run.smithi177.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-27T10:40:20.355 INFO:teuthology.orchestra.run.smithi177.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-27T10:40:20.355 INFO:teuthology.orchestra.run.smithi177.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-27T10:40:20.355 INFO:teuthology.orchestra.run.smithi177.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-27T10:40:20.355 INFO:teuthology.orchestra.run.smithi177.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-27T10:40:20.356 INFO:teuthology.orchestra.run.smithi177.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-27T10:40:20.356 INFO:teuthology.orchestra.run.smithi177.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-27T10:40:20.356 INFO:teuthology.orchestra.run.smithi177.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-27T10:40:20.356 INFO:teuthology.orchestra.run.smithi177.stdout: librados2 x86_64 2:19.0.0-2478.g155268c4.el9 ceph 3.4 M 2024-03-27T10:40:20.356 INFO:teuthology.orchestra.run.smithi177.stdout: librbd1 x86_64 2:19.0.0-2478.g155268c4.el9 ceph 3.1 M 2024-03-27T10:40:20.356 INFO:teuthology.orchestra.run.smithi177.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-27T10:40:20.356 INFO:teuthology.orchestra.run.smithi177.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-27T10:40:20.356 INFO:teuthology.orchestra.run.smithi177.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-27T10:40:20.356 INFO:teuthology.orchestra.run.smithi177.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-27T10:40:20.356 INFO:teuthology.orchestra.run.smithi177.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-27T10:40:20.356 INFO:teuthology.orchestra.run.smithi177.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-27T10:40:20.356 INFO:teuthology.orchestra.run.smithi177.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-27T10:40:20.357 INFO:teuthology.orchestra.run.smithi177.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-27T10:40:20.357 INFO:teuthology.orchestra.run.smithi177.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-27T10:40:20.357 INFO:teuthology.orchestra.run.smithi177.stdout:Installing dependencies: 2024-03-27T10:40:20.357 INFO:teuthology.orchestra.run.smithi177.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-27T10:40:20.357 INFO:teuthology.orchestra.run.smithi177.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-27T10:40:20.357 INFO:teuthology.orchestra.run.smithi177.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-27T10:40:20.357 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-common x86_64 2:19.0.0-2478.g155268c4.el9 ceph 21 M 2024-03-27T10:40:20.357 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2478.g155268c4.el9 ceph-noarch 34 k 2024-03-27T10:40:20.357 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mds x86_64 2:19.0.0-2478.g155268c4.el9 ceph 2.4 M 2024-03-27T10:40:20.357 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2478.g155268c4.el9 ceph-noarch 249 k 2024-03-27T10:40:20.357 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mon x86_64 2:19.0.0-2478.g155268c4.el9 ceph 4.7 M 2024-03-27T10:40:20.357 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-osd x86_64 2:19.0.0-2478.g155268c4.el9 ceph 17 M 2024-03-27T10:40:20.357 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2478.g155268c4.el9 ceph-noarch 16 k 2024-03-27T10:40:20.357 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-selinux x86_64 2:19.0.0-2478.g155268c4.el9 ceph 25 k 2024-03-27T10:40:20.358 INFO:teuthology.orchestra.run.smithi177.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-27T10:40:20.358 INFO:teuthology.orchestra.run.smithi177.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-27T10:40:20.358 INFO:teuthology.orchestra.run.smithi177.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-27T10:40:20.358 INFO:teuthology.orchestra.run.smithi177.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-27T10:40:20.358 INFO:teuthology.orchestra.run.smithi177.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-27T10:40:20.358 INFO:teuthology.orchestra.run.smithi177.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-27T10:40:20.358 INFO:teuthology.orchestra.run.smithi177.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-27T10:40:20.358 INFO:teuthology.orchestra.run.smithi177.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-27T10:40:20.358 INFO:teuthology.orchestra.run.smithi177.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-27T10:40:20.358 INFO:teuthology.orchestra.run.smithi177.stdout: libcephsqlite x86_64 2:19.0.0-2478.g155268c4.el9 ceph 163 k 2024-03-27T10:40:20.358 INFO:teuthology.orchestra.run.smithi177.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-27T10:40:20.358 INFO:teuthology.orchestra.run.smithi177.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-27T10:40:20.359 INFO:teuthology.orchestra.run.smithi177.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-27T10:40:20.359 INFO:teuthology.orchestra.run.smithi177.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-27T10:40:20.359 INFO:teuthology.orchestra.run.smithi177.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-27T10:40:20.359 INFO:teuthology.orchestra.run.smithi177.stdout: libradosstriper1 x86_64 2:19.0.0-2478.g155268c4.el9 ceph 501 k 2024-03-27T10:40:20.359 INFO:teuthology.orchestra.run.smithi177.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-27T10:40:20.359 INFO:teuthology.orchestra.run.smithi177.stdout: librgw2 x86_64 2:19.0.0-2478.g155268c4.el9 ceph 5.1 M 2024-03-27T10:40:20.359 INFO:teuthology.orchestra.run.smithi177.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-27T10:40:20.359 INFO:teuthology.orchestra.run.smithi177.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-27T10:40:20.359 INFO:teuthology.orchestra.run.smithi177.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-27T10:40:20.359 INFO:teuthology.orchestra.run.smithi177.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-27T10:40:20.359 INFO:teuthology.orchestra.run.smithi177.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-27T10:40:20.359 INFO:teuthology.orchestra.run.smithi177.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-27T10:40:20.359 INFO:teuthology.orchestra.run.smithi177.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-27T10:40:20.359 INFO:teuthology.orchestra.run.smithi177.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-27T10:40:20.359 INFO:teuthology.orchestra.run.smithi177.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-27T10:40:20.360 INFO:teuthology.orchestra.run.smithi177.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-27T10:40:20.360 INFO:teuthology.orchestra.run.smithi177.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-03-27T10:40:20.360 INFO:teuthology.orchestra.run.smithi177.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-27T10:40:20.360 INFO:teuthology.orchestra.run.smithi177.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-27T10:40:20.360 INFO:teuthology.orchestra.run.smithi177.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-27T10:40:20.360 INFO:teuthology.orchestra.run.smithi177.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-27T10:40:20.360 INFO:teuthology.orchestra.run.smithi177.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-27T10:40:20.360 INFO:teuthology.orchestra.run.smithi177.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-27T10:40:20.360 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-27T10:40:20.360 INFO:teuthology.orchestra.run.smithi177.stdout: python3-ceph-argparse x86_64 2:19.0.0-2478.g155268c4.el9 ceph 45 k 2024-03-27T10:40:20.360 INFO:teuthology.orchestra.run.smithi177.stdout: python3-ceph-common x86_64 2:19.0.0-2478.g155268c4.el9 ceph 134 k 2024-03-27T10:40:20.360 INFO:teuthology.orchestra.run.smithi177.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-27T10:40:20.360 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-27T10:40:20.360 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-27T10:40:20.361 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-27T10:40:20.361 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-27T10:40:20.361 INFO:teuthology.orchestra.run.smithi177.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-27T10:40:20.361 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-27T10:40:20.361 INFO:teuthology.orchestra.run.smithi177.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-27T10:40:20.361 INFO:teuthology.orchestra.run.smithi177.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-27T10:40:20.361 INFO:teuthology.orchestra.run.smithi177.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-27T10:40:20.361 INFO:teuthology.orchestra.run.smithi177.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-27T10:40:20.361 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-27T10:40:20.361 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-27T10:40:20.361 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-27T10:40:20.361 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-27T10:40:20.361 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-27T10:40:20.361 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-27T10:40:20.361 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-27T10:40:20.362 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-27T10:40:20.362 INFO:teuthology.orchestra.run.smithi177.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-27T10:40:20.362 INFO:teuthology.orchestra.run.smithi177.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-27T10:40:20.362 INFO:teuthology.orchestra.run.smithi177.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-27T10:40:20.362 INFO:teuthology.orchestra.run.smithi177.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-27T10:40:20.362 INFO:teuthology.orchestra.run.smithi177.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-27T10:40:20.362 INFO:teuthology.orchestra.run.smithi177.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-27T10:40:20.362 INFO:teuthology.orchestra.run.smithi177.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-27T10:40:20.362 INFO:teuthology.orchestra.run.smithi177.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-27T10:40:20.362 INFO:teuthology.orchestra.run.smithi177.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-27T10:40:20.362 INFO:teuthology.orchestra.run.smithi177.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-27T10:40:20.362 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-27T10:40:20.363 INFO:teuthology.orchestra.run.smithi177.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-27T10:40:20.363 INFO:teuthology.orchestra.run.smithi177.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-27T10:40:20.363 INFO:teuthology.orchestra.run.smithi177.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-27T10:40:20.363 INFO:teuthology.orchestra.run.smithi177.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-27T10:40:20.363 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-27T10:40:20.363 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-27T10:40:20.363 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-27T10:40:20.363 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-27T10:40:20.363 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-27T10:40:20.363 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-27T10:40:20.363 INFO:teuthology.orchestra.run.smithi177.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-27T10:40:20.363 INFO:teuthology.orchestra.run.smithi177.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-27T10:40:20.363 INFO:teuthology.orchestra.run.smithi177.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-27T10:40:20.363 INFO:teuthology.orchestra.run.smithi177.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-27T10:40:20.363 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-27T10:40:20.364 INFO:teuthology.orchestra.run.smithi177.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-27T10:40:20.364 INFO:teuthology.orchestra.run.smithi177.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-27T10:40:20.364 INFO:teuthology.orchestra.run.smithi177.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-27T10:40:20.364 INFO:teuthology.orchestra.run.smithi177.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-27T10:40:20.364 INFO:teuthology.orchestra.run.smithi177.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-27T10:40:20.364 INFO:teuthology.orchestra.run.smithi177.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-27T10:40:20.364 INFO:teuthology.orchestra.run.smithi177.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-27T10:40:20.364 INFO:teuthology.orchestra.run.smithi177.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-27T10:40:20.364 INFO:teuthology.orchestra.run.smithi177.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-27T10:40:20.364 INFO:teuthology.orchestra.run.smithi177.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-27T10:40:20.364 INFO:teuthology.orchestra.run.smithi177.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-03-27T10:40:20.364 INFO:teuthology.orchestra.run.smithi177.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-03-27T10:40:20.364 INFO:teuthology.orchestra.run.smithi177.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-27T10:40:20.364 INFO:teuthology.orchestra.run.smithi177.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-27T10:40:20.365 INFO:teuthology.orchestra.run.smithi177.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-27T10:40:20.365 INFO:teuthology.orchestra.run.smithi177.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-27T10:40:20.365 INFO:teuthology.orchestra.run.smithi177.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-27T10:40:20.365 INFO:teuthology.orchestra.run.smithi177.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-27T10:40:20.365 INFO:teuthology.orchestra.run.smithi177.stdout:Installing weak dependencies: 2024-03-27T10:40:20.365 INFO:teuthology.orchestra.run.smithi177.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-03-27T10:40:20.365 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T10:40:20.365 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction Summary 2024-03-27T10:40:20.365 INFO:teuthology.orchestra.run.smithi177.stdout:============================================================================================ 2024-03-27T10:40:20.365 INFO:teuthology.orchestra.run.smithi177.stdout:Install 150 Packages 2024-03-27T10:40:20.365 INFO:teuthology.orchestra.run.smithi177.stdout:Upgrade 24 Packages 2024-03-27T10:40:20.365 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T10:40:20.366 INFO:teuthology.orchestra.run.smithi177.stdout:Total download size: 227 M 2024-03-27T10:40:20.366 INFO:teuthology.orchestra.run.smithi177.stdout:Downloading Packages: 2024-03-27T10:40:20.577 INFO:teuthology.orchestra.run.smithi097.stdout:(10/174): ceph-osd-19.0.0-2478.g155268c4.el9.x8 11 MB/s | 17 MB 00:01 2024-03-27T10:40:20.811 INFO:teuthology.orchestra.run.smithi097.stdout:(11/174): ceph-radosgw-19.0.0-2478.g155268c4.el 6.1 MB/s | 9.8 MB 00:01 2024-03-27T10:40:20.845 INFO:teuthology.orchestra.run.smithi097.stdout:(12/174): cephfs-mirror-19.0.0-2478.g155268c4.e 833 kB/s | 221 kB 00:00 2024-03-27T10:40:20.861 INFO:teuthology.orchestra.run.smithi097.stdout:(13/174): cephfs-shell-19.0.0-2478.g155268c4.el 551 kB/s | 27 kB 00:00 2024-03-27T10:40:20.937 INFO:teuthology.orchestra.run.smithi097.stdout:(14/174): libcephfs-devel-19.0.0-2478.g155268c4 347 kB/s | 32 kB 00:00 2024-03-27T10:40:21.012 INFO:teuthology.orchestra.run.smithi097.stdout:(15/174): libcephfs2-19.0.0-2478.g155268c4.el9. 4.9 MB/s | 746 kB 00:00 2024-03-27T10:40:21.021 INFO:teuthology.orchestra.run.smithi177.stdout:(1/174): ceph-19.0.0-2478.g155268c4.el9.x86_64. 21 kB/s | 6.6 kB 00:00 2024-03-27T10:40:21.046 INFO:teuthology.orchestra.run.smithi097.stdout:(16/174): libcephsqlite-19.0.0-2478.g155268c4.e 1.5 MB/s | 163 kB 00:00 2024-03-27T10:40:21.079 INFO:teuthology.orchestra.run.smithi097.stdout:(17/174): librados-devel-19.0.0-2478.g155268c4. 1.9 MB/s | 127 kB 00:00 2024-03-27T10:40:21.146 INFO:teuthology.orchestra.run.smithi097.stdout:(18/174): libradosstriper1-19.0.0-2478.g155268c 4.9 MB/s | 501 kB 00:00 2024-03-27T10:40:21.247 INFO:teuthology.orchestra.run.smithi097.stdout:(19/174): python3-ceph-argparse-19.0.0-2478.g15 455 kB/s | 45 kB 00:00 2024-03-27T10:40:21.396 INFO:teuthology.orchestra.run.smithi097.stdout:(20/174): python3-ceph-common-19.0.0-2478.g1552 905 kB/s | 134 kB 00:00 2024-03-27T10:40:21.514 INFO:teuthology.orchestra.run.smithi097.stdout:(21/174): python3-cephfs-19.0.0-2478.g155268c4. 1.3 MB/s | 162 kB 00:00 2024-03-27T10:40:21.656 INFO:teuthology.orchestra.run.smithi097.stdout:(22/174): python3-rados-19.0.0-2478.g155268c4.e 2.2 MB/s | 320 kB 00:00 2024-03-27T10:40:21.773 INFO:teuthology.orchestra.run.smithi097.stdout:(23/174): python3-rbd-19.0.0-2478.g155268c4.el9 2.5 MB/s | 298 kB 00:00 2024-03-27T10:40:21.923 INFO:teuthology.orchestra.run.smithi097.stdout:(24/174): librgw2-19.0.0-2478.g155268c4.el9.x86 6.1 MB/s | 5.1 MB 00:00 2024-03-27T10:40:21.957 INFO:teuthology.orchestra.run.smithi097.stdout:(25/174): python3-rgw-19.0.0-2478.g155268c4.el9 542 kB/s | 99 kB 00:00 2024-03-27T10:40:21.973 INFO:teuthology.orchestra.run.smithi177.stdout:(2/174): ceph-fuse-19.0.0-2478.g155268c4.el9.x8 936 kB/s | 890 kB 00:00 2024-03-27T10:40:21.990 INFO:teuthology.orchestra.run.smithi097.stdout:(26/174): rbd-fuse-19.0.0-2478.g155268c4.el9.x8 1.3 MB/s | 85 kB 00:00 2024-03-27T10:40:22.082 INFO:teuthology.orchestra.run.smithi097.stdout:(27/174): rbd-nbd-19.0.0-2478.g155268c4.el9.x86 1.8 MB/s | 170 kB 00:00 2024-03-27T10:40:22.149 INFO:teuthology.orchestra.run.smithi097.stdout:(28/174): ceph-grafana-dashboards-19.0.0-2478.g 515 kB/s | 34 kB 00:00 2024-03-27T10:40:22.208 INFO:teuthology.orchestra.run.smithi097.stdout:(29/174): ceph-mgr-cephadm-19.0.0-2478.g155268c 2.4 MB/s | 141 kB 00:00 2024-03-27T10:40:22.254 INFO:teuthology.orchestra.run.smithi177.stdout:(3/174): ceph-immutable-object-cache-19.0.0-247 516 kB/s | 145 kB 00:00 2024-03-27T10:40:22.394 INFO:teuthology.orchestra.run.smithi097.stdout:(30/174): rbd-mirror-19.0.0-2478.g155268c4.el9. 7.1 MB/s | 3.1 MB 00:00 2024-03-27T10:40:22.725 INFO:teuthology.orchestra.run.smithi097.stdout:(31/174): ceph-mgr-dashboard-19.0.0-2478.g15526 6.9 MB/s | 3.5 MB 00:00 2024-03-27T10:40:22.834 INFO:teuthology.orchestra.run.smithi097.stdout:(32/174): ceph-mgr-modules-core-19.0.0-2478.g15 2.3 MB/s | 249 kB 00:00 2024-03-27T10:40:22.901 INFO:teuthology.orchestra.run.smithi097.stdout:(33/174): ceph-mgr-rook-19.0.0-2478.g155268c4.e 750 kB/s | 49 kB 00:00 2024-03-27T10:40:22.951 INFO:teuthology.orchestra.run.smithi097.stdout:(34/174): ceph-prometheus-alerts-19.0.0-2478.g1 335 kB/s | 16 kB 00:00 2024-03-27T10:40:23.010 INFO:teuthology.orchestra.run.smithi097.stdout:(35/174): ceph-volume-19.0.0-2478.g155268c4.el9 4.5 MB/s | 264 kB 00:00 2024-03-27T10:40:23.118 INFO:teuthology.orchestra.run.smithi097.stdout:(36/174): cephadm-19.0.0-2478.g155268c4.el9.noa 6.9 MB/s | 762 kB 00:00 2024-03-27T10:40:23.185 INFO:teuthology.orchestra.run.smithi097.stdout:(37/174): cephfs-top-19.0.0-2478.g155268c4.el9. 278 kB/s | 18 kB 00:00 2024-03-27T10:40:23.544 INFO:teuthology.orchestra.run.smithi097.stdout:(38/174): ceph-mgr-diskprediction-local-19.0.0- 6.4 MB/s | 7.4 MB 00:01 2024-03-27T10:40:23.711 INFO:teuthology.orchestra.run.smithi097.stdout:(39/174): automake-1.16.2-8.el9.noarch.rpm 1.2 MB/s | 662 kB 00:00 2024-03-27T10:40:23.970 INFO:teuthology.orchestra.run.smithi097.stdout:(40/174): bison-3.7.4-5.el9.x86_64.rpm 2.1 MB/s | 920 kB 00:00 2024-03-27T10:40:23.995 INFO:teuthology.orchestra.run.smithi097.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 376 kB/s | 106 kB 00:00 2024-03-27T10:40:24.087 INFO:teuthology.orchestra.run.smithi097.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 2.6 MB/s | 307 kB 00:00 2024-03-27T10:40:24.104 INFO:teuthology.orchestra.run.smithi097.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 294 kB/s | 32 kB 00:00 2024-03-27T10:40:24.171 INFO:teuthology.orchestra.run.smithi177.stdout:(4/174): ceph-base-19.0.0-2478.g155268c4.el9.x8 1.5 MB/s | 5.4 MB 00:03 2024-03-27T10:40:24.362 INFO:teuthology.orchestra.run.smithi097.stdout:(44/174): flexiblas-openblas-openmp-3.0.4-8.el9 66 kB/s | 17 kB 00:00 2024-03-27T10:40:24.521 INFO:teuthology.orchestra.run.smithi097.stdout:(45/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 6.9 MB/s | 3.0 MB 00:00 2024-03-27T10:40:24.538 INFO:teuthology.orchestra.run.smithi097.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 58 kB/s | 10 kB 00:00 2024-03-27T10:40:24.571 INFO:teuthology.orchestra.run.smithi097.stdout:(47/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.3 MB/s | 44 kB 00:00 2024-03-27T10:40:24.597 INFO:teuthology.orchestra.run.smithi097.stdout:(48/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 159 kB 00:00 2024-03-27T10:40:24.621 INFO:teuthology.orchestra.run.smithi177.stdout:(5/174): ceph-mgr-19.0.0-2478.g155268c4.el9.x86 3.3 MB/s | 1.5 MB 00:00 2024-03-27T10:40:24.655 INFO:teuthology.orchestra.run.smithi097.stdout:(49/174): librdkafka-1.6.1-102.el9.x86_64.rpm 7.9 MB/s | 661 kB 00:00 2024-03-27T10:40:24.714 INFO:teuthology.orchestra.run.smithi097.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 5.0 MB/s | 578 kB 00:00 2024-03-27T10:40:24.739 INFO:teuthology.orchestra.run.smithi097.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 593 kB/s | 46 kB 00:00 2024-03-27T10:40:24.782 INFO:teuthology.orchestra.run.smithi097.stdout:(52/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.2 MB/s | 282 kB 00:00 2024-03-27T10:40:24.815 INFO:teuthology.orchestra.run.smithi097.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 2.5 MB/s | 187 kB 00:00 2024-03-27T10:40:24.840 INFO:teuthology.orchestra.run.smithi097.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 3.8 MB/s | 217 kB 00:00 2024-03-27T10:40:24.866 INFO:teuthology.orchestra.run.smithi097.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 719 kB/s | 34 kB 00:00 2024-03-27T10:40:24.888 INFO:teuthology.orchestra.run.smithi177.stdout:(6/174): ceph-common-19.0.0-2478.g155268c4.el9. 4.9 MB/s | 21 MB 00:04 2024-03-27T10:40:25.191 INFO:teuthology.orchestra.run.smithi097.stdout:(56/174): openblas-openmp-0.3.21-2.el9.x86_64.r 14 MB/s | 4.7 MB 00:00 2024-03-27T10:40:25.241 INFO:teuthology.orchestra.run.smithi097.stdout:(57/174): protobuf-3.14.0-13.el9.x86_64.rpm 2.7 MB/s | 1.0 MB 00:00 2024-03-27T10:40:25.267 INFO:teuthology.orchestra.run.smithi097.stdout:(58/174): python3-attrs-20.3.0-7.el9.noarch.rpm 1.1 MB/s | 78 kB 00:00 2024-03-27T10:40:25.434 INFO:teuthology.orchestra.run.smithi097.stdout:(59/174): python3-jsonpatch-1.21-16.el9.noarch. 151 kB/s | 24 kB 00:00 2024-03-27T10:40:25.505 INFO:teuthology.orchestra.run.smithi177.stdout:(7/174): ceph-mds-19.0.0-2478.g155268c4.el9.x86 745 kB/s | 2.4 MB 00:03 2024-03-27T10:40:25.542 INFO:teuthology.orchestra.run.smithi097.stdout:(60/174): python3-babel-2.9.1-2.el9.noarch.rpm 19 MB/s | 5.8 MB 00:00 2024-03-27T10:40:25.567 INFO:teuthology.orchestra.run.smithi097.stdout:(61/174): python3-jsonpointer-2.0-4.el9.noarch. 144 kB/s | 18 kB 00:00 2024-03-27T10:40:25.593 INFO:teuthology.orchestra.run.smithi097.stdout:(62/174): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 144 kB 00:00 2024-03-27T10:40:25.618 INFO:teuthology.orchestra.run.smithi097.stdout:(63/174): python3-markupsafe-1.1.1-12.el9.x86_6 654 kB/s | 32 kB 00:00 2024-03-27T10:40:25.785 INFO:teuthology.orchestra.run.smithi097.stdout:(64/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 25 MB/s | 4.8 MB 00:00 2024-03-27T10:40:25.818 INFO:teuthology.orchestra.run.smithi097.stdout:(65/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.1 MB/s | 217 kB 00:00 2024-03-27T10:40:25.852 INFO:teuthology.orchestra.run.smithi097.stdout:(66/174): python3-oauthlib-3.1.1-5.el9.noarch.r 2.6 MB/s | 174 kB 00:00 2024-03-27T10:40:25.877 INFO:teuthology.orchestra.run.smithi097.stdout:(67/174): python3-packaging-20.9-5.el9.noarch.r 1.2 MB/s | 69 kB 00:00 2024-03-27T10:40:25.903 INFO:teuthology.orchestra.run.smithi097.stdout:(68/174): python3-prettytable-0.7.2-27.el9.noar 844 kB/s | 41 kB 00:00 2024-03-27T10:40:25.936 INFO:teuthology.orchestra.run.smithi097.stdout:(69/174): python3-protobuf-3.14.0-13.el9.noarch 4.1 MB/s | 238 kB 00:00 2024-03-27T10:40:25.953 INFO:teuthology.orchestra.run.smithi097.stdout:(70/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.7 MB/s | 132 kB 00:00 2024-03-27T10:40:25.987 INFO:teuthology.orchestra.run.smithi097.stdout:(71/174): python3-pyasn1-modules-0.4.8-6.el9.no 4.3 MB/s | 211 kB 00:00 2024-03-27T10:40:26.012 INFO:teuthology.orchestra.run.smithi097.stdout:(72/174): python3-pytz-2021.1-5.el9.noarch.rpm 845 kB/s | 47 kB 00:00 2024-03-27T10:40:26.037 INFO:teuthology.orchestra.run.smithi097.stdout:(73/174): python3-requests-oauthlib-1.3.0-12.el 874 kB/s | 43 kB 00:00 2024-03-27T10:40:26.047 INFO:teuthology.orchestra.run.smithi177.stdout:(8/174): ceph-mon-19.0.0-2478.g155268c4.el9.x86 3.3 MB/s | 4.7 MB 00:01 2024-03-27T10:40:26.130 INFO:teuthology.orchestra.run.smithi177.stdout:(9/174): ceph-selinux-19.0.0-2478.g155268c4.el9 306 kB/s | 25 kB 00:00 2024-03-27T10:40:26.254 INFO:teuthology.orchestra.run.smithi097.stdout:(74/174): python3-toml-0.10.2-6.el9.noarch.rpm 190 kB/s | 39 kB 00:00 2024-03-27T10:40:26.497 INFO:teuthology.orchestra.run.smithi097.stdout:(75/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 32 MB/s | 16 MB 00:00 2024-03-27T10:40:26.522 INFO:teuthology.orchestra.run.smithi097.stdout:(76/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 161 kB/s | 41 kB 00:00 2024-03-27T10:40:26.564 INFO:teuthology.orchestra.run.smithi097.stdout:(77/174): uuid-1.6.2-55.el9.x86_64.rpm 1.4 MB/s | 56 kB 00:00 2024-03-27T10:40:26.614 INFO:teuthology.orchestra.run.smithi097.stdout:(78/174): socat-1.7.4.1-5.el9.x86_64.rpm 3.2 MB/s | 300 kB 00:00 2024-03-27T10:40:26.639 INFO:teuthology.orchestra.run.smithi097.stdout:(79/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 868 kB/s | 63 kB 00:00 2024-03-27T10:40:26.665 INFO:teuthology.orchestra.run.smithi097.stdout:(80/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 897 kB/s | 44 kB 00:00 2024-03-27T10:40:26.690 INFO:teuthology.orchestra.run.smithi097.stdout:(81/174): fuse-2.9.9-15.el9.x86_64.rpm 1.6 MB/s | 79 kB 00:00 2024-03-27T10:40:26.714 INFO:teuthology.orchestra.run.smithi177.stdout:(10/174): ceph-osd-19.0.0-2478.g155268c4.el9.x8 9.2 MB/s | 17 MB 00:01 2024-03-27T10:40:26.724 INFO:teuthology.orchestra.run.smithi097.stdout:(82/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.1 MB/s | 241 kB 00:00 2024-03-27T10:40:26.774 INFO:teuthology.orchestra.run.smithi097.stdout:(83/174): python3-cryptography-36.0.1-4.el9.x86 14 MB/s | 1.1 MB 00:00 2024-03-27T10:40:26.789 INFO:teuthology.orchestra.run.smithi177.stdout:(11/174): cephfs-mirror-19.0.0-2478.g155268c4.e 2.9 MB/s | 221 kB 00:00 2024-03-27T10:40:26.799 INFO:teuthology.orchestra.run.smithi097.stdout:(84/174): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 103 kB 00:00 2024-03-27T10:40:26.824 INFO:teuthology.orchestra.run.smithi097.stdout:(85/174): python3-pycparser-2.20-6.el9.noarch.r 2.5 MB/s | 124 kB 00:00 2024-03-27T10:40:26.831 INFO:teuthology.orchestra.run.smithi177.stdout:(12/174): cephfs-shell-19.0.0-2478.g155268c4.el 664 kB/s | 27 kB 00:00 2024-03-27T10:40:26.858 INFO:teuthology.orchestra.run.smithi097.stdout:(86/174): python3-urllib3-1.26.5-5.el9.noarch.r 3.2 MB/s | 187 kB 00:00 2024-03-27T10:40:26.873 INFO:teuthology.orchestra.run.smithi177.stdout:(13/174): libcephfs-devel-19.0.0-2478.g155268c4 778 kB/s | 32 kB 00:00 2024-03-27T10:40:27.007 INFO:teuthology.orchestra.run.smithi177.stdout:(14/174): libcephfs2-19.0.0-2478.g155268c4.el9. 5.5 MB/s | 746 kB 00:00 2024-03-27T10:40:27.057 INFO:teuthology.orchestra.run.smithi177.stdout:(15/174): libcephsqlite-19.0.0-2478.g155268c4.e 3.3 MB/s | 163 kB 00:00 2024-03-27T10:40:27.107 INFO:teuthology.orchestra.run.smithi177.stdout:(16/174): librados-devel-19.0.0-2478.g155268c4. 2.5 MB/s | 127 kB 00:00 2024-03-27T10:40:27.199 INFO:teuthology.orchestra.run.smithi177.stdout:(17/174): libradosstriper1-19.0.0-2478.g155268c 5.4 MB/s | 501 kB 00:00 2024-03-27T10:40:27.384 INFO:teuthology.orchestra.run.smithi097.stdout:(87/174): ceph-test-19.0.0-2478.g155268c4.el9.x 6.3 MB/s | 48 MB 00:07 2024-03-27T10:40:27.626 INFO:teuthology.orchestra.run.smithi097.stdout:(88/174): jq-1.6-16.el9.x86_64.rpm 234 kB/s | 187 kB 00:00 2024-03-27T10:40:27.659 INFO:teuthology.orchestra.run.smithi097.stdout:(89/174): libquadmath-11.4.1-3.el9.x86_64.rpm 712 kB/s | 195 kB 00:00 2024-03-27T10:40:27.693 INFO:teuthology.orchestra.run.smithi097.stdout:(90/174): python3-requests-2.25.1-8.el9.noarch. 1.8 MB/s | 125 kB 00:00 2024-03-27T10:40:27.736 INFO:teuthology.orchestra.run.smithi177.stdout:(18/174): librgw2-19.0.0-2478.g155268c4.el9.x86 9.6 MB/s | 5.1 MB 00:00 2024-03-27T10:40:27.751 INFO:teuthology.orchestra.run.smithi097.stdout:(91/174): libgfortran-11.4.1-3.el9.x86_64.rpm 899 kB/s | 803 kB 00:00 2024-03-27T10:40:27.783 INFO:teuthology.orchestra.run.smithi177.stdout:(19/174): python3-ceph-argparse-19.0.0-2478.g15 973 kB/s | 45 kB 00:00 2024-03-27T10:40:27.834 INFO:teuthology.orchestra.run.smithi177.stdout:(20/174): python3-ceph-common-19.0.0-2478.g1552 2.7 MB/s | 134 kB 00:00 2024-03-27T10:40:27.884 INFO:teuthology.orchestra.run.smithi177.stdout:(21/174): python3-cephfs-19.0.0-2478.g155268c4. 3.2 MB/s | 162 kB 00:00 2024-03-27T10:40:27.959 INFO:teuthology.orchestra.run.smithi177.stdout:(22/174): python3-rados-19.0.0-2478.g155268c4.e 4.2 MB/s | 320 kB 00:00 2024-03-27T10:40:27.985 INFO:teuthology.orchestra.run.smithi097.stdout:(92/174): elfutils-libelf-devel-0.190-2.el9.x86 66 kB/s | 21 kB 00:00 2024-03-27T10:40:28.034 INFO:teuthology.orchestra.run.smithi177.stdout:(23/174): python3-rbd-19.0.0-2478.g155268c4.el9 3.9 MB/s | 298 kB 00:00 2024-03-27T10:40:28.035 INFO:teuthology.orchestra.run.smithi097.stdout:(93/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 74 kB/s | 25 kB 00:00 2024-03-27T10:40:28.085 INFO:teuthology.orchestra.run.smithi177.stdout:(24/174): python3-rgw-19.0.0-2478.g155268c4.el9 2.0 MB/s | 99 kB 00:00 2024-03-27T10:40:28.135 INFO:teuthology.orchestra.run.smithi177.stdout:(25/174): rbd-fuse-19.0.0-2478.g155268c4.el9.x8 1.7 MB/s | 85 kB 00:00 2024-03-27T10:40:28.211 INFO:teuthology.orchestra.run.smithi097.stdout:(94/174): postgresql-13.14-1.el9.x86_64.rpm 7.1 MB/s | 1.6 MB 00:00 2024-03-27T10:40:28.261 INFO:teuthology.orchestra.run.smithi097.stdout:(95/174): postgresql-contrib-13.14-1.el9.x86_64 4.0 MB/s | 911 kB 00:00 2024-03-27T10:40:28.311 INFO:teuthology.orchestra.run.smithi097.stdout:(96/174): postgresql-private-libs-13.14-1.el9.x 1.3 MB/s | 133 kB 00:00 2024-03-27T10:40:28.386 INFO:teuthology.orchestra.run.smithi097.stdout:(97/174): python3-devel-3.9.17-2.el9.x86_64.rpm 3.2 MB/s | 245 kB 00:00 2024-03-27T10:40:28.444 INFO:teuthology.orchestra.run.smithi177.stdout:(26/174): rbd-mirror-19.0.0-2478.g155268c4.el9. 10 MB/s | 3.1 MB 00:00 2024-03-27T10:40:28.494 INFO:teuthology.orchestra.run.smithi177.stdout:(27/174): rbd-nbd-19.0.0-2478.g155268c4.el9.x86 3.4 MB/s | 170 kB 00:00 2024-03-27T10:40:28.503 INFO:teuthology.orchestra.run.smithi097.stdout:(98/174): openssl-devel-3.0.7-27.el9.x86_64.rpm 5.5 MB/s | 4.1 MB 00:00 2024-03-27T10:40:28.537 INFO:teuthology.orchestra.run.smithi097.stdout:(99/174): python3-jinja2-2.11.3-5.el9.noarch.rp 1.6 MB/s | 248 kB 00:00 2024-03-27T10:40:28.544 INFO:teuthology.orchestra.run.smithi177.stdout:(28/174): ceph-grafana-dashboards-19.0.0-2478.g 685 kB/s | 34 kB 00:00 2024-03-27T10:40:28.594 INFO:teuthology.orchestra.run.smithi177.stdout:(29/174): ceph-mgr-cephadm-19.0.0-2478.g155268c 2.8 MB/s | 141 kB 00:00 2024-03-27T10:40:28.596 INFO:teuthology.orchestra.run.smithi097.stdout:(100/174): qatlib-24.02.0-1.el9.x86_64.rpm 2.4 MB/s | 221 kB 00:00 2024-03-27T10:40:28.621 INFO:teuthology.orchestra.run.smithi097.stdout:(101/174): qatlib-service-24.02.0-1.el9.x86_64. 434 kB/s | 36 kB 00:00 2024-03-27T10:40:28.680 INFO:teuthology.orchestra.run.smithi097.stdout:(102/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 573 kB/s | 47 kB 00:00 2024-03-27T10:40:28.705 INFO:teuthology.orchestra.run.smithi097.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.1 MB/s | 94 kB 00:00 2024-03-27T10:40:28.830 INFO:teuthology.orchestra.run.smithi097.stdout:(104/174): postgresql-server-13.14-1.el9.x86_64 10 MB/s | 5.8 MB 00:00 2024-03-27T10:40:28.855 INFO:teuthology.orchestra.run.smithi097.stdout:(105/174): lua-devel-5.4.4-4.el9.x86_64.rpm 128 kB/s | 22 kB 00:00 2024-03-27T10:40:28.953 INFO:teuthology.orchestra.run.smithi177.stdout:(30/174): ceph-mgr-dashboard-19.0.0-2478.g15526 9.9 MB/s | 3.5 MB 00:00 2024-03-27T10:40:29.056 INFO:teuthology.orchestra.run.smithi097.stdout:(106/174): protobuf-compiler-3.14.0-13.el9.x86_ 2.4 MB/s | 863 kB 00:00 2024-03-27T10:40:29.240 INFO:teuthology.orchestra.run.smithi097.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 106 kB/s | 19 kB 00:00 2024-03-27T10:40:29.340 INFO:teuthology.orchestra.run.smithi097.stdout:(108/174): abseil-cpp-20211102.0-3.el9.x86_64.r 1.1 MB/s | 548 kB 00:00 2024-03-27T10:40:29.373 INFO:teuthology.orchestra.run.smithi097.stdout:(109/174): gperftools-libs-2.9.1-2.el9.x86_64.r 598 kB/s | 309 kB 00:00 2024-03-27T10:40:29.532 INFO:teuthology.orchestra.run.smithi097.stdout:(110/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 141 kB/s | 26 kB 00:00 2024-03-27T10:40:29.557 INFO:teuthology.orchestra.run.smithi097.stdout:(111/174): liboath-2.6.7-2.el9.x86_64.rpm 264 kB/s | 49 kB 00:00 2024-03-27T10:40:29.603 INFO:teuthology.orchestra.run.smithi177.stdout:(31/174): ceph-mgr-diskprediction-local-19.0.0- 11 MB/s | 7.4 MB 00:00 2024-03-27T10:40:29.649 INFO:teuthology.orchestra.run.smithi097.stdout:(112/174): libarrow-9.0.0-10.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-03-27T10:40:29.674 INFO:teuthology.orchestra.run.smithi097.stdout:(113/174): libunwind-1.6.2-1.el9.x86_64.rpm 477 kB/s | 67 kB 00:00 2024-03-27T10:40:29.686 INFO:teuthology.orchestra.run.smithi177.stdout:(32/174): ceph-mgr-modules-core-19.0.0-2478.g15 3.0 MB/s | 249 kB 00:00 2024-03-27T10:40:29.699 INFO:teuthology.orchestra.run.smithi097.stdout:(114/174): luarocks-3.9.2-1.el9.noarch.rpm 1.1 MB/s | 151 kB 00:00 2024-03-27T10:40:29.737 INFO:teuthology.orchestra.run.smithi177.stdout:(33/174): ceph-mgr-rook-19.0.0-2478.g155268c4.e 993 kB/s | 49 kB 00:00 2024-03-27T10:40:29.741 INFO:teuthology.orchestra.run.smithi097.stdout:(115/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 9.1 MB/s | 838 kB 00:00 2024-03-27T10:40:29.787 INFO:teuthology.orchestra.run.smithi177.stdout:(34/174): ceph-prometheus-alerts-19.0.0-2478.g1 335 kB/s | 16 kB 00:00 2024-03-27T10:40:29.789 INFO:teuthology.orchestra.run.smithi114.stdout:Extra Packages for Enterprise Linux 1.8 MB/s | 21 MB 00:11 2024-03-27T10:40:29.792 INFO:teuthology.orchestra.run.smithi097.stdout:(116/174): python3-asyncssh-2.13.2-5.el9.noarch 6.4 MB/s | 548 kB 00:00 2024-03-27T10:40:29.817 INFO:teuthology.orchestra.run.smithi097.stdout:(117/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 402 kB/s | 43 kB 00:00 2024-03-27T10:40:29.842 INFO:teuthology.orchestra.run.smithi097.stdout:(118/174): python3-cachetools-4.2.4-1.el9.noarc 322 kB/s | 32 kB 00:00 2024-03-27T10:40:29.846 INFO:teuthology.orchestra.run.smithi177.stdout:(35/174): ceph-volume-19.0.0-2478.g155268c4.el9 4.5 MB/s | 264 kB 00:00 2024-03-27T10:40:29.884 INFO:teuthology.orchestra.run.smithi097.stdout:(119/174): python3-certifi-2021.10.8-2.el9.noar 229 kB/s | 15 kB 00:00 2024-03-27T10:40:29.934 INFO:teuthology.orchestra.run.smithi097.stdout:(120/174): python3-cheroot-8.6.0-4.el9.noarch.r 1.5 MB/s | 172 kB 00:00 2024-03-27T10:40:29.962 INFO:teuthology.orchestra.run.smithi177.stdout:(36/174): cephadm-19.0.0-2478.g155268c4.el9.noa 6.4 MB/s | 762 kB 00:00 2024-03-27T10:40:29.968 INFO:teuthology.orchestra.run.smithi097.stdout:(121/174): python3-cherrypy-18.6.1-2.el9.noarch 2.8 MB/s | 358 kB 00:00 2024-03-27T10:40:30.013 INFO:teuthology.orchestra.run.smithi177.stdout:(37/174): cephfs-top-19.0.0-2478.g155268c4.el9. 371 kB/s | 18 kB 00:00 2024-03-27T10:40:30.018 INFO:teuthology.orchestra.run.smithi097.stdout:(122/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.7 MB/s | 303 kB 00:00 2024-03-27T10:40:30.043 INFO:teuthology.orchestra.run.smithi097.stdout:(123/174): python3-colorama-0.4.4-12.el9.noarch 338 kB/s | 35 kB 00:00 2024-03-27T10:40:30.077 INFO:teuthology.orchestra.run.smithi097.stdout:(124/174): python3-google-auth-2.28.2-1.el9.noa 1.8 MB/s | 203 kB 00:00 2024-03-27T10:40:30.177 INFO:teuthology.orchestra.run.smithi097.stdout:(125/174): python3-grpcio-1.46.7-10.el9.x86_64. 16 MB/s | 2.0 MB 00:00 2024-03-27T10:40:30.202 INFO:teuthology.orchestra.run.smithi097.stdout:(126/174): python3-grpcio-tools-1.46.7-10.el9.x 928 kB/s | 144 kB 00:00 2024-03-27T10:40:30.213 INFO:teuthology.orchestra.run.smithi177.stdout:(38/174): automake-1.16.2-8.el9.noarch.rpm 3.2 MB/s | 662 kB 00:00 2024-03-27T10:40:30.219 INFO:teuthology.orchestra.run.smithi097.stdout:(127/174): python3-jaraco-8.2.1-3.el9.noarch.rp 75 kB/s | 11 kB 00:00 2024-03-27T10:40:30.261 INFO:teuthology.orchestra.run.smithi097.stdout:(128/174): python3-jaraco-classes-3.2.1-5.el9.n 308 kB/s | 18 kB 00:00 2024-03-27T10:40:30.278 INFO:teuthology.orchestra.run.smithi097.stdout:(129/174): python3-jaraco-collections-3.0.0-8.e 312 kB/s | 23 kB 00:00 2024-03-27T10:40:30.280 INFO:teuthology.orchestra.run.smithi177.stdout:(39/174): bison-3.7.4-5.el9.x86_64.rpm 14 MB/s | 920 kB 00:00 2024-03-27T10:40:30.303 INFO:teuthology.orchestra.run.smithi097.stdout:(130/174): python3-jaraco-functools-3.5.0-2.el9 233 kB/s | 19 kB 00:00 2024-03-27T10:40:30.322 INFO:teuthology.orchestra.run.smithi177.stdout:(40/174): boost-program-options-1.75.0-8.el9.x8 2.5 MB/s | 106 kB 00:00 2024-03-27T10:40:30.354 INFO:teuthology.orchestra.run.smithi097.stdout:(131/174): python3-jaraco-text-3.2.0-6.el9.noar 266 kB/s | 20 kB 00:00 2024-03-27T10:40:30.364 INFO:teuthology.orchestra.run.smithi177.stdout:(41/174): flex-2.6.4-9.el9.x86_64.rpm 7.4 MB/s | 307 kB 00:00 2024-03-27T10:40:30.397 INFO:teuthology.orchestra.run.smithi177.stdout:(42/174): flexiblas-3.0.4-8.el9.x86_64.rpm 957 kB/s | 32 kB 00:00 2024-03-27T10:40:30.420 INFO:teuthology.orchestra.run.smithi097.stdout:(132/174): python3-kubernetes-26.1.0-2.el9.noar 7.3 MB/s | 1.0 MB 00:00 2024-03-27T10:40:30.445 INFO:teuthology.orchestra.run.smithi097.stdout:(133/174): python3-logutils-0.3.5-21.el9.noarch 327 kB/s | 46 kB 00:00 2024-03-27T10:40:30.474 INFO:teuthology.orchestra.run.smithi097.stdout:(134/174): python3-more-itertools-8.12.0-2.el9. 708 kB/s | 79 kB 00:00 2024-03-27T10:40:30.514 INFO:teuthology.orchestra.run.smithi177.stdout:(43/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 26 MB/s | 3.0 MB 00:00 2024-03-27T10:40:30.516 INFO:teuthology.orchestra.run.smithi097.stdout:(135/174): python3-natsort-7.1.1-5.el9.noarch.r 843 kB/s | 58 kB 00:00 2024-03-27T10:40:30.549 INFO:teuthology.orchestra.run.smithi097.stdout:(136/174): python3-pecan-1.4.2-3.el9.noarch.rpm 2.6 MB/s | 272 kB 00:00 2024-03-27T10:40:30.575 INFO:teuthology.orchestra.run.smithi097.stdout:(137/174): python3-portend-3.1.0-2.el9.noarch.r 164 kB/s | 16 kB 00:00 2024-03-27T10:40:30.600 INFO:teuthology.orchestra.run.smithi097.stdout:(138/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.1 MB/s | 90 kB 00:00 2024-03-27T10:40:30.642 INFO:teuthology.orchestra.run.smithi097.stdout:(139/174): python3-pyperclip-1.8.2-2.el9.noarch 433 kB/s | 28 kB 00:00 2024-03-27T10:40:30.668 INFO:teuthology.orchestra.run.smithi097.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 339 kB/s | 31 kB 00:00 2024-03-27T10:40:30.690 INFO:teuthology.orchestra.run.smithi177.stdout:(44/174): ceph-radosgw-19.0.0-2478.g155268c4.el 1.9 MB/s | 9.8 MB 00:05 2024-03-27T10:40:30.702 INFO:teuthology.orchestra.run.smithi097.stdout:(141/174): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-03-27T10:40:30.706 INFO:teuthology.orchestra.run.smithi177.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 90 kB/s | 17 kB 00:00 2024-03-27T10:40:30.723 INFO:teuthology.orchestra.run.smithi177.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 314 kB/s | 10 kB 00:00 2024-03-27T10:40:30.727 INFO:teuthology.orchestra.run.smithi097.stdout:(142/174): python3-rsa-4.9-2.el9.noarch.rpm 710 kB/s | 59 kB 00:00 2024-03-27T10:40:30.749 INFO:teuthology.orchestra.run.smithi177.stdout:(47/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.8 MB/s | 159 kB 00:00 2024-03-27T10:40:30.752 INFO:teuthology.orchestra.run.smithi097.stdout:(143/174): python3-tempora-5.0.0-2.el9.noarch.r 438 kB/s | 36 kB 00:00 2024-03-27T10:40:30.766 INFO:teuthology.orchestra.run.smithi177.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.1 MB/s | 44 kB 00:00 2024-03-27T10:40:30.777 INFO:teuthology.orchestra.run.smithi097.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 687 kB/s | 51 kB 00:00 2024-03-27T10:40:30.808 INFO:teuthology.orchestra.run.smithi177.stdout:(49/174): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 661 kB 00:00 2024-03-27T10:40:30.811 INFO:teuthology.orchestra.run.smithi097.stdout:(145/174): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-03-27T10:40:30.836 INFO:teuthology.orchestra.run.smithi097.stdout:(146/174): python3-websocket-client-1.2.3-2.el9 1.1 MB/s | 90 kB 00:00 2024-03-27T10:40:30.842 INFO:teuthology.orchestra.run.smithi177.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 8.2 MB/s | 578 kB 00:00 2024-03-27T10:40:30.867 INFO:teuthology.orchestra.run.smithi177.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 864 kB/s | 46 kB 00:00 2024-03-27T10:40:30.895 INFO:teuthology.orchestra.run.smithi097.stdout:(147/174): python3-werkzeug-2.0.3-3.el9.1.noarc 3.6 MB/s | 427 kB 00:00 2024-03-27T10:40:30.900 INFO:teuthology.orchestra.run.smithi177.stdout:(52/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.8 MB/s | 282 kB 00:00 2024-03-27T10:40:30.920 INFO:teuthology.orchestra.run.smithi097.stdout:(148/174): python3-zc-lockfile-2.0-10.el9.noarc 186 kB/s | 20 kB 00:00 2024-03-27T10:40:30.934 INFO:teuthology.orchestra.run.smithi177.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 2.9 MB/s | 187 kB 00:00 2024-03-27T10:40:30.945 INFO:teuthology.orchestra.run.smithi097.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 1.8 MB/s | 192 kB 00:00 2024-03-27T10:40:30.968 INFO:teuthology.orchestra.run.smithi177.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 3.3 MB/s | 217 kB 00:00 2024-03-27T10:40:30.993 INFO:teuthology.orchestra.run.smithi177.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 614 kB/s | 34 kB 00:00 2024-03-27T10:40:31.012 INFO:teuthology.orchestra.run.smithi097.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-03-27T10:40:31.118 INFO:teuthology.orchestra.run.smithi177.stdout:(56/174): openblas-openmp-0.3.21-2.el9.x86_64.r 32 MB/s | 4.7 MB 00:00 2024-03-27T10:40:31.160 INFO:teuthology.orchestra.run.smithi177.stdout:(57/174): protobuf-3.14.0-13.el9.x86_64.rpm 6.7 MB/s | 1.0 MB 00:00 2024-03-27T10:40:31.185 INFO:teuthology.orchestra.run.smithi177.stdout:(58/174): python3-attrs-20.3.0-7.el9.noarch.rpm 1.2 MB/s | 78 kB 00:00 2024-03-27T10:40:31.296 INFO:teuthology.orchestra.run.smithi097.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 1.8 MB/s | 321 kB 00:00 2024-03-27T10:40:31.328 INFO:teuthology.orchestra.run.smithi177.stdout:(59/174): python3-babel-2.9.1-2.el9.noarch.rpm 35 MB/s | 5.8 MB 00:00 2024-03-27T10:40:31.353 INFO:teuthology.orchestra.run.smithi177.stdout:(60/174): python3-jsonpatch-1.21-16.el9.noarch. 162 kB/s | 24 kB 00:00 2024-03-27T10:40:31.386 INFO:teuthology.orchestra.run.smithi177.stdout:(61/174): python3-jsonpointer-2.0-4.el9.noarch. 564 kB/s | 18 kB 00:00 2024-03-27T10:40:31.411 INFO:teuthology.orchestra.run.smithi177.stdout:(62/174): python3-mako-1.1.4-6.el9.noarch.rpm 2.5 MB/s | 144 kB 00:00 2024-03-27T10:40:31.437 INFO:teuthology.orchestra.run.smithi177.stdout:(63/174): python3-markupsafe-1.1.1-12.el9.x86_6 655 kB/s | 32 kB 00:00 2024-03-27T10:40:31.595 INFO:teuthology.orchestra.run.smithi177.stdout:(64/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 26 MB/s | 4.8 MB 00:00 2024-03-27T10:40:31.620 INFO:teuthology.orchestra.run.smithi177.stdout:(65/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.3 MB/s | 217 kB 00:00 2024-03-27T10:40:31.662 INFO:teuthology.orchestra.run.smithi177.stdout:(66/174): python3-packaging-20.9-5.el9.noarch.r 1.7 MB/s | 69 kB 00:00 2024-03-27T10:40:31.688 INFO:teuthology.orchestra.run.smithi177.stdout:(67/174): python3-oauthlib-3.1.1-5.el9.noarch.r 2.6 MB/s | 174 kB 00:00 2024-03-27T10:40:31.713 INFO:teuthology.orchestra.run.smithi177.stdout:(68/174): python3-prettytable-0.7.2-27.el9.noar 857 kB/s | 41 kB 00:00 2024-03-27T10:40:31.747 INFO:teuthology.orchestra.run.smithi177.stdout:(69/174): python3-protobuf-3.14.0-13.el9.noarch 4.1 MB/s | 238 kB 00:00 2024-03-27T10:40:31.763 INFO:teuthology.orchestra.run.smithi097.stdout:(152/174): librbd1-19.0.0-2478.g155268c4.el9.x8 3.9 MB/s | 3.1 MB 00:00 2024-03-27T10:40:31.772 INFO:teuthology.orchestra.run.smithi177.stdout:(70/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 132 kB 00:00 2024-03-27T10:40:31.805 INFO:teuthology.orchestra.run.smithi177.stdout:(71/174): python3-pyasn1-modules-0.4.8-6.el9.no 3.6 MB/s | 211 kB 00:00 2024-03-27T10:40:31.831 INFO:teuthology.orchestra.run.smithi177.stdout:(72/174): python3-pytz-2021.1-5.el9.noarch.rpm 844 kB/s | 47 kB 00:00 2024-03-27T10:40:31.855 INFO:teuthology.orchestra.run.smithi097.stdout:(153/174): librados2-19.0.0-2478.g155268c4.el9. 3.8 MB/s | 3.4 MB 00:00 2024-03-27T10:40:31.856 INFO:teuthology.orchestra.run.smithi177.stdout:(73/174): python3-requests-oauthlib-1.3.0-12.el 874 kB/s | 43 kB 00:00 2024-03-27T10:40:31.938 INFO:teuthology.orchestra.run.smithi097.stdout:(154/174): NetworkManager-1.47.2-1.el9.x86_64.r 3.6 MB/s | 2.3 MB 00:00 2024-03-27T10:40:32.030 INFO:teuthology.orchestra.run.smithi097.stdout:(155/174): NetworkManager-tui-1.47.2-1.el9.x86_ 2.7 MB/s | 248 kB 00:00 2024-03-27T10:40:32.189 INFO:teuthology.orchestra.run.smithi097.stdout:(156/174): NetworkManager-team-1.47.2-1.el9.x86 126 kB/s | 42 kB 00:00 2024-03-27T10:40:32.198 INFO:teuthology.orchestra.run.smithi177.stdout:(74/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 43 MB/s | 16 MB 00:00 2024-03-27T10:40:32.223 INFO:teuthology.orchestra.run.smithi097.stdout:(157/174): acl-2.3.1-4.el9.x86_64.rpm 373 kB/s | 71 kB 00:00 2024-03-27T10:40:32.231 INFO:teuthology.orchestra.run.smithi177.stdout:(75/174): python3-toml-0.10.2-6.el9.noarch.rpm 107 kB/s | 39 kB 00:00 2024-03-27T10:40:32.273 INFO:teuthology.orchestra.run.smithi177.stdout:(76/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.0 MB/s | 41 kB 00:00 2024-03-27T10:40:32.298 INFO:teuthology.orchestra.run.smithi097.stdout:(158/174): elfutils-debuginfod-client-0.190-2.e 338 kB/s | 36 kB 00:00 2024-03-27T10:40:32.307 INFO:teuthology.orchestra.run.smithi177.stdout:(77/174): socat-1.7.4.1-5.el9.x86_64.rpm 4.0 MB/s | 300 kB 00:00 2024-03-27T10:40:32.332 INFO:teuthology.orchestra.run.smithi097.stdout:(159/174): elfutils-libelf-0.190-2.el9.x86_64.r 1.7 MB/s | 192 kB 00:00 2024-03-27T10:40:32.332 INFO:teuthology.orchestra.run.smithi177.stdout:(78/174): uuid-1.6.2-55.el9.x86_64.rpm 1.0 MB/s | 56 kB 00:00 2024-03-27T10:40:32.358 INFO:teuthology.orchestra.run.smithi177.stdout:(79/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 63 kB 00:00 2024-03-27T10:40:32.383 INFO:teuthology.orchestra.run.smithi177.stdout:(80/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 902 kB/s | 44 kB 00:00 2024-03-27T10:40:32.390 INFO:teuthology.orchestra.run.smithi097.stdout:(160/174): libacl-2.3.1-4.el9.x86_64.rpm 401 kB/s | 23 kB 00:00 2024-03-27T10:40:32.402 INFO:teuthology.orchestra.run.smithi177.stdout:(81/174): fuse-2.9.9-15.el9.x86_64.rpm 1.9 MB/s | 79 kB 00:00 2024-03-27T10:40:32.424 INFO:teuthology.orchestra.run.smithi097.stdout:(161/174): elfutils-libs-0.190-2.el9.x86_64.rpm 2.0 MB/s | 256 kB 00:00 2024-03-27T10:40:32.433 INFO:teuthology.orchestra.run.smithi177.stdout:(82/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.8 MB/s | 241 kB 00:00 2024-03-27T10:40:32.457 INFO:teuthology.orchestra.run.smithi097.stdout:(162/174): libblkid-2.37.4-18.el9.x86_64.rpm 1.6 MB/s | 108 kB 00:00 2024-03-27T10:40:32.484 INFO:teuthology.orchestra.run.smithi177.stdout:(83/174): python3-cryptography-36.0.1-4.el9.x86 14 MB/s | 1.1 MB 00:00 2024-03-27T10:40:32.491 INFO:teuthology.orchestra.run.smithi097.stdout:(163/174): libfdisk-2.37.4-18.el9.x86_64.rpm 2.3 MB/s | 155 kB 00:00 2024-03-27T10:40:32.524 INFO:teuthology.orchestra.run.smithi097.stdout:(164/174): libmount-2.37.4-18.el9.x86_64.rpm 2.0 MB/s | 136 kB 00:00 2024-03-27T10:40:32.534 INFO:teuthology.orchestra.run.smithi177.stdout:(84/174): python3-ply-3.11-14.el9.noarch.rpm 1.1 MB/s | 103 kB 00:00 2024-03-27T10:40:32.550 INFO:teuthology.orchestra.run.smithi097.stdout:(165/174): libsmartcols-2.37.4-18.el9.x86_64.rp 1.1 MB/s | 64 kB 00:00 2024-03-27T10:40:32.559 INFO:teuthology.orchestra.run.smithi177.stdout:(85/174): python3-pycparser-2.20-6.el9.noarch.r 1.6 MB/s | 124 kB 00:00 2024-03-27T10:40:32.593 INFO:teuthology.orchestra.run.smithi177.stdout:(86/174): python3-urllib3-1.26.5-5.el9.noarch.r 3.2 MB/s | 187 kB 00:00 2024-03-27T10:40:32.608 INFO:teuthology.orchestra.run.smithi097.stdout:(166/174): NetworkManager-libnm-1.47.2-1.el9.x8 2.2 MB/s | 1.8 MB 00:00 2024-03-27T10:40:32.634 INFO:teuthology.orchestra.run.smithi097.stdout:(167/174): libuuid-2.37.4-18.el9.x86_64.rpm 265 kB/s | 29 kB 00:00 2024-03-27T10:40:32.718 INFO:teuthology.orchestra.run.smithi177.stdout:(87/174): jq-1.6-16.el9.x86_64.rpm 1.2 MB/s | 187 kB 00:00 2024-03-27T10:40:32.777 INFO:teuthology.orchestra.run.smithi177.stdout:(88/174): libgfortran-11.4.1-3.el9.x86_64.rpm 4.3 MB/s | 803 kB 00:00 2024-03-27T10:40:32.819 INFO:teuthology.orchestra.run.smithi177.stdout:(89/174): libquadmath-11.4.1-3.el9.x86_64.rpm 1.9 MB/s | 195 kB 00:00 2024-03-27T10:40:32.852 INFO:teuthology.orchestra.run.smithi177.stdout:(90/174): python3-requests-2.25.1-8.el9.noarch. 1.6 MB/s | 125 kB 00:00 2024-03-27T10:40:32.859 INFO:teuthology.orchestra.run.smithi097.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 3.9 MB/s | 1.2 MB 00:00 2024-03-27T10:40:32.953 INFO:teuthology.orchestra.run.smithi177.stdout:(91/174): elfutils-libelf-devel-0.190-2.el9.x86 162 kB/s | 21 kB 00:00 2024-03-27T10:40:32.986 INFO:teuthology.orchestra.run.smithi177.stdout:(92/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 191 kB/s | 25 kB 00:00 2024-03-27T10:40:33.010 INFO:teuthology.orchestra.run.smithi097.stdout:(169/174): util-linux-2.37.4-18.el9.x86_64.rpm 6.1 MB/s | 2.3 MB 00:00 2024-03-27T10:40:33.043 INFO:teuthology.orchestra.run.smithi097.stdout:(170/174): util-linux-core-2.37.4-18.el9.x86_64 2.5 MB/s | 465 kB 00:00 2024-03-27T10:40:33.068 INFO:teuthology.orchestra.run.smithi097.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 539 kB/s | 31 kB 00:00 2024-03-27T10:40:33.136 INFO:teuthology.orchestra.run.smithi097.stdout:(172/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 4.1 MB/s | 2.2 MB 00:00 2024-03-27T10:40:33.319 INFO:teuthology.orchestra.run.smithi097.stdout:(173/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 3.9 MB/s | 1.1 MB 00:00 2024-03-27T10:40:33.361 INFO:teuthology.orchestra.run.smithi097.stdout:(174/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 255 kB/s | 74 kB 00:00 2024-03-27T10:40:33.376 INFO:teuthology.orchestra.run.smithi097.stdout:-------------------------------------------------------------------------------- 2024-03-27T10:40:33.376 INFO:teuthology.orchestra.run.smithi097.stdout:Total 14 MB/s | 227 MB 00:16 2024-03-27T10:40:33.395 INFO:teuthology.orchestra.run.smithi177.stdout:(93/174): postgresql-13.14-1.el9.x86_64.rpm 3.9 MB/s | 1.6 MB 00:00 2024-03-27T10:40:33.579 INFO:teuthology.orchestra.run.smithi177.stdout:(94/174): postgresql-contrib-13.14-1.el9.x86_64 4.9 MB/s | 911 kB 00:00 2024-03-27T10:40:33.646 INFO:teuthology.orchestra.run.smithi177.stdout:(95/174): postgresql-private-libs-13.14-1.el9.x 2.0 MB/s | 133 kB 00:00 2024-03-27T10:40:33.762 INFO:teuthology.orchestra.run.smithi177.stdout:(96/174): openssl-devel-3.0.7-27.el9.x86_64.rpm 5.1 MB/s | 4.1 MB 00:00 2024-03-27T10:40:33.829 INFO:teuthology.orchestra.run.smithi177.stdout:(97/174): python3-devel-3.9.17-2.el9.x86_64.rpm 3.6 MB/s | 245 kB 00:00 2024-03-27T10:40:33.896 INFO:teuthology.orchestra.run.smithi177.stdout:(98/174): python3-jinja2-2.11.3-5.el9.noarch.rp 3.7 MB/s | 248 kB 00:00 2024-03-27T10:40:33.964 INFO:teuthology.orchestra.run.smithi177.stdout:(99/174): qatlib-24.02.0-1.el9.x86_64.rpm 3.3 MB/s | 221 kB 00:00 2024-03-27T10:40:34.022 INFO:teuthology.orchestra.run.smithi177.stdout:(100/174): qatlib-service-24.02.0-1.el9.x86_64. 629 kB/s | 36 kB 00:00 2024-03-27T10:40:34.081 INFO:teuthology.orchestra.run.smithi177.stdout:(101/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 824 kB/s | 47 kB 00:00 2024-03-27T10:40:34.147 INFO:teuthology.orchestra.run.smithi177.stdout:(102/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.4 MB/s | 94 kB 00:00 2024-03-27T10:40:34.381 INFO:teuthology.orchestra.run.smithi177.stdout:(103/174): postgresql-server-13.14-1.el9.x86_64 7.9 MB/s | 5.8 MB 00:00 2024-03-27T10:40:34.465 INFO:teuthology.orchestra.run.smithi177.stdout:(104/174): lua-devel-5.4.4-4.el9.x86_64.rpm 71 kB/s | 22 kB 00:00 2024-03-27T10:40:34.723 INFO:teuthology.orchestra.run.smithi177.stdout:(105/174): protobuf-compiler-3.14.0-13.el9.x86_ 2.5 MB/s | 863 kB 00:00 2024-03-27T10:40:35.043 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction check 2024-03-27T10:40:35.123 INFO:teuthology.orchestra.run.smithi177.stdout:(106/174): abseil-cpp-20211102.0-3.el9.x86_64.r 932 kB/s | 548 kB 00:00 2024-03-27T10:40:35.241 INFO:teuthology.orchestra.run.smithi177.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 167 kB/s | 19 kB 00:00 2024-03-27T10:40:35.274 INFO:teuthology.orchestra.run.smithi177.stdout:(108/174): gperftools-libs-2.9.1-2.el9.x86_64.r 562 kB/s | 309 kB 00:00 2024-03-27T10:40:35.407 INFO:teuthology.orchestra.run.smithi177.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 197 kB/s | 26 kB 00:00 2024-03-27T10:40:35.424 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction check succeeded. 2024-03-27T10:40:35.424 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction test 2024-03-27T10:40:35.525 INFO:teuthology.orchestra.run.smithi177.stdout:(110/174): liboath-2.6.7-2.el9.x86_64.rpm 415 kB/s | 49 kB 00:00 2024-03-27T10:40:35.557 INFO:teuthology.orchestra.run.smithi114.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-03-27T10:40:36.100 INFO:teuthology.orchestra.run.smithi177.stdout:(111/174): ceph-test-19.0.0-2478.g155268c4.el9. 4.8 MB/s | 48 MB 00:09 2024-03-27T10:40:36.125 INFO:teuthology.orchestra.run.smithi177.stdout:(112/174): libunwind-1.6.2-1.el9.x86_64.rpm 112 kB/s | 67 kB 00:00 2024-03-27T10:40:36.251 INFO:teuthology.orchestra.run.smithi177.stdout:(113/174): libarrow-9.0.0-10.el9.x86_64.rpm 4.4 MB/s | 4.4 MB 00:01 2024-03-27T10:40:36.509 INFO:teuthology.orchestra.run.smithi177.stdout:(114/174): python3-asyncssh-2.13.2-5.el9.noarch 2.1 MB/s | 548 kB 00:00 2024-03-27T10:40:36.609 INFO:teuthology.orchestra.run.smithi177.stdout:(115/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 1.7 MB/s | 838 kB 00:00 2024-03-27T10:40:36.635 INFO:teuthology.orchestra.run.smithi177.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 349 kB/s | 43 kB 00:00 2024-03-27T10:40:36.686 INFO:teuthology.orchestra.run.smithi177.stdout:(117/174): luarocks-3.9.2-1.el9.noarch.rpm 259 kB/s | 151 kB 00:00 2024-03-27T10:40:36.702 INFO:teuthology.orchestra.run.smithi177.stdout:(118/174): python3-cachetools-4.2.4-1.el9.noarc 352 kB/s | 32 kB 00:00 2024-03-27T10:40:36.727 INFO:teuthology.orchestra.run.smithi177.stdout:(119/174): python3-certifi-2021.10.8-2.el9.noar 164 kB/s | 15 kB 00:00 2024-03-27T10:40:36.844 INFO:teuthology.orchestra.run.smithi177.stdout:(120/174): python3-cherrypy-18.6.1-2.el9.noarch 2.5 MB/s | 358 kB 00:00 2024-03-27T10:40:36.869 INFO:teuthology.orchestra.run.smithi177.stdout:(121/174): python3-cheroot-8.6.0-4.el9.noarch.r 936 kB/s | 172 kB 00:00 2024-03-27T10:40:36.903 INFO:teuthology.orchestra.run.smithi177.stdout:(122/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.7 MB/s | 303 kB 00:00 2024-03-27T10:40:36.953 INFO:teuthology.orchestra.run.smithi177.stdout:(123/174): python3-colorama-0.4.4-12.el9.noarch 327 kB/s | 35 kB 00:00 2024-03-27T10:40:36.987 INFO:teuthology.orchestra.run.smithi177.stdout:(124/174): python3-google-auth-2.28.2-1.el9.noa 1.7 MB/s | 203 kB 00:00 2024-03-27T10:40:37.079 INFO:teuthology.orchestra.run.smithi177.stdout:(125/174): python3-grpcio-tools-1.46.7-10.el9.x 1.1 MB/s | 144 kB 00:00 2024-03-27T10:40:37.121 INFO:teuthology.orchestra.run.smithi177.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 80 kB/s | 11 kB 00:00 2024-03-27T10:40:37.179 INFO:teuthology.orchestra.run.smithi177.stdout:(127/174): python3-jaraco-classes-3.2.1-5.el9.n 178 kB/s | 18 kB 00:00 2024-03-27T10:40:37.221 INFO:teuthology.orchestra.run.smithi177.stdout:(128/174): python3-jaraco-collections-3.0.0-8.e 233 kB/s | 23 kB 00:00 2024-03-27T10:40:37.279 INFO:teuthology.orchestra.run.smithi177.stdout:(129/174): python3-grpcio-1.46.7-10.el9.x86_64. 5.4 MB/s | 2.0 MB 00:00 2024-03-27T10:40:37.336 INFO:teuthology.orchestra.run.smithi177.stdout:(130/174): python3-jaraco-functools-3.5.0-2.el9 125 kB/s | 19 kB 00:00 2024-03-27T10:40:37.355 INFO:teuthology.orchestra.run.smithi177.stdout:(131/174): python3-jaraco-text-3.2.0-6.el9.noar 148 kB/s | 20 kB 00:00 2024-03-27T10:40:37.430 INFO:teuthology.orchestra.run.smithi177.stdout:(132/174): python3-logutils-0.3.5-21.el9.noarch 495 kB/s | 46 kB 00:00 2024-03-27T10:40:37.489 INFO:teuthology.orchestra.run.smithi177.stdout:(133/174): python3-more-itertools-8.12.0-2.el9. 593 kB/s | 79 kB 00:00 2024-03-27T10:40:37.548 INFO:teuthology.orchestra.run.smithi177.stdout:(134/174): python3-natsort-7.1.1-5.el9.noarch.r 496 kB/s | 58 kB 00:00 2024-03-27T10:40:37.589 INFO:teuthology.orchestra.run.smithi177.stdout:(135/174): python3-kubernetes-26.1.0-2.el9.noar 3.3 MB/s | 1.0 MB 00:00 2024-03-27T10:40:37.623 INFO:teuthology.orchestra.run.smithi177.stdout:(136/174): python3-pecan-1.4.2-3.el9.noarch.rpm 2.0 MB/s | 272 kB 00:00 2024-03-27T10:40:37.657 INFO:teuthology.orchestra.run.smithi177.stdout:(137/174): python3-portend-3.1.0-2.el9.noarch.r 152 kB/s | 16 kB 00:00 2024-03-27T10:40:37.707 INFO:teuthology.orchestra.run.smithi177.stdout:(138/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 767 kB/s | 90 kB 00:00 2024-03-27T10:40:37.732 INFO:teuthology.orchestra.run.smithi177.stdout:(139/174): python3-pyperclip-1.8.2-2.el9.noarch 260 kB/s | 28 kB 00:00 2024-03-27T10:40:37.858 INFO:teuthology.orchestra.run.smithi177.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 153 kB/s | 31 kB 00:00 2024-03-27T10:40:37.883 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction test succeeded. 2024-03-27T10:40:37.883 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction 2024-03-27T10:40:37.950 INFO:teuthology.orchestra.run.smithi177.stdout:(141/174): python3-rsa-4.9-2.el9.noarch.rpm 272 kB/s | 59 kB 00:00 2024-03-27T10:40:37.976 INFO:teuthology.orchestra.run.smithi177.stdout:(142/174): python3-tempora-5.0.0-2.el9.noarch.r 308 kB/s | 36 kB 00:00 2024-03-27T10:40:38.001 INFO:teuthology.orchestra.run.smithi177.stdout:(143/174): python3-routes-2.5.1-5.el9.noarch.rp 643 kB/s | 188 kB 00:00 2024-03-27T10:40:38.052 INFO:teuthology.orchestra.run.smithi177.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 510 kB/s | 51 kB 00:00 2024-03-27T10:40:38.110 INFO:teuthology.orchestra.run.smithi177.stdout:(145/174): python3-webob-1.8.7-6.el9.noarch.rpm 1.7 MB/s | 230 kB 00:00 2024-03-27T10:40:38.136 INFO:teuthology.orchestra.run.smithi177.stdout:(146/174): python3-websocket-client-1.2.3-2.el9 670 kB/s | 90 kB 00:00 2024-03-27T10:40:38.203 INFO:teuthology.orchestra.run.smithi177.stdout:(147/174): python3-werkzeug-2.0.3-3.el9.1.noarc 2.8 MB/s | 427 kB 00:00 2024-03-27T10:40:38.214 INFO:teuthology.orchestra.run.smithi114.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-27T10:40:38.216 INFO:teuthology.orchestra.run.smithi114.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-27T10:40:38.228 INFO:teuthology.orchestra.run.smithi177.stdout:(148/174): python3-zc-lockfile-2.0-10.el9.noarc 171 kB/s | 20 kB 00:00 2024-03-27T10:40:38.238 INFO:teuthology.orchestra.run.smithi114.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-27T10:40:38.239 INFO:teuthology.orchestra.run.smithi114.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-27T10:40:38.241 INFO:teuthology.orchestra.run.smithi114.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-27T10:40:38.242 INFO:teuthology.orchestra.run.smithi114.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-27T10:40:38.245 INFO:teuthology.orchestra.run.smithi114.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-27T10:40:38.249 INFO:teuthology.orchestra.run.smithi114.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-27T10:40:38.250 INFO:teuthology.orchestra.run.smithi114.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-27T10:40:38.279 INFO:teuthology.orchestra.run.smithi177.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 1.3 MB/s | 192 kB 00:00 2024-03-27T10:40:38.338 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-03-27T10:40:38.359 INFO:teuthology.orchestra.run.smithi114.stdout:============================================================================================ 2024-03-27T10:40:38.359 INFO:teuthology.orchestra.run.smithi114.stdout: Package Arch Version Repository Size 2024-03-27T10:40:38.359 INFO:teuthology.orchestra.run.smithi114.stdout:============================================================================================ 2024-03-27T10:40:38.359 INFO:teuthology.orchestra.run.smithi114.stdout:Installing: 2024-03-27T10:40:38.359 INFO:teuthology.orchestra.run.smithi114.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-27T10:40:38.359 INFO:teuthology.orchestra.run.smithi114.stdout: ceph x86_64 2:19.0.0-2478.g155268c4.el9 ceph 6.6 k 2024-03-27T10:40:38.359 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-base x86_64 2:19.0.0-2478.g155268c4.el9 ceph 5.4 M 2024-03-27T10:40:38.359 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-fuse x86_64 2:19.0.0-2478.g155268c4.el9 ceph 890 k 2024-03-27T10:40:38.359 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2478.g155268c4.el9 ceph 145 k 2024-03-27T10:40:38.360 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr x86_64 2:19.0.0-2478.g155268c4.el9 ceph 1.5 M 2024-03-27T10:40:38.360 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2478.g155268c4.el9 ceph-noarch 141 k 2024-03-27T10:40:38.360 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2478.g155268c4.el9 ceph-noarch 3.5 M 2024-03-27T10:40:38.360 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2478.g155268c4.el9 ceph-noarch 7.4 M 2024-03-27T10:40:38.360 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-rook noarch 2:19.0.0-2478.g155268c4.el9 ceph-noarch 49 k 2024-03-27T10:40:38.360 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-radosgw x86_64 2:19.0.0-2478.g155268c4.el9 ceph 9.8 M 2024-03-27T10:40:38.360 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-test x86_64 2:19.0.0-2478.g155268c4.el9 ceph 48 M 2024-03-27T10:40:38.360 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-volume noarch 2:19.0.0-2478.g155268c4.el9 ceph-noarch 264 k 2024-03-27T10:40:38.360 INFO:teuthology.orchestra.run.smithi114.stdout: cephadm noarch 2:19.0.0-2478.g155268c4.el9 ceph-noarch 762 k 2024-03-27T10:40:38.360 INFO:teuthology.orchestra.run.smithi114.stdout: cephfs-mirror x86_64 2:19.0.0-2478.g155268c4.el9 ceph 221 k 2024-03-27T10:40:38.360 INFO:teuthology.orchestra.run.smithi114.stdout: cephfs-shell x86_64 2:19.0.0-2478.g155268c4.el9 ceph 27 k 2024-03-27T10:40:38.360 INFO:teuthology.orchestra.run.smithi114.stdout: cephfs-top noarch 2:19.0.0-2478.g155268c4.el9 ceph-noarch 18 k 2024-03-27T10:40:38.360 INFO:teuthology.orchestra.run.smithi114.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-27T10:40:38.360 INFO:teuthology.orchestra.run.smithi114.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-27T10:40:38.361 INFO:teuthology.orchestra.run.smithi114.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-27T10:40:38.361 INFO:teuthology.orchestra.run.smithi114.stdout: libcephfs-devel x86_64 2:19.0.0-2478.g155268c4.el9 ceph 32 k 2024-03-27T10:40:38.361 INFO:teuthology.orchestra.run.smithi114.stdout: libcephfs2 x86_64 2:19.0.0-2478.g155268c4.el9 ceph 746 k 2024-03-27T10:40:38.361 INFO:teuthology.orchestra.run.smithi114.stdout: librados-devel x86_64 2:19.0.0-2478.g155268c4.el9 ceph 127 k 2024-03-27T10:40:38.361 INFO:teuthology.orchestra.run.smithi114.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-27T10:40:38.361 INFO:teuthology.orchestra.run.smithi114.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-27T10:40:38.361 INFO:teuthology.orchestra.run.smithi114.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-27T10:40:38.361 INFO:teuthology.orchestra.run.smithi114.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-03-27T10:40:38.361 INFO:teuthology.orchestra.run.smithi114.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-03-27T10:40:38.361 INFO:teuthology.orchestra.run.smithi114.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-03-27T10:40:38.361 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cephfs x86_64 2:19.0.0-2478.g155268c4.el9 ceph 162 k 2024-03-27T10:40:38.361 INFO:teuthology.orchestra.run.smithi114.stdout: python3-rados x86_64 2:19.0.0-2478.g155268c4.el9 ceph 320 k 2024-03-27T10:40:38.361 INFO:teuthology.orchestra.run.smithi114.stdout: python3-rbd x86_64 2:19.0.0-2478.g155268c4.el9 ceph 298 k 2024-03-27T10:40:38.362 INFO:teuthology.orchestra.run.smithi114.stdout: python3-rgw x86_64 2:19.0.0-2478.g155268c4.el9 ceph 99 k 2024-03-27T10:40:38.362 INFO:teuthology.orchestra.run.smithi114.stdout: rbd-fuse x86_64 2:19.0.0-2478.g155268c4.el9 ceph 85 k 2024-03-27T10:40:38.362 INFO:teuthology.orchestra.run.smithi114.stdout: rbd-mirror x86_64 2:19.0.0-2478.g155268c4.el9 ceph 3.1 M 2024-03-27T10:40:38.362 INFO:teuthology.orchestra.run.smithi114.stdout: rbd-nbd x86_64 2:19.0.0-2478.g155268c4.el9 ceph 170 k 2024-03-27T10:40:38.362 INFO:teuthology.orchestra.run.smithi114.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-27T10:40:38.362 INFO:teuthology.orchestra.run.smithi114.stdout:Upgrading: 2024-03-27T10:40:38.362 INFO:teuthology.orchestra.run.smithi114.stdout: NetworkManager x86_64 1:1.47.2-1.el9 baseos 2.3 M 2024-03-27T10:40:38.362 INFO:teuthology.orchestra.run.smithi114.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 baseos 1.8 M 2024-03-27T10:40:38.362 INFO:teuthology.orchestra.run.smithi114.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 baseos 42 k 2024-03-27T10:40:38.363 INFO:teuthology.orchestra.run.smithi114.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 baseos 248 k 2024-03-27T10:40:38.363 INFO:teuthology.orchestra.run.smithi114.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-27T10:40:38.363 INFO:teuthology.orchestra.run.smithi114.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-27T10:40:38.363 INFO:teuthology.orchestra.run.smithi114.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-27T10:40:38.363 INFO:teuthology.orchestra.run.smithi114.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-27T10:40:38.363 INFO:teuthology.orchestra.run.smithi114.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-27T10:40:38.363 INFO:teuthology.orchestra.run.smithi114.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-27T10:40:38.363 INFO:teuthology.orchestra.run.smithi114.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-27T10:40:38.363 INFO:teuthology.orchestra.run.smithi114.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-27T10:40:38.363 INFO:teuthology.orchestra.run.smithi114.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-27T10:40:38.363 INFO:teuthology.orchestra.run.smithi114.stdout: librados2 x86_64 2:19.0.0-2478.g155268c4.el9 ceph 3.4 M 2024-03-27T10:40:38.363 INFO:teuthology.orchestra.run.smithi114.stdout: librbd1 x86_64 2:19.0.0-2478.g155268c4.el9 ceph 3.1 M 2024-03-27T10:40:38.363 INFO:teuthology.orchestra.run.smithi114.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-27T10:40:38.364 INFO:teuthology.orchestra.run.smithi114.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-27T10:40:38.364 INFO:teuthology.orchestra.run.smithi114.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-27T10:40:38.364 INFO:teuthology.orchestra.run.smithi114.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-27T10:40:38.364 INFO:teuthology.orchestra.run.smithi114.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-27T10:40:38.364 INFO:teuthology.orchestra.run.smithi114.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-27T10:40:38.364 INFO:teuthology.orchestra.run.smithi114.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-27T10:40:38.364 INFO:teuthology.orchestra.run.smithi114.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-27T10:40:38.364 INFO:teuthology.orchestra.run.smithi114.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-27T10:40:38.364 INFO:teuthology.orchestra.run.smithi114.stdout:Installing dependencies: 2024-03-27T10:40:38.364 INFO:teuthology.orchestra.run.smithi114.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-27T10:40:38.364 INFO:teuthology.orchestra.run.smithi114.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-27T10:40:38.364 INFO:teuthology.orchestra.run.smithi114.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-27T10:40:38.364 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-common x86_64 2:19.0.0-2478.g155268c4.el9 ceph 21 M 2024-03-27T10:40:38.365 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2478.g155268c4.el9 ceph-noarch 34 k 2024-03-27T10:40:38.365 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mds x86_64 2:19.0.0-2478.g155268c4.el9 ceph 2.4 M 2024-03-27T10:40:38.365 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2478.g155268c4.el9 ceph-noarch 249 k 2024-03-27T10:40:38.365 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mon x86_64 2:19.0.0-2478.g155268c4.el9 ceph 4.7 M 2024-03-27T10:40:38.365 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-osd x86_64 2:19.0.0-2478.g155268c4.el9 ceph 17 M 2024-03-27T10:40:38.365 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2478.g155268c4.el9 ceph-noarch 16 k 2024-03-27T10:40:38.365 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-selinux x86_64 2:19.0.0-2478.g155268c4.el9 ceph 25 k 2024-03-27T10:40:38.365 INFO:teuthology.orchestra.run.smithi114.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-27T10:40:38.365 INFO:teuthology.orchestra.run.smithi114.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-27T10:40:38.365 INFO:teuthology.orchestra.run.smithi114.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-27T10:40:38.365 INFO:teuthology.orchestra.run.smithi114.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-27T10:40:38.365 INFO:teuthology.orchestra.run.smithi114.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-27T10:40:38.365 INFO:teuthology.orchestra.run.smithi114.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-27T10:40:38.365 INFO:teuthology.orchestra.run.smithi114.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-27T10:40:38.366 INFO:teuthology.orchestra.run.smithi114.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-27T10:40:38.366 INFO:teuthology.orchestra.run.smithi114.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-27T10:40:38.366 INFO:teuthology.orchestra.run.smithi114.stdout: libcephsqlite x86_64 2:19.0.0-2478.g155268c4.el9 ceph 163 k 2024-03-27T10:40:38.366 INFO:teuthology.orchestra.run.smithi114.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-27T10:40:38.366 INFO:teuthology.orchestra.run.smithi114.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-27T10:40:38.366 INFO:teuthology.orchestra.run.smithi114.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-27T10:40:38.366 INFO:teuthology.orchestra.run.smithi114.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-27T10:40:38.366 INFO:teuthology.orchestra.run.smithi114.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-27T10:40:38.366 INFO:teuthology.orchestra.run.smithi114.stdout: libradosstriper1 x86_64 2:19.0.0-2478.g155268c4.el9 ceph 501 k 2024-03-27T10:40:38.366 INFO:teuthology.orchestra.run.smithi114.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-27T10:40:38.366 INFO:teuthology.orchestra.run.smithi114.stdout: librgw2 x86_64 2:19.0.0-2478.g155268c4.el9 ceph 5.1 M 2024-03-27T10:40:38.366 INFO:teuthology.orchestra.run.smithi114.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-27T10:40:38.366 INFO:teuthology.orchestra.run.smithi114.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-27T10:40:38.367 INFO:teuthology.orchestra.run.smithi114.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-27T10:40:38.367 INFO:teuthology.orchestra.run.smithi114.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-27T10:40:38.367 INFO:teuthology.orchestra.run.smithi114.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-27T10:40:38.367 INFO:teuthology.orchestra.run.smithi114.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-27T10:40:38.367 INFO:teuthology.orchestra.run.smithi114.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-27T10:40:38.367 INFO:teuthology.orchestra.run.smithi114.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-27T10:40:38.367 INFO:teuthology.orchestra.run.smithi114.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-27T10:40:38.367 INFO:teuthology.orchestra.run.smithi114.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-27T10:40:38.367 INFO:teuthology.orchestra.run.smithi114.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-03-27T10:40:38.367 INFO:teuthology.orchestra.run.smithi114.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-27T10:40:38.367 INFO:teuthology.orchestra.run.smithi114.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-27T10:40:38.367 INFO:teuthology.orchestra.run.smithi114.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-27T10:40:38.367 INFO:teuthology.orchestra.run.smithi114.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-27T10:40:38.368 INFO:teuthology.orchestra.run.smithi114.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-27T10:40:38.368 INFO:teuthology.orchestra.run.smithi114.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-27T10:40:38.368 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-27T10:40:38.368 INFO:teuthology.orchestra.run.smithi114.stdout: python3-ceph-argparse x86_64 2:19.0.0-2478.g155268c4.el9 ceph 45 k 2024-03-27T10:40:38.368 INFO:teuthology.orchestra.run.smithi114.stdout: python3-ceph-common x86_64 2:19.0.0-2478.g155268c4.el9 ceph 134 k 2024-03-27T10:40:38.368 INFO:teuthology.orchestra.run.smithi114.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-27T10:40:38.368 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-27T10:40:38.368 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-27T10:40:38.368 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-27T10:40:38.368 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-27T10:40:38.368 INFO:teuthology.orchestra.run.smithi114.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-27T10:40:38.368 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-27T10:40:38.368 INFO:teuthology.orchestra.run.smithi114.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-27T10:40:38.369 INFO:teuthology.orchestra.run.smithi114.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-27T10:40:38.369 INFO:teuthology.orchestra.run.smithi114.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-27T10:40:38.369 INFO:teuthology.orchestra.run.smithi114.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-27T10:40:38.369 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-27T10:40:38.369 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-27T10:40:38.369 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-27T10:40:38.369 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-27T10:40:38.369 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-27T10:40:38.369 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-27T10:40:38.369 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-27T10:40:38.369 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-27T10:40:38.369 INFO:teuthology.orchestra.run.smithi114.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-27T10:40:38.369 INFO:teuthology.orchestra.run.smithi114.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-27T10:40:38.369 INFO:teuthology.orchestra.run.smithi114.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-27T10:40:38.370 INFO:teuthology.orchestra.run.smithi114.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-27T10:40:38.370 INFO:teuthology.orchestra.run.smithi114.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-27T10:40:38.370 INFO:teuthology.orchestra.run.smithi114.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-27T10:40:38.370 INFO:teuthology.orchestra.run.smithi114.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-27T10:40:38.370 INFO:teuthology.orchestra.run.smithi114.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-27T10:40:38.370 INFO:teuthology.orchestra.run.smithi114.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-27T10:40:38.370 INFO:teuthology.orchestra.run.smithi114.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-27T10:40:38.370 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-27T10:40:38.370 INFO:teuthology.orchestra.run.smithi114.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-27T10:40:38.370 INFO:teuthology.orchestra.run.smithi114.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-27T10:40:38.370 INFO:teuthology.orchestra.run.smithi114.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-27T10:40:38.370 INFO:teuthology.orchestra.run.smithi114.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-27T10:40:38.370 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-27T10:40:38.371 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-27T10:40:38.371 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-27T10:40:38.371 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-27T10:40:38.371 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-27T10:40:38.371 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-27T10:40:38.371 INFO:teuthology.orchestra.run.smithi114.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-27T10:40:38.371 INFO:teuthology.orchestra.run.smithi114.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-27T10:40:38.371 INFO:teuthology.orchestra.run.smithi114.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-27T10:40:38.371 INFO:teuthology.orchestra.run.smithi114.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-27T10:40:38.371 INFO:teuthology.orchestra.run.smithi114.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-27T10:40:38.371 INFO:teuthology.orchestra.run.smithi114.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-27T10:40:38.371 INFO:teuthology.orchestra.run.smithi114.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-27T10:40:38.371 INFO:teuthology.orchestra.run.smithi114.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-27T10:40:38.372 INFO:teuthology.orchestra.run.smithi114.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-27T10:40:38.372 INFO:teuthology.orchestra.run.smithi114.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-27T10:40:38.372 INFO:teuthology.orchestra.run.smithi114.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-27T10:40:38.372 INFO:teuthology.orchestra.run.smithi114.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-27T10:40:38.372 INFO:teuthology.orchestra.run.smithi114.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-27T10:40:38.372 INFO:teuthology.orchestra.run.smithi114.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-27T10:40:38.372 INFO:teuthology.orchestra.run.smithi114.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-27T10:40:38.372 INFO:teuthology.orchestra.run.smithi114.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-03-27T10:40:38.372 INFO:teuthology.orchestra.run.smithi114.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-03-27T10:40:38.372 INFO:teuthology.orchestra.run.smithi114.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-27T10:40:38.372 INFO:teuthology.orchestra.run.smithi114.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-27T10:40:38.372 INFO:teuthology.orchestra.run.smithi114.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-27T10:40:38.372 INFO:teuthology.orchestra.run.smithi114.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-27T10:40:38.372 INFO:teuthology.orchestra.run.smithi114.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-27T10:40:38.373 INFO:teuthology.orchestra.run.smithi114.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-27T10:40:38.373 INFO:teuthology.orchestra.run.smithi114.stdout:Installing weak dependencies: 2024-03-27T10:40:38.373 INFO:teuthology.orchestra.run.smithi114.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-03-27T10:40:38.373 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-03-27T10:40:38.373 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction Summary 2024-03-27T10:40:38.373 INFO:teuthology.orchestra.run.smithi114.stdout:============================================================================================ 2024-03-27T10:40:38.373 INFO:teuthology.orchestra.run.smithi114.stdout:Install 150 Packages 2024-03-27T10:40:38.373 INFO:teuthology.orchestra.run.smithi114.stdout:Upgrade 24 Packages 2024-03-27T10:40:38.373 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-03-27T10:40:38.374 INFO:teuthology.orchestra.run.smithi114.stdout:Total download size: 227 M 2024-03-27T10:40:38.374 INFO:teuthology.orchestra.run.smithi114.stdout:Downloading Packages: 2024-03-27T10:40:38.487 INFO:teuthology.orchestra.run.smithi177.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 5.6 MB/s | 1.6 MB 00:00 2024-03-27T10:40:38.562 INFO:teuthology.orchestra.run.smithi177.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 4.2 MB/s | 321 kB 00:00 2024-03-27T10:40:38.746 INFO:teuthology.orchestra.run.smithi177.stdout:(152/174): NetworkManager-1.47.2-1.el9.x86_64.r 13 MB/s | 2.3 MB 00:00 2024-03-27T10:40:38.904 INFO:teuthology.orchestra.run.smithi177.stdout:(153/174): NetworkManager-libnm-1.47.2-1.el9.x8 12 MB/s | 1.8 MB 00:00 2024-03-27T10:40:38.963 INFO:teuthology.orchestra.run.smithi177.stdout:(154/174): NetworkManager-team-1.47.2-1.el9.x86 735 kB/s | 42 kB 00:00 2024-03-27T10:40:39.046 INFO:teuthology.orchestra.run.smithi177.stdout:(155/174): NetworkManager-tui-1.47.2-1.el9.x86_ 2.9 MB/s | 248 kB 00:00 2024-03-27T10:40:39.105 INFO:teuthology.orchestra.run.smithi177.stdout:(156/174): acl-2.3.1-4.el9.x86_64.rpm 1.2 MB/s | 71 kB 00:00 2024-03-27T10:40:39.155 INFO:teuthology.orchestra.run.smithi177.stdout:(157/174): elfutils-debuginfod-client-0.190-2.e 743 kB/s | 36 kB 00:00 2024-03-27T10:40:39.381 INFO:teuthology.orchestra.run.smithi177.stdout:(158/174): librbd1-19.0.0-2478.g155268c4.el9.x8 2.9 MB/s | 3.1 MB 00:01 2024-03-27T10:40:39.414 INFO:teuthology.orchestra.run.smithi177.stdout:(159/174): elfutils-libelf-0.190-2.el9.x86_64.r 746 kB/s | 192 kB 00:00 2024-03-27T10:40:39.464 INFO:teuthology.orchestra.run.smithi177.stdout:(160/174): libacl-2.3.1-4.el9.x86_64.rpm 463 kB/s | 23 kB 00:00 2024-03-27T10:40:39.498 INFO:teuthology.orchestra.run.smithi177.stdout:(161/174): elfutils-libs-0.190-2.el9.x86_64.rpm 2.2 MB/s | 256 kB 00:00 2024-03-27T10:40:39.565 INFO:teuthology.orchestra.run.smithi177.stdout:(162/174): libblkid-2.37.4-18.el9.x86_64.rpm 1.1 MB/s | 108 kB 00:00 2024-03-27T10:40:39.598 INFO:teuthology.orchestra.run.smithi177.stdout:(163/174): libfdisk-2.37.4-18.el9.x86_64.rpm 1.5 MB/s | 155 kB 00:00 2024-03-27T10:40:39.657 INFO:teuthology.orchestra.run.smithi177.stdout:(164/174): libmount-2.37.4-18.el9.x86_64.rpm 1.5 MB/s | 136 kB 00:00 2024-03-27T10:40:39.674 INFO:teuthology.orchestra.run.smithi177.stdout:(165/174): libsmartcols-2.37.4-18.el9.x86_64.rp 858 kB/s | 64 kB 00:00 2024-03-27T10:40:39.707 INFO:teuthology.orchestra.run.smithi177.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 579 kB/s | 29 kB 00:00 2024-03-27T10:40:39.750 INFO:teuthology.orchestra.run.smithi114.stdout:(1/174): ceph-19.0.0-2478.g155268c4.el9.x86_64. 19 kB/s | 6.6 kB 00:00 2024-03-27T10:40:39.883 INFO:teuthology.orchestra.run.smithi177.stdout:(167/174): openssl-3.0.7-27.el9.x86_64.rpm 5.8 MB/s | 1.2 MB 00:00 2024-03-27T10:40:40.016 INFO:teuthology.orchestra.run.smithi177.stdout:(168/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 7.0 MB/s | 2.2 MB 00:00 2024-03-27T10:40:40.117 INFO:teuthology.orchestra.run.smithi177.stdout:(169/174): util-linux-core-2.37.4-18.el9.x86_64 4.6 MB/s | 465 kB 00:00 2024-03-27T10:40:40.167 INFO:teuthology.orchestra.run.smithi177.stdout:(170/174): util-linux-user-2.37.4-18.el9.x86_64 635 kB/s | 31 kB 00:00 2024-03-27T10:40:40.242 INFO:teuthology.orchestra.run.smithi177.stdout:(171/174): util-linux-2.37.4-18.el9.x86_64.rpm 6.4 MB/s | 2.3 MB 00:00 2024-03-27T10:40:40.376 INFO:teuthology.orchestra.run.smithi177.stdout:(172/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 5.2 MB/s | 1.1 MB 00:00 2024-03-27T10:40:40.467 INFO:teuthology.orchestra.run.smithi177.stdout:(173/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 331 kB/s | 74 kB 00:00 2024-03-27T10:40:40.559 INFO:teuthology.orchestra.run.smithi177.stdout:(174/174): librados2-19.0.0-2478.g155268c4.el9. 1.5 MB/s | 3.4 MB 00:02 2024-03-27T10:40:40.570 INFO:teuthology.orchestra.run.smithi177.stdout:-------------------------------------------------------------------------------- 2024-03-27T10:40:40.570 INFO:teuthology.orchestra.run.smithi177.stdout:Total 11 MB/s | 227 MB 00:20 2024-03-27T10:40:40.775 INFO:teuthology.orchestra.run.smithi114.stdout:(2/174): ceph-common-19.0.0-2478.g155268c4.el9. 15 MB/s | 21 MB 00:01 2024-03-27T10:40:40.834 INFO:teuthology.orchestra.run.smithi114.stdout:(3/174): ceph-fuse-19.0.0-2478.g155268c4.el9.x8 822 kB/s | 890 kB 00:01 2024-03-27T10:40:40.867 INFO:teuthology.orchestra.run.smithi114.stdout:(4/174): ceph-immutable-object-cache-19.0.0-247 1.6 MB/s | 145 kB 00:00 2024-03-27T10:40:40.984 INFO:teuthology.orchestra.run.smithi114.stdout:(5/174): ceph-mgr-19.0.0-2478.g155268c4.el9.x86 13 MB/s | 1.5 MB 00:00 2024-03-27T10:40:41.260 INFO:teuthology.orchestra.run.smithi114.stdout:(6/174): ceph-mon-19.0.0-2478.g155268c4.el9.x86 17 MB/s | 4.7 MB 00:00 2024-03-27T10:40:42.002 INFO:teuthology.orchestra.run.smithi114.stdout:(7/174): ceph-mds-19.0.0-2478.g155268c4.el9.x86 2.0 MB/s | 2.4 MB 00:01 2024-03-27T10:40:42.190 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction check 2024-03-27T10:40:42.218 INFO:teuthology.orchestra.run.smithi114.stdout:(8/174): ceph-osd-19.0.0-2478.g155268c4.el9.x86 18 MB/s | 17 MB 00:00 2024-03-27T10:40:42.294 INFO:teuthology.orchestra.run.smithi114.stdout:(9/174): ceph-selinux-19.0.0-2478.g155268c4.el9 340 kB/s | 25 kB 00:00 2024-03-27T10:40:42.570 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction check succeeded. 2024-03-27T10:40:42.570 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction test 2024-03-27T10:40:42.686 INFO:teuthology.orchestra.run.smithi114.stdout:(10/174): ceph-base-19.0.0-2478.g155268c4.el9.x 1.6 MB/s | 5.4 MB 00:03 2024-03-27T10:40:42.786 INFO:teuthology.orchestra.run.smithi114.stdout:(11/174): cephfs-mirror-19.0.0-2478.g155268c4.e 2.2 MB/s | 221 kB 00:00 2024-03-27T10:40:42.920 INFO:teuthology.orchestra.run.smithi114.stdout:(12/174): cephfs-shell-19.0.0-2478.g155268c4.el 203 kB/s | 27 kB 00:00 2024-03-27T10:40:43.053 INFO:teuthology.orchestra.run.smithi114.stdout:(13/174): libcephfs-devel-19.0.0-2478.g155268c4 240 kB/s | 32 kB 00:00 2024-03-27T10:40:43.420 INFO:teuthology.orchestra.run.smithi114.stdout:(14/174): libcephfs2-19.0.0-2478.g155268c4.el9. 2.0 MB/s | 746 kB 00:00 2024-03-27T10:40:43.554 INFO:teuthology.orchestra.run.smithi114.stdout:(15/174): libcephsqlite-19.0.0-2478.g155268c4.e 1.2 MB/s | 163 kB 00:00 2024-03-27T10:40:43.687 INFO:teuthology.orchestra.run.smithi114.stdout:(16/174): librados-devel-19.0.0-2478.g155268c4. 960 kB/s | 127 kB 00:00 2024-03-27T10:40:43.929 INFO:teuthology.orchestra.run.smithi114.stdout:(17/174): libradosstriper1-19.0.0-2478.g155268c 2.0 MB/s | 501 kB 00:00 2024-03-27T10:40:44.145 INFO:teuthology.orchestra.run.smithi114.stdout:(18/174): ceph-radosgw-19.0.0-2478.g155268c4.el 4.6 MB/s | 9.8 MB 00:02 2024-03-27T10:40:44.263 INFO:teuthology.orchestra.run.smithi114.stdout:(19/174): python3-ceph-argparse-19.0.0-2478.g15 383 kB/s | 45 kB 00:00 2024-03-27T10:40:44.405 INFO:teuthology.orchestra.run.smithi114.stdout:(20/174): python3-ceph-common-19.0.0-2478.g1552 948 kB/s | 134 kB 00:00 2024-03-27T10:40:44.978 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction test succeeded. 2024-03-27T10:40:44.978 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction 2024-03-27T10:40:44.981 INFO:teuthology.orchestra.run.smithi114.stdout:(21/174): ceph-test-19.0.0-2478.g155268c4.el9.x 18 MB/s | 48 MB 00:02 2024-03-27T10:40:45.014 INFO:teuthology.orchestra.run.smithi114.stdout:(22/174): python3-cephfs-19.0.0-2478.g155268c4. 266 kB/s | 162 kB 00:00 2024-03-27T10:40:45.056 INFO:teuthology.orchestra.run.smithi114.stdout:(23/174): python3-rados-19.0.0-2478.g155268c4.e 4.2 MB/s | 320 kB 00:00 2024-03-27T10:40:45.165 INFO:teuthology.orchestra.run.smithi114.stdout:(24/174): librgw2-19.0.0-2478.g155268c4.el9.x86 4.2 MB/s | 5.1 MB 00:01 2024-03-27T10:40:45.198 INFO:teuthology.orchestra.run.smithi114.stdout:(25/174): python3-rbd-19.0.0-2478.g155268c4.el9 1.6 MB/s | 298 kB 00:00 2024-03-27T10:40:45.253 INFO:teuthology.orchestra.run.smithi114.stdout:(26/174): python3-rgw-19.0.0-2478.g155268c4.el9 505 kB/s | 99 kB 00:00 2024-03-27T10:40:45.282 INFO:teuthology.orchestra.run.smithi097.stdout: Preparing : 1/1 2024-03-27T10:40:45.290 INFO:teuthology.orchestra.run.smithi114.stdout:(27/174): rbd-fuse-19.0.0-2478.g155268c4.el9.x8 684 kB/s | 85 kB 00:00 2024-03-27T10:40:45.324 INFO:teuthology.orchestra.run.smithi114.stdout:(28/174): rbd-nbd-19.0.0-2478.g155268c4.el9.x86 2.4 MB/s | 170 kB 00:00 2024-03-27T10:40:45.368 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-27T10:40:45.383 INFO:teuthology.orchestra.run.smithi114.stdout:(29/174): ceph-mgr-cephadm-19.0.0-2478.g155268c 2.4 MB/s | 141 kB 00:00 2024-03-27T10:40:45.408 INFO:teuthology.orchestra.run.smithi114.stdout:(30/174): ceph-grafana-dashboards-19.0.0-2478.g 291 kB/s | 34 kB 00:00 2024-03-27T10:40:45.509 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-27T10:40:45.659 INFO:teuthology.orchestra.run.smithi114.stdout:(31/174): ceph-mgr-dashboard-19.0.0-2478.g15526 13 MB/s | 3.5 MB 00:00 2024-03-27T10:40:45.717 INFO:teuthology.orchestra.run.smithi114.stdout:(32/174): ceph-mgr-modules-core-19.0.0-2478.g15 4.3 MB/s | 249 kB 00:00 2024-03-27T10:40:45.730 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-27T10:40:45.759 INFO:teuthology.orchestra.run.smithi114.stdout:(33/174): ceph-mgr-rook-19.0.0-2478.g155268c4.e 1.2 MB/s | 49 kB 00:00 2024-03-27T10:40:45.776 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-27T10:40:45.843 INFO:teuthology.orchestra.run.smithi114.stdout:(34/174): rbd-mirror-19.0.0-2478.g155268c4.el9. 4.8 MB/s | 3.1 MB 00:00 2024-03-27T10:40:45.860 INFO:teuthology.orchestra.run.smithi114.stdout:(35/174): ceph-prometheus-alerts-19.0.0-2478.g1 165 kB/s | 16 kB 00:00 2024-03-27T10:40:45.864 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-27T10:40:45.910 INFO:teuthology.orchestra.run.smithi114.stdout:(36/174): ceph-volume-19.0.0-2478.g155268c4.el9 3.9 MB/s | 264 kB 00:00 2024-03-27T10:40:45.972 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-27T10:40:45.992 INFO:teuthology.orchestra.run.smithi114.stdout:(37/174): cephadm-19.0.0-2478.g155268c4.el9.noa 5.7 MB/s | 762 kB 00:00 2024-03-27T10:40:46.019 INFO:teuthology.orchestra.run.smithi114.stdout:(38/174): cephfs-top-19.0.0-2478.g155268c4.el9. 169 kB/s | 18 kB 00:00 2024-03-27T10:40:46.025 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-27T10:40:46.056 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-27T10:40:46.120 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-27T10:40:46.245 INFO:teuthology.orchestra.run.smithi114.stdout:(39/174): automake-1.16.2-8.el9.noarch.rpm 2.6 MB/s | 662 kB 00:00 2024-03-27T10:40:46.286 INFO:teuthology.orchestra.run.smithi114.stdout:(40/174): bison-3.7.4-5.el9.x86_64.rpm 3.4 MB/s | 920 kB 00:00 2024-03-27T10:40:46.312 INFO:teuthology.orchestra.run.smithi114.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 106 kB 00:00 2024-03-27T10:40:46.371 INFO:teuthology.orchestra.run.smithi114.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 3.6 MB/s | 307 kB 00:00 2024-03-27T10:40:46.404 INFO:teuthology.orchestra.run.smithi114.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 357 kB/s | 32 kB 00:00 2024-03-27T10:40:46.538 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-27T10:40:46.546 INFO:teuthology.orchestra.run.smithi114.stdout:(44/174): ceph-mgr-diskprediction-local-19.0.0- 6.5 MB/s | 7.4 MB 00:01 2024-03-27T10:40:46.589 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : librados2-2:19.0.0-2478.g155268c4.el9.x86_64 10/198 2024-03-27T10:40:46.613 INFO:teuthology.orchestra.run.smithi114.stdout:(45/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 12 MB/s | 3.0 MB 00:00 2024-03-27T10:40:46.638 INFO:teuthology.orchestra.run.smithi114.stdout:(46/174): flexiblas-openblas-openmp-3.0.4-8.el9 78 kB/s | 17 kB 00:00 2024-03-27T10:40:46.663 INFO:teuthology.orchestra.run.smithi114.stdout:(47/174): libaio-devel-0.3.111-13.el9.x86_64.rp 87 kB/s | 10 kB 00:00 2024-03-27T10:40:46.688 INFO:teuthology.orchestra.run.smithi114.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 903 kB/s | 44 kB 00:00 2024-03-27T10:40:46.740 INFO:teuthology.orchestra.run.smithi114.stdout:(49/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.5 MB/s | 159 kB 00:00 2024-03-27T10:40:46.772 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librados2-2:19.0.0-2478.g155268c4.el9.x86_64 10/198 2024-03-27T10:40:46.789 INFO:teuthology.orchestra.run.smithi114.stdout:(50/174): librdkafka-1.6.1-102.el9.x86_64.rpm 5.2 MB/s | 661 kB 00:00 2024-03-27T10:40:46.806 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libcephfs2-2:19.0.0-2478.g155268c4.el9.x86_64 11/198 2024-03-27T10:40:46.831 INFO:teuthology.orchestra.run.smithi114.stdout:(51/174): libtool-2.4.6-45.el9.x86_64.rpm 4.2 MB/s | 578 kB 00:00 2024-03-27T10:40:46.856 INFO:teuthology.orchestra.run.smithi114.stdout:(52/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 757 kB/s | 46 kB 00:00 2024-03-27T10:40:46.881 INFO:teuthology.orchestra.run.smithi114.stdout:(53/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.2 MB/s | 282 kB 00:00 2024-03-27T10:40:46.907 INFO:teuthology.orchestra.run.smithi114.stdout:(54/174): lua-5.4.4-4.el9.x86_64.rpm 2.5 MB/s | 187 kB 00:00 2024-03-27T10:40:46.926 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libcephfs2-2:19.0.0-2478.g155268c4.el9.x86_64 11/198 2024-03-27T10:40:46.932 INFO:teuthology.orchestra.run.smithi114.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 693 kB/s | 34 kB 00:00 2024-03-27T10:40:46.963 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-rados-2:19.0.0-2478.g155268c4.el9.x86_64 12/198 2024-03-27T10:40:46.966 INFO:teuthology.orchestra.run.smithi114.stdout:(56/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.5 MB/s | 217 kB 00:00 2024-03-27T10:40:47.022 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-03-27T10:40:47.110 INFO:teuthology.orchestra.run.smithi114.stdout:(57/174): openblas-openmp-0.3.21-2.el9.x86_64.r 23 MB/s | 4.7 MB 00:00 2024-03-27T10:40:47.150 INFO:teuthology.orchestra.run.smithi114.stdout:(58/174): protobuf-3.14.0-13.el9.x86_64.rpm 4.8 MB/s | 1.0 MB 00:00 2024-03-27T10:40:47.175 INFO:teuthology.orchestra.run.smithi114.stdout:(59/174): python3-attrs-20.3.0-7.el9.noarch.rpm 399 kB/s | 78 kB 00:00 2024-03-27T10:40:47.217 INFO:teuthology.orchestra.run.smithi114.stdout:(60/174): python3-jsonpointer-2.0-4.el9.noarch. 454 kB/s | 18 kB 00:00 2024-03-27T10:40:47.242 INFO:teuthology.orchestra.run.smithi114.stdout:(61/174): python3-jsonpatch-1.21-16.el9.noarch. 374 kB/s | 24 kB 00:00 2024-03-27T10:40:47.356 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-27T10:40:47.384 INFO:teuthology.orchestra.run.smithi114.stdout:(62/174): python3-babel-2.9.1-2.el9.noarch.rpm 28 MB/s | 5.8 MB 00:00 2024-03-27T10:40:47.409 INFO:teuthology.orchestra.run.smithi114.stdout:(63/174): python3-mako-1.1.4-6.el9.noarch.rpm 755 kB/s | 144 kB 00:00 2024-03-27T10:40:47.443 INFO:teuthology.orchestra.run.smithi114.stdout:(64/174): python3-markupsafe-1.1.1-12.el9.x86_6 170 kB/s | 32 kB 00:00 2024-03-27T10:40:47.493 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-03-27T10:40:47.509 INFO:teuthology.orchestra.run.smithi114.stdout:(65/174): python3-oauthlib-3.1.1-5.el9.noarch.r 2.6 MB/s | 174 kB 00:00 2024-03-27T10:40:47.543 INFO:teuthology.orchestra.run.smithi114.stdout:(66/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.2 MB/s | 217 kB 00:00 2024-03-27T10:40:47.667 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libcephsqlite-2:19.0.0-2478.g155268c4.el9.x86_64 16/198 2024-03-27T10:40:47.677 INFO:teuthology.orchestra.run.smithi114.stdout:(67/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 21 MB/s | 4.8 MB 00:00 2024-03-27T10:40:47.702 INFO:teuthology.orchestra.run.smithi114.stdout:(68/174): python3-packaging-20.9-5.el9.noarch.r 361 kB/s | 69 kB 00:00 2024-03-27T10:40:47.727 INFO:teuthology.orchestra.run.smithi114.stdout:(69/174): python3-prettytable-0.7.2-27.el9.noar 243 kB/s | 41 kB 00:00 2024-03-27T10:40:47.778 INFO:teuthology.orchestra.run.smithi114.stdout:(70/174): python3-protobuf-3.14.0-13.el9.noarch 4.8 MB/s | 238 kB 00:00 2024-03-27T10:40:47.803 INFO:teuthology.orchestra.run.smithi114.stdout:(71/174): python3-pyasn1-modules-0.4.8-6.el9.no 2.8 MB/s | 211 kB 00:00 2024-03-27T10:40:47.845 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2478.g155268c4.el9.x86_64 16/198 2024-03-27T10:40:47.859 INFO:teuthology.orchestra.run.smithi114.stdout:(72/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.0 MB/s | 132 kB 00:00 2024-03-27T10:40:47.878 INFO:teuthology.orchestra.run.smithi114.stdout:(73/174): python3-pytz-2021.1-5.el9.noarch.rpm 476 kB/s | 47 kB 00:00 2024-03-27T10:40:47.879 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libradosstriper1-2:19.0.0-2478.g155268c4.el9.x86 17/198 2024-03-27T10:40:47.904 INFO:teuthology.orchestra.run.smithi114.stdout:(74/174): python3-requests-oauthlib-1.3.0-12.el 430 kB/s | 43 kB 00:00 2024-03-27T10:40:47.929 INFO:teuthology.orchestra.run.smithi114.stdout:(75/174): python3-toml-0.10.2-6.el9.noarch.rpm 794 kB/s | 39 kB 00:00 2024-03-27T10:40:47.987 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2478.g155268c4.el9.x86 17/198 2024-03-27T10:40:48.049 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-27T10:40:48.081 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-03-27T10:40:48.141 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-03-27T10:40:48.155 INFO:teuthology.orchestra.run.smithi114.stdout:(76/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 164 kB/s | 41 kB 00:00 2024-03-27T10:40:48.229 INFO:teuthology.orchestra.run.smithi114.stdout:(77/174): socat-1.7.4.1-5.el9.x86_64.rpm 1.0 MB/s | 300 kB 00:00 2024-03-27T10:40:48.433 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-03-27T10:40:48.447 INFO:teuthology.orchestra.run.smithi114.stdout:(78/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 28 MB/s | 16 MB 00:00 2024-03-27T10:40:48.472 INFO:teuthology.orchestra.run.smithi114.stdout:(79/174): uuid-1.6.2-55.el9.x86_64.rpm 186 kB/s | 56 kB 00:00 2024-03-27T10:40:48.482 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-03-27T10:40:48.497 INFO:teuthology.orchestra.run.smithi114.stdout:(80/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 238 kB/s | 63 kB 00:00 2024-03-27T10:40:48.522 INFO:teuthology.orchestra.run.smithi114.stdout:(81/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 907 kB/s | 44 kB 00:00 2024-03-27T10:40:48.523 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-27T10:40:48.547 INFO:teuthology.orchestra.run.smithi114.stdout:(82/174): fuse-2.9.9-15.el9.x86_64.rpm 1.0 MB/s | 79 kB 00:00 2024-03-27T10:40:48.607 INFO:teuthology.orchestra.run.smithi114.stdout:(83/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.2 MB/s | 241 kB 00:00 2024-03-27T10:40:48.640 INFO:teuthology.orchestra.run.smithi114.stdout:(84/174): python3-ply-3.11-14.el9.noarch.rpm 1.1 MB/s | 103 kB 00:00 2024-03-27T10:40:48.690 INFO:teuthology.orchestra.run.smithi114.stdout:(85/174): python3-cryptography-36.0.1-4.el9.x86 8.2 MB/s | 1.1 MB 00:00 2024-03-27T10:40:48.715 INFO:teuthology.orchestra.run.smithi114.stdout:(86/174): python3-pycparser-2.20-6.el9.noarch.r 1.1 MB/s | 124 kB 00:00 2024-03-27T10:40:48.740 INFO:teuthology.orchestra.run.smithi114.stdout:(87/174): python3-urllib3-1.26.5-5.el9.noarch.r 2.0 MB/s | 187 kB 00:00 2024-03-27T10:40:48.813 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-27T10:40:48.816 INFO:teuthology.orchestra.run.smithi114.stdout:(88/174): libquadmath-11.4.1-3.el9.x86_64.rpm 2.6 MB/s | 195 kB 00:00 2024-03-27T10:40:48.858 INFO:teuthology.orchestra.run.smithi114.stdout:(89/174): jq-1.6-16.el9.x86_64.rpm 1.3 MB/s | 187 kB 00:00 2024-03-27T10:40:48.908 INFO:teuthology.orchestra.run.smithi114.stdout:(90/174): libgfortran-11.4.1-3.el9.x86_64.rpm 4.1 MB/s | 803 kB 00:00 2024-03-27T10:40:48.977 INFO:teuthology.orchestra.run.smithi114.stdout:(91/174): python3-requests-2.25.1-8.el9.noarch. 781 kB/s | 125 kB 00:00 2024-03-27T10:40:49.007 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-27T10:40:49.119 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-27T10:40:49.184 INFO:teuthology.orchestra.run.smithi114.stdout:(92/174): elfutils-libelf-devel-0.190-2.el9.x86 66 kB/s | 21 kB 00:00 2024-03-27T10:40:49.209 INFO:teuthology.orchestra.run.smithi114.stdout:(93/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 85 kB/s | 25 kB 00:00 2024-03-27T10:40:49.300 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-27T10:40:49.351 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-27T10:40:49.680 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-27T10:40:49.870 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-27T10:40:49.991 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-27T10:40:50.050 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-27T10:40:50.084 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-27T10:40:50.118 INFO:teuthology.orchestra.run.smithi114.stdout:(94/174): postgresql-13.14-1.el9.x86_64.rpm 1.7 MB/s | 1.6 MB 00:00 2024-03-27T10:40:50.148 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-27T10:40:50.268 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-27T10:40:50.350 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-27T10:40:50.400 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-27T10:40:50.502 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-27T10:40:50.563 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-27T10:40:50.627 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-27T10:40:50.635 INFO:teuthology.orchestra.run.smithi114.stdout:(95/174): postgresql-private-libs-13.14-1.el9.x 258 kB/s | 133 kB 00:00 2024-03-27T10:40:50.677 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-27T10:40:50.717 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-27T10:40:50.802 INFO:teuthology.orchestra.run.smithi114.stdout:(96/174): postgresql-contrib-13.14-1.el9.x86_64 572 kB/s | 911 kB 00:01 2024-03-27T10:40:50.835 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-27T10:40:50.871 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-ceph-common-2:19.0.0-2478.g155268c4.el9. 39/198 2024-03-27T10:40:50.923 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-ceph-argparse-2:19.0.0-2478.g155268c4.el 40/198 2024-03-27T10:40:50.969 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cephfs-2:19.0.0-2478.g155268c4.el9.x86_6 41/198 2024-03-27T10:40:51.107 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-27T10:40:51.172 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-27T10:40:51.272 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-27T10:40:51.452 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-27T10:40:51.525 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-27T10:40:51.627 INFO:teuthology.orchestra.run.smithi114.stdout:(97/174): python3-devel-3.9.17-2.el9.x86_64.rpm 298 kB/s | 245 kB 00:00 2024-03-27T10:40:52.214 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-27T10:40:52.305 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-27T10:40:52.369 INFO:teuthology.orchestra.run.smithi114.stdout:(98/174): postgresql-server-13.14-1.el9.x86_64. 3.3 MB/s | 5.8 MB 00:01 2024-03-27T10:40:52.394 INFO:teuthology.orchestra.run.smithi114.stdout:(99/174): python3-jinja2-2.11.3-5.el9.noarch.rp 324 kB/s | 248 kB 00:00 2024-03-27T10:40:52.418 INFO:teuthology.orchestra.run.smithi177.stdout: Preparing : 1/1 2024-03-27T10:40:52.467 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-27T10:40:52.490 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-27T10:40:52.529 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-27T10:40:52.557 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-27T10:40:52.561 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-27T10:40:52.590 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-27T10:40:52.604 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-27T10:40:52.635 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-03-27T10:40:52.685 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-27T10:40:52.694 INFO:teuthology.orchestra.run.smithi114.stdout:(100/174): qatlib-service-24.02.0-1.el9.x86_64. 120 kB/s | 36 kB 00:00 2024-03-27T10:40:52.698 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-27T10:40:52.749 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : librados-devel-2:19.0.0-2478.g155268c4.el9.x86_6 53/198 2024-03-27T10:40:52.792 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-27T10:40:52.797 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-27T10:40:52.853 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-27T10:40:52.869 INFO:teuthology.orchestra.run.smithi114.stdout:(101/174): openssl-devel-3.0.7-27.el9.x86_64.rp 1.1 MB/s | 4.1 MB 00:03 2024-03-27T10:40:52.884 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-27T10:40:52.981 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-27T10:40:53.061 INFO:teuthology.orchestra.run.smithi114.stdout:(102/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 129 kB/s | 47 kB 00:00 2024-03-27T10:40:53.095 INFO:teuthology.orchestra.run.smithi114.stdout:(103/174): qatlib-24.02.0-1.el9.x86_64.rpm 304 kB/s | 221 kB 00:00 2024-03-27T10:40:53.191 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-27T10:40:53.229 INFO:teuthology.orchestra.run.smithi114.stdout:(104/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 262 kB/s | 94 kB 00:00 2024-03-27T10:40:53.398 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-27T10:40:53.438 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-27T10:40:53.441 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : librados2-2:19.0.0-2478.g155268c4.el9.x86_64 10/198 2024-03-27T10:40:53.528 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-27T10:40:53.588 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-27T10:40:53.590 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: librados2-2:19.0.0-2478.g155268c4.el9.x86_64 10/198 2024-03-27T10:40:53.617 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libcephfs2-2:19.0.0-2478.g155268c4.el9.x86_64 11/198 2024-03-27T10:40:53.729 INFO:teuthology.orchestra.run.smithi114.stdout:(105/174): lua-devel-5.4.4-4.el9.x86_64.rpm 33 kB/s | 22 kB 00:00 2024-03-27T10:40:53.740 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: libcephfs2-2:19.0.0-2478.g155268c4.el9.x86_64 11/198 2024-03-27T10:40:53.773 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-rados-2:19.0.0-2478.g155268c4.el9.x86_64 12/198 2024-03-27T10:40:53.789 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-27T10:40:53.829 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-03-27T10:40:53.850 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-27T10:40:53.925 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-27T10:40:53.937 INFO:teuthology.orchestra.run.smithi114.stdout:(106/174): gperftools-libs-2.9.1-2.el9.x86_64.r 1.5 MB/s | 309 kB 00:00 2024-03-27T10:40:53.996 INFO:teuthology.orchestra.run.smithi114.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 334 kB/s | 19 kB 00:00 2024-03-27T10:40:54.021 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-27T10:40:54.084 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-27T10:40:54.118 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-27T10:40:54.130 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-27T10:40:54.171 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-03-27T10:40:54.196 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libcephsqlite-2:19.0.0-2478.g155268c4.el9.x86_64 16/198 2024-03-27T10:40:54.201 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-27T10:40:54.249 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-27T10:40:54.315 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-27T10:40:54.342 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2478.g155268c4.el9.x86_64 16/198 2024-03-27T10:40:54.363 INFO:teuthology.orchestra.run.smithi114.stdout:(108/174): libarrow-9.0.0-10.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-03-27T10:40:54.372 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libradosstriper1-2:19.0.0-2478.g155268c4.el9.x86 17/198 2024-03-27T10:40:54.388 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-27T10:40:54.422 INFO:teuthology.orchestra.run.smithi114.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 449 kB/s | 26 kB 00:00 2024-03-27T10:40:54.445 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2478.g155268c4.el9.x86 17/198 2024-03-27T10:40:54.480 INFO:teuthology.orchestra.run.smithi114.stdout:(110/174): liboath-2.6.7-2.el9.x86_64.rpm 837 kB/s | 49 kB 00:00 2024-03-27T10:40:54.499 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-27T10:40:54.515 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-03-27T10:40:54.547 INFO:teuthology.orchestra.run.smithi114.stdout:(111/174): libunwind-1.6.2-1.el9.x86_64.rpm 1.0 MB/s | 67 kB 00:00 2024-03-27T10:40:54.580 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-03-27T10:40:54.597 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-27T10:40:54.614 INFO:teuthology.orchestra.run.smithi114.stdout:(112/174): luarocks-3.9.2-1.el9.noarch.rpm 2.2 MB/s | 151 kB 00:00 2024-03-27T10:40:54.656 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-27T10:40:54.693 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-27T10:40:54.723 INFO:teuthology.orchestra.run.smithi114.stdout:(113/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 7.6 MB/s | 838 kB 00:00 2024-03-27T10:40:54.751 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-27T10:40:54.806 INFO:teuthology.orchestra.run.smithi114.stdout:(114/174): python3-asyncssh-2.13.2-5.el9.noarch 6.4 MB/s | 548 kB 00:00 2024-03-27T10:40:54.881 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-03-27T10:40:54.884 INFO:teuthology.orchestra.run.smithi114.stdout:(115/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 565 kB/s | 43 kB 00:00 2024-03-27T10:40:54.912 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-27T10:40:54.924 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-03-27T10:40:54.949 INFO:teuthology.orchestra.run.smithi114.stdout:(116/174): python3-cachetools-4.2.4-1.el9.noarc 504 kB/s | 32 kB 00:00 2024-03-27T10:40:54.967 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-03-27T10:40:54.974 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-27T10:40:55.008 INFO:teuthology.orchestra.run.smithi114.stdout:(117/174): protobuf-compiler-3.14.0-13.el9.x86_ 452 kB/s | 863 kB 00:01 2024-03-27T10:40:55.016 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-27T10:40:55.033 INFO:teuthology.orchestra.run.smithi114.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 180 kB/s | 15 kB 00:00 2024-03-27T10:40:55.050 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-27T10:40:55.075 INFO:teuthology.orchestra.run.smithi114.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 2.5 MB/s | 172 kB 00:00 2024-03-27T10:40:55.083 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-27T10:40:55.150 INFO:teuthology.orchestra.run.smithi114.stdout:(120/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.0 MB/s | 303 kB 00:00 2024-03-27T10:40:55.209 INFO:teuthology.orchestra.run.smithi114.stdout:(121/174): python3-colorama-0.4.4-12.el9.noarch 610 kB/s | 35 kB 00:00 2024-03-27T10:40:55.243 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-27T10:40:55.284 INFO:teuthology.orchestra.run.smithi114.stdout:(122/174): python3-google-auth-2.28.2-1.el9.noa 2.7 MB/s | 203 kB 00:00 2024-03-27T10:40:55.308 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-27T10:40:55.317 INFO:teuthology.orchestra.run.smithi114.stdout:(123/174): python3-cherrypy-18.6.1-2.el9.noarch 1.2 MB/s | 358 kB 00:00 2024-03-27T10:40:55.410 INFO:teuthology.orchestra.run.smithi114.stdout:(124/174): python3-grpcio-tools-1.46.7-10.el9.x 1.6 MB/s | 144 kB 00:00 2024-03-27T10:40:55.433 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-27T10:40:55.497 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-27T10:40:55.502 INFO:teuthology.orchestra.run.smithi114.stdout:(125/174): python3-grpcio-1.46.7-10.el9.x86_64. 9.4 MB/s | 2.0 MB 00:00 2024-03-27T10:40:55.527 INFO:teuthology.orchestra.run.smithi114.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 92 kB/s | 11 kB 00:00 2024-03-27T10:40:55.559 INFO:teuthology.orchestra.run.smithi114.stdout:(127/174): python3-jaraco-classes-3.2.1-5.el9.n 315 kB/s | 18 kB 00:00 2024-03-27T10:40:55.579 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-27T10:40:55.584 INFO:teuthology.orchestra.run.smithi114.stdout:(128/174): python3-jaraco-collections-3.0.0-8.e 412 kB/s | 23 kB 00:00 2024-03-27T10:40:55.594 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-27T10:40:55.644 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-27T10:40:55.708 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-27T10:40:55.708 INFO:teuthology.orchestra.run.smithi114.stdout:(129/174): python3-jaraco-functools-3.5.0-2.el9 131 kB/s | 19 kB 00:00 2024-03-27T10:40:55.786 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-27T10:40:55.810 INFO:teuthology.orchestra.run.smithi114.stdout:(130/174): python3-jaraco-text-3.2.0-6.el9.noar 87 kB/s | 20 kB 00:00 2024-03-27T10:40:55.821 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-27T10:40:55.841 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-27T10:40:55.852 INFO:teuthology.orchestra.run.smithi114.stdout:(131/174): abseil-cpp-20211102.0-3.el9.x86_64.r 209 kB/s | 548 kB 00:02 2024-03-27T10:40:55.878 INFO:teuthology.orchestra.run.smithi114.stdout:(132/174): python3-logutils-0.3.5-21.el9.noarch 697 kB/s | 46 kB 00:00 2024-03-27T10:40:55.911 INFO:teuthology.orchestra.run.smithi114.stdout:(133/174): python3-more-itertools-8.12.0-2.el9. 1.3 MB/s | 79 kB 00:00 2024-03-27T10:40:55.953 INFO:teuthology.orchestra.run.smithi114.stdout:(134/174): python3-kubernetes-26.1.0-2.el9.noar 4.2 MB/s | 1.0 MB 00:00 2024-03-27T10:40:56.001 INFO:teuthology.orchestra.run.smithi114.stdout:(135/174): python3-natsort-7.1.1-5.el9.noarch.r 470 kB/s | 58 kB 00:00 2024-03-27T10:40:56.045 INFO:teuthology.orchestra.run.smithi114.stdout:(136/174): python3-pecan-1.4.2-3.el9.noarch.rpm 2.0 MB/s | 272 kB 00:00 2024-03-27T10:40:56.062 INFO:teuthology.orchestra.run.smithi114.stdout:(137/174): python3-portend-3.1.0-2.el9.noarch.r 275 kB/s | 16 kB 00:00 2024-03-27T10:40:56.077 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-27T10:40:56.088 INFO:teuthology.orchestra.run.smithi114.stdout:(138/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.0 MB/s | 90 kB 00:00 2024-03-27T10:40:56.104 INFO:teuthology.orchestra.run.smithi114.stdout:(139/174): python3-pyperclip-1.8.2-2.el9.noarch 487 kB/s | 28 kB 00:00 2024-03-27T10:40:56.122 INFO:teuthology.orchestra.run.smithi114.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 528 kB/s | 31 kB 00:00 2024-03-27T10:40:56.151 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-27T10:40:56.155 INFO:teuthology.orchestra.run.smithi114.stdout:(141/174): python3-routes-2.5.1-5.el9.noarch.rp 2.8 MB/s | 188 kB 00:00 2024-03-27T10:40:56.180 INFO:teuthology.orchestra.run.smithi114.stdout:(142/174): python3-rsa-4.9-2.el9.noarch.rpm 789 kB/s | 59 kB 00:00 2024-03-27T10:40:56.206 INFO:teuthology.orchestra.run.smithi114.stdout:(143/174): python3-tempora-5.0.0-2.el9.noarch.r 430 kB/s | 36 kB 00:00 2024-03-27T10:40:56.209 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-27T10:40:56.231 INFO:teuthology.orchestra.run.smithi114.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 679 kB/s | 51 kB 00:00 2024-03-27T10:40:56.264 INFO:teuthology.orchestra.run.smithi114.stdout:(145/174): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-03-27T10:40:56.290 INFO:teuthology.orchestra.run.smithi114.stdout:(146/174): python3-websocket-client-1.2.3-2.el9 1.1 MB/s | 90 kB 00:00 2024-03-27T10:40:56.324 INFO:teuthology.orchestra.run.smithi114.stdout:(147/174): python3-zc-lockfile-2.0-10.el9.noarc 345 kB/s | 20 kB 00:00 2024-03-27T10:40:56.324 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-27T10:40:56.366 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-03-27T10:40:56.382 INFO:teuthology.orchestra.run.smithi114.stdout:(148/174): python3-werkzeug-2.0.3-3.el9.1.noarc 2.8 MB/s | 427 kB 00:00 2024-03-27T10:40:56.400 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-27T10:40:56.413 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-27T10:40:56.424 INFO:teuthology.orchestra.run.smithi114.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 1.4 MB/s | 192 kB 00:00 2024-03-27T10:40:56.453 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-27T10:40:56.498 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-27T10:40:56.537 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-27T10:40:56.541 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-27T10:40:56.581 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-27T10:40:56.582 INFO:teuthology.orchestra.run.smithi114.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 6.1 MB/s | 1.6 MB 00:00 2024-03-27T10:40:56.609 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-27T10:40:56.618 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-27T10:40:56.658 INFO:teuthology.orchestra.run.smithi114.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 4.2 MB/s | 321 kB 00:00 2024-03-27T10:40:56.668 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-27T10:40:56.680 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-27T10:40:56.739 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-27T10:40:56.809 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-27T10:40:56.850 INFO:teuthology.orchestra.run.smithi114.stdout:(152/174): NetworkManager-1.47.2-1.el9.x86_64.r 12 MB/s | 2.3 MB 00:00 2024-03-27T10:40:56.874 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-27T10:40:56.883 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-27T10:40:56.919 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-27T10:40:56.931 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-27T10:40:56.955 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-27T10:40:57.016 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-27T10:40:57.025 INFO:teuthology.orchestra.run.smithi114.stdout:(153/174): NetworkManager-libnm-1.47.2-1.el9.x8 11 MB/s | 1.8 MB 00:00 2024-03-27T10:40:57.033 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-27T10:40:57.057 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-27T10:40:57.078 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-27T10:40:57.104 INFO:teuthology.orchestra.run.smithi114.stdout:(154/174): NetworkManager-team-1.47.2-1.el9.x86 548 kB/s | 42 kB 00:00 2024-03-27T10:40:57.135 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-27T10:40:57.181 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-27T10:40:57.192 INFO:teuthology.orchestra.run.smithi114.stdout:(155/174): NetworkManager-tui-1.47.2-1.el9.x86_ 2.8 MB/s | 248 kB 00:00 2024-03-27T10:40:57.211 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-27T10:40:57.222 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-27T10:40:57.251 INFO:teuthology.orchestra.run.smithi114.stdout:(156/174): acl-2.3.1-4.el9.x86_64.rpm 1.2 MB/s | 71 kB 00:00 2024-03-27T10:40:57.301 INFO:teuthology.orchestra.run.smithi114.stdout:(157/174): elfutils-debuginfod-client-0.190-2.e 743 kB/s | 36 kB 00:00 2024-03-27T10:40:57.338 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-27T10:40:57.339 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-27T10:40:57.377 INFO:teuthology.orchestra.run.smithi114.stdout:(158/174): librbd1-19.0.0-2478.g155268c4.el9.x8 3.3 MB/s | 3.1 MB 00:00 2024-03-27T10:40:57.377 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-ceph-common-2:19.0.0-2478.g155268c4.el9. 39/198 2024-03-27T10:40:57.396 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-27T10:40:57.433 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-27T10:40:57.438 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-ceph-argparse-2:19.0.0-2478.g155268c4.el 40/198 2024-03-27T10:40:57.443 INFO:teuthology.orchestra.run.smithi114.stdout:(159/174): elfutils-libelf-0.190-2.el9.x86_64.r 1.3 MB/s | 192 kB 00:00 2024-03-27T10:40:57.490 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-27T10:40:57.499 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-cephfs-2:19.0.0-2478.g155268c4.el9.x86_6 41/198 2024-03-27T10:40:57.502 INFO:teuthology.orchestra.run.smithi114.stdout:(160/174): elfutils-libs-0.190-2.el9.x86_64.rpm 2.0 MB/s | 256 kB 00:00 2024-03-27T10:40:57.586 INFO:teuthology.orchestra.run.smithi114.stdout:(161/174): librados2-19.0.0-2478.g155268c4.el9. 2.8 MB/s | 3.4 MB 00:01 2024-03-27T10:40:57.612 INFO:teuthology.orchestra.run.smithi114.stdout:(162/174): libacl-2.3.1-4.el9.x86_64.rpm 138 kB/s | 23 kB 00:00 2024-03-27T10:40:57.622 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-27T10:40:57.626 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-27T10:40:57.653 INFO:teuthology.orchestra.run.smithi114.stdout:(163/174): libblkid-2.37.4-18.el9.x86_64.rpm 725 kB/s | 108 kB 00:00 2024-03-27T10:40:57.681 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-27T10:40:57.695 INFO:teuthology.orchestra.run.smithi114.stdout:(164/174): libfdisk-2.37.4-18.el9.x86_64.rpm 1.4 MB/s | 155 kB 00:00 2024-03-27T10:40:57.716 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-27T10:40:57.720 INFO:teuthology.orchestra.run.smithi114.stdout:(165/174): libmount-2.37.4-18.el9.x86_64.rpm 1.2 MB/s | 136 kB 00:00 2024-03-27T10:40:57.746 INFO:teuthology.orchestra.run.smithi114.stdout:(166/174): libsmartcols-2.37.4-18.el9.x86_64.rp 699 kB/s | 64 kB 00:00 2024-03-27T10:40:57.788 INFO:teuthology.orchestra.run.smithi114.stdout:(167/174): libuuid-2.37.4-18.el9.x86_64.rpm 433 kB/s | 29 kB 00:00 2024-03-27T10:40:57.789 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-27T10:40:57.888 INFO:teuthology.orchestra.run.smithi114.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 7.3 MB/s | 1.2 MB 00:00 2024-03-27T10:40:57.963 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-27T10:40:57.997 INFO:teuthology.orchestra.run.smithi114.stdout:(169/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 8.6 MB/s | 2.2 MB 00:00 2024-03-27T10:40:58.023 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-27T10:40:58.047 INFO:teuthology.orchestra.run.smithi114.stdout:(170/174): util-linux-core-2.37.4-18.el9.x86_64 2.9 MB/s | 465 kB 00:00 2024-03-27T10:40:58.064 INFO:teuthology.orchestra.run.smithi114.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 472 kB/s | 31 kB 00:00 2024-03-27T10:40:58.131 INFO:teuthology.orchestra.run.smithi114.stdout:(172/174): util-linux-2.37.4-18.el9.x86_64.rpm 6.7 MB/s | 2.3 MB 00:00 2024-03-27T10:40:58.239 INFO:teuthology.orchestra.run.smithi114.stdout:(173/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 5.6 MB/s | 1.1 MB 00:00 2024-03-27T10:40:58.498 INFO:teuthology.orchestra.run.smithi114.stdout:(174/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 172 kB/s | 74 kB 00:00 2024-03-27T10:40:58.509 INFO:teuthology.orchestra.run.smithi114.stdout:-------------------------------------------------------------------------------- 2024-03-27T10:40:58.510 INFO:teuthology.orchestra.run.smithi114.stdout:Total 11 MB/s | 227 MB 00:20 2024-03-27T10:40:58.678 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-27T10:40:58.714 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-27T10:40:58.730 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-27T10:40:58.765 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-27T10:40:58.801 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-27T10:40:58.821 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-27T10:40:58.913 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-27T10:40:58.930 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-27T10:40:58.962 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-27T10:40:58.965 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-27T10:40:58.988 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-27T10:40:59.037 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-03-27T10:40:59.099 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-27T10:40:59.153 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : librados-devel-2:19.0.0-2478.g155268c4.el9.x86_6 53/198 2024-03-27T10:40:59.193 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-27T10:40:59.204 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-27T10:40:59.229 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-27T10:40:59.322 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-27T10:40:59.617 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-27T10:40:59.835 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-27T10:40:59.911 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-27T10:40:59.977 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-27T10:41:00.103 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-27T10:41:00.121 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction check 2024-03-27T10:41:00.190 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-27T10:41:00.239 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-27T10:41:00.241 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-27T10:41:00.314 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-27T10:41:00.428 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-27T10:41:00.473 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-27T10:41:00.500 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction check succeeded. 2024-03-27T10:41:00.500 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction test 2024-03-27T10:41:00.512 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-27T10:41:00.580 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-27T10:41:00.620 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-27T10:41:00.684 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-27T10:41:00.758 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-27T10:41:00.949 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-27T10:41:01.010 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-27T10:41:01.048 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-27T10:41:01.105 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-27T10:41:01.237 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-27T10:41:01.271 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-03-27T10:41:01.317 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-27T10:41:01.353 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-27T10:41:01.386 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-27T10:41:01.542 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-27T10:41:01.710 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-27T10:41:01.746 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-27T10:41:01.834 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-27T10:41:01.847 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-27T10:41:01.880 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-27T10:41:01.897 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-27T10:41:01.919 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-27T10:41:01.958 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-27T10:41:01.979 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-27T10:41:02.032 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-27T10:41:02.266 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-27T10:41:02.326 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-27T10:41:02.368 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-27T10:41:02.400 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : librbd1-2:19.0.0-2478.g155268c4.el9.x86_64 124/198 2024-03-27T10:41:02.495 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-27T10:41:02.528 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librbd1-2:19.0.0-2478.g155268c4.el9.x86_64 124/198 2024-03-27T10:41:02.548 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-03-27T10:41:02.557 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-rbd-2:19.0.0-2478.g155268c4.el9.x86_64 125/198 2024-03-27T10:41:02.588 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-27T10:41:02.613 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-27T10:41:02.619 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-27T10:41:02.667 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-27T10:41:02.714 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-27T10:41:02.782 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-27T10:41:02.841 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-27T10:41:02.899 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-27T10:41:02.912 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction test succeeded. 2024-03-27T10:41:02.913 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction 2024-03-27T10:41:03.027 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-27T10:41:03.068 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-27T10:41:03.102 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-27T10:41:03.141 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-27T10:41:03.155 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-27T10:41:03.196 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-27T10:41:03.237 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : librgw2-2:19.0.0-2478.g155268c4.el9.x86_64 128/198 2024-03-27T10:41:03.339 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librgw2-2:19.0.0-2478.g155268c4.el9.x86_64 128/198 2024-03-27T10:41:03.372 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-27T10:41:03.373 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-rgw-2:19.0.0-2478.g155268c4.el9.x86_64 129/198 2024-03-27T10:41:03.485 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-27T10:41:03.544 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-27T10:41:03.580 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-27T10:41:03.636 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-27T10:41:03.772 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-27T10:41:03.837 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-27T10:41:04.788 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-27T10:41:04.844 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-27T10:41:04.878 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-27T10:41:04.936 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-27T10:41:05.045 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-27T10:41:05.095 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-27T10:41:05.328 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-27T10:41:05.367 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-27T10:41:05.459 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-27T10:41:05.965 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-common-2:19.0.0-2478.g155268c4.el9.x86_64 130/198 2024-03-27T10:41:06.011 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-common-2:19.0.0-2478.g155268c4.el9.x86_64 130/198 2024-03-27T10:41:06.243 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-27T10:41:06.369 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-27T10:41:06.644 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-common-2:19.0.0-2478.g155268c4.el9.x86_64 130/198 2024-03-27T10:41:06.690 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-base-2:19.0.0-2478.g155268c4.el9.x86_64 131/198 2024-03-27T10:41:06.888 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-base-2:19.0.0-2478.g155268c4.el9.x86_64 131/198 2024-03-27T10:41:06.888 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-27T10:41:06.888 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-27T10:41:06.888 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T10:41:06.966 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-selinux-2:19.0.0-2478.g155268c4.el9.x86_64 132/198 2024-03-27T10:41:07.874 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-27T10:41:07.954 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-27T10:41:07.992 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-27T10:41:08.022 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-27T10:41:08.078 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-27T10:41:08.448 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-27T10:41:08.478 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : librbd1-2:19.0.0-2478.g155268c4.el9.x86_64 124/198 2024-03-27T10:41:08.597 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: librbd1-2:19.0.0-2478.g155268c4.el9.x86_64 124/198 2024-03-27T10:41:08.626 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-rbd-2:19.0.0-2478.g155268c4.el9.x86_64 125/198 2024-03-27T10:41:08.680 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-27T10:41:09.186 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-27T10:41:09.216 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : librgw2-2:19.0.0-2478.g155268c4.el9.x86_64 128/198 2024-03-27T10:41:09.298 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: librgw2-2:19.0.0-2478.g155268c4.el9.x86_64 128/198 2024-03-27T10:41:09.326 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-rgw-2:19.0.0-2478.g155268c4.el9.x86_64 129/198 2024-03-27T10:41:10.721 INFO:teuthology.orchestra.run.smithi114.stdout: Preparing : 1/1 2024-03-27T10:41:10.875 INFO:teuthology.orchestra.run.smithi114.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-27T10:41:10.922 INFO:teuthology.orchestra.run.smithi114.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-27T10:41:10.960 INFO:teuthology.orchestra.run.smithi114.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-27T10:41:11.006 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-27T10:41:11.089 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-27T10:41:11.195 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-27T10:41:11.251 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-27T10:41:11.284 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-27T10:41:11.376 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-27T10:41:11.788 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-27T10:41:11.832 INFO:teuthology.orchestra.run.smithi114.stdout: Upgrading : librados2-2:19.0.0-2478.g155268c4.el9.x86_64 10/198 2024-03-27T10:41:11.982 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: librados2-2:19.0.0-2478.g155268c4.el9.x86_64 10/198 2024-03-27T10:41:11.987 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-common-2:19.0.0-2478.g155268c4.el9.x86_64 130/198 2024-03-27T10:41:12.016 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libcephfs2-2:19.0.0-2478.g155268c4.el9.x86_64 11/198 2024-03-27T10:41:12.023 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-common-2:19.0.0-2478.g155268c4.el9.x86_64 130/198 2024-03-27T10:41:12.138 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: libcephfs2-2:19.0.0-2478.g155268c4.el9.x86_64 11/198 2024-03-27T10:41:12.173 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-rados-2:19.0.0-2478.g155268c4.el9.x86_64 12/198 2024-03-27T10:41:12.237 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-03-27T10:41:12.558 INFO:teuthology.orchestra.run.smithi114.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-27T10:41:12.613 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-03-27T10:41:12.638 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libcephsqlite-2:19.0.0-2478.g155268c4.el9.x86_64 16/198 2024-03-27T10:41:12.641 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-common-2:19.0.0-2478.g155268c4.el9.x86_64 130/198 2024-03-27T10:41:12.684 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-base-2:19.0.0-2478.g155268c4.el9.x86_64 131/198 2024-03-27T10:41:12.797 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2478.g155268c4.el9.x86_64 16/198 2024-03-27T10:41:12.823 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libradosstriper1-2:19.0.0-2478.g155268c4.el9.x86 17/198 2024-03-27T10:41:12.837 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-base-2:19.0.0-2478.g155268c4.el9.x86_64 131/198 2024-03-27T10:41:12.837 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-27T10:41:12.837 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-27T10:41:12.837 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T10:41:12.893 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-selinux-2:19.0.0-2478.g155268c4.el9.x86_64 132/198 2024-03-27T10:41:12.904 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2478.g155268c4.el9.x86 17/198 2024-03-27T10:41:12.949 INFO:teuthology.orchestra.run.smithi114.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-27T10:41:12.974 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-03-27T10:41:13.034 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-03-27T10:41:13.267 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-03-27T10:41:13.308 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-03-27T10:41:13.349 INFO:teuthology.orchestra.run.smithi114.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-27T10:41:13.688 INFO:teuthology.orchestra.run.smithi114.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-27T10:41:13.879 INFO:teuthology.orchestra.run.smithi114.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-27T10:41:13.962 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-27T10:41:14.135 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-27T10:41:14.185 INFO:teuthology.orchestra.run.smithi114.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-27T10:41:14.493 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-27T10:41:14.663 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-27T10:41:14.795 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-27T10:41:14.832 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-27T10:41:14.868 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-27T10:41:14.927 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-27T10:41:15.061 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-27T10:41:15.132 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-27T10:41:15.174 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-27T10:41:15.255 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-27T10:41:15.295 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-27T10:41:15.378 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-27T10:41:15.425 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-27T10:41:15.458 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-27T10:41:15.573 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-27T10:41:15.612 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-ceph-common-2:19.0.0-2478.g155268c4.el9. 39/198 2024-03-27T10:41:15.661 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-ceph-argparse-2:19.0.0-2478.g155268c4.el 40/198 2024-03-27T10:41:15.693 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-cephfs-2:19.0.0-2478.g155268c4.el9.x86_6 41/198 2024-03-27T10:41:15.834 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-27T10:41:15.895 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-27T10:41:15.995 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-27T10:41:16.164 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-27T10:41:16.232 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-27T10:41:16.916 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-27T10:41:17.012 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-27T10:41:17.136 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-27T10:41:17.222 INFO:teuthology.orchestra.run.smithi114.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-27T10:41:17.250 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-27T10:41:17.296 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-03-27T10:41:17.367 INFO:teuthology.orchestra.run.smithi114.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-27T10:41:17.424 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : librados-devel-2:19.0.0-2478.g155268c4.el9.x86_6 53/198 2024-03-27T10:41:17.478 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-27T10:41:17.864 INFO:teuthology.orchestra.run.smithi114.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-27T10:41:18.093 INFO:teuthology.orchestra.run.smithi114.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-27T10:41:18.162 INFO:teuthology.orchestra.run.smithi114.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-27T10:41:18.234 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-27T10:41:18.438 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-27T10:41:18.517 INFO:teuthology.orchestra.run.smithi114.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-27T10:41:18.601 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-27T10:41:18.697 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-27T10:41:18.767 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-27T10:41:18.807 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-27T10:41:18.878 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-27T10:41:18.923 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-27T10:41:18.986 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-27T10:41:19.051 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-27T10:41:19.262 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-27T10:41:19.313 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-27T10:41:19.350 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-27T10:41:19.406 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-27T10:41:19.535 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-27T10:41:19.573 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-03-27T10:41:19.628 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-27T10:41:19.664 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-27T10:41:19.697 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-27T10:41:19.796 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2478.g155268c4.el9.x86_64 132/198 2024-03-27T10:41:19.796 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /sys 2024-03-27T10:41:19.796 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /proc 2024-03-27T10:41:19.796 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /mnt 2024-03-27T10:41:19.796 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /var/tmp 2024-03-27T10:41:19.797 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /home 2024-03-27T10:41:19.797 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /root 2024-03-27T10:41:19.797 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /tmp 2024-03-27T10:41:19.797 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T10:41:19.857 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-27T10:41:20.036 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-27T10:41:20.038 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-volume-2:19.0.0-2478.g155268c4.el9.noarch 133/198 2024-03-27T10:41:20.083 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-volume-2:19.0.0-2478.g155268c4.el9.noarch 133/198 2024-03-27T10:41:20.083 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-27T10:41:20.083 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-27T10:41:20.084 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T10:41:20.171 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-27T10:41:20.249 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-27T10:41:20.315 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-27T10:41:20.389 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-27T10:41:20.628 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-27T10:41:20.695 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-27T10:41:20.866 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-27T10:41:20.909 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-03-27T10:41:20.974 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-27T10:41:21.022 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-27T10:41:21.063 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-27T10:41:21.110 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-27T10:41:21.178 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-27T10:41:21.234 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-27T10:41:21.275 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-27T10:41:21.430 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-27T10:41:21.470 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-27T10:41:21.496 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-27T10:41:21.548 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-27T10:41:21.590 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-27T10:41:21.702 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-osd-2:19.0.0-2478.g155268c4.el9.x86_64 134/198 2024-03-27T10:41:21.732 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-27T10:41:21.782 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-osd-2:19.0.0-2478.g155268c4.el9.x86_64 134/198 2024-03-27T10:41:21.782 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-27T10:41:21.782 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-27T10:41:21.783 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-27T10:41:21.783 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-27T10:41:21.783 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T10:41:21.853 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-27T10:41:21.904 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-27T10:41:21.941 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-27T10:41:21.997 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-27T10:41:22.060 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mds-2:19.0.0-2478.g155268c4.el9.x86_64 135/198 2024-03-27T10:41:22.128 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mds-2:19.0.0-2478.g155268c4.el9.x86_64 135/198 2024-03-27T10:41:22.128 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-27T10:41:22.129 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-27T10:41:22.129 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-27T10:41:22.129 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-27T10:41:22.129 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T10:41:22.166 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-27T10:41:22.240 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-27T10:41:22.603 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mon-2:19.0.0-2478.g155268c4.el9.x86_64 136/198 2024-03-27T10:41:22.673 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mon-2:19.0.0-2478.g155268c4.el9.x86_64 136/198 2024-03-27T10:41:22.673 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-27T10:41:22.673 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-27T10:41:22.673 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-27T10:41:22.673 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-27T10:41:22.673 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T10:41:22.788 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-27T10:41:23.198 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-27T10:41:23.247 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-27T10:41:23.280 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-27T10:41:23.335 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-27T10:41:23.437 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: cephadm-2:19.0.0-2478.g155268c4.el9.noarch 138/198 2024-03-27T10:41:23.446 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-27T10:41:23.474 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : cephadm-2:19.0.0-2478.g155268c4.el9.noarch 138/198 2024-03-27T10:41:23.489 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-27T10:41:23.523 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2478.g155268c4.e 139/198 2024-03-27T10:41:23.625 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2478.g155268c4. 140/198 2024-03-27T10:41:23.662 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2478.g155268c4.el9.noa 141/198 2024-03-27T10:41:23.746 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-27T10:41:23.787 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-27T10:41:23.920 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-27T10:41:24.722 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-27T10:41:24.848 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-27T10:41:24.955 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2478.g155268c4.el9.noa 141/198 2024-03-27T10:41:25.005 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2478.g155268c4.el9.n 142/198 2024-03-27T10:41:25.771 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2478.g155268c4.el9.x86_64 132/198 2024-03-27T10:41:25.771 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /sys 2024-03-27T10:41:25.771 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /proc 2024-03-27T10:41:25.771 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /mnt 2024-03-27T10:41:25.771 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /var/tmp 2024-03-27T10:41:25.771 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /home 2024-03-27T10:41:25.771 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /root 2024-03-27T10:41:25.771 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /tmp 2024-03-27T10:41:25.771 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T10:41:26.021 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-volume-2:19.0.0-2478.g155268c4.el9.noarch 133/198 2024-03-27T10:41:26.070 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-volume-2:19.0.0-2478.g155268c4.el9.noarch 133/198 2024-03-27T10:41:26.070 INFO:teuthology.orchestra.run.smithi177.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-27T10:41:26.070 INFO:teuthology.orchestra.run.smithi177.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-27T10:41:26.070 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T10:41:26.090 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2478.g155268c4.el9.n 142/198 2024-03-27T10:41:26.122 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2478.g155 143/198 2024-03-27T10:41:26.264 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2478.g155 143/198 2024-03-27T10:41:26.370 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-27T10:41:26.444 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2478.g155268c4.el 144/198 2024-03-27T10:41:26.475 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-2:19.0.0-2478.g155268c4.el9.x86_64 145/198 2024-03-27T10:41:26.539 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2478.g155268c4.el9.x86_64 145/198 2024-03-27T10:41:26.539 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-27T10:41:26.539 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-27T10:41:26.539 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-27T10:41:26.539 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-27T10:41:26.539 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T10:41:26.600 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-rook-2:19.0.0-2478.g155268c4.el9.noarch 146/198 2024-03-27T10:41:26.635 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2478.g155268c4.el9.noarch 146/198 2024-03-27T10:41:26.750 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-2:19.0.0-2478.g155268c4.el9.x86_64 147/198 2024-03-27T10:41:26.922 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-27T10:41:26.970 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-27T10:41:27.038 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-27T10:41:27.089 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-27T10:41:27.502 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-27T10:41:27.539 INFO:teuthology.orchestra.run.smithi114.stdout: Upgrading : librbd1-2:19.0.0-2478.g155268c4.el9.x86_64 124/198 2024-03-27T10:41:27.659 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: librbd1-2:19.0.0-2478.g155268c4.el9.x86_64 124/198 2024-03-27T10:41:27.687 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-rbd-2:19.0.0-2478.g155268c4.el9.x86_64 125/198 2024-03-27T10:41:27.708 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-27T10:41:27.716 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-osd-2:19.0.0-2478.g155268c4.el9.x86_64 134/198 2024-03-27T10:41:27.744 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-radosgw-2:19.0.0-2478.g155268c4.el9.x86_64 149/198 2024-03-27T10:41:27.756 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-27T10:41:27.791 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-osd-2:19.0.0-2478.g155268c4.el9.x86_64 134/198 2024-03-27T10:41:27.792 INFO:teuthology.orchestra.run.smithi177.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-27T10:41:27.792 INFO:teuthology.orchestra.run.smithi177.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-27T10:41:27.792 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-27T10:41:27.792 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-27T10:41:27.792 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T10:41:27.814 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2478.g155268c4.el9.x86_64 149/198 2024-03-27T10:41:27.814 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-27T10:41:27.814 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-27T10:41:27.815 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-27T10:41:27.815 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-27T10:41:27.815 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T10:41:27.870 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2478.g15526 150/198 2024-03-27T10:41:27.954 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2478.g15526 150/198 2024-03-27T10:41:27.954 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-27T10:41:27.954 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-27T10:41:27.954 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T10:41:28.021 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : cephfs-mirror-2:19.0.0-2478.g155268c4.el9.x86_64 151/198 2024-03-27T10:41:28.041 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mds-2:19.0.0-2478.g155268c4.el9.x86_64 135/198 2024-03-27T10:41:28.097 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2478.g155268c4.el9.x86_64 151/198 2024-03-27T10:41:28.097 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-27T10:41:28.097 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-27T10:41:28.097 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-27T10:41:28.097 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-27T10:41:28.098 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T10:41:28.107 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-mds-2:19.0.0-2478.g155268c4.el9.x86_64 135/198 2024-03-27T10:41:28.107 INFO:teuthology.orchestra.run.smithi177.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-27T10:41:28.107 INFO:teuthology.orchestra.run.smithi177.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-27T10:41:28.107 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-27T10:41:28.107 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-27T10:41:28.107 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T10:41:28.318 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-27T10:41:28.342 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : librgw2-2:19.0.0-2478.g155268c4.el9.x86_64 128/198 2024-03-27T10:41:28.435 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: librgw2-2:19.0.0-2478.g155268c4.el9.x86_64 128/198 2024-03-27T10:41:28.454 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : rbd-mirror-2:19.0.0-2478.g155268c4.el9.x86_64 152/198 2024-03-27T10:41:28.461 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-rgw-2:19.0.0-2478.g155268c4.el9.x86_64 129/198 2024-03-27T10:41:28.525 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2478.g155268c4.el9.x86_64 152/198 2024-03-27T10:41:28.526 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-27T10:41:28.526 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-27T10:41:28.526 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-27T10:41:28.526 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-27T10:41:28.526 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T10:41:28.583 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mon-2:19.0.0-2478.g155268c4.el9.x86_64 136/198 2024-03-27T10:41:28.652 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-mon-2:19.0.0-2478.g155268c4.el9.x86_64 136/198 2024-03-27T10:41:28.652 INFO:teuthology.orchestra.run.smithi177.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-27T10:41:28.652 INFO:teuthology.orchestra.run.smithi177.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-27T10:41:28.652 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-27T10:41:28.652 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-27T10:41:28.652 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T10:41:28.777 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-27T10:41:29.424 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: cephadm-2:19.0.0-2478.g155268c4.el9.noarch 138/198 2024-03-27T10:41:29.472 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : cephadm-2:19.0.0-2478.g155268c4.el9.noarch 138/198 2024-03-27T10:41:29.522 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2478.g155268c4.e 139/198 2024-03-27T10:41:29.624 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2478.g155268c4. 140/198 2024-03-27T10:41:29.657 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2478.g155268c4.el9.noa 141/198 2024-03-27T10:41:30.950 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2478.g155268c4.el9.noa 141/198 2024-03-27T10:41:30.984 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2478.g155268c4.el9.n 142/198 2024-03-27T10:41:31.114 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-common-2:19.0.0-2478.g155268c4.el9.x86_64 130/198 2024-03-27T10:41:31.157 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-common-2:19.0.0-2478.g155268c4.el9.x86_64 130/198 2024-03-27T10:41:31.789 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-common-2:19.0.0-2478.g155268c4.el9.x86_64 130/198 2024-03-27T10:41:31.861 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-base-2:19.0.0-2478.g155268c4.el9.x86_64 131/198 2024-03-27T10:41:32.013 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-base-2:19.0.0-2478.g155268c4.el9.x86_64 131/198 2024-03-27T10:41:32.013 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-27T10:41:32.014 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-27T10:41:32.014 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-03-27T10:41:32.053 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-selinux-2:19.0.0-2478.g155268c4.el9.x86_64 132/198 2024-03-27T10:41:32.062 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2478.g155268c4.el9.n 142/198 2024-03-27T10:41:32.092 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2478.g155 143/198 2024-03-27T10:41:32.234 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2478.g155 143/198 2024-03-27T10:41:32.406 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2478.g155268c4.el 144/198 2024-03-27T10:41:32.435 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mgr-2:19.0.0-2478.g155268c4.el9.x86_64 145/198 2024-03-27T10:41:32.507 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2478.g155268c4.el9.x86_64 145/198 2024-03-27T10:41:32.507 INFO:teuthology.orchestra.run.smithi177.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-27T10:41:32.507 INFO:teuthology.orchestra.run.smithi177.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-27T10:41:32.507 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-27T10:41:32.507 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-27T10:41:32.507 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T10:41:32.570 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mgr-rook-2:19.0.0-2478.g155268c4.el9.noarch 146/198 2024-03-27T10:41:32.607 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2478.g155268c4.el9.noarch 146/198 2024-03-27T10:41:32.666 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-2:19.0.0-2478.g155268c4.el9.x86_64 147/198 2024-03-27T10:41:33.334 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-test-2:19.0.0-2478.g155268c4.el9.x86_64 153/198 2024-03-27T10:41:33.401 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : rbd-fuse-2:19.0.0-2478.g155268c4.el9.x86_64 154/198 2024-03-27T10:41:33.443 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : rbd-nbd-2:19.0.0-2478.g155268c4.el9.x86_64 155/198 2024-03-27T10:41:33.490 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-27T10:41:33.670 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-27T10:41:33.705 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-radosgw-2:19.0.0-2478.g155268c4.el9.x86_64 149/198 2024-03-27T10:41:33.771 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2478.g155268c4.el9.x86_64 149/198 2024-03-27T10:41:33.771 INFO:teuthology.orchestra.run.smithi177.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-27T10:41:33.771 INFO:teuthology.orchestra.run.smithi177.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-27T10:41:33.771 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-27T10:41:33.771 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-27T10:41:33.771 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T10:41:33.784 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : cephfs-shell-2:19.0.0-2478.g155268c4.el9.x86_64 157/198 2024-03-27T10:41:33.814 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2478.g15526 150/198 2024-03-27T10:41:33.879 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2478.g15526 150/198 2024-03-27T10:41:33.879 INFO:teuthology.orchestra.run.smithi177.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-27T10:41:33.879 INFO:teuthology.orchestra.run.smithi177.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-27T10:41:33.879 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T10:41:33.904 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 158/198 2024-03-27T10:41:33.957 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : cephfs-mirror-2:19.0.0-2478.g155268c4.el9.x86_64 151/198 2024-03-27T10:41:33.960 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-fuse-2:19.0.0-2478.g155268c4.el9.x86_64 159/198 2024-03-27T10:41:34.008 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-27T10:41:34.030 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2478.g155268c4.el9.x86_64 151/198 2024-03-27T10:41:34.030 INFO:teuthology.orchestra.run.smithi177.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-27T10:41:34.031 INFO:teuthology.orchestra.run.smithi177.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-27T10:41:34.031 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-27T10:41:34.031 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-27T10:41:34.031 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T10:41:34.068 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-27T10:41:34.113 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-27T10:41:34.148 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libcephfs-devel-2:19.0.0-2478.g155268c4.el9.x86_ 163/198 2024-03-27T10:41:34.185 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-27T10:41:34.237 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-03-27T10:41:34.348 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-03-27T10:41:34.365 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : rbd-mirror-2:19.0.0-2478.g155268c4.el9.x86_64 152/198 2024-03-27T10:41:34.433 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2478.g155268c4.el9.x86_64 152/198 2024-03-27T10:41:34.433 INFO:teuthology.orchestra.run.smithi177.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-27T10:41:34.433 INFO:teuthology.orchestra.run.smithi177.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-27T10:41:34.433 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-27T10:41:34.434 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-27T10:41:34.434 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T10:41:34.460 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-27T10:41:34.495 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-27T10:41:35.411 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-27T10:41:35.467 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-27T10:41:35.530 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-27T10:41:36.860 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : cephfs-top-2:19.0.0-2478.g155268c4.el9.noarch 170/198 2024-03-27T10:41:36.948 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-27T10:41:37.000 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-27T10:41:37.133 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-27T10:41:37.167 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-27T10:41:37.200 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-27T10:41:37.235 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-27T10:41:37.277 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-27T10:41:37.277 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-27T10:41:37.422 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-27T10:41:37.473 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-27T10:41:37.474 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-27T10:41:37.627 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-27T10:41:37.672 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-27T10:41:37.718 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-27T10:41:37.751 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-27T10:41:37.783 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-27T10:41:37.853 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-27T10:41:37.927 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-27T10:41:37.971 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-27T10:41:38.005 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-27T10:41:38.068 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-27T10:41:38.105 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-27T10:41:38.139 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-27T10:41:38.176 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-27T10:41:38.176 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-27T10:41:38.489 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-27T10:41:38.490 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-27T10:41:38.628 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-27T10:41:38.659 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-27T10:41:38.714 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-27T10:41:38.753 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-27T10:41:38.794 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-27T10:41:38.911 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-27T10:41:39.193 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-27T10:41:39.193 INFO:teuthology.orchestra.run.smithi097.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-03-27T10:41:39.193 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T10:41:39.196 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-test-2:19.0.0-2478.g155268c4.el9.x86_64 153/198 2024-03-27T10:41:39.239 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : rbd-fuse-2:19.0.0-2478.g155268c4.el9.x86_64 154/198 2024-03-27T10:41:39.276 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : rbd-nbd-2:19.0.0-2478.g155268c4.el9.x86_64 155/198 2024-03-27T10:41:39.314 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-27T10:41:39.568 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : cephfs-shell-2:19.0.0-2478.g155268c4.el9.x86_64 157/198 2024-03-27T10:41:39.697 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 158/198 2024-03-27T10:41:39.745 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-fuse-2:19.0.0-2478.g155268c4.el9.x86_64 159/198 2024-03-27T10:41:39.797 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-27T10:41:39.853 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-27T10:41:39.898 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-27T10:41:39.932 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libcephfs-devel-2:19.0.0-2478.g155268c4.el9.x86_ 163/198 2024-03-27T10:41:39.987 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-27T10:41:40.038 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-03-27T10:41:40.137 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-03-27T10:41:40.241 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-27T10:41:40.279 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-27T10:41:41.210 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-27T10:41:41.259 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-27T10:41:41.308 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-27T10:41:42.642 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : cephfs-top-2:19.0.0-2478.g155268c4.el9.noarch 170/198 2024-03-27T10:41:42.740 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-27T10:41:42.796 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-27T10:41:42.925 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-27T10:41:42.966 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-27T10:41:42.999 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-27T10:41:43.034 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-27T10:41:43.078 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-27T10:41:43.078 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-27T10:41:43.222 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-27T10:41:43.272 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-27T10:41:43.272 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-27T10:41:43.402 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-27T10:41:43.451 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-27T10:41:43.501 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-27T10:41:43.541 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-27T10:41:43.574 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-27T10:41:43.643 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-27T10:41:43.711 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-27T10:41:43.754 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-27T10:41:43.787 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-27T10:41:43.820 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-27T10:41:43.854 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-27T10:41:43.887 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-27T10:41:43.923 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-27T10:41:43.923 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-27T10:41:44.255 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-27T10:41:44.255 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-27T10:41:44.402 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-27T10:41:44.433 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-27T10:41:44.486 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-27T10:41:44.519 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-27T10:41:44.559 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-27T10:41:44.678 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-27T10:41:44.753 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-2:19.0.0-2478.g155268c4.el9.x86_64 1/198 2024-03-27T10:41:44.753 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-base-2:19.0.0-2478.g155268c4.el9.x86_64 2/198 2024-03-27T10:41:44.754 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-common-2:19.0.0-2478.g155268c4.el9.x86_64 3/198 2024-03-27T10:41:44.754 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-fuse-2:19.0.0-2478.g155268c4.el9.x86_64 4/198 2024-03-27T10:41:44.754 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2478.g15526 5/198 2024-03-27T10:41:44.754 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mds-2:19.0.0-2478.g155268c4.el9.x86_64 6/198 2024-03-27T10:41:44.754 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-2:19.0.0-2478.g155268c4.el9.x86_64 7/198 2024-03-27T10:41:44.754 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mon-2:19.0.0-2478.g155268c4.el9.x86_64 8/198 2024-03-27T10:41:44.754 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-osd-2:19.0.0-2478.g155268c4.el9.x86_64 9/198 2024-03-27T10:41:44.754 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-radosgw-2:19.0.0-2478.g155268c4.el9.x86_64 10/198 2024-03-27T10:41:44.754 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-selinux-2:19.0.0-2478.g155268c4.el9.x86_64 11/198 2024-03-27T10:41:44.754 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-test-2:19.0.0-2478.g155268c4.el9.x86_64 12/198 2024-03-27T10:41:44.754 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : cephfs-mirror-2:19.0.0-2478.g155268c4.el9.x86_64 13/198 2024-03-27T10:41:44.755 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : cephfs-shell-2:19.0.0-2478.g155268c4.el9.x86_64 14/198 2024-03-27T10:41:44.755 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libcephfs-devel-2:19.0.0-2478.g155268c4.el9.x86_ 15/198 2024-03-27T10:41:44.755 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libcephfs2-2:19.0.0-2478.g155268c4.el9.x86_64 16/198 2024-03-27T10:41:44.755 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libcephsqlite-2:19.0.0-2478.g155268c4.el9.x86_64 17/198 2024-03-27T10:41:44.755 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librados-devel-2:19.0.0-2478.g155268c4.el9.x86_6 18/198 2024-03-27T10:41:44.755 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libradosstriper1-2:19.0.0-2478.g155268c4.el9.x86 19/198 2024-03-27T10:41:44.755 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librgw2-2:19.0.0-2478.g155268c4.el9.x86_64 20/198 2024-03-27T10:41:44.755 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2478.g155268c4.el 21/198 2024-03-27T10:41:44.755 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-ceph-common-2:19.0.0-2478.g155268c4.el9. 22/198 2024-03-27T10:41:44.755 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cephfs-2:19.0.0-2478.g155268c4.el9.x86_6 23/198 2024-03-27T10:41:44.755 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-rados-2:19.0.0-2478.g155268c4.el9.x86_64 24/198 2024-03-27T10:41:44.755 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-rbd-2:19.0.0-2478.g155268c4.el9.x86_64 25/198 2024-03-27T10:41:44.755 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-rgw-2:19.0.0-2478.g155268c4.el9.x86_64 26/198 2024-03-27T10:41:44.756 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : rbd-fuse-2:19.0.0-2478.g155268c4.el9.x86_64 27/198 2024-03-27T10:41:44.756 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : rbd-mirror-2:19.0.0-2478.g155268c4.el9.x86_64 28/198 2024-03-27T10:41:44.756 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : rbd-nbd-2:19.0.0-2478.g155268c4.el9.x86_64 29/198 2024-03-27T10:41:44.756 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2478.g155268c4. 30/198 2024-03-27T10:41:44.756 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2478.g155268c4.el9.noa 31/198 2024-03-27T10:41:44.756 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2478.g155268c4.el9.n 32/198 2024-03-27T10:41:44.772 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2478.g155 33/198 2024-03-27T10:41:44.772 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2478.g155268c4.el 34/198 2024-03-27T10:41:44.772 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2478.g155268c4.el9.noarch 35/198 2024-03-27T10:41:44.772 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2478.g155268c4.e 36/198 2024-03-27T10:41:44.772 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-volume-2:19.0.0-2478.g155268c4.el9.noarch 37/198 2024-03-27T10:41:44.772 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : cephadm-2:19.0.0-2478.g155268c4.el9.noarch 38/198 2024-03-27T10:41:44.772 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : cephfs-top-2:19.0.0-2478.g155268c4.el9.noarch 39/198 2024-03-27T10:41:44.772 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-27T10:41:44.772 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-27T10:41:44.772 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-27T10:41:44.772 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-27T10:41:44.773 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-27T10:41:44.773 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-27T10:41:44.773 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-27T10:41:44.773 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-27T10:41:44.773 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-27T10:41:44.773 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-27T10:41:44.773 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-27T10:41:44.773 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-27T10:41:44.773 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-27T10:41:44.773 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-27T10:41:44.773 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-27T10:41:44.773 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-27T10:41:44.774 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-27T10:41:44.774 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-27T10:41:44.774 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 58/198 2024-03-27T10:41:44.774 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 59/198 2024-03-27T10:41:44.774 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 60/198 2024-03-27T10:41:44.774 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 61/198 2024-03-27T10:41:44.774 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 62/198 2024-03-27T10:41:44.774 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 63/198 2024-03-27T10:41:44.774 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 64/198 2024-03-27T10:41:44.774 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 65/198 2024-03-27T10:41:44.774 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 66/198 2024-03-27T10:41:44.774 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 67/198 2024-03-27T10:41:44.775 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 68/198 2024-03-27T10:41:44.775 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 69/198 2024-03-27T10:41:44.775 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 70/198 2024-03-27T10:41:44.775 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 71/198 2024-03-27T10:41:44.775 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 72/198 2024-03-27T10:41:44.775 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 73/198 2024-03-27T10:41:44.775 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 74/198 2024-03-27T10:41:44.775 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 75/198 2024-03-27T10:41:44.775 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 76/198 2024-03-27T10:41:44.775 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 77/198 2024-03-27T10:41:44.775 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 78/198 2024-03-27T10:41:44.775 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 79/198 2024-03-27T10:41:44.776 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 80/198 2024-03-27T10:41:44.776 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 81/198 2024-03-27T10:41:44.776 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 82/198 2024-03-27T10:41:44.776 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 83/198 2024-03-27T10:41:44.776 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 84/198 2024-03-27T10:41:44.776 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-ply-3.11-14.el9.noarch 85/198 2024-03-27T10:41:44.776 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 86/198 2024-03-27T10:41:44.776 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 87/198 2024-03-27T10:41:44.776 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : jq-1.6-16.el9.x86_64 88/198 2024-03-27T10:41:44.776 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 89/198 2024-03-27T10:41:44.776 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 90/198 2024-03-27T10:41:44.776 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 91/198 2024-03-27T10:41:44.777 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 92/198 2024-03-27T10:41:44.777 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 93/198 2024-03-27T10:41:44.777 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 94/198 2024-03-27T10:41:44.777 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : postgresql-13.14-1.el9.x86_64 95/198 2024-03-27T10:41:44.777 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 96/198 2024-03-27T10:41:44.777 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 97/198 2024-03-27T10:41:44.777 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 98/198 2024-03-27T10:41:44.777 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-27T10:41:44.777 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-27T10:41:44.777 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 101/198 2024-03-27T10:41:44.777 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 102/198 2024-03-27T10:41:44.777 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 103/198 2024-03-27T10:41:44.777 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-27T10:41:44.778 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-27T10:41:44.778 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-27T10:41:44.778 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-27T10:41:44.778 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-27T10:41:44.778 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-27T10:41:44.778 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-27T10:41:44.778 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-27T10:41:44.778 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-27T10:41:44.778 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-27T10:41:44.778 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-27T10:41:44.778 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-27T10:41:44.778 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-03-27T10:41:44.778 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-27T10:41:44.779 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-27T10:41:44.779 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-27T10:41:44.779 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-27T10:41:44.779 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-27T10:41:44.779 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-27T10:41:44.779 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-27T10:41:44.779 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-03-27T10:41:44.779 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-27T10:41:44.779 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-27T10:41:44.779 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-27T10:41:44.779 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-27T10:41:44.779 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-27T10:41:44.779 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-27T10:41:44.779 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-27T10:41:44.780 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-27T10:41:44.780 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-27T10:41:44.780 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-27T10:41:44.780 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-27T10:41:44.780 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-27T10:41:44.780 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-27T10:41:44.780 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-27T10:41:44.780 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-27T10:41:44.780 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-27T10:41:44.780 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-27T10:41:44.780 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-27T10:41:44.780 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-27T10:41:44.780 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-27T10:41:44.781 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-27T10:41:44.781 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-27T10:41:44.781 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-27T10:41:44.781 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-27T10:41:44.781 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-27T10:41:44.781 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-27T10:41:44.781 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librados2-2:19.0.0-2478.g155268c4.el9.x86_64 151/198 2024-03-27T10:41:44.781 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-27T10:41:44.781 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librbd1-2:19.0.0-2478.g155268c4.el9.x86_64 153/198 2024-03-27T10:41:44.781 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-27T10:41:44.781 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-27T10:41:44.781 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-27T10:41:44.781 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 157/198 2024-03-27T10:41:44.782 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-27T10:41:44.782 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 159/198 2024-03-27T10:41:44.782 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-27T10:41:44.782 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 161/198 2024-03-27T10:41:44.782 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-27T10:41:44.782 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 163/198 2024-03-27T10:41:44.782 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-27T10:41:44.782 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-27T10:41:44.782 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-27T10:41:44.782 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-27T10:41:44.782 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-27T10:41:44.782 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-27T10:41:44.782 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-27T10:41:44.783 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-27T10:41:44.783 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-27T10:41:44.783 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-27T10:41:44.783 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-27T10:41:44.783 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-27T10:41:44.783 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-27T10:41:44.783 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-27T10:41:44.783 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-27T10:41:44.783 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-27T10:41:44.783 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-27T10:41:44.783 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-27T10:41:44.783 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-27T10:41:44.783 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-27T10:41:44.784 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-27T10:41:44.784 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-27T10:41:44.784 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-27T10:41:44.784 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-27T10:41:44.784 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-27T10:41:44.784 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-27T10:41:44.784 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-27T10:41:44.784 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-27T10:41:44.784 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-27T10:41:44.784 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-27T10:41:44.784 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-27T10:41:44.784 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-27T10:41:44.784 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-27T10:41:44.785 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-27T10:41:44.929 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-27T10:41:44.929 INFO:teuthology.orchestra.run.smithi177.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-03-27T10:41:44.929 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T10:41:44.975 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2478.g155268c4.el9.x86_64 132/198 2024-03-27T10:41:44.975 INFO:teuthology.orchestra.run.smithi114.stdout:skipping the directory /sys 2024-03-27T10:41:44.975 INFO:teuthology.orchestra.run.smithi114.stdout:skipping the directory /proc 2024-03-27T10:41:44.975 INFO:teuthology.orchestra.run.smithi114.stdout:skipping the directory /mnt 2024-03-27T10:41:44.975 INFO:teuthology.orchestra.run.smithi114.stdout:skipping the directory /var/tmp 2024-03-27T10:41:44.975 INFO:teuthology.orchestra.run.smithi114.stdout:skipping the directory /home 2024-03-27T10:41:44.975 INFO:teuthology.orchestra.run.smithi114.stdout:skipping the directory /root 2024-03-27T10:41:44.975 INFO:teuthology.orchestra.run.smithi114.stdout:skipping the directory /tmp 2024-03-27T10:41:44.975 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-03-27T10:41:45.199 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-volume-2:19.0.0-2478.g155268c4.el9.noarch 133/198 2024-03-27T10:41:45.246 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-volume-2:19.0.0-2478.g155268c4.el9.noarch 133/198 2024-03-27T10:41:45.246 INFO:teuthology.orchestra.run.smithi114.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-27T10:41:45.246 INFO:teuthology.orchestra.run.smithi114.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-27T10:41:45.246 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-03-27T10:41:46.862 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-osd-2:19.0.0-2478.g155268c4.el9.x86_64 134/198 2024-03-27T10:41:46.936 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-osd-2:19.0.0-2478.g155268c4.el9.x86_64 134/198 2024-03-27T10:41:46.936 INFO:teuthology.orchestra.run.smithi114.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-27T10:41:46.936 INFO:teuthology.orchestra.run.smithi114.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-27T10:41:46.936 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-27T10:41:46.936 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-27T10:41:46.937 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-03-27T10:41:47.187 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-mds-2:19.0.0-2478.g155268c4.el9.x86_64 135/198 2024-03-27T10:41:47.254 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-mds-2:19.0.0-2478.g155268c4.el9.x86_64 135/198 2024-03-27T10:41:47.254 INFO:teuthology.orchestra.run.smithi114.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-27T10:41:47.254 INFO:teuthology.orchestra.run.smithi114.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-27T10:41:47.254 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-27T10:41:47.254 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-27T10:41:47.255 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-03-27T10:41:47.378 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-27T10:41:47.378 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T10:41:47.378 INFO:teuthology.orchestra.run.smithi097.stdout:Upgraded: 2024-03-27T10:41:47.378 INFO:teuthology.orchestra.run.smithi097.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-03-27T10:41:47.378 INFO:teuthology.orchestra.run.smithi097.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-03-27T10:41:47.378 INFO:teuthology.orchestra.run.smithi097.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-03-27T10:41:47.378 INFO:teuthology.orchestra.run.smithi097.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-03-27T10:41:47.379 INFO:teuthology.orchestra.run.smithi097.stdout: acl-2.3.1-4.el9.x86_64 2024-03-27T10:41:47.379 INFO:teuthology.orchestra.run.smithi097.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-27T10:41:47.379 INFO:teuthology.orchestra.run.smithi097.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-27T10:41:47.379 INFO:teuthology.orchestra.run.smithi097.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-27T10:41:47.379 INFO:teuthology.orchestra.run.smithi097.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-27T10:41:47.379 INFO:teuthology.orchestra.run.smithi097.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-27T10:41:47.379 INFO:teuthology.orchestra.run.smithi097.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-27T10:41:47.379 INFO:teuthology.orchestra.run.smithi097.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-27T10:41:47.379 INFO:teuthology.orchestra.run.smithi097.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-27T10:41:47.379 INFO:teuthology.orchestra.run.smithi097.stdout: librados2-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T10:41:47.379 INFO:teuthology.orchestra.run.smithi097.stdout: librbd1-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T10:41:47.379 INFO:teuthology.orchestra.run.smithi097.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-27T10:41:47.379 INFO:teuthology.orchestra.run.smithi097.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-27T10:41:47.379 INFO:teuthology.orchestra.run.smithi097.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-27T10:41:47.380 INFO:teuthology.orchestra.run.smithi097.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-27T10:41:47.380 INFO:teuthology.orchestra.run.smithi097.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-27T10:41:47.380 INFO:teuthology.orchestra.run.smithi097.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-27T10:41:47.380 INFO:teuthology.orchestra.run.smithi097.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-27T10:41:47.380 INFO:teuthology.orchestra.run.smithi097.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-27T10:41:47.380 INFO:teuthology.orchestra.run.smithi097.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-27T10:41:47.380 INFO:teuthology.orchestra.run.smithi097.stdout:Installed: 2024-03-27T10:41:47.380 INFO:teuthology.orchestra.run.smithi097.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-27T10:41:47.380 INFO:teuthology.orchestra.run.smithi097.stdout: automake-1.16.2-8.el9.noarch 2024-03-27T10:41:47.380 INFO:teuthology.orchestra.run.smithi097.stdout: bison-3.7.4-5.el9.x86_64 2024-03-27T10:41:47.380 INFO:teuthology.orchestra.run.smithi097.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-27T10:41:47.380 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T10:41:47.380 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-base-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T10:41:47.381 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-common-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T10:41:47.381 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-fuse-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T10:41:47.381 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-grafana-dashboards-2:19.0.0-2478.g155268c4.el9.noarch 2024-03-27T10:41:47.381 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-immutable-object-cache-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T10:41:47.381 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mds-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T10:41:47.381 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T10:41:47.381 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-cephadm-2:19.0.0-2478.g155268c4.el9.noarch 2024-03-27T10:41:47.381 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-dashboard-2:19.0.0-2478.g155268c4.el9.noarch 2024-03-27T10:41:47.381 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2478.g155268c4.el9.noarch 2024-03-27T10:41:47.381 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-modules-core-2:19.0.0-2478.g155268c4.el9.noarch 2024-03-27T10:41:47.381 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-rook-2:19.0.0-2478.g155268c4.el9.noarch 2024-03-27T10:41:47.381 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mon-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T10:41:47.381 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-osd-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T10:41:47.382 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-prometheus-alerts-2:19.0.0-2478.g155268c4.el9.noarch 2024-03-27T10:41:47.382 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-radosgw-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T10:41:47.382 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-selinux-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T10:41:47.382 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-test-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T10:41:47.382 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-volume-2:19.0.0-2478.g155268c4.el9.noarch 2024-03-27T10:41:47.382 INFO:teuthology.orchestra.run.smithi097.stdout: cephadm-2:19.0.0-2478.g155268c4.el9.noarch 2024-03-27T10:41:47.382 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-mirror-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T10:41:47.382 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-shell-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T10:41:47.382 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-top-2:19.0.0-2478.g155268c4.el9.noarch 2024-03-27T10:41:47.382 INFO:teuthology.orchestra.run.smithi097.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-27T10:41:47.382 INFO:teuthology.orchestra.run.smithi097.stdout: flex-2.6.4-9.el9.x86_64 2024-03-27T10:41:47.382 INFO:teuthology.orchestra.run.smithi097.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-27T10:41:47.382 INFO:teuthology.orchestra.run.smithi097.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-27T10:41:47.383 INFO:teuthology.orchestra.run.smithi097.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-27T10:41:47.383 INFO:teuthology.orchestra.run.smithi097.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-27T10:41:47.383 INFO:teuthology.orchestra.run.smithi097.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-27T10:41:47.383 INFO:teuthology.orchestra.run.smithi097.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-27T10:41:47.383 INFO:teuthology.orchestra.run.smithi097.stdout: jq-1.6-16.el9.x86_64 2024-03-27T10:41:47.383 INFO:teuthology.orchestra.run.smithi097.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-27T10:41:47.383 INFO:teuthology.orchestra.run.smithi097.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-27T10:41:47.383 INFO:teuthology.orchestra.run.smithi097.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-27T10:41:47.383 INFO:teuthology.orchestra.run.smithi097.stdout: libcephfs-devel-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T10:41:47.383 INFO:teuthology.orchestra.run.smithi097.stdout: libcephfs2-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T10:41:47.383 INFO:teuthology.orchestra.run.smithi097.stdout: libcephsqlite-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T10:41:47.383 INFO:teuthology.orchestra.run.smithi097.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-27T10:41:47.383 INFO:teuthology.orchestra.run.smithi097.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-27T10:41:47.383 INFO:teuthology.orchestra.run.smithi097.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-27T10:41:47.383 INFO:teuthology.orchestra.run.smithi097.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-27T10:41:47.384 INFO:teuthology.orchestra.run.smithi097.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-27T10:41:47.384 INFO:teuthology.orchestra.run.smithi097.stdout: librados-devel-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T10:41:47.384 INFO:teuthology.orchestra.run.smithi097.stdout: libradosstriper1-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T10:41:47.384 INFO:teuthology.orchestra.run.smithi097.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-27T10:41:47.384 INFO:teuthology.orchestra.run.smithi097.stdout: librgw2-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T10:41:47.384 INFO:teuthology.orchestra.run.smithi097.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-27T10:41:47.384 INFO:teuthology.orchestra.run.smithi097.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-27T10:41:47.384 INFO:teuthology.orchestra.run.smithi097.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-27T10:41:47.384 INFO:teuthology.orchestra.run.smithi097.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-27T10:41:47.384 INFO:teuthology.orchestra.run.smithi097.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-27T10:41:47.384 INFO:teuthology.orchestra.run.smithi097.stdout: lua-5.4.4-4.el9.x86_64 2024-03-27T10:41:47.384 INFO:teuthology.orchestra.run.smithi097.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-27T10:41:47.384 INFO:teuthology.orchestra.run.smithi097.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-27T10:41:47.384 INFO:teuthology.orchestra.run.smithi097.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-27T10:41:47.385 INFO:teuthology.orchestra.run.smithi097.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-27T10:41:47.385 INFO:teuthology.orchestra.run.smithi097.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-27T10:41:47.385 INFO:teuthology.orchestra.run.smithi097.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-27T10:41:47.385 INFO:teuthology.orchestra.run.smithi097.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-27T10:41:47.385 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-13.14-1.el9.x86_64 2024-03-27T10:41:47.385 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-03-27T10:41:47.385 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-03-27T10:41:47.385 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-server-13.14-1.el9.x86_64 2024-03-27T10:41:47.385 INFO:teuthology.orchestra.run.smithi097.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-27T10:41:47.385 INFO:teuthology.orchestra.run.smithi097.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-27T10:41:47.385 INFO:teuthology.orchestra.run.smithi097.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-27T10:41:47.385 INFO:teuthology.orchestra.run.smithi097.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-27T10:41:47.385 INFO:teuthology.orchestra.run.smithi097.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-27T10:41:47.386 INFO:teuthology.orchestra.run.smithi097.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-27T10:41:47.386 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-27T10:41:47.386 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ceph-argparse-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T10:41:47.386 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ceph-common-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T10:41:47.386 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cephfs-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T10:41:47.386 INFO:teuthology.orchestra.run.smithi097.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-27T10:41:47.386 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-27T10:41:47.386 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-27T10:41:47.386 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-27T10:41:47.386 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-27T10:41:47.386 INFO:teuthology.orchestra.run.smithi097.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-27T10:41:47.386 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-27T10:41:47.386 INFO:teuthology.orchestra.run.smithi097.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-27T10:41:47.386 INFO:teuthology.orchestra.run.smithi097.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-27T10:41:47.387 INFO:teuthology.orchestra.run.smithi097.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-27T10:41:47.387 INFO:teuthology.orchestra.run.smithi097.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-27T10:41:47.387 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-27T10:41:47.387 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-27T10:41:47.387 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-27T10:41:47.387 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-27T10:41:47.387 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-27T10:41:47.387 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-27T10:41:47.387 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-27T10:41:47.387 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-27T10:41:47.387 INFO:teuthology.orchestra.run.smithi097.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-27T10:41:47.387 INFO:teuthology.orchestra.run.smithi097.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-27T10:41:47.387 INFO:teuthology.orchestra.run.smithi097.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-27T10:41:47.387 INFO:teuthology.orchestra.run.smithi097.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-27T10:41:47.388 INFO:teuthology.orchestra.run.smithi097.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-27T10:41:47.388 INFO:teuthology.orchestra.run.smithi097.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-27T10:41:47.388 INFO:teuthology.orchestra.run.smithi097.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-27T10:41:47.388 INFO:teuthology.orchestra.run.smithi097.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-27T10:41:47.388 INFO:teuthology.orchestra.run.smithi097.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-27T10:41:47.388 INFO:teuthology.orchestra.run.smithi097.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-27T10:41:47.388 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-27T10:41:47.388 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ply-3.11-14.el9.noarch 2024-03-27T10:41:47.388 INFO:teuthology.orchestra.run.smithi097.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-27T10:41:47.388 INFO:teuthology.orchestra.run.smithi097.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-27T10:41:47.388 INFO:teuthology.orchestra.run.smithi097.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-27T10:41:47.388 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-27T10:41:47.388 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-27T10:41:47.388 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-27T10:41:47.389 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-27T10:41:47.389 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-27T10:41:47.389 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-27T10:41:47.389 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rados-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T10:41:47.389 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rbd-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T10:41:47.389 INFO:teuthology.orchestra.run.smithi097.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-27T10:41:47.389 INFO:teuthology.orchestra.run.smithi097.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-27T10:41:47.389 INFO:teuthology.orchestra.run.smithi097.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-27T10:41:47.389 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rgw-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T10:41:47.389 INFO:teuthology.orchestra.run.smithi097.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-27T10:41:47.389 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-27T10:41:47.389 INFO:teuthology.orchestra.run.smithi097.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-27T10:41:47.389 INFO:teuthology.orchestra.run.smithi097.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-27T10:41:47.389 INFO:teuthology.orchestra.run.smithi097.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-27T10:41:47.390 INFO:teuthology.orchestra.run.smithi097.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-27T10:41:47.390 INFO:teuthology.orchestra.run.smithi097.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-27T10:41:47.390 INFO:teuthology.orchestra.run.smithi097.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-27T10:41:47.390 INFO:teuthology.orchestra.run.smithi097.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-27T10:41:47.390 INFO:teuthology.orchestra.run.smithi097.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-27T10:41:47.390 INFO:teuthology.orchestra.run.smithi097.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-27T10:41:47.390 INFO:teuthology.orchestra.run.smithi097.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-27T10:41:47.390 INFO:teuthology.orchestra.run.smithi097.stdout: qatlib-24.02.0-1.el9.x86_64 2024-03-27T10:41:47.390 INFO:teuthology.orchestra.run.smithi097.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-03-27T10:41:47.390 INFO:teuthology.orchestra.run.smithi097.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-03-27T10:41:47.390 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-fuse-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T10:41:47.390 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-mirror-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T10:41:47.390 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-nbd-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T10:41:47.390 INFO:teuthology.orchestra.run.smithi097.stdout: re2-1:20211101-3.el9.x86_64 2024-03-27T10:41:47.391 INFO:teuthology.orchestra.run.smithi097.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-27T10:41:47.391 INFO:teuthology.orchestra.run.smithi097.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-27T10:41:47.391 INFO:teuthology.orchestra.run.smithi097.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-27T10:41:47.391 INFO:teuthology.orchestra.run.smithi097.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-27T10:41:47.391 INFO:teuthology.orchestra.run.smithi097.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-27T10:41:47.391 INFO:teuthology.orchestra.run.smithi097.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-27T10:41:47.391 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T10:41:47.391 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2024-03-27T10:41:47.754 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-mon-2:19.0.0-2478.g155268c4.el9.x86_64 136/198 2024-03-27T10:41:47.826 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-mon-2:19.0.0-2478.g155268c4.el9.x86_64 136/198 2024-03-27T10:41:47.826 INFO:teuthology.orchestra.run.smithi114.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-27T10:41:47.826 INFO:teuthology.orchestra.run.smithi114.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-27T10:41:47.826 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-27T10:41:47.826 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-27T10:41:47.826 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-03-27T10:41:47.939 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-27T10:41:48.120 DEBUG:teuthology.parallel:result is None 2024-03-27T10:41:48.584 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: cephadm-2:19.0.0-2478.g155268c4.el9.noarch 138/198 2024-03-27T10:41:48.624 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : cephadm-2:19.0.0-2478.g155268c4.el9.noarch 138/198 2024-03-27T10:41:48.674 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2478.g155268c4.e 139/198 2024-03-27T10:41:48.775 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2478.g155268c4. 140/198 2024-03-27T10:41:48.806 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2478.g155268c4.el9.noa 141/198 2024-03-27T10:41:50.079 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2478.g155268c4.el9.noa 141/198 2024-03-27T10:41:50.121 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2478.g155268c4.el9.n 142/198 2024-03-27T10:41:50.216 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-2:19.0.0-2478.g155268c4.el9.x86_64 1/198 2024-03-27T10:41:50.216 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-base-2:19.0.0-2478.g155268c4.el9.x86_64 2/198 2024-03-27T10:41:50.217 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-common-2:19.0.0-2478.g155268c4.el9.x86_64 3/198 2024-03-27T10:41:50.217 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-fuse-2:19.0.0-2478.g155268c4.el9.x86_64 4/198 2024-03-27T10:41:50.217 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2478.g15526 5/198 2024-03-27T10:41:50.217 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mds-2:19.0.0-2478.g155268c4.el9.x86_64 6/198 2024-03-27T10:41:50.217 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mgr-2:19.0.0-2478.g155268c4.el9.x86_64 7/198 2024-03-27T10:41:50.217 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mon-2:19.0.0-2478.g155268c4.el9.x86_64 8/198 2024-03-27T10:41:50.217 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-osd-2:19.0.0-2478.g155268c4.el9.x86_64 9/198 2024-03-27T10:41:50.217 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-radosgw-2:19.0.0-2478.g155268c4.el9.x86_64 10/198 2024-03-27T10:41:50.217 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-selinux-2:19.0.0-2478.g155268c4.el9.x86_64 11/198 2024-03-27T10:41:50.217 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-test-2:19.0.0-2478.g155268c4.el9.x86_64 12/198 2024-03-27T10:41:50.217 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : cephfs-mirror-2:19.0.0-2478.g155268c4.el9.x86_64 13/198 2024-03-27T10:41:50.217 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : cephfs-shell-2:19.0.0-2478.g155268c4.el9.x86_64 14/198 2024-03-27T10:41:50.217 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libcephfs-devel-2:19.0.0-2478.g155268c4.el9.x86_ 15/198 2024-03-27T10:41:50.217 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libcephfs2-2:19.0.0-2478.g155268c4.el9.x86_64 16/198 2024-03-27T10:41:50.217 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libcephsqlite-2:19.0.0-2478.g155268c4.el9.x86_64 17/198 2024-03-27T10:41:50.218 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librados-devel-2:19.0.0-2478.g155268c4.el9.x86_6 18/198 2024-03-27T10:41:50.218 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libradosstriper1-2:19.0.0-2478.g155268c4.el9.x86 19/198 2024-03-27T10:41:50.218 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librgw2-2:19.0.0-2478.g155268c4.el9.x86_64 20/198 2024-03-27T10:41:50.218 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2478.g155268c4.el 21/198 2024-03-27T10:41:50.218 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-ceph-common-2:19.0.0-2478.g155268c4.el9. 22/198 2024-03-27T10:41:50.218 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-cephfs-2:19.0.0-2478.g155268c4.el9.x86_6 23/198 2024-03-27T10:41:50.218 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-rados-2:19.0.0-2478.g155268c4.el9.x86_64 24/198 2024-03-27T10:41:50.218 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-rbd-2:19.0.0-2478.g155268c4.el9.x86_64 25/198 2024-03-27T10:41:50.236 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-rgw-2:19.0.0-2478.g155268c4.el9.x86_64 26/198 2024-03-27T10:41:50.236 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : rbd-fuse-2:19.0.0-2478.g155268c4.el9.x86_64 27/198 2024-03-27T10:41:50.237 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : rbd-mirror-2:19.0.0-2478.g155268c4.el9.x86_64 28/198 2024-03-27T10:41:50.237 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : rbd-nbd-2:19.0.0-2478.g155268c4.el9.x86_64 29/198 2024-03-27T10:41:50.237 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2478.g155268c4. 30/198 2024-03-27T10:41:50.237 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2478.g155268c4.el9.noa 31/198 2024-03-27T10:41:50.237 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2478.g155268c4.el9.n 32/198 2024-03-27T10:41:50.237 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2478.g155 33/198 2024-03-27T10:41:50.237 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2478.g155268c4.el 34/198 2024-03-27T10:41:50.237 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2478.g155268c4.el9.noarch 35/198 2024-03-27T10:41:50.237 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2478.g155268c4.e 36/198 2024-03-27T10:41:50.237 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-volume-2:19.0.0-2478.g155268c4.el9.noarch 37/198 2024-03-27T10:41:50.237 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : cephadm-2:19.0.0-2478.g155268c4.el9.noarch 38/198 2024-03-27T10:41:50.237 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : cephfs-top-2:19.0.0-2478.g155268c4.el9.noarch 39/198 2024-03-27T10:41:50.237 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-27T10:41:50.238 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-27T10:41:50.238 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-27T10:41:50.238 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-27T10:41:50.238 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-27T10:41:50.238 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-27T10:41:50.238 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-27T10:41:50.238 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-27T10:41:50.238 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-27T10:41:50.238 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-27T10:41:50.238 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-27T10:41:50.238 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-27T10:41:50.238 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-27T10:41:50.238 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-27T10:41:50.239 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-27T10:41:50.239 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-27T10:41:50.239 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-27T10:41:50.239 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-27T10:41:50.239 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 58/198 2024-03-27T10:41:50.239 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 59/198 2024-03-27T10:41:50.239 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 60/198 2024-03-27T10:41:50.239 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 61/198 2024-03-27T10:41:50.239 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 62/198 2024-03-27T10:41:50.239 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 63/198 2024-03-27T10:41:50.239 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 64/198 2024-03-27T10:41:50.239 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 65/198 2024-03-27T10:41:50.239 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 66/198 2024-03-27T10:41:50.239 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 67/198 2024-03-27T10:41:50.240 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 68/198 2024-03-27T10:41:50.240 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 69/198 2024-03-27T10:41:50.240 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 70/198 2024-03-27T10:41:50.240 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 71/198 2024-03-27T10:41:50.240 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 72/198 2024-03-27T10:41:50.240 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 73/198 2024-03-27T10:41:50.240 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 74/198 2024-03-27T10:41:50.240 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 75/198 2024-03-27T10:41:50.240 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 76/198 2024-03-27T10:41:50.240 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 77/198 2024-03-27T10:41:50.240 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 78/198 2024-03-27T10:41:50.240 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 79/198 2024-03-27T10:41:50.240 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 80/198 2024-03-27T10:41:50.240 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 81/198 2024-03-27T10:41:50.241 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 82/198 2024-03-27T10:41:50.241 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 83/198 2024-03-27T10:41:50.241 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 84/198 2024-03-27T10:41:50.241 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-ply-3.11-14.el9.noarch 85/198 2024-03-27T10:41:50.241 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 86/198 2024-03-27T10:41:50.241 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 87/198 2024-03-27T10:41:50.241 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : jq-1.6-16.el9.x86_64 88/198 2024-03-27T10:41:50.241 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 89/198 2024-03-27T10:41:50.241 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 90/198 2024-03-27T10:41:50.241 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 91/198 2024-03-27T10:41:50.241 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 92/198 2024-03-27T10:41:50.241 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 93/198 2024-03-27T10:41:50.241 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 94/198 2024-03-27T10:41:50.242 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : postgresql-13.14-1.el9.x86_64 95/198 2024-03-27T10:41:50.242 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 96/198 2024-03-27T10:41:50.242 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 97/198 2024-03-27T10:41:50.242 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 98/198 2024-03-27T10:41:50.242 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-27T10:41:50.242 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-27T10:41:50.242 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 101/198 2024-03-27T10:41:50.242 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 102/198 2024-03-27T10:41:50.242 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 103/198 2024-03-27T10:41:50.242 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-27T10:41:50.242 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-27T10:41:50.242 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-27T10:41:50.242 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-27T10:41:50.243 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-27T10:41:50.243 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-27T10:41:50.243 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-27T10:41:50.243 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-27T10:41:50.243 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-27T10:41:50.243 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-27T10:41:50.243 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-27T10:41:50.243 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-27T10:41:50.243 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-03-27T10:41:50.243 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-27T10:41:50.243 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-27T10:41:50.243 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-27T10:41:50.243 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-27T10:41:50.243 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-27T10:41:50.244 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-27T10:41:50.244 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-27T10:41:50.244 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-03-27T10:41:50.244 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-27T10:41:50.244 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-27T10:41:50.244 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-27T10:41:50.244 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-27T10:41:50.244 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-27T10:41:50.244 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-27T10:41:50.244 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-27T10:41:50.244 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-27T10:41:50.244 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-27T10:41:50.244 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-27T10:41:50.244 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-27T10:41:50.245 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-27T10:41:50.245 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-27T10:41:50.245 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-27T10:41:50.245 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-27T10:41:50.245 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-27T10:41:50.245 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-27T10:41:50.245 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-27T10:41:50.245 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-27T10:41:50.245 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-27T10:41:50.245 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-27T10:41:50.245 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-27T10:41:50.245 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-27T10:41:50.245 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-27T10:41:50.246 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-27T10:41:50.246 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-27T10:41:50.246 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librados2-2:19.0.0-2478.g155268c4.el9.x86_64 151/198 2024-03-27T10:41:50.246 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-27T10:41:50.246 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librbd1-2:19.0.0-2478.g155268c4.el9.x86_64 153/198 2024-03-27T10:41:50.246 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-27T10:41:50.246 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-27T10:41:50.246 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-27T10:41:50.246 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 157/198 2024-03-27T10:41:50.246 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-27T10:41:50.246 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 159/198 2024-03-27T10:41:50.246 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-27T10:41:50.246 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 161/198 2024-03-27T10:41:50.247 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-27T10:41:50.247 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 163/198 2024-03-27T10:41:50.247 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-27T10:41:50.247 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-27T10:41:50.247 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-27T10:41:50.247 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-27T10:41:50.247 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-27T10:41:50.247 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-27T10:41:50.247 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-27T10:41:50.247 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-27T10:41:50.247 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-27T10:41:50.247 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-27T10:41:50.247 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-27T10:41:50.247 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-27T10:41:50.248 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-27T10:41:50.248 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-27T10:41:50.248 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-27T10:41:50.248 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-27T10:41:50.248 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-27T10:41:50.248 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-27T10:41:50.248 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-27T10:41:50.248 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-27T10:41:50.248 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-27T10:41:50.248 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-27T10:41:50.248 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-27T10:41:50.248 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-27T10:41:50.248 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-27T10:41:50.248 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-27T10:41:50.249 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-27T10:41:50.249 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-27T10:41:50.249 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-27T10:41:50.249 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-27T10:41:50.249 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-27T10:41:50.249 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-27T10:41:50.249 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-27T10:41:50.249 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-27T10:41:51.210 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2478.g155268c4.el9.n 142/198 2024-03-27T10:41:51.237 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2478.g155 143/198 2024-03-27T10:41:51.381 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2478.g155 143/198 2024-03-27T10:41:51.577 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2478.g155268c4.el 144/198 2024-03-27T10:41:51.623 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-mgr-2:19.0.0-2478.g155268c4.el9.x86_64 145/198 2024-03-27T10:41:51.695 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2478.g155268c4.el9.x86_64 145/198 2024-03-27T10:41:51.695 INFO:teuthology.orchestra.run.smithi114.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-27T10:41:51.695 INFO:teuthology.orchestra.run.smithi114.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-27T10:41:51.695 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-27T10:41:51.695 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-27T10:41:51.695 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-03-27T10:41:51.749 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-mgr-rook-2:19.0.0-2478.g155268c4.el9.noarch 146/198 2024-03-27T10:41:51.786 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2478.g155268c4.el9.noarch 146/198 2024-03-27T10:41:51.886 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-2:19.0.0-2478.g155268c4.el9.x86_64 147/198 2024-03-27T10:41:52.735 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-27T10:41:52.735 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T10:41:52.735 INFO:teuthology.orchestra.run.smithi177.stdout:Upgraded: 2024-03-27T10:41:52.735 INFO:teuthology.orchestra.run.smithi177.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-03-27T10:41:52.735 INFO:teuthology.orchestra.run.smithi177.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-03-27T10:41:52.735 INFO:teuthology.orchestra.run.smithi177.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-03-27T10:41:52.735 INFO:teuthology.orchestra.run.smithi177.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-03-27T10:41:52.735 INFO:teuthology.orchestra.run.smithi177.stdout: acl-2.3.1-4.el9.x86_64 2024-03-27T10:41:52.735 INFO:teuthology.orchestra.run.smithi177.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-27T10:41:52.735 INFO:teuthology.orchestra.run.smithi177.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-27T10:41:52.736 INFO:teuthology.orchestra.run.smithi177.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-27T10:41:52.736 INFO:teuthology.orchestra.run.smithi177.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-27T10:41:52.736 INFO:teuthology.orchestra.run.smithi177.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-27T10:41:52.736 INFO:teuthology.orchestra.run.smithi177.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-27T10:41:52.736 INFO:teuthology.orchestra.run.smithi177.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-27T10:41:52.736 INFO:teuthology.orchestra.run.smithi177.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-27T10:41:52.736 INFO:teuthology.orchestra.run.smithi177.stdout: librados2-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T10:41:52.736 INFO:teuthology.orchestra.run.smithi177.stdout: librbd1-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T10:41:52.736 INFO:teuthology.orchestra.run.smithi177.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-27T10:41:52.736 INFO:teuthology.orchestra.run.smithi177.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-27T10:41:52.736 INFO:teuthology.orchestra.run.smithi177.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-27T10:41:52.736 INFO:teuthology.orchestra.run.smithi177.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-27T10:41:52.736 INFO:teuthology.orchestra.run.smithi177.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-27T10:41:52.736 INFO:teuthology.orchestra.run.smithi177.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-27T10:41:52.737 INFO:teuthology.orchestra.run.smithi177.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-27T10:41:52.737 INFO:teuthology.orchestra.run.smithi177.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-27T10:41:52.737 INFO:teuthology.orchestra.run.smithi177.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-27T10:41:52.737 INFO:teuthology.orchestra.run.smithi177.stdout:Installed: 2024-03-27T10:41:52.737 INFO:teuthology.orchestra.run.smithi177.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-27T10:41:52.737 INFO:teuthology.orchestra.run.smithi177.stdout: automake-1.16.2-8.el9.noarch 2024-03-27T10:41:52.737 INFO:teuthology.orchestra.run.smithi177.stdout: bison-3.7.4-5.el9.x86_64 2024-03-27T10:41:52.737 INFO:teuthology.orchestra.run.smithi177.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-27T10:41:52.737 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T10:41:52.737 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-base-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T10:41:52.737 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-common-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T10:41:52.737 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-fuse-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T10:41:52.737 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-grafana-dashboards-2:19.0.0-2478.g155268c4.el9.noarch 2024-03-27T10:41:52.737 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-immutable-object-cache-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T10:41:52.737 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mds-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T10:41:52.738 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T10:41:52.738 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-cephadm-2:19.0.0-2478.g155268c4.el9.noarch 2024-03-27T10:41:52.738 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-dashboard-2:19.0.0-2478.g155268c4.el9.noarch 2024-03-27T10:41:52.738 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2478.g155268c4.el9.noarch 2024-03-27T10:41:52.738 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-modules-core-2:19.0.0-2478.g155268c4.el9.noarch 2024-03-27T10:41:52.738 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-rook-2:19.0.0-2478.g155268c4.el9.noarch 2024-03-27T10:41:52.738 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mon-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T10:41:52.738 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-osd-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T10:41:52.738 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-prometheus-alerts-2:19.0.0-2478.g155268c4.el9.noarch 2024-03-27T10:41:52.738 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-radosgw-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T10:41:52.738 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-selinux-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T10:41:52.738 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-test-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T10:41:52.739 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-volume-2:19.0.0-2478.g155268c4.el9.noarch 2024-03-27T10:41:52.739 INFO:teuthology.orchestra.run.smithi177.stdout: cephadm-2:19.0.0-2478.g155268c4.el9.noarch 2024-03-27T10:41:52.739 INFO:teuthology.orchestra.run.smithi177.stdout: cephfs-mirror-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T10:41:52.739 INFO:teuthology.orchestra.run.smithi177.stdout: cephfs-shell-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T10:41:52.739 INFO:teuthology.orchestra.run.smithi177.stdout: cephfs-top-2:19.0.0-2478.g155268c4.el9.noarch 2024-03-27T10:41:52.739 INFO:teuthology.orchestra.run.smithi177.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-27T10:41:52.739 INFO:teuthology.orchestra.run.smithi177.stdout: flex-2.6.4-9.el9.x86_64 2024-03-27T10:41:52.739 INFO:teuthology.orchestra.run.smithi177.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-27T10:41:52.739 INFO:teuthology.orchestra.run.smithi177.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-27T10:41:52.739 INFO:teuthology.orchestra.run.smithi177.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-27T10:41:52.739 INFO:teuthology.orchestra.run.smithi177.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-27T10:41:52.739 INFO:teuthology.orchestra.run.smithi177.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-27T10:41:52.739 INFO:teuthology.orchestra.run.smithi177.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-27T10:41:52.740 INFO:teuthology.orchestra.run.smithi177.stdout: jq-1.6-16.el9.x86_64 2024-03-27T10:41:52.740 INFO:teuthology.orchestra.run.smithi177.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-27T10:41:52.740 INFO:teuthology.orchestra.run.smithi177.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-27T10:41:52.740 INFO:teuthology.orchestra.run.smithi177.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-27T10:41:52.740 INFO:teuthology.orchestra.run.smithi177.stdout: libcephfs-devel-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T10:41:52.740 INFO:teuthology.orchestra.run.smithi177.stdout: libcephfs2-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T10:41:52.740 INFO:teuthology.orchestra.run.smithi177.stdout: libcephsqlite-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T10:41:52.740 INFO:teuthology.orchestra.run.smithi177.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-27T10:41:52.740 INFO:teuthology.orchestra.run.smithi177.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-27T10:41:52.740 INFO:teuthology.orchestra.run.smithi177.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-27T10:41:52.740 INFO:teuthology.orchestra.run.smithi177.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-27T10:41:52.740 INFO:teuthology.orchestra.run.smithi177.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-27T10:41:52.740 INFO:teuthology.orchestra.run.smithi177.stdout: librados-devel-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T10:41:52.741 INFO:teuthology.orchestra.run.smithi177.stdout: libradosstriper1-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T10:41:52.741 INFO:teuthology.orchestra.run.smithi177.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-27T10:41:52.741 INFO:teuthology.orchestra.run.smithi177.stdout: librgw2-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T10:41:52.741 INFO:teuthology.orchestra.run.smithi177.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-27T10:41:52.741 INFO:teuthology.orchestra.run.smithi177.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-27T10:41:52.741 INFO:teuthology.orchestra.run.smithi177.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-27T10:41:52.741 INFO:teuthology.orchestra.run.smithi177.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-27T10:41:52.741 INFO:teuthology.orchestra.run.smithi177.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-27T10:41:52.741 INFO:teuthology.orchestra.run.smithi177.stdout: lua-5.4.4-4.el9.x86_64 2024-03-27T10:41:52.741 INFO:teuthology.orchestra.run.smithi177.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-27T10:41:52.741 INFO:teuthology.orchestra.run.smithi177.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-27T10:41:52.741 INFO:teuthology.orchestra.run.smithi177.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-27T10:41:52.741 INFO:teuthology.orchestra.run.smithi177.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-27T10:41:52.742 INFO:teuthology.orchestra.run.smithi177.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-27T10:41:52.742 INFO:teuthology.orchestra.run.smithi177.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-27T10:41:52.742 INFO:teuthology.orchestra.run.smithi177.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-27T10:41:52.742 INFO:teuthology.orchestra.run.smithi177.stdout: postgresql-13.14-1.el9.x86_64 2024-03-27T10:41:52.742 INFO:teuthology.orchestra.run.smithi177.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-03-27T10:41:52.742 INFO:teuthology.orchestra.run.smithi177.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-03-27T10:41:52.742 INFO:teuthology.orchestra.run.smithi177.stdout: postgresql-server-13.14-1.el9.x86_64 2024-03-27T10:41:52.742 INFO:teuthology.orchestra.run.smithi177.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-27T10:41:52.742 INFO:teuthology.orchestra.run.smithi177.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-27T10:41:52.742 INFO:teuthology.orchestra.run.smithi177.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-27T10:41:52.742 INFO:teuthology.orchestra.run.smithi177.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-27T10:41:52.742 INFO:teuthology.orchestra.run.smithi177.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-27T10:41:52.743 INFO:teuthology.orchestra.run.smithi177.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-27T10:41:52.743 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-27T10:41:52.743 INFO:teuthology.orchestra.run.smithi177.stdout: python3-ceph-argparse-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T10:41:52.743 INFO:teuthology.orchestra.run.smithi177.stdout: python3-ceph-common-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T10:41:52.743 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cephfs-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T10:41:52.743 INFO:teuthology.orchestra.run.smithi177.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-27T10:41:52.743 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-27T10:41:52.743 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-27T10:41:52.743 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-27T10:41:52.743 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-27T10:41:52.743 INFO:teuthology.orchestra.run.smithi177.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-27T10:41:52.743 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-27T10:41:52.743 INFO:teuthology.orchestra.run.smithi177.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-27T10:41:52.743 INFO:teuthology.orchestra.run.smithi177.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-27T10:41:52.744 INFO:teuthology.orchestra.run.smithi177.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-27T10:41:52.744 INFO:teuthology.orchestra.run.smithi177.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-27T10:41:52.744 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-27T10:41:52.744 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-27T10:41:52.744 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-27T10:41:52.744 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-27T10:41:52.744 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-27T10:41:52.744 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-27T10:41:52.744 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-27T10:41:52.744 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-27T10:41:52.744 INFO:teuthology.orchestra.run.smithi177.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-27T10:41:52.744 INFO:teuthology.orchestra.run.smithi177.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-27T10:41:52.744 INFO:teuthology.orchestra.run.smithi177.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-27T10:41:52.745 INFO:teuthology.orchestra.run.smithi177.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-27T10:41:52.745 INFO:teuthology.orchestra.run.smithi177.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-27T10:41:52.745 INFO:teuthology.orchestra.run.smithi177.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-27T10:41:52.745 INFO:teuthology.orchestra.run.smithi177.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-27T10:41:52.745 INFO:teuthology.orchestra.run.smithi177.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-27T10:41:52.745 INFO:teuthology.orchestra.run.smithi177.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-27T10:41:52.745 INFO:teuthology.orchestra.run.smithi177.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-27T10:41:52.745 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-27T10:41:52.745 INFO:teuthology.orchestra.run.smithi177.stdout: python3-ply-3.11-14.el9.noarch 2024-03-27T10:41:52.745 INFO:teuthology.orchestra.run.smithi177.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-27T10:41:52.745 INFO:teuthology.orchestra.run.smithi177.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-27T10:41:52.745 INFO:teuthology.orchestra.run.smithi177.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-27T10:41:52.745 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-27T10:41:52.746 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-27T10:41:52.746 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-27T10:41:52.746 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-27T10:41:52.746 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-27T10:41:52.746 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-27T10:41:52.746 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rados-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T10:41:52.746 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rbd-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T10:41:52.746 INFO:teuthology.orchestra.run.smithi177.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-27T10:41:52.746 INFO:teuthology.orchestra.run.smithi177.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-27T10:41:52.746 INFO:teuthology.orchestra.run.smithi177.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-27T10:41:52.746 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rgw-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T10:41:52.746 INFO:teuthology.orchestra.run.smithi177.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-27T10:41:52.746 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-27T10:41:52.747 INFO:teuthology.orchestra.run.smithi177.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-27T10:41:52.747 INFO:teuthology.orchestra.run.smithi177.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-27T10:41:52.747 INFO:teuthology.orchestra.run.smithi177.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-27T10:41:52.747 INFO:teuthology.orchestra.run.smithi177.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-27T10:41:52.747 INFO:teuthology.orchestra.run.smithi177.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-27T10:41:52.747 INFO:teuthology.orchestra.run.smithi177.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-27T10:41:52.747 INFO:teuthology.orchestra.run.smithi177.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-27T10:41:52.747 INFO:teuthology.orchestra.run.smithi177.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-27T10:41:52.747 INFO:teuthology.orchestra.run.smithi177.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-27T10:41:52.747 INFO:teuthology.orchestra.run.smithi177.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-27T10:41:52.747 INFO:teuthology.orchestra.run.smithi177.stdout: qatlib-24.02.0-1.el9.x86_64 2024-03-27T10:41:52.747 INFO:teuthology.orchestra.run.smithi177.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-03-27T10:41:52.747 INFO:teuthology.orchestra.run.smithi177.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-03-27T10:41:52.747 INFO:teuthology.orchestra.run.smithi177.stdout: rbd-fuse-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T10:41:52.748 INFO:teuthology.orchestra.run.smithi177.stdout: rbd-mirror-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T10:41:52.748 INFO:teuthology.orchestra.run.smithi177.stdout: rbd-nbd-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T10:41:52.748 INFO:teuthology.orchestra.run.smithi177.stdout: re2-1:20211101-3.el9.x86_64 2024-03-27T10:41:52.748 INFO:teuthology.orchestra.run.smithi177.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-27T10:41:52.748 INFO:teuthology.orchestra.run.smithi177.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-27T10:41:52.748 INFO:teuthology.orchestra.run.smithi177.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-27T10:41:52.748 INFO:teuthology.orchestra.run.smithi177.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-27T10:41:52.748 INFO:teuthology.orchestra.run.smithi177.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-27T10:41:52.748 INFO:teuthology.orchestra.run.smithi177.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-27T10:41:52.748 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T10:41:52.749 INFO:teuthology.orchestra.run.smithi177.stdout:Complete! 2024-03-27T10:41:52.887 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-27T10:41:52.934 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-radosgw-2:19.0.0-2478.g155268c4.el9.x86_64 149/198 2024-03-27T10:41:53.027 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2478.g155268c4.el9.x86_64 149/198 2024-03-27T10:41:53.027 INFO:teuthology.orchestra.run.smithi114.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-27T10:41:53.027 INFO:teuthology.orchestra.run.smithi114.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-27T10:41:53.027 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-27T10:41:53.028 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-27T10:41:53.028 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-03-27T10:41:53.076 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2478.g15526 150/198 2024-03-27T10:41:53.168 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2478.g15526 150/198 2024-03-27T10:41:53.169 INFO:teuthology.orchestra.run.smithi114.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-27T10:41:53.169 INFO:teuthology.orchestra.run.smithi114.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-27T10:41:53.169 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-03-27T10:41:53.252 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : cephfs-mirror-2:19.0.0-2478.g155268c4.el9.x86_64 151/198 2024-03-27T10:41:53.353 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2478.g155268c4.el9.x86_64 151/198 2024-03-27T10:41:53.353 INFO:teuthology.orchestra.run.smithi114.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-27T10:41:53.353 INFO:teuthology.orchestra.run.smithi114.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-27T10:41:53.354 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-27T10:41:53.354 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-27T10:41:53.354 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-03-27T10:41:53.464 DEBUG:teuthology.parallel:result is None 2024-03-27T10:41:53.728 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : rbd-mirror-2:19.0.0-2478.g155268c4.el9.x86_64 152/198 2024-03-27T10:41:53.800 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2478.g155268c4.el9.x86_64 152/198 2024-03-27T10:41:53.800 INFO:teuthology.orchestra.run.smithi114.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-27T10:41:53.801 INFO:teuthology.orchestra.run.smithi114.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-27T10:41:53.801 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-27T10:41:53.801 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-27T10:41:53.801 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-03-27T10:41:58.601 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-test-2:19.0.0-2478.g155268c4.el9.x86_64 153/198 2024-03-27T10:41:58.645 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : rbd-fuse-2:19.0.0-2478.g155268c4.el9.x86_64 154/198 2024-03-27T10:41:58.681 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : rbd-nbd-2:19.0.0-2478.g155268c4.el9.x86_64 155/198 2024-03-27T10:41:58.719 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-27T10:41:58.989 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : cephfs-shell-2:19.0.0-2478.g155268c4.el9.x86_64 157/198 2024-03-27T10:41:59.119 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 158/198 2024-03-27T10:41:59.205 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-fuse-2:19.0.0-2478.g155268c4.el9.x86_64 159/198 2024-03-27T10:41:59.267 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-27T10:41:59.325 INFO:teuthology.orchestra.run.smithi114.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-27T10:41:59.380 INFO:teuthology.orchestra.run.smithi114.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-27T10:41:59.422 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libcephfs-devel-2:19.0.0-2478.g155268c4.el9.x86_ 163/198 2024-03-27T10:41:59.467 INFO:teuthology.orchestra.run.smithi114.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-27T10:41:59.517 INFO:teuthology.orchestra.run.smithi114.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-03-27T10:41:59.633 INFO:teuthology.orchestra.run.smithi114.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-03-27T10:41:59.750 INFO:teuthology.orchestra.run.smithi114.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-27T10:41:59.791 INFO:teuthology.orchestra.run.smithi114.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-27T10:42:00.777 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-27T10:42:00.822 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-27T10:42:00.887 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-27T10:42:02.201 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : cephfs-top-2:19.0.0-2478.g155268c4.el9.noarch 170/198 2024-03-27T10:42:02.285 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-27T10:42:02.341 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-27T10:42:02.466 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-27T10:42:02.529 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-27T10:42:02.562 INFO:teuthology.orchestra.run.smithi114.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-27T10:42:02.596 INFO:teuthology.orchestra.run.smithi114.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-27T10:42:02.638 INFO:teuthology.orchestra.run.smithi114.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-27T10:42:02.638 INFO:teuthology.orchestra.run.smithi114.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-27T10:42:02.783 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-27T10:42:02.834 INFO:teuthology.orchestra.run.smithi114.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-27T10:42:02.834 INFO:teuthology.orchestra.run.smithi114.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-27T10:42:02.980 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-27T10:42:03.025 INFO:teuthology.orchestra.run.smithi114.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-27T10:42:03.071 INFO:teuthology.orchestra.run.smithi114.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-27T10:42:03.104 INFO:teuthology.orchestra.run.smithi114.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-27T10:42:03.136 INFO:teuthology.orchestra.run.smithi114.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-27T10:42:03.205 INFO:teuthology.orchestra.run.smithi114.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-27T10:42:03.283 INFO:teuthology.orchestra.run.smithi114.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-27T10:42:03.324 INFO:teuthology.orchestra.run.smithi114.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-27T10:42:03.357 INFO:teuthology.orchestra.run.smithi114.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-27T10:42:03.391 INFO:teuthology.orchestra.run.smithi114.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-27T10:42:03.425 INFO:teuthology.orchestra.run.smithi114.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-27T10:42:03.458 INFO:teuthology.orchestra.run.smithi114.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-27T10:42:03.493 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-27T10:42:03.493 INFO:teuthology.orchestra.run.smithi114.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-27T10:42:03.800 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-27T10:42:03.800 INFO:teuthology.orchestra.run.smithi114.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-27T10:42:03.939 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-27T10:42:04.003 INFO:teuthology.orchestra.run.smithi114.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-27T10:42:04.056 INFO:teuthology.orchestra.run.smithi114.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-27T10:42:04.089 INFO:teuthology.orchestra.run.smithi114.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-27T10:42:04.129 INFO:teuthology.orchestra.run.smithi114.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-27T10:42:04.248 INFO:teuthology.orchestra.run.smithi114.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-27T10:42:04.546 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-27T10:42:04.547 INFO:teuthology.orchestra.run.smithi114.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-03-27T10:42:04.547 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-03-27T10:42:09.992 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-2:19.0.0-2478.g155268c4.el9.x86_64 1/198 2024-03-27T10:42:09.992 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-base-2:19.0.0-2478.g155268c4.el9.x86_64 2/198 2024-03-27T10:42:09.992 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-common-2:19.0.0-2478.g155268c4.el9.x86_64 3/198 2024-03-27T10:42:09.992 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-fuse-2:19.0.0-2478.g155268c4.el9.x86_64 4/198 2024-03-27T10:42:09.992 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2478.g15526 5/198 2024-03-27T10:42:09.993 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-mds-2:19.0.0-2478.g155268c4.el9.x86_64 6/198 2024-03-27T10:42:09.993 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-mgr-2:19.0.0-2478.g155268c4.el9.x86_64 7/198 2024-03-27T10:42:09.993 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-mon-2:19.0.0-2478.g155268c4.el9.x86_64 8/198 2024-03-27T10:42:09.993 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-osd-2:19.0.0-2478.g155268c4.el9.x86_64 9/198 2024-03-27T10:42:09.993 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-radosgw-2:19.0.0-2478.g155268c4.el9.x86_64 10/198 2024-03-27T10:42:09.993 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-selinux-2:19.0.0-2478.g155268c4.el9.x86_64 11/198 2024-03-27T10:42:09.993 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-test-2:19.0.0-2478.g155268c4.el9.x86_64 12/198 2024-03-27T10:42:09.993 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : cephfs-mirror-2:19.0.0-2478.g155268c4.el9.x86_64 13/198 2024-03-27T10:42:09.993 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : cephfs-shell-2:19.0.0-2478.g155268c4.el9.x86_64 14/198 2024-03-27T10:42:09.993 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libcephfs-devel-2:19.0.0-2478.g155268c4.el9.x86_ 15/198 2024-03-27T10:42:09.993 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libcephfs2-2:19.0.0-2478.g155268c4.el9.x86_64 16/198 2024-03-27T10:42:09.994 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libcephsqlite-2:19.0.0-2478.g155268c4.el9.x86_64 17/198 2024-03-27T10:42:09.994 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : librados-devel-2:19.0.0-2478.g155268c4.el9.x86_6 18/198 2024-03-27T10:42:09.994 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libradosstriper1-2:19.0.0-2478.g155268c4.el9.x86 19/198 2024-03-27T10:42:09.994 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : librgw2-2:19.0.0-2478.g155268c4.el9.x86_64 20/198 2024-03-27T10:42:09.994 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2478.g155268c4.el 21/198 2024-03-27T10:42:09.994 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-ceph-common-2:19.0.0-2478.g155268c4.el9. 22/198 2024-03-27T10:42:09.994 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-cephfs-2:19.0.0-2478.g155268c4.el9.x86_6 23/198 2024-03-27T10:42:09.994 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-rados-2:19.0.0-2478.g155268c4.el9.x86_64 24/198 2024-03-27T10:42:09.994 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-rbd-2:19.0.0-2478.g155268c4.el9.x86_64 25/198 2024-03-27T10:42:09.994 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-rgw-2:19.0.0-2478.g155268c4.el9.x86_64 26/198 2024-03-27T10:42:09.994 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : rbd-fuse-2:19.0.0-2478.g155268c4.el9.x86_64 27/198 2024-03-27T10:42:09.994 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : rbd-mirror-2:19.0.0-2478.g155268c4.el9.x86_64 28/198 2024-03-27T10:42:09.994 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : rbd-nbd-2:19.0.0-2478.g155268c4.el9.x86_64 29/198 2024-03-27T10:42:09.995 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2478.g155268c4. 30/198 2024-03-27T10:42:09.995 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2478.g155268c4.el9.noa 31/198 2024-03-27T10:42:09.995 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2478.g155268c4.el9.n 32/198 2024-03-27T10:42:09.995 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2478.g155 33/198 2024-03-27T10:42:09.995 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2478.g155268c4.el 34/198 2024-03-27T10:42:09.995 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2478.g155268c4.el9.noarch 35/198 2024-03-27T10:42:09.995 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2478.g155268c4.e 36/198 2024-03-27T10:42:09.995 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-volume-2:19.0.0-2478.g155268c4.el9.noarch 37/198 2024-03-27T10:42:09.995 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : cephadm-2:19.0.0-2478.g155268c4.el9.noarch 38/198 2024-03-27T10:42:09.995 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : cephfs-top-2:19.0.0-2478.g155268c4.el9.noarch 39/198 2024-03-27T10:42:09.995 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-27T10:42:09.995 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-27T10:42:09.995 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-27T10:42:09.996 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-27T10:42:09.996 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-27T10:42:09.996 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-27T10:42:09.996 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-27T10:42:09.996 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-27T10:42:09.996 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-27T10:42:09.996 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-27T10:42:09.996 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-27T10:42:09.996 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-27T10:42:09.996 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-27T10:42:09.996 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-27T10:42:09.996 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-27T10:42:09.996 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-27T10:42:09.997 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-27T10:42:09.997 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-27T10:42:09.997 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 58/198 2024-03-27T10:42:09.997 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 59/198 2024-03-27T10:42:09.997 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 60/198 2024-03-27T10:42:09.997 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 61/198 2024-03-27T10:42:09.997 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 62/198 2024-03-27T10:42:09.997 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 63/198 2024-03-27T10:42:09.997 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 64/198 2024-03-27T10:42:09.997 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 65/198 2024-03-27T10:42:09.997 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 66/198 2024-03-27T10:42:09.997 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 67/198 2024-03-27T10:42:09.998 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 68/198 2024-03-27T10:42:09.998 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 69/198 2024-03-27T10:42:09.998 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 70/198 2024-03-27T10:42:09.998 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 71/198 2024-03-27T10:42:09.998 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 72/198 2024-03-27T10:42:09.998 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 73/198 2024-03-27T10:42:09.998 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 74/198 2024-03-27T10:42:09.998 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 75/198 2024-03-27T10:42:09.998 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 76/198 2024-03-27T10:42:09.998 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 77/198 2024-03-27T10:42:09.998 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 78/198 2024-03-27T10:42:09.998 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 79/198 2024-03-27T10:42:09.998 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 80/198 2024-03-27T10:42:09.999 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 81/198 2024-03-27T10:42:09.999 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 82/198 2024-03-27T10:42:09.999 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 83/198 2024-03-27T10:42:09.999 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 84/198 2024-03-27T10:42:09.999 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-ply-3.11-14.el9.noarch 85/198 2024-03-27T10:42:09.999 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 86/198 2024-03-27T10:42:09.999 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 87/198 2024-03-27T10:42:09.999 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : jq-1.6-16.el9.x86_64 88/198 2024-03-27T10:42:09.999 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 89/198 2024-03-27T10:42:09.999 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 90/198 2024-03-27T10:42:09.999 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 91/198 2024-03-27T10:42:09.999 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 92/198 2024-03-27T10:42:10.000 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 93/198 2024-03-27T10:42:10.000 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 94/198 2024-03-27T10:42:10.000 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : postgresql-13.14-1.el9.x86_64 95/198 2024-03-27T10:42:10.000 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 96/198 2024-03-27T10:42:10.000 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 97/198 2024-03-27T10:42:10.000 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 98/198 2024-03-27T10:42:10.000 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-27T10:42:10.000 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-27T10:42:10.000 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 101/198 2024-03-27T10:42:10.000 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 102/198 2024-03-27T10:42:10.000 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 103/198 2024-03-27T10:42:10.000 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-27T10:42:10.000 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-27T10:42:10.000 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-27T10:42:10.001 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-27T10:42:10.001 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-27T10:42:10.001 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-27T10:42:10.001 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-27T10:42:10.001 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-27T10:42:10.001 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-27T10:42:10.001 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-27T10:42:10.001 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-27T10:42:10.001 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-27T10:42:10.001 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-03-27T10:42:10.001 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-27T10:42:10.001 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-27T10:42:10.001 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-27T10:42:10.001 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-27T10:42:10.002 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-27T10:42:10.002 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-27T10:42:10.002 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-27T10:42:10.002 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-03-27T10:42:10.002 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-27T10:42:10.002 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-27T10:42:10.002 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-27T10:42:10.002 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-27T10:42:10.002 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-27T10:42:10.002 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-27T10:42:10.002 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-27T10:42:10.002 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-27T10:42:10.002 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-27T10:42:10.003 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-27T10:42:10.003 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-27T10:42:10.003 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-27T10:42:10.003 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-27T10:42:10.003 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-27T10:42:10.003 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-27T10:42:10.003 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-27T10:42:10.003 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-27T10:42:10.003 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-27T10:42:10.003 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-27T10:42:10.003 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-27T10:42:10.003 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-27T10:42:10.003 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-27T10:42:10.004 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-27T10:42:10.004 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-27T10:42:10.004 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-27T10:42:10.004 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-27T10:42:10.004 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : librados2-2:19.0.0-2478.g155268c4.el9.x86_64 151/198 2024-03-27T10:42:10.004 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-27T10:42:10.004 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : librbd1-2:19.0.0-2478.g155268c4.el9.x86_64 153/198 2024-03-27T10:42:10.004 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-27T10:42:10.004 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-27T10:42:10.004 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-27T10:42:10.004 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 157/198 2024-03-27T10:42:10.004 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-27T10:42:10.004 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 159/198 2024-03-27T10:42:10.004 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-27T10:42:10.005 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 161/198 2024-03-27T10:42:10.005 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-27T10:42:10.005 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 163/198 2024-03-27T10:42:10.005 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-27T10:42:10.005 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-27T10:42:10.005 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-27T10:42:10.005 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-27T10:42:10.005 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-27T10:42:10.005 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-27T10:42:10.005 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-27T10:42:10.005 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-27T10:42:10.005 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-27T10:42:10.005 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-27T10:42:10.006 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-27T10:42:10.006 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-27T10:42:10.006 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-27T10:42:10.006 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-27T10:42:10.006 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-27T10:42:10.006 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-27T10:42:10.006 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-27T10:42:10.006 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-27T10:42:10.006 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-27T10:42:10.006 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-27T10:42:10.006 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-27T10:42:10.006 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-27T10:42:10.007 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-27T10:42:10.007 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-27T10:42:10.007 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-27T10:42:10.007 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-27T10:42:10.007 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-27T10:42:10.007 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-27T10:42:10.007 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-27T10:42:10.007 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-27T10:42:10.007 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-27T10:42:10.007 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-27T10:42:10.007 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-27T10:42:10.007 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-27T10:42:12.595 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-27T10:42:12.596 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-03-27T10:42:12.596 INFO:teuthology.orchestra.run.smithi114.stdout:Upgraded: 2024-03-27T10:42:12.596 INFO:teuthology.orchestra.run.smithi114.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-03-27T10:42:12.596 INFO:teuthology.orchestra.run.smithi114.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-03-27T10:42:12.596 INFO:teuthology.orchestra.run.smithi114.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-03-27T10:42:12.596 INFO:teuthology.orchestra.run.smithi114.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-03-27T10:42:12.596 INFO:teuthology.orchestra.run.smithi114.stdout: acl-2.3.1-4.el9.x86_64 2024-03-27T10:42:12.596 INFO:teuthology.orchestra.run.smithi114.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-27T10:42:12.596 INFO:teuthology.orchestra.run.smithi114.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-27T10:42:12.597 INFO:teuthology.orchestra.run.smithi114.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-27T10:42:12.597 INFO:teuthology.orchestra.run.smithi114.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-27T10:42:12.597 INFO:teuthology.orchestra.run.smithi114.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-27T10:42:12.597 INFO:teuthology.orchestra.run.smithi114.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-27T10:42:12.597 INFO:teuthology.orchestra.run.smithi114.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-27T10:42:12.597 INFO:teuthology.orchestra.run.smithi114.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-27T10:42:12.597 INFO:teuthology.orchestra.run.smithi114.stdout: librados2-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T10:42:12.597 INFO:teuthology.orchestra.run.smithi114.stdout: librbd1-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T10:42:12.597 INFO:teuthology.orchestra.run.smithi114.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-27T10:42:12.597 INFO:teuthology.orchestra.run.smithi114.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-27T10:42:12.598 INFO:teuthology.orchestra.run.smithi114.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-27T10:42:12.598 INFO:teuthology.orchestra.run.smithi114.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-27T10:42:12.598 INFO:teuthology.orchestra.run.smithi114.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-27T10:42:12.598 INFO:teuthology.orchestra.run.smithi114.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-27T10:42:12.598 INFO:teuthology.orchestra.run.smithi114.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-27T10:42:12.598 INFO:teuthology.orchestra.run.smithi114.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-27T10:42:12.598 INFO:teuthology.orchestra.run.smithi114.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-27T10:42:12.598 INFO:teuthology.orchestra.run.smithi114.stdout:Installed: 2024-03-27T10:42:12.598 INFO:teuthology.orchestra.run.smithi114.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-27T10:42:12.598 INFO:teuthology.orchestra.run.smithi114.stdout: automake-1.16.2-8.el9.noarch 2024-03-27T10:42:12.599 INFO:teuthology.orchestra.run.smithi114.stdout: bison-3.7.4-5.el9.x86_64 2024-03-27T10:42:12.599 INFO:teuthology.orchestra.run.smithi114.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-27T10:42:12.599 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T10:42:12.599 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-base-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T10:42:12.599 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-common-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T10:42:12.599 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-fuse-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T10:42:12.599 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-grafana-dashboards-2:19.0.0-2478.g155268c4.el9.noarch 2024-03-27T10:42:12.599 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-immutable-object-cache-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T10:42:12.599 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mds-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T10:42:12.599 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T10:42:12.599 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-cephadm-2:19.0.0-2478.g155268c4.el9.noarch 2024-03-27T10:42:12.599 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-dashboard-2:19.0.0-2478.g155268c4.el9.noarch 2024-03-27T10:42:12.600 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2478.g155268c4.el9.noarch 2024-03-27T10:42:12.600 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-modules-core-2:19.0.0-2478.g155268c4.el9.noarch 2024-03-27T10:42:12.600 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-rook-2:19.0.0-2478.g155268c4.el9.noarch 2024-03-27T10:42:12.600 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mon-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T10:42:12.600 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-osd-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T10:42:12.600 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-prometheus-alerts-2:19.0.0-2478.g155268c4.el9.noarch 2024-03-27T10:42:12.600 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-radosgw-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T10:42:12.600 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-selinux-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T10:42:12.600 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-test-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T10:42:12.600 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-volume-2:19.0.0-2478.g155268c4.el9.noarch 2024-03-27T10:42:12.600 INFO:teuthology.orchestra.run.smithi114.stdout: cephadm-2:19.0.0-2478.g155268c4.el9.noarch 2024-03-27T10:42:12.600 INFO:teuthology.orchestra.run.smithi114.stdout: cephfs-mirror-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T10:42:12.601 INFO:teuthology.orchestra.run.smithi114.stdout: cephfs-shell-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T10:42:12.601 INFO:teuthology.orchestra.run.smithi114.stdout: cephfs-top-2:19.0.0-2478.g155268c4.el9.noarch 2024-03-27T10:42:12.601 INFO:teuthology.orchestra.run.smithi114.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-27T10:42:12.601 INFO:teuthology.orchestra.run.smithi114.stdout: flex-2.6.4-9.el9.x86_64 2024-03-27T10:42:12.601 INFO:teuthology.orchestra.run.smithi114.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-27T10:42:12.601 INFO:teuthology.orchestra.run.smithi114.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-27T10:42:12.601 INFO:teuthology.orchestra.run.smithi114.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-27T10:42:12.601 INFO:teuthology.orchestra.run.smithi114.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-27T10:42:12.601 INFO:teuthology.orchestra.run.smithi114.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-27T10:42:12.601 INFO:teuthology.orchestra.run.smithi114.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-27T10:42:12.602 INFO:teuthology.orchestra.run.smithi114.stdout: jq-1.6-16.el9.x86_64 2024-03-27T10:42:12.602 INFO:teuthology.orchestra.run.smithi114.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-27T10:42:12.602 INFO:teuthology.orchestra.run.smithi114.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-27T10:42:12.602 INFO:teuthology.orchestra.run.smithi114.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-27T10:42:12.602 INFO:teuthology.orchestra.run.smithi114.stdout: libcephfs-devel-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T10:42:12.602 INFO:teuthology.orchestra.run.smithi114.stdout: libcephfs2-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T10:42:12.602 INFO:teuthology.orchestra.run.smithi114.stdout: libcephsqlite-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T10:42:12.602 INFO:teuthology.orchestra.run.smithi114.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-27T10:42:12.602 INFO:teuthology.orchestra.run.smithi114.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-27T10:42:12.602 INFO:teuthology.orchestra.run.smithi114.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-27T10:42:12.602 INFO:teuthology.orchestra.run.smithi114.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-27T10:42:12.603 INFO:teuthology.orchestra.run.smithi114.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-27T10:42:12.603 INFO:teuthology.orchestra.run.smithi114.stdout: librados-devel-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T10:42:12.603 INFO:teuthology.orchestra.run.smithi114.stdout: libradosstriper1-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T10:42:12.603 INFO:teuthology.orchestra.run.smithi114.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-27T10:42:12.603 INFO:teuthology.orchestra.run.smithi114.stdout: librgw2-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T10:42:12.603 INFO:teuthology.orchestra.run.smithi114.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-27T10:42:12.603 INFO:teuthology.orchestra.run.smithi114.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-27T10:42:12.603 INFO:teuthology.orchestra.run.smithi114.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-27T10:42:12.603 INFO:teuthology.orchestra.run.smithi114.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-27T10:42:12.603 INFO:teuthology.orchestra.run.smithi114.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-27T10:42:12.603 INFO:teuthology.orchestra.run.smithi114.stdout: lua-5.4.4-4.el9.x86_64 2024-03-27T10:42:12.604 INFO:teuthology.orchestra.run.smithi114.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-27T10:42:12.604 INFO:teuthology.orchestra.run.smithi114.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-27T10:42:12.604 INFO:teuthology.orchestra.run.smithi114.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-27T10:42:12.604 INFO:teuthology.orchestra.run.smithi114.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-27T10:42:12.604 INFO:teuthology.orchestra.run.smithi114.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-27T10:42:12.604 INFO:teuthology.orchestra.run.smithi114.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-27T10:42:12.604 INFO:teuthology.orchestra.run.smithi114.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-27T10:42:12.604 INFO:teuthology.orchestra.run.smithi114.stdout: postgresql-13.14-1.el9.x86_64 2024-03-27T10:42:12.604 INFO:teuthology.orchestra.run.smithi114.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-03-27T10:42:12.604 INFO:teuthology.orchestra.run.smithi114.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-03-27T10:42:12.604 INFO:teuthology.orchestra.run.smithi114.stdout: postgresql-server-13.14-1.el9.x86_64 2024-03-27T10:42:12.604 INFO:teuthology.orchestra.run.smithi114.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-27T10:42:12.605 INFO:teuthology.orchestra.run.smithi114.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-27T10:42:12.605 INFO:teuthology.orchestra.run.smithi114.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-27T10:42:12.605 INFO:teuthology.orchestra.run.smithi114.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-27T10:42:12.605 INFO:teuthology.orchestra.run.smithi114.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-27T10:42:12.605 INFO:teuthology.orchestra.run.smithi114.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-27T10:42:12.605 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-27T10:42:12.605 INFO:teuthology.orchestra.run.smithi114.stdout: python3-ceph-argparse-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T10:42:12.605 INFO:teuthology.orchestra.run.smithi114.stdout: python3-ceph-common-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T10:42:12.605 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cephfs-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T10:42:12.605 INFO:teuthology.orchestra.run.smithi114.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-27T10:42:12.605 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-27T10:42:12.605 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-27T10:42:12.606 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-27T10:42:12.606 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-27T10:42:12.606 INFO:teuthology.orchestra.run.smithi114.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-27T10:42:12.606 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-27T10:42:12.606 INFO:teuthology.orchestra.run.smithi114.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-27T10:42:12.606 INFO:teuthology.orchestra.run.smithi114.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-27T10:42:12.606 INFO:teuthology.orchestra.run.smithi114.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-27T10:42:12.606 INFO:teuthology.orchestra.run.smithi114.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-27T10:42:12.606 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-27T10:42:12.606 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-27T10:42:12.606 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-27T10:42:12.607 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-27T10:42:12.607 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-27T10:42:12.607 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-27T10:42:12.607 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-27T10:42:12.607 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-27T10:42:12.607 INFO:teuthology.orchestra.run.smithi114.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-27T10:42:12.607 INFO:teuthology.orchestra.run.smithi114.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-27T10:42:12.607 INFO:teuthology.orchestra.run.smithi114.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-27T10:42:12.607 INFO:teuthology.orchestra.run.smithi114.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-27T10:42:12.607 INFO:teuthology.orchestra.run.smithi114.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-27T10:42:12.607 INFO:teuthology.orchestra.run.smithi114.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-27T10:42:12.607 INFO:teuthology.orchestra.run.smithi114.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-27T10:42:12.608 INFO:teuthology.orchestra.run.smithi114.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-27T10:42:12.608 INFO:teuthology.orchestra.run.smithi114.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-27T10:42:12.608 INFO:teuthology.orchestra.run.smithi114.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-27T10:42:12.608 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-27T10:42:12.608 INFO:teuthology.orchestra.run.smithi114.stdout: python3-ply-3.11-14.el9.noarch 2024-03-27T10:42:12.608 INFO:teuthology.orchestra.run.smithi114.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-27T10:42:12.608 INFO:teuthology.orchestra.run.smithi114.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-27T10:42:12.608 INFO:teuthology.orchestra.run.smithi114.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-27T10:42:12.608 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-27T10:42:12.608 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-27T10:42:12.608 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-27T10:42:12.608 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-27T10:42:12.609 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-27T10:42:12.609 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-27T10:42:12.609 INFO:teuthology.orchestra.run.smithi114.stdout: python3-rados-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T10:42:12.609 INFO:teuthology.orchestra.run.smithi114.stdout: python3-rbd-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T10:42:12.609 INFO:teuthology.orchestra.run.smithi114.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-27T10:42:12.609 INFO:teuthology.orchestra.run.smithi114.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-27T10:42:12.609 INFO:teuthology.orchestra.run.smithi114.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-27T10:42:12.609 INFO:teuthology.orchestra.run.smithi114.stdout: python3-rgw-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T10:42:12.609 INFO:teuthology.orchestra.run.smithi114.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-27T10:42:12.609 INFO:teuthology.orchestra.run.smithi114.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-27T10:42:12.609 INFO:teuthology.orchestra.run.smithi114.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-27T10:42:12.609 INFO:teuthology.orchestra.run.smithi114.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-27T10:42:12.610 INFO:teuthology.orchestra.run.smithi114.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-27T10:42:12.610 INFO:teuthology.orchestra.run.smithi114.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-27T10:42:12.610 INFO:teuthology.orchestra.run.smithi114.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-27T10:42:12.610 INFO:teuthology.orchestra.run.smithi114.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-27T10:42:12.610 INFO:teuthology.orchestra.run.smithi114.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-27T10:42:12.610 INFO:teuthology.orchestra.run.smithi114.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-27T10:42:12.610 INFO:teuthology.orchestra.run.smithi114.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-27T10:42:12.610 INFO:teuthology.orchestra.run.smithi114.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-27T10:42:12.610 INFO:teuthology.orchestra.run.smithi114.stdout: qatlib-24.02.0-1.el9.x86_64 2024-03-27T10:42:12.610 INFO:teuthology.orchestra.run.smithi114.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-03-27T10:42:12.610 INFO:teuthology.orchestra.run.smithi114.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-03-27T10:42:12.610 INFO:teuthology.orchestra.run.smithi114.stdout: rbd-fuse-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T10:42:12.611 INFO:teuthology.orchestra.run.smithi114.stdout: rbd-mirror-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T10:42:12.611 INFO:teuthology.orchestra.run.smithi114.stdout: rbd-nbd-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T10:42:12.611 INFO:teuthology.orchestra.run.smithi114.stdout: re2-1:20211101-3.el9.x86_64 2024-03-27T10:42:12.611 INFO:teuthology.orchestra.run.smithi114.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-27T10:42:12.611 INFO:teuthology.orchestra.run.smithi114.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-27T10:42:12.611 INFO:teuthology.orchestra.run.smithi114.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-27T10:42:12.611 INFO:teuthology.orchestra.run.smithi114.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-27T10:42:12.611 INFO:teuthology.orchestra.run.smithi114.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-27T10:42:12.612 INFO:teuthology.orchestra.run.smithi114.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-27T10:42:12.612 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-03-27T10:42:12.612 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-03-27T10:42:13.241 DEBUG:teuthology.parallel:result is None 2024-03-27T10:42:13.241 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=155268c4e432a12433aa833f174f9fe3b1016ae0 2024-03-27T10:42:13.459 DEBUG:teuthology.orchestra.run.smithi097:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-27T10:42:13.530 INFO:teuthology.orchestra.run.smithi097.stdout:19.0.0-2478.g155268c4.el9 2024-03-27T10:42:13.530 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2478.g155268c4.el9 2024-03-27T10:42:13.530 INFO:teuthology.task.install:The correct ceph version 19.0.0-2478.g155268c4 is installed. 2024-03-27T10:42:13.532 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=155268c4e432a12433aa833f174f9fe3b1016ae0 2024-03-27T10:42:13.755 DEBUG:teuthology.orchestra.run.smithi114:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-27T10:42:13.814 INFO:teuthology.orchestra.run.smithi114.stdout:19.0.0-2478.g155268c4.el9 2024-03-27T10:42:13.815 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2478.g155268c4.el9 2024-03-27T10:42:13.815 INFO:teuthology.task.install:The correct ceph version 19.0.0-2478.g155268c4 is installed. 2024-03-27T10:42:13.817 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=155268c4e432a12433aa833f174f9fe3b1016ae0 2024-03-27T10:42:14.032 DEBUG:teuthology.orchestra.run.smithi177:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-27T10:42:14.104 INFO:teuthology.orchestra.run.smithi177.stdout:19.0.0-2478.g155268c4.el9 2024-03-27T10:42:14.104 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2478.g155268c4.el9 2024-03-27T10:42:14.104 INFO:teuthology.task.install:The correct ceph version 19.0.0-2478.g155268c4 is installed. 2024-03-27T10:42:14.106 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-27T10:42:14.106 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-27T10:42:14.106 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-27T10:42:14.209 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-03-27T10:42:14.209 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-27T10:42:14.293 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-03-27T10:42:14.293 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-27T10:42:14.397 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-27T10:42:14.397 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-27T10:42:14.398 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/usr/bin/daemon-helper 2024-03-27T10:42:14.496 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-27T10:42:14.634 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-03-27T10:42:14.634 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/usr/bin/daemon-helper 2024-03-27T10:42:14.713 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-27T10:42:14.826 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-03-27T10:42:14.826 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/usr/bin/daemon-helper 2024-03-27T10:42:14.913 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-27T10:42:15.032 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-27T10:42:15.033 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-27T10:42:15.033 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-27T10:42:15.134 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-27T10:42:15.272 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-03-27T10:42:15.273 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-27T10:42:15.346 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-27T10:42:15.459 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-03-27T10:42:15.459 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-27T10:42:15.546 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-27T10:42:15.669 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-27T10:42:15.669 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-27T10:42:15.669 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/usr/bin/stdin-killer 2024-03-27T10:42:15.764 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-27T10:42:15.857 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-03-27T10:42:15.857 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/usr/bin/stdin-killer 2024-03-27T10:42:15.928 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-27T10:42:16.053 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-03-27T10:42:16.053 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/usr/bin/stdin-killer 2024-03-27T10:42:16.137 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-27T10:42:16.264 INFO:teuthology.run_tasks:Running task cephadm... 2024-03-27T10:42:16.370 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '155268c4e432a12433aa833f174f9fe3b1016ae0', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-03-27T10:42:16.370 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 2024-03-27T10:42:16.370 INFO:tasks.cephadm:Cluster fsid is adca88c4-ec26-11ee-95d0-87774f69a715 2024-03-27T10:42:16.371 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-03-27T10:42:16.371 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.97', 'mon.b': '172.21.15.114', 'mon.c': '172.21.15.177'} 2024-03-27T10:42:16.371 INFO:tasks.cephadm:First mon is mon.a on smithi097 2024-03-27T10:42:16.371 INFO:tasks.cephadm:First mgr is x 2024-03-27T10:42:16.371 INFO:tasks.cephadm:Normalizing hostnames... 2024-03-27T10:42:16.371 DEBUG:teuthology.orchestra.run.smithi097:> sudo hostname $(hostname -s) 2024-03-27T10:42:16.472 DEBUG:teuthology.orchestra.run.smithi114:> sudo hostname $(hostname -s) 2024-03-27T10:42:16.547 DEBUG:teuthology.orchestra.run.smithi177:> sudo hostname $(hostname -s) 2024-03-27T10:42:16.642 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-03-27T10:42:16.643 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=155268c4e432a12433aa833f174f9fe3b1016ae0 2024-03-27T10:42:16.781 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '155268c4e432a12433aa833f174f9fe3b1016ae0', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/77988/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-2478-g155268c4', 'node_name': '172.21.5.35+adami05', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-2478.g155268c4'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240326.131558/155268c4e432a12433aa833f174f9fe3b1016ae0/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-03-26 19:55:12.154042', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-rishabh-testing-20240326.131558', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-rishabh-testing-20240326.131558/155268c4e432a12433aa833f174f9fe3b1016ae0/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-03-27T10:42:16.991 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref wip-rishabh-testing-20240326.131558, sha1 155268c4e432a12433aa833f174f9fe3b1016ae0 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=155268c4e432a12433aa833f174f9fe3b1016ae0 2024-03-27T10:42:16.993 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240326.131558/155268c4e432a12433aa833f174f9fe3b1016ae0/centos/9/x86_64/flavors/default/cephadm 2024-03-27T10:42:16.993 DEBUG:teuthology.orchestra.run.smithi097:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240326.131558/155268c4e432a12433aa833f174f9fe3b1016ae0/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-27T10:42:18.275 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r--. 1 ubuntu ubuntu 781299 Mar 27 10:42 /home/ubuntu/cephtest/cephadm 2024-03-27T10:42:18.276 DEBUG:teuthology.orchestra.run.smithi114:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240326.131558/155268c4e432a12433aa833f174f9fe3b1016ae0/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-27T10:42:18.999 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-r--r--. 1 ubuntu ubuntu 781299 Mar 27 10:42 /home/ubuntu/cephtest/cephadm 2024-03-27T10:42:18.999 DEBUG:teuthology.orchestra.run.smithi177:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240326.131558/155268c4e432a12433aa833f174f9fe3b1016ae0/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-27T10:42:20.267 INFO:teuthology.orchestra.run.smithi177.stdout:-rw-r--r--. 1 ubuntu ubuntu 781299 Mar 27 10:42 /home/ubuntu/cephtest/cephadm 2024-03-27T10:42:20.267 DEBUG:teuthology.orchestra.run.smithi097:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-27T10:42:20.328 DEBUG:teuthology.orchestra.run.smithi114:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-27T10:42:20.387 DEBUG:teuthology.orchestra.run.smithi177:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-27T10:42:20.462 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 on all hosts... 2024-03-27T10:42:20.463 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 pull 2024-03-27T10:42:20.470 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 pull 2024-03-27T10:42:20.477 DEBUG:teuthology.orchestra.run.smithi177:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 pull 2024-03-27T10:42:21.577 INFO:teuthology.orchestra.run.smithi177.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0... 2024-03-27T10:42:21.622 INFO:teuthology.orchestra.run.smithi097.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0... 2024-03-27T10:42:21.782 INFO:teuthology.orchestra.run.smithi114.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0... 2024-03-27T10:42:57.051 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-03-27T10:42:57.051 INFO:teuthology.orchestra.run.smithi177.stdout: "ceph_version": "ceph version 19.0.0-2478-g155268c4 (155268c4e432a12433aa833f174f9fe3b1016ae0) squid (dev)", 2024-03-27T10:42:57.051 INFO:teuthology.orchestra.run.smithi177.stdout: "image_id": "b379bbd8da6b5d0fa9a3523941efac213b6815e1cbf05c6507954f0a3fe7a801", 2024-03-27T10:42:57.052 INFO:teuthology.orchestra.run.smithi177.stdout: "repo_digests": [ 2024-03-27T10:42:57.052 INFO:teuthology.orchestra.run.smithi177.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772" 2024-03-27T10:42:57.052 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-03-27T10:42:57.052 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-03-27T10:43:11.431 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-03-27T10:43:11.431 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_version": "ceph version 19.0.0-2478-g155268c4 (155268c4e432a12433aa833f174f9fe3b1016ae0) squid (dev)", 2024-03-27T10:43:11.431 INFO:teuthology.orchestra.run.smithi097.stdout: "image_id": "b379bbd8da6b5d0fa9a3523941efac213b6815e1cbf05c6507954f0a3fe7a801", 2024-03-27T10:43:11.431 INFO:teuthology.orchestra.run.smithi097.stdout: "repo_digests": [ 2024-03-27T10:43:11.431 INFO:teuthology.orchestra.run.smithi097.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772" 2024-03-27T10:43:11.431 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-03-27T10:43:11.431 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-03-27T10:43:12.302 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-03-27T10:43:12.302 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph_version": "ceph version 19.0.0-2478-g155268c4 (155268c4e432a12433aa833f174f9fe3b1016ae0) squid (dev)", 2024-03-27T10:43:12.303 INFO:teuthology.orchestra.run.smithi114.stdout: "image_id": "b379bbd8da6b5d0fa9a3523941efac213b6815e1cbf05c6507954f0a3fe7a801", 2024-03-27T10:43:12.303 INFO:teuthology.orchestra.run.smithi114.stdout: "repo_digests": [ 2024-03-27T10:43:12.303 INFO:teuthology.orchestra.run.smithi114.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772" 2024-03-27T10:43:12.303 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-03-27T10:43:12.303 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-03-27T10:43:12.342 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /etc/ceph 2024-03-27T10:43:12.445 DEBUG:teuthology.orchestra.run.smithi114:> sudo mkdir -p /etc/ceph 2024-03-27T10:43:12.546 DEBUG:teuthology.orchestra.run.smithi177:> sudo mkdir -p /etc/ceph 2024-03-27T10:43:12.649 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 777 /etc/ceph 2024-03-27T10:43:12.748 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod 777 /etc/ceph 2024-03-27T10:43:12.836 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod 777 /etc/ceph 2024-03-27T10:43:12.932 INFO:tasks.cephadm:Writing seed config... 2024-03-27T10:43:12.933 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-03-27T10:43:12.933 INFO:tasks.cephadm: override: [client] debug client = 20 2024-03-27T10:43:12.933 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-03-27T10:43:12.933 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-03-27T10:43:12.933 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-03-27T10:43:12.933 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-03-27T10:43:12.934 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-03-27T10:43:12.934 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-03-27T10:43:12.934 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-03-27T10:43:12.934 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-03-27T10:43:12.934 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-03-27T10:43:12.934 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-03-27T10:43:12.934 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-03-27T10:43:12.934 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-03-27T10:43:12.934 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-03-27T10:43:12.934 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-03-27T10:43:12.934 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-03-27T10:43:12.934 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-03-27T10:43:12.934 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-03-27T10:43:12.934 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-03-27T10:43:12.935 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-03-27T10:43:12.935 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-03-27T10:43:12.935 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-03-27T10:43:12.935 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-03-27T10:43:12.935 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-03-27T10:43:12.935 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-03-27T10:43:12.935 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-03-27T10:43:12.935 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-03-27T10:43:12.935 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-03-27T10:43:12.935 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-03-27T10:43:12.935 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-03-27T10:43:12.935 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-03-27T10:43:12.935 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-03-27T10:43:12.935 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-03-27T10:43:12.936 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-03-27T10:43:12.936 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-03-27T10:43:12.936 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-03-27T10:43:12.936 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-03-27T10:43:12.936 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-03-27T10:43:12.936 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-03-27T10:43:12.936 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-03-27T10:43:12.936 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-03-27T10:43:12.936 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-03-27T10:43:12.936 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-03-27T10:43:12.936 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-03-27T10:43:12.936 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-03-27T10:43:12.936 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-03-27T10:43:12.937 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-27T10:43:12.937 DEBUG:teuthology.orchestra.run.smithi097:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-03-27T10:43:12.993 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = adca88c4-ec26-11ee-95d0-87774f69a715 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-03-27T10:43:12.994 DEBUG:teuthology.orchestra.run.smithi097:mon.a> sudo journalctl -f -n 0 -u ceph-adca88c4-ec26-11ee-95d0-87774f69a715@mon.a.service 2024-03-27T10:43:13.041 DEBUG:teuthology.orchestra.run.smithi097:mgr.x> sudo journalctl -f -n 0 -u ceph-adca88c4-ec26-11ee-95d0-87774f69a715@mgr.x.service 2024-03-27T10:43:13.085 INFO:tasks.cephadm:Bootstrapping... 2024-03-27T10:43:13.086 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 -v bootstrap --fsid adca88c4-ec26-11ee-95d0-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.97 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-03-27T10:43:13.368 INFO:teuthology.orchestra.run.smithi097.stdout:-------------------------------------------------------------------------------- 2024-03-27T10:43:13.368 INFO:teuthology.orchestra.run.smithi097.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0', '-v', 'bootstrap', '--fsid', 'adca88c4-ec26-11ee-95d0-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.97', '--skip-admin-label'] 2024-03-27T10:43:13.368 INFO:teuthology.orchestra.run.smithi097.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-03-27T10:43:13.368 INFO:teuthology.orchestra.run.smithi097.stdout:Verifying podman|docker is present... 2024-03-27T10:43:13.426 INFO:teuthology.orchestra.run.smithi097.stdout:/bin/podman: stdout 4.9.4-dev 2024-03-27T10:43:13.426 INFO:teuthology.orchestra.run.smithi097.stdout:Verifying lvm2 is present... 2024-03-27T10:43:13.427 INFO:teuthology.orchestra.run.smithi097.stdout:Verifying time synchronization is in place... 2024-03-27T10:43:13.445 INFO:teuthology.orchestra.run.smithi097.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-27T10:43:13.446 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-27T10:43:13.463 INFO:teuthology.orchestra.run.smithi097.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-27T10:43:13.463 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stdout inactive 2024-03-27T10:43:13.486 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stdout enabled 2024-03-27T10:43:13.505 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stdout active 2024-03-27T10:43:13.505 INFO:teuthology.orchestra.run.smithi097.stdout:Unit chronyd.service is enabled and running 2024-03-27T10:43:13.506 INFO:teuthology.orchestra.run.smithi097.stdout:Repeating the final host check... 2024-03-27T10:43:13.560 INFO:teuthology.orchestra.run.smithi097.stdout:/bin/podman: stdout 4.9.4-dev 2024-03-27T10:43:13.560 INFO:teuthology.orchestra.run.smithi097.stdout:podman (/bin/podman) version 4.9.4 is present 2024-03-27T10:43:13.560 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl is present 2024-03-27T10:43:13.560 INFO:teuthology.orchestra.run.smithi097.stdout:lvcreate is present 2024-03-27T10:43:13.574 INFO:teuthology.orchestra.run.smithi097.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-27T10:43:13.574 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-27T10:43:13.591 INFO:teuthology.orchestra.run.smithi097.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-27T10:43:13.591 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stdout inactive 2024-03-27T10:43:13.615 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stdout enabled 2024-03-27T10:43:13.636 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stdout active 2024-03-27T10:43:13.636 INFO:teuthology.orchestra.run.smithi097.stdout:Unit chronyd.service is enabled and running 2024-03-27T10:43:13.637 INFO:teuthology.orchestra.run.smithi097.stdout:Host looks OK 2024-03-27T10:43:13.637 INFO:teuthology.orchestra.run.smithi097.stdout:Cluster fsid: adca88c4-ec26-11ee-95d0-87774f69a715 2024-03-27T10:43:13.637 INFO:teuthology.orchestra.run.smithi097.stdout:Acquiring lock 139708854375088 on /run/cephadm/adca88c4-ec26-11ee-95d0-87774f69a715.lock 2024-03-27T10:43:13.638 INFO:teuthology.orchestra.run.smithi097.stdout:Lock 139708854375088 acquired on /run/cephadm/adca88c4-ec26-11ee-95d0-87774f69a715.lock 2024-03-27T10:43:13.638 INFO:teuthology.orchestra.run.smithi097.stdout:Verifying IP 172.21.15.97 port 3300 ... 2024-03-27T10:43:13.639 INFO:teuthology.orchestra.run.smithi097.stdout:Verifying IP 172.21.15.97 port 6789 ... 2024-03-27T10:43:13.640 INFO:teuthology.orchestra.run.smithi097.stdout:Base mon IP(s) is [172.21.15.97:3300, 172.21.15.97:6789], mon addrv is [v2:172.21.15.97:3300,v1:172.21.15.97:6789] 2024-03-27T10:43:13.655 INFO:teuthology.orchestra.run.smithi097.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.97 metric 100 2024-03-27T10:43:13.655 INFO:teuthology.orchestra.run.smithi097.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.97 metric 100 2024-03-27T10:43:13.667 INFO:teuthology.orchestra.run.smithi097.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-03-27T10:43:13.668 INFO:teuthology.orchestra.run.smithi097.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-03-27T10:43:13.668 INFO:teuthology.orchestra.run.smithi097.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1783sec hoplimit 64 pref medium 2024-03-27T10:43:13.678 INFO:teuthology.orchestra.run.smithi097.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-03-27T10:43:13.679 INFO:teuthology.orchestra.run.smithi097.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-03-27T10:43:13.679 INFO:teuthology.orchestra.run.smithi097.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-27T10:43:13.679 INFO:teuthology.orchestra.run.smithi097.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-03-27T10:43:13.679 INFO:teuthology.orchestra.run.smithi097.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:d805/64 scope link 2024-03-27T10:43:13.680 INFO:teuthology.orchestra.run.smithi097.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-27T10:43:13.682 INFO:teuthology.orchestra.run.smithi097.stdout:Mon IP `172.21.15.97` is in CIDR network `172.21.0.0/20` 2024-03-27T10:43:13.682 INFO:teuthology.orchestra.run.smithi097.stdout:Mon IP `172.21.15.97` is in CIDR network `172.21.0.0/20` 2024-03-27T10:43:13.683 INFO:teuthology.orchestra.run.smithi097.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-03-27T10:43:13.685 INFO:teuthology.orchestra.run.smithi097.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-03-27T10:43:13.688 INFO:teuthology.orchestra.run.smithi097.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0... 2024-03-27T10:43:14.322 INFO:teuthology.orchestra.run.smithi097.stdout:/bin/podman: stdout b379bbd8da6b5d0fa9a3523941efac213b6815e1cbf05c6507954f0a3fe7a801 2024-03-27T10:43:14.322 INFO:teuthology.orchestra.run.smithi097.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0... 2024-03-27T10:43:14.323 INFO:teuthology.orchestra.run.smithi097.stdout:/bin/podman: stderr Getting image source signatures 2024-03-27T10:43:14.323 INFO:teuthology.orchestra.run.smithi097.stdout:/bin/podman: stderr Copying blob sha256:e605917547793151c989dc0ed66d07767a9fed5a2365fd05914b705e7a7b6e21 2024-03-27T10:43:14.323 INFO:teuthology.orchestra.run.smithi097.stdout:/bin/podman: stderr Copying blob sha256:11da15500eca1f826c328eb3265caa30b0a3730510e2a32023084d6fe9cee20c 2024-03-27T10:43:14.323 INFO:teuthology.orchestra.run.smithi097.stdout:/bin/podman: stderr Copying config sha256:b379bbd8da6b5d0fa9a3523941efac213b6815e1cbf05c6507954f0a3fe7a801 2024-03-27T10:43:14.323 INFO:teuthology.orchestra.run.smithi097.stdout:/bin/podman: stderr Writing manifest to image destination 2024-03-27T10:43:15.542 INFO:teuthology.orchestra.run.smithi097.stdout:ceph: stdout ceph version 19.0.0-2478-g155268c4 (155268c4e432a12433aa833f174f9fe3b1016ae0) squid (dev) 2024-03-27T10:43:15.542 INFO:teuthology.orchestra.run.smithi097.stdout:Ceph version: ceph version 19.0.0-2478-g155268c4 (155268c4e432a12433aa833f174f9fe3b1016ae0) squid (dev) 2024-03-27T10:43:15.542 INFO:teuthology.orchestra.run.smithi097.stdout:Extracting ceph user uid/gid from container image... 2024-03-27T10:43:16.615 INFO:teuthology.orchestra.run.smithi097.stdout:stat: stdout 167 167 2024-03-27T10:43:16.615 INFO:teuthology.orchestra.run.smithi097.stdout:Creating initial keys... 2024-03-27T10:43:17.818 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph-authtool: stdout AQBF+ANmFWc8CRAAch4jTXQOdjHLwCVP/HR2gw== 2024-03-27T10:43:18.948 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph-authtool: stdout AQBG+ANmXJIaFBAAvIku4IQdCflInv9/DQHlag== 2024-03-27T10:43:20.052 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph-authtool: stdout AQBH+ANm0CxUHBAARWvol43Lv9wFhXSgYHUrOQ== 2024-03-27T10:43:20.053 INFO:teuthology.orchestra.run.smithi097.stdout:Creating initial monmap... 2024-03-27T10:43:21.251 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-27T10:43:21.252 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-03-27T10:43:21.252 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to adca88c4-ec26-11ee-95d0-87774f69a715 2024-03-27T10:43:21.252 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-27T10:43:21.252 INFO:teuthology.orchestra.run.smithi097.stdout:monmaptool for a [v2:172.21.15.97:3300,v1:172.21.15.97:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-27T10:43:21.252 INFO:teuthology.orchestra.run.smithi097.stdout:setting min_mon_release = quincy 2024-03-27T10:43:21.252 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/monmaptool: set fsid to adca88c4-ec26-11ee-95d0-87774f69a715 2024-03-27T10:43:21.252 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-27T10:43:21.252 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T10:43:21.252 INFO:teuthology.orchestra.run.smithi097.stdout:Creating mon... 2024-03-27T10:43:22.417 INFO:teuthology.orchestra.run.smithi097.stdout:create mon.a on 2024-03-27T10:43:22.765 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-03-27T10:43:23.004 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-03-27T10:43:23.294 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-adca88c4-ec26-11ee-95d0-87774f69a715.target → /etc/systemd/system/ceph-adca88c4-ec26-11ee-95d0-87774f69a715.target. 2024-03-27T10:43:23.294 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-adca88c4-ec26-11ee-95d0-87774f69a715.target → /etc/systemd/system/ceph-adca88c4-ec26-11ee-95d0-87774f69a715.target. 2024-03-27T10:43:23.658 INFO:teuthology.orchestra.run.smithi097.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-adca88c4-ec26-11ee-95d0-87774f69a715@mon.a 2024-03-27T10:43:23.658 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stderr Failed to reset failed state of unit ceph-adca88c4-ec26-11ee-95d0-87774f69a715@mon.a.service: Unit ceph-adca88c4-ec26-11ee-95d0-87774f69a715@mon.a.service not loaded. 2024-03-27T10:43:23.936 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-adca88c4-ec26-11ee-95d0-87774f69a715.target.wants/ceph-adca88c4-ec26-11ee-95d0-87774f69a715@mon.a.service → /etc/systemd/system/ceph-adca88c4-ec26-11ee-95d0-87774f69a715@.service. 2024-03-27T10:43:24.571 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:24 smithi097 podman[31461]: 2024-03-27 10:43:24.25023517 +0000 UTC m=+0.034049305 image pull b379bbd8da6b5d0fa9a3523941efac213b6815e1cbf05c6507954f0a3fe7a801 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 2024-03-27T10:43:24.571 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:24 smithi097 podman[31461]: 2024-03-27 10:43:24.595345794 +0000 UTC m=+0.379159973 container init ed88f7f1eda92e4b43046e530d8dafaf46213e1f5b51945eff51f0c557ae0349 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mon-a, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326) 2024-03-27T10:43:24.680 INFO:teuthology.orchestra.run.smithi097.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-27T10:43:24.680 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stdout disabled 2024-03-27T10:43:24.695 INFO:teuthology.orchestra.run.smithi097.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-27T10:43:24.695 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stdout inactive 2024-03-27T10:43:24.695 INFO:teuthology.orchestra.run.smithi097.stdout:firewalld.service is not enabled 2024-03-27T10:43:24.695 INFO:teuthology.orchestra.run.smithi097.stdout:Not possible to enable service . firewalld.service is not available 2024-03-27T10:43:24.695 INFO:teuthology.orchestra.run.smithi097.stdout:Waiting for mon to start... 2024-03-27T10:43:24.695 INFO:teuthology.orchestra.run.smithi097.stdout:Waiting for mon... 2024-03-27T10:43:24.841 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:24 smithi097 podman[31461]: 2024-03-27 10:43:24.605981048 +0000 UTC m=+0.389795227 container start ed88f7f1eda92e4b43046e530d8dafaf46213e1f5b51945eff51f0c557ae0349 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-03-27T10:43:24.841 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:24 smithi097 bash[31461]: ed88f7f1eda92e4b43046e530d8dafaf46213e1f5b51945eff51f0c557ae0349 2024-03-27T10:43:24.841 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:24 smithi097 systemd[1]: Started Ceph mon.a for adca88c4-ec26-11ee-95d0-87774f69a715. 2024-03-27T10:43:24.841 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:24 smithi097 ceph-mon[31479]: mkfs adca88c4-ec26-11ee-95d0-87774f69a715 2024-03-27T10:43:24.841 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:24 smithi097 ceph-mon[31479]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-27T10:43:26.084 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:25 smithi097 ceph-mon[31479]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-27T10:43:26.084 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:25 smithi097 ceph-mon[31479]: monmap epoch 1 2024-03-27T10:43:26.084 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:25 smithi097 ceph-mon[31479]: fsid adca88c4-ec26-11ee-95d0-87774f69a715 2024-03-27T10:43:26.084 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:25 smithi097 ceph-mon[31479]: last_changed 2024-03-27T10:43:20.535207+0000 2024-03-27T10:43:26.084 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:25 smithi097 ceph-mon[31479]: created 2024-03-27T10:43:20.535207+0000 2024-03-27T10:43:26.084 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:25 smithi097 ceph-mon[31479]: min_mon_release 19 (squid) 2024-03-27T10:43:26.084 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:25 smithi097 ceph-mon[31479]: election_strategy: 1 2024-03-27T10:43:26.084 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:25 smithi097 ceph-mon[31479]: 0: [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] mon.a 2024-03-27T10:43:26.084 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:25 smithi097 ceph-mon[31479]: fsmap 2024-03-27T10:43:26.084 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:25 smithi097 ceph-mon[31479]: osdmap e1: 0 total, 0 up, 0 in 2024-03-27T10:43:26.085 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:25 smithi097 ceph-mon[31479]: mgrmap e1: no daemons active 2024-03-27T10:43:26.085 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:25 smithi097 ceph-mon[31479]: from='client.? 172.21.15.97:0/264673038' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-03-27T10:43:26.097 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout cluster: 2024-03-27T10:43:26.104 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout id: adca88c4-ec26-11ee-95d0-87774f69a715 2024-03-27T10:43:26.104 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-03-27T10:43:26.104 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 2024-03-27T10:43:26.104 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout services: 2024-03-27T10:43:26.104 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.740652s) 2024-03-27T10:43:26.104 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-03-27T10:43:26.104 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-03-27T10:43:26.104 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 2024-03-27T10:43:26.104 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout data: 2024-03-27T10:43:26.104 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-03-27T10:43:26.104 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-03-27T10:43:26.105 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-03-27T10:43:26.105 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout pgs: 2024-03-27T10:43:26.105 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 2024-03-27T10:43:26.105 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:25.328+0000 7f1790200700 1 Processor -- start 2024-03-27T10:43:26.105 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:25.328+0000 7f1790200700 1 -- start start 2024-03-27T10:43:26.105 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:25.329+0000 7f1790200700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17881068d0 0x7f1788106cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:43:26.105 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:25.329+0000 7f1790200700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f178807be50 con 0x7f17881068d0 2024-03-27T10:43:26.105 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:25.330+0000 7f178d800700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17881068d0 0x7f1788106cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:43:26.105 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:25.330+0000 7f178d800700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17881068d0 0x7f1788106cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34120/0 (socket says 172.21.15.97:34120) 2024-03-27T10:43:26.105 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:25.330+0000 7f178d800700 1 -- 172.21.15.97:0/54409753 learned_addr learned my addr 172.21.15.97:0/54409753 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:43:26.105 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:25.330+0000 7f178d800700 1 -- 172.21.15.97:0/54409753 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1788107230 con 0x7f17881068d0 2024-03-27T10:43:26.105 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:25.331+0000 7f178d800700 1 --2- 172.21.15.97:0/54409753 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17881068d0 0x7f1788106cf0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f177c0098a0 tx=0x7f177c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cddae4b48451dd52 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:43:26.106 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:25.331+0000 7f1787e00700 1 -- 172.21.15.97:0/54409753 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f177c004160 con 0x7f17881068d0 2024-03-27T10:43:26.106 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:25.331+0000 7f1787e00700 1 -- 172.21.15.97:0/54409753 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f177c004300 con 0x7f17881068d0 2024-03-27T10:43:26.106 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:25.331+0000 7f1787e00700 1 -- 172.21.15.97:0/54409753 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f177c0036a0 con 0x7f17881068d0 2024-03-27T10:43:26.106 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:25.332+0000 7f1790200700 1 -- 172.21.15.97:0/54409753 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17881068d0 msgr2=0x7f1788106cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:43:26.106 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:25.332+0000 7f1790200700 1 --2- 172.21.15.97:0/54409753 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17881068d0 0x7f1788106cf0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f177c0098a0 tx=0x7f177c009bb0 comp rx=0 tx=0).stop 2024-03-27T10:43:26.106 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:25.332+0000 7f1790200700 1 -- 172.21.15.97:0/54409753 shutdown_connections 2024-03-27T10:43:26.106 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:25.332+0000 7f1790200700 1 --2- 172.21.15.97:0/54409753 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17881068d0 0x7f1788106cf0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:43:26.106 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:25.332+0000 7f1790200700 1 -- 172.21.15.97:0/54409753 >> 172.21.15.97:0/54409753 conn(0x7f1788102490 msgr2=0x7f17881048d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:43:26.106 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:25.332+0000 7f1790200700 1 -- 172.21.15.97:0/54409753 shutdown_connections 2024-03-27T10:43:26.106 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:25.332+0000 7f1790200700 1 -- 172.21.15.97:0/54409753 wait complete. 2024-03-27T10:43:26.106 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:25.333+0000 7f1790200700 1 Processor -- start 2024-03-27T10:43:26.106 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:25.333+0000 7f1790200700 1 -- start start 2024-03-27T10:43:26.107 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:25.333+0000 7f1790200700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17881068d0 0x7f178819d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:43:26.107 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:25.333+0000 7f1790200700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1788107760 con 0x7f17881068d0 2024-03-27T10:43:26.107 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:25.334+0000 7f178d800700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17881068d0 0x7f178819d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:43:26.107 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:25.334+0000 7f178d800700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17881068d0 0x7f178819d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34126/0 (socket says 172.21.15.97:34126) 2024-03-27T10:43:26.107 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:25.334+0000 7f178d800700 1 -- 172.21.15.97:0/264673038 learned_addr learned my addr 172.21.15.97:0/264673038 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:43:26.107 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:25.334+0000 7f178d800700 1 -- 172.21.15.97:0/264673038 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f178819de30 con 0x7f17881068d0 2024-03-27T10:43:26.107 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:25.334+0000 7f178d800700 1 --2- 172.21.15.97:0/264673038 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17881068d0 0x7f178819d8f0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f177c009fb0 tx=0x7f177c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:43:26.107 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:25.334+0000 7f1786000700 1 -- 172.21.15.97:0/264673038 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f177c003de0 con 0x7f17881068d0 2024-03-27T10:43:26.107 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:25.335+0000 7f1786000700 1 -- 172.21.15.97:0/264673038 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f177c011630 con 0x7f17881068d0 2024-03-27T10:43:26.107 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:25.335+0000 7f1786000700 1 -- 172.21.15.97:0/264673038 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f177c011930 con 0x7f17881068d0 2024-03-27T10:43:26.107 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:25.335+0000 7f1790200700 1 -- 172.21.15.97:0/264673038 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f178819e0c0 con 0x7f17881068d0 2024-03-27T10:43:26.107 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:25.335+0000 7f1790200700 1 -- 172.21.15.97:0/264673038 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17881a0db0 con 0x7f17881068d0 2024-03-27T10:43:26.107 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:25.335+0000 7f1786000700 1 -- 172.21.15.97:0/264673038 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f177c024480 con 0x7f17881068d0 2024-03-27T10:43:26.108 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:25.335+0000 7f1786000700 1 -- 172.21.15.97:0/264673038 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f177c01a7e0 con 0x7f17881068d0 2024-03-27T10:43:26.108 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:25.336+0000 7f1790200700 1 -- 172.21.15.97:0/264673038 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1788063c70 con 0x7f17881068d0 2024-03-27T10:43:26.108 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:25.339+0000 7f1786000700 1 -- 172.21.15.97:0/264673038 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f177c01aa80 con 0x7f17881068d0 2024-03-27T10:43:26.108 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:25.418+0000 7f1790200700 1 -- 172.21.15.97:0/264673038 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f17881a1290 con 0x7f17881068d0 2024-03-27T10:43:26.108 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:25.418+0000 7f1786000700 1 -- 172.21.15.97:0/264673038 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f177c01ac60 con 0x7f17881068d0 2024-03-27T10:43:26.108 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:25.419+0000 7f1790200700 1 -- 172.21.15.97:0/264673038 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17881068d0 msgr2=0x7f178819d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:43:26.108 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:25.419+0000 7f1790200700 1 --2- 172.21.15.97:0/264673038 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17881068d0 0x7f178819d8f0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f177c009fb0 tx=0x7f177c004750 comp rx=0 tx=0).stop 2024-03-27T10:43:26.108 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:25.420+0000 7f1790200700 1 -- 172.21.15.97:0/264673038 shutdown_connections 2024-03-27T10:43:26.108 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:25.420+0000 7f1790200700 1 --2- 172.21.15.97:0/264673038 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17881068d0 0x7f178819d8f0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:43:26.108 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:25.420+0000 7f1790200700 1 -- 172.21.15.97:0/264673038 >> 172.21.15.97:0/264673038 conn(0x7f1788102490 msgr2=0x7f1788193f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:43:26.108 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:25.420+0000 7f1790200700 1 -- 172.21.15.97:0/264673038 shutdown_connections 2024-03-27T10:43:26.108 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:25.420+0000 7f1790200700 1 -- 172.21.15.97:0/264673038 wait complete. 2024-03-27T10:43:26.108 INFO:teuthology.orchestra.run.smithi097.stdout:mon is available 2024-03-27T10:43:26.109 INFO:teuthology.orchestra.run.smithi097.stdout:Assimilating anything we can from ceph.conf... 2024-03-27T10:43:27.135 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:26 smithi097 ceph-mon[31479]: from='client.? 172.21.15.97:0/2002708948' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-27T10:43:27.135 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:26 smithi097 ceph-mon[31479]: from='client.? 172.21.15.97:0/2002708948' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-03-27T10:43:27.495 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 2024-03-27T10:43:27.496 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout [global] 2024-03-27T10:43:27.496 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout fsid = adca88c4-ec26-11ee-95d0-87774f69a715 2024-03-27T10:43:27.496 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.97:3300,v1:172.21.15.97:6789] 2024-03-27T10:43:27.496 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-27T10:43:27.496 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-27T10:43:27.496 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-27T10:43:27.496 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-27T10:43:27.496 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 2024-03-27T10:43:27.497 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-27T10:43:27.497 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-27T10:43:27.497 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 2024-03-27T10:43:27.497 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout [osd] 2024-03-27T10:43:27.497 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-03-27T10:43:27.497 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-27T10:43:27.497 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-27T10:43:27.497 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-27T10:43:27.497 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-27T10:43:27.498 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-27T10:43:27.498 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:26.705+0000 7f341b800700 1 Processor -- start 2024-03-27T10:43:27.498 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:26.705+0000 7f341b800700 1 -- start start 2024-03-27T10:43:27.498 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:26.706+0000 7f341b800700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f341410a1f0 0x7f341410a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:43:27.498 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:26.706+0000 7f341b800700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f341410abe0 con 0x7f341410a1f0 2024-03-27T10:43:27.498 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:26.706+0000 7f3418e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f341410a1f0 0x7f341410a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:43:27.498 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:26.706+0000 7f3418e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f341410a1f0 0x7f341410a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34142/0 (socket says 172.21.15.97:34142) 2024-03-27T10:43:27.499 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:26.706+0000 7f3418e00700 1 -- 172.21.15.97:0/1047984935 learned_addr learned my addr 172.21.15.97:0/1047984935 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:43:27.499 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:26.707+0000 7f3418e00700 1 -- 172.21.15.97:0/1047984935 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f341410b3a0 con 0x7f341410a1f0 2024-03-27T10:43:27.499 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:26.707+0000 7f3418e00700 1 --2- 172.21.15.97:0/1047984935 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f341410a1f0 0x7f341410a610 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f34080098a0 tx=0x7f3408009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e825daa079eba54 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:43:27.499 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:26.707+0000 7f3413400700 1 -- 172.21.15.97:0/1047984935 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3408004160 con 0x7f341410a1f0 2024-03-27T10:43:27.499 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:26.707+0000 7f3413400700 1 -- 172.21.15.97:0/1047984935 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3408004300 con 0x7f341410a1f0 2024-03-27T10:43:27.499 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:26.707+0000 7f3413400700 1 -- 172.21.15.97:0/1047984935 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f34080036a0 con 0x7f341410a1f0 2024-03-27T10:43:27.499 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:26.708+0000 7f341b800700 1 -- 172.21.15.97:0/1047984935 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f341410a1f0 msgr2=0x7f341410a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:43:27.499 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:26.708+0000 7f341b800700 1 --2- 172.21.15.97:0/1047984935 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f341410a1f0 0x7f341410a610 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f34080098a0 tx=0x7f3408009bb0 comp rx=0 tx=0).stop 2024-03-27T10:43:27.500 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:26.708+0000 7f341b800700 1 -- 172.21.15.97:0/1047984935 shutdown_connections 2024-03-27T10:43:27.500 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:26.708+0000 7f341b800700 1 --2- 172.21.15.97:0/1047984935 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f341410a1f0 0x7f341410a610 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:43:27.500 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:26.708+0000 7f341b800700 1 -- 172.21.15.97:0/1047984935 >> 172.21.15.97:0/1047984935 conn(0x7f341407d6e0 msgr2=0x7f341407daf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:43:27.500 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:26.708+0000 7f341b800700 1 -- 172.21.15.97:0/1047984935 shutdown_connections 2024-03-27T10:43:27.500 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:26.708+0000 7f341b800700 1 -- 172.21.15.97:0/1047984935 wait complete. 2024-03-27T10:43:27.500 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:26.709+0000 7f341b800700 1 Processor -- start 2024-03-27T10:43:27.500 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:26.709+0000 7f341b800700 1 -- start start 2024-03-27T10:43:27.500 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:26.710+0000 7f341b800700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f341410a1f0 0x7f341419d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:43:27.501 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:26.710+0000 7f341b800700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f341410b8d0 con 0x7f341410a1f0 2024-03-27T10:43:27.501 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:26.710+0000 7f3418e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f341410a1f0 0x7f341419d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:43:27.501 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:26.710+0000 7f3418e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f341410a1f0 0x7f341419d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34152/0 (socket says 172.21.15.97:34152) 2024-03-27T10:43:27.501 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:26.710+0000 7f3418e00700 1 -- 172.21.15.97:0/2002708948 learned_addr learned my addr 172.21.15.97:0/2002708948 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:43:27.501 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:26.710+0000 7f3418e00700 1 -- 172.21.15.97:0/2002708948 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f341419de50 con 0x7f341410a1f0 2024-03-27T10:43:27.501 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:26.710+0000 7f3418e00700 1 --2- 172.21.15.97:0/2002708948 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f341410a1f0 0x7f341419d910 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f3408000c00 tx=0x7f3408004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:43:27.501 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:26.711+0000 7f3411600700 1 -- 172.21.15.97:0/2002708948 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3408003db0 con 0x7f341410a1f0 2024-03-27T10:43:27.501 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:26.711+0000 7f3411600700 1 -- 172.21.15.97:0/2002708948 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3408011630 con 0x7f341410a1f0 2024-03-27T10:43:27.502 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:26.711+0000 7f3411600700 1 -- 172.21.15.97:0/2002708948 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3408011950 con 0x7f341410a1f0 2024-03-27T10:43:27.502 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:26.711+0000 7f341b800700 1 -- 172.21.15.97:0/2002708948 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f341419e0e0 con 0x7f341410a1f0 2024-03-27T10:43:27.502 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:26.711+0000 7f341b800700 1 -- 172.21.15.97:0/2002708948 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34141a0dd0 con 0x7f341410a1f0 2024-03-27T10:43:27.502 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:26.711+0000 7f3411600700 1 -- 172.21.15.97:0/2002708948 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3408011af0 con 0x7f341410a1f0 2024-03-27T10:43:27.502 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:26.711+0000 7f3411600700 1 -- 172.21.15.97:0/2002708948 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f340801c450 con 0x7f341410a1f0 2024-03-27T10:43:27.502 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:26.712+0000 7f341b800700 1 -- 172.21.15.97:0/2002708948 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3414063c70 con 0x7f341410a1f0 2024-03-27T10:43:27.502 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:26.715+0000 7f3411600700 1 -- 172.21.15.97:0/2002708948 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f3408032020 con 0x7f341410a1f0 2024-03-27T10:43:27.503 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:26.791+0000 7f341b800700 1 -- 172.21.15.97:0/2002708948 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f341402deb0 con 0x7f341410a1f0 2024-03-27T10:43:27.503 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:26.797+0000 7f3411600700 1 -- 172.21.15.97:0/2002708948 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+456 (secure 0 0 0) 0x7f3408004260 con 0x7f341410a1f0 2024-03-27T10:43:27.503 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:26.798+0000 7f3411600700 1 -- 172.21.15.97:0/2002708948 <== mon.0 v2:172.21.15.97:3300/0 8 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f340801c650 con 0x7f341410a1f0 2024-03-27T10:43:27.503 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:26.799+0000 7f341b800700 1 -- 172.21.15.97:0/2002708948 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f341410a1f0 msgr2=0x7f341419d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:43:27.503 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:26.799+0000 7f341b800700 1 --2- 172.21.15.97:0/2002708948 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f341410a1f0 0x7f341419d910 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f3408000c00 tx=0x7f3408004750 comp rx=0 tx=0).stop 2024-03-27T10:43:27.503 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:26.799+0000 7f341b800700 1 -- 172.21.15.97:0/2002708948 shutdown_connections 2024-03-27T10:43:27.503 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:26.799+0000 7f341b800700 1 --2- 172.21.15.97:0/2002708948 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f341410a1f0 0x7f341419d910 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:43:27.504 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:26.799+0000 7f341b800700 1 -- 172.21.15.97:0/2002708948 >> 172.21.15.97:0/2002708948 conn(0x7f341407d6e0 msgr2=0x7f34141944f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:43:27.504 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:26.800+0000 7f341b800700 1 -- 172.21.15.97:0/2002708948 shutdown_connections 2024-03-27T10:43:27.504 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:26.800+0000 7f341b800700 1 -- 172.21.15.97:0/2002708948 wait complete. 2024-03-27T10:43:27.504 INFO:teuthology.orchestra.run.smithi097.stdout:Generating new minimal ceph.conf... 2024-03-27T10:43:28.394 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:28 smithi097 ceph-mon[31479]: from='client.? 172.21.15.97:0/111357702' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:43:28.842 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:28.079+0000 7ff565800700 1 Processor -- start 2024-03-27T10:43:28.843 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:28.079+0000 7ff565800700 1 -- start start 2024-03-27T10:43:28.843 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:28.080+0000 7ff565800700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff560106840 0x7ff560106c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:43:28.843 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:28.080+0000 7ff565800700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff56007be70 con 0x7ff560106840 2024-03-27T10:43:28.843 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:28.080+0000 7ff55ea00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff560106840 0x7ff560106c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:43:28.843 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:28.080+0000 7ff55ea00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff560106840 0x7ff560106c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34162/0 (socket says 172.21.15.97:34162) 2024-03-27T10:43:28.843 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:28.080+0000 7ff55ea00700 1 -- 172.21.15.97:0/3476857164 learned_addr learned my addr 172.21.15.97:0/3476857164 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:43:28.843 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:28.081+0000 7ff55ea00700 1 -- 172.21.15.97:0/3476857164 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5601071a0 con 0x7ff560106840 2024-03-27T10:43:28.843 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:28.081+0000 7ff55ea00700 1 --2- 172.21.15.97:0/3476857164 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff560106840 0x7ff560106c60 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff5500098a0 tx=0x7ff550009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c5aa9bed7139603 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:43:28.843 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:28.081+0000 7ff55d600700 1 -- 172.21.15.97:0/3476857164 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff550004160 con 0x7ff560106840 2024-03-27T10:43:28.843 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:28.081+0000 7ff55d600700 1 -- 172.21.15.97:0/3476857164 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7ff550004300 con 0x7ff560106840 2024-03-27T10:43:28.843 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:28.081+0000 7ff55d600700 1 -- 172.21.15.97:0/3476857164 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff550011a80 con 0x7ff560106840 2024-03-27T10:43:28.844 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:28.082+0000 7ff565800700 1 -- 172.21.15.97:0/3476857164 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff560106840 msgr2=0x7ff560106c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:43:28.844 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:28.082+0000 7ff565800700 1 --2- 172.21.15.97:0/3476857164 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff560106840 0x7ff560106c60 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff5500098a0 tx=0x7ff550009bb0 comp rx=0 tx=0).stop 2024-03-27T10:43:28.844 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:28.082+0000 7ff565800700 1 -- 172.21.15.97:0/3476857164 shutdown_connections 2024-03-27T10:43:28.844 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:28.082+0000 7ff565800700 1 --2- 172.21.15.97:0/3476857164 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff560106840 0x7ff560106c60 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:43:28.844 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:28.082+0000 7ff565800700 1 -- 172.21.15.97:0/3476857164 >> 172.21.15.97:0/3476857164 conn(0x7ff5601023e0 msgr2=0x7ff560104860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:43:28.844 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:28.082+0000 7ff565800700 1 -- 172.21.15.97:0/3476857164 shutdown_connections 2024-03-27T10:43:28.844 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:28.083+0000 7ff565800700 1 -- 172.21.15.97:0/3476857164 wait complete. 2024-03-27T10:43:28.844 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:28.083+0000 7ff565800700 1 Processor -- start 2024-03-27T10:43:28.844 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:28.084+0000 7ff565800700 1 -- start start 2024-03-27T10:43:28.844 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:28.084+0000 7ff565800700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff560106840 0x7ff56019e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:43:28.844 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:28.084+0000 7ff565800700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff5601081d0 con 0x7ff560106840 2024-03-27T10:43:28.844 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:28.084+0000 7ff55ea00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff560106840 0x7ff56019e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:43:28.844 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:28.084+0000 7ff55ea00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff560106840 0x7ff56019e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34176/0 (socket says 172.21.15.97:34176) 2024-03-27T10:43:28.845 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:28.084+0000 7ff55ea00700 1 -- 172.21.15.97:0/111357702 learned_addr learned my addr 172.21.15.97:0/111357702 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:43:28.845 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:28.085+0000 7ff55ea00700 1 -- 172.21.15.97:0/111357702 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff56019e6a0 con 0x7ff560106840 2024-03-27T10:43:28.845 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:28.085+0000 7ff55ea00700 1 --2- 172.21.15.97:0/111357702 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff560106840 0x7ff56019e160 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff550011a50 tx=0x7ff550004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:43:28.845 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:28.085+0000 7ff557400700 1 -- 172.21.15.97:0/111357702 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5500039f0 con 0x7ff560106840 2024-03-27T10:43:28.845 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:28.085+0000 7ff557400700 1 -- 172.21.15.97:0/111357702 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7ff550003b90 con 0x7ff560106840 2024-03-27T10:43:28.845 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:28.085+0000 7ff565800700 1 -- 172.21.15.97:0/111357702 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff56019e930 con 0x7ff560106840 2024-03-27T10:43:28.845 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:28.085+0000 7ff557400700 1 -- 172.21.15.97:0/111357702 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff55001a600 con 0x7ff560106840 2024-03-27T10:43:28.845 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:28.085+0000 7ff565800700 1 -- 172.21.15.97:0/111357702 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5601a1620 con 0x7ff560106840 2024-03-27T10:43:28.845 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:28.086+0000 7ff557400700 1 -- 172.21.15.97:0/111357702 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff55001ac60 con 0x7ff560106840 2024-03-27T10:43:28.845 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:28.086+0000 7ff557400700 1 -- 172.21.15.97:0/111357702 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff550019700 con 0x7ff560106840 2024-03-27T10:43:28.845 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:28.087+0000 7ff565800700 1 -- 172.21.15.97:0/111357702 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff560063c70 con 0x7ff560106840 2024-03-27T10:43:28.845 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:28.089+0000 7ff557400700 1 -- 172.21.15.97:0/111357702 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7ff550024a40 con 0x7ff560106840 2024-03-27T10:43:28.845 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:28.166+0000 7ff565800700 1 -- 172.21.15.97:0/111357702 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7ff56002df30 con 0x7ff560106840 2024-03-27T10:43:28.846 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:28.166+0000 7ff557400700 1 -- 172.21.15.97:0/111357702 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7ff550024440 con 0x7ff560106840 2024-03-27T10:43:28.846 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:28.168+0000 7ff565800700 1 -- 172.21.15.97:0/111357702 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff560106840 msgr2=0x7ff56019e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:43:28.846 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:28.168+0000 7ff565800700 1 --2- 172.21.15.97:0/111357702 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff560106840 0x7ff56019e160 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff550011a50 tx=0x7ff550004290 comp rx=0 tx=0).stop 2024-03-27T10:43:28.846 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:28.168+0000 7ff565800700 1 -- 172.21.15.97:0/111357702 shutdown_connections 2024-03-27T10:43:28.846 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:28.168+0000 7ff565800700 1 --2- 172.21.15.97:0/111357702 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff560106840 0x7ff56019e160 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:43:28.846 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:28.168+0000 7ff565800700 1 -- 172.21.15.97:0/111357702 >> 172.21.15.97:0/111357702 conn(0x7ff5601023e0 msgr2=0x7ff560194e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:43:28.846 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:28.168+0000 7ff565800700 1 -- 172.21.15.97:0/111357702 shutdown_connections 2024-03-27T10:43:28.846 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:28.168+0000 7ff565800700 1 -- 172.21.15.97:0/111357702 wait complete. 2024-03-27T10:43:28.847 INFO:teuthology.orchestra.run.smithi097.stdout:Restarting the monitor... 2024-03-27T10:43:29.143 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:28 smithi097 systemd[1]: Stopping Ceph mon.a for adca88c4-ec26-11ee-95d0-87774f69a715... 2024-03-27T10:43:29.644 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:29 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mon-a[31475]: 2024-03-27T10:43:29.329+0000 7f3acea00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-27T10:43:29.644 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:29 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mon-a[31475]: 2024-03-27T10:43:29.329+0000 7f3acea00700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-03-27T10:43:29.644 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:29 smithi097 podman[31733]: 2024-03-27 10:43:29.402210546 +0000 UTC m=+0.210680135 container died ed88f7f1eda92e4b43046e530d8dafaf46213e1f5b51945eff51f0c557ae0349 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240326.131558-155268c, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-27T10:43:29.999 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:29 smithi097 podman[31733]: 2024-03-27 10:43:29.832007124 +0000 UTC m=+0.640476709 container cleanup ed88f7f1eda92e4b43046e530d8dafaf46213e1f5b51945eff51f0c557ae0349 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240326.131558-155268c) 2024-03-27T10:43:29.999 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:29 smithi097 bash[31733]: ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mon-a 2024-03-27T10:43:30.394 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:30 smithi097 podman[31746]: 2024-03-27 10:43:30.024624548 +0000 UTC m=+0.611685266 container remove ed88f7f1eda92e4b43046e530d8dafaf46213e1f5b51945eff51f0c557ae0349 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mon-a, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-03-27T10:43:30.829 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:30 smithi097 systemd[1]: ceph-adca88c4-ec26-11ee-95d0-87774f69a715@mon.a.service: Deactivated successfully. 2024-03-27T10:43:30.829 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:30 smithi097 systemd[1]: Stopped Ceph mon.a for adca88c4-ec26-11ee-95d0-87774f69a715. 2024-03-27T10:43:30.829 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:30 smithi097 systemd[1]: ceph-adca88c4-ec26-11ee-95d0-87774f69a715@mon.a.service: Consumed 1.800s CPU time. 2024-03-27T10:43:30.829 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:30 smithi097 systemd[1]: Starting Ceph mon.a for adca88c4-ec26-11ee-95d0-87774f69a715... 2024-03-27T10:43:31.144 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:30 smithi097 podman[31840]: 2024-03-27 10:43:30.760705995 +0000 UTC m=+0.036854707 image pull b379bbd8da6b5d0fa9a3523941efac213b6815e1cbf05c6507954f0a3fe7a801 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 2024-03-27T10:43:34.144 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:33 smithi097 podman[31840]: 2024-03-27 10:43:33.897118764 +0000 UTC m=+3.173267489 container create 08bedf620c2406d1f45ed195cb2a75fd28bfa2ca98b7b8f0115c3ba5f467131d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240326.131558-155268c, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326) 2024-03-27T10:43:39.928 INFO:teuthology.orchestra.run.smithi097.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-03-27T10:43:40.145 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 podman[31840]: 2024-03-27 10:43:39.85619485 +0000 UTC m=+9.132343578 container init 08bedf620c2406d1f45ed195cb2a75fd28bfa2ca98b7b8f0115c3ba5f467131d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mon-a, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240326.131558-155268c, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e) 2024-03-27T10:43:40.145 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 podman[31840]: 2024-03-27 10:43:39.866630984 +0000 UTC m=+9.142779699 container start 08bedf620c2406d1f45ed195cb2a75fd28bfa2ca98b7b8f0115c3ba5f467131d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-03-27T10:43:40.145 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: set uid:gid to 167:167 (ceph:ceph) 2024-03-27T10:43:40.145 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: ceph version 19.0.0-2478-g155268c4 (155268c4e432a12433aa833f174f9fe3b1016ae0) squid (dev), process ceph-mon, pid 2 2024-03-27T10:43:40.146 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: pidfile_write: ignore empty --pid-file 2024-03-27T10:43:40.146 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: load: jerasure load: lrc 2024-03-27T10:43:40.146 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: RocksDB version: 7.9.2 2024-03-27T10:43:40.146 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Git sha 0 2024-03-27T10:43:40.146 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Compile date 2024-03-26 18:55:08 2024-03-27T10:43:40.146 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: DB SUMMARY 2024-03-27T10:43:40.146 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: DB Session ID: L5YC493V7OIHUDKL3DNK 2024-03-27T10:43:40.146 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: CURRENT file: CURRENT 2024-03-27T10:43:40.146 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: IDENTITY file: IDENTITY 2024-03-27T10:43:40.146 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-03-27T10:43:40.146 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-03-27T10:43:40.147 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 104994 ; 2024-03-27T10:43:40.147 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.error_if_exists: 0 2024-03-27T10:43:40.147 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.create_if_missing: 0 2024-03-27T10:43:40.147 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.paranoid_checks: 1 2024-03-27T10:43:40.147 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-27T10:43:40.147 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-27T10:43:40.147 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-27T10:43:40.147 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.env: 0x56102cd90c60 2024-03-27T10:43:40.147 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.fs: PosixFileSystem 2024-03-27T10:43:40.147 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.info_log: 0x56102ea97b20 2024-03-27T10:43:40.147 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.max_file_opening_threads: 16 2024-03-27T10:43:40.148 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.statistics: (nil) 2024-03-27T10:43:40.148 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.use_fsync: 0 2024-03-27T10:43:40.148 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.max_log_file_size: 0 2024-03-27T10:43:40.148 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-27T10:43:40.148 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-27T10:43:40.148 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.keep_log_file_num: 1000 2024-03-27T10:43:40.148 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.recycle_log_file_num: 0 2024-03-27T10:43:40.149 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.allow_fallocate: 1 2024-03-27T10:43:40.149 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.allow_mmap_reads: 0 2024-03-27T10:43:40.149 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.allow_mmap_writes: 0 2024-03-27T10:43:40.149 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.use_direct_reads: 0 2024-03-27T10:43:40.149 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-27T10:43:40.149 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.create_missing_column_families: 0 2024-03-27T10:43:40.149 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.db_log_dir: 2024-03-27T10:43:40.149 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.wal_dir: 2024-03-27T10:43:40.149 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-27T10:43:40.149 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-27T10:43:40.149 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-27T10:43:40.149 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-27T10:43:40.149 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-27T10:43:40.150 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-27T10:43:40.150 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.advise_random_on_open: 1 2024-03-27T10:43:40.150 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.db_write_buffer_size: 0 2024-03-27T10:43:40.150 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.write_buffer_manager: 0x56102ea681e0 2024-03-27T10:43:40.150 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-27T10:43:40.150 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-27T10:43:40.150 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-27T10:43:40.150 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.rate_limiter: (nil) 2024-03-27T10:43:40.150 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-27T10:43:40.150 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.wal_recovery_mode: 2 2024-03-27T10:43:40.150 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.enable_thread_tracking: 0 2024-03-27T10:43:40.150 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.enable_pipelined_write: 0 2024-03-27T10:43:40.151 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.unordered_write: 0 2024-03-27T10:43:40.151 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-27T10:43:40.151 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-27T10:43:40.151 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-27T10:43:40.151 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-27T10:43:40.151 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.row_cache: None 2024-03-27T10:43:40.151 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.wal_filter: None 2024-03-27T10:43:40.151 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-27T10:43:40.151 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.allow_ingest_behind: 0 2024-03-27T10:43:40.151 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.two_write_queues: 0 2024-03-27T10:43:40.151 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.manual_wal_flush: 0 2024-03-27T10:43:40.152 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.wal_compression: 0 2024-03-27T10:43:40.152 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.atomic_flush: 0 2024-03-27T10:43:40.152 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-27T10:43:40.152 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-27T10:43:40.152 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-27T10:43:40.152 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.log_readahead_size: 0 2024-03-27T10:43:40.152 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-27T10:43:40.152 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.best_efforts_recovery: 0 2024-03-27T10:43:40.152 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-27T10:43:40.152 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-27T10:43:40.152 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.allow_data_in_errors: 0 2024-03-27T10:43:40.153 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.db_host_id: __hostname__ 2024-03-27T10:43:40.153 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-27T10:43:40.153 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.max_background_jobs: 2 2024-03-27T10:43:40.153 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.max_background_compactions: -1 2024-03-27T10:43:40.153 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.max_subcompactions: 1 2024-03-27T10:43:40.153 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-27T10:43:40.153 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-27T10:43:40.153 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-27T10:43:40.153 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.max_total_wal_size: 0 2024-03-27T10:43:40.153 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-27T10:43:40.153 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-27T10:43:40.153 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-27T10:43:40.153 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-27T10:43:40.154 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.max_open_files: -1 2024-03-27T10:43:40.154 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.bytes_per_sync: 0 2024-03-27T10:43:40.154 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-27T10:43:40.154 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-27T10:43:40.154 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.compaction_readahead_size: 0 2024-03-27T10:43:40.154 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.max_background_flushes: -1 2024-03-27T10:43:40.154 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Compression algorithms supported: 2024-03-27T10:43:40.154 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-27T10:43:40.154 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: kZSTD supported: 0 2024-03-27T10:43:40.154 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: kXpressCompression supported: 0 2024-03-27T10:43:40.154 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: kLZ4HCCompression supported: 1 2024-03-27T10:43:40.154 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: kZlibCompression supported: 1 2024-03-27T10:43:40.155 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: kSnappyCompression supported: 1 2024-03-27T10:43:40.156 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: kLZ4Compression supported: 1 2024-03-27T10:43:40.156 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: kBZip2Compression supported: 0 2024-03-27T10:43:40.156 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-27T10:43:40.156 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-27T10:43:40.156 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-03-27T10:43:40.156 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-27T10:43:40.156 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-27T10:43:40.156 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.merge_operator: 2024-03-27T10:43:40.156 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.compaction_filter: None 2024-03-27T10:43:40.156 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.compaction_filter_factory: None 2024-03-27T10:43:40.157 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.sst_partitioner_factory: None 2024-03-27T10:43:40.157 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-27T10:43:40.157 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-27T10:43:40.157 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x56102ea97d00) 2024-03-27T10:43:40.157 INFO:journalctl@ceph.mon.a.smithi097.stdout: cache_index_and_filter_blocks: 1 2024-03-27T10:43:40.157 INFO:journalctl@ceph.mon.a.smithi097.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-27T10:43:40.157 INFO:journalctl@ceph.mon.a.smithi097.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-27T10:43:40.157 INFO:journalctl@ceph.mon.a.smithi097.stdout: pin_top_level_index_and_filter: 1 2024-03-27T10:43:40.157 INFO:journalctl@ceph.mon.a.smithi097.stdout: index_type: 0 2024-03-27T10:43:40.157 INFO:journalctl@ceph.mon.a.smithi097.stdout: data_block_index_type: 0 2024-03-27T10:43:40.157 INFO:journalctl@ceph.mon.a.smithi097.stdout: index_shortening: 1 2024-03-27T10:43:40.157 INFO:journalctl@ceph.mon.a.smithi097.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-27T10:43:40.157 INFO:journalctl@ceph.mon.a.smithi097.stdout: checksum: 4 2024-03-27T10:43:40.158 INFO:journalctl@ceph.mon.a.smithi097.stdout: no_block_cache: 0 2024-03-27T10:43:40.158 INFO:journalctl@ceph.mon.a.smithi097.stdout: block_cache: 0x56102ea9b090 2024-03-27T10:43:40.158 INFO:journalctl@ceph.mon.a.smithi097.stdout: block_cache_name: BinnedLRUCache 2024-03-27T10:43:40.158 INFO:journalctl@ceph.mon.a.smithi097.stdout: block_cache_options: 2024-03-27T10:43:40.158 INFO:journalctl@ceph.mon.a.smithi097.stdout: capacity : 536870912 2024-03-27T10:43:40.158 INFO:journalctl@ceph.mon.a.smithi097.stdout: num_shard_bits : 4 2024-03-27T10:43:40.158 INFO:journalctl@ceph.mon.a.smithi097.stdout: strict_capacity_limit : 0 2024-03-27T10:43:40.158 INFO:journalctl@ceph.mon.a.smithi097.stdout: high_pri_pool_ratio: 0.000 2024-03-27T10:43:40.158 INFO:journalctl@ceph.mon.a.smithi097.stdout: block_cache_compressed: (nil) 2024-03-27T10:43:40.158 INFO:journalctl@ceph.mon.a.smithi097.stdout: persistent_cache: (nil) 2024-03-27T10:43:40.158 INFO:journalctl@ceph.mon.a.smithi097.stdout: block_size: 4096 2024-03-27T10:43:40.158 INFO:journalctl@ceph.mon.a.smithi097.stdout: block_size_deviation: 10 2024-03-27T10:43:40.158 INFO:journalctl@ceph.mon.a.smithi097.stdout: block_restart_interval: 16 2024-03-27T10:43:40.158 INFO:journalctl@ceph.mon.a.smithi097.stdout: index_block_restart_interval: 1 2024-03-27T10:43:40.159 INFO:journalctl@ceph.mon.a.smithi097.stdout: metadata_block_size: 4096 2024-03-27T10:43:40.159 INFO:journalctl@ceph.mon.a.smithi097.stdout: partition_filters: 0 2024-03-27T10:43:40.159 INFO:journalctl@ceph.mon.a.smithi097.stdout: use_delta_encoding: 1 2024-03-27T10:43:40.159 INFO:journalctl@ceph.mon.a.smithi097.stdout: filter_policy: bloomfilter 2024-03-27T10:43:40.159 INFO:journalctl@ceph.mon.a.smithi097.stdout: whole_key_filtering: 1 2024-03-27T10:43:40.159 INFO:journalctl@ceph.mon.a.smithi097.stdout: verify_compression: 0 2024-03-27T10:43:40.159 INFO:journalctl@ceph.mon.a.smithi097.stdout: read_amp_bytes_per_bit: 0 2024-03-27T10:43:40.159 INFO:journalctl@ceph.mon.a.smithi097.stdout: format_version: 5 2024-03-27T10:43:40.159 INFO:journalctl@ceph.mon.a.smithi097.stdout: enable_index_compression: 1 2024-03-27T10:43:40.159 INFO:journalctl@ceph.mon.a.smithi097.stdout: block_align: 0 2024-03-27T10:43:40.160 INFO:journalctl@ceph.mon.a.smithi097.stdout: max_auto_readahead_size: 262144 2024-03-27T10:43:40.160 INFO:journalctl@ceph.mon.a.smithi097.stdout: prepopulate_block_cache: 0 2024-03-27T10:43:40.160 INFO:journalctl@ceph.mon.a.smithi097.stdout: initial_auto_readahead_size: 8192 2024-03-27T10:43:40.160 INFO:journalctl@ceph.mon.a.smithi097.stdout: num_file_reads_for_auto_readahead: 2 2024-03-27T10:43:40.160 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.write_buffer_size: 33554432 2024-03-27T10:43:40.160 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.max_write_buffer_number: 2 2024-03-27T10:43:40.160 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.compression: NoCompression 2024-03-27T10:43:40.160 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.bottommost_compression: Disabled 2024-03-27T10:43:40.160 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.prefix_extractor: nullptr 2024-03-27T10:43:40.160 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-27T10:43:40.160 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.num_levels: 7 2024-03-27T10:43:40.160 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-27T10:43:40.161 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-27T10:43:40.161 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-27T10:43:40.161 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-27T10:43:40.161 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-27T10:43:40.161 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-27T10:43:40.161 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-27T10:43:40.161 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-27T10:43:40.161 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-27T10:43:40.161 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-27T10:43:40.161 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-27T10:43:40.161 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-27T10:43:40.161 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-27T10:43:40.161 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.compression_opts.level: 32767 2024-03-27T10:43:40.161 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.compression_opts.strategy: 0 2024-03-27T10:43:40.162 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-27T10:43:40.162 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-27T10:43:40.162 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-27T10:43:40.162 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-27T10:43:40.162 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.compression_opts.enabled: false 2024-03-27T10:43:40.162 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-27T10:43:40.162 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-27T10:43:40.162 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-27T10:43:40.162 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-27T10:43:40.162 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.target_file_size_base: 67108864 2024-03-27T10:43:40.162 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-27T10:43:40.162 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-27T10:43:40.162 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-27T10:43:40.163 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-27T10:43:40.163 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-27T10:43:40.163 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-27T10:43:40.163 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-27T10:43:40.163 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-27T10:43:40.163 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-27T10:43:40.163 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-27T10:43:40.163 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-27T10:43:40.163 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-27T10:43:40.163 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-27T10:43:40.163 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-27T10:43:40.163 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.arena_block_size: 1048576 2024-03-27T10:43:40.163 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-27T10:43:40.164 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-27T10:43:40.164 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.disable_auto_compactions: 0 2024-03-27T10:43:40.164 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-27T10:43:40.164 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-27T10:43:40.164 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-27T10:43:40.164 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-27T10:43:40.164 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-27T10:43:40.164 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-27T10:43:40.164 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-27T10:43:40.164 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-27T10:43:40.164 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-27T10:43:40.164 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-27T10:43:40.164 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-03-27T10:43:40.165 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.inplace_update_support: 0 2024-03-27T10:43:40.165 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-27T10:43:40.165 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-27T10:43:40.165 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-27T10:43:40.165 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-27T10:43:40.165 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.bloom_locality: 0 2024-03-27T10:43:40.165 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.max_successive_merges: 0 2024-03-27T10:43:40.165 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-27T10:43:40.165 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.paranoid_file_checks: 0 2024-03-27T10:43:40.165 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.force_consistency_checks: 1 2024-03-27T10:43:40.165 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.report_bg_io_stats: 0 2024-03-27T10:43:40.165 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.ttl: 2592000 2024-03-27T10:43:40.165 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-27T10:43:40.166 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-27T10:43:40.166 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-27T10:43:40.166 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.enable_blob_files: false 2024-03-27T10:43:40.166 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.min_blob_size: 0 2024-03-27T10:43:40.166 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.blob_file_size: 268435456 2024-03-27T10:43:40.166 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-27T10:43:40.166 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-27T10:43:40.166 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-27T10:43:40.166 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-27T10:43:40.166 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-27T10:43:40.166 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.blob_file_starting_level: 0 2024-03-27T10:43:40.166 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-27T10:43:40.166 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-03-27T10:43:40.167 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-03-27T10:43:40.167 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 2084c121-94b5-413d-8be3-ba0c2baf60b6 2024-03-27T10:43:40.167 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711536219923434, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-03-27T10:43:40.167 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-03-27T10:43:40.167 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711536219925891, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100122, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 326, "table_properties": {"data_size": 98008, "index_size": 306, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14155, "raw_average_key_size": 49, "raw_value_size": 90344, "raw_average_value_size": 316, "num_data_blocks": 12, "num_entries": 285, "num_filter_entries": 285, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1711536219, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "2084c121-94b5-413d-8be3-ba0c2baf60b6", "db_session_id": "L5YC493V7OIHUDKL3DNK", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-03-27T10:43:40.167 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711536219926093, "job": 1, "event": "recovery_finished"} 2024-03-27T10:43:40.167 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-03-27T10:43:40.167 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-27T10:43:40.167 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x56102eb8a000 2024-03-27T10:43:40.167 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: DB pointer 0x56102eb74000 2024-03-27T10:43:40.167 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-27T10:43:40.167 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-27T10:43:40.167 INFO:journalctl@ceph.mon.a.smithi097.stdout: ** DB Stats ** 2024-03-27T10:43:40.168 INFO:journalctl@ceph.mon.a.smithi097.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-27T10:43:40.168 INFO:journalctl@ceph.mon.a.smithi097.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-27T10:43:40.168 INFO:journalctl@ceph.mon.a.smithi097.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-27T10:43:40.168 INFO:journalctl@ceph.mon.a.smithi097.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-27T10:43:40.168 INFO:journalctl@ceph.mon.a.smithi097.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-27T10:43:40.168 INFO:journalctl@ceph.mon.a.smithi097.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-27T10:43:40.168 INFO:journalctl@ceph.mon.a.smithi097.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-27T10:43:40.168 INFO:journalctl@ceph.mon.a.smithi097.stdout: 2024-03-27T10:43:40.168 INFO:journalctl@ceph.mon.a.smithi097.stdout: ** Compaction Stats [default] ** 2024-03-27T10:43:40.168 INFO:journalctl@ceph.mon.a.smithi097.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-27T10:43:40.168 INFO:journalctl@ceph.mon.a.smithi097.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-27T10:43:40.168 INFO:journalctl@ceph.mon.a.smithi097.stdout: L0 2/0 99.64 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 65.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-27T10:43:40.168 INFO:journalctl@ceph.mon.a.smithi097.stdout: Sum 2/0 99.64 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 65.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-27T10:43:40.169 INFO:journalctl@ceph.mon.a.smithi097.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 65.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-27T10:43:40.169 INFO:journalctl@ceph.mon.a.smithi097.stdout: 2024-03-27T10:43:40.169 INFO:journalctl@ceph.mon.a.smithi097.stdout: ** Compaction Stats [default] ** 2024-03-27T10:43:40.169 INFO:journalctl@ceph.mon.a.smithi097.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-27T10:43:40.169 INFO:journalctl@ceph.mon.a.smithi097.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-27T10:43:40.169 INFO:journalctl@ceph.mon.a.smithi097.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 65.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-27T10:43:40.169 INFO:journalctl@ceph.mon.a.smithi097.stdout: 2024-03-27T10:43:40.169 INFO:journalctl@ceph.mon.a.smithi097.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-27T10:43:40.169 INFO:journalctl@ceph.mon.a.smithi097.stdout: 2024-03-27T10:43:40.169 INFO:journalctl@ceph.mon.a.smithi097.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-27T10:43:40.169 INFO:journalctl@ceph.mon.a.smithi097.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-27T10:43:40.169 INFO:journalctl@ceph.mon.a.smithi097.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-27T10:43:40.169 INFO:journalctl@ceph.mon.a.smithi097.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-27T10:43:40.170 INFO:journalctl@ceph.mon.a.smithi097.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-27T10:43:40.170 INFO:journalctl@ceph.mon.a.smithi097.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-27T10:43:40.170 INFO:journalctl@ceph.mon.a.smithi097.stdout: Cumulative compaction: 0.00 GB write, 10.53 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-27T10:43:40.170 INFO:journalctl@ceph.mon.a.smithi097.stdout: Interval compaction: 0.00 GB write, 10.53 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-27T10:43:40.170 INFO:journalctl@ceph.mon.a.smithi097.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-27T10:43:40.170 INFO:journalctl@ceph.mon.a.smithi097.stdout: Block cache BinnedLRUCache@0x56102ea9b090#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-03-27T10:43:40.170 INFO:journalctl@ceph.mon.a.smithi097.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-03-27T10:43:40.170 INFO:journalctl@ceph.mon.a.smithi097.stdout: 2024-03-27T10:43:40.170 INFO:journalctl@ceph.mon.a.smithi097.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-27T10:43:40.170 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: starting mon.a rank 0 at public addrs [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] at bind addrs [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid adca88c4-ec26-11ee-95d0-87774f69a715 2024-03-27T10:43:40.170 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: mon.a@-1(???) e1 preinit fsid adca88c4-ec26-11ee-95d0-87774f69a715 2024-03-27T10:43:40.170 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: mon.a@-1(???).mds e1 new map 2024-03-27T10:43:40.171 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: mon.a@-1(???).mds e1 print_map 2024-03-27T10:43:40.171 INFO:journalctl@ceph.mon.a.smithi097.stdout: e1 2024-03-27T10:43:40.171 INFO:journalctl@ceph.mon.a.smithi097.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-27T10:43:40.171 INFO:journalctl@ceph.mon.a.smithi097.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-03-27T10:43:40.171 INFO:journalctl@ceph.mon.a.smithi097.stdout: legacy client fscid: -1 2024-03-27T10:43:40.171 INFO:journalctl@ceph.mon.a.smithi097.stdout: 2024-03-27T10:43:40.171 INFO:journalctl@ceph.mon.a.smithi097.stdout: No filesystems configured 2024-03-27T10:43:40.171 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-27T10:43:40.171 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-27T10:43:40.171 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-27T10:43:40.171 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-27T10:43:40.171 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-03-27T10:43:40.171 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 bash[31840]: 08bedf620c2406d1f45ed195cb2a75fd28bfa2ca98b7b8f0115c3ba5f467131d 2024-03-27T10:43:40.171 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 systemd[1]: Started Ceph mon.a for adca88c4-ec26-11ee-95d0-87774f69a715. 2024-03-27T10:43:40.172 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-27T10:43:40.172 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: monmap epoch 1 2024-03-27T10:43:40.172 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: fsid adca88c4-ec26-11ee-95d0-87774f69a715 2024-03-27T10:43:40.172 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: last_changed 2024-03-27T10:43:20.535207+0000 2024-03-27T10:43:40.172 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: created 2024-03-27T10:43:20.535207+0000 2024-03-27T10:43:40.172 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: min_mon_release 19 (squid) 2024-03-27T10:43:40.172 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: election_strategy: 1 2024-03-27T10:43:40.172 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: 0: [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] mon.a 2024-03-27T10:43:40.172 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: fsmap 2024-03-27T10:43:40.172 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: osdmap e1: 0 total, 0 up, 0 in 2024-03-27T10:43:40.172 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:39 smithi097 ceph-mon[31859]: mgrmap e1: no daemons active 2024-03-27T10:43:41.278 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:40.531+0000 7fe55cc00700 1 Processor -- start 2024-03-27T10:43:41.278 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:40.531+0000 7fe55cc00700 1 -- start start 2024-03-27T10:43:41.278 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:40.532+0000 7fe55cc00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe55810ecf0 0x7fe5581110e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:43:41.278 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:40.532+0000 7fe55cc00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe558111620 con 0x7fe55810ecf0 2024-03-27T10:43:41.278 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:40.532+0000 7fe557400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe55810ecf0 0x7fe5581110e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:43:41.278 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:40.533+0000 7fe557400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe55810ecf0 0x7fe5581110e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37008/0 (socket says 172.21.15.97:37008) 2024-03-27T10:43:41.278 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:40.533+0000 7fe557400700 1 -- 172.21.15.97:0/854379837 learned_addr learned my addr 172.21.15.97:0/854379837 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:43:41.278 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:40.533+0000 7fe557400700 1 -- 172.21.15.97:0/854379837 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5581117a0 con 0x7fe55810ecf0 2024-03-27T10:43:41.278 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:40.533+0000 7fe557400700 1 --2- 172.21.15.97:0/854379837 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe55810ecf0 0x7fe5581110e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fe5480098a0 tx=0x7fe548009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6711eb51f6c47a6 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:43:41.278 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:40.534+0000 7fe556000700 1 -- 172.21.15.97:0/854379837 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe548004160 con 0x7fe55810ecf0 2024-03-27T10:43:41.278 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:40.534+0000 7fe556000700 1 -- 172.21.15.97:0/854379837 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fe548004300 con 0x7fe55810ecf0 2024-03-27T10:43:41.279 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:40.534+0000 7fe556000700 1 -- 172.21.15.97:0/854379837 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe548011a80 con 0x7fe55810ecf0 2024-03-27T10:43:41.279 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:40.534+0000 7fe55cc00700 1 -- 172.21.15.97:0/854379837 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe55810ecf0 msgr2=0x7fe5581110e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:43:41.279 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:40.534+0000 7fe55cc00700 1 --2- 172.21.15.97:0/854379837 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe55810ecf0 0x7fe5581110e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fe5480098a0 tx=0x7fe548009bb0 comp rx=0 tx=0).stop 2024-03-27T10:43:41.279 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:40.535+0000 7fe55cc00700 1 -- 172.21.15.97:0/854379837 shutdown_connections 2024-03-27T10:43:41.279 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:40.535+0000 7fe55cc00700 1 --2- 172.21.15.97:0/854379837 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe55810ecf0 0x7fe5581110e0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:43:41.279 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:40.535+0000 7fe55cc00700 1 -- 172.21.15.97:0/854379837 >> 172.21.15.97:0/854379837 conn(0x7fe55807d410 msgr2=0x7fe55807f830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:43:41.279 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:40.535+0000 7fe55cc00700 1 -- 172.21.15.97:0/854379837 shutdown_connections 2024-03-27T10:43:41.279 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:40.535+0000 7fe55cc00700 1 -- 172.21.15.97:0/854379837 wait complete. 2024-03-27T10:43:41.279 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:40.536+0000 7fe55cc00700 1 Processor -- start 2024-03-27T10:43:41.279 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:40.536+0000 7fe55cc00700 1 -- start start 2024-03-27T10:43:41.279 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:40.536+0000 7fe55cc00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe55810ecf0 0x7fe5581a23c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:43:41.279 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:40.536+0000 7fe55cc00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe558112770 con 0x7fe55810ecf0 2024-03-27T10:43:41.280 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:40.536+0000 7fe557400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe55810ecf0 0x7fe5581a23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:43:41.280 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:40.536+0000 7fe557400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe55810ecf0 0x7fe5581a23c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37014/0 (socket says 172.21.15.97:37014) 2024-03-27T10:43:41.280 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:40.536+0000 7fe557400700 1 -- 172.21.15.97:0/505299394 learned_addr learned my addr 172.21.15.97:0/505299394 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:43:41.280 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:40.537+0000 7fe557400700 1 -- 172.21.15.97:0/505299394 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5581a2900 con 0x7fe55810ecf0 2024-03-27T10:43:41.280 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:40.537+0000 7fe557400700 1 --2- 172.21.15.97:0/505299394 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe55810ecf0 0x7fe5581a23c0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fe548011a50 tx=0x7fe548004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:43:41.280 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:40.537+0000 7fe54fe00700 1 -- 172.21.15.97:0/505299394 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe5480039f0 con 0x7fe55810ecf0 2024-03-27T10:43:41.280 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:40.537+0000 7fe54fe00700 1 -- 172.21.15.97:0/505299394 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fe548003b90 con 0x7fe55810ecf0 2024-03-27T10:43:41.280 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:40.537+0000 7fe55cc00700 1 -- 172.21.15.97:0/505299394 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5581a2b90 con 0x7fe55810ecf0 2024-03-27T10:43:41.280 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:40.537+0000 7fe54fe00700 1 -- 172.21.15.97:0/505299394 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe54801a600 con 0x7fe55810ecf0 2024-03-27T10:43:41.280 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:40.538+0000 7fe55cc00700 1 -- 172.21.15.97:0/505299394 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5581a5880 con 0x7fe55810ecf0 2024-03-27T10:43:41.280 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:40.539+0000 7fe54fe00700 1 -- 172.21.15.97:0/505299394 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe54801a920 con 0x7fe55810ecf0 2024-03-27T10:43:41.280 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:40.539+0000 7fe54fe00700 1 -- 172.21.15.97:0/505299394 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe5480193c0 con 0x7fe55810ecf0 2024-03-27T10:43:41.281 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:40.539+0000 7fe55cc00700 1 -- 172.21.15.97:0/505299394 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe558063c70 con 0x7fe55810ecf0 2024-03-27T10:43:41.281 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:40.542+0000 7fe54fe00700 1 -- 172.21.15.97:0/505299394 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fe548024a40 con 0x7fe55810ecf0 2024-03-27T10:43:41.281 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:40.617+0000 7fe55cc00700 1 -- 172.21.15.97:0/505299394 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7fe5581a5a50 con 0x7fe55810ecf0 2024-03-27T10:43:41.281 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:40.621+0000 7fe54fe00700 1 -- 172.21.15.97:0/505299394 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7fe548032030 con 0x7fe55810ecf0 2024-03-27T10:43:41.281 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:40.623+0000 7fe55cc00700 1 -- 172.21.15.97:0/505299394 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe55810ecf0 msgr2=0x7fe5581a23c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:43:41.281 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:40.623+0000 7fe55cc00700 1 --2- 172.21.15.97:0/505299394 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe55810ecf0 0x7fe5581a23c0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fe548011a50 tx=0x7fe548004290 comp rx=0 tx=0).stop 2024-03-27T10:43:41.281 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:40.623+0000 7fe55cc00700 1 -- 172.21.15.97:0/505299394 shutdown_connections 2024-03-27T10:43:41.281 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:40.623+0000 7fe55cc00700 1 --2- 172.21.15.97:0/505299394 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe55810ecf0 0x7fe5581a23c0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:43:41.281 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:40.623+0000 7fe55cc00700 1 -- 172.21.15.97:0/505299394 >> 172.21.15.97:0/505299394 conn(0x7fe55807d410 msgr2=0x7fe55807e010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:43:41.281 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:40.623+0000 7fe55cc00700 1 -- 172.21.15.97:0/505299394 shutdown_connections 2024-03-27T10:43:41.281 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:40.623+0000 7fe55cc00700 1 -- 172.21.15.97:0/505299394 wait complete. 2024-03-27T10:43:41.281 INFO:teuthology.orchestra.run.smithi097.stdout:Wrote config to /etc/ceph/ceph.conf 2024-03-27T10:43:41.299 INFO:teuthology.orchestra.run.smithi097.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-03-27T10:43:41.300 INFO:teuthology.orchestra.run.smithi097.stdout:Creating mgr... 2024-03-27T10:43:41.301 INFO:teuthology.orchestra.run.smithi097.stdout:Verifying port 0.0.0.0:9283 ... 2024-03-27T10:43:41.302 INFO:teuthology.orchestra.run.smithi097.stdout:Verifying port 0.0.0.0:8765 ... 2024-03-27T10:43:41.721 INFO:teuthology.orchestra.run.smithi097.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-adca88c4-ec26-11ee-95d0-87774f69a715@mgr.x 2024-03-27T10:43:41.722 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stderr Failed to reset failed state of unit ceph-adca88c4-ec26-11ee-95d0-87774f69a715@mgr.x.service: Unit ceph-adca88c4-ec26-11ee-95d0-87774f69a715@mgr.x.service not loaded. 2024-03-27T10:43:41.893 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:41 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/505299394' entity='client.admin' 2024-03-27T10:43:41.975 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-adca88c4-ec26-11ee-95d0-87774f69a715.target.wants/ceph-adca88c4-ec26-11ee-95d0-87774f69a715@mgr.x.service → /etc/systemd/system/ceph-adca88c4-ec26-11ee-95d0-87774f69a715@.service. 2024-03-27T10:43:42.362 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:43:42 smithi097 systemd[1]: Starting Ceph mgr.x for adca88c4-ec26-11ee-95d0-87774f69a715... 2024-03-27T10:43:42.624 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:43:42 smithi097 podman[32071]: 2024-03-27 10:43:42.386370103 +0000 UTC m=+0.115988912 container create 662f8153b4104b0eb187f8544269161a090eebbf78819b65b358ee399dcb4086 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.license=GPLv2) 2024-03-27T10:43:42.624 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:43:42 smithi097 podman[32071]: 2024-03-27 10:43:42.305296771 +0000 UTC m=+0.034915546 image pull b379bbd8da6b5d0fa9a3523941efac213b6815e1cbf05c6507954f0a3fe7a801 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 2024-03-27T10:43:42.624 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:43:42 smithi097 podman[32071]: 2024-03-27 10:43:42.647936349 +0000 UTC m=+0.377555131 container init 662f8153b4104b0eb187f8544269161a090eebbf78819b65b358ee399dcb4086 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, RELEASE=wip-rishabh-testing-20240326.131558-155268c, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326) 2024-03-27T10:43:42.739 INFO:teuthology.orchestra.run.smithi097.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-27T10:43:42.740 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stdout disabled 2024-03-27T10:43:42.754 INFO:teuthology.orchestra.run.smithi097.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-27T10:43:42.754 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stdout inactive 2024-03-27T10:43:42.754 INFO:teuthology.orchestra.run.smithi097.stdout:firewalld.service is not enabled 2024-03-27T10:43:42.755 INFO:teuthology.orchestra.run.smithi097.stdout:Not possible to enable service . firewalld.service is not available 2024-03-27T10:43:42.772 INFO:teuthology.orchestra.run.smithi097.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-27T10:43:42.773 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stdout disabled 2024-03-27T10:43:42.787 INFO:teuthology.orchestra.run.smithi097.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-27T10:43:42.788 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stdout inactive 2024-03-27T10:43:42.788 INFO:teuthology.orchestra.run.smithi097.stdout:firewalld.service is not enabled 2024-03-27T10:43:42.788 INFO:teuthology.orchestra.run.smithi097.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-03-27T10:43:42.788 INFO:teuthology.orchestra.run.smithi097.stdout:Waiting for mgr to start... 2024-03-27T10:43:42.788 INFO:teuthology.orchestra.run.smithi097.stdout:Waiting for mgr... 2024-03-27T10:43:42.894 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:43:42 smithi097 podman[32071]: 2024-03-27 10:43:42.657085531 +0000 UTC m=+0.386704310 container start 662f8153b4104b0eb187f8544269161a090eebbf78819b65b358ee399dcb4086 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240326.131558-155268c, CEPH_POINT_RELEASE=, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e) 2024-03-27T10:43:42.894 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:43:42 smithi097 bash[32071]: 662f8153b4104b0eb187f8544269161a090eebbf78819b65b358ee399dcb4086 2024-03-27T10:43:42.894 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:43:42 smithi097 systemd[1]: Started Ceph mgr.x for adca88c4-ec26-11ee-95d0-87774f69a715. 2024-03-27T10:43:43.535 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:43:43 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: 2024-03-27T10:43:43.439+0000 7fdf3acf1200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-27T10:43:43.893 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:43 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/537215747' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-27T10:43:43.894 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:43:43 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: 2024-03-27T10:43:43.567+0000 7fdf3acf1200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-27T10:43:44.249 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 2024-03-27T10:43:44.260 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout { 2024-03-27T10:43:44.260 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "fsid": "adca88c4-ec26-11ee-95d0-87774f69a715", 2024-03-27T10:43:44.260 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "health": { 2024-03-27T10:43:44.260 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-27T10:43:44.261 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-27T10:43:44.261 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-27T10:43:44.261 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-03-27T10:43:44.261 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-27T10:43:44.261 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-27T10:43:44.261 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 0 2024-03-27T10:43:44.261 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout ], 2024-03-27T10:43:44.261 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-27T10:43:44.261 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "a" 2024-03-27T10:43:44.261 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout ], 2024-03-27T10:43:44.261 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-03-27T10:43:44.261 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-27T10:43:44.262 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-27T10:43:44.262 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-27T10:43:44.262 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-27T10:43:44.262 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-03-27T10:43:44.262 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-27T10:43:44.262 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-27T10:43:44.262 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-27T10:43:44.262 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-27T10:43:44.262 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-27T10:43:44.262 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-27T10:43:44.262 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-27T10:43:44.262 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-27T10:43:44.262 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-03-27T10:43:44.263 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-27T10:43:44.263 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-27T10:43:44.263 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-27T10:43:44.263 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-27T10:43:44.263 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-27T10:43:44.263 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-27T10:43:44.263 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-27T10:43:44.263 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-27T10:43:44.263 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-27T10:43:44.263 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-03-27T10:43:44.263 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-27T10:43:44.263 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-27T10:43:44.263 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-27T10:43:44.263 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-27T10:43:44.264 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-03-27T10:43:44.264 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-27T10:43:44.264 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-27T10:43:44.264 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-27T10:43:44.264 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-27T10:43:44.264 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-27T10:43:44.264 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-27T10:43:44.264 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "restful" 2024-03-27T10:43:44.264 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout ], 2024-03-27T10:43:44.264 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-27T10:43:44.264 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-03-27T10:43:44.264 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-27T10:43:44.264 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-27T10:43:44.264 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "modified": "2024-03-27T10:43:24.681303+0000", 2024-03-27T10:43:44.265 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-27T10:43:44.265 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-03-27T10:43:44.265 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-27T10:43:44.265 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout } 2024-03-27T10:43:44.265 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:43.452+0000 7f42efe00700 1 Processor -- start 2024-03-27T10:43:44.265 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:43.452+0000 7f42efe00700 1 -- start start 2024-03-27T10:43:44.265 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:43.452+0000 7f42efe00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f42e8107fe0 0x7f42e8108400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:43:44.265 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:43.452+0000 7f42efe00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f42e81089d0 con 0x7f42e8107fe0 2024-03-27T10:43:44.265 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:43.453+0000 7f42ed400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f42e8107fe0 0x7f42e8108400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:43:44.265 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:43.453+0000 7f42ed400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f42e8107fe0 0x7f42e8108400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37036/0 (socket says 172.21.15.97:37036) 2024-03-27T10:43:44.265 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:43.453+0000 7f42ed400700 1 -- 172.21.15.97:0/3692497348 learned_addr learned my addr 172.21.15.97:0/3692497348 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:43:44.265 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:43.454+0000 7f42ed400700 1 -- 172.21.15.97:0/3692497348 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42e8109190 con 0x7f42e8107fe0 2024-03-27T10:43:44.265 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:43.454+0000 7f42ed400700 1 --2- 172.21.15.97:0/3692497348 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f42e8107fe0 0x7f42e8108400 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f42dc0098a0 tx=0x7f42dc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30b26f09e266ec24 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:43:44.266 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:43.454+0000 7f42e7e00700 1 -- 172.21.15.97:0/3692497348 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f42dc004160 con 0x7f42e8107fe0 2024-03-27T10:43:44.266 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:43.454+0000 7f42e7e00700 1 -- 172.21.15.97:0/3692497348 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f42dc004300 con 0x7f42e8107fe0 2024-03-27T10:43:44.266 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:43.454+0000 7f42e7e00700 1 -- 172.21.15.97:0/3692497348 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f42dc011a80 con 0x7f42e8107fe0 2024-03-27T10:43:44.266 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:43.455+0000 7f42efe00700 1 -- 172.21.15.97:0/3692497348 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f42e8107fe0 msgr2=0x7f42e8108400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:43:44.266 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:43.455+0000 7f42efe00700 1 --2- 172.21.15.97:0/3692497348 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f42e8107fe0 0x7f42e8108400 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f42dc0098a0 tx=0x7f42dc009bb0 comp rx=0 tx=0).stop 2024-03-27T10:43:44.266 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:43.455+0000 7f42efe00700 1 -- 172.21.15.97:0/3692497348 shutdown_connections 2024-03-27T10:43:44.266 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:43.455+0000 7f42efe00700 1 --2- 172.21.15.97:0/3692497348 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f42e8107fe0 0x7f42e8108400 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:43:44.266 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:43.455+0000 7f42efe00700 1 -- 172.21.15.97:0/3692497348 >> 172.21.15.97:0/3692497348 conn(0x7f42e8103540 msgr2=0x7f42e81059c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:43:44.266 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:43.456+0000 7f42efe00700 1 -- 172.21.15.97:0/3692497348 shutdown_connections 2024-03-27T10:43:44.266 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:43.456+0000 7f42efe00700 1 -- 172.21.15.97:0/3692497348 wait complete. 2024-03-27T10:43:44.266 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:43.457+0000 7f42efe00700 1 Processor -- start 2024-03-27T10:43:44.266 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:43.457+0000 7f42efe00700 1 -- start start 2024-03-27T10:43:44.266 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:43.458+0000 7f42efe00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f42e8107fe0 0x7f42e807db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:43:44.267 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:43.458+0000 7f42efe00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f42e810c540 con 0x7f42e8107fe0 2024-03-27T10:43:44.267 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:43.458+0000 7f42ed400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f42e8107fe0 0x7f42e807db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:43:44.267 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:43.458+0000 7f42ed400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f42e8107fe0 0x7f42e807db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37046/0 (socket says 172.21.15.97:37046) 2024-03-27T10:43:44.267 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:43.458+0000 7f42ed400700 1 -- 172.21.15.97:0/537215747 learned_addr learned my addr 172.21.15.97:0/537215747 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:43:44.267 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:43.459+0000 7f42ed400700 1 -- 172.21.15.97:0/537215747 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42e807e0d0 con 0x7f42e8107fe0 2024-03-27T10:43:44.267 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:43.459+0000 7f42ed400700 1 --2- 172.21.15.97:0/537215747 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f42e8107fe0 0x7f42e807db90 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f42dc006af0 tx=0x7f42dc011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:43:44.267 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:43.459+0000 7f42e6000700 1 -- 172.21.15.97:0/537215747 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f42dc003930 con 0x7f42e8107fe0 2024-03-27T10:43:44.267 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:43.459+0000 7f42e6000700 1 -- 172.21.15.97:0/537215747 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f42dc003ad0 con 0x7f42e8107fe0 2024-03-27T10:43:44.267 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:43.459+0000 7f42efe00700 1 -- 172.21.15.97:0/537215747 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42e807c2c0 con 0x7f42e8107fe0 2024-03-27T10:43:44.267 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:43.460+0000 7f42e6000700 1 -- 172.21.15.97:0/537215747 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f42dc0195e0 con 0x7f42e8107fe0 2024-03-27T10:43:44.267 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:43.460+0000 7f42efe00700 1 -- 172.21.15.97:0/537215747 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42e807c800 con 0x7f42e8107fe0 2024-03-27T10:43:44.267 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:43.460+0000 7f42e6000700 1 -- 172.21.15.97:0/537215747 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f42dc019ca0 con 0x7f42e8107fe0 2024-03-27T10:43:44.267 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:43.460+0000 7f42e6000700 1 -- 172.21.15.97:0/537215747 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f42dc01a7c0 con 0x7f42e8107fe0 2024-03-27T10:43:44.267 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:43.461+0000 7f42efe00700 1 -- 172.21.15.97:0/537215747 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42e8063c70 con 0x7f42e8107fe0 2024-03-27T10:43:44.268 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:43.464+0000 7f42e6000700 1 -- 172.21.15.97:0/537215747 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f42dc01aa50 con 0x7f42e8107fe0 2024-03-27T10:43:44.268 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:43.542+0000 7f42efe00700 1 -- 172.21.15.97:0/537215747 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f42e807d100 con 0x7f42e8107fe0 2024-03-27T10:43:44.268 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:43.542+0000 7f42e6000700 1 -- 172.21.15.97:0/537215747 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f42dc00b7a0 con 0x7f42e8107fe0 2024-03-27T10:43:44.268 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:43.544+0000 7f42efe00700 1 -- 172.21.15.97:0/537215747 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f42e8107fe0 msgr2=0x7f42e807db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:43:44.268 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:43.544+0000 7f42efe00700 1 --2- 172.21.15.97:0/537215747 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f42e8107fe0 0x7f42e807db90 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f42dc006af0 tx=0x7f42dc011b70 comp rx=0 tx=0).stop 2024-03-27T10:43:44.268 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:43.544+0000 7f42efe00700 1 -- 172.21.15.97:0/537215747 shutdown_connections 2024-03-27T10:43:44.268 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:43.544+0000 7f42efe00700 1 --2- 172.21.15.97:0/537215747 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f42e8107fe0 0x7f42e807db90 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:43:44.268 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:43.544+0000 7f42efe00700 1 -- 172.21.15.97:0/537215747 >> 172.21.15.97:0/537215747 conn(0x7f42e8103540 msgr2=0x7f42e8104180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:43:44.268 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:43.544+0000 7f42efe00700 1 -- 172.21.15.97:0/537215747 shutdown_connections 2024-03-27T10:43:44.268 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:43.544+0000 7f42efe00700 1 -- 172.21.15.97:0/537215747 wait complete. 2024-03-27T10:43:44.268 INFO:teuthology.orchestra.run.smithi097.stdout:mgr not available, waiting (1/15)... 2024-03-27T10:43:44.644 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:43:44 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: 2024-03-27T10:43:44.382+0000 7fdf3acf1200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-27T10:43:44.644 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:43:44 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: 2024-03-27T10:43:44.500+0000 7fdf3acf1200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-27T10:43:45.144 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:43:44 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: 2024-03-27T10:43:44.691+0000 7fdf3acf1200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-27T10:43:45.235 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:43:44 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: 2024-03-27T10:43:44.813+0000 7fdf3acf1200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-27T10:43:45.235 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:43:44 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: 2024-03-27T10:43:44.921+0000 7fdf3acf1200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-27T10:43:45.594 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:43:45 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: 2024-03-27T10:43:45.249+0000 7fdf3acf1200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-27T10:43:45.594 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:43:45 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: 2024-03-27T10:43:45.388+0000 7fdf3acf1200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-27T10:43:45.594 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:43:45 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: 2024-03-27T10:43:45.497+0000 7fdf3acf1200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-27T10:43:45.893 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:43:45 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: 2024-03-27T10:43:45.618+0000 7fdf3acf1200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-27T10:43:45.894 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:43:45 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: 2024-03-27T10:43:45.832+0000 7fdf3acf1200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-27T10:43:47.334 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:47 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/585010865' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-27T10:43:47.334 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:43:47 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: 2024-03-27T10:43:47.181+0000 7fdf3acf1200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-27T10:43:47.335 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:43:47 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: 2024-03-27T10:43:47.359+0000 7fdf3acf1200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-27T10:43:47.859 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 2024-03-27T10:43:47.860 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout { 2024-03-27T10:43:47.860 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "fsid": "adca88c4-ec26-11ee-95d0-87774f69a715", 2024-03-27T10:43:47.860 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "health": { 2024-03-27T10:43:47.860 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-27T10:43:47.860 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-27T10:43:47.860 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-27T10:43:47.860 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-03-27T10:43:47.860 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-27T10:43:47.860 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-27T10:43:47.860 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 0 2024-03-27T10:43:47.860 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout ], 2024-03-27T10:43:47.860 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-27T10:43:47.860 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "a" 2024-03-27T10:43:47.861 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout ], 2024-03-27T10:43:47.861 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "quorum_age": 7, 2024-03-27T10:43:47.861 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-27T10:43:47.861 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-27T10:43:47.861 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-27T10:43:47.861 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-27T10:43:47.861 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-03-27T10:43:47.861 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-27T10:43:47.861 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-27T10:43:47.861 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-27T10:43:47.861 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-27T10:43:47.861 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-27T10:43:47.861 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-27T10:43:47.861 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-27T10:43:47.862 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-27T10:43:47.862 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-03-27T10:43:47.862 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-27T10:43:47.862 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-27T10:43:47.862 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-27T10:43:47.862 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-27T10:43:47.862 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-27T10:43:47.862 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-27T10:43:47.862 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-27T10:43:47.862 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-27T10:43:47.862 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-27T10:43:47.862 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-03-27T10:43:47.862 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-27T10:43:47.863 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-27T10:43:47.863 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-27T10:43:47.863 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-27T10:43:47.863 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-03-27T10:43:47.863 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-27T10:43:47.863 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-27T10:43:47.863 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-27T10:43:47.863 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-27T10:43:47.863 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-27T10:43:47.863 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-27T10:43:47.863 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "restful" 2024-03-27T10:43:47.863 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout ], 2024-03-27T10:43:47.863 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-27T10:43:47.864 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-03-27T10:43:47.864 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-27T10:43:47.864 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-27T10:43:47.864 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "modified": "2024-03-27T10:43:24.681303+0000", 2024-03-27T10:43:47.864 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-27T10:43:47.864 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-03-27T10:43:47.864 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-27T10:43:47.864 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout } 2024-03-27T10:43:47.864 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:46.951+0000 7fb608a00700 1 Processor -- start 2024-03-27T10:43:47.864 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:46.951+0000 7fb608a00700 1 -- start start 2024-03-27T10:43:47.864 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:46.951+0000 7fb608a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb60407dee0 0x7fb60407c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:43:47.864 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:46.951+0000 7fb608a00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb60407c910 con 0x7fb60407dee0 2024-03-27T10:43:47.864 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:46.952+0000 7fb601e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb60407dee0 0x7fb60407c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:43:47.865 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:46.952+0000 7fb601e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb60407dee0 0x7fb60407c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37062/0 (socket says 172.21.15.97:37062) 2024-03-27T10:43:47.865 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:46.952+0000 7fb601e00700 1 -- 172.21.15.97:0/858427028 learned_addr learned my addr 172.21.15.97:0/858427028 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:43:47.865 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:46.952+0000 7fb601e00700 1 -- 172.21.15.97:0/858427028 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb60407ca90 con 0x7fb60407dee0 2024-03-27T10:43:47.865 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:46.953+0000 7fb601e00700 1 --2- 172.21.15.97:0/858427028 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb60407dee0 0x7fb60407c340 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb5f40098a0 tx=0x7fb5f4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51a29717cc524f46 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:43:47.865 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:46.953+0000 7fb600a00700 1 -- 172.21.15.97:0/858427028 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb5f4004160 con 0x7fb60407dee0 2024-03-27T10:43:47.865 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:46.953+0000 7fb600a00700 1 -- 172.21.15.97:0/858427028 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fb5f4004300 con 0x7fb60407dee0 2024-03-27T10:43:47.865 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:46.953+0000 7fb600a00700 1 -- 172.21.15.97:0/858427028 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb5f4011a80 con 0x7fb60407dee0 2024-03-27T10:43:47.865 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:46.954+0000 7fb608a00700 1 -- 172.21.15.97:0/858427028 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb60407dee0 msgr2=0x7fb60407c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:43:47.865 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:46.954+0000 7fb608a00700 1 --2- 172.21.15.97:0/858427028 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb60407dee0 0x7fb60407c340 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb5f40098a0 tx=0x7fb5f4009bb0 comp rx=0 tx=0).stop 2024-03-27T10:43:47.865 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:46.954+0000 7fb608a00700 1 -- 172.21.15.97:0/858427028 shutdown_connections 2024-03-27T10:43:47.865 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:46.954+0000 7fb608a00700 1 --2- 172.21.15.97:0/858427028 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb60407dee0 0x7fb60407c340 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:43:47.865 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:46.954+0000 7fb608a00700 1 -- 172.21.15.97:0/858427028 >> 172.21.15.97:0/858427028 conn(0x7fb604103570 msgr2=0x7fb6041059d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:43:47.866 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:46.954+0000 7fb608a00700 1 -- 172.21.15.97:0/858427028 shutdown_connections 2024-03-27T10:43:47.866 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:46.954+0000 7fb608a00700 1 -- 172.21.15.97:0/858427028 wait complete. 2024-03-27T10:43:47.866 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:46.955+0000 7fb608a00700 1 Processor -- start 2024-03-27T10:43:47.866 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:46.955+0000 7fb608a00700 1 -- start start 2024-03-27T10:43:47.866 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:46.955+0000 7fb608a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb60407dee0 0x7fb60419e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:43:47.866 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:46.955+0000 7fb608a00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb60410e520 con 0x7fb60407dee0 2024-03-27T10:43:47.866 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:46.956+0000 7fb601e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb60407dee0 0x7fb60419e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:43:47.866 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:46.956+0000 7fb601e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb60407dee0 0x7fb60419e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37078/0 (socket says 172.21.15.97:37078) 2024-03-27T10:43:47.866 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:46.956+0000 7fb601e00700 1 -- 172.21.15.97:0/585010865 learned_addr learned my addr 172.21.15.97:0/585010865 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:43:47.866 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:46.956+0000 7fb601e00700 1 -- 172.21.15.97:0/585010865 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb60419e690 con 0x7fb60407dee0 2024-03-27T10:43:47.866 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:46.956+0000 7fb601e00700 1 --2- 172.21.15.97:0/585010865 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb60407dee0 0x7fb60419e150 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb5f4000c00 tx=0x7fb5f4004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:43:47.866 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:46.957+0000 7fb5faa00700 1 -- 172.21.15.97:0/585010865 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb5f40039c0 con 0x7fb60407dee0 2024-03-27T10:43:47.866 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:46.957+0000 7fb5faa00700 1 -- 172.21.15.97:0/585010865 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fb5f4003b60 con 0x7fb60407dee0 2024-03-27T10:43:47.867 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:46.957+0000 7fb608a00700 1 -- 172.21.15.97:0/585010865 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb60419e920 con 0x7fb60407dee0 2024-03-27T10:43:47.867 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:46.957+0000 7fb5faa00700 1 -- 172.21.15.97:0/585010865 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb5f401a6f0 con 0x7fb60407dee0 2024-03-27T10:43:47.867 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:46.957+0000 7fb608a00700 1 -- 172.21.15.97:0/585010865 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6041a1610 con 0x7fb60407dee0 2024-03-27T10:43:47.867 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:46.957+0000 7fb5faa00700 1 -- 172.21.15.97:0/585010865 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb5f402d3f0 con 0x7fb60407dee0 2024-03-27T10:43:47.867 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:46.957+0000 7fb5faa00700 1 -- 172.21.15.97:0/585010865 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb5f4019700 con 0x7fb60407dee0 2024-03-27T10:43:47.867 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:46.958+0000 7fb608a00700 1 -- 172.21.15.97:0/585010865 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb604063c70 con 0x7fb60407dee0 2024-03-27T10:43:47.867 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:46.961+0000 7fb5faa00700 1 -- 172.21.15.97:0/585010865 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fb5f4024a40 con 0x7fb60407dee0 2024-03-27T10:43:47.867 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:47.040+0000 7fb608a00700 1 -- 172.21.15.97:0/585010865 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fb6041a1b50 con 0x7fb60407dee0 2024-03-27T10:43:47.867 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:47.040+0000 7fb5faa00700 1 -- 172.21.15.97:0/585010865 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fb5f4032030 con 0x7fb60407dee0 2024-03-27T10:43:47.867 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:47.042+0000 7fb608a00700 1 -- 172.21.15.97:0/585010865 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb60407dee0 msgr2=0x7fb60419e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:43:47.867 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:47.042+0000 7fb608a00700 1 --2- 172.21.15.97:0/585010865 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb60407dee0 0x7fb60419e150 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb5f4000c00 tx=0x7fb5f4004290 comp rx=0 tx=0).stop 2024-03-27T10:43:47.867 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:47.042+0000 7fb608a00700 1 -- 172.21.15.97:0/585010865 shutdown_connections 2024-03-27T10:43:47.868 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:47.042+0000 7fb608a00700 1 --2- 172.21.15.97:0/585010865 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb60407dee0 0x7fb60419e150 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:43:47.868 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:47.042+0000 7fb608a00700 1 -- 172.21.15.97:0/585010865 >> 172.21.15.97:0/585010865 conn(0x7fb604103570 msgr2=0x7fb604104150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:43:47.868 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:47.042+0000 7fb608a00700 1 -- 172.21.15.97:0/585010865 shutdown_connections 2024-03-27T10:43:47.868 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:47.042+0000 7fb608a00700 1 -- 172.21.15.97:0/585010865 wait complete. 2024-03-27T10:43:47.868 INFO:teuthology.orchestra.run.smithi097.stdout:mgr not available, waiting (2/15)... 2024-03-27T10:43:48.130 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:43:47 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: 2024-03-27T10:43:47.789+0000 7fdf3acf1200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-27T10:43:48.130 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:43:47 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: 2024-03-27T10:43:47.907+0000 7fdf3acf1200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-27T10:43:48.130 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:43:48 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: 2024-03-27T10:43:48.154+0000 7fdf3acf1200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-27T10:43:48.393 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:43:48 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: 2024-03-27T10:43:48.391+0000 7fdf3acf1200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-27T10:43:49.039 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:43:48 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: 2024-03-27T10:43:48.754+0000 7fdf3acf1200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-27T10:43:49.040 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:43:49 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: 2024-03-27T10:43:49.064+0000 7fdf3acf1200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-27T10:43:49.643 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:43:49 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: 2024-03-27T10:43:49.359+0000 7fdf3acf1200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-27T10:43:50.565 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:43:50 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: 2024-03-27T10:43:50.378+0000 7fdf3acf1200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-27T10:43:50.892 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:50 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/480199087' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-27T10:43:51.180 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:43:51 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: 2024-03-27T10:43:51.078+0000 7fdf3acf1200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-27T10:43:51.207 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 2024-03-27T10:43:51.207 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout { 2024-03-27T10:43:51.207 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "fsid": "adca88c4-ec26-11ee-95d0-87774f69a715", 2024-03-27T10:43:51.208 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "health": { 2024-03-27T10:43:51.208 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-27T10:43:51.208 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-27T10:43:51.208 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-27T10:43:51.208 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-03-27T10:43:51.208 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-27T10:43:51.208 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-27T10:43:51.208 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 0 2024-03-27T10:43:51.208 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout ], 2024-03-27T10:43:51.208 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-27T10:43:51.208 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "a" 2024-03-27T10:43:51.208 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout ], 2024-03-27T10:43:51.209 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-03-27T10:43:51.209 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-27T10:43:51.209 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-27T10:43:51.209 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-27T10:43:51.209 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-27T10:43:51.209 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-03-27T10:43:51.209 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-27T10:43:51.209 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-27T10:43:51.209 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-27T10:43:51.209 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-27T10:43:51.209 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-27T10:43:51.209 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-27T10:43:51.210 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-27T10:43:51.210 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-27T10:43:51.210 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-03-27T10:43:51.210 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-27T10:43:51.210 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-27T10:43:51.210 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-27T10:43:51.210 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-27T10:43:51.210 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-27T10:43:51.210 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-27T10:43:51.210 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-27T10:43:51.210 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-27T10:43:51.211 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-27T10:43:51.211 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-03-27T10:43:51.211 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-27T10:43:51.211 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-27T10:43:51.211 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-27T10:43:51.211 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-27T10:43:51.211 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-03-27T10:43:51.211 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-27T10:43:51.211 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-27T10:43:51.211 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-27T10:43:51.211 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-27T10:43:51.211 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-27T10:43:51.212 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-27T10:43:51.212 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "restful" 2024-03-27T10:43:51.212 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout ], 2024-03-27T10:43:51.212 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-27T10:43:51.212 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-03-27T10:43:51.212 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-27T10:43:51.212 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-27T10:43:51.212 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "modified": "2024-03-27T10:43:24.681303+0000", 2024-03-27T10:43:51.212 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-27T10:43:51.212 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-03-27T10:43:51.212 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-27T10:43:51.212 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout } 2024-03-27T10:43:51.213 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:50.482+0000 7fea36e00700 1 Processor -- start 2024-03-27T10:43:51.213 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:50.482+0000 7fea36e00700 1 -- start start 2024-03-27T10:43:51.213 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:50.482+0000 7fea36e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea3007de80 0x7fea3007c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:43:51.213 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:50.482+0000 7fea36e00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fea3007c8b0 con 0x7fea3007de80 2024-03-27T10:43:51.213 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:50.482+0000 7fea2fe00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea3007de80 0x7fea3007c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:43:51.213 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:50.483+0000 7fea2fe00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea3007de80 0x7fea3007c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45366/0 (socket says 172.21.15.97:45366) 2024-03-27T10:43:51.213 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:50.483+0000 7fea2fe00700 1 -- 172.21.15.97:0/1217415751 learned_addr learned my addr 172.21.15.97:0/1217415751 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:43:51.213 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:50.483+0000 7fea2fe00700 1 -- 172.21.15.97:0/1217415751 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea3007ca30 con 0x7fea3007de80 2024-03-27T10:43:51.213 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:50.483+0000 7fea2fe00700 1 --2- 172.21.15.97:0/1217415751 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea3007de80 0x7fea3007c2e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fea200098a0 tx=0x7fea20009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aa6d9f2b5c2c6a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:43:51.213 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:50.483+0000 7fea2ea00700 1 -- 172.21.15.97:0/1217415751 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fea20004160 con 0x7fea3007de80 2024-03-27T10:43:51.213 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:50.483+0000 7fea2ea00700 1 -- 172.21.15.97:0/1217415751 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fea20004300 con 0x7fea3007de80 2024-03-27T10:43:51.213 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:50.483+0000 7fea2ea00700 1 -- 172.21.15.97:0/1217415751 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fea20011a80 con 0x7fea3007de80 2024-03-27T10:43:51.214 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:50.484+0000 7fea36e00700 1 -- 172.21.15.97:0/1217415751 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea3007de80 msgr2=0x7fea3007c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:43:51.214 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:50.484+0000 7fea36e00700 1 --2- 172.21.15.97:0/1217415751 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea3007de80 0x7fea3007c2e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fea200098a0 tx=0x7fea20009bb0 comp rx=0 tx=0).stop 2024-03-27T10:43:51.214 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:50.484+0000 7fea36e00700 1 -- 172.21.15.97:0/1217415751 shutdown_connections 2024-03-27T10:43:51.214 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:50.484+0000 7fea36e00700 1 --2- 172.21.15.97:0/1217415751 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea3007de80 0x7fea3007c2e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:43:51.214 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:50.484+0000 7fea36e00700 1 -- 172.21.15.97:0/1217415751 >> 172.21.15.97:0/1217415751 conn(0x7fea30103570 msgr2=0x7fea301059d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:43:51.214 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:50.485+0000 7fea36e00700 1 -- 172.21.15.97:0/1217415751 shutdown_connections 2024-03-27T10:43:51.214 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:50.485+0000 7fea36e00700 1 -- 172.21.15.97:0/1217415751 wait complete. 2024-03-27T10:43:51.214 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:50.486+0000 7fea36e00700 1 Processor -- start 2024-03-27T10:43:51.214 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:50.486+0000 7fea36e00700 1 -- start start 2024-03-27T10:43:51.214 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:50.486+0000 7fea36e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea3007de80 0x7fea3019e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:43:51.214 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:50.486+0000 7fea36e00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fea3010e520 con 0x7fea3007de80 2024-03-27T10:43:51.214 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:50.486+0000 7fea2fe00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea3007de80 0x7fea3019e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:43:51.215 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:50.486+0000 7fea2fe00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea3007de80 0x7fea3019e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45376/0 (socket says 172.21.15.97:45376) 2024-03-27T10:43:51.215 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:50.486+0000 7fea2fe00700 1 -- 172.21.15.97:0/480199087 learned_addr learned my addr 172.21.15.97:0/480199087 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:43:51.215 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:50.487+0000 7fea2fe00700 1 -- 172.21.15.97:0/480199087 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea3019e6b0 con 0x7fea3007de80 2024-03-27T10:43:51.215 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:50.487+0000 7fea2fe00700 1 --2- 172.21.15.97:0/480199087 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea3007de80 0x7fea3019e170 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fea20011a50 tx=0x7fea20004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:43:51.215 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:50.487+0000 7fea2cc00700 1 -- 172.21.15.97:0/480199087 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fea200039f0 con 0x7fea3007de80 2024-03-27T10:43:51.215 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:50.487+0000 7fea2cc00700 1 -- 172.21.15.97:0/480199087 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fea20003b90 con 0x7fea3007de80 2024-03-27T10:43:51.215 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:50.487+0000 7fea2cc00700 1 -- 172.21.15.97:0/480199087 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fea2001a600 con 0x7fea3007de80 2024-03-27T10:43:51.215 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:50.487+0000 7fea36e00700 1 -- 172.21.15.97:0/480199087 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea3019e940 con 0x7fea3007de80 2024-03-27T10:43:51.215 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:50.487+0000 7fea36e00700 1 -- 172.21.15.97:0/480199087 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea301a1630 con 0x7fea3007de80 2024-03-27T10:43:51.215 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:50.488+0000 7fea2cc00700 1 -- 172.21.15.97:0/480199087 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fea2001ab30 con 0x7fea3007de80 2024-03-27T10:43:51.215 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:50.488+0000 7fea2cc00700 1 -- 172.21.15.97:0/480199087 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fea20019700 con 0x7fea3007de80 2024-03-27T10:43:51.215 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:50.489+0000 7fea36e00700 1 -- 172.21.15.97:0/480199087 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea30063c70 con 0x7fea3007de80 2024-03-27T10:43:51.216 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:50.491+0000 7fea2cc00700 1 -- 172.21.15.97:0/480199087 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fea20024a40 con 0x7fea3007de80 2024-03-27T10:43:51.216 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:50.571+0000 7fea36e00700 1 -- 172.21.15.97:0/480199087 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fea301a1bf0 con 0x7fea3007de80 2024-03-27T10:43:51.216 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:50.572+0000 7fea2cc00700 1 -- 172.21.15.97:0/480199087 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1240 (secure 0 0 0) 0x7fea20032030 con 0x7fea3007de80 2024-03-27T10:43:51.216 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:50.574+0000 7fea36e00700 1 -- 172.21.15.97:0/480199087 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea3007de80 msgr2=0x7fea3019e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:43:51.216 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:50.574+0000 7fea36e00700 1 --2- 172.21.15.97:0/480199087 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea3007de80 0x7fea3019e170 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fea20011a50 tx=0x7fea20004290 comp rx=0 tx=0).stop 2024-03-27T10:43:51.216 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:50.574+0000 7fea36e00700 1 -- 172.21.15.97:0/480199087 shutdown_connections 2024-03-27T10:43:51.216 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:50.574+0000 7fea36e00700 1 --2- 172.21.15.97:0/480199087 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea3007de80 0x7fea3019e170 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:43:51.216 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:50.574+0000 7fea36e00700 1 -- 172.21.15.97:0/480199087 >> 172.21.15.97:0/480199087 conn(0x7fea30103570 msgr2=0x7fea30104250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:43:51.216 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:50.574+0000 7fea36e00700 1 -- 172.21.15.97:0/480199087 shutdown_connections 2024-03-27T10:43:51.216 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:50.574+0000 7fea36e00700 1 -- 172.21.15.97:0/480199087 wait complete. 2024-03-27T10:43:51.216 INFO:teuthology.orchestra.run.smithi097.stdout:mgr not available, waiting (3/15)... 2024-03-27T10:43:51.459 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:43:51 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: 2024-03-27T10:43:51.205+0000 7fdf3acf1200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-27T10:43:51.894 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:43:51 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: 2024-03-27T10:43:51.483+0000 7fdf3acf1200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-27T10:43:51.894 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:51 smithi097 ceph-mon[31859]: Activating manager daemon x 2024-03-27T10:43:51.894 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:51 smithi097 ceph-mon[31859]: mgrmap e2: x(active, starting, since 0.0114994s) 2024-03-27T10:43:51.894 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:51 smithi097 ceph-mon[31859]: from='mgr.14100 172.21.15.97:0/1381085609' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-27T10:43:51.894 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:51 smithi097 ceph-mon[31859]: from='mgr.14100 172.21.15.97:0/1381085609' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-27T10:43:51.895 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:51 smithi097 ceph-mon[31859]: from='mgr.14100 172.21.15.97:0/1381085609' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-27T10:43:51.895 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:51 smithi097 ceph-mon[31859]: from='mgr.14100 172.21.15.97:0/1381085609' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-27T10:43:51.895 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:51 smithi097 ceph-mon[31859]: from='mgr.14100 172.21.15.97:0/1381085609' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-27T10:43:51.895 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:51 smithi097 ceph-mon[31859]: Manager daemon x is now available 2024-03-27T10:43:51.895 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:51 smithi097 ceph-mon[31859]: from='mgr.14100 172.21.15.97:0/1381085609' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-27T10:43:51.895 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:51 smithi097 ceph-mon[31859]: from='mgr.14100 172.21.15.97:0/1381085609' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-27T10:43:51.895 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:51 smithi097 ceph-mon[31859]: from='mgr.14100 172.21.15.97:0/1381085609' entity='mgr.x' 2024-03-27T10:43:51.895 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:51 smithi097 ceph-mon[31859]: from='mgr.14100 172.21.15.97:0/1381085609' entity='mgr.x' 2024-03-27T10:43:51.895 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:51 smithi097 ceph-mon[31859]: from='mgr.14100 172.21.15.97:0/1381085609' entity='mgr.x' 2024-03-27T10:43:53.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:53 smithi097 ceph-mon[31859]: mgrmap e3: x(active, since 1.02124s) 2024-03-27T10:43:54.797 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:54 smithi097 ceph-mon[31859]: mgrmap e4: x(active, since 2s) 2024-03-27T10:43:54.798 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:54 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/4266264067' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-27T10:43:54.808 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 2024-03-27T10:43:54.808 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout { 2024-03-27T10:43:54.808 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "fsid": "adca88c4-ec26-11ee-95d0-87774f69a715", 2024-03-27T10:43:54.808 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "health": { 2024-03-27T10:43:54.808 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-27T10:43:54.808 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-27T10:43:54.808 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-27T10:43:54.808 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-03-27T10:43:54.808 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-27T10:43:54.809 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-27T10:43:54.809 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 0 2024-03-27T10:43:54.809 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout ], 2024-03-27T10:43:54.809 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-27T10:43:54.809 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "a" 2024-03-27T10:43:54.809 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout ], 2024-03-27T10:43:54.809 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2024-03-27T10:43:54.809 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-27T10:43:54.809 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-27T10:43:54.809 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-27T10:43:54.810 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-27T10:43:54.810 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-03-27T10:43:54.810 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-27T10:43:54.810 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-27T10:43:54.810 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-27T10:43:54.810 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-27T10:43:54.821 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-27T10:43:54.821 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-27T10:43:54.821 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-27T10:43:54.821 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-27T10:43:54.821 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-03-27T10:43:54.821 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-27T10:43:54.821 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-27T10:43:54.821 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-27T10:43:54.822 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-27T10:43:54.822 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-27T10:43:54.822 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-27T10:43:54.822 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-27T10:43:54.822 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-27T10:43:54.822 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-27T10:43:54.822 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-03-27T10:43:54.822 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-27T10:43:54.822 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-27T10:43:54.822 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-27T10:43:54.822 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-27T10:43:54.823 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-03-27T10:43:54.823 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-27T10:43:54.823 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-27T10:43:54.823 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-27T10:43:54.823 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-27T10:43:54.823 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-27T10:43:54.823 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-27T10:43:54.823 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "restful" 2024-03-27T10:43:54.823 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout ], 2024-03-27T10:43:54.823 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-27T10:43:54.823 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-03-27T10:43:54.823 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-27T10:43:54.823 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-27T10:43:54.824 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "modified": "2024-03-27T10:43:24.681303+0000", 2024-03-27T10:43:54.824 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-27T10:43:54.824 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-03-27T10:43:54.824 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-27T10:43:54.824 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout } 2024-03-27T10:43:54.824 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:53.843+0000 7fa8ec600700 1 Processor -- start 2024-03-27T10:43:54.824 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:53.843+0000 7fa8ec600700 1 -- start start 2024-03-27T10:43:54.824 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:53.843+0000 7fa8ec600700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8e410ed80 0x7fa8e4111170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:43:54.824 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:53.843+0000 7fa8ec600700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e4111740 con 0x7fa8e410ed80 2024-03-27T10:43:54.824 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:53.843+0000 7fa8e9c00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8e410ed80 0x7fa8e4111170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:43:54.824 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:53.844+0000 7fa8e9c00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8e410ed80 0x7fa8e4111170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45452/0 (socket says 172.21.15.97:45452) 2024-03-27T10:43:54.824 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:53.844+0000 7fa8e9c00700 1 -- 172.21.15.97:0/3800574269 learned_addr learned my addr 172.21.15.97:0/3800574269 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:43:54.824 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:53.844+0000 7fa8e9c00700 1 -- 172.21.15.97:0/3800574269 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8e41118c0 con 0x7fa8e410ed80 2024-03-27T10:43:54.825 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:53.844+0000 7fa8e9c00700 1 --2- 172.21.15.97:0/3800574269 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8e410ed80 0x7fa8e4111170 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fa8d80098a0 tx=0x7fa8d8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aff7befaa45a2599 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:43:54.825 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:53.844+0000 7fa8e3e00700 1 -- 172.21.15.97:0/3800574269 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa8d8004160 con 0x7fa8e410ed80 2024-03-27T10:43:54.825 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:53.844+0000 7fa8e3e00700 1 -- 172.21.15.97:0/3800574269 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fa8d8004300 con 0x7fa8e410ed80 2024-03-27T10:43:54.825 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:53.844+0000 7fa8e3e00700 1 -- 172.21.15.97:0/3800574269 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa8d8011a80 con 0x7fa8e410ed80 2024-03-27T10:43:54.825 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:53.845+0000 7fa8ec600700 1 -- 172.21.15.97:0/3800574269 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8e410ed80 msgr2=0x7fa8e4111170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:43:54.825 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:53.845+0000 7fa8ec600700 1 --2- 172.21.15.97:0/3800574269 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8e410ed80 0x7fa8e4111170 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fa8d80098a0 tx=0x7fa8d8009bb0 comp rx=0 tx=0).stop 2024-03-27T10:43:54.825 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:53.845+0000 7fa8ec600700 1 -- 172.21.15.97:0/3800574269 shutdown_connections 2024-03-27T10:43:54.825 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:53.845+0000 7fa8ec600700 1 --2- 172.21.15.97:0/3800574269 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8e410ed80 0x7fa8e4111170 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:43:54.825 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:53.845+0000 7fa8ec600700 1 -- 172.21.15.97:0/3800574269 >> 172.21.15.97:0/3800574269 conn(0x7fa8e407d590 msgr2=0x7fa8e407f9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:43:54.825 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:53.845+0000 7fa8ec600700 1 -- 172.21.15.97:0/3800574269 shutdown_connections 2024-03-27T10:43:54.825 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:53.845+0000 7fa8ec600700 1 -- 172.21.15.97:0/3800574269 wait complete. 2024-03-27T10:43:54.825 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:53.846+0000 7fa8ec600700 1 Processor -- start 2024-03-27T10:43:54.826 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:53.846+0000 7fa8ec600700 1 -- start start 2024-03-27T10:43:54.826 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:53.847+0000 7fa8ec600700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8e410ed80 0x7fa8e41a2540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:43:54.826 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:53.847+0000 7fa8ec600700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e4112940 con 0x7fa8e410ed80 2024-03-27T10:43:54.826 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:53.847+0000 7fa8e9c00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8e410ed80 0x7fa8e41a2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:43:54.826 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:53.847+0000 7fa8e9c00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8e410ed80 0x7fa8e41a2540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45466/0 (socket says 172.21.15.97:45466) 2024-03-27T10:43:54.826 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:53.847+0000 7fa8e9c00700 1 -- 172.21.15.97:0/4266264067 learned_addr learned my addr 172.21.15.97:0/4266264067 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:43:54.826 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:53.847+0000 7fa8e9c00700 1 -- 172.21.15.97:0/4266264067 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8e41a2a80 con 0x7fa8e410ed80 2024-03-27T10:43:54.826 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:53.848+0000 7fa8e9c00700 1 --2- 172.21.15.97:0/4266264067 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8e410ed80 0x7fa8e41a2540 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa8d8011a50 tx=0x7fa8d8004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:43:54.826 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:53.848+0000 7fa8e2000700 1 -- 172.21.15.97:0/4266264067 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa8d80039f0 con 0x7fa8e410ed80 2024-03-27T10:43:54.826 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:53.848+0000 7fa8e2000700 1 -- 172.21.15.97:0/4266264067 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fa8d8003b90 con 0x7fa8e410ed80 2024-03-27T10:43:54.826 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:53.848+0000 7fa8e2000700 1 -- 172.21.15.97:0/4266264067 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa8d801a600 con 0x7fa8e410ed80 2024-03-27T10:43:54.826 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:53.848+0000 7fa8ec600700 1 -- 172.21.15.97:0/4266264067 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8e41a2d10 con 0x7fa8e410ed80 2024-03-27T10:43:54.826 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:53.848+0000 7fa8ec600700 1 -- 172.21.15.97:0/4266264067 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8e41a5a00 con 0x7fa8e410ed80 2024-03-27T10:43:54.827 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:53.849+0000 7fa8e2000700 1 -- 172.21.15.97:0/4266264067 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 4) ==== 46455+0+0 (secure 0 0 0) 0x7fa8d8024a40 con 0x7fa8e410ed80 2024-03-27T10:43:54.827 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:53.849+0000 7fa8ec600700 1 -- 172.21.15.97:0/4266264067 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8e4063c70 con 0x7fa8e410ed80 2024-03-27T10:43:54.827 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:53.850+0000 7fa8e2000700 1 --2- 172.21.15.97:0/4266264067 >> [v2:172.21.15.97:6800/98682155,v1:172.21.15.97:6801/98682155] conn(0x7fa8cc0393d0 0x7fa8cc03b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:43:54.827 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:53.850+0000 7fa8e2000700 1 -- 172.21.15.97:0/4266264067 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa8d804d8a0 con 0x7fa8e410ed80 2024-03-27T10:43:54.827 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:53.850+0000 7fa8e9200700 1 --2- 172.21.15.97:0/4266264067 >> [v2:172.21.15.97:6800/98682155,v1:172.21.15.97:6801/98682155] conn(0x7fa8cc0393d0 0x7fa8cc03b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:43:54.827 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:53.851+0000 7fa8e9200700 1 --2- 172.21.15.97:0/4266264067 >> [v2:172.21.15.97:6800/98682155,v1:172.21.15.97:6801/98682155] conn(0x7fa8cc0393d0 0x7fa8cc03b890 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa8d0006fd0 tx=0x7fa8d0006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:43:54.827 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:53.856+0000 7fa8e2000700 1 -- 172.21.15.97:0/4266264067 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7fa8d802d770 con 0x7fa8e410ed80 2024-03-27T10:43:54.827 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:54.167+0000 7fa8ec600700 1 -- 172.21.15.97:0/4266264067 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fa8e41a5bd0 con 0x7fa8e410ed80 2024-03-27T10:43:54.827 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:54.168+0000 7fa8e2000700 1 -- 172.21.15.97:0/4266264067 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fa8e41a5bd0 con 0x7fa8e410ed80 2024-03-27T10:43:54.827 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:54.172+0000 7fa8ec600700 1 -- 172.21.15.97:0/4266264067 >> [v2:172.21.15.97:6800/98682155,v1:172.21.15.97:6801/98682155] conn(0x7fa8cc0393d0 msgr2=0x7fa8cc03b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:43:54.827 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:54.172+0000 7fa8ec600700 1 --2- 172.21.15.97:0/4266264067 >> [v2:172.21.15.97:6800/98682155,v1:172.21.15.97:6801/98682155] conn(0x7fa8cc0393d0 0x7fa8cc03b890 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa8d0006fd0 tx=0x7fa8d0006e40 comp rx=0 tx=0).stop 2024-03-27T10:43:54.827 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:54.173+0000 7fa8ec600700 1 -- 172.21.15.97:0/4266264067 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8e410ed80 msgr2=0x7fa8e41a2540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:43:54.827 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:54.173+0000 7fa8ec600700 1 --2- 172.21.15.97:0/4266264067 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8e410ed80 0x7fa8e41a2540 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa8d8011a50 tx=0x7fa8d8004290 comp rx=0 tx=0).stop 2024-03-27T10:43:54.827 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:54.173+0000 7fa8ec600700 1 -- 172.21.15.97:0/4266264067 shutdown_connections 2024-03-27T10:43:54.828 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:54.173+0000 7fa8ec600700 1 --2- 172.21.15.97:0/4266264067 >> [v2:172.21.15.97:6800/98682155,v1:172.21.15.97:6801/98682155] conn(0x7fa8cc0393d0 0x7fa8cc03b890 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:43:54.828 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:54.173+0000 7fa8ec600700 1 --2- 172.21.15.97:0/4266264067 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8e410ed80 0x7fa8e41a2540 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:43:54.828 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:54.173+0000 7fa8ec600700 1 -- 172.21.15.97:0/4266264067 >> 172.21.15.97:0/4266264067 conn(0x7fa8e407d590 msgr2=0x7fa8e407e160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:43:54.828 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:54.174+0000 7fa8ec600700 1 -- 172.21.15.97:0/4266264067 shutdown_connections 2024-03-27T10:43:54.828 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:54.174+0000 7fa8ec600700 1 -- 172.21.15.97:0/4266264067 wait complete. 2024-03-27T10:43:54.828 INFO:teuthology.orchestra.run.smithi097.stdout:mgr is available 2024-03-27T10:43:55.893 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:55 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/3162549848' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-27T10:43:56.312 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 2024-03-27T10:43:56.312 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout [global] 2024-03-27T10:43:56.312 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout fsid = adca88c4-ec26-11ee-95d0-87774f69a715 2024-03-27T10:43:56.312 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-27T10:43:56.312 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-27T10:43:56.312 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-27T10:43:56.312 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-27T10:43:56.312 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 2024-03-27T10:43:56.312 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-27T10:43:56.313 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-27T10:43:56.313 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 2024-03-27T10:43:56.313 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout [osd] 2024-03-27T10:43:56.313 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-03-27T10:43:56.313 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-27T10:43:56.313 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-27T10:43:56.313 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-27T10:43:56.313 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-27T10:43:56.313 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-27T10:43:56.313 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:55.411+0000 7f1d78c00700 1 Processor -- start 2024-03-27T10:43:56.313 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:55.411+0000 7f1d78c00700 1 -- start start 2024-03-27T10:43:56.313 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:55.412+0000 7f1d78c00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d74109fc0 0x7f1d7410a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:43:56.314 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:55.412+0000 7f1d78c00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1d7410a9b0 con 0x7f1d74109fc0 2024-03-27T10:43:56.314 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:55.412+0000 7f1d73400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d74109fc0 0x7f1d7410a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:43:56.314 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:55.412+0000 7f1d73400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d74109fc0 0x7f1d7410a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45476/0 (socket says 172.21.15.97:45476) 2024-03-27T10:43:56.314 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:55.412+0000 7f1d73400700 1 -- 172.21.15.97:0/144526220 learned_addr learned my addr 172.21.15.97:0/144526220 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:43:56.314 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:55.413+0000 7f1d73400700 1 -- 172.21.15.97:0/144526220 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d7410b1c0 con 0x7f1d74109fc0 2024-03-27T10:43:56.314 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:55.413+0000 7f1d73400700 1 --2- 172.21.15.97:0/144526220 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d74109fc0 0x7f1d7410a3e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f1d640098a0 tx=0x7f1d64009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a78f6396cf6b8611 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:43:56.314 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:55.413+0000 7f1d72000700 1 -- 172.21.15.97:0/144526220 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1d64004160 con 0x7f1d74109fc0 2024-03-27T10:43:56.314 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:55.414+0000 7f1d72000700 1 -- 172.21.15.97:0/144526220 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f1d64004300 con 0x7f1d74109fc0 2024-03-27T10:43:56.314 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:55.414+0000 7f1d72000700 1 -- 172.21.15.97:0/144526220 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1d64011a80 con 0x7f1d74109fc0 2024-03-27T10:43:56.314 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:55.414+0000 7f1d78c00700 1 -- 172.21.15.97:0/144526220 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d74109fc0 msgr2=0x7f1d7410a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:43:56.314 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:55.415+0000 7f1d78c00700 1 --2- 172.21.15.97:0/144526220 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d74109fc0 0x7f1d7410a3e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f1d640098a0 tx=0x7f1d64009bb0 comp rx=0 tx=0).stop 2024-03-27T10:43:56.314 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:55.415+0000 7f1d78c00700 1 -- 172.21.15.97:0/144526220 shutdown_connections 2024-03-27T10:43:56.314 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:55.415+0000 7f1d78c00700 1 --2- 172.21.15.97:0/144526220 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d74109fc0 0x7f1d7410a3e0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:43:56.315 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:55.415+0000 7f1d78c00700 1 -- 172.21.15.97:0/144526220 >> 172.21.15.97:0/144526220 conn(0x7f1d7407d3b0 msgr2=0x7f1d7407d7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:43:56.315 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:55.415+0000 7f1d78c00700 1 -- 172.21.15.97:0/144526220 shutdown_connections 2024-03-27T10:43:56.315 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:55.415+0000 7f1d78c00700 1 -- 172.21.15.97:0/144526220 wait complete. 2024-03-27T10:43:56.315 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:55.416+0000 7f1d78c00700 1 Processor -- start 2024-03-27T10:43:56.315 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:55.416+0000 7f1d78c00700 1 -- start start 2024-03-27T10:43:56.315 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:55.416+0000 7f1d78c00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d74109fc0 0x7f1d7419df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:43:56.315 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:55.416+0000 7f1d78c00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1d7410e570 con 0x7f1d74109fc0 2024-03-27T10:43:56.315 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:55.417+0000 7f1d73400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d74109fc0 0x7f1d7419df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:43:56.315 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:55.417+0000 7f1d73400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d74109fc0 0x7f1d7419df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45482/0 (socket says 172.21.15.97:45482) 2024-03-27T10:43:56.315 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:55.417+0000 7f1d73400700 1 -- 172.21.15.97:0/3162549848 learned_addr learned my addr 172.21.15.97:0/3162549848 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:43:56.315 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:55.417+0000 7f1d73400700 1 -- 172.21.15.97:0/3162549848 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d7419e4b0 con 0x7f1d74109fc0 2024-03-27T10:43:56.316 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:55.417+0000 7f1d73400700 1 --2- 172.21.15.97:0/3162549848 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d74109fc0 0x7f1d7419df70 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f1d64000c00 tx=0x7f1d64011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:43:56.316 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:55.417+0000 7f1d6be00700 1 -- 172.21.15.97:0/3162549848 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1d640039f0 con 0x7f1d74109fc0 2024-03-27T10:43:56.316 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:55.418+0000 7f1d6be00700 1 -- 172.21.15.97:0/3162549848 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f1d64003b90 con 0x7f1d74109fc0 2024-03-27T10:43:56.316 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:55.418+0000 7f1d78c00700 1 -- 172.21.15.97:0/3162549848 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d7419e740 con 0x7f1d74109fc0 2024-03-27T10:43:56.316 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:55.418+0000 7f1d6be00700 1 -- 172.21.15.97:0/3162549848 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1d6401a740 con 0x7f1d74109fc0 2024-03-27T10:43:56.316 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:55.418+0000 7f1d78c00700 1 -- 172.21.15.97:0/3162549848 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d741a1430 con 0x7f1d74109fc0 2024-03-27T10:43:56.316 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:55.419+0000 7f1d6be00700 1 -- 172.21.15.97:0/3162549848 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 4) ==== 46455+0+0 (secure 0 0 0) 0x7f1d6401aa40 con 0x7f1d74109fc0 2024-03-27T10:43:56.316 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:55.419+0000 7f1d78c00700 1 -- 172.21.15.97:0/3162549848 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d74063c70 con 0x7f1d74109fc0 2024-03-27T10:43:56.316 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:55.419+0000 7f1d6be00700 1 --2- 172.21.15.97:0/3162549848 >> [v2:172.21.15.97:6800/98682155,v1:172.21.15.97:6801/98682155] conn(0x7f1d580393d0 0x7f1d5803b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:43:56.316 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:55.419+0000 7f1d6be00700 1 -- 172.21.15.97:0/3162549848 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1d6404d7e0 con 0x7f1d74109fc0 2024-03-27T10:43:56.316 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:55.420+0000 7f1d72a00700 1 --2- 172.21.15.97:0/3162549848 >> [v2:172.21.15.97:6800/98682155,v1:172.21.15.97:6801/98682155] conn(0x7f1d580393d0 0x7f1d5803b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:43:56.316 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:55.420+0000 7f1d72a00700 1 --2- 172.21.15.97:0/3162549848 >> [v2:172.21.15.97:6800/98682155,v1:172.21.15.97:6801/98682155] conn(0x7f1d580393d0 0x7f1d5803b890 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1d60006fd0 tx=0x7f1d60006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:43:56.317 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:55.425+0000 7f1d6be00700 1 -- 172.21.15.97:0/3162549848 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f1d6400b5a0 con 0x7f1d74109fc0 2024-03-27T10:43:56.317 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:55.644+0000 7f1d78c00700 1 -- 172.21.15.97:0/3162549848 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f1d741a1600 con 0x7f1d74109fc0 2024-03-27T10:43:56.317 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:55.646+0000 7f1d6be00700 1 -- 172.21.15.97:0/3162549848 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+400 (secure 0 0 0) 0x7f1d741a1600 con 0x7f1d74109fc0 2024-03-27T10:43:56.317 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:55.649+0000 7f1d78c00700 1 -- 172.21.15.97:0/3162549848 >> [v2:172.21.15.97:6800/98682155,v1:172.21.15.97:6801/98682155] conn(0x7f1d580393d0 msgr2=0x7f1d5803b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:43:56.317 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:55.649+0000 7f1d78c00700 1 --2- 172.21.15.97:0/3162549848 >> [v2:172.21.15.97:6800/98682155,v1:172.21.15.97:6801/98682155] conn(0x7f1d580393d0 0x7f1d5803b890 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1d60006fd0 tx=0x7f1d60006e40 comp rx=0 tx=0).stop 2024-03-27T10:43:56.317 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:55.649+0000 7f1d78c00700 1 -- 172.21.15.97:0/3162549848 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d74109fc0 msgr2=0x7f1d7419df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:43:56.317 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:55.649+0000 7f1d78c00700 1 --2- 172.21.15.97:0/3162549848 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d74109fc0 0x7f1d7419df70 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f1d64000c00 tx=0x7f1d64011b00 comp rx=0 tx=0).stop 2024-03-27T10:43:56.317 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:55.649+0000 7f1d78c00700 1 -- 172.21.15.97:0/3162549848 shutdown_connections 2024-03-27T10:43:56.317 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:55.649+0000 7f1d78c00700 1 --2- 172.21.15.97:0/3162549848 >> [v2:172.21.15.97:6800/98682155,v1:172.21.15.97:6801/98682155] conn(0x7f1d580393d0 0x7f1d5803b890 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:43:56.317 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:55.649+0000 7f1d78c00700 1 --2- 172.21.15.97:0/3162549848 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d74109fc0 0x7f1d7419df70 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:43:56.317 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:55.649+0000 7f1d78c00700 1 -- 172.21.15.97:0/3162549848 >> 172.21.15.97:0/3162549848 conn(0x7f1d7407d3b0 msgr2=0x7f1d74107500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:43:56.317 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:55.649+0000 7f1d78c00700 1 -- 172.21.15.97:0/3162549848 shutdown_connections 2024-03-27T10:43:56.317 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:55.649+0000 7f1d78c00700 1 -- 172.21.15.97:0/3162549848 wait complete. 2024-03-27T10:43:56.318 INFO:teuthology.orchestra.run.smithi097.stdout:Enabling cephadm module... 2024-03-27T10:43:57.624 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:43:57 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: ignoring --setuser ceph since I am not root 2024-03-27T10:43:57.624 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:43:57 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: ignoring --setgroup ceph since I am not root 2024-03-27T10:43:57.624 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:57 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/3763421050' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-03-27T10:43:57.985 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:56.941+0000 7f0698c00700 1 Processor -- start 2024-03-27T10:43:57.986 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:56.941+0000 7f0698c00700 1 -- start start 2024-03-27T10:43:57.986 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:56.942+0000 7f0698c00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f069410ecf0 0x7f06941110e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:43:57.986 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:56.942+0000 7f0698c00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0694111620 con 0x7f069410ecf0 2024-03-27T10:43:57.986 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:56.942+0000 7f0693400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f069410ecf0 0x7f06941110e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:43:57.986 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:56.942+0000 7f0693400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f069410ecf0 0x7f06941110e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45492/0 (socket says 172.21.15.97:45492) 2024-03-27T10:43:57.987 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:56.942+0000 7f0693400700 1 -- 172.21.15.97:0/1565848086 learned_addr learned my addr 172.21.15.97:0/1565848086 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:43:57.987 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:56.943+0000 7f0693400700 1 -- 172.21.15.97:0/1565848086 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06941117a0 con 0x7f069410ecf0 2024-03-27T10:43:57.987 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:56.943+0000 7f0693400700 1 --2- 172.21.15.97:0/1565848086 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f069410ecf0 0x7f06941110e0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f06840098a0 tx=0x7f0684009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be285d7b1f840872 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:43:57.987 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:56.943+0000 7f0692000700 1 -- 172.21.15.97:0/1565848086 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0684004160 con 0x7f069410ecf0 2024-03-27T10:43:57.987 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:56.943+0000 7f0692000700 1 -- 172.21.15.97:0/1565848086 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f0684004300 con 0x7f069410ecf0 2024-03-27T10:43:57.987 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:56.943+0000 7f0692000700 1 -- 172.21.15.97:0/1565848086 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0684011a80 con 0x7f069410ecf0 2024-03-27T10:43:57.987 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:56.944+0000 7f0698c00700 1 -- 172.21.15.97:0/1565848086 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f069410ecf0 msgr2=0x7f06941110e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:43:57.987 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:56.944+0000 7f0698c00700 1 --2- 172.21.15.97:0/1565848086 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f069410ecf0 0x7f06941110e0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f06840098a0 tx=0x7f0684009bb0 comp rx=0 tx=0).stop 2024-03-27T10:43:57.987 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:56.944+0000 7f0698c00700 1 -- 172.21.15.97:0/1565848086 shutdown_connections 2024-03-27T10:43:57.987 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:56.944+0000 7f0698c00700 1 --2- 172.21.15.97:0/1565848086 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f069410ecf0 0x7f06941110e0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:43:57.987 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:56.944+0000 7f0698c00700 1 -- 172.21.15.97:0/1565848086 >> 172.21.15.97:0/1565848086 conn(0x7f069407d410 msgr2=0x7f069407f830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:43:57.988 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:56.944+0000 7f0698c00700 1 -- 172.21.15.97:0/1565848086 shutdown_connections 2024-03-27T10:43:57.988 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:56.944+0000 7f0698c00700 1 -- 172.21.15.97:0/1565848086 wait complete. 2024-03-27T10:43:57.988 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:56.945+0000 7f0698c00700 1 Processor -- start 2024-03-27T10:43:57.988 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:56.945+0000 7f0698c00700 1 -- start start 2024-03-27T10:43:57.988 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:56.946+0000 7f0698c00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f069410ecf0 0x7f06941a23c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:43:57.988 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:56.946+0000 7f0698c00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0694112770 con 0x7f069410ecf0 2024-03-27T10:43:57.988 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:56.946+0000 7f0693400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f069410ecf0 0x7f06941a23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:43:57.988 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:56.946+0000 7f0693400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f069410ecf0 0x7f06941a23c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45500/0 (socket says 172.21.15.97:45500) 2024-03-27T10:43:57.988 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:56.946+0000 7f0693400700 1 -- 172.21.15.97:0/3763421050 learned_addr learned my addr 172.21.15.97:0/3763421050 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:43:57.988 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:56.946+0000 7f0693400700 1 -- 172.21.15.97:0/3763421050 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06941a2900 con 0x7f069410ecf0 2024-03-27T10:43:57.988 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:56.947+0000 7f0693400700 1 --2- 172.21.15.97:0/3763421050 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f069410ecf0 0x7f06941a23c0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0684011a50 tx=0x7f0684004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:43:57.988 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:56.947+0000 7f068be00700 1 -- 172.21.15.97:0/3763421050 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f06840039f0 con 0x7f069410ecf0 2024-03-27T10:43:57.988 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:56.947+0000 7f068be00700 1 -- 172.21.15.97:0/3763421050 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f0684003b90 con 0x7f069410ecf0 2024-03-27T10:43:57.989 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:56.947+0000 7f0698c00700 1 -- 172.21.15.97:0/3763421050 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06941a2b90 con 0x7f069410ecf0 2024-03-27T10:43:57.989 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:56.947+0000 7f068be00700 1 -- 172.21.15.97:0/3763421050 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f068401a600 con 0x7f069410ecf0 2024-03-27T10:43:57.989 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:56.947+0000 7f0698c00700 1 -- 172.21.15.97:0/3763421050 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06941a5880 con 0x7f069410ecf0 2024-03-27T10:43:57.989 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:56.948+0000 7f068be00700 1 -- 172.21.15.97:0/3763421050 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 4) ==== 46455+0+0 (secure 0 0 0) 0x7f0684024a40 con 0x7f069410ecf0 2024-03-27T10:43:57.989 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:56.948+0000 7f0698c00700 1 -- 172.21.15.97:0/3763421050 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0694063c70 con 0x7f069410ecf0 2024-03-27T10:43:57.989 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:56.949+0000 7f068be00700 1 --2- 172.21.15.97:0/3763421050 >> [v2:172.21.15.97:6800/98682155,v1:172.21.15.97:6801/98682155] conn(0x7f06780393d0 0x7f067803b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:43:57.989 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:56.949+0000 7f068be00700 1 -- 172.21.15.97:0/3763421050 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f068404d860 con 0x7f069410ecf0 2024-03-27T10:43:57.989 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:56.949+0000 7f0692a00700 1 --2- 172.21.15.97:0/3763421050 >> [v2:172.21.15.97:6800/98682155,v1:172.21.15.97:6801/98682155] conn(0x7f06780393d0 0x7f067803b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:43:57.989 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:56.950+0000 7f0692a00700 1 --2- 172.21.15.97:0/3763421050 >> [v2:172.21.15.97:6800/98682155,v1:172.21.15.97:6801/98682155] conn(0x7f06780393d0 0x7f067803b890 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0680006fd0 tx=0x7f0680006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:43:57.989 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:56.954+0000 7f068be00700 1 -- 172.21.15.97:0/3763421050 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f068404d280 con 0x7f069410ecf0 2024-03-27T10:43:57.989 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:57.234+0000 7f0698c00700 1 -- 172.21.15.97:0/3763421050 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f06941a5a50 con 0x7f069410ecf0 2024-03-27T10:43:57.989 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:57.287+0000 7f068be00700 1 -- 172.21.15.97:0/3763421050 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f06941a5a50 con 0x7f069410ecf0 2024-03-27T10:43:57.990 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:57.291+0000 7f068be00700 1 -- 172.21.15.97:0/3763421050 <== mon.0 v2:172.21.15.97:3300/0 8 ==== mgrmap(e 5) ==== 46466+0+0 (secure 0 0 0) 0x7f068402d770 con 0x7f069410ecf0 2024-03-27T10:43:57.990 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:57.292+0000 7f0698c00700 1 -- 172.21.15.97:0/3763421050 >> [v2:172.21.15.97:6800/98682155,v1:172.21.15.97:6801/98682155] conn(0x7f06780393d0 msgr2=0x7f067803b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:43:57.990 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:57.292+0000 7f0698c00700 1 --2- 172.21.15.97:0/3763421050 >> [v2:172.21.15.97:6800/98682155,v1:172.21.15.97:6801/98682155] conn(0x7f06780393d0 0x7f067803b890 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0680006fd0 tx=0x7f0680006e40 comp rx=0 tx=0).stop 2024-03-27T10:43:57.990 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:57.292+0000 7f0698c00700 1 -- 172.21.15.97:0/3763421050 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f069410ecf0 msgr2=0x7f06941a23c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:43:57.990 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:57.292+0000 7f0698c00700 1 --2- 172.21.15.97:0/3763421050 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f069410ecf0 0x7f06941a23c0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0684011a50 tx=0x7f0684004290 comp rx=0 tx=0).stop 2024-03-27T10:43:57.990 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:57.293+0000 7f0698c00700 1 -- 172.21.15.97:0/3763421050 shutdown_connections 2024-03-27T10:43:57.990 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:57.293+0000 7f0698c00700 1 --2- 172.21.15.97:0/3763421050 >> [v2:172.21.15.97:6800/98682155,v1:172.21.15.97:6801/98682155] conn(0x7f06780393d0 0x7f067803b890 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:43:57.990 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:57.293+0000 7f0698c00700 1 --2- 172.21.15.97:0/3763421050 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f069410ecf0 0x7f06941a23c0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:43:57.990 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:57.293+0000 7f0698c00700 1 -- 172.21.15.97:0/3763421050 >> 172.21.15.97:0/3763421050 conn(0x7f069407d410 msgr2=0x7f069407e010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:43:57.990 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:57.293+0000 7f0698c00700 1 -- 172.21.15.97:0/3763421050 shutdown_connections 2024-03-27T10:43:57.990 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:57.293+0000 7f0698c00700 1 -- 172.21.15.97:0/3763421050 wait complete. 2024-03-27T10:43:58.268 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:43:58 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: 2024-03-27T10:43:58.062+0000 7fd2aef3c200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-27T10:43:58.269 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:43:58 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: 2024-03-27T10:43:58.181+0000 7fd2aef3c200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-27T10:43:58.573 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:58 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/3763421050' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-03-27T10:43:58.573 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:58 smithi097 ceph-mon[31859]: mgrmap e5: x(active, since 5s) 2024-03-27T10:43:59.143 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:43:58 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: 2024-03-27T10:43:58.981+0000 7fd2aef3c200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-27T10:43:59.143 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:43:59 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: 2024-03-27T10:43:59.100+0000 7fd2aef3c200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-27T10:43:59.600 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:43:59 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/1806622742' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-27T10:43:59.601 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:43:59 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: 2024-03-27T10:43:59.286+0000 7fd2aef3c200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-27T10:43:59.601 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:43:59 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: 2024-03-27T10:43:59.409+0000 7fd2aef3c200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-27T10:43:59.601 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:43:59 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: 2024-03-27T10:43:59.519+0000 7fd2aef3c200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-27T10:43:59.614 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout { 2024-03-27T10:43:59.614 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-03-27T10:43:59.614 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-27T10:43:59.615 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-03-27T10:43:59.615 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-27T10:43:59.615 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout } 2024-03-27T10:43:59.615 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:58.598+0000 7f264b000700 1 Processor -- start 2024-03-27T10:43:59.615 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:58.598+0000 7f264b000700 1 -- start start 2024-03-27T10:43:59.615 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:58.599+0000 7f264b000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f264407dee0 0x7f264407c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:43:59.615 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:58.599+0000 7f264b000700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f264407c910 con 0x7f264407dee0 2024-03-27T10:43:59.615 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:58.599+0000 7f2643e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f264407dee0 0x7f264407c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:43:59.615 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:58.599+0000 7f2643e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f264407dee0 0x7f264407c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45526/0 (socket says 172.21.15.97:45526) 2024-03-27T10:43:59.615 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:58.599+0000 7f2643e00700 1 -- 172.21.15.97:0/2669546839 learned_addr learned my addr 172.21.15.97:0/2669546839 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:43:59.615 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:58.599+0000 7f2643e00700 1 -- 172.21.15.97:0/2669546839 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f264407ca90 con 0x7f264407dee0 2024-03-27T10:43:59.615 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:58.600+0000 7f2643e00700 1 --2- 172.21.15.97:0/2669546839 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f264407dee0 0x7f264407c340 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f26340098a0 tx=0x7f2634009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2888d895be8b882 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:43:59.615 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:58.600+0000 7f2643400700 1 -- 172.21.15.97:0/2669546839 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2634004160 con 0x7f264407dee0 2024-03-27T10:43:59.616 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:58.600+0000 7f2643400700 1 -- 172.21.15.97:0/2669546839 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f2634004300 con 0x7f264407dee0 2024-03-27T10:43:59.616 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:58.600+0000 7f2643400700 1 -- 172.21.15.97:0/2669546839 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2634011a80 con 0x7f264407dee0 2024-03-27T10:43:59.616 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:58.601+0000 7f264b000700 1 -- 172.21.15.97:0/2669546839 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f264407dee0 msgr2=0x7f264407c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:43:59.616 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:58.601+0000 7f264b000700 1 --2- 172.21.15.97:0/2669546839 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f264407dee0 0x7f264407c340 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f26340098a0 tx=0x7f2634009bb0 comp rx=0 tx=0).stop 2024-03-27T10:43:59.616 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:58.601+0000 7f264b000700 1 -- 172.21.15.97:0/2669546839 shutdown_connections 2024-03-27T10:43:59.616 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:58.601+0000 7f264b000700 1 --2- 172.21.15.97:0/2669546839 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f264407dee0 0x7f264407c340 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:43:59.616 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:58.601+0000 7f264b000700 1 -- 172.21.15.97:0/2669546839 >> 172.21.15.97:0/2669546839 conn(0x7f2644103570 msgr2=0x7f26441059d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:43:59.616 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:58.601+0000 7f264b000700 1 -- 172.21.15.97:0/2669546839 shutdown_connections 2024-03-27T10:43:59.616 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:58.601+0000 7f264b000700 1 -- 172.21.15.97:0/2669546839 wait complete. 2024-03-27T10:43:59.616 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:58.602+0000 7f264b000700 1 Processor -- start 2024-03-27T10:43:59.616 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:58.602+0000 7f264b000700 1 -- start start 2024-03-27T10:43:59.616 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:58.602+0000 7f264b000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f264407dee0 0x7f264419e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:43:59.617 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:58.603+0000 7f264b000700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f264410e520 con 0x7f264407dee0 2024-03-27T10:43:59.617 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:58.603+0000 7f2643e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f264407dee0 0x7f264419e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:43:59.617 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:58.603+0000 7f2643e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f264407dee0 0x7f264419e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45538/0 (socket says 172.21.15.97:45538) 2024-03-27T10:43:59.617 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:58.603+0000 7f2643e00700 1 -- 172.21.15.97:0/1806622742 learned_addr learned my addr 172.21.15.97:0/1806622742 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:43:59.617 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:58.603+0000 7f2643e00700 1 -- 172.21.15.97:0/1806622742 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f264419e690 con 0x7f264407dee0 2024-03-27T10:43:59.617 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:58.603+0000 7f2643e00700 1 --2- 172.21.15.97:0/1806622742 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f264407dee0 0x7f264419e150 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2634000c00 tx=0x7f2634004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:43:59.617 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:58.604+0000 7f2641600700 1 -- 172.21.15.97:0/1806622742 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f26340039c0 con 0x7f264407dee0 2024-03-27T10:43:59.617 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:58.604+0000 7f2641600700 1 -- 172.21.15.97:0/1806622742 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f2634003b60 con 0x7f264407dee0 2024-03-27T10:43:59.617 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:58.604+0000 7f2641600700 1 -- 172.21.15.97:0/1806622742 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f263401a6f0 con 0x7f264407dee0 2024-03-27T10:43:59.617 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:58.604+0000 7f264b000700 1 -- 172.21.15.97:0/1806622742 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f264419e920 con 0x7f264407dee0 2024-03-27T10:43:59.617 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:58.604+0000 7f264b000700 1 -- 172.21.15.97:0/1806622742 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26441a1610 con 0x7f264407dee0 2024-03-27T10:43:59.617 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:58.605+0000 7f2641600700 1 -- 172.21.15.97:0/1806622742 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 46466+0+0 (secure 0 0 0) 0x7f263401a9a0 con 0x7f264407dee0 2024-03-27T10:43:59.618 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:58.605+0000 7f264b000700 1 -- 172.21.15.97:0/1806622742 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2644063c70 con 0x7f264407dee0 2024-03-27T10:43:59.618 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:58.605+0000 7f2641600700 1 --2- 172.21.15.97:0/1806622742 >> [v2:172.21.15.97:6800/98682155,v1:172.21.15.97:6801/98682155] conn(0x7f26240393d0 0x7f262403b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:43:59.618 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:58.605+0000 7f2641600700 1 -- 172.21.15.97:0/1806622742 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f263404d7c0 con 0x7f264407dee0 2024-03-27T10:43:59.618 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:58.605+0000 7f263b400700 1 -- 172.21.15.97:0/1806622742 >> [v2:172.21.15.97:6800/98682155,v1:172.21.15.97:6801/98682155] conn(0x7f26240393d0 msgr2=0x7f262403b890 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.97:6800/98682155 2024-03-27T10:43:59.618 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:58.606+0000 7f263b400700 1 --2- 172.21.15.97:0/1806622742 >> [v2:172.21.15.97:6800/98682155,v1:172.21.15.97:6801/98682155] conn(0x7f26240393d0 0x7f262403b890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-27T10:43:59.618 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:58.611+0000 7f2641600700 1 -- 172.21.15.97:0/1806622742 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f2634020240 con 0x7f264407dee0 2024-03-27T10:43:59.618 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:58.807+0000 7f263b400700 1 -- 172.21.15.97:0/1806622742 >> [v2:172.21.15.97:6800/98682155,v1:172.21.15.97:6801/98682155] conn(0x7f26240393d0 msgr2=0x7f262403b890 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.97:6800/98682155 2024-03-27T10:43:59.618 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:58.807+0000 7f263b400700 1 --2- 172.21.15.97:0/1806622742 >> [v2:172.21.15.97:6800/98682155,v1:172.21.15.97:6801/98682155] conn(0x7f26240393d0 0x7f262403b890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-27T10:43:59.618 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:58.895+0000 7f264b000700 1 -- 172.21.15.97:0/1806622742 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f26441a17e0 con 0x7f264407dee0 2024-03-27T10:43:59.618 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:58.896+0000 7f2641600700 1 -- 172.21.15.97:0/1806622742 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f26441a17e0 con 0x7f264407dee0 2024-03-27T10:43:59.618 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:58.899+0000 7f264b000700 1 -- 172.21.15.97:0/1806622742 >> [v2:172.21.15.97:6800/98682155,v1:172.21.15.97:6801/98682155] conn(0x7f26240393d0 msgr2=0x7f262403b890 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-27T10:43:59.619 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:58.899+0000 7f264b000700 1 --2- 172.21.15.97:0/1806622742 >> [v2:172.21.15.97:6800/98682155,v1:172.21.15.97:6801/98682155] conn(0x7f26240393d0 0x7f262403b890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:43:59.619 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:58.899+0000 7f264b000700 1 -- 172.21.15.97:0/1806622742 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f264407dee0 msgr2=0x7f264419e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:43:59.619 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:58.899+0000 7f264b000700 1 --2- 172.21.15.97:0/1806622742 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f264407dee0 0x7f264419e150 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2634000c00 tx=0x7f2634004290 comp rx=0 tx=0).stop 2024-03-27T10:43:59.619 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:58.899+0000 7f264b000700 1 -- 172.21.15.97:0/1806622742 shutdown_connections 2024-03-27T10:43:59.619 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:58.899+0000 7f264b000700 1 --2- 172.21.15.97:0/1806622742 >> [v2:172.21.15.97:6800/98682155,v1:172.21.15.97:6801/98682155] conn(0x7f26240393d0 0x7f262403b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:43:59.619 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:58.899+0000 7f264b000700 1 --2- 172.21.15.97:0/1806622742 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f264407dee0 0x7f264419e150 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:43:59.619 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:58.899+0000 7f264b000700 1 -- 172.21.15.97:0/1806622742 >> 172.21.15.97:0/1806622742 conn(0x7f2644103570 msgr2=0x7f2644104150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:43:59.619 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:58.899+0000 7f264b000700 1 -- 172.21.15.97:0/1806622742 shutdown_connections 2024-03-27T10:43:59.619 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:43:58.899+0000 7f264b000700 1 -- 172.21.15.97:0/1806622742 wait complete. 2024-03-27T10:43:59.619 INFO:teuthology.orchestra.run.smithi097.stdout:Waiting for the mgr to restart... 2024-03-27T10:43:59.620 INFO:teuthology.orchestra.run.smithi097.stdout:Waiting for mgr epoch 5... 2024-03-27T10:43:59.871 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:43:59 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: 2024-03-27T10:43:59.858+0000 7fd2aef3c200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-27T10:44:00.143 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:44:00 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: 2024-03-27T10:44:00.007+0000 7fd2aef3c200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-27T10:44:00.144 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:44:00 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: 2024-03-27T10:44:00.127+0000 7fd2aef3c200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-27T10:44:00.460 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:44:00 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: 2024-03-27T10:44:00.257+0000 7fd2aef3c200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-27T10:44:00.785 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:44:00 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: 2024-03-27T10:44:00.486+0000 7fd2aef3c200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-27T10:44:02.144 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:44:01 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: 2024-03-27T10:44:01.772+0000 7fd2aef3c200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-27T10:44:02.144 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:44:01 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: 2024-03-27T10:44:01.940+0000 7fd2aef3c200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-27T10:44:02.643 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:44:02 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: 2024-03-27T10:44:02.350+0000 7fd2aef3c200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-27T10:44:02.644 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:44:02 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: 2024-03-27T10:44:02.467+0000 7fd2aef3c200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-27T10:44:03.143 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:44:02 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: 2024-03-27T10:44:02.718+0000 7fd2aef3c200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-27T10:44:03.144 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:44:02 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: 2024-03-27T10:44:02.956+0000 7fd2aef3c200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-27T10:44:03.608 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:44:03 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: 2024-03-27T10:44:03.321+0000 7fd2aef3c200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-27T10:44:03.608 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:44:03 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: 2024-03-27T10:44:03.633+0000 7fd2aef3c200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-27T10:44:04.393 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:44:03 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: 2024-03-27T10:44:03.930+0000 7fd2aef3c200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-27T10:44:05.393 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:44:04 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: 2024-03-27T10:44:04.929+0000 7fd2aef3c200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-27T10:44:05.894 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:44:05 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: 2024-03-27T10:44:05.615+0000 7fd2aef3c200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-27T10:44:05.894 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:44:05 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: 2024-03-27T10:44:05.740+0000 7fd2aef3c200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-27T10:44:06.393 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:44:06 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: 2024-03-27T10:44:06.016+0000 7fd2aef3c200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-27T10:44:06.394 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:44:06 smithi097 ceph-mon[31859]: Active manager daemon x restarted 2024-03-27T10:44:06.394 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:44:06 smithi097 ceph-mon[31859]: Activating manager daemon x 2024-03-27T10:44:06.394 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:44:06 smithi097 ceph-mon[31859]: osdmap e2: 0 total, 0 up, 0 in 2024-03-27T10:44:06.394 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:44:06 smithi097 ceph-mon[31859]: mgrmap e6: x(active, starting, since 0.00457814s) 2024-03-27T10:44:06.394 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:44:06 smithi097 ceph-mon[31859]: from='mgr.14120 172.21.15.97:0/4145774229' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-27T10:44:06.394 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:44:06 smithi097 ceph-mon[31859]: from='mgr.14120 172.21.15.97:0/4145774229' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-27T10:44:06.394 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:44:06 smithi097 ceph-mon[31859]: from='mgr.14120 172.21.15.97:0/4145774229' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-27T10:44:06.394 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:44:06 smithi097 ceph-mon[31859]: from='mgr.14120 172.21.15.97:0/4145774229' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-27T10:44:06.394 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:44:06 smithi097 ceph-mon[31859]: from='mgr.14120 172.21.15.97:0/4145774229' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-27T10:44:06.395 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:44:06 smithi097 ceph-mon[31859]: Manager daemon x is now available 2024-03-27T10:44:07.394 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:44:07 smithi097 ceph-mon[31859]: Found migration_current of "None". Setting to last migration. 2024-03-27T10:44:07.394 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:44:07 smithi097 ceph-mon[31859]: from='mgr.14120 172.21.15.97:0/4145774229' entity='mgr.x' 2024-03-27T10:44:07.394 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:44:07 smithi097 ceph-mon[31859]: from='mgr.14120 172.21.15.97:0/4145774229' entity='mgr.x' 2024-03-27T10:44:07.394 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:44:07 smithi097 ceph-mon[31859]: from='mgr.14120 172.21.15.97:0/4145774229' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:44:07.394 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:44:07 smithi097 ceph-mon[31859]: from='mgr.14120 172.21.15.97:0/4145774229' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:44:07.394 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:44:07 smithi097 ceph-mon[31859]: from='mgr.14120 172.21.15.97:0/4145774229' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-27T10:44:07.394 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:44:07 smithi097 ceph-mon[31859]: from='mgr.14120 172.21.15.97:0/4145774229' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-27T10:44:07.394 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:44:07 smithi097 ceph-mon[31859]: from='mgr.14120 172.21.15.97:0/4145774229' entity='mgr.x' 2024-03-27T10:44:07.394 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:44:07 smithi097 ceph-mon[31859]: from='mgr.14120 172.21.15.97:0/4145774229' entity='mgr.x' 2024-03-27T10:44:07.395 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:44:07 smithi097 ceph-mon[31859]: mgrmap e7: x(active, since 1.01391s) 2024-03-27T10:44:07.715 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout { 2024-03-27T10:44:07.718 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-03-27T10:44:07.718 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-27T10:44:07.718 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout } 2024-03-27T10:44:07.718 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:00.199+0000 7f971ec00700 1 Processor -- start 2024-03-27T10:44:07.718 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:00.200+0000 7f971ec00700 1 -- start start 2024-03-27T10:44:07.718 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:00.200+0000 7f971ec00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97181089f0 0x7f971810ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:44:07.718 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:00.200+0000 7f971ec00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f971807bed0 con 0x7f97181089f0 2024-03-27T10:44:07.718 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:00.200+0000 7f9717e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97181089f0 0x7f971810ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:44:07.718 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:00.200+0000 7f9717e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97181089f0 0x7f971810ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54206/0 (socket says 172.21.15.97:54206) 2024-03-27T10:44:07.718 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:00.200+0000 7f9717e00700 1 -- 172.21.15.97:0/1929400603 learned_addr learned my addr 172.21.15.97:0/1929400603 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:44:07.718 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:00.201+0000 7f9717e00700 1 -- 172.21.15.97:0/1929400603 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f971810b320 con 0x7f97181089f0 2024-03-27T10:44:07.719 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:00.201+0000 7f9717e00700 1 --2- 172.21.15.97:0/1929400603 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97181089f0 0x7f971810ade0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f97080098a0 tx=0x7f9708009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1853af8f5b078896 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:44:07.719 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:00.202+0000 7f9716a00700 1 -- 172.21.15.97:0/1929400603 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9708004160 con 0x7f97181089f0 2024-03-27T10:44:07.719 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:00.202+0000 7f9716a00700 1 -- 172.21.15.97:0/1929400603 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f9708004300 con 0x7f97181089f0 2024-03-27T10:44:07.719 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:00.202+0000 7f9716a00700 1 -- 172.21.15.97:0/1929400603 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9708011a80 con 0x7f97181089f0 2024-03-27T10:44:07.719 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:00.202+0000 7f971ec00700 1 -- 172.21.15.97:0/1929400603 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97181089f0 msgr2=0x7f971810ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:44:07.719 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:00.202+0000 7f971ec00700 1 --2- 172.21.15.97:0/1929400603 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97181089f0 0x7f971810ade0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f97080098a0 tx=0x7f9708009bb0 comp rx=0 tx=0).stop 2024-03-27T10:44:07.719 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:00.203+0000 7f971ec00700 1 -- 172.21.15.97:0/1929400603 shutdown_connections 2024-03-27T10:44:07.719 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:00.203+0000 7f971ec00700 1 --2- 172.21.15.97:0/1929400603 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97181089f0 0x7f971810ade0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:44:07.719 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:00.203+0000 7f971ec00700 1 -- 172.21.15.97:0/1929400603 >> 172.21.15.97:0/1929400603 conn(0x7f97181023e0 msgr2=0x7f9718104860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:44:07.719 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:00.203+0000 7f971ec00700 1 -- 172.21.15.97:0/1929400603 shutdown_connections 2024-03-27T10:44:07.719 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:00.203+0000 7f971ec00700 1 -- 172.21.15.97:0/1929400603 wait complete. 2024-03-27T10:44:07.724 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:00.204+0000 7f971ec00700 1 Processor -- start 2024-03-27T10:44:07.724 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:00.204+0000 7f971ec00700 1 -- start start 2024-03-27T10:44:07.724 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:00.205+0000 7f971ec00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97181089f0 0x7f971819e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:44:07.724 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:00.205+0000 7f971ec00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f971810c2f0 con 0x7f97181089f0 2024-03-27T10:44:07.725 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:00.205+0000 7f9717e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97181089f0 0x7f971819e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:44:07.725 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:00.205+0000 7f9717e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97181089f0 0x7f971819e1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54212/0 (socket says 172.21.15.97:54212) 2024-03-27T10:44:07.725 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:00.205+0000 7f9717e00700 1 -- 172.21.15.97:0/3783642320 learned_addr learned my addr 172.21.15.97:0/3783642320 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:44:07.725 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:00.205+0000 7f9717e00700 1 -- 172.21.15.97:0/3783642320 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f971819e710 con 0x7f97181089f0 2024-03-27T10:44:07.725 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:00.206+0000 7f9717e00700 1 --2- 172.21.15.97:0/3783642320 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97181089f0 0x7f971819e1d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9708011a50 tx=0x7f9708004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:44:07.725 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:00.206+0000 7f9714c00700 1 -- 172.21.15.97:0/3783642320 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f97080039f0 con 0x7f97181089f0 2024-03-27T10:44:07.725 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:00.206+0000 7f9714c00700 1 -- 172.21.15.97:0/3783642320 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f9708003b90 con 0x7f97181089f0 2024-03-27T10:44:07.725 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:00.206+0000 7f971ec00700 1 -- 172.21.15.97:0/3783642320 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f971819e9a0 con 0x7f97181089f0 2024-03-27T10:44:07.725 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:00.206+0000 7f9714c00700 1 -- 172.21.15.97:0/3783642320 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f970801a600 con 0x7f97181089f0 2024-03-27T10:44:07.725 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:00.206+0000 7f971ec00700 1 -- 172.21.15.97:0/3783642320 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97181a1690 con 0x7f97181089f0 2024-03-27T10:44:07.725 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:00.207+0000 7f9714c00700 1 -- 172.21.15.97:0/3783642320 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 46466+0+0 (secure 0 0 0) 0x7f970801a910 con 0x7f97181089f0 2024-03-27T10:44:07.725 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:00.208+0000 7f9714c00700 1 --2- 172.21.15.97:0/3783642320 >> [v2:172.21.15.97:6800/98682155,v1:172.21.15.97:6801/98682155] conn(0x7f96fc039420 0x7f96fc03b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:44:07.726 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:00.208+0000 7f9714c00700 1 -- 172.21.15.97:0/3783642320 --> [v2:172.21.15.97:6800/98682155,v1:172.21.15.97:6801/98682155] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f96fc03bff0 con 0x7f96fc039420 2024-03-27T10:44:07.726 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:00.208+0000 7f9714c00700 1 -- 172.21.15.97:0/3783642320 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f970804d840 con 0x7f97181089f0 2024-03-27T10:44:07.726 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:00.208+0000 7f9717400700 1 -- 172.21.15.97:0/3783642320 >> [v2:172.21.15.97:6800/98682155,v1:172.21.15.97:6801/98682155] conn(0x7f96fc039420 msgr2=0x7f96fc03b8e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.97:6800/98682155 2024-03-27T10:44:07.726 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:00.208+0000 7f9717400700 1 --2- 172.21.15.97:0/3783642320 >> [v2:172.21.15.97:6800/98682155,v1:172.21.15.97:6801/98682155] conn(0x7f96fc039420 0x7f96fc03b8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-27T10:44:07.726 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:00.409+0000 7f9717400700 1 -- 172.21.15.97:0/3783642320 >> [v2:172.21.15.97:6800/98682155,v1:172.21.15.97:6801/98682155] conn(0x7f96fc039420 msgr2=0x7f96fc03b8e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.97:6800/98682155 2024-03-27T10:44:07.726 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:00.409+0000 7f9717400700 1 --2- 172.21.15.97:0/3783642320 >> [v2:172.21.15.97:6800/98682155,v1:172.21.15.97:6801/98682155] conn(0x7f96fc039420 0x7f96fc03b8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-27T10:44:07.726 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:00.810+0000 7f9717400700 1 -- 172.21.15.97:0/3783642320 >> [v2:172.21.15.97:6800/98682155,v1:172.21.15.97:6801/98682155] conn(0x7f96fc039420 msgr2=0x7f96fc03b8e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.97:6800/98682155 2024-03-27T10:44:07.726 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:00.810+0000 7f9717400700 1 --2- 172.21.15.97:0/3783642320 >> [v2:172.21.15.97:6800/98682155,v1:172.21.15.97:6801/98682155] conn(0x7f96fc039420 0x7f96fc03b8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-27T10:44:07.726 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:01.611+0000 7f9717400700 1 -- 172.21.15.97:0/3783642320 >> [v2:172.21.15.97:6800/98682155,v1:172.21.15.97:6801/98682155] conn(0x7f96fc039420 msgr2=0x7f96fc03b8e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.97:6800/98682155 2024-03-27T10:44:07.726 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:01.611+0000 7f9717400700 1 --2- 172.21.15.97:0/3783642320 >> [v2:172.21.15.97:6800/98682155,v1:172.21.15.97:6801/98682155] conn(0x7f96fc039420 0x7f96fc03b8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-27T10:44:07.726 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:03.213+0000 7f9717400700 1 -- 172.21.15.97:0/3783642320 >> [v2:172.21.15.97:6800/98682155,v1:172.21.15.97:6801/98682155] conn(0x7f96fc039420 msgr2=0x7f96fc03b8e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.97:6800/98682155 2024-03-27T10:44:07.726 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:03.213+0000 7f9717400700 1 --2- 172.21.15.97:0/3783642320 >> [v2:172.21.15.97:6800/98682155,v1:172.21.15.97:6801/98682155] conn(0x7f96fc039420 0x7f96fc03b8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-27T10:44:07.726 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:06.023+0000 7f9714c00700 1 -- 172.21.15.97:0/3783642320 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mgrmap(e 6) ==== 46233+0+0 (secure 0 0 0) 0x7f9708024a40 con 0x7f97181089f0 2024-03-27T10:44:07.727 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:06.024+0000 7f9714c00700 1 -- 172.21.15.97:0/3783642320 >> [v2:172.21.15.97:6800/98682155,v1:172.21.15.97:6801/98682155] conn(0x7f96fc039420 msgr2=0x7f96fc03b8e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-27T10:44:07.727 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:06.024+0000 7f9714c00700 1 --2- 172.21.15.97:0/3783642320 >> [v2:172.21.15.97:6800/98682155,v1:172.21.15.97:6801/98682155] conn(0x7f96fc039420 0x7f96fc03b8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:44:07.727 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:07.027+0000 7f9714c00700 1 -- 172.21.15.97:0/3783642320 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mgrmap(e 7) ==== 46360+0+0 (secure 0 0 0) 0x7f970804f680 con 0x7f97181089f0 2024-03-27T10:44:07.727 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:07.028+0000 7f9714c00700 1 --2- 172.21.15.97:0/3783642320 >> [v2:172.21.15.97:6800/1685921853,v1:172.21.15.97:6801/1685921853] conn(0x7f96fc03cd10 0x7f96fc03f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:44:07.727 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:07.028+0000 7f9714c00700 1 -- 172.21.15.97:0/3783642320 --> [v2:172.21.15.97:6800/1685921853,v1:172.21.15.97:6801/1685921853] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f970802ee50 con 0x7f96fc03cd10 2024-03-27T10:44:07.727 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:07.030+0000 7f9717400700 1 --2- 172.21.15.97:0/3783642320 >> [v2:172.21.15.97:6800/1685921853,v1:172.21.15.97:6801/1685921853] conn(0x7f96fc03cd10 0x7f96fc03f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:44:07.727 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:07.032+0000 7f9717400700 1 --2- 172.21.15.97:0/3783642320 >> [v2:172.21.15.97:6800/1685921853,v1:172.21.15.97:6801/1685921853] conn(0x7f96fc03cd10 0x7f96fc03f100 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f9704003a10 tx=0x7f97040092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:44:07.727 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:07.035+0000 7f9714c00700 1 -- 172.21.15.97:0/3783642320 <== mgr.14120 v2:172.21.15.97:6800/1685921853 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f970802ee50 con 0x7f96fc03cd10 2024-03-27T10:44:07.727 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:07.042+0000 7f971ec00700 1 -- 172.21.15.97:0/3783642320 --> [v2:172.21.15.97:6800/1685921853,v1:172.21.15.97:6801/1685921853] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f9718063c70 con 0x7f96fc03cd10 2024-03-27T10:44:07.727 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:07.043+0000 7f9714c00700 1 -- 172.21.15.97:0/3783642320 <== mgr.14120 v2:172.21.15.97:6800/1685921853 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f9718063c70 con 0x7f96fc03cd10 2024-03-27T10:44:07.727 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:07.043+0000 7f971ec00700 1 -- 172.21.15.97:0/3783642320 >> [v2:172.21.15.97:6800/1685921853,v1:172.21.15.97:6801/1685921853] conn(0x7f96fc03cd10 msgr2=0x7f96fc03f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:44:07.727 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:07.043+0000 7f971ec00700 1 --2- 172.21.15.97:0/3783642320 >> [v2:172.21.15.97:6800/1685921853,v1:172.21.15.97:6801/1685921853] conn(0x7f96fc03cd10 0x7f96fc03f100 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f9704003a10 tx=0x7f97040092b0 comp rx=0 tx=0).stop 2024-03-27T10:44:07.728 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:07.043+0000 7f971ec00700 1 -- 172.21.15.97:0/3783642320 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97181089f0 msgr2=0x7f971819e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:44:07.728 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:07.043+0000 7f971ec00700 1 --2- 172.21.15.97:0/3783642320 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97181089f0 0x7f971819e1d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9708011a50 tx=0x7f9708004290 comp rx=0 tx=0).stop 2024-03-27T10:44:07.728 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:07.043+0000 7f971ec00700 1 -- 172.21.15.97:0/3783642320 shutdown_connections 2024-03-27T10:44:07.728 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:07.043+0000 7f971ec00700 1 --2- 172.21.15.97:0/3783642320 >> [v2:172.21.15.97:6800/1685921853,v1:172.21.15.97:6801/1685921853] conn(0x7f96fc03cd10 0x7f96fc03f100 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:44:07.728 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:07.043+0000 7f971ec00700 1 --2- 172.21.15.97:0/3783642320 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97181089f0 0x7f971819e1d0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:44:07.728 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:07.043+0000 7f971ec00700 1 --2- 172.21.15.97:0/3783642320 >> [v2:172.21.15.97:6800/98682155,v1:172.21.15.97:6801/98682155] conn(0x7f96fc039420 0x7f96fc03b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:44:07.728 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:07.043+0000 7f971ec00700 1 -- 172.21.15.97:0/3783642320 >> 172.21.15.97:0/3783642320 conn(0x7f97181023e0 msgr2=0x7f9718102fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:44:07.728 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:07.044+0000 7f971ec00700 1 -- 172.21.15.97:0/3783642320 shutdown_connections 2024-03-27T10:44:07.728 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:07.044+0000 7f971ec00700 1 -- 172.21.15.97:0/3783642320 wait complete. 2024-03-27T10:44:07.728 INFO:teuthology.orchestra.run.smithi097.stdout:mgr epoch 5 is available 2024-03-27T10:44:07.728 INFO:teuthology.orchestra.run.smithi097.stdout:Setting orchestrator backend to cephadm... 2024-03-27T10:44:08.348 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:44:08 smithi097 ceph-mon[31859]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-27T10:44:08.348 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:44:08 smithi097 ceph-mon[31859]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-27T10:44:08.349 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:44:08 smithi097 ceph-mon[31859]: [27/Mar/2024:10:44:07] ENGINE Bus STARTING 2024-03-27T10:44:08.349 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:44:08 smithi097 ceph-mon[31859]: [27/Mar/2024:10:44:07] ENGINE Serving on http://172.21.15.97:8765 2024-03-27T10:44:08.349 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:44:08 smithi097 ceph-mon[31859]: [27/Mar/2024:10:44:07] ENGINE Serving on https://172.21.15.97:7150 2024-03-27T10:44:08.349 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:44:08 smithi097 ceph-mon[31859]: [27/Mar/2024:10:44:07] ENGINE Bus STARTED 2024-03-27T10:44:08.349 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:44:08 smithi097 ceph-mon[31859]: from='mgr.14120 172.21.15.97:0/4145774229' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:44:09.320 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:44:09 smithi097 ceph-mon[31859]: mgrmap e8: x(active, since 2s) 2024-03-27T10:44:09.321 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:44:09 smithi097 ceph-mon[31859]: from='mgr.14120 172.21.15.97:0/4145774229' entity='mgr.x' 2024-03-27T10:44:09.321 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:44:09 smithi097 ceph-mon[31859]: from='mgr.14120 172.21.15.97:0/4145774229' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:44:09.334 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:08.373+0000 7fbef1a00700 1 Processor -- start 2024-03-27T10:44:09.334 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:08.373+0000 7fbef1a00700 1 -- start start 2024-03-27T10:44:09.335 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:08.374+0000 7fbef1a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbeec07dec0 0x7fbeec07c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:44:09.335 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:08.374+0000 7fbef1a00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fbeec07c8f0 con 0x7fbeec07dec0 2024-03-27T10:44:09.335 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:08.374+0000 7fbeeaa00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbeec07dec0 0x7fbeec07c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:44:09.335 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:08.374+0000 7fbeeaa00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbeec07dec0 0x7fbeec07c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54278/0 (socket says 172.21.15.97:54278) 2024-03-27T10:44:09.335 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:08.374+0000 7fbeeaa00700 1 -- 172.21.15.97:0/3280789700 learned_addr learned my addr 172.21.15.97:0/3280789700 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:44:09.335 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:08.375+0000 7fbeeaa00700 1 -- 172.21.15.97:0/3280789700 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbeec07ca70 con 0x7fbeec07dec0 2024-03-27T10:44:09.335 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:08.375+0000 7fbeeaa00700 1 --2- 172.21.15.97:0/3280789700 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbeec07dec0 0x7fbeec07c320 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fbedc009e80 tx=0x7fbedc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31bd652c4b85baaf server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:44:09.335 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:08.376+0000 7fbee9600700 1 -- 172.21.15.97:0/3280789700 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbedc017070 con 0x7fbeec07dec0 2024-03-27T10:44:09.335 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:08.376+0000 7fbee9600700 1 -- 172.21.15.97:0/3280789700 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fbedc00b8c0 con 0x7fbeec07dec0 2024-03-27T10:44:09.335 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:08.376+0000 7fbee9600700 1 -- 172.21.15.97:0/3280789700 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbedc004520 con 0x7fbeec07dec0 2024-03-27T10:44:09.335 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:08.376+0000 7fbef1a00700 1 -- 172.21.15.97:0/3280789700 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbeec07dec0 msgr2=0x7fbeec07c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:44:09.335 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:08.376+0000 7fbef1a00700 1 --2- 172.21.15.97:0/3280789700 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbeec07dec0 0x7fbeec07c320 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fbedc009e80 tx=0x7fbedc00b330 comp rx=0 tx=0).stop 2024-03-27T10:44:09.336 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:08.376+0000 7fbef1a00700 1 -- 172.21.15.97:0/3280789700 shutdown_connections 2024-03-27T10:44:09.336 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:08.376+0000 7fbef1a00700 1 --2- 172.21.15.97:0/3280789700 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbeec07dec0 0x7fbeec07c320 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:44:09.336 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:08.376+0000 7fbef1a00700 1 -- 172.21.15.97:0/3280789700 >> 172.21.15.97:0/3280789700 conn(0x7fbeec103570 msgr2=0x7fbeec1059d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:44:09.336 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:08.377+0000 7fbef1a00700 1 -- 172.21.15.97:0/3280789700 shutdown_connections 2024-03-27T10:44:09.336 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:08.377+0000 7fbef1a00700 1 -- 172.21.15.97:0/3280789700 wait complete. 2024-03-27T10:44:09.336 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:08.378+0000 7fbef1a00700 1 Processor -- start 2024-03-27T10:44:09.336 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:08.378+0000 7fbef1a00700 1 -- start start 2024-03-27T10:44:09.336 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:08.378+0000 7fbef1a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbeec07dec0 0x7fbeec19e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:44:09.336 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:08.378+0000 7fbef1a00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fbeec10e540 con 0x7fbeec07dec0 2024-03-27T10:44:09.336 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:08.378+0000 7fbeeaa00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbeec07dec0 0x7fbeec19e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:44:09.336 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:08.378+0000 7fbeeaa00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbeec07dec0 0x7fbeec19e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54294/0 (socket says 172.21.15.97:54294) 2024-03-27T10:44:09.336 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:08.378+0000 7fbeeaa00700 1 -- 172.21.15.97:0/3841879333 learned_addr learned my addr 172.21.15.97:0/3841879333 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:44:09.337 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:08.379+0000 7fbeeaa00700 1 -- 172.21.15.97:0/3841879333 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbeec19e6e0 con 0x7fbeec07dec0 2024-03-27T10:44:09.337 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:08.379+0000 7fbeeaa00700 1 --2- 172.21.15.97:0/3841879333 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbeec07dec0 0x7fbeec19e1a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fbedc000f80 tx=0x7fbedc009640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:44:09.337 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:08.379+0000 7fbee3400700 1 -- 172.21.15.97:0/3841879333 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbedc020020 con 0x7fbeec07dec0 2024-03-27T10:44:09.337 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:08.379+0000 7fbee3400700 1 -- 172.21.15.97:0/3841879333 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fbedc00bba0 con 0x7fbeec07dec0 2024-03-27T10:44:09.337 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:08.379+0000 7fbee3400700 1 -- 172.21.15.97:0/3841879333 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbedc017040 con 0x7fbeec07dec0 2024-03-27T10:44:09.337 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:08.379+0000 7fbef1a00700 1 -- 172.21.15.97:0/3841879333 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbeec19e970 con 0x7fbeec07dec0 2024-03-27T10:44:09.337 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:08.379+0000 7fbef1a00700 1 -- 172.21.15.97:0/3841879333 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbeec1a1660 con 0x7fbeec07dec0 2024-03-27T10:44:09.337 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:08.380+0000 7fbee3400700 1 -- 172.21.15.97:0/3841879333 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) ==== 46466+0+0 (secure 0 0 0) 0x7fbedc023020 con 0x7fbeec07dec0 2024-03-27T10:44:09.337 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:08.380+0000 7fbef1a00700 1 -- 172.21.15.97:0/3841879333 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbeec063c70 con 0x7fbeec07dec0 2024-03-27T10:44:09.337 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:08.381+0000 7fbee3400700 1 --2- 172.21.15.97:0/3841879333 >> [v2:172.21.15.97:6800/1685921853,v1:172.21.15.97:6801/1685921853] conn(0x7fbed00393d0 0x7fbed003b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:44:09.337 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:08.381+0000 7fbee3400700 1 -- 172.21.15.97:0/3841879333 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fbedc00f050 con 0x7fbeec07dec0 2024-03-27T10:44:09.337 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:08.381+0000 7fbeea000700 1 --2- 172.21.15.97:0/3841879333 >> [v2:172.21.15.97:6800/1685921853,v1:172.21.15.97:6801/1685921853] conn(0x7fbed00393d0 0x7fbed003b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:44:09.337 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:08.382+0000 7fbeea000700 1 --2- 172.21.15.97:0/3841879333 >> [v2:172.21.15.97:6800/1685921853,v1:172.21.15.97:6801/1685921853] conn(0x7fbed00393d0 0x7fbed003b890 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbed8006fd0 tx=0x7fbed8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:44:09.338 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:08.387+0000 7fbee3400700 1 -- 172.21.15.97:0/3841879333 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7fbedc02a020 con 0x7fbeec07dec0 2024-03-27T10:44:09.338 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:08.611+0000 7fbef1a00700 1 -- 172.21.15.97:0/3841879333 --> [v2:172.21.15.97:6800/1685921853,v1:172.21.15.97:6801/1685921853] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7fbeec1048c0 con 0x7fbed00393d0 2024-03-27T10:44:09.338 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:08.623+0000 7fbee3400700 1 -- 172.21.15.97:0/3841879333 <== mgr.14120 v2:172.21.15.97:6800/1685921853 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fbeec1048c0 con 0x7fbed00393d0 2024-03-27T10:44:09.338 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:08.626+0000 7fbef1a00700 1 -- 172.21.15.97:0/3841879333 >> [v2:172.21.15.97:6800/1685921853,v1:172.21.15.97:6801/1685921853] conn(0x7fbed00393d0 msgr2=0x7fbed003b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:44:09.338 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:08.626+0000 7fbef1a00700 1 --2- 172.21.15.97:0/3841879333 >> [v2:172.21.15.97:6800/1685921853,v1:172.21.15.97:6801/1685921853] conn(0x7fbed00393d0 0x7fbed003b890 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbed8006fd0 tx=0x7fbed8006e40 comp rx=0 tx=0).stop 2024-03-27T10:44:09.338 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:08.626+0000 7fbef1a00700 1 -- 172.21.15.97:0/3841879333 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbeec07dec0 msgr2=0x7fbeec19e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:44:09.338 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:08.626+0000 7fbef1a00700 1 --2- 172.21.15.97:0/3841879333 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbeec07dec0 0x7fbeec19e1a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fbedc000f80 tx=0x7fbedc009640 comp rx=0 tx=0).stop 2024-03-27T10:44:09.338 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:08.626+0000 7fbef1a00700 1 -- 172.21.15.97:0/3841879333 shutdown_connections 2024-03-27T10:44:09.338 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:08.626+0000 7fbef1a00700 1 --2- 172.21.15.97:0/3841879333 >> [v2:172.21.15.97:6800/1685921853,v1:172.21.15.97:6801/1685921853] conn(0x7fbed00393d0 0x7fbed003b890 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:44:09.338 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:08.626+0000 7fbef1a00700 1 --2- 172.21.15.97:0/3841879333 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbeec07dec0 0x7fbeec19e1a0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:44:09.338 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:08.626+0000 7fbef1a00700 1 -- 172.21.15.97:0/3841879333 >> 172.21.15.97:0/3841879333 conn(0x7fbeec103570 msgr2=0x7fbeec104190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:44:09.338 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:08.627+0000 7fbef1a00700 1 -- 172.21.15.97:0/3841879333 shutdown_connections 2024-03-27T10:44:09.339 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:08.627+0000 7fbef1a00700 1 -- 172.21.15.97:0/3841879333 wait complete. 2024-03-27T10:44:10.143 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:44:10 smithi097 ceph-mon[31859]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T10:44:10.816 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout value unchanged 2024-03-27T10:44:10.816 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:09.910+0000 7fb3ea000700 1 Processor -- start 2024-03-27T10:44:10.816 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:09.910+0000 7fb3ea000700 1 -- start start 2024-03-27T10:44:10.816 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:09.910+0000 7fb3ea000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3e410a210 0x7fb3e410a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:44:10.817 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:09.910+0000 7fb3ea000700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb3e410ac00 con 0x7fb3e410a210 2024-03-27T10:44:10.817 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:09.911+0000 7fb3e3400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3e410a210 0x7fb3e410a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:44:10.817 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:09.911+0000 7fb3e3400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3e410a210 0x7fb3e410a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52370/0 (socket says 172.21.15.97:52370) 2024-03-27T10:44:10.817 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:09.911+0000 7fb3e3400700 1 -- 172.21.15.97:0/2935134891 learned_addr learned my addr 172.21.15.97:0/2935134891 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:44:10.817 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:09.911+0000 7fb3e3400700 1 -- 172.21.15.97:0/2935134891 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3e410b3c0 con 0x7fb3e410a210 2024-03-27T10:44:10.817 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:09.912+0000 7fb3e3400700 1 --2- 172.21.15.97:0/2935134891 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3e410a210 0x7fb3e410a630 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb3d40098a0 tx=0x7fb3d4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ad9af9b87cbaf6b server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:44:10.817 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:09.912+0000 7fb3e2000700 1 -- 172.21.15.97:0/2935134891 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb3d4004160 con 0x7fb3e410a210 2024-03-27T10:44:10.817 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:09.912+0000 7fb3e2000700 1 -- 172.21.15.97:0/2935134891 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fb3d4004300 con 0x7fb3e410a210 2024-03-27T10:44:10.817 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:09.912+0000 7fb3e2000700 1 -- 172.21.15.97:0/2935134891 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb3d4011a80 con 0x7fb3e410a210 2024-03-27T10:44:10.817 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:09.913+0000 7fb3ea000700 1 -- 172.21.15.97:0/2935134891 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3e410a210 msgr2=0x7fb3e410a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:44:10.817 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:09.913+0000 7fb3ea000700 1 --2- 172.21.15.97:0/2935134891 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3e410a210 0x7fb3e410a630 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb3d40098a0 tx=0x7fb3d4009bb0 comp rx=0 tx=0).stop 2024-03-27T10:44:10.817 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:09.913+0000 7fb3ea000700 1 -- 172.21.15.97:0/2935134891 shutdown_connections 2024-03-27T10:44:10.818 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:09.913+0000 7fb3ea000700 1 --2- 172.21.15.97:0/2935134891 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3e410a210 0x7fb3e410a630 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:44:10.818 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:09.913+0000 7fb3ea000700 1 -- 172.21.15.97:0/2935134891 >> 172.21.15.97:0/2935134891 conn(0x7fb3e407d590 msgr2=0x7fb3e407d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:44:10.818 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:09.913+0000 7fb3ea000700 1 -- 172.21.15.97:0/2935134891 shutdown_connections 2024-03-27T10:44:10.818 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:09.913+0000 7fb3ea000700 1 -- 172.21.15.97:0/2935134891 wait complete. 2024-03-27T10:44:10.818 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:09.914+0000 7fb3ea000700 1 Processor -- start 2024-03-27T10:44:10.818 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:09.914+0000 7fb3ea000700 1 -- start start 2024-03-27T10:44:10.818 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:09.915+0000 7fb3ea000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3e410a210 0x7fb3e419e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:44:10.818 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:09.915+0000 7fb3ea000700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb3e410e770 con 0x7fb3e410a210 2024-03-27T10:44:10.818 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:09.915+0000 7fb3e3400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3e410a210 0x7fb3e419e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:44:10.818 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:09.915+0000 7fb3e3400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3e410a210 0x7fb3e419e1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52380/0 (socket says 172.21.15.97:52380) 2024-03-27T10:44:10.818 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:09.915+0000 7fb3e3400700 1 -- 172.21.15.97:0/3673703457 learned_addr learned my addr 172.21.15.97:0/3673703457 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:44:10.818 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:09.915+0000 7fb3e3400700 1 -- 172.21.15.97:0/3673703457 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3e419e710 con 0x7fb3e410a210 2024-03-27T10:44:10.818 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:09.915+0000 7fb3e3400700 1 --2- 172.21.15.97:0/3673703457 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3e410a210 0x7fb3e419e1d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb3d4000c00 tx=0x7fb3d4004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:44:10.819 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:09.916+0000 7fb3dbe00700 1 -- 172.21.15.97:0/3673703457 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb3d40039c0 con 0x7fb3e410a210 2024-03-27T10:44:10.819 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:09.916+0000 7fb3dbe00700 1 -- 172.21.15.97:0/3673703457 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fb3d4003b60 con 0x7fb3e410a210 2024-03-27T10:44:10.819 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:09.916+0000 7fb3dbe00700 1 -- 172.21.15.97:0/3673703457 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb3d401a6f0 con 0x7fb3e410a210 2024-03-27T10:44:10.819 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:09.916+0000 7fb3ea000700 1 -- 172.21.15.97:0/3673703457 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3e419e9a0 con 0x7fb3e410a210 2024-03-27T10:44:10.819 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:09.916+0000 7fb3ea000700 1 -- 172.21.15.97:0/3673703457 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3e41a1690 con 0x7fb3e410a210 2024-03-27T10:44:10.819 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:09.917+0000 7fb3dbe00700 1 -- 172.21.15.97:0/3673703457 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) ==== 46466+0+0 (secure 0 0 0) 0x7fb3d4024a40 con 0x7fb3e410a210 2024-03-27T10:44:10.819 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:09.917+0000 7fb3ea000700 1 -- 172.21.15.97:0/3673703457 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3e4063c70 con 0x7fb3e410a210 2024-03-27T10:44:10.819 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:09.917+0000 7fb3dbe00700 1 --2- 172.21.15.97:0/3673703457 >> [v2:172.21.15.97:6800/1685921853,v1:172.21.15.97:6801/1685921853] conn(0x7fb3c8039420 0x7fb3c803b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:44:10.819 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:09.918+0000 7fb3dbe00700 1 -- 172.21.15.97:0/3673703457 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb3d404d800 con 0x7fb3e410a210 2024-03-27T10:44:10.821 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:09.918+0000 7fb3e2a00700 1 --2- 172.21.15.97:0/3673703457 >> [v2:172.21.15.97:6800/1685921853,v1:172.21.15.97:6801/1685921853] conn(0x7fb3c8039420 0x7fb3c803b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:44:10.821 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:09.918+0000 7fb3e2a00700 1 --2- 172.21.15.97:0/3673703457 >> [v2:172.21.15.97:6800/1685921853,v1:172.21.15.97:6801/1685921853] conn(0x7fb3c8039420 0x7fb3c803b8e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb3d0006fd0 tx=0x7fb3d0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:44:10.821 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:09.923+0000 7fb3dbe00700 1 -- 172.21.15.97:0/3673703457 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7fb3d400f2e0 con 0x7fb3e410a210 2024-03-27T10:44:10.821 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:10.149+0000 7fb3ea000700 1 -- 172.21.15.97:0/3673703457 --> [v2:172.21.15.97:6800/1685921853,v1:172.21.15.97:6801/1685921853] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7fb3e4107d30 con 0x7fb3c8039420 2024-03-27T10:44:10.821 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:10.150+0000 7fb3dbe00700 1 -- 172.21.15.97:0/3673703457 <== mgr.14120 v2:172.21.15.97:6800/1685921853 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7fb3e4107d30 con 0x7fb3c8039420 2024-03-27T10:44:10.821 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:10.153+0000 7fb3ea000700 1 -- 172.21.15.97:0/3673703457 >> [v2:172.21.15.97:6800/1685921853,v1:172.21.15.97:6801/1685921853] conn(0x7fb3c8039420 msgr2=0x7fb3c803b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:44:10.821 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:10.153+0000 7fb3ea000700 1 --2- 172.21.15.97:0/3673703457 >> [v2:172.21.15.97:6800/1685921853,v1:172.21.15.97:6801/1685921853] conn(0x7fb3c8039420 0x7fb3c803b8e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb3d0006fd0 tx=0x7fb3d0006e40 comp rx=0 tx=0).stop 2024-03-27T10:44:10.821 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:10.153+0000 7fb3ea000700 1 -- 172.21.15.97:0/3673703457 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3e410a210 msgr2=0x7fb3e419e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:44:10.821 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:10.153+0000 7fb3ea000700 1 --2- 172.21.15.97:0/3673703457 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3e410a210 0x7fb3e419e1d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb3d4000c00 tx=0x7fb3d4004290 comp rx=0 tx=0).stop 2024-03-27T10:44:10.822 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:10.153+0000 7fb3ea000700 1 -- 172.21.15.97:0/3673703457 shutdown_connections 2024-03-27T10:44:10.822 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:10.153+0000 7fb3ea000700 1 --2- 172.21.15.97:0/3673703457 >> [v2:172.21.15.97:6800/1685921853,v1:172.21.15.97:6801/1685921853] conn(0x7fb3c8039420 0x7fb3c803b8e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:44:10.822 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:10.153+0000 7fb3ea000700 1 --2- 172.21.15.97:0/3673703457 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3e410a210 0x7fb3e419e1d0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:44:10.822 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:10.153+0000 7fb3ea000700 1 -- 172.21.15.97:0/3673703457 >> 172.21.15.97:0/3673703457 conn(0x7fb3e407d590 msgr2=0x7fb3e4107620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:44:10.822 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:10.153+0000 7fb3ea000700 1 -- 172.21.15.97:0/3673703457 shutdown_connections 2024-03-27T10:44:10.822 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:10.153+0000 7fb3ea000700 1 -- 172.21.15.97:0/3673703457 wait complete. 2024-03-27T10:44:10.822 INFO:teuthology.orchestra.run.smithi097.stdout:Generating ssh key... 2024-03-27T10:44:11.370 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:44:11 smithi097 ceph-mon[31859]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T10:44:12.372 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:44:12 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: Generating public/private rsa key pair. 2024-03-27T10:44:12.372 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:44:12 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: Your identification has been saved in /tmp/tmpvcb7b6ld/key. 2024-03-27T10:44:12.372 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:44:12 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: Your public key has been saved in /tmp/tmpvcb7b6ld/key.pub. 2024-03-27T10:44:12.373 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:44:12 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: The key fingerprint is: 2024-03-27T10:44:12.373 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:44:12 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: SHA256:M/gi7QR1G9xly6U6TNyhgtCu6mz2ffP98IHYmJYuUi8 ceph-adca88c4-ec26-11ee-95d0-87774f69a715 2024-03-27T10:44:12.373 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:44:12 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: The key's randomart image is: 2024-03-27T10:44:12.373 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:44:12 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: +---[RSA 3072]----+ 2024-03-27T10:44:12.373 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:44:12 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: | .. + . | 2024-03-27T10:44:12.373 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:44:12 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: | ..o o * = | 2024-03-27T10:44:12.373 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:44:12 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: | .o = = = | 2024-03-27T10:44:12.373 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:44:12 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: | ..o * . | 2024-03-27T10:44:12.373 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:44:12 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: | ... S + | 2024-03-27T10:44:12.373 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:44:12 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: | .o ..o B . | 2024-03-27T10:44:12.373 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:44:12 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: | .. +...* + . | 2024-03-27T10:44:12.373 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:44:12 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: | .+ =..Eo.. o . | 2024-03-27T10:44:12.374 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:44:12 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: | +o.. oo =o ..o | 2024-03-27T10:44:12.374 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:44:12 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: +----[SHA256]-----+ 2024-03-27T10:44:12.706 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:11.472+0000 7f8f79a00700 1 Processor -- start 2024-03-27T10:44:12.707 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:11.472+0000 7f8f79a00700 1 -- start start 2024-03-27T10:44:12.707 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:11.473+0000 7f8f79a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f7410ecf0 0x7f8f741110e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:44:12.707 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:11.473+0000 7f8f79a00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8f74111620 con 0x7f8f7410ecf0 2024-03-27T10:44:12.708 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:11.473+0000 7f8f73e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f7410ecf0 0x7f8f741110e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:44:12.708 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:11.473+0000 7f8f73e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f7410ecf0 0x7f8f741110e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52388/0 (socket says 172.21.15.97:52388) 2024-03-27T10:44:12.708 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:11.473+0000 7f8f73e00700 1 -- 172.21.15.97:0/1354436655 learned_addr learned my addr 172.21.15.97:0/1354436655 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:44:12.708 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:11.473+0000 7f8f73e00700 1 -- 172.21.15.97:0/1354436655 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f741117a0 con 0x7f8f7410ecf0 2024-03-27T10:44:12.709 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:11.474+0000 7f8f73e00700 1 --2- 172.21.15.97:0/1354436655 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f7410ecf0 0x7f8f741110e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8f640098a0 tx=0x7f8f64009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b8a9a24c2f396ba server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:44:12.709 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:11.474+0000 7f8f72a00700 1 -- 172.21.15.97:0/1354436655 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8f64004160 con 0x7f8f7410ecf0 2024-03-27T10:44:12.709 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:11.474+0000 7f8f72a00700 1 -- 172.21.15.97:0/1354436655 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f8f64004300 con 0x7f8f7410ecf0 2024-03-27T10:44:12.709 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:11.474+0000 7f8f72a00700 1 -- 172.21.15.97:0/1354436655 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8f64011a80 con 0x7f8f7410ecf0 2024-03-27T10:44:12.709 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:11.475+0000 7f8f79a00700 1 -- 172.21.15.97:0/1354436655 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f7410ecf0 msgr2=0x7f8f741110e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:44:12.710 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:11.475+0000 7f8f79a00700 1 --2- 172.21.15.97:0/1354436655 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f7410ecf0 0x7f8f741110e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8f640098a0 tx=0x7f8f64009bb0 comp rx=0 tx=0).stop 2024-03-27T10:44:12.710 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:11.475+0000 7f8f79a00700 1 -- 172.21.15.97:0/1354436655 shutdown_connections 2024-03-27T10:44:12.711 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:11.475+0000 7f8f79a00700 1 --2- 172.21.15.97:0/1354436655 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f7410ecf0 0x7f8f741110e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:44:12.711 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:11.475+0000 7f8f79a00700 1 -- 172.21.15.97:0/1354436655 >> 172.21.15.97:0/1354436655 conn(0x7f8f7407d410 msgr2=0x7f8f7407f830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:44:12.711 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:11.475+0000 7f8f79a00700 1 -- 172.21.15.97:0/1354436655 shutdown_connections 2024-03-27T10:44:12.711 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:11.475+0000 7f8f79a00700 1 -- 172.21.15.97:0/1354436655 wait complete. 2024-03-27T10:44:12.712 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:11.476+0000 7f8f79a00700 1 Processor -- start 2024-03-27T10:44:12.712 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:11.477+0000 7f8f79a00700 1 -- start start 2024-03-27T10:44:12.712 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:11.477+0000 7f8f79a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f7410ecf0 0x7f8f741a2370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:44:12.712 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:11.477+0000 7f8f79a00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8f74112770 con 0x7f8f7410ecf0 2024-03-27T10:44:12.712 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:11.477+0000 7f8f73e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f7410ecf0 0x7f8f741a2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:44:12.713 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:11.477+0000 7f8f73e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f7410ecf0 0x7f8f741a2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52402/0 (socket says 172.21.15.97:52402) 2024-03-27T10:44:12.713 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:11.477+0000 7f8f73e00700 1 -- 172.21.15.97:0/3302797309 learned_addr learned my addr 172.21.15.97:0/3302797309 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:44:12.713 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:11.477+0000 7f8f73e00700 1 -- 172.21.15.97:0/3302797309 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f741a28b0 con 0x7f8f7410ecf0 2024-03-27T10:44:12.713 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:11.478+0000 7f8f73e00700 1 --2- 172.21.15.97:0/3302797309 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f7410ecf0 0x7f8f741a2370 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8f64011a50 tx=0x7f8f64004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:44:12.713 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:11.478+0000 7f8f70c00700 1 -- 172.21.15.97:0/3302797309 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8f640039f0 con 0x7f8f7410ecf0 2024-03-27T10:44:12.713 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:11.478+0000 7f8f70c00700 1 -- 172.21.15.97:0/3302797309 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f8f64003b90 con 0x7f8f7410ecf0 2024-03-27T10:44:12.715 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:11.478+0000 7f8f70c00700 1 -- 172.21.15.97:0/3302797309 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8f6401a600 con 0x7f8f7410ecf0 2024-03-27T10:44:12.716 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:11.478+0000 7f8f79a00700 1 -- 172.21.15.97:0/3302797309 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f741a2b40 con 0x7f8f7410ecf0 2024-03-27T10:44:12.716 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:11.478+0000 7f8f79a00700 1 -- 172.21.15.97:0/3302797309 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f741a5830 con 0x7f8f7410ecf0 2024-03-27T10:44:12.716 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:11.479+0000 7f8f70c00700 1 -- 172.21.15.97:0/3302797309 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) ==== 46466+0+0 (secure 0 0 0) 0x7f8f64024a40 con 0x7f8f7410ecf0 2024-03-27T10:44:12.716 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:11.479+0000 7f8f79a00700 1 -- 172.21.15.97:0/3302797309 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f74063c70 con 0x7f8f7410ecf0 2024-03-27T10:44:12.716 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:11.480+0000 7f8f70c00700 1 --2- 172.21.15.97:0/3302797309 >> [v2:172.21.15.97:6800/1685921853,v1:172.21.15.97:6801/1685921853] conn(0x7f8f58039420 0x7f8f5803b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:44:12.716 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:11.480+0000 7f8f70c00700 1 -- 172.21.15.97:0/3302797309 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f8f6404d9b0 con 0x7f8f7410ecf0 2024-03-27T10:44:12.716 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:11.480+0000 7f8f73400700 1 --2- 172.21.15.97:0/3302797309 >> [v2:172.21.15.97:6800/1685921853,v1:172.21.15.97:6801/1685921853] conn(0x7f8f58039420 0x7f8f5803b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:44:12.716 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:11.481+0000 7f8f73400700 1 --2- 172.21.15.97:0/3302797309 >> [v2:172.21.15.97:6800/1685921853,v1:172.21.15.97:6801/1685921853] conn(0x7f8f58039420 0x7f8f5803b8e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8f60006fd0 tx=0x7f8f60006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:44:12.716 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:11.486+0000 7f8f70c00700 1 -- 172.21.15.97:0/3302797309 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f8f6404d300 con 0x7f8f7410ecf0 2024-03-27T10:44:12.716 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:11.716+0000 7f8f79a00700 1 -- 172.21.15.97:0/3302797309 --> [v2:172.21.15.97:6800/1685921853,v1:172.21.15.97:6801/1685921853] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f8f7407e6d0 con 0x7f8f58039420 2024-03-27T10:44:12.716 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:12.039+0000 7f8f70c00700 1 -- 172.21.15.97:0/3302797309 <== mgr.14120 v2:172.21.15.97:6800/1685921853 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8f7407e6d0 con 0x7f8f58039420 2024-03-27T10:44:12.716 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:12.042+0000 7f8f79a00700 1 -- 172.21.15.97:0/3302797309 >> [v2:172.21.15.97:6800/1685921853,v1:172.21.15.97:6801/1685921853] conn(0x7f8f58039420 msgr2=0x7f8f5803b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:44:12.717 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:12.042+0000 7f8f79a00700 1 --2- 172.21.15.97:0/3302797309 >> [v2:172.21.15.97:6800/1685921853,v1:172.21.15.97:6801/1685921853] conn(0x7f8f58039420 0x7f8f5803b8e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8f60006fd0 tx=0x7f8f60006e40 comp rx=0 tx=0).stop 2024-03-27T10:44:12.717 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:12.042+0000 7f8f79a00700 1 -- 172.21.15.97:0/3302797309 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f7410ecf0 msgr2=0x7f8f741a2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:44:12.717 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:12.042+0000 7f8f79a00700 1 --2- 172.21.15.97:0/3302797309 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f7410ecf0 0x7f8f741a2370 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8f64011a50 tx=0x7f8f64004290 comp rx=0 tx=0).stop 2024-03-27T10:44:12.717 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:12.042+0000 7f8f79a00700 1 -- 172.21.15.97:0/3302797309 shutdown_connections 2024-03-27T10:44:12.717 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:12.042+0000 7f8f79a00700 1 --2- 172.21.15.97:0/3302797309 >> [v2:172.21.15.97:6800/1685921853,v1:172.21.15.97:6801/1685921853] conn(0x7f8f58039420 0x7f8f5803b8e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:44:12.717 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:12.042+0000 7f8f79a00700 1 --2- 172.21.15.97:0/3302797309 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f7410ecf0 0x7f8f741a2370 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:44:12.717 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:12.043+0000 7f8f79a00700 1 -- 172.21.15.97:0/3302797309 >> 172.21.15.97:0/3302797309 conn(0x7f8f7407d410 msgr2=0x7f8f7407dfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:44:12.717 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:12.043+0000 7f8f79a00700 1 -- 172.21.15.97:0/3302797309 shutdown_connections 2024-03-27T10:44:12.717 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:12.043+0000 7f8f79a00700 1 -- 172.21.15.97:0/3302797309 wait complete. 2024-03-27T10:44:13.285 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:44:13 smithi097 ceph-mon[31859]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T10:44:13.285 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:44:13 smithi097 ceph-mon[31859]: Generating ssh key... 2024-03-27T10:44:13.285 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:44:13 smithi097 ceph-mon[31859]: from='mgr.14120 172.21.15.97:0/4145774229' entity='mgr.x' 2024-03-27T10:44:13.285 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:44:13 smithi097 ceph-mon[31859]: from='mgr.14120 172.21.15.97:0/4145774229' entity='mgr.x' 2024-03-27T10:44:14.265 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDQJYqv6OO4gJnK9WOA0M0v5I/aOCTRH2Sh6uGB+54a7Ds9qqL4xOVqE15zfpX4CHHV8nlNBlbqj4eGPmGG4HwEi07gGZHiYjUzKKJqnHndejQ5uU7cbQfwiT73MYcVm/lqYgT771kP7wf72m3V8787nTAvKlp0bw91jVA7/M0dki3a+ARyJlzV9S0dEv8+M9NY+f9Pg4PSiUpED7qCP7mIpaK0IGElgKVbtSR1TzfuaUKJJLo7gxS+74AVNfx6UIv3IVKz/WYkr+/3lpl3CLT/Yv7/kcsFKhXQW3+6Xt9nfwVTb1qfxoQUeP8XmIiOUPUYKu8kQxBg2xlj/iCUV1BXYgy6loL8QMwgrEyK96MKV8Yld3y8xoulb8JQOysAGVw9oVa2Yi3ar27mBNgR4tvFq2h82aUBHjDq7t4kyAwA2tHN3Xxzemyt7G8e3mZPM3d8f1CFBNqHuKXJ2+E9oTKv22ZZlb9XAJiww4F9RvLmrUvsmyQ6AjV7smRENTpicS8= ceph-adca88c4-ec26-11ee-95d0-87774f69a715 2024-03-27T10:44:14.266 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:13.387+0000 7f5f87200700 1 Processor -- start 2024-03-27T10:44:14.266 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:13.387+0000 7f5f87200700 1 -- start start 2024-03-27T10:44:14.266 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:13.388+0000 7f5f87200700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f8807dee0 0x7f5f8807c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:44:14.266 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:13.388+0000 7f5f87200700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5f8807c910 con 0x7f5f8807dee0 2024-03-27T10:44:14.266 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:13.388+0000 7f5f85e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f8807dee0 0x7f5f8807c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:44:14.270 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:13.388+0000 7f5f85e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f8807dee0 0x7f5f8807c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52418/0 (socket says 172.21.15.97:52418) 2024-03-27T10:44:14.270 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:13.388+0000 7f5f85e00700 1 -- 172.21.15.97:0/1964166368 learned_addr learned my addr 172.21.15.97:0/1964166368 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:44:14.270 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:13.388+0000 7f5f85e00700 1 -- 172.21.15.97:0/1964166368 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f8807ca90 con 0x7f5f8807dee0 2024-03-27T10:44:14.271 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:13.389+0000 7f5f85e00700 1 --2- 172.21.15.97:0/1964166368 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f8807dee0 0x7f5f8807c340 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5f780098a0 tx=0x7f5f78009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff965ec4d3fa0c40 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:44:14.271 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:13.389+0000 7f5f84a00700 1 -- 172.21.15.97:0/1964166368 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5f78004160 con 0x7f5f8807dee0 2024-03-27T10:44:14.271 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:13.389+0000 7f5f84a00700 1 -- 172.21.15.97:0/1964166368 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f5f78004300 con 0x7f5f8807dee0 2024-03-27T10:44:14.271 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:13.389+0000 7f5f84a00700 1 -- 172.21.15.97:0/1964166368 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5f78011a80 con 0x7f5f8807dee0 2024-03-27T10:44:14.271 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:13.390+0000 7f5f87200700 1 -- 172.21.15.97:0/1964166368 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f8807dee0 msgr2=0x7f5f8807c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:44:14.271 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:13.390+0000 7f5f87200700 1 --2- 172.21.15.97:0/1964166368 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f8807dee0 0x7f5f8807c340 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5f780098a0 tx=0x7f5f78009bb0 comp rx=0 tx=0).stop 2024-03-27T10:44:14.271 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:13.390+0000 7f5f87200700 1 -- 172.21.15.97:0/1964166368 shutdown_connections 2024-03-27T10:44:14.271 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:13.390+0000 7f5f87200700 1 --2- 172.21.15.97:0/1964166368 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f8807dee0 0x7f5f8807c340 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:44:14.271 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:13.390+0000 7f5f87200700 1 -- 172.21.15.97:0/1964166368 >> 172.21.15.97:0/1964166368 conn(0x7f5f881033b0 msgr2=0x7f5f881057d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:44:14.271 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:13.390+0000 7f5f87200700 1 -- 172.21.15.97:0/1964166368 shutdown_connections 2024-03-27T10:44:14.271 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:13.390+0000 7f5f87200700 1 -- 172.21.15.97:0/1964166368 wait complete. 2024-03-27T10:44:14.271 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:13.391+0000 7f5f87200700 1 Processor -- start 2024-03-27T10:44:14.271 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:13.391+0000 7f5f87200700 1 -- start start 2024-03-27T10:44:14.272 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:13.392+0000 7f5f87200700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f8807dee0 0x7f5f8819df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:44:14.272 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:13.392+0000 7f5f87200700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5f8810e360 con 0x7f5f8807dee0 2024-03-27T10:44:14.272 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:13.392+0000 7f5f85e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f8807dee0 0x7f5f8819df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:44:14.272 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:13.392+0000 7f5f85e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f8807dee0 0x7f5f8819df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52424/0 (socket says 172.21.15.97:52424) 2024-03-27T10:44:14.272 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:13.392+0000 7f5f85e00700 1 -- 172.21.15.97:0/3944529160 learned_addr learned my addr 172.21.15.97:0/3944529160 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:44:14.272 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:13.393+0000 7f5f85e00700 1 -- 172.21.15.97:0/3944529160 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f8819e480 con 0x7f5f8807dee0 2024-03-27T10:44:14.272 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:13.393+0000 7f5f85e00700 1 --2- 172.21.15.97:0/3944529160 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f8807dee0 0x7f5f8819df40 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5f78000c00 tx=0x7f5f78011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:44:14.272 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:13.393+0000 7f5f7ea00700 1 -- 172.21.15.97:0/3944529160 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5f780039f0 con 0x7f5f8807dee0 2024-03-27T10:44:14.272 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:13.393+0000 7f5f7ea00700 1 -- 172.21.15.97:0/3944529160 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f5f78003b90 con 0x7f5f8807dee0 2024-03-27T10:44:14.272 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:13.393+0000 7f5f7ea00700 1 -- 172.21.15.97:0/3944529160 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5f7801a740 con 0x7f5f8807dee0 2024-03-27T10:44:14.272 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:13.393+0000 7f5f87200700 1 -- 172.21.15.97:0/3944529160 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f8819e710 con 0x7f5f8807dee0 2024-03-27T10:44:14.272 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:13.393+0000 7f5f87200700 1 -- 172.21.15.97:0/3944529160 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f881a1400 con 0x7f5f8807dee0 2024-03-27T10:44:14.272 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:13.394+0000 7f5f7ea00700 1 -- 172.21.15.97:0/3944529160 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) ==== 46466+0+0 (secure 0 0 0) 0x7f5f78024af0 con 0x7f5f8807dee0 2024-03-27T10:44:14.273 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:13.395+0000 7f5f87200700 1 -- 172.21.15.97:0/3944529160 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f88063c70 con 0x7f5f8807dee0 2024-03-27T10:44:14.273 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:13.395+0000 7f5f7ea00700 1 --2- 172.21.15.97:0/3944529160 >> [v2:172.21.15.97:6800/1685921853,v1:172.21.15.97:6801/1685921853] conn(0x7f5f6c039420 0x7f5f6c03b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:44:14.273 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:13.395+0000 7f5f7ea00700 1 -- 172.21.15.97:0/3944529160 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f5f7804d8a0 con 0x7f5f8807dee0 2024-03-27T10:44:14.273 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:13.395+0000 7f5f85400700 1 --2- 172.21.15.97:0/3944529160 >> [v2:172.21.15.97:6800/1685921853,v1:172.21.15.97:6801/1685921853] conn(0x7f5f6c039420 0x7f5f6c03b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:44:14.273 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:13.396+0000 7f5f85400700 1 --2- 172.21.15.97:0/3944529160 >> [v2:172.21.15.97:6800/1685921853,v1:172.21.15.97:6801/1685921853] conn(0x7f5f6c039420 0x7f5f6c03b8e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5f74006fd0 tx=0x7f5f74006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:44:14.273 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:13.401+0000 7f5f7ea00700 1 -- 172.21.15.97:0/3944529160 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f5f78020db0 con 0x7f5f8807dee0 2024-03-27T10:44:14.275 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:13.625+0000 7f5f87200700 1 -- 172.21.15.97:0/3944529160 --> [v2:172.21.15.97:6800/1685921853,v1:172.21.15.97:6801/1685921853] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f5f881045d0 con 0x7f5f6c039420 2024-03-27T10:44:14.275 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:13.627+0000 7f5f7ea00700 1 -- 172.21.15.97:0/3944529160 <== mgr.14120 v2:172.21.15.97:6800/1685921853 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f5f881045d0 con 0x7f5f6c039420 2024-03-27T10:44:14.275 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:13.630+0000 7f5f87200700 1 -- 172.21.15.97:0/3944529160 >> [v2:172.21.15.97:6800/1685921853,v1:172.21.15.97:6801/1685921853] conn(0x7f5f6c039420 msgr2=0x7f5f6c03b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:44:14.275 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:13.630+0000 7f5f87200700 1 --2- 172.21.15.97:0/3944529160 >> [v2:172.21.15.97:6800/1685921853,v1:172.21.15.97:6801/1685921853] conn(0x7f5f6c039420 0x7f5f6c03b8e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5f74006fd0 tx=0x7f5f74006e40 comp rx=0 tx=0).stop 2024-03-27T10:44:14.275 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:13.631+0000 7f5f87200700 1 -- 172.21.15.97:0/3944529160 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f8807dee0 msgr2=0x7f5f8819df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:44:14.275 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:13.631+0000 7f5f87200700 1 --2- 172.21.15.97:0/3944529160 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f8807dee0 0x7f5f8819df40 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5f78000c00 tx=0x7f5f78011b00 comp rx=0 tx=0).stop 2024-03-27T10:44:14.275 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:13.631+0000 7f5f87200700 1 -- 172.21.15.97:0/3944529160 shutdown_connections 2024-03-27T10:44:14.275 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:13.631+0000 7f5f87200700 1 --2- 172.21.15.97:0/3944529160 >> [v2:172.21.15.97:6800/1685921853,v1:172.21.15.97:6801/1685921853] conn(0x7f5f6c039420 0x7f5f6c03b8e0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:44:14.275 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:13.631+0000 7f5f87200700 1 --2- 172.21.15.97:0/3944529160 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f8807dee0 0x7f5f8819df40 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:44:14.276 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:13.631+0000 7f5f87200700 1 -- 172.21.15.97:0/3944529160 >> 172.21.15.97:0/3944529160 conn(0x7f5f881033b0 msgr2=0x7f5f88103ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:44:14.276 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:13.631+0000 7f5f87200700 1 -- 172.21.15.97:0/3944529160 shutdown_connections 2024-03-27T10:44:14.276 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:13.631+0000 7f5f87200700 1 -- 172.21.15.97:0/3944529160 wait complete. 2024-03-27T10:44:14.276 INFO:teuthology.orchestra.run.smithi097.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-03-27T10:44:14.276 INFO:teuthology.orchestra.run.smithi097.stdout:Adding key to root@localhost authorized_keys... 2024-03-27T10:44:14.276 INFO:teuthology.orchestra.run.smithi097.stdout:Adding host smithi097... 2024-03-27T10:44:14.563 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:44:14 smithi097 ceph-mon[31859]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T10:44:15.746 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:44:15 smithi097 ceph-mon[31859]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi097", "addr": "172.21.15.97", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T10:44:17.644 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:44:17 smithi097 ceph-mon[31859]: Deploying cephadm binary to smithi097 2024-03-27T10:44:22.016 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout Added host 'smithi097' with addr '172.21.15.97' 2024-03-27T10:44:22.017 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:14.876+0000 7f99c2000700 1 Processor -- start 2024-03-27T10:44:22.017 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:14.876+0000 7f99c2000700 1 -- start start 2024-03-27T10:44:22.018 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:14.876+0000 7f99c2000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99bc10a190 0x7f99bc10a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:44:22.018 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:14.876+0000 7f99c2000700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f99bc10ab80 con 0x7f99bc10a190 2024-03-27T10:44:22.018 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:14.877+0000 7f99bb400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99bc10a190 0x7f99bc10a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:44:22.018 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:14.877+0000 7f99bb400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99bc10a190 0x7f99bc10a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52426/0 (socket says 172.21.15.97:52426) 2024-03-27T10:44:22.018 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:14.877+0000 7f99bb400700 1 -- 172.21.15.97:0/2829470770 learned_addr learned my addr 172.21.15.97:0/2829470770 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:44:22.018 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:14.877+0000 7f99bb400700 1 -- 172.21.15.97:0/2829470770 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99bc10b390 con 0x7f99bc10a190 2024-03-27T10:44:22.018 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:14.878+0000 7f99bb400700 1 --2- 172.21.15.97:0/2829470770 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99bc10a190 0x7f99bc10a5b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f99ac0098a0 tx=0x7f99ac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbc5200aa663b1fa server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:44:22.018 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:14.878+0000 7f99ba000700 1 -- 172.21.15.97:0/2829470770 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f99ac004160 con 0x7f99bc10a190 2024-03-27T10:44:22.018 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:14.878+0000 7f99ba000700 1 -- 172.21.15.97:0/2829470770 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f99ac004300 con 0x7f99bc10a190 2024-03-27T10:44:22.018 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:14.878+0000 7f99ba000700 1 -- 172.21.15.97:0/2829470770 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f99ac011a80 con 0x7f99bc10a190 2024-03-27T10:44:22.018 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:14.879+0000 7f99c2000700 1 -- 172.21.15.97:0/2829470770 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99bc10a190 msgr2=0x7f99bc10a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:44:22.022 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:14.879+0000 7f99c2000700 1 --2- 172.21.15.97:0/2829470770 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99bc10a190 0x7f99bc10a5b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f99ac0098a0 tx=0x7f99ac009bb0 comp rx=0 tx=0).stop 2024-03-27T10:44:22.022 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:14.879+0000 7f99c2000700 1 -- 172.21.15.97:0/2829470770 shutdown_connections 2024-03-27T10:44:22.022 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:14.879+0000 7f99c2000700 1 --2- 172.21.15.97:0/2829470770 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99bc10a190 0x7f99bc10a5b0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:44:22.022 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:14.879+0000 7f99c2000700 1 -- 172.21.15.97:0/2829470770 >> 172.21.15.97:0/2829470770 conn(0x7f99bc07d650 msgr2=0x7f99bc07da60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:44:22.022 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:14.879+0000 7f99c2000700 1 -- 172.21.15.97:0/2829470770 shutdown_connections 2024-03-27T10:44:22.023 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:14.879+0000 7f99c2000700 1 -- 172.21.15.97:0/2829470770 wait complete. 2024-03-27T10:44:22.023 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:14.880+0000 7f99c2000700 1 Processor -- start 2024-03-27T10:44:22.023 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:14.880+0000 7f99c2000700 1 -- start start 2024-03-27T10:44:22.023 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:14.880+0000 7f99c2000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99bc10a190 0x7f99bc19e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:44:22.023 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:14.880+0000 7f99c2000700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f99bc10e740 con 0x7f99bc10a190 2024-03-27T10:44:22.023 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:14.881+0000 7f99bb400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99bc10a190 0x7f99bc19e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:44:22.023 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:14.881+0000 7f99bb400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99bc10a190 0x7f99bc19e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52428/0 (socket says 172.21.15.97:52428) 2024-03-27T10:44:22.023 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:14.881+0000 7f99bb400700 1 -- 172.21.15.97:0/167882553 learned_addr learned my addr 172.21.15.97:0/167882553 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:44:22.023 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:14.881+0000 7f99bb400700 1 -- 172.21.15.97:0/167882553 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99bc19e640 con 0x7f99bc10a190 2024-03-27T10:44:22.023 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:14.881+0000 7f99bb400700 1 --2- 172.21.15.97:0/167882553 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99bc10a190 0x7f99bc19e100 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f99ac000c00 tx=0x7f99ac011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:44:22.023 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:14.881+0000 7f99b3e00700 1 -- 172.21.15.97:0/167882553 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f99ac0039f0 con 0x7f99bc10a190 2024-03-27T10:44:22.023 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:14.881+0000 7f99b3e00700 1 -- 172.21.15.97:0/167882553 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f99ac003b90 con 0x7f99bc10a190 2024-03-27T10:44:22.023 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:14.882+0000 7f99b3e00700 1 -- 172.21.15.97:0/167882553 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f99ac01a740 con 0x7f99bc10a190 2024-03-27T10:44:22.024 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:14.882+0000 7f99c2000700 1 -- 172.21.15.97:0/167882553 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99bc19e8d0 con 0x7f99bc10a190 2024-03-27T10:44:22.024 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:14.882+0000 7f99c2000700 1 -- 172.21.15.97:0/167882553 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99bc1a15c0 con 0x7f99bc10a190 2024-03-27T10:44:22.024 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:14.883+0000 7f99b3e00700 1 -- 172.21.15.97:0/167882553 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) ==== 46466+0+0 (secure 0 0 0) 0x7f99ac024af0 con 0x7f99bc10a190 2024-03-27T10:44:22.024 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:14.883+0000 7f99c2000700 1 -- 172.21.15.97:0/167882553 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99bc063c70 con 0x7f99bc10a190 2024-03-27T10:44:22.024 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:14.883+0000 7f99b3e00700 1 --2- 172.21.15.97:0/167882553 >> [v2:172.21.15.97:6800/1685921853,v1:172.21.15.97:6801/1685921853] conn(0x7f99a0039420 0x7f99a003b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:44:22.024 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:14.883+0000 7f99b3e00700 1 -- 172.21.15.97:0/167882553 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f99ac04d890 con 0x7f99bc10a190 2024-03-27T10:44:22.024 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:14.884+0000 7f99baa00700 1 --2- 172.21.15.97:0/167882553 >> [v2:172.21.15.97:6800/1685921853,v1:172.21.15.97:6801/1685921853] conn(0x7f99a0039420 0x7f99a003b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:44:22.024 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:14.884+0000 7f99baa00700 1 --2- 172.21.15.97:0/167882553 >> [v2:172.21.15.97:6800/1685921853,v1:172.21.15.97:6801/1685921853] conn(0x7f99a0039420 0x7f99a003b8e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f99a8006fd0 tx=0x7f99a8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:44:22.024 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:14.889+0000 7f99b3e00700 1 -- 172.21.15.97:0/167882553 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f99ac020db0 con 0x7f99bc10a190 2024-03-27T10:44:22.024 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:15.115+0000 7f99c2000700 1 -- 172.21.15.97:0/167882553 --> [v2:172.21.15.97:6800/1685921853,v1:172.21.15.97:6801/1685921853] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi097", "addr": "172.21.15.97", "target": ["mon-mgr", ""]}) -- 0x7f99bc107c60 con 0x7f99a0039420 2024-03-27T10:44:22.024 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:21.290+0000 7f99b3e00700 1 -- 172.21.15.97:0/167882553 <== mgr.14120 v2:172.21.15.97:6800/1685921853 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f99bc107c60 con 0x7f99a0039420 2024-03-27T10:44:22.024 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:21.294+0000 7f99c2000700 1 -- 172.21.15.97:0/167882553 >> [v2:172.21.15.97:6800/1685921853,v1:172.21.15.97:6801/1685921853] conn(0x7f99a0039420 msgr2=0x7f99a003b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:44:22.024 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:21.294+0000 7f99c2000700 1 --2- 172.21.15.97:0/167882553 >> [v2:172.21.15.97:6800/1685921853,v1:172.21.15.97:6801/1685921853] conn(0x7f99a0039420 0x7f99a003b8e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f99a8006fd0 tx=0x7f99a8006e40 comp rx=0 tx=0).stop 2024-03-27T10:44:22.024 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:21.294+0000 7f99c2000700 1 -- 172.21.15.97:0/167882553 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99bc10a190 msgr2=0x7f99bc19e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:44:22.025 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:21.294+0000 7f99c2000700 1 --2- 172.21.15.97:0/167882553 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99bc10a190 0x7f99bc19e100 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f99ac000c00 tx=0x7f99ac011b00 comp rx=0 tx=0).stop 2024-03-27T10:44:22.025 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:21.294+0000 7f99c2000700 1 -- 172.21.15.97:0/167882553 shutdown_connections 2024-03-27T10:44:22.025 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:21.294+0000 7f99c2000700 1 --2- 172.21.15.97:0/167882553 >> [v2:172.21.15.97:6800/1685921853,v1:172.21.15.97:6801/1685921853] conn(0x7f99a0039420 0x7f99a003b8e0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:44:22.025 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:21.294+0000 7f99c2000700 1 --2- 172.21.15.97:0/167882553 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99bc10a190 0x7f99bc19e100 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:44:22.026 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:21.294+0000 7f99c2000700 1 -- 172.21.15.97:0/167882553 >> 172.21.15.97:0/167882553 conn(0x7f99bc07d650 msgr2=0x7f99bc107550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:44:22.026 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:21.295+0000 7f99c2000700 1 -- 172.21.15.97:0/167882553 shutdown_connections 2024-03-27T10:44:22.026 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:21.295+0000 7f99c2000700 1 -- 172.21.15.97:0/167882553 wait complete. 2024-03-27T10:44:22.026 INFO:teuthology.orchestra.run.smithi097.stdout:Deploying unmanaged mon service... 2024-03-27T10:44:22.624 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:44:22 smithi097 ceph-mon[31859]: from='mgr.14120 172.21.15.97:0/4145774229' entity='mgr.x' 2024-03-27T10:44:22.624 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:44:22 smithi097 ceph-mon[31859]: Added host smithi097 2024-03-27T10:44:22.625 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:44:22 smithi097 ceph-mon[31859]: from='mgr.14120 172.21.15.97:0/4145774229' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:44:24.042 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-03-27T10:44:24.043 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:22.692+0000 7f2055c00700 1 Processor -- start 2024-03-27T10:44:24.043 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:22.693+0000 7f2055c00700 1 -- start start 2024-03-27T10:44:24.043 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:22.693+0000 7f2055c00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20501067e0 0x7f2050106c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:44:24.043 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:22.693+0000 7f2055c00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f205007bed0 con 0x7f20501067e0 2024-03-27T10:44:24.045 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:22.694+0000 7f204ea00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20501067e0 0x7f2050106c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:44:24.045 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:22.694+0000 7f204ea00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20501067e0 0x7f2050106c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40776/0 (socket says 172.21.15.97:40776) 2024-03-27T10:44:24.045 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:22.694+0000 7f204ea00700 1 -- 172.21.15.97:0/1342835824 learned_addr learned my addr 172.21.15.97:0/1342835824 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:44:24.045 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:22.694+0000 7f204ea00700 1 -- 172.21.15.97:0/1342835824 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2050107140 con 0x7f20501067e0 2024-03-27T10:44:24.046 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:22.694+0000 7f204ea00700 1 --2- 172.21.15.97:0/1342835824 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20501067e0 0x7f2050106c00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f20400098a0 tx=0x7f2040009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b3311775eb58d5f server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:44:24.046 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:22.695+0000 7f204d600700 1 -- 172.21.15.97:0/1342835824 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2040004160 con 0x7f20501067e0 2024-03-27T10:44:24.046 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:22.695+0000 7f204d600700 1 -- 172.21.15.97:0/1342835824 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f2040004300 con 0x7f20501067e0 2024-03-27T10:44:24.046 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:22.695+0000 7f204d600700 1 -- 172.21.15.97:0/1342835824 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2040011a80 con 0x7f20501067e0 2024-03-27T10:44:24.046 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:22.695+0000 7f2055c00700 1 -- 172.21.15.97:0/1342835824 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20501067e0 msgr2=0x7f2050106c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:44:24.046 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:22.695+0000 7f2055c00700 1 --2- 172.21.15.97:0/1342835824 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20501067e0 0x7f2050106c00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f20400098a0 tx=0x7f2040009bb0 comp rx=0 tx=0).stop 2024-03-27T10:44:24.046 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:22.695+0000 7f2055c00700 1 -- 172.21.15.97:0/1342835824 shutdown_connections 2024-03-27T10:44:24.046 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:22.696+0000 7f2055c00700 1 --2- 172.21.15.97:0/1342835824 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20501067e0 0x7f2050106c00 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:44:24.046 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:22.696+0000 7f2055c00700 1 -- 172.21.15.97:0/1342835824 >> 172.21.15.97:0/1342835824 conn(0x7f20501023e0 msgr2=0x7f2050104860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:44:24.046 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:22.696+0000 7f2055c00700 1 -- 172.21.15.97:0/1342835824 shutdown_connections 2024-03-27T10:44:24.046 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:22.696+0000 7f2055c00700 1 -- 172.21.15.97:0/1342835824 wait complete. 2024-03-27T10:44:24.046 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:22.697+0000 7f2055c00700 1 Processor -- start 2024-03-27T10:44:24.047 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:22.697+0000 7f2055c00700 1 -- start start 2024-03-27T10:44:24.047 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:22.697+0000 7f2055c00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20501067e0 0x7f205019e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:44:24.047 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:22.697+0000 7f2055c00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2050108170 con 0x7f20501067e0 2024-03-27T10:44:24.047 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:22.697+0000 7f204ea00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20501067e0 0x7f205019e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:44:24.047 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:22.697+0000 7f204ea00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20501067e0 0x7f205019e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40778/0 (socket says 172.21.15.97:40778) 2024-03-27T10:44:24.047 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:22.697+0000 7f204ea00700 1 -- 172.21.15.97:0/4112261547 learned_addr learned my addr 172.21.15.97:0/4112261547 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:44:24.049 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:22.698+0000 7f204ea00700 1 -- 172.21.15.97:0/4112261547 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f205019e6b0 con 0x7f20501067e0 2024-03-27T10:44:24.049 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:22.698+0000 7f204ea00700 1 --2- 172.21.15.97:0/4112261547 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20501067e0 0x7f205019e170 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2040006af0 tx=0x7f2040011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:44:24.049 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:22.698+0000 7f2047400700 1 -- 172.21.15.97:0/4112261547 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2040003930 con 0x7f20501067e0 2024-03-27T10:44:24.049 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:22.698+0000 7f2047400700 1 -- 172.21.15.97:0/4112261547 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f2040003ad0 con 0x7f20501067e0 2024-03-27T10:44:24.049 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:22.698+0000 7f2055c00700 1 -- 172.21.15.97:0/4112261547 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f205019e940 con 0x7f20501067e0 2024-03-27T10:44:24.050 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:22.698+0000 7f2047400700 1 -- 172.21.15.97:0/4112261547 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f20400195e0 con 0x7f20501067e0 2024-03-27T10:44:24.050 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:22.699+0000 7f2055c00700 1 -- 172.21.15.97:0/4112261547 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20501a1630 con 0x7f20501067e0 2024-03-27T10:44:24.050 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:22.700+0000 7f2055c00700 1 -- 172.21.15.97:0/4112261547 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2050063c70 con 0x7f20501067e0 2024-03-27T10:44:24.050 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:22.700+0000 7f2047400700 1 -- 172.21.15.97:0/4112261547 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) ==== 46466+0+0 (secure 0 0 0) 0x7f20400198e0 con 0x7f20501067e0 2024-03-27T10:44:24.050 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:22.700+0000 7f2047400700 1 --2- 172.21.15.97:0/4112261547 >> [v2:172.21.15.97:6800/1685921853,v1:172.21.15.97:6801/1685921853] conn(0x7f2034039380 0x7f203403b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:44:24.050 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:22.700+0000 7f2047400700 1 -- 172.21.15.97:0/4112261547 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f204004d9c0 con 0x7f20501067e0 2024-03-27T10:44:24.050 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:22.701+0000 7f204e000700 1 --2- 172.21.15.97:0/4112261547 >> [v2:172.21.15.97:6800/1685921853,v1:172.21.15.97:6801/1685921853] conn(0x7f2034039380 0x7f203403b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:44:24.050 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:22.701+0000 7f204e000700 1 --2- 172.21.15.97:0/4112261547 >> [v2:172.21.15.97:6800/1685921853,v1:172.21.15.97:6801/1685921853] conn(0x7f2034039380 0x7f203403b840 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f203c006fd0 tx=0x7f203c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:44:24.050 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:22.706+0000 7f2047400700 1 -- 172.21.15.97:0/4112261547 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f204004fe20 con 0x7f20501067e0 2024-03-27T10:44:24.050 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:22.953+0000 7f2055c00700 1 -- 172.21.15.97:0/4112261547 --> [v2:172.21.15.97:6800/1685921853,v1:172.21.15.97:6801/1685921853] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f20500008d0 con 0x7f2034039380 2024-03-27T10:44:24.050 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:22.961+0000 7f2047400700 1 -- 172.21.15.97:0/4112261547 <== mgr.14120 v2:172.21.15.97:6800/1685921853 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f20500008d0 con 0x7f2034039380 2024-03-27T10:44:24.050 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:22.964+0000 7f2055c00700 1 -- 172.21.15.97:0/4112261547 >> [v2:172.21.15.97:6800/1685921853,v1:172.21.15.97:6801/1685921853] conn(0x7f2034039380 msgr2=0x7f203403b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:44:24.051 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:22.964+0000 7f2055c00700 1 --2- 172.21.15.97:0/4112261547 >> [v2:172.21.15.97:6800/1685921853,v1:172.21.15.97:6801/1685921853] conn(0x7f2034039380 0x7f203403b840 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f203c006fd0 tx=0x7f203c006e40 comp rx=0 tx=0).stop 2024-03-27T10:44:24.051 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:22.964+0000 7f2055c00700 1 -- 172.21.15.97:0/4112261547 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20501067e0 msgr2=0x7f205019e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:44:24.051 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:22.964+0000 7f2055c00700 1 --2- 172.21.15.97:0/4112261547 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20501067e0 0x7f205019e170 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2040006af0 tx=0x7f2040011b70 comp rx=0 tx=0).stop 2024-03-27T10:44:24.051 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:22.964+0000 7f2055c00700 1 -- 172.21.15.97:0/4112261547 shutdown_connections 2024-03-27T10:44:24.051 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:22.964+0000 7f2055c00700 1 --2- 172.21.15.97:0/4112261547 >> [v2:172.21.15.97:6800/1685921853,v1:172.21.15.97:6801/1685921853] conn(0x7f2034039380 0x7f203403b840 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:44:24.051 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:22.964+0000 7f2055c00700 1 --2- 172.21.15.97:0/4112261547 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20501067e0 0x7f205019e170 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:44:24.051 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:22.964+0000 7f2055c00700 1 -- 172.21.15.97:0/4112261547 >> 172.21.15.97:0/4112261547 conn(0x7f20501023e0 msgr2=0x7f2050194e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:44:24.051 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:22.964+0000 7f2055c00700 1 -- 172.21.15.97:0/4112261547 shutdown_connections 2024-03-27T10:44:24.051 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:22.964+0000 7f2055c00700 1 -- 172.21.15.97:0/4112261547 wait complete. 2024-03-27T10:44:24.051 INFO:teuthology.orchestra.run.smithi097.stdout:Deploying unmanaged mgr service... 2024-03-27T10:44:24.195 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:44:23 smithi097 ceph-mon[31859]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T10:44:24.196 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:44:23 smithi097 ceph-mon[31859]: Saving service mon spec with placement count:5 2024-03-27T10:44:24.196 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:44:23 smithi097 ceph-mon[31859]: from='mgr.14120 172.21.15.97:0/4145774229' entity='mgr.x' 2024-03-27T10:44:25.787 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:44:25 smithi097 ceph-mon[31859]: from='mgr.14120 172.21.15.97:0/4145774229' entity='mgr.x' 2024-03-27T10:44:25.788 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:44:25 smithi097 ceph-mon[31859]: from='mgr.14120 172.21.15.97:0/4145774229' entity='mgr.x' 2024-03-27T10:44:25.788 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:44:25 smithi097 ceph-mon[31859]: from='mgr.14120 172.21.15.97:0/4145774229' entity='mgr.x' 2024-03-27T10:44:26.106 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-03-27T10:44:26.106 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:25.164+0000 7f329a600700 1 Processor -- start 2024-03-27T10:44:26.106 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:25.164+0000 7f329a600700 1 -- start start 2024-03-27T10:44:26.106 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:25.165+0000 7f329a600700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3294107fc0 0x7f32941083e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:44:26.107 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:25.165+0000 7f329a600700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f32941089b0 con 0x7f3294107fc0 2024-03-27T10:44:26.107 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:25.165+0000 7f3293400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3294107fc0 0x7f32941083e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:44:26.107 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:25.165+0000 7f3293400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3294107fc0 0x7f32941083e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40792/0 (socket says 172.21.15.97:40792) 2024-03-27T10:44:26.107 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:25.165+0000 7f3293400700 1 -- 172.21.15.97:0/2159329801 learned_addr learned my addr 172.21.15.97:0/2159329801 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:44:26.107 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:25.166+0000 7f3293400700 1 -- 172.21.15.97:0/2159329801 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32941091c0 con 0x7f3294107fc0 2024-03-27T10:44:26.107 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:25.166+0000 7f3293400700 1 --2- 172.21.15.97:0/2159329801 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3294107fc0 0x7f32941083e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f32840098a0 tx=0x7f3284009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceb6cb4dcae31313 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:44:26.107 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:25.166+0000 7f3292000700 1 -- 172.21.15.97:0/2159329801 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3284004160 con 0x7f3294107fc0 2024-03-27T10:44:26.107 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:25.166+0000 7f3292000700 1 -- 172.21.15.97:0/2159329801 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3284004300 con 0x7f3294107fc0 2024-03-27T10:44:26.107 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:25.166+0000 7f3292000700 1 -- 172.21.15.97:0/2159329801 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3284011a50 con 0x7f3294107fc0 2024-03-27T10:44:26.107 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:25.167+0000 7f329a600700 1 -- 172.21.15.97:0/2159329801 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3294107fc0 msgr2=0x7f32941083e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:44:26.107 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:25.167+0000 7f329a600700 1 --2- 172.21.15.97:0/2159329801 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3294107fc0 0x7f32941083e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f32840098a0 tx=0x7f3284009bb0 comp rx=0 tx=0).stop 2024-03-27T10:44:26.107 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:25.167+0000 7f329a600700 1 -- 172.21.15.97:0/2159329801 shutdown_connections 2024-03-27T10:44:26.107 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:25.167+0000 7f329a600700 1 --2- 172.21.15.97:0/2159329801 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3294107fc0 0x7f32941083e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:44:26.107 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:25.167+0000 7f329a600700 1 -- 172.21.15.97:0/2159329801 >> 172.21.15.97:0/2159329801 conn(0x7f3294103580 msgr2=0x7f32941059a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:44:26.108 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:25.167+0000 7f329a600700 1 -- 172.21.15.97:0/2159329801 shutdown_connections 2024-03-27T10:44:26.108 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:25.167+0000 7f329a600700 1 -- 172.21.15.97:0/2159329801 wait complete. 2024-03-27T10:44:26.108 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:25.168+0000 7f329a600700 1 Processor -- start 2024-03-27T10:44:26.108 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:25.169+0000 7f329a600700 1 -- start start 2024-03-27T10:44:26.108 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:25.169+0000 7f329a600700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3294107fc0 0x7f329419bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:44:26.108 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:25.169+0000 7f329a600700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f329410c570 con 0x7f3294107fc0 2024-03-27T10:44:26.108 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:25.169+0000 7f3293400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3294107fc0 0x7f329419bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:44:26.108 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:25.169+0000 7f3293400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3294107fc0 0x7f329419bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40808/0 (socket says 172.21.15.97:40808) 2024-03-27T10:44:26.108 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:25.169+0000 7f3293400700 1 -- 172.21.15.97:0/3598886574 learned_addr learned my addr 172.21.15.97:0/3598886574 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:44:26.108 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:25.170+0000 7f3293400700 1 -- 172.21.15.97:0/3598886574 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f329419c400 con 0x7f3294107fc0 2024-03-27T10:44:26.108 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:25.170+0000 7f3293400700 1 --2- 172.21.15.97:0/3598886574 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3294107fc0 0x7f329419bec0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3284006ba0 tx=0x7f3284011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:44:26.108 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:25.170+0000 7f328be00700 1 -- 172.21.15.97:0/3598886574 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3284003af0 con 0x7f3294107fc0 2024-03-27T10:44:26.108 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:25.170+0000 7f328be00700 1 -- 172.21.15.97:0/3598886574 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f328401a8e0 con 0x7f3294107fc0 2024-03-27T10:44:26.109 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:25.170+0000 7f328be00700 1 -- 172.21.15.97:0/3598886574 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3284024900 con 0x7f3294107fc0 2024-03-27T10:44:26.109 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:25.170+0000 7f329a600700 1 -- 172.21.15.97:0/3598886574 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f329419c690 con 0x7f3294107fc0 2024-03-27T10:44:26.109 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:25.171+0000 7f329a600700 1 -- 172.21.15.97:0/3598886574 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f329419f380 con 0x7f3294107fc0 2024-03-27T10:44:26.109 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:25.172+0000 7f328be00700 1 -- 172.21.15.97:0/3598886574 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) ==== 46466+0+0 (secure 0 0 0) 0x7f3284022020 con 0x7f3294107fc0 2024-03-27T10:44:26.109 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:25.172+0000 7f329a600700 1 -- 172.21.15.97:0/3598886574 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3294063c70 con 0x7f3294107fc0 2024-03-27T10:44:26.109 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:25.172+0000 7f328be00700 1 --2- 172.21.15.97:0/3598886574 >> [v2:172.21.15.97:6800/1685921853,v1:172.21.15.97:6801/1685921853] conn(0x7f32780393d0 0x7f327803b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:44:26.109 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:25.172+0000 7f328be00700 1 -- 172.21.15.97:0/3598886574 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f328404d810 con 0x7f3294107fc0 2024-03-27T10:44:26.109 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:25.173+0000 7f3292a00700 1 --2- 172.21.15.97:0/3598886574 >> [v2:172.21.15.97:6800/1685921853,v1:172.21.15.97:6801/1685921853] conn(0x7f32780393d0 0x7f327803b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:44:26.109 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:25.173+0000 7f3292a00700 1 --2- 172.21.15.97:0/3598886574 >> [v2:172.21.15.97:6800/1685921853,v1:172.21.15.97:6801/1685921853] conn(0x7f32780393d0 0x7f327803b890 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3280006fd0 tx=0x7f3280006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:44:26.109 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:25.178+0000 7f328be00700 1 -- 172.21.15.97:0/3598886574 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f328400f2e0 con 0x7f3294107fc0 2024-03-27T10:44:26.109 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:25.430+0000 7f329a600700 1 -- 172.21.15.97:0/3598886574 --> [v2:172.21.15.97:6800/1685921853,v1:172.21.15.97:6801/1685921853] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f3294104730 con 0x7f32780393d0 2024-03-27T10:44:26.109 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:25.439+0000 7f328be00700 1 -- 172.21.15.97:0/3598886574 <== mgr.14120 v2:172.21.15.97:6800/1685921853 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f3294104730 con 0x7f32780393d0 2024-03-27T10:44:26.109 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:25.441+0000 7f329a600700 1 -- 172.21.15.97:0/3598886574 >> [v2:172.21.15.97:6800/1685921853,v1:172.21.15.97:6801/1685921853] conn(0x7f32780393d0 msgr2=0x7f327803b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:44:26.109 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:25.442+0000 7f329a600700 1 --2- 172.21.15.97:0/3598886574 >> [v2:172.21.15.97:6800/1685921853,v1:172.21.15.97:6801/1685921853] conn(0x7f32780393d0 0x7f327803b890 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3280006fd0 tx=0x7f3280006e40 comp rx=0 tx=0).stop 2024-03-27T10:44:26.110 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:25.442+0000 7f329a600700 1 -- 172.21.15.97:0/3598886574 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3294107fc0 msgr2=0x7f329419bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:44:26.110 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:25.442+0000 7f329a600700 1 --2- 172.21.15.97:0/3598886574 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3294107fc0 0x7f329419bec0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3284006ba0 tx=0x7f3284011ad0 comp rx=0 tx=0).stop 2024-03-27T10:44:26.110 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:25.442+0000 7f329a600700 1 -- 172.21.15.97:0/3598886574 shutdown_connections 2024-03-27T10:44:26.110 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:25.442+0000 7f329a600700 1 --2- 172.21.15.97:0/3598886574 >> [v2:172.21.15.97:6800/1685921853,v1:172.21.15.97:6801/1685921853] conn(0x7f32780393d0 0x7f327803b890 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:44:26.110 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:25.442+0000 7f329a600700 1 --2- 172.21.15.97:0/3598886574 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3294107fc0 0x7f329419bec0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:44:26.110 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:25.442+0000 7f329a600700 1 -- 172.21.15.97:0/3598886574 >> 172.21.15.97:0/3598886574 conn(0x7f3294103580 msgr2=0x7f3294104020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:44:26.110 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:25.442+0000 7f329a600700 1 -- 172.21.15.97:0/3598886574 shutdown_connections 2024-03-27T10:44:26.110 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:25.442+0000 7f329a600700 1 -- 172.21.15.97:0/3598886574 wait complete. 2024-03-27T10:44:27.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:44:27 smithi097 ceph-mon[31859]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T10:44:27.394 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:44:27 smithi097 ceph-mon[31859]: Saving service mgr spec with placement count:2 2024-03-27T10:44:27.394 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:44:27 smithi097 ceph-mon[31859]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:44:27.394 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:44:27 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/3620540812' entity='client.admin' 2024-03-27T10:44:28.182 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:26.751+0000 7f3518600700 1 Processor -- start 2024-03-27T10:44:28.182 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:26.751+0000 7f3518600700 1 -- start start 2024-03-27T10:44:28.182 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:26.752+0000 7f3518600700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f351010a1f0 0x7f351010a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:44:28.182 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:26.752+0000 7f3518600700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f351010abe0 con 0x7f351010a1f0 2024-03-27T10:44:28.182 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:26.752+0000 7f3515c00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f351010a1f0 0x7f351010a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:44:28.182 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:26.752+0000 7f3515c00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f351010a1f0 0x7f351010a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40814/0 (socket says 172.21.15.97:40814) 2024-03-27T10:44:28.183 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:26.752+0000 7f3515c00700 1 -- 172.21.15.97:0/3211019348 learned_addr learned my addr 172.21.15.97:0/3211019348 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:44:28.183 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:26.753+0000 7f3515c00700 1 -- 172.21.15.97:0/3211019348 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f351010b3a0 con 0x7f351010a1f0 2024-03-27T10:44:28.183 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:26.753+0000 7f3515c00700 1 --2- 172.21.15.97:0/3211019348 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f351010a1f0 0x7f351010a610 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f35040098a0 tx=0x7f3504009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc635ca26d23509d server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:44:28.183 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:26.753+0000 7f350fe00700 1 -- 172.21.15.97:0/3211019348 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3504004160 con 0x7f351010a1f0 2024-03-27T10:44:28.183 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:26.754+0000 7f350fe00700 1 -- 172.21.15.97:0/3211019348 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3504004300 con 0x7f351010a1f0 2024-03-27T10:44:28.183 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:26.754+0000 7f350fe00700 1 -- 172.21.15.97:0/3211019348 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3504011a50 con 0x7f351010a1f0 2024-03-27T10:44:28.183 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:26.754+0000 7f3518600700 1 -- 172.21.15.97:0/3211019348 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f351010a1f0 msgr2=0x7f351010a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:44:28.183 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:26.754+0000 7f3518600700 1 --2- 172.21.15.97:0/3211019348 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f351010a1f0 0x7f351010a610 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f35040098a0 tx=0x7f3504009bb0 comp rx=0 tx=0).stop 2024-03-27T10:44:28.183 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:26.754+0000 7f3518600700 1 -- 172.21.15.97:0/3211019348 shutdown_connections 2024-03-27T10:44:28.183 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:26.754+0000 7f3518600700 1 --2- 172.21.15.97:0/3211019348 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f351010a1f0 0x7f351010a610 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:44:28.183 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:26.754+0000 7f3518600700 1 -- 172.21.15.97:0/3211019348 >> 172.21.15.97:0/3211019348 conn(0x7f351007d6e0 msgr2=0x7f351007daf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:44:28.184 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:26.755+0000 7f3518600700 1 -- 172.21.15.97:0/3211019348 shutdown_connections 2024-03-27T10:44:28.184 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:26.755+0000 7f3518600700 1 -- 172.21.15.97:0/3211019348 wait complete. 2024-03-27T10:44:28.184 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:26.756+0000 7f3518600700 1 Processor -- start 2024-03-27T10:44:28.184 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:26.756+0000 7f3518600700 1 -- start start 2024-03-27T10:44:28.184 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:26.756+0000 7f3518600700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f351010a1f0 0x7f351019e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:44:28.184 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:26.756+0000 7f3518600700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f351010e750 con 0x7f351010a1f0 2024-03-27T10:44:28.184 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:26.756+0000 7f3515c00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f351010a1f0 0x7f351019e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:44:28.184 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:26.757+0000 7f3515c00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f351010a1f0 0x7f351019e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40826/0 (socket says 172.21.15.97:40826) 2024-03-27T10:44:28.184 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:26.757+0000 7f3515c00700 1 -- 172.21.15.97:0/3620540812 learned_addr learned my addr 172.21.15.97:0/3620540812 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:44:28.184 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:26.757+0000 7f3515c00700 1 -- 172.21.15.97:0/3620540812 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f351019e5b0 con 0x7f351010a1f0 2024-03-27T10:44:28.184 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:26.757+0000 7f3515c00700 1 --2- 172.21.15.97:0/3620540812 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f351010a1f0 0x7f351019e070 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3504006b20 tx=0x7f3504011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:44:28.185 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:26.758+0000 7f350e000700 1 -- 172.21.15.97:0/3620540812 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f35040039e0 con 0x7f351010a1f0 2024-03-27T10:44:28.185 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:26.758+0000 7f350e000700 1 -- 172.21.15.97:0/3620540812 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f350401a8e0 con 0x7f351010a1f0 2024-03-27T10:44:28.185 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:26.758+0000 7f350e000700 1 -- 172.21.15.97:0/3620540812 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f35040247e0 con 0x7f351010a1f0 2024-03-27T10:44:28.185 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:26.758+0000 7f3518600700 1 -- 172.21.15.97:0/3620540812 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f351019e840 con 0x7f351010a1f0 2024-03-27T10:44:28.185 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:26.758+0000 7f3518600700 1 -- 172.21.15.97:0/3620540812 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35101a1530 con 0x7f351010a1f0 2024-03-27T10:44:28.185 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:26.759+0000 7f3518600700 1 -- 172.21.15.97:0/3620540812 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3510063c70 con 0x7f351010a1f0 2024-03-27T10:44:28.185 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:26.759+0000 7f350e000700 1 -- 172.21.15.97:0/3620540812 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) ==== 46466+0+0 (secure 0 0 0) 0x7f3504022020 con 0x7f351010a1f0 2024-03-27T10:44:28.185 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:26.760+0000 7f350e000700 1 --2- 172.21.15.97:0/3620540812 >> [v2:172.21.15.97:6800/1685921853,v1:172.21.15.97:6801/1685921853] conn(0x7f34f80393d0 0x7f34f803b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:44:28.185 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:26.760+0000 7f350e000700 1 -- 172.21.15.97:0/3620540812 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f350404d950 con 0x7f351010a1f0 2024-03-27T10:44:28.185 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:26.760+0000 7f3515200700 1 --2- 172.21.15.97:0/3620540812 >> [v2:172.21.15.97:6800/1685921853,v1:172.21.15.97:6801/1685921853] conn(0x7f34f80393d0 0x7f34f803b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:44:28.185 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:26.761+0000 7f3515200700 1 --2- 172.21.15.97:0/3620540812 >> [v2:172.21.15.97:6800/1685921853,v1:172.21.15.97:6801/1685921853] conn(0x7f34f80393d0 0x7f34f803b890 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f34fc006fd0 tx=0x7f34fc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:44:28.185 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:26.765+0000 7f350e000700 1 -- 172.21.15.97:0/3620540812 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f350400f2e0 con 0x7f351010a1f0 2024-03-27T10:44:28.186 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:26.994+0000 7f3518600700 1 -- 172.21.15.97:0/3620540812 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f35101a1ab0 con 0x7f351010a1f0 2024-03-27T10:44:28.186 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:26.997+0000 7f350e000700 1 -- 172.21.15.97:0/3620540812 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f350404d2e0 con 0x7f351010a1f0 2024-03-27T10:44:28.186 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:27.000+0000 7f3518600700 1 -- 172.21.15.97:0/3620540812 >> [v2:172.21.15.97:6800/1685921853,v1:172.21.15.97:6801/1685921853] conn(0x7f34f80393d0 msgr2=0x7f34f803b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:44:28.186 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:27.000+0000 7f3518600700 1 --2- 172.21.15.97:0/3620540812 >> [v2:172.21.15.97:6800/1685921853,v1:172.21.15.97:6801/1685921853] conn(0x7f34f80393d0 0x7f34f803b890 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f34fc006fd0 tx=0x7f34fc006e40 comp rx=0 tx=0).stop 2024-03-27T10:44:28.186 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:27.000+0000 7f3518600700 1 -- 172.21.15.97:0/3620540812 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f351010a1f0 msgr2=0x7f351019e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:44:28.186 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:27.000+0000 7f3518600700 1 --2- 172.21.15.97:0/3620540812 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f351010a1f0 0x7f351019e070 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3504006b20 tx=0x7f3504011ad0 comp rx=0 tx=0).stop 2024-03-27T10:44:28.186 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:27.001+0000 7f3518600700 1 -- 172.21.15.97:0/3620540812 shutdown_connections 2024-03-27T10:44:28.186 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:27.001+0000 7f3518600700 1 --2- 172.21.15.97:0/3620540812 >> [v2:172.21.15.97:6800/1685921853,v1:172.21.15.97:6801/1685921853] conn(0x7f34f80393d0 0x7f34f803b890 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:44:28.186 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:27.001+0000 7f3518600700 1 --2- 172.21.15.97:0/3620540812 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f351010a1f0 0x7f351019e070 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:44:28.186 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:27.001+0000 7f3518600700 1 -- 172.21.15.97:0/3620540812 >> 172.21.15.97:0/3620540812 conn(0x7f351007d6e0 msgr2=0x7f35101074a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:44:28.186 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:27.001+0000 7f3518600700 1 -- 172.21.15.97:0/3620540812 shutdown_connections 2024-03-27T10:44:28.186 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:27.001+0000 7f3518600700 1 -- 172.21.15.97:0/3620540812 wait complete. 2024-03-27T10:44:29.335 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:44:28 smithi097 ceph-mon[31859]: from='mgr.14120 172.21.15.97:0/4145774229' entity='mgr.x' 2024-03-27T10:44:29.336 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:44:28 smithi097 ceph-mon[31859]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:44:29.701 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:28.756+0000 7f739f200700 1 Processor -- start 2024-03-27T10:44:29.701 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:28.756+0000 7f739f200700 1 -- start start 2024-03-27T10:44:29.701 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:28.756+0000 7f739f200700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73981089f0 0x7f739810ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:44:29.701 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:28.756+0000 7f739f200700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f739807bed0 con 0x7f73981089f0 2024-03-27T10:44:29.701 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:28.757+0000 7f7397e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73981089f0 0x7f739810ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:44:29.701 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:28.757+0000 7f7397e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73981089f0 0x7f739810ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40836/0 (socket says 172.21.15.97:40836) 2024-03-27T10:44:29.701 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:28.757+0000 7f7397e00700 1 -- 172.21.15.97:0/721004505 learned_addr learned my addr 172.21.15.97:0/721004505 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:44:29.702 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:28.757+0000 7f7397e00700 1 -- 172.21.15.97:0/721004505 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f739810b320 con 0x7f73981089f0 2024-03-27T10:44:29.702 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:28.758+0000 7f7397e00700 1 --2- 172.21.15.97:0/721004505 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73981089f0 0x7f739810ade0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f73880098a0 tx=0x7f7388009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b53650a32d10939 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:44:29.702 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:28.758+0000 7f7396a00700 1 -- 172.21.15.97:0/721004505 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7388004160 con 0x7f73981089f0 2024-03-27T10:44:29.702 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:28.758+0000 7f7396a00700 1 -- 172.21.15.97:0/721004505 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7388004300 con 0x7f73981089f0 2024-03-27T10:44:29.702 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:28.758+0000 7f7396a00700 1 -- 172.21.15.97:0/721004505 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7388011a50 con 0x7f73981089f0 2024-03-27T10:44:29.702 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:28.759+0000 7f739f200700 1 -- 172.21.15.97:0/721004505 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73981089f0 msgr2=0x7f739810ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:44:29.702 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:28.759+0000 7f739f200700 1 --2- 172.21.15.97:0/721004505 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73981089f0 0x7f739810ade0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f73880098a0 tx=0x7f7388009bb0 comp rx=0 tx=0).stop 2024-03-27T10:44:29.702 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:28.759+0000 7f739f200700 1 -- 172.21.15.97:0/721004505 shutdown_connections 2024-03-27T10:44:29.702 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:28.759+0000 7f739f200700 1 --2- 172.21.15.97:0/721004505 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73981089f0 0x7f739810ade0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:44:29.702 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:28.759+0000 7f739f200700 1 -- 172.21.15.97:0/721004505 >> 172.21.15.97:0/721004505 conn(0x7f73981023e0 msgr2=0x7f7398104860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:44:29.702 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:28.759+0000 7f739f200700 1 -- 172.21.15.97:0/721004505 shutdown_connections 2024-03-27T10:44:29.702 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:28.759+0000 7f739f200700 1 -- 172.21.15.97:0/721004505 wait complete. 2024-03-27T10:44:29.703 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:28.761+0000 7f739f200700 1 Processor -- start 2024-03-27T10:44:29.703 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:28.761+0000 7f739f200700 1 -- start start 2024-03-27T10:44:29.703 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:28.761+0000 7f739f200700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73981089f0 0x7f739819e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:44:29.703 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:28.761+0000 7f739f200700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f739810c330 con 0x7f73981089f0 2024-03-27T10:44:29.703 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:28.762+0000 7f7397e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73981089f0 0x7f739819e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:44:29.703 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:28.762+0000 7f7397e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73981089f0 0x7f739819e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40850/0 (socket says 172.21.15.97:40850) 2024-03-27T10:44:29.703 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:28.762+0000 7f7397e00700 1 -- 172.21.15.97:0/2084166410 learned_addr learned my addr 172.21.15.97:0/2084166410 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:44:29.703 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:28.762+0000 7f7397e00700 1 -- 172.21.15.97:0/2084166410 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f739819e660 con 0x7f73981089f0 2024-03-27T10:44:29.703 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:28.762+0000 7f7397e00700 1 --2- 172.21.15.97:0/2084166410 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73981089f0 0x7f739819e120 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7388011a00 tx=0x7f7388011ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:44:29.703 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:28.762+0000 7f7394c00700 1 -- 172.21.15.97:0/2084166410 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7388003af0 con 0x7f73981089f0 2024-03-27T10:44:29.703 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:28.762+0000 7f7394c00700 1 -- 172.21.15.97:0/2084166410 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f738801a8e0 con 0x7f73981089f0 2024-03-27T10:44:29.703 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:28.762+0000 7f7394c00700 1 -- 172.21.15.97:0/2084166410 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7388024840 con 0x7f73981089f0 2024-03-27T10:44:29.703 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:28.762+0000 7f739f200700 1 -- 172.21.15.97:0/2084166410 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f739819e8f0 con 0x7f73981089f0 2024-03-27T10:44:29.703 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:28.763+0000 7f739f200700 1 -- 172.21.15.97:0/2084166410 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73981a15e0 con 0x7f73981089f0 2024-03-27T10:44:29.703 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:28.764+0000 7f739f200700 1 -- 172.21.15.97:0/2084166410 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7398063c70 con 0x7f73981089f0 2024-03-27T10:44:29.704 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:28.764+0000 7f7394c00700 1 -- 172.21.15.97:0/2084166410 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) ==== 46466+0+0 (secure 0 0 0) 0x7f7388022020 con 0x7f73981089f0 2024-03-27T10:44:29.704 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:28.765+0000 7f7394c00700 1 --2- 172.21.15.97:0/2084166410 >> [v2:172.21.15.97:6800/1685921853,v1:172.21.15.97:6801/1685921853] conn(0x7f737c039420 0x7f737c03b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:44:29.704 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:28.765+0000 7f7394c00700 1 -- 172.21.15.97:0/2084166410 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f738804d830 con 0x7f73981089f0 2024-03-27T10:44:29.704 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:28.765+0000 7f7397400700 1 --2- 172.21.15.97:0/2084166410 >> [v2:172.21.15.97:6800/1685921853,v1:172.21.15.97:6801/1685921853] conn(0x7f737c039420 0x7f737c03b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:44:29.704 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:28.766+0000 7f7397400700 1 --2- 172.21.15.97:0/2084166410 >> [v2:172.21.15.97:6800/1685921853,v1:172.21.15.97:6801/1685921853] conn(0x7f737c039420 0x7f737c03b8e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f7384006fd0 tx=0x7f7384006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:44:29.704 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:28.770+0000 7f7394c00700 1 -- 172.21.15.97:0/2084166410 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f738800b3e0 con 0x7f73981089f0 2024-03-27T10:44:29.704 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:28.998+0000 7f739f200700 1 -- 172.21.15.97:0/2084166410 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f73981a1c00 con 0x7f73981089f0 2024-03-27T10:44:29.704 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:29.004+0000 7f7394c00700 1 -- 172.21.15.97:0/2084166410 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f738802ddb0 con 0x7f73981089f0 2024-03-27T10:44:29.704 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:29.007+0000 7f739f200700 1 -- 172.21.15.97:0/2084166410 >> [v2:172.21.15.97:6800/1685921853,v1:172.21.15.97:6801/1685921853] conn(0x7f737c039420 msgr2=0x7f737c03b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:44:29.704 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:29.007+0000 7f739f200700 1 --2- 172.21.15.97:0/2084166410 >> [v2:172.21.15.97:6800/1685921853,v1:172.21.15.97:6801/1685921853] conn(0x7f737c039420 0x7f737c03b8e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f7384006fd0 tx=0x7f7384006e40 comp rx=0 tx=0).stop 2024-03-27T10:44:29.704 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:29.007+0000 7f739f200700 1 -- 172.21.15.97:0/2084166410 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73981089f0 msgr2=0x7f739819e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:44:29.704 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:29.007+0000 7f739f200700 1 --2- 172.21.15.97:0/2084166410 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73981089f0 0x7f739819e120 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7388011a00 tx=0x7f7388011ae0 comp rx=0 tx=0).stop 2024-03-27T10:44:29.704 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:29.007+0000 7f739f200700 1 -- 172.21.15.97:0/2084166410 shutdown_connections 2024-03-27T10:44:29.704 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:29.007+0000 7f739f200700 1 --2- 172.21.15.97:0/2084166410 >> [v2:172.21.15.97:6800/1685921853,v1:172.21.15.97:6801/1685921853] conn(0x7f737c039420 0x7f737c03b8e0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:44:29.705 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:29.007+0000 7f739f200700 1 --2- 172.21.15.97:0/2084166410 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73981089f0 0x7f739819e120 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:44:29.705 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:29.007+0000 7f739f200700 1 -- 172.21.15.97:0/2084166410 >> 172.21.15.97:0/2084166410 conn(0x7f73981023e0 msgr2=0x7f7398102f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:44:29.705 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:29.007+0000 7f739f200700 1 -- 172.21.15.97:0/2084166410 shutdown_connections 2024-03-27T10:44:29.705 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:29.008+0000 7f739f200700 1 -- 172.21.15.97:0/2084166410 wait complete. 2024-03-27T10:44:29.705 INFO:teuthology.orchestra.run.smithi097.stdout:Enabling the dashboard module... 2024-03-27T10:44:30.246 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:44:30 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/2084166410' entity='client.admin' 2024-03-27T10:44:31.179 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:44:31 smithi097 ceph-mon[31859]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:44:31.179 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:44:31 smithi097 ceph-mon[31859]: from='mgr.14120 172.21.15.97:0/4145774229' entity='mgr.x' 2024-03-27T10:44:31.179 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:44:31 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/661723432' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-03-27T10:44:31.435 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:44:31 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: ignoring --setuser ceph since I am not root 2024-03-27T10:44:31.435 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:44:31 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: ignoring --setgroup ceph since I am not root 2024-03-27T10:44:32.143 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:44:32 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/661723432' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-03-27T10:44:32.143 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:44:32 smithi097 ceph-mon[31859]: mgrmap e9: x(active, since 25s) 2024-03-27T10:44:32.144 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:44:31 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: 2024-03-27T10:44:31.881+0000 7fb348806200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-27T10:44:32.144 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:44:32 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: 2024-03-27T10:44:32.006+0000 7fb348806200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-27T10:44:32.264 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:30.329+0000 7efff2c00700 1 Processor -- start 2024-03-27T10:44:32.264 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:30.330+0000 7efff2c00700 1 -- start start 2024-03-27T10:44:32.264 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:30.330+0000 7efff2c00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7effec108a30 0x7effec10ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:44:32.264 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:30.330+0000 7efff2c00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7effec07be90 con 0x7effec108a30 2024-03-27T10:44:32.264 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:30.331+0000 7effebe00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7effec108a30 0x7effec10ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:44:32.264 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:30.331+0000 7effebe00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7effec108a30 0x7effec10ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53584/0 (socket says 172.21.15.97:53584) 2024-03-27T10:44:32.264 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:30.331+0000 7effebe00700 1 -- 172.21.15.97:0/2720642160 learned_addr learned my addr 172.21.15.97:0/2720642160 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:44:32.265 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:30.331+0000 7effebe00700 1 -- 172.21.15.97:0/2720642160 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effec10b360 con 0x7effec108a30 2024-03-27T10:44:32.265 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:30.332+0000 7effebe00700 1 --2- 172.21.15.97:0/2720642160 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7effec108a30 0x7effec10ae20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7effdc0098a0 tx=0x7effdc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24ad36cd3d1d672d server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:44:32.265 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:30.332+0000 7effeaa00700 1 -- 172.21.15.97:0/2720642160 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7effdc004160 con 0x7effec108a30 2024-03-27T10:44:32.265 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:30.332+0000 7effeaa00700 1 -- 172.21.15.97:0/2720642160 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7effdc004300 con 0x7effec108a30 2024-03-27T10:44:32.265 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:30.332+0000 7effeaa00700 1 -- 172.21.15.97:0/2720642160 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7effdc011a50 con 0x7effec108a30 2024-03-27T10:44:32.265 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:30.333+0000 7efff2c00700 1 -- 172.21.15.97:0/2720642160 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7effec108a30 msgr2=0x7effec10ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:44:32.265 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:30.333+0000 7efff2c00700 1 --2- 172.21.15.97:0/2720642160 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7effec108a30 0x7effec10ae20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7effdc0098a0 tx=0x7effdc009bb0 comp rx=0 tx=0).stop 2024-03-27T10:44:32.265 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:30.333+0000 7efff2c00700 1 -- 172.21.15.97:0/2720642160 shutdown_connections 2024-03-27T10:44:32.265 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:30.333+0000 7efff2c00700 1 --2- 172.21.15.97:0/2720642160 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7effec108a30 0x7effec10ae20 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:44:32.265 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:30.333+0000 7efff2c00700 1 -- 172.21.15.97:0/2720642160 >> 172.21.15.97:0/2720642160 conn(0x7effec1023e0 msgr2=0x7effec104860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:44:32.265 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:30.333+0000 7efff2c00700 1 -- 172.21.15.97:0/2720642160 shutdown_connections 2024-03-27T10:44:32.265 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:30.333+0000 7efff2c00700 1 -- 172.21.15.97:0/2720642160 wait complete. 2024-03-27T10:44:32.265 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:30.335+0000 7efff2c00700 1 Processor -- start 2024-03-27T10:44:32.266 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:30.335+0000 7efff2c00700 1 -- start start 2024-03-27T10:44:32.266 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:30.336+0000 7efff2c00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7effec108a30 0x7effec1082b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:44:32.266 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:30.336+0000 7efff2c00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7effec10c360 con 0x7effec108a30 2024-03-27T10:44:32.266 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:30.336+0000 7effebe00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7effec108a30 0x7effec1082b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:44:32.266 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:30.336+0000 7effebe00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7effec108a30 0x7effec1082b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53598/0 (socket says 172.21.15.97:53598) 2024-03-27T10:44:32.266 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:30.336+0000 7effebe00700 1 -- 172.21.15.97:0/661723432 learned_addr learned my addr 172.21.15.97:0/661723432 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:44:32.266 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:30.336+0000 7effebe00700 1 -- 172.21.15.97:0/661723432 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effec1087f0 con 0x7effec108a30 2024-03-27T10:44:32.266 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:30.337+0000 7effebe00700 1 --2- 172.21.15.97:0/661723432 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7effec108a30 0x7effec1082b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7effdc003f90 tx=0x7effdc011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:44:32.266 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:30.337+0000 7effe8c00700 1 -- 172.21.15.97:0/661723432 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7effdc0039e0 con 0x7effec108a30 2024-03-27T10:44:32.266 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:30.337+0000 7efff2c00700 1 -- 172.21.15.97:0/661723432 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effec1069e0 con 0x7effec108a30 2024-03-27T10:44:32.266 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:30.337+0000 7effe8c00700 1 -- 172.21.15.97:0/661723432 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7effdc01a8e0 con 0x7effec108a30 2024-03-27T10:44:32.266 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:30.337+0000 7effe8c00700 1 -- 172.21.15.97:0/661723432 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7effdc024760 con 0x7effec108a30 2024-03-27T10:44:32.266 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:30.337+0000 7efff2c00700 1 -- 172.21.15.97:0/661723432 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effec106f20 con 0x7effec108a30 2024-03-27T10:44:32.267 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:30.339+0000 7effe8c00700 1 -- 172.21.15.97:0/661723432 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) ==== 46466+0+0 (secure 0 0 0) 0x7effdc022020 con 0x7effec108a30 2024-03-27T10:44:32.267 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:30.339+0000 7efff2c00700 1 -- 172.21.15.97:0/661723432 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effcc005360 con 0x7effec108a30 2024-03-27T10:44:32.267 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:30.339+0000 7effe8c00700 1 --2- 172.21.15.97:0/661723432 >> [v2:172.21.15.97:6800/1685921853,v1:172.21.15.97:6801/1685921853] conn(0x7effd00393c0 0x7effd003b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:44:32.267 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:30.339+0000 7effe8c00700 1 -- 172.21.15.97:0/661723432 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7effdc04d900 con 0x7effec108a30 2024-03-27T10:44:32.267 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:30.339+0000 7effeb400700 1 --2- 172.21.15.97:0/661723432 >> [v2:172.21.15.97:6800/1685921853,v1:172.21.15.97:6801/1685921853] conn(0x7effd00393c0 0x7effd003b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:44:32.267 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:30.340+0000 7effeb400700 1 --2- 172.21.15.97:0/661723432 >> [v2:172.21.15.97:6800/1685921853,v1:172.21.15.97:6801/1685921853] conn(0x7effd00393c0 0x7effd003b880 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7effd8006fd0 tx=0x7effd8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:44:32.267 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:30.350+0000 7effe8c00700 1 -- 172.21.15.97:0/661723432 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7effdc04d250 con 0x7effec108a30 2024-03-27T10:44:32.267 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:30.634+0000 7efff2c00700 1 -- 172.21.15.97:0/661723432 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7effcc005650 con 0x7effec108a30 2024-03-27T10:44:32.267 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:31.084+0000 7effe8c00700 1 -- 172.21.15.97:0/661723432 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7effdc01ab00 con 0x7effec108a30 2024-03-27T10:44:32.267 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:31.086+0000 7effe8c00700 1 -- 172.21.15.97:0/661723432 <== mon.0 v2:172.21.15.97:3300/0 8 ==== mgrmap(e 9) ==== 46479+0+0 (secure 0 0 0) 0x7effdc02d770 con 0x7effec108a30 2024-03-27T10:44:32.267 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:31.089+0000 7efff2c00700 1 -- 172.21.15.97:0/661723432 >> [v2:172.21.15.97:6800/1685921853,v1:172.21.15.97:6801/1685921853] conn(0x7effd00393c0 msgr2=0x7effd003b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:44:32.267 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:31.089+0000 7efff2c00700 1 --2- 172.21.15.97:0/661723432 >> [v2:172.21.15.97:6800/1685921853,v1:172.21.15.97:6801/1685921853] conn(0x7effd00393c0 0x7effd003b880 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7effd8006fd0 tx=0x7effd8006e40 comp rx=0 tx=0).stop 2024-03-27T10:44:32.267 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:31.089+0000 7efff2c00700 1 -- 172.21.15.97:0/661723432 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7effec108a30 msgr2=0x7effec1082b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:44:32.268 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:31.089+0000 7efff2c00700 1 --2- 172.21.15.97:0/661723432 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7effec108a30 0x7effec1082b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7effdc003f90 tx=0x7effdc011ad0 comp rx=0 tx=0).stop 2024-03-27T10:44:32.268 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:31.089+0000 7efff2c00700 1 -- 172.21.15.97:0/661723432 shutdown_connections 2024-03-27T10:44:32.268 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:31.089+0000 7efff2c00700 1 --2- 172.21.15.97:0/661723432 >> [v2:172.21.15.97:6800/1685921853,v1:172.21.15.97:6801/1685921853] conn(0x7effd00393c0 0x7effd003b880 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:44:32.268 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:31.089+0000 7efff2c00700 1 --2- 172.21.15.97:0/661723432 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7effec108a30 0x7effec1082b0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:44:32.268 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:31.089+0000 7efff2c00700 1 -- 172.21.15.97:0/661723432 >> 172.21.15.97:0/661723432 conn(0x7effec1023e0 msgr2=0x7effec102fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:44:32.268 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:31.090+0000 7efff2c00700 1 -- 172.21.15.97:0/661723432 shutdown_connections 2024-03-27T10:44:32.268 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:31.090+0000 7efff2c00700 1 -- 172.21.15.97:0/661723432 wait complete. 2024-03-27T10:44:32.894 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:44:32 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: 2024-03-27T10:44:32.810+0000 7fb348806200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-27T10:44:33.201 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:44:32 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: 2024-03-27T10:44:32.936+0000 7fb348806200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-27T10:44:33.202 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:44:33 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: 2024-03-27T10:44:33.118+0000 7fb348806200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-27T10:44:33.627 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:44:33 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/2957977708' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-27T10:44:33.627 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:44:33 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: 2024-03-27T10:44:33.246+0000 7fb348806200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-27T10:44:33.627 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:44:33 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: 2024-03-27T10:44:33.359+0000 7fb348806200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-27T10:44:33.893 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:44:33 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: 2024-03-27T10:44:33.692+0000 7fb348806200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-27T10:44:33.893 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:44:33 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: 2024-03-27T10:44:33.832+0000 7fb348806200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-27T10:44:33.988 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout { 2024-03-27T10:44:33.988 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-03-27T10:44:33.989 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-27T10:44:33.989 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-03-27T10:44:33.989 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-27T10:44:33.989 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout } 2024-03-27T10:44:33.989 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:32.951+0000 7f4530c00700 1 Processor -- start 2024-03-27T10:44:33.989 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:32.951+0000 7f4530c00700 1 -- start start 2024-03-27T10:44:33.989 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:32.952+0000 7f4530c00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f452c07dea0 0x7f452c07c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:44:33.989 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:32.952+0000 7f4530c00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f452c07c8d0 con 0x7f452c07dea0 2024-03-27T10:44:33.989 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:32.952+0000 7f4529e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f452c07dea0 0x7f452c07c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:44:33.989 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:32.952+0000 7f4529e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f452c07dea0 0x7f452c07c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53622/0 (socket says 172.21.15.97:53622) 2024-03-27T10:44:33.989 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:32.952+0000 7f4529e00700 1 -- 172.21.15.97:0/97588356 learned_addr learned my addr 172.21.15.97:0/97588356 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:44:33.989 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:32.953+0000 7f4529e00700 1 -- 172.21.15.97:0/97588356 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f452c07ca50 con 0x7f452c07dea0 2024-03-27T10:44:33.990 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:32.953+0000 7f4529e00700 1 --2- 172.21.15.97:0/97588356 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f452c07dea0 0x7f452c07c300 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f451c0098a0 tx=0x7f451c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdb47e7bb2e8d3a0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:44:33.990 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:32.953+0000 7f4528a00700 1 -- 172.21.15.97:0/97588356 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f451c004160 con 0x7f452c07dea0 2024-03-27T10:44:33.990 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:32.953+0000 7f4528a00700 1 -- 172.21.15.97:0/97588356 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f451c004300 con 0x7f452c07dea0 2024-03-27T10:44:33.990 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:32.953+0000 7f4528a00700 1 -- 172.21.15.97:0/97588356 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f451c011a50 con 0x7f452c07dea0 2024-03-27T10:44:33.990 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:32.954+0000 7f4530c00700 1 -- 172.21.15.97:0/97588356 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f452c07dea0 msgr2=0x7f452c07c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:44:33.990 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:32.954+0000 7f4530c00700 1 --2- 172.21.15.97:0/97588356 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f452c07dea0 0x7f452c07c300 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f451c0098a0 tx=0x7f451c009bb0 comp rx=0 tx=0).stop 2024-03-27T10:44:33.990 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:32.954+0000 7f4530c00700 1 -- 172.21.15.97:0/97588356 shutdown_connections 2024-03-27T10:44:33.990 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:32.954+0000 7f4530c00700 1 --2- 172.21.15.97:0/97588356 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f452c07dea0 0x7f452c07c300 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:44:33.990 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:32.954+0000 7f4530c00700 1 -- 172.21.15.97:0/97588356 >> 172.21.15.97:0/97588356 conn(0x7f452c103570 msgr2=0x7f452c1059d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:44:33.990 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:32.954+0000 7f4530c00700 1 -- 172.21.15.97:0/97588356 shutdown_connections 2024-03-27T10:44:33.990 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:32.954+0000 7f4530c00700 1 -- 172.21.15.97:0/97588356 wait complete. 2024-03-27T10:44:33.990 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:32.955+0000 7f4530c00700 1 Processor -- start 2024-03-27T10:44:33.991 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:32.955+0000 7f4530c00700 1 -- start start 2024-03-27T10:44:33.991 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:32.955+0000 7f4530c00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f452c07dea0 0x7f452c19e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:44:33.991 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:32.955+0000 7f4530c00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f452c10e610 con 0x7f452c07dea0 2024-03-27T10:44:33.991 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:32.956+0000 7f4529e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f452c07dea0 0x7f452c19e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:44:33.991 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:32.956+0000 7f4529e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f452c07dea0 0x7f452c19e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53636/0 (socket says 172.21.15.97:53636) 2024-03-27T10:44:33.991 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:32.956+0000 7f4529e00700 1 -- 172.21.15.97:0/2957977708 learned_addr learned my addr 172.21.15.97:0/2957977708 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:44:33.991 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:32.956+0000 7f4529e00700 1 -- 172.21.15.97:0/2957977708 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f452c19e640 con 0x7f452c07dea0 2024-03-27T10:44:33.991 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:32.956+0000 7f4529e00700 1 --2- 172.21.15.97:0/2957977708 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f452c07dea0 0x7f452c19e100 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f451c011a00 tx=0x7f451c011ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:44:33.991 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:32.957+0000 7f4522a00700 1 -- 172.21.15.97:0/2957977708 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f451c003af0 con 0x7f452c07dea0 2024-03-27T10:44:33.991 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:32.957+0000 7f4522a00700 1 -- 172.21.15.97:0/2957977708 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f451c01a8e0 con 0x7f452c07dea0 2024-03-27T10:44:33.991 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:32.957+0000 7f4530c00700 1 -- 172.21.15.97:0/2957977708 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f452c19e8d0 con 0x7f452c07dea0 2024-03-27T10:44:33.991 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:32.957+0000 7f4522a00700 1 -- 172.21.15.97:0/2957977708 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f451c024840 con 0x7f452c07dea0 2024-03-27T10:44:33.991 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:32.957+0000 7f4530c00700 1 -- 172.21.15.97:0/2957977708 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f452c1a15c0 con 0x7f452c07dea0 2024-03-27T10:44:33.992 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:32.958+0000 7f4522a00700 1 -- 172.21.15.97:0/2957977708 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 9) ==== 46479+0+0 (secure 0 0 0) 0x7f451c022020 con 0x7f452c07dea0 2024-03-27T10:44:33.992 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:32.958+0000 7f4530c00700 1 -- 172.21.15.97:0/2957977708 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f452c063c70 con 0x7f452c07dea0 2024-03-27T10:44:33.992 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:32.958+0000 7f4522a00700 1 --2- 172.21.15.97:0/2957977708 >> [v2:172.21.15.97:6800/1685921853,v1:172.21.15.97:6801/1685921853] conn(0x7f4510039480 0x7f451003b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:44:33.992 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:32.958+0000 7f4522a00700 1 -- 172.21.15.97:0/2957977708 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f451c04d920 con 0x7f452c07dea0 2024-03-27T10:44:33.992 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:32.958+0000 7f4529400700 1 -- 172.21.15.97:0/2957977708 >> [v2:172.21.15.97:6800/1685921853,v1:172.21.15.97:6801/1685921853] conn(0x7f4510039480 msgr2=0x7f451003b940 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.97:6800/1685921853 2024-03-27T10:44:33.992 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:32.958+0000 7f4529400700 1 --2- 172.21.15.97:0/2957977708 >> [v2:172.21.15.97:6800/1685921853,v1:172.21.15.97:6801/1685921853] conn(0x7f4510039480 0x7f451003b940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-27T10:44:33.992 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:32.963+0000 7f4522a00700 1 -- 172.21.15.97:0/2957977708 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f451c0172b0 con 0x7f452c07dea0 2024-03-27T10:44:33.992 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:33.159+0000 7f4529400700 1 -- 172.21.15.97:0/2957977708 >> [v2:172.21.15.97:6800/1685921853,v1:172.21.15.97:6801/1685921853] conn(0x7f4510039480 msgr2=0x7f451003b940 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.97:6800/1685921853 2024-03-27T10:44:33.992 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:33.159+0000 7f4529400700 1 --2- 172.21.15.97:0/2957977708 >> [v2:172.21.15.97:6800/1685921853,v1:172.21.15.97:6801/1685921853] conn(0x7f4510039480 0x7f451003b940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-27T10:44:33.992 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:33.253+0000 7f4530c00700 1 -- 172.21.15.97:0/2957977708 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f452c1a1b70 con 0x7f452c07dea0 2024-03-27T10:44:33.992 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:33.254+0000 7f4522a00700 1 -- 172.21.15.97:0/2957977708 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f451c01adb0 con 0x7f452c07dea0 2024-03-27T10:44:33.992 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:33.257+0000 7f4530c00700 1 -- 172.21.15.97:0/2957977708 >> [v2:172.21.15.97:6800/1685921853,v1:172.21.15.97:6801/1685921853] conn(0x7f4510039480 msgr2=0x7f451003b940 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-27T10:44:33.992 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:33.257+0000 7f4530c00700 1 --2- 172.21.15.97:0/2957977708 >> [v2:172.21.15.97:6800/1685921853,v1:172.21.15.97:6801/1685921853] conn(0x7f4510039480 0x7f451003b940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:44:33.992 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:33.257+0000 7f4530c00700 1 -- 172.21.15.97:0/2957977708 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f452c07dea0 msgr2=0x7f452c19e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:44:33.993 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:33.257+0000 7f4530c00700 1 --2- 172.21.15.97:0/2957977708 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f452c07dea0 0x7f452c19e100 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f451c011a00 tx=0x7f451c011ae0 comp rx=0 tx=0).stop 2024-03-27T10:44:33.993 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:33.257+0000 7f4530c00700 1 -- 172.21.15.97:0/2957977708 shutdown_connections 2024-03-27T10:44:33.993 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:33.257+0000 7f4530c00700 1 --2- 172.21.15.97:0/2957977708 >> [v2:172.21.15.97:6800/1685921853,v1:172.21.15.97:6801/1685921853] conn(0x7f4510039480 0x7f451003b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:44:33.993 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:33.257+0000 7f4530c00700 1 --2- 172.21.15.97:0/2957977708 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f452c07dea0 0x7f452c19e100 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:44:33.993 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:33.257+0000 7f4530c00700 1 -- 172.21.15.97:0/2957977708 >> 172.21.15.97:0/2957977708 conn(0x7f452c103570 msgr2=0x7f452c104110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:44:33.993 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:33.257+0000 7f4530c00700 1 -- 172.21.15.97:0/2957977708 shutdown_connections 2024-03-27T10:44:33.993 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:33.257+0000 7f4530c00700 1 -- 172.21.15.97:0/2957977708 wait complete. 2024-03-27T10:44:33.993 INFO:teuthology.orchestra.run.smithi097.stdout:Waiting for the mgr to restart... 2024-03-27T10:44:33.993 INFO:teuthology.orchestra.run.smithi097.stdout:Waiting for mgr epoch 9... 2024-03-27T10:44:34.283 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:44:33 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: 2024-03-27T10:44:33.947+0000 7fb348806200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-27T10:44:34.283 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:44:34 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: 2024-03-27T10:44:34.087+0000 7fb348806200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-27T10:44:34.283 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:44:34 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: 2024-03-27T10:44:34.307+0000 7fb348806200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-27T10:44:35.893 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:44:35 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: 2024-03-27T10:44:35.586+0000 7fb348806200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-27T10:44:35.893 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:44:35 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: 2024-03-27T10:44:35.756+0000 7fb348806200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-27T10:44:36.393 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:44:36 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: 2024-03-27T10:44:36.171+0000 7fb348806200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-27T10:44:36.393 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:44:36 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: 2024-03-27T10:44:36.289+0000 7fb348806200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-27T10:44:36.893 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:44:36 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: 2024-03-27T10:44:36.545+0000 7fb348806200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-27T10:44:36.893 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:44:36 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: 2024-03-27T10:44:36.788+0000 7fb348806200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-27T10:44:37.393 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:44:37 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: 2024-03-27T10:44:37.160+0000 7fb348806200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-27T10:44:37.753 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:44:37 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: 2024-03-27T10:44:37.474+0000 7fb348806200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-27T10:44:37.754 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:44:37 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: 2024-03-27T10:44:37.777+0000 7fb348806200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-27T10:44:39.143 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:44:38 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: 2024-03-27T10:44:38.792+0000 7fb348806200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-27T10:44:39.862 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:44:39 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: 2024-03-27T10:44:39.480+0000 7fb348806200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-27T10:44:39.862 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:44:39 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: 2024-03-27T10:44:39.607+0000 7fb348806200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-27T10:44:40.143 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:44:39 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: 2024-03-27T10:44:39.886+0000 7fb348806200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-27T10:44:40.143 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:44:39 smithi097 ceph-mon[31859]: Active manager daemon x restarted 2024-03-27T10:44:40.143 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:44:39 smithi097 ceph-mon[31859]: Activating manager daemon x 2024-03-27T10:44:40.143 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:44:39 smithi097 ceph-mon[31859]: osdmap e3: 0 total, 0 up, 0 in 2024-03-27T10:44:40.144 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:44:39 smithi097 ceph-mon[31859]: mgrmap e10: x(active, starting, since 0.00395625s) 2024-03-27T10:44:40.144 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:44:39 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-27T10:44:40.144 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:44:39 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-27T10:44:40.144 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:44:39 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-27T10:44:40.144 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:44:39 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-27T10:44:40.144 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:44:39 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-27T10:44:40.144 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:44:39 smithi097 ceph-mon[31859]: Manager daemon x is now available 2024-03-27T10:44:41.144 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:44:40 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:44:41.144 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:44:40 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-27T10:44:41.144 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:44:40 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-27T10:44:41.144 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:44:40 smithi097 ceph-mon[31859]: mgrmap e11: x(active, since 1.00879s) 2024-03-27T10:44:41.554 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout { 2024-03-27T10:44:41.554 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-03-27T10:44:41.554 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-27T10:44:41.554 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout } 2024-03-27T10:44:41.554 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:34.640+0000 7f1e5f800700 1 Processor -- start 2024-03-27T10:44:41.561 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:34.640+0000 7f1e5f800700 1 -- start start 2024-03-27T10:44:41.562 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:34.641+0000 7f1e5f800700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e58108a70 0x7f1e5810ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:44:41.562 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:34.641+0000 7f1e5f800700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1e5807be50 con 0x7f1e58108a70 2024-03-27T10:44:41.562 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:34.641+0000 7f1e5ce00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e58108a70 0x7f1e5810ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:44:41.562 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:34.641+0000 7f1e5ce00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e58108a70 0x7f1e5810ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53646/0 (socket says 172.21.15.97:53646) 2024-03-27T10:44:41.562 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:34.641+0000 7f1e5ce00700 1 -- 172.21.15.97:0/3104957240 learned_addr learned my addr 172.21.15.97:0/3104957240 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:44:41.562 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:34.642+0000 7f1e5ce00700 1 -- 172.21.15.97:0/3104957240 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e5810b3a0 con 0x7f1e58108a70 2024-03-27T10:44:41.562 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:34.642+0000 7f1e5ce00700 1 --2- 172.21.15.97:0/3104957240 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e58108a70 0x7f1e5810ae60 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c0098a0 tx=0x7f1e4c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20cc5d888ef087e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:44:41.562 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:34.642+0000 7f1e57400700 1 -- 172.21.15.97:0/3104957240 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1e4c004160 con 0x7f1e58108a70 2024-03-27T10:44:41.562 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:34.642+0000 7f1e57400700 1 -- 172.21.15.97:0/3104957240 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1e4c004300 con 0x7f1e58108a70 2024-03-27T10:44:41.562 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:34.642+0000 7f1e57400700 1 -- 172.21.15.97:0/3104957240 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1e4c011a50 con 0x7f1e58108a70 2024-03-27T10:44:41.562 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:34.643+0000 7f1e5f800700 1 -- 172.21.15.97:0/3104957240 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e58108a70 msgr2=0x7f1e5810ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:44:41.562 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:34.643+0000 7f1e5f800700 1 --2- 172.21.15.97:0/3104957240 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e58108a70 0x7f1e5810ae60 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c0098a0 tx=0x7f1e4c009bb0 comp rx=0 tx=0).stop 2024-03-27T10:44:41.562 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:34.643+0000 7f1e5f800700 1 -- 172.21.15.97:0/3104957240 shutdown_connections 2024-03-27T10:44:41.563 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:34.643+0000 7f1e5f800700 1 --2- 172.21.15.97:0/3104957240 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e58108a70 0x7f1e5810ae60 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:44:41.563 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:34.643+0000 7f1e5f800700 1 -- 172.21.15.97:0/3104957240 >> 172.21.15.97:0/3104957240 conn(0x7f1e581023e0 msgr2=0x7f1e58104860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:44:41.563 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:34.643+0000 7f1e5f800700 1 -- 172.21.15.97:0/3104957240 shutdown_connections 2024-03-27T10:44:41.563 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:34.643+0000 7f1e5f800700 1 -- 172.21.15.97:0/3104957240 wait complete. 2024-03-27T10:44:41.563 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:34.644+0000 7f1e5f800700 1 Processor -- start 2024-03-27T10:44:41.563 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:34.645+0000 7f1e5f800700 1 -- start start 2024-03-27T10:44:41.563 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:34.645+0000 7f1e5f800700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e58108a70 0x7f1e5819e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:44:41.563 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:34.645+0000 7f1e5f800700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1e5810c340 con 0x7f1e58108a70 2024-03-27T10:44:41.563 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:34.645+0000 7f1e5ce00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e58108a70 0x7f1e5819e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:44:41.563 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:34.645+0000 7f1e5ce00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e58108a70 0x7f1e5819e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53650/0 (socket says 172.21.15.97:53650) 2024-03-27T10:44:41.563 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:34.645+0000 7f1e5ce00700 1 -- 172.21.15.97:0/950828152 learned_addr learned my addr 172.21.15.97:0/950828152 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:44:41.563 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:34.646+0000 7f1e5ce00700 1 -- 172.21.15.97:0/950828152 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e5819e6c0 con 0x7f1e58108a70 2024-03-27T10:44:41.563 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:34.646+0000 7f1e5ce00700 1 --2- 172.21.15.97:0/950828152 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e58108a70 0x7f1e5819e180 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c011a00 tx=0x7f1e4c011ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:44:41.563 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:34.646+0000 7f1e55600700 1 -- 172.21.15.97:0/950828152 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1e4c003af0 con 0x7f1e58108a70 2024-03-27T10:44:41.564 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:34.646+0000 7f1e55600700 1 -- 172.21.15.97:0/950828152 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1e4c01a8e0 con 0x7f1e58108a70 2024-03-27T10:44:41.564 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:34.646+0000 7f1e55600700 1 -- 172.21.15.97:0/950828152 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1e4c024840 con 0x7f1e58108a70 2024-03-27T10:44:41.564 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:34.646+0000 7f1e5f800700 1 -- 172.21.15.97:0/950828152 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e5819e950 con 0x7f1e58108a70 2024-03-27T10:44:41.564 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:34.646+0000 7f1e5f800700 1 -- 172.21.15.97:0/950828152 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e581a1640 con 0x7f1e58108a70 2024-03-27T10:44:41.564 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:34.648+0000 7f1e55600700 1 -- 172.21.15.97:0/950828152 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 9) ==== 46479+0+0 (secure 0 0 0) 0x7f1e4c022020 con 0x7f1e58108a70 2024-03-27T10:44:41.564 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:34.648+0000 7f1e55600700 1 --2- 172.21.15.97:0/950828152 >> [v2:172.21.15.97:6800/1685921853,v1:172.21.15.97:6801/1685921853] conn(0x7f1e40039480 0x7f1e4003b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:44:41.564 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:34.648+0000 7f1e55600700 1 -- 172.21.15.97:0/950828152 --> [v2:172.21.15.97:6800/1685921853,v1:172.21.15.97:6801/1685921853] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f1e4003c050 con 0x7f1e40039480 2024-03-27T10:44:41.564 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:34.648+0000 7f1e55600700 1 -- 172.21.15.97:0/950828152 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f1e4c04d8a0 con 0x7f1e58108a70 2024-03-27T10:44:41.564 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:34.648+0000 7f1e57e00700 1 -- 172.21.15.97:0/950828152 >> [v2:172.21.15.97:6800/1685921853,v1:172.21.15.97:6801/1685921853] conn(0x7f1e40039480 msgr2=0x7f1e4003b940 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.97:6800/1685921853 2024-03-27T10:44:41.564 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:34.648+0000 7f1e57e00700 1 --2- 172.21.15.97:0/950828152 >> [v2:172.21.15.97:6800/1685921853,v1:172.21.15.97:6801/1685921853] conn(0x7f1e40039480 0x7f1e4003b940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-27T10:44:41.564 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:34.849+0000 7f1e57e00700 1 -- 172.21.15.97:0/950828152 >> [v2:172.21.15.97:6800/1685921853,v1:172.21.15.97:6801/1685921853] conn(0x7f1e40039480 msgr2=0x7f1e4003b940 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.97:6800/1685921853 2024-03-27T10:44:41.564 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:34.849+0000 7f1e57e00700 1 --2- 172.21.15.97:0/950828152 >> [v2:172.21.15.97:6800/1685921853,v1:172.21.15.97:6801/1685921853] conn(0x7f1e40039480 0x7f1e4003b940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-27T10:44:41.564 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:35.250+0000 7f1e57e00700 1 -- 172.21.15.97:0/950828152 >> [v2:172.21.15.97:6800/1685921853,v1:172.21.15.97:6801/1685921853] conn(0x7f1e40039480 msgr2=0x7f1e4003b940 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.97:6800/1685921853 2024-03-27T10:44:41.565 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:35.250+0000 7f1e57e00700 1 --2- 172.21.15.97:0/950828152 >> [v2:172.21.15.97:6800/1685921853,v1:172.21.15.97:6801/1685921853] conn(0x7f1e40039480 0x7f1e4003b940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-27T10:44:41.565 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:36.051+0000 7f1e57e00700 1 -- 172.21.15.97:0/950828152 >> [v2:172.21.15.97:6800/1685921853,v1:172.21.15.97:6801/1685921853] conn(0x7f1e40039480 msgr2=0x7f1e4003b940 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.97:6800/1685921853 2024-03-27T10:44:41.565 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:36.051+0000 7f1e57e00700 1 --2- 172.21.15.97:0/950828152 >> [v2:172.21.15.97:6800/1685921853,v1:172.21.15.97:6801/1685921853] conn(0x7f1e40039480 0x7f1e4003b940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-27T10:44:41.565 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:37.653+0000 7f1e57e00700 1 -- 172.21.15.97:0/950828152 >> [v2:172.21.15.97:6800/1685921853,v1:172.21.15.97:6801/1685921853] conn(0x7f1e40039480 msgr2=0x7f1e4003b940 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.97:6800/1685921853 2024-03-27T10:44:41.565 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:37.653+0000 7f1e57e00700 1 --2- 172.21.15.97:0/950828152 >> [v2:172.21.15.97:6800/1685921853,v1:172.21.15.97:6801/1685921853] conn(0x7f1e40039480 0x7f1e4003b940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-27T10:44:41.565 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:39.892+0000 7f1e55600700 1 -- 172.21.15.97:0/950828152 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mgrmap(e 10) ==== 46246+0+0 (secure 0 0 0) 0x7f1e4c0249e0 con 0x7f1e58108a70 2024-03-27T10:44:41.565 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:39.892+0000 7f1e55600700 1 -- 172.21.15.97:0/950828152 >> [v2:172.21.15.97:6800/1685921853,v1:172.21.15.97:6801/1685921853] conn(0x7f1e40039480 msgr2=0x7f1e4003b940 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-27T10:44:41.565 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:39.892+0000 7f1e55600700 1 --2- 172.21.15.97:0/950828152 >> [v2:172.21.15.97:6800/1685921853,v1:172.21.15.97:6801/1685921853] conn(0x7f1e40039480 0x7f1e4003b940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:44:41.565 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:40.894+0000 7f1e55600700 1 -- 172.21.15.97:0/950828152 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mgrmap(e 11) ==== 46373+0+0 (secure 0 0 0) 0x7f1e4c04f6e0 con 0x7f1e58108a70 2024-03-27T10:44:41.565 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:40.894+0000 7f1e55600700 1 --2- 172.21.15.97:0/950828152 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f1e4003cf00 0x7f1e4003f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:44:41.565 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:40.894+0000 7f1e55600700 1 -- 172.21.15.97:0/950828152 --> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f1e4c017ec0 con 0x7f1e4003cf00 2024-03-27T10:44:41.565 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:40.895+0000 7f1e57e00700 1 --2- 172.21.15.97:0/950828152 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f1e4003cf00 0x7f1e4003f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:44:41.565 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:40.896+0000 7f1e57e00700 1 --2- 172.21.15.97:0/950828152 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f1e4003cf00 0x7f1e4003f2f0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f1e44003a10 tx=0x7f1e44009270 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:44:41.566 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:40.897+0000 7f1e55600700 1 -- 172.21.15.97:0/950828152 <== mgr.14152 v2:172.21.15.97:6800/3955463538 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f1e4c017ec0 con 0x7f1e4003cf00 2024-03-27T10:44:41.566 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:40.905+0000 7f1e5f800700 1 -- 172.21.15.97:0/950828152 --> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f1e58063c70 con 0x7f1e4003cf00 2024-03-27T10:44:41.566 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:40.905+0000 7f1e55600700 1 -- 172.21.15.97:0/950828152 <== mgr.14152 v2:172.21.15.97:6800/3955463538 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f1e58063c70 con 0x7f1e4003cf00 2024-03-27T10:44:41.566 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:40.905+0000 7f1e5f800700 1 -- 172.21.15.97:0/950828152 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f1e4003cf00 msgr2=0x7f1e4003f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:44:41.566 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:40.905+0000 7f1e5f800700 1 --2- 172.21.15.97:0/950828152 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f1e4003cf00 0x7f1e4003f2f0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f1e44003a10 tx=0x7f1e44009270 comp rx=0 tx=0).stop 2024-03-27T10:44:41.566 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:40.905+0000 7f1e5f800700 1 -- 172.21.15.97:0/950828152 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e58108a70 msgr2=0x7f1e5819e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:44:41.566 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:40.905+0000 7f1e5f800700 1 --2- 172.21.15.97:0/950828152 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e58108a70 0x7f1e5819e180 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c011a00 tx=0x7f1e4c011ae0 comp rx=0 tx=0).stop 2024-03-27T10:44:41.566 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:40.905+0000 7f1e5f800700 1 -- 172.21.15.97:0/950828152 shutdown_connections 2024-03-27T10:44:41.566 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:40.905+0000 7f1e5f800700 1 --2- 172.21.15.97:0/950828152 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f1e4003cf00 0x7f1e4003f2f0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:44:41.566 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:40.905+0000 7f1e5f800700 1 --2- 172.21.15.97:0/950828152 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e58108a70 0x7f1e5819e180 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:44:41.566 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:40.905+0000 7f1e5f800700 1 --2- 172.21.15.97:0/950828152 >> [v2:172.21.15.97:6800/1685921853,v1:172.21.15.97:6801/1685921853] conn(0x7f1e40039480 0x7f1e4003b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:44:41.566 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:40.906+0000 7f1e5f800700 1 -- 172.21.15.97:0/950828152 >> 172.21.15.97:0/950828152 conn(0x7f1e581023e0 msgr2=0x7f1e58102f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:44:41.566 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:40.906+0000 7f1e5f800700 1 -- 172.21.15.97:0/950828152 shutdown_connections 2024-03-27T10:44:41.566 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:40.906+0000 7f1e5f800700 1 -- 172.21.15.97:0/950828152 wait complete. 2024-03-27T10:44:41.567 INFO:teuthology.orchestra.run.smithi097.stdout:mgr epoch 9 is available 2024-03-27T10:44:41.567 INFO:teuthology.orchestra.run.smithi097.stdout:Generating a dashboard self-signed certificate... 2024-03-27T10:44:42.271 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:44:41 smithi097 ceph-mon[31859]: [27/Mar/2024:10:44:40] ENGINE Bus STARTING 2024-03-27T10:44:42.271 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:44:41 smithi097 ceph-mon[31859]: [27/Mar/2024:10:44:40] ENGINE Serving on https://172.21.15.97:7150 2024-03-27T10:44:42.271 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:44:41 smithi097 ceph-mon[31859]: [27/Mar/2024:10:44:40] ENGINE Serving on http://172.21.15.97:8765 2024-03-27T10:44:42.271 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:44:41 smithi097 ceph-mon[31859]: [27/Mar/2024:10:44:40] ENGINE Bus STARTED 2024-03-27T10:44:42.271 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:44:41 smithi097 ceph-mon[31859]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-27T10:44:42.271 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:44:41 smithi097 ceph-mon[31859]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-27T10:44:43.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:44:42 smithi097 ceph-mon[31859]: mgrmap e12: x(active, since 2s) 2024-03-27T10:44:43.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:44:42 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:44:44.143 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:44:43 smithi097 ceph-mon[31859]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T10:44:44.143 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:44:43 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:44:44.144 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:44:43 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:44:44.155 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-03-27T10:44:44.161 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:42.838+0000 7f84d5000700 1 Processor -- start 2024-03-27T10:44:44.162 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:42.838+0000 7f84d5000700 1 -- start start 2024-03-27T10:44:44.162 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:42.838+0000 7f84d5000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84d0108ab0 0x7f84d010aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:44:44.162 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:42.838+0000 7f84d5000700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f84d007be10 con 0x7f84d0108ab0 2024-03-27T10:44:44.162 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:42.839+0000 7f84cde00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84d0108ab0 0x7f84d010aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:44:44.162 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:42.839+0000 7f84cde00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84d0108ab0 0x7f84d010aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60392/0 (socket says 172.21.15.97:60392) 2024-03-27T10:44:44.162 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:42.839+0000 7f84cde00700 1 -- 172.21.15.97:0/1215626435 learned_addr learned my addr 172.21.15.97:0/1215626435 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:44:44.162 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:42.840+0000 7f84cde00700 1 -- 172.21.15.97:0/1215626435 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84d010b3e0 con 0x7f84d0108ab0 2024-03-27T10:44:44.162 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:42.840+0000 7f84cde00700 1 --2- 172.21.15.97:0/1215626435 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84d0108ab0 0x7f84d010aea0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f84c00098a0 tx=0x7f84c0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7f87dbc562f9e7b server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:44:44.162 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:42.840+0000 7f84cd400700 1 -- 172.21.15.97:0/1215626435 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f84c0004160 con 0x7f84d0108ab0 2024-03-27T10:44:44.162 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:42.840+0000 7f84cd400700 1 -- 172.21.15.97:0/1215626435 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f84c0004300 con 0x7f84d0108ab0 2024-03-27T10:44:44.162 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:42.840+0000 7f84cd400700 1 -- 172.21.15.97:0/1215626435 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f84c0011a50 con 0x7f84d0108ab0 2024-03-27T10:44:44.162 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:42.841+0000 7f84d5000700 1 -- 172.21.15.97:0/1215626435 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84d0108ab0 msgr2=0x7f84d010aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:44:44.163 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:42.841+0000 7f84d5000700 1 --2- 172.21.15.97:0/1215626435 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84d0108ab0 0x7f84d010aea0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f84c00098a0 tx=0x7f84c0009bb0 comp rx=0 tx=0).stop 2024-03-27T10:44:44.163 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:42.841+0000 7f84d5000700 1 -- 172.21.15.97:0/1215626435 shutdown_connections 2024-03-27T10:44:44.163 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:42.841+0000 7f84d5000700 1 --2- 172.21.15.97:0/1215626435 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84d0108ab0 0x7f84d010aea0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:44:44.163 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:42.841+0000 7f84d5000700 1 -- 172.21.15.97:0/1215626435 >> 172.21.15.97:0/1215626435 conn(0x7f84d01023e0 msgr2=0x7f84d0104860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:44:44.163 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:42.841+0000 7f84d5000700 1 -- 172.21.15.97:0/1215626435 shutdown_connections 2024-03-27T10:44:44.163 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:42.841+0000 7f84d5000700 1 -- 172.21.15.97:0/1215626435 wait complete. 2024-03-27T10:44:44.163 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:42.842+0000 7f84d5000700 1 Processor -- start 2024-03-27T10:44:44.163 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:42.843+0000 7f84d5000700 1 -- start start 2024-03-27T10:44:44.163 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:42.843+0000 7f84d5000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84d0108ab0 0x7f84d019e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:44:44.163 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:42.843+0000 7f84d5000700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f84d010c330 con 0x7f84d0108ab0 2024-03-27T10:44:44.163 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:42.843+0000 7f84cde00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84d0108ab0 0x7f84d019e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:44:44.163 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:42.843+0000 7f84cde00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84d0108ab0 0x7f84d019e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60398/0 (socket says 172.21.15.97:60398) 2024-03-27T10:44:44.163 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:42.843+0000 7f84cde00700 1 -- 172.21.15.97:0/3287628675 learned_addr learned my addr 172.21.15.97:0/3287628675 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:44:44.163 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:42.843+0000 7f84cde00700 1 -- 172.21.15.97:0/3287628675 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84d019e640 con 0x7f84d0108ab0 2024-03-27T10:44:44.164 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:42.844+0000 7f84cde00700 1 --2- 172.21.15.97:0/3287628675 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84d0108ab0 0x7f84d019e100 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f84c0011a00 tx=0x7f84c0011ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:44:44.164 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:42.844+0000 7f84c6a00700 1 -- 172.21.15.97:0/3287628675 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f84c0003af0 con 0x7f84d0108ab0 2024-03-27T10:44:44.164 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:42.844+0000 7f84c6a00700 1 -- 172.21.15.97:0/3287628675 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f84c001a8e0 con 0x7f84d0108ab0 2024-03-27T10:44:44.164 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:42.844+0000 7f84c6a00700 1 -- 172.21.15.97:0/3287628675 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f84c0024840 con 0x7f84d0108ab0 2024-03-27T10:44:44.164 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:42.844+0000 7f84d5000700 1 -- 172.21.15.97:0/3287628675 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84d019e8d0 con 0x7f84d0108ab0 2024-03-27T10:44:44.164 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:42.844+0000 7f84d5000700 1 -- 172.21.15.97:0/3287628675 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84d01a15c0 con 0x7f84d0108ab0 2024-03-27T10:44:44.164 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:42.845+0000 7f84c6a00700 1 -- 172.21.15.97:0/3287628675 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 12) ==== 46479+0+0 (secure 0 0 0) 0x7f84c0022020 con 0x7f84d0108ab0 2024-03-27T10:44:44.164 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:42.846+0000 7f84d5000700 1 -- 172.21.15.97:0/3287628675 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84d0063c70 con 0x7f84d0108ab0 2024-03-27T10:44:44.164 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:42.846+0000 7f84c6a00700 1 --2- 172.21.15.97:0/3287628675 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f84b4039420 0x7f84b403b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:44:44.164 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:42.846+0000 7f84c6a00700 1 -- 172.21.15.97:0/3287628675 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f84c004d9e0 con 0x7f84d0108ab0 2024-03-27T10:44:44.164 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:42.846+0000 7f84c7e00700 1 --2- 172.21.15.97:0/3287628675 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f84b4039420 0x7f84b403b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:44:44.164 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:42.847+0000 7f84c7e00700 1 --2- 172.21.15.97:0/3287628675 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f84b4039420 0x7f84b403b8e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f84b8006fd0 tx=0x7f84b8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:44:44.164 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:42.852+0000 7f84c6a00700 1 -- 172.21.15.97:0/3287628675 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f84c000b3e0 con 0x7f84d0108ab0 2024-03-27T10:44:44.165 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:43.104+0000 7f84d5000700 1 -- 172.21.15.97:0/3287628675 --> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f84d01036e0 con 0x7f84b4039420 2024-03-27T10:44:44.165 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:43.443+0000 7f84c6a00700 1 -- 172.21.15.97:0/3287628675 <== mgr.14152 v2:172.21.15.97:6800/3955463538 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f84d01036e0 con 0x7f84b4039420 2024-03-27T10:44:44.165 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:43.446+0000 7f84d5000700 1 -- 172.21.15.97:0/3287628675 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f84b4039420 msgr2=0x7f84b403b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:44:44.165 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:43.446+0000 7f84d5000700 1 --2- 172.21.15.97:0/3287628675 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f84b4039420 0x7f84b403b8e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f84b8006fd0 tx=0x7f84b8006e40 comp rx=0 tx=0).stop 2024-03-27T10:44:44.165 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:43.446+0000 7f84d5000700 1 -- 172.21.15.97:0/3287628675 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84d0108ab0 msgr2=0x7f84d019e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:44:44.165 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:43.446+0000 7f84d5000700 1 --2- 172.21.15.97:0/3287628675 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84d0108ab0 0x7f84d019e100 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f84c0011a00 tx=0x7f84c0011ae0 comp rx=0 tx=0).stop 2024-03-27T10:44:44.165 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:43.446+0000 7f84d5000700 1 -- 172.21.15.97:0/3287628675 shutdown_connections 2024-03-27T10:44:44.165 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:43.446+0000 7f84d5000700 1 --2- 172.21.15.97:0/3287628675 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f84b4039420 0x7f84b403b8e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:44:44.165 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:43.446+0000 7f84d5000700 1 --2- 172.21.15.97:0/3287628675 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84d0108ab0 0x7f84d019e100 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:44:44.165 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:43.446+0000 7f84d5000700 1 -- 172.21.15.97:0/3287628675 >> 172.21.15.97:0/3287628675 conn(0x7f84d01023e0 msgr2=0x7f84d0102fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:44:44.165 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:43.446+0000 7f84d5000700 1 -- 172.21.15.97:0/3287628675 shutdown_connections 2024-03-27T10:44:44.165 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:43.446+0000 7f84d5000700 1 -- 172.21.15.97:0/3287628675 wait complete. 2024-03-27T10:44:44.166 INFO:teuthology.orchestra.run.smithi097.stdout:Creating initial admin user... 2024-03-27T10:44:46.119 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:44:45 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:44:46.119 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:44:45 smithi097 ceph-mon[31859]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T10:44:46.120 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:44:45 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:44:46.135 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$pncAX5Tvx0lp31H1Q98HNOBLulGKb86FspSEamELyPbdJhhs6HPO.", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1711536285, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-03-27T10:44:46.135 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:44.845+0000 7f299d800700 1 Processor -- start 2024-03-27T10:44:46.135 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:44.845+0000 7f299d800700 1 -- start start 2024-03-27T10:44:46.135 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:44.845+0000 7f299d800700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f299807dea0 0x7f299807c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:44:46.135 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:44.846+0000 7f299d800700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f299807c8d0 con 0x7f299807dea0 2024-03-27T10:44:46.135 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:44.846+0000 7f2996a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f299807dea0 0x7f299807c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:44:46.135 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:44.846+0000 7f2996a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f299807dea0 0x7f299807c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60404/0 (socket says 172.21.15.97:60404) 2024-03-27T10:44:46.135 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:44.846+0000 7f2996a00700 1 -- 172.21.15.97:0/3460715663 learned_addr learned my addr 172.21.15.97:0/3460715663 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:44:46.135 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:44.846+0000 7f2996a00700 1 -- 172.21.15.97:0/3460715663 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f299807ca50 con 0x7f299807dea0 2024-03-27T10:44:46.136 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:44.847+0000 7f2996a00700 1 --2- 172.21.15.97:0/3460715663 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f299807dea0 0x7f299807c300 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f29880098a0 tx=0x7f2988009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7fccfea8d7ce9cb server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:44:46.136 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:44.847+0000 7f2995600700 1 -- 172.21.15.97:0/3460715663 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2988004160 con 0x7f299807dea0 2024-03-27T10:44:46.136 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:44.847+0000 7f2995600700 1 -- 172.21.15.97:0/3460715663 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2988004300 con 0x7f299807dea0 2024-03-27T10:44:46.136 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:44.847+0000 7f2995600700 1 -- 172.21.15.97:0/3460715663 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2988011a50 con 0x7f299807dea0 2024-03-27T10:44:46.136 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:44.848+0000 7f299d800700 1 -- 172.21.15.97:0/3460715663 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f299807dea0 msgr2=0x7f299807c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:44:46.136 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:44.848+0000 7f299d800700 1 --2- 172.21.15.97:0/3460715663 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f299807dea0 0x7f299807c300 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f29880098a0 tx=0x7f2988009bb0 comp rx=0 tx=0).stop 2024-03-27T10:44:46.136 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:44.848+0000 7f299d800700 1 -- 172.21.15.97:0/3460715663 shutdown_connections 2024-03-27T10:44:46.136 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:44.848+0000 7f299d800700 1 --2- 172.21.15.97:0/3460715663 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f299807dea0 0x7f299807c300 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:44:46.136 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:44.848+0000 7f299d800700 1 -- 172.21.15.97:0/3460715663 >> 172.21.15.97:0/3460715663 conn(0x7f29981034e0 msgr2=0x7f2998105960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:44:46.136 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:44.848+0000 7f299d800700 1 -- 172.21.15.97:0/3460715663 shutdown_connections 2024-03-27T10:44:46.136 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:44.848+0000 7f299d800700 1 -- 172.21.15.97:0/3460715663 wait complete. 2024-03-27T10:44:46.136 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:44.849+0000 7f299d800700 1 Processor -- start 2024-03-27T10:44:46.136 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:44.849+0000 7f299d800700 1 -- start start 2024-03-27T10:44:46.136 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:44.849+0000 7f299d800700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f299807dea0 0x7f299819e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:44:46.137 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:44.849+0000 7f299d800700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f299810e5a0 con 0x7f299807dea0 2024-03-27T10:44:46.137 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:44.849+0000 7f2996a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f299807dea0 0x7f299819e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:44:46.137 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:44.850+0000 7f2996a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f299807dea0 0x7f299819e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60406/0 (socket says 172.21.15.97:60406) 2024-03-27T10:44:46.137 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:44.850+0000 7f2996a00700 1 -- 172.21.15.97:0/643594553 learned_addr learned my addr 172.21.15.97:0/643594553 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:44:46.137 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:44.850+0000 7f2996a00700 1 -- 172.21.15.97:0/643594553 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f299819e600 con 0x7f299807dea0 2024-03-27T10:44:46.137 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:44.850+0000 7f2996a00700 1 --2- 172.21.15.97:0/643594553 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f299807dea0 0x7f299819e0c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2988011a00 tx=0x7f2988011ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:44:46.137 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:44.850+0000 7f298f400700 1 -- 172.21.15.97:0/643594553 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2988003af0 con 0x7f299807dea0 2024-03-27T10:44:46.137 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:44.851+0000 7f298f400700 1 -- 172.21.15.97:0/643594553 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f298801a8e0 con 0x7f299807dea0 2024-03-27T10:44:46.137 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:44.851+0000 7f299d800700 1 -- 172.21.15.97:0/643594553 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f299819e890 con 0x7f299807dea0 2024-03-27T10:44:46.137 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:44.851+0000 7f298f400700 1 -- 172.21.15.97:0/643594553 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2988024840 con 0x7f299807dea0 2024-03-27T10:44:46.137 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:44.851+0000 7f299d800700 1 -- 172.21.15.97:0/643594553 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29981a1580 con 0x7f299807dea0 2024-03-27T10:44:46.137 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:44.852+0000 7f298f400700 1 -- 172.21.15.97:0/643594553 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 12) ==== 46479+0+0 (secure 0 0 0) 0x7f2988022020 con 0x7f299807dea0 2024-03-27T10:44:46.137 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:44.852+0000 7f299d800700 1 -- 172.21.15.97:0/643594553 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2998063c70 con 0x7f299807dea0 2024-03-27T10:44:46.138 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:44.852+0000 7f298f400700 1 --2- 172.21.15.97:0/643594553 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f297c039480 0x7f297c03b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:44:46.138 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:44.852+0000 7f298f400700 1 -- 172.21.15.97:0/643594553 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f298804d900 con 0x7f299807dea0 2024-03-27T10:44:46.138 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:44.853+0000 7f2996000700 1 --2- 172.21.15.97:0/643594553 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f297c039480 0x7f297c03b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:44:46.138 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:44.853+0000 7f2996000700 1 --2- 172.21.15.97:0/643594553 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f297c039480 0x7f297c03b940 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2984006fd0 tx=0x7f2984006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:44:46.138 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:44.858+0000 7f298f400700 1 -- 172.21.15.97:0/643594553 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f298800b3e0 con 0x7f299807dea0 2024-03-27T10:44:46.138 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:45.114+0000 7f299d800700 1 -- 172.21.15.97:0/643594553 --> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f299819ed70 con 0x7f297c039480 2024-03-27T10:44:46.138 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:45.365+0000 7f298f400700 1 -- 172.21.15.97:0/643594553 <== mgr.14152 v2:172.21.15.97:6800/3955463538 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f299819ed70 con 0x7f297c039480 2024-03-27T10:44:46.138 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:45.368+0000 7f299d800700 1 -- 172.21.15.97:0/643594553 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f297c039480 msgr2=0x7f297c03b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:44:46.138 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:45.368+0000 7f299d800700 1 --2- 172.21.15.97:0/643594553 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f297c039480 0x7f297c03b940 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2984006fd0 tx=0x7f2984006e40 comp rx=0 tx=0).stop 2024-03-27T10:44:46.138 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:45.368+0000 7f299d800700 1 -- 172.21.15.97:0/643594553 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f299807dea0 msgr2=0x7f299819e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:44:46.138 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:45.368+0000 7f299d800700 1 --2- 172.21.15.97:0/643594553 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f299807dea0 0x7f299819e0c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2988011a00 tx=0x7f2988011ae0 comp rx=0 tx=0).stop 2024-03-27T10:44:46.138 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:45.368+0000 7f299d800700 1 -- 172.21.15.97:0/643594553 shutdown_connections 2024-03-27T10:44:46.139 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:45.368+0000 7f299d800700 1 --2- 172.21.15.97:0/643594553 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f297c039480 0x7f297c03b940 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:44:46.139 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:45.368+0000 7f299d800700 1 --2- 172.21.15.97:0/643594553 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f299807dea0 0x7f299819e0c0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:44:46.139 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:45.368+0000 7f299d800700 1 -- 172.21.15.97:0/643594553 >> 172.21.15.97:0/643594553 conn(0x7f29981034e0 msgr2=0x7f29981040b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:44:46.139 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:45.368+0000 7f299d800700 1 -- 172.21.15.97:0/643594553 shutdown_connections 2024-03-27T10:44:46.139 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:45.368+0000 7f299d800700 1 -- 172.21.15.97:0/643594553 wait complete. 2024-03-27T10:44:46.139 INFO:teuthology.orchestra.run.smithi097.stdout:Fetching dashboard port number... 2024-03-27T10:44:47.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:44:47 smithi097 ceph-mon[31859]: mgrmap e13: x(active, since 6s) 2024-03-27T10:44:47.644 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:44:47 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/679049829' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-03-27T10:44:48.465 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 8443 2024-03-27T10:44:48.465 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:47.031+0000 7fda66c00700 1 Processor -- start 2024-03-27T10:44:48.466 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:47.031+0000 7fda66c00700 1 -- start start 2024-03-27T10:44:48.466 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:47.031+0000 7fda66c00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda601062c0 0x7fda601066e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:44:48.466 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:47.031+0000 7fda66c00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fda60106cb0 con 0x7fda601062c0 2024-03-27T10:44:48.466 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:47.032+0000 7fda5fe00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda601062c0 0x7fda601066e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:44:48.467 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:47.032+0000 7fda5fe00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda601062c0 0x7fda601066e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60418/0 (socket says 172.21.15.97:60418) 2024-03-27T10:44:48.467 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:47.032+0000 7fda5fe00700 1 -- 172.21.15.97:0/3416406981 learned_addr learned my addr 172.21.15.97:0/3416406981 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:44:48.467 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:47.032+0000 7fda5fe00700 1 -- 172.21.15.97:0/3416406981 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda601092f0 con 0x7fda601062c0 2024-03-27T10:44:48.467 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:47.033+0000 7fda5fe00700 1 --2- 172.21.15.97:0/3416406981 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda601062c0 0x7fda601066e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fda500098a0 tx=0x7fda50009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13cd204530cf3a8e server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:44:48.467 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:47.033+0000 7fda5ea00700 1 -- 172.21.15.97:0/3416406981 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fda50004160 con 0x7fda601062c0 2024-03-27T10:44:48.467 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:47.033+0000 7fda5ea00700 1 -- 172.21.15.97:0/3416406981 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fda50004300 con 0x7fda601062c0 2024-03-27T10:44:48.467 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:47.033+0000 7fda5ea00700 1 -- 172.21.15.97:0/3416406981 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fda50011a50 con 0x7fda601062c0 2024-03-27T10:44:48.472 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:47.033+0000 7fda66c00700 1 -- 172.21.15.97:0/3416406981 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda601062c0 msgr2=0x7fda601066e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:44:48.472 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:47.033+0000 7fda66c00700 1 --2- 172.21.15.97:0/3416406981 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda601062c0 0x7fda601066e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fda500098a0 tx=0x7fda50009bb0 comp rx=0 tx=0).stop 2024-03-27T10:44:48.472 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:47.034+0000 7fda66c00700 1 -- 172.21.15.97:0/3416406981 shutdown_connections 2024-03-27T10:44:48.472 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:47.034+0000 7fda66c00700 1 --2- 172.21.15.97:0/3416406981 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda601062c0 0x7fda601066e0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:44:48.472 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:47.034+0000 7fda66c00700 1 -- 172.21.15.97:0/3416406981 >> 172.21.15.97:0/3416406981 conn(0x7fda60101c10 msgr2=0x7fda60104030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:44:48.472 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:47.034+0000 7fda66c00700 1 -- 172.21.15.97:0/3416406981 shutdown_connections 2024-03-27T10:44:48.472 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:47.034+0000 7fda66c00700 1 -- 172.21.15.97:0/3416406981 wait complete. 2024-03-27T10:44:48.472 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:47.035+0000 7fda66c00700 1 Processor -- start 2024-03-27T10:44:48.472 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:47.035+0000 7fda66c00700 1 -- start start 2024-03-27T10:44:48.472 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:47.035+0000 7fda66c00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda601062c0 0x7fda6019e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:44:48.473 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:47.035+0000 7fda66c00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fda6010c520 con 0x7fda601062c0 2024-03-27T10:44:48.473 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:47.035+0000 7fda5fe00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda601062c0 0x7fda6019e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:44:48.473 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:47.035+0000 7fda5fe00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda601062c0 0x7fda6019e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60432/0 (socket says 172.21.15.97:60432) 2024-03-27T10:44:48.473 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:47.035+0000 7fda5fe00700 1 -- 172.21.15.97:0/679049829 learned_addr learned my addr 172.21.15.97:0/679049829 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:44:48.473 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:47.036+0000 7fda5fe00700 1 -- 172.21.15.97:0/679049829 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda6019e600 con 0x7fda601062c0 2024-03-27T10:44:48.473 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:47.036+0000 7fda5fe00700 1 --2- 172.21.15.97:0/679049829 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda601062c0 0x7fda6019e0c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fda50006b20 tx=0x7fda50011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:44:48.473 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:47.036+0000 7fda5cc00700 1 -- 172.21.15.97:0/679049829 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fda500039e0 con 0x7fda601062c0 2024-03-27T10:44:48.473 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:47.036+0000 7fda5cc00700 1 -- 172.21.15.97:0/679049829 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fda5001a8e0 con 0x7fda601062c0 2024-03-27T10:44:48.473 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:47.036+0000 7fda66c00700 1 -- 172.21.15.97:0/679049829 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda6019e890 con 0x7fda601062c0 2024-03-27T10:44:48.473 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:47.036+0000 7fda5cc00700 1 -- 172.21.15.97:0/679049829 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fda500247e0 con 0x7fda601062c0 2024-03-27T10:44:48.473 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:47.036+0000 7fda66c00700 1 -- 172.21.15.97:0/679049829 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda601a1580 con 0x7fda601062c0 2024-03-27T10:44:48.473 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:47.037+0000 7fda5cc00700 1 -- 172.21.15.97:0/679049829 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) ==== 46522+0+0 (secure 0 0 0) 0x7fda50022020 con 0x7fda601062c0 2024-03-27T10:44:48.473 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:47.037+0000 7fda66c00700 1 -- 172.21.15.97:0/679049829 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda60063c70 con 0x7fda601062c0 2024-03-27T10:44:48.474 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:47.038+0000 7fda5cc00700 1 --2- 172.21.15.97:0/679049829 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fda44039510 0x7fda4403b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:44:48.474 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:47.038+0000 7fda5cc00700 1 -- 172.21.15.97:0/679049829 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fda5004da50 con 0x7fda601062c0 2024-03-27T10:44:48.474 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:47.038+0000 7fda5f400700 1 --2- 172.21.15.97:0/679049829 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fda44039510 0x7fda4403b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:44:48.474 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:47.039+0000 7fda5f400700 1 --2- 172.21.15.97:0/679049829 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fda44039510 0x7fda4403b9d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fda4c006fd0 tx=0x7fda4c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:44:48.474 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:47.044+0000 7fda5cc00700 1 -- 172.21.15.97:0/679049829 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7fda500212c0 con 0x7fda601062c0 2024-03-27T10:44:48.474 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:47.271+0000 7fda66c00700 1 -- 172.21.15.97:0/679049829 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7fda601a1b30 con 0x7fda601062c0 2024-03-27T10:44:48.474 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:47.272+0000 7fda5cc00700 1 -- 172.21.15.97:0/679049829 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7fda5002d3f0 con 0x7fda601062c0 2024-03-27T10:44:48.474 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:47.274+0000 7fda66c00700 1 -- 172.21.15.97:0/679049829 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fda44039510 msgr2=0x7fda4403b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:44:48.474 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:47.274+0000 7fda66c00700 1 --2- 172.21.15.97:0/679049829 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fda44039510 0x7fda4403b9d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fda4c006fd0 tx=0x7fda4c006e40 comp rx=0 tx=0).stop 2024-03-27T10:44:48.474 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:47.274+0000 7fda66c00700 1 -- 172.21.15.97:0/679049829 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda601062c0 msgr2=0x7fda6019e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:44:48.474 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:47.274+0000 7fda66c00700 1 --2- 172.21.15.97:0/679049829 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda601062c0 0x7fda6019e0c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fda50006b20 tx=0x7fda50011ad0 comp rx=0 tx=0).stop 2024-03-27T10:44:48.474 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:47.274+0000 7fda66c00700 1 -- 172.21.15.97:0/679049829 shutdown_connections 2024-03-27T10:44:48.474 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:47.274+0000 7fda66c00700 1 --2- 172.21.15.97:0/679049829 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fda44039510 0x7fda4403b9d0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:44:48.475 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:47.274+0000 7fda66c00700 1 --2- 172.21.15.97:0/679049829 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda601062c0 0x7fda6019e0c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:44:48.475 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:47.274+0000 7fda66c00700 1 -- 172.21.15.97:0/679049829 >> 172.21.15.97:0/679049829 conn(0x7fda60101c10 msgr2=0x7fda60102700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:44:48.475 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:47.275+0000 7fda66c00700 1 -- 172.21.15.97:0/679049829 shutdown_connections 2024-03-27T10:44:48.475 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:47.275+0000 7fda66c00700 1 -- 172.21.15.97:0/679049829 wait complete. 2024-03-27T10:44:48.501 INFO:teuthology.orchestra.run.smithi097.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-27T10:44:48.502 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stdout disabled 2024-03-27T10:44:48.523 INFO:teuthology.orchestra.run.smithi097.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-27T10:44:48.523 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stdout inactive 2024-03-27T10:44:48.523 INFO:teuthology.orchestra.run.smithi097.stdout:firewalld.service is not enabled 2024-03-27T10:44:48.524 INFO:teuthology.orchestra.run.smithi097.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-03-27T10:44:48.530 INFO:teuthology.orchestra.run.smithi097.stdout:Ceph Dashboard is now available at: 2024-03-27T10:44:48.530 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T10:44:48.530 INFO:teuthology.orchestra.run.smithi097.stdout: URL: https://smithi097.front.sepia.ceph.com:8443/ 2024-03-27T10:44:48.530 INFO:teuthology.orchestra.run.smithi097.stdout: User: admin 2024-03-27T10:44:48.531 INFO:teuthology.orchestra.run.smithi097.stdout: Password: qvdnwf5ukv 2024-03-27T10:44:48.531 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T10:44:48.532 INFO:teuthology.orchestra.run.smithi097.stdout:Saving cluster configuration to /var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/config directory 2024-03-27T10:44:48.533 INFO:teuthology.orchestra.run.smithi097.stdout:Enabling autotune for osd_memory_target 2024-03-27T10:44:50.040 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:49.191+0000 7f809de00700 1 Processor -- start 2024-03-27T10:44:50.040 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:49.191+0000 7f809de00700 1 -- start start 2024-03-27T10:44:50.040 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:49.192+0000 7f809de00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f809807de80 0x7f809807c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:44:50.040 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:49.192+0000 7f809de00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f809807c8b0 con 0x7f809807de80 2024-03-27T10:44:50.040 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:49.192+0000 7f8096a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f809807de80 0x7f809807c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:44:50.040 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:49.192+0000 7f8096a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f809807de80 0x7f809807c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35898/0 (socket says 172.21.15.97:35898) 2024-03-27T10:44:50.040 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:49.192+0000 7f8096a00700 1 -- 172.21.15.97:0/2314386707 learned_addr learned my addr 172.21.15.97:0/2314386707 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:44:50.041 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:49.193+0000 7f8096a00700 1 -- 172.21.15.97:0/2314386707 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f809807ca30 con 0x7f809807de80 2024-03-27T10:44:50.041 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:49.193+0000 7f8096a00700 1 --2- 172.21.15.97:0/2314386707 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f809807de80 0x7f809807c2e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f80880098a0 tx=0x7f8088009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24d86cb0662e23f7 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:44:50.041 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:49.193+0000 7f8095600700 1 -- 172.21.15.97:0/2314386707 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8088004160 con 0x7f809807de80 2024-03-27T10:44:50.041 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:49.194+0000 7f8095600700 1 -- 172.21.15.97:0/2314386707 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8088004300 con 0x7f809807de80 2024-03-27T10:44:50.041 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:49.194+0000 7f8095600700 1 -- 172.21.15.97:0/2314386707 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8088011a50 con 0x7f809807de80 2024-03-27T10:44:50.041 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:49.194+0000 7f809de00700 1 -- 172.21.15.97:0/2314386707 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f809807de80 msgr2=0x7f809807c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:44:50.041 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:49.194+0000 7f809de00700 1 --2- 172.21.15.97:0/2314386707 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f809807de80 0x7f809807c2e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f80880098a0 tx=0x7f8088009bb0 comp rx=0 tx=0).stop 2024-03-27T10:44:50.041 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:49.194+0000 7f809de00700 1 -- 172.21.15.97:0/2314386707 shutdown_connections 2024-03-27T10:44:50.041 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:49.194+0000 7f809de00700 1 --2- 172.21.15.97:0/2314386707 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f809807de80 0x7f809807c2e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:44:50.041 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:49.194+0000 7f809de00700 1 -- 172.21.15.97:0/2314386707 >> 172.21.15.97:0/2314386707 conn(0x7f8098103570 msgr2=0x7f80981059d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:44:50.041 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:49.194+0000 7f809de00700 1 -- 172.21.15.97:0/2314386707 shutdown_connections 2024-03-27T10:44:50.041 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:49.195+0000 7f809de00700 1 -- 172.21.15.97:0/2314386707 wait complete. 2024-03-27T10:44:50.041 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:49.195+0000 7f809de00700 1 Processor -- start 2024-03-27T10:44:50.042 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:49.196+0000 7f809de00700 1 -- start start 2024-03-27T10:44:50.042 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:49.196+0000 7f809de00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f809807de80 0x7f809819e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:44:50.042 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:49.196+0000 7f809de00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f809810e5d0 con 0x7f809807de80 2024-03-27T10:44:50.042 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:49.196+0000 7f8096a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f809807de80 0x7f809819e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:44:50.042 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:49.196+0000 7f8096a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f809807de80 0x7f809819e0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35906/0 (socket says 172.21.15.97:35906) 2024-03-27T10:44:50.042 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:49.196+0000 7f8096a00700 1 -- 172.21.15.97:0/246914284 learned_addr learned my addr 172.21.15.97:0/246914284 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:44:50.042 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:49.197+0000 7f8096a00700 1 -- 172.21.15.97:0/246914284 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f809819e630 con 0x7f809807de80 2024-03-27T10:44:50.042 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:49.197+0000 7f8096a00700 1 --2- 172.21.15.97:0/246914284 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f809807de80 0x7f809819e0f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8088011a00 tx=0x7f8088011ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:44:50.042 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:49.197+0000 7f809ca00700 1 -- 172.21.15.97:0/246914284 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8088003af0 con 0x7f809807de80 2024-03-27T10:44:50.042 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:49.197+0000 7f809ca00700 1 -- 172.21.15.97:0/246914284 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f808801a8e0 con 0x7f809807de80 2024-03-27T10:44:50.042 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:49.197+0000 7f809de00700 1 -- 172.21.15.97:0/246914284 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f809819e8c0 con 0x7f809807de80 2024-03-27T10:44:50.042 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:49.197+0000 7f809ca00700 1 -- 172.21.15.97:0/246914284 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8088024840 con 0x7f809807de80 2024-03-27T10:44:50.042 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:49.197+0000 7f809de00700 1 -- 172.21.15.97:0/246914284 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80981a15b0 con 0x7f809807de80 2024-03-27T10:44:50.043 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:49.199+0000 7f809de00700 1 -- 172.21.15.97:0/246914284 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8098063c70 con 0x7f809807de80 2024-03-27T10:44:50.043 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:49.199+0000 7f809ca00700 1 -- 172.21.15.97:0/246914284 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) ==== 46522+0+0 (secure 0 0 0) 0x7f8088022020 con 0x7f809807de80 2024-03-27T10:44:50.043 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:49.199+0000 7f809ca00700 1 --2- 172.21.15.97:0/246914284 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f807c039520 0x7f807c03b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:44:50.043 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:49.199+0000 7f809ca00700 1 -- 172.21.15.97:0/246914284 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f808804da10 con 0x7f809807de80 2024-03-27T10:44:50.043 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:49.199+0000 7f8096000700 1 --2- 172.21.15.97:0/246914284 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f807c039520 0x7f807c03b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:44:50.043 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:49.200+0000 7f8096000700 1 --2- 172.21.15.97:0/246914284 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f807c039520 0x7f807c03b9e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8084006fd0 tx=0x7f8084006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:44:50.043 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:49.204+0000 7f809ca00700 1 -- 172.21.15.97:0/246914284 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f808800b3e0 con 0x7f809807de80 2024-03-27T10:44:50.043 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:49.426+0000 7f809de00700 1 -- 172.21.15.97:0/246914284 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7f80981a1bd0 con 0x7f809807de80 2024-03-27T10:44:50.043 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:49.427+0000 7f809ca00700 1 -- 172.21.15.97:0/246914284 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7f8088016260 con 0x7f809807de80 2024-03-27T10:44:50.043 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:49.429+0000 7f809de00700 1 -- 172.21.15.97:0/246914284 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f807c039520 msgr2=0x7f807c03b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:44:50.043 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:49.429+0000 7f809de00700 1 --2- 172.21.15.97:0/246914284 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f807c039520 0x7f807c03b9e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8084006fd0 tx=0x7f8084006e40 comp rx=0 tx=0).stop 2024-03-27T10:44:50.043 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:49.430+0000 7f809de00700 1 -- 172.21.15.97:0/246914284 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f809807de80 msgr2=0x7f809819e0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:44:50.043 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:49.430+0000 7f809de00700 1 --2- 172.21.15.97:0/246914284 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f809807de80 0x7f809819e0f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8088011a00 tx=0x7f8088011ae0 comp rx=0 tx=0).stop 2024-03-27T10:44:50.044 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:49.430+0000 7f809de00700 1 -- 172.21.15.97:0/246914284 shutdown_connections 2024-03-27T10:44:50.044 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:49.430+0000 7f809de00700 1 --2- 172.21.15.97:0/246914284 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f807c039520 0x7f807c03b9e0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:44:50.044 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:49.430+0000 7f809de00700 1 --2- 172.21.15.97:0/246914284 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f809807de80 0x7f809819e0f0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:44:50.044 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:49.430+0000 7f809de00700 1 -- 172.21.15.97:0/246914284 >> 172.21.15.97:0/246914284 conn(0x7f8098103570 msgr2=0x7f8098104190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:44:50.044 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:49.430+0000 7f809de00700 1 -- 172.21.15.97:0/246914284 shutdown_connections 2024-03-27T10:44:50.044 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:49.430+0000 7f809de00700 1 -- 172.21.15.97:0/246914284 wait complete. 2024-03-27T10:44:51.636 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:50.648+0000 7fe34c600700 1 Processor -- start 2024-03-27T10:44:51.636 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:50.649+0000 7fe34c600700 1 -- start start 2024-03-27T10:44:51.636 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:50.649+0000 7fe34c600700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe34410a210 0x7fe34410a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:44:51.636 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:50.649+0000 7fe34c600700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe34410ac00 con 0x7fe34410a210 2024-03-27T10:44:51.636 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:50.649+0000 7fe349c00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe34410a210 0x7fe34410a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:44:51.636 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:50.649+0000 7fe349c00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe34410a210 0x7fe34410a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35912/0 (socket says 172.21.15.97:35912) 2024-03-27T10:44:51.636 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:50.649+0000 7fe349c00700 1 -- 172.21.15.97:0/1036687903 learned_addr learned my addr 172.21.15.97:0/1036687903 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:44:51.636 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:50.650+0000 7fe349c00700 1 -- 172.21.15.97:0/1036687903 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe34410b3c0 con 0x7fe34410a210 2024-03-27T10:44:51.636 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:50.650+0000 7fe349c00700 1 --2- 172.21.15.97:0/1036687903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe34410a210 0x7fe34410a630 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe3380098a0 tx=0x7fe338009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13c83c07ae99869c server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:44:51.636 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:50.650+0000 7fe343e00700 1 -- 172.21.15.97:0/1036687903 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe338004160 con 0x7fe34410a210 2024-03-27T10:44:51.637 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:50.651+0000 7fe343e00700 1 -- 172.21.15.97:0/1036687903 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe338004300 con 0x7fe34410a210 2024-03-27T10:44:51.637 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:50.651+0000 7fe343e00700 1 -- 172.21.15.97:0/1036687903 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe338011a50 con 0x7fe34410a210 2024-03-27T10:44:51.637 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:50.651+0000 7fe34c600700 1 -- 172.21.15.97:0/1036687903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe34410a210 msgr2=0x7fe34410a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:44:51.637 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:50.651+0000 7fe34c600700 1 --2- 172.21.15.97:0/1036687903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe34410a210 0x7fe34410a630 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe3380098a0 tx=0x7fe338009bb0 comp rx=0 tx=0).stop 2024-03-27T10:44:51.637 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:50.651+0000 7fe34c600700 1 -- 172.21.15.97:0/1036687903 shutdown_connections 2024-03-27T10:44:51.637 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:50.651+0000 7fe34c600700 1 --2- 172.21.15.97:0/1036687903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe34410a210 0x7fe34410a630 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:44:51.637 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:50.651+0000 7fe34c600700 1 -- 172.21.15.97:0/1036687903 >> 172.21.15.97:0/1036687903 conn(0x7fe34407d590 msgr2=0x7fe34407d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:44:51.637 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:50.651+0000 7fe34c600700 1 -- 172.21.15.97:0/1036687903 shutdown_connections 2024-03-27T10:44:51.637 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:50.652+0000 7fe34c600700 1 -- 172.21.15.97:0/1036687903 wait complete. 2024-03-27T10:44:51.637 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:50.652+0000 7fe34c600700 1 Processor -- start 2024-03-27T10:44:51.638 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:50.653+0000 7fe34c600700 1 -- start start 2024-03-27T10:44:51.638 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:50.653+0000 7fe34c600700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe34410a210 0x7fe34419e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:44:51.638 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:50.653+0000 7fe34c600700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe34410e770 con 0x7fe34410a210 2024-03-27T10:44:51.638 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:50.653+0000 7fe349c00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe34410a210 0x7fe34419e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:44:51.638 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:50.653+0000 7fe349c00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe34410a210 0x7fe34419e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35924/0 (socket says 172.21.15.97:35924) 2024-03-27T10:44:51.638 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:50.653+0000 7fe349c00700 1 -- 172.21.15.97:0/3640397932 learned_addr learned my addr 172.21.15.97:0/3640397932 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:44:51.638 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:50.654+0000 7fe349c00700 1 -- 172.21.15.97:0/3640397932 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe34419e6b0 con 0x7fe34410a210 2024-03-27T10:44:51.638 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:50.654+0000 7fe349c00700 1 --2- 172.21.15.97:0/3640397932 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe34410a210 0x7fe34419e170 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe338011a20 tx=0x7fe338011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:44:51.638 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:50.654+0000 7fe342000700 1 -- 172.21.15.97:0/3640397932 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe338003af0 con 0x7fe34410a210 2024-03-27T10:44:51.638 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:50.654+0000 7fe342000700 1 -- 172.21.15.97:0/3640397932 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe33801a8e0 con 0x7fe34410a210 2024-03-27T10:44:51.638 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:50.654+0000 7fe34c600700 1 -- 172.21.15.97:0/3640397932 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe34419e940 con 0x7fe34410a210 2024-03-27T10:44:51.638 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:50.654+0000 7fe342000700 1 -- 172.21.15.97:0/3640397932 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe338024860 con 0x7fe34410a210 2024-03-27T10:44:51.638 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:50.654+0000 7fe34c600700 1 -- 172.21.15.97:0/3640397932 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3441a1630 con 0x7fe34410a210 2024-03-27T10:44:51.639 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:50.656+0000 7fe342000700 1 -- 172.21.15.97:0/3640397932 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) ==== 46522+0+0 (secure 0 0 0) 0x7fe338022020 con 0x7fe34410a210 2024-03-27T10:44:51.639 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:50.656+0000 7fe34c600700 1 -- 172.21.15.97:0/3640397932 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe344063c70 con 0x7fe34410a210 2024-03-27T10:44:51.639 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:50.656+0000 7fe342000700 1 --2- 172.21.15.97:0/3640397932 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fe32c039570 0x7fe32c03ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:44:51.639 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:50.656+0000 7fe342000700 1 -- 172.21.15.97:0/3640397932 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe33804d9b0 con 0x7fe34410a210 2024-03-27T10:44:51.639 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:50.656+0000 7fe349200700 1 --2- 172.21.15.97:0/3640397932 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fe32c039570 0x7fe32c03ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:44:51.639 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:50.657+0000 7fe349200700 1 --2- 172.21.15.97:0/3640397932 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fe32c039570 0x7fe32c03ba30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe330006fd0 tx=0x7fe330006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:44:51.639 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:50.662+0000 7fe342000700 1 -- 172.21.15.97:0/3640397932 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7fe3380094f0 con 0x7fe34410a210 2024-03-27T10:44:51.639 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:51.002+0000 7fe34c600700 1 -- 172.21.15.97:0/3640397932 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7fe3441a1be0 con 0x7fe34410a210 2024-03-27T10:44:51.639 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:51.007+0000 7fe342000700 1 -- 172.21.15.97:0/3640397932 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7fe33802ad60 con 0x7fe34410a210 2024-03-27T10:44:51.639 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:51.009+0000 7fe34c600700 1 -- 172.21.15.97:0/3640397932 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fe32c039570 msgr2=0x7fe32c03ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:44:51.639 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:51.010+0000 7fe34c600700 1 --2- 172.21.15.97:0/3640397932 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fe32c039570 0x7fe32c03ba30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe330006fd0 tx=0x7fe330006e40 comp rx=0 tx=0).stop 2024-03-27T10:44:51.639 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:51.010+0000 7fe34c600700 1 -- 172.21.15.97:0/3640397932 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe34410a210 msgr2=0x7fe34419e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:44:51.640 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:51.010+0000 7fe34c600700 1 --2- 172.21.15.97:0/3640397932 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe34410a210 0x7fe34419e170 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe338011a20 tx=0x7fe338011b00 comp rx=0 tx=0).stop 2024-03-27T10:44:51.640 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:51.010+0000 7fe34c600700 1 -- 172.21.15.97:0/3640397932 shutdown_connections 2024-03-27T10:44:51.640 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:51.010+0000 7fe34c600700 1 --2- 172.21.15.97:0/3640397932 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fe32c039570 0x7fe32c03ba30 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:44:51.640 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:51.010+0000 7fe34c600700 1 --2- 172.21.15.97:0/3640397932 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe34410a210 0x7fe34419e170 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:44:51.640 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:51.010+0000 7fe34c600700 1 -- 172.21.15.97:0/3640397932 >> 172.21.15.97:0/3640397932 conn(0x7fe34407d590 msgr2=0x7fe344107540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:44:51.640 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:51.010+0000 7fe34c600700 1 -- 172.21.15.97:0/3640397932 shutdown_connections 2024-03-27T10:44:51.640 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-03-27T10:44:51.010+0000 7fe34c600700 1 -- 172.21.15.97:0/3640397932 wait complete. 2024-03-27T10:44:51.640 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-03-27T10:44:51.640 INFO:teuthology.orchestra.run.smithi097.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-03-27T10:44:51.640 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T10:44:51.640 INFO:teuthology.orchestra.run.smithi097.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid adca88c4-ec26-11ee-95d0-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-03-27T10:44:51.640 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T10:44:51.640 INFO:teuthology.orchestra.run.smithi097.stdout:Or, if you are only running a single cluster on this host: 2024-03-27T10:44:51.640 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T10:44:51.641 INFO:teuthology.orchestra.run.smithi097.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-03-27T10:44:51.641 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T10:44:51.641 INFO:teuthology.orchestra.run.smithi097.stdout:Please consider enabling telemetry to help improve Ceph: 2024-03-27T10:44:51.641 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T10:44:51.641 INFO:teuthology.orchestra.run.smithi097.stdout: ceph telemetry on 2024-03-27T10:44:51.641 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T10:44:51.641 INFO:teuthology.orchestra.run.smithi097.stdout:For more information see: 2024-03-27T10:44:51.641 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T10:44:51.641 INFO:teuthology.orchestra.run.smithi097.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-03-27T10:44:51.641 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T10:44:51.641 INFO:teuthology.orchestra.run.smithi097.stdout:Bootstrap complete. 2024-03-27T10:44:51.736 INFO:tasks.cephadm:Fetching config... 2024-03-27T10:44:51.737 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-27T10:44:51.737 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-03-27T10:44:51.825 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-03-27T10:44:51.825 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-27T10:44:51.825 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-03-27T10:44:51.908 INFO:tasks.cephadm:Fetching mon keyring... 2024-03-27T10:44:51.908 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-27T10:44:51.908 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/mon.a/keyring of=/dev/stdout 2024-03-27T10:44:52.048 INFO:tasks.cephadm:Fetching pub ssh key... 2024-03-27T10:44:52.048 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-27T10:44:52.048 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-03-27T10:44:52.102 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-03-27T10:44:52.102 DEBUG:teuthology.orchestra.run.smithi097:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDQJYqv6OO4gJnK9WOA0M0v5I/aOCTRH2Sh6uGB+54a7Ds9qqL4xOVqE15zfpX4CHHV8nlNBlbqj4eGPmGG4HwEi07gGZHiYjUzKKJqnHndejQ5uU7cbQfwiT73MYcVm/lqYgT771kP7wf72m3V8787nTAvKlp0bw91jVA7/M0dki3a+ARyJlzV9S0dEv8+M9NY+f9Pg4PSiUpED7qCP7mIpaK0IGElgKVbtSR1TzfuaUKJJLo7gxS+74AVNfx6UIv3IVKz/WYkr+/3lpl3CLT/Yv7/kcsFKhXQW3+6Xt9nfwVTb1qfxoQUeP8XmIiOUPUYKu8kQxBg2xlj/iCUV1BXYgy6loL8QMwgrEyK96MKV8Yld3y8xoulb8JQOysAGVw9oVa2Yi3ar27mBNgR4tvFq2h82aUBHjDq7t4kyAwA2tHN3Xxzemyt7G8e3mZPM3d8f1CFBNqHuKXJ2+E9oTKv22ZZlb9XAJiww4F9RvLmrUvsmyQ6AjV7smRENTpicS8= ceph-adca88c4-ec26-11ee-95d0-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-27T10:44:52.199 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:44:52 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/3640397932' entity='client.admin' 2024-03-27T10:44:52.206 INFO:teuthology.orchestra.run.smithi097.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDQJYqv6OO4gJnK9WOA0M0v5I/aOCTRH2Sh6uGB+54a7Ds9qqL4xOVqE15zfpX4CHHV8nlNBlbqj4eGPmGG4HwEi07gGZHiYjUzKKJqnHndejQ5uU7cbQfwiT73MYcVm/lqYgT771kP7wf72m3V8787nTAvKlp0bw91jVA7/M0dki3a+ARyJlzV9S0dEv8+M9NY+f9Pg4PSiUpED7qCP7mIpaK0IGElgKVbtSR1TzfuaUKJJLo7gxS+74AVNfx6UIv3IVKz/WYkr+/3lpl3CLT/Yv7/kcsFKhXQW3+6Xt9nfwVTb1qfxoQUeP8XmIiOUPUYKu8kQxBg2xlj/iCUV1BXYgy6loL8QMwgrEyK96MKV8Yld3y8xoulb8JQOysAGVw9oVa2Yi3ar27mBNgR4tvFq2h82aUBHjDq7t4kyAwA2tHN3Xxzemyt7G8e3mZPM3d8f1CFBNqHuKXJ2+E9oTKv22ZZlb9XAJiww4F9RvLmrUvsmyQ6AjV7smRENTpicS8= ceph-adca88c4-ec26-11ee-95d0-87774f69a715 2024-03-27T10:44:52.267 DEBUG:teuthology.orchestra.run.smithi114:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDQJYqv6OO4gJnK9WOA0M0v5I/aOCTRH2Sh6uGB+54a7Ds9qqL4xOVqE15zfpX4CHHV8nlNBlbqj4eGPmGG4HwEi07gGZHiYjUzKKJqnHndejQ5uU7cbQfwiT73MYcVm/lqYgT771kP7wf72m3V8787nTAvKlp0bw91jVA7/M0dki3a+ARyJlzV9S0dEv8+M9NY+f9Pg4PSiUpED7qCP7mIpaK0IGElgKVbtSR1TzfuaUKJJLo7gxS+74AVNfx6UIv3IVKz/WYkr+/3lpl3CLT/Yv7/kcsFKhXQW3+6Xt9nfwVTb1qfxoQUeP8XmIiOUPUYKu8kQxBg2xlj/iCUV1BXYgy6loL8QMwgrEyK96MKV8Yld3y8xoulb8JQOysAGVw9oVa2Yi3ar27mBNgR4tvFq2h82aUBHjDq7t4kyAwA2tHN3Xxzemyt7G8e3mZPM3d8f1CFBNqHuKXJ2+E9oTKv22ZZlb9XAJiww4F9RvLmrUvsmyQ6AjV7smRENTpicS8= ceph-adca88c4-ec26-11ee-95d0-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-27T10:44:52.406 INFO:teuthology.orchestra.run.smithi114.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDQJYqv6OO4gJnK9WOA0M0v5I/aOCTRH2Sh6uGB+54a7Ds9qqL4xOVqE15zfpX4CHHV8nlNBlbqj4eGPmGG4HwEi07gGZHiYjUzKKJqnHndejQ5uU7cbQfwiT73MYcVm/lqYgT771kP7wf72m3V8787nTAvKlp0bw91jVA7/M0dki3a+ARyJlzV9S0dEv8+M9NY+f9Pg4PSiUpED7qCP7mIpaK0IGElgKVbtSR1TzfuaUKJJLo7gxS+74AVNfx6UIv3IVKz/WYkr+/3lpl3CLT/Yv7/kcsFKhXQW3+6Xt9nfwVTb1qfxoQUeP8XmIiOUPUYKu8kQxBg2xlj/iCUV1BXYgy6loL8QMwgrEyK96MKV8Yld3y8xoulb8JQOysAGVw9oVa2Yi3ar27mBNgR4tvFq2h82aUBHjDq7t4kyAwA2tHN3Xxzemyt7G8e3mZPM3d8f1CFBNqHuKXJ2+E9oTKv22ZZlb9XAJiww4F9RvLmrUvsmyQ6AjV7smRENTpicS8= ceph-adca88c4-ec26-11ee-95d0-87774f69a715 2024-03-27T10:44:52.471 DEBUG:teuthology.orchestra.run.smithi177:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDQJYqv6OO4gJnK9WOA0M0v5I/aOCTRH2Sh6uGB+54a7Ds9qqL4xOVqE15zfpX4CHHV8nlNBlbqj4eGPmGG4HwEi07gGZHiYjUzKKJqnHndejQ5uU7cbQfwiT73MYcVm/lqYgT771kP7wf72m3V8787nTAvKlp0bw91jVA7/M0dki3a+ARyJlzV9S0dEv8+M9NY+f9Pg4PSiUpED7qCP7mIpaK0IGElgKVbtSR1TzfuaUKJJLo7gxS+74AVNfx6UIv3IVKz/WYkr+/3lpl3CLT/Yv7/kcsFKhXQW3+6Xt9nfwVTb1qfxoQUeP8XmIiOUPUYKu8kQxBg2xlj/iCUV1BXYgy6loL8QMwgrEyK96MKV8Yld3y8xoulb8JQOysAGVw9oVa2Yi3ar27mBNgR4tvFq2h82aUBHjDq7t4kyAwA2tHN3Xxzemyt7G8e3mZPM3d8f1CFBNqHuKXJ2+E9oTKv22ZZlb9XAJiww4F9RvLmrUvsmyQ6AjV7smRENTpicS8= ceph-adca88c4-ec26-11ee-95d0-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-27T10:44:52.607 INFO:teuthology.orchestra.run.smithi177.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDQJYqv6OO4gJnK9WOA0M0v5I/aOCTRH2Sh6uGB+54a7Ds9qqL4xOVqE15zfpX4CHHV8nlNBlbqj4eGPmGG4HwEi07gGZHiYjUzKKJqnHndejQ5uU7cbQfwiT73MYcVm/lqYgT771kP7wf72m3V8787nTAvKlp0bw91jVA7/M0dki3a+ARyJlzV9S0dEv8+M9NY+f9Pg4PSiUpED7qCP7mIpaK0IGElgKVbtSR1TzfuaUKJJLo7gxS+74AVNfx6UIv3IVKz/WYkr+/3lpl3CLT/Yv7/kcsFKhXQW3+6Xt9nfwVTb1qfxoQUeP8XmIiOUPUYKu8kQxBg2xlj/iCUV1BXYgy6loL8QMwgrEyK96MKV8Yld3y8xoulb8JQOysAGVw9oVa2Yi3ar27mBNgR4tvFq2h82aUBHjDq7t4kyAwA2tHN3Xxzemyt7G8e3mZPM3d8f1CFBNqHuKXJ2+E9oTKv22ZZlb9XAJiww4F9RvLmrUvsmyQ6AjV7smRENTpicS8= ceph-adca88c4-ec26-11ee-95d0-87774f69a715 2024-03-27T10:44:52.669 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid adca88c4-ec26-11ee-95d0-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-03-27T10:44:53.090 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T10:44:55.840 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:44:55 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:44:55.840 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:44:55 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:44:55.840 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:44:55 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi097", "name": "osd_memory_target"}]: dispatch 2024-03-27T10:44:55.840 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:44:55 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:44:55.840 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:44:55 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:44:55.840 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:44:55 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:44:55.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:44:55.913+0000 7f3e4e800700 1 -- 172.21.15.97:0/2962609667 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e481008f0 msgr2=0x7f3e48100d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:44:55.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:44:55.913+0000 7f3e4e800700 1 --2- 172.21.15.97:0/2962609667 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e481008f0 0x7f3e48100d10 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3e38009910 tx=0x7f3e38009c20 comp rx=0 tx=0).stop 2024-03-27T10:44:55.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:44:55.913+0000 7f3e4e800700 1 -- 172.21.15.97:0/2962609667 shutdown_connections 2024-03-27T10:44:55.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:44:55.913+0000 7f3e4e800700 1 --2- 172.21.15.97:0/2962609667 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e481008f0 0x7f3e48100d10 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:44:55.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:44:55.913+0000 7f3e4e800700 1 -- 172.21.15.97:0/2962609667 >> 172.21.15.97:0/2962609667 conn(0x7f3e480fc510 msgr2=0x7f3e480fe950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:44:55.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:44:55.913+0000 7f3e4e800700 1 -- 172.21.15.97:0/2962609667 shutdown_connections 2024-03-27T10:44:55.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:44:55.913+0000 7f3e4e800700 1 -- 172.21.15.97:0/2962609667 wait complete. 2024-03-27T10:44:55.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:44:55.915+0000 7f3e4e800700 1 Processor -- start 2024-03-27T10:44:55.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:44:55.915+0000 7f3e4e800700 1 -- start start 2024-03-27T10:44:55.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:44:55.915+0000 7f3e4e800700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e481008f0 0x7f3e481998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:44:55.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:44:55.915+0000 7f3e4e800700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3e4810a1a0 con 0x7f3e481008f0 2024-03-27T10:44:55.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:44:55.916+0000 7f3e47400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e481008f0 0x7f3e481998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:44:55.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:44:55.916+0000 7f3e47400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e481008f0 0x7f3e481998a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35934/0 (socket says 172.21.15.97:35934) 2024-03-27T10:44:55.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:44:55.916+0000 7f3e47400700 1 -- 172.21.15.97:0/1942165383 learned_addr learned my addr 172.21.15.97:0/1942165383 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:44:55.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:44:55.916+0000 7f3e47400700 1 -- 172.21.15.97:0/1942165383 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e48199de0 con 0x7f3e481008f0 2024-03-27T10:44:55.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:44:55.916+0000 7f3e47400700 1 --2- 172.21.15.97:0/1942165383 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e481008f0 0x7f3e481998a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3e38005b40 tx=0x7f3e38004f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:44:55.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:44:55.917+0000 7f3e3fe00700 1 -- 172.21.15.97:0/1942165383 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3e38027070 con 0x7f3e481008f0 2024-03-27T10:44:55.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:44:55.917+0000 7f3e3fe00700 1 -- 172.21.15.97:0/1942165383 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e3800be10 con 0x7f3e481008f0 2024-03-27T10:44:55.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:44:55.917+0000 7f3e3fe00700 1 -- 172.21.15.97:0/1942165383 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3e38022de0 con 0x7f3e481008f0 2024-03-27T10:44:55.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:44:55.917+0000 7f3e4e800700 1 -- 172.21.15.97:0/1942165383 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e4819a070 con 0x7f3e481008f0 2024-03-27T10:44:55.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:44:55.917+0000 7f3e4e800700 1 -- 172.21.15.97:0/1942165383 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e4819cd60 con 0x7f3e481008f0 2024-03-27T10:44:55.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:44:55.918+0000 7f3e4e800700 1 -- 172.21.15.97:0/1942165383 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e4810a920 con 0x7f3e481008f0 2024-03-27T10:44:55.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:44:55.918+0000 7f3e3fe00700 1 -- 172.21.15.97:0/1942165383 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) ==== 46522+0+0 (secure 0 0 0) 0x7f3e3802a070 con 0x7f3e481008f0 2024-03-27T10:44:55.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:44:55.919+0000 7f3e3fe00700 1 --2- 172.21.15.97:0/1942165383 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f3e2c0394d0 0x7f3e2c03b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:44:55.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:44:55.919+0000 7f3e3fe00700 1 -- 172.21.15.97:0/1942165383 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f3e3801e030 con 0x7f3e481008f0 2024-03-27T10:44:55.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:44:55.919+0000 7f3e46a00700 1 --2- 172.21.15.97:0/1942165383 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f3e2c0394d0 0x7f3e2c03b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:44:55.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:44:55.920+0000 7f3e46a00700 1 --2- 172.21.15.97:0/1942165383 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f3e2c0394d0 0x7f3e2c03b990 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3e34006fd0 tx=0x7f3e34006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:44:55.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:44:55.925+0000 7f3e3fe00700 1 -- 172.21.15.97:0/1942165383 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f3e38013070 con 0x7f3e481008f0 2024-03-27T10:44:56.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:44:56.151+0000 7f3e4e800700 1 -- 172.21.15.97:0/1942165383 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f3e4802de40 con 0x7f3e481008f0 2024-03-27T10:44:56.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:44:56.160+0000 7f3e3fe00700 1 -- 172.21.15.97:0/1942165383 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f3e380310c0 con 0x7f3e481008f0 2024-03-27T10:44:56.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:44:56.163+0000 7f3e4e800700 1 -- 172.21.15.97:0/1942165383 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f3e2c0394d0 msgr2=0x7f3e2c03b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:44:56.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:44:56.163+0000 7f3e4e800700 1 --2- 172.21.15.97:0/1942165383 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f3e2c0394d0 0x7f3e2c03b990 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3e34006fd0 tx=0x7f3e34006e40 comp rx=0 tx=0).stop 2024-03-27T10:44:56.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:44:56.163+0000 7f3e4e800700 1 -- 172.21.15.97:0/1942165383 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e481008f0 msgr2=0x7f3e481998a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:44:56.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:44:56.163+0000 7f3e4e800700 1 --2- 172.21.15.97:0/1942165383 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e481008f0 0x7f3e481998a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3e38005b40 tx=0x7f3e38004f80 comp rx=0 tx=0).stop 2024-03-27T10:44:56.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:44:56.163+0000 7f3e4e800700 1 -- 172.21.15.97:0/1942165383 shutdown_connections 2024-03-27T10:44:56.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:44:56.163+0000 7f3e4e800700 1 --2- 172.21.15.97:0/1942165383 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f3e2c0394d0 0x7f3e2c03b990 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:44:56.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:44:56.163+0000 7f3e4e800700 1 --2- 172.21.15.97:0/1942165383 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e481008f0 0x7f3e481998a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:44:56.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:44:56.163+0000 7f3e4e800700 1 -- 172.21.15.97:0/1942165383 >> 172.21.15.97:0/1942165383 conn(0x7f3e480fc510 msgr2=0x7f3e480fd060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:44:56.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:44:56.164+0000 7f3e4e800700 1 -- 172.21.15.97:0/1942165383 shutdown_connections 2024-03-27T10:44:56.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:44:56.164+0000 7f3e4e800700 1 -- 172.21.15.97:0/1942165383 wait complete. 2024-03-27T10:44:56.977 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-03-27T10:44:56.977 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid adca88c4-ec26-11ee-95d0-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-03-27T10:44:57.394 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:44:57 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/1942165383' entity='client.admin' 2024-03-27T10:44:57.394 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:44:57 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:44:57.394 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:44:57 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:44:57.917 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T10:44:59.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:44:59.882+0000 7f7793200700 1 -- 172.21.15.97:0/995938301 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f778c102090 msgr2=0x7f778c1024b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:44:59.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:44:59.882+0000 7f7793200700 1 --2- 172.21.15.97:0/995938301 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f778c102090 0x7f778c1024b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f777c009910 tx=0x7f777c009c20 comp rx=0 tx=0).stop 2024-03-27T10:44:59.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:44:59.882+0000 7f7793200700 1 -- 172.21.15.97:0/995938301 shutdown_connections 2024-03-27T10:44:59.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:44:59.882+0000 7f7793200700 1 --2- 172.21.15.97:0/995938301 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f778c102090 0x7f778c1024b0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:44:59.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:44:59.882+0000 7f7793200700 1 -- 172.21.15.97:0/995938301 >> 172.21.15.97:0/995938301 conn(0x7f778c0fd630 msgr2=0x7f778c0ffa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:44:59.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:44:59.882+0000 7f7793200700 1 -- 172.21.15.97:0/995938301 shutdown_connections 2024-03-27T10:44:59.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:44:59.882+0000 7f7793200700 1 -- 172.21.15.97:0/995938301 wait complete. 2024-03-27T10:44:59.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:44:59.883+0000 7f7793200700 1 Processor -- start 2024-03-27T10:44:59.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:44:59.883+0000 7f7793200700 1 -- start start 2024-03-27T10:44:59.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:44:59.884+0000 7f7793200700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f778c102090 0x7f778c1976c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:44:59.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:44:59.884+0000 7f7793200700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f778c107fd0 con 0x7f778c102090 2024-03-27T10:44:59.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:44:59.884+0000 7f778be00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f778c102090 0x7f778c1976c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:44:59.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:44:59.884+0000 7f778be00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f778c102090 0x7f778c1976c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38032/0 (socket says 172.21.15.97:38032) 2024-03-27T10:44:59.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:44:59.884+0000 7f778be00700 1 -- 172.21.15.97:0/600966977 learned_addr learned my addr 172.21.15.97:0/600966977 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:44:59.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:44:59.884+0000 7f778be00700 1 -- 172.21.15.97:0/600966977 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f778c197c00 con 0x7f778c102090 2024-03-27T10:44:59.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:44:59.885+0000 7f778be00700 1 --2- 172.21.15.97:0/600966977 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f778c102090 0x7f778c1976c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f777c005950 tx=0x7f777c004fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:44:59.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:44:59.885+0000 7f7788c00700 1 -- 172.21.15.97:0/600966977 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f777c026070 con 0x7f778c102090 2024-03-27T10:44:59.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:44:59.885+0000 7f7788c00700 1 -- 172.21.15.97:0/600966977 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f777c00bc00 con 0x7f778c102090 2024-03-27T10:44:59.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:44:59.885+0000 7f7788c00700 1 -- 172.21.15.97:0/600966977 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f777c0039a0 con 0x7f778c102090 2024-03-27T10:44:59.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:44:59.885+0000 7f7793200700 1 -- 172.21.15.97:0/600966977 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f778c197e90 con 0x7f778c102090 2024-03-27T10:44:59.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:44:59.885+0000 7f7793200700 1 -- 172.21.15.97:0/600966977 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f778c19ab80 con 0x7f778c102090 2024-03-27T10:44:59.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:44:59.886+0000 7f7793200700 1 -- 172.21.15.97:0/600966977 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f778c108750 con 0x7f778c102090 2024-03-27T10:44:59.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:44:59.887+0000 7f7788c00700 1 -- 172.21.15.97:0/600966977 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) ==== 46522+0+0 (secure 0 0 0) 0x7f777c029070 con 0x7f778c102090 2024-03-27T10:44:59.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:44:59.888+0000 7f7788c00700 1 --2- 172.21.15.97:0/600966977 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f77700394d0 0x7f777003b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:44:59.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:44:59.888+0000 7f7788c00700 1 -- 172.21.15.97:0/600966977 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f777c01e070 con 0x7f778c102090 2024-03-27T10:44:59.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:44:59.888+0000 7f778b400700 1 --2- 172.21.15.97:0/600966977 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f77700394d0 0x7f777003b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:44:59.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:44:59.889+0000 7f778b400700 1 --2- 172.21.15.97:0/600966977 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f77700394d0 0x7f777003b990 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7778006fd0 tx=0x7f7778006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:44:59.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:44:59.893+0000 7f7788c00700 1 -- 172.21.15.97:0/600966977 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f777c013070 con 0x7f778c102090 2024-03-27T10:45:00.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:00.126+0000 7f7793200700 1 -- 172.21.15.97:0/600966977 --> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f778c02de30 con 0x7f77700394d0 2024-03-27T10:45:00.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:00.132+0000 7f7788c00700 1 -- 172.21.15.97:0/600966977 <== mgr.14152 v2:172.21.15.97:6800/3955463538 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f778c02de30 con 0x7f77700394d0 2024-03-27T10:45:00.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:00.135+0000 7f7793200700 1 -- 172.21.15.97:0/600966977 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f77700394d0 msgr2=0x7f777003b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:45:00.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:00.135+0000 7f7793200700 1 --2- 172.21.15.97:0/600966977 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f77700394d0 0x7f777003b990 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7778006fd0 tx=0x7f7778006e40 comp rx=0 tx=0).stop 2024-03-27T10:45:00.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:00.135+0000 7f7793200700 1 -- 172.21.15.97:0/600966977 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f778c102090 msgr2=0x7f778c1976c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:45:00.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:00.135+0000 7f7793200700 1 --2- 172.21.15.97:0/600966977 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f778c102090 0x7f778c1976c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f777c005950 tx=0x7f777c004fa0 comp rx=0 tx=0).stop 2024-03-27T10:45:00.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:00.135+0000 7f7793200700 1 -- 172.21.15.97:0/600966977 shutdown_connections 2024-03-27T10:45:00.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:00.135+0000 7f7793200700 1 --2- 172.21.15.97:0/600966977 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f77700394d0 0x7f777003b990 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:45:00.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:00.135+0000 7f7793200700 1 --2- 172.21.15.97:0/600966977 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f778c102090 0x7f778c1976c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:45:00.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:00.135+0000 7f7793200700 1 -- 172.21.15.97:0/600966977 >> 172.21.15.97:0/600966977 conn(0x7f778c0fd630 msgr2=0x7f778c0fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:45:00.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:00.135+0000 7f7793200700 1 -- 172.21.15.97:0/600966977 shutdown_connections 2024-03-27T10:45:00.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:00.135+0000 7f7793200700 1 -- 172.21.15.97:0/600966977 wait complete. 2024-03-27T10:45:00.918 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi114 2024-03-27T10:45:00.918 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-03-27T10:45:00.918 DEBUG:teuthology.orchestra.run.smithi114:> dd of=/etc/ceph/ceph.conf 2024-03-27T10:45:00.974 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-03-27T10:45:00.974 DEBUG:teuthology.orchestra.run.smithi114:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-27T10:45:01.069 INFO:tasks.cephadm:Adding host smithi114 to orchestrator... 2024-03-27T10:45:01.070 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid adca88c4-ec26-11ee-95d0-87774f69a715 -- ceph orch host add smithi114 2024-03-27T10:45:01.132 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:01 smithi097 ceph-mon[31859]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:45:01.133 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:01 smithi097 ceph-mon[31859]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T10:45:01.133 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:01 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:45:01.133 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:01 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:45:01.133 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:01 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:45:01.133 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:01 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:45:01.133 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:01 smithi097 ceph-mon[31859]: Updating smithi097:/etc/ceph/ceph.conf 2024-03-27T10:45:01.471 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T10:45:02.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:02 smithi097 ceph-mon[31859]: Updating smithi097:/var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-27T10:45:03.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:03 smithi097 ceph-mon[31859]: Updating smithi097:/etc/ceph/ceph.client.admin.keyring 2024-03-27T10:45:03.394 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:03 smithi097 ceph-mon[31859]: Updating smithi097:/var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-27T10:45:03.394 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:03 smithi097 ceph-mon[31859]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:45:03.394 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:03 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:45:03.394 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:03 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:45:03.394 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:03 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:45:03.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:03.616+0000 7f48e1c00700 1 -- 172.21.15.97:0/3938895829 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48dc10e640 msgr2=0x7f48dc10ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:45:03.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:03.616+0000 7f48e1c00700 1 --2- 172.21.15.97:0/3938895829 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48dc10e640 0x7f48dc10ea20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f48cc009910 tx=0x7f48cc009c20 comp rx=0 tx=0).stop 2024-03-27T10:45:03.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:03.616+0000 7f48e1c00700 1 -- 172.21.15.97:0/3938895829 shutdown_connections 2024-03-27T10:45:03.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:03.616+0000 7f48e1c00700 1 --2- 172.21.15.97:0/3938895829 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48dc10e640 0x7f48dc10ea20 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:45:03.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:03.616+0000 7f48e1c00700 1 -- 172.21.15.97:0/3938895829 >> 172.21.15.97:0/3938895829 conn(0x7f48dc077de0 msgr2=0x7f48dc07a200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:45:03.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:03.616+0000 7f48e1c00700 1 -- 172.21.15.97:0/3938895829 shutdown_connections 2024-03-27T10:45:03.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:03.617+0000 7f48e1c00700 1 -- 172.21.15.97:0/3938895829 wait complete. 2024-03-27T10:45:03.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:03.618+0000 7f48e1c00700 1 Processor -- start 2024-03-27T10:45:03.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:03.618+0000 7f48e1c00700 1 -- start start 2024-03-27T10:45:03.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:03.618+0000 7f48e1c00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48dc10e640 0x7f48dc19e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:45:03.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:03.618+0000 7f48e1c00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f48dc112b30 con 0x7f48dc10e640 2024-03-27T10:45:03.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:03.618+0000 7f48daa00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48dc10e640 0x7f48dc19e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:45:03.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:03.618+0000 7f48daa00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48dc10e640 0x7f48dc19e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38036/0 (socket says 172.21.15.97:38036) 2024-03-27T10:45:03.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:03.618+0000 7f48daa00700 1 -- 172.21.15.97:0/1052387121 learned_addr learned my addr 172.21.15.97:0/1052387121 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:45:03.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:03.619+0000 7f48daa00700 1 -- 172.21.15.97:0/1052387121 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48dc19e570 con 0x7f48dc10e640 2024-03-27T10:45:03.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:03.619+0000 7f48daa00700 1 --2- 172.21.15.97:0/1052387121 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48dc10e640 0x7f48dc19e030 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f48cc0056d0 tx=0x7f48cc0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:45:03.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:03.619+0000 7f48d3400700 1 -- 172.21.15.97:0/1052387121 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48cc026070 con 0x7f48dc10e640 2024-03-27T10:45:03.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:03.619+0000 7f48d3400700 1 -- 172.21.15.97:0/1052387121 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f48cc00bba0 con 0x7f48dc10e640 2024-03-27T10:45:03.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:03.619+0000 7f48d3400700 1 -- 172.21.15.97:0/1052387121 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48cc0037f0 con 0x7f48dc10e640 2024-03-27T10:45:03.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:03.619+0000 7f48e1c00700 1 -- 172.21.15.97:0/1052387121 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48dc19e860 con 0x7f48dc10e640 2024-03-27T10:45:03.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:03.619+0000 7f48e1c00700 1 -- 172.21.15.97:0/1052387121 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48dc1a2900 con 0x7f48dc10e640 2024-03-27T10:45:03.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:03.621+0000 7f48d3400700 1 -- 172.21.15.97:0/1052387121 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) ==== 46522+0+0 (secure 0 0 0) 0x7f48cc029070 con 0x7f48dc10e640 2024-03-27T10:45:03.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:03.621+0000 7f48e1c00700 1 -- 172.21.15.97:0/1052387121 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48dc109d80 con 0x7f48dc10e640 2024-03-27T10:45:03.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:03.621+0000 7f48d3400700 1 --2- 172.21.15.97:0/1052387121 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f48bc0394d0 0x7f48bc03b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:45:03.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:03.621+0000 7f48d3400700 1 -- 172.21.15.97:0/1052387121 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f48cc01e070 con 0x7f48dc10e640 2024-03-27T10:45:03.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:03.621+0000 7f48d3e00700 1 --2- 172.21.15.97:0/1052387121 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f48bc0394d0 0x7f48bc03b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:45:03.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:03.622+0000 7f48d3e00700 1 --2- 172.21.15.97:0/1052387121 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f48bc0394d0 0x7f48bc03b990 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f48c4006fd0 tx=0x7f48c4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:45:03.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:03.627+0000 7f48d3400700 1 -- 172.21.15.97:0/1052387121 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f48cc013070 con 0x7f48dc10e640 2024-03-27T10:45:03.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:03.855+0000 7f48e1c00700 1 -- 172.21.15.97:0/1052387121 --> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi114", "target": ["mon-mgr", ""]}) -- 0x7f48dc19f390 con 0x7f48bc0394d0 2024-03-27T10:45:05.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:05 smithi097 ceph-mon[31859]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi114", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T10:45:05.394 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:05 smithi097 ceph-mon[31859]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:45:07.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:07 smithi097 ceph-mon[31859]: Deploying cephadm binary to smithi114 2024-03-27T10:45:07.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:07 smithi097 ceph-mon[31859]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:45:09.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:09 smithi097 ceph-mon[31859]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:45:10.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:10.347+0000 7f48d3400700 1 -- 172.21.15.97:0/1052387121 <== mgr.14152 v2:172.21.15.97:6800/3955463538 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f48dc19f390 con 0x7f48bc0394d0 2024-03-27T10:45:10.323 INFO:teuthology.orchestra.run.smithi097.stdout:Added host 'smithi114' with addr '172.21.15.114' 2024-03-27T10:45:10.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:10.350+0000 7f48e1c00700 1 -- 172.21.15.97:0/1052387121 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f48bc0394d0 msgr2=0x7f48bc03b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:45:10.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:10.350+0000 7f48e1c00700 1 --2- 172.21.15.97:0/1052387121 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f48bc0394d0 0x7f48bc03b990 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f48c4006fd0 tx=0x7f48c4006e40 comp rx=0 tx=0).stop 2024-03-27T10:45:10.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:10.350+0000 7f48e1c00700 1 -- 172.21.15.97:0/1052387121 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48dc10e640 msgr2=0x7f48dc19e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:45:10.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:10.350+0000 7f48e1c00700 1 --2- 172.21.15.97:0/1052387121 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48dc10e640 0x7f48dc19e030 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f48cc0056d0 tx=0x7f48cc0057b0 comp rx=0 tx=0).stop 2024-03-27T10:45:10.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:10.351+0000 7f48e1c00700 1 -- 172.21.15.97:0/1052387121 shutdown_connections 2024-03-27T10:45:10.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:10.351+0000 7f48e1c00700 1 --2- 172.21.15.97:0/1052387121 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f48bc0394d0 0x7f48bc03b990 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:45:10.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:10.351+0000 7f48e1c00700 1 --2- 172.21.15.97:0/1052387121 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48dc10e640 0x7f48dc19e030 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:45:10.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:10.351+0000 7f48e1c00700 1 -- 172.21.15.97:0/1052387121 >> 172.21.15.97:0/1052387121 conn(0x7f48dc077de0 msgr2=0x7f48dc110280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:45:10.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:10.351+0000 7f48e1c00700 1 -- 172.21.15.97:0/1052387121 shutdown_connections 2024-03-27T10:45:10.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:10.351+0000 7f48e1c00700 1 -- 172.21.15.97:0/1052387121 wait complete. 2024-03-27T10:45:11.121 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid adca88c4-ec26-11ee-95d0-87774f69a715 -- ceph orch host ls --format=json 2024-03-27T10:45:11.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:11 smithi097 ceph-mon[31859]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:45:11.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:11 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:45:11.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:11 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:45:11.545 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T10:45:12.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:12 smithi097 ceph-mon[31859]: Added host smithi114 2024-03-27T10:45:12.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:12 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:45:12.394 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:12 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:45:13.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:13 smithi097 ceph-mon[31859]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:45:13.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:13.609+0000 7febf2e00700 1 -- 172.21.15.97:0/1862520279 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7febec1044a0 msgr2=0x7febec104880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:45:13.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:13.609+0000 7febf2e00700 1 --2- 172.21.15.97:0/1862520279 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7febec1044a0 0x7febec104880 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7febdc00b0a0 tx=0x7febdc00b3b0 comp rx=0 tx=0).stop 2024-03-27T10:45:13.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:13.609+0000 7febf2e00700 1 -- 172.21.15.97:0/1862520279 shutdown_connections 2024-03-27T10:45:13.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:13.609+0000 7febf2e00700 1 --2- 172.21.15.97:0/1862520279 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7febec1044a0 0x7febec104880 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:45:13.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:13.609+0000 7febf2e00700 1 -- 172.21.15.97:0/1862520279 >> 172.21.15.97:0/1862520279 conn(0x7febec0ffd50 msgr2=0x7febec102170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:45:13.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:13.609+0000 7febf2e00700 1 -- 172.21.15.97:0/1862520279 shutdown_connections 2024-03-27T10:45:13.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:13.609+0000 7febf2e00700 1 -- 172.21.15.97:0/1862520279 wait complete. 2024-03-27T10:45:13.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:13.610+0000 7febf2e00700 1 Processor -- start 2024-03-27T10:45:13.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:13.610+0000 7febf2e00700 1 -- start start 2024-03-27T10:45:13.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:13.611+0000 7febf2e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7febec1044a0 0x7febec19f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:45:13.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:13.611+0000 7febf2e00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7febec10e720 con 0x7febec1044a0 2024-03-27T10:45:13.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:13.611+0000 7febebe00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7febec1044a0 0x7febec19f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:45:13.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:13.611+0000 7febebe00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7febec1044a0 0x7febec19f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37594/0 (socket says 172.21.15.97:37594) 2024-03-27T10:45:13.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:13.611+0000 7febebe00700 1 -- 172.21.15.97:0/3764480555 learned_addr learned my addr 172.21.15.97:0/3764480555 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:45:13.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:13.612+0000 7febebe00700 1 -- 172.21.15.97:0/3764480555 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febec19fd50 con 0x7febec1044a0 2024-03-27T10:45:13.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:13.612+0000 7febebe00700 1 --2- 172.21.15.97:0/3764480555 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7febec1044a0 0x7febec19f810 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7febdc00d010 tx=0x7febdc009790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:45:13.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:13.612+0000 7febe9600700 1 -- 172.21.15.97:0/3764480555 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7febdc017040 con 0x7febec1044a0 2024-03-27T10:45:13.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:13.612+0000 7febe9600700 1 -- 172.21.15.97:0/3764480555 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7febdc022070 con 0x7febec1044a0 2024-03-27T10:45:13.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:13.612+0000 7febe9600700 1 -- 172.21.15.97:0/3764480555 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7febdc01d5e0 con 0x7febec1044a0 2024-03-27T10:45:13.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:13.612+0000 7febf2e00700 1 -- 172.21.15.97:0/3764480555 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febec19ffe0 con 0x7febec1044a0 2024-03-27T10:45:13.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:13.612+0000 7febf2e00700 1 -- 172.21.15.97:0/3764480555 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febec199a20 con 0x7febec1044a0 2024-03-27T10:45:13.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:13.613+0000 7febf2e00700 1 -- 172.21.15.97:0/3764480555 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febec105980 con 0x7febec1044a0 2024-03-27T10:45:13.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:13.613+0000 7febe9600700 1 -- 172.21.15.97:0/3764480555 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) ==== 46522+0+0 (secure 0 0 0) 0x7febdc004040 con 0x7febec1044a0 2024-03-27T10:45:13.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:13.614+0000 7febe9600700 1 --2- 172.21.15.97:0/3764480555 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7febd0039520 0x7febd003b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:45:13.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:13.614+0000 7febe9600700 1 -- 172.21.15.97:0/3764480555 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7febdc00f050 con 0x7febec1044a0 2024-03-27T10:45:13.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:13.614+0000 7febe3400700 1 --2- 172.21.15.97:0/3764480555 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7febd0039520 0x7febd003b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:45:13.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:13.615+0000 7febe3400700 1 --2- 172.21.15.97:0/3764480555 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7febd0039520 0x7febd003b9e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7febd4006fd0 tx=0x7febd4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:45:13.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:13.619+0000 7febe9600700 1 -- 172.21.15.97:0/3764480555 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7febdc01c5f0 con 0x7febec1044a0 2024-03-27T10:45:13.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:13.850+0000 7febf2e00700 1 -- 172.21.15.97:0/3764480555 --> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7febec19af00 con 0x7febd0039520 2024-03-27T10:45:13.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:13.853+0000 7febe9600700 1 -- 172.21.15.97:0/3764480555 <== mgr.14152 v2:172.21.15.97:6800/3955463538 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+160 (secure 0 0 0) 0x7febec19af00 con 0x7febd0039520 2024-03-27T10:45:13.828 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T10:45:13.828 INFO:teuthology.orchestra.run.smithi097.stdout:[{"addr": "172.21.15.97", "hostname": "smithi097", "labels": [], "status": ""}, {"addr": "172.21.15.114", "hostname": "smithi114", "labels": [], "status": ""}] 2024-03-27T10:45:13.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:13.855+0000 7febf2e00700 1 -- 172.21.15.97:0/3764480555 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7febd0039520 msgr2=0x7febd003b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:45:13.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:13.855+0000 7febf2e00700 1 --2- 172.21.15.97:0/3764480555 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7febd0039520 0x7febd003b9e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7febd4006fd0 tx=0x7febd4006e40 comp rx=0 tx=0).stop 2024-03-27T10:45:13.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:13.855+0000 7febf2e00700 1 -- 172.21.15.97:0/3764480555 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7febec1044a0 msgr2=0x7febec19f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:45:13.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:13.855+0000 7febf2e00700 1 --2- 172.21.15.97:0/3764480555 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7febec1044a0 0x7febec19f810 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7febdc00d010 tx=0x7febdc009790 comp rx=0 tx=0).stop 2024-03-27T10:45:13.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:13.855+0000 7febf2e00700 1 -- 172.21.15.97:0/3764480555 shutdown_connections 2024-03-27T10:45:13.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:13.856+0000 7febf2e00700 1 --2- 172.21.15.97:0/3764480555 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7febd0039520 0x7febd003b9e0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:45:13.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:13.856+0000 7febf2e00700 1 --2- 172.21.15.97:0/3764480555 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7febec1044a0 0x7febec19f810 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:45:13.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:13.856+0000 7febf2e00700 1 -- 172.21.15.97:0/3764480555 >> 172.21.15.97:0/3764480555 conn(0x7febec0ffd50 msgr2=0x7febec105e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:45:13.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:13.856+0000 7febf2e00700 1 -- 172.21.15.97:0/3764480555 shutdown_connections 2024-03-27T10:45:13.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:13.856+0000 7febf2e00700 1 -- 172.21.15.97:0/3764480555 wait complete. 2024-03-27T10:45:14.605 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi177 2024-03-27T10:45:14.605 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-03-27T10:45:14.606 DEBUG:teuthology.orchestra.run.smithi177:> dd of=/etc/ceph/ceph.conf 2024-03-27T10:45:14.663 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-03-27T10:45:14.663 DEBUG:teuthology.orchestra.run.smithi177:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-27T10:45:14.757 INFO:tasks.cephadm:Adding host smithi177 to orchestrator... 2024-03-27T10:45:14.757 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid adca88c4-ec26-11ee-95d0-87774f69a715 -- ceph orch host add smithi177 2024-03-27T10:45:15.185 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T10:45:15.483 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:15 smithi097 ceph-mon[31859]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-27T10:45:15.484 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:15 smithi097 ceph-mon[31859]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:45:15.484 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:15 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:45:17.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:17.235+0000 7fbe2ae00700 1 -- 172.21.15.97:0/1568532525 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe24102270 msgr2=0x7fbe24102650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:45:17.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:17.235+0000 7fbe2ae00700 1 --2- 172.21.15.97:0/1568532525 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe24102270 0x7fbe24102650 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fbe14009910 tx=0x7fbe14009c20 comp rx=0 tx=0).stop 2024-03-27T10:45:17.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:17.235+0000 7fbe2ae00700 1 -- 172.21.15.97:0/1568532525 shutdown_connections 2024-03-27T10:45:17.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:17.235+0000 7fbe2ae00700 1 --2- 172.21.15.97:0/1568532525 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe24102270 0x7fbe24102650 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:45:17.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:17.235+0000 7fbe2ae00700 1 -- 172.21.15.97:0/1568532525 >> 172.21.15.97:0/1568532525 conn(0x7fbe240fdb20 msgr2=0x7fbe240fff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:45:17.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:17.235+0000 7fbe2ae00700 1 -- 172.21.15.97:0/1568532525 shutdown_connections 2024-03-27T10:45:17.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:17.235+0000 7fbe2ae00700 1 -- 172.21.15.97:0/1568532525 wait complete. 2024-03-27T10:45:17.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:17.236+0000 7fbe2ae00700 1 Processor -- start 2024-03-27T10:45:17.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:17.236+0000 7fbe2ae00700 1 -- start start 2024-03-27T10:45:17.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:17.237+0000 7fbe2ae00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe24102270 0x7fbe2419f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:45:17.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:17.237+0000 7fbe2ae00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fbe2410c4f0 con 0x7fbe24102270 2024-03-27T10:45:17.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:17.237+0000 7fbe23e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe24102270 0x7fbe2419f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:45:17.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:17.237+0000 7fbe23e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe24102270 0x7fbe2419f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37612/0 (socket says 172.21.15.97:37612) 2024-03-27T10:45:17.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:17.237+0000 7fbe23e00700 1 -- 172.21.15.97:0/1415617030 learned_addr learned my addr 172.21.15.97:0/1415617030 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:45:17.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:17.237+0000 7fbe23e00700 1 -- 172.21.15.97:0/1415617030 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe2419fe10 con 0x7fbe24102270 2024-03-27T10:45:17.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:17.238+0000 7fbe23e00700 1 --2- 172.21.15.97:0/1415617030 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe24102270 0x7fbe2419f8d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fbe14006010 tx=0x7fbe14005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:45:17.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:17.238+0000 7fbe21600700 1 -- 172.21.15.97:0/1415617030 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbe14026070 con 0x7fbe24102270 2024-03-27T10:45:17.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:17.238+0000 7fbe21600700 1 -- 172.21.15.97:0/1415617030 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbe1400bba0 con 0x7fbe24102270 2024-03-27T10:45:17.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:17.238+0000 7fbe21600700 1 -- 172.21.15.97:0/1415617030 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbe140038a0 con 0x7fbe24102270 2024-03-27T10:45:17.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:17.238+0000 7fbe2ae00700 1 -- 172.21.15.97:0/1415617030 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe241a0100 con 0x7fbe24102270 2024-03-27T10:45:17.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:17.238+0000 7fbe2ae00700 1 -- 172.21.15.97:0/1415617030 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe24199a90 con 0x7fbe24102270 2024-03-27T10:45:17.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:17.239+0000 7fbe2ae00700 1 -- 172.21.15.97:0/1415617030 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe24103750 con 0x7fbe24102270 2024-03-27T10:45:17.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:17.239+0000 7fbe21600700 1 -- 172.21.15.97:0/1415617030 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) ==== 46522+0+0 (secure 0 0 0) 0x7fbe14029070 con 0x7fbe24102270 2024-03-27T10:45:17.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:17.240+0000 7fbe21600700 1 --2- 172.21.15.97:0/1415617030 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fbe08039470 0x7fbe0803b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:45:17.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:17.240+0000 7fbe21600700 1 -- 172.21.15.97:0/1415617030 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fbe1401e070 con 0x7fbe24102270 2024-03-27T10:45:17.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:17.240+0000 7fbe1be00700 1 --2- 172.21.15.97:0/1415617030 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fbe08039470 0x7fbe0803b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:45:17.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:17.241+0000 7fbe1be00700 1 --2- 172.21.15.97:0/1415617030 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fbe08039470 0x7fbe0803b930 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c006fd0 tx=0x7fbe0c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:45:17.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:17.246+0000 7fbe21600700 1 -- 172.21.15.97:0/1415617030 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7fbe14013070 con 0x7fbe24102270 2024-03-27T10:45:17.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:17 smithi097 ceph-mon[31859]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:45:17.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:17.477+0000 7fbe2ae00700 1 -- 172.21.15.97:0/1415617030 --> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi177", "target": ["mon-mgr", ""]}) -- 0x7fbe2419b030 con 0x7fbe08039470 2024-03-27T10:45:19.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:19 smithi097 ceph-mon[31859]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi177", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T10:45:19.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:19 smithi097 ceph-mon[31859]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:45:19.644 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:19 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:45:20.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:20 smithi097 ceph-mon[31859]: Deploying cephadm binary to smithi177 2024-03-27T10:45:20.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:20 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:45:20.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:20 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:45:20.644 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:20 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:45:20.644 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:20 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi114", "name": "osd_memory_target"}]: dispatch 2024-03-27T10:45:20.644 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:20 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:45:20.644 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:20 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:45:20.644 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:20 smithi097 ceph-mon[31859]: Updating smithi114:/etc/ceph/ceph.conf 2024-03-27T10:45:21.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:21 smithi097 ceph-mon[31859]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:45:21.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:21 smithi097 ceph-mon[31859]: Updating smithi114:/var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-27T10:45:23.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:23 smithi097 ceph-mon[31859]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:45:23.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:23.975+0000 7fbe21600700 1 -- 172.21.15.97:0/1415617030 <== mgr.14152 v2:172.21.15.97:6800/3955463538 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fbe2419b030 con 0x7fbe08039470 2024-03-27T10:45:23.951 INFO:teuthology.orchestra.run.smithi097.stdout:Added host 'smithi177' with addr '172.21.15.177' 2024-03-27T10:45:23.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:23.979+0000 7fbe2ae00700 1 -- 172.21.15.97:0/1415617030 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fbe08039470 msgr2=0x7fbe0803b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:45:23.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:23.979+0000 7fbe2ae00700 1 --2- 172.21.15.97:0/1415617030 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fbe08039470 0x7fbe0803b930 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c006fd0 tx=0x7fbe0c006e40 comp rx=0 tx=0).stop 2024-03-27T10:45:23.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:23.979+0000 7fbe2ae00700 1 -- 172.21.15.97:0/1415617030 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe24102270 msgr2=0x7fbe2419f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:45:23.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:23.979+0000 7fbe2ae00700 1 --2- 172.21.15.97:0/1415617030 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe24102270 0x7fbe2419f8d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fbe14006010 tx=0x7fbe14005780 comp rx=0 tx=0).stop 2024-03-27T10:45:23.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:23.979+0000 7fbe2ae00700 1 -- 172.21.15.97:0/1415617030 shutdown_connections 2024-03-27T10:45:23.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:23.979+0000 7fbe2ae00700 1 --2- 172.21.15.97:0/1415617030 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fbe08039470 0x7fbe0803b930 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:45:23.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:23.979+0000 7fbe2ae00700 1 --2- 172.21.15.97:0/1415617030 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe24102270 0x7fbe2419f8d0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:45:23.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:23.979+0000 7fbe2ae00700 1 -- 172.21.15.97:0/1415617030 >> 172.21.15.97:0/1415617030 conn(0x7fbe240fdb20 msgr2=0x7fbe24074870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:45:23.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:23.979+0000 7fbe2ae00700 1 -- 172.21.15.97:0/1415617030 shutdown_connections 2024-03-27T10:45:23.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:23.980+0000 7fbe2ae00700 1 -- 172.21.15.97:0/1415617030 wait complete. 2024-03-27T10:45:24.663 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid adca88c4-ec26-11ee-95d0-87774f69a715 -- ceph orch host ls --format=json 2024-03-27T10:45:25.099 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T10:45:25.240 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:24 smithi097 ceph-mon[31859]: Updating smithi114:/etc/ceph/ceph.client.admin.keyring 2024-03-27T10:45:25.241 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:24 smithi097 ceph-mon[31859]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:45:25.241 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:24 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:45:25.241 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:24 smithi097 ceph-mon[31859]: Added host smithi177 2024-03-27T10:45:25.241 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:24 smithi097 ceph-mon[31859]: Updating smithi114:/var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-27T10:45:25.241 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:24 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:45:25.241 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:24 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:45:25.241 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:24 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:45:26.249 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:25 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:45:27.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:27.159+0000 7f7764000700 1 -- 172.21.15.97:0/3195836270 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f775c1044a0 msgr2=0x7f775c104880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:45:27.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:27.159+0000 7f7764000700 1 --2- 172.21.15.97:0/3195836270 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f775c1044a0 0x7f775c104880 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7750009910 tx=0x7f7750009c20 comp rx=0 tx=0).stop 2024-03-27T10:45:27.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:27.160+0000 7f7764000700 1 -- 172.21.15.97:0/3195836270 shutdown_connections 2024-03-27T10:45:27.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:27.160+0000 7f7764000700 1 --2- 172.21.15.97:0/3195836270 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f775c1044a0 0x7f775c104880 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:45:27.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:27.160+0000 7f7764000700 1 -- 172.21.15.97:0/3195836270 >> 172.21.15.97:0/3195836270 conn(0x7f775c0ffd30 msgr2=0x7f775c102150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:45:27.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:27.160+0000 7f7764000700 1 -- 172.21.15.97:0/3195836270 shutdown_connections 2024-03-27T10:45:27.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:27.160+0000 7f7764000700 1 -- 172.21.15.97:0/3195836270 wait complete. 2024-03-27T10:45:27.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:27.161+0000 7f7764000700 1 Processor -- start 2024-03-27T10:45:27.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:27.161+0000 7f7764000700 1 -- start start 2024-03-27T10:45:27.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:27.162+0000 7f7764000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f775c1044a0 0x7f775c19f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:45:27.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:27.162+0000 7f7764000700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f775c10e720 con 0x7f775c1044a0 2024-03-27T10:45:27.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:27.162+0000 7f7761600700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f775c1044a0 0x7f775c19f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:45:27.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:27.162+0000 7f7761600700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f775c1044a0 0x7f775c19f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47590/0 (socket says 172.21.15.97:47590) 2024-03-27T10:45:27.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:27.162+0000 7f7761600700 1 -- 172.21.15.97:0/3025072890 learned_addr learned my addr 172.21.15.97:0/3025072890 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:45:27.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:27.162+0000 7f7761600700 1 -- 172.21.15.97:0/3025072890 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f775c19fe20 con 0x7f775c1044a0 2024-03-27T10:45:27.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:27.163+0000 7f7761600700 1 --2- 172.21.15.97:0/3025072890 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f775c1044a0 0x7f775c19f8e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7750006010 tx=0x7f7750005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:45:27.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:27.163+0000 7f775a000700 1 -- 172.21.15.97:0/3025072890 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7750026070 con 0x7f775c1044a0 2024-03-27T10:45:27.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:27.163+0000 7f775a000700 1 -- 172.21.15.97:0/3025072890 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f775000bba0 con 0x7f775c1044a0 2024-03-27T10:45:27.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:27.163+0000 7f775a000700 1 -- 172.21.15.97:0/3025072890 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f77500038a0 con 0x7f775c1044a0 2024-03-27T10:45:27.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:27.163+0000 7f7764000700 1 -- 172.21.15.97:0/3025072890 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f775c1a0110 con 0x7f775c1044a0 2024-03-27T10:45:27.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:27.163+0000 7f7764000700 1 -- 172.21.15.97:0/3025072890 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f775c199aa0 con 0x7f775c1044a0 2024-03-27T10:45:27.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:27.164+0000 7f775a000700 1 -- 172.21.15.97:0/3025072890 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) ==== 46522+0+0 (secure 0 0 0) 0x7f7750029070 con 0x7f775c1044a0 2024-03-27T10:45:27.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:27.164+0000 7f774f400700 1 -- 172.21.15.97:0/3025072890 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7738005330 con 0x7f775c1044a0 2024-03-27T10:45:27.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:27.165+0000 7f775a000700 1 --2- 172.21.15.97:0/3025072890 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f77440394d0 0x7f774403b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:45:27.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:27.165+0000 7f775a000700 1 -- 172.21.15.97:0/3025072890 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f775001e070 con 0x7f775c1044a0 2024-03-27T10:45:27.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:27.165+0000 7f7760c00700 1 --2- 172.21.15.97:0/3025072890 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f77440394d0 0x7f774403b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:45:27.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:27.166+0000 7f7760c00700 1 --2- 172.21.15.97:0/3025072890 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f77440394d0 0x7f774403b990 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7748006fd0 tx=0x7f7748006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:45:27.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:27.171+0000 7f775a000700 1 -- 172.21.15.97:0/3025072890 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f7750013070 con 0x7f775c1044a0 2024-03-27T10:45:27.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:27.401+0000 7f774f400700 1 -- 172.21.15.97:0/3025072890 --> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7738000bc0 con 0x7f77440394d0 2024-03-27T10:45:27.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:27.404+0000 7f775a000700 1 -- 172.21.15.97:0/3025072890 <== mgr.14152 v2:172.21.15.97:6800/3955463538 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+240 (secure 0 0 0) 0x7f7738000bc0 con 0x7f77440394d0 2024-03-27T10:45:27.379 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T10:45:27.379 INFO:teuthology.orchestra.run.smithi097.stdout:[{"addr": "172.21.15.97", "hostname": "smithi097", "labels": [], "status": ""}, {"addr": "172.21.15.114", "hostname": "smithi114", "labels": [], "status": ""}, {"addr": "172.21.15.177", "hostname": "smithi177", "labels": [], "status": ""}] 2024-03-27T10:45:27.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:27.406+0000 7f774f400700 1 -- 172.21.15.97:0/3025072890 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f77440394d0 msgr2=0x7f774403b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:45:27.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:27.406+0000 7f774f400700 1 --2- 172.21.15.97:0/3025072890 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f77440394d0 0x7f774403b990 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7748006fd0 tx=0x7f7748006e40 comp rx=0 tx=0).stop 2024-03-27T10:45:27.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:27.406+0000 7f774f400700 1 -- 172.21.15.97:0/3025072890 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f775c1044a0 msgr2=0x7f775c19f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:45:27.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:27.406+0000 7f774f400700 1 --2- 172.21.15.97:0/3025072890 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f775c1044a0 0x7f775c19f8e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7750006010 tx=0x7f7750005780 comp rx=0 tx=0).stop 2024-03-27T10:45:27.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:27.407+0000 7f774f400700 1 -- 172.21.15.97:0/3025072890 shutdown_connections 2024-03-27T10:45:27.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:27.407+0000 7f774f400700 1 --2- 172.21.15.97:0/3025072890 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f77440394d0 0x7f774403b990 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:45:27.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:27.407+0000 7f774f400700 1 --2- 172.21.15.97:0/3025072890 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f775c1044a0 0x7f775c19f8e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:45:27.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:27.407+0000 7f774f400700 1 -- 172.21.15.97:0/3025072890 >> 172.21.15.97:0/3025072890 conn(0x7f775c0ffd30 msgr2=0x7f775c105e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:45:27.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:27.407+0000 7f774f400700 1 -- 172.21.15.97:0/3025072890 shutdown_connections 2024-03-27T10:45:27.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:27.407+0000 7f774f400700 1 -- 172.21.15.97:0/3025072890 wait complete. 2024-03-27T10:45:27.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:27 smithi097 ceph-mon[31859]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:45:27.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:27 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:45:27.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:27 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:45:28.138 INFO:tasks.cephadm:Setting crush tunables to default 2024-03-27T10:45:28.138 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid adca88c4-ec26-11ee-95d0-87774f69a715 -- ceph osd crush tunables default 2024-03-27T10:45:28.571 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T10:45:29.358 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:29 smithi097 ceph-mon[31859]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-27T10:45:29.358 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:29 smithi097 ceph-mon[31859]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:45:29.358 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:29 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:45:30.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:30.559+0000 7f7e7ac00700 1 -- 172.21.15.97:0/4131042698 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e741042c0 msgr2=0x7f7e741046a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:45:30.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:30.559+0000 7f7e7ac00700 1 --2- 172.21.15.97:0/4131042698 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e741042c0 0x7f7e741046a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7e68009910 tx=0x7f7e68009c20 comp rx=0 tx=0).stop 2024-03-27T10:45:30.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:30.559+0000 7f7e7ac00700 1 -- 172.21.15.97:0/4131042698 shutdown_connections 2024-03-27T10:45:30.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:30.559+0000 7f7e7ac00700 1 --2- 172.21.15.97:0/4131042698 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e741042c0 0x7f7e741046a0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:45:30.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:30.559+0000 7f7e7ac00700 1 -- 172.21.15.97:0/4131042698 >> 172.21.15.97:0/4131042698 conn(0x7f7e740ffb50 msgr2=0x7f7e74101f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:45:30.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:30.559+0000 7f7e7ac00700 1 -- 172.21.15.97:0/4131042698 shutdown_connections 2024-03-27T10:45:30.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:30.560+0000 7f7e7ac00700 1 -- 172.21.15.97:0/4131042698 wait complete. 2024-03-27T10:45:30.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:30.561+0000 7f7e7ac00700 1 Processor -- start 2024-03-27T10:45:30.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:30.561+0000 7f7e7ac00700 1 -- start start 2024-03-27T10:45:30.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:30.561+0000 7f7e7ac00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e741042c0 0x7f7e7419f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:45:30.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:30.561+0000 7f7e7ac00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7e7410e540 con 0x7f7e741042c0 2024-03-27T10:45:30.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:30.561+0000 7f7e79800700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e741042c0 0x7f7e7419f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:45:30.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:30.562+0000 7f7e79800700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e741042c0 0x7f7e7419f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47012/0 (socket says 172.21.15.97:47012) 2024-03-27T10:45:30.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:30.562+0000 7f7e79800700 1 -- 172.21.15.97:0/2574781535 learned_addr learned my addr 172.21.15.97:0/2574781535 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:45:30.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:30.562+0000 7f7e79800700 1 -- 172.21.15.97:0/2574781535 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e7419fc30 con 0x7f7e741042c0 2024-03-27T10:45:30.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:30.562+0000 7f7e79800700 1 --2- 172.21.15.97:0/2574781535 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e741042c0 0x7f7e7419f6f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7e68006010 tx=0x7f7e68005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:45:30.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:30.562+0000 7f7e72000700 1 -- 172.21.15.97:0/2574781535 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e68026070 con 0x7f7e741042c0 2024-03-27T10:45:30.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:30.562+0000 7f7e72000700 1 -- 172.21.15.97:0/2574781535 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7e6800bba0 con 0x7f7e741042c0 2024-03-27T10:45:30.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:30.562+0000 7f7e7ac00700 1 -- 172.21.15.97:0/2574781535 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e7419ff20 con 0x7f7e741042c0 2024-03-27T10:45:30.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:30.562+0000 7f7e72000700 1 -- 172.21.15.97:0/2574781535 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e680038a0 con 0x7f7e741042c0 2024-03-27T10:45:30.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:30.563+0000 7f7e7ac00700 1 -- 172.21.15.97:0/2574781535 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e741998b0 con 0x7f7e741042c0 2024-03-27T10:45:30.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:30.564+0000 7f7e72000700 1 -- 172.21.15.97:0/2574781535 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) ==== 46522+0+0 (secure 0 0 0) 0x7f7e68029070 con 0x7f7e741042c0 2024-03-27T10:45:30.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:30.564+0000 7f7e7ac00700 1 -- 172.21.15.97:0/2574781535 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e741057a0 con 0x7f7e741042c0 2024-03-27T10:45:30.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:30.564+0000 7f7e72000700 1 --2- 172.21.15.97:0/2574781535 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f7e5c039470 0x7f7e5c03b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:45:30.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:30.564+0000 7f7e72000700 1 -- 172.21.15.97:0/2574781535 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f7e6801e070 con 0x7f7e741042c0 2024-03-27T10:45:30.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:30.564+0000 7f7e78e00700 1 --2- 172.21.15.97:0/2574781535 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f7e5c039470 0x7f7e5c03b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:45:30.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:30.565+0000 7f7e78e00700 1 --2- 172.21.15.97:0/2574781535 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f7e5c039470 0x7f7e5c03b930 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f7e60006fd0 tx=0x7f7e60006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:45:30.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:30.570+0000 7f7e72000700 1 -- 172.21.15.97:0/2574781535 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f7e68013070 con 0x7f7e741042c0 2024-03-27T10:45:30.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:30.800+0000 7f7e7ac00700 1 -- 172.21.15.97:0/2574781535 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f7e7419ade0 con 0x7f7e741042c0 2024-03-27T10:45:31.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:31.194+0000 7f7e72000700 1 -- 172.21.15.97:0/2574781535 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f7e6800bd50 con 0x7f7e741042c0 2024-03-27T10:45:31.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:31.198+0000 7f7e7ac00700 1 -- 172.21.15.97:0/2574781535 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f7e5c039470 msgr2=0x7f7e5c03b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:45:31.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:31.198+0000 7f7e7ac00700 1 --2- 172.21.15.97:0/2574781535 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f7e5c039470 0x7f7e5c03b930 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f7e60006fd0 tx=0x7f7e60006e40 comp rx=0 tx=0).stop 2024-03-27T10:45:31.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:31.198+0000 7f7e7ac00700 1 -- 172.21.15.97:0/2574781535 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e741042c0 msgr2=0x7f7e7419f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:45:31.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:31.198+0000 7f7e7ac00700 1 --2- 172.21.15.97:0/2574781535 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e741042c0 0x7f7e7419f6f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7e68006010 tx=0x7f7e68005780 comp rx=0 tx=0).stop 2024-03-27T10:45:31.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:31.198+0000 7f7e7ac00700 1 -- 172.21.15.97:0/2574781535 shutdown_connections 2024-03-27T10:45:31.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:31.198+0000 7f7e7ac00700 1 --2- 172.21.15.97:0/2574781535 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f7e5c039470 0x7f7e5c03b930 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:45:31.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:31.198+0000 7f7e7ac00700 1 --2- 172.21.15.97:0/2574781535 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e741042c0 0x7f7e7419f6f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:45:31.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:31.198+0000 7f7e7ac00700 1 -- 172.21.15.97:0/2574781535 >> 172.21.15.97:0/2574781535 conn(0x7f7e740ffb50 msgr2=0x7f7e74105c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:45:31.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:31.198+0000 7f7e7ac00700 1 -- 172.21.15.97:0/2574781535 shutdown_connections 2024-03-27T10:45:31.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:45:31.198+0000 7f7e7ac00700 1 -- 172.21.15.97:0/2574781535 wait complete. 2024-03-27T10:45:31.175 INFO:teuthology.orchestra.run.smithi097.stderr:adjusted tunables profile to default 2024-03-27T10:45:31.576 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:31 smithi097 ceph-mon[31859]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:45:31.576 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:31 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/2574781535' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-27T10:45:31.961 INFO:tasks.cephadm:Adding mon.a on smithi097 2024-03-27T10:45:31.962 INFO:tasks.cephadm:Adding mon.b on smithi114 2024-03-27T10:45:31.962 INFO:tasks.cephadm:Adding mon.c on smithi177 2024-03-27T10:45:31.962 DEBUG:teuthology.orchestra.run.smithi177:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid adca88c4-ec26-11ee-95d0-87774f69a715 -- ceph orch apply mon '3;smithi097:172.21.15.97=a;smithi114:172.21.15.114=b;smithi177:172.21.15.177=c' 2024-03-27T10:45:32.388 INFO:teuthology.orchestra.run.smithi177.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-27T10:45:32.494 INFO:teuthology.orchestra.run.smithi177.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-27T10:45:32.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:32 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/2574781535' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-27T10:45:32.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:32 smithi097 ceph-mon[31859]: osdmap e4: 0 total, 0 up, 0 in 2024-03-27T10:45:33.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:33 smithi097 ceph-mon[31859]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:45:34.773 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:34.798+0000 7f2d8ae00700 1 -- 172.21.15.177:0/3246879689 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d84102070 msgr2=0x7f2d84102490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:45:34.773 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:34.798+0000 7f2d8ae00700 1 --2- 172.21.15.177:0/3246879689 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d84102070 0x7f2d84102490 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2d74009910 tx=0x7f2d74009c20 comp rx=0 tx=0).stop 2024-03-27T10:45:34.773 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:34.798+0000 7f2d8ae00700 1 -- 172.21.15.177:0/3246879689 shutdown_connections 2024-03-27T10:45:34.773 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:34.798+0000 7f2d8ae00700 1 --2- 172.21.15.177:0/3246879689 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d84102070 0x7f2d84102490 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:45:34.773 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:34.798+0000 7f2d8ae00700 1 -- 172.21.15.177:0/3246879689 >> 172.21.15.177:0/3246879689 conn(0x7f2d840fd610 msgr2=0x7f2d840ffa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:45:34.774 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:34.798+0000 7f2d8ae00700 1 -- 172.21.15.177:0/3246879689 shutdown_connections 2024-03-27T10:45:34.774 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:34.798+0000 7f2d8ae00700 1 -- 172.21.15.177:0/3246879689 wait complete. 2024-03-27T10:45:34.774 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:34.799+0000 7f2d8ae00700 1 Processor -- start 2024-03-27T10:45:34.774 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:34.799+0000 7f2d8ae00700 1 -- start start 2024-03-27T10:45:34.775 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:34.799+0000 7f2d8ae00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d84102070 0x7f2d841998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:45:34.775 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:34.799+0000 7f2d8ae00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2d84107fb0 con 0x7f2d84102070 2024-03-27T10:45:34.775 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:34.800+0000 7f2d83e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d84102070 0x7f2d841998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:45:34.775 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:34.800+0000 7f2d83e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d84102070 0x7f2d841998e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.177:50264/0 (socket says 172.21.15.177:50264) 2024-03-27T10:45:34.776 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:34.800+0000 7f2d83e00700 1 -- 172.21.15.177:0/3563688642 learned_addr learned my addr 172.21.15.177:0/3563688642 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-03-27T10:45:34.776 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:34.801+0000 7f2d83e00700 1 -- 172.21.15.177:0/3563688642 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d84199e20 con 0x7f2d84102070 2024-03-27T10:45:34.776 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:34.801+0000 7f2d83e00700 1 --2- 172.21.15.177:0/3563688642 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d84102070 0x7f2d841998e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2d74005950 tx=0x7f2d74004fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:45:34.776 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:34.802+0000 7f2d80c00700 1 -- 172.21.15.177:0/3563688642 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d74026070 con 0x7f2d84102070 2024-03-27T10:45:34.776 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:34.802+0000 7f2d8ae00700 1 -- 172.21.15.177:0/3563688642 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d8419a0b0 con 0x7f2d84102070 2024-03-27T10:45:34.777 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:34.802+0000 7f2d80c00700 1 -- 172.21.15.177:0/3563688642 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2d7400bc00 con 0x7f2d84102070 2024-03-27T10:45:34.777 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:34.802+0000 7f2d8ae00700 1 -- 172.21.15.177:0/3563688642 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d8419cda0 con 0x7f2d84102070 2024-03-27T10:45:34.777 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:34.802+0000 7f2d80c00700 1 -- 172.21.15.177:0/3563688642 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d740039a0 con 0x7f2d84102070 2024-03-27T10:45:34.777 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:34.803+0000 7f2d8ae00700 1 -- 172.21.15.177:0/3563688642 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d84108730 con 0x7f2d84102070 2024-03-27T10:45:34.779 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:34.803+0000 7f2d80c00700 1 -- 172.21.15.177:0/3563688642 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) ==== 46522+0+0 (secure 0 0 0) 0x7f2d74029070 con 0x7f2d84102070 2024-03-27T10:45:34.779 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:34.804+0000 7f2d80c00700 1 --2- 172.21.15.177:0/3563688642 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f2d680394d0 0x7f2d6803b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:45:34.779 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:34.804+0000 7f2d80c00700 1 -- 172.21.15.177:0/3563688642 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f2d7401e070 con 0x7f2d84102070 2024-03-27T10:45:34.779 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:34.805+0000 7f2d83400700 1 --2- 172.21.15.177:0/3563688642 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f2d680394d0 0x7f2d6803b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:45:34.780 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:34.806+0000 7f2d83400700 1 --2- 172.21.15.177:0/3563688642 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f2d680394d0 0x7f2d6803b990 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2d70006fd0 tx=0x7f2d70006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:45:34.794 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:34.819+0000 7f2d80c00700 1 -- 172.21.15.177:0/3563688642 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f2d74013070 con 0x7f2d84102070 2024-03-27T10:45:35.032 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:35.057+0000 7f2d8ae00700 1 -- 172.21.15.177:0/3563688642 --> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi097:172.21.15.97=a;smithi114:172.21.15.114=b;smithi177:172.21.15.177=c", "target": ["mon-mgr", ""]}) -- 0x7f2d8402dfc0 con 0x7f2d680394d0 2024-03-27T10:45:35.047 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:35.072+0000 7f2d80c00700 1 -- 172.21.15.177:0/3563688642 <== mgr.14152 v2:172.21.15.97:6800/3955463538 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f2d8402dfc0 con 0x7f2d680394d0 2024-03-27T10:45:35.047 INFO:teuthology.orchestra.run.smithi177.stdout:Scheduled mon update... 2024-03-27T10:45:35.050 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:35.074+0000 7f2d8ae00700 1 -- 172.21.15.177:0/3563688642 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f2d680394d0 msgr2=0x7f2d6803b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:45:35.050 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:35.074+0000 7f2d8ae00700 1 --2- 172.21.15.177:0/3563688642 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f2d680394d0 0x7f2d6803b990 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2d70006fd0 tx=0x7f2d70006e40 comp rx=0 tx=0).stop 2024-03-27T10:45:35.050 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:35.074+0000 7f2d8ae00700 1 -- 172.21.15.177:0/3563688642 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d84102070 msgr2=0x7f2d841998e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:45:35.050 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:35.074+0000 7f2d8ae00700 1 --2- 172.21.15.177:0/3563688642 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d84102070 0x7f2d841998e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2d74005950 tx=0x7f2d74004fa0 comp rx=0 tx=0).stop 2024-03-27T10:45:35.050 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:35.074+0000 7f2d8ae00700 1 -- 172.21.15.177:0/3563688642 shutdown_connections 2024-03-27T10:45:35.050 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:35.074+0000 7f2d8ae00700 1 --2- 172.21.15.177:0/3563688642 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f2d680394d0 0x7f2d6803b990 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:45:35.050 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:35.074+0000 7f2d8ae00700 1 --2- 172.21.15.177:0/3563688642 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d84102070 0x7f2d841998e0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:45:35.050 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:35.074+0000 7f2d8ae00700 1 -- 172.21.15.177:0/3563688642 >> 172.21.15.177:0/3563688642 conn(0x7f2d840fd610 msgr2=0x7f2d840fe1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:45:35.050 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:35.075+0000 7f2d8ae00700 1 -- 172.21.15.177:0/3563688642 shutdown_connections 2024-03-27T10:45:35.050 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:35.075+0000 7f2d8ae00700 1 -- 172.21.15.177:0/3563688642 wait complete. 2024-03-27T10:45:35.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:35 smithi097 ceph-mon[31859]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:45:35.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:35 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:45:35.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:35 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:45:35.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:35 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:45:35.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:35 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:45:35.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:35 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi177", "name": "osd_memory_target"}]: dispatch 2024-03-27T10:45:35.644 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:35 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:45:35.644 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:35 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:45:35.644 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:35 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:45:35.850 DEBUG:teuthology.orchestra.run.smithi114:mon.b> sudo journalctl -f -n 0 -u ceph-adca88c4-ec26-11ee-95d0-87774f69a715@mon.b.service 2024-03-27T10:45:35.858 DEBUG:teuthology.orchestra.run.smithi177:mon.c> sudo journalctl -f -n 0 -u ceph-adca88c4-ec26-11ee-95d0-87774f69a715@mon.c.service 2024-03-27T10:45:35.862 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-27T10:45:35.862 DEBUG:teuthology.orchestra.run.smithi177:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid adca88c4-ec26-11ee-95d0-87774f69a715 -- ceph mon dump -f json 2024-03-27T10:45:36.282 INFO:teuthology.orchestra.run.smithi177.stderr:Inferring config /var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-27T10:45:36.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:36 smithi097 ceph-mon[31859]: Updating smithi177:/etc/ceph/ceph.conf 2024-03-27T10:45:36.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:36 smithi097 ceph-mon[31859]: Updating smithi177:/var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-27T10:45:36.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:36 smithi097 ceph-mon[31859]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi097:172.21.15.97=a;smithi114:172.21.15.114=b;smithi177:172.21.15.177=c", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T10:45:36.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:36 smithi097 ceph-mon[31859]: Saving service mon spec with placement smithi097:172.21.15.97=a;smithi114:172.21.15.114=b;smithi177:172.21.15.177=c;count:3 2024-03-27T10:45:37.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:37 smithi097 ceph-mon[31859]: Updating smithi177:/etc/ceph/ceph.client.admin.keyring 2024-03-27T10:45:37.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:37 smithi097 ceph-mon[31859]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:45:37.644 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:37 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:45:37.644 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:37 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:45:37.644 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:37 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:45:37.644 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:37 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-27T10:45:37.644 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:37 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:45:38.300 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:38.324+0000 7fe6ff800700 1 -- 172.21.15.177:0/3009204764 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6f8102f60 msgr2=0x7fe6f8100470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:45:38.300 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:38.324+0000 7fe6ff800700 1 --2- 172.21.15.177:0/3009204764 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6f8102f60 0x7fe6f8100470 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec009910 tx=0x7fe6ec009c20 comp rx=0 tx=0).stop 2024-03-27T10:45:38.300 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:38.324+0000 7fe6ff800700 1 -- 172.21.15.177:0/3009204764 shutdown_connections 2024-03-27T10:45:38.300 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:38.324+0000 7fe6ff800700 1 --2- 172.21.15.177:0/3009204764 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6f8102f60 0x7fe6f8100470 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:45:38.300 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:38.324+0000 7fe6ff800700 1 -- 172.21.15.177:0/3009204764 >> 172.21.15.177:0/3009204764 conn(0x7fe6f80fc190 msgr2=0x7fe6f80fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:45:38.300 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:38.324+0000 7fe6ff800700 1 -- 172.21.15.177:0/3009204764 shutdown_connections 2024-03-27T10:45:38.300 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:38.324+0000 7fe6ff800700 1 -- 172.21.15.177:0/3009204764 wait complete. 2024-03-27T10:45:38.301 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:38.326+0000 7fe6ff800700 1 Processor -- start 2024-03-27T10:45:38.301 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:38.326+0000 7fe6ff800700 1 -- start start 2024-03-27T10:45:38.301 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:38.326+0000 7fe6ff800700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6f8102f60 0x7fe6f8199d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:45:38.301 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:38.326+0000 7fe6ff800700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe6f810e760 con 0x7fe6f8102f60 2024-03-27T10:45:38.302 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:38.327+0000 7fe6fce00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6f8102f60 0x7fe6f8199d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:45:38.302 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:38.327+0000 7fe6fce00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6f8102f60 0x7fe6f8199d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.177:50278/0 (socket says 172.21.15.177:50278) 2024-03-27T10:45:38.302 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:38.327+0000 7fe6fce00700 1 -- 172.21.15.177:0/2937455704 learned_addr learned my addr 172.21.15.177:0/2937455704 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-03-27T10:45:38.302 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:38.328+0000 7fe6fce00700 1 -- 172.21.15.177:0/2937455704 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6f819a2c0 con 0x7fe6f8102f60 2024-03-27T10:45:38.303 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:38.328+0000 7fe6fce00700 1 --2- 172.21.15.177:0/2937455704 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6f8102f60 0x7fe6f8199d80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec0056d0 tx=0x7fe6ec0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:45:38.303 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:38.328+0000 7fe6f5600700 1 -- 172.21.15.177:0/2937455704 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe6ec026070 con 0x7fe6f8102f60 2024-03-27T10:45:38.303 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:38.328+0000 7fe6ff800700 1 -- 172.21.15.177:0/2937455704 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6f819a550 con 0x7fe6f8102f60 2024-03-27T10:45:38.304 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:38.328+0000 7fe6f5600700 1 -- 172.21.15.177:0/2937455704 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe6ec004370 con 0x7fe6f8102f60 2024-03-27T10:45:38.304 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:38.329+0000 7fe6f5600700 1 -- 172.21.15.177:0/2937455704 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe6ec0039b0 con 0x7fe6f8102f60 2024-03-27T10:45:38.304 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:38.329+0000 7fe6ff800700 1 -- 172.21.15.177:0/2937455704 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6f819a9b0 con 0x7fe6f8102f60 2024-03-27T10:45:38.304 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:38.330+0000 7fe6ff800700 1 -- 172.21.15.177:0/2937455704 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6f8101690 con 0x7fe6f8102f60 2024-03-27T10:45:38.305 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:38.330+0000 7fe6f5600700 1 -- 172.21.15.177:0/2937455704 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) ==== 46522+0+0 (secure 0 0 0) 0x7fe6ec029070 con 0x7fe6f8102f60 2024-03-27T10:45:38.305 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:38.330+0000 7fe6f5600700 1 --2- 172.21.15.177:0/2937455704 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fe6e0039470 0x7fe6e003b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:45:38.305 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:38.331+0000 7fe6f5600700 1 -- 172.21.15.177:0/2937455704 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fe6ec01e070 con 0x7fe6f8102f60 2024-03-27T10:45:38.306 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:38.331+0000 7fe6f7e00700 1 --2- 172.21.15.177:0/2937455704 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fe6e0039470 0x7fe6e003b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:45:38.307 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:38.332+0000 7fe6f7e00700 1 --2- 172.21.15.177:0/2937455704 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fe6e0039470 0x7fe6e003b930 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe6e4006fd0 tx=0x7fe6e4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:45:38.321 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:38.346+0000 7fe6f5600700 1 -- 172.21.15.177:0/2937455704 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7fe6ec013070 con 0x7fe6f8102f60 2024-03-27T10:45:38.626 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:38.651+0000 7fe6ff800700 1 -- 172.21.15.177:0/2937455704 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fe6f819b090 con 0x7fe6f8102f60 2024-03-27T10:45:38.627 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:38.652+0000 7fe6f5600700 1 -- 172.21.15.177:0/2937455704 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7fe6ec04bf30 con 0x7fe6f8102f60 2024-03-27T10:45:38.627 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T10:45:38.627 INFO:teuthology.orchestra.run.smithi177.stdout:{"epoch":1,"fsid":"adca88c4-ec26-11ee-95d0-87774f69a715","modified":"2024-03-27T10:43:20.535207Z","created":"2024-03-27T10:43:20.535207Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:3300","nonce":0},{"type":"v1","addr":"172.21.15.97:6789","nonce":0}]},"addr":"172.21.15.97:6789/0","public_addr":"172.21.15.97:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-27T10:45:38.630 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:38.654+0000 7fe6ff800700 1 -- 172.21.15.177:0/2937455704 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fe6e0039470 msgr2=0x7fe6e003b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:45:38.630 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:38.654+0000 7fe6ff800700 1 --2- 172.21.15.177:0/2937455704 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fe6e0039470 0x7fe6e003b930 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe6e4006fd0 tx=0x7fe6e4006e40 comp rx=0 tx=0).stop 2024-03-27T10:45:38.630 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:38.655+0000 7fe6ff800700 1 -- 172.21.15.177:0/2937455704 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6f8102f60 msgr2=0x7fe6f8199d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:45:38.630 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:38.655+0000 7fe6ff800700 1 --2- 172.21.15.177:0/2937455704 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6f8102f60 0x7fe6f8199d80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec0056d0 tx=0x7fe6ec0057b0 comp rx=0 tx=0).stop 2024-03-27T10:45:38.630 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:38.655+0000 7fe6ff800700 1 -- 172.21.15.177:0/2937455704 shutdown_connections 2024-03-27T10:45:38.630 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:38.655+0000 7fe6ff800700 1 --2- 172.21.15.177:0/2937455704 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fe6e0039470 0x7fe6e003b930 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:45:38.630 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:38.655+0000 7fe6ff800700 1 --2- 172.21.15.177:0/2937455704 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6f8102f60 0x7fe6f8199d80 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:45:38.630 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:38.655+0000 7fe6ff800700 1 -- 172.21.15.177:0/2937455704 >> 172.21.15.177:0/2937455704 conn(0x7fe6f80fc190 msgr2=0x7fe6f80fd210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:45:38.630 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:38.655+0000 7fe6ff800700 1 -- 172.21.15.177:0/2937455704 shutdown_connections 2024-03-27T10:45:38.631 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:38.655+0000 7fe6ff800700 1 -- 172.21.15.177:0/2937455704 wait complete. 2024-03-27T10:45:38.631 INFO:teuthology.orchestra.run.smithi177.stderr:dumped monmap epoch 1 2024-03-27T10:45:38.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:38 smithi097 ceph-mon[31859]: Updating smithi177:/var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-27T10:45:38.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:38 smithi097 ceph-mon[31859]: Deploying daemon mon.c on smithi177 2024-03-27T10:45:39.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:39 smithi097 ceph-mon[31859]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:45:39.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:39 smithi097 ceph-mon[31859]: from='client.? 172.21.15.177:0/2937455704' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-27T10:45:40.753 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-27T10:45:40.753 DEBUG:teuthology.orchestra.run.smithi177:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid adca88c4-ec26-11ee-95d0-87774f69a715 -- ceph mon dump -f json 2024-03-27T10:45:41.500 INFO:teuthology.orchestra.run.smithi177.stderr:Inferring config /var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/mon.c/config 2024-03-27T10:45:41.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:41 smithi097 ceph-mon[31859]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:45:43.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:43 smithi097 ceph-mon[31859]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:45:43.716 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:43 smithi177 podman[35248]: 2024-03-27 10:45:43.614033427 +0000 UTC m=+0.224246914 image pull b379bbd8da6b5d0fa9a3523941efac213b6815e1cbf05c6507954f0a3fe7a801 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772 2024-03-27T10:45:43.716 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:43 smithi177 podman[35248]: 2024-03-27 10:45:43.741052783 +0000 UTC m=+0.351266197 container create 181e3ed3cffed8391460cf6cc9fd5d696fb08e63795ac414596c49fb46791dca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mon-c, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240326.131558-155268c, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-03-27T10:45:44.094 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:44.118+0000 7f8677e00700 1 -- 172.21.15.177:0/221947744 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8678103110 msgr2=0x7f86781034f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:45:44.094 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:44.118+0000 7f8677e00700 1 --2- 172.21.15.177:0/221947744 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8678103110 0x7f86781034f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8668009910 tx=0x7f8668009c20 comp rx=0 tx=0).stop 2024-03-27T10:45:44.095 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:44.119+0000 7f8677e00700 1 -- 172.21.15.177:0/221947744 shutdown_connections 2024-03-27T10:45:44.095 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:44.119+0000 7f8677e00700 1 --2- 172.21.15.177:0/221947744 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8678103110 0x7f86781034f0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:45:44.095 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:44.119+0000 7f8677e00700 1 -- 172.21.15.177:0/221947744 >> 172.21.15.177:0/221947744 conn(0x7f86780fc9a0 msgr2=0x7f86780fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:45:44.095 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:44.119+0000 7f8677e00700 1 -- 172.21.15.177:0/221947744 shutdown_connections 2024-03-27T10:45:44.095 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:44.119+0000 7f8677e00700 1 -- 172.21.15.177:0/221947744 wait complete. 2024-03-27T10:45:44.096 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:44.120+0000 7f8677e00700 1 Processor -- start 2024-03-27T10:45:44.096 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:44.120+0000 7f8677e00700 1 -- start start 2024-03-27T10:45:44.096 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:44.121+0000 7f8677e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8678103110 0x7f8678100d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:45:44.096 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:44.121+0000 7f8677e00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f867810c490 con 0x7f8678103110 2024-03-27T10:45:44.096 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:44.121+0000 7f8676a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8678103110 0x7f8678100d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:45:44.096 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:44.121+0000 7f8676a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8678103110 0x7f8678100d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.177:45426/0 (socket says 172.21.15.177:45426) 2024-03-27T10:45:44.097 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:44.121+0000 7f8676a00700 1 -- 172.21.15.177:0/1798633826 learned_addr learned my addr 172.21.15.177:0/1798633826 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-03-27T10:45:44.097 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:44.122+0000 7f8676a00700 1 -- 172.21.15.177:0/1798633826 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8678102c10 con 0x7f8678103110 2024-03-27T10:45:44.097 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:44.122+0000 7f8676a00700 1 --2- 172.21.15.177:0/1798633826 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8678103110 0x7f8678100d70 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8668006010 tx=0x7f8668005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:45:44.098 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:44.123+0000 7f866f400700 1 -- 172.21.15.177:0/1798633826 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8668026070 con 0x7f8678103110 2024-03-27T10:45:44.098 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:44.123+0000 7f8677e00700 1 -- 172.21.15.177:0/1798633826 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86781012b0 con 0x7f8678103110 2024-03-27T10:45:44.098 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:44.123+0000 7f866f400700 1 -- 172.21.15.177:0/1798633826 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f866800bc00 con 0x7f8678103110 2024-03-27T10:45:44.098 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:44.123+0000 7f866f400700 1 -- 172.21.15.177:0/1798633826 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8668003930 con 0x7f8678103110 2024-03-27T10:45:44.098 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:44.123+0000 7f8677e00700 1 -- 172.21.15.177:0/1798633826 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8678101770 con 0x7f8678103110 2024-03-27T10:45:44.099 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:44.124+0000 7f8677e00700 1 -- 172.21.15.177:0/1798633826 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f867806b100 con 0x7f8678103110 2024-03-27T10:45:44.099 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:44.125+0000 7f866f400700 1 -- 172.21.15.177:0/1798633826 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) ==== 46522+0+0 (secure 0 0 0) 0x7f8668029070 con 0x7f8678103110 2024-03-27T10:45:44.100 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:44.125+0000 7f866f400700 1 --2- 172.21.15.177:0/1798633826 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f865c039470 0x7f865c03b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:45:44.100 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:44.125+0000 7f866f400700 1 -- 172.21.15.177:0/1798633826 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f866801e070 con 0x7f8678103110 2024-03-27T10:45:44.101 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:44.126+0000 7f8676000700 1 --2- 172.21.15.177:0/1798633826 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f865c039470 0x7f865c03b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:45:44.102 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:44.127+0000 7f8676000700 1 --2- 172.21.15.177:0/1798633826 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f865c039470 0x7f865c03b930 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8664006fd0 tx=0x7f8664006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:45:44.116 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:44.141+0000 7f866f400700 1 -- 172.21.15.177:0/1798633826 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f8668013070 con 0x7f8678103110 2024-03-27T10:45:44.227 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:44.252+0000 7f866f400700 1 -- 172.21.15.177:0/1798633826 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f866804cd60 con 0x7f8678103110 2024-03-27T10:45:44.430 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 podman[35248]: 2024-03-27 10:45:44.136397998 +0000 UTC m=+0.746611406 container init 181e3ed3cffed8391460cf6cc9fd5d696fb08e63795ac414596c49fb46791dca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mon-c, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-27T10:45:44.430 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 podman[35248]: 2024-03-27 10:45:44.143920383 +0000 UTC m=+0.754133790 container start 181e3ed3cffed8391460cf6cc9fd5d696fb08e63795ac414596c49fb46791dca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mon-c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-03-27T10:45:44.430 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: set uid:gid to 167:167 (ceph:ceph) 2024-03-27T10:45:44.430 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: ceph version 19.0.0-2478-g155268c4 (155268c4e432a12433aa833f174f9fe3b1016ae0) squid (dev), process ceph-mon, pid 2 2024-03-27T10:45:44.430 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: pidfile_write: ignore empty --pid-file 2024-03-27T10:45:44.431 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: load: jerasure load: lrc 2024-03-27T10:45:44.431 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: RocksDB version: 7.9.2 2024-03-27T10:45:44.431 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Git sha 0 2024-03-27T10:45:44.431 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Compile date 2024-03-26 18:55:08 2024-03-27T10:45:44.431 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: DB SUMMARY 2024-03-27T10:45:44.431 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: DB Session ID: 6B67LX4V13DCESFSSBXQ 2024-03-27T10:45:44.431 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: CURRENT file: CURRENT 2024-03-27T10:45:44.431 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: IDENTITY file: IDENTITY 2024-03-27T10:45:44.431 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-03-27T10:45:44.431 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-03-27T10:45:44.431 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-03-27T10:45:44.431 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.error_if_exists: 0 2024-03-27T10:45:44.432 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.create_if_missing: 0 2024-03-27T10:45:44.432 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.paranoid_checks: 1 2024-03-27T10:45:44.432 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-27T10:45:44.432 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-27T10:45:44.432 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-27T10:45:44.432 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.env: 0x560889d20c60 2024-03-27T10:45:44.432 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.fs: PosixFileSystem 2024-03-27T10:45:44.432 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.info_log: 0x56088c35fb20 2024-03-27T10:45:44.432 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.max_file_opening_threads: 16 2024-03-27T10:45:44.433 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.statistics: (nil) 2024-03-27T10:45:44.433 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.use_fsync: 0 2024-03-27T10:45:44.433 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.max_log_file_size: 0 2024-03-27T10:45:44.433 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-27T10:45:44.433 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-27T10:45:44.433 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.keep_log_file_num: 1000 2024-03-27T10:45:44.433 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.recycle_log_file_num: 0 2024-03-27T10:45:44.433 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.allow_fallocate: 1 2024-03-27T10:45:44.433 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.allow_mmap_reads: 0 2024-03-27T10:45:44.434 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.allow_mmap_writes: 0 2024-03-27T10:45:44.434 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.use_direct_reads: 0 2024-03-27T10:45:44.434 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-27T10:45:44.434 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.create_missing_column_families: 0 2024-03-27T10:45:44.434 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.db_log_dir: 2024-03-27T10:45:44.434 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.wal_dir: 2024-03-27T10:45:44.434 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-27T10:45:44.434 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-27T10:45:44.434 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-27T10:45:44.434 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-27T10:45:44.434 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-27T10:45:44.434 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-27T10:45:44.435 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.advise_random_on_open: 1 2024-03-27T10:45:44.435 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.db_write_buffer_size: 0 2024-03-27T10:45:44.435 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.write_buffer_manager: 0x56088c3301e0 2024-03-27T10:45:44.435 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-27T10:45:44.435 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-27T10:45:44.435 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-27T10:45:44.435 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.rate_limiter: (nil) 2024-03-27T10:45:44.435 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-27T10:45:44.435 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.wal_recovery_mode: 2 2024-03-27T10:45:44.435 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.enable_thread_tracking: 0 2024-03-27T10:45:44.435 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.enable_pipelined_write: 0 2024-03-27T10:45:44.436 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.unordered_write: 0 2024-03-27T10:45:44.436 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-27T10:45:44.436 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-27T10:45:44.436 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-27T10:45:44.436 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-27T10:45:44.436 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.row_cache: None 2024-03-27T10:45:44.436 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.wal_filter: None 2024-03-27T10:45:44.436 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-27T10:45:44.436 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.allow_ingest_behind: 0 2024-03-27T10:45:44.436 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.two_write_queues: 0 2024-03-27T10:45:44.436 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.manual_wal_flush: 0 2024-03-27T10:45:44.436 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.wal_compression: 0 2024-03-27T10:45:44.437 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.atomic_flush: 0 2024-03-27T10:45:44.437 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-27T10:45:44.437 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-27T10:45:44.437 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-27T10:45:44.437 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.log_readahead_size: 0 2024-03-27T10:45:44.437 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-27T10:45:44.437 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.best_efforts_recovery: 0 2024-03-27T10:45:44.437 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-27T10:45:44.437 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-27T10:45:44.437 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.allow_data_in_errors: 0 2024-03-27T10:45:44.437 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.db_host_id: __hostname__ 2024-03-27T10:45:44.437 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-27T10:45:44.437 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.max_background_jobs: 2 2024-03-27T10:45:44.438 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.max_background_compactions: -1 2024-03-27T10:45:44.438 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.max_subcompactions: 1 2024-03-27T10:45:44.438 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-27T10:45:44.438 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-27T10:45:44.438 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-27T10:45:44.438 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.max_total_wal_size: 0 2024-03-27T10:45:44.438 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-27T10:45:44.438 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-27T10:45:44.438 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-27T10:45:44.438 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-27T10:45:44.438 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.max_open_files: -1 2024-03-27T10:45:44.438 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.bytes_per_sync: 0 2024-03-27T10:45:44.438 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-27T10:45:44.439 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-27T10:45:44.439 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.compaction_readahead_size: 0 2024-03-27T10:45:44.439 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.max_background_flushes: -1 2024-03-27T10:45:44.439 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Compression algorithms supported: 2024-03-27T10:45:44.439 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-27T10:45:44.439 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: kZSTD supported: 0 2024-03-27T10:45:44.439 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: kXpressCompression supported: 0 2024-03-27T10:45:44.439 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: kLZ4HCCompression supported: 1 2024-03-27T10:45:44.439 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: kZlibCompression supported: 1 2024-03-27T10:45:44.439 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: kSnappyCompression supported: 1 2024-03-27T10:45:44.441 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: kLZ4Compression supported: 1 2024-03-27T10:45:44.441 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: kBZip2Compression supported: 0 2024-03-27T10:45:44.441 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-27T10:45:44.441 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-27T10:45:44.441 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-03-27T10:45:44.441 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-27T10:45:44.441 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-27T10:45:44.441 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.merge_operator: 2024-03-27T10:45:44.442 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.compaction_filter: None 2024-03-27T10:45:44.442 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.compaction_filter_factory: None 2024-03-27T10:45:44.442 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.sst_partitioner_factory: None 2024-03-27T10:45:44.442 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-27T10:45:44.442 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-27T10:45:44.442 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x56088c35fd00) 2024-03-27T10:45:44.442 INFO:journalctl@ceph.mon.c.smithi177.stdout: cache_index_and_filter_blocks: 1 2024-03-27T10:45:44.442 INFO:journalctl@ceph.mon.c.smithi177.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-27T10:45:44.442 INFO:journalctl@ceph.mon.c.smithi177.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-27T10:45:44.442 INFO:journalctl@ceph.mon.c.smithi177.stdout: pin_top_level_index_and_filter: 1 2024-03-27T10:45:44.442 INFO:journalctl@ceph.mon.c.smithi177.stdout: index_type: 0 2024-03-27T10:45:44.442 INFO:journalctl@ceph.mon.c.smithi177.stdout: data_block_index_type: 0 2024-03-27T10:45:44.443 INFO:journalctl@ceph.mon.c.smithi177.stdout: index_shortening: 1 2024-03-27T10:45:44.443 INFO:journalctl@ceph.mon.c.smithi177.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-27T10:45:44.443 INFO:journalctl@ceph.mon.c.smithi177.stdout: checksum: 4 2024-03-27T10:45:44.443 INFO:journalctl@ceph.mon.c.smithi177.stdout: no_block_cache: 0 2024-03-27T10:45:44.443 INFO:journalctl@ceph.mon.c.smithi177.stdout: block_cache: 0x56088c363090 2024-03-27T10:45:44.443 INFO:journalctl@ceph.mon.c.smithi177.stdout: block_cache_name: BinnedLRUCache 2024-03-27T10:45:44.443 INFO:journalctl@ceph.mon.c.smithi177.stdout: block_cache_options: 2024-03-27T10:45:44.443 INFO:journalctl@ceph.mon.c.smithi177.stdout: capacity : 536870912 2024-03-27T10:45:44.443 INFO:journalctl@ceph.mon.c.smithi177.stdout: num_shard_bits : 4 2024-03-27T10:45:44.443 INFO:journalctl@ceph.mon.c.smithi177.stdout: strict_capacity_limit : 0 2024-03-27T10:45:44.443 INFO:journalctl@ceph.mon.c.smithi177.stdout: high_pri_pool_ratio: 0.000 2024-03-27T10:45:44.443 INFO:journalctl@ceph.mon.c.smithi177.stdout: block_cache_compressed: (nil) 2024-03-27T10:45:44.443 INFO:journalctl@ceph.mon.c.smithi177.stdout: persistent_cache: (nil) 2024-03-27T10:45:44.443 INFO:journalctl@ceph.mon.c.smithi177.stdout: block_size: 4096 2024-03-27T10:45:44.444 INFO:journalctl@ceph.mon.c.smithi177.stdout: block_size_deviation: 10 2024-03-27T10:45:44.444 INFO:journalctl@ceph.mon.c.smithi177.stdout: block_restart_interval: 16 2024-03-27T10:45:44.444 INFO:journalctl@ceph.mon.c.smithi177.stdout: index_block_restart_interval: 1 2024-03-27T10:45:44.444 INFO:journalctl@ceph.mon.c.smithi177.stdout: metadata_block_size: 4096 2024-03-27T10:45:44.444 INFO:journalctl@ceph.mon.c.smithi177.stdout: partition_filters: 0 2024-03-27T10:45:44.444 INFO:journalctl@ceph.mon.c.smithi177.stdout: use_delta_encoding: 1 2024-03-27T10:45:44.444 INFO:journalctl@ceph.mon.c.smithi177.stdout: filter_policy: bloomfilter 2024-03-27T10:45:44.444 INFO:journalctl@ceph.mon.c.smithi177.stdout: whole_key_filtering: 1 2024-03-27T10:45:44.444 INFO:journalctl@ceph.mon.c.smithi177.stdout: verify_compression: 0 2024-03-27T10:45:44.444 INFO:journalctl@ceph.mon.c.smithi177.stdout: read_amp_bytes_per_bit: 0 2024-03-27T10:45:44.444 INFO:journalctl@ceph.mon.c.smithi177.stdout: format_version: 5 2024-03-27T10:45:44.444 INFO:journalctl@ceph.mon.c.smithi177.stdout: enable_index_compression: 1 2024-03-27T10:45:44.445 INFO:journalctl@ceph.mon.c.smithi177.stdout: block_align: 0 2024-03-27T10:45:44.445 INFO:journalctl@ceph.mon.c.smithi177.stdout: max_auto_readahead_size: 262144 2024-03-27T10:45:44.445 INFO:journalctl@ceph.mon.c.smithi177.stdout: prepopulate_block_cache: 0 2024-03-27T10:45:44.445 INFO:journalctl@ceph.mon.c.smithi177.stdout: initial_auto_readahead_size: 8192 2024-03-27T10:45:44.445 INFO:journalctl@ceph.mon.c.smithi177.stdout: num_file_reads_for_auto_readahead: 2 2024-03-27T10:45:44.445 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.write_buffer_size: 33554432 2024-03-27T10:45:44.445 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.max_write_buffer_number: 2 2024-03-27T10:45:44.445 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.compression: NoCompression 2024-03-27T10:45:44.445 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.bottommost_compression: Disabled 2024-03-27T10:45:44.445 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.prefix_extractor: nullptr 2024-03-27T10:45:44.445 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-27T10:45:44.445 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.num_levels: 7 2024-03-27T10:45:44.445 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-27T10:45:44.445 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-27T10:45:44.446 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-27T10:45:44.446 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-27T10:45:44.446 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-27T10:45:44.446 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-27T10:45:44.446 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-27T10:45:44.446 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-27T10:45:44.446 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-27T10:45:44.446 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-27T10:45:44.446 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-27T10:45:44.446 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-27T10:45:44.446 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-27T10:45:44.446 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.compression_opts.level: 32767 2024-03-27T10:45:44.446 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.compression_opts.strategy: 0 2024-03-27T10:45:44.447 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-27T10:45:44.447 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-27T10:45:44.447 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-27T10:45:44.447 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-27T10:45:44.447 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.compression_opts.enabled: false 2024-03-27T10:45:44.447 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-27T10:45:44.447 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-27T10:45:44.447 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-27T10:45:44.447 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-27T10:45:44.447 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.target_file_size_base: 67108864 2024-03-27T10:45:44.447 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-27T10:45:44.447 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-27T10:45:44.447 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-27T10:45:44.447 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-27T10:45:44.448 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-27T10:45:44.448 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-27T10:45:44.448 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-27T10:45:44.448 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-27T10:45:44.448 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-27T10:45:44.448 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-27T10:45:44.448 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-27T10:45:44.448 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-27T10:45:44.448 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-27T10:45:44.448 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-27T10:45:44.448 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.arena_block_size: 1048576 2024-03-27T10:45:44.448 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-27T10:45:44.448 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-27T10:45:44.449 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.disable_auto_compactions: 0 2024-03-27T10:45:44.449 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-27T10:45:44.449 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-27T10:45:44.449 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-27T10:45:44.449 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-27T10:45:44.449 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-27T10:45:44.449 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-27T10:45:44.449 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-27T10:45:44.449 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-27T10:45:44.449 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-27T10:45:44.449 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-27T10:45:44.449 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-03-27T10:45:44.449 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.inplace_update_support: 0 2024-03-27T10:45:44.450 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-27T10:45:44.450 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-27T10:45:44.450 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-27T10:45:44.450 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-27T10:45:44.450 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.bloom_locality: 0 2024-03-27T10:45:44.450 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.max_successive_merges: 0 2024-03-27T10:45:44.450 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-27T10:45:44.450 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.paranoid_file_checks: 0 2024-03-27T10:45:44.450 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.force_consistency_checks: 1 2024-03-27T10:45:44.450 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.report_bg_io_stats: 0 2024-03-27T10:45:44.450 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.ttl: 2592000 2024-03-27T10:45:44.450 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-27T10:45:44.450 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-27T10:45:44.451 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-27T10:45:44.451 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.enable_blob_files: false 2024-03-27T10:45:44.451 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.min_blob_size: 0 2024-03-27T10:45:44.451 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.blob_file_size: 268435456 2024-03-27T10:45:44.451 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-27T10:45:44.451 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-27T10:45:44.451 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-27T10:45:44.451 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-27T10:45:44.451 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-27T10:45:44.451 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.blob_file_starting_level: 0 2024-03-27T10:45:44.451 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-27T10:45:44.451 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-03-27T10:45:44.451 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-03-27T10:45:44.451 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 9e25203a-fe9d-4980-92e2-0fac2522debd 2024-03-27T10:45:44.452 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711536344197255, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-03-27T10:45:44.452 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-03-27T10:45:44.452 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711536344198500, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1711536344, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "9e25203a-fe9d-4980-92e2-0fac2522debd", "db_session_id": "6B67LX4V13DCESFSSBXQ", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-03-27T10:45:44.452 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711536344198820, "job": 1, "event": "recovery_finished"} 2024-03-27T10:45:44.452 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-03-27T10:45:44.452 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-27T10:45:44.452 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x56088c452000 2024-03-27T10:45:44.452 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: DB pointer 0x56088c43c000 2024-03-27T10:45:44.452 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-27T10:45:44.452 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-27T10:45:44.452 INFO:journalctl@ceph.mon.c.smithi177.stdout: ** DB Stats ** 2024-03-27T10:45:44.452 INFO:journalctl@ceph.mon.c.smithi177.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-27T10:45:44.453 INFO:journalctl@ceph.mon.c.smithi177.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-27T10:45:44.453 INFO:journalctl@ceph.mon.c.smithi177.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-27T10:45:44.453 INFO:journalctl@ceph.mon.c.smithi177.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-27T10:45:44.453 INFO:journalctl@ceph.mon.c.smithi177.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-27T10:45:44.453 INFO:journalctl@ceph.mon.c.smithi177.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-27T10:45:44.453 INFO:journalctl@ceph.mon.c.smithi177.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-27T10:45:44.453 INFO:journalctl@ceph.mon.c.smithi177.stdout: 2024-03-27T10:45:44.453 INFO:journalctl@ceph.mon.c.smithi177.stdout: ** Compaction Stats [default] ** 2024-03-27T10:45:44.453 INFO:journalctl@ceph.mon.c.smithi177.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-27T10:45:44.453 INFO:journalctl@ceph.mon.c.smithi177.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-27T10:45:44.453 INFO:journalctl@ceph.mon.c.smithi177.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-27T10:45:44.453 INFO:journalctl@ceph.mon.c.smithi177.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-27T10:45:44.453 INFO:journalctl@ceph.mon.c.smithi177.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-27T10:45:44.453 INFO:journalctl@ceph.mon.c.smithi177.stdout: 2024-03-27T10:45:44.454 INFO:journalctl@ceph.mon.c.smithi177.stdout: ** Compaction Stats [default] ** 2024-03-27T10:45:44.454 INFO:journalctl@ceph.mon.c.smithi177.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-27T10:45:44.454 INFO:journalctl@ceph.mon.c.smithi177.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-27T10:45:44.454 INFO:journalctl@ceph.mon.c.smithi177.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-27T10:45:44.454 INFO:journalctl@ceph.mon.c.smithi177.stdout: 2024-03-27T10:45:44.454 INFO:journalctl@ceph.mon.c.smithi177.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-27T10:45:44.454 INFO:journalctl@ceph.mon.c.smithi177.stdout: 2024-03-27T10:45:44.454 INFO:journalctl@ceph.mon.c.smithi177.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-27T10:45:44.454 INFO:journalctl@ceph.mon.c.smithi177.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-27T10:45:44.454 INFO:journalctl@ceph.mon.c.smithi177.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-27T10:45:44.454 INFO:journalctl@ceph.mon.c.smithi177.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-27T10:45:44.454 INFO:journalctl@ceph.mon.c.smithi177.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-27T10:45:44.454 INFO:journalctl@ceph.mon.c.smithi177.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-27T10:45:44.454 INFO:journalctl@ceph.mon.c.smithi177.stdout: Cumulative compaction: 0.00 GB write, 0.20 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-27T10:45:44.455 INFO:journalctl@ceph.mon.c.smithi177.stdout: Interval compaction: 0.00 GB write, 0.20 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-27T10:45:44.455 INFO:journalctl@ceph.mon.c.smithi177.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-27T10:45:44.455 INFO:journalctl@ceph.mon.c.smithi177.stdout: Block cache BinnedLRUCache@0x56088c363090#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.3e-05 secs_since: 0 2024-03-27T10:45:44.455 INFO:journalctl@ceph.mon.c.smithi177.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-03-27T10:45:44.455 INFO:journalctl@ceph.mon.c.smithi177.stdout: 2024-03-27T10:45:44.455 INFO:journalctl@ceph.mon.c.smithi177.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-27T10:45:44.455 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-03-27T10:45:44.455 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: using public_addr v2:172.21.15.177:0/0 -> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] 2024-03-27T10:45:44.455 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: starting mon.c rank -1 at public addrs [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] at bind addrs [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid adca88c4-ec26-11ee-95d0-87774f69a715 2024-03-27T10:45:44.455 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: mon.c@-1(???) e0 preinit fsid adca88c4-ec26-11ee-95d0-87774f69a715 2024-03-27T10:45:44.455 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 bash[35248]: 181e3ed3cffed8391460cf6cc9fd5d696fb08e63795ac414596c49fb46791dca 2024-03-27T10:45:44.455 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 systemd[1]: Started Ceph mon.c for adca88c4-ec26-11ee-95d0-87774f69a715. 2024-03-27T10:45:44.455 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: mon.c@-1(synchronizing).mds e1 new map 2024-03-27T10:45:44.456 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: mon.c@-1(synchronizing).mds e1 print_map 2024-03-27T10:45:44.456 INFO:journalctl@ceph.mon.c.smithi177.stdout: e1 2024-03-27T10:45:44.456 INFO:journalctl@ceph.mon.c.smithi177.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-27T10:45:44.456 INFO:journalctl@ceph.mon.c.smithi177.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-03-27T10:45:44.456 INFO:journalctl@ceph.mon.c.smithi177.stdout: legacy client fscid: -1 2024-03-27T10:45:44.456 INFO:journalctl@ceph.mon.c.smithi177.stdout: 2024-03-27T10:45:44.456 INFO:journalctl@ceph.mon.c.smithi177.stdout: No filesystems configured 2024-03-27T10:45:44.456 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-03-27T10:45:44.456 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-03-27T10:45:44.456 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-03-27T10:45:44.457 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-03-27T10:45:44.457 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-03-27T10:45:44.457 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-03-27T10:45:44.457 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-27T10:45:44.457 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-27T10:45:44.457 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-27T10:45:44.457 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-27T10:45:44.457 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:45:44.457 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:45:44.457 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:45:44.457 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:45:44.457 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: Added host smithi114 2024-03-27T10:45:44.457 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:45:44.458 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:45:44.458 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:45:44.458 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-27T10:45:44.458 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:45:44.458 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:45:44.458 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:45:44.458 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi177", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T10:45:44.458 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:45:44.458 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:45:44.458 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: Deploying cephadm binary to smithi177 2024-03-27T10:45:44.458 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:45:44.459 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:45:44.459 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:45:44.459 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi114", "name": "osd_memory_target"}]: dispatch 2024-03-27T10:45:44.459 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:45:44.459 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:45:44.459 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: Updating smithi114:/etc/ceph/ceph.conf 2024-03-27T10:45:44.459 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:45:44.459 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: Updating smithi114:/var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-27T10:45:44.459 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:45:44.459 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: Updating smithi114:/etc/ceph/ceph.client.admin.keyring 2024-03-27T10:45:44.459 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:45:44.459 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:45:44.459 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: Added host smithi177 2024-03-27T10:45:44.460 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: Updating smithi114:/var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-27T10:45:44.460 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:45:44.460 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:45:44.460 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:45:44.460 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:45:44.460 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:45:44.460 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:45:44.460 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:45:44.460 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-27T10:45:44.460 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:45:44.460 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:45:44.460 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:45:44.460 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/2574781535' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-27T10:45:44.460 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/2574781535' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-27T10:45:44.461 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: osdmap e4: 0 total, 0 up, 0 in 2024-03-27T10:45:44.461 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:45:44.461 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:45:44.461 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:45:44.461 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:45:44.461 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:45:44.461 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:45:44.461 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi177", "name": "osd_memory_target"}]: dispatch 2024-03-27T10:45:44.461 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:45:44.461 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:45:44.461 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:45:44.461 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: Updating smithi177:/etc/ceph/ceph.conf 2024-03-27T10:45:44.461 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: Updating smithi177:/var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-27T10:45:44.462 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi097:172.21.15.97=a;smithi114:172.21.15.114=b;smithi177:172.21.15.177=c", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T10:45:44.462 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: Saving service mon spec with placement smithi097:172.21.15.97=a;smithi114:172.21.15.114=b;smithi177:172.21.15.177=c;count:3 2024-03-27T10:45:44.462 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: Updating smithi177:/etc/ceph/ceph.client.admin.keyring 2024-03-27T10:45:44.462 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:45:44.462 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:45:44.462 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:45:44.462 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:45:44.462 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-27T10:45:44.462 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:45:44.462 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: Updating smithi177:/var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-27T10:45:44.462 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: Deploying daemon mon.c on smithi177 2024-03-27T10:45:44.462 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:45:44.462 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: from='client.? 172.21.15.177:0/2937455704' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-27T10:45:44.462 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:45:44.463 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:45:44.463 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:44 smithi177 ceph-mon[35294]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-03-27T10:45:44.463 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:44.471+0000 7f8677e00700 1 -- 172.21.15.177:0/1798633826 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f8678102250 con 0x7f8678103110 2024-03-27T10:45:49.253 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:49.277+0000 7f866f400700 1 -- 172.21.15.177:0/1798633826 <== mon.0 v2:172.21.15.97:3300/0 8 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f8668030090 con 0x7f8678103110 2024-03-27T10:45:49.253 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T10:45:49.253 INFO:teuthology.orchestra.run.smithi177.stdout:{"epoch":2,"fsid":"adca88c4-ec26-11ee-95d0-87774f69a715","modified":"2024-03-27T10:45:44.249273Z","created":"2024-03-27T10:43:20.535207Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:3300","nonce":0},{"type":"v1","addr":"172.21.15.97:6789","nonce":0}]},"addr":"172.21.15.97:6789/0","public_addr":"172.21.15.97:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:3300","nonce":0},{"type":"v1","addr":"172.21.15.177:6789","nonce":0}]},"addr":"172.21.15.177:6789/0","public_addr":"172.21.15.177:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-27T10:45:49.257 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:49.282+0000 7f8677e00700 1 -- 172.21.15.177:0/1798633826 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f865c039470 msgr2=0x7f865c03b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:45:49.257 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:49.282+0000 7f8677e00700 1 --2- 172.21.15.177:0/1798633826 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f865c039470 0x7f865c03b930 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8664006fd0 tx=0x7f8664006e40 comp rx=0 tx=0).stop 2024-03-27T10:45:49.257 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:49.282+0000 7f8677e00700 1 -- 172.21.15.177:0/1798633826 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8678103110 msgr2=0x7f8678100d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:45:49.257 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:49.282+0000 7f8677e00700 1 --2- 172.21.15.177:0/1798633826 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8678103110 0x7f8678100d70 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8668006010 tx=0x7f8668005780 comp rx=0 tx=0).stop 2024-03-27T10:45:49.257 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:49.282+0000 7f8677e00700 1 -- 172.21.15.177:0/1798633826 shutdown_connections 2024-03-27T10:45:49.257 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:49.282+0000 7f8677e00700 1 --2- 172.21.15.177:0/1798633826 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f865c039470 0x7f865c03b930 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:45:49.257 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:49.282+0000 7f8677e00700 1 --2- 172.21.15.177:0/1798633826 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8678103110 0x7f8678100d70 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:45:49.257 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:49.282+0000 7f8677e00700 1 -- 172.21.15.177:0/1798633826 >> 172.21.15.177:0/1798633826 conn(0x7f86780fc9a0 msgr2=0x7f8678103e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:45:49.258 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:49.282+0000 7f8677e00700 1 -- 172.21.15.177:0/1798633826 shutdown_connections 2024-03-27T10:45:49.258 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:49.282+0000 7f8677e00700 1 -- 172.21.15.177:0/1798633826 wait complete. 2024-03-27T10:45:49.259 INFO:teuthology.orchestra.run.smithi177.stderr:dumped monmap epoch 2 2024-03-27T10:45:49.591 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:49 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-27T10:45:49.591 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:49 smithi177 ceph-mon[35294]: mon.a calling monitor election 2024-03-27T10:45:49.591 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:49 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-27T10:45:49.591 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:49 smithi177 ceph-mon[35294]: from='client.? 172.21.15.177:0/1798633826' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-27T10:45:49.591 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:49 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-27T10:45:49.591 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:49 smithi177 ceph-mon[35294]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:45:49.591 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:49 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-27T10:45:49.592 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:49 smithi177 ceph-mon[35294]: mon.c calling monitor election 2024-03-27T10:45:49.592 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:49 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-27T10:45:49.592 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:49 smithi177 ceph-mon[35294]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:45:49.592 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:49 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-27T10:45:49.592 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:49 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-27T10:45:49.592 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:49 smithi177 ceph-mon[35294]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-27T10:45:49.592 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:49 smithi177 ceph-mon[35294]: monmap epoch 2 2024-03-27T10:45:49.592 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:49 smithi177 ceph-mon[35294]: fsid adca88c4-ec26-11ee-95d0-87774f69a715 2024-03-27T10:45:49.592 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:49 smithi177 ceph-mon[35294]: last_changed 2024-03-27T10:45:44.249273+0000 2024-03-27T10:45:49.592 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:49 smithi177 ceph-mon[35294]: created 2024-03-27T10:43:20.535207+0000 2024-03-27T10:45:49.592 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:49 smithi177 ceph-mon[35294]: min_mon_release 19 (squid) 2024-03-27T10:45:49.592 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:49 smithi177 ceph-mon[35294]: election_strategy: 1 2024-03-27T10:45:49.592 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:49 smithi177 ceph-mon[35294]: 0: [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] mon.a 2024-03-27T10:45:49.593 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:49 smithi177 ceph-mon[35294]: 1: [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] mon.c 2024-03-27T10:45:49.593 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:49 smithi177 ceph-mon[35294]: fsmap 2024-03-27T10:45:49.593 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:49 smithi177 ceph-mon[35294]: osdmap e4: 0 total, 0 up, 0 in 2024-03-27T10:45:49.593 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:49 smithi177 ceph-mon[35294]: mgrmap e13: x(active, since 69s) 2024-03-27T10:45:49.593 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:49 smithi177 ceph-mon[35294]: overall HEALTH_OK 2024-03-27T10:45:49.593 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:49 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:45:49.593 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:49 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:45:49.644 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:49 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-27T10:45:49.644 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:49 smithi097 ceph-mon[31859]: mon.a calling monitor election 2024-03-27T10:45:49.644 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:49 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-27T10:45:49.644 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:49 smithi097 ceph-mon[31859]: from='client.? 172.21.15.177:0/1798633826' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-27T10:45:49.644 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:49 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-27T10:45:49.644 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:49 smithi097 ceph-mon[31859]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:45:49.645 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:49 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-27T10:45:49.645 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:49 smithi097 ceph-mon[31859]: mon.c calling monitor election 2024-03-27T10:45:49.645 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:49 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-27T10:45:49.645 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:49 smithi097 ceph-mon[31859]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:45:49.645 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:49 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-27T10:45:49.645 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:49 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-27T10:45:49.645 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:49 smithi097 ceph-mon[31859]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-27T10:45:49.645 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:49 smithi097 ceph-mon[31859]: monmap epoch 2 2024-03-27T10:45:49.645 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:49 smithi097 ceph-mon[31859]: fsid adca88c4-ec26-11ee-95d0-87774f69a715 2024-03-27T10:45:49.645 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:49 smithi097 ceph-mon[31859]: last_changed 2024-03-27T10:45:44.249273+0000 2024-03-27T10:45:49.645 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:49 smithi097 ceph-mon[31859]: created 2024-03-27T10:43:20.535207+0000 2024-03-27T10:45:49.645 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:49 smithi097 ceph-mon[31859]: min_mon_release 19 (squid) 2024-03-27T10:45:49.645 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:49 smithi097 ceph-mon[31859]: election_strategy: 1 2024-03-27T10:45:49.645 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:49 smithi097 ceph-mon[31859]: 0: [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] mon.a 2024-03-27T10:45:49.646 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:49 smithi097 ceph-mon[31859]: 1: [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] mon.c 2024-03-27T10:45:49.646 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:49 smithi097 ceph-mon[31859]: fsmap 2024-03-27T10:45:49.646 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:49 smithi097 ceph-mon[31859]: osdmap e4: 0 total, 0 up, 0 in 2024-03-27T10:45:49.646 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:49 smithi097 ceph-mon[31859]: mgrmap e13: x(active, since 69s) 2024-03-27T10:45:49.646 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:49 smithi097 ceph-mon[31859]: overall HEALTH_OK 2024-03-27T10:45:49.646 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:49 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:45:49.646 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:49 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:45:50.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:50 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:45:50.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:50 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-27T10:45:50.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:50 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:45:50.644 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:50 smithi097 ceph-mon[31859]: Deploying daemon mon.b on smithi114 2024-03-27T10:45:50.644 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:50 smithi097 ceph-mon[31859]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:45:50.644 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:50 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-27T10:45:50.679 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:50 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:45:50.679 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:50 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-27T10:45:50.679 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:50 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:45:50.679 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:50 smithi177 ceph-mon[35294]: Deploying daemon mon.b on smithi114 2024-03-27T10:45:50.679 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:50 smithi177 ceph-mon[35294]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:45:50.679 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:50 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-27T10:45:50.947 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-27T10:45:50.947 DEBUG:teuthology.orchestra.run.smithi177:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid adca88c4-ec26-11ee-95d0-87774f69a715 -- ceph mon dump -f json 2024-03-27T10:45:51.392 INFO:teuthology.orchestra.run.smithi177.stderr:Inferring config /var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/mon.c/config 2024-03-27T10:45:51.643 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:45:51 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: 2024-03-27T10:45:51.250+0000 7fb2fda00700 -1 mgr.server handle_report got status from non-daemon mon.c 2024-03-27T10:45:53.122 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:52 smithi177 ceph-mon[35294]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:45:53.289 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:53.314+0000 7f8fbea00700 1 -- 172.21.15.177:0/1654654395 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fb8074a70 msgr2=0x7f8fb8074e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:45:53.290 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:53.314+0000 7f8fbea00700 1 --2- 172.21.15.177:0/1654654395 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fb8074a70 0x7f8fb8074e50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8fa8009910 tx=0x7f8fa8009c20 comp rx=0 tx=0).stop 2024-03-27T10:45:53.290 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:53.314+0000 7f8fbea00700 1 -- 172.21.15.177:0/1654654395 shutdown_connections 2024-03-27T10:45:53.290 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:53.314+0000 7f8fbea00700 1 --2- 172.21.15.177:0/1654654395 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fb8074a70 0x7f8fb8074e50 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:45:53.290 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:53.314+0000 7f8fbea00700 1 -- 172.21.15.177:0/1654654395 >> 172.21.15.177:0/1654654395 conn(0x7f8fb80fdbd0 msgr2=0x7f8fb80ffff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:45:53.290 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:53.314+0000 7f8fbea00700 1 -- 172.21.15.177:0/1654654395 shutdown_connections 2024-03-27T10:45:53.290 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:53.314+0000 7f8fbea00700 1 -- 172.21.15.177:0/1654654395 wait complete. 2024-03-27T10:45:53.291 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:53.315+0000 7f8fbea00700 1 Processor -- start 2024-03-27T10:45:53.291 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:53.316+0000 7f8fbea00700 1 -- start start 2024-03-27T10:45:53.291 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:53.316+0000 7f8fbea00700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8fb8074a70 0x7f8fb8199f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:45:53.291 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:53.316+0000 7f8fbea00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fb819a4d0 0x7f8fb819e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:45:53.291 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:53.316+0000 7f8fbea00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb810e860 con 0x7f8fb819a4d0 2024-03-27T10:45:53.291 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:53.316+0000 7f8fbea00700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb810e6e0 con 0x7f8fb8074a70 2024-03-27T10:45:53.292 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:53.317+0000 7f8fb7e00700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8fb8074a70 0x7f8fb8199f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:45:53.293 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:53.317+0000 7f8fb7e00700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8fb8074a70 0x7f8fb8199f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:34804/0 (socket says 172.21.15.177:34804) 2024-03-27T10:45:53.293 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:53.317+0000 7f8fb7e00700 1 -- 172.21.15.177:0/2026675309 learned_addr learned my addr 172.21.15.177:0/2026675309 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-03-27T10:45:53.293 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:53.317+0000 7f8fb7e00700 1 -- 172.21.15.177:0/2026675309 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8fb8074a70 msgr2=0x7f8fb8199f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-03-27T10:45:53.293 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:53.317+0000 7f8fb7e00700 1 -- 172.21.15.177:0/2026675309 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8fb8074a70 msgr2=0x7f8fb8199f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-27T10:45:53.293 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:53.317+0000 7f8fb7e00700 1 --2- 172.21.15.177:0/2026675309 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8fb8074a70 0x7f8fb8199f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-27T10:45:53.293 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:53.317+0000 7f8fafe00700 1 --2- 172.21.15.177:0/2026675309 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fb819a4d0 0x7f8fb819e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:45:53.293 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:53.317+0000 7f8fb7e00700 1 --2- 172.21.15.177:0/2026675309 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8fb8074a70 0x7f8fb8199f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-27T10:45:53.293 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:53.317+0000 7f8fafe00700 1 -- 172.21.15.177:0/2026675309 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8fb8074a70 msgr2=0x7f8fb8199f90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-27T10:45:53.293 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:53.317+0000 7f8fafe00700 1 --2- 172.21.15.177:0/2026675309 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8fb8074a70 0x7f8fb8199f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:45:53.293 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:53.317+0000 7f8fafe00700 1 -- 172.21.15.177:0/2026675309 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fb819ab10 con 0x7f8fb819a4d0 2024-03-27T10:45:53.293 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:53.318+0000 7f8fafe00700 1 --2- 172.21.15.177:0/2026675309 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fb819a4d0 0x7f8fb819e860 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8fa000b4d0 tx=0x7f8fa000b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:45:53.294 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:53.318+0000 7f8fb5600700 1 -- 172.21.15.177:0/2026675309 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f8fa0011630 con 0x7f8fb819a4d0 2024-03-27T10:45:53.294 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:53.318+0000 7f8fbea00700 1 -- 172.21.15.177:0/2026675309 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fb819eda0 con 0x7f8fb819a4d0 2024-03-27T10:45:53.294 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:53.319+0000 7f8fbea00700 1 -- 172.21.15.177:0/2026675309 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fb819f300 con 0x7f8fb819a4d0 2024-03-27T10:45:53.295 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:53.319+0000 7f8fb5600700 1 -- 172.21.15.177:0/2026675309 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8fa0011c80 con 0x7f8fb819a4d0 2024-03-27T10:45:53.295 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:53.319+0000 7f8fb5600700 1 -- 172.21.15.177:0/2026675309 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f8fa0010da0 con 0x7f8fb819a4d0 2024-03-27T10:45:53.295 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:53.320+0000 7f8fb5600700 1 -- 172.21.15.177:0/2026675309 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) ==== 46522+0+0 (secure 0 0 0) 0x7f8fa0018020 con 0x7f8fb819a4d0 2024-03-27T10:45:53.295 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:53.320+0000 7f8fbea00700 1 -- 172.21.15.177:0/2026675309 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f94005330 con 0x7f8fb819a4d0 2024-03-27T10:45:53.295 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:53.321+0000 7f8fb5600700 1 --2- 172.21.15.177:0/2026675309 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f8f9c039260 0x7f8f9c03b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:45:53.295 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:53.321+0000 7f8fb5600700 1 -- 172.21.15.177:0/2026675309 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f8fa004be10 con 0x7f8fb819a4d0 2024-03-27T10:45:53.296 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:53.321+0000 7f8fb7e00700 1 --2- 172.21.15.177:0/2026675309 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f8f9c039260 0x7f8f9c03b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:45:53.297 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:53.322+0000 7f8fb7e00700 1 --2- 172.21.15.177:0/2026675309 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f8f9c039260 0x7f8f9c03b720 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f8fa8009750 tx=0x7f8fa801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:45:53.314 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:53.338+0000 7f8fb5600700 1 -- 172.21.15.177:0/2026675309 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f8fa001f020 con 0x7f8fb819a4d0 2024-03-27T10:45:53.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:52 smithi097 ceph-mon[31859]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:45:53.628 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:53.653+0000 7f8fbea00700 1 -- 172.21.15.177:0/2026675309 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f8f94005620 con 0x7f8fb819a4d0 2024-03-27T10:45:53.629 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:53.654+0000 7f8fb5600700 1 -- 172.21.15.177:0/2026675309 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f8fa001a400 con 0x7f8fb819a4d0 2024-03-27T10:45:53.629 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T10:45:53.629 INFO:teuthology.orchestra.run.smithi177.stdout:{"epoch":2,"fsid":"adca88c4-ec26-11ee-95d0-87774f69a715","modified":"2024-03-27T10:45:44.249273Z","created":"2024-03-27T10:43:20.535207Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:3300","nonce":0},{"type":"v1","addr":"172.21.15.97:6789","nonce":0}]},"addr":"172.21.15.97:6789/0","public_addr":"172.21.15.97:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:3300","nonce":0},{"type":"v1","addr":"172.21.15.177:6789","nonce":0}]},"addr":"172.21.15.177:6789/0","public_addr":"172.21.15.177:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-27T10:45:53.632 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:53.657+0000 7f8fbea00700 1 -- 172.21.15.177:0/2026675309 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f8f9c039260 msgr2=0x7f8f9c03b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:45:53.632 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:53.657+0000 7f8fbea00700 1 --2- 172.21.15.177:0/2026675309 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f8f9c039260 0x7f8f9c03b720 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f8fa8009750 tx=0x7f8fa801a040 comp rx=0 tx=0).stop 2024-03-27T10:45:53.632 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:53.657+0000 7f8fbea00700 1 -- 172.21.15.177:0/2026675309 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fb819a4d0 msgr2=0x7f8fb819e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:45:53.632 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:53.657+0000 7f8fbea00700 1 --2- 172.21.15.177:0/2026675309 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fb819a4d0 0x7f8fb819e860 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8fa000b4d0 tx=0x7f8fa000b890 comp rx=0 tx=0).stop 2024-03-27T10:45:53.632 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:53.657+0000 7f8fbea00700 1 -- 172.21.15.177:0/2026675309 shutdown_connections 2024-03-27T10:45:53.632 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:53.657+0000 7f8fbea00700 1 --2- 172.21.15.177:0/2026675309 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f8f9c039260 0x7f8f9c03b720 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:45:53.632 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:53.657+0000 7f8fbea00700 1 --2- 172.21.15.177:0/2026675309 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8fb8074a70 0x7f8fb8199f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:45:53.633 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:53.657+0000 7f8fbea00700 1 --2- 172.21.15.177:0/2026675309 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fb819a4d0 0x7f8fb819e860 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:45:53.633 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:53.657+0000 7f8fbea00700 1 -- 172.21.15.177:0/2026675309 >> 172.21.15.177:0/2026675309 conn(0x7f8fb80fdbd0 msgr2=0x7f8fb81083b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:45:53.633 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:53.657+0000 7f8fbea00700 1 -- 172.21.15.177:0/2026675309 shutdown_connections 2024-03-27T10:45:53.633 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:45:53.657+0000 7f8fbea00700 1 -- 172.21.15.177:0/2026675309 wait complete. 2024-03-27T10:45:53.633 INFO:teuthology.orchestra.run.smithi177.stderr:dumped monmap epoch 2 2024-03-27T10:45:54.057 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:45:53 smithi177 ceph-mon[35294]: from='client.? 172.21.15.177:0/2026675309' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-27T10:45:54.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:45:53 smithi097 ceph-mon[31859]: from='client.? 172.21.15.177:0/2026675309' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-27T10:45:54.847 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 podman[35320]: 2024-03-27 10:45:54.870588553 +0000 UTC m=+0.378969569 container init be7a212c5f9df21ebb1eeae7ac16109ac796a86c64eba1e50b8ce424ae0d0033 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mon-b, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240326.131558-155268c, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-03-27T10:45:55.271 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 podman[35320]: 2024-03-27 10:45:54.881976187 +0000 UTC m=+0.390357189 container start be7a212c5f9df21ebb1eeae7ac16109ac796a86c64eba1e50b8ce424ae0d0033 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mon-b, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_BRANCH=HEAD, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-27T10:45:55.271 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: set uid:gid to 167:167 (ceph:ceph) 2024-03-27T10:45:55.271 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: ceph version 19.0.0-2478-g155268c4 (155268c4e432a12433aa833f174f9fe3b1016ae0) squid (dev), process ceph-mon, pid 2 2024-03-27T10:45:55.271 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: pidfile_write: ignore empty --pid-file 2024-03-27T10:45:55.271 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: load: jerasure load: lrc 2024-03-27T10:45:55.271 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: RocksDB version: 7.9.2 2024-03-27T10:45:55.271 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Git sha 0 2024-03-27T10:45:55.271 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Compile date 2024-03-26 18:55:08 2024-03-27T10:45:55.271 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: DB SUMMARY 2024-03-27T10:45:55.271 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: DB Session ID: 20HANU3BKUG27D5MLYXU 2024-03-27T10:45:55.271 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: CURRENT file: CURRENT 2024-03-27T10:45:55.272 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: IDENTITY file: IDENTITY 2024-03-27T10:45:55.272 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-03-27T10:45:55.272 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-03-27T10:45:55.272 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-03-27T10:45:55.272 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.error_if_exists: 0 2024-03-27T10:45:55.272 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.create_if_missing: 0 2024-03-27T10:45:55.272 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.paranoid_checks: 1 2024-03-27T10:45:55.272 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-27T10:45:55.272 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-27T10:45:55.272 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-27T10:45:55.272 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.env: 0x5631ec0e0c60 2024-03-27T10:45:55.272 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.fs: PosixFileSystem 2024-03-27T10:45:55.272 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.info_log: 0x5631ed6a1b20 2024-03-27T10:45:55.272 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.max_file_opening_threads: 16 2024-03-27T10:45:55.273 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.statistics: (nil) 2024-03-27T10:45:55.273 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.use_fsync: 0 2024-03-27T10:45:55.273 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.max_log_file_size: 0 2024-03-27T10:45:55.273 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-27T10:45:55.273 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-27T10:45:55.273 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.keep_log_file_num: 1000 2024-03-27T10:45:55.273 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.recycle_log_file_num: 0 2024-03-27T10:45:55.273 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.allow_fallocate: 1 2024-03-27T10:45:55.274 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.allow_mmap_reads: 0 2024-03-27T10:45:55.274 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.allow_mmap_writes: 0 2024-03-27T10:45:55.274 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.use_direct_reads: 0 2024-03-27T10:45:55.274 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-27T10:45:55.274 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.create_missing_column_families: 0 2024-03-27T10:45:55.274 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.db_log_dir: 2024-03-27T10:45:55.274 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.wal_dir: 2024-03-27T10:45:55.274 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-27T10:45:55.274 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-27T10:45:55.274 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-27T10:45:55.275 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-27T10:45:55.275 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-27T10:45:55.275 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-27T10:45:55.275 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.advise_random_on_open: 1 2024-03-27T10:45:55.275 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.db_write_buffer_size: 0 2024-03-27T10:45:55.275 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.write_buffer_manager: 0x5631ed6721e0 2024-03-27T10:45:55.275 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-27T10:45:55.275 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-27T10:45:55.275 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-27T10:45:55.275 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.rate_limiter: (nil) 2024-03-27T10:45:55.275 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-27T10:45:55.275 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.wal_recovery_mode: 2 2024-03-27T10:45:55.275 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.enable_thread_tracking: 0 2024-03-27T10:45:55.276 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.enable_pipelined_write: 0 2024-03-27T10:45:55.276 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.unordered_write: 0 2024-03-27T10:45:55.276 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-27T10:45:55.276 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-27T10:45:55.276 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-27T10:45:55.276 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-27T10:45:55.276 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.row_cache: None 2024-03-27T10:45:55.276 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.wal_filter: None 2024-03-27T10:45:55.276 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-27T10:45:55.276 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.allow_ingest_behind: 0 2024-03-27T10:45:55.276 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.two_write_queues: 0 2024-03-27T10:45:55.277 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.manual_wal_flush: 0 2024-03-27T10:45:55.277 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.wal_compression: 0 2024-03-27T10:45:55.277 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.atomic_flush: 0 2024-03-27T10:45:55.277 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-27T10:45:55.277 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-27T10:45:55.277 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-27T10:45:55.277 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.log_readahead_size: 0 2024-03-27T10:45:55.277 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-27T10:45:55.277 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.best_efforts_recovery: 0 2024-03-27T10:45:55.277 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-27T10:45:55.277 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-27T10:45:55.277 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.allow_data_in_errors: 0 2024-03-27T10:45:55.277 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.db_host_id: __hostname__ 2024-03-27T10:45:55.278 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-27T10:45:55.278 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.max_background_jobs: 2 2024-03-27T10:45:55.278 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.max_background_compactions: -1 2024-03-27T10:45:55.278 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.max_subcompactions: 1 2024-03-27T10:45:55.278 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-27T10:45:55.278 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-27T10:45:55.278 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-27T10:45:55.278 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.max_total_wal_size: 0 2024-03-27T10:45:55.278 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-27T10:45:55.279 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-27T10:45:55.279 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-27T10:45:55.279 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-27T10:45:55.279 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.max_open_files: -1 2024-03-27T10:45:55.279 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.bytes_per_sync: 0 2024-03-27T10:45:55.279 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-27T10:45:55.279 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-27T10:45:55.279 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.compaction_readahead_size: 0 2024-03-27T10:45:55.279 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.max_background_flushes: -1 2024-03-27T10:45:55.279 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Compression algorithms supported: 2024-03-27T10:45:55.279 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-27T10:45:55.279 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: kZSTD supported: 0 2024-03-27T10:45:55.279 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: kXpressCompression supported: 0 2024-03-27T10:45:55.279 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: kLZ4HCCompression supported: 1 2024-03-27T10:45:55.280 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: kZlibCompression supported: 1 2024-03-27T10:45:55.280 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: kSnappyCompression supported: 1 2024-03-27T10:45:55.280 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: kLZ4Compression supported: 1 2024-03-27T10:45:55.280 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: kBZip2Compression supported: 0 2024-03-27T10:45:55.280 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-27T10:45:55.280 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-27T10:45:55.280 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-03-27T10:45:55.280 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-27T10:45:55.280 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-27T10:45:55.280 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.merge_operator: 2024-03-27T10:45:55.280 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.compaction_filter: None 2024-03-27T10:45:55.280 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.compaction_filter_factory: None 2024-03-27T10:45:55.280 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.sst_partitioner_factory: None 2024-03-27T10:45:55.281 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-27T10:45:55.281 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-27T10:45:55.281 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5631ed6a1d00) 2024-03-27T10:45:55.281 INFO:journalctl@ceph.mon.b.smithi114.stdout: cache_index_and_filter_blocks: 1 2024-03-27T10:45:55.281 INFO:journalctl@ceph.mon.b.smithi114.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-27T10:45:55.281 INFO:journalctl@ceph.mon.b.smithi114.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-27T10:45:55.281 INFO:journalctl@ceph.mon.b.smithi114.stdout: pin_top_level_index_and_filter: 1 2024-03-27T10:45:55.281 INFO:journalctl@ceph.mon.b.smithi114.stdout: index_type: 0 2024-03-27T10:45:55.281 INFO:journalctl@ceph.mon.b.smithi114.stdout: data_block_index_type: 0 2024-03-27T10:45:55.281 INFO:journalctl@ceph.mon.b.smithi114.stdout: index_shortening: 1 2024-03-27T10:45:55.281 INFO:journalctl@ceph.mon.b.smithi114.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-27T10:45:55.281 INFO:journalctl@ceph.mon.b.smithi114.stdout: checksum: 4 2024-03-27T10:45:55.281 INFO:journalctl@ceph.mon.b.smithi114.stdout: no_block_cache: 0 2024-03-27T10:45:55.281 INFO:journalctl@ceph.mon.b.smithi114.stdout: block_cache: 0x5631ed6a5090 2024-03-27T10:45:55.282 INFO:journalctl@ceph.mon.b.smithi114.stdout: block_cache_name: BinnedLRUCache 2024-03-27T10:45:55.282 INFO:journalctl@ceph.mon.b.smithi114.stdout: block_cache_options: 2024-03-27T10:45:55.282 INFO:journalctl@ceph.mon.b.smithi114.stdout: capacity : 536870912 2024-03-27T10:45:55.282 INFO:journalctl@ceph.mon.b.smithi114.stdout: num_shard_bits : 4 2024-03-27T10:45:55.282 INFO:journalctl@ceph.mon.b.smithi114.stdout: strict_capacity_limit : 0 2024-03-27T10:45:55.282 INFO:journalctl@ceph.mon.b.smithi114.stdout: high_pri_pool_ratio: 0.000 2024-03-27T10:45:55.282 INFO:journalctl@ceph.mon.b.smithi114.stdout: block_cache_compressed: (nil) 2024-03-27T10:45:55.282 INFO:journalctl@ceph.mon.b.smithi114.stdout: persistent_cache: (nil) 2024-03-27T10:45:55.282 INFO:journalctl@ceph.mon.b.smithi114.stdout: block_size: 4096 2024-03-27T10:45:55.282 INFO:journalctl@ceph.mon.b.smithi114.stdout: block_size_deviation: 10 2024-03-27T10:45:55.282 INFO:journalctl@ceph.mon.b.smithi114.stdout: block_restart_interval: 16 2024-03-27T10:45:55.282 INFO:journalctl@ceph.mon.b.smithi114.stdout: index_block_restart_interval: 1 2024-03-27T10:45:55.282 INFO:journalctl@ceph.mon.b.smithi114.stdout: metadata_block_size: 4096 2024-03-27T10:45:55.283 INFO:journalctl@ceph.mon.b.smithi114.stdout: partition_filters: 0 2024-03-27T10:45:55.283 INFO:journalctl@ceph.mon.b.smithi114.stdout: use_delta_encoding: 1 2024-03-27T10:45:55.283 INFO:journalctl@ceph.mon.b.smithi114.stdout: filter_policy: bloomfilter 2024-03-27T10:45:55.283 INFO:journalctl@ceph.mon.b.smithi114.stdout: whole_key_filtering: 1 2024-03-27T10:45:55.283 INFO:journalctl@ceph.mon.b.smithi114.stdout: verify_compression: 0 2024-03-27T10:45:55.283 INFO:journalctl@ceph.mon.b.smithi114.stdout: read_amp_bytes_per_bit: 0 2024-03-27T10:45:55.283 INFO:journalctl@ceph.mon.b.smithi114.stdout: format_version: 5 2024-03-27T10:45:55.283 INFO:journalctl@ceph.mon.b.smithi114.stdout: enable_index_compression: 1 2024-03-27T10:45:55.283 INFO:journalctl@ceph.mon.b.smithi114.stdout: block_align: 0 2024-03-27T10:45:55.283 INFO:journalctl@ceph.mon.b.smithi114.stdout: max_auto_readahead_size: 262144 2024-03-27T10:45:55.283 INFO:journalctl@ceph.mon.b.smithi114.stdout: prepopulate_block_cache: 0 2024-03-27T10:45:55.283 INFO:journalctl@ceph.mon.b.smithi114.stdout: initial_auto_readahead_size: 8192 2024-03-27T10:45:55.283 INFO:journalctl@ceph.mon.b.smithi114.stdout: num_file_reads_for_auto_readahead: 2 2024-03-27T10:45:55.283 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.write_buffer_size: 33554432 2024-03-27T10:45:55.284 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.max_write_buffer_number: 2 2024-03-27T10:45:55.284 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.compression: NoCompression 2024-03-27T10:45:55.284 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.bottommost_compression: Disabled 2024-03-27T10:45:55.284 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.prefix_extractor: nullptr 2024-03-27T10:45:55.284 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-27T10:45:55.284 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.num_levels: 7 2024-03-27T10:45:55.284 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-27T10:45:55.285 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-27T10:45:55.285 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-27T10:45:55.286 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-27T10:45:55.286 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-27T10:45:55.286 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-27T10:45:55.286 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-27T10:45:55.286 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-27T10:45:55.286 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-27T10:45:55.286 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-27T10:45:55.286 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-27T10:45:55.286 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-27T10:45:55.286 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-27T10:45:55.286 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.compression_opts.level: 32767 2024-03-27T10:45:55.286 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.compression_opts.strategy: 0 2024-03-27T10:45:55.287 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-27T10:45:55.287 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-27T10:45:55.287 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-27T10:45:55.287 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-27T10:45:55.287 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.compression_opts.enabled: false 2024-03-27T10:45:55.287 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-27T10:45:55.287 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-27T10:45:55.287 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-27T10:45:55.287 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-27T10:45:55.287 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.target_file_size_base: 67108864 2024-03-27T10:45:55.287 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-27T10:45:55.287 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-27T10:45:55.287 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-27T10:45:55.287 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-27T10:45:55.288 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-27T10:45:55.288 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-27T10:45:55.288 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-27T10:45:55.288 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-27T10:45:55.288 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-27T10:45:55.288 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-27T10:45:55.288 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-27T10:45:55.288 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-27T10:45:55.288 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-27T10:45:55.288 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-27T10:45:55.288 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.arena_block_size: 1048576 2024-03-27T10:45:55.288 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-27T10:45:55.288 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-27T10:45:55.289 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.disable_auto_compactions: 0 2024-03-27T10:45:55.289 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-27T10:45:55.289 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-27T10:45:55.289 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-27T10:45:55.289 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-27T10:45:55.289 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-27T10:45:55.289 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-27T10:45:55.289 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-27T10:45:55.289 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-27T10:45:55.289 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-27T10:45:55.290 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-27T10:45:55.290 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-03-27T10:45:55.290 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.inplace_update_support: 0 2024-03-27T10:45:55.290 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-27T10:45:55.290 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-27T10:45:55.290 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-27T10:45:55.290 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-27T10:45:55.290 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.bloom_locality: 0 2024-03-27T10:45:55.291 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.max_successive_merges: 0 2024-03-27T10:45:55.291 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-27T10:45:55.291 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.paranoid_file_checks: 0 2024-03-27T10:45:55.291 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.force_consistency_checks: 1 2024-03-27T10:45:55.291 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.report_bg_io_stats: 0 2024-03-27T10:45:55.291 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.ttl: 2592000 2024-03-27T10:45:55.291 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-27T10:45:55.291 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-27T10:45:55.291 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-27T10:45:55.291 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.enable_blob_files: false 2024-03-27T10:45:55.291 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.min_blob_size: 0 2024-03-27T10:45:55.292 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.blob_file_size: 268435456 2024-03-27T10:45:55.292 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-27T10:45:55.292 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-27T10:45:55.292 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-27T10:45:55.292 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-27T10:45:55.292 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-27T10:45:55.292 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.blob_file_starting_level: 0 2024-03-27T10:45:55.292 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-27T10:45:55.292 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-03-27T10:45:55.292 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-03-27T10:45:55.292 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 3562c6ad-aba9-4f2b-a219-3a529e037e82 2024-03-27T10:45:55.292 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711536354939127, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-03-27T10:45:55.292 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-03-27T10:45:55.293 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711536354940198, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1711536354, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "3562c6ad-aba9-4f2b-a219-3a529e037e82", "db_session_id": "20HANU3BKUG27D5MLYXU", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-03-27T10:45:55.293 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711536354940434, "job": 1, "event": "recovery_finished"} 2024-03-27T10:45:55.293 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-03-27T10:45:55.293 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-27T10:45:55.293 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5631ed794000 2024-03-27T10:45:55.293 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: DB pointer 0x5631ed77e000 2024-03-27T10:45:55.293 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-27T10:45:55.293 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-27T10:45:55.293 INFO:journalctl@ceph.mon.b.smithi114.stdout: ** DB Stats ** 2024-03-27T10:45:55.293 INFO:journalctl@ceph.mon.b.smithi114.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-27T10:45:55.294 INFO:journalctl@ceph.mon.b.smithi114.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-27T10:45:55.294 INFO:journalctl@ceph.mon.b.smithi114.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-27T10:45:55.294 INFO:journalctl@ceph.mon.b.smithi114.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-27T10:45:55.294 INFO:journalctl@ceph.mon.b.smithi114.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-27T10:45:55.294 INFO:journalctl@ceph.mon.b.smithi114.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-27T10:45:55.294 INFO:journalctl@ceph.mon.b.smithi114.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-27T10:45:55.294 INFO:journalctl@ceph.mon.b.smithi114.stdout: 2024-03-27T10:45:55.294 INFO:journalctl@ceph.mon.b.smithi114.stdout: ** Compaction Stats [default] ** 2024-03-27T10:45:55.294 INFO:journalctl@ceph.mon.b.smithi114.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-27T10:45:55.294 INFO:journalctl@ceph.mon.b.smithi114.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-27T10:45:55.294 INFO:journalctl@ceph.mon.b.smithi114.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-27T10:45:55.294 INFO:journalctl@ceph.mon.b.smithi114.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-27T10:45:55.295 INFO:journalctl@ceph.mon.b.smithi114.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-27T10:45:55.295 INFO:journalctl@ceph.mon.b.smithi114.stdout: 2024-03-27T10:45:55.295 INFO:journalctl@ceph.mon.b.smithi114.stdout: ** Compaction Stats [default] ** 2024-03-27T10:45:55.295 INFO:journalctl@ceph.mon.b.smithi114.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-27T10:45:55.295 INFO:journalctl@ceph.mon.b.smithi114.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-27T10:45:55.295 INFO:journalctl@ceph.mon.b.smithi114.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-27T10:45:55.295 INFO:journalctl@ceph.mon.b.smithi114.stdout: 2024-03-27T10:45:55.295 INFO:journalctl@ceph.mon.b.smithi114.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-27T10:45:55.295 INFO:journalctl@ceph.mon.b.smithi114.stdout: 2024-03-27T10:45:55.295 INFO:journalctl@ceph.mon.b.smithi114.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-27T10:45:55.295 INFO:journalctl@ceph.mon.b.smithi114.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-27T10:45:55.295 INFO:journalctl@ceph.mon.b.smithi114.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-27T10:45:55.295 INFO:journalctl@ceph.mon.b.smithi114.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-27T10:45:55.296 INFO:journalctl@ceph.mon.b.smithi114.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-27T10:45:55.296 INFO:journalctl@ceph.mon.b.smithi114.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-27T10:45:55.296 INFO:journalctl@ceph.mon.b.smithi114.stdout: Cumulative compaction: 0.00 GB write, 0.31 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-27T10:45:55.296 INFO:journalctl@ceph.mon.b.smithi114.stdout: Interval compaction: 0.00 GB write, 0.31 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-27T10:45:55.296 INFO:journalctl@ceph.mon.b.smithi114.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-27T10:45:55.296 INFO:journalctl@ceph.mon.b.smithi114.stdout: Block cache BinnedLRUCache@0x5631ed6a5090#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-03-27T10:45:55.296 INFO:journalctl@ceph.mon.b.smithi114.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-03-27T10:45:55.296 INFO:journalctl@ceph.mon.b.smithi114.stdout: 2024-03-27T10:45:55.296 INFO:journalctl@ceph.mon.b.smithi114.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-27T10:45:55.296 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-03-27T10:45:55.296 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: using public_addr v2:172.21.15.114:0/0 -> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] 2024-03-27T10:45:55.296 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: starting mon.b rank -1 at public addrs [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] at bind addrs [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid adca88c4-ec26-11ee-95d0-87774f69a715 2024-03-27T10:45:55.296 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: mon.b@-1(???) e0 preinit fsid adca88c4-ec26-11ee-95d0-87774f69a715 2024-03-27T10:45:55.296 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: mon.b@-1(synchronizing).mds e1 new map 2024-03-27T10:45:55.297 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: mon.b@-1(synchronizing).mds e1 print_map 2024-03-27T10:45:55.297 INFO:journalctl@ceph.mon.b.smithi114.stdout: e1 2024-03-27T10:45:55.297 INFO:journalctl@ceph.mon.b.smithi114.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-27T10:45:55.297 INFO:journalctl@ceph.mon.b.smithi114.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-03-27T10:45:55.297 INFO:journalctl@ceph.mon.b.smithi114.stdout: legacy client fscid: -1 2024-03-27T10:45:55.297 INFO:journalctl@ceph.mon.b.smithi114.stdout: 2024-03-27T10:45:55.297 INFO:journalctl@ceph.mon.b.smithi114.stdout: No filesystems configured 2024-03-27T10:45:55.297 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-03-27T10:45:55.297 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-03-27T10:45:55.297 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-03-27T10:45:55.297 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-03-27T10:45:55.297 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-03-27T10:45:55.297 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-03-27T10:45:55.298 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-27T10:45:55.298 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-27T10:45:55.298 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-27T10:45:55.298 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-27T10:45:55.298 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:45:55.298 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:45:55.298 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:45:55.298 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:45:55.298 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: Added host smithi114 2024-03-27T10:45:55.298 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:45:55.298 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:45:55.298 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:45:55.298 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-27T10:45:55.299 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:45:55.299 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:45:55.299 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:45:55.299 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi177", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T10:45:55.299 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:45:55.299 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:45:55.299 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: Deploying cephadm binary to smithi177 2024-03-27T10:45:55.299 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:45:55.299 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:45:55.299 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:45:55.299 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi114", "name": "osd_memory_target"}]: dispatch 2024-03-27T10:45:55.299 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:45:55.299 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:45:55.299 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: Updating smithi114:/etc/ceph/ceph.conf 2024-03-27T10:45:55.300 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:45:55.300 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: Updating smithi114:/var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-27T10:45:55.300 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:45:55.300 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: Updating smithi114:/etc/ceph/ceph.client.admin.keyring 2024-03-27T10:45:55.300 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:45:55.300 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:45:55.300 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: Added host smithi177 2024-03-27T10:45:55.300 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: Updating smithi114:/var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-27T10:45:55.300 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:45:55.300 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:45:55.300 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:45:55.300 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:45:55.300 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:45:55.301 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:45:55.301 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:45:55.301 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-27T10:45:55.301 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:45:55.301 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:45:55.301 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:45:55.301 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/2574781535' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-27T10:45:55.301 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/2574781535' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-27T10:45:55.301 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: osdmap e4: 0 total, 0 up, 0 in 2024-03-27T10:45:55.301 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:45:55.301 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:45:55.301 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:45:55.301 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:45:55.301 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:45:55.302 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:45:55.302 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi177", "name": "osd_memory_target"}]: dispatch 2024-03-27T10:45:55.302 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:45:55.302 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:45:55.302 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:45:55.302 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: Updating smithi177:/etc/ceph/ceph.conf 2024-03-27T10:45:55.302 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: Updating smithi177:/var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-27T10:45:55.302 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi097:172.21.15.97=a;smithi114:172.21.15.114=b;smithi177:172.21.15.177=c", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T10:45:55.302 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: Saving service mon spec with placement smithi097:172.21.15.97=a;smithi114:172.21.15.114=b;smithi177:172.21.15.177=c;count:3 2024-03-27T10:45:55.302 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: Updating smithi177:/etc/ceph/ceph.client.admin.keyring 2024-03-27T10:45:55.302 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:45:55.302 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:45:55.303 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:45:55.303 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:45:55.303 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-27T10:45:55.303 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:45:55.303 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: Updating smithi177:/var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-27T10:45:55.303 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: Deploying daemon mon.c on smithi177 2024-03-27T10:45:55.303 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:45:55.303 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: from='client.? 172.21.15.177:0/2937455704' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-27T10:45:55.303 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:45:55.303 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:45:55.303 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-27T10:45:55.303 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: mon.a calling monitor election 2024-03-27T10:45:55.303 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-27T10:45:55.304 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: from='client.? 172.21.15.177:0/1798633826' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-27T10:45:55.304 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-27T10:45:55.304 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:45:55.304 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-27T10:45:55.304 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: mon.c calling monitor election 2024-03-27T10:45:55.304 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-27T10:45:55.304 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:45:55.304 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-27T10:45:55.304 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-27T10:45:55.304 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-27T10:45:55.304 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: monmap epoch 2 2024-03-27T10:45:55.304 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: fsid adca88c4-ec26-11ee-95d0-87774f69a715 2024-03-27T10:45:55.304 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: last_changed 2024-03-27T10:45:44.249273+0000 2024-03-27T10:45:55.304 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: created 2024-03-27T10:43:20.535207+0000 2024-03-27T10:45:55.305 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: min_mon_release 19 (squid) 2024-03-27T10:45:55.305 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: election_strategy: 1 2024-03-27T10:45:55.305 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: 0: [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] mon.a 2024-03-27T10:45:55.305 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: 1: [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] mon.c 2024-03-27T10:45:55.305 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: fsmap 2024-03-27T10:45:55.305 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: osdmap e4: 0 total, 0 up, 0 in 2024-03-27T10:45:55.305 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: mgrmap e13: x(active, since 69s) 2024-03-27T10:45:55.305 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: overall HEALTH_OK 2024-03-27T10:45:55.305 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:45:55.305 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:45:55.305 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:45:55.305 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-27T10:45:55.305 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:45:55.306 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: Deploying daemon mon.b on smithi114 2024-03-27T10:45:55.306 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:45:55.306 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-27T10:45:55.306 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:45:55.306 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: from='client.? 172.21.15.177:0/2026675309' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-27T10:45:55.306 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:54 smithi114 ceph-mon[35338]: mon.b@-1(synchronizing).paxosservice(auth 1..4) refresh upgraded, format 0 -> 3 2024-03-27T10:45:55.306 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:55 smithi114 bash[35320]: be7a212c5f9df21ebb1eeae7ac16109ac796a86c64eba1e50b8ce424ae0d0033 2024-03-27T10:45:55.306 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:45:55 smithi114 systemd[1]: Started Ceph mon.b for adca88c4-ec26-11ee-95d0-87774f69a715. 2024-03-27T10:45:55.331 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-27T10:45:55.331 DEBUG:teuthology.orchestra.run.smithi177:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid adca88c4-ec26-11ee-95d0-87774f69a715 -- ceph mon dump -f json 2024-03-27T10:45:55.740 INFO:teuthology.orchestra.run.smithi177.stderr:Inferring config /var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/mon.c/config 2024-03-27T10:46:00.395 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:00 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-27T10:46:00.395 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:00 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-27T10:46:00.395 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:00 smithi097 ceph-mon[31859]: mon.a calling monitor election 2024-03-27T10:46:00.395 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:00 smithi097 ceph-mon[31859]: mon.c calling monitor election 2024-03-27T10:46:00.395 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:00 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-27T10:46:00.395 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:00 smithi097 ceph-mon[31859]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:46:00.395 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:00 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-27T10:46:00.395 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:00 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-27T10:46:00.395 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:00 smithi097 ceph-mon[31859]: mon.b calling monitor election 2024-03-27T10:46:00.395 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:00 smithi097 ceph-mon[31859]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:46:00.396 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:00 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-27T10:46:00.396 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:00 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-27T10:46:00.396 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:00 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-27T10:46:00.396 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:00 smithi097 ceph-mon[31859]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-27T10:46:00.396 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:00 smithi097 ceph-mon[31859]: monmap epoch 3 2024-03-27T10:46:00.396 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:00 smithi097 ceph-mon[31859]: fsid adca88c4-ec26-11ee-95d0-87774f69a715 2024-03-27T10:46:00.396 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:00 smithi097 ceph-mon[31859]: last_changed 2024-03-27T10:45:54.990038+0000 2024-03-27T10:46:00.396 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:00 smithi097 ceph-mon[31859]: created 2024-03-27T10:43:20.535207+0000 2024-03-27T10:46:00.396 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:00 smithi097 ceph-mon[31859]: min_mon_release 19 (squid) 2024-03-27T10:46:00.396 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:00 smithi097 ceph-mon[31859]: election_strategy: 1 2024-03-27T10:46:00.396 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:00 smithi097 ceph-mon[31859]: 0: [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] mon.a 2024-03-27T10:46:00.396 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:00 smithi097 ceph-mon[31859]: 1: [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] mon.c 2024-03-27T10:46:00.396 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:00 smithi097 ceph-mon[31859]: 2: [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] mon.b 2024-03-27T10:46:00.397 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:00 smithi097 ceph-mon[31859]: fsmap 2024-03-27T10:46:00.397 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:00 smithi097 ceph-mon[31859]: osdmap e4: 0 total, 0 up, 0 in 2024-03-27T10:46:00.397 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:00 smithi097 ceph-mon[31859]: mgrmap e13: x(active, since 80s) 2024-03-27T10:46:00.397 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:00 smithi097 ceph-mon[31859]: overall HEALTH_OK 2024-03-27T10:46:00.397 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:00 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:00.397 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:00 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:00.397 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:00 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:00.429 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:00 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-27T10:46:00.429 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:00 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-27T10:46:00.430 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:00 smithi177 ceph-mon[35294]: mon.a calling monitor election 2024-03-27T10:46:00.430 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:00 smithi177 ceph-mon[35294]: mon.c calling monitor election 2024-03-27T10:46:00.430 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:00 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-27T10:46:00.430 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:00 smithi177 ceph-mon[35294]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:46:00.430 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:00 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-27T10:46:00.430 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:00 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-27T10:46:00.430 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:00 smithi177 ceph-mon[35294]: mon.b calling monitor election 2024-03-27T10:46:00.430 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:00 smithi177 ceph-mon[35294]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:46:00.430 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:00 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-27T10:46:00.431 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:00 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-27T10:46:00.431 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:00 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-27T10:46:00.431 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:00 smithi177 ceph-mon[35294]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-27T10:46:00.431 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:00 smithi177 ceph-mon[35294]: monmap epoch 3 2024-03-27T10:46:00.431 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:00 smithi177 ceph-mon[35294]: fsid adca88c4-ec26-11ee-95d0-87774f69a715 2024-03-27T10:46:00.431 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:00 smithi177 ceph-mon[35294]: last_changed 2024-03-27T10:45:54.990038+0000 2024-03-27T10:46:00.431 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:00 smithi177 ceph-mon[35294]: created 2024-03-27T10:43:20.535207+0000 2024-03-27T10:46:00.431 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:00 smithi177 ceph-mon[35294]: min_mon_release 19 (squid) 2024-03-27T10:46:00.431 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:00 smithi177 ceph-mon[35294]: election_strategy: 1 2024-03-27T10:46:00.431 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:00 smithi177 ceph-mon[35294]: 0: [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] mon.a 2024-03-27T10:46:00.431 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:00 smithi177 ceph-mon[35294]: 1: [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] mon.c 2024-03-27T10:46:00.431 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:00 smithi177 ceph-mon[35294]: 2: [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] mon.b 2024-03-27T10:46:00.431 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:00 smithi177 ceph-mon[35294]: fsmap 2024-03-27T10:46:00.432 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:00 smithi177 ceph-mon[35294]: osdmap e4: 0 total, 0 up, 0 in 2024-03-27T10:46:00.432 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:00 smithi177 ceph-mon[35294]: mgrmap e13: x(active, since 80s) 2024-03-27T10:46:00.432 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:00 smithi177 ceph-mon[35294]: overall HEALTH_OK 2024-03-27T10:46:00.432 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:00 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:00.432 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:00 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:00.432 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:00 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:00.521 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:00 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-27T10:46:00.521 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:00 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-27T10:46:00.521 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:00 smithi114 ceph-mon[35338]: mon.a calling monitor election 2024-03-27T10:46:00.521 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:00 smithi114 ceph-mon[35338]: mon.c calling monitor election 2024-03-27T10:46:00.521 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:00 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-27T10:46:00.521 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:00 smithi114 ceph-mon[35338]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:46:00.521 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:00 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-27T10:46:00.521 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:00 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-27T10:46:00.522 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:00 smithi114 ceph-mon[35338]: mon.b calling monitor election 2024-03-27T10:46:00.522 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:00 smithi114 ceph-mon[35338]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:46:00.522 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:00 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-27T10:46:00.522 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:00 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-27T10:46:00.522 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:00 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-27T10:46:00.522 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:00 smithi114 ceph-mon[35338]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-27T10:46:00.522 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:00 smithi114 ceph-mon[35338]: monmap epoch 3 2024-03-27T10:46:00.522 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:00 smithi114 ceph-mon[35338]: fsid adca88c4-ec26-11ee-95d0-87774f69a715 2024-03-27T10:46:00.522 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:00 smithi114 ceph-mon[35338]: last_changed 2024-03-27T10:45:54.990038+0000 2024-03-27T10:46:00.522 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:00 smithi114 ceph-mon[35338]: created 2024-03-27T10:43:20.535207+0000 2024-03-27T10:46:00.522 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:00 smithi114 ceph-mon[35338]: min_mon_release 19 (squid) 2024-03-27T10:46:00.522 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:00 smithi114 ceph-mon[35338]: election_strategy: 1 2024-03-27T10:46:00.523 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:00 smithi114 ceph-mon[35338]: 0: [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] mon.a 2024-03-27T10:46:00.523 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:00 smithi114 ceph-mon[35338]: 1: [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] mon.c 2024-03-27T10:46:00.523 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:00 smithi114 ceph-mon[35338]: 2: [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] mon.b 2024-03-27T10:46:00.523 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:00 smithi114 ceph-mon[35338]: fsmap 2024-03-27T10:46:00.523 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:00 smithi114 ceph-mon[35338]: osdmap e4: 0 total, 0 up, 0 in 2024-03-27T10:46:00.523 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:00 smithi114 ceph-mon[35338]: mgrmap e13: x(active, since 80s) 2024-03-27T10:46:00.523 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:00 smithi114 ceph-mon[35338]: overall HEALTH_OK 2024-03-27T10:46:00.523 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:00 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:00.523 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:00 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:00.523 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:00 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:00.756 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:00.780+0000 7f23c8200700 1 -- 172.21.15.177:0/3675538954 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23c0104580 msgr2=0x7f23a8005610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:46:00.756 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:00.780+0000 7f23c8200700 1 --2- 172.21.15.177:0/3675538954 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23c0104580 0x7f23a8005610 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f23b4005fa0 tx=0x7f23b400ffc0 comp rx=0 tx=0).stop 2024-03-27T10:46:00.756 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:00.780+0000 7f23c8200700 1 -- 172.21.15.177:0/3675538954 shutdown_connections 2024-03-27T10:46:00.756 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:00.780+0000 7f23c8200700 1 --2- 172.21.15.177:0/3675538954 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23c0104580 0x7f23a8005610 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:46:00.756 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:00.780+0000 7f23c8200700 1 -- 172.21.15.177:0/3675538954 >> 172.21.15.177:0/3675538954 conn(0x7f23c00ffe30 msgr2=0x7f23c0102250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:46:00.756 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:00.780+0000 7f23c8200700 1 -- 172.21.15.177:0/3675538954 shutdown_connections 2024-03-27T10:46:00.756 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:00.780+0000 7f23c8200700 1 -- 172.21.15.177:0/3675538954 wait complete. 2024-03-27T10:46:00.756 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:00.782+0000 7f23c8200700 1 Processor -- start 2024-03-27T10:46:00.756 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:00.782+0000 7f23c8200700 1 -- start start 2024-03-27T10:46:00.758 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:00.782+0000 7f23c8200700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f23c0104580 0x7f23c0199df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:46:00.758 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:00.782+0000 7f23c8200700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f23c019a330 0x7f23c019e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:46:00.758 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:00.782+0000 7f23c8200700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23c019ec00 0x7f23c019f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:46:00.758 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:00.782+0000 7f23c8200700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f23c010c210 con 0x7f23c019ec00 2024-03-27T10:46:00.758 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:00.782+0000 7f23c8200700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f23c010c090 con 0x7f23c019a330 2024-03-27T10:46:00.758 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:00.782+0000 7f23c8200700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f23c010c390 con 0x7f23c0104580 2024-03-27T10:46:00.758 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:00.783+0000 7f23c6200700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23c019ec00 0x7f23c019f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:46:00.758 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:00.783+0000 7f23c6200700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23c019ec00 0x7f23c019f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.177:41400/0 (socket says 172.21.15.177:41400) 2024-03-27T10:46:00.758 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:00.783+0000 7f23c6200700 1 -- 172.21.15.177:0/571030660 learned_addr learned my addr 172.21.15.177:0/571030660 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-03-27T10:46:00.759 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:00.784+0000 7f23c5800700 1 --2- 172.21.15.177:0/571030660 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f23c0104580 0x7f23c0199df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:46:00.759 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:00.784+0000 7f23c6200700 1 -- 172.21.15.177:0/571030660 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f23c0104580 msgr2=0x7f23c0199df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:46:00.759 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:00.784+0000 7f23c6200700 1 --2- 172.21.15.177:0/571030660 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f23c0104580 0x7f23c0199df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:46:00.759 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:00.784+0000 7f23c6200700 1 -- 172.21.15.177:0/571030660 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f23c019a330 msgr2=0x7f23c019e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:46:00.759 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:00.784+0000 7f23c6200700 1 --2- 172.21.15.177:0/571030660 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f23c019a330 0x7f23c019e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:46:00.759 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:00.784+0000 7f23c6200700 1 -- 172.21.15.177:0/571030660 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23c01a6970 con 0x7f23c019ec00 2024-03-27T10:46:00.760 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:00.784+0000 7f23c6200700 1 --2- 172.21.15.177:0/571030660 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23c019ec00 0x7f23c019f0b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f23b800ba50 tx=0x7f23b800be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:46:00.760 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:00.785+0000 7f23be000700 1 -- 172.21.15.177:0/571030660 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23b8019070 con 0x7f23c019ec00 2024-03-27T10:46:00.760 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:00.785+0000 7f23be000700 1 -- 172.21.15.177:0/571030660 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f23b8014440 con 0x7f23c019ec00 2024-03-27T10:46:00.760 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:00.785+0000 7f23c8200700 1 -- 172.21.15.177:0/571030660 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23c01a6ba0 con 0x7f23c019ec00 2024-03-27T10:46:00.760 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:00.785+0000 7f23be000700 1 -- 172.21.15.177:0/571030660 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23b8007590 con 0x7f23c019ec00 2024-03-27T10:46:00.760 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:00.785+0000 7f23c8200700 1 -- 172.21.15.177:0/571030660 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23c01a70e0 con 0x7f23c019ec00 2024-03-27T10:46:00.761 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:00.787+0000 7f23c8200700 1 -- 172.21.15.177:0/571030660 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23c0063c70 con 0x7f23c019ec00 2024-03-27T10:46:00.761 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:00.787+0000 7f23be000700 1 -- 172.21.15.177:0/571030660 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) ==== 46522+0+0 (secure 0 0 0) 0x7f23b801c070 con 0x7f23c019ec00 2024-03-27T10:46:00.761 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:00.787+0000 7f23be000700 1 --2- 172.21.15.177:0/571030660 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f23a0039900 0x7f23a003bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:46:00.761 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:00.787+0000 7f23be000700 1 -- 172.21.15.177:0/571030660 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f23b804bc90 con 0x7f23c019ec00 2024-03-27T10:46:00.761 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:00.788+0000 7f23c5800700 1 --2- 172.21.15.177:0/571030660 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f23a0039900 0x7f23a003bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:46:00.762 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:00.789+0000 7f23c5800700 1 --2- 172.21.15.177:0/571030660 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f23a0039900 0x7f23a003bdc0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f23b4005b40 tx=0x7f23b401d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:46:00.773 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:00.799+0000 7f23be000700 1 -- 172.21.15.177:0/571030660 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f23b8014980 con 0x7f23c019ec00 2024-03-27T10:46:01.099 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:01.125+0000 7f23c8200700 1 -- 172.21.15.177:0/571030660 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f23c002def0 con 0x7f23c019ec00 2024-03-27T10:46:01.100 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:01.126+0000 7f23be000700 1 -- 172.21.15.177:0/571030660 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1285 (secure 0 0 0) 0x7f23b800e090 con 0x7f23c019ec00 2024-03-27T10:46:01.100 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-03-27T10:46:01.100 INFO:teuthology.orchestra.run.smithi177.stdout:{"epoch":3,"fsid":"adca88c4-ec26-11ee-95d0-87774f69a715","modified":"2024-03-27T10:45:54.990038Z","created":"2024-03-27T10:43:20.535207Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:3300","nonce":0},{"type":"v1","addr":"172.21.15.97:6789","nonce":0}]},"addr":"172.21.15.97:6789/0","public_addr":"172.21.15.97:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:3300","nonce":0},{"type":"v1","addr":"172.21.15.177:6789","nonce":0}]},"addr":"172.21.15.177:6789/0","public_addr":"172.21.15.177:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:3300","nonce":0},{"type":"v1","addr":"172.21.15.114:6789","nonce":0}]},"addr":"172.21.15.114:6789/0","public_addr":"172.21.15.114:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-03-27T10:46:01.103 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:01.129+0000 7f23c8200700 1 -- 172.21.15.177:0/571030660 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f23a0039900 msgr2=0x7f23a003bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:46:01.103 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:01.129+0000 7f23c8200700 1 --2- 172.21.15.177:0/571030660 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f23a0039900 0x7f23a003bdc0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f23b4005b40 tx=0x7f23b401d040 comp rx=0 tx=0).stop 2024-03-27T10:46:01.103 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:01.129+0000 7f23c8200700 1 -- 172.21.15.177:0/571030660 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23c019ec00 msgr2=0x7f23c019f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:46:01.103 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:01.129+0000 7f23c8200700 1 --2- 172.21.15.177:0/571030660 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23c019ec00 0x7f23c019f0b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f23b800ba50 tx=0x7f23b800be10 comp rx=0 tx=0).stop 2024-03-27T10:46:01.103 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:01.129+0000 7f23c8200700 1 -- 172.21.15.177:0/571030660 shutdown_connections 2024-03-27T10:46:01.103 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:01.129+0000 7f23c8200700 1 --2- 172.21.15.177:0/571030660 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f23a0039900 0x7f23a003bdc0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:46:01.104 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:01.129+0000 7f23c8200700 1 --2- 172.21.15.177:0/571030660 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23c019ec00 0x7f23c019f0b0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:46:01.104 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:01.129+0000 7f23c8200700 1 --2- 172.21.15.177:0/571030660 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f23c0104580 0x7f23c0199df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:46:01.104 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:01.129+0000 7f23c8200700 1 --2- 172.21.15.177:0/571030660 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f23c019a330 0x7f23c019e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:46:01.104 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:01.129+0000 7f23c8200700 1 -- 172.21.15.177:0/571030660 >> 172.21.15.177:0/571030660 conn(0x7f23c00ffe30 msgr2=0x7f23c01010e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:46:01.104 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:01.129+0000 7f23c8200700 1 -- 172.21.15.177:0/571030660 shutdown_connections 2024-03-27T10:46:01.104 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:01.129+0000 7f23c8200700 1 -- 172.21.15.177:0/571030660 wait complete. 2024-03-27T10:46:01.104 INFO:teuthology.orchestra.run.smithi177.stderr:dumped monmap epoch 3 2024-03-27T10:46:01.225 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:01 smithi114 ceph-mon[35338]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:46:01.225 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:01 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:01.225 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:01 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:46:01.225 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:01 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-27T10:46:01.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:01 smithi097 ceph-mon[31859]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:46:01.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:01 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:01.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:01 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:46:01.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:01 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-27T10:46:01.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:01 smithi177 ceph-mon[35294]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:46:01.429 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:01 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:01.429 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:01 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:46:01.429 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:01 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-27T10:46:01.843 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-03-27T10:46:01.843 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid adca88c4-ec26-11ee-95d0-87774f69a715 -- ceph config generate-minimal-conf 2024-03-27T10:46:02.280 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T10:46:02.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:02 smithi097 ceph-mon[31859]: from='client.? 172.21.15.177:0/571030660' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-27T10:46:02.429 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:02 smithi177 ceph-mon[35294]: from='client.? 172.21.15.177:0/571030660' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-27T10:46:02.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:02 smithi114 ceph-mon[35338]: from='client.? 172.21.15.177:0/571030660' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-27T10:46:03.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:03 smithi097 ceph-mon[31859]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:46:03.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:03 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:03.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:03 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:03.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:03 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:03.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:03 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:03.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:03 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:46:03.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:03 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:46:03.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:03 smithi177 ceph-mon[35294]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:46:03.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:03 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:03.429 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:03 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:03.429 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:03 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:03.429 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:03 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:03.429 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:03 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:46:03.429 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:03 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:46:03.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:03 smithi114 ceph-mon[35338]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:46:03.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:03 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:03.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:03 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:03.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:03 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:03.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:03 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:03.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:03 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:46:03.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:03 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:46:04.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:04.236+0000 7ff9a3200700 1 -- 172.21.15.97:0/1348569536 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9a4074a70 msgr2=0x7ff9a4074e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:46:04.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:04.236+0000 7ff9a3200700 1 --2- 172.21.15.97:0/1348569536 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9a4074a70 0x7ff9a4074e50 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff994009910 tx=0x7ff994009c20 comp rx=0 tx=0).stop 2024-03-27T10:46:04.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:04.237+0000 7ff9a3200700 1 -- 172.21.15.97:0/1348569536 shutdown_connections 2024-03-27T10:46:04.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:04.237+0000 7ff9a3200700 1 --2- 172.21.15.97:0/1348569536 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9a4074a70 0x7ff9a4074e50 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:46:04.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:04.237+0000 7ff9a3200700 1 -- 172.21.15.97:0/1348569536 >> 172.21.15.97:0/1348569536 conn(0x7ff9a40fd970 msgr2=0x7ff9a40ffd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:46:04.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:04.237+0000 7ff9a3200700 1 -- 172.21.15.97:0/1348569536 shutdown_connections 2024-03-27T10:46:04.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:04.237+0000 7ff9a3200700 1 -- 172.21.15.97:0/1348569536 wait complete. 2024-03-27T10:46:04.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:04.238+0000 7ff9a3200700 1 Processor -- start 2024-03-27T10:46:04.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:04.238+0000 7ff9a3200700 1 -- start start 2024-03-27T10:46:04.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:04.238+0000 7ff9a3200700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff9a4074a70 0x7ff9a4199db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:46:04.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:04.239+0000 7ff9a3200700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9a419a2f0 0x7ff9a419e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:46:04.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:04.239+0000 7ff9a3200700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff9a419ebc0 0x7ff9a419f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:46:04.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:04.239+0000 7ff9a3200700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a410e5c0 con 0x7ff9a419a2f0 2024-03-27T10:46:04.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:04.239+0000 7ff9a3200700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a410e440 con 0x7ff9a419ebc0 2024-03-27T10:46:04.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:04.239+0000 7ff9a3200700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a410e740 con 0x7ff9a4074a70 2024-03-27T10:46:04.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:04.239+0000 7ff99be00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9a419a2f0 0x7ff9a419e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:46:04.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:04.239+0000 7ff99be00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9a419a2f0 0x7ff9a419e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55356/0 (socket says 172.21.15.97:55356) 2024-03-27T10:46:04.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:04.239+0000 7ff99be00700 1 -- 172.21.15.97:0/1041251462 learned_addr learned my addr 172.21.15.97:0/1041251462 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:46:04.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:04.239+0000 7ff9a1e00700 1 --2- 172.21.15.97:0/1041251462 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff9a4074a70 0x7ff9a4199db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:46:04.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:04.239+0000 7ff9a2800700 1 --2- 172.21.15.97:0/1041251462 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff9a419ebc0 0x7ff9a419f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:46:04.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:04.239+0000 7ff99be00700 1 -- 172.21.15.97:0/1041251462 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff9a4074a70 msgr2=0x7ff9a4199db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:46:04.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:04.239+0000 7ff99be00700 1 --2- 172.21.15.97:0/1041251462 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff9a4074a70 0x7ff9a4199db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:46:04.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:04.239+0000 7ff99be00700 1 -- 172.21.15.97:0/1041251462 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff9a419ebc0 msgr2=0x7ff9a419f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:46:04.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:04.239+0000 7ff99be00700 1 --2- 172.21.15.97:0/1041251462 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff9a419ebc0 0x7ff9a419f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:46:04.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:04.239+0000 7ff99be00700 1 -- 172.21.15.97:0/1041251462 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9a41a6930 con 0x7ff9a419a2f0 2024-03-27T10:46:04.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:04.239+0000 7ff9a2800700 1 --2- 172.21.15.97:0/1041251462 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff9a419ebc0 0x7ff9a419f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:46:04.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:04.240+0000 7ff99be00700 1 --2- 172.21.15.97:0/1041251462 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9a419a2f0 0x7ff9a419e680 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff98c00b780 tx=0x7ff98c00ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:46:04.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:04.240+0000 7ff99aa00700 1 -- 172.21.15.97:0/1041251462 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff98c004230 con 0x7ff9a419a2f0 2024-03-27T10:46:04.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:04.240+0000 7ff99aa00700 1 -- 172.21.15.97:0/1041251462 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff98c0043d0 con 0x7ff9a419a2f0 2024-03-27T10:46:04.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:04.240+0000 7ff9a3200700 1 -- 172.21.15.97:0/1041251462 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9a41a6b60 con 0x7ff9a419a2f0 2024-03-27T10:46:04.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:04.240+0000 7ff99aa00700 1 -- 172.21.15.97:0/1041251462 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff98c011b60 con 0x7ff9a419a2f0 2024-03-27T10:46:04.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:04.240+0000 7ff9a3200700 1 -- 172.21.15.97:0/1041251462 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9a41a70f0 con 0x7ff9a419a2f0 2024-03-27T10:46:04.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:04.241+0000 7ff99aa00700 1 -- 172.21.15.97:0/1041251462 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) ==== 46522+0+0 (secure 0 0 0) 0x7ff98c011d00 con 0x7ff9a419a2f0 2024-03-27T10:46:04.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:04.241+0000 7ff9a3200700 1 -- 172.21.15.97:0/1041251462 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9a4063c70 con 0x7ff9a419a2f0 2024-03-27T10:46:04.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:04.242+0000 7ff99aa00700 1 --2- 172.21.15.97:0/1041251462 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7ff988039900 0x7ff98803bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:46:04.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:04.242+0000 7ff99aa00700 1 -- 172.21.15.97:0/1041251462 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7ff98c04c1f0 con 0x7ff9a419a2f0 2024-03-27T10:46:04.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:04.242+0000 7ff9a1e00700 1 --2- 172.21.15.97:0/1041251462 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7ff988039900 0x7ff98803bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:46:04.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:04.243+0000 7ff9a1e00700 1 --2- 172.21.15.97:0/1041251462 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7ff988039900 0x7ff98803bdc0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff994006010 tx=0x7ff99401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:46:04.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:04.247+0000 7ff99aa00700 1 -- 172.21.15.97:0/1041251462 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7ff98c0204e0 con 0x7ff9a419a2f0 2024-03-27T10:46:04.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:04 smithi097 ceph-mon[31859]: Updating smithi097:/etc/ceph/ceph.conf 2024-03-27T10:46:04.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:04 smithi097 ceph-mon[31859]: Updating smithi114:/etc/ceph/ceph.conf 2024-03-27T10:46:04.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:04 smithi097 ceph-mon[31859]: Updating smithi177:/etc/ceph/ceph.conf 2024-03-27T10:46:04.429 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:04 smithi177 ceph-mon[35294]: Updating smithi097:/etc/ceph/ceph.conf 2024-03-27T10:46:04.429 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:04 smithi177 ceph-mon[35294]: Updating smithi114:/etc/ceph/ceph.conf 2024-03-27T10:46:04.429 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:04 smithi177 ceph-mon[35294]: Updating smithi177:/etc/ceph/ceph.conf 2024-03-27T10:46:04.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:04.479+0000 7ff9a3200700 1 -- 172.21.15.97:0/1041251462 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7ff9a41a73f0 con 0x7ff9a419a2f0 2024-03-27T10:46:04.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:04.480+0000 7ff99aa00700 1 -- 172.21.15.97:0/1041251462 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+275 (secure 0 0 0) 0x7ff98c01a780 con 0x7ff9a419a2f0 2024-03-27T10:46:04.454 INFO:teuthology.orchestra.run.smithi097.stdout:# minimal ceph.conf for adca88c4-ec26-11ee-95d0-87774f69a715 2024-03-27T10:46:04.454 INFO:teuthology.orchestra.run.smithi097.stdout:[global] 2024-03-27T10:46:04.454 INFO:teuthology.orchestra.run.smithi097.stdout: fsid = adca88c4-ec26-11ee-95d0-87774f69a715 2024-03-27T10:46:04.454 INFO:teuthology.orchestra.run.smithi097.stdout: mon_host = [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] 2024-03-27T10:46:04.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:04.483+0000 7ff9a3200700 1 -- 172.21.15.97:0/1041251462 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7ff988039900 msgr2=0x7ff98803bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:46:04.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:04.483+0000 7ff9a3200700 1 --2- 172.21.15.97:0/1041251462 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7ff988039900 0x7ff98803bdc0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff994006010 tx=0x7ff99401a040 comp rx=0 tx=0).stop 2024-03-27T10:46:04.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:04.483+0000 7ff9a3200700 1 -- 172.21.15.97:0/1041251462 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9a419a2f0 msgr2=0x7ff9a419e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:46:04.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:04.483+0000 7ff9a3200700 1 --2- 172.21.15.97:0/1041251462 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9a419a2f0 0x7ff9a419e680 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff98c00b780 tx=0x7ff98c00ba90 comp rx=0 tx=0).stop 2024-03-27T10:46:04.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:04.483+0000 7ff9a3200700 1 -- 172.21.15.97:0/1041251462 shutdown_connections 2024-03-27T10:46:04.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:04.483+0000 7ff9a3200700 1 --2- 172.21.15.97:0/1041251462 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7ff988039900 0x7ff98803bdc0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:46:04.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:04.483+0000 7ff9a3200700 1 --2- 172.21.15.97:0/1041251462 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff9a419ebc0 0x7ff9a419f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:46:04.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:04.483+0000 7ff9a3200700 1 --2- 172.21.15.97:0/1041251462 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff9a4074a70 0x7ff9a4199db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:46:04.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:04.483+0000 7ff9a3200700 1 --2- 172.21.15.97:0/1041251462 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9a419a2f0 0x7ff9a419e680 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:46:04.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:04.483+0000 7ff9a3200700 1 -- 172.21.15.97:0/1041251462 >> 172.21.15.97:0/1041251462 conn(0x7ff9a40fd970 msgr2=0x7ff9a4108110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:46:04.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:04.483+0000 7ff9a3200700 1 -- 172.21.15.97:0/1041251462 shutdown_connections 2024-03-27T10:46:04.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:04.483+0000 7ff9a3200700 1 -- 172.21.15.97:0/1041251462 wait complete. 2024-03-27T10:46:04.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:04 smithi114 ceph-mon[35338]: Updating smithi097:/etc/ceph/ceph.conf 2024-03-27T10:46:04.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:04 smithi114 ceph-mon[35338]: Updating smithi114:/etc/ceph/ceph.conf 2024-03-27T10:46:04.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:04 smithi114 ceph-mon[35338]: Updating smithi177:/etc/ceph/ceph.conf 2024-03-27T10:46:05.271 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-03-27T10:46:05.271 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-27T10:46:05.271 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/ceph/ceph.conf 2024-03-27T10:46:05.363 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-27T10:46:05.363 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-27T10:46:05.394 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:05 smithi097 ceph-mon[31859]: Updating smithi097:/var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-27T10:46:05.394 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:05 smithi097 ceph-mon[31859]: Updating smithi177:/var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-27T10:46:05.394 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:05 smithi097 ceph-mon[31859]: Updating smithi114:/var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-27T10:46:05.394 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:05 smithi097 ceph-mon[31859]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:46:05.394 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:05 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:05.394 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:05 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:05.394 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:05 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:05.394 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:05 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:05.394 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:05 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:05.394 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:05 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:05.395 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:05 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:05.395 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:05 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/1041251462' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:46:05.395 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:05 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:05.395 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:05 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:05.395 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:05 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:05.395 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:05 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:05.395 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:05 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-27T10:46:05.395 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:05 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-27T10:46:05.395 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:05 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:46:05.395 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:05 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:05.429 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:05 smithi177 ceph-mon[35294]: Updating smithi097:/var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-27T10:46:05.429 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:05 smithi177 ceph-mon[35294]: Updating smithi177:/var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-27T10:46:05.429 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:05 smithi177 ceph-mon[35294]: Updating smithi114:/var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-27T10:46:05.430 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:05 smithi177 ceph-mon[35294]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:46:05.430 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:05 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:05.430 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:05 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:05.430 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:05 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:05.430 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:05 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:05.430 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:05 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:05.430 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:05 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:05.430 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:05 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:05.430 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:05 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/1041251462' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:46:05.430 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:05 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:05.430 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:05 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:05.430 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:05 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:05.430 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:05 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:05.430 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:05 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-27T10:46:05.431 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:05 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-27T10:46:05.431 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:05 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:46:05.431 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:05 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:05.481 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-03-27T10:46:05.481 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/etc/ceph/ceph.conf 2024-03-27T10:46:05.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:05 smithi114 ceph-mon[35338]: Updating smithi097:/var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-27T10:46:05.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:05 smithi114 ceph-mon[35338]: Updating smithi177:/var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-27T10:46:05.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:05 smithi114 ceph-mon[35338]: Updating smithi114:/var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-27T10:46:05.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:05 smithi114 ceph-mon[35338]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:46:05.520 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:05 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:05.520 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:05 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:05.520 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:05 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:05.520 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:05 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:05.520 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:05 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:05.520 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:05 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:05.520 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:05 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:05.520 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:05 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/1041251462' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:46:05.520 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:05 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:05.520 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:05 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:05.520 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:05 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:05.520 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:05 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:05.520 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:05 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-27T10:46:05.520 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:05 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-27T10:46:05.521 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:05 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:46:05.521 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:05 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:05.581 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-03-27T10:46:05.581 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-27T10:46:05.721 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-03-27T10:46:05.721 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/etc/ceph/ceph.conf 2024-03-27T10:46:05.819 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-03-27T10:46:05.820 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-27T10:46:05.957 INFO:tasks.cephadm:Adding mgr.x on smithi097 2024-03-27T10:46:05.957 INFO:tasks.cephadm:Adding mgr.y on smithi114 2024-03-27T10:46:05.957 INFO:tasks.cephadm:Adding mgr.z on smithi177 2024-03-27T10:46:05.957 DEBUG:teuthology.orchestra.run.smithi177:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid adca88c4-ec26-11ee-95d0-87774f69a715 -- ceph orch apply mgr '3;smithi097=x;smithi114=y;smithi177=z' 2024-03-27T10:46:06.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:06 smithi097 ceph-mon[31859]: Reconfiguring mon.a (unknown last config time)... 2024-03-27T10:46:06.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:06 smithi097 ceph-mon[31859]: Reconfiguring daemon mon.a on smithi097 2024-03-27T10:46:06.417 INFO:teuthology.orchestra.run.smithi177.stderr:Inferring config /var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/mon.c/config 2024-03-27T10:46:06.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:06 smithi177 ceph-mon[35294]: Reconfiguring mon.a (unknown last config time)... 2024-03-27T10:46:06.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:06 smithi177 ceph-mon[35294]: Reconfiguring daemon mon.a on smithi097 2024-03-27T10:46:06.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:06 smithi114 ceph-mon[35338]: Reconfiguring mon.a (unknown last config time)... 2024-03-27T10:46:06.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:06 smithi114 ceph-mon[35338]: Reconfiguring daemon mon.a on smithi097 2024-03-27T10:46:07.429 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:07 smithi177 ceph-mon[35294]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:46:07.429 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:07 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:07.429 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:07 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:07.429 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:07 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-27T10:46:07.429 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:07 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-27T10:46:07.429 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:07 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:46:07.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:07 smithi114 ceph-mon[35338]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:46:07.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:07 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:07.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:07 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:07.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:07 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-27T10:46:07.520 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:07 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-27T10:46:07.520 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:07 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:46:07.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:07 smithi097 ceph-mon[31859]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:46:07.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:07 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:07.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:07 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:07.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:07 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-27T10:46:07.644 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:07 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-27T10:46:07.644 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:07 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:46:08.406 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:08 smithi177 ceph-mon[35294]: Reconfiguring mon.b (monmap changed)... 2024-03-27T10:46:08.406 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:08 smithi177 ceph-mon[35294]: Reconfiguring daemon mon.b on smithi114 2024-03-27T10:46:08.494 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:08 smithi114 ceph-mon[35338]: Reconfiguring mon.b (monmap changed)... 2024-03-27T10:46:08.494 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:08 smithi114 ceph-mon[35338]: Reconfiguring daemon mon.b on smithi114 2024-03-27T10:46:08.574 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:08.599+0000 7feb69c00700 1 -- 172.21.15.177:0/4200230104 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb64102170 msgr2=0x7feb64102590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:46:08.574 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:08.599+0000 7feb69c00700 1 --2- 172.21.15.177:0/4200230104 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb64102170 0x7feb64102590 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7feb54009910 tx=0x7feb54009c20 comp rx=0 tx=0).stop 2024-03-27T10:46:08.574 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:08.599+0000 7feb69c00700 1 -- 172.21.15.177:0/4200230104 shutdown_connections 2024-03-27T10:46:08.574 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:08.599+0000 7feb69c00700 1 --2- 172.21.15.177:0/4200230104 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb64102170 0x7feb64102590 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:46:08.574 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:08.600+0000 7feb69c00700 1 -- 172.21.15.177:0/4200230104 >> 172.21.15.177:0/4200230104 conn(0x7feb640fd710 msgr2=0x7feb640ffb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:46:08.574 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:08.600+0000 7feb69c00700 1 -- 172.21.15.177:0/4200230104 shutdown_connections 2024-03-27T10:46:08.574 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:08.600+0000 7feb69c00700 1 -- 172.21.15.177:0/4200230104 wait complete. 2024-03-27T10:46:08.575 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:08.601+0000 7feb69c00700 1 Processor -- start 2024-03-27T10:46:08.575 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:08.601+0000 7feb69c00700 1 -- start start 2024-03-27T10:46:08.575 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:08.601+0000 7feb69c00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb64102170 0x7feb64199b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:46:08.576 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:08.601+0000 7feb69c00700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feb6419a040 0x7feb6419f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:46:08.576 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:08.601+0000 7feb69c00700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feb6419a4c0 0x7feb641a1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:46:08.576 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:08.601+0000 7feb69c00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7feb641080b0 con 0x7feb64102170 2024-03-27T10:46:08.576 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:08.601+0000 7feb69c00700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7feb64107f30 con 0x7feb6419a4c0 2024-03-27T10:46:08.576 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:08.601+0000 7feb69c00700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7feb64108230 con 0x7feb6419a040 2024-03-27T10:46:08.576 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:08.602+0000 7feb5a000700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feb6419a040 0x7feb6419f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:46:08.576 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:08.602+0000 7feb5a000700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feb6419a040 0x7feb6419f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:33494/0 (socket says 172.21.15.177:33494) 2024-03-27T10:46:08.576 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:08.602+0000 7feb5a000700 1 -- 172.21.15.177:0/1582610979 learned_addr learned my addr 172.21.15.177:0/1582610979 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-03-27T10:46:08.576 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:08.602+0000 7feb62a00700 1 --2- 172.21.15.177:0/1582610979 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb64102170 0x7feb64199b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:46:08.577 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:08.602+0000 7feb5a000700 1 -- 172.21.15.177:0/1582610979 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feb6419a4c0 msgr2=0x7feb641a1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:46:08.577 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:08.602+0000 7feb63400700 1 --2- 172.21.15.177:0/1582610979 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feb6419a4c0 0x7feb641a1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:46:08.577 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:08.602+0000 7feb5a000700 1 --2- 172.21.15.177:0/1582610979 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feb6419a4c0 0x7feb641a1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:46:08.577 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:08.602+0000 7feb5a000700 1 -- 172.21.15.177:0/1582610979 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb64102170 msgr2=0x7feb64199b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:46:08.577 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:08.602+0000 7feb5a000700 1 --2- 172.21.15.177:0/1582610979 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb64102170 0x7feb64199b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:46:08.578 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:08.602+0000 7feb5a000700 1 -- 172.21.15.177:0/1582610979 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb641a1f10 con 0x7feb6419a040 2024-03-27T10:46:08.578 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:08.602+0000 7feb63400700 1 --2- 172.21.15.177:0/1582610979 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feb6419a4c0 0x7feb641a1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:46:08.578 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:08.602+0000 7feb5a000700 1 --2- 172.21.15.177:0/1582610979 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feb6419a040 0x7feb6419f0b0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7feb4c00b780 tx=0x7feb4c00bb40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:46:08.578 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:08.603+0000 7feb5be00700 1 -- 172.21.15.177:0/1582610979 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb4c004230 con 0x7feb6419a040 2024-03-27T10:46:08.578 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:08.603+0000 7feb5be00700 1 -- 172.21.15.177:0/1582610979 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb4c0043d0 con 0x7feb6419a040 2024-03-27T10:46:08.578 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:08.603+0000 7feb69c00700 1 -- 172.21.15.177:0/1582610979 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb641a2200 con 0x7feb6419a040 2024-03-27T10:46:08.578 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:08.603+0000 7feb5be00700 1 -- 172.21.15.177:0/1582610979 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb4c011b80 con 0x7feb6419a040 2024-03-27T10:46:08.578 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:08.603+0000 7feb69c00700 1 -- 172.21.15.177:0/1582610979 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb641a2810 con 0x7feb6419a040 2024-03-27T10:46:08.579 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:08.604+0000 7feb69c00700 1 -- 172.21.15.177:0/1582610979 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb64063c70 con 0x7feb6419a040 2024-03-27T10:46:08.579 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:08.604+0000 7feb5be00700 1 -- 172.21.15.177:0/1582610979 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 13) ==== 46522+0+0 (secure 0 0 0) 0x7feb4c0036a0 con 0x7feb6419a040 2024-03-27T10:46:08.579 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:08.605+0000 7feb5be00700 1 --2- 172.21.15.177:0/1582610979 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7feb48039900 0x7feb4803bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:46:08.579 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:08.605+0000 7feb5be00700 1 -- 172.21.15.177:0/1582610979 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7feb4c04c2c0 con 0x7feb6419a040 2024-03-27T10:46:08.579 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:08.605+0000 7feb62a00700 1 --2- 172.21.15.177:0/1582610979 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7feb48039900 0x7feb4803bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:46:08.580 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:08.606+0000 7feb62a00700 1 --2- 172.21.15.177:0/1582610979 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7feb48039900 0x7feb4803bdc0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7feb540050f0 tx=0x7feb5401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:46:08.585 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:08.611+0000 7feb5be00700 1 -- 172.21.15.177:0/1582610979 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7feb4c021e00 con 0x7feb6419a040 2024-03-27T10:46:08.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:08 smithi097 ceph-mon[31859]: Reconfiguring mon.b (monmap changed)... 2024-03-27T10:46:08.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:08 smithi097 ceph-mon[31859]: Reconfiguring daemon mon.b on smithi114 2024-03-27T10:46:08.826 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:08.852+0000 7feb69c00700 1 -- 172.21.15.177:0/1582610979 --> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi097=x;smithi114=y;smithi177=z", "target": ["mon-mgr", ""]}) -- 0x7feb64062bd0 con 0x7feb48039900 2024-03-27T10:46:08.840 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:08.866+0000 7feb5be00700 1 -- 172.21.15.177:0/1582610979 <== mgr.14152 v2:172.21.15.97:6800/3955463538 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7feb64062bd0 con 0x7feb48039900 2024-03-27T10:46:08.840 INFO:teuthology.orchestra.run.smithi177.stdout:Scheduled mgr update... 2024-03-27T10:46:08.844 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:08.870+0000 7feb69c00700 1 -- 172.21.15.177:0/1582610979 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7feb48039900 msgr2=0x7feb4803bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:46:08.844 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:08.870+0000 7feb69c00700 1 --2- 172.21.15.177:0/1582610979 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7feb48039900 0x7feb4803bdc0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7feb540050f0 tx=0x7feb5401a040 comp rx=0 tx=0).stop 2024-03-27T10:46:08.844 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:08.870+0000 7feb69c00700 1 -- 172.21.15.177:0/1582610979 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feb6419a040 msgr2=0x7feb6419f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:46:08.844 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:08.870+0000 7feb69c00700 1 --2- 172.21.15.177:0/1582610979 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feb6419a040 0x7feb6419f0b0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7feb4c00b780 tx=0x7feb4c00bb40 comp rx=0 tx=0).stop 2024-03-27T10:46:08.845 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:08.870+0000 7feb69c00700 1 -- 172.21.15.177:0/1582610979 shutdown_connections 2024-03-27T10:46:08.845 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:08.870+0000 7feb69c00700 1 --2- 172.21.15.177:0/1582610979 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7feb48039900 0x7feb4803bdc0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:46:08.845 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:08.870+0000 7feb69c00700 1 --2- 172.21.15.177:0/1582610979 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feb6419a4c0 0x7feb641a1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:46:08.845 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:08.870+0000 7feb69c00700 1 --2- 172.21.15.177:0/1582610979 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb64102170 0x7feb64199b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:46:08.845 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:08.870+0000 7feb69c00700 1 --2- 172.21.15.177:0/1582610979 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feb6419a040 0x7feb6419f0b0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:46:08.845 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:08.870+0000 7feb69c00700 1 -- 172.21.15.177:0/1582610979 >> 172.21.15.177:0/1582610979 conn(0x7feb640fd710 msgr2=0x7feb640ffb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:46:08.845 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:08.870+0000 7feb69c00700 1 -- 172.21.15.177:0/1582610979 shutdown_connections 2024-03-27T10:46:08.845 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:46:08.870+0000 7feb69c00700 1 -- 172.21.15.177:0/1582610979 wait complete. 2024-03-27T10:46:09.429 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:09 smithi177 ceph-mon[35294]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:46:09.429 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:09 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:09.429 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:09 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:09.429 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:09 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:09.429 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:09 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-27T10:46:09.429 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:09 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-27T10:46:09.429 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:09 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:46:09.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:09 smithi114 ceph-mon[35338]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:46:09.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:09 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:09.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:09 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:09.520 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:09 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:09.520 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:09 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-27T10:46:09.520 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:09 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-27T10:46:09.520 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:09 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:46:09.608 DEBUG:teuthology.orchestra.run.smithi114:mgr.y> sudo journalctl -f -n 0 -u ceph-adca88c4-ec26-11ee-95d0-87774f69a715@mgr.y.service 2024-03-27T10:46:09.616 DEBUG:teuthology.orchestra.run.smithi177:mgr.z> sudo journalctl -f -n 0 -u ceph-adca88c4-ec26-11ee-95d0-87774f69a715@mgr.z.service 2024-03-27T10:46:09.623 INFO:tasks.cephadm:Deploying OSDs... 2024-03-27T10:46:09.623 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-27T10:46:09.623 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/scratch_devs of=/dev/stdout 2024-03-27T10:46:09.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:09 smithi097 ceph-mon[31859]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:46:09.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:09 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:09.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:09 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:09.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:09 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:09.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:09 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-27T10:46:09.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:09 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-27T10:46:09.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:09 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:46:09.682 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-27T10:46:09.682 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_1 2024-03-27T10:46:09.778 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-27T10:46:09.779 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-27T10:46:09.779 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 5h/5d Inode: 633 Links: 1 2024-03-27T10:46:09.779 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-27T10:46:09.779 INFO:teuthology.orchestra.run.smithi097.stdout:Context: system_u:object_r:device_t:s0 2024-03-27T10:46:09.779 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2024-03-27 10:44:56.637212442 +0000 2024-03-27T10:46:09.779 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2024-03-27 10:38:48.052977550 +0000 2024-03-27T10:46:09.779 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2024-03-27 10:38:48.052977550 +0000 2024-03-27T10:46:09.779 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: 2024-03-27 10:38:48.052977550 +0000 2024-03-27T10:46:09.779 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-27T10:46:09.864 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-03-27T10:46:09.864 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-03-27T10:46:09.864 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.000372813 s, 1.4 MB/s 2024-03-27T10:46:09.868 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-27T10:46:09.926 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_2 2024-03-27T10:46:09.978 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-27T10:46:09.978 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-27T10:46:09.978 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 5h/5d Inode: 652 Links: 1 2024-03-27T10:46:09.978 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-27T10:46:09.978 INFO:teuthology.orchestra.run.smithi097.stdout:Context: system_u:object_r:device_t:s0 2024-03-27T10:46:09.978 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2024-03-27 10:44:56.723210977 +0000 2024-03-27T10:46:09.979 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2024-03-27 10:38:48.647967372 +0000 2024-03-27T10:46:09.979 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2024-03-27 10:38:48.647967372 +0000 2024-03-27T10:46:09.979 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: 2024-03-27 10:38:48.647967372 +0000 2024-03-27T10:46:09.979 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-27T10:46:10.067 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-03-27T10:46:10.067 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-03-27T10:46:10.067 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.000402788 s, 1.3 MB/s 2024-03-27T10:46:10.071 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-27T10:46:10.123 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_3 2024-03-27T10:46:10.169 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-27T10:46:10.169 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-27T10:46:10.169 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 5h/5d Inode: 671 Links: 1 2024-03-27T10:46:10.169 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-27T10:46:10.169 INFO:teuthology.orchestra.run.smithi097.stdout:Context: system_u:object_r:device_t:s0 2024-03-27T10:46:10.169 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2024-03-27 10:44:56.792209801 +0000 2024-03-27T10:46:10.169 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2024-03-27 10:38:49.214957673 +0000 2024-03-27T10:46:10.169 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2024-03-27 10:38:49.214957673 +0000 2024-03-27T10:46:10.169 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: 2024-03-27 10:38:49.214957673 +0000 2024-03-27T10:46:10.169 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-27T10:46:10.301 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-03-27T10:46:10.302 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-03-27T10:46:10.302 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.000338888 s, 1.5 MB/s 2024-03-27T10:46:10.306 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-27T10:46:10.363 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_4 2024-03-27T10:46:10.412 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-27T10:46:10.412 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-27T10:46:10.412 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 5h/5d Inode: 690 Links: 1 2024-03-27T10:46:10.413 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-27T10:46:10.413 INFO:teuthology.orchestra.run.smithi097.stdout:Context: system_u:object_r:device_t:s0 2024-03-27T10:46:10.413 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2024-03-27 10:44:56.869208490 +0000 2024-03-27T10:46:10.413 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2024-03-27 10:38:49.788947853 +0000 2024-03-27T10:46:10.413 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2024-03-27 10:38:49.788947853 +0000 2024-03-27T10:46:10.413 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: 2024-03-27 10:38:49.788947853 +0000 2024-03-27T10:46:10.414 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-27T10:46:10.429 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:10 smithi177 ceph-mon[35294]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi097=x;smithi114=y;smithi177=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T10:46:10.429 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:10 smithi177 ceph-mon[35294]: Saving service mgr spec with placement smithi097=x;smithi114=y;smithi177=z;count:3 2024-03-27T10:46:10.429 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:10 smithi177 ceph-mon[35294]: Reconfiguring mon.c (monmap changed)... 2024-03-27T10:46:10.429 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:10 smithi177 ceph-mon[35294]: Reconfiguring daemon mon.c on smithi177 2024-03-27T10:46:10.491 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:10 smithi097 ceph-mon[31859]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi097=x;smithi114=y;smithi177=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T10:46:10.491 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:10 smithi097 ceph-mon[31859]: Saving service mgr spec with placement smithi097=x;smithi114=y;smithi177=z;count:3 2024-03-27T10:46:10.491 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:10 smithi097 ceph-mon[31859]: Reconfiguring mon.c (monmap changed)... 2024-03-27T10:46:10.491 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:10 smithi097 ceph-mon[31859]: Reconfiguring daemon mon.c on smithi177 2024-03-27T10:46:10.500 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-03-27T10:46:10.500 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-03-27T10:46:10.500 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.000398964 s, 1.3 MB/s 2024-03-27T10:46:10.504 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-27T10:46:10.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:10 smithi114 ceph-mon[35338]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi097=x;smithi114=y;smithi177=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T10:46:10.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:10 smithi114 ceph-mon[35338]: Saving service mgr spec with placement smithi097=x;smithi114=y;smithi177=z;count:3 2024-03-27T10:46:10.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:10 smithi114 ceph-mon[35338]: Reconfiguring mon.c (monmap changed)... 2024-03-27T10:46:10.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:10 smithi114 ceph-mon[35338]: Reconfiguring daemon mon.c on smithi177 2024-03-27T10:46:10.561 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-03-27T10:46:10.561 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/scratch_devs of=/dev/stdout 2024-03-27T10:46:10.618 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-27T10:46:10.619 DEBUG:teuthology.orchestra.run.smithi114:> stat /dev/vg_nvme/lv_1 2024-03-27T10:46:10.667 INFO:teuthology.orchestra.run.smithi114.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-27T10:46:10.667 INFO:teuthology.orchestra.run.smithi114.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-27T10:46:10.667 INFO:teuthology.orchestra.run.smithi114.stdout:Device: 5h/5d Inode: 634 Links: 1 2024-03-27T10:46:10.668 INFO:teuthology.orchestra.run.smithi114.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-27T10:46:10.668 INFO:teuthology.orchestra.run.smithi114.stdout:Context: system_u:object_r:device_t:s0 2024-03-27T10:46:10.668 INFO:teuthology.orchestra.run.smithi114.stdout:Access: 2024-03-27 10:45:54.125935550 +0000 2024-03-27T10:46:10.668 INFO:teuthology.orchestra.run.smithi114.stdout:Modify: 2024-03-27 10:38:58.727576292 +0000 2024-03-27T10:46:10.668 INFO:teuthology.orchestra.run.smithi114.stdout:Change: 2024-03-27 10:38:58.727576292 +0000 2024-03-27T10:46:10.668 INFO:teuthology.orchestra.run.smithi114.stdout: Birth: 2024-03-27 10:38:58.727576292 +0000 2024-03-27T10:46:10.669 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-27T10:46:10.756 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records in 2024-03-27T10:46:10.756 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records out 2024-03-27T10:46:10.756 INFO:teuthology.orchestra.run.smithi114.stderr:512 bytes copied, 0.000358843 s, 1.4 MB/s 2024-03-27T10:46:10.760 DEBUG:teuthology.orchestra.run.smithi114:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-27T10:46:10.814 DEBUG:teuthology.orchestra.run.smithi114:> stat /dev/vg_nvme/lv_2 2024-03-27T10:46:10.865 INFO:teuthology.orchestra.run.smithi114.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-27T10:46:10.865 INFO:teuthology.orchestra.run.smithi114.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-27T10:46:10.866 INFO:teuthology.orchestra.run.smithi114.stdout:Device: 5h/5d Inode: 651 Links: 1 2024-03-27T10:46:10.866 INFO:teuthology.orchestra.run.smithi114.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-27T10:46:10.866 INFO:teuthology.orchestra.run.smithi114.stdout:Context: system_u:object_r:device_t:s0 2024-03-27T10:46:10.866 INFO:teuthology.orchestra.run.smithi114.stdout:Access: 2024-03-27 10:45:54.126935528 +0000 2024-03-27T10:46:10.866 INFO:teuthology.orchestra.run.smithi114.stdout:Modify: 2024-03-27 10:38:59.288564024 +0000 2024-03-27T10:46:10.866 INFO:teuthology.orchestra.run.smithi114.stdout:Change: 2024-03-27 10:38:59.288564024 +0000 2024-03-27T10:46:10.867 INFO:teuthology.orchestra.run.smithi114.stdout: Birth: 2024-03-27 10:38:59.288564024 +0000 2024-03-27T10:46:10.867 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-27T10:46:10.948 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records in 2024-03-27T10:46:10.948 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records out 2024-03-27T10:46:10.948 INFO:teuthology.orchestra.run.smithi114.stderr:512 bytes copied, 0.000338415 s, 1.5 MB/s 2024-03-27T10:46:10.952 DEBUG:teuthology.orchestra.run.smithi114:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-27T10:46:11.006 DEBUG:teuthology.orchestra.run.smithi114:> stat /dev/vg_nvme/lv_3 2024-03-27T10:46:11.055 INFO:teuthology.orchestra.run.smithi114.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-27T10:46:11.056 INFO:teuthology.orchestra.run.smithi114.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-27T10:46:11.056 INFO:teuthology.orchestra.run.smithi114.stdout:Device: 5h/5d Inode: 671 Links: 1 2024-03-27T10:46:11.056 INFO:teuthology.orchestra.run.smithi114.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-27T10:46:11.056 INFO:teuthology.orchestra.run.smithi114.stdout:Context: system_u:object_r:device_t:s0 2024-03-27T10:46:11.056 INFO:teuthology.orchestra.run.smithi114.stdout:Access: 2024-03-27 10:45:54.127935506 +0000 2024-03-27T10:46:11.056 INFO:teuthology.orchestra.run.smithi114.stdout:Modify: 2024-03-27 10:38:59.849551758 +0000 2024-03-27T10:46:11.056 INFO:teuthology.orchestra.run.smithi114.stdout:Change: 2024-03-27 10:38:59.849551758 +0000 2024-03-27T10:46:11.056 INFO:teuthology.orchestra.run.smithi114.stdout: Birth: 2024-03-27 10:38:59.849551758 +0000 2024-03-27T10:46:11.056 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-27T10:46:11.142 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records in 2024-03-27T10:46:11.142 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records out 2024-03-27T10:46:11.142 INFO:teuthology.orchestra.run.smithi114.stderr:512 bytes copied, 0.000486376 s, 1.1 MB/s 2024-03-27T10:46:11.146 DEBUG:teuthology.orchestra.run.smithi114:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-27T10:46:11.201 DEBUG:teuthology.orchestra.run.smithi114:> stat /dev/vg_nvme/lv_4 2024-03-27T10:46:11.249 INFO:teuthology.orchestra.run.smithi114.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-27T10:46:11.249 INFO:teuthology.orchestra.run.smithi114.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-27T10:46:11.249 INFO:teuthology.orchestra.run.smithi114.stdout:Device: 5h/5d Inode: 690 Links: 1 2024-03-27T10:46:11.249 INFO:teuthology.orchestra.run.smithi114.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-27T10:46:11.249 INFO:teuthology.orchestra.run.smithi114.stdout:Context: system_u:object_r:device_t:s0 2024-03-27T10:46:11.249 INFO:teuthology.orchestra.run.smithi114.stdout:Access: 2024-03-27 10:45:54.127935506 +0000 2024-03-27T10:46:11.249 INFO:teuthology.orchestra.run.smithi114.stdout:Modify: 2024-03-27 10:39:00.422539227 +0000 2024-03-27T10:46:11.249 INFO:teuthology.orchestra.run.smithi114.stdout:Change: 2024-03-27 10:39:00.422539227 +0000 2024-03-27T10:46:11.249 INFO:teuthology.orchestra.run.smithi114.stdout: Birth: 2024-03-27 10:39:00.422539227 +0000 2024-03-27T10:46:11.250 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-27T10:46:11.332 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records in 2024-03-27T10:46:11.332 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records out 2024-03-27T10:46:11.332 INFO:teuthology.orchestra.run.smithi114.stderr:512 bytes copied, 0.000341621 s, 1.5 MB/s 2024-03-27T10:46:11.336 DEBUG:teuthology.orchestra.run.smithi114:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-27T10:46:11.390 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-03-27T10:46:11.390 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/scratch_devs of=/dev/stdout 2024-03-27T10:46:11.439 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-27T10:46:11.440 DEBUG:teuthology.orchestra.run.smithi177:> stat /dev/vg_nvme/lv_1 2024-03-27T10:46:11.520 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:11 smithi114 ceph-mon[35338]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:46:11.520 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:11 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:11.520 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:11 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:11.520 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:11 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:46:11.520 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:11 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:46:11.520 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:11 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:46:11.521 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:11 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:11.521 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:11 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-27T10:46:11.521 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:11 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-27T10:46:11.521 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:11 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-27T10:46:11.521 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:11 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:46:11.530 INFO:teuthology.orchestra.run.smithi177.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-27T10:46:11.530 INFO:teuthology.orchestra.run.smithi177.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-27T10:46:11.530 INFO:teuthology.orchestra.run.smithi177.stdout:Device: 5h/5d Inode: 632 Links: 1 2024-03-27T10:46:11.531 INFO:teuthology.orchestra.run.smithi177.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-27T10:46:11.531 INFO:teuthology.orchestra.run.smithi177.stdout:Context: system_u:object_r:device_t:s0 2024-03-27T10:46:11.531 INFO:teuthology.orchestra.run.smithi177.stdout:Access: 2024-03-27 10:45:42.978769250 +0000 2024-03-27T10:46:11.531 INFO:teuthology.orchestra.run.smithi177.stdout:Modify: 2024-03-27 10:38:44.105899274 +0000 2024-03-27T10:46:11.531 INFO:teuthology.orchestra.run.smithi177.stdout:Change: 2024-03-27 10:38:44.105899274 +0000 2024-03-27T10:46:11.531 INFO:teuthology.orchestra.run.smithi177.stdout: Birth: 2024-03-27 10:38:44.105899274 +0000 2024-03-27T10:46:11.531 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-27T10:46:11.600 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:11 smithi177 ceph-mon[35294]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:46:11.600 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:11 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:11.600 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:11 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:11.600 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:11 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:46:11.600 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:11 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:46:11.600 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:11 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:46:11.600 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:11 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:11.600 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:11 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-27T10:46:11.601 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:11 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-27T10:46:11.601 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:11 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-27T10:46:11.601 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:11 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:46:11.607 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records in 2024-03-27T10:46:11.607 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records out 2024-03-27T10:46:11.607 INFO:teuthology.orchestra.run.smithi177.stderr:512 bytes copied, 0.000324711 s, 1.6 MB/s 2024-03-27T10:46:11.611 DEBUG:teuthology.orchestra.run.smithi177:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-27T10:46:11.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:11 smithi097 ceph-mon[31859]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:46:11.644 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:11 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:11.644 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:11 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:11.644 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:11 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:46:11.644 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:11 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:46:11.644 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:11 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:46:11.644 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:11 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:11.644 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:11 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-27T10:46:11.644 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:11 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-27T10:46:11.644 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:11 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-27T10:46:11.644 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:11 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:46:11.662 DEBUG:teuthology.orchestra.run.smithi177:> stat /dev/vg_nvme/lv_2 2024-03-27T10:46:11.708 INFO:teuthology.orchestra.run.smithi177.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-27T10:46:11.708 INFO:teuthology.orchestra.run.smithi177.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-27T10:46:11.708 INFO:teuthology.orchestra.run.smithi177.stdout:Device: 5h/5d Inode: 652 Links: 1 2024-03-27T10:46:11.708 INFO:teuthology.orchestra.run.smithi177.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-27T10:46:11.708 INFO:teuthology.orchestra.run.smithi177.stdout:Context: system_u:object_r:device_t:s0 2024-03-27T10:46:11.708 INFO:teuthology.orchestra.run.smithi177.stdout:Access: 2024-03-27 10:45:42.979769222 +0000 2024-03-27T10:46:11.708 INFO:teuthology.orchestra.run.smithi177.stdout:Modify: 2024-03-27 10:38:44.649884329 +0000 2024-03-27T10:46:11.708 INFO:teuthology.orchestra.run.smithi177.stdout:Change: 2024-03-27 10:38:44.649884329 +0000 2024-03-27T10:46:11.708 INFO:teuthology.orchestra.run.smithi177.stdout: Birth: 2024-03-27 10:38:44.649884329 +0000 2024-03-27T10:46:11.709 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-27T10:46:11.819 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records in 2024-03-27T10:46:11.819 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records out 2024-03-27T10:46:11.819 INFO:teuthology.orchestra.run.smithi177.stderr:512 bytes copied, 0.000305916 s, 1.7 MB/s 2024-03-27T10:46:11.822 DEBUG:teuthology.orchestra.run.smithi177:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-27T10:46:11.863 DEBUG:teuthology.orchestra.run.smithi177:> stat /dev/vg_nvme/lv_3 2024-03-27T10:46:11.944 INFO:teuthology.orchestra.run.smithi177.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-27T10:46:11.944 INFO:teuthology.orchestra.run.smithi177.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-27T10:46:11.944 INFO:teuthology.orchestra.run.smithi177.stdout:Device: 5h/5d Inode: 672 Links: 1 2024-03-27T10:46:11.944 INFO:teuthology.orchestra.run.smithi177.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-27T10:46:11.944 INFO:teuthology.orchestra.run.smithi177.stdout:Context: system_u:object_r:device_t:s0 2024-03-27T10:46:11.944 INFO:teuthology.orchestra.run.smithi177.stdout:Access: 2024-03-27 10:45:42.979769222 +0000 2024-03-27T10:46:11.944 INFO:teuthology.orchestra.run.smithi177.stdout:Modify: 2024-03-27 10:38:45.211868889 +0000 2024-03-27T10:46:11.944 INFO:teuthology.orchestra.run.smithi177.stdout:Change: 2024-03-27 10:38:45.211868889 +0000 2024-03-27T10:46:11.944 INFO:teuthology.orchestra.run.smithi177.stdout: Birth: 2024-03-27 10:38:45.211868889 +0000 2024-03-27T10:46:11.945 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-27T10:46:12.067 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records in 2024-03-27T10:46:12.067 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records out 2024-03-27T10:46:12.067 INFO:teuthology.orchestra.run.smithi177.stderr:512 bytes copied, 0.000372383 s, 1.4 MB/s 2024-03-27T10:46:12.071 DEBUG:teuthology.orchestra.run.smithi177:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-27T10:46:12.119 DEBUG:teuthology.orchestra.run.smithi177:> stat /dev/vg_nvme/lv_4 2024-03-27T10:46:12.211 INFO:teuthology.orchestra.run.smithi177.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-27T10:46:12.212 INFO:teuthology.orchestra.run.smithi177.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-27T10:46:12.212 INFO:teuthology.orchestra.run.smithi177.stdout:Device: 5h/5d Inode: 691 Links: 1 2024-03-27T10:46:12.212 INFO:teuthology.orchestra.run.smithi177.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-27T10:46:12.212 INFO:teuthology.orchestra.run.smithi177.stdout:Context: system_u:object_r:device_t:s0 2024-03-27T10:46:12.212 INFO:teuthology.orchestra.run.smithi177.stdout:Access: 2024-03-27 10:45:42.980769194 +0000 2024-03-27T10:46:12.212 INFO:teuthology.orchestra.run.smithi177.stdout:Modify: 2024-03-27 10:38:45.794852872 +0000 2024-03-27T10:46:12.212 INFO:teuthology.orchestra.run.smithi177.stdout:Change: 2024-03-27 10:38:45.794852872 +0000 2024-03-27T10:46:12.212 INFO:teuthology.orchestra.run.smithi177.stdout: Birth: 2024-03-27 10:38:45.794852872 +0000 2024-03-27T10:46:12.213 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-27T10:46:12.280 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records in 2024-03-27T10:46:12.281 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records out 2024-03-27T10:46:12.281 INFO:teuthology.orchestra.run.smithi177.stderr:512 bytes copied, 0.000426236 s, 1.2 MB/s 2024-03-27T10:46:12.286 DEBUG:teuthology.orchestra.run.smithi177:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-27T10:46:12.336 INFO:tasks.cephadm:Deploying osd.0 on smithi097 with /dev/vg_nvme/lv_4... 2024-03-27T10:46:12.336 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid adca88c4-ec26-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-27T10:46:12.350 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:12 smithi177 ceph-mon[35294]: Deploying daemon mgr.z on smithi177 2024-03-27T10:46:12.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:12 smithi114 ceph-mon[35338]: Deploying daemon mgr.z on smithi177 2024-03-27T10:46:12.642 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:12 smithi097 ceph-mon[31859]: Deploying daemon mgr.z on smithi177 2024-03-27T10:46:12.766 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T10:46:13.432 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:13 smithi177 ceph-mon[35294]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:46:13.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:13 smithi114 ceph-mon[35338]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:46:13.593 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:13 smithi097 ceph-mon[31859]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:46:14.307 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Mar 27 10:46:13 smithi177 systemd[1]: Starting Ceph mgr.z for adca88c4-ec26-11ee-95d0-87774f69a715... 2024-03-27T10:46:14.601 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Mar 27 10:46:14 smithi177 podman[37151]: 2024-03-27 10:46:14.237691381 +0000 UTC m=+0.032406414 image pull b379bbd8da6b5d0fa9a3523941efac213b6815e1cbf05c6507954f0a3fe7a801 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772 2024-03-27T10:46:14.602 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Mar 27 10:46:14 smithi177 podman[37151]: 2024-03-27 10:46:14.356440098 +0000 UTC m=+0.151155154 container create f9408119d0ecefc054f81700b1359a244e85efc9105597f499358c382cec6714 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-z, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-27T10:46:14.602 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Mar 27 10:46:14 smithi177 podman[37151]: 2024-03-27 10:46:14.626659285 +0000 UTC m=+0.421374385 container init f9408119d0ecefc054f81700b1359a244e85efc9105597f499358c382cec6714 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-z, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240326) 2024-03-27T10:46:14.929 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Mar 27 10:46:14 smithi177 podman[37151]: 2024-03-27 10:46:14.636833491 +0000 UTC m=+0.431548515 container start f9408119d0ecefc054f81700b1359a244e85efc9105597f499358c382cec6714 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-z, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-27T10:46:14.929 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Mar 27 10:46:14 smithi177 ceph-mgr[37170]: -- 172.21.15.177:0/1954478063 <== mon.1 v2:172.21.15.177:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5632ee08c5a0 con 0x5632ee060800 2024-03-27T10:46:14.929 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Mar 27 10:46:14 smithi177 bash[37151]: f9408119d0ecefc054f81700b1359a244e85efc9105597f499358c382cec6714 2024-03-27T10:46:14.929 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Mar 27 10:46:14 smithi177 systemd[1]: Started Ceph mgr.z for adca88c4-ec26-11ee-95d0-87774f69a715. 2024-03-27T10:46:15.352 INFO:teuthology.orchestra.run.smithi097.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-27T10:46:15.352 INFO:teuthology.orchestra.run.smithi097.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-27T10:46:15.352 INFO:teuthology.orchestra.run.smithi097.stderr: stderr: 10+0 records in 2024-03-27T10:46:15.352 INFO:teuthology.orchestra.run.smithi097.stderr:10+0 records out 2024-03-27T10:46:15.352 INFO:teuthology.orchestra.run.smithi097.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0356602 s, 294 MB/s 2024-03-27T10:46:15.353 INFO:teuthology.orchestra.run.smithi097.stderr:--> Zapping successful for: 2024-03-27T10:46:15.483 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:15 smithi177 ceph-mon[35294]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:46:15.484 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:15 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:15.484 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:15 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:15.484 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:15 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:15.484 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:15 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-27T10:46:15.484 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:15 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-27T10:46:15.484 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:15 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-27T10:46:15.484 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:15 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:46:15.484 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Mar 27 10:46:15 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-z[37166]: 2024-03-27T10:46:15.387+0000 7f9fe2047200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-27T10:46:15.485 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Mar 27 10:46:15 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-z[37166]: 2024-03-27T10:46:15.508+0000 7f9fe2047200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-27T10:46:15.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:15 smithi114 ceph-mon[35338]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:46:15.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:15 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:15.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:15 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:15.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:15 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:15.520 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:15 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-27T10:46:15.520 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:15 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-27T10:46:15.520 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:15 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-27T10:46:15.520 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:15 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:46:15.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:15 smithi097 ceph-mon[31859]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:46:15.644 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:15 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:15.644 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:15 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:15.644 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:15 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:15.644 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:15 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-27T10:46:15.644 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:15 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-27T10:46:15.644 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:15 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-27T10:46:15.644 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:15 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:46:16.059 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid adca88c4-ec26-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi097:vg_nvme/lv_4 2024-03-27T10:46:16.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:16 smithi097 ceph-mon[31859]: Deploying daemon mgr.y on smithi114 2024-03-27T10:46:16.494 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T10:46:16.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:16 smithi114 ceph-mon[35338]: Deploying daemon mgr.y on smithi114 2024-03-27T10:46:16.572 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:16 smithi177 ceph-mon[35294]: Deploying daemon mgr.y on smithi114 2024-03-27T10:46:16.572 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Mar 27 10:46:16 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-z[37166]: 2024-03-27T10:46:16.301+0000 7f9fe2047200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-27T10:46:16.572 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Mar 27 10:46:16 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-z[37166]: 2024-03-27T10:46:16.420+0000 7f9fe2047200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-27T10:46:16.572 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Mar 27 10:46:16 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-z[37166]: 2024-03-27T10:46:16.596+0000 7f9fe2047200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-27T10:46:16.929 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Mar 27 10:46:16 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-z[37166]: 2024-03-27T10:46:16.718+0000 7f9fe2047200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-27T10:46:16.929 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Mar 27 10:46:16 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-z[37166]: 2024-03-27T10:46:16.825+0000 7f9fe2047200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-27T10:46:17.269 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:17 smithi114 ceph-mon[35338]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:46:17.388 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:17 smithi177 ceph-mon[35294]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:46:17.389 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Mar 27 10:46:17 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-z[37166]: 2024-03-27T10:46:17.161+0000 7f9fe2047200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-27T10:46:17.389 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Mar 27 10:46:17 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-z[37166]: 2024-03-27T10:46:17.302+0000 7f9fe2047200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-27T10:46:17.389 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Mar 27 10:46:17 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-z[37166]: 2024-03-27T10:46:17.413+0000 7f9fe2047200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-27T10:46:17.468 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:17 smithi097 ceph-mon[31859]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:46:17.678 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Mar 27 10:46:17 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-z[37166]: 2024-03-27T10:46:17.535+0000 7f9fe2047200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-27T10:46:18.162 INFO:journalctl@ceph.mgr.y.smithi114.stdout:Mar 27 10:46:17 smithi114 systemd[1]: Starting Ceph mgr.y for adca88c4-ec26-11ee-95d0-87774f69a715... 2024-03-27T10:46:18.178 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Mar 27 10:46:17 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-z[37166]: 2024-03-27T10:46:17.752+0000 7f9fe2047200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-27T10:46:18.416 INFO:journalctl@ceph.mgr.y.smithi114.stdout:Mar 27 10:46:18 smithi114 podman[36705]: 2024-03-27 10:46:18.187315131 +0000 UTC m=+0.122292445 container create 72bfb2b6dd9bd36c98d50ba3ef8f8ffd9e75cf29306fc8a107727e738ea9bc72 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-y, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240326.131558-155268c, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-27T10:46:18.416 INFO:journalctl@ceph.mgr.y.smithi114.stdout:Mar 27 10:46:18 smithi114 podman[36705]: 2024-03-27 10:46:18.096576844 +0000 UTC m=+0.031554121 image pull b379bbd8da6b5d0fa9a3523941efac213b6815e1cbf05c6507954f0a3fe7a801 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772 2024-03-27T10:46:18.416 INFO:journalctl@ceph.mgr.y.smithi114.stdout:Mar 27 10:46:18 smithi114 podman[36705]: 2024-03-27 10:46:18.441103637 +0000 UTC m=+0.376080958 container init 72bfb2b6dd9bd36c98d50ba3ef8f8ffd9e75cf29306fc8a107727e738ea9bc72 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-y, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240326.131558-155268c) 2024-03-27T10:46:18.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:18.528+0000 7f8755600700 1 -- 172.21.15.97:0/119355379 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8750102760 msgr2=0x7f8750102bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:46:18.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:18.528+0000 7f8755600700 1 --2- 172.21.15.97:0/119355379 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8750102760 0x7f8750102bc0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f873c0099c0 tx=0x7f873c009cd0 comp rx=0 tx=0).stop 2024-03-27T10:46:18.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:18.528+0000 7f8755600700 1 -- 172.21.15.97:0/119355379 shutdown_connections 2024-03-27T10:46:18.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:18.528+0000 7f8755600700 1 --2- 172.21.15.97:0/119355379 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8750103100 0x7f87501099b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:46:18.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:18.528+0000 7f8755600700 1 --2- 172.21.15.97:0/119355379 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8750101560 0x7f8750101980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:46:18.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:18.528+0000 7f8755600700 1 --2- 172.21.15.97:0/119355379 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8750102760 0x7f8750102bc0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:46:18.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:18.528+0000 7f8755600700 1 -- 172.21.15.97:0/119355379 >> 172.21.15.97:0/119355379 conn(0x7f87500fcb00 msgr2=0x7f87500fef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:46:18.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:18.528+0000 7f8755600700 1 -- 172.21.15.97:0/119355379 shutdown_connections 2024-03-27T10:46:18.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:18.528+0000 7f8755600700 1 -- 172.21.15.97:0/119355379 wait complete. 2024-03-27T10:46:18.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:18.529+0000 7f8755600700 1 Processor -- start 2024-03-27T10:46:18.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:18.530+0000 7f8755600700 1 -- start start 2024-03-27T10:46:18.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:18.530+0000 7f8755600700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8750101560 0x7f87501987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:46:18.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:18.530+0000 7f8755600700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8750102760 0x7f8750198d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:46:18.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:18.530+0000 7f8755600700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8750103100 0x7f875019fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:46:18.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:18.530+0000 7f8755600700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f875010d040 con 0x7f8750101560 2024-03-27T10:46:18.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:18.530+0000 7f8755600700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f875010cec0 con 0x7f8750103100 2024-03-27T10:46:18.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:18.530+0000 7f8755600700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f875010d1c0 con 0x7f8750102760 2024-03-27T10:46:18.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:18.530+0000 7f874ea00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8750101560 0x7f87501987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:46:18.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:18.530+0000 7f874ea00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8750101560 0x7f87501987e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37814/0 (socket says 172.21.15.97:37814) 2024-03-27T10:46:18.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:18.530+0000 7f874ea00700 1 -- 172.21.15.97:0/1679624529 learned_addr learned my addr 172.21.15.97:0/1679624529 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:46:18.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:18.530+0000 7f874f400700 1 --2- 172.21.15.97:0/1679624529 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8750103100 0x7f875019fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:46:18.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:18.531+0000 7f874e000700 1 --2- 172.21.15.97:0/1679624529 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8750102760 0x7f8750198d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:46:18.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:18.531+0000 7f874ea00700 1 -- 172.21.15.97:0/1679624529 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8750102760 msgr2=0x7f8750198d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:46:18.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:18.531+0000 7f874ea00700 1 --2- 172.21.15.97:0/1679624529 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8750102760 0x7f8750198d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:46:18.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:18.531+0000 7f874ea00700 1 -- 172.21.15.97:0/1679624529 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8750103100 msgr2=0x7f875019fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:46:18.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:18.531+0000 7f874ea00700 1 --2- 172.21.15.97:0/1679624529 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8750103100 0x7f875019fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:46:18.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:18.531+0000 7f874ea00700 1 -- 172.21.15.97:0/1679624529 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87501a0440 con 0x7f8750101560 2024-03-27T10:46:18.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:18.531+0000 7f874ea00700 1 --2- 172.21.15.97:0/1679624529 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8750101560 0x7f87501987e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f874000c920 tx=0x7f874000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:46:18.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:18.531+0000 7f8747400700 1 -- 172.21.15.97:0/1679624529 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8740004220 con 0x7f8750101560 2024-03-27T10:46:18.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:18.532+0000 7f8747400700 1 -- 172.21.15.97:0/1679624529 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8740007d70 con 0x7f8750101560 2024-03-27T10:46:18.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:18.532+0000 7f8755600700 1 -- 172.21.15.97:0/1679624529 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87501a0730 con 0x7f8750101560 2024-03-27T10:46:18.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:18.532+0000 7f8747400700 1 -- 172.21.15.97:0/1679624529 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8740003ca0 con 0x7f8750101560 2024-03-27T10:46:18.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:18.532+0000 7f8755600700 1 -- 172.21.15.97:0/1679624529 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87501a0c70 con 0x7f8750101560 2024-03-27T10:46:18.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:18.533+0000 7f8747400700 1 -- 172.21.15.97:0/1679624529 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) ==== 46522+0+0 (secure 0 0 0) 0x7f87400078c0 con 0x7f8750101560 2024-03-27T10:46:18.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:18.533+0000 7f8755600700 1 -- 172.21.15.97:0/1679624529 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8750069300 con 0x7f8750101560 2024-03-27T10:46:18.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:18.533+0000 7f8747400700 1 --2- 172.21.15.97:0/1679624529 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f87340396b0 0x7f873403bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:46:18.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:18.533+0000 7f8747400700 1 -- 172.21.15.97:0/1679624529 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f874004c280 con 0x7f8750101560 2024-03-27T10:46:18.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:18.534+0000 7f874e000700 1 --2- 172.21.15.97:0/1679624529 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f87340396b0 0x7f873403bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:46:18.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:18.534+0000 7f874e000700 1 --2- 172.21.15.97:0/1679624529 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f87340396b0 0x7f873403bb70 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f873c0038f0 tx=0x7f873c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:46:18.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:18.539+0000 7f8747400700 1 -- 172.21.15.97:0/1679624529 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f8740014030 con 0x7f8750101560 2024-03-27T10:46:18.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:18.782+0000 7f8755600700 1 -- 172.21.15.97:0/1679624529 --> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f87500fd900 con 0x7f87340396b0 2024-03-27T10:46:18.770 INFO:journalctl@ceph.mgr.y.smithi114.stdout:Mar 27 10:46:18 smithi114 podman[36705]: 2024-03-27 10:46:18.451320638 +0000 UTC m=+0.386297939 container start 72bfb2b6dd9bd36c98d50ba3ef8f8ffd9e75cf29306fc8a107727e738ea9bc72 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-y, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-27T10:46:18.770 INFO:journalctl@ceph.mgr.y.smithi114.stdout:Mar 27 10:46:18 smithi114 ceph-mgr[36724]: -- 172.21.15.114:0/3561200423 <== mon.2 v2:172.21.15.114:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5597093185a0 con 0x5597092ecc00 2024-03-27T10:46:18.770 INFO:journalctl@ceph.mgr.y.smithi114.stdout:Mar 27 10:46:18 smithi114 bash[36705]: 72bfb2b6dd9bd36c98d50ba3ef8f8ffd9e75cf29306fc8a107727e738ea9bc72 2024-03-27T10:46:18.770 INFO:journalctl@ceph.mgr.y.smithi114.stdout:Mar 27 10:46:18 smithi114 systemd[1]: Started Ceph mgr.y for adca88c4-ec26-11ee-95d0-87774f69a715. 2024-03-27T10:46:19.428 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Mar 27 10:46:19 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-z[37166]: 2024-03-27T10:46:19.060+0000 7f9fe2047200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-27T10:46:19.429 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Mar 27 10:46:19 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-z[37166]: 2024-03-27T10:46:19.244+0000 7f9fe2047200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-27T10:46:19.429 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:19 smithi177 ceph-mon[35294]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:46:19.429 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:19 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:19.429 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:19 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:19.429 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:19 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:19.429 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:19 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:19.429 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:19 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-27T10:46:19.429 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:19 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-27T10:46:19.429 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:19 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:46:19.430 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:19 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:46:19.519 INFO:journalctl@ceph.mgr.y.smithi114.stdout:Mar 27 10:46:19 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-y[36720]: 2024-03-27T10:46:19.209+0000 7f1917484200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-27T10:46:19.519 INFO:journalctl@ceph.mgr.y.smithi114.stdout:Mar 27 10:46:19 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-y[36720]: 2024-03-27T10:46:19.335+0000 7f1917484200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-27T10:46:19.520 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:19 smithi114 ceph-mon[35338]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:46:19.520 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:19 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:19.520 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:19 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:19.520 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:19 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:19.520 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:19 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:19.520 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:19 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-27T10:46:19.520 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:19 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-27T10:46:19.520 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:19 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:46:19.520 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:19 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:46:19.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:19 smithi097 ceph-mon[31859]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:46:19.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:19 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:19.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:19 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:19.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:19 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:19.644 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:19 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:19.644 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:19 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-27T10:46:19.644 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:19 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-27T10:46:19.644 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:19 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:46:19.644 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:19 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:46:19.929 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Mar 27 10:46:19 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-z[37166]: 2024-03-27T10:46:19.685+0000 7f9fe2047200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-27T10:46:19.929 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Mar 27 10:46:19 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-z[37166]: 2024-03-27T10:46:19.828+0000 7f9fe2047200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-27T10:46:20.359 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:20 smithi177 ceph-mon[35294]: from='client.14229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T10:46:20.359 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:20 smithi177 ceph-mon[35294]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:46:20.359 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:20 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:20.359 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Mar 27 10:46:20 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-z[37166]: 2024-03-27T10:46:20.133+0000 7f9fe2047200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-27T10:46:20.359 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Mar 27 10:46:20 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-z[37166]: 2024-03-27T10:46:20.383+0000 7f9fe2047200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-27T10:46:20.489 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:20 smithi114 ceph-mon[35338]: from='client.14229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T10:46:20.489 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:20 smithi114 ceph-mon[35338]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:46:20.489 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:20 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:20.489 INFO:journalctl@ceph.mgr.y.smithi114.stdout:Mar 27 10:46:20 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-y[36720]: 2024-03-27T10:46:20.194+0000 7f1917484200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-27T10:46:20.489 INFO:journalctl@ceph.mgr.y.smithi114.stdout:Mar 27 10:46:20 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-y[36720]: 2024-03-27T10:46:20.329+0000 7f1917484200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-27T10:46:20.490 INFO:journalctl@ceph.mgr.y.smithi114.stdout:Mar 27 10:46:20 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-y[36720]: 2024-03-27T10:46:20.514+0000 7f1917484200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-27T10:46:20.634 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:20 smithi097 ceph-mon[31859]: from='client.14229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T10:46:20.634 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:20 smithi097 ceph-mon[31859]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:46:20.634 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:20 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:20.769 INFO:journalctl@ceph.mgr.y.smithi114.stdout:Mar 27 10:46:20 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-y[36720]: 2024-03-27T10:46:20.637+0000 7f1917484200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-27T10:46:20.769 INFO:journalctl@ceph.mgr.y.smithi114.stdout:Mar 27 10:46:20 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-y[36720]: 2024-03-27T10:46:20.759+0000 7f1917484200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-27T10:46:21.096 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Mar 27 10:46:20 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-z[37166]: 2024-03-27T10:46:20.788+0000 7f9fe2047200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-27T10:46:21.096 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Mar 27 10:46:21 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-z[37166]: 2024-03-27T10:46:21.120+0000 7f9fe2047200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-27T10:46:21.346 INFO:journalctl@ceph.mgr.y.smithi114.stdout:Mar 27 10:46:21 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-y[36720]: 2024-03-27T10:46:21.105+0000 7f1917484200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-27T10:46:21.347 INFO:journalctl@ceph.mgr.y.smithi114.stdout:Mar 27 10:46:21 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-y[36720]: 2024-03-27T10:46:21.254+0000 7f1917484200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-27T10:46:21.678 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Mar 27 10:46:21 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-z[37166]: 2024-03-27T10:46:21.420+0000 7f9fe2047200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-27T10:46:21.699 INFO:journalctl@ceph.mgr.y.smithi114.stdout:Mar 27 10:46:21 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-y[36720]: 2024-03-27T10:46:21.372+0000 7f1917484200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-27T10:46:21.699 INFO:journalctl@ceph.mgr.y.smithi114.stdout:Mar 27 10:46:21 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-y[36720]: 2024-03-27T10:46:21.500+0000 7f1917484200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-27T10:46:21.699 INFO:journalctl@ceph.mgr.y.smithi114.stdout:Mar 27 10:46:21 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-y[36720]: 2024-03-27T10:46:21.724+0000 7f1917484200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-27T10:46:22.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:21 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:22.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:21 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:22.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:21 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:22.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:21 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:22.143 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:21 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:22.143 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:21 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:22.143 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:21 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:22.143 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:21 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:22.179 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:21 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:22.179 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:21 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:22.179 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:21 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:22.179 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:21 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:22.678 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Mar 27 10:46:22 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-z[37166]: 2024-03-27T10:46:22.445+0000 7f9fe2047200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-27T10:46:23.118 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:22 smithi177 ceph-mon[35294]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:46:23.118 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:22 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:46:23.119 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:22 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:46:23.119 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:22 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:23.119 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:22 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-27T10:46:23.119 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:22 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-27T10:46:23.119 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:22 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:46:23.119 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:22 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/2656350591' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b634e5f1-6118-4e2f-95e9-830db390f8d3"}]: dispatch 2024-03-27T10:46:23.119 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:22 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/2656350591' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b634e5f1-6118-4e2f-95e9-830db390f8d3"}]': finished 2024-03-27T10:46:23.119 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:22 smithi177 ceph-mon[35294]: osdmap e5: 1 total, 0 up, 1 in 2024-03-27T10:46:23.119 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:22 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-27T10:46:23.143 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:22 smithi097 ceph-mon[31859]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:46:23.143 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:22 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:46:23.144 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:22 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:46:23.144 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:22 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:23.144 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:22 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-27T10:46:23.144 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:22 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-27T10:46:23.144 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:22 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:46:23.144 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:22 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/2656350591' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b634e5f1-6118-4e2f-95e9-830db390f8d3"}]: dispatch 2024-03-27T10:46:23.144 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:22 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/2656350591' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b634e5f1-6118-4e2f-95e9-830db390f8d3"}]': finished 2024-03-27T10:46:23.144 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:22 smithi097 ceph-mon[31859]: osdmap e5: 1 total, 0 up, 1 in 2024-03-27T10:46:23.144 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:22 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-27T10:46:23.183 INFO:journalctl@ceph.mgr.y.smithi114.stdout:Mar 27 10:46:23 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-y[36720]: 2024-03-27T10:46:23.038+0000 7f1917484200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-27T10:46:23.183 INFO:journalctl@ceph.mgr.y.smithi114.stdout:Mar 27 10:46:23 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-y[36720]: 2024-03-27T10:46:23.208+0000 7f1917484200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-27T10:46:23.184 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:22 smithi114 ceph-mon[35338]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:46:23.184 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:22 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:46:23.184 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:22 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:46:23.184 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:22 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:23.184 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:22 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-27T10:46:23.184 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:22 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-27T10:46:23.184 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:22 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:46:23.184 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:22 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/2656350591' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b634e5f1-6118-4e2f-95e9-830db390f8d3"}]: dispatch 2024-03-27T10:46:23.184 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:22 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/2656350591' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b634e5f1-6118-4e2f-95e9-830db390f8d3"}]': finished 2024-03-27T10:46:23.184 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:22 smithi114 ceph-mon[35338]: osdmap e5: 1 total, 0 up, 1 in 2024-03-27T10:46:23.184 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:22 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-27T10:46:23.429 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Mar 27 10:46:23 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-z[37166]: 2024-03-27T10:46:23.142+0000 7f9fe2047200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-27T10:46:23.429 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Mar 27 10:46:23 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-z[37166]: 2024-03-27T10:46:23.270+0000 7f9fe2047200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-27T10:46:23.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:23.855+0000 7f8747400700 1 -- 172.21.15.97:0/1679624529 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mgrmap(e 14) ==== 92418+0+0 (secure 0 0 0) 0x7f8740008cd0 con 0x7f8750101560 2024-03-27T10:46:23.841 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Mar 27 10:46:23 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-z[37166]: 2024-03-27T10:46:23.547+0000 7f9fe2047200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-27T10:46:23.967 INFO:journalctl@ceph.mgr.y.smithi114.stdout:Mar 27 10:46:23 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-y[36720]: 2024-03-27T10:46:23.618+0000 7f1917484200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-27T10:46:23.967 INFO:journalctl@ceph.mgr.y.smithi114.stdout:Mar 27 10:46:23 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-y[36720]: 2024-03-27T10:46:23.735+0000 7f1917484200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-27T10:46:23.967 INFO:journalctl@ceph.mgr.y.smithi114.stdout:Mar 27 10:46:23 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-y[36720]: 2024-03-27T10:46:23.992+0000 7f1917484200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-27T10:46:23.967 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:23 smithi114 ceph-mon[35338]: Reconfiguring mgr.x (unknown last config time)... 2024-03-27T10:46:23.968 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:23 smithi114 ceph-mon[35338]: Reconfiguring daemon mgr.x on smithi097 2024-03-27T10:46:23.968 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:23 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/2783011852' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-27T10:46:23.968 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:23 smithi114 ceph-mon[35338]: Standby manager daemon z started 2024-03-27T10:46:23.968 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:23 smithi114 ceph-mon[35338]: from='mgr.? 172.21.15.177:0/4209467873' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-27T10:46:23.968 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:23 smithi114 ceph-mon[35338]: from='mgr.? 172.21.15.177:0/4209467873' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-27T10:46:23.968 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:23 smithi114 ceph-mon[35338]: from='mgr.? 172.21.15.177:0/4209467873' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-27T10:46:23.968 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:23 smithi114 ceph-mon[35338]: from='mgr.? 172.21.15.177:0/4209467873' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-27T10:46:24.143 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:23 smithi097 ceph-mon[31859]: Reconfiguring mgr.x (unknown last config time)... 2024-03-27T10:46:24.143 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:23 smithi097 ceph-mon[31859]: Reconfiguring daemon mgr.x on smithi097 2024-03-27T10:46:24.143 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:23 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/2783011852' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-27T10:46:24.143 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:23 smithi097 ceph-mon[31859]: Standby manager daemon z started 2024-03-27T10:46:24.143 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:23 smithi097 ceph-mon[31859]: from='mgr.? 172.21.15.177:0/4209467873' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-27T10:46:24.143 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:23 smithi097 ceph-mon[31859]: from='mgr.? 172.21.15.177:0/4209467873' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-27T10:46:24.143 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:23 smithi097 ceph-mon[31859]: from='mgr.? 172.21.15.177:0/4209467873' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-27T10:46:24.143 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:23 smithi097 ceph-mon[31859]: from='mgr.? 172.21.15.177:0/4209467873' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-27T10:46:24.145 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:23 smithi177 ceph-mon[35294]: Reconfiguring mgr.x (unknown last config time)... 2024-03-27T10:46:24.145 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:23 smithi177 ceph-mon[35294]: Reconfiguring daemon mgr.x on smithi097 2024-03-27T10:46:24.145 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:23 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/2783011852' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-27T10:46:24.145 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:23 smithi177 ceph-mon[35294]: Standby manager daemon z started 2024-03-27T10:46:24.145 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:23 smithi177 ceph-mon[35294]: from='mgr.? 172.21.15.177:0/4209467873' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-27T10:46:24.146 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:23 smithi177 ceph-mon[35294]: from='mgr.? 172.21.15.177:0/4209467873' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-27T10:46:24.146 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:23 smithi177 ceph-mon[35294]: from='mgr.? 172.21.15.177:0/4209467873' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-27T10:46:24.146 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:23 smithi177 ceph-mon[35294]: from='mgr.? 172.21.15.177:0/4209467873' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-27T10:46:24.269 INFO:journalctl@ceph.mgr.y.smithi114.stdout:Mar 27 10:46:24 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-y[36720]: 2024-03-27T10:46:24.235+0000 7f1917484200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-27T10:46:24.852 INFO:journalctl@ceph.mgr.y.smithi114.stdout:Mar 27 10:46:24 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-y[36720]: 2024-03-27T10:46:24.606+0000 7f1917484200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-27T10:46:25.143 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:24 smithi097 ceph-mon[31859]: mgrmap e14: x(active, since 103s), standbys: z 2024-03-27T10:46:25.143 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:24 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-27T10:46:25.143 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:24 smithi097 ceph-mon[31859]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:46:25.179 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:24 smithi177 ceph-mon[35294]: mgrmap e14: x(active, since 103s), standbys: z 2024-03-27T10:46:25.179 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:24 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-27T10:46:25.179 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:24 smithi177 ceph-mon[35294]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:46:25.196 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:24 smithi114 ceph-mon[35338]: mgrmap e14: x(active, since 103s), standbys: z 2024-03-27T10:46:25.196 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:24 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-27T10:46:25.196 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:24 smithi114 ceph-mon[35338]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:46:25.196 INFO:journalctl@ceph.mgr.y.smithi114.stdout:Mar 27 10:46:24 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-y[36720]: 2024-03-27T10:46:24.920+0000 7f1917484200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-27T10:46:25.196 INFO:journalctl@ceph.mgr.y.smithi114.stdout:Mar 27 10:46:25 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-y[36720]: 2024-03-27T10:46:25.221+0000 7f1917484200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-27T10:46:26.522 INFO:journalctl@ceph.mgr.y.smithi114.stdout:Mar 27 10:46:26 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-y[36720]: 2024-03-27T10:46:26.232+0000 7f1917484200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-27T10:46:27.143 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:26 smithi097 ceph-mon[31859]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:46:27.269 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:26 smithi114 ceph-mon[35338]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:46:27.269 INFO:journalctl@ceph.mgr.y.smithi114.stdout:Mar 27 10:46:26 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-y[36720]: 2024-03-27T10:46:26.918+0000 7f1917484200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-27T10:46:27.269 INFO:journalctl@ceph.mgr.y.smithi114.stdout:Mar 27 10:46:27 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-y[36720]: 2024-03-27T10:46:27.045+0000 7f1917484200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-27T10:46:27.429 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:26 smithi177 ceph-mon[35294]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:46:27.769 INFO:journalctl@ceph.mgr.y.smithi114.stdout:Mar 27 10:46:27 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-y[36720]: 2024-03-27T10:46:27.325+0000 7f1917484200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-27T10:46:27.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:28.008+0000 7f8747400700 1 -- 172.21.15.97:0/1679624529 <== mon.0 v2:172.21.15.97:3300/0 8 ==== mgrmap(e 15) ==== 138314+0+0 (secure 0 0 0) 0x7f8740020c30 con 0x7f8750101560 2024-03-27T10:46:28.159 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:27 smithi097 ceph-mon[31859]: Standby manager daemon y started 2024-03-27T10:46:28.159 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:27 smithi097 ceph-mon[31859]: from='mgr.? 172.21.15.114:0/3624788731' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-27T10:46:28.159 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:27 smithi097 ceph-mon[31859]: from='mgr.? 172.21.15.114:0/3624788731' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-27T10:46:28.159 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:27 smithi097 ceph-mon[31859]: from='mgr.? 172.21.15.114:0/3624788731' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-27T10:46:28.159 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:27 smithi097 ceph-mon[31859]: from='mgr.? 172.21.15.114:0/3624788731' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-27T10:46:28.269 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:27 smithi114 ceph-mon[35338]: Standby manager daemon y started 2024-03-27T10:46:28.269 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:27 smithi114 ceph-mon[35338]: from='mgr.? 172.21.15.114:0/3624788731' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-27T10:46:28.269 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:27 smithi114 ceph-mon[35338]: from='mgr.? 172.21.15.114:0/3624788731' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-27T10:46:28.269 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:27 smithi114 ceph-mon[35338]: from='mgr.? 172.21.15.114:0/3624788731' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-27T10:46:28.269 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:27 smithi114 ceph-mon[35338]: from='mgr.? 172.21.15.114:0/3624788731' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-27T10:46:28.429 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:27 smithi177 ceph-mon[35294]: Standby manager daemon y started 2024-03-27T10:46:28.429 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:27 smithi177 ceph-mon[35294]: from='mgr.? 172.21.15.114:0/3624788731' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-27T10:46:28.429 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:27 smithi177 ceph-mon[35294]: from='mgr.? 172.21.15.114:0/3624788731' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-27T10:46:28.429 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:27 smithi177 ceph-mon[35294]: from='mgr.? 172.21.15.114:0/3624788731' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-27T10:46:28.429 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:27 smithi177 ceph-mon[35294]: from='mgr.? 172.21.15.114:0/3624788731' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-27T10:46:29.071 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:28 smithi097 ceph-mon[31859]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:46:29.072 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:28 smithi097 ceph-mon[31859]: mgrmap e15: x(active, since 108s), standbys: z, y 2024-03-27T10:46:29.072 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:28 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-27T10:46:29.072 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:28 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:29.072 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:28 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:29.269 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:28 smithi114 ceph-mon[35338]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:46:29.269 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:28 smithi114 ceph-mon[35338]: mgrmap e15: x(active, since 108s), standbys: z, y 2024-03-27T10:46:29.269 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:28 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-27T10:46:29.269 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:28 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:29.269 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:28 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:29.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:28 smithi177 ceph-mon[35294]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:46:29.429 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:28 smithi177 ceph-mon[35294]: mgrmap e15: x(active, since 108s), standbys: z, y 2024-03-27T10:46:29.429 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:28 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-27T10:46:29.429 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:28 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:29.429 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:28 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:30.143 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:30 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:46:30.269 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:30 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:46:30.429 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:30 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:46:31.247 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:31 smithi097 ceph-mon[31859]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:46:31.247 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:31 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:46:31.247 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:31 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:46:31.247 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:31 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:31.269 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:31 smithi114 ceph-mon[35338]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:46:31.269 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:31 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:46:31.269 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:31 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:46:31.269 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:31 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:31.430 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:31 smithi177 ceph-mon[35294]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:46:31.430 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:31 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:46:31.430 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:31 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:46:31.430 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:31 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:33.143 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:33 smithi097 ceph-mon[31859]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:46:33.143 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:33 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-27T10:46:33.143 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:33 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:46:33.143 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:33 smithi097 ceph-mon[31859]: Deploying daemon osd.0 on smithi097 2024-03-27T10:46:33.269 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:33 smithi114 ceph-mon[35338]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:46:33.269 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:33 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-27T10:46:33.269 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:33 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:46:33.269 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:33 smithi114 ceph-mon[35338]: Deploying daemon osd.0 on smithi097 2024-03-27T10:46:33.429 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:33 smithi177 ceph-mon[35294]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:46:33.429 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:33 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-27T10:46:33.429 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:33 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:46:33.429 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:33 smithi177 ceph-mon[35294]: Deploying daemon osd.0 on smithi097 2024-03-27T10:46:35.144 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:35 smithi097 ceph-mon[31859]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:46:35.269 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:35 smithi114 ceph-mon[35338]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:46:35.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:35 smithi177 ceph-mon[35294]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:46:37.238 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:37 smithi097 ceph-mon[31859]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:46:37.269 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:37 smithi114 ceph-mon[35338]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:46:37.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:37 smithi177 ceph-mon[35294]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:46:39.269 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:39 smithi114 ceph-mon[35338]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:46:39.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:39 smithi097 ceph-mon[31859]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:46:39.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:39 smithi177 ceph-mon[35294]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:46:41.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:40 smithi097 ceph-mon[31859]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:46:41.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:40 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:41.054 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:40 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:41.055 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:40 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:46:41.178 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:40 smithi177 ceph-mon[35294]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:46:41.179 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:40 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:41.179 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:40 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:41.179 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:40 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:46:41.269 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:40 smithi114 ceph-mon[35338]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:46:41.269 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:40 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:41.269 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:40 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:41.269 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:40 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:46:43.178 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:42 smithi177 ceph-mon[35294]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:46:43.269 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:42 smithi114 ceph-mon[35338]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:46:43.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:42 smithi097 ceph-mon[31859]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:46:44.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:44 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:44.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:44 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:44.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:44 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:46:44.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:44 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:46:44.520 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:44 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:44.520 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:44 smithi114 ceph-mon[35338]: from='osd.0 [v2:172.21.15.97:6802/1746812080,v1:172.21.15.97:6803/1746812080]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-27T10:46:44.532 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:44 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:44.532 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:44 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:44.532 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:44 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:46:44.532 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:44 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:46:44.532 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:44 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:44.532 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:44 smithi097 ceph-mon[31859]: from='osd.0 [v2:172.21.15.97:6802/1746812080,v1:172.21.15.97:6803/1746812080]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-27T10:46:44.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:44.677+0000 7f8747400700 1 -- 172.21.15.97:0/1679624529 <== mgr.14152 v2:172.21.15.97:6800/3955463538 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f87500fd900 con 0x7f87340396b0 2024-03-27T10:46:44.652 INFO:teuthology.orchestra.run.smithi097.stdout:Created osd(s) 0 on host 'smithi097' 2024-03-27T10:46:44.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:44.681+0000 7f8755600700 1 -- 172.21.15.97:0/1679624529 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f87340396b0 msgr2=0x7f873403bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:46:44.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:44.681+0000 7f8755600700 1 --2- 172.21.15.97:0/1679624529 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f87340396b0 0x7f873403bb70 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f873c0038f0 tx=0x7f873c005fb0 comp rx=0 tx=0).stop 2024-03-27T10:46:44.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:44.681+0000 7f8755600700 1 -- 172.21.15.97:0/1679624529 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8750101560 msgr2=0x7f87501987e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:46:44.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:44.681+0000 7f8755600700 1 --2- 172.21.15.97:0/1679624529 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8750101560 0x7f87501987e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f874000c920 tx=0x7f874000cc30 comp rx=0 tx=0).stop 2024-03-27T10:46:44.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:44.681+0000 7f8755600700 1 -- 172.21.15.97:0/1679624529 shutdown_connections 2024-03-27T10:46:44.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:44.681+0000 7f8755600700 1 --2- 172.21.15.97:0/1679624529 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f87340396b0 0x7f873403bb70 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:46:44.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:44.681+0000 7f8755600700 1 --2- 172.21.15.97:0/1679624529 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8750103100 0x7f875019fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:46:44.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:44.681+0000 7f8755600700 1 --2- 172.21.15.97:0/1679624529 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8750101560 0x7f87501987e0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:46:44.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:44.681+0000 7f8755600700 1 --2- 172.21.15.97:0/1679624529 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8750102760 0x7f8750198d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:46:44.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:44.681+0000 7f8755600700 1 -- 172.21.15.97:0/1679624529 >> 172.21.15.97:0/1679624529 conn(0x7f87500fcb00 msgr2=0x7f87500fd1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:46:44.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:44.681+0000 7f8755600700 1 -- 172.21.15.97:0/1679624529 shutdown_connections 2024-03-27T10:46:44.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:46:44.681+0000 7f8755600700 1 -- 172.21.15.97:0/1679624529 wait complete. 2024-03-27T10:46:44.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:44 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:44.679 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:44 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:44.679 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:44 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:46:44.679 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:44 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:46:44.679 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:44 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:44.679 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:44 smithi177 ceph-mon[35294]: from='osd.0 [v2:172.21.15.97:6802/1746812080,v1:172.21.15.97:6803/1746812080]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-27T10:46:45.258 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:45 smithi097 ceph-mon[31859]: pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:46:45.258 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:45 smithi097 ceph-mon[31859]: from='osd.0 [v2:172.21.15.97:6802/1746812080,v1:172.21.15.97:6803/1746812080]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-27T10:46:45.258 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:45 smithi097 ceph-mon[31859]: osdmap e6: 1 total, 0 up, 1 in 2024-03-27T10:46:45.258 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:45 smithi097 ceph-mon[31859]: from='osd.0 [v2:172.21.15.97:6802/1746812080,v1:172.21.15.97:6803/1746812080]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-03-27T10:46:45.258 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:45 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-27T10:46:45.258 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:45 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:45.258 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:45 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:46:45.258 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:45 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:45.386 DEBUG:teuthology.orchestra.run.smithi097:osd.0> sudo journalctl -f -n 0 -u ceph-adca88c4-ec26-11ee-95d0-87774f69a715@osd.0.service 2024-03-27T10:46:45.391 INFO:tasks.cephadm:Deploying osd.1 on smithi114 with /dev/vg_nvme/lv_4... 2024-03-27T10:46:45.391 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid adca88c4-ec26-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-27T10:46:45.479 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:45 smithi114 ceph-mon[35338]: pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:46:45.479 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:45 smithi114 ceph-mon[35338]: from='osd.0 [v2:172.21.15.97:6802/1746812080,v1:172.21.15.97:6803/1746812080]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-27T10:46:45.479 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:45 smithi114 ceph-mon[35338]: osdmap e6: 1 total, 0 up, 1 in 2024-03-27T10:46:45.479 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:45 smithi114 ceph-mon[35338]: from='osd.0 [v2:172.21.15.97:6802/1746812080,v1:172.21.15.97:6803/1746812080]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-03-27T10:46:45.479 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:45 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-27T10:46:45.479 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:45 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:45.479 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:45 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:46:45.480 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:45 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:45.679 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:45 smithi177 ceph-mon[35294]: pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:46:45.679 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:45 smithi177 ceph-mon[35294]: from='osd.0 [v2:172.21.15.97:6802/1746812080,v1:172.21.15.97:6803/1746812080]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-27T10:46:45.679 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:45 smithi177 ceph-mon[35294]: osdmap e6: 1 total, 0 up, 1 in 2024-03-27T10:46:45.679 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:45 smithi177 ceph-mon[35294]: from='osd.0 [v2:172.21.15.97:6802/1746812080,v1:172.21.15.97:6803/1746812080]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-03-27T10:46:45.679 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:45 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-27T10:46:45.679 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:45 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:45.679 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:45 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:46:45.679 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:45 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:45.835 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/mon.b/config 2024-03-27T10:46:46.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:46 smithi114 ceph-mon[35338]: from='osd.0 [v2:172.21.15.97:6802/1746812080,v1:172.21.15.97:6803/1746812080]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-03-27T10:46:46.520 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:46 smithi114 ceph-mon[35338]: osdmap e7: 1 total, 0 up, 1 in 2024-03-27T10:46:46.520 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:46 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-27T10:46:46.520 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:46 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-27T10:46:46.520 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:46 smithi114 ceph-mon[35338]: pgmap v60: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:46:46.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:46 smithi097 ceph-mon[31859]: from='osd.0 [v2:172.21.15.97:6802/1746812080,v1:172.21.15.97:6803/1746812080]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-03-27T10:46:46.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:46 smithi097 ceph-mon[31859]: osdmap e7: 1 total, 0 up, 1 in 2024-03-27T10:46:46.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:46 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-27T10:46:46.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:46 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-27T10:46:46.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:46 smithi097 ceph-mon[31859]: pgmap v60: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:46:46.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:46 smithi177 ceph-mon[35294]: from='osd.0 [v2:172.21.15.97:6802/1746812080,v1:172.21.15.97:6803/1746812080]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-03-27T10:46:46.679 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:46 smithi177 ceph-mon[35294]: osdmap e7: 1 total, 0 up, 1 in 2024-03-27T10:46:46.679 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:46 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-27T10:46:46.679 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:46 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-27T10:46:46.679 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:46 smithi177 ceph-mon[35294]: pgmap v60: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T10:46:47.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:47 smithi097 ceph-mon[31859]: purged_snaps scrub starts 2024-03-27T10:46:47.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:47 smithi097 ceph-mon[31859]: purged_snaps scrub ok 2024-03-27T10:46:47.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:47 smithi097 ceph-mon[31859]: osd.0 [v2:172.21.15.97:6802/1746812080,v1:172.21.15.97:6803/1746812080] boot 2024-03-27T10:46:47.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:47 smithi097 ceph-mon[31859]: osdmap e8: 1 total, 1 up, 1 in 2024-03-27T10:46:47.394 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:47 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-27T10:46:47.504 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:47 smithi114 ceph-mon[35338]: purged_snaps scrub starts 2024-03-27T10:46:47.504 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:47 smithi114 ceph-mon[35338]: purged_snaps scrub ok 2024-03-27T10:46:47.504 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:47 smithi114 ceph-mon[35338]: osd.0 [v2:172.21.15.97:6802/1746812080,v1:172.21.15.97:6803/1746812080] boot 2024-03-27T10:46:47.504 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:47 smithi114 ceph-mon[35338]: osdmap e8: 1 total, 1 up, 1 in 2024-03-27T10:46:47.504 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:47 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-27T10:46:47.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:47 smithi177 ceph-mon[35294]: purged_snaps scrub starts 2024-03-27T10:46:47.679 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:47 smithi177 ceph-mon[35294]: purged_snaps scrub ok 2024-03-27T10:46:47.679 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:47 smithi177 ceph-mon[35294]: osd.0 [v2:172.21.15.97:6802/1746812080,v1:172.21.15.97:6803/1746812080] boot 2024-03-27T10:46:47.679 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:47 smithi177 ceph-mon[35294]: osdmap e8: 1 total, 1 up, 1 in 2024-03-27T10:46:47.679 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:47 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-27T10:46:48.644 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:48 smithi097 ceph-mon[31859]: osdmap e9: 1 total, 1 up, 1 in 2024-03-27T10:46:48.644 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:48 smithi097 ceph-mon[31859]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-27T10:46:48.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:48 smithi177 ceph-mon[35294]: osdmap e9: 1 total, 1 up, 1 in 2024-03-27T10:46:48.679 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:48 smithi177 ceph-mon[35294]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-27T10:46:48.696 INFO:teuthology.orchestra.run.smithi114.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-27T10:46:48.696 INFO:teuthology.orchestra.run.smithi114.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-27T10:46:48.696 INFO:teuthology.orchestra.run.smithi114.stderr: stderr: 10+0 records in 2024-03-27T10:46:48.696 INFO:teuthology.orchestra.run.smithi114.stderr:10+0 records out 2024-03-27T10:46:48.696 INFO:teuthology.orchestra.run.smithi114.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0330265 s, 317 MB/s 2024-03-27T10:46:48.696 INFO:teuthology.orchestra.run.smithi114.stderr:--> Zapping successful for: 2024-03-27T10:46:48.697 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:48 smithi114 ceph-mon[35338]: osdmap e9: 1 total, 1 up, 1 in 2024-03-27T10:46:48.697 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:48 smithi114 ceph-mon[35338]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-27T10:46:49.561 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid adca88c4-ec26-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi114:vg_nvme/lv_4 2024-03-27T10:46:49.994 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/mon.b/config 2024-03-27T10:46:51.143 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:50 smithi097 ceph-mon[31859]: Detected new or changed devices on smithi097 2024-03-27T10:46:51.143 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:50 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:51.143 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:50 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:51.144 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:50 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-27T10:46:51.144 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:50 smithi097 ceph-mon[31859]: Adjusting osd_memory_target on smithi097 to 17091M 2024-03-27T10:46:51.144 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:50 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:51.144 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:50 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:46:51.144 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:50 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:46:51.144 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:50 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:51.144 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:50 smithi097 ceph-mon[31859]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-27T10:46:51.163 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:50 smithi114 ceph-mon[35338]: Detected new or changed devices on smithi097 2024-03-27T10:46:51.163 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:50 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:51.163 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:50 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:51.163 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:50 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-27T10:46:51.163 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:50 smithi114 ceph-mon[35338]: Adjusting osd_memory_target on smithi097 to 17091M 2024-03-27T10:46:51.163 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:50 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:51.163 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:50 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:46:51.163 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:50 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:46:51.163 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:50 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:51.164 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:50 smithi114 ceph-mon[35338]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-27T10:46:51.179 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:50 smithi177 ceph-mon[35294]: Detected new or changed devices on smithi097 2024-03-27T10:46:51.179 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:50 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:51.179 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:50 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:51.179 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:50 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-27T10:46:51.179 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:50 smithi177 ceph-mon[35294]: Adjusting osd_memory_target on smithi097 to 17091M 2024-03-27T10:46:51.179 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:50 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:51.179 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:50 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:46:51.179 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:50 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:46:51.179 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:50 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:46:51.179 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:50 smithi177 ceph-mon[35294]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-27T10:46:52.096 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:46:52.120+0000 7f8bbb000700 1 -- 172.21.15.114:0/606910002 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8bb4106460 msgr2=0x7f8bb410cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:46:52.096 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:46:52.120+0000 7f8bbb000700 1 --2- 172.21.15.114:0/606910002 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8bb4106460 0x7f8bb410cd10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8bac00b0d0 tx=0x7f8bac00b3e0 comp rx=0 tx=0).stop 2024-03-27T10:46:52.096 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:46:52.121+0000 7f8bbb000700 1 -- 172.21.15.114:0/606910002 shutdown_connections 2024-03-27T10:46:52.096 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:46:52.121+0000 7f8bbb000700 1 --2- 172.21.15.114:0/606910002 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8bb4106460 0x7f8bb410cd10 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:46:52.096 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:46:52.121+0000 7f8bbb000700 1 --2- 172.21.15.114:0/606910002 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8bb41048c0 0x7f8bb4104ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:46:52.096 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:46:52.121+0000 7f8bbb000700 1 --2- 172.21.15.114:0/606910002 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8bb4105ac0 0x7f8bb4105f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:46:52.096 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:46:52.121+0000 7f8bbb000700 1 -- 172.21.15.114:0/606910002 >> 172.21.15.114:0/606910002 conn(0x7f8bb40ffe60 msgr2=0x7f8bb41022a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:46:52.096 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:46:52.121+0000 7f8bbb000700 1 -- 172.21.15.114:0/606910002 shutdown_connections 2024-03-27T10:46:52.096 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:46:52.121+0000 7f8bbb000700 1 -- 172.21.15.114:0/606910002 wait complete. 2024-03-27T10:46:52.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:46:52.122+0000 7f8bbb000700 1 Processor -- start 2024-03-27T10:46:52.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:46:52.122+0000 7f8bbb000700 1 -- start start 2024-03-27T10:46:52.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:46:52.122+0000 7f8bbb000700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8bb41048c0 0x7f8bb419aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:46:52.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:46:52.122+0000 7f8bbb000700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8bb4105ac0 0x7f8bb419b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:46:52.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:46:52.122+0000 7f8bbb000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8bb4106460 0x7f8bb41a2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:46:52.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:46:52.122+0000 7f8bbb000700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb4110450 con 0x7f8bb4106460 2024-03-27T10:46:52.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:46:52.122+0000 7f8bbb000700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb41102d0 con 0x7f8bb41048c0 2024-03-27T10:46:52.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:46:52.122+0000 7f8bbb000700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb41105d0 con 0x7f8bb4105ac0 2024-03-27T10:46:52.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:46:52.122+0000 7f8bb3e00700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8bb41048c0 0x7f8bb419aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:46:52.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:46:52.123+0000 7f8bb3e00700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8bb41048c0 0x7f8bb419aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:59906/0 (socket says 172.21.15.114:59906) 2024-03-27T10:46:52.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:46:52.123+0000 7f8bb3e00700 1 -- 172.21.15.114:0/2348032035 learned_addr learned my addr 172.21.15.114:0/2348032035 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-03-27T10:46:52.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:46:52.123+0000 7f8bb9000700 1 --2- 172.21.15.114:0/2348032035 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8bb4106460 0x7f8bb41a2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:46:52.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:46:52.123+0000 7f8bb3400700 1 --2- 172.21.15.114:0/2348032035 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8bb4105ac0 0x7f8bb419b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:46:52.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:46:52.123+0000 7f8bb3e00700 1 -- 172.21.15.114:0/2348032035 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8bb4105ac0 msgr2=0x7f8bb419b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:46:52.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:46:52.123+0000 7f8bb3e00700 1 --2- 172.21.15.114:0/2348032035 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8bb4105ac0 0x7f8bb419b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:46:52.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:46:52.123+0000 7f8bb3e00700 1 -- 172.21.15.114:0/2348032035 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8bb4106460 msgr2=0x7f8bb41a2080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:46:52.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:46:52.123+0000 7f8bb3e00700 1 --2- 172.21.15.114:0/2348032035 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8bb4106460 0x7f8bb41a2080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:46:52.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:46:52.123+0000 7f8bb3e00700 1 -- 172.21.15.114:0/2348032035 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bb41a2720 con 0x7f8bb41048c0 2024-03-27T10:46:52.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:46:52.123+0000 7f8bb3400700 1 --2- 172.21.15.114:0/2348032035 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8bb4105ac0 0x7f8bb419b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:46:52.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:46:52.123+0000 7f8bb3e00700 1 --2- 172.21.15.114:0/2348032035 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8bb41048c0 0x7f8bb419aac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8ba400b500 tx=0x7f8ba400b8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:46:52.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:46:52.124+0000 7f8bb0c00700 1 -- 172.21.15.114:0/2348032035 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba4015020 con 0x7f8bb41048c0 2024-03-27T10:46:52.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:46:52.124+0000 7f8bbb000700 1 -- 172.21.15.114:0/2348032035 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bb41a2a10 con 0x7f8bb41048c0 2024-03-27T10:46:52.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:46:52.124+0000 7f8bb0c00700 1 -- 172.21.15.114:0/2348032035 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8ba4011ce0 con 0x7f8bb41048c0 2024-03-27T10:46:52.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:46:52.124+0000 7f8bb0c00700 1 -- 172.21.15.114:0/2348032035 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba4010d90 con 0x7f8bb41048c0 2024-03-27T10:46:52.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:46:52.124+0000 7f8bbb000700 1 -- 172.21.15.114:0/2348032035 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bb41a2fa0 con 0x7f8bb41048c0 2024-03-27T10:46:52.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:46:52.125+0000 7f8bbb000700 1 -- 172.21.15.114:0/2348032035 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bb4069300 con 0x7f8bb41048c0 2024-03-27T10:46:52.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:46:52.127+0000 7f8bb0c00700 1 -- 172.21.15.114:0/2348032035 <== mon.2 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 15) ==== 138314+0+0 (secure 0 0 0) 0x7f8ba4018020 con 0x7f8bb41048c0 2024-03-27T10:46:52.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:46:52.129+0000 7f8bb0c00700 1 --2- 172.21.15.114:0/2348032035 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f8b980a3610 0x7f8b980a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:46:52.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:46:52.129+0000 7f8bb0c00700 1 -- 172.21.15.114:0/2348032035 <== mon.2 v2:172.21.15.114:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7f8ba40ce0b0 con 0x7f8bb41048c0 2024-03-27T10:46:52.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:46:52.129+0000 7f8bb3400700 1 --2- 172.21.15.114:0/2348032035 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f8b980a3610 0x7f8b980a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:46:52.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:46:52.130+0000 7f8bb3400700 1 --2- 172.21.15.114:0/2348032035 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f8b980a3610 0x7f8b980a5ad0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8ba0007900 tx=0x7f8ba0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:46:52.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:46:52.132+0000 7f8bb0c00700 1 -- 172.21.15.114:0/2348032035 <== mon.2 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f8ba401f020 con 0x7f8bb41048c0 2024-03-27T10:46:52.340 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:46:52.364+0000 7f8bbb000700 1 -- 172.21.15.114:0/2348032035 --> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi114:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f8bb4062a80 con 0x7f8b980a3610 2024-03-27T10:46:53.269 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:52 smithi114 ceph-mon[35338]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-27T10:46:53.269 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:52 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-27T10:46:53.269 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:52 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-27T10:46:53.269 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:52 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:46:53.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:52 smithi097 ceph-mon[31859]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-27T10:46:53.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:52 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-27T10:46:53.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:52 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-27T10:46:53.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:52 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:46:53.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:52 smithi177 ceph-mon[35294]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-27T10:46:53.429 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:52 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-27T10:46:53.429 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:52 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-27T10:46:53.429 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:52 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:46:54.269 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:53 smithi114 ceph-mon[35338]: from='client.34136 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi114:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T10:46:54.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:53 smithi097 ceph-mon[31859]: from='client.34136 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi114:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T10:46:54.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:53 smithi177 ceph-mon[35294]: from='client.34136 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi114:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T10:46:55.269 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:54 smithi114 ceph-mon[35338]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-27T10:46:55.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:54 smithi097 ceph-mon[31859]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-27T10:46:55.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:54 smithi177 ceph-mon[35294]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-27T10:46:57.269 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:57 smithi114 ceph-mon[35338]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-27T10:46:57.270 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:57 smithi114 ceph-mon[35338]: from='client.? 172.21.15.114:0/770555998' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "441e2770-3284-4388-b0de-c56eb8a373d0"}]: dispatch 2024-03-27T10:46:57.270 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:57 smithi114 ceph-mon[35338]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "441e2770-3284-4388-b0de-c56eb8a373d0"}]: dispatch 2024-03-27T10:46:57.270 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:57 smithi114 ceph-mon[35338]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "441e2770-3284-4388-b0de-c56eb8a373d0"}]': finished 2024-03-27T10:46:57.270 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:57 smithi114 ceph-mon[35338]: osdmap e10: 2 total, 1 up, 2 in 2024-03-27T10:46:57.270 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:57 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-27T10:46:57.270 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:57 smithi114 ceph-mon[35338]: from='client.? 172.21.15.114:0/134414826' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-27T10:46:57.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:57 smithi097 ceph-mon[31859]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-27T10:46:57.394 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:57 smithi097 ceph-mon[31859]: from='client.? 172.21.15.114:0/770555998' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "441e2770-3284-4388-b0de-c56eb8a373d0"}]: dispatch 2024-03-27T10:46:57.394 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:57 smithi097 ceph-mon[31859]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "441e2770-3284-4388-b0de-c56eb8a373d0"}]: dispatch 2024-03-27T10:46:57.394 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:57 smithi097 ceph-mon[31859]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "441e2770-3284-4388-b0de-c56eb8a373d0"}]': finished 2024-03-27T10:46:57.394 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:57 smithi097 ceph-mon[31859]: osdmap e10: 2 total, 1 up, 2 in 2024-03-27T10:46:57.394 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:57 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-27T10:46:57.395 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:57 smithi097 ceph-mon[31859]: from='client.? 172.21.15.114:0/134414826' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-27T10:46:57.430 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:57 smithi177 ceph-mon[35294]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-27T10:46:57.431 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:57 smithi177 ceph-mon[35294]: from='client.? 172.21.15.114:0/770555998' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "441e2770-3284-4388-b0de-c56eb8a373d0"}]: dispatch 2024-03-27T10:46:57.431 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:57 smithi177 ceph-mon[35294]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "441e2770-3284-4388-b0de-c56eb8a373d0"}]: dispatch 2024-03-27T10:46:57.431 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:57 smithi177 ceph-mon[35294]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "441e2770-3284-4388-b0de-c56eb8a373d0"}]': finished 2024-03-27T10:46:57.431 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:57 smithi177 ceph-mon[35294]: osdmap e10: 2 total, 1 up, 2 in 2024-03-27T10:46:57.431 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:57 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-27T10:46:57.431 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:57 smithi177 ceph-mon[35294]: from='client.? 172.21.15.114:0/134414826' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-27T10:46:59.269 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:46:59 smithi114 ceph-mon[35338]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-27T10:46:59.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:46:59 smithi097 ceph-mon[31859]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-27T10:46:59.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:46:59 smithi177 ceph-mon[35294]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-27T10:47:01.269 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:01 smithi114 ceph-mon[35338]: pgmap v70: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-27T10:47:01.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:01 smithi097 ceph-mon[31859]: pgmap v70: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-27T10:47:01.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:01 smithi177 ceph-mon[35294]: pgmap v70: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-27T10:47:03.122 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:03 smithi114 ceph-mon[35338]: pgmap v71: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-27T10:47:03.143 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:03 smithi097 ceph-mon[31859]: pgmap v71: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-27T10:47:03.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:03 smithi177 ceph-mon[35294]: pgmap v71: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-27T10:47:05.234 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:05 smithi114 ceph-mon[35338]: pgmap v72: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-27T10:47:05.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:05 smithi097 ceph-mon[31859]: pgmap v72: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-27T10:47:05.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:05 smithi177 ceph-mon[35294]: pgmap v72: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-27T10:47:06.269 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:06 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-27T10:47:06.269 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:06 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:47:06.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:06 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-27T10:47:06.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:06 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:47:06.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:06 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-27T10:47:06.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:06 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:47:07.269 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:07 smithi114 ceph-mon[35338]: Deploying daemon osd.1 on smithi114 2024-03-27T10:47:07.269 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:07 smithi114 ceph-mon[35338]: pgmap v73: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-27T10:47:07.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:07 smithi097 ceph-mon[31859]: Deploying daemon osd.1 on smithi114 2024-03-27T10:47:07.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:07 smithi097 ceph-mon[31859]: pgmap v73: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-27T10:47:07.429 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:07 smithi177 ceph-mon[35294]: Deploying daemon osd.1 on smithi114 2024-03-27T10:47:07.429 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:07 smithi177 ceph-mon[35294]: pgmap v73: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-27T10:47:09.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:09 smithi097 ceph-mon[31859]: pgmap v74: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-27T10:47:09.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:09 smithi177 ceph-mon[35294]: pgmap v74: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-27T10:47:09.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:09 smithi114 ceph-mon[35338]: pgmap v74: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-27T10:47:11.226 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:11 smithi114 ceph-mon[35338]: pgmap v75: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-27T10:47:11.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:11 smithi097 ceph-mon[31859]: pgmap v75: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-27T10:47:11.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:11 smithi177 ceph-mon[35294]: pgmap v75: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-27T10:47:13.146 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:13 smithi114 ceph-mon[35338]: pgmap v76: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-27T10:47:13.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:13 smithi097 ceph-mon[31859]: pgmap v76: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-27T10:47:13.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:13 smithi177 ceph-mon[35294]: pgmap v76: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-27T10:47:14.769 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:14 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:47:14.769 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:14 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:47:14.769 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:14 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:47:14.769 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:14 smithi114 ceph-mon[35338]: pgmap v77: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-27T10:47:14.928 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:14 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:47:14.929 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:14 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:47:14.929 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:14 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:47:14.929 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:14 smithi177 ceph-mon[35294]: pgmap v77: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-27T10:47:15.143 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:14 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:47:15.143 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:14 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:47:15.143 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:14 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:47:15.143 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:14 smithi097 ceph-mon[31859]: pgmap v77: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-27T10:47:17.268 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:16 smithi114 ceph-mon[35338]: pgmap v78: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-27T10:47:17.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:16 smithi097 ceph-mon[31859]: pgmap v78: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-27T10:47:17.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:16 smithi177 ceph-mon[35294]: pgmap v78: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-27T10:47:18.458 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:18 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:47:18.458 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:18 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:47:18.458 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:18 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:47:18.458 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:18 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:47:18.458 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:18 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:47:18.458 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:18 smithi114 ceph-mon[35338]: from='osd.1 [v2:172.21.15.114:6800/3858821264,v1:172.21.15.114:6801/3858821264]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-27T10:47:18.458 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:18 smithi114 ceph-mon[35338]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-27T10:47:18.459 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:18 smithi114 ceph-mon[35338]: pgmap v79: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-27T10:47:18.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:47:18.568+0000 7f8bb0c00700 1 -- 172.21.15.114:0/2348032035 <== mgr.14152 v2:172.21.15.97:6800/3955463538 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8bb4062a80 con 0x7f8b980a3610 2024-03-27T10:47:18.543 INFO:teuthology.orchestra.run.smithi114.stdout:Created osd(s) 1 on host 'smithi114' 2024-03-27T10:47:18.546 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:47:18.572+0000 7f8bbb000700 1 -- 172.21.15.114:0/2348032035 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f8b980a3610 msgr2=0x7f8b980a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:47:18.546 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:47:18.572+0000 7f8bbb000700 1 --2- 172.21.15.114:0/2348032035 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f8b980a3610 0x7f8b980a5ad0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8ba0007900 tx=0x7f8ba0008040 comp rx=0 tx=0).stop 2024-03-27T10:47:18.546 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:47:18.572+0000 7f8bbb000700 1 -- 172.21.15.114:0/2348032035 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8bb41048c0 msgr2=0x7f8bb419aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:47:18.546 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:47:18.572+0000 7f8bbb000700 1 --2- 172.21.15.114:0/2348032035 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8bb41048c0 0x7f8bb419aac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8ba400b500 tx=0x7f8ba400b8c0 comp rx=0 tx=0).stop 2024-03-27T10:47:18.546 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:47:18.572+0000 7f8bbb000700 1 -- 172.21.15.114:0/2348032035 shutdown_connections 2024-03-27T10:47:18.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:47:18.572+0000 7f8bbb000700 1 --2- 172.21.15.114:0/2348032035 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f8b980a3610 0x7f8b980a5ad0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:47:18.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:47:18.572+0000 7f8bbb000700 1 --2- 172.21.15.114:0/2348032035 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8bb4106460 0x7f8bb41a2080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:47:18.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:47:18.572+0000 7f8bbb000700 1 --2- 172.21.15.114:0/2348032035 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8bb41048c0 0x7f8bb419aac0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:47:18.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:47:18.572+0000 7f8bbb000700 1 --2- 172.21.15.114:0/2348032035 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8bb4105ac0 0x7f8bb419b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:47:18.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:47:18.572+0000 7f8bbb000700 1 -- 172.21.15.114:0/2348032035 >> 172.21.15.114:0/2348032035 conn(0x7f8bb40ffe60 msgr2=0x7f8bb4108dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:47:18.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:47:18.572+0000 7f8bbb000700 1 -- 172.21.15.114:0/2348032035 shutdown_connections 2024-03-27T10:47:18.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:47:18.572+0000 7f8bbb000700 1 -- 172.21.15.114:0/2348032035 wait complete. 2024-03-27T10:47:18.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:18 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:47:18.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:18 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:47:18.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:18 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:47:18.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:18 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:47:18.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:18 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:47:18.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:18 smithi097 ceph-mon[31859]: from='osd.1 [v2:172.21.15.114:6800/3858821264,v1:172.21.15.114:6801/3858821264]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-27T10:47:18.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:18 smithi097 ceph-mon[31859]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-27T10:47:18.644 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:18 smithi097 ceph-mon[31859]: pgmap v79: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-27T10:47:18.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:18 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:47:18.679 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:18 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:47:18.679 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:18 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:47:18.679 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:18 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:47:18.679 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:18 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:47:18.679 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:18 smithi177 ceph-mon[35294]: from='osd.1 [v2:172.21.15.114:6800/3858821264,v1:172.21.15.114:6801/3858821264]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-27T10:47:18.679 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:18 smithi177 ceph-mon[35294]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-27T10:47:18.679 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:18 smithi177 ceph-mon[35294]: pgmap v79: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-27T10:47:19.332 DEBUG:teuthology.orchestra.run.smithi114:osd.1> sudo journalctl -f -n 0 -u ceph-adca88c4-ec26-11ee-95d0-87774f69a715@osd.1.service 2024-03-27T10:47:19.338 INFO:tasks.cephadm:Deploying osd.2 on smithi177 with /dev/vg_nvme/lv_4... 2024-03-27T10:47:19.338 DEBUG:teuthology.orchestra.run.smithi177:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid adca88c4-ec26-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-27T10:47:19.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:19 smithi114 ceph-mon[35338]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-27T10:47:19.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:19 smithi114 ceph-mon[35338]: osdmap e11: 2 total, 1 up, 2 in 2024-03-27T10:47:19.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:19 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-27T10:47:19.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:19 smithi114 ceph-mon[35338]: from='osd.1 [v2:172.21.15.114:6800/3858821264,v1:172.21.15.114:6801/3858821264]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]: dispatch 2024-03-27T10:47:19.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:19 smithi114 ceph-mon[35338]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]: dispatch 2024-03-27T10:47:19.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:19 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:47:19.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:19 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:47:19.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:19 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:47:19.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:19 smithi097 ceph-mon[31859]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-27T10:47:19.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:19 smithi097 ceph-mon[31859]: osdmap e11: 2 total, 1 up, 2 in 2024-03-27T10:47:19.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:19 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-27T10:47:19.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:19 smithi097 ceph-mon[31859]: from='osd.1 [v2:172.21.15.114:6800/3858821264,v1:172.21.15.114:6801/3858821264]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]: dispatch 2024-03-27T10:47:19.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:19 smithi097 ceph-mon[31859]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]: dispatch 2024-03-27T10:47:19.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:19 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:47:19.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:19 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:47:19.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:19 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:47:19.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:19 smithi177 ceph-mon[35294]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-27T10:47:19.679 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:19 smithi177 ceph-mon[35294]: osdmap e11: 2 total, 1 up, 2 in 2024-03-27T10:47:19.679 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:19 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-27T10:47:19.679 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:19 smithi177 ceph-mon[35294]: from='osd.1 [v2:172.21.15.114:6800/3858821264,v1:172.21.15.114:6801/3858821264]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]: dispatch 2024-03-27T10:47:19.679 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:19 smithi177 ceph-mon[35294]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]: dispatch 2024-03-27T10:47:19.679 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:19 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:47:19.679 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:19 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:47:19.679 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:19 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:47:19.779 INFO:teuthology.orchestra.run.smithi177.stderr:Inferring config /var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/mon.c/config 2024-03-27T10:47:20.621 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:20 smithi177 ceph-mon[35294]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]': finished 2024-03-27T10:47:20.621 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:20 smithi177 ceph-mon[35294]: osdmap e12: 2 total, 1 up, 2 in 2024-03-27T10:47:20.621 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:20 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-27T10:47:20.622 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:20 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-27T10:47:20.622 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:20 smithi177 ceph-mon[35294]: pgmap v82: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-27T10:47:20.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:20 smithi097 ceph-mon[31859]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]': finished 2024-03-27T10:47:20.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:20 smithi097 ceph-mon[31859]: osdmap e12: 2 total, 1 up, 2 in 2024-03-27T10:47:20.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:20 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-27T10:47:20.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:20 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-27T10:47:20.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:20 smithi097 ceph-mon[31859]: pgmap v82: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-27T10:47:20.769 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:20 smithi114 ceph-mon[35338]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]': finished 2024-03-27T10:47:20.769 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:20 smithi114 ceph-mon[35338]: osdmap e12: 2 total, 1 up, 2 in 2024-03-27T10:47:20.769 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:20 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-27T10:47:20.770 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:20 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-27T10:47:20.770 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:20 smithi114 ceph-mon[35338]: pgmap v82: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-27T10:47:21.584 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:21 smithi177 ceph-mon[35294]: purged_snaps scrub starts 2024-03-27T10:47:21.584 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:21 smithi177 ceph-mon[35294]: purged_snaps scrub ok 2024-03-27T10:47:21.584 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:21 smithi177 ceph-mon[35294]: osd.1 [v2:172.21.15.114:6800/3858821264,v1:172.21.15.114:6801/3858821264] boot 2024-03-27T10:47:21.584 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:21 smithi177 ceph-mon[35294]: osdmap e13: 2 total, 2 up, 2 in 2024-03-27T10:47:21.584 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:21 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-27T10:47:21.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:21 smithi097 ceph-mon[31859]: purged_snaps scrub starts 2024-03-27T10:47:21.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:21 smithi097 ceph-mon[31859]: purged_snaps scrub ok 2024-03-27T10:47:21.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:21 smithi097 ceph-mon[31859]: osd.1 [v2:172.21.15.114:6800/3858821264,v1:172.21.15.114:6801/3858821264] boot 2024-03-27T10:47:21.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:21 smithi097 ceph-mon[31859]: osdmap e13: 2 total, 2 up, 2 in 2024-03-27T10:47:21.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:21 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-27T10:47:21.769 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:21 smithi114 ceph-mon[35338]: purged_snaps scrub starts 2024-03-27T10:47:21.769 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:21 smithi114 ceph-mon[35338]: purged_snaps scrub ok 2024-03-27T10:47:21.769 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:21 smithi114 ceph-mon[35338]: osd.1 [v2:172.21.15.114:6800/3858821264,v1:172.21.15.114:6801/3858821264] boot 2024-03-27T10:47:21.769 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:21 smithi114 ceph-mon[35338]: osdmap e13: 2 total, 2 up, 2 in 2024-03-27T10:47:21.770 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:21 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-27T10:47:22.460 INFO:teuthology.orchestra.run.smithi177.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-27T10:47:22.460 INFO:teuthology.orchestra.run.smithi177.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-27T10:47:22.460 INFO:teuthology.orchestra.run.smithi177.stderr: stderr: 10+0 records in 2024-03-27T10:47:22.460 INFO:teuthology.orchestra.run.smithi177.stderr:10+0 records out 2024-03-27T10:47:22.460 INFO:teuthology.orchestra.run.smithi177.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0355902 s, 295 MB/s 2024-03-27T10:47:22.460 INFO:teuthology.orchestra.run.smithi177.stderr:--> Zapping successful for: 2024-03-27T10:47:22.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:22 smithi177 ceph-mon[35294]: osdmap e14: 2 total, 2 up, 2 in 2024-03-27T10:47:22.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:22 smithi177 ceph-mon[35294]: pgmap v85: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-27T10:47:22.769 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:22 smithi114 ceph-mon[35338]: osdmap e14: 2 total, 2 up, 2 in 2024-03-27T10:47:22.769 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:22 smithi114 ceph-mon[35338]: pgmap v85: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-27T10:47:22.892 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:22 smithi097 ceph-mon[31859]: osdmap e14: 2 total, 2 up, 2 in 2024-03-27T10:47:22.893 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:22 smithi097 ceph-mon[31859]: pgmap v85: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-27T10:47:23.241 DEBUG:teuthology.orchestra.run.smithi177:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid adca88c4-ec26-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi177:vg_nvme/lv_4 2024-03-27T10:47:23.663 INFO:teuthology.orchestra.run.smithi177.stderr:Inferring config /var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/mon.c/config 2024-03-27T10:47:25.078 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:25 smithi177 ceph-mon[35294]: pgmap v86: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-27T10:47:25.078 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:25 smithi177 ceph-mon[35294]: Detected new or changed devices on smithi114 2024-03-27T10:47:25.078 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:25 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:47:25.078 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:25 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:47:25.078 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:25 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-27T10:47:25.078 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:25 smithi177 ceph-mon[35294]: Adjusting osd_memory_target on smithi114 to 17091M 2024-03-27T10:47:25.078 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:25 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:47:25.078 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:25 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:47:25.079 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:25 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:47:25.079 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:25 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:47:25.269 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:25 smithi114 ceph-mon[35338]: pgmap v86: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-27T10:47:25.269 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:25 smithi114 ceph-mon[35338]: Detected new or changed devices on smithi114 2024-03-27T10:47:25.269 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:25 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:47:25.269 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:25 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:47:25.270 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:25 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-27T10:47:25.270 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:25 smithi114 ceph-mon[35338]: Adjusting osd_memory_target on smithi114 to 17091M 2024-03-27T10:47:25.270 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:25 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:47:25.270 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:25 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:47:25.270 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:25 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:47:25.270 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:25 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:47:25.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:25 smithi097 ceph-mon[31859]: pgmap v86: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-27T10:47:25.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:25 smithi097 ceph-mon[31859]: Detected new or changed devices on smithi114 2024-03-27T10:47:25.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:25 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:47:25.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:25 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:47:25.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:25 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-27T10:47:25.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:25 smithi097 ceph-mon[31859]: Adjusting osd_memory_target on smithi114 to 17091M 2024-03-27T10:47:25.394 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:25 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:47:25.394 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:25 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:47:25.394 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:25 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:47:25.394 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:25 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:47:25.547 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:47:25.572+0000 7f68c1000700 1 -- 172.21.15.177:0/2276435215 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f68bc103850 msgr2=0x7f68bc105ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:47:25.547 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:47:25.572+0000 7f68c1000700 1 --2- 172.21.15.177:0/2276435215 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f68bc103850 0x7f68bc105ce0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f68a4009990 tx=0x7f68a4009ca0 comp rx=0 tx=0).stop 2024-03-27T10:47:25.547 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:47:25.572+0000 7f68c1000700 1 -- 172.21.15.177:0/2276435215 shutdown_connections 2024-03-27T10:47:25.547 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:47:25.572+0000 7f68c1000700 1 --2- 172.21.15.177:0/2276435215 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68bc106220 0x7f68bc1066a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:47:25.547 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:47:25.572+0000 7f68c1000700 1 --2- 172.21.15.177:0/2276435215 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f68bc100ee0 0x7f68bc103310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:47:25.547 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:47:25.572+0000 7f68c1000700 1 --2- 172.21.15.177:0/2276435215 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f68bc103850 0x7f68bc105ce0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:47:25.547 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:47:25.572+0000 7f68c1000700 1 -- 172.21.15.177:0/2276435215 >> 172.21.15.177:0/2276435215 conn(0x7f68bc0fcb00 msgr2=0x7f68bc0fef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:47:25.547 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:47:25.572+0000 7f68c1000700 1 -- 172.21.15.177:0/2276435215 shutdown_connections 2024-03-27T10:47:25.547 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:47:25.572+0000 7f68c1000700 1 -- 172.21.15.177:0/2276435215 wait complete. 2024-03-27T10:47:25.549 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:47:25.573+0000 7f68c1000700 1 Processor -- start 2024-03-27T10:47:25.549 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:47:25.573+0000 7f68c1000700 1 -- start start 2024-03-27T10:47:25.549 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:47:25.573+0000 7f68c1000700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f68bc100ee0 0x7f68bc19ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:47:25.550 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:47:25.573+0000 7f68c1000700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f68bc103850 0x7f68bc19f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:47:25.550 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:47:25.573+0000 7f68c1000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68bc106220 0x7f68bc1a61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:47:25.550 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:47:25.573+0000 7f68c1000700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f68bc1146a0 con 0x7f68bc106220 2024-03-27T10:47:25.550 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:47:25.573+0000 7f68c1000700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f68bc114520 con 0x7f68bc103850 2024-03-27T10:47:25.550 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:47:25.573+0000 7f68c1000700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f68bc114820 con 0x7f68bc100ee0 2024-03-27T10:47:25.550 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:47:25.574+0000 7f68b9e00700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f68bc100ee0 0x7f68bc19ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:47:25.550 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:47:25.574+0000 7f68b9e00700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f68bc100ee0 0x7f68bc19ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:57418/0 (socket says 172.21.15.177:57418) 2024-03-27T10:47:25.550 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:47:25.574+0000 7f68b9e00700 1 -- 172.21.15.177:0/3388284691 learned_addr learned my addr 172.21.15.177:0/3388284691 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-03-27T10:47:25.550 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:47:25.574+0000 7f68b3e00700 1 --2- 172.21.15.177:0/3388284691 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f68bc103850 0x7f68bc19f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:47:25.550 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:47:25.574+0000 7f68b9e00700 1 -- 172.21.15.177:0/3388284691 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f68bc103850 msgr2=0x7f68bc19f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:47:25.551 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:47:25.574+0000 7f68ba800700 1 --2- 172.21.15.177:0/3388284691 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68bc106220 0x7f68bc1a61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:47:25.551 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:47:25.574+0000 7f68b9e00700 1 --2- 172.21.15.177:0/3388284691 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f68bc103850 0x7f68bc19f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:47:25.551 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:47:25.574+0000 7f68b9e00700 1 -- 172.21.15.177:0/3388284691 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68bc106220 msgr2=0x7f68bc1a61f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:47:25.551 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:47:25.574+0000 7f68b9e00700 1 --2- 172.21.15.177:0/3388284691 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68bc106220 0x7f68bc1a61f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:47:25.551 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:47:25.574+0000 7f68b9e00700 1 -- 172.21.15.177:0/3388284691 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68bc1a68c0 con 0x7f68bc100ee0 2024-03-27T10:47:25.551 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:47:25.574+0000 7f68b3e00700 1 --2- 172.21.15.177:0/3388284691 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f68bc103850 0x7f68bc19f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:47:25.551 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:47:25.574+0000 7f68ba800700 1 --2- 172.21.15.177:0/3388284691 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68bc106220 0x7f68bc1a61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:47:25.551 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:47:25.575+0000 7f68b9e00700 1 --2- 172.21.15.177:0/3388284691 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f68bc100ee0 0x7f68bc19ec70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f68ac00c9b0 tx=0x7f68ac00cd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:47:25.551 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:47:25.575+0000 7f68b2a00700 1 -- 172.21.15.177:0/3388284691 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68ac007b00 con 0x7f68bc100ee0 2024-03-27T10:47:25.551 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:47:25.575+0000 7f68c1000700 1 -- 172.21.15.177:0/3388284691 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68bc1a6bb0 con 0x7f68bc100ee0 2024-03-27T10:47:25.551 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:47:25.575+0000 7f68b2a00700 1 -- 172.21.15.177:0/3388284691 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f68ac007ca0 con 0x7f68bc100ee0 2024-03-27T10:47:25.551 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:47:25.575+0000 7f68b2a00700 1 -- 172.21.15.177:0/3388284691 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68ac018730 con 0x7f68bc100ee0 2024-03-27T10:47:25.551 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:47:25.575+0000 7f68c1000700 1 -- 172.21.15.177:0/3388284691 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68bc1a7140 con 0x7f68bc100ee0 2024-03-27T10:47:25.552 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:47:25.576+0000 7f68c1000700 1 -- 172.21.15.177:0/3388284691 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68bc069300 con 0x7f68bc100ee0 2024-03-27T10:47:25.554 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:47:25.580+0000 7f68b2a00700 1 -- 172.21.15.177:0/3388284691 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 15) ==== 138314+0+0 (secure 0 0 0) 0x7f68ac00f450 con 0x7f68bc100ee0 2024-03-27T10:47:25.556 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:47:25.581+0000 7f68b2a00700 1 --2- 172.21.15.177:0/3388284691 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f68a00a3710 0x7f68a00a5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:47:25.556 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:47:25.582+0000 7f68b2a00700 1 -- 172.21.15.177:0/3388284691 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7f68ac0ce800 con 0x7f68bc100ee0 2024-03-27T10:47:25.556 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:47:25.582+0000 7f68b3e00700 1 --2- 172.21.15.177:0/3388284691 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f68a00a3710 0x7f68a00a5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:47:25.557 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:47:25.583+0000 7f68b3e00700 1 --2- 172.21.15.177:0/3388284691 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f68a00a3710 0x7f68a00a5bd0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f68a4005f50 tx=0x7f68a4005e80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:47:25.559 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:47:25.585+0000 7f68b2a00700 1 -- 172.21.15.177:0/3388284691 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f68ac014030 con 0x7f68bc100ee0 2024-03-27T10:47:25.796 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:47:25.821+0000 7f68c1000700 1 -- 172.21.15.177:0/3388284691 --> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi177:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f68bc062a80 con 0x7f68a00a3710 2024-03-27T10:47:26.178 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:26 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-27T10:47:26.178 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:26 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-27T10:47:26.179 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:26 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:47:26.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:26 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-27T10:47:26.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:26 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-27T10:47:26.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:26 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:47:26.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:26 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-27T10:47:26.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:26 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-27T10:47:26.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:26 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:47:27.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:27 smithi097 ceph-mon[31859]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi177:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T10:47:27.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:27 smithi097 ceph-mon[31859]: pgmap v87: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-27T10:47:27.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:27 smithi177 ceph-mon[35294]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi177:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T10:47:27.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:27 smithi177 ceph-mon[35294]: pgmap v87: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-27T10:47:27.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:27 smithi114 ceph-mon[35338]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi177:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T10:47:27.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:27 smithi114 ceph-mon[35338]: pgmap v87: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-27T10:47:29.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:29 smithi097 ceph-mon[31859]: pgmap v88: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-27T10:47:29.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:29 smithi177 ceph-mon[35294]: pgmap v88: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-27T10:47:29.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:29 smithi114 ceph-mon[35338]: pgmap v88: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-27T10:47:30.317 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:30 smithi114 ceph-mon[35338]: from='client.? 172.21.15.177:0/1822220774' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "00108864-1aa6-48d6-8398-e8eb79fda123"}]: dispatch 2024-03-27T10:47:30.318 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:30 smithi114 ceph-mon[35338]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "00108864-1aa6-48d6-8398-e8eb79fda123"}]: dispatch 2024-03-27T10:47:30.318 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:30 smithi114 ceph-mon[35338]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "00108864-1aa6-48d6-8398-e8eb79fda123"}]': finished 2024-03-27T10:47:30.318 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:30 smithi114 ceph-mon[35338]: osdmap e15: 3 total, 2 up, 3 in 2024-03-27T10:47:30.318 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:30 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-27T10:47:30.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:30 smithi097 ceph-mon[31859]: from='client.? 172.21.15.177:0/1822220774' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "00108864-1aa6-48d6-8398-e8eb79fda123"}]: dispatch 2024-03-27T10:47:30.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:30 smithi097 ceph-mon[31859]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "00108864-1aa6-48d6-8398-e8eb79fda123"}]: dispatch 2024-03-27T10:47:30.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:30 smithi097 ceph-mon[31859]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "00108864-1aa6-48d6-8398-e8eb79fda123"}]': finished 2024-03-27T10:47:30.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:30 smithi097 ceph-mon[31859]: osdmap e15: 3 total, 2 up, 3 in 2024-03-27T10:47:30.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:30 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-27T10:47:30.429 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:30 smithi177 ceph-mon[35294]: from='client.? 172.21.15.177:0/1822220774' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "00108864-1aa6-48d6-8398-e8eb79fda123"}]: dispatch 2024-03-27T10:47:30.429 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:30 smithi177 ceph-mon[35294]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "00108864-1aa6-48d6-8398-e8eb79fda123"}]: dispatch 2024-03-27T10:47:30.429 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:30 smithi177 ceph-mon[35294]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "00108864-1aa6-48d6-8398-e8eb79fda123"}]': finished 2024-03-27T10:47:30.429 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:30 smithi177 ceph-mon[35294]: osdmap e15: 3 total, 2 up, 3 in 2024-03-27T10:47:30.429 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:30 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-27T10:47:31.269 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:31 smithi114 ceph-mon[35338]: pgmap v90: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-27T10:47:31.269 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:31 smithi114 ceph-mon[35338]: from='client.? 172.21.15.177:0/969158031' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-27T10:47:31.294 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:31 smithi097 ceph-mon[31859]: pgmap v90: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-27T10:47:31.294 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:31 smithi097 ceph-mon[31859]: from='client.? 172.21.15.177:0/969158031' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-27T10:47:31.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:31 smithi177 ceph-mon[35294]: pgmap v90: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-27T10:47:31.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:31 smithi177 ceph-mon[35294]: from='client.? 172.21.15.177:0/969158031' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-27T10:47:33.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:33 smithi097 ceph-mon[31859]: pgmap v91: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-27T10:47:33.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:33 smithi177 ceph-mon[35294]: pgmap v91: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-27T10:47:33.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:33 smithi114 ceph-mon[35338]: pgmap v91: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-27T10:47:35.178 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:35 smithi177 ceph-mon[35294]: pgmap v92: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-27T10:47:35.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:35 smithi097 ceph-mon[31859]: pgmap v92: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-27T10:47:35.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:35 smithi114 ceph-mon[35338]: pgmap v92: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-27T10:47:37.178 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:37 smithi177 ceph-mon[35294]: pgmap v93: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-27T10:47:37.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:37 smithi097 ceph-mon[31859]: pgmap v93: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-27T10:47:37.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:37 smithi114 ceph-mon[35338]: pgmap v93: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-27T10:47:39.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:39 smithi097 ceph-mon[31859]: pgmap v94: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-27T10:47:39.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:39 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-27T10:47:39.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:39 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:47:39.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:39 smithi177 ceph-mon[35294]: pgmap v94: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-27T10:47:39.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:39 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-27T10:47:39.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:39 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:47:39.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:39 smithi114 ceph-mon[35338]: pgmap v94: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-27T10:47:39.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:39 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-27T10:47:39.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:39 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:47:40.369 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:40 smithi177 ceph-mon[35294]: Deploying daemon osd.2 on smithi177 2024-03-27T10:47:40.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:40 smithi097 ceph-mon[31859]: Deploying daemon osd.2 on smithi177 2024-03-27T10:47:40.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:40 smithi114 ceph-mon[35338]: Deploying daemon osd.2 on smithi177 2024-03-27T10:47:41.342 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:41 smithi177 ceph-mon[35294]: pgmap v95: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-27T10:47:41.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:41 smithi097 ceph-mon[31859]: pgmap v95: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-27T10:47:41.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:41 smithi114 ceph-mon[35338]: pgmap v95: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-27T10:47:43.269 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:43 smithi114 ceph-mon[35338]: pgmap v96: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-27T10:47:43.333 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:43 smithi177 ceph-mon[35294]: pgmap v96: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-27T10:47:43.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:43 smithi097 ceph-mon[31859]: pgmap v96: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-27T10:47:45.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:45 smithi097 ceph-mon[31859]: pgmap v97: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-27T10:47:45.429 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:45 smithi177 ceph-mon[35294]: pgmap v97: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-27T10:47:45.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:45 smithi114 ceph-mon[35338]: pgmap v97: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-27T10:47:47.376 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:47 smithi177 ceph-mon[35294]: pgmap v98: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-27T10:47:47.377 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:47 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:47:47.377 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:47 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:47:47.377 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:47 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:47:47.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:47 smithi097 ceph-mon[31859]: pgmap v98: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-27T10:47:47.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:47 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:47:47.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:47 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:47:47.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:47 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:47:47.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:47 smithi114 ceph-mon[35338]: pgmap v98: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-27T10:47:47.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:47 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:47:47.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:47 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:47:47.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:47 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:47:49.384 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:49 smithi177 ceph-mon[35294]: pgmap v99: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-27T10:47:49.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:49 smithi097 ceph-mon[31859]: pgmap v99: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-27T10:47:49.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:49 smithi114 ceph-mon[35338]: pgmap v99: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-27T10:47:50.828 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:50 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:47:50.829 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:50 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:47:50.829 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:50 smithi177 ceph-mon[35294]: pgmap v100: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-27T10:47:50.829 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:50 smithi177 ceph-mon[35294]: from='osd.2 [v2:172.21.15.177:6800/124240658,v1:172.21.15.177:6801/124240658]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-27T10:47:50.829 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:50 smithi177 ceph-mon[35294]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-27T10:47:50.892 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:50 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:47:50.892 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:50 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:47:50.893 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:50 smithi097 ceph-mon[31859]: pgmap v100: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-27T10:47:50.893 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:50 smithi097 ceph-mon[31859]: from='osd.2 [v2:172.21.15.177:6800/124240658,v1:172.21.15.177:6801/124240658]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-27T10:47:50.893 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:50 smithi097 ceph-mon[31859]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-27T10:47:50.962 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:47:50.988+0000 7f68b2a00700 1 -- 172.21.15.177:0/3388284691 <== mgr.14152 v2:172.21.15.97:6800/3955463538 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f68bc062a80 con 0x7f68a00a3710 2024-03-27T10:47:50.963 INFO:teuthology.orchestra.run.smithi177.stdout:Created osd(s) 2 on host 'smithi177' 2024-03-27T10:47:50.965 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:47:50.992+0000 7f68c1000700 1 -- 172.21.15.177:0/3388284691 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f68a00a3710 msgr2=0x7f68a00a5bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:47:50.965 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:47:50.992+0000 7f68c1000700 1 --2- 172.21.15.177:0/3388284691 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f68a00a3710 0x7f68a00a5bd0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f68a4005f50 tx=0x7f68a4005e80 comp rx=0 tx=0).stop 2024-03-27T10:47:50.966 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:47:50.992+0000 7f68c1000700 1 -- 172.21.15.177:0/3388284691 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f68bc100ee0 msgr2=0x7f68bc19ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:47:50.966 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:47:50.992+0000 7f68c1000700 1 --2- 172.21.15.177:0/3388284691 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f68bc100ee0 0x7f68bc19ec70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f68ac00c9b0 tx=0x7f68ac00cd70 comp rx=0 tx=0).stop 2024-03-27T10:47:50.966 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:47:50.992+0000 7f68c1000700 1 -- 172.21.15.177:0/3388284691 shutdown_connections 2024-03-27T10:47:50.966 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:47:50.992+0000 7f68c1000700 1 --2- 172.21.15.177:0/3388284691 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f68a00a3710 0x7f68a00a5bd0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:47:50.966 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:47:50.992+0000 7f68c1000700 1 --2- 172.21.15.177:0/3388284691 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68bc106220 0x7f68bc1a61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:47:50.966 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:47:50.992+0000 7f68c1000700 1 --2- 172.21.15.177:0/3388284691 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f68bc100ee0 0x7f68bc19ec70 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:47:50.966 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:47:50.992+0000 7f68c1000700 1 --2- 172.21.15.177:0/3388284691 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f68bc103850 0x7f68bc19f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:47:50.966 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:47:50.992+0000 7f68c1000700 1 -- 172.21.15.177:0/3388284691 >> 172.21.15.177:0/3388284691 conn(0x7f68bc0fcb00 msgr2=0x7f68bc101b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:47:50.966 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:47:50.992+0000 7f68c1000700 1 -- 172.21.15.177:0/3388284691 shutdown_connections 2024-03-27T10:47:50.966 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:47:50.992+0000 7f68c1000700 1 -- 172.21.15.177:0/3388284691 wait complete. 2024-03-27T10:47:51.018 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:50 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:47:51.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:50 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:47:51.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:50 smithi114 ceph-mon[35338]: pgmap v100: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-27T10:47:51.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:50 smithi114 ceph-mon[35338]: from='osd.2 [v2:172.21.15.177:6800/124240658,v1:172.21.15.177:6801/124240658]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-27T10:47:51.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:50 smithi114 ceph-mon[35338]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-27T10:47:51.674 DEBUG:teuthology.orchestra.run.smithi177:osd.2> sudo journalctl -f -n 0 -u ceph-adca88c4-ec26-11ee-95d0-87774f69a715@osd.2.service 2024-03-27T10:47:51.680 INFO:tasks.cephadm:Deploying osd.3 on smithi097 with /dev/vg_nvme/lv_3... 2024-03-27T10:47:51.680 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid adca88c4-ec26-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-27T10:47:51.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:51 smithi097 ceph-mon[31859]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-27T10:47:51.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:51 smithi097 ceph-mon[31859]: from='osd.2 [v2:172.21.15.177:6800/124240658,v1:172.21.15.177:6801/124240658]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2024-03-27T10:47:51.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:51 smithi097 ceph-mon[31859]: osdmap e16: 3 total, 2 up, 3 in 2024-03-27T10:47:51.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:51 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-27T10:47:51.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:51 smithi097 ceph-mon[31859]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2024-03-27T10:47:51.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:51 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:47:51.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:51 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:47:51.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:51 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:47:51.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:51 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:47:51.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:51 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:47:51.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:51 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:47:51.928 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:51 smithi177 ceph-mon[35294]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-27T10:47:51.928 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:51 smithi177 ceph-mon[35294]: from='osd.2 [v2:172.21.15.177:6800/124240658,v1:172.21.15.177:6801/124240658]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2024-03-27T10:47:51.928 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:51 smithi177 ceph-mon[35294]: osdmap e16: 3 total, 2 up, 3 in 2024-03-27T10:47:51.929 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:51 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-27T10:47:51.929 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:51 smithi177 ceph-mon[35294]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2024-03-27T10:47:51.929 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:51 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:47:51.929 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:51 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:47:51.929 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:51 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:47:51.929 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:51 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:47:51.929 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:51 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:47:51.929 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:51 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:47:52.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:51 smithi114 ceph-mon[35338]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-27T10:47:52.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:51 smithi114 ceph-mon[35338]: from='osd.2 [v2:172.21.15.177:6800/124240658,v1:172.21.15.177:6801/124240658]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2024-03-27T10:47:52.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:51 smithi114 ceph-mon[35338]: osdmap e16: 3 total, 2 up, 3 in 2024-03-27T10:47:52.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:51 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-27T10:47:52.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:51 smithi114 ceph-mon[35338]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2024-03-27T10:47:52.020 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:51 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:47:52.020 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:51 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:47:52.020 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:51 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:47:52.020 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:51 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:47:52.020 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:51 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:47:52.020 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:51 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:47:52.130 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T10:47:52.893 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:52 smithi097 ceph-mon[31859]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]': finished 2024-03-27T10:47:52.893 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:52 smithi097 ceph-mon[31859]: osdmap e17: 3 total, 2 up, 3 in 2024-03-27T10:47:52.893 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:52 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-27T10:47:52.893 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:52 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-27T10:47:52.893 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:52 smithi097 ceph-mon[31859]: pgmap v103: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-27T10:47:52.918 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:52 smithi177 ceph-mon[35294]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]': finished 2024-03-27T10:47:52.918 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:52 smithi177 ceph-mon[35294]: osdmap e17: 3 total, 2 up, 3 in 2024-03-27T10:47:52.919 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:52 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-27T10:47:52.919 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:52 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-27T10:47:52.919 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:52 smithi177 ceph-mon[35294]: pgmap v103: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-27T10:47:53.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:52 smithi114 ceph-mon[35338]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]': finished 2024-03-27T10:47:53.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:52 smithi114 ceph-mon[35338]: osdmap e17: 3 total, 2 up, 3 in 2024-03-27T10:47:53.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:52 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-27T10:47:53.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:52 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-27T10:47:53.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:52 smithi114 ceph-mon[35338]: pgmap v103: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-27T10:47:53.766 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:53 smithi177 ceph-mon[35294]: purged_snaps scrub starts 2024-03-27T10:47:53.767 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:53 smithi177 ceph-mon[35294]: purged_snaps scrub ok 2024-03-27T10:47:53.767 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:53 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-27T10:47:53.767 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:53 smithi177 ceph-mon[35294]: osd.2 [v2:172.21.15.177:6800/124240658,v1:172.21.15.177:6801/124240658] boot 2024-03-27T10:47:53.767 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:53 smithi177 ceph-mon[35294]: osdmap e18: 3 total, 3 up, 3 in 2024-03-27T10:47:53.767 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:53 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-27T10:47:53.886 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:53 smithi097 ceph-mon[31859]: purged_snaps scrub starts 2024-03-27T10:47:53.887 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:53 smithi097 ceph-mon[31859]: purged_snaps scrub ok 2024-03-27T10:47:53.887 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:53 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-27T10:47:53.887 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:53 smithi097 ceph-mon[31859]: osd.2 [v2:172.21.15.177:6800/124240658,v1:172.21.15.177:6801/124240658] boot 2024-03-27T10:47:53.887 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:53 smithi097 ceph-mon[31859]: osdmap e18: 3 total, 3 up, 3 in 2024-03-27T10:47:53.887 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:53 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-27T10:47:54.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:53 smithi114 ceph-mon[35338]: purged_snaps scrub starts 2024-03-27T10:47:54.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:53 smithi114 ceph-mon[35338]: purged_snaps scrub ok 2024-03-27T10:47:54.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:53 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-27T10:47:54.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:53 smithi114 ceph-mon[35338]: osd.2 [v2:172.21.15.177:6800/124240658,v1:172.21.15.177:6801/124240658] boot 2024-03-27T10:47:54.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:53 smithi114 ceph-mon[35338]: osdmap e18: 3 total, 3 up, 3 in 2024-03-27T10:47:54.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:53 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-27T10:47:54.742 INFO:teuthology.orchestra.run.smithi097.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-27T10:47:54.742 INFO:teuthology.orchestra.run.smithi097.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-27T10:47:54.742 INFO:teuthology.orchestra.run.smithi097.stderr: stderr: 10+0 records in 2024-03-27T10:47:54.742 INFO:teuthology.orchestra.run.smithi097.stderr:10+0 records out 2024-03-27T10:47:54.742 INFO:teuthology.orchestra.run.smithi097.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0357083 s, 294 MB/s 2024-03-27T10:47:54.742 INFO:teuthology.orchestra.run.smithi097.stderr:--> Zapping successful for: 2024-03-27T10:47:54.893 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:54 smithi097 ceph-mon[31859]: osdmap e19: 3 total, 3 up, 3 in 2024-03-27T10:47:54.893 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:54 smithi097 ceph-mon[31859]: pgmap v106: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-27T10:47:54.893 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:54 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-27T10:47:54.928 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:54 smithi177 ceph-mon[35294]: osdmap e19: 3 total, 3 up, 3 in 2024-03-27T10:47:54.928 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:54 smithi177 ceph-mon[35294]: pgmap v106: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-27T10:47:54.929 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:54 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-27T10:47:55.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:54 smithi114 ceph-mon[35338]: osdmap e19: 3 total, 3 up, 3 in 2024-03-27T10:47:55.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:54 smithi114 ceph-mon[35338]: pgmap v106: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-27T10:47:55.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:54 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-27T10:47:55.553 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid adca88c4-ec26-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi097:vg_nvme/lv_3 2024-03-27T10:47:55.892 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:55 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-27T10:47:55.893 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:55 smithi097 ceph-mon[31859]: osdmap e20: 3 total, 3 up, 3 in 2024-03-27T10:47:55.893 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:55 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-27T10:47:55.916 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:55 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-27T10:47:55.917 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:55 smithi177 ceph-mon[35294]: osdmap e20: 3 total, 3 up, 3 in 2024-03-27T10:47:55.917 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:55 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-27T10:47:56.006 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T10:47:56.018 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:55 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-27T10:47:56.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:55 smithi114 ceph-mon[35338]: osdmap e20: 3 total, 3 up, 3 in 2024-03-27T10:47:56.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:55 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-27T10:47:56.309 INFO:journalctl@ceph.osd.0.smithi097.stdout:Mar 27 10:47:55 smithi097 sudo[50997]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-27T10:47:56.309 INFO:journalctl@ceph.osd.0.smithi097.stdout:Mar 27 10:47:55 smithi097 sudo[50997]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-27T10:47:56.309 INFO:journalctl@ceph.osd.0.smithi097.stdout:Mar 27 10:47:55 smithi097 sudo[50997]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-27T10:47:56.309 INFO:journalctl@ceph.osd.0.smithi097.stdout:Mar 27 10:47:55 smithi097 sudo[50997]: pam_unix(sudo:session): session closed for user root 2024-03-27T10:47:56.309 INFO:journalctl@ceph.osd.0.smithi097.stdout:Mar 27 10:47:56 smithi097 sudo[51013]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-27T10:47:56.309 INFO:journalctl@ceph.osd.0.smithi097.stdout:Mar 27 10:47:56 smithi097 sudo[51013]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-27T10:47:56.310 INFO:journalctl@ceph.osd.0.smithi097.stdout:Mar 27 10:47:56 smithi097 sudo[51013]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-27T10:47:56.310 INFO:journalctl@ceph.osd.0.smithi097.stdout:Mar 27 10:47:56 smithi097 sudo[51013]: pam_unix(sudo:session): session closed for user root 2024-03-27T10:47:56.519 INFO:journalctl@ceph.osd.1.smithi114.stdout:Mar 27 10:47:56 smithi114 sudo[43708]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-27T10:47:56.519 INFO:journalctl@ceph.osd.1.smithi114.stdout:Mar 27 10:47:56 smithi114 sudo[43708]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-27T10:47:56.519 INFO:journalctl@ceph.osd.1.smithi114.stdout:Mar 27 10:47:56 smithi114 sudo[43708]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-27T10:47:56.520 INFO:journalctl@ceph.osd.1.smithi114.stdout:Mar 27 10:47:56 smithi114 sudo[43708]: pam_unix(sudo:session): session closed for user root 2024-03-27T10:47:56.520 INFO:journalctl@ceph.osd.1.smithi114.stdout:Mar 27 10:47:56 smithi114 sudo[43711]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-27T10:47:56.520 INFO:journalctl@ceph.osd.1.smithi114.stdout:Mar 27 10:47:56 smithi114 sudo[43711]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-27T10:47:56.520 INFO:journalctl@ceph.osd.1.smithi114.stdout:Mar 27 10:47:56 smithi114 sudo[43711]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-27T10:47:56.520 INFO:journalctl@ceph.osd.1.smithi114.stdout:Mar 27 10:47:56 smithi114 sudo[43711]: pam_unix(sudo:session): session closed for user root 2024-03-27T10:47:56.541 INFO:journalctl@ceph.osd.2.smithi177.stdout:Mar 27 10:47:56 smithi177 sudo[44534]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-27T10:47:56.542 INFO:journalctl@ceph.osd.2.smithi177.stdout:Mar 27 10:47:56 smithi177 sudo[44534]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-27T10:47:56.542 INFO:journalctl@ceph.osd.2.smithi177.stdout:Mar 27 10:47:56 smithi177 sudo[44534]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-27T10:47:56.542 INFO:journalctl@ceph.osd.2.smithi177.stdout:Mar 27 10:47:56 smithi177 sudo[44534]: pam_unix(sudo:session): session closed for user root 2024-03-27T10:47:56.542 INFO:journalctl@ceph.osd.2.smithi177.stdout:Mar 27 10:47:56 smithi177 sudo[44537]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-27T10:47:56.542 INFO:journalctl@ceph.osd.2.smithi177.stdout:Mar 27 10:47:56 smithi177 sudo[44537]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-27T10:47:56.542 INFO:journalctl@ceph.osd.2.smithi177.stdout:Mar 27 10:47:56 smithi177 sudo[44537]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-27T10:47:56.542 INFO:journalctl@ceph.osd.2.smithi177.stdout:Mar 27 10:47:56 smithi177 sudo[44537]: pam_unix(sudo:session): session closed for user root 2024-03-27T10:47:56.569 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:56 smithi097 sudo[51041]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-27T10:47:56.569 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:56 smithi097 sudo[51041]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-27T10:47:56.569 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:56 smithi097 sudo[51041]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-27T10:47:56.569 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:56 smithi097 sudo[51041]: pam_unix(sudo:session): session closed for user root 2024-03-27T10:47:56.569 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:56 smithi097 sudo[51047]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-27T10:47:56.569 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:56 smithi097 sudo[51047]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-27T10:47:56.569 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:56 smithi097 sudo[51047]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-27T10:47:56.569 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:56 smithi097 sudo[51047]: pam_unix(sudo:session): session closed for user root 2024-03-27T10:47:56.892 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:56 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-27T10:47:56.893 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:56 smithi097 ceph-mon[31859]: osdmap e21: 3 total, 3 up, 3 in 2024-03-27T10:47:56.893 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:56 smithi097 ceph-mon[31859]: pgmap v109: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-27T10:47:56.893 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:56 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:47:56.893 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:56 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:47:56.893 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:56 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:47:56.893 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:56 smithi097 ceph-mon[31859]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-27T10:47:56.893 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:56 smithi097 ceph-mon[31859]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-27T10:47:56.893 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:56 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-27T10:47:56.893 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:56 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-27T10:47:56.893 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:56 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-27T10:47:56.894 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:56 smithi097 ceph-mon[31859]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-27T10:47:56.894 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:56 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-27T10:47:56.894 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:56 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-27T10:47:56.894 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:56 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-27T10:47:56.894 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:56 smithi097 ceph-mon[31859]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-27T10:47:56.894 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:56 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:47:56.894 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:56 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-27T10:47:56.894 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:56 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-27T10:47:56.894 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:56 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-27T10:47:56.930 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:56 smithi177 sudo[44540]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-27T10:47:56.931 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:56 smithi177 sudo[44540]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-27T10:47:56.931 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:56 smithi177 sudo[44540]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-27T10:47:56.931 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:56 smithi177 sudo[44540]: pam_unix(sudo:session): session closed for user root 2024-03-27T10:47:56.931 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:56 smithi177 sudo[44543]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-27T10:47:56.931 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:56 smithi177 sudo[44543]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-27T10:47:56.931 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:56 smithi177 sudo[44543]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-27T10:47:56.931 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:56 smithi177 sudo[44543]: pam_unix(sudo:session): session closed for user root 2024-03-27T10:47:56.931 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:56 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-27T10:47:56.931 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:56 smithi177 ceph-mon[35294]: osdmap e21: 3 total, 3 up, 3 in 2024-03-27T10:47:56.931 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:56 smithi177 ceph-mon[35294]: pgmap v109: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-27T10:47:56.931 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:56 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:47:56.932 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:56 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:47:56.932 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:56 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:47:56.932 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:56 smithi177 ceph-mon[35294]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-27T10:47:56.932 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:56 smithi177 ceph-mon[35294]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-27T10:47:56.932 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:56 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-27T10:47:56.932 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:56 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-27T10:47:56.932 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:56 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-27T10:47:56.932 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:56 smithi177 ceph-mon[35294]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-27T10:47:56.932 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:56 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-27T10:47:56.932 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:56 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-27T10:47:56.932 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:56 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-27T10:47:56.932 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:56 smithi177 ceph-mon[35294]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-27T10:47:56.933 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:56 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:47:56.933 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:56 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-27T10:47:56.933 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:56 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-27T10:47:56.933 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:56 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-27T10:47:57.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:56 smithi114 sudo[43714]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-27T10:47:57.020 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:56 smithi114 sudo[43714]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-27T10:47:57.020 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:56 smithi114 sudo[43714]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-27T10:47:57.020 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:56 smithi114 sudo[43714]: pam_unix(sudo:session): session closed for user root 2024-03-27T10:47:57.020 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:56 smithi114 sudo[43717]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-27T10:47:57.020 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:56 smithi114 sudo[43717]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-27T10:47:57.020 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:56 smithi114 sudo[43717]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-27T10:47:57.020 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:56 smithi114 sudo[43717]: pam_unix(sudo:session): session closed for user root 2024-03-27T10:47:57.020 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:56 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-27T10:47:57.020 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:56 smithi114 ceph-mon[35338]: osdmap e21: 3 total, 3 up, 3 in 2024-03-27T10:47:57.020 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:56 smithi114 ceph-mon[35338]: pgmap v109: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-27T10:47:57.020 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:56 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:47:57.020 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:56 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:47:57.021 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:56 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:47:57.021 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:56 smithi114 ceph-mon[35338]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-27T10:47:57.021 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:56 smithi114 ceph-mon[35338]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-27T10:47:57.021 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:56 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-27T10:47:57.021 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:56 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-27T10:47:57.021 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:56 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-27T10:47:57.021 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:56 smithi114 ceph-mon[35338]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-27T10:47:57.021 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:56 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-27T10:47:57.021 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:56 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-27T10:47:57.021 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:56 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-27T10:47:57.021 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:56 smithi114 ceph-mon[35338]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-27T10:47:57.021 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:56 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:47:57.021 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:56 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-27T10:47:57.022 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:56 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-27T10:47:57.022 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:56 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-27T10:47:57.875 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:57 smithi097 ceph-mon[31859]: Detected new or changed devices on smithi177 2024-03-27T10:47:57.875 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:57 smithi097 ceph-mon[31859]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-27T10:47:57.876 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:57 smithi097 ceph-mon[31859]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-27T10:47:57.876 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:57 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:47:57.876 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:57 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-27T10:47:57.876 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:57 smithi097 ceph-mon[31859]: Adjusting osd_memory_target on smithi177 to 17090M 2024-03-27T10:47:57.876 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:57 smithi097 ceph-mon[31859]: mgrmap e16: x(active, since 3m), standbys: z, y 2024-03-27T10:47:57.876 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:57 smithi097 ceph-mon[31859]: osdmap e22: 3 total, 3 up, 3 in 2024-03-27T10:47:57.876 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:57 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:47:57.876 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:57 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:47:57.876 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:57 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:47:57.876 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:57 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:47:58.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:57 smithi114 ceph-mon[35338]: Detected new or changed devices on smithi177 2024-03-27T10:47:58.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:57 smithi114 ceph-mon[35338]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-27T10:47:58.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:57 smithi114 ceph-mon[35338]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-27T10:47:58.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:57 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:47:58.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:57 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-27T10:47:58.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:57 smithi114 ceph-mon[35338]: Adjusting osd_memory_target on smithi177 to 17090M 2024-03-27T10:47:58.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:57 smithi114 ceph-mon[35338]: mgrmap e16: x(active, since 3m), standbys: z, y 2024-03-27T10:47:58.020 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:57 smithi114 ceph-mon[35338]: osdmap e22: 3 total, 3 up, 3 in 2024-03-27T10:47:58.020 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:57 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:47:58.020 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:57 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:47:58.020 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:57 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:47:58.020 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:57 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:47:58.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:47:58.072+0000 7fcb63600700 1 -- 172.21.15.97:0/2420409867 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb5c104210 msgr2=0x7fcb5c10aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:47:58.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:47:58.072+0000 7fcb63600700 1 --2- 172.21.15.97:0/2420409867 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb5c104210 0x7fcb5c10aac0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fcb5400b0d0 tx=0x7fcb5400b3e0 comp rx=0 tx=0).stop 2024-03-27T10:47:58.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:47:58.072+0000 7fcb63600700 1 -- 172.21.15.97:0/2420409867 shutdown_connections 2024-03-27T10:47:58.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:47:58.072+0000 7fcb63600700 1 --2- 172.21.15.97:0/2420409867 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb5c104210 0x7fcb5c10aac0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:47:58.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:47:58.072+0000 7fcb63600700 1 --2- 172.21.15.97:0/2420409867 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcb5c102670 0x7fcb5c102a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:47:58.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:47:58.072+0000 7fcb63600700 1 --2- 172.21.15.97:0/2420409867 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcb5c103870 0x7fcb5c103cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:47:58.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:47:58.072+0000 7fcb63600700 1 -- 172.21.15.97:0/2420409867 >> 172.21.15.97:0/2420409867 conn(0x7fcb5c0fdc50 msgr2=0x7fcb5c100070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:47:58.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:47:58.072+0000 7fcb63600700 1 -- 172.21.15.97:0/2420409867 shutdown_connections 2024-03-27T10:47:58.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:47:58.073+0000 7fcb63600700 1 -- 172.21.15.97:0/2420409867 wait complete. 2024-03-27T10:47:58.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:47:58.073+0000 7fcb63600700 1 Processor -- start 2024-03-27T10:47:58.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:47:58.074+0000 7fcb63600700 1 -- start start 2024-03-27T10:47:58.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:47:58.074+0000 7fcb63600700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcb5c102670 0x7fcb5c0763e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:47:58.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:47:58.074+0000 7fcb63600700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb5c103870 0x7fcb5c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:47:58.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:47:58.074+0000 7fcb63600700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcb5c104210 0x7fcb5c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:47:58.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:47:58.074+0000 7fcb63600700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fcb5c10e140 con 0x7fcb5c103870 2024-03-27T10:47:58.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:47:58.074+0000 7fcb63600700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fcb5c10dfc0 con 0x7fcb5c104210 2024-03-27T10:47:58.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:47:58.074+0000 7fcb63600700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fcb5c10e2c0 con 0x7fcb5c102670 2024-03-27T10:47:58.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:47:58.074+0000 7fcb5be00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb5c103870 0x7fcb5c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:47:58.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:47:58.074+0000 7fcb5be00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb5c103870 0x7fcb5c074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37196/0 (socket says 172.21.15.97:37196) 2024-03-27T10:47:58.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:47:58.074+0000 7fcb5be00700 1 -- 172.21.15.97:0/2334130002 learned_addr learned my addr 172.21.15.97:0/2334130002 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:47:58.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:47:58.075+0000 7fcb60c00700 1 --2- 172.21.15.97:0/2334130002 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcb5c102670 0x7fcb5c0763e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:47:58.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:47:58.075+0000 7fcb61600700 1 --2- 172.21.15.97:0/2334130002 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcb5c104210 0x7fcb5c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:47:58.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:47:58.075+0000 7fcb5be00700 1 -- 172.21.15.97:0/2334130002 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcb5c102670 msgr2=0x7fcb5c0763e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:47:58.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:47:58.075+0000 7fcb5be00700 1 --2- 172.21.15.97:0/2334130002 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcb5c102670 0x7fcb5c0763e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:47:58.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:47:58.075+0000 7fcb5be00700 1 -- 172.21.15.97:0/2334130002 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcb5c104210 msgr2=0x7fcb5c074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:47:58.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:47:58.075+0000 7fcb5be00700 1 --2- 172.21.15.97:0/2334130002 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcb5c104210 0x7fcb5c074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:47:58.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:47:58.075+0000 7fcb5be00700 1 -- 172.21.15.97:0/2334130002 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb5c075820 con 0x7fcb5c103870 2024-03-27T10:47:58.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:47:58.075+0000 7fcb5be00700 1 --2- 172.21.15.97:0/2334130002 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb5c103870 0x7fcb5c074a30 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fcb4800c920 tx=0x7fcb4800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:47:58.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:47:58.076+0000 7fcb59600700 1 -- 172.21.15.97:0/2334130002 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb48007b00 con 0x7fcb5c103870 2024-03-27T10:47:58.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:47:58.076+0000 7fcb59600700 1 -- 172.21.15.97:0/2334130002 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcb48004ce0 con 0x7fcb5c103870 2024-03-27T10:47:58.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:47:58.076+0000 7fcb63600700 1 -- 172.21.15.97:0/2334130002 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb5c1a28c0 con 0x7fcb5c103870 2024-03-27T10:47:58.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:47:58.076+0000 7fcb59600700 1 -- 172.21.15.97:0/2334130002 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb48005600 con 0x7fcb5c103870 2024-03-27T10:47:58.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:47:58.076+0000 7fcb63600700 1 -- 172.21.15.97:0/2334130002 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb5c1a2e00 con 0x7fcb5c103870 2024-03-27T10:47:58.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:47:58.077+0000 7fcb63600700 1 -- 172.21.15.97:0/2334130002 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb5c069300 con 0x7fcb5c103870 2024-03-27T10:47:58.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:47:58.079+0000 7fcb59600700 1 -- 172.21.15.97:0/2334130002 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138370+0+0 (secure 0 0 0) 0x7fcb48007ca0 con 0x7fcb5c103870 2024-03-27T10:47:58.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:47:58.080+0000 7fcb59600700 1 --2- 172.21.15.97:0/2334130002 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fcb440a3720 0x7fcb440a5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:47:58.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:47:58.081+0000 7fcb59600700 1 -- 172.21.15.97:0/2334130002 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 3237+0+0 (secure 0 0 0) 0x7fcb480ce550 con 0x7fcb5c103870 2024-03-27T10:47:58.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:47:58.081+0000 7fcb60c00700 1 --2- 172.21.15.97:0/2334130002 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fcb440a3720 0x7fcb440a5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:47:58.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:47:58.081+0000 7fcb60c00700 1 --2- 172.21.15.97:0/2334130002 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fcb440a3720 0x7fcb440a5be0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fcb50005950 tx=0x7fcb500058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:47:58.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:47:58.085+0000 7fcb59600700 1 -- 172.21.15.97:0/2334130002 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7fcb48014030 con 0x7fcb5c103870 2024-03-27T10:47:58.179 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:57 smithi177 ceph-mon[35294]: Detected new or changed devices on smithi177 2024-03-27T10:47:58.179 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:57 smithi177 ceph-mon[35294]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-27T10:47:58.179 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:57 smithi177 ceph-mon[35294]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-27T10:47:58.179 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:57 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:47:58.179 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:57 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-27T10:47:58.179 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:57 smithi177 ceph-mon[35294]: Adjusting osd_memory_target on smithi177 to 17090M 2024-03-27T10:47:58.179 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:57 smithi177 ceph-mon[35294]: mgrmap e16: x(active, since 3m), standbys: z, y 2024-03-27T10:47:58.179 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:57 smithi177 ceph-mon[35294]: osdmap e22: 3 total, 3 up, 3 in 2024-03-27T10:47:58.179 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:57 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:47:58.179 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:57 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:47:58.179 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:57 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:47:58.179 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:57 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:47:58.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:47:58.318+0000 7fcb63600700 1 -- 172.21.15.97:0/2334130002 --> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fcb5c062a80 con 0x7fcb440a3720 2024-03-27T10:47:59.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:58 smithi114 ceph-mon[35338]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-03-27T10:47:59.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:58 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-27T10:47:59.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:58 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-27T10:47:59.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:58 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:47:59.142 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:58 smithi097 ceph-mon[31859]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-03-27T10:47:59.142 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:58 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-27T10:47:59.142 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:58 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-27T10:47:59.143 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:58 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:47:59.178 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:58 smithi177 ceph-mon[35294]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-03-27T10:47:59.178 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:58 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-27T10:47:59.178 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:58 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-27T10:47:59.178 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:58 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:48:00.012 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:47:59 smithi097 ceph-mon[31859]: from='client.14301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T10:48:00.018 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:47:59 smithi114 ceph-mon[35338]: from='client.14301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T10:48:00.178 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:47:59 smithi177 ceph-mon[35294]: from='client.14301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T10:48:00.914 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:00 smithi097 ceph-mon[31859]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-03-27T10:48:01.018 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:00 smithi114 ceph-mon[35338]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-03-27T10:48:01.178 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:00 smithi177 ceph-mon[35294]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-03-27T10:48:03.068 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:02 smithi097 ceph-mon[31859]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.1 KiB/s rd, 113 KiB/s wr, 7 op/s 2024-03-27T10:48:03.068 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:02 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/499891760' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1860b690-e2dc-4977-88d7-d1aacf7022ca"}]: dispatch 2024-03-27T10:48:03.068 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:02 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/499891760' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1860b690-e2dc-4977-88d7-d1aacf7022ca"}]': finished 2024-03-27T10:48:03.068 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:02 smithi097 ceph-mon[31859]: osdmap e23: 4 total, 3 up, 4 in 2024-03-27T10:48:03.068 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:02 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-27T10:48:03.068 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:02 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/3924247229' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-27T10:48:03.269 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:03 smithi114 ceph-mon[35338]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.1 KiB/s rd, 113 KiB/s wr, 7 op/s 2024-03-27T10:48:03.269 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:03 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/499891760' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1860b690-e2dc-4977-88d7-d1aacf7022ca"}]: dispatch 2024-03-27T10:48:03.269 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:03 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/499891760' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1860b690-e2dc-4977-88d7-d1aacf7022ca"}]': finished 2024-03-27T10:48:03.269 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:03 smithi114 ceph-mon[35338]: osdmap e23: 4 total, 3 up, 4 in 2024-03-27T10:48:03.269 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:03 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-27T10:48:03.269 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:03 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/3924247229' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-27T10:48:03.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:03 smithi177 ceph-mon[35294]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.1 KiB/s rd, 113 KiB/s wr, 7 op/s 2024-03-27T10:48:03.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:03 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/499891760' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1860b690-e2dc-4977-88d7-d1aacf7022ca"}]: dispatch 2024-03-27T10:48:03.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:03 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/499891760' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1860b690-e2dc-4977-88d7-d1aacf7022ca"}]': finished 2024-03-27T10:48:03.429 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:03 smithi177 ceph-mon[35294]: osdmap e23: 4 total, 3 up, 4 in 2024-03-27T10:48:03.429 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:03 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-27T10:48:03.429 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:03 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/3924247229' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-27T10:48:05.178 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:05 smithi177 ceph-mon[35294]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 82 KiB/s wr, 5 op/s 2024-03-27T10:48:05.268 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:05 smithi114 ceph-mon[35338]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 82 KiB/s wr, 5 op/s 2024-03-27T10:48:05.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:05 smithi097 ceph-mon[31859]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 82 KiB/s wr, 5 op/s 2024-03-27T10:48:07.268 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:07 smithi114 ceph-mon[35338]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 64 KiB/s wr, 4 op/s 2024-03-27T10:48:07.355 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:07 smithi097 ceph-mon[31859]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 64 KiB/s wr, 4 op/s 2024-03-27T10:48:07.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:07 smithi177 ceph-mon[35294]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 64 KiB/s wr, 4 op/s 2024-03-27T10:48:09.268 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:09 smithi114 ceph-mon[35338]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 59 KiB/s wr, 3 op/s 2024-03-27T10:48:09.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:09 smithi097 ceph-mon[31859]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 59 KiB/s wr, 3 op/s 2024-03-27T10:48:09.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:09 smithi177 ceph-mon[35294]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 59 KiB/s wr, 3 op/s 2024-03-27T10:48:11.142 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:11 smithi097 ceph-mon[31859]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 59 KiB/s wr, 3 op/s 2024-03-27T10:48:11.268 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:11 smithi114 ceph-mon[35338]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 59 KiB/s wr, 3 op/s 2024-03-27T10:48:11.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:11 smithi177 ceph-mon[35294]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 59 KiB/s wr, 3 op/s 2024-03-27T10:48:12.268 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:12 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-27T10:48:12.269 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:12 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:48:12.375 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:12 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-27T10:48:12.375 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:12 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:48:12.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:12 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-27T10:48:12.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:12 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:48:13.268 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:13 smithi114 ceph-mon[35338]: Deploying daemon osd.3 on smithi097 2024-03-27T10:48:13.269 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:13 smithi114 ceph-mon[35338]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-03-27T10:48:13.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:13 smithi097 ceph-mon[31859]: Deploying daemon osd.3 on smithi097 2024-03-27T10:48:13.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:13 smithi097 ceph-mon[31859]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-03-27T10:48:13.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:13 smithi177 ceph-mon[35294]: Deploying daemon osd.3 on smithi097 2024-03-27T10:48:13.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:13 smithi177 ceph-mon[35294]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-03-27T10:48:15.269 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:15 smithi114 ceph-mon[35338]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-03-27T10:48:15.394 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:15 smithi097 ceph-mon[31859]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-03-27T10:48:15.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:15 smithi177 ceph-mon[35294]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-03-27T10:48:17.143 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:17 smithi097 ceph-mon[31859]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-03-27T10:48:17.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:17 smithi177 ceph-mon[35294]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-03-27T10:48:17.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:17 smithi114 ceph-mon[35338]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-03-27T10:48:19.142 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:19 smithi097 ceph-mon[31859]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-03-27T10:48:19.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:19 smithi177 ceph-mon[35294]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-03-27T10:48:19.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:19 smithi114 ceph-mon[35338]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-03-27T10:48:20.844 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:20 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:48:20.844 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:20 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:48:20.844 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:20 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:48:20.844 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:20 smithi097 ceph-mon[31859]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-03-27T10:48:20.928 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:20 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:48:20.928 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:20 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:48:20.928 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:20 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:48:20.928 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:20 smithi177 ceph-mon[35294]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-03-27T10:48:21.018 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:20 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:48:21.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:20 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:48:21.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:20 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:48:21.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:20 smithi114 ceph-mon[35338]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-03-27T10:48:23.268 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:23 smithi114 ceph-mon[35338]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-03-27T10:48:23.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:23 smithi097 ceph-mon[31859]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-03-27T10:48:23.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:23 smithi177 ceph-mon[35294]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-03-27T10:48:24.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:48:24.515+0000 7fcb59600700 1 -- 172.21.15.97:0/2334130002 <== mgr.14152 v2:172.21.15.97:6800/3955463538 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fcb5c062a80 con 0x7fcb440a3720 2024-03-27T10:48:24.490 INFO:teuthology.orchestra.run.smithi097.stdout:Created osd(s) 3 on host 'smithi097' 2024-03-27T10:48:24.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:48:24.520+0000 7fcb63600700 1 -- 172.21.15.97:0/2334130002 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fcb440a3720 msgr2=0x7fcb440a5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:48:24.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:48:24.520+0000 7fcb63600700 1 --2- 172.21.15.97:0/2334130002 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fcb440a3720 0x7fcb440a5be0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fcb50005950 tx=0x7fcb500058e0 comp rx=0 tx=0).stop 2024-03-27T10:48:24.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:48:24.520+0000 7fcb63600700 1 -- 172.21.15.97:0/2334130002 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb5c103870 msgr2=0x7fcb5c074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:48:24.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:48:24.520+0000 7fcb63600700 1 --2- 172.21.15.97:0/2334130002 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb5c103870 0x7fcb5c074a30 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fcb4800c920 tx=0x7fcb4800cce0 comp rx=0 tx=0).stop 2024-03-27T10:48:24.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:48:24.520+0000 7fcb63600700 1 -- 172.21.15.97:0/2334130002 shutdown_connections 2024-03-27T10:48:24.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:48:24.520+0000 7fcb63600700 1 --2- 172.21.15.97:0/2334130002 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fcb440a3720 0x7fcb440a5be0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:48:24.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:48:24.520+0000 7fcb63600700 1 --2- 172.21.15.97:0/2334130002 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcb5c104210 0x7fcb5c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:48:24.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:48:24.520+0000 7fcb63600700 1 --2- 172.21.15.97:0/2334130002 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcb5c102670 0x7fcb5c0763e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:48:24.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:48:24.520+0000 7fcb63600700 1 --2- 172.21.15.97:0/2334130002 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb5c103870 0x7fcb5c074a30 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:48:24.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:48:24.520+0000 7fcb63600700 1 -- 172.21.15.97:0/2334130002 >> 172.21.15.97:0/2334130002 conn(0x7fcb5c0fdc50 msgr2=0x7fcb5c106b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:48:24.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:48:24.521+0000 7fcb63600700 1 -- 172.21.15.97:0/2334130002 shutdown_connections 2024-03-27T10:48:24.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:48:24.521+0000 7fcb63600700 1 -- 172.21.15.97:0/2334130002 wait complete. 2024-03-27T10:48:24.642 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:24 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:48:24.642 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:24 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:48:24.642 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:24 smithi097 ceph-mon[31859]: from='osd.3 [v2:172.21.15.97:6810/2478179311,v1:172.21.15.97:6811/2478179311]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-27T10:48:24.642 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:24 smithi097 ceph-mon[31859]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-03-27T10:48:24.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:24 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:48:24.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:24 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:48:24.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:24 smithi177 ceph-mon[35294]: from='osd.3 [v2:172.21.15.97:6810/2478179311,v1:172.21.15.97:6811/2478179311]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-27T10:48:24.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:24 smithi177 ceph-mon[35294]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-03-27T10:48:24.768 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:24 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:48:24.769 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:24 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:48:24.769 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:24 smithi114 ceph-mon[35338]: from='osd.3 [v2:172.21.15.97:6810/2478179311,v1:172.21.15.97:6811/2478179311]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-27T10:48:24.769 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:24 smithi114 ceph-mon[35338]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-03-27T10:48:25.450 DEBUG:teuthology.orchestra.run.smithi097:osd.3> sudo journalctl -f -n 0 -u ceph-adca88c4-ec26-11ee-95d0-87774f69a715@osd.3.service 2024-03-27T10:48:25.455 INFO:tasks.cephadm:Deploying osd.4 on smithi114 with /dev/vg_nvme/lv_3... 2024-03-27T10:48:25.455 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid adca88c4-ec26-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-27T10:48:25.642 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:25 smithi097 ceph-mon[31859]: from='osd.3 [v2:172.21.15.97:6810/2478179311,v1:172.21.15.97:6811/2478179311]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-27T10:48:25.642 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:25 smithi097 ceph-mon[31859]: osdmap e24: 4 total, 3 up, 4 in 2024-03-27T10:48:25.642 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:25 smithi097 ceph-mon[31859]: from='osd.3 [v2:172.21.15.97:6810/2478179311,v1:172.21.15.97:6811/2478179311]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-03-27T10:48:25.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:25 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-27T10:48:25.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:25 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:48:25.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:25 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:48:25.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:25 smithi177 ceph-mon[35294]: from='osd.3 [v2:172.21.15.97:6810/2478179311,v1:172.21.15.97:6811/2478179311]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-27T10:48:25.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:25 smithi177 ceph-mon[35294]: osdmap e24: 4 total, 3 up, 4 in 2024-03-27T10:48:25.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:25 smithi177 ceph-mon[35294]: from='osd.3 [v2:172.21.15.97:6810/2478179311,v1:172.21.15.97:6811/2478179311]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-03-27T10:48:25.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:25 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-27T10:48:25.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:25 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:48:25.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:25 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:48:25.768 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:25 smithi114 ceph-mon[35338]: from='osd.3 [v2:172.21.15.97:6810/2478179311,v1:172.21.15.97:6811/2478179311]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-27T10:48:25.768 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:25 smithi114 ceph-mon[35338]: osdmap e24: 4 total, 3 up, 4 in 2024-03-27T10:48:25.768 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:25 smithi114 ceph-mon[35338]: from='osd.3 [v2:172.21.15.97:6810/2478179311,v1:172.21.15.97:6811/2478179311]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-03-27T10:48:25.768 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:25 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-27T10:48:25.769 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:25 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:48:25.769 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:25 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:48:25.900 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/mon.b/config 2024-03-27T10:48:26.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:26 smithi097 ceph-mon[31859]: from='osd.3 [v2:172.21.15.97:6810/2478179311,v1:172.21.15.97:6811/2478179311]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-03-27T10:48:26.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:26 smithi097 ceph-mon[31859]: osdmap e25: 4 total, 3 up, 4 in 2024-03-27T10:48:26.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:26 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-27T10:48:26.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:26 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-27T10:48:26.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:26 smithi097 ceph-mon[31859]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-03-27T10:48:26.644 INFO:journalctl@ceph.osd.3.smithi097.stdout:Mar 27 10:48:26 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-3[53399]: 2024-03-27T10:48:26.382+0000 7f9d4f000700 -1 osd.3 0 waiting for initial osdmap 2024-03-27T10:48:26.644 INFO:journalctl@ceph.osd.3.smithi097.stdout:Mar 27 10:48:26 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-3[53399]: 2024-03-27T10:48:26.444+0000 7f9d44e00700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-27T10:48:26.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:26 smithi177 ceph-mon[35294]: from='osd.3 [v2:172.21.15.97:6810/2478179311,v1:172.21.15.97:6811/2478179311]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-03-27T10:48:26.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:26 smithi177 ceph-mon[35294]: osdmap e25: 4 total, 3 up, 4 in 2024-03-27T10:48:26.679 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:26 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-27T10:48:26.679 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:26 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-27T10:48:26.679 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:26 smithi177 ceph-mon[35294]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-03-27T10:48:26.769 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:26 smithi114 ceph-mon[35338]: from='osd.3 [v2:172.21.15.97:6810/2478179311,v1:172.21.15.97:6811/2478179311]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-03-27T10:48:26.769 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:26 smithi114 ceph-mon[35338]: osdmap e25: 4 total, 3 up, 4 in 2024-03-27T10:48:26.769 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:26 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-27T10:48:26.769 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:26 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-27T10:48:26.769 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:26 smithi114 ceph-mon[35338]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-03-27T10:48:27.447 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:27 smithi097 ceph-mon[31859]: purged_snaps scrub starts 2024-03-27T10:48:27.447 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:27 smithi097 ceph-mon[31859]: purged_snaps scrub ok 2024-03-27T10:48:27.447 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:27 smithi097 ceph-mon[31859]: from='osd.3 [v2:172.21.15.97:6810/2478179311,v1:172.21.15.97:6811/2478179311]' entity='osd.3' 2024-03-27T10:48:27.448 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:27 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-27T10:48:27.448 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:27 smithi097 ceph-mon[31859]: osdmap e26: 4 total, 3 up, 4 in 2024-03-27T10:48:27.448 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:27 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-27T10:48:27.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:27 smithi177 ceph-mon[35294]: purged_snaps scrub starts 2024-03-27T10:48:27.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:27 smithi177 ceph-mon[35294]: purged_snaps scrub ok 2024-03-27T10:48:27.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:27 smithi177 ceph-mon[35294]: from='osd.3 [v2:172.21.15.97:6810/2478179311,v1:172.21.15.97:6811/2478179311]' entity='osd.3' 2024-03-27T10:48:27.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:27 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-27T10:48:27.679 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:27 smithi177 ceph-mon[35294]: osdmap e26: 4 total, 3 up, 4 in 2024-03-27T10:48:27.679 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:27 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-27T10:48:27.722 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:27 smithi114 ceph-mon[35338]: purged_snaps scrub starts 2024-03-27T10:48:27.722 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:27 smithi114 ceph-mon[35338]: purged_snaps scrub ok 2024-03-27T10:48:27.722 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:27 smithi114 ceph-mon[35338]: from='osd.3 [v2:172.21.15.97:6810/2478179311,v1:172.21.15.97:6811/2478179311]' entity='osd.3' 2024-03-27T10:48:27.722 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:27 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-27T10:48:27.722 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:27 smithi114 ceph-mon[35338]: osdmap e26: 4 total, 3 up, 4 in 2024-03-27T10:48:27.722 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:27 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-27T10:48:28.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:28 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-27T10:48:28.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:28 smithi097 ceph-mon[31859]: osd.3 [v2:172.21.15.97:6810/2478179311,v1:172.21.15.97:6811/2478179311] boot 2024-03-27T10:48:28.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:28 smithi097 ceph-mon[31859]: osdmap e27: 4 total, 4 up, 4 in 2024-03-27T10:48:28.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:28 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-27T10:48:28.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:28 smithi097 ceph-mon[31859]: pgmap v131: 1 pgs: 1 remapped+peering; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-03-27T10:48:28.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:28 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-27T10:48:28.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:28 smithi177 ceph-mon[35294]: osd.3 [v2:172.21.15.97:6810/2478179311,v1:172.21.15.97:6811/2478179311] boot 2024-03-27T10:48:28.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:28 smithi177 ceph-mon[35294]: osdmap e27: 4 total, 4 up, 4 in 2024-03-27T10:48:28.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:28 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-27T10:48:28.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:28 smithi177 ceph-mon[35294]: pgmap v131: 1 pgs: 1 remapped+peering; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-03-27T10:48:28.769 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:28 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-27T10:48:28.769 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:28 smithi114 ceph-mon[35338]: osd.3 [v2:172.21.15.97:6810/2478179311,v1:172.21.15.97:6811/2478179311] boot 2024-03-27T10:48:28.769 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:28 smithi114 ceph-mon[35338]: osdmap e27: 4 total, 4 up, 4 in 2024-03-27T10:48:28.769 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:28 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-27T10:48:28.769 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:28 smithi114 ceph-mon[35338]: pgmap v131: 1 pgs: 1 remapped+peering; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-03-27T10:48:28.871 INFO:teuthology.orchestra.run.smithi114.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-27T10:48:28.871 INFO:teuthology.orchestra.run.smithi114.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-27T10:48:28.871 INFO:teuthology.orchestra.run.smithi114.stderr: stderr: 10+0 records in 2024-03-27T10:48:28.871 INFO:teuthology.orchestra.run.smithi114.stderr:10+0 records out 2024-03-27T10:48:28.871 INFO:teuthology.orchestra.run.smithi114.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0327532 s, 320 MB/s 2024-03-27T10:48:28.871 INFO:teuthology.orchestra.run.smithi114.stderr:--> Zapping successful for: 2024-03-27T10:48:29.577 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:29 smithi114 ceph-mon[35338]: osdmap e28: 4 total, 4 up, 4 in 2024-03-27T10:48:29.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:29 smithi097 ceph-mon[31859]: osdmap e28: 4 total, 4 up, 4 in 2024-03-27T10:48:29.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:29 smithi177 ceph-mon[35294]: osdmap e28: 4 total, 4 up, 4 in 2024-03-27T10:48:29.761 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid adca88c4-ec26-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi114:vg_nvme/lv_3 2024-03-27T10:48:30.206 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/mon.b/config 2024-03-27T10:48:30.639 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:30 smithi114 ceph-mon[35338]: osdmap e29: 4 total, 4 up, 4 in 2024-03-27T10:48:30.639 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:30 smithi114 ceph-mon[35338]: pgmap v134: 1 pgs: 1 remapped+peering; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-03-27T10:48:30.639 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:30 smithi114 ceph-mon[35338]: osdmap e30: 4 total, 4 up, 4 in 2024-03-27T10:48:30.640 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:30 smithi114 ceph-mon[35338]: Detected new or changed devices on smithi097 2024-03-27T10:48:30.640 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:30 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:48:30.640 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:30 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:48:30.640 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:30 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-27T10:48:30.640 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:30 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-27T10:48:30.640 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:30 smithi114 ceph-mon[35338]: Adjusting osd_memory_target on smithi097 to 8545M 2024-03-27T10:48:30.640 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:30 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:48:30.640 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:30 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:48:30.640 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:30 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:48:30.640 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:30 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:48:30.682 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:30 smithi177 ceph-mon[35294]: osdmap e29: 4 total, 4 up, 4 in 2024-03-27T10:48:30.682 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:30 smithi177 ceph-mon[35294]: pgmap v134: 1 pgs: 1 remapped+peering; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-03-27T10:48:30.682 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:30 smithi177 ceph-mon[35294]: osdmap e30: 4 total, 4 up, 4 in 2024-03-27T10:48:30.683 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:30 smithi177 ceph-mon[35294]: Detected new or changed devices on smithi097 2024-03-27T10:48:30.683 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:30 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:48:30.683 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:30 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:48:30.683 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:30 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-27T10:48:30.683 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:30 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-27T10:48:30.683 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:30 smithi177 ceph-mon[35294]: Adjusting osd_memory_target on smithi097 to 8545M 2024-03-27T10:48:30.683 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:30 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:48:30.683 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:30 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:48:30.683 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:30 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:48:30.684 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:30 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:48:30.893 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:30 smithi097 ceph-mon[31859]: osdmap e29: 4 total, 4 up, 4 in 2024-03-27T10:48:30.893 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:30 smithi097 ceph-mon[31859]: pgmap v134: 1 pgs: 1 remapped+peering; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-03-27T10:48:30.893 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:30 smithi097 ceph-mon[31859]: osdmap e30: 4 total, 4 up, 4 in 2024-03-27T10:48:30.893 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:30 smithi097 ceph-mon[31859]: Detected new or changed devices on smithi097 2024-03-27T10:48:30.893 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:30 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:48:30.893 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:30 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:48:30.893 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:30 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-27T10:48:30.893 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:30 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-27T10:48:30.894 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:30 smithi097 ceph-mon[31859]: Adjusting osd_memory_target on smithi097 to 8545M 2024-03-27T10:48:30.894 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:30 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:48:30.894 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:30 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:48:30.894 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:30 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:48:30.894 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:30 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:48:31.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:31 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:48:31.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:31 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:48:31.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:31 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:48:31.679 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:31 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:48:31.679 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:31 smithi177 ceph-mon[35294]: osdmap e31: 4 total, 4 up, 4 in 2024-03-27T10:48:31.769 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:31 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:48:31.769 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:31 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:48:31.769 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:31 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:48:31.769 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:31 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:48:31.769 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:31 smithi114 ceph-mon[35338]: osdmap e31: 4 total, 4 up, 4 in 2024-03-27T10:48:31.797 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:31 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:48:31.797 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:31 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:48:31.797 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:31 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:48:31.797 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:31 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:48:31.797 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:31 smithi097 ceph-mon[31859]: osdmap e31: 4 total, 4 up, 4 in 2024-03-27T10:48:32.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:48:32.315+0000 7f8ca0200700 1 -- 172.21.15.114:0/1790540374 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c9810a5a0 msgr2=0x7f8c9810ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:48:32.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:48:32.315+0000 7f8ca0200700 1 --2- 172.21.15.114:0/1790540374 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c9810a5a0 0x7f8c9810ca10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f8c9000b0a0 tx=0x7f8c9000b3b0 comp rx=0 tx=0).stop 2024-03-27T10:48:32.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:48:32.315+0000 7f8ca0200700 1 -- 172.21.15.114:0/1790540374 shutdown_connections 2024-03-27T10:48:32.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:48:32.315+0000 7f8ca0200700 1 --2- 172.21.15.114:0/1790540374 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c9810a5a0 0x7f8c9810ca10 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:48:32.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:48:32.315+0000 7f8ca0200700 1 --2- 172.21.15.114:0/1790540374 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8c980765f0 0x7f8c98074a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:48:32.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:48:32.315+0000 7f8ca0200700 1 --2- 172.21.15.114:0/1790540374 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c98075020 0x7f8c980754a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:48:32.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:48:32.315+0000 7f8ca0200700 1 -- 172.21.15.114:0/1790540374 >> 172.21.15.114:0/1790540374 conn(0x7f8c980fdc40 msgr2=0x7f8c98100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:48:32.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:48:32.315+0000 7f8ca0200700 1 -- 172.21.15.114:0/1790540374 shutdown_connections 2024-03-27T10:48:32.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:48:32.316+0000 7f8ca0200700 1 -- 172.21.15.114:0/1790540374 wait complete. 2024-03-27T10:48:32.292 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:48:32.317+0000 7f8ca0200700 1 Processor -- start 2024-03-27T10:48:32.292 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:48:32.317+0000 7f8ca0200700 1 -- start start 2024-03-27T10:48:32.292 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:48:32.317+0000 7f8ca0200700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c98075020 0x7f8c9819a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:48:32.292 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:48:32.317+0000 7f8ca0200700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8c980765f0 0x7f8c9819ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:48:32.292 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:48:32.317+0000 7f8ca0200700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c9810a5a0 0x7f8c981a1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:48:32.292 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:48:32.317+0000 7f8ca0200700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8c98110310 con 0x7f8c9810a5a0 2024-03-27T10:48:32.292 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:48:32.317+0000 7f8ca0200700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f8c98110190 con 0x7f8c98075020 2024-03-27T10:48:32.292 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:48:32.317+0000 7f8ca0200700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f8c98110490 con 0x7f8c980765f0 2024-03-27T10:48:32.293 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:48:32.317+0000 7f8c9d800700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c98075020 0x7f8c9819a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:48:32.293 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:48:32.317+0000 7f8c9d800700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c98075020 0x7f8c9819a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:45002/0 (socket says 172.21.15.114:45002) 2024-03-27T10:48:32.293 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:48:32.317+0000 7f8c9d800700 1 -- 172.21.15.114:0/1858444085 learned_addr learned my addr 172.21.15.114:0/1858444085 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-03-27T10:48:32.293 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:48:32.318+0000 7f8c9ce00700 1 --2- 172.21.15.114:0/1858444085 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8c980765f0 0x7f8c9819ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:48:32.293 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:48:32.318+0000 7f8c9e200700 1 --2- 172.21.15.114:0/1858444085 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c9810a5a0 0x7f8c981a1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:48:32.293 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:48:32.318+0000 7f8c9d800700 1 -- 172.21.15.114:0/1858444085 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8c980765f0 msgr2=0x7f8c9819ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:48:32.293 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:48:32.318+0000 7f8c9d800700 1 --2- 172.21.15.114:0/1858444085 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8c980765f0 0x7f8c9819ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:48:32.293 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:48:32.318+0000 7f8c9d800700 1 -- 172.21.15.114:0/1858444085 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c9810a5a0 msgr2=0x7f8c981a1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:48:32.293 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:48:32.318+0000 7f8c9d800700 1 --2- 172.21.15.114:0/1858444085 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c9810a5a0 0x7f8c981a1ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:48:32.293 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:48:32.318+0000 7f8c9d800700 1 -- 172.21.15.114:0/1858444085 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c981a2570 con 0x7f8c98075020 2024-03-27T10:48:32.293 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:48:32.318+0000 7f8c9d800700 1 --2- 172.21.15.114:0/1858444085 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c98075020 0x7f8c9819a910 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c00d980 tx=0x7f8c8c00dd40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:48:32.293 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:48:32.319+0000 7f8c96000700 1 -- 172.21.15.114:0/1858444085 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c8c017070 con 0x7f8c98075020 2024-03-27T10:48:32.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:48:32.319+0000 7f8c96000700 1 -- 172.21.15.114:0/1858444085 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8c8c004830 con 0x7f8c98075020 2024-03-27T10:48:32.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:48:32.319+0000 7f8ca0200700 1 -- 172.21.15.114:0/1858444085 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c981a2860 con 0x7f8c98075020 2024-03-27T10:48:32.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:48:32.319+0000 7f8c96000700 1 -- 172.21.15.114:0/1858444085 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c8c005440 con 0x7f8c98075020 2024-03-27T10:48:32.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:48:32.319+0000 7f8ca0200700 1 -- 172.21.15.114:0/1858444085 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c981a2da0 con 0x7f8c98075020 2024-03-27T10:48:32.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:48:32.320+0000 7f8ca0200700 1 -- 172.21.15.114:0/1858444085 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c98069300 con 0x7f8c98075020 2024-03-27T10:48:32.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:48:32.322+0000 7f8c96000700 1 -- 172.21.15.114:0/1858444085 <== mon.2 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 16) ==== 138370+0+0 (secure 0 0 0) 0x7f8c8c0055e0 con 0x7f8c98075020 2024-03-27T10:48:32.296 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:48:32.323+0000 7f8c96000700 1 --2- 172.21.15.114:0/1858444085 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f8c800a3720 0x7f8c800a5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:48:32.296 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:48:32.323+0000 7f8c96000700 1 -- 172.21.15.114:0/1858444085 <== mon.2 v2:172.21.15.114:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 3669+0+0 (secure 0 0 0) 0x7f8c8c0ceed0 con 0x7f8c98075020 2024-03-27T10:48:32.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:48:32.324+0000 7f8c9ce00700 1 --2- 172.21.15.114:0/1858444085 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f8c800a3720 0x7f8c800a5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:48:32.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:48:32.325+0000 7f8c9ce00700 1 --2- 172.21.15.114:0/1858444085 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f8c800a3720 0x7f8c800a5be0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8c84005950 tx=0x7f8c840058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:48:32.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:48:32.327+0000 7f8c96000700 1 -- 172.21.15.114:0/1858444085 <== mon.2 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f8c8c020020 con 0x7f8c98075020 2024-03-27T10:48:32.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:48:32.558+0000 7f8ca0200700 1 -- 172.21.15.114:0/1858444085 --> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi114:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f8c98062b40 con 0x7f8c800a3720 2024-03-27T10:48:32.768 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:32 smithi114 ceph-mon[35338]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-27T10:48:32.813 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:32 smithi097 ceph-mon[31859]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-27T10:48:32.932 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:32 smithi177 ceph-mon[35294]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-27T10:48:33.703 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:33 smithi114 ceph-mon[35338]: from='client.34199 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi114:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T10:48:33.703 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:33 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-27T10:48:33.703 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:33 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-27T10:48:33.703 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:33 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:48:33.893 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:33 smithi097 ceph-mon[31859]: from='client.34199 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi114:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T10:48:33.893 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:33 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-27T10:48:33.893 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:33 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-27T10:48:33.893 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:33 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:48:33.928 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:33 smithi177 ceph-mon[35294]: from='client.34199 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi114:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T10:48:33.928 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:33 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-27T10:48:33.928 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:33 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-27T10:48:33.928 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:33 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:48:34.556 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:34 smithi114 ceph-mon[35338]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 113 MiB used, 357 GiB / 358 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-03-27T10:48:34.892 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:34 smithi097 ceph-mon[31859]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 113 MiB used, 357 GiB / 358 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-03-27T10:48:34.928 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:34 smithi177 ceph-mon[35294]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 113 MiB used, 357 GiB / 358 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-03-27T10:48:37.269 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:37 smithi114 ceph-mon[35338]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 113 MiB used, 357 GiB / 358 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-03-27T10:48:37.269 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:37 smithi114 ceph-mon[35338]: from='client.? 172.21.15.114:0/421413400' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9245a3a4-7d49-4fdb-9d55-112e43177050"}]: dispatch 2024-03-27T10:48:37.269 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:37 smithi114 ceph-mon[35338]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9245a3a4-7d49-4fdb-9d55-112e43177050"}]: dispatch 2024-03-27T10:48:37.269 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:37 smithi114 ceph-mon[35338]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9245a3a4-7d49-4fdb-9d55-112e43177050"}]': finished 2024-03-27T10:48:37.269 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:37 smithi114 ceph-mon[35338]: osdmap e32: 5 total, 4 up, 5 in 2024-03-27T10:48:37.269 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:37 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-27T10:48:37.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:37 smithi097 ceph-mon[31859]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 113 MiB used, 357 GiB / 358 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-03-27T10:48:37.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:37 smithi097 ceph-mon[31859]: from='client.? 172.21.15.114:0/421413400' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9245a3a4-7d49-4fdb-9d55-112e43177050"}]: dispatch 2024-03-27T10:48:37.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:37 smithi097 ceph-mon[31859]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9245a3a4-7d49-4fdb-9d55-112e43177050"}]: dispatch 2024-03-27T10:48:37.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:37 smithi097 ceph-mon[31859]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9245a3a4-7d49-4fdb-9d55-112e43177050"}]': finished 2024-03-27T10:48:37.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:37 smithi097 ceph-mon[31859]: osdmap e32: 5 total, 4 up, 5 in 2024-03-27T10:48:37.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:37 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-27T10:48:37.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:37 smithi177 ceph-mon[35294]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 113 MiB used, 357 GiB / 358 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-03-27T10:48:37.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:37 smithi177 ceph-mon[35294]: from='client.? 172.21.15.114:0/421413400' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9245a3a4-7d49-4fdb-9d55-112e43177050"}]: dispatch 2024-03-27T10:48:37.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:37 smithi177 ceph-mon[35294]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9245a3a4-7d49-4fdb-9d55-112e43177050"}]: dispatch 2024-03-27T10:48:37.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:37 smithi177 ceph-mon[35294]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9245a3a4-7d49-4fdb-9d55-112e43177050"}]': finished 2024-03-27T10:48:37.429 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:37 smithi177 ceph-mon[35294]: osdmap e32: 5 total, 4 up, 5 in 2024-03-27T10:48:37.429 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:37 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-27T10:48:38.268 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:38 smithi114 ceph-mon[35338]: from='client.? 172.21.15.114:0/1649429525' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-27T10:48:38.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:38 smithi097 ceph-mon[31859]: from='client.? 172.21.15.114:0/1649429525' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-27T10:48:38.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:38 smithi177 ceph-mon[35294]: from='client.? 172.21.15.114:0/1649429525' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-27T10:48:39.268 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:39 smithi114 ceph-mon[35338]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-03-27T10:48:39.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:39 smithi097 ceph-mon[31859]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-03-27T10:48:39.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:39 smithi177 ceph-mon[35294]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-03-27T10:48:41.268 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:41 smithi114 ceph-mon[35338]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-03-27T10:48:41.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:41 smithi097 ceph-mon[31859]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-03-27T10:48:41.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:41 smithi177 ceph-mon[35294]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-03-27T10:48:43.044 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:43 smithi114 ceph-mon[35338]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-03-27T10:48:43.143 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:43 smithi097 ceph-mon[31859]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-03-27T10:48:43.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:43 smithi177 ceph-mon[35294]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-03-27T10:48:45.268 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:45 smithi114 ceph-mon[35338]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-03-27T10:48:45.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:45 smithi097 ceph-mon[31859]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-03-27T10:48:45.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:45 smithi177 ceph-mon[35294]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-03-27T10:48:47.370 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:47 smithi114 ceph-mon[35338]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-03-27T10:48:47.370 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:47 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-27T10:48:47.370 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:47 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:48:47.370 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:47 smithi114 ceph-mon[35338]: Deploying daemon osd.4 on smithi114 2024-03-27T10:48:47.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:47 smithi097 ceph-mon[31859]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-03-27T10:48:47.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:47 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-27T10:48:47.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:47 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:48:47.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:47 smithi097 ceph-mon[31859]: Deploying daemon osd.4 on smithi114 2024-03-27T10:48:47.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:47 smithi177 ceph-mon[35294]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-03-27T10:48:47.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:47 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-27T10:48:47.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:47 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:48:47.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:47 smithi177 ceph-mon[35294]: Deploying daemon osd.4 on smithi114 2024-03-27T10:48:49.220 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:49 smithi114 ceph-mon[35338]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-03-27T10:48:49.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:49 smithi097 ceph-mon[31859]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-03-27T10:48:49.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:49 smithi177 ceph-mon[35294]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-03-27T10:48:51.268 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:51 smithi114 ceph-mon[35338]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-03-27T10:48:51.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:51 smithi097 ceph-mon[31859]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-03-27T10:48:51.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:51 smithi177 ceph-mon[35294]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-03-27T10:48:53.226 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:53 smithi114 ceph-mon[35338]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-03-27T10:48:53.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:53 smithi097 ceph-mon[31859]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-03-27T10:48:53.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:53 smithi177 ceph-mon[35294]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-03-27T10:48:55.312 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:55 smithi114 ceph-mon[35338]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-03-27T10:48:55.312 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:55 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:48:55.313 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:55 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:48:55.313 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:55 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:48:55.430 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:55 smithi177 ceph-mon[35294]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-03-27T10:48:55.430 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:55 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:48:55.430 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:55 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:48:55.430 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:55 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:48:55.642 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:55 smithi097 ceph-mon[31859]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-03-27T10:48:55.642 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:55 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:48:55.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:55 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:48:55.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:55 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:48:57.268 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:57 smithi114 ceph-mon[35338]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-03-27T10:48:57.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:57 smithi177 ceph-mon[35294]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-03-27T10:48:57.642 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:57 smithi097 ceph-mon[31859]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-03-27T10:48:58.892 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:58 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:48:58.893 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:58 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:48:58.893 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:58 smithi097 ceph-mon[31859]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-03-27T10:48:58.893 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:58 smithi097 ceph-mon[31859]: from='osd.4 [v2:172.21.15.114:6808/75755945,v1:172.21.15.114:6809/75755945]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-27T10:48:58.893 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:58 smithi097 ceph-mon[31859]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-27T10:48:58.929 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:58 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:48:58.930 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:58 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:48:58.930 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:58 smithi177 ceph-mon[35294]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-03-27T10:48:58.930 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:58 smithi177 ceph-mon[35294]: from='osd.4 [v2:172.21.15.114:6808/75755945,v1:172.21.15.114:6809/75755945]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-27T10:48:58.930 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:58 smithi177 ceph-mon[35294]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-27T10:48:59.004 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:58 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:48:59.005 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:58 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:48:59.005 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:58 smithi114 ceph-mon[35338]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-03-27T10:48:59.005 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:58 smithi114 ceph-mon[35338]: from='osd.4 [v2:172.21.15.114:6808/75755945,v1:172.21.15.114:6809/75755945]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-27T10:48:59.005 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:58 smithi114 ceph-mon[35338]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-27T10:48:59.160 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:48:59.184+0000 7f8c96000700 1 -- 172.21.15.114:0/1858444085 <== mgr.14152 v2:172.21.15.97:6800/3955463538 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8c98062b40 con 0x7f8c800a3720 2024-03-27T10:48:59.160 INFO:teuthology.orchestra.run.smithi114.stdout:Created osd(s) 4 on host 'smithi114' 2024-03-27T10:48:59.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:48:59.188+0000 7f8ca0200700 1 -- 172.21.15.114:0/1858444085 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f8c800a3720 msgr2=0x7f8c800a5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:48:59.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:48:59.188+0000 7f8ca0200700 1 --2- 172.21.15.114:0/1858444085 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f8c800a3720 0x7f8c800a5be0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8c84005950 tx=0x7f8c840058e0 comp rx=0 tx=0).stop 2024-03-27T10:48:59.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:48:59.188+0000 7f8ca0200700 1 -- 172.21.15.114:0/1858444085 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c98075020 msgr2=0x7f8c9819a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:48:59.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:48:59.188+0000 7f8ca0200700 1 --2- 172.21.15.114:0/1858444085 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c98075020 0x7f8c9819a910 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c00d980 tx=0x7f8c8c00dd40 comp rx=0 tx=0).stop 2024-03-27T10:48:59.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:48:59.188+0000 7f8ca0200700 1 -- 172.21.15.114:0/1858444085 shutdown_connections 2024-03-27T10:48:59.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:48:59.188+0000 7f8ca0200700 1 --2- 172.21.15.114:0/1858444085 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f8c800a3720 0x7f8c800a5be0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:48:59.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:48:59.188+0000 7f8ca0200700 1 --2- 172.21.15.114:0/1858444085 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c9810a5a0 0x7f8c981a1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:48:59.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:48:59.188+0000 7f8ca0200700 1 --2- 172.21.15.114:0/1858444085 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c98075020 0x7f8c9819a910 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:48:59.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:48:59.188+0000 7f8ca0200700 1 --2- 172.21.15.114:0/1858444085 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8c980765f0 0x7f8c9819ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:48:59.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:48:59.188+0000 7f8ca0200700 1 -- 172.21.15.114:0/1858444085 >> 172.21.15.114:0/1858444085 conn(0x7f8c980fdc40 msgr2=0x7f8c981064f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:48:59.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:48:59.188+0000 7f8ca0200700 1 -- 172.21.15.114:0/1858444085 shutdown_connections 2024-03-27T10:48:59.164 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:48:59.188+0000 7f8ca0200700 1 -- 172.21.15.114:0/1858444085 wait complete. 2024-03-27T10:48:59.893 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:59 smithi097 ceph-mon[31859]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-27T10:48:59.893 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:59 smithi097 ceph-mon[31859]: osdmap e33: 5 total, 4 up, 5 in 2024-03-27T10:48:59.893 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:59 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-27T10:48:59.893 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:59 smithi097 ceph-mon[31859]: from='osd.4 [v2:172.21.15.114:6808/75755945,v1:172.21.15.114:6809/75755945]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]: dispatch 2024-03-27T10:48:59.893 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:59 smithi097 ceph-mon[31859]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]: dispatch 2024-03-27T10:48:59.893 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:59 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:48:59.893 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:59 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:48:59.893 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:59 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:48:59.893 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:59 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:48:59.893 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:59 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:48:59.894 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:48:59 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:48:59.908 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:59 smithi114 ceph-mon[35338]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-27T10:48:59.909 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:59 smithi114 ceph-mon[35338]: osdmap e33: 5 total, 4 up, 5 in 2024-03-27T10:48:59.909 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:59 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-27T10:48:59.909 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:59 smithi114 ceph-mon[35338]: from='osd.4 [v2:172.21.15.114:6808/75755945,v1:172.21.15.114:6809/75755945]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]: dispatch 2024-03-27T10:48:59.909 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:59 smithi114 ceph-mon[35338]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]: dispatch 2024-03-27T10:48:59.909 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:59 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:48:59.909 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:59 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:48:59.909 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:59 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:48:59.909 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:59 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:48:59.909 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:59 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:48:59.909 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:48:59 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:48:59.928 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:59 smithi177 ceph-mon[35294]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-27T10:48:59.928 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:59 smithi177 ceph-mon[35294]: osdmap e33: 5 total, 4 up, 5 in 2024-03-27T10:48:59.928 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:59 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-27T10:48:59.928 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:59 smithi177 ceph-mon[35294]: from='osd.4 [v2:172.21.15.114:6808/75755945,v1:172.21.15.114:6809/75755945]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]: dispatch 2024-03-27T10:48:59.928 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:59 smithi177 ceph-mon[35294]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]: dispatch 2024-03-27T10:48:59.928 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:59 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:48:59.929 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:59 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:48:59.929 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:59 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:48:59.929 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:59 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:48:59.929 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:59 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:48:59.929 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:48:59 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:48:59.968 DEBUG:teuthology.orchestra.run.smithi114:osd.4> sudo journalctl -f -n 0 -u ceph-adca88c4-ec26-11ee-95d0-87774f69a715@osd.4.service 2024-03-27T10:48:59.975 INFO:tasks.cephadm:Deploying osd.5 on smithi177 with /dev/vg_nvme/lv_3... 2024-03-27T10:48:59.976 DEBUG:teuthology.orchestra.run.smithi177:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid adca88c4-ec26-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-27T10:49:00.408 INFO:teuthology.orchestra.run.smithi177.stderr:Inferring config /var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/mon.c/config 2024-03-27T10:49:00.675 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:00 smithi114 ceph-mon[35338]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]': finished 2024-03-27T10:49:00.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:00 smithi114 ceph-mon[35338]: osdmap e34: 5 total, 4 up, 5 in 2024-03-27T10:49:00.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:00 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-27T10:49:00.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:00 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-27T10:49:00.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:00 smithi114 ceph-mon[35338]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-03-27T10:49:00.676 INFO:journalctl@ceph.osd.4.smithi114.stdout:Mar 27 10:49:00 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-4[46437]: 2024-03-27T10:49:00.682+0000 7f5de3c00700 -1 osd.4 0 waiting for initial osdmap 2024-03-27T10:49:00.825 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:00 smithi177 ceph-mon[35294]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]': finished 2024-03-27T10:49:00.826 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:00 smithi177 ceph-mon[35294]: osdmap e34: 5 total, 4 up, 5 in 2024-03-27T10:49:00.826 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:00 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-27T10:49:00.826 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:00 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-27T10:49:00.826 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:00 smithi177 ceph-mon[35294]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-03-27T10:49:00.892 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:00 smithi097 ceph-mon[31859]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]': finished 2024-03-27T10:49:00.893 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:00 smithi097 ceph-mon[31859]: osdmap e34: 5 total, 4 up, 5 in 2024-03-27T10:49:00.893 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:00 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-27T10:49:00.893 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:00 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-27T10:49:00.893 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:00 smithi097 ceph-mon[31859]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-03-27T10:49:01.018 INFO:journalctl@ceph.osd.4.smithi114.stdout:Mar 27 10:49:00 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-4[46437]: 2024-03-27T10:49:00.721+0000 7f5dd9000700 -1 osd.4 34 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-27T10:49:01.884 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:01 smithi114 ceph-mon[35338]: purged_snaps scrub starts 2024-03-27T10:49:01.884 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:01 smithi114 ceph-mon[35338]: purged_snaps scrub ok 2024-03-27T10:49:01.884 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:01 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-27T10:49:01.884 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:01 smithi114 ceph-mon[35338]: from='osd.4 ' entity='osd.4' 2024-03-27T10:49:01.885 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:01 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-27T10:49:01.892 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:01 smithi097 ceph-mon[31859]: purged_snaps scrub starts 2024-03-27T10:49:01.892 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:01 smithi097 ceph-mon[31859]: purged_snaps scrub ok 2024-03-27T10:49:01.892 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:01 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-27T10:49:01.893 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:01 smithi097 ceph-mon[31859]: from='osd.4 ' entity='osd.4' 2024-03-27T10:49:01.893 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:01 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-27T10:49:01.928 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:01 smithi177 ceph-mon[35294]: purged_snaps scrub starts 2024-03-27T10:49:01.928 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:01 smithi177 ceph-mon[35294]: purged_snaps scrub ok 2024-03-27T10:49:01.928 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:01 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-27T10:49:01.928 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:01 smithi177 ceph-mon[35294]: from='osd.4 ' entity='osd.4' 2024-03-27T10:49:01.928 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:01 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-27T10:49:03.018 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:02 smithi114 ceph-mon[35338]: osd.4 [v2:172.21.15.114:6808/75755945,v1:172.21.15.114:6809/75755945] boot 2024-03-27T10:49:03.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:02 smithi114 ceph-mon[35338]: osdmap e35: 5 total, 5 up, 5 in 2024-03-27T10:49:03.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:02 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-27T10:49:03.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:02 smithi114 ceph-mon[35338]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-03-27T10:49:03.142 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:02 smithi097 ceph-mon[31859]: osd.4 [v2:172.21.15.114:6808/75755945,v1:172.21.15.114:6809/75755945] boot 2024-03-27T10:49:03.143 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:02 smithi097 ceph-mon[31859]: osdmap e35: 5 total, 5 up, 5 in 2024-03-27T10:49:03.143 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:02 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-27T10:49:03.143 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:02 smithi097 ceph-mon[31859]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-03-27T10:49:03.143 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:02 smithi177 ceph-mon[35294]: osd.4 [v2:172.21.15.114:6808/75755945,v1:172.21.15.114:6809/75755945] boot 2024-03-27T10:49:03.143 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:02 smithi177 ceph-mon[35294]: osdmap e35: 5 total, 5 up, 5 in 2024-03-27T10:49:03.144 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:02 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-27T10:49:03.144 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:02 smithi177 ceph-mon[35294]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-03-27T10:49:03.144 INFO:teuthology.orchestra.run.smithi177.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-27T10:49:03.144 INFO:teuthology.orchestra.run.smithi177.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-27T10:49:03.144 INFO:teuthology.orchestra.run.smithi177.stderr: stderr: 10+0 records in 2024-03-27T10:49:03.144 INFO:teuthology.orchestra.run.smithi177.stderr:10+0 records out 2024-03-27T10:49:03.144 INFO:teuthology.orchestra.run.smithi177.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0367143 s, 286 MB/s 2024-03-27T10:49:03.144 INFO:teuthology.orchestra.run.smithi177.stderr:--> Zapping successful for: 2024-03-27T10:49:03.817 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:03 smithi177 ceph-mon[35294]: osdmap e36: 5 total, 5 up, 5 in 2024-03-27T10:49:03.877 DEBUG:teuthology.orchestra.run.smithi177:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid adca88c4-ec26-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi177:vg_nvme/lv_3 2024-03-27T10:49:04.018 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:03 smithi114 ceph-mon[35338]: osdmap e36: 5 total, 5 up, 5 in 2024-03-27T10:49:04.142 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:03 smithi097 ceph-mon[31859]: osdmap e36: 5 total, 5 up, 5 in 2024-03-27T10:49:04.321 INFO:teuthology.orchestra.run.smithi177.stderr:Inferring config /var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/mon.c/config 2024-03-27T10:49:04.814 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:04 smithi177 ceph-mon[35294]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-27T10:49:05.018 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:04 smithi114 ceph-mon[35338]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-27T10:49:05.142 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:04 smithi097 ceph-mon[31859]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-27T10:49:05.826 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:05 smithi177 ceph-mon[35294]: Detected new or changed devices on smithi114 2024-03-27T10:49:05.827 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:05 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:49:05.827 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:05 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:49:05.827 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:05 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-27T10:49:05.827 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:05 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-27T10:49:05.827 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:05 smithi177 ceph-mon[35294]: Adjusting osd_memory_target on smithi114 to 8545M 2024-03-27T10:49:05.827 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:05 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:49:05.827 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:05 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:49:05.827 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:05 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:49:05.827 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:05 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:49:06.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:05 smithi114 ceph-mon[35338]: Detected new or changed devices on smithi114 2024-03-27T10:49:06.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:05 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:49:06.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:05 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:49:06.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:05 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-27T10:49:06.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:05 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-27T10:49:06.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:05 smithi114 ceph-mon[35338]: Adjusting osd_memory_target on smithi114 to 8545M 2024-03-27T10:49:06.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:05 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:49:06.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:05 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:49:06.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:05 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:49:06.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:05 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:49:06.142 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:05 smithi097 ceph-mon[31859]: Detected new or changed devices on smithi114 2024-03-27T10:49:06.143 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:05 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:49:06.143 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:05 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:49:06.143 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:05 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-27T10:49:06.143 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:05 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-27T10:49:06.143 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:05 smithi097 ceph-mon[31859]: Adjusting osd_memory_target on smithi114 to 8545M 2024-03-27T10:49:06.143 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:05 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:49:06.143 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:05 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:49:06.143 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:05 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:49:06.143 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:05 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:49:06.304 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:49:06.330+0000 7efe7d600700 1 -- 172.21.15.177:0/1928794172 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efe78103870 msgr2=0x7efe78103cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:49:06.304 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:49:06.330+0000 7efe7d600700 1 --2- 172.21.15.177:0/1928794172 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efe78103870 0x7efe78103cd0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7efe64009990 tx=0x7efe64009ca0 comp rx=0 tx=0).stop 2024-03-27T10:49:06.304 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:49:06.330+0000 7efe7d600700 1 -- 172.21.15.177:0/1928794172 shutdown_connections 2024-03-27T10:49:06.304 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:49:06.330+0000 7efe7d600700 1 --2- 172.21.15.177:0/1928794172 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe78104210 0x7efe7810aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:49:06.304 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:49:06.330+0000 7efe7d600700 1 --2- 172.21.15.177:0/1928794172 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efe78102670 0x7efe78102a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:49:06.304 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:49:06.330+0000 7efe7d600700 1 --2- 172.21.15.177:0/1928794172 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efe78103870 0x7efe78103cd0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:49:06.304 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:49:06.330+0000 7efe7d600700 1 -- 172.21.15.177:0/1928794172 >> 172.21.15.177:0/1928794172 conn(0x7efe780fdc50 msgr2=0x7efe78100070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:49:06.304 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:49:06.330+0000 7efe7d600700 1 -- 172.21.15.177:0/1928794172 shutdown_connections 2024-03-27T10:49:06.304 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:49:06.330+0000 7efe7d600700 1 -- 172.21.15.177:0/1928794172 wait complete. 2024-03-27T10:49:06.306 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:49:06.331+0000 7efe7d600700 1 Processor -- start 2024-03-27T10:49:06.306 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:49:06.331+0000 7efe7d600700 1 -- start start 2024-03-27T10:49:06.306 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:49:06.332+0000 7efe7d600700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe78102670 0x7efe78198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:49:06.306 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:49:06.332+0000 7efe7d600700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efe78103870 0x7efe78198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:49:06.306 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:49:06.332+0000 7efe7d600700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efe78104210 0x7efe7819ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:49:06.306 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:49:06.332+0000 7efe7d600700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7efe7810e250 con 0x7efe78102670 2024-03-27T10:49:06.306 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:49:06.332+0000 7efe7d600700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7efe7810e0d0 con 0x7efe78103870 2024-03-27T10:49:06.306 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:49:06.332+0000 7efe7d600700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7efe7810e3d0 con 0x7efe78104210 2024-03-27T10:49:06.306 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:49:06.332+0000 7efe77400700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efe78104210 0x7efe7819ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:49:06.306 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:49:06.332+0000 7efe77400700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efe78104210 0x7efe7819ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:58330/0 (socket says 172.21.15.177:58330) 2024-03-27T10:49:06.307 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:49:06.332+0000 7efe77400700 1 -- 172.21.15.177:0/939188439 learned_addr learned my addr 172.21.15.177:0/939188439 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-03-27T10:49:06.307 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:49:06.332+0000 7efe76000700 1 --2- 172.21.15.177:0/939188439 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efe78103870 0x7efe78198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:49:06.307 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:49:06.332+0000 7efe76a00700 1 --2- 172.21.15.177:0/939188439 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe78102670 0x7efe78198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:49:06.307 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:49:06.333+0000 7efe77400700 1 -- 172.21.15.177:0/939188439 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efe78103870 msgr2=0x7efe78198ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:49:06.307 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:49:06.333+0000 7efe77400700 1 --2- 172.21.15.177:0/939188439 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efe78103870 0x7efe78198ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:49:06.307 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:49:06.333+0000 7efe77400700 1 -- 172.21.15.177:0/939188439 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe78102670 msgr2=0x7efe78198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:49:06.307 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:49:06.333+0000 7efe77400700 1 --2- 172.21.15.177:0/939188439 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe78102670 0x7efe78198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:49:06.307 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:49:06.333+0000 7efe77400700 1 -- 172.21.15.177:0/939188439 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe781a05c0 con 0x7efe78104210 2024-03-27T10:49:06.308 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:49:06.333+0000 7efe77400700 1 --2- 172.21.15.177:0/939188439 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efe78104210 0x7efe7819ff20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7efe7000b9a0 tx=0x7efe7000bcb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:49:06.308 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:49:06.333+0000 7efe6f400700 1 -- 172.21.15.177:0/939188439 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe7000d850 con 0x7efe78104210 2024-03-27T10:49:06.308 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:49:06.334+0000 7efe6f400700 1 -- 172.21.15.177:0/939188439 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efe7000e420 con 0x7efe78104210 2024-03-27T10:49:06.308 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:49:06.334+0000 7efe7d600700 1 -- 172.21.15.177:0/939188439 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe781a08b0 con 0x7efe78104210 2024-03-27T10:49:06.308 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:49:06.334+0000 7efe6f400700 1 -- 172.21.15.177:0/939188439 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe70017640 con 0x7efe78104210 2024-03-27T10:49:06.308 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:49:06.334+0000 7efe7d600700 1 -- 172.21.15.177:0/939188439 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe781a0e20 con 0x7efe78104210 2024-03-27T10:49:06.308 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:49:06.335+0000 7efe7d600700 1 -- 172.21.15.177:0/939188439 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe78069300 con 0x7efe78104210 2024-03-27T10:49:06.310 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:49:06.337+0000 7efe6f400700 1 -- 172.21.15.177:0/939188439 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 16) ==== 138370+0+0 (secure 0 0 0) 0x7efe7000e5d0 con 0x7efe78104210 2024-03-27T10:49:06.311 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:49:06.338+0000 7efe6f400700 1 --2- 172.21.15.177:0/939188439 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7efe5c0a36a0 0x7efe5c0a5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:49:06.311 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:49:06.338+0000 7efe6f400700 1 -- 172.21.15.177:0/939188439 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4101+0+0 (secure 0 0 0) 0x7efe700ce9d0 con 0x7efe78104210 2024-03-27T10:49:06.312 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:49:06.339+0000 7efe76a00700 1 --2- 172.21.15.177:0/939188439 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7efe5c0a36a0 0x7efe5c0a5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:49:06.313 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:49:06.340+0000 7efe76a00700 1 --2- 172.21.15.177:0/939188439 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7efe5c0a36a0 0x7efe5c0a5b60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7efe781036d0 tx=0x7efe68009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:49:06.315 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:49:06.342+0000 7efe6f400700 1 -- 172.21.15.177:0/939188439 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7efe70013030 con 0x7efe78104210 2024-03-27T10:49:06.552 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:49:06.579+0000 7efe7d600700 1 -- 172.21.15.177:0/939188439 --> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi177:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7efe78062a80 con 0x7efe5c0a36a0 2024-03-27T10:49:06.928 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:06 smithi177 ceph-mon[35294]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-27T10:49:06.928 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:06 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-27T10:49:06.928 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:06 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-27T10:49:06.928 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:06 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:49:07.018 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:06 smithi114 ceph-mon[35338]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-27T10:49:07.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:06 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-27T10:49:07.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:06 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-27T10:49:07.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:06 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:49:07.142 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:06 smithi097 ceph-mon[31859]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-27T10:49:07.142 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:06 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-27T10:49:07.143 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:06 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-27T10:49:07.143 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:06 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:49:07.779 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:07 smithi177 ceph-mon[35294]: from='client.24223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi177:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T10:49:08.018 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:07 smithi114 ceph-mon[35338]: from='client.24223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi177:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T10:49:08.142 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:07 smithi097 ceph-mon[31859]: from='client.24223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi177:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T10:49:09.018 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:08 smithi114 ceph-mon[35338]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-27T10:49:09.085 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:08 smithi177 ceph-mon[35294]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-27T10:49:09.142 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:08 smithi097 ceph-mon[31859]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-27T10:49:11.178 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:11 smithi177 ceph-mon[35294]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-27T10:49:11.179 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:11 smithi177 ceph-mon[35294]: from='client.? 172.21.15.177:0/94207511' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "51922f99-e8d3-461b-8615-e9bfdcbf5ed3"}]: dispatch 2024-03-27T10:49:11.179 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:11 smithi177 ceph-mon[35294]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "51922f99-e8d3-461b-8615-e9bfdcbf5ed3"}]: dispatch 2024-03-27T10:49:11.179 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:11 smithi177 ceph-mon[35294]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "51922f99-e8d3-461b-8615-e9bfdcbf5ed3"}]': finished 2024-03-27T10:49:11.179 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:11 smithi177 ceph-mon[35294]: osdmap e37: 6 total, 5 up, 6 in 2024-03-27T10:49:11.179 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:11 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-27T10:49:11.269 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:11 smithi114 ceph-mon[35338]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-27T10:49:11.269 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:11 smithi114 ceph-mon[35338]: from='client.? 172.21.15.177:0/94207511' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "51922f99-e8d3-461b-8615-e9bfdcbf5ed3"}]: dispatch 2024-03-27T10:49:11.269 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:11 smithi114 ceph-mon[35338]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "51922f99-e8d3-461b-8615-e9bfdcbf5ed3"}]: dispatch 2024-03-27T10:49:11.269 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:11 smithi114 ceph-mon[35338]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "51922f99-e8d3-461b-8615-e9bfdcbf5ed3"}]': finished 2024-03-27T10:49:11.269 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:11 smithi114 ceph-mon[35338]: osdmap e37: 6 total, 5 up, 6 in 2024-03-27T10:49:11.269 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:11 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-27T10:49:11.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:11 smithi097 ceph-mon[31859]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-27T10:49:11.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:11 smithi097 ceph-mon[31859]: from='client.? 172.21.15.177:0/94207511' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "51922f99-e8d3-461b-8615-e9bfdcbf5ed3"}]: dispatch 2024-03-27T10:49:11.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:11 smithi097 ceph-mon[31859]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "51922f99-e8d3-461b-8615-e9bfdcbf5ed3"}]: dispatch 2024-03-27T10:49:11.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:11 smithi097 ceph-mon[31859]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "51922f99-e8d3-461b-8615-e9bfdcbf5ed3"}]': finished 2024-03-27T10:49:11.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:11 smithi097 ceph-mon[31859]: osdmap e37: 6 total, 5 up, 6 in 2024-03-27T10:49:11.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:11 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-27T10:49:12.268 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:12 smithi114 ceph-mon[35338]: from='client.? 172.21.15.177:0/2744315298' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-27T10:49:12.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:12 smithi097 ceph-mon[31859]: from='client.? 172.21.15.177:0/2744315298' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-27T10:49:12.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:12 smithi177 ceph-mon[35294]: from='client.? 172.21.15.177:0/2744315298' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-27T10:49:13.269 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:13 smithi114 ceph-mon[35338]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-27T10:49:13.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:13 smithi097 ceph-mon[31859]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-27T10:49:13.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:13 smithi177 ceph-mon[35294]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-27T10:49:15.178 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:15 smithi177 ceph-mon[35294]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-27T10:49:15.268 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:15 smithi114 ceph-mon[35338]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-27T10:49:15.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:15 smithi097 ceph-mon[31859]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-27T10:49:17.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:17 smithi097 ceph-mon[31859]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-27T10:49:17.425 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:17 smithi177 ceph-mon[35294]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-27T10:49:17.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:17 smithi114 ceph-mon[35338]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-27T10:49:19.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:19 smithi097 ceph-mon[31859]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-27T10:49:19.399 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:19 smithi177 ceph-mon[35294]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-27T10:49:19.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:19 smithi114 ceph-mon[35338]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-27T10:49:20.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:20 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-27T10:49:20.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:20 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:49:20.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:20 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-27T10:49:20.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:20 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:49:20.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:20 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-27T10:49:20.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:20 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:49:21.161 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:21 smithi177 ceph-mon[35294]: Deploying daemon osd.5 on smithi177 2024-03-27T10:49:21.161 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:21 smithi177 ceph-mon[35294]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-27T10:49:21.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:21 smithi097 ceph-mon[31859]: Deploying daemon osd.5 on smithi177 2024-03-27T10:49:21.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:21 smithi097 ceph-mon[31859]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-27T10:49:21.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:21 smithi114 ceph-mon[35338]: Deploying daemon osd.5 on smithi177 2024-03-27T10:49:21.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:21 smithi114 ceph-mon[35338]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-27T10:49:23.211 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:23 smithi177 ceph-mon[35294]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-27T10:49:23.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:23 smithi097 ceph-mon[31859]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-27T10:49:23.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:23 smithi114 ceph-mon[35338]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-27T10:49:25.179 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:25 smithi177 ceph-mon[35294]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-27T10:49:25.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:25 smithi097 ceph-mon[31859]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-27T10:49:25.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:25 smithi114 ceph-mon[35338]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-27T10:49:27.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:27 smithi097 ceph-mon[31859]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-27T10:49:27.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:27 smithi177 ceph-mon[35294]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-27T10:49:27.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:27 smithi114 ceph-mon[35338]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-27T10:49:28.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:28 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:49:28.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:28 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:49:28.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:28 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:49:28.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:28 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:49:28.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:28 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:49:28.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:28 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:49:28.642 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:28 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:49:28.642 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:28 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:49:28.642 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:28 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:49:29.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:29 smithi177 ceph-mon[35294]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-27T10:49:29.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:29 smithi114 ceph-mon[35338]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-27T10:49:29.642 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:29 smithi097 ceph-mon[31859]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-27T10:49:31.411 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:31 smithi177 ceph-mon[35294]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-27T10:49:31.411 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:31 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:49:31.411 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:31 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:49:31.411 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:31 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:49:31.411 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:31 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:49:31.411 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:31 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:49:31.411 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:31 smithi177 ceph-mon[35294]: from='osd.5 [v2:172.21.15.177:6808/1918493790,v1:172.21.15.177:6809/1918493790]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-27T10:49:31.411 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:31 smithi177 ceph-mon[35294]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-27T10:49:31.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:31 smithi114 ceph-mon[35338]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-27T10:49:31.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:31 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:49:31.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:31 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:49:31.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:31 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:49:31.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:31 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:49:31.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:31 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:49:31.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:31 smithi114 ceph-mon[35338]: from='osd.5 [v2:172.21.15.177:6808/1918493790,v1:172.21.15.177:6809/1918493790]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-27T10:49:31.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:31 smithi114 ceph-mon[35338]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-27T10:49:31.642 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:31 smithi097 ceph-mon[31859]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-27T10:49:31.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:31 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:49:31.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:31 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:49:31.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:31 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:49:31.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:31 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:49:31.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:31 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:49:31.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:31 smithi097 ceph-mon[31859]: from='osd.5 [v2:172.21.15.177:6808/1918493790,v1:172.21.15.177:6809/1918493790]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-27T10:49:31.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:31 smithi097 ceph-mon[31859]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-27T10:49:31.865 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:49:31.890+0000 7efe6f400700 1 -- 172.21.15.177:0/939188439 <== mgr.14152 v2:172.21.15.97:6800/3955463538 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7efe78062a80 con 0x7efe5c0a36a0 2024-03-27T10:49:31.865 INFO:teuthology.orchestra.run.smithi177.stdout:Created osd(s) 5 on host 'smithi177' 2024-03-27T10:49:31.868 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:49:31.894+0000 7efe7d600700 1 -- 172.21.15.177:0/939188439 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7efe5c0a36a0 msgr2=0x7efe5c0a5b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:49:31.868 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:49:31.894+0000 7efe7d600700 1 --2- 172.21.15.177:0/939188439 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7efe5c0a36a0 0x7efe5c0a5b60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7efe781036d0 tx=0x7efe68009190 comp rx=0 tx=0).stop 2024-03-27T10:49:31.868 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:49:31.894+0000 7efe7d600700 1 -- 172.21.15.177:0/939188439 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efe78104210 msgr2=0x7efe7819ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:49:31.868 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:49:31.894+0000 7efe7d600700 1 --2- 172.21.15.177:0/939188439 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efe78104210 0x7efe7819ff20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7efe7000b9a0 tx=0x7efe7000bcb0 comp rx=0 tx=0).stop 2024-03-27T10:49:31.868 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:49:31.894+0000 7efe7d600700 1 -- 172.21.15.177:0/939188439 shutdown_connections 2024-03-27T10:49:31.868 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:49:31.894+0000 7efe7d600700 1 --2- 172.21.15.177:0/939188439 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7efe5c0a36a0 0x7efe5c0a5b60 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:49:31.868 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:49:31.894+0000 7efe7d600700 1 --2- 172.21.15.177:0/939188439 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efe78104210 0x7efe7819ff20 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:49:31.869 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:49:31.894+0000 7efe7d600700 1 --2- 172.21.15.177:0/939188439 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe78102670 0x7efe78198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:49:31.869 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:49:31.894+0000 7efe7d600700 1 --2- 172.21.15.177:0/939188439 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efe78103870 0x7efe78198ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:49:31.869 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:49:31.894+0000 7efe7d600700 1 -- 172.21.15.177:0/939188439 >> 172.21.15.177:0/939188439 conn(0x7efe780fdc50 msgr2=0x7efe78106cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:49:31.869 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:49:31.894+0000 7efe7d600700 1 -- 172.21.15.177:0/939188439 shutdown_connections 2024-03-27T10:49:31.869 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:49:31.894+0000 7efe7d600700 1 -- 172.21.15.177:0/939188439 wait complete. 2024-03-27T10:49:32.663 DEBUG:teuthology.orchestra.run.smithi177:osd.5> sudo journalctl -f -n 0 -u ceph-adca88c4-ec26-11ee-95d0-87774f69a715@osd.5.service 2024-03-27T10:49:32.668 INFO:tasks.cephadm:Deploying osd.6 on smithi097 with /dev/vg_nvme/lv_2... 2024-03-27T10:49:32.668 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid adca88c4-ec26-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-27T10:49:32.883 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:32 smithi177 ceph-mon[35294]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-27T10:49:32.883 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:32 smithi177 ceph-mon[35294]: osdmap e38: 6 total, 5 up, 6 in 2024-03-27T10:49:32.883 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:32 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-27T10:49:32.883 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:32 smithi177 ceph-mon[35294]: from='osd.5 [v2:172.21.15.177:6808/1918493790,v1:172.21.15.177:6809/1918493790]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2024-03-27T10:49:32.883 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:32 smithi177 ceph-mon[35294]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2024-03-27T10:49:32.883 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:32 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:49:32.883 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:32 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:49:32.884 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:32 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:49:32.884 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:32 smithi177 ceph-mon[35294]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-27T10:49:32.892 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:32 smithi097 ceph-mon[31859]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-27T10:49:32.892 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:32 smithi097 ceph-mon[31859]: osdmap e38: 6 total, 5 up, 6 in 2024-03-27T10:49:32.893 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:32 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-27T10:49:32.893 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:32 smithi097 ceph-mon[31859]: from='osd.5 [v2:172.21.15.177:6808/1918493790,v1:172.21.15.177:6809/1918493790]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2024-03-27T10:49:32.893 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:32 smithi097 ceph-mon[31859]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2024-03-27T10:49:32.893 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:32 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:49:32.893 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:32 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:49:32.893 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:32 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:49:32.893 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:32 smithi097 ceph-mon[31859]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-27T10:49:33.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:32 smithi114 ceph-mon[35338]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-27T10:49:33.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:32 smithi114 ceph-mon[35338]: osdmap e38: 6 total, 5 up, 6 in 2024-03-27T10:49:33.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:32 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-27T10:49:33.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:32 smithi114 ceph-mon[35338]: from='osd.5 [v2:172.21.15.177:6808/1918493790,v1:172.21.15.177:6809/1918493790]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2024-03-27T10:49:33.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:32 smithi114 ceph-mon[35338]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2024-03-27T10:49:33.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:32 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:49:33.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:32 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:49:33.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:32 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:49:33.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:32 smithi114 ceph-mon[35338]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-27T10:49:33.112 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T10:49:33.843 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:33 smithi177 ceph-mon[35294]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]': finished 2024-03-27T10:49:33.843 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:33 smithi177 ceph-mon[35294]: osdmap e39: 6 total, 5 up, 6 in 2024-03-27T10:49:33.843 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:33 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-27T10:49:33.843 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:33 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-27T10:49:33.843 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:33 smithi177 ceph-mon[35294]: from='osd.5 ' entity='osd.5' 2024-03-27T10:49:33.843 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:33 smithi177 ceph-mon[35294]: osdmap e40: 6 total, 5 up, 6 in 2024-03-27T10:49:33.843 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:33 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-27T10:49:33.844 INFO:journalctl@ceph.osd.5.smithi177.stdout:Mar 27 10:49:33 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-5[47487]: 2024-03-27T10:49:33.582+0000 7fe11f400700 -1 osd.5 0 waiting for initial osdmap 2024-03-27T10:49:33.844 INFO:journalctl@ceph.osd.5.smithi177.stdout:Mar 27 10:49:33 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-5[47487]: 2024-03-27T10:49:33.666+0000 7fe115200700 -1 osd.5 39 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-27T10:49:33.892 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:33 smithi097 ceph-mon[31859]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]': finished 2024-03-27T10:49:33.892 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:33 smithi097 ceph-mon[31859]: osdmap e39: 6 total, 5 up, 6 in 2024-03-27T10:49:33.892 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:33 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-27T10:49:33.893 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:33 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-27T10:49:33.893 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:33 smithi097 ceph-mon[31859]: from='osd.5 ' entity='osd.5' 2024-03-27T10:49:33.893 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:33 smithi097 ceph-mon[31859]: osdmap e40: 6 total, 5 up, 6 in 2024-03-27T10:49:33.893 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:33 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-27T10:49:34.018 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:33 smithi114 ceph-mon[35338]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]': finished 2024-03-27T10:49:34.018 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:33 smithi114 ceph-mon[35338]: osdmap e39: 6 total, 5 up, 6 in 2024-03-27T10:49:34.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:33 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-27T10:49:34.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:33 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-27T10:49:34.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:33 smithi114 ceph-mon[35338]: from='osd.5 ' entity='osd.5' 2024-03-27T10:49:34.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:33 smithi114 ceph-mon[35338]: osdmap e40: 6 total, 5 up, 6 in 2024-03-27T10:49:34.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:33 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-27T10:49:34.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:34 smithi097 ceph-mon[31859]: purged_snaps scrub starts 2024-03-27T10:49:34.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:34 smithi097 ceph-mon[31859]: purged_snaps scrub ok 2024-03-27T10:49:34.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:34 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-27T10:49:34.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:34 smithi097 ceph-mon[31859]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-27T10:49:34.928 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:34 smithi177 ceph-mon[35294]: purged_snaps scrub starts 2024-03-27T10:49:34.928 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:34 smithi177 ceph-mon[35294]: purged_snaps scrub ok 2024-03-27T10:49:34.928 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:34 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-27T10:49:34.928 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:34 smithi177 ceph-mon[35294]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-27T10:49:35.018 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:34 smithi114 ceph-mon[35338]: purged_snaps scrub starts 2024-03-27T10:49:35.018 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:34 smithi114 ceph-mon[35338]: purged_snaps scrub ok 2024-03-27T10:49:35.018 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:34 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-27T10:49:35.018 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:34 smithi114 ceph-mon[35338]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-27T10:49:35.709 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:35 smithi097 ceph-mon[31859]: osd.5 [v2:172.21.15.177:6808/1918493790,v1:172.21.15.177:6809/1918493790] boot 2024-03-27T10:49:35.709 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:35 smithi097 ceph-mon[31859]: osdmap e41: 6 total, 6 up, 6 in 2024-03-27T10:49:35.709 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:35 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-27T10:49:35.709 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:35 smithi097 ceph-mon[31859]: osdmap e42: 6 total, 6 up, 6 in 2024-03-27T10:49:35.928 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:35 smithi177 ceph-mon[35294]: osd.5 [v2:172.21.15.177:6808/1918493790,v1:172.21.15.177:6809/1918493790] boot 2024-03-27T10:49:35.928 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:35 smithi177 ceph-mon[35294]: osdmap e41: 6 total, 6 up, 6 in 2024-03-27T10:49:35.928 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:35 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-27T10:49:35.928 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:35 smithi177 ceph-mon[35294]: osdmap e42: 6 total, 6 up, 6 in 2024-03-27T10:49:35.995 INFO:teuthology.orchestra.run.smithi097.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-27T10:49:35.995 INFO:teuthology.orchestra.run.smithi097.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-27T10:49:35.995 INFO:teuthology.orchestra.run.smithi097.stderr: stderr: 10+0 records in 2024-03-27T10:49:35.995 INFO:teuthology.orchestra.run.smithi097.stderr:10+0 records out 2024-03-27T10:49:35.995 INFO:teuthology.orchestra.run.smithi097.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0323974 s, 324 MB/s 2024-03-27T10:49:35.995 INFO:teuthology.orchestra.run.smithi097.stderr:--> Zapping successful for: 2024-03-27T10:49:36.018 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:35 smithi114 ceph-mon[35338]: osd.5 [v2:172.21.15.177:6808/1918493790,v1:172.21.15.177:6809/1918493790] boot 2024-03-27T10:49:36.018 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:35 smithi114 ceph-mon[35338]: osdmap e41: 6 total, 6 up, 6 in 2024-03-27T10:49:36.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:35 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-27T10:49:36.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:35 smithi114 ceph-mon[35338]: osdmap e42: 6 total, 6 up, 6 in 2024-03-27T10:49:36.758 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid adca88c4-ec26-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi097:vg_nvme/lv_2 2024-03-27T10:49:37.186 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T10:49:37.331 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:37 smithi097 ceph-mon[31859]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-03-27T10:49:37.331 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:37 smithi097 ceph-mon[31859]: osdmap e43: 6 total, 6 up, 6 in 2024-03-27T10:49:37.331 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:37 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:49:37.331 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:37 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:49:37.331 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:37 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-27T10:49:37.331 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:37 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-27T10:49:37.331 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:37 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:49:37.331 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:37 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:49:37.331 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:37 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:49:37.332 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:37 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:49:37.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:37 smithi177 ceph-mon[35294]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-03-27T10:49:37.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:37 smithi177 ceph-mon[35294]: osdmap e43: 6 total, 6 up, 6 in 2024-03-27T10:49:37.429 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:37 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:49:37.429 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:37 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:49:37.429 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:37 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-27T10:49:37.429 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:37 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-27T10:49:37.429 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:37 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:49:37.429 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:37 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:49:37.429 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:37 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:49:37.429 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:37 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:49:37.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:37 smithi114 ceph-mon[35338]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-03-27T10:49:37.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:37 smithi114 ceph-mon[35338]: osdmap e43: 6 total, 6 up, 6 in 2024-03-27T10:49:37.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:37 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:49:37.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:37 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:49:37.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:37 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-27T10:49:37.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:37 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-27T10:49:37.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:37 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:49:37.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:37 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:49:37.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:37 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:49:37.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:37 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:49:38.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:38 smithi097 ceph-mon[31859]: Detected new or changed devices on smithi177 2024-03-27T10:49:38.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:38 smithi097 ceph-mon[31859]: Adjusting osd_memory_target on smithi177 to 8545M 2024-03-27T10:49:38.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:38 smithi097 ceph-mon[31859]: osdmap e44: 6 total, 6 up, 6 in 2024-03-27T10:49:38.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:38 smithi177 ceph-mon[35294]: Detected new or changed devices on smithi177 2024-03-27T10:49:38.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:38 smithi177 ceph-mon[35294]: Adjusting osd_memory_target on smithi177 to 8545M 2024-03-27T10:49:38.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:38 smithi177 ceph-mon[35294]: osdmap e44: 6 total, 6 up, 6 in 2024-03-27T10:49:38.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:38 smithi114 ceph-mon[35338]: Detected new or changed devices on smithi177 2024-03-27T10:49:38.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:38 smithi114 ceph-mon[35338]: Adjusting osd_memory_target on smithi177 to 8545M 2024-03-27T10:49:38.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:38 smithi114 ceph-mon[35338]: osdmap e44: 6 total, 6 up, 6 in 2024-03-27T10:49:39.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:49:39.261+0000 7f0502200700 1 -- 172.21.15.97:0/2779487551 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04fc101570 msgr2=0x7f04fc101990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:49:39.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:49:39.261+0000 7f0502200700 1 --2- 172.21.15.97:0/2779487551 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04fc101570 0x7f04fc101990 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f04ec0099c0 tx=0x7f04ec009cd0 comp rx=0 tx=0).stop 2024-03-27T10:49:39.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:49:39.261+0000 7f0502200700 1 -- 172.21.15.97:0/2779487551 shutdown_connections 2024-03-27T10:49:39.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:49:39.261+0000 7f0502200700 1 --2- 172.21.15.97:0/2779487551 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f04fc103110 0x7f04fc1099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:49:39.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:49:39.261+0000 7f0502200700 1 --2- 172.21.15.97:0/2779487551 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04fc101570 0x7f04fc101990 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:49:39.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:49:39.261+0000 7f0502200700 1 --2- 172.21.15.97:0/2779487551 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f04fc102770 0x7f04fc102bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:49:39.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:49:39.261+0000 7f0502200700 1 -- 172.21.15.97:0/2779487551 >> 172.21.15.97:0/2779487551 conn(0x7f04fc0fcb10 msgr2=0x7f04fc0fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:49:39.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:49:39.261+0000 7f0502200700 1 -- 172.21.15.97:0/2779487551 shutdown_connections 2024-03-27T10:49:39.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:49:39.261+0000 7f0502200700 1 -- 172.21.15.97:0/2779487551 wait complete. 2024-03-27T10:49:39.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:49:39.262+0000 7f0502200700 1 Processor -- start 2024-03-27T10:49:39.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:49:39.262+0000 7f0502200700 1 -- start start 2024-03-27T10:49:39.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:49:39.262+0000 7f0502200700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f04fc101570 0x7f04fc06ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:49:39.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:49:39.263+0000 7f0502200700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f04fc102770 0x7f04fc19d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:49:39.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:49:39.263+0000 7f0502200700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04fc103110 0x7f04fc19fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:49:39.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:49:39.263+0000 7f0502200700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f04fc10d010 con 0x7f04fc103110 2024-03-27T10:49:39.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:49:39.263+0000 7f0502200700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f04fc10ce90 con 0x7f04fc101570 2024-03-27T10:49:39.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:49:39.263+0000 7f0502200700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f04fc10d190 con 0x7f04fc102770 2024-03-27T10:49:39.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:49:39.263+0000 7f04fbe00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04fc103110 0x7f04fc19fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:49:39.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:49:39.263+0000 7f04fbe00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04fc103110 0x7f04fc19fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34342/0 (socket says 172.21.15.97:34342) 2024-03-27T10:49:39.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:49:39.263+0000 7f04fbe00700 1 -- 172.21.15.97:0/523176158 learned_addr learned my addr 172.21.15.97:0/523176158 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:49:39.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:49:39.263+0000 7f04fb400700 1 --2- 172.21.15.97:0/523176158 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f04fc101570 0x7f04fc06ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:49:39.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:49:39.263+0000 7f04faa00700 1 --2- 172.21.15.97:0/523176158 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f04fc102770 0x7f04fc19d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:49:39.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:49:39.263+0000 7f04fbe00700 1 -- 172.21.15.97:0/523176158 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f04fc102770 msgr2=0x7f04fc19d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:49:39.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:49:39.263+0000 7f04fbe00700 1 --2- 172.21.15.97:0/523176158 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f04fc102770 0x7f04fc19d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:49:39.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:49:39.263+0000 7f04fbe00700 1 -- 172.21.15.97:0/523176158 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f04fc101570 msgr2=0x7f04fc06ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:49:39.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:49:39.263+0000 7f04fbe00700 1 --2- 172.21.15.97:0/523176158 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f04fc101570 0x7f04fc06ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:49:39.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:49:39.263+0000 7f04fbe00700 1 -- 172.21.15.97:0/523176158 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04fc1a0470 con 0x7f04fc103110 2024-03-27T10:49:39.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:49:39.263+0000 7f04fb400700 1 --2- 172.21.15.97:0/523176158 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f04fc101570 0x7f04fc06ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T10:49:39.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:49:39.264+0000 7f04fbe00700 1 --2- 172.21.15.97:0/523176158 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04fc103110 0x7f04fc19fba0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f04f4007ee0 tx=0x7f04f400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:49:39.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:49:39.264+0000 7f04f3e00700 1 -- 172.21.15.97:0/523176158 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f400dbd0 con 0x7f04fc103110 2024-03-27T10:49:39.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:49:39.264+0000 7f04f3e00700 1 -- 172.21.15.97:0/523176158 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f04f400dd70 con 0x7f04fc103110 2024-03-27T10:49:39.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:49:39.264+0000 7f0502200700 1 -- 172.21.15.97:0/523176158 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04fc1a0760 con 0x7f04fc103110 2024-03-27T10:49:39.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:49:39.264+0000 7f04f3e00700 1 -- 172.21.15.97:0/523176158 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f4005a20 con 0x7f04fc103110 2024-03-27T10:49:39.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:49:39.264+0000 7f0502200700 1 -- 172.21.15.97:0/523176158 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04fc1a0cf0 con 0x7f04fc103110 2024-03-27T10:49:39.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:49:39.265+0000 7f0502200700 1 -- 172.21.15.97:0/523176158 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04fc069300 con 0x7f04fc103110 2024-03-27T10:49:39.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:49:39.267+0000 7f04f3e00700 1 -- 172.21.15.97:0/523176158 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138370+0+0 (secure 0 0 0) 0x7f04f401a020 con 0x7f04fc103110 2024-03-27T10:49:39.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:49:39.269+0000 7f04f3e00700 1 --2- 172.21.15.97:0/523176158 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f04e00a3720 0x7f04e00a5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:49:39.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:49:39.269+0000 7f04f3e00700 1 -- 172.21.15.97:0/523176158 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 4533+0+0 (secure 0 0 0) 0x7f04f40cec50 con 0x7f04fc103110 2024-03-27T10:49:39.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:49:39.269+0000 7f04fb400700 1 --2- 172.21.15.97:0/523176158 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f04e00a3720 0x7f04e00a5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:49:39.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:49:39.270+0000 7f04fb400700 1 --2- 172.21.15.97:0/523176158 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f04e00a3720 0x7f04e00a5be0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f04ec000c00 tx=0x7f04ec005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:49:39.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:49:39.272+0000 7f04f3e00700 1 -- 172.21.15.97:0/523176158 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f04f4010070 con 0x7f04fc103110 2024-03-27T10:49:39.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:39 smithi097 ceph-mon[31859]: pgmap v183: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-27T10:49:39.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:39 smithi097 ceph-mon[31859]: osdmap e45: 6 total, 6 up, 6 in 2024-03-27T10:49:39.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:39 smithi177 ceph-mon[35294]: pgmap v183: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-27T10:49:39.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:39 smithi177 ceph-mon[35294]: osdmap e45: 6 total, 6 up, 6 in 2024-03-27T10:49:39.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:49:39.506+0000 7f0502200700 1 -- 172.21.15.97:0/523176158 --> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f04fc062a80 con 0x7f04e00a3720 2024-03-27T10:49:39.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:39 smithi114 ceph-mon[35338]: pgmap v183: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-27T10:49:39.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:39 smithi114 ceph-mon[35338]: osdmap e45: 6 total, 6 up, 6 in 2024-03-27T10:49:40.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:40 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-27T10:49:40.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:40 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-27T10:49:40.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:40 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:49:40.392 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:49:39 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: 2024-03-27T10:49:39.932+0000 7fb2d6a00700 -1 calc_pg_upmaps abort due to max <= 0 2024-03-27T10:49:40.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:40 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-27T10:49:40.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:40 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-27T10:49:40.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:40 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:49:40.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:40 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-27T10:49:40.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:40 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-27T10:49:40.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:40 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:49:41.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:41 smithi097 ceph-mon[31859]: from='client.14361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T10:49:41.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:41 smithi097 ceph-mon[31859]: pgmap v185: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-03-27T10:49:41.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:41 smithi177 ceph-mon[35294]: from='client.14361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T10:49:41.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:41 smithi177 ceph-mon[35294]: pgmap v185: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-03-27T10:49:41.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:41 smithi114 ceph-mon[35338]: from='client.14361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T10:49:41.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:41 smithi114 ceph-mon[35338]: pgmap v185: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-03-27T10:49:43.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:43 smithi097 ceph-mon[31859]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-27T10:49:43.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:43 smithi177 ceph-mon[35294]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-27T10:49:43.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:43 smithi114 ceph-mon[35338]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-27T10:49:44.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:44 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/2388522476' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "06dffb32-05eb-43b3-852e-915c5cc7f25a"}]: dispatch 2024-03-27T10:49:44.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:44 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/2388522476' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "06dffb32-05eb-43b3-852e-915c5cc7f25a"}]': finished 2024-03-27T10:49:44.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:44 smithi097 ceph-mon[31859]: osdmap e46: 7 total, 6 up, 7 in 2024-03-27T10:49:44.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:44 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-27T10:49:44.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:44 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/2388522476' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "06dffb32-05eb-43b3-852e-915c5cc7f25a"}]: dispatch 2024-03-27T10:49:44.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:44 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/2388522476' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "06dffb32-05eb-43b3-852e-915c5cc7f25a"}]': finished 2024-03-27T10:49:44.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:44 smithi177 ceph-mon[35294]: osdmap e46: 7 total, 6 up, 7 in 2024-03-27T10:49:44.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:44 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-27T10:49:44.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:44 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/2388522476' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "06dffb32-05eb-43b3-852e-915c5cc7f25a"}]: dispatch 2024-03-27T10:49:44.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:44 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/2388522476' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "06dffb32-05eb-43b3-852e-915c5cc7f25a"}]': finished 2024-03-27T10:49:44.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:44 smithi114 ceph-mon[35338]: osdmap e46: 7 total, 6 up, 7 in 2024-03-27T10:49:44.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:44 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-27T10:49:45.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:45 smithi114 ceph-mon[35338]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-03-27T10:49:45.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:45 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/1215121362' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-27T10:49:45.642 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:45 smithi097 ceph-mon[31859]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-03-27T10:49:45.642 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:45 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/1215121362' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-27T10:49:45.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:45 smithi177 ceph-mon[35294]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-03-27T10:49:45.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:45 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/1215121362' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-27T10:49:47.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:47 smithi114 ceph-mon[35338]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-03-27T10:49:47.642 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:47 smithi097 ceph-mon[31859]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-03-27T10:49:47.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:47 smithi177 ceph-mon[35294]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-03-27T10:49:49.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:49 smithi114 ceph-mon[35338]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-03-27T10:49:49.642 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:49 smithi097 ceph-mon[31859]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-03-27T10:49:49.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:49 smithi177 ceph-mon[35294]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-03-27T10:49:50.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:50 smithi114 ceph-mon[35338]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-03-27T10:49:50.642 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:50 smithi097 ceph-mon[31859]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-03-27T10:49:50.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:50 smithi177 ceph-mon[35294]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-03-27T10:49:53.268 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:53 smithi114 ceph-mon[35338]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-03-27T10:49:53.382 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:53 smithi177 ceph-mon[35294]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-03-27T10:49:53.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:53 smithi097 ceph-mon[31859]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-03-27T10:49:54.268 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:54 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-27T10:49:54.268 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:54 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:49:54.268 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:54 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-27T10:49:54.269 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:54 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:49:54.417 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:54 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-27T10:49:54.417 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:54 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:49:55.268 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:55 smithi114 ceph-mon[35338]: Deploying daemon osd.6 on smithi097 2024-03-27T10:49:55.268 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:55 smithi114 ceph-mon[35338]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-03-27T10:49:55.286 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:55 smithi097 ceph-mon[31859]: Deploying daemon osd.6 on smithi097 2024-03-27T10:49:55.287 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:55 smithi097 ceph-mon[31859]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-03-27T10:49:55.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:55 smithi177 ceph-mon[35294]: Deploying daemon osd.6 on smithi097 2024-03-27T10:49:55.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:55 smithi177 ceph-mon[35294]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-03-27T10:49:57.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:57 smithi097 ceph-mon[31859]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-03-27T10:49:57.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:57 smithi177 ceph-mon[35294]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-03-27T10:49:57.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:57 smithi114 ceph-mon[35338]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-03-27T10:49:59.064 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:49:59 smithi097 ceph-mon[31859]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-03-27T10:49:59.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:49:59 smithi177 ceph-mon[35294]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-03-27T10:49:59.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:49:59 smithi114 ceph-mon[35338]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-03-27T10:50:00.359 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:00 smithi097 ceph-mon[31859]: overall HEALTH_OK 2024-03-27T10:50:00.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:00 smithi177 ceph-mon[35294]: overall HEALTH_OK 2024-03-27T10:50:00.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:00 smithi114 ceph-mon[35338]: overall HEALTH_OK 2024-03-27T10:50:01.137 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:01 smithi097 ceph-mon[31859]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-03-27T10:50:01.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:01 smithi177 ceph-mon[35294]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-03-27T10:50:01.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:01 smithi114 ceph-mon[35338]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-03-27T10:50:02.815 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:02 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:50:02.816 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:02 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:50:02.816 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:02 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:50:02.816 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:02 smithi097 ceph-mon[31859]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-03-27T10:50:02.928 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:02 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:50:02.928 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:02 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:50:02.928 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:02 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:50:02.928 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:02 smithi177 ceph-mon[35294]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-03-27T10:50:03.018 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:02 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:50:03.018 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:02 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:50:03.018 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:02 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:50:03.018 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:02 smithi114 ceph-mon[35338]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-03-27T10:50:05.268 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:05 smithi114 ceph-mon[35338]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-03-27T10:50:05.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:05 smithi097 ceph-mon[31859]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-03-27T10:50:05.411 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:05 smithi177 ceph-mon[35294]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-03-27T10:50:06.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:06 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:50:06.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:06 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:50:06.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:06 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:50:06.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:06 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:50:06.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:06 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:50:06.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:06 smithi097 ceph-mon[31859]: from='osd.6 [v2:172.21.15.97:6818/2892389563,v1:172.21.15.97:6819/2892389563]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-27T10:50:06.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:06 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:50:06.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:06 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:50:06.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:06 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:50:06.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:06 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:50:06.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:06 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:50:06.429 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:06 smithi177 ceph-mon[35294]: from='osd.6 [v2:172.21.15.97:6818/2892389563,v1:172.21.15.97:6819/2892389563]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-27T10:50:06.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:06 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:50:06.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:06 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:50:06.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:06 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:50:06.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:06 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:50:06.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:06 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:50:06.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:06 smithi114 ceph-mon[35338]: from='osd.6 [v2:172.21.15.97:6818/2892389563,v1:172.21.15.97:6819/2892389563]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-27T10:50:06.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:50:06.547+0000 7f04f3e00700 1 -- 172.21.15.97:0/523176158 <== mgr.14152 v2:172.21.15.97:6800/3955463538 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f04fc062a80 con 0x7f04e00a3720 2024-03-27T10:50:06.521 INFO:teuthology.orchestra.run.smithi097.stdout:Created osd(s) 6 on host 'smithi097' 2024-03-27T10:50:06.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:50:06.550+0000 7f0502200700 1 -- 172.21.15.97:0/523176158 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f04e00a3720 msgr2=0x7f04e00a5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:50:06.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:50:06.550+0000 7f0502200700 1 --2- 172.21.15.97:0/523176158 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f04e00a3720 0x7f04e00a5be0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f04ec000c00 tx=0x7f04ec005fd0 comp rx=0 tx=0).stop 2024-03-27T10:50:06.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:50:06.550+0000 7f0502200700 1 -- 172.21.15.97:0/523176158 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04fc103110 msgr2=0x7f04fc19fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:50:06.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:50:06.550+0000 7f0502200700 1 --2- 172.21.15.97:0/523176158 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04fc103110 0x7f04fc19fba0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f04f4007ee0 tx=0x7f04f400d3b0 comp rx=0 tx=0).stop 2024-03-27T10:50:06.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:50:06.551+0000 7f0502200700 1 -- 172.21.15.97:0/523176158 shutdown_connections 2024-03-27T10:50:06.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:50:06.551+0000 7f0502200700 1 --2- 172.21.15.97:0/523176158 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f04e00a3720 0x7f04e00a5be0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:50:06.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:50:06.551+0000 7f0502200700 1 --2- 172.21.15.97:0/523176158 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04fc103110 0x7f04fc19fba0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:50:06.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:50:06.551+0000 7f0502200700 1 --2- 172.21.15.97:0/523176158 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f04fc101570 0x7f04fc06ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:50:06.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:50:06.551+0000 7f0502200700 1 --2- 172.21.15.97:0/523176158 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f04fc102770 0x7f04fc19d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:50:06.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:50:06.551+0000 7f0502200700 1 -- 172.21.15.97:0/523176158 >> 172.21.15.97:0/523176158 conn(0x7f04fc0fcb10 msgr2=0x7f04fc105b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:50:06.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:50:06.551+0000 7f0502200700 1 -- 172.21.15.97:0/523176158 shutdown_connections 2024-03-27T10:50:06.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:50:06.551+0000 7f0502200700 1 -- 172.21.15.97:0/523176158 wait complete. 2024-03-27T10:50:07.314 DEBUG:teuthology.orchestra.run.smithi097:osd.6> sudo journalctl -f -n 0 -u ceph-adca88c4-ec26-11ee-95d0-87774f69a715@osd.6.service 2024-03-27T10:50:07.319 INFO:tasks.cephadm:Deploying osd.7 on smithi114 with /dev/vg_nvme/lv_2... 2024-03-27T10:50:07.319 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid adca88c4-ec26-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-27T10:50:07.516 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:07 smithi097 ceph-mon[31859]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-03-27T10:50:07.516 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:07 smithi097 ceph-mon[31859]: from='osd.6 [v2:172.21.15.97:6818/2892389563,v1:172.21.15.97:6819/2892389563]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-27T10:50:07.516 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:07 smithi097 ceph-mon[31859]: osdmap e47: 7 total, 6 up, 7 in 2024-03-27T10:50:07.516 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:07 smithi097 ceph-mon[31859]: from='osd.6 [v2:172.21.15.97:6818/2892389563,v1:172.21.15.97:6819/2892389563]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-03-27T10:50:07.516 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:07 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-27T10:50:07.516 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:07 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:50:07.516 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:07 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:50:07.516 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:07 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:50:07.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:07 smithi114 ceph-mon[35338]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-03-27T10:50:07.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:07 smithi114 ceph-mon[35338]: from='osd.6 [v2:172.21.15.97:6818/2892389563,v1:172.21.15.97:6819/2892389563]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-27T10:50:07.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:07 smithi114 ceph-mon[35338]: osdmap e47: 7 total, 6 up, 7 in 2024-03-27T10:50:07.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:07 smithi114 ceph-mon[35338]: from='osd.6 [v2:172.21.15.97:6818/2892389563,v1:172.21.15.97:6819/2892389563]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-03-27T10:50:07.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:07 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-27T10:50:07.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:07 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:50:07.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:07 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:50:07.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:07 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:50:07.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:07 smithi177 ceph-mon[35294]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-03-27T10:50:07.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:07 smithi177 ceph-mon[35294]: from='osd.6 [v2:172.21.15.97:6818/2892389563,v1:172.21.15.97:6819/2892389563]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-27T10:50:07.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:07 smithi177 ceph-mon[35294]: osdmap e47: 7 total, 6 up, 7 in 2024-03-27T10:50:07.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:07 smithi177 ceph-mon[35294]: from='osd.6 [v2:172.21.15.97:6818/2892389563,v1:172.21.15.97:6819/2892389563]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-03-27T10:50:07.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:07 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-27T10:50:07.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:07 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:50:07.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:07 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:50:07.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:07 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:50:07.755 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/mon.b/config 2024-03-27T10:50:08.504 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:08 smithi097 ceph-mon[31859]: from='osd.6 [v2:172.21.15.97:6818/2892389563,v1:172.21.15.97:6819/2892389563]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-03-27T10:50:08.504 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:08 smithi097 ceph-mon[31859]: osdmap e48: 7 total, 6 up, 7 in 2024-03-27T10:50:08.504 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:08 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-27T10:50:08.504 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:08 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-27T10:50:08.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:08 smithi114 ceph-mon[35338]: from='osd.6 [v2:172.21.15.97:6818/2892389563,v1:172.21.15.97:6819/2892389563]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-03-27T10:50:08.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:08 smithi114 ceph-mon[35338]: osdmap e48: 7 total, 6 up, 7 in 2024-03-27T10:50:08.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:08 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-27T10:50:08.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:08 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-27T10:50:08.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:08 smithi177 ceph-mon[35294]: from='osd.6 [v2:172.21.15.97:6818/2892389563,v1:172.21.15.97:6819/2892389563]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-03-27T10:50:08.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:08 smithi177 ceph-mon[35294]: osdmap e48: 7 total, 6 up, 7 in 2024-03-27T10:50:08.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:08 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-27T10:50:08.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:08 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-27T10:50:09.354 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:09 smithi114 ceph-mon[35338]: purged_snaps scrub starts 2024-03-27T10:50:09.393 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:09 smithi114 ceph-mon[35338]: purged_snaps scrub ok 2024-03-27T10:50:09.393 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:09 smithi114 ceph-mon[35338]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-03-27T10:50:09.393 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:09 smithi114 ceph-mon[35338]: osd.6 [v2:172.21.15.97:6818/2892389563,v1:172.21.15.97:6819/2892389563] boot 2024-03-27T10:50:09.393 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:09 smithi114 ceph-mon[35338]: osdmap e49: 7 total, 7 up, 7 in 2024-03-27T10:50:09.393 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:09 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-27T10:50:09.642 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:09 smithi097 ceph-mon[31859]: purged_snaps scrub starts 2024-03-27T10:50:09.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:09 smithi097 ceph-mon[31859]: purged_snaps scrub ok 2024-03-27T10:50:09.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:09 smithi097 ceph-mon[31859]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-03-27T10:50:09.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:09 smithi097 ceph-mon[31859]: osd.6 [v2:172.21.15.97:6818/2892389563,v1:172.21.15.97:6819/2892389563] boot 2024-03-27T10:50:09.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:09 smithi097 ceph-mon[31859]: osdmap e49: 7 total, 7 up, 7 in 2024-03-27T10:50:09.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:09 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-27T10:50:09.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:09 smithi177 ceph-mon[35294]: purged_snaps scrub starts 2024-03-27T10:50:09.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:09 smithi177 ceph-mon[35294]: purged_snaps scrub ok 2024-03-27T10:50:09.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:09 smithi177 ceph-mon[35294]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-03-27T10:50:09.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:09 smithi177 ceph-mon[35294]: osd.6 [v2:172.21.15.97:6818/2892389563,v1:172.21.15.97:6819/2892389563] boot 2024-03-27T10:50:09.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:09 smithi177 ceph-mon[35294]: osdmap e49: 7 total, 7 up, 7 in 2024-03-27T10:50:09.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:09 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-27T10:50:10.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:10 smithi114 ceph-mon[35338]: osdmap e50: 7 total, 7 up, 7 in 2024-03-27T10:50:10.525 INFO:teuthology.orchestra.run.smithi114.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-27T10:50:10.525 INFO:teuthology.orchestra.run.smithi114.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-27T10:50:10.525 INFO:teuthology.orchestra.run.smithi114.stderr: stderr: 10+0 records in 2024-03-27T10:50:10.526 INFO:teuthology.orchestra.run.smithi114.stderr:10+0 records out 2024-03-27T10:50:10.526 INFO:teuthology.orchestra.run.smithi114.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0363506 s, 288 MB/s 2024-03-27T10:50:10.526 INFO:teuthology.orchestra.run.smithi114.stderr:--> Zapping successful for: 2024-03-27T10:50:10.642 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:10 smithi097 ceph-mon[31859]: osdmap e50: 7 total, 7 up, 7 in 2024-03-27T10:50:10.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:10 smithi177 ceph-mon[35294]: osdmap e50: 7 total, 7 up, 7 in 2024-03-27T10:50:11.364 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid adca88c4-ec26-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi114:vg_nvme/lv_2 2024-03-27T10:50:11.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:11 smithi097 ceph-mon[31859]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-03-27T10:50:11.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:11 smithi114 ceph-mon[35338]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-03-27T10:50:11.682 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:11 smithi177 ceph-mon[35294]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-03-27T10:50:11.779 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/mon.b/config 2024-03-27T10:50:13.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:12 smithi114 ceph-mon[35338]: Detected new or changed devices on smithi097 2024-03-27T10:50:13.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:12 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:50:13.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:12 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:50:13.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:12 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-27T10:50:13.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:12 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-27T10:50:13.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:12 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-27T10:50:13.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:12 smithi114 ceph-mon[35338]: Adjusting osd_memory_target on smithi097 to 5697M 2024-03-27T10:50:13.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:12 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:50:13.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:12 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:50:13.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:12 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:50:13.020 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:12 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:50:13.020 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:12 smithi114 ceph-mon[35338]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-03-27T10:50:13.142 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:12 smithi097 ceph-mon[31859]: Detected new or changed devices on smithi097 2024-03-27T10:50:13.143 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:12 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:50:13.143 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:12 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:50:13.143 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:12 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-27T10:50:13.143 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:12 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-27T10:50:13.143 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:12 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-27T10:50:13.143 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:12 smithi097 ceph-mon[31859]: Adjusting osd_memory_target on smithi097 to 5697M 2024-03-27T10:50:13.143 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:12 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:50:13.143 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:12 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:50:13.143 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:12 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:50:13.143 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:12 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:50:13.143 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:12 smithi097 ceph-mon[31859]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-03-27T10:50:13.178 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:12 smithi177 ceph-mon[35294]: Detected new or changed devices on smithi097 2024-03-27T10:50:13.179 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:12 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:50:13.179 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:12 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:50:13.179 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:12 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-27T10:50:13.179 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:12 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-27T10:50:13.179 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:12 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-27T10:50:13.179 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:12 smithi177 ceph-mon[35294]: Adjusting osd_memory_target on smithi097 to 5697M 2024-03-27T10:50:13.179 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:12 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:50:13.179 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:12 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:50:13.179 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:12 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:50:13.179 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:12 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:50:13.179 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:12 smithi177 ceph-mon[35294]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-03-27T10:50:14.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:50:14.144+0000 7f7b76000700 1 -- 172.21.15.114:0/885948726 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7b70106460 msgr2=0x7f7b7010cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:50:14.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:50:14.144+0000 7f7b76000700 1 --2- 172.21.15.114:0/885948726 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7b70106460 0x7f7b7010cd10 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7b6800b0d0 tx=0x7f7b6800b3e0 comp rx=0 tx=0).stop 2024-03-27T10:50:14.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:50:14.145+0000 7f7b76000700 1 -- 172.21.15.114:0/885948726 shutdown_connections 2024-03-27T10:50:14.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:50:14.145+0000 7f7b76000700 1 --2- 172.21.15.114:0/885948726 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7b70106460 0x7f7b7010cd10 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:50:14.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:50:14.145+0000 7f7b76000700 1 --2- 172.21.15.114:0/885948726 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7b701048c0 0x7f7b70104ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:50:14.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:50:14.145+0000 7f7b76000700 1 --2- 172.21.15.114:0/885948726 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b70105ac0 0x7f7b70105f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:50:14.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:50:14.145+0000 7f7b76000700 1 -- 172.21.15.114:0/885948726 >> 172.21.15.114:0/885948726 conn(0x7f7b700ffe60 msgr2=0x7f7b701022a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:50:14.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:50:14.145+0000 7f7b76000700 1 -- 172.21.15.114:0/885948726 shutdown_connections 2024-03-27T10:50:14.120 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:50:14.145+0000 7f7b76000700 1 -- 172.21.15.114:0/885948726 wait complete. 2024-03-27T10:50:14.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:50:14.146+0000 7f7b76000700 1 Processor -- start 2024-03-27T10:50:14.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:50:14.146+0000 7f7b76000700 1 -- start start 2024-03-27T10:50:14.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:50:14.147+0000 7f7b76000700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7b701048c0 0x7f7b7019a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:50:14.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:50:14.147+0000 7f7b76000700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7b70105ac0 0x7f7b7019ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:50:14.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:50:14.147+0000 7f7b76000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b70106460 0x7f7b701a1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:50:14.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:50:14.147+0000 7f7b76000700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7b70110360 con 0x7f7b70106460 2024-03-27T10:50:14.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:50:14.147+0000 7f7b76000700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f7b701101e0 con 0x7f7b70105ac0 2024-03-27T10:50:14.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:50:14.147+0000 7f7b76000700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f7b701104e0 con 0x7f7b701048c0 2024-03-27T10:50:14.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:50:14.147+0000 7f7b6ea00700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7b70105ac0 0x7f7b7019ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:50:14.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:50:14.147+0000 7f7b6ea00700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7b70105ac0 0x7f7b7019ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:45952/0 (socket says 172.21.15.114:45952) 2024-03-27T10:50:14.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:50:14.147+0000 7f7b6ea00700 1 -- 172.21.15.114:0/3821409799 learned_addr learned my addr 172.21.15.114:0/3821409799 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-03-27T10:50:14.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:50:14.147+0000 7f7b6f400700 1 --2- 172.21.15.114:0/3821409799 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7b701048c0 0x7f7b7019a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:50:14.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:50:14.147+0000 7f7b6fe00700 1 --2- 172.21.15.114:0/3821409799 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b70106460 0x7f7b701a1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:50:14.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:50:14.147+0000 7f7b6ea00700 1 -- 172.21.15.114:0/3821409799 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7b701048c0 msgr2=0x7f7b7019a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:50:14.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:50:14.147+0000 7f7b6ea00700 1 --2- 172.21.15.114:0/3821409799 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7b701048c0 0x7f7b7019a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:50:14.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:50:14.147+0000 7f7b6ea00700 1 -- 172.21.15.114:0/3821409799 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b70106460 msgr2=0x7f7b701a1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:50:14.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:50:14.147+0000 7f7b6ea00700 1 --2- 172.21.15.114:0/3821409799 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b70106460 0x7f7b701a1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:50:14.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:50:14.147+0000 7f7b6ea00700 1 -- 172.21.15.114:0/3821409799 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b701a25a0 con 0x7f7b70105ac0 2024-03-27T10:50:14.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:50:14.148+0000 7f7b6ea00700 1 --2- 172.21.15.114:0/3821409799 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7b70105ac0 0x7f7b7019ae80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c00d950 tx=0x7f7b5c00dd10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:50:14.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:50:14.148+0000 7f7b67e00700 1 -- 172.21.15.114:0/3821409799 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b5c016070 con 0x7f7b70105ac0 2024-03-27T10:50:14.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:50:14.148+0000 7f7b76000700 1 -- 172.21.15.114:0/3821409799 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b701a2890 con 0x7f7b70105ac0 2024-03-27T10:50:14.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:50:14.148+0000 7f7b67e00700 1 -- 172.21.15.114:0/3821409799 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7b5c010460 con 0x7f7b70105ac0 2024-03-27T10:50:14.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:50:14.148+0000 7f7b67e00700 1 -- 172.21.15.114:0/3821409799 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b5c020060 con 0x7f7b70105ac0 2024-03-27T10:50:14.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:50:14.148+0000 7f7b76000700 1 -- 172.21.15.114:0/3821409799 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b701a2e20 con 0x7f7b70105ac0 2024-03-27T10:50:14.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:50:14.149+0000 7f7b76000700 1 -- 172.21.15.114:0/3821409799 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b70069300 con 0x7f7b70105ac0 2024-03-27T10:50:14.125 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:50:14.152+0000 7f7b67e00700 1 -- 172.21.15.114:0/3821409799 <== mon.2 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 16) ==== 138370+0+0 (secure 0 0 0) 0x7f7b5c019070 con 0x7f7b70105ac0 2024-03-27T10:50:14.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:50:14.153+0000 7f7b67e00700 1 --2- 172.21.15.114:0/3821409799 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f7b540a36a0 0x7f7b540a5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:50:14.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:50:14.153+0000 7f7b67e00700 1 -- 172.21.15.114:0/3821409799 <== mon.2 v2:172.21.15.114:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4965+0+0 (secure 0 0 0) 0x7f7b5c0cf150 con 0x7f7b70105ac0 2024-03-27T10:50:14.127 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:50:14.154+0000 7f7b6f400700 1 --2- 172.21.15.114:0/3821409799 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f7b540a36a0 0x7f7b540a5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:50:14.127 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:50:14.154+0000 7f7b6f400700 1 --2- 172.21.15.114:0/3821409799 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f7b540a36a0 0x7f7b540a5b60 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7b60005950 tx=0x7f7b600058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:50:14.129 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:50:14.156+0000 7f7b67e00700 1 -- 172.21.15.114:0/3821409799 <== mon.2 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f7b5c09c990 con 0x7f7b70105ac0 2024-03-27T10:50:14.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:50:14.393+0000 7f7b76000700 1 -- 172.21.15.114:0/3821409799 --> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi114:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f7b70062a80 con 0x7f7b540a36a0 2024-03-27T10:50:15.268 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:15 smithi114 ceph-mon[35338]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-03-27T10:50:15.268 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:15 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-27T10:50:15.268 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:15 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-27T10:50:15.268 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:15 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:50:15.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:15 smithi097 ceph-mon[31859]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-03-27T10:50:15.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:15 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-27T10:50:15.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:15 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-27T10:50:15.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:15 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:50:15.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:15 smithi177 ceph-mon[35294]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-03-27T10:50:15.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:15 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-27T10:50:15.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:15 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-27T10:50:15.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:15 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:50:16.371 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:16 smithi114 ceph-mon[35338]: from='client.34262 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi114:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T10:50:16.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:16 smithi097 ceph-mon[31859]: from='client.34262 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi114:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T10:50:16.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:16 smithi177 ceph-mon[35294]: from='client.34262 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi114:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T10:50:17.181 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:17 smithi114 ceph-mon[35338]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-03-27T10:50:17.390 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:17 smithi097 ceph-mon[31859]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-03-27T10:50:17.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:17 smithi177 ceph-mon[35294]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-03-27T10:50:19.268 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:19 smithi114 ceph-mon[35338]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-03-27T10:50:19.268 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:19 smithi114 ceph-mon[35338]: from='client.? 172.21.15.114:0/1146940508' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e68e3519-9c1b-43c5-8e06-5385248b19df"}]: dispatch 2024-03-27T10:50:19.268 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:19 smithi114 ceph-mon[35338]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e68e3519-9c1b-43c5-8e06-5385248b19df"}]: dispatch 2024-03-27T10:50:19.268 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:19 smithi114 ceph-mon[35338]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e68e3519-9c1b-43c5-8e06-5385248b19df"}]': finished 2024-03-27T10:50:19.268 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:19 smithi114 ceph-mon[35338]: osdmap e51: 8 total, 7 up, 8 in 2024-03-27T10:50:19.268 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:19 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-27T10:50:19.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:19 smithi097 ceph-mon[31859]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-03-27T10:50:19.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:19 smithi097 ceph-mon[31859]: from='client.? 172.21.15.114:0/1146940508' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e68e3519-9c1b-43c5-8e06-5385248b19df"}]: dispatch 2024-03-27T10:50:19.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:19 smithi097 ceph-mon[31859]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e68e3519-9c1b-43c5-8e06-5385248b19df"}]: dispatch 2024-03-27T10:50:19.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:19 smithi097 ceph-mon[31859]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e68e3519-9c1b-43c5-8e06-5385248b19df"}]': finished 2024-03-27T10:50:19.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:19 smithi097 ceph-mon[31859]: osdmap e51: 8 total, 7 up, 8 in 2024-03-27T10:50:19.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:19 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-27T10:50:19.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:19 smithi177 ceph-mon[35294]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-03-27T10:50:19.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:19 smithi177 ceph-mon[35294]: from='client.? 172.21.15.114:0/1146940508' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e68e3519-9c1b-43c5-8e06-5385248b19df"}]: dispatch 2024-03-27T10:50:19.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:19 smithi177 ceph-mon[35294]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e68e3519-9c1b-43c5-8e06-5385248b19df"}]: dispatch 2024-03-27T10:50:19.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:19 smithi177 ceph-mon[35294]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e68e3519-9c1b-43c5-8e06-5385248b19df"}]': finished 2024-03-27T10:50:19.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:19 smithi177 ceph-mon[35294]: osdmap e51: 8 total, 7 up, 8 in 2024-03-27T10:50:19.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:19 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-27T10:50:20.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:20 smithi097 ceph-mon[31859]: from='client.? 172.21.15.114:0/3442106656' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-27T10:50:20.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:20 smithi177 ceph-mon[35294]: from='client.? 172.21.15.114:0/3442106656' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-27T10:50:20.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:20 smithi114 ceph-mon[35338]: from='client.? 172.21.15.114:0/3442106656' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-27T10:50:21.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:21 smithi097 ceph-mon[31859]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-03-27T10:50:21.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:21 smithi177 ceph-mon[35294]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-03-27T10:50:21.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:21 smithi114 ceph-mon[35338]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-03-27T10:50:23.304 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:23 smithi114 ceph-mon[35338]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-03-27T10:50:23.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:23 smithi097 ceph-mon[31859]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-03-27T10:50:23.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:23 smithi177 ceph-mon[35294]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-03-27T10:50:25.355 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:25 smithi114 ceph-mon[35338]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-03-27T10:50:25.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:25 smithi097 ceph-mon[31859]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-03-27T10:50:25.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:25 smithi177 ceph-mon[35294]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-03-27T10:50:27.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:27 smithi097 ceph-mon[31859]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-03-27T10:50:27.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:27 smithi177 ceph-mon[35294]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-03-27T10:50:27.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:27 smithi114 ceph-mon[35338]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-03-27T10:50:29.368 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:29 smithi114 ceph-mon[35338]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-03-27T10:50:29.368 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:29 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-27T10:50:29.368 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:29 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:50:29.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:29 smithi097 ceph-mon[31859]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-03-27T10:50:29.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:29 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-27T10:50:29.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:29 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:50:29.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:29 smithi177 ceph-mon[35294]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-03-27T10:50:29.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:29 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-27T10:50:29.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:29 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:50:30.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:30 smithi097 ceph-mon[31859]: Deploying daemon osd.7 on smithi114 2024-03-27T10:50:30.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:30 smithi177 ceph-mon[35294]: Deploying daemon osd.7 on smithi114 2024-03-27T10:50:30.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:30 smithi114 ceph-mon[35338]: Deploying daemon osd.7 on smithi114 2024-03-27T10:50:31.216 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:31 smithi114 ceph-mon[35338]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-03-27T10:50:31.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:31 smithi097 ceph-mon[31859]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-03-27T10:50:31.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:31 smithi177 ceph-mon[35294]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-03-27T10:50:33.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:33 smithi097 ceph-mon[31859]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-03-27T10:50:33.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:33 smithi177 ceph-mon[35294]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-03-27T10:50:33.513 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:33 smithi114 ceph-mon[35338]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-03-27T10:50:35.268 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:35 smithi114 ceph-mon[35338]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-03-27T10:50:35.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:35 smithi097 ceph-mon[31859]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-03-27T10:50:35.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:35 smithi177 ceph-mon[35294]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-03-27T10:50:37.268 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:37 smithi114 ceph-mon[35338]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-03-27T10:50:37.268 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:37 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:50:37.268 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:37 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:50:37.269 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:37 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:50:37.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:37 smithi097 ceph-mon[31859]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-03-27T10:50:37.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:37 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:50:37.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:37 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:50:37.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:37 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:50:37.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:37 smithi177 ceph-mon[35294]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-03-27T10:50:37.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:37 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:50:37.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:37 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:50:37.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:37 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:50:39.268 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:39 smithi114 ceph-mon[35338]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-03-27T10:50:39.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:39 smithi097 ceph-mon[31859]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-03-27T10:50:39.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:39 smithi177 ceph-mon[35294]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-03-27T10:50:41.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:41 smithi097 ceph-mon[31859]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-03-27T10:50:41.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:41 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:50:41.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:41 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:50:41.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:41 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:50:41.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:41 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:50:41.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:41 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:50:41.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:41 smithi097 ceph-mon[31859]: from='osd.7 [v2:172.21.15.114:6816/2445491153,v1:172.21.15.114:6817/2445491153]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-27T10:50:41.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:41 smithi097 ceph-mon[31859]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-27T10:50:41.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:41 smithi177 ceph-mon[35294]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-03-27T10:50:41.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:41 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:50:41.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:41 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:50:41.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:41 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:50:41.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:41 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:50:41.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:41 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:50:41.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:41 smithi177 ceph-mon[35294]: from='osd.7 [v2:172.21.15.114:6816/2445491153,v1:172.21.15.114:6817/2445491153]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-27T10:50:41.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:41 smithi177 ceph-mon[35294]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-27T10:50:41.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:41 smithi114 ceph-mon[35338]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-03-27T10:50:41.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:41 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:50:41.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:41 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:50:41.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:41 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:50:41.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:41 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:50:41.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:41 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:50:41.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:41 smithi114 ceph-mon[35338]: from='osd.7 [v2:172.21.15.114:6816/2445491153,v1:172.21.15.114:6817/2445491153]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-27T10:50:41.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:41 smithi114 ceph-mon[35338]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-27T10:50:41.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:50:41.662+0000 7f7b67e00700 1 -- 172.21.15.114:0/3821409799 <== mgr.14152 v2:172.21.15.97:6800/3955463538 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f7b70062a80 con 0x7f7b540a36a0 2024-03-27T10:50:41.636 INFO:teuthology.orchestra.run.smithi114.stdout:Created osd(s) 7 on host 'smithi114' 2024-03-27T10:50:41.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:50:41.666+0000 7f7b76000700 1 -- 172.21.15.114:0/3821409799 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f7b540a36a0 msgr2=0x7f7b540a5b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:50:41.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:50:41.666+0000 7f7b76000700 1 --2- 172.21.15.114:0/3821409799 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f7b540a36a0 0x7f7b540a5b60 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7b60005950 tx=0x7f7b600058e0 comp rx=0 tx=0).stop 2024-03-27T10:50:41.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:50:41.666+0000 7f7b76000700 1 -- 172.21.15.114:0/3821409799 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7b70105ac0 msgr2=0x7f7b7019ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:50:41.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:50:41.666+0000 7f7b76000700 1 --2- 172.21.15.114:0/3821409799 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7b70105ac0 0x7f7b7019ae80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c00d950 tx=0x7f7b5c00dd10 comp rx=0 tx=0).stop 2024-03-27T10:50:41.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:50:41.666+0000 7f7b76000700 1 -- 172.21.15.114:0/3821409799 shutdown_connections 2024-03-27T10:50:41.641 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:50:41.666+0000 7f7b76000700 1 --2- 172.21.15.114:0/3821409799 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f7b540a36a0 0x7f7b540a5b60 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:50:41.641 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:50:41.666+0000 7f7b76000700 1 --2- 172.21.15.114:0/3821409799 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b70106460 0x7f7b701a1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:50:41.641 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:50:41.666+0000 7f7b76000700 1 --2- 172.21.15.114:0/3821409799 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7b701048c0 0x7f7b7019a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:50:41.641 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:50:41.666+0000 7f7b76000700 1 --2- 172.21.15.114:0/3821409799 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7b70105ac0 0x7f7b7019ae80 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:50:41.641 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:50:41.666+0000 7f7b76000700 1 -- 172.21.15.114:0/3821409799 >> 172.21.15.114:0/3821409799 conn(0x7f7b700ffe60 msgr2=0x7f7b70108dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:50:41.641 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:50:41.666+0000 7f7b76000700 1 -- 172.21.15.114:0/3821409799 shutdown_connections 2024-03-27T10:50:41.641 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:50:41.666+0000 7f7b76000700 1 -- 172.21.15.114:0/3821409799 wait complete. 2024-03-27T10:50:42.489 DEBUG:teuthology.orchestra.run.smithi114:osd.7> sudo journalctl -f -n 0 -u ceph-adca88c4-ec26-11ee-95d0-87774f69a715@osd.7.service 2024-03-27T10:50:42.494 INFO:tasks.cephadm:Deploying osd.8 on smithi177 with /dev/vg_nvme/lv_2... 2024-03-27T10:50:42.495 DEBUG:teuthology.orchestra.run.smithi177:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid adca88c4-ec26-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-27T10:50:42.642 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:42 smithi097 ceph-mon[31859]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-27T10:50:42.642 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:42 smithi097 ceph-mon[31859]: from='osd.7 [v2:172.21.15.114:6816/2445491153,v1:172.21.15.114:6817/2445491153]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]: dispatch 2024-03-27T10:50:42.642 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:42 smithi097 ceph-mon[31859]: osdmap e52: 8 total, 7 up, 8 in 2024-03-27T10:50:42.642 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:42 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-27T10:50:42.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:42 smithi097 ceph-mon[31859]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]: dispatch 2024-03-27T10:50:42.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:42 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:50:42.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:42 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:50:42.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:42 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:50:42.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:42 smithi097 ceph-mon[31859]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-03-27T10:50:42.727 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:42 smithi114 ceph-mon[35338]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-27T10:50:42.727 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:42 smithi114 ceph-mon[35338]: from='osd.7 [v2:172.21.15.114:6816/2445491153,v1:172.21.15.114:6817/2445491153]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]: dispatch 2024-03-27T10:50:42.727 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:42 smithi114 ceph-mon[35338]: osdmap e52: 8 total, 7 up, 8 in 2024-03-27T10:50:42.727 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:42 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-27T10:50:42.727 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:42 smithi114 ceph-mon[35338]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]: dispatch 2024-03-27T10:50:42.728 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:42 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:50:42.728 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:42 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:50:42.728 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:42 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:50:42.728 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:42 smithi114 ceph-mon[35338]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-03-27T10:50:42.927 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:42 smithi177 ceph-mon[35294]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-27T10:50:42.927 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:42 smithi177 ceph-mon[35294]: from='osd.7 [v2:172.21.15.114:6816/2445491153,v1:172.21.15.114:6817/2445491153]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]: dispatch 2024-03-27T10:50:42.928 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:42 smithi177 ceph-mon[35294]: osdmap e52: 8 total, 7 up, 8 in 2024-03-27T10:50:42.928 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:42 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-27T10:50:42.928 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:42 smithi177 ceph-mon[35294]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]: dispatch 2024-03-27T10:50:42.928 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:42 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:50:42.928 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:42 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:50:42.928 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:42 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:50:42.928 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:42 smithi177 ceph-mon[35294]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-03-27T10:50:42.931 INFO:teuthology.orchestra.run.smithi177.stderr:Inferring config /var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/mon.c/config 2024-03-27T10:50:43.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:43 smithi177 ceph-mon[35294]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]': finished 2024-03-27T10:50:43.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:43 smithi177 ceph-mon[35294]: osdmap e53: 8 total, 7 up, 8 in 2024-03-27T10:50:43.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:43 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-27T10:50:43.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:43 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-27T10:50:43.768 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:43 smithi114 ceph-mon[35338]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]': finished 2024-03-27T10:50:43.768 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:43 smithi114 ceph-mon[35338]: osdmap e53: 8 total, 7 up, 8 in 2024-03-27T10:50:43.768 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:43 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-27T10:50:43.768 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:43 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-27T10:50:43.892 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:43 smithi097 ceph-mon[31859]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]': finished 2024-03-27T10:50:43.892 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:43 smithi097 ceph-mon[31859]: osdmap e53: 8 total, 7 up, 8 in 2024-03-27T10:50:43.892 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:43 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-27T10:50:43.892 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:43 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-27T10:50:44.769 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:44 smithi114 ceph-mon[35338]: purged_snaps scrub starts 2024-03-27T10:50:44.769 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:44 smithi114 ceph-mon[35338]: purged_snaps scrub ok 2024-03-27T10:50:44.769 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:44 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-27T10:50:44.769 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:44 smithi114 ceph-mon[35338]: osd.7 [v2:172.21.15.114:6816/2445491153,v1:172.21.15.114:6817/2445491153] boot 2024-03-27T10:50:44.770 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:44 smithi114 ceph-mon[35338]: osdmap e54: 8 total, 8 up, 8 in 2024-03-27T10:50:44.770 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:44 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-27T10:50:44.770 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:44 smithi114 ceph-mon[35338]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-03-27T10:50:44.778 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:44 smithi177 ceph-mon[35294]: purged_snaps scrub starts 2024-03-27T10:50:44.778 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:44 smithi177 ceph-mon[35294]: purged_snaps scrub ok 2024-03-27T10:50:44.779 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:44 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-27T10:50:44.779 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:44 smithi177 ceph-mon[35294]: osd.7 [v2:172.21.15.114:6816/2445491153,v1:172.21.15.114:6817/2445491153] boot 2024-03-27T10:50:44.779 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:44 smithi177 ceph-mon[35294]: osdmap e54: 8 total, 8 up, 8 in 2024-03-27T10:50:44.779 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:44 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-27T10:50:44.779 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:44 smithi177 ceph-mon[35294]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-03-27T10:50:44.892 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:44 smithi097 ceph-mon[31859]: purged_snaps scrub starts 2024-03-27T10:50:44.892 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:44 smithi097 ceph-mon[31859]: purged_snaps scrub ok 2024-03-27T10:50:44.892 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:44 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-27T10:50:44.892 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:44 smithi097 ceph-mon[31859]: osd.7 [v2:172.21.15.114:6816/2445491153,v1:172.21.15.114:6817/2445491153] boot 2024-03-27T10:50:44.892 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:44 smithi097 ceph-mon[31859]: osdmap e54: 8 total, 8 up, 8 in 2024-03-27T10:50:44.892 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:44 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-27T10:50:44.893 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:44 smithi097 ceph-mon[31859]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-03-27T10:50:45.635 INFO:teuthology.orchestra.run.smithi177.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-27T10:50:45.636 INFO:teuthology.orchestra.run.smithi177.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-27T10:50:45.636 INFO:teuthology.orchestra.run.smithi177.stderr: stderr: 10+0 records in 2024-03-27T10:50:45.636 INFO:teuthology.orchestra.run.smithi177.stderr:10+0 records out 2024-03-27T10:50:45.636 INFO:teuthology.orchestra.run.smithi177.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0363519 s, 288 MB/s 2024-03-27T10:50:45.636 INFO:teuthology.orchestra.run.smithi177.stderr:--> Zapping successful for: 2024-03-27T10:50:45.768 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:45 smithi114 ceph-mon[35338]: osdmap e55: 8 total, 8 up, 8 in 2024-03-27T10:50:45.892 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:45 smithi097 ceph-mon[31859]: osdmap e55: 8 total, 8 up, 8 in 2024-03-27T10:50:45.928 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:45 smithi177 ceph-mon[35294]: osdmap e55: 8 total, 8 up, 8 in 2024-03-27T10:50:46.387 DEBUG:teuthology.orchestra.run.smithi177:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid adca88c4-ec26-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi177:vg_nvme/lv_2 2024-03-27T10:50:46.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:46 smithi177 ceph-mon[35294]: osdmap e56: 8 total, 8 up, 8 in 2024-03-27T10:50:46.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:46 smithi177 ceph-mon[35294]: pgmap v229: 1 pgs: 1 remapped+peering; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-03-27T10:50:46.768 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:46 smithi114 ceph-mon[35338]: osdmap e56: 8 total, 8 up, 8 in 2024-03-27T10:50:46.768 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:46 smithi114 ceph-mon[35338]: pgmap v229: 1 pgs: 1 remapped+peering; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-03-27T10:50:46.821 INFO:teuthology.orchestra.run.smithi177.stderr:Inferring config /var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/mon.c/config 2024-03-27T10:50:46.894 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:46 smithi097 ceph-mon[31859]: osdmap e56: 8 total, 8 up, 8 in 2024-03-27T10:50:46.894 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:46 smithi097 ceph-mon[31859]: pgmap v229: 1 pgs: 1 remapped+peering; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-03-27T10:50:47.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:47 smithi177 ceph-mon[35294]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-03-27T10:50:47.679 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:47 smithi177 ceph-mon[35294]: osdmap e57: 8 total, 8 up, 8 in 2024-03-27T10:50:47.679 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:47 smithi177 ceph-mon[35294]: Detected new or changed devices on smithi114 2024-03-27T10:50:47.679 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:47 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:50:47.679 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:47 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:50:47.679 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:47 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-27T10:50:47.679 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:47 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-27T10:50:47.679 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:47 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-27T10:50:47.679 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:47 smithi177 ceph-mon[35294]: Adjusting osd_memory_target on smithi114 to 5697M 2024-03-27T10:50:47.679 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:47 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:50:47.679 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:47 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:50:47.679 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:47 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:50:47.679 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:47 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:50:47.768 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:47 smithi114 ceph-mon[35338]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-03-27T10:50:47.769 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:47 smithi114 ceph-mon[35338]: osdmap e57: 8 total, 8 up, 8 in 2024-03-27T10:50:47.769 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:47 smithi114 ceph-mon[35338]: Detected new or changed devices on smithi114 2024-03-27T10:50:47.769 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:47 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:50:47.769 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:47 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:50:47.769 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:47 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-27T10:50:47.769 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:47 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-27T10:50:47.769 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:47 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-27T10:50:47.769 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:47 smithi114 ceph-mon[35338]: Adjusting osd_memory_target on smithi114 to 5697M 2024-03-27T10:50:47.769 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:47 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:50:47.769 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:47 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:50:47.769 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:47 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:50:47.769 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:47 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:50:47.892 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:47 smithi097 ceph-mon[31859]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-03-27T10:50:47.892 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:47 smithi097 ceph-mon[31859]: osdmap e57: 8 total, 8 up, 8 in 2024-03-27T10:50:47.892 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:47 smithi097 ceph-mon[31859]: Detected new or changed devices on smithi114 2024-03-27T10:50:47.893 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:47 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:50:47.893 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:47 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:50:47.893 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:47 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-27T10:50:47.893 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:47 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-27T10:50:47.893 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:47 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-27T10:50:47.893 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:47 smithi097 ceph-mon[31859]: Adjusting osd_memory_target on smithi114 to 5697M 2024-03-27T10:50:47.893 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:47 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:50:47.893 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:47 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:50:47.893 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:47 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:50:47.893 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:47 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:50:48.771 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:50:48.798+0000 7fd502600700 1 -- 172.21.15.177:0/4019630137 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd4fc1048b0 msgr2=0x7fd4fc104cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:50:48.771 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:50:48.798+0000 7fd502600700 1 --2- 172.21.15.177:0/4019630137 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd4fc1048b0 0x7fd4fc104cd0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec009990 tx=0x7fd4ec009ca0 comp rx=0 tx=0).stop 2024-03-27T10:50:48.771 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:50:48.798+0000 7fd502600700 1 -- 172.21.15.177:0/4019630137 shutdown_connections 2024-03-27T10:50:48.771 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:50:48.798+0000 7fd502600700 1 --2- 172.21.15.177:0/4019630137 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4fc106450 0x7fd4fc10cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:50:48.771 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:50:48.798+0000 7fd502600700 1 --2- 172.21.15.177:0/4019630137 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd4fc1048b0 0x7fd4fc104cd0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:50:48.771 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:50:48.798+0000 7fd502600700 1 --2- 172.21.15.177:0/4019630137 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd4fc105ab0 0x7fd4fc105f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:50:48.771 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:50:48.798+0000 7fd502600700 1 -- 172.21.15.177:0/4019630137 >> 172.21.15.177:0/4019630137 conn(0x7fd4fc0ffe50 msgr2=0x7fd4fc102290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:50:48.772 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:50:48.798+0000 7fd502600700 1 -- 172.21.15.177:0/4019630137 shutdown_connections 2024-03-27T10:50:48.772 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:50:48.798+0000 7fd502600700 1 -- 172.21.15.177:0/4019630137 wait complete. 2024-03-27T10:50:48.773 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:50:48.799+0000 7fd502600700 1 Processor -- start 2024-03-27T10:50:48.773 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:50:48.799+0000 7fd502600700 1 -- start start 2024-03-27T10:50:48.773 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:50:48.800+0000 7fd502600700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd4fc1048b0 0x7fd4fc19aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:50:48.773 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:50:48.800+0000 7fd502600700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4fc105ab0 0x7fd4fc19af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:50:48.774 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:50:48.800+0000 7fd502600700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd4fc106450 0x7fd4fc1a1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:50:48.774 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:50:48.800+0000 7fd502600700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd4fc110340 con 0x7fd4fc105ab0 2024-03-27T10:50:48.774 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:50:48.800+0000 7fd502600700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd4fc1101c0 con 0x7fd4fc106450 2024-03-27T10:50:48.774 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:50:48.800+0000 7fd502600700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd4fc1104c0 con 0x7fd4fc1048b0 2024-03-27T10:50:48.774 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:50:48.800+0000 7fd4fb400700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd4fc1048b0 0x7fd4fc19aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:50:48.774 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:50:48.800+0000 7fd4fb400700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd4fc1048b0 0x7fd4fc19aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:55538/0 (socket says 172.21.15.177:55538) 2024-03-27T10:50:48.774 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:50:48.800+0000 7fd4fb400700 1 -- 172.21.15.177:0/2816609199 learned_addr learned my addr 172.21.15.177:0/2816609199 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-03-27T10:50:48.774 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:50:48.800+0000 7fd4f3e00700 1 --2- 172.21.15.177:0/2816609199 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4fc105ab0 0x7fd4fc19af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:50:48.774 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:50:48.800+0000 7fd4fbe00700 1 --2- 172.21.15.177:0/2816609199 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd4fc106450 0x7fd4fc1a1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:50:48.774 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:50:48.800+0000 7fd4fb400700 1 -- 172.21.15.177:0/2816609199 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd4fc106450 msgr2=0x7fd4fc1a1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:50:48.774 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:50:48.800+0000 7fd4fb400700 1 --2- 172.21.15.177:0/2816609199 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd4fc106450 0x7fd4fc1a1fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:50:48.774 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:50:48.801+0000 7fd4fb400700 1 -- 172.21.15.177:0/2816609199 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4fc105ab0 msgr2=0x7fd4fc19af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:50:48.774 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:50:48.801+0000 7fd4fb400700 1 --2- 172.21.15.177:0/2816609199 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4fc105ab0 0x7fd4fc19af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:50:48.775 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:50:48.801+0000 7fd4fb400700 1 -- 172.21.15.177:0/2816609199 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4fc1a2670 con 0x7fd4fc1048b0 2024-03-27T10:50:48.775 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:50:48.801+0000 7fd4fb400700 1 --2- 172.21.15.177:0/2816609199 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd4fc1048b0 0x7fd4fc19aa10 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec000c00 tx=0x7fd4ec005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:50:48.775 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:50:48.801+0000 7fd4f8c00700 1 -- 172.21.15.177:0/2816609199 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4ec027070 con 0x7fd4fc1048b0 2024-03-27T10:50:48.775 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:50:48.801+0000 7fd4f8c00700 1 -- 172.21.15.177:0/2816609199 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd4ec004180 con 0x7fd4fc1048b0 2024-03-27T10:50:48.776 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:50:48.801+0000 7fd4f8c00700 1 -- 172.21.15.177:0/2816609199 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4ec00f460 con 0x7fd4fc1048b0 2024-03-27T10:50:48.776 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:50:48.801+0000 7fd502600700 1 -- 172.21.15.177:0/2816609199 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4fc1a2900 con 0x7fd4fc1048b0 2024-03-27T10:50:48.776 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:50:48.801+0000 7fd502600700 1 -- 172.21.15.177:0/2816609199 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4fc1a2d50 con 0x7fd4fc1048b0 2024-03-27T10:50:48.776 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:50:48.802+0000 7fd502600700 1 -- 172.21.15.177:0/2816609199 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4fc069300 con 0x7fd4fc1048b0 2024-03-27T10:50:48.777 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:50:48.805+0000 7fd4f8c00700 1 -- 172.21.15.177:0/2816609199 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 16) ==== 138370+0+0 (secure 0 0 0) 0x7fd4ec02a070 con 0x7fd4fc1048b0 2024-03-27T10:50:48.778 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:50:48.806+0000 7fd4f8c00700 1 --2- 172.21.15.177:0/2816609199 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fd4e00a36a0 0x7fd4e00a5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:50:48.778 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:50:48.806+0000 7fd4f8c00700 1 -- 172.21.15.177:0/2816609199 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5397+0+0 (secure 0 0 0) 0x7fd4ec0d0ab0 con 0x7fd4fc1048b0 2024-03-27T10:50:48.779 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:50:48.807+0000 7fd4f3e00700 1 --2- 172.21.15.177:0/2816609199 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fd4e00a36a0 0x7fd4e00a5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:50:48.780 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:50:48.807+0000 7fd4f3e00700 1 --2- 172.21.15.177:0/2816609199 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fd4e00a36a0 0x7fd4e00a5b60 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd4e4005950 tx=0x7fd4e4005eb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:50:48.781 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:50:48.809+0000 7fd4f8c00700 1 -- 172.21.15.177:0/2816609199 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7fd4ec031020 con 0x7fd4fc1048b0 2024-03-27T10:50:48.892 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:48 smithi097 ceph-mon[31859]: osdmap e58: 8 total, 8 up, 8 in 2024-03-27T10:50:48.892 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:48 smithi097 ceph-mon[31859]: pgmap v232: 1 pgs: 1 remapped+peering; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-03-27T10:50:48.927 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:48 smithi177 ceph-mon[35294]: osdmap e58: 8 total, 8 up, 8 in 2024-03-27T10:50:48.927 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:48 smithi177 ceph-mon[35294]: pgmap v232: 1 pgs: 1 remapped+peering; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-03-27T10:50:49.017 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:50:49.044+0000 7fd502600700 1 -- 172.21.15.177:0/2816609199 --> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi177:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fd4fc062a80 con 0x7fd4e00a36a0 2024-03-27T10:50:49.017 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:48 smithi114 ceph-mon[35338]: osdmap e58: 8 total, 8 up, 8 in 2024-03-27T10:50:49.018 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:48 smithi114 ceph-mon[35338]: pgmap v232: 1 pgs: 1 remapped+peering; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-03-27T10:50:49.892 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:49 smithi097 ceph-mon[31859]: from='client.24283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi177:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T10:50:49.892 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:49 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-27T10:50:49.892 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:49 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-27T10:50:49.892 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:49 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:50:49.927 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:49 smithi177 ceph-mon[35294]: from='client.24283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi177:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T10:50:49.927 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:49 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-27T10:50:49.928 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:49 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-27T10:50:49.928 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:49 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:50:50.018 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:49 smithi114 ceph-mon[35338]: from='client.24283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi177:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T10:50:50.018 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:49 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-27T10:50:50.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:49 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-27T10:50:50.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:49 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:50:50.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:50 smithi177 ceph-mon[35294]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-03-27T10:50:50.892 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:50 smithi097 ceph-mon[31859]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-03-27T10:50:51.018 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:50 smithi114 ceph-mon[35338]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-03-27T10:50:51.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:51 smithi177 ceph-mon[35294]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-03-27T10:50:51.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:51 smithi177 ceph-mon[35294]: Cluster is now healthy 2024-03-27T10:50:51.892 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:51 smithi097 ceph-mon[31859]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-03-27T10:50:51.892 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:51 smithi097 ceph-mon[31859]: Cluster is now healthy 2024-03-27T10:50:52.018 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:51 smithi114 ceph-mon[35338]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-03-27T10:50:52.018 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:51 smithi114 ceph-mon[35338]: Cluster is now healthy 2024-03-27T10:50:52.892 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:52 smithi097 ceph-mon[31859]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-03-27T10:50:52.928 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:52 smithi177 ceph-mon[35294]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-03-27T10:50:53.018 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:52 smithi114 ceph-mon[35338]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-03-27T10:50:53.829 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:53 smithi177 ceph-mon[35294]: from='client.? 172.21.15.177:0/3470055550' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "31ac592e-1349-49ee-9af9-8ad85d682978"}]: dispatch 2024-03-27T10:50:53.830 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:53 smithi177 ceph-mon[35294]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "31ac592e-1349-49ee-9af9-8ad85d682978"}]: dispatch 2024-03-27T10:50:53.830 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:53 smithi177 ceph-mon[35294]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "31ac592e-1349-49ee-9af9-8ad85d682978"}]': finished 2024-03-27T10:50:53.830 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:53 smithi177 ceph-mon[35294]: osdmap e59: 9 total, 8 up, 9 in 2024-03-27T10:50:53.830 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:53 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-27T10:50:53.892 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:53 smithi097 ceph-mon[31859]: from='client.? 172.21.15.177:0/3470055550' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "31ac592e-1349-49ee-9af9-8ad85d682978"}]: dispatch 2024-03-27T10:50:53.892 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:53 smithi097 ceph-mon[31859]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "31ac592e-1349-49ee-9af9-8ad85d682978"}]: dispatch 2024-03-27T10:50:53.892 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:53 smithi097 ceph-mon[31859]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "31ac592e-1349-49ee-9af9-8ad85d682978"}]': finished 2024-03-27T10:50:53.892 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:53 smithi097 ceph-mon[31859]: osdmap e59: 9 total, 8 up, 9 in 2024-03-27T10:50:53.892 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:53 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-27T10:50:54.018 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:53 smithi114 ceph-mon[35338]: from='client.? 172.21.15.177:0/3470055550' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "31ac592e-1349-49ee-9af9-8ad85d682978"}]: dispatch 2024-03-27T10:50:54.018 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:53 smithi114 ceph-mon[35338]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "31ac592e-1349-49ee-9af9-8ad85d682978"}]: dispatch 2024-03-27T10:50:54.018 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:53 smithi114 ceph-mon[35338]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "31ac592e-1349-49ee-9af9-8ad85d682978"}]': finished 2024-03-27T10:50:54.018 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:53 smithi114 ceph-mon[35338]: osdmap e59: 9 total, 8 up, 9 in 2024-03-27T10:50:54.018 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:53 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-27T10:50:54.848 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:54 smithi177 ceph-mon[35294]: from='client.? 172.21.15.177:0/1832781220' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-27T10:50:54.849 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:54 smithi177 ceph-mon[35294]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-03-27T10:50:54.892 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:54 smithi097 ceph-mon[31859]: from='client.? 172.21.15.177:0/1832781220' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-27T10:50:54.892 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:54 smithi097 ceph-mon[31859]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-03-27T10:50:55.018 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:54 smithi114 ceph-mon[35338]: from='client.? 172.21.15.177:0/1832781220' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-27T10:50:55.018 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:54 smithi114 ceph-mon[35338]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-03-27T10:50:57.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:57 smithi097 ceph-mon[31859]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-03-27T10:50:57.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:57 smithi177 ceph-mon[35294]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-03-27T10:50:57.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:57 smithi114 ceph-mon[35338]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-03-27T10:50:59.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:50:59 smithi177 ceph-mon[35294]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-03-27T10:50:59.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:50:59 smithi097 ceph-mon[31859]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-03-27T10:50:59.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:50:59 smithi114 ceph-mon[35338]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-03-27T10:51:01.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:01 smithi097 ceph-mon[31859]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-03-27T10:51:01.396 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:01 smithi177 ceph-mon[35294]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-03-27T10:51:01.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:01 smithi114 ceph-mon[35338]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-03-27T10:51:03.321 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:03 smithi114 ceph-mon[35338]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-03-27T10:51:03.321 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:03 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-27T10:51:03.321 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:03 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:51:03.321 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:03 smithi114 ceph-mon[35338]: Deploying daemon osd.8 on smithi177 2024-03-27T10:51:03.371 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:03 smithi177 ceph-mon[35294]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-03-27T10:51:03.372 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:03 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-27T10:51:03.372 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:03 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:51:03.372 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:03 smithi177 ceph-mon[35294]: Deploying daemon osd.8 on smithi177 2024-03-27T10:51:03.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:03 smithi097 ceph-mon[31859]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-03-27T10:51:03.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:03 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-27T10:51:03.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:03 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:51:03.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:03 smithi097 ceph-mon[31859]: Deploying daemon osd.8 on smithi177 2024-03-27T10:51:05.268 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:05 smithi114 ceph-mon[35338]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-03-27T10:51:05.327 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:05 smithi177 ceph-mon[35294]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-03-27T10:51:05.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:05 smithi097 ceph-mon[31859]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-03-27T10:51:07.299 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:07 smithi177 ceph-mon[35294]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-03-27T10:51:07.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:07 smithi097 ceph-mon[31859]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-03-27T10:51:07.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:07 smithi114 ceph-mon[35338]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-03-27T10:51:09.177 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:09 smithi177 ceph-mon[35294]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-03-27T10:51:09.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:09 smithi097 ceph-mon[31859]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-03-27T10:51:09.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:09 smithi114 ceph-mon[35338]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-03-27T10:51:11.248 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:11 smithi177 ceph-mon[35294]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-03-27T10:51:11.248 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:11 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:51:11.248 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:11 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:51:11.248 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:11 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:51:11.268 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:11 smithi114 ceph-mon[35338]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-03-27T10:51:11.268 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:11 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:51:11.268 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:11 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:51:11.268 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:11 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:51:11.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:11 smithi097 ceph-mon[31859]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-03-27T10:51:11.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:11 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:51:11.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:11 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:51:11.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:11 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:51:13.268 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:13 smithi114 ceph-mon[35338]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-03-27T10:51:13.341 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:13 smithi177 ceph-mon[35294]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-03-27T10:51:13.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:13 smithi097 ceph-mon[31859]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-03-27T10:51:14.820 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:14 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:51:14.820 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:14 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:51:14.820 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:14 smithi177 ceph-mon[35294]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-03-27T10:51:14.820 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:14 smithi177 ceph-mon[35294]: from='osd.8 [v2:172.21.15.177:6816/2051134191,v1:172.21.15.177:6817/2051134191]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-27T10:51:14.820 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:14 smithi177 ceph-mon[35294]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-27T10:51:14.891 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:14 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:51:14.891 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:14 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:51:14.892 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:14 smithi097 ceph-mon[31859]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-03-27T10:51:14.892 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:14 smithi097 ceph-mon[31859]: from='osd.8 [v2:172.21.15.177:6816/2051134191,v1:172.21.15.177:6817/2051134191]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-27T10:51:14.892 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:14 smithi097 ceph-mon[31859]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-27T10:51:14.940 INFO:teuthology.orchestra.run.smithi177.stdout:Created osd(s) 8 on host 'smithi177' 2024-03-27T10:51:14.940 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:51:14.965+0000 7fd4f8c00700 1 -- 172.21.15.177:0/2816609199 <== mgr.14152 v2:172.21.15.97:6800/3955463538 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd4fc062a80 con 0x7fd4e00a36a0 2024-03-27T10:51:14.943 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:51:14.969+0000 7fd502600700 1 -- 172.21.15.177:0/2816609199 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fd4e00a36a0 msgr2=0x7fd4e00a5b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:51:14.943 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:51:14.969+0000 7fd502600700 1 --2- 172.21.15.177:0/2816609199 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fd4e00a36a0 0x7fd4e00a5b60 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd4e4005950 tx=0x7fd4e4005eb0 comp rx=0 tx=0).stop 2024-03-27T10:51:14.943 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:51:14.969+0000 7fd502600700 1 -- 172.21.15.177:0/2816609199 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd4fc1048b0 msgr2=0x7fd4fc19aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:51:14.943 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:51:14.969+0000 7fd502600700 1 --2- 172.21.15.177:0/2816609199 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd4fc1048b0 0x7fd4fc19aa10 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec000c00 tx=0x7fd4ec005e80 comp rx=0 tx=0).stop 2024-03-27T10:51:14.943 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:51:14.969+0000 7fd502600700 1 -- 172.21.15.177:0/2816609199 shutdown_connections 2024-03-27T10:51:14.944 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:51:14.969+0000 7fd502600700 1 --2- 172.21.15.177:0/2816609199 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fd4e00a36a0 0x7fd4e00a5b60 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:51:14.944 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:51:14.969+0000 7fd502600700 1 --2- 172.21.15.177:0/2816609199 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd4fc106450 0x7fd4fc1a1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:51:14.944 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:51:14.969+0000 7fd502600700 1 --2- 172.21.15.177:0/2816609199 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd4fc1048b0 0x7fd4fc19aa10 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:51:14.944 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:51:14.969+0000 7fd502600700 1 --2- 172.21.15.177:0/2816609199 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4fc105ab0 0x7fd4fc19af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:51:14.944 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:51:14.969+0000 7fd502600700 1 -- 172.21.15.177:0/2816609199 >> 172.21.15.177:0/2816609199 conn(0x7fd4fc0ffe50 msgr2=0x7fd4fc108e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:51:14.944 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:51:14.970+0000 7fd502600700 1 -- 172.21.15.177:0/2816609199 shutdown_connections 2024-03-27T10:51:14.944 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:51:14.970+0000 7fd502600700 1 -- 172.21.15.177:0/2816609199 wait complete. 2024-03-27T10:51:14.950 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:14 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:51:14.950 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:14 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:51:14.950 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:14 smithi114 ceph-mon[35338]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-03-27T10:51:14.951 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:14 smithi114 ceph-mon[35338]: from='osd.8 [v2:172.21.15.177:6816/2051134191,v1:172.21.15.177:6817/2051134191]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-27T10:51:14.951 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:14 smithi114 ceph-mon[35338]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-27T10:51:15.708 DEBUG:teuthology.orchestra.run.smithi177:osd.8> sudo journalctl -f -n 0 -u ceph-adca88c4-ec26-11ee-95d0-87774f69a715@osd.8.service 2024-03-27T10:51:15.715 INFO:tasks.cephadm:Deploying osd.9 on smithi097 with /dev/vg_nvme/lv_1... 2024-03-27T10:51:15.715 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid adca88c4-ec26-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-27T10:51:15.892 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:15 smithi097 ceph-mon[31859]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-27T10:51:15.892 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:15 smithi097 ceph-mon[31859]: from='osd.8 [v2:172.21.15.177:6816/2051134191,v1:172.21.15.177:6817/2051134191]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2024-03-27T10:51:15.892 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:15 smithi097 ceph-mon[31859]: osdmap e60: 9 total, 8 up, 9 in 2024-03-27T10:51:15.892 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:15 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-27T10:51:15.892 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:15 smithi097 ceph-mon[31859]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2024-03-27T10:51:15.892 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:15 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:51:15.892 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:15 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:51:15.892 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:15 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:51:15.893 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:15 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:51:15.893 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:15 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:51:15.893 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:15 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:51:15.927 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:15 smithi177 ceph-mon[35294]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-27T10:51:15.927 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:15 smithi177 ceph-mon[35294]: from='osd.8 [v2:172.21.15.177:6816/2051134191,v1:172.21.15.177:6817/2051134191]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2024-03-27T10:51:15.927 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:15 smithi177 ceph-mon[35294]: osdmap e60: 9 total, 8 up, 9 in 2024-03-27T10:51:15.927 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:15 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-27T10:51:15.927 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:15 smithi177 ceph-mon[35294]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2024-03-27T10:51:15.928 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:15 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:51:15.928 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:15 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:51:15.928 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:15 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:51:15.928 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:15 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:51:15.928 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:15 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:51:15.928 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:15 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:51:16.018 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:15 smithi114 ceph-mon[35338]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-27T10:51:16.018 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:15 smithi114 ceph-mon[35338]: from='osd.8 [v2:172.21.15.177:6816/2051134191,v1:172.21.15.177:6817/2051134191]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2024-03-27T10:51:16.018 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:15 smithi114 ceph-mon[35338]: osdmap e60: 9 total, 8 up, 9 in 2024-03-27T10:51:16.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:15 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-27T10:51:16.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:15 smithi114 ceph-mon[35338]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2024-03-27T10:51:16.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:15 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:51:16.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:15 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:51:16.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:15 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:51:16.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:15 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:51:16.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:15 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:51:16.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:15 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:51:16.161 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T10:51:16.787 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 10:51:16 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T10:51:16.692+0000 7f06a9800700 -1 osd.8 0 waiting for initial osdmap 2024-03-27T10:51:16.787 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 10:51:16 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T10:51:16.725+0000 7f069f600700 -1 osd.8 62 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-27T10:51:16.788 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:16 smithi177 ceph-mon[35294]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]': finished 2024-03-27T10:51:16.788 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:16 smithi177 ceph-mon[35294]: osdmap e61: 9 total, 8 up, 9 in 2024-03-27T10:51:16.788 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:16 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-27T10:51:16.788 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:16 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-27T10:51:16.788 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:16 smithi177 ceph-mon[35294]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-03-27T10:51:16.788 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:16 smithi177 ceph-mon[35294]: osdmap e62: 9 total, 8 up, 9 in 2024-03-27T10:51:16.893 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:16 smithi097 ceph-mon[31859]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]': finished 2024-03-27T10:51:16.893 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:16 smithi097 ceph-mon[31859]: osdmap e61: 9 total, 8 up, 9 in 2024-03-27T10:51:16.893 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:16 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-27T10:51:16.893 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:16 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-27T10:51:16.893 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:16 smithi097 ceph-mon[31859]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-03-27T10:51:16.893 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:16 smithi097 ceph-mon[31859]: osdmap e62: 9 total, 8 up, 9 in 2024-03-27T10:51:17.018 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:16 smithi114 ceph-mon[35338]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]': finished 2024-03-27T10:51:17.018 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:16 smithi114 ceph-mon[35338]: osdmap e61: 9 total, 8 up, 9 in 2024-03-27T10:51:17.018 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:16 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-27T10:51:17.018 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:16 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-27T10:51:17.018 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:16 smithi114 ceph-mon[35338]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-03-27T10:51:17.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:16 smithi114 ceph-mon[35338]: osdmap e62: 9 total, 8 up, 9 in 2024-03-27T10:51:17.866 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:17 smithi177 ceph-mon[35294]: purged_snaps scrub starts 2024-03-27T10:51:17.867 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:17 smithi177 ceph-mon[35294]: purged_snaps scrub ok 2024-03-27T10:51:17.867 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:17 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-27T10:51:17.867 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:17 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-27T10:51:17.867 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:17 smithi177 ceph-mon[35294]: from='osd.8 ' entity='osd.8' 2024-03-27T10:51:17.867 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:17 smithi177 ceph-mon[35294]: osd.8 [v2:172.21.15.177:6816/2051134191,v1:172.21.15.177:6817/2051134191] boot 2024-03-27T10:51:17.892 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:17 smithi097 ceph-mon[31859]: purged_snaps scrub starts 2024-03-27T10:51:17.892 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:17 smithi097 ceph-mon[31859]: purged_snaps scrub ok 2024-03-27T10:51:17.892 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:17 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-27T10:51:17.892 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:17 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-27T10:51:17.893 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:17 smithi097 ceph-mon[31859]: from='osd.8 ' entity='osd.8' 2024-03-27T10:51:17.893 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:17 smithi097 ceph-mon[31859]: osd.8 [v2:172.21.15.177:6816/2051134191,v1:172.21.15.177:6817/2051134191] boot 2024-03-27T10:51:18.018 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:17 smithi114 ceph-mon[35338]: purged_snaps scrub starts 2024-03-27T10:51:18.018 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:17 smithi114 ceph-mon[35338]: purged_snaps scrub ok 2024-03-27T10:51:18.018 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:17 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-27T10:51:18.018 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:17 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-27T10:51:18.018 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:17 smithi114 ceph-mon[35338]: from='osd.8 ' entity='osd.8' 2024-03-27T10:51:18.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:17 smithi114 ceph-mon[35338]: osd.8 [v2:172.21.15.177:6816/2051134191,v1:172.21.15.177:6817/2051134191] boot 2024-03-27T10:51:18.928 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:18 smithi177 ceph-mon[35294]: osdmap e63: 9 total, 9 up, 9 in 2024-03-27T10:51:18.928 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:18 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-27T10:51:18.928 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:18 smithi177 ceph-mon[35294]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-03-27T10:51:19.018 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:18 smithi114 ceph-mon[35338]: osdmap e63: 9 total, 9 up, 9 in 2024-03-27T10:51:19.018 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:18 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-27T10:51:19.018 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:18 smithi114 ceph-mon[35338]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-03-27T10:51:19.076 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:18 smithi097 ceph-mon[31859]: osdmap e63: 9 total, 9 up, 9 in 2024-03-27T10:51:19.077 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:18 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-27T10:51:19.077 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:18 smithi097 ceph-mon[31859]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-03-27T10:51:19.077 INFO:teuthology.orchestra.run.smithi097.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-27T10:51:19.077 INFO:teuthology.orchestra.run.smithi097.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-27T10:51:19.077 INFO:teuthology.orchestra.run.smithi097.stderr: stderr: 10+0 records in 2024-03-27T10:51:19.077 INFO:teuthology.orchestra.run.smithi097.stderr:10+0 records out 2024-03-27T10:51:19.077 INFO:teuthology.orchestra.run.smithi097.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0372702 s, 281 MB/s 2024-03-27T10:51:19.077 INFO:teuthology.orchestra.run.smithi097.stderr:--> Zapping successful for: 2024-03-27T10:51:19.882 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:19 smithi097 ceph-mon[31859]: osdmap e64: 9 total, 9 up, 9 in 2024-03-27T10:51:19.928 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:19 smithi177 ceph-mon[35294]: osdmap e64: 9 total, 9 up, 9 in 2024-03-27T10:51:19.940 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid adca88c4-ec26-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi097:vg_nvme/lv_1 2024-03-27T10:51:20.018 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:19 smithi114 ceph-mon[35338]: osdmap e64: 9 total, 9 up, 9 in 2024-03-27T10:51:20.390 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T10:51:20.707 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:20 smithi097 ceph-mon[31859]: osdmap e65: 9 total, 9 up, 9 in 2024-03-27T10:51:20.707 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:20 smithi097 ceph-mon[31859]: pgmap v255: 1 pgs: 1 remapped+peering; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-03-27T10:51:20.707 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:20 smithi097 ceph-mon[31859]: osdmap e66: 9 total, 9 up, 9 in 2024-03-27T10:51:20.791 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:20 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:51:20.792 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:20 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:51:20.792 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:20 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-27T10:51:20.792 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:20 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-27T10:51:20.792 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:20 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-27T10:51:20.792 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:20 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:51:20.792 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:20 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:51:20.792 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:20 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:51:20.792 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:20 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:51:21.018 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:20 smithi114 ceph-mon[35338]: osdmap e65: 9 total, 9 up, 9 in 2024-03-27T10:51:21.018 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:20 smithi114 ceph-mon[35338]: pgmap v255: 1 pgs: 1 remapped+peering; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-03-27T10:51:21.018 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:20 smithi114 ceph-mon[35338]: osdmap e66: 9 total, 9 up, 9 in 2024-03-27T10:51:21.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:20 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:51:21.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:20 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:51:21.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:20 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-27T10:51:21.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:20 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-27T10:51:21.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:20 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-27T10:51:21.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:20 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:51:21.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:20 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:51:21.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:20 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:51:21.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:20 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:51:21.178 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:20 smithi177 ceph-mon[35294]: osdmap e65: 9 total, 9 up, 9 in 2024-03-27T10:51:21.178 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:20 smithi177 ceph-mon[35294]: pgmap v255: 1 pgs: 1 remapped+peering; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-03-27T10:51:21.179 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:20 smithi177 ceph-mon[35294]: osdmap e66: 9 total, 9 up, 9 in 2024-03-27T10:51:21.179 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:20 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:51:21.179 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:20 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:51:21.179 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:20 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-27T10:51:21.179 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:20 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-27T10:51:21.179 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:20 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-27T10:51:21.179 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:20 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:51:21.179 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:20 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:51:21.179 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:20 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:51:21.179 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:20 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:51:22.242 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:22 smithi097 ceph-mon[31859]: Detected new or changed devices on smithi177 2024-03-27T10:51:22.243 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:22 smithi097 ceph-mon[31859]: Adjusting osd_memory_target on smithi177 to 5696M 2024-03-27T10:51:22.243 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:22 smithi097 ceph-mon[31859]: osdmap e67: 9 total, 9 up, 9 in 2024-03-27T10:51:22.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:51:22.440+0000 7f5882e00700 1 -- 172.21.15.97:0/835359286 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f587c1015a0 msgr2=0x7f587c1019c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:51:22.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:51:22.440+0000 7f5882e00700 1 --2- 172.21.15.97:0/835359286 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f587c1015a0 0x7f587c1019c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f586c0099c0 tx=0x7f586c009cd0 comp rx=0 tx=0).stop 2024-03-27T10:51:22.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:51:22.440+0000 7f5882e00700 1 -- 172.21.15.97:0/835359286 shutdown_connections 2024-03-27T10:51:22.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:51:22.440+0000 7f5882e00700 1 --2- 172.21.15.97:0/835359286 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f587c103140 0x7f587c1099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:51:22.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:51:22.440+0000 7f5882e00700 1 --2- 172.21.15.97:0/835359286 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f587c1015a0 0x7f587c1019c0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:51:22.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:51:22.440+0000 7f5882e00700 1 --2- 172.21.15.97:0/835359286 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f587c1027a0 0x7f587c102c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:51:22.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:51:22.440+0000 7f5882e00700 1 -- 172.21.15.97:0/835359286 >> 172.21.15.97:0/835359286 conn(0x7f587c0fcb40 msgr2=0x7f587c0fef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:51:22.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:51:22.440+0000 7f5882e00700 1 -- 172.21.15.97:0/835359286 shutdown_connections 2024-03-27T10:51:22.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:51:22.440+0000 7f5882e00700 1 -- 172.21.15.97:0/835359286 wait complete. 2024-03-27T10:51:22.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:51:22.441+0000 7f5882e00700 1 Processor -- start 2024-03-27T10:51:22.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:51:22.442+0000 7f5882e00700 1 -- start start 2024-03-27T10:51:22.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:51:22.442+0000 7f5882e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f587c1015a0 0x7f587c1987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:51:22.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:51:22.442+0000 7f5882e00700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f587c1027a0 0x7f587c198d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:51:22.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:51:22.442+0000 7f5882e00700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f587c103140 0x7f587c19fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:51:22.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:51:22.442+0000 7f5882e00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f587c10d020 con 0x7f587c1015a0 2024-03-27T10:51:22.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:51:22.442+0000 7f5882e00700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f587c10cea0 con 0x7f587c1027a0 2024-03-27T10:51:22.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:51:22.442+0000 7f5882e00700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f587c10d1a0 con 0x7f587c103140 2024-03-27T10:51:22.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:51:22.442+0000 7f587be00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f587c1015a0 0x7f587c1987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:51:22.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:51:22.442+0000 7f587be00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f587c1015a0 0x7f587c1987c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47950/0 (socket says 172.21.15.97:47950) 2024-03-27T10:51:22.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:51:22.442+0000 7f587be00700 1 -- 172.21.15.97:0/4156309143 learned_addr learned my addr 172.21.15.97:0/4156309143 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:51:22.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:51:22.443+0000 7f587b400700 1 --2- 172.21.15.97:0/4156309143 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f587c1027a0 0x7f587c198d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:51:22.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:51:22.443+0000 7f5880e00700 1 --2- 172.21.15.97:0/4156309143 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f587c103140 0x7f587c19fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:51:22.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:51:22.443+0000 7f587be00700 1 -- 172.21.15.97:0/4156309143 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f587c103140 msgr2=0x7f587c19fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:51:22.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:51:22.443+0000 7f587be00700 1 --2- 172.21.15.97:0/4156309143 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f587c103140 0x7f587c19fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:51:22.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:51:22.443+0000 7f587be00700 1 -- 172.21.15.97:0/4156309143 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f587c1027a0 msgr2=0x7f587c198d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:51:22.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:51:22.443+0000 7f587be00700 1 --2- 172.21.15.97:0/4156309143 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f587c1027a0 0x7f587c198d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:51:22.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:51:22.443+0000 7f587be00700 1 -- 172.21.15.97:0/4156309143 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f587c1a0420 con 0x7f587c1015a0 2024-03-27T10:51:22.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:51:22.443+0000 7f587be00700 1 --2- 172.21.15.97:0/4156309143 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f587c1015a0 0x7f587c1987c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f586c00b580 tx=0x7f586c009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:51:22.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:51:22.444+0000 7f5878c00700 1 -- 172.21.15.97:0/4156309143 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f586c027070 con 0x7f587c1015a0 2024-03-27T10:51:22.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:51:22.444+0000 7f5878c00700 1 -- 172.21.15.97:0/4156309143 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f586c0040d0 con 0x7f587c1015a0 2024-03-27T10:51:22.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:51:22.444+0000 7f5882e00700 1 -- 172.21.15.97:0/4156309143 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f587c1a06b0 con 0x7f587c1015a0 2024-03-27T10:51:22.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:51:22.444+0000 7f5878c00700 1 -- 172.21.15.97:0/4156309143 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f586c022d90 con 0x7f587c1015a0 2024-03-27T10:51:22.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:51:22.444+0000 7f5882e00700 1 -- 172.21.15.97:0/4156309143 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f587c1a0b90 con 0x7f587c1015a0 2024-03-27T10:51:22.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:51:22.445+0000 7f5882e00700 1 -- 172.21.15.97:0/4156309143 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f587c069300 con 0x7f587c1015a0 2024-03-27T10:51:22.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:51:22.448+0000 7f5878c00700 1 -- 172.21.15.97:0/4156309143 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138370+0+0 (secure 0 0 0) 0x7f586c02a070 con 0x7f587c1015a0 2024-03-27T10:51:22.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:51:22.450+0000 7f5878c00700 1 --2- 172.21.15.97:0/4156309143 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f58600a36a0 0x7f58600a5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:51:22.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:51:22.450+0000 7f5878c00700 1 -- 172.21.15.97:0/4156309143 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5829+0+0 (secure 0 0 0) 0x7f586c0d0a10 con 0x7f587c1015a0 2024-03-27T10:51:22.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:51:22.450+0000 7f587b400700 1 --2- 172.21.15.97:0/4156309143 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f58600a36a0 0x7f58600a5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:51:22.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:51:22.451+0000 7f587b400700 1 --2- 172.21.15.97:0/4156309143 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f58600a36a0 0x7f58600a5b60 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5868005ea0 tx=0x7f5868005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:51:22.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:51:22.453+0000 7f5878c00700 1 -- 172.21.15.97:0/4156309143 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f586c031020 con 0x7f587c1015a0 2024-03-27T10:51:22.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:22 smithi177 ceph-mon[35294]: Detected new or changed devices on smithi177 2024-03-27T10:51:22.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:22 smithi177 ceph-mon[35294]: Adjusting osd_memory_target on smithi177 to 5696M 2024-03-27T10:51:22.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:22 smithi177 ceph-mon[35294]: osdmap e67: 9 total, 9 up, 9 in 2024-03-27T10:51:22.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:22 smithi114 ceph-mon[35338]: Detected new or changed devices on smithi177 2024-03-27T10:51:22.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:22 smithi114 ceph-mon[35338]: Adjusting osd_memory_target on smithi177 to 5696M 2024-03-27T10:51:22.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:22 smithi114 ceph-mon[35338]: osdmap e67: 9 total, 9 up, 9 in 2024-03-27T10:51:22.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:51:22.687+0000 7f5882e00700 1 -- 172.21.15.97:0/4156309143 --> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f587c0fecd0 con 0x7f58600a36a0 2024-03-27T10:51:23.330 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:23 smithi177 ceph-mon[35294]: pgmap v258: 1 pgs: 1 remapped+peering; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-03-27T10:51:23.330 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:23 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-27T10:51:23.330 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:23 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-27T10:51:23.330 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:23 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:51:23.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:23 smithi097 ceph-mon[31859]: pgmap v258: 1 pgs: 1 remapped+peering; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-03-27T10:51:23.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:23 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-27T10:51:23.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:23 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-27T10:51:23.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:23 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:51:23.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:23 smithi114 ceph-mon[35338]: pgmap v258: 1 pgs: 1 remapped+peering; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-03-27T10:51:23.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:23 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-27T10:51:23.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:23 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-27T10:51:23.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:23 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:51:24.321 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:24 smithi097 ceph-mon[31859]: from='client.14412 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T10:51:24.346 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:24 smithi177 ceph-mon[35294]: from='client.14412 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T10:51:24.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:24 smithi114 ceph-mon[35338]: from='client.14412 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T10:51:25.262 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:25 smithi097 ceph-mon[31859]: pgmap v259: 1 pgs: 1 remapped+peering; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-03-27T10:51:25.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:25 smithi177 ceph-mon[35294]: pgmap v259: 1 pgs: 1 remapped+peering; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-03-27T10:51:25.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:25 smithi114 ceph-mon[35338]: pgmap v259: 1 pgs: 1 remapped+peering; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-03-27T10:51:27.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:27 smithi097 ceph-mon[31859]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-03-27T10:51:27.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:27 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/309801764' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0171086a-dd41-4eb4-a389-16f0c46a64db"}]: dispatch 2024-03-27T10:51:27.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:27 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/309801764' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0171086a-dd41-4eb4-a389-16f0c46a64db"}]': finished 2024-03-27T10:51:27.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:27 smithi097 ceph-mon[31859]: osdmap e68: 10 total, 9 up, 10 in 2024-03-27T10:51:27.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:27 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-27T10:51:27.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:27 smithi177 ceph-mon[35294]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-03-27T10:51:27.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:27 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/309801764' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0171086a-dd41-4eb4-a389-16f0c46a64db"}]: dispatch 2024-03-27T10:51:27.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:27 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/309801764' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0171086a-dd41-4eb4-a389-16f0c46a64db"}]': finished 2024-03-27T10:51:27.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:27 smithi177 ceph-mon[35294]: osdmap e68: 10 total, 9 up, 10 in 2024-03-27T10:51:27.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:27 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-27T10:51:27.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:27 smithi114 ceph-mon[35338]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-03-27T10:51:27.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:27 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/309801764' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0171086a-dd41-4eb4-a389-16f0c46a64db"}]: dispatch 2024-03-27T10:51:27.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:27 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/309801764' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0171086a-dd41-4eb4-a389-16f0c46a64db"}]': finished 2024-03-27T10:51:27.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:27 smithi114 ceph-mon[35338]: osdmap e68: 10 total, 9 up, 10 in 2024-03-27T10:51:27.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:27 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-27T10:51:28.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:28 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/3905448347' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-27T10:51:28.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:28 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/3905448347' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-27T10:51:28.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:28 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/3905448347' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-27T10:51:29.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:29 smithi097 ceph-mon[31859]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-03-27T10:51:29.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:29 smithi177 ceph-mon[35294]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-03-27T10:51:29.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:29 smithi114 ceph-mon[35338]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-03-27T10:51:31.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:31 smithi097 ceph-mon[31859]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-03-27T10:51:31.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:31 smithi177 ceph-mon[35294]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-03-27T10:51:31.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:31 smithi114 ceph-mon[35338]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-03-27T10:51:33.263 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:33 smithi097 ceph-mon[31859]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-27T10:51:33.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:33 smithi177 ceph-mon[35294]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-27T10:51:33.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:33 smithi114 ceph-mon[35338]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-27T10:51:35.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:35 smithi097 ceph-mon[31859]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-27T10:51:35.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:35 smithi177 ceph-mon[35294]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-27T10:51:35.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:35 smithi114 ceph-mon[35338]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-27T10:51:36.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:36 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-27T10:51:36.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:36 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:51:36.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:36 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-27T10:51:36.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:36 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:51:36.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:36 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-27T10:51:36.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:36 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:51:37.376 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:37 smithi097 ceph-mon[31859]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 804 GiB / 805 GiB avail 2024-03-27T10:51:37.376 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:37 smithi097 ceph-mon[31859]: Deploying daemon osd.9 on smithi097 2024-03-27T10:51:37.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:37 smithi177 ceph-mon[35294]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 804 GiB / 805 GiB avail 2024-03-27T10:51:37.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:37 smithi177 ceph-mon[35294]: Deploying daemon osd.9 on smithi097 2024-03-27T10:51:37.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:37 smithi114 ceph-mon[35338]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 804 GiB / 805 GiB avail 2024-03-27T10:51:37.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:37 smithi114 ceph-mon[35338]: Deploying daemon osd.9 on smithi097 2024-03-27T10:51:39.257 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:39 smithi097 ceph-mon[31859]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 804 GiB / 805 GiB avail 2024-03-27T10:51:39.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:39 smithi177 ceph-mon[35294]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 804 GiB / 805 GiB avail 2024-03-27T10:51:39.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:39 smithi114 ceph-mon[35338]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 804 GiB / 805 GiB avail 2024-03-27T10:51:41.363 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:41 smithi097 ceph-mon[31859]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 804 GiB / 805 GiB avail 2024-03-27T10:51:41.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:41 smithi177 ceph-mon[35294]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 804 GiB / 805 GiB avail 2024-03-27T10:51:41.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:41 smithi114 ceph-mon[35338]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 804 GiB / 805 GiB avail 2024-03-27T10:51:43.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:43 smithi177 ceph-mon[35294]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 804 GiB / 805 GiB avail 2024-03-27T10:51:43.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:43 smithi114 ceph-mon[35338]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 804 GiB / 805 GiB avail 2024-03-27T10:51:43.641 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:43 smithi097 ceph-mon[31859]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 804 GiB / 805 GiB avail 2024-03-27T10:51:45.485 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:45 smithi097 ceph-mon[31859]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 804 GiB / 805 GiB avail 2024-03-27T10:51:45.485 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:45 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:51:45.485 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:45 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:51:45.485 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:45 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:51:45.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:45 smithi114 ceph-mon[35338]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 804 GiB / 805 GiB avail 2024-03-27T10:51:45.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:45 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:51:45.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:45 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:51:45.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:45 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:51:45.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:45 smithi177 ceph-mon[35294]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 804 GiB / 805 GiB avail 2024-03-27T10:51:45.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:45 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:51:45.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:45 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:51:45.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:45 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:51:47.492 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:47 smithi097 ceph-mon[31859]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 804 GiB / 805 GiB avail 2024-03-27T10:51:47.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:47 smithi114 ceph-mon[35338]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 804 GiB / 805 GiB avail 2024-03-27T10:51:47.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:47 smithi177 ceph-mon[35294]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 804 GiB / 805 GiB avail 2024-03-27T10:51:48.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:48 smithi097 ceph-mon[31859]: from='osd.9 [v2:172.21.15.97:6826/1578881271,v1:172.21.15.97:6827/1578881271]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-27T10:51:48.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:48 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:51:48.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:48 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:51:48.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:48 smithi114 ceph-mon[35338]: from='osd.9 [v2:172.21.15.97:6826/1578881271,v1:172.21.15.97:6827/1578881271]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-27T10:51:48.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:48 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:51:48.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:48 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:51:48.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:48 smithi177 ceph-mon[35294]: from='osd.9 [v2:172.21.15.97:6826/1578881271,v1:172.21.15.97:6827/1578881271]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-27T10:51:48.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:48 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:51:48.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:48 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:51:49.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:51:49.179+0000 7f5878c00700 1 -- 172.21.15.97:0/4156309143 <== mgr.14152 v2:172.21.15.97:6800/3955463538 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f587c0fecd0 con 0x7f58600a36a0 2024-03-27T10:51:49.152 INFO:teuthology.orchestra.run.smithi097.stdout:Created osd(s) 9 on host 'smithi097' 2024-03-27T10:51:49.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:51:49.182+0000 7f5882e00700 1 -- 172.21.15.97:0/4156309143 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f58600a36a0 msgr2=0x7f58600a5b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:51:49.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:51:49.182+0000 7f5882e00700 1 --2- 172.21.15.97:0/4156309143 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f58600a36a0 0x7f58600a5b60 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5868005ea0 tx=0x7f5868005e30 comp rx=0 tx=0).stop 2024-03-27T10:51:49.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:51:49.182+0000 7f5882e00700 1 -- 172.21.15.97:0/4156309143 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f587c1015a0 msgr2=0x7f587c1987c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:51:49.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:51:49.182+0000 7f5882e00700 1 --2- 172.21.15.97:0/4156309143 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f587c1015a0 0x7f587c1987c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f586c00b580 tx=0x7f586c009580 comp rx=0 tx=0).stop 2024-03-27T10:51:49.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:51:49.182+0000 7f5882e00700 1 -- 172.21.15.97:0/4156309143 shutdown_connections 2024-03-27T10:51:49.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:51:49.182+0000 7f5882e00700 1 --2- 172.21.15.97:0/4156309143 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f58600a36a0 0x7f58600a5b60 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:51:49.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:51:49.182+0000 7f5882e00700 1 --2- 172.21.15.97:0/4156309143 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f587c103140 0x7f587c19fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:51:49.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:51:49.182+0000 7f5882e00700 1 --2- 172.21.15.97:0/4156309143 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f587c1015a0 0x7f587c1987c0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:51:49.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:51:49.182+0000 7f5882e00700 1 --2- 172.21.15.97:0/4156309143 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f587c1027a0 0x7f587c198d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:51:49.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:51:49.182+0000 7f5882e00700 1 -- 172.21.15.97:0/4156309143 >> 172.21.15.97:0/4156309143 conn(0x7f587c0fcb40 msgr2=0x7f587c0fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:51:49.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:51:49.183+0000 7f5882e00700 1 -- 172.21.15.97:0/4156309143 shutdown_connections 2024-03-27T10:51:49.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:51:49.183+0000 7f5882e00700 1 -- 172.21.15.97:0/4156309143 wait complete. 2024-03-27T10:51:49.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:49 smithi097 ceph-mon[31859]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 804 GiB / 805 GiB avail 2024-03-27T10:51:49.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:49 smithi097 ceph-mon[31859]: from='osd.9 [v2:172.21.15.97:6826/1578881271,v1:172.21.15.97:6827/1578881271]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-27T10:51:49.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:49 smithi097 ceph-mon[31859]: osdmap e69: 10 total, 9 up, 10 in 2024-03-27T10:51:49.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:49 smithi097 ceph-mon[31859]: from='osd.9 [v2:172.21.15.97:6826/1578881271,v1:172.21.15.97:6827/1578881271]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-03-27T10:51:49.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:49 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-27T10:51:49.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:49 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:51:49.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:49 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:51:49.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:49 smithi114 ceph-mon[35338]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 804 GiB / 805 GiB avail 2024-03-27T10:51:49.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:49 smithi114 ceph-mon[35338]: from='osd.9 [v2:172.21.15.97:6826/1578881271,v1:172.21.15.97:6827/1578881271]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-27T10:51:49.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:49 smithi114 ceph-mon[35338]: osdmap e69: 10 total, 9 up, 10 in 2024-03-27T10:51:49.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:49 smithi114 ceph-mon[35338]: from='osd.9 [v2:172.21.15.97:6826/1578881271,v1:172.21.15.97:6827/1578881271]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-03-27T10:51:49.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:49 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-27T10:51:49.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:49 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:51:49.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:49 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:51:49.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:49 smithi177 ceph-mon[35294]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 804 GiB / 805 GiB avail 2024-03-27T10:51:49.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:49 smithi177 ceph-mon[35294]: from='osd.9 [v2:172.21.15.97:6826/1578881271,v1:172.21.15.97:6827/1578881271]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-27T10:51:49.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:49 smithi177 ceph-mon[35294]: osdmap e69: 10 total, 9 up, 10 in 2024-03-27T10:51:49.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:49 smithi177 ceph-mon[35294]: from='osd.9 [v2:172.21.15.97:6826/1578881271,v1:172.21.15.97:6827/1578881271]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-03-27T10:51:49.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:49 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-27T10:51:49.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:49 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:51:49.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:49 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:51:50.058 DEBUG:teuthology.orchestra.run.smithi097:osd.9> sudo journalctl -f -n 0 -u ceph-adca88c4-ec26-11ee-95d0-87774f69a715@osd.9.service 2024-03-27T10:51:50.064 INFO:tasks.cephadm:Deploying osd.10 on smithi114 with /dev/vg_nvme/lv_1... 2024-03-27T10:51:50.064 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid adca88c4-ec26-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-27T10:51:50.261 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 10:51:50 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T10:51:50.274+0000 7fddca600700 -1 osd.9 0 waiting for initial osdmap 2024-03-27T10:51:50.506 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/mon.b/config 2024-03-27T10:51:50.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:50 smithi114 ceph-mon[35338]: from='osd.9 [v2:172.21.15.97:6826/1578881271,v1:172.21.15.97:6827/1578881271]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-03-27T10:51:50.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:50 smithi114 ceph-mon[35338]: osdmap e70: 10 total, 9 up, 10 in 2024-03-27T10:51:50.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:50 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-27T10:51:50.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:50 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-27T10:51:50.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:50 smithi114 ceph-mon[35338]: from='osd.9 [v2:172.21.15.97:6826/1578881271,v1:172.21.15.97:6827/1578881271]' entity='osd.9' 2024-03-27T10:51:50.563 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:50 smithi097 ceph-mon[31859]: from='osd.9 [v2:172.21.15.97:6826/1578881271,v1:172.21.15.97:6827/1578881271]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-03-27T10:51:50.563 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:50 smithi097 ceph-mon[31859]: osdmap e70: 10 total, 9 up, 10 in 2024-03-27T10:51:50.563 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:50 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-27T10:51:50.563 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:50 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-27T10:51:50.563 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:50 smithi097 ceph-mon[31859]: from='osd.9 [v2:172.21.15.97:6826/1578881271,v1:172.21.15.97:6827/1578881271]' entity='osd.9' 2024-03-27T10:51:50.563 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 10:51:50 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T10:51:50.357+0000 7fddc0400700 -1 osd.9 70 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-27T10:51:50.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:50 smithi177 ceph-mon[35294]: from='osd.9 [v2:172.21.15.97:6826/1578881271,v1:172.21.15.97:6827/1578881271]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-03-27T10:51:50.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:50 smithi177 ceph-mon[35294]: osdmap e70: 10 total, 9 up, 10 in 2024-03-27T10:51:50.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:50 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-27T10:51:50.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:50 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-27T10:51:50.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:50 smithi177 ceph-mon[35294]: from='osd.9 [v2:172.21.15.97:6826/1578881271,v1:172.21.15.97:6827/1578881271]' entity='osd.9' 2024-03-27T10:51:51.488 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:51 smithi097 ceph-mon[31859]: purged_snaps scrub starts 2024-03-27T10:51:51.488 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:51 smithi097 ceph-mon[31859]: purged_snaps scrub ok 2024-03-27T10:51:51.488 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:51 smithi097 ceph-mon[31859]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 804 GiB / 805 GiB avail 2024-03-27T10:51:51.489 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:51 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-27T10:51:51.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:51 smithi114 ceph-mon[35338]: purged_snaps scrub starts 2024-03-27T10:51:51.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:51 smithi114 ceph-mon[35338]: purged_snaps scrub ok 2024-03-27T10:51:51.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:51 smithi114 ceph-mon[35338]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 804 GiB / 805 GiB avail 2024-03-27T10:51:51.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:51 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-27T10:51:51.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:51 smithi177 ceph-mon[35294]: purged_snaps scrub starts 2024-03-27T10:51:51.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:51 smithi177 ceph-mon[35294]: purged_snaps scrub ok 2024-03-27T10:51:51.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:51 smithi177 ceph-mon[35294]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 804 GiB / 805 GiB avail 2024-03-27T10:51:51.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:51 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-27T10:51:52.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:52 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-27T10:51:52.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:52 smithi097 ceph-mon[31859]: osd.9 [v2:172.21.15.97:6826/1578881271,v1:172.21.15.97:6827/1578881271] boot 2024-03-27T10:51:52.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:52 smithi097 ceph-mon[31859]: osdmap e71: 10 total, 10 up, 10 in 2024-03-27T10:51:52.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:52 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-27T10:51:52.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:52 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-27T10:51:52.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:52 smithi114 ceph-mon[35338]: osd.9 [v2:172.21.15.97:6826/1578881271,v1:172.21.15.97:6827/1578881271] boot 2024-03-27T10:51:52.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:52 smithi114 ceph-mon[35338]: osdmap e71: 10 total, 10 up, 10 in 2024-03-27T10:51:52.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:52 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-27T10:51:52.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:52 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-27T10:51:52.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:52 smithi177 ceph-mon[35294]: osd.9 [v2:172.21.15.97:6826/1578881271,v1:172.21.15.97:6827/1578881271] boot 2024-03-27T10:51:52.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:52 smithi177 ceph-mon[35294]: osdmap e71: 10 total, 10 up, 10 in 2024-03-27T10:51:52.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:52 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-27T10:51:53.414 INFO:teuthology.orchestra.run.smithi114.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-27T10:51:53.415 INFO:teuthology.orchestra.run.smithi114.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-27T10:51:53.415 INFO:teuthology.orchestra.run.smithi114.stderr: stderr: 10+0 records in 2024-03-27T10:51:53.415 INFO:teuthology.orchestra.run.smithi114.stderr:10+0 records out 2024-03-27T10:51:53.415 INFO:teuthology.orchestra.run.smithi114.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0359317 s, 292 MB/s 2024-03-27T10:51:53.415 INFO:teuthology.orchestra.run.smithi114.stderr:--> Zapping successful for: 2024-03-27T10:51:53.579 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:53 smithi177 ceph-mon[35294]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 804 GiB / 805 GiB avail 2024-03-27T10:51:53.579 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:53 smithi177 ceph-mon[35294]: osdmap e72: 10 total, 10 up, 10 in 2024-03-27T10:51:53.641 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:53 smithi097 ceph-mon[31859]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 804 GiB / 805 GiB avail 2024-03-27T10:51:53.642 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:53 smithi097 ceph-mon[31859]: osdmap e72: 10 total, 10 up, 10 in 2024-03-27T10:51:53.754 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:53 smithi114 ceph-mon[35338]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 804 GiB / 805 GiB avail 2024-03-27T10:51:53.754 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:53 smithi114 ceph-mon[35338]: osdmap e72: 10 total, 10 up, 10 in 2024-03-27T10:51:54.268 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid adca88c4-ec26-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi114:vg_nvme/lv_1 2024-03-27T10:51:54.695 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/mon.b/config 2024-03-27T10:51:55.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:55 smithi114 ceph-mon[35338]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-27T10:51:55.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:55 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:51:55.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:55 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:51:55.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:55 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-27T10:51:55.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:55 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-27T10:51:55.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:55 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-27T10:51:55.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:55 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-27T10:51:55.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:55 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:51:55.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:55 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:51:55.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:55 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:51:55.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:55 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:51:55.642 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:55 smithi097 ceph-mon[31859]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-27T10:51:55.642 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:55 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:51:55.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:55 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:51:55.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:55 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-27T10:51:55.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:55 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-27T10:51:55.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:55 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-27T10:51:55.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:55 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-27T10:51:55.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:55 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:51:55.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:55 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:51:55.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:55 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:51:55.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:55 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:51:55.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:55 smithi177 ceph-mon[35294]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-27T10:51:55.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:55 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:51:55.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:55 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:51:55.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:55 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-27T10:51:55.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:55 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-27T10:51:55.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:55 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-27T10:51:55.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:55 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-27T10:51:55.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:55 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:51:55.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:55 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:51:55.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:55 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:51:55.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:55 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:51:56.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:56 smithi114 ceph-mon[35338]: Detected new or changed devices on smithi097 2024-03-27T10:51:56.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:56 smithi114 ceph-mon[35338]: Adjusting osd_memory_target on smithi097 to 4272M 2024-03-27T10:51:56.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:56 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:51:56.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:56 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:51:56.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:56 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:51:56.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:56 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:51:56.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:56 smithi114 ceph-mon[35338]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-27T10:51:56.642 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:56 smithi097 ceph-mon[31859]: Detected new or changed devices on smithi097 2024-03-27T10:51:56.642 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:56 smithi097 ceph-mon[31859]: Adjusting osd_memory_target on smithi097 to 4272M 2024-03-27T10:51:56.642 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:56 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:51:56.642 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:56 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:51:56.642 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:56 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:51:56.642 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:56 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:51:56.642 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:56 smithi097 ceph-mon[31859]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-27T10:51:56.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:56 smithi177 ceph-mon[35294]: Detected new or changed devices on smithi097 2024-03-27T10:51:56.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:56 smithi177 ceph-mon[35294]: Adjusting osd_memory_target on smithi097 to 4272M 2024-03-27T10:51:56.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:56 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:51:56.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:56 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:51:56.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:56 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:51:56.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:56 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:51:56.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:56 smithi177 ceph-mon[35294]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-27T10:51:57.023 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:51:57.049+0000 7f8310200700 1 -- 172.21.15.114:0/2101058008 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8308104210 msgr2=0x7f830810aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:51:57.023 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:51:57.049+0000 7f8310200700 1 --2- 172.21.15.114:0/2101058008 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8308104210 0x7f830810aac0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f830000b0d0 tx=0x7f830000b3e0 comp rx=0 tx=0).stop 2024-03-27T10:51:57.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:51:57.049+0000 7f8310200700 1 -- 172.21.15.114:0/2101058008 shutdown_connections 2024-03-27T10:51:57.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:51:57.049+0000 7f8310200700 1 --2- 172.21.15.114:0/2101058008 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8308104210 0x7f830810aac0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:51:57.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:51:57.049+0000 7f8310200700 1 --2- 172.21.15.114:0/2101058008 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8308102670 0x7f8308102a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:51:57.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:51:57.049+0000 7f8310200700 1 --2- 172.21.15.114:0/2101058008 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8308103870 0x7f8308103cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:51:57.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:51:57.049+0000 7f8310200700 1 -- 172.21.15.114:0/2101058008 >> 172.21.15.114:0/2101058008 conn(0x7f83080fdc50 msgr2=0x7f8308100070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:51:57.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:51:57.049+0000 7f8310200700 1 -- 172.21.15.114:0/2101058008 shutdown_connections 2024-03-27T10:51:57.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:51:57.049+0000 7f8310200700 1 -- 172.21.15.114:0/2101058008 wait complete. 2024-03-27T10:51:57.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:51:57.050+0000 7f8310200700 1 Processor -- start 2024-03-27T10:51:57.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:51:57.051+0000 7f8310200700 1 -- start start 2024-03-27T10:51:57.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:51:57.051+0000 7f8310200700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8308102670 0x7f830819a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:51:57.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:51:57.051+0000 7f8310200700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8308103870 0x7f830819aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:51:57.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:51:57.051+0000 7f8310200700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8308104210 0x7f83081a1f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:51:57.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:51:57.051+0000 7f8310200700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f830810e110 con 0x7f8308104210 2024-03-27T10:51:57.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:51:57.051+0000 7f8310200700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f830810df90 con 0x7f8308103870 2024-03-27T10:51:57.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:51:57.051+0000 7f8310200700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f830810e290 con 0x7f8308102670 2024-03-27T10:51:57.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:51:57.051+0000 7f830ce00700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8308103870 0x7f830819aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:51:57.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:51:57.051+0000 7f830ce00700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8308103870 0x7f830819aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:45308/0 (socket says 172.21.15.114:45308) 2024-03-27T10:51:57.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:51:57.051+0000 7f830ce00700 1 -- 172.21.15.114:0/3235871731 learned_addr learned my addr 172.21.15.114:0/3235871731 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-03-27T10:51:57.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:51:57.051+0000 7f830d800700 1 --2- 172.21.15.114:0/3235871731 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8308102670 0x7f830819a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:51:57.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:51:57.052+0000 7f830ce00700 1 -- 172.21.15.114:0/3235871731 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8308102670 msgr2=0x7f830819a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:51:57.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:51:57.052+0000 7f830ce00700 1 --2- 172.21.15.114:0/3235871731 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8308102670 0x7f830819a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:51:57.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:51:57.052+0000 7f830e200700 1 --2- 172.21.15.114:0/3235871731 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8308104210 0x7f83081a1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:51:57.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:51:57.052+0000 7f830ce00700 1 -- 172.21.15.114:0/3235871731 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8308104210 msgr2=0x7f83081a1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:51:57.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:51:57.052+0000 7f830ce00700 1 --2- 172.21.15.114:0/3235871731 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8308104210 0x7f83081a1f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:51:57.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:51:57.052+0000 7f830ce00700 1 -- 172.21.15.114:0/3235871731 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83081a2610 con 0x7f8308103870 2024-03-27T10:51:57.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:51:57.052+0000 7f830ce00700 1 --2- 172.21.15.114:0/3235871731 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8308103870 0x7f830819aef0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f82f400e9c0 tx=0x7f82f400ed80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:51:57.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:51:57.052+0000 7f8306000700 1 -- 172.21.15.114:0/3235871731 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82f400cbd0 con 0x7f8308103870 2024-03-27T10:51:57.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:51:57.052+0000 7f8306000700 1 -- 172.21.15.114:0/3235871731 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f82f400cd70 con 0x7f8308103870 2024-03-27T10:51:57.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:51:57.052+0000 7f8306000700 1 -- 172.21.15.114:0/3235871731 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82f4010430 con 0x7f8308103870 2024-03-27T10:51:57.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:51:57.052+0000 7f8310200700 1 -- 172.21.15.114:0/3235871731 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83081a2900 con 0x7f8308103870 2024-03-27T10:51:57.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:51:57.052+0000 7f8310200700 1 -- 172.21.15.114:0/3235871731 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83081a2e40 con 0x7f8308103870 2024-03-27T10:51:57.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:51:57.054+0000 7f8310200700 1 -- 172.21.15.114:0/3235871731 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8308069300 con 0x7f8308103870 2024-03-27T10:51:57.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:51:57.056+0000 7f8306000700 1 -- 172.21.15.114:0/3235871731 <== mon.2 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 16) ==== 138370+0+0 (secure 0 0 0) 0x7f82f4004750 con 0x7f8308103870 2024-03-27T10:51:57.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:51:57.057+0000 7f8306000700 1 --2- 172.21.15.114:0/3235871731 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f82f00a3720 0x7f82f00a5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:51:57.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:51:57.057+0000 7f8306000700 1 -- 172.21.15.114:0/3235871731 <== mon.2 v2:172.21.15.114:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 6261+0+0 (secure 0 0 0) 0x7f82f40cf960 con 0x7f8308103870 2024-03-27T10:51:57.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:51:57.058+0000 7f830d800700 1 --2- 172.21.15.114:0/3235871731 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f82f00a3720 0x7f82f00a5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:51:57.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:51:57.059+0000 7f830d800700 1 --2- 172.21.15.114:0/3235871731 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f82f00a3720 0x7f82f00a5be0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f83081036d0 tx=0x7f82fc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:51:57.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:51:57.061+0000 7f8306000700 1 -- 172.21.15.114:0/3235871731 <== mon.2 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f82f409cc90 con 0x7f8308103870 2024-03-27T10:51:57.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:51:57.295+0000 7f8310200700 1 -- 172.21.15.114:0/3235871731 --> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi114:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f8308075f90 con 0x7f82f00a3720 2024-03-27T10:51:57.768 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:57 smithi114 ceph-mon[35338]: from='client.34328 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi114:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T10:51:57.768 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:57 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-27T10:51:57.768 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:57 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-27T10:51:57.768 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:57 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:51:57.891 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:57 smithi097 ceph-mon[31859]: from='client.34328 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi114:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T10:51:57.892 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:57 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-27T10:51:57.892 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:57 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-27T10:51:57.892 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:57 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:51:57.927 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:57 smithi177 ceph-mon[35294]: from='client.34328 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi114:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T10:51:57.927 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:57 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-27T10:51:57.928 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:57 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-27T10:51:57.928 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:57 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:51:58.768 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:51:58 smithi114 ceph-mon[35338]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-27T10:51:58.891 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:51:58 smithi097 ceph-mon[31859]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-27T10:51:58.927 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:51:58 smithi177 ceph-mon[35294]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-27T10:52:01.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:01 smithi097 ceph-mon[31859]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-27T10:52:01.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:01 smithi097 ceph-mon[31859]: from='client.? 172.21.15.114:0/2366902215' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "64754e31-b124-4601-ad98-2f70b1dd5359"}]: dispatch 2024-03-27T10:52:01.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:01 smithi097 ceph-mon[31859]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "64754e31-b124-4601-ad98-2f70b1dd5359"}]: dispatch 2024-03-27T10:52:01.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:01 smithi097 ceph-mon[31859]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "64754e31-b124-4601-ad98-2f70b1dd5359"}]': finished 2024-03-27T10:52:01.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:01 smithi177 ceph-mon[35294]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-27T10:52:01.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:01 smithi177 ceph-mon[35294]: from='client.? 172.21.15.114:0/2366902215' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "64754e31-b124-4601-ad98-2f70b1dd5359"}]: dispatch 2024-03-27T10:52:01.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:01 smithi177 ceph-mon[35294]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "64754e31-b124-4601-ad98-2f70b1dd5359"}]: dispatch 2024-03-27T10:52:01.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:01 smithi177 ceph-mon[35294]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "64754e31-b124-4601-ad98-2f70b1dd5359"}]': finished 2024-03-27T10:52:01.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:01 smithi114 ceph-mon[35338]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-27T10:52:01.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:01 smithi114 ceph-mon[35338]: from='client.? 172.21.15.114:0/2366902215' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "64754e31-b124-4601-ad98-2f70b1dd5359"}]: dispatch 2024-03-27T10:52:01.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:01 smithi114 ceph-mon[35338]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "64754e31-b124-4601-ad98-2f70b1dd5359"}]: dispatch 2024-03-27T10:52:01.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:01 smithi114 ceph-mon[35338]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "64754e31-b124-4601-ad98-2f70b1dd5359"}]': finished 2024-03-27T10:52:02.337 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:02 smithi097 ceph-mon[31859]: osdmap e73: 11 total, 10 up, 11 in 2024-03-27T10:52:02.338 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:02 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-27T10:52:02.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:02 smithi177 ceph-mon[35294]: osdmap e73: 11 total, 10 up, 11 in 2024-03-27T10:52:02.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:02 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-27T10:52:02.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:02 smithi114 ceph-mon[35338]: osdmap e73: 11 total, 10 up, 11 in 2024-03-27T10:52:02.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:02 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-27T10:52:03.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:03 smithi097 ceph-mon[31859]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-27T10:52:03.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:03 smithi097 ceph-mon[31859]: from='client.? 172.21.15.114:0/496735121' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-27T10:52:03.420 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:03 smithi114 ceph-mon[35338]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-27T10:52:03.420 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:03 smithi114 ceph-mon[35338]: from='client.? 172.21.15.114:0/496735121' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-27T10:52:03.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:03 smithi177 ceph-mon[35294]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-27T10:52:03.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:03 smithi177 ceph-mon[35294]: from='client.? 172.21.15.114:0/496735121' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-27T10:52:05.381 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:05 smithi114 ceph-mon[35338]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-27T10:52:05.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:05 smithi097 ceph-mon[31859]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-27T10:52:05.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:05 smithi177 ceph-mon[35294]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-27T10:52:07.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:07 smithi097 ceph-mon[31859]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-27T10:52:07.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:07 smithi177 ceph-mon[35294]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-27T10:52:07.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:07 smithi114 ceph-mon[35338]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-27T10:52:09.358 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:09 smithi114 ceph-mon[35338]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-27T10:52:09.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:09 smithi097 ceph-mon[31859]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-27T10:52:09.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:09 smithi177 ceph-mon[35294]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-27T10:52:11.267 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:11 smithi114 ceph-mon[35338]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-27T10:52:11.268 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:11 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-27T10:52:11.268 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:11 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:52:11.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:11 smithi097 ceph-mon[31859]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-27T10:52:11.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:11 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-27T10:52:11.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:11 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:52:11.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:11 smithi177 ceph-mon[35294]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-27T10:52:11.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:11 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-27T10:52:11.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:11 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:52:12.267 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:12 smithi114 ceph-mon[35338]: Deploying daemon osd.10 on smithi114 2024-03-27T10:52:12.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:12 smithi097 ceph-mon[31859]: Deploying daemon osd.10 on smithi114 2024-03-27T10:52:12.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:12 smithi177 ceph-mon[35294]: Deploying daemon osd.10 on smithi114 2024-03-27T10:52:13.268 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:13 smithi114 ceph-mon[35338]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-27T10:52:13.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:13 smithi097 ceph-mon[31859]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-27T10:52:13.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:13 smithi177 ceph-mon[35294]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-27T10:52:15.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:15 smithi097 ceph-mon[31859]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-27T10:52:15.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:15 smithi177 ceph-mon[35294]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-27T10:52:15.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:15 smithi114 ceph-mon[35338]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-27T10:52:17.268 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:17 smithi114 ceph-mon[35338]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-27T10:52:17.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:17 smithi097 ceph-mon[31859]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-27T10:52:17.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:17 smithi177 ceph-mon[35294]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-27T10:52:19.357 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:19 smithi114 ceph-mon[35338]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-27T10:52:19.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:19 smithi097 ceph-mon[31859]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-27T10:52:19.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:19 smithi177 ceph-mon[35294]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-27T10:52:20.927 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:20 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:52:20.927 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:20 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:52:20.927 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:20 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:52:20.928 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:20 smithi177 ceph-mon[35294]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-27T10:52:20.936 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:20 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:52:20.936 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:20 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:52:20.936 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:20 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:52:20.936 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:20 smithi114 ceph-mon[35338]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-27T10:52:21.141 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:20 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:52:21.142 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:20 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:52:21.142 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:20 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:52:21.142 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:20 smithi097 ceph-mon[31859]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-27T10:52:23.203 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:23 smithi114 ceph-mon[35338]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-27T10:52:23.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:23 smithi097 ceph-mon[31859]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-27T10:52:23.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:23 smithi177 ceph-mon[35294]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-27T10:52:24.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:24 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:52:24.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:24 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:52:24.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:24 smithi114 ceph-mon[35338]: from='osd.10 [v2:172.21.15.114:6824/2893111461,v1:172.21.15.114:6825/2893111461]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-27T10:52:24.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:24 smithi114 ceph-mon[35338]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-27T10:52:24.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:24 smithi114 ceph-mon[35338]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-27T10:52:24.641 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:24 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:52:24.642 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:24 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:52:24.642 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:24 smithi097 ceph-mon[31859]: from='osd.10 [v2:172.21.15.114:6824/2893111461,v1:172.21.15.114:6825/2893111461]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-27T10:52:24.642 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:24 smithi097 ceph-mon[31859]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-27T10:52:24.642 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:24 smithi097 ceph-mon[31859]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-27T10:52:24.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:24 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:52:24.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:24 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:52:24.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:24 smithi177 ceph-mon[35294]: from='osd.10 [v2:172.21.15.114:6824/2893111461,v1:172.21.15.114:6825/2893111461]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-27T10:52:24.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:24 smithi177 ceph-mon[35294]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-27T10:52:24.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:24 smithi177 ceph-mon[35294]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-27T10:52:24.934 INFO:teuthology.orchestra.run.smithi114.stdout:Created osd(s) 10 on host 'smithi114' 2024-03-27T10:52:24.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:52:24.961+0000 7f8306000700 1 -- 172.21.15.114:0/3235871731 <== mgr.14152 v2:172.21.15.97:6800/3955463538 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f8308075f90 con 0x7f82f00a3720 2024-03-27T10:52:24.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:52:24.965+0000 7f8310200700 1 -- 172.21.15.114:0/3235871731 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f82f00a3720 msgr2=0x7f82f00a5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:52:24.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:52:24.965+0000 7f8310200700 1 --2- 172.21.15.114:0/3235871731 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f82f00a3720 0x7f82f00a5be0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f83081036d0 tx=0x7f82fc009260 comp rx=0 tx=0).stop 2024-03-27T10:52:24.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:52:24.965+0000 7f8310200700 1 -- 172.21.15.114:0/3235871731 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8308103870 msgr2=0x7f830819aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:52:24.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:52:24.965+0000 7f8310200700 1 --2- 172.21.15.114:0/3235871731 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8308103870 0x7f830819aef0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f82f400e9c0 tx=0x7f82f400ed80 comp rx=0 tx=0).stop 2024-03-27T10:52:24.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:52:24.965+0000 7f8310200700 1 -- 172.21.15.114:0/3235871731 shutdown_connections 2024-03-27T10:52:24.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:52:24.965+0000 7f8310200700 1 --2- 172.21.15.114:0/3235871731 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f82f00a3720 0x7f82f00a5be0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:52:24.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:52:24.965+0000 7f8310200700 1 --2- 172.21.15.114:0/3235871731 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8308104210 0x7f83081a1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:52:24.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:52:24.965+0000 7f8310200700 1 --2- 172.21.15.114:0/3235871731 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8308102670 0x7f830819a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:52:24.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:52:24.965+0000 7f8310200700 1 --2- 172.21.15.114:0/3235871731 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8308103870 0x7f830819aef0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:52:24.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:52:24.965+0000 7f8310200700 1 -- 172.21.15.114:0/3235871731 >> 172.21.15.114:0/3235871731 conn(0x7f83080fdc50 msgr2=0x7f8308074870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:52:24.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:52:24.965+0000 7f8310200700 1 -- 172.21.15.114:0/3235871731 shutdown_connections 2024-03-27T10:52:24.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-03-27T10:52:24.965+0000 7f8310200700 1 -- 172.21.15.114:0/3235871731 wait complete. 2024-03-27T10:52:25.642 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:25 smithi097 ceph-mon[31859]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-27T10:52:25.642 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:25 smithi097 ceph-mon[31859]: osdmap e74: 11 total, 10 up, 11 in 2024-03-27T10:52:25.642 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:25 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-27T10:52:25.642 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:25 smithi097 ceph-mon[31859]: from='osd.10 [v2:172.21.15.114:6824/2893111461,v1:172.21.15.114:6825/2893111461]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]: dispatch 2024-03-27T10:52:25.642 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:25 smithi097 ceph-mon[31859]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]: dispatch 2024-03-27T10:52:25.642 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:25 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:52:25.642 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:25 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:52:25.642 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:25 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:52:25.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:25 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:52:25.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:25 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:52:25.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:25 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:52:25.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:25 smithi177 ceph-mon[35294]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-27T10:52:25.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:25 smithi177 ceph-mon[35294]: osdmap e74: 11 total, 10 up, 11 in 2024-03-27T10:52:25.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:25 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-27T10:52:25.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:25 smithi177 ceph-mon[35294]: from='osd.10 [v2:172.21.15.114:6824/2893111461,v1:172.21.15.114:6825/2893111461]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]: dispatch 2024-03-27T10:52:25.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:25 smithi177 ceph-mon[35294]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]: dispatch 2024-03-27T10:52:25.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:25 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:52:25.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:25 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:52:25.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:25 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:52:25.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:25 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:52:25.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:25 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:52:25.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:25 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:52:25.729 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:25 smithi114 ceph-mon[35338]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-27T10:52:25.729 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:25 smithi114 ceph-mon[35338]: osdmap e74: 11 total, 10 up, 11 in 2024-03-27T10:52:25.729 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:25 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-27T10:52:25.729 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:25 smithi114 ceph-mon[35338]: from='osd.10 [v2:172.21.15.114:6824/2893111461,v1:172.21.15.114:6825/2893111461]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]: dispatch 2024-03-27T10:52:25.729 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:25 smithi114 ceph-mon[35338]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]: dispatch 2024-03-27T10:52:25.730 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:25 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:52:25.730 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:25 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:52:25.730 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:25 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:52:25.730 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:25 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:52:25.730 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:25 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:52:25.730 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:25 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:52:25.779 DEBUG:teuthology.orchestra.run.smithi114:osd.10> sudo journalctl -f -n 0 -u ceph-adca88c4-ec26-11ee-95d0-87774f69a715@osd.10.service 2024-03-27T10:52:25.785 INFO:tasks.cephadm:Deploying osd.11 on smithi177 with /dev/vg_nvme/lv_1... 2024-03-27T10:52:25.785 DEBUG:teuthology.orchestra.run.smithi177:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid adca88c4-ec26-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-27T10:52:26.229 INFO:teuthology.orchestra.run.smithi177.stderr:Inferring config /var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/mon.c/config 2024-03-27T10:52:26.641 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:26 smithi097 ceph-mon[31859]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]': finished 2024-03-27T10:52:26.642 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:26 smithi097 ceph-mon[31859]: osdmap e75: 11 total, 10 up, 11 in 2024-03-27T10:52:26.642 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:26 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-27T10:52:26.642 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:26 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-27T10:52:26.642 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:26 smithi097 ceph-mon[31859]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-27T10:52:26.642 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:26 smithi097 ceph-mon[31859]: from='osd.10 ' entity='osd.10' 2024-03-27T10:52:26.651 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:26 smithi177 ceph-mon[35294]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]': finished 2024-03-27T10:52:26.651 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:26 smithi177 ceph-mon[35294]: osdmap e75: 11 total, 10 up, 11 in 2024-03-27T10:52:26.651 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:26 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-27T10:52:26.651 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:26 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-27T10:52:26.651 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:26 smithi177 ceph-mon[35294]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-27T10:52:26.651 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:26 smithi177 ceph-mon[35294]: from='osd.10 ' entity='osd.10' 2024-03-27T10:52:26.768 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:26 smithi114 ceph-mon[35338]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]': finished 2024-03-27T10:52:26.768 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:26 smithi114 ceph-mon[35338]: osdmap e75: 11 total, 10 up, 11 in 2024-03-27T10:52:26.768 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:26 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-27T10:52:26.768 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:26 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-27T10:52:26.768 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:26 smithi114 ceph-mon[35338]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-27T10:52:26.768 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:26 smithi114 ceph-mon[35338]: from='osd.10 ' entity='osd.10' 2024-03-27T10:52:26.769 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 10:52:26 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T10:52:26.351+0000 7f831ec00700 -1 osd.10 0 waiting for initial osdmap 2024-03-27T10:52:26.769 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 10:52:26 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T10:52:26.425+0000 7f8314000700 -1 osd.10 76 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-27T10:52:27.464 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:27 smithi114 ceph-mon[35338]: purged_snaps scrub starts 2024-03-27T10:52:27.465 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:27 smithi114 ceph-mon[35338]: purged_snaps scrub ok 2024-03-27T10:52:27.465 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:27 smithi114 ceph-mon[35338]: osdmap e76: 11 total, 10 up, 11 in 2024-03-27T10:52:27.465 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:27 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-27T10:52:27.641 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:27 smithi097 ceph-mon[31859]: purged_snaps scrub starts 2024-03-27T10:52:27.641 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:27 smithi097 ceph-mon[31859]: purged_snaps scrub ok 2024-03-27T10:52:27.642 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:27 smithi097 ceph-mon[31859]: osdmap e76: 11 total, 10 up, 11 in 2024-03-27T10:52:27.642 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:27 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-27T10:52:27.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:27 smithi177 ceph-mon[35294]: purged_snaps scrub starts 2024-03-27T10:52:27.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:27 smithi177 ceph-mon[35294]: purged_snaps scrub ok 2024-03-27T10:52:27.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:27 smithi177 ceph-mon[35294]: osdmap e76: 11 total, 10 up, 11 in 2024-03-27T10:52:27.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:27 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-27T10:52:28.642 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:28 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-27T10:52:28.642 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:28 smithi097 ceph-mon[31859]: osd.10 [v2:172.21.15.114:6824/2893111461,v1:172.21.15.114:6825/2893111461] boot 2024-03-27T10:52:28.642 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:28 smithi097 ceph-mon[31859]: osdmap e77: 11 total, 11 up, 11 in 2024-03-27T10:52:28.642 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:28 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-27T10:52:28.642 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:28 smithi097 ceph-mon[31859]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-27T10:52:28.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:28 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-27T10:52:28.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:28 smithi177 ceph-mon[35294]: osd.10 [v2:172.21.15.114:6824/2893111461,v1:172.21.15.114:6825/2893111461] boot 2024-03-27T10:52:28.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:28 smithi177 ceph-mon[35294]: osdmap e77: 11 total, 11 up, 11 in 2024-03-27T10:52:28.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:28 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-27T10:52:28.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:28 smithi177 ceph-mon[35294]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-27T10:52:28.767 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:28 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-27T10:52:28.768 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:28 smithi114 ceph-mon[35338]: osd.10 [v2:172.21.15.114:6824/2893111461,v1:172.21.15.114:6825/2893111461] boot 2024-03-27T10:52:28.768 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:28 smithi114 ceph-mon[35338]: osdmap e77: 11 total, 11 up, 11 in 2024-03-27T10:52:28.768 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:28 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-27T10:52:28.768 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:28 smithi114 ceph-mon[35338]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-03-27T10:52:28.934 INFO:teuthology.orchestra.run.smithi177.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-27T10:52:28.934 INFO:teuthology.orchestra.run.smithi177.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-27T10:52:28.934 INFO:teuthology.orchestra.run.smithi177.stderr: stderr: 10+0 records in 2024-03-27T10:52:28.934 INFO:teuthology.orchestra.run.smithi177.stderr:10+0 records out 2024-03-27T10:52:28.934 INFO:teuthology.orchestra.run.smithi177.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0352004 s, 298 MB/s 2024-03-27T10:52:28.935 INFO:teuthology.orchestra.run.smithi177.stderr:--> Zapping successful for: 2024-03-27T10:52:29.620 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:29 smithi177 ceph-mon[35294]: osdmap e78: 11 total, 11 up, 11 in 2024-03-27T10:52:29.641 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:29 smithi097 ceph-mon[31859]: osdmap e78: 11 total, 11 up, 11 in 2024-03-27T10:52:29.739 DEBUG:teuthology.orchestra.run.smithi177:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid adca88c4-ec26-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi177:vg_nvme/lv_1 2024-03-27T10:52:29.767 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:29 smithi114 ceph-mon[35338]: osdmap e78: 11 total, 11 up, 11 in 2024-03-27T10:52:30.158 INFO:teuthology.orchestra.run.smithi177.stderr:Inferring config /var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/mon.c/config 2024-03-27T10:52:30.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:30 smithi114 ceph-mon[35338]: osdmap e79: 11 total, 11 up, 11 in 2024-03-27T10:52:30.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:30 smithi114 ceph-mon[35338]: pgmap v304: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2024-03-27T10:52:30.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:30 smithi114 ceph-mon[35338]: osdmap e80: 11 total, 11 up, 11 in 2024-03-27T10:52:30.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:30 smithi114 ceph-mon[35338]: Detected new or changed devices on smithi114 2024-03-27T10:52:30.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:30 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:52:30.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:30 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:52:30.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:30 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-27T10:52:30.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:30 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-27T10:52:30.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:30 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-27T10:52:30.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:30 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-27T10:52:30.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:30 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:52:30.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:30 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:52:30.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:30 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:52:30.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:30 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:52:30.548 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:30 smithi177 ceph-mon[35294]: osdmap e79: 11 total, 11 up, 11 in 2024-03-27T10:52:30.548 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:30 smithi177 ceph-mon[35294]: pgmap v304: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2024-03-27T10:52:30.548 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:30 smithi177 ceph-mon[35294]: osdmap e80: 11 total, 11 up, 11 in 2024-03-27T10:52:30.548 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:30 smithi177 ceph-mon[35294]: Detected new or changed devices on smithi114 2024-03-27T10:52:30.548 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:30 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:52:30.548 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:30 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:52:30.548 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:30 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-27T10:52:30.548 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:30 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-27T10:52:30.549 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:30 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-27T10:52:30.549 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:30 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-27T10:52:30.549 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:30 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:52:30.549 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:30 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:52:30.549 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:30 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:52:30.549 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:30 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:52:30.892 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:30 smithi097 ceph-mon[31859]: osdmap e79: 11 total, 11 up, 11 in 2024-03-27T10:52:30.892 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:30 smithi097 ceph-mon[31859]: pgmap v304: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2024-03-27T10:52:30.892 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:30 smithi097 ceph-mon[31859]: osdmap e80: 11 total, 11 up, 11 in 2024-03-27T10:52:30.892 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:30 smithi097 ceph-mon[31859]: Detected new or changed devices on smithi114 2024-03-27T10:52:30.892 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:30 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:52:30.892 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:30 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:52:30.892 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:30 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-27T10:52:30.892 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:30 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-27T10:52:30.892 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:30 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-27T10:52:30.893 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:30 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-27T10:52:30.893 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:30 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:52:30.893 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:30 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:52:30.893 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:30 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:52:30.893 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:30 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:52:31.672 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:31 smithi177 ceph-mon[35294]: Adjusting osd_memory_target on smithi114 to 4272M 2024-03-27T10:52:31.672 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:31 smithi177 ceph-mon[35294]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-03-27T10:52:31.672 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:31 smithi177 ceph-mon[35294]: osdmap e81: 11 total, 11 up, 11 in 2024-03-27T10:52:31.767 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:31 smithi114 ceph-mon[35338]: Adjusting osd_memory_target on smithi114 to 4272M 2024-03-27T10:52:31.767 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:31 smithi114 ceph-mon[35338]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-03-27T10:52:31.768 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:31 smithi114 ceph-mon[35338]: osdmap e81: 11 total, 11 up, 11 in 2024-03-27T10:52:31.793 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:31 smithi097 ceph-mon[31859]: Adjusting osd_memory_target on smithi114 to 4272M 2024-03-27T10:52:31.793 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:31 smithi097 ceph-mon[31859]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-03-27T10:52:31.793 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:31 smithi097 ceph-mon[31859]: osdmap e81: 11 total, 11 up, 11 in 2024-03-27T10:52:32.097 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:52:32.124+0000 7f85a9000700 1 -- 172.21.15.177:0/1960691695 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f85a4103860 msgr2=0x7f85a4105cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:52:32.097 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:52:32.124+0000 7f85a9000700 1 --2- 172.21.15.177:0/1960691695 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f85a4103860 0x7f85a4105cf0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f858c009990 tx=0x7f858c009ca0 comp rx=0 tx=0).stop 2024-03-27T10:52:32.097 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:52:32.124+0000 7f85a9000700 1 -- 172.21.15.177:0/1960691695 shutdown_connections 2024-03-27T10:52:32.097 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:52:32.124+0000 7f85a9000700 1 --2- 172.21.15.177:0/1960691695 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85a4106230 0x7f85a41066b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:52:32.097 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:52:32.124+0000 7f85a9000700 1 --2- 172.21.15.177:0/1960691695 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f85a4100ef0 0x7f85a4103320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:52:32.097 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:52:32.124+0000 7f85a9000700 1 --2- 172.21.15.177:0/1960691695 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f85a4103860 0x7f85a4105cf0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:52:32.097 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:52:32.124+0000 7f85a9000700 1 -- 172.21.15.177:0/1960691695 >> 172.21.15.177:0/1960691695 conn(0x7f85a40fcb10 msgr2=0x7f85a40fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:52:32.098 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:52:32.125+0000 7f85a9000700 1 -- 172.21.15.177:0/1960691695 shutdown_connections 2024-03-27T10:52:32.098 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:52:32.125+0000 7f85a9000700 1 -- 172.21.15.177:0/1960691695 wait complete. 2024-03-27T10:52:32.098 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:52:32.126+0000 7f85a9000700 1 Processor -- start 2024-03-27T10:52:32.098 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:52:32.126+0000 7f85a9000700 1 -- start start 2024-03-27T10:52:32.099 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:52:32.126+0000 7f85a9000700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f85a4100ef0 0x7f85a419ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:52:32.100 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:52:32.126+0000 7f85a9000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85a4103860 0x7f85a419f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:52:32.100 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:52:32.126+0000 7f85a9000700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f85a4106230 0x7f85a41a6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:52:32.100 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:52:32.126+0000 7f85a9000700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f85a41146d0 con 0x7f85a4103860 2024-03-27T10:52:32.100 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:52:32.126+0000 7f85a9000700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f85a4114550 con 0x7f85a4106230 2024-03-27T10:52:32.100 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:52:32.126+0000 7f85a9000700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f85a4114850 con 0x7f85a4100ef0 2024-03-27T10:52:32.100 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:52:32.127+0000 7f85a1e00700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f85a4100ef0 0x7f85a419ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:52:32.100 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:52:32.127+0000 7f85a1e00700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f85a4100ef0 0x7f85a419ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:48478/0 (socket says 172.21.15.177:48478) 2024-03-27T10:52:32.100 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:52:32.127+0000 7f85a1e00700 1 -- 172.21.15.177:0/793597423 learned_addr learned my addr 172.21.15.177:0/793597423 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-03-27T10:52:32.100 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:52:32.127+0000 7f85a2800700 1 --2- 172.21.15.177:0/793597423 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f85a4106230 0x7f85a41a6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:52:32.100 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:52:32.127+0000 7f8599400700 1 --2- 172.21.15.177:0/793597423 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85a4103860 0x7f85a419f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:52:32.100 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:52:32.127+0000 7f85a1e00700 1 -- 172.21.15.177:0/793597423 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f85a4106230 msgr2=0x7f85a41a6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:52:32.100 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:52:32.127+0000 7f85a1e00700 1 --2- 172.21.15.177:0/793597423 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f85a4106230 0x7f85a41a6310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:52:32.101 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:52:32.127+0000 7f85a1e00700 1 -- 172.21.15.177:0/793597423 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85a4103860 msgr2=0x7f85a419f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:52:32.101 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:52:32.127+0000 7f85a1e00700 1 --2- 172.21.15.177:0/793597423 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85a4103860 0x7f85a419f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:52:32.101 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:52:32.127+0000 7f85a1e00700 1 -- 172.21.15.177:0/793597423 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85a41a69e0 con 0x7f85a4100ef0 2024-03-27T10:52:32.101 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:52:32.128+0000 7f85a1e00700 1 --2- 172.21.15.177:0/793597423 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f85a4100ef0 0x7f85a419ed90 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f8594007f10 tx=0x7f859400d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:52:32.101 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:52:32.128+0000 7f859b400700 1 -- 172.21.15.177:0/793597423 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f859400dc90 con 0x7f85a4100ef0 2024-03-27T10:52:32.102 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:52:32.128+0000 7f859b400700 1 -- 172.21.15.177:0/793597423 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f859400de30 con 0x7f85a4100ef0 2024-03-27T10:52:32.102 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:52:32.128+0000 7f85a9000700 1 -- 172.21.15.177:0/793597423 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85a41a6cd0 con 0x7f85a4100ef0 2024-03-27T10:52:32.102 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:52:32.128+0000 7f859b400700 1 -- 172.21.15.177:0/793597423 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85940133f0 con 0x7f85a4100ef0 2024-03-27T10:52:32.102 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:52:32.128+0000 7f85a9000700 1 -- 172.21.15.177:0/793597423 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85a41a71c0 con 0x7f85a4100ef0 2024-03-27T10:52:32.102 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:52:32.129+0000 7f85a9000700 1 -- 172.21.15.177:0/793597423 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85a4069300 con 0x7f85a4100ef0 2024-03-27T10:52:32.103 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:52:32.131+0000 7f859b400700 1 -- 172.21.15.177:0/793597423 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 16) ==== 138370+0+0 (secure 0 0 0) 0x7f859401a020 con 0x7f85a4100ef0 2024-03-27T10:52:32.105 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:52:32.132+0000 7f859b400700 1 --2- 172.21.15.177:0/793597423 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f85880a36a0 0x7f85880a5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:52:32.105 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:52:32.133+0000 7f859b400700 1 -- 172.21.15.177:0/793597423 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6694+0+0 (secure 0 0 0) 0x7f85940cefa0 con 0x7f85a4100ef0 2024-03-27T10:52:32.105 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:52:32.133+0000 7f8599400700 1 --2- 172.21.15.177:0/793597423 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f85880a36a0 0x7f85880a5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:52:32.106 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:52:32.134+0000 7f8599400700 1 --2- 172.21.15.177:0/793597423 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f85880a36a0 0x7f85880a5b60 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f858c006010 tx=0x7f858c003870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:52:32.109 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:52:32.137+0000 7f859b400700 1 -- 172.21.15.177:0/793597423 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f8594010070 con 0x7f85a4100ef0 2024-03-27T10:52:32.346 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:52:32.373+0000 7f85a9000700 1 -- 172.21.15.177:0/793597423 --> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi177:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f85a4062a80 con 0x7f85880a36a0 2024-03-27T10:52:32.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:32 smithi177 ceph-mon[35294]: pgmap v307: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2024-03-27T10:52:32.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:32 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-27T10:52:32.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:32 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-27T10:52:32.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:32 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:52:32.768 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:32 smithi114 ceph-mon[35338]: pgmap v307: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2024-03-27T10:52:32.768 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:32 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-27T10:52:32.768 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:32 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-27T10:52:32.768 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:32 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:52:32.821 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:32 smithi097 ceph-mon[31859]: pgmap v307: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2024-03-27T10:52:32.821 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:32 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-27T10:52:32.821 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:32 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-27T10:52:32.821 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:32 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:52:33.476 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:33 smithi097 ceph-mon[31859]: from='client.24346 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi177:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T10:52:33.605 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:33 smithi177 ceph-mon[35294]: from='client.24346 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi177:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T10:52:33.669 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:33 smithi114 ceph-mon[35338]: from='client.24346 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi177:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T10:52:34.641 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:34 smithi097 ceph-mon[31859]: pgmap v308: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2024-03-27T10:52:34.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:34 smithi177 ceph-mon[35294]: pgmap v308: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2024-03-27T10:52:34.768 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:34 smithi114 ceph-mon[35338]: pgmap v308: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2024-03-27T10:52:36.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:36 smithi097 ceph-mon[31859]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-03-27T10:52:36.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:36 smithi097 ceph-mon[31859]: Cluster is now healthy 2024-03-27T10:52:36.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:36 smithi177 ceph-mon[35294]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-03-27T10:52:36.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:36 smithi177 ceph-mon[35294]: Cluster is now healthy 2024-03-27T10:52:36.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:36 smithi114 ceph-mon[35338]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-03-27T10:52:36.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:36 smithi114 ceph-mon[35338]: Cluster is now healthy 2024-03-27T10:52:37.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:37 smithi097 ceph-mon[31859]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-03-27T10:52:37.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:37 smithi097 ceph-mon[31859]: from='client.? 172.21.15.177:0/2510417232' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9ffc752c-adf7-4037-89d3-535787d7475d"}]: dispatch 2024-03-27T10:52:37.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:37 smithi097 ceph-mon[31859]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9ffc752c-adf7-4037-89d3-535787d7475d"}]: dispatch 2024-03-27T10:52:37.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:37 smithi097 ceph-mon[31859]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9ffc752c-adf7-4037-89d3-535787d7475d"}]': finished 2024-03-27T10:52:37.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:37 smithi097 ceph-mon[31859]: osdmap e82: 12 total, 11 up, 12 in 2024-03-27T10:52:37.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:37 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-27T10:52:37.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:37 smithi177 ceph-mon[35294]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-03-27T10:52:37.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:37 smithi177 ceph-mon[35294]: from='client.? 172.21.15.177:0/2510417232' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9ffc752c-adf7-4037-89d3-535787d7475d"}]: dispatch 2024-03-27T10:52:37.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:37 smithi177 ceph-mon[35294]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9ffc752c-adf7-4037-89d3-535787d7475d"}]: dispatch 2024-03-27T10:52:37.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:37 smithi177 ceph-mon[35294]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9ffc752c-adf7-4037-89d3-535787d7475d"}]': finished 2024-03-27T10:52:37.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:37 smithi177 ceph-mon[35294]: osdmap e82: 12 total, 11 up, 12 in 2024-03-27T10:52:37.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:37 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-27T10:52:37.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:37 smithi114 ceph-mon[35338]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-03-27T10:52:37.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:37 smithi114 ceph-mon[35338]: from='client.? 172.21.15.177:0/2510417232' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9ffc752c-adf7-4037-89d3-535787d7475d"}]: dispatch 2024-03-27T10:52:37.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:37 smithi114 ceph-mon[35338]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9ffc752c-adf7-4037-89d3-535787d7475d"}]: dispatch 2024-03-27T10:52:37.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:37 smithi114 ceph-mon[35338]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9ffc752c-adf7-4037-89d3-535787d7475d"}]': finished 2024-03-27T10:52:37.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:37 smithi114 ceph-mon[35338]: osdmap e82: 12 total, 11 up, 12 in 2024-03-27T10:52:37.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:37 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-27T10:52:38.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:38 smithi097 ceph-mon[31859]: from='client.? 172.21.15.177:0/2159903078' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-27T10:52:38.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:38 smithi177 ceph-mon[35294]: from='client.? 172.21.15.177:0/2159903078' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-27T10:52:38.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:38 smithi114 ceph-mon[35338]: from='client.? 172.21.15.177:0/2159903078' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-27T10:52:39.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:39 smithi097 ceph-mon[31859]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-03-27T10:52:39.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:39 smithi177 ceph-mon[35294]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-03-27T10:52:39.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:39 smithi114 ceph-mon[35338]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-03-27T10:52:41.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:41 smithi097 ceph-mon[31859]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-03-27T10:52:41.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:41 smithi177 ceph-mon[35294]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-03-27T10:52:41.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:41 smithi114 ceph-mon[35338]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-03-27T10:52:43.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:43 smithi097 ceph-mon[31859]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-27T10:52:43.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:43 smithi177 ceph-mon[35294]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-27T10:52:43.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:43 smithi114 ceph-mon[35338]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-27T10:52:45.334 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:45 smithi177 ceph-mon[35294]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-27T10:52:45.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:45 smithi097 ceph-mon[31859]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-27T10:52:45.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:45 smithi114 ceph-mon[35338]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-27T10:52:46.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:46 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-27T10:52:46.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:46 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:52:46.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:46 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-27T10:52:46.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:46 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:52:46.641 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:46 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-27T10:52:46.641 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:46 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:52:47.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:47 smithi177 ceph-mon[35294]: Deploying daemon osd.11 on smithi177 2024-03-27T10:52:47.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:47 smithi177 ceph-mon[35294]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-03-27T10:52:47.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:47 smithi114 ceph-mon[35338]: Deploying daemon osd.11 on smithi177 2024-03-27T10:52:47.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:47 smithi114 ceph-mon[35338]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-03-27T10:52:47.641 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:47 smithi097 ceph-mon[31859]: Deploying daemon osd.11 on smithi177 2024-03-27T10:52:47.641 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:47 smithi097 ceph-mon[31859]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-03-27T10:52:49.314 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:49 smithi177 ceph-mon[35294]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-03-27T10:52:49.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:49 smithi114 ceph-mon[35338]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-03-27T10:52:49.641 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:49 smithi097 ceph-mon[31859]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-03-27T10:52:51.375 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:51 smithi177 ceph-mon[35294]: pgmap v317: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-03-27T10:52:51.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:51 smithi114 ceph-mon[35338]: pgmap v317: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-03-27T10:52:51.641 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:51 smithi097 ceph-mon[31859]: pgmap v317: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-03-27T10:52:53.370 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:53 smithi177 ceph-mon[35294]: pgmap v318: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-03-27T10:52:53.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:53 smithi114 ceph-mon[35338]: pgmap v318: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-03-27T10:52:53.641 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:53 smithi097 ceph-mon[31859]: pgmap v318: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-03-27T10:52:54.920 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:54 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:52:54.920 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:54 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:52:54.921 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:54 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:52:54.921 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:54 smithi177 ceph-mon[35294]: pgmap v319: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-03-27T10:52:55.141 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:54 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:52:55.142 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:54 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:52:55.142 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:54 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:52:55.142 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:54 smithi097 ceph-mon[31859]: pgmap v319: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-03-27T10:52:55.267 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:54 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:52:55.268 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:54 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:52:55.268 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:54 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:52:55.268 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:54 smithi114 ceph-mon[35338]: pgmap v319: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-03-27T10:52:57.177 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:57 smithi177 ceph-mon[35294]: pgmap v320: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-03-27T10:52:57.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:57 smithi097 ceph-mon[31859]: pgmap v320: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-03-27T10:52:57.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:57 smithi114 ceph-mon[35338]: pgmap v320: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-03-27T10:52:58.614 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:52:58.640+0000 7f859b400700 1 -- 172.21.15.177:0/793597423 <== mgr.14152 v2:172.21.15.97:6800/3955463538 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f85a4062a80 con 0x7f85880a36a0 2024-03-27T10:52:58.614 INFO:teuthology.orchestra.run.smithi177.stdout:Created osd(s) 11 on host 'smithi177' 2024-03-27T10:52:58.617 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:52:58.643+0000 7f85a9000700 1 -- 172.21.15.177:0/793597423 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f85880a36a0 msgr2=0x7f85880a5b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:52:58.617 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:52:58.643+0000 7f85a9000700 1 --2- 172.21.15.177:0/793597423 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f85880a36a0 0x7f85880a5b60 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f858c006010 tx=0x7f858c003870 comp rx=0 tx=0).stop 2024-03-27T10:52:58.617 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:52:58.643+0000 7f85a9000700 1 -- 172.21.15.177:0/793597423 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f85a4100ef0 msgr2=0x7f85a419ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:52:58.617 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:52:58.643+0000 7f85a9000700 1 --2- 172.21.15.177:0/793597423 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f85a4100ef0 0x7f85a419ed90 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f8594007f10 tx=0x7f859400d3b0 comp rx=0 tx=0).stop 2024-03-27T10:52:58.617 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:52:58.643+0000 7f85a9000700 1 -- 172.21.15.177:0/793597423 shutdown_connections 2024-03-27T10:52:58.617 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:52:58.643+0000 7f85a9000700 1 --2- 172.21.15.177:0/793597423 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f85880a36a0 0x7f85880a5b60 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:52:58.617 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:52:58.643+0000 7f85a9000700 1 --2- 172.21.15.177:0/793597423 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f85a4106230 0x7f85a41a6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:52:58.617 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:52:58.643+0000 7f85a9000700 1 --2- 172.21.15.177:0/793597423 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f85a4100ef0 0x7f85a419ed90 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:52:58.617 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:52:58.643+0000 7f85a9000700 1 --2- 172.21.15.177:0/793597423 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85a4103860 0x7f85a419f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:52:58.618 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:52:58.643+0000 7f85a9000700 1 -- 172.21.15.177:0/793597423 >> 172.21.15.177:0/793597423 conn(0x7f85a40fcb10 msgr2=0x7f85a41018d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:52:58.618 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:52:58.644+0000 7f85a9000700 1 -- 172.21.15.177:0/793597423 shutdown_connections 2024-03-27T10:52:58.618 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:52:58.644+0000 7f85a9000700 1 -- 172.21.15.177:0/793597423 wait complete. 2024-03-27T10:52:58.768 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:58 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:52:58.768 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:58 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:52:58.768 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:58 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:52:58.768 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:58 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:52:58.768 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:58 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:52:58.768 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:58 smithi114 ceph-mon[35338]: from='osd.11 [v2:172.21.15.177:6824/2333030391,v1:172.21.15.177:6825/2333030391]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-27T10:52:58.768 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:58 smithi114 ceph-mon[35338]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-27T10:52:58.768 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:58 smithi114 ceph-mon[35338]: pgmap v321: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-03-27T10:52:58.891 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:58 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:52:58.892 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:58 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:52:58.892 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:58 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:52:58.892 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:58 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:52:58.892 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:58 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:52:58.892 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:58 smithi097 ceph-mon[31859]: from='osd.11 [v2:172.21.15.177:6824/2333030391,v1:172.21.15.177:6825/2333030391]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-27T10:52:58.892 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:58 smithi097 ceph-mon[31859]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-27T10:52:58.892 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:58 smithi097 ceph-mon[31859]: pgmap v321: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-03-27T10:52:58.927 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:58 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:52:58.927 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:58 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:52:58.927 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:58 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:52:58.927 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:58 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:52:58.928 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:58 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:52:58.928 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:58 smithi177 ceph-mon[35294]: from='osd.11 [v2:172.21.15.177:6824/2333030391,v1:172.21.15.177:6825/2333030391]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-27T10:52:58.928 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:58 smithi177 ceph-mon[35294]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-27T10:52:58.928 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:58 smithi177 ceph-mon[35294]: pgmap v321: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-03-27T10:52:59.293 DEBUG:teuthology.orchestra.run.smithi177:osd.11> sudo journalctl -f -n 0 -u ceph-adca88c4-ec26-11ee-95d0-87774f69a715@osd.11.service 2024-03-27T10:52:59.296 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-03-27T10:52:59.297 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid adca88c4-ec26-11ee-95d0-87774f69a715 -- ceph osd stat -f json 2024-03-27T10:52:59.641 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:59 smithi097 ceph-mon[31859]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-27T10:52:59.641 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:59 smithi097 ceph-mon[31859]: osdmap e83: 12 total, 11 up, 12 in 2024-03-27T10:52:59.641 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:59 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-27T10:52:59.642 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:59 smithi097 ceph-mon[31859]: from='osd.11 [v2:172.21.15.177:6824/2333030391,v1:172.21.15.177:6825/2333030391]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2024-03-27T10:52:59.642 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:59 smithi097 ceph-mon[31859]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2024-03-27T10:52:59.642 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:59 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:52:59.642 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:59 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:52:59.642 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:52:59 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:52:59.743 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T10:52:59.783 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:59 smithi177 ceph-mon[35294]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-27T10:52:59.783 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:59 smithi177 ceph-mon[35294]: osdmap e83: 12 total, 11 up, 12 in 2024-03-27T10:52:59.783 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:59 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-27T10:52:59.784 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:59 smithi177 ceph-mon[35294]: from='osd.11 [v2:172.21.15.177:6824/2333030391,v1:172.21.15.177:6825/2333030391]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2024-03-27T10:52:59.784 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:59 smithi177 ceph-mon[35294]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2024-03-27T10:52:59.784 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:59 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:52:59.784 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:59 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:52:59.784 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:52:59 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:52:59.784 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 10:52:59 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T10:52:59.569+0000 7f5a5ac00700 -1 osd.11 0 waiting for initial osdmap 2024-03-27T10:52:59.784 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 10:52:59 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T10:52:59.624+0000 7f5a55a00700 -1 osd.11 84 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-27T10:53:00.018 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:59 smithi114 ceph-mon[35338]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-27T10:53:00.018 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:59 smithi114 ceph-mon[35338]: osdmap e83: 12 total, 11 up, 12 in 2024-03-27T10:53:00.018 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:59 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-27T10:53:00.018 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:59 smithi114 ceph-mon[35338]: from='osd.11 [v2:172.21.15.177:6824/2333030391,v1:172.21.15.177:6825/2333030391]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2024-03-27T10:53:00.018 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:59 smithi114 ceph-mon[35338]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2024-03-27T10:53:00.018 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:59 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:00.018 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:59 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:00.018 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:52:59 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:53:00.774 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:00 smithi177 ceph-mon[35294]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]': finished 2024-03-27T10:53:00.774 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:00 smithi177 ceph-mon[35294]: osdmap e84: 12 total, 11 up, 12 in 2024-03-27T10:53:00.774 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:00 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-27T10:53:00.774 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:00 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-27T10:53:00.774 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:00 smithi177 ceph-mon[35294]: pgmap v324: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-03-27T10:53:00.892 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:00 smithi097 ceph-mon[31859]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]': finished 2024-03-27T10:53:00.892 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:00 smithi097 ceph-mon[31859]: osdmap e84: 12 total, 11 up, 12 in 2024-03-27T10:53:00.892 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:00 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-27T10:53:00.892 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:00 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-27T10:53:00.892 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:00 smithi097 ceph-mon[31859]: pgmap v324: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-03-27T10:53:01.018 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:00 smithi114 ceph-mon[35338]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]': finished 2024-03-27T10:53:01.018 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:00 smithi114 ceph-mon[35338]: osdmap e84: 12 total, 11 up, 12 in 2024-03-27T10:53:01.018 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:00 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-27T10:53:01.018 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:00 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-27T10:53:01.018 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:00 smithi114 ceph-mon[35338]: pgmap v324: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-03-27T10:53:01.652 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:01 smithi097 ceph-mon[31859]: purged_snaps scrub starts 2024-03-27T10:53:01.652 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:01 smithi097 ceph-mon[31859]: purged_snaps scrub ok 2024-03-27T10:53:01.652 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:01 smithi097 ceph-mon[31859]: osd.11 [v2:172.21.15.177:6824/2333030391,v1:172.21.15.177:6825/2333030391] boot 2024-03-27T10:53:01.652 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:01 smithi097 ceph-mon[31859]: osdmap e85: 12 total, 12 up, 12 in 2024-03-27T10:53:01.652 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:01 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-27T10:53:01.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:01.935+0000 7f4b9b200700 1 -- 172.21.15.97:0/1890448467 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b94106230 msgr2=0x7f4b941066b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:01.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:01.935+0000 7f4b9b200700 1 --2- 172.21.15.97:0/1890448467 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b94106230 0x7f4b941066b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c00b0a0 tx=0x7f4b8c00b3b0 comp rx=0 tx=0).stop 2024-03-27T10:53:01.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:01.935+0000 7f4b9b200700 1 -- 172.21.15.97:0/1890448467 shutdown_connections 2024-03-27T10:53:01.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:01.935+0000 7f4b9b200700 1 --2- 172.21.15.97:0/1890448467 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b94106230 0x7f4b941066b0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:01.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:01.935+0000 7f4b9b200700 1 --2- 172.21.15.97:0/1890448467 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4b94100ef0 0x7f4b94103320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:01.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:01.935+0000 7f4b9b200700 1 --2- 172.21.15.97:0/1890448467 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4b94103860 0x7f4b94105cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:01.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:01.935+0000 7f4b9b200700 1 -- 172.21.15.97:0/1890448467 >> 172.21.15.97:0/1890448467 conn(0x7f4b940fcb10 msgr2=0x7f4b940fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:53:01.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:01.935+0000 7f4b9b200700 1 -- 172.21.15.97:0/1890448467 shutdown_connections 2024-03-27T10:53:01.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:01.935+0000 7f4b9b200700 1 -- 172.21.15.97:0/1890448467 wait complete. 2024-03-27T10:53:01.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:01.936+0000 7f4b9b200700 1 Processor -- start 2024-03-27T10:53:01.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:01.937+0000 7f4b9b200700 1 -- start start 2024-03-27T10:53:01.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:01.937+0000 7f4b9b200700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4b94100ef0 0x7f4b9419edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:53:01.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:01.937+0000 7f4b9b200700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4b94103860 0x7f4b9419f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:53:01.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:01.937+0000 7f4b9b200700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b94106230 0x7f4b941a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:53:01.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:01.937+0000 7f4b9b200700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4b94114730 con 0x7f4b94106230 2024-03-27T10:53:01.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:01.937+0000 7f4b9b200700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f4b941145b0 con 0x7f4b94100ef0 2024-03-27T10:53:01.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:01.937+0000 7f4b9b200700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f4b941148b0 con 0x7f4b94103860 2024-03-27T10:53:01.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:01.937+0000 7f4b99200700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b94106230 0x7f4b941a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:53:01.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:01.937+0000 7f4b99200700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b94106230 0x7f4b941a6380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34058/0 (socket says 172.21.15.97:34058) 2024-03-27T10:53:01.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:01.937+0000 7f4b99200700 1 -- 172.21.15.97:0/2673695137 learned_addr learned my addr 172.21.15.97:0/2673695137 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:53:01.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:01.937+0000 7f4b93400700 1 --2- 172.21.15.97:0/2673695137 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4b94103860 0x7f4b9419f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:53:01.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:01.938+0000 7f4b93e00700 1 --2- 172.21.15.97:0/2673695137 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4b94100ef0 0x7f4b9419edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:53:01.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:01.938+0000 7f4b99200700 1 -- 172.21.15.97:0/2673695137 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4b94103860 msgr2=0x7f4b9419f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:01.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:01.938+0000 7f4b99200700 1 --2- 172.21.15.97:0/2673695137 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4b94103860 0x7f4b9419f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:01.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:01.938+0000 7f4b99200700 1 -- 172.21.15.97:0/2673695137 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4b94100ef0 msgr2=0x7f4b9419edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:01.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:01.938+0000 7f4b99200700 1 --2- 172.21.15.97:0/2673695137 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4b94100ef0 0x7f4b9419edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:01.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:01.938+0000 7f4b99200700 1 -- 172.21.15.97:0/2673695137 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b941a6a20 con 0x7f4b94106230 2024-03-27T10:53:01.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:01.938+0000 7f4b99200700 1 --2- 172.21.15.97:0/2673695137 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b94106230 0x7f4b941a6380 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c00cf10 tx=0x7f4b8c00cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:53:01.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:01.938+0000 7f4b90c00700 1 -- 172.21.15.97:0/2673695137 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b8c021020 con 0x7f4b94106230 2024-03-27T10:53:01.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:01.939+0000 7f4b90c00700 1 -- 172.21.15.97:0/2673695137 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4b8c01d920 con 0x7f4b94106230 2024-03-27T10:53:01.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:01.939+0000 7f4b90c00700 1 -- 172.21.15.97:0/2673695137 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b8c005580 con 0x7f4b94106230 2024-03-27T10:53:01.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:01.939+0000 7f4b9b200700 1 -- 172.21.15.97:0/2673695137 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b941a6cb0 con 0x7f4b94106230 2024-03-27T10:53:01.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:01.939+0000 7f4b9b200700 1 -- 172.21.15.97:0/2673695137 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b941a7110 con 0x7f4b94106230 2024-03-27T10:53:01.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:01.940+0000 7f4b9b200700 1 -- 172.21.15.97:0/2673695137 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b94069340 con 0x7f4b94106230 2024-03-27T10:53:01.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:01.942+0000 7f4b90c00700 1 -- 172.21.15.97:0/2673695137 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138370+0+0 (secure 0 0 0) 0x7f4b8c024020 con 0x7f4b94106230 2024-03-27T10:53:01.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:01.943+0000 7f4b90c00700 1 --2- 172.21.15.97:0/2673695137 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f4b780a3720 0x7f4b780a5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:53:01.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:01.943+0000 7f4b90c00700 1 -- 172.21.15.97:0/2673695137 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7127+0+0 (secure 0 0 0) 0x7f4b8c00f020 con 0x7f4b94106230 2024-03-27T10:53:01.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:01.944+0000 7f4b93e00700 1 --2- 172.21.15.97:0/2673695137 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f4b780a3720 0x7f4b780a5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:53:01.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:01.944+0000 7f4b93e00700 1 --2- 172.21.15.97:0/2673695137 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f4b780a3720 0x7f4b780a5be0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4b84006fd0 tx=0x7f4b84008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:53:01.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:01.948+0000 7f4b90c00700 1 -- 172.21.15.97:0/2673695137 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f4b8c009cb0 con 0x7f4b94106230 2024-03-27T10:53:01.927 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:01 smithi177 ceph-mon[35294]: purged_snaps scrub starts 2024-03-27T10:53:01.927 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:01 smithi177 ceph-mon[35294]: purged_snaps scrub ok 2024-03-27T10:53:01.927 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:01 smithi177 ceph-mon[35294]: osd.11 [v2:172.21.15.177:6824/2333030391,v1:172.21.15.177:6825/2333030391] boot 2024-03-27T10:53:01.928 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:01 smithi177 ceph-mon[35294]: osdmap e85: 12 total, 12 up, 12 in 2024-03-27T10:53:01.928 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:01 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-27T10:53:02.017 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:01 smithi114 ceph-mon[35338]: purged_snaps scrub starts 2024-03-27T10:53:02.018 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:01 smithi114 ceph-mon[35338]: purged_snaps scrub ok 2024-03-27T10:53:02.018 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:01 smithi114 ceph-mon[35338]: osd.11 [v2:172.21.15.177:6824/2333030391,v1:172.21.15.177:6825/2333030391] boot 2024-03-27T10:53:02.018 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:01 smithi114 ceph-mon[35338]: osdmap e85: 12 total, 12 up, 12 in 2024-03-27T10:53:02.018 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:01 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-27T10:53:02.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:02.171+0000 7f4b9b200700 1 -- 172.21.15.97:0/2673695137 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f4b94063250 con 0x7f4b94106230 2024-03-27T10:53:02.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:02.171+0000 7f4b90c00700 1 -- 172.21.15.97:0/2673695137 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v86) ==== 74+0+133 (secure 0 0 0) 0x7f4b8c09df50 con 0x7f4b94106230 2024-03-27T10:53:02.145 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T10:53:02.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:02.174+0000 7f4b9b200700 1 -- 172.21.15.97:0/2673695137 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f4b780a3720 msgr2=0x7f4b780a5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:02.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:02.174+0000 7f4b9b200700 1 --2- 172.21.15.97:0/2673695137 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f4b780a3720 0x7f4b780a5be0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4b84006fd0 tx=0x7f4b84008040 comp rx=0 tx=0).stop 2024-03-27T10:53:02.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:02.174+0000 7f4b9b200700 1 -- 172.21.15.97:0/2673695137 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b94106230 msgr2=0x7f4b941a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:02.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:02.174+0000 7f4b9b200700 1 --2- 172.21.15.97:0/2673695137 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b94106230 0x7f4b941a6380 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c00cf10 tx=0x7f4b8c00cf40 comp rx=0 tx=0).stop 2024-03-27T10:53:02.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:02.174+0000 7f4b9b200700 1 -- 172.21.15.97:0/2673695137 shutdown_connections 2024-03-27T10:53:02.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:02.174+0000 7f4b9b200700 1 --2- 172.21.15.97:0/2673695137 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f4b780a3720 0x7f4b780a5be0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:02.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:02.174+0000 7f4b9b200700 1 --2- 172.21.15.97:0/2673695137 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b94106230 0x7f4b941a6380 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:02.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:02.174+0000 7f4b9b200700 1 --2- 172.21.15.97:0/2673695137 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4b94100ef0 0x7f4b9419edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:02.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:02.174+0000 7f4b9b200700 1 --2- 172.21.15.97:0/2673695137 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4b94103860 0x7f4b9419f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:02.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:02.174+0000 7f4b9b200700 1 -- 172.21.15.97:0/2673695137 >> 172.21.15.97:0/2673695137 conn(0x7f4b940fcb10 msgr2=0x7f4b94101c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:53:02.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:02.175+0000 7f4b9b200700 1 -- 172.21.15.97:0/2673695137 shutdown_connections 2024-03-27T10:53:02.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:02.175+0000 7f4b9b200700 1 -- 172.21.15.97:0/2673695137 wait complete. 2024-03-27T10:53:02.849 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:02 smithi177 ceph-mon[35294]: osdmap e86: 12 total, 12 up, 12 in 2024-03-27T10:53:02.849 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:02 smithi177 ceph-mon[35294]: pgmap v327: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T10:53:02.849 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:02 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/2673695137' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-27T10:53:02.862 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:02 smithi097 ceph-mon[31859]: osdmap e86: 12 total, 12 up, 12 in 2024-03-27T10:53:02.862 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:02 smithi097 ceph-mon[31859]: pgmap v327: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T10:53:02.862 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:02 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/2673695137' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-27T10:53:02.929 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":86,"num_osds":12,"num_up_osds":12,"osd_up_since":1711536780,"num_in_osds":12,"osd_in_since":1711536756,"num_remapped_pgs":0} 2024-03-27T10:53:02.929 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell --fsid adca88c4-ec26-11ee-95d0-87774f69a715 -- ceph osd dump --format=json 2024-03-27T10:53:03.017 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:02 smithi114 ceph-mon[35338]: osdmap e86: 12 total, 12 up, 12 in 2024-03-27T10:53:03.018 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:02 smithi114 ceph-mon[35338]: pgmap v327: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T10:53:03.018 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:02 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/2673695137' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-27T10:53:03.382 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T10:53:04.866 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:04 smithi097 ceph-mon[31859]: Detected new or changed devices on smithi177 2024-03-27T10:53:04.866 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:04 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:04.866 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:04 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:04.867 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:04 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-27T10:53:04.867 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:04 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-27T10:53:04.867 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:04 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-27T10:53:04.867 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:04 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-27T10:53:04.867 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:04 smithi097 ceph-mon[31859]: Adjusting osd_memory_target on smithi177 to 4272M 2024-03-27T10:53:04.867 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:04 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:04.867 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:04 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:53:04.867 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:04 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:53:04.867 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:04 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:04.867 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:04 smithi097 ceph-mon[31859]: pgmap v328: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T10:53:04.927 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:04 smithi177 ceph-mon[35294]: Detected new or changed devices on smithi177 2024-03-27T10:53:04.928 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:04 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:04.928 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:04 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:04.928 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:04 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-27T10:53:04.928 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:04 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-27T10:53:04.928 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:04 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-27T10:53:04.928 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:04 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-27T10:53:04.928 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:04 smithi177 ceph-mon[35294]: Adjusting osd_memory_target on smithi177 to 4272M 2024-03-27T10:53:04.928 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:04 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:04.928 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:04 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:53:04.928 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:04 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:53:04.928 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:04 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:04.928 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:04 smithi177 ceph-mon[35294]: pgmap v328: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T10:53:05.018 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:04 smithi114 ceph-mon[35338]: Detected new or changed devices on smithi177 2024-03-27T10:53:05.018 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:04 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:05.018 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:04 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:05.018 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:04 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-27T10:53:05.018 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:04 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-27T10:53:05.018 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:04 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-27T10:53:05.018 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:04 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-27T10:53:05.018 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:04 smithi114 ceph-mon[35338]: Adjusting osd_memory_target on smithi177 to 4272M 2024-03-27T10:53:05.018 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:04 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:05.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:04 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:53:05.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:04 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:53:05.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:04 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:05.019 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:04 smithi114 ceph-mon[35338]: pgmap v328: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T10:53:05.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:05.390+0000 7ff1ede00700 1 -- 172.21.15.97:0/2761639639 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1e8102690 msgr2=0x7ff1e8102b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:05.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:05.390+0000 7ff1ede00700 1 --2- 172.21.15.97:0/2761639639 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1e8102690 0x7ff1e8102b10 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff1d0009990 tx=0x7ff1d0009ca0 comp rx=0 tx=0).stop 2024-03-27T10:53:05.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:05.391+0000 7ff1ede00700 1 -- 172.21.15.97:0/2761639639 shutdown_connections 2024-03-27T10:53:05.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:05.391+0000 7ff1ede00700 1 --2- 172.21.15.97:0/2761639639 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff1e8103050 0x7ff1e810f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:05.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:05.391+0000 7ff1ede00700 1 --2- 172.21.15.97:0/2761639639 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff1e8108680 0x7ff1e8108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:05.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:05.391+0000 7ff1ede00700 1 --2- 172.21.15.97:0/2761639639 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1e8102690 0x7ff1e8102b10 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:05.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:05.391+0000 7ff1ede00700 1 -- 172.21.15.97:0/2761639639 >> 172.21.15.97:0/2761639639 conn(0x7ff1e80797a0 msgr2=0x7ff1e8079bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:53:05.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:05.391+0000 7ff1ede00700 1 -- 172.21.15.97:0/2761639639 shutdown_connections 2024-03-27T10:53:05.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:05.391+0000 7ff1ede00700 1 -- 172.21.15.97:0/2761639639 wait complete. 2024-03-27T10:53:05.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:05.392+0000 7ff1ede00700 1 Processor -- start 2024-03-27T10:53:05.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:05.393+0000 7ff1ede00700 1 -- start start 2024-03-27T10:53:05.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:05.393+0000 7ff1ede00700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff1e8102690 0x7ff1e8074940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:53:05.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:05.393+0000 7ff1ede00700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff1e8103050 0x7ff1e8074e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:53:05.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:05.393+0000 7ff1ede00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1e8108680 0x7ff1e80753c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:53:05.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:05.393+0000 7ff1ede00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff1e8112910 con 0x7ff1e8108680 2024-03-27T10:53:05.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:05.393+0000 7ff1ede00700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff1e8112790 con 0x7ff1e8103050 2024-03-27T10:53:05.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:05.393+0000 7ff1ede00700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff1e8112a90 con 0x7ff1e8102690 2024-03-27T10:53:05.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:05.394+0000 7ff1e7400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1e8108680 0x7ff1e80753c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:53:05.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:05.394+0000 7ff1e7400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1e8108680 0x7ff1e80753c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34076/0 (socket says 172.21.15.97:34076) 2024-03-27T10:53:05.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:05.394+0000 7ff1e7400700 1 -- 172.21.15.97:0/3654930426 learned_addr learned my addr 172.21.15.97:0/3654930426 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:53:05.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:05.394+0000 7ff1e6a00700 1 --2- 172.21.15.97:0/3654930426 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff1e8102690 0x7ff1e8074940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:53:05.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:05.394+0000 7ff1dfe00700 1 --2- 172.21.15.97:0/3654930426 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff1e8103050 0x7ff1e8074e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:53:05.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:05.394+0000 7ff1e7400700 1 -- 172.21.15.97:0/3654930426 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff1e8102690 msgr2=0x7ff1e8074940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:05.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:05.394+0000 7ff1e7400700 1 --2- 172.21.15.97:0/3654930426 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff1e8102690 0x7ff1e8074940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:05.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:05.394+0000 7ff1e7400700 1 -- 172.21.15.97:0/3654930426 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff1e8103050 msgr2=0x7ff1e8074e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:05.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:05.394+0000 7ff1e7400700 1 --2- 172.21.15.97:0/3654930426 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff1e8103050 0x7ff1e8074e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:05.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:05.394+0000 7ff1e7400700 1 -- 172.21.15.97:0/3654930426 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1e81a7330 con 0x7ff1e8108680 2024-03-27T10:53:05.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:05.395+0000 7ff1e7400700 1 --2- 172.21.15.97:0/3654930426 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1e8108680 0x7ff1e80753c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff1e000ab50 tx=0x7ff1e000ae60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:53:05.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:05.395+0000 7ff1eca00700 1 -- 172.21.15.97:0/3654930426 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e0017020 con 0x7ff1e8108680 2024-03-27T10:53:05.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:05.395+0000 7ff1eca00700 1 -- 172.21.15.97:0/3654930426 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff1e0004bb0 con 0x7ff1e8108680 2024-03-27T10:53:05.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:05.395+0000 7ff1ede00700 1 -- 172.21.15.97:0/3654930426 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1e81a7620 con 0x7ff1e8108680 2024-03-27T10:53:05.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:05.395+0000 7ff1eca00700 1 -- 172.21.15.97:0/3654930426 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e0005350 con 0x7ff1e8108680 2024-03-27T10:53:05.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:05.395+0000 7ff1ede00700 1 -- 172.21.15.97:0/3654930426 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1e81a7b10 con 0x7ff1e8108680 2024-03-27T10:53:05.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:05.396+0000 7ff1ede00700 1 -- 172.21.15.97:0/3654930426 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1e8069300 con 0x7ff1e8108680 2024-03-27T10:53:05.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:05.399+0000 7ff1eca00700 1 -- 172.21.15.97:0/3654930426 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138370+0+0 (secure 0 0 0) 0x7ff1e00054f0 con 0x7ff1e8108680 2024-03-27T10:53:05.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:05.400+0000 7ff1eca00700 1 --2- 172.21.15.97:0/3654930426 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7ff1cc0a36a0 0x7ff1cc0a5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:53:05.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:05.400+0000 7ff1eca00700 1 -- 172.21.15.97:0/3654930426 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7127+0+0 (secure 0 0 0) 0x7ff1e00cf550 con 0x7ff1e8108680 2024-03-27T10:53:05.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:05.400+0000 7ff1e6a00700 1 --2- 172.21.15.97:0/3654930426 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7ff1cc0a36a0 0x7ff1cc0a5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:53:05.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:05.401+0000 7ff1e6a00700 1 --2- 172.21.15.97:0/3654930426 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7ff1cc0a36a0 0x7ff1cc0a5b60 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff1d8005950 tx=0x7ff1d800a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:53:05.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:05.403+0000 7ff1eca00700 1 -- 172.21.15.97:0/3654930426 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7ff1e0021020 con 0x7ff1e8108680 2024-03-27T10:53:05.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:05.628+0000 7ff1ede00700 1 -- 172.21.15.97:0/3654930426 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff1e802dd30 con 0x7ff1e8108680 2024-03-27T10:53:05.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:05.631+0000 7ff1eca00700 1 -- 172.21.15.97:0/3654930426 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+18448 (secure 0 0 0) 0x7ff1e009c5b0 con 0x7ff1e8108680 2024-03-27T10:53:05.604 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T10:53:05.605 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":86,"fsid":"adca88c4-ec26-11ee-95d0-87774f69a715","created":"2024-03-27T10:43:24.680694+0000","modified":"2024-03-27T10:53:01.568999+0000","last_up_change":"2024-03-27T10:53:00.552513+0000","last_in_change":"2024-03-27T10:52:36.302986+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-27T10:47:54.052810+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b634e5f1-6118-4e2f-95e9-830db390f8d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1746812080},{"type":"v1","addr":"172.21.15.97:6803","nonce":1746812080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1746812080},{"type":"v1","addr":"172.21.15.97:6805","nonce":1746812080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":1746812080},{"type":"v1","addr":"172.21.15.97:6809","nonce":1746812080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1746812080},{"type":"v1","addr":"172.21.15.97:6807","nonce":1746812080}]},"public_addr":"172.21.15.97:6803/1746812080","cluster_addr":"172.21.15.97:6805/1746812080","heartbeat_back_addr":"172.21.15.97:6809/1746812080","heartbeat_front_addr":"172.21.15.97:6807/1746812080","state":["exists","up"]},{"osd":1,"uuid":"441e2770-3284-4388-b0de-c56eb8a373d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":3858821264},{"type":"v1","addr":"172.21.15.114:6801","nonce":3858821264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":3858821264},{"type":"v1","addr":"172.21.15.114:6803","nonce":3858821264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":3858821264},{"type":"v1","addr":"172.21.15.114:6807","nonce":3858821264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":3858821264},{"type":"v1","addr":"172.21.15.114:6805","nonce":3858821264}]},"public_addr":"172.21.15.114:6801/3858821264","cluster_addr":"172.21.15.114:6803/3858821264","heartbeat_back_addr":"172.21.15.114:6807/3858821264","heartbeat_front_addr":"172.21.15.114:6805/3858821264","state":["exists","up"]},{"osd":2,"uuid":"00108864-1aa6-48d6-8398-e8eb79fda123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":124240658},{"type":"v1","addr":"172.21.15.177:6801","nonce":124240658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":124240658},{"type":"v1","addr":"172.21.15.177:6803","nonce":124240658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":124240658},{"type":"v1","addr":"172.21.15.177:6807","nonce":124240658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":124240658},{"type":"v1","addr":"172.21.15.177:6805","nonce":124240658}]},"public_addr":"172.21.15.177:6801/124240658","cluster_addr":"172.21.15.177:6803/124240658","heartbeat_back_addr":"172.21.15.177:6807/124240658","heartbeat_front_addr":"172.21.15.177:6805/124240658","state":["exists","up"]},{"osd":3,"uuid":"1860b690-e2dc-4977-88d7-d1aacf7022ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2478179311},{"type":"v1","addr":"172.21.15.97:6811","nonce":2478179311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2478179311},{"type":"v1","addr":"172.21.15.97:6813","nonce":2478179311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":2478179311},{"type":"v1","addr":"172.21.15.97:6817","nonce":2478179311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2478179311},{"type":"v1","addr":"172.21.15.97:6815","nonce":2478179311}]},"public_addr":"172.21.15.97:6811/2478179311","cluster_addr":"172.21.15.97:6813/2478179311","heartbeat_back_addr":"172.21.15.97:6817/2478179311","heartbeat_front_addr":"172.21.15.97:6815/2478179311","state":["exists","up"]},{"osd":4,"uuid":"9245a3a4-7d49-4fdb-9d55-112e43177050","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":75755945},{"type":"v1","addr":"172.21.15.114:6809","nonce":75755945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":75755945},{"type":"v1","addr":"172.21.15.114:6811","nonce":75755945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":75755945},{"type":"v1","addr":"172.21.15.114:6815","nonce":75755945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":75755945},{"type":"v1","addr":"172.21.15.114:6813","nonce":75755945}]},"public_addr":"172.21.15.114:6809/75755945","cluster_addr":"172.21.15.114:6811/75755945","heartbeat_back_addr":"172.21.15.114:6815/75755945","heartbeat_front_addr":"172.21.15.114:6813/75755945","state":["exists","up"]},{"osd":5,"uuid":"51922f99-e8d3-461b-8615-e9bfdcbf5ed3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":1918493790},{"type":"v1","addr":"172.21.15.177:6809","nonce":1918493790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":1918493790},{"type":"v1","addr":"172.21.15.177:6811","nonce":1918493790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":1918493790},{"type":"v1","addr":"172.21.15.177:6815","nonce":1918493790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":1918493790},{"type":"v1","addr":"172.21.15.177:6813","nonce":1918493790}]},"public_addr":"172.21.15.177:6809/1918493790","cluster_addr":"172.21.15.177:6811/1918493790","heartbeat_back_addr":"172.21.15.177:6815/1918493790","heartbeat_front_addr":"172.21.15.177:6813/1918493790","state":["exists","up"]},{"osd":6,"uuid":"06dffb32-05eb-43b3-852e-915c5cc7f25a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2892389563},{"type":"v1","addr":"172.21.15.97:6819","nonce":2892389563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2892389563},{"type":"v1","addr":"172.21.15.97:6821","nonce":2892389563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2892389563},{"type":"v1","addr":"172.21.15.97:6825","nonce":2892389563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2892389563},{"type":"v1","addr":"172.21.15.97:6823","nonce":2892389563}]},"public_addr":"172.21.15.97:6819/2892389563","cluster_addr":"172.21.15.97:6821/2892389563","heartbeat_back_addr":"172.21.15.97:6825/2892389563","heartbeat_front_addr":"172.21.15.97:6823/2892389563","state":["exists","up"]},{"osd":7,"uuid":"e68e3519-9c1b-43c5-8e06-5385248b19df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":2445491153},{"type":"v1","addr":"172.21.15.114:6817","nonce":2445491153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":2445491153},{"type":"v1","addr":"172.21.15.114:6819","nonce":2445491153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2445491153},{"type":"v1","addr":"172.21.15.114:6823","nonce":2445491153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":2445491153},{"type":"v1","addr":"172.21.15.114:6821","nonce":2445491153}]},"public_addr":"172.21.15.114:6817/2445491153","cluster_addr":"172.21.15.114:6819/2445491153","heartbeat_back_addr":"172.21.15.114:6823/2445491153","heartbeat_front_addr":"172.21.15.114:6821/2445491153","state":["exists","up"]},{"osd":8,"uuid":"31ac592e-1349-49ee-9af9-8ad85d682978","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":2051134191},{"type":"v1","addr":"172.21.15.177:6817","nonce":2051134191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":2051134191},{"type":"v1","addr":"172.21.15.177:6819","nonce":2051134191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":2051134191},{"type":"v1","addr":"172.21.15.177:6823","nonce":2051134191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":2051134191},{"type":"v1","addr":"172.21.15.177:6821","nonce":2051134191}]},"public_addr":"172.21.15.177:6817/2051134191","cluster_addr":"172.21.15.177:6819/2051134191","heartbeat_back_addr":"172.21.15.177:6823/2051134191","heartbeat_front_addr":"172.21.15.177:6821/2051134191","state":["exists","up"]},{"osd":9,"uuid":"0171086a-dd41-4eb4-a389-16f0c46a64db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":1578881271},{"type":"v1","addr":"172.21.15.97:6827","nonce":1578881271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1578881271},{"type":"v1","addr":"172.21.15.97:6829","nonce":1578881271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1578881271},{"type":"v1","addr":"172.21.15.97:6833","nonce":1578881271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1578881271},{"type":"v1","addr":"172.21.15.97:6831","nonce":1578881271}]},"public_addr":"172.21.15.97:6827/1578881271","cluster_addr":"172.21.15.97:6829/1578881271","heartbeat_back_addr":"172.21.15.97:6833/1578881271","heartbeat_front_addr":"172.21.15.97:6831/1578881271","state":["exists","up"]},{"osd":10,"uuid":"64754e31-b124-4601-ad98-2f70b1dd5359","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":2893111461},{"type":"v1","addr":"172.21.15.114:6825","nonce":2893111461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":2893111461},{"type":"v1","addr":"172.21.15.114:6827","nonce":2893111461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2893111461},{"type":"v1","addr":"172.21.15.114:6831","nonce":2893111461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":2893111461},{"type":"v1","addr":"172.21.15.114:6829","nonce":2893111461}]},"public_addr":"172.21.15.114:6825/2893111461","cluster_addr":"172.21.15.114:6827/2893111461","heartbeat_back_addr":"172.21.15.114:6831/2893111461","heartbeat_front_addr":"172.21.15.114:6829/2893111461","state":["exists","up"]},{"osd":11,"uuid":"9ffc752c-adf7-4037-89d3-535787d7475d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":85,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":2333030391},{"type":"v1","addr":"172.21.15.177:6825","nonce":2333030391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":2333030391},{"type":"v1","addr":"172.21.15.177:6827","nonce":2333030391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":2333030391},{"type":"v1","addr":"172.21.15.177:6831","nonce":2333030391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":2333030391},{"type":"v1","addr":"172.21.15.177:6829","nonce":2333030391}]},"public_addr":"172.21.15.177:6825/2333030391","cluster_addr":"172.21.15.177:6827/2333030391","heartbeat_back_addr":"172.21.15.177:6831/2333030391","heartbeat_front_addr":"172.21.15.177:6829/2333030391","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:46:45.143519+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:47:18.656947+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:47:51.316345+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:48:24.575512+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:48:59.113555+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:49:32.103865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:50:06.615539+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:50:41.578341+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:51:15.254011+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:51:49.099165+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:52:24.544821+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:52:58.838484+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/359646":"2024-03-28T10:44:39.889550+0000","172.21.15.97:6801/98682155":"2024-03-28T10:44:06.019075+0000","172.21.15.97:6801/1685921853":"2024-03-28T10:44:39.889550+0000","172.21.15.97:6800/98682155":"2024-03-28T10:44:06.019075+0000","172.21.15.97:0/107690680":"2024-03-28T10:44:06.019075+0000","172.21.15.97:6800/1685921853":"2024-03-28T10:44:39.889550+0000","172.21.15.97:0/437043413":"2024-03-28T10:44:39.889550+0000","172.21.15.97:0/2319706992":"2024-03-28T10:44:39.889550+0000","172.21.15.97:0/3911532654":"2024-03-28T10:44:06.019075+0000","172.21.15.97:0/3347321845":"2024-03-28T10:44:06.019075+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-27T10:53:05.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:05.633+0000 7ff1ede00700 1 -- 172.21.15.97:0/3654930426 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7ff1cc0a36a0 msgr2=0x7ff1cc0a5b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:05.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:05.633+0000 7ff1ede00700 1 --2- 172.21.15.97:0/3654930426 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7ff1cc0a36a0 0x7ff1cc0a5b60 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff1d8005950 tx=0x7ff1d800a300 comp rx=0 tx=0).stop 2024-03-27T10:53:05.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:05.634+0000 7ff1ede00700 1 -- 172.21.15.97:0/3654930426 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1e8108680 msgr2=0x7ff1e80753c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:05.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:05.634+0000 7ff1ede00700 1 --2- 172.21.15.97:0/3654930426 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1e8108680 0x7ff1e80753c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff1e000ab50 tx=0x7ff1e000ae60 comp rx=0 tx=0).stop 2024-03-27T10:53:05.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:05.634+0000 7ff1ede00700 1 -- 172.21.15.97:0/3654930426 shutdown_connections 2024-03-27T10:53:05.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:05.634+0000 7ff1ede00700 1 --2- 172.21.15.97:0/3654930426 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7ff1cc0a36a0 0x7ff1cc0a5b60 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:05.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:05.634+0000 7ff1ede00700 1 --2- 172.21.15.97:0/3654930426 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1e8108680 0x7ff1e80753c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:05.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:05.634+0000 7ff1ede00700 1 --2- 172.21.15.97:0/3654930426 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff1e8102690 0x7ff1e8074940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:05.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:05.634+0000 7ff1ede00700 1 --2- 172.21.15.97:0/3654930426 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff1e8103050 0x7ff1e8074e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:05.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:05.634+0000 7ff1ede00700 1 -- 172.21.15.97:0/3654930426 >> 172.21.15.97:0/3654930426 conn(0x7ff1e80797a0 msgr2=0x7ff1e8100840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:53:05.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:05.634+0000 7ff1ede00700 1 -- 172.21.15.97:0/3654930426 shutdown_connections 2024-03-27T10:53:05.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:05.634+0000 7ff1ede00700 1 -- 172.21.15.97:0/3654930426 wait complete. 2024-03-27T10:53:05.891 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:05 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/3654930426' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T10:53:05.927 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:05 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/3654930426' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T10:53:06.017 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:05 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/3654930426' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T10:53:06.488 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-27T10:47:54.052810+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-27T10:53:06.489 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell --fsid adca88c4-ec26-11ee-95d0-87774f69a715 -- ceph osd pool get .mgr pg_num 2024-03-27T10:53:06.930 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T10:53:07.017 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:06 smithi114 ceph-mon[35338]: pgmap v329: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T10:53:07.074 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:06 smithi097 ceph-mon[31859]: pgmap v329: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T10:53:07.177 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:06 smithi177 ceph-mon[35294]: pgmap v329: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T10:53:09.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:09.143+0000 7ff0c7200700 1 -- 172.21.15.97:0/1886809560 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0c0104850 msgr2=0x7ff0c0104cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:09.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:09.143+0000 7ff0c7200700 1 --2- 172.21.15.97:0/1886809560 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0c0104850 0x7ff0c0104cd0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac009990 tx=0x7ff0ac009ca0 comp rx=0 tx=0).stop 2024-03-27T10:53:09.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:09.143+0000 7ff0c7200700 1 -- 172.21.15.97:0/1886809560 shutdown_connections 2024-03-27T10:53:09.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:09.143+0000 7ff0c7200700 1 --2- 172.21.15.97:0/1886809560 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff0c0105210 0x7ff0c0111720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:09.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:09.143+0000 7ff0c7200700 1 --2- 172.21.15.97:0/1886809560 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff0c010a850 0x7ff0c010ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:09.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:09.143+0000 7ff0c7200700 1 --2- 172.21.15.97:0/1886809560 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0c0104850 0x7ff0c0104cd0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:09.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:09.143+0000 7ff0c7200700 1 -- 172.21.15.97:0/1886809560 >> 172.21.15.97:0/1886809560 conn(0x7ff0c0100320 msgr2=0x7ff0c0102740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:53:09.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:09.143+0000 7ff0c7200700 1 -- 172.21.15.97:0/1886809560 shutdown_connections 2024-03-27T10:53:09.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:09.143+0000 7ff0c7200700 1 -- 172.21.15.97:0/1886809560 wait complete. 2024-03-27T10:53:09.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:09.145+0000 7ff0c7200700 1 Processor -- start 2024-03-27T10:53:09.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:09.145+0000 7ff0c7200700 1 -- start start 2024-03-27T10:53:09.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:09.145+0000 7ff0c7200700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0c0104850 0x7ff0c019aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:53:09.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:09.145+0000 7ff0c7200700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff0c0105210 0x7ff0c019b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:53:09.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:09.145+0000 7ff0c7200700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff0c010a850 0x7ff0c019f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:53:09.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:09.145+0000 7ff0c7200700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff0c0114b40 con 0x7ff0c0104850 2024-03-27T10:53:09.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:09.145+0000 7ff0c7200700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff0c01149c0 con 0x7ff0c010a850 2024-03-27T10:53:09.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:09.145+0000 7ff0c7200700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff0c0114cc0 con 0x7ff0c0105210 2024-03-27T10:53:09.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:09.146+0000 7ff0bfe00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0c0104850 0x7ff0c019aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:53:09.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:09.146+0000 7ff0bfe00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0c0104850 0x7ff0c019aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51658/0 (socket says 172.21.15.97:51658) 2024-03-27T10:53:09.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:09.146+0000 7ff0bfe00700 1 -- 172.21.15.97:0/2084856469 learned_addr learned my addr 172.21.15.97:0/2084856469 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:53:09.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:09.146+0000 7ff0bf400700 1 --2- 172.21.15.97:0/2084856469 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff0c0105210 0x7ff0c019b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:53:09.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:09.146+0000 7ff0c5200700 1 --2- 172.21.15.97:0/2084856469 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff0c010a850 0x7ff0c019f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:53:09.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:09.146+0000 7ff0bfe00700 1 -- 172.21.15.97:0/2084856469 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff0c0105210 msgr2=0x7ff0c019b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:09.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:09.146+0000 7ff0bfe00700 1 --2- 172.21.15.97:0/2084856469 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff0c0105210 0x7ff0c019b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:09.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:09.146+0000 7ff0bfe00700 1 -- 172.21.15.97:0/2084856469 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff0c010a850 msgr2=0x7ff0c019f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:09.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:09.146+0000 7ff0bfe00700 1 --2- 172.21.15.97:0/2084856469 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff0c010a850 0x7ff0c019f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:09.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:09.146+0000 7ff0bfe00700 1 -- 172.21.15.97:0/2084856469 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0c01a0000 con 0x7ff0c0104850 2024-03-27T10:53:09.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:09.147+0000 7ff0bfe00700 1 --2- 172.21.15.97:0/2084856469 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0c0104850 0x7ff0c019aff0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff0b000b530 tx=0x7ff0b000b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:53:09.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:09.147+0000 7ff0bcc00700 1 -- 172.21.15.97:0/2084856469 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0b0015020 con 0x7ff0c0104850 2024-03-27T10:53:09.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:09.147+0000 7ff0bcc00700 1 -- 172.21.15.97:0/2084856469 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff0b0011ce0 con 0x7ff0c0104850 2024-03-27T10:53:09.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:09.147+0000 7ff0bcc00700 1 -- 172.21.15.97:0/2084856469 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0b0010d60 con 0x7ff0c0104850 2024-03-27T10:53:09.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:09.147+0000 7ff0c7200700 1 -- 172.21.15.97:0/2084856469 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0c01a02f0 con 0x7ff0c0104850 2024-03-27T10:53:09.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:09.147+0000 7ff0c7200700 1 -- 172.21.15.97:0/2084856469 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0c01a7b70 con 0x7ff0c0104850 2024-03-27T10:53:09.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:09.148+0000 7ff0c7200700 1 -- 172.21.15.97:0/2084856469 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0c0069300 con 0x7ff0c0104850 2024-03-27T10:53:09.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:09.150+0000 7ff0bcc00700 1 -- 172.21.15.97:0/2084856469 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138370+0+0 (secure 0 0 0) 0x7ff0b0018020 con 0x7ff0c0104850 2024-03-27T10:53:09.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:09.151+0000 7ff0bcc00700 1 --2- 172.21.15.97:0/2084856469 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7ff0a40a36a0 0x7ff0a40a5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:53:09.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:09.152+0000 7ff0bcc00700 1 -- 172.21.15.97:0/2084856469 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7127+0+0 (secure 0 0 0) 0x7ff0b00cf1d0 con 0x7ff0c0104850 2024-03-27T10:53:09.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:09.152+0000 7ff0bf400700 1 --2- 172.21.15.97:0/2084856469 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7ff0a40a36a0 0x7ff0a40a5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:53:09.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:09.152+0000 7ff0bf400700 1 --2- 172.21.15.97:0/2084856469 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7ff0a40a36a0 0x7ff0a40a5b60 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac00b5c0 tx=0x7ff0ac01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:53:09.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:09.156+0000 7ff0bcc00700 1 -- 172.21.15.97:0/2084856469 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7ff0b001f020 con 0x7ff0c0104850 2024-03-27T10:53:09.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:09.378+0000 7ff0c7200700 1 -- 172.21.15.97:0/2084856469 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff0c002dd30 con 0x7ff0c0104850 2024-03-27T10:53:09.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:09 smithi097 ceph-mon[31859]: pgmap v330: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T10:53:09.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:09.379+0000 7ff0bcc00700 1 -- 172.21.15.97:0/2084856469 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) ==== 93+0+10 (secure 0 0 0) 0x7ff0b009c1a0 con 0x7ff0c0104850 2024-03-27T10:53:09.352 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-03-27T10:53:09.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:09.382+0000 7ff0c7200700 1 -- 172.21.15.97:0/2084856469 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7ff0a40a36a0 msgr2=0x7ff0a40a5b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:09.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:09.382+0000 7ff0c7200700 1 --2- 172.21.15.97:0/2084856469 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7ff0a40a36a0 0x7ff0a40a5b60 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac00b5c0 tx=0x7ff0ac01a040 comp rx=0 tx=0).stop 2024-03-27T10:53:09.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:09.382+0000 7ff0c7200700 1 -- 172.21.15.97:0/2084856469 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0c0104850 msgr2=0x7ff0c019aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:09.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:09.382+0000 7ff0c7200700 1 --2- 172.21.15.97:0/2084856469 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0c0104850 0x7ff0c019aff0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff0b000b530 tx=0x7ff0b000b8f0 comp rx=0 tx=0).stop 2024-03-27T10:53:09.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:09.382+0000 7ff0c7200700 1 -- 172.21.15.97:0/2084856469 shutdown_connections 2024-03-27T10:53:09.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:09.382+0000 7ff0c7200700 1 --2- 172.21.15.97:0/2084856469 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7ff0a40a36a0 0x7ff0a40a5b60 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:09.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:09.382+0000 7ff0c7200700 1 --2- 172.21.15.97:0/2084856469 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff0c010a850 0x7ff0c019f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:09.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:09.382+0000 7ff0c7200700 1 --2- 172.21.15.97:0/2084856469 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0c0104850 0x7ff0c019aff0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:09.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:09.382+0000 7ff0c7200700 1 --2- 172.21.15.97:0/2084856469 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff0c0105210 0x7ff0c019b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:09.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:09.382+0000 7ff0c7200700 1 -- 172.21.15.97:0/2084856469 >> 172.21.15.97:0/2084856469 conn(0x7ff0c0100320 msgr2=0x7ff0c0100960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:53:09.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:09.383+0000 7ff0c7200700 1 -- 172.21.15.97:0/2084856469 shutdown_connections 2024-03-27T10:53:09.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:09.383+0000 7ff0c7200700 1 -- 172.21.15.97:0/2084856469 wait complete. 2024-03-27T10:53:09.359 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:09 smithi177 ceph-mon[35294]: pgmap v330: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T10:53:09.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:09 smithi114 ceph-mon[35338]: pgmap v330: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T10:53:10.139 INFO:tasks.cephadm:Adding mds.a on smithi097 2024-03-27T10:53:10.139 INFO:tasks.cephadm:Adding mds.d on smithi097 2024-03-27T10:53:10.139 INFO:tasks.cephadm:Adding mds.g on smithi097 2024-03-27T10:53:10.139 INFO:tasks.cephadm:Adding mds.j on smithi097 2024-03-27T10:53:10.139 INFO:tasks.cephadm:Adding mds.b on smithi114 2024-03-27T10:53:10.139 INFO:tasks.cephadm:Adding mds.e on smithi114 2024-03-27T10:53:10.139 INFO:tasks.cephadm:Adding mds.h on smithi114 2024-03-27T10:53:10.140 INFO:tasks.cephadm:Adding mds.k on smithi114 2024-03-27T10:53:10.140 INFO:tasks.cephadm:Adding mds.c on smithi177 2024-03-27T10:53:10.140 INFO:tasks.cephadm:Adding mds.f on smithi177 2024-03-27T10:53:10.140 INFO:tasks.cephadm:Adding mds.i on smithi177 2024-03-27T10:53:10.140 INFO:tasks.cephadm:Adding mds.l on smithi177 2024-03-27T10:53:10.140 DEBUG:teuthology.orchestra.run.smithi177:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid adca88c4-ec26-11ee-95d0-87774f69a715 -- ceph orch apply mds all '12;smithi097=a;smithi097=d;smithi097=g;smithi097=j;smithi114=b;smithi114=e;smithi114=h;smithi114=k;smithi177=c;smithi177=f;smithi177=i;smithi177=l' 2024-03-27T10:53:10.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:10 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/2084856469' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-27T10:53:10.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:10 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/2084856469' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-27T10:53:10.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:10 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/2084856469' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-27T10:53:10.591 INFO:teuthology.orchestra.run.smithi177.stderr:Inferring config /var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/mon.c/config 2024-03-27T10:53:11.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:11 smithi097 ceph-mon[31859]: pgmap v331: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T10:53:11.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:11 smithi177 ceph-mon[35294]: pgmap v331: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T10:53:11.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:11 smithi114 ceph-mon[35338]: pgmap v331: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T10:53:12.671 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:53:12.698+0000 7f33d1a00700 1 -- 172.21.15.177:0/2361023796 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f33cc076670 msgr2=0x7f33cc074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:12.671 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:53:12.698+0000 7f33d1a00700 1 --2- 172.21.15.177:0/2361023796 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f33cc076670 0x7f33cc074ad0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f33bc009990 tx=0x7f33bc009ca0 comp rx=0 tx=0).stop 2024-03-27T10:53:12.671 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:53:12.698+0000 7f33d1a00700 1 -- 172.21.15.177:0/2361023796 shutdown_connections 2024-03-27T10:53:12.671 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:53:12.698+0000 7f33d1a00700 1 --2- 172.21.15.177:0/2361023796 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33cc10ec60 0x7f33cc111050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:12.671 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:53:12.698+0000 7f33d1a00700 1 --2- 172.21.15.177:0/2361023796 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f33cc076670 0x7f33cc074ad0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:12.671 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:53:12.698+0000 7f33d1a00700 1 --2- 172.21.15.177:0/2361023796 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f33cc0750a0 0x7f33cc075520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:12.671 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:53:12.698+0000 7f33d1a00700 1 -- 172.21.15.177:0/2361023796 >> 172.21.15.177:0/2361023796 conn(0x7f33cc0fdbe0 msgr2=0x7f33cc100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:53:12.671 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:53:12.698+0000 7f33d1a00700 1 -- 172.21.15.177:0/2361023796 shutdown_connections 2024-03-27T10:53:12.671 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:53:12.698+0000 7f33d1a00700 1 -- 172.21.15.177:0/2361023796 wait complete. 2024-03-27T10:53:12.673 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:53:12.699+0000 7f33d1a00700 1 Processor -- start 2024-03-27T10:53:12.673 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:53:12.699+0000 7f33d1a00700 1 -- start start 2024-03-27T10:53:12.673 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:53:12.700+0000 7f33d1a00700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f33cc0750a0 0x7f33cc19ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:53:12.674 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:53:12.700+0000 7f33d1a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33cc076670 0x7f33cc19f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:53:12.674 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:53:12.700+0000 7f33d1a00700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f33cc10ec60 0x7f33cc1a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:53:12.674 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:53:12.700+0000 7f33d1a00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f33cc114760 con 0x7f33cc076670 2024-03-27T10:53:12.674 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:53:12.700+0000 7f33d1a00700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f33cc1145e0 con 0x7f33cc0750a0 2024-03-27T10:53:12.674 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:53:12.700+0000 7f33d1a00700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f33cc1148e0 con 0x7f33cc10ec60 2024-03-27T10:53:12.674 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:53:12.700+0000 7f33cb400700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f33cc10ec60 0x7f33cc1a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:53:12.674 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:53:12.700+0000 7f33cb400700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f33cc10ec60 0x7f33cc1a6410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:37896/0 (socket says 172.21.15.177:37896) 2024-03-27T10:53:12.674 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:53:12.700+0000 7f33cb400700 1 -- 172.21.15.177:0/3763766969 learned_addr learned my addr 172.21.15.177:0/3763766969 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-03-27T10:53:12.674 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:53:12.700+0000 7f33caa00700 1 --2- 172.21.15.177:0/3763766969 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f33cc0750a0 0x7f33cc19ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:53:12.674 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:53:12.700+0000 7f33ca000700 1 --2- 172.21.15.177:0/3763766969 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33cc076670 0x7f33cc19f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:53:12.674 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:53:12.700+0000 7f33cb400700 1 -- 172.21.15.177:0/3763766969 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f33cc0750a0 msgr2=0x7f33cc19ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:12.674 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:53:12.700+0000 7f33cb400700 1 --2- 172.21.15.177:0/3763766969 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f33cc0750a0 0x7f33cc19ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:12.675 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:53:12.700+0000 7f33cb400700 1 -- 172.21.15.177:0/3763766969 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33cc076670 msgr2=0x7f33cc19f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:12.675 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:53:12.700+0000 7f33cb400700 1 --2- 172.21.15.177:0/3763766969 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33cc076670 0x7f33cc19f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:12.675 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:53:12.700+0000 7f33cb400700 1 -- 172.21.15.177:0/3763766969 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33cc1a6ab0 con 0x7f33cc10ec60 2024-03-27T10:53:12.675 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:53:12.701+0000 7f33caa00700 1 --2- 172.21.15.177:0/3763766969 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f33cc0750a0 0x7f33cc19ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T10:53:12.675 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:53:12.701+0000 7f33cb400700 1 --2- 172.21.15.177:0/3763766969 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f33cc10ec60 0x7f33cc1a6410 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f33c4007f10 tx=0x7f33c400d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:53:12.675 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:53:12.701+0000 7f33c3400700 1 -- 172.21.15.177:0/3763766969 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c4004bb0 con 0x7f33cc10ec60 2024-03-27T10:53:12.675 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:53:12.701+0000 7f33c3400700 1 -- 172.21.15.177:0/3763766969 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f33c4004d50 con 0x7f33cc10ec60 2024-03-27T10:53:12.675 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:53:12.701+0000 7f33c3400700 1 -- 172.21.15.177:0/3763766969 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c4005990 con 0x7f33cc10ec60 2024-03-27T10:53:12.675 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:53:12.701+0000 7f33d1a00700 1 -- 172.21.15.177:0/3763766969 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33cc1a6da0 con 0x7f33cc10ec60 2024-03-27T10:53:12.675 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:53:12.701+0000 7f33d1a00700 1 -- 172.21.15.177:0/3763766969 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33cc1a7200 con 0x7f33cc10ec60 2024-03-27T10:53:12.675 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:53:12.702+0000 7f33d1a00700 1 -- 172.21.15.177:0/3763766969 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33cc02dd30 con 0x7f33cc10ec60 2024-03-27T10:53:12.678 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:53:12.706+0000 7f33c3400700 1 -- 172.21.15.177:0/3763766969 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 16) ==== 138370+0+0 (secure 0 0 0) 0x7f33c401a020 con 0x7f33cc10ec60 2024-03-27T10:53:12.679 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:53:12.707+0000 7f33c3400700 1 --2- 172.21.15.177:0/3763766969 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f33b00a3720 0x7f33b00a5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:53:12.680 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:53:12.708+0000 7f33c3400700 1 -- 172.21.15.177:0/3763766969 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7127+0+0 (secure 0 0 0) 0x7f33c40cf6f0 con 0x7f33cc10ec60 2024-03-27T10:53:12.680 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:53:12.708+0000 7f33caa00700 1 --2- 172.21.15.177:0/3763766969 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f33b00a3720 0x7f33b00a5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:53:12.681 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:53:12.709+0000 7f33caa00700 1 --2- 172.21.15.177:0/3763766969 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f33b00a3720 0x7f33b00a5be0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f33bc00b5c0 tx=0x7f33bc005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:53:12.683 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:53:12.711+0000 7f33c3400700 1 -- 172.21.15.177:0/3763766969 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f33c4010070 con 0x7f33cc10ec60 2024-03-27T10:53:12.919 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:53:12.947+0000 7f33d1a00700 1 -- 172.21.15.177:0/3763766969 --> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi097=a;smithi097=d;smithi097=g;smithi097=j;smithi114=b;smithi114=e;smithi114=h;smithi114=k;smithi177=c;smithi177=f;smithi177=i;smithi177=l", "target": ["mon-mgr", ""]}) -- 0x7f33cc062b80 con 0x7f33b00a3720 2024-03-27T10:53:12.937 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:53:12.965+0000 7f33c3400700 1 -- 172.21.15.177:0/3763766969 <== mgr.14152 v2:172.21.15.97:6800/3955463538 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f33cc062b80 con 0x7f33b00a3720 2024-03-27T10:53:12.937 INFO:teuthology.orchestra.run.smithi177.stdout:Scheduled mds.all update... 2024-03-27T10:53:12.940 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:53:12.968+0000 7f33d1a00700 1 -- 172.21.15.177:0/3763766969 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f33b00a3720 msgr2=0x7f33b00a5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:12.940 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:53:12.968+0000 7f33d1a00700 1 --2- 172.21.15.177:0/3763766969 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f33b00a3720 0x7f33b00a5be0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f33bc00b5c0 tx=0x7f33bc005fd0 comp rx=0 tx=0).stop 2024-03-27T10:53:12.941 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:53:12.968+0000 7f33d1a00700 1 -- 172.21.15.177:0/3763766969 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f33cc10ec60 msgr2=0x7f33cc1a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:12.941 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:53:12.968+0000 7f33d1a00700 1 --2- 172.21.15.177:0/3763766969 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f33cc10ec60 0x7f33cc1a6410 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f33c4007f10 tx=0x7f33c400d3b0 comp rx=0 tx=0).stop 2024-03-27T10:53:12.941 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:53:12.968+0000 7f33d1a00700 1 -- 172.21.15.177:0/3763766969 shutdown_connections 2024-03-27T10:53:12.941 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:53:12.968+0000 7f33d1a00700 1 --2- 172.21.15.177:0/3763766969 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f33b00a3720 0x7f33b00a5be0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:12.941 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:53:12.968+0000 7f33d1a00700 1 --2- 172.21.15.177:0/3763766969 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f33cc10ec60 0x7f33cc1a6410 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:12.941 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:53:12.968+0000 7f33d1a00700 1 --2- 172.21.15.177:0/3763766969 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f33cc0750a0 0x7f33cc19ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:12.941 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:53:12.968+0000 7f33d1a00700 1 --2- 172.21.15.177:0/3763766969 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33cc076670 0x7f33cc19f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:12.941 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:53:12.968+0000 7f33d1a00700 1 -- 172.21.15.177:0/3763766969 >> 172.21.15.177:0/3763766969 conn(0x7f33cc0fdbe0 msgr2=0x7f33cc102000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:53:12.941 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:53:12.969+0000 7f33d1a00700 1 -- 172.21.15.177:0/3763766969 shutdown_connections 2024-03-27T10:53:12.941 INFO:teuthology.orchestra.run.smithi177.stderr:2024-03-27T10:53:12.969+0000 7f33d1a00700 1 -- 172.21.15.177:0/3763766969 wait complete. 2024-03-27T10:53:13.285 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:13 smithi177 ceph-mon[35294]: pgmap v332: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T10:53:13.285 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:13 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:13.285 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:13 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:53:13.285 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:13 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:53:13.285 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:13 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:53:13.285 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:13 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:13.285 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:13 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-27T10:53:13.285 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:13 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-27T10:53:13.286 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:13 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:53:13.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:13 smithi097 ceph-mon[31859]: pgmap v332: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T10:53:13.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:13 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:13.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:13 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:53:13.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:13 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:53:13.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:13 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:53:13.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:13 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:13.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:13 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-27T10:53:13.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:13 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-27T10:53:13.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:13 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:53:13.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:13 smithi114 ceph-mon[35338]: pgmap v332: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T10:53:13.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:13 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:13.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:13 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:53:13.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:13 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:53:13.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:13 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:53:13.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:13 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:13.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:13 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-27T10:53:13.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:13 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-27T10:53:13.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:13 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:53:13.614 DEBUG:teuthology.orchestra.run.smithi097:mds.a> sudo journalctl -f -n 0 -u ceph-adca88c4-ec26-11ee-95d0-87774f69a715@mds.a.service 2024-03-27T10:53:13.623 DEBUG:teuthology.orchestra.run.smithi097:mds.d> sudo journalctl -f -n 0 -u ceph-adca88c4-ec26-11ee-95d0-87774f69a715@mds.d.service 2024-03-27T10:53:13.670 DEBUG:teuthology.orchestra.run.smithi097:mds.g> sudo journalctl -f -n 0 -u ceph-adca88c4-ec26-11ee-95d0-87774f69a715@mds.g.service 2024-03-27T10:53:13.716 DEBUG:teuthology.orchestra.run.smithi097:mds.j> sudo journalctl -f -n 0 -u ceph-adca88c4-ec26-11ee-95d0-87774f69a715@mds.j.service 2024-03-27T10:53:13.761 DEBUG:teuthology.orchestra.run.smithi114:mds.b> sudo journalctl -f -n 0 -u ceph-adca88c4-ec26-11ee-95d0-87774f69a715@mds.b.service 2024-03-27T10:53:13.769 DEBUG:teuthology.orchestra.run.smithi114:mds.e> sudo journalctl -f -n 0 -u ceph-adca88c4-ec26-11ee-95d0-87774f69a715@mds.e.service 2024-03-27T10:53:13.817 DEBUG:teuthology.orchestra.run.smithi114:mds.h> sudo journalctl -f -n 0 -u ceph-adca88c4-ec26-11ee-95d0-87774f69a715@mds.h.service 2024-03-27T10:53:13.863 DEBUG:teuthology.orchestra.run.smithi114:mds.k> sudo journalctl -f -n 0 -u ceph-adca88c4-ec26-11ee-95d0-87774f69a715@mds.k.service 2024-03-27T10:53:13.909 DEBUG:teuthology.orchestra.run.smithi177:mds.c> sudo journalctl -f -n 0 -u ceph-adca88c4-ec26-11ee-95d0-87774f69a715@mds.c.service 2024-03-27T10:53:13.991 DEBUG:teuthology.orchestra.run.smithi177:mds.f> sudo journalctl -f -n 0 -u ceph-adca88c4-ec26-11ee-95d0-87774f69a715@mds.f.service 2024-03-27T10:53:14.039 DEBUG:teuthology.orchestra.run.smithi177:mds.i> sudo journalctl -f -n 0 -u ceph-adca88c4-ec26-11ee-95d0-87774f69a715@mds.i.service 2024-03-27T10:53:14.088 DEBUG:teuthology.orchestra.run.smithi177:mds.l> sudo journalctl -f -n 0 -u ceph-adca88c4-ec26-11ee-95d0-87774f69a715@mds.l.service 2024-03-27T10:53:14.133 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-03-27T10:53:14.134 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-27T10:53:14.184 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:14 smithi177 ceph-mon[35294]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi097=a;smithi097=d;smithi097=g;smithi097=j;smithi114=b;smithi114=e;smithi114=h;smithi114=k;smithi177=c;smithi177=f;smithi177=i;smithi177=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T10:53:14.184 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:14 smithi177 ceph-mon[35294]: Saving service mds.all spec with placement smithi097=a;smithi097=d;smithi097=g;smithi097=j;smithi114=b;smithi114=e;smithi114=h;smithi114=k;smithi177=c;smithi177=f;smithi177=i;smithi177=l;count:12 2024-03-27T10:53:14.184 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:14 smithi177 ceph-mon[35294]: Deploying daemon mds.l on smithi177 2024-03-27T10:53:14.267 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:14 smithi114 ceph-mon[35338]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi097=a;smithi097=d;smithi097=g;smithi097=j;smithi114=b;smithi114=e;smithi114=h;smithi114=k;smithi177=c;smithi177=f;smithi177=i;smithi177=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T10:53:14.267 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:14 smithi114 ceph-mon[35338]: Saving service mds.all spec with placement smithi097=a;smithi097=d;smithi097=g;smithi097=j;smithi114=b;smithi114=e;smithi114=h;smithi114=k;smithi177=c;smithi177=f;smithi177=i;smithi177=l;count:12 2024-03-27T10:53:14.267 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:14 smithi114 ceph-mon[35338]: Deploying daemon mds.l on smithi177 2024-03-27T10:53:14.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:14 smithi097 ceph-mon[31859]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi097=a;smithi097=d;smithi097=g;smithi097=j;smithi114=b;smithi114=e;smithi114=h;smithi114=k;smithi177=c;smithi177=f;smithi177=i;smithi177=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T10:53:14.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:14 smithi097 ceph-mon[31859]: Saving service mds.all spec with placement smithi097=a;smithi097=d;smithi097=g;smithi097=j;smithi114=b;smithi114=e;smithi114=h;smithi114=k;smithi177=c;smithi177=f;smithi177=i;smithi177=l;count:12 2024-03-27T10:53:14.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:14 smithi097 ceph-mon[31859]: Deploying daemon mds.l on smithi177 2024-03-27T10:53:14.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:14.445+0000 7f483be00640 1 -- 172.21.15.97:0/3074981980 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f483c103510 msgr2=0x7f483c109da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:14.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:14.445+0000 7f483be00640 1 --2- 172.21.15.97:0/3074981980 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f483c103510 0x7f483c109da0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4834009ca0 tx=0x7f483402c0f0 comp rx=0 tx=0).stop 2024-03-27T10:53:14.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:14.445+0000 7f483be00640 1 -- 172.21.15.97:0/3074981980 shutdown_connections 2024-03-27T10:53:14.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:14.445+0000 7f483be00640 1 --2- 172.21.15.97:0/3074981980 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f483c103510 0x7f483c109da0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:14.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:14.445+0000 7f483be00640 1 --2- 172.21.15.97:0/3074981980 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f483c102b50 0x7f483c102fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:14.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:14.445+0000 7f483be00640 1 --2- 172.21.15.97:0/3074981980 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f483c101960 0x7f483c101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:14.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:14.445+0000 7f483be00640 1 -- 172.21.15.97:0/3074981980 >> 172.21.15.97:0/3074981980 conn(0x7f483c0fceb0 msgr2=0x7f483c0ff310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:53:14.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:14.446+0000 7f483be00640 1 -- 172.21.15.97:0/3074981980 shutdown_connections 2024-03-27T10:53:14.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:14.446+0000 7f483be00640 1 -- 172.21.15.97:0/3074981980 wait complete. 2024-03-27T10:53:14.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:14.447+0000 7f483be00640 1 Processor -- start 2024-03-27T10:53:14.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:14.447+0000 7f483be00640 1 -- start start 2024-03-27T10:53:14.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:14.447+0000 7f483be00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f483c101960 0x7f483c199dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:53:14.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:14.447+0000 7f483be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f483c102b50 0x7f483c19a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:53:14.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:14.447+0000 7f483be00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f483c103510 0x7f483c1a1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:53:14.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:14.447+0000 7f483be00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f483c10d490 con 0x7f483c102b50 2024-03-27T10:53:14.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:14.447+0000 7f483be00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f483c10d310 con 0x7f483c101960 2024-03-27T10:53:14.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:14.447+0000 7f483be00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f483c10d610 con 0x7f483c103510 2024-03-27T10:53:14.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:14.448+0000 7f483a000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f483c102b50 0x7f483c19a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:53:14.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:14.448+0000 7f483a000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f483c102b50 0x7f483c19a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51668/0 (socket says 172.21.15.97:51668) 2024-03-27T10:53:14.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:14.448+0000 7f483a000640 1 -- 172.21.15.97:0/2926691208 learned_addr learned my addr 172.21.15.97:0/2926691208 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:53:14.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:14.448+0000 7f483b400640 1 --2- 172.21.15.97:0/2926691208 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f483c103510 0x7f483c1a1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:53:14.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:14.448+0000 7f483aa00640 1 --2- 172.21.15.97:0/2926691208 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f483c101960 0x7f483c199dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:53:14.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:14.448+0000 7f483a000640 1 -- 172.21.15.97:0/2926691208 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f483c103510 msgr2=0x7f483c1a1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:14.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:14.448+0000 7f483a000640 1 --2- 172.21.15.97:0/2926691208 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f483c103510 0x7f483c1a1330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:14.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:14.448+0000 7f483a000640 1 -- 172.21.15.97:0/2926691208 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f483c101960 msgr2=0x7f483c199dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:14.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:14.448+0000 7f483a000640 1 --2- 172.21.15.97:0/2926691208 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f483c101960 0x7f483c199dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:14.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:14.448+0000 7f483a000640 1 -- 172.21.15.97:0/2926691208 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f483c1a19d0 con 0x7f483c102b50 2024-03-27T10:53:14.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:14.448+0000 7f483a000640 1 --2- 172.21.15.97:0/2926691208 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f483c102b50 0x7f483c19a300 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f482800d950 tx=0x7f482800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:53:14.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:14.449+0000 7f4840a00640 1 -- 172.21.15.97:0/2926691208 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4828017070 con 0x7f483c102b50 2024-03-27T10:53:14.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:14.449+0000 7f4840a00640 1 -- 172.21.15.97:0/2926691208 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4828009c30 con 0x7f483c102b50 2024-03-27T10:53:14.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:14.449+0000 7f483be00640 1 -- 172.21.15.97:0/2926691208 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f483c1a1cc0 con 0x7f483c102b50 2024-03-27T10:53:14.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:14.449+0000 7f4840a00640 1 -- 172.21.15.97:0/2926691208 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f482800f620 con 0x7f483c102b50 2024-03-27T10:53:14.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:14.449+0000 7f483be00640 1 -- 172.21.15.97:0/2926691208 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f483c1a21c0 con 0x7f483c102b50 2024-03-27T10:53:14.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:14.450+0000 7f483be00640 1 -- 172.21.15.97:0/2926691208 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f483c1a2390 con 0x7f483c102b50 2024-03-27T10:53:14.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:14.452+0000 7f4840a00640 1 -- 172.21.15.97:0/2926691208 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138370+0+0 (secure 0 0 0) 0x7f48280109a0 con 0x7f483c102b50 2024-03-27T10:53:14.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:14.454+0000 7f4840a00640 1 --2- 172.21.15.97:0/2926691208 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f48200a3610 0x7f48200a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:53:14.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:14.454+0000 7f4840a00640 1 -- 172.21.15.97:0/2926691208 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7127+0+0 (secure 0 0 0) 0x7f48280cf6c0 con 0x7f483c102b50 2024-03-27T10:53:14.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:14.454+0000 7f483aa00640 1 --2- 172.21.15.97:0/2926691208 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f48200a3610 0x7f48200a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:53:14.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:14.455+0000 7f483aa00640 1 --2- 172.21.15.97:0/2926691208 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f48200a3610 0x7f48200a5ad0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f483c19b290 tx=0x7f482c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:53:14.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:14.458+0000 7f4840a00640 1 -- 172.21.15.97:0/2926691208 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f4828009de0 con 0x7f483c102b50 2024-03-27T10:53:14.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:14.660+0000 7f483be00640 1 -- 172.21.15.97:0/2926691208 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f483c1a2b60 con 0x7f483c102b50 2024-03-27T10:53:14.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:14.663+0000 7f4840a00640 1 -- 172.21.15.97:0/2926691208 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+18448 (secure 0 0 0) 0x7f482809c690 con 0x7f483c102b50 2024-03-27T10:53:14.636 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T10:53:14.637 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":86,"fsid":"adca88c4-ec26-11ee-95d0-87774f69a715","created":"2024-03-27T10:43:24.680694+0000","modified":"2024-03-27T10:53:01.568999+0000","last_up_change":"2024-03-27T10:53:00.552513+0000","last_in_change":"2024-03-27T10:52:36.302986+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-27T10:47:54.052810+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b634e5f1-6118-4e2f-95e9-830db390f8d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1746812080},{"type":"v1","addr":"172.21.15.97:6803","nonce":1746812080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1746812080},{"type":"v1","addr":"172.21.15.97:6805","nonce":1746812080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":1746812080},{"type":"v1","addr":"172.21.15.97:6809","nonce":1746812080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1746812080},{"type":"v1","addr":"172.21.15.97:6807","nonce":1746812080}]},"public_addr":"172.21.15.97:6803/1746812080","cluster_addr":"172.21.15.97:6805/1746812080","heartbeat_back_addr":"172.21.15.97:6809/1746812080","heartbeat_front_addr":"172.21.15.97:6807/1746812080","state":["exists","up"]},{"osd":1,"uuid":"441e2770-3284-4388-b0de-c56eb8a373d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":3858821264},{"type":"v1","addr":"172.21.15.114:6801","nonce":3858821264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":3858821264},{"type":"v1","addr":"172.21.15.114:6803","nonce":3858821264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":3858821264},{"type":"v1","addr":"172.21.15.114:6807","nonce":3858821264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":3858821264},{"type":"v1","addr":"172.21.15.114:6805","nonce":3858821264}]},"public_addr":"172.21.15.114:6801/3858821264","cluster_addr":"172.21.15.114:6803/3858821264","heartbeat_back_addr":"172.21.15.114:6807/3858821264","heartbeat_front_addr":"172.21.15.114:6805/3858821264","state":["exists","up"]},{"osd":2,"uuid":"00108864-1aa6-48d6-8398-e8eb79fda123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":124240658},{"type":"v1","addr":"172.21.15.177:6801","nonce":124240658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":124240658},{"type":"v1","addr":"172.21.15.177:6803","nonce":124240658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":124240658},{"type":"v1","addr":"172.21.15.177:6807","nonce":124240658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":124240658},{"type":"v1","addr":"172.21.15.177:6805","nonce":124240658}]},"public_addr":"172.21.15.177:6801/124240658","cluster_addr":"172.21.15.177:6803/124240658","heartbeat_back_addr":"172.21.15.177:6807/124240658","heartbeat_front_addr":"172.21.15.177:6805/124240658","state":["exists","up"]},{"osd":3,"uuid":"1860b690-e2dc-4977-88d7-d1aacf7022ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2478179311},{"type":"v1","addr":"172.21.15.97:6811","nonce":2478179311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2478179311},{"type":"v1","addr":"172.21.15.97:6813","nonce":2478179311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":2478179311},{"type":"v1","addr":"172.21.15.97:6817","nonce":2478179311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2478179311},{"type":"v1","addr":"172.21.15.97:6815","nonce":2478179311}]},"public_addr":"172.21.15.97:6811/2478179311","cluster_addr":"172.21.15.97:6813/2478179311","heartbeat_back_addr":"172.21.15.97:6817/2478179311","heartbeat_front_addr":"172.21.15.97:6815/2478179311","state":["exists","up"]},{"osd":4,"uuid":"9245a3a4-7d49-4fdb-9d55-112e43177050","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":75755945},{"type":"v1","addr":"172.21.15.114:6809","nonce":75755945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":75755945},{"type":"v1","addr":"172.21.15.114:6811","nonce":75755945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":75755945},{"type":"v1","addr":"172.21.15.114:6815","nonce":75755945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":75755945},{"type":"v1","addr":"172.21.15.114:6813","nonce":75755945}]},"public_addr":"172.21.15.114:6809/75755945","cluster_addr":"172.21.15.114:6811/75755945","heartbeat_back_addr":"172.21.15.114:6815/75755945","heartbeat_front_addr":"172.21.15.114:6813/75755945","state":["exists","up"]},{"osd":5,"uuid":"51922f99-e8d3-461b-8615-e9bfdcbf5ed3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":1918493790},{"type":"v1","addr":"172.21.15.177:6809","nonce":1918493790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":1918493790},{"type":"v1","addr":"172.21.15.177:6811","nonce":1918493790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":1918493790},{"type":"v1","addr":"172.21.15.177:6815","nonce":1918493790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":1918493790},{"type":"v1","addr":"172.21.15.177:6813","nonce":1918493790}]},"public_addr":"172.21.15.177:6809/1918493790","cluster_addr":"172.21.15.177:6811/1918493790","heartbeat_back_addr":"172.21.15.177:6815/1918493790","heartbeat_front_addr":"172.21.15.177:6813/1918493790","state":["exists","up"]},{"osd":6,"uuid":"06dffb32-05eb-43b3-852e-915c5cc7f25a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2892389563},{"type":"v1","addr":"172.21.15.97:6819","nonce":2892389563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2892389563},{"type":"v1","addr":"172.21.15.97:6821","nonce":2892389563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2892389563},{"type":"v1","addr":"172.21.15.97:6825","nonce":2892389563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2892389563},{"type":"v1","addr":"172.21.15.97:6823","nonce":2892389563}]},"public_addr":"172.21.15.97:6819/2892389563","cluster_addr":"172.21.15.97:6821/2892389563","heartbeat_back_addr":"172.21.15.97:6825/2892389563","heartbeat_front_addr":"172.21.15.97:6823/2892389563","state":["exists","up"]},{"osd":7,"uuid":"e68e3519-9c1b-43c5-8e06-5385248b19df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":2445491153},{"type":"v1","addr":"172.21.15.114:6817","nonce":2445491153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":2445491153},{"type":"v1","addr":"172.21.15.114:6819","nonce":2445491153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2445491153},{"type":"v1","addr":"172.21.15.114:6823","nonce":2445491153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":2445491153},{"type":"v1","addr":"172.21.15.114:6821","nonce":2445491153}]},"public_addr":"172.21.15.114:6817/2445491153","cluster_addr":"172.21.15.114:6819/2445491153","heartbeat_back_addr":"172.21.15.114:6823/2445491153","heartbeat_front_addr":"172.21.15.114:6821/2445491153","state":["exists","up"]},{"osd":8,"uuid":"31ac592e-1349-49ee-9af9-8ad85d682978","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":2051134191},{"type":"v1","addr":"172.21.15.177:6817","nonce":2051134191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":2051134191},{"type":"v1","addr":"172.21.15.177:6819","nonce":2051134191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":2051134191},{"type":"v1","addr":"172.21.15.177:6823","nonce":2051134191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":2051134191},{"type":"v1","addr":"172.21.15.177:6821","nonce":2051134191}]},"public_addr":"172.21.15.177:6817/2051134191","cluster_addr":"172.21.15.177:6819/2051134191","heartbeat_back_addr":"172.21.15.177:6823/2051134191","heartbeat_front_addr":"172.21.15.177:6821/2051134191","state":["exists","up"]},{"osd":9,"uuid":"0171086a-dd41-4eb4-a389-16f0c46a64db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":1578881271},{"type":"v1","addr":"172.21.15.97:6827","nonce":1578881271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1578881271},{"type":"v1","addr":"172.21.15.97:6829","nonce":1578881271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1578881271},{"type":"v1","addr":"172.21.15.97:6833","nonce":1578881271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1578881271},{"type":"v1","addr":"172.21.15.97:6831","nonce":1578881271}]},"public_addr":"172.21.15.97:6827/1578881271","cluster_addr":"172.21.15.97:6829/1578881271","heartbeat_back_addr":"172.21.15.97:6833/1578881271","heartbeat_front_addr":"172.21.15.97:6831/1578881271","state":["exists","up"]},{"osd":10,"uuid":"64754e31-b124-4601-ad98-2f70b1dd5359","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":2893111461},{"type":"v1","addr":"172.21.15.114:6825","nonce":2893111461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":2893111461},{"type":"v1","addr":"172.21.15.114:6827","nonce":2893111461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2893111461},{"type":"v1","addr":"172.21.15.114:6831","nonce":2893111461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":2893111461},{"type":"v1","addr":"172.21.15.114:6829","nonce":2893111461}]},"public_addr":"172.21.15.114:6825/2893111461","cluster_addr":"172.21.15.114:6827/2893111461","heartbeat_back_addr":"172.21.15.114:6831/2893111461","heartbeat_front_addr":"172.21.15.114:6829/2893111461","state":["exists","up"]},{"osd":11,"uuid":"9ffc752c-adf7-4037-89d3-535787d7475d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":85,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":2333030391},{"type":"v1","addr":"172.21.15.177:6825","nonce":2333030391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":2333030391},{"type":"v1","addr":"172.21.15.177:6827","nonce":2333030391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":2333030391},{"type":"v1","addr":"172.21.15.177:6831","nonce":2333030391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":2333030391},{"type":"v1","addr":"172.21.15.177:6829","nonce":2333030391}]},"public_addr":"172.21.15.177:6825/2333030391","cluster_addr":"172.21.15.177:6827/2333030391","heartbeat_back_addr":"172.21.15.177:6831/2333030391","heartbeat_front_addr":"172.21.15.177:6829/2333030391","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:46:45.143519+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:47:18.656947+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:47:51.316345+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:48:24.575512+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:48:59.113555+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:49:32.103865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:50:06.615539+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:50:41.578341+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:51:15.254011+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:51:49.099165+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:52:24.544821+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:52:58.838484+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/359646":"2024-03-28T10:44:39.889550+0000","172.21.15.97:6801/98682155":"2024-03-28T10:44:06.019075+0000","172.21.15.97:6801/1685921853":"2024-03-28T10:44:39.889550+0000","172.21.15.97:6800/98682155":"2024-03-28T10:44:06.019075+0000","172.21.15.97:0/107690680":"2024-03-28T10:44:06.019075+0000","172.21.15.97:6800/1685921853":"2024-03-28T10:44:39.889550+0000","172.21.15.97:0/437043413":"2024-03-28T10:44:39.889550+0000","172.21.15.97:0/2319706992":"2024-03-28T10:44:39.889550+0000","172.21.15.97:0/3911532654":"2024-03-28T10:44:06.019075+0000","172.21.15.97:0/3347321845":"2024-03-28T10:44:06.019075+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-27T10:53:14.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:14.665+0000 7f483be00640 1 -- 172.21.15.97:0/2926691208 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f48200a3610 msgr2=0x7f48200a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:14.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:14.665+0000 7f483be00640 1 --2- 172.21.15.97:0/2926691208 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f48200a3610 0x7f48200a5ad0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f483c19b290 tx=0x7f482c005f90 comp rx=0 tx=0).stop 2024-03-27T10:53:14.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:14.665+0000 7f483be00640 1 -- 172.21.15.97:0/2926691208 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f483c102b50 msgr2=0x7f483c19a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:14.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:14.665+0000 7f483be00640 1 --2- 172.21.15.97:0/2926691208 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f483c102b50 0x7f483c19a300 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f482800d950 tx=0x7f482800de10 comp rx=0 tx=0).stop 2024-03-27T10:53:14.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:14.665+0000 7f483be00640 1 -- 172.21.15.97:0/2926691208 shutdown_connections 2024-03-27T10:53:14.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:14.665+0000 7f483be00640 1 --2- 172.21.15.97:0/2926691208 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f483c103510 0x7f483c1a1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:14.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:14.665+0000 7f483be00640 1 --2- 172.21.15.97:0/2926691208 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f483c102b50 0x7f483c19a300 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:14.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:14.665+0000 7f483be00640 1 --2- 172.21.15.97:0/2926691208 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f48200a3610 0x7f48200a5ad0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:14.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:14.665+0000 7f483be00640 1 --2- 172.21.15.97:0/2926691208 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f483c101960 0x7f483c199dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:14.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:14.665+0000 7f483be00640 1 -- 172.21.15.97:0/2926691208 >> 172.21.15.97:0/2926691208 conn(0x7f483c0fceb0 msgr2=0x7f483c073d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:53:14.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:14.666+0000 7f483be00640 1 -- 172.21.15.97:0/2926691208 shutdown_connections 2024-03-27T10:53:14.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:14.666+0000 7f483be00640 1 -- 172.21.15.97:0/2926691208 wait complete. 2024-03-27T10:53:14.660 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-27T10:47:54.052810+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-27T10:53:14.661 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-27T10:53:14.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:14.897+0000 7f51f3c00640 1 -- 172.21.15.97:0/595665966 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51ec104f80 msgr2=0x7f51ec105400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:14.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:14.897+0000 7f51f3c00640 1 --2- 172.21.15.97:0/595665966 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51ec104f80 0x7f51ec105400 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f51d4009900 tx=0x7f51d402bc50 comp rx=0 tx=0).stop 2024-03-27T10:53:14.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:14.897+0000 7f51f3c00640 1 -- 172.21.15.97:0/595665966 shutdown_connections 2024-03-27T10:53:14.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:14.897+0000 7f51f3c00640 1 --2- 172.21.15.97:0/595665966 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51ec105940 0x7f51ec10c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:14.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:14.897+0000 7f51f3c00640 1 --2- 172.21.15.97:0/595665966 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51ec104f80 0x7f51ec105400 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:14.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:14.897+0000 7f51f3c00640 1 --2- 172.21.15.97:0/595665966 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f51ec103d90 0x7f51ec1041b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:14.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:14.897+0000 7f51f3c00640 1 -- 172.21.15.97:0/595665966 >> 172.21.15.97:0/595665966 conn(0x7f51ec0ff300 msgr2=0x7f51ec101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:53:14.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:14.898+0000 7f51f3c00640 1 -- 172.21.15.97:0/595665966 shutdown_connections 2024-03-27T10:53:14.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:14.898+0000 7f51f3c00640 1 -- 172.21.15.97:0/595665966 wait complete. 2024-03-27T10:53:14.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:14.898+0000 7f51f3c00640 1 Processor -- start 2024-03-27T10:53:14.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:14.899+0000 7f51f3c00640 1 -- start start 2024-03-27T10:53:14.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:14.899+0000 7f51f3c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51ec103d90 0x7f51ec19a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:53:14.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:14.899+0000 7f51f3c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51ec104f80 0x7f51ec19a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:53:14.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:14.899+0000 7f51f3c00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f51ec105940 0x7f51ec1a1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:53:14.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:14.899+0000 7f51f3c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f51ec10f980 con 0x7f51ec103d90 2024-03-27T10:53:14.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:14.899+0000 7f51f3c00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f51ec10f800 con 0x7f51ec104f80 2024-03-27T10:53:14.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:14.899+0000 7f51f3c00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f51ec10fb00 con 0x7f51ec105940 2024-03-27T10:53:14.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:14.899+0000 7f51f1200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51ec103d90 0x7f51ec19a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:53:14.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:14.899+0000 7f51f1200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51ec103d90 0x7f51ec19a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51700/0 (socket says 172.21.15.97:51700) 2024-03-27T10:53:14.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:14.899+0000 7f51f1200640 1 -- 172.21.15.97:0/1714977752 learned_addr learned my addr 172.21.15.97:0/1714977752 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:53:14.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:14.900+0000 7f51ebe00640 1 --2- 172.21.15.97:0/1714977752 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51ec104f80 0x7f51ec19a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:53:14.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:14.900+0000 7f51f1200640 1 -- 172.21.15.97:0/1714977752 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f51ec105940 msgr2=0x7f51ec1a1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:14.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:14.900+0000 7f51f1200640 1 --2- 172.21.15.97:0/1714977752 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f51ec105940 0x7f51ec1a1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:14.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:14.900+0000 7f51f1200640 1 -- 172.21.15.97:0/1714977752 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51ec104f80 msgr2=0x7f51ec19a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:14.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:14.900+0000 7f51f1200640 1 --2- 172.21.15.97:0/1714977752 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51ec104f80 0x7f51ec19a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:14.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:14.900+0000 7f51f1200640 1 -- 172.21.15.97:0/1714977752 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51ec1a1c70 con 0x7f51ec103d90 2024-03-27T10:53:14.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:14.900+0000 7f51ebe00640 1 --2- 172.21.15.97:0/1714977752 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51ec104f80 0x7f51ec19a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:53:14.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:14.900+0000 7f51f1200640 1 --2- 172.21.15.97:0/1714977752 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51ec103d90 0x7f51ec19a050 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f51dc00b9c0 tx=0x7f51dc00be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:53:14.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:14.900+0000 7f51e9600640 1 -- 172.21.15.97:0/1714977752 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51dc019070 con 0x7f51ec103d90 2024-03-27T10:53:14.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:14.900+0000 7f51e9600640 1 -- 172.21.15.97:0/1714977752 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f51dc00cba0 con 0x7f51ec103d90 2024-03-27T10:53:14.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:14.900+0000 7f51e9600640 1 -- 172.21.15.97:0/1714977752 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51dc0125a0 con 0x7f51ec103d90 2024-03-27T10:53:14.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:14.900+0000 7f51f3c00640 1 -- 172.21.15.97:0/1714977752 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51ec1a1f60 con 0x7f51ec103d90 2024-03-27T10:53:14.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:14.900+0000 7f51f3c00640 1 -- 172.21.15.97:0/1714977752 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51ec1a2450 con 0x7f51ec103d90 2024-03-27T10:53:14.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:14.902+0000 7f51f3c00640 1 -- 172.21.15.97:0/1714977752 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51ec10b150 con 0x7f51ec103d90 2024-03-27T10:53:14.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:14.905+0000 7f51e9600640 1 -- 172.21.15.97:0/1714977752 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138370+0+0 (secure 0 0 0) 0x7f51dc01c070 con 0x7f51ec103d90 2024-03-27T10:53:14.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:14.906+0000 7f51e9600640 1 --2- 172.21.15.97:0/1714977752 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f51d00a3610 0x7f51d00a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:53:14.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:14.907+0000 7f51e9600640 1 -- 172.21.15.97:0/1714977752 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7127+0+0 (secure 0 0 0) 0x7f51dc0cf560 con 0x7f51ec103d90 2024-03-27T10:53:14.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:14.907+0000 7f51ebe00640 1 --2- 172.21.15.97:0/1714977752 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f51d00a3610 0x7f51d00a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:53:14.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:14.907+0000 7f51ebe00640 1 --2- 172.21.15.97:0/1714977752 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f51d00a3610 0x7f51d00a5ad0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f51d4036040 tx=0x7f51d4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:53:14.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:14.909+0000 7f51e9600640 1 -- 172.21.15.97:0/1714977752 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f51dc09c530 con 0x7f51ec103d90 2024-03-27T10:53:15.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:15.108+0000 7f51f3c00640 1 -- 172.21.15.97:0/1714977752 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f51ec02d5c0 con 0x7f51ec103d90 2024-03-27T10:53:15.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:15.109+0000 7f51e9600640 1 -- 172.21.15.97:0/1714977752 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) ==== 93+0+10 (secure 0 0 0) 0x7f51dc09be30 con 0x7f51ec103d90 2024-03-27T10:53:15.082 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-03-27T10:53:15.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:15.112+0000 7f51f3c00640 1 -- 172.21.15.97:0/1714977752 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f51d00a3610 msgr2=0x7f51d00a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:15.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:15.112+0000 7f51f3c00640 1 --2- 172.21.15.97:0/1714977752 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f51d00a3610 0x7f51d00a5ad0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f51d4036040 tx=0x7f51d4005fd0 comp rx=0 tx=0).stop 2024-03-27T10:53:15.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:15.112+0000 7f51f3c00640 1 -- 172.21.15.97:0/1714977752 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51ec103d90 msgr2=0x7f51ec19a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:15.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:15.112+0000 7f51f3c00640 1 --2- 172.21.15.97:0/1714977752 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51ec103d90 0x7f51ec19a050 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f51dc00b9c0 tx=0x7f51dc00be80 comp rx=0 tx=0).stop 2024-03-27T10:53:15.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:15.112+0000 7f51f3c00640 1 -- 172.21.15.97:0/1714977752 shutdown_connections 2024-03-27T10:53:15.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:15.112+0000 7f51f3c00640 1 --2- 172.21.15.97:0/1714977752 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f51ec105940 0x7f51ec1a1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:15.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:15.112+0000 7f51f3c00640 1 --2- 172.21.15.97:0/1714977752 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f51d00a3610 0x7f51d00a5ad0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:15.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:15.112+0000 7f51f3c00640 1 --2- 172.21.15.97:0/1714977752 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51ec104f80 0x7f51ec19a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:15.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:15.112+0000 7f51f3c00640 1 --2- 172.21.15.97:0/1714977752 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51ec103d90 0x7f51ec19a050 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:15.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:15.112+0000 7f51f3c00640 1 -- 172.21.15.97:0/1714977752 >> 172.21.15.97:0/1714977752 conn(0x7f51ec0ff300 msgr2=0x7f51ec108320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:53:15.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:15.112+0000 7f51f3c00640 1 -- 172.21.15.97:0/1714977752 shutdown_connections 2024-03-27T10:53:15.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:15.112+0000 7f51f3c00640 1 -- 172.21.15.97:0/1714977752 wait complete. 2024-03-27T10:53:15.108 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-27T10:53:15.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:15.349+0000 7fe37c200640 1 -- 172.21.15.97:0/1188354232 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe374104f80 msgr2=0x7fe374105400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:15.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:15.349+0000 7fe37c200640 1 --2- 172.21.15.97:0/1188354232 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe374104f80 0x7fe374105400 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe368009900 tx=0x7fe36802bc30 comp rx=0 tx=0).stop 2024-03-27T10:53:15.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:15.349+0000 7fe37c200640 1 -- 172.21.15.97:0/1188354232 shutdown_connections 2024-03-27T10:53:15.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:15.349+0000 7fe37c200640 1 --2- 172.21.15.97:0/1188354232 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe374105940 0x7fe37410c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:15.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:15.349+0000 7fe37c200640 1 --2- 172.21.15.97:0/1188354232 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe374104f80 0x7fe374105400 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:15.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:15.349+0000 7fe37c200640 1 --2- 172.21.15.97:0/1188354232 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe374103d90 0x7fe3741041b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:15.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:15.349+0000 7fe37c200640 1 -- 172.21.15.97:0/1188354232 >> 172.21.15.97:0/1188354232 conn(0x7fe3740ff300 msgr2=0x7fe374101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:53:15.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:15.349+0000 7fe37c200640 1 -- 172.21.15.97:0/1188354232 shutdown_connections 2024-03-27T10:53:15.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:15.349+0000 7fe37c200640 1 -- 172.21.15.97:0/1188354232 wait complete. 2024-03-27T10:53:15.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:15.350+0000 7fe37c200640 1 Processor -- start 2024-03-27T10:53:15.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:15.350+0000 7fe37c200640 1 -- start start 2024-03-27T10:53:15.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:15.350+0000 7fe37c200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe374103d90 0x7fe374199f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:53:15.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:15.350+0000 7fe37c200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe374104f80 0x7fe37419a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:53:15.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:15.350+0000 7fe37c200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe374105940 0x7fe3741a1510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:53:15.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:15.350+0000 7fe37c200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe37410f910 con 0x7fe374105940 2024-03-27T10:53:15.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:15.350+0000 7fe37c200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe37410f790 con 0x7fe374104f80 2024-03-27T10:53:15.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:15.350+0000 7fe37c200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe37410fa90 con 0x7fe374103d90 2024-03-27T10:53:15.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:15.351+0000 7fe37a200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe374105940 0x7fe3741a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:53:15.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:15.351+0000 7fe37a200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe374105940 0x7fe3741a1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51718/0 (socket says 172.21.15.97:51718) 2024-03-27T10:53:15.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:15.351+0000 7fe37a200640 1 -- 172.21.15.97:0/2983414014 learned_addr learned my addr 172.21.15.97:0/2983414014 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:53:15.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:15.351+0000 7fe379800640 1 --2- 172.21.15.97:0/2983414014 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe374103d90 0x7fe374199f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:53:15.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:15.351+0000 7fe378e00640 1 --2- 172.21.15.97:0/2983414014 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe374104f80 0x7fe37419a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:53:15.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:15.351+0000 7fe37a200640 1 -- 172.21.15.97:0/2983414014 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe374103d90 msgr2=0x7fe374199f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:15.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:15.351+0000 7fe37a200640 1 --2- 172.21.15.97:0/2983414014 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe374103d90 0x7fe374199f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:15.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:15.351+0000 7fe37a200640 1 -- 172.21.15.97:0/2983414014 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe374104f80 msgr2=0x7fe37419a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:15.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:15.351+0000 7fe37a200640 1 --2- 172.21.15.97:0/2983414014 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe374104f80 0x7fe37419a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:15.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:15.351+0000 7fe37a200640 1 -- 172.21.15.97:0/2983414014 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3741a1b80 con 0x7fe374105940 2024-03-27T10:53:15.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:15.351+0000 7fe378e00640 1 --2- 172.21.15.97:0/2983414014 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe374104f80 0x7fe37419a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:53:15.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:15.351+0000 7fe37a200640 1 --2- 172.21.15.97:0/2983414014 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe374105940 0x7fe3741a1510 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe36c00eaa0 tx=0x7fe36c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:53:15.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:15.352+0000 7fe372000640 1 -- 172.21.15.97:0/2983414014 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe36c00cdb0 con 0x7fe374105940 2024-03-27T10:53:15.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:15.352+0000 7fe372000640 1 -- 172.21.15.97:0/2983414014 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe36c005020 con 0x7fe374105940 2024-03-27T10:53:15.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:15.352+0000 7fe372000640 1 -- 172.21.15.97:0/2983414014 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe36c0116d0 con 0x7fe374105940 2024-03-27T10:53:15.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:15.352+0000 7fe37c200640 1 -- 172.21.15.97:0/2983414014 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3741a1e70 con 0x7fe374105940 2024-03-27T10:53:15.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:15.352+0000 7fe37c200640 1 -- 172.21.15.97:0/2983414014 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3741a2330 con 0x7fe374105940 2024-03-27T10:53:15.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:15.353+0000 7fe37c200640 1 -- 172.21.15.97:0/2983414014 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe37410b0b0 con 0x7fe374105940 2024-03-27T10:53:15.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:15.355+0000 7fe372000640 1 -- 172.21.15.97:0/2983414014 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138370+0+0 (secure 0 0 0) 0x7fe36c0051d0 con 0x7fe374105940 2024-03-27T10:53:15.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:15.357+0000 7fe372000640 1 --2- 172.21.15.97:0/2983414014 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fe35c0a3610 0x7fe35c0a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:53:15.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:15.357+0000 7fe372000640 1 -- 172.21.15.97:0/2983414014 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7127+0+0 (secure 0 0 0) 0x7fe36c0cfe30 con 0x7fe374105940 2024-03-27T10:53:15.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:15.357+0000 7fe379800640 1 --2- 172.21.15.97:0/2983414014 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fe35c0a3610 0x7fe35c0a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:53:15.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:15.357+0000 7fe379800640 1 --2- 172.21.15.97:0/2983414014 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fe35c0a3610 0x7fe35c0a5ad0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe364005d00 tx=0x7fe364005c50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:53:15.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:15.360+0000 7fe372000640 1 -- 172.21.15.97:0/2983414014 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7fe36c016030 con 0x7fe374105940 2024-03-27T10:53:15.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:15 smithi097 ceph-mon[31859]: pgmap v333: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T10:53:15.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:15 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/2926691208' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T10:53:15.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:15 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/1714977752' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-27T10:53:15.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:15 smithi177 ceph-mon[35294]: pgmap v333: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T10:53:15.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:15 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/2926691208' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T10:53:15.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:15 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/1714977752' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-27T10:53:15.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:15 smithi114 ceph-mon[35338]: pgmap v333: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T10:53:15.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:15 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/2926691208' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T10:53:15.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:15 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/1714977752' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-27T10:53:15.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:15.598+0000 7fe37c200640 1 -- 172.21.15.97:0/2983414014 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe37402d370 con 0x7fe374105940 2024-03-27T10:53:15.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:15.598+0000 7fe372000640 1 -- 172.21.15.97:0/2983414014 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7fe36c09ce00 con 0x7fe374105940 2024-03-27T10:53:15.572 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T10:53:15.572 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-03-27T10:53:15.572 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 1 2024-03-27T10:53:15.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:15.601+0000 7fe37c200640 1 -- 172.21.15.97:0/2983414014 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fe35c0a3610 msgr2=0x7fe35c0a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:15.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:15.601+0000 7fe37c200640 1 --2- 172.21.15.97:0/2983414014 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fe35c0a3610 0x7fe35c0a5ad0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe364005d00 tx=0x7fe364005c50 comp rx=0 tx=0).stop 2024-03-27T10:53:15.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:15.601+0000 7fe37c200640 1 -- 172.21.15.97:0/2983414014 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe374105940 msgr2=0x7fe3741a1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:15.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:15.601+0000 7fe37c200640 1 --2- 172.21.15.97:0/2983414014 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe374105940 0x7fe3741a1510 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe36c00eaa0 tx=0x7fe36c00ef60 comp rx=0 tx=0).stop 2024-03-27T10:53:15.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:15.601+0000 7fe37c200640 1 -- 172.21.15.97:0/2983414014 shutdown_connections 2024-03-27T10:53:15.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:15.601+0000 7fe37c200640 1 --2- 172.21.15.97:0/2983414014 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe374105940 0x7fe3741a1510 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:15.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:15.601+0000 7fe37c200640 1 --2- 172.21.15.97:0/2983414014 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fe35c0a3610 0x7fe35c0a5ad0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:15.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:15.601+0000 7fe37c200640 1 --2- 172.21.15.97:0/2983414014 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe374104f80 0x7fe37419a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:15.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:15.601+0000 7fe37c200640 1 --2- 172.21.15.97:0/2983414014 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe374103d90 0x7fe374199f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:15.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:15.601+0000 7fe37c200640 1 -- 172.21.15.97:0/2983414014 >> 172.21.15.97:0/2983414014 conn(0x7fe3740ff300 msgr2=0x7fe374108280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:53:15.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:15.601+0000 7fe37c200640 1 -- 172.21.15.97:0/2983414014 shutdown_connections 2024-03-27T10:53:15.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:15.601+0000 7fe37c200640 1 -- 172.21.15.97:0/2983414014 wait complete. 2024-03-27T10:53:16.104 INFO:journalctl@ceph.mds.l.smithi177.stdout:Mar 27 10:53:15 smithi177 systemd[1]: Starting Ceph mds.l for adca88c4-ec26-11ee-95d0-87774f69a715... 2024-03-27T10:53:16.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:16 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/2983414014' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T10:53:16.416 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:16 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/2983414014' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T10:53:16.416 INFO:journalctl@ceph.mds.l.smithi177.stdout:Mar 27 10:53:16 smithi177 podman[65822]: 2024-03-27 10:53:16.1057657 +0000 UTC m=+0.034955804 image pull b379bbd8da6b5d0fa9a3523941efac213b6815e1cbf05c6507954f0a3fe7a801 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772 2024-03-27T10:53:16.416 INFO:journalctl@ceph.mds.l.smithi177.stdout:Mar 27 10:53:16 smithi177 podman[65822]: 2024-03-27 10:53:16.206667349 +0000 UTC m=+0.135857459 container create 5ebaf095548b340079f8207835a4684bf862e553ad22da4c434b12885197431d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-l, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240326) 2024-03-27T10:53:16.416 INFO:journalctl@ceph.mds.l.smithi177.stdout:Mar 27 10:53:16 smithi177 podman[65822]: 2024-03-27 10:53:16.44366356 +0000 UTC m=+0.372853682 container init 5ebaf095548b340079f8207835a4684bf862e553ad22da4c434b12885197431d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-l, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-27T10:53:16.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:16 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/2983414014' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T10:53:16.677 INFO:journalctl@ceph.mds.l.smithi177.stdout:Mar 27 10:53:16 smithi177 podman[65822]: 2024-03-27 10:53:16.455151464 +0000 UTC m=+0.384341617 container start 5ebaf095548b340079f8207835a4684bf862e553ad22da4c434b12885197431d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-l, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True) 2024-03-27T10:53:16.678 INFO:journalctl@ceph.mds.l.smithi177.stdout:Mar 27 10:53:16 smithi177 ceph-mds[65840]: -- 172.21.15.177:0/699413818 <== mon.1 v2:172.21.15.177:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5561efc0fa40 con 0x5561efbe2c00 2024-03-27T10:53:16.678 INFO:journalctl@ceph.mds.l.smithi177.stdout:Mar 27 10:53:16 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-l[65836]: starting mds.l at 2024-03-27T10:53:16.678 INFO:journalctl@ceph.mds.l.smithi177.stdout:Mar 27 10:53:16 smithi177 bash[65822]: 5ebaf095548b340079f8207835a4684bf862e553ad22da4c434b12885197431d 2024-03-27T10:53:16.678 INFO:journalctl@ceph.mds.l.smithi177.stdout:Mar 27 10:53:16 smithi177 systemd[1]: Started Ceph mds.l for adca88c4-ec26-11ee-95d0-87774f69a715. 2024-03-27T10:53:17.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:17 smithi097 ceph-mon[31859]: pgmap v334: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T10:53:17.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:17 smithi097 ceph-mon[31859]: mds.? [v2:172.21.15.177:6832/3787981632,v1:172.21.15.177:6833/3787981632] up:boot 2024-03-27T10:53:17.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:17 smithi097 ceph-mon[31859]: fsmap 1 up:standby 2024-03-27T10:53:17.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:17 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-27T10:53:17.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:17 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:17.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:17 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:17.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:17 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:17.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:17 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-27T10:53:17.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:17 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-27T10:53:17.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:17 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:53:17.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:17 smithi177 ceph-mon[35294]: pgmap v334: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T10:53:17.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:17 smithi177 ceph-mon[35294]: mds.? [v2:172.21.15.177:6832/3787981632,v1:172.21.15.177:6833/3787981632] up:boot 2024-03-27T10:53:17.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:17 smithi177 ceph-mon[35294]: fsmap 1 up:standby 2024-03-27T10:53:17.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:17 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-27T10:53:17.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:17 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:17.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:17 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:17.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:17 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:17.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:17 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-27T10:53:17.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:17 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-27T10:53:17.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:17 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:53:17.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:17 smithi114 ceph-mon[35338]: pgmap v334: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T10:53:17.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:17 smithi114 ceph-mon[35338]: mds.? [v2:172.21.15.177:6832/3787981632,v1:172.21.15.177:6833/3787981632] up:boot 2024-03-27T10:53:17.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:17 smithi114 ceph-mon[35338]: fsmap 1 up:standby 2024-03-27T10:53:17.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:17 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-27T10:53:17.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:17 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:17.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:17 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:17.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:17 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:17.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:17 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-27T10:53:17.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:17 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-27T10:53:17.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:17 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:53:17.592 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-27T10:53:17.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:17.839+0000 7f429b000640 1 -- 172.21.15.97:0/195159030 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4294109180 msgr2=0x7f4294109600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:17.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:17.839+0000 7f429b000640 1 --2- 172.21.15.97:0/195159030 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4294109180 0x7f4294109600 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4280009900 tx=0x7f428002bc30 comp rx=0 tx=0).stop 2024-03-27T10:53:17.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:17.840+0000 7f429b000640 1 -- 172.21.15.97:0/195159030 shutdown_connections 2024-03-27T10:53:17.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:17.840+0000 7f429b000640 1 --2- 172.21.15.97:0/195159030 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4294109b40 0x7f42941103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:17.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:17.840+0000 7f429b000640 1 --2- 172.21.15.97:0/195159030 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4294109180 0x7f4294109600 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:17.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:17.840+0000 7f429b000640 1 --2- 172.21.15.97:0/195159030 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4294107f90 0x7f42941083b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:17.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:17.840+0000 7f429b000640 1 -- 172.21.15.97:0/195159030 >> 172.21.15.97:0/195159030 conn(0x7f4294076f20 msgr2=0x7f4294079360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:53:17.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:17.840+0000 7f429b000640 1 -- 172.21.15.97:0/195159030 shutdown_connections 2024-03-27T10:53:17.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:17.840+0000 7f429b000640 1 -- 172.21.15.97:0/195159030 wait complete. 2024-03-27T10:53:17.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:17.841+0000 7f429b000640 1 Processor -- start 2024-03-27T10:53:17.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:17.841+0000 7f429b000640 1 -- start start 2024-03-27T10:53:17.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:17.841+0000 7f429b000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4294107f90 0x7f429419e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:53:17.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:17.841+0000 7f429b000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4294109180 0x7f429419e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:53:17.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:17.841+0000 7f429b000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4294109b40 0x7f42941a5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:53:17.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:17.841+0000 7f429b000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4294113b20 con 0x7f4294107f90 2024-03-27T10:53:17.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:17.841+0000 7f429b000640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f42941139a0 con 0x7f4294109b40 2024-03-27T10:53:17.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:17.841+0000 7f429b000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f4294113ca0 con 0x7f4294109180 2024-03-27T10:53:17.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:17.841+0000 7f4299c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4294107f90 0x7f429419e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:53:17.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:17.841+0000 7f4299c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4294107f90 0x7f429419e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51736/0 (socket says 172.21.15.97:51736) 2024-03-27T10:53:17.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:17.841+0000 7f4299c00640 1 -- 172.21.15.97:0/2595852951 learned_addr learned my addr 172.21.15.97:0/2595852951 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:53:17.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:17.842+0000 7f429a600640 1 --2- 172.21.15.97:0/2595852951 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4294109b40 0x7f42941a5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:53:17.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:17.842+0000 7f4299200640 1 --2- 172.21.15.97:0/2595852951 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4294109180 0x7f429419e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:53:17.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:17.842+0000 7f4299c00640 1 -- 172.21.15.97:0/2595852951 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4294109180 msgr2=0x7f429419e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:17.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:17.842+0000 7f4299c00640 1 --2- 172.21.15.97:0/2595852951 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4294109180 0x7f429419e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:17.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:17.842+0000 7f4299c00640 1 -- 172.21.15.97:0/2595852951 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4294109b40 msgr2=0x7f42941a5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:17.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:17.842+0000 7f4299c00640 1 --2- 172.21.15.97:0/2595852951 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4294109b40 0x7f42941a5690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:17.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:17.842+0000 7f4299c00640 1 -- 172.21.15.97:0/2595852951 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42941a5d00 con 0x7f4294107f90 2024-03-27T10:53:17.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:17.842+0000 7f4299c00640 1 --2- 172.21.15.97:0/2595852951 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4294107f90 0x7f429419e0e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f428800c970 tx=0x7f428800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:53:17.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:17.842+0000 7f4292000640 1 -- 172.21.15.97:0/2595852951 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4288004220 con 0x7f4294107f90 2024-03-27T10:53:17.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:17.842+0000 7f4292000640 1 -- 172.21.15.97:0/2595852951 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f42880043c0 con 0x7f4294107f90 2024-03-27T10:53:17.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:17.842+0000 7f429b000640 1 -- 172.21.15.97:0/2595852951 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42941a5ff0 con 0x7f4294107f90 2024-03-27T10:53:17.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:17.842+0000 7f4292000640 1 -- 172.21.15.97:0/2595852951 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4288004a00 con 0x7f4294107f90 2024-03-27T10:53:17.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:17.842+0000 7f429b000640 1 -- 172.21.15.97:0/2595852951 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42941a6450 con 0x7f4294107f90 2024-03-27T10:53:17.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:17.843+0000 7f429b000640 1 -- 172.21.15.97:0/2595852951 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f429410f2b0 con 0x7f4294107f90 2024-03-27T10:53:17.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:17.845+0000 7f4292000640 1 -- 172.21.15.97:0/2595852951 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138370+0+0 (secure 0 0 0) 0x7f42880079b0 con 0x7f4294107f90 2024-03-27T10:53:17.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:17.847+0000 7f4292000640 1 --2- 172.21.15.97:0/2595852951 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f427c0a3610 0x7f427c0a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:53:17.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:17.847+0000 7f4292000640 1 -- 172.21.15.97:0/2595852951 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7127+0+0 (secure 0 0 0) 0x7f42880cf730 con 0x7f4294107f90 2024-03-27T10:53:17.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:17.847+0000 7f4299200640 1 --2- 172.21.15.97:0/2595852951 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f427c0a3610 0x7f427c0a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:53:17.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:17.848+0000 7f4299200640 1 --2- 172.21.15.97:0/2595852951 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f427c0a3610 0x7f427c0a5ad0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f42800096c0 tx=0x7f4280005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:53:17.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:17.851+0000 7f4292000640 1 -- 172.21.15.97:0/2595852951 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f4288014030 con 0x7f4294107f90 2024-03-27T10:53:18.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:18.089+0000 7f429b000640 1 -- 172.21.15.97:0/2595852951 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f429402d370 con 0x7f4294107f90 2024-03-27T10:53:18.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:18.090+0000 7f4292000640 1 -- 172.21.15.97:0/2595852951 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7f428809c700 con 0x7f4294107f90 2024-03-27T10:53:18.063 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T10:53:18.063 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3787981632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3787981632},{"type":"v1","addr":"172.21.15.177:6833","nonce":3787981632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-03-27T10:53:18.063 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 2 2024-03-27T10:53:18.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:18.092+0000 7f429b000640 1 -- 172.21.15.97:0/2595852951 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f427c0a3610 msgr2=0x7f427c0a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:18.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:18.092+0000 7f429b000640 1 --2- 172.21.15.97:0/2595852951 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f427c0a3610 0x7f427c0a5ad0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f42800096c0 tx=0x7f4280005fd0 comp rx=0 tx=0).stop 2024-03-27T10:53:18.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:18.092+0000 7f429b000640 1 -- 172.21.15.97:0/2595852951 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4294107f90 msgr2=0x7f429419e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:18.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:18.092+0000 7f429b000640 1 --2- 172.21.15.97:0/2595852951 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4294107f90 0x7f429419e0e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f428800c970 tx=0x7f428800ce30 comp rx=0 tx=0).stop 2024-03-27T10:53:18.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:18.092+0000 7f429b000640 1 -- 172.21.15.97:0/2595852951 shutdown_connections 2024-03-27T10:53:18.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:18.092+0000 7f429b000640 1 --2- 172.21.15.97:0/2595852951 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f427c0a3610 0x7f427c0a5ad0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:18.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:18.092+0000 7f429b000640 1 --2- 172.21.15.97:0/2595852951 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4294109b40 0x7f42941a5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:18.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:18.092+0000 7f429b000640 1 --2- 172.21.15.97:0/2595852951 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4294109180 0x7f429419e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:18.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:18.092+0000 7f429b000640 1 --2- 172.21.15.97:0/2595852951 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4294107f90 0x7f429419e0e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:18.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:18.092+0000 7f429b000640 1 -- 172.21.15.97:0/2595852951 >> 172.21.15.97:0/2595852951 conn(0x7f4294076f20 msgr2=0x7f429410c480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:53:18.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:18.092+0000 7f429b000640 1 -- 172.21.15.97:0/2595852951 shutdown_connections 2024-03-27T10:53:18.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:18.092+0000 7f429b000640 1 -- 172.21.15.97:0/2595852951 wait complete. 2024-03-27T10:53:18.268 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:18 smithi114 ceph-mon[35338]: Deploying daemon mds.k on smithi114 2024-03-27T10:53:18.268 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:18 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/2595852951' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T10:53:18.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:18 smithi097 ceph-mon[31859]: Deploying daemon mds.k on smithi114 2024-03-27T10:53:18.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:18 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/2595852951' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T10:53:18.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:18 smithi177 ceph-mon[35294]: Deploying daemon mds.k on smithi114 2024-03-27T10:53:18.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:18 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/2595852951' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T10:53:19.267 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:19 smithi114 ceph-mon[35338]: pgmap v335: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T10:53:19.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:19 smithi097 ceph-mon[31859]: pgmap v335: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T10:53:19.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:19 smithi177 ceph-mon[35294]: pgmap v335: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T10:53:20.086 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-27T10:53:20.203 INFO:journalctl@ceph.mds.k.smithi114.stdout:Mar 27 10:53:19 smithi114 systemd[1]: Starting Ceph mds.k for adca88c4-ec26-11ee-95d0-87774f69a715... 2024-03-27T10:53:20.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:20.330+0000 7fd604c00640 1 -- 172.21.15.97:0/898506451 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6001094b0 msgr2=0x7fd60010fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:20.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:20.330+0000 7fd604c00640 1 --2- 172.21.15.97:0/898506451 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6001094b0 0x7fd60010fe90 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd5e8009900 tx=0x7fd5e802bc30 comp rx=0 tx=0).stop 2024-03-27T10:53:20.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:20.330+0000 7fd604c00640 1 -- 172.21.15.97:0/898506451 shutdown_connections 2024-03-27T10:53:20.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:20.330+0000 7fd604c00640 1 --2- 172.21.15.97:0/898506451 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd6001103d0 0x7fd6001127c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:20.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:20.330+0000 7fd604c00640 1 --2- 172.21.15.97:0/898506451 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6001094b0 0x7fd60010fe90 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:20.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:20.330+0000 7fd604c00640 1 --2- 172.21.15.97:0/898506451 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd600108b50 0x7fd600108f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:20.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:20.330+0000 7fd604c00640 1 -- 172.21.15.97:0/898506451 >> 172.21.15.97:0/898506451 conn(0x7fd60006ed40 msgr2=0x7fd600071160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:53:20.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:20.330+0000 7fd604c00640 1 -- 172.21.15.97:0/898506451 shutdown_connections 2024-03-27T10:53:20.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:20.330+0000 7fd604c00640 1 -- 172.21.15.97:0/898506451 wait complete. 2024-03-27T10:53:20.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:20.331+0000 7fd604c00640 1 Processor -- start 2024-03-27T10:53:20.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:20.331+0000 7fd604c00640 1 -- start start 2024-03-27T10:53:20.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:20.331+0000 7fd604c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd600108b50 0x7fd60010b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:53:20.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:20.331+0000 7fd604c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd6001094b0 0x7fd600109e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:53:20.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:20.331+0000 7fd604c00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd6001103d0 0x7fd60010a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:53:20.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:20.331+0000 7fd604c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd600115ee0 con 0x7fd600108b50 2024-03-27T10:53:20.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:20.331+0000 7fd604c00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd600115d60 con 0x7fd6001094b0 2024-03-27T10:53:20.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:20.331+0000 7fd604c00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd600116060 con 0x7fd6001103d0 2024-03-27T10:53:20.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:20.332+0000 7fd5fde00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd600108b50 0x7fd60010b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:53:20.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:20.332+0000 7fd5fde00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd600108b50 0x7fd60010b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38346/0 (socket says 172.21.15.97:38346) 2024-03-27T10:53:20.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:20.332+0000 7fd5fde00640 1 -- 172.21.15.97:0/3535972522 learned_addr learned my addr 172.21.15.97:0/3535972522 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:53:20.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:20.332+0000 7fd5f5400640 1 --2- 172.21.15.97:0/3535972522 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd6001094b0 0x7fd600109e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:53:20.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:20.332+0000 7fd5fe800640 1 --2- 172.21.15.97:0/3535972522 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd6001103d0 0x7fd60010a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:53:20.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:20.332+0000 7fd5fde00640 1 -- 172.21.15.97:0/3535972522 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd6001103d0 msgr2=0x7fd60010a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:20.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:20.332+0000 7fd5fde00640 1 --2- 172.21.15.97:0/3535972522 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd6001103d0 0x7fd60010a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:20.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:20.332+0000 7fd5fde00640 1 -- 172.21.15.97:0/3535972522 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd6001094b0 msgr2=0x7fd600109e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:20.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:20.332+0000 7fd5fde00640 1 --2- 172.21.15.97:0/3535972522 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd6001094b0 0x7fd600109e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:20.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:20.332+0000 7fd5fde00640 1 -- 172.21.15.97:0/3535972522 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd60010abf0 con 0x7fd600108b50 2024-03-27T10:53:20.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:20.332+0000 7fd5f5400640 1 --2- 172.21.15.97:0/3535972522 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd6001094b0 0x7fd600109e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T10:53:20.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:20.332+0000 7fd5fe800640 1 --2- 172.21.15.97:0/3535972522 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd6001103d0 0x7fd60010a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:53:20.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:20.332+0000 7fd5fde00640 1 --2- 172.21.15.97:0/3535972522 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd600108b50 0x7fd60010b7e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd5f000c970 tx=0x7fd5f000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:53:20.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:20.333+0000 7fd5f7400640 1 -- 172.21.15.97:0/3535972522 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f0004220 con 0x7fd600108b50 2024-03-27T10:53:20.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:20.333+0000 7fd5f7400640 1 -- 172.21.15.97:0/3535972522 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd5f00043c0 con 0x7fd600108b50 2024-03-27T10:53:20.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:20.333+0000 7fd5f7400640 1 -- 172.21.15.97:0/3535972522 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f000f450 con 0x7fd600108b50 2024-03-27T10:53:20.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:20.333+0000 7fd604c00640 1 -- 172.21.15.97:0/3535972522 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6001aa600 con 0x7fd600108b50 2024-03-27T10:53:20.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:20.333+0000 7fd604c00640 1 -- 172.21.15.97:0/3535972522 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6001aab00 con 0x7fd600108b50 2024-03-27T10:53:20.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:20.334+0000 7fd604c00640 1 -- 172.21.15.97:0/3535972522 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd60010eca0 con 0x7fd600108b50 2024-03-27T10:53:20.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:20.336+0000 7fd5f7400640 1 -- 172.21.15.97:0/3535972522 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138370+0+0 (secure 0 0 0) 0x7fd5f00079b0 con 0x7fd600108b50 2024-03-27T10:53:20.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:20.337+0000 7fd5f7400640 1 --2- 172.21.15.97:0/3535972522 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fd5e40a3690 0x7fd5e40a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:53:20.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:20.338+0000 7fd5f7400640 1 -- 172.21.15.97:0/3535972522 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7127+0+0 (secure 0 0 0) 0x7fd5f00cf7c0 con 0x7fd600108b50 2024-03-27T10:53:20.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:20.338+0000 7fd5f5400640 1 --2- 172.21.15.97:0/3535972522 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fd5e40a3690 0x7fd5e40a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:53:20.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:20.338+0000 7fd5f5400640 1 --2- 172.21.15.97:0/3535972522 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fd5e40a3690 0x7fd5e40a5b50 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd60010b550 tx=0x7fd5e8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:53:20.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:20.342+0000 7fd5f7400640 1 -- 172.21.15.97:0/3535972522 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7fd5f0014030 con 0x7fd600108b50 2024-03-27T10:53:20.487 INFO:journalctl@ceph.mds.k.smithi114.stdout:Mar 27 10:53:20 smithi114 podman[64930]: 2024-03-27 10:53:20.136559841 +0000 UTC m=+0.035176293 image pull b379bbd8da6b5d0fa9a3523941efac213b6815e1cbf05c6507954f0a3fe7a801 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772 2024-03-27T10:53:20.487 INFO:journalctl@ceph.mds.k.smithi114.stdout:Mar 27 10:53:20 smithi114 podman[64930]: 2024-03-27 10:53:20.26026615 +0000 UTC m=+0.158882640 container create 012040f065d4fea01dd5ced86d8fae509821420a41664023970e25053584d551 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-k, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0) 2024-03-27T10:53:20.487 INFO:journalctl@ceph.mds.k.smithi114.stdout:Mar 27 10:53:20 smithi114 podman[64930]: 2024-03-27 10:53:20.51406611 +0000 UTC m=+0.412682607 container init 012040f065d4fea01dd5ced86d8fae509821420a41664023970e25053584d551 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-k, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-rishabh-testing-20240326.131558-155268c, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-03-27T10:53:20.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:20.579+0000 7fd604c00640 1 -- 172.21.15.97:0/3535972522 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd60002d370 con 0x7fd600108b50 2024-03-27T10:53:20.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:20.579+0000 7fd5f7400640 1 -- 172.21.15.97:0/3535972522 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7fd5f009c790 con 0x7fd600108b50 2024-03-27T10:53:20.553 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T10:53:20.553 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3787981632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3787981632},{"type":"v1","addr":"172.21.15.177:6833","nonce":3787981632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-03-27T10:53:20.553 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 2 2024-03-27T10:53:20.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:20.582+0000 7fd604c00640 1 -- 172.21.15.97:0/3535972522 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fd5e40a3690 msgr2=0x7fd5e40a5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:20.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:20.582+0000 7fd604c00640 1 --2- 172.21.15.97:0/3535972522 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fd5e40a3690 0x7fd5e40a5b50 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd60010b550 tx=0x7fd5e8005fd0 comp rx=0 tx=0).stop 2024-03-27T10:53:20.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:20.582+0000 7fd604c00640 1 -- 172.21.15.97:0/3535972522 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd600108b50 msgr2=0x7fd60010b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:20.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:20.582+0000 7fd604c00640 1 --2- 172.21.15.97:0/3535972522 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd600108b50 0x7fd60010b7e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd5f000c970 tx=0x7fd5f000ce30 comp rx=0 tx=0).stop 2024-03-27T10:53:20.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:20.582+0000 7fd604c00640 1 -- 172.21.15.97:0/3535972522 shutdown_connections 2024-03-27T10:53:20.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:20.582+0000 7fd604c00640 1 --2- 172.21.15.97:0/3535972522 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd6001103d0 0x7fd60010a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:20.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:20.582+0000 7fd604c00640 1 --2- 172.21.15.97:0/3535972522 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fd5e40a3690 0x7fd5e40a5b50 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:20.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:20.582+0000 7fd604c00640 1 --2- 172.21.15.97:0/3535972522 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd6001094b0 0x7fd600109e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:20.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:20.582+0000 7fd604c00640 1 --2- 172.21.15.97:0/3535972522 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd600108b50 0x7fd60010b7e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:20.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:20.582+0000 7fd604c00640 1 -- 172.21.15.97:0/3535972522 >> 172.21.15.97:0/3535972522 conn(0x7fd60006ed40 msgr2=0x7fd60010be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:53:20.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:20.583+0000 7fd604c00640 1 -- 172.21.15.97:0/3535972522 shutdown_connections 2024-03-27T10:53:20.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:20.583+0000 7fd604c00640 1 -- 172.21.15.97:0/3535972522 wait complete. 2024-03-27T10:53:20.768 INFO:journalctl@ceph.mds.k.smithi114.stdout:Mar 27 10:53:20 smithi114 podman[64930]: 2024-03-27 10:53:20.523501066 +0000 UTC m=+0.422117534 container start 012040f065d4fea01dd5ced86d8fae509821420a41664023970e25053584d551 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-k, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-03-27T10:53:20.768 INFO:journalctl@ceph.mds.k.smithi114.stdout:Mar 27 10:53:20 smithi114 ceph-mds[64949]: -- 172.21.15.114:0/3572385052 <== mon.2 v2:172.21.15.114:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c338f49a40 con 0x55c338f1d000 2024-03-27T10:53:20.768 INFO:journalctl@ceph.mds.k.smithi114.stdout:Mar 27 10:53:20 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-k[64945]: starting mds.k at 2024-03-27T10:53:20.768 INFO:journalctl@ceph.mds.k.smithi114.stdout:Mar 27 10:53:20 smithi114 bash[64930]: 012040f065d4fea01dd5ced86d8fae509821420a41664023970e25053584d551 2024-03-27T10:53:20.768 INFO:journalctl@ceph.mds.k.smithi114.stdout:Mar 27 10:53:20 smithi114 systemd[1]: Started Ceph mds.k for adca88c4-ec26-11ee-95d0-87774f69a715. 2024-03-27T10:53:21.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:21 smithi097 ceph-mon[31859]: pgmap v336: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T10:53:21.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:21 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/3535972522' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T10:53:21.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:21 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:21.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:21 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:21.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:21 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:21.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:21 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-27T10:53:21.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:21 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-27T10:53:21.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:21 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:53:21.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:21 smithi177 ceph-mon[35294]: pgmap v336: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T10:53:21.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:21 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/3535972522' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T10:53:21.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:21 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:21.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:21 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:21.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:21 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:21.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:21 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-27T10:53:21.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:21 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-27T10:53:21.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:21 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:53:21.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:21 smithi114 ceph-mon[35338]: pgmap v336: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T10:53:21.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:21 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/3535972522' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T10:53:21.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:21 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:21.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:21 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:21.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:21 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:21.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:21 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-27T10:53:21.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:21 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-27T10:53:21.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:21 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:53:22.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:22 smithi097 ceph-mon[31859]: Deploying daemon mds.j on smithi097 2024-03-27T10:53:22.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:22 smithi097 ceph-mon[31859]: mds.? [v2:172.21.15.114:6832/4052517902,v1:172.21.15.114:6833/4052517902] up:boot 2024-03-27T10:53:22.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:22 smithi097 ceph-mon[31859]: fsmap 2 up:standby 2024-03-27T10:53:22.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:22 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-27T10:53:22.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:22 smithi177 ceph-mon[35294]: Deploying daemon mds.j on smithi097 2024-03-27T10:53:22.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:22 smithi177 ceph-mon[35294]: mds.? [v2:172.21.15.114:6832/4052517902,v1:172.21.15.114:6833/4052517902] up:boot 2024-03-27T10:53:22.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:22 smithi177 ceph-mon[35294]: fsmap 2 up:standby 2024-03-27T10:53:22.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:22 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-27T10:53:22.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:22 smithi114 ceph-mon[35338]: Deploying daemon mds.j on smithi097 2024-03-27T10:53:22.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:22 smithi114 ceph-mon[35338]: mds.? [v2:172.21.15.114:6832/4052517902,v1:172.21.15.114:6833/4052517902] up:boot 2024-03-27T10:53:22.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:22 smithi114 ceph-mon[35338]: fsmap 2 up:standby 2024-03-27T10:53:22.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:22 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-27T10:53:22.575 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-27T10:53:22.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:22.812+0000 7fec34800640 1 -- 172.21.15.97:0/1650312989 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fec2c108170 msgr2=0x7fec2c108590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:22.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:22.812+0000 7fec34800640 1 --2- 172.21.15.97:0/1650312989 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fec2c108170 0x7fec2c108590 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fec20009900 tx=0x7fec2002bc30 comp rx=0 tx=0).stop 2024-03-27T10:53:22.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:22.812+0000 7fec34800640 1 -- 172.21.15.97:0/1650312989 shutdown_connections 2024-03-27T10:53:22.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:22.812+0000 7fec34800640 1 --2- 172.21.15.97:0/1650312989 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fec2c109d20 0x7fec2c1105b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:22.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:22.812+0000 7fec34800640 1 --2- 172.21.15.97:0/1650312989 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fec2c109360 0x7fec2c1097e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:22.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:22.812+0000 7fec34800640 1 --2- 172.21.15.97:0/1650312989 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fec2c108170 0x7fec2c108590 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:22.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:22.812+0000 7fec34800640 1 -- 172.21.15.97:0/1650312989 >> 172.21.15.97:0/1650312989 conn(0x7fec2c077130 msgr2=0x7fec2c079550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:53:22.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:22.812+0000 7fec34800640 1 -- 172.21.15.97:0/1650312989 shutdown_connections 2024-03-27T10:53:22.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:22.812+0000 7fec34800640 1 -- 172.21.15.97:0/1650312989 wait complete. 2024-03-27T10:53:22.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:22.813+0000 7fec34800640 1 Processor -- start 2024-03-27T10:53:22.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:22.813+0000 7fec34800640 1 -- start start 2024-03-27T10:53:22.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:22.813+0000 7fec34800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fec2c108170 0x7fec2c19e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:53:22.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:22.814+0000 7fec34800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fec2c109360 0x7fec2c19e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:53:22.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:22.814+0000 7fec34800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fec2c109d20 0x7fec2c1a5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:53:22.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:22.814+0000 7fec34800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fec2c113cb0 con 0x7fec2c108170 2024-03-27T10:53:22.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:22.814+0000 7fec34800640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fec2c113b30 con 0x7fec2c109360 2024-03-27T10:53:22.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:22.814+0000 7fec34800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fec2c113e30 con 0x7fec2c109d20 2024-03-27T10:53:22.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:22.814+0000 7fec31e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fec2c108170 0x7fec2c19e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:53:22.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:22.814+0000 7fec31e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fec2c108170 0x7fec2c19e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38360/0 (socket says 172.21.15.97:38360) 2024-03-27T10:53:22.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:22.814+0000 7fec31e00640 1 -- 172.21.15.97:0/3240933702 learned_addr learned my addr 172.21.15.97:0/3240933702 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:53:22.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:22.814+0000 7fec31e00640 1 -- 172.21.15.97:0/3240933702 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fec2c109d20 msgr2=0x7fec2c1a5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:22.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:22.814+0000 7fec31400640 1 --2- 172.21.15.97:0/3240933702 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fec2c109360 0x7fec2c19e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:53:22.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:22.814+0000 7fec32800640 1 --2- 172.21.15.97:0/3240933702 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fec2c109d20 0x7fec2c1a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:53:22.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:22.814+0000 7fec31e00640 1 --2- 172.21.15.97:0/3240933702 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fec2c109d20 0x7fec2c1a5970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:22.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:22.814+0000 7fec31e00640 1 -- 172.21.15.97:0/3240933702 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fec2c109360 msgr2=0x7fec2c19e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:22.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:22.814+0000 7fec31e00640 1 --2- 172.21.15.97:0/3240933702 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fec2c109360 0x7fec2c19e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:22.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:22.814+0000 7fec31e00640 1 -- 172.21.15.97:0/3240933702 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec2c1a5fe0 con 0x7fec2c108170 2024-03-27T10:53:22.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:22.815+0000 7fec31e00640 1 --2- 172.21.15.97:0/3240933702 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fec2c108170 0x7fec2c19e3c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fec20000c00 tx=0x7fec20004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:53:22.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:22.815+0000 7fec26a00640 1 -- 172.21.15.97:0/3240933702 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec20043020 con 0x7fec2c108170 2024-03-27T10:53:22.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:22.815+0000 7fec26a00640 1 -- 172.21.15.97:0/3240933702 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fec20004440 con 0x7fec2c108170 2024-03-27T10:53:22.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:22.815+0000 7fec34800640 1 -- 172.21.15.97:0/3240933702 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec2c1a6270 con 0x7fec2c108170 2024-03-27T10:53:22.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:22.815+0000 7fec26a00640 1 -- 172.21.15.97:0/3240933702 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec20003c90 con 0x7fec2c108170 2024-03-27T10:53:22.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:22.815+0000 7fec34800640 1 -- 172.21.15.97:0/3240933702 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec2c1a67c0 con 0x7fec2c108170 2024-03-27T10:53:22.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:22.816+0000 7fec34800640 1 -- 172.21.15.97:0/3240933702 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec2c10f5b0 con 0x7fec2c108170 2024-03-27T10:53:22.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:22.818+0000 7fec26a00640 1 -- 172.21.15.97:0/3240933702 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138370+0+0 (secure 0 0 0) 0x7fec20046020 con 0x7fec2c108170 2024-03-27T10:53:22.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:22.819+0000 7fec26a00640 1 --2- 172.21.15.97:0/3240933702 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fec140a3610 0x7fec140a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:53:22.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:22.820+0000 7fec26a00640 1 -- 172.21.15.97:0/3240933702 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7127+0+0 (secure 0 0 0) 0x7fec200f1c60 con 0x7fec2c108170 2024-03-27T10:53:22.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:22.820+0000 7fec31400640 1 --2- 172.21.15.97:0/3240933702 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fec140a3610 0x7fec140a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:53:22.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:22.820+0000 7fec31400640 1 --2- 172.21.15.97:0/3240933702 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fec140a3610 0x7fec140a5ad0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fec2c19f8d0 tx=0x7fec1c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:53:22.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:22.823+0000 7fec26a00640 1 -- 172.21.15.97:0/3240933702 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7fec2004d020 con 0x7fec2c108170 2024-03-27T10:53:23.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:23.063+0000 7fec34800640 1 -- 172.21.15.97:0/3240933702 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fec2c02d370 con 0x7fec2c108170 2024-03-27T10:53:23.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:23.064+0000 7fec26a00640 1 -- 172.21.15.97:0/3240933702 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2224 (secure 0 0 0) 0x7fec200bec30 con 0x7fec2c108170 2024-03-27T10:53:23.038 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T10:53:23.038 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3787981632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3787981632},{"type":"v1","addr":"172.21.15.177:6833","nonce":3787981632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":34415,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6833/4052517902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":4052517902},{"type":"v1","addr":"172.21.15.114:6833","nonce":4052517902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-03-27T10:53:23.039 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 3 2024-03-27T10:53:23.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:23.067+0000 7fec34800640 1 -- 172.21.15.97:0/3240933702 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fec140a3610 msgr2=0x7fec140a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:23.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:23.067+0000 7fec34800640 1 --2- 172.21.15.97:0/3240933702 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fec140a3610 0x7fec140a5ad0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fec2c19f8d0 tx=0x7fec1c005f90 comp rx=0 tx=0).stop 2024-03-27T10:53:23.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:23.067+0000 7fec34800640 1 -- 172.21.15.97:0/3240933702 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fec2c108170 msgr2=0x7fec2c19e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:23.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:23.067+0000 7fec34800640 1 --2- 172.21.15.97:0/3240933702 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fec2c108170 0x7fec2c19e3c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fec20000c00 tx=0x7fec20004040 comp rx=0 tx=0).stop 2024-03-27T10:53:23.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:23.067+0000 7fec34800640 1 -- 172.21.15.97:0/3240933702 shutdown_connections 2024-03-27T10:53:23.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:23.067+0000 7fec34800640 1 --2- 172.21.15.97:0/3240933702 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fec2c109d20 0x7fec2c1a5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:23.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:23.068+0000 7fec34800640 1 --2- 172.21.15.97:0/3240933702 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fec140a3610 0x7fec140a5ad0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:23.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:23.068+0000 7fec34800640 1 --2- 172.21.15.97:0/3240933702 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fec2c109360 0x7fec2c19e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:23.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:23.068+0000 7fec34800640 1 --2- 172.21.15.97:0/3240933702 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fec2c108170 0x7fec2c19e3c0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:23.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:23.068+0000 7fec34800640 1 -- 172.21.15.97:0/3240933702 >> 172.21.15.97:0/3240933702 conn(0x7fec2c077130 msgr2=0x7fec2c10c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:53:23.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:23.068+0000 7fec34800640 1 -- 172.21.15.97:0/3240933702 shutdown_connections 2024-03-27T10:53:23.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:23.068+0000 7fec34800640 1 -- 172.21.15.97:0/3240933702 wait complete. 2024-03-27T10:53:23.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:23 smithi097 ceph-mon[31859]: pgmap v337: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T10:53:23.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:23 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/3240933702' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T10:53:23.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:23 smithi177 ceph-mon[35294]: pgmap v337: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T10:53:23.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:23 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/3240933702' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T10:53:23.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:23 smithi114 ceph-mon[35338]: pgmap v337: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T10:53:23.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:23 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/3240933702' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T10:53:24.135 INFO:journalctl@ceph.mds.j.smithi097.stdout:Mar 27 10:53:23 smithi097 systemd[1]: Starting Ceph mds.j for adca88c4-ec26-11ee-95d0-87774f69a715... 2024-03-27T10:53:24.391 INFO:journalctl@ceph.mds.j.smithi097.stdout:Mar 27 10:53:24 smithi097 podman[72417]: 2024-03-27 10:53:24.066750191 +0000 UTC m=+0.032863091 image pull b379bbd8da6b5d0fa9a3523941efac213b6815e1cbf05c6507954f0a3fe7a801 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772 2024-03-27T10:53:24.392 INFO:journalctl@ceph.mds.j.smithi097.stdout:Mar 27 10:53:24 smithi097 podman[72417]: 2024-03-27 10:53:24.23946771 +0000 UTC m=+0.205580667 container create c8fdfbf183e03bb5e20a05450e8ba6458851e39713b58307de8143280bae40b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-j, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240326.131558-155268c) 2024-03-27T10:53:24.891 INFO:journalctl@ceph.mds.j.smithi097.stdout:Mar 27 10:53:24 smithi097 podman[72417]: 2024-03-27 10:53:24.493512995 +0000 UTC m=+0.459625938 container init c8fdfbf183e03bb5e20a05450e8ba6458851e39713b58307de8143280bae40b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-j, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-27T10:53:24.891 INFO:journalctl@ceph.mds.j.smithi097.stdout:Mar 27 10:53:24 smithi097 podman[72417]: 2024-03-27 10:53:24.502751329 +0000 UTC m=+0.468864256 container start c8fdfbf183e03bb5e20a05450e8ba6458851e39713b58307de8143280bae40b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-j, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.build-date=20240326, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-27T10:53:24.891 INFO:journalctl@ceph.mds.j.smithi097.stdout:Mar 27 10:53:24 smithi097 ceph-mds[72435]: -- 172.21.15.97:0/4113428984 <== mon.0 v2:172.21.15.97:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560875dcba40 con 0x560875d9e800 2024-03-27T10:53:24.891 INFO:journalctl@ceph.mds.j.smithi097.stdout:Mar 27 10:53:24 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-j[72431]: starting mds.j at 2024-03-27T10:53:24.891 INFO:journalctl@ceph.mds.j.smithi097.stdout:Mar 27 10:53:24 smithi097 bash[72417]: c8fdfbf183e03bb5e20a05450e8ba6458851e39713b58307de8143280bae40b2 2024-03-27T10:53:24.892 INFO:journalctl@ceph.mds.j.smithi097.stdout:Mar 27 10:53:24 smithi097 systemd[1]: Started Ceph mds.j for adca88c4-ec26-11ee-95d0-87774f69a715. 2024-03-27T10:53:25.060 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-27T10:53:25.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:25.305+0000 7f5d89a00640 1 -- 172.21.15.97:0/974847608 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d84105920 msgr2=0x7f5d8410c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:25.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:25.305+0000 7f5d89a00640 1 --2- 172.21.15.97:0/974847608 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d84105920 0x7f5d8410c1b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c009ca0 tx=0x7f5d7c02c0f0 comp rx=0 tx=0).stop 2024-03-27T10:53:25.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:25.305+0000 7f5d89a00640 1 -- 172.21.15.97:0/974847608 shutdown_connections 2024-03-27T10:53:25.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:25.305+0000 7f5d89a00640 1 --2- 172.21.15.97:0/974847608 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d84105920 0x7f5d8410c1b0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:25.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:25.305+0000 7f5d89a00640 1 --2- 172.21.15.97:0/974847608 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5d84104f60 0x7f5d841053e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:25.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:25.305+0000 7f5d89a00640 1 --2- 172.21.15.97:0/974847608 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5d84103d70 0x7f5d84104190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:25.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:25.305+0000 7f5d89a00640 1 -- 172.21.15.97:0/974847608 >> 172.21.15.97:0/974847608 conn(0x7f5d840ff300 msgr2=0x7f5d84101720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:53:25.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:25.305+0000 7f5d89a00640 1 -- 172.21.15.97:0/974847608 shutdown_connections 2024-03-27T10:53:25.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:25.305+0000 7f5d89a00640 1 -- 172.21.15.97:0/974847608 wait complete. 2024-03-27T10:53:25.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:25.306+0000 7f5d89a00640 1 Processor -- start 2024-03-27T10:53:25.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:25.306+0000 7f5d89a00640 1 -- start start 2024-03-27T10:53:25.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:25.306+0000 7f5d89a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5d84103d70 0x7f5d84199e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:53:25.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:25.306+0000 7f5d89a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d84104f60 0x7f5d8419a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:53:25.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:25.306+0000 7f5d89a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5d84105920 0x7f5d841a1430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:53:25.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:25.306+0000 7f5d89a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5d8410f8a0 con 0x7f5d84104f60 2024-03-27T10:53:25.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:25.306+0000 7f5d89a00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f5d8410f720 con 0x7f5d84103d70 2024-03-27T10:53:25.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:25.306+0000 7f5d89a00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f5d8410fa20 con 0x7f5d84105920 2024-03-27T10:53:25.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:25.306+0000 7f5d82000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d84104f60 0x7f5d8419a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:53:25.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:25.307+0000 7f5d82000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d84104f60 0x7f5d8419a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38392/0 (socket says 172.21.15.97:38392) 2024-03-27T10:53:25.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:25.307+0000 7f5d82000640 1 -- 172.21.15.97:0/184784943 learned_addr learned my addr 172.21.15.97:0/184784943 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:53:25.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:25.307+0000 7f5d83400640 1 --2- 172.21.15.97:0/184784943 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5d84105920 0x7f5d841a1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:53:25.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:25.307+0000 7f5d82a00640 1 --2- 172.21.15.97:0/184784943 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5d84103d70 0x7f5d84199e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:53:25.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:25.307+0000 7f5d82000640 1 -- 172.21.15.97:0/184784943 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5d84105920 msgr2=0x7f5d841a1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:25.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:25.307+0000 7f5d82000640 1 --2- 172.21.15.97:0/184784943 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5d84105920 0x7f5d841a1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:25.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:25.307+0000 7f5d82000640 1 -- 172.21.15.97:0/184784943 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5d84103d70 msgr2=0x7f5d84199e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:25.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:25.307+0000 7f5d82000640 1 --2- 172.21.15.97:0/184784943 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5d84103d70 0x7f5d84199e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:25.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:25.307+0000 7f5d82000640 1 -- 172.21.15.97:0/184784943 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d841a1aa0 con 0x7f5d84104f60 2024-03-27T10:53:25.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:25.307+0000 7f5d82a00640 1 --2- 172.21.15.97:0/184784943 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5d84103d70 0x7f5d84199e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:53:25.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:25.307+0000 7f5d83400640 1 --2- 172.21.15.97:0/184784943 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5d84105920 0x7f5d841a1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:53:25.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:25.307+0000 7f5d82000640 1 --2- 172.21.15.97:0/184784943 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d84104f60 0x7f5d8419a3c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5d7000d770 tx=0x7f5d7000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:53:25.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:25.307+0000 7f5d7b400640 1 -- 172.21.15.97:0/184784943 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d70017070 con 0x7f5d84104f60 2024-03-27T10:53:25.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:25.308+0000 7f5d7b400640 1 -- 172.21.15.97:0/184784943 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5d700054d0 con 0x7f5d84104f60 2024-03-27T10:53:25.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:25.308+0000 7f5d89a00640 1 -- 172.21.15.97:0/184784943 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d841a1d90 con 0x7f5d84104f60 2024-03-27T10:53:25.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:25.308+0000 7f5d7b400640 1 -- 172.21.15.97:0/184784943 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d70004cf0 con 0x7f5d84104f60 2024-03-27T10:53:25.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:25.308+0000 7f5d89a00640 1 -- 172.21.15.97:0/184784943 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d841a2280 con 0x7f5d84104f60 2024-03-27T10:53:25.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:25.309+0000 7f5d89a00640 1 -- 172.21.15.97:0/184784943 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d8410b030 con 0x7f5d84104f60 2024-03-27T10:53:25.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:25.312+0000 7f5d7b400640 1 -- 172.21.15.97:0/184784943 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138370+0+0 (secure 0 0 0) 0x7f5d70004e90 con 0x7f5d84104f60 2024-03-27T10:53:25.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:25.314+0000 7f5d7b400640 1 --2- 172.21.15.97:0/184784943 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f5d680a3690 0x7f5d680a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:53:25.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:25.314+0000 7f5d7b400640 1 -- 172.21.15.97:0/184784943 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7127+0+0 (secure 0 0 0) 0x7f5d700cf3b0 con 0x7f5d84104f60 2024-03-27T10:53:25.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:25.314+0000 7f5d82a00640 1 --2- 172.21.15.97:0/184784943 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f5d680a3690 0x7f5d680a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:53:25.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:25.315+0000 7f5d82a00640 1 --2- 172.21.15.97:0/184784943 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f5d680a3690 0x7f5d680a5b50 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5d84104dc0 tx=0x7f5d74005db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:53:25.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:25.317+0000 7f5d7b400640 1 -- 172.21.15.97:0/184784943 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f5d7009c380 con 0x7f5d84104f60 2024-03-27T10:53:25.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:25 smithi097 ceph-mon[31859]: pgmap v338: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T10:53:25.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:25 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:25.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:25 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:25.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:25 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:25.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:25 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-27T10:53:25.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:25 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-27T10:53:25.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:25 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:53:25.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:25 smithi177 ceph-mon[35294]: pgmap v338: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T10:53:25.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:25 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:25.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:25 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:25.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:25 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:25.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:25 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-27T10:53:25.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:25 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-27T10:53:25.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:25 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:53:25.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:25 smithi114 ceph-mon[35338]: pgmap v338: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T10:53:25.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:25 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:25.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:25 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:25.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:25 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:25.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:25 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-27T10:53:25.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:25 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-27T10:53:25.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:25 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:53:25.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:25.575+0000 7f5d89a00640 1 -- 172.21.15.97:0/184784943 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5d8402d370 con 0x7f5d84104f60 2024-03-27T10:53:25.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:25.576+0000 7f5d7b400640 1 -- 172.21.15.97:0/184784943 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3049 (secure 0 0 0) 0x7f5d700bc0d0 con 0x7f5d84104f60 2024-03-27T10:53:25.549 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T10:53:25.549 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1009423904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1009423904},{"type":"v1","addr":"172.21.15.97:6835","nonce":1009423904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3787981632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3787981632},{"type":"v1","addr":"172.21.15.177:6833","nonce":3787981632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":34415,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6833/4052517902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":4052517902},{"type":"v1","addr":"172.21.15.114:6833","nonce":4052517902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-03-27T10:53:25.549 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 4 2024-03-27T10:53:25.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:25.578+0000 7f5d89a00640 1 -- 172.21.15.97:0/184784943 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f5d680a3690 msgr2=0x7f5d680a5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:25.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:25.578+0000 7f5d89a00640 1 --2- 172.21.15.97:0/184784943 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f5d680a3690 0x7f5d680a5b50 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5d84104dc0 tx=0x7f5d74005db0 comp rx=0 tx=0).stop 2024-03-27T10:53:25.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:25.578+0000 7f5d89a00640 1 -- 172.21.15.97:0/184784943 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d84104f60 msgr2=0x7f5d8419a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:25.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:25.578+0000 7f5d89a00640 1 --2- 172.21.15.97:0/184784943 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d84104f60 0x7f5d8419a3c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5d7000d770 tx=0x7f5d7000dc30 comp rx=0 tx=0).stop 2024-03-27T10:53:25.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:25.578+0000 7f5d89a00640 1 -- 172.21.15.97:0/184784943 shutdown_connections 2024-03-27T10:53:25.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:25.578+0000 7f5d89a00640 1 --2- 172.21.15.97:0/184784943 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5d84105920 0x7f5d841a1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:25.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:25.578+0000 7f5d89a00640 1 --2- 172.21.15.97:0/184784943 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d84104f60 0x7f5d8419a3c0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:25.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:25.578+0000 7f5d89a00640 1 --2- 172.21.15.97:0/184784943 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f5d680a3690 0x7f5d680a5b50 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:25.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:25.578+0000 7f5d89a00640 1 --2- 172.21.15.97:0/184784943 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5d84103d70 0x7f5d84199e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:25.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:25.578+0000 7f5d89a00640 1 -- 172.21.15.97:0/184784943 >> 172.21.15.97:0/184784943 conn(0x7f5d840ff300 msgr2=0x7f5d84108200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:53:25.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:25.578+0000 7f5d89a00640 1 -- 172.21.15.97:0/184784943 shutdown_connections 2024-03-27T10:53:25.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:25.578+0000 7f5d89a00640 1 -- 172.21.15.97:0/184784943 wait complete. 2024-03-27T10:53:26.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:26 smithi177 ceph-mon[35294]: Deploying daemon mds.d on smithi097 2024-03-27T10:53:26.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:26 smithi177 ceph-mon[35294]: mds.? [v2:172.21.15.97:6834/1009423904,v1:172.21.15.97:6835/1009423904] up:boot 2024-03-27T10:53:26.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:26 smithi177 ceph-mon[35294]: fsmap 3 up:standby 2024-03-27T10:53:26.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:26 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-27T10:53:26.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:26 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/184784943' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T10:53:26.478 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:26 smithi097 ceph-mon[31859]: Deploying daemon mds.d on smithi097 2024-03-27T10:53:26.478 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:26 smithi097 ceph-mon[31859]: mds.? [v2:172.21.15.97:6834/1009423904,v1:172.21.15.97:6835/1009423904] up:boot 2024-03-27T10:53:26.478 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:26 smithi097 ceph-mon[31859]: fsmap 3 up:standby 2024-03-27T10:53:26.478 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:26 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-27T10:53:26.478 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:26 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/184784943' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T10:53:26.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:26 smithi114 ceph-mon[35338]: Deploying daemon mds.d on smithi097 2024-03-27T10:53:26.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:26 smithi114 ceph-mon[35338]: mds.? [v2:172.21.15.97:6834/1009423904,v1:172.21.15.97:6835/1009423904] up:boot 2024-03-27T10:53:26.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:26 smithi114 ceph-mon[35338]: fsmap 3 up:standby 2024-03-27T10:53:26.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:26 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-27T10:53:26.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:26 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/184784943' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T10:53:27.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:27 smithi097 ceph-mon[31859]: pgmap v339: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T10:53:27.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:27 smithi114 ceph-mon[35338]: pgmap v339: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T10:53:27.572 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-27T10:53:27.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:27 smithi177 ceph-mon[35294]: pgmap v339: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T10:53:27.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:27.866+0000 7f2cc1000640 1 -- 172.21.15.97:0/2102470773 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2cbc1078a0 msgr2=0x7f2cbc109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:27.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:27.866+0000 7f2cc1000640 1 --2- 172.21.15.97:0/2102470773 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2cbc1078a0 0x7f2cbc109c90 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2cb4009900 tx=0x7f2cb402bc30 comp rx=0 tx=0).stop 2024-03-27T10:53:27.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:27.866+0000 7f2cc1000640 1 -- 172.21.15.97:0/2102470773 shutdown_connections 2024-03-27T10:53:27.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:27.866+0000 7f2cc1000640 1 --2- 172.21.15.97:0/2102470773 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2cbc1078a0 0x7f2cbc109c90 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:27.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:27.866+0000 7f2cc1000640 1 --2- 172.21.15.97:0/2102470773 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2cbc104f70 0x7f2cbc107360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:27.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:27.866+0000 7f2cc1000640 1 --2- 172.21.15.97:0/2102470773 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2cbc102640 0x7f2cbc104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:27.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:27.866+0000 7f2cc1000640 1 -- 172.21.15.97:0/2102470773 >> 172.21.15.97:0/2102470773 conn(0x7f2cbc0fbfb0 msgr2=0x7f2cbc0fe3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:53:27.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:27.866+0000 7f2cc1000640 1 -- 172.21.15.97:0/2102470773 shutdown_connections 2024-03-27T10:53:27.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:27.866+0000 7f2cc1000640 1 -- 172.21.15.97:0/2102470773 wait complete. 2024-03-27T10:53:27.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:27.867+0000 7f2cc1000640 1 Processor -- start 2024-03-27T10:53:27.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:27.867+0000 7f2cc1000640 1 -- start start 2024-03-27T10:53:27.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:27.867+0000 7f2cc1000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2cbc102640 0x7f2cbc101fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:53:27.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:27.867+0000 7f2cc1000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2cbc104f70 0x7f2cbc1005f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:53:27.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:27.868+0000 7f2cc1000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2cbc1078a0 0x7f2cbc100b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:53:27.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:27.868+0000 7f2cc1000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2cbc10d6e0 con 0x7f2cbc104f70 2024-03-27T10:53:27.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:27.868+0000 7f2cc1000640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f2cbc10d560 con 0x7f2cbc1078a0 2024-03-27T10:53:27.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:27.868+0000 7f2cc1000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2cbc10d860 con 0x7f2cbc102640 2024-03-27T10:53:27.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:27.868+0000 7f2cb9400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2cbc104f70 0x7f2cbc1005f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:53:27.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:27.868+0000 7f2cb9400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2cbc104f70 0x7f2cbc1005f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38396/0 (socket says 172.21.15.97:38396) 2024-03-27T10:53:27.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:27.868+0000 7f2cb9400640 1 -- 172.21.15.97:0/557982944 learned_addr learned my addr 172.21.15.97:0/557982944 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:53:27.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:27.868+0000 7f2cb9e00640 1 --2- 172.21.15.97:0/557982944 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2cbc102640 0x7f2cbc101fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:53:27.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:27.868+0000 7f2cba800640 1 --2- 172.21.15.97:0/557982944 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2cbc1078a0 0x7f2cbc100b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:53:27.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:27.868+0000 7f2cb9400640 1 -- 172.21.15.97:0/557982944 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2cbc102640 msgr2=0x7f2cbc101fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:27.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:27.868+0000 7f2cb9400640 1 --2- 172.21.15.97:0/557982944 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2cbc102640 0x7f2cbc101fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:27.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:27.868+0000 7f2cb9400640 1 -- 172.21.15.97:0/557982944 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2cbc1078a0 msgr2=0x7f2cbc100b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:27.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:27.868+0000 7f2cb9400640 1 --2- 172.21.15.97:0/557982944 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2cbc1078a0 0x7f2cbc100b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:27.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:27.868+0000 7f2cb9400640 1 -- 172.21.15.97:0/557982944 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cbc1013c0 con 0x7f2cbc104f70 2024-03-27T10:53:27.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:27.869+0000 7f2cb9400640 1 --2- 172.21.15.97:0/557982944 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2cbc104f70 0x7f2cbc1005f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2ca800d770 tx=0x7f2ca800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:53:27.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:27.869+0000 7f2cb2a00640 1 -- 172.21.15.97:0/557982944 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca8017070 con 0x7f2cbc104f70 2024-03-27T10:53:27.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:27.869+0000 7f2cb2a00640 1 -- 172.21.15.97:0/557982944 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ca8009c30 con 0x7f2cbc104f70 2024-03-27T10:53:27.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:27.869+0000 7f2cb2a00640 1 -- 172.21.15.97:0/557982944 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca800f620 con 0x7f2cbc104f70 2024-03-27T10:53:27.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:27.869+0000 7f2cc1000640 1 -- 172.21.15.97:0/557982944 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cbc1a1e70 con 0x7f2cbc104f70 2024-03-27T10:53:27.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:27.869+0000 7f2cc1000640 1 -- 172.21.15.97:0/557982944 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cbc1a2280 con 0x7f2cbc104f70 2024-03-27T10:53:27.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:27.870+0000 7f2cc1000640 1 -- 172.21.15.97:0/557982944 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cbc0fdaa0 con 0x7f2cbc104f70 2024-03-27T10:53:27.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:27.873+0000 7f2cb2a00640 1 -- 172.21.15.97:0/557982944 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138370+0+0 (secure 0 0 0) 0x7f2ca801a070 con 0x7f2cbc104f70 2024-03-27T10:53:27.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:27.874+0000 7f2cb2a00640 1 --2- 172.21.15.97:0/557982944 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f2ca00a3610 0x7f2ca00a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:53:27.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:27.874+0000 7f2cb2a00640 1 -- 172.21.15.97:0/557982944 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7127+0+0 (secure 0 0 0) 0x7f2ca80cf380 con 0x7f2cbc104f70 2024-03-27T10:53:27.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:27.875+0000 7f2cb9e00640 1 --2- 172.21.15.97:0/557982944 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f2ca00a3610 0x7f2ca00a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:53:27.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:27.876+0000 7f2cb9e00640 1 --2- 172.21.15.97:0/557982944 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f2ca00a3610 0x7f2ca00a5ad0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2cac005fd0 tx=0x7f2cac009870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:53:27.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:27.880+0000 7f2cb2a00640 1 -- 172.21.15.97:0/557982944 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f2ca8009de0 con 0x7f2cbc104f70 2024-03-27T10:53:28.133 INFO:journalctl@ceph.mds.d.smithi097.stdout:Mar 27 10:53:27 smithi097 systemd[1]: Starting Ceph mds.d for adca88c4-ec26-11ee-95d0-87774f69a715... 2024-03-27T10:53:28.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:28.165+0000 7f2cc1000640 1 -- 172.21.15.97:0/557982944 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2cbc02d370 con 0x7f2cbc104f70 2024-03-27T10:53:28.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:28.165+0000 7f2cb2a00640 1 -- 172.21.15.97:0/557982944 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3049 (secure 0 0 0) 0x7f2ca809c3d0 con 0x7f2cbc104f70 2024-03-27T10:53:28.139 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T10:53:28.139 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1009423904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1009423904},{"type":"v1","addr":"172.21.15.97:6835","nonce":1009423904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3787981632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3787981632},{"type":"v1","addr":"172.21.15.177:6833","nonce":3787981632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":34415,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6833/4052517902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":4052517902},{"type":"v1","addr":"172.21.15.114:6833","nonce":4052517902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-03-27T10:53:28.139 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 4 2024-03-27T10:53:28.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:28.168+0000 7f2cc1000640 1 -- 172.21.15.97:0/557982944 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f2ca00a3610 msgr2=0x7f2ca00a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:28.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:28.168+0000 7f2cc1000640 1 --2- 172.21.15.97:0/557982944 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f2ca00a3610 0x7f2ca00a5ad0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2cac005fd0 tx=0x7f2cac009870 comp rx=0 tx=0).stop 2024-03-27T10:53:28.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:28.168+0000 7f2cc1000640 1 -- 172.21.15.97:0/557982944 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2cbc104f70 msgr2=0x7f2cbc1005f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:28.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:28.168+0000 7f2cc1000640 1 --2- 172.21.15.97:0/557982944 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2cbc104f70 0x7f2cbc1005f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2ca800d770 tx=0x7f2ca800dc30 comp rx=0 tx=0).stop 2024-03-27T10:53:28.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:28.168+0000 7f2cc1000640 1 -- 172.21.15.97:0/557982944 shutdown_connections 2024-03-27T10:53:28.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:28.168+0000 7f2cc1000640 1 --2- 172.21.15.97:0/557982944 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f2ca00a3610 0x7f2ca00a5ad0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:28.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:28.168+0000 7f2cc1000640 1 --2- 172.21.15.97:0/557982944 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2cbc1078a0 0x7f2cbc100b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:28.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:28.168+0000 7f2cc1000640 1 --2- 172.21.15.97:0/557982944 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2cbc104f70 0x7f2cbc1005f0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:28.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:28.168+0000 7f2cc1000640 1 --2- 172.21.15.97:0/557982944 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2cbc102640 0x7f2cbc101fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:28.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:28.168+0000 7f2cc1000640 1 -- 172.21.15.97:0/557982944 >> 172.21.15.97:0/557982944 conn(0x7f2cbc0fbfb0 msgr2=0x7f2cbc103120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:53:28.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:28.168+0000 7f2cc1000640 1 -- 172.21.15.97:0/557982944 shutdown_connections 2024-03-27T10:53:28.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:28.169+0000 7f2cc1000640 1 -- 172.21.15.97:0/557982944 wait complete. 2024-03-27T10:53:28.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:28 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/557982944' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T10:53:28.392 INFO:journalctl@ceph.mds.d.smithi097.stdout:Mar 27 10:53:28 smithi097 podman[72791]: 2024-03-27 10:53:28.067990179 +0000 UTC m=+0.037137202 image pull b379bbd8da6b5d0fa9a3523941efac213b6815e1cbf05c6507954f0a3fe7a801 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772 2024-03-27T10:53:28.392 INFO:journalctl@ceph.mds.d.smithi097.stdout:Mar 27 10:53:28 smithi097 podman[72791]: 2024-03-27 10:53:28.185018359 +0000 UTC m=+0.154165352 container create 693e794650a1cd93a6351af8dbd720ef93b5ff1ff097a10e2aeaf5d74c1138ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-d, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-27T10:53:28.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:28 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/557982944' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T10:53:28.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:28 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/557982944' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T10:53:28.893 INFO:journalctl@ceph.mds.d.smithi097.stdout:Mar 27 10:53:28 smithi097 podman[72791]: 2024-03-27 10:53:28.440824245 +0000 UTC m=+0.409971249 container init 693e794650a1cd93a6351af8dbd720ef93b5ff1ff097a10e2aeaf5d74c1138ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-d, RELEASE=wip-rishabh-testing-20240326.131558-155268c, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-03-27T10:53:28.893 INFO:journalctl@ceph.mds.d.smithi097.stdout:Mar 27 10:53:28 smithi097 podman[72791]: 2024-03-27 10:53:28.451100514 +0000 UTC m=+0.420247515 container start 693e794650a1cd93a6351af8dbd720ef93b5ff1ff097a10e2aeaf5d74c1138ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-d, RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-03-27T10:53:28.893 INFO:journalctl@ceph.mds.d.smithi097.stdout:Mar 27 10:53:28 smithi097 ceph-mds[72812]: -- 172.21.15.97:0/3863844521 <== mon.0 v2:172.21.15.97:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556d2f7fda40 con 0x556d2f7d1000 2024-03-27T10:53:28.893 INFO:journalctl@ceph.mds.d.smithi097.stdout:Mar 27 10:53:28 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-d[72808]: starting mds.d at 2024-03-27T10:53:28.893 INFO:journalctl@ceph.mds.d.smithi097.stdout:Mar 27 10:53:28 smithi097 bash[72791]: 693e794650a1cd93a6351af8dbd720ef93b5ff1ff097a10e2aeaf5d74c1138ad 2024-03-27T10:53:28.893 INFO:journalctl@ceph.mds.d.smithi097.stdout:Mar 27 10:53:28 smithi097 systemd[1]: Started Ceph mds.d for adca88c4-ec26-11ee-95d0-87774f69a715. 2024-03-27T10:53:29.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:29 smithi114 ceph-mon[35338]: pgmap v340: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T10:53:29.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:29 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:29.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:29 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:29.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:29 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:29.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:29 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-27T10:53:29.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:29 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-27T10:53:29.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:29 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:53:29.641 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:29 smithi097 ceph-mon[31859]: pgmap v340: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T10:53:29.641 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:29 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:29.642 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:29 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:29.642 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:29 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:29.642 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:29 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-27T10:53:29.642 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:29 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-27T10:53:29.642 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:29 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:53:29.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:29 smithi177 ceph-mon[35294]: pgmap v340: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T10:53:29.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:29 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:29.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:29 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:29.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:29 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:29.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:29 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-27T10:53:29.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:29 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-27T10:53:29.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:29 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:53:30.166 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-27T10:53:30.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:30.417+0000 7fccb0000640 1 -- 172.21.15.97:0/4230340892 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcca80780f0 msgr2=0x7fcca8078990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:30.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:30.417+0000 7fccb0000640 1 --2- 172.21.15.97:0/4230340892 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcca80780f0 0x7fcca8078990 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fcca0009ca0 tx=0x7fcca002c100 comp rx=0 tx=0).stop 2024-03-27T10:53:30.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:30.417+0000 7fccb0000640 1 -- 172.21.15.97:0/4230340892 shutdown_connections 2024-03-27T10:53:30.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:30.417+0000 7fccb0000640 1 --2- 172.21.15.97:0/4230340892 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcca80780f0 0x7fcca8078990 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:30.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:30.417+0000 7fccb0000640 1 --2- 172.21.15.97:0/4230340892 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcca8077750 0x7fcca8077bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:30.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:30.417+0000 7fccb0000640 1 --2- 172.21.15.97:0/4230340892 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcca8076510 0x7fcca8076930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:30.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:30.417+0000 7fccb0000640 1 -- 172.21.15.97:0/4230340892 >> 172.21.15.97:0/4230340892 conn(0x7fcca80ff300 msgr2=0x7fcca8101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:53:30.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:30.418+0000 7fccb0000640 1 -- 172.21.15.97:0/4230340892 shutdown_connections 2024-03-27T10:53:30.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:30.418+0000 7fccb0000640 1 -- 172.21.15.97:0/4230340892 wait complete. 2024-03-27T10:53:30.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:30.419+0000 7fccb0000640 1 Processor -- start 2024-03-27T10:53:30.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:30.419+0000 7fccb0000640 1 -- start start 2024-03-27T10:53:30.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:30.419+0000 7fccb0000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcca8076510 0x7fcca819e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:53:30.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:30.419+0000 7fccb0000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcca8077750 0x7fcca819e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:53:30.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:30.419+0000 7fccb0000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcca80780f0 0x7fcca81a58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:53:30.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:30.419+0000 7fccb0000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fcca8113cd0 con 0x7fcca80780f0 2024-03-27T10:53:30.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:30.419+0000 7fccb0000640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fcca8113b50 con 0x7fcca8077750 2024-03-27T10:53:30.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:30.419+0000 7fccb0000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fcca8113e50 con 0x7fcca8076510 2024-03-27T10:53:30.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:30.419+0000 7fccae000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcca80780f0 0x7fcca81a58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:53:30.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:30.420+0000 7fccae000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcca80780f0 0x7fcca81a58d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41250/0 (socket says 172.21.15.97:41250) 2024-03-27T10:53:30.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:30.420+0000 7fccae000640 1 -- 172.21.15.97:0/3745791305 learned_addr learned my addr 172.21.15.97:0/3745791305 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:53:30.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:30.420+0000 7fccad600640 1 --2- 172.21.15.97:0/3745791305 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcca8076510 0x7fcca819e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:53:30.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:30.420+0000 7fccacc00640 1 --2- 172.21.15.97:0/3745791305 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcca8077750 0x7fcca819e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:53:30.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:30.420+0000 7fccae000640 1 -- 172.21.15.97:0/3745791305 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcca8076510 msgr2=0x7fcca819e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:30.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:30.420+0000 7fccae000640 1 --2- 172.21.15.97:0/3745791305 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcca8076510 0x7fcca819e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:30.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:30.420+0000 7fccae000640 1 -- 172.21.15.97:0/3745791305 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcca8077750 msgr2=0x7fcca819e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:30.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:30.420+0000 7fccae000640 1 --2- 172.21.15.97:0/3745791305 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcca8077750 0x7fcca819e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:30.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:30.420+0000 7fccae000640 1 -- 172.21.15.97:0/3745791305 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcca81a5f40 con 0x7fcca80780f0 2024-03-27T10:53:30.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:30.420+0000 7fccacc00640 1 --2- 172.21.15.97:0/3745791305 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcca8077750 0x7fcca819e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:53:30.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:30.420+0000 7fccae000640 1 --2- 172.21.15.97:0/3745791305 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcca80780f0 0x7fcca81a58d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fcca0032520 tx=0x7fcca0034880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:53:30.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:30.420+0000 7fcca6000640 1 -- 172.21.15.97:0/3745791305 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca0039040 con 0x7fcca80780f0 2024-03-27T10:53:30.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:30.420+0000 7fcca6000640 1 -- 172.21.15.97:0/3745791305 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcca0044070 con 0x7fcca80780f0 2024-03-27T10:53:30.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:30.421+0000 7fccb0000640 1 -- 172.21.15.97:0/3745791305 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcca81a61d0 con 0x7fcca80780f0 2024-03-27T10:53:30.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:30.421+0000 7fcca6000640 1 -- 172.21.15.97:0/3745791305 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca00043c0 con 0x7fcca80780f0 2024-03-27T10:53:30.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:30.421+0000 7fccb0000640 1 -- 172.21.15.97:0/3745791305 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcca81a6710 con 0x7fcca80780f0 2024-03-27T10:53:30.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:30.422+0000 7fccb0000640 1 -- 172.21.15.97:0/3745791305 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcca81108a0 con 0x7fcca80780f0 2024-03-27T10:53:30.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:30.424+0000 7fcca6000640 1 -- 172.21.15.97:0/3745791305 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138370+0+0 (secure 0 0 0) 0x7fcca0004560 con 0x7fcca80780f0 2024-03-27T10:53:30.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:30.425+0000 7fcca6000640 1 --2- 172.21.15.97:0/3745791305 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fcc900a3690 0x7fcc900a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:53:30.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:30.425+0000 7fcca6000640 1 -- 172.21.15.97:0/3745791305 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7127+0+0 (secure 0 0 0) 0x7fcca00f1ff0 con 0x7fcca80780f0 2024-03-27T10:53:30.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:30.425+0000 7fccad600640 1 --2- 172.21.15.97:0/3745791305 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fcc900a3690 0x7fcc900a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:53:30.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:30.426+0000 7fccad600640 1 --2- 172.21.15.97:0/3745791305 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fcc900a3690 0x7fcc900a5b50 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c005fd0 tx=0x7fcc9c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:53:30.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:30.429+0000 7fcca6000640 1 -- 172.21.15.97:0/3745791305 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7fcca00333a0 con 0x7fcca80780f0 2024-03-27T10:53:30.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:30 smithi114 ceph-mon[35338]: Deploying daemon mds.e on smithi114 2024-03-27T10:53:30.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:30 smithi114 ceph-mon[35338]: mds.? [v2:172.21.15.97:6836/3251173433,v1:172.21.15.97:6837/3251173433] up:boot 2024-03-27T10:53:30.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:30 smithi114 ceph-mon[35338]: fsmap 4 up:standby 2024-03-27T10:53:30.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:30 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-27T10:53:30.641 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:30 smithi097 ceph-mon[31859]: Deploying daemon mds.e on smithi114 2024-03-27T10:53:30.641 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:30 smithi097 ceph-mon[31859]: mds.? [v2:172.21.15.97:6836/3251173433,v1:172.21.15.97:6837/3251173433] up:boot 2024-03-27T10:53:30.641 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:30 smithi097 ceph-mon[31859]: fsmap 4 up:standby 2024-03-27T10:53:30.641 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:30 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-27T10:53:30.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:30.672+0000 7fccb0000640 1 -- 172.21.15.97:0/3745791305 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcca8062750 con 0x7fcca80780f0 2024-03-27T10:53:30.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:30.673+0000 7fcca6000640 1 -- 172.21.15.97:0/3745791305 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3874 (secure 0 0 0) 0x7fcca00bf000 con 0x7fcca80780f0 2024-03-27T10:53:30.645 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T10:53:30.645 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1009423904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1009423904},{"type":"v1","addr":"172.21.15.97:6835","nonce":1009423904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3251173433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3251173433},{"type":"v1","addr":"172.21.15.97:6837","nonce":3251173433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3787981632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3787981632},{"type":"v1","addr":"172.21.15.177:6833","nonce":3787981632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":34415,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6833/4052517902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":4052517902},{"type":"v1","addr":"172.21.15.114:6833","nonce":4052517902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-03-27T10:53:30.646 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 5 2024-03-27T10:53:30.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:30.676+0000 7fccb0000640 1 -- 172.21.15.97:0/3745791305 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fcc900a3690 msgr2=0x7fcc900a5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:30.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:30.676+0000 7fccb0000640 1 --2- 172.21.15.97:0/3745791305 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fcc900a3690 0x7fcc900a5b50 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c005fd0 tx=0x7fcc9c005850 comp rx=0 tx=0).stop 2024-03-27T10:53:30.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:30.676+0000 7fccb0000640 1 -- 172.21.15.97:0/3745791305 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcca80780f0 msgr2=0x7fcca81a58d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:30.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:30.676+0000 7fccb0000640 1 --2- 172.21.15.97:0/3745791305 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcca80780f0 0x7fcca81a58d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fcca0032520 tx=0x7fcca0034880 comp rx=0 tx=0).stop 2024-03-27T10:53:30.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:30.676+0000 7fccb0000640 1 -- 172.21.15.97:0/3745791305 shutdown_connections 2024-03-27T10:53:30.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:30.676+0000 7fccb0000640 1 --2- 172.21.15.97:0/3745791305 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcca80780f0 0x7fcca81a58d0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:30.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:30.676+0000 7fccb0000640 1 --2- 172.21.15.97:0/3745791305 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fcc900a3690 0x7fcc900a5b50 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:30.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:30.676+0000 7fccb0000640 1 --2- 172.21.15.97:0/3745791305 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcca8077750 0x7fcca819e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:30.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:30.676+0000 7fccb0000640 1 --2- 172.21.15.97:0/3745791305 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcca8076510 0x7fcca819e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:30.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:30.676+0000 7fccb0000640 1 -- 172.21.15.97:0/3745791305 >> 172.21.15.97:0/3745791305 conn(0x7fcca80ff300 msgr2=0x7fcca8100a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:53:30.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:30.676+0000 7fccb0000640 1 -- 172.21.15.97:0/3745791305 shutdown_connections 2024-03-27T10:53:30.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:30.676+0000 7fccb0000640 1 -- 172.21.15.97:0/3745791305 wait complete. 2024-03-27T10:53:30.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:30 smithi177 ceph-mon[35294]: Deploying daemon mds.e on smithi114 2024-03-27T10:53:30.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:30 smithi177 ceph-mon[35294]: mds.? [v2:172.21.15.97:6836/3251173433,v1:172.21.15.97:6837/3251173433] up:boot 2024-03-27T10:53:30.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:30 smithi177 ceph-mon[35294]: fsmap 4 up:standby 2024-03-27T10:53:30.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:30 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-27T10:53:31.507 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:31 smithi097 ceph-mon[31859]: pgmap v341: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T10:53:31.507 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:31 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/3745791305' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T10:53:31.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:31 smithi114 ceph-mon[35338]: pgmap v341: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T10:53:31.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:31 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/3745791305' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T10:53:31.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:31 smithi177 ceph-mon[35294]: pgmap v341: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T10:53:31.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:31 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/3745791305' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T10:53:32.267 INFO:journalctl@ceph.mds.e.smithi114.stdout:Mar 27 10:53:31 smithi114 systemd[1]: Starting Ceph mds.e for adca88c4-ec26-11ee-95d0-87774f69a715... 2024-03-27T10:53:32.604 INFO:journalctl@ceph.mds.e.smithi114.stdout:Mar 27 10:53:32 smithi114 podman[65227]: 2024-03-27 10:53:32.338349431 +0000 UTC m=+0.123922625 container create 427c9c4785157f00a0cfc142b9766f8948dde380c8db8dd00c52e104d44d7163 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-e, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, RELEASE=wip-rishabh-testing-20240326.131558-155268c, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326) 2024-03-27T10:53:32.604 INFO:journalctl@ceph.mds.e.smithi114.stdout:Mar 27 10:53:32 smithi114 podman[65227]: 2024-03-27 10:53:32.247728767 +0000 UTC m=+0.033301956 image pull b379bbd8da6b5d0fa9a3523941efac213b6815e1cbf05c6507954f0a3fe7a801 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772 2024-03-27T10:53:32.604 INFO:journalctl@ceph.mds.e.smithi114.stdout:Mar 27 10:53:32 smithi114 podman[65227]: 2024-03-27 10:53:32.575033235 +0000 UTC m=+0.360606436 container init 427c9c4785157f00a0cfc142b9766f8948dde380c8db8dd00c52e104d44d7163 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-e, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, RELEASE=wip-rishabh-testing-20240326.131558-155268c, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-03-27T10:53:32.604 INFO:journalctl@ceph.mds.e.smithi114.stdout:Mar 27 10:53:32 smithi114 podman[65227]: 2024-03-27 10:53:32.584286536 +0000 UTC m=+0.369859721 container start 427c9c4785157f00a0cfc142b9766f8948dde380c8db8dd00c52e104d44d7163 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-e, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240326, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-03-27T10:53:32.667 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-27T10:53:32.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:32.885+0000 7f8b3a600640 1 -- 172.21.15.97:0/3893827800 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b34101970 msgr2=0x7f8b34101d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:32.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:32.885+0000 7f8b3a600640 1 --2- 172.21.15.97:0/3893827800 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b34101970 0x7f8b34101d90 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8b28009900 tx=0x7f8b2802bc30 comp rx=0 tx=0).stop 2024-03-27T10:53:32.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:32.885+0000 7f8b3a600640 1 -- 172.21.15.97:0/3893827800 shutdown_connections 2024-03-27T10:53:32.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:32.885+0000 7f8b3a600640 1 --2- 172.21.15.97:0/3893827800 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8b34103520 0x7f8b34109db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:32.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:32.885+0000 7f8b3a600640 1 --2- 172.21.15.97:0/3893827800 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8b34102b60 0x7f8b34102fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:32.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:32.885+0000 7f8b3a600640 1 --2- 172.21.15.97:0/3893827800 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b34101970 0x7f8b34101d90 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:32.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:32.885+0000 7f8b3a600640 1 -- 172.21.15.97:0/3893827800 >> 172.21.15.97:0/3893827800 conn(0x7f8b340fcee0 msgr2=0x7f8b340ff320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:53:32.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:32.885+0000 7f8b3a600640 1 -- 172.21.15.97:0/3893827800 shutdown_connections 2024-03-27T10:53:32.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:32.885+0000 7f8b3a600640 1 -- 172.21.15.97:0/3893827800 wait complete. 2024-03-27T10:53:32.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:32.886+0000 7f8b3a600640 1 Processor -- start 2024-03-27T10:53:32.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:32.886+0000 7f8b3a600640 1 -- start start 2024-03-27T10:53:32.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:32.886+0000 7f8b3a600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8b34101970 0x7f8b34199db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:53:32.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:32.886+0000 7f8b3a600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8b34102b60 0x7f8b3419a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:53:32.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:32.886+0000 7f8b3a600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b34103520 0x7f8b341a1360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:53:32.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:32.886+0000 7f8b3a600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8b3410d4e0 con 0x7f8b34103520 2024-03-27T10:53:32.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:32.886+0000 7f8b3a600640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f8b3410d360 con 0x7f8b34101970 2024-03-27T10:53:32.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:32.886+0000 7f8b3a600640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f8b3410d660 con 0x7f8b34102b60 2024-03-27T10:53:32.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:32.887+0000 7f8b39c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b34103520 0x7f8b341a1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:53:32.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:32.887+0000 7f8b39c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b34103520 0x7f8b341a1360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41280/0 (socket says 172.21.15.97:41280) 2024-03-27T10:53:32.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:32.887+0000 7f8b39c00640 1 -- 172.21.15.97:0/2371390946 learned_addr learned my addr 172.21.15.97:0/2371390946 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:53:32.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:32.887+0000 7f8b39200640 1 --2- 172.21.15.97:0/2371390946 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8b34101970 0x7f8b34199db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:53:32.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:32.887+0000 7f8b33e00640 1 --2- 172.21.15.97:0/2371390946 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8b34102b60 0x7f8b3419a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:53:32.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:32.887+0000 7f8b39c00640 1 -- 172.21.15.97:0/2371390946 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8b34102b60 msgr2=0x7f8b3419a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:32.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:32.887+0000 7f8b39c00640 1 --2- 172.21.15.97:0/2371390946 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8b34102b60 0x7f8b3419a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:32.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:32.887+0000 7f8b39c00640 1 -- 172.21.15.97:0/2371390946 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8b34101970 msgr2=0x7f8b34199db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:32.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:32.887+0000 7f8b39c00640 1 --2- 172.21.15.97:0/2371390946 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8b34101970 0x7f8b34199db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:32.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:32.887+0000 7f8b39c00640 1 -- 172.21.15.97:0/2371390946 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b341a19d0 con 0x7f8b34103520 2024-03-27T10:53:32.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:32.887+0000 7f8b39200640 1 --2- 172.21.15.97:0/2371390946 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8b34101970 0x7f8b34199db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:53:32.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:32.887+0000 7f8b39c00640 1 --2- 172.21.15.97:0/2371390946 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b34103520 0x7f8b341a1360 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c00eaa0 tx=0x7f8b2c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:53:32.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:32.888+0000 7f8b31600640 1 -- 172.21.15.97:0/2371390946 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b2c00cdb0 con 0x7f8b34103520 2024-03-27T10:53:32.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:32.888+0000 7f8b31600640 1 -- 172.21.15.97:0/2371390946 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8b2c005020 con 0x7f8b34103520 2024-03-27T10:53:32.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:32.888+0000 7f8b3a600640 1 -- 172.21.15.97:0/2371390946 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b341a1cc0 con 0x7f8b34103520 2024-03-27T10:53:32.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:32.888+0000 7f8b31600640 1 -- 172.21.15.97:0/2371390946 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b2c0116d0 con 0x7f8b34103520 2024-03-27T10:53:32.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:32.888+0000 7f8b3a600640 1 -- 172.21.15.97:0/2371390946 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b341a21c0 con 0x7f8b34103520 2024-03-27T10:53:32.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:32.889+0000 7f8b3a600640 1 -- 172.21.15.97:0/2371390946 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b341a2450 con 0x7f8b34103520 2024-03-27T10:53:32.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:32.891+0000 7f8b31600640 1 -- 172.21.15.97:0/2371390946 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138370+0+0 (secure 0 0 0) 0x7f8b2c0051d0 con 0x7f8b34103520 2024-03-27T10:53:32.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:32.892+0000 7f8b31600640 1 --2- 172.21.15.97:0/2371390946 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f8b1c0a3690 0x7f8b1c0a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:53:32.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:32.892+0000 7f8b31600640 1 -- 172.21.15.97:0/2371390946 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7127+0+0 (secure 0 0 0) 0x7f8b2c0cfaa0 con 0x7f8b34103520 2024-03-27T10:53:32.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:32.892+0000 7f8b39200640 1 --2- 172.21.15.97:0/2371390946 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f8b1c0a3690 0x7f8b1c0a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:53:32.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:32.893+0000 7f8b39200640 1 --2- 172.21.15.97:0/2371390946 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f8b1c0a3690 0x7f8b1c0a5b50 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8b28036040 tx=0x7f8b28005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:53:32.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:32.899+0000 7f8b31600640 1 -- 172.21.15.97:0/2371390946 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f8b2c016030 con 0x7f8b34103520 2024-03-27T10:53:33.018 INFO:journalctl@ceph.mds.e.smithi114.stdout:Mar 27 10:53:32 smithi114 ceph-mds[65246]: -- 172.21.15.114:0/615835153 <== mon.2 v2:172.21.15.114:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e5767ada40 con 0x55e576781000 2024-03-27T10:53:33.018 INFO:journalctl@ceph.mds.e.smithi114.stdout:Mar 27 10:53:32 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-e[65242]: starting mds.e at 2024-03-27T10:53:33.018 INFO:journalctl@ceph.mds.e.smithi114.stdout:Mar 27 10:53:32 smithi114 bash[65227]: 427c9c4785157f00a0cfc142b9766f8948dde380c8db8dd00c52e104d44d7163 2024-03-27T10:53:33.018 INFO:journalctl@ceph.mds.e.smithi114.stdout:Mar 27 10:53:32 smithi114 systemd[1]: Started Ceph mds.e for adca88c4-ec26-11ee-95d0-87774f69a715. 2024-03-27T10:53:33.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:33.141+0000 7f8b3a600640 1 -- 172.21.15.97:0/2371390946 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8b341a2620 con 0x7f8b34103520 2024-03-27T10:53:33.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:33.142+0000 7f8b31600640 1 -- 172.21.15.97:0/2371390946 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3874 (secure 0 0 0) 0x7f8b2c09cb10 con 0x7f8b34103520 2024-03-27T10:53:33.115 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T10:53:33.115 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1009423904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1009423904},{"type":"v1","addr":"172.21.15.97:6835","nonce":1009423904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3251173433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3251173433},{"type":"v1","addr":"172.21.15.97:6837","nonce":3251173433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3787981632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3787981632},{"type":"v1","addr":"172.21.15.177:6833","nonce":3787981632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":34415,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6833/4052517902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":4052517902},{"type":"v1","addr":"172.21.15.114:6833","nonce":4052517902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-03-27T10:53:33.115 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 5 2024-03-27T10:53:33.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:33.145+0000 7f8b3a600640 1 -- 172.21.15.97:0/2371390946 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f8b1c0a3690 msgr2=0x7f8b1c0a5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:33.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:33.145+0000 7f8b3a600640 1 --2- 172.21.15.97:0/2371390946 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f8b1c0a3690 0x7f8b1c0a5b50 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8b28036040 tx=0x7f8b28005fd0 comp rx=0 tx=0).stop 2024-03-27T10:53:33.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:33.145+0000 7f8b3a600640 1 -- 172.21.15.97:0/2371390946 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b34103520 msgr2=0x7f8b341a1360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:33.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:33.145+0000 7f8b3a600640 1 --2- 172.21.15.97:0/2371390946 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b34103520 0x7f8b341a1360 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c00eaa0 tx=0x7f8b2c00ef60 comp rx=0 tx=0).stop 2024-03-27T10:53:33.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:33.146+0000 7f8b3a600640 1 -- 172.21.15.97:0/2371390946 shutdown_connections 2024-03-27T10:53:33.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:33.146+0000 7f8b3a600640 1 --2- 172.21.15.97:0/2371390946 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b34103520 0x7f8b341a1360 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:33.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:33.146+0000 7f8b3a600640 1 --2- 172.21.15.97:0/2371390946 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8b34102b60 0x7f8b3419a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:33.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:33.146+0000 7f8b3a600640 1 --2- 172.21.15.97:0/2371390946 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f8b1c0a3690 0x7f8b1c0a5b50 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:33.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:33.146+0000 7f8b3a600640 1 --2- 172.21.15.97:0/2371390946 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8b34101970 0x7f8b34199db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:33.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:33.146+0000 7f8b3a600640 1 -- 172.21.15.97:0/2371390946 >> 172.21.15.97:0/2371390946 conn(0x7f8b340fcee0 msgr2=0x7f8b34073d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:53:33.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:33.146+0000 7f8b3a600640 1 -- 172.21.15.97:0/2371390946 shutdown_connections 2024-03-27T10:53:33.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:33.146+0000 7f8b3a600640 1 -- 172.21.15.97:0/2371390946 wait complete. 2024-03-27T10:53:33.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:33 smithi097 ceph-mon[31859]: pgmap v342: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T10:53:33.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:33 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:33.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:33 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:33.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:33 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:33.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:33 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-27T10:53:33.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:33 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-27T10:53:33.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:33 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:53:33.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:33 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/2371390946' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T10:53:33.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:33 smithi114 ceph-mon[35338]: pgmap v342: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T10:53:33.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:33 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:33.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:33 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:33.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:33 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:33.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:33 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-27T10:53:33.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:33 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-27T10:53:33.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:33 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:53:33.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:33 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/2371390946' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T10:53:33.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:33 smithi177 ceph-mon[35294]: pgmap v342: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T10:53:33.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:33 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:33.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:33 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:33.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:33 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:33.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:33 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-27T10:53:33.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:33 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-27T10:53:33.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:33 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:53:33.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:33 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/2371390946' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T10:53:34.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:34 smithi177 ceph-mon[35294]: Deploying daemon mds.f on smithi177 2024-03-27T10:53:34.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:34 smithi177 ceph-mon[35294]: mds.? [v2:172.21.15.114:6834/3581918222,v1:172.21.15.114:6835/3581918222] up:boot 2024-03-27T10:53:34.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:34 smithi177 ceph-mon[35294]: fsmap 5 up:standby 2024-03-27T10:53:34.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:34 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-27T10:53:34.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:34 smithi114 ceph-mon[35338]: Deploying daemon mds.f on smithi177 2024-03-27T10:53:34.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:34 smithi114 ceph-mon[35338]: mds.? [v2:172.21.15.114:6834/3581918222,v1:172.21.15.114:6835/3581918222] up:boot 2024-03-27T10:53:34.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:34 smithi114 ceph-mon[35338]: fsmap 5 up:standby 2024-03-27T10:53:34.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:34 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-27T10:53:34.641 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:34 smithi097 ceph-mon[31859]: Deploying daemon mds.f on smithi177 2024-03-27T10:53:34.641 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:34 smithi097 ceph-mon[31859]: mds.? [v2:172.21.15.114:6834/3581918222,v1:172.21.15.114:6835/3581918222] up:boot 2024-03-27T10:53:34.641 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:34 smithi097 ceph-mon[31859]: fsmap 5 up:standby 2024-03-27T10:53:34.641 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:34 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-27T10:53:35.137 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-27T10:53:35.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:35.392+0000 7f189e400640 1 -- 172.21.15.97:0/816876426 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1898109b40 msgr2=0x7f18981103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:35.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:35.392+0000 7f189e400640 1 --2- 172.21.15.97:0/816876426 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1898109b40 0x7f18981103d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1890009ca0 tx=0x7f189002c100 comp rx=0 tx=0).stop 2024-03-27T10:53:35.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:35.392+0000 7f189e400640 1 -- 172.21.15.97:0/816876426 shutdown_connections 2024-03-27T10:53:35.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:35.392+0000 7f189e400640 1 --2- 172.21.15.97:0/816876426 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1898109b40 0x7f18981103d0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:35.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:35.392+0000 7f189e400640 1 --2- 172.21.15.97:0/816876426 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1898109180 0x7f1898109600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:35.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:35.392+0000 7f189e400640 1 --2- 172.21.15.97:0/816876426 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1898107f90 0x7f18981083b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:35.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:35.392+0000 7f189e400640 1 -- 172.21.15.97:0/816876426 >> 172.21.15.97:0/816876426 conn(0x7f1898076f20 msgr2=0x7f1898079360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:53:35.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:35.392+0000 7f189e400640 1 -- 172.21.15.97:0/816876426 shutdown_connections 2024-03-27T10:53:35.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:35.392+0000 7f189e400640 1 -- 172.21.15.97:0/816876426 wait complete. 2024-03-27T10:53:35.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:35.393+0000 7f189e400640 1 Processor -- start 2024-03-27T10:53:35.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:35.393+0000 7f189e400640 1 -- start start 2024-03-27T10:53:35.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:35.393+0000 7f189e400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1898107f90 0x7f189819e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:53:35.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:35.393+0000 7f189e400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1898109180 0x7f189819e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:53:35.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:35.393+0000 7f189e400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1898109b40 0x7f18981a5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:53:35.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:35.393+0000 7f189e400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1898113ad0 con 0x7f1898109b40 2024-03-27T10:53:35.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:35.393+0000 7f189e400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f1898113950 con 0x7f1898109180 2024-03-27T10:53:35.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:35.393+0000 7f189e400640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1898113c50 con 0x7f1898107f90 2024-03-27T10:53:35.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:35.394+0000 7f189da00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1898109b40 0x7f18981a5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:53:35.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:35.394+0000 7f189d000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1898107f90 0x7f189819e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:53:35.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:35.394+0000 7f189da00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1898109b40 0x7f18981a5610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41296/0 (socket says 172.21.15.97:41296) 2024-03-27T10:53:35.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:35.394+0000 7f189d000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1898107f90 0x7f189819e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.97:51472/0 (socket says 172.21.15.97:51472) 2024-03-27T10:53:35.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:35.394+0000 7f189da00640 1 -- 172.21.15.97:0/511934408 learned_addr learned my addr 172.21.15.97:0/511934408 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:53:35.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:35.394+0000 7f1897e00640 1 --2- 172.21.15.97:0/511934408 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1898109180 0x7f189819e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:53:35.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:35.394+0000 7f189da00640 1 -- 172.21.15.97:0/511934408 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1898107f90 msgr2=0x7f189819e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:35.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:35.394+0000 7f189da00640 1 --2- 172.21.15.97:0/511934408 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1898107f90 0x7f189819e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:35.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:35.394+0000 7f189da00640 1 -- 172.21.15.97:0/511934408 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1898109180 msgr2=0x7f189819e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:35.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:35.394+0000 7f189da00640 1 --2- 172.21.15.97:0/511934408 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1898109180 0x7f189819e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:35.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:35.394+0000 7f189da00640 1 -- 172.21.15.97:0/511934408 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18981a5c80 con 0x7f1898109b40 2024-03-27T10:53:35.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:35.394+0000 7f189da00640 1 --2- 172.21.15.97:0/511934408 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1898109b40 0x7f18981a5610 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1890032520 tx=0x7f1890035ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:53:35.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:35.395+0000 7f1895600640 1 -- 172.21.15.97:0/511934408 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1890039040 con 0x7f1898109b40 2024-03-27T10:53:35.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:35.395+0000 7f189e400640 1 -- 172.21.15.97:0/511934408 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18981a5f10 con 0x7f1898109b40 2024-03-27T10:53:35.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:35.395+0000 7f1895600640 1 -- 172.21.15.97:0/511934408 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1890044070 con 0x7f1898109b40 2024-03-27T10:53:35.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:35.395+0000 7f1895600640 1 -- 172.21.15.97:0/511934408 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18900043c0 con 0x7f1898109b40 2024-03-27T10:53:35.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:35.395+0000 7f189e400640 1 -- 172.21.15.97:0/511934408 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18981a6380 con 0x7f1898109b40 2024-03-27T10:53:35.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:35.396+0000 7f189e400640 1 -- 172.21.15.97:0/511934408 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f189810f250 con 0x7f1898109b40 2024-03-27T10:53:35.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:35.398+0000 7f1895600640 1 -- 172.21.15.97:0/511934408 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138370+0+0 (secure 0 0 0) 0x7f1890004560 con 0x7f1898109b40 2024-03-27T10:53:35.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:35.399+0000 7f1895600640 1 --2- 172.21.15.97:0/511934408 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f18800a3610 0x7f18800a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:53:35.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:35.400+0000 7f1895600640 1 -- 172.21.15.97:0/511934408 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7127+0+0 (secure 0 0 0) 0x7f18900f1ff0 con 0x7f1898109b40 2024-03-27T10:53:35.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:35.400+0000 7f189d000640 1 --2- 172.21.15.97:0/511934408 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f18800a3610 0x7f18800a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:53:35.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:35.400+0000 7f189d000640 1 --2- 172.21.15.97:0/511934408 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f18800a3610 0x7f18800a5ad0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1898108fe0 tx=0x7f188c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:53:35.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:35.404+0000 7f1895600640 1 -- 172.21.15.97:0/511934408 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f18900bf020 con 0x7f1898109b40 2024-03-27T10:53:35.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:35 smithi114 ceph-mon[35338]: pgmap v343: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T10:53:35.587 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:35 smithi177 ceph-mon[35294]: pgmap v343: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T10:53:35.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:35.646+0000 7f189e400640 1 -- 172.21.15.97:0/511934408 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f18980627f0 con 0x7f1898109b40 2024-03-27T10:53:35.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:35.646+0000 7f1895600640 1 -- 172.21.15.97:0/511934408 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4702 (secure 0 0 0) 0x7f18900be920 con 0x7f1898109b40 2024-03-27T10:53:35.619 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T10:53:35.619 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1009423904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1009423904},{"type":"v1","addr":"172.21.15.97:6835","nonce":1009423904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3251173433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3251173433},{"type":"v1","addr":"172.21.15.97:6837","nonce":3251173433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3787981632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3787981632},{"type":"v1","addr":"172.21.15.177:6833","nonce":3787981632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":34415,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6833/4052517902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":4052517902},{"type":"v1","addr":"172.21.15.114:6833","nonce":4052517902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34445,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/3581918222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3581918222},{"type":"v1","addr":"172.21.15.114:6835","nonce":3581918222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-03-27T10:53:35.619 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 6 2024-03-27T10:53:35.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:35.649+0000 7f189e400640 1 -- 172.21.15.97:0/511934408 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f18800a3610 msgr2=0x7f18800a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:35.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:35.649+0000 7f189e400640 1 --2- 172.21.15.97:0/511934408 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f18800a3610 0x7f18800a5ad0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1898108fe0 tx=0x7f188c005f90 comp rx=0 tx=0).stop 2024-03-27T10:53:35.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:35.649+0000 7f189e400640 1 -- 172.21.15.97:0/511934408 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1898109b40 msgr2=0x7f18981a5610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:35.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:35.649+0000 7f189e400640 1 --2- 172.21.15.97:0/511934408 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1898109b40 0x7f18981a5610 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1890032520 tx=0x7f1890035ce0 comp rx=0 tx=0).stop 2024-03-27T10:53:35.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:35.649+0000 7f189e400640 1 -- 172.21.15.97:0/511934408 shutdown_connections 2024-03-27T10:53:35.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:35.649+0000 7f189e400640 1 --2- 172.21.15.97:0/511934408 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1898109b40 0x7f18981a5610 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:35.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:35.649+0000 7f189e400640 1 --2- 172.21.15.97:0/511934408 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f18800a3610 0x7f18800a5ad0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:35.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:35.649+0000 7f189e400640 1 --2- 172.21.15.97:0/511934408 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1898109180 0x7f189819e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:35.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:35.649+0000 7f189e400640 1 --2- 172.21.15.97:0/511934408 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1898107f90 0x7f189819e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:35.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:35.649+0000 7f189e400640 1 -- 172.21.15.97:0/511934408 >> 172.21.15.97:0/511934408 conn(0x7f1898076f20 msgr2=0x7f189810c420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:53:35.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:35.649+0000 7f189e400640 1 -- 172.21.15.97:0/511934408 shutdown_connections 2024-03-27T10:53:35.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:35.650+0000 7f189e400640 1 -- 172.21.15.97:0/511934408 wait complete. 2024-03-27T10:53:35.636 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:35 smithi097 ceph-mon[31859]: pgmap v343: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T10:53:35.927 INFO:journalctl@ceph.mds.f.smithi177.stdout:Mar 27 10:53:35 smithi177 systemd[1]: Starting Ceph mds.f for adca88c4-ec26-11ee-95d0-87774f69a715... 2024-03-27T10:53:36.236 INFO:journalctl@ceph.mds.f.smithi177.stdout:Mar 27 10:53:36 smithi177 podman[66143]: 2024-03-27 10:53:35.907459116 +0000 UTC m=+0.031216668 image pull b379bbd8da6b5d0fa9a3523941efac213b6815e1cbf05c6507954f0a3fe7a801 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772 2024-03-27T10:53:36.236 INFO:journalctl@ceph.mds.f.smithi177.stdout:Mar 27 10:53:36 smithi177 podman[66143]: 2024-03-27 10:53:36.00796366 +0000 UTC m=+0.131721252 container create 4a238616523cba3fbe16e747258fb61c033ec87300db453cea07faf41a5f7132 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-f, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-03-27T10:53:36.236 INFO:journalctl@ceph.mds.f.smithi177.stdout:Mar 27 10:53:36 smithi177 podman[66143]: 2024-03-27 10:53:36.261982314 +0000 UTC m=+0.385739912 container init 4a238616523cba3fbe16e747258fb61c033ec87300db453cea07faf41a5f7132 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-f, org.label-schema.build-date=20240326, io.buildah.version=1.33.5, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240326.131558-155268c, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-03-27T10:53:36.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:36 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/511934408' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T10:53:36.641 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:36 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/511934408' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T10:53:36.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:36 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/511934408' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T10:53:36.678 INFO:journalctl@ceph.mds.f.smithi177.stdout:Mar 27 10:53:36 smithi177 podman[66143]: 2024-03-27 10:53:36.271396278 +0000 UTC m=+0.395153913 container start 4a238616523cba3fbe16e747258fb61c033ec87300db453cea07faf41a5f7132 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-f, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240326.131558-155268c, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.vendor=CentOS) 2024-03-27T10:53:36.678 INFO:journalctl@ceph.mds.f.smithi177.stdout:Mar 27 10:53:36 smithi177 ceph-mds[66161]: -- 172.21.15.177:0/834965490 <== mon.1 v2:172.21.15.177:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55bba3fd9a40 con 0x55bba3fad000 2024-03-27T10:53:36.678 INFO:journalctl@ceph.mds.f.smithi177.stdout:Mar 27 10:53:36 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-f[66157]: starting mds.f at 2024-03-27T10:53:36.678 INFO:journalctl@ceph.mds.f.smithi177.stdout:Mar 27 10:53:36 smithi177 bash[66143]: 4a238616523cba3fbe16e747258fb61c033ec87300db453cea07faf41a5f7132 2024-03-27T10:53:36.678 INFO:journalctl@ceph.mds.f.smithi177.stdout:Mar 27 10:53:36 smithi177 systemd[1]: Started Ceph mds.f for adca88c4-ec26-11ee-95d0-87774f69a715. 2024-03-27T10:53:37.602 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:37 smithi177 ceph-mon[35294]: pgmap v344: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T10:53:37.602 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:37 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:37.602 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:37 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:37.602 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:37 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:37.602 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:37 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-27T10:53:37.602 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:37 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-27T10:53:37.603 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:37 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:53:37.639 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-27T10:53:37.647 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:37 smithi097 ceph-mon[31859]: pgmap v344: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T10:53:37.648 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:37 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:37.648 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:37 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:37.648 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:37 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:37.648 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:37 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-27T10:53:37.648 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:37 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-27T10:53:37.648 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:37 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:53:37.767 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:37 smithi114 ceph-mon[35338]: pgmap v344: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T10:53:37.768 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:37 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:37.768 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:37 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:37.768 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:37 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:37.768 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:37 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-27T10:53:37.768 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:37 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-27T10:53:37.768 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:37 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:53:37.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:37.890+0000 7f9172000640 1 -- 172.21.15.97:0/4073180144 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f916c103d90 msgr2=0x7f916c1041b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:37.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:37.890+0000 7f9172000640 1 --2- 172.21.15.97:0/4073180144 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f916c103d90 0x7f916c1041b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9154009900 tx=0x7f915402bc30 comp rx=0 tx=0).stop 2024-03-27T10:53:37.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:37.890+0000 7f9172000640 1 -- 172.21.15.97:0/4073180144 shutdown_connections 2024-03-27T10:53:37.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:37.890+0000 7f9172000640 1 --2- 172.21.15.97:0/4073180144 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f916c105940 0x7f916c10c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:37.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:37.890+0000 7f9172000640 1 --2- 172.21.15.97:0/4073180144 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f916c104f80 0x7f916c105400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:37.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:37.890+0000 7f9172000640 1 --2- 172.21.15.97:0/4073180144 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f916c103d90 0x7f916c1041b0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:37.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:37.890+0000 7f9172000640 1 -- 172.21.15.97:0/4073180144 >> 172.21.15.97:0/4073180144 conn(0x7f916c0ff300 msgr2=0x7f916c101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:53:37.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:37.890+0000 7f9172000640 1 -- 172.21.15.97:0/4073180144 shutdown_connections 2024-03-27T10:53:37.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:37.890+0000 7f9172000640 1 -- 172.21.15.97:0/4073180144 wait complete. 2024-03-27T10:53:37.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:37.891+0000 7f9172000640 1 Processor -- start 2024-03-27T10:53:37.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:37.891+0000 7f9172000640 1 -- start start 2024-03-27T10:53:37.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:37.892+0000 7f9172000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f916c103d90 0x7f916c199fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:53:37.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:37.892+0000 7f9172000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f916c104f80 0x7f916c19a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:53:37.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:37.892+0000 7f9172000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f916c105940 0x7f916c1a1550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:53:37.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:37.892+0000 7f9172000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f916c10f8e0 con 0x7f916c105940 2024-03-27T10:53:37.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:37.892+0000 7f9172000640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f916c10f760 con 0x7f916c103d90 2024-03-27T10:53:37.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:37.892+0000 7f9172000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f916c10fa60 con 0x7f916c104f80 2024-03-27T10:53:37.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:37.892+0000 7f916be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f916c105940 0x7f916c1a1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:53:37.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:37.892+0000 7f916be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f916c105940 0x7f916c1a1550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41320/0 (socket says 172.21.15.97:41320) 2024-03-27T10:53:37.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:37.892+0000 7f916be00640 1 -- 172.21.15.97:0/3814563568 learned_addr learned my addr 172.21.15.97:0/3814563568 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:53:37.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:37.892+0000 7f916b400640 1 --2- 172.21.15.97:0/3814563568 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f916c103d90 0x7f916c199fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:53:37.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:37.893+0000 7f916aa00640 1 --2- 172.21.15.97:0/3814563568 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f916c104f80 0x7f916c19a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:53:37.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:37.893+0000 7f916be00640 1 -- 172.21.15.97:0/3814563568 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f916c104f80 msgr2=0x7f916c19a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:37.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:37.893+0000 7f916be00640 1 --2- 172.21.15.97:0/3814563568 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f916c104f80 0x7f916c19a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:37.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:37.893+0000 7f916be00640 1 -- 172.21.15.97:0/3814563568 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f916c103d90 msgr2=0x7f916c199fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:37.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:37.893+0000 7f916be00640 1 --2- 172.21.15.97:0/3814563568 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f916c103d90 0x7f916c199fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:37.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:37.893+0000 7f916be00640 1 -- 172.21.15.97:0/3814563568 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f916c1a1bc0 con 0x7f916c105940 2024-03-27T10:53:37.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:37.893+0000 7f916b400640 1 --2- 172.21.15.97:0/3814563568 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f916c103d90 0x7f916c199fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T10:53:37.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:37.893+0000 7f916be00640 1 --2- 172.21.15.97:0/3814563568 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f916c105940 0x7f916c1a1550 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f916400eaa0 tx=0x7f916400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:53:37.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:37.893+0000 7f9163e00640 1 -- 172.21.15.97:0/3814563568 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f916400cdb0 con 0x7f916c105940 2024-03-27T10:53:37.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:37.893+0000 7f9163e00640 1 -- 172.21.15.97:0/3814563568 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9164005020 con 0x7f916c105940 2024-03-27T10:53:37.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:37.893+0000 7f9172000640 1 -- 172.21.15.97:0/3814563568 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f916c1a1eb0 con 0x7f916c105940 2024-03-27T10:53:37.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:37.893+0000 7f9163e00640 1 -- 172.21.15.97:0/3814563568 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91640116d0 con 0x7f916c105940 2024-03-27T10:53:37.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:37.894+0000 7f9172000640 1 -- 172.21.15.97:0/3814563568 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f916c1a2320 con 0x7f916c105940 2024-03-27T10:53:37.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:37.895+0000 7f9172000640 1 -- 172.21.15.97:0/3814563568 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f916c10b180 con 0x7f916c105940 2024-03-27T10:53:37.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:37.897+0000 7f9163e00640 1 -- 172.21.15.97:0/3814563568 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138370+0+0 (secure 0 0 0) 0x7f91640051d0 con 0x7f916c105940 2024-03-27T10:53:37.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:37.898+0000 7f9163e00640 1 --2- 172.21.15.97:0/3814563568 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f91500a3610 0x7f91500a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:53:37.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:37.898+0000 7f9163e00640 1 -- 172.21.15.97:0/3814563568 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7127+0+0 (secure 0 0 0) 0x7f91640cfb40 con 0x7f916c105940 2024-03-27T10:53:37.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:37.898+0000 7f916b400640 1 --2- 172.21.15.97:0/3814563568 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f91500a3610 0x7f91500a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:53:37.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:37.899+0000 7f916b400640 1 --2- 172.21.15.97:0/3814563568 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f91500a3610 0x7f91500a5ad0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f915402c240 tx=0x7f9154005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:53:37.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:37.902+0000 7f9163e00640 1 -- 172.21.15.97:0/3814563568 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f9164016030 con 0x7f916c105940 2024-03-27T10:53:38.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:38.135+0000 7f9172000640 1 -- 172.21.15.97:0/3814563568 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f916c02d370 con 0x7f916c105940 2024-03-27T10:53:38.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:38.136+0000 7f9163e00640 1 -- 172.21.15.97:0/3814563568 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5530 (secure 0 0 0) 0x7f916409cb10 con 0x7f916c105940 2024-03-27T10:53:38.108 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T10:53:38.108 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1009423904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1009423904},{"type":"v1","addr":"172.21.15.97:6835","nonce":1009423904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3251173433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3251173433},{"type":"v1","addr":"172.21.15.97:6837","nonce":3251173433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3787981632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3787981632},{"type":"v1","addr":"172.21.15.177:6833","nonce":3787981632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/3349112132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":3349112132},{"type":"v1","addr":"172.21.15.177:6835","nonce":3349112132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":34415,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6833/4052517902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":4052517902},{"type":"v1","addr":"172.21.15.114:6833","nonce":4052517902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34445,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/3581918222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3581918222},{"type":"v1","addr":"172.21.15.114:6835","nonce":3581918222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-03-27T10:53:38.109 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 7 2024-03-27T10:53:38.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:38.138+0000 7f9172000640 1 -- 172.21.15.97:0/3814563568 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f91500a3610 msgr2=0x7f91500a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:38.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:38.138+0000 7f9172000640 1 --2- 172.21.15.97:0/3814563568 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f91500a3610 0x7f91500a5ad0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f915402c240 tx=0x7f9154005fd0 comp rx=0 tx=0).stop 2024-03-27T10:53:38.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:38.139+0000 7f9172000640 1 -- 172.21.15.97:0/3814563568 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f916c105940 msgr2=0x7f916c1a1550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:38.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:38.139+0000 7f9172000640 1 --2- 172.21.15.97:0/3814563568 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f916c105940 0x7f916c1a1550 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f916400eaa0 tx=0x7f916400ef60 comp rx=0 tx=0).stop 2024-03-27T10:53:38.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:38.139+0000 7f9172000640 1 -- 172.21.15.97:0/3814563568 shutdown_connections 2024-03-27T10:53:38.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:38.139+0000 7f9172000640 1 --2- 172.21.15.97:0/3814563568 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f916c105940 0x7f916c1a1550 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:38.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:38.139+0000 7f9172000640 1 --2- 172.21.15.97:0/3814563568 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f916c104f80 0x7f916c19a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:38.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:38.139+0000 7f9172000640 1 --2- 172.21.15.97:0/3814563568 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f91500a3610 0x7f91500a5ad0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:38.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:38.139+0000 7f9172000640 1 --2- 172.21.15.97:0/3814563568 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f916c103d90 0x7f916c199fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:38.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:38.139+0000 7f9172000640 1 -- 172.21.15.97:0/3814563568 >> 172.21.15.97:0/3814563568 conn(0x7f916c0ff300 msgr2=0x7f916c108350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:53:38.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:38.139+0000 7f9172000640 1 -- 172.21.15.97:0/3814563568 shutdown_connections 2024-03-27T10:53:38.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:38.139+0000 7f9172000640 1 -- 172.21.15.97:0/3814563568 wait complete. 2024-03-27T10:53:38.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:38 smithi097 ceph-mon[31859]: Deploying daemon mds.c on smithi177 2024-03-27T10:53:38.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:38 smithi097 ceph-mon[31859]: mds.? [v2:172.21.15.177:6834/3349112132,v1:172.21.15.177:6835/3349112132] up:boot 2024-03-27T10:53:38.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:38 smithi097 ceph-mon[31859]: fsmap 6 up:standby 2024-03-27T10:53:38.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:38 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-27T10:53:38.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:38 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/3814563568' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T10:53:38.583 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:38 smithi177 ceph-mon[35294]: Deploying daemon mds.c on smithi177 2024-03-27T10:53:38.583 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:38 smithi177 ceph-mon[35294]: mds.? [v2:172.21.15.177:6834/3349112132,v1:172.21.15.177:6835/3349112132] up:boot 2024-03-27T10:53:38.583 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:38 smithi177 ceph-mon[35294]: fsmap 6 up:standby 2024-03-27T10:53:38.583 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:38 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-27T10:53:38.583 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:38 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/3814563568' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T10:53:38.767 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:38 smithi114 ceph-mon[35338]: Deploying daemon mds.c on smithi177 2024-03-27T10:53:38.768 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:38 smithi114 ceph-mon[35338]: mds.? [v2:172.21.15.177:6834/3349112132,v1:172.21.15.177:6835/3349112132] up:boot 2024-03-27T10:53:38.768 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:38 smithi114 ceph-mon[35338]: fsmap 6 up:standby 2024-03-27T10:53:38.768 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:38 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-27T10:53:38.768 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:38 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/3814563568' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T10:53:39.368 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:39 smithi177 ceph-mon[35294]: pgmap v345: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T10:53:39.641 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:39 smithi097 ceph-mon[31859]: pgmap v345: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T10:53:39.677 INFO:journalctl@ceph.mds.c.smithi177.stdout:Mar 27 10:53:39 smithi177 systemd[1]: Starting Ceph mds.c for adca88c4-ec26-11ee-95d0-87774f69a715... 2024-03-27T10:53:39.767 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:39 smithi114 ceph-mon[35338]: pgmap v345: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T10:53:40.005 INFO:journalctl@ceph.mds.c.smithi177.stdout:Mar 27 10:53:39 smithi177 podman[66442]: 2024-03-27 10:53:39.745178356 +0000 UTC m=+0.126415282 container create 5ffb343bec61f7e87208fb4fd64bc226bdbe48924ae95e1c4efdf4729ce9fc7c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-c, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-03-27T10:53:40.006 INFO:journalctl@ceph.mds.c.smithi177.stdout:Mar 27 10:53:39 smithi177 podman[66442]: 2024-03-27 10:53:39.653948415 +0000 UTC m=+0.035185311 image pull b379bbd8da6b5d0fa9a3523941efac213b6815e1cbf05c6507954f0a3fe7a801 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772 2024-03-27T10:53:40.006 INFO:journalctl@ceph.mds.c.smithi177.stdout:Mar 27 10:53:40 smithi177 podman[66442]: 2024-03-27 10:53:40.032486833 +0000 UTC m=+0.413723770 container init 5ffb343bec61f7e87208fb4fd64bc226bdbe48924ae95e1c4efdf4729ce9fc7c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-c, RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_CLEAN=True, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-03-27T10:53:40.131 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-27T10:53:40.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:40.383+0000 7f99e4c00640 1 -- 172.21.15.97:0/662573599 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99e0104f80 msgr2=0x7f99e0105400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:40.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:40.383+0000 7f99e4c00640 1 --2- 172.21.15.97:0/662573599 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99e0104f80 0x7f99e0105400 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f99c8009900 tx=0x7f99c802bc30 comp rx=0 tx=0).stop 2024-03-27T10:53:40.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:40.383+0000 7f99e4c00640 1 -- 172.21.15.97:0/662573599 shutdown_connections 2024-03-27T10:53:40.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:40.383+0000 7f99e4c00640 1 --2- 172.21.15.97:0/662573599 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f99e0105940 0x7f99e010c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:40.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:40.383+0000 7f99e4c00640 1 --2- 172.21.15.97:0/662573599 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99e0104f80 0x7f99e0105400 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:40.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:40.383+0000 7f99e4c00640 1 --2- 172.21.15.97:0/662573599 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f99e0103d90 0x7f99e01041b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:40.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:40.383+0000 7f99e4c00640 1 -- 172.21.15.97:0/662573599 >> 172.21.15.97:0/662573599 conn(0x7f99e00ff300 msgr2=0x7f99e0101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:53:40.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:40.383+0000 7f99e4c00640 1 -- 172.21.15.97:0/662573599 shutdown_connections 2024-03-27T10:53:40.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:40.383+0000 7f99e4c00640 1 -- 172.21.15.97:0/662573599 wait complete. 2024-03-27T10:53:40.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:40.384+0000 7f99e4c00640 1 Processor -- start 2024-03-27T10:53:40.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:40.384+0000 7f99e4c00640 1 -- start start 2024-03-27T10:53:40.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:40.384+0000 7f99e4c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f99e0103d90 0x7f99e0199f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:53:40.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:40.384+0000 7f99e4c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99e0104f80 0x7f99e019a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:53:40.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:40.384+0000 7f99e4c00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f99e0105940 0x7f99e01a1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:53:40.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:40.384+0000 7f99e4c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f99e010f910 con 0x7f99e0104f80 2024-03-27T10:53:40.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:40.384+0000 7f99e4c00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f99e010f790 con 0x7f99e0103d90 2024-03-27T10:53:40.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:40.384+0000 7f99e4c00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f99e010fa90 con 0x7f99e0105940 2024-03-27T10:53:40.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:40.384+0000 7f99d7e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99e0104f80 0x7f99e019a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:53:40.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:40.384+0000 7f99d7e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99e0104f80 0x7f99e019a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45452/0 (socket says 172.21.15.97:45452) 2024-03-27T10:53:40.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:40.384+0000 7f99d7e00640 1 -- 172.21.15.97:0/513478591 learned_addr learned my addr 172.21.15.97:0/513478591 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:53:40.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:40.385+0000 7f99dde00640 1 --2- 172.21.15.97:0/513478591 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f99e0103d90 0x7f99e0199f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:53:40.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:40.385+0000 7f99d7e00640 1 -- 172.21.15.97:0/513478591 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f99e0105940 msgr2=0x7f99e01a1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:40.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:40.385+0000 7f99de800640 1 --2- 172.21.15.97:0/513478591 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f99e0105940 0x7f99e01a1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:53:40.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:40.385+0000 7f99d7e00640 1 --2- 172.21.15.97:0/513478591 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f99e0105940 0x7f99e01a1540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:40.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:40.385+0000 7f99d7e00640 1 -- 172.21.15.97:0/513478591 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f99e0103d90 msgr2=0x7f99e0199f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:40.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:40.385+0000 7f99d7e00640 1 --2- 172.21.15.97:0/513478591 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f99e0103d90 0x7f99e0199f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:40.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:40.385+0000 7f99d7e00640 1 -- 172.21.15.97:0/513478591 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99e01a1bb0 con 0x7f99e0104f80 2024-03-27T10:53:40.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:40.385+0000 7f99d7e00640 1 --2- 172.21.15.97:0/513478591 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99e0104f80 0x7f99e019a4d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f99c8000c00 tx=0x7f99c80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:53:40.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:40.385+0000 7f99d6a00640 1 -- 172.21.15.97:0/513478591 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c8043020 con 0x7f99e0104f80 2024-03-27T10:53:40.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:40.385+0000 7f99d6a00640 1 -- 172.21.15.97:0/513478591 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f99c8004aa0 con 0x7f99e0104f80 2024-03-27T10:53:40.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:40.385+0000 7f99e4c00640 1 -- 172.21.15.97:0/513478591 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99e01a1e40 con 0x7f99e0104f80 2024-03-27T10:53:40.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:40.386+0000 7f99d6a00640 1 -- 172.21.15.97:0/513478591 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c8003d70 con 0x7f99e0104f80 2024-03-27T10:53:40.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:40.386+0000 7f99e4c00640 1 -- 172.21.15.97:0/513478591 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99e01a2300 con 0x7f99e0104f80 2024-03-27T10:53:40.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:40.387+0000 7f99e4c00640 1 -- 172.21.15.97:0/513478591 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99e010b130 con 0x7f99e0104f80 2024-03-27T10:53:40.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:40.388+0000 7f99d6a00640 1 -- 172.21.15.97:0/513478591 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138370+0+0 (secure 0 0 0) 0x7f99c8046020 con 0x7f99e0104f80 2024-03-27T10:53:40.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:40.390+0000 7f99d6a00640 1 --2- 172.21.15.97:0/513478591 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f99c40a3610 0x7f99c40a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:53:40.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:40.390+0000 7f99d6a00640 1 -- 172.21.15.97:0/513478591 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7127+0+0 (secure 0 0 0) 0x7f99c80f1c60 con 0x7f99e0104f80 2024-03-27T10:53:40.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:40.390+0000 7f99dde00640 1 --2- 172.21.15.97:0/513478591 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f99c40a3610 0x7f99c40a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:53:40.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:40.391+0000 7f99dde00640 1 --2- 172.21.15.97:0/513478591 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f99c40a3610 0x7f99c40a5ad0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f99e0104de0 tx=0x7f99d0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:53:40.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:40.394+0000 7f99d6a00640 1 -- 172.21.15.97:0/513478591 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f99c804d020 con 0x7f99e0104f80 2024-03-27T10:53:40.427 INFO:journalctl@ceph.mds.c.smithi177.stdout:Mar 27 10:53:40 smithi177 podman[66442]: 2024-03-27 10:53:40.042453075 +0000 UTC m=+0.423689974 container start 5ffb343bec61f7e87208fb4fd64bc226bdbe48924ae95e1c4efdf4729ce9fc7c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-c, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240326.131558-155268c, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-03-27T10:53:40.427 INFO:journalctl@ceph.mds.c.smithi177.stdout:Mar 27 10:53:40 smithi177 ceph-mds[66460]: -- 172.21.15.177:0/1811039610 <== mon.1 v2:172.21.15.177:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564f51355a40 con 0x564f51328c00 2024-03-27T10:53:40.427 INFO:journalctl@ceph.mds.c.smithi177.stdout:Mar 27 10:53:40 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-c[66456]: starting mds.c at 2024-03-27T10:53:40.428 INFO:journalctl@ceph.mds.c.smithi177.stdout:Mar 27 10:53:40 smithi177 bash[66442]: 5ffb343bec61f7e87208fb4fd64bc226bdbe48924ae95e1c4efdf4729ce9fc7c 2024-03-27T10:53:40.428 INFO:journalctl@ceph.mds.c.smithi177.stdout:Mar 27 10:53:40 smithi177 systemd[1]: Started Ceph mds.c for adca88c4-ec26-11ee-95d0-87774f69a715. 2024-03-27T10:53:40.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:40.634+0000 7f99e4c00640 1 -- 172.21.15.97:0/513478591 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f99e00627f0 con 0x7f99e0104f80 2024-03-27T10:53:40.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:40.636+0000 7f99d6a00640 1 -- 172.21.15.97:0/513478591 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6358 (secure 0 0 0) 0x7f99c80bec30 con 0x7f99e0104f80 2024-03-27T10:53:40.608 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T10:53:40.608 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1009423904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1009423904},{"type":"v1","addr":"172.21.15.97:6835","nonce":1009423904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3251173433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3251173433},{"type":"v1","addr":"172.21.15.97:6837","nonce":3251173433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3787981632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3787981632},{"type":"v1","addr":"172.21.15.177:6833","nonce":3787981632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/3349112132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":3349112132},{"type":"v1","addr":"172.21.15.177:6835","nonce":3349112132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6837/3244432640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":3244432640},{"type":"v1","addr":"172.21.15.177:6837","nonce":3244432640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":34415,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6833/4052517902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":4052517902},{"type":"v1","addr":"172.21.15.114:6833","nonce":4052517902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34445,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/3581918222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3581918222},{"type":"v1","addr":"172.21.15.114:6835","nonce":3581918222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-03-27T10:53:40.608 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 8 2024-03-27T10:53:40.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:40.638+0000 7f99e4c00640 1 -- 172.21.15.97:0/513478591 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f99c40a3610 msgr2=0x7f99c40a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:40.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:40.638+0000 7f99e4c00640 1 --2- 172.21.15.97:0/513478591 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f99c40a3610 0x7f99c40a5ad0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f99e0104de0 tx=0x7f99d0005f90 comp rx=0 tx=0).stop 2024-03-27T10:53:40.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:40.638+0000 7f99e4c00640 1 -- 172.21.15.97:0/513478591 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99e0104f80 msgr2=0x7f99e019a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:40.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:40.638+0000 7f99e4c00640 1 --2- 172.21.15.97:0/513478591 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99e0104f80 0x7f99e019a4d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f99c8000c00 tx=0x7f99c80042f0 comp rx=0 tx=0).stop 2024-03-27T10:53:40.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:40.638+0000 7f99e4c00640 1 -- 172.21.15.97:0/513478591 shutdown_connections 2024-03-27T10:53:40.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:40.638+0000 7f99e4c00640 1 --2- 172.21.15.97:0/513478591 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f99e0105940 0x7f99e01a1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:40.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:40.638+0000 7f99e4c00640 1 --2- 172.21.15.97:0/513478591 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99e0104f80 0x7f99e019a4d0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:40.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:40.638+0000 7f99e4c00640 1 --2- 172.21.15.97:0/513478591 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f99c40a3610 0x7f99c40a5ad0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:40.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:40.638+0000 7f99e4c00640 1 --2- 172.21.15.97:0/513478591 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f99e0103d90 0x7f99e0199f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:40.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:40.638+0000 7f99e4c00640 1 -- 172.21.15.97:0/513478591 >> 172.21.15.97:0/513478591 conn(0x7f99e00ff300 msgr2=0x7f99e0108300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:53:40.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:40.639+0000 7f99e4c00640 1 -- 172.21.15.97:0/513478591 shutdown_connections 2024-03-27T10:53:40.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:40.639+0000 7f99e4c00640 1 -- 172.21.15.97:0/513478591 wait complete. 2024-03-27T10:53:41.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:41 smithi114 ceph-mon[35338]: pgmap v346: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T10:53:41.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:41 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:41.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:41 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:41.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:41 smithi114 ceph-mon[35338]: mds.? [v2:172.21.15.177:6836/3244432640,v1:172.21.15.177:6837/3244432640] up:boot 2024-03-27T10:53:41.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:41 smithi114 ceph-mon[35338]: fsmap 7 up:standby 2024-03-27T10:53:41.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:41 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-27T10:53:41.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:41 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:41.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:41 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-27T10:53:41.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:41 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-27T10:53:41.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:41 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:53:41.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:41 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/513478591' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T10:53:41.642 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:41 smithi097 ceph-mon[31859]: pgmap v346: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T10:53:41.642 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:41 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:41.642 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:41 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:41.642 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:41 smithi097 ceph-mon[31859]: mds.? [v2:172.21.15.177:6836/3244432640,v1:172.21.15.177:6837/3244432640] up:boot 2024-03-27T10:53:41.642 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:41 smithi097 ceph-mon[31859]: fsmap 7 up:standby 2024-03-27T10:53:41.642 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:41 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-27T10:53:41.642 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:41 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:41.642 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:41 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-27T10:53:41.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:41 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-27T10:53:41.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:41 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:53:41.643 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:41 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/513478591' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T10:53:41.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:41 smithi177 ceph-mon[35294]: pgmap v346: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T10:53:41.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:41 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:41.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:41 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:41.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:41 smithi177 ceph-mon[35294]: mds.? [v2:172.21.15.177:6836/3244432640,v1:172.21.15.177:6837/3244432640] up:boot 2024-03-27T10:53:41.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:41 smithi177 ceph-mon[35294]: fsmap 7 up:standby 2024-03-27T10:53:41.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:41 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-27T10:53:41.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:41 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:41.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:41 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-27T10:53:41.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:41 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-27T10:53:41.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:41 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:53:41.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:41 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/513478591' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T10:53:42.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:42 smithi114 ceph-mon[35338]: Deploying daemon mds.h on smithi114 2024-03-27T10:53:42.629 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-27T10:53:42.641 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:42 smithi097 ceph-mon[31859]: Deploying daemon mds.h on smithi114 2024-03-27T10:53:42.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:42 smithi177 ceph-mon[35294]: Deploying daemon mds.h on smithi114 2024-03-27T10:53:42.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:42.883+0000 7f3b14800640 1 -- 172.21.15.97:0/3214161428 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b0c105920 msgr2=0x7f3b0c10c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:42.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:42.883+0000 7f3b14800640 1 --2- 172.21.15.97:0/3214161428 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b0c105920 0x7f3b0c10c1b0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f3b08009ca0 tx=0x7f3b0802c0f0 comp rx=0 tx=0).stop 2024-03-27T10:53:42.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:42.883+0000 7f3b14800640 1 -- 172.21.15.97:0/3214161428 shutdown_connections 2024-03-27T10:53:42.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:42.883+0000 7f3b14800640 1 --2- 172.21.15.97:0/3214161428 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b0c105920 0x7f3b0c10c1b0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:42.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:42.883+0000 7f3b14800640 1 --2- 172.21.15.97:0/3214161428 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3b0c104f60 0x7f3b0c1053e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:42.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:42.883+0000 7f3b14800640 1 --2- 172.21.15.97:0/3214161428 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3b0c103d70 0x7f3b0c104190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:42.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:42.883+0000 7f3b14800640 1 -- 172.21.15.97:0/3214161428 >> 172.21.15.97:0/3214161428 conn(0x7f3b0c0ff300 msgr2=0x7f3b0c101720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:53:42.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:42.883+0000 7f3b14800640 1 -- 172.21.15.97:0/3214161428 shutdown_connections 2024-03-27T10:53:42.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:42.884+0000 7f3b14800640 1 -- 172.21.15.97:0/3214161428 wait complete. 2024-03-27T10:53:42.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:42.884+0000 7f3b14800640 1 Processor -- start 2024-03-27T10:53:42.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:42.885+0000 7f3b14800640 1 -- start start 2024-03-27T10:53:42.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:42.885+0000 7f3b14800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b0c103d70 0x7f3b0c199f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:53:42.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:42.885+0000 7f3b14800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3b0c104f60 0x7f3b0c19a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:53:42.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:42.885+0000 7f3b14800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3b0c105920 0x7f3b0c1a14c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:53:42.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:42.885+0000 7f3b14800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3b0c10f8a0 con 0x7f3b0c103d70 2024-03-27T10:53:42.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:42.885+0000 7f3b14800640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f3b0c10f720 con 0x7f3b0c104f60 2024-03-27T10:53:42.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:42.885+0000 7f3b14800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f3b0c10fa20 con 0x7f3b0c105920 2024-03-27T10:53:42.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:42.885+0000 7f3b11e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b0c103d70 0x7f3b0c199f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:53:42.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:42.885+0000 7f3b11e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b0c103d70 0x7f3b0c199f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45466/0 (socket says 172.21.15.97:45466) 2024-03-27T10:53:42.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:42.885+0000 7f3b11e00640 1 -- 172.21.15.97:0/3988212496 learned_addr learned my addr 172.21.15.97:0/3988212496 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:53:42.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:42.886+0000 7f3b11400640 1 --2- 172.21.15.97:0/3988212496 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3b0c104f60 0x7f3b0c19a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:53:42.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:42.886+0000 7f3b12800640 1 --2- 172.21.15.97:0/3988212496 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3b0c105920 0x7f3b0c1a14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:53:42.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:42.886+0000 7f3b11e00640 1 -- 172.21.15.97:0/3988212496 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3b0c105920 msgr2=0x7f3b0c1a14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:42.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:42.886+0000 7f3b11e00640 1 --2- 172.21.15.97:0/3988212496 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3b0c105920 0x7f3b0c1a14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:42.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:42.886+0000 7f3b11e00640 1 -- 172.21.15.97:0/3988212496 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3b0c104f60 msgr2=0x7f3b0c19a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:42.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:42.886+0000 7f3b11e00640 1 --2- 172.21.15.97:0/3988212496 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3b0c104f60 0x7f3b0c19a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:42.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:42.886+0000 7f3b11e00640 1 -- 172.21.15.97:0/3988212496 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b0c1a1b30 con 0x7f3b0c103d70 2024-03-27T10:53:42.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:42.886+0000 7f3b11400640 1 --2- 172.21.15.97:0/3988212496 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3b0c104f60 0x7f3b0c19a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:53:42.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:42.886+0000 7f3b12800640 1 --2- 172.21.15.97:0/3988212496 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3b0c105920 0x7f3b0c1a14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:53:42.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:42.886+0000 7f3b11e00640 1 --2- 172.21.15.97:0/3988212496 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b0c103d70 0x7f3b0c199f10 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3b0000d950 tx=0x7f3b0000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:53:42.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:42.886+0000 7f3b06a00640 1 -- 172.21.15.97:0/3988212496 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b00017070 con 0x7f3b0c103d70 2024-03-27T10:53:42.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:42.886+0000 7f3b06a00640 1 -- 172.21.15.97:0/3988212496 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3b00009c30 con 0x7f3b0c103d70 2024-03-27T10:53:42.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:42.886+0000 7f3b14800640 1 -- 172.21.15.97:0/3988212496 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b0c1a1e20 con 0x7f3b0c103d70 2024-03-27T10:53:42.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:42.886+0000 7f3b06a00640 1 -- 172.21.15.97:0/3988212496 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b0000f620 con 0x7f3b0c103d70 2024-03-27T10:53:42.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:42.886+0000 7f3b14800640 1 -- 172.21.15.97:0/3988212496 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b0c1a2280 con 0x7f3b0c103d70 2024-03-27T10:53:42.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:42.888+0000 7f3b14800640 1 -- 172.21.15.97:0/3988212496 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b0c10b0b0 con 0x7f3b0c103d70 2024-03-27T10:53:42.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:42.890+0000 7f3b06a00640 1 -- 172.21.15.97:0/3988212496 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138370+0+0 (secure 0 0 0) 0x7f3b0001a070 con 0x7f3b0c103d70 2024-03-27T10:53:42.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:42.891+0000 7f3b06a00640 1 --2- 172.21.15.97:0/3988212496 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f3af40a3690 0x7f3af40a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:53:42.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:42.891+0000 7f3b06a00640 1 -- 172.21.15.97:0/3988212496 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7127+0+0 (secure 0 0 0) 0x7f3b000cf3b0 con 0x7f3b0c103d70 2024-03-27T10:53:42.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:42.891+0000 7f3b11400640 1 --2- 172.21.15.97:0/3988212496 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f3af40a3690 0x7f3af40a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:53:42.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:42.892+0000 7f3b11400640 1 --2- 172.21.15.97:0/3988212496 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f3af40a3690 0x7f3af40a5b50 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c19b420 tx=0x7f3afc005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:53:42.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:42.895+0000 7f3b06a00640 1 -- 172.21.15.97:0/3988212496 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f3b00009de0 con 0x7f3b0c103d70 2024-03-27T10:53:43.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:43.134+0000 7f3b14800640 1 -- 172.21.15.97:0/3988212496 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3b0c02d620 con 0x7f3b0c103d70 2024-03-27T10:53:43.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:43.135+0000 7f3b06a00640 1 -- 172.21.15.97:0/3988212496 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6358 (secure 0 0 0) 0x7f3b0009c380 con 0x7f3b0c103d70 2024-03-27T10:53:43.107 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T10:53:43.107 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1009423904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1009423904},{"type":"v1","addr":"172.21.15.97:6835","nonce":1009423904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3251173433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3251173433},{"type":"v1","addr":"172.21.15.97:6837","nonce":3251173433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3787981632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3787981632},{"type":"v1","addr":"172.21.15.177:6833","nonce":3787981632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/3349112132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":3349112132},{"type":"v1","addr":"172.21.15.177:6835","nonce":3349112132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6837/3244432640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":3244432640},{"type":"v1","addr":"172.21.15.177:6837","nonce":3244432640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":34415,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6833/4052517902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":4052517902},{"type":"v1","addr":"172.21.15.114:6833","nonce":4052517902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34445,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/3581918222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3581918222},{"type":"v1","addr":"172.21.15.114:6835","nonce":3581918222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-03-27T10:53:43.107 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 8 2024-03-27T10:53:43.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:43.137+0000 7f3b14800640 1 -- 172.21.15.97:0/3988212496 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f3af40a3690 msgr2=0x7f3af40a5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:43.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:43.137+0000 7f3b14800640 1 --2- 172.21.15.97:0/3988212496 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f3af40a3690 0x7f3af40a5b50 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c19b420 tx=0x7f3afc005e50 comp rx=0 tx=0).stop 2024-03-27T10:53:43.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:43.137+0000 7f3b14800640 1 -- 172.21.15.97:0/3988212496 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b0c103d70 msgr2=0x7f3b0c199f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:43.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:43.137+0000 7f3b14800640 1 --2- 172.21.15.97:0/3988212496 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b0c103d70 0x7f3b0c199f10 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3b0000d950 tx=0x7f3b0000de10 comp rx=0 tx=0).stop 2024-03-27T10:53:43.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:43.137+0000 7f3b14800640 1 -- 172.21.15.97:0/3988212496 shutdown_connections 2024-03-27T10:53:43.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:43.137+0000 7f3b14800640 1 --2- 172.21.15.97:0/3988212496 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3b0c105920 0x7f3b0c1a14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:43.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:43.137+0000 7f3b14800640 1 --2- 172.21.15.97:0/3988212496 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f3af40a3690 0x7f3af40a5b50 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:43.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:43.137+0000 7f3b14800640 1 --2- 172.21.15.97:0/3988212496 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3b0c104f60 0x7f3b0c19a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:43.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:43.137+0000 7f3b14800640 1 --2- 172.21.15.97:0/3988212496 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b0c103d70 0x7f3b0c199f10 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:43.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:43.137+0000 7f3b14800640 1 -- 172.21.15.97:0/3988212496 >> 172.21.15.97:0/3988212496 conn(0x7f3b0c0ff300 msgr2=0x7f3b0c108280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:53:43.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:43.137+0000 7f3b14800640 1 -- 172.21.15.97:0/3988212496 shutdown_connections 2024-03-27T10:53:43.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:43.137+0000 7f3b14800640 1 -- 172.21.15.97:0/3988212496 wait complete. 2024-03-27T10:53:43.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:43 smithi097 ceph-mon[31859]: pgmap v347: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T10:53:43.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:43 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/3988212496' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T10:53:43.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:43 smithi114 ceph-mon[35338]: pgmap v347: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T10:53:43.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:43 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/3988212496' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T10:53:43.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:43 smithi177 ceph-mon[35294]: pgmap v347: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T10:53:43.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:43 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/3988212496' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T10:53:43.884 INFO:journalctl@ceph.mds.h.smithi114.stdout:Mar 27 10:53:43 smithi114 systemd[1]: Starting Ceph mds.h for adca88c4-ec26-11ee-95d0-87774f69a715... 2024-03-27T10:53:44.178 INFO:journalctl@ceph.mds.h.smithi114.stdout:Mar 27 10:53:43 smithi114 podman[65540]: 2024-03-27 10:53:43.815993271 +0000 UTC m=+0.032730843 image pull b379bbd8da6b5d0fa9a3523941efac213b6815e1cbf05c6507954f0a3fe7a801 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772 2024-03-27T10:53:44.178 INFO:journalctl@ceph.mds.h.smithi114.stdout:Mar 27 10:53:43 smithi114 podman[65540]: 2024-03-27 10:53:43.919889176 +0000 UTC m=+0.136626772 container create 21762b79bb89e8819bfc30cec93becabe29481c20677e6f5ca410dc2e47ad7d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-h, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, RELEASE=wip-rishabh-testing-20240326.131558-155268c, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-27T10:53:44.178 INFO:journalctl@ceph.mds.h.smithi114.stdout:Mar 27 10:53:44 smithi114 podman[65540]: 2024-03-27 10:53:44.14889049 +0000 UTC m=+0.365628092 container init 21762b79bb89e8819bfc30cec93becabe29481c20677e6f5ca410dc2e47ad7d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-h, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-03-27T10:53:44.178 INFO:journalctl@ceph.mds.h.smithi114.stdout:Mar 27 10:53:44 smithi114 podman[65540]: 2024-03-27 10:53:44.158290334 +0000 UTC m=+0.375027937 container start 21762b79bb89e8819bfc30cec93becabe29481c20677e6f5ca410dc2e47ad7d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-h, GIT_CLEAN=True, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-03-27T10:53:44.517 INFO:journalctl@ceph.mds.h.smithi114.stdout:Mar 27 10:53:44 smithi114 ceph-mds[65559]: -- 172.21.15.114:0/2739434379 <== mon.2 v2:172.21.15.114:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563b891b1a40 con 0x563b89184c00 2024-03-27T10:53:44.518 INFO:journalctl@ceph.mds.h.smithi114.stdout:Mar 27 10:53:44 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-h[65555]: starting mds.h at 2024-03-27T10:53:44.518 INFO:journalctl@ceph.mds.h.smithi114.stdout:Mar 27 10:53:44 smithi114 bash[65540]: 21762b79bb89e8819bfc30cec93becabe29481c20677e6f5ca410dc2e47ad7d6 2024-03-27T10:53:44.518 INFO:journalctl@ceph.mds.h.smithi114.stdout:Mar 27 10:53:44 smithi114 systemd[1]: Started Ceph mds.h for adca88c4-ec26-11ee-95d0-87774f69a715. 2024-03-27T10:53:45.128 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-27T10:53:45.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:45.382+0000 7f535d400640 1 -- 172.21.15.97:0/4190137724 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5358101b60 msgr2=0x7f5358101f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:45.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:45.382+0000 7f535d400640 1 --2- 172.21.15.97:0/4190137724 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5358101b60 0x7f5358101f80 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5348009900 tx=0x7f534802bc30 comp rx=0 tx=0).stop 2024-03-27T10:53:45.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:45.383+0000 7f535d400640 1 -- 172.21.15.97:0/4190137724 shutdown_connections 2024-03-27T10:53:45.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:45.383+0000 7f535d400640 1 --2- 172.21.15.97:0/4190137724 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5358103710 0x7f5358109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:45.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:45.383+0000 7f535d400640 1 --2- 172.21.15.97:0/4190137724 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5358102d50 0x7f53581031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:45.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:45.383+0000 7f535d400640 1 --2- 172.21.15.97:0/4190137724 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5358101b60 0x7f5358101f80 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:45.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:45.383+0000 7f535d400640 1 -- 172.21.15.97:0/4190137724 >> 172.21.15.97:0/4190137724 conn(0x7f53580fd0f0 msgr2=0x7f53580ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:53:45.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:45.383+0000 7f535d400640 1 -- 172.21.15.97:0/4190137724 shutdown_connections 2024-03-27T10:53:45.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:45.383+0000 7f535d400640 1 -- 172.21.15.97:0/4190137724 wait complete. 2024-03-27T10:53:45.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:45.384+0000 7f535d400640 1 Processor -- start 2024-03-27T10:53:45.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:45.384+0000 7f535d400640 1 -- start start 2024-03-27T10:53:45.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:45.384+0000 7f535d400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5358101b60 0x7f5358075900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:53:45.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:45.384+0000 7f535d400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5358102d50 0x7f5358073f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:53:45.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:45.384+0000 7f535d400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5358103710 0x7f5358074480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:53:45.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:45.384+0000 7f535d400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f535810d6e0 con 0x7f5358102d50 2024-03-27T10:53:45.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:45.384+0000 7f535d400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f535810d560 con 0x7f5358103710 2024-03-27T10:53:45.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:45.384+0000 7f535d400640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f535810d860 con 0x7f5358101b60 2024-03-27T10:53:45.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:45.384+0000 7f5355e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5358102d50 0x7f5358073f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:53:45.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:45.384+0000 7f5355e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5358102d50 0x7f5358073f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45474/0 (socket says 172.21.15.97:45474) 2024-03-27T10:53:45.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:45.384+0000 7f5355e00640 1 -- 172.21.15.97:0/3640727730 learned_addr learned my addr 172.21.15.97:0/3640727730 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:53:45.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:45.385+0000 7f5356800640 1 --2- 172.21.15.97:0/3640727730 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5358101b60 0x7f5358075900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:53:45.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:45.385+0000 7f5355e00640 1 -- 172.21.15.97:0/3640727730 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5358101b60 msgr2=0x7f5358075900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:45.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:45.385+0000 7f5357200640 1 --2- 172.21.15.97:0/3640727730 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5358103710 0x7f5358074480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:53:45.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:45.385+0000 7f5355e00640 1 --2- 172.21.15.97:0/3640727730 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5358101b60 0x7f5358075900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:45.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:45.385+0000 7f5355e00640 1 -- 172.21.15.97:0/3640727730 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5358103710 msgr2=0x7f5358074480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:45.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:45.385+0000 7f5355e00640 1 --2- 172.21.15.97:0/3640727730 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5358103710 0x7f5358074480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:45.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:45.385+0000 7f5355e00640 1 -- 172.21.15.97:0/3640727730 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5358074d10 con 0x7f5358102d50 2024-03-27T10:53:45.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:45.385+0000 7f5355e00640 1 --2- 172.21.15.97:0/3640727730 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5358102d50 0x7f5358073f40 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f534400d950 tx=0x7f534400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:53:45.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:45.385+0000 7f534f400640 1 -- 172.21.15.97:0/3640727730 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5344017070 con 0x7f5358102d50 2024-03-27T10:53:45.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:45.385+0000 7f534f400640 1 -- 172.21.15.97:0/3640727730 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5344009c30 con 0x7f5358102d50 2024-03-27T10:53:45.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:45.385+0000 7f534f400640 1 -- 172.21.15.97:0/3640727730 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f534400f620 con 0x7f5358102d50 2024-03-27T10:53:45.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:45.385+0000 7f535d400640 1 -- 172.21.15.97:0/3640727730 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53581a1ea0 con 0x7f5358102d50 2024-03-27T10:53:45.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:45.386+0000 7f535d400640 1 -- 172.21.15.97:0/3640727730 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53581a2310 con 0x7f5358102d50 2024-03-27T10:53:45.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:45.387+0000 7f535d400640 1 -- 172.21.15.97:0/3640727730 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5358108e80 con 0x7f5358102d50 2024-03-27T10:53:45.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:45.389+0000 7f534f400640 1 -- 172.21.15.97:0/3640727730 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138370+0+0 (secure 0 0 0) 0x7f53440109a0 con 0x7f5358102d50 2024-03-27T10:53:45.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:45.391+0000 7f534f400640 1 --2- 172.21.15.97:0/3640727730 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f533c0a3690 0x7f533c0a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:53:45.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:45.391+0000 7f534f400640 1 -- 172.21.15.97:0/3640727730 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7127+0+0 (secure 0 0 0) 0x7f53440cf520 con 0x7f5358102d50 2024-03-27T10:53:45.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:45.391+0000 7f5356800640 1 --2- 172.21.15.97:0/3640727730 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f533c0a3690 0x7f533c0a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:53:45.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:45.392+0000 7f5356800640 1 --2- 172.21.15.97:0/3640727730 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f533c0a3690 0x7f533c0a5b50 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f53480096c0 tx=0x7f5348005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:53:45.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:45.396+0000 7f534f400640 1 -- 172.21.15.97:0/3640727730 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f5344009de0 con 0x7f5358102d50 2024-03-27T10:53:45.526 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:45 smithi177 ceph-mon[35294]: pgmap v348: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T10:53:45.527 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:45 smithi177 ceph-mon[35294]: mds.? [v2:172.21.15.114:6836/616775588,v1:172.21.15.114:6837/616775588] up:boot 2024-03-27T10:53:45.527 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:45 smithi177 ceph-mon[35294]: fsmap 8 up:standby 2024-03-27T10:53:45.527 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:45 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-27T10:53:45.527 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:45 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:45.527 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:45 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:45.527 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:45 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:45.527 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:45 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-27T10:53:45.527 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:45 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-27T10:53:45.527 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:45 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:53:45.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:45.633+0000 7f535d400640 1 -- 172.21.15.97:0/3640727730 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f53580627f0 con 0x7f5358102d50 2024-03-27T10:53:45.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:45.634+0000 7f534f400640 1 -- 172.21.15.97:0/3640727730 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7183 (secure 0 0 0) 0x7f534409c4f0 con 0x7f5358102d50 2024-03-27T10:53:45.606 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T10:53:45.606 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1009423904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1009423904},{"type":"v1","addr":"172.21.15.97:6835","nonce":1009423904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3251173433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3251173433},{"type":"v1","addr":"172.21.15.97:6837","nonce":3251173433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3787981632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3787981632},{"type":"v1","addr":"172.21.15.177:6833","nonce":3787981632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/3349112132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":3349112132},{"type":"v1","addr":"172.21.15.177:6835","nonce":3349112132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6837/3244432640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":3244432640},{"type":"v1","addr":"172.21.15.177:6837","nonce":3244432640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":34415,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6833/4052517902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":4052517902},{"type":"v1","addr":"172.21.15.114:6833","nonce":4052517902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34445,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/3581918222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3581918222},{"type":"v1","addr":"172.21.15.114:6835","nonce":3581918222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34478,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/616775588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":616775588},{"type":"v1","addr":"172.21.15.114:6837","nonce":616775588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-03-27T10:53:45.607 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 9 2024-03-27T10:53:45.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:45.637+0000 7f535d400640 1 -- 172.21.15.97:0/3640727730 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f533c0a3690 msgr2=0x7f533c0a5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:45.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:45.637+0000 7f535d400640 1 --2- 172.21.15.97:0/3640727730 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f533c0a3690 0x7f533c0a5b50 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f53480096c0 tx=0x7f5348005fd0 comp rx=0 tx=0).stop 2024-03-27T10:53:45.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:45.637+0000 7f535d400640 1 -- 172.21.15.97:0/3640727730 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5358102d50 msgr2=0x7f5358073f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:45.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:45.637+0000 7f535d400640 1 --2- 172.21.15.97:0/3640727730 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5358102d50 0x7f5358073f40 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f534400d950 tx=0x7f534400de10 comp rx=0 tx=0).stop 2024-03-27T10:53:45.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:45.637+0000 7f535d400640 1 -- 172.21.15.97:0/3640727730 shutdown_connections 2024-03-27T10:53:45.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:45.637+0000 7f535d400640 1 --2- 172.21.15.97:0/3640727730 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f533c0a3690 0x7f533c0a5b50 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:45.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:45.637+0000 7f535d400640 1 --2- 172.21.15.97:0/3640727730 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5358103710 0x7f5358074480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:45.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:45.637+0000 7f535d400640 1 --2- 172.21.15.97:0/3640727730 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5358102d50 0x7f5358073f40 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:45.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:45.637+0000 7f535d400640 1 --2- 172.21.15.97:0/3640727730 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5358101b60 0x7f5358075900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:45.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:45.637+0000 7f535d400640 1 -- 172.21.15.97:0/3640727730 >> 172.21.15.97:0/3640727730 conn(0x7f53580fd0f0 msgr2=0x7f5358106050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:53:45.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:45.637+0000 7f535d400640 1 -- 172.21.15.97:0/3640727730 shutdown_connections 2024-03-27T10:53:45.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:45.637+0000 7f535d400640 1 -- 172.21.15.97:0/3640727730 wait complete. 2024-03-27T10:53:45.626 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:45 smithi097 ceph-mon[31859]: pgmap v348: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T10:53:45.626 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:45 smithi097 ceph-mon[31859]: mds.? [v2:172.21.15.114:6836/616775588,v1:172.21.15.114:6837/616775588] up:boot 2024-03-27T10:53:45.626 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:45 smithi097 ceph-mon[31859]: fsmap 8 up:standby 2024-03-27T10:53:45.626 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:45 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-27T10:53:45.626 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:45 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:45.626 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:45 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:45.626 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:45 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:45.627 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:45 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-27T10:53:45.627 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:45 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-27T10:53:45.627 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:45 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:53:45.768 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:45 smithi114 ceph-mon[35338]: pgmap v348: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T10:53:45.768 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:45 smithi114 ceph-mon[35338]: mds.? [v2:172.21.15.114:6836/616775588,v1:172.21.15.114:6837/616775588] up:boot 2024-03-27T10:53:45.768 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:45 smithi114 ceph-mon[35338]: fsmap 8 up:standby 2024-03-27T10:53:45.768 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:45 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-27T10:53:45.768 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:45 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:45.768 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:45 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:45.768 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:45 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:45.768 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:45 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-27T10:53:45.768 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:45 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-27T10:53:45.768 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:45 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:53:46.485 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:46 smithi177 ceph-mon[35294]: Deploying daemon mds.i on smithi177 2024-03-27T10:53:46.485 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:46 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/3640727730' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T10:53:46.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:46 smithi114 ceph-mon[35338]: Deploying daemon mds.i on smithi177 2024-03-27T10:53:46.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:46 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/3640727730' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T10:53:46.641 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:46 smithi097 ceph-mon[31859]: Deploying daemon mds.i on smithi177 2024-03-27T10:53:46.641 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:46 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/3640727730' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T10:53:47.631 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-27T10:53:47.641 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:47 smithi097 ceph-mon[31859]: pgmap v349: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T10:53:47.664 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:47 smithi177 ceph-mon[35294]: pgmap v349: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T10:53:47.664 INFO:journalctl@ceph.mds.i.smithi177.stdout:Mar 27 10:53:47 smithi177 systemd[1]: Starting Ceph mds.i for adca88c4-ec26-11ee-95d0-87774f69a715... 2024-03-27T10:53:47.767 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:47 smithi114 ceph-mon[35338]: pgmap v349: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T10:53:47.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:47.877+0000 7fb3ed200640 1 -- 172.21.15.97:0/2127475458 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3e8105700 msgr2=0x7fb3e810bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:47.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:47.877+0000 7fb3ed200640 1 --2- 172.21.15.97:0/2127475458 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3e8105700 0x7fb3e810bf90 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb3e0009ca0 tx=0x7fb3e002c0f0 comp rx=0 tx=0).stop 2024-03-27T10:53:47.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:47.877+0000 7fb3ed200640 1 -- 172.21.15.97:0/2127475458 shutdown_connections 2024-03-27T10:53:47.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:47.877+0000 7fb3ed200640 1 --2- 172.21.15.97:0/2127475458 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3e8105700 0x7fb3e810bf90 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:47.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:47.877+0000 7fb3ed200640 1 --2- 172.21.15.97:0/2127475458 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb3e8104d40 0x7fb3e81051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:47.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:47.877+0000 7fb3ed200640 1 --2- 172.21.15.97:0/2127475458 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb3e8103b50 0x7fb3e8103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:47.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:47.877+0000 7fb3ed200640 1 -- 172.21.15.97:0/2127475458 >> 172.21.15.97:0/2127475458 conn(0x7fb3e80ff0c0 msgr2=0x7fb3e8101500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:53:47.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:47.877+0000 7fb3ed200640 1 -- 172.21.15.97:0/2127475458 shutdown_connections 2024-03-27T10:53:47.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:47.877+0000 7fb3ed200640 1 -- 172.21.15.97:0/2127475458 wait complete. 2024-03-27T10:53:47.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:47.878+0000 7fb3ed200640 1 Processor -- start 2024-03-27T10:53:47.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:47.878+0000 7fb3ed200640 1 -- start start 2024-03-27T10:53:47.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:47.879+0000 7fb3ed200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3e8103b50 0x7fb3e8199d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:53:47.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:47.879+0000 7fb3ed200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb3e8104d40 0x7fb3e819a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:53:47.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:47.879+0000 7fb3ed200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb3e8105700 0x7fb3e81a12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:53:47.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:47.879+0000 7fb3ed200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb3e810f680 con 0x7fb3e8103b50 2024-03-27T10:53:47.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:47.879+0000 7fb3ed200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb3e810f500 con 0x7fb3e8104d40 2024-03-27T10:53:47.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:47.879+0000 7fb3ed200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb3e810f800 con 0x7fb3e8105700 2024-03-27T10:53:47.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:47.879+0000 7fb3e7400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3e8103b50 0x7fb3e8199d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:53:47.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:47.879+0000 7fb3e7400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3e8103b50 0x7fb3e8199d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45490/0 (socket says 172.21.15.97:45490) 2024-03-27T10:53:47.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:47.879+0000 7fb3e7400640 1 -- 172.21.15.97:0/922002537 learned_addr learned my addr 172.21.15.97:0/922002537 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:53:47.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:47.879+0000 7fb3e7e00640 1 --2- 172.21.15.97:0/922002537 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb3e8105700 0x7fb3e81a12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:53:47.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:47.879+0000 7fb3e6a00640 1 --2- 172.21.15.97:0/922002537 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb3e8104d40 0x7fb3e819a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:53:47.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:47.879+0000 7fb3e7400640 1 -- 172.21.15.97:0/922002537 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb3e8105700 msgr2=0x7fb3e81a12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:47.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:47.879+0000 7fb3e7400640 1 --2- 172.21.15.97:0/922002537 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb3e8105700 0x7fb3e81a12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:47.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:47.879+0000 7fb3e7400640 1 -- 172.21.15.97:0/922002537 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb3e8104d40 msgr2=0x7fb3e819a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:47.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:47.879+0000 7fb3e7400640 1 --2- 172.21.15.97:0/922002537 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb3e8104d40 0x7fb3e819a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:47.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:47.879+0000 7fb3e7400640 1 -- 172.21.15.97:0/922002537 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3e81a1940 con 0x7fb3e8103b50 2024-03-27T10:53:47.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:47.879+0000 7fb3e6a00640 1 --2- 172.21.15.97:0/922002537 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb3e8104d40 0x7fb3e819a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:53:47.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:47.880+0000 7fb3e7400640 1 --2- 172.21.15.97:0/922002537 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3e8103b50 0x7fb3e8199d30 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb3d800d770 tx=0x7fb3d800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:53:47.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:47.880+0000 7fb3dfe00640 1 -- 172.21.15.97:0/922002537 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d8017070 con 0x7fb3e8103b50 2024-03-27T10:53:47.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:47.880+0000 7fb3dfe00640 1 -- 172.21.15.97:0/922002537 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb3d8009c30 con 0x7fb3e8103b50 2024-03-27T10:53:47.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:47.880+0000 7fb3dfe00640 1 -- 172.21.15.97:0/922002537 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d8003d00 con 0x7fb3e8103b50 2024-03-27T10:53:47.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:47.880+0000 7fb3ed200640 1 -- 172.21.15.97:0/922002537 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3e81a1c30 con 0x7fb3e8103b50 2024-03-27T10:53:47.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:47.880+0000 7fb3ed200640 1 -- 172.21.15.97:0/922002537 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3e81a2180 con 0x7fb3e8103b50 2024-03-27T10:53:47.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:47.881+0000 7fb3ed200640 1 -- 172.21.15.97:0/922002537 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3e810af10 con 0x7fb3e8103b50 2024-03-27T10:53:47.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:47.883+0000 7fb3dfe00640 1 -- 172.21.15.97:0/922002537 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138370+0+0 (secure 0 0 0) 0x7fb3d8003ea0 con 0x7fb3e8103b50 2024-03-27T10:53:47.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:47.885+0000 7fb3dfe00640 1 --2- 172.21.15.97:0/922002537 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fb3cc0a3610 0x7fb3cc0a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:53:47.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:47.885+0000 7fb3dfe00640 1 -- 172.21.15.97:0/922002537 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7127+0+0 (secure 0 0 0) 0x7fb3d80cf7b0 con 0x7fb3e8103b50 2024-03-27T10:53:47.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:47.885+0000 7fb3e6a00640 1 --2- 172.21.15.97:0/922002537 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fb3cc0a3610 0x7fb3cc0a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:53:47.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:47.886+0000 7fb3e6a00640 1 --2- 172.21.15.97:0/922002537 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fb3cc0a3610 0x7fb3cc0a5ad0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb3d4009b50 tx=0x7fb3d4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:53:47.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:47.891+0000 7fb3dfe00640 1 -- 172.21.15.97:0/922002537 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7fb3d809c780 con 0x7fb3e8103b50 2024-03-27T10:53:47.928 INFO:journalctl@ceph.mds.i.smithi177.stdout:Mar 27 10:53:47 smithi177 podman[66743]: 2024-03-27 10:53:47.595981952 +0000 UTC m=+0.033040430 image pull b379bbd8da6b5d0fa9a3523941efac213b6815e1cbf05c6507954f0a3fe7a801 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772 2024-03-27T10:53:47.928 INFO:journalctl@ceph.mds.i.smithi177.stdout:Mar 27 10:53:47 smithi177 podman[66743]: 2024-03-27 10:53:47.714531603 +0000 UTC m=+0.151590118 container create 2e7f049a1279bada5ae2740f8b74503a5da9c6e28459983ebd986c2543ecaf41 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-i, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e) 2024-03-27T10:53:48.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:48.131+0000 7fb3ed200640 1 -- 172.21.15.97:0/922002537 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb3e802d370 con 0x7fb3e8103b50 2024-03-27T10:53:48.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:48.133+0000 7fb3dfe00640 1 -- 172.21.15.97:0/922002537 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7183 (secure 0 0 0) 0x7fb3d8009de0 con 0x7fb3e8103b50 2024-03-27T10:53:48.105 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T10:53:48.105 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1009423904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1009423904},{"type":"v1","addr":"172.21.15.97:6835","nonce":1009423904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3251173433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3251173433},{"type":"v1","addr":"172.21.15.97:6837","nonce":3251173433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3787981632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3787981632},{"type":"v1","addr":"172.21.15.177:6833","nonce":3787981632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/3349112132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":3349112132},{"type":"v1","addr":"172.21.15.177:6835","nonce":3349112132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6837/3244432640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":3244432640},{"type":"v1","addr":"172.21.15.177:6837","nonce":3244432640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":34415,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6833/4052517902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":4052517902},{"type":"v1","addr":"172.21.15.114:6833","nonce":4052517902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34445,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/3581918222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3581918222},{"type":"v1","addr":"172.21.15.114:6835","nonce":3581918222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34478,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/616775588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":616775588},{"type":"v1","addr":"172.21.15.114:6837","nonce":616775588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-03-27T10:53:48.105 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 9 2024-03-27T10:53:48.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:48.135+0000 7fb3ed200640 1 -- 172.21.15.97:0/922002537 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fb3cc0a3610 msgr2=0x7fb3cc0a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:48.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:48.135+0000 7fb3ed200640 1 --2- 172.21.15.97:0/922002537 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fb3cc0a3610 0x7fb3cc0a5ad0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb3d4009b50 tx=0x7fb3d4005f90 comp rx=0 tx=0).stop 2024-03-27T10:53:48.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:48.135+0000 7fb3ed200640 1 -- 172.21.15.97:0/922002537 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3e8103b50 msgr2=0x7fb3e8199d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:48.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:48.135+0000 7fb3ed200640 1 --2- 172.21.15.97:0/922002537 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3e8103b50 0x7fb3e8199d30 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb3d800d770 tx=0x7fb3d800dc30 comp rx=0 tx=0).stop 2024-03-27T10:53:48.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:48.135+0000 7fb3ed200640 1 -- 172.21.15.97:0/922002537 shutdown_connections 2024-03-27T10:53:48.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:48.135+0000 7fb3ed200640 1 --2- 172.21.15.97:0/922002537 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb3e8105700 0x7fb3e81a12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:48.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:48.135+0000 7fb3ed200640 1 --2- 172.21.15.97:0/922002537 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fb3cc0a3610 0x7fb3cc0a5ad0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:48.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:48.135+0000 7fb3ed200640 1 --2- 172.21.15.97:0/922002537 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb3e8104d40 0x7fb3e819a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:48.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:48.135+0000 7fb3ed200640 1 --2- 172.21.15.97:0/922002537 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3e8103b50 0x7fb3e8199d30 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:48.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:48.135+0000 7fb3ed200640 1 -- 172.21.15.97:0/922002537 >> 172.21.15.97:0/922002537 conn(0x7fb3e80ff0c0 msgr2=0x7fb3e81080e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:53:48.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:48.136+0000 7fb3ed200640 1 -- 172.21.15.97:0/922002537 shutdown_connections 2024-03-27T10:53:48.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:48.136+0000 7fb3ed200640 1 -- 172.21.15.97:0/922002537 wait complete. 2024-03-27T10:53:48.300 INFO:journalctl@ceph.mds.i.smithi177.stdout:Mar 27 10:53:47 smithi177 podman[66743]: 2024-03-27 10:53:47.968575385 +0000 UTC m=+0.405633937 container init 2e7f049a1279bada5ae2740f8b74503a5da9c6e28459983ebd986c2543ecaf41 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-i, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, RELEASE=wip-rishabh-testing-20240326.131558-155268c, io.buildah.version=1.33.5, ceph=True) 2024-03-27T10:53:48.300 INFO:journalctl@ceph.mds.i.smithi177.stdout:Mar 27 10:53:47 smithi177 podman[66743]: 2024-03-27 10:53:47.978366253 +0000 UTC m=+0.415424794 container start 2e7f049a1279bada5ae2740f8b74503a5da9c6e28459983ebd986c2543ecaf41 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-i, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240326.131558-155268c, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-03-27T10:53:48.300 INFO:journalctl@ceph.mds.i.smithi177.stdout:Mar 27 10:53:48 smithi177 ceph-mds[66761]: -- 172.21.15.177:0/3937081425 <== mon.1 v2:172.21.15.177:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561d8a1dda40 con 0x561d8a1b1000 2024-03-27T10:53:48.300 INFO:journalctl@ceph.mds.i.smithi177.stdout:Mar 27 10:53:48 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-i[66757]: starting mds.i at 2024-03-27T10:53:48.300 INFO:journalctl@ceph.mds.i.smithi177.stdout:Mar 27 10:53:48 smithi177 bash[66743]: 2e7f049a1279bada5ae2740f8b74503a5da9c6e28459983ebd986c2543ecaf41 2024-03-27T10:53:48.300 INFO:journalctl@ceph.mds.i.smithi177.stdout:Mar 27 10:53:48 smithi177 systemd[1]: Started Ceph mds.i for adca88c4-ec26-11ee-95d0-87774f69a715. 2024-03-27T10:53:48.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:48 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/922002537' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T10:53:48.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:48 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:48.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:48 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:48.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:48 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:48.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:48 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-27T10:53:48.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:48 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-27T10:53:48.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:48 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:53:48.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:48 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/922002537' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T10:53:48.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:48 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:48.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:48 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:48.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:48 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:48.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:48 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-27T10:53:48.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:48 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-27T10:53:48.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:48 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:53:48.768 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:48 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/922002537' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T10:53:48.768 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:48 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:48.768 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:48 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:48.768 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:48 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:48.768 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:48 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-27T10:53:48.768 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:48 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-27T10:53:48.768 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:48 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:53:49.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:49 smithi097 ceph-mon[31859]: pgmap v350: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T10:53:49.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:49 smithi097 ceph-mon[31859]: Deploying daemon mds.g on smithi097 2024-03-27T10:53:49.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:49 smithi097 ceph-mon[31859]: mds.? [v2:172.21.15.177:6838/3340454605,v1:172.21.15.177:6839/3340454605] up:boot 2024-03-27T10:53:49.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:49 smithi097 ceph-mon[31859]: fsmap 9 up:standby 2024-03-27T10:53:49.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:49 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-27T10:53:49.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:49 smithi177 ceph-mon[35294]: pgmap v350: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T10:53:49.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:49 smithi177 ceph-mon[35294]: Deploying daemon mds.g on smithi097 2024-03-27T10:53:49.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:49 smithi177 ceph-mon[35294]: mds.? [v2:172.21.15.177:6838/3340454605,v1:172.21.15.177:6839/3340454605] up:boot 2024-03-27T10:53:49.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:49 smithi177 ceph-mon[35294]: fsmap 9 up:standby 2024-03-27T10:53:49.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:49 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-27T10:53:49.768 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:49 smithi114 ceph-mon[35338]: pgmap v350: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T10:53:49.768 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:49 smithi114 ceph-mon[35338]: Deploying daemon mds.g on smithi097 2024-03-27T10:53:49.768 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:49 smithi114 ceph-mon[35338]: mds.? [v2:172.21.15.177:6838/3340454605,v1:172.21.15.177:6839/3340454605] up:boot 2024-03-27T10:53:49.768 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:49 smithi114 ceph-mon[35338]: fsmap 9 up:standby 2024-03-27T10:53:49.768 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:49 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-27T10:53:50.127 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-27T10:53:50.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:50.821+0000 7f6823200640 1 -- 172.21.15.97:0/560872214 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6824102ba0 msgr2=0x7f6824103020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:50.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:50.821+0000 7f6820a00640 1 -- 172.21.15.97:0/560872214 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f680c02cbd0 con 0x7f6824102ba0 2024-03-27T10:53:50.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:50.821+0000 7f6823200640 1 --2- 172.21.15.97:0/560872214 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6824102ba0 0x7f6824103020 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f680c009900 tx=0x7f680c02bc80 comp rx=0 tx=0).stop 2024-03-27T10:53:50.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:50.822+0000 7f6823200640 1 -- 172.21.15.97:0/560872214 shutdown_connections 2024-03-27T10:53:50.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:50.822+0000 7f6823200640 1 --2- 172.21.15.97:0/560872214 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6824103560 0x7f6824109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:50.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:50.822+0000 7f6823200640 1 --2- 172.21.15.97:0/560872214 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6824102ba0 0x7f6824103020 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:50.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:50.822+0000 7f6823200640 1 --2- 172.21.15.97:0/560872214 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f68241019b0 0x7f6824101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:50.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:50.822+0000 7f6823200640 1 -- 172.21.15.97:0/560872214 >> 172.21.15.97:0/560872214 conn(0x7f68240fcf20 msgr2=0x7f68240ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:53:50.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:50.822+0000 7f6823200640 1 -- 172.21.15.97:0/560872214 shutdown_connections 2024-03-27T10:53:50.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:50.822+0000 7f6823200640 1 -- 172.21.15.97:0/560872214 wait complete. 2024-03-27T10:53:50.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:50.824+0000 7f6823200640 1 Processor -- start 2024-03-27T10:53:50.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:50.824+0000 7f6823200640 1 -- start start 2024-03-27T10:53:50.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:50.824+0000 7f6823200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68241019b0 0x7f6824199cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:53:50.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:50.824+0000 7f6823200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6824102ba0 0x7f682419a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:53:50.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:50.824+0000 7f6823200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6824103560 0x7f68241a1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:53:50.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:50.824+0000 7f6823200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f682410d620 con 0x7f68241019b0 2024-03-27T10:53:50.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:50.824+0000 7f6823200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f682410d4a0 con 0x7f6824103560 2024-03-27T10:53:50.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:50.824+0000 7f6823200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f682410d7a0 con 0x7f6824102ba0 2024-03-27T10:53:50.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:50.825+0000 7f6821400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6824102ba0 0x7f682419a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:53:50.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:50.825+0000 7f6821e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68241019b0 0x7f6824199cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:53:50.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:50.825+0000 7f6821400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6824102ba0 0x7f682419a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.97:42934/0 (socket says 172.21.15.97:42934) 2024-03-27T10:53:50.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:50.825+0000 7f6821400640 1 -- 172.21.15.97:0/3083957668 learned_addr learned my addr 172.21.15.97:0/3083957668 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:53:50.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:50.825+0000 7f6821e00640 1 --2- 172.21.15.97:0/3083957668 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68241019b0 0x7f6824199cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43808/0 (socket says 172.21.15.97:43808) 2024-03-27T10:53:50.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:50.825+0000 7f6822800640 1 --2- 172.21.15.97:0/3083957668 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6824103560 0x7f68241a1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:53:50.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:50.826+0000 7f6821400640 1 -- 172.21.15.97:0/3083957668 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6824103560 msgr2=0x7f68241a1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:50.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:50.826+0000 7f6821400640 1 --2- 172.21.15.97:0/3083957668 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6824103560 0x7f68241a1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:50.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:50.826+0000 7f6821400640 1 -- 172.21.15.97:0/3083957668 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68241019b0 msgr2=0x7f6824199cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:50.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:50.826+0000 7f6821400640 1 --2- 172.21.15.97:0/3083957668 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68241019b0 0x7f6824199cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:50.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:50.826+0000 7f6821400640 1 -- 172.21.15.97:0/3083957668 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68241a1900 con 0x7f6824102ba0 2024-03-27T10:53:50.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:50.826+0000 7f6821e00640 1 --2- 172.21.15.97:0/3083957668 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68241019b0 0x7f6824199cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T10:53:50.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:50.826+0000 7f6821400640 1 --2- 172.21.15.97:0/3083957668 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6824102ba0 0x7f682419a200 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f680c005ec0 tx=0x7f680c02caf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:53:50.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:50.827+0000 7f681aa00640 1 -- 172.21.15.97:0/3083957668 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f680c043020 con 0x7f6824102ba0 2024-03-27T10:53:50.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:50.827+0000 7f6823200640 1 -- 172.21.15.97:0/3083957668 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68241a1b90 con 0x7f6824102ba0 2024-03-27T10:53:50.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:50.827+0000 7f6823200640 1 -- 172.21.15.97:0/3083957668 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68241a2130 con 0x7f6824102ba0 2024-03-27T10:53:50.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:50.828+0000 7f681aa00640 1 -- 172.21.15.97:0/3083957668 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f680c02cce0 con 0x7f6824102ba0 2024-03-27T10:53:50.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:50.828+0000 7f681aa00640 1 -- 172.21.15.97:0/3083957668 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f680c003d80 con 0x7f6824102ba0 2024-03-27T10:53:50.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:50.829+0000 7f6823200640 1 -- 172.21.15.97:0/3083957668 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6800005290 con 0x7f6824102ba0 2024-03-27T10:53:50.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:50.833+0000 7f681aa00640 1 -- 172.21.15.97:0/3083957668 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 16) ==== 138370+0+0 (secure 0 0 0) 0x7f680c046020 con 0x7f6824102ba0 2024-03-27T10:53:50.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:50.834+0000 7f681aa00640 1 --2- 172.21.15.97:0/3083957668 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f68080a36d0 0x7f68080a5b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:53:50.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:50.834+0000 7f681aa00640 1 -- 172.21.15.97:0/3083957668 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7127+0+0 (secure 0 0 0) 0x7f680c0f2030 con 0x7f6824102ba0 2024-03-27T10:53:50.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:50.834+0000 7f6821e00640 1 --2- 172.21.15.97:0/3083957668 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f68080a36d0 0x7f68080a5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:53:50.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:50.835+0000 7f6821e00640 1 --2- 172.21.15.97:0/3083957668 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f68080a36d0 0x7f68080a5b90 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f682419ab90 tx=0x7f68140090e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:53:50.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:50.846+0000 7f681aa00640 1 -- 172.21.15.97:0/3083957668 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f680c04d020 con 0x7f6824102ba0 2024-03-27T10:53:51.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:51.091+0000 7f6823200640 1 -- 172.21.15.97:0/3083957668 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6800005580 con 0x7f6824102ba0 2024-03-27T10:53:51.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:51.093+0000 7f681aa00640 1 -- 172.21.15.97:0/3083957668 <== mon.1 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8013 (secure 0 0 0) 0x7f680c0bf080 con 0x7f6824102ba0 2024-03-27T10:53:51.065 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T10:53:51.065 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1009423904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1009423904},{"type":"v1","addr":"172.21.15.97:6835","nonce":1009423904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3251173433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3251173433},{"type":"v1","addr":"172.21.15.97:6837","nonce":3251173433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3787981632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3787981632},{"type":"v1","addr":"172.21.15.177:6833","nonce":3787981632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/3349112132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":3349112132},{"type":"v1","addr":"172.21.15.177:6835","nonce":3349112132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6837/3244432640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":3244432640},{"type":"v1","addr":"172.21.15.177:6837","nonce":3244432640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24487,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6839/3340454605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":3340454605},{"type":"v1","addr":"172.21.15.177:6839","nonce":3340454605}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34415,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6833/4052517902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":4052517902},{"type":"v1","addr":"172.21.15.114:6833","nonce":4052517902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34445,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/3581918222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3581918222},{"type":"v1","addr":"172.21.15.114:6835","nonce":3581918222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34478,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/616775588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":616775588},{"type":"v1","addr":"172.21.15.114:6837","nonce":616775588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-03-27T10:53:51.065 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 10 2024-03-27T10:53:51.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:51.097+0000 7f6823200640 1 -- 172.21.15.97:0/3083957668 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f68080a36d0 msgr2=0x7f68080a5b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:51.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:51.097+0000 7f6823200640 1 --2- 172.21.15.97:0/3083957668 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f68080a36d0 0x7f68080a5b90 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f682419ab90 tx=0x7f68140090e0 comp rx=0 tx=0).stop 2024-03-27T10:53:51.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:51.097+0000 7f6823200640 1 -- 172.21.15.97:0/3083957668 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6824102ba0 msgr2=0x7f682419a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:51.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:51.097+0000 7f6823200640 1 --2- 172.21.15.97:0/3083957668 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6824102ba0 0x7f682419a200 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f680c005ec0 tx=0x7f680c02caf0 comp rx=0 tx=0).stop 2024-03-27T10:53:51.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:51.097+0000 7f6823200640 1 -- 172.21.15.97:0/3083957668 shutdown_connections 2024-03-27T10:53:51.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:51.097+0000 7f6823200640 1 --2- 172.21.15.97:0/3083957668 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f68080a36d0 0x7f68080a5b90 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:51.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:51.097+0000 7f6823200640 1 --2- 172.21.15.97:0/3083957668 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6824103560 0x7f68241a1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:51.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:51.097+0000 7f6823200640 1 --2- 172.21.15.97:0/3083957668 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6824102ba0 0x7f682419a200 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:51.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:51.097+0000 7f6823200640 1 --2- 172.21.15.97:0/3083957668 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68241019b0 0x7f6824199cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:51.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:51.097+0000 7f6823200640 1 -- 172.21.15.97:0/3083957668 >> 172.21.15.97:0/3083957668 conn(0x7f68240fcf20 msgr2=0x7f6824073d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:53:51.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:51.097+0000 7f6823200640 1 -- 172.21.15.97:0/3083957668 shutdown_connections 2024-03-27T10:53:51.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:51.098+0000 7f6823200640 1 -- 172.21.15.97:0/3083957668 wait complete. 2024-03-27T10:53:51.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:51 smithi097 ceph-mon[31859]: pgmap v351: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T10:53:51.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:51 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/3083957668' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T10:53:51.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:51 smithi177 ceph-mon[35294]: pgmap v351: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T10:53:51.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:51 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/3083957668' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T10:53:51.766 INFO:journalctl@ceph.mds.g.smithi097.stdout:Mar 27 10:53:51 smithi097 podman[73475]: 2024-03-27 10:53:51.498264369 +0000 UTC m=+0.142541749 container create d58bb8d449717d12e61a7c321da2487155a9c921960565fb8bc0078c03f97745 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-g, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, ceph=True, maintainer=Guillaume Abrioux ) 2024-03-27T10:53:51.766 INFO:journalctl@ceph.mds.g.smithi097.stdout:Mar 27 10:53:51 smithi097 podman[73475]: 2024-03-27 10:53:51.793862018 +0000 UTC m=+0.438139377 container init d58bb8d449717d12e61a7c321da2487155a9c921960565fb8bc0078c03f97745 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-g, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-03-27T10:53:51.767 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:51 smithi114 ceph-mon[35338]: pgmap v351: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T10:53:51.767 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:51 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/3083957668' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T10:53:52.141 INFO:journalctl@ceph.mds.g.smithi097.stdout:Mar 27 10:53:51 smithi097 podman[73475]: 2024-03-27 10:53:51.802086808 +0000 UTC m=+0.446364158 container start d58bb8d449717d12e61a7c321da2487155a9c921960565fb8bc0078c03f97745 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-g, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.license=GPLv2) 2024-03-27T10:53:52.141 INFO:journalctl@ceph.mds.g.smithi097.stdout:Mar 27 10:53:51 smithi097 ceph-mds[73502]: -- 172.21.15.97:0/3955189271 <== mon.0 v2:172.21.15.97:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d0ea385a40 con 0x55d0ea358800 2024-03-27T10:53:52.141 INFO:journalctl@ceph.mds.g.smithi097.stdout:Mar 27 10:53:51 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-g[73493]: starting mds.g at 2024-03-27T10:53:52.141 INFO:journalctl@ceph.mds.g.smithi097.stdout:Mar 27 10:53:51 smithi097 bash[73475]: d58bb8d449717d12e61a7c321da2487155a9c921960565fb8bc0078c03f97745 2024-03-27T10:53:52.141 INFO:journalctl@ceph.mds.g.smithi097.stdout:Mar 27 10:53:51 smithi097 systemd[1]: Started Ceph mds.g for adca88c4-ec26-11ee-95d0-87774f69a715. 2024-03-27T10:53:53.093 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-27T10:53:53.268 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:53 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:53.268 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:53 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:53.268 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:53 smithi114 ceph-mon[35338]: pgmap v352: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T10:53:53.268 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:53 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:53.268 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:53 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-27T10:53:53.268 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:53 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-27T10:53:53.268 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:53 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:53:53.269 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:53 smithi114 ceph-mon[35338]: Deploying daemon mds.a on smithi097 2024-03-27T10:53:53.269 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:53 smithi114 ceph-mon[35338]: mds.? [v2:172.21.15.97:6838/1057596821,v1:172.21.15.97:6839/1057596821] up:boot 2024-03-27T10:53:53.269 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:53 smithi114 ceph-mon[35338]: fsmap 10 up:standby 2024-03-27T10:53:53.269 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:53 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-27T10:53:53.287 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:53 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:53.287 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:53 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:53.287 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:53 smithi097 ceph-mon[31859]: pgmap v352: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T10:53:53.287 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:53 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:53.287 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:53 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-27T10:53:53.287 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:53 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-27T10:53:53.287 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:53 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:53:53.287 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:53 smithi097 ceph-mon[31859]: Deploying daemon mds.a on smithi097 2024-03-27T10:53:53.288 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:53 smithi097 ceph-mon[31859]: mds.? [v2:172.21.15.97:6838/1057596821,v1:172.21.15.97:6839/1057596821] up:boot 2024-03-27T10:53:53.288 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:53 smithi097 ceph-mon[31859]: fsmap 10 up:standby 2024-03-27T10:53:53.288 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:53 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-27T10:53:53.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:53.333+0000 7fe6e2000640 1 -- 172.21.15.97:0/1547522565 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe6dc1016e0 msgr2=0x7fe6dc101b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:53.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:53.333+0000 7fe6e2000640 1 --2- 172.21.15.97:0/1547522565 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe6dc1016e0 0x7fe6dc101b60 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe6d4009ca0 tx=0x7fe6d402c100 comp rx=0 tx=0).stop 2024-03-27T10:53:53.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:53.333+0000 7fe6e2000640 1 -- 172.21.15.97:0/1547522565 shutdown_connections 2024-03-27T10:53:53.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:53.333+0000 7fe6e2000640 1 --2- 172.21.15.97:0/1547522565 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe6dc1016e0 0x7fe6dc101b60 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:53.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:53.333+0000 7fe6e2000640 1 --2- 172.21.15.97:0/1547522565 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6dc100d40 0x7fe6dc1011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:53.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:53.333+0000 7fe6e2000640 1 --2- 172.21.15.97:0/1547522565 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe6dc1003e0 0x7fe6dc100800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:53.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:53.333+0000 7fe6e2000640 1 -- 172.21.15.97:0/1547522565 >> 172.21.15.97:0/1547522565 conn(0x7fe6dc0fbfe0 msgr2=0x7fe6dc0fe420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:53:53.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:53.333+0000 7fe6e2000640 1 -- 172.21.15.97:0/1547522565 shutdown_connections 2024-03-27T10:53:53.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:53.333+0000 7fe6e2000640 1 -- 172.21.15.97:0/1547522565 wait complete. 2024-03-27T10:53:53.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:53.335+0000 7fe6e2000640 1 Processor -- start 2024-03-27T10:53:53.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:53.335+0000 7fe6e2000640 1 -- start start 2024-03-27T10:53:53.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:53.335+0000 7fe6e2000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6dc1003e0 0x7fe6dc199f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:53:53.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:53.335+0000 7fe6e2000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe6dc100d40 0x7fe6dc19a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:53:53.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:53.335+0000 7fe6e2000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe6dc1016e0 0x7fe6dc1a14d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:53:53.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:53.335+0000 7fe6e2000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe6dc10f870 con 0x7fe6dc1003e0 2024-03-27T10:53:53.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:53.335+0000 7fe6e2000640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe6dc10f6f0 con 0x7fe6dc100d40 2024-03-27T10:53:53.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:53.335+0000 7fe6e2000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe6dc10f9f0 con 0x7fe6dc1016e0 2024-03-27T10:53:53.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:53.335+0000 7fe6db400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6dc1003e0 0x7fe6dc199f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:53:53.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:53.335+0000 7fe6db400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6dc1003e0 0x7fe6dc199f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43846/0 (socket says 172.21.15.97:43846) 2024-03-27T10:53:53.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:53.335+0000 7fe6db400640 1 -- 172.21.15.97:0/1851281217 learned_addr learned my addr 172.21.15.97:0/1851281217 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:53:53.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:53.336+0000 7fe6daa00640 1 --2- 172.21.15.97:0/1851281217 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe6dc100d40 0x7fe6dc19a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:53:53.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:53.336+0000 7fe6dbe00640 1 --2- 172.21.15.97:0/1851281217 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe6dc1016e0 0x7fe6dc1a14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:53:53.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:53.336+0000 7fe6db400640 1 -- 172.21.15.97:0/1851281217 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe6dc1016e0 msgr2=0x7fe6dc1a14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:53.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:53.336+0000 7fe6db400640 1 --2- 172.21.15.97:0/1851281217 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe6dc1016e0 0x7fe6dc1a14d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:53.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:53.336+0000 7fe6db400640 1 -- 172.21.15.97:0/1851281217 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe6dc100d40 msgr2=0x7fe6dc19a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:53.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:53.336+0000 7fe6db400640 1 --2- 172.21.15.97:0/1851281217 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe6dc100d40 0x7fe6dc19a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:53.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:53.336+0000 7fe6db400640 1 -- 172.21.15.97:0/1851281217 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6dc1a1b40 con 0x7fe6dc1003e0 2024-03-27T10:53:53.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:53.336+0000 7fe6daa00640 1 --2- 172.21.15.97:0/1851281217 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe6dc100d40 0x7fe6dc19a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:53:53.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:53.336+0000 7fe6db400640 1 --2- 172.21.15.97:0/1851281217 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6dc1003e0 0x7fe6dc199f20 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc00ea60 tx=0x7fe6cc00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:53:53.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:53.337+0000 7fe6d3e00640 1 -- 172.21.15.97:0/1851281217 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6cc00cde0 con 0x7fe6dc1003e0 2024-03-27T10:53:53.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:53.337+0000 7fe6d3e00640 1 -- 172.21.15.97:0/1851281217 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe6cc0035f0 con 0x7fe6dc1003e0 2024-03-27T10:53:53.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:53.337+0000 7fe6e2000640 1 -- 172.21.15.97:0/1851281217 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6dc1a1e30 con 0x7fe6dc1003e0 2024-03-27T10:53:53.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:53.337+0000 7fe6d3e00640 1 -- 172.21.15.97:0/1851281217 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6cc010690 con 0x7fe6dc1003e0 2024-03-27T10:53:53.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:53.337+0000 7fe6e2000640 1 -- 172.21.15.97:0/1851281217 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6dc1a2290 con 0x7fe6dc1003e0 2024-03-27T10:53:53.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:53.338+0000 7fe6e2000640 1 -- 172.21.15.97:0/1851281217 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6dc10b2b0 con 0x7fe6dc1003e0 2024-03-27T10:53:53.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:53.340+0000 7fe6d3e00640 1 -- 172.21.15.97:0/1851281217 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138370+0+0 (secure 0 0 0) 0x7fe6cc0037a0 con 0x7fe6dc1003e0 2024-03-27T10:53:53.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:53.342+0000 7fe6d3e00640 1 --2- 172.21.15.97:0/1851281217 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fe6c00a3610 0x7fe6c00a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:53:53.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:53.342+0000 7fe6d3e00640 1 -- 172.21.15.97:0/1851281217 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7127+0+0 (secure 0 0 0) 0x7fe6cc0cfd00 con 0x7fe6dc1003e0 2024-03-27T10:53:53.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:53.342+0000 7fe6daa00640 1 --2- 172.21.15.97:0/1851281217 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fe6c00a3610 0x7fe6c00a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:53:53.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:53.343+0000 7fe6daa00640 1 --2- 172.21.15.97:0/1851281217 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fe6c00a3610 0x7fe6c00a5ad0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe6c8005d00 tx=0x7fe6c8009550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:53:53.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:53.347+0000 7fe6d3e00640 1 -- 172.21.15.97:0/1851281217 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7fe6cc09ccd0 con 0x7fe6dc1003e0 2024-03-27T10:53:53.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:53 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:53.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:53 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:53.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:53 smithi177 ceph-mon[35294]: pgmap v352: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T10:53:53.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:53 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:53.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:53 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-27T10:53:53.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:53 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-27T10:53:53.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:53 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:53:53.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:53 smithi177 ceph-mon[35294]: Deploying daemon mds.a on smithi097 2024-03-27T10:53:53.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:53 smithi177 ceph-mon[35294]: mds.? [v2:172.21.15.97:6838/1057596821,v1:172.21.15.97:6839/1057596821] up:boot 2024-03-27T10:53:53.429 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:53 smithi177 ceph-mon[35294]: fsmap 10 up:standby 2024-03-27T10:53:53.429 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:53 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-27T10:53:53.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:53.591+0000 7fe6e2000640 1 -- 172.21.15.97:0/1851281217 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe6dc02d370 con 0x7fe6dc1003e0 2024-03-27T10:53:53.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:53.594+0000 7fe6d3e00640 1 -- 172.21.15.97:0/1851281217 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8839 (secure 0 0 0) 0x7fe6cc09c5d0 con 0x7fe6dc1003e0 2024-03-27T10:53:53.566 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T10:53:53.566 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1009423904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1009423904},{"type":"v1","addr":"172.21.15.97:6835","nonce":1009423904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3251173433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3251173433},{"type":"v1","addr":"172.21.15.97:6837","nonce":3251173433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14646,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6839/1057596821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1057596821},{"type":"v1","addr":"172.21.15.97:6839","nonce":1057596821}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3787981632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3787981632},{"type":"v1","addr":"172.21.15.177:6833","nonce":3787981632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/3349112132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":3349112132},{"type":"v1","addr":"172.21.15.177:6835","nonce":3349112132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6837/3244432640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":3244432640},{"type":"v1","addr":"172.21.15.177:6837","nonce":3244432640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24487,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6839/3340454605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":3340454605},{"type":"v1","addr":"172.21.15.177:6839","nonce":3340454605}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34415,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6833/4052517902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":4052517902},{"type":"v1","addr":"172.21.15.114:6833","nonce":4052517902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34445,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/3581918222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3581918222},{"type":"v1","addr":"172.21.15.114:6835","nonce":3581918222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34478,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/616775588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":616775588},{"type":"v1","addr":"172.21.15.114:6837","nonce":616775588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-03-27T10:53:53.566 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 11 2024-03-27T10:53:53.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:53.596+0000 7fe6e2000640 1 -- 172.21.15.97:0/1851281217 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fe6c00a3610 msgr2=0x7fe6c00a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:53.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:53.596+0000 7fe6e2000640 1 --2- 172.21.15.97:0/1851281217 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fe6c00a3610 0x7fe6c00a5ad0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe6c8005d00 tx=0x7fe6c8009550 comp rx=0 tx=0).stop 2024-03-27T10:53:53.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:53.596+0000 7fe6e2000640 1 -- 172.21.15.97:0/1851281217 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6dc1003e0 msgr2=0x7fe6dc199f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:53.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:53.596+0000 7fe6e2000640 1 --2- 172.21.15.97:0/1851281217 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6dc1003e0 0x7fe6dc199f20 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc00ea60 tx=0x7fe6cc00ef20 comp rx=0 tx=0).stop 2024-03-27T10:53:53.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:53.596+0000 7fe6e2000640 1 -- 172.21.15.97:0/1851281217 shutdown_connections 2024-03-27T10:53:53.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:53.596+0000 7fe6e2000640 1 --2- 172.21.15.97:0/1851281217 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe6dc1016e0 0x7fe6dc1a14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:53.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:53.596+0000 7fe6e2000640 1 --2- 172.21.15.97:0/1851281217 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fe6c00a3610 0x7fe6c00a5ad0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:53.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:53.596+0000 7fe6e2000640 1 --2- 172.21.15.97:0/1851281217 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe6dc100d40 0x7fe6dc19a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:53.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:53.596+0000 7fe6e2000640 1 --2- 172.21.15.97:0/1851281217 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6dc1003e0 0x7fe6dc199f20 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:53.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:53.596+0000 7fe6e2000640 1 -- 172.21.15.97:0/1851281217 >> 172.21.15.97:0/1851281217 conn(0x7fe6dc0fbfe0 msgr2=0x7fe6dc0fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:53:53.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:53.597+0000 7fe6e2000640 1 -- 172.21.15.97:0/1851281217 shutdown_connections 2024-03-27T10:53:53.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:53.597+0000 7fe6e2000640 1 -- 172.21.15.97:0/1851281217 wait complete. 2024-03-27T10:53:54.098 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:54 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/1851281217' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T10:53:54.267 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:54 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/1851281217' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T10:53:54.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:54 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/1851281217' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T10:53:55.141 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:55 smithi097 ceph-mon[31859]: pgmap v353: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T10:53:55.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:55 smithi177 ceph-mon[35294]: pgmap v353: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T10:53:55.492 INFO:journalctl@ceph.mds.a.smithi097.stdout:Mar 27 10:53:55 smithi097 podman[73830]: 2024-03-27 10:53:55.294869721 +0000 UTC m=+0.180310763 container create c886b2875232f40ad088dbdd0f95feefc2590f10073fbcd9ae6dd9afb670f3f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True) 2024-03-27T10:53:55.492 INFO:journalctl@ceph.mds.a.smithi097.stdout:Mar 27 10:53:55 smithi097 podman[73830]: 2024-03-27 10:53:55.517075973 +0000 UTC m=+0.402517055 container init c886b2875232f40ad088dbdd0f95feefc2590f10073fbcd9ae6dd9afb670f3f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-a, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-27T10:53:55.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:55 smithi114 ceph-mon[35338]: pgmap v353: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T10:53:55.587 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-27T10:53:55.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:55.819+0000 7efe26c00640 1 -- 172.21.15.97:0/3895088647 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe200780f0 msgr2=0x7efe20078990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:55.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:55.819+0000 7efe26c00640 1 --2- 172.21.15.97:0/3895088647 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe200780f0 0x7efe20078990 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7efe18009ca0 tx=0x7efe1802c0f0 comp rx=0 tx=0).stop 2024-03-27T10:53:55.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:55.819+0000 7efe26c00640 1 -- 172.21.15.97:0/3895088647 shutdown_connections 2024-03-27T10:53:55.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:55.819+0000 7efe26c00640 1 --2- 172.21.15.97:0/3895088647 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe200780f0 0x7efe20078990 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:55.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:55.819+0000 7efe26c00640 1 --2- 172.21.15.97:0/3895088647 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efe20077750 0x7efe20077bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:55.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:55.819+0000 7efe26c00640 1 --2- 172.21.15.97:0/3895088647 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efe20076510 0x7efe20076930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:55.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:55.819+0000 7efe26c00640 1 -- 172.21.15.97:0/3895088647 >> 172.21.15.97:0/3895088647 conn(0x7efe200ff0c0 msgr2=0x7efe20101500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:53:55.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:55.819+0000 7efe26c00640 1 -- 172.21.15.97:0/3895088647 shutdown_connections 2024-03-27T10:53:55.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:55.819+0000 7efe26c00640 1 -- 172.21.15.97:0/3895088647 wait complete. 2024-03-27T10:53:55.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:55.820+0000 7efe26c00640 1 Processor -- start 2024-03-27T10:53:55.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:55.820+0000 7efe26c00640 1 -- start start 2024-03-27T10:53:55.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:55.821+0000 7efe26c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efe20076510 0x7efe2019e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:53:55.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:55.821+0000 7efe26c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe20077750 0x7efe2019e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:53:55.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:55.821+0000 7efe26c00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efe200780f0 0x7efe201a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:53:55.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:55.821+0000 7efe26c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7efe20113a90 con 0x7efe20077750 2024-03-27T10:53:55.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:55.821+0000 7efe26c00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7efe20113910 con 0x7efe20076510 2024-03-27T10:53:55.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:55.821+0000 7efe26c00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7efe20113c10 con 0x7efe200780f0 2024-03-27T10:53:55.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:55.821+0000 7efe24e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe20077750 0x7efe2019e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:53:55.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:55.821+0000 7efe24e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe20077750 0x7efe2019e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43878/0 (socket says 172.21.15.97:43878) 2024-03-27T10:53:55.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:55.821+0000 7efe24e00640 1 -- 172.21.15.97:0/2960055335 learned_addr learned my addr 172.21.15.97:0/2960055335 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:53:55.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:55.821+0000 7efe25800640 1 --2- 172.21.15.97:0/2960055335 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efe20076510 0x7efe2019e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:53:55.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:55.821+0000 7efe26200640 1 --2- 172.21.15.97:0/2960055335 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efe200780f0 0x7efe201a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:53:55.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:55.821+0000 7efe24e00640 1 -- 172.21.15.97:0/2960055335 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efe200780f0 msgr2=0x7efe201a56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:55.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:55.821+0000 7efe24e00640 1 --2- 172.21.15.97:0/2960055335 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efe200780f0 0x7efe201a56a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:55.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:55.821+0000 7efe24e00640 1 -- 172.21.15.97:0/2960055335 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efe20076510 msgr2=0x7efe2019e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:55.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:55.821+0000 7efe24e00640 1 --2- 172.21.15.97:0/2960055335 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efe20076510 0x7efe2019e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:55.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:55.821+0000 7efe24e00640 1 -- 172.21.15.97:0/2960055335 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe201a5d10 con 0x7efe20077750 2024-03-27T10:53:55.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:55.821+0000 7efe25800640 1 --2- 172.21.15.97:0/2960055335 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efe20076510 0x7efe2019e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T10:53:55.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:55.822+0000 7efe24e00640 1 --2- 172.21.15.97:0/2960055335 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe20077750 0x7efe2019e630 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7efe0c00d770 tx=0x7efe0c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:53:55.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:55.822+0000 7efe1e000640 1 -- 172.21.15.97:0/2960055335 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe0c017070 con 0x7efe20077750 2024-03-27T10:53:55.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:55.822+0000 7efe1e000640 1 -- 172.21.15.97:0/2960055335 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efe0c0054d0 con 0x7efe20077750 2024-03-27T10:53:55.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:55.822+0000 7efe26c00640 1 -- 172.21.15.97:0/2960055335 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe201a5fa0 con 0x7efe20077750 2024-03-27T10:53:55.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:55.822+0000 7efe1e000640 1 -- 172.21.15.97:0/2960055335 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe0c004cf0 con 0x7efe20077750 2024-03-27T10:53:55.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:55.822+0000 7efe26c00640 1 -- 172.21.15.97:0/2960055335 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe201a6470 con 0x7efe20077750 2024-03-27T10:53:55.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:55.823+0000 7efe26c00640 1 -- 172.21.15.97:0/2960055335 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe201104c0 con 0x7efe20077750 2024-03-27T10:53:55.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:55.826+0000 7efe1e000640 1 -- 172.21.15.97:0/2960055335 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138370+0+0 (secure 0 0 0) 0x7efe0c004e90 con 0x7efe20077750 2024-03-27T10:53:55.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:55.827+0000 7efe1e000640 1 --2- 172.21.15.97:0/2960055335 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7efe080a3610 0x7efe080a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:53:55.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:55.827+0000 7efe1e000640 1 -- 172.21.15.97:0/2960055335 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7127+0+0 (secure 0 0 0) 0x7efe0c0cf410 con 0x7efe20077750 2024-03-27T10:53:55.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:55.827+0000 7efe25800640 1 --2- 172.21.15.97:0/2960055335 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7efe080a3610 0x7efe080a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:53:55.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:55.828+0000 7efe25800640 1 --2- 172.21.15.97:0/2960055335 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7efe080a3610 0x7efe080a5ad0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7efe14009c20 tx=0x7efe14005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:53:55.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:55.832+0000 7efe1e000640 1 -- 172.21.15.97:0/2960055335 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7efe0c09c3e0 con 0x7efe20077750 2024-03-27T10:53:55.892 INFO:journalctl@ceph.mds.a.smithi097.stdout:Mar 27 10:53:55 smithi097 podman[73830]: 2024-03-27 10:53:55.529724416 +0000 UTC m=+0.415165491 container start c886b2875232f40ad088dbdd0f95feefc2590f10073fbcd9ae6dd9afb670f3f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-a, RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, ceph=True) 2024-03-27T10:53:55.892 INFO:journalctl@ceph.mds.a.smithi097.stdout:Mar 27 10:53:55 smithi097 ceph-mds[73849]: -- 172.21.15.97:0/822350559 <== mon.0 v2:172.21.15.97:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c947547a40 con 0x55c94751ac00 2024-03-27T10:53:55.892 INFO:journalctl@ceph.mds.a.smithi097.stdout:Mar 27 10:53:55 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-a[73845]: starting mds.a at 2024-03-27T10:53:55.892 INFO:journalctl@ceph.mds.a.smithi097.stdout:Mar 27 10:53:55 smithi097 bash[73830]: c886b2875232f40ad088dbdd0f95feefc2590f10073fbcd9ae6dd9afb670f3f5 2024-03-27T10:53:55.892 INFO:journalctl@ceph.mds.a.smithi097.stdout:Mar 27 10:53:55 smithi097 systemd[1]: Started Ceph mds.a for adca88c4-ec26-11ee-95d0-87774f69a715. 2024-03-27T10:53:56.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:56.076+0000 7efe26c00640 1 -- 172.21.15.97:0/2960055335 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efe2002d5b0 con 0x7efe20077750 2024-03-27T10:53:56.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:56.077+0000 7efe1e000640 1 -- 172.21.15.97:0/2960055335 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9665 (secure 0 0 0) 0x7efe0c09bce0 con 0x7efe20077750 2024-03-27T10:53:56.050 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T10:53:56.050 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1009423904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1009423904},{"type":"v1","addr":"172.21.15.97:6835","nonce":1009423904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3251173433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3251173433},{"type":"v1","addr":"172.21.15.97:6837","nonce":3251173433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14646,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6839/1057596821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1057596821},{"type":"v1","addr":"172.21.15.97:6839","nonce":1057596821}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14658,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6841/1672464856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6840","nonce":1672464856},{"type":"v1","addr":"172.21.15.97:6841","nonce":1672464856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3787981632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3787981632},{"type":"v1","addr":"172.21.15.177:6833","nonce":3787981632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/3349112132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":3349112132},{"type":"v1","addr":"172.21.15.177:6835","nonce":3349112132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6837/3244432640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":3244432640},{"type":"v1","addr":"172.21.15.177:6837","nonce":3244432640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24487,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6839/3340454605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":3340454605},{"type":"v1","addr":"172.21.15.177:6839","nonce":3340454605}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34415,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6833/4052517902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":4052517902},{"type":"v1","addr":"172.21.15.114:6833","nonce":4052517902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34445,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/3581918222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3581918222},{"type":"v1","addr":"172.21.15.114:6835","nonce":3581918222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34478,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/616775588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":616775588},{"type":"v1","addr":"172.21.15.114:6837","nonce":616775588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-03-27T10:53:56.050 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 12 2024-03-27T10:53:56.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:56.079+0000 7efe26c00640 1 -- 172.21.15.97:0/2960055335 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7efe080a3610 msgr2=0x7efe080a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:56.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:56.079+0000 7efe26c00640 1 --2- 172.21.15.97:0/2960055335 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7efe080a3610 0x7efe080a5ad0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7efe14009c20 tx=0x7efe14005f90 comp rx=0 tx=0).stop 2024-03-27T10:53:56.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:56.079+0000 7efe26c00640 1 -- 172.21.15.97:0/2960055335 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe20077750 msgr2=0x7efe2019e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:56.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:56.079+0000 7efe26c00640 1 --2- 172.21.15.97:0/2960055335 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe20077750 0x7efe2019e630 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7efe0c00d770 tx=0x7efe0c00dc30 comp rx=0 tx=0).stop 2024-03-27T10:53:56.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:56.079+0000 7efe26c00640 1 -- 172.21.15.97:0/2960055335 shutdown_connections 2024-03-27T10:53:56.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:56.079+0000 7efe26c00640 1 --2- 172.21.15.97:0/2960055335 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efe200780f0 0x7efe201a56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:56.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:56.079+0000 7efe26c00640 1 --2- 172.21.15.97:0/2960055335 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe20077750 0x7efe2019e630 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:56.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:56.079+0000 7efe26c00640 1 --2- 172.21.15.97:0/2960055335 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7efe080a3610 0x7efe080a5ad0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:56.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:56.079+0000 7efe26c00640 1 --2- 172.21.15.97:0/2960055335 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efe20076510 0x7efe2019e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:56.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:56.079+0000 7efe26c00640 1 -- 172.21.15.97:0/2960055335 >> 172.21.15.97:0/2960055335 conn(0x7efe200ff0c0 msgr2=0x7efe20100bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:53:56.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:56.080+0000 7efe26c00640 1 -- 172.21.15.97:0/2960055335 shutdown_connections 2024-03-27T10:53:56.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:56.080+0000 7efe26c00640 1 -- 172.21.15.97:0/2960055335 wait complete. 2024-03-27T10:53:56.963 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:56 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:56.963 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:56 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:56.963 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:56 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:56.963 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:56 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-27T10:53:56.963 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:56 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-27T10:53:56.963 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:56 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:53:56.963 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:56 smithi114 ceph-mon[35338]: Deploying daemon mds.b on smithi114 2024-03-27T10:53:56.963 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:56 smithi114 ceph-mon[35338]: pgmap v354: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T10:53:56.963 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:56 smithi114 ceph-mon[35338]: mds.? [v2:172.21.15.97:6840/1672464856,v1:172.21.15.97:6841/1672464856] up:boot 2024-03-27T10:53:56.963 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:56 smithi114 ceph-mon[35338]: fsmap 11 up:standby 2024-03-27T10:53:56.963 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:56 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-27T10:53:56.964 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:56 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/2960055335' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T10:53:57.141 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:56 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:57.142 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:56 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:57.142 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:56 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:57.142 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:56 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-27T10:53:57.142 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:56 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-27T10:53:57.142 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:56 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:53:57.142 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:56 smithi097 ceph-mon[31859]: Deploying daemon mds.b on smithi114 2024-03-27T10:53:57.142 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:56 smithi097 ceph-mon[31859]: pgmap v354: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T10:53:57.142 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:56 smithi097 ceph-mon[31859]: mds.? [v2:172.21.15.97:6840/1672464856,v1:172.21.15.97:6841/1672464856] up:boot 2024-03-27T10:53:57.142 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:56 smithi097 ceph-mon[31859]: fsmap 11 up:standby 2024-03-27T10:53:57.142 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:56 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-27T10:53:57.142 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:56 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/2960055335' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T10:53:57.177 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:56 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:57.178 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:56 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:57.178 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:56 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:53:57.178 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:56 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-27T10:53:57.178 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:56 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-27T10:53:57.178 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:56 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:53:57.178 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:56 smithi177 ceph-mon[35294]: Deploying daemon mds.b on smithi114 2024-03-27T10:53:57.178 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:56 smithi177 ceph-mon[35294]: pgmap v354: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T10:53:57.178 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:56 smithi177 ceph-mon[35294]: mds.? [v2:172.21.15.97:6840/1672464856,v1:172.21.15.97:6841/1672464856] up:boot 2024-03-27T10:53:57.178 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:56 smithi177 ceph-mon[35294]: fsmap 11 up:standby 2024-03-27T10:53:57.178 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:56 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-27T10:53:57.178 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:56 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/2960055335' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T10:53:58.071 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-27T10:53:58.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:58.320+0000 7fb459e00640 1 -- 172.21.15.97:0/2923067939 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb454076510 msgr2=0x7fb454076930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:58.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:58.320+0000 7fb459e00640 1 --2- 172.21.15.97:0/2923067939 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb454076510 0x7fb454076930 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb444009900 tx=0x7fb44402bc30 comp rx=0 tx=0).stop 2024-03-27T10:53:58.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:58.320+0000 7fb459e00640 1 -- 172.21.15.97:0/2923067939 shutdown_connections 2024-03-27T10:53:58.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:58.320+0000 7fb459e00640 1 --2- 172.21.15.97:0/2923067939 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb4540780f0 0x7fb454078990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:58.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:58.320+0000 7fb459e00640 1 --2- 172.21.15.97:0/2923067939 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb454077750 0x7fb454077bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:58.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:58.320+0000 7fb459e00640 1 --2- 172.21.15.97:0/2923067939 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb454076510 0x7fb454076930 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:58.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:58.320+0000 7fb459e00640 1 -- 172.21.15.97:0/2923067939 >> 172.21.15.97:0/2923067939 conn(0x7fb4540ff300 msgr2=0x7fb454101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:53:58.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:58.320+0000 7fb459e00640 1 -- 172.21.15.97:0/2923067939 shutdown_connections 2024-03-27T10:53:58.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:58.320+0000 7fb459e00640 1 -- 172.21.15.97:0/2923067939 wait complete. 2024-03-27T10:53:58.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:58.321+0000 7fb459e00640 1 Processor -- start 2024-03-27T10:53:58.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:58.321+0000 7fb459e00640 1 -- start start 2024-03-27T10:53:58.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:58.321+0000 7fb459e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb454076510 0x7fb45419e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:53:58.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:58.322+0000 7fb459e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb454077750 0x7fb45419e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:53:58.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:58.322+0000 7fb459e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4540780f0 0x7fb4541a58c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:53:58.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:58.322+0000 7fb459e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb454113d10 con 0x7fb4540780f0 2024-03-27T10:53:58.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:58.322+0000 7fb459e00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb454113b90 con 0x7fb454077750 2024-03-27T10:53:58.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:58.322+0000 7fb459e00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb454113e90 con 0x7fb454076510 2024-03-27T10:53:58.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:58.322+0000 7fb453400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4540780f0 0x7fb4541a58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:53:58.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:58.322+0000 7fb453400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4540780f0 0x7fb4541a58c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43888/0 (socket says 172.21.15.97:43888) 2024-03-27T10:53:58.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:58.322+0000 7fb453400640 1 -- 172.21.15.97:0/3295116502 learned_addr learned my addr 172.21.15.97:0/3295116502 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:53:58.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:58.322+0000 7fb452a00640 1 --2- 172.21.15.97:0/3295116502 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb454076510 0x7fb45419e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:53:58.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:58.322+0000 7fb452000640 1 --2- 172.21.15.97:0/3295116502 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb454077750 0x7fb45419e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:53:58.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:58.322+0000 7fb453400640 1 -- 172.21.15.97:0/3295116502 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb454076510 msgr2=0x7fb45419e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:58.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:58.322+0000 7fb453400640 1 --2- 172.21.15.97:0/3295116502 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb454076510 0x7fb45419e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:58.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:58.322+0000 7fb453400640 1 -- 172.21.15.97:0/3295116502 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb454077750 msgr2=0x7fb45419e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:58.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:58.322+0000 7fb453400640 1 --2- 172.21.15.97:0/3295116502 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb454077750 0x7fb45419e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:58.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:58.322+0000 7fb453400640 1 -- 172.21.15.97:0/3295116502 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4541a5f30 con 0x7fb4540780f0 2024-03-27T10:53:58.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:58.322+0000 7fb452a00640 1 --2- 172.21.15.97:0/3295116502 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb454076510 0x7fb45419e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:53:58.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:58.322+0000 7fb452000640 1 --2- 172.21.15.97:0/3295116502 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb454077750 0x7fb45419e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:53:58.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:58.323+0000 7fb453400640 1 --2- 172.21.15.97:0/3295116502 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4540780f0 0x7fb4541a58c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb44c00eaa0 tx=0x7fb44c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:53:58.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:58.323+0000 7fb458a00640 1 -- 172.21.15.97:0/3295116502 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb44c00cdb0 con 0x7fb4540780f0 2024-03-27T10:53:58.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:58.323+0000 7fb458a00640 1 -- 172.21.15.97:0/3295116502 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb44c005020 con 0x7fb4540780f0 2024-03-27T10:53:58.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:58.323+0000 7fb459e00640 1 -- 172.21.15.97:0/3295116502 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4541a6220 con 0x7fb4540780f0 2024-03-27T10:53:58.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:58.323+0000 7fb458a00640 1 -- 172.21.15.97:0/3295116502 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb44c0116d0 con 0x7fb4540780f0 2024-03-27T10:53:58.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:58.323+0000 7fb459e00640 1 -- 172.21.15.97:0/3295116502 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4541a67c0 con 0x7fb4540780f0 2024-03-27T10:53:58.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:58.324+0000 7fb459e00640 1 -- 172.21.15.97:0/3295116502 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb454110850 con 0x7fb4540780f0 2024-03-27T10:53:58.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:58.326+0000 7fb458a00640 1 -- 172.21.15.97:0/3295116502 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138370+0+0 (secure 0 0 0) 0x7fb44c0051d0 con 0x7fb4540780f0 2024-03-27T10:53:58.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:58.328+0000 7fb458a00640 1 --2- 172.21.15.97:0/3295116502 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fb4380a3690 0x7fb4380a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:53:58.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:58.328+0000 7fb458a00640 1 -- 172.21.15.97:0/3295116502 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7127+0+0 (secure 0 0 0) 0x7fb44c0cfa70 con 0x7fb4540780f0 2024-03-27T10:53:58.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:58.328+0000 7fb452a00640 1 --2- 172.21.15.97:0/3295116502 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fb4380a3690 0x7fb4380a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:53:58.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:58.329+0000 7fb452a00640 1 --2- 172.21.15.97:0/3295116502 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fb4380a3690 0x7fb4380a5b50 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb444036040 tx=0x7fb444005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:53:58.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:58.333+0000 7fb458a00640 1 -- 172.21.15.97:0/3295116502 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7fb44c016030 con 0x7fb4540780f0 2024-03-27T10:53:58.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:58.576+0000 7fb459e00640 1 -- 172.21.15.97:0/3295116502 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb454062750 con 0x7fb4540780f0 2024-03-27T10:53:58.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:58.577+0000 7fb458a00640 1 -- 172.21.15.97:0/3295116502 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9665 (secure 0 0 0) 0x7fb44c09cae0 con 0x7fb4540780f0 2024-03-27T10:53:58.551 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T10:53:58.551 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1009423904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1009423904},{"type":"v1","addr":"172.21.15.97:6835","nonce":1009423904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3251173433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3251173433},{"type":"v1","addr":"172.21.15.97:6837","nonce":3251173433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14646,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6839/1057596821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1057596821},{"type":"v1","addr":"172.21.15.97:6839","nonce":1057596821}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14658,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6841/1672464856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6840","nonce":1672464856},{"type":"v1","addr":"172.21.15.97:6841","nonce":1672464856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3787981632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3787981632},{"type":"v1","addr":"172.21.15.177:6833","nonce":3787981632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/3349112132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":3349112132},{"type":"v1","addr":"172.21.15.177:6835","nonce":3349112132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6837/3244432640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":3244432640},{"type":"v1","addr":"172.21.15.177:6837","nonce":3244432640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24487,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6839/3340454605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":3340454605},{"type":"v1","addr":"172.21.15.177:6839","nonce":3340454605}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34415,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6833/4052517902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":4052517902},{"type":"v1","addr":"172.21.15.114:6833","nonce":4052517902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34445,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/3581918222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3581918222},{"type":"v1","addr":"172.21.15.114:6835","nonce":3581918222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34478,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/616775588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":616775588},{"type":"v1","addr":"172.21.15.114:6837","nonce":616775588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-03-27T10:53:58.551 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 12 2024-03-27T10:53:58.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:58.580+0000 7fb459e00640 1 -- 172.21.15.97:0/3295116502 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fb4380a3690 msgr2=0x7fb4380a5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:58.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:58.580+0000 7fb459e00640 1 --2- 172.21.15.97:0/3295116502 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fb4380a3690 0x7fb4380a5b50 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb444036040 tx=0x7fb444005fd0 comp rx=0 tx=0).stop 2024-03-27T10:53:58.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:58.580+0000 7fb459e00640 1 -- 172.21.15.97:0/3295116502 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4540780f0 msgr2=0x7fb4541a58c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:53:58.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:58.580+0000 7fb459e00640 1 --2- 172.21.15.97:0/3295116502 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4540780f0 0x7fb4541a58c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb44c00eaa0 tx=0x7fb44c00ef60 comp rx=0 tx=0).stop 2024-03-27T10:53:58.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:58.580+0000 7fb459e00640 1 -- 172.21.15.97:0/3295116502 shutdown_connections 2024-03-27T10:53:58.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:58.580+0000 7fb459e00640 1 --2- 172.21.15.97:0/3295116502 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4540780f0 0x7fb4541a58c0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:58.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:58.580+0000 7fb459e00640 1 --2- 172.21.15.97:0/3295116502 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fb4380a3690 0x7fb4380a5b50 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:58.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:58.580+0000 7fb459e00640 1 --2- 172.21.15.97:0/3295116502 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb454077750 0x7fb45419e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:58.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:58.580+0000 7fb459e00640 1 --2- 172.21.15.97:0/3295116502 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb454076510 0x7fb45419e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:53:58.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:58.580+0000 7fb459e00640 1 -- 172.21.15.97:0/3295116502 >> 172.21.15.97:0/3295116502 conn(0x7fb4540ff300 msgr2=0x7fb4541009e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:53:58.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:58.581+0000 7fb459e00640 1 -- 172.21.15.97:0/3295116502 shutdown_connections 2024-03-27T10:53:58.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:53:58.581+0000 7fb459e00640 1 -- 172.21.15.97:0/3295116502 wait complete. 2024-03-27T10:53:59.268 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:59 smithi114 ceph-mon[35338]: pgmap v355: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T10:53:59.268 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:53:59 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/3295116502' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T10:53:59.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:59 smithi097 ceph-mon[31859]: pgmap v355: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T10:53:59.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:53:59 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/3295116502' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T10:53:59.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:59 smithi177 ceph-mon[35294]: pgmap v355: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T10:53:59.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:53:59 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/3295116502' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T10:54:00.572 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-27T10:54:00.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:00.818+0000 7f7c25000640 1 -- 172.21.15.97:0/1469792403 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c200780f0 msgr2=0x7f7c20078990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:00.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:00.818+0000 7f7c25000640 1 --2- 172.21.15.97:0/1469792403 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c200780f0 0x7f7c20078990 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7c18009ca0 tx=0x7f7c1802c0f0 comp rx=0 tx=0).stop 2024-03-27T10:54:00.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:00.818+0000 7f7c25000640 1 -- 172.21.15.97:0/1469792403 shutdown_connections 2024-03-27T10:54:00.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:00.818+0000 7f7c25000640 1 --2- 172.21.15.97:0/1469792403 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c200780f0 0x7f7c20078990 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:00.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:00.818+0000 7f7c25000640 1 --2- 172.21.15.97:0/1469792403 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7c20077750 0x7f7c20077bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:00.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:00.818+0000 7f7c25000640 1 --2- 172.21.15.97:0/1469792403 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7c20076510 0x7f7c20076930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:00.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:00.818+0000 7f7c25000640 1 -- 172.21.15.97:0/1469792403 >> 172.21.15.97:0/1469792403 conn(0x7f7c200ff300 msgr2=0x7f7c20101720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:54:00.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:00.818+0000 7f7c25000640 1 -- 172.21.15.97:0/1469792403 shutdown_connections 2024-03-27T10:54:00.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:00.818+0000 7f7c25000640 1 -- 172.21.15.97:0/1469792403 wait complete. 2024-03-27T10:54:00.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:00.819+0000 7f7c25000640 1 Processor -- start 2024-03-27T10:54:00.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:00.819+0000 7f7c25000640 1 -- start start 2024-03-27T10:54:00.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:00.820+0000 7f7c25000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c20076510 0x7f7c2019e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:00.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:00.820+0000 7f7c25000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7c20077750 0x7f7c2019e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:00.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:00.820+0000 7f7c25000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7c200780f0 0x7f7c201a58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:00.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:00.820+0000 7f7c25000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7c20113cb0 con 0x7f7c20076510 2024-03-27T10:54:00.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:00.820+0000 7f7c25000640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f7c20113b30 con 0x7f7c20077750 2024-03-27T10:54:00.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:00.820+0000 7f7c25000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f7c20113e30 con 0x7f7c200780f0 2024-03-27T10:54:00.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:00.820+0000 7f7c1de00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c20076510 0x7f7c2019e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:00.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:00.820+0000 7f7c1de00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c20076510 0x7f7c2019e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56904/0 (socket says 172.21.15.97:56904) 2024-03-27T10:54:00.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:00.820+0000 7f7c1de00640 1 -- 172.21.15.97:0/3115140885 learned_addr learned my addr 172.21.15.97:0/3115140885 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:54:00.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:00.820+0000 7f7c1d400640 1 --2- 172.21.15.97:0/3115140885 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7c20077750 0x7f7c2019e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:00.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:00.820+0000 7f7c1e800640 1 --2- 172.21.15.97:0/3115140885 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7c200780f0 0x7f7c201a58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:00.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:00.820+0000 7f7c1de00640 1 -- 172.21.15.97:0/3115140885 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7c200780f0 msgr2=0x7f7c201a58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:00.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:00.820+0000 7f7c1de00640 1 --2- 172.21.15.97:0/3115140885 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7c200780f0 0x7f7c201a58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:00.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:00.820+0000 7f7c1de00640 1 -- 172.21.15.97:0/3115140885 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7c20077750 msgr2=0x7f7c2019e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:00.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:00.820+0000 7f7c1de00640 1 --2- 172.21.15.97:0/3115140885 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7c20077750 0x7f7c2019e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:00.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:00.820+0000 7f7c1de00640 1 -- 172.21.15.97:0/3115140885 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c201a5f50 con 0x7f7c20076510 2024-03-27T10:54:00.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:00.821+0000 7f7c1de00640 1 --2- 172.21.15.97:0/3115140885 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c20076510 0x7f7c2019e330 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7c1000ea60 tx=0x7f7c1000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:54:00.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:00.821+0000 7f7c16a00640 1 -- 172.21.15.97:0/3115140885 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c1000cde0 con 0x7f7c20076510 2024-03-27T10:54:00.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:00.821+0000 7f7c16a00640 1 -- 172.21.15.97:0/3115140885 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7c100035f0 con 0x7f7c20076510 2024-03-27T10:54:00.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:00.821+0000 7f7c25000640 1 -- 172.21.15.97:0/3115140885 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c201a6240 con 0x7f7c20076510 2024-03-27T10:54:00.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:00.821+0000 7f7c16a00640 1 -- 172.21.15.97:0/3115140885 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c10010690 con 0x7f7c20076510 2024-03-27T10:54:00.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:00.821+0000 7f7c25000640 1 -- 172.21.15.97:0/3115140885 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c201a67e0 con 0x7f7c20076510 2024-03-27T10:54:00.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:00.823+0000 7f7c25000640 1 -- 172.21.15.97:0/3115140885 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c201108d0 con 0x7f7c20076510 2024-03-27T10:54:00.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:00.825+0000 7f7c16a00640 1 -- 172.21.15.97:0/3115140885 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138370+0+0 (secure 0 0 0) 0x7f7c100037a0 con 0x7f7c20076510 2024-03-27T10:54:00.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:00.826+0000 7f7c16a00640 1 --2- 172.21.15.97:0/3115140885 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f7c040a3690 0x7f7c040a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:00.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:00.826+0000 7f7c16a00640 1 -- 172.21.15.97:0/3115140885 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7127+0+0 (secure 0 0 0) 0x7f7c100cfe00 con 0x7f7c20076510 2024-03-27T10:54:00.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:00.826+0000 7f7c1d400640 1 --2- 172.21.15.97:0/3115140885 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f7c040a3690 0x7f7c040a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:00.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:00.827+0000 7f7c1d400640 1 --2- 172.21.15.97:0/3115140885 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f7c040a3690 0x7f7c040a5b50 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7c2019f840 tx=0x7f7c0c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:54:00.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:00.829+0000 7f7c16a00640 1 -- 172.21.15.97:0/3115140885 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f7c10053cb0 con 0x7f7c20076510 2024-03-27T10:54:01.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.078+0000 7f7c25000640 1 -- 172.21.15.97:0/3115140885 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7c2002d370 con 0x7f7c20076510 2024-03-27T10:54:01.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.079+0000 7f7c16a00640 1 -- 172.21.15.97:0/3115140885 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10494 (secure 0 0 0) 0x7f7c1009cdd0 con 0x7f7c20076510 2024-03-27T10:54:01.052 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T10:54:01.052 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1009423904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1009423904},{"type":"v1","addr":"172.21.15.97:6835","nonce":1009423904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3251173433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3251173433},{"type":"v1","addr":"172.21.15.97:6837","nonce":3251173433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14646,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6839/1057596821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1057596821},{"type":"v1","addr":"172.21.15.97:6839","nonce":1057596821}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14658,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6841/1672464856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6840","nonce":1672464856},{"type":"v1","addr":"172.21.15.97:6841","nonce":1672464856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3787981632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3787981632},{"type":"v1","addr":"172.21.15.177:6833","nonce":3787981632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/3349112132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":3349112132},{"type":"v1","addr":"172.21.15.177:6835","nonce":3349112132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6837/3244432640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":3244432640},{"type":"v1","addr":"172.21.15.177:6837","nonce":3244432640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24487,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6839/3340454605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":3340454605},{"type":"v1","addr":"172.21.15.177:6839","nonce":3340454605}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34415,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6833/4052517902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":4052517902},{"type":"v1","addr":"172.21.15.114:6833","nonce":4052517902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34445,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/3581918222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3581918222},{"type":"v1","addr":"172.21.15.114:6835","nonce":3581918222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34478,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/616775588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":616775588},{"type":"v1","addr":"172.21.15.114:6837","nonce":616775588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":34508,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6839/3937193784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":3937193784},{"type":"v1","addr":"172.21.15.114:6839","nonce":3937193784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-03-27T10:54:01.053 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 13 2024-03-27T10:54:01.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.083+0000 7f7c25000640 1 -- 172.21.15.97:0/3115140885 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f7c040a3690 msgr2=0x7f7c040a5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:01.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.083+0000 7f7c25000640 1 --2- 172.21.15.97:0/3115140885 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f7c040a3690 0x7f7c040a5b50 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7c2019f840 tx=0x7f7c0c005f90 comp rx=0 tx=0).stop 2024-03-27T10:54:01.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.083+0000 7f7c25000640 1 -- 172.21.15.97:0/3115140885 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c20076510 msgr2=0x7f7c2019e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:01.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.083+0000 7f7c25000640 1 --2- 172.21.15.97:0/3115140885 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c20076510 0x7f7c2019e330 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7c1000ea60 tx=0x7f7c1000ef20 comp rx=0 tx=0).stop 2024-03-27T10:54:01.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.083+0000 7f7c25000640 1 -- 172.21.15.97:0/3115140885 shutdown_connections 2024-03-27T10:54:01.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.083+0000 7f7c25000640 1 --2- 172.21.15.97:0/3115140885 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7c200780f0 0x7f7c201a58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:01.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.083+0000 7f7c25000640 1 --2- 172.21.15.97:0/3115140885 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f7c040a3690 0x7f7c040a5b50 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:01.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.083+0000 7f7c25000640 1 --2- 172.21.15.97:0/3115140885 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7c20077750 0x7f7c2019e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:01.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.083+0000 7f7c25000640 1 --2- 172.21.15.97:0/3115140885 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c20076510 0x7f7c2019e330 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:01.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.083+0000 7f7c25000640 1 -- 172.21.15.97:0/3115140885 >> 172.21.15.97:0/3115140885 conn(0x7f7c200ff300 msgr2=0x7f7c20100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:54:01.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.083+0000 7f7c25000640 1 -- 172.21.15.97:0/3115140885 shutdown_connections 2024-03-27T10:54:01.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.083+0000 7f7c25000640 1 -- 172.21.15.97:0/3115140885 wait complete. 2024-03-27T10:54:01.078 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:00 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:54:01.078 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:00 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:54:01.078 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:00 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:54:01.078 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:00 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:54:01.078 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:00 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:54:01.078 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:00 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:54:01.078 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:00 smithi097 ceph-mon[31859]: pgmap v356: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T10:54:01.078 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:00 smithi097 ceph-mon[31859]: mds.? [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] up:boot 2024-03-27T10:54:01.078 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:00 smithi097 ceph-mon[31859]: fsmap 12 up:standby 2024-03-27T10:54:01.079 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:00 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-27T10:54:01.079 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:00 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:54:01.081 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-27T10:54:01.177 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:00 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:54:01.177 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:00 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:54:01.177 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:00 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:54:01.177 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:00 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:54:01.177 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:00 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:54:01.177 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:00 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:54:01.177 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:00 smithi177 ceph-mon[35294]: pgmap v356: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T10:54:01.178 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:00 smithi177 ceph-mon[35294]: mds.? [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] up:boot 2024-03-27T10:54:01.178 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:00 smithi177 ceph-mon[35294]: fsmap 12 up:standby 2024-03-27T10:54:01.178 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:00 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-27T10:54:01.178 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:00 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:54:01.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.295+0000 7fcd97000640 1 -- 172.21.15.97:0/2558634747 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd90100d40 msgr2=0x7fcd901011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:01.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.295+0000 7fcd97000640 1 --2- 172.21.15.97:0/2558634747 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd90100d40 0x7fcd901011a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fcd78009900 tx=0x7fcd7802bc30 comp rx=0 tx=0).stop 2024-03-27T10:54:01.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.295+0000 7fcd97000640 1 -- 172.21.15.97:0/2558634747 shutdown_connections 2024-03-27T10:54:01.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.295+0000 7fcd97000640 1 --2- 172.21.15.97:0/2558634747 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcd901016e0 0x7fcd90101b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:01.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.295+0000 7fcd97000640 1 --2- 172.21.15.97:0/2558634747 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd90100d40 0x7fcd901011a0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:01.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.295+0000 7fcd97000640 1 --2- 172.21.15.97:0/2558634747 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcd901003e0 0x7fcd90100800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:01.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.295+0000 7fcd97000640 1 -- 172.21.15.97:0/2558634747 >> 172.21.15.97:0/2558634747 conn(0x7fcd900fbfe0 msgr2=0x7fcd900fe420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:54:01.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.295+0000 7fcd97000640 1 -- 172.21.15.97:0/2558634747 shutdown_connections 2024-03-27T10:54:01.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.295+0000 7fcd97000640 1 -- 172.21.15.97:0/2558634747 wait complete. 2024-03-27T10:54:01.269 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:00 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:54:01.269 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:00 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:54:01.269 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:00 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:54:01.270 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:00 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:54:01.270 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:00 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:54:01.270 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:00 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:54:01.270 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:00 smithi114 ceph-mon[35338]: pgmap v356: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T10:54:01.270 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:00 smithi114 ceph-mon[35338]: mds.? [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] up:boot 2024-03-27T10:54:01.270 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:00 smithi114 ceph-mon[35338]: fsmap 12 up:standby 2024-03-27T10:54:01.270 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:00 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-27T10:54:01.270 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:00 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:54:01.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.296+0000 7fcd97000640 1 Processor -- start 2024-03-27T10:54:01.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.296+0000 7fcd97000640 1 -- start start 2024-03-27T10:54:01.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.296+0000 7fcd97000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcd901003e0 0x7fcd90199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:01.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.297+0000 7fcd97000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcd90100d40 0x7fcd9019a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:01.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.297+0000 7fcd97000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd901016e0 0x7fcd901a1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:01.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.297+0000 7fcd97000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fcd9010f8e0 con 0x7fcd901016e0 2024-03-27T10:54:01.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.297+0000 7fcd97000640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fcd9010f760 con 0x7fcd90100d40 2024-03-27T10:54:01.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.297+0000 7fcd97000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fcd9010fa60 con 0x7fcd901003e0 2024-03-27T10:54:01.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.297+0000 7fcd8fe00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcd901003e0 0x7fcd90199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:01.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.297+0000 7fcd95000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd901016e0 0x7fcd901a1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:01.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.297+0000 7fcd8fe00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcd901003e0 0x7fcd90199f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.97:46334/0 (socket says 172.21.15.97:46334) 2024-03-27T10:54:01.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.297+0000 7fcd8fe00640 1 -- 172.21.15.97:0/256680903 learned_addr learned my addr 172.21.15.97:0/256680903 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:54:01.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.297+0000 7fcd87e00640 1 --2- 172.21.15.97:0/256680903 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcd90100d40 0x7fcd9019a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:01.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.297+0000 7fcd95000640 1 -- 172.21.15.97:0/256680903 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcd901003e0 msgr2=0x7fcd90199f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:01.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.297+0000 7fcd95000640 1 --2- 172.21.15.97:0/256680903 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcd901003e0 0x7fcd90199f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:01.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.297+0000 7fcd95000640 1 -- 172.21.15.97:0/256680903 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcd90100d40 msgr2=0x7fcd9019a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:01.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.297+0000 7fcd95000640 1 --2- 172.21.15.97:0/256680903 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcd90100d40 0x7fcd9019a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:01.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.297+0000 7fcd95000640 1 -- 172.21.15.97:0/256680903 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd901a1b70 con 0x7fcd901016e0 2024-03-27T10:54:01.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.297+0000 7fcd87e00640 1 --2- 172.21.15.97:0/256680903 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcd90100d40 0x7fcd9019a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T10:54:01.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.298+0000 7fcd95000640 1 --2- 172.21.15.97:0/256680903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd901016e0 0x7fcd901a1500 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fcd88009fd0 tx=0x7fcd8800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:54:01.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.298+0000 7fcd8d600640 1 -- 172.21.15.97:0/256680903 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd8800ce20 con 0x7fcd901016e0 2024-03-27T10:54:01.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.298+0000 7fcd8d600640 1 -- 172.21.15.97:0/256680903 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcd88005020 con 0x7fcd901016e0 2024-03-27T10:54:01.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.298+0000 7fcd97000640 1 -- 172.21.15.97:0/256680903 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd901a1e60 con 0x7fcd901016e0 2024-03-27T10:54:01.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.298+0000 7fcd8d600640 1 -- 172.21.15.97:0/256680903 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd88010690 con 0x7fcd901016e0 2024-03-27T10:54:01.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.298+0000 7fcd97000640 1 -- 172.21.15.97:0/256680903 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd901a2320 con 0x7fcd901016e0 2024-03-27T10:54:01.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.300+0000 7fcd97000640 1 -- 172.21.15.97:0/256680903 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd9010b2c0 con 0x7fcd901016e0 2024-03-27T10:54:01.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.302+0000 7fcd8d600640 1 -- 172.21.15.97:0/256680903 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138370+0+0 (secure 0 0 0) 0x7fcd880051d0 con 0x7fcd901016e0 2024-03-27T10:54:01.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.303+0000 7fcd8d600640 1 --2- 172.21.15.97:0/256680903 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fcd740a3610 0x7fcd740a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:01.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.303+0000 7fcd8d600640 1 -- 172.21.15.97:0/256680903 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7127+0+0 (secure 0 0 0) 0x7fcd880d0070 con 0x7fcd901016e0 2024-03-27T10:54:01.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.303+0000 7fcd8fe00640 1 --2- 172.21.15.97:0/256680903 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fcd740a3610 0x7fcd740a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:01.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.304+0000 7fcd8fe00640 1 --2- 172.21.15.97:0/256680903 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fcd740a3610 0x7fcd740a5ad0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fcd80005fd0 tx=0x7fcd8000b480 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:54:01.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.308+0000 7fcd8d600640 1 -- 172.21.15.97:0/256680903 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7fcd8809d040 con 0x7fcd901016e0 2024-03-27T10:54:01.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.508+0000 7fcd97000640 1 -- 172.21.15.97:0/256680903 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcd900627f0 con 0x7fcd901016e0 2024-03-27T10:54:01.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.510+0000 7fcd8d600640 1 -- 172.21.15.97:0/256680903 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+18448 (secure 0 0 0) 0x7fcd8809c940 con 0x7fcd901016e0 2024-03-27T10:54:01.483 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T10:54:01.483 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":86,"fsid":"adca88c4-ec26-11ee-95d0-87774f69a715","created":"2024-03-27T10:43:24.680694+0000","modified":"2024-03-27T10:53:01.568999+0000","last_up_change":"2024-03-27T10:53:00.552513+0000","last_in_change":"2024-03-27T10:52:36.302986+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-27T10:47:54.052810+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b634e5f1-6118-4e2f-95e9-830db390f8d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1746812080},{"type":"v1","addr":"172.21.15.97:6803","nonce":1746812080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1746812080},{"type":"v1","addr":"172.21.15.97:6805","nonce":1746812080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":1746812080},{"type":"v1","addr":"172.21.15.97:6809","nonce":1746812080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1746812080},{"type":"v1","addr":"172.21.15.97:6807","nonce":1746812080}]},"public_addr":"172.21.15.97:6803/1746812080","cluster_addr":"172.21.15.97:6805/1746812080","heartbeat_back_addr":"172.21.15.97:6809/1746812080","heartbeat_front_addr":"172.21.15.97:6807/1746812080","state":["exists","up"]},{"osd":1,"uuid":"441e2770-3284-4388-b0de-c56eb8a373d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":3858821264},{"type":"v1","addr":"172.21.15.114:6801","nonce":3858821264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":3858821264},{"type":"v1","addr":"172.21.15.114:6803","nonce":3858821264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":3858821264},{"type":"v1","addr":"172.21.15.114:6807","nonce":3858821264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":3858821264},{"type":"v1","addr":"172.21.15.114:6805","nonce":3858821264}]},"public_addr":"172.21.15.114:6801/3858821264","cluster_addr":"172.21.15.114:6803/3858821264","heartbeat_back_addr":"172.21.15.114:6807/3858821264","heartbeat_front_addr":"172.21.15.114:6805/3858821264","state":["exists","up"]},{"osd":2,"uuid":"00108864-1aa6-48d6-8398-e8eb79fda123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":124240658},{"type":"v1","addr":"172.21.15.177:6801","nonce":124240658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":124240658},{"type":"v1","addr":"172.21.15.177:6803","nonce":124240658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":124240658},{"type":"v1","addr":"172.21.15.177:6807","nonce":124240658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":124240658},{"type":"v1","addr":"172.21.15.177:6805","nonce":124240658}]},"public_addr":"172.21.15.177:6801/124240658","cluster_addr":"172.21.15.177:6803/124240658","heartbeat_back_addr":"172.21.15.177:6807/124240658","heartbeat_front_addr":"172.21.15.177:6805/124240658","state":["exists","up"]},{"osd":3,"uuid":"1860b690-e2dc-4977-88d7-d1aacf7022ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2478179311},{"type":"v1","addr":"172.21.15.97:6811","nonce":2478179311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2478179311},{"type":"v1","addr":"172.21.15.97:6813","nonce":2478179311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":2478179311},{"type":"v1","addr":"172.21.15.97:6817","nonce":2478179311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2478179311},{"type":"v1","addr":"172.21.15.97:6815","nonce":2478179311}]},"public_addr":"172.21.15.97:6811/2478179311","cluster_addr":"172.21.15.97:6813/2478179311","heartbeat_back_addr":"172.21.15.97:6817/2478179311","heartbeat_front_addr":"172.21.15.97:6815/2478179311","state":["exists","up"]},{"osd":4,"uuid":"9245a3a4-7d49-4fdb-9d55-112e43177050","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":75755945},{"type":"v1","addr":"172.21.15.114:6809","nonce":75755945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":75755945},{"type":"v1","addr":"172.21.15.114:6811","nonce":75755945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":75755945},{"type":"v1","addr":"172.21.15.114:6815","nonce":75755945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":75755945},{"type":"v1","addr":"172.21.15.114:6813","nonce":75755945}]},"public_addr":"172.21.15.114:6809/75755945","cluster_addr":"172.21.15.114:6811/75755945","heartbeat_back_addr":"172.21.15.114:6815/75755945","heartbeat_front_addr":"172.21.15.114:6813/75755945","state":["exists","up"]},{"osd":5,"uuid":"51922f99-e8d3-461b-8615-e9bfdcbf5ed3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":1918493790},{"type":"v1","addr":"172.21.15.177:6809","nonce":1918493790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":1918493790},{"type":"v1","addr":"172.21.15.177:6811","nonce":1918493790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":1918493790},{"type":"v1","addr":"172.21.15.177:6815","nonce":1918493790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":1918493790},{"type":"v1","addr":"172.21.15.177:6813","nonce":1918493790}]},"public_addr":"172.21.15.177:6809/1918493790","cluster_addr":"172.21.15.177:6811/1918493790","heartbeat_back_addr":"172.21.15.177:6815/1918493790","heartbeat_front_addr":"172.21.15.177:6813/1918493790","state":["exists","up"]},{"osd":6,"uuid":"06dffb32-05eb-43b3-852e-915c5cc7f25a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2892389563},{"type":"v1","addr":"172.21.15.97:6819","nonce":2892389563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2892389563},{"type":"v1","addr":"172.21.15.97:6821","nonce":2892389563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2892389563},{"type":"v1","addr":"172.21.15.97:6825","nonce":2892389563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2892389563},{"type":"v1","addr":"172.21.15.97:6823","nonce":2892389563}]},"public_addr":"172.21.15.97:6819/2892389563","cluster_addr":"172.21.15.97:6821/2892389563","heartbeat_back_addr":"172.21.15.97:6825/2892389563","heartbeat_front_addr":"172.21.15.97:6823/2892389563","state":["exists","up"]},{"osd":7,"uuid":"e68e3519-9c1b-43c5-8e06-5385248b19df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":2445491153},{"type":"v1","addr":"172.21.15.114:6817","nonce":2445491153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":2445491153},{"type":"v1","addr":"172.21.15.114:6819","nonce":2445491153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2445491153},{"type":"v1","addr":"172.21.15.114:6823","nonce":2445491153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":2445491153},{"type":"v1","addr":"172.21.15.114:6821","nonce":2445491153}]},"public_addr":"172.21.15.114:6817/2445491153","cluster_addr":"172.21.15.114:6819/2445491153","heartbeat_back_addr":"172.21.15.114:6823/2445491153","heartbeat_front_addr":"172.21.15.114:6821/2445491153","state":["exists","up"]},{"osd":8,"uuid":"31ac592e-1349-49ee-9af9-8ad85d682978","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":2051134191},{"type":"v1","addr":"172.21.15.177:6817","nonce":2051134191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":2051134191},{"type":"v1","addr":"172.21.15.177:6819","nonce":2051134191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":2051134191},{"type":"v1","addr":"172.21.15.177:6823","nonce":2051134191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":2051134191},{"type":"v1","addr":"172.21.15.177:6821","nonce":2051134191}]},"public_addr":"172.21.15.177:6817/2051134191","cluster_addr":"172.21.15.177:6819/2051134191","heartbeat_back_addr":"172.21.15.177:6823/2051134191","heartbeat_front_addr":"172.21.15.177:6821/2051134191","state":["exists","up"]},{"osd":9,"uuid":"0171086a-dd41-4eb4-a389-16f0c46a64db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":1578881271},{"type":"v1","addr":"172.21.15.97:6827","nonce":1578881271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1578881271},{"type":"v1","addr":"172.21.15.97:6829","nonce":1578881271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1578881271},{"type":"v1","addr":"172.21.15.97:6833","nonce":1578881271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1578881271},{"type":"v1","addr":"172.21.15.97:6831","nonce":1578881271}]},"public_addr":"172.21.15.97:6827/1578881271","cluster_addr":"172.21.15.97:6829/1578881271","heartbeat_back_addr":"172.21.15.97:6833/1578881271","heartbeat_front_addr":"172.21.15.97:6831/1578881271","state":["exists","up"]},{"osd":10,"uuid":"64754e31-b124-4601-ad98-2f70b1dd5359","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":2893111461},{"type":"v1","addr":"172.21.15.114:6825","nonce":2893111461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":2893111461},{"type":"v1","addr":"172.21.15.114:6827","nonce":2893111461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2893111461},{"type":"v1","addr":"172.21.15.114:6831","nonce":2893111461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":2893111461},{"type":"v1","addr":"172.21.15.114:6829","nonce":2893111461}]},"public_addr":"172.21.15.114:6825/2893111461","cluster_addr":"172.21.15.114:6827/2893111461","heartbeat_back_addr":"172.21.15.114:6831/2893111461","heartbeat_front_addr":"172.21.15.114:6829/2893111461","state":["exists","up"]},{"osd":11,"uuid":"9ffc752c-adf7-4037-89d3-535787d7475d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":85,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":2333030391},{"type":"v1","addr":"172.21.15.177:6825","nonce":2333030391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":2333030391},{"type":"v1","addr":"172.21.15.177:6827","nonce":2333030391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":2333030391},{"type":"v1","addr":"172.21.15.177:6831","nonce":2333030391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":2333030391},{"type":"v1","addr":"172.21.15.177:6829","nonce":2333030391}]},"public_addr":"172.21.15.177:6825/2333030391","cluster_addr":"172.21.15.177:6827/2333030391","heartbeat_back_addr":"172.21.15.177:6831/2333030391","heartbeat_front_addr":"172.21.15.177:6829/2333030391","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:46:45.143519+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:47:18.656947+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:47:51.316345+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:48:24.575512+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:48:59.113555+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:49:32.103865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:50:06.615539+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:50:41.578341+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:51:15.254011+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:51:49.099165+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:52:24.544821+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:52:58.838484+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/359646":"2024-03-28T10:44:39.889550+0000","172.21.15.97:6801/98682155":"2024-03-28T10:44:06.019075+0000","172.21.15.97:6801/1685921853":"2024-03-28T10:44:39.889550+0000","172.21.15.97:6800/98682155":"2024-03-28T10:44:06.019075+0000","172.21.15.97:0/107690680":"2024-03-28T10:44:06.019075+0000","172.21.15.97:6800/1685921853":"2024-03-28T10:44:39.889550+0000","172.21.15.97:0/437043413":"2024-03-28T10:44:39.889550+0000","172.21.15.97:0/2319706992":"2024-03-28T10:44:39.889550+0000","172.21.15.97:0/3911532654":"2024-03-28T10:44:06.019075+0000","172.21.15.97:0/3347321845":"2024-03-28T10:44:06.019075+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-27T10:54:01.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.513+0000 7fcd97000640 1 -- 172.21.15.97:0/256680903 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fcd740a3610 msgr2=0x7fcd740a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:01.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.513+0000 7fcd97000640 1 --2- 172.21.15.97:0/256680903 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fcd740a3610 0x7fcd740a5ad0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fcd80005fd0 tx=0x7fcd8000b480 comp rx=0 tx=0).stop 2024-03-27T10:54:01.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.513+0000 7fcd97000640 1 -- 172.21.15.97:0/256680903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd901016e0 msgr2=0x7fcd901a1500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:01.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.513+0000 7fcd97000640 1 --2- 172.21.15.97:0/256680903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd901016e0 0x7fcd901a1500 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fcd88009fd0 tx=0x7fcd8800ef10 comp rx=0 tx=0).stop 2024-03-27T10:54:01.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.513+0000 7fcd97000640 1 -- 172.21.15.97:0/256680903 shutdown_connections 2024-03-27T10:54:01.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.513+0000 7fcd97000640 1 --2- 172.21.15.97:0/256680903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd901016e0 0x7fcd901a1500 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:01.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.513+0000 7fcd97000640 1 --2- 172.21.15.97:0/256680903 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fcd740a3610 0x7fcd740a5ad0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:01.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.513+0000 7fcd97000640 1 --2- 172.21.15.97:0/256680903 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcd90100d40 0x7fcd9019a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:01.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.513+0000 7fcd97000640 1 --2- 172.21.15.97:0/256680903 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcd901003e0 0x7fcd90199f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:01.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.513+0000 7fcd97000640 1 -- 172.21.15.97:0/256680903 >> 172.21.15.97:0/256680903 conn(0x7fcd900fbfe0 msgr2=0x7fcd900fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:54:01.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.513+0000 7fcd97000640 1 -- 172.21.15.97:0/256680903 shutdown_connections 2024-03-27T10:54:01.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.513+0000 7fcd97000640 1 -- 172.21.15.97:0/256680903 wait complete. 2024-03-27T10:54:01.506 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-27T10:47:54.052810+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-27T10:54:01.506 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-27T10:54:01.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.735+0000 7fabdf200640 1 -- 172.21.15.97:0/3620832491 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fabe0101a50 msgr2=0x7fabe0101e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:01.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.735+0000 7fabdf200640 1 --2- 172.21.15.97:0/3620832491 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fabe0101a50 0x7fabe0101e70 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fabd0009900 tx=0x7fabd002bc30 comp rx=0 tx=0).stop 2024-03-27T10:54:01.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.735+0000 7fabdf200640 1 -- 172.21.15.97:0/3620832491 shutdown_connections 2024-03-27T10:54:01.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.735+0000 7fabdf200640 1 --2- 172.21.15.97:0/3620832491 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fabe0103600 0x7fabe0109e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:01.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.735+0000 7fabdf200640 1 --2- 172.21.15.97:0/3620832491 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fabe0102c40 0x7fabe01030c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:01.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.735+0000 7fabdf200640 1 --2- 172.21.15.97:0/3620832491 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fabe0101a50 0x7fabe0101e70 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:01.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.735+0000 7fabdf200640 1 -- 172.21.15.97:0/3620832491 >> 172.21.15.97:0/3620832491 conn(0x7fabe00fcfc0 msgr2=0x7fabe00ff400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:54:01.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.735+0000 7fabdf200640 1 -- 172.21.15.97:0/3620832491 shutdown_connections 2024-03-27T10:54:01.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.735+0000 7fabdf200640 1 -- 172.21.15.97:0/3620832491 wait complete. 2024-03-27T10:54:01.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.736+0000 7fabdf200640 1 Processor -- start 2024-03-27T10:54:01.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.736+0000 7fabdf200640 1 -- start start 2024-03-27T10:54:01.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.737+0000 7fabdf200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fabe0101a50 0x7fabe0075900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:01.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.737+0000 7fabdf200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fabe0102c40 0x7fabe0073f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:01.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.737+0000 7fabdf200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fabe0103600 0x7fabe0074480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:01.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.737+0000 7fabdf200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fabe010d5c0 con 0x7fabe0103600 2024-03-27T10:54:01.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.737+0000 7fabdf200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fabe010d440 con 0x7fabe0102c40 2024-03-27T10:54:01.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.737+0000 7fabdf200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fabe010d740 con 0x7fabe0101a50 2024-03-27T10:54:01.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.737+0000 7fabde800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fabe0103600 0x7fabe0074480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:01.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.737+0000 7fabdd400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fabe0102c40 0x7fabe0073f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:01.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.737+0000 7fabde800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fabe0103600 0x7fabe0074480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56946/0 (socket says 172.21.15.97:56946) 2024-03-27T10:54:01.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.737+0000 7fabdd400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fabe0102c40 0x7fabe0073f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:43848/0 (socket says 172.21.15.97:43848) 2024-03-27T10:54:01.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.737+0000 7fabde800640 1 -- 172.21.15.97:0/3622939122 learned_addr learned my addr 172.21.15.97:0/3622939122 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:54:01.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.737+0000 7fabdde00640 1 --2- 172.21.15.97:0/3622939122 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fabe0101a50 0x7fabe0075900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:01.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.738+0000 7fabde800640 1 -- 172.21.15.97:0/3622939122 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fabe0101a50 msgr2=0x7fabe0075900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:01.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.738+0000 7fabde800640 1 --2- 172.21.15.97:0/3622939122 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fabe0101a50 0x7fabe0075900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:01.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.738+0000 7fabde800640 1 -- 172.21.15.97:0/3622939122 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fabe0102c40 msgr2=0x7fabe0073f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:01.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.738+0000 7fabde800640 1 --2- 172.21.15.97:0/3622939122 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fabe0102c40 0x7fabe0073f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:01.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.738+0000 7fabde800640 1 -- 172.21.15.97:0/3622939122 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabe0074d10 con 0x7fabe0103600 2024-03-27T10:54:01.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.738+0000 7fabde800640 1 --2- 172.21.15.97:0/3622939122 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fabe0103600 0x7fabe0074480 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fabd800eaa0 tx=0x7fabd800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:54:01.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.738+0000 7fabd6a00640 1 -- 172.21.15.97:0/3622939122 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabd800cdb0 con 0x7fabe0103600 2024-03-27T10:54:01.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.738+0000 7fabd6a00640 1 -- 172.21.15.97:0/3622939122 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fabd8005020 con 0x7fabe0103600 2024-03-27T10:54:01.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.738+0000 7fabdf200640 1 -- 172.21.15.97:0/3622939122 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabe01a1de0 con 0x7fabe0103600 2024-03-27T10:54:01.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.738+0000 7fabd6a00640 1 -- 172.21.15.97:0/3622939122 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabd80116d0 con 0x7fabe0103600 2024-03-27T10:54:01.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.738+0000 7fabdf200640 1 -- 172.21.15.97:0/3622939122 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabe01a2240 con 0x7fabe0103600 2024-03-27T10:54:01.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.739+0000 7fabdf200640 1 -- 172.21.15.97:0/3622939122 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabe0108de0 con 0x7fabe0103600 2024-03-27T10:54:01.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.741+0000 7fabd6a00640 1 -- 172.21.15.97:0/3622939122 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138370+0+0 (secure 0 0 0) 0x7fabd80051d0 con 0x7fabe0103600 2024-03-27T10:54:01.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.742+0000 7fabd6a00640 1 --2- 172.21.15.97:0/3622939122 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fabc40a3690 0x7fabc40a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:01.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.743+0000 7fabd6a00640 1 -- 172.21.15.97:0/3622939122 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7127+0+0 (secure 0 0 0) 0x7fabd80cf900 con 0x7fabe0103600 2024-03-27T10:54:01.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.743+0000 7fabdde00640 1 --2- 172.21.15.97:0/3622939122 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fabc40a3690 0x7fabc40a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:01.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.743+0000 7fabdde00640 1 --2- 172.21.15.97:0/3622939122 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fabc40a3690 0x7fabc40a5b50 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fabd002c240 tx=0x7fabd002c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:54:01.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.747+0000 7fabd6a00640 1 -- 172.21.15.97:0/3622939122 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7fabd8016030 con 0x7fabe0103600 2024-03-27T10:54:01.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.945+0000 7fabdf200640 1 -- 172.21.15.97:0/3622939122 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fabe0062750 con 0x7fabe0103600 2024-03-27T10:54:01.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.945+0000 7fabd6a00640 1 -- 172.21.15.97:0/3622939122 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) ==== 93+0+10 (secure 0 0 0) 0x7fabd809c320 con 0x7fabe0103600 2024-03-27T10:54:01.918 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-03-27T10:54:01.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.948+0000 7fabdf200640 1 -- 172.21.15.97:0/3622939122 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fabc40a3690 msgr2=0x7fabc40a5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:01.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.948+0000 7fabdf200640 1 --2- 172.21.15.97:0/3622939122 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fabc40a3690 0x7fabc40a5b50 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fabd002c240 tx=0x7fabd002c7e0 comp rx=0 tx=0).stop 2024-03-27T10:54:01.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.948+0000 7fabdf200640 1 -- 172.21.15.97:0/3622939122 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fabe0103600 msgr2=0x7fabe0074480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:01.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.948+0000 7fabdf200640 1 --2- 172.21.15.97:0/3622939122 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fabe0103600 0x7fabe0074480 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fabd800eaa0 tx=0x7fabd800ef60 comp rx=0 tx=0).stop 2024-03-27T10:54:01.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.948+0000 7fabdf200640 1 -- 172.21.15.97:0/3622939122 shutdown_connections 2024-03-27T10:54:01.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.948+0000 7fabdf200640 1 --2- 172.21.15.97:0/3622939122 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fabe0103600 0x7fabe0074480 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:01.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.948+0000 7fabdf200640 1 --2- 172.21.15.97:0/3622939122 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fabc40a3690 0x7fabc40a5b50 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:01.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.948+0000 7fabdf200640 1 --2- 172.21.15.97:0/3622939122 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fabe0102c40 0x7fabe0073f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:01.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.948+0000 7fabdf200640 1 --2- 172.21.15.97:0/3622939122 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fabe0101a50 0x7fabe0075900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:01.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.948+0000 7fabdf200640 1 -- 172.21.15.97:0/3622939122 >> 172.21.15.97:0/3622939122 conn(0x7fabe00fcfc0 msgr2=0x7fabe0105fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:54:01.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.948+0000 7fabdf200640 1 -- 172.21.15.97:0/3622939122 shutdown_connections 2024-03-27T10:54:01.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:01.948+0000 7fabdf200640 1 -- 172.21.15.97:0/3622939122 wait complete. 2024-03-27T10:54:01.939 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-27T10:54:02.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:02.157+0000 7f21a5800640 1 -- 172.21.15.97:0/71541497 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21a0103720 msgr2=0x7f21a0109fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:02.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:02.157+0000 7f21a5800640 1 --2- 172.21.15.97:0/71541497 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21a0103720 0x7f21a0109fb0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2198009ca0 tx=0x7f219802c0f0 comp rx=0 tx=0).stop 2024-03-27T10:54:02.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:02.157+0000 7f21a5800640 1 -- 172.21.15.97:0/71541497 shutdown_connections 2024-03-27T10:54:02.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:02.157+0000 7f21a5800640 1 --2- 172.21.15.97:0/71541497 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21a0103720 0x7f21a0109fb0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:02.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:02.157+0000 7f21a5800640 1 --2- 172.21.15.97:0/71541497 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f21a0102d60 0x7f21a01031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:02.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:02.157+0000 7f21a5800640 1 --2- 172.21.15.97:0/71541497 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f21a0101b70 0x7f21a0101f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:02.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:02.157+0000 7f21a5800640 1 -- 172.21.15.97:0/71541497 >> 172.21.15.97:0/71541497 conn(0x7f21a00fd100 msgr2=0x7f21a00ff520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:54:02.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:02.157+0000 7f21a5800640 1 -- 172.21.15.97:0/71541497 shutdown_connections 2024-03-27T10:54:02.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:02.157+0000 7f21a5800640 1 -- 172.21.15.97:0/71541497 wait complete. 2024-03-27T10:54:02.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:02.158+0000 7f21a5800640 1 Processor -- start 2024-03-27T10:54:02.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:02.158+0000 7f21a5800640 1 -- start start 2024-03-27T10:54:02.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:02.158+0000 7f21a5800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f21a0101b70 0x7f21a0197d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:02.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:02.158+0000 7f21a5800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f21a0102d60 0x7f21a0198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:02.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:02.158+0000 7f21a5800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21a0103720 0x7f21a019f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:02.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:02.158+0000 7f21a5800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f21a010d6a0 con 0x7f21a0103720 2024-03-27T10:54:02.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:02.158+0000 7f21a5800640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f21a010d520 con 0x7f21a0102d60 2024-03-27T10:54:02.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:02.158+0000 7f21a5800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f21a010d820 con 0x7f21a0101b70 2024-03-27T10:54:02.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:02.159+0000 7f219f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21a0103720 0x7f21a019f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:02.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:02.159+0000 7f219f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21a0103720 0x7f21a019f300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56968/0 (socket says 172.21.15.97:56968) 2024-03-27T10:54:02.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:02.159+0000 7f219f400640 1 -- 172.21.15.97:0/2914925744 learned_addr learned my addr 172.21.15.97:0/2914925744 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:54:02.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:02.159+0000 7f219ea00640 1 --2- 172.21.15.97:0/2914925744 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f21a0101b70 0x7f21a0197d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:02.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:02.159+0000 7f219e000640 1 --2- 172.21.15.97:0/2914925744 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f21a0102d60 0x7f21a0198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:02.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:02.159+0000 7f219f400640 1 -- 172.21.15.97:0/2914925744 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f21a0101b70 msgr2=0x7f21a0197d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:02.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:02.159+0000 7f219f400640 1 --2- 172.21.15.97:0/2914925744 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f21a0101b70 0x7f21a0197d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:02.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:02.159+0000 7f219f400640 1 -- 172.21.15.97:0/2914925744 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f21a0102d60 msgr2=0x7f21a0198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:02.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:02.159+0000 7f219f400640 1 --2- 172.21.15.97:0/2914925744 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f21a0102d60 0x7f21a0198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:02.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:02.159+0000 7f219f400640 1 -- 172.21.15.97:0/2914925744 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21a019f970 con 0x7f21a0103720 2024-03-27T10:54:02.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:02.159+0000 7f219f400640 1 --2- 172.21.15.97:0/2914925744 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21a0103720 0x7f21a019f300 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2198009dc0 tx=0x7f21980328a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:54:02.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:02.160+0000 7f2197400640 1 -- 172.21.15.97:0/2914925744 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2198041020 con 0x7f21a0103720 2024-03-27T10:54:02.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:02.160+0000 7f2197400640 1 -- 172.21.15.97:0/2914925744 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2198003e20 con 0x7f21a0103720 2024-03-27T10:54:02.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:02.160+0000 7f21a5800640 1 -- 172.21.15.97:0/2914925744 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21a019fc00 con 0x7f21a0103720 2024-03-27T10:54:02.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:02.160+0000 7f2197400640 1 -- 172.21.15.97:0/2914925744 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2198039040 con 0x7f21a0103720 2024-03-27T10:54:02.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:02.160+0000 7f21a5800640 1 -- 172.21.15.97:0/2914925744 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21a01a00f0 con 0x7f21a0103720 2024-03-27T10:54:02.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:02.161+0000 7f21a5800640 1 -- 172.21.15.97:0/2914925744 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21a0108e30 con 0x7f21a0103720 2024-03-27T10:54:02.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:02.163+0000 7f2197400640 1 -- 172.21.15.97:0/2914925744 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138370+0+0 (secure 0 0 0) 0x7f2198044020 con 0x7f21a0103720 2024-03-27T10:54:02.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:02.164+0000 7f2197400640 1 --2- 172.21.15.97:0/2914925744 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f21840a3610 0x7f21840a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:02.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:02.165+0000 7f2197400640 1 -- 172.21.15.97:0/2914925744 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7127+0+0 (secure 0 0 0) 0x7f21980f22e0 con 0x7f21a0103720 2024-03-27T10:54:02.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:02.165+0000 7f219ea00640 1 --2- 172.21.15.97:0/2914925744 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f21840a3610 0x7f21840a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:02.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:02.165+0000 7f219ea00640 1 --2- 172.21.15.97:0/2914925744 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f21840a3610 0x7f21840a5ad0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f21a0102bc0 tx=0x7f2190005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:54:02.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:02.169+0000 7f2197400640 1 -- 172.21.15.97:0/2914925744 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f219804b020 con 0x7f21a0103720 2024-03-27T10:54:02.267 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:02 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/3115140885' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T10:54:02.267 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:02 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/256680903' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T10:54:02.267 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:02 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/3622939122' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-27T10:54:02.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:02.376+0000 7f21a5800640 1 -- 172.21.15.97:0/2914925744 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f21a002d4f0 con 0x7f21a0103720 2024-03-27T10:54:02.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:02.377+0000 7f2197400640 1 -- 172.21.15.97:0/2914925744 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v86) ==== 84+0+65 (secure 0 0 0) 0x7f21980bf2b0 con 0x7f21a0103720 2024-03-27T10:54:02.350 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T10:54:02.350 INFO:teuthology.orchestra.run.smithi097.stdout:[ 2024-03-27T10:54:02.350 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T10:54:02.350 INFO:teuthology.orchestra.run.smithi097.stdout: "poolnum": 1, 2024-03-27T10:54:02.350 INFO:teuthology.orchestra.run.smithi097.stdout: "poolname": ".mgr" 2024-03-27T10:54:02.350 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T10:54:02.350 INFO:teuthology.orchestra.run.smithi097.stdout:] 2024-03-27T10:54:02.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:02.379+0000 7f21a5800640 1 -- 172.21.15.97:0/2914925744 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f21840a3610 msgr2=0x7f21840a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:02.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:02.379+0000 7f21a5800640 1 --2- 172.21.15.97:0/2914925744 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f21840a3610 0x7f21840a5ad0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f21a0102bc0 tx=0x7f2190005f90 comp rx=0 tx=0).stop 2024-03-27T10:54:02.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:02.379+0000 7f21a5800640 1 -- 172.21.15.97:0/2914925744 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21a0103720 msgr2=0x7f21a019f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:02.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:02.379+0000 7f21a5800640 1 --2- 172.21.15.97:0/2914925744 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21a0103720 0x7f21a019f300 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2198009dc0 tx=0x7f21980328a0 comp rx=0 tx=0).stop 2024-03-27T10:54:02.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:02.380+0000 7f21a5800640 1 -- 172.21.15.97:0/2914925744 shutdown_connections 2024-03-27T10:54:02.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:02.380+0000 7f21a5800640 1 --2- 172.21.15.97:0/2914925744 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21a0103720 0x7f21a019f300 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:02.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:02.380+0000 7f21a5800640 1 --2- 172.21.15.97:0/2914925744 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f21840a3610 0x7f21840a5ad0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:02.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:02.380+0000 7f21a5800640 1 --2- 172.21.15.97:0/2914925744 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f21a0102d60 0x7f21a0198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:02.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:02.380+0000 7f21a5800640 1 --2- 172.21.15.97:0/2914925744 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f21a0101b70 0x7f21a0197d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:02.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:02.380+0000 7f21a5800640 1 -- 172.21.15.97:0/2914925744 >> 172.21.15.97:0/2914925744 conn(0x7f21a00fd100 msgr2=0x7f21a0106000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:54:02.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:02.380+0000 7f21a5800640 1 -- 172.21.15.97:0/2914925744 shutdown_connections 2024-03-27T10:54:02.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:02.380+0000 7f21a5800640 1 -- 172.21.15.97:0/2914925744 wait complete. 2024-03-27T10:54:02.369 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:02 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/3115140885' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T10:54:02.369 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:02 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/256680903' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T10:54:02.369 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:02 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/3622939122' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-27T10:54:02.372 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-27T10:54:02.372 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-27T10:54:02.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:02 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/3115140885' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T10:54:02.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:02 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/256680903' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T10:54:02.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:02 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/3622939122' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-27T10:54:02.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:02.587+0000 7f6b4da00640 1 -- 172.21.15.97:0/947833973 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6b481025f0 msgr2=0x7f6b481049e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:02.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:02.587+0000 7f6b4da00640 1 --2- 172.21.15.97:0/947833973 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6b481025f0 0x7f6b481049e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6b38009900 tx=0x7f6b3802bc30 comp rx=0 tx=0).stop 2024-03-27T10:54:02.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:02.587+0000 7f6b4da00640 1 -- 172.21.15.97:0/947833973 shutdown_connections 2024-03-27T10:54:02.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:02.587+0000 7f6b4da00640 1 --2- 172.21.15.97:0/947833973 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b48107850 0x7f6b48109ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:02.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:02.587+0000 7f6b4da00640 1 --2- 172.21.15.97:0/947833973 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6b48104f20 0x7f6b48107310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:02.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:02.587+0000 7f6b4da00640 1 --2- 172.21.15.97:0/947833973 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6b481025f0 0x7f6b481049e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:02.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:02.587+0000 7f6b4da00640 1 -- 172.21.15.97:0/947833973 >> 172.21.15.97:0/947833973 conn(0x7f6b480fbfe0 msgr2=0x7f6b480fe420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:54:02.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:02.587+0000 7f6b4da00640 1 -- 172.21.15.97:0/947833973 shutdown_connections 2024-03-27T10:54:02.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:02.587+0000 7f6b4da00640 1 -- 172.21.15.97:0/947833973 wait complete. 2024-03-27T10:54:02.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:02.588+0000 7f6b4da00640 1 Processor -- start 2024-03-27T10:54:02.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:02.588+0000 7f6b4da00640 1 -- start start 2024-03-27T10:54:02.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:02.589+0000 7f6b4da00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6b481025f0 0x7f6b48101f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:02.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:02.589+0000 7f6b4da00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6b48104f20 0x7f6b481005a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:02.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:02.589+0000 7f6b4da00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b48107850 0x7f6b48100ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:02.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:02.589+0000 7f6b4da00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6b4810d6d0 con 0x7f6b48107850 2024-03-27T10:54:02.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:02.589+0000 7f6b4da00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f6b4810d550 con 0x7f6b48104f20 2024-03-27T10:54:02.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:02.589+0000 7f6b4da00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6b4810d850 con 0x7f6b481025f0 2024-03-27T10:54:02.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:02.589+0000 7f6b3fe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6b48104f20 0x7f6b481005a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:02.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:02.589+0000 7f6b47400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b48107850 0x7f6b48100ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:02.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:02.589+0000 7f6b47400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b48107850 0x7f6b48100ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56972/0 (socket says 172.21.15.97:56972) 2024-03-27T10:54:02.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:02.589+0000 7f6b47400640 1 -- 172.21.15.97:0/2243804266 learned_addr learned my addr 172.21.15.97:0/2243804266 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:54:02.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:02.589+0000 7f6b46a00640 1 --2- 172.21.15.97:0/2243804266 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6b481025f0 0x7f6b48101f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:02.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:02.590+0000 7f6b47400640 1 -- 172.21.15.97:0/2243804266 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6b481025f0 msgr2=0x7f6b48101f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:02.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:02.590+0000 7f6b47400640 1 --2- 172.21.15.97:0/2243804266 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6b481025f0 0x7f6b48101f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:02.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:02.590+0000 7f6b47400640 1 -- 172.21.15.97:0/2243804266 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6b48104f20 msgr2=0x7f6b481005a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:02.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:02.590+0000 7f6b47400640 1 --2- 172.21.15.97:0/2243804266 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6b48104f20 0x7f6b481005a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:02.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:02.590+0000 7f6b47400640 1 -- 172.21.15.97:0/2243804266 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b48101370 con 0x7f6b48107850 2024-03-27T10:54:02.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:02.590+0000 7f6b47400640 1 --2- 172.21.15.97:0/2243804266 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b48107850 0x7f6b48100ae0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6b4000efd0 tx=0x7f6b4000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:54:02.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:02.590+0000 7f6b3f400640 1 -- 172.21.15.97:0/2243804266 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b40003b70 con 0x7f6b48107850 2024-03-27T10:54:02.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:02.590+0000 7f6b3f400640 1 -- 172.21.15.97:0/2243804266 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6b40003d10 con 0x7f6b48107850 2024-03-27T10:54:02.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:02.590+0000 7f6b4da00640 1 -- 172.21.15.97:0/2243804266 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b481a1fd0 con 0x7f6b48107850 2024-03-27T10:54:02.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:02.591+0000 7f6b3f400640 1 -- 172.21.15.97:0/2243804266 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b40010980 con 0x7f6b48107850 2024-03-27T10:54:02.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:02.591+0000 7f6b4da00640 1 -- 172.21.15.97:0/2243804266 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b481a23e0 con 0x7f6b48107850 2024-03-27T10:54:02.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:02.592+0000 7f6b4da00640 1 -- 172.21.15.97:0/2243804266 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b480fdad0 con 0x7f6b48107850 2024-03-27T10:54:02.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:02.595+0000 7f6b3f400640 1 -- 172.21.15.97:0/2243804266 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138370+0+0 (secure 0 0 0) 0x7f6b40004020 con 0x7f6b48107850 2024-03-27T10:54:02.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:02.596+0000 7f6b3f400640 1 --2- 172.21.15.97:0/2243804266 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f6b2c0a3610 0x7f6b2c0a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:02.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:02.596+0000 7f6b3f400640 1 -- 172.21.15.97:0/2243804266 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7127+0+0 (secure 0 0 0) 0x7f6b400d0090 con 0x7f6b48107850 2024-03-27T10:54:02.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:02.597+0000 7f6b46a00640 1 --2- 172.21.15.97:0/2243804266 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f6b2c0a3610 0x7f6b2c0a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:02.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:02.597+0000 7f6b46a00640 1 --2- 172.21.15.97:0/2243804266 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f6b2c0a3610 0x7f6b2c0a5ad0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6b38009230 tx=0x7f6b38005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:54:02.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:02.603+0000 7f6b3f400640 1 -- 172.21.15.97:0/2243804266 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f6b4009d0a0 con 0x7f6b48107850 2024-03-27T10:54:02.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:02.831+0000 7f6b4da00640 1 -- 172.21.15.97:0/2243804266 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f6b48062750 con 0x7f6b48107850 2024-03-27T10:54:02.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:03.015+0000 7f6b3f400640 1 -- 172.21.15.97:0/2243804266 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v87) ==== 138+0+0 (secure 0 0 0) 0x7f6b4009c9a0 con 0x7f6b48107850 2024-03-27T10:54:02.989 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_metadata' created 2024-03-27T10:54:02.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:03.019+0000 7f6b4da00640 1 -- 172.21.15.97:0/2243804266 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f6b2c0a3610 msgr2=0x7f6b2c0a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:02.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:03.019+0000 7f6b4da00640 1 --2- 172.21.15.97:0/2243804266 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f6b2c0a3610 0x7f6b2c0a5ad0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6b38009230 tx=0x7f6b38005fd0 comp rx=0 tx=0).stop 2024-03-27T10:54:02.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:03.019+0000 7f6b4da00640 1 -- 172.21.15.97:0/2243804266 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b48107850 msgr2=0x7f6b48100ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:02.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:03.019+0000 7f6b4da00640 1 --2- 172.21.15.97:0/2243804266 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b48107850 0x7f6b48100ae0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6b4000efd0 tx=0x7f6b4000c6a0 comp rx=0 tx=0).stop 2024-03-27T10:54:02.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:03.019+0000 7f6b4da00640 1 -- 172.21.15.97:0/2243804266 shutdown_connections 2024-03-27T10:54:02.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:03.019+0000 7f6b4da00640 1 --2- 172.21.15.97:0/2243804266 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b48107850 0x7f6b48100ae0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:02.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:03.019+0000 7f6b4da00640 1 --2- 172.21.15.97:0/2243804266 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f6b2c0a3610 0x7f6b2c0a5ad0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:02.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:03.019+0000 7f6b4da00640 1 --2- 172.21.15.97:0/2243804266 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6b48104f20 0x7f6b481005a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:02.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:03.019+0000 7f6b4da00640 1 --2- 172.21.15.97:0/2243804266 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6b481025f0 0x7f6b48101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:02.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:03.019+0000 7f6b4da00640 1 -- 172.21.15.97:0/2243804266 >> 172.21.15.97:0/2243804266 conn(0x7f6b480fbfe0 msgr2=0x7f6b48103090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:54:02.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:03.020+0000 7f6b4da00640 1 -- 172.21.15.97:0/2243804266 shutdown_connections 2024-03-27T10:54:02.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:03.020+0000 7f6b4da00640 1 -- 172.21.15.97:0/2243804266 wait complete. 2024-03-27T10:54:03.018 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-27T10:54:03.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:03.247+0000 7f9c22c00640 1 -- 172.21.15.97:0/4055395958 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c1c102e00 msgr2=0x7f9c1c103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:03.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:03.247+0000 7f9c22c00640 1 --2- 172.21.15.97:0/4055395958 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c1c102e00 0x7f9c1c103280 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9c08009930 tx=0x7f9c0802bc20 comp rx=0 tx=0).stop 2024-03-27T10:54:03.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:03.247+0000 7f9c22c00640 1 -- 172.21.15.97:0/4055395958 shutdown_connections 2024-03-27T10:54:03.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:03.247+0000 7f9c22c00640 1 --2- 172.21.15.97:0/4055395958 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9c1c1037c0 0x7f9c1c10a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:03.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:03.247+0000 7f9c22c00640 1 --2- 172.21.15.97:0/4055395958 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c1c102e00 0x7f9c1c103280 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:03.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:03.247+0000 7f9c22c00640 1 --2- 172.21.15.97:0/4055395958 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9c1c0ffe30 0x7f9c1c100250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:03.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:03.247+0000 7f9c22c00640 1 -- 172.21.15.97:0/4055395958 >> 172.21.15.97:0/4055395958 conn(0x7f9c1c0fb740 msgr2=0x7f9c1c0fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:54:03.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:03.247+0000 7f9c22c00640 1 -- 172.21.15.97:0/4055395958 shutdown_connections 2024-03-27T10:54:03.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:03.247+0000 7f9c22c00640 1 -- 172.21.15.97:0/4055395958 wait complete. 2024-03-27T10:54:03.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:03.248+0000 7f9c22c00640 1 Processor -- start 2024-03-27T10:54:03.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:03.248+0000 7f9c22c00640 1 -- start start 2024-03-27T10:54:03.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:03.248+0000 7f9c22c00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9c1c0ffe30 0x7f9c1c197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:03.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:03.249+0000 7f9c22c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9c1c102e00 0x7f9c1c198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:03.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:03.249+0000 7f9c22c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c1c1037c0 0x7f9c1c19f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:03.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:03.249+0000 7f9c22c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9c1c10d6f0 con 0x7f9c1c1037c0 2024-03-27T10:54:03.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:03.249+0000 7f9c22c00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9c1c10d570 con 0x7f9c1c102e00 2024-03-27T10:54:03.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:03.249+0000 7f9c22c00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f9c1c10d870 con 0x7f9c1c0ffe30 2024-03-27T10:54:03.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:03.249+0000 7f9c20c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c1c1037c0 0x7f9c1c19f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:03.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:03.249+0000 7f9c20c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c1c1037c0 0x7f9c1c19f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56990/0 (socket says 172.21.15.97:56990) 2024-03-27T10:54:03.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:03.249+0000 7f9c20c00640 1 -- 172.21.15.97:0/919515696 learned_addr learned my addr 172.21.15.97:0/919515696 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:54:03.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:03.249+0000 7f9c1be00640 1 --2- 172.21.15.97:0/919515696 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9c1c0ffe30 0x7f9c1c197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:03.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:03.249+0000 7f9c1b400640 1 --2- 172.21.15.97:0/919515696 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9c1c102e00 0x7f9c1c198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:03.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:03.249+0000 7f9c20c00640 1 -- 172.21.15.97:0/919515696 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9c1c0ffe30 msgr2=0x7f9c1c197df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:03.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:03.249+0000 7f9c20c00640 1 --2- 172.21.15.97:0/919515696 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9c1c0ffe30 0x7f9c1c197df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:03.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:03.249+0000 7f9c20c00640 1 -- 172.21.15.97:0/919515696 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9c1c102e00 msgr2=0x7f9c1c198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:03.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:03.249+0000 7f9c20c00640 1 --2- 172.21.15.97:0/919515696 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9c1c102e00 0x7f9c1c198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:03.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:03.249+0000 7f9c20c00640 1 -- 172.21.15.97:0/919515696 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c1c19fa10 con 0x7f9c1c1037c0 2024-03-27T10:54:03.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:03.250+0000 7f9c20c00640 1 --2- 172.21.15.97:0/919515696 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c1c1037c0 0x7f9c1c19f3a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9c1400efd0 tx=0x7f9c1400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:54:03.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:03.250+0000 7f9c18c00640 1 -- 172.21.15.97:0/919515696 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c14003b70 con 0x7f9c1c1037c0 2024-03-27T10:54:03.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:03.250+0000 7f9c22c00640 1 -- 172.21.15.97:0/919515696 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c1c19fd00 con 0x7f9c1c1037c0 2024-03-27T10:54:03.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:03.250+0000 7f9c18c00640 1 -- 172.21.15.97:0/919515696 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9c14003d10 con 0x7f9c1c1037c0 2024-03-27T10:54:03.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:03.250+0000 7f9c18c00640 1 -- 172.21.15.97:0/919515696 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c14010980 con 0x7f9c1c1037c0 2024-03-27T10:54:03.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:03.250+0000 7f9c22c00640 1 -- 172.21.15.97:0/919515696 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c1c1a01d0 con 0x7f9c1c1037c0 2024-03-27T10:54:03.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:03.251+0000 7f9c22c00640 1 -- 172.21.15.97:0/919515696 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c1c108fd0 con 0x7f9c1c1037c0 2024-03-27T10:54:03.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:03.254+0000 7f9c18c00640 1 -- 172.21.15.97:0/919515696 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138370+0+0 (secure 0 0 0) 0x7f9c14004b40 con 0x7f9c1c1037c0 2024-03-27T10:54:03.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:03.255+0000 7f9c18c00640 1 --2- 172.21.15.97:0/919515696 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f9c000a3690 0x7f9c000a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:03.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:03.255+0000 7f9c18c00640 1 -- 172.21.15.97:0/919515696 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7497+0+0 (secure 0 0 0) 0x7f9c140d01b0 con 0x7f9c1c1037c0 2024-03-27T10:54:03.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:03.255+0000 7f9c1be00640 1 --2- 172.21.15.97:0/919515696 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f9c000a3690 0x7f9c000a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:03.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:03.256+0000 7f9c1be00640 1 --2- 172.21.15.97:0/919515696 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f9c000a3690 0x7f9c000a5b50 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c005fd0 tx=0x7f9c0c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:54:03.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:03.258+0000 7f9c18c00640 1 -- 172.21.15.97:0/919515696 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f9c1409d010 con 0x7f9c1c1037c0 2024-03-27T10:54:03.267 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:03 smithi114 ceph-mon[35338]: pgmap v357: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T10:54:03.267 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:03 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/2914925744' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-27T10:54:03.267 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:03 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/2243804266' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-27T10:54:03.267 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:03 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:54:03.268 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:03 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:54:03.268 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:03 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:54:03.268 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:03 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:54:03.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:03 smithi097 ceph-mon[31859]: pgmap v357: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T10:54:03.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:03 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/2914925744' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-27T10:54:03.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:03 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/2243804266' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-27T10:54:03.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:03 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:54:03.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:03 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:54:03.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:03 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:54:03.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:03 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:54:03.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:03 smithi177 ceph-mon[35294]: pgmap v357: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T10:54:03.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:03 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/2914925744' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-27T10:54:03.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:03 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/2243804266' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-27T10:54:03.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:03 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:54:03.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:03 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:54:03.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:03 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:54:03.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:03 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:54:03.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:03.461+0000 7f9c22c00640 1 -- 172.21.15.97:0/919515696 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f9c1c02d370 con 0x7f9c1c1037c0 2024-03-27T10:54:03.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:04.024+0000 7f9c18c00640 1 -- 172.21.15.97:0/919515696 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v88) ==== 170+0+0 (secure 0 0 0) 0x7f9c1409c910 con 0x7f9c1c1037c0 2024-03-27T10:54:03.997 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_data' created 2024-03-27T10:54:04.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:04.028+0000 7f9c22c00640 1 -- 172.21.15.97:0/919515696 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f9c000a3690 msgr2=0x7f9c000a5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:04.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:04.028+0000 7f9c22c00640 1 --2- 172.21.15.97:0/919515696 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f9c000a3690 0x7f9c000a5b50 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c005fd0 tx=0x7f9c0c00b410 comp rx=0 tx=0).stop 2024-03-27T10:54:04.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:04.028+0000 7f9c22c00640 1 -- 172.21.15.97:0/919515696 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c1c1037c0 msgr2=0x7f9c1c19f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:04.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:04.028+0000 7f9c22c00640 1 --2- 172.21.15.97:0/919515696 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c1c1037c0 0x7f9c1c19f3a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9c1400efd0 tx=0x7f9c1400c6a0 comp rx=0 tx=0).stop 2024-03-27T10:54:04.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:04.028+0000 7f9c22c00640 1 -- 172.21.15.97:0/919515696 shutdown_connections 2024-03-27T10:54:04.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:04.028+0000 7f9c22c00640 1 --2- 172.21.15.97:0/919515696 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c1c1037c0 0x7f9c1c19f3a0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:04.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:04.028+0000 7f9c22c00640 1 --2- 172.21.15.97:0/919515696 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f9c000a3690 0x7f9c000a5b50 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:04.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:04.028+0000 7f9c22c00640 1 --2- 172.21.15.97:0/919515696 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9c1c102e00 0x7f9c1c198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:04.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:04.028+0000 7f9c22c00640 1 --2- 172.21.15.97:0/919515696 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9c1c0ffe30 0x7f9c1c197df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:04.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:04.028+0000 7f9c22c00640 1 -- 172.21.15.97:0/919515696 >> 172.21.15.97:0/919515696 conn(0x7f9c1c0fb740 msgr2=0x7f9c1c1061a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:54:04.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:04.028+0000 7f9c22c00640 1 -- 172.21.15.97:0/919515696 shutdown_connections 2024-03-27T10:54:04.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:04.029+0000 7f9c22c00640 1 -- 172.21.15.97:0/919515696 wait complete. 2024-03-27T10:54:04.033 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-27T10:54:04.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:04.248+0000 7fc7f0a00640 1 -- 172.21.15.97:0/986605375 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7ec0745b0 msgr2=0x7fc7ec074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:04.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:04.248+0000 7fc7f0a00640 1 --2- 172.21.15.97:0/986605375 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7ec0745b0 0x7fc7ec074a30 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fc7d8009930 tx=0x7fc7d802bc30 comp rx=0 tx=0).stop 2024-03-27T10:54:04.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:04.248+0000 7fc7f0a00640 1 -- 172.21.15.97:0/986605375 shutdown_connections 2024-03-27T10:54:04.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:04.248+0000 7fc7f0a00640 1 --2- 172.21.15.97:0/986605375 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc7ec10df20 0x7fc7ec110310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:04.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:04.248+0000 7fc7f0a00640 1 --2- 172.21.15.97:0/986605375 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7ec0745b0 0x7fc7ec074a30 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:04.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:04.248+0000 7fc7f0a00640 1 --2- 172.21.15.97:0/986605375 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc7ec075b90 0x7fc7ec073fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:04.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:04.248+0000 7fc7f0a00640 1 -- 172.21.15.97:0/986605375 >> 172.21.15.97:0/986605375 conn(0x7fc7ec0fcee0 msgr2=0x7fc7ec0ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:54:04.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:04.249+0000 7fc7f0a00640 1 -- 172.21.15.97:0/986605375 shutdown_connections 2024-03-27T10:54:04.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:04.249+0000 7fc7f0a00640 1 -- 172.21.15.97:0/986605375 wait complete. 2024-03-27T10:54:04.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:04.249+0000 7fc7f0a00640 1 Processor -- start 2024-03-27T10:54:04.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:04.250+0000 7fc7f0a00640 1 -- start start 2024-03-27T10:54:04.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:04.250+0000 7fc7f0a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc7ec0745b0 0x7fc7ec19e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:04.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:04.250+0000 7fc7f0a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc7ec075b90 0x7fc7ec19e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:04.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:04.250+0000 7fc7f0a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7ec10df20 0x7fc7ec1a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:04.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:04.250+0000 7fc7f0a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc7ec113ac0 con 0x7fc7ec10df20 2024-03-27T10:54:04.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:04.250+0000 7fc7f0a00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc7ec113940 con 0x7fc7ec0745b0 2024-03-27T10:54:04.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:04.250+0000 7fc7f0a00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc7ec113c40 con 0x7fc7ec075b90 2024-03-27T10:54:04.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:04.250+0000 7fc7ebe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7ec10df20 0x7fc7ec1a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:04.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:04.250+0000 7fc7ebe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7ec10df20 0x7fc7ec1a5780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57008/0 (socket says 172.21.15.97:57008) 2024-03-27T10:54:04.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:04.250+0000 7fc7ebe00640 1 -- 172.21.15.97:0/276538256 learned_addr learned my addr 172.21.15.97:0/276538256 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:54:04.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:04.250+0000 7fc7eb400640 1 --2- 172.21.15.97:0/276538256 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc7ec0745b0 0x7fc7ec19e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:04.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:04.250+0000 7fc7eaa00640 1 --2- 172.21.15.97:0/276538256 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc7ec075b90 0x7fc7ec19e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:04.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:04.251+0000 7fc7ebe00640 1 -- 172.21.15.97:0/276538256 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc7ec075b90 msgr2=0x7fc7ec19e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:04.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:04.251+0000 7fc7ebe00640 1 --2- 172.21.15.97:0/276538256 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc7ec075b90 0x7fc7ec19e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:04.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:04.251+0000 7fc7ebe00640 1 -- 172.21.15.97:0/276538256 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc7ec0745b0 msgr2=0x7fc7ec19e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:04.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:04.251+0000 7fc7ebe00640 1 --2- 172.21.15.97:0/276538256 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc7ec0745b0 0x7fc7ec19e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:04.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:04.251+0000 7fc7ebe00640 1 -- 172.21.15.97:0/276538256 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7ec1a5df0 con 0x7fc7ec10df20 2024-03-27T10:54:04.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:04.251+0000 7fc7eb400640 1 --2- 172.21.15.97:0/276538256 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc7ec0745b0 0x7fc7ec19e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T10:54:04.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:04.251+0000 7fc7ebe00640 1 --2- 172.21.15.97:0/276538256 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7ec10df20 0x7fc7ec1a5780 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc7e400cad0 tx=0x7fc7e400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:54:04.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:04.251+0000 7fc7e3e00640 1 -- 172.21.15.97:0/276538256 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e400ae20 con 0x7fc7ec10df20 2024-03-27T10:54:04.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:04.251+0000 7fc7e3e00640 1 -- 172.21.15.97:0/276538256 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc7e4005020 con 0x7fc7ec10df20 2024-03-27T10:54:04.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:04.251+0000 7fc7f0a00640 1 -- 172.21.15.97:0/276538256 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7ec1a60e0 con 0x7fc7ec10df20 2024-03-27T10:54:04.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:04.251+0000 7fc7e3e00640 1 -- 172.21.15.97:0/276538256 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e400f6b0 con 0x7fc7ec10df20 2024-03-27T10:54:04.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:04.252+0000 7fc7f0a00640 1 -- 172.21.15.97:0/276538256 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7ec1a6600 con 0x7fc7ec10df20 2024-03-27T10:54:04.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:04.253+0000 7fc7f0a00640 1 -- 172.21.15.97:0/276538256 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7ec068860 con 0x7fc7ec10df20 2024-03-27T10:54:04.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:04.255+0000 7fc7e3e00640 1 -- 172.21.15.97:0/276538256 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138370+0+0 (secure 0 0 0) 0x7fc7e40051d0 con 0x7fc7ec10df20 2024-03-27T10:54:04.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:04.256+0000 7fc7e3e00640 1 --2- 172.21.15.97:0/276538256 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fc7d00a3690 0x7fc7d00a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:04.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:04.256+0000 7fc7e3e00640 1 -- 172.21.15.97:0/276538256 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7879+0+0 (secure 0 0 0) 0x7fc7e40d0020 con 0x7fc7ec10df20 2024-03-27T10:54:04.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:04.256+0000 7fc7eb400640 1 --2- 172.21.15.97:0/276538256 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fc7d00a3690 0x7fc7d00a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:04.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:04.257+0000 7fc7eb400640 1 --2- 172.21.15.97:0/276538256 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fc7d00a3690 0x7fc7d00a5b50 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc7dc0058c0 tx=0x7fc7dc005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:54:04.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:04.260+0000 7fc7e3e00640 1 -- 172.21.15.97:0/276538256 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7fc7e4014030 con 0x7fc7ec10df20 2024-03-27T10:54:04.267 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:04 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/2243804266' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-27T10:54:04.268 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:04 smithi114 ceph-mon[35338]: osdmap e87: 12 total, 12 up, 12 in 2024-03-27T10:54:04.268 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:04 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:54:04.268 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:04 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:54:04.268 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:04 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/919515696' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-27T10:54:04.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:04 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/2243804266' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-27T10:54:04.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:04 smithi097 ceph-mon[31859]: osdmap e87: 12 total, 12 up, 12 in 2024-03-27T10:54:04.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:04 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:54:04.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:04 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:54:04.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:04 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/919515696' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-27T10:54:04.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:04 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/2243804266' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-27T10:54:04.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:04 smithi177 ceph-mon[35294]: osdmap e87: 12 total, 12 up, 12 in 2024-03-27T10:54:04.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:04 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:54:04.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:04 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:54:04.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:04 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/919515696' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-27T10:54:04.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:04.507+0000 7fc7f0a00640 1 -- 172.21.15.97:0/276538256 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fc7ec103fe0 con 0x7fc7ec10df20 2024-03-27T10:54:05.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.232+0000 7fc7e3e00640 1 -- 172.21.15.97:0/276538256 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7fc7e409ce20 con 0x7fc7ec10df20 2024-03-27T10:54:05.206 INFO:teuthology.orchestra.run.smithi097.stderr:new fs with metadata pool 2 and data pool 3 2024-03-27T10:54:05.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.235+0000 7fc7f0a00640 1 -- 172.21.15.97:0/276538256 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fc7d00a3690 msgr2=0x7fc7d00a5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:05.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.235+0000 7fc7f0a00640 1 --2- 172.21.15.97:0/276538256 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fc7d00a3690 0x7fc7d00a5b50 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc7dc0058c0 tx=0x7fc7dc005850 comp rx=0 tx=0).stop 2024-03-27T10:54:05.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.235+0000 7fc7f0a00640 1 -- 172.21.15.97:0/276538256 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7ec10df20 msgr2=0x7fc7ec1a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:05.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.235+0000 7fc7f0a00640 1 --2- 172.21.15.97:0/276538256 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7ec10df20 0x7fc7ec1a5780 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc7e400cad0 tx=0x7fc7e400cf90 comp rx=0 tx=0).stop 2024-03-27T10:54:05.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.236+0000 7fc7f0a00640 1 -- 172.21.15.97:0/276538256 shutdown_connections 2024-03-27T10:54:05.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.236+0000 7fc7f0a00640 1 --2- 172.21.15.97:0/276538256 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7ec10df20 0x7fc7ec1a5780 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:05.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.236+0000 7fc7f0a00640 1 --2- 172.21.15.97:0/276538256 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc7ec075b90 0x7fc7ec19e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:05.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.236+0000 7fc7f0a00640 1 --2- 172.21.15.97:0/276538256 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fc7d00a3690 0x7fc7d00a5b50 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:05.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.236+0000 7fc7f0a00640 1 --2- 172.21.15.97:0/276538256 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc7ec0745b0 0x7fc7ec19e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:05.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.236+0000 7fc7f0a00640 1 -- 172.21.15.97:0/276538256 >> 172.21.15.97:0/276538256 conn(0x7fc7ec0fcee0 msgr2=0x7fc7ec1011b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:54:05.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.236+0000 7fc7f0a00640 1 -- 172.21.15.97:0/276538256 shutdown_connections 2024-03-27T10:54:05.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.236+0000 7fc7f0a00640 1 -- 172.21.15.97:0/276538256 wait complete. 2024-03-27T10:54:05.232 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-27T10:54:05.292 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:05 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/919515696' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-27T10:54:05.292 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:05 smithi097 ceph-mon[31859]: osdmap e88: 12 total, 12 up, 12 in 2024-03-27T10:54:05.292 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:05 smithi097 ceph-mon[31859]: pgmap v360: 97 pgs: 14 creating+peering, 1 active+clean, 82 unknown; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T10:54:05.292 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:05 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:54:05.292 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:05 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:54:05.292 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:05 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:54:05.292 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:05 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/276538256' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-27T10:54:05.292 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:05 smithi097 ceph-mon[31859]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-27T10:54:05.292 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:05 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:54:05.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.447+0000 7f2dd8e00640 1 -- 172.21.15.97:0/179730503 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2dd4104d10 msgr2=0x7f2dd4107100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:05.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.447+0000 7f2dd8e00640 1 --2- 172.21.15.97:0/179730503 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2dd4104d10 0x7f2dd4107100 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2dc0009900 tx=0x7f2dc002bc30 comp rx=0 tx=0).stop 2024-03-27T10:54:05.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.447+0000 7f2dd8e00640 1 -- 172.21.15.97:0/179730503 shutdown_connections 2024-03-27T10:54:05.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.447+0000 7f2dd8e00640 1 --2- 172.21.15.97:0/179730503 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2dd4107640 0x7f2dd4109a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:05.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.447+0000 7f2dd8e00640 1 --2- 172.21.15.97:0/179730503 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2dd4104d10 0x7f2dd4107100 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:05.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.447+0000 7f2dd8e00640 1 --2- 172.21.15.97:0/179730503 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2dd41023e0 0x7f2dd41047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:05.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.447+0000 7f2dd8e00640 1 -- 172.21.15.97:0/179730503 >> 172.21.15.97:0/179730503 conn(0x7f2dd40fbdd0 msgr2=0x7f2dd40fe210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:54:05.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.447+0000 7f2dd8e00640 1 -- 172.21.15.97:0/179730503 shutdown_connections 2024-03-27T10:54:05.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.447+0000 7f2dd8e00640 1 -- 172.21.15.97:0/179730503 wait complete. 2024-03-27T10:54:05.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.448+0000 7f2dd8e00640 1 Processor -- start 2024-03-27T10:54:05.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.448+0000 7f2dd8e00640 1 -- start start 2024-03-27T10:54:05.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.448+0000 7f2dd8e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2dd41023e0 0x7f2dd4199db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:05.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.448+0000 7f2dd8e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2dd4104d10 0x7f2dd419a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:05.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.448+0000 7f2dd8e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2dd4107640 0x7f2dd41a1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:05.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.448+0000 7f2dd8e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd410d4e0 con 0x7f2dd41023e0 2024-03-27T10:54:05.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.448+0000 7f2dd8e00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd410d360 con 0x7f2dd4104d10 2024-03-27T10:54:05.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.448+0000 7f2dd8e00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd410d660 con 0x7f2dd4107640 2024-03-27T10:54:05.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.448+0000 7f2dd3400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2dd41023e0 0x7f2dd4199db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:05.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.449+0000 7f2dd3400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2dd41023e0 0x7f2dd4199db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57026/0 (socket says 172.21.15.97:57026) 2024-03-27T10:54:05.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.449+0000 7f2dd3400640 1 -- 172.21.15.97:0/2397407242 learned_addr learned my addr 172.21.15.97:0/2397407242 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:54:05.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.449+0000 7f2dd2a00640 1 --2- 172.21.15.97:0/2397407242 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2dd4104d10 0x7f2dd419a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:05.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.449+0000 7f2dd3e00640 1 --2- 172.21.15.97:0/2397407242 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2dd4107640 0x7f2dd41a1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:05.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.449+0000 7f2dd3400640 1 -- 172.21.15.97:0/2397407242 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2dd4107640 msgr2=0x7f2dd41a1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:05.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.449+0000 7f2dd3400640 1 --2- 172.21.15.97:0/2397407242 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2dd4107640 0x7f2dd41a1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:05.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.449+0000 7f2dd3400640 1 -- 172.21.15.97:0/2397407242 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2dd4104d10 msgr2=0x7f2dd419a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:05.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.449+0000 7f2dd3400640 1 --2- 172.21.15.97:0/2397407242 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2dd4104d10 0x7f2dd419a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:05.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.449+0000 7f2dd3400640 1 -- 172.21.15.97:0/2397407242 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dd41a19c0 con 0x7f2dd41023e0 2024-03-27T10:54:05.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.449+0000 7f2dd2a00640 1 --2- 172.21.15.97:0/2397407242 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2dd4104d10 0x7f2dd419a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T10:54:05.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.449+0000 7f2dd3400640 1 --2- 172.21.15.97:0/2397407242 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2dd41023e0 0x7f2dd4199db0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2dc400e9f0 tx=0x7f2dc400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:54:05.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.449+0000 7f2dcbe00640 1 -- 172.21.15.97:0/2397407242 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dc400cd80 con 0x7f2dd41023e0 2024-03-27T10:54:05.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.450+0000 7f2dcbe00640 1 -- 172.21.15.97:0/2397407242 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2dc40108e0 con 0x7f2dd41023e0 2024-03-27T10:54:05.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.450+0000 7f2dd8e00640 1 -- 172.21.15.97:0/2397407242 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2dd41a1cb0 con 0x7f2dd41023e0 2024-03-27T10:54:05.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.450+0000 7f2dcbe00640 1 -- 172.21.15.97:0/2397407242 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dc4018920 con 0x7f2dd41023e0 2024-03-27T10:54:05.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.450+0000 7f2dd8e00640 1 -- 172.21.15.97:0/2397407242 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2dd41a21a0 con 0x7f2dd41023e0 2024-03-27T10:54:05.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.451+0000 7f2dd8e00640 1 -- 172.21.15.97:0/2397407242 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2dd4109190 con 0x7f2dd41023e0 2024-03-27T10:54:05.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.453+0000 7f2dcbe00640 1 -- 172.21.15.97:0/2397407242 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138370+0+0 (secure 0 0 0) 0x7f2dc4010a90 con 0x7f2dd41023e0 2024-03-27T10:54:05.426 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:05 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/919515696' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-27T10:54:05.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:05 smithi177 ceph-mon[35294]: osdmap e88: 12 total, 12 up, 12 in 2024-03-27T10:54:05.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:05 smithi177 ceph-mon[35294]: pgmap v360: 97 pgs: 14 creating+peering, 1 active+clean, 82 unknown; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T10:54:05.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:05 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:54:05.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:05 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:54:05.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:05 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:54:05.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:05 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/276538256' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-27T10:54:05.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:05 smithi177 ceph-mon[35294]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-27T10:54:05.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:05 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:54:05.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.455+0000 7f2dcbe00640 1 --2- 172.21.15.97:0/2397407242 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f2db80a3690 0x7f2db80a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:05.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.455+0000 7f2dcbe00640 1 -- 172.21.15.97:0/2397407242 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7f2dc40cff60 con 0x7f2dd41023e0 2024-03-27T10:54:05.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.455+0000 7f2dd2a00640 1 --2- 172.21.15.97:0/2397407242 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f2db80a3690 0x7f2db80a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:05.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.456+0000 7f2dd2a00640 1 --2- 172.21.15.97:0/2397407242 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f2db80a3690 0x7f2db80a5b50 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2dc002c240 tx=0x7f2dc0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:54:05.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.460+0000 7f2dcbe00640 1 -- 172.21.15.97:0/2397407242 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f2dc409cc80 con 0x7f2dd41023e0 2024-03-27T10:54:05.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:05 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/919515696' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-27T10:54:05.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:05 smithi114 ceph-mon[35338]: osdmap e88: 12 total, 12 up, 12 in 2024-03-27T10:54:05.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:05 smithi114 ceph-mon[35338]: pgmap v360: 97 pgs: 14 creating+peering, 1 active+clean, 82 unknown; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T10:54:05.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:05 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:54:05.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:05 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:54:05.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:05 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:54:05.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:05 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/276538256' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-27T10:54:05.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:05 smithi114 ceph-mon[35338]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-27T10:54:05.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:05 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:54:05.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.676+0000 7f2dd8e00640 1 -- 172.21.15.97:0/2397407242 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2dd402d4f0 con 0x7f2dd41023e0 2024-03-27T10:54:05.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.678+0000 7f2dcbe00640 1 -- 172.21.15.97:0/2397407242 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+22224 (secure 0 0 0) 0x7f2dc409c580 con 0x7f2dd41023e0 2024-03-27T10:54:05.651 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T10:54:05.652 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":89,"fsid":"adca88c4-ec26-11ee-95d0-87774f69a715","created":"2024-03-27T10:43:24.680694+0000","modified":"2024-03-27T10:54:04.510278+0000","last_up_change":"2024-03-27T10:53:00.552513+0000","last_in_change":"2024-03-27T10:52:36.302986+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-27T10:47:54.052810+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-27T10:54:02.845060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-27T10:54:03.474151+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b634e5f1-6118-4e2f-95e9-830db390f8d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1746812080},{"type":"v1","addr":"172.21.15.97:6803","nonce":1746812080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1746812080},{"type":"v1","addr":"172.21.15.97:6805","nonce":1746812080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":1746812080},{"type":"v1","addr":"172.21.15.97:6809","nonce":1746812080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1746812080},{"type":"v1","addr":"172.21.15.97:6807","nonce":1746812080}]},"public_addr":"172.21.15.97:6803/1746812080","cluster_addr":"172.21.15.97:6805/1746812080","heartbeat_back_addr":"172.21.15.97:6809/1746812080","heartbeat_front_addr":"172.21.15.97:6807/1746812080","state":["exists","up"]},{"osd":1,"uuid":"441e2770-3284-4388-b0de-c56eb8a373d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":3858821264},{"type":"v1","addr":"172.21.15.114:6801","nonce":3858821264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":3858821264},{"type":"v1","addr":"172.21.15.114:6803","nonce":3858821264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":3858821264},{"type":"v1","addr":"172.21.15.114:6807","nonce":3858821264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":3858821264},{"type":"v1","addr":"172.21.15.114:6805","nonce":3858821264}]},"public_addr":"172.21.15.114:6801/3858821264","cluster_addr":"172.21.15.114:6803/3858821264","heartbeat_back_addr":"172.21.15.114:6807/3858821264","heartbeat_front_addr":"172.21.15.114:6805/3858821264","state":["exists","up"]},{"osd":2,"uuid":"00108864-1aa6-48d6-8398-e8eb79fda123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":124240658},{"type":"v1","addr":"172.21.15.177:6801","nonce":124240658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":124240658},{"type":"v1","addr":"172.21.15.177:6803","nonce":124240658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":124240658},{"type":"v1","addr":"172.21.15.177:6807","nonce":124240658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":124240658},{"type":"v1","addr":"172.21.15.177:6805","nonce":124240658}]},"public_addr":"172.21.15.177:6801/124240658","cluster_addr":"172.21.15.177:6803/124240658","heartbeat_back_addr":"172.21.15.177:6807/124240658","heartbeat_front_addr":"172.21.15.177:6805/124240658","state":["exists","up"]},{"osd":3,"uuid":"1860b690-e2dc-4977-88d7-d1aacf7022ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2478179311},{"type":"v1","addr":"172.21.15.97:6811","nonce":2478179311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2478179311},{"type":"v1","addr":"172.21.15.97:6813","nonce":2478179311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":2478179311},{"type":"v1","addr":"172.21.15.97:6817","nonce":2478179311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2478179311},{"type":"v1","addr":"172.21.15.97:6815","nonce":2478179311}]},"public_addr":"172.21.15.97:6811/2478179311","cluster_addr":"172.21.15.97:6813/2478179311","heartbeat_back_addr":"172.21.15.97:6817/2478179311","heartbeat_front_addr":"172.21.15.97:6815/2478179311","state":["exists","up"]},{"osd":4,"uuid":"9245a3a4-7d49-4fdb-9d55-112e43177050","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":75755945},{"type":"v1","addr":"172.21.15.114:6809","nonce":75755945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":75755945},{"type":"v1","addr":"172.21.15.114:6811","nonce":75755945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":75755945},{"type":"v1","addr":"172.21.15.114:6815","nonce":75755945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":75755945},{"type":"v1","addr":"172.21.15.114:6813","nonce":75755945}]},"public_addr":"172.21.15.114:6809/75755945","cluster_addr":"172.21.15.114:6811/75755945","heartbeat_back_addr":"172.21.15.114:6815/75755945","heartbeat_front_addr":"172.21.15.114:6813/75755945","state":["exists","up"]},{"osd":5,"uuid":"51922f99-e8d3-461b-8615-e9bfdcbf5ed3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":1918493790},{"type":"v1","addr":"172.21.15.177:6809","nonce":1918493790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":1918493790},{"type":"v1","addr":"172.21.15.177:6811","nonce":1918493790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":1918493790},{"type":"v1","addr":"172.21.15.177:6815","nonce":1918493790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":1918493790},{"type":"v1","addr":"172.21.15.177:6813","nonce":1918493790}]},"public_addr":"172.21.15.177:6809/1918493790","cluster_addr":"172.21.15.177:6811/1918493790","heartbeat_back_addr":"172.21.15.177:6815/1918493790","heartbeat_front_addr":"172.21.15.177:6813/1918493790","state":["exists","up"]},{"osd":6,"uuid":"06dffb32-05eb-43b3-852e-915c5cc7f25a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2892389563},{"type":"v1","addr":"172.21.15.97:6819","nonce":2892389563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2892389563},{"type":"v1","addr":"172.21.15.97:6821","nonce":2892389563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2892389563},{"type":"v1","addr":"172.21.15.97:6825","nonce":2892389563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2892389563},{"type":"v1","addr":"172.21.15.97:6823","nonce":2892389563}]},"public_addr":"172.21.15.97:6819/2892389563","cluster_addr":"172.21.15.97:6821/2892389563","heartbeat_back_addr":"172.21.15.97:6825/2892389563","heartbeat_front_addr":"172.21.15.97:6823/2892389563","state":["exists","up"]},{"osd":7,"uuid":"e68e3519-9c1b-43c5-8e06-5385248b19df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":2445491153},{"type":"v1","addr":"172.21.15.114:6817","nonce":2445491153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":2445491153},{"type":"v1","addr":"172.21.15.114:6819","nonce":2445491153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2445491153},{"type":"v1","addr":"172.21.15.114:6823","nonce":2445491153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":2445491153},{"type":"v1","addr":"172.21.15.114:6821","nonce":2445491153}]},"public_addr":"172.21.15.114:6817/2445491153","cluster_addr":"172.21.15.114:6819/2445491153","heartbeat_back_addr":"172.21.15.114:6823/2445491153","heartbeat_front_addr":"172.21.15.114:6821/2445491153","state":["exists","up"]},{"osd":8,"uuid":"31ac592e-1349-49ee-9af9-8ad85d682978","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":2051134191},{"type":"v1","addr":"172.21.15.177:6817","nonce":2051134191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":2051134191},{"type":"v1","addr":"172.21.15.177:6819","nonce":2051134191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":2051134191},{"type":"v1","addr":"172.21.15.177:6823","nonce":2051134191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":2051134191},{"type":"v1","addr":"172.21.15.177:6821","nonce":2051134191}]},"public_addr":"172.21.15.177:6817/2051134191","cluster_addr":"172.21.15.177:6819/2051134191","heartbeat_back_addr":"172.21.15.177:6823/2051134191","heartbeat_front_addr":"172.21.15.177:6821/2051134191","state":["exists","up"]},{"osd":9,"uuid":"0171086a-dd41-4eb4-a389-16f0c46a64db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":1578881271},{"type":"v1","addr":"172.21.15.97:6827","nonce":1578881271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1578881271},{"type":"v1","addr":"172.21.15.97:6829","nonce":1578881271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1578881271},{"type":"v1","addr":"172.21.15.97:6833","nonce":1578881271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1578881271},{"type":"v1","addr":"172.21.15.97:6831","nonce":1578881271}]},"public_addr":"172.21.15.97:6827/1578881271","cluster_addr":"172.21.15.97:6829/1578881271","heartbeat_back_addr":"172.21.15.97:6833/1578881271","heartbeat_front_addr":"172.21.15.97:6831/1578881271","state":["exists","up"]},{"osd":10,"uuid":"64754e31-b124-4601-ad98-2f70b1dd5359","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":2893111461},{"type":"v1","addr":"172.21.15.114:6825","nonce":2893111461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":2893111461},{"type":"v1","addr":"172.21.15.114:6827","nonce":2893111461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2893111461},{"type":"v1","addr":"172.21.15.114:6831","nonce":2893111461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":2893111461},{"type":"v1","addr":"172.21.15.114:6829","nonce":2893111461}]},"public_addr":"172.21.15.114:6825/2893111461","cluster_addr":"172.21.15.114:6827/2893111461","heartbeat_back_addr":"172.21.15.114:6831/2893111461","heartbeat_front_addr":"172.21.15.114:6829/2893111461","state":["exists","up"]},{"osd":11,"uuid":"9ffc752c-adf7-4037-89d3-535787d7475d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":85,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":2333030391},{"type":"v1","addr":"172.21.15.177:6825","nonce":2333030391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":2333030391},{"type":"v1","addr":"172.21.15.177:6827","nonce":2333030391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":2333030391},{"type":"v1","addr":"172.21.15.177:6831","nonce":2333030391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":2333030391},{"type":"v1","addr":"172.21.15.177:6829","nonce":2333030391}]},"public_addr":"172.21.15.177:6825/2333030391","cluster_addr":"172.21.15.177:6827/2333030391","heartbeat_back_addr":"172.21.15.177:6831/2333030391","heartbeat_front_addr":"172.21.15.177:6829/2333030391","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:46:45.143519+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:47:18.656947+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:47:51.316345+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:48:24.575512+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:48:59.113555+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:49:32.103865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:50:06.615539+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:50:41.578341+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:51:15.254011+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:51:49.099165+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:52:24.544821+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:52:58.838484+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/359646":"2024-03-28T10:44:39.889550+0000","172.21.15.97:6801/98682155":"2024-03-28T10:44:06.019075+0000","172.21.15.97:6801/1685921853":"2024-03-28T10:44:39.889550+0000","172.21.15.97:6800/98682155":"2024-03-28T10:44:06.019075+0000","172.21.15.97:0/107690680":"2024-03-28T10:44:06.019075+0000","172.21.15.97:6800/1685921853":"2024-03-28T10:44:39.889550+0000","172.21.15.97:0/437043413":"2024-03-28T10:44:39.889550+0000","172.21.15.97:0/2319706992":"2024-03-28T10:44:39.889550+0000","172.21.15.97:0/3911532654":"2024-03-28T10:44:06.019075+0000","172.21.15.97:0/3347321845":"2024-03-28T10:44:06.019075+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-27T10:54:05.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.681+0000 7f2dd8e00640 1 -- 172.21.15.97:0/2397407242 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f2db80a3690 msgr2=0x7f2db80a5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:05.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.681+0000 7f2dd8e00640 1 --2- 172.21.15.97:0/2397407242 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f2db80a3690 0x7f2db80a5b50 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2dc002c240 tx=0x7f2dc0005fd0 comp rx=0 tx=0).stop 2024-03-27T10:54:05.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.681+0000 7f2dd8e00640 1 -- 172.21.15.97:0/2397407242 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2dd41023e0 msgr2=0x7f2dd4199db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:05.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.681+0000 7f2dd8e00640 1 --2- 172.21.15.97:0/2397407242 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2dd41023e0 0x7f2dd4199db0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2dc400e9f0 tx=0x7f2dc400eeb0 comp rx=0 tx=0).stop 2024-03-27T10:54:05.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.681+0000 7f2dd8e00640 1 -- 172.21.15.97:0/2397407242 shutdown_connections 2024-03-27T10:54:05.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.681+0000 7f2dd8e00640 1 --2- 172.21.15.97:0/2397407242 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2dd4107640 0x7f2dd41a1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:05.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.681+0000 7f2dd8e00640 1 --2- 172.21.15.97:0/2397407242 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f2db80a3690 0x7f2db80a5b50 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:05.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.681+0000 7f2dd8e00640 1 --2- 172.21.15.97:0/2397407242 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2dd4104d10 0x7f2dd419a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:05.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.681+0000 7f2dd8e00640 1 --2- 172.21.15.97:0/2397407242 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2dd41023e0 0x7f2dd4199db0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:05.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.681+0000 7f2dd8e00640 1 -- 172.21.15.97:0/2397407242 >> 172.21.15.97:0/2397407242 conn(0x7f2dd40fbdd0 msgr2=0x7f2dd4101ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:54:05.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.682+0000 7f2dd8e00640 1 -- 172.21.15.97:0/2397407242 shutdown_connections 2024-03-27T10:54:05.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.682+0000 7f2dd8e00640 1 -- 172.21.15.97:0/2397407242 wait complete. 2024-03-27T10:54:05.673 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-27T10:54:05.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.888+0000 7fa50ae00640 1 -- 172.21.15.97:0/2419645550 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa504102620 msgr2=0x7fa504108eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:05.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.888+0000 7fa50ae00640 1 --2- 172.21.15.97:0/2419645550 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa504102620 0x7fa504108eb0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc009ca0 tx=0x7fa4fc02c0f0 comp rx=0 tx=0).stop 2024-03-27T10:54:05.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.888+0000 7fa50ae00640 1 -- 172.21.15.97:0/2419645550 shutdown_connections 2024-03-27T10:54:05.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.888+0000 7fa50ae00640 1 --2- 172.21.15.97:0/2419645550 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa504102620 0x7fa504108eb0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:05.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.888+0000 7fa50ae00640 1 --2- 172.21.15.97:0/2419645550 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa504101c60 0x7fa5041020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:05.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.888+0000 7fa50ae00640 1 --2- 172.21.15.97:0/2419645550 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa504100a70 0x7fa504100e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:05.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.888+0000 7fa50ae00640 1 -- 172.21.15.97:0/2419645550 >> 172.21.15.97:0/2419645550 conn(0x7fa5040fc020 msgr2=0x7fa5040fe440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:54:05.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.888+0000 7fa50ae00640 1 -- 172.21.15.97:0/2419645550 shutdown_connections 2024-03-27T10:54:05.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.888+0000 7fa50ae00640 1 -- 172.21.15.97:0/2419645550 wait complete. 2024-03-27T10:54:05.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.889+0000 7fa50ae00640 1 Processor -- start 2024-03-27T10:54:05.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.889+0000 7fa50ae00640 1 -- start start 2024-03-27T10:54:05.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.889+0000 7fa50ae00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa504100a70 0x7fa504199ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:05.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.889+0000 7fa50ae00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa504101c60 0x7fa50419a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:05.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.889+0000 7fa50ae00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa504102620 0x7fa5041a1490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:05.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.889+0000 7fa50ae00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa50410c5a0 con 0x7fa504102620 2024-03-27T10:54:05.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.889+0000 7fa50ae00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa50410c420 con 0x7fa504101c60 2024-03-27T10:54:05.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.889+0000 7fa50ae00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa50410c720 con 0x7fa504100a70 2024-03-27T10:54:05.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.890+0000 7fa508e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa504102620 0x7fa5041a1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:05.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.890+0000 7fa508e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa504102620 0x7fa5041a1490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57038/0 (socket says 172.21.15.97:57038) 2024-03-27T10:54:05.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.890+0000 7fa508e00640 1 -- 172.21.15.97:0/774566865 learned_addr learned my addr 172.21.15.97:0/774566865 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:54:05.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.890+0000 7fa503e00640 1 --2- 172.21.15.97:0/774566865 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa504100a70 0x7fa504199ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:05.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.890+0000 7fa503400640 1 --2- 172.21.15.97:0/774566865 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa504101c60 0x7fa50419a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:05.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.890+0000 7fa508e00640 1 -- 172.21.15.97:0/774566865 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa504100a70 msgr2=0x7fa504199ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:05.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.890+0000 7fa508e00640 1 --2- 172.21.15.97:0/774566865 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa504100a70 0x7fa504199ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:05.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.890+0000 7fa508e00640 1 -- 172.21.15.97:0/774566865 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa504101c60 msgr2=0x7fa50419a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:05.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.890+0000 7fa508e00640 1 --2- 172.21.15.97:0/774566865 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa504101c60 0x7fa50419a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:05.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.890+0000 7fa508e00640 1 -- 172.21.15.97:0/774566865 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5041a1b00 con 0x7fa504102620 2024-03-27T10:54:05.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.890+0000 7fa503400640 1 --2- 172.21.15.97:0/774566865 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa504101c60 0x7fa50419a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:54:05.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.890+0000 7fa508e00640 1 --2- 172.21.15.97:0/774566865 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa504102620 0x7fa5041a1490 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc032380 tx=0x7fa4fc032820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:54:05.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.891+0000 7fa500c00640 1 -- 172.21.15.97:0/774566865 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4fc041020 con 0x7fa504102620 2024-03-27T10:54:05.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.891+0000 7fa50ae00640 1 -- 172.21.15.97:0/774566865 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5041a1d90 con 0x7fa504102620 2024-03-27T10:54:05.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.891+0000 7fa500c00640 1 -- 172.21.15.97:0/774566865 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa4fc003e20 con 0x7fa504102620 2024-03-27T10:54:05.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.891+0000 7fa500c00640 1 -- 172.21.15.97:0/774566865 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4fc039040 con 0x7fa504102620 2024-03-27T10:54:05.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.891+0000 7fa50ae00640 1 -- 172.21.15.97:0/774566865 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5041a2220 con 0x7fa504102620 2024-03-27T10:54:05.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.892+0000 7fa50ae00640 1 -- 172.21.15.97:0/774566865 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5041a2450 con 0x7fa504102620 2024-03-27T10:54:05.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.894+0000 7fa500c00640 1 -- 172.21.15.97:0/774566865 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138370+0+0 (secure 0 0 0) 0x7fa4fc044020 con 0x7fa504102620 2024-03-27T10:54:05.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.895+0000 7fa500c00640 1 --2- 172.21.15.97:0/774566865 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fa4e80a3610 0x7fa4e80a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:05.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.896+0000 7fa500c00640 1 -- 172.21.15.97:0/774566865 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7fa4fc0f18c0 con 0x7fa504102620 2024-03-27T10:54:05.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.896+0000 7fa503e00640 1 --2- 172.21.15.97:0/774566865 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fa4e80a3610 0x7fa4e80a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:05.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.896+0000 7fa503e00640 1 --2- 172.21.15.97:0/774566865 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fa4e80a3610 0x7fa4e80a5ad0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa4f4005fd0 tx=0x7fa4f4005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:54:05.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:05.900+0000 7fa500c00640 1 -- 172.21.15.97:0/774566865 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7fa4fc04b070 con 0x7fa504102620 2024-03-27T10:54:06.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:06.113+0000 7fa50ae00640 1 -- 172.21.15.97:0/774566865 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa5041a27b0 con 0x7fa504102620 2024-03-27T10:54:06.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:06.115+0000 7fa500c00640 1 -- 172.21.15.97:0/774566865 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22211 (secure 0 0 0) 0x7fa4fc0be5e0 con 0x7fa504102620 2024-03-27T10:54:06.088 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T10:54:06.088 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":90,"fsid":"adca88c4-ec26-11ee-95d0-87774f69a715","created":"2024-03-27T10:43:24.680694+0000","modified":"2024-03-27T10:54:06.066477+0000","last_up_change":"2024-03-27T10:53:00.552513+0000","last_in_change":"2024-03-27T10:52:36.302986+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-27T10:47:54.052810+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-27T10:54:02.845060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-27T10:54:03.474151+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b634e5f1-6118-4e2f-95e9-830db390f8d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1746812080},{"type":"v1","addr":"172.21.15.97:6803","nonce":1746812080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1746812080},{"type":"v1","addr":"172.21.15.97:6805","nonce":1746812080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":1746812080},{"type":"v1","addr":"172.21.15.97:6809","nonce":1746812080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1746812080},{"type":"v1","addr":"172.21.15.97:6807","nonce":1746812080}]},"public_addr":"172.21.15.97:6803/1746812080","cluster_addr":"172.21.15.97:6805/1746812080","heartbeat_back_addr":"172.21.15.97:6809/1746812080","heartbeat_front_addr":"172.21.15.97:6807/1746812080","state":["exists","up"]},{"osd":1,"uuid":"441e2770-3284-4388-b0de-c56eb8a373d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":3858821264},{"type":"v1","addr":"172.21.15.114:6801","nonce":3858821264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":3858821264},{"type":"v1","addr":"172.21.15.114:6803","nonce":3858821264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":3858821264},{"type":"v1","addr":"172.21.15.114:6807","nonce":3858821264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":3858821264},{"type":"v1","addr":"172.21.15.114:6805","nonce":3858821264}]},"public_addr":"172.21.15.114:6801/3858821264","cluster_addr":"172.21.15.114:6803/3858821264","heartbeat_back_addr":"172.21.15.114:6807/3858821264","heartbeat_front_addr":"172.21.15.114:6805/3858821264","state":["exists","up"]},{"osd":2,"uuid":"00108864-1aa6-48d6-8398-e8eb79fda123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":124240658},{"type":"v1","addr":"172.21.15.177:6801","nonce":124240658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":124240658},{"type":"v1","addr":"172.21.15.177:6803","nonce":124240658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":124240658},{"type":"v1","addr":"172.21.15.177:6807","nonce":124240658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":124240658},{"type":"v1","addr":"172.21.15.177:6805","nonce":124240658}]},"public_addr":"172.21.15.177:6801/124240658","cluster_addr":"172.21.15.177:6803/124240658","heartbeat_back_addr":"172.21.15.177:6807/124240658","heartbeat_front_addr":"172.21.15.177:6805/124240658","state":["exists","up"]},{"osd":3,"uuid":"1860b690-e2dc-4977-88d7-d1aacf7022ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2478179311},{"type":"v1","addr":"172.21.15.97:6811","nonce":2478179311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2478179311},{"type":"v1","addr":"172.21.15.97:6813","nonce":2478179311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":2478179311},{"type":"v1","addr":"172.21.15.97:6817","nonce":2478179311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2478179311},{"type":"v1","addr":"172.21.15.97:6815","nonce":2478179311}]},"public_addr":"172.21.15.97:6811/2478179311","cluster_addr":"172.21.15.97:6813/2478179311","heartbeat_back_addr":"172.21.15.97:6817/2478179311","heartbeat_front_addr":"172.21.15.97:6815/2478179311","state":["exists","up"]},{"osd":4,"uuid":"9245a3a4-7d49-4fdb-9d55-112e43177050","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":75755945},{"type":"v1","addr":"172.21.15.114:6809","nonce":75755945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":75755945},{"type":"v1","addr":"172.21.15.114:6811","nonce":75755945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":75755945},{"type":"v1","addr":"172.21.15.114:6815","nonce":75755945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":75755945},{"type":"v1","addr":"172.21.15.114:6813","nonce":75755945}]},"public_addr":"172.21.15.114:6809/75755945","cluster_addr":"172.21.15.114:6811/75755945","heartbeat_back_addr":"172.21.15.114:6815/75755945","heartbeat_front_addr":"172.21.15.114:6813/75755945","state":["exists","up"]},{"osd":5,"uuid":"51922f99-e8d3-461b-8615-e9bfdcbf5ed3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":1918493790},{"type":"v1","addr":"172.21.15.177:6809","nonce":1918493790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":1918493790},{"type":"v1","addr":"172.21.15.177:6811","nonce":1918493790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":1918493790},{"type":"v1","addr":"172.21.15.177:6815","nonce":1918493790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":1918493790},{"type":"v1","addr":"172.21.15.177:6813","nonce":1918493790}]},"public_addr":"172.21.15.177:6809/1918493790","cluster_addr":"172.21.15.177:6811/1918493790","heartbeat_back_addr":"172.21.15.177:6815/1918493790","heartbeat_front_addr":"172.21.15.177:6813/1918493790","state":["exists","up"]},{"osd":6,"uuid":"06dffb32-05eb-43b3-852e-915c5cc7f25a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2892389563},{"type":"v1","addr":"172.21.15.97:6819","nonce":2892389563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2892389563},{"type":"v1","addr":"172.21.15.97:6821","nonce":2892389563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2892389563},{"type":"v1","addr":"172.21.15.97:6825","nonce":2892389563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2892389563},{"type":"v1","addr":"172.21.15.97:6823","nonce":2892389563}]},"public_addr":"172.21.15.97:6819/2892389563","cluster_addr":"172.21.15.97:6821/2892389563","heartbeat_back_addr":"172.21.15.97:6825/2892389563","heartbeat_front_addr":"172.21.15.97:6823/2892389563","state":["exists","up"]},{"osd":7,"uuid":"e68e3519-9c1b-43c5-8e06-5385248b19df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":2445491153},{"type":"v1","addr":"172.21.15.114:6817","nonce":2445491153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":2445491153},{"type":"v1","addr":"172.21.15.114:6819","nonce":2445491153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2445491153},{"type":"v1","addr":"172.21.15.114:6823","nonce":2445491153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":2445491153},{"type":"v1","addr":"172.21.15.114:6821","nonce":2445491153}]},"public_addr":"172.21.15.114:6817/2445491153","cluster_addr":"172.21.15.114:6819/2445491153","heartbeat_back_addr":"172.21.15.114:6823/2445491153","heartbeat_front_addr":"172.21.15.114:6821/2445491153","state":["exists","up"]},{"osd":8,"uuid":"31ac592e-1349-49ee-9af9-8ad85d682978","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":2051134191},{"type":"v1","addr":"172.21.15.177:6817","nonce":2051134191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":2051134191},{"type":"v1","addr":"172.21.15.177:6819","nonce":2051134191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":2051134191},{"type":"v1","addr":"172.21.15.177:6823","nonce":2051134191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":2051134191},{"type":"v1","addr":"172.21.15.177:6821","nonce":2051134191}]},"public_addr":"172.21.15.177:6817/2051134191","cluster_addr":"172.21.15.177:6819/2051134191","heartbeat_back_addr":"172.21.15.177:6823/2051134191","heartbeat_front_addr":"172.21.15.177:6821/2051134191","state":["exists","up"]},{"osd":9,"uuid":"0171086a-dd41-4eb4-a389-16f0c46a64db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":1578881271},{"type":"v1","addr":"172.21.15.97:6827","nonce":1578881271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1578881271},{"type":"v1","addr":"172.21.15.97:6829","nonce":1578881271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1578881271},{"type":"v1","addr":"172.21.15.97:6833","nonce":1578881271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1578881271},{"type":"v1","addr":"172.21.15.97:6831","nonce":1578881271}]},"public_addr":"172.21.15.97:6827/1578881271","cluster_addr":"172.21.15.97:6829/1578881271","heartbeat_back_addr":"172.21.15.97:6833/1578881271","heartbeat_front_addr":"172.21.15.97:6831/1578881271","state":["exists","up"]},{"osd":10,"uuid":"64754e31-b124-4601-ad98-2f70b1dd5359","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":2893111461},{"type":"v1","addr":"172.21.15.114:6825","nonce":2893111461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":2893111461},{"type":"v1","addr":"172.21.15.114:6827","nonce":2893111461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2893111461},{"type":"v1","addr":"172.21.15.114:6831","nonce":2893111461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":2893111461},{"type":"v1","addr":"172.21.15.114:6829","nonce":2893111461}]},"public_addr":"172.21.15.114:6825/2893111461","cluster_addr":"172.21.15.114:6827/2893111461","heartbeat_back_addr":"172.21.15.114:6831/2893111461","heartbeat_front_addr":"172.21.15.114:6829/2893111461","state":["exists","up"]},{"osd":11,"uuid":"9ffc752c-adf7-4037-89d3-535787d7475d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":85,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":2333030391},{"type":"v1","addr":"172.21.15.177:6825","nonce":2333030391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":2333030391},{"type":"v1","addr":"172.21.15.177:6827","nonce":2333030391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":2333030391},{"type":"v1","addr":"172.21.15.177:6831","nonce":2333030391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":2333030391},{"type":"v1","addr":"172.21.15.177:6829","nonce":2333030391}]},"public_addr":"172.21.15.177:6825/2333030391","cluster_addr":"172.21.15.177:6827/2333030391","heartbeat_back_addr":"172.21.15.177:6831/2333030391","heartbeat_front_addr":"172.21.15.177:6829/2333030391","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:46:45.143519+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:47:18.656947+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:47:51.316345+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:48:24.575512+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:48:59.113555+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:49:32.103865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:50:06.615539+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:50:41.578341+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:51:15.254011+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:51:49.099165+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:52:24.544821+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:52:58.838484+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/359646":"2024-03-28T10:44:39.889550+0000","172.21.15.97:6801/98682155":"2024-03-28T10:44:06.019075+0000","172.21.15.97:6801/1685921853":"2024-03-28T10:44:39.889550+0000","172.21.15.97:6800/98682155":"2024-03-28T10:44:06.019075+0000","172.21.15.97:0/107690680":"2024-03-28T10:44:06.019075+0000","172.21.15.97:6800/1685921853":"2024-03-28T10:44:39.889550+0000","172.21.15.97:0/437043413":"2024-03-28T10:44:39.889550+0000","172.21.15.97:0/2319706992":"2024-03-28T10:44:39.889550+0000","172.21.15.97:0/3911532654":"2024-03-28T10:44:06.019075+0000","172.21.15.97:0/3347321845":"2024-03-28T10:44:06.019075+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-27T10:54:06.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:06.117+0000 7fa50ae00640 1 -- 172.21.15.97:0/774566865 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fa4e80a3610 msgr2=0x7fa4e80a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:06.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:06.117+0000 7fa50ae00640 1 --2- 172.21.15.97:0/774566865 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fa4e80a3610 0x7fa4e80a5ad0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa4f4005fd0 tx=0x7fa4f4005850 comp rx=0 tx=0).stop 2024-03-27T10:54:06.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:06.117+0000 7fa50ae00640 1 -- 172.21.15.97:0/774566865 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa504102620 msgr2=0x7fa5041a1490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:06.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:06.117+0000 7fa50ae00640 1 --2- 172.21.15.97:0/774566865 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa504102620 0x7fa5041a1490 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc032380 tx=0x7fa4fc032820 comp rx=0 tx=0).stop 2024-03-27T10:54:06.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:06.118+0000 7fa50ae00640 1 -- 172.21.15.97:0/774566865 shutdown_connections 2024-03-27T10:54:06.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:06.118+0000 7fa50ae00640 1 --2- 172.21.15.97:0/774566865 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa504102620 0x7fa5041a1490 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:06.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:06.118+0000 7fa50ae00640 1 --2- 172.21.15.97:0/774566865 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fa4e80a3610 0x7fa4e80a5ad0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:06.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:06.118+0000 7fa50ae00640 1 --2- 172.21.15.97:0/774566865 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa504101c60 0x7fa50419a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:06.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:06.118+0000 7fa50ae00640 1 --2- 172.21.15.97:0/774566865 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa504100a70 0x7fa504199ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:06.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:06.118+0000 7fa50ae00640 1 -- 172.21.15.97:0/774566865 >> 172.21.15.97:0/774566865 conn(0x7fa5040fc020 msgr2=0x7fa5040fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:54:06.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:06.118+0000 7fa50ae00640 1 -- 172.21.15.97:0/774566865 shutdown_connections 2024-03-27T10:54:06.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:06.118+0000 7fa50ae00640 1 -- 172.21.15.97:0/774566865 wait complete. 2024-03-27T10:54:06.107 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-27T10:54:06.267 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:06 smithi114 ceph-mon[35338]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-27T10:54:06.267 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:06 smithi114 ceph-mon[35338]: osdmap e89: 12 total, 12 up, 12 in 2024-03-27T10:54:06.267 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:06 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/276538256' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-27T10:54:06.268 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:06 smithi114 ceph-mon[35338]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-27T10:54:06.268 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:06 smithi114 ceph-mon[35338]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-27T10:54:06.268 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:06 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/2397407242' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T10:54:06.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:06.321+0000 7f7a2fa00640 1 -- 172.21.15.97:0/1003805936 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a28105940 msgr2=0x7f7a2810c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:06.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:06.321+0000 7f7a2fa00640 1 --2- 172.21.15.97:0/1003805936 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a28105940 0x7f7a2810c1d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7a20009ca0 tx=0x7f7a2002c0f0 comp rx=0 tx=0).stop 2024-03-27T10:54:06.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:06.321+0000 7f7a2fa00640 1 -- 172.21.15.97:0/1003805936 shutdown_connections 2024-03-27T10:54:06.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:06.322+0000 7f7a2fa00640 1 --2- 172.21.15.97:0/1003805936 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a28105940 0x7f7a2810c1d0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:06.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:06.322+0000 7f7a2fa00640 1 --2- 172.21.15.97:0/1003805936 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7a28104f80 0x7f7a28105400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:06.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:06.322+0000 7f7a2fa00640 1 --2- 172.21.15.97:0/1003805936 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7a28103d90 0x7f7a281041b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:06.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:06.322+0000 7f7a2fa00640 1 -- 172.21.15.97:0/1003805936 >> 172.21.15.97:0/1003805936 conn(0x7f7a280ff300 msgr2=0x7f7a28101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:54:06.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:06.322+0000 7f7a2fa00640 1 -- 172.21.15.97:0/1003805936 shutdown_connections 2024-03-27T10:54:06.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:06.322+0000 7f7a2fa00640 1 -- 172.21.15.97:0/1003805936 wait complete. 2024-03-27T10:54:06.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:06.323+0000 7f7a2fa00640 1 Processor -- start 2024-03-27T10:54:06.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:06.323+0000 7f7a2fa00640 1 -- start start 2024-03-27T10:54:06.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:06.323+0000 7f7a2fa00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7a28103d90 0x7f7a28199f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:06.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:06.323+0000 7f7a2fa00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7a28104f80 0x7f7a2819a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:06.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:06.323+0000 7f7a2fa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a28105940 0x7f7a281a1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:06.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:06.323+0000 7f7a2fa00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7a2810f8a0 con 0x7f7a28105940 2024-03-27T10:54:06.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:06.323+0000 7f7a2fa00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f7a2810f720 con 0x7f7a28104f80 2024-03-27T10:54:06.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:06.323+0000 7f7a2fa00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f7a2810fa20 con 0x7f7a28103d90 2024-03-27T10:54:06.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:06.323+0000 7f7a2da00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a28105940 0x7f7a281a1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:06.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:06.323+0000 7f7a2da00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a28105940 0x7f7a281a1520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57064/0 (socket says 172.21.15.97:57064) 2024-03-27T10:54:06.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:06.323+0000 7f7a2da00640 1 -- 172.21.15.97:0/1963960333 learned_addr learned my addr 172.21.15.97:0/1963960333 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:54:06.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:06.324+0000 7f7a27e00640 1 --2- 172.21.15.97:0/1963960333 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7a28104f80 0x7f7a2819a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:06.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:06.324+0000 7f7a2d000640 1 --2- 172.21.15.97:0/1963960333 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7a28103d90 0x7f7a28199f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:06.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:06.324+0000 7f7a2da00640 1 -- 172.21.15.97:0/1963960333 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7a28103d90 msgr2=0x7f7a28199f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:06.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:06.324+0000 7f7a2da00640 1 --2- 172.21.15.97:0/1963960333 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7a28103d90 0x7f7a28199f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:06.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:06.324+0000 7f7a2da00640 1 -- 172.21.15.97:0/1963960333 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7a28104f80 msgr2=0x7f7a2819a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:06.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:06.324+0000 7f7a2da00640 1 --2- 172.21.15.97:0/1963960333 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7a28104f80 0x7f7a2819a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:06.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:06.324+0000 7f7a2da00640 1 -- 172.21.15.97:0/1963960333 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a281a1b90 con 0x7f7a28105940 2024-03-27T10:54:06.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:06.324+0000 7f7a2da00640 1 --2- 172.21.15.97:0/1963960333 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a28105940 0x7f7a281a1520 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7a20032380 tx=0x7f7a20032820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:54:06.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:06.324+0000 7f7a25600640 1 -- 172.21.15.97:0/1963960333 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a20041020 con 0x7f7a28105940 2024-03-27T10:54:06.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:06.324+0000 7f7a25600640 1 -- 172.21.15.97:0/1963960333 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7a20003e20 con 0x7f7a28105940 2024-03-27T10:54:06.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:06.324+0000 7f7a25600640 1 -- 172.21.15.97:0/1963960333 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a20039040 con 0x7f7a28105940 2024-03-27T10:54:06.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:06.324+0000 7f7a2fa00640 1 -- 172.21.15.97:0/1963960333 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a281a1e20 con 0x7f7a28105940 2024-03-27T10:54:06.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:06.325+0000 7f7a2fa00640 1 -- 172.21.15.97:0/1963960333 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a281a22e0 con 0x7f7a28105940 2024-03-27T10:54:06.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:06.326+0000 7f7a2fa00640 1 -- 172.21.15.97:0/1963960333 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a2810afe0 con 0x7f7a28105940 2024-03-27T10:54:06.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:06.327+0000 7f7a25600640 1 -- 172.21.15.97:0/1963960333 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138370+0+0 (secure 0 0 0) 0x7f7a20044020 con 0x7f7a28105940 2024-03-27T10:54:06.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:06.329+0000 7f7a25600640 1 --2- 172.21.15.97:0/1963960333 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f7a100a3690 0x7f7a100a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:06.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:06.329+0000 7f7a25600640 1 -- 172.21.15.97:0/1963960333 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 7979+0+0 (secure 0 0 0) 0x7f7a200f1e70 con 0x7f7a28105940 2024-03-27T10:54:06.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:06.329+0000 7f7a2d000640 1 --2- 172.21.15.97:0/1963960333 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f7a100a3690 0x7f7a100a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:06.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:06.330+0000 7f7a2d000640 1 --2- 172.21.15.97:0/1963960333 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f7a100a3690 0x7f7a100a5b50 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7a28104de0 tx=0x7f7a1c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:54:06.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:06.333+0000 7f7a25600640 1 -- 172.21.15.97:0/1963960333 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f7a2004b020 con 0x7f7a28105940 2024-03-27T10:54:06.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:06 smithi097 ceph-mon[31859]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-27T10:54:06.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:06 smithi097 ceph-mon[31859]: osdmap e89: 12 total, 12 up, 12 in 2024-03-27T10:54:06.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:06 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/276538256' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-27T10:54:06.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:06 smithi097 ceph-mon[31859]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-27T10:54:06.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:06 smithi097 ceph-mon[31859]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-27T10:54:06.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:06 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/2397407242' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T10:54:06.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:06 smithi177 ceph-mon[35294]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-27T10:54:06.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:06 smithi177 ceph-mon[35294]: osdmap e89: 12 total, 12 up, 12 in 2024-03-27T10:54:06.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:06 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/276538256' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-27T10:54:06.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:06 smithi177 ceph-mon[35294]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-27T10:54:06.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:06 smithi177 ceph-mon[35294]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-27T10:54:06.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:06 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/2397407242' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T10:54:06.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:06.588+0000 7f7a2fa00640 1 -- 172.21.15.97:0/1963960333 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f7a280627f0 con 0x7f7a28105940 2024-03-27T10:54:07.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:07.246+0000 7f7a25600640 1 -- 172.21.15.97:0/1963960333 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7f7a2003b2f0 con 0x7f7a28105940 2024-03-27T10:54:07.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:07.249+0000 7f7a2fa00640 1 -- 172.21.15.97:0/1963960333 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f7a100a3690 msgr2=0x7f7a100a5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:07.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:07.249+0000 7f7a2fa00640 1 --2- 172.21.15.97:0/1963960333 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f7a100a3690 0x7f7a100a5b50 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7a28104de0 tx=0x7f7a1c005f90 comp rx=0 tx=0).stop 2024-03-27T10:54:07.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:07.249+0000 7f7a2fa00640 1 -- 172.21.15.97:0/1963960333 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a28105940 msgr2=0x7f7a281a1520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:07.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:07.249+0000 7f7a2fa00640 1 --2- 172.21.15.97:0/1963960333 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a28105940 0x7f7a281a1520 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7a20032380 tx=0x7f7a20032820 comp rx=0 tx=0).stop 2024-03-27T10:54:07.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:07.249+0000 7f7a2fa00640 1 -- 172.21.15.97:0/1963960333 shutdown_connections 2024-03-27T10:54:07.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:07.249+0000 7f7a2fa00640 1 --2- 172.21.15.97:0/1963960333 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a28105940 0x7f7a281a1520 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:07.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:07.249+0000 7f7a2fa00640 1 --2- 172.21.15.97:0/1963960333 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f7a100a3690 0x7f7a100a5b50 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:07.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:07.249+0000 7f7a2fa00640 1 --2- 172.21.15.97:0/1963960333 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7a28104f80 0x7f7a2819a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:07.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:07.249+0000 7f7a2fa00640 1 --2- 172.21.15.97:0/1963960333 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7a28103d90 0x7f7a28199f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:07.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:07.249+0000 7f7a2fa00640 1 -- 172.21.15.97:0/1963960333 >> 172.21.15.97:0/1963960333 conn(0x7f7a280ff300 msgr2=0x7f7a281081b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:54:07.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:07.249+0000 7f7a2fa00640 1 -- 172.21.15.97:0/1963960333 shutdown_connections 2024-03-27T10:54:07.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:07.249+0000 7f7a2fa00640 1 -- 172.21.15.97:0/1963960333 wait complete. 2024-03-27T10:54:07.241 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-03-27T10:54:07.242 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2024-03-27T10:54:07.307 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:07 smithi097 ceph-mon[31859]: pgmap v362: 97 pgs: 12 active+clean, 33 creating+peering, 52 unknown; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T10:54:07.307 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:07 smithi097 ceph-mon[31859]: osdmap e90: 12 total, 12 up, 12 in 2024-03-27T10:54:07.307 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:07 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/774566865' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T10:54:07.307 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:07 smithi097 ceph-mon[31859]: mds.? [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] up:active 2024-03-27T10:54:07.307 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:07 smithi097 ceph-mon[31859]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-27T10:54:07.308 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:07 smithi097 ceph-mon[31859]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-27T10:54:07.308 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:07 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/1963960333' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-27T10:54:07.308 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:07 smithi097 ceph-mon[31859]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-03-27T10:54:07.308 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:07 smithi097 ceph-mon[31859]: Cluster is now healthy 2024-03-27T10:54:07.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:07 smithi177 ceph-mon[35294]: pgmap v362: 97 pgs: 12 active+clean, 33 creating+peering, 52 unknown; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T10:54:07.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:07 smithi177 ceph-mon[35294]: osdmap e90: 12 total, 12 up, 12 in 2024-03-27T10:54:07.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:07 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/774566865' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T10:54:07.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:07 smithi177 ceph-mon[35294]: mds.? [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] up:active 2024-03-27T10:54:07.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:07 smithi177 ceph-mon[35294]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-27T10:54:07.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:07 smithi177 ceph-mon[35294]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-27T10:54:07.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:07 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/1963960333' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-27T10:54:07.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:07 smithi177 ceph-mon[35294]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-03-27T10:54:07.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:07 smithi177 ceph-mon[35294]: Cluster is now healthy 2024-03-27T10:54:07.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:07.488+0000 7fd247a00640 1 -- 172.21.15.97:0/41691774 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd240104f80 msgr2=0x7fd240105400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:07.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:07.488+0000 7fd247a00640 1 --2- 172.21.15.97:0/41691774 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd240104f80 0x7fd240105400 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd228009900 tx=0x7fd22802bc30 comp rx=0 tx=0).stop 2024-03-27T10:54:07.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:07.488+0000 7fd247a00640 1 -- 172.21.15.97:0/41691774 shutdown_connections 2024-03-27T10:54:07.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:07.488+0000 7fd247a00640 1 --2- 172.21.15.97:0/41691774 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd240105940 0x7fd24010c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:07.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:07.488+0000 7fd247a00640 1 --2- 172.21.15.97:0/41691774 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd240104f80 0x7fd240105400 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:07.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:07.488+0000 7fd247a00640 1 --2- 172.21.15.97:0/41691774 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd240103d90 0x7fd2401041b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:07.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:07.488+0000 7fd247a00640 1 -- 172.21.15.97:0/41691774 >> 172.21.15.97:0/41691774 conn(0x7fd2400ff300 msgr2=0x7fd240101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:54:07.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:07.488+0000 7fd247a00640 1 -- 172.21.15.97:0/41691774 shutdown_connections 2024-03-27T10:54:07.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:07.488+0000 7fd247a00640 1 -- 172.21.15.97:0/41691774 wait complete. 2024-03-27T10:54:07.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:07.489+0000 7fd247a00640 1 Processor -- start 2024-03-27T10:54:07.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:07.489+0000 7fd247a00640 1 -- start start 2024-03-27T10:54:07.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:07.489+0000 7fd247a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd240103d90 0x7fd240199fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:07.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:07.489+0000 7fd247a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd240104f80 0x7fd24019a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:07.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:07.489+0000 7fd247a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd240105940 0x7fd2401a1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:07.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:07.489+0000 7fd247a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd24010f8c0 con 0x7fd240105940 2024-03-27T10:54:07.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:07.489+0000 7fd247a00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd24010f740 con 0x7fd240103d90 2024-03-27T10:54:07.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:07.489+0000 7fd247a00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd24010fa40 con 0x7fd240104f80 2024-03-27T10:54:07.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:07.490+0000 7fd245a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd240105940 0x7fd2401a1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:07.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:07.490+0000 7fd245a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd240105940 0x7fd2401a1590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57076/0 (socket says 172.21.15.97:57076) 2024-03-27T10:54:07.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:07.490+0000 7fd245a00640 1 -- 172.21.15.97:0/2276738718 learned_addr learned my addr 172.21.15.97:0/2276738718 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:54:07.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:07.490+0000 7fd245000640 1 --2- 172.21.15.97:0/2276738718 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd240103d90 0x7fd240199fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:07.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:07.490+0000 7fd23fe00640 1 --2- 172.21.15.97:0/2276738718 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd240104f80 0x7fd24019a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:07.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:07.490+0000 7fd245a00640 1 -- 172.21.15.97:0/2276738718 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd240104f80 msgr2=0x7fd24019a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:07.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:07.490+0000 7fd245a00640 1 --2- 172.21.15.97:0/2276738718 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd240104f80 0x7fd24019a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:07.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:07.490+0000 7fd245a00640 1 -- 172.21.15.97:0/2276738718 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd240103d90 msgr2=0x7fd240199fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:07.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:07.490+0000 7fd245a00640 1 --2- 172.21.15.97:0/2276738718 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd240103d90 0x7fd240199fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:07.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:07.490+0000 7fd245a00640 1 -- 172.21.15.97:0/2276738718 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2401a1c00 con 0x7fd240105940 2024-03-27T10:54:07.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:07.491+0000 7fd245a00640 1 --2- 172.21.15.97:0/2276738718 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd240105940 0x7fd2401a1590 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd23800eb10 tx=0x7fd23800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:54:07.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:07.492+0000 7fd23d600640 1 -- 172.21.15.97:0/2276738718 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd238004b20 con 0x7fd240105940 2024-03-27T10:54:07.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:07.492+0000 7fd247a00640 1 -- 172.21.15.97:0/2276738718 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2401a1ef0 con 0x7fd240105940 2024-03-27T10:54:07.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:07.492+0000 7fd23d600640 1 -- 172.21.15.97:0/2276738718 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd238004cc0 con 0x7fd240105940 2024-03-27T10:54:07.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:07.492+0000 7fd23d600640 1 -- 172.21.15.97:0/2276738718 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd238010690 con 0x7fd240105940 2024-03-27T10:54:07.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:07.492+0000 7fd247a00640 1 -- 172.21.15.97:0/2276738718 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2401a2400 con 0x7fd240105940 2024-03-27T10:54:07.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:07.493+0000 7fd247a00640 1 -- 172.21.15.97:0/2276738718 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd24010b150 con 0x7fd240105940 2024-03-27T10:54:07.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:07.495+0000 7fd23d600640 1 -- 172.21.15.97:0/2276738718 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138370+0+0 (secure 0 0 0) 0x7fd238005000 con 0x7fd240105940 2024-03-27T10:54:07.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:07.497+0000 7fd23d600640 1 --2- 172.21.15.97:0/2276738718 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fd2240a3610 0x7fd2240a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:07.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:07.497+0000 7fd23d600640 1 -- 172.21.15.97:0/2276738718 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 7979+0+0 (secure 0 0 0) 0x7fd2380d0950 con 0x7fd240105940 2024-03-27T10:54:07.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:07.497+0000 7fd245000640 1 --2- 172.21.15.97:0/2276738718 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fd2240a3610 0x7fd2240a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:07.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:07.498+0000 7fd245000640 1 --2- 172.21.15.97:0/2276738718 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fd2240a3610 0x7fd2240a5ad0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd240104de0 tx=0x7fd230005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:54:07.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:07.499+0000 7fd23d600640 1 -- 172.21.15.97:0/2276738718 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7fd23809d670 con 0x7fd240105940 2024-03-27T10:54:07.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:07 smithi114 ceph-mon[35338]: pgmap v362: 97 pgs: 12 active+clean, 33 creating+peering, 52 unknown; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T10:54:07.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:07 smithi114 ceph-mon[35338]: osdmap e90: 12 total, 12 up, 12 in 2024-03-27T10:54:07.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:07 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/774566865' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T10:54:07.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:07 smithi114 ceph-mon[35338]: mds.? [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] up:active 2024-03-27T10:54:07.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:07 smithi114 ceph-mon[35338]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-27T10:54:07.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:07 smithi114 ceph-mon[35338]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-27T10:54:07.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:07 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/1963960333' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-27T10:54:07.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:07 smithi114 ceph-mon[35338]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-03-27T10:54:07.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:07 smithi114 ceph-mon[35338]: Cluster is now healthy 2024-03-27T10:54:07.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:07.763+0000 7fd247a00640 1 -- 172.21.15.97:0/2276738718 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7fd24002d370 con 0x7fd240105940 2024-03-27T10:54:08.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:08 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/1963960333' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-27T10:54:08.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:08 smithi114 ceph-mon[35338]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-27T10:54:08.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:08 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:54:08.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:08 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:54:08.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:08 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:54:08.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:08 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:54:08.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:08 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:54:08.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:08 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:54:08.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:08 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:54:08.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:08 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:54:08.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:08 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:54:08.519 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:08 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/2276738718' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-03-27T10:54:08.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:08.575+0000 7fd23d600640 1 -- 172.21.15.97:0/2276738718 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7fd23809cf70 con 0x7fd240105940 2024-03-27T10:54:08.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:08.579+0000 7fd247a00640 1 -- 172.21.15.97:0/2276738718 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fd2240a3610 msgr2=0x7fd2240a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:08.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:08.579+0000 7fd247a00640 1 --2- 172.21.15.97:0/2276738718 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fd2240a3610 0x7fd2240a5ad0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd240104de0 tx=0x7fd230005f90 comp rx=0 tx=0).stop 2024-03-27T10:54:08.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:08.579+0000 7fd247a00640 1 -- 172.21.15.97:0/2276738718 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd240105940 msgr2=0x7fd2401a1590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:08.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:08.579+0000 7fd247a00640 1 --2- 172.21.15.97:0/2276738718 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd240105940 0x7fd2401a1590 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd23800eb10 tx=0x7fd23800c2d0 comp rx=0 tx=0).stop 2024-03-27T10:54:08.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:08.579+0000 7fd247a00640 1 -- 172.21.15.97:0/2276738718 shutdown_connections 2024-03-27T10:54:08.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:08.579+0000 7fd247a00640 1 --2- 172.21.15.97:0/2276738718 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd240105940 0x7fd2401a1590 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:08.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:08.579+0000 7fd247a00640 1 --2- 172.21.15.97:0/2276738718 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd240104f80 0x7fd24019a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:08.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:08.579+0000 7fd247a00640 1 --2- 172.21.15.97:0/2276738718 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fd2240a3610 0x7fd2240a5ad0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:08.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:08.579+0000 7fd247a00640 1 --2- 172.21.15.97:0/2276738718 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd240103d90 0x7fd240199fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:08.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:08.579+0000 7fd247a00640 1 -- 172.21.15.97:0/2276738718 >> 172.21.15.97:0/2276738718 conn(0x7fd2400ff300 msgr2=0x7fd240108320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:54:08.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:08.579+0000 7fd247a00640 1 -- 172.21.15.97:0/2276738718 shutdown_connections 2024-03-27T10:54:08.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:08.579+0000 7fd247a00640 1 -- 172.21.15.97:0/2276738718 wait complete. 2024-03-27T10:54:08.571 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:08 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/1963960333' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-27T10:54:08.571 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:08 smithi097 ceph-mon[31859]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-27T10:54:08.571 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:08 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:54:08.571 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:08 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:54:08.571 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:08 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:54:08.571 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:08 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:54:08.571 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:08 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:54:08.572 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:08 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:54:08.572 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:08 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:54:08.572 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:08 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:54:08.572 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:08 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:54:08.572 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:08 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/2276738718' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-03-27T10:54:08.579 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-03-27T10:54:08.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:08 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/1963960333' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-27T10:54:08.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:08 smithi177 ceph-mon[35294]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-27T10:54:08.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:08 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:54:08.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:08 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:54:08.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:08 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:54:08.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:08 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:54:08.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:08 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:54:08.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:08 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:54:08.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:08 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:54:08.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:08 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:54:08.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:08 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:54:08.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:08 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/2276738718' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-03-27T10:54:08.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:08.813+0000 7f3029000640 1 -- 172.21.15.97:0/3069399746 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3024108170 msgr2=0x7f3024108590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:08.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:08.813+0000 7f3029000640 1 --2- 172.21.15.97:0/3069399746 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3024108170 0x7f3024108590 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3014009900 tx=0x7f301402bc30 comp rx=0 tx=0).stop 2024-03-27T10:54:08.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:08.813+0000 7f3029000640 1 -- 172.21.15.97:0/3069399746 shutdown_connections 2024-03-27T10:54:08.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:08.813+0000 7f3029000640 1 --2- 172.21.15.97:0/3069399746 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3024109d20 0x7f30241105b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:08.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:08.813+0000 7f3029000640 1 --2- 172.21.15.97:0/3069399746 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3024109360 0x7f30241097e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:08.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:08.813+0000 7f3029000640 1 --2- 172.21.15.97:0/3069399746 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3024108170 0x7f3024108590 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:08.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:08.813+0000 7f3029000640 1 -- 172.21.15.97:0/3069399746 >> 172.21.15.97:0/3069399746 conn(0x7f3024077110 msgr2=0x7f3024079550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:54:08.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:08.814+0000 7f3029000640 1 -- 172.21.15.97:0/3069399746 shutdown_connections 2024-03-27T10:54:08.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:08.814+0000 7f3029000640 1 -- 172.21.15.97:0/3069399746 wait complete. 2024-03-27T10:54:08.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:08.814+0000 7f3029000640 1 Processor -- start 2024-03-27T10:54:08.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:08.815+0000 7f3029000640 1 -- start start 2024-03-27T10:54:08.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:08.815+0000 7f3029000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3024108170 0x7f302419e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:08.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:08.815+0000 7f3029000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3024109360 0x7f302419e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:08.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:08.815+0000 7f3029000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3024109d20 0x7f30241a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:08.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:08.815+0000 7f3029000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3024113cd0 con 0x7f3024108170 2024-03-27T10:54:08.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:08.815+0000 7f3029000640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f3024113b50 con 0x7f3024109360 2024-03-27T10:54:08.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:08.815+0000 7f3029000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f3024113e50 con 0x7f3024109d20 2024-03-27T10:54:08.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:08.815+0000 7f3021e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3024108170 0x7f302419e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:08.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:08.815+0000 7f3021e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3024108170 0x7f302419e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57108/0 (socket says 172.21.15.97:57108) 2024-03-27T10:54:08.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:08.815+0000 7f3021e00640 1 -- 172.21.15.97:0/1621064938 learned_addr learned my addr 172.21.15.97:0/1621064938 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:54:08.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:08.815+0000 7f3021400640 1 --2- 172.21.15.97:0/1621064938 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3024109360 0x7f302419e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:08.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:08.816+0000 7f3022800640 1 --2- 172.21.15.97:0/1621064938 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3024109d20 0x7f30241a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:08.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:08.816+0000 7f3021e00640 1 -- 172.21.15.97:0/1621064938 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3024109d20 msgr2=0x7f30241a5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:08.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:08.816+0000 7f3021e00640 1 --2- 172.21.15.97:0/1621064938 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3024109d20 0x7f30241a5990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:08.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:08.816+0000 7f3021e00640 1 -- 172.21.15.97:0/1621064938 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3024109360 msgr2=0x7f302419e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:08.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:08.816+0000 7f3021e00640 1 --2- 172.21.15.97:0/1621064938 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3024109360 0x7f302419e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:08.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:08.816+0000 7f3021e00640 1 -- 172.21.15.97:0/1621064938 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30241a6000 con 0x7f3024108170 2024-03-27T10:54:08.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:08.816+0000 7f3021e00640 1 --2- 172.21.15.97:0/1621064938 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3024108170 0x7f302419e3e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f3014005ab0 tx=0x7f30140042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:54:08.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:08.816+0000 7f301aa00640 1 -- 172.21.15.97:0/1621064938 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3014043020 con 0x7f3024108170 2024-03-27T10:54:08.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:08.816+0000 7f301aa00640 1 -- 172.21.15.97:0/1621064938 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f30140054c0 con 0x7f3024108170 2024-03-27T10:54:08.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:08.816+0000 7f3029000640 1 -- 172.21.15.97:0/1621064938 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30241a6290 con 0x7f3024108170 2024-03-27T10:54:08.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:08.816+0000 7f301aa00640 1 -- 172.21.15.97:0/1621064938 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3014003d10 con 0x7f3024108170 2024-03-27T10:54:08.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:08.816+0000 7f3029000640 1 -- 172.21.15.97:0/1621064938 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30241a6830 con 0x7f3024108170 2024-03-27T10:54:08.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:08.818+0000 7f3029000640 1 -- 172.21.15.97:0/1621064938 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f302410f580 con 0x7f3024108170 2024-03-27T10:54:08.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:08.819+0000 7f301aa00640 1 -- 172.21.15.97:0/1621064938 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138370+0+0 (secure 0 0 0) 0x7f3014046020 con 0x7f3024108170 2024-03-27T10:54:08.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:08.821+0000 7f301aa00640 1 --2- 172.21.15.97:0/1621064938 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f30080a3690 0x7f30080a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:08.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:08.821+0000 7f301aa00640 1 -- 172.21.15.97:0/1621064938 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 7979+0+0 (secure 0 0 0) 0x7f30140f1ff0 con 0x7f3024108170 2024-03-27T10:54:08.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:08.821+0000 7f3021400640 1 --2- 172.21.15.97:0/1621064938 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f30080a3690 0x7f30080a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:08.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:08.821+0000 7f3021400640 1 --2- 172.21.15.97:0/1621064938 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f30080a3690 0x7f30080a5b50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3010005fd0 tx=0x7f3010005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:54:08.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:08.825+0000 7f301aa00640 1 -- 172.21.15.97:0/1621064938 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f301404d020 con 0x7f3024108170 2024-03-27T10:54:09.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:09.052+0000 7f3029000640 1 -- 172.21.15.97:0/1621064938 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f302402d370 con 0x7f3024108170 2024-03-27T10:54:09.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:09 smithi114 ceph-mon[35338]: pgmap v364: 97 pgs: 44 active+clean, 33 creating+peering, 20 unknown; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 203 B/s wr, 1 op/s 2024-03-27T10:54:09.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:09 smithi114 ceph-mon[35338]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-27T10:54:09.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:09 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/2276738718' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-03-27T10:54:09.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:09 smithi114 ceph-mon[35338]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-27T10:54:09.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:09 smithi114 ceph-mon[35338]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-27T10:54:09.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:09 smithi114 ceph-mon[35338]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-03-27T10:54:09.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:09 smithi114 ceph-mon[35338]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-03-27T10:54:09.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:09 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/1621064938' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-27T10:54:09.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:09.589+0000 7f301aa00640 1 -- 172.21.15.97:0/1621064938 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v20) ==== 121+0+0 (secure 0 0 0) 0x7f30140bec70 con 0x7f3024108170 2024-03-27T10:54:09.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:09.592+0000 7f3029000640 1 -- 172.21.15.97:0/1621064938 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f30080a3690 msgr2=0x7f30080a5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:09.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:09.592+0000 7f3029000640 1 --2- 172.21.15.97:0/1621064938 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f30080a3690 0x7f30080a5b50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3010005fd0 tx=0x7f3010005850 comp rx=0 tx=0).stop 2024-03-27T10:54:09.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:09.592+0000 7f3029000640 1 -- 172.21.15.97:0/1621064938 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3024108170 msgr2=0x7f302419e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:09.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:09.592+0000 7f3029000640 1 --2- 172.21.15.97:0/1621064938 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3024108170 0x7f302419e3e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f3014005ab0 tx=0x7f30140042f0 comp rx=0 tx=0).stop 2024-03-27T10:54:09.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:09.593+0000 7f3029000640 1 -- 172.21.15.97:0/1621064938 shutdown_connections 2024-03-27T10:54:09.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:09.593+0000 7f3029000640 1 --2- 172.21.15.97:0/1621064938 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3024109d20 0x7f30241a5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:09.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:09.593+0000 7f3029000640 1 --2- 172.21.15.97:0/1621064938 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f30080a3690 0x7f30080a5b50 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:09.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:09.593+0000 7f3029000640 1 --2- 172.21.15.97:0/1621064938 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3024109360 0x7f302419e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:09.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:09.593+0000 7f3029000640 1 --2- 172.21.15.97:0/1621064938 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3024108170 0x7f302419e3e0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:09.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:09.593+0000 7f3029000640 1 -- 172.21.15.97:0/1621064938 >> 172.21.15.97:0/1621064938 conn(0x7f3024077110 msgr2=0x7f302410c750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:54:09.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:09.593+0000 7f3029000640 1 -- 172.21.15.97:0/1621064938 shutdown_connections 2024-03-27T10:54:09.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:09.593+0000 7f3029000640 1 -- 172.21.15.97:0/1621064938 wait complete. 2024-03-27T10:54:09.583 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:09 smithi097 ceph-mon[31859]: pgmap v364: 97 pgs: 44 active+clean, 33 creating+peering, 20 unknown; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 203 B/s wr, 1 op/s 2024-03-27T10:54:09.583 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:09 smithi097 ceph-mon[31859]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-27T10:54:09.583 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:09 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/2276738718' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-03-27T10:54:09.583 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:09 smithi097 ceph-mon[31859]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-27T10:54:09.583 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:09 smithi097 ceph-mon[31859]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-27T10:54:09.583 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:09 smithi097 ceph-mon[31859]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-03-27T10:54:09.583 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:09 smithi097 ceph-mon[31859]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-03-27T10:54:09.583 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:09 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/1621064938' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-27T10:54:09.585 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-03-27T10:54:09.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:09 smithi177 ceph-mon[35294]: pgmap v364: 97 pgs: 44 active+clean, 33 creating+peering, 20 unknown; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 203 B/s wr, 1 op/s 2024-03-27T10:54:09.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:09 smithi177 ceph-mon[35294]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-27T10:54:09.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:09 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/2276738718' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-03-27T10:54:09.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:09 smithi177 ceph-mon[35294]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-27T10:54:09.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:09 smithi177 ceph-mon[35294]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-27T10:54:09.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:09 smithi177 ceph-mon[35294]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-03-27T10:54:09.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:09 smithi177 ceph-mon[35294]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-03-27T10:54:09.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:09 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/1621064938' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-27T10:54:09.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:09.813+0000 7f25c2600640 1 -- 172.21.15.97:0/1858581980 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25bc104f80 msgr2=0x7f25bc105400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:09.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:09.813+0000 7f25c2600640 1 --2- 172.21.15.97:0/1858581980 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25bc104f80 0x7f25bc105400 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f25ac009900 tx=0x7f25ac02bc40 comp rx=0 tx=0).stop 2024-03-27T10:54:09.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:09.813+0000 7f25c2600640 1 -- 172.21.15.97:0/1858581980 shutdown_connections 2024-03-27T10:54:09.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:09.813+0000 7f25c2600640 1 --2- 172.21.15.97:0/1858581980 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f25bc105940 0x7f25bc10c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:09.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:09.813+0000 7f25c2600640 1 --2- 172.21.15.97:0/1858581980 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25bc104f80 0x7f25bc105400 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:09.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:09.813+0000 7f25c2600640 1 --2- 172.21.15.97:0/1858581980 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f25bc103d90 0x7f25bc1041b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:09.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:09.813+0000 7f25c2600640 1 -- 172.21.15.97:0/1858581980 >> 172.21.15.97:0/1858581980 conn(0x7f25bc0ff300 msgr2=0x7f25bc101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:54:09.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:09.813+0000 7f25c2600640 1 -- 172.21.15.97:0/1858581980 shutdown_connections 2024-03-27T10:54:09.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:09.813+0000 7f25c2600640 1 -- 172.21.15.97:0/1858581980 wait complete. 2024-03-27T10:54:09.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:09.814+0000 7f25c2600640 1 Processor -- start 2024-03-27T10:54:09.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:09.814+0000 7f25c2600640 1 -- start start 2024-03-27T10:54:09.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:09.814+0000 7f25c2600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f25bc103d90 0x7f25bc19a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:09.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:09.814+0000 7f25c2600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25bc104f80 0x7f25bc19a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:09.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:09.814+0000 7f25c2600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f25bc105940 0x7f25bc1a15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:09.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:09.814+0000 7f25c2600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f25bc10f970 con 0x7f25bc104f80 2024-03-27T10:54:09.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:09.814+0000 7f25c2600640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f25bc10f7f0 con 0x7f25bc105940 2024-03-27T10:54:09.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:09.814+0000 7f25c2600640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f25bc10faf0 con 0x7f25bc103d90 2024-03-27T10:54:09.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:09.814+0000 7f25baa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25bc104f80 0x7f25bc19a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:09.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:09.814+0000 7f25baa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25bc104f80 0x7f25bc19a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43110/0 (socket says 172.21.15.97:43110) 2024-03-27T10:54:09.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:09.814+0000 7f25baa00640 1 -- 172.21.15.97:0/3343560827 learned_addr learned my addr 172.21.15.97:0/3343560827 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:54:09.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:09.815+0000 7f25bbe00640 1 --2- 172.21.15.97:0/3343560827 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f25bc105940 0x7f25bc1a15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:09.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:09.815+0000 7f25baa00640 1 -- 172.21.15.97:0/3343560827 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f25bc103d90 msgr2=0x7f25bc19a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:09.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:09.815+0000 7f25bb400640 1 --2- 172.21.15.97:0/3343560827 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f25bc103d90 0x7f25bc19a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:09.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:09.815+0000 7f25baa00640 1 --2- 172.21.15.97:0/3343560827 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f25bc103d90 0x7f25bc19a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:09.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:09.815+0000 7f25baa00640 1 -- 172.21.15.97:0/3343560827 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f25bc105940 msgr2=0x7f25bc1a15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:09.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:09.815+0000 7f25baa00640 1 --2- 172.21.15.97:0/3343560827 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f25bc105940 0x7f25bc1a15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:09.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:09.815+0000 7f25baa00640 1 -- 172.21.15.97:0/3343560827 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25bc1a1c40 con 0x7f25bc104f80 2024-03-27T10:54:09.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:09.815+0000 7f25baa00640 1 --2- 172.21.15.97:0/3343560827 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25bc104f80 0x7f25bc19a560 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f25ac000c00 tx=0x7f25ac0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:54:09.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:09.815+0000 7f25b3e00640 1 -- 172.21.15.97:0/3343560827 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25ac043020 con 0x7f25bc104f80 2024-03-27T10:54:09.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:09.815+0000 7f25b3e00640 1 -- 172.21.15.97:0/3343560827 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f25ac0054c0 con 0x7f25bc104f80 2024-03-27T10:54:09.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:09.815+0000 7f25c2600640 1 -- 172.21.15.97:0/3343560827 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25bc1a1ed0 con 0x7f25bc104f80 2024-03-27T10:54:09.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:09.815+0000 7f25b3e00640 1 -- 172.21.15.97:0/3343560827 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25ac003d40 con 0x7f25bc104f80 2024-03-27T10:54:09.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:09.816+0000 7f25c2600640 1 -- 172.21.15.97:0/3343560827 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25bc1a2460 con 0x7f25bc104f80 2024-03-27T10:54:09.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:09.817+0000 7f25c2600640 1 -- 172.21.15.97:0/3343560827 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25bc10b1d0 con 0x7f25bc104f80 2024-03-27T10:54:09.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:09.819+0000 7f25b3e00640 1 -- 172.21.15.97:0/3343560827 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138370+0+0 (secure 0 0 0) 0x7f25ac046020 con 0x7f25bc104f80 2024-03-27T10:54:09.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:09.820+0000 7f25b3e00640 1 --2- 172.21.15.97:0/3343560827 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f25a00a3690 0x7f25a00a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:09.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:09.820+0000 7f25b3e00640 1 -- 172.21.15.97:0/3343560827 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 7979+0+0 (secure 0 0 0) 0x7f25ac0f21a0 con 0x7f25bc104f80 2024-03-27T10:54:09.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:09.821+0000 7f25bb400640 1 --2- 172.21.15.97:0/3343560827 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f25a00a3690 0x7f25a00a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:09.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:09.821+0000 7f25bb400640 1 --2- 172.21.15.97:0/3343560827 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f25a00a3690 0x7f25a00a5b50 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f25a4005fd0 tx=0x7f25a4005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:54:09.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:09.824+0000 7f25b3e00640 1 -- 172.21.15.97:0/3343560827 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f25ac04d020 con 0x7f25bc104f80 2024-03-27T10:54:10.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.050+0000 7f25c2600640 1 -- 172.21.15.97:0/3343560827 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f25bc02d370 con 0x7f25bc104f80 2024-03-27T10:54:10.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.147+0000 7f25b3e00640 1 -- 172.21.15.97:0/3343560827 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v22) ==== 115+0+0 (secure 0 0 0) 0x7f25ac0bef40 con 0x7f25bc104f80 2024-03-27T10:54:10.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.151+0000 7f25c2600640 1 -- 172.21.15.97:0/3343560827 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f25a00a3690 msgr2=0x7f25a00a5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:10.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.151+0000 7f25c2600640 1 --2- 172.21.15.97:0/3343560827 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f25a00a3690 0x7f25a00a5b50 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f25a4005fd0 tx=0x7f25a4005850 comp rx=0 tx=0).stop 2024-03-27T10:54:10.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.151+0000 7f25c2600640 1 -- 172.21.15.97:0/3343560827 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25bc104f80 msgr2=0x7f25bc19a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:10.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.151+0000 7f25c2600640 1 --2- 172.21.15.97:0/3343560827 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25bc104f80 0x7f25bc19a560 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f25ac000c00 tx=0x7f25ac0042f0 comp rx=0 tx=0).stop 2024-03-27T10:54:10.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.151+0000 7f25c2600640 1 -- 172.21.15.97:0/3343560827 shutdown_connections 2024-03-27T10:54:10.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.151+0000 7f25c2600640 1 --2- 172.21.15.97:0/3343560827 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f25a00a3690 0x7f25a00a5b50 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:10.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.151+0000 7f25c2600640 1 --2- 172.21.15.97:0/3343560827 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f25bc105940 0x7f25bc1a15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:10.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.151+0000 7f25c2600640 1 --2- 172.21.15.97:0/3343560827 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25bc104f80 0x7f25bc19a560 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:10.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.151+0000 7f25c2600640 1 --2- 172.21.15.97:0/3343560827 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f25bc103d90 0x7f25bc19a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:10.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.151+0000 7f25c2600640 1 -- 172.21.15.97:0/3343560827 >> 172.21.15.97:0/3343560827 conn(0x7f25bc0ff300 msgr2=0x7f25bc1083a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:54:10.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.151+0000 7f25c2600640 1 -- 172.21.15.97:0/3343560827 shutdown_connections 2024-03-27T10:54:10.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.152+0000 7f25c2600640 1 -- 172.21.15.97:0/3343560827 wait complete. 2024-03-27T10:54:10.151 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated'} subvols for filesystem 'cephfs' 2024-03-27T10:54:10.152 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-03-27T10:54:10.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.367+0000 7f6779e00640 1 -- 172.21.15.97:0/2657084970 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6774077750 msgr2=0x7f6774077bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:10.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.367+0000 7f6779e00640 1 --2- 172.21.15.97:0/2657084970 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6774077750 0x7f6774077bb0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f675c009900 tx=0x7f675c02bc30 comp rx=0 tx=0).stop 2024-03-27T10:54:10.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.367+0000 7f6779e00640 1 -- 172.21.15.97:0/2657084970 shutdown_connections 2024-03-27T10:54:10.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.367+0000 7f6779e00640 1 --2- 172.21.15.97:0/2657084970 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f67740780f0 0x7f6774078990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:10.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.367+0000 7f6779e00640 1 --2- 172.21.15.97:0/2657084970 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6774077750 0x7f6774077bb0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:10.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.367+0000 7f6779e00640 1 --2- 172.21.15.97:0/2657084970 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6774076510 0x7f6774076930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:10.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.367+0000 7f6779e00640 1 -- 172.21.15.97:0/2657084970 >> 172.21.15.97:0/2657084970 conn(0x7f67740ff300 msgr2=0x7f6774101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:54:10.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.367+0000 7f6779e00640 1 -- 172.21.15.97:0/2657084970 shutdown_connections 2024-03-27T10:54:10.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.367+0000 7f6779e00640 1 -- 172.21.15.97:0/2657084970 wait complete. 2024-03-27T10:54:10.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.368+0000 7f6779e00640 1 Processor -- start 2024-03-27T10:54:10.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.368+0000 7f6779e00640 1 -- start start 2024-03-27T10:54:10.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.368+0000 7f6779e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6774076510 0x7f677419e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:10.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.368+0000 7f6779e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6774077750 0x7f677419e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:10.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.369+0000 7f6779e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f67740780f0 0x7f67741a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:10.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.369+0000 7f6779e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6774113d10 con 0x7f6774077750 2024-03-27T10:54:10.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.369+0000 7f6779e00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f6774113b90 con 0x7f6774076510 2024-03-27T10:54:10.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.369+0000 7f6779e00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6774113e90 con 0x7f67740780f0 2024-03-27T10:54:10.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.369+0000 7f676be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6774077750 0x7f677419e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:10.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.369+0000 7f676be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6774077750 0x7f677419e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43126/0 (socket says 172.21.15.97:43126) 2024-03-27T10:54:10.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.369+0000 7f676be00640 1 -- 172.21.15.97:0/3794056100 learned_addr learned my addr 172.21.15.97:0/3794056100 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:54:10.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.369+0000 7f6772a00640 1 --2- 172.21.15.97:0/3794056100 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6774076510 0x7f677419e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:10.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.369+0000 7f6773400640 1 --2- 172.21.15.97:0/3794056100 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f67740780f0 0x7f67741a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:10.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.369+0000 7f676be00640 1 -- 172.21.15.97:0/3794056100 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f67740780f0 msgr2=0x7f67741a5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:10.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.369+0000 7f676be00640 1 --2- 172.21.15.97:0/3794056100 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f67740780f0 0x7f67741a5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:10.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.369+0000 7f676be00640 1 -- 172.21.15.97:0/3794056100 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6774076510 msgr2=0x7f677419e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:10.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.369+0000 7f676be00640 1 --2- 172.21.15.97:0/3794056100 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6774076510 0x7f677419e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:10.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.369+0000 7f676be00640 1 -- 172.21.15.97:0/3794056100 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67741a6000 con 0x7f6774077750 2024-03-27T10:54:10.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.370+0000 7f676be00640 1 --2- 172.21.15.97:0/3794056100 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6774077750 0x7f677419e920 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f675c000c00 tx=0x7f675c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:54:10.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.370+0000 7f6778a00640 1 -- 172.21.15.97:0/3794056100 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f675c043020 con 0x7f6774077750 2024-03-27T10:54:10.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.370+0000 7f6778a00640 1 -- 172.21.15.97:0/3794056100 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f675c0054c0 con 0x7f6774077750 2024-03-27T10:54:10.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.370+0000 7f6778a00640 1 -- 172.21.15.97:0/3794056100 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f675c003d70 con 0x7f6774077750 2024-03-27T10:54:10.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.370+0000 7f6779e00640 1 -- 172.21.15.97:0/3794056100 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67741a6290 con 0x7f6774077750 2024-03-27T10:54:10.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.370+0000 7f6779e00640 1 -- 172.21.15.97:0/3794056100 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67741a6790 con 0x7f6774077750 2024-03-27T10:54:10.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.371+0000 7f6779e00640 1 -- 172.21.15.97:0/3794056100 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6774110700 con 0x7f6774077750 2024-03-27T10:54:10.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.373+0000 7f6778a00640 1 -- 172.21.15.97:0/3794056100 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138370+0+0 (secure 0 0 0) 0x7f675c046020 con 0x7f6774077750 2024-03-27T10:54:10.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.374+0000 7f6778a00640 1 --2- 172.21.15.97:0/3794056100 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f67580a3690 0x7f67580a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:10.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.374+0000 7f6778a00640 1 -- 172.21.15.97:0/3794056100 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 7979+0+0 (secure 0 0 0) 0x7f675c0f2060 con 0x7f6774077750 2024-03-27T10:54:10.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.374+0000 7f6772a00640 1 --2- 172.21.15.97:0/3794056100 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f67580a3690 0x7f67580a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:10.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.375+0000 7f6772a00640 1 --2- 172.21.15.97:0/3794056100 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f67580a3690 0x7f67580a5b50 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f67640058c0 tx=0x7f676400a250 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:54:10.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.378+0000 7f6778a00640 1 -- 172.21.15.97:0/3794056100 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f675c04d020 con 0x7f6774077750 2024-03-27T10:54:10.517 INFO:journalctl@ceph.mds.b.smithi114.stdout:Mar 27 10:54:10 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-b[65857]: 2024-03-27T10:54:10.122+0000 7f0399000700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-03-27T10:54:10.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.609+0000 7f6779e00640 1 -- 172.21.15.97:0/3794056100 --> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f6774061f90 con 0x7f67580a3690 2024-03-27T10:54:10.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.664+0000 7f6778a00640 1 -- 172.21.15.97:0/3794056100 <== mgr.14152 v2:172.21.15.97:6800/3955463538 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6774061f90 con 0x7f67580a3690 2024-03-27T10:54:10.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.667+0000 7f6779e00640 1 -- 172.21.15.97:0/3794056100 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f67580a3690 msgr2=0x7f67580a5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:10.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.667+0000 7f6779e00640 1 --2- 172.21.15.97:0/3794056100 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f67580a3690 0x7f67580a5b50 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f67640058c0 tx=0x7f676400a250 comp rx=0 tx=0).stop 2024-03-27T10:54:10.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.667+0000 7f6779e00640 1 -- 172.21.15.97:0/3794056100 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6774077750 msgr2=0x7f677419e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:10.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.667+0000 7f6779e00640 1 --2- 172.21.15.97:0/3794056100 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6774077750 0x7f677419e920 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f675c000c00 tx=0x7f675c0042f0 comp rx=0 tx=0).stop 2024-03-27T10:54:10.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.667+0000 7f6779e00640 1 -- 172.21.15.97:0/3794056100 shutdown_connections 2024-03-27T10:54:10.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.667+0000 7f6779e00640 1 --2- 172.21.15.97:0/3794056100 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f67740780f0 0x7f67741a5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:10.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.667+0000 7f6779e00640 1 --2- 172.21.15.97:0/3794056100 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6774077750 0x7f677419e920 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:10.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.667+0000 7f6779e00640 1 --2- 172.21.15.97:0/3794056100 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f67580a3690 0x7f67580a5b50 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:10.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.667+0000 7f6779e00640 1 --2- 172.21.15.97:0/3794056100 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6774076510 0x7f677419e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:10.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.667+0000 7f6779e00640 1 -- 172.21.15.97:0/3794056100 >> 172.21.15.97:0/3794056100 conn(0x7f67740ff300 msgr2=0x7f6774100e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:54:10.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.667+0000 7f6779e00640 1 -- 172.21.15.97:0/3794056100 shutdown_connections 2024-03-27T10:54:10.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.667+0000 7f6779e00640 1 -- 172.21.15.97:0/3794056100 wait complete. 2024-03-27T10:54:10.658 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-03-27T10:54:10.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.881+0000 7f2271a00640 1 -- 172.21.15.97:0/2387391339 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f226c105940 msgr2=0x7f226c10c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:10.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.881+0000 7f2271a00640 1 --2- 172.21.15.97:0/2387391339 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f226c105940 0x7f226c10c1d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2264009ca0 tx=0x7f226402c0f0 comp rx=0 tx=0).stop 2024-03-27T10:54:10.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.881+0000 7f2271a00640 1 -- 172.21.15.97:0/2387391339 shutdown_connections 2024-03-27T10:54:10.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.881+0000 7f2271a00640 1 --2- 172.21.15.97:0/2387391339 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f226c105940 0x7f226c10c1d0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:10.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.881+0000 7f2271a00640 1 --2- 172.21.15.97:0/2387391339 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f226c104f80 0x7f226c105400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:10.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.881+0000 7f2271a00640 1 --2- 172.21.15.97:0/2387391339 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f226c103d90 0x7f226c1041b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:10.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.881+0000 7f2271a00640 1 -- 172.21.15.97:0/2387391339 >> 172.21.15.97:0/2387391339 conn(0x7f226c0ff300 msgr2=0x7f226c101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:54:10.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.881+0000 7f2271a00640 1 -- 172.21.15.97:0/2387391339 shutdown_connections 2024-03-27T10:54:10.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.881+0000 7f2271a00640 1 -- 172.21.15.97:0/2387391339 wait complete. 2024-03-27T10:54:10.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.882+0000 7f2271a00640 1 Processor -- start 2024-03-27T10:54:10.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.882+0000 7f2271a00640 1 -- start start 2024-03-27T10:54:10.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.882+0000 7f2271a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f226c103d90 0x7f226c19a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:10.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.882+0000 7f2271a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f226c104f80 0x7f226c19a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:10.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.882+0000 7f2271a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f226c105940 0x7f226c1a15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:10.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.882+0000 7f2271a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f226c10f930 con 0x7f226c105940 2024-03-27T10:54:10.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.882+0000 7f2271a00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f226c10f7b0 con 0x7f226c104f80 2024-03-27T10:54:10.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.882+0000 7f2271a00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f226c10fab0 con 0x7f226c103d90 2024-03-27T10:54:10.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.883+0000 7f226b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f226c105940 0x7f226c1a15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:10.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.883+0000 7f226b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f226c105940 0x7f226c1a15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43164/0 (socket says 172.21.15.97:43164) 2024-03-27T10:54:10.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.883+0000 7f226b400640 1 -- 172.21.15.97:0/1238339924 learned_addr learned my addr 172.21.15.97:0/1238339924 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:54:10.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.883+0000 7f226aa00640 1 --2- 172.21.15.97:0/1238339924 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f226c103d90 0x7f226c19a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:10.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.883+0000 7f226a000640 1 --2- 172.21.15.97:0/1238339924 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f226c104f80 0x7f226c19a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:10.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.883+0000 7f226b400640 1 -- 172.21.15.97:0/1238339924 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f226c103d90 msgr2=0x7f226c19a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:10.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.883+0000 7f226b400640 1 --2- 172.21.15.97:0/1238339924 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f226c103d90 0x7f226c19a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:10.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.883+0000 7f226b400640 1 -- 172.21.15.97:0/1238339924 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f226c104f80 msgr2=0x7f226c19a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:10.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.883+0000 7f226b400640 1 --2- 172.21.15.97:0/1238339924 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f226c104f80 0x7f226c19a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:10.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.883+0000 7f226b400640 1 -- 172.21.15.97:0/1238339924 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f226c1a1c20 con 0x7f226c105940 2024-03-27T10:54:10.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.883+0000 7f226a000640 1 --2- 172.21.15.97:0/1238339924 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f226c104f80 0x7f226c19a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:54:10.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.883+0000 7f226b400640 1 --2- 172.21.15.97:0/1238339924 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f226c105940 0x7f226c1a15b0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f226402c700 tx=0x7f22640328a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:54:10.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.884+0000 7f2263400640 1 -- 172.21.15.97:0/1238339924 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2264041020 con 0x7f226c105940 2024-03-27T10:54:10.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.884+0000 7f2263400640 1 -- 172.21.15.97:0/1238339924 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2264003e20 con 0x7f226c105940 2024-03-27T10:54:10.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.884+0000 7f2271a00640 1 -- 172.21.15.97:0/1238339924 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f226c1a1eb0 con 0x7f226c105940 2024-03-27T10:54:10.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.884+0000 7f2263400640 1 -- 172.21.15.97:0/1238339924 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2264039040 con 0x7f226c105940 2024-03-27T10:54:10.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.884+0000 7f2271a00640 1 -- 172.21.15.97:0/1238339924 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f226c1a2390 con 0x7f226c105940 2024-03-27T10:54:10.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.885+0000 7f2271a00640 1 -- 172.21.15.97:0/1238339924 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f226c10b0b0 con 0x7f226c105940 2024-03-27T10:54:10.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.888+0000 7f2263400640 1 -- 172.21.15.97:0/1238339924 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138370+0+0 (secure 0 0 0) 0x7f2264044020 con 0x7f226c105940 2024-03-27T10:54:10.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.889+0000 7f2263400640 1 --2- 172.21.15.97:0/1238339924 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f22500a3610 0x7f22500a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:10.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.890+0000 7f2263400640 1 -- 172.21.15.97:0/1238339924 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 7979+0+0 (secure 0 0 0) 0x7f22640f2500 con 0x7f226c105940 2024-03-27T10:54:10.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.890+0000 7f226aa00640 1 --2- 172.21.15.97:0/1238339924 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f22500a3610 0x7f22500a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:10.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.890+0000 7f226aa00640 1 --2- 172.21.15.97:0/1238339924 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f22500a3610 0x7f22500a5ad0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f226c104de0 tx=0x7f225c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:54:10.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:10.892+0000 7f2263400640 1 -- 172.21.15.97:0/1238339924 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f226404b020 con 0x7f226c105940 2024-03-27T10:54:10.891 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:54:10 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: 2024-03-27T10:54:10.623+0000 7fb29e200700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-27T10:54:10.891 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:54:10 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: 2024-03-27T10:54:10.623+0000 7fb29e200700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-27T10:54:10.891 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:54:10 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: 2024-03-27T10:54:10.623+0000 7fb29e200700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-27T10:54:10.891 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:54:10 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: 2024-03-27T10:54:10.623+0000 7fb29e200700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-27T10:54:10.891 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 10:54:10 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: 2024-03-27T10:54:10.623+0000 7fb29e200700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-27T10:54:10.892 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:10 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/1621064938' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-27T10:54:10.892 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:10 smithi097 ceph-mon[31859]: mds.? [v2:172.21.15.114:6836/616775588,v1:172.21.15.114:6837/616775588] up:active 2024-03-27T10:54:10.892 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:10 smithi097 ceph-mon[31859]: daemon mds.e assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-27T10:54:10.892 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:10 smithi097 ceph-mon[31859]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-03-27T10:54:10.892 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:10 smithi097 ceph-mon[31859]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:creating} 9 up:standby 2024-03-27T10:54:10.892 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:10 smithi097 ceph-mon[31859]: daemon mds.e is now active in filesystem cephfs as rank 2 2024-03-27T10:54:10.892 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:10 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/3343560827' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-27T10:54:10.892 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:10 smithi097 ceph-mon[31859]: pgmap v365: 97 pgs: 97 active+clean; 580 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 10 op/s 2024-03-27T10:54:10.892 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:10 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/3343560827' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-27T10:54:10.892 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:10 smithi097 ceph-mon[31859]: mds.? [v2:172.21.15.114:6834/3581918222,v1:172.21.15.114:6835/3581918222] up:active 2024-03-27T10:54:10.892 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:10 smithi097 ceph-mon[31859]: daemon mds.f assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-03-27T10:54:10.892 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:10 smithi097 ceph-mon[31859]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-03-27T10:54:10.892 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:10 smithi097 ceph-mon[31859]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=e=up:active,3=f=up:creating} 3 up:standby-replay 5 up:standby 2024-03-27T10:54:10.892 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:10 smithi097 ceph-mon[31859]: daemon mds.f is now active in filesystem cephfs as rank 3 2024-03-27T10:54:10.927 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:10 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/1621064938' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-27T10:54:10.927 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:10 smithi177 ceph-mon[35294]: mds.? [v2:172.21.15.114:6836/616775588,v1:172.21.15.114:6837/616775588] up:active 2024-03-27T10:54:10.927 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:10 smithi177 ceph-mon[35294]: daemon mds.e assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-27T10:54:10.928 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:10 smithi177 ceph-mon[35294]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-03-27T10:54:10.928 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:10 smithi177 ceph-mon[35294]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:creating} 9 up:standby 2024-03-27T10:54:10.928 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:10 smithi177 ceph-mon[35294]: daemon mds.e is now active in filesystem cephfs as rank 2 2024-03-27T10:54:10.928 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:10 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/3343560827' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-27T10:54:10.928 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:10 smithi177 ceph-mon[35294]: pgmap v365: 97 pgs: 97 active+clean; 580 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 10 op/s 2024-03-27T10:54:10.928 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:10 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/3343560827' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-27T10:54:10.928 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:10 smithi177 ceph-mon[35294]: mds.? [v2:172.21.15.114:6834/3581918222,v1:172.21.15.114:6835/3581918222] up:active 2024-03-27T10:54:10.928 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:10 smithi177 ceph-mon[35294]: daemon mds.f assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-03-27T10:54:10.928 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:10 smithi177 ceph-mon[35294]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-03-27T10:54:10.928 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:10 smithi177 ceph-mon[35294]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=e=up:active,3=f=up:creating} 3 up:standby-replay 5 up:standby 2024-03-27T10:54:10.928 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:10 smithi177 ceph-mon[35294]: daemon mds.f is now active in filesystem cephfs as rank 3 2024-03-27T10:54:11.017 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:10 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/1621064938' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-27T10:54:11.017 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:10 smithi114 ceph-mon[35338]: mds.? [v2:172.21.15.114:6836/616775588,v1:172.21.15.114:6837/616775588] up:active 2024-03-27T10:54:11.017 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:10 smithi114 ceph-mon[35338]: daemon mds.e assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-27T10:54:11.017 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:10 smithi114 ceph-mon[35338]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-03-27T10:54:11.017 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:10 smithi114 ceph-mon[35338]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:creating} 9 up:standby 2024-03-27T10:54:11.018 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:10 smithi114 ceph-mon[35338]: daemon mds.e is now active in filesystem cephfs as rank 2 2024-03-27T10:54:11.018 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:10 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/3343560827' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-27T10:54:11.018 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:10 smithi114 ceph-mon[35338]: pgmap v365: 97 pgs: 97 active+clean; 580 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 10 op/s 2024-03-27T10:54:11.018 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:10 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/3343560827' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-27T10:54:11.018 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:10 smithi114 ceph-mon[35338]: mds.? [v2:172.21.15.114:6834/3581918222,v1:172.21.15.114:6835/3581918222] up:active 2024-03-27T10:54:11.018 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:10 smithi114 ceph-mon[35338]: daemon mds.f assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-03-27T10:54:11.018 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:10 smithi114 ceph-mon[35338]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-03-27T10:54:11.018 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:10 smithi114 ceph-mon[35338]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=e=up:active,3=f=up:creating} 3 up:standby-replay 5 up:standby 2024-03-27T10:54:11.018 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:10 smithi114 ceph-mon[35338]: daemon mds.f is now active in filesystem cephfs as rank 3 2024-03-27T10:54:11.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.120+0000 7f2271a00640 1 -- 172.21.15.97:0/1238339924 --> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f226c062020 con 0x7f22500a3610 2024-03-27T10:54:11.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.167+0000 7f2263400640 1 -- 172.21.15.97:0/1238339924 <== mgr.14152 v2:172.21.15.97:6800/3955463538 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f226c062020 con 0x7f22500a3610 2024-03-27T10:54:11.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.170+0000 7f2271a00640 1 -- 172.21.15.97:0/1238339924 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f22500a3610 msgr2=0x7f22500a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:11.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.170+0000 7f2271a00640 1 --2- 172.21.15.97:0/1238339924 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f22500a3610 0x7f22500a5ad0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f226c104de0 tx=0x7f225c008040 comp rx=0 tx=0).stop 2024-03-27T10:54:11.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.170+0000 7f2271a00640 1 -- 172.21.15.97:0/1238339924 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f226c105940 msgr2=0x7f226c1a15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:11.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.170+0000 7f2271a00640 1 --2- 172.21.15.97:0/1238339924 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f226c105940 0x7f226c1a15b0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f226402c700 tx=0x7f22640328a0 comp rx=0 tx=0).stop 2024-03-27T10:54:11.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.170+0000 7f2271a00640 1 -- 172.21.15.97:0/1238339924 shutdown_connections 2024-03-27T10:54:11.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.170+0000 7f2271a00640 1 --2- 172.21.15.97:0/1238339924 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f226c105940 0x7f226c1a15b0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:11.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.170+0000 7f2271a00640 1 --2- 172.21.15.97:0/1238339924 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f22500a3610 0x7f22500a5ad0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:11.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.170+0000 7f2271a00640 1 --2- 172.21.15.97:0/1238339924 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f226c104f80 0x7f226c19a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:11.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.170+0000 7f2271a00640 1 --2- 172.21.15.97:0/1238339924 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f226c103d90 0x7f226c19a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:11.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.170+0000 7f2271a00640 1 -- 172.21.15.97:0/1238339924 >> 172.21.15.97:0/1238339924 conn(0x7f226c0ff300 msgr2=0x7f226c108280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:54:11.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.170+0000 7f2271a00640 1 -- 172.21.15.97:0/1238339924 shutdown_connections 2024-03-27T10:54:11.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.170+0000 7f2271a00640 1 -- 172.21.15.97:0/1238339924 wait complete. 2024-03-27T10:54:11.171 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-03-27T10:54:11.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.388+0000 7f40b4000640 1 -- 172.21.15.97:0/2309834907 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40ac106670 msgr2=0x7f40ac108b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:11.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.388+0000 7f40b4000640 1 --2- 172.21.15.97:0/2309834907 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40ac106670 0x7f40ac108b00 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f40a4009900 tx=0x7f40a402bc30 comp rx=0 tx=0).stop 2024-03-27T10:54:11.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.388+0000 7f40b4000640 1 -- 172.21.15.97:0/2309834907 shutdown_connections 2024-03-27T10:54:11.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.388+0000 7f40b4000640 1 --2- 172.21.15.97:0/2309834907 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40ac106670 0x7f40ac108b00 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:11.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.388+0000 7f40b4000640 1 --2- 172.21.15.97:0/2309834907 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f40ac103c60 0x7f40ac106050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:11.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.388+0000 7f40b4000640 1 --2- 172.21.15.97:0/2309834907 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f40ac101330 0x7f40ac103720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:11.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.388+0000 7f40b4000640 1 -- 172.21.15.97:0/2309834907 >> 172.21.15.97:0/2309834907 conn(0x7f40ac0fb310 msgr2=0x7f40ac0fd750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:54:11.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.388+0000 7f40b4000640 1 -- 172.21.15.97:0/2309834907 shutdown_connections 2024-03-27T10:54:11.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.388+0000 7f40b4000640 1 -- 172.21.15.97:0/2309834907 wait complete. 2024-03-27T10:54:11.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.390+0000 7f40b4000640 1 Processor -- start 2024-03-27T10:54:11.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.390+0000 7f40b4000640 1 -- start start 2024-03-27T10:54:11.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.390+0000 7f40b4000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f40ac101330 0x7f40ac199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:11.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.390+0000 7f40b4000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40ac103c60 0x7f40ac19a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:11.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.390+0000 7f40b4000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f40ac106670 0x7f40ac1a1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:11.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.390+0000 7f40b4000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f40ac10c5c0 con 0x7f40ac103c60 2024-03-27T10:54:11.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.390+0000 7f40b4000640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f40ac10c440 con 0x7f40ac106670 2024-03-27T10:54:11.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.390+0000 7f40b4000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f40ac10c740 con 0x7f40ac101330 2024-03-27T10:54:11.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.391+0000 7f40b0c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40ac103c60 0x7f40ac19a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:11.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.391+0000 7f40b0c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40ac103c60 0x7f40ac19a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43180/0 (socket says 172.21.15.97:43180) 2024-03-27T10:54:11.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.391+0000 7f40b0c00640 1 -- 172.21.15.97:0/3885680678 learned_addr learned my addr 172.21.15.97:0/3885680678 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:54:11.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.391+0000 7f40b1600640 1 --2- 172.21.15.97:0/3885680678 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f40ac101330 0x7f40ac199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:11.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.391+0000 7f40b2000640 1 --2- 172.21.15.97:0/3885680678 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f40ac106670 0x7f40ac1a1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:11.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.391+0000 7f40b0c00640 1 -- 172.21.15.97:0/3885680678 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f40ac101330 msgr2=0x7f40ac199fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:11.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.391+0000 7f40b0c00640 1 --2- 172.21.15.97:0/3885680678 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f40ac101330 0x7f40ac199fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:11.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.391+0000 7f40b0c00640 1 -- 172.21.15.97:0/3885680678 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f40ac106670 msgr2=0x7f40ac1a1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:11.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.391+0000 7f40b0c00640 1 --2- 172.21.15.97:0/3885680678 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f40ac106670 0x7f40ac1a1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:11.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.391+0000 7f40b0c00640 1 -- 172.21.15.97:0/3885680678 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40ac1a1bd0 con 0x7f40ac103c60 2024-03-27T10:54:11.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.391+0000 7f40b0c00640 1 --2- 172.21.15.97:0/3885680678 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40ac103c60 0x7f40ac19a4f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f409800d950 tx=0x7f409800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:54:11.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.392+0000 7f40aa000640 1 -- 172.21.15.97:0/3885680678 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4098017070 con 0x7f40ac103c60 2024-03-27T10:54:11.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.392+0000 7f40aa000640 1 -- 172.21.15.97:0/3885680678 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4098009c30 con 0x7f40ac103c60 2024-03-27T10:54:11.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.392+0000 7f40b4000640 1 -- 172.21.15.97:0/3885680678 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40ac1a1ec0 con 0x7f40ac103c60 2024-03-27T10:54:11.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.392+0000 7f40aa000640 1 -- 172.21.15.97:0/3885680678 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f409800f620 con 0x7f40ac103c60 2024-03-27T10:54:11.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.392+0000 7f40b4000640 1 -- 172.21.15.97:0/3885680678 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40ac1a2380 con 0x7f40ac103c60 2024-03-27T10:54:11.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.393+0000 7f40b4000640 1 -- 172.21.15.97:0/3885680678 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40ac1a25b0 con 0x7f40ac103c60 2024-03-27T10:54:11.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.396+0000 7f40aa000640 1 -- 172.21.15.97:0/3885680678 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138370+0+0 (secure 0 0 0) 0x7f40980109a0 con 0x7f40ac103c60 2024-03-27T10:54:11.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.398+0000 7f40aa000640 1 --2- 172.21.15.97:0/3885680678 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f40900a3610 0x7f40900a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:11.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.398+0000 7f40aa000640 1 -- 172.21.15.97:0/3885680678 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 7979+0+0 (secure 0 0 0) 0x7f40980cf7e0 con 0x7f40ac103c60 2024-03-27T10:54:11.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.398+0000 7f40b1600640 1 --2- 172.21.15.97:0/3885680678 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f40900a3610 0x7f40900a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:11.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.399+0000 7f40b1600640 1 --2- 172.21.15.97:0/3885680678 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f40900a3610 0x7f40900a5ad0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f40ac0ffe40 tx=0x7f409c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:54:11.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.404+0000 7f40aa000640 1 -- 172.21.15.97:0/3885680678 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f4098009de0 con 0x7f40ac103c60 2024-03-27T10:54:11.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.636+0000 7f40b4000640 1 -- 172.21.15.97:0/3885680678 --> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f40ac107a60 con 0x7f40900a3610 2024-03-27T10:54:11.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.652+0000 7f40aa000640 1 -- 172.21.15.97:0/3885680678 <== mgr.14152 v2:172.21.15.97:6800/3955463538 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f40ac107a60 con 0x7f40900a3610 2024-03-27T10:54:11.625 INFO:teuthology.orchestra.run.smithi097.stdout:/volumes/qa/sv_0/22bc04cd-8106-4d11-83f1-2aead5019d07 2024-03-27T10:54:11.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.655+0000 7f40b4000640 1 -- 172.21.15.97:0/3885680678 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f40900a3610 msgr2=0x7f40900a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:11.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.655+0000 7f40b4000640 1 --2- 172.21.15.97:0/3885680678 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f40900a3610 0x7f40900a5ad0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f40ac0ffe40 tx=0x7f409c009500 comp rx=0 tx=0).stop 2024-03-27T10:54:11.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.655+0000 7f40b4000640 1 -- 172.21.15.97:0/3885680678 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40ac103c60 msgr2=0x7f40ac19a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:11.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.655+0000 7f40b4000640 1 --2- 172.21.15.97:0/3885680678 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40ac103c60 0x7f40ac19a4f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f409800d950 tx=0x7f409800de10 comp rx=0 tx=0).stop 2024-03-27T10:54:11.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.655+0000 7f40b4000640 1 -- 172.21.15.97:0/3885680678 shutdown_connections 2024-03-27T10:54:11.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.655+0000 7f40b4000640 1 --2- 172.21.15.97:0/3885680678 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f40900a3610 0x7f40900a5ad0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:11.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.655+0000 7f40b4000640 1 --2- 172.21.15.97:0/3885680678 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f40ac106670 0x7f40ac1a1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:11.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.655+0000 7f40b4000640 1 --2- 172.21.15.97:0/3885680678 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40ac103c60 0x7f40ac19a4f0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:11.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.655+0000 7f40b4000640 1 --2- 172.21.15.97:0/3885680678 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f40ac101330 0x7f40ac199fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:11.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.655+0000 7f40b4000640 1 -- 172.21.15.97:0/3885680678 >> 172.21.15.97:0/3885680678 conn(0x7f40ac0fb310 msgr2=0x7f40ac100560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:54:11.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.655+0000 7f40b4000640 1 -- 172.21.15.97:0/3885680678 shutdown_connections 2024-03-27T10:54:11.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.656+0000 7f40b4000640 1 -- 172.21.15.97:0/3885680678 wait complete. 2024-03-27T10:54:11.646 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-03-27T10:54:11.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.878+0000 7f1b4ee00640 1 -- 172.21.15.97:0/86407132 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b48103d90 msgr2=0x7f1b481041b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:11.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.878+0000 7f1b4ee00640 1 --2- 172.21.15.97:0/86407132 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b48103d90 0x7f1b481041b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f1b38009900 tx=0x7f1b3802bc30 comp rx=0 tx=0).stop 2024-03-27T10:54:11.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.878+0000 7f1b4ee00640 1 -- 172.21.15.97:0/86407132 shutdown_connections 2024-03-27T10:54:11.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.878+0000 7f1b4ee00640 1 --2- 172.21.15.97:0/86407132 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1b48105940 0x7f1b4810c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:11.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.878+0000 7f1b4ee00640 1 --2- 172.21.15.97:0/86407132 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1b48104f80 0x7f1b48105400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:11.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.878+0000 7f1b4ee00640 1 --2- 172.21.15.97:0/86407132 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b48103d90 0x7f1b481041b0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:11.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.878+0000 7f1b4ee00640 1 -- 172.21.15.97:0/86407132 >> 172.21.15.97:0/86407132 conn(0x7f1b480ff300 msgr2=0x7f1b48101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:54:11.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.878+0000 7f1b4ee00640 1 -- 172.21.15.97:0/86407132 shutdown_connections 2024-03-27T10:54:11.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.878+0000 7f1b4ee00640 1 -- 172.21.15.97:0/86407132 wait complete. 2024-03-27T10:54:11.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.879+0000 7f1b4ee00640 1 Processor -- start 2024-03-27T10:54:11.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.879+0000 7f1b4ee00640 1 -- start start 2024-03-27T10:54:11.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.880+0000 7f1b4ee00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1b48103d90 0x7f1b48199f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:11.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.880+0000 7f1b4ee00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1b48104f80 0x7f1b4819a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:11.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.880+0000 7f1b4ee00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b48105940 0x7f1b481a1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:11.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.880+0000 7f1b4ee00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1b4810f8e0 con 0x7f1b48105940 2024-03-27T10:54:11.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.880+0000 7f1b4ee00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f1b4810f760 con 0x7f1b48103d90 2024-03-27T10:54:11.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.880+0000 7f1b4ee00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1b4810fa60 con 0x7f1b48104f80 2024-03-27T10:54:11.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.880+0000 7f1b4ce00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b48105940 0x7f1b481a1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:11.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.880+0000 7f1b4ce00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b48105940 0x7f1b481a1540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43188/0 (socket says 172.21.15.97:43188) 2024-03-27T10:54:11.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.880+0000 7f1b4ce00640 1 -- 172.21.15.97:0/2703156949 learned_addr learned my addr 172.21.15.97:0/2703156949 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:54:11.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.880+0000 7f1b47e00640 1 --2- 172.21.15.97:0/2703156949 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1b48103d90 0x7f1b48199f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:11.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.880+0000 7f1b47400640 1 --2- 172.21.15.97:0/2703156949 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1b48104f80 0x7f1b4819a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:11.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.880+0000 7f1b4ce00640 1 -- 172.21.15.97:0/2703156949 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1b48104f80 msgr2=0x7f1b4819a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:11.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.880+0000 7f1b4ce00640 1 --2- 172.21.15.97:0/2703156949 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1b48104f80 0x7f1b4819a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:11.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.880+0000 7f1b4ce00640 1 -- 172.21.15.97:0/2703156949 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1b48103d90 msgr2=0x7f1b48199f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:11.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.880+0000 7f1b4ce00640 1 --2- 172.21.15.97:0/2703156949 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1b48103d90 0x7f1b48199f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:11.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.880+0000 7f1b4ce00640 1 -- 172.21.15.97:0/2703156949 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b481a1bb0 con 0x7f1b48105940 2024-03-27T10:54:11.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.880+0000 7f1b47e00640 1 --2- 172.21.15.97:0/2703156949 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1b48103d90 0x7f1b48199f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T10:54:11.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.880+0000 7f1b4ce00640 1 --2- 172.21.15.97:0/2703156949 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b48105940 0x7f1b481a1540 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1b4000cce0 tx=0x7f1b4000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:54:11.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.881+0000 7f1b44c00640 1 -- 172.21.15.97:0/2703156949 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b4000f450 con 0x7f1b48105940 2024-03-27T10:54:11.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.881+0000 7f1b44c00640 1 -- 172.21.15.97:0/2703156949 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1b4000faa0 con 0x7f1b48105940 2024-03-27T10:54:11.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.881+0000 7f1b44c00640 1 -- 172.21.15.97:0/2703156949 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b40018b70 con 0x7f1b48105940 2024-03-27T10:54:11.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.881+0000 7f1b4ee00640 1 -- 172.21.15.97:0/2703156949 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b481a1ea0 con 0x7f1b48105940 2024-03-27T10:54:11.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.881+0000 7f1b4ee00640 1 -- 172.21.15.97:0/2703156949 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b481a2360 con 0x7f1b48105940 2024-03-27T10:54:11.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.882+0000 7f1b4ee00640 1 -- 172.21.15.97:0/2703156949 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b4810b1d0 con 0x7f1b48105940 2024-03-27T10:54:11.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.884+0000 7f1b44c00640 1 -- 172.21.15.97:0/2703156949 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138370+0+0 (secure 0 0 0) 0x7f1b40018d10 con 0x7f1b48105940 2024-03-27T10:54:11.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.885+0000 7f1b44c00640 1 --2- 172.21.15.97:0/2703156949 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f1b2c0a3690 0x7f1b2c0a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:11.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.885+0000 7f1b44c00640 1 -- 172.21.15.97:0/2703156949 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 7979+0+0 (secure 0 0 0) 0x7f1b400d00f0 con 0x7f1b48105940 2024-03-27T10:54:11.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.885+0000 7f1b47e00640 1 --2- 172.21.15.97:0/2703156949 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f1b2c0a3690 0x7f1b2c0a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:11.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.886+0000 7f1b47e00640 1 --2- 172.21.15.97:0/2703156949 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f1b2c0a3690 0x7f1b2c0a5b50 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1b3802c240 tx=0x7f1b38005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:54:11.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:11.889+0000 7f1b44c00640 1 -- 172.21.15.97:0/2703156949 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f1b40014030 con 0x7f1b48105940 2024-03-27T10:54:12.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.124+0000 7f1b4ee00640 1 -- 172.21.15.97:0/2703156949 --> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f1b48061f90 con 0x7f1b2c0a3690 2024-03-27T10:54:12.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.186+0000 7f1b44c00640 1 -- 172.21.15.97:0/2703156949 <== mgr.14152 v2:172.21.15.97:6800/3955463538 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f1b48061f90 con 0x7f1b2c0a3690 2024-03-27T10:54:12.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.189+0000 7f1b4ee00640 1 -- 172.21.15.97:0/2703156949 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f1b2c0a3690 msgr2=0x7f1b2c0a5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:12.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.189+0000 7f1b4ee00640 1 --2- 172.21.15.97:0/2703156949 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f1b2c0a3690 0x7f1b2c0a5b50 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1b3802c240 tx=0x7f1b38005fd0 comp rx=0 tx=0).stop 2024-03-27T10:54:12.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.189+0000 7f1b4ee00640 1 -- 172.21.15.97:0/2703156949 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b48105940 msgr2=0x7f1b481a1540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:12.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.189+0000 7f1b4ee00640 1 --2- 172.21.15.97:0/2703156949 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b48105940 0x7f1b481a1540 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1b4000cce0 tx=0x7f1b4000a620 comp rx=0 tx=0).stop 2024-03-27T10:54:12.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.190+0000 7f1b4ee00640 1 -- 172.21.15.97:0/2703156949 shutdown_connections 2024-03-27T10:54:12.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.190+0000 7f1b4ee00640 1 --2- 172.21.15.97:0/2703156949 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b48105940 0x7f1b481a1540 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:12.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.190+0000 7f1b4ee00640 1 --2- 172.21.15.97:0/2703156949 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1b48104f80 0x7f1b4819a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:12.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.190+0000 7f1b4ee00640 1 --2- 172.21.15.97:0/2703156949 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f1b2c0a3690 0x7f1b2c0a5b50 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:12.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.190+0000 7f1b4ee00640 1 --2- 172.21.15.97:0/2703156949 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1b48103d90 0x7f1b48199f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:12.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.190+0000 7f1b4ee00640 1 -- 172.21.15.97:0/2703156949 >> 172.21.15.97:0/2703156949 conn(0x7f1b480ff300 msgr2=0x7f1b481083a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:54:12.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.190+0000 7f1b4ee00640 1 -- 172.21.15.97:0/2703156949 shutdown_connections 2024-03-27T10:54:12.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.190+0000 7f1b4ee00640 1 -- 172.21.15.97:0/2703156949 wait complete. 2024-03-27T10:54:12.183 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-03-27T10:54:12.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.402+0000 7ff51c400640 1 -- 172.21.15.97:0/2647607951 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff514109a20 msgr2=0x7ff51410beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:12.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.402+0000 7ff51c400640 1 --2- 172.21.15.97:0/2647607951 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff514109a20 0x7ff51410beb0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff50c009ca0 tx=0x7ff50c02c0f0 comp rx=0 tx=0).stop 2024-03-27T10:54:12.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.402+0000 7ff51c400640 1 -- 172.21.15.97:0/2647607951 shutdown_connections 2024-03-27T10:54:12.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.402+0000 7ff51c400640 1 --2- 172.21.15.97:0/2647607951 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff514109a20 0x7ff51410beb0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:12.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.402+0000 7ff51c400640 1 --2- 172.21.15.97:0/2647607951 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff5140745b0 0x7ff514074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:12.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.402+0000 7ff51c400640 1 --2- 172.21.15.97:0/2647607951 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff514075b90 0x7ff514073fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:12.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.402+0000 7ff51c400640 1 -- 172.21.15.97:0/2647607951 >> 172.21.15.97:0/2647607951 conn(0x7ff5140fd0f0 msgr2=0x7ff5140ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:54:12.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.402+0000 7ff51c400640 1 -- 172.21.15.97:0/2647607951 shutdown_connections 2024-03-27T10:54:12.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.402+0000 7ff51c400640 1 -- 172.21.15.97:0/2647607951 wait complete. 2024-03-27T10:54:12.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.403+0000 7ff51c400640 1 Processor -- start 2024-03-27T10:54:12.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.403+0000 7ff51c400640 1 -- start start 2024-03-27T10:54:12.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.403+0000 7ff51c400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff5140745b0 0x7ff514199f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:12.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.403+0000 7ff51c400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff514075b90 0x7ff51419a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:12.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.403+0000 7ff51c400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff514109a20 0x7ff5141a14d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:12.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.403+0000 7ff51c400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff51410f850 con 0x7ff514075b90 2024-03-27T10:54:12.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.403+0000 7ff51c400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff51410f6d0 con 0x7ff5140745b0 2024-03-27T10:54:12.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.403+0000 7ff51c400640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff51410f9d0 con 0x7ff514109a20 2024-03-27T10:54:12.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.404+0000 7ff519000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff514075b90 0x7ff51419a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:12.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.404+0000 7ff519000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff514075b90 0x7ff51419a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43198/0 (socket says 172.21.15.97:43198) 2024-03-27T10:54:12.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.404+0000 7ff519000640 1 -- 172.21.15.97:0/752212200 learned_addr learned my addr 172.21.15.97:0/752212200 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:54:12.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.404+0000 7ff51a400640 1 --2- 172.21.15.97:0/752212200 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff514109a20 0x7ff5141a14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:12.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.404+0000 7ff519a00640 1 --2- 172.21.15.97:0/752212200 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff5140745b0 0x7ff514199f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:12.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.404+0000 7ff519000640 1 -- 172.21.15.97:0/752212200 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff514109a20 msgr2=0x7ff5141a14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:12.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.404+0000 7ff519000640 1 --2- 172.21.15.97:0/752212200 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff514109a20 0x7ff5141a14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:12.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.404+0000 7ff519000640 1 -- 172.21.15.97:0/752212200 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff5140745b0 msgr2=0x7ff514199f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:12.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.404+0000 7ff519000640 1 --2- 172.21.15.97:0/752212200 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff5140745b0 0x7ff514199f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:12.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.404+0000 7ff519000640 1 -- 172.21.15.97:0/752212200 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5141a1b40 con 0x7ff514075b90 2024-03-27T10:54:12.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.404+0000 7ff519000640 1 --2- 172.21.15.97:0/752212200 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff514075b90 0x7ff51419a460 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff50000e9f0 tx=0x7ff50000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:54:12.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.404+0000 7ff512000640 1 -- 172.21.15.97:0/752212200 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff50000cd80 con 0x7ff514075b90 2024-03-27T10:54:12.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.405+0000 7ff512000640 1 -- 172.21.15.97:0/752212200 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff5000108e0 con 0x7ff514075b90 2024-03-27T10:54:12.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.405+0000 7ff51c400640 1 -- 172.21.15.97:0/752212200 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5141a1e30 con 0x7ff514075b90 2024-03-27T10:54:12.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.405+0000 7ff512000640 1 -- 172.21.15.97:0/752212200 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff500018920 con 0x7ff514075b90 2024-03-27T10:54:12.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.405+0000 7ff51c400640 1 -- 172.21.15.97:0/752212200 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5141a22f0 con 0x7ff514075b90 2024-03-27T10:54:12.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.406+0000 7ff51c400640 1 -- 172.21.15.97:0/752212200 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff514108750 con 0x7ff514075b90 2024-03-27T10:54:12.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.408+0000 7ff512000640 1 -- 172.21.15.97:0/752212200 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138370+0+0 (secure 0 0 0) 0x7ff500010a90 con 0x7ff514075b90 2024-03-27T10:54:12.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.410+0000 7ff512000640 1 --2- 172.21.15.97:0/752212200 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7ff4fc0a3610 0x7ff4fc0a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:12.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.410+0000 7ff512000640 1 -- 172.21.15.97:0/752212200 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 7979+0+0 (secure 0 0 0) 0x7ff5000cff50 con 0x7ff514075b90 2024-03-27T10:54:12.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.410+0000 7ff519a00640 1 --2- 172.21.15.97:0/752212200 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7ff4fc0a3610 0x7ff4fc0a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:12.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.411+0000 7ff519a00640 1 --2- 172.21.15.97:0/752212200 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7ff4fc0a3610 0x7ff4fc0a5ad0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff508009b60 tx=0x7ff508005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:54:12.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.418+0000 7ff512000640 1 -- 172.21.15.97:0/752212200 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7ff50009cc70 con 0x7ff514075b90 2024-03-27T10:54:12.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:12 smithi097 ceph-mon[31859]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T10:54:12.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:12 smithi097 ceph-mon[31859]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T10:54:12.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:12 smithi097 ceph-mon[31859]: mds.? [v2:172.21.15.177:6834/3349112132,v1:172.21.15.177:6835/3349112132] up:active 2024-03-27T10:54:12.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:12 smithi097 ceph-mon[31859]: daemon mds.l assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-03-27T10:54:12.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:12 smithi097 ceph-mon[31859]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-27T10:54:12.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:12 smithi097 ceph-mon[31859]: Cluster is now healthy 2024-03-27T10:54:12.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:12 smithi097 ceph-mon[31859]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=e=up:active,3=f=up:active} 3 up:standby-replay 5 up:standby 2024-03-27T10:54:12.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:12 smithi097 ceph-mon[31859]: fsmap cephfs:5 3 up:standby-replay 1 up:creating 4 up:standby 4 up:active 2024-03-27T10:54:12.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:12 smithi097 ceph-mon[31859]: daemon mds.l is now active in filesystem cephfs as rank 4 2024-03-27T10:54:12.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:12 smithi177 ceph-mon[35294]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T10:54:12.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:12 smithi177 ceph-mon[35294]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T10:54:12.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:12 smithi177 ceph-mon[35294]: mds.? [v2:172.21.15.177:6834/3349112132,v1:172.21.15.177:6835/3349112132] up:active 2024-03-27T10:54:12.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:12 smithi177 ceph-mon[35294]: daemon mds.l assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-03-27T10:54:12.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:12 smithi177 ceph-mon[35294]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-27T10:54:12.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:12 smithi177 ceph-mon[35294]: Cluster is now healthy 2024-03-27T10:54:12.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:12 smithi177 ceph-mon[35294]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=e=up:active,3=f=up:active} 3 up:standby-replay 5 up:standby 2024-03-27T10:54:12.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:12 smithi177 ceph-mon[35294]: fsmap cephfs:5 3 up:standby-replay 1 up:creating 4 up:standby 4 up:active 2024-03-27T10:54:12.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:12 smithi177 ceph-mon[35294]: daemon mds.l is now active in filesystem cephfs as rank 4 2024-03-27T10:54:12.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:12 smithi114 ceph-mon[35338]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T10:54:12.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:12 smithi114 ceph-mon[35338]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T10:54:12.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:12 smithi114 ceph-mon[35338]: mds.? [v2:172.21.15.177:6834/3349112132,v1:172.21.15.177:6835/3349112132] up:active 2024-03-27T10:54:12.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:12 smithi114 ceph-mon[35338]: daemon mds.l assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-03-27T10:54:12.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:12 smithi114 ceph-mon[35338]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-27T10:54:12.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:12 smithi114 ceph-mon[35338]: Cluster is now healthy 2024-03-27T10:54:12.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:12 smithi114 ceph-mon[35338]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=e=up:active,3=f=up:active} 3 up:standby-replay 5 up:standby 2024-03-27T10:54:12.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:12 smithi114 ceph-mon[35338]: fsmap cephfs:5 3 up:standby-replay 1 up:creating 4 up:standby 4 up:active 2024-03-27T10:54:12.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:12 smithi114 ceph-mon[35338]: daemon mds.l is now active in filesystem cephfs as rank 4 2024-03-27T10:54:12.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.646+0000 7ff51c400640 1 -- 172.21.15.97:0/752212200 --> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7ff514062020 con 0x7ff4fc0a3610 2024-03-27T10:54:12.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.653+0000 7ff512000640 1 -- 172.21.15.97:0/752212200 <== mgr.14152 v2:172.21.15.97:6800/3955463538 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7ff514062020 con 0x7ff4fc0a3610 2024-03-27T10:54:12.626 INFO:teuthology.orchestra.run.smithi097.stdout:/volumes/qa/sv_1/44398619-4f71-4f0c-990c-3a021a08a282 2024-03-27T10:54:12.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.655+0000 7ff51c400640 1 -- 172.21.15.97:0/752212200 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7ff4fc0a3610 msgr2=0x7ff4fc0a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:12.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.656+0000 7ff51c400640 1 --2- 172.21.15.97:0/752212200 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7ff4fc0a3610 0x7ff4fc0a5ad0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff508009b60 tx=0x7ff508005f90 comp rx=0 tx=0).stop 2024-03-27T10:54:12.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.656+0000 7ff51c400640 1 -- 172.21.15.97:0/752212200 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff514075b90 msgr2=0x7ff51419a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:12.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.656+0000 7ff51c400640 1 --2- 172.21.15.97:0/752212200 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff514075b90 0x7ff51419a460 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff50000e9f0 tx=0x7ff50000eeb0 comp rx=0 tx=0).stop 2024-03-27T10:54:12.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.656+0000 7ff51c400640 1 -- 172.21.15.97:0/752212200 shutdown_connections 2024-03-27T10:54:12.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.656+0000 7ff51c400640 1 --2- 172.21.15.97:0/752212200 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff514109a20 0x7ff5141a14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:12.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.656+0000 7ff51c400640 1 --2- 172.21.15.97:0/752212200 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff514075b90 0x7ff51419a460 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:12.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.656+0000 7ff51c400640 1 --2- 172.21.15.97:0/752212200 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7ff4fc0a3610 0x7ff4fc0a5ad0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:12.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.656+0000 7ff51c400640 1 --2- 172.21.15.97:0/752212200 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff5140745b0 0x7ff514199f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:12.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.656+0000 7ff51c400640 1 -- 172.21.15.97:0/752212200 >> 172.21.15.97:0/752212200 conn(0x7ff5140fd0f0 msgr2=0x7ff514105920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:54:12.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.656+0000 7ff51c400640 1 -- 172.21.15.97:0/752212200 shutdown_connections 2024-03-27T10:54:12.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.656+0000 7ff51c400640 1 -- 172.21.15.97:0/752212200 wait complete. 2024-03-27T10:54:12.646 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-27T10:54:12.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.879+0000 7f3117800640 1 -- 172.21.15.97:0/1794719133 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31101005b0 msgr2=0x7f311010a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:12.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.879+0000 7f3117800640 1 --2- 172.21.15.97:0/1794719133 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31101005b0 0x7f311010a160 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f3108009ca0 tx=0x7f310802c100 comp rx=0 tx=0).stop 2024-03-27T10:54:12.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.880+0000 7f3117800640 1 -- 172.21.15.97:0/1794719133 shutdown_connections 2024-03-27T10:54:12.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.880+0000 7f3117800640 1 --2- 172.21.15.97:0/1794719133 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31101005b0 0x7f311010a160 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:12.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.880+0000 7f3117800640 1 --2- 172.21.15.97:0/1794719133 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f31100ffbf0 0x7f3110100070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:12.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.880+0000 7f3117800640 1 --2- 172.21.15.97:0/1794719133 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3110103180 0x7f31101035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:12.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.880+0000 7f3117800640 1 -- 172.21.15.97:0/1794719133 >> 172.21.15.97:0/1794719133 conn(0x7f31100fb730 msgr2=0x7f31100fdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:54:12.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.880+0000 7f3117800640 1 -- 172.21.15.97:0/1794719133 shutdown_connections 2024-03-27T10:54:12.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.880+0000 7f3117800640 1 -- 172.21.15.97:0/1794719133 wait complete. 2024-03-27T10:54:12.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.881+0000 7f3117800640 1 Processor -- start 2024-03-27T10:54:12.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.881+0000 7f3117800640 1 -- start start 2024-03-27T10:54:12.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.881+0000 7f3117800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31100ffbf0 0x7f3110197d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:12.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.881+0000 7f3117800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f31101005b0 0x7f3110198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:12.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.881+0000 7f3117800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3110103180 0x7f311019f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:12.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.881+0000 7f3117800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f311010d690 con 0x7f31100ffbf0 2024-03-27T10:54:12.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.881+0000 7f3117800640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f311010d510 con 0x7f3110103180 2024-03-27T10:54:12.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.881+0000 7f3117800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f311010d810 con 0x7f31101005b0 2024-03-27T10:54:12.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.881+0000 7f3114e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31100ffbf0 0x7f3110197d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:12.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.881+0000 7f3114e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31100ffbf0 0x7f3110197d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43226/0 (socket says 172.21.15.97:43226) 2024-03-27T10:54:12.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.881+0000 7f3114e00640 1 -- 172.21.15.97:0/491441007 learned_addr learned my addr 172.21.15.97:0/491441007 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:54:12.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.881+0000 7f3115800640 1 --2- 172.21.15.97:0/491441007 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3110103180 0x7f311019f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:12.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.881+0000 7f3107e00640 1 --2- 172.21.15.97:0/491441007 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f31101005b0 0x7f3110198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:12.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.882+0000 7f3114e00640 1 -- 172.21.15.97:0/491441007 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f31101005b0 msgr2=0x7f3110198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:12.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.882+0000 7f3114e00640 1 --2- 172.21.15.97:0/491441007 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f31101005b0 0x7f3110198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:12.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.882+0000 7f3114e00640 1 -- 172.21.15.97:0/491441007 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3110103180 msgr2=0x7f311019f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:12.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.882+0000 7f3114e00640 1 --2- 172.21.15.97:0/491441007 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3110103180 0x7f311019f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:12.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.882+0000 7f3114e00640 1 -- 172.21.15.97:0/491441007 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f311019f960 con 0x7f31100ffbf0 2024-03-27T10:54:12.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.882+0000 7f3115800640 1 --2- 172.21.15.97:0/491441007 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3110103180 0x7f311019f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T10:54:12.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.882+0000 7f3114e00640 1 --2- 172.21.15.97:0/491441007 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31100ffbf0 0x7f3110197d50 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f30f800e9c0 tx=0x7f30f800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:54:12.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.882+0000 7f310e000640 1 -- 172.21.15.97:0/491441007 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f800cd80 con 0x7f31100ffbf0 2024-03-27T10:54:12.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.882+0000 7f310e000640 1 -- 172.21.15.97:0/491441007 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f30f80035f0 con 0x7f31100ffbf0 2024-03-27T10:54:12.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.882+0000 7f310e000640 1 -- 172.21.15.97:0/491441007 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f8010690 con 0x7f31100ffbf0 2024-03-27T10:54:12.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.882+0000 7f3117800640 1 -- 172.21.15.97:0/491441007 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f311019fc50 con 0x7f31100ffbf0 2024-03-27T10:54:12.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.882+0000 7f3117800640 1 -- 172.21.15.97:0/491441007 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31101a0150 con 0x7f31100ffbf0 2024-03-27T10:54:12.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.883+0000 7f3117800640 1 -- 172.21.15.97:0/491441007 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3110109040 con 0x7f31100ffbf0 2024-03-27T10:54:12.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.885+0000 7f310e000640 1 -- 172.21.15.97:0/491441007 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138370+0+0 (secure 0 0 0) 0x7f30f80037a0 con 0x7f31100ffbf0 2024-03-27T10:54:12.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.887+0000 7f310e000640 1 --2- 172.21.15.97:0/491441007 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f30f40a3690 0x7f30f40a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:12.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.888+0000 7f310e000640 1 -- 172.21.15.97:0/491441007 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 7979+0+0 (secure 0 0 0) 0x7f30f80d0150 con 0x7f31100ffbf0 2024-03-27T10:54:12.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.888+0000 7f3107e00640 1 --2- 172.21.15.97:0/491441007 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f30f40a3690 0x7f30f40a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:12.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.888+0000 7f3107e00640 1 --2- 172.21.15.97:0/491441007 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f30f40a3690 0x7f30f40a5b50 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3100005fd0 tx=0x7f3100005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:54:12.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:12.892+0000 7f310e000640 1 -- 172.21.15.97:0/491441007 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f30f809cdd0 con 0x7f31100ffbf0 2024-03-27T10:54:13.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.139+0000 7f3117800640 1 -- 172.21.15.97:0/491441007 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f311002d370 con 0x7f31100ffbf0 2024-03-27T10:54:13.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.141+0000 7f310e000640 1 -- 172.21.15.97:0/491441007 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11944 (secure 0 0 0) 0x7f30f8098030 con 0x7f31100ffbf0 2024-03-27T10:54:13.114 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T10:54:13.114 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1009423904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1009423904},{"type":"v1","addr":"172.21.15.97:6835","nonce":1009423904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3251173433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3251173433},{"type":"v1","addr":"172.21.15.97:6837","nonce":3251173433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-27T10:54:04.510178+0000","modified":"2024-03-27T10:54:12.180047+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34508,"mds_1":34478,"mds_2":34445,"mds_3":24454,"mds_4":24400},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14646":{"gid":14646,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6839/1057596821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1057596821},{"type":"v1","addr":"172.21.15.97:6839","nonce":1057596821}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14658":{"gid":14658,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6841/1672464856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6840","nonce":1672464856},{"type":"v1","addr":"172.21.15.97:6841","nonce":1672464856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.177:6833/3787981632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3787981632},{"type":"v1","addr":"172.21.15.177:6833","nonce":3787981632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.177:6835/3349112132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":3349112132},{"type":"v1","addr":"172.21.15.177:6835","nonce":3349112132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6837/3244432640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":3244432640},{"type":"v1","addr":"172.21.15.177:6837","nonce":3244432640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6839/3340454605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":3340454605},{"type":"v1","addr":"172.21.15.177:6839","nonce":3340454605}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34415":{"gid":34415,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6833/4052517902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":4052517902},{"type":"v1","addr":"172.21.15.114:6833","nonce":4052517902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34445":{"gid":34445,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.114:6835/3581918222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3581918222},{"type":"v1","addr":"172.21.15.114:6835","nonce":3581918222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34478":{"gid":34478,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.114:6837/616775588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":616775588},{"type":"v1","addr":"172.21.15.114:6837","nonce":616775588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34508":{"gid":34508,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.114:6839/3937193784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":3937193784},{"type":"v1","addr":"172.21.15.114:6839","nonce":3937193784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34508,"qdb_cluster":[34445,34508,24400,24454,34478]},"id":1}]} 2024-03-27T10:54:13.114 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 27 2024-03-27T10:54:13.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.143+0000 7f3117800640 1 -- 172.21.15.97:0/491441007 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f30f40a3690 msgr2=0x7f30f40a5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:13.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.143+0000 7f3117800640 1 --2- 172.21.15.97:0/491441007 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f30f40a3690 0x7f30f40a5b50 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3100005fd0 tx=0x7f3100005850 comp rx=0 tx=0).stop 2024-03-27T10:54:13.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.143+0000 7f3117800640 1 -- 172.21.15.97:0/491441007 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31100ffbf0 msgr2=0x7f3110197d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:13.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.143+0000 7f3117800640 1 --2- 172.21.15.97:0/491441007 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31100ffbf0 0x7f3110197d50 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f30f800e9c0 tx=0x7f30f800ee80 comp rx=0 tx=0).stop 2024-03-27T10:54:13.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.144+0000 7f3117800640 1 -- 172.21.15.97:0/491441007 shutdown_connections 2024-03-27T10:54:13.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.144+0000 7f3117800640 1 --2- 172.21.15.97:0/491441007 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f30f40a3690 0x7f30f40a5b50 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:13.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.144+0000 7f3117800640 1 --2- 172.21.15.97:0/491441007 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3110103180 0x7f311019f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:13.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.144+0000 7f3117800640 1 --2- 172.21.15.97:0/491441007 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f31101005b0 0x7f3110198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:13.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.144+0000 7f3117800640 1 --2- 172.21.15.97:0/491441007 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31100ffbf0 0x7f3110197d50 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:13.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.144+0000 7f3117800640 1 -- 172.21.15.97:0/491441007 >> 172.21.15.97:0/491441007 conn(0x7f31100fb730 msgr2=0x7f3110106210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:54:13.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.144+0000 7f3117800640 1 -- 172.21.15.97:0/491441007 shutdown_connections 2024-03-27T10:54:13.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.144+0000 7f3117800640 1 -- 172.21.15.97:0/491441007 wait complete. 2024-03-27T10:54:13.137 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-27T10:54:13.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.368+0000 7f4bbee00640 1 -- 172.21.15.97:0/2013970038 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4bb8107640 msgr2=0x7f4bb8109a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:13.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.368+0000 7f4bbee00640 1 --2- 172.21.15.97:0/2013970038 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4bb8107640 0x7f4bb8109a90 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f4bb0009ca0 tx=0x7f4bb002c0f0 comp rx=0 tx=0).stop 2024-03-27T10:54:13.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.368+0000 7f4bbee00640 1 -- 172.21.15.97:0/2013970038 shutdown_connections 2024-03-27T10:54:13.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.368+0000 7f4bbee00640 1 --2- 172.21.15.97:0/2013970038 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4bb8107640 0x7f4bb8109a90 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:13.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.368+0000 7f4bbee00640 1 --2- 172.21.15.97:0/2013970038 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4bb8104d10 0x7f4bb8107100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:13.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.368+0000 7f4bbee00640 1 --2- 172.21.15.97:0/2013970038 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4bb81023e0 0x7f4bb81047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:13.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.368+0000 7f4bbee00640 1 -- 172.21.15.97:0/2013970038 >> 172.21.15.97:0/2013970038 conn(0x7f4bb80fbdd0 msgr2=0x7f4bb80fe210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:54:13.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.368+0000 7f4bbee00640 1 -- 172.21.15.97:0/2013970038 shutdown_connections 2024-03-27T10:54:13.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.368+0000 7f4bbee00640 1 -- 172.21.15.97:0/2013970038 wait complete. 2024-03-27T10:54:13.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.369+0000 7f4bbee00640 1 Processor -- start 2024-03-27T10:54:13.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.369+0000 7f4bbee00640 1 -- start start 2024-03-27T10:54:13.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.369+0000 7f4bbee00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4bb81023e0 0x7f4bb8199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:13.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.369+0000 7f4bbee00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4bb8104d10 0x7f4bb819a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:13.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.369+0000 7f4bbee00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4bb8107640 0x7f4bb81a12b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:13.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.369+0000 7f4bbee00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb810d490 con 0x7f4bb8104d10 2024-03-27T10:54:13.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.369+0000 7f4bbee00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb810d310 con 0x7f4bb81023e0 2024-03-27T10:54:13.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.369+0000 7f4bbee00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb810d610 con 0x7f4bb8107640 2024-03-27T10:54:13.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.370+0000 7f4bbd000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4bb8104d10 0x7f4bb819a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:13.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.370+0000 7f4bbd000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4bb8104d10 0x7f4bb819a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43250/0 (socket says 172.21.15.97:43250) 2024-03-27T10:54:13.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.370+0000 7f4bbd000640 1 -- 172.21.15.97:0/2453445887 learned_addr learned my addr 172.21.15.97:0/2453445887 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:54:13.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.370+0000 7f4bbe400640 1 --2- 172.21.15.97:0/2453445887 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4bb8107640 0x7f4bb81a12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:13.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.370+0000 7f4bbda00640 1 --2- 172.21.15.97:0/2453445887 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4bb81023e0 0x7f4bb8199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:13.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.370+0000 7f4bbd000640 1 -- 172.21.15.97:0/2453445887 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4bb8107640 msgr2=0x7f4bb81a12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:13.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.370+0000 7f4bbd000640 1 --2- 172.21.15.97:0/2453445887 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4bb8107640 0x7f4bb81a12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:13.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.370+0000 7f4bbd000640 1 -- 172.21.15.97:0/2453445887 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4bb81023e0 msgr2=0x7f4bb8199d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:13.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.370+0000 7f4bbd000640 1 --2- 172.21.15.97:0/2453445887 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4bb81023e0 0x7f4bb8199d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:13.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.370+0000 7f4bbd000640 1 -- 172.21.15.97:0/2453445887 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bb81a1950 con 0x7f4bb8104d10 2024-03-27T10:54:13.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.370+0000 7f4bbe400640 1 --2- 172.21.15.97:0/2453445887 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4bb8107640 0x7f4bb81a12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T10:54:13.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.370+0000 7f4bbd000640 1 --2- 172.21.15.97:0/2453445887 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4bb8104d10 0x7f4bb819a280 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4ba400e9c0 tx=0x7f4ba400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:54:13.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.370+0000 7f4bb6000640 1 -- 172.21.15.97:0/2453445887 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ba400cd80 con 0x7f4bb8104d10 2024-03-27T10:54:13.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.371+0000 7f4bb6000640 1 -- 172.21.15.97:0/2453445887 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4ba40054d0 con 0x7f4bb8104d10 2024-03-27T10:54:13.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.371+0000 7f4bbee00640 1 -- 172.21.15.97:0/2453445887 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bb81a1c40 con 0x7f4bb8104d10 2024-03-27T10:54:13.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.371+0000 7f4bb6000640 1 -- 172.21.15.97:0/2453445887 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ba4010430 con 0x7f4bb8104d10 2024-03-27T10:54:13.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.371+0000 7f4bbee00640 1 -- 172.21.15.97:0/2453445887 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bb81a2190 con 0x7f4bb8104d10 2024-03-27T10:54:13.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.372+0000 7f4bbee00640 1 -- 172.21.15.97:0/2453445887 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bb8109130 con 0x7f4bb8104d10 2024-03-27T10:54:13.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.375+0000 7f4bb6000640 1 -- 172.21.15.97:0/2453445887 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138421+0+0 (secure 0 0 0) 0x7f4ba4004fd0 con 0x7f4bb8104d10 2024-03-27T10:54:13.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.376+0000 7f4bb6000640 1 --2- 172.21.15.97:0/2453445887 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f4ba00a36a0 0x7f4ba00a5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:13.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.376+0000 7f4bb6000640 1 -- 172.21.15.97:0/2453445887 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 7979+0+0 (secure 0 0 0) 0x7f4ba40d0370 con 0x7f4bb8104d10 2024-03-27T10:54:13.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.376+0000 7f4bbda00640 1 --2- 172.21.15.97:0/2453445887 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f4ba00a36a0 0x7f4ba00a5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:13.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.377+0000 7f4bbda00640 1 --2- 172.21.15.97:0/2453445887 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f4ba00a36a0 0x7f4ba00a5b60 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f4bac005d00 tx=0x7f4bac005c70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:54:13.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.380+0000 7f4bb6000640 1 -- 172.21.15.97:0/2453445887 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f4ba400b2c0 con 0x7f4bb8104d10 2024-03-27T10:54:13.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:13 smithi097 ceph-mon[31859]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T10:54:13.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:13 smithi097 ceph-mon[31859]: pgmap v366: 97 pgs: 97 active+clean; 580 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 8 op/s 2024-03-27T10:54:13.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:13 smithi097 ceph-mon[31859]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T10:54:13.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:13 smithi097 ceph-mon[31859]: mds.? [v2:172.21.15.177:6832/3787981632,v1:172.21.15.177:6833/3787981632] up:active 2024-03-27T10:54:13.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:13 smithi097 ceph-mon[31859]: fsmap cephfs:5 3 up:standby-replay 4 up:standby 5 up:active 2024-03-27T10:54:13.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:13 smithi097 ceph-mon[31859]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T10:54:13.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:13 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/491441007' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T10:54:13.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:13 smithi177 ceph-mon[35294]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T10:54:13.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:13 smithi177 ceph-mon[35294]: pgmap v366: 97 pgs: 97 active+clean; 580 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 8 op/s 2024-03-27T10:54:13.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:13 smithi177 ceph-mon[35294]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T10:54:13.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:13 smithi177 ceph-mon[35294]: mds.? [v2:172.21.15.177:6832/3787981632,v1:172.21.15.177:6833/3787981632] up:active 2024-03-27T10:54:13.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:13 smithi177 ceph-mon[35294]: fsmap cephfs:5 3 up:standby-replay 4 up:standby 5 up:active 2024-03-27T10:54:13.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:13 smithi177 ceph-mon[35294]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T10:54:13.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:13 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/491441007' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T10:54:13.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:13 smithi114 ceph-mon[35338]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T10:54:13.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:13 smithi114 ceph-mon[35338]: pgmap v366: 97 pgs: 97 active+clean; 580 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 8 op/s 2024-03-27T10:54:13.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:13 smithi114 ceph-mon[35338]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T10:54:13.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:13 smithi114 ceph-mon[35338]: mds.? [v2:172.21.15.177:6832/3787981632,v1:172.21.15.177:6833/3787981632] up:active 2024-03-27T10:54:13.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:13 smithi114 ceph-mon[35338]: fsmap cephfs:5 3 up:standby-replay 4 up:standby 5 up:active 2024-03-27T10:54:13.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:13 smithi114 ceph-mon[35338]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T10:54:13.518 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:13 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/491441007' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T10:54:13.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.581+0000 7f4bbee00640 1 -- 172.21.15.97:0/2453445887 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4bb802d590 con 0x7f4bb8104d10 2024-03-27T10:54:13.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.584+0000 7f4bb6000640 1 -- 172.21.15.97:0/2453445887 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22211 (secure 0 0 0) 0x7f4ba409ce90 con 0x7f4bb8104d10 2024-03-27T10:54:13.557 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T10:54:13.557 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":90,"fsid":"adca88c4-ec26-11ee-95d0-87774f69a715","created":"2024-03-27T10:43:24.680694+0000","modified":"2024-03-27T10:54:06.066477+0000","last_up_change":"2024-03-27T10:53:00.552513+0000","last_in_change":"2024-03-27T10:52:36.302986+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-27T10:47:54.052810+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-27T10:54:02.845060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-27T10:54:03.474151+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b634e5f1-6118-4e2f-95e9-830db390f8d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1746812080},{"type":"v1","addr":"172.21.15.97:6803","nonce":1746812080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1746812080},{"type":"v1","addr":"172.21.15.97:6805","nonce":1746812080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":1746812080},{"type":"v1","addr":"172.21.15.97:6809","nonce":1746812080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1746812080},{"type":"v1","addr":"172.21.15.97:6807","nonce":1746812080}]},"public_addr":"172.21.15.97:6803/1746812080","cluster_addr":"172.21.15.97:6805/1746812080","heartbeat_back_addr":"172.21.15.97:6809/1746812080","heartbeat_front_addr":"172.21.15.97:6807/1746812080","state":["exists","up"]},{"osd":1,"uuid":"441e2770-3284-4388-b0de-c56eb8a373d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":3858821264},{"type":"v1","addr":"172.21.15.114:6801","nonce":3858821264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":3858821264},{"type":"v1","addr":"172.21.15.114:6803","nonce":3858821264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":3858821264},{"type":"v1","addr":"172.21.15.114:6807","nonce":3858821264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":3858821264},{"type":"v1","addr":"172.21.15.114:6805","nonce":3858821264}]},"public_addr":"172.21.15.114:6801/3858821264","cluster_addr":"172.21.15.114:6803/3858821264","heartbeat_back_addr":"172.21.15.114:6807/3858821264","heartbeat_front_addr":"172.21.15.114:6805/3858821264","state":["exists","up"]},{"osd":2,"uuid":"00108864-1aa6-48d6-8398-e8eb79fda123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":124240658},{"type":"v1","addr":"172.21.15.177:6801","nonce":124240658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":124240658},{"type":"v1","addr":"172.21.15.177:6803","nonce":124240658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":124240658},{"type":"v1","addr":"172.21.15.177:6807","nonce":124240658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":124240658},{"type":"v1","addr":"172.21.15.177:6805","nonce":124240658}]},"public_addr":"172.21.15.177:6801/124240658","cluster_addr":"172.21.15.177:6803/124240658","heartbeat_back_addr":"172.21.15.177:6807/124240658","heartbeat_front_addr":"172.21.15.177:6805/124240658","state":["exists","up"]},{"osd":3,"uuid":"1860b690-e2dc-4977-88d7-d1aacf7022ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2478179311},{"type":"v1","addr":"172.21.15.97:6811","nonce":2478179311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2478179311},{"type":"v1","addr":"172.21.15.97:6813","nonce":2478179311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":2478179311},{"type":"v1","addr":"172.21.15.97:6817","nonce":2478179311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2478179311},{"type":"v1","addr":"172.21.15.97:6815","nonce":2478179311}]},"public_addr":"172.21.15.97:6811/2478179311","cluster_addr":"172.21.15.97:6813/2478179311","heartbeat_back_addr":"172.21.15.97:6817/2478179311","heartbeat_front_addr":"172.21.15.97:6815/2478179311","state":["exists","up"]},{"osd":4,"uuid":"9245a3a4-7d49-4fdb-9d55-112e43177050","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":75755945},{"type":"v1","addr":"172.21.15.114:6809","nonce":75755945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":75755945},{"type":"v1","addr":"172.21.15.114:6811","nonce":75755945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":75755945},{"type":"v1","addr":"172.21.15.114:6815","nonce":75755945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":75755945},{"type":"v1","addr":"172.21.15.114:6813","nonce":75755945}]},"public_addr":"172.21.15.114:6809/75755945","cluster_addr":"172.21.15.114:6811/75755945","heartbeat_back_addr":"172.21.15.114:6815/75755945","heartbeat_front_addr":"172.21.15.114:6813/75755945","state":["exists","up"]},{"osd":5,"uuid":"51922f99-e8d3-461b-8615-e9bfdcbf5ed3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":1918493790},{"type":"v1","addr":"172.21.15.177:6809","nonce":1918493790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":1918493790},{"type":"v1","addr":"172.21.15.177:6811","nonce":1918493790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":1918493790},{"type":"v1","addr":"172.21.15.177:6815","nonce":1918493790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":1918493790},{"type":"v1","addr":"172.21.15.177:6813","nonce":1918493790}]},"public_addr":"172.21.15.177:6809/1918493790","cluster_addr":"172.21.15.177:6811/1918493790","heartbeat_back_addr":"172.21.15.177:6815/1918493790","heartbeat_front_addr":"172.21.15.177:6813/1918493790","state":["exists","up"]},{"osd":6,"uuid":"06dffb32-05eb-43b3-852e-915c5cc7f25a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2892389563},{"type":"v1","addr":"172.21.15.97:6819","nonce":2892389563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2892389563},{"type":"v1","addr":"172.21.15.97:6821","nonce":2892389563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2892389563},{"type":"v1","addr":"172.21.15.97:6825","nonce":2892389563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2892389563},{"type":"v1","addr":"172.21.15.97:6823","nonce":2892389563}]},"public_addr":"172.21.15.97:6819/2892389563","cluster_addr":"172.21.15.97:6821/2892389563","heartbeat_back_addr":"172.21.15.97:6825/2892389563","heartbeat_front_addr":"172.21.15.97:6823/2892389563","state":["exists","up"]},{"osd":7,"uuid":"e68e3519-9c1b-43c5-8e06-5385248b19df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":2445491153},{"type":"v1","addr":"172.21.15.114:6817","nonce":2445491153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":2445491153},{"type":"v1","addr":"172.21.15.114:6819","nonce":2445491153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2445491153},{"type":"v1","addr":"172.21.15.114:6823","nonce":2445491153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":2445491153},{"type":"v1","addr":"172.21.15.114:6821","nonce":2445491153}]},"public_addr":"172.21.15.114:6817/2445491153","cluster_addr":"172.21.15.114:6819/2445491153","heartbeat_back_addr":"172.21.15.114:6823/2445491153","heartbeat_front_addr":"172.21.15.114:6821/2445491153","state":["exists","up"]},{"osd":8,"uuid":"31ac592e-1349-49ee-9af9-8ad85d682978","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":2051134191},{"type":"v1","addr":"172.21.15.177:6817","nonce":2051134191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":2051134191},{"type":"v1","addr":"172.21.15.177:6819","nonce":2051134191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":2051134191},{"type":"v1","addr":"172.21.15.177:6823","nonce":2051134191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":2051134191},{"type":"v1","addr":"172.21.15.177:6821","nonce":2051134191}]},"public_addr":"172.21.15.177:6817/2051134191","cluster_addr":"172.21.15.177:6819/2051134191","heartbeat_back_addr":"172.21.15.177:6823/2051134191","heartbeat_front_addr":"172.21.15.177:6821/2051134191","state":["exists","up"]},{"osd":9,"uuid":"0171086a-dd41-4eb4-a389-16f0c46a64db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":1578881271},{"type":"v1","addr":"172.21.15.97:6827","nonce":1578881271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1578881271},{"type":"v1","addr":"172.21.15.97:6829","nonce":1578881271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1578881271},{"type":"v1","addr":"172.21.15.97:6833","nonce":1578881271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1578881271},{"type":"v1","addr":"172.21.15.97:6831","nonce":1578881271}]},"public_addr":"172.21.15.97:6827/1578881271","cluster_addr":"172.21.15.97:6829/1578881271","heartbeat_back_addr":"172.21.15.97:6833/1578881271","heartbeat_front_addr":"172.21.15.97:6831/1578881271","state":["exists","up"]},{"osd":10,"uuid":"64754e31-b124-4601-ad98-2f70b1dd5359","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":2893111461},{"type":"v1","addr":"172.21.15.114:6825","nonce":2893111461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":2893111461},{"type":"v1","addr":"172.21.15.114:6827","nonce":2893111461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2893111461},{"type":"v1","addr":"172.21.15.114:6831","nonce":2893111461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":2893111461},{"type":"v1","addr":"172.21.15.114:6829","nonce":2893111461}]},"public_addr":"172.21.15.114:6825/2893111461","cluster_addr":"172.21.15.114:6827/2893111461","heartbeat_back_addr":"172.21.15.114:6831/2893111461","heartbeat_front_addr":"172.21.15.114:6829/2893111461","state":["exists","up"]},{"osd":11,"uuid":"9ffc752c-adf7-4037-89d3-535787d7475d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":85,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":2333030391},{"type":"v1","addr":"172.21.15.177:6825","nonce":2333030391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":2333030391},{"type":"v1","addr":"172.21.15.177:6827","nonce":2333030391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":2333030391},{"type":"v1","addr":"172.21.15.177:6831","nonce":2333030391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":2333030391},{"type":"v1","addr":"172.21.15.177:6829","nonce":2333030391}]},"public_addr":"172.21.15.177:6825/2333030391","cluster_addr":"172.21.15.177:6827/2333030391","heartbeat_back_addr":"172.21.15.177:6831/2333030391","heartbeat_front_addr":"172.21.15.177:6829/2333030391","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:46:45.143519+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:47:18.656947+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:47:51.316345+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:48:24.575512+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:48:59.113555+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:49:32.103865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:50:06.615539+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:50:41.578341+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:51:15.254011+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:51:49.099165+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:52:24.544821+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:52:58.838484+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/359646":"2024-03-28T10:44:39.889550+0000","172.21.15.97:6801/98682155":"2024-03-28T10:44:06.019075+0000","172.21.15.97:6801/1685921853":"2024-03-28T10:44:39.889550+0000","172.21.15.97:6800/98682155":"2024-03-28T10:44:06.019075+0000","172.21.15.97:0/107690680":"2024-03-28T10:44:06.019075+0000","172.21.15.97:6800/1685921853":"2024-03-28T10:44:39.889550+0000","172.21.15.97:0/437043413":"2024-03-28T10:44:39.889550+0000","172.21.15.97:0/2319706992":"2024-03-28T10:44:39.889550+0000","172.21.15.97:0/3911532654":"2024-03-28T10:44:06.019075+0000","172.21.15.97:0/3347321845":"2024-03-28T10:44:06.019075+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-27T10:54:13.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.586+0000 7f4bbee00640 1 -- 172.21.15.97:0/2453445887 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f4ba00a36a0 msgr2=0x7f4ba00a5b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:13.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.586+0000 7f4bbee00640 1 --2- 172.21.15.97:0/2453445887 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f4ba00a36a0 0x7f4ba00a5b60 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f4bac005d00 tx=0x7f4bac005c70 comp rx=0 tx=0).stop 2024-03-27T10:54:13.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.586+0000 7f4bbee00640 1 -- 172.21.15.97:0/2453445887 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4bb8104d10 msgr2=0x7f4bb819a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:13.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.586+0000 7f4bbee00640 1 --2- 172.21.15.97:0/2453445887 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4bb8104d10 0x7f4bb819a280 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4ba400e9c0 tx=0x7f4ba400ee80 comp rx=0 tx=0).stop 2024-03-27T10:54:13.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.586+0000 7f4bbee00640 1 -- 172.21.15.97:0/2453445887 shutdown_connections 2024-03-27T10:54:13.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.586+0000 7f4bbee00640 1 --2- 172.21.15.97:0/2453445887 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4bb8107640 0x7f4bb81a12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:13.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.586+0000 7f4bbee00640 1 --2- 172.21.15.97:0/2453445887 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4bb8104d10 0x7f4bb819a280 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:13.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.586+0000 7f4bbee00640 1 --2- 172.21.15.97:0/2453445887 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f4ba00a36a0 0x7f4ba00a5b60 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:13.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.586+0000 7f4bbee00640 1 --2- 172.21.15.97:0/2453445887 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4bb81023e0 0x7f4bb8199d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:13.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.586+0000 7f4bbee00640 1 -- 172.21.15.97:0/2453445887 >> 172.21.15.97:0/2453445887 conn(0x7f4bb80fbdd0 msgr2=0x7f4bb8101b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:54:13.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.587+0000 7f4bbee00640 1 -- 172.21.15.97:0/2453445887 shutdown_connections 2024-03-27T10:54:13.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.587+0000 7f4bbee00640 1 -- 172.21.15.97:0/2453445887 wait complete. 2024-03-27T10:54:13.580 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-27T10:54:13.580 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-27T10:54:13.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.822+0000 7f1358200640 1 -- 172.21.15.97:0/3906416822 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13501025f0 msgr2=0x7f13501049e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:13.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.822+0000 7f1358200640 1 --2- 172.21.15.97:0/3906416822 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13501025f0 0x7f13501049e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f1344009900 tx=0x7f134402bc30 comp rx=0 tx=0).stop 2024-03-27T10:54:13.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.822+0000 7f1358200640 1 -- 172.21.15.97:0/3906416822 shutdown_connections 2024-03-27T10:54:13.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.822+0000 7f1358200640 1 --2- 172.21.15.97:0/3906416822 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1350107850 0x7f1350109ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:13.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.822+0000 7f1358200640 1 --2- 172.21.15.97:0/3906416822 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1350104f20 0x7f1350107310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:13.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.822+0000 7f1358200640 1 --2- 172.21.15.97:0/3906416822 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13501025f0 0x7f13501049e0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:13.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.822+0000 7f1358200640 1 -- 172.21.15.97:0/3906416822 >> 172.21.15.97:0/3906416822 conn(0x7f13500fbfe0 msgr2=0x7f13500fe420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:54:13.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.822+0000 7f1358200640 1 -- 172.21.15.97:0/3906416822 shutdown_connections 2024-03-27T10:54:13.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.822+0000 7f1358200640 1 -- 172.21.15.97:0/3906416822 wait complete. 2024-03-27T10:54:13.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.823+0000 7f1358200640 1 Processor -- start 2024-03-27T10:54:13.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.823+0000 7f1358200640 1 -- start start 2024-03-27T10:54:13.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.824+0000 7f1358200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13501025f0 0x7f1350199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:13.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.824+0000 7f1358200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1350104f20 0x7f135019a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:13.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.824+0000 7f1358200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1350107850 0x7f13501a1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:13.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.824+0000 7f1358200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f135010d720 con 0x7f13501025f0 2024-03-27T10:54:13.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.824+0000 7f1358200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f135010d5a0 con 0x7f1350107850 2024-03-27T10:54:13.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.824+0000 7f1358200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f135010d8a0 con 0x7f1350104f20 2024-03-27T10:54:13.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.824+0000 7f1355800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13501025f0 0x7f1350199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:13.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.824+0000 7f1355800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13501025f0 0x7f1350199fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43282/0 (socket says 172.21.15.97:43282) 2024-03-27T10:54:13.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.824+0000 7f1355800640 1 -- 172.21.15.97:0/3667891195 learned_addr learned my addr 172.21.15.97:0/3667891195 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:54:13.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.824+0000 7f1354e00640 1 --2- 172.21.15.97:0/3667891195 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1350104f20 0x7f135019a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:13.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.824+0000 7f1355800640 1 -- 172.21.15.97:0/3667891195 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1350104f20 msgr2=0x7f135019a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:13.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.824+0000 7f1356200640 1 --2- 172.21.15.97:0/3667891195 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1350107850 0x7f13501a1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:13.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.824+0000 7f1355800640 1 --2- 172.21.15.97:0/3667891195 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1350104f20 0x7f135019a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:13.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.824+0000 7f1355800640 1 -- 172.21.15.97:0/3667891195 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1350107850 msgr2=0x7f13501a1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:13.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.824+0000 7f1355800640 1 --2- 172.21.15.97:0/3667891195 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1350107850 0x7f13501a1560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:13.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.824+0000 7f1355800640 1 -- 172.21.15.97:0/3667891195 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13501a1bd0 con 0x7f13501025f0 2024-03-27T10:54:13.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.825+0000 7f1355800640 1 --2- 172.21.15.97:0/3667891195 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13501025f0 0x7f1350199fb0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1344000c00 tx=0x7f13440042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:54:13.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.825+0000 7f134e000640 1 -- 172.21.15.97:0/3667891195 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1344043020 con 0x7f13501025f0 2024-03-27T10:54:13.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.825+0000 7f134e000640 1 -- 172.21.15.97:0/3667891195 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f13440054c0 con 0x7f13501025f0 2024-03-27T10:54:13.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.825+0000 7f1358200640 1 -- 172.21.15.97:0/3667891195 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13501a1e60 con 0x7f13501025f0 2024-03-27T10:54:13.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.825+0000 7f134e000640 1 -- 172.21.15.97:0/3667891195 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1344003d40 con 0x7f13501025f0 2024-03-27T10:54:13.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.825+0000 7f1358200640 1 -- 172.21.15.97:0/3667891195 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13501a2320 con 0x7f13501025f0 2024-03-27T10:54:13.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.826+0000 7f1358200640 1 -- 172.21.15.97:0/3667891195 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13501093a0 con 0x7f13501025f0 2024-03-27T10:54:13.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.828+0000 7f134e000640 1 -- 172.21.15.97:0/3667891195 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138421+0+0 (secure 0 0 0) 0x7f1344046020 con 0x7f13501025f0 2024-03-27T10:54:13.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.829+0000 7f134e000640 1 --2- 172.21.15.97:0/3667891195 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f13380a3720 0x7f13380a5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:13.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.829+0000 7f134e000640 1 -- 172.21.15.97:0/3667891195 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 7979+0+0 (secure 0 0 0) 0x7f13440f2220 con 0x7f13501025f0 2024-03-27T10:54:13.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.830+0000 7f1354e00640 1 --2- 172.21.15.97:0/3667891195 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f13380a3720 0x7f13380a5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:13.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.830+0000 7f1354e00640 1 --2- 172.21.15.97:0/3667891195 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f13380a3720 0x7f13380a5be0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f133c005fd0 tx=0x7f133c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:54:13.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:13.833+0000 7f134e000640 1 -- 172.21.15.97:0/3667891195 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f134404d020 con 0x7f13501025f0 2024-03-27T10:54:14.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:14.032+0000 7f1358200640 1 -- 172.21.15.97:0/3667891195 --> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1350061f90 con 0x7f13380a3720 2024-03-27T10:54:14.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:14.047+0000 7f134e000640 1 -- 172.21.15.97:0/3667891195 <== mgr.14152 v2:172.21.15.97:6800/3955463538 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258756 (secure 0 0 0) 0x7f1350061f90 con 0x7f13380a3720 2024-03-27T10:54:14.025 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T10:54:14.026 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-03-27T10:54:14.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:14.053+0000 7f1358200640 1 -- 172.21.15.97:0/3667891195 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f13380a3720 msgr2=0x7f13380a5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:14.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:14.053+0000 7f1358200640 1 --2- 172.21.15.97:0/3667891195 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f13380a3720 0x7f13380a5be0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f133c005fd0 tx=0x7f133c005850 comp rx=0 tx=0).stop 2024-03-27T10:54:14.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:14.054+0000 7f1358200640 1 -- 172.21.15.97:0/3667891195 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13501025f0 msgr2=0x7f1350199fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:14.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:14.054+0000 7f1358200640 1 --2- 172.21.15.97:0/3667891195 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13501025f0 0x7f1350199fb0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1344000c00 tx=0x7f13440042f0 comp rx=0 tx=0).stop 2024-03-27T10:54:14.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:14.054+0000 7f1358200640 1 -- 172.21.15.97:0/3667891195 shutdown_connections 2024-03-27T10:54:14.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:14.054+0000 7f1358200640 1 --2- 172.21.15.97:0/3667891195 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f13380a3720 0x7f13380a5be0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:14.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:14.054+0000 7f1358200640 1 --2- 172.21.15.97:0/3667891195 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1350107850 0x7f13501a1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:14.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:14.054+0000 7f1358200640 1 --2- 172.21.15.97:0/3667891195 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1350104f20 0x7f135019a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:14.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:14.054+0000 7f1358200640 1 --2- 172.21.15.97:0/3667891195 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13501025f0 0x7f1350199fb0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:14.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:14.054+0000 7f1358200640 1 -- 172.21.15.97:0/3667891195 >> 172.21.15.97:0/3667891195 conn(0x7f13500fbfe0 msgr2=0x7f1350101c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:54:14.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:14.054+0000 7f1358200640 1 -- 172.21.15.97:0/3667891195 shutdown_connections 2024-03-27T10:54:14.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:14.054+0000 7f1358200640 1 -- 172.21.15.97:0/3667891195 wait complete. 2024-03-27T10:54:14.047 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":366,"stamp":"2024-03-27T10:54:12.057461+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594004,"num_objects":34,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":248,"num_write_kb":2956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":198,"ondisk_log_size":198,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":258,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":370760,"kb_used_data":13236,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124571064,"statfs":{"total":1151940427776,"available":1151560769536,"internally_reserved":0,"allocated":13553664,"data_stored":7336292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":365933881},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3636,"num_objects":32,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":67,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.004787"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:07.006848+0000","last_change":"2024-03-27T10:54:05.077845+0000","last_active":"2024-03-27T10:54:07.006848+0000","last_peered":"2024-03-27T10:54:07.006848+0000","last_clean":"2024-03-27T10:54:07.006848+0000","last_became_active":"2024-03-27T10:54:05.077494+0000","last_became_peered":"2024-03-27T10:54:05.077494+0000","last_unstale":"2024-03-27T10:54:07.006848+0000","last_undegraded":"2024-03-27T10:54:07.006848+0000","last_fullsized":"2024-03-27T10:54:07.006848+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:16:07.094746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:07.003179+0000","last_change":"2024-03-27T10:54:05.078830+0000","last_active":"2024-03-27T10:54:07.003179+0000","last_peered":"2024-03-27T10:54:07.003179+0000","last_clean":"2024-03-27T10:54:07.003179+0000","last_became_active":"2024-03-27T10:54:05.078474+0000","last_became_peered":"2024-03-27T10:54:05.078474+0000","last_unstale":"2024-03-27T10:54:07.003179+0000","last_undegraded":"2024-03-27T10:54:07.003179+0000","last_fullsized":"2024-03-27T10:54:07.003179+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T20:11:09.900802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:07.003325+0000","last_change":"2024-03-27T10:54:05.082554+0000","last_active":"2024-03-27T10:54:07.003325+0000","last_peered":"2024-03-27T10:54:07.003325+0000","last_clean":"2024-03-27T10:54:07.003325+0000","last_became_active":"2024-03-27T10:54:05.081395+0000","last_became_peered":"2024-03-27T10:54:05.081395+0000","last_unstale":"2024-03-27T10:54:07.003325+0000","last_undegraded":"2024-03-27T10:54:07.003325+0000","last_fullsized":"2024-03-27T10:54:07.003325+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T20:02:58.866792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:07.003675+0000","last_change":"2024-03-27T10:54:05.086700+0000","last_active":"2024-03-27T10:54:07.003675+0000","last_peered":"2024-03-27T10:54:07.003675+0000","last_clean":"2024-03-27T10:54:07.003675+0000","last_became_active":"2024-03-27T10:54:05.086347+0000","last_became_peered":"2024-03-27T10:54:05.086347+0000","last_unstale":"2024-03-27T10:54:07.003675+0000","last_undegraded":"2024-03-27T10:54:07.003675+0000","last_fullsized":"2024-03-27T10:54:07.003675+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T22:03:11.336365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:06.257719+0000","last_change":"2024-03-27T10:54:05.083479+0000","last_active":"2024-03-27T10:54:06.257719+0000","last_peered":"2024-03-27T10:54:06.257719+0000","last_clean":"2024-03-27T10:54:06.257719+0000","last_became_active":"2024-03-27T10:54:05.083162+0000","last_became_peered":"2024-03-27T10:54:05.083162+0000","last_unstale":"2024-03-27T10:54:06.257719+0000","last_undegraded":"2024-03-27T10:54:06.257719+0000","last_fullsized":"2024-03-27T10:54:06.257719+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:23:42.265668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:07.004206+0000","last_change":"2024-03-27T10:54:05.079096+0000","last_active":"2024-03-27T10:54:07.004206+0000","last_peered":"2024-03-27T10:54:07.004206+0000","last_clean":"2024-03-27T10:54:07.004206+0000","last_became_active":"2024-03-27T10:54:05.078778+0000","last_became_peered":"2024-03-27T10:54:05.078778+0000","last_unstale":"2024-03-27T10:54:07.004206+0000","last_undegraded":"2024-03-27T10:54:07.004206+0000","last_fullsized":"2024-03-27T10:54:07.004206+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T19:26:47.534594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:06.074712+0000","last_change":"2024-03-27T10:54:05.098926+0000","last_active":"2024-03-27T10:54:06.074712+0000","last_peered":"2024-03-27T10:54:06.074712+0000","last_clean":"2024-03-27T10:54:06.074712+0000","last_became_active":"2024-03-27T10:54:05.098139+0000","last_became_peered":"2024-03-27T10:54:05.098139+0000","last_unstale":"2024-03-27T10:54:06.074712+0000","last_undegraded":"2024-03-27T10:54:06.074712+0000","last_fullsized":"2024-03-27T10:54:06.074712+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:35:14.335947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:06.192561+0000","last_change":"2024-03-27T10:54:05.093786+0000","last_active":"2024-03-27T10:54:06.192561+0000","last_peered":"2024-03-27T10:54:06.192561+0000","last_clean":"2024-03-27T10:54:06.192561+0000","last_became_active":"2024-03-27T10:54:05.092047+0000","last_became_peered":"2024-03-27T10:54:05.092047+0000","last_unstale":"2024-03-27T10:54:06.192561+0000","last_undegraded":"2024-03-27T10:54:06.192561+0000","last_fullsized":"2024-03-27T10:54:06.192561+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T22:50:30.778358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:07.005190+0000","last_change":"2024-03-27T10:54:05.081678+0000","last_active":"2024-03-27T10:54:07.005190+0000","last_peered":"2024-03-27T10:54:07.005190+0000","last_clean":"2024-03-27T10:54:07.005190+0000","last_became_active":"2024-03-27T10:54:05.080264+0000","last_became_peered":"2024-03-27T10:54:05.080264+0000","last_unstale":"2024-03-27T10:54:07.005190+0000","last_undegraded":"2024-03-27T10:54:07.005190+0000","last_fullsized":"2024-03-27T10:54:07.005190+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T22:41:04.183282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:07.004320+0000","last_change":"2024-03-27T10:54:05.095172+0000","last_active":"2024-03-27T10:54:07.004320+0000","last_peered":"2024-03-27T10:54:07.004320+0000","last_clean":"2024-03-27T10:54:07.004320+0000","last_became_active":"2024-03-27T10:54:05.088438+0000","last_became_peered":"2024-03-27T10:54:05.088438+0000","last_unstale":"2024-03-27T10:54:07.004320+0000","last_undegraded":"2024-03-27T10:54:07.004320+0000","last_fullsized":"2024-03-27T10:54:07.004320+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T11:19:50.422788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:06.075675+0000","last_change":"2024-03-27T10:54:05.093599+0000","last_active":"2024-03-27T10:54:06.075675+0000","last_peered":"2024-03-27T10:54:06.075675+0000","last_clean":"2024-03-27T10:54:06.075675+0000","last_became_active":"2024-03-27T10:54:05.092248+0000","last_became_peered":"2024-03-27T10:54:05.092248+0000","last_unstale":"2024-03-27T10:54:06.075675+0000","last_undegraded":"2024-03-27T10:54:06.075675+0000","last_fullsized":"2024-03-27T10:54:06.075675+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:22:38.712442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:07.003467+0000","last_change":"2024-03-27T10:54:05.094677+0000","last_active":"2024-03-27T10:54:07.003467+0000","last_peered":"2024-03-27T10:54:07.003467+0000","last_clean":"2024-03-27T10:54:07.003467+0000","last_became_active":"2024-03-27T10:54:05.094326+0000","last_became_peered":"2024-03-27T10:54:05.094326+0000","last_unstale":"2024-03-27T10:54:07.003467+0000","last_undegraded":"2024-03-27T10:54:07.003467+0000","last_fullsized":"2024-03-27T10:54:07.003467+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T21:44:51.923881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:06.073405+0000","last_change":"2024-03-27T10:54:05.083178+0000","last_active":"2024-03-27T10:54:06.073405+0000","last_peered":"2024-03-27T10:54:06.073405+0000","last_clean":"2024-03-27T10:54:06.073405+0000","last_became_active":"2024-03-27T10:54:05.082805+0000","last_became_peered":"2024-03-27T10:54:05.082805+0000","last_unstale":"2024-03-27T10:54:06.073405+0000","last_undegraded":"2024-03-27T10:54:06.073405+0000","last_fullsized":"2024-03-27T10:54:06.073405+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T19:03:07.558006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:07.002556+0000","last_change":"2024-03-27T10:54:05.098092+0000","last_active":"2024-03-27T10:54:07.002556+0000","last_peered":"2024-03-27T10:54:07.002556+0000","last_clean":"2024-03-27T10:54:07.002556+0000","last_became_active":"2024-03-27T10:54:05.097759+0000","last_became_peered":"2024-03-27T10:54:05.097759+0000","last_unstale":"2024-03-27T10:54:07.002556+0000","last_undegraded":"2024-03-27T10:54:07.002556+0000","last_fullsized":"2024-03-27T10:54:07.002556+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:14:07.884759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:07.002729+0000","last_change":"2024-03-27T10:54:05.086089+0000","last_active":"2024-03-27T10:54:07.002729+0000","last_peered":"2024-03-27T10:54:07.002729+0000","last_clean":"2024-03-27T10:54:07.002729+0000","last_became_active":"2024-03-27T10:54:05.084830+0000","last_became_peered":"2024-03-27T10:54:05.084830+0000","last_unstale":"2024-03-27T10:54:07.002729+0000","last_undegraded":"2024-03-27T10:54:07.002729+0000","last_fullsized":"2024-03-27T10:54:07.002729+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T20:37:44.390102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:07.004753+0000","last_change":"2024-03-27T10:54:05.092190+0000","last_active":"2024-03-27T10:54:07.004753+0000","last_peered":"2024-03-27T10:54:07.004753+0000","last_clean":"2024-03-27T10:54:07.004753+0000","last_became_active":"2024-03-27T10:54:05.091827+0000","last_became_peered":"2024-03-27T10:54:05.091827+0000","last_unstale":"2024-03-27T10:54:07.004753+0000","last_undegraded":"2024-03-27T10:54:07.004753+0000","last_fullsized":"2024-03-27T10:54:07.004753+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T21:12:13.452829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:07.006737+0000","last_change":"2024-03-27T10:54:05.086924+0000","last_active":"2024-03-27T10:54:07.006737+0000","last_peered":"2024-03-27T10:54:07.006737+0000","last_clean":"2024-03-27T10:54:07.006737+0000","last_became_active":"2024-03-27T10:54:05.085701+0000","last_became_peered":"2024-03-27T10:54:05.085701+0000","last_unstale":"2024-03-27T10:54:07.006737+0000","last_undegraded":"2024-03-27T10:54:07.006737+0000","last_fullsized":"2024-03-27T10:54:07.006737+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:43:38.612288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:06.254562+0000","last_change":"2024-03-27T10:54:05.082532+0000","last_active":"2024-03-27T10:54:06.254562+0000","last_peered":"2024-03-27T10:54:06.254562+0000","last_clean":"2024-03-27T10:54:06.254562+0000","last_became_active":"2024-03-27T10:54:05.082200+0000","last_became_peered":"2024-03-27T10:54:05.082200+0000","last_unstale":"2024-03-27T10:54:06.254562+0000","last_undegraded":"2024-03-27T10:54:06.254562+0000","last_fullsized":"2024-03-27T10:54:06.254562+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T19:07:55.232728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:07.005871+0000","last_change":"2024-03-27T10:54:05.088204+0000","last_active":"2024-03-27T10:54:07.005871+0000","last_peered":"2024-03-27T10:54:07.005871+0000","last_clean":"2024-03-27T10:54:07.005871+0000","last_became_active":"2024-03-27T10:54:05.087705+0000","last_became_peered":"2024-03-27T10:54:05.087705+0000","last_unstale":"2024-03-27T10:54:07.005871+0000","last_undegraded":"2024-03-27T10:54:07.005871+0000","last_fullsized":"2024-03-27T10:54:07.005871+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T19:24:52.450205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'4","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:06.254536+0000","last_change":"2024-03-27T10:54:04.072182+0000","last_active":"2024-03-27T10:54:06.254536+0000","last_peered":"2024-03-27T10:54:06.254536+0000","last_clean":"2024-03-27T10:54:06.254536+0000","last_became_active":"2024-03-27T10:54:04.071827+0000","last_became_peered":"2024-03-27T10:54:04.071827+0000","last_unstale":"2024-03-27T10:54:06.254536+0000","last_undegraded":"2024-03-27T10:54:06.254536+0000","last_fullsized":"2024-03-27T10:54:06.254536+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T18:40:19.776684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:06.192635+0000","last_change":"2024-03-27T10:54:05.095772+0000","last_active":"2024-03-27T10:54:06.192635+0000","last_peered":"2024-03-27T10:54:06.192635+0000","last_clean":"2024-03-27T10:54:06.192635+0000","last_became_active":"2024-03-27T10:54:05.095385+0000","last_became_peered":"2024-03-27T10:54:05.095385+0000","last_unstale":"2024-03-27T10:54:06.192635+0000","last_undegraded":"2024-03-27T10:54:06.192635+0000","last_fullsized":"2024-03-27T10:54:06.192635+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T12:07:21.744097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"89'1","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:06.192710+0000","last_change":"2024-03-27T10:54:04.069600+0000","last_active":"2024-03-27T10:54:06.192710+0000","last_peered":"2024-03-27T10:54:06.192710+0000","last_clean":"2024-03-27T10:54:06.192710+0000","last_became_active":"2024-03-27T10:54:04.067770+0000","last_became_peered":"2024-03-27T10:54:04.067770+0000","last_unstale":"2024-03-27T10:54:06.192710+0000","last_undegraded":"2024-03-27T10:54:06.192710+0000","last_fullsized":"2024-03-27T10:54:06.192710+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T12:26:36.930945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:07.003297+0000","last_change":"2024-03-27T10:54:05.081779+0000","last_active":"2024-03-27T10:54:07.003297+0000","last_peered":"2024-03-27T10:54:07.003297+0000","last_clean":"2024-03-27T10:54:07.003297+0000","last_became_active":"2024-03-27T10:54:05.081492+0000","last_became_peered":"2024-03-27T10:54:05.081492+0000","last_unstale":"2024-03-27T10:54:07.003297+0000","last_undegraded":"2024-03-27T10:54:07.003297+0000","last_fullsized":"2024-03-27T10:54:07.003297+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T19:07:12.838048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"89'2","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:06.258171+0000","last_change":"2024-03-27T10:54:04.070550+0000","last_active":"2024-03-27T10:54:06.258171+0000","last_peered":"2024-03-27T10:54:06.258171+0000","last_clean":"2024-03-27T10:54:06.258171+0000","last_became_active":"2024-03-27T10:54:04.070010+0000","last_became_peered":"2024-03-27T10:54:04.070010+0000","last_unstale":"2024-03-27T10:54:06.258171+0000","last_undegraded":"2024-03-27T10:54:06.258171+0000","last_fullsized":"2024-03-27T10:54:06.258171+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:46:34.168771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:07.005025+0000","last_change":"2024-03-27T10:54:05.086372+0000","last_active":"2024-03-27T10:54:07.005025+0000","last_peered":"2024-03-27T10:54:07.005025+0000","last_clean":"2024-03-27T10:54:07.005025+0000","last_became_active":"2024-03-27T10:54:05.086013+0000","last_became_peered":"2024-03-27T10:54:05.086013+0000","last_unstale":"2024-03-27T10:54:07.005025+0000","last_undegraded":"2024-03-27T10:54:07.005025+0000","last_fullsized":"2024-03-27T10:54:07.005025+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:17:39.059071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"89'2","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:06.257760+0000","last_change":"2024-03-27T10:54:04.071365+0000","last_active":"2024-03-27T10:54:06.257760+0000","last_peered":"2024-03-27T10:54:06.257760+0000","last_clean":"2024-03-27T10:54:06.257760+0000","last_became_active":"2024-03-27T10:54:04.070996+0000","last_became_peered":"2024-03-27T10:54:04.070996+0000","last_unstale":"2024-03-27T10:54:06.257760+0000","last_undegraded":"2024-03-27T10:54:06.257760+0000","last_fullsized":"2024-03-27T10:54:06.257760+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:41:42.174140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:06.075485+0000","last_change":"2024-03-27T10:54:05.087580+0000","last_active":"2024-03-27T10:54:06.075485+0000","last_peered":"2024-03-27T10:54:06.075485+0000","last_clean":"2024-03-27T10:54:06.075485+0000","last_became_active":"2024-03-27T10:54:05.087128+0000","last_became_peered":"2024-03-27T10:54:05.087128+0000","last_unstale":"2024-03-27T10:54:06.075485+0000","last_undegraded":"2024-03-27T10:54:06.075485+0000","last_fullsized":"2024-03-27T10:54:06.075485+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T22:03:46.861258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"90'5","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:08.619759+0000","last_change":"2024-03-27T10:54:04.066312+0000","last_active":"2024-03-27T10:54:08.619759+0000","last_peered":"2024-03-27T10:54:08.619759+0000","last_clean":"2024-03-27T10:54:08.619759+0000","last_became_active":"2024-03-27T10:54:04.065909+0000","last_became_peered":"2024-03-27T10:54:04.065909+0000","last_unstale":"2024-03-27T10:54:08.619759+0000","last_undegraded":"2024-03-27T10:54:08.619759+0000","last_fullsized":"2024-03-27T10:54:08.619759+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:08:45.257808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:06.257851+0000","last_change":"2024-03-27T10:54:05.081096+0000","last_active":"2024-03-27T10:54:06.257851+0000","last_peered":"2024-03-27T10:54:06.257851+0000","last_clean":"2024-03-27T10:54:06.257851+0000","last_became_active":"2024-03-27T10:54:05.080729+0000","last_became_peered":"2024-03-27T10:54:05.080729+0000","last_unstale":"2024-03-27T10:54:06.257851+0000","last_undegraded":"2024-03-27T10:54:06.257851+0000","last_fullsized":"2024-03-27T10:54:06.257851+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T18:51:29.345559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"89'1","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:07.003943+0000","last_change":"2024-03-27T10:54:04.072451+0000","last_active":"2024-03-27T10:54:07.003943+0000","last_peered":"2024-03-27T10:54:07.003943+0000","last_clean":"2024-03-27T10:54:07.003943+0000","last_became_active":"2024-03-27T10:54:04.072042+0000","last_became_peered":"2024-03-27T10:54:04.072042+0000","last_unstale":"2024-03-27T10:54:07.003943+0000","last_undegraded":"2024-03-27T10:54:07.003943+0000","last_fullsized":"2024-03-27T10:54:07.003943+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T12:20:43.231073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:06.074910+0000","last_change":"2024-03-27T10:54:05.086055+0000","last_active":"2024-03-27T10:54:06.074910+0000","last_peered":"2024-03-27T10:54:06.074910+0000","last_clean":"2024-03-27T10:54:06.074910+0000","last_became_active":"2024-03-27T10:54:05.084429+0000","last_became_peered":"2024-03-27T10:54:05.084429+0000","last_unstale":"2024-03-27T10:54:06.074910+0000","last_undegraded":"2024-03-27T10:54:06.074910+0000","last_fullsized":"2024-03-27T10:54:06.074910+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T12:41:12.868245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:06.073172+0000","last_change":"2024-03-27T10:54:04.059076+0000","last_active":"2024-03-27T10:54:06.073172+0000","last_peered":"2024-03-27T10:54:06.073172+0000","last_clean":"2024-03-27T10:54:06.073172+0000","last_became_active":"2024-03-27T10:54:04.057614+0000","last_became_peered":"2024-03-27T10:54:04.057614+0000","last_unstale":"2024-03-27T10:54:06.073172+0000","last_undegraded":"2024-03-27T10:54:06.073172+0000","last_fullsized":"2024-03-27T10:54:06.073172+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T22:26:16.299840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:06.192466+0000","last_change":"2024-03-27T10:54:05.092955+0000","last_active":"2024-03-27T10:54:06.192466+0000","last_peered":"2024-03-27T10:54:06.192466+0000","last_clean":"2024-03-27T10:54:06.192466+0000","last_became_active":"2024-03-27T10:54:05.091867+0000","last_became_peered":"2024-03-27T10:54:05.091867+0000","last_unstale":"2024-03-27T10:54:06.192466+0000","last_undegraded":"2024-03-27T10:54:06.192466+0000","last_fullsized":"2024-03-27T10:54:06.192466+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T21:41:04.636480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:06.074430+0000","last_change":"2024-03-27T10:54:04.058833+0000","last_active":"2024-03-27T10:54:06.074430+0000","last_peered":"2024-03-27T10:54:06.074430+0000","last_clean":"2024-03-27T10:54:06.074430+0000","last_became_active":"2024-03-27T10:54:04.057750+0000","last_became_peered":"2024-03-27T10:54:04.057750+0000","last_unstale":"2024-03-27T10:54:06.074430+0000","last_undegraded":"2024-03-27T10:54:06.074430+0000","last_fullsized":"2024-03-27T10:54:06.074430+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:03:18.771094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:06.253770+0000","last_change":"2024-03-27T10:54:05.078933+0000","last_active":"2024-03-27T10:54:06.253770+0000","last_peered":"2024-03-27T10:54:06.253770+0000","last_clean":"2024-03-27T10:54:06.253770+0000","last_became_active":"2024-03-27T10:54:05.078544+0000","last_became_peered":"2024-03-27T10:54:05.078544+0000","last_unstale":"2024-03-27T10:54:06.253770+0000","last_undegraded":"2024-03-27T10:54:06.253770+0000","last_fullsized":"2024-03-27T10:54:06.253770+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T19:40:25.529775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:07.004178+0000","last_change":"2024-03-27T10:54:04.063974+0000","last_active":"2024-03-27T10:54:07.004178+0000","last_peered":"2024-03-27T10:54:07.004178+0000","last_clean":"2024-03-27T10:54:07.004178+0000","last_became_active":"2024-03-27T10:54:04.058171+0000","last_became_peered":"2024-03-27T10:54:04.058171+0000","last_unstale":"2024-03-27T10:54:07.004178+0000","last_undegraded":"2024-03-27T10:54:07.004178+0000","last_fullsized":"2024-03-27T10:54:07.004178+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T19:14:15.597889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:07.005572+0000","last_change":"2024-03-27T10:54:05.085727+0000","last_active":"2024-03-27T10:54:07.005572+0000","last_peered":"2024-03-27T10:54:07.005572+0000","last_clean":"2024-03-27T10:54:07.005572+0000","last_became_active":"2024-03-27T10:54:05.085354+0000","last_became_peered":"2024-03-27T10:54:05.085354+0000","last_unstale":"2024-03-27T10:54:07.005572+0000","last_undegraded":"2024-03-27T10:54:07.005572+0000","last_fullsized":"2024-03-27T10:54:07.005572+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:33:24.330313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:08.612483+0000","last_change":"2024-03-27T10:54:04.061757+0000","last_active":"2024-03-27T10:54:08.612483+0000","last_peered":"2024-03-27T10:54:08.612483+0000","last_clean":"2024-03-27T10:54:08.612483+0000","last_became_active":"2024-03-27T10:54:04.059211+0000","last_became_peered":"2024-03-27T10:54:04.059211+0000","last_unstale":"2024-03-27T10:54:08.612483+0000","last_undegraded":"2024-03-27T10:54:08.612483+0000","last_fullsized":"2024-03-27T10:54:08.612483+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T11:20:41.391143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:07.003425+0000","last_change":"2024-03-27T10:54:05.090919+0000","last_active":"2024-03-27T10:54:07.003425+0000","last_peered":"2024-03-27T10:54:07.003425+0000","last_clean":"2024-03-27T10:54:07.003425+0000","last_became_active":"2024-03-27T10:54:05.087971+0000","last_became_peered":"2024-03-27T10:54:05.087971+0000","last_unstale":"2024-03-27T10:54:07.003425+0000","last_undegraded":"2024-03-27T10:54:07.003425+0000","last_fullsized":"2024-03-27T10:54:07.003425+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:00:45.195957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:06.075360+0000","last_change":"2024-03-27T10:54:04.064828+0000","last_active":"2024-03-27T10:54:06.075360+0000","last_peered":"2024-03-27T10:54:06.075360+0000","last_clean":"2024-03-27T10:54:06.075360+0000","last_became_active":"2024-03-27T10:54:04.064359+0000","last_became_peered":"2024-03-27T10:54:04.064359+0000","last_unstale":"2024-03-27T10:54:06.075360+0000","last_undegraded":"2024-03-27T10:54:06.075360+0000","last_fullsized":"2024-03-27T10:54:06.075360+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:05:19.905707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:07.005145+0000","last_change":"2024-03-27T10:54:05.095617+0000","last_active":"2024-03-27T10:54:07.005145+0000","last_peered":"2024-03-27T10:54:07.005145+0000","last_clean":"2024-03-27T10:54:07.005145+0000","last_became_active":"2024-03-27T10:54:05.095253+0000","last_became_peered":"2024-03-27T10:54:05.095253+0000","last_unstale":"2024-03-27T10:54:07.005145+0000","last_undegraded":"2024-03-27T10:54:07.005145+0000","last_fullsized":"2024-03-27T10:54:07.005145+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:54:21.420739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"89'6","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:07.003989+0000","last_change":"2024-03-27T10:54:04.061913+0000","last_active":"2024-03-27T10:54:07.003989+0000","last_peered":"2024-03-27T10:54:07.003989+0000","last_clean":"2024-03-27T10:54:07.003989+0000","last_became_active":"2024-03-27T10:54:04.058366+0000","last_became_peered":"2024-03-27T10:54:04.058366+0000","last_unstale":"2024-03-27T10:54:07.003989+0000","last_undegraded":"2024-03-27T10:54:07.003989+0000","last_fullsized":"2024-03-27T10:54:07.003989+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:45:28.365554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:06.258071+0000","last_change":"2024-03-27T10:54:05.090840+0000","last_active":"2024-03-27T10:54:06.258071+0000","last_peered":"2024-03-27T10:54:06.258071+0000","last_clean":"2024-03-27T10:54:06.258071+0000","last_became_active":"2024-03-27T10:54:05.090525+0000","last_became_peered":"2024-03-27T10:54:05.090525+0000","last_unstale":"2024-03-27T10:54:06.258071+0000","last_undegraded":"2024-03-27T10:54:06.258071+0000","last_fullsized":"2024-03-27T10:54:06.258071+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T21:26:10.559917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"90'4","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:08.616244+0000","last_change":"2024-03-27T10:54:04.061205+0000","last_active":"2024-03-27T10:54:08.616244+0000","last_peered":"2024-03-27T10:54:08.616244+0000","last_clean":"2024-03-27T10:54:08.616244+0000","last_became_active":"2024-03-27T10:54:04.059558+0000","last_became_peered":"2024-03-27T10:54:04.059558+0000","last_unstale":"2024-03-27T10:54:08.616244+0000","last_undegraded":"2024-03-27T10:54:08.616244+0000","last_fullsized":"2024-03-27T10:54:08.616244+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T12:00:00.182233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:06.073197+0000","last_change":"2024-03-27T10:54:05.090149+0000","last_active":"2024-03-27T10:54:06.073197+0000","last_peered":"2024-03-27T10:54:06.073197+0000","last_clean":"2024-03-27T10:54:06.073197+0000","last_became_active":"2024-03-27T10:54:05.089317+0000","last_became_peered":"2024-03-27T10:54:05.089317+0000","last_unstale":"2024-03-27T10:54:06.073197+0000","last_undegraded":"2024-03-27T10:54:06.073197+0000","last_fullsized":"2024-03-27T10:54:06.073197+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T11:50:06.477472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:08.618008+0000","last_change":"2024-03-27T10:54:04.061797+0000","last_active":"2024-03-27T10:54:08.618008+0000","last_peered":"2024-03-27T10:54:08.618008+0000","last_clean":"2024-03-27T10:54:08.618008+0000","last_became_active":"2024-03-27T10:54:04.060974+0000","last_became_peered":"2024-03-27T10:54:04.060974+0000","last_unstale":"2024-03-27T10:54:08.618008+0000","last_undegraded":"2024-03-27T10:54:08.618008+0000","last_fullsized":"2024-03-27T10:54:08.618008+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T18:34:33.486126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:06.075870+0000","last_change":"2024-03-27T10:54:05.097916+0000","last_active":"2024-03-27T10:54:06.075870+0000","last_peered":"2024-03-27T10:54:06.075870+0000","last_clean":"2024-03-27T10:54:06.075870+0000","last_became_active":"2024-03-27T10:54:05.097494+0000","last_became_peered":"2024-03-27T10:54:05.097494+0000","last_unstale":"2024-03-27T10:54:06.075870+0000","last_undegraded":"2024-03-27T10:54:06.075870+0000","last_fullsized":"2024-03-27T10:54:06.075870+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T11:22:00.834995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:07.002680+0000","last_change":"2024-03-27T10:54:04.060042+0000","last_active":"2024-03-27T10:54:07.002680+0000","last_peered":"2024-03-27T10:54:07.002680+0000","last_clean":"2024-03-27T10:54:07.002680+0000","last_became_active":"2024-03-27T10:54:04.059415+0000","last_became_peered":"2024-03-27T10:54:04.059415+0000","last_unstale":"2024-03-27T10:54:07.002680+0000","last_undegraded":"2024-03-27T10:54:07.002680+0000","last_fullsized":"2024-03-27T10:54:07.002680+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:52:06.846942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:06.074396+0000","last_change":"2024-03-27T10:54:05.100088+0000","last_active":"2024-03-27T10:54:06.074396+0000","last_peered":"2024-03-27T10:54:06.074396+0000","last_clean":"2024-03-27T10:54:06.074396+0000","last_became_active":"2024-03-27T10:54:05.098709+0000","last_became_peered":"2024-03-27T10:54:05.098709+0000","last_unstale":"2024-03-27T10:54:06.074396+0000","last_undegraded":"2024-03-27T10:54:06.074396+0000","last_fullsized":"2024-03-27T10:54:06.074396+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T21:25:13.329710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"90'10","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:08.617645+0000","last_change":"2024-03-27T10:54:04.066812+0000","last_active":"2024-03-27T10:54:08.617645+0000","last_peered":"2024-03-27T10:54:08.617645+0000","last_clean":"2024-03-27T10:54:08.617645+0000","last_became_active":"2024-03-27T10:54:04.066364+0000","last_became_peered":"2024-03-27T10:54:04.066364+0000","last_unstale":"2024-03-27T10:54:08.617645+0000","last_undegraded":"2024-03-27T10:54:08.617645+0000","last_fullsized":"2024-03-27T10:54:08.617645+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T19:31:51.616785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:06.074690+0000","last_change":"2024-03-27T10:54:05.089367+0000","last_active":"2024-03-27T10:54:06.074690+0000","last_peered":"2024-03-27T10:54:06.074690+0000","last_clean":"2024-03-27T10:54:06.074690+0000","last_became_active":"2024-03-27T10:54:05.088718+0000","last_became_peered":"2024-03-27T10:54:05.088718+0000","last_unstale":"2024-03-27T10:54:06.074690+0000","last_undegraded":"2024-03-27T10:54:06.074690+0000","last_fullsized":"2024-03-27T10:54:06.074690+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:56:08.768859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'1","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:06.258044+0000","last_change":"2024-03-27T10:54:04.059951+0000","last_active":"2024-03-27T10:54:06.258044+0000","last_peered":"2024-03-27T10:54:06.258044+0000","last_clean":"2024-03-27T10:54:06.258044+0000","last_became_active":"2024-03-27T10:54:04.057245+0000","last_became_peered":"2024-03-27T10:54:04.057245+0000","last_unstale":"2024-03-27T10:54:06.258044+0000","last_undegraded":"2024-03-27T10:54:06.258044+0000","last_fullsized":"2024-03-27T10:54:06.258044+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T19:36:48.881920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:07.006364+0000","last_change":"2024-03-27T10:54:05.089004+0000","last_active":"2024-03-27T10:54:07.006364+0000","last_peered":"2024-03-27T10:54:07.006364+0000","last_clean":"2024-03-27T10:54:07.006364+0000","last_became_active":"2024-03-27T10:54:05.083764+0000","last_became_peered":"2024-03-27T10:54:05.083764+0000","last_unstale":"2024-03-27T10:54:07.006364+0000","last_undegraded":"2024-03-27T10:54:07.006364+0000","last_fullsized":"2024-03-27T10:54:07.006364+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:09:53.883490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:06.254127+0000","last_change":"2024-03-27T10:54:04.050508+0000","last_active":"2024-03-27T10:54:06.254127+0000","last_peered":"2024-03-27T10:54:06.254127+0000","last_clean":"2024-03-27T10:54:06.254127+0000","last_became_active":"2024-03-27T10:54:04.048877+0000","last_became_peered":"2024-03-27T10:54:04.048877+0000","last_unstale":"2024-03-27T10:54:06.254127+0000","last_undegraded":"2024-03-27T10:54:06.254127+0000","last_fullsized":"2024-03-27T10:54:06.254127+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:44:11.578684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:07.004493+0000","last_change":"2024-03-27T10:54:05.083931+0000","last_active":"2024-03-27T10:54:07.004493+0000","last_peered":"2024-03-27T10:54:07.004493+0000","last_clean":"2024-03-27T10:54:07.004493+0000","last_became_active":"2024-03-27T10:54:05.083544+0000","last_became_peered":"2024-03-27T10:54:05.083544+0000","last_unstale":"2024-03-27T10:54:07.004493+0000","last_undegraded":"2024-03-27T10:54:07.004493+0000","last_fullsized":"2024-03-27T10:54:07.004493+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T20:54:19.172355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:06.074827+0000","last_change":"2024-03-27T10:54:04.057428+0000","last_active":"2024-03-27T10:54:06.074827+0000","last_peered":"2024-03-27T10:54:06.074827+0000","last_clean":"2024-03-27T10:54:06.074827+0000","last_became_active":"2024-03-27T10:54:04.052966+0000","last_became_peered":"2024-03-27T10:54:04.052966+0000","last_unstale":"2024-03-27T10:54:06.074827+0000","last_undegraded":"2024-03-27T10:54:06.074827+0000","last_fullsized":"2024-03-27T10:54:06.074827+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T18:50:31.170411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:07.004381+0000","last_change":"2024-03-27T10:54:05.081283+0000","last_active":"2024-03-27T10:54:07.004381+0000","last_peered":"2024-03-27T10:54:07.004381+0000","last_clean":"2024-03-27T10:54:07.004381+0000","last_became_active":"2024-03-27T10:54:05.080122+0000","last_became_peered":"2024-03-27T10:54:05.080122+0000","last_unstale":"2024-03-27T10:54:07.004381+0000","last_undegraded":"2024-03-27T10:54:07.004381+0000","last_fullsized":"2024-03-27T10:54:07.004381+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T21:19:34.331347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:06.074966+0000","last_change":"2024-03-27T10:54:05.090255+0000","last_active":"2024-03-27T10:54:06.074966+0000","last_peered":"2024-03-27T10:54:06.074966+0000","last_clean":"2024-03-27T10:54:06.074966+0000","last_became_active":"2024-03-27T10:54:05.088833+0000","last_became_peered":"2024-03-27T10:54:05.088833+0000","last_unstale":"2024-03-27T10:54:06.074966+0000","last_undegraded":"2024-03-27T10:54:06.074966+0000","last_fullsized":"2024-03-27T10:54:06.074966+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:15:55.934698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'1","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:06.073023+0000","last_change":"2024-03-27T10:54:04.059941+0000","last_active":"2024-03-27T10:54:06.073023+0000","last_peered":"2024-03-27T10:54:06.073023+0000","last_clean":"2024-03-27T10:54:06.073023+0000","last_became_active":"2024-03-27T10:54:04.058197+0000","last_became_peered":"2024-03-27T10:54:04.058197+0000","last_unstale":"2024-03-27T10:54:06.073023+0000","last_undegraded":"2024-03-27T10:54:06.073023+0000","last_fullsized":"2024-03-27T10:54:06.073023+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:44:46.374054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:06.192864+0000","last_change":"2024-03-27T10:54:05.095884+0000","last_active":"2024-03-27T10:54:06.192864+0000","last_peered":"2024-03-27T10:54:06.192864+0000","last_clean":"2024-03-27T10:54:06.192864+0000","last_became_active":"2024-03-27T10:54:05.095428+0000","last_became_peered":"2024-03-27T10:54:05.095428+0000","last_unstale":"2024-03-27T10:54:06.192864+0000","last_undegraded":"2024-03-27T10:54:06.192864+0000","last_fullsized":"2024-03-27T10:54:06.192864+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T11:19:49.656608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:07.003112+0000","last_change":"2024-03-27T10:54:04.064017+0000","last_active":"2024-03-27T10:54:07.003112+0000","last_peered":"2024-03-27T10:54:07.003112+0000","last_clean":"2024-03-27T10:54:07.003112+0000","last_became_active":"2024-03-27T10:54:04.062881+0000","last_became_peered":"2024-03-27T10:54:04.062881+0000","last_unstale":"2024-03-27T10:54:07.003112+0000","last_undegraded":"2024-03-27T10:54:07.003112+0000","last_fullsized":"2024-03-27T10:54:07.003112+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:21:21.638623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":344,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:06.074174+0000","last_change":"2024-03-27T10:52:31.127495+0000","last_active":"2024-03-27T10:54:06.074174+0000","last_peered":"2024-03-27T10:54:06.074174+0000","last_clean":"2024-03-27T10:54:06.074174+0000","last_became_active":"2024-03-27T10:52:31.125418+0000","last_became_peered":"2024-03-27T10:52:31.125418+0000","last_unstale":"2024-03-27T10:54:06.074174+0000","last_undegraded":"2024-03-27T10:54:06.074174+0000","last_fullsized":"2024-03-27T10:54:06.074174+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:47:54.649615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:47:54.649615+0000","last_clean_scrub_stamp":"2024-03-27T10:47:54.649615+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T18:49:24.591580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:06.073036+0000","last_change":"2024-03-27T10:54:05.085558+0000","last_active":"2024-03-27T10:54:06.073036+0000","last_peered":"2024-03-27T10:54:06.073036+0000","last_clean":"2024-03-27T10:54:06.073036+0000","last_became_active":"2024-03-27T10:54:05.083896+0000","last_became_peered":"2024-03-27T10:54:05.083896+0000","last_unstale":"2024-03-27T10:54:06.073036+0000","last_undegraded":"2024-03-27T10:54:06.073036+0000","last_fullsized":"2024-03-27T10:54:06.073036+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:15:29.697609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'1","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:06.074575+0000","last_change":"2024-03-27T10:54:04.072448+0000","last_active":"2024-03-27T10:54:06.074575+0000","last_peered":"2024-03-27T10:54:06.074575+0000","last_clean":"2024-03-27T10:54:06.074575+0000","last_became_active":"2024-03-27T10:54:04.071933+0000","last_became_peered":"2024-03-27T10:54:04.071933+0000","last_unstale":"2024-03-27T10:54:06.074575+0000","last_undegraded":"2024-03-27T10:54:06.074575+0000","last_fullsized":"2024-03-27T10:54:06.074575+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:34:49.204345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:06.258028+0000","last_change":"2024-03-27T10:54:04.067308+0000","last_active":"2024-03-27T10:54:06.258028+0000","last_peered":"2024-03-27T10:54:06.258028+0000","last_clean":"2024-03-27T10:54:06.258028+0000","last_became_active":"2024-03-27T10:54:04.066927+0000","last_became_peered":"2024-03-27T10:54:04.066927+0000","last_unstale":"2024-03-27T10:54:06.258028+0000","last_undegraded":"2024-03-27T10:54:06.258028+0000","last_fullsized":"2024-03-27T10:54:06.258028+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:08:04.849245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:06.073197+0000","last_change":"2024-03-27T10:54:05.075154+0000","last_active":"2024-03-27T10:54:06.073197+0000","last_peered":"2024-03-27T10:54:06.073197+0000","last_clean":"2024-03-27T10:54:06.073197+0000","last_became_active":"2024-03-27T10:54:05.074772+0000","last_became_peered":"2024-03-27T10:54:05.074772+0000","last_unstale":"2024-03-27T10:54:06.073197+0000","last_undegraded":"2024-03-27T10:54:06.073197+0000","last_fullsized":"2024-03-27T10:54:06.073197+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T20:49:16.286229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"90'1","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:06.257094+0000","last_change":"2024-03-27T10:54:04.066805+0000","last_active":"2024-03-27T10:54:06.257094+0000","last_peered":"2024-03-27T10:54:06.257094+0000","last_clean":"2024-03-27T10:54:06.257094+0000","last_became_active":"2024-03-27T10:54:04.066500+0000","last_became_peered":"2024-03-27T10:54:04.066500+0000","last_unstale":"2024-03-27T10:54:06.257094+0000","last_undegraded":"2024-03-27T10:54:06.257094+0000","last_fullsized":"2024-03-27T10:54:06.257094+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T12:37:44.853575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:06.192933+0000","last_change":"2024-03-27T10:54:05.094314+0000","last_active":"2024-03-27T10:54:06.192933+0000","last_peered":"2024-03-27T10:54:06.192933+0000","last_clean":"2024-03-27T10:54:06.192933+0000","last_became_active":"2024-03-27T10:54:05.093896+0000","last_became_peered":"2024-03-27T10:54:05.093896+0000","last_unstale":"2024-03-27T10:54:06.192933+0000","last_undegraded":"2024-03-27T10:54:06.192933+0000","last_fullsized":"2024-03-27T10:54:06.192933+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:30:02.441317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"89'2","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:07.003376+0000","last_change":"2024-03-27T10:54:04.065525+0000","last_active":"2024-03-27T10:54:07.003376+0000","last_peered":"2024-03-27T10:54:07.003376+0000","last_clean":"2024-03-27T10:54:07.003376+0000","last_became_active":"2024-03-27T10:54:04.064979+0000","last_became_peered":"2024-03-27T10:54:04.064979+0000","last_unstale":"2024-03-27T10:54:07.003376+0000","last_undegraded":"2024-03-27T10:54:07.003376+0000","last_fullsized":"2024-03-27T10:54:07.003376+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:58:25.110861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:07.003122+0000","last_change":"2024-03-27T10:54:05.084038+0000","last_active":"2024-03-27T10:54:07.003122+0000","last_peered":"2024-03-27T10:54:07.003122+0000","last_clean":"2024-03-27T10:54:07.003122+0000","last_became_active":"2024-03-27T10:54:05.083733+0000","last_became_peered":"2024-03-27T10:54:05.083733+0000","last_unstale":"2024-03-27T10:54:07.003122+0000","last_undegraded":"2024-03-27T10:54:07.003122+0000","last_fullsized":"2024-03-27T10:54:07.003122+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T18:51:14.832602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:08.617286+0000","last_change":"2024-03-27T10:54:04.064506+0000","last_active":"2024-03-27T10:54:08.617286+0000","last_peered":"2024-03-27T10:54:08.617286+0000","last_clean":"2024-03-27T10:54:08.617286+0000","last_became_active":"2024-03-27T10:54:04.064178+0000","last_became_peered":"2024-03-27T10:54:04.064178+0000","last_unstale":"2024-03-27T10:54:08.617286+0000","last_undegraded":"2024-03-27T10:54:08.617286+0000","last_fullsized":"2024-03-27T10:54:08.617286+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:07:19.465782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:06.257550+0000","last_change":"2024-03-27T10:54:05.088241+0000","last_active":"2024-03-27T10:54:06.257550+0000","last_peered":"2024-03-27T10:54:06.257550+0000","last_clean":"2024-03-27T10:54:06.257550+0000","last_became_active":"2024-03-27T10:54:05.087852+0000","last_became_peered":"2024-03-27T10:54:05.087852+0000","last_unstale":"2024-03-27T10:54:06.257550+0000","last_undegraded":"2024-03-27T10:54:06.257550+0000","last_fullsized":"2024-03-27T10:54:06.257550+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:47:09.633654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:06.192080+0000","last_change":"2024-03-27T10:54:04.065744+0000","last_active":"2024-03-27T10:54:06.192080+0000","last_peered":"2024-03-27T10:54:06.192080+0000","last_clean":"2024-03-27T10:54:06.192080+0000","last_became_active":"2024-03-27T10:54:04.065155+0000","last_became_peered":"2024-03-27T10:54:04.065155+0000","last_unstale":"2024-03-27T10:54:06.192080+0000","last_undegraded":"2024-03-27T10:54:06.192080+0000","last_fullsized":"2024-03-27T10:54:06.192080+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T12:35:00.556458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:07.004318+0000","last_change":"2024-03-27T10:54:05.077954+0000","last_active":"2024-03-27T10:54:07.004318+0000","last_peered":"2024-03-27T10:54:07.004318+0000","last_clean":"2024-03-27T10:54:07.004318+0000","last_became_active":"2024-03-27T10:54:05.077589+0000","last_became_peered":"2024-03-27T10:54:05.077589+0000","last_unstale":"2024-03-27T10:54:07.004318+0000","last_undegraded":"2024-03-27T10:54:07.004318+0000","last_fullsized":"2024-03-27T10:54:07.004318+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T20:18:27.411887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:07.005056+0000","last_change":"2024-03-27T10:54:04.067013+0000","last_active":"2024-03-27T10:54:07.005056+0000","last_peered":"2024-03-27T10:54:07.005056+0000","last_clean":"2024-03-27T10:54:07.005056+0000","last_became_active":"2024-03-27T10:54:04.066603+0000","last_became_peered":"2024-03-27T10:54:04.066603+0000","last_unstale":"2024-03-27T10:54:07.005056+0000","last_undegraded":"2024-03-27T10:54:07.005056+0000","last_fullsized":"2024-03-27T10:54:07.005056+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:41:21.678765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:06.192620+0000","last_change":"2024-03-27T10:54:05.088250+0000","last_active":"2024-03-27T10:54:06.192620+0000","last_peered":"2024-03-27T10:54:06.192620+0000","last_clean":"2024-03-27T10:54:06.192620+0000","last_became_active":"2024-03-27T10:54:05.087942+0000","last_became_peered":"2024-03-27T10:54:05.087942+0000","last_unstale":"2024-03-27T10:54:06.192620+0000","last_undegraded":"2024-03-27T10:54:06.192620+0000","last_fullsized":"2024-03-27T10:54:06.192620+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:57:45.266347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:08.619350+0000","last_change":"2024-03-27T10:54:04.070513+0000","last_active":"2024-03-27T10:54:08.619350+0000","last_peered":"2024-03-27T10:54:08.619350+0000","last_clean":"2024-03-27T10:54:08.619350+0000","last_became_active":"2024-03-27T10:54:04.070042+0000","last_became_peered":"2024-03-27T10:54:04.070042+0000","last_unstale":"2024-03-27T10:54:08.619350+0000","last_undegraded":"2024-03-27T10:54:08.619350+0000","last_fullsized":"2024-03-27T10:54:08.619350+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:40:25.216899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:06.075411+0000","last_change":"2024-03-27T10:54:05.095067+0000","last_active":"2024-03-27T10:54:06.075411+0000","last_peered":"2024-03-27T10:54:06.075411+0000","last_clean":"2024-03-27T10:54:06.075411+0000","last_became_active":"2024-03-27T10:54:05.094618+0000","last_became_peered":"2024-03-27T10:54:05.094618+0000","last_unstale":"2024-03-27T10:54:06.075411+0000","last_undegraded":"2024-03-27T10:54:06.075411+0000","last_fullsized":"2024-03-27T10:54:06.075411+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:57:59.557310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:07.005890+0000","last_change":"2024-03-27T10:54:04.051352+0000","last_active":"2024-03-27T10:54:07.005890+0000","last_peered":"2024-03-27T10:54:07.005890+0000","last_clean":"2024-03-27T10:54:07.005890+0000","last_became_active":"2024-03-27T10:54:04.050850+0000","last_became_peered":"2024-03-27T10:54:04.050850+0000","last_unstale":"2024-03-27T10:54:07.005890+0000","last_undegraded":"2024-03-27T10:54:07.005890+0000","last_fullsized":"2024-03-27T10:54:07.005890+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:50:16.719451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:06.076078+0000","last_change":"2024-03-27T10:54:05.094650+0000","last_active":"2024-03-27T10:54:06.076078+0000","last_peered":"2024-03-27T10:54:06.076078+0000","last_clean":"2024-03-27T10:54:06.076078+0000","last_became_active":"2024-03-27T10:54:05.094168+0000","last_became_peered":"2024-03-27T10:54:05.094168+0000","last_unstale":"2024-03-27T10:54:06.076078+0000","last_undegraded":"2024-03-27T10:54:06.076078+0000","last_fullsized":"2024-03-27T10:54:06.076078+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:29:56.491733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"89'1","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:06.075077+0000","last_change":"2024-03-27T10:54:04.067755+0000","last_active":"2024-03-27T10:54:06.075077+0000","last_peered":"2024-03-27T10:54:06.075077+0000","last_clean":"2024-03-27T10:54:06.075077+0000","last_became_active":"2024-03-27T10:54:04.067320+0000","last_became_peered":"2024-03-27T10:54:04.067320+0000","last_unstale":"2024-03-27T10:54:06.075077+0000","last_undegraded":"2024-03-27T10:54:06.075077+0000","last_fullsized":"2024-03-27T10:54:06.075077+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T12:16:25.949829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:06.074766+0000","last_change":"2024-03-27T10:54:05.100715+0000","last_active":"2024-03-27T10:54:06.074766+0000","last_peered":"2024-03-27T10:54:06.074766+0000","last_clean":"2024-03-27T10:54:06.074766+0000","last_became_active":"2024-03-27T10:54:05.097861+0000","last_became_peered":"2024-03-27T10:54:05.097861+0000","last_unstale":"2024-03-27T10:54:06.074766+0000","last_undegraded":"2024-03-27T10:54:06.074766+0000","last_fullsized":"2024-03-27T10:54:06.074766+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:02:39.107232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"89'2","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:06.073541+0000","last_change":"2024-03-27T10:54:04.063894+0000","last_active":"2024-03-27T10:54:06.073541+0000","last_peered":"2024-03-27T10:54:06.073541+0000","last_clean":"2024-03-27T10:54:06.073541+0000","last_became_active":"2024-03-27T10:54:04.063163+0000","last_became_peered":"2024-03-27T10:54:04.063163+0000","last_unstale":"2024-03-27T10:54:06.073541+0000","last_undegraded":"2024-03-27T10:54:06.073541+0000","last_fullsized":"2024-03-27T10:54:06.073541+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:30:41.025752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:06.076166+0000","last_change":"2024-03-27T10:54:05.101980+0000","last_active":"2024-03-27T10:54:06.076166+0000","last_peered":"2024-03-27T10:54:06.076166+0000","last_clean":"2024-03-27T10:54:06.076166+0000","last_became_active":"2024-03-27T10:54:05.101448+0000","last_became_peered":"2024-03-27T10:54:05.101448+0000","last_unstale":"2024-03-27T10:54:06.076166+0000","last_undegraded":"2024-03-27T10:54:06.076166+0000","last_fullsized":"2024-03-27T10:54:06.076166+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:45:11.137181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:07.004605+0000","last_change":"2024-03-27T10:54:05.084026+0000","last_active":"2024-03-27T10:54:07.004605+0000","last_peered":"2024-03-27T10:54:07.004605+0000","last_clean":"2024-03-27T10:54:07.004605+0000","last_became_active":"2024-03-27T10:54:05.083410+0000","last_became_peered":"2024-03-27T10:54:05.083410+0000","last_unstale":"2024-03-27T10:54:07.004605+0000","last_undegraded":"2024-03-27T10:54:07.004605+0000","last_fullsized":"2024-03-27T10:54:07.004605+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T22:39:34.586363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:07.005234+0000","last_change":"2024-03-27T10:54:05.083592+0000","last_active":"2024-03-27T10:54:07.005234+0000","last_peered":"2024-03-27T10:54:07.005234+0000","last_clean":"2024-03-27T10:54:07.005234+0000","last_became_active":"2024-03-27T10:54:05.082537+0000","last_became_peered":"2024-03-27T10:54:05.082537+0000","last_unstale":"2024-03-27T10:54:07.005234+0000","last_undegraded":"2024-03-27T10:54:07.005234+0000","last_fullsized":"2024-03-27T10:54:07.005234+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T21:15:36.830805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:07.005069+0000","last_change":"2024-03-27T10:54:05.090226+0000","last_active":"2024-03-27T10:54:07.005069+0000","last_peered":"2024-03-27T10:54:07.005069+0000","last_clean":"2024-03-27T10:54:07.005069+0000","last_became_active":"2024-03-27T10:54:05.089831+0000","last_became_peered":"2024-03-27T10:54:05.089831+0000","last_unstale":"2024-03-27T10:54:07.005069+0000","last_undegraded":"2024-03-27T10:54:07.005069+0000","last_fullsized":"2024-03-27T10:54:07.005069+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T12:45:57.802266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:07.002774+0000","last_change":"2024-03-27T10:54:05.080196+0000","last_active":"2024-03-27T10:54:07.002774+0000","last_peered":"2024-03-27T10:54:07.002774+0000","last_clean":"2024-03-27T10:54:07.002774+0000","last_became_active":"2024-03-27T10:54:05.079822+0000","last_became_peered":"2024-03-27T10:54:05.079822+0000","last_unstale":"2024-03-27T10:54:07.002774+0000","last_undegraded":"2024-03-27T10:54:07.002774+0000","last_fullsized":"2024-03-27T10:54:07.002774+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:53:11.556503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:06.073502+0000","last_change":"2024-03-27T10:54:05.087264+0000","last_active":"2024-03-27T10:54:06.073502+0000","last_peered":"2024-03-27T10:54:06.073502+0000","last_clean":"2024-03-27T10:54:06.073502+0000","last_became_active":"2024-03-27T10:54:05.085440+0000","last_became_peered":"2024-03-27T10:54:05.085440+0000","last_unstale":"2024-03-27T10:54:06.073502+0000","last_undegraded":"2024-03-27T10:54:06.073502+0000","last_fullsized":"2024-03-27T10:54:06.073502+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T22:41:40.665643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:07.005423+0000","last_change":"2024-03-27T10:54:05.094306+0000","last_active":"2024-03-27T10:54:07.005423+0000","last_peered":"2024-03-27T10:54:07.005423+0000","last_clean":"2024-03-27T10:54:07.005423+0000","last_became_active":"2024-03-27T10:54:05.093858+0000","last_became_peered":"2024-03-27T10:54:05.093858+0000","last_unstale":"2024-03-27T10:54:07.005423+0000","last_undegraded":"2024-03-27T10:54:07.005423+0000","last_fullsized":"2024-03-27T10:54:07.005423+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T21:22:57.624833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:07.006193+0000","last_change":"2024-03-27T10:54:05.094089+0000","last_active":"2024-03-27T10:54:07.006193+0000","last_peered":"2024-03-27T10:54:07.006193+0000","last_clean":"2024-03-27T10:54:07.006193+0000","last_became_active":"2024-03-27T10:54:05.092317+0000","last_became_peered":"2024-03-27T10:54:05.092317+0000","last_unstale":"2024-03-27T10:54:07.006193+0000","last_undegraded":"2024-03-27T10:54:07.006193+0000","last_fullsized":"2024-03-27T10:54:07.006193+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:34:57.606250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:06.192922+0000","last_change":"2024-03-27T10:54:05.099995+0000","last_active":"2024-03-27T10:54:06.192922+0000","last_peered":"2024-03-27T10:54:06.192922+0000","last_clean":"2024-03-27T10:54:06.192922+0000","last_became_active":"2024-03-27T10:54:05.099641+0000","last_became_peered":"2024-03-27T10:54:05.099641+0000","last_unstale":"2024-03-27T10:54:06.192922+0000","last_undegraded":"2024-03-27T10:54:06.192922+0000","last_fullsized":"2024-03-27T10:54:06.192922+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:01:16.050180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:06.074370+0000","last_change":"2024-03-27T10:54:05.082138+0000","last_active":"2024-03-27T10:54:06.074370+0000","last_peered":"2024-03-27T10:54:06.074370+0000","last_clean":"2024-03-27T10:54:06.074370+0000","last_became_active":"2024-03-27T10:54:05.081825+0000","last_became_peered":"2024-03-27T10:54:05.081825+0000","last_unstale":"2024-03-27T10:54:06.074370+0000","last_undegraded":"2024-03-27T10:54:06.074370+0000","last_fullsized":"2024-03-27T10:54:06.074370+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:51:07.635412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:06.192125+0000","last_change":"2024-03-27T10:54:05.096812+0000","last_active":"2024-03-27T10:54:06.192125+0000","last_peered":"2024-03-27T10:54:06.192125+0000","last_clean":"2024-03-27T10:54:06.192125+0000","last_became_active":"2024-03-27T10:54:05.096323+0000","last_became_peered":"2024-03-27T10:54:05.096323+0000","last_unstale":"2024-03-27T10:54:06.192125+0000","last_undegraded":"2024-03-27T10:54:06.192125+0000","last_fullsized":"2024-03-27T10:54:06.192125+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:39:29.827291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:07.005005+0000","last_change":"2024-03-27T10:54:05.089137+0000","last_active":"2024-03-27T10:54:07.005005+0000","last_peered":"2024-03-27T10:54:07.005005+0000","last_clean":"2024-03-27T10:54:07.005005+0000","last_became_active":"2024-03-27T10:54:05.088795+0000","last_became_peered":"2024-03-27T10:54:05.088795+0000","last_unstale":"2024-03-27T10:54:07.005005+0000","last_undegraded":"2024-03-27T10:54:07.005005+0000","last_fullsized":"2024-03-27T10:54:07.005005+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T19:47:35.231749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:06.073082+0000","last_change":"2024-03-27T10:54:05.089683+0000","last_active":"2024-03-27T10:54:06.073082+0000","last_peered":"2024-03-27T10:54:06.073082+0000","last_clean":"2024-03-27T10:54:06.073082+0000","last_became_active":"2024-03-27T10:54:05.089235+0000","last_became_peered":"2024-03-27T10:54:05.089235+0000","last_unstale":"2024-03-27T10:54:06.073082+0000","last_undegraded":"2024-03-27T10:54:06.073082+0000","last_fullsized":"2024-03-27T10:54:06.073082+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:43:37.012814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:07.004811+0000","last_change":"2024-03-27T10:54:05.085043+0000","last_active":"2024-03-27T10:54:07.004811+0000","last_peered":"2024-03-27T10:54:07.004811+0000","last_clean":"2024-03-27T10:54:07.004811+0000","last_became_active":"2024-03-27T10:54:05.084695+0000","last_became_peered":"2024-03-27T10:54:05.084695+0000","last_unstale":"2024-03-27T10:54:07.004811+0000","last_undegraded":"2024-03-27T10:54:07.004811+0000","last_fullsized":"2024-03-27T10:54:07.004811+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T12:07:31.327006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":3636,"num_objects":32,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":67,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":212992,"data_stored":34105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":61,"ondisk_log_size":61,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":85,"seq":365072220177,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":463333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":10,"up_from":77,"seq":330712481816,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32732,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712420,"statfs":{"total":95995035648,"available":95961518080,"internally_reserved":0,"allocated":1585152,"data_stored":1054315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":9,"up_from":71,"seq":304942678046,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":970752,"data_stored":462723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":63,"seq":270582939685,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36296,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708856,"statfs":{"total":95995035648,"available":95957868544,"internally_reserved":0,"allocated":974848,"data_stored":462745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":7,"up_from":54,"seq":231928234028,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28552,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716600,"statfs":{"total":95995035648,"available":95965798400,"internally_reserved":0,"allocated":1572864,"data_stored":1053713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":49,"seq":210453397554,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":966656,"data_stored":462141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574933,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":470935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738459,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":464207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411405,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":983040,"data_stored":463787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117064,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":974848,"data_stored":462245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":35,"seq":150323855425,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36292,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708860,"statfs":{"total":95995035648,"available":95957872640,"internally_reserved":0,"allocated":970752,"data_stored":462163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":41,"seq":176093659194,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32728,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712424,"statfs":{"total":95995035648,"available":95961522176,"internally_reserved":0,"allocated":1589248,"data_stored":1053985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":22647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-27T10:54:14.052 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-27T10:54:14.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:14.340+0000 7fd954000640 1 -- 172.21.15.97:0/3551403930 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd94c076510 msgr2=0x7fd94c076930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:14.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:14.340+0000 7fd954000640 1 --2- 172.21.15.97:0/3551403930 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd94c076510 0x7fd94c076930 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd940009900 tx=0x7fd94002bc30 comp rx=0 tx=0).stop 2024-03-27T10:54:14.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:14.340+0000 7fd954000640 1 -- 172.21.15.97:0/3551403930 shutdown_connections 2024-03-27T10:54:14.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:14.340+0000 7fd954000640 1 --2- 172.21.15.97:0/3551403930 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd94c0780f0 0x7fd94c078990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:14.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:14.340+0000 7fd954000640 1 --2- 172.21.15.97:0/3551403930 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd94c077750 0x7fd94c077bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:14.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:14.340+0000 7fd954000640 1 --2- 172.21.15.97:0/3551403930 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd94c076510 0x7fd94c076930 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:14.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:14.340+0000 7fd954000640 1 -- 172.21.15.97:0/3551403930 >> 172.21.15.97:0/3551403930 conn(0x7fd94c0ff300 msgr2=0x7fd94c101720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:54:14.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:14.340+0000 7fd954000640 1 -- 172.21.15.97:0/3551403930 shutdown_connections 2024-03-27T10:54:14.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:14.340+0000 7fd954000640 1 -- 172.21.15.97:0/3551403930 wait complete. 2024-03-27T10:54:14.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:14.341+0000 7fd954000640 1 Processor -- start 2024-03-27T10:54:14.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:14.341+0000 7fd954000640 1 -- start start 2024-03-27T10:54:14.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:14.342+0000 7fd954000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd94c076510 0x7fd94c19e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:14.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:14.342+0000 7fd954000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd94c077750 0x7fd94c19e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:14.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:14.342+0000 7fd954000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd94c0780f0 0x7fd94c1a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:14.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:14.342+0000 7fd954000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd94c113cf0 con 0x7fd94c0780f0 2024-03-27T10:54:14.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:14.342+0000 7fd954000640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd94c113b70 con 0x7fd94c077750 2024-03-27T10:54:14.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:14.342+0000 7fd954000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd94c113e70 con 0x7fd94c076510 2024-03-27T10:54:14.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:14.342+0000 7fd952000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd94c0780f0 0x7fd94c1a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:14.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:14.342+0000 7fd950c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd94c077750 0x7fd94c19e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:14.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:14.342+0000 7fd952000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd94c0780f0 0x7fd94c1a5990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43304/0 (socket says 172.21.15.97:43304) 2024-03-27T10:54:14.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:14.342+0000 7fd952000640 1 -- 172.21.15.97:0/3088677193 learned_addr learned my addr 172.21.15.97:0/3088677193 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:54:14.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:14.342+0000 7fd951600640 1 --2- 172.21.15.97:0/3088677193 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd94c076510 0x7fd94c19e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:14.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:14.343+0000 7fd952000640 1 -- 172.21.15.97:0/3088677193 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd94c076510 msgr2=0x7fd94c19e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:14.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:14.343+0000 7fd952000640 1 --2- 172.21.15.97:0/3088677193 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd94c076510 0x7fd94c19e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:14.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:14.343+0000 7fd952000640 1 -- 172.21.15.97:0/3088677193 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd94c077750 msgr2=0x7fd94c19e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:14.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:14.343+0000 7fd952000640 1 --2- 172.21.15.97:0/3088677193 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd94c077750 0x7fd94c19e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:14.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:14.343+0000 7fd952000640 1 -- 172.21.15.97:0/3088677193 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd94c1a6000 con 0x7fd94c0780f0 2024-03-27T10:54:14.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:14.343+0000 7fd952000640 1 --2- 172.21.15.97:0/3088677193 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd94c0780f0 0x7fd94c1a5990 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd94400eb10 tx=0x7fd94400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:54:14.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:14.343+0000 7fd94a000640 1 -- 172.21.15.97:0/3088677193 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd944004b20 con 0x7fd94c0780f0 2024-03-27T10:54:14.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:14.343+0000 7fd954000640 1 -- 172.21.15.97:0/3088677193 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd94c1a62f0 con 0x7fd94c0780f0 2024-03-27T10:54:14.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:14.343+0000 7fd94a000640 1 -- 172.21.15.97:0/3088677193 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd944004cc0 con 0x7fd94c0780f0 2024-03-27T10:54:14.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:14.343+0000 7fd94a000640 1 -- 172.21.15.97:0/3088677193 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd944010690 con 0x7fd94c0780f0 2024-03-27T10:54:14.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:14.343+0000 7fd954000640 1 -- 172.21.15.97:0/3088677193 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd94c1a6840 con 0x7fd94c0780f0 2024-03-27T10:54:14.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:14.345+0000 7fd954000640 1 -- 172.21.15.97:0/3088677193 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd94c1106e0 con 0x7fd94c0780f0 2024-03-27T10:54:14.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:14.347+0000 7fd94a000640 1 -- 172.21.15.97:0/3088677193 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138421+0+0 (secure 0 0 0) 0x7fd944005020 con 0x7fd94c0780f0 2024-03-27T10:54:14.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:14.349+0000 7fd94a000640 1 --2- 172.21.15.97:0/3088677193 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fd9340a36a0 0x7fd9340a5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:14.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:14.349+0000 7fd94a000640 1 -- 172.21.15.97:0/3088677193 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 7979+0+0 (secure 0 0 0) 0x7fd9440d03e0 con 0x7fd94c0780f0 2024-03-27T10:54:14.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:14.349+0000 7fd951600640 1 --2- 172.21.15.97:0/3088677193 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fd9340a36a0 0x7fd9340a5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:14.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:14.350+0000 7fd951600640 1 --2- 172.21.15.97:0/3088677193 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fd9340a36a0 0x7fd9340a5b60 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd940006010 tx=0x7fd940000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:54:14.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:14.353+0000 7fd94a000640 1 -- 172.21.15.97:0/3088677193 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7fd94409d000 con 0x7fd94c0780f0 2024-03-27T10:54:14.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:14 smithi097 ceph-mon[31859]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T10:54:14.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:14 smithi097 ceph-mon[31859]: mgrmap e17: x(active, since 9m), standbys: z, y 2024-03-27T10:54:14.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:14 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/2453445887' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T10:54:14.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:14 smithi177 ceph-mon[35294]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T10:54:14.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:14 smithi177 ceph-mon[35294]: mgrmap e17: x(active, since 9m), standbys: z, y 2024-03-27T10:54:14.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:14 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/2453445887' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T10:54:14.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:14 smithi114 ceph-mon[35338]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T10:54:14.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:14 smithi114 ceph-mon[35338]: mgrmap e17: x(active, since 9m), standbys: z, y 2024-03-27T10:54:14.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:14 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/2453445887' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T10:54:14.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:14.550+0000 7fd954000640 1 -- 172.21.15.97:0/3088677193 --> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd94c061f90 con 0x7fd9340a36a0 2024-03-27T10:54:14.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:14.574+0000 7fd94a000640 1 -- 172.21.15.97:0/3088677193 <== mgr.14152 v2:172.21.15.97:6800/3955463538 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258852 (secure 0 0 0) 0x7fd94c061f90 con 0x7fd9340a36a0 2024-03-27T10:54:14.550 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T10:54:14.552 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-03-27T10:54:14.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:14.580+0000 7fd954000640 1 -- 172.21.15.97:0/3088677193 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fd9340a36a0 msgr2=0x7fd9340a5b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:14.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:14.580+0000 7fd954000640 1 --2- 172.21.15.97:0/3088677193 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fd9340a36a0 0x7fd9340a5b60 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd940006010 tx=0x7fd940000bc0 comp rx=0 tx=0).stop 2024-03-27T10:54:14.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:14.580+0000 7fd954000640 1 -- 172.21.15.97:0/3088677193 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd94c0780f0 msgr2=0x7fd94c1a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:14.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:14.580+0000 7fd954000640 1 --2- 172.21.15.97:0/3088677193 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd94c0780f0 0x7fd94c1a5990 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd94400eb10 tx=0x7fd94400c2d0 comp rx=0 tx=0).stop 2024-03-27T10:54:14.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:14.580+0000 7fd954000640 1 -- 172.21.15.97:0/3088677193 shutdown_connections 2024-03-27T10:54:14.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:14.580+0000 7fd954000640 1 --2- 172.21.15.97:0/3088677193 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd94c0780f0 0x7fd94c1a5990 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:14.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:14.580+0000 7fd954000640 1 --2- 172.21.15.97:0/3088677193 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fd9340a36a0 0x7fd9340a5b60 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:14.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:14.580+0000 7fd954000640 1 --2- 172.21.15.97:0/3088677193 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd94c077750 0x7fd94c19e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:14.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:14.580+0000 7fd954000640 1 --2- 172.21.15.97:0/3088677193 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd94c076510 0x7fd94c19e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:14.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:14.580+0000 7fd954000640 1 -- 172.21.15.97:0/3088677193 >> 172.21.15.97:0/3088677193 conn(0x7fd94c0ff300 msgr2=0x7fd94c100e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:54:14.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:14.580+0000 7fd954000640 1 -- 172.21.15.97:0/3088677193 shutdown_connections 2024-03-27T10:54:14.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:14.580+0000 7fd954000640 1 -- 172.21.15.97:0/3088677193 wait complete. 2024-03-27T10:54:14.574 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":367,"stamp":"2024-03-27T10:54:14.058337+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":634119,"num_objects":87,"num_object_clones":0,"num_object_copies":261,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":0,"num_read":172,"num_read_kb":180,"num_write":375,"num_write_kb":3025,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":57,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":304,"ondisk_log_size":304,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":277,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":375116,"kb_used_data":13472,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124566708,"statfs":{"total":1151940427776,"available":1151556308992,"internally_reserved":0,"allocated":13795328,"data_stored":7416898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":43751,"num_objects":85,"num_object_clones":0,"num_object_copies":255,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85,"num_whiteouts":0,"num_read":46,"num_read_kb":71,"num_write":194,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":57,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.011218"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:07.006848+0000","last_change":"2024-03-27T10:54:05.077845+0000","last_active":"2024-03-27T10:54:07.006848+0000","last_peered":"2024-03-27T10:54:07.006848+0000","last_clean":"2024-03-27T10:54:07.006848+0000","last_became_active":"2024-03-27T10:54:05.077494+0000","last_became_peered":"2024-03-27T10:54:05.077494+0000","last_unstale":"2024-03-27T10:54:07.006848+0000","last_undegraded":"2024-03-27T10:54:07.006848+0000","last_fullsized":"2024-03-27T10:54:07.006848+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:16:07.094746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:07.003179+0000","last_change":"2024-03-27T10:54:05.078830+0000","last_active":"2024-03-27T10:54:07.003179+0000","last_peered":"2024-03-27T10:54:07.003179+0000","last_clean":"2024-03-27T10:54:07.003179+0000","last_became_active":"2024-03-27T10:54:05.078474+0000","last_became_peered":"2024-03-27T10:54:05.078474+0000","last_unstale":"2024-03-27T10:54:07.003179+0000","last_undegraded":"2024-03-27T10:54:07.003179+0000","last_fullsized":"2024-03-27T10:54:07.003179+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T20:11:09.900802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:07.003325+0000","last_change":"2024-03-27T10:54:05.082554+0000","last_active":"2024-03-27T10:54:07.003325+0000","last_peered":"2024-03-27T10:54:07.003325+0000","last_clean":"2024-03-27T10:54:07.003325+0000","last_became_active":"2024-03-27T10:54:05.081395+0000","last_became_peered":"2024-03-27T10:54:05.081395+0000","last_unstale":"2024-03-27T10:54:07.003325+0000","last_undegraded":"2024-03-27T10:54:07.003325+0000","last_fullsized":"2024-03-27T10:54:07.003325+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T20:02:58.866792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:07.003675+0000","last_change":"2024-03-27T10:54:05.086700+0000","last_active":"2024-03-27T10:54:07.003675+0000","last_peered":"2024-03-27T10:54:07.003675+0000","last_clean":"2024-03-27T10:54:07.003675+0000","last_became_active":"2024-03-27T10:54:05.086347+0000","last_became_peered":"2024-03-27T10:54:05.086347+0000","last_unstale":"2024-03-27T10:54:07.003675+0000","last_undegraded":"2024-03-27T10:54:07.003675+0000","last_fullsized":"2024-03-27T10:54:07.003675+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T22:03:11.336365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:06.257719+0000","last_change":"2024-03-27T10:54:05.083479+0000","last_active":"2024-03-27T10:54:06.257719+0000","last_peered":"2024-03-27T10:54:06.257719+0000","last_clean":"2024-03-27T10:54:06.257719+0000","last_became_active":"2024-03-27T10:54:05.083162+0000","last_became_peered":"2024-03-27T10:54:05.083162+0000","last_unstale":"2024-03-27T10:54:06.257719+0000","last_undegraded":"2024-03-27T10:54:06.257719+0000","last_fullsized":"2024-03-27T10:54:06.257719+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:23:42.265668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:07.004206+0000","last_change":"2024-03-27T10:54:05.079096+0000","last_active":"2024-03-27T10:54:07.004206+0000","last_peered":"2024-03-27T10:54:07.004206+0000","last_clean":"2024-03-27T10:54:07.004206+0000","last_became_active":"2024-03-27T10:54:05.078778+0000","last_became_peered":"2024-03-27T10:54:05.078778+0000","last_unstale":"2024-03-27T10:54:07.004206+0000","last_undegraded":"2024-03-27T10:54:07.004206+0000","last_fullsized":"2024-03-27T10:54:07.004206+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T19:26:47.534594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:06.074712+0000","last_change":"2024-03-27T10:54:05.098926+0000","last_active":"2024-03-27T10:54:06.074712+0000","last_peered":"2024-03-27T10:54:06.074712+0000","last_clean":"2024-03-27T10:54:06.074712+0000","last_became_active":"2024-03-27T10:54:05.098139+0000","last_became_peered":"2024-03-27T10:54:05.098139+0000","last_unstale":"2024-03-27T10:54:06.074712+0000","last_undegraded":"2024-03-27T10:54:06.074712+0000","last_fullsized":"2024-03-27T10:54:06.074712+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:35:14.335947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:06.192561+0000","last_change":"2024-03-27T10:54:05.093786+0000","last_active":"2024-03-27T10:54:06.192561+0000","last_peered":"2024-03-27T10:54:06.192561+0000","last_clean":"2024-03-27T10:54:06.192561+0000","last_became_active":"2024-03-27T10:54:05.092047+0000","last_became_peered":"2024-03-27T10:54:05.092047+0000","last_unstale":"2024-03-27T10:54:06.192561+0000","last_undegraded":"2024-03-27T10:54:06.192561+0000","last_fullsized":"2024-03-27T10:54:06.192561+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T22:50:30.778358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:07.005190+0000","last_change":"2024-03-27T10:54:05.081678+0000","last_active":"2024-03-27T10:54:07.005190+0000","last_peered":"2024-03-27T10:54:07.005190+0000","last_clean":"2024-03-27T10:54:07.005190+0000","last_became_active":"2024-03-27T10:54:05.080264+0000","last_became_peered":"2024-03-27T10:54:05.080264+0000","last_unstale":"2024-03-27T10:54:07.005190+0000","last_undegraded":"2024-03-27T10:54:07.005190+0000","last_fullsized":"2024-03-27T10:54:07.005190+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T22:41:04.183282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:07.004320+0000","last_change":"2024-03-27T10:54:05.095172+0000","last_active":"2024-03-27T10:54:07.004320+0000","last_peered":"2024-03-27T10:54:07.004320+0000","last_clean":"2024-03-27T10:54:07.004320+0000","last_became_active":"2024-03-27T10:54:05.088438+0000","last_became_peered":"2024-03-27T10:54:05.088438+0000","last_unstale":"2024-03-27T10:54:07.004320+0000","last_undegraded":"2024-03-27T10:54:07.004320+0000","last_fullsized":"2024-03-27T10:54:07.004320+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T11:19:50.422788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:06.075675+0000","last_change":"2024-03-27T10:54:05.093599+0000","last_active":"2024-03-27T10:54:06.075675+0000","last_peered":"2024-03-27T10:54:06.075675+0000","last_clean":"2024-03-27T10:54:06.075675+0000","last_became_active":"2024-03-27T10:54:05.092248+0000","last_became_peered":"2024-03-27T10:54:05.092248+0000","last_unstale":"2024-03-27T10:54:06.075675+0000","last_undegraded":"2024-03-27T10:54:06.075675+0000","last_fullsized":"2024-03-27T10:54:06.075675+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:22:38.712442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:07.003467+0000","last_change":"2024-03-27T10:54:05.094677+0000","last_active":"2024-03-27T10:54:07.003467+0000","last_peered":"2024-03-27T10:54:07.003467+0000","last_clean":"2024-03-27T10:54:07.003467+0000","last_became_active":"2024-03-27T10:54:05.094326+0000","last_became_peered":"2024-03-27T10:54:05.094326+0000","last_unstale":"2024-03-27T10:54:07.003467+0000","last_undegraded":"2024-03-27T10:54:07.003467+0000","last_fullsized":"2024-03-27T10:54:07.003467+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T21:44:51.923881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:06.073405+0000","last_change":"2024-03-27T10:54:05.083178+0000","last_active":"2024-03-27T10:54:06.073405+0000","last_peered":"2024-03-27T10:54:06.073405+0000","last_clean":"2024-03-27T10:54:06.073405+0000","last_became_active":"2024-03-27T10:54:05.082805+0000","last_became_peered":"2024-03-27T10:54:05.082805+0000","last_unstale":"2024-03-27T10:54:06.073405+0000","last_undegraded":"2024-03-27T10:54:06.073405+0000","last_fullsized":"2024-03-27T10:54:06.073405+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T19:03:07.558006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:07.002556+0000","last_change":"2024-03-27T10:54:05.098092+0000","last_active":"2024-03-27T10:54:07.002556+0000","last_peered":"2024-03-27T10:54:07.002556+0000","last_clean":"2024-03-27T10:54:07.002556+0000","last_became_active":"2024-03-27T10:54:05.097759+0000","last_became_peered":"2024-03-27T10:54:05.097759+0000","last_unstale":"2024-03-27T10:54:07.002556+0000","last_undegraded":"2024-03-27T10:54:07.002556+0000","last_fullsized":"2024-03-27T10:54:07.002556+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:14:07.884759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:07.002729+0000","last_change":"2024-03-27T10:54:05.086089+0000","last_active":"2024-03-27T10:54:07.002729+0000","last_peered":"2024-03-27T10:54:07.002729+0000","last_clean":"2024-03-27T10:54:07.002729+0000","last_became_active":"2024-03-27T10:54:05.084830+0000","last_became_peered":"2024-03-27T10:54:05.084830+0000","last_unstale":"2024-03-27T10:54:07.002729+0000","last_undegraded":"2024-03-27T10:54:07.002729+0000","last_fullsized":"2024-03-27T10:54:07.002729+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T20:37:44.390102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:07.004753+0000","last_change":"2024-03-27T10:54:05.092190+0000","last_active":"2024-03-27T10:54:07.004753+0000","last_peered":"2024-03-27T10:54:07.004753+0000","last_clean":"2024-03-27T10:54:07.004753+0000","last_became_active":"2024-03-27T10:54:05.091827+0000","last_became_peered":"2024-03-27T10:54:05.091827+0000","last_unstale":"2024-03-27T10:54:07.004753+0000","last_undegraded":"2024-03-27T10:54:07.004753+0000","last_fullsized":"2024-03-27T10:54:07.004753+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T21:12:13.452829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:07.006737+0000","last_change":"2024-03-27T10:54:05.086924+0000","last_active":"2024-03-27T10:54:07.006737+0000","last_peered":"2024-03-27T10:54:07.006737+0000","last_clean":"2024-03-27T10:54:07.006737+0000","last_became_active":"2024-03-27T10:54:05.085701+0000","last_became_peered":"2024-03-27T10:54:05.085701+0000","last_unstale":"2024-03-27T10:54:07.006737+0000","last_undegraded":"2024-03-27T10:54:07.006737+0000","last_fullsized":"2024-03-27T10:54:07.006737+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:43:38.612288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:06.254562+0000","last_change":"2024-03-27T10:54:05.082532+0000","last_active":"2024-03-27T10:54:06.254562+0000","last_peered":"2024-03-27T10:54:06.254562+0000","last_clean":"2024-03-27T10:54:06.254562+0000","last_became_active":"2024-03-27T10:54:05.082200+0000","last_became_peered":"2024-03-27T10:54:05.082200+0000","last_unstale":"2024-03-27T10:54:06.254562+0000","last_undegraded":"2024-03-27T10:54:06.254562+0000","last_fullsized":"2024-03-27T10:54:06.254562+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T19:07:55.232728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:07.005871+0000","last_change":"2024-03-27T10:54:05.088204+0000","last_active":"2024-03-27T10:54:07.005871+0000","last_peered":"2024-03-27T10:54:07.005871+0000","last_clean":"2024-03-27T10:54:07.005871+0000","last_became_active":"2024-03-27T10:54:05.087705+0000","last_became_peered":"2024-03-27T10:54:05.087705+0000","last_unstale":"2024-03-27T10:54:07.005871+0000","last_undegraded":"2024-03-27T10:54:07.005871+0000","last_fullsized":"2024-03-27T10:54:07.005871+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T19:24:52.450205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'13","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:11.194429+0000","last_change":"2024-03-27T10:54:04.072182+0000","last_active":"2024-03-27T10:54:11.194429+0000","last_peered":"2024-03-27T10:54:11.194429+0000","last_clean":"2024-03-27T10:54:11.194429+0000","last_became_active":"2024-03-27T10:54:04.071827+0000","last_became_peered":"2024-03-27T10:54:04.071827+0000","last_unstale":"2024-03-27T10:54:11.194429+0000","last_undegraded":"2024-03-27T10:54:11.194429+0000","last_fullsized":"2024-03-27T10:54:11.194429+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T18:40:19.776684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:06.192635+0000","last_change":"2024-03-27T10:54:05.095772+0000","last_active":"2024-03-27T10:54:06.192635+0000","last_peered":"2024-03-27T10:54:06.192635+0000","last_clean":"2024-03-27T10:54:06.192635+0000","last_became_active":"2024-03-27T10:54:05.095385+0000","last_became_peered":"2024-03-27T10:54:05.095385+0000","last_unstale":"2024-03-27T10:54:06.192635+0000","last_undegraded":"2024-03-27T10:54:06.192635+0000","last_fullsized":"2024-03-27T10:54:06.192635+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T12:07:21.744097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:06.192792+0000","last_change":"2024-03-27T10:54:04.069600+0000","last_active":"2024-03-27T10:54:06.192792+0000","last_peered":"2024-03-27T10:54:06.192792+0000","last_clean":"2024-03-27T10:54:06.192792+0000","last_became_active":"2024-03-27T10:54:04.067770+0000","last_became_peered":"2024-03-27T10:54:04.067770+0000","last_unstale":"2024-03-27T10:54:06.192792+0000","last_undegraded":"2024-03-27T10:54:06.192792+0000","last_fullsized":"2024-03-27T10:54:06.192792+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T12:26:36.930945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:07.003297+0000","last_change":"2024-03-27T10:54:05.081779+0000","last_active":"2024-03-27T10:54:07.003297+0000","last_peered":"2024-03-27T10:54:07.003297+0000","last_clean":"2024-03-27T10:54:07.003297+0000","last_became_active":"2024-03-27T10:54:05.081492+0000","last_became_peered":"2024-03-27T10:54:05.081492+0000","last_unstale":"2024-03-27T10:54:07.003297+0000","last_undegraded":"2024-03-27T10:54:07.003297+0000","last_fullsized":"2024-03-27T10:54:07.003297+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T19:07:12.838048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"90'12","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:12.184907+0000","last_change":"2024-03-27T10:54:04.070550+0000","last_active":"2024-03-27T10:54:12.184907+0000","last_peered":"2024-03-27T10:54:12.184907+0000","last_clean":"2024-03-27T10:54:12.184907+0000","last_became_active":"2024-03-27T10:54:04.070010+0000","last_became_peered":"2024-03-27T10:54:04.070010+0000","last_unstale":"2024-03-27T10:54:12.184907+0000","last_undegraded":"2024-03-27T10:54:12.184907+0000","last_fullsized":"2024-03-27T10:54:12.184907+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:46:34.168771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:07.005025+0000","last_change":"2024-03-27T10:54:05.086372+0000","last_active":"2024-03-27T10:54:07.005025+0000","last_peered":"2024-03-27T10:54:07.005025+0000","last_clean":"2024-03-27T10:54:07.005025+0000","last_became_active":"2024-03-27T10:54:05.086013+0000","last_became_peered":"2024-03-27T10:54:05.086013+0000","last_unstale":"2024-03-27T10:54:07.005025+0000","last_undegraded":"2024-03-27T10:54:07.005025+0000","last_fullsized":"2024-03-27T10:54:07.005025+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:17:39.059071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"90'5","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:11.196735+0000","last_change":"2024-03-27T10:54:04.071365+0000","last_active":"2024-03-27T10:54:11.196735+0000","last_peered":"2024-03-27T10:54:11.196735+0000","last_clean":"2024-03-27T10:54:11.196735+0000","last_became_active":"2024-03-27T10:54:04.070996+0000","last_became_peered":"2024-03-27T10:54:04.070996+0000","last_unstale":"2024-03-27T10:54:11.196735+0000","last_undegraded":"2024-03-27T10:54:11.196735+0000","last_fullsized":"2024-03-27T10:54:11.196735+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:41:42.174140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:06.075485+0000","last_change":"2024-03-27T10:54:05.087580+0000","last_active":"2024-03-27T10:54:06.075485+0000","last_peered":"2024-03-27T10:54:06.075485+0000","last_clean":"2024-03-27T10:54:06.075485+0000","last_became_active":"2024-03-27T10:54:05.087128+0000","last_became_peered":"2024-03-27T10:54:05.087128+0000","last_unstale":"2024-03-27T10:54:06.075485+0000","last_undegraded":"2024-03-27T10:54:06.075485+0000","last_fullsized":"2024-03-27T10:54:06.075485+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T22:03:46.861258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"90'5","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:08.619759+0000","last_change":"2024-03-27T10:54:04.066312+0000","last_active":"2024-03-27T10:54:08.619759+0000","last_peered":"2024-03-27T10:54:08.619759+0000","last_clean":"2024-03-27T10:54:08.619759+0000","last_became_active":"2024-03-27T10:54:04.065909+0000","last_became_peered":"2024-03-27T10:54:04.065909+0000","last_unstale":"2024-03-27T10:54:08.619759+0000","last_undegraded":"2024-03-27T10:54:08.619759+0000","last_fullsized":"2024-03-27T10:54:08.619759+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:08:45.257808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:06.257851+0000","last_change":"2024-03-27T10:54:05.081096+0000","last_active":"2024-03-27T10:54:06.257851+0000","last_peered":"2024-03-27T10:54:06.257851+0000","last_clean":"2024-03-27T10:54:06.257851+0000","last_became_active":"2024-03-27T10:54:05.080729+0000","last_became_peered":"2024-03-27T10:54:05.080729+0000","last_unstale":"2024-03-27T10:54:06.257851+0000","last_undegraded":"2024-03-27T10:54:06.257851+0000","last_fullsized":"2024-03-27T10:54:06.257851+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T18:51:29.345559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"90'7","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:10.179363+0000","last_change":"2024-03-27T10:54:04.072451+0000","last_active":"2024-03-27T10:54:10.179363+0000","last_peered":"2024-03-27T10:54:10.179363+0000","last_clean":"2024-03-27T10:54:10.179363+0000","last_became_active":"2024-03-27T10:54:04.072042+0000","last_became_peered":"2024-03-27T10:54:04.072042+0000","last_unstale":"2024-03-27T10:54:10.179363+0000","last_undegraded":"2024-03-27T10:54:10.179363+0000","last_fullsized":"2024-03-27T10:54:10.179363+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T12:20:43.231073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:06.074910+0000","last_change":"2024-03-27T10:54:05.086055+0000","last_active":"2024-03-27T10:54:06.074910+0000","last_peered":"2024-03-27T10:54:06.074910+0000","last_clean":"2024-03-27T10:54:06.074910+0000","last_became_active":"2024-03-27T10:54:05.084429+0000","last_became_peered":"2024-03-27T10:54:05.084429+0000","last_unstale":"2024-03-27T10:54:06.074910+0000","last_undegraded":"2024-03-27T10:54:06.074910+0000","last_fullsized":"2024-03-27T10:54:06.074910+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T12:41:12.868245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:06.073172+0000","last_change":"2024-03-27T10:54:04.059076+0000","last_active":"2024-03-27T10:54:06.073172+0000","last_peered":"2024-03-27T10:54:06.073172+0000","last_clean":"2024-03-27T10:54:06.073172+0000","last_became_active":"2024-03-27T10:54:04.057614+0000","last_became_peered":"2024-03-27T10:54:04.057614+0000","last_unstale":"2024-03-27T10:54:06.073172+0000","last_undegraded":"2024-03-27T10:54:06.073172+0000","last_fullsized":"2024-03-27T10:54:06.073172+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T22:26:16.299840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:06.192466+0000","last_change":"2024-03-27T10:54:05.092955+0000","last_active":"2024-03-27T10:54:06.192466+0000","last_peered":"2024-03-27T10:54:06.192466+0000","last_clean":"2024-03-27T10:54:06.192466+0000","last_became_active":"2024-03-27T10:54:05.091867+0000","last_became_peered":"2024-03-27T10:54:05.091867+0000","last_unstale":"2024-03-27T10:54:06.192466+0000","last_undegraded":"2024-03-27T10:54:06.192466+0000","last_fullsized":"2024-03-27T10:54:06.192466+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T21:41:04.636480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:06.074430+0000","last_change":"2024-03-27T10:54:04.058833+0000","last_active":"2024-03-27T10:54:06.074430+0000","last_peered":"2024-03-27T10:54:06.074430+0000","last_clean":"2024-03-27T10:54:06.074430+0000","last_became_active":"2024-03-27T10:54:04.057750+0000","last_became_peered":"2024-03-27T10:54:04.057750+0000","last_unstale":"2024-03-27T10:54:06.074430+0000","last_undegraded":"2024-03-27T10:54:06.074430+0000","last_fullsized":"2024-03-27T10:54:06.074430+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:03:18.771094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:06.253770+0000","last_change":"2024-03-27T10:54:05.078933+0000","last_active":"2024-03-27T10:54:06.253770+0000","last_peered":"2024-03-27T10:54:06.253770+0000","last_clean":"2024-03-27T10:54:06.253770+0000","last_became_active":"2024-03-27T10:54:05.078544+0000","last_became_peered":"2024-03-27T10:54:05.078544+0000","last_unstale":"2024-03-27T10:54:06.253770+0000","last_undegraded":"2024-03-27T10:54:06.253770+0000","last_fullsized":"2024-03-27T10:54:06.253770+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T19:40:25.529775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"90'4","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:13.174834+0000","last_change":"2024-03-27T10:54:04.063974+0000","last_active":"2024-03-27T10:54:13.174834+0000","last_peered":"2024-03-27T10:54:13.174834+0000","last_clean":"2024-03-27T10:54:13.174834+0000","last_became_active":"2024-03-27T10:54:04.058171+0000","last_became_peered":"2024-03-27T10:54:04.058171+0000","last_unstale":"2024-03-27T10:54:13.174834+0000","last_undegraded":"2024-03-27T10:54:13.174834+0000","last_fullsized":"2024-03-27T10:54:13.174834+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T19:14:15.597889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:07.005572+0000","last_change":"2024-03-27T10:54:05.085727+0000","last_active":"2024-03-27T10:54:07.005572+0000","last_peered":"2024-03-27T10:54:07.005572+0000","last_clean":"2024-03-27T10:54:07.005572+0000","last_became_active":"2024-03-27T10:54:05.085354+0000","last_became_peered":"2024-03-27T10:54:05.085354+0000","last_unstale":"2024-03-27T10:54:07.005572+0000","last_undegraded":"2024-03-27T10:54:07.005572+0000","last_fullsized":"2024-03-27T10:54:07.005572+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:33:24.330313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:08.612483+0000","last_change":"2024-03-27T10:54:04.061757+0000","last_active":"2024-03-27T10:54:08.612483+0000","last_peered":"2024-03-27T10:54:08.612483+0000","last_clean":"2024-03-27T10:54:08.612483+0000","last_became_active":"2024-03-27T10:54:04.059211+0000","last_became_peered":"2024-03-27T10:54:04.059211+0000","last_unstale":"2024-03-27T10:54:08.612483+0000","last_undegraded":"2024-03-27T10:54:08.612483+0000","last_fullsized":"2024-03-27T10:54:08.612483+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T11:20:41.391143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:07.003425+0000","last_change":"2024-03-27T10:54:05.090919+0000","last_active":"2024-03-27T10:54:07.003425+0000","last_peered":"2024-03-27T10:54:07.003425+0000","last_clean":"2024-03-27T10:54:07.003425+0000","last_became_active":"2024-03-27T10:54:05.087971+0000","last_became_peered":"2024-03-27T10:54:05.087971+0000","last_unstale":"2024-03-27T10:54:07.003425+0000","last_undegraded":"2024-03-27T10:54:07.003425+0000","last_fullsized":"2024-03-27T10:54:07.003425+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:00:45.195957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:06.075360+0000","last_change":"2024-03-27T10:54:04.064828+0000","last_active":"2024-03-27T10:54:06.075360+0000","last_peered":"2024-03-27T10:54:06.075360+0000","last_clean":"2024-03-27T10:54:06.075360+0000","last_became_active":"2024-03-27T10:54:04.064359+0000","last_became_peered":"2024-03-27T10:54:04.064359+0000","last_unstale":"2024-03-27T10:54:06.075360+0000","last_undegraded":"2024-03-27T10:54:06.075360+0000","last_fullsized":"2024-03-27T10:54:06.075360+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:05:19.905707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:07.005145+0000","last_change":"2024-03-27T10:54:05.095617+0000","last_active":"2024-03-27T10:54:07.005145+0000","last_peered":"2024-03-27T10:54:07.005145+0000","last_clean":"2024-03-27T10:54:07.005145+0000","last_became_active":"2024-03-27T10:54:05.095253+0000","last_became_peered":"2024-03-27T10:54:05.095253+0000","last_unstale":"2024-03-27T10:54:07.005145+0000","last_undegraded":"2024-03-27T10:54:07.005145+0000","last_fullsized":"2024-03-27T10:54:07.005145+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:54:21.420739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"90'21","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:13.174148+0000","last_change":"2024-03-27T10:54:04.061913+0000","last_active":"2024-03-27T10:54:13.174148+0000","last_peered":"2024-03-27T10:54:13.174148+0000","last_clean":"2024-03-27T10:54:13.174148+0000","last_became_active":"2024-03-27T10:54:04.058366+0000","last_became_peered":"2024-03-27T10:54:04.058366+0000","last_unstale":"2024-03-27T10:54:13.174148+0000","last_undegraded":"2024-03-27T10:54:13.174148+0000","last_fullsized":"2024-03-27T10:54:13.174148+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:45:28.365554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38193,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":12,"num_read_kb":42,"num_write":23,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:06.258071+0000","last_change":"2024-03-27T10:54:05.090840+0000","last_active":"2024-03-27T10:54:06.258071+0000","last_peered":"2024-03-27T10:54:06.258071+0000","last_clean":"2024-03-27T10:54:06.258071+0000","last_became_active":"2024-03-27T10:54:05.090525+0000","last_became_peered":"2024-03-27T10:54:05.090525+0000","last_unstale":"2024-03-27T10:54:06.258071+0000","last_undegraded":"2024-03-27T10:54:06.258071+0000","last_fullsized":"2024-03-27T10:54:06.258071+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T21:26:10.559917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"90'9","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:11.194763+0000","last_change":"2024-03-27T10:54:04.061205+0000","last_active":"2024-03-27T10:54:11.194763+0000","last_peered":"2024-03-27T10:54:11.194763+0000","last_clean":"2024-03-27T10:54:11.194763+0000","last_became_active":"2024-03-27T10:54:04.059558+0000","last_became_peered":"2024-03-27T10:54:04.059558+0000","last_unstale":"2024-03-27T10:54:11.194763+0000","last_undegraded":"2024-03-27T10:54:11.194763+0000","last_fullsized":"2024-03-27T10:54:11.194763+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T12:00:00.182233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:06.073197+0000","last_change":"2024-03-27T10:54:05.090149+0000","last_active":"2024-03-27T10:54:06.073197+0000","last_peered":"2024-03-27T10:54:06.073197+0000","last_clean":"2024-03-27T10:54:06.073197+0000","last_became_active":"2024-03-27T10:54:05.089317+0000","last_became_peered":"2024-03-27T10:54:05.089317+0000","last_unstale":"2024-03-27T10:54:06.073197+0000","last_undegraded":"2024-03-27T10:54:06.073197+0000","last_fullsized":"2024-03-27T10:54:06.073197+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T11:50:06.477472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:08.618008+0000","last_change":"2024-03-27T10:54:04.061797+0000","last_active":"2024-03-27T10:54:08.618008+0000","last_peered":"2024-03-27T10:54:08.618008+0000","last_clean":"2024-03-27T10:54:08.618008+0000","last_became_active":"2024-03-27T10:54:04.060974+0000","last_became_peered":"2024-03-27T10:54:04.060974+0000","last_unstale":"2024-03-27T10:54:08.618008+0000","last_undegraded":"2024-03-27T10:54:08.618008+0000","last_fullsized":"2024-03-27T10:54:08.618008+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T18:34:33.486126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:06.075870+0000","last_change":"2024-03-27T10:54:05.097916+0000","last_active":"2024-03-27T10:54:06.075870+0000","last_peered":"2024-03-27T10:54:06.075870+0000","last_clean":"2024-03-27T10:54:06.075870+0000","last_became_active":"2024-03-27T10:54:05.097494+0000","last_became_peered":"2024-03-27T10:54:05.097494+0000","last_unstale":"2024-03-27T10:54:06.075870+0000","last_undegraded":"2024-03-27T10:54:06.075870+0000","last_fullsized":"2024-03-27T10:54:06.075870+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T11:22:00.834995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"90'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:11.173339+0000","last_change":"2024-03-27T10:54:04.060042+0000","last_active":"2024-03-27T10:54:11.173339+0000","last_peered":"2024-03-27T10:54:11.173339+0000","last_clean":"2024-03-27T10:54:11.173339+0000","last_became_active":"2024-03-27T10:54:04.059415+0000","last_became_peered":"2024-03-27T10:54:04.059415+0000","last_unstale":"2024-03-27T10:54:11.173339+0000","last_undegraded":"2024-03-27T10:54:11.173339+0000","last_fullsized":"2024-03-27T10:54:11.173339+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:52:06.846942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":7,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:06.074396+0000","last_change":"2024-03-27T10:54:05.100088+0000","last_active":"2024-03-27T10:54:06.074396+0000","last_peered":"2024-03-27T10:54:06.074396+0000","last_clean":"2024-03-27T10:54:06.074396+0000","last_became_active":"2024-03-27T10:54:05.098709+0000","last_became_peered":"2024-03-27T10:54:05.098709+0000","last_unstale":"2024-03-27T10:54:06.074396+0000","last_undegraded":"2024-03-27T10:54:06.074396+0000","last_fullsized":"2024-03-27T10:54:06.074396+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T21:25:13.329710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"90'17","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:10.179349+0000","last_change":"2024-03-27T10:54:04.066812+0000","last_active":"2024-03-27T10:54:10.179349+0000","last_peered":"2024-03-27T10:54:10.179349+0000","last_clean":"2024-03-27T10:54:10.179349+0000","last_became_active":"2024-03-27T10:54:04.066364+0000","last_became_peered":"2024-03-27T10:54:04.066364+0000","last_unstale":"2024-03-27T10:54:10.179349+0000","last_undegraded":"2024-03-27T10:54:10.179349+0000","last_fullsized":"2024-03-27T10:54:10.179349+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T19:31:51.616785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:06.074690+0000","last_change":"2024-03-27T10:54:05.089367+0000","last_active":"2024-03-27T10:54:06.074690+0000","last_peered":"2024-03-27T10:54:06.074690+0000","last_clean":"2024-03-27T10:54:06.074690+0000","last_became_active":"2024-03-27T10:54:05.088718+0000","last_became_peered":"2024-03-27T10:54:05.088718+0000","last_unstale":"2024-03-27T10:54:06.074690+0000","last_undegraded":"2024-03-27T10:54:06.074690+0000","last_fullsized":"2024-03-27T10:54:06.074690+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:56:08.768859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:10.156788+0000","last_change":"2024-03-27T10:54:04.059951+0000","last_active":"2024-03-27T10:54:10.156788+0000","last_peered":"2024-03-27T10:54:10.156788+0000","last_clean":"2024-03-27T10:54:10.156788+0000","last_became_active":"2024-03-27T10:54:04.057245+0000","last_became_peered":"2024-03-27T10:54:04.057245+0000","last_unstale":"2024-03-27T10:54:10.156788+0000","last_undegraded":"2024-03-27T10:54:10.156788+0000","last_fullsized":"2024-03-27T10:54:10.156788+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T19:36:48.881920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:07.006364+0000","last_change":"2024-03-27T10:54:05.089004+0000","last_active":"2024-03-27T10:54:07.006364+0000","last_peered":"2024-03-27T10:54:07.006364+0000","last_clean":"2024-03-27T10:54:07.006364+0000","last_became_active":"2024-03-27T10:54:05.083764+0000","last_became_peered":"2024-03-27T10:54:05.083764+0000","last_unstale":"2024-03-27T10:54:07.006364+0000","last_undegraded":"2024-03-27T10:54:07.006364+0000","last_fullsized":"2024-03-27T10:54:07.006364+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:09:53.883490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"90'1","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:07.310236+0000","last_change":"2024-03-27T10:54:04.050508+0000","last_active":"2024-03-27T10:54:07.310236+0000","last_peered":"2024-03-27T10:54:07.310236+0000","last_clean":"2024-03-27T10:54:07.310236+0000","last_became_active":"2024-03-27T10:54:04.048877+0000","last_became_peered":"2024-03-27T10:54:04.048877+0000","last_unstale":"2024-03-27T10:54:07.310236+0000","last_undegraded":"2024-03-27T10:54:07.310236+0000","last_fullsized":"2024-03-27T10:54:07.310236+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:44:11.578684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"90'1","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:08.396819+0000","last_change":"2024-03-27T10:54:05.083931+0000","last_active":"2024-03-27T10:54:08.396819+0000","last_peered":"2024-03-27T10:54:08.396819+0000","last_clean":"2024-03-27T10:54:08.396819+0000","last_became_active":"2024-03-27T10:54:05.083544+0000","last_became_peered":"2024-03-27T10:54:05.083544+0000","last_unstale":"2024-03-27T10:54:08.396819+0000","last_undegraded":"2024-03-27T10:54:08.396819+0000","last_fullsized":"2024-03-27T10:54:08.396819+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T20:54:19.172355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:06.074827+0000","last_change":"2024-03-27T10:54:04.057428+0000","last_active":"2024-03-27T10:54:06.074827+0000","last_peered":"2024-03-27T10:54:06.074827+0000","last_clean":"2024-03-27T10:54:06.074827+0000","last_became_active":"2024-03-27T10:54:04.052966+0000","last_became_peered":"2024-03-27T10:54:04.052966+0000","last_unstale":"2024-03-27T10:54:06.074827+0000","last_undegraded":"2024-03-27T10:54:06.074827+0000","last_fullsized":"2024-03-27T10:54:06.074827+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T18:50:31.170411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:07.004381+0000","last_change":"2024-03-27T10:54:05.081283+0000","last_active":"2024-03-27T10:54:07.004381+0000","last_peered":"2024-03-27T10:54:07.004381+0000","last_clean":"2024-03-27T10:54:07.004381+0000","last_became_active":"2024-03-27T10:54:05.080122+0000","last_became_peered":"2024-03-27T10:54:05.080122+0000","last_unstale":"2024-03-27T10:54:07.004381+0000","last_undegraded":"2024-03-27T10:54:07.004381+0000","last_fullsized":"2024-03-27T10:54:07.004381+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T21:19:34.331347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:06.074966+0000","last_change":"2024-03-27T10:54:05.090255+0000","last_active":"2024-03-27T10:54:06.074966+0000","last_peered":"2024-03-27T10:54:06.074966+0000","last_clean":"2024-03-27T10:54:06.074966+0000","last_became_active":"2024-03-27T10:54:05.088833+0000","last_became_peered":"2024-03-27T10:54:05.088833+0000","last_unstale":"2024-03-27T10:54:06.074966+0000","last_undegraded":"2024-03-27T10:54:06.074966+0000","last_fullsized":"2024-03-27T10:54:06.074966+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:15:55.934698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'1","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:06.073023+0000","last_change":"2024-03-27T10:54:04.059941+0000","last_active":"2024-03-27T10:54:06.073023+0000","last_peered":"2024-03-27T10:54:06.073023+0000","last_clean":"2024-03-27T10:54:06.073023+0000","last_became_active":"2024-03-27T10:54:04.058197+0000","last_became_peered":"2024-03-27T10:54:04.058197+0000","last_unstale":"2024-03-27T10:54:06.073023+0000","last_undegraded":"2024-03-27T10:54:06.073023+0000","last_fullsized":"2024-03-27T10:54:06.073023+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:44:46.374054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:06.192864+0000","last_change":"2024-03-27T10:54:05.095884+0000","last_active":"2024-03-27T10:54:06.192864+0000","last_peered":"2024-03-27T10:54:06.192864+0000","last_clean":"2024-03-27T10:54:06.192864+0000","last_became_active":"2024-03-27T10:54:05.095428+0000","last_became_peered":"2024-03-27T10:54:05.095428+0000","last_unstale":"2024-03-27T10:54:06.192864+0000","last_undegraded":"2024-03-27T10:54:06.192864+0000","last_fullsized":"2024-03-27T10:54:06.192864+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T11:19:49.656608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'6","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:11.194438+0000","last_change":"2024-03-27T10:54:04.064017+0000","last_active":"2024-03-27T10:54:11.194438+0000","last_peered":"2024-03-27T10:54:11.194438+0000","last_clean":"2024-03-27T10:54:11.194438+0000","last_became_active":"2024-03-27T10:54:04.062881+0000","last_became_peered":"2024-03-27T10:54:04.062881+0000","last_unstale":"2024-03-27T10:54:11.194438+0000","last_undegraded":"2024-03-27T10:54:11.194438+0000","last_fullsized":"2024-03-27T10:54:11.194438+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:21:21.638623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":344,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:06.074174+0000","last_change":"2024-03-27T10:52:31.127495+0000","last_active":"2024-03-27T10:54:06.074174+0000","last_peered":"2024-03-27T10:54:06.074174+0000","last_clean":"2024-03-27T10:54:06.074174+0000","last_became_active":"2024-03-27T10:52:31.125418+0000","last_became_peered":"2024-03-27T10:52:31.125418+0000","last_unstale":"2024-03-27T10:54:06.074174+0000","last_undegraded":"2024-03-27T10:54:06.074174+0000","last_fullsized":"2024-03-27T10:54:06.074174+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:47:54.649615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:47:54.649615+0000","last_clean_scrub_stamp":"2024-03-27T10:47:54.649615+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T18:49:24.591580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:06.073036+0000","last_change":"2024-03-27T10:54:05.085558+0000","last_active":"2024-03-27T10:54:06.073036+0000","last_peered":"2024-03-27T10:54:06.073036+0000","last_clean":"2024-03-27T10:54:06.073036+0000","last_became_active":"2024-03-27T10:54:05.083896+0000","last_became_peered":"2024-03-27T10:54:05.083896+0000","last_unstale":"2024-03-27T10:54:06.073036+0000","last_undegraded":"2024-03-27T10:54:06.073036+0000","last_fullsized":"2024-03-27T10:54:06.073036+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:15:29.697609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'1","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:06.074575+0000","last_change":"2024-03-27T10:54:04.072448+0000","last_active":"2024-03-27T10:54:06.074575+0000","last_peered":"2024-03-27T10:54:06.074575+0000","last_clean":"2024-03-27T10:54:06.074575+0000","last_became_active":"2024-03-27T10:54:04.071933+0000","last_became_peered":"2024-03-27T10:54:04.071933+0000","last_unstale":"2024-03-27T10:54:06.074575+0000","last_undegraded":"2024-03-27T10:54:06.074575+0000","last_fullsized":"2024-03-27T10:54:06.074575+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:34:49.204345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"90'4","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:10.182397+0000","last_change":"2024-03-27T10:54:04.067308+0000","last_active":"2024-03-27T10:54:10.182397+0000","last_peered":"2024-03-27T10:54:10.182397+0000","last_clean":"2024-03-27T10:54:10.182397+0000","last_became_active":"2024-03-27T10:54:04.066927+0000","last_became_peered":"2024-03-27T10:54:04.066927+0000","last_unstale":"2024-03-27T10:54:10.182397+0000","last_undegraded":"2024-03-27T10:54:10.182397+0000","last_fullsized":"2024-03-27T10:54:10.182397+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:08:04.849245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:06.073197+0000","last_change":"2024-03-27T10:54:05.075154+0000","last_active":"2024-03-27T10:54:06.073197+0000","last_peered":"2024-03-27T10:54:06.073197+0000","last_clean":"2024-03-27T10:54:06.073197+0000","last_became_active":"2024-03-27T10:54:05.074772+0000","last_became_peered":"2024-03-27T10:54:05.074772+0000","last_unstale":"2024-03-27T10:54:06.073197+0000","last_undegraded":"2024-03-27T10:54:06.073197+0000","last_fullsized":"2024-03-27T10:54:06.073197+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T20:49:16.286229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"90'9","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:11.248705+0000","last_change":"2024-03-27T10:54:04.066805+0000","last_active":"2024-03-27T10:54:11.248705+0000","last_peered":"2024-03-27T10:54:11.248705+0000","last_clean":"2024-03-27T10:54:11.248705+0000","last_became_active":"2024-03-27T10:54:04.066500+0000","last_became_peered":"2024-03-27T10:54:04.066500+0000","last_unstale":"2024-03-27T10:54:11.248705+0000","last_undegraded":"2024-03-27T10:54:11.248705+0000","last_fullsized":"2024-03-27T10:54:11.248705+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T12:37:44.853575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:06.192933+0000","last_change":"2024-03-27T10:54:05.094314+0000","last_active":"2024-03-27T10:54:06.192933+0000","last_peered":"2024-03-27T10:54:06.192933+0000","last_clean":"2024-03-27T10:54:06.192933+0000","last_became_active":"2024-03-27T10:54:05.093896+0000","last_became_peered":"2024-03-27T10:54:05.093896+0000","last_unstale":"2024-03-27T10:54:06.192933+0000","last_undegraded":"2024-03-27T10:54:06.192933+0000","last_fullsized":"2024-03-27T10:54:06.192933+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:30:02.441317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"90'7","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:11.195293+0000","last_change":"2024-03-27T10:54:04.065525+0000","last_active":"2024-03-27T10:54:11.195293+0000","last_peered":"2024-03-27T10:54:11.195293+0000","last_clean":"2024-03-27T10:54:11.195293+0000","last_became_active":"2024-03-27T10:54:04.064979+0000","last_became_peered":"2024-03-27T10:54:04.064979+0000","last_unstale":"2024-03-27T10:54:11.195293+0000","last_undegraded":"2024-03-27T10:54:11.195293+0000","last_fullsized":"2024-03-27T10:54:11.195293+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:58:25.110861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:07.003122+0000","last_change":"2024-03-27T10:54:05.084038+0000","last_active":"2024-03-27T10:54:07.003122+0000","last_peered":"2024-03-27T10:54:07.003122+0000","last_clean":"2024-03-27T10:54:07.003122+0000","last_became_active":"2024-03-27T10:54:05.083733+0000","last_became_peered":"2024-03-27T10:54:05.083733+0000","last_unstale":"2024-03-27T10:54:07.003122+0000","last_undegraded":"2024-03-27T10:54:07.003122+0000","last_fullsized":"2024-03-27T10:54:07.003122+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T18:51:14.832602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:08.617286+0000","last_change":"2024-03-27T10:54:04.064506+0000","last_active":"2024-03-27T10:54:08.617286+0000","last_peered":"2024-03-27T10:54:08.617286+0000","last_clean":"2024-03-27T10:54:08.617286+0000","last_became_active":"2024-03-27T10:54:04.064178+0000","last_became_peered":"2024-03-27T10:54:04.064178+0000","last_unstale":"2024-03-27T10:54:08.617286+0000","last_undegraded":"2024-03-27T10:54:08.617286+0000","last_fullsized":"2024-03-27T10:54:08.617286+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:07:19.465782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:06.257550+0000","last_change":"2024-03-27T10:54:05.088241+0000","last_active":"2024-03-27T10:54:06.257550+0000","last_peered":"2024-03-27T10:54:06.257550+0000","last_clean":"2024-03-27T10:54:06.257550+0000","last_became_active":"2024-03-27T10:54:05.087852+0000","last_became_peered":"2024-03-27T10:54:05.087852+0000","last_unstale":"2024-03-27T10:54:06.257550+0000","last_undegraded":"2024-03-27T10:54:06.257550+0000","last_fullsized":"2024-03-27T10:54:06.257550+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:47:09.633654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"90'7","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:11.193916+0000","last_change":"2024-03-27T10:54:04.065744+0000","last_active":"2024-03-27T10:54:11.193916+0000","last_peered":"2024-03-27T10:54:11.193916+0000","last_clean":"2024-03-27T10:54:11.193916+0000","last_became_active":"2024-03-27T10:54:04.065155+0000","last_became_peered":"2024-03-27T10:54:04.065155+0000","last_unstale":"2024-03-27T10:54:11.193916+0000","last_undegraded":"2024-03-27T10:54:11.193916+0000","last_fullsized":"2024-03-27T10:54:11.193916+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T12:35:00.556458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:07.004318+0000","last_change":"2024-03-27T10:54:05.077954+0000","last_active":"2024-03-27T10:54:07.004318+0000","last_peered":"2024-03-27T10:54:07.004318+0000","last_clean":"2024-03-27T10:54:07.004318+0000","last_became_active":"2024-03-27T10:54:05.077589+0000","last_became_peered":"2024-03-27T10:54:05.077589+0000","last_unstale":"2024-03-27T10:54:07.004318+0000","last_undegraded":"2024-03-27T10:54:07.004318+0000","last_fullsized":"2024-03-27T10:54:07.004318+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T20:18:27.411887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"90'3","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:10.180949+0000","last_change":"2024-03-27T10:54:04.067013+0000","last_active":"2024-03-27T10:54:10.180949+0000","last_peered":"2024-03-27T10:54:10.180949+0000","last_clean":"2024-03-27T10:54:10.180949+0000","last_became_active":"2024-03-27T10:54:04.066603+0000","last_became_peered":"2024-03-27T10:54:04.066603+0000","last_unstale":"2024-03-27T10:54:10.180949+0000","last_undegraded":"2024-03-27T10:54:10.180949+0000","last_fullsized":"2024-03-27T10:54:10.180949+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:41:21.678765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:06.192620+0000","last_change":"2024-03-27T10:54:05.088250+0000","last_active":"2024-03-27T10:54:06.192620+0000","last_peered":"2024-03-27T10:54:06.192620+0000","last_clean":"2024-03-27T10:54:06.192620+0000","last_became_active":"2024-03-27T10:54:05.087942+0000","last_became_peered":"2024-03-27T10:54:05.087942+0000","last_unstale":"2024-03-27T10:54:06.192620+0000","last_undegraded":"2024-03-27T10:54:06.192620+0000","last_fullsized":"2024-03-27T10:54:06.192620+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:57:45.266347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:08.619790+0000","last_change":"2024-03-27T10:54:04.070513+0000","last_active":"2024-03-27T10:54:08.619790+0000","last_peered":"2024-03-27T10:54:08.619790+0000","last_clean":"2024-03-27T10:54:08.619790+0000","last_became_active":"2024-03-27T10:54:04.070042+0000","last_became_peered":"2024-03-27T10:54:04.070042+0000","last_unstale":"2024-03-27T10:54:08.619790+0000","last_undegraded":"2024-03-27T10:54:08.619790+0000","last_fullsized":"2024-03-27T10:54:08.619790+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:40:25.216899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:06.075411+0000","last_change":"2024-03-27T10:54:05.095067+0000","last_active":"2024-03-27T10:54:06.075411+0000","last_peered":"2024-03-27T10:54:06.075411+0000","last_clean":"2024-03-27T10:54:06.075411+0000","last_became_active":"2024-03-27T10:54:05.094618+0000","last_became_peered":"2024-03-27T10:54:05.094618+0000","last_unstale":"2024-03-27T10:54:06.075411+0000","last_undegraded":"2024-03-27T10:54:06.075411+0000","last_fullsized":"2024-03-27T10:54:06.075411+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:57:59.557310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"90'5","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:12.201433+0000","last_change":"2024-03-27T10:54:04.051352+0000","last_active":"2024-03-27T10:54:12.201433+0000","last_peered":"2024-03-27T10:54:12.201433+0000","last_clean":"2024-03-27T10:54:12.201433+0000","last_became_active":"2024-03-27T10:54:04.050850+0000","last_became_peered":"2024-03-27T10:54:04.050850+0000","last_unstale":"2024-03-27T10:54:12.201433+0000","last_undegraded":"2024-03-27T10:54:12.201433+0000","last_fullsized":"2024-03-27T10:54:12.201433+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:50:16.719451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:06.076078+0000","last_change":"2024-03-27T10:54:05.094650+0000","last_active":"2024-03-27T10:54:06.076078+0000","last_peered":"2024-03-27T10:54:06.076078+0000","last_clean":"2024-03-27T10:54:06.076078+0000","last_became_active":"2024-03-27T10:54:05.094168+0000","last_became_peered":"2024-03-27T10:54:05.094168+0000","last_unstale":"2024-03-27T10:54:06.076078+0000","last_undegraded":"2024-03-27T10:54:06.076078+0000","last_fullsized":"2024-03-27T10:54:06.076078+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:29:56.491733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"89'1","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:06.075077+0000","last_change":"2024-03-27T10:54:04.067755+0000","last_active":"2024-03-27T10:54:06.075077+0000","last_peered":"2024-03-27T10:54:06.075077+0000","last_clean":"2024-03-27T10:54:06.075077+0000","last_became_active":"2024-03-27T10:54:04.067320+0000","last_became_peered":"2024-03-27T10:54:04.067320+0000","last_unstale":"2024-03-27T10:54:06.075077+0000","last_undegraded":"2024-03-27T10:54:06.075077+0000","last_fullsized":"2024-03-27T10:54:06.075077+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T12:16:25.949829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:06.074766+0000","last_change":"2024-03-27T10:54:05.100715+0000","last_active":"2024-03-27T10:54:06.074766+0000","last_peered":"2024-03-27T10:54:06.074766+0000","last_clean":"2024-03-27T10:54:06.074766+0000","last_became_active":"2024-03-27T10:54:05.097861+0000","last_became_peered":"2024-03-27T10:54:05.097861+0000","last_unstale":"2024-03-27T10:54:06.074766+0000","last_undegraded":"2024-03-27T10:54:06.074766+0000","last_fullsized":"2024-03-27T10:54:06.074766+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:02:39.107232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"89'2","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:06.073541+0000","last_change":"2024-03-27T10:54:04.063894+0000","last_active":"2024-03-27T10:54:06.073541+0000","last_peered":"2024-03-27T10:54:06.073541+0000","last_clean":"2024-03-27T10:54:06.073541+0000","last_became_active":"2024-03-27T10:54:04.063163+0000","last_became_peered":"2024-03-27T10:54:04.063163+0000","last_unstale":"2024-03-27T10:54:06.073541+0000","last_undegraded":"2024-03-27T10:54:06.073541+0000","last_fullsized":"2024-03-27T10:54:06.073541+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:30:41.025752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:06.076166+0000","last_change":"2024-03-27T10:54:05.101980+0000","last_active":"2024-03-27T10:54:06.076166+0000","last_peered":"2024-03-27T10:54:06.076166+0000","last_clean":"2024-03-27T10:54:06.076166+0000","last_became_active":"2024-03-27T10:54:05.101448+0000","last_became_peered":"2024-03-27T10:54:05.101448+0000","last_unstale":"2024-03-27T10:54:06.076166+0000","last_undegraded":"2024-03-27T10:54:06.076166+0000","last_fullsized":"2024-03-27T10:54:06.076166+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:45:11.137181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:07.004605+0000","last_change":"2024-03-27T10:54:05.084026+0000","last_active":"2024-03-27T10:54:07.004605+0000","last_peered":"2024-03-27T10:54:07.004605+0000","last_clean":"2024-03-27T10:54:07.004605+0000","last_became_active":"2024-03-27T10:54:05.083410+0000","last_became_peered":"2024-03-27T10:54:05.083410+0000","last_unstale":"2024-03-27T10:54:07.004605+0000","last_undegraded":"2024-03-27T10:54:07.004605+0000","last_fullsized":"2024-03-27T10:54:07.004605+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T22:39:34.586363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:07.005234+0000","last_change":"2024-03-27T10:54:05.083592+0000","last_active":"2024-03-27T10:54:07.005234+0000","last_peered":"2024-03-27T10:54:07.005234+0000","last_clean":"2024-03-27T10:54:07.005234+0000","last_became_active":"2024-03-27T10:54:05.082537+0000","last_became_peered":"2024-03-27T10:54:05.082537+0000","last_unstale":"2024-03-27T10:54:07.005234+0000","last_undegraded":"2024-03-27T10:54:07.005234+0000","last_fullsized":"2024-03-27T10:54:07.005234+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T21:15:36.830805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:07.005069+0000","last_change":"2024-03-27T10:54:05.090226+0000","last_active":"2024-03-27T10:54:07.005069+0000","last_peered":"2024-03-27T10:54:07.005069+0000","last_clean":"2024-03-27T10:54:07.005069+0000","last_became_active":"2024-03-27T10:54:05.089831+0000","last_became_peered":"2024-03-27T10:54:05.089831+0000","last_unstale":"2024-03-27T10:54:07.005069+0000","last_undegraded":"2024-03-27T10:54:07.005069+0000","last_fullsized":"2024-03-27T10:54:07.005069+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T12:45:57.802266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:07.002774+0000","last_change":"2024-03-27T10:54:05.080196+0000","last_active":"2024-03-27T10:54:07.002774+0000","last_peered":"2024-03-27T10:54:07.002774+0000","last_clean":"2024-03-27T10:54:07.002774+0000","last_became_active":"2024-03-27T10:54:05.079822+0000","last_became_peered":"2024-03-27T10:54:05.079822+0000","last_unstale":"2024-03-27T10:54:07.002774+0000","last_undegraded":"2024-03-27T10:54:07.002774+0000","last_fullsized":"2024-03-27T10:54:07.002774+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:53:11.556503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:06.073502+0000","last_change":"2024-03-27T10:54:05.087264+0000","last_active":"2024-03-27T10:54:06.073502+0000","last_peered":"2024-03-27T10:54:06.073502+0000","last_clean":"2024-03-27T10:54:06.073502+0000","last_became_active":"2024-03-27T10:54:05.085440+0000","last_became_peered":"2024-03-27T10:54:05.085440+0000","last_unstale":"2024-03-27T10:54:06.073502+0000","last_undegraded":"2024-03-27T10:54:06.073502+0000","last_fullsized":"2024-03-27T10:54:06.073502+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T22:41:40.665643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:07.005423+0000","last_change":"2024-03-27T10:54:05.094306+0000","last_active":"2024-03-27T10:54:07.005423+0000","last_peered":"2024-03-27T10:54:07.005423+0000","last_clean":"2024-03-27T10:54:07.005423+0000","last_became_active":"2024-03-27T10:54:05.093858+0000","last_became_peered":"2024-03-27T10:54:05.093858+0000","last_unstale":"2024-03-27T10:54:07.005423+0000","last_undegraded":"2024-03-27T10:54:07.005423+0000","last_fullsized":"2024-03-27T10:54:07.005423+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T21:22:57.624833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:07.006193+0000","last_change":"2024-03-27T10:54:05.094089+0000","last_active":"2024-03-27T10:54:07.006193+0000","last_peered":"2024-03-27T10:54:07.006193+0000","last_clean":"2024-03-27T10:54:07.006193+0000","last_became_active":"2024-03-27T10:54:05.092317+0000","last_became_peered":"2024-03-27T10:54:05.092317+0000","last_unstale":"2024-03-27T10:54:07.006193+0000","last_undegraded":"2024-03-27T10:54:07.006193+0000","last_fullsized":"2024-03-27T10:54:07.006193+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:34:57.606250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:06.192922+0000","last_change":"2024-03-27T10:54:05.099995+0000","last_active":"2024-03-27T10:54:06.192922+0000","last_peered":"2024-03-27T10:54:06.192922+0000","last_clean":"2024-03-27T10:54:06.192922+0000","last_became_active":"2024-03-27T10:54:05.099641+0000","last_became_peered":"2024-03-27T10:54:05.099641+0000","last_unstale":"2024-03-27T10:54:06.192922+0000","last_undegraded":"2024-03-27T10:54:06.192922+0000","last_fullsized":"2024-03-27T10:54:06.192922+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:01:16.050180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:06.074370+0000","last_change":"2024-03-27T10:54:05.082138+0000","last_active":"2024-03-27T10:54:06.074370+0000","last_peered":"2024-03-27T10:54:06.074370+0000","last_clean":"2024-03-27T10:54:06.074370+0000","last_became_active":"2024-03-27T10:54:05.081825+0000","last_became_peered":"2024-03-27T10:54:05.081825+0000","last_unstale":"2024-03-27T10:54:06.074370+0000","last_undegraded":"2024-03-27T10:54:06.074370+0000","last_fullsized":"2024-03-27T10:54:06.074370+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:51:07.635412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:06.192125+0000","last_change":"2024-03-27T10:54:05.096812+0000","last_active":"2024-03-27T10:54:06.192125+0000","last_peered":"2024-03-27T10:54:06.192125+0000","last_clean":"2024-03-27T10:54:06.192125+0000","last_became_active":"2024-03-27T10:54:05.096323+0000","last_became_peered":"2024-03-27T10:54:05.096323+0000","last_unstale":"2024-03-27T10:54:06.192125+0000","last_undegraded":"2024-03-27T10:54:06.192125+0000","last_fullsized":"2024-03-27T10:54:06.192125+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:39:29.827291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:07.005005+0000","last_change":"2024-03-27T10:54:05.089137+0000","last_active":"2024-03-27T10:54:07.005005+0000","last_peered":"2024-03-27T10:54:07.005005+0000","last_clean":"2024-03-27T10:54:07.005005+0000","last_became_active":"2024-03-27T10:54:05.088795+0000","last_became_peered":"2024-03-27T10:54:05.088795+0000","last_unstale":"2024-03-27T10:54:07.005005+0000","last_undegraded":"2024-03-27T10:54:07.005005+0000","last_fullsized":"2024-03-27T10:54:07.005005+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T19:47:35.231749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:06.073082+0000","last_change":"2024-03-27T10:54:05.089683+0000","last_active":"2024-03-27T10:54:06.073082+0000","last_peered":"2024-03-27T10:54:06.073082+0000","last_clean":"2024-03-27T10:54:06.073082+0000","last_became_active":"2024-03-27T10:54:05.089235+0000","last_became_peered":"2024-03-27T10:54:05.089235+0000","last_unstale":"2024-03-27T10:54:06.073082+0000","last_undegraded":"2024-03-27T10:54:06.073082+0000","last_fullsized":"2024-03-27T10:54:06.073082+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:43:37.012814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-27T10:54:07.004811+0000","last_change":"2024-03-27T10:54:05.085043+0000","last_active":"2024-03-27T10:54:07.004811+0000","last_peered":"2024-03-27T10:54:07.004811+0000","last_clean":"2024-03-27T10:54:07.004811+0000","last_became_active":"2024-03-27T10:54:05.084695+0000","last_became_peered":"2024-03-27T10:54:05.084695+0000","last_unstale":"2024-03-27T10:54:07.004811+0000","last_undegraded":"2024-03-27T10:54:07.004811+0000","last_fullsized":"2024-03-27T10:54:07.004811+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T12:07:31.327006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":43634,"num_objects":84,"num_object_clones":0,"num_object_copies":252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84,"num_whiteouts":0,"num_read":46,"num_read_kb":71,"num_write":193,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":57,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":113716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":166,"ondisk_log_size":166,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":85,"seq":365072220177,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":463333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":10,"up_from":77,"seq":330712481817,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36956,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708196,"statfs":{"total":95995035648,"available":95957192704,"internally_reserved":0,"allocated":1658880,"data_stored":1092946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":71,"seq":304942678047,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":1003520,"data_stored":463263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":8,"up_from":63,"seq":270582939686,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36312,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708840,"statfs":{"total":95995035648,"available":95957852160,"internally_reserved":0,"allocated":999424,"data_stored":463641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":54,"seq":231928234028,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28552,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716600,"statfs":{"total":95995035648,"available":95965798400,"internally_reserved":0,"allocated":1572864,"data_stored":1053713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":49,"seq":210453397554,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":966656,"data_stored":462141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574933,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":470935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738459,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":464207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411406,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1048576,"data_stored":502697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117065,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":1003520,"data_stored":462805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":35,"seq":150323855426,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36300,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708852,"statfs":{"total":95995035648,"available":95957864448,"internally_reserved":0,"allocated":987136,"data_stored":463232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":41,"seq":176093659194,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32728,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712424,"statfs":{"total":95995035648,"available":95961522176,"internally_reserved":0,"allocated":1589248,"data_stored":1053985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":22647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":40556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":40320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-27T10:54:14.579 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-27T10:54:14.580 INFO:tasks.cephadm:Setting up client nodes... 2024-03-27T10:54:14.580 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid adca88c4-ec26-11ee-95d0-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-03-27T10:54:15.057 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T10:54:15.493 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:15 smithi097 ceph-mon[31859]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-27T10:54:15.493 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:15 smithi097 ceph-mon[31859]: pgmap v367: 97 pgs: 97 active+clean; 619 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 9.7 KiB/s wr, 26 op/s 2024-03-27T10:54:15.517 INFO:journalctl@ceph.mds.b.smithi114.stdout:Mar 27 10:54:15 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-b[65857]: 2024-03-27T10:54:15.121+0000 7f0399000700 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-03-27T10:54:15.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:15 smithi114 ceph-mon[35338]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-27T10:54:15.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:15 smithi114 ceph-mon[35338]: pgmap v367: 97 pgs: 97 active+clean; 619 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 9.7 KiB/s wr, 26 op/s 2024-03-27T10:54:15.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:15 smithi177 ceph-mon[35294]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-27T10:54:15.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:15 smithi177 ceph-mon[35294]: pgmap v367: 97 pgs: 97 active+clean; 619 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 9.7 KiB/s wr, 26 op/s 2024-03-27T10:54:16.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:16 smithi114 ceph-mon[35338]: from='client.14808 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-27T10:54:16.595 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:16 smithi097 ceph-mon[31859]: from='client.14808 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-27T10:54:16.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:16 smithi177 ceph-mon[35294]: from='client.14808 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-27T10:54:17.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:17.201+0000 7f8dcbc00700 1 -- 172.21.15.97:0/60416545 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8dc4103890 msgr2=0x7f8dc4103cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:17.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:17.201+0000 7f8dcbc00700 1 --2- 172.21.15.97:0/60416545 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8dc4103890 0x7f8dc4103cf0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f8db00099c0 tx=0x7f8db0009cd0 comp rx=0 tx=0).stop 2024-03-27T10:54:17.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:17.201+0000 7f8dcbc00700 1 -- 172.21.15.97:0/60416545 shutdown_connections 2024-03-27T10:54:17.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:17.201+0000 7f8dcbc00700 1 --2- 172.21.15.97:0/60416545 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8dc4104230 0x7f8dc410aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:17.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:17.201+0000 7f8dcbc00700 1 --2- 172.21.15.97:0/60416545 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8dc4102690 0x7f8dc4102ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:17.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:17.201+0000 7f8dcbc00700 1 --2- 172.21.15.97:0/60416545 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8dc4103890 0x7f8dc4103cf0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:17.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:17.201+0000 7f8dcbc00700 1 -- 172.21.15.97:0/60416545 >> 172.21.15.97:0/60416545 conn(0x7f8dc40fdc50 msgr2=0x7f8dc4100070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:54:17.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:17.201+0000 7f8dcbc00700 1 -- 172.21.15.97:0/60416545 shutdown_connections 2024-03-27T10:54:17.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:17.201+0000 7f8dcbc00700 1 -- 172.21.15.97:0/60416545 wait complete. 2024-03-27T10:54:17.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:17.202+0000 7f8dcbc00700 1 Processor -- start 2024-03-27T10:54:17.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:17.202+0000 7f8dcbc00700 1 -- start start 2024-03-27T10:54:17.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:17.203+0000 7f8dcbc00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8dc4102690 0x7f8dc40763e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:17.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:17.203+0000 7f8dcbc00700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8dc4103890 0x7f8dc4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:17.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:17.203+0000 7f8dcbc00700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8dc4104230 0x7f8dc4074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:17.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:17.203+0000 7f8dcbc00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8dc410e180 con 0x7f8dc4102690 2024-03-27T10:54:17.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:17.203+0000 7f8dcbc00700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f8dc410e000 con 0x7f8dc4104230 2024-03-27T10:54:17.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:17.203+0000 7f8dcbc00700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f8dc410e300 con 0x7f8dc4103890 2024-03-27T10:54:17.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:17.203+0000 7f8dc9200700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8dc4102690 0x7f8dc40763e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:17.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:17.203+0000 7f8dc9200700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8dc4102690 0x7f8dc40763e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43324/0 (socket says 172.21.15.97:43324) 2024-03-27T10:54:17.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:17.203+0000 7f8dc9200700 1 -- 172.21.15.97:0/2946693768 learned_addr learned my addr 172.21.15.97:0/2946693768 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:54:17.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:17.203+0000 7f8dc9c00700 1 --2- 172.21.15.97:0/2946693768 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8dc4104230 0x7f8dc4074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:17.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:17.203+0000 7f8dc3e00700 1 --2- 172.21.15.97:0/2946693768 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8dc4103890 0x7f8dc4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:17.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:17.203+0000 7f8dc9200700 1 -- 172.21.15.97:0/2946693768 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8dc4103890 msgr2=0x7f8dc4074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:17.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:17.203+0000 7f8dc9200700 1 --2- 172.21.15.97:0/2946693768 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8dc4103890 0x7f8dc4074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:17.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:17.203+0000 7f8dc9200700 1 -- 172.21.15.97:0/2946693768 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8dc4104230 msgr2=0x7f8dc4074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:17.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:17.204+0000 7f8dc9200700 1 --2- 172.21.15.97:0/2946693768 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8dc4104230 0x7f8dc4074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:17.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:17.204+0000 7f8dc9200700 1 -- 172.21.15.97:0/2946693768 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dc4075820 con 0x7f8dc4102690 2024-03-27T10:54:17.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:17.204+0000 7f8dc9200700 1 --2- 172.21.15.97:0/2946693768 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8dc4102690 0x7f8dc40763e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8db800e970 tx=0x7f8db800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:54:17.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:17.204+0000 7f8dc1600700 1 -- 172.21.15.97:0/2946693768 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8db800cbd0 con 0x7f8dc4102690 2024-03-27T10:54:17.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:17.204+0000 7f8dc1600700 1 -- 172.21.15.97:0/2946693768 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8db80044d0 con 0x7f8dc4102690 2024-03-27T10:54:17.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:17.204+0000 7f8dcbc00700 1 -- 172.21.15.97:0/2946693768 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8dc41a28a0 con 0x7f8dc4102690 2024-03-27T10:54:17.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:17.204+0000 7f8dc1600700 1 -- 172.21.15.97:0/2946693768 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8db8003da0 con 0x7f8dc4102690 2024-03-27T10:54:17.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:17.204+0000 7f8dcbc00700 1 -- 172.21.15.97:0/2946693768 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8dc41a2de0 con 0x7f8dc4102690 2024-03-27T10:54:17.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:17.206+0000 7f8dcbc00700 1 -- 172.21.15.97:0/2946693768 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8dc4050b50 con 0x7f8dc4102690 2024-03-27T10:54:17.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:17.208+0000 7f8dc1600700 1 -- 172.21.15.97:0/2946693768 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138421+0+0 (secure 0 0 0) 0x7f8db8003f40 con 0x7f8dc4102690 2024-03-27T10:54:17.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:17.210+0000 7f8dc1600700 1 --2- 172.21.15.97:0/2946693768 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f8dac0a3860 0x7f8dac0a5d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:17.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:17.210+0000 7f8dc1600700 1 -- 172.21.15.97:0/2946693768 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 7979+0+0 (secure 0 0 0) 0x7f8db80d0160 con 0x7f8dc4102690 2024-03-27T10:54:17.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:17.210+0000 7f8dc3e00700 1 --2- 172.21.15.97:0/2946693768 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f8dac0a3860 0x7f8dac0a5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:17.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:17.211+0000 7f8dc3e00700 1 --2- 172.21.15.97:0/2946693768 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f8dac0a3860 0x7f8dac0a5d20 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8db0000c00 tx=0x7f8db0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:54:17.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:17.213+0000 7f8dc1600700 1 -- 172.21.15.97:0/2946693768 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f8db809cd10 con 0x7f8dc4102690 2024-03-27T10:54:17.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:17 smithi097 ceph-mon[31859]: pgmap v368: 97 pgs: 97 active+clean; 621 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 11 KiB/s wr, 29 op/s 2024-03-27T10:54:17.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:17.524+0000 7f8dcbc00700 1 -- 172.21.15.97:0/2946693768 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f8dc402ddb0 con 0x7f8dc4102690 2024-03-27T10:54:17.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:17.528+0000 7f8dc1600700 1 -- 172.21.15.97:0/2946693768 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v33) ==== 170+0+59 (secure 0 0 0) 0x7f8db809c610 con 0x7f8dc4102690 2024-03-27T10:54:17.502 INFO:teuthology.orchestra.run.smithi097.stdout:[client.0] 2024-03-27T10:54:17.502 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQDZ+gNmfrleHxAA9EE8CJobWKM2newy3+xkaw== 2024-03-27T10:54:17.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:17.531+0000 7f8dcbc00700 1 -- 172.21.15.97:0/2946693768 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f8dac0a3860 msgr2=0x7f8dac0a5d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:17.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:17.531+0000 7f8dcbc00700 1 --2- 172.21.15.97:0/2946693768 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f8dac0a3860 0x7f8dac0a5d20 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8db0000c00 tx=0x7f8db0005fb0 comp rx=0 tx=0).stop 2024-03-27T10:54:17.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:17.531+0000 7f8dcbc00700 1 -- 172.21.15.97:0/2946693768 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8dc4102690 msgr2=0x7f8dc40763e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:17.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:17.531+0000 7f8dcbc00700 1 --2- 172.21.15.97:0/2946693768 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8dc4102690 0x7f8dc40763e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8db800e970 tx=0x7f8db800ec80 comp rx=0 tx=0).stop 2024-03-27T10:54:17.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:17.532+0000 7f8dcbc00700 1 -- 172.21.15.97:0/2946693768 shutdown_connections 2024-03-27T10:54:17.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:17.532+0000 7f8dcbc00700 1 --2- 172.21.15.97:0/2946693768 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f8dac0a3860 0x7f8dac0a5d20 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:17.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:17.532+0000 7f8dcbc00700 1 --2- 172.21.15.97:0/2946693768 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8dc4104230 0x7f8dc4074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:17.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:17.532+0000 7f8dcbc00700 1 --2- 172.21.15.97:0/2946693768 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8dc4102690 0x7f8dc40763e0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:17.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:17.532+0000 7f8dcbc00700 1 --2- 172.21.15.97:0/2946693768 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8dc4103890 0x7f8dc4074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:17.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:17.532+0000 7f8dcbc00700 1 -- 172.21.15.97:0/2946693768 >> 172.21.15.97:0/2946693768 conn(0x7f8dc40fdc50 msgr2=0x7f8dc4106b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:54:17.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:17.532+0000 7f8dcbc00700 1 -- 172.21.15.97:0/2946693768 shutdown_connections 2024-03-27T10:54:17.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:17.532+0000 7f8dcbc00700 1 -- 172.21.15.97:0/2946693768 wait complete. 2024-03-27T10:54:17.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:17 smithi114 ceph-mon[35338]: pgmap v368: 97 pgs: 97 active+clean; 621 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 11 KiB/s wr, 29 op/s 2024-03-27T10:54:17.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:17 smithi177 ceph-mon[35294]: pgmap v368: 97 pgs: 97 active+clean; 621 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 11 KiB/s wr, 29 op/s 2024-03-27T10:54:18.269 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-27T10:54:18.269 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-03-27T10:54:18.269 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-27T10:54:18.417 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-27T10:54:18.418 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-03-27T10:54:18.418 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell --fsid adca88c4-ec26-11ee-95d0-87774f69a715 -- ceph mgr dump --format=json 2024-03-27T10:54:18.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:18 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/2946693768' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-27T10:54:18.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:18 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/2946693768' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-27T10:54:18.545 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:18 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/2946693768' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-27T10:54:18.546 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:18 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/2946693768' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-27T10:54:18.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:18 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/2946693768' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-27T10:54:18.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:18 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/2946693768' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-27T10:54:18.912 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T10:54:19.381 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:19 smithi097 ceph-mon[31859]: pgmap v369: 97 pgs: 97 active+clean; 621 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 8.9 KiB/s wr, 25 op/s 2024-03-27T10:54:19.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:19 smithi114 ceph-mon[35338]: pgmap v369: 97 pgs: 97 active+clean; 621 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 8.9 KiB/s wr, 25 op/s 2024-03-27T10:54:19.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:19 smithi177 ceph-mon[35294]: pgmap v369: 97 pgs: 97 active+clean; 621 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 8.9 KiB/s wr, 25 op/s 2024-03-27T10:54:21.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:21.096+0000 7fb1b9400700 1 -- 172.21.15.97:0/2179710500 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1b4108680 msgr2=0x7fb1b4108a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:21.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:21.096+0000 7fb1b9400700 1 --2- 172.21.15.97:0/2179710500 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1b4108680 0x7fb1b4108a60 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb1a4009990 tx=0x7fb1a4009ca0 comp rx=0 tx=0).stop 2024-03-27T10:54:21.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:21.096+0000 7fb1b9400700 1 -- 172.21.15.97:0/2179710500 shutdown_connections 2024-03-27T10:54:21.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:21.096+0000 7fb1b9400700 1 --2- 172.21.15.97:0/2179710500 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb1b4103050 0x7fb1b410f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:21.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:21.096+0000 7fb1b9400700 1 --2- 172.21.15.97:0/2179710500 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1b4108680 0x7fb1b4108a60 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:21.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:21.096+0000 7fb1b9400700 1 --2- 172.21.15.97:0/2179710500 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb1b4102690 0x7fb1b4102b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:21.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:21.096+0000 7fb1b9400700 1 -- 172.21.15.97:0/2179710500 >> 172.21.15.97:0/2179710500 conn(0x7fb1b40797a0 msgr2=0x7fb1b4079bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:54:21.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:21.097+0000 7fb1b9400700 1 -- 172.21.15.97:0/2179710500 shutdown_connections 2024-03-27T10:54:21.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:21.097+0000 7fb1b9400700 1 -- 172.21.15.97:0/2179710500 wait complete. 2024-03-27T10:54:21.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:21.098+0000 7fb1b9400700 1 Processor -- start 2024-03-27T10:54:21.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:21.098+0000 7fb1b9400700 1 -- start start 2024-03-27T10:54:21.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:21.098+0000 7fb1b9400700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb1b4102690 0x7fb1b4198d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:21.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:21.098+0000 7fb1b9400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1b4103050 0x7fb1b41992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:21.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:21.098+0000 7fb1b9400700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb1b4108680 0x7fb1b419d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:21.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:21.098+0000 7fb1b9400700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b41128b0 con 0x7fb1b4103050 2024-03-27T10:54:21.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:21.098+0000 7fb1b9400700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b4112730 con 0x7fb1b4108680 2024-03-27T10:54:21.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:21.098+0000 7fb1b9400700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b4112a30 con 0x7fb1b4102690 2024-03-27T10:54:21.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:21.099+0000 7fb1b2800700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb1b4102690 0x7fb1b4198d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:21.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:21.099+0000 7fb1b1e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1b4103050 0x7fb1b41992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:21.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:21.099+0000 7fb1b3200700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb1b4108680 0x7fb1b419d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:21.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:21.099+0000 7fb1b1e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1b4103050 0x7fb1b41992c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58554/0 (socket says 172.21.15.97:58554) 2024-03-27T10:54:21.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:21.099+0000 7fb1b1e00700 1 -- 172.21.15.97:0/2289051059 learned_addr learned my addr 172.21.15.97:0/2289051059 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:54:21.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:21.099+0000 7fb1b1e00700 1 -- 172.21.15.97:0/2289051059 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb1b4102690 msgr2=0x7fb1b4198d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:21.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:21.099+0000 7fb1b1e00700 1 --2- 172.21.15.97:0/2289051059 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb1b4102690 0x7fb1b4198d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:21.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:21.099+0000 7fb1b1e00700 1 -- 172.21.15.97:0/2289051059 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb1b4108680 msgr2=0x7fb1b419d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:21.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:21.099+0000 7fb1b1e00700 1 --2- 172.21.15.97:0/2289051059 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb1b4108680 0x7fb1b419d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:21.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:21.099+0000 7fb1b1e00700 1 -- 172.21.15.97:0/2289051059 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1b419dd90 con 0x7fb1b4103050 2024-03-27T10:54:21.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:21.099+0000 7fb1b3200700 1 --2- 172.21.15.97:0/2289051059 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb1b4108680 0x7fb1b419d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T10:54:21.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:21.100+0000 7fb1b1e00700 1 --2- 172.21.15.97:0/2289051059 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1b4103050 0x7fb1b41992c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb1a000eb30 tx=0x7fb1a000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:54:21.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:21.100+0000 7fb1ab400700 1 -- 172.21.15.97:0/2289051059 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a000cbd0 con 0x7fb1b4103050 2024-03-27T10:54:21.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:21.100+0000 7fb1ab400700 1 -- 172.21.15.97:0/2289051059 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb1a00044d0 con 0x7fb1b4103050 2024-03-27T10:54:21.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:21.100+0000 7fb1b9400700 1 -- 172.21.15.97:0/2289051059 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1b419e080 con 0x7fb1b4103050 2024-03-27T10:54:21.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:21.100+0000 7fb1ab400700 1 -- 172.21.15.97:0/2289051059 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a0003e30 con 0x7fb1b4103050 2024-03-27T10:54:21.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:21.100+0000 7fb1b9400700 1 -- 172.21.15.97:0/2289051059 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1b41a5900 con 0x7fb1b4103050 2024-03-27T10:54:21.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:21.101+0000 7fb1b9400700 1 -- 172.21.15.97:0/2289051059 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1b4069300 con 0x7fb1b4103050 2024-03-27T10:54:21.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:21.103+0000 7fb1ab400700 1 -- 172.21.15.97:0/2289051059 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138421+0+0 (secure 0 0 0) 0x7fb1a0004000 con 0x7fb1b4103050 2024-03-27T10:54:21.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:21.104+0000 7fb1ab400700 1 --2- 172.21.15.97:0/2289051059 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fb1980a37b0 0x7fb1980a5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:21.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:21.105+0000 7fb1ab400700 1 -- 172.21.15.97:0/2289051059 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 7979+0+0 (secure 0 0 0) 0x7fb1a00d00b0 con 0x7fb1b4103050 2024-03-27T10:54:21.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:21.105+0000 7fb1b2800700 1 --2- 172.21.15.97:0/2289051059 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fb1980a37b0 0x7fb1980a5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:21.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:21.105+0000 7fb1b2800700 1 --2- 172.21.15.97:0/2289051059 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fb1980a37b0 0x7fb1980a5c70 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb1a400b2e0 tx=0x7fb1a4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:54:21.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:21.109+0000 7fb1ab400700 1 -- 172.21.15.97:0/2289051059 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7fb1a00042f0 con 0x7fb1b4103050 2024-03-27T10:54:21.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:21.398+0000 7fb1b9400700 1 -- 172.21.15.97:0/2289051059 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fb1b419a110 con 0x7fb1b4103050 2024-03-27T10:54:21.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:21.404+0000 7fb1ab400700 1 -- 172.21.15.97:0/2289051059 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+264395 (secure 0 0 0) 0x7fb1b419a110 con 0x7fb1b4103050 2024-03-27T10:54:21.378 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T10:54:21.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:21.410+0000 7fb1b9400700 1 -- 172.21.15.97:0/2289051059 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fb1980a37b0 msgr2=0x7fb1980a5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:21.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:21.410+0000 7fb1b9400700 1 --2- 172.21.15.97:0/2289051059 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fb1980a37b0 0x7fb1980a5c70 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb1a400b2e0 tx=0x7fb1a4005fd0 comp rx=0 tx=0).stop 2024-03-27T10:54:21.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:21.410+0000 7fb1b9400700 1 -- 172.21.15.97:0/2289051059 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1b4103050 msgr2=0x7fb1b41992c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:21.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:21.410+0000 7fb1b9400700 1 --2- 172.21.15.97:0/2289051059 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1b4103050 0x7fb1b41992c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb1a000eb30 tx=0x7fb1a000ee40 comp rx=0 tx=0).stop 2024-03-27T10:54:21.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:21.410+0000 7fb1b9400700 1 -- 172.21.15.97:0/2289051059 shutdown_connections 2024-03-27T10:54:21.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:21.410+0000 7fb1b9400700 1 --2- 172.21.15.97:0/2289051059 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fb1980a37b0 0x7fb1980a5c70 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:21.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:21.410+0000 7fb1b9400700 1 --2- 172.21.15.97:0/2289051059 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb1b4108680 0x7fb1b419d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:21.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:21.410+0000 7fb1b9400700 1 --2- 172.21.15.97:0/2289051059 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb1b4102690 0x7fb1b4198d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:21.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:21.410+0000 7fb1b9400700 1 --2- 172.21.15.97:0/2289051059 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1b4103050 0x7fb1b41992c0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:21.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:21.410+0000 7fb1b9400700 1 -- 172.21.15.97:0/2289051059 >> 172.21.15.97:0/2289051059 conn(0x7fb1b40797a0 msgr2=0x7fb1b4100740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:54:21.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:21.411+0000 7fb1b9400700 1 -- 172.21.15.97:0/2289051059 shutdown_connections 2024-03-27T10:54:21.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:21.411+0000 7fb1b9400700 1 -- 172.21.15.97:0/2289051059 wait complete. 2024-03-27T10:54:21.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:21 smithi114 ceph-mon[35338]: pgmap v370: 97 pgs: 97 active+clean; 626 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.3 KiB/s wr, 28 op/s 2024-03-27T10:54:21.641 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:21 smithi097 ceph-mon[31859]: pgmap v370: 97 pgs: 97 active+clean; 626 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.3 KiB/s wr, 28 op/s 2024-03-27T10:54:21.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:21 smithi177 ceph-mon[35294]: pgmap v370: 97 pgs: 97 active+clean; 626 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.3 KiB/s wr, 28 op/s 2024-03-27T10:54:22.245 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3955463538},{"type":"v1","addr":"172.21.15.97:6801","nonce":3955463538}]},"active_addr":"172.21.15.97:6801/3955463538","active_change":"2024-03-27T10:44:39.889650+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":34109,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.97:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.97:0","nonce":1887548681}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.97:0","nonce":2584280197}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.97:0","nonce":3268554894}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.97:0","nonce":1075669214}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.97:0","nonce":2177463268}]}]} 2024-03-27T10:54:22.250 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-03-27T10:54:22.250 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-03-27T10:54:22.250 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell --fsid adca88c4-ec26-11ee-95d0-87774f69a715 -- ceph osd dump --format=json 2024-03-27T10:54:22.364 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:22 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/2289051059' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-27T10:54:22.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:22 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/2289051059' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-27T10:54:22.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:22 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/2289051059' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-27T10:54:22.727 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T10:54:23.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:23 smithi114 ceph-mon[35338]: pgmap v371: 97 pgs: 97 active+clean; 626 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.9 KiB/s wr, 23 op/s 2024-03-27T10:54:23.579 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:23 smithi177 ceph-mon[35294]: pgmap v371: 97 pgs: 97 active+clean; 626 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.9 KiB/s wr, 23 op/s 2024-03-27T10:54:23.641 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:23 smithi097 ceph-mon[31859]: pgmap v371: 97 pgs: 97 active+clean; 626 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.9 KiB/s wr, 23 op/s 2024-03-27T10:54:24.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:24.889+0000 7f4757a00700 1 -- 172.21.15.97:0/2136417361 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4750075190 msgr2=0x7f475010eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:24.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:24.889+0000 7f4757a00700 1 --2- 172.21.15.97:0/2136417361 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4750075190 0x7f475010eb00 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f473c009990 tx=0x7f473c009ca0 comp rx=0 tx=0).stop 2024-03-27T10:54:24.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:24.889+0000 7f4757a00700 1 -- 172.21.15.97:0/2136417361 shutdown_connections 2024-03-27T10:54:24.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:24.890+0000 7f4757a00700 1 --2- 172.21.15.97:0/2136417361 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f475010f040 0x7f4750111430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:24.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:24.890+0000 7f4757a00700 1 --2- 172.21.15.97:0/2136417361 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4750074870 0x7f4750074c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:24.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:24.890+0000 7f4757a00700 1 --2- 172.21.15.97:0/2136417361 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4750075190 0x7f475010eb00 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:24.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:24.890+0000 7f4757a00700 1 -- 172.21.15.97:0/2136417361 >> 172.21.15.97:0/2136417361 conn(0x7f47500797a0 msgr2=0x7f4750079bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:54:24.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:24.890+0000 7f4757a00700 1 -- 172.21.15.97:0/2136417361 shutdown_connections 2024-03-27T10:54:24.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:24.890+0000 7f4757a00700 1 -- 172.21.15.97:0/2136417361 wait complete. 2024-03-27T10:54:24.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:24.891+0000 7f4757a00700 1 Processor -- start 2024-03-27T10:54:24.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:24.891+0000 7f4757a00700 1 -- start start 2024-03-27T10:54:24.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:24.892+0000 7f4757a00700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4750074870 0x7f475019af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:24.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:24.892+0000 7f4757a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4750075190 0x7f475019b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:24.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:24.892+0000 7f4757a00700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f475010f040 0x7f475019f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:24.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:24.892+0000 7f4757a00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4750114aa0 con 0x7f4750075190 2024-03-27T10:54:24.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:24.892+0000 7f4757a00700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f4750114920 con 0x7f4750074870 2024-03-27T10:54:24.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:24.892+0000 7f4757a00700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f4750114c20 con 0x7f475010f040 2024-03-27T10:54:24.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:24.892+0000 7f4747e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4750075190 0x7f475019b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:24.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:24.892+0000 7f4747e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4750075190 0x7f475019b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58574/0 (socket says 172.21.15.97:58574) 2024-03-27T10:54:24.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:24.892+0000 7f4747e00700 1 -- 172.21.15.97:0/144714168 learned_addr learned my addr 172.21.15.97:0/144714168 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:54:24.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:24.892+0000 7f4755000700 1 --2- 172.21.15.97:0/144714168 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4750074870 0x7f475019af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:24.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:24.892+0000 7f4755a00700 1 --2- 172.21.15.97:0/144714168 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f475010f040 0x7f475019f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:24.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:24.892+0000 7f4747e00700 1 -- 172.21.15.97:0/144714168 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f475010f040 msgr2=0x7f475019f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:24.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:24.892+0000 7f4747e00700 1 --2- 172.21.15.97:0/144714168 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f475010f040 0x7f475019f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:24.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:24.892+0000 7f4747e00700 1 -- 172.21.15.97:0/144714168 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4750074870 msgr2=0x7f475019af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:24.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:24.892+0000 7f4747e00700 1 --2- 172.21.15.97:0/144714168 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4750074870 0x7f475019af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:24.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:24.893+0000 7f4747e00700 1 -- 172.21.15.97:0/144714168 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f475019ff20 con 0x7f4750075190 2024-03-27T10:54:24.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:24.893+0000 7f4747e00700 1 --2- 172.21.15.97:0/144714168 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4750075190 0x7f475019b450 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f473c003ab0 tx=0x7f473c003ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:54:24.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:24.893+0000 7f474e000700 1 -- 172.21.15.97:0/144714168 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f473c027070 con 0x7f4750075190 2024-03-27T10:54:24.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:24.893+0000 7f474e000700 1 -- 172.21.15.97:0/144714168 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f473c00bdc0 con 0x7f4750075190 2024-03-27T10:54:24.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:24.893+0000 7f4757a00700 1 -- 172.21.15.97:0/144714168 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47501a01b0 con 0x7f4750075190 2024-03-27T10:54:24.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:24.893+0000 7f474e000700 1 -- 172.21.15.97:0/144714168 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f473c022ca0 con 0x7f4750075190 2024-03-27T10:54:24.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:24.893+0000 7f4757a00700 1 -- 172.21.15.97:0/144714168 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47501a7a90 con 0x7f4750075190 2024-03-27T10:54:24.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:24.894+0000 7f4757a00700 1 -- 172.21.15.97:0/144714168 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4750069300 con 0x7f4750075190 2024-03-27T10:54:24.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:24.896+0000 7f474e000700 1 -- 172.21.15.97:0/144714168 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138421+0+0 (secure 0 0 0) 0x7f473c02a070 con 0x7f4750075190 2024-03-27T10:54:24.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:24.898+0000 7f474e000700 1 --2- 172.21.15.97:0/144714168 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f47340a3730 0x7f47340a5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:24.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:24.898+0000 7f474e000700 1 -- 172.21.15.97:0/144714168 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 7979+0+0 (secure 0 0 0) 0x7f473c0d0ca0 con 0x7f4750075190 2024-03-27T10:54:24.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:24.898+0000 7f4755000700 1 --2- 172.21.15.97:0/144714168 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f47340a3730 0x7f47340a5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:24.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:24.899+0000 7f4755000700 1 --2- 172.21.15.97:0/144714168 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f47340a3730 0x7f47340a5bf0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4740005950 tx=0x7f474000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:54:24.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:24.902+0000 7f474e000700 1 -- 172.21.15.97:0/144714168 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f473c05ecb0 con 0x7f4750075190 2024-03-27T10:54:25.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:25.127+0000 7f4757a00700 1 -- 172.21.15.97:0/144714168 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f475002dd30 con 0x7f4750075190 2024-03-27T10:54:25.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:25.131+0000 7f474e000700 1 -- 172.21.15.97:0/144714168 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22211 (secure 0 0 0) 0x7f473c013070 con 0x7f4750075190 2024-03-27T10:54:25.104 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T10:54:25.105 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":90,"fsid":"adca88c4-ec26-11ee-95d0-87774f69a715","created":"2024-03-27T10:43:24.680694+0000","modified":"2024-03-27T10:54:06.066477+0000","last_up_change":"2024-03-27T10:53:00.552513+0000","last_in_change":"2024-03-27T10:52:36.302986+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-27T10:47:54.052810+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-27T10:54:02.845060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-27T10:54:03.474151+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b634e5f1-6118-4e2f-95e9-830db390f8d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1746812080},{"type":"v1","addr":"172.21.15.97:6803","nonce":1746812080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1746812080},{"type":"v1","addr":"172.21.15.97:6805","nonce":1746812080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":1746812080},{"type":"v1","addr":"172.21.15.97:6809","nonce":1746812080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1746812080},{"type":"v1","addr":"172.21.15.97:6807","nonce":1746812080}]},"public_addr":"172.21.15.97:6803/1746812080","cluster_addr":"172.21.15.97:6805/1746812080","heartbeat_back_addr":"172.21.15.97:6809/1746812080","heartbeat_front_addr":"172.21.15.97:6807/1746812080","state":["exists","up"]},{"osd":1,"uuid":"441e2770-3284-4388-b0de-c56eb8a373d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":3858821264},{"type":"v1","addr":"172.21.15.114:6801","nonce":3858821264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":3858821264},{"type":"v1","addr":"172.21.15.114:6803","nonce":3858821264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":3858821264},{"type":"v1","addr":"172.21.15.114:6807","nonce":3858821264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":3858821264},{"type":"v1","addr":"172.21.15.114:6805","nonce":3858821264}]},"public_addr":"172.21.15.114:6801/3858821264","cluster_addr":"172.21.15.114:6803/3858821264","heartbeat_back_addr":"172.21.15.114:6807/3858821264","heartbeat_front_addr":"172.21.15.114:6805/3858821264","state":["exists","up"]},{"osd":2,"uuid":"00108864-1aa6-48d6-8398-e8eb79fda123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":124240658},{"type":"v1","addr":"172.21.15.177:6801","nonce":124240658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":124240658},{"type":"v1","addr":"172.21.15.177:6803","nonce":124240658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":124240658},{"type":"v1","addr":"172.21.15.177:6807","nonce":124240658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":124240658},{"type":"v1","addr":"172.21.15.177:6805","nonce":124240658}]},"public_addr":"172.21.15.177:6801/124240658","cluster_addr":"172.21.15.177:6803/124240658","heartbeat_back_addr":"172.21.15.177:6807/124240658","heartbeat_front_addr":"172.21.15.177:6805/124240658","state":["exists","up"]},{"osd":3,"uuid":"1860b690-e2dc-4977-88d7-d1aacf7022ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2478179311},{"type":"v1","addr":"172.21.15.97:6811","nonce":2478179311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2478179311},{"type":"v1","addr":"172.21.15.97:6813","nonce":2478179311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":2478179311},{"type":"v1","addr":"172.21.15.97:6817","nonce":2478179311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2478179311},{"type":"v1","addr":"172.21.15.97:6815","nonce":2478179311}]},"public_addr":"172.21.15.97:6811/2478179311","cluster_addr":"172.21.15.97:6813/2478179311","heartbeat_back_addr":"172.21.15.97:6817/2478179311","heartbeat_front_addr":"172.21.15.97:6815/2478179311","state":["exists","up"]},{"osd":4,"uuid":"9245a3a4-7d49-4fdb-9d55-112e43177050","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":75755945},{"type":"v1","addr":"172.21.15.114:6809","nonce":75755945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":75755945},{"type":"v1","addr":"172.21.15.114:6811","nonce":75755945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":75755945},{"type":"v1","addr":"172.21.15.114:6815","nonce":75755945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":75755945},{"type":"v1","addr":"172.21.15.114:6813","nonce":75755945}]},"public_addr":"172.21.15.114:6809/75755945","cluster_addr":"172.21.15.114:6811/75755945","heartbeat_back_addr":"172.21.15.114:6815/75755945","heartbeat_front_addr":"172.21.15.114:6813/75755945","state":["exists","up"]},{"osd":5,"uuid":"51922f99-e8d3-461b-8615-e9bfdcbf5ed3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":1918493790},{"type":"v1","addr":"172.21.15.177:6809","nonce":1918493790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":1918493790},{"type":"v1","addr":"172.21.15.177:6811","nonce":1918493790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":1918493790},{"type":"v1","addr":"172.21.15.177:6815","nonce":1918493790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":1918493790},{"type":"v1","addr":"172.21.15.177:6813","nonce":1918493790}]},"public_addr":"172.21.15.177:6809/1918493790","cluster_addr":"172.21.15.177:6811/1918493790","heartbeat_back_addr":"172.21.15.177:6815/1918493790","heartbeat_front_addr":"172.21.15.177:6813/1918493790","state":["exists","up"]},{"osd":6,"uuid":"06dffb32-05eb-43b3-852e-915c5cc7f25a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2892389563},{"type":"v1","addr":"172.21.15.97:6819","nonce":2892389563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2892389563},{"type":"v1","addr":"172.21.15.97:6821","nonce":2892389563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2892389563},{"type":"v1","addr":"172.21.15.97:6825","nonce":2892389563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2892389563},{"type":"v1","addr":"172.21.15.97:6823","nonce":2892389563}]},"public_addr":"172.21.15.97:6819/2892389563","cluster_addr":"172.21.15.97:6821/2892389563","heartbeat_back_addr":"172.21.15.97:6825/2892389563","heartbeat_front_addr":"172.21.15.97:6823/2892389563","state":["exists","up"]},{"osd":7,"uuid":"e68e3519-9c1b-43c5-8e06-5385248b19df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":2445491153},{"type":"v1","addr":"172.21.15.114:6817","nonce":2445491153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":2445491153},{"type":"v1","addr":"172.21.15.114:6819","nonce":2445491153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2445491153},{"type":"v1","addr":"172.21.15.114:6823","nonce":2445491153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":2445491153},{"type":"v1","addr":"172.21.15.114:6821","nonce":2445491153}]},"public_addr":"172.21.15.114:6817/2445491153","cluster_addr":"172.21.15.114:6819/2445491153","heartbeat_back_addr":"172.21.15.114:6823/2445491153","heartbeat_front_addr":"172.21.15.114:6821/2445491153","state":["exists","up"]},{"osd":8,"uuid":"31ac592e-1349-49ee-9af9-8ad85d682978","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":2051134191},{"type":"v1","addr":"172.21.15.177:6817","nonce":2051134191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":2051134191},{"type":"v1","addr":"172.21.15.177:6819","nonce":2051134191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":2051134191},{"type":"v1","addr":"172.21.15.177:6823","nonce":2051134191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":2051134191},{"type":"v1","addr":"172.21.15.177:6821","nonce":2051134191}]},"public_addr":"172.21.15.177:6817/2051134191","cluster_addr":"172.21.15.177:6819/2051134191","heartbeat_back_addr":"172.21.15.177:6823/2051134191","heartbeat_front_addr":"172.21.15.177:6821/2051134191","state":["exists","up"]},{"osd":9,"uuid":"0171086a-dd41-4eb4-a389-16f0c46a64db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":1578881271},{"type":"v1","addr":"172.21.15.97:6827","nonce":1578881271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1578881271},{"type":"v1","addr":"172.21.15.97:6829","nonce":1578881271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1578881271},{"type":"v1","addr":"172.21.15.97:6833","nonce":1578881271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1578881271},{"type":"v1","addr":"172.21.15.97:6831","nonce":1578881271}]},"public_addr":"172.21.15.97:6827/1578881271","cluster_addr":"172.21.15.97:6829/1578881271","heartbeat_back_addr":"172.21.15.97:6833/1578881271","heartbeat_front_addr":"172.21.15.97:6831/1578881271","state":["exists","up"]},{"osd":10,"uuid":"64754e31-b124-4601-ad98-2f70b1dd5359","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":2893111461},{"type":"v1","addr":"172.21.15.114:6825","nonce":2893111461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":2893111461},{"type":"v1","addr":"172.21.15.114:6827","nonce":2893111461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2893111461},{"type":"v1","addr":"172.21.15.114:6831","nonce":2893111461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":2893111461},{"type":"v1","addr":"172.21.15.114:6829","nonce":2893111461}]},"public_addr":"172.21.15.114:6825/2893111461","cluster_addr":"172.21.15.114:6827/2893111461","heartbeat_back_addr":"172.21.15.114:6831/2893111461","heartbeat_front_addr":"172.21.15.114:6829/2893111461","state":["exists","up"]},{"osd":11,"uuid":"9ffc752c-adf7-4037-89d3-535787d7475d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":85,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":2333030391},{"type":"v1","addr":"172.21.15.177:6825","nonce":2333030391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":2333030391},{"type":"v1","addr":"172.21.15.177:6827","nonce":2333030391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":2333030391},{"type":"v1","addr":"172.21.15.177:6831","nonce":2333030391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":2333030391},{"type":"v1","addr":"172.21.15.177:6829","nonce":2333030391}]},"public_addr":"172.21.15.177:6825/2333030391","cluster_addr":"172.21.15.177:6827/2333030391","heartbeat_back_addr":"172.21.15.177:6831/2333030391","heartbeat_front_addr":"172.21.15.177:6829/2333030391","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:46:45.143519+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:47:18.656947+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:47:51.316345+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:48:24.575512+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:48:59.113555+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:49:32.103865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:50:06.615539+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:50:41.578341+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:51:15.254011+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:51:49.099165+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:52:24.544821+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:52:58.838484+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/359646":"2024-03-28T10:44:39.889550+0000","172.21.15.97:6801/98682155":"2024-03-28T10:44:06.019075+0000","172.21.15.97:6801/1685921853":"2024-03-28T10:44:39.889550+0000","172.21.15.97:6800/98682155":"2024-03-28T10:44:06.019075+0000","172.21.15.97:0/107690680":"2024-03-28T10:44:06.019075+0000","172.21.15.97:6800/1685921853":"2024-03-28T10:44:39.889550+0000","172.21.15.97:0/437043413":"2024-03-28T10:44:39.889550+0000","172.21.15.97:0/2319706992":"2024-03-28T10:44:39.889550+0000","172.21.15.97:0/3911532654":"2024-03-28T10:44:06.019075+0000","172.21.15.97:0/3347321845":"2024-03-28T10:44:06.019075+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-27T10:54:25.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:25.134+0000 7f4757a00700 1 -- 172.21.15.97:0/144714168 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f47340a3730 msgr2=0x7f47340a5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:25.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:25.134+0000 7f4757a00700 1 --2- 172.21.15.97:0/144714168 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f47340a3730 0x7f47340a5bf0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4740005950 tx=0x7f474000b410 comp rx=0 tx=0).stop 2024-03-27T10:54:25.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:25.134+0000 7f4757a00700 1 -- 172.21.15.97:0/144714168 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4750075190 msgr2=0x7f475019b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:25.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:25.134+0000 7f4757a00700 1 --2- 172.21.15.97:0/144714168 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4750075190 0x7f475019b450 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f473c003ab0 tx=0x7f473c003ae0 comp rx=0 tx=0).stop 2024-03-27T10:54:25.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:25.134+0000 7f4757a00700 1 -- 172.21.15.97:0/144714168 shutdown_connections 2024-03-27T10:54:25.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:25.134+0000 7f4757a00700 1 --2- 172.21.15.97:0/144714168 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f47340a3730 0x7f47340a5bf0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:25.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:25.134+0000 7f4757a00700 1 --2- 172.21.15.97:0/144714168 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f475010f040 0x7f475019f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:25.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:25.134+0000 7f4757a00700 1 --2- 172.21.15.97:0/144714168 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4750074870 0x7f475019af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:25.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:25.134+0000 7f4757a00700 1 --2- 172.21.15.97:0/144714168 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4750075190 0x7f475019b450 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:25.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:25.134+0000 7f4757a00700 1 -- 172.21.15.97:0/144714168 >> 172.21.15.97:0/144714168 conn(0x7f47500797a0 msgr2=0x7f47500ff8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:54:25.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:25.134+0000 7f4757a00700 1 -- 172.21.15.97:0/144714168 shutdown_connections 2024-03-27T10:54:25.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:25.135+0000 7f4757a00700 1 -- 172.21.15.97:0/144714168 wait complete. 2024-03-27T10:54:25.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:25 smithi097 ceph-mon[31859]: pgmap v372: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.2 KiB/s wr, 27 op/s 2024-03-27T10:54:25.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:25 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/144714168' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T10:54:25.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:25 smithi114 ceph-mon[35338]: pgmap v372: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.2 KiB/s wr, 27 op/s 2024-03-27T10:54:25.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:25 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/144714168' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T10:54:25.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:25 smithi177 ceph-mon[35294]: pgmap v372: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.2 KiB/s wr, 27 op/s 2024-03-27T10:54:25.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:25 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/144714168' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T10:54:25.902 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-03-27T10:54:25.903 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell --fsid adca88c4-ec26-11ee-95d0-87774f69a715 -- ceph osd dump --format=json 2024-03-27T10:54:26.353 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T10:54:27.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:27 smithi114 ceph-mon[35338]: pgmap v373: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 3.4 KiB/s wr, 13 op/s 2024-03-27T10:54:27.578 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:27 smithi097 ceph-mon[31859]: pgmap v373: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 3.4 KiB/s wr, 13 op/s 2024-03-27T10:54:27.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:27 smithi177 ceph-mon[35294]: pgmap v373: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 3.4 KiB/s wr, 13 op/s 2024-03-27T10:54:28.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:28.487+0000 7f0b96e00700 1 -- 172.21.15.97:0/953817530 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b90074870 msgr2=0x7f0b90074c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:28.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:28.487+0000 7f0b96e00700 1 --2- 172.21.15.97:0/953817530 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b90074870 0x7f0b90074c50 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0b80009990 tx=0x7f0b80009ca0 comp rx=0 tx=0).stop 2024-03-27T10:54:28.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:28.487+0000 7f0b96e00700 1 -- 172.21.15.97:0/953817530 shutdown_connections 2024-03-27T10:54:28.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:28.487+0000 7f0b96e00700 1 --2- 172.21.15.97:0/953817530 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0b9010f040 0x7f0b90111430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:28.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:28.487+0000 7f0b96e00700 1 --2- 172.21.15.97:0/953817530 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b90074870 0x7f0b90074c50 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:28.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:28.487+0000 7f0b96e00700 1 --2- 172.21.15.97:0/953817530 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b90075190 0x7f0b9010eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:28.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:28.487+0000 7f0b96e00700 1 -- 172.21.15.97:0/953817530 >> 172.21.15.97:0/953817530 conn(0x7f0b90079770 msgr2=0x7f0b90079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:54:28.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:28.487+0000 7f0b96e00700 1 -- 172.21.15.97:0/953817530 shutdown_connections 2024-03-27T10:54:28.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:28.487+0000 7f0b96e00700 1 -- 172.21.15.97:0/953817530 wait complete. 2024-03-27T10:54:28.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:28.488+0000 7f0b96e00700 1 Processor -- start 2024-03-27T10:54:28.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:28.488+0000 7f0b96e00700 1 -- start start 2024-03-27T10:54:28.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:28.489+0000 7f0b96e00700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b90074870 0x7f0b9019aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:28.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:28.489+0000 7f0b96e00700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0b90075190 0x7f0b9019b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:28.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:28.489+0000 7f0b96e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b9010f040 0x7f0b9019f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:28.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:28.489+0000 7f0b96e00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0b901149e0 con 0x7f0b9010f040 2024-03-27T10:54:28.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:28.489+0000 7f0b96e00700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f0b90114860 con 0x7f0b90074870 2024-03-27T10:54:28.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:28.489+0000 7f0b96e00700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f0b90114b60 con 0x7f0b90075190 2024-03-27T10:54:28.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:28.489+0000 7f0b8fe00700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b90074870 0x7f0b9019aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:28.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:28.489+0000 7f0b94e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b9010f040 0x7f0b9019f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:28.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:28.489+0000 7f0b8fe00700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b90074870 0x7f0b9019aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:53568/0 (socket says 172.21.15.97:53568) 2024-03-27T10:54:28.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:28.489+0000 7f0b8fe00700 1 -- 172.21.15.97:0/1201968350 learned_addr learned my addr 172.21.15.97:0/1201968350 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:54:28.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:28.489+0000 7f0b8f400700 1 --2- 172.21.15.97:0/1201968350 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0b90075190 0x7f0b9019b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:28.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:28.490+0000 7f0b94e00700 1 -- 172.21.15.97:0/1201968350 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0b90075190 msgr2=0x7f0b9019b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:28.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:28.490+0000 7f0b94e00700 1 --2- 172.21.15.97:0/1201968350 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0b90075190 0x7f0b9019b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:28.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:28.490+0000 7f0b94e00700 1 -- 172.21.15.97:0/1201968350 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b90074870 msgr2=0x7f0b9019aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:28.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:28.490+0000 7f0b94e00700 1 --2- 172.21.15.97:0/1201968350 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b90074870 0x7f0b9019aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:28.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:28.490+0000 7f0b94e00700 1 -- 172.21.15.97:0/1201968350 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b9019ff00 con 0x7f0b9010f040 2024-03-27T10:54:28.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:28.490+0000 7f0b94e00700 1 --2- 172.21.15.97:0/1201968350 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b9010f040 0x7f0b9019f7c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0b88007f10 tx=0x7f0b8800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:54:28.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:28.490+0000 7f0b8cc00700 1 -- 172.21.15.97:0/1201968350 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b88004db0 con 0x7f0b9010f040 2024-03-27T10:54:28.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:28.490+0000 7f0b8cc00700 1 -- 172.21.15.97:0/1201968350 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0b88009e20 con 0x7f0b9010f040 2024-03-27T10:54:28.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:28.490+0000 7f0b96e00700 1 -- 172.21.15.97:0/1201968350 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b901a01f0 con 0x7f0b9010f040 2024-03-27T10:54:28.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:28.490+0000 7f0b8cc00700 1 -- 172.21.15.97:0/1201968350 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b88015400 con 0x7f0b9010f040 2024-03-27T10:54:28.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:28.490+0000 7f0b96e00700 1 -- 172.21.15.97:0/1201968350 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b901a7a70 con 0x7f0b9010f040 2024-03-27T10:54:28.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:28.491+0000 7f0b96e00700 1 -- 172.21.15.97:0/1201968350 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b90069300 con 0x7f0b9010f040 2024-03-27T10:54:28.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:28.493+0000 7f0b8cc00700 1 -- 172.21.15.97:0/1201968350 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138421+0+0 (secure 0 0 0) 0x7f0b8800db20 con 0x7f0b9010f040 2024-03-27T10:54:28.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:28.495+0000 7f0b8cc00700 1 --2- 172.21.15.97:0/1201968350 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f0b740a3730 0x7f0b740a5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:28.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:28.495+0000 7f0b8cc00700 1 -- 172.21.15.97:0/1201968350 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 7979+0+0 (secure 0 0 0) 0x7f0b880cf740 con 0x7f0b9010f040 2024-03-27T10:54:28.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:28.495+0000 7f0b8fe00700 1 --2- 172.21.15.97:0/1201968350 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f0b740a3730 0x7f0b740a5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:28.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:28.496+0000 7f0b8fe00700 1 --2- 172.21.15.97:0/1201968350 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f0b740a3730 0x7f0b740a5bf0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f0b80006010 tx=0x7f0b8000b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:54:28.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:28.500+0000 7f0b8cc00700 1 -- 172.21.15.97:0/1201968350 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f0b88010070 con 0x7f0b9010f040 2024-03-27T10:54:28.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:28.725+0000 7f0b96e00700 1 -- 172.21.15.97:0/1201968350 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0b9002dd30 con 0x7f0b9010f040 2024-03-27T10:54:28.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:28.727+0000 7f0b8cc00700 1 -- 172.21.15.97:0/1201968350 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22211 (secure 0 0 0) 0x7f0b8809c260 con 0x7f0b9010f040 2024-03-27T10:54:28.699 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T10:54:28.699 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":90,"fsid":"adca88c4-ec26-11ee-95d0-87774f69a715","created":"2024-03-27T10:43:24.680694+0000","modified":"2024-03-27T10:54:06.066477+0000","last_up_change":"2024-03-27T10:53:00.552513+0000","last_in_change":"2024-03-27T10:52:36.302986+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-27T10:47:54.052810+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-27T10:54:02.845060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-27T10:54:03.474151+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b634e5f1-6118-4e2f-95e9-830db390f8d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1746812080},{"type":"v1","addr":"172.21.15.97:6803","nonce":1746812080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1746812080},{"type":"v1","addr":"172.21.15.97:6805","nonce":1746812080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":1746812080},{"type":"v1","addr":"172.21.15.97:6809","nonce":1746812080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1746812080},{"type":"v1","addr":"172.21.15.97:6807","nonce":1746812080}]},"public_addr":"172.21.15.97:6803/1746812080","cluster_addr":"172.21.15.97:6805/1746812080","heartbeat_back_addr":"172.21.15.97:6809/1746812080","heartbeat_front_addr":"172.21.15.97:6807/1746812080","state":["exists","up"]},{"osd":1,"uuid":"441e2770-3284-4388-b0de-c56eb8a373d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":3858821264},{"type":"v1","addr":"172.21.15.114:6801","nonce":3858821264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":3858821264},{"type":"v1","addr":"172.21.15.114:6803","nonce":3858821264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":3858821264},{"type":"v1","addr":"172.21.15.114:6807","nonce":3858821264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":3858821264},{"type":"v1","addr":"172.21.15.114:6805","nonce":3858821264}]},"public_addr":"172.21.15.114:6801/3858821264","cluster_addr":"172.21.15.114:6803/3858821264","heartbeat_back_addr":"172.21.15.114:6807/3858821264","heartbeat_front_addr":"172.21.15.114:6805/3858821264","state":["exists","up"]},{"osd":2,"uuid":"00108864-1aa6-48d6-8398-e8eb79fda123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":124240658},{"type":"v1","addr":"172.21.15.177:6801","nonce":124240658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":124240658},{"type":"v1","addr":"172.21.15.177:6803","nonce":124240658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":124240658},{"type":"v1","addr":"172.21.15.177:6807","nonce":124240658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":124240658},{"type":"v1","addr":"172.21.15.177:6805","nonce":124240658}]},"public_addr":"172.21.15.177:6801/124240658","cluster_addr":"172.21.15.177:6803/124240658","heartbeat_back_addr":"172.21.15.177:6807/124240658","heartbeat_front_addr":"172.21.15.177:6805/124240658","state":["exists","up"]},{"osd":3,"uuid":"1860b690-e2dc-4977-88d7-d1aacf7022ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2478179311},{"type":"v1","addr":"172.21.15.97:6811","nonce":2478179311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2478179311},{"type":"v1","addr":"172.21.15.97:6813","nonce":2478179311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":2478179311},{"type":"v1","addr":"172.21.15.97:6817","nonce":2478179311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2478179311},{"type":"v1","addr":"172.21.15.97:6815","nonce":2478179311}]},"public_addr":"172.21.15.97:6811/2478179311","cluster_addr":"172.21.15.97:6813/2478179311","heartbeat_back_addr":"172.21.15.97:6817/2478179311","heartbeat_front_addr":"172.21.15.97:6815/2478179311","state":["exists","up"]},{"osd":4,"uuid":"9245a3a4-7d49-4fdb-9d55-112e43177050","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":75755945},{"type":"v1","addr":"172.21.15.114:6809","nonce":75755945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":75755945},{"type":"v1","addr":"172.21.15.114:6811","nonce":75755945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":75755945},{"type":"v1","addr":"172.21.15.114:6815","nonce":75755945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":75755945},{"type":"v1","addr":"172.21.15.114:6813","nonce":75755945}]},"public_addr":"172.21.15.114:6809/75755945","cluster_addr":"172.21.15.114:6811/75755945","heartbeat_back_addr":"172.21.15.114:6815/75755945","heartbeat_front_addr":"172.21.15.114:6813/75755945","state":["exists","up"]},{"osd":5,"uuid":"51922f99-e8d3-461b-8615-e9bfdcbf5ed3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":1918493790},{"type":"v1","addr":"172.21.15.177:6809","nonce":1918493790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":1918493790},{"type":"v1","addr":"172.21.15.177:6811","nonce":1918493790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":1918493790},{"type":"v1","addr":"172.21.15.177:6815","nonce":1918493790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":1918493790},{"type":"v1","addr":"172.21.15.177:6813","nonce":1918493790}]},"public_addr":"172.21.15.177:6809/1918493790","cluster_addr":"172.21.15.177:6811/1918493790","heartbeat_back_addr":"172.21.15.177:6815/1918493790","heartbeat_front_addr":"172.21.15.177:6813/1918493790","state":["exists","up"]},{"osd":6,"uuid":"06dffb32-05eb-43b3-852e-915c5cc7f25a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2892389563},{"type":"v1","addr":"172.21.15.97:6819","nonce":2892389563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2892389563},{"type":"v1","addr":"172.21.15.97:6821","nonce":2892389563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2892389563},{"type":"v1","addr":"172.21.15.97:6825","nonce":2892389563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2892389563},{"type":"v1","addr":"172.21.15.97:6823","nonce":2892389563}]},"public_addr":"172.21.15.97:6819/2892389563","cluster_addr":"172.21.15.97:6821/2892389563","heartbeat_back_addr":"172.21.15.97:6825/2892389563","heartbeat_front_addr":"172.21.15.97:6823/2892389563","state":["exists","up"]},{"osd":7,"uuid":"e68e3519-9c1b-43c5-8e06-5385248b19df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":2445491153},{"type":"v1","addr":"172.21.15.114:6817","nonce":2445491153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":2445491153},{"type":"v1","addr":"172.21.15.114:6819","nonce":2445491153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2445491153},{"type":"v1","addr":"172.21.15.114:6823","nonce":2445491153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":2445491153},{"type":"v1","addr":"172.21.15.114:6821","nonce":2445491153}]},"public_addr":"172.21.15.114:6817/2445491153","cluster_addr":"172.21.15.114:6819/2445491153","heartbeat_back_addr":"172.21.15.114:6823/2445491153","heartbeat_front_addr":"172.21.15.114:6821/2445491153","state":["exists","up"]},{"osd":8,"uuid":"31ac592e-1349-49ee-9af9-8ad85d682978","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":2051134191},{"type":"v1","addr":"172.21.15.177:6817","nonce":2051134191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":2051134191},{"type":"v1","addr":"172.21.15.177:6819","nonce":2051134191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":2051134191},{"type":"v1","addr":"172.21.15.177:6823","nonce":2051134191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":2051134191},{"type":"v1","addr":"172.21.15.177:6821","nonce":2051134191}]},"public_addr":"172.21.15.177:6817/2051134191","cluster_addr":"172.21.15.177:6819/2051134191","heartbeat_back_addr":"172.21.15.177:6823/2051134191","heartbeat_front_addr":"172.21.15.177:6821/2051134191","state":["exists","up"]},{"osd":9,"uuid":"0171086a-dd41-4eb4-a389-16f0c46a64db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":1578881271},{"type":"v1","addr":"172.21.15.97:6827","nonce":1578881271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1578881271},{"type":"v1","addr":"172.21.15.97:6829","nonce":1578881271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1578881271},{"type":"v1","addr":"172.21.15.97:6833","nonce":1578881271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1578881271},{"type":"v1","addr":"172.21.15.97:6831","nonce":1578881271}]},"public_addr":"172.21.15.97:6827/1578881271","cluster_addr":"172.21.15.97:6829/1578881271","heartbeat_back_addr":"172.21.15.97:6833/1578881271","heartbeat_front_addr":"172.21.15.97:6831/1578881271","state":["exists","up"]},{"osd":10,"uuid":"64754e31-b124-4601-ad98-2f70b1dd5359","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":2893111461},{"type":"v1","addr":"172.21.15.114:6825","nonce":2893111461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":2893111461},{"type":"v1","addr":"172.21.15.114:6827","nonce":2893111461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2893111461},{"type":"v1","addr":"172.21.15.114:6831","nonce":2893111461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":2893111461},{"type":"v1","addr":"172.21.15.114:6829","nonce":2893111461}]},"public_addr":"172.21.15.114:6825/2893111461","cluster_addr":"172.21.15.114:6827/2893111461","heartbeat_back_addr":"172.21.15.114:6831/2893111461","heartbeat_front_addr":"172.21.15.114:6829/2893111461","state":["exists","up"]},{"osd":11,"uuid":"9ffc752c-adf7-4037-89d3-535787d7475d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":85,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":2333030391},{"type":"v1","addr":"172.21.15.177:6825","nonce":2333030391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":2333030391},{"type":"v1","addr":"172.21.15.177:6827","nonce":2333030391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":2333030391},{"type":"v1","addr":"172.21.15.177:6831","nonce":2333030391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":2333030391},{"type":"v1","addr":"172.21.15.177:6829","nonce":2333030391}]},"public_addr":"172.21.15.177:6825/2333030391","cluster_addr":"172.21.15.177:6827/2333030391","heartbeat_back_addr":"172.21.15.177:6831/2333030391","heartbeat_front_addr":"172.21.15.177:6829/2333030391","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:46:45.143519+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:47:18.656947+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:47:51.316345+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:48:24.575512+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:48:59.113555+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:49:32.103865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:50:06.615539+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:50:41.578341+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:51:15.254011+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:51:49.099165+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:52:24.544821+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:52:58.838484+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/359646":"2024-03-28T10:44:39.889550+0000","172.21.15.97:6801/98682155":"2024-03-28T10:44:06.019075+0000","172.21.15.97:6801/1685921853":"2024-03-28T10:44:39.889550+0000","172.21.15.97:6800/98682155":"2024-03-28T10:44:06.019075+0000","172.21.15.97:0/107690680":"2024-03-28T10:44:06.019075+0000","172.21.15.97:6800/1685921853":"2024-03-28T10:44:39.889550+0000","172.21.15.97:0/437043413":"2024-03-28T10:44:39.889550+0000","172.21.15.97:0/2319706992":"2024-03-28T10:44:39.889550+0000","172.21.15.97:0/3911532654":"2024-03-28T10:44:06.019075+0000","172.21.15.97:0/3347321845":"2024-03-28T10:44:06.019075+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-27T10:54:28.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:28.730+0000 7f0b96e00700 1 -- 172.21.15.97:0/1201968350 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f0b740a3730 msgr2=0x7f0b740a5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:28.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:28.730+0000 7f0b96e00700 1 --2- 172.21.15.97:0/1201968350 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f0b740a3730 0x7f0b740a5bf0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f0b80006010 tx=0x7f0b8000b540 comp rx=0 tx=0).stop 2024-03-27T10:54:28.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:28.730+0000 7f0b96e00700 1 -- 172.21.15.97:0/1201968350 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b9010f040 msgr2=0x7f0b9019f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:28.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:28.730+0000 7f0b96e00700 1 --2- 172.21.15.97:0/1201968350 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b9010f040 0x7f0b9019f7c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0b88007f10 tx=0x7f0b8800d3b0 comp rx=0 tx=0).stop 2024-03-27T10:54:28.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:28.730+0000 7f0b96e00700 1 -- 172.21.15.97:0/1201968350 shutdown_connections 2024-03-27T10:54:28.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:28.730+0000 7f0b96e00700 1 --2- 172.21.15.97:0/1201968350 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f0b740a3730 0x7f0b740a5bf0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:28.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:28.730+0000 7f0b96e00700 1 --2- 172.21.15.97:0/1201968350 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b9010f040 0x7f0b9019f7c0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:28.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:28.730+0000 7f0b96e00700 1 --2- 172.21.15.97:0/1201968350 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b90074870 0x7f0b9019aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:28.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:28.730+0000 7f0b96e00700 1 --2- 172.21.15.97:0/1201968350 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0b90075190 0x7f0b9019b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:28.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:28.730+0000 7f0b96e00700 1 -- 172.21.15.97:0/1201968350 >> 172.21.15.97:0/1201968350 conn(0x7f0b90079770 msgr2=0x7f0b900ff850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:54:28.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:28.730+0000 7f0b96e00700 1 -- 172.21.15.97:0/1201968350 shutdown_connections 2024-03-27T10:54:28.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:28.730+0000 7f0b96e00700 1 -- 172.21.15.97:0/1201968350 wait complete. 2024-03-27T10:54:29.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:29 smithi097 ceph-mon[31859]: pgmap v374: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.8 KiB/s wr, 10 op/s 2024-03-27T10:54:29.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:29 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/1201968350' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T10:54:29.489 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell --fsid adca88c4-ec26-11ee-95d0-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-03-27T10:54:29.489 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell --fsid adca88c4-ec26-11ee-95d0-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-03-27T10:54:29.489 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell --fsid adca88c4-ec26-11ee-95d0-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-03-27T10:54:29.490 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell --fsid adca88c4-ec26-11ee-95d0-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2024-03-27T10:54:29.490 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell --fsid adca88c4-ec26-11ee-95d0-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2024-03-27T10:54:29.491 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell --fsid adca88c4-ec26-11ee-95d0-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2024-03-27T10:54:29.491 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell --fsid adca88c4-ec26-11ee-95d0-87774f69a715 -- ceph tell osd.6 flush_pg_stats 2024-03-27T10:54:29.491 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell --fsid adca88c4-ec26-11ee-95d0-87774f69a715 -- ceph tell osd.7 flush_pg_stats 2024-03-27T10:54:29.492 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell --fsid adca88c4-ec26-11ee-95d0-87774f69a715 -- ceph tell osd.8 flush_pg_stats 2024-03-27T10:54:29.492 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell --fsid adca88c4-ec26-11ee-95d0-87774f69a715 -- ceph tell osd.9 flush_pg_stats 2024-03-27T10:54:29.493 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell --fsid adca88c4-ec26-11ee-95d0-87774f69a715 -- ceph tell osd.10 flush_pg_stats 2024-03-27T10:54:29.493 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell --fsid adca88c4-ec26-11ee-95d0-87774f69a715 -- ceph tell osd.11 flush_pg_stats 2024-03-27T10:54:29.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:29 smithi114 ceph-mon[35338]: pgmap v374: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.8 KiB/s wr, 10 op/s 2024-03-27T10:54:29.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:29 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/1201968350' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T10:54:29.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:29 smithi177 ceph-mon[35294]: pgmap v374: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.8 KiB/s wr, 10 op/s 2024-03-27T10:54:29.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:29 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/1201968350' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T10:54:30.429 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T10:54:30.830 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T10:54:30.831 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T10:54:30.852 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T10:54:30.858 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T10:54:30.873 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T10:54:30.880 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T10:54:30.882 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T10:54:31.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:31 smithi097 ceph-mon[31859]: pgmap v375: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-03-27T10:54:31.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:31 smithi114 ceph-mon[35338]: pgmap v375: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-03-27T10:54:31.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:31 smithi177 ceph-mon[35294]: pgmap v375: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-03-27T10:54:33.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:33 smithi114 ceph-mon[35338]: pgmap v376: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-03-27T10:54:33.641 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:33 smithi097 ceph-mon[31859]: pgmap v376: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-03-27T10:54:33.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:33 smithi177 ceph-mon[35294]: pgmap v376: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-03-27T10:54:35.448 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:35 smithi097 ceph-mon[31859]: pgmap v377: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-03-27T10:54:35.482 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T10:54:35.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:35 smithi114 ceph-mon[35338]: pgmap v377: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-03-27T10:54:35.592 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:35 smithi177 ceph-mon[35294]: pgmap v377: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-03-27T10:54:36.113 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T10:54:36.114 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T10:54:36.118 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T10:54:37.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:37 smithi114 ceph-mon[35338]: pgmap v378: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 9 op/s 2024-03-27T10:54:37.605 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:37 smithi097 ceph-mon[31859]: pgmap v378: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 9 op/s 2024-03-27T10:54:37.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:37 smithi177 ceph-mon[35294]: pgmap v378: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 9 op/s 2024-03-27T10:54:39.385 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:39 smithi097 ceph-mon[31859]: pgmap v379: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 9 op/s 2024-03-27T10:54:39.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:39.403+0000 7fdab3000700 1 -- 172.21.15.97:0/1362520246 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdaac102690 msgr2=0x7fdaac102b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:39.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:39.404+0000 7fdab3000700 1 --2- 172.21.15.97:0/1362520246 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdaac102690 0x7fdaac102b10 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fda94009990 tx=0x7fda94009ca0 comp rx=0 tx=0).stop 2024-03-27T10:54:39.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:39.404+0000 7fdab3000700 1 -- 172.21.15.97:0/1362520246 shutdown_connections 2024-03-27T10:54:39.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:39.404+0000 7fdab3000700 1 --2- 172.21.15.97:0/1362520246 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdaac103050 0x7fdaac10f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:39.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:39.404+0000 7fdab3000700 1 --2- 172.21.15.97:0/1362520246 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdaac108680 0x7fdaac108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:39.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:39.404+0000 7fdab3000700 1 --2- 172.21.15.97:0/1362520246 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdaac102690 0x7fdaac102b10 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:39.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:39.404+0000 7fdab3000700 1 -- 172.21.15.97:0/1362520246 >> 172.21.15.97:0/1362520246 conn(0x7fdaac0797a0 msgr2=0x7fdaac079bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:54:39.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:39.404+0000 7fdab3000700 1 -- 172.21.15.97:0/1362520246 shutdown_connections 2024-03-27T10:54:39.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:39.404+0000 7fdab3000700 1 -- 172.21.15.97:0/1362520246 wait complete. 2024-03-27T10:54:39.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:39.405+0000 7fdab3000700 1 Processor -- start 2024-03-27T10:54:39.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:39.405+0000 7fdab3000700 1 -- start start 2024-03-27T10:54:39.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:39.405+0000 7fdab3000700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdaac102690 0x7fdaac19af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:39.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:39.405+0000 7fdab3000700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdaac103050 0x7fdaac19b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:39.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:39.405+0000 7fdab3000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdaac108680 0x7fdaac19f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:39.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:39.405+0000 7fdab3000700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdaac1128b0 con 0x7fdaac108680 2024-03-27T10:54:39.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:39.405+0000 7fdab3000700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fdaac112730 con 0x7fdaac102690 2024-03-27T10:54:39.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:39.405+0000 7fdab3000700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fdaac112a30 con 0x7fdaac103050 2024-03-27T10:54:39.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:39.406+0000 7fdab1000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdaac108680 0x7fdaac19f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:39.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:39.406+0000 7fdab1000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdaac108680 0x7fdaac19f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45378/0 (socket says 172.21.15.97:45378) 2024-03-27T10:54:39.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:39.406+0000 7fdab1000700 1 -- 172.21.15.97:0/2596887492 learned_addr learned my addr 172.21.15.97:0/2596887492 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:54:39.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:39.406+0000 7fdaabe00700 1 --2- 172.21.15.97:0/2596887492 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdaac102690 0x7fdaac19af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:39.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:39.406+0000 7fdaa3e00700 1 --2- 172.21.15.97:0/2596887492 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdaac103050 0x7fdaac19b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:39.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:39.406+0000 7fdab1000700 1 -- 172.21.15.97:0/2596887492 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdaac103050 msgr2=0x7fdaac19b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:39.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:39.406+0000 7fdab1000700 1 --2- 172.21.15.97:0/2596887492 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdaac103050 0x7fdaac19b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:39.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:39.406+0000 7fdab1000700 1 -- 172.21.15.97:0/2596887492 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdaac102690 msgr2=0x7fdaac19af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:39.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:39.406+0000 7fdab1000700 1 --2- 172.21.15.97:0/2596887492 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdaac102690 0x7fdaac19af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:39.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:39.406+0000 7fdab1000700 1 -- 172.21.15.97:0/2596887492 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaac19ff50 con 0x7fdaac108680 2024-03-27T10:54:39.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:39.406+0000 7fdab1000700 1 --2- 172.21.15.97:0/2596887492 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdaac108680 0x7fdaac19f810 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fdaa400cca0 tx=0x7fdaa400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:54:39.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:39.407+0000 7fdaa9600700 1 -- 172.21.15.97:0/2596887492 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa4016020 con 0x7fdaac108680 2024-03-27T10:54:39.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:39.407+0000 7fdaa9600700 1 -- 172.21.15.97:0/2596887492 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdaa400aba0 con 0x7fdaac108680 2024-03-27T10:54:39.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:39.407+0000 7fdaa9600700 1 -- 172.21.15.97:0/2596887492 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa40115e0 con 0x7fdaac108680 2024-03-27T10:54:39.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:39.407+0000 7fdab3000700 1 -- 172.21.15.97:0/2596887492 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaac1a0240 con 0x7fdaac108680 2024-03-27T10:54:39.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:39.407+0000 7fdab3000700 1 -- 172.21.15.97:0/2596887492 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaac1a7ac0 con 0x7fdaac108680 2024-03-27T10:54:39.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:39.408+0000 7fdab3000700 1 -- 172.21.15.97:0/2596887492 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdaac10e2f0 con 0x7fdaac108680 2024-03-27T10:54:39.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:39.410+0000 7fdaa9600700 1 -- 172.21.15.97:0/2596887492 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138421+0+0 (secure 0 0 0) 0x7fdaa4019020 con 0x7fdaac108680 2024-03-27T10:54:39.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:39.411+0000 7fdaa9600700 1 --2- 172.21.15.97:0/2596887492 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fda900a3830 0x7fda900a5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:39.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:39.411+0000 7fdaa9600700 1 -- 172.21.15.97:0/2596887492 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 7979+0+0 (secure 0 0 0) 0x7fdaa40d0030 con 0x7fdaac108680 2024-03-27T10:54:39.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:39.411+0000 7fdaabe00700 1 --2- 172.21.15.97:0/2596887492 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fda900a3830 0x7fda900a5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:39.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:39.412+0000 7fdaa9600700 1 --2- 172.21.15.97:0/2596887492 >> [v2:172.21.15.114:6808/75755945,v1:172.21.15.114:6809/75755945] conn(0x7fda900ad010 0x7fda900af430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:39.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:39.412+0000 7fdaa9600700 1 -- 172.21.15.97:0/2596887492 --> [v2:172.21.15.114:6808/75755945,v1:172.21.15.114:6809/75755945] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdaa401eeb0 con 0x7fda900ad010 2024-03-27T10:54:39.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:39.412+0000 7fdaa9600700 1 -- 172.21.15.97:0/2596887492 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fdaa40d0420 con 0x7fdaac108680 2024-03-27T10:54:39.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:39.412+0000 7fdaabe00700 1 --2- 172.21.15.97:0/2596887492 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fda900a3830 0x7fda900a5cf0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fda9c005950 tx=0x7fda9c00a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:54:39.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:39.412+0000 7fdaa3e00700 1 --2- 172.21.15.97:0/2596887492 >> [v2:172.21.15.114:6808/75755945,v1:172.21.15.114:6809/75755945] conn(0x7fda900ad010 0x7fda900af430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:39.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:39.413+0000 7fdaa3e00700 1 --2- 172.21.15.97:0/2596887492 >> [v2:172.21.15.114:6808/75755945,v1:172.21.15.114:6809/75755945] conn(0x7fda900ad010 0x7fda900af430 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:54:39.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:39.415+0000 7fdaa9600700 1 -- 172.21.15.97:0/2596887492 <== osd.4 v2:172.21.15.114:6808/75755945 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdaa401eeb0 con 0x7fda900ad010 2024-03-27T10:54:39.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:39.434+0000 7fdab3000700 1 -- 172.21.15.97:0/2596887492 --> [v2:172.21.15.114:6808/75755945,v1:172.21.15.114:6809/75755945] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fdaac103dc0 con 0x7fda900ad010 2024-03-27T10:54:39.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:39.434+0000 7fdaa9600700 1 -- 172.21.15.97:0/2596887492 <== osd.4 v2:172.21.15.114:6808/75755945 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fdaac103dc0 con 0x7fda900ad010 2024-03-27T10:54:39.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:39.435+0000 7fdab3000700 1 -- 172.21.15.97:0/2596887492 >> [v2:172.21.15.114:6808/75755945,v1:172.21.15.114:6809/75755945] conn(0x7fda900ad010 msgr2=0x7fda900af430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:39.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:39.435+0000 7fdab3000700 1 --2- 172.21.15.97:0/2596887492 >> [v2:172.21.15.114:6808/75755945,v1:172.21.15.114:6809/75755945] conn(0x7fda900ad010 0x7fda900af430 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:39.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:39.435+0000 7fdab3000700 1 -- 172.21.15.97:0/2596887492 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fda900a3830 msgr2=0x7fda900a5cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:39.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:39.435+0000 7fdab3000700 1 --2- 172.21.15.97:0/2596887492 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fda900a3830 0x7fda900a5cf0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fda9c005950 tx=0x7fda9c00a300 comp rx=0 tx=0).stop 2024-03-27T10:54:39.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:39.435+0000 7fdab3000700 1 -- 172.21.15.97:0/2596887492 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdaac108680 msgr2=0x7fdaac19f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:39.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:39.435+0000 7fdab3000700 1 --2- 172.21.15.97:0/2596887492 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdaac108680 0x7fdaac19f810 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fdaa400cca0 tx=0x7fdaa400a060 comp rx=0 tx=0).stop 2024-03-27T10:54:39.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:39.435+0000 7fdab1000700 1 -- 172.21.15.97:0/2596887492 reap_dead start 2024-03-27T10:54:39.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:39.435+0000 7fdab3000700 1 -- 172.21.15.97:0/2596887492 shutdown_connections 2024-03-27T10:54:39.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:39.435+0000 7fdab3000700 1 -- 172.21.15.97:0/2596887492 >> 172.21.15.97:0/2596887492 conn(0x7fdaac0797a0 msgr2=0x7fdaac10d910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:54:39.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:39.435+0000 7fdab3000700 1 -- 172.21.15.97:0/2596887492 shutdown_connections 2024-03-27T10:54:39.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:39.436+0000 7fdab3000700 1 -- 172.21.15.97:0/2596887492 wait complete. 2024-03-27T10:54:39.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:39 smithi177 ceph-mon[35294]: pgmap v379: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 9 op/s 2024-03-27T10:54:39.767 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:39 smithi114 ceph-mon[35338]: pgmap v379: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 9 op/s 2024-03-27T10:54:40.641 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:40 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-03-27T10:54:40.641 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:40 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-03-27T10:54:40.641 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:40 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]: dispatch 2024-03-27T10:54:40.641 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:40 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-03-27T10:54:40.641 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:40 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-03-27T10:54:40.642 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:40 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 7]}]: dispatch 2024-03-27T10:54:40.642 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:40 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]: dispatch 2024-03-27T10:54:40.642 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:40 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [3, 6]}]: dispatch 2024-03-27T10:54:40.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:40 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-03-27T10:54:40.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:40 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-03-27T10:54:40.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:40 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]: dispatch 2024-03-27T10:54:40.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:40 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-03-27T10:54:40.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:40 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-03-27T10:54:40.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:40 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 7]}]: dispatch 2024-03-27T10:54:40.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:40 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]: dispatch 2024-03-27T10:54:40.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:40 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [3, 6]}]: dispatch 2024-03-27T10:54:40.767 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:40 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-03-27T10:54:40.767 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:40 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-03-27T10:54:40.767 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:40 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]: dispatch 2024-03-27T10:54:40.767 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:40 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-03-27T10:54:40.768 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:40 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-03-27T10:54:40.768 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:40 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 7]}]: dispatch 2024-03-27T10:54:40.768 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:40 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]: dispatch 2024-03-27T10:54:40.768 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:40 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [3, 6]}]: dispatch 2024-03-27T10:54:40.776 INFO:teuthology.orchestra.run.smithi097.stdout:150323855432 2024-03-27T10:54:40.776 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell --fsid adca88c4-ec26-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.4 2024-03-27T10:54:41.508 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:41 smithi097 ceph-mon[31859]: pgmap v380: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 426 B/s wr, 12 op/s 2024-03-27T10:54:41.508 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:41 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-03-27T10:54:41.508 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:41 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-03-27T10:54:41.508 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:41 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]': finished 2024-03-27T10:54:41.508 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:41 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-03-27T10:54:41.508 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:41 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-03-27T10:54:41.508 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:41 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 7]}]': finished 2024-03-27T10:54:41.508 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:41 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]': finished 2024-03-27T10:54:41.508 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:41 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [3, 6]}]': finished 2024-03-27T10:54:41.508 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:41 smithi097 ceph-mon[31859]: osdmap e91: 12 total, 12 up, 12 in 2024-03-27T10:54:41.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:41 smithi177 ceph-mon[35294]: pgmap v380: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 426 B/s wr, 12 op/s 2024-03-27T10:54:41.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:41 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-03-27T10:54:41.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:41 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-03-27T10:54:41.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:41 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]': finished 2024-03-27T10:54:41.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:41 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-03-27T10:54:41.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:41 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-03-27T10:54:41.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:41 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 7]}]': finished 2024-03-27T10:54:41.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:41 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]': finished 2024-03-27T10:54:41.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:41 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [3, 6]}]': finished 2024-03-27T10:54:41.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:41 smithi177 ceph-mon[35294]: osdmap e91: 12 total, 12 up, 12 in 2024-03-27T10:54:41.767 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:41 smithi114 ceph-mon[35338]: pgmap v380: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 426 B/s wr, 12 op/s 2024-03-27T10:54:41.767 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:41 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-03-27T10:54:41.767 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:41 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-03-27T10:54:41.767 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:41 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]': finished 2024-03-27T10:54:41.767 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:41 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-03-27T10:54:41.768 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:41 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-03-27T10:54:41.768 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:41 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 7]}]': finished 2024-03-27T10:54:41.768 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:41 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]': finished 2024-03-27T10:54:41.768 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:41 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [3, 6]}]': finished 2024-03-27T10:54:41.768 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:41 smithi114 ceph-mon[35338]: osdmap e91: 12 total, 12 up, 12 in 2024-03-27T10:54:41.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:41.862+0000 7f181b800700 1 -- 172.21.15.97:0/1384974896 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f181410a840 msgr2=0x7f181410ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:41.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:41.862+0000 7f181b800700 1 --2- 172.21.15.97:0/1384974896 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f181410a840 0x7f181410ac20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1808009990 tx=0x7f1808009ca0 comp rx=0 tx=0).stop 2024-03-27T10:54:41.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:41.862+0000 7f181b800700 1 -- 172.21.15.97:0/1384974896 shutdown_connections 2024-03-27T10:54:41.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:41.862+0000 7f181b800700 1 --2- 172.21.15.97:0/1384974896 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1814105200 0x7f1814111710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:41.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:41.862+0000 7f181b800700 1 --2- 172.21.15.97:0/1384974896 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f181410a840 0x7f181410ac20 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:41.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:41.862+0000 7f181b800700 1 --2- 172.21.15.97:0/1384974896 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1814104840 0x7f1814104cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:41.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:41.862+0000 7f181b800700 1 -- 172.21.15.97:0/1384974896 >> 172.21.15.97:0/1384974896 conn(0x7f1814100310 msgr2=0x7f1814102730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:54:41.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:41.862+0000 7f181b800700 1 -- 172.21.15.97:0/1384974896 shutdown_connections 2024-03-27T10:54:41.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:41.862+0000 7f181b800700 1 -- 172.21.15.97:0/1384974896 wait complete. 2024-03-27T10:54:41.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:41.863+0000 7f181b800700 1 Processor -- start 2024-03-27T10:54:41.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:41.863+0000 7f181b800700 1 -- start start 2024-03-27T10:54:41.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:41.864+0000 7f181b800700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1814104840 0x7f181419af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:41.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:41.864+0000 7f181b800700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1814105200 0x7f181419b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:41.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:41.864+0000 7f181b800700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f181410a840 0x7f181419f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:41.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:41.864+0000 7f181b800700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1814114a70 con 0x7f1814104840 2024-03-27T10:54:41.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:41.864+0000 7f181b800700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f18141148f0 con 0x7f181410a840 2024-03-27T10:54:41.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:41.864+0000 7f181b800700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1814114bf0 con 0x7f1814105200 2024-03-27T10:54:41.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:41.864+0000 7f1818e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1814104840 0x7f181419af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:41.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:41.864+0000 7f1818e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1814104840 0x7f181419af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45384/0 (socket says 172.21.15.97:45384) 2024-03-27T10:54:41.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:41.864+0000 7f1818e00700 1 -- 172.21.15.97:0/4274524773 learned_addr learned my addr 172.21.15.97:0/4274524773 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:54:41.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:41.865+0000 7f1813e00700 1 --2- 172.21.15.97:0/4274524773 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1814105200 0x7f181419b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:41.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:41.865+0000 7f1819800700 1 --2- 172.21.15.97:0/4274524773 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f181410a840 0x7f181419f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:41.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:41.865+0000 7f1818e00700 1 -- 172.21.15.97:0/4274524773 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1814105200 msgr2=0x7f181419b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:41.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:41.865+0000 7f1818e00700 1 --2- 172.21.15.97:0/4274524773 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1814105200 0x7f181419b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:41.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:41.865+0000 7f1818e00700 1 -- 172.21.15.97:0/4274524773 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f181410a840 msgr2=0x7f181419f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:41.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:41.865+0000 7f1818e00700 1 --2- 172.21.15.97:0/4274524773 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f181410a840 0x7f181419f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:41.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:41.865+0000 7f1818e00700 1 -- 172.21.15.97:0/4274524773 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f181419ff70 con 0x7f1814104840 2024-03-27T10:54:41.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:41.865+0000 7f1818e00700 1 --2- 172.21.15.97:0/4274524773 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1814104840 0x7f181419af60 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f1808000c00 tx=0x7f1808005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:54:41.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:41.865+0000 7f1811600700 1 -- 172.21.15.97:0/4274524773 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1808027070 con 0x7f1814104840 2024-03-27T10:54:41.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:41.865+0000 7f181b800700 1 -- 172.21.15.97:0/4274524773 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18141a0200 con 0x7f1814104840 2024-03-27T10:54:41.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:41.865+0000 7f1811600700 1 -- 172.21.15.97:0/4274524773 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1808022470 con 0x7f1814104840 2024-03-27T10:54:41.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:41.866+0000 7f1811600700 1 -- 172.21.15.97:0/4274524773 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1808017630 con 0x7f1814104840 2024-03-27T10:54:41.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:41.866+0000 7f181b800700 1 -- 172.21.15.97:0/4274524773 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18141a7ae0 con 0x7f1814104840 2024-03-27T10:54:41.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:41.867+0000 7f181b800700 1 -- 172.21.15.97:0/4274524773 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1814062dd0 con 0x7f1814104840 2024-03-27T10:54:41.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:41.870+0000 7f1811600700 1 -- 172.21.15.97:0/4274524773 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138421+0+0 (secure 0 0 0) 0x7f180802a070 con 0x7f1814104840 2024-03-27T10:54:41.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:41.871+0000 7f1811600700 1 --2- 172.21.15.97:0/4274524773 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f17fc0a3730 0x7f17fc0a5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:41.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:41.871+0000 7f1811600700 1 -- 172.21.15.97:0/4274524773 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f18080d0fe0 con 0x7f1814104840 2024-03-27T10:54:41.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:41.872+0000 7f1813e00700 1 --2- 172.21.15.97:0/4274524773 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f17fc0a3730 0x7f17fc0a5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:41.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:41.872+0000 7f1811600700 1 --2- 172.21.15.97:0/4274524773 >> [v2:172.21.15.97:6818/2892389563,v1:172.21.15.97:6819/2892389563] conn(0x7f17fc0ad270 0x7f17fc0af710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:41.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:41.872+0000 7f1811600700 1 -- 172.21.15.97:0/4274524773 --> [v2:172.21.15.97:6818/2892389563,v1:172.21.15.97:6819/2892389563] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1808024ec0 con 0x7f17fc0ad270 2024-03-27T10:54:41.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:41.872+0000 7f1811600700 1 -- 172.21.15.97:0/4274524773 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f18080d1410 con 0x7f1814104840 2024-03-27T10:54:41.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:41.872+0000 7f1819800700 1 --2- 172.21.15.97:0/4274524773 >> [v2:172.21.15.97:6818/2892389563,v1:172.21.15.97:6819/2892389563] conn(0x7f17fc0ad270 0x7f17fc0af710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:41.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:41.873+0000 7f1813e00700 1 --2- 172.21.15.97:0/4274524773 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f17fc0a3730 0x7f17fc0a5bf0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1800005950 tx=0x7f18000058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:54:41.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:41.873+0000 7f1819800700 1 --2- 172.21.15.97:0/4274524773 >> [v2:172.21.15.97:6818/2892389563,v1:172.21.15.97:6819/2892389563] conn(0x7f17fc0ad270 0x7f17fc0af710 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:54:41.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:41.874+0000 7f1811600700 1 -- 172.21.15.97:0/4274524773 <== osd.6 v2:172.21.15.97:6818/2892389563 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1808024ec0 con 0x7f17fc0ad270 2024-03-27T10:54:41.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:41.892+0000 7f181b800700 1 -- 172.21.15.97:0/4274524773 --> [v2:172.21.15.97:6818/2892389563,v1:172.21.15.97:6819/2892389563] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1814063250 con 0x7f17fc0ad270 2024-03-27T10:54:41.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:41.892+0000 7f1811600700 1 -- 172.21.15.97:0/4274524773 <== osd.6 v2:172.21.15.97:6818/2892389563 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f1814063250 con 0x7f17fc0ad270 2024-03-27T10:54:41.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:41.893+0000 7f181b800700 1 -- 172.21.15.97:0/4274524773 >> [v2:172.21.15.97:6818/2892389563,v1:172.21.15.97:6819/2892389563] conn(0x7f17fc0ad270 msgr2=0x7f17fc0af710 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:41.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:41.893+0000 7f181b800700 1 --2- 172.21.15.97:0/4274524773 >> [v2:172.21.15.97:6818/2892389563,v1:172.21.15.97:6819/2892389563] conn(0x7f17fc0ad270 0x7f17fc0af710 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:41.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:41.893+0000 7f181b800700 1 -- 172.21.15.97:0/4274524773 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f17fc0a3730 msgr2=0x7f17fc0a5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:41.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:41.893+0000 7f181b800700 1 --2- 172.21.15.97:0/4274524773 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f17fc0a3730 0x7f17fc0a5bf0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1800005950 tx=0x7f18000058e0 comp rx=0 tx=0).stop 2024-03-27T10:54:41.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:41.893+0000 7f181b800700 1 -- 172.21.15.97:0/4274524773 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1814104840 msgr2=0x7f181419af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:41.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:41.893+0000 7f181b800700 1 --2- 172.21.15.97:0/4274524773 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1814104840 0x7f181419af60 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f1808000c00 tx=0x7f1808005ef0 comp rx=0 tx=0).stop 2024-03-27T10:54:41.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:41.893+0000 7f1819800700 1 -- 172.21.15.97:0/4274524773 reap_dead start 2024-03-27T10:54:41.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:41.893+0000 7f181b800700 1 -- 172.21.15.97:0/4274524773 shutdown_connections 2024-03-27T10:54:41.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:41.893+0000 7f181b800700 1 -- 172.21.15.97:0/4274524773 >> 172.21.15.97:0/4274524773 conn(0x7f1814100310 msgr2=0x7f1814100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:54:41.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:41.893+0000 7f181b800700 1 -- 172.21.15.97:0/4274524773 shutdown_connections 2024-03-27T10:54:41.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:41.893+0000 7f181b800700 1 -- 172.21.15.97:0/4274524773 wait complete. 2024-03-27T10:54:42.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:42 smithi097 ceph-mon[31859]: osdmap e92: 12 total, 12 up, 12 in 2024-03-27T10:54:42.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:42 smithi177 ceph-mon[35294]: osdmap e92: 12 total, 12 up, 12 in 2024-03-27T10:54:42.767 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:42 smithi114 ceph-mon[35338]: osdmap e92: 12 total, 12 up, 12 in 2024-03-27T10:54:43.641 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:43 smithi097 ceph-mon[31859]: pgmap v383: 97 pgs: 97 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s; 72 B/s, 0 objects/s recovering 2024-03-27T10:54:43.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:43 smithi177 ceph-mon[35294]: pgmap v383: 97 pgs: 97 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s; 72 B/s, 0 objects/s recovering 2024-03-27T10:54:43.767 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:43 smithi114 ceph-mon[35338]: pgmap v383: 97 pgs: 97 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s; 72 B/s, 0 objects/s recovering 2024-03-27T10:54:44.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:44.187+0000 7fd05dc00700 1 -- 172.21.15.97:0/568850787 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd05806acb0 msgr2=0x7fd05810f4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:44.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:44.187+0000 7fd05dc00700 1 --2- 172.21.15.97:0/568850787 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd05806acb0 0x7fd05810f4d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd05000b0d0 tx=0x7fd05000b3e0 comp rx=0 tx=0).stop 2024-03-27T10:54:44.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:44.188+0000 7fd05dc00700 1 -- 172.21.15.97:0/568850787 shutdown_connections 2024-03-27T10:54:44.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:44.188+0000 7fd05dc00700 1 --2- 172.21.15.97:0/568850787 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd05806acb0 0x7fd05810f4d0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:44.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:44.188+0000 7fd05dc00700 1 --2- 172.21.15.97:0/568850787 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd058108670 0x7fd058108a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:44.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:44.188+0000 7fd05dc00700 1 --2- 172.21.15.97:0/568850787 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd05806a2f0 0x7fd05806a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:44.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:44.188+0000 7fd05dc00700 1 -- 172.21.15.97:0/568850787 >> 172.21.15.97:0/568850787 conn(0x7fd058078470 msgr2=0x7fd058078880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:54:44.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:44.188+0000 7fd05dc00700 1 -- 172.21.15.97:0/568850787 shutdown_connections 2024-03-27T10:54:44.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:44.188+0000 7fd05dc00700 1 -- 172.21.15.97:0/568850787 wait complete. 2024-03-27T10:54:44.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:44.189+0000 7fd05dc00700 1 Processor -- start 2024-03-27T10:54:44.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:44.189+0000 7fd05dc00700 1 -- start start 2024-03-27T10:54:44.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:44.189+0000 7fd05dc00700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd05806a2f0 0x7fd058101610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:44.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:44.189+0000 7fd05dc00700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd05806acb0 0x7fd058101b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:44.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:44.189+0000 7fd05dc00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd058108670 0x7fd058102090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:44.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:44.189+0000 7fd05dc00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd0581127c0 con 0x7fd058108670 2024-03-27T10:54:44.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:44.189+0000 7fd05dc00700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd058112640 con 0x7fd05806acb0 2024-03-27T10:54:44.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:44.189+0000 7fd05dc00700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd058112940 con 0x7fd05806a2f0 2024-03-27T10:54:44.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:44.190+0000 7fd057400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd058108670 0x7fd058102090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:44.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:44.190+0000 7fd057400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd058108670 0x7fd058102090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45402/0 (socket says 172.21.15.97:45402) 2024-03-27T10:54:44.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:44.190+0000 7fd057400700 1 -- 172.21.15.97:0/2988406719 learned_addr learned my addr 172.21.15.97:0/2988406719 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:54:44.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:44.190+0000 7fd04fe00700 1 --2- 172.21.15.97:0/2988406719 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd05806acb0 0x7fd058101b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:44.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:44.190+0000 7fd056a00700 1 --2- 172.21.15.97:0/2988406719 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd05806a2f0 0x7fd058101610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:44.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:44.190+0000 7fd057400700 1 -- 172.21.15.97:0/2988406719 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd05806a2f0 msgr2=0x7fd058101610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:44.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:44.190+0000 7fd057400700 1 --2- 172.21.15.97:0/2988406719 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd05806a2f0 0x7fd058101610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:44.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:44.190+0000 7fd057400700 1 -- 172.21.15.97:0/2988406719 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd05806acb0 msgr2=0x7fd058101b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:44.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:44.190+0000 7fd057400700 1 --2- 172.21.15.97:0/2988406719 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd05806acb0 0x7fd058101b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:44.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:44.190+0000 7fd057400700 1 -- 172.21.15.97:0/2988406719 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0581a7140 con 0x7fd058108670 2024-03-27T10:54:44.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:44.190+0000 7fd057400700 1 --2- 172.21.15.97:0/2988406719 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd058108670 0x7fd058102090 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd0500079a0 tx=0x7fd050007a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:54:44.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:44.191+0000 7fd04f400700 1 -- 172.21.15.97:0/2988406719 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd05001f070 con 0x7fd058108670 2024-03-27T10:54:44.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:44.191+0000 7fd05dc00700 1 -- 172.21.15.97:0/2988406719 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0581a73d0 con 0x7fd058108670 2024-03-27T10:54:44.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:44.191+0000 7fd04f400700 1 -- 172.21.15.97:0/2988406719 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd050003c00 con 0x7fd058108670 2024-03-27T10:54:44.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:44.191+0000 7fd04f400700 1 -- 172.21.15.97:0/2988406719 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd05001b6a0 con 0x7fd058108670 2024-03-27T10:54:44.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:44.191+0000 7fd05dc00700 1 -- 172.21.15.97:0/2988406719 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0581a7820 con 0x7fd058108670 2024-03-27T10:54:44.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:44.192+0000 7fd05dc00700 1 -- 172.21.15.97:0/2988406719 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd058062dd0 con 0x7fd058108670 2024-03-27T10:54:44.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:44.194+0000 7fd04f400700 1 -- 172.21.15.97:0/2988406719 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138421+0+0 (secure 0 0 0) 0x7fd050022070 con 0x7fd058108670 2024-03-27T10:54:44.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:44.195+0000 7fd04f400700 1 --2- 172.21.15.97:0/2988406719 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fd03c0a3730 0x7fd03c0a5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:44.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:44.195+0000 7fd04f400700 1 -- 172.21.15.97:0/2988406719 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fd05000f020 con 0x7fd058108670 2024-03-27T10:54:44.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:44.195+0000 7fd056a00700 1 --2- 172.21.15.97:0/2988406719 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fd03c0a3730 0x7fd03c0a5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:44.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:44.196+0000 7fd04f400700 1 --2- 172.21.15.97:0/2988406719 >> [v2:172.21.15.177:6800/124240658,v1:172.21.15.177:6801/124240658] conn(0x7fd03c0ad270 0x7fd03c0af710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:44.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:44.196+0000 7fd04f400700 1 -- 172.21.15.97:0/2988406719 --> [v2:172.21.15.177:6800/124240658,v1:172.21.15.177:6801/124240658] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd03c0afde0 con 0x7fd03c0ad270 2024-03-27T10:54:44.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:44.196+0000 7fd04f400700 1 -- 172.21.15.97:0/2988406719 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7fd0500d1710 con 0x7fd058108670 2024-03-27T10:54:44.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:44.196+0000 7fd056a00700 1 --2- 172.21.15.97:0/2988406719 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fd03c0a3730 0x7fd03c0a5bf0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd048005950 tx=0x7fd0480058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:54:44.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:44.196+0000 7fd04fe00700 1 --2- 172.21.15.97:0/2988406719 >> [v2:172.21.15.177:6800/124240658,v1:172.21.15.177:6801/124240658] conn(0x7fd03c0ad270 0x7fd03c0af710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:44.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:44.197+0000 7fd04fe00700 1 --2- 172.21.15.97:0/2988406719 >> [v2:172.21.15.177:6800/124240658,v1:172.21.15.177:6801/124240658] conn(0x7fd03c0ad270 0x7fd03c0af710 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:54:44.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:44.200+0000 7fd04f400700 1 -- 172.21.15.97:0/2988406719 <== osd.2 v2:172.21.15.177:6800/124240658 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd03c0afde0 con 0x7fd03c0ad270 2024-03-27T10:54:44.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:44.218+0000 7fd05dc00700 1 -- 172.21.15.97:0/2988406719 --> [v2:172.21.15.177:6800/124240658,v1:172.21.15.177:6801/124240658] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd058063250 con 0x7fd03c0ad270 2024-03-27T10:54:44.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:44.219+0000 7fd04f400700 1 -- 172.21.15.97:0/2988406719 <== osd.2 v2:172.21.15.177:6800/124240658 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fd058063250 con 0x7fd03c0ad270 2024-03-27T10:54:44.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:44.219+0000 7fd05dc00700 1 -- 172.21.15.97:0/2988406719 >> [v2:172.21.15.177:6800/124240658,v1:172.21.15.177:6801/124240658] conn(0x7fd03c0ad270 msgr2=0x7fd03c0af710 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:44.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:44.219+0000 7fd05dc00700 1 --2- 172.21.15.97:0/2988406719 >> [v2:172.21.15.177:6800/124240658,v1:172.21.15.177:6801/124240658] conn(0x7fd03c0ad270 0x7fd03c0af710 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:44.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:44.220+0000 7fd05dc00700 1 -- 172.21.15.97:0/2988406719 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fd03c0a3730 msgr2=0x7fd03c0a5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:44.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:44.220+0000 7fd05dc00700 1 --2- 172.21.15.97:0/2988406719 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fd03c0a3730 0x7fd03c0a5bf0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd048005950 tx=0x7fd0480058e0 comp rx=0 tx=0).stop 2024-03-27T10:54:44.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:44.220+0000 7fd05dc00700 1 -- 172.21.15.97:0/2988406719 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd058108670 msgr2=0x7fd058102090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:44.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:44.220+0000 7fd05dc00700 1 --2- 172.21.15.97:0/2988406719 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd058108670 0x7fd058102090 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd0500079a0 tx=0x7fd050007a80 comp rx=0 tx=0).stop 2024-03-27T10:54:44.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:44.220+0000 7fd057400700 1 -- 172.21.15.97:0/2988406719 reap_dead start 2024-03-27T10:54:44.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:44.220+0000 7fd05dc00700 1 -- 172.21.15.97:0/2988406719 shutdown_connections 2024-03-27T10:54:44.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:44.220+0000 7fd05dc00700 1 -- 172.21.15.97:0/2988406719 >> 172.21.15.97:0/2988406719 conn(0x7fd058078470 msgr2=0x7fd0581000b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:54:44.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:44.220+0000 7fd05dc00700 1 -- 172.21.15.97:0/2988406719 shutdown_connections 2024-03-27T10:54:44.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:44.220+0000 7fd05dc00700 1 -- 172.21.15.97:0/2988406719 wait complete. 2024-03-27T10:54:45.493 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:45 smithi097 ceph-mon[31859]: pgmap v384: 97 pgs: 97 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 10 op/s; 72 B/s, 0 objects/s recovering 2024-03-27T10:54:45.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:45 smithi177 ceph-mon[35294]: pgmap v384: 97 pgs: 97 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 10 op/s; 72 B/s, 0 objects/s recovering 2024-03-27T10:54:45.767 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:45 smithi114 ceph-mon[35338]: pgmap v384: 97 pgs: 97 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 10 op/s; 72 B/s, 0 objects/s recovering 2024-03-27T10:54:46.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:46.247+0000 7f88bec00700 1 -- 172.21.15.97:0/2057352330 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88b810f040 msgr2=0x7f88b8111430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:46.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:46.247+0000 7f88bec00700 1 --2- 172.21.15.97:0/2057352330 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88b810f040 0x7f88b8111430 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f88b000b0a0 tx=0x7f88b000b3b0 comp rx=0 tx=0).stop 2024-03-27T10:54:46.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:46.247+0000 7f88bec00700 1 -- 172.21.15.97:0/2057352330 shutdown_connections 2024-03-27T10:54:46.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:46.247+0000 7f88bec00700 1 --2- 172.21.15.97:0/2057352330 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88b810f040 0x7f88b8111430 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:46.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:46.247+0000 7f88bec00700 1 --2- 172.21.15.97:0/2057352330 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f88b8074870 0x7f88b8074c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:46.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:46.247+0000 7f88bec00700 1 --2- 172.21.15.97:0/2057352330 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f88b8075190 0x7f88b810eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:46.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:46.247+0000 7f88bec00700 1 -- 172.21.15.97:0/2057352330 >> 172.21.15.97:0/2057352330 conn(0x7f88b8079770 msgr2=0x7f88b8079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:54:46.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:46.248+0000 7f88bec00700 1 -- 172.21.15.97:0/2057352330 shutdown_connections 2024-03-27T10:54:46.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:46.248+0000 7f88bec00700 1 -- 172.21.15.97:0/2057352330 wait complete. 2024-03-27T10:54:46.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:46.249+0000 7f88bec00700 1 Processor -- start 2024-03-27T10:54:46.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:46.250+0000 7f88bec00700 1 -- start start 2024-03-27T10:54:46.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:46.250+0000 7f88bec00700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f88b8074870 0x7f88b819af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:46.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:46.250+0000 7f88bec00700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f88b8075190 0x7f88b819b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:46.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:46.250+0000 7f88bec00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88b810f040 0x7f88b819f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:46.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:46.250+0000 7f88bec00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f88b81149f0 con 0x7f88b810f040 2024-03-27T10:54:46.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:46.250+0000 7f88bec00700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f88b8114870 con 0x7f88b8075190 2024-03-27T10:54:46.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:46.250+0000 7f88bec00700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f88b8114b70 con 0x7f88b8074870 2024-03-27T10:54:46.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:46.251+0000 7f88b7400700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f88b8075190 0x7f88b819b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:46.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:46.251+0000 7f88bcc00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88b810f040 0x7f88b819f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:46.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:46.251+0000 7f88b7e00700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f88b8074870 0x7f88b819af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:46.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:46.251+0000 7f88bcc00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88b810f040 0x7f88b819f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45420/0 (socket says 172.21.15.97:45420) 2024-03-27T10:54:46.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:46.251+0000 7f88bcc00700 1 -- 172.21.15.97:0/809923255 learned_addr learned my addr 172.21.15.97:0/809923255 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:54:46.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:46.251+0000 7f88b7400700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f88b8075190 0x7f88b819b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:50246/0 (socket says 172.21.15.97:50246) 2024-03-27T10:54:46.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:46.251+0000 7f88bcc00700 1 -- 172.21.15.97:0/809923255 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f88b8074870 msgr2=0x7f88b819af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:46.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:46.251+0000 7f88bcc00700 1 --2- 172.21.15.97:0/809923255 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f88b8074870 0x7f88b819af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:46.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:46.251+0000 7f88bcc00700 1 -- 172.21.15.97:0/809923255 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f88b8075190 msgr2=0x7f88b819b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:46.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:46.251+0000 7f88bcc00700 1 --2- 172.21.15.97:0/809923255 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f88b8075190 0x7f88b819b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:46.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:46.251+0000 7f88bcc00700 1 -- 172.21.15.97:0/809923255 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88b819fee0 con 0x7f88b810f040 2024-03-27T10:54:46.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:46.252+0000 7f88bcc00700 1 --2- 172.21.15.97:0/809923255 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88b810f040 0x7f88b819f7a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f88b0009fd0 tx=0x7f88b0004dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:54:46.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:46.252+0000 7f88b4c00700 1 -- 172.21.15.97:0/809923255 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b0021070 con 0x7f88b810f040 2024-03-27T10:54:46.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:46.252+0000 7f88b4c00700 1 -- 172.21.15.97:0/809923255 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f88b0007b80 con 0x7f88b810f040 2024-03-27T10:54:46.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:46.252+0000 7f88b4c00700 1 -- 172.21.15.97:0/809923255 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b00052a0 con 0x7f88b810f040 2024-03-27T10:54:46.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:46.252+0000 7f88bec00700 1 -- 172.21.15.97:0/809923255 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88b81a0170 con 0x7f88b810f040 2024-03-27T10:54:46.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:46.252+0000 7f88bec00700 1 -- 172.21.15.97:0/809923255 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88b81a7a50 con 0x7f88b810f040 2024-03-27T10:54:46.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:46.253+0000 7f88bec00700 1 -- 172.21.15.97:0/809923255 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f88b8062d90 con 0x7f88b810f040 2024-03-27T10:54:46.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:46.255+0000 7f88b4c00700 1 -- 172.21.15.97:0/809923255 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138421+0+0 (secure 0 0 0) 0x7f88b0024070 con 0x7f88b810f040 2024-03-27T10:54:46.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:46.256+0000 7f88b4c00700 1 --2- 172.21.15.97:0/809923255 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f889c0a37b0 0x7f889c0a5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:46.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:46.257+0000 7f88b4c00700 1 -- 172.21.15.97:0/809923255 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f88b00d0fa0 con 0x7f88b810f040 2024-03-27T10:54:46.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:46.257+0000 7f88b7e00700 1 --2- 172.21.15.97:0/809923255 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f889c0a37b0 0x7f889c0a5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:46.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:46.257+0000 7f88b4c00700 1 --2- 172.21.15.97:0/809923255 >> [v2:172.21.15.97:6826/1578881271,v1:172.21.15.97:6827/1578881271] conn(0x7f889c0ad2f0 0x7f889c0af790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:46.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:46.257+0000 7f88b4c00700 1 -- 172.21.15.97:0/809923255 --> [v2:172.21.15.97:6826/1578881271,v1:172.21.15.97:6827/1578881271] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f889c0afe60 con 0x7f889c0ad2f0 2024-03-27T10:54:46.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:46.257+0000 7f88b4c00700 1 -- 172.21.15.97:0/809923255 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f88b00d1350 con 0x7f88b810f040 2024-03-27T10:54:46.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:46.257+0000 7f88b7400700 1 --2- 172.21.15.97:0/809923255 >> [v2:172.21.15.97:6826/1578881271,v1:172.21.15.97:6827/1578881271] conn(0x7f889c0ad2f0 0x7f889c0af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:46.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:46.258+0000 7f88b7e00700 1 --2- 172.21.15.97:0/809923255 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f889c0a37b0 0x7f889c0a5c70 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f88a8004630 tx=0x7f88a800b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:54:46.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:46.258+0000 7f88b7400700 1 --2- 172.21.15.97:0/809923255 >> [v2:172.21.15.97:6826/1578881271,v1:172.21.15.97:6827/1578881271] conn(0x7f889c0ad2f0 0x7f889c0af790 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:54:46.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:46.259+0000 7f88b4c00700 1 -- 172.21.15.97:0/809923255 <== osd.9 v2:172.21.15.97:6826/1578881271 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f889c0afe60 con 0x7f889c0ad2f0 2024-03-27T10:54:46.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:46.277+0000 7f88bec00700 1 -- 172.21.15.97:0/809923255 --> [v2:172.21.15.97:6826/1578881271,v1:172.21.15.97:6827/1578881271] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f88b8063210 con 0x7f889c0ad2f0 2024-03-27T10:54:46.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:46.277+0000 7f88b4c00700 1 -- 172.21.15.97:0/809923255 <== osd.9 v2:172.21.15.97:6826/1578881271 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f88b8063210 con 0x7f889c0ad2f0 2024-03-27T10:54:46.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:46.278+0000 7f88bec00700 1 -- 172.21.15.97:0/809923255 >> [v2:172.21.15.97:6826/1578881271,v1:172.21.15.97:6827/1578881271] conn(0x7f889c0ad2f0 msgr2=0x7f889c0af790 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:46.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:46.278+0000 7f88bec00700 1 --2- 172.21.15.97:0/809923255 >> [v2:172.21.15.97:6826/1578881271,v1:172.21.15.97:6827/1578881271] conn(0x7f889c0ad2f0 0x7f889c0af790 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:46.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:46.278+0000 7f88bec00700 1 -- 172.21.15.97:0/809923255 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f889c0a37b0 msgr2=0x7f889c0a5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:46.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:46.278+0000 7f88bec00700 1 --2- 172.21.15.97:0/809923255 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f889c0a37b0 0x7f889c0a5c70 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f88a8004630 tx=0x7f88a800b3c0 comp rx=0 tx=0).stop 2024-03-27T10:54:46.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:46.278+0000 7f88bec00700 1 -- 172.21.15.97:0/809923255 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88b810f040 msgr2=0x7f88b819f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:46.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:46.278+0000 7f88bec00700 1 --2- 172.21.15.97:0/809923255 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88b810f040 0x7f88b819f7a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f88b0009fd0 tx=0x7f88b0004dc0 comp rx=0 tx=0).stop 2024-03-27T10:54:46.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:46.278+0000 7f88bcc00700 1 -- 172.21.15.97:0/809923255 reap_dead start 2024-03-27T10:54:46.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:46.278+0000 7f88bec00700 1 -- 172.21.15.97:0/809923255 shutdown_connections 2024-03-27T10:54:46.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:46.278+0000 7f88bec00700 1 -- 172.21.15.97:0/809923255 >> 172.21.15.97:0/809923255 conn(0x7f88b8079770 msgr2=0x7f88b80ff8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:54:46.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:46.279+0000 7f88bec00700 1 -- 172.21.15.97:0/809923255 shutdown_connections 2024-03-27T10:54:46.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:46.279+0000 7f88bec00700 1 -- 172.21.15.97:0/809923255 wait complete. 2024-03-27T10:54:47.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.078+0000 7f7b52000700 1 -- 172.21.15.97:0/3664304141 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b4c06ad40 msgr2=0x7f7b4c06b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:47.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.078+0000 7f7b52000700 1 --2- 172.21.15.97:0/3664304141 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b4c06ad40 0x7f7b4c06b1a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7b380099c0 tx=0x7f7b38009cd0 comp rx=0 tx=0).stop 2024-03-27T10:54:47.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.079+0000 7f7b52000700 1 -- 172.21.15.97:0/3664304141 shutdown_connections 2024-03-27T10:54:47.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.079+0000 7f7b52000700 1 --2- 172.21.15.97:0/3664304141 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7b4c06b6e0 0x7f7b4c10f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:47.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.079+0000 7f7b52000700 1 --2- 172.21.15.97:0/3664304141 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7b4c108630 0x7f7b4c108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:47.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.079+0000 7f7b52000700 1 --2- 172.21.15.97:0/3664304141 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b4c06ad40 0x7f7b4c06b1a0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:47.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.079+0000 7f7b52000700 1 -- 172.21.15.97:0/3664304141 >> 172.21.15.97:0/3664304141 conn(0x7f7b4c077560 msgr2=0x7f7b4c077970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:54:47.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.079+0000 7f7b52000700 1 -- 172.21.15.97:0/3664304141 shutdown_connections 2024-03-27T10:54:47.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.079+0000 7f7b52000700 1 -- 172.21.15.97:0/3664304141 wait complete. 2024-03-27T10:54:47.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.080+0000 7f7b52000700 1 Processor -- start 2024-03-27T10:54:47.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.080+0000 7f7b52000700 1 -- start start 2024-03-27T10:54:47.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.081+0000 7f7b52000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b4c06ad40 0x7f7b4c19aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:47.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.081+0000 7f7b52000700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7b4c06b6e0 0x7f7b4c19b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:47.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.081+0000 7f7b52000700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7b4c108630 0x7f7b4c19f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:47.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.081+0000 7f7b52000700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7b4c1127d0 con 0x7f7b4c06ad40 2024-03-27T10:54:47.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.081+0000 7f7b52000700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f7b4c112650 con 0x7f7b4c108630 2024-03-27T10:54:47.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.081+0000 7f7b52000700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f7b4c112950 con 0x7f7b4c06b6e0 2024-03-27T10:54:47.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.081+0000 7f7b4b400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b4c06ad40 0x7f7b4c19aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:47.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.081+0000 7f7b4b400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b4c06ad40 0x7f7b4c19aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45432/0 (socket says 172.21.15.97:45432) 2024-03-27T10:54:47.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.081+0000 7f7b4b400700 1 -- 172.21.15.97:0/1651932298 learned_addr learned my addr 172.21.15.97:0/1651932298 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:54:47.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.081+0000 7f7b4be00700 1 --2- 172.21.15.97:0/1651932298 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7b4c108630 0x7f7b4c19f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:47.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.081+0000 7f7b4aa00700 1 --2- 172.21.15.97:0/1651932298 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7b4c06b6e0 0x7f7b4c19b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:47.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.081+0000 7f7b4b400700 1 -- 172.21.15.97:0/1651932298 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7b4c06b6e0 msgr2=0x7f7b4c19b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:47.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.081+0000 7f7b4b400700 1 --2- 172.21.15.97:0/1651932298 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7b4c06b6e0 0x7f7b4c19b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:47.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.081+0000 7f7b4b400700 1 -- 172.21.15.97:0/1651932298 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7b4c108630 msgr2=0x7f7b4c19f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:47.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.081+0000 7f7b4b400700 1 --2- 172.21.15.97:0/1651932298 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7b4c108630 0x7f7b4c19f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:47.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.081+0000 7f7b4b400700 1 -- 172.21.15.97:0/1651932298 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b4c19fe60 con 0x7f7b4c06ad40 2024-03-27T10:54:47.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.082+0000 7f7b4aa00700 1 --2- 172.21.15.97:0/1651932298 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7b4c06b6e0 0x7f7b4c19b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:54:47.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.082+0000 7f7b4b400700 1 --2- 172.21.15.97:0/1651932298 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b4c06ad40 0x7f7b4c19aea0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c00d950 tx=0x7f7b3c00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:54:47.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.082+0000 7f7b43e00700 1 -- 172.21.15.97:0/1651932298 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b3c016070 con 0x7f7b4c06ad40 2024-03-27T10:54:47.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.082+0000 7f7b43e00700 1 -- 172.21.15.97:0/1651932298 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7b3c010460 con 0x7f7b4c06ad40 2024-03-27T10:54:47.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.082+0000 7f7b52000700 1 -- 172.21.15.97:0/1651932298 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b4c1a0150 con 0x7f7b4c06ad40 2024-03-27T10:54:47.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.082+0000 7f7b43e00700 1 -- 172.21.15.97:0/1651932298 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b3c00f620 con 0x7f7b4c06ad40 2024-03-27T10:54:47.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.082+0000 7f7b52000700 1 -- 172.21.15.97:0/1651932298 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b4c1a79d0 con 0x7f7b4c06ad40 2024-03-27T10:54:47.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.083+0000 7f7b52000700 1 -- 172.21.15.97:0/1651932298 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7b4c102540 con 0x7f7b4c06ad40 2024-03-27T10:54:47.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.086+0000 7f7b43e00700 1 -- 172.21.15.97:0/1651932298 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138421+0+0 (secure 0 0 0) 0x7f7b3c019070 con 0x7f7b4c06ad40 2024-03-27T10:54:47.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.087+0000 7f7b43e00700 1 --2- 172.21.15.97:0/1651932298 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f7b300a37b0 0x7f7b300a5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:47.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.087+0000 7f7b43e00700 1 -- 172.21.15.97:0/1651932298 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f7b3c0cff00 con 0x7f7b4c06ad40 2024-03-27T10:54:47.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.087+0000 7f7b4aa00700 1 --2- 172.21.15.97:0/1651932298 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f7b300a37b0 0x7f7b300a5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:47.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.087+0000 7f7b43e00700 1 --2- 172.21.15.97:0/1651932298 >> [v2:172.21.15.97:6810/2478179311,v1:172.21.15.97:6811/2478179311] conn(0x7f7b300ad2f0 0x7f7b300af790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:47.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.087+0000 7f7b43e00700 1 -- 172.21.15.97:0/1651932298 --> [v2:172.21.15.97:6810/2478179311,v1:172.21.15.97:6811/2478179311] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7b3c005690 con 0x7f7b300ad2f0 2024-03-27T10:54:47.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.087+0000 7f7b43e00700 1 -- 172.21.15.97:0/1651932298 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f7b3c0d0350 con 0x7f7b4c06ad40 2024-03-27T10:54:47.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.088+0000 7f7b4be00700 1 --2- 172.21.15.97:0/1651932298 >> [v2:172.21.15.97:6810/2478179311,v1:172.21.15.97:6811/2478179311] conn(0x7f7b300ad2f0 0x7f7b300af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:47.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.088+0000 7f7b4aa00700 1 --2- 172.21.15.97:0/1651932298 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f7b300a37b0 0x7f7b300a5c70 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f7b3800b580 tx=0x7f7b38009f30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:54:47.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.088+0000 7f7b4be00700 1 --2- 172.21.15.97:0/1651932298 >> [v2:172.21.15.97:6810/2478179311,v1:172.21.15.97:6811/2478179311] conn(0x7f7b300ad2f0 0x7f7b300af790 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:54:47.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.089+0000 7f7b43e00700 1 -- 172.21.15.97:0/1651932298 <== osd.3 v2:172.21.15.97:6810/2478179311 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7b3c005690 con 0x7f7b300ad2f0 2024-03-27T10:54:47.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.110+0000 7f7b52000700 1 -- 172.21.15.97:0/1651932298 --> [v2:172.21.15.97:6810/2478179311,v1:172.21.15.97:6811/2478179311] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7b4c103d70 con 0x7f7b300ad2f0 2024-03-27T10:54:47.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.110+0000 7f7b43e00700 1 -- 172.21.15.97:0/1651932298 <== osd.3 v2:172.21.15.97:6810/2478179311 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f7b4c103d70 con 0x7f7b300ad2f0 2024-03-27T10:54:47.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.111+0000 7f7b52000700 1 -- 172.21.15.97:0/1651932298 >> [v2:172.21.15.97:6810/2478179311,v1:172.21.15.97:6811/2478179311] conn(0x7f7b300ad2f0 msgr2=0x7f7b300af790 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:47.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.111+0000 7f7b52000700 1 --2- 172.21.15.97:0/1651932298 >> [v2:172.21.15.97:6810/2478179311,v1:172.21.15.97:6811/2478179311] conn(0x7f7b300ad2f0 0x7f7b300af790 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:47.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.111+0000 7f7b52000700 1 -- 172.21.15.97:0/1651932298 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f7b300a37b0 msgr2=0x7f7b300a5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:47.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.111+0000 7f7b52000700 1 --2- 172.21.15.97:0/1651932298 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f7b300a37b0 0x7f7b300a5c70 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f7b3800b580 tx=0x7f7b38009f30 comp rx=0 tx=0).stop 2024-03-27T10:54:47.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.111+0000 7f7b52000700 1 -- 172.21.15.97:0/1651932298 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b4c06ad40 msgr2=0x7f7b4c19aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:47.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.111+0000 7f7b52000700 1 --2- 172.21.15.97:0/1651932298 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b4c06ad40 0x7f7b4c19aea0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c00d950 tx=0x7f7b3c00dd10 comp rx=0 tx=0).stop 2024-03-27T10:54:47.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.111+0000 7f7b4be00700 1 -- 172.21.15.97:0/1651932298 reap_dead start 2024-03-27T10:54:47.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.111+0000 7f7b52000700 1 -- 172.21.15.97:0/1651932298 shutdown_connections 2024-03-27T10:54:47.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.111+0000 7f7b52000700 1 -- 172.21.15.97:0/1651932298 >> 172.21.15.97:0/1651932298 conn(0x7f7b4c077560 msgr2=0x7f7b4c0ff800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:54:47.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.111+0000 7f7b52000700 1 -- 172.21.15.97:0/1651932298 shutdown_connections 2024-03-27T10:54:47.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.111+0000 7f7b52000700 1 -- 172.21.15.97:0/1651932298 wait complete. 2024-03-27T10:54:47.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.293+0000 7fc36c000700 1 -- 172.21.15.97:0/2302254420 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc364108670 msgr2=0x7fc364108a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:47.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.293+0000 7fc36c000700 1 --2- 172.21.15.97:0/2302254420 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc364108670 0x7fc364108a50 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc35800b0d0 tx=0x7fc35800b3e0 comp rx=0 tx=0).stop 2024-03-27T10:54:47.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.293+0000 7fc36c000700 1 -- 172.21.15.97:0/2302254420 shutdown_connections 2024-03-27T10:54:47.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.293+0000 7fc36c000700 1 --2- 172.21.15.97:0/2302254420 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc36406acb0 0x7fc36410f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:47.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.293+0000 7fc36c000700 1 --2- 172.21.15.97:0/2302254420 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc364108670 0x7fc364108a50 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:47.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.293+0000 7fc36c000700 1 --2- 172.21.15.97:0/2302254420 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc36406a2f0 0x7fc36406a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:47.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.293+0000 7fc36c000700 1 -- 172.21.15.97:0/2302254420 >> 172.21.15.97:0/2302254420 conn(0x7fc364078470 msgr2=0x7fc364078880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:54:47.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.293+0000 7fc36c000700 1 -- 172.21.15.97:0/2302254420 shutdown_connections 2024-03-27T10:54:47.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.293+0000 7fc36c000700 1 -- 172.21.15.97:0/2302254420 wait complete. 2024-03-27T10:54:47.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.294+0000 7fc36c000700 1 Processor -- start 2024-03-27T10:54:47.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.295+0000 7fc36c000700 1 -- start start 2024-03-27T10:54:47.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.295+0000 7fc36c000700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc36406a2f0 0x7fc36419ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:47.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.295+0000 7fc36c000700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc36406acb0 0x7fc36419b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:47.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.295+0000 7fc36c000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc364108670 0x7fc36419f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:47.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.295+0000 7fc36c000700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc364112780 con 0x7fc364108670 2024-03-27T10:54:47.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.295+0000 7fc36c000700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc364112600 con 0x7fc36406a2f0 2024-03-27T10:54:47.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.295+0000 7fc36c000700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc364112900 con 0x7fc36406acb0 2024-03-27T10:54:47.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.295+0000 7fc36a000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc364108670 0x7fc36419f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:47.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.295+0000 7fc36a000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc364108670 0x7fc36419f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45458/0 (socket says 172.21.15.97:45458) 2024-03-27T10:54:47.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.295+0000 7fc36a000700 1 -- 172.21.15.97:0/3242275322 learned_addr learned my addr 172.21.15.97:0/3242275322 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:54:47.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.296+0000 7fc369600700 1 --2- 172.21.15.97:0/3242275322 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc36406a2f0 0x7fc36419ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:47.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.296+0000 7fc368c00700 1 --2- 172.21.15.97:0/3242275322 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc36406acb0 0x7fc36419b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:47.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.296+0000 7fc36a000700 1 -- 172.21.15.97:0/3242275322 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc36406acb0 msgr2=0x7fc36419b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:47.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.296+0000 7fc36a000700 1 --2- 172.21.15.97:0/3242275322 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc36406acb0 0x7fc36419b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:47.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.296+0000 7fc36a000700 1 -- 172.21.15.97:0/3242275322 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc36406a2f0 msgr2=0x7fc36419ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:47.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.296+0000 7fc36a000700 1 --2- 172.21.15.97:0/3242275322 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc36406a2f0 0x7fc36419ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:47.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.296+0000 7fc36a000700 1 -- 172.21.15.97:0/3242275322 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc36419fdf0 con 0x7fc364108670 2024-03-27T10:54:47.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.296+0000 7fc368c00700 1 --2- 172.21.15.97:0/3242275322 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc36406acb0 0x7fc36419b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T10:54:47.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.296+0000 7fc36a000700 1 --2- 172.21.15.97:0/3242275322 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc364108670 0x7fc36419f6b0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc35c00c970 tx=0x7fc35c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:54:47.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.297+0000 7fc362000700 1 -- 172.21.15.97:0/3242275322 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc35c00ce60 con 0x7fc364108670 2024-03-27T10:54:47.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.297+0000 7fc362000700 1 -- 172.21.15.97:0/3242275322 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc35c004830 con 0x7fc364108670 2024-03-27T10:54:47.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.297+0000 7fc362000700 1 -- 172.21.15.97:0/3242275322 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc35c0054c0 con 0x7fc364108670 2024-03-27T10:54:47.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.297+0000 7fc36c000700 1 -- 172.21.15.97:0/3242275322 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3641a00e0 con 0x7fc364108670 2024-03-27T10:54:47.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.297+0000 7fc36c000700 1 -- 172.21.15.97:0/3242275322 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3641a7960 con 0x7fc364108670 2024-03-27T10:54:47.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.298+0000 7fc36c000700 1 -- 172.21.15.97:0/3242275322 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc364100a00 con 0x7fc364108670 2024-03-27T10:54:47.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.300+0000 7fc362000700 1 -- 172.21.15.97:0/3242275322 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138421+0+0 (secure 0 0 0) 0x7fc35c0049e0 con 0x7fc364108670 2024-03-27T10:54:47.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.302+0000 7fc362000700 1 --2- 172.21.15.97:0/3242275322 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fc34c0a3730 0x7fc34c0a5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:47.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.302+0000 7fc362000700 1 -- 172.21.15.97:0/3242275322 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fc35c0cfe50 con 0x7fc364108670 2024-03-27T10:54:47.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.302+0000 7fc369600700 1 --2- 172.21.15.97:0/3242275322 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fc34c0a3730 0x7fc34c0a5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:47.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.302+0000 7fc362000700 1 --2- 172.21.15.97:0/3242275322 >> [v2:172.21.15.114:6800/3858821264,v1:172.21.15.114:6801/3858821264] conn(0x7fc34c0ad270 0x7fc34c0af710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:47.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.302+0000 7fc362000700 1 -- 172.21.15.97:0/3242275322 --> [v2:172.21.15.114:6800/3858821264,v1:172.21.15.114:6801/3858821264] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc35c012ec0 con 0x7fc34c0ad270 2024-03-27T10:54:47.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.302+0000 7fc362000700 1 -- 172.21.15.97:0/3242275322 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7fc35c0d0200 con 0x7fc364108670 2024-03-27T10:54:47.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.303+0000 7fc369600700 1 --2- 172.21.15.97:0/3242275322 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fc34c0a3730 0x7fc34c0a5bf0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc358009e60 tx=0x7fc35800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:54:47.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.303+0000 7fc368c00700 1 --2- 172.21.15.97:0/3242275322 >> [v2:172.21.15.114:6800/3858821264,v1:172.21.15.114:6801/3858821264] conn(0x7fc34c0ad270 0x7fc34c0af710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:47.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.304+0000 7fc368c00700 1 --2- 172.21.15.97:0/3242275322 >> [v2:172.21.15.114:6800/3858821264,v1:172.21.15.114:6801/3858821264] conn(0x7fc34c0ad270 0x7fc34c0af710 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:54:47.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.306+0000 7fc362000700 1 -- 172.21.15.97:0/3242275322 <== osd.1 v2:172.21.15.114:6800/3858821264 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc35c012ec0 con 0x7fc34c0ad270 2024-03-27T10:54:47.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.325+0000 7fc36c000700 1 -- 172.21.15.97:0/3242275322 --> [v2:172.21.15.114:6800/3858821264,v1:172.21.15.114:6801/3858821264] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc364103d40 con 0x7fc34c0ad270 2024-03-27T10:54:47.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.326+0000 7fc362000700 1 -- 172.21.15.97:0/3242275322 <== osd.1 v2:172.21.15.114:6800/3858821264 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fc364103d40 con 0x7fc34c0ad270 2024-03-27T10:54:47.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.327+0000 7fc36c000700 1 -- 172.21.15.97:0/3242275322 >> [v2:172.21.15.114:6800/3858821264,v1:172.21.15.114:6801/3858821264] conn(0x7fc34c0ad270 msgr2=0x7fc34c0af710 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:47.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.327+0000 7fc36c000700 1 --2- 172.21.15.97:0/3242275322 >> [v2:172.21.15.114:6800/3858821264,v1:172.21.15.114:6801/3858821264] conn(0x7fc34c0ad270 0x7fc34c0af710 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:47.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.327+0000 7fc36c000700 1 -- 172.21.15.97:0/3242275322 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fc34c0a3730 msgr2=0x7fc34c0a5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:47.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.327+0000 7fc36c000700 1 --2- 172.21.15.97:0/3242275322 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fc34c0a3730 0x7fc34c0a5bf0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc358009e60 tx=0x7fc35800b040 comp rx=0 tx=0).stop 2024-03-27T10:54:47.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.327+0000 7fc36c000700 1 -- 172.21.15.97:0/3242275322 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc364108670 msgr2=0x7fc36419f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:47.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.327+0000 7fc36c000700 1 --2- 172.21.15.97:0/3242275322 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc364108670 0x7fc36419f6b0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc35c00c970 tx=0x7fc35c00cc80 comp rx=0 tx=0).stop 2024-03-27T10:54:47.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.327+0000 7fc36a000700 1 -- 172.21.15.97:0/3242275322 reap_dead start 2024-03-27T10:54:47.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.328+0000 7fc36c000700 1 -- 172.21.15.97:0/3242275322 shutdown_connections 2024-03-27T10:54:47.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.328+0000 7fc36c000700 1 -- 172.21.15.97:0/3242275322 >> 172.21.15.97:0/3242275322 conn(0x7fc364078470 msgr2=0x7fc3640fffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:54:47.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.328+0000 7fc36c000700 1 -- 172.21.15.97:0/3242275322 shutdown_connections 2024-03-27T10:54:47.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:47.328+0000 7fc36c000700 1 -- 172.21.15.97:0/3242275322 wait complete. 2024-03-27T10:54:47.589 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:47 smithi097 ceph-mon[31859]: pgmap v385: 97 pgs: 97 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 255 B/s wr, 11 op/s; 72 B/s, 0 objects/s recovering 2024-03-27T10:54:47.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:47 smithi177 ceph-mon[35294]: pgmap v385: 97 pgs: 97 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 255 B/s wr, 11 op/s; 72 B/s, 0 objects/s recovering 2024-03-27T10:54:47.767 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:47 smithi114 ceph-mon[35338]: pgmap v385: 97 pgs: 97 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 255 B/s wr, 11 op/s; 72 B/s, 0 objects/s recovering 2024-03-27T10:54:49.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.262+0000 7fc5a8200700 1 -- 172.21.15.97:0/1757584152 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5a006a2f0 msgr2=0x7fc5a006a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:49.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.262+0000 7fc5a8200700 1 --2- 172.21.15.97:0/1757584152 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5a006a2f0 0x7fc5a006a6d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fc5940098f0 tx=0x7fc594009c00 comp rx=0 tx=0).stop 2024-03-27T10:54:49.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.263+0000 7fc5a8200700 1 -- 172.21.15.97:0/1757584152 shutdown_connections 2024-03-27T10:54:49.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.263+0000 7fc5a8200700 1 --2- 172.21.15.97:0/1757584152 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc5a010f0f0 0x7fc5a01114e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:49.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.263+0000 7fc5a8200700 1 --2- 172.21.15.97:0/1757584152 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5a006a2f0 0x7fc5a006a6d0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:49.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.263+0000 7fc5a8200700 1 --2- 172.21.15.97:0/1757584152 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc5a006ac10 0x7fc5a010ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:49.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.263+0000 7fc5a8200700 1 -- 172.21.15.97:0/1757584152 >> 172.21.15.97:0/1757584152 conn(0x7fc5a0078470 msgr2=0x7fc5a0078880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:54:49.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.263+0000 7fc5a8200700 1 -- 172.21.15.97:0/1757584152 shutdown_connections 2024-03-27T10:54:49.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.263+0000 7fc5a8200700 1 -- 172.21.15.97:0/1757584152 wait complete. 2024-03-27T10:54:49.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.264+0000 7fc5a8200700 1 Processor -- start 2024-03-27T10:54:49.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.264+0000 7fc5a8200700 1 -- start start 2024-03-27T10:54:49.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.264+0000 7fc5a8200700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5a006a2f0 0x7fc5a019af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:49.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.264+0000 7fc5a8200700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc5a006ac10 0x7fc5a019b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:49.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.264+0000 7fc5a8200700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc5a010f0f0 0x7fc5a019f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:49.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.265+0000 7fc5a8200700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a0114a00 con 0x7fc5a006a2f0 2024-03-27T10:54:49.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.265+0000 7fc5a8200700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a0114880 con 0x7fc5a006ac10 2024-03-27T10:54:49.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.265+0000 7fc5a8200700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a0114b80 con 0x7fc5a010f0f0 2024-03-27T10:54:49.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.265+0000 7fc5a4e00700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc5a006ac10 0x7fc5a019b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:49.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.265+0000 7fc5a4e00700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc5a006ac10 0x7fc5a019b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:50302/0 (socket says 172.21.15.97:50302) 2024-03-27T10:54:49.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.265+0000 7fc5a4e00700 1 -- 172.21.15.97:0/2459267586 learned_addr learned my addr 172.21.15.97:0/2459267586 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:54:49.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.265+0000 7fc5a6200700 1 --2- 172.21.15.97:0/2459267586 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc5a010f0f0 0x7fc5a019f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:49.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.265+0000 7fc5a5800700 1 --2- 172.21.15.97:0/2459267586 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5a006a2f0 0x7fc5a019af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:49.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.265+0000 7fc5a4e00700 1 -- 172.21.15.97:0/2459267586 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc5a010f0f0 msgr2=0x7fc5a019f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:49.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.265+0000 7fc5a4e00700 1 --2- 172.21.15.97:0/2459267586 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc5a010f0f0 0x7fc5a019f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:49.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.265+0000 7fc5a4e00700 1 -- 172.21.15.97:0/2459267586 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5a006a2f0 msgr2=0x7fc5a019af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:49.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.265+0000 7fc5a4e00700 1 --2- 172.21.15.97:0/2459267586 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5a006a2f0 0x7fc5a019af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:49.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.265+0000 7fc5a4e00700 1 -- 172.21.15.97:0/2459267586 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5a019ff10 con 0x7fc5a006ac10 2024-03-27T10:54:49.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.265+0000 7fc5a5800700 1 --2- 172.21.15.97:0/2459267586 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5a006a2f0 0x7fc5a019af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:54:49.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.266+0000 7fc5a4e00700 1 --2- 172.21.15.97:0/2459267586 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc5a006ac10 0x7fc5a019b490 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc58c00ec10 tx=0x7fc58c00c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:54:49.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.266+0000 7fc59e000700 1 -- 172.21.15.97:0/2459267586 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc58c00cde0 con 0x7fc5a006ac10 2024-03-27T10:54:49.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.266+0000 7fc5a8200700 1 -- 172.21.15.97:0/2459267586 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5a01a0200 con 0x7fc5a006ac10 2024-03-27T10:54:49.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.266+0000 7fc59e000700 1 -- 172.21.15.97:0/2459267586 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc58c0044d0 con 0x7fc5a006ac10 2024-03-27T10:54:49.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.266+0000 7fc59e000700 1 -- 172.21.15.97:0/2459267586 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc58c010690 con 0x7fc5a006ac10 2024-03-27T10:54:49.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.266+0000 7fc5a8200700 1 -- 172.21.15.97:0/2459267586 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5a01a7a80 con 0x7fc5a006ac10 2024-03-27T10:54:49.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.267+0000 7fc5a8200700 1 -- 172.21.15.97:0/2459267586 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc5a0062dd0 con 0x7fc5a006ac10 2024-03-27T10:54:49.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.270+0000 7fc59e000700 1 -- 172.21.15.97:0/2459267586 <== mon.2 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 17) ==== 138421+0+0 (secure 0 0 0) 0x7fc58c003f00 con 0x7fc5a006ac10 2024-03-27T10:54:49.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.271+0000 7fc59e000700 1 --2- 172.21.15.97:0/2459267586 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fc5880a37b0 0x7fc5880a5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:49.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.271+0000 7fc59e000700 1 -- 172.21.15.97:0/2459267586 <== mon.2 v2:172.21.15.114:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fc58c0d0500 con 0x7fc5a006ac10 2024-03-27T10:54:49.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.271+0000 7fc5a5800700 1 --2- 172.21.15.97:0/2459267586 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fc5880a37b0 0x7fc5880a5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:49.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.271+0000 7fc59e000700 1 --2- 172.21.15.97:0/2459267586 >> [v2:172.21.15.114:6824/2893111461,v1:172.21.15.114:6825/2893111461] conn(0x7fc5880ad2f0 0x7fc5880af790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:49.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.272+0000 7fc59e000700 1 -- 172.21.15.97:0/2459267586 --> [v2:172.21.15.114:6824/2893111461,v1:172.21.15.114:6825/2893111461] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc58c022e50 con 0x7fc5880ad2f0 2024-03-27T10:54:49.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.272+0000 7fc59e000700 1 -- 172.21.15.97:0/2459267586 <== mon.2 v2:172.21.15.114:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7fc58c0d08b0 con 0x7fc5a006ac10 2024-03-27T10:54:49.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.272+0000 7fc5a6200700 1 --2- 172.21.15.97:0/2459267586 >> [v2:172.21.15.114:6824/2893111461,v1:172.21.15.114:6825/2893111461] conn(0x7fc5880ad2f0 0x7fc5880af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:49.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.272+0000 7fc5a5800700 1 --2- 172.21.15.97:0/2459267586 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fc5880a37b0 0x7fc5880a5c70 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc594009230 tx=0x7fc594005b40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:54:49.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.272+0000 7fc5a6200700 1 --2- 172.21.15.97:0/2459267586 >> [v2:172.21.15.114:6824/2893111461,v1:172.21.15.114:6825/2893111461] conn(0x7fc5880ad2f0 0x7fc5880af790 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:54:49.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.274+0000 7fc59e000700 1 -- 172.21.15.97:0/2459267586 <== osd.10 v2:172.21.15.114:6824/2893111461 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc58c022e50 con 0x7fc5880ad2f0 2024-03-27T10:54:49.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.292+0000 7fc5a8200700 1 -- 172.21.15.97:0/2459267586 --> [v2:172.21.15.114:6824/2893111461,v1:172.21.15.114:6825/2893111461] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc5a00631d0 con 0x7fc5880ad2f0 2024-03-27T10:54:49.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.292+0000 7fc59e000700 1 -- 172.21.15.97:0/2459267586 <== osd.10 v2:172.21.15.114:6824/2893111461 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fc5a00631d0 con 0x7fc5880ad2f0 2024-03-27T10:54:49.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.293+0000 7fc5a8200700 1 -- 172.21.15.97:0/2459267586 >> [v2:172.21.15.114:6824/2893111461,v1:172.21.15.114:6825/2893111461] conn(0x7fc5880ad2f0 msgr2=0x7fc5880af790 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:49.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.293+0000 7fc5a8200700 1 --2- 172.21.15.97:0/2459267586 >> [v2:172.21.15.114:6824/2893111461,v1:172.21.15.114:6825/2893111461] conn(0x7fc5880ad2f0 0x7fc5880af790 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:49.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.293+0000 7fc5a8200700 1 -- 172.21.15.97:0/2459267586 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fc5880a37b0 msgr2=0x7fc5880a5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:49.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.293+0000 7fc5a8200700 1 --2- 172.21.15.97:0/2459267586 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fc5880a37b0 0x7fc5880a5c70 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc594009230 tx=0x7fc594005b40 comp rx=0 tx=0).stop 2024-03-27T10:54:49.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.293+0000 7fc5a8200700 1 -- 172.21.15.97:0/2459267586 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc5a006ac10 msgr2=0x7fc5a019b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:49.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.293+0000 7fc5a8200700 1 --2- 172.21.15.97:0/2459267586 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc5a006ac10 0x7fc5a019b490 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc58c00ec10 tx=0x7fc58c00c2d0 comp rx=0 tx=0).stop 2024-03-27T10:54:49.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.293+0000 7fc5a6200700 1 -- 172.21.15.97:0/2459267586 reap_dead start 2024-03-27T10:54:49.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.293+0000 7fc5a8200700 1 -- 172.21.15.97:0/2459267586 shutdown_connections 2024-03-27T10:54:49.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.293+0000 7fc5a8200700 1 -- 172.21.15.97:0/2459267586 >> 172.21.15.97:0/2459267586 conn(0x7fc5a0078470 msgr2=0x7fc5a01108b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:54:49.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.293+0000 7fc5a8200700 1 -- 172.21.15.97:0/2459267586 shutdown_connections 2024-03-27T10:54:49.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.293+0000 7fc5a8200700 1 -- 172.21.15.97:0/2459267586 wait complete. 2024-03-27T10:54:49.566 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:49 smithi097 ceph-mon[31859]: pgmap v386: 97 pgs: 97 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s; 72 B/s, 0 objects/s recovering 2024-03-27T10:54:49.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:49 smithi177 ceph-mon[35294]: pgmap v386: 97 pgs: 97 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s; 72 B/s, 0 objects/s recovering 2024-03-27T10:54:49.767 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:49 smithi114 ceph-mon[35338]: pgmap v386: 97 pgs: 97 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s; 72 B/s, 0 objects/s recovering 2024-03-27T10:54:49.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:49.594+0000 7f4a2e400700 1 -- 172.21.15.97:0/3212748096 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a28107550 msgr2=0x7f4a28107930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:49.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:49.594+0000 7f4a2e400700 1 --2- 172.21.15.97:0/3212748096 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a28107550 0x7f4a28107930 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f4a18009990 tx=0x7f4a18009ca0 comp rx=0 tx=0).stop 2024-03-27T10:54:49.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:49.594+0000 7f4a2e400700 1 -- 172.21.15.97:0/3212748096 shutdown_connections 2024-03-27T10:54:49.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:49.594+0000 7f4a2e400700 1 --2- 172.21.15.97:0/3212748096 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4a28101f00 0x7f4a2810e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:49.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:49.594+0000 7f4a2e400700 1 --2- 172.21.15.97:0/3212748096 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a28107550 0x7f4a28107930 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:49.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:49.594+0000 7f4a2e400700 1 --2- 172.21.15.97:0/3212748096 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4a28101540 0x7f4a281019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:49.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:49.594+0000 7f4a2e400700 1 -- 172.21.15.97:0/3212748096 >> 172.21.15.97:0/3212748096 conn(0x7f4a28078470 msgr2=0x7f4a28078880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:54:49.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:49.594+0000 7f4a2e400700 1 -- 172.21.15.97:0/3212748096 shutdown_connections 2024-03-27T10:54:49.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:49.594+0000 7f4a2e400700 1 -- 172.21.15.97:0/3212748096 wait complete. 2024-03-27T10:54:49.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:49.595+0000 7f4a2e400700 1 Processor -- start 2024-03-27T10:54:49.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:49.595+0000 7f4a2e400700 1 -- start start 2024-03-27T10:54:49.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:49.596+0000 7f4a2e400700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4a28101540 0x7f4a2819ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:49.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:49.596+0000 7f4a2e400700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4a28101f00 0x7f4a2819ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:49.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:49.596+0000 7f4a2e400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a28107550 0x7f4a28198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:49.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:49.596+0000 7f4a2e400700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4a28111760 con 0x7f4a28107550 2024-03-27T10:54:49.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:49.596+0000 7f4a2e400700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f4a281115e0 con 0x7f4a28101f00 2024-03-27T10:54:49.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:49.596+0000 7f4a2e400700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f4a281118e0 con 0x7f4a28101540 2024-03-27T10:54:49.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:49.596+0000 7f4a27e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a28107550 0x7f4a28198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:49.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:49.596+0000 7f4a27e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a28107550 0x7f4a28198b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33954/0 (socket says 172.21.15.97:33954) 2024-03-27T10:54:49.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:49.596+0000 7f4a27e00700 1 -- 172.21.15.97:0/401876346 learned_addr learned my addr 172.21.15.97:0/401876346 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:54:49.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:49.596+0000 7f4a27400700 1 --2- 172.21.15.97:0/401876346 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4a28101540 0x7f4a2819ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:49.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:49.596+0000 7f4a1fe00700 1 --2- 172.21.15.97:0/401876346 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4a28101f00 0x7f4a2819ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:49.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:49.597+0000 7f4a27e00700 1 -- 172.21.15.97:0/401876346 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4a28101540 msgr2=0x7f4a2819ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:49.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:49.597+0000 7f4a27e00700 1 --2- 172.21.15.97:0/401876346 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4a28101540 0x7f4a2819ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:49.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:49.597+0000 7f4a27e00700 1 -- 172.21.15.97:0/401876346 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4a28101f00 msgr2=0x7f4a2819ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:49.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:49.597+0000 7f4a27e00700 1 --2- 172.21.15.97:0/401876346 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4a28101f00 0x7f4a2819ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:49.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:49.597+0000 7f4a27e00700 1 -- 172.21.15.97:0/401876346 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a28199400 con 0x7f4a28107550 2024-03-27T10:54:49.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:49.597+0000 7f4a1fe00700 1 --2- 172.21.15.97:0/401876346 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4a28101f00 0x7f4a2819ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T10:54:49.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:49.597+0000 7f4a27e00700 1 --2- 172.21.15.97:0/401876346 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a28107550 0x7f4a28198b20 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f4a2000ada0 tx=0x7f4a2000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:54:49.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:49.597+0000 7f4a24c00700 1 -- 172.21.15.97:0/401876346 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a20017020 con 0x7f4a28107550 2024-03-27T10:54:49.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:49.597+0000 7f4a2e400700 1 -- 172.21.15.97:0/401876346 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a281996f0 con 0x7f4a28107550 2024-03-27T10:54:49.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:49.597+0000 7f4a24c00700 1 -- 172.21.15.97:0/401876346 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4a2000ce30 con 0x7f4a28107550 2024-03-27T10:54:49.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:49.597+0000 7f4a24c00700 1 -- 172.21.15.97:0/401876346 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a20005460 con 0x7f4a28107550 2024-03-27T10:54:49.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:49.597+0000 7f4a2e400700 1 -- 172.21.15.97:0/401876346 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a281a59d0 con 0x7f4a28107550 2024-03-27T10:54:49.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:49.598+0000 7f4a2e400700 1 -- 172.21.15.97:0/401876346 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4a2806a660 con 0x7f4a28107550 2024-03-27T10:54:49.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:49.601+0000 7f4a24c00700 1 -- 172.21.15.97:0/401876346 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138421+0+0 (secure 0 0 0) 0x7f4a2000c980 con 0x7f4a28107550 2024-03-27T10:54:49.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:49.602+0000 7f4a24c00700 1 --2- 172.21.15.97:0/401876346 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f4a0c0a37b0 0x7f4a0c0a5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:49.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:49.602+0000 7f4a24c00700 1 -- 172.21.15.97:0/401876346 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f4a200d00c0 con 0x7f4a28107550 2024-03-27T10:54:49.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:49.602+0000 7f4a27400700 1 --2- 172.21.15.97:0/401876346 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f4a0c0a37b0 0x7f4a0c0a5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:49.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:49.602+0000 7f4a24c00700 1 --2- 172.21.15.97:0/401876346 >> [v2:172.21.15.177:6808/1918493790,v1:172.21.15.177:6809/1918493790] conn(0x7f4a0c0ad350 0x7f4a0c0af7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:49.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:49.602+0000 7f4a24c00700 1 -- 172.21.15.97:0/401876346 --> [v2:172.21.15.177:6808/1918493790,v1:172.21.15.177:6809/1918493790] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4a20019e50 con 0x7f4a0c0ad350 2024-03-27T10:54:49.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:49.602+0000 7f4a24c00700 1 -- 172.21.15.97:0/401876346 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f4a200d0510 con 0x7f4a28107550 2024-03-27T10:54:49.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:49.603+0000 7f4a27400700 1 --2- 172.21.15.97:0/401876346 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f4a0c0a37b0 0x7f4a0c0a5c70 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4a18009750 tx=0x7f4a1801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:54:49.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:49.603+0000 7f4a1fe00700 1 --2- 172.21.15.97:0/401876346 >> [v2:172.21.15.177:6808/1918493790,v1:172.21.15.177:6809/1918493790] conn(0x7f4a0c0ad350 0x7f4a0c0af7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:49.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:49.604+0000 7f4a1fe00700 1 --2- 172.21.15.97:0/401876346 >> [v2:172.21.15.177:6808/1918493790,v1:172.21.15.177:6809/1918493790] conn(0x7f4a0c0ad350 0x7f4a0c0af7f0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:54:49.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:49.606+0000 7f4a24c00700 1 -- 172.21.15.97:0/401876346 <== osd.5 v2:172.21.15.177:6808/1918493790 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4a20019e50 con 0x7f4a0c0ad350 2024-03-27T10:54:49.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:49.624+0000 7f4a2e400700 1 -- 172.21.15.97:0/401876346 --> [v2:172.21.15.177:6808/1918493790,v1:172.21.15.177:6809/1918493790] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4a28102c30 con 0x7f4a0c0ad350 2024-03-27T10:54:49.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:49.625+0000 7f4a24c00700 1 -- 172.21.15.97:0/401876346 <== osd.5 v2:172.21.15.177:6808/1918493790 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f4a28102c30 con 0x7f4a0c0ad350 2024-03-27T10:54:49.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:49.626+0000 7f4a2e400700 1 -- 172.21.15.97:0/401876346 >> [v2:172.21.15.177:6808/1918493790,v1:172.21.15.177:6809/1918493790] conn(0x7f4a0c0ad350 msgr2=0x7f4a0c0af7f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:49.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:49.626+0000 7f4a2e400700 1 --2- 172.21.15.97:0/401876346 >> [v2:172.21.15.177:6808/1918493790,v1:172.21.15.177:6809/1918493790] conn(0x7f4a0c0ad350 0x7f4a0c0af7f0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:49.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:49.626+0000 7f4a2e400700 1 -- 172.21.15.97:0/401876346 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f4a0c0a37b0 msgr2=0x7f4a0c0a5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:49.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:49.626+0000 7f4a2e400700 1 --2- 172.21.15.97:0/401876346 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f4a0c0a37b0 0x7f4a0c0a5c70 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4a18009750 tx=0x7f4a1801a040 comp rx=0 tx=0).stop 2024-03-27T10:54:49.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:49.626+0000 7f4a2e400700 1 -- 172.21.15.97:0/401876346 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a28107550 msgr2=0x7f4a28198b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:49.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:49.626+0000 7f4a2e400700 1 --2- 172.21.15.97:0/401876346 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a28107550 0x7f4a28198b20 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f4a2000ada0 tx=0x7f4a2000c330 comp rx=0 tx=0).stop 2024-03-27T10:54:49.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:49.626+0000 7f4a27e00700 1 -- 172.21.15.97:0/401876346 reap_dead start 2024-03-27T10:54:49.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:49.626+0000 7f4a2e400700 1 -- 172.21.15.97:0/401876346 shutdown_connections 2024-03-27T10:54:49.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:49.626+0000 7f4a2e400700 1 -- 172.21.15.97:0/401876346 >> 172.21.15.97:0/401876346 conn(0x7f4a28078470 msgr2=0x7f4a281008d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:54:49.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:49.626+0000 7f4a2e400700 1 -- 172.21.15.97:0/401876346 shutdown_connections 2024-03-27T10:54:49.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:49.626+0000 7f4a2e400700 1 -- 172.21.15.97:0/401876346 wait complete. 2024-03-27T10:54:50.641 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:50 smithi097 ceph-mon[31859]: pgmap v387: 97 pgs: 97 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 209 B/s wr, 10 op/s; 59 B/s, 0 objects/s recovering 2024-03-27T10:54:50.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:50 smithi177 ceph-mon[35294]: pgmap v387: 97 pgs: 97 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 209 B/s wr, 10 op/s; 59 B/s, 0 objects/s recovering 2024-03-27T10:54:50.767 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:50 smithi114 ceph-mon[35338]: pgmap v387: 97 pgs: 97 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 209 B/s wr, 10 op/s; 59 B/s, 0 objects/s recovering 2024-03-27T10:54:51.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:51.209+0000 7f5ffd800700 1 -- 172.21.15.97:0/3126418118 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ff8109860 msgr2=0x7f5ff810d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:51.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:51.209+0000 7f5ffd800700 1 --2- 172.21.15.97:0/3126418118 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ff8109860 0x7f5ff810d8b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f5fe40099c0 tx=0x7f5fe4009cd0 comp rx=0 tx=0).stop 2024-03-27T10:54:51.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:51.209+0000 7f5ffd800700 1 -- 172.21.15.97:0/3126418118 shutdown_connections 2024-03-27T10:54:51.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:51.209+0000 7f5ffd800700 1 --2- 172.21.15.97:0/3126418118 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5ff810df90 0x7f5ff8115b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:51.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:51.209+0000 7f5ffd800700 1 --2- 172.21.15.97:0/3126418118 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5ff8108eb0 0x7f5ff8109290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:51.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:51.209+0000 7f5ffd800700 1 --2- 172.21.15.97:0/3126418118 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ff8109860 0x7f5ff810d8b0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:51.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:51.209+0000 7f5ffd800700 1 -- 172.21.15.97:0/3126418118 >> 172.21.15.97:0/3126418118 conn(0x7f5ff8075e10 msgr2=0x7f5ff8078230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:54:51.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:51.209+0000 7f5ffd800700 1 -- 172.21.15.97:0/3126418118 shutdown_connections 2024-03-27T10:54:51.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:51.209+0000 7f5ffd800700 1 -- 172.21.15.97:0/3126418118 wait complete. 2024-03-27T10:54:51.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:51.210+0000 7f5ffd800700 1 Processor -- start 2024-03-27T10:54:51.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:51.210+0000 7f5ffd800700 1 -- start start 2024-03-27T10:54:51.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:51.211+0000 7f5ffd800700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5ff8108eb0 0x7f5ff819f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:51.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:51.211+0000 7f5ffd800700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5ff8109860 0x7f5ff819f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:51.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:51.211+0000 7f5ffd800700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ff810df90 0x7f5ff81a3c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:51.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:51.211+0000 7f5ffd800700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff8118e70 con 0x7f5ff810df90 2024-03-27T10:54:51.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:51.211+0000 7f5ffd800700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff8118cf0 con 0x7f5ff8109860 2024-03-27T10:54:51.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:51.211+0000 7f5ffd800700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff8118ff0 con 0x7f5ff8108eb0 2024-03-27T10:54:51.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:51.211+0000 7f5ff7400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ff810df90 0x7f5ff81a3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:51.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:51.211+0000 7f5ff6000700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5ff8109860 0x7f5ff819f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:51.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:51.211+0000 7f5ff7400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ff810df90 0x7f5ff81a3c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33974/0 (socket says 172.21.15.97:33974) 2024-03-27T10:54:51.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:51.211+0000 7f5ff7400700 1 -- 172.21.15.97:0/168668005 learned_addr learned my addr 172.21.15.97:0/168668005 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:54:51.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:51.211+0000 7f5ff6000700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5ff8109860 0x7f5ff819f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:39560/0 (socket says 172.21.15.97:39560) 2024-03-27T10:54:51.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:51.211+0000 7f5ff6a00700 1 --2- 172.21.15.97:0/168668005 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5ff8108eb0 0x7f5ff819f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:51.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:51.212+0000 7f5ff7400700 1 -- 172.21.15.97:0/168668005 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5ff8108eb0 msgr2=0x7f5ff819f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:51.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:51.212+0000 7f5ff7400700 1 --2- 172.21.15.97:0/168668005 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5ff8108eb0 0x7f5ff819f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:51.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:51.212+0000 7f5ff7400700 1 -- 172.21.15.97:0/168668005 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5ff8109860 msgr2=0x7f5ff819f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:51.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:51.212+0000 7f5ff7400700 1 --2- 172.21.15.97:0/168668005 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5ff8109860 0x7f5ff819f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:51.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:51.212+0000 7f5ff7400700 1 -- 172.21.15.97:0/168668005 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ff81a4370 con 0x7f5ff810df90 2024-03-27T10:54:51.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:51.212+0000 7f5ff7400700 1 --2- 172.21.15.97:0/168668005 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ff810df90 0x7f5ff81a3c30 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f5ff0007ee0 tx=0x7f5ff000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:54:51.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:51.213+0000 7f5fef400700 1 -- 172.21.15.97:0/168668005 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff0004bb0 con 0x7f5ff810df90 2024-03-27T10:54:51.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:51.213+0000 7f5ffd800700 1 -- 172.21.15.97:0/168668005 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ff81a4660 con 0x7f5ff810df90 2024-03-27T10:54:51.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:51.213+0000 7f5fef400700 1 -- 172.21.15.97:0/168668005 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5ff0004d50 con 0x7f5ff810df90 2024-03-27T10:54:51.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:51.213+0000 7f5fef400700 1 -- 172.21.15.97:0/168668005 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff00058e0 con 0x7f5ff810df90 2024-03-27T10:54:51.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:51.213+0000 7f5ffd800700 1 -- 172.21.15.97:0/168668005 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ff81abee0 con 0x7f5ff810df90 2024-03-27T10:54:51.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:51.214+0000 7f5ffd800700 1 -- 172.21.15.97:0/168668005 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5ff8062d50 con 0x7f5ff810df90 2024-03-27T10:54:51.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:51.216+0000 7f5fef400700 1 -- 172.21.15.97:0/168668005 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138421+0+0 (secure 0 0 0) 0x7f5ff001a020 con 0x7f5ff810df90 2024-03-27T10:54:51.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:51.217+0000 7f5fef400700 1 --2- 172.21.15.97:0/168668005 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f5fdc0a3730 0x7f5fdc0a5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:51.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:51.217+0000 7f5fef400700 1 -- 172.21.15.97:0/168668005 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f5ff00cfc70 con 0x7f5ff810df90 2024-03-27T10:54:51.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:51.218+0000 7f5ff6a00700 1 --2- 172.21.15.97:0/168668005 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f5fdc0a3730 0x7f5fdc0a5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:51.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:51.218+0000 7f5fef400700 1 --2- 172.21.15.97:0/168668005 >> [v2:172.21.15.177:6824/2333030391,v1:172.21.15.177:6825/2333030391] conn(0x7f5fdc0ad270 0x7f5fdc0af710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:51.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:51.218+0000 7f5fef400700 1 -- 172.21.15.97:0/168668005 --> [v2:172.21.15.177:6824/2333030391,v1:172.21.15.177:6825/2333030391] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5fdc0afde0 con 0x7f5fdc0ad270 2024-03-27T10:54:51.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:51.218+0000 7f5fef400700 1 -- 172.21.15.97:0/168668005 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f5ff00d0020 con 0x7f5ff810df90 2024-03-27T10:54:51.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:51.218+0000 7f5ff6a00700 1 --2- 172.21.15.97:0/168668005 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f5fdc0a3730 0x7f5fdc0a5bf0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5fe8004630 tx=0x7f5fe800b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:54:51.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:51.218+0000 7f5ff6000700 1 --2- 172.21.15.97:0/168668005 >> [v2:172.21.15.177:6824/2333030391,v1:172.21.15.177:6825/2333030391] conn(0x7f5fdc0ad270 0x7f5fdc0af710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:51.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:51.219+0000 7f5ff6000700 1 --2- 172.21.15.97:0/168668005 >> [v2:172.21.15.177:6824/2333030391,v1:172.21.15.177:6825/2333030391] conn(0x7f5fdc0ad270 0x7f5fdc0af710 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:54:51.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:51.222+0000 7f5fef400700 1 -- 172.21.15.97:0/168668005 <== osd.11 v2:172.21.15.177:6824/2333030391 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5fdc0afde0 con 0x7f5fdc0ad270 2024-03-27T10:54:51.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:51.240+0000 7f5ffd800700 1 -- 172.21.15.97:0/168668005 --> [v2:172.21.15.177:6824/2333030391,v1:172.21.15.177:6825/2333030391] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5ff80631d0 con 0x7f5fdc0ad270 2024-03-27T10:54:51.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:51.241+0000 7f5fef400700 1 -- 172.21.15.97:0/168668005 <== osd.11 v2:172.21.15.177:6824/2333030391 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f5ff80631d0 con 0x7f5fdc0ad270 2024-03-27T10:54:51.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:51.242+0000 7f5ffd800700 1 -- 172.21.15.97:0/168668005 >> [v2:172.21.15.177:6824/2333030391,v1:172.21.15.177:6825/2333030391] conn(0x7f5fdc0ad270 msgr2=0x7f5fdc0af710 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:51.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:51.242+0000 7f5ffd800700 1 --2- 172.21.15.97:0/168668005 >> [v2:172.21.15.177:6824/2333030391,v1:172.21.15.177:6825/2333030391] conn(0x7f5fdc0ad270 0x7f5fdc0af710 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:51.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:51.242+0000 7f5ffd800700 1 -- 172.21.15.97:0/168668005 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f5fdc0a3730 msgr2=0x7f5fdc0a5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:51.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:51.242+0000 7f5ffd800700 1 --2- 172.21.15.97:0/168668005 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f5fdc0a3730 0x7f5fdc0a5bf0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5fe8004630 tx=0x7f5fe800b3c0 comp rx=0 tx=0).stop 2024-03-27T10:54:51.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:51.242+0000 7f5ffd800700 1 -- 172.21.15.97:0/168668005 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ff810df90 msgr2=0x7f5ff81a3c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:51.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:51.242+0000 7f5ffd800700 1 --2- 172.21.15.97:0/168668005 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ff810df90 0x7f5ff81a3c30 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f5ff0007ee0 tx=0x7f5ff000d3b0 comp rx=0 tx=0).stop 2024-03-27T10:54:51.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:51.242+0000 7f5ff7400700 1 -- 172.21.15.97:0/168668005 reap_dead start 2024-03-27T10:54:51.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:51.242+0000 7f5ffd800700 1 -- 172.21.15.97:0/168668005 shutdown_connections 2024-03-27T10:54:51.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:51.242+0000 7f5ffd800700 1 -- 172.21.15.97:0/168668005 >> 172.21.15.97:0/168668005 conn(0x7f5ff8075e10 msgr2=0x7f5ff8078140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:54:51.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:51.243+0000 7f5ffd800700 1 -- 172.21.15.97:0/168668005 shutdown_connections 2024-03-27T10:54:51.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:51.243+0000 7f5ffd800700 1 -- 172.21.15.97:0/168668005 wait complete. 2024-03-27T10:54:51.761 INFO:teuthology.orchestra.run.smithi097.stdout:55834574941 2024-03-27T10:54:51.761 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell --fsid adca88c4-ec26-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-03-27T10:54:53.222 INFO:teuthology.orchestra.run.smithi097.stdout:115964117072 2024-03-27T10:54:53.222 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell --fsid adca88c4-ec26-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.3 2024-03-27T10:54:53.380 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:53 smithi177 ceph-mon[35294]: pgmap v388: 97 pgs: 97 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 190 B/s wr, 9 op/s; 54 B/s, 0 objects/s recovering 2024-03-27T10:54:53.390 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:53 smithi097 ceph-mon[31859]: pgmap v388: 97 pgs: 97 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 190 B/s wr, 9 op/s; 54 B/s, 0 objects/s recovering 2024-03-27T10:54:53.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:53 smithi114 ceph-mon[35338]: pgmap v388: 97 pgs: 97 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 190 B/s wr, 9 op/s; 54 B/s, 0 objects/s recovering 2024-03-27T10:54:53.551 INFO:teuthology.orchestra.run.smithi097.stdout:176093659202 2024-03-27T10:54:53.551 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell --fsid adca88c4-ec26-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.5 2024-03-27T10:54:53.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.384+0000 7f035aa00700 1 -- 172.21.15.97:0/2976478824 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f035410f040 msgr2=0x7f0354111430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:53.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.384+0000 7f035aa00700 1 --2- 172.21.15.97:0/2976478824 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f035410f040 0x7f0354111430 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f034c00b0a0 tx=0x7f034c00b3b0 comp rx=0 tx=0).stop 2024-03-27T10:54:53.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.384+0000 7f035aa00700 1 -- 172.21.15.97:0/2976478824 shutdown_connections 2024-03-27T10:54:53.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.384+0000 7f035aa00700 1 --2- 172.21.15.97:0/2976478824 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f035410f040 0x7f0354111430 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:53.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.384+0000 7f035aa00700 1 --2- 172.21.15.97:0/2976478824 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0354074870 0x7f0354074c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:53.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.384+0000 7f035aa00700 1 --2- 172.21.15.97:0/2976478824 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0354075190 0x7f035410eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:53.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.385+0000 7f035aa00700 1 -- 172.21.15.97:0/2976478824 >> 172.21.15.97:0/2976478824 conn(0x7f0354079770 msgr2=0x7f0354079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:54:53.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.385+0000 7f035aa00700 1 -- 172.21.15.97:0/2976478824 shutdown_connections 2024-03-27T10:54:53.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.385+0000 7f035aa00700 1 -- 172.21.15.97:0/2976478824 wait complete. 2024-03-27T10:54:53.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.386+0000 7f035aa00700 1 Processor -- start 2024-03-27T10:54:53.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.386+0000 7f035aa00700 1 -- start start 2024-03-27T10:54:53.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.386+0000 7f035aa00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0354074870 0x7f035419af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:53.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.386+0000 7f035aa00700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0354075190 0x7f035419b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:53.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.386+0000 7f035aa00700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f035410f040 0x7f035419f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:53.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.386+0000 7f035aa00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0354114a00 con 0x7f0354074870 2024-03-27T10:54:53.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.386+0000 7f035aa00700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f0354114880 con 0x7f0354075190 2024-03-27T10:54:53.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.386+0000 7f035aa00700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f0354114b80 con 0x7f035410f040 2024-03-27T10:54:53.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.387+0000 7f0353e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0354074870 0x7f035419af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:53.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.387+0000 7f0353e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0354074870 0x7f035419af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45498/0 (socket says 172.21.15.97:45498) 2024-03-27T10:54:53.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.387+0000 7f0353e00700 1 -- 172.21.15.97:0/3607773686 learned_addr learned my addr 172.21.15.97:0/3607773686 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:54:53.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.387+0000 7f0353400700 1 --2- 172.21.15.97:0/3607773686 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0354075190 0x7f035419b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:53.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.387+0000 7f0353e00700 1 -- 172.21.15.97:0/3607773686 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f035410f040 msgr2=0x7f035419f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:53.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.387+0000 7f0358a00700 1 --2- 172.21.15.97:0/3607773686 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f035410f040 0x7f035419f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:53.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.387+0000 7f0353e00700 1 --2- 172.21.15.97:0/3607773686 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f035410f040 0x7f035419f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:53.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.387+0000 7f0353e00700 1 -- 172.21.15.97:0/3607773686 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0354075190 msgr2=0x7f035419b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:53.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.387+0000 7f0353e00700 1 --2- 172.21.15.97:0/3607773686 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0354075190 0x7f035419b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:53.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.387+0000 7f0353e00700 1 -- 172.21.15.97:0/3607773686 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f035419ff60 con 0x7f0354074870 2024-03-27T10:54:53.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.387+0000 7f0353400700 1 --2- 172.21.15.97:0/3607773686 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0354075190 0x7f035419b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:54:53.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.387+0000 7f0358a00700 1 --2- 172.21.15.97:0/3607773686 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f035410f040 0x7f035419f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:54:53.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.387+0000 7f0353e00700 1 --2- 172.21.15.97:0/3607773686 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0354074870 0x7f035419af50 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f034400d950 tx=0x7f034400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:54:53.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.388+0000 7f0350c00700 1 -- 172.21.15.97:0/3607773686 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0344016070 con 0x7f0354074870 2024-03-27T10:54:53.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.388+0000 7f0350c00700 1 -- 172.21.15.97:0/3607773686 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0344010460 con 0x7f0354074870 2024-03-27T10:54:53.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.388+0000 7f035aa00700 1 -- 172.21.15.97:0/3607773686 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03541a0250 con 0x7f0354074870 2024-03-27T10:54:53.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.388+0000 7f0350c00700 1 -- 172.21.15.97:0/3607773686 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f034400f620 con 0x7f0354074870 2024-03-27T10:54:53.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.388+0000 7f035aa00700 1 -- 172.21.15.97:0/3607773686 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03541a7ad0 con 0x7f0354074870 2024-03-27T10:54:53.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.389+0000 7f035aa00700 1 -- 172.21.15.97:0/3607773686 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0354062d50 con 0x7f0354074870 2024-03-27T10:54:53.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.391+0000 7f0350c00700 1 -- 172.21.15.97:0/3607773686 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138421+0+0 (secure 0 0 0) 0x7f0344019070 con 0x7f0354074870 2024-03-27T10:54:53.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.392+0000 7f0350c00700 1 --2- 172.21.15.97:0/3607773686 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f03380a3730 0x7f03380a5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:53.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.392+0000 7f0350c00700 1 -- 172.21.15.97:0/3607773686 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f03440d0080 con 0x7f0354074870 2024-03-27T10:54:53.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.392+0000 7f0353400700 1 --2- 172.21.15.97:0/3607773686 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f03380a3730 0x7f03380a5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:53.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.393+0000 7f0350c00700 1 --2- 172.21.15.97:0/3607773686 >> [v2:172.21.15.114:6816/2445491153,v1:172.21.15.114:6817/2445491153] conn(0x7f03380ad270 0x7f03380af710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:53.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.393+0000 7f0350c00700 1 -- 172.21.15.97:0/3607773686 --> [v2:172.21.15.114:6816/2445491153,v1:172.21.15.114:6817/2445491153] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0344005690 con 0x7f03380ad270 2024-03-27T10:54:53.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.393+0000 7f0350c00700 1 -- 172.21.15.97:0/3607773686 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f03440d0430 con 0x7f0354074870 2024-03-27T10:54:53.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.393+0000 7f0353400700 1 --2- 172.21.15.97:0/3607773686 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f03380a3730 0x7f03380a5bf0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0354110e40 tx=0x7f0340009550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:54:53.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.393+0000 7f0358a00700 1 --2- 172.21.15.97:0/3607773686 >> [v2:172.21.15.114:6816/2445491153,v1:172.21.15.114:6817/2445491153] conn(0x7f03380ad270 0x7f03380af710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:53.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.394+0000 7f0358a00700 1 --2- 172.21.15.97:0/3607773686 >> [v2:172.21.15.114:6816/2445491153,v1:172.21.15.114:6817/2445491153] conn(0x7f03380ad270 0x7f03380af710 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:54:53.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.396+0000 7f0350c00700 1 -- 172.21.15.97:0/3607773686 <== osd.7 v2:172.21.15.114:6816/2445491153 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0344005690 con 0x7f03380ad270 2024-03-27T10:54:53.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.414+0000 7f035aa00700 1 -- 172.21.15.97:0/3607773686 --> [v2:172.21.15.114:6816/2445491153,v1:172.21.15.114:6817/2445491153] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f03540631d0 con 0x7f03380ad270 2024-03-27T10:54:53.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.414+0000 7f0350c00700 1 -- 172.21.15.97:0/3607773686 <== osd.7 v2:172.21.15.114:6816/2445491153 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f03540631d0 con 0x7f03380ad270 2024-03-27T10:54:53.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.415+0000 7f035aa00700 1 -- 172.21.15.97:0/3607773686 >> [v2:172.21.15.114:6816/2445491153,v1:172.21.15.114:6817/2445491153] conn(0x7f03380ad270 msgr2=0x7f03380af710 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:53.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.415+0000 7f035aa00700 1 --2- 172.21.15.97:0/3607773686 >> [v2:172.21.15.114:6816/2445491153,v1:172.21.15.114:6817/2445491153] conn(0x7f03380ad270 0x7f03380af710 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:53.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.415+0000 7f035aa00700 1 -- 172.21.15.97:0/3607773686 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f03380a3730 msgr2=0x7f03380a5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:53.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.415+0000 7f035aa00700 1 --2- 172.21.15.97:0/3607773686 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f03380a3730 0x7f03380a5bf0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0354110e40 tx=0x7f0340009550 comp rx=0 tx=0).stop 2024-03-27T10:54:53.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.415+0000 7f035aa00700 1 -- 172.21.15.97:0/3607773686 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0354074870 msgr2=0x7f035419af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:53.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.415+0000 7f035aa00700 1 --2- 172.21.15.97:0/3607773686 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0354074870 0x7f035419af50 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f034400d950 tx=0x7f034400dd10 comp rx=0 tx=0).stop 2024-03-27T10:54:53.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.415+0000 7f0358a00700 1 -- 172.21.15.97:0/3607773686 reap_dead start 2024-03-27T10:54:53.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.415+0000 7f035aa00700 1 -- 172.21.15.97:0/3607773686 shutdown_connections 2024-03-27T10:54:53.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.415+0000 7f035aa00700 1 -- 172.21.15.97:0/3607773686 >> 172.21.15.97:0/3607773686 conn(0x7f0354079770 msgr2=0x7f03540ff910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:54:53.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.415+0000 7f035aa00700 1 -- 172.21.15.97:0/3607773686 shutdown_connections 2024-03-27T10:54:53.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:48.416+0000 7f035aa00700 1 -- 172.21.15.97:0/3607773686 wait complete. 2024-03-27T10:54:55.141 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:55 smithi097 ceph-mon[31859]: pgmap v389: 97 pgs: 97 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 170 B/s wr, 11 op/s 2024-03-27T10:54:55.426 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:55 smithi177 ceph-mon[35294]: pgmap v389: 97 pgs: 97 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 170 B/s wr, 11 op/s 2024-03-27T10:54:55.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:55 smithi114 ceph-mon[35338]: pgmap v389: 97 pgs: 97 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 170 B/s wr, 11 op/s 2024-03-27T10:54:56.020 INFO:teuthology.orchestra.run.smithi097.stdout:330712481824 2024-03-27T10:54:56.020 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell --fsid adca88c4-ec26-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.10 2024-03-27T10:54:56.502 INFO:teuthology.orchestra.run.smithi097.stdout:304942678054 2024-03-27T10:54:56.502 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell --fsid adca88c4-ec26-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.9 2024-03-27T10:54:56.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:55.751+0000 7fa184400700 1 -- 172.21.15.97:0/1361246372 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa17c1085f0 msgr2=0x7fa17c1089d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:56.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:55.751+0000 7fa184400700 1 --2- 172.21.15.97:0/1361246372 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa17c1085f0 0x7fa17c1089d0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa17000b0d0 tx=0x7fa17000b3e0 comp rx=0 tx=0).stop 2024-03-27T10:54:56.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:55.751+0000 7fa184400700 1 -- 172.21.15.97:0/1361246372 shutdown_connections 2024-03-27T10:54:56.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:55.751+0000 7fa184400700 1 --2- 172.21.15.97:0/1361246372 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa17c06b540 0x7fa17c10f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:56.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:55.751+0000 7fa184400700 1 --2- 172.21.15.97:0/1361246372 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa17c1085f0 0x7fa17c1089d0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:56.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:55.751+0000 7fa184400700 1 --2- 172.21.15.97:0/1361246372 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa17c06ab80 0x7fa17c06b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:56.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:55.751+0000 7fa184400700 1 -- 172.21.15.97:0/1361246372 >> 172.21.15.97:0/1361246372 conn(0x7fa17c077590 msgr2=0x7fa17c0779a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:54:56.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:55.751+0000 7fa184400700 1 -- 172.21.15.97:0/1361246372 shutdown_connections 2024-03-27T10:54:56.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:55.751+0000 7fa184400700 1 -- 172.21.15.97:0/1361246372 wait complete. 2024-03-27T10:54:56.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:55.753+0000 7fa184400700 1 Processor -- start 2024-03-27T10:54:56.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:55.753+0000 7fa184400700 1 -- start start 2024-03-27T10:54:56.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:55.754+0000 7fa184400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa17c06ab80 0x7fa17c101640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:56.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:55.754+0000 7fa184400700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa17c06b540 0x7fa17c101b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:56.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:55.754+0000 7fa184400700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa17c1085f0 0x7fa17c1020c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:56.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:55.754+0000 7fa184400700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa17c112830 con 0x7fa17c06ab80 2024-03-27T10:54:56.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:55.754+0000 7fa184400700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa17c1126b0 con 0x7fa17c1085f0 2024-03-27T10:54:56.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:55.754+0000 7fa184400700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa17c1129b0 con 0x7fa17c06b540 2024-03-27T10:54:56.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:55.754+0000 7fa181a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa17c06ab80 0x7fa17c101640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:56.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:55.754+0000 7fa181a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa17c06ab80 0x7fa17c101640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34002/0 (socket says 172.21.15.97:34002) 2024-03-27T10:54:56.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:55.754+0000 7fa181a00700 1 -- 172.21.15.97:0/2031147667 learned_addr learned my addr 172.21.15.97:0/2031147667 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:54:56.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:55.754+0000 7fa181000700 1 --2- 172.21.15.97:0/2031147667 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa17c06b540 0x7fa17c101b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:56.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:55.754+0000 7fa182400700 1 --2- 172.21.15.97:0/2031147667 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa17c1085f0 0x7fa17c1020c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:56.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:55.754+0000 7fa181a00700 1 -- 172.21.15.97:0/2031147667 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa17c06b540 msgr2=0x7fa17c101b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:56.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:55.754+0000 7fa181a00700 1 --2- 172.21.15.97:0/2031147667 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa17c06b540 0x7fa17c101b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:56.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:55.754+0000 7fa181a00700 1 -- 172.21.15.97:0/2031147667 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa17c1085f0 msgr2=0x7fa17c1020c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:56.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:55.754+0000 7fa181a00700 1 --2- 172.21.15.97:0/2031147667 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa17c1085f0 0x7fa17c1020c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:56.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:55.754+0000 7fa181a00700 1 -- 172.21.15.97:0/2031147667 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa17c1a72f0 con 0x7fa17c06ab80 2024-03-27T10:54:56.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:55.754+0000 7fa182400700 1 --2- 172.21.15.97:0/2031147667 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa17c1085f0 0x7fa17c1020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:54:56.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:55.754+0000 7fa181000700 1 --2- 172.21.15.97:0/2031147667 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa17c06b540 0x7fa17c101b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:54:56.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:55.755+0000 7fa181a00700 1 --2- 172.21.15.97:0/2031147667 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa17c06ab80 0x7fa17c101640 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa170009fd0 tx=0x7fa170007ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:54:56.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:55.755+0000 7fa17a000700 1 -- 172.21.15.97:0/2031147667 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa170021020 con 0x7fa17c06ab80 2024-03-27T10:54:56.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:55.755+0000 7fa17a000700 1 -- 172.21.15.97:0/2031147667 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa170004c50 con 0x7fa17c06ab80 2024-03-27T10:54:56.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:55.755+0000 7fa184400700 1 -- 172.21.15.97:0/2031147667 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa17c1a7580 con 0x7fa17c06ab80 2024-03-27T10:54:56.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:55.755+0000 7fa17a000700 1 -- 172.21.15.97:0/2031147667 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa170005340 con 0x7fa17c06ab80 2024-03-27T10:54:56.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:55.755+0000 7fa184400700 1 -- 172.21.15.97:0/2031147667 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa17c1a7a80 con 0x7fa17c06ab80 2024-03-27T10:54:56.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:55.756+0000 7fa184400700 1 -- 172.21.15.97:0/2031147667 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa17c062d50 con 0x7fa17c06ab80 2024-03-27T10:54:56.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:55.758+0000 7fa17a000700 1 -- 172.21.15.97:0/2031147667 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138421+0+0 (secure 0 0 0) 0x7fa170024020 con 0x7fa17c06ab80 2024-03-27T10:54:56.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:55.760+0000 7fa17a000700 1 --2- 172.21.15.97:0/2031147667 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fa1640a37b0 0x7fa1640a5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:56.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:55.760+0000 7fa17a000700 1 -- 172.21.15.97:0/2031147667 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fa17000f020 con 0x7fa17c06ab80 2024-03-27T10:54:56.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:55.760+0000 7fa181000700 1 --2- 172.21.15.97:0/2031147667 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fa1640a37b0 0x7fa1640a5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:56.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:55.760+0000 7fa17a000700 1 --2- 172.21.15.97:0/2031147667 >> [v2:172.21.15.177:6816/2051134191,v1:172.21.15.177:6817/2051134191] conn(0x7fa1640ad2f0 0x7fa1640af790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:56.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:55.760+0000 7fa17a000700 1 -- 172.21.15.97:0/2031147667 --> [v2:172.21.15.177:6816/2051134191,v1:172.21.15.177:6817/2051134191] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa17001ae50 con 0x7fa1640ad2f0 2024-03-27T10:54:56.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:55.760+0000 7fa17a000700 1 -- 172.21.15.97:0/2031147667 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7fa1700d1730 con 0x7fa17c06ab80 2024-03-27T10:54:56.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:55.760+0000 7fa181000700 1 --2- 172.21.15.97:0/2031147667 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fa1640a37b0 0x7fa1640a5c70 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa168005fd0 tx=0x7fa168005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:54:56.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:55.768+0000 7fa182400700 1 --2- 172.21.15.97:0/2031147667 >> [v2:172.21.15.177:6816/2051134191,v1:172.21.15.177:6817/2051134191] conn(0x7fa1640ad2f0 0x7fa1640af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:56.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:55.769+0000 7fa182400700 1 --2- 172.21.15.97:0/2031147667 >> [v2:172.21.15.177:6816/2051134191,v1:172.21.15.177:6817/2051134191] conn(0x7fa1640ad2f0 0x7fa1640af790 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:54:56.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:55.771+0000 7fa17a000700 1 -- 172.21.15.97:0/2031147667 <== osd.8 v2:172.21.15.177:6816/2051134191 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa17001ae50 con 0x7fa1640ad2f0 2024-03-27T10:54:56.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:55.790+0000 7fa184400700 1 -- 172.21.15.97:0/2031147667 --> [v2:172.21.15.177:6816/2051134191,v1:172.21.15.177:6817/2051134191] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa17c063150 con 0x7fa1640ad2f0 2024-03-27T10:54:56.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:55.791+0000 7fa17a000700 1 -- 172.21.15.97:0/2031147667 <== osd.8 v2:172.21.15.177:6816/2051134191 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa17c063150 con 0x7fa1640ad2f0 2024-03-27T10:54:56.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:55.792+0000 7fa184400700 1 -- 172.21.15.97:0/2031147667 >> [v2:172.21.15.177:6816/2051134191,v1:172.21.15.177:6817/2051134191] conn(0x7fa1640ad2f0 msgr2=0x7fa1640af790 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:56.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:55.792+0000 7fa184400700 1 --2- 172.21.15.97:0/2031147667 >> [v2:172.21.15.177:6816/2051134191,v1:172.21.15.177:6817/2051134191] conn(0x7fa1640ad2f0 0x7fa1640af790 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:56.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:55.792+0000 7fa184400700 1 -- 172.21.15.97:0/2031147667 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fa1640a37b0 msgr2=0x7fa1640a5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:56.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:55.792+0000 7fa184400700 1 --2- 172.21.15.97:0/2031147667 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fa1640a37b0 0x7fa1640a5c70 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa168005fd0 tx=0x7fa168005ee0 comp rx=0 tx=0).stop 2024-03-27T10:54:56.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:55.792+0000 7fa184400700 1 -- 172.21.15.97:0/2031147667 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa17c06ab80 msgr2=0x7fa17c101640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:56.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:55.792+0000 7fa184400700 1 --2- 172.21.15.97:0/2031147667 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa17c06ab80 0x7fa17c101640 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa170009fd0 tx=0x7fa170007ad0 comp rx=0 tx=0).stop 2024-03-27T10:54:56.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:55.792+0000 7fa182400700 1 -- 172.21.15.97:0/2031147667 reap_dead start 2024-03-27T10:54:56.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:55.792+0000 7fa184400700 1 -- 172.21.15.97:0/2031147667 shutdown_connections 2024-03-27T10:54:56.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:55.792+0000 7fa184400700 1 -- 172.21.15.97:0/2031147667 >> 172.21.15.97:0/2031147667 conn(0x7fa17c077590 msgr2=0x7fa17c1003e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:54:56.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:55.792+0000 7fa184400700 1 -- 172.21.15.97:0/2031147667 shutdown_connections 2024-03-27T10:54:56.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:55.792+0000 7fa184400700 1 -- 172.21.15.97:0/2031147667 wait complete. 2024-03-27T10:54:57.256 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:57 smithi097 ceph-mon[31859]: pgmap v390: 97 pgs: 97 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-03-27T10:54:57.259 INFO:teuthology.orchestra.run.smithi097.stdout:231928234036 2024-03-27T10:54:57.259 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell --fsid adca88c4-ec26-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.7 2024-03-27T10:54:57.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:57 smithi177 ceph-mon[35294]: pgmap v390: 97 pgs: 97 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-03-27T10:54:57.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:57 smithi114 ceph-mon[35338]: pgmap v390: 97 pgs: 97 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-03-27T10:54:57.693 INFO:teuthology.orchestra.run.smithi097.stdout:270582939695 2024-03-27T10:54:57.693 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell --fsid adca88c4-ec26-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.8 2024-03-27T10:54:58.206 INFO:teuthology.orchestra.run.smithi097.stdout:210453397561 2024-03-27T10:54:58.206 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell --fsid adca88c4-ec26-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.6 2024-03-27T10:54:58.676 INFO:teuthology.orchestra.run.smithi097.stdout:365072220185 2024-03-27T10:54:58.677 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell --fsid adca88c4-ec26-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.11 2024-03-27T10:54:58.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:58.977+0000 7f30bf000700 1 -- 172.21.15.97:0/2538050016 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30b8108620 msgr2=0x7f30b8108a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:58.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:58.977+0000 7f30bf000700 1 --2- 172.21.15.97:0/2538050016 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30b8108620 0x7f30b8108a00 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f30a8009990 tx=0x7f30a8009ca0 comp rx=0 tx=0).stop 2024-03-27T10:54:58.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:58.977+0000 7f30bf000700 1 -- 172.21.15.97:0/2538050016 shutdown_connections 2024-03-27T10:54:58.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:58.977+0000 7f30bf000700 1 --2- 172.21.15.97:0/2538050016 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f30b8102fe0 0x7f30b810f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:58.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:58.977+0000 7f30bf000700 1 --2- 172.21.15.97:0/2538050016 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30b8108620 0x7f30b8108a00 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:58.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:58.977+0000 7f30bf000700 1 --2- 172.21.15.97:0/2538050016 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f30b8102620 0x7f30b8102aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:58.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:58.977+0000 7f30bf000700 1 -- 172.21.15.97:0/2538050016 >> 172.21.15.97:0/2538050016 conn(0x7f30b80fe0f0 msgr2=0x7f30b8100510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:54:58.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:58.977+0000 7f30bf000700 1 -- 172.21.15.97:0/2538050016 shutdown_connections 2024-03-27T10:54:58.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:58.977+0000 7f30bf000700 1 -- 172.21.15.97:0/2538050016 wait complete. 2024-03-27T10:54:58.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:58.978+0000 7f30bf000700 1 Processor -- start 2024-03-27T10:54:58.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:58.978+0000 7f30bf000700 1 -- start start 2024-03-27T10:54:58.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:58.978+0000 7f30bf000700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f30b8102620 0x7f30b819af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:58.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:58.978+0000 7f30bf000700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f30b8102fe0 0x7f30b819b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:58.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:58.978+0000 7f30bf000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30b8108620 0x7f30b819f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:58.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:58.978+0000 7f30bf000700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f30b8112850 con 0x7f30b8108620 2024-03-27T10:54:58.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:58.978+0000 7f30bf000700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f30b81126d0 con 0x7f30b8102620 2024-03-27T10:54:58.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:58.978+0000 7f30bf000700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f30b81129d0 con 0x7f30b8102fe0 2024-03-27T10:54:58.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:58.979+0000 7f30bd000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30b8108620 0x7f30b819f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:58.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:58.979+0000 7f30bd000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30b8108620 0x7f30b819f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54024/0 (socket says 172.21.15.97:54024) 2024-03-27T10:54:58.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:58.979+0000 7f30bd000700 1 -- 172.21.15.97:0/3239081216 learned_addr learned my addr 172.21.15.97:0/3239081216 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:54:58.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:58.979+0000 7f30b7e00700 1 --2- 172.21.15.97:0/3239081216 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f30b8102620 0x7f30b819af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:58.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:58.979+0000 7f30b7400700 1 --2- 172.21.15.97:0/3239081216 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f30b8102fe0 0x7f30b819b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:58.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:58.979+0000 7f30bd000700 1 -- 172.21.15.97:0/3239081216 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f30b8102fe0 msgr2=0x7f30b819b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:58.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:58.979+0000 7f30bd000700 1 --2- 172.21.15.97:0/3239081216 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f30b8102fe0 0x7f30b819b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:58.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:58.979+0000 7f30bd000700 1 -- 172.21.15.97:0/3239081216 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f30b8102620 msgr2=0x7f30b819af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:58.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:58.979+0000 7f30bd000700 1 --2- 172.21.15.97:0/3239081216 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f30b8102620 0x7f30b819af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:58.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:58.980+0000 7f30bd000700 1 -- 172.21.15.97:0/3239081216 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30b819ff10 con 0x7f30b8108620 2024-03-27T10:54:58.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:58.980+0000 7f30bd000700 1 --2- 172.21.15.97:0/3239081216 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30b8108620 0x7f30b819f7d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f30b0007ee0 tx=0x7f30b000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:54:58.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:58.980+0000 7f30b4c00700 1 -- 172.21.15.97:0/3239081216 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b0004bb0 con 0x7f30b8108620 2024-03-27T10:54:58.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:58.980+0000 7f30bf000700 1 -- 172.21.15.97:0/3239081216 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30b81a0200 con 0x7f30b8108620 2024-03-27T10:54:58.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:58.980+0000 7f30bf000700 1 -- 172.21.15.97:0/3239081216 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30b81a7a80 con 0x7f30b8108620 2024-03-27T10:54:58.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:58.980+0000 7f30b4c00700 1 -- 172.21.15.97:0/3239081216 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f30b0004d50 con 0x7f30b8108620 2024-03-27T10:54:58.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:58.980+0000 7f30b4c00700 1 -- 172.21.15.97:0/3239081216 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b00058e0 con 0x7f30b8108620 2024-03-27T10:54:58.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:58.981+0000 7f30bf000700 1 -- 172.21.15.97:0/3239081216 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f30b80ff940 con 0x7f30b8108620 2024-03-27T10:54:58.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:58.984+0000 7f30b4c00700 1 -- 172.21.15.97:0/3239081216 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138421+0+0 (secure 0 0 0) 0x7f30b001a020 con 0x7f30b8108620 2024-03-27T10:54:58.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:58.985+0000 7f30b4c00700 1 --2- 172.21.15.97:0/3239081216 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f309c0a37b0 0x7f309c0a5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:58.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:58.985+0000 7f30b4c00700 1 -- 172.21.15.97:0/3239081216 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f30b00cfc30 con 0x7f30b8108620 2024-03-27T10:54:58.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:58.986+0000 7f30b7e00700 1 --2- 172.21.15.97:0/3239081216 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f309c0a37b0 0x7f309c0a5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:58.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:58.986+0000 7f30b4c00700 1 --2- 172.21.15.97:0/3239081216 >> [v2:172.21.15.97:6802/1746812080,v1:172.21.15.97:6803/1746812080] conn(0x7f309c0ad2f0 0x7f309c0af790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:54:58.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:58.986+0000 7f30b4c00700 1 -- 172.21.15.97:0/3239081216 --> [v2:172.21.15.97:6802/1746812080,v1:172.21.15.97:6803/1746812080] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f30b0021e70 con 0x7f309c0ad2f0 2024-03-27T10:54:58.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:58.986+0000 7f30b4c00700 1 -- 172.21.15.97:0/3239081216 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f30b00cffe0 con 0x7f30b8108620 2024-03-27T10:54:58.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:58.986+0000 7f30b7e00700 1 --2- 172.21.15.97:0/3239081216 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f309c0a37b0 0x7f309c0a5c70 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f30a8006010 tx=0x7f30a800b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:54:58.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:58.987+0000 7f30b7400700 1 --2- 172.21.15.97:0/3239081216 >> [v2:172.21.15.97:6802/1746812080,v1:172.21.15.97:6803/1746812080] conn(0x7f309c0ad2f0 0x7f309c0af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:54:58.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:58.987+0000 7f30b7400700 1 --2- 172.21.15.97:0/3239081216 >> [v2:172.21.15.97:6802/1746812080,v1:172.21.15.97:6803/1746812080] conn(0x7f309c0ad2f0 0x7f309c0af790 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:54:58.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:58.988+0000 7f30b4c00700 1 -- 172.21.15.97:0/3239081216 <== osd.0 v2:172.21.15.97:6802/1746812080 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f30b0021e70 con 0x7f309c0ad2f0 2024-03-27T10:54:58.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:59.009+0000 7f30bf000700 1 -- 172.21.15.97:0/3239081216 --> [v2:172.21.15.97:6802/1746812080,v1:172.21.15.97:6803/1746812080] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f30b8103d60 con 0x7f309c0ad2f0 2024-03-27T10:54:58.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:59.010+0000 7f30b4c00700 1 -- 172.21.15.97:0/3239081216 <== osd.0 v2:172.21.15.97:6802/1746812080 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f30b8103d60 con 0x7f309c0ad2f0 2024-03-27T10:54:58.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:59.011+0000 7f30bf000700 1 -- 172.21.15.97:0/3239081216 >> [v2:172.21.15.97:6802/1746812080,v1:172.21.15.97:6803/1746812080] conn(0x7f309c0ad2f0 msgr2=0x7f309c0af790 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:58.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:59.011+0000 7f30bf000700 1 --2- 172.21.15.97:0/3239081216 >> [v2:172.21.15.97:6802/1746812080,v1:172.21.15.97:6803/1746812080] conn(0x7f309c0ad2f0 0x7f309c0af790 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:54:58.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:59.011+0000 7f30bf000700 1 -- 172.21.15.97:0/3239081216 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f309c0a37b0 msgr2=0x7f309c0a5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:58.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:59.011+0000 7f30bf000700 1 --2- 172.21.15.97:0/3239081216 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f309c0a37b0 0x7f309c0a5c70 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f30a8006010 tx=0x7f30a800b540 comp rx=0 tx=0).stop 2024-03-27T10:54:58.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:59.011+0000 7f30bf000700 1 -- 172.21.15.97:0/3239081216 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30b8108620 msgr2=0x7f30b819f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:54:58.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:59.011+0000 7f30bf000700 1 --2- 172.21.15.97:0/3239081216 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30b8108620 0x7f30b819f7d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f30b0007ee0 tx=0x7f30b000d3b0 comp rx=0 tx=0).stop 2024-03-27T10:54:58.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:59.011+0000 7f30bd000700 1 -- 172.21.15.97:0/3239081216 reap_dead start 2024-03-27T10:54:58.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:59.011+0000 7f30bf000700 1 -- 172.21.15.97:0/3239081216 shutdown_connections 2024-03-27T10:54:58.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:59.011+0000 7f30bf000700 1 -- 172.21.15.97:0/3239081216 >> 172.21.15.97:0/3239081216 conn(0x7f30b80fe0f0 msgr2=0x7f30b810dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:54:58.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:59.011+0000 7f30bf000700 1 -- 172.21.15.97:0/3239081216 shutdown_connections 2024-03-27T10:54:58.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:54:59.011+0000 7f30bf000700 1 -- 172.21.15.97:0/3239081216 wait complete. 2024-03-27T10:54:59.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:54:59 smithi097 ceph-mon[31859]: pgmap v391: 97 pgs: 97 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-27T10:54:59.426 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:54:59 smithi177 ceph-mon[35294]: pgmap v391: 97 pgs: 97 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-27T10:54:59.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:54:59 smithi114 ceph-mon[35338]: pgmap v391: 97 pgs: 97 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-27T10:54:59.920 INFO:teuthology.orchestra.run.smithi097.stdout:34359738469 2024-03-27T10:54:59.920 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell --fsid adca88c4-ec26-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-03-27T10:55:00.237 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T10:55:00.302 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T10:55:00.306 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T10:55:00.308 INFO:teuthology.orchestra.run.smithi097.stdout:77309411413 2024-03-27T10:55:00.308 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell --fsid adca88c4-ec26-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-03-27T10:55:00.371 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T10:55:00.478 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T10:55:00.485 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T10:55:00.486 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T10:55:00.488 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T10:55:00.489 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T10:55:00.498 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T10:55:01.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:55:01 smithi097 ceph-mon[31859]: pgmap v392: 97 pgs: 97 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-03-27T10:55:01.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:55:01 smithi177 ceph-mon[35294]: pgmap v392: 97 pgs: 97 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-03-27T10:55:01.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:55:01 smithi114 ceph-mon[35338]: pgmap v392: 97 pgs: 97 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-03-27T10:55:03.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:55:03 smithi097 ceph-mon[31859]: pgmap v393: 97 pgs: 97 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-27T10:55:03.426 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:55:03 smithi177 ceph-mon[35294]: pgmap v393: 97 pgs: 97 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-27T10:55:03.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:55:03 smithi114 ceph-mon[35338]: pgmap v393: 97 pgs: 97 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-27T10:55:04.735 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T10:55:05.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:55:05 smithi177 ceph-mon[35294]: pgmap v394: 97 pgs: 97 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-03-27T10:55:05.449 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:55:05 smithi097 ceph-mon[31859]: pgmap v394: 97 pgs: 97 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-03-27T10:55:05.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:55:05 smithi114 ceph-mon[35338]: pgmap v394: 97 pgs: 97 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-03-27T10:55:07.378 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:55:07 smithi097 ceph-mon[31859]: pgmap v395: 97 pgs: 97 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-03-27T10:55:07.378 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:55:07 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/401420460' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-27T10:55:07.426 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:55:07 smithi177 ceph-mon[35294]: pgmap v395: 97 pgs: 97 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-03-27T10:55:07.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:55:07 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/401420460' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-27T10:55:07.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:55:07 smithi114 ceph-mon[35338]: pgmap v395: 97 pgs: 97 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-03-27T10:55:07.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:55:07 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/401420460' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-27T10:55:07.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:06.424+0000 7f3042200700 1 -- 172.21.15.97:0/885313016 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f303c1051f0 msgr2=0x7f303c111700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:07.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:06.424+0000 7f3042200700 1 --2- 172.21.15.97:0/885313016 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f303c1051f0 0x7f303c111700 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f303400b0a0 tx=0x7f303400b3b0 comp rx=0 tx=0).stop 2024-03-27T10:55:07.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:06.424+0000 7f3042200700 1 -- 172.21.15.97:0/885313016 shutdown_connections 2024-03-27T10:55:07.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:06.424+0000 7f3042200700 1 --2- 172.21.15.97:0/885313016 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f303c1051f0 0x7f303c111700 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:07.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:06.424+0000 7f3042200700 1 --2- 172.21.15.97:0/885313016 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f303c10a830 0x7f303c10ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:07.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:06.424+0000 7f3042200700 1 --2- 172.21.15.97:0/885313016 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f303c104830 0x7f303c104cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:07.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:06.424+0000 7f3042200700 1 -- 172.21.15.97:0/885313016 >> 172.21.15.97:0/885313016 conn(0x7f303c100320 msgr2=0x7f303c102740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:55:07.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:06.424+0000 7f3042200700 1 -- 172.21.15.97:0/885313016 shutdown_connections 2024-03-27T10:55:07.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:06.424+0000 7f3042200700 1 -- 172.21.15.97:0/885313016 wait complete. 2024-03-27T10:55:07.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:06.425+0000 7f3042200700 1 Processor -- start 2024-03-27T10:55:07.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:06.425+0000 7f3042200700 1 -- start start 2024-03-27T10:55:07.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:06.426+0000 7f3042200700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f303c104830 0x7f303c19ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:07.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:06.426+0000 7f3042200700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f303c1051f0 0x7f303c19b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:07.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:06.426+0000 7f3042200700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f303c10a830 0x7f303c19f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:07.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:06.426+0000 7f3042200700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f303c114a30 con 0x7f303c104830 2024-03-27T10:55:07.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:06.426+0000 7f3042200700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f303c1148b0 con 0x7f303c1051f0 2024-03-27T10:55:07.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:06.426+0000 7f3042200700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f303c114bb0 con 0x7f303c10a830 2024-03-27T10:55:07.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:06.426+0000 7f303b400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f303c104830 0x7f303c19ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:07.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:06.426+0000 7f303b400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f303c104830 0x7f303c19ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54044/0 (socket says 172.21.15.97:54044) 2024-03-27T10:55:07.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:06.426+0000 7f303b400700 1 -- 172.21.15.97:0/401420460 learned_addr learned my addr 172.21.15.97:0/401420460 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:55:07.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:06.426+0000 7f3032a00700 1 --2- 172.21.15.97:0/401420460 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f303c1051f0 0x7f303c19b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:07.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:06.426+0000 7f303b400700 1 -- 172.21.15.97:0/401420460 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f303c10a830 msgr2=0x7f303c19f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:07.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:06.426+0000 7f303b400700 1 --2- 172.21.15.97:0/401420460 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f303c10a830 0x7f303c19f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:07.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:06.426+0000 7f303b400700 1 -- 172.21.15.97:0/401420460 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f303c1051f0 msgr2=0x7f303c19b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:07.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:06.426+0000 7f303b400700 1 --2- 172.21.15.97:0/401420460 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f303c1051f0 0x7f303c19b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:07.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:06.427+0000 7f303b400700 1 -- 172.21.15.97:0/401420460 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f303c19fe50 con 0x7f303c104830 2024-03-27T10:55:07.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:06.427+0000 7f3032a00700 1 --2- 172.21.15.97:0/401420460 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f303c1051f0 0x7f303c19b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:55:07.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:06.427+0000 7f303b400700 1 --2- 172.21.15.97:0/401420460 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f303c104830 0x7f303c19ae40 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f302c00d980 tx=0x7f302c00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:55:07.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:06.427+0000 7f3038c00700 1 -- 172.21.15.97:0/401420460 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f302c016070 con 0x7f303c104830 2024-03-27T10:55:07.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:06.427+0000 7f3038c00700 1 -- 172.21.15.97:0/401420460 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f302c0044d0 con 0x7f303c104830 2024-03-27T10:55:07.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:06.427+0000 7f3038c00700 1 -- 172.21.15.97:0/401420460 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f302c004cf0 con 0x7f303c104830 2024-03-27T10:55:07.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:06.427+0000 7f3042200700 1 -- 172.21.15.97:0/401420460 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f303c1a0140 con 0x7f303c104830 2024-03-27T10:55:07.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:06.427+0000 7f3042200700 1 -- 172.21.15.97:0/401420460 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f303c1a79c0 con 0x7f303c104830 2024-03-27T10:55:07.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:06.428+0000 7f3042200700 1 -- 172.21.15.97:0/401420460 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f303c069300 con 0x7f303c104830 2024-03-27T10:55:07.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:06.430+0000 7f3038c00700 1 -- 172.21.15.97:0/401420460 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138421+0+0 (secure 0 0 0) 0x7f302c004020 con 0x7f303c104830 2024-03-27T10:55:07.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:06.432+0000 7f3038c00700 1 --2- 172.21.15.97:0/401420460 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f30200a37b0 0x7f30200a5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:07.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:06.432+0000 7f3038c00700 1 -- 172.21.15.97:0/401420460 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f302c0cfc30 con 0x7f303c104830 2024-03-27T10:55:07.632 INFO:teuthology.orchestra.run.smithi097.stdout:176093659206 2024-03-27T10:55:07.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:06.432+0000 7f3032a00700 1 --2- 172.21.15.97:0/401420460 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f30200a37b0 0x7f30200a5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:07.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:06.433+0000 7f3032a00700 1 --2- 172.21.15.97:0/401420460 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f30200a37b0 0x7f30200a5c70 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f303c109df0 tx=0x7f3024009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:55:07.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:06.437+0000 7f3038c00700 1 -- 172.21.15.97:0/401420460 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f302c01f020 con 0x7f303c104830 2024-03-27T10:55:07.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:06.673+0000 7f3042200700 1 -- 172.21.15.97:0/401420460 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f303c02de50 con 0x7f303c104830 2024-03-27T10:55:07.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:06.673+0000 7f3038c00700 1 -- 172.21.15.97:0/401420460 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f302c09c650 con 0x7f303c104830 2024-03-27T10:55:07.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:06.677+0000 7f3042200700 1 -- 172.21.15.97:0/401420460 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f30200a37b0 msgr2=0x7f30200a5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:07.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:06.677+0000 7f3042200700 1 --2- 172.21.15.97:0/401420460 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f30200a37b0 0x7f30200a5c70 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f303c109df0 tx=0x7f3024009260 comp rx=0 tx=0).stop 2024-03-27T10:55:07.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:06.677+0000 7f3042200700 1 -- 172.21.15.97:0/401420460 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f303c104830 msgr2=0x7f303c19ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:07.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:06.677+0000 7f3042200700 1 --2- 172.21.15.97:0/401420460 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f303c104830 0x7f303c19ae40 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f302c00d980 tx=0x7f302c00dd40 comp rx=0 tx=0).stop 2024-03-27T10:55:07.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:06.677+0000 7f3042200700 1 -- 172.21.15.97:0/401420460 shutdown_connections 2024-03-27T10:55:07.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:06.677+0000 7f3042200700 1 --2- 172.21.15.97:0/401420460 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f30200a37b0 0x7f30200a5c70 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:07.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:06.677+0000 7f3042200700 1 --2- 172.21.15.97:0/401420460 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f303c10a830 0x7f303c19f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:07.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:06.677+0000 7f3042200700 1 --2- 172.21.15.97:0/401420460 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f303c104830 0x7f303c19ae40 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:07.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:06.677+0000 7f3042200700 1 --2- 172.21.15.97:0/401420460 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f303c1051f0 0x7f303c19b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:07.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:06.677+0000 7f3042200700 1 -- 172.21.15.97:0/401420460 >> 172.21.15.97:0/401420460 conn(0x7f303c100320 msgr2=0x7f303c1008a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:55:07.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:06.677+0000 7f3042200700 1 -- 172.21.15.97:0/401420460 shutdown_connections 2024-03-27T10:55:07.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:06.677+0000 7f3042200700 1 -- 172.21.15.97:0/401420460 wait complete. 2024-03-27T10:55:08.325 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:55:08 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:55:08.426 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:55:08 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:55:08.516 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:55:08 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:55:09.245 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:55:09 smithi097 ceph-mon[31859]: pgmap v396: 97 pgs: 97 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-03-27T10:55:09.426 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:55:09 smithi177 ceph-mon[35294]: pgmap v396: 97 pgs: 97 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-03-27T10:55:09.514 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:55:09 smithi114 ceph-mon[35338]: pgmap v396: 97 pgs: 97 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-03-27T10:55:11.267 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:55:11 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:55:11.267 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:55:11 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:55:11.267 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:55:11 smithi114 ceph-mon[35338]: pgmap v397: 97 pgs: 97 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-03-27T10:55:11.267 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:55:11 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:55:11.267 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:55:11 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:55:11.267 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:55:11 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:55:11.267 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:55:11 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:55:11.294 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:55:11 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:55:11.294 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:55:11 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:55:11.294 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:55:11 smithi097 ceph-mon[31859]: pgmap v397: 97 pgs: 97 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-03-27T10:55:11.294 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:55:11 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:55:11.294 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:55:11 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:55:11.294 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:55:11 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:55:11.294 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:55:11 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:55:11.426 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:55:11 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:55:11.426 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:55:11 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:55:11.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:55:11 smithi177 ceph-mon[35294]: pgmap v397: 97 pgs: 97 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-03-27T10:55:11.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:55:11 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:55:11.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:55:11 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:55:11.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:55:11 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:55:11.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:55:11 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:55:12.225 INFO:tasks.cephadm.ceph_manager.ceph:need seq 176093659202 got 176093659206 for osd.5 2024-03-27T10:55:12.225 DEBUG:teuthology.parallel:result is None 2024-03-27T10:55:13.368 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:55:13 smithi097 ceph-mon[31859]: pgmap v398: 97 pgs: 97 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-27T10:55:13.426 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:55:13 smithi177 ceph-mon[35294]: pgmap v398: 97 pgs: 97 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-27T10:55:13.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:55:13 smithi114 ceph-mon[35338]: pgmap v398: 97 pgs: 97 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-27T10:55:15.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:14.066+0000 7fcc97e00700 1 -- 172.21.15.97:0/1334953489 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc981050f0 msgr2=0x7fcc98111600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:15.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:14.066+0000 7fcc97e00700 1 --2- 172.21.15.97:0/1334953489 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc981050f0 0x7fcc98111600 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fcc9000b0a0 tx=0x7fcc9000b3b0 comp rx=0 tx=0).stop 2024-03-27T10:55:15.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:14.067+0000 7fcc97e00700 1 -- 172.21.15.97:0/1334953489 shutdown_connections 2024-03-27T10:55:15.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:14.067+0000 7fcc97e00700 1 --2- 172.21.15.97:0/1334953489 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc981050f0 0x7fcc98111600 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:15.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:14.067+0000 7fcc97e00700 1 --2- 172.21.15.97:0/1334953489 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcc9810a730 0x7fcc9810ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:15.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:14.067+0000 7fcc97e00700 1 --2- 172.21.15.97:0/1334953489 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcc98104730 0x7fcc98104bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:15.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:14.067+0000 7fcc97e00700 1 -- 172.21.15.97:0/1334953489 >> 172.21.15.97:0/1334953489 conn(0x7fcc98100260 msgr2=0x7fcc98102680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:55:15.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:14.067+0000 7fcc97e00700 1 -- 172.21.15.97:0/1334953489 shutdown_connections 2024-03-27T10:55:15.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:14.067+0000 7fcc97e00700 1 -- 172.21.15.97:0/1334953489 wait complete. 2024-03-27T10:55:15.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:14.068+0000 7fcc97e00700 1 Processor -- start 2024-03-27T10:55:15.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:14.068+0000 7fcc97e00700 1 -- start start 2024-03-27T10:55:15.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:14.068+0000 7fcc97e00700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcc98104730 0x7fcc9819ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:15.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:14.068+0000 7fcc97e00700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcc981050f0 0x7fcc9819b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:15.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:14.068+0000 7fcc97e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc9810a730 0x7fcc9819f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:15.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:14.068+0000 7fcc97e00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fcc98114930 con 0x7fcc9810a730 2024-03-27T10:55:15.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:14.068+0000 7fcc97e00700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fcc981147b0 con 0x7fcc981050f0 2024-03-27T10:55:15.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:14.068+0000 7fcc97e00700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fcc98114ab0 con 0x7fcc98104730 2024-03-27T10:55:15.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:14.069+0000 7fcc97400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc9810a730 0x7fcc9819f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:15.105 INFO:teuthology.orchestra.run.smithi097.stdout:115964117078 2024-03-27T10:55:15.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:14.069+0000 7fcc97400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc9810a730 0x7fcc9819f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37062/0 (socket says 172.21.15.97:37062) 2024-03-27T10:55:15.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:14.069+0000 7fcc97400700 1 -- 172.21.15.97:0/1417378084 learned_addr learned my addr 172.21.15.97:0/1417378084 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:55:15.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:14.069+0000 7fcc96a00700 1 --2- 172.21.15.97:0/1417378084 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcc98104730 0x7fcc9819ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:15.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:14.069+0000 7fcc96000700 1 --2- 172.21.15.97:0/1417378084 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcc981050f0 0x7fcc9819b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:15.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:14.069+0000 7fcc97400700 1 -- 172.21.15.97:0/1417378084 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcc98104730 msgr2=0x7fcc9819ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:15.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:14.069+0000 7fcc97400700 1 --2- 172.21.15.97:0/1417378084 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcc98104730 0x7fcc9819ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:15.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:14.069+0000 7fcc97400700 1 -- 172.21.15.97:0/1417378084 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcc981050f0 msgr2=0x7fcc9819b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:15.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:14.069+0000 7fcc97400700 1 --2- 172.21.15.97:0/1417378084 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcc981050f0 0x7fcc9819b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:15.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:14.069+0000 7fcc97400700 1 -- 172.21.15.97:0/1417378084 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc9819fd50 con 0x7fcc9810a730 2024-03-27T10:55:15.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:14.069+0000 7fcc97400700 1 --2- 172.21.15.97:0/1417378084 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc9810a730 0x7fcc9819f610 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fcc9000b830 tx=0x7fcc90004d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:55:15.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:14.070+0000 7fcc8f400700 1 -- 172.21.15.97:0/1417378084 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc90021070 con 0x7fcc9810a730 2024-03-27T10:55:15.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:14.070+0000 7fcc8f400700 1 -- 172.21.15.97:0/1417378084 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcc90003c00 con 0x7fcc9810a730 2024-03-27T10:55:15.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:14.070+0000 7fcc97e00700 1 -- 172.21.15.97:0/1417378084 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc9819ffe0 con 0x7fcc9810a730 2024-03-27T10:55:15.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:14.070+0000 7fcc8f400700 1 -- 172.21.15.97:0/1417378084 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc90004790 con 0x7fcc9810a730 2024-03-27T10:55:15.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:14.070+0000 7fcc97e00700 1 -- 172.21.15.97:0/1417378084 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc981a78c0 con 0x7fcc9810a730 2024-03-27T10:55:15.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:14.071+0000 7fcc97e00700 1 -- 172.21.15.97:0/1417378084 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc98069300 con 0x7fcc9810a730 2024-03-27T10:55:15.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:14.073+0000 7fcc8f400700 1 -- 172.21.15.97:0/1417378084 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138421+0+0 (secure 0 0 0) 0x7fcc90007a00 con 0x7fcc9810a730 2024-03-27T10:55:15.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:14.074+0000 7fcc8f400700 1 --2- 172.21.15.97:0/1417378084 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fcc7c0a37b0 0x7fcc7c0a5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:15.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:14.074+0000 7fcc8f400700 1 -- 172.21.15.97:0/1417378084 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fcc9000f020 con 0x7fcc9810a730 2024-03-27T10:55:15.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:14.074+0000 7fcc96a00700 1 --2- 172.21.15.97:0/1417378084 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fcc7c0a37b0 0x7fcc7c0a5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:15.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:14.075+0000 7fcc96a00700 1 --2- 172.21.15.97:0/1417378084 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fcc7c0a37b0 0x7fcc7c0a5c70 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fcc88005950 tx=0x7fcc880058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:55:15.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:14.077+0000 7fcc8f400700 1 -- 172.21.15.97:0/1417378084 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7fcc9002a020 con 0x7fcc9810a730 2024-03-27T10:55:15.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:14.347+0000 7fcc97e00700 1 -- 172.21.15.97:0/1417378084 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fcc9802dd30 con 0x7fcc9810a730 2024-03-27T10:55:15.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:14.348+0000 7fcc8f400700 1 -- 172.21.15.97:0/1417378084 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fcc9002e3e0 con 0x7fcc9810a730 2024-03-27T10:55:15.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:14.351+0000 7fcc97e00700 1 -- 172.21.15.97:0/1417378084 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fcc7c0a37b0 msgr2=0x7fcc7c0a5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:15.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:14.351+0000 7fcc97e00700 1 --2- 172.21.15.97:0/1417378084 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fcc7c0a37b0 0x7fcc7c0a5c70 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fcc88005950 tx=0x7fcc880058e0 comp rx=0 tx=0).stop 2024-03-27T10:55:15.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:14.351+0000 7fcc97e00700 1 -- 172.21.15.97:0/1417378084 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc9810a730 msgr2=0x7fcc9819f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:15.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:14.351+0000 7fcc97e00700 1 --2- 172.21.15.97:0/1417378084 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc9810a730 0x7fcc9819f610 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fcc9000b830 tx=0x7fcc90004d90 comp rx=0 tx=0).stop 2024-03-27T10:55:15.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:14.351+0000 7fcc97e00700 1 -- 172.21.15.97:0/1417378084 shutdown_connections 2024-03-27T10:55:15.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:14.351+0000 7fcc97e00700 1 --2- 172.21.15.97:0/1417378084 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fcc7c0a37b0 0x7fcc7c0a5c70 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:15.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:14.351+0000 7fcc97e00700 1 --2- 172.21.15.97:0/1417378084 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc9810a730 0x7fcc9819f610 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:15.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:14.351+0000 7fcc97e00700 1 --2- 172.21.15.97:0/1417378084 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcc98104730 0x7fcc9819ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:15.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:14.351+0000 7fcc97e00700 1 --2- 172.21.15.97:0/1417378084 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcc981050f0 0x7fcc9819b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:15.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:14.351+0000 7fcc97e00700 1 -- 172.21.15.97:0/1417378084 >> 172.21.15.97:0/1417378084 conn(0x7fcc98100260 msgr2=0x7fcc981007e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:55:15.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:14.351+0000 7fcc97e00700 1 -- 172.21.15.97:0/1417378084 shutdown_connections 2024-03-27T10:55:15.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:14.351+0000 7fcc97e00700 1 -- 172.21.15.97:0/1417378084 wait complete. 2024-03-27T10:55:15.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:55:15 smithi097 ceph-mon[31859]: pgmap v399: 97 pgs: 97 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-03-27T10:55:15.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:55:15 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/1417378084' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-27T10:55:15.426 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:55:15 smithi177 ceph-mon[35294]: pgmap v399: 97 pgs: 97 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-03-27T10:55:15.426 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:55:15 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/1417378084' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-27T10:55:15.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:55:15 smithi114 ceph-mon[35338]: pgmap v399: 97 pgs: 97 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-03-27T10:55:15.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:55:15 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/1417378084' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-27T10:55:16.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:15.913+0000 7f60e3800700 1 -- 172.21.15.97:0/2974972464 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60dc06ab80 msgr2=0x7f60dc06b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:16.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:15.913+0000 7f60e3800700 1 --2- 172.21.15.97:0/2974972464 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60dc06ab80 0x7f60dc06b000 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f60c80099c0 tx=0x7f60c8009cd0 comp rx=0 tx=0).stop 2024-03-27T10:55:16.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:15.913+0000 7f60e3800700 1 -- 172.21.15.97:0/2974972464 shutdown_connections 2024-03-27T10:55:16.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:15.913+0000 7f60e3800700 1 --2- 172.21.15.97:0/2974972464 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f60dc06b540 0x7f60dc10f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:16.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:15.913+0000 7f60e3800700 1 --2- 172.21.15.97:0/2974972464 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f60dc1085f0 0x7f60dc1089d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:16.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:15.913+0000 7f60e3800700 1 --2- 172.21.15.97:0/2974972464 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60dc06ab80 0x7f60dc06b000 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:16.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:15.913+0000 7f60e3800700 1 -- 172.21.15.97:0/2974972464 >> 172.21.15.97:0/2974972464 conn(0x7f60dc077590 msgr2=0x7f60dc0779a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:55:16.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:15.913+0000 7f60e3800700 1 -- 172.21.15.97:0/2974972464 shutdown_connections 2024-03-27T10:55:16.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:15.913+0000 7f60e3800700 1 -- 172.21.15.97:0/2974972464 wait complete. 2024-03-27T10:55:16.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:15.914+0000 7f60e3800700 1 Processor -- start 2024-03-27T10:55:16.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:15.914+0000 7f60e3800700 1 -- start start 2024-03-27T10:55:16.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:15.915+0000 7f60e3800700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60dc06ab80 0x7f60dc101640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:16.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:15.915+0000 7f60e3800700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f60dc06b540 0x7f60dc101b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:16.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:15.915+0000 7f60e3800700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f60dc1085f0 0x7f60dc1020c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:16.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:15.915+0000 7f60e3800700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f60dc112800 con 0x7f60dc06ab80 2024-03-27T10:55:16.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:15.915+0000 7f60e3800700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f60dc112680 con 0x7f60dc1085f0 2024-03-27T10:55:16.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:15.915+0000 7f60e3800700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f60dc112980 con 0x7f60dc06b540 2024-03-27T10:55:16.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:15.915+0000 7f60e0e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60dc06ab80 0x7f60dc101640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:16.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:15.915+0000 7f60e0e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60dc06ab80 0x7f60dc101640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37078/0 (socket says 172.21.15.97:37078) 2024-03-27T10:55:16.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:15.915+0000 7f60e0e00700 1 -- 172.21.15.97:0/1869198430 learned_addr learned my addr 172.21.15.97:0/1869198430 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:55:16.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:15.915+0000 7f60e1800700 1 --2- 172.21.15.97:0/1869198430 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f60dc1085f0 0x7f60dc1020c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:16.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:15.915+0000 7f60e0e00700 1 -- 172.21.15.97:0/1869198430 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f60dc06b540 msgr2=0x7f60dc101b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:16.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:15.915+0000 7f60e0e00700 1 --2- 172.21.15.97:0/1869198430 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f60dc06b540 0x7f60dc101b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:16.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:15.915+0000 7f60e0e00700 1 -- 172.21.15.97:0/1869198430 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f60dc1085f0 msgr2=0x7f60dc1020c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:16.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:15.915+0000 7f60e0e00700 1 --2- 172.21.15.97:0/1869198430 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f60dc1085f0 0x7f60dc1020c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:16.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:15.915+0000 7f60e0e00700 1 -- 172.21.15.97:0/1869198430 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60dc1a7280 con 0x7f60dc06ab80 2024-03-27T10:55:16.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:15.916+0000 7f60e0e00700 1 --2- 172.21.15.97:0/1869198430 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60dc06ab80 0x7f60dc101640 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f60d000d980 tx=0x7f60d000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:55:16.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:15.916+0000 7f60d9600700 1 -- 172.21.15.97:0/1869198430 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d0016070 con 0x7f60dc06ab80 2024-03-27T10:55:16.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:15.916+0000 7f60d9600700 1 -- 172.21.15.97:0/1869198430 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f60d00044d0 con 0x7f60dc06ab80 2024-03-27T10:55:16.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:15.916+0000 7f60e3800700 1 -- 172.21.15.97:0/1869198430 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60dc1a7570 con 0x7f60dc06ab80 2024-03-27T10:55:16.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:15.916+0000 7f60d9600700 1 -- 172.21.15.97:0/1869198430 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d0004cf0 con 0x7f60dc06ab80 2024-03-27T10:55:16.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:15.916+0000 7f60e3800700 1 -- 172.21.15.97:0/1869198430 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60dc1a7b00 con 0x7f60dc06ab80 2024-03-27T10:55:16.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:15.917+0000 7f60e3800700 1 -- 172.21.15.97:0/1869198430 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60dc069300 con 0x7f60dc06ab80 2024-03-27T10:55:16.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:15.919+0000 7f60d9600700 1 -- 172.21.15.97:0/1869198430 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138421+0+0 (secure 0 0 0) 0x7f60d0004020 con 0x7f60dc06ab80 2024-03-27T10:55:16.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:15.920+0000 7f60d9600700 1 --2- 172.21.15.97:0/1869198430 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f60c40a37b0 0x7f60c40a5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:16.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:15.921+0000 7f60d9600700 1 -- 172.21.15.97:0/1869198430 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f60d00cfc30 con 0x7f60dc06ab80 2024-03-27T10:55:16.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:15.921+0000 7f60dbe00700 1 --2- 172.21.15.97:0/1869198430 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f60c40a37b0 0x7f60c40a5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:16.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:15.921+0000 7f60dbe00700 1 --2- 172.21.15.97:0/1869198430 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f60c40a37b0 0x7f60c40a5c70 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f60c80098e0 tx=0x7f60c8000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:55:16.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:15.925+0000 7f60d9600700 1 -- 172.21.15.97:0/1869198430 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f60d001f020 con 0x7f60dc06ab80 2024-03-27T10:55:16.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:16.157+0000 7f60e3800700 1 -- 172.21.15.97:0/1869198430 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f60dc02def0 con 0x7f60dc06ab80 2024-03-27T10:55:16.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:16.157+0000 7f60d9600700 1 -- 172.21.15.97:0/1869198430 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f60d009c650 con 0x7f60dc06ab80 2024-03-27T10:55:16.130 INFO:teuthology.orchestra.run.smithi097.stdout:55834574946 2024-03-27T10:55:16.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:16.160+0000 7f60e3800700 1 -- 172.21.15.97:0/1869198430 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f60c40a37b0 msgr2=0x7f60c40a5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:16.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:16.160+0000 7f60e3800700 1 --2- 172.21.15.97:0/1869198430 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f60c40a37b0 0x7f60c40a5c70 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f60c80098e0 tx=0x7f60c8000bc0 comp rx=0 tx=0).stop 2024-03-27T10:55:16.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:16.160+0000 7f60e3800700 1 -- 172.21.15.97:0/1869198430 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60dc06ab80 msgr2=0x7f60dc101640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:16.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:16.160+0000 7f60e3800700 1 --2- 172.21.15.97:0/1869198430 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60dc06ab80 0x7f60dc101640 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f60d000d980 tx=0x7f60d000dd40 comp rx=0 tx=0).stop 2024-03-27T10:55:16.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:16.160+0000 7f60e3800700 1 -- 172.21.15.97:0/1869198430 shutdown_connections 2024-03-27T10:55:16.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:16.160+0000 7f60e3800700 1 --2- 172.21.15.97:0/1869198430 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f60c40a37b0 0x7f60c40a5c70 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:16.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:16.160+0000 7f60e3800700 1 --2- 172.21.15.97:0/1869198430 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f60dc1085f0 0x7f60dc1020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:16.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:16.160+0000 7f60e3800700 1 --2- 172.21.15.97:0/1869198430 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60dc06ab80 0x7f60dc101640 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:16.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:16.160+0000 7f60e3800700 1 --2- 172.21.15.97:0/1869198430 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f60dc06b540 0x7f60dc101b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:16.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:16.160+0000 7f60e3800700 1 -- 172.21.15.97:0/1869198430 >> 172.21.15.97:0/1869198430 conn(0x7f60dc077590 msgr2=0x7f60dc100310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:55:16.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:16.161+0000 7f60e3800700 1 -- 172.21.15.97:0/1869198430 shutdown_connections 2024-03-27T10:55:16.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:16.161+0000 7f60e3800700 1 -- 172.21.15.97:0/1869198430 wait complete. 2024-03-27T10:55:16.987 INFO:tasks.cephadm.ceph_manager.ceph:need seq 115964117072 got 115964117078 for osd.3 2024-03-27T10:55:16.988 DEBUG:teuthology.parallel:result is None 2024-03-27T10:55:17.209 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:55:17 smithi097 ceph-mon[31859]: pgmap v400: 97 pgs: 97 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-27T10:55:17.209 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:55:17 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/1869198430' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-27T10:55:17.209 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:55:17 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/1873154819' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-27T10:55:17.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:55:17 smithi177 ceph-mon[35294]: pgmap v400: 97 pgs: 97 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-27T10:55:17.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:55:17 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/1869198430' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-27T10:55:17.428 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:55:17 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/1873154819' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-27T10:55:17.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:55:17 smithi114 ceph-mon[35338]: pgmap v400: 97 pgs: 97 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-27T10:55:17.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:55:17 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/1869198430' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-27T10:55:17.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:55:17 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/1873154819' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-27T10:55:17.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:16.438+0000 7f859e200700 1 -- 172.21.15.97:0/581079247 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8598102690 msgr2=0x7f8598102b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:17.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:16.438+0000 7f859e200700 1 --2- 172.21.15.97:0/581079247 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8598102690 0x7f8598102b10 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f85840099c0 tx=0x7f8584009cd0 comp rx=0 tx=0).stop 2024-03-27T10:55:17.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:16.439+0000 7f859e200700 1 -- 172.21.15.97:0/581079247 shutdown_connections 2024-03-27T10:55:17.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:16.439+0000 7f859e200700 1 --2- 172.21.15.97:0/581079247 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8598103050 0x7f859810f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:17.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:16.439+0000 7f859e200700 1 --2- 172.21.15.97:0/581079247 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8598108670 0x7f8598108a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:17.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:16.439+0000 7f859e200700 1 --2- 172.21.15.97:0/581079247 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8598102690 0x7f8598102b10 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:17.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:16.439+0000 7f859e200700 1 -- 172.21.15.97:0/581079247 >> 172.21.15.97:0/581079247 conn(0x7f8598079770 msgr2=0x7f8598079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:55:17.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:16.439+0000 7f859e200700 1 -- 172.21.15.97:0/581079247 shutdown_connections 2024-03-27T10:55:17.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:16.439+0000 7f859e200700 1 -- 172.21.15.97:0/581079247 wait complete. 2024-03-27T10:55:17.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:16.440+0000 7f859e200700 1 Processor -- start 2024-03-27T10:55:17.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:16.440+0000 7f859e200700 1 -- start start 2024-03-27T10:55:17.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:16.440+0000 7f859e200700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8598103050 0x7f859819b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:17.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:16.440+0000 7f859e200700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8598108670 0x7f859819b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:17.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:16.440+0000 7f859e200700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f859819fa50 0x7f859819fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:17.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:16.440+0000 7f859e200700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f85981127f0 con 0x7f859819fa50 2024-03-27T10:55:17.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:16.440+0000 7f859e200700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f8598112670 con 0x7f8598103050 2024-03-27T10:55:17.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:16.440+0000 7f859e200700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f8598112970 con 0x7f8598108670 2024-03-27T10:55:17.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:16.441+0000 7f8597e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f859819fa50 0x7f859819fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:17.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:16.441+0000 7f8597e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f859819fa50 0x7f859819fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37090/0 (socket says 172.21.15.97:37090) 2024-03-27T10:55:17.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:16.441+0000 7f8597e00700 1 -- 172.21.15.97:0/1873154819 learned_addr learned my addr 172.21.15.97:0/1873154819 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:55:17.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:16.441+0000 7f8597400700 1 --2- 172.21.15.97:0/1873154819 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8598103050 0x7f859819b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:17.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:16.441+0000 7f8596a00700 1 --2- 172.21.15.97:0/1873154819 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8598108670 0x7f859819b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:17.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:16.441+0000 7f8597e00700 1 -- 172.21.15.97:0/1873154819 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8598108670 msgr2=0x7f859819b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:17.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:16.441+0000 7f8597e00700 1 --2- 172.21.15.97:0/1873154819 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8598108670 0x7f859819b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:17.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:16.441+0000 7f8597e00700 1 -- 172.21.15.97:0/1873154819 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8598103050 msgr2=0x7f859819b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:17.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:16.441+0000 7f8597e00700 1 --2- 172.21.15.97:0/1873154819 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8598103050 0x7f859819b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:17.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:16.441+0000 7f8597e00700 1 -- 172.21.15.97:0/1873154819 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85981a0570 con 0x7f859819fa50 2024-03-27T10:55:17.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:16.441+0000 7f8597400700 1 --2- 172.21.15.97:0/1873154819 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8598103050 0x7f859819b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:55:17.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:16.441+0000 7f8596a00700 1 --2- 172.21.15.97:0/1873154819 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8598108670 0x7f859819b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:55:17.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:16.441+0000 7f8597e00700 1 --2- 172.21.15.97:0/1873154819 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f859819fa50 0x7f859819fed0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f8590007ee0 tx=0x7f859000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:55:17.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:16.442+0000 7f858fe00700 1 -- 172.21.15.97:0/1873154819 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f859000dbd0 con 0x7f859819fa50 2024-03-27T10:55:17.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:16.442+0000 7f858fe00700 1 -- 172.21.15.97:0/1873154819 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f859000dd70 con 0x7f859819fa50 2024-03-27T10:55:17.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:16.442+0000 7f859e200700 1 -- 172.21.15.97:0/1873154819 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85981a0860 con 0x7f859819fa50 2024-03-27T10:55:17.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:16.442+0000 7f858fe00700 1 -- 172.21.15.97:0/1873154819 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8590005a20 con 0x7f859819fa50 2024-03-27T10:55:17.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:16.442+0000 7f859e200700 1 -- 172.21.15.97:0/1873154819 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85981a8040 con 0x7f859819fa50 2024-03-27T10:55:17.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:16.443+0000 7f859e200700 1 -- 172.21.15.97:0/1873154819 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8598069300 con 0x7f859819fa50 2024-03-27T10:55:17.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:16.445+0000 7f858fe00700 1 -- 172.21.15.97:0/1873154819 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138421+0+0 (secure 0 0 0) 0x7f859001a020 con 0x7f859819fa50 2024-03-27T10:55:17.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:16.447+0000 7f858fe00700 1 --2- 172.21.15.97:0/1873154819 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f857c0a37b0 0x7f857c0a5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:17.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:16.448+0000 7f858fe00700 1 -- 172.21.15.97:0/1873154819 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f85900cf670 con 0x7f859819fa50 2024-03-27T10:55:17.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:16.448+0000 7f8597400700 1 --2- 172.21.15.97:0/1873154819 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f857c0a37b0 0x7f857c0a5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:17.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:16.448+0000 7f8597400700 1 --2- 172.21.15.97:0/1873154819 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f857c0a37b0 0x7f857c0a5c70 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8588005fd0 tx=0x7f8588005d40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:55:17.594 INFO:teuthology.orchestra.run.smithi097.stdout:231928234042 2024-03-27T10:55:17.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:16.452+0000 7f858fe00700 1 -- 172.21.15.97:0/1873154819 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f8590010070 con 0x7f859819fa50 2024-03-27T10:55:17.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:16.682+0000 7f859e200700 1 -- 172.21.15.97:0/1873154819 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f85981a8300 con 0x7f859819fa50 2024-03-27T10:55:17.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:16.682+0000 7f858fe00700 1 -- 172.21.15.97:0/1873154819 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f859009c090 con 0x7f859819fa50 2024-03-27T10:55:17.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:16.685+0000 7f859e200700 1 -- 172.21.15.97:0/1873154819 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f857c0a37b0 msgr2=0x7f857c0a5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:17.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:16.685+0000 7f859e200700 1 --2- 172.21.15.97:0/1873154819 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f857c0a37b0 0x7f857c0a5c70 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8588005fd0 tx=0x7f8588005d40 comp rx=0 tx=0).stop 2024-03-27T10:55:17.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:16.685+0000 7f859e200700 1 -- 172.21.15.97:0/1873154819 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f859819fa50 msgr2=0x7f859819fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:17.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:16.685+0000 7f859e200700 1 --2- 172.21.15.97:0/1873154819 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f859819fa50 0x7f859819fed0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f8590007ee0 tx=0x7f859000d3b0 comp rx=0 tx=0).stop 2024-03-27T10:55:17.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:16.685+0000 7f859e200700 1 -- 172.21.15.97:0/1873154819 shutdown_connections 2024-03-27T10:55:17.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:16.685+0000 7f859e200700 1 --2- 172.21.15.97:0/1873154819 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f857c0a37b0 0x7f857c0a5c70 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:17.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:16.685+0000 7f859e200700 1 --2- 172.21.15.97:0/1873154819 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f859819fa50 0x7f859819fed0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:17.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:16.685+0000 7f859e200700 1 --2- 172.21.15.97:0/1873154819 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8598103050 0x7f859819b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:17.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:16.685+0000 7f859e200700 1 --2- 172.21.15.97:0/1873154819 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8598108670 0x7f859819b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:17.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:16.685+0000 7f859e200700 1 -- 172.21.15.97:0/1873154819 >> 172.21.15.97:0/1873154819 conn(0x7f8598079770 msgr2=0x7f8598101320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:55:17.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:16.685+0000 7f859e200700 1 -- 172.21.15.97:0/1873154819 shutdown_connections 2024-03-27T10:55:17.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:16.685+0000 7f859e200700 1 -- 172.21.15.97:0/1873154819 wait complete. 2024-03-27T10:55:18.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.088+0000 7f71d0400700 1 -- 172.21.15.97:0/3513417068 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71c810f040 msgr2=0x7f71c8111430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:18.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.088+0000 7f71d0400700 1 --2- 172.21.15.97:0/3513417068 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71c810f040 0x7f71c8111430 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f71c000b0a0 tx=0x7f71c000b3b0 comp rx=0 tx=0).stop 2024-03-27T10:55:18.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.088+0000 7f71d0400700 1 -- 172.21.15.97:0/3513417068 shutdown_connections 2024-03-27T10:55:18.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.088+0000 7f71d0400700 1 --2- 172.21.15.97:0/3513417068 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71c810f040 0x7f71c8111430 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:18.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.088+0000 7f71d0400700 1 --2- 172.21.15.97:0/3513417068 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f71c8074870 0x7f71c8074c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:18.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.088+0000 7f71d0400700 1 --2- 172.21.15.97:0/3513417068 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f71c8075190 0x7f71c810eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:18.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.088+0000 7f71d0400700 1 -- 172.21.15.97:0/3513417068 >> 172.21.15.97:0/3513417068 conn(0x7f71c8079770 msgr2=0x7f71c8079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:55:18.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.088+0000 7f71d0400700 1 -- 172.21.15.97:0/3513417068 shutdown_connections 2024-03-27T10:55:18.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.088+0000 7f71d0400700 1 -- 172.21.15.97:0/3513417068 wait complete. 2024-03-27T10:55:18.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.089+0000 7f71d0400700 1 Processor -- start 2024-03-27T10:55:18.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.090+0000 7f71d0400700 1 -- start start 2024-03-27T10:55:18.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.090+0000 7f71d0400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71c8074870 0x7f71c819af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:18.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.090+0000 7f71d0400700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f71c8075190 0x7f71c819b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:18.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.090+0000 7f71d0400700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f71c810f040 0x7f71c819f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:18.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.090+0000 7f71d0400700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f71c81149e0 con 0x7f71c8074870 2024-03-27T10:55:18.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.090+0000 7f71d0400700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f71c8114860 con 0x7f71c810f040 2024-03-27T10:55:18.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.090+0000 7f71d0400700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f71c8114b60 con 0x7f71c8075190 2024-03-27T10:55:18.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.090+0000 7f71cda00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71c8074870 0x7f71c819af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:18.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.090+0000 7f71cda00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71c8074870 0x7f71c819af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37100/0 (socket says 172.21.15.97:37100) 2024-03-27T10:55:18.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.090+0000 7f71cda00700 1 -- 172.21.15.97:0/724967155 learned_addr learned my addr 172.21.15.97:0/724967155 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:55:18.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.091+0000 7f71cd000700 1 --2- 172.21.15.97:0/724967155 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f71c8075190 0x7f71c819b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:18.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.091+0000 7f71cda00700 1 -- 172.21.15.97:0/724967155 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f71c8075190 msgr2=0x7f71c819b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:18.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.091+0000 7f71ce400700 1 --2- 172.21.15.97:0/724967155 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f71c810f040 0x7f71c819f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:18.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.091+0000 7f71cda00700 1 --2- 172.21.15.97:0/724967155 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f71c8075190 0x7f71c819b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:18.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.091+0000 7f71cda00700 1 -- 172.21.15.97:0/724967155 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f71c810f040 msgr2=0x7f71c819f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:18.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.091+0000 7f71cda00700 1 --2- 172.21.15.97:0/724967155 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f71c810f040 0x7f71c819f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:18.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.091+0000 7f71cda00700 1 -- 172.21.15.97:0/724967155 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71c819ff30 con 0x7f71c8074870 2024-03-27T10:55:18.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.091+0000 7f71cd000700 1 --2- 172.21.15.97:0/724967155 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f71c8075190 0x7f71c819b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:55:18.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.091+0000 7f71ce400700 1 --2- 172.21.15.97:0/724967155 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f71c810f040 0x7f71c819f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:55:18.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.091+0000 7f71cda00700 1 --2- 172.21.15.97:0/724967155 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71c8074870 0x7f71c819af70 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f71bc00d950 tx=0x7f71bc00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:55:18.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.091+0000 7f71c6000700 1 -- 172.21.15.97:0/724967155 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71bc016070 con 0x7f71c8074870 2024-03-27T10:55:18.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.091+0000 7f71d0400700 1 -- 172.21.15.97:0/724967155 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71c81a0220 con 0x7f71c8074870 2024-03-27T10:55:18.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.091+0000 7f71c6000700 1 -- 172.21.15.97:0/724967155 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f71bc010460 con 0x7f71c8074870 2024-03-27T10:55:18.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.091+0000 7f71c6000700 1 -- 172.21.15.97:0/724967155 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71bc00f620 con 0x7f71c8074870 2024-03-27T10:55:18.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.091+0000 7f71d0400700 1 -- 172.21.15.97:0/724967155 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71c81a7aa0 con 0x7f71c8074870 2024-03-27T10:55:18.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.092+0000 7f71d0400700 1 -- 172.21.15.97:0/724967155 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71c8069300 con 0x7f71c8074870 2024-03-27T10:55:18.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.095+0000 7f71c6000700 1 -- 172.21.15.97:0/724967155 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138421+0+0 (secure 0 0 0) 0x7f71bc019070 con 0x7f71c8074870 2024-03-27T10:55:18.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.096+0000 7f71c6000700 1 --2- 172.21.15.97:0/724967155 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f71b00a37b0 0x7f71b00a5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:18.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.096+0000 7f71c6000700 1 -- 172.21.15.97:0/724967155 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f71bc0cfa90 con 0x7f71c8074870 2024-03-27T10:55:18.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.096+0000 7f71cd000700 1 --2- 172.21.15.97:0/724967155 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f71b00a37b0 0x7f71b00a5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:18.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.097+0000 7f71cd000700 1 --2- 172.21.15.97:0/724967155 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f71b00a37b0 0x7f71b00a5c70 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f71b4005fd0 tx=0x7f71b4005d40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:55:18.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.103+0000 7f71c6000700 1 -- 172.21.15.97:0/724967155 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f71bc09c510 con 0x7f71c8074870 2024-03-27T10:55:18.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.338+0000 7f71d0400700 1 -- 172.21.15.97:0/724967155 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f71c802dd30 con 0x7f71c8074870 2024-03-27T10:55:18.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.339+0000 7f71c6000700 1 -- 172.21.15.97:0/724967155 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f71bc09be10 con 0x7f71c8074870 2024-03-27T10:55:18.312 INFO:teuthology.orchestra.run.smithi097.stdout:150323855440 2024-03-27T10:55:18.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.343+0000 7f71d0400700 1 -- 172.21.15.97:0/724967155 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f71b00a37b0 msgr2=0x7f71b00a5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:18.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.343+0000 7f71d0400700 1 --2- 172.21.15.97:0/724967155 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f71b00a37b0 0x7f71b00a5c70 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f71b4005fd0 tx=0x7f71b4005d40 comp rx=0 tx=0).stop 2024-03-27T10:55:18.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.343+0000 7f71d0400700 1 -- 172.21.15.97:0/724967155 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71c8074870 msgr2=0x7f71c819af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:18.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.343+0000 7f71d0400700 1 --2- 172.21.15.97:0/724967155 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71c8074870 0x7f71c819af70 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f71bc00d950 tx=0x7f71bc00dd10 comp rx=0 tx=0).stop 2024-03-27T10:55:18.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.343+0000 7f71d0400700 1 -- 172.21.15.97:0/724967155 shutdown_connections 2024-03-27T10:55:18.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.343+0000 7f71d0400700 1 --2- 172.21.15.97:0/724967155 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f71b00a37b0 0x7f71b00a5c70 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:18.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.343+0000 7f71d0400700 1 --2- 172.21.15.97:0/724967155 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f71c810f040 0x7f71c819f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:18.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.343+0000 7f71d0400700 1 --2- 172.21.15.97:0/724967155 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71c8074870 0x7f71c819af70 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:18.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.343+0000 7f71d0400700 1 --2- 172.21.15.97:0/724967155 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f71c8075190 0x7f71c819b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:18.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.343+0000 7f71d0400700 1 -- 172.21.15.97:0/724967155 >> 172.21.15.97:0/724967155 conn(0x7f71c8079770 msgr2=0x7f71c80ff8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:55:18.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.343+0000 7f71d0400700 1 -- 172.21.15.97:0/724967155 shutdown_connections 2024-03-27T10:55:18.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.343+0000 7f71d0400700 1 -- 172.21.15.97:0/724967155 wait complete. 2024-03-27T10:55:19.387 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:55:19 smithi097 ceph-mon[31859]: pgmap v401: 97 pgs: 97 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-27T10:55:19.387 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:55:19 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/724967155' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-27T10:55:19.387 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:55:19 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/3363413056' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-27T10:55:19.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:55:19 smithi177 ceph-mon[35294]: pgmap v401: 97 pgs: 97 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-27T10:55:19.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:55:19 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/724967155' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-27T10:55:19.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:55:19 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/3363413056' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-27T10:55:19.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.438+0000 7f698e400700 1 -- 172.21.15.97:0/3711360077 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6988102690 msgr2=0x7f6988102b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:19.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.438+0000 7f698e400700 1 --2- 172.21.15.97:0/3711360077 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6988102690 0x7f6988102b10 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6970009990 tx=0x7f6970009ca0 comp rx=0 tx=0).stop 2024-03-27T10:55:19.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.438+0000 7f698e400700 1 -- 172.21.15.97:0/3711360077 shutdown_connections 2024-03-27T10:55:19.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.438+0000 7f698e400700 1 --2- 172.21.15.97:0/3711360077 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6988103050 0x7f698810f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:19.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.438+0000 7f698e400700 1 --2- 172.21.15.97:0/3711360077 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6988108680 0x7f6988108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:19.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.438+0000 7f698e400700 1 --2- 172.21.15.97:0/3711360077 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6988102690 0x7f6988102b10 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:19.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.438+0000 7f698e400700 1 -- 172.21.15.97:0/3711360077 >> 172.21.15.97:0/3711360077 conn(0x7f69880797a0 msgr2=0x7f6988079bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:55:19.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.438+0000 7f698e400700 1 -- 172.21.15.97:0/3711360077 shutdown_connections 2024-03-27T10:55:19.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.438+0000 7f698e400700 1 -- 172.21.15.97:0/3711360077 wait complete. 2024-03-27T10:55:19.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.439+0000 7f698e400700 1 Processor -- start 2024-03-27T10:55:19.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.439+0000 7f698e400700 1 -- start start 2024-03-27T10:55:19.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.440+0000 7f698e400700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6988102690 0x7f698819afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:19.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.440+0000 7f698e400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6988103050 0x7f698819b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:19.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.440+0000 7f698e400700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6988108680 0x7f698819f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:19.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.440+0000 7f698e400700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f69881128d0 con 0x7f6988103050 2024-03-27T10:55:19.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.440+0000 7f698e400700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f6988112750 con 0x7f6988102690 2024-03-27T10:55:19.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.440+0000 7f698e400700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6988112a50 con 0x7f6988108680 2024-03-27T10:55:19.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.440+0000 7f697fe00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6988103050 0x7f698819b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:19.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.440+0000 7f697fe00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6988103050 0x7f698819b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33174/0 (socket says 172.21.15.97:33174) 2024-03-27T10:55:19.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.440+0000 7f697fe00700 1 -- 172.21.15.97:0/1446723600 learned_addr learned my addr 172.21.15.97:0/1446723600 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:55:19.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.440+0000 7f6987400700 1 --2- 172.21.15.97:0/1446723600 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6988102690 0x7f698819afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:19.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.441+0000 7f697fe00700 1 -- 172.21.15.97:0/1446723600 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6988108680 msgr2=0x7f698819f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:19.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.441+0000 7f697fe00700 1 --2- 172.21.15.97:0/1446723600 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6988108680 0x7f698819f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:19.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.441+0000 7f697fe00700 1 -- 172.21.15.97:0/1446723600 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6988102690 msgr2=0x7f698819afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:19.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.441+0000 7f697fe00700 1 --2- 172.21.15.97:0/1446723600 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6988102690 0x7f698819afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:19.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.441+0000 7f697fe00700 1 -- 172.21.15.97:0/1446723600 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f698819ffd0 con 0x7f6988103050 2024-03-27T10:55:19.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.441+0000 7f697fe00700 1 --2- 172.21.15.97:0/1446723600 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6988103050 0x7f698819b500 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f6970000c00 tx=0x7f69700057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:55:19.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.441+0000 7f6984c00700 1 -- 172.21.15.97:0/1446723600 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6970027070 con 0x7f6988103050 2024-03-27T10:55:19.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.441+0000 7f698e400700 1 -- 172.21.15.97:0/1446723600 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69881a0260 con 0x7f6988103050 2024-03-27T10:55:19.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.441+0000 7f6984c00700 1 -- 172.21.15.97:0/1446723600 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6970003f10 con 0x7f6988103050 2024-03-27T10:55:19.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.441+0000 7f6984c00700 1 -- 172.21.15.97:0/1446723600 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6970022cd0 con 0x7f6988103050 2024-03-27T10:55:19.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.441+0000 7f698e400700 1 -- 172.21.15.97:0/1446723600 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69881a7b40 con 0x7f6988103050 2024-03-27T10:55:19.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.442+0000 7f698e400700 1 -- 172.21.15.97:0/1446723600 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6988069300 con 0x7f6988103050 2024-03-27T10:55:19.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.445+0000 7f6984c00700 1 -- 172.21.15.97:0/1446723600 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138421+0+0 (secure 0 0 0) 0x7f697002a070 con 0x7f6988103050 2024-03-27T10:55:19.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.446+0000 7f6984c00700 1 --2- 172.21.15.97:0/1446723600 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f696c0a37b0 0x7f696c0a5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:19.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.446+0000 7f6984c00700 1 -- 172.21.15.97:0/1446723600 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f69700d15d0 con 0x7f6988103050 2024-03-27T10:55:19.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.447+0000 7f6987400700 1 --2- 172.21.15.97:0/1446723600 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f696c0a37b0 0x7f696c0a5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:19.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.447+0000 7f6987400700 1 --2- 172.21.15.97:0/1446723600 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f696c0a37b0 0x7f696c0a5c70 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6978003f10 tx=0x7f6978009180 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:55:19.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.449+0000 7f6984c00700 1 -- 172.21.15.97:0/1446723600 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f6970013070 con 0x7f6988103050 2024-03-27T10:55:19.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:55:19 smithi114 ceph-mon[35338]: pgmap v401: 97 pgs: 97 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-27T10:55:19.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:55:19 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/724967155' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-27T10:55:19.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:55:19 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/3363413056' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-27T10:55:19.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.563+0000 7fd49cc00700 1 -- 172.21.15.97:0/4013085082 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd498108620 msgr2=0x7fd498108a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:19.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.563+0000 7fd49cc00700 1 --2- 172.21.15.97:0/4013085082 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd498108620 0x7fd498108a00 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fd488009990 tx=0x7fd488009ca0 comp rx=0 tx=0).stop 2024-03-27T10:55:19.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.564+0000 7fd49cc00700 1 -- 172.21.15.97:0/4013085082 shutdown_connections 2024-03-27T10:55:19.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.564+0000 7fd49cc00700 1 --2- 172.21.15.97:0/4013085082 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd498102fe0 0x7fd49810f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:19.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.564+0000 7fd49cc00700 1 --2- 172.21.15.97:0/4013085082 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd498108620 0x7fd498108a00 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:19.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.564+0000 7fd49cc00700 1 --2- 172.21.15.97:0/4013085082 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd498102620 0x7fd498102aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:19.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.564+0000 7fd49cc00700 1 -- 172.21.15.97:0/4013085082 >> 172.21.15.97:0/4013085082 conn(0x7fd4980fe0f0 msgr2=0x7fd498100510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:55:19.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.564+0000 7fd49cc00700 1 -- 172.21.15.97:0/4013085082 shutdown_connections 2024-03-27T10:55:19.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.564+0000 7fd49cc00700 1 -- 172.21.15.97:0/4013085082 wait complete. 2024-03-27T10:55:19.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.565+0000 7fd49cc00700 1 Processor -- start 2024-03-27T10:55:19.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.565+0000 7fd49cc00700 1 -- start start 2024-03-27T10:55:19.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.565+0000 7fd49cc00700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd498102620 0x7fd49819af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:19.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.565+0000 7fd49cc00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd498102fe0 0x7fd49819b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:19.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.565+0000 7fd49cc00700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd498108620 0x7fd49819f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:19.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.565+0000 7fd49cc00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd498112850 con 0x7fd498102fe0 2024-03-27T10:55:19.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.565+0000 7fd49cc00700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd4981126d0 con 0x7fd498108620 2024-03-27T10:55:19.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.565+0000 7fd49cc00700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd4981129d0 con 0x7fd498102620 2024-03-27T10:55:19.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.566+0000 7fd495400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd498102fe0 0x7fd49819b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:19.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.566+0000 7fd495400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd498102fe0 0x7fd49819b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33192/0 (socket says 172.21.15.97:33192) 2024-03-27T10:55:19.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.566+0000 7fd495400700 1 -- 172.21.15.97:0/2868713377 learned_addr learned my addr 172.21.15.97:0/2868713377 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:55:19.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.566+0000 7fd496800700 1 --2- 172.21.15.97:0/2868713377 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd498108620 0x7fd49819f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:19.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.566+0000 7fd495e00700 1 --2- 172.21.15.97:0/2868713377 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd498102620 0x7fd49819af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:19.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.566+0000 7fd495400700 1 -- 172.21.15.97:0/2868713377 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd498102620 msgr2=0x7fd49819af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:19.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.566+0000 7fd495400700 1 --2- 172.21.15.97:0/2868713377 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd498102620 0x7fd49819af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:19.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.566+0000 7fd495400700 1 -- 172.21.15.97:0/2868713377 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd498108620 msgr2=0x7fd49819f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:19.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.566+0000 7fd495400700 1 --2- 172.21.15.97:0/2868713377 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd498108620 0x7fd49819f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:19.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.566+0000 7fd495400700 1 -- 172.21.15.97:0/2868713377 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd49819ff10 con 0x7fd498102fe0 2024-03-27T10:55:19.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.566+0000 7fd496800700 1 --2- 172.21.15.97:0/2868713377 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd498108620 0x7fd49819f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T10:55:19.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.566+0000 7fd495e00700 1 --2- 172.21.15.97:0/2868713377 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd498102620 0x7fd49819af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T10:55:19.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.567+0000 7fd495400700 1 --2- 172.21.15.97:0/2868713377 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd498102fe0 0x7fd49819b440 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd48400d950 tx=0x7fd48400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:55:19.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.567+0000 7fd48ea00700 1 -- 172.21.15.97:0/2868713377 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd484016070 con 0x7fd498102fe0 2024-03-27T10:55:19.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.567+0000 7fd48ea00700 1 -- 172.21.15.97:0/2868713377 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd484010460 con 0x7fd498102fe0 2024-03-27T10:55:19.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.567+0000 7fd49cc00700 1 -- 172.21.15.97:0/2868713377 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4981a0200 con 0x7fd498102fe0 2024-03-27T10:55:19.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.567+0000 7fd48ea00700 1 -- 172.21.15.97:0/2868713377 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd48400f620 con 0x7fd498102fe0 2024-03-27T10:55:19.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.567+0000 7fd49cc00700 1 -- 172.21.15.97:0/2868713377 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4981a7a80 con 0x7fd498102fe0 2024-03-27T10:55:19.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.568+0000 7fd49cc00700 1 -- 172.21.15.97:0/2868713377 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd498069300 con 0x7fd498102fe0 2024-03-27T10:55:19.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.570+0000 7fd48ea00700 1 -- 172.21.15.97:0/2868713377 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138421+0+0 (secure 0 0 0) 0x7fd484019070 con 0x7fd498102fe0 2024-03-27T10:55:19.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.571+0000 7fd48ea00700 1 --2- 172.21.15.97:0/2868713377 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fd47c0a37b0 0x7fd47c0a5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:19.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.572+0000 7fd48ea00700 1 -- 172.21.15.97:0/2868713377 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fd4840cfaf0 con 0x7fd498102fe0 2024-03-27T10:55:19.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.572+0000 7fd495e00700 1 --2- 172.21.15.97:0/2868713377 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fd47c0a37b0 0x7fd47c0a5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:19.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.573+0000 7fd495e00700 1 --2- 172.21.15.97:0/2868713377 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fd47c0a37b0 0x7fd47c0a5c70 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd48800b5c0 tx=0x7fd488005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:55:19.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.576+0000 7fd48ea00700 1 -- 172.21.15.97:0/2868713377 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7fd48409c510 con 0x7fd498102fe0 2024-03-27T10:55:19.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.681+0000 7f698e400700 1 -- 172.21.15.97:0/1446723600 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f69881a7d10 con 0x7f6988103050 2024-03-27T10:55:19.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.681+0000 7f6984c00700 1 -- 172.21.15.97:0/1446723600 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f697009e080 con 0x7f6988103050 2024-03-27T10:55:19.654 INFO:teuthology.orchestra.run.smithi097.stdout:34359738473 2024-03-27T10:55:19.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.684+0000 7f698e400700 1 -- 172.21.15.97:0/1446723600 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f696c0a37b0 msgr2=0x7f696c0a5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:19.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.684+0000 7f698e400700 1 --2- 172.21.15.97:0/1446723600 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f696c0a37b0 0x7f696c0a5c70 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6978003f10 tx=0x7f6978009180 comp rx=0 tx=0).stop 2024-03-27T10:55:19.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.684+0000 7f698e400700 1 -- 172.21.15.97:0/1446723600 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6988103050 msgr2=0x7f698819b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:19.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.684+0000 7f698e400700 1 --2- 172.21.15.97:0/1446723600 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6988103050 0x7f698819b500 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f6970000c00 tx=0x7f69700057c0 comp rx=0 tx=0).stop 2024-03-27T10:55:19.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.684+0000 7f698e400700 1 -- 172.21.15.97:0/1446723600 shutdown_connections 2024-03-27T10:55:19.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.684+0000 7f698e400700 1 --2- 172.21.15.97:0/1446723600 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f696c0a37b0 0x7f696c0a5c70 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:19.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.684+0000 7f698e400700 1 --2- 172.21.15.97:0/1446723600 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6988108680 0x7f698819f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:19.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.684+0000 7f698e400700 1 --2- 172.21.15.97:0/1446723600 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6988102690 0x7f698819afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:19.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.684+0000 7f698e400700 1 --2- 172.21.15.97:0/1446723600 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6988103050 0x7f698819b500 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:19.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.684+0000 7f698e400700 1 -- 172.21.15.97:0/1446723600 >> 172.21.15.97:0/1446723600 conn(0x7f69880797a0 msgr2=0x7f698810d910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:55:19.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.684+0000 7f698e400700 1 -- 172.21.15.97:0/1446723600 shutdown_connections 2024-03-27T10:55:19.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.684+0000 7f698e400700 1 -- 172.21.15.97:0/1446723600 wait complete. 2024-03-27T10:55:19.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.808+0000 7fd49cc00700 1 -- 172.21.15.97:0/2868713377 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fd4981a7ce0 con 0x7fd498102fe0 2024-03-27T10:55:19.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.809+0000 7fd48ea00700 1 -- 172.21.15.97:0/2868713377 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fd48409be10 con 0x7fd498102fe0 2024-03-27T10:55:19.782 INFO:teuthology.orchestra.run.smithi097.stdout:210453397568 2024-03-27T10:55:19.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.812+0000 7fd49cc00700 1 -- 172.21.15.97:0/2868713377 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fd47c0a37b0 msgr2=0x7fd47c0a5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:19.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.812+0000 7fd49cc00700 1 --2- 172.21.15.97:0/2868713377 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fd47c0a37b0 0x7fd47c0a5c70 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd48800b5c0 tx=0x7fd488005fd0 comp rx=0 tx=0).stop 2024-03-27T10:55:19.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.812+0000 7fd49cc00700 1 -- 172.21.15.97:0/2868713377 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd498102fe0 msgr2=0x7fd49819b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:19.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.812+0000 7fd49cc00700 1 --2- 172.21.15.97:0/2868713377 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd498102fe0 0x7fd49819b440 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd48400d950 tx=0x7fd48400dd10 comp rx=0 tx=0).stop 2024-03-27T10:55:19.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.812+0000 7fd49cc00700 1 -- 172.21.15.97:0/2868713377 shutdown_connections 2024-03-27T10:55:19.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.812+0000 7fd49cc00700 1 --2- 172.21.15.97:0/2868713377 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fd47c0a37b0 0x7fd47c0a5c70 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:19.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.812+0000 7fd49cc00700 1 --2- 172.21.15.97:0/2868713377 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd498108620 0x7fd49819f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:19.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.812+0000 7fd49cc00700 1 --2- 172.21.15.97:0/2868713377 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd498102620 0x7fd49819af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:19.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.812+0000 7fd49cc00700 1 --2- 172.21.15.97:0/2868713377 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd498102fe0 0x7fd49819b440 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:19.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.812+0000 7fd49cc00700 1 -- 172.21.15.97:0/2868713377 >> 172.21.15.97:0/2868713377 conn(0x7fd4980fe0f0 msgr2=0x7fd49810dd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:55:19.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.812+0000 7fd49cc00700 1 -- 172.21.15.97:0/2868713377 shutdown_connections 2024-03-27T10:55:19.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:19.813+0000 7fd49cc00700 1 -- 172.21.15.97:0/2868713377 wait complete. 2024-03-27T10:55:19.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.196+0000 7fef2ca00700 1 -- 172.21.15.97:0/2703791835 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef28108670 msgr2=0x7fef28108a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:19.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.196+0000 7fef2ca00700 1 --2- 172.21.15.97:0/2703791835 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef28108670 0x7fef28108a50 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fef180099c0 tx=0x7fef18009cd0 comp rx=0 tx=0).stop 2024-03-27T10:55:19.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.196+0000 7fef2ca00700 1 -- 172.21.15.97:0/2703791835 shutdown_connections 2024-03-27T10:55:19.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.196+0000 7fef2ca00700 1 --2- 172.21.15.97:0/2703791835 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fef2806acb0 0x7fef2810f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:19.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.196+0000 7fef2ca00700 1 --2- 172.21.15.97:0/2703791835 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef28108670 0x7fef28108a50 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:19.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.196+0000 7fef2ca00700 1 --2- 172.21.15.97:0/2703791835 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fef2806a2f0 0x7fef2806a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:19.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.196+0000 7fef2ca00700 1 -- 172.21.15.97:0/2703791835 >> 172.21.15.97:0/2703791835 conn(0x7fef28078470 msgr2=0x7fef28078880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:55:19.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.196+0000 7fef2ca00700 1 -- 172.21.15.97:0/2703791835 shutdown_connections 2024-03-27T10:55:19.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.196+0000 7fef2ca00700 1 -- 172.21.15.97:0/2703791835 wait complete. 2024-03-27T10:55:19.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.197+0000 7fef2ca00700 1 Processor -- start 2024-03-27T10:55:19.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.197+0000 7fef2ca00700 1 -- start start 2024-03-27T10:55:19.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.198+0000 7fef2ca00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef2806a2f0 0x7fef2819e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:19.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.198+0000 7fef2ca00700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fef2806acb0 0x7fef2819eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:19.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.198+0000 7fef2ca00700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fef28108670 0x7fef28198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:19.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.198+0000 7fef2ca00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fef28112780 con 0x7fef2806a2f0 2024-03-27T10:55:19.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.198+0000 7fef2ca00700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fef28112600 con 0x7fef28108670 2024-03-27T10:55:19.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.198+0000 7fef2ca00700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fef28112900 con 0x7fef2806acb0 2024-03-27T10:55:19.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.198+0000 7fef25e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef2806a2f0 0x7fef2819e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:19.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.198+0000 7fef25e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef2806a2f0 0x7fef2819e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37116/0 (socket says 172.21.15.97:37116) 2024-03-27T10:55:19.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.198+0000 7fef25e00700 1 -- 172.21.15.97:0/3363413056 learned_addr learned my addr 172.21.15.97:0/3363413056 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:55:19.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.198+0000 7fef25400700 1 --2- 172.21.15.97:0/3363413056 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fef2806acb0 0x7fef2819eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:19.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.198+0000 7fef26800700 1 --2- 172.21.15.97:0/3363413056 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fef28108670 0x7fef28198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:19.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.199+0000 7fef25e00700 1 -- 172.21.15.97:0/3363413056 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fef2806acb0 msgr2=0x7fef2819eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:19.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.199+0000 7fef25e00700 1 --2- 172.21.15.97:0/3363413056 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fef2806acb0 0x7fef2819eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:19.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.199+0000 7fef25e00700 1 -- 172.21.15.97:0/3363413056 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fef28108670 msgr2=0x7fef28198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:19.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.199+0000 7fef25e00700 1 --2- 172.21.15.97:0/3363413056 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fef28108670 0x7fef28198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:19.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.199+0000 7fef25e00700 1 -- 172.21.15.97:0/3363413056 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef28199180 con 0x7fef2806a2f0 2024-03-27T10:55:19.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.199+0000 7fef26800700 1 --2- 172.21.15.97:0/3363413056 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fef28108670 0x7fef28198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T10:55:19.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.199+0000 7fef25e00700 1 --2- 172.21.15.97:0/3363413056 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef2806a2f0 0x7fef2819e860 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fef18005b40 tx=0x7fef180057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:55:19.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.199+0000 7fef1ea00700 1 -- 172.21.15.97:0/3363413056 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef18026070 con 0x7fef2806a2f0 2024-03-27T10:55:19.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.199+0000 7fef1ea00700 1 -- 172.21.15.97:0/3363413056 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fef18003f50 con 0x7fef2806a2f0 2024-03-27T10:55:19.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.199+0000 7fef1ea00700 1 -- 172.21.15.97:0/3363413056 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef1800fd70 con 0x7fef2806a2f0 2024-03-27T10:55:19.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.200+0000 7fef2ca00700 1 -- 172.21.15.97:0/3363413056 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef28199410 con 0x7fef2806a2f0 2024-03-27T10:55:19.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.200+0000 7fef2ca00700 1 -- 172.21.15.97:0/3363413056 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef281a5730 con 0x7fef2806a2f0 2024-03-27T10:55:19.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.201+0000 7fef2ca00700 1 -- 172.21.15.97:0/3363413056 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef28103d40 con 0x7fef2806a2f0 2024-03-27T10:55:19.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.203+0000 7fef1ea00700 1 -- 172.21.15.97:0/3363413056 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138421+0+0 (secure 0 0 0) 0x7fef18029070 con 0x7fef2806a2f0 2024-03-27T10:55:19.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.205+0000 7fef1ea00700 1 --2- 172.21.15.97:0/3363413056 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fef0c0a37b0 0x7fef0c0a5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:19.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.205+0000 7fef1ea00700 1 -- 172.21.15.97:0/3363413056 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fef180d0b20 con 0x7fef2806a2f0 2024-03-27T10:55:19.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.206+0000 7fef25400700 1 --2- 172.21.15.97:0/3363413056 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fef0c0a37b0 0x7fef0c0a5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:19.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.206+0000 7fef25400700 1 --2- 172.21.15.97:0/3363413056 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fef0c0a37b0 0x7fef0c0a5c70 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fef28078a70 tx=0x7fef1400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:55:19.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.207+0000 7fef1ea00700 1 -- 172.21.15.97:0/3363413056 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7fef18030080 con 0x7fef2806a2f0 2024-03-27T10:55:19.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.453+0000 7fef2ca00700 1 -- 172.21.15.97:0/3363413056 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7fef2802dd30 con 0x7fef2806a2f0 2024-03-27T10:55:19.927 INFO:teuthology.orchestra.run.smithi097.stdout:365072220191 2024-03-27T10:55:19.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.454+0000 7fef1ea00700 1 -- 172.21.15.97:0/3363413056 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fef18013070 con 0x7fef2806a2f0 2024-03-27T10:55:19.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.457+0000 7fef2ca00700 1 -- 172.21.15.97:0/3363413056 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fef0c0a37b0 msgr2=0x7fef0c0a5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:19.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.457+0000 7fef2ca00700 1 --2- 172.21.15.97:0/3363413056 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fef0c0a37b0 0x7fef0c0a5c70 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fef28078a70 tx=0x7fef1400b410 comp rx=0 tx=0).stop 2024-03-27T10:55:19.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.457+0000 7fef2ca00700 1 -- 172.21.15.97:0/3363413056 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef2806a2f0 msgr2=0x7fef2819e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:19.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.457+0000 7fef2ca00700 1 --2- 172.21.15.97:0/3363413056 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef2806a2f0 0x7fef2819e860 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fef18005b40 tx=0x7fef180057c0 comp rx=0 tx=0).stop 2024-03-27T10:55:19.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.457+0000 7fef2ca00700 1 -- 172.21.15.97:0/3363413056 shutdown_connections 2024-03-27T10:55:19.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.457+0000 7fef2ca00700 1 --2- 172.21.15.97:0/3363413056 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fef0c0a37b0 0x7fef0c0a5c70 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:19.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.457+0000 7fef2ca00700 1 --2- 172.21.15.97:0/3363413056 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fef28108670 0x7fef28198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:19.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.457+0000 7fef2ca00700 1 --2- 172.21.15.97:0/3363413056 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef2806a2f0 0x7fef2819e860 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:19.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.457+0000 7fef2ca00700 1 --2- 172.21.15.97:0/3363413056 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fef2806acb0 0x7fef2819eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:19.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.457+0000 7fef2ca00700 1 -- 172.21.15.97:0/3363413056 >> 172.21.15.97:0/3363413056 conn(0x7fef28078470 msgr2=0x7fef28100120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:55:19.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.457+0000 7fef2ca00700 1 -- 172.21.15.97:0/3363413056 shutdown_connections 2024-03-27T10:55:19.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:18.457+0000 7fef2ca00700 1 -- 172.21.15.97:0/3363413056 wait complete. 2024-03-27T10:55:20.256 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:55:20 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/1446723600' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-27T10:55:20.256 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:55:20 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/2868713377' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-27T10:55:20.426 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:55:20 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/1446723600' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-27T10:55:20.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:55:20 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/2868713377' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-27T10:55:20.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:55:20 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/1446723600' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-27T10:55:20.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:55:20 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/2868713377' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-27T10:55:21.208 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738469 got 34359738473 for osd.0 2024-03-27T10:55:21.208 DEBUG:teuthology.parallel:result is None 2024-03-27T10:55:21.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:55:21 smithi097 ceph-mon[31859]: pgmap v402: 97 pgs: 97 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-27T10:55:21.426 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:55:21 smithi177 ceph-mon[35294]: pgmap v402: 97 pgs: 97 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-27T10:55:21.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:55:21 smithi114 ceph-mon[35338]: pgmap v402: 97 pgs: 97 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-27T10:55:21.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:21.820+0000 7fc82fe00700 1 -- 172.21.15.97:0/1111378184 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc82810f040 msgr2=0x7fc828111430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:21.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:21.820+0000 7fc82fe00700 1 --2- 172.21.15.97:0/1111378184 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc82810f040 0x7fc828111430 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fc82000b0a0 tx=0x7fc82000b3b0 comp rx=0 tx=0).stop 2024-03-27T10:55:21.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:21.820+0000 7fc82fe00700 1 -- 172.21.15.97:0/1111378184 shutdown_connections 2024-03-27T10:55:21.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:21.820+0000 7fc82fe00700 1 --2- 172.21.15.97:0/1111378184 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc82810f040 0x7fc828111430 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:21.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:21.820+0000 7fc82fe00700 1 --2- 172.21.15.97:0/1111378184 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc828074870 0x7fc828074c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:21.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:21.820+0000 7fc82fe00700 1 --2- 172.21.15.97:0/1111378184 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc828075190 0x7fc82810eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:21.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:21.820+0000 7fc82fe00700 1 -- 172.21.15.97:0/1111378184 >> 172.21.15.97:0/1111378184 conn(0x7fc8280797a0 msgr2=0x7fc828079bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:55:21.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:21.820+0000 7fc82fe00700 1 -- 172.21.15.97:0/1111378184 shutdown_connections 2024-03-27T10:55:21.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:21.820+0000 7fc82fe00700 1 -- 172.21.15.97:0/1111378184 wait complete. 2024-03-27T10:55:21.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:21.821+0000 7fc82fe00700 1 Processor -- start 2024-03-27T10:55:21.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:21.822+0000 7fc82fe00700 1 -- start start 2024-03-27T10:55:21.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:21.822+0000 7fc82fe00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc828074870 0x7fc82819af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:21.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:21.822+0000 7fc82fe00700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc828075190 0x7fc82819b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:21.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:21.822+0000 7fc82fe00700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc82810f040 0x7fc82819f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:21.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:21.822+0000 7fc82fe00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc828114a90 con 0x7fc828074870 2024-03-27T10:55:21.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:21.822+0000 7fc82fe00700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc828114910 con 0x7fc82810f040 2024-03-27T10:55:21.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:21.822+0000 7fc82fe00700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc828114c10 con 0x7fc828075190 2024-03-27T10:55:21.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:21.823+0000 7fc82d400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc828074870 0x7fc82819af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:21.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:21.823+0000 7fc82d400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc828074870 0x7fc82819af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33200/0 (socket says 172.21.15.97:33200) 2024-03-27T10:55:21.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:21.823+0000 7fc82d400700 1 -- 172.21.15.97:0/753964683 learned_addr learned my addr 172.21.15.97:0/753964683 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:55:21.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:21.823+0000 7fc82ca00700 1 --2- 172.21.15.97:0/753964683 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc828075190 0x7fc82819b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:21.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:21.823+0000 7fc82de00700 1 --2- 172.21.15.97:0/753964683 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc82810f040 0x7fc82819f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:21.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:21.823+0000 7fc82d400700 1 -- 172.21.15.97:0/753964683 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc828075190 msgr2=0x7fc82819b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:21.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:21.823+0000 7fc82d400700 1 --2- 172.21.15.97:0/753964683 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc828075190 0x7fc82819b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:21.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:21.823+0000 7fc82d400700 1 -- 172.21.15.97:0/753964683 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc82810f040 msgr2=0x7fc82819f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:21.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:21.823+0000 7fc82d400700 1 --2- 172.21.15.97:0/753964683 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc82810f040 0x7fc82819f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:21.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:21.823+0000 7fc82d400700 1 -- 172.21.15.97:0/753964683 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc82819ff00 con 0x7fc828074870 2024-03-27T10:55:21.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:21.823+0000 7fc82ca00700 1 --2- 172.21.15.97:0/753964683 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc828075190 0x7fc82819b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T10:55:21.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:21.824+0000 7fc82d400700 1 --2- 172.21.15.97:0/753964683 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc828074870 0x7fc82819af40 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc81c00d950 tx=0x7fc81c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:55:21.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:21.824+0000 7fc826000700 1 -- 172.21.15.97:0/753964683 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc81c016070 con 0x7fc828074870 2024-03-27T10:55:21.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:21.824+0000 7fc82fe00700 1 -- 172.21.15.97:0/753964683 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8281a01f0 con 0x7fc828074870 2024-03-27T10:55:21.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:21.824+0000 7fc826000700 1 -- 172.21.15.97:0/753964683 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc81c009df0 con 0x7fc828074870 2024-03-27T10:55:21.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:21.824+0000 7fc826000700 1 -- 172.21.15.97:0/753964683 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc81c00f3c0 con 0x7fc828074870 2024-03-27T10:55:21.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:21.824+0000 7fc82fe00700 1 -- 172.21.15.97:0/753964683 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8281a7a70 con 0x7fc828074870 2024-03-27T10:55:21.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:21.826+0000 7fc82fe00700 1 -- 172.21.15.97:0/753964683 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc808005330 con 0x7fc828074870 2024-03-27T10:55:21.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:21.829+0000 7fc826000700 1 -- 172.21.15.97:0/753964683 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138421+0+0 (secure 0 0 0) 0x7fc81c019070 con 0x7fc828074870 2024-03-27T10:55:21.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:21.830+0000 7fc826000700 1 --2- 172.21.15.97:0/753964683 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fc8100a3730 0x7fc8100a5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:21.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:21.831+0000 7fc826000700 1 -- 172.21.15.97:0/753964683 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fc81c0cf9c0 con 0x7fc828074870 2024-03-27T10:55:21.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:21.831+0000 7fc82ca00700 1 --2- 172.21.15.97:0/753964683 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fc8100a3730 0x7fc8100a5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:21.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:21.832+0000 7fc82ca00700 1 --2- 172.21.15.97:0/753964683 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fc8100a3730 0x7fc8100a5bf0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc814005950 tx=0x7fc8140058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:55:21.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:21.835+0000 7fc826000700 1 -- 172.21.15.97:0/753964683 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7fc81c09c3e0 con 0x7fc828074870 2024-03-27T10:55:22.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:22.068+0000 7fc82fe00700 1 -- 172.21.15.97:0/753964683 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7fc8080058f0 con 0x7fc828074870 2024-03-27T10:55:22.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:22.069+0000 7fc826000700 1 -- 172.21.15.97:0/753964683 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fc81c09bce0 con 0x7fc828074870 2024-03-27T10:55:22.042 INFO:teuthology.orchestra.run.smithi097.stdout:330712481831 2024-03-27T10:55:22.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:22.073+0000 7fc82fe00700 1 -- 172.21.15.97:0/753964683 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fc8100a3730 msgr2=0x7fc8100a5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:22.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:22.073+0000 7fc82fe00700 1 --2- 172.21.15.97:0/753964683 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fc8100a3730 0x7fc8100a5bf0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc814005950 tx=0x7fc8140058e0 comp rx=0 tx=0).stop 2024-03-27T10:55:22.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:22.073+0000 7fc82fe00700 1 -- 172.21.15.97:0/753964683 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc828074870 msgr2=0x7fc82819af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:22.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:22.073+0000 7fc82fe00700 1 --2- 172.21.15.97:0/753964683 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc828074870 0x7fc82819af40 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc81c00d950 tx=0x7fc81c00dc60 comp rx=0 tx=0).stop 2024-03-27T10:55:22.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:22.074+0000 7fc82fe00700 1 -- 172.21.15.97:0/753964683 shutdown_connections 2024-03-27T10:55:22.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:22.074+0000 7fc82fe00700 1 --2- 172.21.15.97:0/753964683 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fc8100a3730 0x7fc8100a5bf0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:22.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:22.074+0000 7fc82fe00700 1 --2- 172.21.15.97:0/753964683 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc82810f040 0x7fc82819f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:22.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:22.074+0000 7fc82fe00700 1 --2- 172.21.15.97:0/753964683 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc828074870 0x7fc82819af40 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:22.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:22.074+0000 7fc82fe00700 1 --2- 172.21.15.97:0/753964683 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc828075190 0x7fc82819b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:22.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:22.074+0000 7fc82fe00700 1 -- 172.21.15.97:0/753964683 >> 172.21.15.97:0/753964683 conn(0x7fc8280797a0 msgr2=0x7fc8280ff850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:55:22.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:22.074+0000 7fc82fe00700 1 -- 172.21.15.97:0/753964683 shutdown_connections 2024-03-27T10:55:22.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:22.074+0000 7fc82fe00700 1 -- 172.21.15.97:0/753964683 wait complete. 2024-03-27T10:55:22.302 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:55:22 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/753964683' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-27T10:55:22.362 INFO:tasks.cephadm.ceph_manager.ceph:need seq 150323855432 got 150323855440 for osd.4 2024-03-27T10:55:22.362 DEBUG:teuthology.parallel:result is None 2024-03-27T10:55:22.426 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:55:22 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/753964683' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-27T10:55:22.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:55:22 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/753964683' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-27T10:55:22.911 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574941 got 55834574946 for osd.1 2024-03-27T10:55:22.911 DEBUG:teuthology.parallel:result is None 2024-03-27T10:55:23.028 INFO:tasks.cephadm.ceph_manager.ceph:need seq 365072220185 got 365072220191 for osd.11 2024-03-27T10:55:23.028 DEBUG:teuthology.parallel:result is None 2024-03-27T10:55:23.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:55:23 smithi097 ceph-mon[31859]: pgmap v403: 97 pgs: 97 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T10:55:23.426 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:55:23 smithi177 ceph-mon[35294]: pgmap v403: 97 pgs: 97 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T10:55:23.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:55:23 smithi114 ceph-mon[35338]: pgmap v403: 97 pgs: 97 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T10:55:23.725 INFO:tasks.cephadm.ceph_manager.ceph:need seq 210453397561 got 210453397568 for osd.6 2024-03-27T10:55:23.726 DEBUG:teuthology.parallel:result is None 2024-03-27T10:55:24.239 INFO:tasks.cephadm.ceph_manager.ceph:need seq 231928234036 got 231928234042 for osd.7 2024-03-27T10:55:24.239 DEBUG:teuthology.parallel:result is None 2024-03-27T10:55:24.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:23.315+0000 7fd959600700 1 -- 172.21.15.97:0/3732203839 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd95406ad10 msgr2=0x7fd95406b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:24.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:23.315+0000 7fd959600700 1 --2- 172.21.15.97:0/3732203839 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd95406ad10 0x7fd95406b190 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd9400099c0 tx=0x7fd940009cd0 comp rx=0 tx=0).stop 2024-03-27T10:55:24.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:23.316+0000 7fd959600700 1 -- 172.21.15.97:0/3732203839 shutdown_connections 2024-03-27T10:55:24.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:23.316+0000 7fd959600700 1 --2- 172.21.15.97:0/3732203839 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd95406b6d0 0x7fd95410f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:24.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:23.316+0000 7fd959600700 1 --2- 172.21.15.97:0/3732203839 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd9541085e0 0x7fd9541089c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:24.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:23.316+0000 7fd959600700 1 --2- 172.21.15.97:0/3732203839 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd95406ad10 0x7fd95406b190 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:24.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:23.316+0000 7fd959600700 1 -- 172.21.15.97:0/3732203839 >> 172.21.15.97:0/3732203839 conn(0x7fd9540784b0 msgr2=0x7fd9540788c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:55:24.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:23.316+0000 7fd959600700 1 -- 172.21.15.97:0/3732203839 shutdown_connections 2024-03-27T10:55:24.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:23.316+0000 7fd959600700 1 -- 172.21.15.97:0/3732203839 wait complete. 2024-03-27T10:55:24.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:23.317+0000 7fd959600700 1 Processor -- start 2024-03-27T10:55:24.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:23.317+0000 7fd959600700 1 -- start start 2024-03-27T10:55:24.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:23.317+0000 7fd959600700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd95406ad10 0x7fd95419e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:24.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:23.317+0000 7fd959600700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd95406b6d0 0x7fd95419ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:24.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:23.317+0000 7fd959600700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9541085e0 0x7fd954198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:24.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:23.317+0000 7fd959600700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd9541127c0 con 0x7fd95406ad10 2024-03-27T10:55:24.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:23.317+0000 7fd959600700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd954112640 con 0x7fd9541085e0 2024-03-27T10:55:24.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:23.317+0000 7fd959600700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd954112940 con 0x7fd95406b6d0 2024-03-27T10:55:24.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:23.317+0000 7fd952a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd95406ad10 0x7fd95419e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:24.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:23.317+0000 7fd952a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd95406ad10 0x7fd95419e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33212/0 (socket says 172.21.15.97:33212) 2024-03-27T10:55:24.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:23.317+0000 7fd952a00700 1 -- 172.21.15.97:0/2524259156 learned_addr learned my addr 172.21.15.97:0/2524259156 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:55:24.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:23.318+0000 7fd952000700 1 --2- 172.21.15.97:0/2524259156 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd95406b6d0 0x7fd95419ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:24.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:23.318+0000 7fd952a00700 1 -- 172.21.15.97:0/2524259156 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd95406b6d0 msgr2=0x7fd95419ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:24.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:23.318+0000 7fd952a00700 1 --2- 172.21.15.97:0/2524259156 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd95406b6d0 0x7fd95419ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:24.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:23.318+0000 7fd953400700 1 --2- 172.21.15.97:0/2524259156 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9541085e0 0x7fd954198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:24.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:23.318+0000 7fd952a00700 1 -- 172.21.15.97:0/2524259156 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9541085e0 msgr2=0x7fd954198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:24.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:23.318+0000 7fd952a00700 1 --2- 172.21.15.97:0/2524259156 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9541085e0 0x7fd954198b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:24.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:23.318+0000 7fd952a00700 1 -- 172.21.15.97:0/2524259156 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd954199260 con 0x7fd95406ad10 2024-03-27T10:55:24.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:23.318+0000 7fd952a00700 1 --2- 172.21.15.97:0/2524259156 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd95406ad10 0x7fd95419e8f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd94400d980 tx=0x7fd94400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:55:24.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:23.318+0000 7fd94b400700 1 -- 172.21.15.97:0/2524259156 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd944017070 con 0x7fd95406ad10 2024-03-27T10:55:24.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:23.318+0000 7fd94b400700 1 -- 172.21.15.97:0/2524259156 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd944004830 con 0x7fd95406ad10 2024-03-27T10:55:24.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:23.318+0000 7fd94b400700 1 -- 172.21.15.97:0/2524259156 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd944005440 con 0x7fd95406ad10 2024-03-27T10:55:24.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:23.318+0000 7fd959600700 1 -- 172.21.15.97:0/2524259156 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd954199550 con 0x7fd95406ad10 2024-03-27T10:55:24.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:23.319+0000 7fd959600700 1 -- 172.21.15.97:0/2524259156 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9541a57c0 con 0x7fd95406ad10 2024-03-27T10:55:24.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:23.319+0000 7fd959600700 1 -- 172.21.15.97:0/2524259156 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd954103d60 con 0x7fd95406ad10 2024-03-27T10:55:24.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:23.321+0000 7fd94b400700 1 -- 172.21.15.97:0/2524259156 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138421+0+0 (secure 0 0 0) 0x7fd9440055e0 con 0x7fd95406ad10 2024-03-27T10:55:24.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:23.323+0000 7fd94b400700 1 --2- 172.21.15.97:0/2524259156 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fd9380a37b0 0x7fd9380a5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:24.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:23.323+0000 7fd94b400700 1 -- 172.21.15.97:0/2524259156 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fd9440d0130 con 0x7fd95406ad10 2024-03-27T10:55:24.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:23.323+0000 7fd952000700 1 --2- 172.21.15.97:0/2524259156 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fd9380a37b0 0x7fd9380a5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:24.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:23.323+0000 7fd952000700 1 --2- 172.21.15.97:0/2524259156 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fd9380a37b0 0x7fd9380a5c70 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd940006010 tx=0x7fd940000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:55:24.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:23.325+0000 7fd94b400700 1 -- 172.21.15.97:0/2524259156 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7fd944020020 con 0x7fd95406ad10 2024-03-27T10:55:24.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:23.541+0000 7fd959600700 1 -- 172.21.15.97:0/2524259156 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7fd95402dd30 con 0x7fd95406ad10 2024-03-27T10:55:24.249 INFO:teuthology.orchestra.run.smithi097.stdout:304942678061 2024-03-27T10:55:24.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:23.541+0000 7fd94b400700 1 -- 172.21.15.97:0/2524259156 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fd94409cc30 con 0x7fd95406ad10 2024-03-27T10:55:24.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:23.544+0000 7fd959600700 1 -- 172.21.15.97:0/2524259156 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fd9380a37b0 msgr2=0x7fd9380a5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:24.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:23.544+0000 7fd959600700 1 --2- 172.21.15.97:0/2524259156 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fd9380a37b0 0x7fd9380a5c70 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd940006010 tx=0x7fd940000bc0 comp rx=0 tx=0).stop 2024-03-27T10:55:24.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:23.544+0000 7fd959600700 1 -- 172.21.15.97:0/2524259156 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd95406ad10 msgr2=0x7fd95419e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:24.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:23.544+0000 7fd959600700 1 --2- 172.21.15.97:0/2524259156 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd95406ad10 0x7fd95419e8f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd94400d980 tx=0x7fd94400dd40 comp rx=0 tx=0).stop 2024-03-27T10:55:24.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:23.544+0000 7fd959600700 1 -- 172.21.15.97:0/2524259156 shutdown_connections 2024-03-27T10:55:24.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:23.544+0000 7fd959600700 1 --2- 172.21.15.97:0/2524259156 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fd9380a37b0 0x7fd9380a5c70 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:24.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:23.544+0000 7fd959600700 1 --2- 172.21.15.97:0/2524259156 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9541085e0 0x7fd954198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:24.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:23.545+0000 7fd959600700 1 --2- 172.21.15.97:0/2524259156 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd95406ad10 0x7fd95419e8f0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:24.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:23.545+0000 7fd959600700 1 --2- 172.21.15.97:0/2524259156 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd95406b6d0 0x7fd95419ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:24.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:23.545+0000 7fd959600700 1 -- 172.21.15.97:0/2524259156 >> 172.21.15.97:0/2524259156 conn(0x7fd9540784b0 msgr2=0x7fd9541001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:55:24.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:23.545+0000 7fd959600700 1 -- 172.21.15.97:0/2524259156 shutdown_connections 2024-03-27T10:55:24.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:23.545+0000 7fd959600700 1 -- 172.21.15.97:0/2524259156 wait complete. 2024-03-27T10:55:24.426 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:55:24 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/2524259156' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-27T10:55:24.460 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T10:55:24.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:55:24 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/2524259156' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-27T10:55:24.596 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:55:24 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/2524259156' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-27T10:55:24.993 INFO:tasks.cephadm.ceph_manager.ceph:need seq 304942678054 got 304942678061 for osd.9 2024-03-27T10:55:24.993 DEBUG:teuthology.parallel:result is None 2024-03-27T10:55:25.294 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:55:25 smithi097 ceph-mon[31859]: pgmap v404: 97 pgs: 97 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-27T10:55:25.426 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:55:25 smithi177 ceph-mon[35294]: pgmap v404: 97 pgs: 97 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-27T10:55:25.516 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:55:25 smithi114 ceph-mon[35338]: pgmap v404: 97 pgs: 97 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-27T10:55:25.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:25.682+0000 7f553d800700 1 -- 172.21.15.97:0/1890810145 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f55381134b0 msgr2=0x7f55381158a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:25.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:25.682+0000 7f553d800700 1 --2- 172.21.15.97:0/1890810145 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f55381134b0 0x7f55381158a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f553000b0a0 tx=0x7f553000b3b0 comp rx=0 tx=0).stop 2024-03-27T10:55:25.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:25.682+0000 7f553d800700 1 -- 172.21.15.97:0/1890810145 shutdown_connections 2024-03-27T10:55:25.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:25.682+0000 7f553d800700 1 --2- 172.21.15.97:0/1890810145 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f55381134b0 0x7f55381158a0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:25.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:25.682+0000 7f553d800700 1 --2- 172.21.15.97:0/1890810145 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f55380787a0 0x7f5538078b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:25.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:25.682+0000 7f553d800700 1 --2- 172.21.15.97:0/1890810145 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5538079150 0x7f5538112f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:25.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:25.682+0000 7f553d800700 1 -- 172.21.15.97:0/1890810145 >> 172.21.15.97:0/1890810145 conn(0x7f55381002e0 msgr2=0x7f5538102700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:55:25.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:25.682+0000 7f553d800700 1 -- 172.21.15.97:0/1890810145 shutdown_connections 2024-03-27T10:55:25.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:25.682+0000 7f553d800700 1 -- 172.21.15.97:0/1890810145 wait complete. 2024-03-27T10:55:25.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:25.684+0000 7f553d800700 1 Processor -- start 2024-03-27T10:55:25.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:25.684+0000 7f553d800700 1 -- start start 2024-03-27T10:55:25.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:25.684+0000 7f553d800700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f55380787a0 0x7f553819f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:25.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:25.684+0000 7f553d800700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5538079150 0x7f553819f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:25.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:25.684+0000 7f553d800700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f55381134b0 0x7f55381a3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:25.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:25.684+0000 7f553d800700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5538118ec0 con 0x7f5538079150 2024-03-27T10:55:25.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:25.684+0000 7f553d800700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f5538118d40 con 0x7f55380787a0 2024-03-27T10:55:25.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:25.684+0000 7f553d800700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f5538119040 con 0x7f55381134b0 2024-03-27T10:55:25.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:25.685+0000 7f5536000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5538079150 0x7f553819f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:25.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:25.685+0000 7f5536000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5538079150 0x7f553819f9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33234/0 (socket says 172.21.15.97:33234) 2024-03-27T10:55:25.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:25.685+0000 7f5536000700 1 -- 172.21.15.97:0/3354613193 learned_addr learned my addr 172.21.15.97:0/3354613193 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:55:25.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:25.685+0000 7f5536a00700 1 --2- 172.21.15.97:0/3354613193 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f55380787a0 0x7f553819f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:25.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:25.685+0000 7f5537400700 1 --2- 172.21.15.97:0/3354613193 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f55381134b0 0x7f55381a3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:25.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:25.685+0000 7f5536000700 1 -- 172.21.15.97:0/3354613193 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f55381134b0 msgr2=0x7f55381a3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:25.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:25.685+0000 7f5536000700 1 --2- 172.21.15.97:0/3354613193 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f55381134b0 0x7f55381a3cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:25.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:25.685+0000 7f5536000700 1 -- 172.21.15.97:0/3354613193 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f55380787a0 msgr2=0x7f553819f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:25.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:25.685+0000 7f5536000700 1 --2- 172.21.15.97:0/3354613193 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f55380787a0 0x7f553819f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:25.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:25.685+0000 7f5536000700 1 -- 172.21.15.97:0/3354613193 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55381a4430 con 0x7f5538079150 2024-03-27T10:55:25.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:25.686+0000 7f5536000700 1 --2- 172.21.15.97:0/3354613193 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5538079150 0x7f553819f9b0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f552400d950 tx=0x7f552400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:55:25.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:25.686+0000 7f552f400700 1 -- 172.21.15.97:0/3354613193 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5524016070 con 0x7f5538079150 2024-03-27T10:55:25.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:25.686+0000 7f552f400700 1 -- 172.21.15.97:0/3354613193 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5524009df0 con 0x7f5538079150 2024-03-27T10:55:25.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:25.686+0000 7f553d800700 1 -- 172.21.15.97:0/3354613193 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55381a4720 con 0x7f5538079150 2024-03-27T10:55:25.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:25.686+0000 7f552f400700 1 -- 172.21.15.97:0/3354613193 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f552400f3c0 con 0x7f5538079150 2024-03-27T10:55:25.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:25.686+0000 7f553d800700 1 -- 172.21.15.97:0/3354613193 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55381abfa0 con 0x7f5538079150 2024-03-27T10:55:25.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:25.687+0000 7f553d800700 1 -- 172.21.15.97:0/3354613193 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5538069300 con 0x7f5538079150 2024-03-27T10:55:25.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:25.689+0000 7f552f400700 1 -- 172.21.15.97:0/3354613193 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138421+0+0 (secure 0 0 0) 0x7f5524019070 con 0x7f5538079150 2024-03-27T10:55:25.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:25.690+0000 7f552f400700 1 --2- 172.21.15.97:0/3354613193 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f551c0a3730 0x7f551c0a5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:25.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:25.690+0000 7f552f400700 1 -- 172.21.15.97:0/3354613193 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f55240cf4b0 con 0x7f5538079150 2024-03-27T10:55:25.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:25.691+0000 7f5536a00700 1 --2- 172.21.15.97:0/3354613193 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f551c0a3730 0x7f551c0a5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:25.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:25.691+0000 7f5536a00700 1 --2- 172.21.15.97:0/3354613193 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f551c0a3730 0x7f551c0a5bf0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5528005950 tx=0x7f552800a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:55:25.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:25.696+0000 7f552f400700 1 -- 172.21.15.97:0/3354613193 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f552409bed0 con 0x7f5538079150 2024-03-27T10:55:25.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:25.928+0000 7f553d800700 1 -- 172.21.15.97:0/3354613193 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f55381a0780 con 0x7f5538079150 2024-03-27T10:55:25.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:25.930+0000 7f552f400700 1 -- 172.21.15.97:0/3354613193 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f552409b7d0 con 0x7f5538079150 2024-03-27T10:55:25.901 INFO:teuthology.orchestra.run.smithi097.stdout:270582939701 2024-03-27T10:55:25.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:25.932+0000 7f553d800700 1 -- 172.21.15.97:0/3354613193 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f551c0a3730 msgr2=0x7f551c0a5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:25.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:25.932+0000 7f553d800700 1 --2- 172.21.15.97:0/3354613193 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f551c0a3730 0x7f551c0a5bf0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5528005950 tx=0x7f552800a300 comp rx=0 tx=0).stop 2024-03-27T10:55:25.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:25.932+0000 7f553d800700 1 -- 172.21.15.97:0/3354613193 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5538079150 msgr2=0x7f553819f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:25.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:25.932+0000 7f553d800700 1 --2- 172.21.15.97:0/3354613193 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5538079150 0x7f553819f9b0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f552400d950 tx=0x7f552400dc60 comp rx=0 tx=0).stop 2024-03-27T10:55:25.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:25.933+0000 7f553d800700 1 -- 172.21.15.97:0/3354613193 shutdown_connections 2024-03-27T10:55:25.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:25.933+0000 7f553d800700 1 --2- 172.21.15.97:0/3354613193 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f551c0a3730 0x7f551c0a5bf0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:25.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:25.933+0000 7f553d800700 1 --2- 172.21.15.97:0/3354613193 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f55381134b0 0x7f55381a3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:25.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:25.933+0000 7f553d800700 1 --2- 172.21.15.97:0/3354613193 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f55380787a0 0x7f553819f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:25.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:25.933+0000 7f553d800700 1 --2- 172.21.15.97:0/3354613193 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5538079150 0x7f553819f9b0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:25.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:25.933+0000 7f553d800700 1 -- 172.21.15.97:0/3354613193 >> 172.21.15.97:0/3354613193 conn(0x7f55381002e0 msgr2=0x7f55381026d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:55:25.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:25.933+0000 7f553d800700 1 -- 172.21.15.97:0/3354613193 shutdown_connections 2024-03-27T10:55:25.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:25.933+0000 7f553d800700 1 -- 172.21.15.97:0/3354613193 wait complete. 2024-03-27T10:55:26.239 INFO:tasks.cephadm.ceph_manager.ceph:need seq 330712481824 got 330712481831 for osd.10 2024-03-27T10:55:26.239 DEBUG:teuthology.parallel:result is None 2024-03-27T10:55:26.426 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:55:26 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/3354613193' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-27T10:55:26.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:55:26 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/3354613193' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-27T10:55:26.641 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:55:26 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/3354613193' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-27T10:55:27.442 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:55:27 smithi097 ceph-mon[31859]: pgmap v405: 97 pgs: 97 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-27T10:55:27.445 INFO:tasks.cephadm.ceph_manager.ceph:need seq 270582939695 got 270582939701 for osd.8 2024-03-27T10:55:27.446 DEBUG:teuthology.parallel:result is None 2024-03-27T10:55:27.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:55:27 smithi114 ceph-mon[35338]: pgmap v405: 97 pgs: 97 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-27T10:55:27.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:55:27 smithi177 ceph-mon[35294]: pgmap v405: 97 pgs: 97 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-27T10:55:29.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:29.154+0000 7fc013e00700 1 -- 172.21.15.97:0/3412313881 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc00c108620 msgr2=0x7fc00c108a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:29.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:29.154+0000 7fc013e00700 1 --2- 172.21.15.97:0/3412313881 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc00c108620 0x7fc00c108a00 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc000009990 tx=0x7fc000009ca0 comp rx=0 tx=0).stop 2024-03-27T10:55:29.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:29.154+0000 7fc013e00700 1 -- 172.21.15.97:0/3412313881 shutdown_connections 2024-03-27T10:55:29.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:29.154+0000 7fc013e00700 1 --2- 172.21.15.97:0/3412313881 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc00c102fe0 0x7fc00c10f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:29.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:29.154+0000 7fc013e00700 1 --2- 172.21.15.97:0/3412313881 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc00c108620 0x7fc00c108a00 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:29.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:29.154+0000 7fc013e00700 1 --2- 172.21.15.97:0/3412313881 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc00c102620 0x7fc00c102aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:29.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:29.154+0000 7fc013e00700 1 -- 172.21.15.97:0/3412313881 >> 172.21.15.97:0/3412313881 conn(0x7fc00c0fe0f0 msgr2=0x7fc00c100510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:55:29.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:29.154+0000 7fc013e00700 1 -- 172.21.15.97:0/3412313881 shutdown_connections 2024-03-27T10:55:29.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:29.154+0000 7fc013e00700 1 -- 172.21.15.97:0/3412313881 wait complete. 2024-03-27T10:55:29.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:29.156+0000 7fc013e00700 1 Processor -- start 2024-03-27T10:55:29.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:29.156+0000 7fc013e00700 1 -- start start 2024-03-27T10:55:29.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:29.156+0000 7fc013e00700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc00c102620 0x7fc00c19af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:29.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:29.156+0000 7fc013e00700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc00c102fe0 0x7fc00c19b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:29.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:29.156+0000 7fc013e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc00c108620 0x7fc00c19f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:29.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:29.156+0000 7fc013e00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc00c112850 con 0x7fc00c108620 2024-03-27T10:55:29.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:29.156+0000 7fc013e00700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc00c1126d0 con 0x7fc00c102fe0 2024-03-27T10:55:29.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:29.156+0000 7fc013e00700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc00c1129d0 con 0x7fc00c102620 2024-03-27T10:55:29.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:29.157+0000 7fc011e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc00c108620 0x7fc00c19f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:29.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:29.157+0000 7fc011e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc00c108620 0x7fc00c19f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50376/0 (socket says 172.21.15.97:50376) 2024-03-27T10:55:29.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:29.157+0000 7fc011e00700 1 -- 172.21.15.97:0/1943719151 learned_addr learned my addr 172.21.15.97:0/1943719151 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:55:29.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:29.157+0000 7fc010a00700 1 --2- 172.21.15.97:0/1943719151 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc00c102fe0 0x7fc00c19b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:29.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:29.157+0000 7fc011400700 1 --2- 172.21.15.97:0/1943719151 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc00c102620 0x7fc00c19af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:29.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:29.157+0000 7fc011e00700 1 -- 172.21.15.97:0/1943719151 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc00c102620 msgr2=0x7fc00c19af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:29.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:29.157+0000 7fc011e00700 1 --2- 172.21.15.97:0/1943719151 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc00c102620 0x7fc00c19af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:29.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:29.157+0000 7fc011e00700 1 -- 172.21.15.97:0/1943719151 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc00c102fe0 msgr2=0x7fc00c19b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:29.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:29.157+0000 7fc011e00700 1 --2- 172.21.15.97:0/1943719151 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc00c102fe0 0x7fc00c19b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:29.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:29.157+0000 7fc011e00700 1 -- 172.21.15.97:0/1943719151 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc00c19ff20 con 0x7fc00c108620 2024-03-27T10:55:29.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:29.157+0000 7fc011e00700 1 --2- 172.21.15.97:0/1943719151 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc00c108620 0x7fc00c19f7e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc004007d70 tx=0x7fc00400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:55:29.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:29.158+0000 7fc00a000700 1 -- 172.21.15.97:0/1943719151 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc004016020 con 0x7fc00c108620 2024-03-27T10:55:29.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:29.158+0000 7fc00a000700 1 -- 172.21.15.97:0/1943719151 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc00400ad40 con 0x7fc00c108620 2024-03-27T10:55:29.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:29.158+0000 7fc013e00700 1 -- 172.21.15.97:0/1943719151 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc00c1a0210 con 0x7fc00c108620 2024-03-27T10:55:29.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:29.158+0000 7fc00a000700 1 -- 172.21.15.97:0/1943719151 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0040115e0 con 0x7fc00c108620 2024-03-27T10:55:29.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:29.158+0000 7fc013e00700 1 -- 172.21.15.97:0/1943719151 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc00c1a7a90 con 0x7fc00c108620 2024-03-27T10:55:29.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:29.159+0000 7fc013e00700 1 -- 172.21.15.97:0/1943719151 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc00c069300 con 0x7fc00c108620 2024-03-27T10:55:29.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:29.163+0000 7fc00a000700 1 -- 172.21.15.97:0/1943719151 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138421+0+0 (secure 0 0 0) 0x7fc004019020 con 0x7fc00c108620 2024-03-27T10:55:29.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:29.164+0000 7fc00a000700 1 --2- 172.21.15.97:0/1943719151 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fbff40a3730 0x7fbff40a5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:29.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:29.164+0000 7fc00a000700 1 -- 172.21.15.97:0/1943719151 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fc0040cfcc0 con 0x7fc00c108620 2024-03-27T10:55:29.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:29.164+0000 7fc011400700 1 --2- 172.21.15.97:0/1943719151 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fbff40a3730 0x7fbff40a5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:29.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:29.165+0000 7fc011400700 1 --2- 172.21.15.97:0/1943719151 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fbff40a3730 0x7fbff40a5bf0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc00000b5c0 tx=0x7fc00001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:55:29.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:29.169+0000 7fc00a000700 1 -- 172.21.15.97:0/1943719151 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7fc004020020 con 0x7fc00c108620 2024-03-27T10:55:29.390 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:55:29 smithi097 ceph-mon[31859]: pgmap v406: 97 pgs: 97 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-27T10:55:29.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:29.472+0000 7fc013e00700 1 -- 172.21.15.97:0/1943719151 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fc00c1a7c60 con 0x7fc00c108620 2024-03-27T10:55:29.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:29.472+0000 7fc00a000700 1 -- 172.21.15.97:0/1943719151 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc00409c850 con 0x7fc00c108620 2024-03-27T10:55:29.444 INFO:teuthology.orchestra.run.smithi097.stdout:77309411421 2024-03-27T10:55:29.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:29.476+0000 7fc013e00700 1 -- 172.21.15.97:0/1943719151 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fbff40a3730 msgr2=0x7fbff40a5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:29.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:29.476+0000 7fc013e00700 1 --2- 172.21.15.97:0/1943719151 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fbff40a3730 0x7fbff40a5bf0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc00000b5c0 tx=0x7fc00001a040 comp rx=0 tx=0).stop 2024-03-27T10:55:29.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:29.476+0000 7fc013e00700 1 -- 172.21.15.97:0/1943719151 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc00c108620 msgr2=0x7fc00c19f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:29.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:29.476+0000 7fc013e00700 1 --2- 172.21.15.97:0/1943719151 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc00c108620 0x7fc00c19f7e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc004007d70 tx=0x7fc00400a340 comp rx=0 tx=0).stop 2024-03-27T10:55:29.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:29.476+0000 7fc013e00700 1 -- 172.21.15.97:0/1943719151 shutdown_connections 2024-03-27T10:55:29.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:29.476+0000 7fc013e00700 1 --2- 172.21.15.97:0/1943719151 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fbff40a3730 0x7fbff40a5bf0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:29.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:29.476+0000 7fc013e00700 1 --2- 172.21.15.97:0/1943719151 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc00c108620 0x7fc00c19f7e0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:29.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:29.476+0000 7fc013e00700 1 --2- 172.21.15.97:0/1943719151 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc00c102620 0x7fc00c19af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:29.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:29.476+0000 7fc013e00700 1 --2- 172.21.15.97:0/1943719151 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc00c102fe0 0x7fc00c19b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:29.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:29.477+0000 7fc013e00700 1 -- 172.21.15.97:0/1943719151 >> 172.21.15.97:0/1943719151 conn(0x7fc00c0fe0f0 msgr2=0x7fc00c0fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:55:29.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:29.477+0000 7fc013e00700 1 -- 172.21.15.97:0/1943719151 shutdown_connections 2024-03-27T10:55:29.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:29.477+0000 7fc013e00700 1 -- 172.21.15.97:0/1943719151 wait complete. 2024-03-27T10:55:29.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:55:29 smithi114 ceph-mon[35338]: pgmap v406: 97 pgs: 97 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-27T10:55:29.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:55:29 smithi177 ceph-mon[35294]: pgmap v406: 97 pgs: 97 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-27T10:55:30.189 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411413 got 77309411421 for osd.2 2024-03-27T10:55:30.189 DEBUG:teuthology.parallel:result is None 2024-03-27T10:55:30.190 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-03-27T10:55:30.190 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell --fsid adca88c4-ec26-11ee-95d0-87774f69a715 -- ceph pg dump --format=json 2024-03-27T10:55:30.390 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:55:30 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/1943719151' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-27T10:55:30.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:55:30 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/1943719151' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-27T10:55:30.643 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T10:55:30.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:55:30 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/1943719151' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-27T10:55:31.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:55:31 smithi097 ceph-mon[31859]: pgmap v407: 97 pgs: 97 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-27T10:55:31.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:55:31 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:55:31.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:55:31 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:55:31.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:55:31 smithi114 ceph-mon[35338]: pgmap v407: 97 pgs: 97 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-27T10:55:31.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:55:31 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:55:31.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:55:31 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:55:31.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:55:31 smithi177 ceph-mon[35294]: pgmap v407: 97 pgs: 97 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-27T10:55:31.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:55:31 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:55:31.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:55:31 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:55:32.499 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:55:32 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:55:32.499 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:55:32 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:55:32.499 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:55:32 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:55:32.516 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:55:32 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:55:32.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:55:32 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:55:32.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:55:32 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:55:32.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:55:32 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:55:32.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:55:32 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:55:32.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:55:32 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:55:32.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:32.793+0000 7f531e200700 1 -- 172.21.15.97:0/922268426 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5318102690 msgr2=0x7f5318102b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:32.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:32.793+0000 7f531e200700 1 --2- 172.21.15.97:0/922268426 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5318102690 0x7f5318102b10 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f53040099c0 tx=0x7f5304009cd0 comp rx=0 tx=0).stop 2024-03-27T10:55:32.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:32.793+0000 7f531e200700 1 -- 172.21.15.97:0/922268426 shutdown_connections 2024-03-27T10:55:32.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:32.793+0000 7f531e200700 1 --2- 172.21.15.97:0/922268426 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5318103050 0x7f531810f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:32.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:32.793+0000 7f531e200700 1 --2- 172.21.15.97:0/922268426 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5318108670 0x7f5318108a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:32.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:32.793+0000 7f531e200700 1 --2- 172.21.15.97:0/922268426 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5318102690 0x7f5318102b10 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:32.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:32.793+0000 7f531e200700 1 -- 172.21.15.97:0/922268426 >> 172.21.15.97:0/922268426 conn(0x7f5318079770 msgr2=0x7f5318079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:55:32.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:32.793+0000 7f531e200700 1 -- 172.21.15.97:0/922268426 shutdown_connections 2024-03-27T10:55:32.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:32.793+0000 7f531e200700 1 -- 172.21.15.97:0/922268426 wait complete. 2024-03-27T10:55:32.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:32.795+0000 7f531e200700 1 Processor -- start 2024-03-27T10:55:32.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:32.795+0000 7f531e200700 1 -- start start 2024-03-27T10:55:32.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:32.795+0000 7f531e200700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5318102690 0x7f5318074940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:32.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:32.795+0000 7f531e200700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5318103050 0x7f5318074e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:32.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:32.795+0000 7f531e200700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5318108670 0x7f53180753c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:32.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:32.795+0000 7f531e200700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f53181127f0 con 0x7f5318102690 2024-03-27T10:55:32.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:32.796+0000 7f531e200700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f5318112670 con 0x7f5318108670 2024-03-27T10:55:32.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:32.796+0000 7f531e200700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f5318112970 con 0x7f5318103050 2024-03-27T10:55:32.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:32.796+0000 7f5317400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5318102690 0x7f5318074940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:32.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:32.796+0000 7f5317400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5318102690 0x7f5318074940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50394/0 (socket says 172.21.15.97:50394) 2024-03-27T10:55:32.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:32.796+0000 7f5317400700 1 -- 172.21.15.97:0/1287548843 learned_addr learned my addr 172.21.15.97:0/1287548843 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:55:32.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:32.796+0000 7f5317e00700 1 --2- 172.21.15.97:0/1287548843 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5318108670 0x7f53180753c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:32.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:32.796+0000 7f5316a00700 1 --2- 172.21.15.97:0/1287548843 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5318103050 0x7f5318074e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:32.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:32.796+0000 7f5317400700 1 -- 172.21.15.97:0/1287548843 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5318103050 msgr2=0x7f5318074e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:32.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:32.796+0000 7f5317400700 1 --2- 172.21.15.97:0/1287548843 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5318103050 0x7f5318074e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:32.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:32.796+0000 7f5317400700 1 -- 172.21.15.97:0/1287548843 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5318108670 msgr2=0x7f53180753c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:32.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:32.796+0000 7f5317400700 1 --2- 172.21.15.97:0/1287548843 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5318108670 0x7f53180753c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:32.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:32.796+0000 7f5317400700 1 -- 172.21.15.97:0/1287548843 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53181a72d0 con 0x7f5318102690 2024-03-27T10:55:32.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:32.797+0000 7f5317400700 1 --2- 172.21.15.97:0/1287548843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5318102690 0x7f5318074940 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f530800c9b0 tx=0x7f530800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:55:32.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:32.797+0000 7f530fe00700 1 -- 172.21.15.97:0/1287548843 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5308007b00 con 0x7f5318102690 2024-03-27T10:55:32.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:32.797+0000 7f530fe00700 1 -- 172.21.15.97:0/1287548843 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f530800ce50 con 0x7f5318102690 2024-03-27T10:55:32.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:32.797+0000 7f530fe00700 1 -- 172.21.15.97:0/1287548843 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5308018660 con 0x7f5318102690 2024-03-27T10:55:32.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:32.797+0000 7f531e200700 1 -- 172.21.15.97:0/1287548843 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53181a75c0 con 0x7f5318102690 2024-03-27T10:55:32.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:32.797+0000 7f531e200700 1 -- 172.21.15.97:0/1287548843 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53181a7a80 con 0x7f5318102690 2024-03-27T10:55:32.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:32.798+0000 7f531e200700 1 -- 172.21.15.97:0/1287548843 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5318069300 con 0x7f5318102690 2024-03-27T10:55:32.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:32.800+0000 7f530fe00700 1 -- 172.21.15.97:0/1287548843 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138421+0+0 (secure 0 0 0) 0x7f530800f450 con 0x7f5318102690 2024-03-27T10:55:32.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:32.802+0000 7f530fe00700 1 --2- 172.21.15.97:0/1287548843 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f52fc0a37b0 0x7f52fc0a5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:32.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:32.802+0000 7f530fe00700 1 -- 172.21.15.97:0/1287548843 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f53080cfd40 con 0x7f5318102690 2024-03-27T10:55:32.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:32.802+0000 7f5316a00700 1 --2- 172.21.15.97:0/1287548843 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f52fc0a37b0 0x7f52fc0a5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:32.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:32.803+0000 7f5316a00700 1 --2- 172.21.15.97:0/1287548843 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f52fc0a37b0 0x7f52fc0a5c70 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f53180763d0 tx=0x7f5304005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:55:32.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:32.806+0000 7f530fe00700 1 -- 172.21.15.97:0/1287548843 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f5308014030 con 0x7f5318102690 2024-03-27T10:55:32.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:33.024+0000 7f531e200700 1 -- 172.21.15.97:0/1287548843 --> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5318062a80 con 0x7f52fc0a37b0 2024-03-27T10:55:33.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:33.037+0000 7f530fe00700 1 -- 172.21.15.97:0/1287548843 <== mgr.14152 v2:172.21.15.97:6800/3955463538 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258768 (secure 0 0 0) 0x7f5318062a80 con 0x7f52fc0a37b0 2024-03-27T10:55:33.015 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T10:55:33.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:33.044+0000 7f531e200700 1 -- 172.21.15.97:0/1287548843 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f52fc0a37b0 msgr2=0x7f52fc0a5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:33.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:33.044+0000 7f531e200700 1 --2- 172.21.15.97:0/1287548843 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f52fc0a37b0 0x7f52fc0a5c70 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f53180763d0 tx=0x7f5304005fb0 comp rx=0 tx=0).stop 2024-03-27T10:55:33.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:33.044+0000 7f531e200700 1 -- 172.21.15.97:0/1287548843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5318102690 msgr2=0x7f5318074940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:33.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:33.044+0000 7f531e200700 1 --2- 172.21.15.97:0/1287548843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5318102690 0x7f5318074940 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f530800c9b0 tx=0x7f530800ccc0 comp rx=0 tx=0).stop 2024-03-27T10:55:33.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:33.044+0000 7f531e200700 1 -- 172.21.15.97:0/1287548843 shutdown_connections 2024-03-27T10:55:33.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:33.044+0000 7f531e200700 1 --2- 172.21.15.97:0/1287548843 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f52fc0a37b0 0x7f52fc0a5c70 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:33.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:33.044+0000 7f531e200700 1 --2- 172.21.15.97:0/1287548843 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5318108670 0x7f53180753c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:33.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:33.044+0000 7f531e200700 1 --2- 172.21.15.97:0/1287548843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5318102690 0x7f5318074940 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:33.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:33.044+0000 7f531e200700 1 --2- 172.21.15.97:0/1287548843 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5318103050 0x7f5318074e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:33.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:33.044+0000 7f531e200700 1 -- 172.21.15.97:0/1287548843 >> 172.21.15.97:0/1287548843 conn(0x7f5318079770 msgr2=0x7f5318101490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:55:33.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:33.045+0000 7f531e200700 1 -- 172.21.15.97:0/1287548843 shutdown_connections 2024-03-27T10:55:33.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:33.045+0000 7f531e200700 1 -- 172.21.15.97:0/1287548843 wait complete. 2024-03-27T10:55:33.018 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-03-27T10:55:33.375 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:55:33 smithi097 ceph-mon[31859]: pgmap v408: 97 pgs: 97 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T10:55:33.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:55:33 smithi114 ceph-mon[35338]: pgmap v408: 97 pgs: 97 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T10:55:33.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:55:33 smithi177 ceph-mon[35294]: pgmap v408: 97 pgs: 97 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T10:55:33.873 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":408,"stamp":"2024-03-27T10:55:32.091211+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658852,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":946,"num_read_kb":592,"num_write":422,"num_write_kb":3073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1181318,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":346,"ondisk_log_size":346,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":375632,"kb_used_data":13932,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124566192,"statfs":{"total":1151940427776,"available":1151555780608,"internally_reserved":0,"allocated":14266368,"data_stored":7727568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":100,"num_read_kb":50,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005439"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.332110+0000","last_change":"2024-03-27T10:54:05.077845+0000","last_active":"2024-03-27T10:54:41.332110+0000","last_peered":"2024-03-27T10:54:41.332110+0000","last_clean":"2024-03-27T10:54:41.332110+0000","last_became_active":"2024-03-27T10:54:05.077494+0000","last_became_peered":"2024-03-27T10:54:05.077494+0000","last_unstale":"2024-03-27T10:54:41.332110+0000","last_undegraded":"2024-03-27T10:54:41.332110+0000","last_fullsized":"2024-03-27T10:54:41.332110+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:16:07.094746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.437521+0000","last_change":"2024-03-27T10:54:05.078830+0000","last_active":"2024-03-27T10:54:41.437521+0000","last_peered":"2024-03-27T10:54:41.437521+0000","last_clean":"2024-03-27T10:54:41.437521+0000","last_became_active":"2024-03-27T10:54:05.078474+0000","last_became_peered":"2024-03-27T10:54:05.078474+0000","last_unstale":"2024-03-27T10:54:41.437521+0000","last_undegraded":"2024-03-27T10:54:41.437521+0000","last_fullsized":"2024-03-27T10:54:41.437521+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T20:11:09.900802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.437181+0000","last_change":"2024-03-27T10:54:05.082554+0000","last_active":"2024-03-27T10:54:41.437181+0000","last_peered":"2024-03-27T10:54:41.437181+0000","last_clean":"2024-03-27T10:54:41.437181+0000","last_became_active":"2024-03-27T10:54:05.081395+0000","last_became_peered":"2024-03-27T10:54:05.081395+0000","last_unstale":"2024-03-27T10:54:41.437181+0000","last_undegraded":"2024-03-27T10:54:41.437181+0000","last_fullsized":"2024-03-27T10:54:41.437181+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T20:02:58.866792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.327250+0000","last_change":"2024-03-27T10:54:05.086700+0000","last_active":"2024-03-27T10:54:41.327250+0000","last_peered":"2024-03-27T10:54:41.327250+0000","last_clean":"2024-03-27T10:54:41.327250+0000","last_became_active":"2024-03-27T10:54:05.086347+0000","last_became_peered":"2024-03-27T10:54:05.086347+0000","last_unstale":"2024-03-27T10:54:41.327250+0000","last_undegraded":"2024-03-27T10:54:41.327250+0000","last_fullsized":"2024-03-27T10:54:41.327250+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T22:03:11.336365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.327431+0000","last_change":"2024-03-27T10:54:05.083479+0000","last_active":"2024-03-27T10:54:41.327431+0000","last_peered":"2024-03-27T10:54:41.327431+0000","last_clean":"2024-03-27T10:54:41.327431+0000","last_became_active":"2024-03-27T10:54:05.083162+0000","last_became_peered":"2024-03-27T10:54:05.083162+0000","last_unstale":"2024-03-27T10:54:41.327431+0000","last_undegraded":"2024-03-27T10:54:41.327431+0000","last_fullsized":"2024-03-27T10:54:41.327431+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:23:42.265668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.331297+0000","last_change":"2024-03-27T10:54:05.079096+0000","last_active":"2024-03-27T10:54:41.331297+0000","last_peered":"2024-03-27T10:54:41.331297+0000","last_clean":"2024-03-27T10:54:41.331297+0000","last_became_active":"2024-03-27T10:54:05.078778+0000","last_became_peered":"2024-03-27T10:54:05.078778+0000","last_unstale":"2024-03-27T10:54:41.331297+0000","last_undegraded":"2024-03-27T10:54:41.331297+0000","last_fullsized":"2024-03-27T10:54:41.331297+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T19:26:47.534594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.333561+0000","last_change":"2024-03-27T10:54:05.098926+0000","last_active":"2024-03-27T10:54:41.333561+0000","last_peered":"2024-03-27T10:54:41.333561+0000","last_clean":"2024-03-27T10:54:41.333561+0000","last_became_active":"2024-03-27T10:54:05.098139+0000","last_became_peered":"2024-03-27T10:54:05.098139+0000","last_unstale":"2024-03-27T10:54:41.333561+0000","last_undegraded":"2024-03-27T10:54:41.333561+0000","last_fullsized":"2024-03-27T10:54:41.333561+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:35:14.335947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.335843+0000","last_change":"2024-03-27T10:54:05.093786+0000","last_active":"2024-03-27T10:54:41.335843+0000","last_peered":"2024-03-27T10:54:41.335843+0000","last_clean":"2024-03-27T10:54:41.335843+0000","last_became_active":"2024-03-27T10:54:05.092047+0000","last_became_peered":"2024-03-27T10:54:05.092047+0000","last_unstale":"2024-03-27T10:54:41.335843+0000","last_undegraded":"2024-03-27T10:54:41.335843+0000","last_fullsized":"2024-03-27T10:54:41.335843+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T22:50:30.778358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.330842+0000","last_change":"2024-03-27T10:54:05.081678+0000","last_active":"2024-03-27T10:54:41.330842+0000","last_peered":"2024-03-27T10:54:41.330842+0000","last_clean":"2024-03-27T10:54:41.330842+0000","last_became_active":"2024-03-27T10:54:05.080264+0000","last_became_peered":"2024-03-27T10:54:05.080264+0000","last_unstale":"2024-03-27T10:54:41.330842+0000","last_undegraded":"2024-03-27T10:54:41.330842+0000","last_fullsized":"2024-03-27T10:54:41.330842+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T22:41:04.183282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.334975+0000","last_change":"2024-03-27T10:54:05.095172+0000","last_active":"2024-03-27T10:54:41.334975+0000","last_peered":"2024-03-27T10:54:41.334975+0000","last_clean":"2024-03-27T10:54:41.334975+0000","last_became_active":"2024-03-27T10:54:05.088438+0000","last_became_peered":"2024-03-27T10:54:05.088438+0000","last_unstale":"2024-03-27T10:54:41.334975+0000","last_undegraded":"2024-03-27T10:54:41.334975+0000","last_fullsized":"2024-03-27T10:54:41.334975+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T11:19:50.422788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.333441+0000","last_change":"2024-03-27T10:54:05.093599+0000","last_active":"2024-03-27T10:54:41.333441+0000","last_peered":"2024-03-27T10:54:41.333441+0000","last_clean":"2024-03-27T10:54:41.333441+0000","last_became_active":"2024-03-27T10:54:05.092248+0000","last_became_peered":"2024-03-27T10:54:05.092248+0000","last_unstale":"2024-03-27T10:54:41.333441+0000","last_undegraded":"2024-03-27T10:54:41.333441+0000","last_fullsized":"2024-03-27T10:54:41.333441+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:22:38.712442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.327198+0000","last_change":"2024-03-27T10:54:05.094677+0000","last_active":"2024-03-27T10:54:41.327198+0000","last_peered":"2024-03-27T10:54:41.327198+0000","last_clean":"2024-03-27T10:54:41.327198+0000","last_became_active":"2024-03-27T10:54:05.094326+0000","last_became_peered":"2024-03-27T10:54:05.094326+0000","last_unstale":"2024-03-27T10:54:41.327198+0000","last_undegraded":"2024-03-27T10:54:41.327198+0000","last_fullsized":"2024-03-27T10:54:41.327198+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T21:44:51.923881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.369637+0000","last_change":"2024-03-27T10:54:05.083178+0000","last_active":"2024-03-27T10:54:41.369637+0000","last_peered":"2024-03-27T10:54:41.369637+0000","last_clean":"2024-03-27T10:54:41.369637+0000","last_became_active":"2024-03-27T10:54:05.082805+0000","last_became_peered":"2024-03-27T10:54:05.082805+0000","last_unstale":"2024-03-27T10:54:41.369637+0000","last_undegraded":"2024-03-27T10:54:41.369637+0000","last_fullsized":"2024-03-27T10:54:41.369637+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T19:03:07.558006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.436813+0000","last_change":"2024-03-27T10:54:05.098092+0000","last_active":"2024-03-27T10:54:41.436813+0000","last_peered":"2024-03-27T10:54:41.436813+0000","last_clean":"2024-03-27T10:54:41.436813+0000","last_became_active":"2024-03-27T10:54:05.097759+0000","last_became_peered":"2024-03-27T10:54:05.097759+0000","last_unstale":"2024-03-27T10:54:41.436813+0000","last_undegraded":"2024-03-27T10:54:41.436813+0000","last_fullsized":"2024-03-27T10:54:41.436813+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:14:07.884759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.436833+0000","last_change":"2024-03-27T10:54:05.086089+0000","last_active":"2024-03-27T10:54:41.436833+0000","last_peered":"2024-03-27T10:54:41.436833+0000","last_clean":"2024-03-27T10:54:41.436833+0000","last_became_active":"2024-03-27T10:54:05.084830+0000","last_became_peered":"2024-03-27T10:54:05.084830+0000","last_unstale":"2024-03-27T10:54:41.436833+0000","last_undegraded":"2024-03-27T10:54:41.436833+0000","last_fullsized":"2024-03-27T10:54:41.436833+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T20:37:44.390102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.330070+0000","last_change":"2024-03-27T10:54:05.092190+0000","last_active":"2024-03-27T10:54:41.330070+0000","last_peered":"2024-03-27T10:54:41.330070+0000","last_clean":"2024-03-27T10:54:41.330070+0000","last_became_active":"2024-03-27T10:54:05.091827+0000","last_became_peered":"2024-03-27T10:54:05.091827+0000","last_unstale":"2024-03-27T10:54:41.330070+0000","last_undegraded":"2024-03-27T10:54:41.330070+0000","last_fullsized":"2024-03-27T10:54:41.330070+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T21:12:13.452829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.332416+0000","last_change":"2024-03-27T10:54:05.086924+0000","last_active":"2024-03-27T10:54:41.332416+0000","last_peered":"2024-03-27T10:54:41.332416+0000","last_clean":"2024-03-27T10:54:41.332416+0000","last_became_active":"2024-03-27T10:54:05.085701+0000","last_became_peered":"2024-03-27T10:54:05.085701+0000","last_unstale":"2024-03-27T10:54:41.332416+0000","last_undegraded":"2024-03-27T10:54:41.332416+0000","last_fullsized":"2024-03-27T10:54:41.332416+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:43:38.612288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.326652+0000","last_change":"2024-03-27T10:54:05.082532+0000","last_active":"2024-03-27T10:54:41.326652+0000","last_peered":"2024-03-27T10:54:41.326652+0000","last_clean":"2024-03-27T10:54:41.326652+0000","last_became_active":"2024-03-27T10:54:05.082200+0000","last_became_peered":"2024-03-27T10:54:05.082200+0000","last_unstale":"2024-03-27T10:54:41.326652+0000","last_undegraded":"2024-03-27T10:54:41.326652+0000","last_fullsized":"2024-03-27T10:54:41.326652+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T19:07:55.232728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.336188+0000","last_change":"2024-03-27T10:54:05.088204+0000","last_active":"2024-03-27T10:54:41.336188+0000","last_peered":"2024-03-27T10:54:41.336188+0000","last_clean":"2024-03-27T10:54:41.336188+0000","last_became_active":"2024-03-27T10:54:05.087705+0000","last_became_peered":"2024-03-27T10:54:05.087705+0000","last_unstale":"2024-03-27T10:54:41.336188+0000","last_undegraded":"2024-03-27T10:54:41.336188+0000","last_fullsized":"2024-03-27T10:54:41.336188+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T19:24:52.450205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'13","reported_seq":108,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:25.378238+0000","last_change":"2024-03-27T10:54:04.072182+0000","last_active":"2024-03-27T10:55:25.378238+0000","last_peered":"2024-03-27T10:55:25.378238+0000","last_clean":"2024-03-27T10:55:25.378238+0000","last_became_active":"2024-03-27T10:54:04.071827+0000","last_became_peered":"2024-03-27T10:54:04.071827+0000","last_unstale":"2024-03-27T10:55:25.378238+0000","last_undegraded":"2024-03-27T10:55:25.378238+0000","last_fullsized":"2024-03-27T10:55:25.378238+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T18:40:19.776684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":76,"num_read_kb":76,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.336909+0000","last_change":"2024-03-27T10:54:05.095772+0000","last_active":"2024-03-27T10:54:41.336909+0000","last_peered":"2024-03-27T10:54:41.336909+0000","last_clean":"2024-03-27T10:54:41.336909+0000","last_became_active":"2024-03-27T10:54:05.095385+0000","last_became_peered":"2024-03-27T10:54:05.095385+0000","last_unstale":"2024-03-27T10:54:41.336909+0000","last_undegraded":"2024-03-27T10:54:41.336909+0000","last_fullsized":"2024-03-27T10:54:41.336909+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T12:07:21.744097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.337143+0000","last_change":"2024-03-27T10:54:04.069600+0000","last_active":"2024-03-27T10:54:41.337143+0000","last_peered":"2024-03-27T10:54:41.337143+0000","last_clean":"2024-03-27T10:54:41.337143+0000","last_became_active":"2024-03-27T10:54:04.067770+0000","last_became_peered":"2024-03-27T10:54:04.067770+0000","last_unstale":"2024-03-27T10:54:41.337143+0000","last_undegraded":"2024-03-27T10:54:41.337143+0000","last_fullsized":"2024-03-27T10:54:41.337143+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T12:26:36.930945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.437383+0000","last_change":"2024-03-27T10:54:05.081779+0000","last_active":"2024-03-27T10:54:41.437383+0000","last_peered":"2024-03-27T10:54:41.437383+0000","last_clean":"2024-03-27T10:54:41.437383+0000","last_became_active":"2024-03-27T10:54:05.081492+0000","last_became_peered":"2024-03-27T10:54:05.081492+0000","last_unstale":"2024-03-27T10:54:41.437383+0000","last_undegraded":"2024-03-27T10:54:41.437383+0000","last_fullsized":"2024-03-27T10:54:41.437383+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T19:07:12.838048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"90'12","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.329099+0000","last_change":"2024-03-27T10:54:04.070550+0000","last_active":"2024-03-27T10:54:41.329099+0000","last_peered":"2024-03-27T10:54:41.329099+0000","last_clean":"2024-03-27T10:54:41.329099+0000","last_became_active":"2024-03-27T10:54:04.070010+0000","last_became_peered":"2024-03-27T10:54:04.070010+0000","last_unstale":"2024-03-27T10:54:41.329099+0000","last_undegraded":"2024-03-27T10:54:41.329099+0000","last_fullsized":"2024-03-27T10:54:41.329099+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:46:34.168771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.331327+0000","last_change":"2024-03-27T10:54:05.086372+0000","last_active":"2024-03-27T10:54:41.331327+0000","last_peered":"2024-03-27T10:54:41.331327+0000","last_clean":"2024-03-27T10:54:41.331327+0000","last_became_active":"2024-03-27T10:54:05.086013+0000","last_became_peered":"2024-03-27T10:54:05.086013+0000","last_unstale":"2024-03-27T10:54:41.331327+0000","last_undegraded":"2024-03-27T10:54:41.331327+0000","last_fullsized":"2024-03-27T10:54:41.331327+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:17:39.059071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"90'5","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.347107+0000","last_change":"2024-03-27T10:54:41.346475+0000","last_active":"2024-03-27T10:54:41.347107+0000","last_peered":"2024-03-27T10:54:41.347107+0000","last_clean":"2024-03-27T10:54:41.347107+0000","last_became_active":"2024-03-27T10:54:41.337431+0000","last_became_peered":"2024-03-27T10:54:41.337431+0000","last_unstale":"2024-03-27T10:54:41.347107+0000","last_undegraded":"2024-03-27T10:54:41.347107+0000","last_fullsized":"2024-03-27T10:54:41.347107+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T20:57:10.552047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.334709+0000","last_change":"2024-03-27T10:54:05.087580+0000","last_active":"2024-03-27T10:54:41.334709+0000","last_peered":"2024-03-27T10:54:41.334709+0000","last_clean":"2024-03-27T10:54:41.334709+0000","last_became_active":"2024-03-27T10:54:05.087128+0000","last_became_peered":"2024-03-27T10:54:05.087128+0000","last_unstale":"2024-03-27T10:54:41.334709+0000","last_undegraded":"2024-03-27T10:54:41.334709+0000","last_fullsized":"2024-03-27T10:54:41.334709+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T22:03:46.861258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"90'6","reported_seq":108,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:28.351176+0000","last_change":"2024-03-27T10:54:04.066312+0000","last_active":"2024-03-27T10:55:28.351176+0000","last_peered":"2024-03-27T10:55:28.351176+0000","last_clean":"2024-03-27T10:55:28.351176+0000","last_became_active":"2024-03-27T10:54:04.065909+0000","last_became_peered":"2024-03-27T10:54:04.065909+0000","last_unstale":"2024-03-27T10:55:28.351176+0000","last_undegraded":"2024-03-27T10:55:28.351176+0000","last_fullsized":"2024-03-27T10:55:28.351176+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:08:45.257808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":83,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.370991+0000","last_change":"2024-03-27T10:54:41.370991+0000","last_active":"2024-03-27T10:54:41.370991+0000","last_peered":"2024-03-27T10:54:41.370991+0000","last_clean":"2024-03-27T10:54:41.370991+0000","last_became_active":"2024-03-27T10:54:41.370596+0000","last_became_peered":"2024-03-27T10:54:41.370596+0000","last_unstale":"2024-03-27T10:54:41.370991+0000","last_undegraded":"2024-03-27T10:54:41.370991+0000","last_fullsized":"2024-03-27T10:54:41.370991+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:29:44.187503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"90'7","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.331143+0000","last_change":"2024-03-27T10:54:04.072451+0000","last_active":"2024-03-27T10:54:41.331143+0000","last_peered":"2024-03-27T10:54:41.331143+0000","last_clean":"2024-03-27T10:54:41.331143+0000","last_became_active":"2024-03-27T10:54:04.072042+0000","last_became_peered":"2024-03-27T10:54:04.072042+0000","last_unstale":"2024-03-27T10:54:41.331143+0000","last_undegraded":"2024-03-27T10:54:41.331143+0000","last_fullsized":"2024-03-27T10:54:41.331143+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T12:20:43.231073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.334437+0000","last_change":"2024-03-27T10:54:05.086055+0000","last_active":"2024-03-27T10:54:41.334437+0000","last_peered":"2024-03-27T10:54:41.334437+0000","last_clean":"2024-03-27T10:54:41.334437+0000","last_became_active":"2024-03-27T10:54:05.084429+0000","last_became_peered":"2024-03-27T10:54:05.084429+0000","last_unstale":"2024-03-27T10:54:41.334437+0000","last_undegraded":"2024-03-27T10:54:41.334437+0000","last_fullsized":"2024-03-27T10:54:41.334437+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T12:41:12.868245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"90'3","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.369278+0000","last_change":"2024-03-27T10:54:04.059076+0000","last_active":"2024-03-27T10:54:41.369278+0000","last_peered":"2024-03-27T10:54:41.369278+0000","last_clean":"2024-03-27T10:54:41.369278+0000","last_became_active":"2024-03-27T10:54:04.057614+0000","last_became_peered":"2024-03-27T10:54:04.057614+0000","last_unstale":"2024-03-27T10:54:41.369278+0000","last_undegraded":"2024-03-27T10:54:41.369278+0000","last_fullsized":"2024-03-27T10:54:41.369278+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T22:26:16.299840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.335569+0000","last_change":"2024-03-27T10:54:05.092955+0000","last_active":"2024-03-27T10:54:41.335569+0000","last_peered":"2024-03-27T10:54:41.335569+0000","last_clean":"2024-03-27T10:54:41.335569+0000","last_became_active":"2024-03-27T10:54:05.091867+0000","last_became_peered":"2024-03-27T10:54:05.091867+0000","last_unstale":"2024-03-27T10:54:41.335569+0000","last_undegraded":"2024-03-27T10:54:41.335569+0000","last_fullsized":"2024-03-27T10:54:41.335569+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T21:41:04.636480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"90'3","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.331670+0000","last_change":"2024-03-27T10:54:04.058833+0000","last_active":"2024-03-27T10:54:41.331670+0000","last_peered":"2024-03-27T10:54:41.331670+0000","last_clean":"2024-03-27T10:54:41.331670+0000","last_became_active":"2024-03-27T10:54:04.057750+0000","last_became_peered":"2024-03-27T10:54:04.057750+0000","last_unstale":"2024-03-27T10:54:41.331670+0000","last_undegraded":"2024-03-27T10:54:41.331670+0000","last_fullsized":"2024-03-27T10:54:41.331670+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:03:18.771094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.326145+0000","last_change":"2024-03-27T10:54:05.078933+0000","last_active":"2024-03-27T10:54:41.326145+0000","last_peered":"2024-03-27T10:54:41.326145+0000","last_clean":"2024-03-27T10:54:41.326145+0000","last_became_active":"2024-03-27T10:54:05.078544+0000","last_became_peered":"2024-03-27T10:54:05.078544+0000","last_unstale":"2024-03-27T10:54:41.326145+0000","last_undegraded":"2024-03-27T10:54:41.326145+0000","last_fullsized":"2024-03-27T10:54:41.326145+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T19:40:25.529775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"90'4","reported_seq":179,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:28.350036+0000","last_change":"2024-03-27T10:54:04.063974+0000","last_active":"2024-03-27T10:55:28.350036+0000","last_peered":"2024-03-27T10:55:28.350036+0000","last_clean":"2024-03-27T10:55:28.350036+0000","last_became_active":"2024-03-27T10:54:04.058171+0000","last_became_peered":"2024-03-27T10:54:04.058171+0000","last_unstale":"2024-03-27T10:55:28.350036+0000","last_undegraded":"2024-03-27T10:55:28.350036+0000","last_fullsized":"2024-03-27T10:55:28.350036+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T19:14:15.597889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":156,"num_read_kb":156,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.332046+0000","last_change":"2024-03-27T10:54:05.085727+0000","last_active":"2024-03-27T10:54:41.332046+0000","last_peered":"2024-03-27T10:54:41.332046+0000","last_clean":"2024-03-27T10:54:41.332046+0000","last_became_active":"2024-03-27T10:54:05.085354+0000","last_became_peered":"2024-03-27T10:54:05.085354+0000","last_unstale":"2024-03-27T10:54:41.332046+0000","last_undegraded":"2024-03-27T10:54:41.332046+0000","last_fullsized":"2024-03-27T10:54:41.332046+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:33:24.330313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"90'4","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.333492+0000","last_change":"2024-03-27T10:54:04.061757+0000","last_active":"2024-03-27T10:54:41.333492+0000","last_peered":"2024-03-27T10:54:41.333492+0000","last_clean":"2024-03-27T10:54:41.333492+0000","last_became_active":"2024-03-27T10:54:04.059211+0000","last_became_peered":"2024-03-27T10:54:04.059211+0000","last_unstale":"2024-03-27T10:54:41.333492+0000","last_undegraded":"2024-03-27T10:54:41.333492+0000","last_fullsized":"2024-03-27T10:54:41.333492+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T11:20:41.391143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.437453+0000","last_change":"2024-03-27T10:54:05.090919+0000","last_active":"2024-03-27T10:54:41.437453+0000","last_peered":"2024-03-27T10:54:41.437453+0000","last_clean":"2024-03-27T10:54:41.437453+0000","last_became_active":"2024-03-27T10:54:05.087971+0000","last_became_peered":"2024-03-27T10:54:05.087971+0000","last_unstale":"2024-03-27T10:54:41.437453+0000","last_undegraded":"2024-03-27T10:54:41.437453+0000","last_fullsized":"2024-03-27T10:54:41.437453+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:00:45.195957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"90'5","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.334805+0000","last_change":"2024-03-27T10:54:04.064828+0000","last_active":"2024-03-27T10:54:41.334805+0000","last_peered":"2024-03-27T10:54:41.334805+0000","last_clean":"2024-03-27T10:54:41.334805+0000","last_became_active":"2024-03-27T10:54:04.064359+0000","last_became_peered":"2024-03-27T10:54:04.064359+0000","last_unstale":"2024-03-27T10:54:41.334805+0000","last_undegraded":"2024-03-27T10:54:41.334805+0000","last_fullsized":"2024-03-27T10:54:41.334805+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:05:19.905707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.337153+0000","last_change":"2024-03-27T10:54:41.337153+0000","last_active":"2024-03-27T10:54:41.337153+0000","last_peered":"2024-03-27T10:54:41.337153+0000","last_clean":"2024-03-27T10:54:41.337153+0000","last_became_active":"2024-03-27T10:54:41.336433+0000","last_became_peered":"2024-03-27T10:54:41.336433+0000","last_unstale":"2024-03-27T10:54:41.337153+0000","last_undegraded":"2024-03-27T10:54:41.337153+0000","last_fullsized":"2024-03-27T10:54:41.337153+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:00:25.944147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,7],"acting":[1,8,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"90'30","reported_seq":215,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:28.346828+0000","last_change":"2024-03-27T10:54:04.061913+0000","last_active":"2024-03-27T10:55:28.346828+0000","last_peered":"2024-03-27T10:55:28.346828+0000","last_clean":"2024-03-27T10:55:28.346828+0000","last_became_active":"2024-03-27T10:54:04.058366+0000","last_became_peered":"2024-03-27T10:54:04.058366+0000","last_unstale":"2024-03-27T10:55:28.346828+0000","last_undegraded":"2024-03-27T10:55:28.346828+0000","last_fullsized":"2024-03-27T10:55:28.346828+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:45:28.365554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61046,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":166,"num_read_kb":142,"num_write":32,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.328195+0000","last_change":"2024-03-27T10:54:05.090840+0000","last_active":"2024-03-27T10:54:41.328195+0000","last_peered":"2024-03-27T10:54:41.328195+0000","last_clean":"2024-03-27T10:54:41.328195+0000","last_became_active":"2024-03-27T10:54:05.090525+0000","last_became_peered":"2024-03-27T10:54:05.090525+0000","last_unstale":"2024-03-27T10:54:41.328195+0000","last_undegraded":"2024-03-27T10:54:41.328195+0000","last_fullsized":"2024-03-27T10:54:41.328195+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T21:26:10.559917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"90'9","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.334274+0000","last_change":"2024-03-27T10:54:04.061205+0000","last_active":"2024-03-27T10:54:41.334274+0000","last_peered":"2024-03-27T10:54:41.334274+0000","last_clean":"2024-03-27T10:54:41.334274+0000","last_became_active":"2024-03-27T10:54:04.059558+0000","last_became_peered":"2024-03-27T10:54:04.059558+0000","last_unstale":"2024-03-27T10:54:41.334274+0000","last_undegraded":"2024-03-27T10:54:41.334274+0000","last_fullsized":"2024-03-27T10:54:41.334274+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T12:00:00.182233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.369558+0000","last_change":"2024-03-27T10:54:05.090149+0000","last_active":"2024-03-27T10:54:41.369558+0000","last_peered":"2024-03-27T10:54:41.369558+0000","last_clean":"2024-03-27T10:54:41.369558+0000","last_became_active":"2024-03-27T10:54:05.089317+0000","last_became_peered":"2024-03-27T10:54:05.089317+0000","last_unstale":"2024-03-27T10:54:41.369558+0000","last_undegraded":"2024-03-27T10:54:41.369558+0000","last_fullsized":"2024-03-27T10:54:41.369558+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T11:50:06.477472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"90'5","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.333525+0000","last_change":"2024-03-27T10:54:04.061797+0000","last_active":"2024-03-27T10:54:41.333525+0000","last_peered":"2024-03-27T10:54:41.333525+0000","last_clean":"2024-03-27T10:54:41.333525+0000","last_became_active":"2024-03-27T10:54:04.060974+0000","last_became_peered":"2024-03-27T10:54:04.060974+0000","last_unstale":"2024-03-27T10:54:41.333525+0000","last_undegraded":"2024-03-27T10:54:41.333525+0000","last_fullsized":"2024-03-27T10:54:41.333525+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T18:34:33.486126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.370832+0000","last_change":"2024-03-27T10:54:41.370832+0000","last_active":"2024-03-27T10:54:41.370832+0000","last_peered":"2024-03-27T10:54:41.370832+0000","last_clean":"2024-03-27T10:54:41.370832+0000","last_became_active":"2024-03-27T10:54:41.370229+0000","last_became_peered":"2024-03-27T10:54:41.370229+0000","last_unstale":"2024-03-27T10:54:41.370832+0000","last_undegraded":"2024-03-27T10:54:41.370832+0000","last_fullsized":"2024-03-27T10:54:41.370832+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:53:19.887501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"90'6","reported_seq":106,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:26.357408+0000","last_change":"2024-03-27T10:54:04.060042+0000","last_active":"2024-03-27T10:55:26.357408+0000","last_peered":"2024-03-27T10:55:26.357408+0000","last_clean":"2024-03-27T10:55:26.357408+0000","last_became_active":"2024-03-27T10:54:04.059415+0000","last_became_peered":"2024-03-27T10:54:04.059415+0000","last_unstale":"2024-03-27T10:55:26.357408+0000","last_undegraded":"2024-03-27T10:55:26.357408+0000","last_fullsized":"2024-03-27T10:55:26.357408+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:52:06.846942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.331929+0000","last_change":"2024-03-27T10:54:05.100088+0000","last_active":"2024-03-27T10:54:41.331929+0000","last_peered":"2024-03-27T10:54:41.331929+0000","last_clean":"2024-03-27T10:54:41.331929+0000","last_became_active":"2024-03-27T10:54:05.098709+0000","last_became_peered":"2024-03-27T10:54:05.098709+0000","last_unstale":"2024-03-27T10:54:41.331929+0000","last_undegraded":"2024-03-27T10:54:41.331929+0000","last_fullsized":"2024-03-27T10:54:41.331929+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T21:25:13.329710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"90'17","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.327421+0000","last_change":"2024-03-27T10:54:04.066812+0000","last_active":"2024-03-27T10:54:41.327421+0000","last_peered":"2024-03-27T10:54:41.327421+0000","last_clean":"2024-03-27T10:54:41.327421+0000","last_became_active":"2024-03-27T10:54:04.066364+0000","last_became_peered":"2024-03-27T10:54:04.066364+0000","last_unstale":"2024-03-27T10:54:41.327421+0000","last_undegraded":"2024-03-27T10:54:41.327421+0000","last_fullsized":"2024-03-27T10:54:41.327421+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T19:31:51.616785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.333942+0000","last_change":"2024-03-27T10:54:05.089367+0000","last_active":"2024-03-27T10:54:41.333942+0000","last_peered":"2024-03-27T10:54:41.333942+0000","last_clean":"2024-03-27T10:54:41.333942+0000","last_became_active":"2024-03-27T10:54:05.088718+0000","last_became_peered":"2024-03-27T10:54:05.088718+0000","last_unstale":"2024-03-27T10:54:41.333942+0000","last_undegraded":"2024-03-27T10:54:41.333942+0000","last_fullsized":"2024-03-27T10:54:41.333942+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:56:08.768859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.327949+0000","last_change":"2024-03-27T10:54:04.059951+0000","last_active":"2024-03-27T10:54:41.327949+0000","last_peered":"2024-03-27T10:54:41.327949+0000","last_clean":"2024-03-27T10:54:41.327949+0000","last_became_active":"2024-03-27T10:54:04.057245+0000","last_became_peered":"2024-03-27T10:54:04.057245+0000","last_unstale":"2024-03-27T10:54:41.327949+0000","last_undegraded":"2024-03-27T10:54:41.327949+0000","last_fullsized":"2024-03-27T10:54:41.327949+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T19:36:48.881920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.331808+0000","last_change":"2024-03-27T10:54:05.089004+0000","last_active":"2024-03-27T10:54:41.331808+0000","last_peered":"2024-03-27T10:54:41.331808+0000","last_clean":"2024-03-27T10:54:41.331808+0000","last_became_active":"2024-03-27T10:54:05.083764+0000","last_became_peered":"2024-03-27T10:54:05.083764+0000","last_unstale":"2024-03-27T10:54:41.331808+0000","last_undegraded":"2024-03-27T10:54:41.331808+0000","last_fullsized":"2024-03-27T10:54:41.331808+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:09:53.883490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"90'1","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.326382+0000","last_change":"2024-03-27T10:54:04.050508+0000","last_active":"2024-03-27T10:54:41.326382+0000","last_peered":"2024-03-27T10:54:41.326382+0000","last_clean":"2024-03-27T10:54:41.326382+0000","last_became_active":"2024-03-27T10:54:04.048877+0000","last_became_peered":"2024-03-27T10:54:04.048877+0000","last_unstale":"2024-03-27T10:54:41.326382+0000","last_undegraded":"2024-03-27T10:54:41.326382+0000","last_fullsized":"2024-03-27T10:54:41.326382+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:44:11.578684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"90'1","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.334664+0000","last_change":"2024-03-27T10:54:05.083931+0000","last_active":"2024-03-27T10:54:41.334664+0000","last_peered":"2024-03-27T10:54:41.334664+0000","last_clean":"2024-03-27T10:54:41.334664+0000","last_became_active":"2024-03-27T10:54:05.083544+0000","last_became_peered":"2024-03-27T10:54:05.083544+0000","last_unstale":"2024-03-27T10:54:41.334664+0000","last_undegraded":"2024-03-27T10:54:41.334664+0000","last_fullsized":"2024-03-27T10:54:41.334664+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T20:54:19.172355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"90'5","reported_seq":103,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:28.386457+0000","last_change":"2024-03-27T10:54:04.057428+0000","last_active":"2024-03-27T10:55:28.386457+0000","last_peered":"2024-03-27T10:55:28.386457+0000","last_clean":"2024-03-27T10:55:28.386457+0000","last_became_active":"2024-03-27T10:54:04.052966+0000","last_became_peered":"2024-03-27T10:54:04.052966+0000","last_unstale":"2024-03-27T10:55:28.386457+0000","last_undegraded":"2024-03-27T10:55:28.386457+0000","last_fullsized":"2024-03-27T10:55:28.386457+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T18:50:31.170411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":79,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.438829+0000","last_change":"2024-03-27T10:54:41.438829+0000","last_active":"2024-03-27T10:54:41.438829+0000","last_peered":"2024-03-27T10:54:41.438829+0000","last_clean":"2024-03-27T10:54:41.438829+0000","last_became_active":"2024-03-27T10:54:41.438031+0000","last_became_peered":"2024-03-27T10:54:41.438031+0000","last_unstale":"2024-03-27T10:54:41.438829+0000","last_undegraded":"2024-03-27T10:54:41.438829+0000","last_fullsized":"2024-03-27T10:54:41.438829+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:33:04.052747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.332074+0000","last_change":"2024-03-27T10:54:05.090255+0000","last_active":"2024-03-27T10:54:41.332074+0000","last_peered":"2024-03-27T10:54:41.332074+0000","last_clean":"2024-03-27T10:54:41.332074+0000","last_became_active":"2024-03-27T10:54:05.088833+0000","last_became_peered":"2024-03-27T10:54:05.088833+0000","last_unstale":"2024-03-27T10:54:41.332074+0000","last_undegraded":"2024-03-27T10:54:41.332074+0000","last_fullsized":"2024-03-27T10:54:41.332074+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:15:55.934698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'5","reported_seq":104,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:27.358991+0000","last_change":"2024-03-27T10:54:04.059941+0000","last_active":"2024-03-27T10:55:27.358991+0000","last_peered":"2024-03-27T10:55:27.358991+0000","last_clean":"2024-03-27T10:55:27.358991+0000","last_became_active":"2024-03-27T10:54:04.058197+0000","last_became_peered":"2024-03-27T10:54:04.058197+0000","last_unstale":"2024-03-27T10:55:27.358991+0000","last_undegraded":"2024-03-27T10:55:27.358991+0000","last_fullsized":"2024-03-27T10:55:27.358991+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:44:46.374054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":80,"num_read_kb":80,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.338903+0000","last_change":"2024-03-27T10:54:41.337917+0000","last_active":"2024-03-27T10:54:41.338903+0000","last_peered":"2024-03-27T10:54:41.338903+0000","last_clean":"2024-03-27T10:54:41.338903+0000","last_became_active":"2024-03-27T10:54:41.337624+0000","last_became_peered":"2024-03-27T10:54:41.337624+0000","last_unstale":"2024-03-27T10:54:41.338903+0000","last_undegraded":"2024-03-27T10:54:41.338903+0000","last_fullsized":"2024-03-27T10:54:41.338903+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T11:50:48.275234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"90'6","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.327492+0000","last_change":"2024-03-27T10:54:04.064017+0000","last_active":"2024-03-27T10:54:41.327492+0000","last_peered":"2024-03-27T10:54:41.327492+0000","last_clean":"2024-03-27T10:54:41.327492+0000","last_became_active":"2024-03-27T10:54:04.062881+0000","last_became_peered":"2024-03-27T10:54:04.062881+0000","last_unstale":"2024-03-27T10:54:41.327492+0000","last_undegraded":"2024-03-27T10:54:41.327492+0000","last_fullsized":"2024-03-27T10:54:41.327492+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:21:21.638623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":349,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.333171+0000","last_change":"2024-03-27T10:52:31.127495+0000","last_active":"2024-03-27T10:54:41.333171+0000","last_peered":"2024-03-27T10:54:41.333171+0000","last_clean":"2024-03-27T10:54:41.333171+0000","last_became_active":"2024-03-27T10:52:31.125418+0000","last_became_peered":"2024-03-27T10:52:31.125418+0000","last_unstale":"2024-03-27T10:54:41.333171+0000","last_undegraded":"2024-03-27T10:54:41.333171+0000","last_fullsized":"2024-03-27T10:54:41.333171+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:47:54.649615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:47:54.649615+0000","last_clean_scrub_stamp":"2024-03-27T10:47:54.649615+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T18:49:24.591580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.369133+0000","last_change":"2024-03-27T10:54:05.085558+0000","last_active":"2024-03-27T10:54:41.369133+0000","last_peered":"2024-03-27T10:54:41.369133+0000","last_clean":"2024-03-27T10:54:41.369133+0000","last_became_active":"2024-03-27T10:54:05.083896+0000","last_became_peered":"2024-03-27T10:54:05.083896+0000","last_unstale":"2024-03-27T10:54:41.369133+0000","last_undegraded":"2024-03-27T10:54:41.369133+0000","last_fullsized":"2024-03-27T10:54:41.369133+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:15:29.697609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'4","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.333242+0000","last_change":"2024-03-27T10:54:04.072448+0000","last_active":"2024-03-27T10:54:41.333242+0000","last_peered":"2024-03-27T10:54:41.333242+0000","last_clean":"2024-03-27T10:54:41.333242+0000","last_became_active":"2024-03-27T10:54:04.071933+0000","last_became_peered":"2024-03-27T10:54:04.071933+0000","last_unstale":"2024-03-27T10:54:41.333242+0000","last_undegraded":"2024-03-27T10:54:41.333242+0000","last_fullsized":"2024-03-27T10:54:41.333242+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:34:49.204345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"90'4","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.328648+0000","last_change":"2024-03-27T10:54:04.067308+0000","last_active":"2024-03-27T10:54:41.328648+0000","last_peered":"2024-03-27T10:54:41.328648+0000","last_clean":"2024-03-27T10:54:41.328648+0000","last_became_active":"2024-03-27T10:54:04.066927+0000","last_became_peered":"2024-03-27T10:54:04.066927+0000","last_unstale":"2024-03-27T10:54:41.328648+0000","last_undegraded":"2024-03-27T10:54:41.328648+0000","last_fullsized":"2024-03-27T10:54:41.328648+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:08:04.849245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.369740+0000","last_change":"2024-03-27T10:54:05.075154+0000","last_active":"2024-03-27T10:54:41.369740+0000","last_peered":"2024-03-27T10:54:41.369740+0000","last_clean":"2024-03-27T10:54:41.369740+0000","last_became_active":"2024-03-27T10:54:05.074772+0000","last_became_peered":"2024-03-27T10:54:05.074772+0000","last_unstale":"2024-03-27T10:54:41.369740+0000","last_undegraded":"2024-03-27T10:54:41.369740+0000","last_fullsized":"2024-03-27T10:54:41.369740+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T20:49:16.286229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"90'13","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.326583+0000","last_change":"2024-03-27T10:54:04.066805+0000","last_active":"2024-03-27T10:54:41.326583+0000","last_peered":"2024-03-27T10:54:41.326583+0000","last_clean":"2024-03-27T10:54:41.326583+0000","last_became_active":"2024-03-27T10:54:04.066500+0000","last_became_peered":"2024-03-27T10:54:04.066500+0000","last_unstale":"2024-03-27T10:54:41.326583+0000","last_undegraded":"2024-03-27T10:54:41.326583+0000","last_fullsized":"2024-03-27T10:54:41.326583+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T12:37:44.853575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.337096+0000","last_change":"2024-03-27T10:54:05.094314+0000","last_active":"2024-03-27T10:54:41.337096+0000","last_peered":"2024-03-27T10:54:41.337096+0000","last_clean":"2024-03-27T10:54:41.337096+0000","last_became_active":"2024-03-27T10:54:05.093896+0000","last_became_peered":"2024-03-27T10:54:05.093896+0000","last_unstale":"2024-03-27T10:54:41.337096+0000","last_undegraded":"2024-03-27T10:54:41.337096+0000","last_fullsized":"2024-03-27T10:54:41.337096+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:30:02.441317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"90'7","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.328683+0000","last_change":"2024-03-27T10:54:04.065525+0000","last_active":"2024-03-27T10:54:41.328683+0000","last_peered":"2024-03-27T10:54:41.328683+0000","last_clean":"2024-03-27T10:54:41.328683+0000","last_became_active":"2024-03-27T10:54:04.064979+0000","last_became_peered":"2024-03-27T10:54:04.064979+0000","last_unstale":"2024-03-27T10:54:41.328683+0000","last_undegraded":"2024-03-27T10:54:41.328683+0000","last_fullsized":"2024-03-27T10:54:41.328683+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:58:25.110861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.437342+0000","last_change":"2024-03-27T10:54:05.084038+0000","last_active":"2024-03-27T10:54:41.437342+0000","last_peered":"2024-03-27T10:54:41.437342+0000","last_clean":"2024-03-27T10:54:41.437342+0000","last_became_active":"2024-03-27T10:54:05.083733+0000","last_became_peered":"2024-03-27T10:54:05.083733+0000","last_unstale":"2024-03-27T10:54:41.437342+0000","last_undegraded":"2024-03-27T10:54:41.437342+0000","last_fullsized":"2024-03-27T10:54:41.437342+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T18:51:14.832602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.330469+0000","last_change":"2024-03-27T10:54:04.064506+0000","last_active":"2024-03-27T10:54:41.330469+0000","last_peered":"2024-03-27T10:54:41.330469+0000","last_clean":"2024-03-27T10:54:41.330469+0000","last_became_active":"2024-03-27T10:54:04.064178+0000","last_became_peered":"2024-03-27T10:54:04.064178+0000","last_unstale":"2024-03-27T10:54:41.330469+0000","last_undegraded":"2024-03-27T10:54:41.330469+0000","last_fullsized":"2024-03-27T10:54:41.330469+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:07:19.465782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.327649+0000","last_change":"2024-03-27T10:54:05.088241+0000","last_active":"2024-03-27T10:54:41.327649+0000","last_peered":"2024-03-27T10:54:41.327649+0000","last_clean":"2024-03-27T10:54:41.327649+0000","last_became_active":"2024-03-27T10:54:05.087852+0000","last_became_peered":"2024-03-27T10:54:05.087852+0000","last_unstale":"2024-03-27T10:54:41.327649+0000","last_undegraded":"2024-03-27T10:54:41.327649+0000","last_fullsized":"2024-03-27T10:54:41.327649+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:47:09.633654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"90'7","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.336358+0000","last_change":"2024-03-27T10:54:04.065744+0000","last_active":"2024-03-27T10:54:41.336358+0000","last_peered":"2024-03-27T10:54:41.336358+0000","last_clean":"2024-03-27T10:54:41.336358+0000","last_became_active":"2024-03-27T10:54:04.065155+0000","last_became_peered":"2024-03-27T10:54:04.065155+0000","last_unstale":"2024-03-27T10:54:41.336358+0000","last_undegraded":"2024-03-27T10:54:41.336358+0000","last_fullsized":"2024-03-27T10:54:41.336358+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T12:35:00.556458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.331050+0000","last_change":"2024-03-27T10:54:05.077954+0000","last_active":"2024-03-27T10:54:41.331050+0000","last_peered":"2024-03-27T10:54:41.331050+0000","last_clean":"2024-03-27T10:54:41.331050+0000","last_became_active":"2024-03-27T10:54:05.077589+0000","last_became_peered":"2024-03-27T10:54:05.077589+0000","last_unstale":"2024-03-27T10:54:41.331050+0000","last_undegraded":"2024-03-27T10:54:41.331050+0000","last_fullsized":"2024-03-27T10:54:41.331050+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T20:18:27.411887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"90'3","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.353201+0000","last_change":"2024-03-27T10:54:41.352576+0000","last_active":"2024-03-27T10:54:41.353201+0000","last_peered":"2024-03-27T10:54:41.353201+0000","last_clean":"2024-03-27T10:54:41.353201+0000","last_became_active":"2024-03-27T10:54:41.338775+0000","last_became_peered":"2024-03-27T10:54:41.338775+0000","last_unstale":"2024-03-27T10:54:41.353201+0000","last_undegraded":"2024-03-27T10:54:41.353201+0000","last_fullsized":"2024-03-27T10:54:41.353201+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T18:23:14.194920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.336442+0000","last_change":"2024-03-27T10:54:05.088250+0000","last_active":"2024-03-27T10:54:41.336442+0000","last_peered":"2024-03-27T10:54:41.336442+0000","last_clean":"2024-03-27T10:54:41.336442+0000","last_became_active":"2024-03-27T10:54:05.087942+0000","last_became_peered":"2024-03-27T10:54:05.087942+0000","last_unstale":"2024-03-27T10:54:41.336442+0000","last_undegraded":"2024-03-27T10:54:41.336442+0000","last_fullsized":"2024-03-27T10:54:41.336442+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:57:45.266347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.327668+0000","last_change":"2024-03-27T10:54:04.070513+0000","last_active":"2024-03-27T10:54:41.327668+0000","last_peered":"2024-03-27T10:54:41.327668+0000","last_clean":"2024-03-27T10:54:41.327668+0000","last_became_active":"2024-03-27T10:54:04.070042+0000","last_became_peered":"2024-03-27T10:54:04.070042+0000","last_unstale":"2024-03-27T10:54:41.327668+0000","last_undegraded":"2024-03-27T10:54:41.327668+0000","last_fullsized":"2024-03-27T10:54:41.327668+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:40:25.216899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.334218+0000","last_change":"2024-03-27T10:54:05.095067+0000","last_active":"2024-03-27T10:54:41.334218+0000","last_peered":"2024-03-27T10:54:41.334218+0000","last_clean":"2024-03-27T10:54:41.334218+0000","last_became_active":"2024-03-27T10:54:05.094618+0000","last_became_peered":"2024-03-27T10:54:05.094618+0000","last_unstale":"2024-03-27T10:54:41.334218+0000","last_undegraded":"2024-03-27T10:54:41.334218+0000","last_fullsized":"2024-03-27T10:54:41.334218+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:57:59.557310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"90'5","reported_seq":103,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:27.385863+0000","last_change":"2024-03-27T10:54:04.051352+0000","last_active":"2024-03-27T10:55:27.385863+0000","last_peered":"2024-03-27T10:55:27.385863+0000","last_clean":"2024-03-27T10:55:27.385863+0000","last_became_active":"2024-03-27T10:54:04.050850+0000","last_became_peered":"2024-03-27T10:54:04.050850+0000","last_unstale":"2024-03-27T10:55:27.385863+0000","last_undegraded":"2024-03-27T10:55:27.385863+0000","last_fullsized":"2024-03-27T10:55:27.385863+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:50:16.719451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":79,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.333591+0000","last_change":"2024-03-27T10:54:05.094650+0000","last_active":"2024-03-27T10:54:41.333591+0000","last_peered":"2024-03-27T10:54:41.333591+0000","last_clean":"2024-03-27T10:54:41.333591+0000","last_became_active":"2024-03-27T10:54:05.094168+0000","last_became_peered":"2024-03-27T10:54:05.094168+0000","last_unstale":"2024-03-27T10:54:41.333591+0000","last_undegraded":"2024-03-27T10:54:41.333591+0000","last_fullsized":"2024-03-27T10:54:41.333591+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:29:56.491733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"90'3","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.334578+0000","last_change":"2024-03-27T10:54:04.067755+0000","last_active":"2024-03-27T10:54:41.334578+0000","last_peered":"2024-03-27T10:54:41.334578+0000","last_clean":"2024-03-27T10:54:41.334578+0000","last_became_active":"2024-03-27T10:54:04.067320+0000","last_became_peered":"2024-03-27T10:54:04.067320+0000","last_unstale":"2024-03-27T10:54:41.334578+0000","last_undegraded":"2024-03-27T10:54:41.334578+0000","last_fullsized":"2024-03-27T10:54:41.334578+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T12:16:25.949829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"90'1","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.333325+0000","last_change":"2024-03-27T10:54:05.100715+0000","last_active":"2024-03-27T10:54:41.333325+0000","last_peered":"2024-03-27T10:54:41.333325+0000","last_clean":"2024-03-27T10:54:41.333325+0000","last_became_active":"2024-03-27T10:54:05.097861+0000","last_became_peered":"2024-03-27T10:54:05.097861+0000","last_unstale":"2024-03-27T10:54:41.333325+0000","last_undegraded":"2024-03-27T10:54:41.333325+0000","last_fullsized":"2024-03-27T10:54:41.333325+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:02:39.107232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"90'5","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.369686+0000","last_change":"2024-03-27T10:54:04.063894+0000","last_active":"2024-03-27T10:54:41.369686+0000","last_peered":"2024-03-27T10:54:41.369686+0000","last_clean":"2024-03-27T10:54:41.369686+0000","last_became_active":"2024-03-27T10:54:04.063163+0000","last_became_peered":"2024-03-27T10:54:04.063163+0000","last_unstale":"2024-03-27T10:54:41.369686+0000","last_undegraded":"2024-03-27T10:54:41.369686+0000","last_fullsized":"2024-03-27T10:54:41.369686+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:30:41.025752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.333820+0000","last_change":"2024-03-27T10:54:05.101980+0000","last_active":"2024-03-27T10:54:41.333820+0000","last_peered":"2024-03-27T10:54:41.333820+0000","last_clean":"2024-03-27T10:54:41.333820+0000","last_became_active":"2024-03-27T10:54:05.101448+0000","last_became_peered":"2024-03-27T10:54:05.101448+0000","last_unstale":"2024-03-27T10:54:41.333820+0000","last_undegraded":"2024-03-27T10:54:41.333820+0000","last_fullsized":"2024-03-27T10:54:41.333820+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:45:11.137181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.331340+0000","last_change":"2024-03-27T10:54:05.084026+0000","last_active":"2024-03-27T10:54:41.331340+0000","last_peered":"2024-03-27T10:54:41.331340+0000","last_clean":"2024-03-27T10:54:41.331340+0000","last_became_active":"2024-03-27T10:54:05.083410+0000","last_became_peered":"2024-03-27T10:54:05.083410+0000","last_unstale":"2024-03-27T10:54:41.331340+0000","last_undegraded":"2024-03-27T10:54:41.331340+0000","last_fullsized":"2024-03-27T10:54:41.331340+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T22:39:34.586363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.334292+0000","last_change":"2024-03-27T10:54:05.083592+0000","last_active":"2024-03-27T10:54:41.334292+0000","last_peered":"2024-03-27T10:54:41.334292+0000","last_clean":"2024-03-27T10:54:41.334292+0000","last_became_active":"2024-03-27T10:54:05.082537+0000","last_became_peered":"2024-03-27T10:54:05.082537+0000","last_unstale":"2024-03-27T10:54:41.334292+0000","last_undegraded":"2024-03-27T10:54:41.334292+0000","last_fullsized":"2024-03-27T10:54:41.334292+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T21:15:36.830805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.334858+0000","last_change":"2024-03-27T10:54:05.090226+0000","last_active":"2024-03-27T10:54:41.334858+0000","last_peered":"2024-03-27T10:54:41.334858+0000","last_clean":"2024-03-27T10:54:41.334858+0000","last_became_active":"2024-03-27T10:54:05.089831+0000","last_became_peered":"2024-03-27T10:54:05.089831+0000","last_unstale":"2024-03-27T10:54:41.334858+0000","last_undegraded":"2024-03-27T10:54:41.334858+0000","last_fullsized":"2024-03-27T10:54:41.334858+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T12:45:57.802266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.437134+0000","last_change":"2024-03-27T10:54:05.080196+0000","last_active":"2024-03-27T10:54:41.437134+0000","last_peered":"2024-03-27T10:54:41.437134+0000","last_clean":"2024-03-27T10:54:41.437134+0000","last_became_active":"2024-03-27T10:54:05.079822+0000","last_became_peered":"2024-03-27T10:54:05.079822+0000","last_unstale":"2024-03-27T10:54:41.437134+0000","last_undegraded":"2024-03-27T10:54:41.437134+0000","last_fullsized":"2024-03-27T10:54:41.437134+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:53:11.556503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.369643+0000","last_change":"2024-03-27T10:54:05.087264+0000","last_active":"2024-03-27T10:54:41.369643+0000","last_peered":"2024-03-27T10:54:41.369643+0000","last_clean":"2024-03-27T10:54:41.369643+0000","last_became_active":"2024-03-27T10:54:05.085440+0000","last_became_peered":"2024-03-27T10:54:05.085440+0000","last_unstale":"2024-03-27T10:54:41.369643+0000","last_undegraded":"2024-03-27T10:54:41.369643+0000","last_fullsized":"2024-03-27T10:54:41.369643+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T22:41:40.665643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.331178+0000","last_change":"2024-03-27T10:54:05.094306+0000","last_active":"2024-03-27T10:54:41.331178+0000","last_peered":"2024-03-27T10:54:41.331178+0000","last_clean":"2024-03-27T10:54:41.331178+0000","last_became_active":"2024-03-27T10:54:05.093858+0000","last_became_peered":"2024-03-27T10:54:05.093858+0000","last_unstale":"2024-03-27T10:54:41.331178+0000","last_undegraded":"2024-03-27T10:54:41.331178+0000","last_fullsized":"2024-03-27T10:54:41.331178+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T21:22:57.624833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.334445+0000","last_change":"2024-03-27T10:54:05.094089+0000","last_active":"2024-03-27T10:54:41.334445+0000","last_peered":"2024-03-27T10:54:41.334445+0000","last_clean":"2024-03-27T10:54:41.334445+0000","last_became_active":"2024-03-27T10:54:05.092317+0000","last_became_peered":"2024-03-27T10:54:05.092317+0000","last_unstale":"2024-03-27T10:54:41.334445+0000","last_undegraded":"2024-03-27T10:54:41.334445+0000","last_fullsized":"2024-03-27T10:54:41.334445+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:34:57.606250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.336968+0000","last_change":"2024-03-27T10:54:05.099995+0000","last_active":"2024-03-27T10:54:41.336968+0000","last_peered":"2024-03-27T10:54:41.336968+0000","last_clean":"2024-03-27T10:54:41.336968+0000","last_became_active":"2024-03-27T10:54:05.099641+0000","last_became_peered":"2024-03-27T10:54:05.099641+0000","last_unstale":"2024-03-27T10:54:41.336968+0000","last_undegraded":"2024-03-27T10:54:41.336968+0000","last_fullsized":"2024-03-27T10:54:41.336968+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:01:16.050180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.371472+0000","last_change":"2024-03-27T10:54:41.371472+0000","last_active":"2024-03-27T10:54:41.371472+0000","last_peered":"2024-03-27T10:54:41.371472+0000","last_clean":"2024-03-27T10:54:41.371472+0000","last_became_active":"2024-03-27T10:54:41.370912+0000","last_became_peered":"2024-03-27T10:54:41.370912+0000","last_unstale":"2024-03-27T10:54:41.371472+0000","last_undegraded":"2024-03-27T10:54:41.371472+0000","last_fullsized":"2024-03-27T10:54:41.371472+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T20:21:28.950573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6],"acting":[5,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.335834+0000","last_change":"2024-03-27T10:54:05.096812+0000","last_active":"2024-03-27T10:54:41.335834+0000","last_peered":"2024-03-27T10:54:41.335834+0000","last_clean":"2024-03-27T10:54:41.335834+0000","last_became_active":"2024-03-27T10:54:05.096323+0000","last_became_peered":"2024-03-27T10:54:05.096323+0000","last_unstale":"2024-03-27T10:54:41.335834+0000","last_undegraded":"2024-03-27T10:54:41.335834+0000","last_fullsized":"2024-03-27T10:54:41.335834+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:39:29.827291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.332559+0000","last_change":"2024-03-27T10:54:05.089137+0000","last_active":"2024-03-27T10:54:41.332559+0000","last_peered":"2024-03-27T10:54:41.332559+0000","last_clean":"2024-03-27T10:54:41.332559+0000","last_became_active":"2024-03-27T10:54:05.088795+0000","last_became_peered":"2024-03-27T10:54:05.088795+0000","last_unstale":"2024-03-27T10:54:41.332559+0000","last_undegraded":"2024-03-27T10:54:41.332559+0000","last_fullsized":"2024-03-27T10:54:41.332559+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T19:47:35.231749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.369503+0000","last_change":"2024-03-27T10:54:05.089683+0000","last_active":"2024-03-27T10:54:41.369503+0000","last_peered":"2024-03-27T10:54:41.369503+0000","last_clean":"2024-03-27T10:54:41.369503+0000","last_became_active":"2024-03-27T10:54:05.089235+0000","last_became_peered":"2024-03-27T10:54:05.089235+0000","last_unstale":"2024-03-27T10:54:41.369503+0000","last_undegraded":"2024-03-27T10:54:41.369503+0000","last_fullsized":"2024-03-27T10:54:41.369503+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:43:37.012814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.332471+0000","last_change":"2024-03-27T10:54:05.085043+0000","last_active":"2024-03-27T10:54:41.332471+0000","last_peered":"2024-03-27T10:54:41.332471+0000","last_clean":"2024-03-27T10:54:41.332471+0000","last_became_active":"2024-03-27T10:54:05.084695+0000","last_became_peered":"2024-03-27T10:54:05.084695+0000","last_unstale":"2024-03-27T10:54:41.332471+0000","last_undegraded":"2024-03-27T10:54:41.332471+0000","last_fullsized":"2024-03-27T10:54:41.332471+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T12:07:31.327006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68250,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":820,"num_read_kb":483,"num_write":239,"num_write_kb":133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":204750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":207,"ondisk_log_size":207,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":85,"seq":365072220194,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1007616,"data_stored":481025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":77,"seq":330712481833,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37004,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708148,"statfs":{"total":95995035648,"available":95957143552,"internally_reserved":0,"allocated":1703936,"data_stored":1132909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":71,"seq":304942678063,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1028096,"data_stored":480373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":63,"seq":270582939702,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1019904,"data_stored":480169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":54,"seq":231928234045,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28600,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716552,"statfs":{"total":95995035648,"available":95965749248,"internally_reserved":0,"allocated":1622016,"data_stored":1073660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":49,"seq":210453397571,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1011712,"data_stored":480090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574950,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1073152,"data_stored":542049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738476,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1032192,"data_stored":481429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411422,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1093632,"data_stored":542660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117081,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1028096,"data_stored":479915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":35,"seq":150323855442,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36332,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708820,"statfs":{"total":95995035648,"available":95957831680,"internally_reserved":0,"allocated":1011712,"data_stored":480342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":41,"seq":176093659211,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32780,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712372,"statfs":{"total":95995035648,"available":95961468928,"internally_reserved":0,"allocated":1634304,"data_stored":1072947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":62798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":63409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":63173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-27T10:55:33.878 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell --fsid adca88c4-ec26-11ee-95d0-87774f69a715 -- ceph pg dump --format=json 2024-03-27T10:55:34.366 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T10:55:34.514 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:55:34 smithi097 ceph-mon[31859]: from='client.14979 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-27T10:55:34.516 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:55:34 smithi114 ceph-mon[35338]: from='client.14979 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-27T10:55:34.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:55:34 smithi177 ceph-mon[35294]: from='client.14979 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-27T10:55:35.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:55:35 smithi114 ceph-mon[35338]: pgmap v409: 97 pgs: 97 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-27T10:55:35.533 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:55:35 smithi097 ceph-mon[31859]: pgmap v409: 97 pgs: 97 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-27T10:55:35.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:55:35 smithi177 ceph-mon[35294]: pgmap v409: 97 pgs: 97 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-27T10:55:36.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:36.517+0000 7f65aae00700 1 -- 172.21.15.97:0/3293524781 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f65a4104830 msgr2=0x7f65a4104cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:36.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:36.517+0000 7f65aae00700 1 --2- 172.21.15.97:0/3293524781 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f65a4104830 0x7f65a4104cb0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f6590009990 tx=0x7f6590009ca0 comp rx=0 tx=0).stop 2024-03-27T10:55:36.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:36.517+0000 7f65aae00700 1 -- 172.21.15.97:0/3293524781 shutdown_connections 2024-03-27T10:55:36.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:36.517+0000 7f65aae00700 1 --2- 172.21.15.97:0/3293524781 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f65a41051f0 0x7f65a4111700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:36.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:36.517+0000 7f65aae00700 1 --2- 172.21.15.97:0/3293524781 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f65a410a830 0x7f65a410ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:36.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:36.517+0000 7f65aae00700 1 --2- 172.21.15.97:0/3293524781 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f65a4104830 0x7f65a4104cb0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:36.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:36.517+0000 7f65aae00700 1 -- 172.21.15.97:0/3293524781 >> 172.21.15.97:0/3293524781 conn(0x7f65a4100320 msgr2=0x7f65a4102740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:55:36.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:36.517+0000 7f65aae00700 1 -- 172.21.15.97:0/3293524781 shutdown_connections 2024-03-27T10:55:36.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:36.517+0000 7f65aae00700 1 -- 172.21.15.97:0/3293524781 wait complete. 2024-03-27T10:55:36.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:36.518+0000 7f65aae00700 1 Processor -- start 2024-03-27T10:55:36.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:36.518+0000 7f65aae00700 1 -- start start 2024-03-27T10:55:36.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:36.519+0000 7f65aae00700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f65a4104830 0x7f65a419af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:36.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:36.519+0000 7f65aae00700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f65a41051f0 0x7f65a419b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:36.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:36.519+0000 7f65aae00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f65a410a830 0x7f65a419f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:36.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:36.519+0000 7f65aae00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f65a4114a80 con 0x7f65a410a830 2024-03-27T10:55:36.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:36.519+0000 7f65aae00700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f65a4114900 con 0x7f65a41051f0 2024-03-27T10:55:36.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:36.519+0000 7f65aae00700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f65a4114c00 con 0x7f65a4104830 2024-03-27T10:55:36.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:36.519+0000 7f65a8e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f65a410a830 0x7f65a419f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:36.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:36.519+0000 7f65a8e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f65a410a830 0x7f65a419f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50400/0 (socket says 172.21.15.97:50400) 2024-03-27T10:55:36.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:36.519+0000 7f65a8e00700 1 -- 172.21.15.97:0/558321434 learned_addr learned my addr 172.21.15.97:0/558321434 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:55:36.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:36.519+0000 7f65a3e00700 1 --2- 172.21.15.97:0/558321434 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f65a4104830 0x7f65a419af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:36.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:36.519+0000 7f65a3400700 1 --2- 172.21.15.97:0/558321434 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f65a41051f0 0x7f65a419b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:36.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:36.520+0000 7f65a8e00700 1 -- 172.21.15.97:0/558321434 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f65a4104830 msgr2=0x7f65a419af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:36.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:36.520+0000 7f65a8e00700 1 --2- 172.21.15.97:0/558321434 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f65a4104830 0x7f65a419af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:36.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:36.520+0000 7f65a8e00700 1 -- 172.21.15.97:0/558321434 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f65a41051f0 msgr2=0x7f65a419b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:36.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:36.520+0000 7f65a8e00700 1 --2- 172.21.15.97:0/558321434 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f65a41051f0 0x7f65a419b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:36.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:36.520+0000 7f65a8e00700 1 -- 172.21.15.97:0/558321434 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65a419ff10 con 0x7f65a410a830 2024-03-27T10:55:36.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:36.520+0000 7f65a8e00700 1 --2- 172.21.15.97:0/558321434 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f65a410a830 0x7f65a419f7d0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f659c00ef30 tx=0x7f659c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:55:36.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:36.520+0000 7f65a0c00700 1 -- 172.21.15.97:0/558321434 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f659c0092e0 con 0x7f65a410a830 2024-03-27T10:55:36.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:36.520+0000 7f65a0c00700 1 -- 172.21.15.97:0/558321434 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f659c004bb0 con 0x7f65a410a830 2024-03-27T10:55:36.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:36.520+0000 7f65aae00700 1 -- 172.21.15.97:0/558321434 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65a41a01a0 con 0x7f65a410a830 2024-03-27T10:55:36.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:36.520+0000 7f65a0c00700 1 -- 172.21.15.97:0/558321434 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f659c004020 con 0x7f65a410a830 2024-03-27T10:55:36.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:36.521+0000 7f65aae00700 1 -- 172.21.15.97:0/558321434 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65a41a7a80 con 0x7f65a410a830 2024-03-27T10:55:36.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:36.522+0000 7f65aae00700 1 -- 172.21.15.97:0/558321434 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65a4069300 con 0x7f65a410a830 2024-03-27T10:55:36.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:36.524+0000 7f65a0c00700 1 -- 172.21.15.97:0/558321434 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138421+0+0 (secure 0 0 0) 0x7f659c01b030 con 0x7f65a410a830 2024-03-27T10:55:36.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:36.525+0000 7f65a0c00700 1 --2- 172.21.15.97:0/558321434 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f65880a37b0 0x7f65880a5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:36.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:36.525+0000 7f65a0c00700 1 -- 172.21.15.97:0/558321434 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f659c0cf980 con 0x7f65a410a830 2024-03-27T10:55:36.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:36.525+0000 7f65a3e00700 1 --2- 172.21.15.97:0/558321434 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f65880a37b0 0x7f65880a5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:36.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:36.526+0000 7f65a3e00700 1 --2- 172.21.15.97:0/558321434 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f65880a37b0 0x7f65880a5c70 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6594009be0 tx=0x7f6594009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:55:36.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:36.530+0000 7f65a0c00700 1 -- 172.21.15.97:0/558321434 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f659c022020 con 0x7f65a410a830 2024-03-27T10:55:36.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:36.751+0000 7f65aae00700 1 -- 172.21.15.97:0/558321434 --> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f65a4062a80 con 0x7f65880a37b0 2024-03-27T10:55:36.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:36.774+0000 7f65a0c00700 1 -- 172.21.15.97:0/558321434 <== mgr.14152 v2:172.21.15.97:6800/3955463538 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258768 (secure 0 0 0) 0x7f65a4062a80 con 0x7f65880a37b0 2024-03-27T10:55:36.750 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T10:55:36.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:36.780+0000 7f65aae00700 1 -- 172.21.15.97:0/558321434 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f65880a37b0 msgr2=0x7f65880a5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:36.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:36.780+0000 7f65aae00700 1 --2- 172.21.15.97:0/558321434 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f65880a37b0 0x7f65880a5c70 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6594009be0 tx=0x7f6594009260 comp rx=0 tx=0).stop 2024-03-27T10:55:36.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:36.780+0000 7f65aae00700 1 -- 172.21.15.97:0/558321434 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f65a410a830 msgr2=0x7f65a419f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:36.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:36.780+0000 7f65aae00700 1 --2- 172.21.15.97:0/558321434 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f65a410a830 0x7f65a419f7d0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f659c00ef30 tx=0x7f659c00c3a0 comp rx=0 tx=0).stop 2024-03-27T10:55:36.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:36.780+0000 7f65aae00700 1 -- 172.21.15.97:0/558321434 shutdown_connections 2024-03-27T10:55:36.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:36.780+0000 7f65aae00700 1 --2- 172.21.15.97:0/558321434 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f65880a37b0 0x7f65880a5c70 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:36.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:36.780+0000 7f65aae00700 1 --2- 172.21.15.97:0/558321434 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f65a410a830 0x7f65a419f7d0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:36.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:36.780+0000 7f65aae00700 1 --2- 172.21.15.97:0/558321434 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f65a4104830 0x7f65a419af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:36.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:36.780+0000 7f65aae00700 1 --2- 172.21.15.97:0/558321434 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f65a41051f0 0x7f65a419b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:36.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:36.780+0000 7f65aae00700 1 -- 172.21.15.97:0/558321434 >> 172.21.15.97:0/558321434 conn(0x7f65a4100320 msgr2=0x7f65a41009d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:55:36.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:36.780+0000 7f65aae00700 1 -- 172.21.15.97:0/558321434 shutdown_connections 2024-03-27T10:55:36.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:36.780+0000 7f65aae00700 1 -- 172.21.15.97:0/558321434 wait complete. 2024-03-27T10:55:36.753 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-03-27T10:55:37.390 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:55:37 smithi097 ceph-mon[31859]: pgmap v410: 97 pgs: 97 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-27T10:55:37.490 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":410,"stamp":"2024-03-27T10:55:36.092866+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658852,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":996,"num_read_kb":617,"num_write":422,"num_write_kb":3073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1181318,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":346,"ondisk_log_size":346,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":375632,"kb_used_data":13932,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124566192,"statfs":{"total":1151940427776,"available":1151555780608,"internally_reserved":0,"allocated":14266368,"data_stored":7727568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":115,"num_read_kb":55,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005644"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.332110+0000","last_change":"2024-03-27T10:54:05.077845+0000","last_active":"2024-03-27T10:54:41.332110+0000","last_peered":"2024-03-27T10:54:41.332110+0000","last_clean":"2024-03-27T10:54:41.332110+0000","last_became_active":"2024-03-27T10:54:05.077494+0000","last_became_peered":"2024-03-27T10:54:05.077494+0000","last_unstale":"2024-03-27T10:54:41.332110+0000","last_undegraded":"2024-03-27T10:54:41.332110+0000","last_fullsized":"2024-03-27T10:54:41.332110+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:16:07.094746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.437521+0000","last_change":"2024-03-27T10:54:05.078830+0000","last_active":"2024-03-27T10:54:41.437521+0000","last_peered":"2024-03-27T10:54:41.437521+0000","last_clean":"2024-03-27T10:54:41.437521+0000","last_became_active":"2024-03-27T10:54:05.078474+0000","last_became_peered":"2024-03-27T10:54:05.078474+0000","last_unstale":"2024-03-27T10:54:41.437521+0000","last_undegraded":"2024-03-27T10:54:41.437521+0000","last_fullsized":"2024-03-27T10:54:41.437521+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T20:11:09.900802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.437181+0000","last_change":"2024-03-27T10:54:05.082554+0000","last_active":"2024-03-27T10:54:41.437181+0000","last_peered":"2024-03-27T10:54:41.437181+0000","last_clean":"2024-03-27T10:54:41.437181+0000","last_became_active":"2024-03-27T10:54:05.081395+0000","last_became_peered":"2024-03-27T10:54:05.081395+0000","last_unstale":"2024-03-27T10:54:41.437181+0000","last_undegraded":"2024-03-27T10:54:41.437181+0000","last_fullsized":"2024-03-27T10:54:41.437181+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T20:02:58.866792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.327250+0000","last_change":"2024-03-27T10:54:05.086700+0000","last_active":"2024-03-27T10:54:41.327250+0000","last_peered":"2024-03-27T10:54:41.327250+0000","last_clean":"2024-03-27T10:54:41.327250+0000","last_became_active":"2024-03-27T10:54:05.086347+0000","last_became_peered":"2024-03-27T10:54:05.086347+0000","last_unstale":"2024-03-27T10:54:41.327250+0000","last_undegraded":"2024-03-27T10:54:41.327250+0000","last_fullsized":"2024-03-27T10:54:41.327250+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T22:03:11.336365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.327431+0000","last_change":"2024-03-27T10:54:05.083479+0000","last_active":"2024-03-27T10:54:41.327431+0000","last_peered":"2024-03-27T10:54:41.327431+0000","last_clean":"2024-03-27T10:54:41.327431+0000","last_became_active":"2024-03-27T10:54:05.083162+0000","last_became_peered":"2024-03-27T10:54:05.083162+0000","last_unstale":"2024-03-27T10:54:41.327431+0000","last_undegraded":"2024-03-27T10:54:41.327431+0000","last_fullsized":"2024-03-27T10:54:41.327431+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:23:42.265668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.331297+0000","last_change":"2024-03-27T10:54:05.079096+0000","last_active":"2024-03-27T10:54:41.331297+0000","last_peered":"2024-03-27T10:54:41.331297+0000","last_clean":"2024-03-27T10:54:41.331297+0000","last_became_active":"2024-03-27T10:54:05.078778+0000","last_became_peered":"2024-03-27T10:54:05.078778+0000","last_unstale":"2024-03-27T10:54:41.331297+0000","last_undegraded":"2024-03-27T10:54:41.331297+0000","last_fullsized":"2024-03-27T10:54:41.331297+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T19:26:47.534594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.333561+0000","last_change":"2024-03-27T10:54:05.098926+0000","last_active":"2024-03-27T10:54:41.333561+0000","last_peered":"2024-03-27T10:54:41.333561+0000","last_clean":"2024-03-27T10:54:41.333561+0000","last_became_active":"2024-03-27T10:54:05.098139+0000","last_became_peered":"2024-03-27T10:54:05.098139+0000","last_unstale":"2024-03-27T10:54:41.333561+0000","last_undegraded":"2024-03-27T10:54:41.333561+0000","last_fullsized":"2024-03-27T10:54:41.333561+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:35:14.335947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.335843+0000","last_change":"2024-03-27T10:54:05.093786+0000","last_active":"2024-03-27T10:54:41.335843+0000","last_peered":"2024-03-27T10:54:41.335843+0000","last_clean":"2024-03-27T10:54:41.335843+0000","last_became_active":"2024-03-27T10:54:05.092047+0000","last_became_peered":"2024-03-27T10:54:05.092047+0000","last_unstale":"2024-03-27T10:54:41.335843+0000","last_undegraded":"2024-03-27T10:54:41.335843+0000","last_fullsized":"2024-03-27T10:54:41.335843+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T22:50:30.778358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.330842+0000","last_change":"2024-03-27T10:54:05.081678+0000","last_active":"2024-03-27T10:54:41.330842+0000","last_peered":"2024-03-27T10:54:41.330842+0000","last_clean":"2024-03-27T10:54:41.330842+0000","last_became_active":"2024-03-27T10:54:05.080264+0000","last_became_peered":"2024-03-27T10:54:05.080264+0000","last_unstale":"2024-03-27T10:54:41.330842+0000","last_undegraded":"2024-03-27T10:54:41.330842+0000","last_fullsized":"2024-03-27T10:54:41.330842+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T22:41:04.183282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.334975+0000","last_change":"2024-03-27T10:54:05.095172+0000","last_active":"2024-03-27T10:54:41.334975+0000","last_peered":"2024-03-27T10:54:41.334975+0000","last_clean":"2024-03-27T10:54:41.334975+0000","last_became_active":"2024-03-27T10:54:05.088438+0000","last_became_peered":"2024-03-27T10:54:05.088438+0000","last_unstale":"2024-03-27T10:54:41.334975+0000","last_undegraded":"2024-03-27T10:54:41.334975+0000","last_fullsized":"2024-03-27T10:54:41.334975+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T11:19:50.422788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.333441+0000","last_change":"2024-03-27T10:54:05.093599+0000","last_active":"2024-03-27T10:54:41.333441+0000","last_peered":"2024-03-27T10:54:41.333441+0000","last_clean":"2024-03-27T10:54:41.333441+0000","last_became_active":"2024-03-27T10:54:05.092248+0000","last_became_peered":"2024-03-27T10:54:05.092248+0000","last_unstale":"2024-03-27T10:54:41.333441+0000","last_undegraded":"2024-03-27T10:54:41.333441+0000","last_fullsized":"2024-03-27T10:54:41.333441+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:22:38.712442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.327198+0000","last_change":"2024-03-27T10:54:05.094677+0000","last_active":"2024-03-27T10:54:41.327198+0000","last_peered":"2024-03-27T10:54:41.327198+0000","last_clean":"2024-03-27T10:54:41.327198+0000","last_became_active":"2024-03-27T10:54:05.094326+0000","last_became_peered":"2024-03-27T10:54:05.094326+0000","last_unstale":"2024-03-27T10:54:41.327198+0000","last_undegraded":"2024-03-27T10:54:41.327198+0000","last_fullsized":"2024-03-27T10:54:41.327198+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T21:44:51.923881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.369637+0000","last_change":"2024-03-27T10:54:05.083178+0000","last_active":"2024-03-27T10:54:41.369637+0000","last_peered":"2024-03-27T10:54:41.369637+0000","last_clean":"2024-03-27T10:54:41.369637+0000","last_became_active":"2024-03-27T10:54:05.082805+0000","last_became_peered":"2024-03-27T10:54:05.082805+0000","last_unstale":"2024-03-27T10:54:41.369637+0000","last_undegraded":"2024-03-27T10:54:41.369637+0000","last_fullsized":"2024-03-27T10:54:41.369637+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T19:03:07.558006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.436813+0000","last_change":"2024-03-27T10:54:05.098092+0000","last_active":"2024-03-27T10:54:41.436813+0000","last_peered":"2024-03-27T10:54:41.436813+0000","last_clean":"2024-03-27T10:54:41.436813+0000","last_became_active":"2024-03-27T10:54:05.097759+0000","last_became_peered":"2024-03-27T10:54:05.097759+0000","last_unstale":"2024-03-27T10:54:41.436813+0000","last_undegraded":"2024-03-27T10:54:41.436813+0000","last_fullsized":"2024-03-27T10:54:41.436813+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:14:07.884759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.436833+0000","last_change":"2024-03-27T10:54:05.086089+0000","last_active":"2024-03-27T10:54:41.436833+0000","last_peered":"2024-03-27T10:54:41.436833+0000","last_clean":"2024-03-27T10:54:41.436833+0000","last_became_active":"2024-03-27T10:54:05.084830+0000","last_became_peered":"2024-03-27T10:54:05.084830+0000","last_unstale":"2024-03-27T10:54:41.436833+0000","last_undegraded":"2024-03-27T10:54:41.436833+0000","last_fullsized":"2024-03-27T10:54:41.436833+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T20:37:44.390102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.330070+0000","last_change":"2024-03-27T10:54:05.092190+0000","last_active":"2024-03-27T10:54:41.330070+0000","last_peered":"2024-03-27T10:54:41.330070+0000","last_clean":"2024-03-27T10:54:41.330070+0000","last_became_active":"2024-03-27T10:54:05.091827+0000","last_became_peered":"2024-03-27T10:54:05.091827+0000","last_unstale":"2024-03-27T10:54:41.330070+0000","last_undegraded":"2024-03-27T10:54:41.330070+0000","last_fullsized":"2024-03-27T10:54:41.330070+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T21:12:13.452829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.332416+0000","last_change":"2024-03-27T10:54:05.086924+0000","last_active":"2024-03-27T10:54:41.332416+0000","last_peered":"2024-03-27T10:54:41.332416+0000","last_clean":"2024-03-27T10:54:41.332416+0000","last_became_active":"2024-03-27T10:54:05.085701+0000","last_became_peered":"2024-03-27T10:54:05.085701+0000","last_unstale":"2024-03-27T10:54:41.332416+0000","last_undegraded":"2024-03-27T10:54:41.332416+0000","last_fullsized":"2024-03-27T10:54:41.332416+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:43:38.612288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.326652+0000","last_change":"2024-03-27T10:54:05.082532+0000","last_active":"2024-03-27T10:54:41.326652+0000","last_peered":"2024-03-27T10:54:41.326652+0000","last_clean":"2024-03-27T10:54:41.326652+0000","last_became_active":"2024-03-27T10:54:05.082200+0000","last_became_peered":"2024-03-27T10:54:05.082200+0000","last_unstale":"2024-03-27T10:54:41.326652+0000","last_undegraded":"2024-03-27T10:54:41.326652+0000","last_fullsized":"2024-03-27T10:54:41.326652+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T19:07:55.232728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.336188+0000","last_change":"2024-03-27T10:54:05.088204+0000","last_active":"2024-03-27T10:54:41.336188+0000","last_peered":"2024-03-27T10:54:41.336188+0000","last_clean":"2024-03-27T10:54:41.336188+0000","last_became_active":"2024-03-27T10:54:05.087705+0000","last_became_peered":"2024-03-27T10:54:05.087705+0000","last_unstale":"2024-03-27T10:54:41.336188+0000","last_undegraded":"2024-03-27T10:54:41.336188+0000","last_fullsized":"2024-03-27T10:54:41.336188+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T19:24:52.450205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'13","reported_seq":113,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:30.389513+0000","last_change":"2024-03-27T10:54:04.072182+0000","last_active":"2024-03-27T10:55:30.389513+0000","last_peered":"2024-03-27T10:55:30.389513+0000","last_clean":"2024-03-27T10:55:30.389513+0000","last_became_active":"2024-03-27T10:54:04.071827+0000","last_became_peered":"2024-03-27T10:54:04.071827+0000","last_unstale":"2024-03-27T10:55:30.389513+0000","last_undegraded":"2024-03-27T10:55:30.389513+0000","last_fullsized":"2024-03-27T10:55:30.389513+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T18:40:19.776684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":81,"num_read_kb":81,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.336909+0000","last_change":"2024-03-27T10:54:05.095772+0000","last_active":"2024-03-27T10:54:41.336909+0000","last_peered":"2024-03-27T10:54:41.336909+0000","last_clean":"2024-03-27T10:54:41.336909+0000","last_became_active":"2024-03-27T10:54:05.095385+0000","last_became_peered":"2024-03-27T10:54:05.095385+0000","last_unstale":"2024-03-27T10:54:41.336909+0000","last_undegraded":"2024-03-27T10:54:41.336909+0000","last_fullsized":"2024-03-27T10:54:41.336909+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T12:07:21.744097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.337143+0000","last_change":"2024-03-27T10:54:04.069600+0000","last_active":"2024-03-27T10:54:41.337143+0000","last_peered":"2024-03-27T10:54:41.337143+0000","last_clean":"2024-03-27T10:54:41.337143+0000","last_became_active":"2024-03-27T10:54:04.067770+0000","last_became_peered":"2024-03-27T10:54:04.067770+0000","last_unstale":"2024-03-27T10:54:41.337143+0000","last_undegraded":"2024-03-27T10:54:41.337143+0000","last_fullsized":"2024-03-27T10:54:41.337143+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T12:26:36.930945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.437383+0000","last_change":"2024-03-27T10:54:05.081779+0000","last_active":"2024-03-27T10:54:41.437383+0000","last_peered":"2024-03-27T10:54:41.437383+0000","last_clean":"2024-03-27T10:54:41.437383+0000","last_became_active":"2024-03-27T10:54:05.081492+0000","last_became_peered":"2024-03-27T10:54:05.081492+0000","last_unstale":"2024-03-27T10:54:41.437383+0000","last_undegraded":"2024-03-27T10:54:41.437383+0000","last_fullsized":"2024-03-27T10:54:41.437383+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T19:07:12.838048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"90'12","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.329099+0000","last_change":"2024-03-27T10:54:04.070550+0000","last_active":"2024-03-27T10:54:41.329099+0000","last_peered":"2024-03-27T10:54:41.329099+0000","last_clean":"2024-03-27T10:54:41.329099+0000","last_became_active":"2024-03-27T10:54:04.070010+0000","last_became_peered":"2024-03-27T10:54:04.070010+0000","last_unstale":"2024-03-27T10:54:41.329099+0000","last_undegraded":"2024-03-27T10:54:41.329099+0000","last_fullsized":"2024-03-27T10:54:41.329099+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:46:34.168771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.331327+0000","last_change":"2024-03-27T10:54:05.086372+0000","last_active":"2024-03-27T10:54:41.331327+0000","last_peered":"2024-03-27T10:54:41.331327+0000","last_clean":"2024-03-27T10:54:41.331327+0000","last_became_active":"2024-03-27T10:54:05.086013+0000","last_became_peered":"2024-03-27T10:54:05.086013+0000","last_unstale":"2024-03-27T10:54:41.331327+0000","last_undegraded":"2024-03-27T10:54:41.331327+0000","last_fullsized":"2024-03-27T10:54:41.331327+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:17:39.059071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"90'5","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.347107+0000","last_change":"2024-03-27T10:54:41.346475+0000","last_active":"2024-03-27T10:54:41.347107+0000","last_peered":"2024-03-27T10:54:41.347107+0000","last_clean":"2024-03-27T10:54:41.347107+0000","last_became_active":"2024-03-27T10:54:41.337431+0000","last_became_peered":"2024-03-27T10:54:41.337431+0000","last_unstale":"2024-03-27T10:54:41.347107+0000","last_undegraded":"2024-03-27T10:54:41.347107+0000","last_fullsized":"2024-03-27T10:54:41.347107+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T20:57:10.552047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.334709+0000","last_change":"2024-03-27T10:54:05.087580+0000","last_active":"2024-03-27T10:54:41.334709+0000","last_peered":"2024-03-27T10:54:41.334709+0000","last_clean":"2024-03-27T10:54:41.334709+0000","last_became_active":"2024-03-27T10:54:05.087128+0000","last_became_peered":"2024-03-27T10:54:05.087128+0000","last_unstale":"2024-03-27T10:54:41.334709+0000","last_undegraded":"2024-03-27T10:54:41.334709+0000","last_fullsized":"2024-03-27T10:54:41.334709+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T22:03:46.861258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"90'6","reported_seq":113,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:33.361772+0000","last_change":"2024-03-27T10:54:04.066312+0000","last_active":"2024-03-27T10:55:33.361772+0000","last_peered":"2024-03-27T10:55:33.361772+0000","last_clean":"2024-03-27T10:55:33.361772+0000","last_became_active":"2024-03-27T10:54:04.065909+0000","last_became_peered":"2024-03-27T10:54:04.065909+0000","last_unstale":"2024-03-27T10:55:33.361772+0000","last_undegraded":"2024-03-27T10:55:33.361772+0000","last_fullsized":"2024-03-27T10:55:33.361772+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:08:45.257808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":88,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.370991+0000","last_change":"2024-03-27T10:54:41.370991+0000","last_active":"2024-03-27T10:54:41.370991+0000","last_peered":"2024-03-27T10:54:41.370991+0000","last_clean":"2024-03-27T10:54:41.370991+0000","last_became_active":"2024-03-27T10:54:41.370596+0000","last_became_peered":"2024-03-27T10:54:41.370596+0000","last_unstale":"2024-03-27T10:54:41.370991+0000","last_undegraded":"2024-03-27T10:54:41.370991+0000","last_fullsized":"2024-03-27T10:54:41.370991+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:29:44.187503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"90'7","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.331143+0000","last_change":"2024-03-27T10:54:04.072451+0000","last_active":"2024-03-27T10:54:41.331143+0000","last_peered":"2024-03-27T10:54:41.331143+0000","last_clean":"2024-03-27T10:54:41.331143+0000","last_became_active":"2024-03-27T10:54:04.072042+0000","last_became_peered":"2024-03-27T10:54:04.072042+0000","last_unstale":"2024-03-27T10:54:41.331143+0000","last_undegraded":"2024-03-27T10:54:41.331143+0000","last_fullsized":"2024-03-27T10:54:41.331143+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T12:20:43.231073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.334437+0000","last_change":"2024-03-27T10:54:05.086055+0000","last_active":"2024-03-27T10:54:41.334437+0000","last_peered":"2024-03-27T10:54:41.334437+0000","last_clean":"2024-03-27T10:54:41.334437+0000","last_became_active":"2024-03-27T10:54:05.084429+0000","last_became_peered":"2024-03-27T10:54:05.084429+0000","last_unstale":"2024-03-27T10:54:41.334437+0000","last_undegraded":"2024-03-27T10:54:41.334437+0000","last_fullsized":"2024-03-27T10:54:41.334437+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T12:41:12.868245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"90'3","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.369278+0000","last_change":"2024-03-27T10:54:04.059076+0000","last_active":"2024-03-27T10:54:41.369278+0000","last_peered":"2024-03-27T10:54:41.369278+0000","last_clean":"2024-03-27T10:54:41.369278+0000","last_became_active":"2024-03-27T10:54:04.057614+0000","last_became_peered":"2024-03-27T10:54:04.057614+0000","last_unstale":"2024-03-27T10:54:41.369278+0000","last_undegraded":"2024-03-27T10:54:41.369278+0000","last_fullsized":"2024-03-27T10:54:41.369278+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T22:26:16.299840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.335569+0000","last_change":"2024-03-27T10:54:05.092955+0000","last_active":"2024-03-27T10:54:41.335569+0000","last_peered":"2024-03-27T10:54:41.335569+0000","last_clean":"2024-03-27T10:54:41.335569+0000","last_became_active":"2024-03-27T10:54:05.091867+0000","last_became_peered":"2024-03-27T10:54:05.091867+0000","last_unstale":"2024-03-27T10:54:41.335569+0000","last_undegraded":"2024-03-27T10:54:41.335569+0000","last_fullsized":"2024-03-27T10:54:41.335569+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T21:41:04.636480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"90'3","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.331670+0000","last_change":"2024-03-27T10:54:04.058833+0000","last_active":"2024-03-27T10:54:41.331670+0000","last_peered":"2024-03-27T10:54:41.331670+0000","last_clean":"2024-03-27T10:54:41.331670+0000","last_became_active":"2024-03-27T10:54:04.057750+0000","last_became_peered":"2024-03-27T10:54:04.057750+0000","last_unstale":"2024-03-27T10:54:41.331670+0000","last_undegraded":"2024-03-27T10:54:41.331670+0000","last_fullsized":"2024-03-27T10:54:41.331670+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:03:18.771094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.326145+0000","last_change":"2024-03-27T10:54:05.078933+0000","last_active":"2024-03-27T10:54:41.326145+0000","last_peered":"2024-03-27T10:54:41.326145+0000","last_clean":"2024-03-27T10:54:41.326145+0000","last_became_active":"2024-03-27T10:54:05.078544+0000","last_became_peered":"2024-03-27T10:54:05.078544+0000","last_unstale":"2024-03-27T10:54:41.326145+0000","last_undegraded":"2024-03-27T10:54:41.326145+0000","last_fullsized":"2024-03-27T10:54:41.326145+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T19:40:25.529775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"90'4","reported_seq":189,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:33.360721+0000","last_change":"2024-03-27T10:54:04.063974+0000","last_active":"2024-03-27T10:55:33.360721+0000","last_peered":"2024-03-27T10:55:33.360721+0000","last_clean":"2024-03-27T10:55:33.360721+0000","last_became_active":"2024-03-27T10:54:04.058171+0000","last_became_peered":"2024-03-27T10:54:04.058171+0000","last_unstale":"2024-03-27T10:55:33.360721+0000","last_undegraded":"2024-03-27T10:55:33.360721+0000","last_fullsized":"2024-03-27T10:55:33.360721+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T19:14:15.597889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":166,"num_read_kb":166,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.332046+0000","last_change":"2024-03-27T10:54:05.085727+0000","last_active":"2024-03-27T10:54:41.332046+0000","last_peered":"2024-03-27T10:54:41.332046+0000","last_clean":"2024-03-27T10:54:41.332046+0000","last_became_active":"2024-03-27T10:54:05.085354+0000","last_became_peered":"2024-03-27T10:54:05.085354+0000","last_unstale":"2024-03-27T10:54:41.332046+0000","last_undegraded":"2024-03-27T10:54:41.332046+0000","last_fullsized":"2024-03-27T10:54:41.332046+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:33:24.330313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"90'4","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.333492+0000","last_change":"2024-03-27T10:54:04.061757+0000","last_active":"2024-03-27T10:54:41.333492+0000","last_peered":"2024-03-27T10:54:41.333492+0000","last_clean":"2024-03-27T10:54:41.333492+0000","last_became_active":"2024-03-27T10:54:04.059211+0000","last_became_peered":"2024-03-27T10:54:04.059211+0000","last_unstale":"2024-03-27T10:54:41.333492+0000","last_undegraded":"2024-03-27T10:54:41.333492+0000","last_fullsized":"2024-03-27T10:54:41.333492+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T11:20:41.391143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.437453+0000","last_change":"2024-03-27T10:54:05.090919+0000","last_active":"2024-03-27T10:54:41.437453+0000","last_peered":"2024-03-27T10:54:41.437453+0000","last_clean":"2024-03-27T10:54:41.437453+0000","last_became_active":"2024-03-27T10:54:05.087971+0000","last_became_peered":"2024-03-27T10:54:05.087971+0000","last_unstale":"2024-03-27T10:54:41.437453+0000","last_undegraded":"2024-03-27T10:54:41.437453+0000","last_fullsized":"2024-03-27T10:54:41.437453+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:00:45.195957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"90'5","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.334805+0000","last_change":"2024-03-27T10:54:04.064828+0000","last_active":"2024-03-27T10:54:41.334805+0000","last_peered":"2024-03-27T10:54:41.334805+0000","last_clean":"2024-03-27T10:54:41.334805+0000","last_became_active":"2024-03-27T10:54:04.064359+0000","last_became_peered":"2024-03-27T10:54:04.064359+0000","last_unstale":"2024-03-27T10:54:41.334805+0000","last_undegraded":"2024-03-27T10:54:41.334805+0000","last_fullsized":"2024-03-27T10:54:41.334805+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:05:19.905707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.337153+0000","last_change":"2024-03-27T10:54:41.337153+0000","last_active":"2024-03-27T10:54:41.337153+0000","last_peered":"2024-03-27T10:54:41.337153+0000","last_clean":"2024-03-27T10:54:41.337153+0000","last_became_active":"2024-03-27T10:54:41.336433+0000","last_became_peered":"2024-03-27T10:54:41.336433+0000","last_unstale":"2024-03-27T10:54:41.337153+0000","last_undegraded":"2024-03-27T10:54:41.337153+0000","last_fullsized":"2024-03-27T10:54:41.337153+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:00:25.944147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,7],"acting":[1,8,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"90'30","reported_seq":225,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:33.356991+0000","last_change":"2024-03-27T10:54:04.061913+0000","last_active":"2024-03-27T10:55:33.356991+0000","last_peered":"2024-03-27T10:55:33.356991+0000","last_clean":"2024-03-27T10:55:33.356991+0000","last_became_active":"2024-03-27T10:54:04.058366+0000","last_became_peered":"2024-03-27T10:54:04.058366+0000","last_unstale":"2024-03-27T10:55:33.356991+0000","last_undegraded":"2024-03-27T10:55:33.356991+0000","last_fullsized":"2024-03-27T10:55:33.356991+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:45:28.365554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61046,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":176,"num_read_kb":147,"num_write":32,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.328195+0000","last_change":"2024-03-27T10:54:05.090840+0000","last_active":"2024-03-27T10:54:41.328195+0000","last_peered":"2024-03-27T10:54:41.328195+0000","last_clean":"2024-03-27T10:54:41.328195+0000","last_became_active":"2024-03-27T10:54:05.090525+0000","last_became_peered":"2024-03-27T10:54:05.090525+0000","last_unstale":"2024-03-27T10:54:41.328195+0000","last_undegraded":"2024-03-27T10:54:41.328195+0000","last_fullsized":"2024-03-27T10:54:41.328195+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T21:26:10.559917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"90'9","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.334274+0000","last_change":"2024-03-27T10:54:04.061205+0000","last_active":"2024-03-27T10:54:41.334274+0000","last_peered":"2024-03-27T10:54:41.334274+0000","last_clean":"2024-03-27T10:54:41.334274+0000","last_became_active":"2024-03-27T10:54:04.059558+0000","last_became_peered":"2024-03-27T10:54:04.059558+0000","last_unstale":"2024-03-27T10:54:41.334274+0000","last_undegraded":"2024-03-27T10:54:41.334274+0000","last_fullsized":"2024-03-27T10:54:41.334274+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T12:00:00.182233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.369558+0000","last_change":"2024-03-27T10:54:05.090149+0000","last_active":"2024-03-27T10:54:41.369558+0000","last_peered":"2024-03-27T10:54:41.369558+0000","last_clean":"2024-03-27T10:54:41.369558+0000","last_became_active":"2024-03-27T10:54:05.089317+0000","last_became_peered":"2024-03-27T10:54:05.089317+0000","last_unstale":"2024-03-27T10:54:41.369558+0000","last_undegraded":"2024-03-27T10:54:41.369558+0000","last_fullsized":"2024-03-27T10:54:41.369558+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T11:50:06.477472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"90'5","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.333525+0000","last_change":"2024-03-27T10:54:04.061797+0000","last_active":"2024-03-27T10:54:41.333525+0000","last_peered":"2024-03-27T10:54:41.333525+0000","last_clean":"2024-03-27T10:54:41.333525+0000","last_became_active":"2024-03-27T10:54:04.060974+0000","last_became_peered":"2024-03-27T10:54:04.060974+0000","last_unstale":"2024-03-27T10:54:41.333525+0000","last_undegraded":"2024-03-27T10:54:41.333525+0000","last_fullsized":"2024-03-27T10:54:41.333525+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T18:34:33.486126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.370832+0000","last_change":"2024-03-27T10:54:41.370832+0000","last_active":"2024-03-27T10:54:41.370832+0000","last_peered":"2024-03-27T10:54:41.370832+0000","last_clean":"2024-03-27T10:54:41.370832+0000","last_became_active":"2024-03-27T10:54:41.370229+0000","last_became_peered":"2024-03-27T10:54:41.370229+0000","last_unstale":"2024-03-27T10:54:41.370832+0000","last_undegraded":"2024-03-27T10:54:41.370832+0000","last_fullsized":"2024-03-27T10:54:41.370832+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:53:19.887501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"90'6","reported_seq":111,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:31.369661+0000","last_change":"2024-03-27T10:54:04.060042+0000","last_active":"2024-03-27T10:55:31.369661+0000","last_peered":"2024-03-27T10:55:31.369661+0000","last_clean":"2024-03-27T10:55:31.369661+0000","last_became_active":"2024-03-27T10:54:04.059415+0000","last_became_peered":"2024-03-27T10:54:04.059415+0000","last_unstale":"2024-03-27T10:55:31.369661+0000","last_undegraded":"2024-03-27T10:55:31.369661+0000","last_fullsized":"2024-03-27T10:55:31.369661+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:52:06.846942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":87,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.331929+0000","last_change":"2024-03-27T10:54:05.100088+0000","last_active":"2024-03-27T10:54:41.331929+0000","last_peered":"2024-03-27T10:54:41.331929+0000","last_clean":"2024-03-27T10:54:41.331929+0000","last_became_active":"2024-03-27T10:54:05.098709+0000","last_became_peered":"2024-03-27T10:54:05.098709+0000","last_unstale":"2024-03-27T10:54:41.331929+0000","last_undegraded":"2024-03-27T10:54:41.331929+0000","last_fullsized":"2024-03-27T10:54:41.331929+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T21:25:13.329710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"90'17","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.327421+0000","last_change":"2024-03-27T10:54:04.066812+0000","last_active":"2024-03-27T10:54:41.327421+0000","last_peered":"2024-03-27T10:54:41.327421+0000","last_clean":"2024-03-27T10:54:41.327421+0000","last_became_active":"2024-03-27T10:54:04.066364+0000","last_became_peered":"2024-03-27T10:54:04.066364+0000","last_unstale":"2024-03-27T10:54:41.327421+0000","last_undegraded":"2024-03-27T10:54:41.327421+0000","last_fullsized":"2024-03-27T10:54:41.327421+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T19:31:51.616785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.333942+0000","last_change":"2024-03-27T10:54:05.089367+0000","last_active":"2024-03-27T10:54:41.333942+0000","last_peered":"2024-03-27T10:54:41.333942+0000","last_clean":"2024-03-27T10:54:41.333942+0000","last_became_active":"2024-03-27T10:54:05.088718+0000","last_became_peered":"2024-03-27T10:54:05.088718+0000","last_unstale":"2024-03-27T10:54:41.333942+0000","last_undegraded":"2024-03-27T10:54:41.333942+0000","last_fullsized":"2024-03-27T10:54:41.333942+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:56:08.768859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.327949+0000","last_change":"2024-03-27T10:54:04.059951+0000","last_active":"2024-03-27T10:54:41.327949+0000","last_peered":"2024-03-27T10:54:41.327949+0000","last_clean":"2024-03-27T10:54:41.327949+0000","last_became_active":"2024-03-27T10:54:04.057245+0000","last_became_peered":"2024-03-27T10:54:04.057245+0000","last_unstale":"2024-03-27T10:54:41.327949+0000","last_undegraded":"2024-03-27T10:54:41.327949+0000","last_fullsized":"2024-03-27T10:54:41.327949+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T19:36:48.881920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.331808+0000","last_change":"2024-03-27T10:54:05.089004+0000","last_active":"2024-03-27T10:54:41.331808+0000","last_peered":"2024-03-27T10:54:41.331808+0000","last_clean":"2024-03-27T10:54:41.331808+0000","last_became_active":"2024-03-27T10:54:05.083764+0000","last_became_peered":"2024-03-27T10:54:05.083764+0000","last_unstale":"2024-03-27T10:54:41.331808+0000","last_undegraded":"2024-03-27T10:54:41.331808+0000","last_fullsized":"2024-03-27T10:54:41.331808+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:09:53.883490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"90'1","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.326382+0000","last_change":"2024-03-27T10:54:04.050508+0000","last_active":"2024-03-27T10:54:41.326382+0000","last_peered":"2024-03-27T10:54:41.326382+0000","last_clean":"2024-03-27T10:54:41.326382+0000","last_became_active":"2024-03-27T10:54:04.048877+0000","last_became_peered":"2024-03-27T10:54:04.048877+0000","last_unstale":"2024-03-27T10:54:41.326382+0000","last_undegraded":"2024-03-27T10:54:41.326382+0000","last_fullsized":"2024-03-27T10:54:41.326382+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:44:11.578684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"90'1","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.334664+0000","last_change":"2024-03-27T10:54:05.083931+0000","last_active":"2024-03-27T10:54:41.334664+0000","last_peered":"2024-03-27T10:54:41.334664+0000","last_clean":"2024-03-27T10:54:41.334664+0000","last_became_active":"2024-03-27T10:54:05.083544+0000","last_became_peered":"2024-03-27T10:54:05.083544+0000","last_unstale":"2024-03-27T10:54:41.334664+0000","last_undegraded":"2024-03-27T10:54:41.334664+0000","last_fullsized":"2024-03-27T10:54:41.334664+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T20:54:19.172355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"90'5","reported_seq":108,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:33.398095+0000","last_change":"2024-03-27T10:54:04.057428+0000","last_active":"2024-03-27T10:55:33.398095+0000","last_peered":"2024-03-27T10:55:33.398095+0000","last_clean":"2024-03-27T10:55:33.398095+0000","last_became_active":"2024-03-27T10:54:04.052966+0000","last_became_peered":"2024-03-27T10:54:04.052966+0000","last_unstale":"2024-03-27T10:55:33.398095+0000","last_undegraded":"2024-03-27T10:55:33.398095+0000","last_fullsized":"2024-03-27T10:55:33.398095+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T18:50:31.170411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":84,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.438829+0000","last_change":"2024-03-27T10:54:41.438829+0000","last_active":"2024-03-27T10:54:41.438829+0000","last_peered":"2024-03-27T10:54:41.438829+0000","last_clean":"2024-03-27T10:54:41.438829+0000","last_became_active":"2024-03-27T10:54:41.438031+0000","last_became_peered":"2024-03-27T10:54:41.438031+0000","last_unstale":"2024-03-27T10:54:41.438829+0000","last_undegraded":"2024-03-27T10:54:41.438829+0000","last_fullsized":"2024-03-27T10:54:41.438829+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:33:04.052747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.332074+0000","last_change":"2024-03-27T10:54:05.090255+0000","last_active":"2024-03-27T10:54:41.332074+0000","last_peered":"2024-03-27T10:54:41.332074+0000","last_clean":"2024-03-27T10:54:41.332074+0000","last_became_active":"2024-03-27T10:54:05.088833+0000","last_became_peered":"2024-03-27T10:54:05.088833+0000","last_unstale":"2024-03-27T10:54:41.332074+0000","last_undegraded":"2024-03-27T10:54:41.332074+0000","last_fullsized":"2024-03-27T10:54:41.332074+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:15:55.934698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'5","reported_seq":109,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:32.371164+0000","last_change":"2024-03-27T10:54:04.059941+0000","last_active":"2024-03-27T10:55:32.371164+0000","last_peered":"2024-03-27T10:55:32.371164+0000","last_clean":"2024-03-27T10:55:32.371164+0000","last_became_active":"2024-03-27T10:54:04.058197+0000","last_became_peered":"2024-03-27T10:54:04.058197+0000","last_unstale":"2024-03-27T10:55:32.371164+0000","last_undegraded":"2024-03-27T10:55:32.371164+0000","last_fullsized":"2024-03-27T10:55:32.371164+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:44:46.374054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":85,"num_read_kb":85,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.338903+0000","last_change":"2024-03-27T10:54:41.337917+0000","last_active":"2024-03-27T10:54:41.338903+0000","last_peered":"2024-03-27T10:54:41.338903+0000","last_clean":"2024-03-27T10:54:41.338903+0000","last_became_active":"2024-03-27T10:54:41.337624+0000","last_became_peered":"2024-03-27T10:54:41.337624+0000","last_unstale":"2024-03-27T10:54:41.338903+0000","last_undegraded":"2024-03-27T10:54:41.338903+0000","last_fullsized":"2024-03-27T10:54:41.338903+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T11:50:48.275234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"90'6","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.327492+0000","last_change":"2024-03-27T10:54:04.064017+0000","last_active":"2024-03-27T10:54:41.327492+0000","last_peered":"2024-03-27T10:54:41.327492+0000","last_clean":"2024-03-27T10:54:41.327492+0000","last_became_active":"2024-03-27T10:54:04.062881+0000","last_became_peered":"2024-03-27T10:54:04.062881+0000","last_unstale":"2024-03-27T10:54:41.327492+0000","last_undegraded":"2024-03-27T10:54:41.327492+0000","last_fullsized":"2024-03-27T10:54:41.327492+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:21:21.638623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":349,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.333171+0000","last_change":"2024-03-27T10:52:31.127495+0000","last_active":"2024-03-27T10:54:41.333171+0000","last_peered":"2024-03-27T10:54:41.333171+0000","last_clean":"2024-03-27T10:54:41.333171+0000","last_became_active":"2024-03-27T10:52:31.125418+0000","last_became_peered":"2024-03-27T10:52:31.125418+0000","last_unstale":"2024-03-27T10:54:41.333171+0000","last_undegraded":"2024-03-27T10:54:41.333171+0000","last_fullsized":"2024-03-27T10:54:41.333171+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:47:54.649615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:47:54.649615+0000","last_clean_scrub_stamp":"2024-03-27T10:47:54.649615+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T18:49:24.591580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.369133+0000","last_change":"2024-03-27T10:54:05.085558+0000","last_active":"2024-03-27T10:54:41.369133+0000","last_peered":"2024-03-27T10:54:41.369133+0000","last_clean":"2024-03-27T10:54:41.369133+0000","last_became_active":"2024-03-27T10:54:05.083896+0000","last_became_peered":"2024-03-27T10:54:05.083896+0000","last_unstale":"2024-03-27T10:54:41.369133+0000","last_undegraded":"2024-03-27T10:54:41.369133+0000","last_fullsized":"2024-03-27T10:54:41.369133+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:15:29.697609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'4","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.333242+0000","last_change":"2024-03-27T10:54:04.072448+0000","last_active":"2024-03-27T10:54:41.333242+0000","last_peered":"2024-03-27T10:54:41.333242+0000","last_clean":"2024-03-27T10:54:41.333242+0000","last_became_active":"2024-03-27T10:54:04.071933+0000","last_became_peered":"2024-03-27T10:54:04.071933+0000","last_unstale":"2024-03-27T10:54:41.333242+0000","last_undegraded":"2024-03-27T10:54:41.333242+0000","last_fullsized":"2024-03-27T10:54:41.333242+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:34:49.204345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"90'4","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.328648+0000","last_change":"2024-03-27T10:54:04.067308+0000","last_active":"2024-03-27T10:54:41.328648+0000","last_peered":"2024-03-27T10:54:41.328648+0000","last_clean":"2024-03-27T10:54:41.328648+0000","last_became_active":"2024-03-27T10:54:04.066927+0000","last_became_peered":"2024-03-27T10:54:04.066927+0000","last_unstale":"2024-03-27T10:54:41.328648+0000","last_undegraded":"2024-03-27T10:54:41.328648+0000","last_fullsized":"2024-03-27T10:54:41.328648+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:08:04.849245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.369740+0000","last_change":"2024-03-27T10:54:05.075154+0000","last_active":"2024-03-27T10:54:41.369740+0000","last_peered":"2024-03-27T10:54:41.369740+0000","last_clean":"2024-03-27T10:54:41.369740+0000","last_became_active":"2024-03-27T10:54:05.074772+0000","last_became_peered":"2024-03-27T10:54:05.074772+0000","last_unstale":"2024-03-27T10:54:41.369740+0000","last_undegraded":"2024-03-27T10:54:41.369740+0000","last_fullsized":"2024-03-27T10:54:41.369740+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T20:49:16.286229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"90'13","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.326583+0000","last_change":"2024-03-27T10:54:04.066805+0000","last_active":"2024-03-27T10:54:41.326583+0000","last_peered":"2024-03-27T10:54:41.326583+0000","last_clean":"2024-03-27T10:54:41.326583+0000","last_became_active":"2024-03-27T10:54:04.066500+0000","last_became_peered":"2024-03-27T10:54:04.066500+0000","last_unstale":"2024-03-27T10:54:41.326583+0000","last_undegraded":"2024-03-27T10:54:41.326583+0000","last_fullsized":"2024-03-27T10:54:41.326583+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T12:37:44.853575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.337096+0000","last_change":"2024-03-27T10:54:05.094314+0000","last_active":"2024-03-27T10:54:41.337096+0000","last_peered":"2024-03-27T10:54:41.337096+0000","last_clean":"2024-03-27T10:54:41.337096+0000","last_became_active":"2024-03-27T10:54:05.093896+0000","last_became_peered":"2024-03-27T10:54:05.093896+0000","last_unstale":"2024-03-27T10:54:41.337096+0000","last_undegraded":"2024-03-27T10:54:41.337096+0000","last_fullsized":"2024-03-27T10:54:41.337096+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:30:02.441317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"90'7","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.328683+0000","last_change":"2024-03-27T10:54:04.065525+0000","last_active":"2024-03-27T10:54:41.328683+0000","last_peered":"2024-03-27T10:54:41.328683+0000","last_clean":"2024-03-27T10:54:41.328683+0000","last_became_active":"2024-03-27T10:54:04.064979+0000","last_became_peered":"2024-03-27T10:54:04.064979+0000","last_unstale":"2024-03-27T10:54:41.328683+0000","last_undegraded":"2024-03-27T10:54:41.328683+0000","last_fullsized":"2024-03-27T10:54:41.328683+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:58:25.110861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.437342+0000","last_change":"2024-03-27T10:54:05.084038+0000","last_active":"2024-03-27T10:54:41.437342+0000","last_peered":"2024-03-27T10:54:41.437342+0000","last_clean":"2024-03-27T10:54:41.437342+0000","last_became_active":"2024-03-27T10:54:05.083733+0000","last_became_peered":"2024-03-27T10:54:05.083733+0000","last_unstale":"2024-03-27T10:54:41.437342+0000","last_undegraded":"2024-03-27T10:54:41.437342+0000","last_fullsized":"2024-03-27T10:54:41.437342+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T18:51:14.832602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.330469+0000","last_change":"2024-03-27T10:54:04.064506+0000","last_active":"2024-03-27T10:54:41.330469+0000","last_peered":"2024-03-27T10:54:41.330469+0000","last_clean":"2024-03-27T10:54:41.330469+0000","last_became_active":"2024-03-27T10:54:04.064178+0000","last_became_peered":"2024-03-27T10:54:04.064178+0000","last_unstale":"2024-03-27T10:54:41.330469+0000","last_undegraded":"2024-03-27T10:54:41.330469+0000","last_fullsized":"2024-03-27T10:54:41.330469+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:07:19.465782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.327649+0000","last_change":"2024-03-27T10:54:05.088241+0000","last_active":"2024-03-27T10:54:41.327649+0000","last_peered":"2024-03-27T10:54:41.327649+0000","last_clean":"2024-03-27T10:54:41.327649+0000","last_became_active":"2024-03-27T10:54:05.087852+0000","last_became_peered":"2024-03-27T10:54:05.087852+0000","last_unstale":"2024-03-27T10:54:41.327649+0000","last_undegraded":"2024-03-27T10:54:41.327649+0000","last_fullsized":"2024-03-27T10:54:41.327649+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:47:09.633654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"90'7","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.336358+0000","last_change":"2024-03-27T10:54:04.065744+0000","last_active":"2024-03-27T10:54:41.336358+0000","last_peered":"2024-03-27T10:54:41.336358+0000","last_clean":"2024-03-27T10:54:41.336358+0000","last_became_active":"2024-03-27T10:54:04.065155+0000","last_became_peered":"2024-03-27T10:54:04.065155+0000","last_unstale":"2024-03-27T10:54:41.336358+0000","last_undegraded":"2024-03-27T10:54:41.336358+0000","last_fullsized":"2024-03-27T10:54:41.336358+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T12:35:00.556458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.331050+0000","last_change":"2024-03-27T10:54:05.077954+0000","last_active":"2024-03-27T10:54:41.331050+0000","last_peered":"2024-03-27T10:54:41.331050+0000","last_clean":"2024-03-27T10:54:41.331050+0000","last_became_active":"2024-03-27T10:54:05.077589+0000","last_became_peered":"2024-03-27T10:54:05.077589+0000","last_unstale":"2024-03-27T10:54:41.331050+0000","last_undegraded":"2024-03-27T10:54:41.331050+0000","last_fullsized":"2024-03-27T10:54:41.331050+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T20:18:27.411887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"90'3","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.353201+0000","last_change":"2024-03-27T10:54:41.352576+0000","last_active":"2024-03-27T10:54:41.353201+0000","last_peered":"2024-03-27T10:54:41.353201+0000","last_clean":"2024-03-27T10:54:41.353201+0000","last_became_active":"2024-03-27T10:54:41.338775+0000","last_became_peered":"2024-03-27T10:54:41.338775+0000","last_unstale":"2024-03-27T10:54:41.353201+0000","last_undegraded":"2024-03-27T10:54:41.353201+0000","last_fullsized":"2024-03-27T10:54:41.353201+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T18:23:14.194920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.336442+0000","last_change":"2024-03-27T10:54:05.088250+0000","last_active":"2024-03-27T10:54:41.336442+0000","last_peered":"2024-03-27T10:54:41.336442+0000","last_clean":"2024-03-27T10:54:41.336442+0000","last_became_active":"2024-03-27T10:54:05.087942+0000","last_became_peered":"2024-03-27T10:54:05.087942+0000","last_unstale":"2024-03-27T10:54:41.336442+0000","last_undegraded":"2024-03-27T10:54:41.336442+0000","last_fullsized":"2024-03-27T10:54:41.336442+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:57:45.266347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.327668+0000","last_change":"2024-03-27T10:54:04.070513+0000","last_active":"2024-03-27T10:54:41.327668+0000","last_peered":"2024-03-27T10:54:41.327668+0000","last_clean":"2024-03-27T10:54:41.327668+0000","last_became_active":"2024-03-27T10:54:04.070042+0000","last_became_peered":"2024-03-27T10:54:04.070042+0000","last_unstale":"2024-03-27T10:54:41.327668+0000","last_undegraded":"2024-03-27T10:54:41.327668+0000","last_fullsized":"2024-03-27T10:54:41.327668+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:40:25.216899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.334218+0000","last_change":"2024-03-27T10:54:05.095067+0000","last_active":"2024-03-27T10:54:41.334218+0000","last_peered":"2024-03-27T10:54:41.334218+0000","last_clean":"2024-03-27T10:54:41.334218+0000","last_became_active":"2024-03-27T10:54:05.094618+0000","last_became_peered":"2024-03-27T10:54:05.094618+0000","last_unstale":"2024-03-27T10:54:41.334218+0000","last_undegraded":"2024-03-27T10:54:41.334218+0000","last_fullsized":"2024-03-27T10:54:41.334218+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:57:59.557310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"90'5","reported_seq":108,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:32.397666+0000","last_change":"2024-03-27T10:54:04.051352+0000","last_active":"2024-03-27T10:55:32.397666+0000","last_peered":"2024-03-27T10:55:32.397666+0000","last_clean":"2024-03-27T10:55:32.397666+0000","last_became_active":"2024-03-27T10:54:04.050850+0000","last_became_peered":"2024-03-27T10:54:04.050850+0000","last_unstale":"2024-03-27T10:55:32.397666+0000","last_undegraded":"2024-03-27T10:55:32.397666+0000","last_fullsized":"2024-03-27T10:55:32.397666+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:50:16.719451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":84,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.333591+0000","last_change":"2024-03-27T10:54:05.094650+0000","last_active":"2024-03-27T10:54:41.333591+0000","last_peered":"2024-03-27T10:54:41.333591+0000","last_clean":"2024-03-27T10:54:41.333591+0000","last_became_active":"2024-03-27T10:54:05.094168+0000","last_became_peered":"2024-03-27T10:54:05.094168+0000","last_unstale":"2024-03-27T10:54:41.333591+0000","last_undegraded":"2024-03-27T10:54:41.333591+0000","last_fullsized":"2024-03-27T10:54:41.333591+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:29:56.491733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"90'3","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.334578+0000","last_change":"2024-03-27T10:54:04.067755+0000","last_active":"2024-03-27T10:54:41.334578+0000","last_peered":"2024-03-27T10:54:41.334578+0000","last_clean":"2024-03-27T10:54:41.334578+0000","last_became_active":"2024-03-27T10:54:04.067320+0000","last_became_peered":"2024-03-27T10:54:04.067320+0000","last_unstale":"2024-03-27T10:54:41.334578+0000","last_undegraded":"2024-03-27T10:54:41.334578+0000","last_fullsized":"2024-03-27T10:54:41.334578+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T12:16:25.949829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"90'1","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.333325+0000","last_change":"2024-03-27T10:54:05.100715+0000","last_active":"2024-03-27T10:54:41.333325+0000","last_peered":"2024-03-27T10:54:41.333325+0000","last_clean":"2024-03-27T10:54:41.333325+0000","last_became_active":"2024-03-27T10:54:05.097861+0000","last_became_peered":"2024-03-27T10:54:05.097861+0000","last_unstale":"2024-03-27T10:54:41.333325+0000","last_undegraded":"2024-03-27T10:54:41.333325+0000","last_fullsized":"2024-03-27T10:54:41.333325+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:02:39.107232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"90'5","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.369686+0000","last_change":"2024-03-27T10:54:04.063894+0000","last_active":"2024-03-27T10:54:41.369686+0000","last_peered":"2024-03-27T10:54:41.369686+0000","last_clean":"2024-03-27T10:54:41.369686+0000","last_became_active":"2024-03-27T10:54:04.063163+0000","last_became_peered":"2024-03-27T10:54:04.063163+0000","last_unstale":"2024-03-27T10:54:41.369686+0000","last_undegraded":"2024-03-27T10:54:41.369686+0000","last_fullsized":"2024-03-27T10:54:41.369686+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:30:41.025752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.333820+0000","last_change":"2024-03-27T10:54:05.101980+0000","last_active":"2024-03-27T10:54:41.333820+0000","last_peered":"2024-03-27T10:54:41.333820+0000","last_clean":"2024-03-27T10:54:41.333820+0000","last_became_active":"2024-03-27T10:54:05.101448+0000","last_became_peered":"2024-03-27T10:54:05.101448+0000","last_unstale":"2024-03-27T10:54:41.333820+0000","last_undegraded":"2024-03-27T10:54:41.333820+0000","last_fullsized":"2024-03-27T10:54:41.333820+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:45:11.137181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.331340+0000","last_change":"2024-03-27T10:54:05.084026+0000","last_active":"2024-03-27T10:54:41.331340+0000","last_peered":"2024-03-27T10:54:41.331340+0000","last_clean":"2024-03-27T10:54:41.331340+0000","last_became_active":"2024-03-27T10:54:05.083410+0000","last_became_peered":"2024-03-27T10:54:05.083410+0000","last_unstale":"2024-03-27T10:54:41.331340+0000","last_undegraded":"2024-03-27T10:54:41.331340+0000","last_fullsized":"2024-03-27T10:54:41.331340+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T22:39:34.586363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.334292+0000","last_change":"2024-03-27T10:54:05.083592+0000","last_active":"2024-03-27T10:54:41.334292+0000","last_peered":"2024-03-27T10:54:41.334292+0000","last_clean":"2024-03-27T10:54:41.334292+0000","last_became_active":"2024-03-27T10:54:05.082537+0000","last_became_peered":"2024-03-27T10:54:05.082537+0000","last_unstale":"2024-03-27T10:54:41.334292+0000","last_undegraded":"2024-03-27T10:54:41.334292+0000","last_fullsized":"2024-03-27T10:54:41.334292+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T21:15:36.830805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.334858+0000","last_change":"2024-03-27T10:54:05.090226+0000","last_active":"2024-03-27T10:54:41.334858+0000","last_peered":"2024-03-27T10:54:41.334858+0000","last_clean":"2024-03-27T10:54:41.334858+0000","last_became_active":"2024-03-27T10:54:05.089831+0000","last_became_peered":"2024-03-27T10:54:05.089831+0000","last_unstale":"2024-03-27T10:54:41.334858+0000","last_undegraded":"2024-03-27T10:54:41.334858+0000","last_fullsized":"2024-03-27T10:54:41.334858+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T12:45:57.802266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.437134+0000","last_change":"2024-03-27T10:54:05.080196+0000","last_active":"2024-03-27T10:54:41.437134+0000","last_peered":"2024-03-27T10:54:41.437134+0000","last_clean":"2024-03-27T10:54:41.437134+0000","last_became_active":"2024-03-27T10:54:05.079822+0000","last_became_peered":"2024-03-27T10:54:05.079822+0000","last_unstale":"2024-03-27T10:54:41.437134+0000","last_undegraded":"2024-03-27T10:54:41.437134+0000","last_fullsized":"2024-03-27T10:54:41.437134+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:53:11.556503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.369643+0000","last_change":"2024-03-27T10:54:05.087264+0000","last_active":"2024-03-27T10:54:41.369643+0000","last_peered":"2024-03-27T10:54:41.369643+0000","last_clean":"2024-03-27T10:54:41.369643+0000","last_became_active":"2024-03-27T10:54:05.085440+0000","last_became_peered":"2024-03-27T10:54:05.085440+0000","last_unstale":"2024-03-27T10:54:41.369643+0000","last_undegraded":"2024-03-27T10:54:41.369643+0000","last_fullsized":"2024-03-27T10:54:41.369643+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T22:41:40.665643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.331178+0000","last_change":"2024-03-27T10:54:05.094306+0000","last_active":"2024-03-27T10:54:41.331178+0000","last_peered":"2024-03-27T10:54:41.331178+0000","last_clean":"2024-03-27T10:54:41.331178+0000","last_became_active":"2024-03-27T10:54:05.093858+0000","last_became_peered":"2024-03-27T10:54:05.093858+0000","last_unstale":"2024-03-27T10:54:41.331178+0000","last_undegraded":"2024-03-27T10:54:41.331178+0000","last_fullsized":"2024-03-27T10:54:41.331178+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T21:22:57.624833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.334445+0000","last_change":"2024-03-27T10:54:05.094089+0000","last_active":"2024-03-27T10:54:41.334445+0000","last_peered":"2024-03-27T10:54:41.334445+0000","last_clean":"2024-03-27T10:54:41.334445+0000","last_became_active":"2024-03-27T10:54:05.092317+0000","last_became_peered":"2024-03-27T10:54:05.092317+0000","last_unstale":"2024-03-27T10:54:41.334445+0000","last_undegraded":"2024-03-27T10:54:41.334445+0000","last_fullsized":"2024-03-27T10:54:41.334445+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:34:57.606250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.336968+0000","last_change":"2024-03-27T10:54:05.099995+0000","last_active":"2024-03-27T10:54:41.336968+0000","last_peered":"2024-03-27T10:54:41.336968+0000","last_clean":"2024-03-27T10:54:41.336968+0000","last_became_active":"2024-03-27T10:54:05.099641+0000","last_became_peered":"2024-03-27T10:54:05.099641+0000","last_unstale":"2024-03-27T10:54:41.336968+0000","last_undegraded":"2024-03-27T10:54:41.336968+0000","last_fullsized":"2024-03-27T10:54:41.336968+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:01:16.050180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.371472+0000","last_change":"2024-03-27T10:54:41.371472+0000","last_active":"2024-03-27T10:54:41.371472+0000","last_peered":"2024-03-27T10:54:41.371472+0000","last_clean":"2024-03-27T10:54:41.371472+0000","last_became_active":"2024-03-27T10:54:41.370912+0000","last_became_peered":"2024-03-27T10:54:41.370912+0000","last_unstale":"2024-03-27T10:54:41.371472+0000","last_undegraded":"2024-03-27T10:54:41.371472+0000","last_fullsized":"2024-03-27T10:54:41.371472+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T20:21:28.950573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6],"acting":[5,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.335834+0000","last_change":"2024-03-27T10:54:05.096812+0000","last_active":"2024-03-27T10:54:41.335834+0000","last_peered":"2024-03-27T10:54:41.335834+0000","last_clean":"2024-03-27T10:54:41.335834+0000","last_became_active":"2024-03-27T10:54:05.096323+0000","last_became_peered":"2024-03-27T10:54:05.096323+0000","last_unstale":"2024-03-27T10:54:41.335834+0000","last_undegraded":"2024-03-27T10:54:41.335834+0000","last_fullsized":"2024-03-27T10:54:41.335834+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:39:29.827291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.332559+0000","last_change":"2024-03-27T10:54:05.089137+0000","last_active":"2024-03-27T10:54:41.332559+0000","last_peered":"2024-03-27T10:54:41.332559+0000","last_clean":"2024-03-27T10:54:41.332559+0000","last_became_active":"2024-03-27T10:54:05.088795+0000","last_became_peered":"2024-03-27T10:54:05.088795+0000","last_unstale":"2024-03-27T10:54:41.332559+0000","last_undegraded":"2024-03-27T10:54:41.332559+0000","last_fullsized":"2024-03-27T10:54:41.332559+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T19:47:35.231749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.369503+0000","last_change":"2024-03-27T10:54:05.089683+0000","last_active":"2024-03-27T10:54:41.369503+0000","last_peered":"2024-03-27T10:54:41.369503+0000","last_clean":"2024-03-27T10:54:41.369503+0000","last_became_active":"2024-03-27T10:54:05.089235+0000","last_became_peered":"2024-03-27T10:54:05.089235+0000","last_unstale":"2024-03-27T10:54:41.369503+0000","last_undegraded":"2024-03-27T10:54:41.369503+0000","last_fullsized":"2024-03-27T10:54:41.369503+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:43:37.012814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.332471+0000","last_change":"2024-03-27T10:54:05.085043+0000","last_active":"2024-03-27T10:54:41.332471+0000","last_peered":"2024-03-27T10:54:41.332471+0000","last_clean":"2024-03-27T10:54:41.332471+0000","last_became_active":"2024-03-27T10:54:05.084695+0000","last_became_peered":"2024-03-27T10:54:05.084695+0000","last_unstale":"2024-03-27T10:54:41.332471+0000","last_undegraded":"2024-03-27T10:54:41.332471+0000","last_fullsized":"2024-03-27T10:54:41.332471+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T12:07:31.327006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68250,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":870,"num_read_kb":508,"num_write":239,"num_write_kb":133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":204750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":207,"ondisk_log_size":207,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":85,"seq":365072220194,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1007616,"data_stored":481025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":77,"seq":330712481834,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37004,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708148,"statfs":{"total":95995035648,"available":95957143552,"internally_reserved":0,"allocated":1703936,"data_stored":1132909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":71,"seq":304942678064,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1028096,"data_stored":480373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":63,"seq":270582939703,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1019904,"data_stored":480169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":54,"seq":231928234046,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28600,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716552,"statfs":{"total":95995035648,"available":95965749248,"internally_reserved":0,"allocated":1622016,"data_stored":1073660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":49,"seq":210453397572,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1011712,"data_stored":480090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574950,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1073152,"data_stored":542049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738476,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1032192,"data_stored":481429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411423,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1093632,"data_stored":542660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117082,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1028096,"data_stored":479915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":35,"seq":150323855443,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36332,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708820,"statfs":{"total":95995035648,"available":95957831680,"internally_reserved":0,"allocated":1011712,"data_stored":480342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":41,"seq":176093659212,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32780,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712372,"statfs":{"total":95995035648,"available":95961468928,"internally_reserved":0,"allocated":1634304,"data_stored":1072947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":62798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":63409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":63173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-27T10:55:37.494 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-03-27T10:55:37.495 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-27T10:55:37.495 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-03-27T10:55:37.495 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell --fsid adca88c4-ec26-11ee-95d0-87774f69a715 -- ceph health --format=json 2024-03-27T10:55:37.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:55:37 smithi114 ceph-mon[35338]: pgmap v410: 97 pgs: 97 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-27T10:55:37.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:55:37 smithi177 ceph-mon[35294]: pgmap v410: 97 pgs: 97 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-27T10:55:37.988 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T10:55:38.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:55:38 smithi097 ceph-mon[31859]: from='client.14985 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-27T10:55:38.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:55:38 smithi114 ceph-mon[35338]: from='client.14985 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-27T10:55:38.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:55:38 smithi177 ceph-mon[35294]: from='client.14985 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-27T10:55:39.479 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:55:39 smithi097 ceph-mon[31859]: pgmap v411: 97 pgs: 97 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-27T10:55:39.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:55:39 smithi114 ceph-mon[35338]: pgmap v411: 97 pgs: 97 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-27T10:55:39.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:55:39 smithi177 ceph-mon[35294]: pgmap v411: 97 pgs: 97 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-27T10:55:39.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:39.994+0000 7f8acd600700 1 -- 172.21.15.97:0/2845389521 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ac8109860 msgr2=0x7f8ac810d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:39.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:39.994+0000 7f8acd600700 1 --2- 172.21.15.97:0/2845389521 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ac8109860 0x7f8ac810d8b0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8ab40099c0 tx=0x7f8ab4009cd0 comp rx=0 tx=0).stop 2024-03-27T10:55:39.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:39.994+0000 7f8acd600700 1 -- 172.21.15.97:0/2845389521 shutdown_connections 2024-03-27T10:55:39.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:39.994+0000 7f8acd600700 1 --2- 172.21.15.97:0/2845389521 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8ac810df90 0x7f8ac8115b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:39.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:39.994+0000 7f8acd600700 1 --2- 172.21.15.97:0/2845389521 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8ac8108eb0 0x7f8ac8109290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:39.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:39.994+0000 7f8acd600700 1 --2- 172.21.15.97:0/2845389521 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ac8109860 0x7f8ac810d8b0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:39.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:39.994+0000 7f8acd600700 1 -- 172.21.15.97:0/2845389521 >> 172.21.15.97:0/2845389521 conn(0x7f8ac8075e10 msgr2=0x7f8ac8078230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:55:39.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:39.995+0000 7f8acd600700 1 -- 172.21.15.97:0/2845389521 shutdown_connections 2024-03-27T10:55:39.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:39.996+0000 7f8acd600700 1 -- 172.21.15.97:0/2845389521 wait complete. 2024-03-27T10:55:39.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:39.997+0000 7f8acd600700 1 Processor -- start 2024-03-27T10:55:39.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:39.997+0000 7f8acd600700 1 -- start start 2024-03-27T10:55:39.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:39.998+0000 7f8acd600700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ac8108eb0 0x7f8ac819f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:39.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:39.998+0000 7f8acd600700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8ac8109860 0x7f8ac819f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:39.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:39.998+0000 7f8acd600700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8ac810df90 0x7f8ac81a3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:39.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:39.998+0000 7f8acd600700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8ac8118e80 con 0x7f8ac8108eb0 2024-03-27T10:55:39.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:39.998+0000 7f8acd600700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f8ac8118d00 con 0x7f8ac810df90 2024-03-27T10:55:39.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:39.998+0000 7f8acd600700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f8ac8119000 con 0x7f8ac8109860 2024-03-27T10:55:39.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:39.998+0000 7f8ac6a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ac8108eb0 0x7f8ac819f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:39.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:39.998+0000 7f8ac6a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ac8108eb0 0x7f8ac819f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40542/0 (socket says 172.21.15.97:40542) 2024-03-27T10:55:39.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:39.998+0000 7f8ac6a00700 1 -- 172.21.15.97:0/492896990 learned_addr learned my addr 172.21.15.97:0/492896990 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:55:39.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:39.999+0000 7f8ac6000700 1 --2- 172.21.15.97:0/492896990 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8ac8109860 0x7f8ac819f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:39.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:39.999+0000 7f8ac6a00700 1 -- 172.21.15.97:0/492896990 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8ac8109860 msgr2=0x7f8ac819f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:39.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:39.999+0000 7f8ac6a00700 1 --2- 172.21.15.97:0/492896990 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8ac8109860 0x7f8ac819f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:39.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:39.999+0000 7f8ac6a00700 1 -- 172.21.15.97:0/492896990 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8ac810df90 msgr2=0x7f8ac81a3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:39.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:39.999+0000 7f8ac6a00700 1 --2- 172.21.15.97:0/492896990 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8ac810df90 0x7f8ac81a3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:39.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:39.999+0000 7f8ac6a00700 1 -- 172.21.15.97:0/492896990 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ac81a4340 con 0x7f8ac8108eb0 2024-03-27T10:55:39.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:39.999+0000 7f8ac6a00700 1 --2- 172.21.15.97:0/492896990 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ac8108eb0 0x7f8ac819f330 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8ab800e9a0 tx=0x7f8ab800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:55:39.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:40.000+0000 7f8abf400700 1 -- 172.21.15.97:0/492896990 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ab800cbd0 con 0x7f8ac8108eb0 2024-03-27T10:55:39.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:40.000+0000 7f8abf400700 1 -- 172.21.15.97:0/492896990 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8ab800cd70 con 0x7f8ac8108eb0 2024-03-27T10:55:39.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:40.000+0000 7f8acd600700 1 -- 172.21.15.97:0/492896990 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ac81a4630 con 0x7f8ac8108eb0 2024-03-27T10:55:39.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:40.000+0000 7f8abf400700 1 -- 172.21.15.97:0/492896990 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ab8018750 con 0x7f8ac8108eb0 2024-03-27T10:55:39.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:40.000+0000 7f8acd600700 1 -- 172.21.15.97:0/492896990 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ac81abeb0 con 0x7f8ac8108eb0 2024-03-27T10:55:39.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:40.002+0000 7f8acd600700 1 -- 172.21.15.97:0/492896990 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ac8069300 con 0x7f8ac8108eb0 2024-03-27T10:55:39.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:40.005+0000 7f8abf400700 1 -- 172.21.15.97:0/492896990 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138421+0+0 (secure 0 0 0) 0x7f8ab8010430 con 0x7f8ac8108eb0 2024-03-27T10:55:39.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:40.010+0000 7f8abf400700 1 --2- 172.21.15.97:0/492896990 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f8aac0a3830 0x7f8aac0a5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:39.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:40.011+0000 7f8abf400700 1 -- 172.21.15.97:0/492896990 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f8ab80d0140 con 0x7f8ac8108eb0 2024-03-27T10:55:39.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:40.013+0000 7f8ac6000700 1 --2- 172.21.15.97:0/492896990 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f8aac0a3830 0x7f8aac0a5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:39.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:40.014+0000 7f8ac6000700 1 --2- 172.21.15.97:0/492896990 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f8aac0a3830 0x7f8aac0a5cf0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f8ab4000c00 tx=0x7f8ab4005940 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:55:39.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:40.025+0000 7f8abf400700 1 -- 172.21.15.97:0/492896990 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f8ab809cb60 con 0x7f8ac8108eb0 2024-03-27T10:55:40.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:40.313+0000 7f8abf400700 1 -- 172.21.15.97:0/492896990 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f8ab8098ad0 con 0x7f8ac8108eb0 2024-03-27T10:55:40.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:40.339+0000 7f8acd600700 1 -- 172.21.15.97:0/492896990 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f8ac802def0 con 0x7f8ac8108eb0 2024-03-27T10:55:40.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:40.340+0000 7f8abf400700 1 -- 172.21.15.97:0/492896990 <== mon.0 v2:172.21.15.97:3300/0 8 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f8ab80c0400 con 0x7f8ac8108eb0 2024-03-27T10:55:40.311 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T10:55:40.311 INFO:teuthology.orchestra.run.smithi097.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-27T10:55:40.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:40.343+0000 7f8acd600700 1 -- 172.21.15.97:0/492896990 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f8aac0a3830 msgr2=0x7f8aac0a5cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:40.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:40.343+0000 7f8acd600700 1 --2- 172.21.15.97:0/492896990 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f8aac0a3830 0x7f8aac0a5cf0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f8ab4000c00 tx=0x7f8ab4005940 comp rx=0 tx=0).stop 2024-03-27T10:55:40.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:40.343+0000 7f8acd600700 1 -- 172.21.15.97:0/492896990 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ac8108eb0 msgr2=0x7f8ac819f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:40.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:40.343+0000 7f8acd600700 1 --2- 172.21.15.97:0/492896990 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ac8108eb0 0x7f8ac819f330 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8ab800e9a0 tx=0x7f8ab800ed60 comp rx=0 tx=0).stop 2024-03-27T10:55:40.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:40.343+0000 7f8acd600700 1 -- 172.21.15.97:0/492896990 shutdown_connections 2024-03-27T10:55:40.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:40.343+0000 7f8acd600700 1 --2- 172.21.15.97:0/492896990 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f8aac0a3830 0x7f8aac0a5cf0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:40.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:40.343+0000 7f8acd600700 1 --2- 172.21.15.97:0/492896990 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8ac810df90 0x7f8ac81a3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:40.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:40.343+0000 7f8acd600700 1 --2- 172.21.15.97:0/492896990 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ac8108eb0 0x7f8ac819f330 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:40.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:40.343+0000 7f8acd600700 1 --2- 172.21.15.97:0/492896990 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8ac8109860 0x7f8ac819f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:40.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:40.343+0000 7f8acd600700 1 -- 172.21.15.97:0/492896990 >> 172.21.15.97:0/492896990 conn(0x7f8ac8075e10 msgr2=0x7f8ac80780f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:55:40.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:40.343+0000 7f8acd600700 1 -- 172.21.15.97:0/492896990 shutdown_connections 2024-03-27T10:55:40.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:40.343+0000 7f8acd600700 1 -- 172.21.15.97:0/492896990 wait complete. 2024-03-27T10:55:41.038 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-03-27T10:55:41.038 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-27T10:55:41.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.288+0000 7fd352000640 1 -- 172.21.15.97:0/3564535001 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd34c1016b0 msgr2=0x7fd34c101b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:41.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.288+0000 7fd352000640 1 --2- 172.21.15.97:0/3564535001 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd34c1016b0 0x7fd34c101b30 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fd344009ca0 tx=0x7fd34402c0f0 comp rx=0 tx=0).stop 2024-03-27T10:55:41.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.288+0000 7fd352000640 1 -- 172.21.15.97:0/3564535001 shutdown_connections 2024-03-27T10:55:41.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.288+0000 7fd352000640 1 --2- 172.21.15.97:0/3564535001 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd34c1016b0 0x7fd34c101b30 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:41.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.288+0000 7fd352000640 1 --2- 172.21.15.97:0/3564535001 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd34c100d10 0x7fd34c101170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:41.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.288+0000 7fd352000640 1 --2- 172.21.15.97:0/3564535001 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd34c1003b0 0x7fd34c1007d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:41.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.288+0000 7fd352000640 1 -- 172.21.15.97:0/3564535001 >> 172.21.15.97:0/3564535001 conn(0x7fd34c0fbfb0 msgr2=0x7fd34c0fe3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:55:41.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.289+0000 7fd352000640 1 -- 172.21.15.97:0/3564535001 shutdown_connections 2024-03-27T10:55:41.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.289+0000 7fd352000640 1 -- 172.21.15.97:0/3564535001 wait complete. 2024-03-27T10:55:41.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.290+0000 7fd352000640 1 Processor -- start 2024-03-27T10:55:41.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.290+0000 7fd352000640 1 -- start start 2024-03-27T10:55:41.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.290+0000 7fd352000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd34c1003b0 0x7fd34c199f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:41.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.290+0000 7fd352000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd34c100d10 0x7fd34c19a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:41.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.290+0000 7fd352000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd34c1016b0 0x7fd34c1a14e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:41.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.290+0000 7fd352000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd34c10f860 con 0x7fd34c1016b0 2024-03-27T10:55:41.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.290+0000 7fd352000640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd34c10f6e0 con 0x7fd34c100d10 2024-03-27T10:55:41.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.290+0000 7fd352000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd34c10f9e0 con 0x7fd34c1003b0 2024-03-27T10:55:41.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.290+0000 7fd34be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd34c1016b0 0x7fd34c1a14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:41.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.290+0000 7fd34be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd34c1016b0 0x7fd34c1a14e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40560/0 (socket says 172.21.15.97:40560) 2024-03-27T10:55:41.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.290+0000 7fd34be00640 1 -- 172.21.15.97:0/1228356902 learned_addr learned my addr 172.21.15.97:0/1228356902 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:55:41.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.291+0000 7fd34b400640 1 --2- 172.21.15.97:0/1228356902 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd34c1003b0 0x7fd34c199f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:41.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.291+0000 7fd342a00640 1 --2- 172.21.15.97:0/1228356902 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd34c100d10 0x7fd34c19a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:41.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.291+0000 7fd34be00640 1 -- 172.21.15.97:0/1228356902 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd34c1003b0 msgr2=0x7fd34c199f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:41.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.291+0000 7fd34be00640 1 --2- 172.21.15.97:0/1228356902 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd34c1003b0 0x7fd34c199f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:41.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.291+0000 7fd34be00640 1 -- 172.21.15.97:0/1228356902 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd34c100d10 msgr2=0x7fd34c19a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:41.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.291+0000 7fd34be00640 1 --2- 172.21.15.97:0/1228356902 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd34c100d10 0x7fd34c19a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:41.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.291+0000 7fd34be00640 1 -- 172.21.15.97:0/1228356902 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd34c1a1b50 con 0x7fd34c1016b0 2024-03-27T10:55:41.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.291+0000 7fd342a00640 1 --2- 172.21.15.97:0/1228356902 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd34c100d10 0x7fd34c19a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T10:55:41.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.291+0000 7fd34be00640 1 --2- 172.21.15.97:0/1228356902 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd34c1016b0 0x7fd34c1a14e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd34402c700 tx=0x7fd3440328a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:55:41.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.291+0000 7fd348c00640 1 -- 172.21.15.97:0/1228356902 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd344041020 con 0x7fd34c1016b0 2024-03-27T10:55:41.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.291+0000 7fd348c00640 1 -- 172.21.15.97:0/1228356902 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd344003e20 con 0x7fd34c1016b0 2024-03-27T10:55:41.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.291+0000 7fd352000640 1 -- 172.21.15.97:0/1228356902 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd34c1a1de0 con 0x7fd34c1016b0 2024-03-27T10:55:41.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.292+0000 7fd348c00640 1 -- 172.21.15.97:0/1228356902 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd344039040 con 0x7fd34c1016b0 2024-03-27T10:55:41.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.292+0000 7fd352000640 1 -- 172.21.15.97:0/1228356902 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd34c1a22a0 con 0x7fd34c1016b0 2024-03-27T10:55:41.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.293+0000 7fd352000640 1 -- 172.21.15.97:0/1228356902 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd34c10b300 con 0x7fd34c1016b0 2024-03-27T10:55:41.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.295+0000 7fd348c00640 1 -- 172.21.15.97:0/1228356902 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7fd344044020 con 0x7fd34c1016b0 2024-03-27T10:55:41.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.297+0000 7fd348c00640 1 --2- 172.21.15.97:0/1228356902 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fd3300a3690 0x7fd3300a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:41.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.298+0000 7fd34b400640 1 --2- 172.21.15.97:0/1228356902 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fd3300a3690 0x7fd3300a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:41.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.298+0000 7fd348c00640 1 -- 172.21.15.97:0/1228356902 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fd3440f2570 con 0x7fd34c1016b0 2024-03-27T10:55:41.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.298+0000 7fd34b400640 1 --2- 172.21.15.97:0/1228356902 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fd3300a3690 0x7fd3300a5b50 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd33c005fd0 tx=0x7fd33c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:55:41.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.300+0000 7fd348c00640 1 -- 172.21.15.97:0/1228356902 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7fd34404b020 con 0x7fd34c1016b0 2024-03-27T10:55:41.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.503+0000 7fd352000640 1 -- 172.21.15.97:0/1228356902 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd34c0627f0 con 0x7fd34c1016b0 2024-03-27T10:55:41.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.506+0000 7fd348c00640 1 -- 172.21.15.97:0/1228356902 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+22644 (secure 0 0 0) 0x7fd3440bf0f0 con 0x7fd34c1016b0 2024-03-27T10:55:41.478 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T10:55:41.478 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":92,"fsid":"adca88c4-ec26-11ee-95d0-87774f69a715","created":"2024-03-27T10:43:24.680694+0000","modified":"2024-03-27T10:54:41.310812+0000","last_up_change":"2024-03-27T10:53:00.552513+0000","last_in_change":"2024-03-27T10:52:36.302986+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-27T10:47:54.052810+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-27T10:54:02.845060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-27T10:54:03.474151+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b634e5f1-6118-4e2f-95e9-830db390f8d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1746812080},{"type":"v1","addr":"172.21.15.97:6803","nonce":1746812080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1746812080},{"type":"v1","addr":"172.21.15.97:6805","nonce":1746812080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":1746812080},{"type":"v1","addr":"172.21.15.97:6809","nonce":1746812080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1746812080},{"type":"v1","addr":"172.21.15.97:6807","nonce":1746812080}]},"public_addr":"172.21.15.97:6803/1746812080","cluster_addr":"172.21.15.97:6805/1746812080","heartbeat_back_addr":"172.21.15.97:6809/1746812080","heartbeat_front_addr":"172.21.15.97:6807/1746812080","state":["exists","up"]},{"osd":1,"uuid":"441e2770-3284-4388-b0de-c56eb8a373d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":3858821264},{"type":"v1","addr":"172.21.15.114:6801","nonce":3858821264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":3858821264},{"type":"v1","addr":"172.21.15.114:6803","nonce":3858821264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":3858821264},{"type":"v1","addr":"172.21.15.114:6807","nonce":3858821264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":3858821264},{"type":"v1","addr":"172.21.15.114:6805","nonce":3858821264}]},"public_addr":"172.21.15.114:6801/3858821264","cluster_addr":"172.21.15.114:6803/3858821264","heartbeat_back_addr":"172.21.15.114:6807/3858821264","heartbeat_front_addr":"172.21.15.114:6805/3858821264","state":["exists","up"]},{"osd":2,"uuid":"00108864-1aa6-48d6-8398-e8eb79fda123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":124240658},{"type":"v1","addr":"172.21.15.177:6801","nonce":124240658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":124240658},{"type":"v1","addr":"172.21.15.177:6803","nonce":124240658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":124240658},{"type":"v1","addr":"172.21.15.177:6807","nonce":124240658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":124240658},{"type":"v1","addr":"172.21.15.177:6805","nonce":124240658}]},"public_addr":"172.21.15.177:6801/124240658","cluster_addr":"172.21.15.177:6803/124240658","heartbeat_back_addr":"172.21.15.177:6807/124240658","heartbeat_front_addr":"172.21.15.177:6805/124240658","state":["exists","up"]},{"osd":3,"uuid":"1860b690-e2dc-4977-88d7-d1aacf7022ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2478179311},{"type":"v1","addr":"172.21.15.97:6811","nonce":2478179311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2478179311},{"type":"v1","addr":"172.21.15.97:6813","nonce":2478179311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":2478179311},{"type":"v1","addr":"172.21.15.97:6817","nonce":2478179311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2478179311},{"type":"v1","addr":"172.21.15.97:6815","nonce":2478179311}]},"public_addr":"172.21.15.97:6811/2478179311","cluster_addr":"172.21.15.97:6813/2478179311","heartbeat_back_addr":"172.21.15.97:6817/2478179311","heartbeat_front_addr":"172.21.15.97:6815/2478179311","state":["exists","up"]},{"osd":4,"uuid":"9245a3a4-7d49-4fdb-9d55-112e43177050","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":75755945},{"type":"v1","addr":"172.21.15.114:6809","nonce":75755945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":75755945},{"type":"v1","addr":"172.21.15.114:6811","nonce":75755945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":75755945},{"type":"v1","addr":"172.21.15.114:6815","nonce":75755945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":75755945},{"type":"v1","addr":"172.21.15.114:6813","nonce":75755945}]},"public_addr":"172.21.15.114:6809/75755945","cluster_addr":"172.21.15.114:6811/75755945","heartbeat_back_addr":"172.21.15.114:6815/75755945","heartbeat_front_addr":"172.21.15.114:6813/75755945","state":["exists","up"]},{"osd":5,"uuid":"51922f99-e8d3-461b-8615-e9bfdcbf5ed3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":1918493790},{"type":"v1","addr":"172.21.15.177:6809","nonce":1918493790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":1918493790},{"type":"v1","addr":"172.21.15.177:6811","nonce":1918493790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":1918493790},{"type":"v1","addr":"172.21.15.177:6815","nonce":1918493790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":1918493790},{"type":"v1","addr":"172.21.15.177:6813","nonce":1918493790}]},"public_addr":"172.21.15.177:6809/1918493790","cluster_addr":"172.21.15.177:6811/1918493790","heartbeat_back_addr":"172.21.15.177:6815/1918493790","heartbeat_front_addr":"172.21.15.177:6813/1918493790","state":["exists","up"]},{"osd":6,"uuid":"06dffb32-05eb-43b3-852e-915c5cc7f25a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2892389563},{"type":"v1","addr":"172.21.15.97:6819","nonce":2892389563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2892389563},{"type":"v1","addr":"172.21.15.97:6821","nonce":2892389563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2892389563},{"type":"v1","addr":"172.21.15.97:6825","nonce":2892389563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2892389563},{"type":"v1","addr":"172.21.15.97:6823","nonce":2892389563}]},"public_addr":"172.21.15.97:6819/2892389563","cluster_addr":"172.21.15.97:6821/2892389563","heartbeat_back_addr":"172.21.15.97:6825/2892389563","heartbeat_front_addr":"172.21.15.97:6823/2892389563","state":["exists","up"]},{"osd":7,"uuid":"e68e3519-9c1b-43c5-8e06-5385248b19df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":2445491153},{"type":"v1","addr":"172.21.15.114:6817","nonce":2445491153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":2445491153},{"type":"v1","addr":"172.21.15.114:6819","nonce":2445491153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2445491153},{"type":"v1","addr":"172.21.15.114:6823","nonce":2445491153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":2445491153},{"type":"v1","addr":"172.21.15.114:6821","nonce":2445491153}]},"public_addr":"172.21.15.114:6817/2445491153","cluster_addr":"172.21.15.114:6819/2445491153","heartbeat_back_addr":"172.21.15.114:6823/2445491153","heartbeat_front_addr":"172.21.15.114:6821/2445491153","state":["exists","up"]},{"osd":8,"uuid":"31ac592e-1349-49ee-9af9-8ad85d682978","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":2051134191},{"type":"v1","addr":"172.21.15.177:6817","nonce":2051134191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":2051134191},{"type":"v1","addr":"172.21.15.177:6819","nonce":2051134191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":2051134191},{"type":"v1","addr":"172.21.15.177:6823","nonce":2051134191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":2051134191},{"type":"v1","addr":"172.21.15.177:6821","nonce":2051134191}]},"public_addr":"172.21.15.177:6817/2051134191","cluster_addr":"172.21.15.177:6819/2051134191","heartbeat_back_addr":"172.21.15.177:6823/2051134191","heartbeat_front_addr":"172.21.15.177:6821/2051134191","state":["exists","up"]},{"osd":9,"uuid":"0171086a-dd41-4eb4-a389-16f0c46a64db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":1578881271},{"type":"v1","addr":"172.21.15.97:6827","nonce":1578881271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1578881271},{"type":"v1","addr":"172.21.15.97:6829","nonce":1578881271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1578881271},{"type":"v1","addr":"172.21.15.97:6833","nonce":1578881271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1578881271},{"type":"v1","addr":"172.21.15.97:6831","nonce":1578881271}]},"public_addr":"172.21.15.97:6827/1578881271","cluster_addr":"172.21.15.97:6829/1578881271","heartbeat_back_addr":"172.21.15.97:6833/1578881271","heartbeat_front_addr":"172.21.15.97:6831/1578881271","state":["exists","up"]},{"osd":10,"uuid":"64754e31-b124-4601-ad98-2f70b1dd5359","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":2893111461},{"type":"v1","addr":"172.21.15.114:6825","nonce":2893111461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":2893111461},{"type":"v1","addr":"172.21.15.114:6827","nonce":2893111461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2893111461},{"type":"v1","addr":"172.21.15.114:6831","nonce":2893111461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":2893111461},{"type":"v1","addr":"172.21.15.114:6829","nonce":2893111461}]},"public_addr":"172.21.15.114:6825/2893111461","cluster_addr":"172.21.15.114:6827/2893111461","heartbeat_back_addr":"172.21.15.114:6831/2893111461","heartbeat_front_addr":"172.21.15.114:6829/2893111461","state":["exists","up"]},{"osd":11,"uuid":"9ffc752c-adf7-4037-89d3-535787d7475d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":85,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":2333030391},{"type":"v1","addr":"172.21.15.177:6825","nonce":2333030391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":2333030391},{"type":"v1","addr":"172.21.15.177:6827","nonce":2333030391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":2333030391},{"type":"v1","addr":"172.21.15.177:6831","nonce":2333030391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":2333030391},{"type":"v1","addr":"172.21.15.177:6829","nonce":2333030391}]},"public_addr":"172.21.15.177:6825/2333030391","cluster_addr":"172.21.15.177:6827/2333030391","heartbeat_back_addr":"172.21.15.177:6831/2333030391","heartbeat_front_addr":"172.21.15.177:6829/2333030391","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:46:45.143519+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:47:18.656947+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:47:51.316345+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:48:24.575512+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:48:59.113555+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:49:32.103865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:50:06.615539+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:50:41.578341+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:51:15.254011+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:51:49.099165+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:52:24.544821+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:52:58.838484+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.28","mappings":[{"from":3,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/359646":"2024-03-28T10:44:39.889550+0000","172.21.15.97:6801/98682155":"2024-03-28T10:44:06.019075+0000","172.21.15.97:6801/1685921853":"2024-03-28T10:44:39.889550+0000","172.21.15.97:6800/98682155":"2024-03-28T10:44:06.019075+0000","172.21.15.97:0/107690680":"2024-03-28T10:44:06.019075+0000","172.21.15.97:6800/1685921853":"2024-03-28T10:44:39.889550+0000","172.21.15.97:0/437043413":"2024-03-28T10:44:39.889550+0000","172.21.15.97:0/2319706992":"2024-03-28T10:44:39.889550+0000","172.21.15.97:0/3911532654":"2024-03-28T10:44:06.019075+0000","172.21.15.97:0/3347321845":"2024-03-28T10:44:06.019075+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-27T10:55:41.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.509+0000 7fd352000640 1 -- 172.21.15.97:0/1228356902 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fd3300a3690 msgr2=0x7fd3300a5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:41.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.509+0000 7fd352000640 1 --2- 172.21.15.97:0/1228356902 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fd3300a3690 0x7fd3300a5b50 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd33c005fd0 tx=0x7fd33c005850 comp rx=0 tx=0).stop 2024-03-27T10:55:41.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.509+0000 7fd352000640 1 -- 172.21.15.97:0/1228356902 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd34c1016b0 msgr2=0x7fd34c1a14e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:41.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.509+0000 7fd352000640 1 --2- 172.21.15.97:0/1228356902 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd34c1016b0 0x7fd34c1a14e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd34402c700 tx=0x7fd3440328a0 comp rx=0 tx=0).stop 2024-03-27T10:55:41.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.509+0000 7fd352000640 1 -- 172.21.15.97:0/1228356902 shutdown_connections 2024-03-27T10:55:41.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.509+0000 7fd352000640 1 --2- 172.21.15.97:0/1228356902 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd34c1016b0 0x7fd34c1a14e0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:41.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.509+0000 7fd352000640 1 --2- 172.21.15.97:0/1228356902 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fd3300a3690 0x7fd3300a5b50 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:41.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.509+0000 7fd352000640 1 --2- 172.21.15.97:0/1228356902 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd34c100d10 0x7fd34c19a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:41.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.509+0000 7fd352000640 1 --2- 172.21.15.97:0/1228356902 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd34c1003b0 0x7fd34c199f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:41.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.509+0000 7fd352000640 1 -- 172.21.15.97:0/1228356902 >> 172.21.15.97:0/1228356902 conn(0x7fd34c0fbfb0 msgr2=0x7fd34c0fd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:55:41.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.509+0000 7fd352000640 1 -- 172.21.15.97:0/1228356902 shutdown_connections 2024-03-27T10:55:41.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.509+0000 7fd352000640 1 -- 172.21.15.97:0/1228356902 wait complete. 2024-03-27T10:55:41.497 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:55:41 smithi097 ceph-mon[31859]: pgmap v412: 97 pgs: 97 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-27T10:55:41.497 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:55:41 smithi097 ceph-mon[31859]: mgrmap e18: x(active, since 11m), standbys: z, y 2024-03-27T10:55:41.497 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:55:41 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/492896990' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-27T10:55:41.499 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-27T10:47:54.052810+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-27T10:54:02.845060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-27T10:54:03.474151+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-27T10:55:41.500 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-27T10:55:41.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:55:41 smithi177 ceph-mon[35294]: pgmap v412: 97 pgs: 97 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-27T10:55:41.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:55:41 smithi177 ceph-mon[35294]: mgrmap e18: x(active, since 11m), standbys: z, y 2024-03-27T10:55:41.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:55:41 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/492896990' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-27T10:55:41.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.736+0000 7fe845000640 1 -- 172.21.15.97:0/121046026 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe840077750 msgr2=0x7fe840077bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:41.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.736+0000 7fe845000640 1 --2- 172.21.15.97:0/121046026 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe840077750 0x7fe840077bb0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fe828009900 tx=0x7fe82802bc30 comp rx=0 tx=0).stop 2024-03-27T10:55:41.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.736+0000 7fe845000640 1 -- 172.21.15.97:0/121046026 shutdown_connections 2024-03-27T10:55:41.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.736+0000 7fe845000640 1 --2- 172.21.15.97:0/121046026 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe8400780f0 0x7fe840078990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:41.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.736+0000 7fe845000640 1 --2- 172.21.15.97:0/121046026 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe840077750 0x7fe840077bb0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:41.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.736+0000 7fe845000640 1 --2- 172.21.15.97:0/121046026 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe840076510 0x7fe840076930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:41.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.736+0000 7fe845000640 1 -- 172.21.15.97:0/121046026 >> 172.21.15.97:0/121046026 conn(0x7fe8400ff300 msgr2=0x7fe840101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:55:41.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.736+0000 7fe845000640 1 -- 172.21.15.97:0/121046026 shutdown_connections 2024-03-27T10:55:41.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.736+0000 7fe845000640 1 -- 172.21.15.97:0/121046026 wait complete. 2024-03-27T10:55:41.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.737+0000 7fe845000640 1 Processor -- start 2024-03-27T10:55:41.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.737+0000 7fe845000640 1 -- start start 2024-03-27T10:55:41.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.737+0000 7fe845000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe840076510 0x7fe84019e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:41.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.737+0000 7fe845000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe840077750 0x7fe84019e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:41.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.738+0000 7fe845000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe8400780f0 0x7fe8401a5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:41.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.738+0000 7fe845000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe840113c90 con 0x7fe840077750 2024-03-27T10:55:41.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.738+0000 7fe845000640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe840113b10 con 0x7fe8400780f0 2024-03-27T10:55:41.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.738+0000 7fe845000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe840113e10 con 0x7fe840076510 2024-03-27T10:55:41.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.738+0000 7fe835400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe840077750 0x7fe84019e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:41.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.738+0000 7fe835400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe840077750 0x7fe84019e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40582/0 (socket says 172.21.15.97:40582) 2024-03-27T10:55:41.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.738+0000 7fe835400640 1 -- 172.21.15.97:0/4075245056 learned_addr learned my addr 172.21.15.97:0/4075245056 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:55:41.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.738+0000 7fe83de00640 1 --2- 172.21.15.97:0/4075245056 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe840076510 0x7fe84019e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:41.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.738+0000 7fe83e800640 1 --2- 172.21.15.97:0/4075245056 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe8400780f0 0x7fe8401a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:41.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.738+0000 7fe835400640 1 -- 172.21.15.97:0/4075245056 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe840076510 msgr2=0x7fe84019e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:41.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.738+0000 7fe835400640 1 --2- 172.21.15.97:0/4075245056 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe840076510 0x7fe84019e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:41.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.738+0000 7fe835400640 1 -- 172.21.15.97:0/4075245056 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe8400780f0 msgr2=0x7fe8401a5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:41.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.738+0000 7fe835400640 1 --2- 172.21.15.97:0/4075245056 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe8400780f0 0x7fe8401a5970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:41.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.738+0000 7fe835400640 1 -- 172.21.15.97:0/4075245056 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8401a5fe0 con 0x7fe840077750 2024-03-27T10:55:41.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.739+0000 7fe835400640 1 --2- 172.21.15.97:0/4075245056 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe840077750 0x7fe84019e900 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fe828006010 tx=0x7fe828004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:55:41.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.739+0000 7fe837400640 1 -- 172.21.15.97:0/4075245056 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe828043020 con 0x7fe840077750 2024-03-27T10:55:41.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.739+0000 7fe837400640 1 -- 172.21.15.97:0/4075245056 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe828004440 con 0x7fe840077750 2024-03-27T10:55:41.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.739+0000 7fe845000640 1 -- 172.21.15.97:0/4075245056 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8401a6270 con 0x7fe840077750 2024-03-27T10:55:41.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.739+0000 7fe837400640 1 -- 172.21.15.97:0/4075245056 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe828003bf0 con 0x7fe840077750 2024-03-27T10:55:41.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.739+0000 7fe845000640 1 -- 172.21.15.97:0/4075245056 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8401a6810 con 0x7fe840077750 2024-03-27T10:55:41.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.740+0000 7fe845000640 1 -- 172.21.15.97:0/4075245056 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe840110700 con 0x7fe840077750 2024-03-27T10:55:41.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.742+0000 7fe837400640 1 -- 172.21.15.97:0/4075245056 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7fe828046020 con 0x7fe840077750 2024-03-27T10:55:41.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.744+0000 7fe837400640 1 --2- 172.21.15.97:0/4075245056 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fe8240a3690 0x7fe8240a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:41.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.744+0000 7fe837400640 1 -- 172.21.15.97:0/4075245056 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fe8280f2180 con 0x7fe840077750 2024-03-27T10:55:41.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.745+0000 7fe83de00640 1 --2- 172.21.15.97:0/4075245056 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fe8240a3690 0x7fe8240a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:41.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.745+0000 7fe83de00640 1 --2- 172.21.15.97:0/4075245056 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fe8240a3690 0x7fe8240a5b50 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe830005fd0 tx=0x7fe830009870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:55:41.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.749+0000 7fe837400640 1 -- 172.21.15.97:0/4075245056 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7fe82804d020 con 0x7fe840077750 2024-03-27T10:55:41.767 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:55:41 smithi114 ceph-mon[35338]: pgmap v412: 97 pgs: 97 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-27T10:55:41.767 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:55:41 smithi114 ceph-mon[35338]: mgrmap e18: x(active, since 11m), standbys: z, y 2024-03-27T10:55:41.767 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:55:41 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/492896990' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-27T10:55:41.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.948+0000 7fe845000640 1 -- 172.21.15.97:0/4075245056 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe840062750 con 0x7fe840077750 2024-03-27T10:55:41.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.949+0000 7fe837400640 1 -- 172.21.15.97:0/4075245056 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) ==== 93+0+10 (secure 0 0 0) 0x7fe8280bed00 con 0x7fe840077750 2024-03-27T10:55:41.921 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-03-27T10:55:41.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.952+0000 7fe845000640 1 -- 172.21.15.97:0/4075245056 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fe8240a3690 msgr2=0x7fe8240a5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:41.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.952+0000 7fe845000640 1 --2- 172.21.15.97:0/4075245056 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fe8240a3690 0x7fe8240a5b50 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe830005fd0 tx=0x7fe830009870 comp rx=0 tx=0).stop 2024-03-27T10:55:41.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.952+0000 7fe845000640 1 -- 172.21.15.97:0/4075245056 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe840077750 msgr2=0x7fe84019e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:41.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.952+0000 7fe845000640 1 --2- 172.21.15.97:0/4075245056 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe840077750 0x7fe84019e900 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fe828006010 tx=0x7fe828004040 comp rx=0 tx=0).stop 2024-03-27T10:55:41.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.952+0000 7fe845000640 1 -- 172.21.15.97:0/4075245056 shutdown_connections 2024-03-27T10:55:41.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.952+0000 7fe845000640 1 --2- 172.21.15.97:0/4075245056 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fe8240a3690 0x7fe8240a5b50 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:41.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.952+0000 7fe845000640 1 --2- 172.21.15.97:0/4075245056 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe8400780f0 0x7fe8401a5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:41.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.952+0000 7fe845000640 1 --2- 172.21.15.97:0/4075245056 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe840077750 0x7fe84019e900 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:41.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.952+0000 7fe845000640 1 --2- 172.21.15.97:0/4075245056 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe840076510 0x7fe84019e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:41.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.952+0000 7fe845000640 1 -- 172.21.15.97:0/4075245056 >> 172.21.15.97:0/4075245056 conn(0x7fe8400ff300 msgr2=0x7fe840102660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:55:41.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.952+0000 7fe845000640 1 -- 172.21.15.97:0/4075245056 shutdown_connections 2024-03-27T10:55:41.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:41.952+0000 7fe845000640 1 -- 172.21.15.97:0/4075245056 wait complete. 2024-03-27T10:55:41.943 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-27T10:55:42.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.186+0000 7fc2c8200640 1 -- 172.21.15.97:0/21552139 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2c00780f0 msgr2=0x7fc2c0078990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:42.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.186+0000 7fc2c8200640 1 --2- 172.21.15.97:0/21552139 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2c00780f0 0x7fc2c0078990 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fc2b8009900 tx=0x7fc2b802bc30 comp rx=0 tx=0).stop 2024-03-27T10:55:42.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.186+0000 7fc2c8200640 1 -- 172.21.15.97:0/21552139 shutdown_connections 2024-03-27T10:55:42.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.186+0000 7fc2c8200640 1 --2- 172.21.15.97:0/21552139 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2c00780f0 0x7fc2c0078990 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:42.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.186+0000 7fc2c8200640 1 --2- 172.21.15.97:0/21552139 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc2c0077750 0x7fc2c0077bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:42.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.186+0000 7fc2c8200640 1 --2- 172.21.15.97:0/21552139 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc2c0076510 0x7fc2c0076930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:42.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.186+0000 7fc2c8200640 1 -- 172.21.15.97:0/21552139 >> 172.21.15.97:0/21552139 conn(0x7fc2c00ff300 msgr2=0x7fc2c0101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:55:42.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.186+0000 7fc2c8200640 1 -- 172.21.15.97:0/21552139 shutdown_connections 2024-03-27T10:55:42.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.186+0000 7fc2c8200640 1 -- 172.21.15.97:0/21552139 wait complete. 2024-03-27T10:55:42.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.187+0000 7fc2c8200640 1 Processor -- start 2024-03-27T10:55:42.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.187+0000 7fc2c8200640 1 -- start start 2024-03-27T10:55:42.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.188+0000 7fc2c8200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc2c0076510 0x7fc2c019e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:42.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.188+0000 7fc2c8200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2c0077750 0x7fc2c019e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:42.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.188+0000 7fc2c8200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc2c00780f0 0x7fc2c01a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:42.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.188+0000 7fc2c8200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c0113d10 con 0x7fc2c0077750 2024-03-27T10:55:42.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.188+0000 7fc2c8200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c0113b90 con 0x7fc2c0076510 2024-03-27T10:55:42.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.188+0000 7fc2c8200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c0113e90 con 0x7fc2c00780f0 2024-03-27T10:55:42.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.188+0000 7fc2c4e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2c0077750 0x7fc2c019e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:42.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.188+0000 7fc2c4e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2c0077750 0x7fc2c019e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40602/0 (socket says 172.21.15.97:40602) 2024-03-27T10:55:42.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.188+0000 7fc2c5800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc2c0076510 0x7fc2c019e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:42.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.188+0000 7fc2c4e00640 1 -- 172.21.15.97:0/3780533157 learned_addr learned my addr 172.21.15.97:0/3780533157 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:55:42.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.188+0000 7fc2c6200640 1 --2- 172.21.15.97:0/3780533157 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc2c00780f0 0x7fc2c01a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:42.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.188+0000 7fc2c4e00640 1 -- 172.21.15.97:0/3780533157 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc2c00780f0 msgr2=0x7fc2c01a5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:42.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.188+0000 7fc2c4e00640 1 --2- 172.21.15.97:0/3780533157 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc2c00780f0 0x7fc2c01a5990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:42.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.188+0000 7fc2c4e00640 1 -- 172.21.15.97:0/3780533157 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc2c0076510 msgr2=0x7fc2c019e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:42.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.188+0000 7fc2c4e00640 1 --2- 172.21.15.97:0/3780533157 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc2c0076510 0x7fc2c019e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:42.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.189+0000 7fc2c4e00640 1 -- 172.21.15.97:0/3780533157 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2c01a6000 con 0x7fc2c0077750 2024-03-27T10:55:42.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.189+0000 7fc2c4e00640 1 --2- 172.21.15.97:0/3780533157 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2c0077750 0x7fc2c019e970 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac00d770 tx=0x7fc2ac00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:55:42.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.189+0000 7fc2be000640 1 -- 172.21.15.97:0/3780533157 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac017070 con 0x7fc2c0077750 2024-03-27T10:55:42.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.189+0000 7fc2c8200640 1 -- 172.21.15.97:0/3780533157 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2c01a62f0 con 0x7fc2c0077750 2024-03-27T10:55:42.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.189+0000 7fc2be000640 1 -- 172.21.15.97:0/3780533157 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc2ac0054d0 con 0x7fc2c0077750 2024-03-27T10:55:42.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.189+0000 7fc2be000640 1 -- 172.21.15.97:0/3780533157 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac004cf0 con 0x7fc2c0077750 2024-03-27T10:55:42.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.189+0000 7fc2c8200640 1 -- 172.21.15.97:0/3780533157 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2c01a6840 con 0x7fc2c0077750 2024-03-27T10:55:42.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.190+0000 7fc2c8200640 1 -- 172.21.15.97:0/3780533157 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2c0110700 con 0x7fc2c0077750 2024-03-27T10:55:42.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.193+0000 7fc2be000640 1 -- 172.21.15.97:0/3780533157 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7fc2ac004e90 con 0x7fc2c0077750 2024-03-27T10:55:42.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.194+0000 7fc2be000640 1 --2- 172.21.15.97:0/3780533157 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fc2a40a3610 0x7fc2a40a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:42.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.194+0000 7fc2be000640 1 -- 172.21.15.97:0/3780533157 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fc2ac0cf7e0 con 0x7fc2c0077750 2024-03-27T10:55:42.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.194+0000 7fc2c5800640 1 --2- 172.21.15.97:0/3780533157 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fc2a40a3610 0x7fc2a40a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:42.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.195+0000 7fc2c5800640 1 --2- 172.21.15.97:0/3780533157 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fc2a40a3610 0x7fc2a40a5ad0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc2b00099a0 tx=0x7fc2b0009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:55:42.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.199+0000 7fc2be000640 1 -- 172.21.15.97:0/3780533157 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7fc2ac09c360 con 0x7fc2c0077750 2024-03-27T10:55:42.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.408+0000 7fc2c8200640 1 -- 172.21.15.97:0/3780533157 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc2c0062750 con 0x7fc2c0077750 2024-03-27T10:55:42.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.408+0000 7fc2be000640 1 -- 172.21.15.97:0/3780533157 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) ==== 104+0+11 (secure 0 0 0) 0x7fc2ac09bc60 con 0x7fc2c0077750 2024-03-27T10:55:42.380 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-03-27T10:55:42.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.411+0000 7fc2c8200640 1 -- 172.21.15.97:0/3780533157 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fc2a40a3610 msgr2=0x7fc2a40a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:42.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.411+0000 7fc2c8200640 1 --2- 172.21.15.97:0/3780533157 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fc2a40a3610 0x7fc2a40a5ad0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc2b00099a0 tx=0x7fc2b0009500 comp rx=0 tx=0).stop 2024-03-27T10:55:42.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.411+0000 7fc2c8200640 1 -- 172.21.15.97:0/3780533157 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2c0077750 msgr2=0x7fc2c019e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:42.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.411+0000 7fc2c8200640 1 --2- 172.21.15.97:0/3780533157 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2c0077750 0x7fc2c019e970 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac00d770 tx=0x7fc2ac00dc30 comp rx=0 tx=0).stop 2024-03-27T10:55:42.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.411+0000 7fc2c8200640 1 -- 172.21.15.97:0/3780533157 shutdown_connections 2024-03-27T10:55:42.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.411+0000 7fc2c8200640 1 --2- 172.21.15.97:0/3780533157 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc2c00780f0 0x7fc2c01a5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:42.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.411+0000 7fc2c8200640 1 --2- 172.21.15.97:0/3780533157 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2c0077750 0x7fc2c019e970 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:42.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.411+0000 7fc2c8200640 1 --2- 172.21.15.97:0/3780533157 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fc2a40a3610 0x7fc2a40a5ad0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:42.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.411+0000 7fc2c8200640 1 --2- 172.21.15.97:0/3780533157 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc2c0076510 0x7fc2c019e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:42.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.411+0000 7fc2c8200640 1 -- 172.21.15.97:0/3780533157 >> 172.21.15.97:0/3780533157 conn(0x7fc2c00ff300 msgr2=0x7fc2c0100e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:55:42.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.411+0000 7fc2c8200640 1 -- 172.21.15.97:0/3780533157 shutdown_connections 2024-03-27T10:55:42.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.411+0000 7fc2c8200640 1 -- 172.21.15.97:0/3780533157 wait complete. 2024-03-27T10:55:42.402 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-27T10:55:42.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.646+0000 7f0f1cc00640 1 -- 172.21.15.97:0/958361989 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f18104f80 msgr2=0x7f0f18105400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:42.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.646+0000 7f0f1cc00640 1 --2- 172.21.15.97:0/958361989 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f18104f80 0x7f0f18105400 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f0f00009900 tx=0x7f0f0002bc30 comp rx=0 tx=0).stop 2024-03-27T10:55:42.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.647+0000 7f0f1cc00640 1 -- 172.21.15.97:0/958361989 shutdown_connections 2024-03-27T10:55:42.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.647+0000 7f0f1cc00640 1 --2- 172.21.15.97:0/958361989 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0f18105940 0x7f0f1810c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:42.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.647+0000 7f0f1cc00640 1 --2- 172.21.15.97:0/958361989 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f18104f80 0x7f0f18105400 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:42.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.647+0000 7f0f1cc00640 1 --2- 172.21.15.97:0/958361989 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f18103d90 0x7f0f181041b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:42.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.647+0000 7f0f1cc00640 1 -- 172.21.15.97:0/958361989 >> 172.21.15.97:0/958361989 conn(0x7f0f180ff300 msgr2=0x7f0f18101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:55:42.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.647+0000 7f0f1cc00640 1 -- 172.21.15.97:0/958361989 shutdown_connections 2024-03-27T10:55:42.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.647+0000 7f0f1cc00640 1 -- 172.21.15.97:0/958361989 wait complete. 2024-03-27T10:55:42.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.648+0000 7f0f1cc00640 1 Processor -- start 2024-03-27T10:55:42.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.648+0000 7f0f1cc00640 1 -- start start 2024-03-27T10:55:42.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.648+0000 7f0f1cc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f18103d90 0x7f0f18199fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:42.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.648+0000 7f0f1cc00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0f18104f80 0x7f0f1819a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:42.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.648+0000 7f0f1cc00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f18105940 0x7f0f181a1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:42.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.648+0000 7f0f1cc00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0f1810f910 con 0x7f0f18103d90 2024-03-27T10:55:42.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.648+0000 7f0f1cc00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f0f1810f790 con 0x7f0f18105940 2024-03-27T10:55:42.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.648+0000 7f0f1cc00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f0f1810fa90 con 0x7f0f18104f80 2024-03-27T10:55:42.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.648+0000 7f0f15e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f18103d90 0x7f0f18199fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:42.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.648+0000 7f0f15e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f18103d90 0x7f0f18199fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40626/0 (socket says 172.21.15.97:40626) 2024-03-27T10:55:42.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.649+0000 7f0f15e00640 1 -- 172.21.15.97:0/1550939693 learned_addr learned my addr 172.21.15.97:0/1550939693 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:55:42.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.649+0000 7f0f15e00640 1 -- 172.21.15.97:0/1550939693 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0f18104f80 msgr2=0x7f0f1819a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:42.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.649+0000 7f0f15e00640 1 --2- 172.21.15.97:0/1550939693 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0f18104f80 0x7f0f1819a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:42.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.649+0000 7f0f15e00640 1 -- 172.21.15.97:0/1550939693 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f18105940 msgr2=0x7f0f181a1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:42.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.649+0000 7f0f16800640 1 --2- 172.21.15.97:0/1550939693 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f18105940 0x7f0f181a1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:42.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.649+0000 7f0f15e00640 1 --2- 172.21.15.97:0/1550939693 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f18105940 0x7f0f181a1590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:42.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.649+0000 7f0f15e00640 1 -- 172.21.15.97:0/1550939693 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f181a1c00 con 0x7f0f18103d90 2024-03-27T10:55:42.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.649+0000 7f0f15e00640 1 --2- 172.21.15.97:0/1550939693 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f18103d90 0x7f0f18199fe0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f0f0800d770 tx=0x7f0f0800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:55:42.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.649+0000 7f0f0f400640 1 -- 172.21.15.97:0/1550939693 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f08017070 con 0x7f0f18103d90 2024-03-27T10:55:42.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.649+0000 7f0f0f400640 1 -- 172.21.15.97:0/1550939693 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0f08009c30 con 0x7f0f18103d90 2024-03-27T10:55:42.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.649+0000 7f0f0f400640 1 -- 172.21.15.97:0/1550939693 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f08003d00 con 0x7f0f18103d90 2024-03-27T10:55:42.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.649+0000 7f0f1cc00640 1 -- 172.21.15.97:0/1550939693 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f181a1ef0 con 0x7f0f18103d90 2024-03-27T10:55:42.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.650+0000 7f0f1cc00640 1 -- 172.21.15.97:0/1550939693 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f181a2360 con 0x7f0f18103d90 2024-03-27T10:55:42.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.651+0000 7f0f1cc00640 1 -- 172.21.15.97:0/1550939693 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f1810b150 con 0x7f0f18103d90 2024-03-27T10:55:42.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.652+0000 7f0f0f400640 1 -- 172.21.15.97:0/1550939693 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f0f08003ea0 con 0x7f0f18103d90 2024-03-27T10:55:42.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.654+0000 7f0f0f400640 1 --2- 172.21.15.97:0/1550939693 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f0efc0a3610 0x7f0efc0a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:42.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.654+0000 7f0f0f400640 1 -- 172.21.15.97:0/1550939693 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f0f080cfb50 con 0x7f0f18103d90 2024-03-27T10:55:42.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.654+0000 7f0f0d400640 1 --2- 172.21.15.97:0/1550939693 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f0efc0a3610 0x7f0efc0a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:42.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.655+0000 7f0f0d400640 1 --2- 172.21.15.97:0/1550939693 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f0efc0a3610 0x7f0efc0a5ad0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0f00005c20 tx=0x7f0f00000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:55:42.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.658+0000 7f0f0f400640 1 -- 172.21.15.97:0/1550939693 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f0f0809c6d0 con 0x7f0f18103d90 2024-03-27T10:55:42.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:55:42 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/1228356902' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T10:55:42.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:55:42 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/4075245056' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-27T10:55:42.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:55:42 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/1228356902' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T10:55:42.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:55:42 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/4075245056' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-27T10:55:42.767 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:55:42 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/1228356902' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T10:55:42.767 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:55:42 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/4075245056' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-27T10:55:42.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.859+0000 7f0f1cc00640 1 -- 172.21.15.97:0/1550939693 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f0f18062830 con 0x7f0f18103d90 2024-03-27T10:55:42.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.860+0000 7f0f0f400640 1 -- 172.21.15.97:0/1550939693 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) ==== 100+0+11 (secure 0 0 0) 0x7f0f08009de0 con 0x7f0f18103d90 2024-03-27T10:55:42.831 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-03-27T10:55:42.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.862+0000 7f0f1cc00640 1 -- 172.21.15.97:0/1550939693 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f0efc0a3610 msgr2=0x7f0efc0a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:42.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.862+0000 7f0f1cc00640 1 --2- 172.21.15.97:0/1550939693 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f0efc0a3610 0x7f0efc0a5ad0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0f00005c20 tx=0x7f0f00000bc0 comp rx=0 tx=0).stop 2024-03-27T10:55:42.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.862+0000 7f0f1cc00640 1 -- 172.21.15.97:0/1550939693 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f18103d90 msgr2=0x7f0f18199fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:42.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.862+0000 7f0f1cc00640 1 --2- 172.21.15.97:0/1550939693 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f18103d90 0x7f0f18199fe0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f0f0800d770 tx=0x7f0f0800dc30 comp rx=0 tx=0).stop 2024-03-27T10:55:42.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.862+0000 7f0f1cc00640 1 -- 172.21.15.97:0/1550939693 shutdown_connections 2024-03-27T10:55:42.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.862+0000 7f0f1cc00640 1 --2- 172.21.15.97:0/1550939693 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f0efc0a3610 0x7f0efc0a5ad0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:42.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.862+0000 7f0f1cc00640 1 --2- 172.21.15.97:0/1550939693 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f18105940 0x7f0f181a1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:42.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.862+0000 7f0f1cc00640 1 --2- 172.21.15.97:0/1550939693 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0f18104f80 0x7f0f1819a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:42.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.862+0000 7f0f1cc00640 1 --2- 172.21.15.97:0/1550939693 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f18103d90 0x7f0f18199fe0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:42.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.862+0000 7f0f1cc00640 1 -- 172.21.15.97:0/1550939693 >> 172.21.15.97:0/1550939693 conn(0x7f0f180ff300 msgr2=0x7f0f18108320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:55:42.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.863+0000 7f0f1cc00640 1 -- 172.21.15.97:0/1550939693 shutdown_connections 2024-03-27T10:55:42.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:42.863+0000 7f0f1cc00640 1 -- 172.21.15.97:0/1550939693 wait complete. 2024-03-27T10:55:42.855 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-27T10:55:43.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.096+0000 7f8d65e00640 1 -- 172.21.15.97:0/16901291 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d60103710 msgr2=0x7f8d60109fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:43.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.096+0000 7f8d65e00640 1 --2- 172.21.15.97:0/16901291 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d60103710 0x7f8d60109fa0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f8d58009ca0 tx=0x7f8d5802c0f0 comp rx=0 tx=0).stop 2024-03-27T10:55:43.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.096+0000 7f8d65e00640 1 -- 172.21.15.97:0/16901291 shutdown_connections 2024-03-27T10:55:43.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.096+0000 7f8d65e00640 1 --2- 172.21.15.97:0/16901291 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d60103710 0x7f8d60109fa0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:43.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.096+0000 7f8d65e00640 1 --2- 172.21.15.97:0/16901291 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8d60102d50 0x7f8d601031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:43.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.096+0000 7f8d65e00640 1 --2- 172.21.15.97:0/16901291 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8d60101b60 0x7f8d60101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:43.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.096+0000 7f8d65e00640 1 -- 172.21.15.97:0/16901291 >> 172.21.15.97:0/16901291 conn(0x7f8d600fd0f0 msgr2=0x7f8d600ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:55:43.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.096+0000 7f8d65e00640 1 -- 172.21.15.97:0/16901291 shutdown_connections 2024-03-27T10:55:43.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.096+0000 7f8d65e00640 1 -- 172.21.15.97:0/16901291 wait complete. 2024-03-27T10:55:43.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.097+0000 7f8d65e00640 1 Processor -- start 2024-03-27T10:55:43.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.097+0000 7f8d65e00640 1 -- start start 2024-03-27T10:55:43.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.097+0000 7f8d65e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8d60101b60 0x7f8d60075900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:43.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.098+0000 7f8d65e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d60102d50 0x7f8d60073f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:43.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.098+0000 7f8d65e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8d60103710 0x7f8d60074480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:43.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.098+0000 7f8d65e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8d6010d690 con 0x7f8d60102d50 2024-03-27T10:55:43.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.098+0000 7f8d65e00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f8d6010d510 con 0x7f8d60103710 2024-03-27T10:55:43.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.098+0000 7f8d65e00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f8d6010d810 con 0x7f8d60101b60 2024-03-27T10:55:43.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.098+0000 7f8d56000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d60102d50 0x7f8d60073f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:43.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.098+0000 7f8d56000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d60102d50 0x7f8d60073f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40648/0 (socket says 172.21.15.97:40648) 2024-03-27T10:55:43.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.098+0000 7f8d56000640 1 -- 172.21.15.97:0/105730681 learned_addr learned my addr 172.21.15.97:0/105730681 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:55:43.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.098+0000 7f8d5f400640 1 --2- 172.21.15.97:0/105730681 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8d60103710 0x7f8d60074480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:43.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.098+0000 7f8d5ea00640 1 --2- 172.21.15.97:0/105730681 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8d60101b60 0x7f8d60075900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:43.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.098+0000 7f8d56000640 1 -- 172.21.15.97:0/105730681 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8d60101b60 msgr2=0x7f8d60075900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:43.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.098+0000 7f8d56000640 1 --2- 172.21.15.97:0/105730681 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8d60101b60 0x7f8d60075900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:43.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.098+0000 7f8d56000640 1 -- 172.21.15.97:0/105730681 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8d60103710 msgr2=0x7f8d60074480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:43.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.098+0000 7f8d56000640 1 --2- 172.21.15.97:0/105730681 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8d60103710 0x7f8d60074480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:43.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.098+0000 7f8d56000640 1 -- 172.21.15.97:0/105730681 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d60074d10 con 0x7f8d60102d50 2024-03-27T10:55:43.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.099+0000 7f8d56000640 1 --2- 172.21.15.97:0/105730681 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d60102d50 0x7f8d60073f40 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f8d4800d770 tx=0x7f8d4800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:55:43.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.099+0000 7f8d64a00640 1 -- 172.21.15.97:0/105730681 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d48017070 con 0x7f8d60102d50 2024-03-27T10:55:43.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.099+0000 7f8d64a00640 1 -- 172.21.15.97:0/105730681 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8d480054d0 con 0x7f8d60102d50 2024-03-27T10:55:43.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.099+0000 7f8d64a00640 1 -- 172.21.15.97:0/105730681 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d48004cf0 con 0x7f8d60102d50 2024-03-27T10:55:43.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.099+0000 7f8d65e00640 1 -- 172.21.15.97:0/105730681 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d601a1e50 con 0x7f8d60102d50 2024-03-27T10:55:43.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.099+0000 7f8d65e00640 1 -- 172.21.15.97:0/105730681 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d601a2260 con 0x7f8d60102d50 2024-03-27T10:55:43.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.100+0000 7f8d65e00640 1 -- 172.21.15.97:0/105730681 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d60108e80 con 0x7f8d60102d50 2024-03-27T10:55:43.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.102+0000 7f8d64a00640 1 -- 172.21.15.97:0/105730681 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f8d48004e90 con 0x7f8d60102d50 2024-03-27T10:55:43.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.103+0000 7f8d64a00640 1 --2- 172.21.15.97:0/105730681 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f8d440a3610 0x7f8d440a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:43.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.103+0000 7f8d64a00640 1 -- 172.21.15.97:0/105730681 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f8d480cf7f0 con 0x7f8d60102d50 2024-03-27T10:55:43.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.104+0000 7f8d5ea00640 1 --2- 172.21.15.97:0/105730681 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f8d440a3610 0x7f8d440a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:43.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.104+0000 7f8d5ea00640 1 --2- 172.21.15.97:0/105730681 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f8d440a3610 0x7f8d440a5ad0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8d50005fd0 tx=0x7f8d50005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:55:43.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.107+0000 7f8d64a00640 1 -- 172.21.15.97:0/105730681 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f8d4809c370 con 0x7f8d60102d50 2024-03-27T10:55:43.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.338+0000 7f8d65e00640 1 -- 172.21.15.97:0/105730681 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8d60062890 con 0x7f8d60102d50 2024-03-27T10:55:43.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.339+0000 7f8d64a00640 1 -- 172.21.15.97:0/105730681 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11944 (secure 0 0 0) 0x7f8d480bc0c0 con 0x7f8d60102d50 2024-03-27T10:55:43.312 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T10:55:43.312 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1009423904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1009423904},{"type":"v1","addr":"172.21.15.97:6835","nonce":1009423904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3251173433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3251173433},{"type":"v1","addr":"172.21.15.97:6837","nonce":3251173433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-27T10:54:04.510178+0000","modified":"2024-03-27T10:54:12.180047+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34508,"mds_1":34478,"mds_2":34445,"mds_3":24454,"mds_4":24400},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14646":{"gid":14646,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6839/1057596821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1057596821},{"type":"v1","addr":"172.21.15.97:6839","nonce":1057596821}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14658":{"gid":14658,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6841/1672464856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6840","nonce":1672464856},{"type":"v1","addr":"172.21.15.97:6841","nonce":1672464856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.177:6833/3787981632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3787981632},{"type":"v1","addr":"172.21.15.177:6833","nonce":3787981632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.177:6835/3349112132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":3349112132},{"type":"v1","addr":"172.21.15.177:6835","nonce":3349112132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6837/3244432640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":3244432640},{"type":"v1","addr":"172.21.15.177:6837","nonce":3244432640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6839/3340454605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":3340454605},{"type":"v1","addr":"172.21.15.177:6839","nonce":3340454605}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34415":{"gid":34415,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6833/4052517902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":4052517902},{"type":"v1","addr":"172.21.15.114:6833","nonce":4052517902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34445":{"gid":34445,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.114:6835/3581918222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3581918222},{"type":"v1","addr":"172.21.15.114:6835","nonce":3581918222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34478":{"gid":34478,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.114:6837/616775588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":616775588},{"type":"v1","addr":"172.21.15.114:6837","nonce":616775588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34508":{"gid":34508,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.114:6839/3937193784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":3937193784},{"type":"v1","addr":"172.21.15.114:6839","nonce":3937193784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34508,"qdb_cluster":[34445,34508,24400,24454,34478]},"id":1}]} 2024-03-27T10:55:43.313 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 27 2024-03-27T10:55:43.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.342+0000 7f8d65e00640 1 -- 172.21.15.97:0/105730681 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f8d440a3610 msgr2=0x7f8d440a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:43.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.342+0000 7f8d65e00640 1 --2- 172.21.15.97:0/105730681 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f8d440a3610 0x7f8d440a5ad0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8d50005fd0 tx=0x7f8d50005850 comp rx=0 tx=0).stop 2024-03-27T10:55:43.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.342+0000 7f8d65e00640 1 -- 172.21.15.97:0/105730681 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d60102d50 msgr2=0x7f8d60073f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:43.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.342+0000 7f8d65e00640 1 --2- 172.21.15.97:0/105730681 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d60102d50 0x7f8d60073f40 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f8d4800d770 tx=0x7f8d4800dc30 comp rx=0 tx=0).stop 2024-03-27T10:55:43.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.342+0000 7f8d65e00640 1 -- 172.21.15.97:0/105730681 shutdown_connections 2024-03-27T10:55:43.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.342+0000 7f8d65e00640 1 --2- 172.21.15.97:0/105730681 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f8d440a3610 0x7f8d440a5ad0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:43.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.342+0000 7f8d65e00640 1 --2- 172.21.15.97:0/105730681 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8d60103710 0x7f8d60074480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:43.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.342+0000 7f8d65e00640 1 --2- 172.21.15.97:0/105730681 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d60102d50 0x7f8d60073f40 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:43.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.342+0000 7f8d65e00640 1 --2- 172.21.15.97:0/105730681 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8d60101b60 0x7f8d60075900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:43.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.342+0000 7f8d65e00640 1 -- 172.21.15.97:0/105730681 >> 172.21.15.97:0/105730681 conn(0x7f8d600fd0f0 msgr2=0x7f8d60106050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:55:43.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.342+0000 7f8d65e00640 1 -- 172.21.15.97:0/105730681 shutdown_connections 2024-03-27T10:55:43.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.342+0000 7f8d65e00640 1 -- 172.21.15.97:0/105730681 wait complete. 2024-03-27T10:55:43.335 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-27T10:55:43.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.577+0000 7f920c400640 1 -- 172.21.15.97:0/1888119362 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92041003b0 msgr2=0x7f92041027e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:43.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.577+0000 7f920c400640 1 --2- 172.21.15.97:0/1888119362 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92041003b0 0x7f92041027e0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f91f8009900 tx=0x7f91f802bc30 comp rx=0 tx=0).stop 2024-03-27T10:55:43.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.577+0000 7f920c400640 1 -- 172.21.15.97:0/1888119362 shutdown_connections 2024-03-27T10:55:43.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.577+0000 7f920c400640 1 --2- 172.21.15.97:0/1888119362 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f92041056d0 0x7f9204105b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:43.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.577+0000 7f920c400640 1 --2- 172.21.15.97:0/1888119362 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9204102d20 0x7f9204105190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:43.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.577+0000 7f920c400640 1 --2- 172.21.15.97:0/1888119362 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92041003b0 0x7f92041027e0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:43.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.578+0000 7f920c400640 1 -- 172.21.15.97:0/1888119362 >> 172.21.15.97:0/1888119362 conn(0x7f92040fbfb0 msgr2=0x7f92040fe3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:55:43.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.578+0000 7f920c400640 1 -- 172.21.15.97:0/1888119362 shutdown_connections 2024-03-27T10:55:43.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.578+0000 7f920c400640 1 -- 172.21.15.97:0/1888119362 wait complete. 2024-03-27T10:55:43.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.578+0000 7f920c400640 1 Processor -- start 2024-03-27T10:55:43.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.579+0000 7f920c400640 1 -- start start 2024-03-27T10:55:43.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.579+0000 7f920c400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f92041003b0 0x7f920419e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:43.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.579+0000 7f920c400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9204102d20 0x7f920419e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:43.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.579+0000 7f920c400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f92041056d0 0x7f92041a58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:43.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.579+0000 7f920c400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9204113ce0 con 0x7f9204102d20 2024-03-27T10:55:43.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.579+0000 7f920c400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9204113b60 con 0x7f92041056d0 2024-03-27T10:55:43.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.579+0000 7f920c400640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f9204113e60 con 0x7f92041003b0 2024-03-27T10:55:43.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.579+0000 7f9209000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9204102d20 0x7f920419e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:43.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.579+0000 7f9209000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9204102d20 0x7f920419e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40662/0 (socket says 172.21.15.97:40662) 2024-03-27T10:55:43.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.579+0000 7f9209000640 1 -- 172.21.15.97:0/1288220922 learned_addr learned my addr 172.21.15.97:0/1288220922 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:55:43.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.579+0000 7f9209a00640 1 --2- 172.21.15.97:0/1288220922 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f92041003b0 0x7f920419e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:43.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.580+0000 7f920a400640 1 --2- 172.21.15.97:0/1288220922 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f92041056d0 0x7f92041a58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:43.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.580+0000 7f9209000640 1 -- 172.21.15.97:0/1288220922 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f92041003b0 msgr2=0x7f920419e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:43.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.580+0000 7f9209000640 1 --2- 172.21.15.97:0/1288220922 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f92041003b0 0x7f920419e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:43.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.580+0000 7f9209000640 1 -- 172.21.15.97:0/1288220922 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f92041056d0 msgr2=0x7f92041a58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:43.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.580+0000 7f9209000640 1 --2- 172.21.15.97:0/1288220922 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f92041056d0 0x7f92041a58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:43.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.580+0000 7f9209000640 1 -- 172.21.15.97:0/1288220922 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92041a5f50 con 0x7f9204102d20 2024-03-27T10:55:43.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.580+0000 7f9209000640 1 --2- 172.21.15.97:0/1288220922 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9204102d20 0x7f920419e870 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f91f000d950 tx=0x7f91f000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:55:43.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.580+0000 7f9202000640 1 -- 172.21.15.97:0/1288220922 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f0017030 con 0x7f9204102d20 2024-03-27T10:55:43.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.580+0000 7f9202000640 1 -- 172.21.15.97:0/1288220922 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f91f0004020 con 0x7f9204102d20 2024-03-27T10:55:43.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.580+0000 7f920c400640 1 -- 172.21.15.97:0/1288220922 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92041a6240 con 0x7f9204102d20 2024-03-27T10:55:43.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.580+0000 7f9202000640 1 -- 172.21.15.97:0/1288220922 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f00047a0 con 0x7f9204102d20 2024-03-27T10:55:43.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.580+0000 7f920c400640 1 -- 172.21.15.97:0/1288220922 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92041a6790 con 0x7f9204102d20 2024-03-27T10:55:43.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.581+0000 7f920c400640 1 -- 172.21.15.97:0/1288220922 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92040fdaa0 con 0x7f9204102d20 2024-03-27T10:55:43.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.584+0000 7f9202000640 1 -- 172.21.15.97:0/1288220922 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f91f00041d0 con 0x7f9204102d20 2024-03-27T10:55:43.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.585+0000 7f9202000640 1 --2- 172.21.15.97:0/1288220922 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f91ec0a3610 0x7f91ec0a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:43.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.585+0000 7f9202000640 1 -- 172.21.15.97:0/1288220922 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f91f00cf930 con 0x7f9204102d20 2024-03-27T10:55:43.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.586+0000 7f9209a00640 1 --2- 172.21.15.97:0/1288220922 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f91ec0a3610 0x7f91ec0a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:43.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.586+0000 7f9209a00640 1 --2- 172.21.15.97:0/1288220922 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f91ec0a3610 0x7f91ec0a5ad0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f91f8006010 tx=0x7f91f8000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:55:43.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.590+0000 7f9202000640 1 -- 172.21.15.97:0/1288220922 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f91f009c4b0 con 0x7f9204102d20 2024-03-27T10:55:43.641 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:55:43 smithi097 ceph-mon[31859]: pgmap v413: 97 pgs: 97 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T10:55:43.641 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:55:43 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/3780533157' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-27T10:55:43.641 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:55:43 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/1550939693' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-27T10:55:43.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:55:43 smithi177 ceph-mon[35294]: pgmap v413: 97 pgs: 97 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T10:55:43.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:55:43 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/3780533157' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-27T10:55:43.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:55:43 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/1550939693' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-27T10:55:43.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.792+0000 7f920c400640 1 -- 172.21.15.97:0/1288220922 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9204062750 con 0x7f9204102d20 2024-03-27T10:55:43.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.794+0000 7f9202000640 1 -- 172.21.15.97:0/1288220922 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+22644 (secure 0 0 0) 0x7f91f009bdb0 con 0x7f9204102d20 2024-03-27T10:55:43.766 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T10:55:43.767 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:55:43 smithi114 ceph-mon[35338]: pgmap v413: 97 pgs: 97 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T10:55:43.767 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:55:43 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/3780533157' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-27T10:55:43.767 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:55:43 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/1550939693' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-27T10:55:43.772 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":92,"fsid":"adca88c4-ec26-11ee-95d0-87774f69a715","created":"2024-03-27T10:43:24.680694+0000","modified":"2024-03-27T10:54:41.310812+0000","last_up_change":"2024-03-27T10:53:00.552513+0000","last_in_change":"2024-03-27T10:52:36.302986+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-27T10:47:54.052810+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-27T10:54:02.845060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-27T10:54:03.474151+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b634e5f1-6118-4e2f-95e9-830db390f8d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1746812080},{"type":"v1","addr":"172.21.15.97:6803","nonce":1746812080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1746812080},{"type":"v1","addr":"172.21.15.97:6805","nonce":1746812080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":1746812080},{"type":"v1","addr":"172.21.15.97:6809","nonce":1746812080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1746812080},{"type":"v1","addr":"172.21.15.97:6807","nonce":1746812080}]},"public_addr":"172.21.15.97:6803/1746812080","cluster_addr":"172.21.15.97:6805/1746812080","heartbeat_back_addr":"172.21.15.97:6809/1746812080","heartbeat_front_addr":"172.21.15.97:6807/1746812080","state":["exists","up"]},{"osd":1,"uuid":"441e2770-3284-4388-b0de-c56eb8a373d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":3858821264},{"type":"v1","addr":"172.21.15.114:6801","nonce":3858821264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":3858821264},{"type":"v1","addr":"172.21.15.114:6803","nonce":3858821264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":3858821264},{"type":"v1","addr":"172.21.15.114:6807","nonce":3858821264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":3858821264},{"type":"v1","addr":"172.21.15.114:6805","nonce":3858821264}]},"public_addr":"172.21.15.114:6801/3858821264","cluster_addr":"172.21.15.114:6803/3858821264","heartbeat_back_addr":"172.21.15.114:6807/3858821264","heartbeat_front_addr":"172.21.15.114:6805/3858821264","state":["exists","up"]},{"osd":2,"uuid":"00108864-1aa6-48d6-8398-e8eb79fda123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":124240658},{"type":"v1","addr":"172.21.15.177:6801","nonce":124240658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":124240658},{"type":"v1","addr":"172.21.15.177:6803","nonce":124240658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":124240658},{"type":"v1","addr":"172.21.15.177:6807","nonce":124240658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":124240658},{"type":"v1","addr":"172.21.15.177:6805","nonce":124240658}]},"public_addr":"172.21.15.177:6801/124240658","cluster_addr":"172.21.15.177:6803/124240658","heartbeat_back_addr":"172.21.15.177:6807/124240658","heartbeat_front_addr":"172.21.15.177:6805/124240658","state":["exists","up"]},{"osd":3,"uuid":"1860b690-e2dc-4977-88d7-d1aacf7022ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2478179311},{"type":"v1","addr":"172.21.15.97:6811","nonce":2478179311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2478179311},{"type":"v1","addr":"172.21.15.97:6813","nonce":2478179311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":2478179311},{"type":"v1","addr":"172.21.15.97:6817","nonce":2478179311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2478179311},{"type":"v1","addr":"172.21.15.97:6815","nonce":2478179311}]},"public_addr":"172.21.15.97:6811/2478179311","cluster_addr":"172.21.15.97:6813/2478179311","heartbeat_back_addr":"172.21.15.97:6817/2478179311","heartbeat_front_addr":"172.21.15.97:6815/2478179311","state":["exists","up"]},{"osd":4,"uuid":"9245a3a4-7d49-4fdb-9d55-112e43177050","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":75755945},{"type":"v1","addr":"172.21.15.114:6809","nonce":75755945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":75755945},{"type":"v1","addr":"172.21.15.114:6811","nonce":75755945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":75755945},{"type":"v1","addr":"172.21.15.114:6815","nonce":75755945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":75755945},{"type":"v1","addr":"172.21.15.114:6813","nonce":75755945}]},"public_addr":"172.21.15.114:6809/75755945","cluster_addr":"172.21.15.114:6811/75755945","heartbeat_back_addr":"172.21.15.114:6815/75755945","heartbeat_front_addr":"172.21.15.114:6813/75755945","state":["exists","up"]},{"osd":5,"uuid":"51922f99-e8d3-461b-8615-e9bfdcbf5ed3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":1918493790},{"type":"v1","addr":"172.21.15.177:6809","nonce":1918493790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":1918493790},{"type":"v1","addr":"172.21.15.177:6811","nonce":1918493790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":1918493790},{"type":"v1","addr":"172.21.15.177:6815","nonce":1918493790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":1918493790},{"type":"v1","addr":"172.21.15.177:6813","nonce":1918493790}]},"public_addr":"172.21.15.177:6809/1918493790","cluster_addr":"172.21.15.177:6811/1918493790","heartbeat_back_addr":"172.21.15.177:6815/1918493790","heartbeat_front_addr":"172.21.15.177:6813/1918493790","state":["exists","up"]},{"osd":6,"uuid":"06dffb32-05eb-43b3-852e-915c5cc7f25a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2892389563},{"type":"v1","addr":"172.21.15.97:6819","nonce":2892389563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2892389563},{"type":"v1","addr":"172.21.15.97:6821","nonce":2892389563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2892389563},{"type":"v1","addr":"172.21.15.97:6825","nonce":2892389563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2892389563},{"type":"v1","addr":"172.21.15.97:6823","nonce":2892389563}]},"public_addr":"172.21.15.97:6819/2892389563","cluster_addr":"172.21.15.97:6821/2892389563","heartbeat_back_addr":"172.21.15.97:6825/2892389563","heartbeat_front_addr":"172.21.15.97:6823/2892389563","state":["exists","up"]},{"osd":7,"uuid":"e68e3519-9c1b-43c5-8e06-5385248b19df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":2445491153},{"type":"v1","addr":"172.21.15.114:6817","nonce":2445491153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":2445491153},{"type":"v1","addr":"172.21.15.114:6819","nonce":2445491153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2445491153},{"type":"v1","addr":"172.21.15.114:6823","nonce":2445491153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":2445491153},{"type":"v1","addr":"172.21.15.114:6821","nonce":2445491153}]},"public_addr":"172.21.15.114:6817/2445491153","cluster_addr":"172.21.15.114:6819/2445491153","heartbeat_back_addr":"172.21.15.114:6823/2445491153","heartbeat_front_addr":"172.21.15.114:6821/2445491153","state":["exists","up"]},{"osd":8,"uuid":"31ac592e-1349-49ee-9af9-8ad85d682978","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":2051134191},{"type":"v1","addr":"172.21.15.177:6817","nonce":2051134191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":2051134191},{"type":"v1","addr":"172.21.15.177:6819","nonce":2051134191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":2051134191},{"type":"v1","addr":"172.21.15.177:6823","nonce":2051134191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":2051134191},{"type":"v1","addr":"172.21.15.177:6821","nonce":2051134191}]},"public_addr":"172.21.15.177:6817/2051134191","cluster_addr":"172.21.15.177:6819/2051134191","heartbeat_back_addr":"172.21.15.177:6823/2051134191","heartbeat_front_addr":"172.21.15.177:6821/2051134191","state":["exists","up"]},{"osd":9,"uuid":"0171086a-dd41-4eb4-a389-16f0c46a64db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":1578881271},{"type":"v1","addr":"172.21.15.97:6827","nonce":1578881271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1578881271},{"type":"v1","addr":"172.21.15.97:6829","nonce":1578881271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1578881271},{"type":"v1","addr":"172.21.15.97:6833","nonce":1578881271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1578881271},{"type":"v1","addr":"172.21.15.97:6831","nonce":1578881271}]},"public_addr":"172.21.15.97:6827/1578881271","cluster_addr":"172.21.15.97:6829/1578881271","heartbeat_back_addr":"172.21.15.97:6833/1578881271","heartbeat_front_addr":"172.21.15.97:6831/1578881271","state":["exists","up"]},{"osd":10,"uuid":"64754e31-b124-4601-ad98-2f70b1dd5359","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":2893111461},{"type":"v1","addr":"172.21.15.114:6825","nonce":2893111461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":2893111461},{"type":"v1","addr":"172.21.15.114:6827","nonce":2893111461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2893111461},{"type":"v1","addr":"172.21.15.114:6831","nonce":2893111461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":2893111461},{"type":"v1","addr":"172.21.15.114:6829","nonce":2893111461}]},"public_addr":"172.21.15.114:6825/2893111461","cluster_addr":"172.21.15.114:6827/2893111461","heartbeat_back_addr":"172.21.15.114:6831/2893111461","heartbeat_front_addr":"172.21.15.114:6829/2893111461","state":["exists","up"]},{"osd":11,"uuid":"9ffc752c-adf7-4037-89d3-535787d7475d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":85,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":2333030391},{"type":"v1","addr":"172.21.15.177:6825","nonce":2333030391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":2333030391},{"type":"v1","addr":"172.21.15.177:6827","nonce":2333030391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":2333030391},{"type":"v1","addr":"172.21.15.177:6831","nonce":2333030391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":2333030391},{"type":"v1","addr":"172.21.15.177:6829","nonce":2333030391}]},"public_addr":"172.21.15.177:6825/2333030391","cluster_addr":"172.21.15.177:6827/2333030391","heartbeat_back_addr":"172.21.15.177:6831/2333030391","heartbeat_front_addr":"172.21.15.177:6829/2333030391","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:46:45.143519+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:47:18.656947+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:47:51.316345+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:48:24.575512+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:48:59.113555+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:49:32.103865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:50:06.615539+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:50:41.578341+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:51:15.254011+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:51:49.099165+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:52:24.544821+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:52:58.838484+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.28","mappings":[{"from":3,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/359646":"2024-03-28T10:44:39.889550+0000","172.21.15.97:6801/98682155":"2024-03-28T10:44:06.019075+0000","172.21.15.97:6801/1685921853":"2024-03-28T10:44:39.889550+0000","172.21.15.97:6800/98682155":"2024-03-28T10:44:06.019075+0000","172.21.15.97:0/107690680":"2024-03-28T10:44:06.019075+0000","172.21.15.97:6800/1685921853":"2024-03-28T10:44:39.889550+0000","172.21.15.97:0/437043413":"2024-03-28T10:44:39.889550+0000","172.21.15.97:0/2319706992":"2024-03-28T10:44:39.889550+0000","172.21.15.97:0/3911532654":"2024-03-28T10:44:06.019075+0000","172.21.15.97:0/3347321845":"2024-03-28T10:44:06.019075+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-27T10:55:43.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.797+0000 7f920c400640 1 -- 172.21.15.97:0/1288220922 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f91ec0a3610 msgr2=0x7f91ec0a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:43.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.797+0000 7f920c400640 1 --2- 172.21.15.97:0/1288220922 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f91ec0a3610 0x7f91ec0a5ad0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f91f8006010 tx=0x7f91f8000bc0 comp rx=0 tx=0).stop 2024-03-27T10:55:43.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.797+0000 7f920c400640 1 -- 172.21.15.97:0/1288220922 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9204102d20 msgr2=0x7f920419e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:43.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.797+0000 7f920c400640 1 --2- 172.21.15.97:0/1288220922 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9204102d20 0x7f920419e870 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f91f000d950 tx=0x7f91f000de10 comp rx=0 tx=0).stop 2024-03-27T10:55:43.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.797+0000 7f920c400640 1 -- 172.21.15.97:0/1288220922 shutdown_connections 2024-03-27T10:55:43.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.797+0000 7f920c400640 1 --2- 172.21.15.97:0/1288220922 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f91ec0a3610 0x7f91ec0a5ad0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:43.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.797+0000 7f920c400640 1 --2- 172.21.15.97:0/1288220922 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f92041056d0 0x7f92041a58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:43.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.797+0000 7f920c400640 1 --2- 172.21.15.97:0/1288220922 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9204102d20 0x7f920419e870 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:43.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.797+0000 7f920c400640 1 --2- 172.21.15.97:0/1288220922 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f92041003b0 0x7f920419e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:43.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.797+0000 7f920c400640 1 -- 172.21.15.97:0/1288220922 >> 172.21.15.97:0/1288220922 conn(0x7f92040fbfb0 msgr2=0x7f9204103860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:55:43.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.797+0000 7f920c400640 1 -- 172.21.15.97:0/1288220922 shutdown_connections 2024-03-27T10:55:43.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:43.797+0000 7f920c400640 1 -- 172.21.15.97:0/1288220922 wait complete. 2024-03-27T10:55:43.787 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-27T10:47:54.052810+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-27T10:54:02.845060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-27T10:54:03.474151+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-27T10:55:43.788 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-27T10:55:44.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.035+0000 7f03c4c00640 1 -- 172.21.15.97:0/2968972421 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03c0109d20 msgr2=0x7f03c01105b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:44.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.035+0000 7f03c4c00640 1 --2- 172.21.15.97:0/2968972421 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03c0109d20 0x7f03c01105b0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f03b8009ca0 tx=0x7f03b802c0f0 comp rx=0 tx=0).stop 2024-03-27T10:55:44.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.036+0000 7f03c4c00640 1 -- 172.21.15.97:0/2968972421 shutdown_connections 2024-03-27T10:55:44.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.036+0000 7f03c4c00640 1 --2- 172.21.15.97:0/2968972421 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03c0109d20 0x7f03c01105b0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:44.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.036+0000 7f03c4c00640 1 --2- 172.21.15.97:0/2968972421 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f03c0109360 0x7f03c01097e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:44.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.036+0000 7f03c4c00640 1 --2- 172.21.15.97:0/2968972421 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f03c0108170 0x7f03c0108590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:44.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.036+0000 7f03c4c00640 1 -- 172.21.15.97:0/2968972421 >> 172.21.15.97:0/2968972421 conn(0x7f03c0077110 msgr2=0x7f03c0079550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:55:44.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.036+0000 7f03c4c00640 1 -- 172.21.15.97:0/2968972421 shutdown_connections 2024-03-27T10:55:44.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.036+0000 7f03c4c00640 1 -- 172.21.15.97:0/2968972421 wait complete. 2024-03-27T10:55:44.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.037+0000 7f03c4c00640 1 Processor -- start 2024-03-27T10:55:44.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.037+0000 7f03c4c00640 1 -- start start 2024-03-27T10:55:44.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.037+0000 7f03c4c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f03c0108170 0x7f03c019e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:44.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.037+0000 7f03c4c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03c0109360 0x7f03c019e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:44.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.037+0000 7f03c4c00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f03c0109d20 0x7f03c01a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:44.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.037+0000 7f03c4c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f03c0113cb0 con 0x7f03c0109360 2024-03-27T10:55:44.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.037+0000 7f03c4c00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f03c0113b30 con 0x7f03c0108170 2024-03-27T10:55:44.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.037+0000 7f03c4c00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f03c0113e30 con 0x7f03c0109d20 2024-03-27T10:55:44.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.037+0000 7f03b7e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03c0109360 0x7f03c019e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:44.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.037+0000 7f03b7e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03c0109360 0x7f03c019e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40688/0 (socket says 172.21.15.97:40688) 2024-03-27T10:55:44.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.037+0000 7f03b7e00640 1 -- 172.21.15.97:0/2629037319 learned_addr learned my addr 172.21.15.97:0/2629037319 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:55:44.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.038+0000 7f03b7e00640 1 -- 172.21.15.97:0/2629037319 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f03c0109d20 msgr2=0x7f03c01a58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:44.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.038+0000 7f03b7e00640 1 --2- 172.21.15.97:0/2629037319 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f03c0109d20 0x7f03c01a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:44.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.038+0000 7f03b7e00640 1 -- 172.21.15.97:0/2629037319 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f03c0108170 msgr2=0x7f03c019e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:44.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.038+0000 7f03bde00640 1 --2- 172.21.15.97:0/2629037319 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f03c0108170 0x7f03c019e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:44.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.038+0000 7f03b7e00640 1 --2- 172.21.15.97:0/2629037319 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f03c0108170 0x7f03c019e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:44.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.038+0000 7f03b7e00640 1 -- 172.21.15.97:0/2629037319 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03c01a5f60 con 0x7f03c0109360 2024-03-27T10:55:44.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.038+0000 7f03b7e00640 1 --2- 172.21.15.97:0/2629037319 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03c0109360 0x7f03c019e880 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f03a800d770 tx=0x7f03a800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:55:44.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.038+0000 7f03b6a00640 1 -- 172.21.15.97:0/2629037319 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a8017070 con 0x7f03c0109360 2024-03-27T10:55:44.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.038+0000 7f03b6a00640 1 -- 172.21.15.97:0/2629037319 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f03a80054d0 con 0x7f03c0109360 2024-03-27T10:55:44.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.038+0000 7f03c4c00640 1 -- 172.21.15.97:0/2629037319 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03c01a6250 con 0x7f03c0109360 2024-03-27T10:55:44.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.038+0000 7f03b6a00640 1 -- 172.21.15.97:0/2629037319 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a8004cf0 con 0x7f03c0109360 2024-03-27T10:55:44.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.038+0000 7f03c4c00640 1 -- 172.21.15.97:0/2629037319 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03c01a6750 con 0x7f03c0109360 2024-03-27T10:55:44.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.039+0000 7f03c4c00640 1 -- 172.21.15.97:0/2629037319 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03c010f490 con 0x7f03c0109360 2024-03-27T10:55:44.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.041+0000 7f03b6a00640 1 -- 172.21.15.97:0/2629037319 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f03a8004e90 con 0x7f03c0109360 2024-03-27T10:55:44.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.043+0000 7f03b6a00640 1 --2- 172.21.15.97:0/2629037319 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f03a40a3610 0x7f03a40a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:44.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.043+0000 7f03b6a00640 1 -- 172.21.15.97:0/2629037319 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f03a80cfcc0 con 0x7f03c0109360 2024-03-27T10:55:44.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.043+0000 7f03bde00640 1 --2- 172.21.15.97:0/2629037319 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f03a40a3610 0x7f03a40a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:44.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.044+0000 7f03bde00640 1 --2- 172.21.15.97:0/2629037319 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f03a40a3610 0x7f03a40a5ad0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f03b0005fd0 tx=0x7f03b0005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:55:44.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.045+0000 7f03b6a00640 1 -- 172.21.15.97:0/2629037319 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f03a809c840 con 0x7f03c0109360 2024-03-27T10:55:44.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.246+0000 7f03c4c00640 1 -- 172.21.15.97:0/2629037319 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f03c002d590 con 0x7f03c0109360 2024-03-27T10:55:44.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.247+0000 7f03b6a00640 1 -- 172.21.15.97:0/2629037319 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) ==== 93+0+10 (secure 0 0 0) 0x7f03a809c140 con 0x7f03c0109360 2024-03-27T10:55:44.219 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-03-27T10:55:44.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.250+0000 7f03c4c00640 1 -- 172.21.15.97:0/2629037319 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f03a40a3610 msgr2=0x7f03a40a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:44.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.250+0000 7f03c4c00640 1 --2- 172.21.15.97:0/2629037319 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f03a40a3610 0x7f03a40a5ad0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f03b0005fd0 tx=0x7f03b0005850 comp rx=0 tx=0).stop 2024-03-27T10:55:44.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.250+0000 7f03c4c00640 1 -- 172.21.15.97:0/2629037319 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03c0109360 msgr2=0x7f03c019e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:44.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.250+0000 7f03c4c00640 1 --2- 172.21.15.97:0/2629037319 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03c0109360 0x7f03c019e880 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f03a800d770 tx=0x7f03a800dc30 comp rx=0 tx=0).stop 2024-03-27T10:55:44.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.250+0000 7f03c4c00640 1 -- 172.21.15.97:0/2629037319 shutdown_connections 2024-03-27T10:55:44.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.250+0000 7f03c4c00640 1 --2- 172.21.15.97:0/2629037319 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f03c0109d20 0x7f03c01a58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:44.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.250+0000 7f03c4c00640 1 --2- 172.21.15.97:0/2629037319 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03c0109360 0x7f03c019e880 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:44.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.250+0000 7f03c4c00640 1 --2- 172.21.15.97:0/2629037319 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f03a40a3610 0x7f03a40a5ad0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:44.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.250+0000 7f03c4c00640 1 --2- 172.21.15.97:0/2629037319 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f03c0108170 0x7f03c019e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:44.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.250+0000 7f03c4c00640 1 -- 172.21.15.97:0/2629037319 >> 172.21.15.97:0/2629037319 conn(0x7f03c0077110 msgr2=0x7f03c010c660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:55:44.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.250+0000 7f03c4c00640 1 -- 172.21.15.97:0/2629037319 shutdown_connections 2024-03-27T10:55:44.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.250+0000 7f03c4c00640 1 -- 172.21.15.97:0/2629037319 wait complete. 2024-03-27T10:55:44.239 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-27T10:55:44.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.476+0000 7fa547c00640 1 -- 172.21.15.97:0/3392139766 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa540076510 msgr2=0x7fa540076930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:44.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.476+0000 7fa547c00640 1 --2- 172.21.15.97:0/3392139766 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa540076510 0x7fa540076930 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fa530009900 tx=0x7fa53002bc30 comp rx=0 tx=0).stop 2024-03-27T10:55:44.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.477+0000 7fa547c00640 1 -- 172.21.15.97:0/3392139766 shutdown_connections 2024-03-27T10:55:44.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.477+0000 7fa547c00640 1 --2- 172.21.15.97:0/3392139766 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa5400780f0 0x7fa540078990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:44.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.477+0000 7fa547c00640 1 --2- 172.21.15.97:0/3392139766 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa540077750 0x7fa540077bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:44.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.477+0000 7fa547c00640 1 --2- 172.21.15.97:0/3392139766 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa540076510 0x7fa540076930 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:44.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.477+0000 7fa547c00640 1 -- 172.21.15.97:0/3392139766 >> 172.21.15.97:0/3392139766 conn(0x7fa5400ff300 msgr2=0x7fa540101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:55:44.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.477+0000 7fa547c00640 1 -- 172.21.15.97:0/3392139766 shutdown_connections 2024-03-27T10:55:44.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.477+0000 7fa547c00640 1 -- 172.21.15.97:0/3392139766 wait complete. 2024-03-27T10:55:44.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.478+0000 7fa547c00640 1 Processor -- start 2024-03-27T10:55:44.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.478+0000 7fa547c00640 1 -- start start 2024-03-27T10:55:44.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.478+0000 7fa547c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa540076510 0x7fa54019e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:44.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.478+0000 7fa547c00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa540077750 0x7fa54019e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:44.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.478+0000 7fa547c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5400780f0 0x7fa5401a5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:44.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.478+0000 7fa547c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa540113d10 con 0x7fa5400780f0 2024-03-27T10:55:44.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.478+0000 7fa547c00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa540113b90 con 0x7fa540076510 2024-03-27T10:55:44.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.478+0000 7fa547c00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa540113e90 con 0x7fa540077750 2024-03-27T10:55:44.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.479+0000 7fa545c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5400780f0 0x7fa5401a5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:44.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.479+0000 7fa545c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5400780f0 0x7fa5401a5940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40702/0 (socket says 172.21.15.97:40702) 2024-03-27T10:55:44.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.479+0000 7fa545c00640 1 -- 172.21.15.97:0/1089538878 learned_addr learned my addr 172.21.15.97:0/1089538878 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:55:44.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.479+0000 7fa545200640 1 --2- 172.21.15.97:0/1089538878 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa540076510 0x7fa54019e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:44.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.479+0000 7fa537e00640 1 --2- 172.21.15.97:0/1089538878 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa540077750 0x7fa54019e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:44.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.479+0000 7fa545c00640 1 -- 172.21.15.97:0/1089538878 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa540077750 msgr2=0x7fa54019e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:44.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.479+0000 7fa545c00640 1 --2- 172.21.15.97:0/1089538878 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa540077750 0x7fa54019e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:44.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.479+0000 7fa545c00640 1 -- 172.21.15.97:0/1089538878 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa540076510 msgr2=0x7fa54019e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:44.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.479+0000 7fa545c00640 1 --2- 172.21.15.97:0/1089538878 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa540076510 0x7fa54019e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:44.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.479+0000 7fa545c00640 1 -- 172.21.15.97:0/1089538878 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5401a5fb0 con 0x7fa5400780f0 2024-03-27T10:55:44.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.479+0000 7fa537e00640 1 --2- 172.21.15.97:0/1089538878 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa540077750 0x7fa54019e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T10:55:44.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.480+0000 7fa545c00640 1 --2- 172.21.15.97:0/1089538878 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5400780f0 0x7fa5401a5940 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa53800eaa0 tx=0x7fa53800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:55:44.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.480+0000 7fa53e000640 1 -- 172.21.15.97:0/1089538878 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa53800cdb0 con 0x7fa5400780f0 2024-03-27T10:55:44.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.480+0000 7fa53e000640 1 -- 172.21.15.97:0/1089538878 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa538005020 con 0x7fa5400780f0 2024-03-27T10:55:44.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.480+0000 7fa547c00640 1 -- 172.21.15.97:0/1089538878 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5401a62a0 con 0x7fa5400780f0 2024-03-27T10:55:44.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.480+0000 7fa53e000640 1 -- 172.21.15.97:0/1089538878 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5380116d0 con 0x7fa5400780f0 2024-03-27T10:55:44.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.480+0000 7fa547c00640 1 -- 172.21.15.97:0/1089538878 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5401a6840 con 0x7fa5400780f0 2024-03-27T10:55:44.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.482+0000 7fa547c00640 1 -- 172.21.15.97:0/1089538878 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5401108f0 con 0x7fa5400780f0 2024-03-27T10:55:44.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.484+0000 7fa53e000640 1 -- 172.21.15.97:0/1089538878 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7fa5380051d0 con 0x7fa5400780f0 2024-03-27T10:55:44.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.485+0000 7fa53e000640 1 --2- 172.21.15.97:0/1089538878 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fa5240a3690 0x7fa5240a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:44.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.485+0000 7fa53e000640 1 -- 172.21.15.97:0/1089538878 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fa5380cfeb0 con 0x7fa5400780f0 2024-03-27T10:55:44.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.485+0000 7fa545200640 1 --2- 172.21.15.97:0/1089538878 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fa5240a3690 0x7fa5240a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:44.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.486+0000 7fa545200640 1 --2- 172.21.15.97:0/1089538878 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fa5240a3690 0x7fa5240a5b50 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa530000c00 tx=0x7fa53002c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:55:44.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.491+0000 7fa53e000640 1 -- 172.21.15.97:0/1089538878 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7fa538016030 con 0x7fa5400780f0 2024-03-27T10:55:44.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:55:44 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/105730681' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T10:55:44.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:55:44 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/1288220922' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T10:55:44.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:55:44 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/2629037319' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-27T10:55:44.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.692+0000 7fa547c00640 1 -- 172.21.15.97:0/1089538878 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa540062750 con 0x7fa5400780f0 2024-03-27T10:55:44.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.693+0000 7fa53e000640 1 -- 172.21.15.97:0/1089538878 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) ==== 104+0+11 (secure 0 0 0) 0x7fa53809ca30 con 0x7fa5400780f0 2024-03-27T10:55:44.665 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-03-27T10:55:44.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.696+0000 7fa547c00640 1 -- 172.21.15.97:0/1089538878 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fa5240a3690 msgr2=0x7fa5240a5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:44.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.696+0000 7fa547c00640 1 --2- 172.21.15.97:0/1089538878 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fa5240a3690 0x7fa5240a5b50 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa530000c00 tx=0x7fa53002c7e0 comp rx=0 tx=0).stop 2024-03-27T10:55:44.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.696+0000 7fa547c00640 1 -- 172.21.15.97:0/1089538878 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5400780f0 msgr2=0x7fa5401a5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:44.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.696+0000 7fa547c00640 1 --2- 172.21.15.97:0/1089538878 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5400780f0 0x7fa5401a5940 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa53800eaa0 tx=0x7fa53800ef60 comp rx=0 tx=0).stop 2024-03-27T10:55:44.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.696+0000 7fa547c00640 1 -- 172.21.15.97:0/1089538878 shutdown_connections 2024-03-27T10:55:44.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.696+0000 7fa547c00640 1 --2- 172.21.15.97:0/1089538878 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5400780f0 0x7fa5401a5940 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:44.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.696+0000 7fa547c00640 1 --2- 172.21.15.97:0/1089538878 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa540077750 0x7fa54019e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:44.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.696+0000 7fa547c00640 1 --2- 172.21.15.97:0/1089538878 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fa5240a3690 0x7fa5240a5b50 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:44.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.696+0000 7fa547c00640 1 --2- 172.21.15.97:0/1089538878 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa540076510 0x7fa54019e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:44.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.696+0000 7fa547c00640 1 -- 172.21.15.97:0/1089538878 >> 172.21.15.97:0/1089538878 conn(0x7fa5400ff300 msgr2=0x7fa540100aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:55:44.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.696+0000 7fa547c00640 1 -- 172.21.15.97:0/1089538878 shutdown_connections 2024-03-27T10:55:44.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.696+0000 7fa547c00640 1 -- 172.21.15.97:0/1089538878 wait complete. 2024-03-27T10:55:44.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:55:44 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/105730681' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T10:55:44.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:55:44 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/1288220922' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T10:55:44.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:55:44 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/2629037319' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-27T10:55:44.686 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-27T10:55:44.767 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:55:44 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/105730681' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T10:55:44.767 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:55:44 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/1288220922' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T10:55:44.767 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:55:44 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/2629037319' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-27T10:55:44.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.932+0000 7f8c90e00640 1 -- 172.21.15.97:0/1618828388 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c8c0745b0 msgr2=0x7f8c8c074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:44.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.932+0000 7f8c90e00640 1 --2- 172.21.15.97:0/1618828388 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c8c0745b0 0x7f8c8c074a30 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f8c74009900 tx=0x7f8c7402bc30 comp rx=0 tx=0).stop 2024-03-27T10:55:44.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.933+0000 7f8c90e00640 1 -- 172.21.15.97:0/1618828388 shutdown_connections 2024-03-27T10:55:44.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.933+0000 7f8c90e00640 1 --2- 172.21.15.97:0/1618828388 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c8c1098f0 0x7f8c8c10bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:44.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.933+0000 7f8c90e00640 1 --2- 172.21.15.97:0/1618828388 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c8c0745b0 0x7f8c8c074a30 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:44.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.933+0000 7f8c90e00640 1 --2- 172.21.15.97:0/1618828388 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8c8c075b90 0x7f8c8c073fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:44.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.933+0000 7f8c90e00640 1 -- 172.21.15.97:0/1618828388 >> 172.21.15.97:0/1618828388 conn(0x7f8c8c0fcfc0 msgr2=0x7f8c8c0ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:55:44.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.933+0000 7f8c90e00640 1 -- 172.21.15.97:0/1618828388 shutdown_connections 2024-03-27T10:55:44.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.933+0000 7f8c90e00640 1 -- 172.21.15.97:0/1618828388 wait complete. 2024-03-27T10:55:44.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.934+0000 7f8c90e00640 1 Processor -- start 2024-03-27T10:55:44.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.934+0000 7f8c90e00640 1 -- start start 2024-03-27T10:55:44.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.934+0000 7f8c90e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c8c0745b0 0x7f8c8c199e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:44.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.934+0000 7f8c90e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c8c075b90 0x7f8c8c19a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:44.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.934+0000 7f8c90e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8c8c1098f0 0x7f8c8c1a13d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:44.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.934+0000 7f8c90e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8c8c10f790 con 0x7f8c8c0745b0 2024-03-27T10:55:44.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.934+0000 7f8c90e00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f8c8c10f610 con 0x7f8c8c075b90 2024-03-27T10:55:44.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.934+0000 7f8c90e00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f8c8c10f910 con 0x7f8c8c1098f0 2024-03-27T10:55:44.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.934+0000 7f8c8b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c8c0745b0 0x7f8c8c199e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:44.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.934+0000 7f8c8b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c8c0745b0 0x7f8c8c199e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40718/0 (socket says 172.21.15.97:40718) 2024-03-27T10:55:44.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.934+0000 7f8c8b400640 1 -- 172.21.15.97:0/342683264 learned_addr learned my addr 172.21.15.97:0/342683264 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:55:44.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.935+0000 7f8c83e00640 1 --2- 172.21.15.97:0/342683264 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c8c075b90 0x7f8c8c19a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:44.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.935+0000 7f8c8b400640 1 -- 172.21.15.97:0/342683264 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8c8c1098f0 msgr2=0x7f8c8c1a13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:44.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.935+0000 7f8c8b400640 1 --2- 172.21.15.97:0/342683264 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8c8c1098f0 0x7f8c8c1a13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:44.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.935+0000 7f8c8b400640 1 -- 172.21.15.97:0/342683264 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c8c075b90 msgr2=0x7f8c8c19a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:44.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.935+0000 7f8c8b400640 1 --2- 172.21.15.97:0/342683264 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c8c075b90 0x7f8c8c19a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:44.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.935+0000 7f8c8b400640 1 -- 172.21.15.97:0/342683264 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c8c1a1a40 con 0x7f8c8c0745b0 2024-03-27T10:55:44.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.935+0000 7f8c83e00640 1 --2- 172.21.15.97:0/342683264 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c8c075b90 0x7f8c8c19a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:55:44.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.935+0000 7f8c8b400640 1 --2- 172.21.15.97:0/342683264 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c8c0745b0 0x7f8c8c199e20 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c00d770 tx=0x7f8c7c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:55:44.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.935+0000 7f8c88c00640 1 -- 172.21.15.97:0/342683264 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c7c017070 con 0x7f8c8c0745b0 2024-03-27T10:55:44.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.935+0000 7f8c88c00640 1 -- 172.21.15.97:0/342683264 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8c7c0054d0 con 0x7f8c8c0745b0 2024-03-27T10:55:44.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.935+0000 7f8c90e00640 1 -- 172.21.15.97:0/342683264 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c8c1a1d30 con 0x7f8c8c0745b0 2024-03-27T10:55:44.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.935+0000 7f8c88c00640 1 -- 172.21.15.97:0/342683264 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c7c004cf0 con 0x7f8c8c0745b0 2024-03-27T10:55:44.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.936+0000 7f8c90e00640 1 -- 172.21.15.97:0/342683264 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c8c1a21d0 con 0x7f8c8c0745b0 2024-03-27T10:55:44.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.937+0000 7f8c90e00640 1 -- 172.21.15.97:0/342683264 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c8c108620 con 0x7f8c8c0745b0 2024-03-27T10:55:44.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.939+0000 7f8c88c00640 1 -- 172.21.15.97:0/342683264 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f8c7c004e90 con 0x7f8c8c0745b0 2024-03-27T10:55:44.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.940+0000 7f8c88c00640 1 --2- 172.21.15.97:0/342683264 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f8c700a3690 0x7f8c700a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:44.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.940+0000 7f8c88c00640 1 -- 172.21.15.97:0/342683264 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f8c7c0cf810 con 0x7f8c8c0745b0 2024-03-27T10:55:44.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.940+0000 7f8c83e00640 1 --2- 172.21.15.97:0/342683264 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f8c700a3690 0x7f8c700a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:44.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.941+0000 7f8c83e00640 1 --2- 172.21.15.97:0/342683264 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f8c700a3690 0x7f8c700a5b50 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8c74036040 tx=0x7f8c74005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:55:44.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:44.944+0000 7f8c88c00640 1 -- 172.21.15.97:0/342683264 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f8c7c09c390 con 0x7f8c8c0745b0 2024-03-27T10:55:45.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.141+0000 7f8c90e00640 1 -- 172.21.15.97:0/342683264 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f8c8c062820 con 0x7f8c8c0745b0 2024-03-27T10:55:45.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.142+0000 7f8c88c00640 1 -- 172.21.15.97:0/342683264 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) ==== 100+0+11 (secure 0 0 0) 0x7f8c7c09bc90 con 0x7f8c8c0745b0 2024-03-27T10:55:45.113 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-03-27T10:55:45.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.144+0000 7f8c90e00640 1 -- 172.21.15.97:0/342683264 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f8c700a3690 msgr2=0x7f8c700a5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:45.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.144+0000 7f8c90e00640 1 --2- 172.21.15.97:0/342683264 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f8c700a3690 0x7f8c700a5b50 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8c74036040 tx=0x7f8c74005fd0 comp rx=0 tx=0).stop 2024-03-27T10:55:45.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.144+0000 7f8c90e00640 1 -- 172.21.15.97:0/342683264 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c8c0745b0 msgr2=0x7f8c8c199e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:45.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.144+0000 7f8c90e00640 1 --2- 172.21.15.97:0/342683264 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c8c0745b0 0x7f8c8c199e20 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c00d770 tx=0x7f8c7c00dc30 comp rx=0 tx=0).stop 2024-03-27T10:55:45.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.144+0000 7f8c90e00640 1 -- 172.21.15.97:0/342683264 shutdown_connections 2024-03-27T10:55:45.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.144+0000 7f8c90e00640 1 --2- 172.21.15.97:0/342683264 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8c8c1098f0 0x7f8c8c1a13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:45.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.145+0000 7f8c90e00640 1 --2- 172.21.15.97:0/342683264 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f8c700a3690 0x7f8c700a5b50 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:45.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.145+0000 7f8c90e00640 1 --2- 172.21.15.97:0/342683264 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c8c075b90 0x7f8c8c19a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:45.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.145+0000 7f8c90e00640 1 --2- 172.21.15.97:0/342683264 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c8c0745b0 0x7f8c8c199e20 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:45.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.145+0000 7f8c90e00640 1 -- 172.21.15.97:0/342683264 >> 172.21.15.97:0/342683264 conn(0x7f8c8c0fcfc0 msgr2=0x7f8c8c1057f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:55:45.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.145+0000 7f8c90e00640 1 -- 172.21.15.97:0/342683264 shutdown_connections 2024-03-27T10:55:45.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.145+0000 7f8c90e00640 1 -- 172.21.15.97:0/342683264 wait complete. 2024-03-27T10:55:45.135 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-27T10:55:45.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.372+0000 7f5fda200640 1 -- 172.21.15.97:0/2983264315 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5fd4104f20 msgr2=0x7f5fd4107310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:45.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.372+0000 7f5fda200640 1 --2- 172.21.15.97:0/2983264315 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5fd4104f20 0x7f5fd4107310 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc009900 tx=0x7f5fbc02bc30 comp rx=0 tx=0).stop 2024-03-27T10:55:45.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.372+0000 7f5fda200640 1 -- 172.21.15.97:0/2983264315 shutdown_connections 2024-03-27T10:55:45.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.372+0000 7f5fda200640 1 --2- 172.21.15.97:0/2983264315 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5fd4107850 0x7f5fd4109ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:45.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.372+0000 7f5fda200640 1 --2- 172.21.15.97:0/2983264315 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5fd4104f20 0x7f5fd4107310 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:45.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.372+0000 7f5fda200640 1 --2- 172.21.15.97:0/2983264315 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5fd41025f0 0x7f5fd41049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:45.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.372+0000 7f5fda200640 1 -- 172.21.15.97:0/2983264315 >> 172.21.15.97:0/2983264315 conn(0x7f5fd40fbfe0 msgr2=0x7f5fd40fe420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:55:45.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.372+0000 7f5fda200640 1 -- 172.21.15.97:0/2983264315 shutdown_connections 2024-03-27T10:55:45.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.372+0000 7f5fda200640 1 -- 172.21.15.97:0/2983264315 wait complete. 2024-03-27T10:55:45.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.373+0000 7f5fda200640 1 Processor -- start 2024-03-27T10:55:45.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.373+0000 7f5fda200640 1 -- start start 2024-03-27T10:55:45.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.374+0000 7f5fda200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5fd41025f0 0x7f5fd419a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:45.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.374+0000 7f5fda200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5fd4104f20 0x7f5fd419a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:45.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.374+0000 7f5fda200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5fd4107850 0x7f5fd41a15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:45.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.374+0000 7f5fda200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd410d6f0 con 0x7f5fd4107850 2024-03-27T10:55:45.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.374+0000 7f5fda200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd410d570 con 0x7f5fd4104f20 2024-03-27T10:55:45.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.374+0000 7f5fda200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd410d870 con 0x7f5fd41025f0 2024-03-27T10:55:45.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.374+0000 7f5fd3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5fd4107850 0x7f5fd41a15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:45.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.374+0000 7f5fd3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5fd4107850 0x7f5fd41a15d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40736/0 (socket says 172.21.15.97:40736) 2024-03-27T10:55:45.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.374+0000 7f5fd3e00640 1 -- 172.21.15.97:0/2680180006 learned_addr learned my addr 172.21.15.97:0/2680180006 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:55:45.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.374+0000 7f5fd2a00640 1 --2- 172.21.15.97:0/2680180006 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5fd4104f20 0x7f5fd419a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:45.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.374+0000 7f5fd3400640 1 --2- 172.21.15.97:0/2680180006 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5fd41025f0 0x7f5fd419a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:45.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.374+0000 7f5fd3e00640 1 -- 172.21.15.97:0/2680180006 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5fd41025f0 msgr2=0x7f5fd419a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:45.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.374+0000 7f5fd3e00640 1 --2- 172.21.15.97:0/2680180006 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5fd41025f0 0x7f5fd419a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:45.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.374+0000 7f5fd3e00640 1 -- 172.21.15.97:0/2680180006 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5fd4104f20 msgr2=0x7f5fd419a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:45.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.374+0000 7f5fd3e00640 1 --2- 172.21.15.97:0/2680180006 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5fd4104f20 0x7f5fd419a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:45.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.374+0000 7f5fd3e00640 1 -- 172.21.15.97:0/2680180006 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fd41a1c40 con 0x7f5fd4107850 2024-03-27T10:55:45.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.375+0000 7f5fd3e00640 1 --2- 172.21.15.97:0/2680180006 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5fd4107850 0x7f5fd41a15d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc00cad0 tx=0x7f5fcc00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:55:45.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.375+0000 7f5fcbe00640 1 -- 172.21.15.97:0/2680180006 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fcc00ae20 con 0x7f5fd4107850 2024-03-27T10:55:45.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.375+0000 7f5fcbe00640 1 -- 172.21.15.97:0/2680180006 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5fcc005020 con 0x7f5fd4107850 2024-03-27T10:55:45.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.375+0000 7f5fda200640 1 -- 172.21.15.97:0/2680180006 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fd41a1f30 con 0x7f5fd4107850 2024-03-27T10:55:45.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.375+0000 7f5fcbe00640 1 -- 172.21.15.97:0/2680180006 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fcc00f6b0 con 0x7f5fd4107850 2024-03-27T10:55:45.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.375+0000 7f5fda200640 1 -- 172.21.15.97:0/2680180006 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fd41a23f0 con 0x7f5fd4107850 2024-03-27T10:55:45.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.376+0000 7f5fda200640 1 -- 172.21.15.97:0/2680180006 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fd41093a0 con 0x7f5fd4107850 2024-03-27T10:55:45.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.378+0000 7f5fcbe00640 1 -- 172.21.15.97:0/2680180006 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f5fcc0051d0 con 0x7f5fd4107850 2024-03-27T10:55:45.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.380+0000 7f5fcbe00640 1 --2- 172.21.15.97:0/2680180006 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f5fb40a3610 0x7f5fb40a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:45.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.380+0000 7f5fcbe00640 1 -- 172.21.15.97:0/2680180006 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f5fcc0cfd50 con 0x7f5fd4107850 2024-03-27T10:55:45.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.380+0000 7f5fd3400640 1 --2- 172.21.15.97:0/2680180006 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f5fb40a3610 0x7f5fb40a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:45.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.381+0000 7f5fd3400640 1 --2- 172.21.15.97:0/2680180006 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f5fb40a3610 0x7f5fb40a5ad0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f5fc4005d00 tx=0x7f5fc4005c50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:55:45.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.385+0000 7f5fcbe00640 1 -- 172.21.15.97:0/2680180006 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f5fcc014030 con 0x7f5fd4107850 2024-03-27T10:55:45.390 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:55:45 smithi097 ceph-mon[31859]: pgmap v414: 97 pgs: 97 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 426 B/s wr, 11 op/s 2024-03-27T10:55:45.390 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:55:45 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/1089538878' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-27T10:55:45.390 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:55:45 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/342683264' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-27T10:55:45.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.624+0000 7f5fda200640 1 -- 172.21.15.97:0/2680180006 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5fd40627f0 con 0x7f5fd4107850 2024-03-27T10:55:45.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.626+0000 7f5fcbe00640 1 -- 172.21.15.97:0/2680180006 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11944 (secure 0 0 0) 0x7f5fcc09c8d0 con 0x7f5fd4107850 2024-03-27T10:55:45.597 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T10:55:45.597 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1009423904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1009423904},{"type":"v1","addr":"172.21.15.97:6835","nonce":1009423904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3251173433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3251173433},{"type":"v1","addr":"172.21.15.97:6837","nonce":3251173433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-27T10:54:04.510178+0000","modified":"2024-03-27T10:54:12.180047+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34508,"mds_1":34478,"mds_2":34445,"mds_3":24454,"mds_4":24400},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14646":{"gid":14646,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6839/1057596821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1057596821},{"type":"v1","addr":"172.21.15.97:6839","nonce":1057596821}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14658":{"gid":14658,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6841/1672464856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6840","nonce":1672464856},{"type":"v1","addr":"172.21.15.97:6841","nonce":1672464856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.177:6833/3787981632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3787981632},{"type":"v1","addr":"172.21.15.177:6833","nonce":3787981632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.177:6835/3349112132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":3349112132},{"type":"v1","addr":"172.21.15.177:6835","nonce":3349112132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6837/3244432640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":3244432640},{"type":"v1","addr":"172.21.15.177:6837","nonce":3244432640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6839/3340454605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":3340454605},{"type":"v1","addr":"172.21.15.177:6839","nonce":3340454605}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34415":{"gid":34415,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6833/4052517902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":4052517902},{"type":"v1","addr":"172.21.15.114:6833","nonce":4052517902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34445":{"gid":34445,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.114:6835/3581918222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3581918222},{"type":"v1","addr":"172.21.15.114:6835","nonce":3581918222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34478":{"gid":34478,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.114:6837/616775588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":616775588},{"type":"v1","addr":"172.21.15.114:6837","nonce":616775588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34508":{"gid":34508,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.114:6839/3937193784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":3937193784},{"type":"v1","addr":"172.21.15.114:6839","nonce":3937193784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34508,"qdb_cluster":[34445,34508,24400,24454,34478]},"id":1}]} 2024-03-27T10:55:45.598 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 27 2024-03-27T10:55:45.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.628+0000 7f5fda200640 1 -- 172.21.15.97:0/2680180006 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f5fb40a3610 msgr2=0x7f5fb40a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:45.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.628+0000 7f5fda200640 1 --2- 172.21.15.97:0/2680180006 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f5fb40a3610 0x7f5fb40a5ad0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f5fc4005d00 tx=0x7f5fc4005c50 comp rx=0 tx=0).stop 2024-03-27T10:55:45.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.628+0000 7f5fda200640 1 -- 172.21.15.97:0/2680180006 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5fd4107850 msgr2=0x7f5fd41a15d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:45.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.628+0000 7f5fda200640 1 --2- 172.21.15.97:0/2680180006 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5fd4107850 0x7f5fd41a15d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc00cad0 tx=0x7f5fcc00cf90 comp rx=0 tx=0).stop 2024-03-27T10:55:45.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.628+0000 7f5fda200640 1 -- 172.21.15.97:0/2680180006 shutdown_connections 2024-03-27T10:55:45.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.628+0000 7f5fda200640 1 --2- 172.21.15.97:0/2680180006 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5fd4107850 0x7f5fd41a15d0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:45.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.628+0000 7f5fda200640 1 --2- 172.21.15.97:0/2680180006 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f5fb40a3610 0x7f5fb40a5ad0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:45.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.628+0000 7f5fda200640 1 --2- 172.21.15.97:0/2680180006 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5fd4104f20 0x7f5fd419a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:45.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.628+0000 7f5fda200640 1 --2- 172.21.15.97:0/2680180006 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5fd41025f0 0x7f5fd419a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:45.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.628+0000 7f5fda200640 1 -- 172.21.15.97:0/2680180006 >> 172.21.15.97:0/2680180006 conn(0x7f5fd40fbfe0 msgr2=0x7f5fd4101ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:55:45.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.629+0000 7f5fda200640 1 -- 172.21.15.97:0/2680180006 shutdown_connections 2024-03-27T10:55:45.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.629+0000 7f5fda200640 1 -- 172.21.15.97:0/2680180006 wait complete. 2024-03-27T10:55:45.617 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-27T10:55:45.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:55:45 smithi177 ceph-mon[35294]: pgmap v414: 97 pgs: 97 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 426 B/s wr, 11 op/s 2024-03-27T10:55:45.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:55:45 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/1089538878' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-27T10:55:45.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:55:45 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/342683264' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-27T10:55:45.767 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:55:45 smithi114 ceph-mon[35338]: pgmap v414: 97 pgs: 97 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 426 B/s wr, 11 op/s 2024-03-27T10:55:45.767 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:55:45 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/1089538878' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-27T10:55:45.767 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:55:45 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/342683264' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-27T10:55:45.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.854+0000 7f64c2200640 1 -- 172.21.15.97:0/2199133543 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64bc109b40 msgr2=0x7f64bc1103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:45.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.854+0000 7f64c2200640 1 --2- 172.21.15.97:0/2199133543 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64bc109b40 0x7f64bc1103d0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f64b4009ca0 tx=0x7f64b402c0f0 comp rx=0 tx=0).stop 2024-03-27T10:55:45.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.854+0000 7f64c2200640 1 -- 172.21.15.97:0/2199133543 shutdown_connections 2024-03-27T10:55:45.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.855+0000 7f64c2200640 1 --2- 172.21.15.97:0/2199133543 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64bc109b40 0x7f64bc1103d0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:45.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.855+0000 7f64c2200640 1 --2- 172.21.15.97:0/2199133543 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f64bc109180 0x7f64bc109600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:45.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.855+0000 7f64c2200640 1 --2- 172.21.15.97:0/2199133543 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f64bc107f90 0x7f64bc1083b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:45.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.855+0000 7f64c2200640 1 -- 172.21.15.97:0/2199133543 >> 172.21.15.97:0/2199133543 conn(0x7f64bc076f20 msgr2=0x7f64bc079360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:55:45.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.855+0000 7f64c2200640 1 -- 172.21.15.97:0/2199133543 shutdown_connections 2024-03-27T10:55:45.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.855+0000 7f64c2200640 1 -- 172.21.15.97:0/2199133543 wait complete. 2024-03-27T10:55:45.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.855+0000 7f64c2200640 1 Processor -- start 2024-03-27T10:55:45.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.856+0000 7f64c2200640 1 -- start start 2024-03-27T10:55:45.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.856+0000 7f64c2200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f64bc107f90 0x7f64bc19e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:45.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.856+0000 7f64c2200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64bc109180 0x7f64bc19e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:45.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.856+0000 7f64c2200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f64bc109b40 0x7f64bc1a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:45.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.856+0000 7f64c2200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f64bc113ad0 con 0x7f64bc109180 2024-03-27T10:55:45.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.856+0000 7f64c2200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f64bc113950 con 0x7f64bc109b40 2024-03-27T10:55:45.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.856+0000 7f64c2200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f64bc113c50 con 0x7f64bc107f90 2024-03-27T10:55:45.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.856+0000 7f64bbe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64bc109180 0x7f64bc19e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:45.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.856+0000 7f64bbe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64bc109180 0x7f64bc19e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40760/0 (socket says 172.21.15.97:40760) 2024-03-27T10:55:45.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.856+0000 7f64bbe00640 1 -- 172.21.15.97:0/4239863144 learned_addr learned my addr 172.21.15.97:0/4239863144 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:55:45.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.856+0000 7f64c0e00640 1 --2- 172.21.15.97:0/4239863144 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f64bc107f90 0x7f64bc19e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:45.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.857+0000 7f64c1800640 1 --2- 172.21.15.97:0/4239863144 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f64bc109b40 0x7f64bc1a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:45.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.857+0000 7f64bbe00640 1 -- 172.21.15.97:0/4239863144 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f64bc107f90 msgr2=0x7f64bc19e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:45.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.857+0000 7f64bbe00640 1 --2- 172.21.15.97:0/4239863144 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f64bc107f90 0x7f64bc19e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:45.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.857+0000 7f64bbe00640 1 -- 172.21.15.97:0/4239863144 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f64bc109b40 msgr2=0x7f64bc1a5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:45.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.857+0000 7f64bbe00640 1 --2- 172.21.15.97:0/4239863144 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f64bc109b40 0x7f64bc1a5700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:45.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.857+0000 7f64bbe00640 1 -- 172.21.15.97:0/4239863144 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64bc1a5d70 con 0x7f64bc109180 2024-03-27T10:55:45.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.857+0000 7f64c1800640 1 --2- 172.21.15.97:0/4239863144 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f64bc109b40 0x7f64bc1a5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:55:45.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.857+0000 7f64bbe00640 1 --2- 172.21.15.97:0/4239863144 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64bc109180 0x7f64bc19e690 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f64a800d770 tx=0x7f64a800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:55:45.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.857+0000 7f64b9600640 1 -- 172.21.15.97:0/4239863144 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a8017070 con 0x7f64bc109180 2024-03-27T10:55:45.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.857+0000 7f64b9600640 1 -- 172.21.15.97:0/4239863144 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f64a80054d0 con 0x7f64bc109180 2024-03-27T10:55:45.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.857+0000 7f64c2200640 1 -- 172.21.15.97:0/4239863144 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64bc1a6060 con 0x7f64bc109180 2024-03-27T10:55:45.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.857+0000 7f64b9600640 1 -- 172.21.15.97:0/4239863144 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a8004cf0 con 0x7f64bc109180 2024-03-27T10:55:45.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.857+0000 7f64c2200640 1 -- 172.21.15.97:0/4239863144 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64bc1a6560 con 0x7f64bc109180 2024-03-27T10:55:45.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.859+0000 7f64c2200640 1 -- 172.21.15.97:0/4239863144 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64bc10f2b0 con 0x7f64bc109180 2024-03-27T10:55:45.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.861+0000 7f64b9600640 1 -- 172.21.15.97:0/4239863144 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f64a8004e90 con 0x7f64bc109180 2024-03-27T10:55:45.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.862+0000 7f64b9600640 1 --2- 172.21.15.97:0/4239863144 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f64a40a3610 0x7f64a40a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:45.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.862+0000 7f64b9600640 1 -- 172.21.15.97:0/4239863144 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f64a80cf890 con 0x7f64bc109180 2024-03-27T10:55:45.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.862+0000 7f64c0e00640 1 --2- 172.21.15.97:0/4239863144 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f64a40a3610 0x7f64a40a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:45.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.863+0000 7f64c0e00640 1 --2- 172.21.15.97:0/4239863144 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f64a40a3610 0x7f64a40a5ad0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f64b0005fd0 tx=0x7f64b0005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:55:45.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:45.867+0000 7f64b9600640 1 -- 172.21.15.97:0/4239863144 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f64a809c410 con 0x7f64bc109180 2024-03-27T10:55:46.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.055+0000 7f64c2200640 1 -- 172.21.15.97:0/4239863144 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f64bc062780 con 0x7f64bc109180 2024-03-27T10:55:46.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.058+0000 7f64b9600640 1 -- 172.21.15.97:0/4239863144 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+22644 (secure 0 0 0) 0x7f64a809bd10 con 0x7f64bc109180 2024-03-27T10:55:46.030 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T10:55:46.030 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":92,"fsid":"adca88c4-ec26-11ee-95d0-87774f69a715","created":"2024-03-27T10:43:24.680694+0000","modified":"2024-03-27T10:54:41.310812+0000","last_up_change":"2024-03-27T10:53:00.552513+0000","last_in_change":"2024-03-27T10:52:36.302986+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-27T10:47:54.052810+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-27T10:54:02.845060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-27T10:54:03.474151+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b634e5f1-6118-4e2f-95e9-830db390f8d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1746812080},{"type":"v1","addr":"172.21.15.97:6803","nonce":1746812080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1746812080},{"type":"v1","addr":"172.21.15.97:6805","nonce":1746812080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":1746812080},{"type":"v1","addr":"172.21.15.97:6809","nonce":1746812080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1746812080},{"type":"v1","addr":"172.21.15.97:6807","nonce":1746812080}]},"public_addr":"172.21.15.97:6803/1746812080","cluster_addr":"172.21.15.97:6805/1746812080","heartbeat_back_addr":"172.21.15.97:6809/1746812080","heartbeat_front_addr":"172.21.15.97:6807/1746812080","state":["exists","up"]},{"osd":1,"uuid":"441e2770-3284-4388-b0de-c56eb8a373d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":3858821264},{"type":"v1","addr":"172.21.15.114:6801","nonce":3858821264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":3858821264},{"type":"v1","addr":"172.21.15.114:6803","nonce":3858821264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":3858821264},{"type":"v1","addr":"172.21.15.114:6807","nonce":3858821264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":3858821264},{"type":"v1","addr":"172.21.15.114:6805","nonce":3858821264}]},"public_addr":"172.21.15.114:6801/3858821264","cluster_addr":"172.21.15.114:6803/3858821264","heartbeat_back_addr":"172.21.15.114:6807/3858821264","heartbeat_front_addr":"172.21.15.114:6805/3858821264","state":["exists","up"]},{"osd":2,"uuid":"00108864-1aa6-48d6-8398-e8eb79fda123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":124240658},{"type":"v1","addr":"172.21.15.177:6801","nonce":124240658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":124240658},{"type":"v1","addr":"172.21.15.177:6803","nonce":124240658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":124240658},{"type":"v1","addr":"172.21.15.177:6807","nonce":124240658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":124240658},{"type":"v1","addr":"172.21.15.177:6805","nonce":124240658}]},"public_addr":"172.21.15.177:6801/124240658","cluster_addr":"172.21.15.177:6803/124240658","heartbeat_back_addr":"172.21.15.177:6807/124240658","heartbeat_front_addr":"172.21.15.177:6805/124240658","state":["exists","up"]},{"osd":3,"uuid":"1860b690-e2dc-4977-88d7-d1aacf7022ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2478179311},{"type":"v1","addr":"172.21.15.97:6811","nonce":2478179311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2478179311},{"type":"v1","addr":"172.21.15.97:6813","nonce":2478179311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":2478179311},{"type":"v1","addr":"172.21.15.97:6817","nonce":2478179311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2478179311},{"type":"v1","addr":"172.21.15.97:6815","nonce":2478179311}]},"public_addr":"172.21.15.97:6811/2478179311","cluster_addr":"172.21.15.97:6813/2478179311","heartbeat_back_addr":"172.21.15.97:6817/2478179311","heartbeat_front_addr":"172.21.15.97:6815/2478179311","state":["exists","up"]},{"osd":4,"uuid":"9245a3a4-7d49-4fdb-9d55-112e43177050","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":75755945},{"type":"v1","addr":"172.21.15.114:6809","nonce":75755945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":75755945},{"type":"v1","addr":"172.21.15.114:6811","nonce":75755945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":75755945},{"type":"v1","addr":"172.21.15.114:6815","nonce":75755945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":75755945},{"type":"v1","addr":"172.21.15.114:6813","nonce":75755945}]},"public_addr":"172.21.15.114:6809/75755945","cluster_addr":"172.21.15.114:6811/75755945","heartbeat_back_addr":"172.21.15.114:6815/75755945","heartbeat_front_addr":"172.21.15.114:6813/75755945","state":["exists","up"]},{"osd":5,"uuid":"51922f99-e8d3-461b-8615-e9bfdcbf5ed3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":1918493790},{"type":"v1","addr":"172.21.15.177:6809","nonce":1918493790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":1918493790},{"type":"v1","addr":"172.21.15.177:6811","nonce":1918493790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":1918493790},{"type":"v1","addr":"172.21.15.177:6815","nonce":1918493790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":1918493790},{"type":"v1","addr":"172.21.15.177:6813","nonce":1918493790}]},"public_addr":"172.21.15.177:6809/1918493790","cluster_addr":"172.21.15.177:6811/1918493790","heartbeat_back_addr":"172.21.15.177:6815/1918493790","heartbeat_front_addr":"172.21.15.177:6813/1918493790","state":["exists","up"]},{"osd":6,"uuid":"06dffb32-05eb-43b3-852e-915c5cc7f25a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2892389563},{"type":"v1","addr":"172.21.15.97:6819","nonce":2892389563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2892389563},{"type":"v1","addr":"172.21.15.97:6821","nonce":2892389563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2892389563},{"type":"v1","addr":"172.21.15.97:6825","nonce":2892389563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2892389563},{"type":"v1","addr":"172.21.15.97:6823","nonce":2892389563}]},"public_addr":"172.21.15.97:6819/2892389563","cluster_addr":"172.21.15.97:6821/2892389563","heartbeat_back_addr":"172.21.15.97:6825/2892389563","heartbeat_front_addr":"172.21.15.97:6823/2892389563","state":["exists","up"]},{"osd":7,"uuid":"e68e3519-9c1b-43c5-8e06-5385248b19df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":2445491153},{"type":"v1","addr":"172.21.15.114:6817","nonce":2445491153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":2445491153},{"type":"v1","addr":"172.21.15.114:6819","nonce":2445491153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2445491153},{"type":"v1","addr":"172.21.15.114:6823","nonce":2445491153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":2445491153},{"type":"v1","addr":"172.21.15.114:6821","nonce":2445491153}]},"public_addr":"172.21.15.114:6817/2445491153","cluster_addr":"172.21.15.114:6819/2445491153","heartbeat_back_addr":"172.21.15.114:6823/2445491153","heartbeat_front_addr":"172.21.15.114:6821/2445491153","state":["exists","up"]},{"osd":8,"uuid":"31ac592e-1349-49ee-9af9-8ad85d682978","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":2051134191},{"type":"v1","addr":"172.21.15.177:6817","nonce":2051134191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":2051134191},{"type":"v1","addr":"172.21.15.177:6819","nonce":2051134191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":2051134191},{"type":"v1","addr":"172.21.15.177:6823","nonce":2051134191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":2051134191},{"type":"v1","addr":"172.21.15.177:6821","nonce":2051134191}]},"public_addr":"172.21.15.177:6817/2051134191","cluster_addr":"172.21.15.177:6819/2051134191","heartbeat_back_addr":"172.21.15.177:6823/2051134191","heartbeat_front_addr":"172.21.15.177:6821/2051134191","state":["exists","up"]},{"osd":9,"uuid":"0171086a-dd41-4eb4-a389-16f0c46a64db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":1578881271},{"type":"v1","addr":"172.21.15.97:6827","nonce":1578881271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1578881271},{"type":"v1","addr":"172.21.15.97:6829","nonce":1578881271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1578881271},{"type":"v1","addr":"172.21.15.97:6833","nonce":1578881271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1578881271},{"type":"v1","addr":"172.21.15.97:6831","nonce":1578881271}]},"public_addr":"172.21.15.97:6827/1578881271","cluster_addr":"172.21.15.97:6829/1578881271","heartbeat_back_addr":"172.21.15.97:6833/1578881271","heartbeat_front_addr":"172.21.15.97:6831/1578881271","state":["exists","up"]},{"osd":10,"uuid":"64754e31-b124-4601-ad98-2f70b1dd5359","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":2893111461},{"type":"v1","addr":"172.21.15.114:6825","nonce":2893111461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":2893111461},{"type":"v1","addr":"172.21.15.114:6827","nonce":2893111461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2893111461},{"type":"v1","addr":"172.21.15.114:6831","nonce":2893111461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":2893111461},{"type":"v1","addr":"172.21.15.114:6829","nonce":2893111461}]},"public_addr":"172.21.15.114:6825/2893111461","cluster_addr":"172.21.15.114:6827/2893111461","heartbeat_back_addr":"172.21.15.114:6831/2893111461","heartbeat_front_addr":"172.21.15.114:6829/2893111461","state":["exists","up"]},{"osd":11,"uuid":"9ffc752c-adf7-4037-89d3-535787d7475d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":85,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":2333030391},{"type":"v1","addr":"172.21.15.177:6825","nonce":2333030391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":2333030391},{"type":"v1","addr":"172.21.15.177:6827","nonce":2333030391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":2333030391},{"type":"v1","addr":"172.21.15.177:6831","nonce":2333030391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":2333030391},{"type":"v1","addr":"172.21.15.177:6829","nonce":2333030391}]},"public_addr":"172.21.15.177:6825/2333030391","cluster_addr":"172.21.15.177:6827/2333030391","heartbeat_back_addr":"172.21.15.177:6831/2333030391","heartbeat_front_addr":"172.21.15.177:6829/2333030391","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:46:45.143519+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:47:18.656947+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:47:51.316345+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:48:24.575512+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:48:59.113555+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:49:32.103865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:50:06.615539+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:50:41.578341+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:51:15.254011+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:51:49.099165+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:52:24.544821+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:52:58.838484+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.28","mappings":[{"from":3,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/359646":"2024-03-28T10:44:39.889550+0000","172.21.15.97:6801/98682155":"2024-03-28T10:44:06.019075+0000","172.21.15.97:6801/1685921853":"2024-03-28T10:44:39.889550+0000","172.21.15.97:6800/98682155":"2024-03-28T10:44:06.019075+0000","172.21.15.97:0/107690680":"2024-03-28T10:44:06.019075+0000","172.21.15.97:6800/1685921853":"2024-03-28T10:44:39.889550+0000","172.21.15.97:0/437043413":"2024-03-28T10:44:39.889550+0000","172.21.15.97:0/2319706992":"2024-03-28T10:44:39.889550+0000","172.21.15.97:0/3911532654":"2024-03-28T10:44:06.019075+0000","172.21.15.97:0/3347321845":"2024-03-28T10:44:06.019075+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-27T10:55:46.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.060+0000 7f64c2200640 1 -- 172.21.15.97:0/4239863144 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f64a40a3610 msgr2=0x7f64a40a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:46.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.060+0000 7f64c2200640 1 --2- 172.21.15.97:0/4239863144 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f64a40a3610 0x7f64a40a5ad0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f64b0005fd0 tx=0x7f64b0005850 comp rx=0 tx=0).stop 2024-03-27T10:55:46.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.060+0000 7f64c2200640 1 -- 172.21.15.97:0/4239863144 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64bc109180 msgr2=0x7f64bc19e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:46.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.060+0000 7f64c2200640 1 --2- 172.21.15.97:0/4239863144 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64bc109180 0x7f64bc19e690 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f64a800d770 tx=0x7f64a800dc30 comp rx=0 tx=0).stop 2024-03-27T10:55:46.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.060+0000 7f64c2200640 1 -- 172.21.15.97:0/4239863144 shutdown_connections 2024-03-27T10:55:46.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.061+0000 7f64c2200640 1 --2- 172.21.15.97:0/4239863144 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f64a40a3610 0x7f64a40a5ad0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:46.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.061+0000 7f64c2200640 1 --2- 172.21.15.97:0/4239863144 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f64bc109b40 0x7f64bc1a5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:46.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.061+0000 7f64c2200640 1 --2- 172.21.15.97:0/4239863144 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64bc109180 0x7f64bc19e690 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:46.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.061+0000 7f64c2200640 1 --2- 172.21.15.97:0/4239863144 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f64bc107f90 0x7f64bc19e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:46.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.061+0000 7f64c2200640 1 -- 172.21.15.97:0/4239863144 >> 172.21.15.97:0/4239863144 conn(0x7f64bc076f20 msgr2=0x7f64bc10c480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:55:46.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.061+0000 7f64c2200640 1 -- 172.21.15.97:0/4239863144 shutdown_connections 2024-03-27T10:55:46.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.061+0000 7f64c2200640 1 -- 172.21.15.97:0/4239863144 wait complete. 2024-03-27T10:55:46.051 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-27T10:55:46.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.269+0000 7f3aea200640 1 -- 172.21.15.97:0/2718539102 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ae4102d40 msgr2=0x7f3ae41031c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:46.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.269+0000 7f3aea200640 1 --2- 172.21.15.97:0/2718539102 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ae4102d40 0x7f3ae41031c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f3ad0009900 tx=0x7f3ad002bc30 comp rx=0 tx=0).stop 2024-03-27T10:55:46.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.269+0000 7f3aea200640 1 -- 172.21.15.97:0/2718539102 shutdown_connections 2024-03-27T10:55:46.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.269+0000 7f3aea200640 1 --2- 172.21.15.97:0/2718539102 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3ae4103700 0x7f3ae4109f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:46.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.269+0000 7f3aea200640 1 --2- 172.21.15.97:0/2718539102 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ae4102d40 0x7f3ae41031c0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:46.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.269+0000 7f3aea200640 1 --2- 172.21.15.97:0/2718539102 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3ae4101b50 0x7f3ae4101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:46.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.269+0000 7f3aea200640 1 -- 172.21.15.97:0/2718539102 >> 172.21.15.97:0/2718539102 conn(0x7f3ae40fd0c0 msgr2=0x7f3ae40ff500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:55:46.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.270+0000 7f3aea200640 1 -- 172.21.15.97:0/2718539102 shutdown_connections 2024-03-27T10:55:46.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.270+0000 7f3aea200640 1 -- 172.21.15.97:0/2718539102 wait complete. 2024-03-27T10:55:46.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.270+0000 7f3aea200640 1 Processor -- start 2024-03-27T10:55:46.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.271+0000 7f3aea200640 1 -- start start 2024-03-27T10:55:46.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.271+0000 7f3aea200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3ae4101b50 0x7f3ae4075900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:46.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.271+0000 7f3aea200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ae4102d40 0x7f3ae4073f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:46.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.271+0000 7f3aea200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3ae4103700 0x7f3ae4074480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:46.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.271+0000 7f3aea200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae410d660 con 0x7f3ae4102d40 2024-03-27T10:55:46.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.271+0000 7f3aea200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae410d4e0 con 0x7f3ae4101b50 2024-03-27T10:55:46.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.271+0000 7f3aea200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae410d7e0 con 0x7f3ae4103700 2024-03-27T10:55:46.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.271+0000 7f3ae2a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ae4102d40 0x7f3ae4073f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:46.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.271+0000 7f3ae2a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ae4102d40 0x7f3ae4073f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40770/0 (socket says 172.21.15.97:40770) 2024-03-27T10:55:46.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.271+0000 7f3ae2a00640 1 -- 172.21.15.97:0/1144338998 learned_addr learned my addr 172.21.15.97:0/1144338998 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:55:46.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.271+0000 7f3ae3400640 1 --2- 172.21.15.97:0/1144338998 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3ae4101b50 0x7f3ae4075900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:46.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.271+0000 7f3ae3e00640 1 --2- 172.21.15.97:0/1144338998 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3ae4103700 0x7f3ae4074480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:46.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.271+0000 7f3ae2a00640 1 -- 172.21.15.97:0/1144338998 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3ae4103700 msgr2=0x7f3ae4074480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:46.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.271+0000 7f3ae2a00640 1 --2- 172.21.15.97:0/1144338998 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3ae4103700 0x7f3ae4074480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:46.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.271+0000 7f3ae2a00640 1 -- 172.21.15.97:0/1144338998 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3ae4101b50 msgr2=0x7f3ae4075900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:46.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.271+0000 7f3ae2a00640 1 --2- 172.21.15.97:0/1144338998 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3ae4101b50 0x7f3ae4075900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:46.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.271+0000 7f3ae2a00640 1 -- 172.21.15.97:0/1144338998 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ae4074d10 con 0x7f3ae4102d40 2024-03-27T10:55:46.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.271+0000 7f3ae3e00640 1 --2- 172.21.15.97:0/1144338998 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3ae4103700 0x7f3ae4074480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:55:46.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.272+0000 7f3ae2a00640 1 --2- 172.21.15.97:0/1144338998 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ae4102d40 0x7f3ae4073f40 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f3ad0006010 tx=0x7f3ad00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:55:46.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.272+0000 7f3adbe00640 1 -- 172.21.15.97:0/1144338998 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad0043020 con 0x7f3ae4102d40 2024-03-27T10:55:46.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.272+0000 7f3adbe00640 1 -- 172.21.15.97:0/1144338998 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3ad0004aa0 con 0x7f3ae4102d40 2024-03-27T10:55:46.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.272+0000 7f3adbe00640 1 -- 172.21.15.97:0/1144338998 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad0003c60 con 0x7f3ae4102d40 2024-03-27T10:55:46.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.272+0000 7f3aea200640 1 -- 172.21.15.97:0/1144338998 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ae41a1ea0 con 0x7f3ae4102d40 2024-03-27T10:55:46.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.272+0000 7f3aea200640 1 -- 172.21.15.97:0/1144338998 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ae41a22b0 con 0x7f3ae4102d40 2024-03-27T10:55:46.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.273+0000 7f3aea200640 1 -- 172.21.15.97:0/1144338998 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ae4108f40 con 0x7f3ae4102d40 2024-03-27T10:55:46.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.276+0000 7f3adbe00640 1 -- 172.21.15.97:0/1144338998 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f3ad0046020 con 0x7f3ae4102d40 2024-03-27T10:55:46.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.277+0000 7f3adbe00640 1 --2- 172.21.15.97:0/1144338998 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f3ac80a3690 0x7f3ac80a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:46.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.277+0000 7f3adbe00640 1 -- 172.21.15.97:0/1144338998 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f3ad00f2030 con 0x7f3ae4102d40 2024-03-27T10:55:46.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.277+0000 7f3ae3400640 1 --2- 172.21.15.97:0/1144338998 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f3ac80a3690 0x7f3ac80a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:46.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.278+0000 7f3ae3400640 1 --2- 172.21.15.97:0/1144338998 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f3ac80a3690 0x7f3ac80a5b50 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f3ae4102ba0 tx=0x7f3ad400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:55:46.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.281+0000 7f3adbe00640 1 -- 172.21.15.97:0/1144338998 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f3ad004d020 con 0x7f3ae4102d40 2024-03-27T10:55:46.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.508+0000 7f3aea200640 1 -- 172.21.15.97:0/1144338998 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3ae402d370 con 0x7f3ae4102d40 2024-03-27T10:55:46.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.510+0000 7f3adbe00640 1 -- 172.21.15.97:0/1144338998 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11944 (secure 0 0 0) 0x7f3ad00bec10 con 0x7f3ae4102d40 2024-03-27T10:55:46.481 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T10:55:46.482 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1009423904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1009423904},{"type":"v1","addr":"172.21.15.97:6835","nonce":1009423904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3251173433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3251173433},{"type":"v1","addr":"172.21.15.97:6837","nonce":3251173433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-27T10:54:04.510178+0000","modified":"2024-03-27T10:54:12.180047+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34508,"mds_1":34478,"mds_2":34445,"mds_3":24454,"mds_4":24400},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14646":{"gid":14646,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6839/1057596821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1057596821},{"type":"v1","addr":"172.21.15.97:6839","nonce":1057596821}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14658":{"gid":14658,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6841/1672464856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6840","nonce":1672464856},{"type":"v1","addr":"172.21.15.97:6841","nonce":1672464856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.177:6833/3787981632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3787981632},{"type":"v1","addr":"172.21.15.177:6833","nonce":3787981632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.177:6835/3349112132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":3349112132},{"type":"v1","addr":"172.21.15.177:6835","nonce":3349112132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6837/3244432640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":3244432640},{"type":"v1","addr":"172.21.15.177:6837","nonce":3244432640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6839/3340454605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":3340454605},{"type":"v1","addr":"172.21.15.177:6839","nonce":3340454605}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34415":{"gid":34415,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6833/4052517902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":4052517902},{"type":"v1","addr":"172.21.15.114:6833","nonce":4052517902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34445":{"gid":34445,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.114:6835/3581918222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3581918222},{"type":"v1","addr":"172.21.15.114:6835","nonce":3581918222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34478":{"gid":34478,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.114:6837/616775588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":616775588},{"type":"v1","addr":"172.21.15.114:6837","nonce":616775588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34508":{"gid":34508,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.114:6839/3937193784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":3937193784},{"type":"v1","addr":"172.21.15.114:6839","nonce":3937193784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34508,"qdb_cluster":[34445,34508,24400,24454,34478]},"id":1}]} 2024-03-27T10:55:46.482 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 27 2024-03-27T10:55:46.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.512+0000 7f3aea200640 1 -- 172.21.15.97:0/1144338998 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f3ac80a3690 msgr2=0x7f3ac80a5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:46.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.512+0000 7f3aea200640 1 --2- 172.21.15.97:0/1144338998 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f3ac80a3690 0x7f3ac80a5b50 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f3ae4102ba0 tx=0x7f3ad400b390 comp rx=0 tx=0).stop 2024-03-27T10:55:46.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.512+0000 7f3aea200640 1 -- 172.21.15.97:0/1144338998 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ae4102d40 msgr2=0x7f3ae4073f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:46.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.512+0000 7f3aea200640 1 --2- 172.21.15.97:0/1144338998 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ae4102d40 0x7f3ae4073f40 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f3ad0006010 tx=0x7f3ad00042f0 comp rx=0 tx=0).stop 2024-03-27T10:55:46.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.512+0000 7f3aea200640 1 -- 172.21.15.97:0/1144338998 shutdown_connections 2024-03-27T10:55:46.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.512+0000 7f3aea200640 1 --2- 172.21.15.97:0/1144338998 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3ae4103700 0x7f3ae4074480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:46.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.512+0000 7f3aea200640 1 --2- 172.21.15.97:0/1144338998 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ae4102d40 0x7f3ae4073f40 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:46.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.512+0000 7f3aea200640 1 --2- 172.21.15.97:0/1144338998 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f3ac80a3690 0x7f3ac80a5b50 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:46.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.512+0000 7f3aea200640 1 --2- 172.21.15.97:0/1144338998 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3ae4101b50 0x7f3ae4075900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:46.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.512+0000 7f3aea200640 1 -- 172.21.15.97:0/1144338998 >> 172.21.15.97:0/1144338998 conn(0x7f3ae40fd0c0 msgr2=0x7f3ae4106110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:55:46.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.513+0000 7f3aea200640 1 -- 172.21.15.97:0/1144338998 shutdown_connections 2024-03-27T10:55:46.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.513+0000 7f3aea200640 1 -- 172.21.15.97:0/1144338998 wait complete. 2024-03-27T10:55:46.501 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-03-27T10:54:04.510178+0000', 'modified': '2024-03-27T10:54:12.180047+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 34508, 'mds_1': 34478, 'mds_2': 34445, 'mds_3': 24454, 'mds_4': 24400}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14646': {'gid': 14646, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.97:6839/1057596821', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6838', 'nonce': 1057596821}, {'type': 'v1', 'addr': '172.21.15.97:6839', 'nonce': 1057596821}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14658': {'gid': 14658, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.97:6841/1672464856', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6840', 'nonce': 1672464856}, {'type': 'v1', 'addr': '172.21.15.97:6841', 'nonce': 1672464856}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24400': {'gid': 24400, 'name': 'l', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 15, 'addr': '172.21.15.177:6833/3787981632', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6832', 'nonce': 3787981632}, {'type': 'v1', 'addr': '172.21.15.177:6833', 'nonce': 3787981632}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24454': {'gid': 24454, 'name': 'f', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.177:6835/3349112132', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6834', 'nonce': 3349112132}, {'type': 'v1', 'addr': '172.21.15.177:6835', 'nonce': 3349112132}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24466': {'gid': 24466, 'name': 'c', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.177:6837/3244432640', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6836', 'nonce': 3244432640}, {'type': 'v1', 'addr': '172.21.15.177:6837', 'nonce': 3244432640}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24487': {'gid': 24487, 'name': 'i', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.177:6839/3340454605', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6838', 'nonce': 3340454605}, {'type': 'v1', 'addr': '172.21.15.177:6839', 'nonce': 3340454605}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34415': {'gid': 34415, 'name': 'k', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.114:6833/4052517902', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6832', 'nonce': 4052517902}, {'type': 'v1', 'addr': '172.21.15.114:6833', 'nonce': 4052517902}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34445': {'gid': 34445, 'name': 'e', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.114:6835/3581918222', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6834', 'nonce': 3581918222}, {'type': 'v1', 'addr': '172.21.15.114:6835', 'nonce': 3581918222}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34478': {'gid': 34478, 'name': 'h', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.114:6837/616775588', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6836', 'nonce': 616775588}, {'type': 'v1', 'addr': '172.21.15.114:6837', 'nonce': 616775588}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34508': {'gid': 34508, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.114:6839/3937193784', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6838', 'nonce': 3937193784}, {'type': 'v1', 'addr': '172.21.15.114:6839', 'nonce': 3937193784}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 34508, 'qdb_cluster': [34445, 34508, 24400, 24454, 34478]} 2024-03-27T10:55:46.501 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-03-27T10:55:46.501 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.l status 2024-03-27T10:55:46.575 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:55:46 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/2680180006' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T10:55:46.575 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:55:46 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/4239863144' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T10:55:46.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:55:46 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/2680180006' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T10:55:46.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:55:46 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/4239863144' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T10:55:46.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.734+0000 7f8b52600640 1 -- 172.21.15.97:0/4210871626 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b4c103710 msgr2=0x7f8b4c109fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:46.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.734+0000 7f8b52600640 1 --2- 172.21.15.97:0/4210871626 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b4c103710 0x7f8b4c109fa0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f8b44009900 tx=0x7f8b4402bc30 comp rx=0 tx=0).stop 2024-03-27T10:55:46.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.735+0000 7f8b52600640 1 -- 172.21.15.97:0/4210871626 shutdown_connections 2024-03-27T10:55:46.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.735+0000 7f8b52600640 1 --2- 172.21.15.97:0/4210871626 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b4c103710 0x7f8b4c109fa0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:46.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.735+0000 7f8b52600640 1 --2- 172.21.15.97:0/4210871626 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8b4c102d50 0x7f8b4c1031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:46.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.735+0000 7f8b52600640 1 --2- 172.21.15.97:0/4210871626 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8b4c101b60 0x7f8b4c101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:46.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.735+0000 7f8b52600640 1 -- 172.21.15.97:0/4210871626 >> 172.21.15.97:0/4210871626 conn(0x7f8b4c0fd0f0 msgr2=0x7f8b4c0ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:55:46.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.735+0000 7f8b52600640 1 -- 172.21.15.97:0/4210871626 shutdown_connections 2024-03-27T10:55:46.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.735+0000 7f8b52600640 1 -- 172.21.15.97:0/4210871626 wait complete. 2024-03-27T10:55:46.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.736+0000 7f8b52600640 1 Processor -- start 2024-03-27T10:55:46.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.736+0000 7f8b52600640 1 -- start start 2024-03-27T10:55:46.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.736+0000 7f8b52600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8b4c101b60 0x7f8b4c199f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:46.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.736+0000 7f8b52600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8b4c102d50 0x7f8b4c19a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:46.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.736+0000 7f8b52600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b4c103710 0x7f8b4c1a1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:46.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.736+0000 7f8b52600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8b4c10d6e0 con 0x7f8b4c103710 2024-03-27T10:55:46.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.736+0000 7f8b52600640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f8b4c10d560 con 0x7f8b4c101b60 2024-03-27T10:55:46.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.736+0000 7f8b52600640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f8b4c10d860 con 0x7f8b4c102d50 2024-03-27T10:55:46.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.736+0000 7f8b4be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b4c103710 0x7f8b4c1a1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:46.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.736+0000 7f8b4be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b4c103710 0x7f8b4c1a1520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40792/0 (socket says 172.21.15.97:40792) 2024-03-27T10:55:46.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.736+0000 7f8b4be00640 1 -- 172.21.15.97:0/1279832180 learned_addr learned my addr 172.21.15.97:0/1279832180 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:55:46.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.736+0000 7f8b4b400640 1 --2- 172.21.15.97:0/1279832180 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8b4c101b60 0x7f8b4c199f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:46.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.737+0000 7f8b4aa00640 1 --2- 172.21.15.97:0/1279832180 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8b4c102d50 0x7f8b4c19a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:46.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.737+0000 7f8b4be00640 1 -- 172.21.15.97:0/1279832180 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8b4c102d50 msgr2=0x7f8b4c19a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:46.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.737+0000 7f8b4be00640 1 --2- 172.21.15.97:0/1279832180 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8b4c102d50 0x7f8b4c19a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:46.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.737+0000 7f8b4be00640 1 -- 172.21.15.97:0/1279832180 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8b4c101b60 msgr2=0x7f8b4c199f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:46.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.737+0000 7f8b4be00640 1 --2- 172.21.15.97:0/1279832180 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8b4c101b60 0x7f8b4c199f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:46.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.737+0000 7f8b4be00640 1 -- 172.21.15.97:0/1279832180 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b4c1a1b90 con 0x7f8b4c103710 2024-03-27T10:55:46.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.737+0000 7f8b4be00640 1 --2- 172.21.15.97:0/1279832180 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b4c103710 0x7f8b4c1a1520 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f8b44036040 tx=0x7f8b4402cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:55:46.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.737+0000 7f8b43e00640 1 -- 172.21.15.97:0/1279832180 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b44043020 con 0x7f8b4c103710 2024-03-27T10:55:46.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.737+0000 7f8b43e00640 1 -- 172.21.15.97:0/1279832180 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8b440044c0 con 0x7f8b4c103710 2024-03-27T10:55:46.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.737+0000 7f8b43e00640 1 -- 172.21.15.97:0/1279832180 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b44003bf0 con 0x7f8b4c103710 2024-03-27T10:55:46.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.737+0000 7f8b52600640 1 -- 172.21.15.97:0/1279832180 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b4c1a1e20 con 0x7f8b4c103710 2024-03-27T10:55:46.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.737+0000 7f8b52600640 1 -- 172.21.15.97:0/1279832180 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b4c1a2330 con 0x7f8b4c103710 2024-03-27T10:55:46.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.741+0000 7f8b553b2740 1 Processor -- start 2024-03-27T10:55:46.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.742+0000 7f8b43e00640 1 -- 172.21.15.97:0/1279832180 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f8b44046020 con 0x7f8b4c103710 2024-03-27T10:55:46.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.742+0000 7f8b553b2740 1 -- start start 2024-03-27T10:55:46.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.742+0000 7f8b553b2740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558d15a00f20 0x558d15a01340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:46.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.742+0000 7f8b553b2740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558d15a01910 0x558d15a06410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:46.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.742+0000 7f8b553b2740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558d15a06a80 0x558d15a08f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:46.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.742+0000 7f8b553b2740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x558d159c3920 con 0x558d15a01910 2024-03-27T10:55:46.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.742+0000 7f8b553b2740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x558d15913fa0 con 0x558d15a06a80 2024-03-27T10:55:46.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.742+0000 7f8b553b2740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x558d15965f90 con 0x558d15a00f20 2024-03-27T10:55:46.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.742+0000 7f8b4b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558d15a01910 0x558d15a06410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:46.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.742+0000 7f8b4b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558d15a01910 0x558d15a06410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40808/0 (socket says 172.21.15.97:40808) 2024-03-27T10:55:46.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.742+0000 7f8b4b400640 1 -- 172.21.15.97:0/4155095458 learned_addr learned my addr 172.21.15.97:0/4155095458 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:55:46.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.743+0000 7f8b4aa00640 1 --2- 172.21.15.97:0/4155095458 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558d15a06a80 0x558d15a08f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:46.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.743+0000 7f8b4aa00640 1 --2- 172.21.15.97:0/4155095458 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558d15a00f20 0x558d15a01340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:46.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.743+0000 7f8b4b400640 1 -- 172.21.15.97:0/4155095458 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558d15a00f20 msgr2=0x558d15a01340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:55:46.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.743+0000 7f8b4b400640 1 --2- 172.21.15.97:0/4155095458 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558d15a00f20 0x558d15a01340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:46.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.743+0000 7f8b4b400640 1 -- 172.21.15.97:0/4155095458 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558d15a06a80 msgr2=0x558d15a08f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:55:46.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.743+0000 7f8b4b400640 1 --2- 172.21.15.97:0/4155095458 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558d15a06a80 0x558d15a08f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:46.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.743+0000 7f8b4b400640 1 -- 172.21.15.97:0/4155095458 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d1567a2d0 con 0x558d15a01910 2024-03-27T10:55:46.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.743+0000 7f8b4aa00640 1 --2- 172.21.15.97:0/4155095458 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558d15a06a80 0x558d15a08f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T10:55:46.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.743+0000 7f8b43e00640 1 --2- 172.21.15.97:0/1279832180 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f8b2c0a3690 0x7f8b2c0a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:46.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.743+0000 7f8b4b400640 1 --2- 172.21.15.97:0/4155095458 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558d15a01910 0x558d15a06410 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c009ca0 tx=0x7f8b3c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0f5a12b587d1f2 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:55:46.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.743+0000 7f8b43e00640 1 -- 172.21.15.97:0/1279832180 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f8b440f1fe0 con 0x7f8b4c103710 2024-03-27T10:55:46.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.743+0000 7f8b4aa00640 1 --2- 172.21.15.97:0/1279832180 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f8b2c0a3690 0x7f8b2c0a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:46.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.743+0000 7f8b40c00640 1 -- 172.21.15.97:0/4155095458 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b3c004220 con 0x558d15a01910 2024-03-27T10:55:46.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.743+0000 7f8b40c00640 1 -- 172.21.15.97:0/4155095458 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8b3c0043c0 con 0x558d15a01910 2024-03-27T10:55:46.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.743+0000 7f8b40c00640 1 -- 172.21.15.97:0/4155095458 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b3c0198b0 con 0x558d15a01910 2024-03-27T10:55:46.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.744+0000 7f8b553b2740 1 -- 172.21.15.97:0/4155095458 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558d15a01910 msgr2=0x558d15a06410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:46.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.744+0000 7f8b553b2740 1 --2- 172.21.15.97:0/4155095458 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558d15a01910 0x558d15a06410 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c009ca0 tx=0x7f8b3c011810 comp rx=0 tx=0).stop 2024-03-27T10:55:46.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.744+0000 7f8b4aa00640 1 --2- 172.21.15.97:0/1279832180 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f8b2c0a3690 0x7f8b2c0a5b50 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8b34009fd0 tx=0x7f8b34014040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:55:46.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.744+0000 7f8b553b2740 1 -- 172.21.15.97:0/4155095458 shutdown_connections 2024-03-27T10:55:46.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.744+0000 7f8b553b2740 1 --2- 172.21.15.97:0/4155095458 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558d15a06a80 0x558d15a08f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:46.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.744+0000 7f8b553b2740 1 --2- 172.21.15.97:0/4155095458 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558d15a01910 0x558d15a06410 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:46.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.744+0000 7f8b553b2740 1 --2- 172.21.15.97:0/4155095458 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558d15a00f20 0x558d15a01340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:46.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.744+0000 7f8b553b2740 1 -- 172.21.15.97:0/4155095458 >> 172.21.15.97:0/4155095458 conn(0x558d159fc490 msgr2=0x558d159fc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:55:46.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.744+0000 7f8b553b2740 1 -- 172.21.15.97:0/4155095458 shutdown_connections 2024-03-27T10:55:46.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.744+0000 7f8b553b2740 1 -- 172.21.15.97:0/4155095458 wait complete. 2024-03-27T10:55:46.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.745+0000 7f8b553b2740 1 -- start start 2024-03-27T10:55:46.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.745+0000 7f8b553b2740 1 Processor -- start 2024-03-27T10:55:46.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.745+0000 7f8b553b2740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558d15a00f20 0x558d15a0b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:46.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.745+0000 7f8b553b2740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558d15a01910 0x558d15a0b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:46.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.745+0000 7f8b553b2740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558d15a06a80 0x558d15a126b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:46.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.745+0000 7f8b553b2740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x558d15961f40 con 0x558d15a00f20 2024-03-27T10:55:46.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.745+0000 7f8b553b2740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x558d1590e650 con 0x558d15a01910 2024-03-27T10:55:46.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.745+0000 7f8b553b2740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x558d15921a10 con 0x558d15a06a80 2024-03-27T10:55:46.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.746+0000 7f8b553b2740 10 client.0 _ll_register_callbacks cb 0x558d15960ff0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:55:46.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.746+0000 7f8b4b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558d15a00f20 0x558d15a0b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:46.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.746+0000 7f8b4b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558d15a00f20 0x558d15a0b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40816/0 (socket says 172.21.15.97:40816) 2024-03-27T10:55:46.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.746+0000 7f8b4b400640 1 -- 172.21.15.97:0/2632697465 learned_addr learned my addr 172.21.15.97:0/2632697465 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:55:46.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.746+0000 7f8b4aa00640 1 --2- 172.21.15.97:0/2632697465 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558d15a01910 0x558d15a0b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:46.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.746+0000 7f8b4be00640 1 --2- 172.21.15.97:0/2632697465 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558d15a06a80 0x558d15a126b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:46.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.746+0000 7f8b4b400640 1 -- 172.21.15.97:0/2632697465 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558d15a06a80 msgr2=0x558d15a126b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:55:46.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.746+0000 7f8b4b400640 1 --2- 172.21.15.97:0/2632697465 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558d15a06a80 0x558d15a126b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:46.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.746+0000 7f8b4b400640 1 -- 172.21.15.97:0/2632697465 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558d15a01910 msgr2=0x558d15a0b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:55:46.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.746+0000 7f8b4b400640 1 --2- 172.21.15.97:0/2632697465 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558d15a01910 0x558d15a0b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:46.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.746+0000 7f8b4b400640 1 -- 172.21.15.97:0/2632697465 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d157446b0 con 0x558d15a00f20 2024-03-27T10:55:46.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.746+0000 7f8b4b400640 1 --2- 172.21.15.97:0/2632697465 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558d15a00f20 0x558d15a0b1a0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c01b040 tx=0x7f8b3c003800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca70b6fbb23af757 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:55:46.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.746+0000 7f8b39600640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T10:55:46.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.747+0000 7f8b39600640 1 -- 172.21.15.97:0/2632697465 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b3c028070 con 0x558d15a00f20 2024-03-27T10:55:46.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.747+0000 7f8b39600640 1 -- 172.21.15.97:0/2632697465 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f8b3c005040 con 0x558d15a00f20 2024-03-27T10:55:46.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.747+0000 7f8b553b2740 1 -- 172.21.15.97:0/2632697465 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b3c0043c0 con 0x558d15a00f20 2024-03-27T10:55:46.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.747+0000 7f8b39600640 1 -- 172.21.15.97:0/2632697465 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8b3c034050 con 0x558d15a00f20 2024-03-27T10:55:46.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.747+0000 7f8b553b2740 10 client.15075 fetch_fsmap learned FSMap version 27 2024-03-27T10:55:46.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.747+0000 7f8b553b2740 1 -- 172.21.15.97:0/2632697465 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558d15a15a20 con 0x558d15a00f20 2024-03-27T10:55:46.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.747+0000 7f8b39600640 1 -- 172.21.15.97:0/2632697465 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8b3c0043c0 con 0x558d15a00f20 2024-03-27T10:55:46.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.747+0000 7f8b39600640 1 -- 172.21.15.97:0/2632697465 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f8b3c004b70 con 0x558d15a00f20 2024-03-27T10:55:46.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.747+0000 7f8b553b2740 10 client.15075 fetch_fsmap finished waiting for FSMap version 27 2024-03-27T10:55:46.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.748+0000 7f8b553b2740 10 client.15075 resolve_mds: resolved name 'l' to daemon mds.l 2024-03-27T10:55:46.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.748+0000 7f8b553b2740 20 client.15075 populate_metadata read hostname 'smithi097' 2024-03-27T10:55:46.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.748+0000 7f8b553b2740 1 --2- 172.21.15.97:0/2632697465 >> [v2:172.21.15.177:6832/3787981632,v1:172.21.15.177:6833/3787981632] conn(0x558d15a17e90 0x558d15a19300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:46.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.748+0000 7f8b553b2740 4 client.15075 mds_command: new command op to 24400 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T10:55:46.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.748+0000 7f8b553b2740 1 -- 172.21.15.97:0/2632697465 --> [v2:172.21.15.177:6832/3787981632,v1:172.21.15.177:6833/3787981632] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558d159734d0 con 0x558d15a17e90 2024-03-27T10:55:46.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.748+0000 7f8b4aa00640 1 --2- 172.21.15.97:0/2632697465 >> [v2:172.21.15.177:6832/3787981632,v1:172.21.15.177:6833/3787981632] conn(0x558d15a17e90 0x558d15a19300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:46.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.749+0000 7f8b4aa00640 1 --2- 172.21.15.97:0/2632697465 >> [v2:172.21.15.177:6832/3787981632,v1:172.21.15.177:6833/3787981632] conn(0x558d15a17e90 0x558d15a19300 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=330733216f3fa3d3 server_cookie=d030f6f03c87976a in_seq=0 out_seq=0 2024-03-27T10:55:46.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.749+0000 7f8b39600640 10 client.15075 ms_handle_connect on v2:172.21.15.177:6832/3787981632 2024-03-27T10:55:46.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.752+0000 7f8b39600640 1 -- 172.21.15.97:0/2632697465 <== mds.4 v2:172.21.15.177:6832/3787981632 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x558d159734d0 con 0x558d15a17e90 2024-03-27T10:55:46.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.752+0000 7f8b39600640 10 client.15075 handle_command_reply: tid=0 2024-03-27T10:55:46.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.752+0000 7f8b553b2740 1 client.15075 shutdown 2024-03-27T10:55:46.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.752+0000 7f8b553b2740 1 -- 172.21.15.97:0/2632697465 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558d15a00f20 msgr2=0x558d15a0b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:46.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.752+0000 7f8b553b2740 1 --2- 172.21.15.97:0/2632697465 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558d15a00f20 0x558d15a0b1a0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c01b040 tx=0x7f8b3c003800 comp rx=0 tx=0).stop 2024-03-27T10:55:46.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.752+0000 7f8b553b2740 1 -- 172.21.15.97:0/2632697465 shutdown_connections 2024-03-27T10:55:46.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.753+0000 7f8b553b2740 1 --2- 172.21.15.97:0/2632697465 >> [v2:172.21.15.177:6832/3787981632,v1:172.21.15.177:6833/3787981632] conn(0x558d15a17e90 0x558d15a19300 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:46.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.753+0000 7f8b553b2740 1 --2- 172.21.15.97:0/2632697465 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558d15a06a80 0x558d15a126b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:46.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.753+0000 7f8b39600640 0 client.15075 ms_handle_reset on v2:172.21.15.177:6832/3787981632 2024-03-27T10:55:46.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.753+0000 7f8b553b2740 1 --2- 172.21.15.97:0/2632697465 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558d15a01910 0x558d15a0b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:46.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.753+0000 7f8b553b2740 1 --2- 172.21.15.97:0/2632697465 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558d15a00f20 0x558d15a0b1a0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:46.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.753+0000 7f8b553b2740 1 -- 172.21.15.97:0/2632697465 >> 172.21.15.97:0/2632697465 conn(0x558d159fc490 msgr2=0x558d159fb690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:55:46.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.753+0000 7f8b553b2740 1 -- 172.21.15.97:0/2632697465 shutdown_connections 2024-03-27T10:55:46.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.753+0000 7f8b553b2740 1 -- 172.21.15.97:0/2632697465 wait complete. 2024-03-27T10:55:46.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.753+0000 7f8b553b2740 20 client.15075 trim_cache size 0 max 16384 2024-03-27T10:55:46.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.775+0000 7f8b553b2740 1 Processor -- start 2024-03-27T10:55:46.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.775+0000 7f8b553b2740 1 -- start start 2024-03-27T10:55:46.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.775+0000 7f8b553b2740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558d15a00f20 0x558d15a09cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:46.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.775+0000 7f8b553b2740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558d15a01910 0x558d15a12670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:46.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.776+0000 7f8b553b2740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558d15a06a80 0x558d15a16e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:46.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.776+0000 7f8b553b2740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x558d15a17640 con 0x558d15a00f20 2024-03-27T10:55:46.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.776+0000 7f8b553b2740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x558d15a17d10 con 0x558d15a01910 2024-03-27T10:55:46.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.776+0000 7f8b553b2740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x558d15a183e0 con 0x558d15a06a80 2024-03-27T10:55:46.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.776+0000 7f8b4b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558d15a00f20 0x558d15a09cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:46.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.776+0000 7f8b4b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558d15a00f20 0x558d15a09cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40818/0 (socket says 172.21.15.97:40818) 2024-03-27T10:55:46.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.776+0000 7f8b4b400640 1 -- 172.21.15.97:0/4234222875 learned_addr learned my addr 172.21.15.97:0/4234222875 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:55:46.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.776+0000 7f8b4aa00640 1 --2- 172.21.15.97:0/4234222875 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558d15a01910 0x558d15a12670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:46.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.776+0000 7f8b4b400640 1 -- 172.21.15.97:0/4234222875 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558d15a06a80 msgr2=0x558d15a16e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:55:46.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.776+0000 7f8b4b400640 1 --2- 172.21.15.97:0/4234222875 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558d15a06a80 0x558d15a16e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:46.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.776+0000 7f8b4b400640 1 -- 172.21.15.97:0/4234222875 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558d15a01910 msgr2=0x558d15a12670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:55:46.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.776+0000 7f8b4b400640 1 --2- 172.21.15.97:0/4234222875 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558d15a01910 0x558d15a12670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:46.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.776+0000 7f8b4b400640 1 -- 172.21.15.97:0/4234222875 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d15a01cf0 con 0x558d15a00f20 2024-03-27T10:55:46.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.776+0000 7f8b4b400640 1 --2- 172.21.15.97:0/4234222875 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558d15a00f20 0x558d15a09cb0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c009380 tx=0x7f8b3c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bcd3e7f9050b41e server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:55:46.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.777+0000 7f8b3be00640 1 -- 172.21.15.97:0/4234222875 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b3c028040 con 0x558d15a00f20 2024-03-27T10:55:46.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.777+0000 7f8b3be00640 1 -- 172.21.15.97:0/4234222875 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8b3c031030 con 0x558d15a00f20 2024-03-27T10:55:46.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.777+0000 7f8b3be00640 1 -- 172.21.15.97:0/4234222875 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b3c0052c0 con 0x558d15a00f20 2024-03-27T10:55:46.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.777+0000 7f8b553b2740 1 -- 172.21.15.97:0/4234222875 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558d15a00f20 msgr2=0x558d15a09cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:46.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.777+0000 7f8b553b2740 1 --2- 172.21.15.97:0/4234222875 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558d15a00f20 0x558d15a09cb0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c009380 tx=0x7f8b3c005f90 comp rx=0 tx=0).stop 2024-03-27T10:55:46.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.777+0000 7f8b553b2740 1 -- 172.21.15.97:0/4234222875 shutdown_connections 2024-03-27T10:55:46.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.777+0000 7f8b553b2740 1 --2- 172.21.15.97:0/4234222875 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558d15a06a80 0x558d15a16e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:46.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.777+0000 7f8b553b2740 1 --2- 172.21.15.97:0/4234222875 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558d15a01910 0x558d15a12670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:46.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.777+0000 7f8b553b2740 1 --2- 172.21.15.97:0/4234222875 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558d15a00f20 0x558d15a09cb0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:46.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.777+0000 7f8b553b2740 1 -- 172.21.15.97:0/4234222875 >> 172.21.15.97:0/4234222875 conn(0x558d159fc490 msgr2=0x558d159fff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:55:46.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.777+0000 7f8b553b2740 1 -- 172.21.15.97:0/4234222875 shutdown_connections 2024-03-27T10:55:46.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.777+0000 7f8b553b2740 1 -- 172.21.15.97:0/4234222875 wait complete. 2024-03-27T10:55:46.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.778+0000 7f8b553b2740 1 -- start start 2024-03-27T10:55:46.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.778+0000 7f8b553b2740 1 Processor -- start 2024-03-27T10:55:46.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.778+0000 7f8b553b2740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558d15a00f20 0x558d15a10660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:46.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.778+0000 7f8b553b2740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558d15a01910 0x558d15a823f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:46.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.778+0000 7f8b553b2740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558d15a06a80 0x558d15a84940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:46.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.778+0000 7f8b553b2740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x558d15a19180 con 0x558d15a01910 2024-03-27T10:55:46.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.778+0000 7f8b553b2740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x558d15a18ab0 con 0x558d15a06a80 2024-03-27T10:55:46.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.778+0000 7f8b553b2740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x558d15a19850 con 0x558d15a00f20 2024-03-27T10:55:46.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.778+0000 7f8b553b2740 10 client.0 _ll_register_callbacks cb 0x7f8b3c009af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:55:46.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.778+0000 7f8b4aa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558d15a01910 0x558d15a823f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:46.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.779+0000 7f8b4aa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558d15a01910 0x558d15a823f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40824/0 (socket says 172.21.15.97:40824) 2024-03-27T10:55:46.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.779+0000 7f8b4aa00640 1 -- 172.21.15.97:0/2954669214 learned_addr learned my addr 172.21.15.97:0/2954669214 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:55:46.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.779+0000 7f8b4be00640 1 --2- 172.21.15.97:0/2954669214 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558d15a06a80 0x558d15a84940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:46.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.779+0000 7f8b4b400640 1 --2- 172.21.15.97:0/2954669214 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558d15a00f20 0x558d15a10660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:46.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.779+0000 7f8b4aa00640 1 -- 172.21.15.97:0/2954669214 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558d15a00f20 msgr2=0x558d15a10660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:55:46.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.779+0000 7f8b4aa00640 1 --2- 172.21.15.97:0/2954669214 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558d15a00f20 0x558d15a10660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:46.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.779+0000 7f8b4aa00640 1 -- 172.21.15.97:0/2954669214 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558d15a06a80 msgr2=0x558d15a84940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:55:46.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.779+0000 7f8b4aa00640 1 --2- 172.21.15.97:0/2954669214 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558d15a06a80 0x558d15a84940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:46.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.779+0000 7f8b4aa00640 1 -- 172.21.15.97:0/2954669214 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d15a85050 con 0x558d15a01910 2024-03-27T10:55:46.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.779+0000 7f8b4aa00640 1 --2- 172.21.15.97:0/2954669214 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558d15a01910 0x558d15a823f0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x558d15a02170 tx=0x7f8b34009710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70af00621fc9fe28 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:55:46.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.779+0000 7f8b39600640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T10:55:46.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.779+0000 7f8b39600640 1 -- 172.21.15.97:0/2954669214 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b34018030 con 0x558d15a01910 2024-03-27T10:55:46.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.779+0000 7f8b39600640 1 -- 172.21.15.97:0/2954669214 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f8b3402a600 con 0x558d15a01910 2024-03-27T10:55:46.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.779+0000 7f8b553b2740 1 -- 172.21.15.97:0/2954669214 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b3c031030 con 0x558d15a01910 2024-03-27T10:55:46.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.780+0000 7f8b39600640 1 -- 172.21.15.97:0/2954669214 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8b3402a800 con 0x558d15a01910 2024-03-27T10:55:46.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.780+0000 7f8b553b2740 10 client.15081 fetch_fsmap learned FSMap version 27 2024-03-27T10:55:46.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.780+0000 7f8b553b2740 1 -- 172.21.15.97:0/2954669214 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558d15a86450 con 0x558d15a01910 2024-03-27T10:55:46.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.780+0000 7f8b39600640 1 -- 172.21.15.97:0/2954669214 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8b3c031030 con 0x558d15a01910 2024-03-27T10:55:46.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.780+0000 7f8b39600640 1 -- 172.21.15.97:0/2954669214 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f8b34020430 con 0x558d15a01910 2024-03-27T10:55:46.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.780+0000 7f8b553b2740 10 client.15081 fetch_fsmap finished waiting for FSMap version 27 2024-03-27T10:55:46.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.780+0000 7f8b553b2740 10 client.15081 resolve_mds: resolved name 'l' to daemon mds.l 2024-03-27T10:55:46.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.780+0000 7f8b553b2740 20 client.15081 populate_metadata read hostname 'smithi097' 2024-03-27T10:55:46.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.780+0000 7f8b553b2740 1 --2- 172.21.15.97:0/2954669214 >> [v2:172.21.15.177:6832/3787981632,v1:172.21.15.177:6833/3787981632] conn(0x558d15a8afe0 0x558d15a89cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:46.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.780+0000 7f8b553b2740 4 client.15081 mds_command: new command op to 24400 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-27T10:55:46.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.780+0000 7f8b553b2740 1 -- 172.21.15.97:0/2954669214 --> [v2:172.21.15.177:6832/3787981632,v1:172.21.15.177:6833/3787981632] -- command(tid 0: {"prefix": "status"}) -- 0x558d1596d820 con 0x558d15a8afe0 2024-03-27T10:55:46.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.781+0000 7f8b4b400640 1 --2- 172.21.15.97:0/2954669214 >> [v2:172.21.15.177:6832/3787981632,v1:172.21.15.177:6833/3787981632] conn(0x558d15a8afe0 0x558d15a89cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:46.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.782+0000 7f8b4b400640 1 --2- 172.21.15.97:0/2954669214 >> [v2:172.21.15.177:6832/3787981632,v1:172.21.15.177:6833/3787981632] conn(0x558d15a8afe0 0x558d15a89cf0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7ae9280a56e4b8db server_cookie=e5e06a02c921aa27 in_seq=0 out_seq=0 2024-03-27T10:55:46.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.782+0000 7f8b39600640 10 client.15081 ms_handle_connect on v2:172.21.15.177:6832/3787981632 2024-03-27T10:55:46.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.783+0000 7f8b39600640 1 -- 172.21.15.97:0/2954669214 <== mds.4 v2:172.21.15.177:6832/3787981632 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x558d1596d820 con 0x558d15a8afe0 2024-03-27T10:55:46.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.783+0000 7f8b39600640 10 client.15081 handle_command_reply: tid=0 2024-03-27T10:55:46.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.783+0000 7f8b553b2740 1 client.15081 shutdown 2024-03-27T10:55:46.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.783+0000 7f8b553b2740 1 -- 172.21.15.97:0/2954669214 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558d15a01910 msgr2=0x558d15a823f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:46.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.783+0000 7f8b553b2740 1 --2- 172.21.15.97:0/2954669214 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558d15a01910 0x558d15a823f0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x558d15a02170 tx=0x7f8b34009710 comp rx=0 tx=0).stop 2024-03-27T10:55:46.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.783+0000 7f8b553b2740 1 -- 172.21.15.97:0/2954669214 shutdown_connections 2024-03-27T10:55:46.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.783+0000 7f8b553b2740 1 --2- 172.21.15.97:0/2954669214 >> [v2:172.21.15.177:6832/3787981632,v1:172.21.15.177:6833/3787981632] conn(0x558d15a8afe0 0x558d15a89cf0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:46.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.783+0000 7f8b553b2740 1 --2- 172.21.15.97:0/2954669214 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558d15a06a80 0x558d15a84940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:46.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.783+0000 7f8b553b2740 1 --2- 172.21.15.97:0/2954669214 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558d15a01910 0x558d15a823f0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:46.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.783+0000 7f8b553b2740 1 --2- 172.21.15.97:0/2954669214 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558d15a00f20 0x558d15a10660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:46.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.783+0000 7f8b553b2740 1 -- 172.21.15.97:0/2954669214 >> 172.21.15.97:0/2954669214 conn(0x558d159fc490 msgr2=0x558d159fff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:55:46.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.783+0000 7f8b39600640 0 client.15081 ms_handle_reset on v2:172.21.15.177:6832/3787981632 2024-03-27T10:55:46.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.783+0000 7f8b553b2740 1 -- 172.21.15.97:0/2954669214 shutdown_connections 2024-03-27T10:55:46.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.783+0000 7f8b553b2740 1 -- 172.21.15.97:0/2954669214 wait complete. 2024-03-27T10:55:46.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.783+0000 7f8b553b2740 20 client.15081 trim_cache size 0 max 16384 2024-03-27T10:55:46.759 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-03-27T10:55:46.759 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "adca88c4-ec26-11ee-95d0-87774f69a715", 2024-03-27T10:55:46.759 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 4, 2024-03-27T10:55:46.759 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 24400, 2024-03-27T10:55:46.759 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2024-03-27T10:55:46.759 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2024-03-27T10:55:46.759 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2024-03-27T10:55:46.759 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 95.607191412999995, 2024-03-27T10:55:46.759 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 27, 2024-03-27T10:55:46.760 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 92, 2024-03-27T10:55:46.760 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 90, 2024-03-27T10:55:46.760 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 150.278854575 2024-03-27T10:55:46.760 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-03-27T10:55:46.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.784+0000 7f8b52600640 1 -- 172.21.15.97:0/1279832180 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f8b2c0a3690 msgr2=0x7f8b2c0a5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:46.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.784+0000 7f8b52600640 1 --2- 172.21.15.97:0/1279832180 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f8b2c0a3690 0x7f8b2c0a5b50 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8b34009fd0 tx=0x7f8b34014040 comp rx=0 tx=0).stop 2024-03-27T10:55:46.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.784+0000 7f8b52600640 1 -- 172.21.15.97:0/1279832180 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b4c103710 msgr2=0x7f8b4c1a1520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:46.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.784+0000 7f8b52600640 1 --2- 172.21.15.97:0/1279832180 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b4c103710 0x7f8b4c1a1520 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f8b44036040 tx=0x7f8b4402cdd0 comp rx=0 tx=0).stop 2024-03-27T10:55:46.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.784+0000 7f8b52600640 1 -- 172.21.15.97:0/1279832180 shutdown_connections 2024-03-27T10:55:46.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.784+0000 7f8b52600640 1 --2- 172.21.15.97:0/1279832180 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b4c103710 0x7f8b4c1a1520 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:46.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.784+0000 7f8b52600640 1 --2- 172.21.15.97:0/1279832180 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8b4c102d50 0x7f8b4c19a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:46.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.784+0000 7f8b52600640 1 --2- 172.21.15.97:0/1279832180 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f8b2c0a3690 0x7f8b2c0a5b50 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:46.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.784+0000 7f8b52600640 1 --2- 172.21.15.97:0/1279832180 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8b4c101b60 0x7f8b4c199f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:46.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.784+0000 7f8b52600640 1 -- 172.21.15.97:0/1279832180 >> 172.21.15.97:0/1279832180 conn(0x7f8b4c0fd0f0 msgr2=0x7f8b4c073d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:55:46.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.784+0000 7f8b52600640 1 -- 172.21.15.97:0/1279832180 shutdown_connections 2024-03-27T10:55:46.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.784+0000 7f8b52600640 1 -- 172.21.15.97:0/1279832180 wait complete. 2024-03-27T10:55:46.767 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:55:46 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/2680180006' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T10:55:46.767 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:55:46 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/4239863144' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T10:55:46.771 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2024-03-27T10:55:46.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.995+0000 7fb0afc00640 1 -- 172.21.15.97:0/3656353599 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0a8103710 msgr2=0x7fb0a8109fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:46.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.995+0000 7fb0afc00640 1 --2- 172.21.15.97:0/3656353599 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0a8103710 0x7fb0a8109fa0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb0a0009be0 tx=0x7fb0a002bef0 comp rx=0 tx=0).stop 2024-03-27T10:55:46.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.995+0000 7fb0afc00640 1 -- 172.21.15.97:0/3656353599 shutdown_connections 2024-03-27T10:55:46.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.995+0000 7fb0afc00640 1 --2- 172.21.15.97:0/3656353599 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0a8103710 0x7fb0a8109fa0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:46.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.995+0000 7fb0afc00640 1 --2- 172.21.15.97:0/3656353599 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb0a8102d50 0x7fb0a81031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:46.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.995+0000 7fb0afc00640 1 --2- 172.21.15.97:0/3656353599 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb0a8101b60 0x7fb0a8101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:46.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.995+0000 7fb0afc00640 1 -- 172.21.15.97:0/3656353599 >> 172.21.15.97:0/3656353599 conn(0x7fb0a80fd0f0 msgr2=0x7fb0a80ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:55:46.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.995+0000 7fb0afc00640 1 -- 172.21.15.97:0/3656353599 shutdown_connections 2024-03-27T10:55:46.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.995+0000 7fb0afc00640 1 -- 172.21.15.97:0/3656353599 wait complete. 2024-03-27T10:55:46.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.996+0000 7fb0afc00640 1 Processor -- start 2024-03-27T10:55:46.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.996+0000 7fb0afc00640 1 -- start start 2024-03-27T10:55:46.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.996+0000 7fb0afc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0a8101b60 0x7fb0a8199ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:46.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.996+0000 7fb0afc00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb0a8102d50 0x7fb0a819a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:46.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.996+0000 7fb0afc00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb0a8103710 0x7fb0a81a14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:46.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.996+0000 7fb0afc00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb0a810d6c0 con 0x7fb0a8101b60 2024-03-27T10:55:46.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.996+0000 7fb0afc00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb0a810d540 con 0x7fb0a8103710 2024-03-27T10:55:46.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.996+0000 7fb0afc00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb0a810d840 con 0x7fb0a8102d50 2024-03-27T10:55:46.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.997+0000 7fb0ad200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0a8101b60 0x7fb0a8199ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:46.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.997+0000 7fb0ad200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0a8101b60 0x7fb0a8199ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40840/0 (socket says 172.21.15.97:40840) 2024-03-27T10:55:46.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.997+0000 7fb0ad200640 1 -- 172.21.15.97:0/1402070204 learned_addr learned my addr 172.21.15.97:0/1402070204 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:55:46.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.997+0000 7fb0adc00640 1 --2- 172.21.15.97:0/1402070204 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb0a8103710 0x7fb0a81a14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:46.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.997+0000 7fb0a7e00640 1 --2- 172.21.15.97:0/1402070204 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb0a8102d50 0x7fb0a819a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:46.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.997+0000 7fb0ad200640 1 -- 172.21.15.97:0/1402070204 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb0a8102d50 msgr2=0x7fb0a819a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:46.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.997+0000 7fb0ad200640 1 --2- 172.21.15.97:0/1402070204 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb0a8102d50 0x7fb0a819a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:46.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.997+0000 7fb0ad200640 1 -- 172.21.15.97:0/1402070204 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb0a8103710 msgr2=0x7fb0a81a14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:46.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.997+0000 7fb0ad200640 1 --2- 172.21.15.97:0/1402070204 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb0a8103710 0x7fb0a81a14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:46.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.997+0000 7fb0ad200640 1 -- 172.21.15.97:0/1402070204 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0a81a1b10 con 0x7fb0a8101b60 2024-03-27T10:55:46.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.997+0000 7fb0a7e00640 1 --2- 172.21.15.97:0/1402070204 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb0a8102d50 0x7fb0a819a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:55:46.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.997+0000 7fb0ad200640 1 --2- 172.21.15.97:0/1402070204 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0a8101b60 0x7fb0a8199ef0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb09c00d770 tx=0x7fb09c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:55:46.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.997+0000 7fb0a5600640 1 -- 172.21.15.97:0/1402070204 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb09c017070 con 0x7fb0a8101b60 2024-03-27T10:55:46.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.997+0000 7fb0afc00640 1 -- 172.21.15.97:0/1402070204 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0a81a1e00 con 0x7fb0a8101b60 2024-03-27T10:55:46.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.997+0000 7fb0a5600640 1 -- 172.21.15.97:0/1402070204 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb09c009c30 con 0x7fb0a8101b60 2024-03-27T10:55:46.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.998+0000 7fb0a5600640 1 -- 172.21.15.97:0/1402070204 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb09c003d00 con 0x7fb0a8101b60 2024-03-27T10:55:46.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:46.998+0000 7fb0afc00640 1 -- 172.21.15.97:0/1402070204 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0a81a2260 con 0x7fb0a8101b60 2024-03-27T10:55:46.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.001+0000 7fb0b29b1740 1 Processor -- start 2024-03-27T10:55:46.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.001+0000 7fb0a5600640 1 -- 172.21.15.97:0/1402070204 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7fb09c003ea0 con 0x7fb0a8101b60 2024-03-27T10:55:46.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.001+0000 7fb0b29b1740 1 -- start start 2024-03-27T10:55:46.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.001+0000 7fb0b29b1740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5649f8611f20 0x5649f8612340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:46.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.001+0000 7fb0b29b1740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5649f8612910 0x5649f8617410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:46.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.001+0000 7fb0b29b1740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649f8617a80 0x5649f8619f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:46.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.001+0000 7fb0b29b1740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5649f85d4920 con 0x5649f8611f20 2024-03-27T10:55:46.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.001+0000 7fb0b29b1740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5649f8524fa0 con 0x5649f8617a80 2024-03-27T10:55:46.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.001+0000 7fb0b29b1740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5649f8576f90 con 0x5649f8612910 2024-03-27T10:55:46.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.001+0000 7fb0adc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5649f8611f20 0x5649f8612340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:46.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.001+0000 7fb0adc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5649f8611f20 0x5649f8612340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40842/0 (socket says 172.21.15.97:40842) 2024-03-27T10:55:46.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.001+0000 7fb0adc00640 1 -- 172.21.15.97:0/3375518240 learned_addr learned my addr 172.21.15.97:0/3375518240 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:55:46.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.001+0000 7fb0a7e00640 1 --2- 172.21.15.97:0/3375518240 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649f8617a80 0x5649f8619f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:46.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.002+0000 7fb0ad200640 1 --2- 172.21.15.97:0/3375518240 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5649f8612910 0x5649f8617410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:46.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.002+0000 7fb0adc00640 1 -- 172.21.15.97:0/3375518240 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5649f8612910 msgr2=0x5649f8617410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:55:46.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.002+0000 7fb0adc00640 1 --2- 172.21.15.97:0/3375518240 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5649f8612910 0x5649f8617410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:46.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.002+0000 7fb0adc00640 1 -- 172.21.15.97:0/3375518240 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649f8617a80 msgr2=0x5649f8619f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:55:46.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.002+0000 7fb0adc00640 1 --2- 172.21.15.97:0/3375518240 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649f8617a80 0x5649f8619f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:46.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.002+0000 7fb0adc00640 1 -- 172.21.15.97:0/3375518240 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649f828b2d0 con 0x5649f8611f20 2024-03-27T10:55:46.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.002+0000 7fb0adc00640 1 --2- 172.21.15.97:0/3375518240 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5649f8611f20 0x5649f8612340 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb0a0004af0 tx=0x7fb0a0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e66fa7a83f152d11 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:55:46.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.002+0000 7fb0a5600640 1 --2- 172.21.15.97:0/1402070204 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fb0900a3610 0x7fb0900a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:46.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.002+0000 7fb0ad200640 1 --2- 172.21.15.97:0/1402070204 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fb0900a3610 0x7fb0900a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:46.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.002+0000 7fb0a5600640 1 -- 172.21.15.97:0/1402070204 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fb09c0cfb10 con 0x7fb0a8101b60 2024-03-27T10:55:46.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.002+0000 7fb09a000640 1 -- 172.21.15.97:0/3375518240 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a0043070 con 0x5649f8611f20 2024-03-27T10:55:46.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.003+0000 7fb09a000640 1 -- 172.21.15.97:0/3375518240 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb0a0033590 con 0x5649f8611f20 2024-03-27T10:55:46.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.003+0000 7fb09a000640 1 -- 172.21.15.97:0/3375518240 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a002c890 con 0x5649f8611f20 2024-03-27T10:55:46.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.003+0000 7fb0b29b1740 1 -- 172.21.15.97:0/3375518240 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5649f8611f20 msgr2=0x5649f8612340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:46.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.003+0000 7fb0ad200640 1 --2- 172.21.15.97:0/1402070204 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fb0900a3610 0x7fb0900a5ad0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x5649f8612d10 tx=0x7fb09c005f30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:55:46.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.003+0000 7fb0b29b1740 1 --2- 172.21.15.97:0/3375518240 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5649f8611f20 0x5649f8612340 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb0a0004af0 tx=0x7fb0a0009840 comp rx=0 tx=0).stop 2024-03-27T10:55:46.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.003+0000 7fb0b29b1740 1 -- 172.21.15.97:0/3375518240 shutdown_connections 2024-03-27T10:55:46.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.003+0000 7fb0b29b1740 1 --2- 172.21.15.97:0/3375518240 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649f8617a80 0x5649f8619f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:46.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.003+0000 7fb0b29b1740 1 --2- 172.21.15.97:0/3375518240 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5649f8612910 0x5649f8617410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:46.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.003+0000 7fb0b29b1740 1 --2- 172.21.15.97:0/3375518240 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5649f8611f20 0x5649f8612340 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:46.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.003+0000 7fb0b29b1740 1 -- 172.21.15.97:0/3375518240 >> 172.21.15.97:0/3375518240 conn(0x5649f860d490 msgr2=0x5649f860d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:55:46.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.003+0000 7fb0b29b1740 1 -- 172.21.15.97:0/3375518240 shutdown_connections 2024-03-27T10:55:46.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.003+0000 7fb0b29b1740 1 -- 172.21.15.97:0/3375518240 wait complete. 2024-03-27T10:55:46.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.004+0000 7fb0b29b1740 1 -- start start 2024-03-27T10:55:46.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.004+0000 7fb0b29b1740 1 Processor -- start 2024-03-27T10:55:46.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.004+0000 7fb0b29b1740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649f8611f20 0x5649f861c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:46.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.004+0000 7fb0b29b1740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5649f8612910 0x5649f861c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:46.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.005+0000 7fb0b29b1740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5649f8617a80 0x5649f86236b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:46.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.005+0000 7fb0b29b1740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5649f8572f40 con 0x5649f8617a80 2024-03-27T10:55:46.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.005+0000 7fb0b29b1740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5649f851f650 con 0x5649f8611f20 2024-03-27T10:55:46.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.005+0000 7fb0b29b1740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5649f8532a10 con 0x5649f8612910 2024-03-27T10:55:46.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.005+0000 7fb0b29b1740 10 client.0 _ll_register_callbacks cb 0x5649f8571ff0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:55:46.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.005+0000 7fb0adc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5649f8617a80 0x5649f86236b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:46.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.005+0000 7fb0adc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5649f8617a80 0x5649f86236b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40848/0 (socket says 172.21.15.97:40848) 2024-03-27T10:55:46.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.005+0000 7fb0adc00640 1 -- 172.21.15.97:0/1571822720 learned_addr learned my addr 172.21.15.97:0/1571822720 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:55:46.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.005+0000 7fb0adc00640 1 --2- 172.21.15.97:0/1571822720 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649f8611f20 0x5649f861c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:46.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.005+0000 7fb0a7e00640 1 --2- 172.21.15.97:0/1571822720 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5649f8612910 0x5649f861c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:46.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.005+0000 7fb0adc00640 1 -- 172.21.15.97:0/1571822720 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5649f8612910 msgr2=0x5649f861c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:55:46.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.005+0000 7fb0adc00640 1 --2- 172.21.15.97:0/1571822720 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5649f8612910 0x5649f861c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:46.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.005+0000 7fb0adc00640 1 -- 172.21.15.97:0/1571822720 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649f8611f20 msgr2=0x5649f861c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:55:46.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.005+0000 7fb0adc00640 1 --2- 172.21.15.97:0/1571822720 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649f8611f20 0x5649f861c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:46.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.005+0000 7fb0adc00640 1 -- 172.21.15.97:0/1571822720 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649f83556b0 con 0x5649f8617a80 2024-03-27T10:55:46.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.005+0000 7fb0adc00640 1 --2- 172.21.15.97:0/1571822720 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5649f8617a80 0x5649f86236b0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb0a002cfd0 tx=0x7fb0a002c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b73d39dbcc5116be server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:55:46.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.005+0000 7fb08ea00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T10:55:46.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.006+0000 7fb08ea00640 1 -- 172.21.15.97:0/1571822720 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a0043030 con 0x5649f8617a80 2024-03-27T10:55:46.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.006+0000 7fb08ea00640 1 -- 172.21.15.97:0/1571822720 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fb0a0033740 con 0x5649f8617a80 2024-03-27T10:55:46.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.006+0000 7fb0b29b1740 1 -- 172.21.15.97:0/1571822720 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0a0033590 con 0x5649f8617a80 2024-03-27T10:55:46.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.006+0000 7fb08ea00640 1 -- 172.21.15.97:0/1571822720 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb0a002c900 con 0x5649f8617a80 2024-03-27T10:55:46.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.006+0000 7fb0b29b1740 10 client.15093 fetch_fsmap learned FSMap version 27 2024-03-27T10:55:46.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.006+0000 7fb0b29b1740 1 -- 172.21.15.97:0/1571822720 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5649f8626a00 con 0x5649f8617a80 2024-03-27T10:55:46.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.006+0000 7fb08ea00640 1 -- 172.21.15.97:0/1571822720 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb0a0033590 con 0x5649f8617a80 2024-03-27T10:55:46.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.006+0000 7fb08ea00640 1 -- 172.21.15.97:0/1571822720 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fb0a00046d0 con 0x5649f8617a80 2024-03-27T10:55:46.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.006+0000 7fb0b29b1740 10 client.15093 fetch_fsmap finished waiting for FSMap version 27 2024-03-27T10:55:46.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.007+0000 7fb0b29b1740 10 client.15093 resolve_mds: resolved name 'f' to daemon mds.f 2024-03-27T10:55:46.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.007+0000 7fb0b29b1740 20 client.15093 populate_metadata read hostname 'smithi097' 2024-03-27T10:55:46.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.007+0000 7fb0b29b1740 1 --2- 172.21.15.97:0/1571822720 >> [v2:172.21.15.177:6834/3349112132,v1:172.21.15.177:6835/3349112132] conn(0x5649f8628bc0 0x5649f862a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:46.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.007+0000 7fb0b29b1740 4 client.15093 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T10:55:46.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.007+0000 7fb0b29b1740 1 -- 172.21.15.97:0/1571822720 --> [v2:172.21.15.177:6834/3349112132,v1:172.21.15.177:6835/3349112132] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5649f85844d0 con 0x5649f8628bc0 2024-03-27T10:55:46.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.007+0000 7fb0a7e00640 1 --2- 172.21.15.97:0/1571822720 >> [v2:172.21.15.177:6834/3349112132,v1:172.21.15.177:6835/3349112132] conn(0x5649f8628bc0 0x5649f862a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:46.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.008+0000 7fb0a7e00640 1 --2- 172.21.15.97:0/1571822720 >> [v2:172.21.15.177:6834/3349112132,v1:172.21.15.177:6835/3349112132] conn(0x5649f8628bc0 0x5649f862a030 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=15322f57cebc0ed5 server_cookie=235897230cf340ac in_seq=0 out_seq=0 2024-03-27T10:55:46.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.008+0000 7fb08ea00640 10 client.15093 ms_handle_connect on v2:172.21.15.177:6834/3349112132 2024-03-27T10:55:46.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.010+0000 7fb08ea00640 1 -- 172.21.15.97:0/1571822720 <== mds.3 v2:172.21.15.177:6834/3349112132 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5649f85844d0 con 0x5649f8628bc0 2024-03-27T10:55:46.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.010+0000 7fb08ea00640 10 client.15093 handle_command_reply: tid=0 2024-03-27T10:55:46.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.010+0000 7fb0b29b1740 1 client.15093 shutdown 2024-03-27T10:55:46.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.010+0000 7fb0b29b1740 1 -- 172.21.15.97:0/1571822720 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5649f8617a80 msgr2=0x5649f86236b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:46.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.010+0000 7fb0b29b1740 1 --2- 172.21.15.97:0/1571822720 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5649f8617a80 0x5649f86236b0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb0a002cfd0 tx=0x7fb0a002c890 comp rx=0 tx=0).stop 2024-03-27T10:55:46.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.010+0000 7fb0b29b1740 1 -- 172.21.15.97:0/1571822720 shutdown_connections 2024-03-27T10:55:46.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.010+0000 7fb0b29b1740 1 --2- 172.21.15.97:0/1571822720 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5649f8617a80 0x5649f86236b0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:46.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.010+0000 7fb0b29b1740 1 --2- 172.21.15.97:0/1571822720 >> [v2:172.21.15.177:6834/3349112132,v1:172.21.15.177:6835/3349112132] conn(0x5649f8628bc0 0x5649f862a030 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:46.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.010+0000 7fb0b29b1740 1 --2- 172.21.15.97:0/1571822720 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5649f8612910 0x5649f861c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:46.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.010+0000 7fb0b29b1740 1 --2- 172.21.15.97:0/1571822720 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649f8611f20 0x5649f861c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:46.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.010+0000 7fb0b29b1740 1 -- 172.21.15.97:0/1571822720 >> 172.21.15.97:0/1571822720 conn(0x5649f860d490 msgr2=0x5649f860c690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:55:46.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.010+0000 7fb08ea00640 0 client.15093 ms_handle_reset on v2:172.21.15.177:6834/3349112132 2024-03-27T10:55:46.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.010+0000 7fb0b29b1740 1 -- 172.21.15.97:0/1571822720 shutdown_connections 2024-03-27T10:55:46.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.010+0000 7fb0b29b1740 1 -- 172.21.15.97:0/1571822720 wait complete. 2024-03-27T10:55:46.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.011+0000 7fb0b29b1740 20 client.15093 trim_cache size 0 max 16384 2024-03-27T10:55:47.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.029+0000 7fb0b29b1740 1 Processor -- start 2024-03-27T10:55:47.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.029+0000 7fb0b29b1740 1 -- start start 2024-03-27T10:55:47.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.029+0000 7fb0b29b1740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5649f8611f20 0x5649f8627d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:47.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.029+0000 7fb0b29b1740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5649f8612910 0x5649f861c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:47.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.029+0000 7fb0b29b1740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649f8617a80 0x5649f86235d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:47.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.029+0000 7fb0b29b1740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5649f8628370 con 0x5649f8612910 2024-03-27T10:55:47.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.029+0000 7fb0b29b1740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5649f8628a40 con 0x5649f8617a80 2024-03-27T10:55:47.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.029+0000 7fb0b29b1740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5649f8629110 con 0x5649f8611f20 2024-03-27T10:55:47.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.030+0000 7fb0a7e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5649f8612910 0x5649f861c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:47.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.030+0000 7fb0a7e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5649f8612910 0x5649f861c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40864/0 (socket says 172.21.15.97:40864) 2024-03-27T10:55:47.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.030+0000 7fb0a7e00640 1 -- 172.21.15.97:0/2578266703 learned_addr learned my addr 172.21.15.97:0/2578266703 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:55:47.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.030+0000 7fb0adc00640 1 --2- 172.21.15.97:0/2578266703 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5649f8611f20 0x5649f8627d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:47.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.030+0000 7fb0adc00640 1 --2- 172.21.15.97:0/2578266703 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649f8617a80 0x5649f86235d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:47.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.030+0000 7fb0a7e00640 1 -- 172.21.15.97:0/2578266703 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5649f8611f20 msgr2=0x5649f8627d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:55:47.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.030+0000 7fb0a7e00640 1 --2- 172.21.15.97:0/2578266703 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5649f8611f20 0x5649f8627d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.030+0000 7fb0a7e00640 1 -- 172.21.15.97:0/2578266703 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649f8617a80 msgr2=0x5649f86235d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:55:47.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.030+0000 7fb0a7e00640 1 --2- 172.21.15.97:0/2578266703 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649f8617a80 0x5649f86235d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.030+0000 7fb0a7e00640 1 -- 172.21.15.97:0/2578266703 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649f860e200 con 0x5649f8612910 2024-03-27T10:55:47.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.030+0000 7fb0adc00640 1 --2- 172.21.15.97:0/2578266703 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5649f8611f20 0x5649f8627d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:55:47.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.030+0000 7fb0a7e00640 1 --2- 172.21.15.97:0/2578266703 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5649f8612910 0x5649f861c5d0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fb0a819b400 tx=0x7fb09401ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a3d771389b3c2b4 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:55:47.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.031+0000 7fb099600640 1 -- 172.21.15.97:0/2578266703 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb09401e430 con 0x5649f8612910 2024-03-27T10:55:47.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.031+0000 7fb099600640 1 -- 172.21.15.97:0/2578266703 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb09401e5d0 con 0x5649f8612910 2024-03-27T10:55:47.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.031+0000 7fb099600640 1 -- 172.21.15.97:0/2578266703 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0940047a0 con 0x5649f8612910 2024-03-27T10:55:47.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.031+0000 7fb0b29b1740 1 -- 172.21.15.97:0/2578266703 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5649f8612910 msgr2=0x5649f861c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:47.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.031+0000 7fb0b29b1740 1 --2- 172.21.15.97:0/2578266703 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5649f8612910 0x5649f861c5d0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fb0a819b400 tx=0x7fb09401ef60 comp rx=0 tx=0).stop 2024-03-27T10:55:47.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.031+0000 7fb0b29b1740 1 -- 172.21.15.97:0/2578266703 shutdown_connections 2024-03-27T10:55:47.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.031+0000 7fb0b29b1740 1 --2- 172.21.15.97:0/2578266703 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649f8617a80 0x5649f86235d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.031+0000 7fb0b29b1740 1 --2- 172.21.15.97:0/2578266703 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5649f8612910 0x5649f861c5d0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.031+0000 7fb0b29b1740 1 --2- 172.21.15.97:0/2578266703 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5649f8611f20 0x5649f8627d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.031+0000 7fb0b29b1740 1 -- 172.21.15.97:0/2578266703 >> 172.21.15.97:0/2578266703 conn(0x5649f860d490 msgr2=0x5649f861ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:55:47.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.031+0000 7fb0b29b1740 1 -- 172.21.15.97:0/2578266703 shutdown_connections 2024-03-27T10:55:47.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.031+0000 7fb0b29b1740 1 -- 172.21.15.97:0/2578266703 wait complete. 2024-03-27T10:55:47.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.032+0000 7fb0b29b1740 1 -- start start 2024-03-27T10:55:47.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.032+0000 7fb0b29b1740 1 Processor -- start 2024-03-27T10:55:47.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.032+0000 7fb0b29b1740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649f8611f20 0x5649f861eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:47.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.032+0000 7fb0b29b1740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5649f8612910 0x5649f861f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:47.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.032+0000 7fb0b29b1740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5649f8617a80 0x5649f8622400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:47.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.032+0000 7fb0b29b1740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5649f8629eb0 con 0x5649f8612910 2024-03-27T10:55:47.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.032+0000 7fb0b29b1740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5649f86297e0 con 0x5649f8611f20 2024-03-27T10:55:47.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.032+0000 7fb0b29b1740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5649f862a580 con 0x5649f8617a80 2024-03-27T10:55:47.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.032+0000 7fb0b29b1740 10 client.0 _ll_register_callbacks cb 0x7fb0940094a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:55:47.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.032+0000 7fb0a7e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5649f8612910 0x5649f861f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:47.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.033+0000 7fb0a7e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5649f8612910 0x5649f861f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40876/0 (socket says 172.21.15.97:40876) 2024-03-27T10:55:47.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.033+0000 7fb0a7e00640 1 -- 172.21.15.97:0/1627314106 learned_addr learned my addr 172.21.15.97:0/1627314106 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:55:47.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.033+0000 7fb0adc00640 1 --2- 172.21.15.97:0/1627314106 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649f8611f20 0x5649f861eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:47.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.033+0000 7fb0adc00640 1 --2- 172.21.15.97:0/1627314106 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5649f8617a80 0x5649f8622400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:47.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.033+0000 7fb0a7e00640 1 -- 172.21.15.97:0/1627314106 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5649f8617a80 msgr2=0x5649f8622400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:55:47.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.033+0000 7fb0a7e00640 1 --2- 172.21.15.97:0/1627314106 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5649f8617a80 0x5649f8622400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.033+0000 7fb0a7e00640 1 -- 172.21.15.97:0/1627314106 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649f8611f20 msgr2=0x5649f861eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:55:47.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.033+0000 7fb0a7e00640 1 --2- 172.21.15.97:0/1627314106 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649f8611f20 0x5649f861eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.033+0000 7fb0a7e00640 1 -- 172.21.15.97:0/1627314106 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649f86229f0 con 0x5649f8612910 2024-03-27T10:55:47.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.033+0000 7fb0adc00640 1 --2- 172.21.15.97:0/1627314106 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649f8611f20 0x5649f861eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T10:55:47.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.033+0000 7fb0a7e00640 1 --2- 172.21.15.97:0/1627314106 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5649f8612910 0x5649f861f3e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fb0940096c0 tx=0x7fb09401e780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7a2e960b28786b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:55:47.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.033+0000 7fb08ea00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T10:55:47.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.033+0000 7fb08ea00640 1 -- 172.21.15.97:0/1627314106 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb094014290 con 0x5649f8612910 2024-03-27T10:55:47.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.033+0000 7fb08ea00640 1 -- 172.21.15.97:0/1627314106 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fb094014a60 con 0x5649f8612910 2024-03-27T10:55:47.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.033+0000 7fb0b29b1740 1 -- 172.21.15.97:0/1627314106 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb09401e5d0 con 0x5649f8612910 2024-03-27T10:55:47.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.034+0000 7fb08ea00640 1 -- 172.21.15.97:0/1627314106 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb094037050 con 0x5649f8612910 2024-03-27T10:55:47.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.034+0000 7fb0b29b1740 10 client.15099 fetch_fsmap learned FSMap version 27 2024-03-27T10:55:47.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.034+0000 7fb0b29b1740 1 -- 172.21.15.97:0/1627314106 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5649f868e9a0 con 0x5649f8612910 2024-03-27T10:55:47.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.034+0000 7fb08ea00640 1 -- 172.21.15.97:0/1627314106 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb09401e5d0 con 0x5649f8612910 2024-03-27T10:55:47.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.034+0000 7fb08ea00640 1 -- 172.21.15.97:0/1627314106 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fb09400f040 con 0x5649f8612910 2024-03-27T10:55:47.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.034+0000 7fb0b29b1740 10 client.15099 fetch_fsmap finished waiting for FSMap version 27 2024-03-27T10:55:47.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.034+0000 7fb0b29b1740 10 client.15099 resolve_mds: resolved name 'f' to daemon mds.f 2024-03-27T10:55:47.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.034+0000 7fb0b29b1740 20 client.15099 populate_metadata read hostname 'smithi097' 2024-03-27T10:55:47.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.034+0000 7fb0b29b1740 1 --2- 172.21.15.97:0/1627314106 >> [v2:172.21.15.177:6834/3349112132,v1:172.21.15.177:6835/3349112132] conn(0x5649f8693230 0x5649f8691f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:47.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.034+0000 7fb0b29b1740 4 client.15099 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-27T10:55:47.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.034+0000 7fb0b29b1740 1 -- 172.21.15.97:0/1627314106 --> [v2:172.21.15.177:6834/3349112132,v1:172.21.15.177:6835/3349112132] -- command(tid 0: {"prefix": "status"}) -- 0x5649f857e820 con 0x5649f8693230 2024-03-27T10:55:47.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.035+0000 7fb0adc00640 1 --2- 172.21.15.97:0/1627314106 >> [v2:172.21.15.177:6834/3349112132,v1:172.21.15.177:6835/3349112132] conn(0x5649f8693230 0x5649f8691f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:47.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.036+0000 7fb0adc00640 1 --2- 172.21.15.97:0/1627314106 >> [v2:172.21.15.177:6834/3349112132,v1:172.21.15.177:6835/3349112132] conn(0x5649f8693230 0x5649f8691f40 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5f5d6b3fdee428eb server_cookie=17c04728c1fee6de in_seq=0 out_seq=0 2024-03-27T10:55:47.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.036+0000 7fb08ea00640 10 client.15099 ms_handle_connect on v2:172.21.15.177:6834/3349112132 2024-03-27T10:55:47.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.036+0000 7fb08ea00640 1 -- 172.21.15.97:0/1627314106 <== mds.3 v2:172.21.15.177:6834/3349112132 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5649f857e820 con 0x5649f8693230 2024-03-27T10:55:47.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.036+0000 7fb08ea00640 10 client.15099 handle_command_reply: tid=0 2024-03-27T10:55:47.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.036+0000 7fb0b29b1740 1 client.15099 shutdown 2024-03-27T10:55:47.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.036+0000 7fb0b29b1740 1 -- 172.21.15.97:0/1627314106 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5649f8612910 msgr2=0x5649f861f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:47.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.036+0000 7fb0b29b1740 1 --2- 172.21.15.97:0/1627314106 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5649f8612910 0x5649f861f3e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fb0940096c0 tx=0x7fb09401e780 comp rx=0 tx=0).stop 2024-03-27T10:55:47.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.036+0000 7fb0b29b1740 1 -- 172.21.15.97:0/1627314106 shutdown_connections 2024-03-27T10:55:47.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.036+0000 7fb0b29b1740 1 --2- 172.21.15.97:0/1627314106 >> [v2:172.21.15.177:6834/3349112132,v1:172.21.15.177:6835/3349112132] conn(0x5649f8693230 0x5649f8691f40 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.036+0000 7fb0b29b1740 1 --2- 172.21.15.97:0/1627314106 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5649f8617a80 0x5649f8622400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.036+0000 7fb0b29b1740 1 --2- 172.21.15.97:0/1627314106 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5649f8612910 0x5649f861f3e0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.036+0000 7fb0b29b1740 1 --2- 172.21.15.97:0/1627314106 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649f8611f20 0x5649f861eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.036+0000 7fb08ea00640 0 client.15099 ms_handle_reset on v2:172.21.15.177:6834/3349112132 2024-03-27T10:55:47.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.036+0000 7fb0b29b1740 1 -- 172.21.15.97:0/1627314106 >> 172.21.15.97:0/1627314106 conn(0x5649f860d490 msgr2=0x5649f860d870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:55:47.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.037+0000 7fb0b29b1740 1 -- 172.21.15.97:0/1627314106 shutdown_connections 2024-03-27T10:55:47.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.037+0000 7fb0b29b1740 1 -- 172.21.15.97:0/1627314106 wait complete. 2024-03-27T10:55:47.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.037+0000 7fb0b29b1740 20 client.15099 trim_cache size 0 max 16384 2024-03-27T10:55:47.012 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-03-27T10:55:47.012 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "adca88c4-ec26-11ee-95d0-87774f69a715", 2024-03-27T10:55:47.013 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 3, 2024-03-27T10:55:47.013 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 24454, 2024-03-27T10:55:47.013 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2024-03-27T10:55:47.013 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2024-03-27T10:55:47.013 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2024-03-27T10:55:47.013 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 96.874799104000004, 2024-03-27T10:55:47.013 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 27, 2024-03-27T10:55:47.013 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 92, 2024-03-27T10:55:47.013 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 90, 2024-03-27T10:55:47.013 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 130.71227698600001 2024-03-27T10:55:47.013 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-03-27T10:55:47.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.037+0000 7fb0afc00640 1 -- 172.21.15.97:0/1402070204 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fb0900a3610 msgr2=0x7fb0900a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:47.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.037+0000 7fb0afc00640 1 --2- 172.21.15.97:0/1402070204 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fb0900a3610 0x7fb0900a5ad0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x5649f8612d10 tx=0x7fb09c005f30 comp rx=0 tx=0).stop 2024-03-27T10:55:47.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.037+0000 7fb0afc00640 1 -- 172.21.15.97:0/1402070204 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0a8101b60 msgr2=0x7fb0a8199ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:47.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.037+0000 7fb0afc00640 1 --2- 172.21.15.97:0/1402070204 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0a8101b60 0x7fb0a8199ef0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb09c00d770 tx=0x7fb09c00dc30 comp rx=0 tx=0).stop 2024-03-27T10:55:47.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.037+0000 7fb0afc00640 1 -- 172.21.15.97:0/1402070204 shutdown_connections 2024-03-27T10:55:47.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.037+0000 7fb0afc00640 1 --2- 172.21.15.97:0/1402070204 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fb0900a3610 0x7fb0900a5ad0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.037+0000 7fb0afc00640 1 --2- 172.21.15.97:0/1402070204 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb0a8103710 0x7fb0a81a14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.037+0000 7fb0afc00640 1 --2- 172.21.15.97:0/1402070204 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb0a8102d50 0x7fb0a819a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.037+0000 7fb0afc00640 1 --2- 172.21.15.97:0/1402070204 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0a8101b60 0x7fb0a8199ef0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.037+0000 7fb0afc00640 1 -- 172.21.15.97:0/1402070204 >> 172.21.15.97:0/1402070204 conn(0x7fb0a80fd0f0 msgr2=0x7fb0a8073d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:55:47.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.038+0000 7fb0afc00640 1 -- 172.21.15.97:0/1402070204 shutdown_connections 2024-03-27T10:55:47.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.038+0000 7fb0afc00640 1 -- 172.21.15.97:0/1402070204 wait complete. 2024-03-27T10:55:47.024 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2024-03-27T10:55:47.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.240+0000 7f17d1000640 1 -- 172.21.15.97:0/817585401 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17cc109380 msgr2=0x7f17cc109800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:47.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.240+0000 7f17d1000640 1 --2- 172.21.15.97:0/817585401 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17cc109380 0x7f17cc109800 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f17b4009900 tx=0x7f17b402bc30 comp rx=0 tx=0).stop 2024-03-27T10:55:47.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.240+0000 7f17d1000640 1 -- 172.21.15.97:0/817585401 shutdown_connections 2024-03-27T10:55:47.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.240+0000 7f17d1000640 1 --2- 172.21.15.97:0/817585401 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f17cc109d40 0x7f17cc1105d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.240+0000 7f17d1000640 1 --2- 172.21.15.97:0/817585401 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17cc109380 0x7f17cc109800 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.240+0000 7f17d1000640 1 --2- 172.21.15.97:0/817585401 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f17cc108190 0x7f17cc1085b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.240+0000 7f17d1000640 1 -- 172.21.15.97:0/817585401 >> 172.21.15.97:0/817585401 conn(0x7f17cc077120 msgr2=0x7f17cc079560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:55:47.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.240+0000 7f17d1000640 1 -- 172.21.15.97:0/817585401 shutdown_connections 2024-03-27T10:55:47.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.240+0000 7f17d1000640 1 -- 172.21.15.97:0/817585401 wait complete. 2024-03-27T10:55:47.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.241+0000 7f17d1000640 1 Processor -- start 2024-03-27T10:55:47.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.242+0000 7f17d1000640 1 -- start start 2024-03-27T10:55:47.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.242+0000 7f17d1000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f17cc108190 0x7f17cc19e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:47.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.242+0000 7f17d1000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f17cc109380 0x7f17cc19e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:47.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.242+0000 7f17d1000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17cc109d40 0x7f17cc1a5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:47.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.242+0000 7f17d1000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f17cc113cd0 con 0x7f17cc109d40 2024-03-27T10:55:47.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.242+0000 7f17d1000640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f17cc113b50 con 0x7f17cc108190 2024-03-27T10:55:47.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.242+0000 7f17d1000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f17cc113e50 con 0x7f17cc109380 2024-03-27T10:55:47.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.242+0000 7f17ca800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17cc109d40 0x7f17cc1a5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:47.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.242+0000 7f17ca800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17cc109d40 0x7f17cc1a5910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40888/0 (socket says 172.21.15.97:40888) 2024-03-27T10:55:47.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.242+0000 7f17ca800640 1 -- 172.21.15.97:0/3888939373 learned_addr learned my addr 172.21.15.97:0/3888939373 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:55:47.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.242+0000 7f17c9e00640 1 --2- 172.21.15.97:0/3888939373 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f17cc108190 0x7f17cc19e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:47.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.243+0000 7f17c1400640 1 --2- 172.21.15.97:0/3888939373 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f17cc109380 0x7f17cc19e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:47.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.243+0000 7f17ca800640 1 -- 172.21.15.97:0/3888939373 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f17cc109380 msgr2=0x7f17cc19e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:47.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.243+0000 7f17ca800640 1 --2- 172.21.15.97:0/3888939373 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f17cc109380 0x7f17cc19e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.243+0000 7f17ca800640 1 -- 172.21.15.97:0/3888939373 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f17cc108190 msgr2=0x7f17cc19e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:47.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.243+0000 7f17ca800640 1 --2- 172.21.15.97:0/3888939373 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f17cc108190 0x7f17cc19e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.243+0000 7f17ca800640 1 -- 172.21.15.97:0/3888939373 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17cc1a5f80 con 0x7f17cc109d40 2024-03-27T10:55:47.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.243+0000 7f17ca800640 1 --2- 172.21.15.97:0/3888939373 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17cc109d40 0x7f17cc1a5910 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f17c400eaa0 tx=0x7f17c400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:55:47.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.243+0000 7f17c3400640 1 -- 172.21.15.97:0/3888939373 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c400cdb0 con 0x7f17cc109d40 2024-03-27T10:55:47.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.243+0000 7f17c3400640 1 -- 172.21.15.97:0/3888939373 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f17c4005020 con 0x7f17cc109d40 2024-03-27T10:55:47.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.243+0000 7f17d1000640 1 -- 172.21.15.97:0/3888939373 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17cc1a6270 con 0x7f17cc109d40 2024-03-27T10:55:47.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.243+0000 7f17c3400640 1 -- 172.21.15.97:0/3888939373 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c40116d0 con 0x7f17cc109d40 2024-03-27T10:55:47.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.243+0000 7f17d1000640 1 -- 172.21.15.97:0/3888939373 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17cc1a67c0 con 0x7f17cc109d40 2024-03-27T10:55:47.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.246+0000 7f17d3e26740 1 Processor -- start 2024-03-27T10:55:47.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.246+0000 7f17c3400640 1 -- 172.21.15.97:0/3888939373 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f17c40051d0 con 0x7f17cc109d40 2024-03-27T10:55:47.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.247+0000 7f17d3e26740 1 -- start start 2024-03-27T10:55:47.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.247+0000 7f17d3e26740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bd94693f20 0x55bd94694340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:47.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.247+0000 7f17d3e26740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd94694910 0x55bd94699410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:47.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.247+0000 7f17d3e26740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd94699a80 0x55bd9469bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:47.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.247+0000 7f17d3e26740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55bd94656920 con 0x55bd94694910 2024-03-27T10:55:47.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.247+0000 7f17d3e26740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55bd945a6fa0 con 0x55bd94699a80 2024-03-27T10:55:47.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.247+0000 7f17d3e26740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55bd945f8f90 con 0x55bd94693f20 2024-03-27T10:55:47.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.247+0000 7f17c9e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd94694910 0x55bd94699410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:47.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.247+0000 7f17c9e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd94694910 0x55bd94699410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40900/0 (socket says 172.21.15.97:40900) 2024-03-27T10:55:47.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.247+0000 7f17c9e00640 1 -- 172.21.15.97:0/422800501 learned_addr learned my addr 172.21.15.97:0/422800501 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:55:47.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.247+0000 7f17c1400640 1 --2- 172.21.15.97:0/422800501 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd94699a80 0x55bd9469bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:47.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.248+0000 7f17c1400640 1 --2- 172.21.15.97:0/422800501 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bd94693f20 0x55bd94694340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:47.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.248+0000 7f17c9e00640 1 -- 172.21.15.97:0/422800501 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bd94693f20 msgr2=0x55bd94694340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:55:47.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.248+0000 7f17c9e00640 1 --2- 172.21.15.97:0/422800501 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bd94693f20 0x55bd94694340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.248+0000 7f17c9e00640 1 -- 172.21.15.97:0/422800501 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd94699a80 msgr2=0x55bd9469bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:55:47.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.248+0000 7f17c9e00640 1 --2- 172.21.15.97:0/422800501 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd94699a80 0x55bd9469bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.248+0000 7f17c9e00640 1 -- 172.21.15.97:0/422800501 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd9430d2d0 con 0x55bd94694910 2024-03-27T10:55:47.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.248+0000 7f17c1400640 1 --2- 172.21.15.97:0/422800501 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd94699a80 0x55bd9469bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:55:47.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.248+0000 7f17c9e00640 1 --2- 172.21.15.97:0/422800501 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd94694910 0x55bd94699410 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f17bc00b7d0 tx=0x7f17bc004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc1b06ea345dd405 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:55:47.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.248+0000 7f17c3400640 1 --2- 172.21.15.97:0/3888939373 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f17b00a3610 0x7f17b00a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:47.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.248+0000 7f17bb400640 1 -- 172.21.15.97:0/422800501 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17bc003dd0 con 0x55bd94694910 2024-03-27T10:55:47.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.248+0000 7f17bb400640 1 -- 172.21.15.97:0/422800501 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f17bc015070 con 0x55bd94694910 2024-03-27T10:55:47.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.248+0000 7f17bb400640 1 -- 172.21.15.97:0/422800501 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17bc011730 con 0x55bd94694910 2024-03-27T10:55:47.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.248+0000 7f17c3400640 1 -- 172.21.15.97:0/3888939373 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f17c40cfe60 con 0x7f17cc109d40 2024-03-27T10:55:47.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.248+0000 7f17c1400640 1 --2- 172.21.15.97:0/3888939373 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f17b00a3610 0x7f17b00a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:47.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.249+0000 7f17d3e26740 1 -- 172.21.15.97:0/422800501 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd94694910 msgr2=0x55bd94699410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:47.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.249+0000 7f17d3e26740 1 --2- 172.21.15.97:0/422800501 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd94694910 0x55bd94699410 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f17bc00b7d0 tx=0x7f17bc004610 comp rx=0 tx=0).stop 2024-03-27T10:55:47.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.249+0000 7f17c1400640 1 --2- 172.21.15.97:0/3888939373 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f17b00a3610 0x7f17b00a5ad0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f17cc19f870 tx=0x7f17b4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:55:47.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.249+0000 7f17d3e26740 1 -- 172.21.15.97:0/422800501 shutdown_connections 2024-03-27T10:55:47.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.249+0000 7f17d3e26740 1 --2- 172.21.15.97:0/422800501 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd94699a80 0x55bd9469bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.249+0000 7f17d3e26740 1 --2- 172.21.15.97:0/422800501 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd94694910 0x55bd94699410 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.249+0000 7f17d3e26740 1 --2- 172.21.15.97:0/422800501 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bd94693f20 0x55bd94694340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.249+0000 7f17d3e26740 1 -- 172.21.15.97:0/422800501 >> 172.21.15.97:0/422800501 conn(0x55bd9468f490 msgr2=0x55bd9468f890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:55:47.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.249+0000 7f17d3e26740 1 -- 172.21.15.97:0/422800501 shutdown_connections 2024-03-27T10:55:47.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.249+0000 7f17d3e26740 1 -- 172.21.15.97:0/422800501 wait complete. 2024-03-27T10:55:47.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.250+0000 7f17d3e26740 1 -- start start 2024-03-27T10:55:47.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.250+0000 7f17d3e26740 1 Processor -- start 2024-03-27T10:55:47.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.250+0000 7f17d3e26740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bd94693f20 0x55bd9469e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:47.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.250+0000 7f17d3e26740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd94694910 0x55bd9469e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:47.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.250+0000 7f17d3e26740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd94699a80 0x55bd946a5690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:47.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.250+0000 7f17d3e26740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55bd945f4f40 con 0x55bd94694910 2024-03-27T10:55:47.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.250+0000 7f17d3e26740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55bd945a1650 con 0x55bd94699a80 2024-03-27T10:55:47.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.250+0000 7f17d3e26740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55bd945b4a10 con 0x55bd94693f20 2024-03-27T10:55:47.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.250+0000 7f17d3e26740 10 client.0 _ll_register_callbacks cb 0x55bd945f3ff0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:55:47.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.250+0000 7f17c1400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd94694910 0x55bd9469e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:47.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.251+0000 7f17c1400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd94694910 0x55bd9469e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40910/0 (socket says 172.21.15.97:40910) 2024-03-27T10:55:47.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.251+0000 7f17c1400640 1 -- 172.21.15.97:0/1779655394 learned_addr learned my addr 172.21.15.97:0/1779655394 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:55:47.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.251+0000 7f17ca800640 1 --2- 172.21.15.97:0/1779655394 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd94699a80 0x55bd946a5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:47.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.251+0000 7f17c9e00640 1 --2- 172.21.15.97:0/1779655394 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bd94693f20 0x55bd9469e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:47.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.251+0000 7f17c1400640 1 -- 172.21.15.97:0/1779655394 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bd94693f20 msgr2=0x55bd9469e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:55:47.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.251+0000 7f17c1400640 1 --2- 172.21.15.97:0/1779655394 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bd94693f20 0x55bd9469e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.251+0000 7f17c1400640 1 -- 172.21.15.97:0/1779655394 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd94699a80 msgr2=0x55bd946a5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:55:47.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.251+0000 7f17c1400640 1 --2- 172.21.15.97:0/1779655394 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd94699a80 0x55bd946a5690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.251+0000 7f17c1400640 1 -- 172.21.15.97:0/1779655394 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd943d76b0 con 0x55bd94694910 2024-03-27T10:55:47.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.251+0000 7f17c1400640 1 --2- 172.21.15.97:0/1779655394 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd94694910 0x55bd9469e650 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f17b4009900 tx=0x7f17b4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8685eedfebf60248 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:55:47.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.251+0000 7f17afe00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T10:55:47.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.252+0000 7f17afe00640 1 -- 172.21.15.97:0/1779655394 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17b404f020 con 0x55bd94694910 2024-03-27T10:55:47.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.252+0000 7f17afe00640 1 -- 172.21.15.97:0/1779655394 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f17b40048d0 con 0x55bd94694910 2024-03-27T10:55:47.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.252+0000 7f17d3e26740 1 -- 172.21.15.97:0/1779655394 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17bc015070 con 0x55bd94694910 2024-03-27T10:55:47.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.252+0000 7f17afe00640 1 -- 172.21.15.97:0/1779655394 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f17b40055d0 con 0x55bd94694910 2024-03-27T10:55:47.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.252+0000 7f17d3e26740 10 client.15111 fetch_fsmap learned FSMap version 27 2024-03-27T10:55:47.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.252+0000 7f17d3e26740 1 -- 172.21.15.97:0/1779655394 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55bd946a89e0 con 0x55bd94694910 2024-03-27T10:55:47.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.252+0000 7f17afe00640 1 -- 172.21.15.97:0/1779655394 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f17bc015070 con 0x55bd94694910 2024-03-27T10:55:47.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.252+0000 7f17afe00640 1 -- 172.21.15.97:0/1779655394 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f17b4003c40 con 0x55bd94694910 2024-03-27T10:55:47.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.252+0000 7f17d3e26740 10 client.15111 fetch_fsmap finished waiting for FSMap version 27 2024-03-27T10:55:47.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.252+0000 7f17d3e26740 10 client.15111 resolve_mds: resolved name 'e' to daemon mds.e 2024-03-27T10:55:47.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.253+0000 7f17d3e26740 20 client.15111 populate_metadata read hostname 'smithi097' 2024-03-27T10:55:47.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.253+0000 7f17d3e26740 1 --2- 172.21.15.97:0/1779655394 >> [v2:172.21.15.114:6834/3581918222,v1:172.21.15.114:6835/3581918222] conn(0x55bd946ab0f0 0x55bd946ac560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:47.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.253+0000 7f17d3e26740 4 client.15111 mds_command: new command op to 34445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T10:55:47.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.253+0000 7f17d3e26740 1 -- 172.21.15.97:0/1779655394 --> [v2:172.21.15.114:6834/3581918222,v1:172.21.15.114:6835/3581918222] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bd946064d0 con 0x55bd946ab0f0 2024-03-27T10:55:47.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.253+0000 7f17c9e00640 1 --2- 172.21.15.97:0/1779655394 >> [v2:172.21.15.114:6834/3581918222,v1:172.21.15.114:6835/3581918222] conn(0x55bd946ab0f0 0x55bd946ac560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:47.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.254+0000 7f17c9e00640 1 --2- 172.21.15.97:0/1779655394 >> [v2:172.21.15.114:6834/3581918222,v1:172.21.15.114:6835/3581918222] conn(0x55bd946ab0f0 0x55bd946ac560 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8fa029b479a5ca8b server_cookie=ab4fadbdcaf2ee30 in_seq=0 out_seq=0 2024-03-27T10:55:47.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.254+0000 7f17afe00640 10 client.15111 ms_handle_connect on v2:172.21.15.114:6834/3581918222 2024-03-27T10:55:47.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.255+0000 7f17afe00640 1 -- 172.21.15.97:0/1779655394 <== mds.2 v2:172.21.15.114:6834/3581918222 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55bd946064d0 con 0x55bd946ab0f0 2024-03-27T10:55:47.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.255+0000 7f17afe00640 10 client.15111 handle_command_reply: tid=0 2024-03-27T10:55:47.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.255+0000 7f17d3e26740 1 client.15111 shutdown 2024-03-27T10:55:47.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.256+0000 7f17d3e26740 1 -- 172.21.15.97:0/1779655394 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd94694910 msgr2=0x55bd9469e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:47.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.256+0000 7f17d3e26740 1 --2- 172.21.15.97:0/1779655394 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd94694910 0x55bd9469e650 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f17b4009900 tx=0x7f17b4000bc0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.256+0000 7f17d3e26740 1 -- 172.21.15.97:0/1779655394 shutdown_connections 2024-03-27T10:55:47.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.256+0000 7f17d3e26740 1 --2- 172.21.15.97:0/1779655394 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd94699a80 0x55bd946a5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.256+0000 7f17d3e26740 1 --2- 172.21.15.97:0/1779655394 >> [v2:172.21.15.114:6834/3581918222,v1:172.21.15.114:6835/3581918222] conn(0x55bd946ab0f0 0x55bd946ac560 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.256+0000 7f17d3e26740 1 --2- 172.21.15.97:0/1779655394 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd94694910 0x55bd9469e650 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.256+0000 7f17d3e26740 1 --2- 172.21.15.97:0/1779655394 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bd94693f20 0x55bd9469e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.256+0000 7f17afe00640 0 client.15111 ms_handle_reset on v2:172.21.15.114:6834/3581918222 2024-03-27T10:55:47.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.256+0000 7f17d3e26740 1 -- 172.21.15.97:0/1779655394 >> 172.21.15.97:0/1779655394 conn(0x55bd9468f490 msgr2=0x55bd9468e690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:55:47.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.256+0000 7f17d3e26740 1 -- 172.21.15.97:0/1779655394 shutdown_connections 2024-03-27T10:55:47.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.256+0000 7f17d3e26740 1 -- 172.21.15.97:0/1779655394 wait complete. 2024-03-27T10:55:47.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.256+0000 7f17d3e26740 20 client.15111 trim_cache size 0 max 16384 2024-03-27T10:55:47.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.275+0000 7f17d3e26740 1 Processor -- start 2024-03-27T10:55:47.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.276+0000 7f17d3e26740 1 -- start start 2024-03-27T10:55:47.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.276+0000 7f17d3e26740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd94693f20 0x55bd94695dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:47.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.276+0000 7f17d3e26740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bd94694910 0x55bd94696310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:47.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.276+0000 7f17d3e26740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd94699a80 0x55bd946a5650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:47.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.276+0000 7f17d3e26740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55bd946aa8a0 con 0x55bd94699a80 2024-03-27T10:55:47.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.276+0000 7f17d3e26740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55bd946aaf70 con 0x55bd94693f20 2024-03-27T10:55:47.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.276+0000 7f17d3e26740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55bd946ab640 con 0x55bd94694910 2024-03-27T10:55:47.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.276+0000 7f17ca800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd94699a80 0x55bd946a5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:47.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.276+0000 7f17ca800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd94699a80 0x55bd946a5650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40926/0 (socket says 172.21.15.97:40926) 2024-03-27T10:55:47.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.276+0000 7f17ca800640 1 -- 172.21.15.97:0/2245020152 learned_addr learned my addr 172.21.15.97:0/2245020152 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:55:47.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.276+0000 7f17c9e00640 1 --2- 172.21.15.97:0/2245020152 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd94693f20 0x55bd94695dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:47.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.276+0000 7f17c1400640 1 --2- 172.21.15.97:0/2245020152 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bd94694910 0x55bd94696310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:47.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.277+0000 7f17ca800640 1 -- 172.21.15.97:0/2245020152 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bd94694910 msgr2=0x55bd94696310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:55:47.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.277+0000 7f17ca800640 1 --2- 172.21.15.97:0/2245020152 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bd94694910 0x55bd94696310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.277+0000 7f17ca800640 1 -- 172.21.15.97:0/2245020152 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd94693f20 msgr2=0x55bd94695dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:55:47.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.277+0000 7f17ca800640 1 --2- 172.21.15.97:0/2245020152 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd94693f20 0x55bd94695dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.277+0000 7f17ca800640 1 -- 172.21.15.97:0/2245020152 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd9469f290 con 0x55bd94699a80 2024-03-27T10:55:47.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.277+0000 7f17c9e00640 1 --2- 172.21.15.97:0/2245020152 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd94693f20 0x55bd94695dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T10:55:47.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.277+0000 7f17ca800640 1 --2- 172.21.15.97:0/2245020152 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd94699a80 0x55bd946a5650 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f17c40cd9c0 tx=0x7f17c409cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5436bd5adab9aa49 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:55:47.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.277+0000 7f17baa00640 1 -- 172.21.15.97:0/2245020152 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c401f040 con 0x55bd94699a80 2024-03-27T10:55:47.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.277+0000 7f17baa00640 1 -- 172.21.15.97:0/2245020152 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f17c40b5070 con 0x55bd94699a80 2024-03-27T10:55:47.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.277+0000 7f17baa00640 1 -- 172.21.15.97:0/2245020152 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c40a0170 con 0x55bd94699a80 2024-03-27T10:55:47.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.278+0000 7f17d3e26740 1 -- 172.21.15.97:0/2245020152 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd94699a80 msgr2=0x55bd946a5650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:47.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.278+0000 7f17d3e26740 1 --2- 172.21.15.97:0/2245020152 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd94699a80 0x55bd946a5650 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f17c40cd9c0 tx=0x7f17c409cb00 comp rx=0 tx=0).stop 2024-03-27T10:55:47.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.278+0000 7f17d3e26740 1 -- 172.21.15.97:0/2245020152 shutdown_connections 2024-03-27T10:55:47.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.278+0000 7f17d3e26740 1 --2- 172.21.15.97:0/2245020152 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd94699a80 0x55bd946a5650 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.278+0000 7f17d3e26740 1 --2- 172.21.15.97:0/2245020152 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bd94694910 0x55bd94696310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.278+0000 7f17d3e26740 1 --2- 172.21.15.97:0/2245020152 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd94693f20 0x55bd94695dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.278+0000 7f17d3e26740 1 -- 172.21.15.97:0/2245020152 >> 172.21.15.97:0/2245020152 conn(0x55bd9468f490 msgr2=0x55bd9468f870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:55:47.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.278+0000 7f17d3e26740 1 -- 172.21.15.97:0/2245020152 shutdown_connections 2024-03-27T10:55:47.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.278+0000 7f17d3e26740 1 -- 172.21.15.97:0/2245020152 wait complete. 2024-03-27T10:55:47.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.278+0000 7f17d3e26740 1 -- start start 2024-03-27T10:55:47.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.279+0000 7f17d3e26740 1 Processor -- start 2024-03-27T10:55:47.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.279+0000 7f17d3e26740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd94693f20 0x55bd946a1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:47.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.279+0000 7f17d3e26740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bd94694910 0x55bd946a17d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:47.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.279+0000 7f17d3e26740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd94699a80 0x55bd946a47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:47.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.279+0000 7f17d3e26740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55bd946ac3e0 con 0x55bd94699a80 2024-03-27T10:55:47.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.279+0000 7f17d3e26740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55bd946abd10 con 0x55bd94693f20 2024-03-27T10:55:47.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.279+0000 7f17d3e26740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55bd946acab0 con 0x55bd94694910 2024-03-27T10:55:47.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.279+0000 7f17d3e26740 10 client.0 _ll_register_callbacks cb 0x7f17b4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:55:47.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.279+0000 7f17ca800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd94699a80 0x55bd946a47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:47.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.279+0000 7f17ca800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd94699a80 0x55bd946a47f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40936/0 (socket says 172.21.15.97:40936) 2024-03-27T10:55:47.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.279+0000 7f17ca800640 1 -- 172.21.15.97:0/3715681976 learned_addr learned my addr 172.21.15.97:0/3715681976 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:55:47.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.279+0000 7f17c9e00640 1 --2- 172.21.15.97:0/3715681976 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd94693f20 0x55bd946a1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:47.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.279+0000 7f17c1400640 1 --2- 172.21.15.97:0/3715681976 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bd94694910 0x55bd946a17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:47.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.280+0000 7f17ca800640 1 -- 172.21.15.97:0/3715681976 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bd94694910 msgr2=0x55bd946a17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:55:47.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.280+0000 7f17ca800640 1 --2- 172.21.15.97:0/3715681976 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bd94694910 0x55bd946a17d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.280+0000 7f17ca800640 1 -- 172.21.15.97:0/3715681976 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd94693f20 msgr2=0x55bd946a1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:55:47.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.280+0000 7f17ca800640 1 --2- 172.21.15.97:0/3715681976 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd94693f20 0x55bd946a1290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.280+0000 7f17ca800640 1 -- 172.21.15.97:0/3715681976 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd946a4de0 con 0x55bd94699a80 2024-03-27T10:55:47.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.280+0000 7f17c1400640 1 --2- 172.21.15.97:0/3715681976 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bd94694910 0x55bd946a17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:55:47.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.280+0000 7f17ca800640 1 --2- 172.21.15.97:0/3715681976 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd94699a80 0x55bd946a47f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f17c409c040 tx=0x7f17c40a4fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16a39e954572b097 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:55:47.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.280+0000 7f17afe00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T10:55:47.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.280+0000 7f17afe00640 1 -- 172.21.15.97:0/3715681976 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c401f070 con 0x55bd94699a80 2024-03-27T10:55:47.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.280+0000 7f17afe00640 1 -- 172.21.15.97:0/3715681976 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f17c409a640 con 0x55bd94699a80 2024-03-27T10:55:47.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.280+0000 7f17d3e26740 1 -- 172.21.15.97:0/3715681976 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17c40b5070 con 0x55bd94699a80 2024-03-27T10:55:47.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.280+0000 7f17afe00640 1 -- 172.21.15.97:0/3715681976 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f17c4005560 con 0x55bd94699a80 2024-03-27T10:55:47.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.280+0000 7f17d3e26740 10 client.15117 fetch_fsmap learned FSMap version 27 2024-03-27T10:55:47.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.280+0000 7f17d3e26740 1 -- 172.21.15.97:0/3715681976 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55bd947112c0 con 0x55bd94699a80 2024-03-27T10:55:47.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.280+0000 7f17afe00640 1 -- 172.21.15.97:0/3715681976 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f17c40b5070 con 0x55bd94699a80 2024-03-27T10:55:47.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.281+0000 7f17afe00640 1 -- 172.21.15.97:0/3715681976 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f17c400f040 con 0x55bd94699a80 2024-03-27T10:55:47.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.281+0000 7f17d3e26740 10 client.15117 fetch_fsmap finished waiting for FSMap version 27 2024-03-27T10:55:47.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.281+0000 7f17d3e26740 10 client.15117 resolve_mds: resolved name 'e' to daemon mds.e 2024-03-27T10:55:47.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.281+0000 7f17d3e26740 20 client.15117 populate_metadata read hostname 'smithi097' 2024-03-27T10:55:47.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.281+0000 7f17d3e26740 1 --2- 172.21.15.97:0/3715681976 >> [v2:172.21.15.114:6834/3581918222,v1:172.21.15.114:6835/3581918222] conn(0x55bd947159e0 0x55bd947146f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:47.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.281+0000 7f17d3e26740 4 client.15117 mds_command: new command op to 34445 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-27T10:55:47.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.281+0000 7f17d3e26740 1 -- 172.21.15.97:0/3715681976 --> [v2:172.21.15.114:6834/3581918222,v1:172.21.15.114:6835/3581918222] -- command(tid 0: {"prefix": "status"}) -- 0x55bd94600820 con 0x55bd947159e0 2024-03-27T10:55:47.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.281+0000 7f17c9e00640 1 --2- 172.21.15.97:0/3715681976 >> [v2:172.21.15.114:6834/3581918222,v1:172.21.15.114:6835/3581918222] conn(0x55bd947159e0 0x55bd947146f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:47.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.282+0000 7f17c9e00640 1 --2- 172.21.15.97:0/3715681976 >> [v2:172.21.15.114:6834/3581918222,v1:172.21.15.114:6835/3581918222] conn(0x55bd947159e0 0x55bd947146f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d9d9352a1f018dbc server_cookie=60c609715017881 in_seq=0 out_seq=0 2024-03-27T10:55:47.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.282+0000 7f17afe00640 10 client.15117 ms_handle_connect on v2:172.21.15.114:6834/3581918222 2024-03-27T10:55:47.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.283+0000 7f17afe00640 1 -- 172.21.15.97:0/3715681976 <== mds.2 v2:172.21.15.114:6834/3581918222 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55bd94600820 con 0x55bd947159e0 2024-03-27T10:55:47.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.283+0000 7f17afe00640 10 client.15117 handle_command_reply: tid=0 2024-03-27T10:55:47.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.283+0000 7f17d3e26740 1 client.15117 shutdown 2024-03-27T10:55:47.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.283+0000 7f17d3e26740 1 -- 172.21.15.97:0/3715681976 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd94699a80 msgr2=0x55bd946a47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:47.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.283+0000 7f17d3e26740 1 --2- 172.21.15.97:0/3715681976 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd94699a80 0x55bd946a47f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f17c409c040 tx=0x7f17c40a4fb0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.283+0000 7f17d3e26740 1 -- 172.21.15.97:0/3715681976 shutdown_connections 2024-03-27T10:55:47.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.283+0000 7f17d3e26740 1 --2- 172.21.15.97:0/3715681976 >> [v2:172.21.15.114:6834/3581918222,v1:172.21.15.114:6835/3581918222] conn(0x55bd947159e0 0x55bd947146f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.283+0000 7f17d3e26740 1 --2- 172.21.15.97:0/3715681976 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd94699a80 0x55bd946a47f0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.283+0000 7f17d3e26740 1 --2- 172.21.15.97:0/3715681976 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bd94694910 0x55bd946a17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.283+0000 7f17d3e26740 1 --2- 172.21.15.97:0/3715681976 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd94693f20 0x55bd946a1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.283+0000 7f17afe00640 0 client.15117 ms_handle_reset on v2:172.21.15.114:6834/3581918222 2024-03-27T10:55:47.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.283+0000 7f17d3e26740 1 -- 172.21.15.97:0/3715681976 >> 172.21.15.97:0/3715681976 conn(0x55bd9468f490 msgr2=0x55bd94695dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:55:47.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.283+0000 7f17d3e26740 1 -- 172.21.15.97:0/3715681976 shutdown_connections 2024-03-27T10:55:47.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.283+0000 7f17d3e26740 1 -- 172.21.15.97:0/3715681976 wait complete. 2024-03-27T10:55:47.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.283+0000 7f17d3e26740 20 client.15117 trim_cache size 0 max 16384 2024-03-27T10:55:47.259 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-03-27T10:55:47.259 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "adca88c4-ec26-11ee-95d0-87774f69a715", 2024-03-27T10:55:47.259 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 2, 2024-03-27T10:55:47.259 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 34445, 2024-03-27T10:55:47.259 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2024-03-27T10:55:47.259 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2024-03-27T10:55:47.259 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2024-03-27T10:55:47.259 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 97.679338888999993, 2024-03-27T10:55:47.260 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 27, 2024-03-27T10:55:47.260 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 92, 2024-03-27T10:55:47.260 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 90, 2024-03-27T10:55:47.260 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 134.64528395900001 2024-03-27T10:55:47.260 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-03-27T10:55:47.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.284+0000 7f17d1000640 1 -- 172.21.15.97:0/3888939373 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f17b00a3610 msgr2=0x7f17b00a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:47.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.284+0000 7f17d1000640 1 --2- 172.21.15.97:0/3888939373 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f17b00a3610 0x7f17b00a5ad0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f17cc19f870 tx=0x7f17b4009260 comp rx=0 tx=0).stop 2024-03-27T10:55:47.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.284+0000 7f17d1000640 1 -- 172.21.15.97:0/3888939373 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17cc109d40 msgr2=0x7f17cc1a5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:47.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.284+0000 7f17d1000640 1 --2- 172.21.15.97:0/3888939373 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17cc109d40 0x7f17cc1a5910 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f17c400eaa0 tx=0x7f17c400ef60 comp rx=0 tx=0).stop 2024-03-27T10:55:47.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.284+0000 7f17d1000640 1 -- 172.21.15.97:0/3888939373 shutdown_connections 2024-03-27T10:55:47.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.284+0000 7f17d1000640 1 --2- 172.21.15.97:0/3888939373 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17cc109d40 0x7f17cc1a5910 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.284+0000 7f17d1000640 1 --2- 172.21.15.97:0/3888939373 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f17cc109380 0x7f17cc19e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.284+0000 7f17d1000640 1 --2- 172.21.15.97:0/3888939373 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f17b00a3610 0x7f17b00a5ad0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.284+0000 7f17d1000640 1 --2- 172.21.15.97:0/3888939373 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f17cc108190 0x7f17cc19e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.284+0000 7f17d1000640 1 -- 172.21.15.97:0/3888939373 >> 172.21.15.97:0/3888939373 conn(0x7f17cc077120 msgr2=0x7f17cc10c680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:55:47.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.284+0000 7f17d1000640 1 -- 172.21.15.97:0/3888939373 shutdown_connections 2024-03-27T10:55:47.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.284+0000 7f17d1000640 1 -- 172.21.15.97:0/3888939373 wait complete. 2024-03-27T10:55:47.272 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-03-27T10:55:47.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.500+0000 7fb8a5800640 1 -- 172.21.15.97:0/1368957304 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8a0105730 msgr2=0x7fb8a010bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:47.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.500+0000 7fb8a5800640 1 --2- 172.21.15.97:0/1368957304 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8a0105730 0x7fb8a010bfc0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb898009900 tx=0x7fb89802bc30 comp rx=0 tx=0).stop 2024-03-27T10:55:47.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.500+0000 7fb8a5800640 1 -- 172.21.15.97:0/1368957304 shutdown_connections 2024-03-27T10:55:47.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.500+0000 7fb8a5800640 1 --2- 172.21.15.97:0/1368957304 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8a0105730 0x7fb8a010bfc0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.500+0000 7fb8a5800640 1 --2- 172.21.15.97:0/1368957304 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb8a0104d70 0x7fb8a01051f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.500+0000 7fb8a5800640 1 --2- 172.21.15.97:0/1368957304 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb8a0103b80 0x7fb8a0103fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.500+0000 7fb8a5800640 1 -- 172.21.15.97:0/1368957304 >> 172.21.15.97:0/1368957304 conn(0x7fb8a00ff0f0 msgr2=0x7fb8a0101530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:55:47.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.501+0000 7fb8a5800640 1 -- 172.21.15.97:0/1368957304 shutdown_connections 2024-03-27T10:55:47.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.501+0000 7fb8a5800640 1 -- 172.21.15.97:0/1368957304 wait complete. 2024-03-27T10:55:47.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.501+0000 7fb8a5800640 1 Processor -- start 2024-03-27T10:55:47.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.502+0000 7fb8a5800640 1 -- start start 2024-03-27T10:55:47.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.502+0000 7fb8a5800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb8a0103b80 0x7fb8a0199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:47.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.502+0000 7fb8a5800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8a0104d70 0x7fb8a019a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:47.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.502+0000 7fb8a5800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb8a0105730 0x7fb8a01a1300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:47.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.502+0000 7fb8a5800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb8a010f6b0 con 0x7fb8a0104d70 2024-03-27T10:55:47.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.502+0000 7fb8a5800640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb8a010f530 con 0x7fb8a0105730 2024-03-27T10:55:47.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.502+0000 7fb8a5800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb8a010f830 con 0x7fb8a0103b80 2024-03-27T10:55:47.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.502+0000 7fb897400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8a0104d70 0x7fb8a019a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:47.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.502+0000 7fb897400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8a0104d70 0x7fb8a019a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40956/0 (socket says 172.21.15.97:40956) 2024-03-27T10:55:47.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.502+0000 7fb897400640 1 -- 172.21.15.97:0/1059456944 learned_addr learned my addr 172.21.15.97:0/1059456944 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:55:47.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.502+0000 7fb8a4e00640 1 --2- 172.21.15.97:0/1059456944 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb8a0105730 0x7fb8a01a1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:47.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.502+0000 7fb89fe00640 1 --2- 172.21.15.97:0/1059456944 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb8a0103b80 0x7fb8a0199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:47.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.503+0000 7fb897400640 1 -- 172.21.15.97:0/1059456944 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb8a0103b80 msgr2=0x7fb8a0199d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:47.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.503+0000 7fb897400640 1 --2- 172.21.15.97:0/1059456944 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb8a0103b80 0x7fb8a0199d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.503+0000 7fb897400640 1 -- 172.21.15.97:0/1059456944 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb8a0105730 msgr2=0x7fb8a01a1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:47.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.503+0000 7fb897400640 1 --2- 172.21.15.97:0/1059456944 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb8a0105730 0x7fb8a01a1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.503+0000 7fb897400640 1 -- 172.21.15.97:0/1059456944 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8a01a1970 con 0x7fb8a0104d70 2024-03-27T10:55:47.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.503+0000 7fb8a4e00640 1 --2- 172.21.15.97:0/1059456944 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb8a0105730 0x7fb8a01a1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:55:47.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.503+0000 7fb897400640 1 --2- 172.21.15.97:0/1059456944 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8a0104d70 0x7fb8a019a290 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb88800d770 tx=0x7fb88800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:55:47.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.503+0000 7fb89d600640 1 -- 172.21.15.97:0/1059456944 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb888017070 con 0x7fb8a0104d70 2024-03-27T10:55:47.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.503+0000 7fb8a5800640 1 -- 172.21.15.97:0/1059456944 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8a01a1c60 con 0x7fb8a0104d70 2024-03-27T10:55:47.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.503+0000 7fb89d600640 1 -- 172.21.15.97:0/1059456944 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb8880054d0 con 0x7fb8a0104d70 2024-03-27T10:55:47.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.503+0000 7fb89d600640 1 -- 172.21.15.97:0/1059456944 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb888004cf0 con 0x7fb8a0104d70 2024-03-27T10:55:47.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.503+0000 7fb8a5800640 1 -- 172.21.15.97:0/1059456944 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8a01a2160 con 0x7fb8a0104d70 2024-03-27T10:55:47.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.506+0000 7fb89d600640 1 -- 172.21.15.97:0/1059456944 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7fb888004e90 con 0x7fb8a0104d70 2024-03-27T10:55:47.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.506+0000 7fb8a9b7d740 1 Processor -- start 2024-03-27T10:55:47.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.507+0000 7fb8a9b7d740 1 -- start start 2024-03-27T10:55:47.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.507+0000 7fb8a9b7d740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d0aeafc120 0x55d0aeafc540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:47.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.507+0000 7fb8a9b7d740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0aeafcb10 0x55d0aeb01610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:47.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.507+0000 7fb8a9b7d740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d0aeb01c80 0x55d0aeb04110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:47.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.507+0000 7fb8a9b7d740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55d0aeabe920 con 0x55d0aeafcb10 2024-03-27T10:55:47.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.507+0000 7fb8a9b7d740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55d0aea0efa0 con 0x55d0aeafc120 2024-03-27T10:55:47.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.507+0000 7fb8a9b7d740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55d0aea60f90 con 0x55d0aeb01c80 2024-03-27T10:55:47.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.507+0000 7fb89fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0aeafcb10 0x55d0aeb01610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:47.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.507+0000 7fb89fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0aeafcb10 0x55d0aeb01610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40972/0 (socket says 172.21.15.97:40972) 2024-03-27T10:55:47.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.507+0000 7fb89fe00640 1 -- 172.21.15.97:0/977427670 learned_addr learned my addr 172.21.15.97:0/977427670 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:55:47.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.507+0000 7fb8a4e00640 1 --2- 172.21.15.97:0/977427670 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d0aeafc120 0x55d0aeafc540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:47.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.508+0000 7fb897400640 1 --2- 172.21.15.97:0/977427670 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d0aeb01c80 0x55d0aeb04110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:47.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.508+0000 7fb89fe00640 1 -- 172.21.15.97:0/977427670 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d0aeb01c80 msgr2=0x55d0aeb04110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:55:47.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.508+0000 7fb89fe00640 1 --2- 172.21.15.97:0/977427670 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d0aeb01c80 0x55d0aeb04110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.508+0000 7fb89fe00640 1 -- 172.21.15.97:0/977427670 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d0aeafc120 msgr2=0x55d0aeafc540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:55:47.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.508+0000 7fb89fe00640 1 --2- 172.21.15.97:0/977427670 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d0aeafc120 0x55d0aeafc540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.508+0000 7fb89fe00640 1 -- 172.21.15.97:0/977427670 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0ae7752d0 con 0x55d0aeafcb10 2024-03-27T10:55:47.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.508+0000 7fb897400640 1 --2- 172.21.15.97:0/977427670 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d0aeb01c80 0x55d0aeb04110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:55:47.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.508+0000 7fb8a4e00640 1 --2- 172.21.15.97:0/977427670 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d0aeafc120 0x55d0aeafc540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T10:55:47.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.508+0000 7fb89fe00640 1 --2- 172.21.15.97:0/977427670 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0aeafcb10 0x55d0aeb01610 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fb890009f50 tx=0x7fb89000e780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f140e61bf29b674 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:55:47.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.508+0000 7fb89d600640 1 --2- 172.21.15.97:0/1059456944 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fb8840a3610 0x7fb8840a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:47.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.508+0000 7fb895600640 1 -- 172.21.15.97:0/977427670 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb890016070 con 0x55d0aeafcb10 2024-03-27T10:55:47.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.508+0000 7fb89d600640 1 -- 172.21.15.97:0/1059456944 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fb8880cf7d0 con 0x7fb8a0104d70 2024-03-27T10:55:47.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.508+0000 7fb895600640 1 -- 172.21.15.97:0/977427670 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb890011b20 con 0x55d0aeafcb10 2024-03-27T10:55:47.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.508+0000 7fb8a4e00640 1 --2- 172.21.15.97:0/1059456944 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fb8840a3610 0x7fb8840a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:47.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.508+0000 7fb895600640 1 -- 172.21.15.97:0/977427670 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb890010b10 con 0x55d0aeafcb10 2024-03-27T10:55:47.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.509+0000 7fb8a9b7d740 1 -- 172.21.15.97:0/977427670 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0aeafcb10 msgr2=0x55d0aeb01610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:47.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.509+0000 7fb8a4e00640 1 --2- 172.21.15.97:0/1059456944 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fb8840a3610 0x7fb8840a5ad0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb898005ab0 tx=0x7fb898005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:55:47.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.509+0000 7fb8a9b7d740 1 --2- 172.21.15.97:0/977427670 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0aeafcb10 0x55d0aeb01610 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fb890009f50 tx=0x7fb89000e780 comp rx=0 tx=0).stop 2024-03-27T10:55:47.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.509+0000 7fb8a9b7d740 1 -- 172.21.15.97:0/977427670 shutdown_connections 2024-03-27T10:55:47.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.509+0000 7fb8a9b7d740 1 --2- 172.21.15.97:0/977427670 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d0aeb01c80 0x55d0aeb04110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.509+0000 7fb8a9b7d740 1 --2- 172.21.15.97:0/977427670 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0aeafcb10 0x55d0aeb01610 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.509+0000 7fb8a9b7d740 1 --2- 172.21.15.97:0/977427670 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d0aeafc120 0x55d0aeafc540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.509+0000 7fb8a9b7d740 1 -- 172.21.15.97:0/977427670 >> 172.21.15.97:0/977427670 conn(0x55d0aeaf7690 msgr2=0x55d0aeaf7a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:55:47.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.510+0000 7fb8a9b7d740 1 -- 172.21.15.97:0/977427670 shutdown_connections 2024-03-27T10:55:47.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.510+0000 7fb8a9b7d740 1 -- 172.21.15.97:0/977427670 wait complete. 2024-03-27T10:55:47.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.510+0000 7fb8a9b7d740 1 -- start start 2024-03-27T10:55:47.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.510+0000 7fb8a9b7d740 1 Processor -- start 2024-03-27T10:55:47.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.511+0000 7fb8a9b7d740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d0aeafc120 0x55d0aeb06360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:47.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.511+0000 7fb8a9b7d740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d0aeafcb10 0x55d0aeb068a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:47.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.511+0000 7fb8a9b7d740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0aeb01c80 0x55d0aeb0d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:47.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.511+0000 7fb8a9b7d740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55d0aea5cf40 con 0x55d0aeb01c80 2024-03-27T10:55:47.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.511+0000 7fb8a9b7d740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55d0aea09650 con 0x55d0aeafcb10 2024-03-27T10:55:47.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.511+0000 7fb8a9b7d740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55d0aea1ca10 con 0x55d0aeafc120 2024-03-27T10:55:47.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.511+0000 7fb8a9b7d740 10 client.0 _ll_register_callbacks cb 0x55d0aea5bff0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:55:47.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.511+0000 7fb8a4e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0aeb01c80 0x55d0aeb0d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:47.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.511+0000 7fb8a4e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0aeb01c80 0x55d0aeb0d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40988/0 (socket says 172.21.15.97:40988) 2024-03-27T10:55:47.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.511+0000 7fb8a4e00640 1 -- 172.21.15.97:0/383563734 learned_addr learned my addr 172.21.15.97:0/383563734 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:55:47.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.511+0000 7fb897400640 1 --2- 172.21.15.97:0/383563734 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d0aeafcb10 0x55d0aeb068a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:47.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.511+0000 7fb89fe00640 1 --2- 172.21.15.97:0/383563734 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d0aeafc120 0x55d0aeb06360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:47.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.511+0000 7fb8a4e00640 1 -- 172.21.15.97:0/383563734 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d0aeafc120 msgr2=0x55d0aeb06360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:55:47.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.511+0000 7fb8a4e00640 1 --2- 172.21.15.97:0/383563734 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d0aeafc120 0x55d0aeb06360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.511+0000 7fb8a4e00640 1 -- 172.21.15.97:0/383563734 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d0aeafcb10 msgr2=0x55d0aeb068a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:55:47.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.511+0000 7fb8a4e00640 1 --2- 172.21.15.97:0/383563734 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d0aeafcb10 0x55d0aeb068a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.512+0000 7fb8a4e00640 1 -- 172.21.15.97:0/383563734 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0ae83f6b0 con 0x55d0aeb01c80 2024-03-27T10:55:47.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.512+0000 7fb8a4e00640 1 --2- 172.21.15.97:0/383563734 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0aeb01c80 0x55d0aeb0d8e0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fb898035000 tx=0x7fb89802bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=907fd4de2ee381ef server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:55:47.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.512+0000 7fb88e000640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T10:55:47.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.512+0000 7fb88e000640 1 -- 172.21.15.97:0/383563734 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb89804f020 con 0x55d0aeb01c80 2024-03-27T10:55:47.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.512+0000 7fb88e000640 1 -- 172.21.15.97:0/383563734 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fb898004ad0 con 0x55d0aeb01c80 2024-03-27T10:55:47.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.512+0000 7fb8a9b7d740 1 -- 172.21.15.97:0/383563734 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb890011b20 con 0x55d0aeb01c80 2024-03-27T10:55:47.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.512+0000 7fb88e000640 1 -- 172.21.15.97:0/383563734 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb8980055c0 con 0x55d0aeb01c80 2024-03-27T10:55:47.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.512+0000 7fb8a9b7d740 10 client.15129 fetch_fsmap learned FSMap version 27 2024-03-27T10:55:47.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.512+0000 7fb8a9b7d740 1 -- 172.21.15.97:0/383563734 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d0aeb10c30 con 0x55d0aeb01c80 2024-03-27T10:55:47.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.513+0000 7fb88e000640 1 -- 172.21.15.97:0/383563734 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb890011b20 con 0x55d0aeb01c80 2024-03-27T10:55:47.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.513+0000 7fb88e000640 1 -- 172.21.15.97:0/383563734 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fb898034440 con 0x55d0aeb01c80 2024-03-27T10:55:47.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.513+0000 7fb8a9b7d740 10 client.15129 fetch_fsmap finished waiting for FSMap version 27 2024-03-27T10:55:47.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.513+0000 7fb8a9b7d740 10 client.15129 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-27T10:55:47.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.513+0000 7fb8a9b7d740 20 client.15129 populate_metadata read hostname 'smithi097' 2024-03-27T10:55:47.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.513+0000 7fb8a9b7d740 1 --2- 172.21.15.97:0/383563734 >> [v2:172.21.15.114:6836/616775588,v1:172.21.15.114:6837/616775588] conn(0x55d0aeb13360 0x55d0aeb147d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:47.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.513+0000 7fb8a9b7d740 4 client.15129 mds_command: new command op to 34478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T10:55:47.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.513+0000 7fb8a9b7d740 1 -- 172.21.15.97:0/383563734 --> [v2:172.21.15.114:6836/616775588,v1:172.21.15.114:6837/616775588] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d0aea6e4d0 con 0x55d0aeb13360 2024-03-27T10:55:47.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.514+0000 7fb89fe00640 1 --2- 172.21.15.97:0/383563734 >> [v2:172.21.15.114:6836/616775588,v1:172.21.15.114:6837/616775588] conn(0x55d0aeb13360 0x55d0aeb147d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:47.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.515+0000 7fb89fe00640 1 --2- 172.21.15.97:0/383563734 >> [v2:172.21.15.114:6836/616775588,v1:172.21.15.114:6837/616775588] conn(0x55d0aeb13360 0x55d0aeb147d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a2fedc9b0891ca30 server_cookie=432048076c31b279 in_seq=0 out_seq=0 2024-03-27T10:55:47.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.515+0000 7fb88e000640 10 client.15129 ms_handle_connect on v2:172.21.15.114:6836/616775588 2024-03-27T10:55:47.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.516+0000 7fb88e000640 1 -- 172.21.15.97:0/383563734 <== mds.1 v2:172.21.15.114:6836/616775588 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55d0aea6e4d0 con 0x55d0aeb13360 2024-03-27T10:55:47.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.516+0000 7fb88e000640 10 client.15129 handle_command_reply: tid=0 2024-03-27T10:55:47.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.516+0000 7fb8a9b7d740 1 client.15129 shutdown 2024-03-27T10:55:47.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.517+0000 7fb8a9b7d740 1 -- 172.21.15.97:0/383563734 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0aeb01c80 msgr2=0x55d0aeb0d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:47.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.517+0000 7fb8a9b7d740 1 --2- 172.21.15.97:0/383563734 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0aeb01c80 0x55d0aeb0d8e0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fb898035000 tx=0x7fb89802bc30 comp rx=0 tx=0).stop 2024-03-27T10:55:47.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.517+0000 7fb8a9b7d740 1 -- 172.21.15.97:0/383563734 shutdown_connections 2024-03-27T10:55:47.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.517+0000 7fb8a9b7d740 1 --2- 172.21.15.97:0/383563734 >> [v2:172.21.15.114:6836/616775588,v1:172.21.15.114:6837/616775588] conn(0x55d0aeb13360 0x55d0aeb147d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.517+0000 7fb8a9b7d740 1 --2- 172.21.15.97:0/383563734 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0aeb01c80 0x55d0aeb0d8e0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.517+0000 7fb8a9b7d740 1 --2- 172.21.15.97:0/383563734 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d0aeafcb10 0x55d0aeb068a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.517+0000 7fb8a9b7d740 1 --2- 172.21.15.97:0/383563734 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d0aeafc120 0x55d0aeb06360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.517+0000 7fb8a9b7d740 1 -- 172.21.15.97:0/383563734 >> 172.21.15.97:0/383563734 conn(0x55d0aeaf7690 msgr2=0x55d0aeaf6890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:55:47.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.517+0000 7fb88e000640 0 client.15129 ms_handle_reset on v2:172.21.15.114:6836/616775588 2024-03-27T10:55:47.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.517+0000 7fb8a9b7d740 1 -- 172.21.15.97:0/383563734 shutdown_connections 2024-03-27T10:55:47.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.517+0000 7fb8a9b7d740 1 -- 172.21.15.97:0/383563734 wait complete. 2024-03-27T10:55:47.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.517+0000 7fb8a9b7d740 20 client.15129 trim_cache size 0 max 16384 2024-03-27T10:55:47.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.537+0000 7fb8a9b7d740 1 Processor -- start 2024-03-27T10:55:47.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.537+0000 7fb8a9b7d740 1 -- start start 2024-03-27T10:55:47.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.537+0000 7fb8a9b7d740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0aeafc120 0x55d0aeb06360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:47.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.537+0000 7fb8a9b7d740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d0aeafcb10 0x55d0aeb068a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:47.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.537+0000 7fb8a9b7d740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d0aeb01c80 0x55d0aeb0d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:47.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.537+0000 7fb8a9b7d740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55d0aeb12b10 con 0x55d0aeafc120 2024-03-27T10:55:47.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.537+0000 7fb8a9b7d740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55d0aeb131e0 con 0x55d0aeafcb10 2024-03-27T10:55:47.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.537+0000 7fb8a9b7d740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55d0aeb138b0 con 0x55d0aeb01c80 2024-03-27T10:55:47.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.537+0000 7fb89fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0aeafc120 0x55d0aeb06360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:47.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.537+0000 7fb89fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0aeafc120 0x55d0aeb06360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40994/0 (socket says 172.21.15.97:40994) 2024-03-27T10:55:47.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.537+0000 7fb89fe00640 1 -- 172.21.15.97:0/2219152502 learned_addr learned my addr 172.21.15.97:0/2219152502 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:55:47.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.538+0000 7fb897400640 1 --2- 172.21.15.97:0/2219152502 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d0aeafcb10 0x55d0aeb068a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:47.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.538+0000 7fb8a4e00640 1 --2- 172.21.15.97:0/2219152502 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d0aeb01c80 0x55d0aeb0d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:47.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.538+0000 7fb89fe00640 1 -- 172.21.15.97:0/2219152502 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d0aeb01c80 msgr2=0x55d0aeb0d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:55:47.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.538+0000 7fb89fe00640 1 --2- 172.21.15.97:0/2219152502 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d0aeb01c80 0x55d0aeb0d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.538+0000 7fb89fe00640 1 -- 172.21.15.97:0/2219152502 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d0aeafcb10 msgr2=0x55d0aeb068a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:55:47.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.538+0000 7fb89fe00640 1 --2- 172.21.15.97:0/2219152502 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d0aeafcb10 0x55d0aeb068a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.538+0000 7fb89fe00640 1 -- 172.21.15.97:0/2219152502 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0aeb14b00 con 0x55d0aeafc120 2024-03-27T10:55:47.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.538+0000 7fb89fe00640 1 --2- 172.21.15.97:0/2219152502 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0aeafc120 0x55d0aeb06360 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fb890000f80 tx=0x7fb890009740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12663d05a2c7c89d server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:55:47.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.538+0000 7fb894c00640 1 -- 172.21.15.97:0/2219152502 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb89002d020 con 0x55d0aeafc120 2024-03-27T10:55:47.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.538+0000 7fb894c00640 1 -- 172.21.15.97:0/2219152502 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb890005bd0 con 0x55d0aeafc120 2024-03-27T10:55:47.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.538+0000 7fb894c00640 1 -- 172.21.15.97:0/2219152502 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb890016040 con 0x55d0aeafc120 2024-03-27T10:55:47.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.539+0000 7fb8a9b7d740 1 -- 172.21.15.97:0/2219152502 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0aeafc120 msgr2=0x55d0aeb06360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:47.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.539+0000 7fb8a9b7d740 1 --2- 172.21.15.97:0/2219152502 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0aeafc120 0x55d0aeb06360 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fb890000f80 tx=0x7fb890009740 comp rx=0 tx=0).stop 2024-03-27T10:55:47.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.539+0000 7fb8a9b7d740 1 -- 172.21.15.97:0/2219152502 shutdown_connections 2024-03-27T10:55:47.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.539+0000 7fb8a9b7d740 1 --2- 172.21.15.97:0/2219152502 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d0aeb01c80 0x55d0aeb0d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.539+0000 7fb8a9b7d740 1 --2- 172.21.15.97:0/2219152502 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d0aeafcb10 0x55d0aeb068a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.539+0000 7fb8a9b7d740 1 --2- 172.21.15.97:0/2219152502 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0aeafc120 0x55d0aeb06360 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.539+0000 7fb8a9b7d740 1 -- 172.21.15.97:0/2219152502 >> 172.21.15.97:0/2219152502 conn(0x55d0aeaf7690 msgr2=0x55d0aeb04ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:55:47.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.539+0000 7fb8a9b7d740 1 -- 172.21.15.97:0/2219152502 shutdown_connections 2024-03-27T10:55:47.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.539+0000 7fb8a9b7d740 1 -- 172.21.15.97:0/2219152502 wait complete. 2024-03-27T10:55:47.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.540+0000 7fb8a9b7d740 1 -- start start 2024-03-27T10:55:47.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.540+0000 7fb8a9b7d740 1 Processor -- start 2024-03-27T10:55:47.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.540+0000 7fb8a9b7d740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0aeafc120 0x55d0aeb093d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:47.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.540+0000 7fb8a9b7d740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d0aeafcb10 0x55d0aeb09910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:47.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.540+0000 7fb8a9b7d740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d0aeb01c80 0x55d0aeb0c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:47.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.540+0000 7fb8a9b7d740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55d0aeb14650 con 0x55d0aeafc120 2024-03-27T10:55:47.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.540+0000 7fb8a9b7d740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55d0aeb13f80 con 0x55d0aeafcb10 2024-03-27T10:55:47.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.540+0000 7fb8a9b7d740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55d0aeb14d20 con 0x55d0aeb01c80 2024-03-27T10:55:47.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.540+0000 7fb8a9b7d740 10 client.0 _ll_register_callbacks cb 0x7fb898009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:55:47.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.540+0000 7fb89fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0aeafc120 0x55d0aeb093d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:47.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.540+0000 7fb89fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0aeafc120 0x55d0aeb093d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41004/0 (socket says 172.21.15.97:41004) 2024-03-27T10:55:47.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.540+0000 7fb89fe00640 1 -- 172.21.15.97:0/14620400 learned_addr learned my addr 172.21.15.97:0/14620400 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:55:47.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.541+0000 7fb897400640 1 --2- 172.21.15.97:0/14620400 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d0aeafcb10 0x55d0aeb09910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:47.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.541+0000 7fb89fe00640 1 -- 172.21.15.97:0/14620400 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d0aeb01c80 msgr2=0x55d0aeb0c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:55:47.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.541+0000 7fb89fe00640 1 --2- 172.21.15.97:0/14620400 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d0aeb01c80 0x55d0aeb0c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.541+0000 7fb89fe00640 1 -- 172.21.15.97:0/14620400 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d0aeafcb10 msgr2=0x55d0aeb09910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:55:47.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.541+0000 7fb89fe00640 1 --2- 172.21.15.97:0/14620400 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d0aeafcb10 0x55d0aeb09910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.541+0000 7fb89fe00640 1 -- 172.21.15.97:0/14620400 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0aeb0cf20 con 0x55d0aeafc120 2024-03-27T10:55:47.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.541+0000 7fb897400640 1 --2- 172.21.15.97:0/14620400 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d0aeafcb10 0x55d0aeb09910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:55:47.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.541+0000 7fb89fe00640 1 --2- 172.21.15.97:0/14620400 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0aeafc120 0x55d0aeb093d0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fb890009e70 tx=0x7fb890004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40715f4aa489f3bc server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:55:47.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.541+0000 7fb88e000640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T10:55:47.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.541+0000 7fb88e000640 1 -- 172.21.15.97:0/14620400 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb89002d050 con 0x55d0aeafc120 2024-03-27T10:55:47.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.541+0000 7fb88e000640 1 -- 172.21.15.97:0/14620400 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fb890029630 con 0x55d0aeafc120 2024-03-27T10:55:47.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.541+0000 7fb8a9b7d740 1 -- 172.21.15.97:0/14620400 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb890005bd0 con 0x55d0aeafc120 2024-03-27T10:55:47.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.541+0000 7fb88e000640 1 -- 172.21.15.97:0/14620400 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb890041050 con 0x55d0aeafc120 2024-03-27T10:55:47.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.541+0000 7fb8a9b7d740 10 client.15135 fetch_fsmap learned FSMap version 27 2024-03-27T10:55:47.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.542+0000 7fb8a9b7d740 1 -- 172.21.15.97:0/14620400 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d0aeb79420 con 0x55d0aeafc120 2024-03-27T10:55:47.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.542+0000 7fb88e000640 1 -- 172.21.15.97:0/14620400 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb890005bd0 con 0x55d0aeafc120 2024-03-27T10:55:47.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.542+0000 7fb88e000640 1 -- 172.21.15.97:0/14620400 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fb89000f040 con 0x55d0aeafc120 2024-03-27T10:55:47.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.542+0000 7fb8a9b7d740 10 client.15135 fetch_fsmap finished waiting for FSMap version 27 2024-03-27T10:55:47.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.542+0000 7fb8a9b7d740 10 client.15135 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-27T10:55:47.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.542+0000 7fb8a9b7d740 20 client.15135 populate_metadata read hostname 'smithi097' 2024-03-27T10:55:47.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.542+0000 7fb8a9b7d740 1 --2- 172.21.15.97:0/14620400 >> [v2:172.21.15.114:6836/616775588,v1:172.21.15.114:6837/616775588] conn(0x55d0aeb7da60 0x55d0aeb7c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:47.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.542+0000 7fb8a9b7d740 4 client.15135 mds_command: new command op to 34478 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-27T10:55:47.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.542+0000 7fb8a9b7d740 1 -- 172.21.15.97:0/14620400 --> [v2:172.21.15.114:6836/616775588,v1:172.21.15.114:6837/616775588] -- command(tid 0: {"prefix": "status"}) -- 0x55d0aea68820 con 0x55d0aeb7da60 2024-03-27T10:55:47.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.543+0000 7fb897400640 1 --2- 172.21.15.97:0/14620400 >> [v2:172.21.15.114:6836/616775588,v1:172.21.15.114:6837/616775588] conn(0x55d0aeb7da60 0x55d0aeb7c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:47.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.544+0000 7fb897400640 1 --2- 172.21.15.97:0/14620400 >> [v2:172.21.15.114:6836/616775588,v1:172.21.15.114:6837/616775588] conn(0x55d0aeb7da60 0x55d0aeb7c770 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7c71b512a888d863 server_cookie=67eaa4711bfb0e80 in_seq=0 out_seq=0 2024-03-27T10:55:47.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.544+0000 7fb88e000640 10 client.15135 ms_handle_connect on v2:172.21.15.114:6836/616775588 2024-03-27T10:55:47.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.544+0000 7fb88e000640 1 -- 172.21.15.97:0/14620400 <== mds.1 v2:172.21.15.114:6836/616775588 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55d0aea68820 con 0x55d0aeb7da60 2024-03-27T10:55:47.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.544+0000 7fb88e000640 10 client.15135 handle_command_reply: tid=0 2024-03-27T10:55:47.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.544+0000 7fb8a9b7d740 1 client.15135 shutdown 2024-03-27T10:55:47.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.544+0000 7fb8a9b7d740 1 -- 172.21.15.97:0/14620400 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0aeafc120 msgr2=0x55d0aeb093d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:47.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.544+0000 7fb8a9b7d740 1 --2- 172.21.15.97:0/14620400 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0aeafc120 0x55d0aeb093d0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fb890009e70 tx=0x7fb890004ad0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.544+0000 7fb8a9b7d740 1 -- 172.21.15.97:0/14620400 shutdown_connections 2024-03-27T10:55:47.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.544+0000 7fb8a9b7d740 1 --2- 172.21.15.97:0/14620400 >> [v2:172.21.15.114:6836/616775588,v1:172.21.15.114:6837/616775588] conn(0x55d0aeb7da60 0x55d0aeb7c770 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.544+0000 7fb8a9b7d740 1 --2- 172.21.15.97:0/14620400 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d0aeb01c80 0x55d0aeb0c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.544+0000 7fb8a9b7d740 1 --2- 172.21.15.97:0/14620400 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d0aeafcb10 0x55d0aeb09910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.544+0000 7fb8a9b7d740 1 --2- 172.21.15.97:0/14620400 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0aeafc120 0x55d0aeb093d0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.544+0000 7fb8a9b7d740 1 -- 172.21.15.97:0/14620400 >> 172.21.15.97:0/14620400 conn(0x55d0aeaf7690 msgr2=0x55d0aeb06360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:55:47.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.544+0000 7fb88e000640 0 client.15135 ms_handle_reset on v2:172.21.15.114:6836/616775588 2024-03-27T10:55:47.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.544+0000 7fb8a9b7d740 1 -- 172.21.15.97:0/14620400 shutdown_connections 2024-03-27T10:55:47.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.545+0000 7fb8a9b7d740 1 -- 172.21.15.97:0/14620400 wait complete. 2024-03-27T10:55:47.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.545+0000 7fb8a9b7d740 20 client.15135 trim_cache size 0 max 16384 2024-03-27T10:55:47.520 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-03-27T10:55:47.521 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "adca88c4-ec26-11ee-95d0-87774f69a715", 2024-03-27T10:55:47.521 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 1, 2024-03-27T10:55:47.521 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 34478, 2024-03-27T10:55:47.521 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2024-03-27T10:55:47.521 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2024-03-27T10:55:47.521 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2024-03-27T10:55:47.521 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 98.954485114999997, 2024-03-27T10:55:47.521 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 27, 2024-03-27T10:55:47.521 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 92, 2024-03-27T10:55:47.521 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 90, 2024-03-27T10:55:47.521 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 123.331673724 2024-03-27T10:55:47.521 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-03-27T10:55:47.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.545+0000 7fb8a5800640 1 -- 172.21.15.97:0/1059456944 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fb8840a3610 msgr2=0x7fb8840a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:47.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.545+0000 7fb8a5800640 1 --2- 172.21.15.97:0/1059456944 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fb8840a3610 0x7fb8840a5ad0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb898005ab0 tx=0x7fb898005a20 comp rx=0 tx=0).stop 2024-03-27T10:55:47.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.545+0000 7fb8a5800640 1 -- 172.21.15.97:0/1059456944 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8a0104d70 msgr2=0x7fb8a019a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:47.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.545+0000 7fb8a5800640 1 --2- 172.21.15.97:0/1059456944 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8a0104d70 0x7fb8a019a290 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb88800d770 tx=0x7fb88800dc30 comp rx=0 tx=0).stop 2024-03-27T10:55:47.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.545+0000 7fb8a5800640 1 -- 172.21.15.97:0/1059456944 shutdown_connections 2024-03-27T10:55:47.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.545+0000 7fb8a5800640 1 --2- 172.21.15.97:0/1059456944 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fb8840a3610 0x7fb8840a5ad0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.545+0000 7fb8a5800640 1 --2- 172.21.15.97:0/1059456944 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb8a0105730 0x7fb8a01a1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.545+0000 7fb8a5800640 1 --2- 172.21.15.97:0/1059456944 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8a0104d70 0x7fb8a019a290 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.545+0000 7fb8a5800640 1 --2- 172.21.15.97:0/1059456944 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb8a0103b80 0x7fb8a0199d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.545+0000 7fb8a5800640 1 -- 172.21.15.97:0/1059456944 >> 172.21.15.97:0/1059456944 conn(0x7fb8a00ff0f0 msgr2=0x7fb8a0108070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:55:47.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.546+0000 7fb8a5800640 1 -- 172.21.15.97:0/1059456944 shutdown_connections 2024-03-27T10:55:47.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.546+0000 7fb8a5800640 1 -- 172.21.15.97:0/1059456944 wait complete. 2024-03-27T10:55:47.531 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:55:47 smithi097 ceph-mon[31859]: pgmap v415: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 10 op/s 2024-03-27T10:55:47.531 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:55:47 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/1144338998' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T10:55:47.534 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-27T10:55:47.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:55:47 smithi177 ceph-mon[35294]: pgmap v415: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 10 op/s 2024-03-27T10:55:47.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:55:47 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/1144338998' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T10:55:47.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.763+0000 7fddcdc00640 1 -- 172.21.15.97:0/2843325245 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fddc8104f80 msgr2=0x7fddc8105400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:47.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.763+0000 7fddcdc00640 1 --2- 172.21.15.97:0/2843325245 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fddc8104f80 0x7fddc8105400 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fddb0009900 tx=0x7fddb002bc30 comp rx=0 tx=0).stop 2024-03-27T10:55:47.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.763+0000 7fddcdc00640 1 -- 172.21.15.97:0/2843325245 shutdown_connections 2024-03-27T10:55:47.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.763+0000 7fddcdc00640 1 --2- 172.21.15.97:0/2843325245 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fddc8105940 0x7fddc810c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.763+0000 7fddcdc00640 1 --2- 172.21.15.97:0/2843325245 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fddc8104f80 0x7fddc8105400 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.763+0000 7fddcdc00640 1 --2- 172.21.15.97:0/2843325245 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fddc8103d90 0x7fddc81041b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.763+0000 7fddcdc00640 1 -- 172.21.15.97:0/2843325245 >> 172.21.15.97:0/2843325245 conn(0x7fddc80ff300 msgr2=0x7fddc8101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:55:47.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.763+0000 7fddcdc00640 1 -- 172.21.15.97:0/2843325245 shutdown_connections 2024-03-27T10:55:47.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.763+0000 7fddcdc00640 1 -- 172.21.15.97:0/2843325245 wait complete. 2024-03-27T10:55:47.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.764+0000 7fddcdc00640 1 Processor -- start 2024-03-27T10:55:47.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.764+0000 7fddcdc00640 1 -- start start 2024-03-27T10:55:47.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.764+0000 7fddcdc00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fddc8103d90 0x7fddc8199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:47.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.764+0000 7fddcdc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fddc8104f80 0x7fddc819a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:47.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.764+0000 7fddcdc00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fddc8105940 0x7fddc81a1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:47.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.764+0000 7fddcdc00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fddc810f910 con 0x7fddc8104f80 2024-03-27T10:55:47.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.764+0000 7fddcdc00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fddc810f790 con 0x7fddc8103d90 2024-03-27T10:55:47.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.764+0000 7fddcdc00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fddc810fa90 con 0x7fddc8105940 2024-03-27T10:55:47.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.765+0000 7fddc6000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fddc8104f80 0x7fddc819a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:47.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.765+0000 7fddc6000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fddc8104f80 0x7fddc819a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41008/0 (socket says 172.21.15.97:41008) 2024-03-27T10:55:47.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.765+0000 7fddc6000640 1 -- 172.21.15.97:0/1736571697 learned_addr learned my addr 172.21.15.97:0/1736571697 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:55:47.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.765+0000 7fddc6a00640 1 --2- 172.21.15.97:0/1736571697 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fddc8103d90 0x7fddc8199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:47.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.765+0000 7fddc7400640 1 --2- 172.21.15.97:0/1736571697 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fddc8105940 0x7fddc81a1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:47.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.765+0000 7fddc6000640 1 -- 172.21.15.97:0/1736571697 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fddc8105940 msgr2=0x7fddc81a1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:47.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.765+0000 7fddc6000640 1 --2- 172.21.15.97:0/1736571697 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fddc8105940 0x7fddc81a1560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.765+0000 7fddc6000640 1 -- 172.21.15.97:0/1736571697 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fddc8103d90 msgr2=0x7fddc8199fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:47.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.765+0000 7fddc6000640 1 --2- 172.21.15.97:0/1736571697 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fddc8103d90 0x7fddc8199fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.765+0000 7fddc6000640 1 -- 172.21.15.97:0/1736571697 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddc81a1bd0 con 0x7fddc8104f80 2024-03-27T10:55:47.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.765+0000 7fddc6a00640 1 --2- 172.21.15.97:0/1736571697 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fddc8103d90 0x7fddc8199fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:55:47.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.765+0000 7fddc6000640 1 --2- 172.21.15.97:0/1736571697 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fddc8104f80 0x7fddc819a4f0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fddb0000c00 tx=0x7fddb00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:55:47.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.765+0000 7fddbf400640 1 -- 172.21.15.97:0/1736571697 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb0043020 con 0x7fddc8104f80 2024-03-27T10:55:47.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.766+0000 7fddbf400640 1 -- 172.21.15.97:0/1736571697 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fddb00054c0 con 0x7fddc8104f80 2024-03-27T10:55:47.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.766+0000 7fddbf400640 1 -- 172.21.15.97:0/1736571697 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb0003d40 con 0x7fddc8104f80 2024-03-27T10:55:47.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.766+0000 7fddcdc00640 1 -- 172.21.15.97:0/1736571697 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddc81a1e60 con 0x7fddc8104f80 2024-03-27T10:55:47.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.766+0000 7fddcdc00640 1 -- 172.21.15.97:0/1736571697 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddc81a2320 con 0x7fddc8104f80 2024-03-27T10:55:47.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.769+0000 7fddbf400640 1 -- 172.21.15.97:0/1736571697 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7fddb0046020 con 0x7fddc8104f80 2024-03-27T10:55:47.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.769+0000 7fddd0861740 1 Processor -- start 2024-03-27T10:55:47.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.769+0000 7fddd0861740 1 -- start start 2024-03-27T10:55:47.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.770+0000 7fddd0861740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5585287b9f20 0x5585287ba340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:47.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.770+0000 7fddd0861740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5585287ba910 0x5585287bf410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:47.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.770+0000 7fddd0861740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5585287bfa80 0x5585287c1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:47.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.770+0000 7fddd0861740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55852877c920 con 0x5585287ba910 2024-03-27T10:55:47.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.770+0000 7fddd0861740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5585286ccfa0 con 0x5585287b9f20 2024-03-27T10:55:47.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.770+0000 7fddd0861740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55852871ef90 con 0x5585287bfa80 2024-03-27T10:55:47.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.770+0000 7fddc6a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5585287ba910 0x5585287bf410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:47.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.770+0000 7fddc6a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5585287ba910 0x5585287bf410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41020/0 (socket says 172.21.15.97:41020) 2024-03-27T10:55:47.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.770+0000 7fddc6a00640 1 -- 172.21.15.97:0/2071846237 learned_addr learned my addr 172.21.15.97:0/2071846237 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:55:47.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.770+0000 7fddc7400640 1 --2- 172.21.15.97:0/2071846237 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5585287b9f20 0x5585287ba340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:47.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.770+0000 7fddc6000640 1 --2- 172.21.15.97:0/2071846237 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5585287bfa80 0x5585287c1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:47.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.770+0000 7fddc6a00640 1 -- 172.21.15.97:0/2071846237 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5585287bfa80 msgr2=0x5585287c1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:55:47.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.770+0000 7fddc6a00640 1 --2- 172.21.15.97:0/2071846237 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5585287bfa80 0x5585287c1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.770+0000 7fddc6a00640 1 -- 172.21.15.97:0/2071846237 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5585287b9f20 msgr2=0x5585287ba340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:55:47.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.770+0000 7fddc6a00640 1 --2- 172.21.15.97:0/2071846237 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5585287b9f20 0x5585287ba340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.770+0000 7fddc6a00640 1 -- 172.21.15.97:0/2071846237 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585284332d0 con 0x5585287ba910 2024-03-27T10:55:47.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.770+0000 7fddc7400640 1 --2- 172.21.15.97:0/2071846237 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5585287b9f20 0x5585287ba340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T10:55:47.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.771+0000 7fddc6a00640 1 --2- 172.21.15.97:0/2071846237 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5585287ba910 0x5585287bf410 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fddb800e320 tx=0x7fddb800e7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfce997b6a460642 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:55:47.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.771+0000 7fddbf400640 1 --2- 172.21.15.97:0/1736571697 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fddac0a3610 0x7fddac0a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:47.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.771+0000 7fddb7e00640 1 -- 172.21.15.97:0/2071846237 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb8016070 con 0x5585287ba910 2024-03-27T10:55:47.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.771+0000 7fddb7e00640 1 -- 172.21.15.97:0/2071846237 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fddb8010b40 con 0x5585287ba910 2024-03-27T10:55:47.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.771+0000 7fddb7e00640 1 -- 172.21.15.97:0/2071846237 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb8011bf0 con 0x5585287ba910 2024-03-27T10:55:47.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.771+0000 7fddbf400640 1 -- 172.21.15.97:0/1736571697 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fddb00f21e0 con 0x7fddc8104f80 2024-03-27T10:55:47.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.771+0000 7fddc7400640 1 --2- 172.21.15.97:0/1736571697 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fddac0a3610 0x7fddac0a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:47.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.772+0000 7fddd0861740 1 -- 172.21.15.97:0/2071846237 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5585287ba910 msgr2=0x5585287bf410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:47.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.772+0000 7fddd0861740 1 --2- 172.21.15.97:0/2071846237 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5585287ba910 0x5585287bf410 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fddb800e320 tx=0x7fddb800e7e0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.772+0000 7fddc7400640 1 --2- 172.21.15.97:0/1736571697 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fddac0a3610 0x7fddac0a5ad0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x5585287bb170 tx=0x7fddc0005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:55:47.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.772+0000 7fddd0861740 1 -- 172.21.15.97:0/2071846237 shutdown_connections 2024-03-27T10:55:47.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.772+0000 7fddd0861740 1 --2- 172.21.15.97:0/2071846237 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5585287bfa80 0x5585287c1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.772+0000 7fddd0861740 1 --2- 172.21.15.97:0/2071846237 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5585287ba910 0x5585287bf410 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.772+0000 7fddd0861740 1 --2- 172.21.15.97:0/2071846237 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5585287b9f20 0x5585287ba340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.772+0000 7fddd0861740 1 -- 172.21.15.97:0/2071846237 >> 172.21.15.97:0/2071846237 conn(0x5585287b5490 msgr2=0x5585287b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:55:47.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.772+0000 7fddd0861740 1 -- 172.21.15.97:0/2071846237 shutdown_connections 2024-03-27T10:55:47.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.772+0000 7fddd0861740 1 -- 172.21.15.97:0/2071846237 wait complete. 2024-03-27T10:55:47.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.773+0000 7fddd0861740 1 -- start start 2024-03-27T10:55:47.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.773+0000 7fddd0861740 1 Processor -- start 2024-03-27T10:55:47.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.773+0000 7fddd0861740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5585287b9f20 0x5585287c4110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:47.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.773+0000 7fddd0861740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5585287ba910 0x5585287c4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:47.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.773+0000 7fddd0861740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5585287bfa80 0x5585287cb690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:47.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.773+0000 7fddd0861740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55852871af40 con 0x5585287b9f20 2024-03-27T10:55:47.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.773+0000 7fddd0861740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5585286c7650 con 0x5585287bfa80 2024-03-27T10:55:47.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.773+0000 7fddd0861740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5585286daa10 con 0x5585287ba910 2024-03-27T10:55:47.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.773+0000 7fddd0861740 10 client.0 _ll_register_callbacks cb 0x558528719ff0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:55:47.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.774+0000 7fddc6a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5585287b9f20 0x5585287c4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:47.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.774+0000 7fddc6a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5585287b9f20 0x5585287c4110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41028/0 (socket says 172.21.15.97:41028) 2024-03-27T10:55:47.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.774+0000 7fddc6a00640 1 -- 172.21.15.97:0/290540859 learned_addr learned my addr 172.21.15.97:0/290540859 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:55:47.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.774+0000 7fddc6000640 1 --2- 172.21.15.97:0/290540859 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5585287ba910 0x5585287c4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:47.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.774+0000 7fddc7400640 1 --2- 172.21.15.97:0/290540859 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5585287bfa80 0x5585287cb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:47.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.774+0000 7fddc6a00640 1 -- 172.21.15.97:0/290540859 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5585287ba910 msgr2=0x5585287c4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:55:47.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.774+0000 7fddc6a00640 1 --2- 172.21.15.97:0/290540859 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5585287ba910 0x5585287c4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.774+0000 7fddc6a00640 1 -- 172.21.15.97:0/290540859 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5585287bfa80 msgr2=0x5585287cb690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:55:47.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.774+0000 7fddc6a00640 1 --2- 172.21.15.97:0/290540859 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5585287bfa80 0x5585287cb690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.774+0000 7fddc6a00640 1 -- 172.21.15.97:0/290540859 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585284fd6b0 con 0x5585287b9f20 2024-03-27T10:55:47.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.774+0000 7fddc7400640 1 --2- 172.21.15.97:0/290540859 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5585287bfa80 0x5585287cb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:55:47.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.774+0000 7fddc6000640 1 --2- 172.21.15.97:0/290540859 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5585287ba910 0x5585287c4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:55:47.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.774+0000 7fddc6a00640 1 --2- 172.21.15.97:0/290540859 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5585287b9f20 0x5585287c4110 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fddb8000f80 tx=0x7fddb8011690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb10d66c909b441e server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:55:47.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.774+0000 7fddb4c00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T10:55:47.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.775+0000 7fddb4c00640 1 -- 172.21.15.97:0/290540859 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb8016050 con 0x5585287b9f20 2024-03-27T10:55:47.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.775+0000 7fddb4c00640 1 -- 172.21.15.97:0/290540859 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fddb8011bd0 con 0x5585287b9f20 2024-03-27T10:55:47.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.775+0000 7fddd0861740 1 -- 172.21.15.97:0/290540859 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fddb8010b40 con 0x5585287b9f20 2024-03-27T10:55:47.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.775+0000 7fddb4c00640 1 -- 172.21.15.97:0/290540859 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fddb8004e10 con 0x5585287b9f20 2024-03-27T10:55:47.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.775+0000 7fddd0861740 10 client.15147 fetch_fsmap learned FSMap version 27 2024-03-27T10:55:47.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.775+0000 7fddd0861740 1 -- 172.21.15.97:0/290540859 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5585287cea00 con 0x5585287b9f20 2024-03-27T10:55:47.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.775+0000 7fddb4c00640 1 -- 172.21.15.97:0/290540859 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fddb8010b40 con 0x5585287b9f20 2024-03-27T10:55:47.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.775+0000 7fddb4c00640 1 -- 172.21.15.97:0/290540859 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fddb802b030 con 0x5585287b9f20 2024-03-27T10:55:47.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.775+0000 7fddd0861740 10 client.15147 fetch_fsmap finished waiting for FSMap version 27 2024-03-27T10:55:47.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.776+0000 7fddd0861740 10 client.15147 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-27T10:55:47.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.776+0000 7fddd0861740 20 client.15147 populate_metadata read hostname 'smithi097' 2024-03-27T10:55:47.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.776+0000 7fddd0861740 1 --2- 172.21.15.97:0/290540859 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5585287d0d50 0x5585287d21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:47.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.776+0000 7fddd0861740 4 client.15147 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T10:55:47.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.776+0000 7fddd0861740 1 -- 172.21.15.97:0/290540859 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55852872c4d0 con 0x5585287d0d50 2024-03-27T10:55:47.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.776+0000 7fddc6000640 1 --2- 172.21.15.97:0/290540859 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5585287d0d50 0x5585287d21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:47.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.777+0000 7fddc6000640 1 --2- 172.21.15.97:0/290540859 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5585287d0d50 0x5585287d21c0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7b78b6d476ac3ea server_cookie=a177e4520680a885 in_seq=0 out_seq=0 2024-03-27T10:55:47.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.777+0000 7fddb4c00640 10 client.15147 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T10:55:47.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.778+0000 7fddb4c00640 1 -- 172.21.15.97:0/290540859 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55852872c4d0 con 0x5585287d0d50 2024-03-27T10:55:47.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.778+0000 7fddb4c00640 10 client.15147 handle_command_reply: tid=0 2024-03-27T10:55:47.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.778+0000 7fddd0861740 1 client.15147 shutdown 2024-03-27T10:55:47.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.779+0000 7fddd0861740 1 -- 172.21.15.97:0/290540859 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5585287b9f20 msgr2=0x5585287c4110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:47.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.779+0000 7fddd0861740 1 --2- 172.21.15.97:0/290540859 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5585287b9f20 0x5585287c4110 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fddb8000f80 tx=0x7fddb8011690 comp rx=0 tx=0).stop 2024-03-27T10:55:47.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.779+0000 7fddd0861740 1 -- 172.21.15.97:0/290540859 shutdown_connections 2024-03-27T10:55:47.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.779+0000 7fddd0861740 1 --2- 172.21.15.97:0/290540859 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5585287d0d50 0x5585287d21c0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.779+0000 7fddd0861740 1 --2- 172.21.15.97:0/290540859 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5585287bfa80 0x5585287cb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.779+0000 7fddd0861740 1 --2- 172.21.15.97:0/290540859 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5585287ba910 0x5585287c4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.779+0000 7fddd0861740 1 --2- 172.21.15.97:0/290540859 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5585287b9f20 0x5585287c4110 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.779+0000 7fddd0861740 1 -- 172.21.15.97:0/290540859 >> 172.21.15.97:0/290540859 conn(0x5585287b5490 msgr2=0x5585287b4690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:55:47.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.779+0000 7fddb4c00640 0 client.15147 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T10:55:47.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.779+0000 7fddd0861740 1 -- 172.21.15.97:0/290540859 shutdown_connections 2024-03-27T10:55:47.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.779+0000 7fddd0861740 1 -- 172.21.15.97:0/290540859 wait complete. 2024-03-27T10:55:47.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.779+0000 7fddd0861740 20 client.15147 trim_cache size 0 max 16384 2024-03-27T10:55:47.766 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:55:47 smithi114 ceph-mon[35338]: pgmap v415: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 10 op/s 2024-03-27T10:55:47.766 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:55:47 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/1144338998' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T10:55:47.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.798+0000 7fddd0861740 1 Processor -- start 2024-03-27T10:55:47.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.799+0000 7fddd0861740 1 -- start start 2024-03-27T10:55:47.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.799+0000 7fddd0861740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5585287b9f20 0x5585287d0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:47.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.799+0000 7fddd0861740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5585287ba910 0x5585287cb650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:47.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.799+0000 7fddd0861740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5585287bfa80 0x5585287b0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:47.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.799+0000 7fddd0861740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5585287d0500 con 0x5585287b9f20 2024-03-27T10:55:47.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.799+0000 7fddd0861740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5585287d0bd0 con 0x5585287bfa80 2024-03-27T10:55:47.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.799+0000 7fddd0861740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5585287d12a0 con 0x5585287ba910 2024-03-27T10:55:47.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.799+0000 7fddc6a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5585287b9f20 0x5585287d0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:47.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.799+0000 7fddc6a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5585287b9f20 0x5585287d0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41036/0 (socket says 172.21.15.97:41036) 2024-03-27T10:55:47.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.799+0000 7fddc6a00640 1 -- 172.21.15.97:0/1807933813 learned_addr learned my addr 172.21.15.97:0/1807933813 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:55:47.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.799+0000 7fddc7400640 1 --2- 172.21.15.97:0/1807933813 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5585287bfa80 0x5585287b0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:47.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.799+0000 7fddc6a00640 1 -- 172.21.15.97:0/1807933813 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5585287ba910 msgr2=0x5585287cb650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:55:47.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.799+0000 7fddc6a00640 1 --2- 172.21.15.97:0/1807933813 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5585287ba910 0x5585287cb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.799+0000 7fddc6a00640 1 -- 172.21.15.97:0/1807933813 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5585287bfa80 msgr2=0x5585287b0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:55:47.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.799+0000 7fddc6a00640 1 --2- 172.21.15.97:0/1807933813 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5585287bfa80 0x5585287b0210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.799+0000 7fddc6a00640 1 -- 172.21.15.97:0/1807933813 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585287bacf0 con 0x5585287b9f20 2024-03-27T10:55:47.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.800+0000 7fddc6a00640 1 --2- 172.21.15.97:0/1807933813 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5585287b9f20 0x5585287d0680 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fddb800e820 tx=0x7fddb800efc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51ccb5a8662316b6 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:55:47.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.800+0000 7fddb7400640 1 -- 172.21.15.97:0/1807933813 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb803b070 con 0x5585287b9f20 2024-03-27T10:55:47.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.800+0000 7fddb7400640 1 -- 172.21.15.97:0/1807933813 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fddb8011bb0 con 0x5585287b9f20 2024-03-27T10:55:47.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.800+0000 7fddb7400640 1 -- 172.21.15.97:0/1807933813 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb8016040 con 0x5585287b9f20 2024-03-27T10:55:47.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.801+0000 7fddd0861740 1 -- 172.21.15.97:0/1807933813 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5585287b9f20 msgr2=0x5585287d0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:47.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.801+0000 7fddd0861740 1 --2- 172.21.15.97:0/1807933813 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5585287b9f20 0x5585287d0680 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fddb800e820 tx=0x7fddb800efc0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.801+0000 7fddd0861740 1 -- 172.21.15.97:0/1807933813 shutdown_connections 2024-03-27T10:55:47.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.801+0000 7fddd0861740 1 --2- 172.21.15.97:0/1807933813 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5585287bfa80 0x5585287b0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.801+0000 7fddd0861740 1 --2- 172.21.15.97:0/1807933813 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5585287ba910 0x5585287cb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.801+0000 7fddd0861740 1 --2- 172.21.15.97:0/1807933813 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5585287b9f20 0x5585287d0680 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.801+0000 7fddd0861740 1 -- 172.21.15.97:0/1807933813 >> 172.21.15.97:0/1807933813 conn(0x5585287b5490 msgr2=0x5585287b5870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:55:47.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.801+0000 7fddd0861740 1 -- 172.21.15.97:0/1807933813 shutdown_connections 2024-03-27T10:55:47.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.801+0000 7fddd0861740 1 -- 172.21.15.97:0/1807933813 wait complete. 2024-03-27T10:55:47.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.801+0000 7fddd0861740 1 -- start start 2024-03-27T10:55:47.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.801+0000 7fddd0861740 1 Processor -- start 2024-03-27T10:55:47.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.802+0000 7fddd0861740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5585287b9f20 0x5585287c7490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:47.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.802+0000 7fddd0861740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5585287ba910 0x5585287c79d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:47.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.802+0000 7fddd0861740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5585287bfa80 0x5585287ca9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:47.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.802+0000 7fddd0861740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5585287d2040 con 0x5585287b9f20 2024-03-27T10:55:47.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.802+0000 7fddd0861740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5585287d1970 con 0x5585287bfa80 2024-03-27T10:55:47.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.802+0000 7fddd0861740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5585287d2710 con 0x5585287ba910 2024-03-27T10:55:47.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.802+0000 7fddd0861740 10 client.0 _ll_register_callbacks cb 0x7fddb8009e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:55:47.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.802+0000 7fddc6a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5585287b9f20 0x5585287c7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:47.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.802+0000 7fddc6a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5585287b9f20 0x5585287c7490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41042/0 (socket says 172.21.15.97:41042) 2024-03-27T10:55:47.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.802+0000 7fddc6a00640 1 -- 172.21.15.97:0/1649158583 learned_addr learned my addr 172.21.15.97:0/1649158583 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:55:47.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.802+0000 7fddc7400640 1 --2- 172.21.15.97:0/1649158583 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5585287bfa80 0x5585287ca9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:47.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.802+0000 7fddc6a00640 1 -- 172.21.15.97:0/1649158583 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5585287ba910 msgr2=0x5585287c79d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:55:47.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.802+0000 7fddc6a00640 1 --2- 172.21.15.97:0/1649158583 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5585287ba910 0x5585287c79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.802+0000 7fddc6a00640 1 -- 172.21.15.97:0/1649158583 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5585287bfa80 msgr2=0x5585287ca9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:55:47.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.802+0000 7fddc6a00640 1 --2- 172.21.15.97:0/1649158583 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5585287bfa80 0x5585287ca9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.802+0000 7fddc6a00640 1 -- 172.21.15.97:0/1649158583 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585287cafe0 con 0x5585287b9f20 2024-03-27T10:55:47.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.803+0000 7fddc6a00640 1 --2- 172.21.15.97:0/1649158583 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5585287b9f20 0x5585287c7490 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fddb8039040 tx=0x7fddb8011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4894754ef31e3f2 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:55:47.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.803+0000 7fddb4c00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T10:55:47.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.803+0000 7fddb4c00640 1 -- 172.21.15.97:0/1649158583 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb8043060 con 0x5585287b9f20 2024-03-27T10:55:47.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.803+0000 7fddb4c00640 1 -- 172.21.15.97:0/1649158583 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fddb8011880 con 0x5585287b9f20 2024-03-27T10:55:47.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.803+0000 7fddd0861740 1 -- 172.21.15.97:0/1649158583 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fddb8011bb0 con 0x5585287b9f20 2024-03-27T10:55:47.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.803+0000 7fddb4c00640 1 -- 172.21.15.97:0/1649158583 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fddb803b070 con 0x5585287b9f20 2024-03-27T10:55:47.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.803+0000 7fddd0861740 10 client.15153 fetch_fsmap learned FSMap version 27 2024-03-27T10:55:47.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.803+0000 7fddd0861740 1 -- 172.21.15.97:0/1649158583 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55852883f9d0 con 0x5585287b9f20 2024-03-27T10:55:47.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.803+0000 7fddb4c00640 1 -- 172.21.15.97:0/1649158583 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fddb8011bb0 con 0x5585287b9f20 2024-03-27T10:55:47.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.804+0000 7fddb4c00640 1 -- 172.21.15.97:0/1649158583 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fddb802b070 con 0x5585287b9f20 2024-03-27T10:55:47.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.804+0000 7fddd0861740 10 client.15153 fetch_fsmap finished waiting for FSMap version 27 2024-03-27T10:55:47.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.804+0000 7fddd0861740 10 client.15153 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-27T10:55:47.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.804+0000 7fddd0861740 20 client.15153 populate_metadata read hostname 'smithi097' 2024-03-27T10:55:47.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.804+0000 7fddd0861740 1 --2- 172.21.15.97:0/1649158583 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5585288441a0 0x5585288405e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:47.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.804+0000 7fddd0861740 4 client.15153 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-27T10:55:47.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.804+0000 7fddd0861740 1 -- 172.21.15.97:0/1649158583 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "status"}) -- 0x558528726820 con 0x5585288441a0 2024-03-27T10:55:47.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.804+0000 7fddc6000640 1 --2- 172.21.15.97:0/1649158583 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5585288441a0 0x5585288405e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:47.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.805+0000 7fddc6000640 1 --2- 172.21.15.97:0/1649158583 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5585288441a0 0x5585288405e0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cdbb753ae5307793 server_cookie=1f85f399a8609a81 in_seq=0 out_seq=0 2024-03-27T10:55:47.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.805+0000 7fddb4c00640 10 client.15153 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T10:55:47.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.806+0000 7fddb4c00640 1 -- 172.21.15.97:0/1649158583 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x558528726820 con 0x5585288441a0 2024-03-27T10:55:47.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.806+0000 7fddb4c00640 10 client.15153 handle_command_reply: tid=0 2024-03-27T10:55:47.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.806+0000 7fddd0861740 1 client.15153 shutdown 2024-03-27T10:55:47.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.806+0000 7fddd0861740 1 -- 172.21.15.97:0/1649158583 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5585287b9f20 msgr2=0x5585287c7490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:47.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.806+0000 7fddd0861740 1 --2- 172.21.15.97:0/1649158583 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5585287b9f20 0x5585287c7490 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fddb8039040 tx=0x7fddb8011000 comp rx=0 tx=0).stop 2024-03-27T10:55:47.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.806+0000 7fddd0861740 1 -- 172.21.15.97:0/1649158583 shutdown_connections 2024-03-27T10:55:47.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.806+0000 7fddd0861740 1 --2- 172.21.15.97:0/1649158583 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5585288441a0 0x5585288405e0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.806+0000 7fddd0861740 1 --2- 172.21.15.97:0/1649158583 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5585287bfa80 0x5585287ca9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.806+0000 7fddb4c00640 0 client.15153 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T10:55:47.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.806+0000 7fddd0861740 1 --2- 172.21.15.97:0/1649158583 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5585287ba910 0x5585287c79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.806+0000 7fddd0861740 1 --2- 172.21.15.97:0/1649158583 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5585287b9f20 0x5585287c7490 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.806+0000 7fddd0861740 1 -- 172.21.15.97:0/1649158583 >> 172.21.15.97:0/1649158583 conn(0x5585287b5490 msgr2=0x5585287b5870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:55:47.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.806+0000 7fddd0861740 1 -- 172.21.15.97:0/1649158583 shutdown_connections 2024-03-27T10:55:47.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.806+0000 7fddd0861740 1 -- 172.21.15.97:0/1649158583 wait complete. 2024-03-27T10:55:47.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.806+0000 7fddd0861740 20 client.15153 trim_cache size 0 max 16384 2024-03-27T10:55:47.782 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-03-27T10:55:47.782 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "adca88c4-ec26-11ee-95d0-87774f69a715", 2024-03-27T10:55:47.783 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 0, 2024-03-27T10:55:47.783 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 34508, 2024-03-27T10:55:47.783 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2024-03-27T10:55:47.783 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2024-03-27T10:55:47.783 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2024-03-27T10:55:47.783 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 102.568957029, 2024-03-27T10:55:47.783 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 27, 2024-03-27T10:55:47.783 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 92, 2024-03-27T10:55:47.783 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 90, 2024-03-27T10:55:47.783 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 108.124514121 2024-03-27T10:55:47.783 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-03-27T10:55:47.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.807+0000 7fddcdc00640 1 -- 172.21.15.97:0/1736571697 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fddac0a3610 msgr2=0x7fddac0a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:47.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.807+0000 7fddcdc00640 1 --2- 172.21.15.97:0/1736571697 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fddac0a3610 0x7fddac0a5ad0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x5585287bb170 tx=0x7fddc0005e50 comp rx=0 tx=0).stop 2024-03-27T10:55:47.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.807+0000 7fddcdc00640 1 -- 172.21.15.97:0/1736571697 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fddc8104f80 msgr2=0x7fddc819a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:47.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.807+0000 7fddcdc00640 1 --2- 172.21.15.97:0/1736571697 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fddc8104f80 0x7fddc819a4f0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fddb0000c00 tx=0x7fddb00042f0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.807+0000 7fddcdc00640 1 -- 172.21.15.97:0/1736571697 shutdown_connections 2024-03-27T10:55:47.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.807+0000 7fddcdc00640 1 --2- 172.21.15.97:0/1736571697 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fddc8105940 0x7fddc81a1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.807+0000 7fddcdc00640 1 --2- 172.21.15.97:0/1736571697 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fddc8104f80 0x7fddc819a4f0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.807+0000 7fddcdc00640 1 --2- 172.21.15.97:0/1736571697 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fddac0a3610 0x7fddac0a5ad0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.807+0000 7fddcdc00640 1 --2- 172.21.15.97:0/1736571697 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fddc8103d90 0x7fddc8199fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:47.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.807+0000 7fddcdc00640 1 -- 172.21.15.97:0/1736571697 >> 172.21.15.97:0/1736571697 conn(0x7fddc80ff300 msgr2=0x7fddc81082d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:55:47.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.807+0000 7fddcdc00640 1 -- 172.21.15.97:0/1736571697 shutdown_connections 2024-03-27T10:55:47.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:47.807+0000 7fddcdc00640 1 -- 172.21.15.97:0/1736571697 wait complete. 2024-03-27T10:55:47.795 INFO:tasks.cephadm:Setup complete, yielding 2024-03-27T10:55:47.796 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-27T10:55:47.805 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi097.front.sepia.ceph.com 2024-03-27T10:55:47.806 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid adca88c4-ec26-11ee-95d0-87774f69a715 -- bash -c 'ceph orch status' 2024-03-27T10:55:48.241 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T10:55:49.417 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:55:49 smithi097 ceph-mon[31859]: pgmap v416: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 10 op/s 2024-03-27T10:55:49.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:55:49 smithi177 ceph-mon[35294]: pgmap v416: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 10 op/s 2024-03-27T10:55:49.767 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:55:49 smithi114 ceph-mon[35338]: pgmap v416: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 10 op/s 2024-03-27T10:55:50.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:50.305+0000 7f2da7400700 1 -- 172.21.15.97:0/3159999642 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2da0105a90 msgr2=0x7f2da0107e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:50.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:50.305+0000 7f2da7400700 1 --2- 172.21.15.97:0/3159999642 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2da0105a90 0x7f2da0107e80 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c009990 tx=0x7f2d8c009ca0 comp rx=0 tx=0).stop 2024-03-27T10:55:50.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:50.306+0000 7f2da7400700 1 -- 172.21.15.97:0/3159999642 shutdown_connections 2024-03-27T10:55:50.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:50.306+0000 7f2da7400700 1 --2- 172.21.15.97:0/3159999642 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2da01083c0 0x7f2da010a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:50.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:50.306+0000 7f2da7400700 1 --2- 172.21.15.97:0/3159999642 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2da0103160 0x7f2da0105550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:50.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:50.306+0000 7f2da7400700 1 --2- 172.21.15.97:0/3159999642 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2da0105a90 0x7f2da0107e80 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:50.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:50.306+0000 7f2da7400700 1 -- 172.21.15.97:0/3159999642 >> 172.21.15.97:0/3159999642 conn(0x7f2da00fcb10 msgr2=0x7f2da00fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:55:50.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:50.306+0000 7f2da7400700 1 -- 172.21.15.97:0/3159999642 shutdown_connections 2024-03-27T10:55:50.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:50.306+0000 7f2da7400700 1 -- 172.21.15.97:0/3159999642 wait complete. 2024-03-27T10:55:50.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:50.307+0000 7f2da7400700 1 Processor -- start 2024-03-27T10:55:50.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:50.307+0000 7f2da7400700 1 -- start start 2024-03-27T10:55:50.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:50.308+0000 7f2da7400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2da0103160 0x7f2da0198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:50.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:50.308+0000 7f2da7400700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2da0105a90 0x7f2da0198dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:50.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:50.308+0000 7f2da7400700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2da01083c0 0x7f2da019fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:50.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:50.308+0000 7f2da7400700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2da010e1c0 con 0x7f2da0103160 2024-03-27T10:55:50.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:50.308+0000 7f2da7400700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f2da010e040 con 0x7f2da0105a90 2024-03-27T10:55:50.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:50.308+0000 7f2da7400700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2da010e340 con 0x7f2da01083c0 2024-03-27T10:55:50.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:50.308+0000 7f2da4a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2da0103160 0x7f2da0198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:50.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:50.308+0000 7f2da4a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2da0103160 0x7f2da0198890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58354/0 (socket says 172.21.15.97:58354) 2024-03-27T10:55:50.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:50.308+0000 7f2da4a00700 1 -- 172.21.15.97:0/2353572765 learned_addr learned my addr 172.21.15.97:0/2353572765 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:55:50.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:50.308+0000 7f2d9fe00700 1 --2- 172.21.15.97:0/2353572765 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2da0105a90 0x7f2da0198dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:50.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:50.308+0000 7f2da4a00700 1 -- 172.21.15.97:0/2353572765 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2da01083c0 msgr2=0x7f2da019fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:50.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:50.308+0000 7f2da4a00700 1 --2- 172.21.15.97:0/2353572765 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2da01083c0 0x7f2da019fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:50.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:50.308+0000 7f2da4a00700 1 -- 172.21.15.97:0/2353572765 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2da0105a90 msgr2=0x7f2da0198dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:50.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:50.308+0000 7f2da4a00700 1 --2- 172.21.15.97:0/2353572765 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2da0105a90 0x7f2da0198dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:50.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:50.308+0000 7f2da4a00700 1 -- 172.21.15.97:0/2353572765 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2da01a04f0 con 0x7f2da0103160 2024-03-27T10:55:50.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:50.309+0000 7f2d9fe00700 1 --2- 172.21.15.97:0/2353572765 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2da0105a90 0x7f2da0198dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T10:55:50.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:50.309+0000 7f2da4a00700 1 --2- 172.21.15.97:0/2353572765 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2da0103160 0x7f2da0198890 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f2d9400d980 tx=0x7f2d9400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:55:50.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:50.309+0000 7f2d9d600700 1 -- 172.21.15.97:0/2353572765 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d94016070 con 0x7f2da0103160 2024-03-27T10:55:50.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:50.309+0000 7f2d9d600700 1 -- 172.21.15.97:0/2353572765 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2d94010460 con 0x7f2da0103160 2024-03-27T10:55:50.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:50.309+0000 7f2da7400700 1 -- 172.21.15.97:0/2353572765 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2da01a07e0 con 0x7f2da0103160 2024-03-27T10:55:50.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:50.309+0000 7f2d9d600700 1 -- 172.21.15.97:0/2353572765 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d9400f620 con 0x7f2da0103160 2024-03-27T10:55:50.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:50.309+0000 7f2da7400700 1 -- 172.21.15.97:0/2353572765 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2da01a0d20 con 0x7f2da0103160 2024-03-27T10:55:50.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:50.310+0000 7f2da7400700 1 -- 172.21.15.97:0/2353572765 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2da0069300 con 0x7f2da0103160 2024-03-27T10:55:50.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:50.313+0000 7f2d9d600700 1 -- 172.21.15.97:0/2353572765 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f2d94019070 con 0x7f2da0103160 2024-03-27T10:55:50.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:50.314+0000 7f2d9d600700 1 --2- 172.21.15.97:0/2353572765 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f2d880a36a0 0x7f2d880a5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:50.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:50.314+0000 7f2d9d600700 1 -- 172.21.15.97:0/2353572765 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f2d940cf900 con 0x7f2da0103160 2024-03-27T10:55:50.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:50.314+0000 7f2d9fe00700 1 --2- 172.21.15.97:0/2353572765 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f2d880a36a0 0x7f2d880a5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:50.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:50.315+0000 7f2d9fe00700 1 --2- 172.21.15.97:0/2353572765 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f2d880a36a0 0x7f2d880a5b60 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c00b5c0 tx=0x7f2d8c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:55:50.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:50.317+0000 7f2d9d600700 1 -- 172.21.15.97:0/2353572765 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f2d9409c480 con 0x7f2da0103160 2024-03-27T10:55:50.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:50.562+0000 7f2da7400700 1 -- 172.21.15.97:0/2353572765 --> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f2da0062a80 con 0x7f2d880a36a0 2024-03-27T10:55:50.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:50.564+0000 7f2d9d600700 1 -- 172.21.15.97:0/2353572765 <== mgr.14152 v2:172.21.15.97:6800/3955463538 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f2da0062a80 con 0x7f2d880a36a0 2024-03-27T10:55:50.537 INFO:teuthology.orchestra.run.smithi097.stdout:Backend: cephadm 2024-03-27T10:55:50.537 INFO:teuthology.orchestra.run.smithi097.stdout:Available: Yes 2024-03-27T10:55:50.537 INFO:teuthology.orchestra.run.smithi097.stdout:Paused: No 2024-03-27T10:55:50.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:50.567+0000 7f2da7400700 1 -- 172.21.15.97:0/2353572765 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f2d880a36a0 msgr2=0x7f2d880a5b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:50.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:50.567+0000 7f2da7400700 1 --2- 172.21.15.97:0/2353572765 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f2d880a36a0 0x7f2d880a5b60 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c00b5c0 tx=0x7f2d8c005fb0 comp rx=0 tx=0).stop 2024-03-27T10:55:50.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:50.567+0000 7f2da7400700 1 -- 172.21.15.97:0/2353572765 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2da0103160 msgr2=0x7f2da0198890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:50.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:50.567+0000 7f2da7400700 1 --2- 172.21.15.97:0/2353572765 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2da0103160 0x7f2da0198890 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f2d9400d980 tx=0x7f2d9400dd40 comp rx=0 tx=0).stop 2024-03-27T10:55:50.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:50.567+0000 7f2da7400700 1 -- 172.21.15.97:0/2353572765 shutdown_connections 2024-03-27T10:55:50.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:50.567+0000 7f2da7400700 1 --2- 172.21.15.97:0/2353572765 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f2d880a36a0 0x7f2d880a5b60 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:50.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:50.567+0000 7f2da7400700 1 --2- 172.21.15.97:0/2353572765 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2da01083c0 0x7f2da019fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:50.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:50.567+0000 7f2da7400700 1 --2- 172.21.15.97:0/2353572765 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2da0103160 0x7f2da0198890 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:50.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:50.567+0000 7f2da7400700 1 --2- 172.21.15.97:0/2353572765 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2da0105a90 0x7f2da0198dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:50.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:50.567+0000 7f2da7400700 1 -- 172.21.15.97:0/2353572765 >> 172.21.15.97:0/2353572765 conn(0x7f2da00fcb10 msgr2=0x7f2da0103a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:55:50.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:50.568+0000 7f2da7400700 1 -- 172.21.15.97:0/2353572765 shutdown_connections 2024-03-27T10:55:50.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:50.568+0000 7f2da7400700 1 -- 172.21.15.97:0/2353572765 wait complete. 2024-03-27T10:55:51.320 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid adca88c4-ec26-11ee-95d0-87774f69a715 -- bash -c 'ceph orch ps' 2024-03-27T10:55:51.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:55:51 smithi097 ceph-mon[31859]: pgmap v417: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 597 B/s wr, 12 op/s 2024-03-27T10:55:51.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:55:51 smithi177 ceph-mon[35294]: pgmap v417: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 597 B/s wr, 12 op/s 2024-03-27T10:55:51.767 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:55:51 smithi114 ceph-mon[35338]: pgmap v417: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 597 B/s wr, 12 op/s 2024-03-27T10:55:51.767 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T10:55:52.489 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:55:52 smithi097 ceph-mon[31859]: from='client.15159 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T10:55:52.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:55:52 smithi177 ceph-mon[35294]: from='client.15159 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T10:55:52.766 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:55:52 smithi114 ceph-mon[35338]: from='client.15159 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T10:55:53.641 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:55:53 smithi097 ceph-mon[31859]: pgmap v418: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 597 B/s wr, 9 op/s 2024-03-27T10:55:53.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:55:53 smithi177 ceph-mon[35294]: pgmap v418: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 597 B/s wr, 9 op/s 2024-03-27T10:55:53.767 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:55:53 smithi114 ceph-mon[35338]: pgmap v418: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 597 B/s wr, 9 op/s 2024-03-27T10:55:53.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:53.840+0000 7f5c6a600700 1 -- 172.21.15.97:0/3954823722 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c64105ac0 msgr2=0x7f5c64105f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:53.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:53.840+0000 7f5c6a600700 1 --2- 172.21.15.97:0/3954823722 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c64105ac0 0x7f5c64105f20 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f5c500099c0 tx=0x7f5c50009cd0 comp rx=0 tx=0).stop 2024-03-27T10:55:53.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:53.840+0000 7f5c6a600700 1 -- 172.21.15.97:0/3954823722 shutdown_connections 2024-03-27T10:55:53.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:53.840+0000 7f5c6a600700 1 --2- 172.21.15.97:0/3954823722 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5c64106460 0x7f5c6410cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:53.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:53.840+0000 7f5c6a600700 1 --2- 172.21.15.97:0/3954823722 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5c641048c0 0x7f5c64104ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:53.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:53.840+0000 7f5c6a600700 1 --2- 172.21.15.97:0/3954823722 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c64105ac0 0x7f5c64105f20 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:53.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:53.840+0000 7f5c6a600700 1 -- 172.21.15.97:0/3954823722 >> 172.21.15.97:0/3954823722 conn(0x7f5c640ffe60 msgr2=0x7f5c641022a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:55:53.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:53.840+0000 7f5c6a600700 1 -- 172.21.15.97:0/3954823722 shutdown_connections 2024-03-27T10:55:53.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:53.840+0000 7f5c6a600700 1 -- 172.21.15.97:0/3954823722 wait complete. 2024-03-27T10:55:53.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:53.841+0000 7f5c6a600700 1 Processor -- start 2024-03-27T10:55:53.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:53.841+0000 7f5c6a600700 1 -- start start 2024-03-27T10:55:53.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:53.842+0000 7f5c6a600700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5c641048c0 0x7f5c6419a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:53.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:53.842+0000 7f5c6a600700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c64105ac0 0x7f5c6419af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:53.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:53.842+0000 7f5c6a600700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5c64106460 0x7f5c641a1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:53.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:53.842+0000 7f5c6a600700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5c64110380 con 0x7f5c64105ac0 2024-03-27T10:55:53.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:53.842+0000 7f5c6a600700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f5c64110200 con 0x7f5c641048c0 2024-03-27T10:55:53.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:53.842+0000 7f5c6a600700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f5c64110500 con 0x7f5c64106460 2024-03-27T10:55:53.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:53.842+0000 7f5c62a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c64105ac0 0x7f5c6419af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:53.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:53.842+0000 7f5c62a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c64105ac0 0x7f5c6419af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58360/0 (socket says 172.21.15.97:58360) 2024-03-27T10:55:53.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:53.842+0000 7f5c62a00700 1 -- 172.21.15.97:0/2952171497 learned_addr learned my addr 172.21.15.97:0/2952171497 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:55:53.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:53.842+0000 7f5c63400700 1 --2- 172.21.15.97:0/2952171497 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5c641048c0 0x7f5c6419a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:53.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:53.843+0000 7f5c62a00700 1 -- 172.21.15.97:0/2952171497 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5c64106460 msgr2=0x7f5c641a1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:53.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:53.843+0000 7f5c63e00700 1 --2- 172.21.15.97:0/2952171497 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5c64106460 0x7f5c641a1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:53.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:53.843+0000 7f5c62a00700 1 --2- 172.21.15.97:0/2952171497 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5c64106460 0x7f5c641a1fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:53.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:53.843+0000 7f5c62a00700 1 -- 172.21.15.97:0/2952171497 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5c641048c0 msgr2=0x7f5c6419a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:53.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:53.843+0000 7f5c62a00700 1 --2- 172.21.15.97:0/2952171497 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5c641048c0 0x7f5c6419a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:53.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:53.843+0000 7f5c62a00700 1 -- 172.21.15.97:0/2952171497 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c641a2650 con 0x7f5c64105ac0 2024-03-27T10:55:53.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:53.843+0000 7f5c63400700 1 --2- 172.21.15.97:0/2952171497 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5c641048c0 0x7f5c6419a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T10:55:53.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:53.843+0000 7f5c62a00700 1 --2- 172.21.15.97:0/2952171497 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c64105ac0 0x7f5c6419af30 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f5c5000b580 tx=0x7f5c50003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:55:53.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:53.843+0000 7f5c5be00700 1 -- 172.21.15.97:0/2952171497 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c50027070 con 0x7f5c64105ac0 2024-03-27T10:55:53.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:53.843+0000 7f5c5be00700 1 -- 172.21.15.97:0/2952171497 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5c50004210 con 0x7f5c64105ac0 2024-03-27T10:55:53.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:53.843+0000 7f5c5be00700 1 -- 172.21.15.97:0/2952171497 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c5000f460 con 0x7f5c64105ac0 2024-03-27T10:55:53.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:53.843+0000 7f5c6a600700 1 -- 172.21.15.97:0/2952171497 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c641a28e0 con 0x7f5c64105ac0 2024-03-27T10:55:53.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:53.843+0000 7f5c6a600700 1 -- 172.21.15.97:0/2952171497 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c641a2ce0 con 0x7f5c64105ac0 2024-03-27T10:55:53.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:53.844+0000 7f5c6a600700 1 -- 172.21.15.97:0/2952171497 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c64069300 con 0x7f5c64105ac0 2024-03-27T10:55:53.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:53.846+0000 7f5c5be00700 1 -- 172.21.15.97:0/2952171497 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f5c5002a070 con 0x7f5c64105ac0 2024-03-27T10:55:53.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:53.848+0000 7f5c5be00700 1 --2- 172.21.15.97:0/2952171497 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f5c480a3720 0x7f5c480a5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:53.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:53.848+0000 7f5c5be00700 1 -- 172.21.15.97:0/2952171497 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f5c500d16e0 con 0x7f5c64105ac0 2024-03-27T10:55:53.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:53.848+0000 7f5c63400700 1 --2- 172.21.15.97:0/2952171497 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f5c480a3720 0x7f5c480a5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:53.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:53.848+0000 7f5c63400700 1 --2- 172.21.15.97:0/2952171497 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f5c480a3720 0x7f5c480a5be0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5c64105920 tx=0x7f5c54005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:55:53.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:53.850+0000 7f5c5be00700 1 -- 172.21.15.97:0/2952171497 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f5c50031020 con 0x7f5c64105ac0 2024-03-27T10:55:54.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:54.081+0000 7f5c6a600700 1 -- 172.21.15.97:0/2952171497 --> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f5c64062a80 con 0x7f5c480a3720 2024-03-27T10:55:54.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:54.104+0000 7f5c5be00700 1 -- 172.21.15.97:0/2952171497 <== mgr.14152 v2:172.21.15.97:6800/3955463538 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4185 (secure 0 0 0) 0x7f5c64062a80 con 0x7f5c480a3720 2024-03-27T10:55:54.077 INFO:teuthology.orchestra.run.smithi097.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-27T10:55:54.077 INFO:teuthology.orchestra.run.smithi097.stdout:mds.a smithi097 running (118s) 23s ago 118s 15.9M - 19.0.0-2478-g155268c4 b379bbd8da6b c886b2875232 2024-03-27T10:55:54.077 INFO:teuthology.orchestra.run.smithi097.stdout:mds.b smithi114 running (114s) 43s ago 114s 24.7M - 19.0.0-2478-g155268c4 b379bbd8da6b 0a34b4897eb0 2024-03-27T10:55:54.077 INFO:teuthology.orchestra.run.smithi097.stdout:mds.c smithi177 running (2m) 43s ago 2m 15.5M - 19.0.0-2478-g155268c4 b379bbd8da6b 5ffb343bec61 2024-03-27T10:55:54.077 INFO:teuthology.orchestra.run.smithi097.stdout:mds.d smithi097 running (2m) 23s ago 2m 12.5M - 19.0.0-2478-g155268c4 b379bbd8da6b 693e794650a1 2024-03-27T10:55:54.077 INFO:teuthology.orchestra.run.smithi097.stdout:mds.e smithi114 running (2m) 43s ago 2m 16.6M - 19.0.0-2478-g155268c4 b379bbd8da6b 427c9c478515 2024-03-27T10:55:54.077 INFO:teuthology.orchestra.run.smithi097.stdout:mds.f smithi177 running (2m) 43s ago 2m 16.4M - 19.0.0-2478-g155268c4 b379bbd8da6b 4a238616523c 2024-03-27T10:55:54.077 INFO:teuthology.orchestra.run.smithi097.stdout:mds.g smithi097 running (2m) 23s ago 2m 16.0M - 19.0.0-2478-g155268c4 b379bbd8da6b d58bb8d44971 2024-03-27T10:55:54.077 INFO:teuthology.orchestra.run.smithi097.stdout:mds.h smithi114 running (2m) 43s ago 2m 16.7M - 19.0.0-2478-g155268c4 b379bbd8da6b 21762b79bb89 2024-03-27T10:55:54.078 INFO:teuthology.orchestra.run.smithi097.stdout:mds.i smithi177 running (2m) 43s ago 2m 15.5M - 19.0.0-2478-g155268c4 b379bbd8da6b 2e7f049a1279 2024-03-27T10:55:54.078 INFO:teuthology.orchestra.run.smithi097.stdout:mds.j smithi097 running (2m) 23s ago 2m 12.5M - 19.0.0-2478-g155268c4 b379bbd8da6b c8fdfbf183e0 2024-03-27T10:55:54.078 INFO:teuthology.orchestra.run.smithi097.stdout:mds.k smithi114 running (2m) 43s ago 2m 16.9M - 19.0.0-2478-g155268c4 b379bbd8da6b 012040f065d4 2024-03-27T10:55:54.078 INFO:teuthology.orchestra.run.smithi097.stdout:mds.l smithi177 running (2m) 43s ago 2m 16.2M - 19.0.0-2478-g155268c4 b379bbd8da6b 5ebaf095548b 2024-03-27T10:55:54.078 INFO:teuthology.orchestra.run.smithi097.stdout:mgr.x smithi097 *:9283,8765 running (12m) 23s ago 12m 514M - 19.0.0-2478-g155268c4 b379bbd8da6b 662f8153b410 2024-03-27T10:55:54.078 INFO:teuthology.orchestra.run.smithi097.stdout:mgr.y smithi114 *:8443,8765 running (9m) 43s ago 9m 452M - 19.0.0-2478-g155268c4 b379bbd8da6b 72bfb2b6dd9b 2024-03-27T10:55:54.078 INFO:teuthology.orchestra.run.smithi097.stdout:mgr.z smithi177 *:8443,8765 running (9m) 43s ago 9m 451M - 19.0.0-2478-g155268c4 b379bbd8da6b f9408119d0ec 2024-03-27T10:55:54.078 INFO:teuthology.orchestra.run.smithi097.stdout:mon.a smithi097 running (12m) 23s ago 12m 50.6M 2048M 19.0.0-2478-g155268c4 b379bbd8da6b 08bedf620c24 2024-03-27T10:55:54.078 INFO:teuthology.orchestra.run.smithi097.stdout:mon.b smithi114 running (9m) 43s ago 9m 40.1M 2048M 19.0.0-2478-g155268c4 b379bbd8da6b be7a212c5f9d 2024-03-27T10:55:54.078 INFO:teuthology.orchestra.run.smithi097.stdout:mon.c smithi177 running (10m) 43s ago 10m 40.7M 2048M 19.0.0-2478-g155268c4 b379bbd8da6b 181e3ed3cffe 2024-03-27T10:55:54.078 INFO:teuthology.orchestra.run.smithi097.stdout:osd.0 smithi097 running (9m) 23s ago 9m 47.2M 4272M 19.0.0-2478-g155268c4 b379bbd8da6b 50851d00ea4c 2024-03-27T10:55:54.078 INFO:teuthology.orchestra.run.smithi097.stdout:osd.1 smithi114 running (8m) 43s ago 8m 48.2M 4272M 19.0.0-2478-g155268c4 b379bbd8da6b 079e646c3b07 2024-03-27T10:55:54.079 INFO:teuthology.orchestra.run.smithi097.stdout:osd.2 smithi177 running (8m) 43s ago 8m 49.4M 4272M 19.0.0-2478-g155268c4 b379bbd8da6b 17f47ec38255 2024-03-27T10:55:54.079 INFO:teuthology.orchestra.run.smithi097.stdout:osd.3 smithi097 running (7m) 23s ago 7m 56.4M 4272M 19.0.0-2478-g155268c4 b379bbd8da6b c5a89c0f5607 2024-03-27T10:55:54.079 INFO:teuthology.orchestra.run.smithi097.stdout:osd.4 smithi114 running (7m) 43s ago 7m 51.2M 4272M 19.0.0-2478-g155268c4 b379bbd8da6b 3e575c8ed6a5 2024-03-27T10:55:54.079 INFO:teuthology.orchestra.run.smithi097.stdout:osd.5 smithi177 running (6m) 43s ago 6m 53.4M 4272M 19.0.0-2478-g155268c4 b379bbd8da6b b7b79355c0dc 2024-03-27T10:55:54.079 INFO:teuthology.orchestra.run.smithi097.stdout:osd.6 smithi097 running (5m) 23s ago 5m 44.4M 4272M 19.0.0-2478-g155268c4 b379bbd8da6b 864b71757d0d 2024-03-27T10:55:54.079 INFO:teuthology.orchestra.run.smithi097.stdout:osd.7 smithi114 running (5m) 43s ago 5m 45.6M 4272M 19.0.0-2478-g155268c4 b379bbd8da6b c6da1bc97668 2024-03-27T10:55:54.079 INFO:teuthology.orchestra.run.smithi097.stdout:osd.8 smithi177 running (4m) 43s ago 4m 52.2M 4272M 19.0.0-2478-g155268c4 b379bbd8da6b 80d92c661786 2024-03-27T10:55:54.079 INFO:teuthology.orchestra.run.smithi097.stdout:osd.9 smithi097 running (4m) 23s ago 4m 49.2M 4272M 19.0.0-2478-g155268c4 b379bbd8da6b ddeecfa0134c 2024-03-27T10:55:54.079 INFO:teuthology.orchestra.run.smithi097.stdout:osd.10 smithi114 running (3m) 43s ago 3m 52.6M 4272M 19.0.0-2478-g155268c4 b379bbd8da6b 2eeb80282945 2024-03-27T10:55:54.079 INFO:teuthology.orchestra.run.smithi097.stdout:osd.11 smithi177 running (3m) 43s ago 3m 42.2M 4272M 19.0.0-2478-g155268c4 b379bbd8da6b 40b79e22b3c8 2024-03-27T10:55:54.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:54.108+0000 7f5c6a600700 1 -- 172.21.15.97:0/2952171497 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f5c480a3720 msgr2=0x7f5c480a5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:54.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:54.108+0000 7f5c6a600700 1 --2- 172.21.15.97:0/2952171497 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f5c480a3720 0x7f5c480a5be0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5c64105920 tx=0x7f5c54005f90 comp rx=0 tx=0).stop 2024-03-27T10:55:54.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:54.108+0000 7f5c6a600700 1 -- 172.21.15.97:0/2952171497 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c64105ac0 msgr2=0x7f5c6419af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:54.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:54.108+0000 7f5c6a600700 1 --2- 172.21.15.97:0/2952171497 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c64105ac0 0x7f5c6419af30 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f5c5000b580 tx=0x7f5c50003680 comp rx=0 tx=0).stop 2024-03-27T10:55:54.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:54.108+0000 7f5c6a600700 1 -- 172.21.15.97:0/2952171497 shutdown_connections 2024-03-27T10:55:54.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:54.108+0000 7f5c6a600700 1 --2- 172.21.15.97:0/2952171497 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f5c480a3720 0x7f5c480a5be0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:54.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:54.108+0000 7f5c6a600700 1 --2- 172.21.15.97:0/2952171497 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5c64106460 0x7f5c641a1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:54.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:54.108+0000 7f5c6a600700 1 --2- 172.21.15.97:0/2952171497 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5c641048c0 0x7f5c6419a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:54.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:54.108+0000 7f5c6a600700 1 --2- 172.21.15.97:0/2952171497 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c64105ac0 0x7f5c6419af30 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:54.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:54.108+0000 7f5c6a600700 1 -- 172.21.15.97:0/2952171497 >> 172.21.15.97:0/2952171497 conn(0x7f5c640ffe60 msgr2=0x7f5c64108e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:55:54.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:54.108+0000 7f5c6a600700 1 -- 172.21.15.97:0/2952171497 shutdown_connections 2024-03-27T10:55:54.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:54.108+0000 7f5c6a600700 1 -- 172.21.15.97:0/2952171497 wait complete. 2024-03-27T10:55:54.885 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid adca88c4-ec26-11ee-95d0-87774f69a715 -- bash -c 'ceph orch ls' 2024-03-27T10:55:55.327 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T10:55:55.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:55:55 smithi097 ceph-mon[31859]: from='client.15165 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T10:55:55.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:55:55 smithi097 ceph-mon[31859]: pgmap v419: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 597 B/s wr, 12 op/s 2024-03-27T10:55:55.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:55:55 smithi177 ceph-mon[35294]: from='client.15165 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T10:55:55.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:55:55 smithi177 ceph-mon[35294]: pgmap v419: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 597 B/s wr, 12 op/s 2024-03-27T10:55:55.767 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:55:55 smithi114 ceph-mon[35338]: from='client.15165 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T10:55:55.767 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:55:55 smithi114 ceph-mon[35338]: pgmap v419: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 597 B/s wr, 12 op/s 2024-03-27T10:55:56.514 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:55:56 smithi097 ceph-mon[31859]: pgmap v420: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-03-27T10:55:56.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:55:56 smithi177 ceph-mon[35294]: pgmap v420: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-03-27T10:55:56.767 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:55:56 smithi114 ceph-mon[35338]: pgmap v420: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-03-27T10:55:57.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:57.356+0000 7f6aa9200700 1 -- 172.21.15.97:0/4248487794 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6aa4076670 msgr2=0x7f6aa4074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:57.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:57.356+0000 7f6aa9200700 1 --2- 172.21.15.97:0/4248487794 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6aa4076670 0x7f6aa4074ad0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f6a9400b0a0 tx=0x7f6a9400b3b0 comp rx=0 tx=0).stop 2024-03-27T10:55:57.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:57.356+0000 7f6aa9200700 1 -- 172.21.15.97:0/4248487794 shutdown_connections 2024-03-27T10:55:57.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:57.356+0000 7f6aa9200700 1 --2- 172.21.15.97:0/4248487794 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6aa410ec60 0x7f6aa4111050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:57.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:57.356+0000 7f6aa9200700 1 --2- 172.21.15.97:0/4248487794 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6aa4076670 0x7f6aa4074ad0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:57.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:57.356+0000 7f6aa9200700 1 --2- 172.21.15.97:0/4248487794 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6aa40750a0 0x7f6aa4075520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:57.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:57.356+0000 7f6aa9200700 1 -- 172.21.15.97:0/4248487794 >> 172.21.15.97:0/4248487794 conn(0x7f6aa40fdbe0 msgr2=0x7f6aa4100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:55:57.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:57.356+0000 7f6aa9200700 1 -- 172.21.15.97:0/4248487794 shutdown_connections 2024-03-27T10:55:57.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:57.357+0000 7f6aa9200700 1 -- 172.21.15.97:0/4248487794 wait complete. 2024-03-27T10:55:57.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:57.357+0000 7f6aa9200700 1 Processor -- start 2024-03-27T10:55:57.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:57.358+0000 7f6aa9200700 1 -- start start 2024-03-27T10:55:57.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:57.358+0000 7f6aa9200700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6aa40750a0 0x7f6aa419edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:57.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:57.358+0000 7f6aa9200700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6aa4076670 0x7f6aa419f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:57.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:57.358+0000 7f6aa9200700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6aa410ec60 0x7f6aa41a63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:57.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:57.358+0000 7f6aa9200700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa4114740 con 0x7f6aa4076670 2024-03-27T10:55:57.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:57.358+0000 7f6aa9200700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa41145c0 con 0x7f6aa410ec60 2024-03-27T10:55:57.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:57.358+0000 7f6aa9200700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa41148c0 con 0x7f6aa40750a0 2024-03-27T10:55:57.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:57.358+0000 7f6a99400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6aa4076670 0x7f6aa419f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:57.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:57.358+0000 7f6aa2800700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6aa410ec60 0x7f6aa41a63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:57.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:57.358+0000 7f6a99400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6aa4076670 0x7f6aa419f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58384/0 (socket says 172.21.15.97:58384) 2024-03-27T10:55:57.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:57.358+0000 7f6a99400700 1 -- 172.21.15.97:0/423467713 learned_addr learned my addr 172.21.15.97:0/423467713 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:55:57.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:57.359+0000 7f6aa1e00700 1 --2- 172.21.15.97:0/423467713 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6aa40750a0 0x7f6aa419edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:57.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:57.359+0000 7f6a99400700 1 -- 172.21.15.97:0/423467713 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6aa40750a0 msgr2=0x7f6aa419edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:57.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:57.359+0000 7f6a99400700 1 --2- 172.21.15.97:0/423467713 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6aa40750a0 0x7f6aa419edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:57.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:57.359+0000 7f6a99400700 1 -- 172.21.15.97:0/423467713 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6aa410ec60 msgr2=0x7f6aa41a63b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:57.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:57.359+0000 7f6a99400700 1 --2- 172.21.15.97:0/423467713 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6aa410ec60 0x7f6aa41a63b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:57.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:57.359+0000 7f6a99400700 1 -- 172.21.15.97:0/423467713 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6aa41a6a50 con 0x7f6aa4076670 2024-03-27T10:55:57.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:57.359+0000 7f6aa1e00700 1 --2- 172.21.15.97:0/423467713 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6aa40750a0 0x7f6aa419edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T10:55:57.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:57.359+0000 7f6a99400700 1 --2- 172.21.15.97:0/423467713 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6aa4076670 0x7f6aa419f330 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c00c9a0 tx=0x7f6a8c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:55:57.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:57.359+0000 7f6a9b400700 1 -- 172.21.15.97:0/423467713 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a8c004220 con 0x7f6aa4076670 2024-03-27T10:55:57.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:57.360+0000 7f6a9b400700 1 -- 172.21.15.97:0/423467713 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6a8c007d10 con 0x7f6aa4076670 2024-03-27T10:55:57.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:57.360+0000 7f6aa9200700 1 -- 172.21.15.97:0/423467713 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6aa41a6d40 con 0x7f6aa4076670 2024-03-27T10:55:57.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:57.360+0000 7f6a9b400700 1 -- 172.21.15.97:0/423467713 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a8c003d00 con 0x7f6aa4076670 2024-03-27T10:55:57.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:57.360+0000 7f6aa9200700 1 -- 172.21.15.97:0/423467713 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6aa41a7220 con 0x7f6aa4076670 2024-03-27T10:55:57.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:57.361+0000 7f6aa9200700 1 -- 172.21.15.97:0/423467713 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6aa4069300 con 0x7f6aa4076670 2024-03-27T10:55:57.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:57.363+0000 7f6a9b400700 1 -- 172.21.15.97:0/423467713 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f6a8c007860 con 0x7f6aa4076670 2024-03-27T10:55:57.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:57.364+0000 7f6a9b400700 1 --2- 172.21.15.97:0/423467713 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f6a880a36a0 0x7f6a880a5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:55:57.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:57.364+0000 7f6a9b400700 1 -- 172.21.15.97:0/423467713 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f6a8c0cf970 con 0x7f6aa4076670 2024-03-27T10:55:57.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:57.364+0000 7f6aa1e00700 1 --2- 172.21.15.97:0/423467713 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f6a880a36a0 0x7f6a880a5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:55:57.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:57.365+0000 7f6aa1e00700 1 --2- 172.21.15.97:0/423467713 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f6a880a36a0 0x7f6a880a5b60 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f6a940060b0 tx=0x7f6a94006020 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:55:57.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:57.371+0000 7f6a9b400700 1 -- 172.21.15.97:0/423467713 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f6a8c014030 con 0x7f6aa4076670 2024-03-27T10:55:57.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:57.607+0000 7f6aa9200700 1 -- 172.21.15.97:0/423467713 --> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f6aa4062a80 con 0x7f6a880a36a0 2024-03-27T10:55:57.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:57.625+0000 7f6a9b400700 1 -- 172.21.15.97:0/423467713 <== mgr.14152 v2:172.21.15.97:6800/3955463538 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f6aa4062a80 con 0x7f6a880a36a0 2024-03-27T10:55:57.597 INFO:teuthology.orchestra.run.smithi097.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-03-27T10:55:57.597 INFO:teuthology.orchestra.run.smithi097.stdout:mds.all 12/12 46s ago 2m smithi097=a;smithi097=d;smithi097=g;smithi097=j;smithi114=b;smithi114=e;smithi114=h;smithi114=k;smithi177=c;smithi177=f;smithi177=i;smithi177=l;count:12 2024-03-27T10:55:57.598 INFO:teuthology.orchestra.run.smithi097.stdout:mgr 3/3 46s ago 9m smithi097=x;smithi114=y;smithi177=z;count:3 2024-03-27T10:55:57.598 INFO:teuthology.orchestra.run.smithi097.stdout:mon 3/3 46s ago 10m smithi097:172.21.15.97=a;smithi114:172.21.15.114=b;smithi177:172.21.15.177=c;count:3 2024-03-27T10:55:57.598 INFO:teuthology.orchestra.run.smithi097.stdout:osd 12 46s ago - 2024-03-27T10:55:57.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:57.627+0000 7f6aa9200700 1 -- 172.21.15.97:0/423467713 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f6a880a36a0 msgr2=0x7f6a880a5b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:57.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:57.627+0000 7f6aa9200700 1 --2- 172.21.15.97:0/423467713 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f6a880a36a0 0x7f6a880a5b60 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f6a940060b0 tx=0x7f6a94006020 comp rx=0 tx=0).stop 2024-03-27T10:55:57.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:57.627+0000 7f6aa9200700 1 -- 172.21.15.97:0/423467713 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6aa4076670 msgr2=0x7f6aa419f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:55:57.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:57.627+0000 7f6aa9200700 1 --2- 172.21.15.97:0/423467713 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6aa4076670 0x7f6aa419f330 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c00c9a0 tx=0x7f6a8c00ccb0 comp rx=0 tx=0).stop 2024-03-27T10:55:57.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:57.628+0000 7f6aa9200700 1 -- 172.21.15.97:0/423467713 shutdown_connections 2024-03-27T10:55:57.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:57.628+0000 7f6aa9200700 1 --2- 172.21.15.97:0/423467713 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f6a880a36a0 0x7f6a880a5b60 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:57.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:57.628+0000 7f6aa9200700 1 --2- 172.21.15.97:0/423467713 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6aa410ec60 0x7f6aa41a63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:57.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:57.628+0000 7f6aa9200700 1 --2- 172.21.15.97:0/423467713 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6aa40750a0 0x7f6aa419edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:57.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:57.628+0000 7f6aa9200700 1 --2- 172.21.15.97:0/423467713 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6aa4076670 0x7f6aa419f330 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:55:57.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:57.628+0000 7f6aa9200700 1 -- 172.21.15.97:0/423467713 >> 172.21.15.97:0/423467713 conn(0x7f6aa40fdbe0 msgr2=0x7f6aa4102000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:55:57.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:57.628+0000 7f6aa9200700 1 -- 172.21.15.97:0/423467713 shutdown_connections 2024-03-27T10:55:57.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:55:57.628+0000 7f6aa9200700 1 -- 172.21.15.97:0/423467713 wait complete. 2024-03-27T10:55:58.423 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid adca88c4-ec26-11ee-95d0-87774f69a715 -- bash -c 'ceph orch host ls' 2024-03-27T10:55:58.872 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T10:55:59.293 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:55:59 smithi097 ceph-mon[31859]: from='client.15171 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T10:55:59.293 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:55:59 smithi097 ceph-mon[31859]: pgmap v421: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-27T10:55:59.426 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:55:59 smithi177 ceph-mon[35294]: from='client.15171 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T10:55:59.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:55:59 smithi177 ceph-mon[35294]: pgmap v421: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-27T10:55:59.516 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:55:59 smithi114 ceph-mon[35338]: from='client.15171 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T10:55:59.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:55:59 smithi114 ceph-mon[35338]: pgmap v421: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-27T10:56:00.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:00.926+0000 7f1f5a000700 1 -- 172.21.15.97:0/1628544827 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f541018b0 msgr2=0x7f1f54101d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:00.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:00.926+0000 7f1f5a000700 1 --2- 172.21.15.97:0/1628544827 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f541018b0 0x7f1f54101d30 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f1f40009990 tx=0x7f1f40009ca0 comp rx=0 tx=0).stop 2024-03-27T10:56:00.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:00.926+0000 7f1f5a000700 1 -- 172.21.15.97:0/1628544827 shutdown_connections 2024-03-27T10:56:00.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:00.926+0000 7f1f5a000700 1 --2- 172.21.15.97:0/1628544827 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1f54102270 0x7f1f541026f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:00.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:00.926+0000 7f1f5a000700 1 --2- 172.21.15.97:0/1628544827 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1f54100f50 0x7f1f54101370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:00.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:00.926+0000 7f1f5a000700 1 --2- 172.21.15.97:0/1628544827 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f541018b0 0x7f1f54101d30 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:00.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:00.926+0000 7f1f5a000700 1 -- 172.21.15.97:0/1628544827 >> 172.21.15.97:0/1628544827 conn(0x7f1f540fcb10 msgr2=0x7f1f540fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:00.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:00.927+0000 7f1f5a000700 1 -- 172.21.15.97:0/1628544827 shutdown_connections 2024-03-27T10:56:00.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:00.927+0000 7f1f5a000700 1 -- 172.21.15.97:0/1628544827 wait complete. 2024-03-27T10:56:00.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:00.928+0000 7f1f5a000700 1 Processor -- start 2024-03-27T10:56:00.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:00.928+0000 7f1f5a000700 1 -- start start 2024-03-27T10:56:00.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:00.928+0000 7f1f5a000700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1f54100f50 0x7f1f5419aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:00.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:00.928+0000 7f1f5a000700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1f541018b0 0x7f1f5419af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:00.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:00.928+0000 7f1f5a000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f54102270 0x7f1f541a2010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:00.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:00.928+0000 7f1f5a000700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1f54110340 con 0x7f1f54102270 2024-03-27T10:56:00.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:00.928+0000 7f1f5a000700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f1f541101c0 con 0x7f1f54100f50 2024-03-27T10:56:00.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:00.928+0000 7f1f5a000700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1f541104c0 con 0x7f1f541018b0 2024-03-27T10:56:00.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:00.929+0000 7f1f53e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f54102270 0x7f1f541a2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:00.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:00.929+0000 7f1f53e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f54102270 0x7f1f541a2010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56732/0 (socket says 172.21.15.97:56732) 2024-03-27T10:56:00.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:00.929+0000 7f1f53e00700 1 -- 172.21.15.97:0/2439359186 learned_addr learned my addr 172.21.15.97:0/2439359186 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:56:00.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:00.929+0000 7f1f52a00700 1 --2- 172.21.15.97:0/2439359186 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1f541018b0 0x7f1f5419af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:00.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:00.929+0000 7f1f53400700 1 --2- 172.21.15.97:0/2439359186 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1f54100f50 0x7f1f5419aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:00.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:00.929+0000 7f1f53e00700 1 -- 172.21.15.97:0/2439359186 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1f541018b0 msgr2=0x7f1f5419af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:00.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:00.929+0000 7f1f53e00700 1 --2- 172.21.15.97:0/2439359186 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1f541018b0 0x7f1f5419af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:00.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:00.929+0000 7f1f53e00700 1 -- 172.21.15.97:0/2439359186 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1f54100f50 msgr2=0x7f1f5419aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:00.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:00.929+0000 7f1f53e00700 1 --2- 172.21.15.97:0/2439359186 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1f54100f50 0x7f1f5419aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:00.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:00.929+0000 7f1f53e00700 1 -- 172.21.15.97:0/2439359186 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f541a26b0 con 0x7f1f54102270 2024-03-27T10:56:00.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:00.929+0000 7f1f52a00700 1 --2- 172.21.15.97:0/2439359186 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1f541018b0 0x7f1f5419af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T10:56:00.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:00.929+0000 7f1f53e00700 1 --2- 172.21.15.97:0/2439359186 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f54102270 0x7f1f541a2010 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c00d9b0 tx=0x7f1f4c00dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:00.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:00.930+0000 7f1f4be00700 1 -- 172.21.15.97:0/2439359186 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f4c017070 con 0x7f1f54102270 2024-03-27T10:56:00.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:00.930+0000 7f1f4be00700 1 -- 172.21.15.97:0/2439359186 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1f4c009df0 con 0x7f1f54102270 2024-03-27T10:56:00.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:00.930+0000 7f1f5a000700 1 -- 172.21.15.97:0/2439359186 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f541a29a0 con 0x7f1f54102270 2024-03-27T10:56:00.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:00.930+0000 7f1f4be00700 1 -- 172.21.15.97:0/2439359186 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f4c0053a0 con 0x7f1f54102270 2024-03-27T10:56:00.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:00.930+0000 7f1f5a000700 1 -- 172.21.15.97:0/2439359186 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f541a2e90 con 0x7f1f54102270 2024-03-27T10:56:00.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:00.931+0000 7f1f5a000700 1 -- 172.21.15.97:0/2439359186 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f54069300 con 0x7f1f54102270 2024-03-27T10:56:00.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:00.933+0000 7f1f4be00700 1 -- 172.21.15.97:0/2439359186 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f1f4c004750 con 0x7f1f54102270 2024-03-27T10:56:00.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:00.934+0000 7f1f4be00700 1 --2- 172.21.15.97:0/2439359186 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f1f380a36a0 0x7f1f380a5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:00.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:00.934+0000 7f1f4be00700 1 -- 172.21.15.97:0/2439359186 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f1f4c0cfc30 con 0x7f1f54102270 2024-03-27T10:56:00.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:00.935+0000 7f1f53400700 1 --2- 172.21.15.97:0/2439359186 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f1f380a36a0 0x7f1f380a5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:00.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:00.935+0000 7f1f53400700 1 --2- 172.21.15.97:0/2439359186 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f1f380a36a0 0x7f1f380a5b60 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1f44009c10 tx=0x7f1f44009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:00.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:00.939+0000 7f1f4be00700 1 -- 172.21.15.97:0/2439359186 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f1f4c020020 con 0x7f1f54102270 2024-03-27T10:56:01.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:01.173+0000 7f1f5a000700 1 -- 172.21.15.97:0/2439359186 --> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f1f54062b40 con 0x7f1f380a36a0 2024-03-27T10:56:01.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:01.175+0000 7f1f4be00700 1 -- 172.21.15.97:0/2439359186 <== mgr.14152 v2:172.21.15.97:6800/3955463538 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f1f54062b40 con 0x7f1f380a36a0 2024-03-27T10:56:01.147 INFO:teuthology.orchestra.run.smithi097.stdout:HOST ADDR LABELS STATUS 2024-03-27T10:56:01.148 INFO:teuthology.orchestra.run.smithi097.stdout:smithi097 172.21.15.97 2024-03-27T10:56:01.148 INFO:teuthology.orchestra.run.smithi097.stdout:smithi114 172.21.15.114 2024-03-27T10:56:01.148 INFO:teuthology.orchestra.run.smithi097.stdout:smithi177 172.21.15.177 2024-03-27T10:56:01.148 INFO:teuthology.orchestra.run.smithi097.stdout:3 hosts in cluster 2024-03-27T10:56:01.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:01.178+0000 7f1f5a000700 1 -- 172.21.15.97:0/2439359186 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f1f380a36a0 msgr2=0x7f1f380a5b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:01.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:01.178+0000 7f1f5a000700 1 --2- 172.21.15.97:0/2439359186 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f1f380a36a0 0x7f1f380a5b60 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1f44009c10 tx=0x7f1f44009040 comp rx=0 tx=0).stop 2024-03-27T10:56:01.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:01.178+0000 7f1f5a000700 1 -- 172.21.15.97:0/2439359186 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f54102270 msgr2=0x7f1f541a2010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:01.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:01.178+0000 7f1f5a000700 1 --2- 172.21.15.97:0/2439359186 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f54102270 0x7f1f541a2010 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c00d9b0 tx=0x7f1f4c00dd70 comp rx=0 tx=0).stop 2024-03-27T10:56:01.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:01.178+0000 7f1f5a000700 1 -- 172.21.15.97:0/2439359186 shutdown_connections 2024-03-27T10:56:01.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:01.178+0000 7f1f5a000700 1 --2- 172.21.15.97:0/2439359186 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f1f380a36a0 0x7f1f380a5b60 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:01.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:01.178+0000 7f1f5a000700 1 --2- 172.21.15.97:0/2439359186 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f54102270 0x7f1f541a2010 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:01.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:01.178+0000 7f1f5a000700 1 --2- 172.21.15.97:0/2439359186 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1f54100f50 0x7f1f5419aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:01.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:01.178+0000 7f1f5a000700 1 --2- 172.21.15.97:0/2439359186 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1f541018b0 0x7f1f5419af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:01.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:01.178+0000 7f1f5a000700 1 -- 172.21.15.97:0/2439359186 >> 172.21.15.97:0/2439359186 conn(0x7f1f540fcb10 msgr2=0x7f1f540fe610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:01.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:01.178+0000 7f1f5a000700 1 -- 172.21.15.97:0/2439359186 shutdown_connections 2024-03-27T10:56:01.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:01.178+0000 7f1f5a000700 1 -- 172.21.15.97:0/2439359186 wait complete. 2024-03-27T10:56:01.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:56:01 smithi097 ceph-mon[31859]: pgmap v422: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-03-27T10:56:01.426 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:56:01 smithi177 ceph-mon[35294]: pgmap v422: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-03-27T10:56:01.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:56:01 smithi114 ceph-mon[35338]: pgmap v422: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-03-27T10:56:01.934 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid adca88c4-ec26-11ee-95d0-87774f69a715 -- bash -c 'ceph orch device ls' 2024-03-27T10:56:02.385 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T10:56:02.426 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:56:02 smithi177 ceph-mon[35294]: from='client.15177 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T10:56:02.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:56:02 smithi114 ceph-mon[35338]: from='client.15177 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T10:56:02.534 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:56:02 smithi097 ceph-mon[31859]: from='client.15177 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T10:56:03.426 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:56:03 smithi177 ceph-mon[35294]: pgmap v423: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-03-27T10:56:03.516 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:56:03 smithi114 ceph-mon[35338]: pgmap v423: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-03-27T10:56:03.641 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:56:03 smithi097 ceph-mon[31859]: pgmap v423: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-03-27T10:56:04.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:04.456+0000 7ff527400700 1 -- 172.21.15.97:0/2411843093 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5201048b0 msgr2=0x7ff520104cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:04.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:04.456+0000 7ff527400700 1 --2- 172.21.15.97:0/2411843093 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5201048b0 0x7ff520104cd0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7ff51000b0d0 tx=0x7ff51000b3e0 comp rx=0 tx=0).stop 2024-03-27T10:56:04.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:04.456+0000 7ff527400700 1 -- 172.21.15.97:0/2411843093 shutdown_connections 2024-03-27T10:56:04.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:04.456+0000 7ff527400700 1 --2- 172.21.15.97:0/2411843093 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff520106450 0x7ff52010cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:04.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:04.456+0000 7ff527400700 1 --2- 172.21.15.97:0/2411843093 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5201048b0 0x7ff520104cd0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:04.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:04.456+0000 7ff527400700 1 --2- 172.21.15.97:0/2411843093 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff520105ab0 0x7ff520105f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:04.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:04.457+0000 7ff527400700 1 -- 172.21.15.97:0/2411843093 >> 172.21.15.97:0/2411843093 conn(0x7ff5200ffe50 msgr2=0x7ff520102290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:04.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:04.457+0000 7ff527400700 1 -- 172.21.15.97:0/2411843093 shutdown_connections 2024-03-27T10:56:04.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:04.457+0000 7ff527400700 1 -- 172.21.15.97:0/2411843093 wait complete. 2024-03-27T10:56:04.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:04.458+0000 7ff527400700 1 Processor -- start 2024-03-27T10:56:04.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:04.458+0000 7ff527400700 1 -- start start 2024-03-27T10:56:04.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:04.458+0000 7ff527400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5201048b0 0x7ff52019aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:04.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:04.458+0000 7ff527400700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff520105ab0 0x7ff52019af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:04.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:04.458+0000 7ff527400700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff520106450 0x7ff5201a2010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:04.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:04.458+0000 7ff527400700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff520110410 con 0x7ff5201048b0 2024-03-27T10:56:04.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:04.458+0000 7ff527400700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff520110290 con 0x7ff520105ab0 2024-03-27T10:56:04.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:04.458+0000 7ff527400700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff520110590 con 0x7ff520106450 2024-03-27T10:56:04.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:04.458+0000 7ff524a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5201048b0 0x7ff52019aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:04.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:04.458+0000 7ff524a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5201048b0 0x7ff52019aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56760/0 (socket says 172.21.15.97:56760) 2024-03-27T10:56:04.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:04.458+0000 7ff524a00700 1 -- 172.21.15.97:0/1700068279 learned_addr learned my addr 172.21.15.97:0/1700068279 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:56:04.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:04.459+0000 7ff517e00700 1 --2- 172.21.15.97:0/1700068279 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff520105ab0 0x7ff52019af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:04.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:04.459+0000 7ff525400700 1 --2- 172.21.15.97:0/1700068279 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff520106450 0x7ff5201a2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:04.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:04.459+0000 7ff524a00700 1 -- 172.21.15.97:0/1700068279 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff520106450 msgr2=0x7ff5201a2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:04.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:04.459+0000 7ff524a00700 1 --2- 172.21.15.97:0/1700068279 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff520106450 0x7ff5201a2010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:04.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:04.459+0000 7ff524a00700 1 -- 172.21.15.97:0/1700068279 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff520105ab0 msgr2=0x7ff52019af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:04.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:04.459+0000 7ff524a00700 1 --2- 172.21.15.97:0/1700068279 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff520105ab0 0x7ff52019af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:04.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:04.459+0000 7ff524a00700 1 -- 172.21.15.97:0/1700068279 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5201a26b0 con 0x7ff5201048b0 2024-03-27T10:56:04.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:04.459+0000 7ff525400700 1 --2- 172.21.15.97:0/1700068279 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff520106450 0x7ff5201a2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:56:04.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:04.459+0000 7ff524a00700 1 --2- 172.21.15.97:0/1700068279 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5201048b0 0x7ff52019aa50 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7ff510000f80 tx=0x7ff510004ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:04.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:04.460+0000 7ff51e000700 1 -- 172.21.15.97:0/1700068279 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff51001f030 con 0x7ff5201048b0 2024-03-27T10:56:04.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:04.460+0000 7ff51e000700 1 -- 172.21.15.97:0/1700068279 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff510007bb0 con 0x7ff5201048b0 2024-03-27T10:56:04.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:04.460+0000 7ff527400700 1 -- 172.21.15.97:0/1700068279 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5201a2940 con 0x7ff5201048b0 2024-03-27T10:56:04.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:04.460+0000 7ff51e000700 1 -- 172.21.15.97:0/1700068279 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5100047f0 con 0x7ff5201048b0 2024-03-27T10:56:04.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:04.460+0000 7ff527400700 1 -- 172.21.15.97:0/1700068279 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5201a2e70 con 0x7ff5201048b0 2024-03-27T10:56:04.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:04.461+0000 7ff527400700 1 -- 172.21.15.97:0/1700068279 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff520069300 con 0x7ff5201048b0 2024-03-27T10:56:04.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:04.464+0000 7ff51e000700 1 -- 172.21.15.97:0/1700068279 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7ff510004000 con 0x7ff5201048b0 2024-03-27T10:56:04.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:04.466+0000 7ff51e000700 1 --2- 172.21.15.97:0/1700068279 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7ff5040a3720 0x7ff5040a5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:04.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:04.466+0000 7ff51e000700 1 -- 172.21.15.97:0/1700068279 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7ff51000f020 con 0x7ff5201048b0 2024-03-27T10:56:04.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:04.466+0000 7ff517e00700 1 --2- 172.21.15.97:0/1700068279 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7ff5040a3720 0x7ff5040a5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:04.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:04.467+0000 7ff517e00700 1 --2- 172.21.15.97:0/1700068279 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7ff5040a3720 0x7ff5040a5be0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff50c005950 tx=0x7ff50c005eb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:04.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:04.468+0000 7ff51e000700 1 -- 172.21.15.97:0/1700068279 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7ff510029020 con 0x7ff5201048b0 2024-03-27T10:56:04.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:04.706+0000 7ff527400700 1 -- 172.21.15.97:0/1700068279 --> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7ff520062a80 con 0x7ff5040a3720 2024-03-27T10:56:04.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:04.709+0000 7ff51e000700 1 -- 172.21.15.97:0/1700068279 <== mgr.14152 v2:172.21.15.97:6800/3955463538 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7ff520062a80 con 0x7ff5040a3720 2024-03-27T10:56:04.682 INFO:teuthology.orchestra.run.smithi097.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-03-27T10:56:04.682 INFO:teuthology.orchestra.run.smithi097.stdout:smithi097 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT6204013T400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-03-27T10:56:04.682 INFO:teuthology.orchestra.run.smithi097.stdout:smithi114 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400WB400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-03-27T10:56:04.682 INFO:teuthology.orchestra.run.smithi097.stdout:smithi177 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6233007S400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-03-27T10:56:04.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:04.712+0000 7ff527400700 1 -- 172.21.15.97:0/1700068279 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7ff5040a3720 msgr2=0x7ff5040a5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:04.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:04.712+0000 7ff527400700 1 --2- 172.21.15.97:0/1700068279 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7ff5040a3720 0x7ff5040a5be0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff50c005950 tx=0x7ff50c005eb0 comp rx=0 tx=0).stop 2024-03-27T10:56:04.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:04.712+0000 7ff527400700 1 -- 172.21.15.97:0/1700068279 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5201048b0 msgr2=0x7ff52019aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:04.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:04.712+0000 7ff527400700 1 --2- 172.21.15.97:0/1700068279 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5201048b0 0x7ff52019aa50 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7ff510000f80 tx=0x7ff510004ef0 comp rx=0 tx=0).stop 2024-03-27T10:56:04.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:04.712+0000 7ff527400700 1 -- 172.21.15.97:0/1700068279 shutdown_connections 2024-03-27T10:56:04.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:04.712+0000 7ff527400700 1 --2- 172.21.15.97:0/1700068279 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7ff5040a3720 0x7ff5040a5be0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:04.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:04.712+0000 7ff527400700 1 --2- 172.21.15.97:0/1700068279 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff520106450 0x7ff5201a2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:04.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:04.712+0000 7ff527400700 1 --2- 172.21.15.97:0/1700068279 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5201048b0 0x7ff52019aa50 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:04.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:04.712+0000 7ff527400700 1 --2- 172.21.15.97:0/1700068279 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff520105ab0 0x7ff52019af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:04.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:04.712+0000 7ff527400700 1 -- 172.21.15.97:0/1700068279 >> 172.21.15.97:0/1700068279 conn(0x7ff5200ffe50 msgr2=0x7ff520108db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:04.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:04.712+0000 7ff527400700 1 -- 172.21.15.97:0/1700068279 shutdown_connections 2024-03-27T10:56:04.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:04.713+0000 7ff527400700 1 -- 172.21.15.97:0/1700068279 wait complete. 2024-03-27T10:56:05.390 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:56:05 smithi097 ceph-mon[31859]: pgmap v424: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-03-27T10:56:05.426 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:56:05 smithi177 ceph-mon[35294]: pgmap v424: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-03-27T10:56:05.506 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-27T10:56:05.516 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi097.front.sepia.ceph.com 2024-03-27T10:56:05.517 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid adca88c4-ec26-11ee-95d0-87774f69a715 -- bash -c 'ceph fs dump' 2024-03-27T10:56:05.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:56:05 smithi114 ceph-mon[35338]: pgmap v424: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-03-27T10:56:05.966 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T10:56:06.390 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:56:06 smithi097 ceph-mon[31859]: from='client.15183 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T10:56:06.426 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:56:06 smithi177 ceph-mon[35294]: from='client.15183 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T10:56:06.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:56:06 smithi114 ceph-mon[35338]: from='client.15183 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T10:56:07.426 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:56:07 smithi177 ceph-mon[35294]: pgmap v425: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-03-27T10:56:07.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:56:07 smithi114 ceph-mon[35338]: pgmap v425: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-03-27T10:56:07.534 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:56:07 smithi097 ceph-mon[31859]: pgmap v425: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-03-27T10:56:08.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:08.062+0000 7fc43bc00700 1 -- 172.21.15.97:0/4065320128 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc434108350 msgr2=0x7fc43410a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:08.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:08.062+0000 7fc43bc00700 1 --2- 172.21.15.97:0/4065320128 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc434108350 0x7fc43410a7c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fc42c00b0a0 tx=0x7fc42c00b3b0 comp rx=0 tx=0).stop 2024-03-27T10:56:08.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:08.062+0000 7fc43bc00700 1 -- 172.21.15.97:0/4065320128 shutdown_connections 2024-03-27T10:56:08.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:08.062+0000 7fc43bc00700 1 --2- 172.21.15.97:0/4065320128 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc434108350 0x7fc43410a7c0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:08.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:08.062+0000 7fc43bc00700 1 --2- 172.21.15.97:0/4065320128 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc4341030f0 0x7fc4341054e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:08.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:08.062+0000 7fc43bc00700 1 --2- 172.21.15.97:0/4065320128 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc434105a20 0x7fc434107e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:08.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:08.062+0000 7fc43bc00700 1 -- 172.21.15.97:0/4065320128 >> 172.21.15.97:0/4065320128 conn(0x7fc4340fcb00 msgr2=0x7fc4340fef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:08.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:08.062+0000 7fc43bc00700 1 -- 172.21.15.97:0/4065320128 shutdown_connections 2024-03-27T10:56:08.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:08.062+0000 7fc43bc00700 1 -- 172.21.15.97:0/4065320128 wait complete. 2024-03-27T10:56:08.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:08.063+0000 7fc43bc00700 1 Processor -- start 2024-03-27T10:56:08.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:08.063+0000 7fc43bc00700 1 -- start start 2024-03-27T10:56:08.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:08.064+0000 7fc43bc00700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc4341030f0 0x7fc43419a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:08.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:08.064+0000 7fc43bc00700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc434105a20 0x7fc43419af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:08.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:08.064+0000 7fc43bc00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc434108350 0x7fc4341a1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:08.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:08.064+0000 7fc43bc00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc43410e140 con 0x7fc434108350 2024-03-27T10:56:08.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:08.064+0000 7fc43bc00700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc43410dfc0 con 0x7fc434105a20 2024-03-27T10:56:08.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:08.064+0000 7fc43bc00700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc43410e2c0 con 0x7fc4341030f0 2024-03-27T10:56:08.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:08.064+0000 7fc439c00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc434108350 0x7fc4341a1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:08.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:08.064+0000 7fc439c00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc434108350 0x7fc4341a1f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56786/0 (socket says 172.21.15.97:56786) 2024-03-27T10:56:08.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:08.064+0000 7fc439c00700 1 -- 172.21.15.97:0/1786137699 learned_addr learned my addr 172.21.15.97:0/1786137699 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:56:08.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:08.064+0000 7fc433e00700 1 --2- 172.21.15.97:0/1786137699 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc434105a20 0x7fc43419af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:08.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:08.064+0000 7fc439200700 1 --2- 172.21.15.97:0/1786137699 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc4341030f0 0x7fc43419a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:08.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:08.065+0000 7fc439c00700 1 -- 172.21.15.97:0/1786137699 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc4341030f0 msgr2=0x7fc43419a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:08.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:08.065+0000 7fc439c00700 1 --2- 172.21.15.97:0/1786137699 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc4341030f0 0x7fc43419a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:08.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:08.065+0000 7fc439c00700 1 -- 172.21.15.97:0/1786137699 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc434105a20 msgr2=0x7fc43419af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:08.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:08.065+0000 7fc439c00700 1 --2- 172.21.15.97:0/1786137699 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc434105a20 0x7fc43419af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:08.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:08.065+0000 7fc439c00700 1 -- 172.21.15.97:0/1786137699 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4341a2620 con 0x7fc434108350 2024-03-27T10:56:08.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:08.065+0000 7fc439c00700 1 --2- 172.21.15.97:0/1786137699 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc434108350 0x7fc4341a1f80 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fc42c00b070 tx=0x7fc42c00ce50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:08.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:08.065+0000 7fc431600700 1 -- 172.21.15.97:0/1786137699 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc42c021070 con 0x7fc434108350 2024-03-27T10:56:08.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:08.065+0000 7fc431600700 1 -- 172.21.15.97:0/1786137699 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc42c007b80 con 0x7fc434108350 2024-03-27T10:56:08.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:08.065+0000 7fc43bc00700 1 -- 172.21.15.97:0/1786137699 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4341a28b0 con 0x7fc434108350 2024-03-27T10:56:08.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:08.065+0000 7fc431600700 1 -- 172.21.15.97:0/1786137699 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc42c0047b0 con 0x7fc434108350 2024-03-27T10:56:08.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:08.065+0000 7fc43bc00700 1 -- 172.21.15.97:0/1786137699 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4341a2e40 con 0x7fc434108350 2024-03-27T10:56:08.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:08.066+0000 7fc43bc00700 1 -- 172.21.15.97:0/1786137699 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc434069300 con 0x7fc434108350 2024-03-27T10:56:08.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:08.068+0000 7fc431600700 1 -- 172.21.15.97:0/1786137699 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7fc42c003a00 con 0x7fc434108350 2024-03-27T10:56:08.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:08.070+0000 7fc431600700 1 --2- 172.21.15.97:0/1786137699 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fc41c0a3720 0x7fc41c0a5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:08.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:08.070+0000 7fc431600700 1 -- 172.21.15.97:0/1786137699 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fc42c00f020 con 0x7fc434108350 2024-03-27T10:56:08.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:08.070+0000 7fc439200700 1 --2- 172.21.15.97:0/1786137699 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fc41c0a3720 0x7fc41c0a5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:08.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:08.071+0000 7fc439200700 1 --2- 172.21.15.97:0/1786137699 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fc41c0a3720 0x7fc41c0a5be0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc428007c60 tx=0x7fc4280073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:08.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:08.075+0000 7fc431600700 1 -- 172.21.15.97:0/1786137699 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7fc42c02a020 con 0x7fc434108350 2024-03-27T10:56:08.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:08.351+0000 7fc43bc00700 1 -- 172.21.15.97:0/1786137699 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fc434063250 con 0x7fc434108350 2024-03-27T10:56:08.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:08.352+0000 7fc431600700 1 -- 172.21.15.97:0/1786137699 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 27 v27) ==== 76+0+3099 (secure 0 0 0) 0x7fc42c09df00 con 0x7fc434108350 2024-03-27T10:56:08.325 INFO:teuthology.orchestra.run.smithi097.stdout:e27 2024-03-27T10:56:08.325 INFO:teuthology.orchestra.run.smithi097.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-27T10:56:08.325 INFO:teuthology.orchestra.run.smithi097.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-03-27T10:56:08.325 INFO:teuthology.orchestra.run.smithi097.stdout:legacy client fscid: 1 2024-03-27T10:56:08.326 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T10:56:08.326 INFO:teuthology.orchestra.run.smithi097.stdout:Filesystem 'cephfs' (1) 2024-03-27T10:56:08.326 INFO:teuthology.orchestra.run.smithi097.stdout:fs_name cephfs 2024-03-27T10:56:08.326 INFO:teuthology.orchestra.run.smithi097.stdout:epoch 27 2024-03-27T10:56:08.326 INFO:teuthology.orchestra.run.smithi097.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-27T10:56:08.326 INFO:teuthology.orchestra.run.smithi097.stdout:created 2024-03-27T10:54:04.510178+0000 2024-03-27T10:56:08.326 INFO:teuthology.orchestra.run.smithi097.stdout:modified 2024-03-27T10:54:12.180047+0000 2024-03-27T10:56:08.326 INFO:teuthology.orchestra.run.smithi097.stdout:tableserver 0 2024-03-27T10:56:08.326 INFO:teuthology.orchestra.run.smithi097.stdout:root 0 2024-03-27T10:56:08.326 INFO:teuthology.orchestra.run.smithi097.stdout:session_timeout 300 2024-03-27T10:56:08.326 INFO:teuthology.orchestra.run.smithi097.stdout:session_autoclose 300 2024-03-27T10:56:08.326 INFO:teuthology.orchestra.run.smithi097.stdout:max_file_size 1099511627776 2024-03-27T10:56:08.327 INFO:teuthology.orchestra.run.smithi097.stdout:max_xattr_size 65536 2024-03-27T10:56:08.327 INFO:teuthology.orchestra.run.smithi097.stdout:required_client_features {} 2024-03-27T10:56:08.327 INFO:teuthology.orchestra.run.smithi097.stdout:last_failure 0 2024-03-27T10:56:08.327 INFO:teuthology.orchestra.run.smithi097.stdout:last_failure_osd_epoch 0 2024-03-27T10:56:08.327 INFO:teuthology.orchestra.run.smithi097.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-03-27T10:56:08.327 INFO:teuthology.orchestra.run.smithi097.stdout:max_mds 5 2024-03-27T10:56:08.327 INFO:teuthology.orchestra.run.smithi097.stdout:in 0,1,2,3,4 2024-03-27T10:56:08.327 INFO:teuthology.orchestra.run.smithi097.stdout:up {0=34508,1=34478,2=34445,3=24454,4=24400} 2024-03-27T10:56:08.327 INFO:teuthology.orchestra.run.smithi097.stdout:failed 2024-03-27T10:56:08.327 INFO:teuthology.orchestra.run.smithi097.stdout:damaged 2024-03-27T10:56:08.327 INFO:teuthology.orchestra.run.smithi097.stdout:stopped 2024-03-27T10:56:08.327 INFO:teuthology.orchestra.run.smithi097.stdout:data_pools [3] 2024-03-27T10:56:08.327 INFO:teuthology.orchestra.run.smithi097.stdout:metadata_pool 2 2024-03-27T10:56:08.327 INFO:teuthology.orchestra.run.smithi097.stdout:inline_data disabled 2024-03-27T10:56:08.328 INFO:teuthology.orchestra.run.smithi097.stdout:balancer 2024-03-27T10:56:08.328 INFO:teuthology.orchestra.run.smithi097.stdout:bal_rank_mask -1 2024-03-27T10:56:08.328 INFO:teuthology.orchestra.run.smithi097.stdout:standby_count_wanted 0 2024-03-27T10:56:08.328 INFO:teuthology.orchestra.run.smithi097.stdout:qdb_cluster leader: 34508 members: 34445,34508,24400,24454,34478 2024-03-27T10:56:08.328 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.b{0:34508} state up:active seq 3 addr [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] compat {c=[1],r=[1],i=[1fff]}] 2024-03-27T10:56:08.328 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.k{0:34415} state up:standby-replay seq 1 addr [v2:172.21.15.114:6832/4052517902,v1:172.21.15.114:6833/4052517902] compat {c=[1],r=[1],i=[1fff]}] 2024-03-27T10:56:08.328 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.h{1:34478} state up:active seq 8 addr [v2:172.21.15.114:6836/616775588,v1:172.21.15.114:6837/616775588] compat {c=[1],r=[1],i=[1fff]}] 2024-03-27T10:56:08.328 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.i{1:24487} state up:standby-replay seq 1 addr [v2:172.21.15.177:6838/3340454605,v1:172.21.15.177:6839/3340454605] compat {c=[1],r=[1],i=[1fff]}] 2024-03-27T10:56:08.328 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.e{2:34445} state up:active seq 11 addr [v2:172.21.15.114:6834/3581918222,v1:172.21.15.114:6835/3581918222] compat {c=[1],r=[1],i=[1fff]}] 2024-03-27T10:56:08.328 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.c{2:24466} state up:standby-replay seq 1 addr [v2:172.21.15.177:6836/3244432640,v1:172.21.15.177:6837/3244432640] compat {c=[1],r=[1],i=[1fff]}] 2024-03-27T10:56:08.328 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.f{3:24454} state up:active seq 10 addr [v2:172.21.15.177:6834/3349112132,v1:172.21.15.177:6835/3349112132] compat {c=[1],r=[1],i=[1fff]}] 2024-03-27T10:56:08.328 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.a{3:14658} state up:standby-replay seq 1 addr [v2:172.21.15.97:6840/1672464856,v1:172.21.15.97:6841/1672464856] compat {c=[1],r=[1],i=[1fff]}] 2024-03-27T10:56:08.328 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.l{4:24400} state up:active seq 15 addr [v2:172.21.15.177:6832/3787981632,v1:172.21.15.177:6833/3787981632] compat {c=[1],r=[1],i=[1fff]}] 2024-03-27T10:56:08.328 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.g{4:14646} state up:standby-replay seq 1 addr [v2:172.21.15.97:6838/1057596821,v1:172.21.15.97:6839/1057596821] compat {c=[1],r=[1],i=[1fff]}] 2024-03-27T10:56:08.329 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T10:56:08.329 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T10:56:08.329 INFO:teuthology.orchestra.run.smithi097.stdout:Standby daemons: 2024-03-27T10:56:08.329 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T10:56:08.329 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.j{-1:14553} state up:standby seq 1 addr [v2:172.21.15.97:6834/1009423904,v1:172.21.15.97:6835/1009423904] compat {c=[1],r=[1],i=[1fff]}] 2024-03-27T10:56:08.329 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.d{-1:14571} state up:standby seq 1 addr [v2:172.21.15.97:6836/3251173433,v1:172.21.15.97:6837/3251173433] compat {c=[1],r=[1],i=[1fff]}] 2024-03-27T10:56:08.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:08.354+0000 7fc43bc00700 1 -- 172.21.15.97:0/1786137699 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fc41c0a3720 msgr2=0x7fc41c0a5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:08.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:08.354+0000 7fc43bc00700 1 --2- 172.21.15.97:0/1786137699 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fc41c0a3720 0x7fc41c0a5be0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc428007c60 tx=0x7fc4280073d0 comp rx=0 tx=0).stop 2024-03-27T10:56:08.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:08.354+0000 7fc43bc00700 1 -- 172.21.15.97:0/1786137699 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc434108350 msgr2=0x7fc4341a1f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:08.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:08.354+0000 7fc43bc00700 1 --2- 172.21.15.97:0/1786137699 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc434108350 0x7fc4341a1f80 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fc42c00b070 tx=0x7fc42c00ce50 comp rx=0 tx=0).stop 2024-03-27T10:56:08.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:08.355+0000 7fc43bc00700 1 -- 172.21.15.97:0/1786137699 shutdown_connections 2024-03-27T10:56:08.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:08.355+0000 7fc43bc00700 1 --2- 172.21.15.97:0/1786137699 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fc41c0a3720 0x7fc41c0a5be0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:08.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:08.355+0000 7fc43bc00700 1 --2- 172.21.15.97:0/1786137699 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc434108350 0x7fc4341a1f80 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:08.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:08.355+0000 7fc43bc00700 1 --2- 172.21.15.97:0/1786137699 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc4341030f0 0x7fc43419a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:08.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:08.355+0000 7fc43bc00700 1 --2- 172.21.15.97:0/1786137699 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc434105a20 0x7fc43419af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:08.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:08.355+0000 7fc43bc00700 1 -- 172.21.15.97:0/1786137699 >> 172.21.15.97:0/1786137699 conn(0x7fc4340fcb00 msgr2=0x7fc434102870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:08.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:08.355+0000 7fc43bc00700 1 -- 172.21.15.97:0/1786137699 shutdown_connections 2024-03-27T10:56:08.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:08.355+0000 7fc43bc00700 1 -- 172.21.15.97:0/1786137699 wait complete. 2024-03-27T10:56:08.331 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 27 2024-03-27T10:56:09.151 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid adca88c4-ec26-11ee-95d0-87774f69a715 -- bash -c 'ceph osd dump' 2024-03-27T10:56:09.390 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:56:09 smithi097 ceph-mon[31859]: pgmap v426: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-03-27T10:56:09.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:56:09 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/1786137699' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-27T10:56:09.426 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:56:09 smithi177 ceph-mon[35294]: pgmap v426: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-03-27T10:56:09.426 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:56:09 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/1786137699' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-27T10:56:09.516 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:56:09 smithi114 ceph-mon[35338]: pgmap v426: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-03-27T10:56:09.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:56:09 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/1786137699' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-27T10:56:09.588 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T10:56:11.483 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:56:11 smithi097 ceph-mon[31859]: pgmap v427: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-03-27T10:56:11.516 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:56:11 smithi114 ceph-mon[35338]: pgmap v427: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-03-27T10:56:11.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:11.677+0000 7fac88400700 1 -- 172.21.15.97:0/4055901116 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac801048c0 msgr2=0x7fac80104ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:11.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:11.677+0000 7fac88400700 1 --2- 172.21.15.97:0/4055901116 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac801048c0 0x7fac80104ce0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fac740099c0 tx=0x7fac74009cd0 comp rx=0 tx=0).stop 2024-03-27T10:56:11.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:11.677+0000 7fac88400700 1 -- 172.21.15.97:0/4055901116 shutdown_connections 2024-03-27T10:56:11.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:11.677+0000 7fac88400700 1 --2- 172.21.15.97:0/4055901116 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fac80106460 0x7fac8010cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:11.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:11.677+0000 7fac88400700 1 --2- 172.21.15.97:0/4055901116 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac801048c0 0x7fac80104ce0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:11.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:11.677+0000 7fac88400700 1 --2- 172.21.15.97:0/4055901116 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fac80105ac0 0x7fac80105f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:11.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:11.677+0000 7fac88400700 1 -- 172.21.15.97:0/4055901116 >> 172.21.15.97:0/4055901116 conn(0x7fac800ffe60 msgr2=0x7fac801022a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:11.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:11.677+0000 7fac88400700 1 -- 172.21.15.97:0/4055901116 shutdown_connections 2024-03-27T10:56:11.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:11.677+0000 7fac88400700 1 -- 172.21.15.97:0/4055901116 wait complete. 2024-03-27T10:56:11.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:11.678+0000 7fac88400700 1 Processor -- start 2024-03-27T10:56:11.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:11.678+0000 7fac88400700 1 -- start start 2024-03-27T10:56:11.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:11.679+0000 7fac88400700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fac801048c0 0x7fac8019a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:11.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:11.679+0000 7fac88400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac80105ac0 0x7fac8019aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:11.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:11.679+0000 7fac88400700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fac80106460 0x7fac801a1f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:11.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:11.679+0000 7fac88400700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fac80110380 con 0x7fac80105ac0 2024-03-27T10:56:11.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:11.679+0000 7fac88400700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fac80110200 con 0x7fac80106460 2024-03-27T10:56:11.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:11.679+0000 7fac88400700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fac80110500 con 0x7fac801048c0 2024-03-27T10:56:11.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:11.679+0000 7fac85000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac80105ac0 0x7fac8019aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:11.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:11.679+0000 7fac85000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac80105ac0 0x7fac8019aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45222/0 (socket says 172.21.15.97:45222) 2024-03-27T10:56:11.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:11.679+0000 7fac85000700 1 -- 172.21.15.97:0/600348106 learned_addr learned my addr 172.21.15.97:0/600348106 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:56:11.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:11.679+0000 7fac86400700 1 --2- 172.21.15.97:0/600348106 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fac80106460 0x7fac801a1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:11.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:11.679+0000 7fac85a00700 1 --2- 172.21.15.97:0/600348106 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fac801048c0 0x7fac8019a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:11.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:11.680+0000 7fac85000700 1 -- 172.21.15.97:0/600348106 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fac801048c0 msgr2=0x7fac8019a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:11.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:11.680+0000 7fac85000700 1 --2- 172.21.15.97:0/600348106 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fac801048c0 0x7fac8019a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:11.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:11.680+0000 7fac85000700 1 -- 172.21.15.97:0/600348106 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fac80106460 msgr2=0x7fac801a1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:11.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:11.680+0000 7fac85000700 1 --2- 172.21.15.97:0/600348106 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fac80106460 0x7fac801a1f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:11.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:11.680+0000 7fac85000700 1 -- 172.21.15.97:0/600348106 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac801a25f0 con 0x7fac80105ac0 2024-03-27T10:56:11.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:11.680+0000 7fac85000700 1 --2- 172.21.15.97:0/600348106 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac80105ac0 0x7fac8019aed0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fac6c00c950 tx=0x7fac6c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:11.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:11.680+0000 7fac7e000700 1 -- 172.21.15.97:0/600348106 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac6c0078d0 con 0x7fac80105ac0 2024-03-27T10:56:11.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:11.680+0000 7fac7e000700 1 -- 172.21.15.97:0/600348106 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fac6c00f450 con 0x7fac80105ac0 2024-03-27T10:56:11.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:11.680+0000 7fac7e000700 1 -- 172.21.15.97:0/600348106 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac6c018660 con 0x7fac80105ac0 2024-03-27T10:56:11.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:11.680+0000 7fac88400700 1 -- 172.21.15.97:0/600348106 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac801a28e0 con 0x7fac80105ac0 2024-03-27T10:56:11.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:11.680+0000 7fac88400700 1 -- 172.21.15.97:0/600348106 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac801a2e70 con 0x7fac80105ac0 2024-03-27T10:56:11.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:11.682+0000 7fac88400700 1 -- 172.21.15.97:0/600348106 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac80069300 con 0x7fac80105ac0 2024-03-27T10:56:11.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:11.684+0000 7fac7e000700 1 -- 172.21.15.97:0/600348106 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7fac6c00f600 con 0x7fac80105ac0 2024-03-27T10:56:11.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:11.685+0000 7fac7e000700 1 --2- 172.21.15.97:0/600348106 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fac680a3720 0x7fac680a5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:11.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:11.686+0000 7fac7e000700 1 -- 172.21.15.97:0/600348106 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fac6c0cfbd0 con 0x7fac80105ac0 2024-03-27T10:56:11.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:11.686+0000 7fac85a00700 1 --2- 172.21.15.97:0/600348106 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fac680a3720 0x7fac680a5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:11.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:11.686+0000 7fac85a00700 1 --2- 172.21.15.97:0/600348106 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fac680a3720 0x7fac680a5be0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fac74000c00 tx=0x7fac74005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:11.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:11.688+0000 7fac7e000700 1 -- 172.21.15.97:0/600348106 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7fac6c014030 con 0x7fac80105ac0 2024-03-27T10:56:11.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:56:11 smithi177 ceph-mon[35294]: pgmap v427: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-03-27T10:56:11.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:11.914+0000 7fac88400700 1 -- 172.21.15.97:0/600348106 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7fac8002ddb0 con 0x7fac80105ac0 2024-03-27T10:56:11.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:11.916+0000 7fac7e000700 1 -- 172.21.15.97:0/600348106 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v92) ==== 56+0+5275 (secure 0 0 0) 0x7fac6c09c860 con 0x7fac80105ac0 2024-03-27T10:56:11.889 INFO:teuthology.orchestra.run.smithi097.stdout:epoch 92 2024-03-27T10:56:11.889 INFO:teuthology.orchestra.run.smithi097.stdout:fsid adca88c4-ec26-11ee-95d0-87774f69a715 2024-03-27T10:56:11.889 INFO:teuthology.orchestra.run.smithi097.stdout:created 2024-03-27T10:43:24.680694+0000 2024-03-27T10:56:11.889 INFO:teuthology.orchestra.run.smithi097.stdout:modified 2024-03-27T10:54:41.310812+0000 2024-03-27T10:56:11.889 INFO:teuthology.orchestra.run.smithi097.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-03-27T10:56:11.889 INFO:teuthology.orchestra.run.smithi097.stdout:crush_version 26 2024-03-27T10:56:11.890 INFO:teuthology.orchestra.run.smithi097.stdout:full_ratio 0.95 2024-03-27T10:56:11.890 INFO:teuthology.orchestra.run.smithi097.stdout:backfillfull_ratio 0.9 2024-03-27T10:56:11.890 INFO:teuthology.orchestra.run.smithi097.stdout:nearfull_ratio 0.85 2024-03-27T10:56:11.890 INFO:teuthology.orchestra.run.smithi097.stdout:require_min_compat_client luminous 2024-03-27T10:56:11.890 INFO:teuthology.orchestra.run.smithi097.stdout:min_compat_client luminous 2024-03-27T10:56:11.890 INFO:teuthology.orchestra.run.smithi097.stdout:require_osd_release squid 2024-03-27T10:56:11.890 INFO:teuthology.orchestra.run.smithi097.stdout:stretch_mode_enabled false 2024-03-27T10:56:11.890 INFO:teuthology.orchestra.run.smithi097.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-03-27T10:56:11.890 INFO:teuthology.orchestra.run.smithi097.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 89 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-03-27T10:56:11.890 INFO:teuthology.orchestra.run.smithi097.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 90 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-03-27T10:56:11.890 INFO:teuthology.orchestra.run.smithi097.stdout:max_osd 12 2024-03-27T10:56:11.890 INFO:teuthology.orchestra.run.smithi097.stdout:osd.0 up in weight 1 up_from 8 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.97:6802/1746812080,v1:172.21.15.97:6803/1746812080] [v2:172.21.15.97:6804/1746812080,v1:172.21.15.97:6805/1746812080] exists,up b634e5f1-6118-4e2f-95e9-830db390f8d3 2024-03-27T10:56:11.891 INFO:teuthology.orchestra.run.smithi097.stdout:osd.1 up in weight 1 up_from 13 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.114:6800/3858821264,v1:172.21.15.114:6801/3858821264] [v2:172.21.15.114:6802/3858821264,v1:172.21.15.114:6803/3858821264] exists,up 441e2770-3284-4388-b0de-c56eb8a373d0 2024-03-27T10:56:11.891 INFO:teuthology.orchestra.run.smithi097.stdout:osd.2 up in weight 1 up_from 18 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.177:6800/124240658,v1:172.21.15.177:6801/124240658] [v2:172.21.15.177:6802/124240658,v1:172.21.15.177:6803/124240658] exists,up 00108864-1aa6-48d6-8398-e8eb79fda123 2024-03-27T10:56:11.891 INFO:teuthology.orchestra.run.smithi097.stdout:osd.3 up in weight 1 up_from 27 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.97:6810/2478179311,v1:172.21.15.97:6811/2478179311] [v2:172.21.15.97:6812/2478179311,v1:172.21.15.97:6813/2478179311] exists,up 1860b690-e2dc-4977-88d7-d1aacf7022ca 2024-03-27T10:56:11.891 INFO:teuthology.orchestra.run.smithi097.stdout:osd.4 up in weight 1 up_from 35 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.114:6808/75755945,v1:172.21.15.114:6809/75755945] [v2:172.21.15.114:6810/75755945,v1:172.21.15.114:6811/75755945] exists,up 9245a3a4-7d49-4fdb-9d55-112e43177050 2024-03-27T10:56:11.891 INFO:teuthology.orchestra.run.smithi097.stdout:osd.5 up in weight 1 up_from 41 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.177:6808/1918493790,v1:172.21.15.177:6809/1918493790] [v2:172.21.15.177:6810/1918493790,v1:172.21.15.177:6811/1918493790] exists,up 51922f99-e8d3-461b-8615-e9bfdcbf5ed3 2024-03-27T10:56:11.891 INFO:teuthology.orchestra.run.smithi097.stdout:osd.6 up in weight 1 up_from 49 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.97:6818/2892389563,v1:172.21.15.97:6819/2892389563] [v2:172.21.15.97:6820/2892389563,v1:172.21.15.97:6821/2892389563] exists,up 06dffb32-05eb-43b3-852e-915c5cc7f25a 2024-03-27T10:56:11.891 INFO:teuthology.orchestra.run.smithi097.stdout:osd.7 up in weight 1 up_from 54 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.114:6816/2445491153,v1:172.21.15.114:6817/2445491153] [v2:172.21.15.114:6818/2445491153,v1:172.21.15.114:6819/2445491153] exists,up e68e3519-9c1b-43c5-8e06-5385248b19df 2024-03-27T10:56:11.891 INFO:teuthology.orchestra.run.smithi097.stdout:osd.8 up in weight 1 up_from 63 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.177:6816/2051134191,v1:172.21.15.177:6817/2051134191] [v2:172.21.15.177:6818/2051134191,v1:172.21.15.177:6819/2051134191] exists,up 31ac592e-1349-49ee-9af9-8ad85d682978 2024-03-27T10:56:11.891 INFO:teuthology.orchestra.run.smithi097.stdout:osd.9 up in weight 1 up_from 71 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.97:6826/1578881271,v1:172.21.15.97:6827/1578881271] [v2:172.21.15.97:6828/1578881271,v1:172.21.15.97:6829/1578881271] exists,up 0171086a-dd41-4eb4-a389-16f0c46a64db 2024-03-27T10:56:11.891 INFO:teuthology.orchestra.run.smithi097.stdout:osd.10 up in weight 1 up_from 77 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.114:6824/2893111461,v1:172.21.15.114:6825/2893111461] [v2:172.21.15.114:6826/2893111461,v1:172.21.15.114:6827/2893111461] exists,up 64754e31-b124-4601-ad98-2f70b1dd5359 2024-03-27T10:56:11.891 INFO:teuthology.orchestra.run.smithi097.stdout:osd.11 up in weight 1 up_from 85 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.177:6824/2333030391,v1:172.21.15.177:6825/2333030391] [v2:172.21.15.177:6826/2333030391,v1:172.21.15.177:6827/2333030391] exists,up 9ffc752c-adf7-4037-89d3-535787d7475d 2024-03-27T10:56:11.891 INFO:teuthology.orchestra.run.smithi097.stdout:pg_upmap_items 2.a [2,11] 2024-03-27T10:56:11.891 INFO:teuthology.orchestra.run.smithi097.stdout:pg_upmap_items 2.1c [8,11] 2024-03-27T10:56:11.891 INFO:teuthology.orchestra.run.smithi097.stdout:pg_upmap_items 3.2 [3,9] 2024-03-27T10:56:11.892 INFO:teuthology.orchestra.run.smithi097.stdout:pg_upmap_items 3.3 [4,9] 2024-03-27T10:56:11.892 INFO:teuthology.orchestra.run.smithi097.stdout:pg_upmap_items 3.13 [4,9] 2024-03-27T10:56:11.892 INFO:teuthology.orchestra.run.smithi097.stdout:pg_upmap_items 3.14 [4,7] 2024-03-27T10:56:11.892 INFO:teuthology.orchestra.run.smithi097.stdout:pg_upmap_items 3.1a [5,6] 2024-03-27T10:56:11.892 INFO:teuthology.orchestra.run.smithi097.stdout:pg_upmap_items 3.28 [3,6] 2024-03-27T10:56:11.892 INFO:teuthology.orchestra.run.smithi097.stdout:blocklist 172.21.15.97:0/359646 expires 2024-03-28T10:44:39.889550+0000 2024-03-27T10:56:11.892 INFO:teuthology.orchestra.run.smithi097.stdout:blocklist 172.21.15.97:6801/98682155 expires 2024-03-28T10:44:06.019075+0000 2024-03-27T10:56:11.892 INFO:teuthology.orchestra.run.smithi097.stdout:blocklist 172.21.15.97:6801/1685921853 expires 2024-03-28T10:44:39.889550+0000 2024-03-27T10:56:11.892 INFO:teuthology.orchestra.run.smithi097.stdout:blocklist 172.21.15.97:6800/98682155 expires 2024-03-28T10:44:06.019075+0000 2024-03-27T10:56:11.892 INFO:teuthology.orchestra.run.smithi097.stdout:blocklist 172.21.15.97:0/107690680 expires 2024-03-28T10:44:06.019075+0000 2024-03-27T10:56:11.892 INFO:teuthology.orchestra.run.smithi097.stdout:blocklist 172.21.15.97:6800/1685921853 expires 2024-03-28T10:44:39.889550+0000 2024-03-27T10:56:11.892 INFO:teuthology.orchestra.run.smithi097.stdout:blocklist 172.21.15.97:0/437043413 expires 2024-03-28T10:44:39.889550+0000 2024-03-27T10:56:11.892 INFO:teuthology.orchestra.run.smithi097.stdout:blocklist 172.21.15.97:0/2319706992 expires 2024-03-28T10:44:39.889550+0000 2024-03-27T10:56:11.892 INFO:teuthology.orchestra.run.smithi097.stdout:blocklist 172.21.15.97:0/3911532654 expires 2024-03-28T10:44:06.019075+0000 2024-03-27T10:56:11.892 INFO:teuthology.orchestra.run.smithi097.stdout:blocklist 172.21.15.97:0/3347321845 expires 2024-03-28T10:44:06.019075+0000 2024-03-27T10:56:11.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:11.919+0000 7fac88400700 1 -- 172.21.15.97:0/600348106 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fac680a3720 msgr2=0x7fac680a5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:11.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:11.919+0000 7fac88400700 1 --2- 172.21.15.97:0/600348106 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fac680a3720 0x7fac680a5be0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fac74000c00 tx=0x7fac74005fb0 comp rx=0 tx=0).stop 2024-03-27T10:56:11.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:11.919+0000 7fac88400700 1 -- 172.21.15.97:0/600348106 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac80105ac0 msgr2=0x7fac8019aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:11.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:11.919+0000 7fac88400700 1 --2- 172.21.15.97:0/600348106 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac80105ac0 0x7fac8019aed0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fac6c00c950 tx=0x7fac6c00cc60 comp rx=0 tx=0).stop 2024-03-27T10:56:11.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:11.919+0000 7fac88400700 1 -- 172.21.15.97:0/600348106 shutdown_connections 2024-03-27T10:56:11.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:11.919+0000 7fac88400700 1 --2- 172.21.15.97:0/600348106 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fac680a3720 0x7fac680a5be0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:11.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:11.919+0000 7fac88400700 1 --2- 172.21.15.97:0/600348106 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fac80106460 0x7fac801a1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:11.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:11.919+0000 7fac88400700 1 --2- 172.21.15.97:0/600348106 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fac801048c0 0x7fac8019a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:11.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:11.919+0000 7fac88400700 1 --2- 172.21.15.97:0/600348106 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac80105ac0 0x7fac8019aed0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:11.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:11.919+0000 7fac88400700 1 -- 172.21.15.97:0/600348106 >> 172.21.15.97:0/600348106 conn(0x7fac800ffe60 msgr2=0x7fac80108dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:11.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:11.919+0000 7fac88400700 1 -- 172.21.15.97:0/600348106 shutdown_connections 2024-03-27T10:56:11.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:11.919+0000 7fac88400700 1 -- 172.21.15.97:0/600348106 wait complete. 2024-03-27T10:56:12.516 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:56:12 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/600348106' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-27T10:56:12.559 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:56:12 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/600348106' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-27T10:56:12.625 INFO:teuthology.run_tasks:Running task fs.ready... 2024-03-27T10:56:12.636 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-27T10:56:12.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:56:12 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/600348106' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-27T10:56:12.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:12.889+0000 7f859f400640 1 -- 172.21.15.97:0/3276743719 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85980780f0 msgr2=0x7f8598078990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:12.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:12.889+0000 7f859f400640 1 --2- 172.21.15.97:0/3276743719 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85980780f0 0x7f8598078990 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f8590009ca0 tx=0x7f859002c100 comp rx=0 tx=0).stop 2024-03-27T10:56:12.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:12.889+0000 7f859f400640 1 -- 172.21.15.97:0/3276743719 shutdown_connections 2024-03-27T10:56:12.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:12.889+0000 7f859f400640 1 --2- 172.21.15.97:0/3276743719 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85980780f0 0x7f8598078990 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:12.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:12.889+0000 7f859f400640 1 --2- 172.21.15.97:0/3276743719 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8598077750 0x7f8598077bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:12.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:12.889+0000 7f859f400640 1 --2- 172.21.15.97:0/3276743719 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8598076510 0x7f8598076930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:12.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:12.889+0000 7f859f400640 1 -- 172.21.15.97:0/3276743719 >> 172.21.15.97:0/3276743719 conn(0x7f85980ff300 msgr2=0x7f8598101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:12.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:12.889+0000 7f859f400640 1 -- 172.21.15.97:0/3276743719 shutdown_connections 2024-03-27T10:56:12.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:12.889+0000 7f859f400640 1 -- 172.21.15.97:0/3276743719 wait complete. 2024-03-27T10:56:12.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:12.890+0000 7f859f400640 1 Processor -- start 2024-03-27T10:56:12.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:12.890+0000 7f859f400640 1 -- start start 2024-03-27T10:56:12.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:12.890+0000 7f859f400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8598076510 0x7f859819e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:12.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:12.890+0000 7f859f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8598077750 0x7f859819e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:12.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:12.890+0000 7f859f400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f85980780f0 0x7f85981a58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:12.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:12.890+0000 7f859f400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8598113cd0 con 0x7f8598077750 2024-03-27T10:56:12.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:12.890+0000 7f859f400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f8598113b50 con 0x7f8598076510 2024-03-27T10:56:12.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:12.890+0000 7f859f400640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f8598113e50 con 0x7f85980780f0 2024-03-27T10:56:12.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:12.890+0000 7f8597e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8598077750 0x7f859819e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:12.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:12.890+0000 7f8597e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8598077750 0x7f859819e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45236/0 (socket says 172.21.15.97:45236) 2024-03-27T10:56:12.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:12.890+0000 7f8597e00640 1 -- 172.21.15.97:0/2601604688 learned_addr learned my addr 172.21.15.97:0/2601604688 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:56:12.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:12.891+0000 7f859ca00640 1 --2- 172.21.15.97:0/2601604688 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8598076510 0x7f859819e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:12.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:12.891+0000 7f859d400640 1 --2- 172.21.15.97:0/2601604688 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f85980780f0 0x7f85981a58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:12.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:12.891+0000 7f8597e00640 1 -- 172.21.15.97:0/2601604688 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f85980780f0 msgr2=0x7f85981a58d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:12.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:12.891+0000 7f8597e00640 1 --2- 172.21.15.97:0/2601604688 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f85980780f0 0x7f85981a58d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:12.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:12.891+0000 7f8597e00640 1 -- 172.21.15.97:0/2601604688 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8598076510 msgr2=0x7f859819e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:12.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:12.891+0000 7f8597e00640 1 --2- 172.21.15.97:0/2601604688 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8598076510 0x7f859819e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:12.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:12.891+0000 7f8597e00640 1 -- 172.21.15.97:0/2601604688 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85981a5f40 con 0x7f8598077750 2024-03-27T10:56:12.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:12.891+0000 7f859ca00640 1 --2- 172.21.15.97:0/2601604688 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8598076510 0x7f859819e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T10:56:12.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:12.891+0000 7f8597e00640 1 --2- 172.21.15.97:0/2601604688 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8598077750 0x7f859819e860 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f858400d950 tx=0x7f858400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:12.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:12.891+0000 7f8595600640 1 -- 172.21.15.97:0/2601604688 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8584017070 con 0x7f8598077750 2024-03-27T10:56:12.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:12.891+0000 7f8595600640 1 -- 172.21.15.97:0/2601604688 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8584009c30 con 0x7f8598077750 2024-03-27T10:56:12.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:12.891+0000 7f859f400640 1 -- 172.21.15.97:0/2601604688 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85981a6230 con 0x7f8598077750 2024-03-27T10:56:12.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:12.891+0000 7f8595600640 1 -- 172.21.15.97:0/2601604688 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f858400f620 con 0x7f8598077750 2024-03-27T10:56:12.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:12.892+0000 7f859f400640 1 -- 172.21.15.97:0/2601604688 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85981a6780 con 0x7f8598077750 2024-03-27T10:56:12.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:12.893+0000 7f859f400640 1 -- 172.21.15.97:0/2601604688 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85981108a0 con 0x7f8598077750 2024-03-27T10:56:12.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:12.895+0000 7f8595600640 1 -- 172.21.15.97:0/2601604688 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f85840109a0 con 0x7f8598077750 2024-03-27T10:56:12.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:12.897+0000 7f8595600640 1 --2- 172.21.15.97:0/2601604688 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f85800a3610 0x7f85800a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:12.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:12.897+0000 7f8595600640 1 -- 172.21.15.97:0/2601604688 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f85840cfa80 con 0x7f8598077750 2024-03-27T10:56:12.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:12.897+0000 7f859ca00640 1 --2- 172.21.15.97:0/2601604688 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f85800a3610 0x7f85800a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:12.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:12.898+0000 7f859ca00640 1 --2- 172.21.15.97:0/2601604688 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f85800a3610 0x7f85800a5ad0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f858c0058c0 tx=0x7f858c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:12.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:12.901+0000 7f8595600640 1 -- 172.21.15.97:0/2601604688 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f85840d1e20 con 0x7f8598077750 2024-03-27T10:56:13.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.108+0000 7f859f400640 1 -- 172.21.15.97:0/2601604688 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8598062750 con 0x7f8598077750 2024-03-27T10:56:13.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.112+0000 7f8595600640 1 -- 172.21.15.97:0/2601604688 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+22644 (secure 0 0 0) 0x7f8584009de0 con 0x7f8598077750 2024-03-27T10:56:13.085 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T10:56:13.086 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":92,"fsid":"adca88c4-ec26-11ee-95d0-87774f69a715","created":"2024-03-27T10:43:24.680694+0000","modified":"2024-03-27T10:54:41.310812+0000","last_up_change":"2024-03-27T10:53:00.552513+0000","last_in_change":"2024-03-27T10:52:36.302986+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-27T10:47:54.052810+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-27T10:54:02.845060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-27T10:54:03.474151+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b634e5f1-6118-4e2f-95e9-830db390f8d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1746812080},{"type":"v1","addr":"172.21.15.97:6803","nonce":1746812080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1746812080},{"type":"v1","addr":"172.21.15.97:6805","nonce":1746812080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":1746812080},{"type":"v1","addr":"172.21.15.97:6809","nonce":1746812080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1746812080},{"type":"v1","addr":"172.21.15.97:6807","nonce":1746812080}]},"public_addr":"172.21.15.97:6803/1746812080","cluster_addr":"172.21.15.97:6805/1746812080","heartbeat_back_addr":"172.21.15.97:6809/1746812080","heartbeat_front_addr":"172.21.15.97:6807/1746812080","state":["exists","up"]},{"osd":1,"uuid":"441e2770-3284-4388-b0de-c56eb8a373d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":3858821264},{"type":"v1","addr":"172.21.15.114:6801","nonce":3858821264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":3858821264},{"type":"v1","addr":"172.21.15.114:6803","nonce":3858821264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":3858821264},{"type":"v1","addr":"172.21.15.114:6807","nonce":3858821264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":3858821264},{"type":"v1","addr":"172.21.15.114:6805","nonce":3858821264}]},"public_addr":"172.21.15.114:6801/3858821264","cluster_addr":"172.21.15.114:6803/3858821264","heartbeat_back_addr":"172.21.15.114:6807/3858821264","heartbeat_front_addr":"172.21.15.114:6805/3858821264","state":["exists","up"]},{"osd":2,"uuid":"00108864-1aa6-48d6-8398-e8eb79fda123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":124240658},{"type":"v1","addr":"172.21.15.177:6801","nonce":124240658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":124240658},{"type":"v1","addr":"172.21.15.177:6803","nonce":124240658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":124240658},{"type":"v1","addr":"172.21.15.177:6807","nonce":124240658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":124240658},{"type":"v1","addr":"172.21.15.177:6805","nonce":124240658}]},"public_addr":"172.21.15.177:6801/124240658","cluster_addr":"172.21.15.177:6803/124240658","heartbeat_back_addr":"172.21.15.177:6807/124240658","heartbeat_front_addr":"172.21.15.177:6805/124240658","state":["exists","up"]},{"osd":3,"uuid":"1860b690-e2dc-4977-88d7-d1aacf7022ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2478179311},{"type":"v1","addr":"172.21.15.97:6811","nonce":2478179311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2478179311},{"type":"v1","addr":"172.21.15.97:6813","nonce":2478179311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":2478179311},{"type":"v1","addr":"172.21.15.97:6817","nonce":2478179311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2478179311},{"type":"v1","addr":"172.21.15.97:6815","nonce":2478179311}]},"public_addr":"172.21.15.97:6811/2478179311","cluster_addr":"172.21.15.97:6813/2478179311","heartbeat_back_addr":"172.21.15.97:6817/2478179311","heartbeat_front_addr":"172.21.15.97:6815/2478179311","state":["exists","up"]},{"osd":4,"uuid":"9245a3a4-7d49-4fdb-9d55-112e43177050","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":75755945},{"type":"v1","addr":"172.21.15.114:6809","nonce":75755945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":75755945},{"type":"v1","addr":"172.21.15.114:6811","nonce":75755945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":75755945},{"type":"v1","addr":"172.21.15.114:6815","nonce":75755945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":75755945},{"type":"v1","addr":"172.21.15.114:6813","nonce":75755945}]},"public_addr":"172.21.15.114:6809/75755945","cluster_addr":"172.21.15.114:6811/75755945","heartbeat_back_addr":"172.21.15.114:6815/75755945","heartbeat_front_addr":"172.21.15.114:6813/75755945","state":["exists","up"]},{"osd":5,"uuid":"51922f99-e8d3-461b-8615-e9bfdcbf5ed3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":1918493790},{"type":"v1","addr":"172.21.15.177:6809","nonce":1918493790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":1918493790},{"type":"v1","addr":"172.21.15.177:6811","nonce":1918493790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":1918493790},{"type":"v1","addr":"172.21.15.177:6815","nonce":1918493790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":1918493790},{"type":"v1","addr":"172.21.15.177:6813","nonce":1918493790}]},"public_addr":"172.21.15.177:6809/1918493790","cluster_addr":"172.21.15.177:6811/1918493790","heartbeat_back_addr":"172.21.15.177:6815/1918493790","heartbeat_front_addr":"172.21.15.177:6813/1918493790","state":["exists","up"]},{"osd":6,"uuid":"06dffb32-05eb-43b3-852e-915c5cc7f25a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2892389563},{"type":"v1","addr":"172.21.15.97:6819","nonce":2892389563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2892389563},{"type":"v1","addr":"172.21.15.97:6821","nonce":2892389563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2892389563},{"type":"v1","addr":"172.21.15.97:6825","nonce":2892389563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2892389563},{"type":"v1","addr":"172.21.15.97:6823","nonce":2892389563}]},"public_addr":"172.21.15.97:6819/2892389563","cluster_addr":"172.21.15.97:6821/2892389563","heartbeat_back_addr":"172.21.15.97:6825/2892389563","heartbeat_front_addr":"172.21.15.97:6823/2892389563","state":["exists","up"]},{"osd":7,"uuid":"e68e3519-9c1b-43c5-8e06-5385248b19df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":2445491153},{"type":"v1","addr":"172.21.15.114:6817","nonce":2445491153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":2445491153},{"type":"v1","addr":"172.21.15.114:6819","nonce":2445491153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2445491153},{"type":"v1","addr":"172.21.15.114:6823","nonce":2445491153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":2445491153},{"type":"v1","addr":"172.21.15.114:6821","nonce":2445491153}]},"public_addr":"172.21.15.114:6817/2445491153","cluster_addr":"172.21.15.114:6819/2445491153","heartbeat_back_addr":"172.21.15.114:6823/2445491153","heartbeat_front_addr":"172.21.15.114:6821/2445491153","state":["exists","up"]},{"osd":8,"uuid":"31ac592e-1349-49ee-9af9-8ad85d682978","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":2051134191},{"type":"v1","addr":"172.21.15.177:6817","nonce":2051134191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":2051134191},{"type":"v1","addr":"172.21.15.177:6819","nonce":2051134191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":2051134191},{"type":"v1","addr":"172.21.15.177:6823","nonce":2051134191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":2051134191},{"type":"v1","addr":"172.21.15.177:6821","nonce":2051134191}]},"public_addr":"172.21.15.177:6817/2051134191","cluster_addr":"172.21.15.177:6819/2051134191","heartbeat_back_addr":"172.21.15.177:6823/2051134191","heartbeat_front_addr":"172.21.15.177:6821/2051134191","state":["exists","up"]},{"osd":9,"uuid":"0171086a-dd41-4eb4-a389-16f0c46a64db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":1578881271},{"type":"v1","addr":"172.21.15.97:6827","nonce":1578881271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1578881271},{"type":"v1","addr":"172.21.15.97:6829","nonce":1578881271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1578881271},{"type":"v1","addr":"172.21.15.97:6833","nonce":1578881271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1578881271},{"type":"v1","addr":"172.21.15.97:6831","nonce":1578881271}]},"public_addr":"172.21.15.97:6827/1578881271","cluster_addr":"172.21.15.97:6829/1578881271","heartbeat_back_addr":"172.21.15.97:6833/1578881271","heartbeat_front_addr":"172.21.15.97:6831/1578881271","state":["exists","up"]},{"osd":10,"uuid":"64754e31-b124-4601-ad98-2f70b1dd5359","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":2893111461},{"type":"v1","addr":"172.21.15.114:6825","nonce":2893111461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":2893111461},{"type":"v1","addr":"172.21.15.114:6827","nonce":2893111461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2893111461},{"type":"v1","addr":"172.21.15.114:6831","nonce":2893111461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":2893111461},{"type":"v1","addr":"172.21.15.114:6829","nonce":2893111461}]},"public_addr":"172.21.15.114:6825/2893111461","cluster_addr":"172.21.15.114:6827/2893111461","heartbeat_back_addr":"172.21.15.114:6831/2893111461","heartbeat_front_addr":"172.21.15.114:6829/2893111461","state":["exists","up"]},{"osd":11,"uuid":"9ffc752c-adf7-4037-89d3-535787d7475d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":85,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":2333030391},{"type":"v1","addr":"172.21.15.177:6825","nonce":2333030391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":2333030391},{"type":"v1","addr":"172.21.15.177:6827","nonce":2333030391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":2333030391},{"type":"v1","addr":"172.21.15.177:6831","nonce":2333030391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":2333030391},{"type":"v1","addr":"172.21.15.177:6829","nonce":2333030391}]},"public_addr":"172.21.15.177:6825/2333030391","cluster_addr":"172.21.15.177:6827/2333030391","heartbeat_back_addr":"172.21.15.177:6831/2333030391","heartbeat_front_addr":"172.21.15.177:6829/2333030391","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:46:45.143519+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:47:18.656947+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:47:51.316345+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:48:24.575512+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:48:59.113555+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:49:32.103865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:50:06.615539+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:50:41.578341+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:51:15.254011+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:51:49.099165+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:52:24.544821+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:52:58.838484+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.28","mappings":[{"from":3,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/359646":"2024-03-28T10:44:39.889550+0000","172.21.15.97:6801/98682155":"2024-03-28T10:44:06.019075+0000","172.21.15.97:6801/1685921853":"2024-03-28T10:44:39.889550+0000","172.21.15.97:6800/98682155":"2024-03-28T10:44:06.019075+0000","172.21.15.97:0/107690680":"2024-03-28T10:44:06.019075+0000","172.21.15.97:6800/1685921853":"2024-03-28T10:44:39.889550+0000","172.21.15.97:0/437043413":"2024-03-28T10:44:39.889550+0000","172.21.15.97:0/2319706992":"2024-03-28T10:44:39.889550+0000","172.21.15.97:0/3911532654":"2024-03-28T10:44:06.019075+0000","172.21.15.97:0/3347321845":"2024-03-28T10:44:06.019075+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-27T10:56:13.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.116+0000 7f859f400640 1 -- 172.21.15.97:0/2601604688 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f85800a3610 msgr2=0x7f85800a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:13.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.116+0000 7f859f400640 1 --2- 172.21.15.97:0/2601604688 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f85800a3610 0x7f85800a5ad0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f858c0058c0 tx=0x7f858c005850 comp rx=0 tx=0).stop 2024-03-27T10:56:13.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.116+0000 7f859f400640 1 -- 172.21.15.97:0/2601604688 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8598077750 msgr2=0x7f859819e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:13.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.116+0000 7f859f400640 1 --2- 172.21.15.97:0/2601604688 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8598077750 0x7f859819e860 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f858400d950 tx=0x7f858400de10 comp rx=0 tx=0).stop 2024-03-27T10:56:13.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.116+0000 7f859f400640 1 -- 172.21.15.97:0/2601604688 shutdown_connections 2024-03-27T10:56:13.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.116+0000 7f859f400640 1 --2- 172.21.15.97:0/2601604688 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f85980780f0 0x7f85981a58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:13.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.116+0000 7f859f400640 1 --2- 172.21.15.97:0/2601604688 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8598077750 0x7f859819e860 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:13.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.116+0000 7f859f400640 1 --2- 172.21.15.97:0/2601604688 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f85800a3610 0x7f85800a5ad0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:13.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.116+0000 7f859f400640 1 --2- 172.21.15.97:0/2601604688 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8598076510 0x7f859819e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:13.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.116+0000 7f859f400640 1 -- 172.21.15.97:0/2601604688 >> 172.21.15.97:0/2601604688 conn(0x7f85980ff300 msgr2=0x7f8598100a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:13.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.116+0000 7f859f400640 1 -- 172.21.15.97:0/2601604688 shutdown_connections 2024-03-27T10:56:13.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.116+0000 7f859f400640 1 -- 172.21.15.97:0/2601604688 wait complete. 2024-03-27T10:56:13.107 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-27T10:47:54.052810+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-27T10:54:02.845060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-27T10:54:03.474151+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-27T10:56:13.108 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-27T10:56:13.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.332+0000 7efda3600640 1 -- 172.21.15.97:0/2167097789 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd9c076510 msgr2=0x7efd9c076930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:13.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.332+0000 7efda3600640 1 --2- 172.21.15.97:0/2167097789 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd9c076510 0x7efd9c076930 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7efd88009900 tx=0x7efd8802bc30 comp rx=0 tx=0).stop 2024-03-27T10:56:13.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.332+0000 7efda3600640 1 -- 172.21.15.97:0/2167097789 shutdown_connections 2024-03-27T10:56:13.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.332+0000 7efda3600640 1 --2- 172.21.15.97:0/2167097789 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efd9c0780f0 0x7efd9c078990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:13.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.332+0000 7efda3600640 1 --2- 172.21.15.97:0/2167097789 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efd9c077750 0x7efd9c077bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:13.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.332+0000 7efda3600640 1 --2- 172.21.15.97:0/2167097789 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd9c076510 0x7efd9c076930 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:13.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.332+0000 7efda3600640 1 -- 172.21.15.97:0/2167097789 >> 172.21.15.97:0/2167097789 conn(0x7efd9c0ff300 msgr2=0x7efd9c101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:13.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.332+0000 7efda3600640 1 -- 172.21.15.97:0/2167097789 shutdown_connections 2024-03-27T10:56:13.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.332+0000 7efda3600640 1 -- 172.21.15.97:0/2167097789 wait complete. 2024-03-27T10:56:13.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.333+0000 7efda3600640 1 Processor -- start 2024-03-27T10:56:13.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.333+0000 7efda3600640 1 -- start start 2024-03-27T10:56:13.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.333+0000 7efda3600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efd9c076510 0x7efd9c19e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:13.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.333+0000 7efda3600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd9c077750 0x7efd9c19e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:13.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.334+0000 7efda3600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efd9c0780f0 0x7efd9c1a5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:13.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.334+0000 7efda3600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7efd9c113d10 con 0x7efd9c077750 2024-03-27T10:56:13.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.334+0000 7efda3600640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7efd9c113b90 con 0x7efd9c076510 2024-03-27T10:56:13.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.334+0000 7efda3600640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7efd9c113e90 con 0x7efd9c0780f0 2024-03-27T10:56:13.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.334+0000 7efd9be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd9c077750 0x7efd9c19e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:13.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.334+0000 7efd9be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd9c077750 0x7efd9c19e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45248/0 (socket says 172.21.15.97:45248) 2024-03-27T10:56:13.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.334+0000 7efd9be00640 1 -- 172.21.15.97:0/1394250086 learned_addr learned my addr 172.21.15.97:0/1394250086 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:56:13.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.334+0000 7efda0c00640 1 --2- 172.21.15.97:0/1394250086 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efd9c076510 0x7efd9c19e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:13.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.334+0000 7efda1600640 1 --2- 172.21.15.97:0/1394250086 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efd9c0780f0 0x7efd9c1a5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:13.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.334+0000 7efd9be00640 1 -- 172.21.15.97:0/1394250086 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efd9c0780f0 msgr2=0x7efd9c1a5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:13.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.334+0000 7efd9be00640 1 --2- 172.21.15.97:0/1394250086 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efd9c0780f0 0x7efd9c1a5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:13.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.334+0000 7efd9be00640 1 -- 172.21.15.97:0/1394250086 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efd9c076510 msgr2=0x7efd9c19e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:13.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.334+0000 7efd9be00640 1 --2- 172.21.15.97:0/1394250086 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efd9c076510 0x7efd9c19e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:13.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.334+0000 7efd9be00640 1 -- 172.21.15.97:0/1394250086 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd9c1a5fb0 con 0x7efd9c077750 2024-03-27T10:56:13.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.334+0000 7efda1600640 1 --2- 172.21.15.97:0/1394250086 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efd9c0780f0 0x7efd9c1a5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:56:13.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.334+0000 7efda0c00640 1 --2- 172.21.15.97:0/1394250086 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efd9c076510 0x7efd9c19e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T10:56:13.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.334+0000 7efd9be00640 1 --2- 172.21.15.97:0/1394250086 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd9c077750 0x7efd9c19e8d0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7efd9000e9f0 tx=0x7efd9000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:13.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.335+0000 7efd99600640 1 -- 172.21.15.97:0/1394250086 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd9000cd80 con 0x7efd9c077750 2024-03-27T10:56:13.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.335+0000 7efd99600640 1 -- 172.21.15.97:0/1394250086 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efd900108e0 con 0x7efd9c077750 2024-03-27T10:56:13.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.335+0000 7efda3600640 1 -- 172.21.15.97:0/1394250086 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd9c1a62a0 con 0x7efd9c077750 2024-03-27T10:56:13.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.335+0000 7efd99600640 1 -- 172.21.15.97:0/1394250086 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd90018920 con 0x7efd9c077750 2024-03-27T10:56:13.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.335+0000 7efda3600640 1 -- 172.21.15.97:0/1394250086 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd9c1a67f0 con 0x7efd9c077750 2024-03-27T10:56:13.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.336+0000 7efda3600640 1 -- 172.21.15.97:0/1394250086 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd9c1108f0 con 0x7efd9c077750 2024-03-27T10:56:13.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.338+0000 7efd99600640 1 -- 172.21.15.97:0/1394250086 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7efd90010a90 con 0x7efd9c077750 2024-03-27T10:56:13.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.340+0000 7efd99600640 1 --2- 172.21.15.97:0/1394250086 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7efd840a3610 0x7efd840a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:13.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.340+0000 7efd99600640 1 -- 172.21.15.97:0/1394250086 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7efd900d0170 con 0x7efd9c077750 2024-03-27T10:56:13.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.341+0000 7efda0c00640 1 --2- 172.21.15.97:0/1394250086 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7efd840a3610 0x7efd840a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:13.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.341+0000 7efda0c00640 1 --2- 172.21.15.97:0/1394250086 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7efd840a3610 0x7efd840a5ad0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7efd88006010 tx=0x7efd8802c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:13.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.345+0000 7efd99600640 1 -- 172.21.15.97:0/1394250086 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7efd9009cd50 con 0x7efd9c077750 2024-03-27T10:56:13.390 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:56:13 smithi097 ceph-mon[31859]: pgmap v428: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T10:56:13.390 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:56:13 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/2601604688' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T10:56:13.516 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:56:13 smithi114 ceph-mon[35338]: pgmap v428: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T10:56:13.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:56:13 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/2601604688' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T10:56:13.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.545+0000 7efda3600640 1 -- 172.21.15.97:0/1394250086 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7efd9c062780 con 0x7efd9c077750 2024-03-27T10:56:13.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.546+0000 7efd99600640 1 -- 172.21.15.97:0/1394250086 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) ==== 93+0+10 (secure 0 0 0) 0x7efd9009c650 con 0x7efd9c077750 2024-03-27T10:56:13.518 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-03-27T10:56:13.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.548+0000 7efda3600640 1 -- 172.21.15.97:0/1394250086 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7efd840a3610 msgr2=0x7efd840a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:13.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.548+0000 7efda3600640 1 --2- 172.21.15.97:0/1394250086 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7efd840a3610 0x7efd840a5ad0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7efd88006010 tx=0x7efd8802c7e0 comp rx=0 tx=0).stop 2024-03-27T10:56:13.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.548+0000 7efda3600640 1 -- 172.21.15.97:0/1394250086 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd9c077750 msgr2=0x7efd9c19e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:13.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.548+0000 7efda3600640 1 --2- 172.21.15.97:0/1394250086 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd9c077750 0x7efd9c19e8d0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7efd9000e9f0 tx=0x7efd9000eeb0 comp rx=0 tx=0).stop 2024-03-27T10:56:13.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.548+0000 7efda3600640 1 -- 172.21.15.97:0/1394250086 shutdown_connections 2024-03-27T10:56:13.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.548+0000 7efda3600640 1 --2- 172.21.15.97:0/1394250086 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efd9c0780f0 0x7efd9c1a5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:13.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.548+0000 7efda3600640 1 --2- 172.21.15.97:0/1394250086 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd9c077750 0x7efd9c19e8d0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:13.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.548+0000 7efda3600640 1 --2- 172.21.15.97:0/1394250086 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7efd840a3610 0x7efd840a5ad0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:13.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.548+0000 7efda3600640 1 --2- 172.21.15.97:0/1394250086 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efd9c076510 0x7efd9c19e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:13.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.548+0000 7efda3600640 1 -- 172.21.15.97:0/1394250086 >> 172.21.15.97:0/1394250086 conn(0x7efd9c0ff300 msgr2=0x7efd9c100aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:13.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.549+0000 7efda3600640 1 -- 172.21.15.97:0/1394250086 shutdown_connections 2024-03-27T10:56:13.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.549+0000 7efda3600640 1 -- 172.21.15.97:0/1394250086 wait complete. 2024-03-27T10:56:13.539 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-27T10:56:13.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:56:13 smithi177 ceph-mon[35294]: pgmap v428: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T10:56:13.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:56:13 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/2601604688' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T10:56:13.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.781+0000 7f59dca00640 1 -- 172.21.15.97:0/3426533125 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59d8103d70 msgr2=0x7f59d8104190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:13.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.781+0000 7f59dca00640 1 --2- 172.21.15.97:0/3426533125 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59d8103d70 0x7f59d8104190 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f59c8009900 tx=0x7f59c802bc30 comp rx=0 tx=0).stop 2024-03-27T10:56:13.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.782+0000 7f59dca00640 1 -- 172.21.15.97:0/3426533125 shutdown_connections 2024-03-27T10:56:13.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.782+0000 7f59dca00640 1 --2- 172.21.15.97:0/3426533125 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f59d8105920 0x7f59d810c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:13.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.782+0000 7f59dca00640 1 --2- 172.21.15.97:0/3426533125 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f59d8104f60 0x7f59d81053e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:13.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.782+0000 7f59dca00640 1 --2- 172.21.15.97:0/3426533125 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59d8103d70 0x7f59d8104190 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:13.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.782+0000 7f59dca00640 1 -- 172.21.15.97:0/3426533125 >> 172.21.15.97:0/3426533125 conn(0x7f59d80ff300 msgr2=0x7f59d8101720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:13.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.782+0000 7f59dca00640 1 -- 172.21.15.97:0/3426533125 shutdown_connections 2024-03-27T10:56:13.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.782+0000 7f59dca00640 1 -- 172.21.15.97:0/3426533125 wait complete. 2024-03-27T10:56:13.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.783+0000 7f59dca00640 1 Processor -- start 2024-03-27T10:56:13.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.783+0000 7f59dca00640 1 -- start start 2024-03-27T10:56:13.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.783+0000 7f59dca00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f59d8103d70 0x7f59d819a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:13.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.783+0000 7f59dca00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f59d8104f60 0x7f59d819a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:13.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.784+0000 7f59dca00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59d8105920 0x7f59d81a15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:13.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.784+0000 7f59dca00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f59d810f8e0 con 0x7f59d8105920 2024-03-27T10:56:13.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.784+0000 7f59dca00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f59d810f760 con 0x7f59d8104f60 2024-03-27T10:56:13.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.784+0000 7f59dca00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f59d810fa60 con 0x7f59d8103d70 2024-03-27T10:56:13.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.784+0000 7f59d6800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59d8105920 0x7f59d81a15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:13.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.784+0000 7f59d6800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59d8105920 0x7f59d81a15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45278/0 (socket says 172.21.15.97:45278) 2024-03-27T10:56:13.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.784+0000 7f59d6800640 1 -- 172.21.15.97:0/3946847116 learned_addr learned my addr 172.21.15.97:0/3946847116 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:56:13.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.784+0000 7f59d5400640 1 --2- 172.21.15.97:0/3946847116 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f59d8104f60 0x7f59d819a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:13.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.784+0000 7f59d5e00640 1 --2- 172.21.15.97:0/3946847116 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f59d8103d70 0x7f59d819a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:13.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.784+0000 7f59d6800640 1 -- 172.21.15.97:0/3946847116 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f59d8103d70 msgr2=0x7f59d819a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:13.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.784+0000 7f59d6800640 1 --2- 172.21.15.97:0/3946847116 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f59d8103d70 0x7f59d819a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:13.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.784+0000 7f59d6800640 1 -- 172.21.15.97:0/3946847116 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f59d8104f60 msgr2=0x7f59d819a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:13.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.784+0000 7f59d6800640 1 --2- 172.21.15.97:0/3946847116 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f59d8104f60 0x7f59d819a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:13.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.784+0000 7f59d6800640 1 -- 172.21.15.97:0/3946847116 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59d81a1c30 con 0x7f59d8105920 2024-03-27T10:56:13.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.784+0000 7f59d5e00640 1 --2- 172.21.15.97:0/3946847116 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f59d8103d70 0x7f59d819a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T10:56:13.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.785+0000 7f59d6800640 1 --2- 172.21.15.97:0/3946847116 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59d8105920 0x7f59d81a15c0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f59d000eb10 tx=0x7f59d000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:13.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.785+0000 7f59cea00640 1 -- 172.21.15.97:0/3946847116 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d0004b20 con 0x7f59d8105920 2024-03-27T10:56:13.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.785+0000 7f59dca00640 1 -- 172.21.15.97:0/3946847116 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59d81a1f20 con 0x7f59d8105920 2024-03-27T10:56:13.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.785+0000 7f59cea00640 1 -- 172.21.15.97:0/3946847116 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f59d0004cc0 con 0x7f59d8105920 2024-03-27T10:56:13.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.785+0000 7f59cea00640 1 -- 172.21.15.97:0/3946847116 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d0010690 con 0x7f59d8105920 2024-03-27T10:56:13.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.785+0000 7f59dca00640 1 -- 172.21.15.97:0/3946847116 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59d81a2390 con 0x7f59d8105920 2024-03-27T10:56:13.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.787+0000 7f59dca00640 1 -- 172.21.15.97:0/3946847116 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59d810b200 con 0x7f59d8105920 2024-03-27T10:56:13.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.788+0000 7f59cea00640 1 -- 172.21.15.97:0/3946847116 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f59d0005020 con 0x7f59d8105920 2024-03-27T10:56:13.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.790+0000 7f59cea00640 1 --2- 172.21.15.97:0/3946847116 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f59bc0a3610 0x7f59bc0a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:13.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.790+0000 7f59cea00640 1 -- 172.21.15.97:0/3946847116 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f59d00d0390 con 0x7f59d8105920 2024-03-27T10:56:13.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.790+0000 7f59d5e00640 1 --2- 172.21.15.97:0/3946847116 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f59bc0a3610 0x7f59bc0a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:13.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.791+0000 7f59d5e00640 1 --2- 172.21.15.97:0/3946847116 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f59bc0a3610 0x7f59bc0a5ad0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f59c802c240 tx=0x7f59c8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:13.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.794+0000 7f59cea00640 1 -- 172.21.15.97:0/3946847116 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f59d009cf10 con 0x7f59d8105920 2024-03-27T10:56:13.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.994+0000 7f59dca00640 1 -- 172.21.15.97:0/3946847116 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f59d80627f0 con 0x7f59d8105920 2024-03-27T10:56:14.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.995+0000 7f59cea00640 1 -- 172.21.15.97:0/3946847116 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) ==== 104+0+11 (secure 0 0 0) 0x7f59d009c810 con 0x7f59d8105920 2024-03-27T10:56:14.030 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-03-27T10:56:14.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.998+0000 7f59dca00640 1 -- 172.21.15.97:0/3946847116 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f59bc0a3610 msgr2=0x7f59bc0a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:14.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.998+0000 7f59dca00640 1 --2- 172.21.15.97:0/3946847116 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f59bc0a3610 0x7f59bc0a5ad0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f59c802c240 tx=0x7f59c8005fd0 comp rx=0 tx=0).stop 2024-03-27T10:56:14.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.998+0000 7f59dca00640 1 -- 172.21.15.97:0/3946847116 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59d8105920 msgr2=0x7f59d81a15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:14.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.998+0000 7f59dca00640 1 --2- 172.21.15.97:0/3946847116 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59d8105920 0x7f59d81a15c0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f59d000eb10 tx=0x7f59d000c2d0 comp rx=0 tx=0).stop 2024-03-27T10:56:14.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.998+0000 7f59dca00640 1 -- 172.21.15.97:0/3946847116 shutdown_connections 2024-03-27T10:56:14.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.998+0000 7f59dca00640 1 --2- 172.21.15.97:0/3946847116 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59d8105920 0x7f59d81a15c0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:14.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.998+0000 7f59dca00640 1 --2- 172.21.15.97:0/3946847116 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f59bc0a3610 0x7f59bc0a5ad0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:14.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.998+0000 7f59dca00640 1 --2- 172.21.15.97:0/3946847116 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f59d8104f60 0x7f59d819a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:14.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.998+0000 7f59dca00640 1 --2- 172.21.15.97:0/3946847116 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f59d8103d70 0x7f59d819a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:14.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.998+0000 7f59dca00640 1 -- 172.21.15.97:0/3946847116 >> 172.21.15.97:0/3946847116 conn(0x7f59d80ff300 msgr2=0x7f59d81083d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:14.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.998+0000 7f59dca00640 1 -- 172.21.15.97:0/3946847116 shutdown_connections 2024-03-27T10:56:14.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:13.998+0000 7f59dca00640 1 -- 172.21.15.97:0/3946847116 wait complete. 2024-03-27T10:56:14.033 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-27T10:56:14.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.283+0000 7f3d41800640 1 -- 172.21.15.97:0/3678997126 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d3c103d90 msgr2=0x7f3d3c1041b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:14.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.283+0000 7f3d41800640 1 --2- 172.21.15.97:0/3678997126 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d3c103d90 0x7f3d3c1041b0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f3d24009900 tx=0x7f3d2402bc30 comp rx=0 tx=0).stop 2024-03-27T10:56:14.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.283+0000 7f3d41800640 1 -- 172.21.15.97:0/3678997126 shutdown_connections 2024-03-27T10:56:14.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.283+0000 7f3d41800640 1 --2- 172.21.15.97:0/3678997126 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3d3c105940 0x7f3d3c10c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:14.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.283+0000 7f3d41800640 1 --2- 172.21.15.97:0/3678997126 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3d3c104f80 0x7f3d3c105400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:14.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.283+0000 7f3d41800640 1 --2- 172.21.15.97:0/3678997126 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d3c103d90 0x7f3d3c1041b0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:14.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.283+0000 7f3d41800640 1 -- 172.21.15.97:0/3678997126 >> 172.21.15.97:0/3678997126 conn(0x7f3d3c0ff300 msgr2=0x7f3d3c101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:14.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.283+0000 7f3d41800640 1 -- 172.21.15.97:0/3678997126 shutdown_connections 2024-03-27T10:56:14.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.283+0000 7f3d41800640 1 -- 172.21.15.97:0/3678997126 wait complete. 2024-03-27T10:56:14.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.284+0000 7f3d41800640 1 Processor -- start 2024-03-27T10:56:14.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.284+0000 7f3d41800640 1 -- start start 2024-03-27T10:56:14.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.284+0000 7f3d41800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d3c103d90 0x7f3d3c19a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:14.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.284+0000 7f3d41800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3d3c104f80 0x7f3d3c19a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:14.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.284+0000 7f3d41800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3d3c105940 0x7f3d3c1a15f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:14.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.284+0000 7f3d41800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3d3c10f8e0 con 0x7f3d3c103d90 2024-03-27T10:56:14.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.284+0000 7f3d41800640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f3d3c10f760 con 0x7f3d3c104f80 2024-03-27T10:56:14.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.284+0000 7f3d41800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f3d3c10fa60 con 0x7f3d3c105940 2024-03-27T10:56:14.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.285+0000 7f3d3aa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d3c103d90 0x7f3d3c19a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:14.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.285+0000 7f3d3aa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d3c103d90 0x7f3d3c19a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45290/0 (socket says 172.21.15.97:45290) 2024-03-27T10:56:14.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.285+0000 7f3d3aa00640 1 -- 172.21.15.97:0/932378804 learned_addr learned my addr 172.21.15.97:0/932378804 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:56:14.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.285+0000 7f3d3b400640 1 --2- 172.21.15.97:0/932378804 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3d3c105940 0x7f3d3c1a15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:14.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.285+0000 7f3d3a000640 1 --2- 172.21.15.97:0/932378804 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3d3c104f80 0x7f3d3c19a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:14.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.285+0000 7f3d3aa00640 1 -- 172.21.15.97:0/932378804 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3d3c105940 msgr2=0x7f3d3c1a15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:14.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.285+0000 7f3d3aa00640 1 --2- 172.21.15.97:0/932378804 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3d3c105940 0x7f3d3c1a15f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:14.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.285+0000 7f3d3aa00640 1 -- 172.21.15.97:0/932378804 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3d3c104f80 msgr2=0x7f3d3c19a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:14.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.285+0000 7f3d3aa00640 1 --2- 172.21.15.97:0/932378804 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3d3c104f80 0x7f3d3c19a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:14.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.285+0000 7f3d3aa00640 1 -- 172.21.15.97:0/932378804 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d3c1a1c60 con 0x7f3d3c103d90 2024-03-27T10:56:14.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.285+0000 7f3d3a000640 1 --2- 172.21.15.97:0/932378804 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3d3c104f80 0x7f3d3c19a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:56:14.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.285+0000 7f3d3aa00640 1 --2- 172.21.15.97:0/932378804 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d3c103d90 0x7f3d3c19a040 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f3d24005ab0 tx=0x7f3d240042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:14.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.286+0000 7f3d33400640 1 -- 172.21.15.97:0/932378804 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d24043020 con 0x7f3d3c103d90 2024-03-27T10:56:14.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.286+0000 7f3d33400640 1 -- 172.21.15.97:0/932378804 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3d240054c0 con 0x7f3d3c103d90 2024-03-27T10:56:14.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.286+0000 7f3d41800640 1 -- 172.21.15.97:0/932378804 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d3c1a1ef0 con 0x7f3d3c103d90 2024-03-27T10:56:14.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.286+0000 7f3d33400640 1 -- 172.21.15.97:0/932378804 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d24003d10 con 0x7f3d3c103d90 2024-03-27T10:56:14.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.286+0000 7f3d41800640 1 -- 172.21.15.97:0/932378804 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d3c1a2350 con 0x7f3d3c103d90 2024-03-27T10:56:14.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.287+0000 7f3d41800640 1 -- 172.21.15.97:0/932378804 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d3c10b1d0 con 0x7f3d3c103d90 2024-03-27T10:56:14.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.289+0000 7f3d33400640 1 -- 172.21.15.97:0/932378804 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f3d24046020 con 0x7f3d3c103d90 2024-03-27T10:56:14.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.290+0000 7f3d33400640 1 --2- 172.21.15.97:0/932378804 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f3d200a3610 0x7f3d200a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:14.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.290+0000 7f3d33400640 1 -- 172.21.15.97:0/932378804 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f3d240f2190 con 0x7f3d3c103d90 2024-03-27T10:56:14.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.290+0000 7f3d3a000640 1 --2- 172.21.15.97:0/932378804 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f3d200a3610 0x7f3d200a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:14.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.291+0000 7f3d3a000640 1 --2- 172.21.15.97:0/932378804 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f3d200a3610 0x7f3d200a5ad0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c19b550 tx=0x7f3d2c009510 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:14.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.294+0000 7f3d33400640 1 -- 172.21.15.97:0/932378804 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f3d2404d020 con 0x7f3d3c103d90 2024-03-27T10:56:14.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:56:14 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/1394250086' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-27T10:56:14.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:56:14 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/3946847116' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-27T10:56:14.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.496+0000 7f3d41800640 1 -- 172.21.15.97:0/932378804 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f3d3c062820 con 0x7f3d3c103d90 2024-03-27T10:56:14.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.497+0000 7f3d33400640 1 -- 172.21.15.97:0/932378804 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) ==== 100+0+11 (secure 0 0 0) 0x7f3d240bee30 con 0x7f3d3c103d90 2024-03-27T10:56:14.470 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-03-27T10:56:14.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.500+0000 7f3d41800640 1 -- 172.21.15.97:0/932378804 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f3d200a3610 msgr2=0x7f3d200a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:14.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.500+0000 7f3d41800640 1 --2- 172.21.15.97:0/932378804 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f3d200a3610 0x7f3d200a5ad0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c19b550 tx=0x7f3d2c009510 comp rx=0 tx=0).stop 2024-03-27T10:56:14.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.500+0000 7f3d41800640 1 -- 172.21.15.97:0/932378804 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d3c103d90 msgr2=0x7f3d3c19a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:14.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.500+0000 7f3d41800640 1 --2- 172.21.15.97:0/932378804 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d3c103d90 0x7f3d3c19a040 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f3d24005ab0 tx=0x7f3d240042f0 comp rx=0 tx=0).stop 2024-03-27T10:56:14.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.500+0000 7f3d41800640 1 -- 172.21.15.97:0/932378804 shutdown_connections 2024-03-27T10:56:14.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.500+0000 7f3d41800640 1 --2- 172.21.15.97:0/932378804 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3d3c105940 0x7f3d3c1a15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:14.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.500+0000 7f3d41800640 1 --2- 172.21.15.97:0/932378804 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f3d200a3610 0x7f3d200a5ad0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:14.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.500+0000 7f3d41800640 1 --2- 172.21.15.97:0/932378804 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3d3c104f80 0x7f3d3c19a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:14.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.500+0000 7f3d41800640 1 --2- 172.21.15.97:0/932378804 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d3c103d90 0x7f3d3c19a040 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:14.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.500+0000 7f3d41800640 1 -- 172.21.15.97:0/932378804 >> 172.21.15.97:0/932378804 conn(0x7f3d3c0ff300 msgr2=0x7f3d3c1083a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:14.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.501+0000 7f3d41800640 1 -- 172.21.15.97:0/932378804 shutdown_connections 2024-03-27T10:56:14.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.501+0000 7f3d41800640 1 -- 172.21.15.97:0/932378804 wait complete. 2024-03-27T10:56:14.493 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-27T10:56:14.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:56:14 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/1394250086' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-27T10:56:14.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:56:14 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/3946847116' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-27T10:56:14.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:56:14 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/1394250086' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-27T10:56:14.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:56:14 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/3946847116' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-27T10:56:14.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.719+0000 7f5ead200640 1 -- 172.21.15.97:0/2888504151 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ea8103510 msgr2=0x7f5ea8109da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:14.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.719+0000 7f5ead200640 1 --2- 172.21.15.97:0/2888504151 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ea8103510 0x7f5ea8109da0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f5ea0009900 tx=0x7f5ea002bc30 comp rx=0 tx=0).stop 2024-03-27T10:56:14.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.720+0000 7f5ead200640 1 -- 172.21.15.97:0/2888504151 shutdown_connections 2024-03-27T10:56:14.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.720+0000 7f5ead200640 1 --2- 172.21.15.97:0/2888504151 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ea8103510 0x7f5ea8109da0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:14.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.720+0000 7f5ead200640 1 --2- 172.21.15.97:0/2888504151 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5ea8102b50 0x7f5ea8102fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:14.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.720+0000 7f5ead200640 1 --2- 172.21.15.97:0/2888504151 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5ea8101960 0x7f5ea8101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:14.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.720+0000 7f5ead200640 1 -- 172.21.15.97:0/2888504151 >> 172.21.15.97:0/2888504151 conn(0x7f5ea80fceb0 msgr2=0x7f5ea80ff310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:14.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.720+0000 7f5ead200640 1 -- 172.21.15.97:0/2888504151 shutdown_connections 2024-03-27T10:56:14.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.720+0000 7f5ead200640 1 -- 172.21.15.97:0/2888504151 wait complete. 2024-03-27T10:56:14.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.721+0000 7f5ead200640 1 Processor -- start 2024-03-27T10:56:14.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.721+0000 7f5ead200640 1 -- start start 2024-03-27T10:56:14.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.721+0000 7f5ead200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5ea8101960 0x7f5ea8199d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:14.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.721+0000 7f5ead200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5ea8102b50 0x7f5ea819a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:14.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.721+0000 7f5ead200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ea8103510 0x7f5ea81a1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:14.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.721+0000 7f5ead200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5ea810d4e0 con 0x7f5ea8103510 2024-03-27T10:56:14.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.721+0000 7f5ead200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f5ea810d360 con 0x7f5ea8101960 2024-03-27T10:56:14.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.721+0000 7f5ead200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f5ea810d660 con 0x7f5ea8102b50 2024-03-27T10:56:14.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.721+0000 7f5ea7e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ea8103510 0x7f5ea81a1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:14.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.721+0000 7f5ea7e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ea8103510 0x7f5ea81a1270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45312/0 (socket says 172.21.15.97:45312) 2024-03-27T10:56:14.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.721+0000 7f5ea7e00640 1 -- 172.21.15.97:0/2541657673 learned_addr learned my addr 172.21.15.97:0/2541657673 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:56:14.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.722+0000 7f5ea7400640 1 --2- 172.21.15.97:0/2541657673 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5ea8101960 0x7f5ea8199d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:14.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.722+0000 7f5e9ea00640 1 --2- 172.21.15.97:0/2541657673 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5ea8102b50 0x7f5ea819a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:14.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.722+0000 7f5ea7e00640 1 -- 172.21.15.97:0/2541657673 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5ea8102b50 msgr2=0x7f5ea819a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:14.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.722+0000 7f5ea7e00640 1 --2- 172.21.15.97:0/2541657673 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5ea8102b50 0x7f5ea819a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:14.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.722+0000 7f5ea7e00640 1 -- 172.21.15.97:0/2541657673 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5ea8101960 msgr2=0x7f5ea8199d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:14.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.722+0000 7f5ea7e00640 1 --2- 172.21.15.97:0/2541657673 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5ea8101960 0x7f5ea8199d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:14.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.722+0000 7f5ea7e00640 1 -- 172.21.15.97:0/2541657673 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ea81a1910 con 0x7f5ea8103510 2024-03-27T10:56:14.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.722+0000 7f5e9ea00640 1 --2- 172.21.15.97:0/2541657673 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5ea8102b50 0x7f5ea819a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:56:14.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.722+0000 7f5ea7e00640 1 --2- 172.21.15.97:0/2541657673 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ea8103510 0x7f5ea81a1270 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f5ea0036040 tx=0x7f5ea0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:14.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.722+0000 7f5ea4c00640 1 -- 172.21.15.97:0/2541657673 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea0043020 con 0x7f5ea8103510 2024-03-27T10:56:14.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.722+0000 7f5ea4c00640 1 -- 172.21.15.97:0/2541657673 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5ea0004490 con 0x7f5ea8103510 2024-03-27T10:56:14.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.722+0000 7f5ea4c00640 1 -- 172.21.15.97:0/2541657673 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea0003d10 con 0x7f5ea8103510 2024-03-27T10:56:14.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.722+0000 7f5ead200640 1 -- 172.21.15.97:0/2541657673 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ea81a1ba0 con 0x7f5ea8103510 2024-03-27T10:56:14.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.723+0000 7f5ead200640 1 -- 172.21.15.97:0/2541657673 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ea81a20a0 con 0x7f5ea8103510 2024-03-27T10:56:14.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.724+0000 7f5ead200640 1 -- 172.21.15.97:0/2541657673 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ea81a2330 con 0x7f5ea8103510 2024-03-27T10:56:14.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.726+0000 7f5ea4c00640 1 -- 172.21.15.97:0/2541657673 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f5ea0046020 con 0x7f5ea8103510 2024-03-27T10:56:14.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.727+0000 7f5ea4c00640 1 --2- 172.21.15.97:0/2541657673 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f5e8c0a3690 0x7f5e8c0a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:14.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.727+0000 7f5ea4c00640 1 -- 172.21.15.97:0/2541657673 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f5ea00f25b0 con 0x7f5ea8103510 2024-03-27T10:56:14.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.728+0000 7f5ea7400640 1 --2- 172.21.15.97:0/2541657673 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f5e8c0a3690 0x7f5e8c0a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:14.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.728+0000 7f5ea7400640 1 --2- 172.21.15.97:0/2541657673 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f5e8c0a3690 0x7f5e8c0a5b50 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5ea819b1d0 tx=0x7f5e980094a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:14.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.730+0000 7f5ea4c00640 1 -- 172.21.15.97:0/2541657673 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f5ea004d020 con 0x7f5ea8103510 2024-03-27T10:56:14.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.968+0000 7f5ead200640 1 -- 172.21.15.97:0/2541657673 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5ea81a2500 con 0x7f5ea8103510 2024-03-27T10:56:14.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.969+0000 7f5ea4c00640 1 -- 172.21.15.97:0/2541657673 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11944 (secure 0 0 0) 0x7f5ea00bf130 con 0x7f5ea8103510 2024-03-27T10:56:14.941 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T10:56:14.941 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1009423904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1009423904},{"type":"v1","addr":"172.21.15.97:6835","nonce":1009423904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3251173433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3251173433},{"type":"v1","addr":"172.21.15.97:6837","nonce":3251173433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-27T10:54:04.510178+0000","modified":"2024-03-27T10:54:12.180047+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34508,"mds_1":34478,"mds_2":34445,"mds_3":24454,"mds_4":24400},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14646":{"gid":14646,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6839/1057596821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1057596821},{"type":"v1","addr":"172.21.15.97:6839","nonce":1057596821}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14658":{"gid":14658,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6841/1672464856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6840","nonce":1672464856},{"type":"v1","addr":"172.21.15.97:6841","nonce":1672464856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.177:6833/3787981632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3787981632},{"type":"v1","addr":"172.21.15.177:6833","nonce":3787981632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.177:6835/3349112132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":3349112132},{"type":"v1","addr":"172.21.15.177:6835","nonce":3349112132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6837/3244432640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":3244432640},{"type":"v1","addr":"172.21.15.177:6837","nonce":3244432640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6839/3340454605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":3340454605},{"type":"v1","addr":"172.21.15.177:6839","nonce":3340454605}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34415":{"gid":34415,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6833/4052517902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":4052517902},{"type":"v1","addr":"172.21.15.114:6833","nonce":4052517902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34445":{"gid":34445,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.114:6835/3581918222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3581918222},{"type":"v1","addr":"172.21.15.114:6835","nonce":3581918222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34478":{"gid":34478,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.114:6837/616775588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":616775588},{"type":"v1","addr":"172.21.15.114:6837","nonce":616775588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34508":{"gid":34508,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.114:6839/3937193784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":3937193784},{"type":"v1","addr":"172.21.15.114:6839","nonce":3937193784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34508,"qdb_cluster":[34445,34508,24400,24454,34478]},"id":1}]} 2024-03-27T10:56:14.943 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 27 2024-03-27T10:56:14.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.971+0000 7f5ead200640 1 -- 172.21.15.97:0/2541657673 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f5e8c0a3690 msgr2=0x7f5e8c0a5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:14.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.971+0000 7f5ead200640 1 --2- 172.21.15.97:0/2541657673 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f5e8c0a3690 0x7f5e8c0a5b50 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5ea819b1d0 tx=0x7f5e980094a0 comp rx=0 tx=0).stop 2024-03-27T10:56:14.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.971+0000 7f5ead200640 1 -- 172.21.15.97:0/2541657673 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ea8103510 msgr2=0x7f5ea81a1270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:14.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.971+0000 7f5ead200640 1 --2- 172.21.15.97:0/2541657673 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ea8103510 0x7f5ea81a1270 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f5ea0036040 tx=0x7f5ea0004270 comp rx=0 tx=0).stop 2024-03-27T10:56:14.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.971+0000 7f5ead200640 1 -- 172.21.15.97:0/2541657673 shutdown_connections 2024-03-27T10:56:14.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.972+0000 7f5ead200640 1 --2- 172.21.15.97:0/2541657673 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ea8103510 0x7f5ea81a1270 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:14.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.972+0000 7f5ead200640 1 --2- 172.21.15.97:0/2541657673 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5ea8102b50 0x7f5ea819a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:14.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.972+0000 7f5ead200640 1 --2- 172.21.15.97:0/2541657673 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f5e8c0a3690 0x7f5e8c0a5b50 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:14.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.972+0000 7f5ead200640 1 --2- 172.21.15.97:0/2541657673 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5ea8101960 0x7f5ea8199d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:14.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.972+0000 7f5ead200640 1 -- 172.21.15.97:0/2541657673 >> 172.21.15.97:0/2541657673 conn(0x7f5ea80fceb0 msgr2=0x7f5ea8073d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:14.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.972+0000 7f5ead200640 1 -- 172.21.15.97:0/2541657673 shutdown_connections 2024-03-27T10:56:14.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:14.972+0000 7f5ead200640 1 -- 172.21.15.97:0/2541657673 wait complete. 2024-03-27T10:56:14.964 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-27T10:56:15.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.185+0000 7f5b35a00640 1 -- 172.21.15.97:0/2318105308 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b30105940 msgr2=0x7f5b3010c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:15.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.185+0000 7f5b35a00640 1 --2- 172.21.15.97:0/2318105308 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b30105940 0x7f5b3010c1d0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f5b28009ca0 tx=0x7f5b2802c0f0 comp rx=0 tx=0).stop 2024-03-27T10:56:15.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.185+0000 7f5b35a00640 1 -- 172.21.15.97:0/2318105308 shutdown_connections 2024-03-27T10:56:15.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.185+0000 7f5b35a00640 1 --2- 172.21.15.97:0/2318105308 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b30105940 0x7f5b3010c1d0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:15.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.185+0000 7f5b35a00640 1 --2- 172.21.15.97:0/2318105308 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5b30104f80 0x7f5b30105400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:15.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.185+0000 7f5b35a00640 1 --2- 172.21.15.97:0/2318105308 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5b30103d90 0x7f5b301041b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:15.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.185+0000 7f5b35a00640 1 -- 172.21.15.97:0/2318105308 >> 172.21.15.97:0/2318105308 conn(0x7f5b300ff300 msgr2=0x7f5b30101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:15.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.185+0000 7f5b35a00640 1 -- 172.21.15.97:0/2318105308 shutdown_connections 2024-03-27T10:56:15.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.185+0000 7f5b35a00640 1 -- 172.21.15.97:0/2318105308 wait complete. 2024-03-27T10:56:15.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.186+0000 7f5b35a00640 1 Processor -- start 2024-03-27T10:56:15.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.186+0000 7f5b35a00640 1 -- start start 2024-03-27T10:56:15.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.187+0000 7f5b35a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5b30103d90 0x7f5b30199f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:15.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.187+0000 7f5b35a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b30104f80 0x7f5b3019a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:15.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.187+0000 7f5b35a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5b30105940 0x7f5b301a14e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:15.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.187+0000 7f5b35a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5b3010f8c0 con 0x7f5b30104f80 2024-03-27T10:56:15.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.187+0000 7f5b35a00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f5b3010f740 con 0x7f5b30105940 2024-03-27T10:56:15.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.187+0000 7f5b35a00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f5b3010fa40 con 0x7f5b30103d90 2024-03-27T10:56:15.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.187+0000 7f5b27e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b30104f80 0x7f5b3019a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:15.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.187+0000 7f5b27e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b30104f80 0x7f5b3019a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45330/0 (socket says 172.21.15.97:45330) 2024-03-27T10:56:15.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.187+0000 7f5b27e00640 1 -- 172.21.15.97:0/238232066 learned_addr learned my addr 172.21.15.97:0/238232066 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:56:15.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.187+0000 7f5b2ea00640 1 --2- 172.21.15.97:0/238232066 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5b30103d90 0x7f5b30199f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:15.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.187+0000 7f5b2f400640 1 --2- 172.21.15.97:0/238232066 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5b30105940 0x7f5b301a14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:15.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.187+0000 7f5b27e00640 1 -- 172.21.15.97:0/238232066 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5b30103d90 msgr2=0x7f5b30199f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:15.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.187+0000 7f5b27e00640 1 --2- 172.21.15.97:0/238232066 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5b30103d90 0x7f5b30199f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:15.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.187+0000 7f5b27e00640 1 -- 172.21.15.97:0/238232066 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5b30105940 msgr2=0x7f5b301a14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:15.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.187+0000 7f5b27e00640 1 --2- 172.21.15.97:0/238232066 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5b30105940 0x7f5b301a14e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:15.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.187+0000 7f5b27e00640 1 -- 172.21.15.97:0/238232066 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b301a1b50 con 0x7f5b30104f80 2024-03-27T10:56:15.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.187+0000 7f5b2f400640 1 --2- 172.21.15.97:0/238232066 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5b30105940 0x7f5b301a14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:56:15.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.188+0000 7f5b27e00640 1 --2- 172.21.15.97:0/238232066 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b30104f80 0x7f5b3019a470 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f5b1800d770 tx=0x7f5b1800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:15.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.188+0000 7f5b27400640 1 -- 172.21.15.97:0/238232066 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b18017070 con 0x7f5b30104f80 2024-03-27T10:56:15.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.188+0000 7f5b27400640 1 -- 172.21.15.97:0/238232066 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5b180054d0 con 0x7f5b30104f80 2024-03-27T10:56:15.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.188+0000 7f5b35a00640 1 -- 172.21.15.97:0/238232066 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b301a1e40 con 0x7f5b30104f80 2024-03-27T10:56:15.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.188+0000 7f5b27400640 1 -- 172.21.15.97:0/238232066 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b18004cf0 con 0x7f5b30104f80 2024-03-27T10:56:15.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.188+0000 7f5b35a00640 1 -- 172.21.15.97:0/238232066 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b301a22b0 con 0x7f5b30104f80 2024-03-27T10:56:15.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.189+0000 7f5b35a00640 1 -- 172.21.15.97:0/238232066 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b3010b130 con 0x7f5b30104f80 2024-03-27T10:56:15.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.191+0000 7f5b27400640 1 -- 172.21.15.97:0/238232066 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f5b18004e90 con 0x7f5b30104f80 2024-03-27T10:56:15.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.192+0000 7f5b27400640 1 --2- 172.21.15.97:0/238232066 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f5b140a3610 0x7f5b140a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:15.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.192+0000 7f5b27400640 1 -- 172.21.15.97:0/238232066 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f5b180cf790 con 0x7f5b30104f80 2024-03-27T10:56:15.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.193+0000 7f5b2ea00640 1 --2- 172.21.15.97:0/238232066 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f5b140a3610 0x7f5b140a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:15.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.193+0000 7f5b2ea00640 1 --2- 172.21.15.97:0/238232066 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f5b140a3610 0x7f5b140a5ad0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5b20005d00 tx=0x7f5b20005c50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:15.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.197+0000 7f5b27400640 1 -- 172.21.15.97:0/238232066 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f5b1809c370 con 0x7f5b30104f80 2024-03-27T10:56:15.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.400+0000 7f5b35a00640 1 -- 172.21.15.97:0/238232066 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5b300627f0 con 0x7f5b30104f80 2024-03-27T10:56:15.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.402+0000 7f5b27400640 1 -- 172.21.15.97:0/238232066 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+22644 (secure 0 0 0) 0x7f5b1809bc70 con 0x7f5b30104f80 2024-03-27T10:56:15.375 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T10:56:15.375 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":92,"fsid":"adca88c4-ec26-11ee-95d0-87774f69a715","created":"2024-03-27T10:43:24.680694+0000","modified":"2024-03-27T10:54:41.310812+0000","last_up_change":"2024-03-27T10:53:00.552513+0000","last_in_change":"2024-03-27T10:52:36.302986+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-27T10:47:54.052810+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-27T10:54:02.845060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-27T10:54:03.474151+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b634e5f1-6118-4e2f-95e9-830db390f8d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1746812080},{"type":"v1","addr":"172.21.15.97:6803","nonce":1746812080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1746812080},{"type":"v1","addr":"172.21.15.97:6805","nonce":1746812080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":1746812080},{"type":"v1","addr":"172.21.15.97:6809","nonce":1746812080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1746812080},{"type":"v1","addr":"172.21.15.97:6807","nonce":1746812080}]},"public_addr":"172.21.15.97:6803/1746812080","cluster_addr":"172.21.15.97:6805/1746812080","heartbeat_back_addr":"172.21.15.97:6809/1746812080","heartbeat_front_addr":"172.21.15.97:6807/1746812080","state":["exists","up"]},{"osd":1,"uuid":"441e2770-3284-4388-b0de-c56eb8a373d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":3858821264},{"type":"v1","addr":"172.21.15.114:6801","nonce":3858821264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":3858821264},{"type":"v1","addr":"172.21.15.114:6803","nonce":3858821264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":3858821264},{"type":"v1","addr":"172.21.15.114:6807","nonce":3858821264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":3858821264},{"type":"v1","addr":"172.21.15.114:6805","nonce":3858821264}]},"public_addr":"172.21.15.114:6801/3858821264","cluster_addr":"172.21.15.114:6803/3858821264","heartbeat_back_addr":"172.21.15.114:6807/3858821264","heartbeat_front_addr":"172.21.15.114:6805/3858821264","state":["exists","up"]},{"osd":2,"uuid":"00108864-1aa6-48d6-8398-e8eb79fda123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":124240658},{"type":"v1","addr":"172.21.15.177:6801","nonce":124240658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":124240658},{"type":"v1","addr":"172.21.15.177:6803","nonce":124240658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":124240658},{"type":"v1","addr":"172.21.15.177:6807","nonce":124240658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":124240658},{"type":"v1","addr":"172.21.15.177:6805","nonce":124240658}]},"public_addr":"172.21.15.177:6801/124240658","cluster_addr":"172.21.15.177:6803/124240658","heartbeat_back_addr":"172.21.15.177:6807/124240658","heartbeat_front_addr":"172.21.15.177:6805/124240658","state":["exists","up"]},{"osd":3,"uuid":"1860b690-e2dc-4977-88d7-d1aacf7022ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2478179311},{"type":"v1","addr":"172.21.15.97:6811","nonce":2478179311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2478179311},{"type":"v1","addr":"172.21.15.97:6813","nonce":2478179311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":2478179311},{"type":"v1","addr":"172.21.15.97:6817","nonce":2478179311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2478179311},{"type":"v1","addr":"172.21.15.97:6815","nonce":2478179311}]},"public_addr":"172.21.15.97:6811/2478179311","cluster_addr":"172.21.15.97:6813/2478179311","heartbeat_back_addr":"172.21.15.97:6817/2478179311","heartbeat_front_addr":"172.21.15.97:6815/2478179311","state":["exists","up"]},{"osd":4,"uuid":"9245a3a4-7d49-4fdb-9d55-112e43177050","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":75755945},{"type":"v1","addr":"172.21.15.114:6809","nonce":75755945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":75755945},{"type":"v1","addr":"172.21.15.114:6811","nonce":75755945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":75755945},{"type":"v1","addr":"172.21.15.114:6815","nonce":75755945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":75755945},{"type":"v1","addr":"172.21.15.114:6813","nonce":75755945}]},"public_addr":"172.21.15.114:6809/75755945","cluster_addr":"172.21.15.114:6811/75755945","heartbeat_back_addr":"172.21.15.114:6815/75755945","heartbeat_front_addr":"172.21.15.114:6813/75755945","state":["exists","up"]},{"osd":5,"uuid":"51922f99-e8d3-461b-8615-e9bfdcbf5ed3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":1918493790},{"type":"v1","addr":"172.21.15.177:6809","nonce":1918493790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":1918493790},{"type":"v1","addr":"172.21.15.177:6811","nonce":1918493790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":1918493790},{"type":"v1","addr":"172.21.15.177:6815","nonce":1918493790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":1918493790},{"type":"v1","addr":"172.21.15.177:6813","nonce":1918493790}]},"public_addr":"172.21.15.177:6809/1918493790","cluster_addr":"172.21.15.177:6811/1918493790","heartbeat_back_addr":"172.21.15.177:6815/1918493790","heartbeat_front_addr":"172.21.15.177:6813/1918493790","state":["exists","up"]},{"osd":6,"uuid":"06dffb32-05eb-43b3-852e-915c5cc7f25a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2892389563},{"type":"v1","addr":"172.21.15.97:6819","nonce":2892389563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2892389563},{"type":"v1","addr":"172.21.15.97:6821","nonce":2892389563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2892389563},{"type":"v1","addr":"172.21.15.97:6825","nonce":2892389563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2892389563},{"type":"v1","addr":"172.21.15.97:6823","nonce":2892389563}]},"public_addr":"172.21.15.97:6819/2892389563","cluster_addr":"172.21.15.97:6821/2892389563","heartbeat_back_addr":"172.21.15.97:6825/2892389563","heartbeat_front_addr":"172.21.15.97:6823/2892389563","state":["exists","up"]},{"osd":7,"uuid":"e68e3519-9c1b-43c5-8e06-5385248b19df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":2445491153},{"type":"v1","addr":"172.21.15.114:6817","nonce":2445491153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":2445491153},{"type":"v1","addr":"172.21.15.114:6819","nonce":2445491153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2445491153},{"type":"v1","addr":"172.21.15.114:6823","nonce":2445491153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":2445491153},{"type":"v1","addr":"172.21.15.114:6821","nonce":2445491153}]},"public_addr":"172.21.15.114:6817/2445491153","cluster_addr":"172.21.15.114:6819/2445491153","heartbeat_back_addr":"172.21.15.114:6823/2445491153","heartbeat_front_addr":"172.21.15.114:6821/2445491153","state":["exists","up"]},{"osd":8,"uuid":"31ac592e-1349-49ee-9af9-8ad85d682978","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":2051134191},{"type":"v1","addr":"172.21.15.177:6817","nonce":2051134191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":2051134191},{"type":"v1","addr":"172.21.15.177:6819","nonce":2051134191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":2051134191},{"type":"v1","addr":"172.21.15.177:6823","nonce":2051134191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":2051134191},{"type":"v1","addr":"172.21.15.177:6821","nonce":2051134191}]},"public_addr":"172.21.15.177:6817/2051134191","cluster_addr":"172.21.15.177:6819/2051134191","heartbeat_back_addr":"172.21.15.177:6823/2051134191","heartbeat_front_addr":"172.21.15.177:6821/2051134191","state":["exists","up"]},{"osd":9,"uuid":"0171086a-dd41-4eb4-a389-16f0c46a64db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":1578881271},{"type":"v1","addr":"172.21.15.97:6827","nonce":1578881271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1578881271},{"type":"v1","addr":"172.21.15.97:6829","nonce":1578881271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1578881271},{"type":"v1","addr":"172.21.15.97:6833","nonce":1578881271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1578881271},{"type":"v1","addr":"172.21.15.97:6831","nonce":1578881271}]},"public_addr":"172.21.15.97:6827/1578881271","cluster_addr":"172.21.15.97:6829/1578881271","heartbeat_back_addr":"172.21.15.97:6833/1578881271","heartbeat_front_addr":"172.21.15.97:6831/1578881271","state":["exists","up"]},{"osd":10,"uuid":"64754e31-b124-4601-ad98-2f70b1dd5359","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":2893111461},{"type":"v1","addr":"172.21.15.114:6825","nonce":2893111461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":2893111461},{"type":"v1","addr":"172.21.15.114:6827","nonce":2893111461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2893111461},{"type":"v1","addr":"172.21.15.114:6831","nonce":2893111461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":2893111461},{"type":"v1","addr":"172.21.15.114:6829","nonce":2893111461}]},"public_addr":"172.21.15.114:6825/2893111461","cluster_addr":"172.21.15.114:6827/2893111461","heartbeat_back_addr":"172.21.15.114:6831/2893111461","heartbeat_front_addr":"172.21.15.114:6829/2893111461","state":["exists","up"]},{"osd":11,"uuid":"9ffc752c-adf7-4037-89d3-535787d7475d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":85,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":2333030391},{"type":"v1","addr":"172.21.15.177:6825","nonce":2333030391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":2333030391},{"type":"v1","addr":"172.21.15.177:6827","nonce":2333030391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":2333030391},{"type":"v1","addr":"172.21.15.177:6831","nonce":2333030391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":2333030391},{"type":"v1","addr":"172.21.15.177:6829","nonce":2333030391}]},"public_addr":"172.21.15.177:6825/2333030391","cluster_addr":"172.21.15.177:6827/2333030391","heartbeat_back_addr":"172.21.15.177:6831/2333030391","heartbeat_front_addr":"172.21.15.177:6829/2333030391","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:46:45.143519+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:47:18.656947+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:47:51.316345+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:48:24.575512+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:48:59.113555+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:49:32.103865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:50:06.615539+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:50:41.578341+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:51:15.254011+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:51:49.099165+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:52:24.544821+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:52:58.838484+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.28","mappings":[{"from":3,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/359646":"2024-03-28T10:44:39.889550+0000","172.21.15.97:6801/98682155":"2024-03-28T10:44:06.019075+0000","172.21.15.97:6801/1685921853":"2024-03-28T10:44:39.889550+0000","172.21.15.97:6800/98682155":"2024-03-28T10:44:06.019075+0000","172.21.15.97:0/107690680":"2024-03-28T10:44:06.019075+0000","172.21.15.97:6800/1685921853":"2024-03-28T10:44:39.889550+0000","172.21.15.97:0/437043413":"2024-03-28T10:44:39.889550+0000","172.21.15.97:0/2319706992":"2024-03-28T10:44:39.889550+0000","172.21.15.97:0/3911532654":"2024-03-28T10:44:06.019075+0000","172.21.15.97:0/3347321845":"2024-03-28T10:44:06.019075+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-27T10:56:15.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.405+0000 7f5b35a00640 1 -- 172.21.15.97:0/238232066 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f5b140a3610 msgr2=0x7f5b140a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:15.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.405+0000 7f5b35a00640 1 --2- 172.21.15.97:0/238232066 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f5b140a3610 0x7f5b140a5ad0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5b20005d00 tx=0x7f5b20005c50 comp rx=0 tx=0).stop 2024-03-27T10:56:15.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.405+0000 7f5b35a00640 1 -- 172.21.15.97:0/238232066 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b30104f80 msgr2=0x7f5b3019a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:15.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.405+0000 7f5b35a00640 1 --2- 172.21.15.97:0/238232066 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b30104f80 0x7f5b3019a470 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f5b1800d770 tx=0x7f5b1800dc30 comp rx=0 tx=0).stop 2024-03-27T10:56:15.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.405+0000 7f5b35a00640 1 -- 172.21.15.97:0/238232066 shutdown_connections 2024-03-27T10:56:15.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.405+0000 7f5b35a00640 1 --2- 172.21.15.97:0/238232066 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f5b140a3610 0x7f5b140a5ad0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:15.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.405+0000 7f5b35a00640 1 --2- 172.21.15.97:0/238232066 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5b30105940 0x7f5b301a14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:15.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.405+0000 7f5b35a00640 1 --2- 172.21.15.97:0/238232066 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b30104f80 0x7f5b3019a470 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:15.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.405+0000 7f5b35a00640 1 --2- 172.21.15.97:0/238232066 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5b30103d90 0x7f5b30199f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:15.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.405+0000 7f5b35a00640 1 -- 172.21.15.97:0/238232066 >> 172.21.15.97:0/238232066 conn(0x7f5b300ff300 msgr2=0x7f5b30108300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:15.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.405+0000 7f5b35a00640 1 -- 172.21.15.97:0/238232066 shutdown_connections 2024-03-27T10:56:15.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.405+0000 7f5b35a00640 1 -- 172.21.15.97:0/238232066 wait complete. 2024-03-27T10:56:15.390 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:56:15 smithi097 ceph-mon[31859]: pgmap v429: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-27T10:56:15.390 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:56:15 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/932378804' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-27T10:56:15.390 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:56:15 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/2541657673' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T10:56:15.395 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-27T10:47:54.052810+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-27T10:54:02.845060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-27T10:54:03.474151+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-27T10:56:15.395 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-27T10:56:15.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:56:15 smithi114 ceph-mon[35338]: pgmap v429: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-27T10:56:15.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:56:15 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/932378804' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-27T10:56:15.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:56:15 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/2541657673' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T10:56:15.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.639+0000 7f2958c00640 1 -- 172.21.15.97:0/867691726 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2954100420 msgr2=0x7f2954100840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:15.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.639+0000 7f2958c00640 1 --2- 172.21.15.97:0/867691726 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2954100420 0x7f2954100840 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f2944009900 tx=0x7f294402bc30 comp rx=0 tx=0).stop 2024-03-27T10:56:15.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.639+0000 7f2958c00640 1 -- 172.21.15.97:0/867691726 shutdown_connections 2024-03-27T10:56:15.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.639+0000 7f2958c00640 1 --2- 172.21.15.97:0/867691726 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2954101720 0x7f2954101ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:15.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.639+0000 7f2958c00640 1 --2- 172.21.15.97:0/867691726 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2954100d80 0x7f29541011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:15.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.639+0000 7f2958c00640 1 --2- 172.21.15.97:0/867691726 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2954100420 0x7f2954100840 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:15.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.639+0000 7f2958c00640 1 -- 172.21.15.97:0/867691726 >> 172.21.15.97:0/867691726 conn(0x7f29540fbfe0 msgr2=0x7f29540fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:15.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.639+0000 7f2958c00640 1 -- 172.21.15.97:0/867691726 shutdown_connections 2024-03-27T10:56:15.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.639+0000 7f2958c00640 1 -- 172.21.15.97:0/867691726 wait complete. 2024-03-27T10:56:15.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.640+0000 7f2958c00640 1 Processor -- start 2024-03-27T10:56:15.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.640+0000 7f2958c00640 1 -- start start 2024-03-27T10:56:15.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.641+0000 7f2958c00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2954100420 0x7f2954199fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:15.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.641+0000 7f2958c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2954100d80 0x7f295419a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:15.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.641+0000 7f2958c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2954101720 0x7f29541a1570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:15.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.641+0000 7f2958c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f295410f8b0 con 0x7f2954101720 2024-03-27T10:56:15.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.641+0000 7f2958c00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f295410f730 con 0x7f2954100d80 2024-03-27T10:56:15.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.641+0000 7f2958c00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f295410fa30 con 0x7f2954100420 2024-03-27T10:56:15.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.641+0000 7f2952800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2954101720 0x7f29541a1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:15.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.641+0000 7f2952800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2954101720 0x7f29541a1570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45346/0 (socket says 172.21.15.97:45346) 2024-03-27T10:56:15.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.641+0000 7f2952800640 1 -- 172.21.15.97:0/1313658666 learned_addr learned my addr 172.21.15.97:0/1313658666 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:56:15.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.641+0000 7f2951e00640 1 --2- 172.21.15.97:0/1313658666 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2954100420 0x7f2954199fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:15.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.641+0000 7f2951400640 1 --2- 172.21.15.97:0/1313658666 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2954100d80 0x7f295419a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:15.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.641+0000 7f2952800640 1 -- 172.21.15.97:0/1313658666 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2954100420 msgr2=0x7f2954199fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:15.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.641+0000 7f2952800640 1 --2- 172.21.15.97:0/1313658666 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2954100420 0x7f2954199fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:15.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.641+0000 7f2952800640 1 -- 172.21.15.97:0/1313658666 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2954100d80 msgr2=0x7f295419a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:15.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.641+0000 7f2952800640 1 --2- 172.21.15.97:0/1313658666 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2954100d80 0x7f295419a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:15.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.641+0000 7f2952800640 1 -- 172.21.15.97:0/1313658666 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29541a1be0 con 0x7f2954101720 2024-03-27T10:56:15.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.642+0000 7f2952800640 1 --2- 172.21.15.97:0/1313658666 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2954101720 0x7f29541a1570 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f294c00d750 tx=0x7f294c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:15.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.642+0000 7f294aa00640 1 -- 172.21.15.97:0/1313658666 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f294c017070 con 0x7f2954101720 2024-03-27T10:56:15.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.642+0000 7f294aa00640 1 -- 172.21.15.97:0/1313658666 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f294c0054d0 con 0x7f2954101720 2024-03-27T10:56:15.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.642+0000 7f2958c00640 1 -- 172.21.15.97:0/1313658666 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29541a1ed0 con 0x7f2954101720 2024-03-27T10:56:15.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.642+0000 7f294aa00640 1 -- 172.21.15.97:0/1313658666 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f294c004c60 con 0x7f2954101720 2024-03-27T10:56:15.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.642+0000 7f2958c00640 1 -- 172.21.15.97:0/1313658666 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29541a23e0 con 0x7f2954101720 2024-03-27T10:56:15.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.644+0000 7f2958c00640 1 -- 172.21.15.97:0/1313658666 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f295410b350 con 0x7f2954101720 2024-03-27T10:56:15.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.645+0000 7f294aa00640 1 -- 172.21.15.97:0/1313658666 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f294c004e00 con 0x7f2954101720 2024-03-27T10:56:15.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.647+0000 7f294aa00640 1 --2- 172.21.15.97:0/1313658666 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f29340a3690 0x7f29340a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:15.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.647+0000 7f294aa00640 1 -- 172.21.15.97:0/1313658666 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f294c0cf6c0 con 0x7f2954101720 2024-03-27T10:56:15.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.647+0000 7f2951e00640 1 --2- 172.21.15.97:0/1313658666 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f29340a3690 0x7f29340a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:15.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.648+0000 7f2951e00640 1 --2- 172.21.15.97:0/1313658666 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f29340a3690 0x7f29340a5b50 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2944006010 tx=0x7f2944000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:15.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.651+0000 7f294aa00640 1 -- 172.21.15.97:0/1313658666 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f294c09c2a0 con 0x7f2954101720 2024-03-27T10:56:15.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:56:15 smithi177 ceph-mon[35294]: pgmap v429: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-27T10:56:15.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:56:15 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/932378804' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-27T10:56:15.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:56:15 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/2541657673' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T10:56:15.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.853+0000 7f2958c00640 1 -- 172.21.15.97:0/1313658666 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f29540627f0 con 0x7f2954101720 2024-03-27T10:56:15.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.854+0000 7f294aa00640 1 -- 172.21.15.97:0/1313658666 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) ==== 93+0+10 (secure 0 0 0) 0x7f294c09bba0 con 0x7f2954101720 2024-03-27T10:56:15.827 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-03-27T10:56:15.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.857+0000 7f2958c00640 1 -- 172.21.15.97:0/1313658666 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f29340a3690 msgr2=0x7f29340a5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:15.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.857+0000 7f2958c00640 1 --2- 172.21.15.97:0/1313658666 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f29340a3690 0x7f29340a5b50 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2944006010 tx=0x7f2944000bc0 comp rx=0 tx=0).stop 2024-03-27T10:56:15.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.857+0000 7f2958c00640 1 -- 172.21.15.97:0/1313658666 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2954101720 msgr2=0x7f29541a1570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:15.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.857+0000 7f2958c00640 1 --2- 172.21.15.97:0/1313658666 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2954101720 0x7f29541a1570 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f294c00d750 tx=0x7f294c00dc10 comp rx=0 tx=0).stop 2024-03-27T10:56:15.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.857+0000 7f2958c00640 1 -- 172.21.15.97:0/1313658666 shutdown_connections 2024-03-27T10:56:15.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.857+0000 7f2958c00640 1 --2- 172.21.15.97:0/1313658666 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2954101720 0x7f29541a1570 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:15.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.857+0000 7f2958c00640 1 --2- 172.21.15.97:0/1313658666 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f29340a3690 0x7f29340a5b50 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:15.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.857+0000 7f2958c00640 1 --2- 172.21.15.97:0/1313658666 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2954100d80 0x7f295419a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:15.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.857+0000 7f2958c00640 1 --2- 172.21.15.97:0/1313658666 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2954100420 0x7f2954199fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:15.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.857+0000 7f2958c00640 1 -- 172.21.15.97:0/1313658666 >> 172.21.15.97:0/1313658666 conn(0x7f29540fbfe0 msgr2=0x7f29540fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:15.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.857+0000 7f2958c00640 1 -- 172.21.15.97:0/1313658666 shutdown_connections 2024-03-27T10:56:15.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:15.857+0000 7f2958c00640 1 -- 172.21.15.97:0/1313658666 wait complete. 2024-03-27T10:56:15.849 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-27T10:56:16.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.093+0000 7f4b0ca00640 1 -- 172.21.15.97:0/1715866740 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b08076510 msgr2=0x7f4b08076930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:16.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.093+0000 7f4b0ca00640 1 --2- 172.21.15.97:0/1715866740 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b08076510 0x7f4b08076930 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f4af8009900 tx=0x7f4af802bc30 comp rx=0 tx=0).stop 2024-03-27T10:56:16.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.093+0000 7f4b0ca00640 1 -- 172.21.15.97:0/1715866740 shutdown_connections 2024-03-27T10:56:16.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.093+0000 7f4b0ca00640 1 --2- 172.21.15.97:0/1715866740 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4b080780f0 0x7f4b08078990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:16.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.093+0000 7f4b0ca00640 1 --2- 172.21.15.97:0/1715866740 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4b08077750 0x7f4b08077bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:16.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.093+0000 7f4b0ca00640 1 --2- 172.21.15.97:0/1715866740 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b08076510 0x7f4b08076930 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:16.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.093+0000 7f4b0ca00640 1 -- 172.21.15.97:0/1715866740 >> 172.21.15.97:0/1715866740 conn(0x7f4b080ff300 msgr2=0x7f4b08101720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:16.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.093+0000 7f4b0ca00640 1 -- 172.21.15.97:0/1715866740 shutdown_connections 2024-03-27T10:56:16.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.093+0000 7f4b0ca00640 1 -- 172.21.15.97:0/1715866740 wait complete. 2024-03-27T10:56:16.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.094+0000 7f4b0ca00640 1 Processor -- start 2024-03-27T10:56:16.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.094+0000 7f4b0ca00640 1 -- start start 2024-03-27T10:56:16.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.095+0000 7f4b0ca00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4b08076510 0x7f4b0819e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:16.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.095+0000 7f4b0ca00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4b08077750 0x7f4b0819e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:16.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.095+0000 7f4b0ca00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b080780f0 0x7f4b081a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:16.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.095+0000 7f4b0ca00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4b08113cf0 con 0x7f4b080780f0 2024-03-27T10:56:16.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.095+0000 7f4b0ca00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f4b08113b70 con 0x7f4b08077750 2024-03-27T10:56:16.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.095+0000 7f4b0ca00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f4b08113e70 con 0x7f4b08076510 2024-03-27T10:56:16.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.095+0000 7f4b06800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b080780f0 0x7f4b081a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:16.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.095+0000 7f4b06800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b080780f0 0x7f4b081a5990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45368/0 (socket says 172.21.15.97:45368) 2024-03-27T10:56:16.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.095+0000 7f4b06800640 1 -- 172.21.15.97:0/1309498523 learned_addr learned my addr 172.21.15.97:0/1309498523 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:56:16.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.095+0000 7f4b05400640 1 --2- 172.21.15.97:0/1309498523 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4b08077750 0x7f4b0819e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:16.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.095+0000 7f4b05e00640 1 --2- 172.21.15.97:0/1309498523 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4b08076510 0x7f4b0819e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:16.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.096+0000 7f4b06800640 1 -- 172.21.15.97:0/1309498523 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4b08076510 msgr2=0x7f4b0819e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:16.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.096+0000 7f4b06800640 1 --2- 172.21.15.97:0/1309498523 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4b08076510 0x7f4b0819e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:16.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.096+0000 7f4b06800640 1 -- 172.21.15.97:0/1309498523 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4b08077750 msgr2=0x7f4b0819e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:16.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.096+0000 7f4b06800640 1 --2- 172.21.15.97:0/1309498523 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4b08077750 0x7f4b0819e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:16.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.096+0000 7f4b06800640 1 -- 172.21.15.97:0/1309498523 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b081a6000 con 0x7f4b080780f0 2024-03-27T10:56:16.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.096+0000 7f4b06800640 1 --2- 172.21.15.97:0/1309498523 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b080780f0 0x7f4b081a5990 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f4b0000eaa0 tx=0x7f4b0000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:16.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.096+0000 7f4afea00640 1 -- 172.21.15.97:0/1309498523 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b0000cdb0 con 0x7f4b080780f0 2024-03-27T10:56:16.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.096+0000 7f4afea00640 1 -- 172.21.15.97:0/1309498523 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4b00005020 con 0x7f4b080780f0 2024-03-27T10:56:16.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.096+0000 7f4b0ca00640 1 -- 172.21.15.97:0/1309498523 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b081a62f0 con 0x7f4b080780f0 2024-03-27T10:56:16.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.096+0000 7f4afea00640 1 -- 172.21.15.97:0/1309498523 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b000116d0 con 0x7f4b080780f0 2024-03-27T10:56:16.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.096+0000 7f4b0ca00640 1 -- 172.21.15.97:0/1309498523 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b081a6890 con 0x7f4b080780f0 2024-03-27T10:56:16.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.097+0000 7f4b0ca00640 1 -- 172.21.15.97:0/1309498523 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b081106e0 con 0x7f4b080780f0 2024-03-27T10:56:16.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.099+0000 7f4afea00640 1 -- 172.21.15.97:0/1309498523 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f4b000051d0 con 0x7f4b080780f0 2024-03-27T10:56:16.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.101+0000 7f4afea00640 1 --2- 172.21.15.97:0/1309498523 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f4aec0a3690 0x7f4aec0a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:16.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.101+0000 7f4afea00640 1 -- 172.21.15.97:0/1309498523 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f4b000cfeb0 con 0x7f4b080780f0 2024-03-27T10:56:16.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.101+0000 7f4b05e00640 1 --2- 172.21.15.97:0/1309498523 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f4aec0a3690 0x7f4aec0a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:16.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.102+0000 7f4b05e00640 1 --2- 172.21.15.97:0/1309498523 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f4aec0a3690 0x7f4aec0a5b50 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f4af8006010 tx=0x7f4af8000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:16.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.105+0000 7f4afea00640 1 -- 172.21.15.97:0/1309498523 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f4b00016030 con 0x7f4b080780f0 2024-03-27T10:56:16.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.308+0000 7f4b0ca00640 1 -- 172.21.15.97:0/1309498523 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f4b08062750 con 0x7f4b080780f0 2024-03-27T10:56:16.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.308+0000 7f4afea00640 1 -- 172.21.15.97:0/1309498523 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) ==== 104+0+11 (secure 0 0 0) 0x7f4b0009ca30 con 0x7f4b080780f0 2024-03-27T10:56:16.281 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-03-27T10:56:16.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.311+0000 7f4b0ca00640 1 -- 172.21.15.97:0/1309498523 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f4aec0a3690 msgr2=0x7f4aec0a5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:16.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.311+0000 7f4b0ca00640 1 --2- 172.21.15.97:0/1309498523 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f4aec0a3690 0x7f4aec0a5b50 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f4af8006010 tx=0x7f4af8000bc0 comp rx=0 tx=0).stop 2024-03-27T10:56:16.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.311+0000 7f4b0ca00640 1 -- 172.21.15.97:0/1309498523 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b080780f0 msgr2=0x7f4b081a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:16.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.311+0000 7f4b0ca00640 1 --2- 172.21.15.97:0/1309498523 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b080780f0 0x7f4b081a5990 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f4b0000eaa0 tx=0x7f4b0000ef60 comp rx=0 tx=0).stop 2024-03-27T10:56:16.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.311+0000 7f4b0ca00640 1 -- 172.21.15.97:0/1309498523 shutdown_connections 2024-03-27T10:56:16.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.311+0000 7f4b0ca00640 1 --2- 172.21.15.97:0/1309498523 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b080780f0 0x7f4b081a5990 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:16.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.311+0000 7f4b0ca00640 1 --2- 172.21.15.97:0/1309498523 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f4aec0a3690 0x7f4aec0a5b50 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:16.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.311+0000 7f4b0ca00640 1 --2- 172.21.15.97:0/1309498523 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4b08077750 0x7f4b0819e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:16.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.311+0000 7f4b0ca00640 1 --2- 172.21.15.97:0/1309498523 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4b08076510 0x7f4b0819e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:16.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.311+0000 7f4b0ca00640 1 -- 172.21.15.97:0/1309498523 >> 172.21.15.97:0/1309498523 conn(0x7f4b080ff300 msgr2=0x7f4b08100e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:16.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.312+0000 7f4b0ca00640 1 -- 172.21.15.97:0/1309498523 shutdown_connections 2024-03-27T10:56:16.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.312+0000 7f4b0ca00640 1 -- 172.21.15.97:0/1309498523 wait complete. 2024-03-27T10:56:16.303 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-27T10:56:16.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.542+0000 7f8d26c00640 1 -- 172.21.15.97:0/2530633468 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d20101a50 msgr2=0x7f8d20101e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:16.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.542+0000 7f8d26c00640 1 --2- 172.21.15.97:0/2530633468 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d20101a50 0x7f8d20101e70 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f8d14009900 tx=0x7f8d1402bc30 comp rx=0 tx=0).stop 2024-03-27T10:56:16.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.543+0000 7f8d26c00640 1 -- 172.21.15.97:0/2530633468 shutdown_connections 2024-03-27T10:56:16.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.543+0000 7f8d26c00640 1 --2- 172.21.15.97:0/2530633468 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8d20103600 0x7f8d20109e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:16.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.543+0000 7f8d26c00640 1 --2- 172.21.15.97:0/2530633468 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8d20102c40 0x7f8d201030c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:16.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.543+0000 7f8d26c00640 1 --2- 172.21.15.97:0/2530633468 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d20101a50 0x7f8d20101e70 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:16.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.543+0000 7f8d26c00640 1 -- 172.21.15.97:0/2530633468 >> 172.21.15.97:0/2530633468 conn(0x7f8d200fcfc0 msgr2=0x7f8d200ff400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:16.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.543+0000 7f8d26c00640 1 -- 172.21.15.97:0/2530633468 shutdown_connections 2024-03-27T10:56:16.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.543+0000 7f8d26c00640 1 -- 172.21.15.97:0/2530633468 wait complete. 2024-03-27T10:56:16.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:56:16 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/238232066' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T10:56:16.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:56:16 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/1313658666' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-27T10:56:16.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.544+0000 7f8d26c00640 1 Processor -- start 2024-03-27T10:56:16.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.544+0000 7f8d26c00640 1 -- start start 2024-03-27T10:56:16.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.544+0000 7f8d26c00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8d20101a50 0x7f8d20075900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:16.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.544+0000 7f8d26c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8d20102c40 0x7f8d20073f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:16.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.544+0000 7f8d26c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d20103600 0x7f8d20074480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:16.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.544+0000 7f8d26c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8d2010d6a0 con 0x7f8d20103600 2024-03-27T10:56:16.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.544+0000 7f8d26c00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f8d2010d520 con 0x7f8d20102c40 2024-03-27T10:56:16.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.544+0000 7f8d26c00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f8d2010d820 con 0x7f8d20101a50 2024-03-27T10:56:16.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.544+0000 7f8d26200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d20103600 0x7f8d20074480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:16.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.544+0000 7f8d26200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d20103600 0x7f8d20074480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45384/0 (socket says 172.21.15.97:45384) 2024-03-27T10:56:16.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.544+0000 7f8d26200640 1 -- 172.21.15.97:0/4033855709 learned_addr learned my addr 172.21.15.97:0/4033855709 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:56:16.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.545+0000 7f8d25800640 1 --2- 172.21.15.97:0/4033855709 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8d20101a50 0x7f8d20075900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:16.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.545+0000 7f8d24e00640 1 --2- 172.21.15.97:0/4033855709 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8d20102c40 0x7f8d20073f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:16.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.545+0000 7f8d26200640 1 -- 172.21.15.97:0/4033855709 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8d20101a50 msgr2=0x7f8d20075900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:16.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.545+0000 7f8d26200640 1 --2- 172.21.15.97:0/4033855709 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8d20101a50 0x7f8d20075900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:16.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.545+0000 7f8d26200640 1 -- 172.21.15.97:0/4033855709 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8d20102c40 msgr2=0x7f8d20073f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:16.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.545+0000 7f8d26200640 1 --2- 172.21.15.97:0/4033855709 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8d20102c40 0x7f8d20073f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:16.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.545+0000 7f8d26200640 1 -- 172.21.15.97:0/4033855709 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d20074d10 con 0x7f8d20103600 2024-03-27T10:56:16.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.545+0000 7f8d25800640 1 --2- 172.21.15.97:0/4033855709 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8d20101a50 0x7f8d20075900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:56:16.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.545+0000 7f8d24e00640 1 --2- 172.21.15.97:0/4033855709 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8d20102c40 0x7f8d20073f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:56:16.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.545+0000 7f8d26200640 1 --2- 172.21.15.97:0/4033855709 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d20103600 0x7f8d20074480 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f8d1800b970 tx=0x7f8d1800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:16.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.545+0000 7f8d1e000640 1 -- 172.21.15.97:0/4033855709 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d18019070 con 0x7f8d20103600 2024-03-27T10:56:16.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.545+0000 7f8d1e000640 1 -- 172.21.15.97:0/4033855709 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8d18014440 con 0x7f8d20103600 2024-03-27T10:56:16.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.546+0000 7f8d26c00640 1 -- 172.21.15.97:0/4033855709 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d201a1f00 con 0x7f8d20103600 2024-03-27T10:56:16.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.546+0000 7f8d1e000640 1 -- 172.21.15.97:0/4033855709 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d180075d0 con 0x7f8d20103600 2024-03-27T10:56:16.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.546+0000 7f8d26c00640 1 -- 172.21.15.97:0/4033855709 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d201a2390 con 0x7f8d20103600 2024-03-27T10:56:16.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.547+0000 7f8d26c00640 1 -- 172.21.15.97:0/4033855709 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d20108e90 con 0x7f8d20103600 2024-03-27T10:56:16.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.549+0000 7f8d1e000640 1 -- 172.21.15.97:0/4033855709 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f8d1801c070 con 0x7f8d20103600 2024-03-27T10:56:16.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.550+0000 7f8d1e000640 1 --2- 172.21.15.97:0/4033855709 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f8d080a3610 0x7f8d080a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:16.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.550+0000 7f8d1e000640 1 -- 172.21.15.97:0/4033855709 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f8d180cf840 con 0x7f8d20103600 2024-03-27T10:56:16.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.551+0000 7f8d25800640 1 --2- 172.21.15.97:0/4033855709 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f8d080a3610 0x7f8d080a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:16.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.551+0000 7f8d25800640 1 --2- 172.21.15.97:0/4033855709 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f8d080a3610 0x7f8d080a5ad0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f8d14036040 tx=0x7f8d14005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:16.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.553+0000 7f8d1e000640 1 -- 172.21.15.97:0/4033855709 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f8d1809c3c0 con 0x7f8d20103600 2024-03-27T10:56:16.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:56:16 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/238232066' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T10:56:16.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:56:16 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/1313658666' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-27T10:56:16.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:56:16 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/238232066' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T10:56:16.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:56:16 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/1313658666' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-27T10:56:16.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.751+0000 7f8d26c00640 1 -- 172.21.15.97:0/4033855709 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f8d20062750 con 0x7f8d20103600 2024-03-27T10:56:16.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.752+0000 7f8d1e000640 1 -- 172.21.15.97:0/4033855709 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) ==== 100+0+11 (secure 0 0 0) 0x7f8d1809bcc0 con 0x7f8d20103600 2024-03-27T10:56:16.725 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-03-27T10:56:16.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.755+0000 7f8d26c00640 1 -- 172.21.15.97:0/4033855709 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f8d080a3610 msgr2=0x7f8d080a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:16.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.755+0000 7f8d26c00640 1 --2- 172.21.15.97:0/4033855709 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f8d080a3610 0x7f8d080a5ad0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f8d14036040 tx=0x7f8d14005fd0 comp rx=0 tx=0).stop 2024-03-27T10:56:16.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.755+0000 7f8d26c00640 1 -- 172.21.15.97:0/4033855709 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d20103600 msgr2=0x7f8d20074480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:16.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.755+0000 7f8d26c00640 1 --2- 172.21.15.97:0/4033855709 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d20103600 0x7f8d20074480 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f8d1800b970 tx=0x7f8d1800be30 comp rx=0 tx=0).stop 2024-03-27T10:56:16.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.755+0000 7f8d26c00640 1 -- 172.21.15.97:0/4033855709 shutdown_connections 2024-03-27T10:56:16.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.755+0000 7f8d26c00640 1 --2- 172.21.15.97:0/4033855709 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d20103600 0x7f8d20074480 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:16.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.755+0000 7f8d26c00640 1 --2- 172.21.15.97:0/4033855709 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f8d080a3610 0x7f8d080a5ad0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:16.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.755+0000 7f8d26c00640 1 --2- 172.21.15.97:0/4033855709 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8d20102c40 0x7f8d20073f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:16.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.755+0000 7f8d26c00640 1 --2- 172.21.15.97:0/4033855709 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8d20101a50 0x7f8d20075900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:16.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.755+0000 7f8d26c00640 1 -- 172.21.15.97:0/4033855709 >> 172.21.15.97:0/4033855709 conn(0x7f8d200fcfc0 msgr2=0x7f8d20106060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:16.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.755+0000 7f8d26c00640 1 -- 172.21.15.97:0/4033855709 shutdown_connections 2024-03-27T10:56:16.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.755+0000 7f8d26c00640 1 -- 172.21.15.97:0/4033855709 wait complete. 2024-03-27T10:56:16.747 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-27T10:56:16.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.979+0000 7f1c97200640 1 -- 172.21.15.97:0/2627814240 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c90103d90 msgr2=0x7f1c901041b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:16.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.979+0000 7f1c97200640 1 --2- 172.21.15.97:0/2627814240 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c90103d90 0x7f1c901041b0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f1c80009900 tx=0x7f1c8002bc30 comp rx=0 tx=0).stop 2024-03-27T10:56:16.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.979+0000 7f1c97200640 1 -- 172.21.15.97:0/2627814240 shutdown_connections 2024-03-27T10:56:16.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.979+0000 7f1c97200640 1 --2- 172.21.15.97:0/2627814240 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c90105940 0x7f1c9010c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:16.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.979+0000 7f1c97200640 1 --2- 172.21.15.97:0/2627814240 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1c90104f80 0x7f1c90105400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:16.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.979+0000 7f1c97200640 1 --2- 172.21.15.97:0/2627814240 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c90103d90 0x7f1c901041b0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:16.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.979+0000 7f1c97200640 1 -- 172.21.15.97:0/2627814240 >> 172.21.15.97:0/2627814240 conn(0x7f1c900ff300 msgr2=0x7f1c90101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:16.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.979+0000 7f1c97200640 1 -- 172.21.15.97:0/2627814240 shutdown_connections 2024-03-27T10:56:16.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.979+0000 7f1c97200640 1 -- 172.21.15.97:0/2627814240 wait complete. 2024-03-27T10:56:16.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.980+0000 7f1c97200640 1 Processor -- start 2024-03-27T10:56:16.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.980+0000 7f1c97200640 1 -- start start 2024-03-27T10:56:16.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.981+0000 7f1c97200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1c90103d90 0x7f1c9019a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:16.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.981+0000 7f1c97200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c90104f80 0x7f1c9019a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:16.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.981+0000 7f1c97200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c90105940 0x7f1c901a15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:16.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.981+0000 7f1c97200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1c9010f910 con 0x7f1c90105940 2024-03-27T10:56:16.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.981+0000 7f1c97200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f1c9010f790 con 0x7f1c90104f80 2024-03-27T10:56:16.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.981+0000 7f1c97200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1c9010fa90 con 0x7f1c90103d90 2024-03-27T10:56:16.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.981+0000 7f1c95200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c90105940 0x7f1c901a15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:16.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.981+0000 7f1c95200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c90105940 0x7f1c901a15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45400/0 (socket says 172.21.15.97:45400) 2024-03-27T10:56:16.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.981+0000 7f1c95200640 1 -- 172.21.15.97:0/183932976 learned_addr learned my addr 172.21.15.97:0/183932976 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:56:16.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.981+0000 7f1c8fe00640 1 --2- 172.21.15.97:0/183932976 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1c90103d90 0x7f1c9019a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:16.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.981+0000 7f1c87e00640 1 --2- 172.21.15.97:0/183932976 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c90104f80 0x7f1c9019a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:16.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.981+0000 7f1c95200640 1 -- 172.21.15.97:0/183932976 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1c90103d90 msgr2=0x7f1c9019a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:16.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.981+0000 7f1c95200640 1 --2- 172.21.15.97:0/183932976 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1c90103d90 0x7f1c9019a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:16.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.981+0000 7f1c95200640 1 -- 172.21.15.97:0/183932976 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c90104f80 msgr2=0x7f1c9019a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:16.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.981+0000 7f1c95200640 1 --2- 172.21.15.97:0/183932976 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c90104f80 0x7f1c9019a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:16.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.981+0000 7f1c95200640 1 -- 172.21.15.97:0/183932976 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c901a1c30 con 0x7f1c90105940 2024-03-27T10:56:16.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.982+0000 7f1c95200640 1 --2- 172.21.15.97:0/183932976 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c90105940 0x7f1c901a15c0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f1c8800eaa0 tx=0x7f1c8800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:16.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.982+0000 7f1c8d600640 1 -- 172.21.15.97:0/183932976 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c8800cdb0 con 0x7f1c90105940 2024-03-27T10:56:16.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.982+0000 7f1c8d600640 1 -- 172.21.15.97:0/183932976 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1c88005020 con 0x7f1c90105940 2024-03-27T10:56:16.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.982+0000 7f1c97200640 1 -- 172.21.15.97:0/183932976 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c901a1f20 con 0x7f1c90105940 2024-03-27T10:56:16.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.982+0000 7f1c8d600640 1 -- 172.21.15.97:0/183932976 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c880116d0 con 0x7f1c90105940 2024-03-27T10:56:16.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.982+0000 7f1c97200640 1 -- 172.21.15.97:0/183932976 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c901a2410 con 0x7f1c90105940 2024-03-27T10:56:16.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.983+0000 7f1c97200640 1 -- 172.21.15.97:0/183932976 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c9010b120 con 0x7f1c90105940 2024-03-27T10:56:16.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.985+0000 7f1c8d600640 1 -- 172.21.15.97:0/183932976 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f1c880051d0 con 0x7f1c90105940 2024-03-27T10:56:16.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.986+0000 7f1c8d600640 1 --2- 172.21.15.97:0/183932976 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f1c740a3610 0x7f1c740a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:16.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.986+0000 7f1c8d600640 1 -- 172.21.15.97:0/183932976 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f1c880cffb0 con 0x7f1c90105940 2024-03-27T10:56:16.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.987+0000 7f1c8fe00640 1 --2- 172.21.15.97:0/183932976 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f1c740a3610 0x7f1c740a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:16.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.987+0000 7f1c8fe00640 1 --2- 172.21.15.97:0/183932976 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f1c740a3610 0x7f1c740a5ad0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f1c80006010 tx=0x7f1c80005be0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:16.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:16.991+0000 7f1c8d600640 1 -- 172.21.15.97:0/183932976 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f1c88016030 con 0x7f1c90105940 2024-03-27T10:56:17.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.227+0000 7f1c97200640 1 -- 172.21.15.97:0/183932976 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1c9002d370 con 0x7f1c90105940 2024-03-27T10:56:17.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.228+0000 7f1c8d600640 1 -- 172.21.15.97:0/183932976 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11944 (secure 0 0 0) 0x7f1c8809cb90 con 0x7f1c90105940 2024-03-27T10:56:17.201 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T10:56:17.201 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1009423904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1009423904},{"type":"v1","addr":"172.21.15.97:6835","nonce":1009423904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3251173433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3251173433},{"type":"v1","addr":"172.21.15.97:6837","nonce":3251173433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-27T10:54:04.510178+0000","modified":"2024-03-27T10:54:12.180047+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34508,"mds_1":34478,"mds_2":34445,"mds_3":24454,"mds_4":24400},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14646":{"gid":14646,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6839/1057596821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1057596821},{"type":"v1","addr":"172.21.15.97:6839","nonce":1057596821}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14658":{"gid":14658,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6841/1672464856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6840","nonce":1672464856},{"type":"v1","addr":"172.21.15.97:6841","nonce":1672464856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.177:6833/3787981632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3787981632},{"type":"v1","addr":"172.21.15.177:6833","nonce":3787981632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.177:6835/3349112132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":3349112132},{"type":"v1","addr":"172.21.15.177:6835","nonce":3349112132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6837/3244432640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":3244432640},{"type":"v1","addr":"172.21.15.177:6837","nonce":3244432640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6839/3340454605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":3340454605},{"type":"v1","addr":"172.21.15.177:6839","nonce":3340454605}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34415":{"gid":34415,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6833/4052517902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":4052517902},{"type":"v1","addr":"172.21.15.114:6833","nonce":4052517902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34445":{"gid":34445,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.114:6835/3581918222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3581918222},{"type":"v1","addr":"172.21.15.114:6835","nonce":3581918222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34478":{"gid":34478,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.114:6837/616775588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":616775588},{"type":"v1","addr":"172.21.15.114:6837","nonce":616775588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34508":{"gid":34508,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.114:6839/3937193784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":3937193784},{"type":"v1","addr":"172.21.15.114:6839","nonce":3937193784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34508,"qdb_cluster":[34445,34508,24400,24454,34478]},"id":1}]} 2024-03-27T10:56:17.201 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 27 2024-03-27T10:56:17.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.230+0000 7f1c97200640 1 -- 172.21.15.97:0/183932976 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f1c740a3610 msgr2=0x7f1c740a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:17.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.230+0000 7f1c97200640 1 --2- 172.21.15.97:0/183932976 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f1c740a3610 0x7f1c740a5ad0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f1c80006010 tx=0x7f1c80005be0 comp rx=0 tx=0).stop 2024-03-27T10:56:17.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.231+0000 7f1c97200640 1 -- 172.21.15.97:0/183932976 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c90105940 msgr2=0x7f1c901a15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:17.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.231+0000 7f1c97200640 1 --2- 172.21.15.97:0/183932976 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c90105940 0x7f1c901a15c0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f1c8800eaa0 tx=0x7f1c8800ef60 comp rx=0 tx=0).stop 2024-03-27T10:56:17.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.231+0000 7f1c97200640 1 -- 172.21.15.97:0/183932976 shutdown_connections 2024-03-27T10:56:17.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.231+0000 7f1c97200640 1 --2- 172.21.15.97:0/183932976 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c90105940 0x7f1c901a15c0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:17.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.231+0000 7f1c97200640 1 --2- 172.21.15.97:0/183932976 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f1c740a3610 0x7f1c740a5ad0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:17.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.231+0000 7f1c97200640 1 --2- 172.21.15.97:0/183932976 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c90104f80 0x7f1c9019a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:17.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.231+0000 7f1c97200640 1 --2- 172.21.15.97:0/183932976 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1c90103d90 0x7f1c9019a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:17.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.231+0000 7f1c97200640 1 -- 172.21.15.97:0/183932976 >> 172.21.15.97:0/183932976 conn(0x7f1c900ff300 msgr2=0x7f1c901082f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:17.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.231+0000 7f1c97200640 1 -- 172.21.15.97:0/183932976 shutdown_connections 2024-03-27T10:56:17.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.231+0000 7f1c97200640 1 -- 172.21.15.97:0/183932976 wait complete. 2024-03-27T10:56:17.223 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-27T10:56:17.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.459+0000 7fc306400640 1 -- 172.21.15.97:0/2516335377 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc300105730 msgr2=0x7fc30010bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:17.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.459+0000 7fc306400640 1 --2- 172.21.15.97:0/2516335377 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc300105730 0x7fc30010bfc0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8009ca0 tx=0x7fc2f802c0f0 comp rx=0 tx=0).stop 2024-03-27T10:56:17.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.459+0000 7fc306400640 1 -- 172.21.15.97:0/2516335377 shutdown_connections 2024-03-27T10:56:17.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.459+0000 7fc306400640 1 --2- 172.21.15.97:0/2516335377 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc300105730 0x7fc30010bfc0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:17.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.459+0000 7fc306400640 1 --2- 172.21.15.97:0/2516335377 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc300104d70 0x7fc3001051f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:17.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.459+0000 7fc306400640 1 --2- 172.21.15.97:0/2516335377 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc300103b80 0x7fc300103fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:17.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.459+0000 7fc306400640 1 -- 172.21.15.97:0/2516335377 >> 172.21.15.97:0/2516335377 conn(0x7fc3000ff0f0 msgr2=0x7fc300101530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:17.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.459+0000 7fc306400640 1 -- 172.21.15.97:0/2516335377 shutdown_connections 2024-03-27T10:56:17.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.459+0000 7fc306400640 1 -- 172.21.15.97:0/2516335377 wait complete. 2024-03-27T10:56:17.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.460+0000 7fc306400640 1 Processor -- start 2024-03-27T10:56:17.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.460+0000 7fc306400640 1 -- start start 2024-03-27T10:56:17.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.461+0000 7fc306400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc300103b80 0x7fc300199cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:17.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.461+0000 7fc306400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc300104d70 0x7fc30019a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:17.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.461+0000 7fc306400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc300105730 0x7fc3001a1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:17.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.461+0000 7fc306400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc30010f690 con 0x7fc300103b80 2024-03-27T10:56:17.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.461+0000 7fc306400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc30010f510 con 0x7fc300105730 2024-03-27T10:56:17.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.461+0000 7fc306400640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc30010f810 con 0x7fc300104d70 2024-03-27T10:56:17.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.461+0000 7fc305000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc300103b80 0x7fc300199cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:17.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.461+0000 7fc305000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc300103b80 0x7fc300199cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45418/0 (socket says 172.21.15.97:45418) 2024-03-27T10:56:17.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.461+0000 7fc305000640 1 -- 172.21.15.97:0/3817441582 learned_addr learned my addr 172.21.15.97:0/3817441582 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:56:17.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.461+0000 7fc305a00640 1 --2- 172.21.15.97:0/3817441582 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc300105730 0x7fc3001a1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:17.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.461+0000 7fc305000640 1 -- 172.21.15.97:0/3817441582 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc300104d70 msgr2=0x7fc30019a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:17.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.461+0000 7fc2f7e00640 1 --2- 172.21.15.97:0/3817441582 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc300104d70 0x7fc30019a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:17.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.461+0000 7fc305000640 1 --2- 172.21.15.97:0/3817441582 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc300104d70 0x7fc30019a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:17.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.461+0000 7fc305000640 1 -- 172.21.15.97:0/3817441582 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc300105730 msgr2=0x7fc3001a1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:17.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.461+0000 7fc305000640 1 --2- 172.21.15.97:0/3817441582 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc300105730 0x7fc3001a1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:17.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.461+0000 7fc305000640 1 -- 172.21.15.97:0/3817441582 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3001a1900 con 0x7fc300103b80 2024-03-27T10:56:17.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.461+0000 7fc305a00640 1 --2- 172.21.15.97:0/3817441582 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc300105730 0x7fc3001a1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T10:56:17.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.462+0000 7fc305000640 1 --2- 172.21.15.97:0/3817441582 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc300103b80 0x7fc300199cf0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fc2e800d770 tx=0x7fc2e800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:17.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.462+0000 7fc2fe000640 1 -- 172.21.15.97:0/3817441582 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e8017070 con 0x7fc300103b80 2024-03-27T10:56:17.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.462+0000 7fc2fe000640 1 -- 172.21.15.97:0/3817441582 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc2e8009c30 con 0x7fc300103b80 2024-03-27T10:56:17.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.462+0000 7fc306400640 1 -- 172.21.15.97:0/3817441582 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3001a1bf0 con 0x7fc300103b80 2024-03-27T10:56:17.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.462+0000 7fc2fe000640 1 -- 172.21.15.97:0/3817441582 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e8003d00 con 0x7fc300103b80 2024-03-27T10:56:17.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.462+0000 7fc306400640 1 -- 172.21.15.97:0/3817441582 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3001a2190 con 0x7fc300103b80 2024-03-27T10:56:17.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.463+0000 7fc306400640 1 -- 172.21.15.97:0/3817441582 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc30010aea0 con 0x7fc300103b80 2024-03-27T10:56:17.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.465+0000 7fc2fe000640 1 -- 172.21.15.97:0/3817441582 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7fc2e8003ea0 con 0x7fc300103b80 2024-03-27T10:56:17.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.466+0000 7fc2fe000640 1 --2- 172.21.15.97:0/3817441582 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fc2e00a3690 0x7fc2e00a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:17.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.467+0000 7fc2fe000640 1 -- 172.21.15.97:0/3817441582 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fc2e80cfb30 con 0x7fc300103b80 2024-03-27T10:56:17.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.467+0000 7fc2f7e00640 1 --2- 172.21.15.97:0/3817441582 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fc2e00a3690 0x7fc2e00a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:17.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.467+0000 7fc2f7e00640 1 --2- 172.21.15.97:0/3817441582 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fc2e00a3690 0x7fc2e00a5b50 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc2f00058c0 tx=0x7fc2f0005c60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:17.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.471+0000 7fc2fe000640 1 -- 172.21.15.97:0/3817441582 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7fc2e809c710 con 0x7fc300103b80 2024-03-27T10:56:17.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:56:17 smithi114 ceph-mon[35338]: pgmap v430: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-27T10:56:17.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:56:17 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/1309498523' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-27T10:56:17.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:56:17 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/4033855709' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-27T10:56:17.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:56:17 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/183932976' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T10:56:17.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:56:17 smithi097 ceph-mon[31859]: pgmap v430: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-27T10:56:17.641 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:56:17 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/1309498523' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-27T10:56:17.641 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:56:17 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/4033855709' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-27T10:56:17.641 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:56:17 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/183932976' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T10:56:17.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.674+0000 7fc306400640 1 -- 172.21.15.97:0/3817441582 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc3000627f0 con 0x7fc300103b80 2024-03-27T10:56:17.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.678+0000 7fc2fe000640 1 -- 172.21.15.97:0/3817441582 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+22644 (secure 0 0 0) 0x7fc2e8009de0 con 0x7fc300103b80 2024-03-27T10:56:17.651 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T10:56:17.651 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":92,"fsid":"adca88c4-ec26-11ee-95d0-87774f69a715","created":"2024-03-27T10:43:24.680694+0000","modified":"2024-03-27T10:54:41.310812+0000","last_up_change":"2024-03-27T10:53:00.552513+0000","last_in_change":"2024-03-27T10:52:36.302986+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-27T10:47:54.052810+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-27T10:54:02.845060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-27T10:54:03.474151+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b634e5f1-6118-4e2f-95e9-830db390f8d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1746812080},{"type":"v1","addr":"172.21.15.97:6803","nonce":1746812080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1746812080},{"type":"v1","addr":"172.21.15.97:6805","nonce":1746812080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":1746812080},{"type":"v1","addr":"172.21.15.97:6809","nonce":1746812080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1746812080},{"type":"v1","addr":"172.21.15.97:6807","nonce":1746812080}]},"public_addr":"172.21.15.97:6803/1746812080","cluster_addr":"172.21.15.97:6805/1746812080","heartbeat_back_addr":"172.21.15.97:6809/1746812080","heartbeat_front_addr":"172.21.15.97:6807/1746812080","state":["exists","up"]},{"osd":1,"uuid":"441e2770-3284-4388-b0de-c56eb8a373d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":3858821264},{"type":"v1","addr":"172.21.15.114:6801","nonce":3858821264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":3858821264},{"type":"v1","addr":"172.21.15.114:6803","nonce":3858821264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":3858821264},{"type":"v1","addr":"172.21.15.114:6807","nonce":3858821264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":3858821264},{"type":"v1","addr":"172.21.15.114:6805","nonce":3858821264}]},"public_addr":"172.21.15.114:6801/3858821264","cluster_addr":"172.21.15.114:6803/3858821264","heartbeat_back_addr":"172.21.15.114:6807/3858821264","heartbeat_front_addr":"172.21.15.114:6805/3858821264","state":["exists","up"]},{"osd":2,"uuid":"00108864-1aa6-48d6-8398-e8eb79fda123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":124240658},{"type":"v1","addr":"172.21.15.177:6801","nonce":124240658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":124240658},{"type":"v1","addr":"172.21.15.177:6803","nonce":124240658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":124240658},{"type":"v1","addr":"172.21.15.177:6807","nonce":124240658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":124240658},{"type":"v1","addr":"172.21.15.177:6805","nonce":124240658}]},"public_addr":"172.21.15.177:6801/124240658","cluster_addr":"172.21.15.177:6803/124240658","heartbeat_back_addr":"172.21.15.177:6807/124240658","heartbeat_front_addr":"172.21.15.177:6805/124240658","state":["exists","up"]},{"osd":3,"uuid":"1860b690-e2dc-4977-88d7-d1aacf7022ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2478179311},{"type":"v1","addr":"172.21.15.97:6811","nonce":2478179311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2478179311},{"type":"v1","addr":"172.21.15.97:6813","nonce":2478179311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":2478179311},{"type":"v1","addr":"172.21.15.97:6817","nonce":2478179311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2478179311},{"type":"v1","addr":"172.21.15.97:6815","nonce":2478179311}]},"public_addr":"172.21.15.97:6811/2478179311","cluster_addr":"172.21.15.97:6813/2478179311","heartbeat_back_addr":"172.21.15.97:6817/2478179311","heartbeat_front_addr":"172.21.15.97:6815/2478179311","state":["exists","up"]},{"osd":4,"uuid":"9245a3a4-7d49-4fdb-9d55-112e43177050","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":75755945},{"type":"v1","addr":"172.21.15.114:6809","nonce":75755945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":75755945},{"type":"v1","addr":"172.21.15.114:6811","nonce":75755945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":75755945},{"type":"v1","addr":"172.21.15.114:6815","nonce":75755945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":75755945},{"type":"v1","addr":"172.21.15.114:6813","nonce":75755945}]},"public_addr":"172.21.15.114:6809/75755945","cluster_addr":"172.21.15.114:6811/75755945","heartbeat_back_addr":"172.21.15.114:6815/75755945","heartbeat_front_addr":"172.21.15.114:6813/75755945","state":["exists","up"]},{"osd":5,"uuid":"51922f99-e8d3-461b-8615-e9bfdcbf5ed3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":1918493790},{"type":"v1","addr":"172.21.15.177:6809","nonce":1918493790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":1918493790},{"type":"v1","addr":"172.21.15.177:6811","nonce":1918493790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":1918493790},{"type":"v1","addr":"172.21.15.177:6815","nonce":1918493790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":1918493790},{"type":"v1","addr":"172.21.15.177:6813","nonce":1918493790}]},"public_addr":"172.21.15.177:6809/1918493790","cluster_addr":"172.21.15.177:6811/1918493790","heartbeat_back_addr":"172.21.15.177:6815/1918493790","heartbeat_front_addr":"172.21.15.177:6813/1918493790","state":["exists","up"]},{"osd":6,"uuid":"06dffb32-05eb-43b3-852e-915c5cc7f25a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2892389563},{"type":"v1","addr":"172.21.15.97:6819","nonce":2892389563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2892389563},{"type":"v1","addr":"172.21.15.97:6821","nonce":2892389563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2892389563},{"type":"v1","addr":"172.21.15.97:6825","nonce":2892389563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2892389563},{"type":"v1","addr":"172.21.15.97:6823","nonce":2892389563}]},"public_addr":"172.21.15.97:6819/2892389563","cluster_addr":"172.21.15.97:6821/2892389563","heartbeat_back_addr":"172.21.15.97:6825/2892389563","heartbeat_front_addr":"172.21.15.97:6823/2892389563","state":["exists","up"]},{"osd":7,"uuid":"e68e3519-9c1b-43c5-8e06-5385248b19df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":2445491153},{"type":"v1","addr":"172.21.15.114:6817","nonce":2445491153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":2445491153},{"type":"v1","addr":"172.21.15.114:6819","nonce":2445491153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2445491153},{"type":"v1","addr":"172.21.15.114:6823","nonce":2445491153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":2445491153},{"type":"v1","addr":"172.21.15.114:6821","nonce":2445491153}]},"public_addr":"172.21.15.114:6817/2445491153","cluster_addr":"172.21.15.114:6819/2445491153","heartbeat_back_addr":"172.21.15.114:6823/2445491153","heartbeat_front_addr":"172.21.15.114:6821/2445491153","state":["exists","up"]},{"osd":8,"uuid":"31ac592e-1349-49ee-9af9-8ad85d682978","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":2051134191},{"type":"v1","addr":"172.21.15.177:6817","nonce":2051134191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":2051134191},{"type":"v1","addr":"172.21.15.177:6819","nonce":2051134191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":2051134191},{"type":"v1","addr":"172.21.15.177:6823","nonce":2051134191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":2051134191},{"type":"v1","addr":"172.21.15.177:6821","nonce":2051134191}]},"public_addr":"172.21.15.177:6817/2051134191","cluster_addr":"172.21.15.177:6819/2051134191","heartbeat_back_addr":"172.21.15.177:6823/2051134191","heartbeat_front_addr":"172.21.15.177:6821/2051134191","state":["exists","up"]},{"osd":9,"uuid":"0171086a-dd41-4eb4-a389-16f0c46a64db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":1578881271},{"type":"v1","addr":"172.21.15.97:6827","nonce":1578881271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1578881271},{"type":"v1","addr":"172.21.15.97:6829","nonce":1578881271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1578881271},{"type":"v1","addr":"172.21.15.97:6833","nonce":1578881271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1578881271},{"type":"v1","addr":"172.21.15.97:6831","nonce":1578881271}]},"public_addr":"172.21.15.97:6827/1578881271","cluster_addr":"172.21.15.97:6829/1578881271","heartbeat_back_addr":"172.21.15.97:6833/1578881271","heartbeat_front_addr":"172.21.15.97:6831/1578881271","state":["exists","up"]},{"osd":10,"uuid":"64754e31-b124-4601-ad98-2f70b1dd5359","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":2893111461},{"type":"v1","addr":"172.21.15.114:6825","nonce":2893111461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":2893111461},{"type":"v1","addr":"172.21.15.114:6827","nonce":2893111461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2893111461},{"type":"v1","addr":"172.21.15.114:6831","nonce":2893111461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":2893111461},{"type":"v1","addr":"172.21.15.114:6829","nonce":2893111461}]},"public_addr":"172.21.15.114:6825/2893111461","cluster_addr":"172.21.15.114:6827/2893111461","heartbeat_back_addr":"172.21.15.114:6831/2893111461","heartbeat_front_addr":"172.21.15.114:6829/2893111461","state":["exists","up"]},{"osd":11,"uuid":"9ffc752c-adf7-4037-89d3-535787d7475d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":85,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":2333030391},{"type":"v1","addr":"172.21.15.177:6825","nonce":2333030391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":2333030391},{"type":"v1","addr":"172.21.15.177:6827","nonce":2333030391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":2333030391},{"type":"v1","addr":"172.21.15.177:6831","nonce":2333030391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":2333030391},{"type":"v1","addr":"172.21.15.177:6829","nonce":2333030391}]},"public_addr":"172.21.15.177:6825/2333030391","cluster_addr":"172.21.15.177:6827/2333030391","heartbeat_back_addr":"172.21.15.177:6831/2333030391","heartbeat_front_addr":"172.21.15.177:6829/2333030391","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:46:45.143519+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:47:18.656947+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:47:51.316345+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:48:24.575512+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:48:59.113555+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:49:32.103865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:50:06.615539+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:50:41.578341+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:51:15.254011+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:51:49.099165+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:52:24.544821+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:52:58.838484+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.28","mappings":[{"from":3,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/359646":"2024-03-28T10:44:39.889550+0000","172.21.15.97:6801/98682155":"2024-03-28T10:44:06.019075+0000","172.21.15.97:6801/1685921853":"2024-03-28T10:44:39.889550+0000","172.21.15.97:6800/98682155":"2024-03-28T10:44:06.019075+0000","172.21.15.97:0/107690680":"2024-03-28T10:44:06.019075+0000","172.21.15.97:6800/1685921853":"2024-03-28T10:44:39.889550+0000","172.21.15.97:0/437043413":"2024-03-28T10:44:39.889550+0000","172.21.15.97:0/2319706992":"2024-03-28T10:44:39.889550+0000","172.21.15.97:0/3911532654":"2024-03-28T10:44:06.019075+0000","172.21.15.97:0/3347321845":"2024-03-28T10:44:06.019075+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-27T10:56:17.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.681+0000 7fc306400640 1 -- 172.21.15.97:0/3817441582 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fc2e00a3690 msgr2=0x7fc2e00a5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:17.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.681+0000 7fc306400640 1 --2- 172.21.15.97:0/3817441582 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fc2e00a3690 0x7fc2e00a5b50 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc2f00058c0 tx=0x7fc2f0005c60 comp rx=0 tx=0).stop 2024-03-27T10:56:17.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.681+0000 7fc306400640 1 -- 172.21.15.97:0/3817441582 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc300103b80 msgr2=0x7fc300199cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:17.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.681+0000 7fc306400640 1 --2- 172.21.15.97:0/3817441582 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc300103b80 0x7fc300199cf0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fc2e800d770 tx=0x7fc2e800dc30 comp rx=0 tx=0).stop 2024-03-27T10:56:17.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.681+0000 7fc306400640 1 -- 172.21.15.97:0/3817441582 shutdown_connections 2024-03-27T10:56:17.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.681+0000 7fc306400640 1 --2- 172.21.15.97:0/3817441582 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fc2e00a3690 0x7fc2e00a5b50 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:17.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.681+0000 7fc306400640 1 --2- 172.21.15.97:0/3817441582 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc300105730 0x7fc3001a1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:17.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.681+0000 7fc306400640 1 --2- 172.21.15.97:0/3817441582 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc300104d70 0x7fc30019a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:17.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.681+0000 7fc306400640 1 --2- 172.21.15.97:0/3817441582 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc300103b80 0x7fc300199cf0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:17.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.681+0000 7fc306400640 1 -- 172.21.15.97:0/3817441582 >> 172.21.15.97:0/3817441582 conn(0x7fc3000ff0f0 msgr2=0x7fc300108070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:17.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.681+0000 7fc306400640 1 -- 172.21.15.97:0/3817441582 shutdown_connections 2024-03-27T10:56:17.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.681+0000 7fc306400640 1 -- 172.21.15.97:0/3817441582 wait complete. 2024-03-27T10:56:17.671 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-03-27T10:54:04.510178+0000', 'modified': '2024-03-27T10:54:12.180047+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 34508, 'mds_1': 34478, 'mds_2': 34445, 'mds_3': 24454, 'mds_4': 24400}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14646': {'gid': 14646, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.97:6839/1057596821', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6838', 'nonce': 1057596821}, {'type': 'v1', 'addr': '172.21.15.97:6839', 'nonce': 1057596821}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14658': {'gid': 14658, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.97:6841/1672464856', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6840', 'nonce': 1672464856}, {'type': 'v1', 'addr': '172.21.15.97:6841', 'nonce': 1672464856}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24400': {'gid': 24400, 'name': 'l', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 15, 'addr': '172.21.15.177:6833/3787981632', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6832', 'nonce': 3787981632}, {'type': 'v1', 'addr': '172.21.15.177:6833', 'nonce': 3787981632}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24454': {'gid': 24454, 'name': 'f', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.177:6835/3349112132', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6834', 'nonce': 3349112132}, {'type': 'v1', 'addr': '172.21.15.177:6835', 'nonce': 3349112132}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24466': {'gid': 24466, 'name': 'c', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.177:6837/3244432640', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6836', 'nonce': 3244432640}, {'type': 'v1', 'addr': '172.21.15.177:6837', 'nonce': 3244432640}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24487': {'gid': 24487, 'name': 'i', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.177:6839/3340454605', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6838', 'nonce': 3340454605}, {'type': 'v1', 'addr': '172.21.15.177:6839', 'nonce': 3340454605}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34415': {'gid': 34415, 'name': 'k', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.114:6833/4052517902', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6832', 'nonce': 4052517902}, {'type': 'v1', 'addr': '172.21.15.114:6833', 'nonce': 4052517902}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34445': {'gid': 34445, 'name': 'e', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.114:6835/3581918222', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6834', 'nonce': 3581918222}, {'type': 'v1', 'addr': '172.21.15.114:6835', 'nonce': 3581918222}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34478': {'gid': 34478, 'name': 'h', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.114:6837/616775588', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6836', 'nonce': 616775588}, {'type': 'v1', 'addr': '172.21.15.114:6837', 'nonce': 616775588}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34508': {'gid': 34508, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.114:6839/3937193784', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6838', 'nonce': 3937193784}, {'type': 'v1', 'addr': '172.21.15.114:6839', 'nonce': 3937193784}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 34508, 'qdb_cluster': [34445, 34508, 24400, 24454, 34478]} 2024-03-27T10:56:17.671 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-03-27T10:56:17.671 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.l status 2024-03-27T10:56:17.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:56:17 smithi177 ceph-mon[35294]: pgmap v430: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-27T10:56:17.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:56:17 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/1309498523' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-27T10:56:17.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:56:17 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/4033855709' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-27T10:56:17.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:56:17 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/183932976' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T10:56:17.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.904+0000 7f45ffa00640 1 -- 172.21.15.97:0/361257576 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45f8109aa0 msgr2=0x7f45f810bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:17.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.904+0000 7f45ffa00640 1 --2- 172.21.15.97:0/361257576 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45f8109aa0 0x7f45f810bf30 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f45e8009900 tx=0x7f45e802bc30 comp rx=0 tx=0).stop 2024-03-27T10:56:17.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.904+0000 7f45ffa00640 1 -- 172.21.15.97:0/361257576 shutdown_connections 2024-03-27T10:56:17.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.904+0000 7f45ffa00640 1 --2- 172.21.15.97:0/361257576 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45f8109aa0 0x7f45f810bf30 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:17.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.904+0000 7f45ffa00640 1 --2- 172.21.15.97:0/361257576 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f45f80745b0 0x7f45f8074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:17.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.905+0000 7f45ffa00640 1 --2- 172.21.15.97:0/361257576 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f45f8075b90 0x7f45f8073fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:17.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.905+0000 7f45ffa00640 1 -- 172.21.15.97:0/361257576 >> 172.21.15.97:0/361257576 conn(0x7f45f80fd150 msgr2=0x7f45f80ff590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:17.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.905+0000 7f45ffa00640 1 -- 172.21.15.97:0/361257576 shutdown_connections 2024-03-27T10:56:17.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.905+0000 7f45ffa00640 1 -- 172.21.15.97:0/361257576 wait complete. 2024-03-27T10:56:17.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.905+0000 7f45ffa00640 1 Processor -- start 2024-03-27T10:56:17.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.906+0000 7f45ffa00640 1 -- start start 2024-03-27T10:56:17.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.906+0000 7f45ffa00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f45f80745b0 0x7f45f8199f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:17.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.906+0000 7f45ffa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45f8075b90 0x7f45f819a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:17.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.906+0000 7f45ffa00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f45f8109aa0 0x7f45f81a14e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:17.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.906+0000 7f45ffa00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f45f810f8d0 con 0x7f45f8075b90 2024-03-27T10:56:17.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.906+0000 7f45ffa00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f45f810f750 con 0x7f45f8109aa0 2024-03-27T10:56:17.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.906+0000 7f45ffa00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f45f810fa50 con 0x7f45f80745b0 2024-03-27T10:56:17.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.906+0000 7f45f7e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45f8075b90 0x7f45f819a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:17.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.906+0000 7f45f7e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45f8075b90 0x7f45f819a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45428/0 (socket says 172.21.15.97:45428) 2024-03-27T10:56:17.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.906+0000 7f45f7e00640 1 -- 172.21.15.97:0/770669457 learned_addr learned my addr 172.21.15.97:0/770669457 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:56:17.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.906+0000 7f45fd000640 1 --2- 172.21.15.97:0/770669457 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f45f80745b0 0x7f45f8199f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:17.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.907+0000 7f45fda00640 1 --2- 172.21.15.97:0/770669457 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f45f8109aa0 0x7f45f81a14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:17.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.907+0000 7f45f7e00640 1 -- 172.21.15.97:0/770669457 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f45f80745b0 msgr2=0x7f45f8199f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:17.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.907+0000 7f45f7e00640 1 --2- 172.21.15.97:0/770669457 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f45f80745b0 0x7f45f8199f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:17.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.907+0000 7f45f7e00640 1 -- 172.21.15.97:0/770669457 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f45f8109aa0 msgr2=0x7f45f81a14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:17.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.907+0000 7f45f7e00640 1 --2- 172.21.15.97:0/770669457 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f45f8109aa0 0x7f45f81a14e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:17.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.907+0000 7f45f7e00640 1 -- 172.21.15.97:0/770669457 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45f81a1b50 con 0x7f45f8075b90 2024-03-27T10:56:17.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.907+0000 7f45fd000640 1 --2- 172.21.15.97:0/770669457 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f45f80745b0 0x7f45f8199f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:56:17.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.907+0000 7f45f7e00640 1 --2- 172.21.15.97:0/770669457 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45f8075b90 0x7f45f819a470 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f45e000d950 tx=0x7f45e000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:17.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.907+0000 7f45f5600640 1 -- 172.21.15.97:0/770669457 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e0017070 con 0x7f45f8075b90 2024-03-27T10:56:17.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.907+0000 7f45f5600640 1 -- 172.21.15.97:0/770669457 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f45e0009c30 con 0x7f45f8075b90 2024-03-27T10:56:17.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.907+0000 7f45f5600640 1 -- 172.21.15.97:0/770669457 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e000f620 con 0x7f45f8075b90 2024-03-27T10:56:17.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.907+0000 7f45ffa00640 1 -- 172.21.15.97:0/770669457 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45f81a1e40 con 0x7f45f8075b90 2024-03-27T10:56:17.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.907+0000 7f45ffa00640 1 -- 172.21.15.97:0/770669457 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45f81a2300 con 0x7f45f8075b90 2024-03-27T10:56:17.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.910+0000 7f460261c740 1 Processor -- start 2024-03-27T10:56:17.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.911+0000 7f45f5600640 1 -- 172.21.15.97:0/770669457 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f45e00109a0 con 0x7f45f8075b90 2024-03-27T10:56:17.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.911+0000 7f460261c740 1 -- start start 2024-03-27T10:56:17.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.911+0000 7f460261c740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e830632f20 0x55e830633340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:17.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.911+0000 7f460261c740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e830633910 0x55e830638410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:17.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.911+0000 7f460261c740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e830638a80 0x55e83063af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:17.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.911+0000 7f460261c740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55e8305f5920 con 0x55e830633910 2024-03-27T10:56:17.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.911+0000 7f460261c740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55e830545fa0 con 0x55e830632f20 2024-03-27T10:56:17.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.911+0000 7f460261c740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55e830597f90 con 0x55e830638a80 2024-03-27T10:56:17.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.911+0000 7f45fd000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e830633910 0x55e830638410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:17.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.911+0000 7f45fd000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e830633910 0x55e830638410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45430/0 (socket says 172.21.15.97:45430) 2024-03-27T10:56:17.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.911+0000 7f45fd000640 1 -- 172.21.15.97:0/3693305730 learned_addr learned my addr 172.21.15.97:0/3693305730 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:56:17.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.911+0000 7f45fda00640 1 --2- 172.21.15.97:0/3693305730 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e830632f20 0x55e830633340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:17.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.912+0000 7f45f7e00640 1 --2- 172.21.15.97:0/3693305730 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e830638a80 0x55e83063af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:17.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.912+0000 7f45fd000640 1 -- 172.21.15.97:0/3693305730 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e830638a80 msgr2=0x55e83063af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:56:17.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.912+0000 7f45fd000640 1 --2- 172.21.15.97:0/3693305730 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e830638a80 0x55e83063af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:17.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.912+0000 7f45fd000640 1 -- 172.21.15.97:0/3693305730 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e830632f20 msgr2=0x55e830633340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:56:17.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.912+0000 7f45fd000640 1 --2- 172.21.15.97:0/3693305730 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e830632f20 0x55e830633340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:17.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.912+0000 7f45fd000640 1 -- 172.21.15.97:0/3693305730 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8302ac2d0 con 0x55e830633910 2024-03-27T10:56:17.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.912+0000 7f45f7e00640 1 --2- 172.21.15.97:0/3693305730 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e830638a80 0x55e83063af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:56:17.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.912+0000 7f45fd000640 1 --2- 172.21.15.97:0/3693305730 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e830633910 0x55e830638410 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f45f0009f20 tx=0x7f45f000f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a98f9f3701ffe604 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:17.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.912+0000 7f45f5600640 1 --2- 172.21.15.97:0/770669457 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f45dc0a3610 0x7f45dc0a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:17.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.912+0000 7f45ee000640 1 -- 172.21.15.97:0/3693305730 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f0004d20 con 0x55e830633910 2024-03-27T10:56:17.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.912+0000 7f45ee000640 1 -- 172.21.15.97:0/3693305730 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f45f00054d0 con 0x55e830633910 2024-03-27T10:56:17.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.912+0000 7f45f5600640 1 -- 172.21.15.97:0/770669457 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f45e00cf9d0 con 0x7f45f8075b90 2024-03-27T10:56:17.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.912+0000 7f45ee000640 1 -- 172.21.15.97:0/3693305730 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f0011890 con 0x55e830633910 2024-03-27T10:56:17.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.912+0000 7f45fda00640 1 --2- 172.21.15.97:0/770669457 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f45dc0a3610 0x7f45dc0a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:17.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.913+0000 7f460261c740 1 -- 172.21.15.97:0/3693305730 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e830633910 msgr2=0x55e830638410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:17.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.913+0000 7f45fda00640 1 --2- 172.21.15.97:0/770669457 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f45dc0a3610 0x7f45dc0a5ad0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f45e8009230 tx=0x7f45e802c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:17.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.913+0000 7f460261c740 1 --2- 172.21.15.97:0/3693305730 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e830633910 0x55e830638410 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f45f0009f20 tx=0x7f45f000f7f0 comp rx=0 tx=0).stop 2024-03-27T10:56:17.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.913+0000 7f460261c740 1 -- 172.21.15.97:0/3693305730 shutdown_connections 2024-03-27T10:56:17.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.913+0000 7f460261c740 1 --2- 172.21.15.97:0/3693305730 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e830638a80 0x55e83063af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:17.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.913+0000 7f460261c740 1 --2- 172.21.15.97:0/3693305730 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e830633910 0x55e830638410 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:17.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.913+0000 7f460261c740 1 --2- 172.21.15.97:0/3693305730 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e830632f20 0x55e830633340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:17.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.913+0000 7f460261c740 1 -- 172.21.15.97:0/3693305730 >> 172.21.15.97:0/3693305730 conn(0x55e83062e490 msgr2=0x55e83062e890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:17.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.913+0000 7f460261c740 1 -- 172.21.15.97:0/3693305730 shutdown_connections 2024-03-27T10:56:17.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.913+0000 7f460261c740 1 -- 172.21.15.97:0/3693305730 wait complete. 2024-03-27T10:56:17.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.914+0000 7f460261c740 1 -- start start 2024-03-27T10:56:17.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.914+0000 7f460261c740 1 Processor -- start 2024-03-27T10:56:17.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.914+0000 7f460261c740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e830632f20 0x55e83063d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:17.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.914+0000 7f460261c740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e830633910 0x55e83063d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:17.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.914+0000 7f460261c740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e830638a80 0x55e830644710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:17.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.914+0000 7f460261c740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55e830593f40 con 0x55e830632f20 2024-03-27T10:56:17.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.914+0000 7f460261c740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55e830540650 con 0x55e830638a80 2024-03-27T10:56:17.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.914+0000 7f460261c740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55e830553a10 con 0x55e830633910 2024-03-27T10:56:17.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.915+0000 7f460261c740 10 client.0 _ll_register_callbacks cb 0x55e830592ff0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:56:17.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.915+0000 7f45fd000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e830632f20 0x55e83063d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:17.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.915+0000 7f45fd000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e830632f20 0x55e83063d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45434/0 (socket says 172.21.15.97:45434) 2024-03-27T10:56:17.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.915+0000 7f45fd000640 1 -- 172.21.15.97:0/3896816163 learned_addr learned my addr 172.21.15.97:0/3896816163 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:56:17.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.915+0000 7f45fd000640 1 -- 172.21.15.97:0/3896816163 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e830633910 msgr2=0x55e83063d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:56:17.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.915+0000 7f45fd000640 1 --2- 172.21.15.97:0/3896816163 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e830633910 0x55e83063d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:17.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.915+0000 7f45fd000640 1 -- 172.21.15.97:0/3896816163 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e830638a80 msgr2=0x55e830644710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:56:17.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.915+0000 7f45fd000640 1 --2- 172.21.15.97:0/3896816163 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e830638a80 0x55e830644710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:17.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.915+0000 7f45fd000640 1 -- 172.21.15.97:0/3896816163 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8303766b0 con 0x55e830632f20 2024-03-27T10:56:17.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.915+0000 7f45fd000640 1 --2- 172.21.15.97:0/3896816163 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e830632f20 0x55e83063d190 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f45f0006010 tx=0x7f45f0005300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a4f23791a07ca43 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:17.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.915+0000 7f45e6a00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T10:56:17.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.915+0000 7f45e6a00640 1 -- 172.21.15.97:0/3896816163 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f00049e0 con 0x55e830632f20 2024-03-27T10:56:17.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.916+0000 7f45e6a00640 1 -- 172.21.15.97:0/3896816163 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f45f0004b80 con 0x55e830632f20 2024-03-27T10:56:17.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.916+0000 7f460261c740 1 -- 172.21.15.97:0/3896816163 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45f00054d0 con 0x55e830632f20 2024-03-27T10:56:17.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.916+0000 7f45e6a00640 1 -- 172.21.15.97:0/3896816163 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f45f0004d80 con 0x55e830632f20 2024-03-27T10:56:17.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.916+0000 7f460261c740 10 client.15273 fetch_fsmap learned FSMap version 27 2024-03-27T10:56:17.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.916+0000 7f460261c740 1 -- 172.21.15.97:0/3896816163 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e830647a60 con 0x55e830632f20 2024-03-27T10:56:17.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.916+0000 7f45e6a00640 1 -- 172.21.15.97:0/3896816163 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f45f00054d0 con 0x55e830632f20 2024-03-27T10:56:17.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.916+0000 7f45e6a00640 1 -- 172.21.15.97:0/3896816163 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f45f0019800 con 0x55e830632f20 2024-03-27T10:56:17.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.916+0000 7f460261c740 10 client.15273 fetch_fsmap finished waiting for FSMap version 27 2024-03-27T10:56:17.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.916+0000 7f460261c740 10 client.15273 resolve_mds: resolved name 'l' to daemon mds.l 2024-03-27T10:56:17.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.916+0000 7f460261c740 20 client.15273 populate_metadata read hostname 'smithi097' 2024-03-27T10:56:17.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.916+0000 7f460261c740 1 --2- 172.21.15.97:0/3896816163 >> [v2:172.21.15.177:6832/3787981632,v1:172.21.15.177:6833/3787981632] conn(0x55e830649f40 0x55e83064b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:17.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.916+0000 7f460261c740 4 client.15273 mds_command: new command op to 24400 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T10:56:17.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.916+0000 7f460261c740 1 -- 172.21.15.97:0/3896816163 --> [v2:172.21.15.177:6832/3787981632,v1:172.21.15.177:6833/3787981632] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e8305a54d0 con 0x55e830649f40 2024-03-27T10:56:17.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.917+0000 7f45f7e00640 1 --2- 172.21.15.97:0/3896816163 >> [v2:172.21.15.177:6832/3787981632,v1:172.21.15.177:6833/3787981632] conn(0x55e830649f40 0x55e83064b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:17.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.918+0000 7f45f7e00640 1 --2- 172.21.15.97:0/3896816163 >> [v2:172.21.15.177:6832/3787981632,v1:172.21.15.177:6833/3787981632] conn(0x55e830649f40 0x55e83064b3b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4ce8e6cbe96e7075 server_cookie=1be16d916a3b7ecf in_seq=0 out_seq=0 2024-03-27T10:56:17.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.918+0000 7f45e6a00640 10 client.15273 ms_handle_connect on v2:172.21.15.177:6832/3787981632 2024-03-27T10:56:17.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.920+0000 7f45e6a00640 1 -- 172.21.15.97:0/3896816163 <== mds.4 v2:172.21.15.177:6832/3787981632 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55e8305a54d0 con 0x55e830649f40 2024-03-27T10:56:17.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.920+0000 7f45e6a00640 10 client.15273 handle_command_reply: tid=0 2024-03-27T10:56:17.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.920+0000 7f460261c740 1 client.15273 shutdown 2024-03-27T10:56:17.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.920+0000 7f460261c740 1 -- 172.21.15.97:0/3896816163 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e830632f20 msgr2=0x55e83063d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:17.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.920+0000 7f460261c740 1 --2- 172.21.15.97:0/3896816163 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e830632f20 0x55e83063d190 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f45f0006010 tx=0x7f45f0005300 comp rx=0 tx=0).stop 2024-03-27T10:56:17.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.920+0000 7f460261c740 1 -- 172.21.15.97:0/3896816163 shutdown_connections 2024-03-27T10:56:17.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.920+0000 7f460261c740 1 --2- 172.21.15.97:0/3896816163 >> [v2:172.21.15.177:6832/3787981632,v1:172.21.15.177:6833/3787981632] conn(0x55e830649f40 0x55e83064b3b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:17.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.920+0000 7f460261c740 1 --2- 172.21.15.97:0/3896816163 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e830638a80 0x55e830644710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:17.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.920+0000 7f460261c740 1 --2- 172.21.15.97:0/3896816163 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e830633910 0x55e83063d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:17.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.920+0000 7f45e6a00640 0 client.15273 ms_handle_reset on v2:172.21.15.177:6832/3787981632 2024-03-27T10:56:17.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.920+0000 7f460261c740 1 --2- 172.21.15.97:0/3896816163 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e830632f20 0x55e83063d190 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:17.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.920+0000 7f460261c740 1 -- 172.21.15.97:0/3896816163 >> 172.21.15.97:0/3896816163 conn(0x55e83062e490 msgr2=0x55e83062d690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:17.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.920+0000 7f460261c740 1 -- 172.21.15.97:0/3896816163 shutdown_connections 2024-03-27T10:56:17.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.920+0000 7f460261c740 1 -- 172.21.15.97:0/3896816163 wait complete. 2024-03-27T10:56:17.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.920+0000 7f460261c740 20 client.15273 trim_cache size 0 max 16384 2024-03-27T10:56:17.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.939+0000 7f460261c740 1 Processor -- start 2024-03-27T10:56:17.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.939+0000 7f460261c740 1 -- start start 2024-03-27T10:56:17.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.939+0000 7f460261c740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e830632f20 0x55e830631e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:17.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.939+0000 7f460261c740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e830633910 0x55e8306446d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:17.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.939+0000 7f460261c740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e830638a80 0x55e830652680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:17.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.939+0000 7f460261c740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55e8306496f0 con 0x55e830632f20 2024-03-27T10:56:17.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.939+0000 7f460261c740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55e830649dc0 con 0x55e830638a80 2024-03-27T10:56:17.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.939+0000 7f460261c740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55e83064a490 con 0x55e830633910 2024-03-27T10:56:17.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.940+0000 7f45fd000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e830632f20 0x55e830631e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:17.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.940+0000 7f45fd000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e830632f20 0x55e830631e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45442/0 (socket says 172.21.15.97:45442) 2024-03-27T10:56:17.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.940+0000 7f45fd000640 1 -- 172.21.15.97:0/3961669655 learned_addr learned my addr 172.21.15.97:0/3961669655 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:56:17.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.940+0000 7f45f7e00640 1 --2- 172.21.15.97:0/3961669655 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e830633910 0x55e8306446d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:17.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.940+0000 7f45fda00640 1 --2- 172.21.15.97:0/3961669655 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e830638a80 0x55e830652680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:17.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.940+0000 7f45fd000640 1 -- 172.21.15.97:0/3961669655 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e830633910 msgr2=0x55e8306446d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:56:17.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.940+0000 7f45fd000640 1 --2- 172.21.15.97:0/3961669655 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e830633910 0x55e8306446d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:17.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.940+0000 7f45fd000640 1 -- 172.21.15.97:0/3961669655 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e830638a80 msgr2=0x55e830652680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:56:17.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.940+0000 7f45fd000640 1 --2- 172.21.15.97:0/3961669655 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e830638a80 0x55e830652680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:17.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.940+0000 7f45fd000640 1 -- 172.21.15.97:0/3961669655 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e830633cf0 con 0x55e830632f20 2024-03-27T10:56:17.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.940+0000 7f45fda00640 1 --2- 172.21.15.97:0/3961669655 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e830638a80 0x55e830652680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:56:17.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.940+0000 7f45fd000640 1 --2- 172.21.15.97:0/3961669655 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e830632f20 0x55e830631e00 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f45f00093e0 tx=0x7f45f0005990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecc2bc7e16c2bb6a server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:17.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.940+0000 7f45ed600640 1 -- 172.21.15.97:0/3961669655 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f0004b80 con 0x55e830632f20 2024-03-27T10:56:17.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.941+0000 7f45ed600640 1 -- 172.21.15.97:0/3961669655 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f45f0028070 con 0x55e830632f20 2024-03-27T10:56:17.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.941+0000 7f45ed600640 1 -- 172.21.15.97:0/3961669655 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f0023900 con 0x55e830632f20 2024-03-27T10:56:17.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.941+0000 7f460261c740 1 -- 172.21.15.97:0/3961669655 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e830632f20 msgr2=0x55e830631e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:17.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.941+0000 7f460261c740 1 --2- 172.21.15.97:0/3961669655 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e830632f20 0x55e830631e00 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f45f00093e0 tx=0x7f45f0005990 comp rx=0 tx=0).stop 2024-03-27T10:56:17.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.941+0000 7f460261c740 1 -- 172.21.15.97:0/3961669655 shutdown_connections 2024-03-27T10:56:17.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.941+0000 7f460261c740 1 --2- 172.21.15.97:0/3961669655 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e830638a80 0x55e830652680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:17.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.941+0000 7f460261c740 1 --2- 172.21.15.97:0/3961669655 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e830633910 0x55e8306446d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:17.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.941+0000 7f460261c740 1 --2- 172.21.15.97:0/3961669655 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e830632f20 0x55e830631e00 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:17.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.941+0000 7f460261c740 1 -- 172.21.15.97:0/3961669655 >> 172.21.15.97:0/3961669655 conn(0x55e83062e490 msgr2=0x55e8306316f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:17.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.941+0000 7f460261c740 1 -- 172.21.15.97:0/3961669655 shutdown_connections 2024-03-27T10:56:17.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.942+0000 7f460261c740 1 -- 172.21.15.97:0/3961669655 wait complete. 2024-03-27T10:56:17.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.942+0000 7f460261c740 1 -- start start 2024-03-27T10:56:17.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.942+0000 7f460261c740 1 Processor -- start 2024-03-27T10:56:17.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.942+0000 7f460261c740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e830632f20 0x55e83063fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:17.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.942+0000 7f460261c740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e830633910 0x55e830640180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:17.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.942+0000 7f460261c740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e830638a80 0x55e8306431a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:17.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.942+0000 7f460261c740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55e83064b230 con 0x55e830632f20 2024-03-27T10:56:17.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.942+0000 7f460261c740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55e83064ab60 con 0x55e830633910 2024-03-27T10:56:17.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.942+0000 7f460261c740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55e83064b900 con 0x55e830638a80 2024-03-27T10:56:17.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.942+0000 7f460261c740 10 client.0 _ll_register_callbacks cb 0x7f45f0009d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:56:17.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.943+0000 7f45fd000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e830632f20 0x55e83063fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:17.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.943+0000 7f45fd000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e830632f20 0x55e83063fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45458/0 (socket says 172.21.15.97:45458) 2024-03-27T10:56:17.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.943+0000 7f45fd000640 1 -- 172.21.15.97:0/2383053445 learned_addr learned my addr 172.21.15.97:0/2383053445 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:56:17.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.943+0000 7f45fd000640 1 -- 172.21.15.97:0/2383053445 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e830638a80 msgr2=0x55e8306431a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:56:17.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.943+0000 7f45fd000640 1 --2- 172.21.15.97:0/2383053445 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e830638a80 0x55e8306431a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:17.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.943+0000 7f45fd000640 1 -- 172.21.15.97:0/2383053445 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e830633910 msgr2=0x55e830640180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:56:17.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.943+0000 7f45fd000640 1 --2- 172.21.15.97:0/2383053445 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e830633910 0x55e830640180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:17.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.943+0000 7f45fd000640 1 -- 172.21.15.97:0/2383053445 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e830643790 con 0x55e830632f20 2024-03-27T10:56:17.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.943+0000 7f45fd000640 1 --2- 172.21.15.97:0/2383053445 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e830632f20 0x55e83063fc40 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f45f0005ae0 tx=0x7f45f0009790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd9788c0f8cbae2a server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:17.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.943+0000 7f45e6a00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T10:56:17.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.943+0000 7f45e6a00640 1 -- 172.21.15.97:0/2383053445 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f0019630 con 0x55e830632f20 2024-03-27T10:56:17.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.943+0000 7f45e6a00640 1 -- 172.21.15.97:0/2383053445 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f45f00197d0 con 0x55e830632f20 2024-03-27T10:56:17.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.943+0000 7f460261c740 1 -- 172.21.15.97:0/2383053445 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45f0028070 con 0x55e830632f20 2024-03-27T10:56:17.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.944+0000 7f45e6a00640 1 -- 172.21.15.97:0/2383053445 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f45f00052e0 con 0x55e830632f20 2024-03-27T10:56:17.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.944+0000 7f460261c740 10 client.15279 fetch_fsmap learned FSMap version 27 2024-03-27T10:56:17.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.944+0000 7f460261c740 1 -- 172.21.15.97:0/2383053445 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e830644a00 con 0x55e830632f20 2024-03-27T10:56:17.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.944+0000 7f45e6a00640 1 -- 172.21.15.97:0/2383053445 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f45f0028070 con 0x55e830632f20 2024-03-27T10:56:17.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.944+0000 7f45e6a00640 1 -- 172.21.15.97:0/2383053445 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f45f0037030 con 0x55e830632f20 2024-03-27T10:56:17.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.944+0000 7f460261c740 10 client.15279 fetch_fsmap finished waiting for FSMap version 27 2024-03-27T10:56:17.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.944+0000 7f460261c740 10 client.15279 resolve_mds: resolved name 'l' to daemon mds.l 2024-03-27T10:56:17.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.944+0000 7f460261c740 20 client.15279 populate_metadata read hostname 'smithi097' 2024-03-27T10:56:17.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.944+0000 7f460261c740 1 --2- 172.21.15.97:0/2383053445 >> [v2:172.21.15.177:6832/3787981632,v1:172.21.15.177:6833/3787981632] conn(0x55e8306b4310 0x55e8306b3020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:17.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.944+0000 7f460261c740 4 client.15279 mds_command: new command op to 24400 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-27T10:56:17.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.944+0000 7f460261c740 1 -- 172.21.15.97:0/2383053445 --> [v2:172.21.15.177:6832/3787981632,v1:172.21.15.177:6833/3787981632] -- command(tid 0: {"prefix": "status"}) -- 0x55e83059f820 con 0x55e8306b4310 2024-03-27T10:56:17.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.945+0000 7f45f7e00640 1 --2- 172.21.15.97:0/2383053445 >> [v2:172.21.15.177:6832/3787981632,v1:172.21.15.177:6833/3787981632] conn(0x55e8306b4310 0x55e8306b3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:17.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.945+0000 7f45f7e00640 1 --2- 172.21.15.97:0/2383053445 >> [v2:172.21.15.177:6832/3787981632,v1:172.21.15.177:6833/3787981632] conn(0x55e8306b4310 0x55e8306b3020 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=23dd168d007353d8 server_cookie=40230cde79b89a8e in_seq=0 out_seq=0 2024-03-27T10:56:17.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.945+0000 7f45e6a00640 10 client.15279 ms_handle_connect on v2:172.21.15.177:6832/3787981632 2024-03-27T10:56:17.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.946+0000 7f45e6a00640 1 -- 172.21.15.97:0/2383053445 <== mds.4 v2:172.21.15.177:6832/3787981632 1 ==== command_reply(tid 0: 0 ) ==== 8+0+321 (crc 0 0 0) 0x55e83059f820 con 0x55e8306b4310 2024-03-27T10:56:17.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.946+0000 7f45e6a00640 10 client.15279 handle_command_reply: tid=0 2024-03-27T10:56:17.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.946+0000 7f460261c740 1 client.15279 shutdown 2024-03-27T10:56:17.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.946+0000 7f460261c740 1 -- 172.21.15.97:0/2383053445 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e830632f20 msgr2=0x55e83063fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:17.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.946+0000 7f460261c740 1 --2- 172.21.15.97:0/2383053445 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e830632f20 0x55e83063fc40 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f45f0005ae0 tx=0x7f45f0009790 comp rx=0 tx=0).stop 2024-03-27T10:56:17.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.946+0000 7f460261c740 1 -- 172.21.15.97:0/2383053445 shutdown_connections 2024-03-27T10:56:17.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.946+0000 7f460261c740 1 --2- 172.21.15.97:0/2383053445 >> [v2:172.21.15.177:6832/3787981632,v1:172.21.15.177:6833/3787981632] conn(0x55e8306b4310 0x55e8306b3020 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:17.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.946+0000 7f460261c740 1 --2- 172.21.15.97:0/2383053445 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e830638a80 0x55e8306431a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:17.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.946+0000 7f460261c740 1 --2- 172.21.15.97:0/2383053445 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e830633910 0x55e830640180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:17.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.946+0000 7f45e6a00640 0 client.15279 ms_handle_reset on v2:172.21.15.177:6832/3787981632 2024-03-27T10:56:17.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.946+0000 7f460261c740 1 --2- 172.21.15.97:0/2383053445 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e830632f20 0x55e83063fc40 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:17.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.947+0000 7f460261c740 1 -- 172.21.15.97:0/2383053445 >> 172.21.15.97:0/2383053445 conn(0x55e83062e490 msgr2=0x55e830631f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:17.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.947+0000 7f460261c740 1 -- 172.21.15.97:0/2383053445 shutdown_connections 2024-03-27T10:56:17.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.947+0000 7f460261c740 1 -- 172.21.15.97:0/2383053445 wait complete. 2024-03-27T10:56:17.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.947+0000 7f460261c740 20 client.15279 trim_cache size 0 max 16384 2024-03-27T10:56:17.923 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-03-27T10:56:17.923 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "adca88c4-ec26-11ee-95d0-87774f69a715", 2024-03-27T10:56:17.923 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 4, 2024-03-27T10:56:17.923 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 24400, 2024-03-27T10:56:17.923 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2024-03-27T10:56:17.923 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2024-03-27T10:56:17.923 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2024-03-27T10:56:17.923 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 126.77178341, 2024-03-27T10:56:17.923 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 27, 2024-03-27T10:56:17.923 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 92, 2024-03-27T10:56:17.923 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 90, 2024-03-27T10:56:17.923 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 181.443450031 2024-03-27T10:56:17.924 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-03-27T10:56:17.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.947+0000 7f45ffa00640 1 -- 172.21.15.97:0/770669457 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f45dc0a3610 msgr2=0x7f45dc0a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:17.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.947+0000 7f45ffa00640 1 --2- 172.21.15.97:0/770669457 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f45dc0a3610 0x7f45dc0a5ad0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f45e8009230 tx=0x7f45e802c7e0 comp rx=0 tx=0).stop 2024-03-27T10:56:17.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.947+0000 7f45ffa00640 1 -- 172.21.15.97:0/770669457 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45f8075b90 msgr2=0x7f45f819a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:17.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.947+0000 7f45ffa00640 1 --2- 172.21.15.97:0/770669457 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45f8075b90 0x7f45f819a470 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f45e000d950 tx=0x7f45e000de10 comp rx=0 tx=0).stop 2024-03-27T10:56:17.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.947+0000 7f45ffa00640 1 -- 172.21.15.97:0/770669457 shutdown_connections 2024-03-27T10:56:17.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.947+0000 7f45ffa00640 1 --2- 172.21.15.97:0/770669457 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f45dc0a3610 0x7f45dc0a5ad0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:17.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.947+0000 7f45ffa00640 1 --2- 172.21.15.97:0/770669457 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f45f8109aa0 0x7f45f81a14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:17.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.947+0000 7f45ffa00640 1 --2- 172.21.15.97:0/770669457 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45f8075b90 0x7f45f819a470 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:17.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.947+0000 7f45ffa00640 1 --2- 172.21.15.97:0/770669457 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f45f80745b0 0x7f45f8199f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:17.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.947+0000 7f45ffa00640 1 -- 172.21.15.97:0/770669457 >> 172.21.15.97:0/770669457 conn(0x7f45f80fd150 msgr2=0x7f45f81059a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:17.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.948+0000 7f45ffa00640 1 -- 172.21.15.97:0/770669457 shutdown_connections 2024-03-27T10:56:17.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:17.948+0000 7f45ffa00640 1 -- 172.21.15.97:0/770669457 wait complete. 2024-03-27T10:56:17.935 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2024-03-27T10:56:18.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.159+0000 7ff122600640 1 -- 172.21.15.97:0/704715292 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff11c077750 msgr2=0x7ff11c077bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:18.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.159+0000 7ff122600640 1 --2- 172.21.15.97:0/704715292 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff11c077750 0x7ff11c077bb0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7ff104009900 tx=0x7ff10402bc50 comp rx=0 tx=0).stop 2024-03-27T10:56:18.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.159+0000 7ff122600640 1 -- 172.21.15.97:0/704715292 shutdown_connections 2024-03-27T10:56:18.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.159+0000 7ff122600640 1 --2- 172.21.15.97:0/704715292 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff11c0780f0 0x7ff11c078990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.159+0000 7ff122600640 1 --2- 172.21.15.97:0/704715292 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff11c077750 0x7ff11c077bb0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.159+0000 7ff122600640 1 --2- 172.21.15.97:0/704715292 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff11c076510 0x7ff11c076930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.160+0000 7ff122600640 1 -- 172.21.15.97:0/704715292 >> 172.21.15.97:0/704715292 conn(0x7ff11c0ff300 msgr2=0x7ff11c101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:18.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.160+0000 7ff122600640 1 -- 172.21.15.97:0/704715292 shutdown_connections 2024-03-27T10:56:18.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.160+0000 7ff122600640 1 -- 172.21.15.97:0/704715292 wait complete. 2024-03-27T10:56:18.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.160+0000 7ff122600640 1 Processor -- start 2024-03-27T10:56:18.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.161+0000 7ff122600640 1 -- start start 2024-03-27T10:56:18.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.161+0000 7ff122600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff11c076510 0x7ff11c19e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:18.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.161+0000 7ff122600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff11c077750 0x7ff11c19e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:18.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.161+0000 7ff122600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff11c0780f0 0x7ff11c1a5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:18.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.161+0000 7ff122600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff11c113d10 con 0x7ff11c077750 2024-03-27T10:56:18.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.161+0000 7ff122600640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff11c113b90 con 0x7ff11c0780f0 2024-03-27T10:56:18.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.161+0000 7ff122600640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff11c113e90 con 0x7ff11c076510 2024-03-27T10:56:18.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.161+0000 7ff11aa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff11c077750 0x7ff11c19e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:18.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.161+0000 7ff11aa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff11c077750 0x7ff11c19e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45478/0 (socket says 172.21.15.97:45478) 2024-03-27T10:56:18.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.161+0000 7ff11aa00640 1 -- 172.21.15.97:0/3150313474 learned_addr learned my addr 172.21.15.97:0/3150313474 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:56:18.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.161+0000 7ff11b400640 1 --2- 172.21.15.97:0/3150313474 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff11c076510 0x7ff11c19e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:18.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.161+0000 7ff11be00640 1 --2- 172.21.15.97:0/3150313474 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff11c0780f0 0x7ff11c1a5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:18.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.162+0000 7ff11aa00640 1 -- 172.21.15.97:0/3150313474 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff11c076510 msgr2=0x7ff11c19e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:18.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.162+0000 7ff11aa00640 1 --2- 172.21.15.97:0/3150313474 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff11c076510 0x7ff11c19e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.162+0000 7ff11aa00640 1 -- 172.21.15.97:0/3150313474 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff11c0780f0 msgr2=0x7ff11c1a5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:18.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.162+0000 7ff11aa00640 1 --2- 172.21.15.97:0/3150313474 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff11c0780f0 0x7ff11c1a5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.162+0000 7ff11aa00640 1 -- 172.21.15.97:0/3150313474 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff11c1a5fc0 con 0x7ff11c077750 2024-03-27T10:56:18.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.162+0000 7ff11aa00640 1 --2- 172.21.15.97:0/3150313474 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff11c077750 0x7ff11c19e8e0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ff104000c00 tx=0x7ff1040042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:18.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.162+0000 7ff113e00640 1 -- 172.21.15.97:0/3150313474 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff104043020 con 0x7ff11c077750 2024-03-27T10:56:18.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.162+0000 7ff113e00640 1 -- 172.21.15.97:0/3150313474 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff1040054c0 con 0x7ff11c077750 2024-03-27T10:56:18.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.162+0000 7ff122600640 1 -- 172.21.15.97:0/3150313474 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff11c1a6250 con 0x7ff11c077750 2024-03-27T10:56:18.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.162+0000 7ff113e00640 1 -- 172.21.15.97:0/3150313474 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff104003d70 con 0x7ff11c077750 2024-03-27T10:56:18.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.162+0000 7ff122600640 1 -- 172.21.15.97:0/3150313474 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff11c1a67f0 con 0x7ff11c077750 2024-03-27T10:56:18.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.165+0000 7ff125259740 1 Processor -- start 2024-03-27T10:56:18.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.165+0000 7ff113e00640 1 -- 172.21.15.97:0/3150313474 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7ff104046020 con 0x7ff11c077750 2024-03-27T10:56:18.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.165+0000 7ff125259740 1 -- start start 2024-03-27T10:56:18.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.165+0000 7ff125259740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557c9375ef20 0x557c9375f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:18.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.165+0000 7ff125259740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c9375f910 0x557c93764410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:18.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.166+0000 7ff125259740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557c93764a80 0x557c93766f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:18.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.166+0000 7ff125259740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x557c93721920 con 0x557c9375f910 2024-03-27T10:56:18.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.166+0000 7ff125259740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x557c93671fa0 con 0x557c9375ef20 2024-03-27T10:56:18.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.166+0000 7ff125259740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x557c936c3f90 con 0x557c93764a80 2024-03-27T10:56:18.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.166+0000 7ff11b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c9375f910 0x557c93764410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:18.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.166+0000 7ff11b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c9375f910 0x557c93764410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45486/0 (socket says 172.21.15.97:45486) 2024-03-27T10:56:18.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.166+0000 7ff11b400640 1 -- 172.21.15.97:0/1320782313 learned_addr learned my addr 172.21.15.97:0/1320782313 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:56:18.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.166+0000 7ff11be00640 1 --2- 172.21.15.97:0/1320782313 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557c9375ef20 0x557c9375f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:18.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.166+0000 7ff11aa00640 1 --2- 172.21.15.97:0/1320782313 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557c93764a80 0x557c93766f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:18.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.166+0000 7ff11b400640 1 -- 172.21.15.97:0/1320782313 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557c93764a80 msgr2=0x557c93766f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:56:18.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.166+0000 7ff11b400640 1 --2- 172.21.15.97:0/1320782313 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557c93764a80 0x557c93766f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.166+0000 7ff11b400640 1 -- 172.21.15.97:0/1320782313 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557c9375ef20 msgr2=0x557c9375f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:56:18.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.166+0000 7ff11b400640 1 --2- 172.21.15.97:0/1320782313 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557c9375ef20 0x557c9375f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.166+0000 7ff11b400640 1 -- 172.21.15.97:0/1320782313 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c933d82d0 con 0x557c9375f910 2024-03-27T10:56:18.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.166+0000 7ff11aa00640 1 --2- 172.21.15.97:0/1320782313 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557c93764a80 0x557c93766f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:56:18.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.166+0000 7ff11be00640 1 --2- 172.21.15.97:0/1320782313 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557c9375ef20 0x557c9375f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T10:56:18.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.166+0000 7ff11b400640 1 --2- 172.21.15.97:0/1320782313 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c9375f910 0x557c93764410 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ff10c00bcf0 tx=0x7ff10c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfbf1a5257b938ef server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:18.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.167+0000 7ff110c00640 1 -- 172.21.15.97:0/1320782313 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff10c015070 con 0x557c9375f910 2024-03-27T10:56:18.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.167+0000 7ff110c00640 1 -- 172.21.15.97:0/1320782313 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff10c01a910 con 0x557c9375f910 2024-03-27T10:56:18.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.167+0000 7ff110c00640 1 -- 172.21.15.97:0/1320782313 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff10c0118e0 con 0x557c9375f910 2024-03-27T10:56:18.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.167+0000 7ff113e00640 1 --2- 172.21.15.97:0/3150313474 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7ff1000a3690 0x7ff1000a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:18.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.167+0000 7ff113e00640 1 -- 172.21.15.97:0/3150313474 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7ff1040f21a0 con 0x7ff11c077750 2024-03-27T10:56:18.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.167+0000 7ff11be00640 1 --2- 172.21.15.97:0/3150313474 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7ff1000a3690 0x7ff1000a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:18.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.167+0000 7ff125259740 1 -- 172.21.15.97:0/1320782313 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c9375f910 msgr2=0x557c93764410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:18.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.167+0000 7ff125259740 1 --2- 172.21.15.97:0/1320782313 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c9375f910 0x557c93764410 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ff10c00bcf0 tx=0x7ff10c00f730 comp rx=0 tx=0).stop 2024-03-27T10:56:18.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.168+0000 7ff125259740 1 -- 172.21.15.97:0/1320782313 shutdown_connections 2024-03-27T10:56:18.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.168+0000 7ff125259740 1 --2- 172.21.15.97:0/1320782313 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557c93764a80 0x557c93766f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.168+0000 7ff125259740 1 --2- 172.21.15.97:0/1320782313 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c9375f910 0x557c93764410 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.168+0000 7ff125259740 1 --2- 172.21.15.97:0/1320782313 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557c9375ef20 0x557c9375f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.168+0000 7ff125259740 1 -- 172.21.15.97:0/1320782313 >> 172.21.15.97:0/1320782313 conn(0x557c9375a490 msgr2=0x557c9375a890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:18.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.168+0000 7ff125259740 1 -- 172.21.15.97:0/1320782313 shutdown_connections 2024-03-27T10:56:18.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.168+0000 7ff11be00640 1 --2- 172.21.15.97:0/3150313474 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7ff1000a3690 0x7ff1000a5b50 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x557c93760170 tx=0x7ff114008860 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:18.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.168+0000 7ff125259740 1 -- 172.21.15.97:0/1320782313 wait complete. 2024-03-27T10:56:18.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.168+0000 7ff125259740 1 -- start start 2024-03-27T10:56:18.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.168+0000 7ff125259740 1 Processor -- start 2024-03-27T10:56:18.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.169+0000 7ff125259740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c9375ef20 0x557c93769120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:18.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.169+0000 7ff125259740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557c9375f910 0x557c93769660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:18.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.169+0000 7ff125259740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557c93764a80 0x557c937706a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:18.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.169+0000 7ff125259740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x557c936bff40 con 0x557c9375ef20 2024-03-27T10:56:18.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.169+0000 7ff125259740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x557c9366c650 con 0x557c9375f910 2024-03-27T10:56:18.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.169+0000 7ff125259740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x557c9367fa10 con 0x557c93764a80 2024-03-27T10:56:18.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.169+0000 7ff125259740 10 client.0 _ll_register_callbacks cb 0x557c936beff0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:56:18.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.169+0000 7ff11b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c9375ef20 0x557c93769120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:18.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.169+0000 7ff11b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c9375ef20 0x557c93769120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45492/0 (socket says 172.21.15.97:45492) 2024-03-27T10:56:18.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.169+0000 7ff11b400640 1 -- 172.21.15.97:0/2976067910 learned_addr learned my addr 172.21.15.97:0/2976067910 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:56:18.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.169+0000 7ff11b400640 1 -- 172.21.15.97:0/2976067910 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557c93764a80 msgr2=0x557c937706a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:56:18.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.169+0000 7ff11be00640 1 --2- 172.21.15.97:0/2976067910 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557c93764a80 0x557c937706a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:18.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.169+0000 7ff11aa00640 1 --2- 172.21.15.97:0/2976067910 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557c9375f910 0x557c93769660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:18.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.169+0000 7ff11b400640 1 --2- 172.21.15.97:0/2976067910 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557c93764a80 0x557c937706a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.169+0000 7ff11b400640 1 -- 172.21.15.97:0/2976067910 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557c9375f910 msgr2=0x557c93769660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:56:18.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.169+0000 7ff11b400640 1 --2- 172.21.15.97:0/2976067910 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557c9375f910 0x557c93769660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.169+0000 7ff11b400640 1 -- 172.21.15.97:0/2976067910 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c934a26b0 con 0x557c9375ef20 2024-03-27T10:56:18.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.170+0000 7ff11b400640 1 --2- 172.21.15.97:0/2976067910 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c9375ef20 0x557c93769120 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7ff10c019000 tx=0x7ff10c01a690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99d6d81ce8b3b8ba server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:18.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.170+0000 7ff109600640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T10:56:18.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.170+0000 7ff109600640 1 -- 172.21.15.97:0/2976067910 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff10c028060 con 0x557c9375ef20 2024-03-27T10:56:18.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.170+0000 7ff109600640 1 -- 172.21.15.97:0/2976067910 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7ff10c01acd0 con 0x557c9375ef20 2024-03-27T10:56:18.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.170+0000 7ff125259740 1 -- 172.21.15.97:0/2976067910 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff10c01a910 con 0x557c9375ef20 2024-03-27T10:56:18.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.170+0000 7ff109600640 1 -- 172.21.15.97:0/2976067910 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff10c015070 con 0x557c9375ef20 2024-03-27T10:56:18.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.170+0000 7ff125259740 10 client.15291 fetch_fsmap learned FSMap version 27 2024-03-27T10:56:18.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.170+0000 7ff125259740 1 -- 172.21.15.97:0/2976067910 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557c93773980 con 0x557c9375ef20 2024-03-27T10:56:18.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.170+0000 7ff109600640 1 -- 172.21.15.97:0/2976067910 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff10c01a910 con 0x557c9375ef20 2024-03-27T10:56:18.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.171+0000 7ff109600640 1 -- 172.21.15.97:0/2976067910 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ff10c004a30 con 0x557c9375ef20 2024-03-27T10:56:18.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.171+0000 7ff125259740 10 client.15291 fetch_fsmap finished waiting for FSMap version 27 2024-03-27T10:56:18.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.171+0000 7ff125259740 10 client.15291 resolve_mds: resolved name 'f' to daemon mds.f 2024-03-27T10:56:18.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.171+0000 7ff125259740 20 client.15291 populate_metadata read hostname 'smithi097' 2024-03-27T10:56:18.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.171+0000 7ff125259740 1 --2- 172.21.15.97:0/2976067910 >> [v2:172.21.15.177:6834/3349112132,v1:172.21.15.177:6835/3349112132] conn(0x557c93775c00 0x557c93777070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:18.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.171+0000 7ff125259740 4 client.15291 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T10:56:18.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.171+0000 7ff125259740 1 -- 172.21.15.97:0/2976067910 --> [v2:172.21.15.177:6834/3349112132,v1:172.21.15.177:6835/3349112132] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557c936d14d0 con 0x557c93775c00 2024-03-27T10:56:18.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.171+0000 7ff11aa00640 1 --2- 172.21.15.97:0/2976067910 >> [v2:172.21.15.177:6834/3349112132,v1:172.21.15.177:6835/3349112132] conn(0x557c93775c00 0x557c93777070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:18.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.172+0000 7ff11aa00640 1 --2- 172.21.15.97:0/2976067910 >> [v2:172.21.15.177:6834/3349112132,v1:172.21.15.177:6835/3349112132] conn(0x557c93775c00 0x557c93777070 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3724bf6c3e9496ba server_cookie=1268fe848c97b456 in_seq=0 out_seq=0 2024-03-27T10:56:18.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.172+0000 7ff109600640 10 client.15291 ms_handle_connect on v2:172.21.15.177:6834/3349112132 2024-03-27T10:56:18.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.174+0000 7ff109600640 1 -- 172.21.15.97:0/2976067910 <== mds.3 v2:172.21.15.177:6834/3349112132 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x557c936d14d0 con 0x557c93775c00 2024-03-27T10:56:18.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.174+0000 7ff109600640 10 client.15291 handle_command_reply: tid=0 2024-03-27T10:56:18.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.174+0000 7ff125259740 1 client.15291 shutdown 2024-03-27T10:56:18.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.174+0000 7ff125259740 1 -- 172.21.15.97:0/2976067910 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c9375ef20 msgr2=0x557c93769120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:18.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.174+0000 7ff125259740 1 --2- 172.21.15.97:0/2976067910 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c9375ef20 0x557c93769120 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7ff10c019000 tx=0x7ff10c01a690 comp rx=0 tx=0).stop 2024-03-27T10:56:18.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.174+0000 7ff125259740 1 -- 172.21.15.97:0/2976067910 shutdown_connections 2024-03-27T10:56:18.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.174+0000 7ff125259740 1 --2- 172.21.15.97:0/2976067910 >> [v2:172.21.15.177:6834/3349112132,v1:172.21.15.177:6835/3349112132] conn(0x557c93775c00 0x557c93777070 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.174+0000 7ff125259740 1 --2- 172.21.15.97:0/2976067910 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557c93764a80 0x557c937706a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.174+0000 7ff125259740 1 --2- 172.21.15.97:0/2976067910 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557c9375f910 0x557c93769660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.174+0000 7ff109600640 0 client.15291 ms_handle_reset on v2:172.21.15.177:6834/3349112132 2024-03-27T10:56:18.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.174+0000 7ff125259740 1 --2- 172.21.15.97:0/2976067910 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c9375ef20 0x557c93769120 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.174+0000 7ff125259740 1 -- 172.21.15.97:0/2976067910 >> 172.21.15.97:0/2976067910 conn(0x557c9375a490 msgr2=0x557c93759690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:18.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.174+0000 7ff125259740 1 -- 172.21.15.97:0/2976067910 shutdown_connections 2024-03-27T10:56:18.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.174+0000 7ff125259740 1 -- 172.21.15.97:0/2976067910 wait complete. 2024-03-27T10:56:18.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.174+0000 7ff125259740 20 client.15291 trim_cache size 0 max 16384 2024-03-27T10:56:18.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.193+0000 7ff125259740 1 Processor -- start 2024-03-27T10:56:18.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.194+0000 7ff125259740 1 -- start start 2024-03-27T10:56:18.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.194+0000 7ff125259740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557c9375ef20 0x557c93764440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:18.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.194+0000 7ff125259740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c9375f910 0x557c93770660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:18.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.194+0000 7ff125259740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557c93764a80 0x557c93758e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:18.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.194+0000 7ff125259740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x557c937753b0 con 0x557c9375f910 2024-03-27T10:56:18.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.194+0000 7ff125259740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x557c93775a80 con 0x557c93764a80 2024-03-27T10:56:18.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.194+0000 7ff125259740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x557c93776150 con 0x557c9375ef20 2024-03-27T10:56:18.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.194+0000 7ff11aa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c9375f910 0x557c93770660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:18.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.194+0000 7ff11aa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c9375f910 0x557c93770660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45502/0 (socket says 172.21.15.97:45502) 2024-03-27T10:56:18.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.194+0000 7ff11aa00640 1 -- 172.21.15.97:0/1022668690 learned_addr learned my addr 172.21.15.97:0/1022668690 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:56:18.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.194+0000 7ff11b400640 1 --2- 172.21.15.97:0/1022668690 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557c9375ef20 0x557c93764440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:18.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.194+0000 7ff11be00640 1 --2- 172.21.15.97:0/1022668690 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557c93764a80 0x557c93758e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:18.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.194+0000 7ff11aa00640 1 -- 172.21.15.97:0/1022668690 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557c9375ef20 msgr2=0x557c93764440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:56:18.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.194+0000 7ff11aa00640 1 --2- 172.21.15.97:0/1022668690 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557c9375ef20 0x557c93764440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.194+0000 7ff11aa00640 1 -- 172.21.15.97:0/1022668690 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557c93764a80 msgr2=0x557c93758e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:56:18.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.194+0000 7ff11aa00640 1 --2- 172.21.15.97:0/1022668690 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557c93764a80 0x557c93758e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.195+0000 7ff11aa00640 1 -- 172.21.15.97:0/1022668690 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c937751e0 con 0x557c9375f910 2024-03-27T10:56:18.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.195+0000 7ff11b400640 1 --2- 172.21.15.97:0/1022668690 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557c9375ef20 0x557c93764440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:56:18.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.195+0000 7ff11be00640 1 --2- 172.21.15.97:0/1022668690 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557c93764a80 0x557c93758e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:56:18.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.195+0000 7ff11aa00640 1 --2- 172.21.15.97:0/1022668690 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c9375f910 0x557c93770660 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7ff104006010 tx=0x7ff1040efb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a25cca60ea649520 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:18.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.195+0000 7ff10be00640 1 -- 172.21.15.97:0/1022668690 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1040dc070 con 0x557c9375f910 2024-03-27T10:56:18.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.195+0000 7ff10be00640 1 -- 172.21.15.97:0/1022668690 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff104004c50 con 0x557c9375f910 2024-03-27T10:56:18.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.195+0000 7ff10be00640 1 -- 172.21.15.97:0/1022668690 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1040d7030 con 0x557c9375f910 2024-03-27T10:56:18.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.196+0000 7ff125259740 1 -- 172.21.15.97:0/1022668690 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c9375f910 msgr2=0x557c93770660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:18.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.196+0000 7ff125259740 1 --2- 172.21.15.97:0/1022668690 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c9375f910 0x557c93770660 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7ff104006010 tx=0x7ff1040efb80 comp rx=0 tx=0).stop 2024-03-27T10:56:18.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.196+0000 7ff125259740 1 -- 172.21.15.97:0/1022668690 shutdown_connections 2024-03-27T10:56:18.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.196+0000 7ff125259740 1 --2- 172.21.15.97:0/1022668690 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557c93764a80 0x557c93758e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.196+0000 7ff125259740 1 --2- 172.21.15.97:0/1022668690 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c9375f910 0x557c93770660 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.196+0000 7ff125259740 1 --2- 172.21.15.97:0/1022668690 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557c9375ef20 0x557c93764440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.196+0000 7ff125259740 1 -- 172.21.15.97:0/1022668690 >> 172.21.15.97:0/1022668690 conn(0x557c9375a490 msgr2=0x557c9375a870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:18.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.196+0000 7ff125259740 1 -- 172.21.15.97:0/1022668690 shutdown_connections 2024-03-27T10:56:18.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.196+0000 7ff125259740 1 -- 172.21.15.97:0/1022668690 wait complete. 2024-03-27T10:56:18.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.196+0000 7ff125259740 1 -- start start 2024-03-27T10:56:18.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.196+0000 7ff125259740 1 Processor -- start 2024-03-27T10:56:18.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.197+0000 7ff125259740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557c9375ef20 0x557c9376e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:18.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.197+0000 7ff125259740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c9375f910 0x557c937d7950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:18.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.197+0000 7ff125259740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557c93764a80 0x557c937d9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:18.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.197+0000 7ff125259740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x557c93776ef0 con 0x557c9375f910 2024-03-27T10:56:18.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.197+0000 7ff125259740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x557c93776820 con 0x557c9375ef20 2024-03-27T10:56:18.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.197+0000 7ff125259740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x557c937775c0 con 0x557c93764a80 2024-03-27T10:56:18.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.197+0000 7ff125259740 10 client.0 _ll_register_callbacks cb 0x7ff10c00bb40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:56:18.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.197+0000 7ff11aa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c9375f910 0x557c937d7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:18.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.197+0000 7ff11aa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c9375f910 0x557c937d7950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45512/0 (socket says 172.21.15.97:45512) 2024-03-27T10:56:18.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.197+0000 7ff11aa00640 1 -- 172.21.15.97:0/3687197859 learned_addr learned my addr 172.21.15.97:0/3687197859 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:56:18.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.197+0000 7ff11b400640 1 --2- 172.21.15.97:0/3687197859 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557c9375ef20 0x557c9376e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:18.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.197+0000 7ff11be00640 1 --2- 172.21.15.97:0/3687197859 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557c93764a80 0x557c937d9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:18.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.197+0000 7ff11aa00640 1 -- 172.21.15.97:0/3687197859 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557c93764a80 msgr2=0x557c937d9ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:56:18.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.197+0000 7ff11aa00640 1 --2- 172.21.15.97:0/3687197859 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557c93764a80 0x557c937d9ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.197+0000 7ff11aa00640 1 -- 172.21.15.97:0/3687197859 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557c9375ef20 msgr2=0x557c9376e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:56:18.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.197+0000 7ff11aa00640 1 --2- 172.21.15.97:0/3687197859 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557c9375ef20 0x557c9376e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.197+0000 7ff11aa00640 1 -- 172.21.15.97:0/3687197859 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c937da590 con 0x557c9375f910 2024-03-27T10:56:18.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.197+0000 7ff11b400640 1 --2- 172.21.15.97:0/3687197859 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557c9375ef20 0x557c9376e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:56:18.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.197+0000 7ff11be00640 1 --2- 172.21.15.97:0/3687197859 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557c93764a80 0x557c937d9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:56:18.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.198+0000 7ff11aa00640 1 --2- 172.21.15.97:0/3687197859 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c9375f910 0x557c937d7950 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7ff1040efd00 tx=0x7ff1040bab50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cbfb1a3a7520d9d server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:18.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.198+0000 7ff109600640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T10:56:18.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.198+0000 7ff109600640 1 -- 172.21.15.97:0/3687197859 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1040a2060 con 0x557c9375f910 2024-03-27T10:56:18.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.198+0000 7ff109600640 1 -- 172.21.15.97:0/3687197859 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7ff1040bae10 con 0x557c9375f910 2024-03-27T10:56:18.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.198+0000 7ff125259740 1 -- 172.21.15.97:0/3687197859 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff104004c50 con 0x557c9375f910 2024-03-27T10:56:18.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.198+0000 7ff109600640 1 -- 172.21.15.97:0/3687197859 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff1040dc070 con 0x557c9375f910 2024-03-27T10:56:18.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.198+0000 7ff125259740 10 client.15297 fetch_fsmap learned FSMap version 27 2024-03-27T10:56:18.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.198+0000 7ff125259740 1 -- 172.21.15.97:0/3687197859 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557c937db7e0 con 0x557c9375f910 2024-03-27T10:56:18.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.198+0000 7ff109600640 1 -- 172.21.15.97:0/3687197859 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff104004c50 con 0x557c9375f910 2024-03-27T10:56:18.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.198+0000 7ff109600640 1 -- 172.21.15.97:0/3687197859 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ff1040c7010 con 0x557c9375f910 2024-03-27T10:56:18.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.198+0000 7ff125259740 10 client.15297 fetch_fsmap finished waiting for FSMap version 27 2024-03-27T10:56:18.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.199+0000 7ff125259740 10 client.15297 resolve_mds: resolved name 'f' to daemon mds.f 2024-03-27T10:56:18.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.199+0000 7ff125259740 20 client.15297 populate_metadata read hostname 'smithi097' 2024-03-27T10:56:18.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.199+0000 7ff125259740 1 --2- 172.21.15.97:0/3687197859 >> [v2:172.21.15.177:6834/3349112132,v1:172.21.15.177:6835/3349112132] conn(0x557c937e0280 0x557c937def90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:18.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.199+0000 7ff125259740 4 client.15297 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-27T10:56:18.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.199+0000 7ff125259740 1 -- 172.21.15.97:0/3687197859 --> [v2:172.21.15.177:6834/3349112132,v1:172.21.15.177:6835/3349112132] -- command(tid 0: {"prefix": "status"}) -- 0x557c936cb820 con 0x557c937e0280 2024-03-27T10:56:18.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.199+0000 7ff11b400640 1 --2- 172.21.15.97:0/3687197859 >> [v2:172.21.15.177:6834/3349112132,v1:172.21.15.177:6835/3349112132] conn(0x557c937e0280 0x557c937def90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:18.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.200+0000 7ff11b400640 1 --2- 172.21.15.97:0/3687197859 >> [v2:172.21.15.177:6834/3349112132,v1:172.21.15.177:6835/3349112132] conn(0x557c937e0280 0x557c937def90 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=23cc6c8af4a32933 server_cookie=7c86405331bb1a9d in_seq=0 out_seq=0 2024-03-27T10:56:18.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.200+0000 7ff109600640 10 client.15297 ms_handle_connect on v2:172.21.15.177:6834/3349112132 2024-03-27T10:56:18.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.201+0000 7ff109600640 1 -- 172.21.15.97:0/3687197859 <== mds.3 v2:172.21.15.177:6834/3349112132 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x557c936cb820 con 0x557c937e0280 2024-03-27T10:56:18.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.201+0000 7ff109600640 10 client.15297 handle_command_reply: tid=0 2024-03-27T10:56:18.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.201+0000 7ff125259740 1 client.15297 shutdown 2024-03-27T10:56:18.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.201+0000 7ff125259740 1 -- 172.21.15.97:0/3687197859 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c9375f910 msgr2=0x557c937d7950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:18.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.201+0000 7ff125259740 1 --2- 172.21.15.97:0/3687197859 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c9375f910 0x557c937d7950 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7ff1040efd00 tx=0x7ff1040bab50 comp rx=0 tx=0).stop 2024-03-27T10:56:18.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.201+0000 7ff125259740 1 -- 172.21.15.97:0/3687197859 shutdown_connections 2024-03-27T10:56:18.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.201+0000 7ff125259740 1 --2- 172.21.15.97:0/3687197859 >> [v2:172.21.15.177:6834/3349112132,v1:172.21.15.177:6835/3349112132] conn(0x557c937e0280 0x557c937def90 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.201+0000 7ff125259740 1 --2- 172.21.15.97:0/3687197859 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557c93764a80 0x557c937d9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.201+0000 7ff125259740 1 --2- 172.21.15.97:0/3687197859 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c9375f910 0x557c937d7950 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.201+0000 7ff109600640 0 client.15297 ms_handle_reset on v2:172.21.15.177:6834/3349112132 2024-03-27T10:56:18.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.201+0000 7ff125259740 1 --2- 172.21.15.97:0/3687197859 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557c9375ef20 0x557c9376e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.201+0000 7ff125259740 1 -- 172.21.15.97:0/3687197859 >> 172.21.15.97:0/3687197859 conn(0x557c9375a490 msgr2=0x557c9375d6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:18.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.201+0000 7ff125259740 1 -- 172.21.15.97:0/3687197859 shutdown_connections 2024-03-27T10:56:18.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.201+0000 7ff125259740 1 -- 172.21.15.97:0/3687197859 wait complete. 2024-03-27T10:56:18.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.201+0000 7ff125259740 20 client.15297 trim_cache size 0 max 16384 2024-03-27T10:56:18.179 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-03-27T10:56:18.179 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "adca88c4-ec26-11ee-95d0-87774f69a715", 2024-03-27T10:56:18.179 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 3, 2024-03-27T10:56:18.179 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 24454, 2024-03-27T10:56:18.179 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2024-03-27T10:56:18.179 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2024-03-27T10:56:18.179 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2024-03-27T10:56:18.179 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 128.04030953899999, 2024-03-27T10:56:18.179 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 27, 2024-03-27T10:56:18.179 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 92, 2024-03-27T10:56:18.179 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 90, 2024-03-27T10:56:18.179 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 161.87778290899999 2024-03-27T10:56:18.180 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-03-27T10:56:18.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.202+0000 7ff122600640 1 -- 172.21.15.97:0/3150313474 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7ff1000a3690 msgr2=0x7ff1000a5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:18.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.202+0000 7ff122600640 1 --2- 172.21.15.97:0/3150313474 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7ff1000a3690 0x7ff1000a5b50 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x557c93760170 tx=0x7ff114008860 comp rx=0 tx=0).stop 2024-03-27T10:56:18.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.202+0000 7ff122600640 1 -- 172.21.15.97:0/3150313474 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff11c077750 msgr2=0x7ff11c19e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:18.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.202+0000 7ff122600640 1 --2- 172.21.15.97:0/3150313474 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff11c077750 0x7ff11c19e8e0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ff104000c00 tx=0x7ff1040042f0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.202+0000 7ff122600640 1 -- 172.21.15.97:0/3150313474 shutdown_connections 2024-03-27T10:56:18.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.202+0000 7ff122600640 1 --2- 172.21.15.97:0/3150313474 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7ff1000a3690 0x7ff1000a5b50 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.202+0000 7ff122600640 1 --2- 172.21.15.97:0/3150313474 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff11c0780f0 0x7ff11c1a5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.202+0000 7ff122600640 1 --2- 172.21.15.97:0/3150313474 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff11c077750 0x7ff11c19e8e0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.202+0000 7ff122600640 1 --2- 172.21.15.97:0/3150313474 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff11c076510 0x7ff11c19e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.202+0000 7ff122600640 1 -- 172.21.15.97:0/3150313474 >> 172.21.15.97:0/3150313474 conn(0x7ff11c0ff300 msgr2=0x7ff11c100a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:18.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.202+0000 7ff122600640 1 -- 172.21.15.97:0/3150313474 shutdown_connections 2024-03-27T10:56:18.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.202+0000 7ff122600640 1 -- 172.21.15.97:0/3150313474 wait complete. 2024-03-27T10:56:18.191 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2024-03-27T10:56:18.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.419+0000 7f8d0e600640 1 -- 172.21.15.97:0/3218644347 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d08102d50 msgr2=0x7f8d081031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:18.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.419+0000 7f8d0e600640 1 --2- 172.21.15.97:0/3218644347 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d08102d50 0x7f8d081031d0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f8cf0009900 tx=0x7f8cf002bc30 comp rx=0 tx=0).stop 2024-03-27T10:56:18.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.419+0000 7f8d0e600640 1 -- 172.21.15.97:0/3218644347 shutdown_connections 2024-03-27T10:56:18.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.419+0000 7f8d0e600640 1 --2- 172.21.15.97:0/3218644347 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8d08103710 0x7f8d08109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.419+0000 7f8d0e600640 1 --2- 172.21.15.97:0/3218644347 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d08102d50 0x7f8d081031d0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.419+0000 7f8d0e600640 1 --2- 172.21.15.97:0/3218644347 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8d08101b60 0x7f8d08101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.419+0000 7f8d0e600640 1 -- 172.21.15.97:0/3218644347 >> 172.21.15.97:0/3218644347 conn(0x7f8d080fd0f0 msgr2=0x7f8d080ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:18.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.419+0000 7f8d0e600640 1 -- 172.21.15.97:0/3218644347 shutdown_connections 2024-03-27T10:56:18.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.419+0000 7f8d0e600640 1 -- 172.21.15.97:0/3218644347 wait complete. 2024-03-27T10:56:18.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.420+0000 7f8d0e600640 1 Processor -- start 2024-03-27T10:56:18.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.420+0000 7f8d0e600640 1 -- start start 2024-03-27T10:56:18.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.421+0000 7f8d0e600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8d08101b60 0x7f8d08197d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:18.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.421+0000 7f8d0e600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d08102d50 0x7f8d08198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:18.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.421+0000 7f8d0e600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8d08103710 0x7f8d0819f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:18.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.421+0000 7f8d0e600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8d0810d6c0 con 0x7f8d08102d50 2024-03-27T10:56:18.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.421+0000 7f8d0e600640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f8d0810d540 con 0x7f8d08101b60 2024-03-27T10:56:18.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.421+0000 7f8d0e600640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f8d0810d840 con 0x7f8d08103710 2024-03-27T10:56:18.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.421+0000 7f8cfea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d08102d50 0x7f8d08198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:18.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.421+0000 7f8cfea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d08102d50 0x7f8d08198260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45532/0 (socket says 172.21.15.97:45532) 2024-03-27T10:56:18.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.421+0000 7f8cfea00640 1 -- 172.21.15.97:0/1147827399 learned_addr learned my addr 172.21.15.97:0/1147827399 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:56:18.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.421+0000 7f8d07400640 1 --2- 172.21.15.97:0/1147827399 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8d08101b60 0x7f8d08197d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:18.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.421+0000 7f8d07e00640 1 --2- 172.21.15.97:0/1147827399 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8d08103710 0x7f8d0819f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:18.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.421+0000 7f8cfea00640 1 -- 172.21.15.97:0/1147827399 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8d08103710 msgr2=0x7f8d0819f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:18.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.421+0000 7f8cfea00640 1 --2- 172.21.15.97:0/1147827399 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8d08103710 0x7f8d0819f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.421+0000 7f8cfea00640 1 -- 172.21.15.97:0/1147827399 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8d08101b60 msgr2=0x7f8d08197d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:18.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.421+0000 7f8cfea00640 1 --2- 172.21.15.97:0/1147827399 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8d08101b60 0x7f8d08197d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.421+0000 7f8cfea00640 1 -- 172.21.15.97:0/1147827399 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d0819f930 con 0x7f8d08102d50 2024-03-27T10:56:18.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.421+0000 7f8d07400640 1 --2- 172.21.15.97:0/1147827399 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8d08101b60 0x7f8d08197d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T10:56:18.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.422+0000 7f8cfea00640 1 --2- 172.21.15.97:0/1147827399 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d08102d50 0x7f8d08198260 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f8cf0000c00 tx=0x7f8cf00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:18.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.422+0000 7f8d04c00640 1 -- 172.21.15.97:0/1147827399 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cf0043020 con 0x7f8d08102d50 2024-03-27T10:56:18.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.422+0000 7f8d04c00640 1 -- 172.21.15.97:0/1147827399 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8cf0004aa0 con 0x7f8d08102d50 2024-03-27T10:56:18.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.422+0000 7f8d0e600640 1 -- 172.21.15.97:0/1147827399 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d0819fbc0 con 0x7f8d08102d50 2024-03-27T10:56:18.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.422+0000 7f8d04c00640 1 -- 172.21.15.97:0/1147827399 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cf0003d00 con 0x7f8d08102d50 2024-03-27T10:56:18.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.422+0000 7f8d0e600640 1 -- 172.21.15.97:0/1147827399 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d081a0110 con 0x7f8d08102d50 2024-03-27T10:56:18.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.425+0000 7f8d11363740 1 Processor -- start 2024-03-27T10:56:18.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.425+0000 7f8d11363740 1 -- start start 2024-03-27T10:56:18.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.425+0000 7f8d04c00640 1 -- 172.21.15.97:0/1147827399 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f8cf0046020 con 0x7f8d08102d50 2024-03-27T10:56:18.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.426+0000 7f8d11363740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556034f77f20 0x556034f78340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:18.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.426+0000 7f8d11363740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556034f78910 0x556034f7d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:18.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.426+0000 7f8d11363740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556034f7da80 0x556034f7ff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:18.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.426+0000 7f8d11363740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x556034f3a920 con 0x556034f77f20 2024-03-27T10:56:18.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.426+0000 7f8d11363740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x556034e8afa0 con 0x556034f7da80 2024-03-27T10:56:18.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.426+0000 7f8d11363740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x556034edcf90 con 0x556034f78910 2024-03-27T10:56:18.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.426+0000 7f8d07e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556034f77f20 0x556034f78340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:18.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.426+0000 7f8d07e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556034f77f20 0x556034f78340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45548/0 (socket says 172.21.15.97:45548) 2024-03-27T10:56:18.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.426+0000 7f8d07e00640 1 -- 172.21.15.97:0/1559495165 learned_addr learned my addr 172.21.15.97:0/1559495165 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:56:18.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.426+0000 7f8cfea00640 1 --2- 172.21.15.97:0/1559495165 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556034f7da80 0x556034f7ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:18.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.426+0000 7f8d07400640 1 --2- 172.21.15.97:0/1559495165 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556034f78910 0x556034f7d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:18.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.426+0000 7f8d07e00640 1 -- 172.21.15.97:0/1559495165 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556034f78910 msgr2=0x556034f7d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:56:18.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.426+0000 7f8d07e00640 1 --2- 172.21.15.97:0/1559495165 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556034f78910 0x556034f7d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.426+0000 7f8d07e00640 1 -- 172.21.15.97:0/1559495165 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556034f7da80 msgr2=0x556034f7ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:56:18.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.426+0000 7f8d07e00640 1 --2- 172.21.15.97:0/1559495165 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556034f7da80 0x556034f7ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.426+0000 7f8d07e00640 1 -- 172.21.15.97:0/1559495165 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556034bf12d0 con 0x556034f77f20 2024-03-27T10:56:18.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.426+0000 7f8cfea00640 1 --2- 172.21.15.97:0/1559495165 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556034f7da80 0x556034f7ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T10:56:18.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.427+0000 7f8d07e00640 1 --2- 172.21.15.97:0/1559495165 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556034f77f20 0x556034f78340 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f8d0000cbe0 tx=0x7f8d0000e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87cf230cc0d1ddf6 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:18.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.427+0000 7f8cfcc00640 1 -- 172.21.15.97:0/1559495165 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d00005220 con 0x556034f77f20 2024-03-27T10:56:18.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.427+0000 7f8cfcc00640 1 -- 172.21.15.97:0/1559495165 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8d00004c50 con 0x556034f77f20 2024-03-27T10:56:18.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.427+0000 7f8cfcc00640 1 -- 172.21.15.97:0/1559495165 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d00010750 con 0x556034f77f20 2024-03-27T10:56:18.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.427+0000 7f8d04c00640 1 --2- 172.21.15.97:0/1147827399 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f8cec0a3610 0x7f8cec0a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:18.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.427+0000 7f8d04c00640 1 -- 172.21.15.97:0/1147827399 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f8cf00f21a0 con 0x7f8d08102d50 2024-03-27T10:56:18.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.428+0000 7f8d07400640 1 --2- 172.21.15.97:0/1147827399 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f8cec0a3610 0x7f8cec0a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:18.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.428+0000 7f8d11363740 1 -- 172.21.15.97:0/1559495165 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556034f77f20 msgr2=0x556034f78340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:18.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.428+0000 7f8d11363740 1 --2- 172.21.15.97:0/1559495165 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556034f77f20 0x556034f78340 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f8d0000cbe0 tx=0x7f8d0000e720 comp rx=0 tx=0).stop 2024-03-27T10:56:18.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.428+0000 7f8d11363740 1 -- 172.21.15.97:0/1559495165 shutdown_connections 2024-03-27T10:56:18.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.428+0000 7f8d11363740 1 --2- 172.21.15.97:0/1559495165 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556034f7da80 0x556034f7ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.428+0000 7f8d11363740 1 --2- 172.21.15.97:0/1559495165 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556034f78910 0x556034f7d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.428+0000 7f8d11363740 1 --2- 172.21.15.97:0/1559495165 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556034f77f20 0x556034f78340 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.428+0000 7f8d11363740 1 -- 172.21.15.97:0/1559495165 >> 172.21.15.97:0/1559495165 conn(0x556034f73490 msgr2=0x556034f73890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:18.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.428+0000 7f8d11363740 1 -- 172.21.15.97:0/1559495165 shutdown_connections 2024-03-27T10:56:18.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.428+0000 7f8d11363740 1 -- 172.21.15.97:0/1559495165 wait complete. 2024-03-27T10:56:18.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.428+0000 7f8d07400640 1 --2- 172.21.15.97:0/1147827399 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f8cec0a3610 0x7f8cec0a5ad0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x556034f78d10 tx=0x7f8cf8009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:18.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.429+0000 7f8d11363740 1 -- start start 2024-03-27T10:56:18.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.429+0000 7f8d11363740 1 Processor -- start 2024-03-27T10:56:18.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.429+0000 7f8d11363740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556034f77f20 0x556034f82130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:18.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.429+0000 7f8d11363740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556034f78910 0x556034f82670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:18.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.429+0000 7f8d11363740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556034f7da80 0x556034f896b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:18.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.429+0000 7f8d11363740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x556034ed8f40 con 0x556034f78910 2024-03-27T10:56:18.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.429+0000 7f8d11363740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x556034e85650 con 0x556034f77f20 2024-03-27T10:56:18.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.429+0000 7f8d11363740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x556034e98a10 con 0x556034f7da80 2024-03-27T10:56:18.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.429+0000 7f8d11363740 10 client.0 _ll_register_callbacks cb 0x556034ed7ff0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:56:18.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.430+0000 7f8cfea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556034f78910 0x556034f82670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:18.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.430+0000 7f8cfea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556034f78910 0x556034f82670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45554/0 (socket says 172.21.15.97:45554) 2024-03-27T10:56:18.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.430+0000 7f8cfea00640 1 -- 172.21.15.97:0/2314535494 learned_addr learned my addr 172.21.15.97:0/2314535494 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:56:18.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.430+0000 7f8d07400640 1 --2- 172.21.15.97:0/2314535494 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556034f77f20 0x556034f82130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:18.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.430+0000 7f8d07e00640 1 --2- 172.21.15.97:0/2314535494 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556034f7da80 0x556034f896b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:18.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.430+0000 7f8cfea00640 1 -- 172.21.15.97:0/2314535494 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556034f7da80 msgr2=0x556034f896b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:56:18.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.430+0000 7f8cfea00640 1 --2- 172.21.15.97:0/2314535494 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556034f7da80 0x556034f896b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.430+0000 7f8cfea00640 1 -- 172.21.15.97:0/2314535494 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556034f77f20 msgr2=0x556034f82130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:56:18.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.430+0000 7f8cfea00640 1 --2- 172.21.15.97:0/2314535494 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556034f77f20 0x556034f82130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.430+0000 7f8cfea00640 1 -- 172.21.15.97:0/2314535494 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556034cbb6b0 con 0x556034f78910 2024-03-27T10:56:18.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.430+0000 7f8d07e00640 1 --2- 172.21.15.97:0/2314535494 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556034f7da80 0x556034f896b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:56:18.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.430+0000 7f8cfea00640 1 --2- 172.21.15.97:0/2314535494 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556034f78910 0x556034f82670 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f8cf00be610 tx=0x7f8cf004f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38fb83e1ad318314 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:18.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.430+0000 7f8cf5600640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T10:56:18.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.430+0000 7f8cf5600640 1 -- 172.21.15.97:0/2314535494 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cf00f1050 con 0x556034f78910 2024-03-27T10:56:18.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.430+0000 7f8cf5600640 1 -- 172.21.15.97:0/2314535494 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f8cf00beae0 con 0x556034f78910 2024-03-27T10:56:18.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.431+0000 7f8d11363740 1 -- 172.21.15.97:0/2314535494 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d00004c50 con 0x556034f78910 2024-03-27T10:56:18.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.431+0000 7f8cf5600640 1 -- 172.21.15.97:0/2314535494 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8cf00be450 con 0x556034f78910 2024-03-27T10:56:18.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.431+0000 7f8d11363740 10 client.15309 fetch_fsmap learned FSMap version 27 2024-03-27T10:56:18.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.431+0000 7f8d11363740 1 -- 172.21.15.97:0/2314535494 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556034f8ca20 con 0x556034f78910 2024-03-27T10:56:18.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.431+0000 7f8cf5600640 1 -- 172.21.15.97:0/2314535494 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8d00004c50 con 0x556034f78910 2024-03-27T10:56:18.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.431+0000 7f8cf5600640 1 -- 172.21.15.97:0/2314535494 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f8cf00dd240 con 0x556034f78910 2024-03-27T10:56:18.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.431+0000 7f8d11363740 10 client.15309 fetch_fsmap finished waiting for FSMap version 27 2024-03-27T10:56:18.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.431+0000 7f8d11363740 10 client.15309 resolve_mds: resolved name 'e' to daemon mds.e 2024-03-27T10:56:18.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.431+0000 7f8d11363740 20 client.15309 populate_metadata read hostname 'smithi097' 2024-03-27T10:56:18.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.431+0000 7f8d11363740 1 --2- 172.21.15.97:0/2314535494 >> [v2:172.21.15.114:6834/3581918222,v1:172.21.15.114:6835/3581918222] conn(0x556034f8f180 0x556034f905f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:18.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.431+0000 7f8d11363740 4 client.15309 mds_command: new command op to 34445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T10:56:18.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.431+0000 7f8d11363740 1 -- 172.21.15.97:0/2314535494 --> [v2:172.21.15.114:6834/3581918222,v1:172.21.15.114:6835/3581918222] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556034eea4d0 con 0x556034f8f180 2024-03-27T10:56:18.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.432+0000 7f8d07400640 1 --2- 172.21.15.97:0/2314535494 >> [v2:172.21.15.114:6834/3581918222,v1:172.21.15.114:6835/3581918222] conn(0x556034f8f180 0x556034f905f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:18.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.433+0000 7f8d07400640 1 --2- 172.21.15.97:0/2314535494 >> [v2:172.21.15.114:6834/3581918222,v1:172.21.15.114:6835/3581918222] conn(0x556034f8f180 0x556034f905f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bb9dcb41faa254b7 server_cookie=a843e0915f8c1859 in_seq=0 out_seq=0 2024-03-27T10:56:18.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.433+0000 7f8cf5600640 10 client.15309 ms_handle_connect on v2:172.21.15.114:6834/3581918222 2024-03-27T10:56:18.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.434+0000 7f8cf5600640 1 -- 172.21.15.97:0/2314535494 <== mds.2 v2:172.21.15.114:6834/3581918222 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x556034eea4d0 con 0x556034f8f180 2024-03-27T10:56:18.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.434+0000 7f8cf5600640 10 client.15309 handle_command_reply: tid=0 2024-03-27T10:56:18.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.435+0000 7f8d11363740 1 client.15309 shutdown 2024-03-27T10:56:18.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.435+0000 7f8d11363740 1 -- 172.21.15.97:0/2314535494 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556034f78910 msgr2=0x556034f82670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:18.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.435+0000 7f8d11363740 1 --2- 172.21.15.97:0/2314535494 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556034f78910 0x556034f82670 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f8cf00be610 tx=0x7f8cf004f000 comp rx=0 tx=0).stop 2024-03-27T10:56:18.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.435+0000 7f8d11363740 1 -- 172.21.15.97:0/2314535494 shutdown_connections 2024-03-27T10:56:18.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.435+0000 7f8d11363740 1 --2- 172.21.15.97:0/2314535494 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556034f7da80 0x556034f896b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.435+0000 7f8d11363740 1 --2- 172.21.15.97:0/2314535494 >> [v2:172.21.15.114:6834/3581918222,v1:172.21.15.114:6835/3581918222] conn(0x556034f8f180 0x556034f905f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.435+0000 7f8d11363740 1 --2- 172.21.15.97:0/2314535494 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556034f78910 0x556034f82670 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.435+0000 7f8d11363740 1 --2- 172.21.15.97:0/2314535494 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556034f77f20 0x556034f82130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.435+0000 7f8cf5600640 0 client.15309 ms_handle_reset on v2:172.21.15.114:6834/3581918222 2024-03-27T10:56:18.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.435+0000 7f8d11363740 1 -- 172.21.15.97:0/2314535494 >> 172.21.15.97:0/2314535494 conn(0x556034f73490 msgr2=0x556034f72690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:18.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.435+0000 7f8d11363740 1 -- 172.21.15.97:0/2314535494 shutdown_connections 2024-03-27T10:56:18.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.435+0000 7f8d11363740 1 -- 172.21.15.97:0/2314535494 wait complete. 2024-03-27T10:56:18.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.435+0000 7f8d11363740 20 client.15309 trim_cache size 0 max 16384 2024-03-27T10:56:18.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.454+0000 7f8d11363740 1 Processor -- start 2024-03-27T10:56:18.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.454+0000 7f8d11363740 1 -- start start 2024-03-27T10:56:18.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.454+0000 7f8d11363740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556034f77f20 0x556034f8dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:18.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.454+0000 7f8d11363740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556034f78910 0x556034f89670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:18.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.454+0000 7f8d11363740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556034f7da80 0x556034f8e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:18.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.454+0000 7f8d11363740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x556034f8e930 con 0x556034f7da80 2024-03-27T10:56:18.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.454+0000 7f8d11363740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x556034f8f000 con 0x556034f77f20 2024-03-27T10:56:18.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.454+0000 7f8d11363740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x556034f8f6d0 con 0x556034f78910 2024-03-27T10:56:18.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.455+0000 7f8d07e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556034f7da80 0x556034f8e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:18.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.455+0000 7f8d07400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556034f77f20 0x556034f8dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:18.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.455+0000 7f8d07e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556034f7da80 0x556034f8e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45560/0 (socket says 172.21.15.97:45560) 2024-03-27T10:56:18.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.455+0000 7f8d07e00640 1 -- 172.21.15.97:0/208812481 learned_addr learned my addr 172.21.15.97:0/208812481 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:56:18.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.455+0000 7f8cfea00640 1 --2- 172.21.15.97:0/208812481 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556034f78910 0x556034f89670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:18.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.455+0000 7f8d07e00640 1 -- 172.21.15.97:0/208812481 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556034f78910 msgr2=0x556034f89670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:56:18.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.455+0000 7f8d07e00640 1 --2- 172.21.15.97:0/208812481 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556034f78910 0x556034f89670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.455+0000 7f8d07e00640 1 -- 172.21.15.97:0/208812481 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556034f77f20 msgr2=0x556034f8dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:56:18.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.455+0000 7f8d07e00640 1 --2- 172.21.15.97:0/208812481 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556034f77f20 0x556034f8dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.455+0000 7f8d07e00640 1 -- 172.21.15.97:0/208812481 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556034f791a0 con 0x556034f7da80 2024-03-27T10:56:18.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.455+0000 7f8d07e00640 1 --2- 172.21.15.97:0/208812481 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556034f7da80 0x556034f8e2b0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f8d00000c00 tx=0x7f8d0000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dcb9a00eca9b990 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:18.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.456+0000 7f8cf7e00640 1 -- 172.21.15.97:0/208812481 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d000050c0 con 0x556034f7da80 2024-03-27T10:56:18.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.456+0000 7f8cf7e00640 1 -- 172.21.15.97:0/208812481 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8d000035f0 con 0x556034f7da80 2024-03-27T10:56:18.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.456+0000 7f8cf7e00640 1 -- 172.21.15.97:0/208812481 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d00010700 con 0x556034f7da80 2024-03-27T10:56:18.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.456+0000 7f8d11363740 1 -- 172.21.15.97:0/208812481 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556034f7da80 msgr2=0x556034f8e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:18.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.456+0000 7f8d11363740 1 --2- 172.21.15.97:0/208812481 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556034f7da80 0x556034f8e2b0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f8d00000c00 tx=0x7f8d0000c880 comp rx=0 tx=0).stop 2024-03-27T10:56:18.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.456+0000 7f8d11363740 1 -- 172.21.15.97:0/208812481 shutdown_connections 2024-03-27T10:56:18.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.456+0000 7f8d11363740 1 --2- 172.21.15.97:0/208812481 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556034f7da80 0x556034f8e2b0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.456+0000 7f8d11363740 1 --2- 172.21.15.97:0/208812481 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556034f78910 0x556034f89670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.456+0000 7f8d11363740 1 --2- 172.21.15.97:0/208812481 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556034f77f20 0x556034f8dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.456+0000 7f8d11363740 1 -- 172.21.15.97:0/208812481 >> 172.21.15.97:0/208812481 conn(0x556034f73490 msgr2=0x556034f73870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:18.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.456+0000 7f8d11363740 1 -- 172.21.15.97:0/208812481 shutdown_connections 2024-03-27T10:56:18.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.457+0000 7f8d11363740 1 -- 172.21.15.97:0/208812481 wait complete. 2024-03-27T10:56:18.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.457+0000 7f8d11363740 1 -- start start 2024-03-27T10:56:18.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.457+0000 7f8d11363740 1 Processor -- start 2024-03-27T10:56:18.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.457+0000 7f8d11363740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556034f77f20 0x556034f84e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:18.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.457+0000 7f8d11363740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556034f78910 0x556034f853a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:18.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.457+0000 7f8d11363740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556034f7da80 0x556034f883c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:18.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.457+0000 7f8d11363740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x556034f90470 con 0x556034f78910 2024-03-27T10:56:18.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.457+0000 7f8d11363740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x556034f8fda0 con 0x556034f77f20 2024-03-27T10:56:18.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.457+0000 7f8d11363740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x556034f90b40 con 0x556034f7da80 2024-03-27T10:56:18.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.457+0000 7f8d11363740 10 client.0 _ll_register_callbacks cb 0x7f8cf0003ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:56:18.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.458+0000 7f8cfea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556034f78910 0x556034f853a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:18.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.458+0000 7f8cfea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556034f78910 0x556034f853a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45570/0 (socket says 172.21.15.97:45570) 2024-03-27T10:56:18.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.458+0000 7f8cfea00640 1 -- 172.21.15.97:0/919758731 learned_addr learned my addr 172.21.15.97:0/919758731 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:56:18.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.458+0000 7f8d07400640 1 --2- 172.21.15.97:0/919758731 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556034f77f20 0x556034f84e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:18.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.458+0000 7f8d07e00640 1 --2- 172.21.15.97:0/919758731 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556034f7da80 0x556034f883c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:18.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.458+0000 7f8cfea00640 1 -- 172.21.15.97:0/919758731 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556034f7da80 msgr2=0x556034f883c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:56:18.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.458+0000 7f8cfea00640 1 --2- 172.21.15.97:0/919758731 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556034f7da80 0x556034f883c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.458+0000 7f8cfea00640 1 -- 172.21.15.97:0/919758731 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556034f77f20 msgr2=0x556034f84e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:56:18.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.458+0000 7f8cfea00640 1 --2- 172.21.15.97:0/919758731 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556034f77f20 0x556034f84e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.458+0000 7f8cfea00640 1 -- 172.21.15.97:0/919758731 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556034f889b0 con 0x556034f78910 2024-03-27T10:56:18.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.458+0000 7f8d07e00640 1 --2- 172.21.15.97:0/919758731 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556034f7da80 0x556034f883c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:56:18.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.458+0000 7f8cfea00640 1 --2- 172.21.15.97:0/919758731 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556034f78910 0x556034f853a0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f8cf00bafc0 tx=0x7f8cf00dd4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3038708ad1e85df1 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:18.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.458+0000 7f8cf5600640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T10:56:18.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.458+0000 7f8cf5600640 1 -- 172.21.15.97:0/919758731 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cf00f1040 con 0x556034f78910 2024-03-27T10:56:18.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.459+0000 7f8cf5600640 1 -- 172.21.15.97:0/919758731 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f8cf00a7b80 con 0x556034f78910 2024-03-27T10:56:18.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.459+0000 7f8d11363740 1 -- 172.21.15.97:0/919758731 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d000035f0 con 0x556034f78910 2024-03-27T10:56:18.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.459+0000 7f8cf5600640 1 -- 172.21.15.97:0/919758731 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8cf00044e0 con 0x556034f78910 2024-03-27T10:56:18.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.459+0000 7f8d11363740 10 client.15315 fetch_fsmap learned FSMap version 27 2024-03-27T10:56:18.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.459+0000 7f8d11363740 1 -- 172.21.15.97:0/919758731 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556034ffd880 con 0x556034f78910 2024-03-27T10:56:18.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.459+0000 7f8cf5600640 1 -- 172.21.15.97:0/919758731 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8d000035f0 con 0x556034f78910 2024-03-27T10:56:18.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.459+0000 7f8cf5600640 1 -- 172.21.15.97:0/919758731 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f8cf00890f0 con 0x556034f78910 2024-03-27T10:56:18.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.459+0000 7f8d11363740 10 client.15315 fetch_fsmap finished waiting for FSMap version 27 2024-03-27T10:56:18.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.459+0000 7f8d11363740 10 client.15315 resolve_mds: resolved name 'e' to daemon mds.e 2024-03-27T10:56:18.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.459+0000 7f8d11363740 20 client.15315 populate_metadata read hostname 'smithi097' 2024-03-27T10:56:18.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.459+0000 7f8d11363740 1 --2- 172.21.15.97:0/919758731 >> [v2:172.21.15.114:6834/3581918222,v1:172.21.15.114:6835/3581918222] conn(0x5560350022b0 0x556034ffe6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:18.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.459+0000 7f8d11363740 4 client.15315 mds_command: new command op to 34445 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-27T10:56:18.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.459+0000 7f8d11363740 1 -- 172.21.15.97:0/919758731 --> [v2:172.21.15.114:6834/3581918222,v1:172.21.15.114:6835/3581918222] -- command(tid 0: {"prefix": "status"}) -- 0x556034ee4820 con 0x5560350022b0 2024-03-27T10:56:18.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.460+0000 7f8d07400640 1 --2- 172.21.15.97:0/919758731 >> [v2:172.21.15.114:6834/3581918222,v1:172.21.15.114:6835/3581918222] conn(0x5560350022b0 0x556034ffe6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:18.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.461+0000 7f8d07400640 1 --2- 172.21.15.97:0/919758731 >> [v2:172.21.15.114:6834/3581918222,v1:172.21.15.114:6835/3581918222] conn(0x5560350022b0 0x556034ffe6f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=17d8578419523244 server_cookie=bae68189faeee37 in_seq=0 out_seq=0 2024-03-27T10:56:18.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.461+0000 7f8cf5600640 10 client.15315 ms_handle_connect on v2:172.21.15.114:6834/3581918222 2024-03-27T10:56:18.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.461+0000 7f8cf5600640 1 -- 172.21.15.97:0/919758731 <== mds.2 v2:172.21.15.114:6834/3581918222 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x556034ee4820 con 0x5560350022b0 2024-03-27T10:56:18.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.461+0000 7f8cf5600640 10 client.15315 handle_command_reply: tid=0 2024-03-27T10:56:18.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.461+0000 7f8d11363740 1 client.15315 shutdown 2024-03-27T10:56:18.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.461+0000 7f8d11363740 1 -- 172.21.15.97:0/919758731 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556034f78910 msgr2=0x556034f853a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:18.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.461+0000 7f8d11363740 1 --2- 172.21.15.97:0/919758731 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556034f78910 0x556034f853a0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f8cf00bafc0 tx=0x7f8cf00dd4f0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.461+0000 7f8d11363740 1 -- 172.21.15.97:0/919758731 shutdown_connections 2024-03-27T10:56:18.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.461+0000 7f8d11363740 1 --2- 172.21.15.97:0/919758731 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556034f7da80 0x556034f883c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.461+0000 7f8d11363740 1 --2- 172.21.15.97:0/919758731 >> [v2:172.21.15.114:6834/3581918222,v1:172.21.15.114:6835/3581918222] conn(0x5560350022b0 0x556034ffe6f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.461+0000 7f8d11363740 1 --2- 172.21.15.97:0/919758731 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556034f78910 0x556034f853a0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.461+0000 7f8d11363740 1 --2- 172.21.15.97:0/919758731 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556034f77f20 0x556034f84e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.461+0000 7f8cf5600640 0 client.15315 ms_handle_reset on v2:172.21.15.114:6834/3581918222 2024-03-27T10:56:18.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.461+0000 7f8d11363740 1 -- 172.21.15.97:0/919758731 >> 172.21.15.97:0/919758731 conn(0x556034f73490 msgr2=0x556034f79dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:18.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.462+0000 7f8d11363740 1 -- 172.21.15.97:0/919758731 shutdown_connections 2024-03-27T10:56:18.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.462+0000 7f8d11363740 1 -- 172.21.15.97:0/919758731 wait complete. 2024-03-27T10:56:18.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.462+0000 7f8d11363740 20 client.15315 trim_cache size 0 max 16384 2024-03-27T10:56:18.439 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-03-27T10:56:18.439 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "adca88c4-ec26-11ee-95d0-87774f69a715", 2024-03-27T10:56:18.439 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 2, 2024-03-27T10:56:18.439 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 34445, 2024-03-27T10:56:18.439 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2024-03-27T10:56:18.439 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2024-03-27T10:56:18.440 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2024-03-27T10:56:18.440 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 128.858841149, 2024-03-27T10:56:18.440 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 27, 2024-03-27T10:56:18.440 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 92, 2024-03-27T10:56:18.440 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 90, 2024-03-27T10:56:18.440 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 165.824784486 2024-03-27T10:56:18.440 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-03-27T10:56:18.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.462+0000 7f8d0e600640 1 -- 172.21.15.97:0/1147827399 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f8cec0a3610 msgr2=0x7f8cec0a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:18.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.462+0000 7f8d0e600640 1 --2- 172.21.15.97:0/1147827399 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f8cec0a3610 0x7f8cec0a5ad0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x556034f78d10 tx=0x7f8cf8009500 comp rx=0 tx=0).stop 2024-03-27T10:56:18.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.462+0000 7f8d0e600640 1 -- 172.21.15.97:0/1147827399 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d08102d50 msgr2=0x7f8d08198260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:18.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.462+0000 7f8d0e600640 1 --2- 172.21.15.97:0/1147827399 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d08102d50 0x7f8d08198260 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f8cf0000c00 tx=0x7f8cf00042f0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.462+0000 7f8d0e600640 1 -- 172.21.15.97:0/1147827399 shutdown_connections 2024-03-27T10:56:18.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.462+0000 7f8d0e600640 1 --2- 172.21.15.97:0/1147827399 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8d08103710 0x7f8d0819f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.462+0000 7f8d0e600640 1 --2- 172.21.15.97:0/1147827399 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d08102d50 0x7f8d08198260 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.462+0000 7f8d0e600640 1 --2- 172.21.15.97:0/1147827399 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f8cec0a3610 0x7f8cec0a5ad0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.462+0000 7f8d0e600640 1 --2- 172.21.15.97:0/1147827399 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8d08101b60 0x7f8d08197d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.462+0000 7f8d0e600640 1 -- 172.21.15.97:0/1147827399 >> 172.21.15.97:0/1147827399 conn(0x7f8d080fd0f0 msgr2=0x7f8d08106050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:18.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.463+0000 7f8d0e600640 1 -- 172.21.15.97:0/1147827399 shutdown_connections 2024-03-27T10:56:18.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.463+0000 7f8d0e600640 1 -- 172.21.15.97:0/1147827399 wait complete. 2024-03-27T10:56:18.448 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:56:18 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/3817441582' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T10:56:18.452 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-03-27T10:56:18.516 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:56:18 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/3817441582' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T10:56:18.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.683+0000 7fdafca00640 1 -- 172.21.15.97:0/3495849961 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdaf80780f0 msgr2=0x7fdaf8078990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:18.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.683+0000 7fdafca00640 1 --2- 172.21.15.97:0/3495849961 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdaf80780f0 0x7fdaf8078990 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fdaf0009900 tx=0x7fdaf002bc30 comp rx=0 tx=0).stop 2024-03-27T10:56:18.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.683+0000 7fdafca00640 1 -- 172.21.15.97:0/3495849961 shutdown_connections 2024-03-27T10:56:18.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.683+0000 7fdafca00640 1 --2- 172.21.15.97:0/3495849961 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdaf80780f0 0x7fdaf8078990 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.683+0000 7fdafca00640 1 --2- 172.21.15.97:0/3495849961 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdaf8077750 0x7fdaf8077bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.683+0000 7fdafca00640 1 --2- 172.21.15.97:0/3495849961 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdaf8076510 0x7fdaf8076930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.683+0000 7fdafca00640 1 -- 172.21.15.97:0/3495849961 >> 172.21.15.97:0/3495849961 conn(0x7fdaf80ff300 msgr2=0x7fdaf8101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:18.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.683+0000 7fdafca00640 1 -- 172.21.15.97:0/3495849961 shutdown_connections 2024-03-27T10:56:18.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.683+0000 7fdafca00640 1 -- 172.21.15.97:0/3495849961 wait complete. 2024-03-27T10:56:18.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.684+0000 7fdafca00640 1 Processor -- start 2024-03-27T10:56:18.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.684+0000 7fdafca00640 1 -- start start 2024-03-27T10:56:18.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.684+0000 7fdafca00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdaf8076510 0x7fdaf819e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:18.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.684+0000 7fdafca00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdaf8077750 0x7fdaf819e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:18.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.684+0000 7fdafca00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdaf80780f0 0x7fdaf81a58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:18.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.684+0000 7fdafca00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf8113d10 con 0x7fdaf80780f0 2024-03-27T10:56:18.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.684+0000 7fdafca00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf8113b90 con 0x7fdaf8077750 2024-03-27T10:56:18.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.684+0000 7fdafca00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf8113e90 con 0x7fdaf8076510 2024-03-27T10:56:18.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.684+0000 7fdaf6800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdaf80780f0 0x7fdaf81a58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:18.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.684+0000 7fdaf6800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdaf80780f0 0x7fdaf81a58e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45590/0 (socket says 172.21.15.97:45590) 2024-03-27T10:56:18.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.685+0000 7fdaf6800640 1 -- 172.21.15.97:0/176215569 learned_addr learned my addr 172.21.15.97:0/176215569 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:56:18.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.685+0000 7fdaed400640 1 --2- 172.21.15.97:0/176215569 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdaf8077750 0x7fdaf819e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:18.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.685+0000 7fdaf5e00640 1 --2- 172.21.15.97:0/176215569 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdaf8076510 0x7fdaf819e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:18.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.685+0000 7fdaf6800640 1 -- 172.21.15.97:0/176215569 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdaf8076510 msgr2=0x7fdaf819e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:18.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.685+0000 7fdaf6800640 1 --2- 172.21.15.97:0/176215569 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdaf8076510 0x7fdaf819e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.685+0000 7fdaf6800640 1 -- 172.21.15.97:0/176215569 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdaf8077750 msgr2=0x7fdaf819e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:18.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.685+0000 7fdaf6800640 1 --2- 172.21.15.97:0/176215569 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdaf8077750 0x7fdaf819e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.685+0000 7fdaf6800640 1 -- 172.21.15.97:0/176215569 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaf81a5f50 con 0x7fdaf80780f0 2024-03-27T10:56:18.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.685+0000 7fdaed400640 1 --2- 172.21.15.97:0/176215569 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdaf8077750 0x7fdaf819e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T10:56:18.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.685+0000 7fdaf6800640 1 --2- 172.21.15.97:0/176215569 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdaf80780f0 0x7fdaf81a58e0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fdaf0036040 tx=0x7fdaf002cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:18.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.686+0000 7fdaef400640 1 -- 172.21.15.97:0/176215569 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf0043020 con 0x7fdaf80780f0 2024-03-27T10:56:18.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.686+0000 7fdafca00640 1 -- 172.21.15.97:0/176215569 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaf81a61e0 con 0x7fdaf80780f0 2024-03-27T10:56:18.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.686+0000 7fdaef400640 1 -- 172.21.15.97:0/176215569 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdaf00044c0 con 0x7fdaf80780f0 2024-03-27T10:56:18.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.686+0000 7fdaef400640 1 -- 172.21.15.97:0/176215569 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf0003bf0 con 0x7fdaf80780f0 2024-03-27T10:56:18.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.686+0000 7fdafca00640 1 -- 172.21.15.97:0/176215569 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaf81a6730 con 0x7fdaf80780f0 2024-03-27T10:56:18.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.690+0000 7fdaff7ec740 1 Processor -- start 2024-03-27T10:56:18.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.690+0000 7fdaef400640 1 -- 172.21.15.97:0/176215569 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7fdaf0046020 con 0x7fdaf80780f0 2024-03-27T10:56:18.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.690+0000 7fdaff7ec740 1 -- start start 2024-03-27T10:56:18.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.690+0000 7fdaff7ec740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bc23759f20 0x55bc2375a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:18.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.690+0000 7fdaff7ec740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bc2375a910 0x55bc2375f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:18.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.690+0000 7fdaff7ec740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bc2375fa80 0x55bc23761f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:18.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.691+0000 7fdaff7ec740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55bc2371c920 con 0x55bc2375fa80 2024-03-27T10:56:18.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.691+0000 7fdaff7ec740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55bc2366cfa0 con 0x55bc2375a910 2024-03-27T10:56:18.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.691+0000 7fdaff7ec740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55bc236bef90 con 0x55bc23759f20 2024-03-27T10:56:18.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.691+0000 7fdaed400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bc2375fa80 0x55bc23761f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:18.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.691+0000 7fdaf5e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bc2375a910 0x55bc2375f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:18.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.691+0000 7fdaed400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bc2375fa80 0x55bc23761f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45602/0 (socket says 172.21.15.97:45602) 2024-03-27T10:56:18.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.691+0000 7fdaed400640 1 -- 172.21.15.97:0/1466013953 learned_addr learned my addr 172.21.15.97:0/1466013953 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:56:18.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.691+0000 7fdaed400640 1 --2- 172.21.15.97:0/1466013953 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bc23759f20 0x55bc2375a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:18.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.691+0000 7fdaed400640 1 -- 172.21.15.97:0/1466013953 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bc23759f20 msgr2=0x55bc2375a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:56:18.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.691+0000 7fdaed400640 1 --2- 172.21.15.97:0/1466013953 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bc23759f20 0x55bc2375a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.691+0000 7fdaed400640 1 -- 172.21.15.97:0/1466013953 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bc2375a910 msgr2=0x55bc2375f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:56:18.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.691+0000 7fdaed400640 1 --2- 172.21.15.97:0/1466013953 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bc2375a910 0x55bc2375f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.691+0000 7fdaed400640 1 -- 172.21.15.97:0/1466013953 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc233d32d0 con 0x55bc2375fa80 2024-03-27T10:56:18.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.692+0000 7fdaed400640 1 --2- 172.21.15.97:0/1466013953 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bc2375fa80 0x55bc23761f10 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fdae0009560 tx=0x7fdae0009760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=957a057ba6162d11 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:18.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.692+0000 7fdaef400640 1 --2- 172.21.15.97:0/176215569 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fdadc0a3610 0x7fdadc0a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:18.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.692+0000 7fdae7000640 1 -- 172.21.15.97:0/1466013953 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae0010710 con 0x55bc2375fa80 2024-03-27T10:56:18.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.692+0000 7fdae7000640 1 -- 172.21.15.97:0/1466013953 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdae0010d60 con 0x55bc2375fa80 2024-03-27T10:56:18.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.692+0000 7fdae7000640 1 -- 172.21.15.97:0/1466013953 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae001dd30 con 0x55bc2375fa80 2024-03-27T10:56:18.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.692+0000 7fdaef400640 1 -- 172.21.15.97:0/176215569 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fdaf00f2120 con 0x7fdaf80780f0 2024-03-27T10:56:18.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.692+0000 7fdaf5e00640 1 --2- 172.21.15.97:0/176215569 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fdadc0a3610 0x7fdadc0a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:18.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.693+0000 7fdaff7ec740 1 -- 172.21.15.97:0/1466013953 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bc2375fa80 msgr2=0x55bc23761f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:18.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.693+0000 7fdaff7ec740 1 --2- 172.21.15.97:0/1466013953 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bc2375fa80 0x55bc23761f10 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fdae0009560 tx=0x7fdae0009760 comp rx=0 tx=0).stop 2024-03-27T10:56:18.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.693+0000 7fdaf5e00640 1 --2- 172.21.15.97:0/176215569 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fdadc0a3610 0x7fdadc0a5ad0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x55bc2375ad10 tx=0x7fdae80097c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:18.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.693+0000 7fdaff7ec740 1 -- 172.21.15.97:0/1466013953 shutdown_connections 2024-03-27T10:56:18.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.693+0000 7fdaff7ec740 1 --2- 172.21.15.97:0/1466013953 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bc2375fa80 0x55bc23761f10 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.693+0000 7fdaff7ec740 1 --2- 172.21.15.97:0/1466013953 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bc2375a910 0x55bc2375f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.693+0000 7fdaff7ec740 1 --2- 172.21.15.97:0/1466013953 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bc23759f20 0x55bc2375a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.693+0000 7fdaff7ec740 1 -- 172.21.15.97:0/1466013953 >> 172.21.15.97:0/1466013953 conn(0x55bc23755490 msgr2=0x55bc23755890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:18.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.693+0000 7fdaff7ec740 1 -- 172.21.15.97:0/1466013953 shutdown_connections 2024-03-27T10:56:18.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.693+0000 7fdaff7ec740 1 -- 172.21.15.97:0/1466013953 wait complete. 2024-03-27T10:56:18.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.693+0000 7fdaff7ec740 1 -- start start 2024-03-27T10:56:18.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.694+0000 7fdaff7ec740 1 Processor -- start 2024-03-27T10:56:18.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.694+0000 7fdaff7ec740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bc23759f20 0x55bc23764140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:18.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.694+0000 7fdaff7ec740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bc2375a910 0x55bc23764680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:18.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.694+0000 7fdaff7ec740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bc2375fa80 0x55bc2376b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:18.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.694+0000 7fdaff7ec740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55bc236baf40 con 0x55bc2375a910 2024-03-27T10:56:18.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.694+0000 7fdaff7ec740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55bc23667650 con 0x55bc2375fa80 2024-03-27T10:56:18.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.694+0000 7fdaff7ec740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55bc2367aa10 con 0x55bc23759f20 2024-03-27T10:56:18.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.694+0000 7fdaff7ec740 10 client.0 _ll_register_callbacks cb 0x55bc236b9ff0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:56:18.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.694+0000 7fdaed400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bc2375a910 0x55bc23764680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:18.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.694+0000 7fdaed400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bc2375a910 0x55bc23764680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45606/0 (socket says 172.21.15.97:45606) 2024-03-27T10:56:18.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.694+0000 7fdaed400640 1 -- 172.21.15.97:0/2216553891 learned_addr learned my addr 172.21.15.97:0/2216553891 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:56:18.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.695+0000 7fdaf6800640 1 --2- 172.21.15.97:0/2216553891 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bc2375fa80 0x55bc2376b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:18.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.695+0000 7fdaed400640 1 -- 172.21.15.97:0/2216553891 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bc23759f20 msgr2=0x55bc23764140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:56:18.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.695+0000 7fdaf5e00640 1 --2- 172.21.15.97:0/2216553891 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bc23759f20 0x55bc23764140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:18.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.695+0000 7fdaed400640 1 --2- 172.21.15.97:0/2216553891 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bc23759f20 0x55bc23764140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.695+0000 7fdaed400640 1 -- 172.21.15.97:0/2216553891 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bc2375fa80 msgr2=0x55bc2376b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:56:18.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.695+0000 7fdaed400640 1 --2- 172.21.15.97:0/2216553891 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bc2375fa80 0x55bc2376b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.695+0000 7fdaed400640 1 -- 172.21.15.97:0/2216553891 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc2349d6b0 con 0x55bc2375a910 2024-03-27T10:56:18.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.695+0000 7fdaf6800640 1 --2- 172.21.15.97:0/2216553891 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bc2375fa80 0x55bc2376b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:56:18.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.695+0000 7fdaed400640 1 --2- 172.21.15.97:0/2216553891 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bc2375a910 0x55bc23764680 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fdae001d8a0 tx=0x7fdae001dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3575ea64cb3f8e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:18.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.695+0000 7fdadb400640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T10:56:18.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.695+0000 7fdadb400640 1 -- 172.21.15.97:0/2216553891 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae0012240 con 0x55bc2375a910 2024-03-27T10:56:18.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.695+0000 7fdadb400640 1 -- 172.21.15.97:0/2216553891 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fdae00123e0 con 0x55bc2375a910 2024-03-27T10:56:18.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.695+0000 7fdaff7ec740 1 -- 172.21.15.97:0/2216553891 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdae0010d60 con 0x55bc2375a910 2024-03-27T10:56:18.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.696+0000 7fdadb400640 1 -- 172.21.15.97:0/2216553891 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdae0005040 con 0x55bc2375a910 2024-03-27T10:56:18.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.696+0000 7fdaff7ec740 10 client.15327 fetch_fsmap learned FSMap version 27 2024-03-27T10:56:18.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.696+0000 7fdaff7ec740 1 -- 172.21.15.97:0/2216553891 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55bc2376ea30 con 0x55bc2375a910 2024-03-27T10:56:18.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.696+0000 7fdadb400640 1 -- 172.21.15.97:0/2216553891 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdae0010d60 con 0x55bc2375a910 2024-03-27T10:56:18.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.696+0000 7fdadb400640 1 -- 172.21.15.97:0/2216553891 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fdae001c070 con 0x55bc2375a910 2024-03-27T10:56:18.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.696+0000 7fdaff7ec740 10 client.15327 fetch_fsmap finished waiting for FSMap version 27 2024-03-27T10:56:18.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.696+0000 7fdaff7ec740 10 client.15327 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-27T10:56:18.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.696+0000 7fdaff7ec740 20 client.15327 populate_metadata read hostname 'smithi097' 2024-03-27T10:56:18.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.696+0000 7fdaff7ec740 1 --2- 172.21.15.97:0/2216553891 >> [v2:172.21.15.114:6836/616775588,v1:172.21.15.114:6837/616775588] conn(0x55bc23770e20 0x55bc23772290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:18.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.696+0000 7fdaff7ec740 4 client.15327 mds_command: new command op to 34478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T10:56:18.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.696+0000 7fdaff7ec740 1 -- 172.21.15.97:0/2216553891 --> [v2:172.21.15.114:6836/616775588,v1:172.21.15.114:6837/616775588] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc236cc4d0 con 0x55bc23770e20 2024-03-27T10:56:18.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.697+0000 7fdaf5e00640 1 --2- 172.21.15.97:0/2216553891 >> [v2:172.21.15.114:6836/616775588,v1:172.21.15.114:6837/616775588] conn(0x55bc23770e20 0x55bc23772290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:18.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.698+0000 7fdaf5e00640 1 --2- 172.21.15.97:0/2216553891 >> [v2:172.21.15.114:6836/616775588,v1:172.21.15.114:6837/616775588] conn(0x55bc23770e20 0x55bc23772290 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1016a502bcfc4c60 server_cookie=e02e974ba4e7166a in_seq=0 out_seq=0 2024-03-27T10:56:18.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.698+0000 7fdadb400640 10 client.15327 ms_handle_connect on v2:172.21.15.114:6836/616775588 2024-03-27T10:56:18.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.699+0000 7fdadb400640 1 -- 172.21.15.97:0/2216553891 <== mds.1 v2:172.21.15.114:6836/616775588 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55bc236cc4d0 con 0x55bc23770e20 2024-03-27T10:56:18.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.699+0000 7fdadb400640 10 client.15327 handle_command_reply: tid=0 2024-03-27T10:56:18.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.699+0000 7fdaff7ec740 1 client.15327 shutdown 2024-03-27T10:56:18.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.699+0000 7fdaff7ec740 1 -- 172.21.15.97:0/2216553891 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bc2375a910 msgr2=0x55bc23764680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:18.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.699+0000 7fdaff7ec740 1 --2- 172.21.15.97:0/2216553891 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bc2375a910 0x55bc23764680 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fdae001d8a0 tx=0x7fdae001dd40 comp rx=0 tx=0).stop 2024-03-27T10:56:18.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.699+0000 7fdaff7ec740 1 -- 172.21.15.97:0/2216553891 shutdown_connections 2024-03-27T10:56:18.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.699+0000 7fdaff7ec740 1 --2- 172.21.15.97:0/2216553891 >> [v2:172.21.15.114:6836/616775588,v1:172.21.15.114:6837/616775588] conn(0x55bc23770e20 0x55bc23772290 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.699+0000 7fdaff7ec740 1 --2- 172.21.15.97:0/2216553891 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bc2375fa80 0x55bc2376b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.700+0000 7fdaff7ec740 1 --2- 172.21.15.97:0/2216553891 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bc2375a910 0x55bc23764680 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.700+0000 7fdaff7ec740 1 --2- 172.21.15.97:0/2216553891 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bc23759f20 0x55bc23764140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.700+0000 7fdadb400640 0 client.15327 ms_handle_reset on v2:172.21.15.114:6836/616775588 2024-03-27T10:56:18.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.700+0000 7fdaff7ec740 1 -- 172.21.15.97:0/2216553891 >> 172.21.15.97:0/2216553891 conn(0x55bc23755490 msgr2=0x55bc23754690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:18.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.700+0000 7fdaff7ec740 1 -- 172.21.15.97:0/2216553891 shutdown_connections 2024-03-27T10:56:18.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.700+0000 7fdaff7ec740 1 -- 172.21.15.97:0/2216553891 wait complete. 2024-03-27T10:56:18.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.700+0000 7fdaff7ec740 20 client.15327 trim_cache size 0 max 16384 2024-03-27T10:56:18.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:56:18 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/3817441582' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T10:56:18.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.719+0000 7fdaff7ec740 1 Processor -- start 2024-03-27T10:56:18.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.719+0000 7fdaff7ec740 1 -- start start 2024-03-27T10:56:18.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.720+0000 7fdaff7ec740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bc23759f20 0x55bc237645e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:18.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.720+0000 7fdaff7ec740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bc2375a910 0x55bc2376b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:18.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.720+0000 7fdaff7ec740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bc2375fa80 0x55bc23762cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:18.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.720+0000 7fdaff7ec740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55bc237705d0 con 0x55bc23759f20 2024-03-27T10:56:18.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.720+0000 7fdaff7ec740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55bc23770ca0 con 0x55bc2375a910 2024-03-27T10:56:18.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.720+0000 7fdaff7ec740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55bc23771370 con 0x55bc2375fa80 2024-03-27T10:56:18.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.720+0000 7fdaf5e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bc23759f20 0x55bc237645e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:18.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.720+0000 7fdaf5e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bc23759f20 0x55bc237645e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45608/0 (socket says 172.21.15.97:45608) 2024-03-27T10:56:18.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.720+0000 7fdaf5e00640 1 -- 172.21.15.97:0/1369299415 learned_addr learned my addr 172.21.15.97:0/1369299415 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:56:18.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.720+0000 7fdaed400640 1 --2- 172.21.15.97:0/1369299415 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bc2375a910 0x55bc2376b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:18.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.720+0000 7fdaf6800640 1 --2- 172.21.15.97:0/1369299415 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bc2375fa80 0x55bc23762cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:18.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.720+0000 7fdaf5e00640 1 -- 172.21.15.97:0/1369299415 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bc2375fa80 msgr2=0x55bc23762cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:56:18.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.720+0000 7fdaf5e00640 1 --2- 172.21.15.97:0/1369299415 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bc2375fa80 0x55bc23762cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.720+0000 7fdaf5e00640 1 -- 172.21.15.97:0/1369299415 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bc2375a910 msgr2=0x55bc2376b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:56:18.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.720+0000 7fdaf5e00640 1 --2- 172.21.15.97:0/1369299415 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bc2375a910 0x55bc2376b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.720+0000 7fdaf5e00640 1 -- 172.21.15.97:0/1369299415 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc23756200 con 0x55bc23759f20 2024-03-27T10:56:18.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.720+0000 7fdaf6800640 1 --2- 172.21.15.97:0/1369299415 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bc2375fa80 0x55bc23762cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:56:18.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.720+0000 7fdaed400640 1 --2- 172.21.15.97:0/1369299415 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bc2375a910 0x55bc2376b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T10:56:18.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.721+0000 7fdaf5e00640 1 --2- 172.21.15.97:0/1369299415 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bc23759f20 0x55bc237645e0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x55bc2375b380 tx=0x7fdae8014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcd48ff9db22a37 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:18.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.721+0000 7fdae6600640 1 -- 172.21.15.97:0/1369299415 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae8004d20 con 0x55bc23759f20 2024-03-27T10:56:18.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.721+0000 7fdae6600640 1 -- 172.21.15.97:0/1369299415 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdae8010e60 con 0x55bc23759f20 2024-03-27T10:56:18.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.721+0000 7fdae6600640 1 -- 172.21.15.97:0/1369299415 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae8005020 con 0x55bc23759f20 2024-03-27T10:56:18.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.722+0000 7fdaff7ec740 1 -- 172.21.15.97:0/1369299415 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bc23759f20 msgr2=0x55bc237645e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:18.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.722+0000 7fdaff7ec740 1 --2- 172.21.15.97:0/1369299415 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bc23759f20 0x55bc237645e0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x55bc2375b380 tx=0x7fdae8014040 comp rx=0 tx=0).stop 2024-03-27T10:56:18.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.722+0000 7fdaff7ec740 1 -- 172.21.15.97:0/1369299415 shutdown_connections 2024-03-27T10:56:18.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.722+0000 7fdaff7ec740 1 --2- 172.21.15.97:0/1369299415 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bc2375fa80 0x55bc23762cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.722+0000 7fdaff7ec740 1 --2- 172.21.15.97:0/1369299415 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bc2375a910 0x55bc2376b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.722+0000 7fdaff7ec740 1 --2- 172.21.15.97:0/1369299415 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bc23759f20 0x55bc237645e0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.722+0000 7fdaff7ec740 1 -- 172.21.15.97:0/1369299415 >> 172.21.15.97:0/1369299415 conn(0x55bc23755490 msgr2=0x55bc2375b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:18.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.722+0000 7fdaff7ec740 1 -- 172.21.15.97:0/1369299415 shutdown_connections 2024-03-27T10:56:18.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.722+0000 7fdaff7ec740 1 -- 172.21.15.97:0/1369299415 wait complete. 2024-03-27T10:56:18.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.722+0000 7fdaff7ec740 1 -- start start 2024-03-27T10:56:18.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.722+0000 7fdaff7ec740 1 Processor -- start 2024-03-27T10:56:18.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.723+0000 7fdaff7ec740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bc2375a910 0x55bc237678b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:18.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.723+0000 7fdaff7ec740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bc2375fa80 0x55bc23767df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:18.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.723+0000 7fdaff7ec740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bc2376a7b0 0x55bc2376ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:18.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.723+0000 7fdaff7ec740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55bc23772110 con 0x55bc2375a910 2024-03-27T10:56:18.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.723+0000 7fdaff7ec740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55bc23771a40 con 0x55bc2375fa80 2024-03-27T10:56:18.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.723+0000 7fdaff7ec740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55bc237727e0 con 0x55bc2376a7b0 2024-03-27T10:56:18.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.723+0000 7fdaff7ec740 10 client.0 _ll_register_callbacks cb 0x7fdae8009660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:56:18.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.723+0000 7fdaf5e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bc2375a910 0x55bc237678b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:18.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.723+0000 7fdaf5e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bc2375a910 0x55bc237678b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45624/0 (socket says 172.21.15.97:45624) 2024-03-27T10:56:18.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.723+0000 7fdaf5e00640 1 -- 172.21.15.97:0/303514322 learned_addr learned my addr 172.21.15.97:0/303514322 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:56:18.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.723+0000 7fdaed400640 1 --2- 172.21.15.97:0/303514322 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bc2375fa80 0x55bc23767df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:18.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.723+0000 7fdaf6800640 1 --2- 172.21.15.97:0/303514322 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bc2376a7b0 0x55bc2376ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:18.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.723+0000 7fdaf5e00640 1 -- 172.21.15.97:0/303514322 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bc2376a7b0 msgr2=0x55bc2376ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:56:18.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.723+0000 7fdaf5e00640 1 --2- 172.21.15.97:0/303514322 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bc2376a7b0 0x55bc2376ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.723+0000 7fdaf5e00640 1 -- 172.21.15.97:0/303514322 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bc2375fa80 msgr2=0x55bc23767df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:56:18.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.723+0000 7fdaf5e00640 1 --2- 172.21.15.97:0/303514322 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bc2375fa80 0x55bc23767df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.723+0000 7fdaf5e00640 1 -- 172.21.15.97:0/303514322 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc2376b150 con 0x55bc2375a910 2024-03-27T10:56:18.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.724+0000 7fdaf5e00640 1 --2- 172.21.15.97:0/303514322 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bc2375a910 0x55bc237678b0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fdae8028040 tx=0x7fdae8010790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2200c898bae0abc2 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:18.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.724+0000 7fdadb400640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T10:56:18.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.724+0000 7fdadb400640 1 -- 172.21.15.97:0/303514322 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae8004c40 con 0x55bc2375a910 2024-03-27T10:56:18.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.724+0000 7fdaff7ec740 1 -- 172.21.15.97:0/303514322 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdae8010e60 con 0x55bc2375a910 2024-03-27T10:56:18.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.724+0000 7fdadb400640 1 -- 172.21.15.97:0/303514322 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fdae8004de0 con 0x55bc2375a910 2024-03-27T10:56:18.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.724+0000 7fdadb400640 1 -- 172.21.15.97:0/303514322 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdae80287f0 con 0x55bc2375a910 2024-03-27T10:56:18.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.724+0000 7fdaff7ec740 10 client.15333 fetch_fsmap learned FSMap version 27 2024-03-27T10:56:18.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.724+0000 7fdaff7ec740 1 -- 172.21.15.97:0/303514322 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55bc237d7430 con 0x55bc2375a910 2024-03-27T10:56:18.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.725+0000 7fdadb400640 1 -- 172.21.15.97:0/303514322 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdae8010e60 con 0x55bc2375a910 2024-03-27T10:56:18.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.725+0000 7fdadb400640 1 -- 172.21.15.97:0/303514322 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fdae8025070 con 0x55bc2375a910 2024-03-27T10:56:18.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.725+0000 7fdaff7ec740 10 client.15333 fetch_fsmap finished waiting for FSMap version 27 2024-03-27T10:56:18.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.725+0000 7fdaff7ec740 10 client.15333 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-27T10:56:18.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.725+0000 7fdaff7ec740 20 client.15333 populate_metadata read hostname 'smithi097' 2024-03-27T10:56:18.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.725+0000 7fdaff7ec740 1 --2- 172.21.15.97:0/303514322 >> [v2:172.21.15.114:6836/616775588,v1:172.21.15.114:6837/616775588] conn(0x55bc237dbc70 0x55bc237d8780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:18.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.725+0000 7fdaff7ec740 4 client.15333 mds_command: new command op to 34478 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-27T10:56:18.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.725+0000 7fdaff7ec740 1 -- 172.21.15.97:0/303514322 --> [v2:172.21.15.114:6836/616775588,v1:172.21.15.114:6837/616775588] -- command(tid 0: {"prefix": "status"}) -- 0x55bc236c6820 con 0x55bc237dbc70 2024-03-27T10:56:18.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.725+0000 7fdaed400640 1 --2- 172.21.15.97:0/303514322 >> [v2:172.21.15.114:6836/616775588,v1:172.21.15.114:6837/616775588] conn(0x55bc237dbc70 0x55bc237d8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:18.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.726+0000 7fdaed400640 1 --2- 172.21.15.97:0/303514322 >> [v2:172.21.15.114:6836/616775588,v1:172.21.15.114:6837/616775588] conn(0x55bc237dbc70 0x55bc237d8780 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=dee6540d4b9aa988 server_cookie=974527dc1d65c20 in_seq=0 out_seq=0 2024-03-27T10:56:18.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.726+0000 7fdadb400640 10 client.15333 ms_handle_connect on v2:172.21.15.114:6836/616775588 2024-03-27T10:56:18.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.727+0000 7fdadb400640 1 -- 172.21.15.97:0/303514322 <== mds.1 v2:172.21.15.114:6836/616775588 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55bc236c6820 con 0x55bc237dbc70 2024-03-27T10:56:18.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.727+0000 7fdadb400640 10 client.15333 handle_command_reply: tid=0 2024-03-27T10:56:18.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.727+0000 7fdaff7ec740 1 client.15333 shutdown 2024-03-27T10:56:18.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.727+0000 7fdaff7ec740 1 -- 172.21.15.97:0/303514322 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bc2375a910 msgr2=0x55bc237678b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:18.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.727+0000 7fdaff7ec740 1 --2- 172.21.15.97:0/303514322 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bc2375a910 0x55bc237678b0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fdae8028040 tx=0x7fdae8010790 comp rx=0 tx=0).stop 2024-03-27T10:56:18.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.727+0000 7fdaff7ec740 1 -- 172.21.15.97:0/303514322 shutdown_connections 2024-03-27T10:56:18.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.727+0000 7fdaff7ec740 1 --2- 172.21.15.97:0/303514322 >> [v2:172.21.15.114:6836/616775588,v1:172.21.15.114:6837/616775588] conn(0x55bc237dbc70 0x55bc237d8780 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.727+0000 7fdaff7ec740 1 --2- 172.21.15.97:0/303514322 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bc2376a7b0 0x55bc2376ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.727+0000 7fdaff7ec740 1 --2- 172.21.15.97:0/303514322 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bc2375fa80 0x55bc23767df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.727+0000 7fdadb400640 0 client.15333 ms_handle_reset on v2:172.21.15.114:6836/616775588 2024-03-27T10:56:18.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.727+0000 7fdaff7ec740 1 --2- 172.21.15.97:0/303514322 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bc2375a910 0x55bc237678b0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.727+0000 7fdaff7ec740 1 -- 172.21.15.97:0/303514322 >> 172.21.15.97:0/303514322 conn(0x55bc23755490 msgr2=0x55bc23762cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:18.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.727+0000 7fdaff7ec740 1 -- 172.21.15.97:0/303514322 shutdown_connections 2024-03-27T10:56:18.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.727+0000 7fdaff7ec740 1 -- 172.21.15.97:0/303514322 wait complete. 2024-03-27T10:56:18.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.727+0000 7fdaff7ec740 20 client.15333 trim_cache size 0 max 16384 2024-03-27T10:56:18.704 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-03-27T10:56:18.704 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "adca88c4-ec26-11ee-95d0-87774f69a715", 2024-03-27T10:56:18.704 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 1, 2024-03-27T10:56:18.704 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 34478, 2024-03-27T10:56:18.705 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2024-03-27T10:56:18.705 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2024-03-27T10:56:18.705 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2024-03-27T10:56:18.705 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 130.138074808, 2024-03-27T10:56:18.705 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 27, 2024-03-27T10:56:18.705 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 92, 2024-03-27T10:56:18.705 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 90, 2024-03-27T10:56:18.705 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 154.51525404700001 2024-03-27T10:56:18.705 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-03-27T10:56:18.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.728+0000 7fdafca00640 1 -- 172.21.15.97:0/176215569 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fdadc0a3610 msgr2=0x7fdadc0a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:18.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.728+0000 7fdafca00640 1 --2- 172.21.15.97:0/176215569 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fdadc0a3610 0x7fdadc0a5ad0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x55bc2375ad10 tx=0x7fdae80097c0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.728+0000 7fdafca00640 1 -- 172.21.15.97:0/176215569 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdaf80780f0 msgr2=0x7fdaf81a58e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:18.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.728+0000 7fdafca00640 1 --2- 172.21.15.97:0/176215569 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdaf80780f0 0x7fdaf81a58e0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fdaf0036040 tx=0x7fdaf002cdd0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.728+0000 7fdafca00640 1 -- 172.21.15.97:0/176215569 shutdown_connections 2024-03-27T10:56:18.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.728+0000 7fdafca00640 1 --2- 172.21.15.97:0/176215569 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdaf80780f0 0x7fdaf81a58e0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.728+0000 7fdafca00640 1 --2- 172.21.15.97:0/176215569 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fdadc0a3610 0x7fdadc0a5ad0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.728+0000 7fdafca00640 1 --2- 172.21.15.97:0/176215569 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdaf8077750 0x7fdaf819e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.728+0000 7fdafca00640 1 --2- 172.21.15.97:0/176215569 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdaf8076510 0x7fdaf819e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.728+0000 7fdafca00640 1 -- 172.21.15.97:0/176215569 >> 172.21.15.97:0/176215569 conn(0x7fdaf80ff300 msgr2=0x7fdaf8100a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:18.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.728+0000 7fdafca00640 1 -- 172.21.15.97:0/176215569 shutdown_connections 2024-03-27T10:56:18.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.728+0000 7fdafca00640 1 -- 172.21.15.97:0/176215569 wait complete. 2024-03-27T10:56:18.717 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-27T10:56:18.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.937+0000 7fe0c6800640 1 -- 172.21.15.97:0/915200671 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe0c0103d90 msgr2=0x7fe0c01041b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:18.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.937+0000 7fe0c6800640 1 --2- 172.21.15.97:0/915200671 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe0c0103d90 0x7fe0c01041b0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fe0b0009900 tx=0x7fe0b002bc40 comp rx=0 tx=0).stop 2024-03-27T10:56:18.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.937+0000 7fe0c6800640 1 -- 172.21.15.97:0/915200671 shutdown_connections 2024-03-27T10:56:18.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.937+0000 7fe0c6800640 1 --2- 172.21.15.97:0/915200671 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe0c0105940 0x7fe0c010c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.937+0000 7fe0c6800640 1 --2- 172.21.15.97:0/915200671 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe0c0104f80 0x7fe0c0105400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.937+0000 7fe0c6800640 1 --2- 172.21.15.97:0/915200671 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe0c0103d90 0x7fe0c01041b0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.937+0000 7fe0c6800640 1 -- 172.21.15.97:0/915200671 >> 172.21.15.97:0/915200671 conn(0x7fe0c00ff300 msgr2=0x7fe0c0101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:18.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.937+0000 7fe0c6800640 1 -- 172.21.15.97:0/915200671 shutdown_connections 2024-03-27T10:56:18.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.937+0000 7fe0c6800640 1 -- 172.21.15.97:0/915200671 wait complete. 2024-03-27T10:56:18.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.938+0000 7fe0c6800640 1 Processor -- start 2024-03-27T10:56:18.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.938+0000 7fe0c6800640 1 -- start start 2024-03-27T10:56:18.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.939+0000 7fe0c6800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe0c0103d90 0x7fe0c0199fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:18.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.939+0000 7fe0c6800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe0c0104f80 0x7fe0c019a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:18.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.939+0000 7fe0c6800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe0c0105940 0x7fe0c01a1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:18.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.939+0000 7fe0c6800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c010f8e0 con 0x7fe0c0104f80 2024-03-27T10:56:18.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.939+0000 7fe0c6800640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c010f760 con 0x7fe0c0103d90 2024-03-27T10:56:18.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.939+0000 7fe0c6800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c010fa60 con 0x7fe0c0105940 2024-03-27T10:56:18.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.939+0000 7fe0bea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe0c0104f80 0x7fe0c019a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:18.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.939+0000 7fe0bea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe0c0104f80 0x7fe0c019a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42464/0 (socket says 172.21.15.97:42464) 2024-03-27T10:56:18.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.939+0000 7fe0bea00640 1 -- 172.21.15.97:0/704401233 learned_addr learned my addr 172.21.15.97:0/704401233 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:56:18.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.939+0000 7fe0bf400640 1 --2- 172.21.15.97:0/704401233 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe0c0103d90 0x7fe0c0199fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:18.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.939+0000 7fe0bfe00640 1 --2- 172.21.15.97:0/704401233 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe0c0105940 0x7fe0c01a1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:18.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.939+0000 7fe0bea00640 1 -- 172.21.15.97:0/704401233 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe0c0105940 msgr2=0x7fe0c01a1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:18.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.939+0000 7fe0bea00640 1 --2- 172.21.15.97:0/704401233 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe0c0105940 0x7fe0c01a1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.939+0000 7fe0bea00640 1 -- 172.21.15.97:0/704401233 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe0c0103d90 msgr2=0x7fe0c0199fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:18.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.939+0000 7fe0bea00640 1 --2- 172.21.15.97:0/704401233 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe0c0103d90 0x7fe0c0199fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.939+0000 7fe0bea00640 1 -- 172.21.15.97:0/704401233 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0c01a1c00 con 0x7fe0c0104f80 2024-03-27T10:56:18.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.939+0000 7fe0bf400640 1 --2- 172.21.15.97:0/704401233 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe0c0103d90 0x7fe0c0199fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T10:56:18.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.940+0000 7fe0bea00640 1 --2- 172.21.15.97:0/704401233 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe0c0104f80 0x7fe0c019a520 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fe0a800d950 tx=0x7fe0a800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:18.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.940+0000 7fe0b7e00640 1 -- 172.21.15.97:0/704401233 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a8017070 con 0x7fe0c0104f80 2024-03-27T10:56:18.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.940+0000 7fe0b7e00640 1 -- 172.21.15.97:0/704401233 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe0a8009c30 con 0x7fe0c0104f80 2024-03-27T10:56:18.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.940+0000 7fe0c6800640 1 -- 172.21.15.97:0/704401233 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0c01a1ef0 con 0x7fe0c0104f80 2024-03-27T10:56:18.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.940+0000 7fe0b7e00640 1 -- 172.21.15.97:0/704401233 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a800f620 con 0x7fe0c0104f80 2024-03-27T10:56:18.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.940+0000 7fe0c6800640 1 -- 172.21.15.97:0/704401233 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0c01a23b0 con 0x7fe0c0104f80 2024-03-27T10:56:18.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.943+0000 7fe0c95a0740 1 Processor -- start 2024-03-27T10:56:18.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.943+0000 7fe0b7e00640 1 -- 172.21.15.97:0/704401233 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7fe0a80109a0 con 0x7fe0c0104f80 2024-03-27T10:56:18.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.943+0000 7fe0c95a0740 1 -- start start 2024-03-27T10:56:18.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.944+0000 7fe0c95a0740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ed3939df20 0x55ed3939e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:18.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.944+0000 7fe0c95a0740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed3939e910 0x55ed393a3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:18.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.944+0000 7fe0c95a0740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ed393a3a80 0x55ed393a5f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:18.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.944+0000 7fe0c95a0740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ed39360920 con 0x55ed3939e910 2024-03-27T10:56:18.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.944+0000 7fe0c95a0740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55ed392b0fa0 con 0x55ed393a3a80 2024-03-27T10:56:18.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.944+0000 7fe0c95a0740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55ed39302f90 con 0x55ed3939df20 2024-03-27T10:56:18.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.944+0000 7fe0bf400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed3939e910 0x55ed393a3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:18.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.944+0000 7fe0bf400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed3939e910 0x55ed393a3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42478/0 (socket says 172.21.15.97:42478) 2024-03-27T10:56:18.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.944+0000 7fe0bf400640 1 -- 172.21.15.97:0/3074842670 learned_addr learned my addr 172.21.15.97:0/3074842670 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:56:18.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.944+0000 7fe0bea00640 1 --2- 172.21.15.97:0/3074842670 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ed393a3a80 0x55ed393a5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:18.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.944+0000 7fe0bfe00640 1 --2- 172.21.15.97:0/3074842670 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ed3939df20 0x55ed3939e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:18.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.944+0000 7fe0bf400640 1 -- 172.21.15.97:0/3074842670 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ed3939df20 msgr2=0x55ed3939e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:56:18.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.944+0000 7fe0bf400640 1 --2- 172.21.15.97:0/3074842670 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ed3939df20 0x55ed3939e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.944+0000 7fe0bf400640 1 -- 172.21.15.97:0/3074842670 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ed393a3a80 msgr2=0x55ed393a5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:56:18.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.944+0000 7fe0bf400640 1 --2- 172.21.15.97:0/3074842670 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ed393a3a80 0x55ed393a5f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.944+0000 7fe0bf400640 1 -- 172.21.15.97:0/3074842670 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed390172d0 con 0x55ed3939e910 2024-03-27T10:56:18.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.945+0000 7fe0b7e00640 1 --2- 172.21.15.97:0/704401233 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fe0a40a3610 0x7fe0a40a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:18.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.945+0000 7fe0bf400640 1 --2- 172.21.15.97:0/3074842670 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed3939e910 0x55ed393a3410 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fe0b002c850 tx=0x7fe0b002c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c458f07f44c9db9a server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:18.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.945+0000 7fe0b7e00640 1 -- 172.21.15.97:0/704401233 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fe0a80cfa30 con 0x7fe0c0104f80 2024-03-27T10:56:18.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.945+0000 7fe0b4c00640 1 -- 172.21.15.97:0/3074842670 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b0044020 con 0x55ed3939e910 2024-03-27T10:56:18.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.945+0000 7fe0bfe00640 1 --2- 172.21.15.97:0/704401233 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fe0a40a3610 0x7fe0a40a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:18.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.945+0000 7fe0b4c00640 1 -- 172.21.15.97:0/3074842670 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe0b0005040 con 0x55ed3939e910 2024-03-27T10:56:18.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.945+0000 7fe0b4c00640 1 -- 172.21.15.97:0/3074842670 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b0004990 con 0x55ed3939e910 2024-03-27T10:56:18.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.946+0000 7fe0c95a0740 1 -- 172.21.15.97:0/3074842670 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed3939e910 msgr2=0x55ed393a3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:18.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.946+0000 7fe0c95a0740 1 --2- 172.21.15.97:0/3074842670 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed3939e910 0x55ed393a3410 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fe0b002c850 tx=0x7fe0b002c8f0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.946+0000 7fe0bfe00640 1 --2- 172.21.15.97:0/704401233 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fe0a40a3610 0x7fe0a40a5ad0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x55ed3939f170 tx=0x7fe0b8008370 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:18.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.946+0000 7fe0c95a0740 1 -- 172.21.15.97:0/3074842670 shutdown_connections 2024-03-27T10:56:18.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.946+0000 7fe0c95a0740 1 --2- 172.21.15.97:0/3074842670 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ed393a3a80 0x55ed393a5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.946+0000 7fe0c95a0740 1 --2- 172.21.15.97:0/3074842670 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed3939e910 0x55ed393a3410 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.946+0000 7fe0c95a0740 1 --2- 172.21.15.97:0/3074842670 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ed3939df20 0x55ed3939e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.946+0000 7fe0c95a0740 1 -- 172.21.15.97:0/3074842670 >> 172.21.15.97:0/3074842670 conn(0x55ed39399490 msgr2=0x55ed39399890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:18.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.946+0000 7fe0c95a0740 1 -- 172.21.15.97:0/3074842670 shutdown_connections 2024-03-27T10:56:18.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.946+0000 7fe0c95a0740 1 -- 172.21.15.97:0/3074842670 wait complete. 2024-03-27T10:56:18.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.947+0000 7fe0c95a0740 1 -- start start 2024-03-27T10:56:18.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.947+0000 7fe0c95a0740 1 Processor -- start 2024-03-27T10:56:18.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.947+0000 7fe0c95a0740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ed3939df20 0x55ed393a8070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:18.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.947+0000 7fe0c95a0740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ed3939e910 0x55ed393a85b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:18.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.947+0000 7fe0c95a0740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed393a3a80 0x55ed393af5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:18.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.947+0000 7fe0c95a0740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ed392fef40 con 0x55ed393a3a80 2024-03-27T10:56:18.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.947+0000 7fe0c95a0740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55ed392ab650 con 0x55ed3939e910 2024-03-27T10:56:18.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.947+0000 7fe0c95a0740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55ed392bea10 con 0x55ed3939df20 2024-03-27T10:56:18.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.947+0000 7fe0c95a0740 10 client.0 _ll_register_callbacks cb 0x55ed392fdff0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:56:18.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.947+0000 7fe0bfe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed393a3a80 0x55ed393af5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:18.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.947+0000 7fe0bfe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed393a3a80 0x55ed393af5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42482/0 (socket says 172.21.15.97:42482) 2024-03-27T10:56:18.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.947+0000 7fe0bfe00640 1 -- 172.21.15.97:0/2737399717 learned_addr learned my addr 172.21.15.97:0/2737399717 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:56:18.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.948+0000 7fe0bea00640 1 --2- 172.21.15.97:0/2737399717 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ed3939e910 0x55ed393a85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:18.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.948+0000 7fe0bf400640 1 --2- 172.21.15.97:0/2737399717 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ed3939df20 0x55ed393a8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:18.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.948+0000 7fe0bfe00640 1 -- 172.21.15.97:0/2737399717 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ed3939df20 msgr2=0x55ed393a8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:56:18.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.948+0000 7fe0bfe00640 1 --2- 172.21.15.97:0/2737399717 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ed3939df20 0x55ed393a8070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.948+0000 7fe0bfe00640 1 -- 172.21.15.97:0/2737399717 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ed3939e910 msgr2=0x55ed393a85b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:56:18.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.948+0000 7fe0bfe00640 1 --2- 172.21.15.97:0/2737399717 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ed3939e910 0x55ed393a85b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.948+0000 7fe0bfe00640 1 -- 172.21.15.97:0/2737399717 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed390e16b0 con 0x55ed393a3a80 2024-03-27T10:56:18.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.948+0000 7fe0bfe00640 1 --2- 172.21.15.97:0/2737399717 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed393a3a80 0x55ed393af5f0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fe0b8011e40 tx=0x7fe0b8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c37efbd97d987264 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:18.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.948+0000 7fe0ad600640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T10:56:18.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.948+0000 7fe0ad600640 1 -- 172.21.15.97:0/2737399717 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b801e420 con 0x55ed393a3a80 2024-03-27T10:56:18.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.948+0000 7fe0ad600640 1 -- 172.21.15.97:0/2737399717 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fe0b801e5c0 con 0x55ed393a3a80 2024-03-27T10:56:18.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.948+0000 7fe0c95a0740 1 -- 172.21.15.97:0/2737399717 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0b0005040 con 0x55ed393a3a80 2024-03-27T10:56:18.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.949+0000 7fe0ad600640 1 -- 172.21.15.97:0/2737399717 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe0b801e7c0 con 0x55ed393a3a80 2024-03-27T10:56:18.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.949+0000 7fe0c95a0740 10 client.15345 fetch_fsmap learned FSMap version 27 2024-03-27T10:56:18.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.949+0000 7fe0c95a0740 1 -- 172.21.15.97:0/2737399717 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ed393b2a80 con 0x55ed393a3a80 2024-03-27T10:56:18.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.949+0000 7fe0ad600640 1 -- 172.21.15.97:0/2737399717 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe0b0005040 con 0x55ed393a3a80 2024-03-27T10:56:18.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.949+0000 7fe0ad600640 1 -- 172.21.15.97:0/2737399717 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fe0b801ca40 con 0x55ed393a3a80 2024-03-27T10:56:18.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.949+0000 7fe0c95a0740 10 client.15345 fetch_fsmap finished waiting for FSMap version 27 2024-03-27T10:56:18.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.949+0000 7fe0c95a0740 10 client.15345 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-27T10:56:18.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.949+0000 7fe0c95a0740 20 client.15345 populate_metadata read hostname 'smithi097' 2024-03-27T10:56:18.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.949+0000 7fe0c95a0740 1 --2- 172.21.15.97:0/2737399717 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55ed393b51d0 0x55ed393b6640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:18.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.949+0000 7fe0c95a0740 4 client.15345 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T10:56:18.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.949+0000 7fe0c95a0740 1 -- 172.21.15.97:0/2737399717 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ed393104d0 con 0x55ed393b51d0 2024-03-27T10:56:18.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.950+0000 7fe0bf400640 1 --2- 172.21.15.97:0/2737399717 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55ed393b51d0 0x55ed393b6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:18.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.951+0000 7fe0bf400640 1 --2- 172.21.15.97:0/2737399717 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55ed393b51d0 0x55ed393b6640 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3b5048f13ab43dc server_cookie=637a7ebb70440156 in_seq=0 out_seq=0 2024-03-27T10:56:18.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.951+0000 7fe0ad600640 10 client.15345 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T10:56:18.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.953+0000 7fe0ad600640 1 -- 172.21.15.97:0/2737399717 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55ed393104d0 con 0x55ed393b51d0 2024-03-27T10:56:18.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.953+0000 7fe0ad600640 10 client.15345 handle_command_reply: tid=0 2024-03-27T10:56:18.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.953+0000 7fe0c95a0740 1 client.15345 shutdown 2024-03-27T10:56:18.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.953+0000 7fe0c95a0740 1 -- 172.21.15.97:0/2737399717 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed393a3a80 msgr2=0x55ed393af5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:18.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.953+0000 7fe0c95a0740 1 --2- 172.21.15.97:0/2737399717 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed393a3a80 0x55ed393af5f0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fe0b8011e40 tx=0x7fe0b8004a40 comp rx=0 tx=0).stop 2024-03-27T10:56:18.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.953+0000 7fe0c95a0740 1 -- 172.21.15.97:0/2737399717 shutdown_connections 2024-03-27T10:56:18.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.953+0000 7fe0c95a0740 1 --2- 172.21.15.97:0/2737399717 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55ed393b51d0 0x55ed393b6640 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.953+0000 7fe0c95a0740 1 --2- 172.21.15.97:0/2737399717 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed393a3a80 0x55ed393af5f0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.953+0000 7fe0c95a0740 1 --2- 172.21.15.97:0/2737399717 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ed3939e910 0x55ed393a85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.953+0000 7fe0c95a0740 1 --2- 172.21.15.97:0/2737399717 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ed3939df20 0x55ed393a8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.953+0000 7fe0ad600640 0 client.15345 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T10:56:18.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.953+0000 7fe0c95a0740 1 -- 172.21.15.97:0/2737399717 >> 172.21.15.97:0/2737399717 conn(0x55ed39399490 msgr2=0x55ed39398690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:18.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.954+0000 7fe0c95a0740 1 -- 172.21.15.97:0/2737399717 shutdown_connections 2024-03-27T10:56:18.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.954+0000 7fe0c95a0740 1 -- 172.21.15.97:0/2737399717 wait complete. 2024-03-27T10:56:18.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.954+0000 7fe0c95a0740 20 client.15345 trim_cache size 0 max 16384 2024-03-27T10:56:18.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.973+0000 7fe0c95a0740 1 Processor -- start 2024-03-27T10:56:18.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.973+0000 7fe0c95a0740 1 -- start start 2024-03-27T10:56:18.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.974+0000 7fe0c95a0740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ed3939df20 0x55ed393a7340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:18.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.974+0000 7fe0c95a0740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ed3939e910 0x55ed393af5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:18.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.974+0000 7fe0c95a0740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed393a3a80 0x55ed393a7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:18.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.974+0000 7fe0c95a0740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ed393b4980 con 0x55ed393a3a80 2024-03-27T10:56:18.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.974+0000 7fe0c95a0740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55ed393b5050 con 0x55ed3939df20 2024-03-27T10:56:18.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.974+0000 7fe0c95a0740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55ed393b5720 con 0x55ed3939e910 2024-03-27T10:56:18.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.974+0000 7fe0bfe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed393a3a80 0x55ed393a7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:18.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.974+0000 7fe0bfe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed393a3a80 0x55ed393a7880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42488/0 (socket says 172.21.15.97:42488) 2024-03-27T10:56:18.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.974+0000 7fe0bfe00640 1 -- 172.21.15.97:0/3450460428 learned_addr learned my addr 172.21.15.97:0/3450460428 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:56:18.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.974+0000 7fe0bf400640 1 --2- 172.21.15.97:0/3450460428 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ed3939df20 0x55ed393a7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:18.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.974+0000 7fe0bea00640 1 --2- 172.21.15.97:0/3450460428 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ed3939e910 0x55ed393af5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:18.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.974+0000 7fe0bfe00640 1 -- 172.21.15.97:0/3450460428 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ed3939e910 msgr2=0x55ed393af5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:56:18.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.974+0000 7fe0bfe00640 1 --2- 172.21.15.97:0/3450460428 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ed3939e910 0x55ed393af5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.974+0000 7fe0bfe00640 1 -- 172.21.15.97:0/3450460428 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ed3939df20 msgr2=0x55ed393a7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:56:18.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.974+0000 7fe0bfe00640 1 --2- 172.21.15.97:0/3450460428 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ed3939df20 0x55ed393a7340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.974+0000 7fe0bfe00640 1 -- 172.21.15.97:0/3450460428 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed3939f1a0 con 0x55ed393a3a80 2024-03-27T10:56:18.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.975+0000 7fe0bfe00640 1 --2- 172.21.15.97:0/3450460428 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed393a3a80 0x55ed393a7880 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fe0b8000f80 tx=0x7fe0b8011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b57e0605f3c73e server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:18.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.975+0000 7fe0afe00640 1 -- 172.21.15.97:0/3450460428 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b800ece0 con 0x55ed393a3a80 2024-03-27T10:56:18.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.975+0000 7fe0afe00640 1 -- 172.21.15.97:0/3450460428 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe0b800ee80 con 0x55ed393a3a80 2024-03-27T10:56:18.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.975+0000 7fe0afe00640 1 -- 172.21.15.97:0/3450460428 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b801c730 con 0x55ed393a3a80 2024-03-27T10:56:18.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.976+0000 7fe0c95a0740 1 -- 172.21.15.97:0/3450460428 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed393a3a80 msgr2=0x55ed393a7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:18.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.976+0000 7fe0c95a0740 1 --2- 172.21.15.97:0/3450460428 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed393a3a80 0x55ed393a7880 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fe0b8000f80 tx=0x7fe0b8011b20 comp rx=0 tx=0).stop 2024-03-27T10:56:18.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.976+0000 7fe0c95a0740 1 -- 172.21.15.97:0/3450460428 shutdown_connections 2024-03-27T10:56:18.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.976+0000 7fe0c95a0740 1 --2- 172.21.15.97:0/3450460428 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed393a3a80 0x55ed393a7880 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.976+0000 7fe0c95a0740 1 --2- 172.21.15.97:0/3450460428 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ed3939e910 0x55ed393af5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.976+0000 7fe0c95a0740 1 --2- 172.21.15.97:0/3450460428 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ed3939df20 0x55ed393a7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.976+0000 7fe0c95a0740 1 -- 172.21.15.97:0/3450460428 >> 172.21.15.97:0/3450460428 conn(0x55ed39399490 msgr2=0x55ed393a8070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:18.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.976+0000 7fe0c95a0740 1 -- 172.21.15.97:0/3450460428 shutdown_connections 2024-03-27T10:56:18.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.976+0000 7fe0c95a0740 1 -- 172.21.15.97:0/3450460428 wait complete. 2024-03-27T10:56:18.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.976+0000 7fe0c95a0740 1 -- start start 2024-03-27T10:56:18.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.976+0000 7fe0c95a0740 1 Processor -- start 2024-03-27T10:56:18.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.977+0000 7fe0c95a0740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ed3939df20 0x55ed393ab7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:18.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.977+0000 7fe0c95a0740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ed3939e910 0x55ed393abd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:18.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.977+0000 7fe0c95a0740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed393a3a80 0x55ed393ae970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:18.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.977+0000 7fe0c95a0740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ed393b64c0 con 0x55ed393a3a80 2024-03-27T10:56:18.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.977+0000 7fe0c95a0740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55ed393b5df0 con 0x55ed3939df20 2024-03-27T10:56:18.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.977+0000 7fe0c95a0740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55ed393b6b90 con 0x55ed3939e910 2024-03-27T10:56:18.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.977+0000 7fe0c95a0740 10 client.0 _ll_register_callbacks cb 0x7fe0b8008540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:56:18.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.977+0000 7fe0bfe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed393a3a80 0x55ed393ae970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:18.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.977+0000 7fe0bfe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed393a3a80 0x55ed393ae970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42496/0 (socket says 172.21.15.97:42496) 2024-03-27T10:56:18.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.977+0000 7fe0bfe00640 1 -- 172.21.15.97:0/805971391 learned_addr learned my addr 172.21.15.97:0/805971391 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:56:18.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.977+0000 7fe0bf400640 1 --2- 172.21.15.97:0/805971391 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ed3939df20 0x55ed393ab7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:18.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.977+0000 7fe0bea00640 1 --2- 172.21.15.97:0/805971391 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ed3939e910 0x55ed393abd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:18.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.977+0000 7fe0bfe00640 1 -- 172.21.15.97:0/805971391 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ed3939e910 msgr2=0x55ed393abd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:56:18.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.977+0000 7fe0bfe00640 1 --2- 172.21.15.97:0/805971391 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ed3939e910 0x55ed393abd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.977+0000 7fe0bfe00640 1 -- 172.21.15.97:0/805971391 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ed3939df20 msgr2=0x55ed393ab7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:56:18.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.977+0000 7fe0bfe00640 1 --2- 172.21.15.97:0/805971391 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ed3939df20 0x55ed393ab7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.977+0000 7fe0bfe00640 1 -- 172.21.15.97:0/805971391 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed393aefb0 con 0x55ed393a3a80 2024-03-27T10:56:18.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.977+0000 7fe0bf400640 1 --2- 172.21.15.97:0/805971391 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ed3939df20 0x55ed393ab7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T10:56:18.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.978+0000 7fe0bfe00640 1 --2- 172.21.15.97:0/805971391 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed393a3a80 0x55ed393ae970 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fe0b8011010 tx=0x7fe0b8004c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba31ecbf5e973a09 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:18.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.978+0000 7fe0ad600640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T10:56:18.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.978+0000 7fe0ad600640 1 -- 172.21.15.97:0/805971391 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b8035040 con 0x55ed393a3a80 2024-03-27T10:56:18.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.978+0000 7fe0ad600640 1 -- 172.21.15.97:0/805971391 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fe0b801c6c0 con 0x55ed393a3a80 2024-03-27T10:56:18.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.978+0000 7fe0c95a0740 1 -- 172.21.15.97:0/805971391 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ed3930a820 con 0x55ed393a3a80 2024-03-27T10:56:18.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.978+0000 7fe0ad600640 1 -- 172.21.15.97:0/805971391 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe0b800ece0 con 0x55ed393a3a80 2024-03-27T10:56:18.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.978+0000 7fe0c95a0740 10 client.15351 fetch_fsmap learned FSMap version 27 2024-03-27T10:56:18.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.978+0000 7fe0c95a0740 1 -- 172.21.15.97:0/805971391 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ed3941b680 con 0x55ed393a3a80 2024-03-27T10:56:18.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.979+0000 7fe0ad600640 1 -- 172.21.15.97:0/805971391 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55ed3930a820 con 0x55ed393a3a80 2024-03-27T10:56:18.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.979+0000 7fe0ad600640 1 -- 172.21.15.97:0/805971391 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fe0b8005560 con 0x55ed393a3a80 2024-03-27T10:56:18.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.979+0000 7fe0c95a0740 10 client.15351 fetch_fsmap finished waiting for FSMap version 27 2024-03-27T10:56:18.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.979+0000 7fe0c95a0740 10 client.15351 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-27T10:56:18.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.979+0000 7fe0c95a0740 20 client.15351 populate_metadata read hostname 'smithi097' 2024-03-27T10:56:18.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.979+0000 7fe0c95a0740 1 --2- 172.21.15.97:0/805971391 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55ed39420020 0x55ed3941ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:18.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.979+0000 7fe0c95a0740 4 client.15351 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-27T10:56:18.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.979+0000 7fe0c95a0740 1 -- 172.21.15.97:0/805971391 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "status"}) -- 0x55ed392888c0 con 0x55ed39420020 2024-03-27T10:56:18.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.979+0000 7fe0bf400640 1 --2- 172.21.15.97:0/805971391 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55ed39420020 0x55ed3941ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:18.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.980+0000 7fe0bf400640 1 --2- 172.21.15.97:0/805971391 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55ed39420020 0x55ed3941ed30 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e41e3ca769ee5e9 server_cookie=221309919766decf in_seq=0 out_seq=0 2024-03-27T10:56:18.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.980+0000 7fe0ad600640 10 client.15351 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T10:56:18.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.981+0000 7fe0ad600640 1 -- 172.21.15.97:0/805971391 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55ed392888c0 con 0x55ed39420020 2024-03-27T10:56:18.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.981+0000 7fe0ad600640 10 client.15351 handle_command_reply: tid=0 2024-03-27T10:56:18.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.981+0000 7fe0c95a0740 1 client.15351 shutdown 2024-03-27T10:56:18.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.981+0000 7fe0c95a0740 1 -- 172.21.15.97:0/805971391 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed393a3a80 msgr2=0x55ed393ae970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:18.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.981+0000 7fe0c95a0740 1 --2- 172.21.15.97:0/805971391 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed393a3a80 0x55ed393ae970 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fe0b8011010 tx=0x7fe0b8004c30 comp rx=0 tx=0).stop 2024-03-27T10:56:18.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.981+0000 7fe0c95a0740 1 -- 172.21.15.97:0/805971391 shutdown_connections 2024-03-27T10:56:18.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.981+0000 7fe0c95a0740 1 --2- 172.21.15.97:0/805971391 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55ed39420020 0x55ed3941ed30 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.981+0000 7fe0c95a0740 1 --2- 172.21.15.97:0/805971391 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed393a3a80 0x55ed393ae970 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.981+0000 7fe0c95a0740 1 --2- 172.21.15.97:0/805971391 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ed3939e910 0x55ed393abd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.981+0000 7fe0c95a0740 1 --2- 172.21.15.97:0/805971391 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ed3939df20 0x55ed393ab7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.981+0000 7fe0ad600640 0 client.15351 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T10:56:18.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.981+0000 7fe0c95a0740 1 -- 172.21.15.97:0/805971391 >> 172.21.15.97:0/805971391 conn(0x55ed39399490 msgr2=0x55ed393a7340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:18.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.981+0000 7fe0c95a0740 1 -- 172.21.15.97:0/805971391 shutdown_connections 2024-03-27T10:56:18.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.981+0000 7fe0c95a0740 1 -- 172.21.15.97:0/805971391 wait complete. 2024-03-27T10:56:18.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.981+0000 7fe0c95a0740 20 client.15351 trim_cache size 0 max 16384 2024-03-27T10:56:18.959 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-03-27T10:56:18.959 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "adca88c4-ec26-11ee-95d0-87774f69a715", 2024-03-27T10:56:18.960 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 0, 2024-03-27T10:56:18.960 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 34508, 2024-03-27T10:56:18.960 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2024-03-27T10:56:18.960 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2024-03-27T10:56:18.960 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2024-03-27T10:56:18.960 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 133.74512269499999, 2024-03-27T10:56:18.960 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 27, 2024-03-27T10:56:18.960 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 92, 2024-03-27T10:56:18.960 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 90, 2024-03-27T10:56:18.960 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 139.300747989 2024-03-27T10:56:18.960 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-03-27T10:56:18.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.982+0000 7fe0c6800640 1 -- 172.21.15.97:0/704401233 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fe0a40a3610 msgr2=0x7fe0a40a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:18.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.982+0000 7fe0c6800640 1 --2- 172.21.15.97:0/704401233 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fe0a40a3610 0x7fe0a40a5ad0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x55ed3939f170 tx=0x7fe0b8008370 comp rx=0 tx=0).stop 2024-03-27T10:56:18.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.982+0000 7fe0c6800640 1 -- 172.21.15.97:0/704401233 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe0c0104f80 msgr2=0x7fe0c019a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:18.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.982+0000 7fe0c6800640 1 --2- 172.21.15.97:0/704401233 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe0c0104f80 0x7fe0c019a520 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fe0a800d950 tx=0x7fe0a800de10 comp rx=0 tx=0).stop 2024-03-27T10:56:18.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.982+0000 7fe0c6800640 1 -- 172.21.15.97:0/704401233 shutdown_connections 2024-03-27T10:56:18.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.982+0000 7fe0c6800640 1 --2- 172.21.15.97:0/704401233 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe0c0105940 0x7fe0c01a1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.982+0000 7fe0c6800640 1 --2- 172.21.15.97:0/704401233 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe0c0104f80 0x7fe0c019a520 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.982+0000 7fe0c6800640 1 --2- 172.21.15.97:0/704401233 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fe0a40a3610 0x7fe0a40a5ad0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.982+0000 7fe0c6800640 1 --2- 172.21.15.97:0/704401233 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe0c0103d90 0x7fe0c0199fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:18.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.982+0000 7fe0c6800640 1 -- 172.21.15.97:0/704401233 >> 172.21.15.97:0/704401233 conn(0x7fe0c00ff300 msgr2=0x7fe0c01082f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:18.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.982+0000 7fe0c6800640 1 -- 172.21.15.97:0/704401233 shutdown_connections 2024-03-27T10:56:18.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:18.982+0000 7fe0c6800640 1 -- 172.21.15.97:0/704401233 wait complete. 2024-03-27T10:56:18.972 INFO:teuthology.run_tasks:Running task sequential... 2024-03-27T10:56:18.982 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-03-27T10:56:18.982 INFO:teuthology.task.sequential:In sequential, running task print... 2024-03-27T10:56:18.983 INFO:teuthology.task.print:Enabling mgr modules 2024-03-27T10:56:18.983 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-03-27T10:56:19.012 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-03-27T10:56:19.012 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2024-03-27T10:56:19.012 INFO:tasks.ceph_fuse:client.0 config is {'mount_subvol_num': 0} 2024-03-27T10:56:19.012 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-27T10:56:19.012 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-27T10:56:19.013 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-27T10:56:19.013 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2024-03-27T10:56:19.068 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-27T10:56:19.068 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link delete ceph-brx 2024-03-27T10:56:19.164 INFO:teuthology.orchestra.run.smithi097.stderr:Cannot find device "ceph-brx" 2024-03-27T10:56:19.168 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-27T10:56:19.168 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-03-27T10:56:19.168 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell --fsid adca88c4-ec26-11ee-95d0-87774f69a715 -- ceph fs ls 2024-03-27T10:56:19.516 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:56:19 smithi114 ceph-mon[35338]: pgmap v431: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-27T10:56:19.613 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T10:56:19.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:56:19 smithi097 ceph-mon[31859]: pgmap v431: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-27T10:56:19.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:56:19 smithi177 ceph-mon[35294]: pgmap v431: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-27T10:56:21.510 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:56:21 smithi097 ceph-mon[31859]: pgmap v432: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-27T10:56:21.516 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:56:21 smithi114 ceph-mon[35338]: pgmap v432: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-27T10:56:21.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:56:21 smithi177 ceph-mon[35294]: pgmap v432: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-27T10:56:21.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:21.800+0000 7f3f55800700 1 -- 172.21.15.97:0/1480575178 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f5010df90 msgr2=0x7f3f50115b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:21.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:21.800+0000 7f3f55800700 1 --2- 172.21.15.97:0/1480575178 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f5010df90 0x7f3f50115b60 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f3f4800b0d0 tx=0x7f3f4800b3e0 comp rx=0 tx=0).stop 2024-03-27T10:56:21.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:21.801+0000 7f3f55800700 1 -- 172.21.15.97:0/1480575178 shutdown_connections 2024-03-27T10:56:21.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:21.801+0000 7f3f55800700 1 --2- 172.21.15.97:0/1480575178 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f5010df90 0x7f3f50115b60 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:21.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:21.801+0000 7f3f55800700 1 --2- 172.21.15.97:0/1480575178 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3f50108eb0 0x7f3f50109290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:21.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:21.801+0000 7f3f55800700 1 --2- 172.21.15.97:0/1480575178 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3f50109860 0x7f3f5010d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:21.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:21.801+0000 7f3f55800700 1 -- 172.21.15.97:0/1480575178 >> 172.21.15.97:0/1480575178 conn(0x7f3f50075e10 msgr2=0x7f3f50078230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:21.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:21.801+0000 7f3f55800700 1 -- 172.21.15.97:0/1480575178 shutdown_connections 2024-03-27T10:56:21.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:21.801+0000 7f3f55800700 1 -- 172.21.15.97:0/1480575178 wait complete. 2024-03-27T10:56:21.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:21.802+0000 7f3f55800700 1 Processor -- start 2024-03-27T10:56:21.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:21.802+0000 7f3f55800700 1 -- start start 2024-03-27T10:56:21.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:21.802+0000 7f3f55800700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3f50108eb0 0x7f3f5019f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:21.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:21.802+0000 7f3f55800700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f50109860 0x7f3f5019f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:21.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:21.802+0000 7f3f55800700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3f5010df90 0x7f3f501a3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:21.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:21.802+0000 7f3f55800700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3f50118e20 con 0x7f3f50109860 2024-03-27T10:56:21.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:21.802+0000 7f3f55800700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f3f50118ca0 con 0x7f3f50108eb0 2024-03-27T10:56:21.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:21.802+0000 7f3f55800700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f3f50118fa0 con 0x7f3f5010df90 2024-03-27T10:56:21.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:21.803+0000 7f3f47e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f50109860 0x7f3f5019f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:21.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:21.803+0000 7f3f47e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f50109860 0x7f3f5019f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42528/0 (socket says 172.21.15.97:42528) 2024-03-27T10:56:21.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:21.803+0000 7f3f47e00700 1 -- 172.21.15.97:0/1253678374 learned_addr learned my addr 172.21.15.97:0/1253678374 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:56:21.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:21.803+0000 7f3f4ea00700 1 --2- 172.21.15.97:0/1253678374 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3f50108eb0 0x7f3f5019f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:21.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:21.803+0000 7f3f4f400700 1 --2- 172.21.15.97:0/1253678374 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3f5010df90 0x7f3f501a3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:21.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:21.803+0000 7f3f47e00700 1 -- 172.21.15.97:0/1253678374 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3f5010df90 msgr2=0x7f3f501a3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:21.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:21.803+0000 7f3f47e00700 1 --2- 172.21.15.97:0/1253678374 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3f5010df90 0x7f3f501a3ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:21.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:21.803+0000 7f3f47e00700 1 -- 172.21.15.97:0/1253678374 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3f50108eb0 msgr2=0x7f3f5019f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:21.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:21.803+0000 7f3f47e00700 1 --2- 172.21.15.97:0/1253678374 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3f50108eb0 0x7f3f5019f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:21.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:21.803+0000 7f3f47e00700 1 -- 172.21.15.97:0/1253678374 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f501a4220 con 0x7f3f50109860 2024-03-27T10:56:21.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:21.804+0000 7f3f47e00700 1 --2- 172.21.15.97:0/1253678374 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f50109860 0x7f3f5019f750 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f3f3800d980 tx=0x7f3f3800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:21.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:21.804+0000 7f3f47400700 1 -- 172.21.15.97:0/1253678374 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f38017070 con 0x7f3f50109860 2024-03-27T10:56:21.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:21.804+0000 7f3f55800700 1 -- 172.21.15.97:0/1253678374 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f501a4510 con 0x7f3f50109860 2024-03-27T10:56:21.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:21.804+0000 7f3f47400700 1 -- 172.21.15.97:0/1253678374 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3f38009de0 con 0x7f3f50109860 2024-03-27T10:56:21.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:21.804+0000 7f3f47400700 1 -- 172.21.15.97:0/1253678374 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f3800f3c0 con 0x7f3f50109860 2024-03-27T10:56:21.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:21.804+0000 7f3f55800700 1 -- 172.21.15.97:0/1253678374 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f501abd90 con 0x7f3f50109860 2024-03-27T10:56:21.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:21.805+0000 7f3f55800700 1 -- 172.21.15.97:0/1253678374 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f50069300 con 0x7f3f50109860 2024-03-27T10:56:21.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:21.807+0000 7f3f47400700 1 -- 172.21.15.97:0/1253678374 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f3f38009930 con 0x7f3f50109860 2024-03-27T10:56:21.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:21.808+0000 7f3f47400700 1 --2- 172.21.15.97:0/1253678374 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f3f340a3720 0x7f3f340a5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:21.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:21.808+0000 7f3f47400700 1 -- 172.21.15.97:0/1253678374 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f3f380cf790 con 0x7f3f50109860 2024-03-27T10:56:21.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:21.809+0000 7f3f4ea00700 1 --2- 172.21.15.97:0/1253678374 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f3f340a3720 0x7f3f340a5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:21.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:21.809+0000 7f3f4ea00700 1 --2- 172.21.15.97:0/1253678374 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f3f340a3720 0x7f3f340a5be0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f3f4000ac50 tx=0x7f3f4000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:21.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:21.813+0000 7f3f47400700 1 -- 172.21.15.97:0/1253678374 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f3f3809c310 con 0x7f3f50109860 2024-03-27T10:56:22.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:22.088+0000 7f3f55800700 1 -- 172.21.15.97:0/1253678374 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f3f501a0570 con 0x7f3f50109860 2024-03-27T10:56:22.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:22.089+0000 7f3f47400700 1 -- 172.21.15.97:0/1253678374 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) ==== 53+0+73 (secure 0 0 0) 0x7f3f501a0570 con 0x7f3f50109860 2024-03-27T10:56:22.062 INFO:teuthology.orchestra.run.smithi097.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-27T10:56:22.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:22.092+0000 7f3f55800700 1 -- 172.21.15.97:0/1253678374 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f3f340a3720 msgr2=0x7f3f340a5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:22.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:22.092+0000 7f3f55800700 1 --2- 172.21.15.97:0/1253678374 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f3f340a3720 0x7f3f340a5be0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f3f4000ac50 tx=0x7f3f4000a380 comp rx=0 tx=0).stop 2024-03-27T10:56:22.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:22.092+0000 7f3f55800700 1 -- 172.21.15.97:0/1253678374 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f50109860 msgr2=0x7f3f5019f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:22.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:22.092+0000 7f3f55800700 1 --2- 172.21.15.97:0/1253678374 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f50109860 0x7f3f5019f750 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f3f3800d980 tx=0x7f3f3800dc90 comp rx=0 tx=0).stop 2024-03-27T10:56:22.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:22.092+0000 7f3f55800700 1 -- 172.21.15.97:0/1253678374 shutdown_connections 2024-03-27T10:56:22.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:22.092+0000 7f3f55800700 1 --2- 172.21.15.97:0/1253678374 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f3f340a3720 0x7f3f340a5be0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:22.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:22.092+0000 7f3f55800700 1 --2- 172.21.15.97:0/1253678374 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3f5010df90 0x7f3f501a3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:22.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:22.092+0000 7f3f55800700 1 --2- 172.21.15.97:0/1253678374 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3f50108eb0 0x7f3f5019f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:22.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:22.092+0000 7f3f55800700 1 --2- 172.21.15.97:0/1253678374 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f50109860 0x7f3f5019f750 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:22.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:22.092+0000 7f3f55800700 1 -- 172.21.15.97:0/1253678374 >> 172.21.15.97:0/1253678374 conn(0x7f3f50075e10 msgr2=0x7f3f50076870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:22.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:22.093+0000 7f3f55800700 1 -- 172.21.15.97:0/1253678374 shutdown_connections 2024-03-27T10:56:22.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:22.093+0000 7f3f55800700 1 -- 172.21.15.97:0/1253678374 wait complete. 2024-03-27T10:56:22.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:56:22 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/1253678374' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-27T10:56:22.516 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:56:22 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/1253678374' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-27T10:56:22.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:56:22 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/1253678374' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-27T10:56:22.953 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-27T10:56:22.953 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-27T10:56:22.953 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi097.front.sepia.ceph.com 2024-03-27T10:56:22.953 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-27T10:56:22.953 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-27T10:56:22.953 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-27T10:56:22.953 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-27T10:56:22.953 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-27T10:56:22.953 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-27T10:56:22.953 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-27T10:56:22.954 DEBUG:teuthology.orchestra.run.smithi097:> ip addr 2024-03-27T10:56:23.008 INFO:teuthology.orchestra.run.smithi097.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-27T10:56:23.008 INFO:teuthology.orchestra.run.smithi097.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-27T10:56:23.008 INFO:teuthology.orchestra.run.smithi097.stdout: inet 127.0.0.1/8 scope host lo 2024-03-27T10:56:23.008 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-03-27T10:56:23.008 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 ::1/128 scope host 2024-03-27T10:56:23.008 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-03-27T10:56:23.008 INFO:teuthology.orchestra.run.smithi097.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-27T10:56:23.008 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0e brd ff:ff:ff:ff:ff:ff 2024-03-27T10:56:23.008 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f0 2024-03-27T10:56:23.008 INFO:teuthology.orchestra.run.smithi097.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-27T10:56:23.008 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0f brd ff:ff:ff:ff:ff:ff 2024-03-27T10:56:23.009 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f1 2024-03-27T10:56:23.009 INFO:teuthology.orchestra.run.smithi097.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-27T10:56:23.009 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:04 brd ff:ff:ff:ff:ff:ff 2024-03-27T10:56:23.009 INFO:teuthology.orchestra.run.smithi097.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-27T10:56:23.009 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:05 brd ff:ff:ff:ff:ff:ff 2024-03-27T10:56:23.009 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.21.15.97/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-27T10:56:23.009 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft 41805sec preferred_lft 41805sec 2024-03-27T10:56:23.009 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::ec4:7aff:fe8f:d805/64 scope link 2024-03-27T10:56:23.009 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-03-27T10:56:23.010 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-27T10:56:23.010 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-27T10:56:23.011 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-03-27T10:56:23.011 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link add name ceph-brx type bridge 2024-03-27T10:56:23.011 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip addr flush dev ceph-brx 2024-03-27T10:56:23.011 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set ceph-brx up 2024-03-27T10:56:23.011 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-27T10:56:23.011 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-03-27T10:56:23.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-27T10:56:23.401 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:56:23 smithi097 ceph-mon[31859]: pgmap v433: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T10:56:23.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-27T10:56:23.502 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-27T10:56:23.502 DEBUG:teuthology.orchestra.run.smithi097:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-27T10:56:23.516 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:56:23 smithi114 ceph-mon[35338]: pgmap v433: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T10:56:23.575 INFO:teuthology.orchestra.run.smithi097.stdout:1 2024-03-27T10:56:23.578 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:56:23 smithi177 ceph-mon[35294]: pgmap v433: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T10:56:23.579 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-27T10:56:23.579 DEBUG:teuthology.orchestra.run.smithi097:> route 2024-03-27T10:56:23.641 INFO:teuthology.orchestra.run.smithi097.stdout:Kernel IP routing table 2024-03-27T10:56:23.641 INFO:teuthology.orchestra.run.smithi097.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-27T10:56:23.641 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-27T10:56:23.641 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-27T10:56:23.641 INFO:teuthology.orchestra.run.smithi097.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-27T10:56:23.642 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-27T10:56:23.642 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-03-27T10:56:23.643 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-27T10:56:23.643 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-27T10:56:23.643 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-27T10:56:23.643 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-03-27T10:56:23.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-27T10:56:24.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-27T10:56:24.386 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-27T10:56:24.387 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2024-03-27T10:56:24.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-27T10:56:24.437 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list-id 2024-03-27T10:56:24.484 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-27T10:56:24.484 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-03-27T10:56:24.484 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-27T10:56:24.484 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-03-27T10:56:24.484 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-03-27T10:56:24.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-27T10:56:24.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-27T10:56:24.664 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-03-27T10:56:24.664 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-27T10:56:24.664 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-03-27T10:56:24.664 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-03-27T10:56:24.664 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-27T10:56:24.665 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-03-27T10:56:24.665 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-03-27T10:56:24.665 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-03-27T10:56:24.665 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-03-27T10:56:24.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-27T10:56:24.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-27T10:56:24.982 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-27T10:56:24.982 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-03-27T10:56:24.982 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set brx.0 up 2024-03-27T10:56:24.983 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set dev brx.0 master ceph-brx 2024-03-27T10:56:24.983 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-03-27T10:56:25.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-27T10:56:25.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-27T10:56:25.199 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {'mount_subvol_num': 0} 2024-03-27T10:56:25.199 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-27T10:56:25.199 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-27T10:56:25.246 INFO:teuthology.orchestra.run.smithi097.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-27T10:56:25.246 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-27T10:56:25.246 DEBUG:teuthology.orchestra.run.smithi097:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-27T10:56:25.335 DEBUG:teuthology.orchestra.run.smithi097:> sudo modprobe fuse 2024-03-27T10:56:25.401 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:56:25 smithi097 ceph-mon[31859]: pgmap v434: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-27T10:56:25.413 DEBUG:teuthology.orchestra.run.smithi097:> cat /proc/self/mounts | awk '{print $2}' 2024-03-27T10:56:25.457 INFO:teuthology.orchestra.run.smithi097.stdout:/proc 2024-03-27T10:56:25.458 INFO:teuthology.orchestra.run.smithi097.stdout:/sys 2024-03-27T10:56:25.458 INFO:teuthology.orchestra.run.smithi097.stdout:/dev 2024-03-27T10:56:25.458 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/security 2024-03-27T10:56:25.458 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/shm 2024-03-27T10:56:25.458 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/pts 2024-03-27T10:56:25.458 INFO:teuthology.orchestra.run.smithi097.stdout:/run 2024-03-27T10:56:25.458 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/cgroup 2024-03-27T10:56:25.458 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/pstore 2024-03-27T10:56:25.458 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/bpf 2024-03-27T10:56:25.458 INFO:teuthology.orchestra.run.smithi097.stdout:/ 2024-03-27T10:56:25.458 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/selinux 2024-03-27T10:56:25.459 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/sys/fs/binfmt_misc 2024-03-27T10:56:25.459 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/hugepages 2024-03-27T10:56:25.459 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/mqueue 2024-03-27T10:56:25.459 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/debug 2024-03-27T10:56:25.459 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/tracing 2024-03-27T10:56:25.459 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-27T10:56:25.459 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/fuse/connections 2024-03-27T10:56:25.459 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/config 2024-03-27T10:56:25.459 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-sysctl.service 2024-03-27T10:56:25.459 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-27T10:56:25.459 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/nfs/rpc_pipefs 2024-03-27T10:56:25.459 INFO:teuthology.orchestra.run.smithi097.stdout:/run/user/1000 2024-03-27T10:56:25.459 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph 2024-03-27T10:56:25.459 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/containers/storage/overlay 2024-03-27T10:56:25.460 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/containers/storage/overlay/fd52e3d90990cd52f9e47dd102deb1ea1680d0f0c4ab0d9ae240811634b3768d/merged 2024-03-27T10:56:25.460 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/containers/storage/overlay/fc41024860037e01df60c3d4a6f07f4d9d527b5dbef5fe3fad7b5c283469199c/merged 2024-03-27T10:56:25.460 INFO:teuthology.orchestra.run.smithi097.stdout:/run/user/0 2024-03-27T10:56:25.460 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/sys/fs/binfmt_misc 2024-03-27T10:56:25.460 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/containers/storage/overlay/72648f82c94a9036bb3f9d2091130cb193fc92102cea1674eb0ea680e16c2fa8/merged 2024-03-27T10:56:25.460 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/containers/storage/overlay/18b57e8dc21154edc4e609ec412c9049f0e30baeaae2e921ca6cf22fc9f1cfae/merged 2024-03-27T10:56:25.460 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/containers/storage/overlay/85f6d221948763f505cbce83e7395da5158f47d89bc27bf13e983fbc90970fe3/merged 2024-03-27T10:56:25.460 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/containers/storage/overlay/dbd5f08f69ed0f749ea5a54b9a8273e0e9ef7924679f1d3e4e1675c64aa70b7b/merged 2024-03-27T10:56:25.460 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/containers/storage/overlay/cb10540150b4485b1abdf9448da0ffda177f7195342c8f44d144812ebc3cda6e/merged 2024-03-27T10:56:25.460 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/containers/storage/overlay/28c9cb3198f2410789a2fe7fe6979c3aaefeacfc1d3fee1000689f976ff968c3/merged 2024-03-27T10:56:25.460 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/containers/storage/overlay/fd55632977228ad83453a03d787fd880cae87bcf6acae44c00cc6327c2d001b4/merged 2024-03-27T10:56:25.460 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/containers/storage/overlay/2de6114599b10bcc03a81253909baee34d1c6f505ff90072f903ecfd4877635c/merged 2024-03-27T10:56:25.460 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns 2024-03-27T10:56:25.460 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-27T10:56:25.461 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-27T10:56:25.461 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-27T10:56:25.461 DEBUG:teuthology.orchestra.run.smithi097:> ls /sys/fs/fuse/connections 2024-03-27T10:56:25.516 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:56:25 smithi114 ceph-mon[35338]: pgmap v434: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-27T10:56:25.553 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-27T10:56:25.553 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0 --client_mountpoint=/volumes/qa/sv_0/22bc04cd-8106-4d11-83f1-2aead5019d07 --client_fs=cephfs) 2024-03-27T10:56:25.557 DEBUG:teuthology.orchestra.run.smithi097:> sudo modprobe fuse 2024-03-27T10:56:25.629 DEBUG:teuthology.orchestra.run.smithi097:> cat /proc/self/mounts | awk '{print $2}' 2024-03-27T10:56:25.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:56:25 smithi177 ceph-mon[35294]: pgmap v434: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-27T10:56:25.679 INFO:teuthology.orchestra.run.smithi097.stdout:/proc 2024-03-27T10:56:25.679 INFO:teuthology.orchestra.run.smithi097.stdout:/sys 2024-03-27T10:56:25.679 INFO:teuthology.orchestra.run.smithi097.stdout:/dev 2024-03-27T10:56:25.680 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/security 2024-03-27T10:56:25.680 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/shm 2024-03-27T10:56:25.680 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/pts 2024-03-27T10:56:25.680 INFO:teuthology.orchestra.run.smithi097.stdout:/run 2024-03-27T10:56:25.680 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/cgroup 2024-03-27T10:56:25.680 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/pstore 2024-03-27T10:56:25.680 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/bpf 2024-03-27T10:56:25.680 INFO:teuthology.orchestra.run.smithi097.stdout:/ 2024-03-27T10:56:25.680 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/selinux 2024-03-27T10:56:25.680 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/sys/fs/binfmt_misc 2024-03-27T10:56:25.680 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/hugepages 2024-03-27T10:56:25.680 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/mqueue 2024-03-27T10:56:25.681 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/debug 2024-03-27T10:56:25.681 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/tracing 2024-03-27T10:56:25.681 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-27T10:56:25.681 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/fuse/connections 2024-03-27T10:56:25.681 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/config 2024-03-27T10:56:25.681 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-sysctl.service 2024-03-27T10:56:25.681 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-27T10:56:25.681 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/nfs/rpc_pipefs 2024-03-27T10:56:25.681 INFO:teuthology.orchestra.run.smithi097.stdout:/run/user/1000 2024-03-27T10:56:25.681 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph 2024-03-27T10:56:25.682 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/containers/storage/overlay 2024-03-27T10:56:25.682 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/containers/storage/overlay/fd52e3d90990cd52f9e47dd102deb1ea1680d0f0c4ab0d9ae240811634b3768d/merged 2024-03-27T10:56:25.682 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/containers/storage/overlay/fc41024860037e01df60c3d4a6f07f4d9d527b5dbef5fe3fad7b5c283469199c/merged 2024-03-27T10:56:25.682 INFO:teuthology.orchestra.run.smithi097.stdout:/run/user/0 2024-03-27T10:56:25.682 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/sys/fs/binfmt_misc 2024-03-27T10:56:25.682 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/containers/storage/overlay/72648f82c94a9036bb3f9d2091130cb193fc92102cea1674eb0ea680e16c2fa8/merged 2024-03-27T10:56:25.682 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/containers/storage/overlay/18b57e8dc21154edc4e609ec412c9049f0e30baeaae2e921ca6cf22fc9f1cfae/merged 2024-03-27T10:56:25.682 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/containers/storage/overlay/85f6d221948763f505cbce83e7395da5158f47d89bc27bf13e983fbc90970fe3/merged 2024-03-27T10:56:25.682 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/containers/storage/overlay/dbd5f08f69ed0f749ea5a54b9a8273e0e9ef7924679f1d3e4e1675c64aa70b7b/merged 2024-03-27T10:56:25.682 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/containers/storage/overlay/cb10540150b4485b1abdf9448da0ffda177f7195342c8f44d144812ebc3cda6e/merged 2024-03-27T10:56:25.682 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/containers/storage/overlay/28c9cb3198f2410789a2fe7fe6979c3aaefeacfc1d3fee1000689f976ff968c3/merged 2024-03-27T10:56:25.683 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/containers/storage/overlay/fd55632977228ad83453a03d787fd880cae87bcf6acae44c00cc6327c2d001b4/merged 2024-03-27T10:56:25.683 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/containers/storage/overlay/2de6114599b10bcc03a81253909baee34d1c6f505ff90072f903ecfd4877635c/merged 2024-03-27T10:56:25.683 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns 2024-03-27T10:56:25.683 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-27T10:56:25.683 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-27T10:56:25.684 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-27T10:56:25.684 DEBUG:teuthology.orchestra.run.smithi097:> ls /sys/fs/fuse/connections 2024-03-27T10:56:25.778 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:2024-03-27T10:56:25.807+0000 7f69764d6480 -1 init, newargv = 0x5644067ee470 newargc=15 2024-03-27T10:56:25.778 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:ceph-fuse[84404]: starting ceph client 2024-03-27T10:56:25.818 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:ceph-fuse[84404]: starting fuse 2024-03-27T10:56:26.872 DEBUG:teuthology.orchestra.run.smithi097:> sudo modprobe fuse 2024-03-27T10:56:26.979 DEBUG:teuthology.orchestra.run.smithi097:> cat /proc/self/mounts | awk '{print $2}' 2024-03-27T10:56:27.032 INFO:teuthology.orchestra.run.smithi097.stdout:/proc 2024-03-27T10:56:27.032 INFO:teuthology.orchestra.run.smithi097.stdout:/sys 2024-03-27T10:56:27.032 INFO:teuthology.orchestra.run.smithi097.stdout:/dev 2024-03-27T10:56:27.032 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/security 2024-03-27T10:56:27.032 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/shm 2024-03-27T10:56:27.032 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/pts 2024-03-27T10:56:27.032 INFO:teuthology.orchestra.run.smithi097.stdout:/run 2024-03-27T10:56:27.032 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/cgroup 2024-03-27T10:56:27.032 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/pstore 2024-03-27T10:56:27.033 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/bpf 2024-03-27T10:56:27.033 INFO:teuthology.orchestra.run.smithi097.stdout:/ 2024-03-27T10:56:27.033 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/selinux 2024-03-27T10:56:27.033 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/sys/fs/binfmt_misc 2024-03-27T10:56:27.033 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/hugepages 2024-03-27T10:56:27.033 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/mqueue 2024-03-27T10:56:27.033 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/debug 2024-03-27T10:56:27.033 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/tracing 2024-03-27T10:56:27.033 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-27T10:56:27.033 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/fuse/connections 2024-03-27T10:56:27.033 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/config 2024-03-27T10:56:27.033 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-sysctl.service 2024-03-27T10:56:27.033 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-27T10:56:27.034 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/nfs/rpc_pipefs 2024-03-27T10:56:27.034 INFO:teuthology.orchestra.run.smithi097.stdout:/run/user/1000 2024-03-27T10:56:27.034 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph 2024-03-27T10:56:27.034 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/containers/storage/overlay 2024-03-27T10:56:27.034 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/containers/storage/overlay/fd52e3d90990cd52f9e47dd102deb1ea1680d0f0c4ab0d9ae240811634b3768d/merged 2024-03-27T10:56:27.034 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/containers/storage/overlay/fc41024860037e01df60c3d4a6f07f4d9d527b5dbef5fe3fad7b5c283469199c/merged 2024-03-27T10:56:27.034 INFO:teuthology.orchestra.run.smithi097.stdout:/run/user/0 2024-03-27T10:56:27.034 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/sys/fs/binfmt_misc 2024-03-27T10:56:27.034 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/containers/storage/overlay/72648f82c94a9036bb3f9d2091130cb193fc92102cea1674eb0ea680e16c2fa8/merged 2024-03-27T10:56:27.034 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/containers/storage/overlay/18b57e8dc21154edc4e609ec412c9049f0e30baeaae2e921ca6cf22fc9f1cfae/merged 2024-03-27T10:56:27.034 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/containers/storage/overlay/85f6d221948763f505cbce83e7395da5158f47d89bc27bf13e983fbc90970fe3/merged 2024-03-27T10:56:27.034 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/containers/storage/overlay/dbd5f08f69ed0f749ea5a54b9a8273e0e9ef7924679f1d3e4e1675c64aa70b7b/merged 2024-03-27T10:56:27.035 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/containers/storage/overlay/cb10540150b4485b1abdf9448da0ffda177f7195342c8f44d144812ebc3cda6e/merged 2024-03-27T10:56:27.035 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/containers/storage/overlay/28c9cb3198f2410789a2fe7fe6979c3aaefeacfc1d3fee1000689f976ff968c3/merged 2024-03-27T10:56:27.035 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/containers/storage/overlay/fd55632977228ad83453a03d787fd880cae87bcf6acae44c00cc6327c2d001b4/merged 2024-03-27T10:56:27.035 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/containers/storage/overlay/2de6114599b10bcc03a81253909baee34d1c6f505ff90072f903ecfd4877635c/merged 2024-03-27T10:56:27.035 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns 2024-03-27T10:56:27.035 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-27T10:56:27.035 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-27T10:56:27.035 INFO:teuthology.orchestra.run.smithi097.stdout:/home/ubuntu/cephtest/mnt.0 2024-03-27T10:56:27.036 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-27T10:56:27.036 DEBUG:teuthology.orchestra.run.smithi097:> ls /sys/fs/fuse/connections 2024-03-27T10:56:27.085 INFO:teuthology.orchestra.run.smithi097.stdout:118 2024-03-27T10:56:27.085 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [118] 2024-03-27T10:56:27.086 DEBUG:teuthology.orchestra.run.smithi097:> sudo stdin-killer -- python3 -c ' 2024-03-27T10:56:27.086 DEBUG:teuthology.orchestra.run.smithi097:> import glob 2024-03-27T10:56:27.086 DEBUG:teuthology.orchestra.run.smithi097:> import re 2024-03-27T10:56:27.086 DEBUG:teuthology.orchestra.run.smithi097:> import os 2024-03-27T10:56:27.086 DEBUG:teuthology.orchestra.run.smithi097:> import subprocess 2024-03-27T10:56:27.086 DEBUG:teuthology.orchestra.run.smithi097:> 2024-03-27T10:56:27.086 DEBUG:teuthology.orchestra.run.smithi097:> def _find_admin_socket(client_name): 2024-03-27T10:56:27.086 DEBUG:teuthology.orchestra.run.smithi097:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-03-27T10:56:27.086 DEBUG:teuthology.orchestra.run.smithi097:> files = glob.glob(asok_path) 2024-03-27T10:56:27.086 DEBUG:teuthology.orchestra.run.smithi097:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-03-27T10:56:27.086 DEBUG:teuthology.orchestra.run.smithi097:> 2024-03-27T10:56:27.086 DEBUG:teuthology.orchestra.run.smithi097:> # Given a non-glob path, it better be there 2024-03-27T10:56:27.086 DEBUG:teuthology.orchestra.run.smithi097:> if "*" not in asok_path: 2024-03-27T10:56:27.087 DEBUG:teuthology.orchestra.run.smithi097:> assert(len(files) == 1) 2024-03-27T10:56:27.087 DEBUG:teuthology.orchestra.run.smithi097:> return files[0] 2024-03-27T10:56:27.087 DEBUG:teuthology.orchestra.run.smithi097:> 2024-03-27T10:56:27.087 DEBUG:teuthology.orchestra.run.smithi097:> for f in files: 2024-03-27T10:56:27.087 DEBUG:teuthology.orchestra.run.smithi097:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-27T10:56:27.087 DEBUG:teuthology.orchestra.run.smithi097:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-27T10:56:27.087 DEBUG:teuthology.orchestra.run.smithi097:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-27T10:56:27.087 DEBUG:teuthology.orchestra.run.smithi097:> contents = proc_f.read() 2024-03-27T10:56:27.087 DEBUG:teuthology.orchestra.run.smithi097:> if mountpoint in contents: 2024-03-27T10:56:27.087 DEBUG:teuthology.orchestra.run.smithi097:> return f 2024-03-27T10:56:27.087 DEBUG:teuthology.orchestra.run.smithi097:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-27T10:56:27.087 DEBUG:teuthology.orchestra.run.smithi097:> 2024-03-27T10:56:27.087 DEBUG:teuthology.orchestra.run.smithi097:> print(_find_admin_socket("client.0")) 2024-03-27T10:56:27.087 DEBUG:teuthology.orchestra.run.smithi097:> ' 2024-03-27T10:56:27.259 INFO:teuthology.orchestra.run.smithi097.stdout:/var/run/ceph/ceph-client.0.84404.asok 2024-03-27T10:56:27.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-27T10:56:27.278 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.84404.asok 2024-03-27T10:56:27.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-27T10:56:27.279 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.84404.asok status 2024-03-27T10:56:27.517 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-03-27T10:56:27.517 INFO:teuthology.orchestra.run.smithi097.stdout: "metadata": { 2024-03-27T10:56:27.517 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_sha1": "155268c4e432a12433aa833f174f9fe3b1016ae0", 2024-03-27T10:56:27.517 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_version": "ceph version 19.0.0-2478-g155268c4 (155268c4e432a12433aa833f174f9fe3b1016ae0) squid (dev)", 2024-03-27T10:56:27.517 INFO:teuthology.orchestra.run.smithi097.stdout: "entity_id": "0", 2024-03-27T10:56:27.517 INFO:teuthology.orchestra.run.smithi097.stdout: "hostname": "smithi097", 2024-03-27T10:56:27.517 INFO:teuthology.orchestra.run.smithi097.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-03-27T10:56:27.517 INFO:teuthology.orchestra.run.smithi097.stdout: "pid": "84404", 2024-03-27T10:56:27.517 INFO:teuthology.orchestra.run.smithi097.stdout: "root": "/volumes/qa/sv_0/22bc04cd-8106-4d11-83f1-2aead5019d07" 2024-03-27T10:56:27.517 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T10:56:27.518 INFO:teuthology.orchestra.run.smithi097.stdout: "dentry_count": 0, 2024-03-27T10:56:27.518 INFO:teuthology.orchestra.run.smithi097.stdout: "dentry_pinned_count": 0, 2024-03-27T10:56:27.518 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 15363, 2024-03-27T10:56:27.518 INFO:teuthology.orchestra.run.smithi097.stdout: "inst": { 2024-03-27T10:56:27.518 INFO:teuthology.orchestra.run.smithi097.stdout: "name": { 2024-03-27T10:56:27.518 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "client", 2024-03-27T10:56:27.518 INFO:teuthology.orchestra.run.smithi097.stdout: "num": 15363 2024-03-27T10:56:27.518 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T10:56:27.518 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": { 2024-03-27T10:56:27.518 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2024-03-27T10:56:27.518 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "192.168.0.1:0", 2024-03-27T10:56:27.518 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 1182911793 2024-03-27T10:56:27.518 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T10:56:27.519 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T10:56:27.519 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": { 2024-03-27T10:56:27.519 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2024-03-27T10:56:27.519 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "192.168.0.1:0", 2024-03-27T10:56:27.519 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 1182911793 2024-03-27T10:56:27.519 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T10:56:27.519 INFO:teuthology.orchestra.run.smithi097.stdout: "inst_str": "client.15363 192.168.0.1:0/1182911793", 2024-03-27T10:56:27.519 INFO:teuthology.orchestra.run.smithi097.stdout: "addr_str": "192.168.0.1:0/1182911793", 2024-03-27T10:56:27.519 INFO:teuthology.orchestra.run.smithi097.stdout: "inode_count": 5, 2024-03-27T10:56:27.519 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_epoch": 27, 2024-03-27T10:56:27.519 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_epoch": 92, 2024-03-27T10:56:27.519 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_epoch_barrier": 0, 2024-03-27T10:56:27.519 INFO:teuthology.orchestra.run.smithi097.stdout: "blocklisted": false, 2024-03-27T10:56:27.519 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs" 2024-03-27T10:56:27.519 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-03-27T10:56:27.528 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:56:27 smithi097 ceph-mon[31859]: pgmap v435: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-27T10:56:27.532 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-27T10:56:27.532 DEBUG:teuthology.orchestra.run.smithi097:> stat --file-system '--printf=%T 2024-03-27T10:56:27.533 DEBUG:teuthology.orchestra.run.smithi097:> ' -- /home/ubuntu/cephtest/mnt.0 2024-03-27T10:56:27.580 INFO:teuthology.orchestra.run.smithi097.stdout:fuseblk 2024-03-27T10:56:27.581 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-03-27T10:56:27.581 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-27T10:56:27.581 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-27T10:56:27.675 INFO:teuthology.run_tasks:Running task exec... 2024-03-27T10:56:27.686 INFO:teuthology.task.exec:Executing custom commands... 2024-03-27T10:56:27.686 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi097.front.sepia.ceph.com 2024-03-27T10:56:27.686 DEBUG:teuthology.orchestra.run.smithi097:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-03-27T10:56:27.687 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:56:27 smithi177 ceph-mon[35294]: pgmap v435: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-27T10:56:27.766 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:56:27 smithi114 ceph-mon[35338]: pgmap v435: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-27T10:56:27.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:27.918+0000 7fba94200640 1 -- 172.21.15.97:0/2757538503 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba8c108b50 msgr2=0x7fba8c10af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:27.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:27.918+0000 7fba94200640 1 --2- 172.21.15.97:0/2757538503 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba8c108b50 0x7fba8c10af80 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fba80009900 tx=0x7fba8002bcd0 comp rx=0 tx=0).stop 2024-03-27T10:56:27.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:27.918+0000 7fba94200640 1 -- 172.21.15.97:0/2757538503 shutdown_connections 2024-03-27T10:56:27.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:27.918+0000 7fba94200640 1 --2- 172.21.15.97:0/2757538503 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fba8c10b940 0x7fba8c1147e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:27.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:27.918+0000 7fba94200640 1 --2- 172.21.15.97:0/2757538503 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fba8c10b4c0 0x7fba8c112290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:27.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:27.918+0000 7fba94200640 1 --2- 172.21.15.97:0/2757538503 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba8c108b50 0x7fba8c10af80 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:27.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:27.918+0000 7fba94200640 1 -- 172.21.15.97:0/2757538503 >> 172.21.15.97:0/2757538503 conn(0x7fba8c06ef80 msgr2=0x7fba8c0713c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:27.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:27.919+0000 7fba94200640 1 -- 172.21.15.97:0/2757538503 shutdown_connections 2024-03-27T10:56:27.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:27.919+0000 7fba94200640 1 -- 172.21.15.97:0/2757538503 wait complete. 2024-03-27T10:56:27.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:27.920+0000 7fba94200640 1 Processor -- start 2024-03-27T10:56:27.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:27.920+0000 7fba94200640 1 -- start start 2024-03-27T10:56:27.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:27.920+0000 7fba94200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fba8c108b50 0x7fba8c1a2750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:27.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:27.920+0000 7fba94200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba8c10b4c0 0x7fba8c1a2c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:27.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:27.920+0000 7fba94200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fba8c10b940 0x7fba8c1a9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:27.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:27.920+0000 7fba94200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fba8c118100 con 0x7fba8c10b4c0 2024-03-27T10:56:27.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:27.920+0000 7fba94200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fba8c117f80 con 0x7fba8c10b940 2024-03-27T10:56:27.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:27.920+0000 7fba94200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fba8c118280 con 0x7fba8c108b50 2024-03-27T10:56:27.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:27.920+0000 7fba90e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba8c10b4c0 0x7fba8c1a2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:27.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:27.920+0000 7fba90e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba8c10b4c0 0x7fba8c1a2c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42548/0 (socket says 172.21.15.97:42548) 2024-03-27T10:56:27.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:27.920+0000 7fba90e00640 1 -- 172.21.15.97:0/3201667636 learned_addr learned my addr 172.21.15.97:0/3201667636 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:56:27.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:27.921+0000 7fba91800640 1 --2- 172.21.15.97:0/3201667636 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fba8c108b50 0x7fba8c1a2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:27.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:27.921+0000 7fba92200640 1 --2- 172.21.15.97:0/3201667636 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fba8c10b940 0x7fba8c1a9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:27.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:27.921+0000 7fba90e00640 1 -- 172.21.15.97:0/3201667636 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fba8c108b50 msgr2=0x7fba8c1a2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:27.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:27.921+0000 7fba90e00640 1 --2- 172.21.15.97:0/3201667636 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fba8c108b50 0x7fba8c1a2750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:27.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:27.921+0000 7fba90e00640 1 -- 172.21.15.97:0/3201667636 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fba8c10b940 msgr2=0x7fba8c1a9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:27.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:27.921+0000 7fba90e00640 1 --2- 172.21.15.97:0/3201667636 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fba8c10b940 0x7fba8c1a9cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:27.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:27.921+0000 7fba90e00640 1 -- 172.21.15.97:0/3201667636 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba8c1aa320 con 0x7fba8c10b4c0 2024-03-27T10:56:27.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:27.921+0000 7fba90e00640 1 --2- 172.21.15.97:0/3201667636 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba8c10b4c0 0x7fba8c1a2c90 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fba7800d770 tx=0x7fba7800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:27.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:27.922+0000 7fba8a000640 1 -- 172.21.15.97:0/3201667636 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba78017070 con 0x7fba8c10b4c0 2024-03-27T10:56:27.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:27.922+0000 7fba94200640 1 -- 172.21.15.97:0/3201667636 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba8c1aa610 con 0x7fba8c10b4c0 2024-03-27T10:56:27.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:27.922+0000 7fba8a000640 1 -- 172.21.15.97:0/3201667636 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fba78009c30 con 0x7fba8c10b4c0 2024-03-27T10:56:27.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:27.922+0000 7fba8a000640 1 -- 172.21.15.97:0/3201667636 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba78003d00 con 0x7fba8c10b4c0 2024-03-27T10:56:27.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:27.922+0000 7fba94200640 1 -- 172.21.15.97:0/3201667636 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba8c1aabb0 con 0x7fba8c10b4c0 2024-03-27T10:56:27.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:27.923+0000 7fba94200640 1 -- 172.21.15.97:0/3201667636 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba8c1139a0 con 0x7fba8c10b4c0 2024-03-27T10:56:27.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:27.926+0000 7fba8a000640 1 -- 172.21.15.97:0/3201667636 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7fba78003ea0 con 0x7fba8c10b4c0 2024-03-27T10:56:27.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:27.928+0000 7fba8a000640 1 --2- 172.21.15.97:0/3201667636 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fba740a3730 0x7fba740a5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:27.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:27.928+0000 7fba8a000640 1 -- 172.21.15.97:0/3201667636 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fba780cfa20 con 0x7fba8c10b4c0 2024-03-27T10:56:27.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:27.928+0000 7fba91800640 1 --2- 172.21.15.97:0/3201667636 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fba740a3730 0x7fba740a5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:27.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:27.929+0000 7fba91800640 1 --2- 172.21.15.97:0/3201667636 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fba740a3730 0x7fba740a5bf0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fba80000c00 tx=0x7fba8002c880 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:27.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:27.933+0000 7fba8a000640 1 -- 172.21.15.97:0/3201667636 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7fba7809c5a0 con 0x7fba8c10b4c0 2024-03-27T10:56:28.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:28.178+0000 7fba94200640 1 -- 172.21.15.97:0/3201667636 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7fba8c062750 con 0x7fba8c10b4c0 2024-03-27T10:56:28.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:28.313+0000 7fba8a000640 1 -- 172.21.15.97:0/3201667636 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v28) ==== 117+0+0 (secure 0 0 0) 0x7fba78009de0 con 0x7fba8c10b4c0 2024-03-27T10:56:28.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:28.317+0000 7fba94200640 1 -- 172.21.15.97:0/3201667636 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fba740a3730 msgr2=0x7fba740a5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:28.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:28.317+0000 7fba94200640 1 --2- 172.21.15.97:0/3201667636 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fba740a3730 0x7fba740a5bf0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fba80000c00 tx=0x7fba8002c880 comp rx=0 tx=0).stop 2024-03-27T10:56:28.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:28.317+0000 7fba94200640 1 -- 172.21.15.97:0/3201667636 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba8c10b4c0 msgr2=0x7fba8c1a2c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:28.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:28.317+0000 7fba94200640 1 --2- 172.21.15.97:0/3201667636 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba8c10b4c0 0x7fba8c1a2c90 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fba7800d770 tx=0x7fba7800dc30 comp rx=0 tx=0).stop 2024-03-27T10:56:28.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:28.317+0000 7fba94200640 1 -- 172.21.15.97:0/3201667636 shutdown_connections 2024-03-27T10:56:28.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:28.317+0000 7fba94200640 1 --2- 172.21.15.97:0/3201667636 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fba740a3730 0x7fba740a5bf0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:28.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:28.317+0000 7fba94200640 1 --2- 172.21.15.97:0/3201667636 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fba8c10b940 0x7fba8c1a9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:28.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:28.317+0000 7fba94200640 1 --2- 172.21.15.97:0/3201667636 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba8c10b4c0 0x7fba8c1a2c90 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:28.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:28.317+0000 7fba94200640 1 --2- 172.21.15.97:0/3201667636 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fba8c108b50 0x7fba8c1a2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:28.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:28.317+0000 7fba94200640 1 -- 172.21.15.97:0/3201667636 >> 172.21.15.97:0/3201667636 conn(0x7fba8c06ef80 msgr2=0x7fba8c071130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:28.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:28.317+0000 7fba94200640 1 -- 172.21.15.97:0/3201667636 shutdown_connections 2024-03-27T10:56:28.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:28.317+0000 7fba94200640 1 -- 172.21.15.97:0/3201667636 wait complete. 2024-03-27T10:56:28.309 INFO:teuthology.run_tasks:Running task check-counter... 2024-03-27T10:56:28.320 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-03-27T10:56:28.332 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-27T10:56:28.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:28.580+0000 7f1d8aa00640 1 -- 172.21.15.97:0/2592627750 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1d84104d70 msgr2=0x7f1d841051f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:28.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:28.580+0000 7f1d8aa00640 1 --2- 172.21.15.97:0/2592627750 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1d84104d70 0x7f1d841051f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1d70009900 tx=0x7f1d7002bc30 comp rx=0 tx=0).stop 2024-03-27T10:56:28.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:28.580+0000 7f1d8aa00640 1 -- 172.21.15.97:0/2592627750 shutdown_connections 2024-03-27T10:56:28.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:28.580+0000 7f1d8aa00640 1 --2- 172.21.15.97:0/2592627750 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d84105730 0x7f1d8410bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:28.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:28.580+0000 7f1d8aa00640 1 --2- 172.21.15.97:0/2592627750 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1d84104d70 0x7f1d841051f0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:28.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:28.580+0000 7f1d8aa00640 1 --2- 172.21.15.97:0/2592627750 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d84103b80 0x7f1d84103fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:28.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:28.580+0000 7f1d8aa00640 1 -- 172.21.15.97:0/2592627750 >> 172.21.15.97:0/2592627750 conn(0x7f1d840ff0f0 msgr2=0x7f1d84101530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:28.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:28.580+0000 7f1d8aa00640 1 -- 172.21.15.97:0/2592627750 shutdown_connections 2024-03-27T10:56:28.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:28.580+0000 7f1d8aa00640 1 -- 172.21.15.97:0/2592627750 wait complete. 2024-03-27T10:56:28.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:28.581+0000 7f1d8aa00640 1 Processor -- start 2024-03-27T10:56:28.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:28.581+0000 7f1d8aa00640 1 -- start start 2024-03-27T10:56:28.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:28.582+0000 7f1d8aa00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1d84103b80 0x7f1d84199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:28.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:28.582+0000 7f1d8aa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d84104d70 0x7f1d8419a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:28.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:28.582+0000 7f1d8aa00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d84105730 0x7f1d841a1300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:28.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:28.582+0000 7f1d8aa00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1d8410f6b0 con 0x7f1d84104d70 2024-03-27T10:56:28.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:28.582+0000 7f1d8aa00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f1d8410f530 con 0x7f1d84103b80 2024-03-27T10:56:28.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:28.582+0000 7f1d8aa00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1d8410f830 con 0x7f1d84105730 2024-03-27T10:56:28.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:28.582+0000 7f1d88c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d84104d70 0x7f1d8419a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:28.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:28.582+0000 7f1d88c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d84104d70 0x7f1d8419a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42568/0 (socket says 172.21.15.97:42568) 2024-03-27T10:56:28.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:28.582+0000 7f1d88c00640 1 -- 172.21.15.97:0/1700934925 learned_addr learned my addr 172.21.15.97:0/1700934925 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:56:28.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:28.582+0000 7f1d89600640 1 --2- 172.21.15.97:0/1700934925 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1d84103b80 0x7f1d84199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:28.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:28.582+0000 7f1d8a000640 1 --2- 172.21.15.97:0/1700934925 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d84105730 0x7f1d841a1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:28.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:28.582+0000 7f1d88c00640 1 -- 172.21.15.97:0/1700934925 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d84105730 msgr2=0x7f1d841a1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:28.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:28.582+0000 7f1d88c00640 1 --2- 172.21.15.97:0/1700934925 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d84105730 0x7f1d841a1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:28.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:28.582+0000 7f1d88c00640 1 -- 172.21.15.97:0/1700934925 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1d84103b80 msgr2=0x7f1d84199d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:28.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:28.582+0000 7f1d88c00640 1 --2- 172.21.15.97:0/1700934925 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1d84103b80 0x7f1d84199d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:28.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:28.582+0000 7f1d88c00640 1 -- 172.21.15.97:0/1700934925 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d841a1970 con 0x7f1d84104d70 2024-03-27T10:56:28.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:28.582+0000 7f1d89600640 1 --2- 172.21.15.97:0/1700934925 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1d84103b80 0x7f1d84199d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T10:56:28.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:28.582+0000 7f1d8a000640 1 --2- 172.21.15.97:0/1700934925 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d84105730 0x7f1d841a1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T10:56:28.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:28.583+0000 7f1d88c00640 1 --2- 172.21.15.97:0/1700934925 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d84104d70 0x7f1d8419a290 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f1d70006010 tx=0x7f1d70005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:28.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:28.583+0000 7f1d82000640 1 -- 172.21.15.97:0/1700934925 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d70043020 con 0x7f1d84104d70 2024-03-27T10:56:28.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:28.583+0000 7f1d82000640 1 -- 172.21.15.97:0/1700934925 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1d700043d0 con 0x7f1d84104d70 2024-03-27T10:56:28.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:28.583+0000 7f1d8aa00640 1 -- 172.21.15.97:0/1700934925 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d841a1c00 con 0x7f1d84104d70 2024-03-27T10:56:28.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:28.583+0000 7f1d82000640 1 -- 172.21.15.97:0/1700934925 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d70003b80 con 0x7f1d84104d70 2024-03-27T10:56:28.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:28.583+0000 7f1d8aa00640 1 -- 172.21.15.97:0/1700934925 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d841a2150 con 0x7f1d84104d70 2024-03-27T10:56:28.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:28.584+0000 7f1d8aa00640 1 -- 172.21.15.97:0/1700934925 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d8410aea0 con 0x7f1d84104d70 2024-03-27T10:56:28.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:28.586+0000 7f1d82000640 1 -- 172.21.15.97:0/1700934925 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f1d70046020 con 0x7f1d84104d70 2024-03-27T10:56:28.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:28.588+0000 7f1d82000640 1 --2- 172.21.15.97:0/1700934925 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f1d6c0a3610 0x7f1d6c0a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:28.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:28.588+0000 7f1d82000640 1 -- 172.21.15.97:0/1700934925 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f1d700f26e0 con 0x7f1d84104d70 2024-03-27T10:56:28.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:28.588+0000 7f1d89600640 1 --2- 172.21.15.97:0/1700934925 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f1d6c0a3610 0x7f1d6c0a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:28.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:28.589+0000 7f1d89600640 1 --2- 172.21.15.97:0/1700934925 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f1d6c0a3610 0x7f1d6c0a5ad0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1d84104bd0 tx=0x7f1d78005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:28.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:28.590+0000 7f1d82000640 1 -- 172.21.15.97:0/1700934925 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f1d7004d020 con 0x7f1d84104d70 2024-03-27T10:56:28.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:56:28 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/3201667636' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-03-27T10:56:28.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:56:28 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/3201667636' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-03-27T10:56:28.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:28.791+0000 7f1d8aa00640 1 -- 172.21.15.97:0/1700934925 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1d8402d5e0 con 0x7f1d84104d70 2024-03-27T10:56:28.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:28.794+0000 7f1d82000640 1 -- 172.21.15.97:0/1700934925 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+22644 (secure 0 0 0) 0x7f1d700bf260 con 0x7f1d84104d70 2024-03-27T10:56:28.767 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T10:56:28.767 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":92,"fsid":"adca88c4-ec26-11ee-95d0-87774f69a715","created":"2024-03-27T10:43:24.680694+0000","modified":"2024-03-27T10:54:41.310812+0000","last_up_change":"2024-03-27T10:53:00.552513+0000","last_in_change":"2024-03-27T10:52:36.302986+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-27T10:47:54.052810+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-27T10:54:02.845060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-27T10:54:03.474151+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b634e5f1-6118-4e2f-95e9-830db390f8d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1746812080},{"type":"v1","addr":"172.21.15.97:6803","nonce":1746812080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1746812080},{"type":"v1","addr":"172.21.15.97:6805","nonce":1746812080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":1746812080},{"type":"v1","addr":"172.21.15.97:6809","nonce":1746812080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1746812080},{"type":"v1","addr":"172.21.15.97:6807","nonce":1746812080}]},"public_addr":"172.21.15.97:6803/1746812080","cluster_addr":"172.21.15.97:6805/1746812080","heartbeat_back_addr":"172.21.15.97:6809/1746812080","heartbeat_front_addr":"172.21.15.97:6807/1746812080","state":["exists","up"]},{"osd":1,"uuid":"441e2770-3284-4388-b0de-c56eb8a373d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":3858821264},{"type":"v1","addr":"172.21.15.114:6801","nonce":3858821264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":3858821264},{"type":"v1","addr":"172.21.15.114:6803","nonce":3858821264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":3858821264},{"type":"v1","addr":"172.21.15.114:6807","nonce":3858821264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":3858821264},{"type":"v1","addr":"172.21.15.114:6805","nonce":3858821264}]},"public_addr":"172.21.15.114:6801/3858821264","cluster_addr":"172.21.15.114:6803/3858821264","heartbeat_back_addr":"172.21.15.114:6807/3858821264","heartbeat_front_addr":"172.21.15.114:6805/3858821264","state":["exists","up"]},{"osd":2,"uuid":"00108864-1aa6-48d6-8398-e8eb79fda123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":124240658},{"type":"v1","addr":"172.21.15.177:6801","nonce":124240658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":124240658},{"type":"v1","addr":"172.21.15.177:6803","nonce":124240658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":124240658},{"type":"v1","addr":"172.21.15.177:6807","nonce":124240658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":124240658},{"type":"v1","addr":"172.21.15.177:6805","nonce":124240658}]},"public_addr":"172.21.15.177:6801/124240658","cluster_addr":"172.21.15.177:6803/124240658","heartbeat_back_addr":"172.21.15.177:6807/124240658","heartbeat_front_addr":"172.21.15.177:6805/124240658","state":["exists","up"]},{"osd":3,"uuid":"1860b690-e2dc-4977-88d7-d1aacf7022ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2478179311},{"type":"v1","addr":"172.21.15.97:6811","nonce":2478179311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2478179311},{"type":"v1","addr":"172.21.15.97:6813","nonce":2478179311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":2478179311},{"type":"v1","addr":"172.21.15.97:6817","nonce":2478179311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2478179311},{"type":"v1","addr":"172.21.15.97:6815","nonce":2478179311}]},"public_addr":"172.21.15.97:6811/2478179311","cluster_addr":"172.21.15.97:6813/2478179311","heartbeat_back_addr":"172.21.15.97:6817/2478179311","heartbeat_front_addr":"172.21.15.97:6815/2478179311","state":["exists","up"]},{"osd":4,"uuid":"9245a3a4-7d49-4fdb-9d55-112e43177050","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":75755945},{"type":"v1","addr":"172.21.15.114:6809","nonce":75755945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":75755945},{"type":"v1","addr":"172.21.15.114:6811","nonce":75755945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":75755945},{"type":"v1","addr":"172.21.15.114:6815","nonce":75755945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":75755945},{"type":"v1","addr":"172.21.15.114:6813","nonce":75755945}]},"public_addr":"172.21.15.114:6809/75755945","cluster_addr":"172.21.15.114:6811/75755945","heartbeat_back_addr":"172.21.15.114:6815/75755945","heartbeat_front_addr":"172.21.15.114:6813/75755945","state":["exists","up"]},{"osd":5,"uuid":"51922f99-e8d3-461b-8615-e9bfdcbf5ed3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":1918493790},{"type":"v1","addr":"172.21.15.177:6809","nonce":1918493790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":1918493790},{"type":"v1","addr":"172.21.15.177:6811","nonce":1918493790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":1918493790},{"type":"v1","addr":"172.21.15.177:6815","nonce":1918493790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":1918493790},{"type":"v1","addr":"172.21.15.177:6813","nonce":1918493790}]},"public_addr":"172.21.15.177:6809/1918493790","cluster_addr":"172.21.15.177:6811/1918493790","heartbeat_back_addr":"172.21.15.177:6815/1918493790","heartbeat_front_addr":"172.21.15.177:6813/1918493790","state":["exists","up"]},{"osd":6,"uuid":"06dffb32-05eb-43b3-852e-915c5cc7f25a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2892389563},{"type":"v1","addr":"172.21.15.97:6819","nonce":2892389563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2892389563},{"type":"v1","addr":"172.21.15.97:6821","nonce":2892389563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2892389563},{"type":"v1","addr":"172.21.15.97:6825","nonce":2892389563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2892389563},{"type":"v1","addr":"172.21.15.97:6823","nonce":2892389563}]},"public_addr":"172.21.15.97:6819/2892389563","cluster_addr":"172.21.15.97:6821/2892389563","heartbeat_back_addr":"172.21.15.97:6825/2892389563","heartbeat_front_addr":"172.21.15.97:6823/2892389563","state":["exists","up"]},{"osd":7,"uuid":"e68e3519-9c1b-43c5-8e06-5385248b19df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":2445491153},{"type":"v1","addr":"172.21.15.114:6817","nonce":2445491153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":2445491153},{"type":"v1","addr":"172.21.15.114:6819","nonce":2445491153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2445491153},{"type":"v1","addr":"172.21.15.114:6823","nonce":2445491153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":2445491153},{"type":"v1","addr":"172.21.15.114:6821","nonce":2445491153}]},"public_addr":"172.21.15.114:6817/2445491153","cluster_addr":"172.21.15.114:6819/2445491153","heartbeat_back_addr":"172.21.15.114:6823/2445491153","heartbeat_front_addr":"172.21.15.114:6821/2445491153","state":["exists","up"]},{"osd":8,"uuid":"31ac592e-1349-49ee-9af9-8ad85d682978","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":2051134191},{"type":"v1","addr":"172.21.15.177:6817","nonce":2051134191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":2051134191},{"type":"v1","addr":"172.21.15.177:6819","nonce":2051134191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":2051134191},{"type":"v1","addr":"172.21.15.177:6823","nonce":2051134191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":2051134191},{"type":"v1","addr":"172.21.15.177:6821","nonce":2051134191}]},"public_addr":"172.21.15.177:6817/2051134191","cluster_addr":"172.21.15.177:6819/2051134191","heartbeat_back_addr":"172.21.15.177:6823/2051134191","heartbeat_front_addr":"172.21.15.177:6821/2051134191","state":["exists","up"]},{"osd":9,"uuid":"0171086a-dd41-4eb4-a389-16f0c46a64db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":1578881271},{"type":"v1","addr":"172.21.15.97:6827","nonce":1578881271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1578881271},{"type":"v1","addr":"172.21.15.97:6829","nonce":1578881271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1578881271},{"type":"v1","addr":"172.21.15.97:6833","nonce":1578881271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1578881271},{"type":"v1","addr":"172.21.15.97:6831","nonce":1578881271}]},"public_addr":"172.21.15.97:6827/1578881271","cluster_addr":"172.21.15.97:6829/1578881271","heartbeat_back_addr":"172.21.15.97:6833/1578881271","heartbeat_front_addr":"172.21.15.97:6831/1578881271","state":["exists","up"]},{"osd":10,"uuid":"64754e31-b124-4601-ad98-2f70b1dd5359","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":2893111461},{"type":"v1","addr":"172.21.15.114:6825","nonce":2893111461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":2893111461},{"type":"v1","addr":"172.21.15.114:6827","nonce":2893111461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2893111461},{"type":"v1","addr":"172.21.15.114:6831","nonce":2893111461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":2893111461},{"type":"v1","addr":"172.21.15.114:6829","nonce":2893111461}]},"public_addr":"172.21.15.114:6825/2893111461","cluster_addr":"172.21.15.114:6827/2893111461","heartbeat_back_addr":"172.21.15.114:6831/2893111461","heartbeat_front_addr":"172.21.15.114:6829/2893111461","state":["exists","up"]},{"osd":11,"uuid":"9ffc752c-adf7-4037-89d3-535787d7475d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":85,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":2333030391},{"type":"v1","addr":"172.21.15.177:6825","nonce":2333030391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":2333030391},{"type":"v1","addr":"172.21.15.177:6827","nonce":2333030391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":2333030391},{"type":"v1","addr":"172.21.15.177:6831","nonce":2333030391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":2333030391},{"type":"v1","addr":"172.21.15.177:6829","nonce":2333030391}]},"public_addr":"172.21.15.177:6825/2333030391","cluster_addr":"172.21.15.177:6827/2333030391","heartbeat_back_addr":"172.21.15.177:6831/2333030391","heartbeat_front_addr":"172.21.15.177:6829/2333030391","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:46:45.143519+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:47:18.656947+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:47:51.316345+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:48:24.575512+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:48:59.113555+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:49:32.103865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:50:06.615539+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:50:41.578341+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:51:15.254011+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:51:49.099165+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:52:24.544821+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:52:58.838484+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.28","mappings":[{"from":3,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/359646":"2024-03-28T10:44:39.889550+0000","172.21.15.97:6801/98682155":"2024-03-28T10:44:06.019075+0000","172.21.15.97:6801/1685921853":"2024-03-28T10:44:39.889550+0000","172.21.15.97:6800/98682155":"2024-03-28T10:44:06.019075+0000","172.21.15.97:0/107690680":"2024-03-28T10:44:06.019075+0000","172.21.15.97:6800/1685921853":"2024-03-28T10:44:39.889550+0000","172.21.15.97:0/437043413":"2024-03-28T10:44:39.889550+0000","172.21.15.97:0/2319706992":"2024-03-28T10:44:39.889550+0000","172.21.15.97:0/3911532654":"2024-03-28T10:44:06.019075+0000","172.21.15.97:0/3347321845":"2024-03-28T10:44:06.019075+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-27T10:56:28.767 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:56:28 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/3201667636' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-03-27T10:56:28.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:28.797+0000 7f1d8aa00640 1 -- 172.21.15.97:0/1700934925 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f1d6c0a3610 msgr2=0x7f1d6c0a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:28.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:28.797+0000 7f1d8aa00640 1 --2- 172.21.15.97:0/1700934925 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f1d6c0a3610 0x7f1d6c0a5ad0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1d84104bd0 tx=0x7f1d78005f90 comp rx=0 tx=0).stop 2024-03-27T10:56:28.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:28.797+0000 7f1d8aa00640 1 -- 172.21.15.97:0/1700934925 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d84104d70 msgr2=0x7f1d8419a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:28.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:28.797+0000 7f1d8aa00640 1 --2- 172.21.15.97:0/1700934925 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d84104d70 0x7f1d8419a290 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f1d70006010 tx=0x7f1d70005e40 comp rx=0 tx=0).stop 2024-03-27T10:56:28.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:28.797+0000 7f1d8aa00640 1 -- 172.21.15.97:0/1700934925 shutdown_connections 2024-03-27T10:56:28.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:28.797+0000 7f1d8aa00640 1 --2- 172.21.15.97:0/1700934925 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d84105730 0x7f1d841a1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:28.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:28.797+0000 7f1d8aa00640 1 --2- 172.21.15.97:0/1700934925 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d84104d70 0x7f1d8419a290 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:28.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:28.797+0000 7f1d8aa00640 1 --2- 172.21.15.97:0/1700934925 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f1d6c0a3610 0x7f1d6c0a5ad0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:28.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:28.797+0000 7f1d8aa00640 1 --2- 172.21.15.97:0/1700934925 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1d84103b80 0x7f1d84199d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:28.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:28.797+0000 7f1d8aa00640 1 -- 172.21.15.97:0/1700934925 >> 172.21.15.97:0/1700934925 conn(0x7f1d840ff0f0 msgr2=0x7f1d84108070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:28.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:28.797+0000 7f1d8aa00640 1 -- 172.21.15.97:0/1700934925 shutdown_connections 2024-03-27T10:56:28.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:28.797+0000 7f1d8aa00640 1 -- 172.21.15.97:0/1700934925 wait complete. 2024-03-27T10:56:28.787 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-27T10:47:54.052810+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-27T10:54:02.845060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-27T10:54:03.474151+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-27T10:56:28.787 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-27T10:56:28.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.021+0000 7fe5b0800640 1 -- 172.21.15.97:0/3017666683 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5a8109d20 msgr2=0x7fe5a81105b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:28.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.021+0000 7fe5b0800640 1 --2- 172.21.15.97:0/3017666683 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5a8109d20 0x7fe5a81105b0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fe5a4009ca0 tx=0x7fe5a402c0f0 comp rx=0 tx=0).stop 2024-03-27T10:56:28.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.021+0000 7fe5b0800640 1 -- 172.21.15.97:0/3017666683 shutdown_connections 2024-03-27T10:56:28.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.021+0000 7fe5b0800640 1 --2- 172.21.15.97:0/3017666683 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5a8109d20 0x7fe5a81105b0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:28.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.021+0000 7fe5b0800640 1 --2- 172.21.15.97:0/3017666683 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe5a8109360 0x7fe5a81097e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:28.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.021+0000 7fe5b0800640 1 --2- 172.21.15.97:0/3017666683 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe5a8108170 0x7fe5a8108590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:28.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.021+0000 7fe5b0800640 1 -- 172.21.15.97:0/3017666683 >> 172.21.15.97:0/3017666683 conn(0x7fe5a8077110 msgr2=0x7fe5a8079550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:28.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.021+0000 7fe5b0800640 1 -- 172.21.15.97:0/3017666683 shutdown_connections 2024-03-27T10:56:28.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.021+0000 7fe5b0800640 1 -- 172.21.15.97:0/3017666683 wait complete. 2024-03-27T10:56:28.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.022+0000 7fe5b0800640 1 Processor -- start 2024-03-27T10:56:28.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.022+0000 7fe5b0800640 1 -- start start 2024-03-27T10:56:28.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.022+0000 7fe5b0800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe5a8108170 0x7fe5a819e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:28.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.022+0000 7fe5b0800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5a8109360 0x7fe5a819e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:28.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.022+0000 7fe5b0800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe5a8109d20 0x7fe5a81a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:28.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.022+0000 7fe5b0800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a8113cb0 con 0x7fe5a8109360 2024-03-27T10:56:28.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.022+0000 7fe5b0800640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a8113b30 con 0x7fe5a8109d20 2024-03-27T10:56:28.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.022+0000 7fe5b0800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a8113e30 con 0x7fe5a8108170 2024-03-27T10:56:28.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.023+0000 7fe5ad400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5a8109360 0x7fe5a819e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:28.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.023+0000 7fe5ad400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5a8109360 0x7fe5a819e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53366/0 (socket says 172.21.15.97:53366) 2024-03-27T10:56:28.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.023+0000 7fe5ad400640 1 -- 172.21.15.97:0/468512933 learned_addr learned my addr 172.21.15.97:0/468512933 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:56:28.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.023+0000 7fe5ae800640 1 --2- 172.21.15.97:0/468512933 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe5a8109d20 0x7fe5a81a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:28.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.023+0000 7fe5ade00640 1 --2- 172.21.15.97:0/468512933 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe5a8108170 0x7fe5a819e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:28.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.023+0000 7fe5ad400640 1 -- 172.21.15.97:0/468512933 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe5a8108170 msgr2=0x7fe5a819e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:28.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.023+0000 7fe5ad400640 1 --2- 172.21.15.97:0/468512933 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe5a8108170 0x7fe5a819e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:28.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.023+0000 7fe5ad400640 1 -- 172.21.15.97:0/468512933 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe5a8109d20 msgr2=0x7fe5a81a5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:28.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.023+0000 7fe5ad400640 1 --2- 172.21.15.97:0/468512933 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe5a8109d20 0x7fe5a81a5900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:28.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.023+0000 7fe5ad400640 1 -- 172.21.15.97:0/468512933 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5a81a5f70 con 0x7fe5a8109360 2024-03-27T10:56:28.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.023+0000 7fe5ad400640 1 --2- 172.21.15.97:0/468512933 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5a8109360 0x7fe5a819e890 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fe59800d950 tx=0x7fe59800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:28.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.024+0000 7fe5a2a00640 1 -- 172.21.15.97:0/468512933 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe598017070 con 0x7fe5a8109360 2024-03-27T10:56:28.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.024+0000 7fe5a2a00640 1 -- 172.21.15.97:0/468512933 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe598009c30 con 0x7fe5a8109360 2024-03-27T10:56:28.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.024+0000 7fe5b0800640 1 -- 172.21.15.97:0/468512933 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5a81a6260 con 0x7fe5a8109360 2024-03-27T10:56:28.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.024+0000 7fe5a2a00640 1 -- 172.21.15.97:0/468512933 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe59800f620 con 0x7fe5a8109360 2024-03-27T10:56:28.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.024+0000 7fe5b0800640 1 -- 172.21.15.97:0/468512933 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5a81a67b0 con 0x7fe5a8109360 2024-03-27T10:56:28.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.025+0000 7fe5b0800640 1 -- 172.21.15.97:0/468512933 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5a810f4b0 con 0x7fe5a8109360 2024-03-27T10:56:28.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.027+0000 7fe5a2a00640 1 -- 172.21.15.97:0/468512933 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7fe5980109a0 con 0x7fe5a8109360 2024-03-27T10:56:28.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.028+0000 7fe5a2a00640 1 --2- 172.21.15.97:0/468512933 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fe5900a3610 0x7fe5900a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:29.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.028+0000 7fe5a2a00640 1 -- 172.21.15.97:0/468512933 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fe5980cfa80 con 0x7fe5a8109360 2024-03-27T10:56:29.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.029+0000 7fe5ade00640 1 --2- 172.21.15.97:0/468512933 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fe5900a3610 0x7fe5900a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:29.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.029+0000 7fe5ade00640 1 --2- 172.21.15.97:0/468512933 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fe5900a3610 0x7fe5900a5ad0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe59c005fd0 tx=0x7fe59c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:29.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.033+0000 7fe5a2a00640 1 -- 172.21.15.97:0/468512933 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7fe5980d1e20 con 0x7fe5a8109360 2024-03-27T10:56:29.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.239+0000 7fe5b0800640 1 -- 172.21.15.97:0/468512933 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe5a8062780 con 0x7fe5a8109360 2024-03-27T10:56:29.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.240+0000 7fe5a2a00640 1 -- 172.21.15.97:0/468512933 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) ==== 93+0+10 (secure 0 0 0) 0x7fe598009de0 con 0x7fe5a8109360 2024-03-27T10:56:29.212 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-03-27T10:56:29.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.243+0000 7fe5b0800640 1 -- 172.21.15.97:0/468512933 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fe5900a3610 msgr2=0x7fe5900a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:29.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.243+0000 7fe5b0800640 1 --2- 172.21.15.97:0/468512933 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fe5900a3610 0x7fe5900a5ad0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe59c005fd0 tx=0x7fe59c005850 comp rx=0 tx=0).stop 2024-03-27T10:56:29.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.243+0000 7fe5b0800640 1 -- 172.21.15.97:0/468512933 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5a8109360 msgr2=0x7fe5a819e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:29.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.243+0000 7fe5b0800640 1 --2- 172.21.15.97:0/468512933 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5a8109360 0x7fe5a819e890 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fe59800d950 tx=0x7fe59800de10 comp rx=0 tx=0).stop 2024-03-27T10:56:29.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.243+0000 7fe5b0800640 1 -- 172.21.15.97:0/468512933 shutdown_connections 2024-03-27T10:56:29.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.243+0000 7fe5b0800640 1 --2- 172.21.15.97:0/468512933 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fe5900a3610 0x7fe5900a5ad0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:29.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.243+0000 7fe5b0800640 1 --2- 172.21.15.97:0/468512933 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe5a8109d20 0x7fe5a81a5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:29.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.243+0000 7fe5b0800640 1 --2- 172.21.15.97:0/468512933 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5a8109360 0x7fe5a819e890 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:29.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.243+0000 7fe5b0800640 1 --2- 172.21.15.97:0/468512933 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe5a8108170 0x7fe5a819e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:29.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.243+0000 7fe5b0800640 1 -- 172.21.15.97:0/468512933 >> 172.21.15.97:0/468512933 conn(0x7fe5a8077110 msgr2=0x7fe5a810c680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:29.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.243+0000 7fe5b0800640 1 -- 172.21.15.97:0/468512933 shutdown_connections 2024-03-27T10:56:29.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.243+0000 7fe5b0800640 1 -- 172.21.15.97:0/468512933 wait complete. 2024-03-27T10:56:29.232 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-27T10:56:29.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.475+0000 7f2cc4e00640 1 -- 172.21.15.97:0/1012441905 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2cc00780f0 msgr2=0x7f2cc0078990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:29.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.475+0000 7f2cc4e00640 1 --2- 172.21.15.97:0/1012441905 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2cc00780f0 0x7f2cc0078990 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f2cb8009ca0 tx=0x7f2cb802c0f0 comp rx=0 tx=0).stop 2024-03-27T10:56:29.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.475+0000 7f2cc4e00640 1 -- 172.21.15.97:0/1012441905 shutdown_connections 2024-03-27T10:56:29.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.475+0000 7f2cc4e00640 1 --2- 172.21.15.97:0/1012441905 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2cc00780f0 0x7f2cc0078990 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:29.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.475+0000 7f2cc4e00640 1 --2- 172.21.15.97:0/1012441905 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2cc0077750 0x7f2cc0077bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:29.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.475+0000 7f2cc4e00640 1 --2- 172.21.15.97:0/1012441905 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2cc0076510 0x7f2cc0076930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:29.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.475+0000 7f2cc4e00640 1 -- 172.21.15.97:0/1012441905 >> 172.21.15.97:0/1012441905 conn(0x7f2cc00ff300 msgr2=0x7f2cc0101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:29.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.475+0000 7f2cc4e00640 1 -- 172.21.15.97:0/1012441905 shutdown_connections 2024-03-27T10:56:29.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.475+0000 7f2cc4e00640 1 -- 172.21.15.97:0/1012441905 wait complete. 2024-03-27T10:56:29.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.476+0000 7f2cc4e00640 1 Processor -- start 2024-03-27T10:56:29.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.476+0000 7f2cc4e00640 1 -- start start 2024-03-27T10:56:29.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.476+0000 7f2cc4e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2cc0076510 0x7f2cc019e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:29.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.476+0000 7f2cc4e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2cc0077750 0x7f2cc019e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:29.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.476+0000 7f2cc4e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2cc00780f0 0x7f2cc01a58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:29.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.476+0000 7f2cc4e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc0113cd0 con 0x7f2cc00780f0 2024-03-27T10:56:29.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.476+0000 7f2cc4e00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc0113b50 con 0x7f2cc0077750 2024-03-27T10:56:29.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.476+0000 7f2cc4e00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc0113e50 con 0x7f2cc0076510 2024-03-27T10:56:29.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.477+0000 7f2cbe800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2cc00780f0 0x7f2cc01a58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:29.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.477+0000 7f2cbde00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2cc0076510 0x7f2cc019e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:29.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.477+0000 7f2cbe800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2cc00780f0 0x7f2cc01a58d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53376/0 (socket says 172.21.15.97:53376) 2024-03-27T10:56:29.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.477+0000 7f2cbe800640 1 -- 172.21.15.97:0/1212173468 learned_addr learned my addr 172.21.15.97:0/1212173468 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:56:29.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.477+0000 7f2cb5400640 1 --2- 172.21.15.97:0/1212173468 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2cc0077750 0x7f2cc019e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:29.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.477+0000 7f2cbe800640 1 -- 172.21.15.97:0/1212173468 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2cc0076510 msgr2=0x7f2cc019e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:29.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.477+0000 7f2cbe800640 1 --2- 172.21.15.97:0/1212173468 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2cc0076510 0x7f2cc019e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:29.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.477+0000 7f2cbe800640 1 -- 172.21.15.97:0/1212173468 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2cc0077750 msgr2=0x7f2cc019e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:29.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.477+0000 7f2cbe800640 1 --2- 172.21.15.97:0/1212173468 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2cc0077750 0x7f2cc019e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:29.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.477+0000 7f2cbe800640 1 -- 172.21.15.97:0/1212173468 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cc01a5f40 con 0x7f2cc00780f0 2024-03-27T10:56:29.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.478+0000 7f2cbe800640 1 --2- 172.21.15.97:0/1212173468 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2cc00780f0 0x7f2cc01a58d0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f2cb802c700 tx=0x7f2cb80328a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:29.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.478+0000 7f2cb7400640 1 -- 172.21.15.97:0/1212173468 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb8041020 con 0x7f2cc00780f0 2024-03-27T10:56:29.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.478+0000 7f2cc4e00640 1 -- 172.21.15.97:0/1212173468 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cc01a61d0 con 0x7f2cc00780f0 2024-03-27T10:56:29.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.478+0000 7f2cb7400640 1 -- 172.21.15.97:0/1212173468 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2cb8003e20 con 0x7f2cc00780f0 2024-03-27T10:56:29.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.478+0000 7f2cb7400640 1 -- 172.21.15.97:0/1212173468 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb8039040 con 0x7f2cc00780f0 2024-03-27T10:56:29.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.478+0000 7f2cc4e00640 1 -- 172.21.15.97:0/1212173468 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cc01a66c0 con 0x7f2cc00780f0 2024-03-27T10:56:29.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.479+0000 7f2cc4e00640 1 -- 172.21.15.97:0/1212173468 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cc0110700 con 0x7f2cc00780f0 2024-03-27T10:56:29.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.481+0000 7f2cb7400640 1 -- 172.21.15.97:0/1212173468 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f2cb8044020 con 0x7f2cc00780f0 2024-03-27T10:56:29.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.482+0000 7f2cb7400640 1 --2- 172.21.15.97:0/1212173468 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f2ca40a3610 0x7f2ca40a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:29.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.483+0000 7f2cb7400640 1 -- 172.21.15.97:0/1212173468 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f2cb80f1f10 con 0x7f2cc00780f0 2024-03-27T10:56:29.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.483+0000 7f2cbde00640 1 --2- 172.21.15.97:0/1212173468 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f2ca40a3610 0x7f2ca40a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:29.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.483+0000 7f2cbde00640 1 --2- 172.21.15.97:0/1212173468 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f2ca40a3610 0x7f2ca40a5ad0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f2cb0009bd0 tx=0x7f2cb0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:29.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.487+0000 7f2cb7400640 1 -- 172.21.15.97:0/1212173468 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f2cb804b020 con 0x7f2cc00780f0 2024-03-27T10:56:29.641 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:56:29 smithi097 ceph-mon[31859]: pgmap v436: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-27T10:56:29.641 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:56:29 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/3201667636' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-03-27T10:56:29.641 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:56:29 smithi097 ceph-mon[31859]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T10:56:29.641 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:56:29 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/1700934925' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T10:56:29.641 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:56:29 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/468512933' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-27T10:56:29.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.691+0000 7f2cc4e00640 1 -- 172.21.15.97:0/1212173468 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2cc0062750 con 0x7f2cc00780f0 2024-03-27T10:56:29.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.691+0000 7f2cb7400640 1 -- 172.21.15.97:0/1212173468 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) ==== 104+0+11 (secure 0 0 0) 0x7f2cb80bea90 con 0x7f2cc00780f0 2024-03-27T10:56:29.663 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-03-27T10:56:29.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.694+0000 7f2cc4e00640 1 -- 172.21.15.97:0/1212173468 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f2ca40a3610 msgr2=0x7f2ca40a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:29.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.694+0000 7f2cc4e00640 1 --2- 172.21.15.97:0/1212173468 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f2ca40a3610 0x7f2ca40a5ad0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f2cb0009bd0 tx=0x7f2cb0005f90 comp rx=0 tx=0).stop 2024-03-27T10:56:29.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.694+0000 7f2cc4e00640 1 -- 172.21.15.97:0/1212173468 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2cc00780f0 msgr2=0x7f2cc01a58d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:29.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.694+0000 7f2cc4e00640 1 --2- 172.21.15.97:0/1212173468 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2cc00780f0 0x7f2cc01a58d0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f2cb802c700 tx=0x7f2cb80328a0 comp rx=0 tx=0).stop 2024-03-27T10:56:29.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.694+0000 7f2cc4e00640 1 -- 172.21.15.97:0/1212173468 shutdown_connections 2024-03-27T10:56:29.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.694+0000 7f2cc4e00640 1 --2- 172.21.15.97:0/1212173468 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2cc00780f0 0x7f2cc01a58d0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:29.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.694+0000 7f2cc4e00640 1 --2- 172.21.15.97:0/1212173468 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f2ca40a3610 0x7f2ca40a5ad0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:29.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.694+0000 7f2cc4e00640 1 --2- 172.21.15.97:0/1212173468 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2cc0077750 0x7f2cc019e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:29.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.694+0000 7f2cc4e00640 1 --2- 172.21.15.97:0/1212173468 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2cc0076510 0x7f2cc019e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:29.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.694+0000 7f2cc4e00640 1 -- 172.21.15.97:0/1212173468 >> 172.21.15.97:0/1212173468 conn(0x7f2cc00ff300 msgr2=0x7f2cc0100e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:29.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.694+0000 7f2cc4e00640 1 -- 172.21.15.97:0/1212173468 shutdown_connections 2024-03-27T10:56:29.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.694+0000 7f2cc4e00640 1 -- 172.21.15.97:0/1212173468 wait complete. 2024-03-27T10:56:29.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:56:29 smithi177 ceph-mon[35294]: pgmap v436: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-27T10:56:29.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:56:29 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/3201667636' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-03-27T10:56:29.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:56:29 smithi177 ceph-mon[35294]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T10:56:29.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:56:29 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/1700934925' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T10:56:29.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:56:29 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/468512933' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-27T10:56:29.682 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-27T10:56:29.766 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:56:29 smithi114 ceph-mon[35338]: pgmap v436: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-27T10:56:29.766 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:56:29 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/3201667636' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-03-27T10:56:29.766 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:56:29 smithi114 ceph-mon[35338]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T10:56:29.766 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:56:29 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/1700934925' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T10:56:29.766 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:56:29 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/468512933' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-27T10:56:29.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.922+0000 7f6930c00640 1 -- 172.21.15.97:0/2674263255 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f692c105940 msgr2=0x7f692c10c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:29.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.922+0000 7f6930c00640 1 --2- 172.21.15.97:0/2674263255 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f692c105940 0x7f692c10c1d0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f6924009ca0 tx=0x7f692402c0f0 comp rx=0 tx=0).stop 2024-03-27T10:56:29.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.922+0000 7f6930c00640 1 -- 172.21.15.97:0/2674263255 shutdown_connections 2024-03-27T10:56:29.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.922+0000 7f6930c00640 1 --2- 172.21.15.97:0/2674263255 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f692c105940 0x7f692c10c1d0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:29.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.922+0000 7f6930c00640 1 --2- 172.21.15.97:0/2674263255 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f692c104f80 0x7f692c105400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:29.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.922+0000 7f6930c00640 1 --2- 172.21.15.97:0/2674263255 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f692c103d90 0x7f692c1041b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:29.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.922+0000 7f6930c00640 1 -- 172.21.15.97:0/2674263255 >> 172.21.15.97:0/2674263255 conn(0x7f692c0ff300 msgr2=0x7f692c101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:29.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.922+0000 7f6930c00640 1 -- 172.21.15.97:0/2674263255 shutdown_connections 2024-03-27T10:56:29.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.922+0000 7f6930c00640 1 -- 172.21.15.97:0/2674263255 wait complete. 2024-03-27T10:56:29.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.923+0000 7f6930c00640 1 Processor -- start 2024-03-27T10:56:29.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.923+0000 7f6930c00640 1 -- start start 2024-03-27T10:56:29.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.924+0000 7f6930c00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f692c103d90 0x7f692c199f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:29.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.924+0000 7f6930c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f692c104f80 0x7f692c19a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:29.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.924+0000 7f6930c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f692c105940 0x7f692c1a1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:29.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.924+0000 7f6930c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f692c10f8c0 con 0x7f692c105940 2024-03-27T10:56:29.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.924+0000 7f6930c00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f692c10f740 con 0x7f692c104f80 2024-03-27T10:56:29.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.924+0000 7f6930c00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f692c10fa40 con 0x7f692c103d90 2024-03-27T10:56:29.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.924+0000 7f692a800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f692c105940 0x7f692c1a1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:29.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.924+0000 7f692a800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f692c105940 0x7f692c1a1520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53386/0 (socket says 172.21.15.97:53386) 2024-03-27T10:56:29.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.924+0000 7f692a800640 1 -- 172.21.15.97:0/347457135 learned_addr learned my addr 172.21.15.97:0/347457135 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:56:29.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.924+0000 7f6929400640 1 --2- 172.21.15.97:0/347457135 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f692c104f80 0x7f692c19a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:29.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.924+0000 7f6929e00640 1 --2- 172.21.15.97:0/347457135 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f692c103d90 0x7f692c199f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:29.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.924+0000 7f692a800640 1 -- 172.21.15.97:0/347457135 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f692c103d90 msgr2=0x7f692c199f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:29.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.924+0000 7f692a800640 1 --2- 172.21.15.97:0/347457135 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f692c103d90 0x7f692c199f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:29.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.924+0000 7f692a800640 1 -- 172.21.15.97:0/347457135 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f692c104f80 msgr2=0x7f692c19a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:29.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.924+0000 7f692a800640 1 --2- 172.21.15.97:0/347457135 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f692c104f80 0x7f692c19a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:29.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.924+0000 7f692a800640 1 -- 172.21.15.97:0/347457135 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f692c1a1b90 con 0x7f692c105940 2024-03-27T10:56:29.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.925+0000 7f692a800640 1 --2- 172.21.15.97:0/347457135 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f692c105940 0x7f692c1a1520 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f6924032380 tx=0x7f6924032820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:29.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.925+0000 7f6922a00640 1 -- 172.21.15.97:0/347457135 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6924041020 con 0x7f692c105940 2024-03-27T10:56:29.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.925+0000 7f6922a00640 1 -- 172.21.15.97:0/347457135 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6924003e20 con 0x7f692c105940 2024-03-27T10:56:29.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.925+0000 7f6930c00640 1 -- 172.21.15.97:0/347457135 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f692c1a1e20 con 0x7f692c105940 2024-03-27T10:56:29.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.925+0000 7f6922a00640 1 -- 172.21.15.97:0/347457135 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6924039040 con 0x7f692c105940 2024-03-27T10:56:29.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.925+0000 7f6930c00640 1 -- 172.21.15.97:0/347457135 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f692c1a22e0 con 0x7f692c105940 2024-03-27T10:56:29.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.926+0000 7f6930c00640 1 -- 172.21.15.97:0/347457135 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f692c10b1d0 con 0x7f692c105940 2024-03-27T10:56:29.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.928+0000 7f6922a00640 1 -- 172.21.15.97:0/347457135 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f6924044020 con 0x7f692c105940 2024-03-27T10:56:29.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.930+0000 7f6922a00640 1 --2- 172.21.15.97:0/347457135 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f69100a3690 0x7f69100a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:29.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.930+0000 7f6922a00640 1 -- 172.21.15.97:0/347457135 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f69240f1ef0 con 0x7f692c105940 2024-03-27T10:56:29.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.930+0000 7f6929e00640 1 --2- 172.21.15.97:0/347457135 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f69100a3690 0x7f69100a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:29.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.931+0000 7f6929e00640 1 --2- 172.21.15.97:0/347457135 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f69100a3690 0x7f69100a5b50 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f692c104de0 tx=0x7f6918005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:29.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:29.934+0000 7f6922a00640 1 -- 172.21.15.97:0/347457135 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f692404b020 con 0x7f692c105940 2024-03-27T10:56:30.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.134+0000 7f6930c00640 1 -- 172.21.15.97:0/347457135 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f692c0627f0 con 0x7f692c105940 2024-03-27T10:56:30.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.135+0000 7f6922a00640 1 -- 172.21.15.97:0/347457135 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) ==== 100+0+11 (secure 0 0 0) 0x7f692403b2f0 con 0x7f692c105940 2024-03-27T10:56:30.106 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-03-27T10:56:30.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.137+0000 7f6930c00640 1 -- 172.21.15.97:0/347457135 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f69100a3690 msgr2=0x7f69100a5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:30.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.137+0000 7f6930c00640 1 --2- 172.21.15.97:0/347457135 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f69100a3690 0x7f69100a5b50 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f692c104de0 tx=0x7f6918005f90 comp rx=0 tx=0).stop 2024-03-27T10:56:30.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.137+0000 7f6930c00640 1 -- 172.21.15.97:0/347457135 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f692c105940 msgr2=0x7f692c1a1520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:30.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.137+0000 7f6930c00640 1 --2- 172.21.15.97:0/347457135 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f692c105940 0x7f692c1a1520 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f6924032380 tx=0x7f6924032820 comp rx=0 tx=0).stop 2024-03-27T10:56:30.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.137+0000 7f6930c00640 1 -- 172.21.15.97:0/347457135 shutdown_connections 2024-03-27T10:56:30.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.137+0000 7f6930c00640 1 --2- 172.21.15.97:0/347457135 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f692c105940 0x7f692c1a1520 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:30.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.137+0000 7f6930c00640 1 --2- 172.21.15.97:0/347457135 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f69100a3690 0x7f69100a5b50 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:30.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.138+0000 7f6930c00640 1 --2- 172.21.15.97:0/347457135 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f692c104f80 0x7f692c19a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:30.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.138+0000 7f6930c00640 1 --2- 172.21.15.97:0/347457135 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f692c103d90 0x7f692c199f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:30.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.138+0000 7f6930c00640 1 -- 172.21.15.97:0/347457135 >> 172.21.15.97:0/347457135 conn(0x7f692c0ff300 msgr2=0x7f692c1083a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:30.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.138+0000 7f6930c00640 1 -- 172.21.15.97:0/347457135 shutdown_connections 2024-03-27T10:56:30.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.138+0000 7f6930c00640 1 -- 172.21.15.97:0/347457135 wait complete. 2024-03-27T10:56:30.127 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-27T10:56:30.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.363+0000 7fde08a00640 1 -- 172.21.15.97:0/3412225536 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde04105920 msgr2=0x7fde0410c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:30.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.363+0000 7fde08a00640 1 --2- 172.21.15.97:0/3412225536 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde04105920 0x7fde0410c1b0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fddf4009900 tx=0x7fddf402bc50 comp rx=0 tx=0).stop 2024-03-27T10:56:30.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.363+0000 7fde08a00640 1 -- 172.21.15.97:0/3412225536 shutdown_connections 2024-03-27T10:56:30.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.363+0000 7fde08a00640 1 --2- 172.21.15.97:0/3412225536 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde04105920 0x7fde0410c1b0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:30.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.363+0000 7fde08a00640 1 --2- 172.21.15.97:0/3412225536 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fde04104f60 0x7fde041053e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:30.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.363+0000 7fde08a00640 1 --2- 172.21.15.97:0/3412225536 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fde04103d70 0x7fde04104190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:30.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.363+0000 7fde08a00640 1 -- 172.21.15.97:0/3412225536 >> 172.21.15.97:0/3412225536 conn(0x7fde040ff300 msgr2=0x7fde04101720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:30.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.363+0000 7fde08a00640 1 -- 172.21.15.97:0/3412225536 shutdown_connections 2024-03-27T10:56:30.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.363+0000 7fde08a00640 1 -- 172.21.15.97:0/3412225536 wait complete. 2024-03-27T10:56:30.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.364+0000 7fde08a00640 1 Processor -- start 2024-03-27T10:56:30.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.364+0000 7fde08a00640 1 -- start start 2024-03-27T10:56:30.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.364+0000 7fde08a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde04103d70 0x7fde04199f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:30.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.364+0000 7fde08a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fde04104f60 0x7fde0419a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:30.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.364+0000 7fde08a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fde04105920 0x7fde041a14c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:30.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.364+0000 7fde08a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fde0410f8f0 con 0x7fde04103d70 2024-03-27T10:56:30.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.364+0000 7fde08a00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fde0410f770 con 0x7fde04104f60 2024-03-27T10:56:30.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.364+0000 7fde08a00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fde0410fa70 con 0x7fde04105920 2024-03-27T10:56:30.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.365+0000 7fde01e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde04103d70 0x7fde04199f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:30.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.365+0000 7fde01e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde04103d70 0x7fde04199f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53408/0 (socket says 172.21.15.97:53408) 2024-03-27T10:56:30.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.365+0000 7fde01e00640 1 -- 172.21.15.97:0/2966831902 learned_addr learned my addr 172.21.15.97:0/2966831902 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:56:30.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.365+0000 7fde01400640 1 --2- 172.21.15.97:0/2966831902 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fde04104f60 0x7fde0419a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:30.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.365+0000 7fde02800640 1 --2- 172.21.15.97:0/2966831902 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fde04105920 0x7fde041a14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:30.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.365+0000 7fde01e00640 1 -- 172.21.15.97:0/2966831902 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fde04105920 msgr2=0x7fde041a14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:30.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.365+0000 7fde01e00640 1 --2- 172.21.15.97:0/2966831902 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fde04105920 0x7fde041a14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:30.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.365+0000 7fde01e00640 1 -- 172.21.15.97:0/2966831902 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fde04104f60 msgr2=0x7fde0419a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:30.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.365+0000 7fde01e00640 1 --2- 172.21.15.97:0/2966831902 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fde04104f60 0x7fde0419a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:30.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.365+0000 7fde01e00640 1 -- 172.21.15.97:0/2966831902 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde041a1b30 con 0x7fde04103d70 2024-03-27T10:56:30.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.366+0000 7fde01e00640 1 --2- 172.21.15.97:0/2966831902 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde04103d70 0x7fde04199f10 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fddfc00eaa0 tx=0x7fddfc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:30.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.366+0000 7fddfaa00640 1 -- 172.21.15.97:0/2966831902 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddfc016070 con 0x7fde04103d70 2024-03-27T10:56:30.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.366+0000 7fde08a00640 1 -- 172.21.15.97:0/2966831902 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde041a1e20 con 0x7fde04103d70 2024-03-27T10:56:30.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.366+0000 7fddfaa00640 1 -- 172.21.15.97:0/2966831902 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fddfc003e20 con 0x7fde04103d70 2024-03-27T10:56:30.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.366+0000 7fddfaa00640 1 -- 172.21.15.97:0/2966831902 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddfc0116d0 con 0x7fde04103d70 2024-03-27T10:56:30.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.366+0000 7fde08a00640 1 -- 172.21.15.97:0/2966831902 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde041a2230 con 0x7fde04103d70 2024-03-27T10:56:30.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.367+0000 7fde08a00640 1 -- 172.21.15.97:0/2966831902 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde0410b090 con 0x7fde04103d70 2024-03-27T10:56:30.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.369+0000 7fddfaa00640 1 -- 172.21.15.97:0/2966831902 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7fddfc019070 con 0x7fde04103d70 2024-03-27T10:56:30.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.370+0000 7fddfaa00640 1 --2- 172.21.15.97:0/2966831902 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fdde80a3610 0x7fdde80a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:30.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.371+0000 7fddfaa00640 1 -- 172.21.15.97:0/2966831902 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fddfc0d0250 con 0x7fde04103d70 2024-03-27T10:56:30.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.371+0000 7fde01400640 1 --2- 172.21.15.97:0/2966831902 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fdde80a3610 0x7fdde80a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:30.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.371+0000 7fde01400640 1 --2- 172.21.15.97:0/2966831902 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fdde80a3610 0x7fdde80a5ad0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fde0419b420 tx=0x7fddec00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:30.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.373+0000 7fddfaa00640 1 -- 172.21.15.97:0/2966831902 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7fddfc09cdd0 con 0x7fde04103d70 2024-03-27T10:56:30.390 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:56:30 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/1212173468' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-27T10:56:30.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:56:30 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/347457135' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-27T10:56:30.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.575+0000 7fde08a00640 1 -- 172.21.15.97:0/2966831902 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fde0402d540 con 0x7fde04103d70 2024-03-27T10:56:30.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.579+0000 7fddfaa00640 1 -- 172.21.15.97:0/2966831902 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+22644 (secure 0 0 0) 0x7fddfc09c6d0 con 0x7fde04103d70 2024-03-27T10:56:30.551 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T10:56:30.551 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":92,"fsid":"adca88c4-ec26-11ee-95d0-87774f69a715","created":"2024-03-27T10:43:24.680694+0000","modified":"2024-03-27T10:54:41.310812+0000","last_up_change":"2024-03-27T10:53:00.552513+0000","last_in_change":"2024-03-27T10:52:36.302986+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-27T10:47:54.052810+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-27T10:54:02.845060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-27T10:54:03.474151+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b634e5f1-6118-4e2f-95e9-830db390f8d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1746812080},{"type":"v1","addr":"172.21.15.97:6803","nonce":1746812080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1746812080},{"type":"v1","addr":"172.21.15.97:6805","nonce":1746812080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":1746812080},{"type":"v1","addr":"172.21.15.97:6809","nonce":1746812080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1746812080},{"type":"v1","addr":"172.21.15.97:6807","nonce":1746812080}]},"public_addr":"172.21.15.97:6803/1746812080","cluster_addr":"172.21.15.97:6805/1746812080","heartbeat_back_addr":"172.21.15.97:6809/1746812080","heartbeat_front_addr":"172.21.15.97:6807/1746812080","state":["exists","up"]},{"osd":1,"uuid":"441e2770-3284-4388-b0de-c56eb8a373d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":3858821264},{"type":"v1","addr":"172.21.15.114:6801","nonce":3858821264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":3858821264},{"type":"v1","addr":"172.21.15.114:6803","nonce":3858821264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":3858821264},{"type":"v1","addr":"172.21.15.114:6807","nonce":3858821264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":3858821264},{"type":"v1","addr":"172.21.15.114:6805","nonce":3858821264}]},"public_addr":"172.21.15.114:6801/3858821264","cluster_addr":"172.21.15.114:6803/3858821264","heartbeat_back_addr":"172.21.15.114:6807/3858821264","heartbeat_front_addr":"172.21.15.114:6805/3858821264","state":["exists","up"]},{"osd":2,"uuid":"00108864-1aa6-48d6-8398-e8eb79fda123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":124240658},{"type":"v1","addr":"172.21.15.177:6801","nonce":124240658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":124240658},{"type":"v1","addr":"172.21.15.177:6803","nonce":124240658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":124240658},{"type":"v1","addr":"172.21.15.177:6807","nonce":124240658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":124240658},{"type":"v1","addr":"172.21.15.177:6805","nonce":124240658}]},"public_addr":"172.21.15.177:6801/124240658","cluster_addr":"172.21.15.177:6803/124240658","heartbeat_back_addr":"172.21.15.177:6807/124240658","heartbeat_front_addr":"172.21.15.177:6805/124240658","state":["exists","up"]},{"osd":3,"uuid":"1860b690-e2dc-4977-88d7-d1aacf7022ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2478179311},{"type":"v1","addr":"172.21.15.97:6811","nonce":2478179311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2478179311},{"type":"v1","addr":"172.21.15.97:6813","nonce":2478179311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":2478179311},{"type":"v1","addr":"172.21.15.97:6817","nonce":2478179311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2478179311},{"type":"v1","addr":"172.21.15.97:6815","nonce":2478179311}]},"public_addr":"172.21.15.97:6811/2478179311","cluster_addr":"172.21.15.97:6813/2478179311","heartbeat_back_addr":"172.21.15.97:6817/2478179311","heartbeat_front_addr":"172.21.15.97:6815/2478179311","state":["exists","up"]},{"osd":4,"uuid":"9245a3a4-7d49-4fdb-9d55-112e43177050","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":75755945},{"type":"v1","addr":"172.21.15.114:6809","nonce":75755945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":75755945},{"type":"v1","addr":"172.21.15.114:6811","nonce":75755945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":75755945},{"type":"v1","addr":"172.21.15.114:6815","nonce":75755945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":75755945},{"type":"v1","addr":"172.21.15.114:6813","nonce":75755945}]},"public_addr":"172.21.15.114:6809/75755945","cluster_addr":"172.21.15.114:6811/75755945","heartbeat_back_addr":"172.21.15.114:6815/75755945","heartbeat_front_addr":"172.21.15.114:6813/75755945","state":["exists","up"]},{"osd":5,"uuid":"51922f99-e8d3-461b-8615-e9bfdcbf5ed3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":1918493790},{"type":"v1","addr":"172.21.15.177:6809","nonce":1918493790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":1918493790},{"type":"v1","addr":"172.21.15.177:6811","nonce":1918493790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":1918493790},{"type":"v1","addr":"172.21.15.177:6815","nonce":1918493790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":1918493790},{"type":"v1","addr":"172.21.15.177:6813","nonce":1918493790}]},"public_addr":"172.21.15.177:6809/1918493790","cluster_addr":"172.21.15.177:6811/1918493790","heartbeat_back_addr":"172.21.15.177:6815/1918493790","heartbeat_front_addr":"172.21.15.177:6813/1918493790","state":["exists","up"]},{"osd":6,"uuid":"06dffb32-05eb-43b3-852e-915c5cc7f25a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2892389563},{"type":"v1","addr":"172.21.15.97:6819","nonce":2892389563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2892389563},{"type":"v1","addr":"172.21.15.97:6821","nonce":2892389563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2892389563},{"type":"v1","addr":"172.21.15.97:6825","nonce":2892389563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2892389563},{"type":"v1","addr":"172.21.15.97:6823","nonce":2892389563}]},"public_addr":"172.21.15.97:6819/2892389563","cluster_addr":"172.21.15.97:6821/2892389563","heartbeat_back_addr":"172.21.15.97:6825/2892389563","heartbeat_front_addr":"172.21.15.97:6823/2892389563","state":["exists","up"]},{"osd":7,"uuid":"e68e3519-9c1b-43c5-8e06-5385248b19df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":2445491153},{"type":"v1","addr":"172.21.15.114:6817","nonce":2445491153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":2445491153},{"type":"v1","addr":"172.21.15.114:6819","nonce":2445491153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2445491153},{"type":"v1","addr":"172.21.15.114:6823","nonce":2445491153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":2445491153},{"type":"v1","addr":"172.21.15.114:6821","nonce":2445491153}]},"public_addr":"172.21.15.114:6817/2445491153","cluster_addr":"172.21.15.114:6819/2445491153","heartbeat_back_addr":"172.21.15.114:6823/2445491153","heartbeat_front_addr":"172.21.15.114:6821/2445491153","state":["exists","up"]},{"osd":8,"uuid":"31ac592e-1349-49ee-9af9-8ad85d682978","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":2051134191},{"type":"v1","addr":"172.21.15.177:6817","nonce":2051134191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":2051134191},{"type":"v1","addr":"172.21.15.177:6819","nonce":2051134191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":2051134191},{"type":"v1","addr":"172.21.15.177:6823","nonce":2051134191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":2051134191},{"type":"v1","addr":"172.21.15.177:6821","nonce":2051134191}]},"public_addr":"172.21.15.177:6817/2051134191","cluster_addr":"172.21.15.177:6819/2051134191","heartbeat_back_addr":"172.21.15.177:6823/2051134191","heartbeat_front_addr":"172.21.15.177:6821/2051134191","state":["exists","up"]},{"osd":9,"uuid":"0171086a-dd41-4eb4-a389-16f0c46a64db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":1578881271},{"type":"v1","addr":"172.21.15.97:6827","nonce":1578881271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1578881271},{"type":"v1","addr":"172.21.15.97:6829","nonce":1578881271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1578881271},{"type":"v1","addr":"172.21.15.97:6833","nonce":1578881271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1578881271},{"type":"v1","addr":"172.21.15.97:6831","nonce":1578881271}]},"public_addr":"172.21.15.97:6827/1578881271","cluster_addr":"172.21.15.97:6829/1578881271","heartbeat_back_addr":"172.21.15.97:6833/1578881271","heartbeat_front_addr":"172.21.15.97:6831/1578881271","state":["exists","up"]},{"osd":10,"uuid":"64754e31-b124-4601-ad98-2f70b1dd5359","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":2893111461},{"type":"v1","addr":"172.21.15.114:6825","nonce":2893111461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":2893111461},{"type":"v1","addr":"172.21.15.114:6827","nonce":2893111461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2893111461},{"type":"v1","addr":"172.21.15.114:6831","nonce":2893111461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":2893111461},{"type":"v1","addr":"172.21.15.114:6829","nonce":2893111461}]},"public_addr":"172.21.15.114:6825/2893111461","cluster_addr":"172.21.15.114:6827/2893111461","heartbeat_back_addr":"172.21.15.114:6831/2893111461","heartbeat_front_addr":"172.21.15.114:6829/2893111461","state":["exists","up"]},{"osd":11,"uuid":"9ffc752c-adf7-4037-89d3-535787d7475d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":85,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":2333030391},{"type":"v1","addr":"172.21.15.177:6825","nonce":2333030391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":2333030391},{"type":"v1","addr":"172.21.15.177:6827","nonce":2333030391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":2333030391},{"type":"v1","addr":"172.21.15.177:6831","nonce":2333030391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":2333030391},{"type":"v1","addr":"172.21.15.177:6829","nonce":2333030391}]},"public_addr":"172.21.15.177:6825/2333030391","cluster_addr":"172.21.15.177:6827/2333030391","heartbeat_back_addr":"172.21.15.177:6831/2333030391","heartbeat_front_addr":"172.21.15.177:6829/2333030391","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:46:45.143519+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:47:18.656947+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:47:51.316345+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:48:24.575512+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:48:59.113555+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:49:32.103865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:50:06.615539+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:50:41.578341+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:51:15.254011+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:51:49.099165+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:52:24.544821+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:52:58.838484+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.28","mappings":[{"from":3,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/359646":"2024-03-28T10:44:39.889550+0000","172.21.15.97:6801/98682155":"2024-03-28T10:44:06.019075+0000","172.21.15.97:6801/1685921853":"2024-03-28T10:44:39.889550+0000","172.21.15.97:6800/98682155":"2024-03-28T10:44:06.019075+0000","172.21.15.97:0/107690680":"2024-03-28T10:44:06.019075+0000","172.21.15.97:6800/1685921853":"2024-03-28T10:44:39.889550+0000","172.21.15.97:0/437043413":"2024-03-28T10:44:39.889550+0000","172.21.15.97:0/2319706992":"2024-03-28T10:44:39.889550+0000","172.21.15.97:0/3911532654":"2024-03-28T10:44:06.019075+0000","172.21.15.97:0/3347321845":"2024-03-28T10:44:06.019075+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-27T10:56:30.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.582+0000 7fde08a00640 1 -- 172.21.15.97:0/2966831902 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fdde80a3610 msgr2=0x7fdde80a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:30.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.582+0000 7fde08a00640 1 --2- 172.21.15.97:0/2966831902 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fdde80a3610 0x7fdde80a5ad0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fde0419b420 tx=0x7fddec00b410 comp rx=0 tx=0).stop 2024-03-27T10:56:30.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.582+0000 7fde08a00640 1 -- 172.21.15.97:0/2966831902 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde04103d70 msgr2=0x7fde04199f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:30.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.582+0000 7fde08a00640 1 --2- 172.21.15.97:0/2966831902 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde04103d70 0x7fde04199f10 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fddfc00eaa0 tx=0x7fddfc00ef60 comp rx=0 tx=0).stop 2024-03-27T10:56:30.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.582+0000 7fde08a00640 1 -- 172.21.15.97:0/2966831902 shutdown_connections 2024-03-27T10:56:30.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.582+0000 7fde08a00640 1 --2- 172.21.15.97:0/2966831902 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fde04105920 0x7fde041a14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:30.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.582+0000 7fde08a00640 1 --2- 172.21.15.97:0/2966831902 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fdde80a3610 0x7fdde80a5ad0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:30.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.582+0000 7fde08a00640 1 --2- 172.21.15.97:0/2966831902 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fde04104f60 0x7fde0419a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:30.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.582+0000 7fde08a00640 1 --2- 172.21.15.97:0/2966831902 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde04103d70 0x7fde04199f10 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:30.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.583+0000 7fde08a00640 1 -- 172.21.15.97:0/2966831902 >> 172.21.15.97:0/2966831902 conn(0x7fde040ff300 msgr2=0x7fde04108260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:30.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.583+0000 7fde08a00640 1 -- 172.21.15.97:0/2966831902 shutdown_connections 2024-03-27T10:56:30.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.583+0000 7fde08a00640 1 -- 172.21.15.97:0/2966831902 wait complete. 2024-03-27T10:56:30.573 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-27T10:47:54.052810+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-27T10:54:02.845060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-27T10:54:03.474151+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-27T10:56:30.574 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-27T10:56:30.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:56:30 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/1212173468' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-27T10:56:30.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:56:30 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/347457135' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-27T10:56:30.766 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:56:30 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/1212173468' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-27T10:56:30.766 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:56:30 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/347457135' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-27T10:56:30.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.822+0000 7f4134600640 1 -- 172.21.15.97:0/2699995487 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f412c109260 msgr2=0x7f412c109680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:30.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.822+0000 7f4134600640 1 --2- 172.21.15.97:0/2699995487 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f412c109260 0x7f412c109680 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f4120009900 tx=0x7f412002bc30 comp rx=0 tx=0).stop 2024-03-27T10:56:30.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.823+0000 7f4134600640 1 -- 172.21.15.97:0/2699995487 shutdown_connections 2024-03-27T10:56:30.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.823+0000 7f4134600640 1 --2- 172.21.15.97:0/2699995487 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f412c10ae10 0x7f412c1116a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:30.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.823+0000 7f4134600640 1 --2- 172.21.15.97:0/2699995487 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f412c10a450 0x7f412c10a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:30.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.823+0000 7f4134600640 1 --2- 172.21.15.97:0/2699995487 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f412c109260 0x7f412c109680 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:30.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.823+0000 7f4134600640 1 -- 172.21.15.97:0/2699995487 >> 172.21.15.97:0/2699995487 conn(0x7f412c06f9e0 msgr2=0x7f412c071e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:30.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.823+0000 7f4134600640 1 -- 172.21.15.97:0/2699995487 shutdown_connections 2024-03-27T10:56:30.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.823+0000 7f4134600640 1 -- 172.21.15.97:0/2699995487 wait complete. 2024-03-27T10:56:30.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.824+0000 7f4134600640 1 Processor -- start 2024-03-27T10:56:30.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.824+0000 7f4134600640 1 -- start start 2024-03-27T10:56:30.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.824+0000 7f4134600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f412c109260 0x7f412c1a2890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:30.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.824+0000 7f4134600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f412c10a450 0x7f412c1a2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:30.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.824+0000 7f4134600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f412c10ae10 0x7f412c1a9e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:30.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.824+0000 7f4134600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f412c114dc0 con 0x7f412c10a450 2024-03-27T10:56:30.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.824+0000 7f4134600640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f412c114c40 con 0x7f412c10ae10 2024-03-27T10:56:30.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.824+0000 7f4134600640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f412c114f40 con 0x7f412c109260 2024-03-27T10:56:30.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.825+0000 7f4131200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f412c10a450 0x7f412c1a2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:30.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.825+0000 7f4131200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f412c10a450 0x7f412c1a2dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53436/0 (socket says 172.21.15.97:53436) 2024-03-27T10:56:30.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.825+0000 7f4131200640 1 -- 172.21.15.97:0/3988627381 learned_addr learned my addr 172.21.15.97:0/3988627381 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:56:30.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.825+0000 7f4131c00640 1 --2- 172.21.15.97:0/3988627381 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f412c109260 0x7f412c1a2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:30.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.825+0000 7f4132600640 1 --2- 172.21.15.97:0/3988627381 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f412c10ae10 0x7f412c1a9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:30.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.825+0000 7f4131200640 1 -- 172.21.15.97:0/3988627381 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f412c109260 msgr2=0x7f412c1a2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:30.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.825+0000 7f4131200640 1 --2- 172.21.15.97:0/3988627381 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f412c109260 0x7f412c1a2890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:30.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.825+0000 7f4131200640 1 -- 172.21.15.97:0/3988627381 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f412c10ae10 msgr2=0x7f412c1a9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:30.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.825+0000 7f4131200640 1 --2- 172.21.15.97:0/3988627381 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f412c10ae10 0x7f412c1a9e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:30.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.825+0000 7f4131200640 1 -- 172.21.15.97:0/3988627381 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f412c1aa4b0 con 0x7f412c10a450 2024-03-27T10:56:30.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.825+0000 7f4131200640 1 --2- 172.21.15.97:0/3988627381 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f412c10a450 0x7f412c1a2dd0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f411800e9f0 tx=0x7f411800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:30.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.826+0000 7f412a000640 1 -- 172.21.15.97:0/3988627381 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f411800cd80 con 0x7f412c10a450 2024-03-27T10:56:30.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.826+0000 7f412a000640 1 -- 172.21.15.97:0/3988627381 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f41180108e0 con 0x7f412c10a450 2024-03-27T10:56:30.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.826+0000 7f4134600640 1 -- 172.21.15.97:0/3988627381 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f412c1aa7a0 con 0x7f412c10a450 2024-03-27T10:56:30.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.826+0000 7f412a000640 1 -- 172.21.15.97:0/3988627381 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4118018920 con 0x7f412c10a450 2024-03-27T10:56:30.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.826+0000 7f4134600640 1 -- 172.21.15.97:0/3988627381 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f412c1aad40 con 0x7f412c10a450 2024-03-27T10:56:30.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.827+0000 7f4134600640 1 -- 172.21.15.97:0/3988627381 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f412c1aafa0 con 0x7f412c10a450 2024-03-27T10:56:30.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.830+0000 7f412a000640 1 -- 172.21.15.97:0/3988627381 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f4118010a90 con 0x7f412c10a450 2024-03-27T10:56:30.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.831+0000 7f412a000640 1 --2- 172.21.15.97:0/3988627381 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f41140a3690 0x7f41140a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:30.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.831+0000 7f412a000640 1 -- 172.21.15.97:0/3988627381 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f41180d0150 con 0x7f412c10a450 2024-03-27T10:56:30.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.831+0000 7f4131c00640 1 --2- 172.21.15.97:0/3988627381 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f41140a3690 0x7f41140a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:30.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.832+0000 7f4131c00640 1 --2- 172.21.15.97:0/3988627381 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f41140a3690 0x7f41140a5b50 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f412002c240 tx=0x7f4120005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:30.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:30.835+0000 7f412a000640 1 -- 172.21.15.97:0/3988627381 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f411809ccd0 con 0x7f412c10a450 2024-03-27T10:56:31.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.037+0000 7f4134600640 1 -- 172.21.15.97:0/3988627381 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f412c1ab170 con 0x7f412c10a450 2024-03-27T10:56:31.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.038+0000 7f412a000640 1 -- 172.21.15.97:0/3988627381 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) ==== 93+0+10 (secure 0 0 0) 0x7f411809c5d0 con 0x7f412c10a450 2024-03-27T10:56:31.009 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-03-27T10:56:31.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.040+0000 7f4134600640 1 -- 172.21.15.97:0/3988627381 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f41140a3690 msgr2=0x7f41140a5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:31.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.040+0000 7f4134600640 1 --2- 172.21.15.97:0/3988627381 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f41140a3690 0x7f41140a5b50 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f412002c240 tx=0x7f4120005fd0 comp rx=0 tx=0).stop 2024-03-27T10:56:31.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.040+0000 7f4134600640 1 -- 172.21.15.97:0/3988627381 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f412c10a450 msgr2=0x7f412c1a2dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:31.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.040+0000 7f4134600640 1 --2- 172.21.15.97:0/3988627381 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f412c10a450 0x7f412c1a2dd0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f411800e9f0 tx=0x7f411800eeb0 comp rx=0 tx=0).stop 2024-03-27T10:56:31.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.040+0000 7f4134600640 1 -- 172.21.15.97:0/3988627381 shutdown_connections 2024-03-27T10:56:31.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.040+0000 7f4134600640 1 --2- 172.21.15.97:0/3988627381 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f41140a3690 0x7f41140a5b50 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:31.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.040+0000 7f4134600640 1 --2- 172.21.15.97:0/3988627381 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f412c10ae10 0x7f412c1a9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:31.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.040+0000 7f4134600640 1 --2- 172.21.15.97:0/3988627381 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f412c10a450 0x7f412c1a2dd0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:31.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.040+0000 7f4134600640 1 --2- 172.21.15.97:0/3988627381 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f412c109260 0x7f412c1a2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:31.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.040+0000 7f4134600640 1 -- 172.21.15.97:0/3988627381 >> 172.21.15.97:0/3988627381 conn(0x7f412c06f9e0 msgr2=0x7f412c06c500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:31.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.041+0000 7f4134600640 1 -- 172.21.15.97:0/3988627381 shutdown_connections 2024-03-27T10:56:31.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.041+0000 7f4134600640 1 -- 172.21.15.97:0/3988627381 wait complete. 2024-03-27T10:56:31.032 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-27T10:56:31.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.266+0000 7f0ddc000640 1 -- 172.21.15.97:0/1151899579 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0dd4105940 msgr2=0x7f0dd410c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:31.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.266+0000 7f0ddc000640 1 --2- 172.21.15.97:0/1151899579 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0dd4105940 0x7f0dd410c1d0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc009ca0 tx=0x7f0dcc02c100 comp rx=0 tx=0).stop 2024-03-27T10:56:31.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.267+0000 7f0ddc000640 1 -- 172.21.15.97:0/1151899579 shutdown_connections 2024-03-27T10:56:31.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.267+0000 7f0ddc000640 1 --2- 172.21.15.97:0/1151899579 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0dd4105940 0x7f0dd410c1d0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:31.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.267+0000 7f0ddc000640 1 --2- 172.21.15.97:0/1151899579 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0dd4104f80 0x7f0dd4105400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:31.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.267+0000 7f0ddc000640 1 --2- 172.21.15.97:0/1151899579 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0dd4103d90 0x7f0dd41041b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:31.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.267+0000 7f0ddc000640 1 -- 172.21.15.97:0/1151899579 >> 172.21.15.97:0/1151899579 conn(0x7f0dd40ff300 msgr2=0x7f0dd4101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:31.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.267+0000 7f0ddc000640 1 -- 172.21.15.97:0/1151899579 shutdown_connections 2024-03-27T10:56:31.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.267+0000 7f0ddc000640 1 -- 172.21.15.97:0/1151899579 wait complete. 2024-03-27T10:56:31.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.268+0000 7f0ddc000640 1 Processor -- start 2024-03-27T10:56:31.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.268+0000 7f0ddc000640 1 -- start start 2024-03-27T10:56:31.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.268+0000 7f0ddc000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0dd4103d90 0x7f0dd4199f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:31.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.268+0000 7f0ddc000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0dd4104f80 0x7f0dd419a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:31.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.268+0000 7f0ddc000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0dd4105940 0x7f0dd41a1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:31.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.268+0000 7f0ddc000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd410f8c0 con 0x7f0dd4104f80 2024-03-27T10:56:31.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.268+0000 7f0ddc000640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd410f740 con 0x7f0dd4103d90 2024-03-27T10:56:31.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.268+0000 7f0ddc000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd410fa40 con 0x7f0dd4105940 2024-03-27T10:56:31.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.268+0000 7f0dd8c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0dd4104f80 0x7f0dd419a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:31.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.268+0000 7f0dd8c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0dd4104f80 0x7f0dd419a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53450/0 (socket says 172.21.15.97:53450) 2024-03-27T10:56:31.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.268+0000 7f0dd8c00640 1 -- 172.21.15.97:0/3478131444 learned_addr learned my addr 172.21.15.97:0/3478131444 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:56:31.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.269+0000 7f0dd9600640 1 --2- 172.21.15.97:0/3478131444 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0dd4103d90 0x7f0dd4199f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:31.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.269+0000 7f0dda000640 1 --2- 172.21.15.97:0/3478131444 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0dd4105940 0x7f0dd41a1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:31.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.269+0000 7f0dd8c00640 1 -- 172.21.15.97:0/3478131444 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0dd4105940 msgr2=0x7f0dd41a1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:31.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.269+0000 7f0dd8c00640 1 --2- 172.21.15.97:0/3478131444 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0dd4105940 0x7f0dd41a1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:31.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.269+0000 7f0dd8c00640 1 -- 172.21.15.97:0/3478131444 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0dd4103d90 msgr2=0x7f0dd4199f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:31.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.269+0000 7f0dd8c00640 1 --2- 172.21.15.97:0/3478131444 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0dd4103d90 0x7f0dd4199f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:31.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.269+0000 7f0dd8c00640 1 -- 172.21.15.97:0/3478131444 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dd41a1b90 con 0x7f0dd4104f80 2024-03-27T10:56:31.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.269+0000 7f0dd8c00640 1 --2- 172.21.15.97:0/3478131444 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0dd4104f80 0x7f0dd419a4b0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f0dc800e9f0 tx=0x7f0dc800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:31.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.269+0000 7f0dd2000640 1 -- 172.21.15.97:0/3478131444 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc800cd80 con 0x7f0dd4104f80 2024-03-27T10:56:31.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.269+0000 7f0dd2000640 1 -- 172.21.15.97:0/3478131444 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0dc80054d0 con 0x7f0dd4104f80 2024-03-27T10:56:31.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.269+0000 7f0ddc000640 1 -- 172.21.15.97:0/3478131444 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dd41a1e80 con 0x7f0dd4104f80 2024-03-27T10:56:31.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.270+0000 7f0dd2000640 1 -- 172.21.15.97:0/3478131444 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc8010490 con 0x7f0dd4104f80 2024-03-27T10:56:31.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.270+0000 7f0ddc000640 1 -- 172.21.15.97:0/3478131444 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dd41a22f0 con 0x7f0dd4104f80 2024-03-27T10:56:31.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.271+0000 7f0ddc000640 1 -- 172.21.15.97:0/3478131444 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0dd410b1f0 con 0x7f0dd4104f80 2024-03-27T10:56:31.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.273+0000 7f0dd2000640 1 -- 172.21.15.97:0/3478131444 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f0dc8005020 con 0x7f0dd4104f80 2024-03-27T10:56:31.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.274+0000 7f0dd2000640 1 --2- 172.21.15.97:0/3478131444 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f0dbc0a3610 0x7f0dbc0a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:31.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.274+0000 7f0dd2000640 1 -- 172.21.15.97:0/3478131444 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f0dc80d01c0 con 0x7f0dd4104f80 2024-03-27T10:56:31.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.274+0000 7f0dd9600640 1 --2- 172.21.15.97:0/3478131444 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f0dbc0a3610 0x7f0dbc0a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:31.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.275+0000 7f0dd9600640 1 --2- 172.21.15.97:0/3478131444 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f0dbc0a3610 0x7f0dbc0a5ad0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f0dd4104de0 tx=0x7f0dc4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:31.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.278+0000 7f0dd2000640 1 -- 172.21.15.97:0/3478131444 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f0dc809cda0 con 0x7f0dd4104f80 2024-03-27T10:56:31.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.492+0000 7f0ddc000640 1 -- 172.21.15.97:0/3478131444 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0dd40627f0 con 0x7f0dd4104f80 2024-03-27T10:56:31.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.493+0000 7f0dd2000640 1 -- 172.21.15.97:0/3478131444 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) ==== 104+0+11 (secure 0 0 0) 0x7f0dc809c6a0 con 0x7f0dd4104f80 2024-03-27T10:56:31.464 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-03-27T10:56:31.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.495+0000 7f0ddc000640 1 -- 172.21.15.97:0/3478131444 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f0dbc0a3610 msgr2=0x7f0dbc0a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:31.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.495+0000 7f0ddc000640 1 --2- 172.21.15.97:0/3478131444 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f0dbc0a3610 0x7f0dbc0a5ad0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f0dd4104de0 tx=0x7f0dc4005f90 comp rx=0 tx=0).stop 2024-03-27T10:56:31.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.496+0000 7f0ddc000640 1 -- 172.21.15.97:0/3478131444 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0dd4104f80 msgr2=0x7f0dd419a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:31.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.496+0000 7f0ddc000640 1 --2- 172.21.15.97:0/3478131444 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0dd4104f80 0x7f0dd419a4b0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f0dc800e9f0 tx=0x7f0dc800edf0 comp rx=0 tx=0).stop 2024-03-27T10:56:31.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.496+0000 7f0ddc000640 1 -- 172.21.15.97:0/3478131444 shutdown_connections 2024-03-27T10:56:31.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.496+0000 7f0ddc000640 1 --2- 172.21.15.97:0/3478131444 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0dd4105940 0x7f0dd41a1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:31.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.496+0000 7f0ddc000640 1 --2- 172.21.15.97:0/3478131444 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0dd4104f80 0x7f0dd419a4b0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:31.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.496+0000 7f0ddc000640 1 --2- 172.21.15.97:0/3478131444 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f0dbc0a3610 0x7f0dbc0a5ad0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:31.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.496+0000 7f0ddc000640 1 --2- 172.21.15.97:0/3478131444 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0dd4103d90 0x7f0dd4199f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:31.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.496+0000 7f0ddc000640 1 -- 172.21.15.97:0/3478131444 >> 172.21.15.97:0/3478131444 conn(0x7f0dd40ff300 msgr2=0x7f0dd41083c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:31.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.496+0000 7f0ddc000640 1 -- 172.21.15.97:0/3478131444 shutdown_connections 2024-03-27T10:56:31.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.496+0000 7f0ddc000640 1 -- 172.21.15.97:0/3478131444 wait complete. 2024-03-27T10:56:31.486 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-27T10:56:31.562 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:56:31 smithi097 ceph-mon[31859]: pgmap v437: 97 pgs: 97 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1023 B/s wr, 13 op/s 2024-03-27T10:56:31.562 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:56:31 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/2966831902' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T10:56:31.563 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:56:31 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/3988627381' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-27T10:56:31.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:56:31 smithi177 ceph-mon[35294]: pgmap v437: 97 pgs: 97 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1023 B/s wr, 13 op/s 2024-03-27T10:56:31.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:56:31 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/2966831902' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T10:56:31.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:56:31 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/3988627381' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-27T10:56:31.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.730+0000 7f3e77200640 1 -- 172.21.15.97:0/801602950 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e78109260 msgr2=0x7f3e781096e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:31.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.730+0000 7f3e77200640 1 --2- 172.21.15.97:0/801602950 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e78109260 0x7f3e781096e0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f3e64009900 tx=0x7f3e6402bc30 comp rx=0 tx=0).stop 2024-03-27T10:56:31.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.730+0000 7f3e77200640 1 -- 172.21.15.97:0/801602950 shutdown_connections 2024-03-27T10:56:31.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.730+0000 7f3e77200640 1 --2- 172.21.15.97:0/801602950 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3e78109c20 0x7f3e781104b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:31.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.730+0000 7f3e77200640 1 --2- 172.21.15.97:0/801602950 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e78109260 0x7f3e781096e0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:31.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.730+0000 7f3e77200640 1 --2- 172.21.15.97:0/801602950 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3e78108070 0x7f3e78108490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:31.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.730+0000 7f3e77200640 1 -- 172.21.15.97:0/801602950 >> 172.21.15.97:0/801602950 conn(0x7f3e78077020 msgr2=0x7f3e78079440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:31.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.730+0000 7f3e77200640 1 -- 172.21.15.97:0/801602950 shutdown_connections 2024-03-27T10:56:31.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.730+0000 7f3e77200640 1 -- 172.21.15.97:0/801602950 wait complete. 2024-03-27T10:56:31.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.731+0000 7f3e77200640 1 Processor -- start 2024-03-27T10:56:31.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.731+0000 7f3e77200640 1 -- start start 2024-03-27T10:56:31.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.731+0000 7f3e77200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3e78108070 0x7f3e7819e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:31.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.731+0000 7f3e77200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e78109260 0x7f3e7819e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:31.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.732+0000 7f3e77200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3e78109c20 0x7f3e781a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:31.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.732+0000 7f3e77200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3e78113c00 con 0x7f3e78109260 2024-03-27T10:56:31.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.732+0000 7f3e77200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f3e78113a80 con 0x7f3e78109c20 2024-03-27T10:56:31.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.732+0000 7f3e77200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f3e78113d80 con 0x7f3e78108070 2024-03-27T10:56:31.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.732+0000 7f3e75400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e78109260 0x7f3e7819e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:31.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.732+0000 7f3e75400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e78109260 0x7f3e7819e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53468/0 (socket says 172.21.15.97:53468) 2024-03-27T10:56:31.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.732+0000 7f3e75400640 1 -- 172.21.15.97:0/389632256 learned_addr learned my addr 172.21.15.97:0/389632256 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:56:31.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.732+0000 7f3e76800640 1 --2- 172.21.15.97:0/389632256 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3e78109c20 0x7f3e781a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:31.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.732+0000 7f3e75e00640 1 --2- 172.21.15.97:0/389632256 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3e78108070 0x7f3e7819e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:31.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.732+0000 7f3e75400640 1 -- 172.21.15.97:0/389632256 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3e78108070 msgr2=0x7f3e7819e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:31.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.732+0000 7f3e75400640 1 --2- 172.21.15.97:0/389632256 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3e78108070 0x7f3e7819e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:31.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.732+0000 7f3e75400640 1 -- 172.21.15.97:0/389632256 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3e78109c20 msgr2=0x7f3e781a57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:31.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.732+0000 7f3e75400640 1 --2- 172.21.15.97:0/389632256 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3e78109c20 0x7f3e781a57b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:31.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.732+0000 7f3e75400640 1 -- 172.21.15.97:0/389632256 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e781a5e20 con 0x7f3e78109260 2024-03-27T10:56:31.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.732+0000 7f3e76800640 1 --2- 172.21.15.97:0/389632256 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3e78109c20 0x7f3e781a57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T10:56:31.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.733+0000 7f3e75400640 1 --2- 172.21.15.97:0/389632256 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e78109260 0x7f3e7819e740 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f3e64000c00 tx=0x7f3e640042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:31.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.733+0000 7f3e6ea00640 1 -- 172.21.15.97:0/389632256 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e64043020 con 0x7f3e78109260 2024-03-27T10:56:31.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.733+0000 7f3e6ea00640 1 -- 172.21.15.97:0/389632256 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e640054c0 con 0x7f3e78109260 2024-03-27T10:56:31.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.733+0000 7f3e6ea00640 1 -- 172.21.15.97:0/389632256 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e64003d40 con 0x7f3e78109260 2024-03-27T10:56:31.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.733+0000 7f3e77200640 1 -- 172.21.15.97:0/389632256 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e781a60b0 con 0x7f3e78109260 2024-03-27T10:56:31.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.733+0000 7f3e77200640 1 -- 172.21.15.97:0/389632256 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e781a6600 con 0x7f3e78109260 2024-03-27T10:56:31.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.734+0000 7f3e77200640 1 -- 172.21.15.97:0/389632256 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e7810f330 con 0x7f3e78109260 2024-03-27T10:56:31.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.736+0000 7f3e6ea00640 1 -- 172.21.15.97:0/389632256 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f3e64046020 con 0x7f3e78109260 2024-03-27T10:56:31.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.737+0000 7f3e6ea00640 1 --2- 172.21.15.97:0/389632256 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f3e5c0a3610 0x7f3e5c0a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:31.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.738+0000 7f3e6ea00640 1 -- 172.21.15.97:0/389632256 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f3e640f2690 con 0x7f3e78109260 2024-03-27T10:56:31.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.738+0000 7f3e75e00640 1 --2- 172.21.15.97:0/389632256 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f3e5c0a3610 0x7f3e5c0a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:31.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.738+0000 7f3e75e00640 1 --2- 172.21.15.97:0/389632256 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f3e5c0a3610 0x7f3e5c0a5ad0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f3e68005fd0 tx=0x7f3e68005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:31.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.740+0000 7f3e6ea00640 1 -- 172.21.15.97:0/389632256 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f3e6404d020 con 0x7f3e78109260 2024-03-27T10:56:31.767 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:56:31 smithi114 ceph-mon[35338]: pgmap v437: 97 pgs: 97 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1023 B/s wr, 13 op/s 2024-03-27T10:56:31.767 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:56:31 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/2966831902' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T10:56:31.767 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:56:31 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/3988627381' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-27T10:56:31.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.940+0000 7f3e77200640 1 -- 172.21.15.97:0/389632256 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f3e7802d5c0 con 0x7f3e78109260 2024-03-27T10:56:31.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.941+0000 7f3e6ea00640 1 -- 172.21.15.97:0/389632256 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) ==== 100+0+11 (secure 0 0 0) 0x7f3e640bf330 con 0x7f3e78109260 2024-03-27T10:56:31.913 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-03-27T10:56:31.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.944+0000 7f3e77200640 1 -- 172.21.15.97:0/389632256 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f3e5c0a3610 msgr2=0x7f3e5c0a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:31.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.944+0000 7f3e77200640 1 --2- 172.21.15.97:0/389632256 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f3e5c0a3610 0x7f3e5c0a5ad0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f3e68005fd0 tx=0x7f3e68005850 comp rx=0 tx=0).stop 2024-03-27T10:56:31.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.944+0000 7f3e77200640 1 -- 172.21.15.97:0/389632256 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e78109260 msgr2=0x7f3e7819e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:31.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.944+0000 7f3e77200640 1 --2- 172.21.15.97:0/389632256 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e78109260 0x7f3e7819e740 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f3e64000c00 tx=0x7f3e640042f0 comp rx=0 tx=0).stop 2024-03-27T10:56:31.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.944+0000 7f3e77200640 1 -- 172.21.15.97:0/389632256 shutdown_connections 2024-03-27T10:56:31.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.944+0000 7f3e77200640 1 --2- 172.21.15.97:0/389632256 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f3e5c0a3610 0x7f3e5c0a5ad0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:31.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.944+0000 7f3e77200640 1 --2- 172.21.15.97:0/389632256 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3e78109c20 0x7f3e781a57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:31.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.944+0000 7f3e77200640 1 --2- 172.21.15.97:0/389632256 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e78109260 0x7f3e7819e740 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:31.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.944+0000 7f3e77200640 1 --2- 172.21.15.97:0/389632256 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3e78108070 0x7f3e7819e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:31.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.944+0000 7f3e77200640 1 -- 172.21.15.97:0/389632256 >> 172.21.15.97:0/389632256 conn(0x7f3e78077020 msgr2=0x7f3e7810c500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:31.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.944+0000 7f3e77200640 1 -- 172.21.15.97:0/389632256 shutdown_connections 2024-03-27T10:56:31.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:31.944+0000 7f3e77200640 1 -- 172.21.15.97:0/389632256 wait complete. 2024-03-27T10:56:31.935 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-03-27T10:56:31.935 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-27T10:56:32.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.172+0000 7fb0ca800640 1 -- 172.21.15.97:0/3905504582 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0c40780f0 msgr2=0x7fb0c4078990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:32.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.172+0000 7fb0ca800640 1 --2- 172.21.15.97:0/3905504582 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0c40780f0 0x7fb0c4078990 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc009ca0 tx=0x7fb0bc02c0f0 comp rx=0 tx=0).stop 2024-03-27T10:56:32.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.172+0000 7fb0ca800640 1 -- 172.21.15.97:0/3905504582 shutdown_connections 2024-03-27T10:56:32.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.172+0000 7fb0ca800640 1 --2- 172.21.15.97:0/3905504582 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0c40780f0 0x7fb0c4078990 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:32.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.172+0000 7fb0ca800640 1 --2- 172.21.15.97:0/3905504582 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb0c4077750 0x7fb0c4077bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:32.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.172+0000 7fb0ca800640 1 --2- 172.21.15.97:0/3905504582 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb0c4076510 0x7fb0c4076930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:32.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.172+0000 7fb0ca800640 1 -- 172.21.15.97:0/3905504582 >> 172.21.15.97:0/3905504582 conn(0x7fb0c40ff300 msgr2=0x7fb0c4101720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:32.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.172+0000 7fb0ca800640 1 -- 172.21.15.97:0/3905504582 shutdown_connections 2024-03-27T10:56:32.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.173+0000 7fb0ca800640 1 -- 172.21.15.97:0/3905504582 wait complete. 2024-03-27T10:56:32.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.173+0000 7fb0ca800640 1 Processor -- start 2024-03-27T10:56:32.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.174+0000 7fb0ca800640 1 -- start start 2024-03-27T10:56:32.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.174+0000 7fb0ca800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb0c4076510 0x7fb0c419e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:32.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.174+0000 7fb0ca800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0c4077750 0x7fb0c419e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:32.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.174+0000 7fb0ca800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb0c40780f0 0x7fb0c41a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:32.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.174+0000 7fb0ca800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c4113cb0 con 0x7fb0c4077750 2024-03-27T10:56:32.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.174+0000 7fb0ca800640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c4113b30 con 0x7fb0c40780f0 2024-03-27T10:56:32.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.174+0000 7fb0ca800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c4113e30 con 0x7fb0c4076510 2024-03-27T10:56:32.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.174+0000 7fb0c2a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0c4077750 0x7fb0c419e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:32.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.174+0000 7fb0c2a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0c4077750 0x7fb0c419e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53484/0 (socket says 172.21.15.97:53484) 2024-03-27T10:56:32.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.174+0000 7fb0c2a00640 1 -- 172.21.15.97:0/1532575121 learned_addr learned my addr 172.21.15.97:0/1532575121 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:56:32.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.174+0000 7fb0c3e00640 1 --2- 172.21.15.97:0/1532575121 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb0c40780f0 0x7fb0c41a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:32.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.174+0000 7fb0c3400640 1 --2- 172.21.15.97:0/1532575121 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb0c4076510 0x7fb0c419e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:32.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.175+0000 7fb0c2a00640 1 -- 172.21.15.97:0/1532575121 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb0c4076510 msgr2=0x7fb0c419e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:32.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.175+0000 7fb0c2a00640 1 --2- 172.21.15.97:0/1532575121 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb0c4076510 0x7fb0c419e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:32.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.175+0000 7fb0c2a00640 1 -- 172.21.15.97:0/1532575121 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb0c40780f0 msgr2=0x7fb0c41a5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:32.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.175+0000 7fb0c2a00640 1 --2- 172.21.15.97:0/1532575121 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb0c40780f0 0x7fb0c41a5900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:32.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.175+0000 7fb0c2a00640 1 -- 172.21.15.97:0/1532575121 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0c41a5f70 con 0x7fb0c4077750 2024-03-27T10:56:32.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.175+0000 7fb0c3e00640 1 --2- 172.21.15.97:0/1532575121 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb0c40780f0 0x7fb0c41a5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T10:56:32.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.175+0000 7fb0c2a00640 1 --2- 172.21.15.97:0/1532575121 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0c4077750 0x7fb0c419e8e0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fb0b000d950 tx=0x7fb0b000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:32.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.175+0000 7fb0bbe00640 1 -- 172.21.15.97:0/1532575121 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b0017070 con 0x7fb0c4077750 2024-03-27T10:56:32.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.175+0000 7fb0bbe00640 1 -- 172.21.15.97:0/1532575121 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb0b0009c30 con 0x7fb0c4077750 2024-03-27T10:56:32.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.175+0000 7fb0ca800640 1 -- 172.21.15.97:0/1532575121 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0c41a6260 con 0x7fb0c4077750 2024-03-27T10:56:32.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.175+0000 7fb0bbe00640 1 -- 172.21.15.97:0/1532575121 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b000f620 con 0x7fb0c4077750 2024-03-27T10:56:32.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.175+0000 7fb0ca800640 1 -- 172.21.15.97:0/1532575121 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0c41a67b0 con 0x7fb0c4077750 2024-03-27T10:56:32.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.177+0000 7fb0ca800640 1 -- 172.21.15.97:0/1532575121 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0c41106e0 con 0x7fb0c4077750 2024-03-27T10:56:32.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.179+0000 7fb0bbe00640 1 -- 172.21.15.97:0/1532575121 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7fb0b00109a0 con 0x7fb0c4077750 2024-03-27T10:56:32.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.181+0000 7fb0bbe00640 1 --2- 172.21.15.97:0/1532575121 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fb0a80a3610 0x7fb0a80a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:32.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.181+0000 7fb0bbe00640 1 -- 172.21.15.97:0/1532575121 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fb0b00cfa80 con 0x7fb0c4077750 2024-03-27T10:56:32.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.181+0000 7fb0c3400640 1 --2- 172.21.15.97:0/1532575121 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fb0a80a3610 0x7fb0a80a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:32.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.181+0000 7fb0c3400640 1 --2- 172.21.15.97:0/1532575121 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fb0a80a3610 0x7fb0a80a5ad0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fb0b4005fd0 tx=0x7fb0b4005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:32.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.185+0000 7fb0bbe00640 1 -- 172.21.15.97:0/1532575121 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7fb0b00d1e20 con 0x7fb0c4077750 2024-03-27T10:56:32.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.434+0000 7fb0ca800640 1 -- 172.21.15.97:0/1532575121 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb0c4062750 con 0x7fb0c4077750 2024-03-27T10:56:32.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.435+0000 7fb0bbe00640 1 -- 172.21.15.97:0/1532575121 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11944 (secure 0 0 0) 0x7fb0b0009de0 con 0x7fb0c4077750 2024-03-27T10:56:32.407 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T10:56:32.407 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1009423904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1009423904},{"type":"v1","addr":"172.21.15.97:6835","nonce":1009423904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3251173433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3251173433},{"type":"v1","addr":"172.21.15.97:6837","nonce":3251173433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-27T10:54:04.510178+0000","modified":"2024-03-27T10:56:28.179534+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34508,"mds_1":34478,"mds_2":34445,"mds_3":24454,"mds_4":24400},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14646":{"gid":14646,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6839/1057596821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1057596821},{"type":"v1","addr":"172.21.15.97:6839","nonce":1057596821}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14658":{"gid":14658,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6841/1672464856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6840","nonce":1672464856},{"type":"v1","addr":"172.21.15.97:6841","nonce":1672464856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.177:6833/3787981632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3787981632},{"type":"v1","addr":"172.21.15.177:6833","nonce":3787981632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.177:6835/3349112132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":3349112132},{"type":"v1","addr":"172.21.15.177:6835","nonce":3349112132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6837/3244432640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":3244432640},{"type":"v1","addr":"172.21.15.177:6837","nonce":3244432640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6839/3340454605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":3340454605},{"type":"v1","addr":"172.21.15.177:6839","nonce":3340454605}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34415":{"gid":34415,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6833/4052517902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":4052517902},{"type":"v1","addr":"172.21.15.114:6833","nonce":4052517902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34445":{"gid":34445,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.114:6835/3581918222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3581918222},{"type":"v1","addr":"172.21.15.114:6835","nonce":3581918222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34478":{"gid":34478,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.114:6837/616775588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":616775588},{"type":"v1","addr":"172.21.15.114:6837","nonce":616775588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34508":{"gid":34508,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.114:6839/3937193784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":3937193784},{"type":"v1","addr":"172.21.15.114:6839","nonce":3937193784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34508,"qdb_cluster":[34478,24454,24400,34445,34508]},"id":1}]} 2024-03-27T10:56:32.407 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 28 2024-03-27T10:56:32.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.438+0000 7fb0ca800640 1 -- 172.21.15.97:0/1532575121 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fb0a80a3610 msgr2=0x7fb0a80a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:32.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.438+0000 7fb0ca800640 1 --2- 172.21.15.97:0/1532575121 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fb0a80a3610 0x7fb0a80a5ad0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fb0b4005fd0 tx=0x7fb0b4005850 comp rx=0 tx=0).stop 2024-03-27T10:56:32.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.438+0000 7fb0ca800640 1 -- 172.21.15.97:0/1532575121 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0c4077750 msgr2=0x7fb0c419e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:32.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.438+0000 7fb0ca800640 1 --2- 172.21.15.97:0/1532575121 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0c4077750 0x7fb0c419e8e0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fb0b000d950 tx=0x7fb0b000de10 comp rx=0 tx=0).stop 2024-03-27T10:56:32.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.438+0000 7fb0ca800640 1 -- 172.21.15.97:0/1532575121 shutdown_connections 2024-03-27T10:56:32.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.438+0000 7fb0ca800640 1 --2- 172.21.15.97:0/1532575121 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fb0a80a3610 0x7fb0a80a5ad0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:32.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.438+0000 7fb0ca800640 1 --2- 172.21.15.97:0/1532575121 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb0c40780f0 0x7fb0c41a5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:32.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.438+0000 7fb0ca800640 1 --2- 172.21.15.97:0/1532575121 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0c4077750 0x7fb0c419e8e0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:32.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.438+0000 7fb0ca800640 1 --2- 172.21.15.97:0/1532575121 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb0c4076510 0x7fb0c419e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:32.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.438+0000 7fb0ca800640 1 -- 172.21.15.97:0/1532575121 >> 172.21.15.97:0/1532575121 conn(0x7fb0c40ff300 msgr2=0x7fb0c4100e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:32.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.438+0000 7fb0ca800640 1 -- 172.21.15.97:0/1532575121 shutdown_connections 2024-03-27T10:56:32.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.438+0000 7fb0ca800640 1 -- 172.21.15.97:0/1532575121 wait complete. 2024-03-27T10:56:32.427 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-03-27T10:56:32.428 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-03-27T10:56:32.428 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-27T10:56:32.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.647+0000 7f9f83200640 1 -- 172.21.15.97:0/4208647008 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f7c107640 msgr2=0x7f9f7c109a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:32.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.647+0000 7f9f83200640 1 --2- 172.21.15.97:0/4208647008 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f7c107640 0x7f9f7c109a90 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f9f78009ca0 tx=0x7f9f7802c0f0 comp rx=0 tx=0).stop 2024-03-27T10:56:32.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.647+0000 7f9f83200640 1 -- 172.21.15.97:0/4208647008 shutdown_connections 2024-03-27T10:56:32.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.647+0000 7f9f83200640 1 --2- 172.21.15.97:0/4208647008 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f7c107640 0x7f9f7c109a90 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:32.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.647+0000 7f9f83200640 1 --2- 172.21.15.97:0/4208647008 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9f7c104d10 0x7f9f7c107100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:32.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.647+0000 7f9f83200640 1 --2- 172.21.15.97:0/4208647008 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9f7c1023e0 0x7f9f7c1047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:32.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.647+0000 7f9f83200640 1 -- 172.21.15.97:0/4208647008 >> 172.21.15.97:0/4208647008 conn(0x7f9f7c0fbdb0 msgr2=0x7f9f7c0fe210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:32.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.647+0000 7f9f83200640 1 -- 172.21.15.97:0/4208647008 shutdown_connections 2024-03-27T10:56:32.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.647+0000 7f9f83200640 1 -- 172.21.15.97:0/4208647008 wait complete. 2024-03-27T10:56:32.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.648+0000 7f9f83200640 1 Processor -- start 2024-03-27T10:56:32.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.648+0000 7f9f83200640 1 -- start start 2024-03-27T10:56:32.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.649+0000 7f9f83200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9f7c1023e0 0x7f9f7c199d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:32.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.649+0000 7f9f83200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9f7c104d10 0x7f9f7c19a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:32.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.649+0000 7f9f83200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f7c107640 0x7f9f7c1a1290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:32.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.649+0000 7f9f83200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9f7c10d4b0 con 0x7f9f7c107640 2024-03-27T10:56:32.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.649+0000 7f9f83200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9f7c10d330 con 0x7f9f7c104d10 2024-03-27T10:56:32.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.649+0000 7f9f83200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f9f7c10d630 con 0x7f9f7c1023e0 2024-03-27T10:56:32.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.649+0000 7f9f82800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f7c107640 0x7f9f7c1a1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:32.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.649+0000 7f9f82800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f7c107640 0x7f9f7c1a1290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53506/0 (socket says 172.21.15.97:53506) 2024-03-27T10:56:32.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.649+0000 7f9f82800640 1 -- 172.21.15.97:0/2616035847 learned_addr learned my addr 172.21.15.97:0/2616035847 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:56:32.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.649+0000 7f9f81e00640 1 --2- 172.21.15.97:0/2616035847 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9f7c1023e0 0x7f9f7c199d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:32.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.649+0000 7f9f81400640 1 --2- 172.21.15.97:0/2616035847 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9f7c104d10 0x7f9f7c19a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:32.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.649+0000 7f9f82800640 1 -- 172.21.15.97:0/2616035847 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9f7c1023e0 msgr2=0x7f9f7c199d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:32.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.649+0000 7f9f82800640 1 --2- 172.21.15.97:0/2616035847 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9f7c1023e0 0x7f9f7c199d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:32.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.649+0000 7f9f82800640 1 -- 172.21.15.97:0/2616035847 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9f7c104d10 msgr2=0x7f9f7c19a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:32.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.649+0000 7f9f82800640 1 --2- 172.21.15.97:0/2616035847 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9f7c104d10 0x7f9f7c19a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:32.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.649+0000 7f9f82800640 1 -- 172.21.15.97:0/2616035847 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f7c1a1930 con 0x7f9f7c107640 2024-03-27T10:56:32.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.650+0000 7f9f81400640 1 --2- 172.21.15.97:0/2616035847 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9f7c104d10 0x7f9f7c19a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T10:56:32.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.650+0000 7f9f82800640 1 --2- 172.21.15.97:0/2616035847 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f7c107640 0x7f9f7c1a1290 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f9f7802c700 tx=0x7f9f780328a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:32.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.650+0000 7f9f76a00640 1 -- 172.21.15.97:0/2616035847 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f78041020 con 0x7f9f7c107640 2024-03-27T10:56:32.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.650+0000 7f9f76a00640 1 -- 172.21.15.97:0/2616035847 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9f78003e20 con 0x7f9f7c107640 2024-03-27T10:56:32.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.650+0000 7f9f83200640 1 -- 172.21.15.97:0/2616035847 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f7c1a1bc0 con 0x7f9f7c107640 2024-03-27T10:56:32.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.650+0000 7f9f76a00640 1 -- 172.21.15.97:0/2616035847 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f78039040 con 0x7f9f7c107640 2024-03-27T10:56:32.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.650+0000 7f9f83200640 1 -- 172.21.15.97:0/2616035847 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f7c1a2100 con 0x7f9f7c107640 2024-03-27T10:56:32.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.652+0000 7f9f83200640 1 -- 172.21.15.97:0/2616035847 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f7c109130 con 0x7f9f7c107640 2024-03-27T10:56:32.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.653+0000 7f9f76a00640 1 -- 172.21.15.97:0/2616035847 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f9f78044020 con 0x7f9f7c107640 2024-03-27T10:56:32.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.655+0000 7f9f76a00640 1 --2- 172.21.15.97:0/2616035847 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f9f640a3690 0x7f9f640a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:32.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.655+0000 7f9f76a00640 1 -- 172.21.15.97:0/2616035847 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f9f780f2030 con 0x7f9f7c107640 2024-03-27T10:56:32.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.655+0000 7f9f81e00640 1 --2- 172.21.15.97:0/2616035847 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f9f640a3690 0x7f9f640a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:32.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.656+0000 7f9f81e00640 1 --2- 172.21.15.97:0/2616035847 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f9f640a3690 0x7f9f640a5b50 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c19b1f0 tx=0x7f9f7000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:32.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.659+0000 7f9f76a00640 1 -- 172.21.15.97:0/2616035847 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f9f7804b020 con 0x7f9f7c107640 2024-03-27T10:56:32.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:56:32 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/3478131444' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-27T10:56:32.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:56:32 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/389632256' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-27T10:56:32.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:56:32 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/3478131444' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-27T10:56:32.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:56:32 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/389632256' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-27T10:56:32.767 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:56:32 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/3478131444' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-27T10:56:32.767 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:56:32 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/389632256' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-27T10:56:32.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.857+0000 7f9f83200640 1 -- 172.21.15.97:0/2616035847 --> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9f7c061f90 con 0x7f9f640a3690 2024-03-27T10:56:32.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.868+0000 7f9f76a00640 1 -- 172.21.15.97:0/2616035847 <== mgr.14152 v2:172.21.15.97:6800/3955463538 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258915 (secure 0 0 0) 0x7f9f7c061f90 con 0x7f9f640a3690 2024-03-27T10:56:32.843 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T10:56:32.845 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-03-27T10:56:32.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.874+0000 7f9f83200640 1 -- 172.21.15.97:0/2616035847 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f9f640a3690 msgr2=0x7f9f640a5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:32.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.874+0000 7f9f83200640 1 --2- 172.21.15.97:0/2616035847 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f9f640a3690 0x7f9f640a5b50 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c19b1f0 tx=0x7f9f7000b390 comp rx=0 tx=0).stop 2024-03-27T10:56:32.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.874+0000 7f9f83200640 1 -- 172.21.15.97:0/2616035847 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f7c107640 msgr2=0x7f9f7c1a1290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:32.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.874+0000 7f9f83200640 1 --2- 172.21.15.97:0/2616035847 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f7c107640 0x7f9f7c1a1290 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f9f7802c700 tx=0x7f9f780328a0 comp rx=0 tx=0).stop 2024-03-27T10:56:32.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.874+0000 7f9f83200640 1 -- 172.21.15.97:0/2616035847 shutdown_connections 2024-03-27T10:56:32.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.874+0000 7f9f83200640 1 --2- 172.21.15.97:0/2616035847 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f7c107640 0x7f9f7c1a1290 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:32.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.874+0000 7f9f83200640 1 --2- 172.21.15.97:0/2616035847 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f9f640a3690 0x7f9f640a5b50 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:32.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.874+0000 7f9f83200640 1 --2- 172.21.15.97:0/2616035847 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9f7c104d10 0x7f9f7c19a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:32.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.874+0000 7f9f83200640 1 --2- 172.21.15.97:0/2616035847 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9f7c1023e0 0x7f9f7c199d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:32.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.874+0000 7f9f83200640 1 -- 172.21.15.97:0/2616035847 >> 172.21.15.97:0/2616035847 conn(0x7f9f7c0fbdb0 msgr2=0x7f9f7c101b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:32.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.874+0000 7f9f83200640 1 -- 172.21.15.97:0/2616035847 shutdown_connections 2024-03-27T10:56:32.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:32.874+0000 7f9f83200640 1 -- 172.21.15.97:0/2616035847 wait complete. 2024-03-27T10:56:32.865 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":438,"stamp":"2024-03-27T10:56:32.116811+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":661848,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1550,"num_read_kb":897,"num_write":447,"num_write_kb":3093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1181318,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":867,"ondisk_log_size":867,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":379728,"kb_used_data":13936,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124562096,"statfs":{"total":1151940427776,"available":1151551586304,"internally_reserved":0,"allocated":14270464,"data_stored":7740870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":374453561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2465,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":105,"num_read_kb":55,"num_write":16,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005541"},"pg_stats":[{"pgid":"3.3f","version":"92'8","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.077510+0000","last_change":"2024-03-27T10:54:05.077845+0000","last_active":"2024-03-27T10:55:40.077510+0000","last_peered":"2024-03-27T10:55:40.077510+0000","last_clean":"2024-03-27T10:55:40.077510+0000","last_became_active":"2024-03-27T10:54:05.077494+0000","last_became_peered":"2024-03-27T10:54:05.077494+0000","last_unstale":"2024-03-27T10:55:40.077510+0000","last_undegraded":"2024-03-27T10:55:40.077510+0000","last_fullsized":"2024-03-27T10:55:40.077510+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:16:07.094746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.048785+0000","last_change":"2024-03-27T10:54:05.078830+0000","last_active":"2024-03-27T10:55:40.048785+0000","last_peered":"2024-03-27T10:55:40.048785+0000","last_clean":"2024-03-27T10:55:40.048785+0000","last_became_active":"2024-03-27T10:54:05.078474+0000","last_became_peered":"2024-03-27T10:54:05.078474+0000","last_unstale":"2024-03-27T10:55:40.048785+0000","last_undegraded":"2024-03-27T10:55:40.048785+0000","last_fullsized":"2024-03-27T10:55:40.048785+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T20:11:09.900802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.075900+0000","last_change":"2024-03-27T10:54:05.082554+0000","last_active":"2024-03-27T10:55:40.075900+0000","last_peered":"2024-03-27T10:55:40.075900+0000","last_clean":"2024-03-27T10:55:40.075900+0000","last_became_active":"2024-03-27T10:54:05.081395+0000","last_became_peered":"2024-03-27T10:54:05.081395+0000","last_unstale":"2024-03-27T10:55:40.075900+0000","last_undegraded":"2024-03-27T10:55:40.075900+0000","last_fullsized":"2024-03-27T10:55:40.075900+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T20:02:58.866792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.033675+0000","last_change":"2024-03-27T10:54:05.086700+0000","last_active":"2024-03-27T10:55:40.033675+0000","last_peered":"2024-03-27T10:55:40.033675+0000","last_clean":"2024-03-27T10:55:40.033675+0000","last_became_active":"2024-03-27T10:54:05.086347+0000","last_became_peered":"2024-03-27T10:54:05.086347+0000","last_unstale":"2024-03-27T10:55:40.033675+0000","last_undegraded":"2024-03-27T10:55:40.033675+0000","last_fullsized":"2024-03-27T10:55:40.033675+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T22:03:11.336365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.047556+0000","last_change":"2024-03-27T10:54:05.083479+0000","last_active":"2024-03-27T10:55:40.047556+0000","last_peered":"2024-03-27T10:55:40.047556+0000","last_clean":"2024-03-27T10:55:40.047556+0000","last_became_active":"2024-03-27T10:54:05.083162+0000","last_became_peered":"2024-03-27T10:54:05.083162+0000","last_unstale":"2024-03-27T10:55:40.047556+0000","last_undegraded":"2024-03-27T10:55:40.047556+0000","last_fullsized":"2024-03-27T10:55:40.047556+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:23:42.265668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.050257+0000","last_change":"2024-03-27T10:54:05.079096+0000","last_active":"2024-03-27T10:55:40.050257+0000","last_peered":"2024-03-27T10:55:40.050257+0000","last_clean":"2024-03-27T10:55:40.050257+0000","last_became_active":"2024-03-27T10:54:05.078778+0000","last_became_peered":"2024-03-27T10:54:05.078778+0000","last_unstale":"2024-03-27T10:55:40.050257+0000","last_undegraded":"2024-03-27T10:55:40.050257+0000","last_fullsized":"2024-03-27T10:55:40.050257+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T19:26:47.534594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.053852+0000","last_change":"2024-03-27T10:54:05.098926+0000","last_active":"2024-03-27T10:55:40.053852+0000","last_peered":"2024-03-27T10:55:40.053852+0000","last_clean":"2024-03-27T10:55:40.053852+0000","last_became_active":"2024-03-27T10:54:05.098139+0000","last_became_peered":"2024-03-27T10:54:05.098139+0000","last_unstale":"2024-03-27T10:55:40.053852+0000","last_undegraded":"2024-03-27T10:55:40.053852+0000","last_fullsized":"2024-03-27T10:55:40.053852+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:35:14.335947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.050624+0000","last_change":"2024-03-27T10:54:05.093786+0000","last_active":"2024-03-27T10:55:40.050624+0000","last_peered":"2024-03-27T10:55:40.050624+0000","last_clean":"2024-03-27T10:55:40.050624+0000","last_became_active":"2024-03-27T10:54:05.092047+0000","last_became_peered":"2024-03-27T10:54:05.092047+0000","last_unstale":"2024-03-27T10:55:40.050624+0000","last_undegraded":"2024-03-27T10:55:40.050624+0000","last_fullsized":"2024-03-27T10:55:40.050624+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T22:50:30.778358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.073745+0000","last_change":"2024-03-27T10:54:05.081678+0000","last_active":"2024-03-27T10:55:40.073745+0000","last_peered":"2024-03-27T10:55:40.073745+0000","last_clean":"2024-03-27T10:55:40.073745+0000","last_became_active":"2024-03-27T10:54:05.080264+0000","last_became_peered":"2024-03-27T10:54:05.080264+0000","last_unstale":"2024-03-27T10:55:40.073745+0000","last_undegraded":"2024-03-27T10:55:40.073745+0000","last_fullsized":"2024-03-27T10:55:40.073745+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T22:41:04.183282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.063980+0000","last_change":"2024-03-27T10:54:05.095172+0000","last_active":"2024-03-27T10:55:40.063980+0000","last_peered":"2024-03-27T10:55:40.063980+0000","last_clean":"2024-03-27T10:55:40.063980+0000","last_became_active":"2024-03-27T10:54:05.088438+0000","last_became_peered":"2024-03-27T10:54:05.088438+0000","last_unstale":"2024-03-27T10:55:40.063980+0000","last_undegraded":"2024-03-27T10:55:40.063980+0000","last_fullsized":"2024-03-27T10:55:40.063980+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T11:19:50.422788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.076689+0000","last_change":"2024-03-27T10:54:05.093599+0000","last_active":"2024-03-27T10:55:40.076689+0000","last_peered":"2024-03-27T10:55:40.076689+0000","last_clean":"2024-03-27T10:55:40.076689+0000","last_became_active":"2024-03-27T10:54:05.092248+0000","last_became_peered":"2024-03-27T10:54:05.092248+0000","last_unstale":"2024-03-27T10:55:40.076689+0000","last_undegraded":"2024-03-27T10:55:40.076689+0000","last_fullsized":"2024-03-27T10:55:40.076689+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:22:38.712442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.039555+0000","last_change":"2024-03-27T10:54:05.094677+0000","last_active":"2024-03-27T10:55:40.039555+0000","last_peered":"2024-03-27T10:55:40.039555+0000","last_clean":"2024-03-27T10:55:40.039555+0000","last_became_active":"2024-03-27T10:54:05.094326+0000","last_became_peered":"2024-03-27T10:54:05.094326+0000","last_unstale":"2024-03-27T10:55:40.039555+0000","last_undegraded":"2024-03-27T10:55:40.039555+0000","last_fullsized":"2024-03-27T10:55:40.039555+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T21:44:51.923881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.052563+0000","last_change":"2024-03-27T10:54:05.083178+0000","last_active":"2024-03-27T10:55:40.052563+0000","last_peered":"2024-03-27T10:55:40.052563+0000","last_clean":"2024-03-27T10:55:40.052563+0000","last_became_active":"2024-03-27T10:54:05.082805+0000","last_became_peered":"2024-03-27T10:54:05.082805+0000","last_unstale":"2024-03-27T10:55:40.052563+0000","last_undegraded":"2024-03-27T10:55:40.052563+0000","last_fullsized":"2024-03-27T10:55:40.052563+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T19:03:07.558006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.040999+0000","last_change":"2024-03-27T10:54:05.098092+0000","last_active":"2024-03-27T10:55:40.040999+0000","last_peered":"2024-03-27T10:55:40.040999+0000","last_clean":"2024-03-27T10:55:40.040999+0000","last_became_active":"2024-03-27T10:54:05.097759+0000","last_became_peered":"2024-03-27T10:54:05.097759+0000","last_unstale":"2024-03-27T10:55:40.040999+0000","last_undegraded":"2024-03-27T10:55:40.040999+0000","last_fullsized":"2024-03-27T10:55:40.040999+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:14:07.884759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.033846+0000","last_change":"2024-03-27T10:54:05.086089+0000","last_active":"2024-03-27T10:55:40.033846+0000","last_peered":"2024-03-27T10:55:40.033846+0000","last_clean":"2024-03-27T10:55:40.033846+0000","last_became_active":"2024-03-27T10:54:05.084830+0000","last_became_peered":"2024-03-27T10:54:05.084830+0000","last_unstale":"2024-03-27T10:55:40.033846+0000","last_undegraded":"2024-03-27T10:55:40.033846+0000","last_fullsized":"2024-03-27T10:55:40.033846+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T20:37:44.390102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.080920+0000","last_change":"2024-03-27T10:54:05.092190+0000","last_active":"2024-03-27T10:55:40.080920+0000","last_peered":"2024-03-27T10:55:40.080920+0000","last_clean":"2024-03-27T10:55:40.080920+0000","last_became_active":"2024-03-27T10:54:05.091827+0000","last_became_peered":"2024-03-27T10:54:05.091827+0000","last_unstale":"2024-03-27T10:55:40.080920+0000","last_undegraded":"2024-03-27T10:55:40.080920+0000","last_fullsized":"2024-03-27T10:55:40.080920+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T21:12:13.452829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.077720+0000","last_change":"2024-03-27T10:54:05.086924+0000","last_active":"2024-03-27T10:55:40.077720+0000","last_peered":"2024-03-27T10:55:40.077720+0000","last_clean":"2024-03-27T10:55:40.077720+0000","last_became_active":"2024-03-27T10:54:05.085701+0000","last_became_peered":"2024-03-27T10:54:05.085701+0000","last_unstale":"2024-03-27T10:55:40.077720+0000","last_undegraded":"2024-03-27T10:55:40.077720+0000","last_fullsized":"2024-03-27T10:55:40.077720+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:43:38.612288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.070593+0000","last_change":"2024-03-27T10:54:05.082532+0000","last_active":"2024-03-27T10:55:40.070593+0000","last_peered":"2024-03-27T10:55:40.070593+0000","last_clean":"2024-03-27T10:55:40.070593+0000","last_became_active":"2024-03-27T10:54:05.082200+0000","last_became_peered":"2024-03-27T10:54:05.082200+0000","last_unstale":"2024-03-27T10:55:40.070593+0000","last_undegraded":"2024-03-27T10:55:40.070593+0000","last_fullsized":"2024-03-27T10:55:40.070593+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T19:07:55.232728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.048428+0000","last_change":"2024-03-27T10:54:05.088204+0000","last_active":"2024-03-27T10:55:40.048428+0000","last_peered":"2024-03-27T10:55:40.048428+0000","last_clean":"2024-03-27T10:55:40.048428+0000","last_became_active":"2024-03-27T10:54:05.087705+0000","last_became_peered":"2024-03-27T10:54:05.087705+0000","last_unstale":"2024-03-27T10:55:40.048428+0000","last_undegraded":"2024-03-27T10:55:40.048428+0000","last_fullsized":"2024-03-27T10:55:40.048428+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T19:24:52.450205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"92'14","reported_seq":170,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:56:25.842018+0000","last_change":"2024-03-27T10:54:04.072182+0000","last_active":"2024-03-27T10:56:25.842018+0000","last_peered":"2024-03-27T10:56:25.842018+0000","last_clean":"2024-03-27T10:56:25.842018+0000","last_became_active":"2024-03-27T10:54:04.071827+0000","last_became_peered":"2024-03-27T10:54:04.071827+0000","last_unstale":"2024-03-27T10:56:25.842018+0000","last_undegraded":"2024-03-27T10:56:25.842018+0000","last_fullsized":"2024-03-27T10:56:25.842018+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T18:40:19.776684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":136,"num_read_kb":136,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.043065+0000","last_change":"2024-03-27T10:54:05.095772+0000","last_active":"2024-03-27T10:55:40.043065+0000","last_peered":"2024-03-27T10:55:40.043065+0000","last_clean":"2024-03-27T10:55:40.043065+0000","last_became_active":"2024-03-27T10:54:05.095385+0000","last_became_peered":"2024-03-27T10:54:05.095385+0000","last_unstale":"2024-03-27T10:55:40.043065+0000","last_undegraded":"2024-03-27T10:55:40.043065+0000","last_fullsized":"2024-03-27T10:55:40.043065+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T12:07:21.744097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.337196+0000","last_change":"2024-03-27T10:54:04.069600+0000","last_active":"2024-03-27T10:54:41.337196+0000","last_peered":"2024-03-27T10:54:41.337196+0000","last_clean":"2024-03-27T10:54:41.337196+0000","last_became_active":"2024-03-27T10:54:04.067770+0000","last_became_peered":"2024-03-27T10:54:04.067770+0000","last_unstale":"2024-03-27T10:54:41.337196+0000","last_undegraded":"2024-03-27T10:54:41.337196+0000","last_fullsized":"2024-03-27T10:54:41.337196+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T12:26:36.930945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.076797+0000","last_change":"2024-03-27T10:54:05.081779+0000","last_active":"2024-03-27T10:55:40.076797+0000","last_peered":"2024-03-27T10:55:40.076797+0000","last_clean":"2024-03-27T10:55:40.076797+0000","last_became_active":"2024-03-27T10:54:05.081492+0000","last_became_peered":"2024-03-27T10:54:05.081492+0000","last_unstale":"2024-03-27T10:55:40.076797+0000","last_undegraded":"2024-03-27T10:55:40.076797+0000","last_fullsized":"2024-03-27T10:55:40.076797+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T19:07:12.838048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"90'12","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.329099+0000","last_change":"2024-03-27T10:54:04.070550+0000","last_active":"2024-03-27T10:54:41.329099+0000","last_peered":"2024-03-27T10:54:41.329099+0000","last_clean":"2024-03-27T10:54:41.329099+0000","last_became_active":"2024-03-27T10:54:04.070010+0000","last_became_peered":"2024-03-27T10:54:04.070010+0000","last_unstale":"2024-03-27T10:54:41.329099+0000","last_undegraded":"2024-03-27T10:54:41.329099+0000","last_fullsized":"2024-03-27T10:54:41.329099+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:46:34.168771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.079329+0000","last_change":"2024-03-27T10:54:05.086372+0000","last_active":"2024-03-27T10:55:40.079329+0000","last_peered":"2024-03-27T10:55:40.079329+0000","last_clean":"2024-03-27T10:55:40.079329+0000","last_became_active":"2024-03-27T10:54:05.086013+0000","last_became_peered":"2024-03-27T10:54:05.086013+0000","last_unstale":"2024-03-27T10:55:40.079329+0000","last_undegraded":"2024-03-27T10:55:40.079329+0000","last_fullsized":"2024-03-27T10:55:40.079329+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:17:39.059071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"90'5","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.347107+0000","last_change":"2024-03-27T10:54:41.346475+0000","last_active":"2024-03-27T10:54:41.347107+0000","last_peered":"2024-03-27T10:54:41.347107+0000","last_clean":"2024-03-27T10:54:41.347107+0000","last_became_active":"2024-03-27T10:54:41.337431+0000","last_became_peered":"2024-03-27T10:54:41.337431+0000","last_unstale":"2024-03-27T10:54:41.347107+0000","last_undegraded":"2024-03-27T10:54:41.347107+0000","last_fullsized":"2024-03-27T10:54:41.347107+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T20:57:10.552047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.075638+0000","last_change":"2024-03-27T10:54:05.087580+0000","last_active":"2024-03-27T10:55:40.075638+0000","last_peered":"2024-03-27T10:55:40.075638+0000","last_clean":"2024-03-27T10:55:40.075638+0000","last_became_active":"2024-03-27T10:54:05.087128+0000","last_became_peered":"2024-03-27T10:54:05.087128+0000","last_unstale":"2024-03-27T10:55:40.075638+0000","last_undegraded":"2024-03-27T10:55:40.075638+0000","last_fullsized":"2024-03-27T10:55:40.075638+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T22:03:46.861258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"92'8","reported_seq":172,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:56:28.513097+0000","last_change":"2024-03-27T10:54:04.066312+0000","last_active":"2024-03-27T10:56:28.513097+0000","last_peered":"2024-03-27T10:56:28.513097+0000","last_clean":"2024-03-27T10:56:28.513097+0000","last_became_active":"2024-03-27T10:54:04.065909+0000","last_became_peered":"2024-03-27T10:54:04.065909+0000","last_unstale":"2024-03-27T10:56:28.513097+0000","last_undegraded":"2024-03-27T10:56:28.513097+0000","last_fullsized":"2024-03-27T10:56:28.513097+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:08:45.257808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1647,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":145,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.040498+0000","last_change":"2024-03-27T10:54:41.370991+0000","last_active":"2024-03-27T10:55:40.040498+0000","last_peered":"2024-03-27T10:55:40.040498+0000","last_clean":"2024-03-27T10:55:40.040498+0000","last_became_active":"2024-03-27T10:54:41.370596+0000","last_became_peered":"2024-03-27T10:54:41.370596+0000","last_unstale":"2024-03-27T10:55:40.040498+0000","last_undegraded":"2024-03-27T10:55:40.040498+0000","last_fullsized":"2024-03-27T10:55:40.040498+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:29:44.187503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"90'7","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.331143+0000","last_change":"2024-03-27T10:54:04.072451+0000","last_active":"2024-03-27T10:54:41.331143+0000","last_peered":"2024-03-27T10:54:41.331143+0000","last_clean":"2024-03-27T10:54:41.331143+0000","last_became_active":"2024-03-27T10:54:04.072042+0000","last_became_peered":"2024-03-27T10:54:04.072042+0000","last_unstale":"2024-03-27T10:54:41.331143+0000","last_undegraded":"2024-03-27T10:54:41.331143+0000","last_fullsized":"2024-03-27T10:54:41.331143+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T12:20:43.231073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.047114+0000","last_change":"2024-03-27T10:54:05.086055+0000","last_active":"2024-03-27T10:55:40.047114+0000","last_peered":"2024-03-27T10:55:40.047114+0000","last_clean":"2024-03-27T10:55:40.047114+0000","last_became_active":"2024-03-27T10:54:05.084429+0000","last_became_peered":"2024-03-27T10:54:05.084429+0000","last_unstale":"2024-03-27T10:55:40.047114+0000","last_undegraded":"2024-03-27T10:55:40.047114+0000","last_fullsized":"2024-03-27T10:55:40.047114+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T12:41:12.868245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"90'3","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.369391+0000","last_change":"2024-03-27T10:54:04.059076+0000","last_active":"2024-03-27T10:54:41.369391+0000","last_peered":"2024-03-27T10:54:41.369391+0000","last_clean":"2024-03-27T10:54:41.369391+0000","last_became_active":"2024-03-27T10:54:04.057614+0000","last_became_peered":"2024-03-27T10:54:04.057614+0000","last_unstale":"2024-03-27T10:54:41.369391+0000","last_undegraded":"2024-03-27T10:54:41.369391+0000","last_fullsized":"2024-03-27T10:54:41.369391+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T22:26:16.299840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.047600+0000","last_change":"2024-03-27T10:54:05.092955+0000","last_active":"2024-03-27T10:55:40.047600+0000","last_peered":"2024-03-27T10:55:40.047600+0000","last_clean":"2024-03-27T10:55:40.047600+0000","last_became_active":"2024-03-27T10:54:05.091867+0000","last_became_peered":"2024-03-27T10:54:05.091867+0000","last_unstale":"2024-03-27T10:55:40.047600+0000","last_undegraded":"2024-03-27T10:55:40.047600+0000","last_fullsized":"2024-03-27T10:55:40.047600+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T21:41:04.636480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"90'3","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.331834+0000","last_change":"2024-03-27T10:54:04.058833+0000","last_active":"2024-03-27T10:54:41.331834+0000","last_peered":"2024-03-27T10:54:41.331834+0000","last_clean":"2024-03-27T10:54:41.331834+0000","last_became_active":"2024-03-27T10:54:04.057750+0000","last_became_peered":"2024-03-27T10:54:04.057750+0000","last_unstale":"2024-03-27T10:54:41.331834+0000","last_undegraded":"2024-03-27T10:54:41.331834+0000","last_fullsized":"2024-03-27T10:54:41.331834+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:03:18.771094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.048019+0000","last_change":"2024-03-27T10:54:05.078933+0000","last_active":"2024-03-27T10:55:40.048019+0000","last_peered":"2024-03-27T10:55:40.048019+0000","last_clean":"2024-03-27T10:55:40.048019+0000","last_became_active":"2024-03-27T10:54:05.078544+0000","last_became_peered":"2024-03-27T10:54:05.078544+0000","last_unstale":"2024-03-27T10:55:40.048019+0000","last_undegraded":"2024-03-27T10:55:40.048019+0000","last_fullsized":"2024-03-27T10:55:40.048019+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T19:40:25.529775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"92'7","reported_seq":301,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:56:27.533739+0000","last_change":"2024-03-27T10:54:04.063974+0000","last_active":"2024-03-27T10:56:27.533739+0000","last_peered":"2024-03-27T10:56:27.533739+0000","last_clean":"2024-03-27T10:56:27.533739+0000","last_became_active":"2024-03-27T10:54:04.058171+0000","last_became_peered":"2024-03-27T10:54:04.058171+0000","last_unstale":"2024-03-27T10:56:27.533739+0000","last_undegraded":"2024-03-27T10:56:27.533739+0000","last_fullsized":"2024-03-27T10:56:27.533739+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T19:14:15.597889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":274,"num_read_kb":274,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.079116+0000","last_change":"2024-03-27T10:54:05.085727+0000","last_active":"2024-03-27T10:55:40.079116+0000","last_peered":"2024-03-27T10:55:40.079116+0000","last_clean":"2024-03-27T10:55:40.079116+0000","last_became_active":"2024-03-27T10:54:05.085354+0000","last_became_peered":"2024-03-27T10:54:05.085354+0000","last_unstale":"2024-03-27T10:55:40.079116+0000","last_undegraded":"2024-03-27T10:55:40.079116+0000","last_fullsized":"2024-03-27T10:55:40.079116+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:33:24.330313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"90'4","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.333606+0000","last_change":"2024-03-27T10:54:04.061757+0000","last_active":"2024-03-27T10:54:41.333606+0000","last_peered":"2024-03-27T10:54:41.333606+0000","last_clean":"2024-03-27T10:54:41.333606+0000","last_became_active":"2024-03-27T10:54:04.059211+0000","last_became_peered":"2024-03-27T10:54:04.059211+0000","last_unstale":"2024-03-27T10:54:41.333606+0000","last_undegraded":"2024-03-27T10:54:41.333606+0000","last_fullsized":"2024-03-27T10:54:41.333606+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T11:20:41.391143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.045487+0000","last_change":"2024-03-27T10:54:05.090919+0000","last_active":"2024-03-27T10:55:40.045487+0000","last_peered":"2024-03-27T10:55:40.045487+0000","last_clean":"2024-03-27T10:55:40.045487+0000","last_became_active":"2024-03-27T10:54:05.087971+0000","last_became_peered":"2024-03-27T10:54:05.087971+0000","last_unstale":"2024-03-27T10:55:40.045487+0000","last_undegraded":"2024-03-27T10:55:40.045487+0000","last_fullsized":"2024-03-27T10:55:40.045487+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:00:45.195957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"90'5","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.334856+0000","last_change":"2024-03-27T10:54:04.064828+0000","last_active":"2024-03-27T10:54:41.334856+0000","last_peered":"2024-03-27T10:54:41.334856+0000","last_clean":"2024-03-27T10:54:41.334856+0000","last_became_active":"2024-03-27T10:54:04.064359+0000","last_became_peered":"2024-03-27T10:54:04.064359+0000","last_unstale":"2024-03-27T10:54:41.334856+0000","last_undegraded":"2024-03-27T10:54:41.334856+0000","last_fullsized":"2024-03-27T10:54:41.334856+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:05:19.905707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.081122+0000","last_change":"2024-03-27T10:54:41.337153+0000","last_active":"2024-03-27T10:55:40.081122+0000","last_peered":"2024-03-27T10:55:40.081122+0000","last_clean":"2024-03-27T10:55:40.081122+0000","last_became_active":"2024-03-27T10:54:41.336433+0000","last_became_peered":"2024-03-27T10:54:41.336433+0000","last_unstale":"2024-03-27T10:55:40.081122+0000","last_undegraded":"2024-03-27T10:55:40.081122+0000","last_fullsized":"2024-03-27T10:55:40.081122+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:00:25.944147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,7],"acting":[1,8,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"92'35","reported_seq":341,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:56:27.489303+0000","last_change":"2024-03-27T10:54:04.061913+0000","last_active":"2024-03-27T10:56:27.489303+0000","last_peered":"2024-03-27T10:56:27.489303+0000","last_clean":"2024-03-27T10:56:27.489303+0000","last_became_active":"2024-03-27T10:54:04.058366+0000","last_became_peered":"2024-03-27T10:54:04.058366+0000","last_unstale":"2024-03-27T10:56:27.489303+0000","last_undegraded":"2024-03-27T10:56:27.489303+0000","last_fullsized":"2024-03-27T10:56:27.489303+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:45:28.365554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61788,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":286,"num_read_kb":203,"num_write":37,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.061525+0000","last_change":"2024-03-27T10:54:05.090840+0000","last_active":"2024-03-27T10:55:40.061525+0000","last_peered":"2024-03-27T10:55:40.061525+0000","last_clean":"2024-03-27T10:55:40.061525+0000","last_became_active":"2024-03-27T10:54:05.090525+0000","last_became_peered":"2024-03-27T10:54:05.090525+0000","last_unstale":"2024-03-27T10:55:40.061525+0000","last_undegraded":"2024-03-27T10:55:40.061525+0000","last_fullsized":"2024-03-27T10:55:40.061525+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T21:26:10.559917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"90'9","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.336322+0000","last_change":"2024-03-27T10:54:04.061205+0000","last_active":"2024-03-27T10:54:41.336322+0000","last_peered":"2024-03-27T10:54:41.336322+0000","last_clean":"2024-03-27T10:54:41.336322+0000","last_became_active":"2024-03-27T10:54:04.059558+0000","last_became_peered":"2024-03-27T10:54:04.059558+0000","last_unstale":"2024-03-27T10:54:41.336322+0000","last_undegraded":"2024-03-27T10:54:41.336322+0000","last_fullsized":"2024-03-27T10:54:41.336322+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T12:00:00.182233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.068118+0000","last_change":"2024-03-27T10:54:05.090149+0000","last_active":"2024-03-27T10:55:40.068118+0000","last_peered":"2024-03-27T10:55:40.068118+0000","last_clean":"2024-03-27T10:55:40.068118+0000","last_became_active":"2024-03-27T10:54:05.089317+0000","last_became_peered":"2024-03-27T10:54:05.089317+0000","last_unstale":"2024-03-27T10:55:40.068118+0000","last_undegraded":"2024-03-27T10:55:40.068118+0000","last_fullsized":"2024-03-27T10:55:40.068118+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T11:50:06.477472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"90'5","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.333600+0000","last_change":"2024-03-27T10:54:04.061797+0000","last_active":"2024-03-27T10:54:41.333600+0000","last_peered":"2024-03-27T10:54:41.333600+0000","last_clean":"2024-03-27T10:54:41.333600+0000","last_became_active":"2024-03-27T10:54:04.060974+0000","last_became_peered":"2024-03-27T10:54:04.060974+0000","last_unstale":"2024-03-27T10:54:41.333600+0000","last_undegraded":"2024-03-27T10:54:41.333600+0000","last_fullsized":"2024-03-27T10:54:41.333600+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T18:34:33.486126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.057613+0000","last_change":"2024-03-27T10:54:41.370832+0000","last_active":"2024-03-27T10:55:40.057613+0000","last_peered":"2024-03-27T10:55:40.057613+0000","last_clean":"2024-03-27T10:55:40.057613+0000","last_became_active":"2024-03-27T10:54:41.370229+0000","last_became_peered":"2024-03-27T10:54:41.370229+0000","last_unstale":"2024-03-27T10:55:40.057613+0000","last_undegraded":"2024-03-27T10:55:40.057613+0000","last_fullsized":"2024-03-27T10:55:40.057613+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:53:19.887501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"92'8","reported_seq":171,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:56:27.517918+0000","last_change":"2024-03-27T10:54:04.060042+0000","last_active":"2024-03-27T10:56:27.517918+0000","last_peered":"2024-03-27T10:56:27.517918+0000","last_clean":"2024-03-27T10:56:27.517918+0000","last_became_active":"2024-03-27T10:54:04.059415+0000","last_became_peered":"2024-03-27T10:54:04.059415+0000","last_unstale":"2024-03-27T10:56:27.517918+0000","last_undegraded":"2024-03-27T10:56:27.517918+0000","last_fullsized":"2024-03-27T10:56:27.517918+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:52:06.846942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1613,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":144,"num_read_kb":6,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.066681+0000","last_change":"2024-03-27T10:54:05.100088+0000","last_active":"2024-03-27T10:55:40.066681+0000","last_peered":"2024-03-27T10:55:40.066681+0000","last_clean":"2024-03-27T10:55:40.066681+0000","last_became_active":"2024-03-27T10:54:05.098709+0000","last_became_peered":"2024-03-27T10:54:05.098709+0000","last_unstale":"2024-03-27T10:55:40.066681+0000","last_undegraded":"2024-03-27T10:55:40.066681+0000","last_fullsized":"2024-03-27T10:55:40.066681+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T21:25:13.329710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"90'17","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.327516+0000","last_change":"2024-03-27T10:54:04.066812+0000","last_active":"2024-03-27T10:54:41.327516+0000","last_peered":"2024-03-27T10:54:41.327516+0000","last_clean":"2024-03-27T10:54:41.327516+0000","last_became_active":"2024-03-27T10:54:04.066364+0000","last_became_peered":"2024-03-27T10:54:04.066364+0000","last_unstale":"2024-03-27T10:54:41.327516+0000","last_undegraded":"2024-03-27T10:54:41.327516+0000","last_fullsized":"2024-03-27T10:54:41.327516+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T19:31:51.616785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.074821+0000","last_change":"2024-03-27T10:54:05.089367+0000","last_active":"2024-03-27T10:55:40.074821+0000","last_peered":"2024-03-27T10:55:40.074821+0000","last_clean":"2024-03-27T10:55:40.074821+0000","last_became_active":"2024-03-27T10:54:05.088718+0000","last_became_peered":"2024-03-27T10:54:05.088718+0000","last_unstale":"2024-03-27T10:55:40.074821+0000","last_undegraded":"2024-03-27T10:55:40.074821+0000","last_fullsized":"2024-03-27T10:55:40.074821+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:56:08.768859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"92'3","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:56:18.706651+0000","last_change":"2024-03-27T10:54:04.059951+0000","last_active":"2024-03-27T10:56:18.706651+0000","last_peered":"2024-03-27T10:56:18.706651+0000","last_clean":"2024-03-27T10:56:18.706651+0000","last_became_active":"2024-03-27T10:54:04.057245+0000","last_became_peered":"2024-03-27T10:54:04.057245+0000","last_unstale":"2024-03-27T10:56:18.706651+0000","last_undegraded":"2024-03-27T10:56:18.706651+0000","last_fullsized":"2024-03-27T10:56:18.706651+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T19:36:48.881920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.051789+0000","last_change":"2024-03-27T10:54:05.089004+0000","last_active":"2024-03-27T10:55:40.051789+0000","last_peered":"2024-03-27T10:55:40.051789+0000","last_clean":"2024-03-27T10:55:40.051789+0000","last_became_active":"2024-03-27T10:54:05.083764+0000","last_became_peered":"2024-03-27T10:54:05.083764+0000","last_unstale":"2024-03-27T10:55:40.051789+0000","last_undegraded":"2024-03-27T10:55:40.051789+0000","last_fullsized":"2024-03-27T10:55:40.051789+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:09:53.883490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"90'1","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.326444+0000","last_change":"2024-03-27T10:54:04.050508+0000","last_active":"2024-03-27T10:54:41.326444+0000","last_peered":"2024-03-27T10:54:41.326444+0000","last_clean":"2024-03-27T10:54:41.326444+0000","last_became_active":"2024-03-27T10:54:04.048877+0000","last_became_peered":"2024-03-27T10:54:04.048877+0000","last_unstale":"2024-03-27T10:54:41.326444+0000","last_undegraded":"2024-03-27T10:54:41.326444+0000","last_fullsized":"2024-03-27T10:54:41.326444+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:44:11.578684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"92'5","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.045703+0000","last_change":"2024-03-27T10:54:05.083931+0000","last_active":"2024-03-27T10:55:40.045703+0000","last_peered":"2024-03-27T10:55:40.045703+0000","last_clean":"2024-03-27T10:55:40.045703+0000","last_became_active":"2024-03-27T10:54:05.083544+0000","last_became_peered":"2024-03-27T10:54:05.083544+0000","last_unstale":"2024-03-27T10:55:40.045703+0000","last_undegraded":"2024-03-27T10:55:40.045703+0000","last_fullsized":"2024-03-27T10:55:40.045703+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T20:54:19.172355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"92'6","reported_seq":166,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:56:28.536227+0000","last_change":"2024-03-27T10:54:04.057428+0000","last_active":"2024-03-27T10:56:28.536227+0000","last_peered":"2024-03-27T10:56:28.536227+0000","last_clean":"2024-03-27T10:56:28.536227+0000","last_became_active":"2024-03-27T10:54:04.052966+0000","last_became_peered":"2024-03-27T10:54:04.052966+0000","last_unstale":"2024-03-27T10:56:28.536227+0000","last_undegraded":"2024-03-27T10:56:28.536227+0000","last_fullsized":"2024-03-27T10:56:28.536227+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T18:50:31.170411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1081,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":140,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.050606+0000","last_change":"2024-03-27T10:54:41.438829+0000","last_active":"2024-03-27T10:55:40.050606+0000","last_peered":"2024-03-27T10:55:40.050606+0000","last_clean":"2024-03-27T10:55:40.050606+0000","last_became_active":"2024-03-27T10:54:41.438031+0000","last_became_peered":"2024-03-27T10:54:41.438031+0000","last_unstale":"2024-03-27T10:55:40.050606+0000","last_undegraded":"2024-03-27T10:55:40.050606+0000","last_fullsized":"2024-03-27T10:55:40.050606+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:33:04.052747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.035927+0000","last_change":"2024-03-27T10:54:05.090255+0000","last_active":"2024-03-27T10:55:40.035927+0000","last_peered":"2024-03-27T10:55:40.035927+0000","last_clean":"2024-03-27T10:55:40.035927+0000","last_became_active":"2024-03-27T10:54:05.088833+0000","last_became_peered":"2024-03-27T10:54:05.088833+0000","last_unstale":"2024-03-27T10:55:40.035927+0000","last_undegraded":"2024-03-27T10:55:40.035927+0000","last_fullsized":"2024-03-27T10:55:40.035927+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:15:55.934698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"92'7","reported_seq":167,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:56:27.516617+0000","last_change":"2024-03-27T10:54:04.059941+0000","last_active":"2024-03-27T10:56:27.516617+0000","last_peered":"2024-03-27T10:56:27.516617+0000","last_clean":"2024-03-27T10:56:27.516617+0000","last_became_active":"2024-03-27T10:54:04.058197+0000","last_became_peered":"2024-03-27T10:54:04.058197+0000","last_unstale":"2024-03-27T10:56:27.516617+0000","last_undegraded":"2024-03-27T10:56:27.516617+0000","last_fullsized":"2024-03-27T10:56:27.516617+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:44:46.374054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":140,"num_read_kb":140,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.078902+0000","last_change":"2024-03-27T10:54:41.337917+0000","last_active":"2024-03-27T10:55:40.078902+0000","last_peered":"2024-03-27T10:55:40.078902+0000","last_clean":"2024-03-27T10:55:40.078902+0000","last_became_active":"2024-03-27T10:54:41.337624+0000","last_became_peered":"2024-03-27T10:54:41.337624+0000","last_unstale":"2024-03-27T10:55:40.078902+0000","last_undegraded":"2024-03-27T10:55:40.078902+0000","last_fullsized":"2024-03-27T10:55:40.078902+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T11:50:48.275234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"90'6","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.327556+0000","last_change":"2024-03-27T10:54:04.064017+0000","last_active":"2024-03-27T10:54:41.327556+0000","last_peered":"2024-03-27T10:54:41.327556+0000","last_clean":"2024-03-27T10:54:41.327556+0000","last_became_active":"2024-03-27T10:54:04.062881+0000","last_became_peered":"2024-03-27T10:54:04.062881+0000","last_unstale":"2024-03-27T10:54:41.327556+0000","last_undegraded":"2024-03-27T10:54:41.327556+0000","last_fullsized":"2024-03-27T10:54:41.327556+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:21:21.638623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":349,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.333171+0000","last_change":"2024-03-27T10:52:31.127495+0000","last_active":"2024-03-27T10:54:41.333171+0000","last_peered":"2024-03-27T10:54:41.333171+0000","last_clean":"2024-03-27T10:54:41.333171+0000","last_became_active":"2024-03-27T10:52:31.125418+0000","last_became_peered":"2024-03-27T10:52:31.125418+0000","last_unstale":"2024-03-27T10:54:41.333171+0000","last_undegraded":"2024-03-27T10:54:41.333171+0000","last_fullsized":"2024-03-27T10:54:41.333171+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:47:54.649615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:47:54.649615+0000","last_clean_scrub_stamp":"2024-03-27T10:47:54.649615+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T18:49:24.591580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.047107+0000","last_change":"2024-03-27T10:54:05.085558+0000","last_active":"2024-03-27T10:55:40.047107+0000","last_peered":"2024-03-27T10:55:40.047107+0000","last_clean":"2024-03-27T10:55:40.047107+0000","last_became_active":"2024-03-27T10:54:05.083896+0000","last_became_peered":"2024-03-27T10:54:05.083896+0000","last_unstale":"2024-03-27T10:55:40.047107+0000","last_undegraded":"2024-03-27T10:55:40.047107+0000","last_fullsized":"2024-03-27T10:55:40.047107+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:15:29.697609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'4","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.333395+0000","last_change":"2024-03-27T10:54:04.072448+0000","last_active":"2024-03-27T10:54:41.333395+0000","last_peered":"2024-03-27T10:54:41.333395+0000","last_clean":"2024-03-27T10:54:41.333395+0000","last_became_active":"2024-03-27T10:54:04.071933+0000","last_became_peered":"2024-03-27T10:54:04.071933+0000","last_unstale":"2024-03-27T10:54:41.333395+0000","last_undegraded":"2024-03-27T10:54:41.333395+0000","last_fullsized":"2024-03-27T10:54:41.333395+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:34:49.204345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"90'4","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.328806+0000","last_change":"2024-03-27T10:54:04.067308+0000","last_active":"2024-03-27T10:54:41.328806+0000","last_peered":"2024-03-27T10:54:41.328806+0000","last_clean":"2024-03-27T10:54:41.328806+0000","last_became_active":"2024-03-27T10:54:04.066927+0000","last_became_peered":"2024-03-27T10:54:04.066927+0000","last_unstale":"2024-03-27T10:54:41.328806+0000","last_undegraded":"2024-03-27T10:54:41.328806+0000","last_fullsized":"2024-03-27T10:54:41.328806+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:08:04.849245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.049057+0000","last_change":"2024-03-27T10:54:05.075154+0000","last_active":"2024-03-27T10:55:40.049057+0000","last_peered":"2024-03-27T10:55:40.049057+0000","last_clean":"2024-03-27T10:55:40.049057+0000","last_became_active":"2024-03-27T10:54:05.074772+0000","last_became_peered":"2024-03-27T10:54:05.074772+0000","last_unstale":"2024-03-27T10:55:40.049057+0000","last_undegraded":"2024-03-27T10:55:40.049057+0000","last_fullsized":"2024-03-27T10:55:40.049057+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T20:49:16.286229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"92'15","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:56:17.327925+0000","last_change":"2024-03-27T10:54:04.066805+0000","last_active":"2024-03-27T10:56:17.327925+0000","last_peered":"2024-03-27T10:56:17.327925+0000","last_clean":"2024-03-27T10:56:17.327925+0000","last_became_active":"2024-03-27T10:54:04.066500+0000","last_became_peered":"2024-03-27T10:54:04.066500+0000","last_unstale":"2024-03-27T10:56:17.327925+0000","last_undegraded":"2024-03-27T10:56:17.327925+0000","last_fullsized":"2024-03-27T10:56:17.327925+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T12:37:44.853575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.070237+0000","last_change":"2024-03-27T10:54:05.094314+0000","last_active":"2024-03-27T10:55:40.070237+0000","last_peered":"2024-03-27T10:55:40.070237+0000","last_clean":"2024-03-27T10:55:40.070237+0000","last_became_active":"2024-03-27T10:54:05.093896+0000","last_became_peered":"2024-03-27T10:54:05.093896+0000","last_unstale":"2024-03-27T10:55:40.070237+0000","last_undegraded":"2024-03-27T10:55:40.070237+0000","last_fullsized":"2024-03-27T10:55:40.070237+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:30:02.441317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"90'7","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.328737+0000","last_change":"2024-03-27T10:54:04.065525+0000","last_active":"2024-03-27T10:54:41.328737+0000","last_peered":"2024-03-27T10:54:41.328737+0000","last_clean":"2024-03-27T10:54:41.328737+0000","last_became_active":"2024-03-27T10:54:04.064979+0000","last_became_peered":"2024-03-27T10:54:04.064979+0000","last_unstale":"2024-03-27T10:54:41.328737+0000","last_undegraded":"2024-03-27T10:54:41.328737+0000","last_fullsized":"2024-03-27T10:54:41.328737+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:58:25.110861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.042948+0000","last_change":"2024-03-27T10:54:05.084038+0000","last_active":"2024-03-27T10:55:40.042948+0000","last_peered":"2024-03-27T10:55:40.042948+0000","last_clean":"2024-03-27T10:55:40.042948+0000","last_became_active":"2024-03-27T10:54:05.083733+0000","last_became_peered":"2024-03-27T10:54:05.083733+0000","last_unstale":"2024-03-27T10:55:40.042948+0000","last_undegraded":"2024-03-27T10:55:40.042948+0000","last_fullsized":"2024-03-27T10:55:40.042948+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T18:51:14.832602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.330550+0000","last_change":"2024-03-27T10:54:04.064506+0000","last_active":"2024-03-27T10:54:41.330550+0000","last_peered":"2024-03-27T10:54:41.330550+0000","last_clean":"2024-03-27T10:54:41.330550+0000","last_became_active":"2024-03-27T10:54:04.064178+0000","last_became_peered":"2024-03-27T10:54:04.064178+0000","last_unstale":"2024-03-27T10:54:41.330550+0000","last_undegraded":"2024-03-27T10:54:41.330550+0000","last_fullsized":"2024-03-27T10:54:41.330550+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:07:19.465782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.038119+0000","last_change":"2024-03-27T10:54:05.088241+0000","last_active":"2024-03-27T10:55:40.038119+0000","last_peered":"2024-03-27T10:55:40.038119+0000","last_clean":"2024-03-27T10:55:40.038119+0000","last_became_active":"2024-03-27T10:54:05.087852+0000","last_became_peered":"2024-03-27T10:54:05.087852+0000","last_unstale":"2024-03-27T10:55:40.038119+0000","last_undegraded":"2024-03-27T10:55:40.038119+0000","last_fullsized":"2024-03-27T10:55:40.038119+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:47:09.633654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"90'7","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.336541+0000","last_change":"2024-03-27T10:54:04.065744+0000","last_active":"2024-03-27T10:54:41.336541+0000","last_peered":"2024-03-27T10:54:41.336541+0000","last_clean":"2024-03-27T10:54:41.336541+0000","last_became_active":"2024-03-27T10:54:04.065155+0000","last_became_peered":"2024-03-27T10:54:04.065155+0000","last_unstale":"2024-03-27T10:54:41.336541+0000","last_undegraded":"2024-03-27T10:54:41.336541+0000","last_fullsized":"2024-03-27T10:54:41.336541+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T12:35:00.556458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.049948+0000","last_change":"2024-03-27T10:54:05.077954+0000","last_active":"2024-03-27T10:55:40.049948+0000","last_peered":"2024-03-27T10:55:40.049948+0000","last_clean":"2024-03-27T10:55:40.049948+0000","last_became_active":"2024-03-27T10:54:05.077589+0000","last_became_peered":"2024-03-27T10:54:05.077589+0000","last_unstale":"2024-03-27T10:55:40.049948+0000","last_undegraded":"2024-03-27T10:55:40.049948+0000","last_fullsized":"2024-03-27T10:55:40.049948+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T20:18:27.411887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"92'4","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.574937+0000","last_change":"2024-03-27T10:54:41.352576+0000","last_active":"2024-03-27T10:54:41.574937+0000","last_peered":"2024-03-27T10:54:41.574937+0000","last_clean":"2024-03-27T10:54:41.574937+0000","last_became_active":"2024-03-27T10:54:41.338775+0000","last_became_peered":"2024-03-27T10:54:41.338775+0000","last_unstale":"2024-03-27T10:54:41.574937+0000","last_undegraded":"2024-03-27T10:54:41.574937+0000","last_fullsized":"2024-03-27T10:54:41.574937+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T18:23:14.194920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.051269+0000","last_change":"2024-03-27T10:54:05.088250+0000","last_active":"2024-03-27T10:55:40.051269+0000","last_peered":"2024-03-27T10:55:40.051269+0000","last_clean":"2024-03-27T10:55:40.051269+0000","last_became_active":"2024-03-27T10:54:05.087942+0000","last_became_peered":"2024-03-27T10:54:05.087942+0000","last_unstale":"2024-03-27T10:55:40.051269+0000","last_undegraded":"2024-03-27T10:55:40.051269+0000","last_fullsized":"2024-03-27T10:55:40.051269+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:57:45.266347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.327767+0000","last_change":"2024-03-27T10:54:04.070513+0000","last_active":"2024-03-27T10:54:41.327767+0000","last_peered":"2024-03-27T10:54:41.327767+0000","last_clean":"2024-03-27T10:54:41.327767+0000","last_became_active":"2024-03-27T10:54:04.070042+0000","last_became_peered":"2024-03-27T10:54:04.070042+0000","last_unstale":"2024-03-27T10:54:41.327767+0000","last_undegraded":"2024-03-27T10:54:41.327767+0000","last_fullsized":"2024-03-27T10:54:41.327767+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:40:25.216899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.080525+0000","last_change":"2024-03-27T10:54:05.095067+0000","last_active":"2024-03-27T10:55:40.080525+0000","last_peered":"2024-03-27T10:55:40.080525+0000","last_clean":"2024-03-27T10:55:40.080525+0000","last_became_active":"2024-03-27T10:54:05.094618+0000","last_became_peered":"2024-03-27T10:54:05.094618+0000","last_unstale":"2024-03-27T10:55:40.080525+0000","last_undegraded":"2024-03-27T10:55:40.080525+0000","last_fullsized":"2024-03-27T10:55:40.080525+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:57:59.557310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"92'6","reported_seq":166,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:56:27.535118+0000","last_change":"2024-03-27T10:54:04.051352+0000","last_active":"2024-03-27T10:56:27.535118+0000","last_peered":"2024-03-27T10:56:27.535118+0000","last_clean":"2024-03-27T10:56:27.535118+0000","last_became_active":"2024-03-27T10:54:04.050850+0000","last_became_peered":"2024-03-27T10:54:04.050850+0000","last_unstale":"2024-03-27T10:56:27.535118+0000","last_undegraded":"2024-03-27T10:56:27.535118+0000","last_fullsized":"2024-03-27T10:56:27.535118+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:50:16.719451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1663,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":140,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.080038+0000","last_change":"2024-03-27T10:54:05.094650+0000","last_active":"2024-03-27T10:55:40.080038+0000","last_peered":"2024-03-27T10:55:40.080038+0000","last_clean":"2024-03-27T10:55:40.080038+0000","last_became_active":"2024-03-27T10:54:05.094168+0000","last_became_peered":"2024-03-27T10:54:05.094168+0000","last_unstale":"2024-03-27T10:55:40.080038+0000","last_undegraded":"2024-03-27T10:55:40.080038+0000","last_fullsized":"2024-03-27T10:55:40.080038+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:29:56.491733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"90'3","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.334649+0000","last_change":"2024-03-27T10:54:04.067755+0000","last_active":"2024-03-27T10:54:41.334649+0000","last_peered":"2024-03-27T10:54:41.334649+0000","last_clean":"2024-03-27T10:54:41.334649+0000","last_became_active":"2024-03-27T10:54:04.067320+0000","last_became_peered":"2024-03-27T10:54:04.067320+0000","last_unstale":"2024-03-27T10:54:41.334649+0000","last_undegraded":"2024-03-27T10:54:41.334649+0000","last_fullsized":"2024-03-27T10:54:41.334649+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T12:16:25.949829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"92'7","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.062408+0000","last_change":"2024-03-27T10:54:05.100715+0000","last_active":"2024-03-27T10:55:40.062408+0000","last_peered":"2024-03-27T10:55:40.062408+0000","last_clean":"2024-03-27T10:55:40.062408+0000","last_became_active":"2024-03-27T10:54:05.097861+0000","last_became_peered":"2024-03-27T10:54:05.097861+0000","last_unstale":"2024-03-27T10:55:40.062408+0000","last_undegraded":"2024-03-27T10:55:40.062408+0000","last_fullsized":"2024-03-27T10:55:40.062408+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:02:39.107232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"90'5","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.369747+0000","last_change":"2024-03-27T10:54:04.063894+0000","last_active":"2024-03-27T10:54:41.369747+0000","last_peered":"2024-03-27T10:54:41.369747+0000","last_clean":"2024-03-27T10:54:41.369747+0000","last_became_active":"2024-03-27T10:54:04.063163+0000","last_became_peered":"2024-03-27T10:54:04.063163+0000","last_unstale":"2024-03-27T10:54:41.369747+0000","last_undegraded":"2024-03-27T10:54:41.369747+0000","last_fullsized":"2024-03-27T10:54:41.369747+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:30:41.025752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.069291+0000","last_change":"2024-03-27T10:54:05.101980+0000","last_active":"2024-03-27T10:55:40.069291+0000","last_peered":"2024-03-27T10:55:40.069291+0000","last_clean":"2024-03-27T10:55:40.069291+0000","last_became_active":"2024-03-27T10:54:05.101448+0000","last_became_peered":"2024-03-27T10:54:05.101448+0000","last_unstale":"2024-03-27T10:55:40.069291+0000","last_undegraded":"2024-03-27T10:55:40.069291+0000","last_fullsized":"2024-03-27T10:55:40.069291+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:45:11.137181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.053342+0000","last_change":"2024-03-27T10:54:05.084026+0000","last_active":"2024-03-27T10:55:40.053342+0000","last_peered":"2024-03-27T10:55:40.053342+0000","last_clean":"2024-03-27T10:55:40.053342+0000","last_became_active":"2024-03-27T10:54:05.083410+0000","last_became_peered":"2024-03-27T10:54:05.083410+0000","last_unstale":"2024-03-27T10:55:40.053342+0000","last_undegraded":"2024-03-27T10:55:40.053342+0000","last_fullsized":"2024-03-27T10:55:40.053342+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T22:39:34.586363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.047447+0000","last_change":"2024-03-27T10:54:05.083592+0000","last_active":"2024-03-27T10:55:40.047447+0000","last_peered":"2024-03-27T10:55:40.047447+0000","last_clean":"2024-03-27T10:55:40.047447+0000","last_became_active":"2024-03-27T10:54:05.082537+0000","last_became_peered":"2024-03-27T10:54:05.082537+0000","last_unstale":"2024-03-27T10:55:40.047447+0000","last_undegraded":"2024-03-27T10:55:40.047447+0000","last_fullsized":"2024-03-27T10:55:40.047447+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T21:15:36.830805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.040301+0000","last_change":"2024-03-27T10:54:05.090226+0000","last_active":"2024-03-27T10:55:40.040301+0000","last_peered":"2024-03-27T10:55:40.040301+0000","last_clean":"2024-03-27T10:55:40.040301+0000","last_became_active":"2024-03-27T10:54:05.089831+0000","last_became_peered":"2024-03-27T10:54:05.089831+0000","last_unstale":"2024-03-27T10:55:40.040301+0000","last_undegraded":"2024-03-27T10:55:40.040301+0000","last_fullsized":"2024-03-27T10:55:40.040301+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T12:45:57.802266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.047183+0000","last_change":"2024-03-27T10:54:05.080196+0000","last_active":"2024-03-27T10:55:40.047183+0000","last_peered":"2024-03-27T10:55:40.047183+0000","last_clean":"2024-03-27T10:55:40.047183+0000","last_became_active":"2024-03-27T10:54:05.079822+0000","last_became_peered":"2024-03-27T10:54:05.079822+0000","last_unstale":"2024-03-27T10:55:40.047183+0000","last_undegraded":"2024-03-27T10:55:40.047183+0000","last_fullsized":"2024-03-27T10:55:40.047183+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:53:11.556503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.040513+0000","last_change":"2024-03-27T10:54:05.087264+0000","last_active":"2024-03-27T10:55:40.040513+0000","last_peered":"2024-03-27T10:55:40.040513+0000","last_clean":"2024-03-27T10:55:40.040513+0000","last_became_active":"2024-03-27T10:54:05.085440+0000","last_became_peered":"2024-03-27T10:54:05.085440+0000","last_unstale":"2024-03-27T10:55:40.040513+0000","last_undegraded":"2024-03-27T10:55:40.040513+0000","last_fullsized":"2024-03-27T10:55:40.040513+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T22:41:40.665643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.050820+0000","last_change":"2024-03-27T10:54:05.094306+0000","last_active":"2024-03-27T10:55:40.050820+0000","last_peered":"2024-03-27T10:55:40.050820+0000","last_clean":"2024-03-27T10:55:40.050820+0000","last_became_active":"2024-03-27T10:54:05.093858+0000","last_became_peered":"2024-03-27T10:54:05.093858+0000","last_unstale":"2024-03-27T10:55:40.050820+0000","last_undegraded":"2024-03-27T10:55:40.050820+0000","last_fullsized":"2024-03-27T10:55:40.050820+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T21:22:57.624833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.077439+0000","last_change":"2024-03-27T10:54:05.094089+0000","last_active":"2024-03-27T10:55:40.077439+0000","last_peered":"2024-03-27T10:55:40.077439+0000","last_clean":"2024-03-27T10:55:40.077439+0000","last_became_active":"2024-03-27T10:54:05.092317+0000","last_became_peered":"2024-03-27T10:54:05.092317+0000","last_unstale":"2024-03-27T10:55:40.077439+0000","last_undegraded":"2024-03-27T10:55:40.077439+0000","last_fullsized":"2024-03-27T10:55:40.077439+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:34:57.606250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.042440+0000","last_change":"2024-03-27T10:54:05.099995+0000","last_active":"2024-03-27T10:55:40.042440+0000","last_peered":"2024-03-27T10:55:40.042440+0000","last_clean":"2024-03-27T10:55:40.042440+0000","last_became_active":"2024-03-27T10:54:05.099641+0000","last_became_peered":"2024-03-27T10:54:05.099641+0000","last_unstale":"2024-03-27T10:55:40.042440+0000","last_undegraded":"2024-03-27T10:55:40.042440+0000","last_fullsized":"2024-03-27T10:55:40.042440+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:01:16.050180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.043273+0000","last_change":"2024-03-27T10:54:41.371472+0000","last_active":"2024-03-27T10:55:40.043273+0000","last_peered":"2024-03-27T10:55:40.043273+0000","last_clean":"2024-03-27T10:55:40.043273+0000","last_became_active":"2024-03-27T10:54:41.370912+0000","last_became_peered":"2024-03-27T10:54:41.370912+0000","last_unstale":"2024-03-27T10:55:40.043273+0000","last_undegraded":"2024-03-27T10:55:40.043273+0000","last_fullsized":"2024-03-27T10:55:40.043273+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T20:21:28.950573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6],"acting":[5,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.046487+0000","last_change":"2024-03-27T10:54:05.096812+0000","last_active":"2024-03-27T10:55:40.046487+0000","last_peered":"2024-03-27T10:55:40.046487+0000","last_clean":"2024-03-27T10:55:40.046487+0000","last_became_active":"2024-03-27T10:54:05.096323+0000","last_became_peered":"2024-03-27T10:54:05.096323+0000","last_unstale":"2024-03-27T10:55:40.046487+0000","last_undegraded":"2024-03-27T10:55:40.046487+0000","last_fullsized":"2024-03-27T10:55:40.046487+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:39:29.827291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.080404+0000","last_change":"2024-03-27T10:54:05.089137+0000","last_active":"2024-03-27T10:55:40.080404+0000","last_peered":"2024-03-27T10:55:40.080404+0000","last_clean":"2024-03-27T10:55:40.080404+0000","last_became_active":"2024-03-27T10:54:05.088795+0000","last_became_peered":"2024-03-27T10:54:05.088795+0000","last_unstale":"2024-03-27T10:55:40.080404+0000","last_undegraded":"2024-03-27T10:55:40.080404+0000","last_fullsized":"2024-03-27T10:55:40.080404+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T19:47:35.231749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.060769+0000","last_change":"2024-03-27T10:54:05.089683+0000","last_active":"2024-03-27T10:55:40.060769+0000","last_peered":"2024-03-27T10:55:40.060769+0000","last_clean":"2024-03-27T10:55:40.060769+0000","last_became_active":"2024-03-27T10:54:05.089235+0000","last_became_peered":"2024-03-27T10:54:05.089235+0000","last_unstale":"2024-03-27T10:55:40.060769+0000","last_undegraded":"2024-03-27T10:55:40.060769+0000","last_fullsized":"2024-03-27T10:55:40.060769+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:43:37.012814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.049661+0000","last_change":"2024-03-27T10:54:05.085043+0000","last_active":"2024-03-27T10:55:40.049661+0000","last_peered":"2024-03-27T10:55:40.049661+0000","last_clean":"2024-03-27T10:55:40.049661+0000","last_became_active":"2024-03-27T10:54:05.084695+0000","last_became_peered":"2024-03-27T10:54:05.084695+0000","last_unstale":"2024-03-27T10:55:40.049661+0000","last_undegraded":"2024-03-27T10:55:40.049661+0000","last_fullsized":"2024-03-27T10:55:40.049661+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T12:07:31.327006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":71246,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1424,"num_read_kb":788,"num_write":264,"num_write_kb":153,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":569344,"data_stored":218052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":228,"ondisk_log_size":228,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":85,"seq":365072220206,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":481518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":77,"seq":330712481845,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37000,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708152,"statfs":{"total":95995035648,"available":95957147648,"internally_reserved":0,"allocated":1703936,"data_stored":1134285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":71,"seq":304942678075,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1028096,"data_stored":480373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":63,"seq":270582939714,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36332,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708820,"statfs":{"total":95995035648,"available":95957831680,"internally_reserved":0,"allocated":1019904,"data_stored":480169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":54,"seq":231928234057,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28600,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716552,"statfs":{"total":95995035648,"available":95965749248,"internally_reserved":0,"allocated":1622016,"data_stored":1075421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":49,"seq":210453397583,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":480583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574962,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1077248,"data_stored":547598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738488,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":481922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411434,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1093632,"data_stored":544529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117093,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1028096,"data_stored":479915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":35,"seq":150323855454,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36324,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708828,"statfs":{"total":95995035648,"available":95957839872,"internally_reserved":0,"allocated":1011712,"data_stored":480976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":41,"seq":176093659223,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36932,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708220,"statfs":{"total":95995035648,"available":95957217280,"internally_reserved":0,"allocated":1634304,"data_stored":1073581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":68347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":65278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":64549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-27T10:56:32.870 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-27T10:56:33.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.130+0000 7f6128200640 1 -- 172.21.15.97:0/934226238 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61201025e0 msgr2=0x7f61201049d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:33.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.130+0000 7f6128200640 1 --2- 172.21.15.97:0/934226238 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61201025e0 0x7f61201049d0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f6114009900 tx=0x7f611402bc40 comp rx=0 tx=0).stop 2024-03-27T10:56:33.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.130+0000 7f6128200640 1 -- 172.21.15.97:0/934226238 shutdown_connections 2024-03-27T10:56:33.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.130+0000 7f6128200640 1 --2- 172.21.15.97:0/934226238 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6120107840 0x7f6120109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:33.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.130+0000 7f6128200640 1 --2- 172.21.15.97:0/934226238 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6120104f10 0x7f6120107300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:33.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.130+0000 7f6128200640 1 --2- 172.21.15.97:0/934226238 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61201025e0 0x7f61201049d0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:33.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.130+0000 7f6128200640 1 -- 172.21.15.97:0/934226238 >> 172.21.15.97:0/934226238 conn(0x7f61200fbfd0 msgr2=0x7f61200fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:33.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.130+0000 7f6128200640 1 -- 172.21.15.97:0/934226238 shutdown_connections 2024-03-27T10:56:33.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.131+0000 7f6128200640 1 -- 172.21.15.97:0/934226238 wait complete. 2024-03-27T10:56:33.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.131+0000 7f6128200640 1 Processor -- start 2024-03-27T10:56:33.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.132+0000 7f6128200640 1 -- start start 2024-03-27T10:56:33.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.132+0000 7f6128200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61201025e0 0x7f6120197dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:33.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.132+0000 7f6128200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6120104f10 0x7f6120198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:33.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.132+0000 7f6128200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6120107840 0x7f612019f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:33.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.132+0000 7f6128200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f612010d6c0 con 0x7f61201025e0 2024-03-27T10:56:33.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.132+0000 7f6128200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f612010d540 con 0x7f6120104f10 2024-03-27T10:56:33.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.132+0000 7f6128200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f612010d840 con 0x7f6120107840 2024-03-27T10:56:33.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.132+0000 7f6125800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61201025e0 0x7f6120197dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:33.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.132+0000 7f6125800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61201025e0 0x7f6120197dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53526/0 (socket says 172.21.15.97:53526) 2024-03-27T10:56:33.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.132+0000 7f6125800640 1 -- 172.21.15.97:0/4270420020 learned_addr learned my addr 172.21.15.97:0/4270420020 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:56:33.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.132+0000 7f6124e00640 1 --2- 172.21.15.97:0/4270420020 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6120104f10 0x7f6120198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:33.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.132+0000 7f6126200640 1 --2- 172.21.15.97:0/4270420020 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6120107840 0x7f612019f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:33.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.132+0000 7f6125800640 1 -- 172.21.15.97:0/4270420020 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6120107840 msgr2=0x7f612019f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:33.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.132+0000 7f6125800640 1 --2- 172.21.15.97:0/4270420020 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6120107840 0x7f612019f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:33.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.132+0000 7f6125800640 1 -- 172.21.15.97:0/4270420020 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6120104f10 msgr2=0x7f6120198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:33.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.133+0000 7f6125800640 1 --2- 172.21.15.97:0/4270420020 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6120104f10 0x7f6120198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:33.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.133+0000 7f6125800640 1 -- 172.21.15.97:0/4270420020 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f612019f9d0 con 0x7f61201025e0 2024-03-27T10:56:33.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.133+0000 7f6126200640 1 --2- 172.21.15.97:0/4270420020 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6120107840 0x7f612019f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:56:33.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.133+0000 7f6124e00640 1 --2- 172.21.15.97:0/4270420020 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6120104f10 0x7f6120198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T10:56:33.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.133+0000 7f6125800640 1 --2- 172.21.15.97:0/4270420020 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61201025e0 0x7f6120197dc0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f6114006010 tx=0x7f61140042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:33.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.133+0000 7f611e000640 1 -- 172.21.15.97:0/4270420020 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6114043020 con 0x7f61201025e0 2024-03-27T10:56:33.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.133+0000 7f6128200640 1 -- 172.21.15.97:0/4270420020 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f612019fc60 con 0x7f61201025e0 2024-03-27T10:56:33.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.133+0000 7f611e000640 1 -- 172.21.15.97:0/4270420020 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6114004aa0 con 0x7f61201025e0 2024-03-27T10:56:33.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.133+0000 7f611e000640 1 -- 172.21.15.97:0/4270420020 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6114003c60 con 0x7f61201025e0 2024-03-27T10:56:33.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.133+0000 7f6128200640 1 -- 172.21.15.97:0/4270420020 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61201a01b0 con 0x7f61201025e0 2024-03-27T10:56:33.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.135+0000 7f6128200640 1 -- 172.21.15.97:0/4270420020 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61200fdac0 con 0x7f61201025e0 2024-03-27T10:56:33.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.138+0000 7f611e000640 1 -- 172.21.15.97:0/4270420020 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f6114046020 con 0x7f61201025e0 2024-03-27T10:56:33.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.140+0000 7f611e000640 1 --2- 172.21.15.97:0/4270420020 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f61080a3690 0x7f61080a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:33.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.141+0000 7f611e000640 1 -- 172.21.15.97:0/4270420020 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f61140f2200 con 0x7f61201025e0 2024-03-27T10:56:33.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.141+0000 7f6124e00640 1 --2- 172.21.15.97:0/4270420020 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f61080a3690 0x7f61080a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:33.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.141+0000 7f6124e00640 1 --2- 172.21.15.97:0/4270420020 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f61080a3690 0x7f61080a5b50 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f610c0058c0 tx=0x7f610c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:33.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.145+0000 7f611e000640 1 -- 172.21.15.97:0/4270420020 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f611404d020 con 0x7f61201025e0 2024-03-27T10:56:33.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.359+0000 7f6128200640 1 -- 172.21.15.97:0/4270420020 --> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6120061f90 con 0x7f61080a3690 2024-03-27T10:56:33.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.375+0000 7f611e000640 1 -- 172.21.15.97:0/4270420020 <== mgr.14152 v2:172.21.15.97:6800/3955463538 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258915 (secure 0 0 0) 0x7f6120061f90 con 0x7f61080a3690 2024-03-27T10:56:33.350 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T10:56:33.351 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-03-27T10:56:33.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.380+0000 7f6128200640 1 -- 172.21.15.97:0/4270420020 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f61080a3690 msgr2=0x7f61080a5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:33.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.380+0000 7f6128200640 1 --2- 172.21.15.97:0/4270420020 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f61080a3690 0x7f61080a5b50 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f610c0058c0 tx=0x7f610c005850 comp rx=0 tx=0).stop 2024-03-27T10:56:33.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.380+0000 7f6128200640 1 -- 172.21.15.97:0/4270420020 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61201025e0 msgr2=0x7f6120197dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:33.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.380+0000 7f6128200640 1 --2- 172.21.15.97:0/4270420020 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61201025e0 0x7f6120197dc0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f6114006010 tx=0x7f61140042f0 comp rx=0 tx=0).stop 2024-03-27T10:56:33.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.380+0000 7f6128200640 1 -- 172.21.15.97:0/4270420020 shutdown_connections 2024-03-27T10:56:33.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.380+0000 7f6128200640 1 --2- 172.21.15.97:0/4270420020 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6120107840 0x7f612019f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:33.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.380+0000 7f6128200640 1 --2- 172.21.15.97:0/4270420020 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f61080a3690 0x7f61080a5b50 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:33.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.380+0000 7f6128200640 1 --2- 172.21.15.97:0/4270420020 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6120104f10 0x7f6120198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:33.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.380+0000 7f6128200640 1 --2- 172.21.15.97:0/4270420020 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61201025e0 0x7f6120197dc0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:33.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.380+0000 7f6128200640 1 -- 172.21.15.97:0/4270420020 >> 172.21.15.97:0/4270420020 conn(0x7f61200fbfd0 msgr2=0x7f6120102f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:33.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.381+0000 7f6128200640 1 -- 172.21.15.97:0/4270420020 shutdown_connections 2024-03-27T10:56:33.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.381+0000 7f6128200640 1 -- 172.21.15.97:0/4270420020 wait complete. 2024-03-27T10:56:33.371 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":438,"stamp":"2024-03-27T10:56:32.116811+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":661848,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1550,"num_read_kb":897,"num_write":447,"num_write_kb":3093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1181318,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":867,"ondisk_log_size":867,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":379728,"kb_used_data":13936,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124562096,"statfs":{"total":1151940427776,"available":1151551586304,"internally_reserved":0,"allocated":14270464,"data_stored":7740870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":374453561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2465,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":105,"num_read_kb":55,"num_write":16,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005541"},"pg_stats":[{"pgid":"3.3f","version":"92'8","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.077510+0000","last_change":"2024-03-27T10:54:05.077845+0000","last_active":"2024-03-27T10:55:40.077510+0000","last_peered":"2024-03-27T10:55:40.077510+0000","last_clean":"2024-03-27T10:55:40.077510+0000","last_became_active":"2024-03-27T10:54:05.077494+0000","last_became_peered":"2024-03-27T10:54:05.077494+0000","last_unstale":"2024-03-27T10:55:40.077510+0000","last_undegraded":"2024-03-27T10:55:40.077510+0000","last_fullsized":"2024-03-27T10:55:40.077510+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:16:07.094746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.048785+0000","last_change":"2024-03-27T10:54:05.078830+0000","last_active":"2024-03-27T10:55:40.048785+0000","last_peered":"2024-03-27T10:55:40.048785+0000","last_clean":"2024-03-27T10:55:40.048785+0000","last_became_active":"2024-03-27T10:54:05.078474+0000","last_became_peered":"2024-03-27T10:54:05.078474+0000","last_unstale":"2024-03-27T10:55:40.048785+0000","last_undegraded":"2024-03-27T10:55:40.048785+0000","last_fullsized":"2024-03-27T10:55:40.048785+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T20:11:09.900802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.075900+0000","last_change":"2024-03-27T10:54:05.082554+0000","last_active":"2024-03-27T10:55:40.075900+0000","last_peered":"2024-03-27T10:55:40.075900+0000","last_clean":"2024-03-27T10:55:40.075900+0000","last_became_active":"2024-03-27T10:54:05.081395+0000","last_became_peered":"2024-03-27T10:54:05.081395+0000","last_unstale":"2024-03-27T10:55:40.075900+0000","last_undegraded":"2024-03-27T10:55:40.075900+0000","last_fullsized":"2024-03-27T10:55:40.075900+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T20:02:58.866792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.033675+0000","last_change":"2024-03-27T10:54:05.086700+0000","last_active":"2024-03-27T10:55:40.033675+0000","last_peered":"2024-03-27T10:55:40.033675+0000","last_clean":"2024-03-27T10:55:40.033675+0000","last_became_active":"2024-03-27T10:54:05.086347+0000","last_became_peered":"2024-03-27T10:54:05.086347+0000","last_unstale":"2024-03-27T10:55:40.033675+0000","last_undegraded":"2024-03-27T10:55:40.033675+0000","last_fullsized":"2024-03-27T10:55:40.033675+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T22:03:11.336365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.047556+0000","last_change":"2024-03-27T10:54:05.083479+0000","last_active":"2024-03-27T10:55:40.047556+0000","last_peered":"2024-03-27T10:55:40.047556+0000","last_clean":"2024-03-27T10:55:40.047556+0000","last_became_active":"2024-03-27T10:54:05.083162+0000","last_became_peered":"2024-03-27T10:54:05.083162+0000","last_unstale":"2024-03-27T10:55:40.047556+0000","last_undegraded":"2024-03-27T10:55:40.047556+0000","last_fullsized":"2024-03-27T10:55:40.047556+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:23:42.265668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.050257+0000","last_change":"2024-03-27T10:54:05.079096+0000","last_active":"2024-03-27T10:55:40.050257+0000","last_peered":"2024-03-27T10:55:40.050257+0000","last_clean":"2024-03-27T10:55:40.050257+0000","last_became_active":"2024-03-27T10:54:05.078778+0000","last_became_peered":"2024-03-27T10:54:05.078778+0000","last_unstale":"2024-03-27T10:55:40.050257+0000","last_undegraded":"2024-03-27T10:55:40.050257+0000","last_fullsized":"2024-03-27T10:55:40.050257+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T19:26:47.534594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.053852+0000","last_change":"2024-03-27T10:54:05.098926+0000","last_active":"2024-03-27T10:55:40.053852+0000","last_peered":"2024-03-27T10:55:40.053852+0000","last_clean":"2024-03-27T10:55:40.053852+0000","last_became_active":"2024-03-27T10:54:05.098139+0000","last_became_peered":"2024-03-27T10:54:05.098139+0000","last_unstale":"2024-03-27T10:55:40.053852+0000","last_undegraded":"2024-03-27T10:55:40.053852+0000","last_fullsized":"2024-03-27T10:55:40.053852+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:35:14.335947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.050624+0000","last_change":"2024-03-27T10:54:05.093786+0000","last_active":"2024-03-27T10:55:40.050624+0000","last_peered":"2024-03-27T10:55:40.050624+0000","last_clean":"2024-03-27T10:55:40.050624+0000","last_became_active":"2024-03-27T10:54:05.092047+0000","last_became_peered":"2024-03-27T10:54:05.092047+0000","last_unstale":"2024-03-27T10:55:40.050624+0000","last_undegraded":"2024-03-27T10:55:40.050624+0000","last_fullsized":"2024-03-27T10:55:40.050624+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T22:50:30.778358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.073745+0000","last_change":"2024-03-27T10:54:05.081678+0000","last_active":"2024-03-27T10:55:40.073745+0000","last_peered":"2024-03-27T10:55:40.073745+0000","last_clean":"2024-03-27T10:55:40.073745+0000","last_became_active":"2024-03-27T10:54:05.080264+0000","last_became_peered":"2024-03-27T10:54:05.080264+0000","last_unstale":"2024-03-27T10:55:40.073745+0000","last_undegraded":"2024-03-27T10:55:40.073745+0000","last_fullsized":"2024-03-27T10:55:40.073745+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T22:41:04.183282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.063980+0000","last_change":"2024-03-27T10:54:05.095172+0000","last_active":"2024-03-27T10:55:40.063980+0000","last_peered":"2024-03-27T10:55:40.063980+0000","last_clean":"2024-03-27T10:55:40.063980+0000","last_became_active":"2024-03-27T10:54:05.088438+0000","last_became_peered":"2024-03-27T10:54:05.088438+0000","last_unstale":"2024-03-27T10:55:40.063980+0000","last_undegraded":"2024-03-27T10:55:40.063980+0000","last_fullsized":"2024-03-27T10:55:40.063980+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T11:19:50.422788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.076689+0000","last_change":"2024-03-27T10:54:05.093599+0000","last_active":"2024-03-27T10:55:40.076689+0000","last_peered":"2024-03-27T10:55:40.076689+0000","last_clean":"2024-03-27T10:55:40.076689+0000","last_became_active":"2024-03-27T10:54:05.092248+0000","last_became_peered":"2024-03-27T10:54:05.092248+0000","last_unstale":"2024-03-27T10:55:40.076689+0000","last_undegraded":"2024-03-27T10:55:40.076689+0000","last_fullsized":"2024-03-27T10:55:40.076689+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:22:38.712442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.039555+0000","last_change":"2024-03-27T10:54:05.094677+0000","last_active":"2024-03-27T10:55:40.039555+0000","last_peered":"2024-03-27T10:55:40.039555+0000","last_clean":"2024-03-27T10:55:40.039555+0000","last_became_active":"2024-03-27T10:54:05.094326+0000","last_became_peered":"2024-03-27T10:54:05.094326+0000","last_unstale":"2024-03-27T10:55:40.039555+0000","last_undegraded":"2024-03-27T10:55:40.039555+0000","last_fullsized":"2024-03-27T10:55:40.039555+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T21:44:51.923881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.052563+0000","last_change":"2024-03-27T10:54:05.083178+0000","last_active":"2024-03-27T10:55:40.052563+0000","last_peered":"2024-03-27T10:55:40.052563+0000","last_clean":"2024-03-27T10:55:40.052563+0000","last_became_active":"2024-03-27T10:54:05.082805+0000","last_became_peered":"2024-03-27T10:54:05.082805+0000","last_unstale":"2024-03-27T10:55:40.052563+0000","last_undegraded":"2024-03-27T10:55:40.052563+0000","last_fullsized":"2024-03-27T10:55:40.052563+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T19:03:07.558006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.040999+0000","last_change":"2024-03-27T10:54:05.098092+0000","last_active":"2024-03-27T10:55:40.040999+0000","last_peered":"2024-03-27T10:55:40.040999+0000","last_clean":"2024-03-27T10:55:40.040999+0000","last_became_active":"2024-03-27T10:54:05.097759+0000","last_became_peered":"2024-03-27T10:54:05.097759+0000","last_unstale":"2024-03-27T10:55:40.040999+0000","last_undegraded":"2024-03-27T10:55:40.040999+0000","last_fullsized":"2024-03-27T10:55:40.040999+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:14:07.884759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.033846+0000","last_change":"2024-03-27T10:54:05.086089+0000","last_active":"2024-03-27T10:55:40.033846+0000","last_peered":"2024-03-27T10:55:40.033846+0000","last_clean":"2024-03-27T10:55:40.033846+0000","last_became_active":"2024-03-27T10:54:05.084830+0000","last_became_peered":"2024-03-27T10:54:05.084830+0000","last_unstale":"2024-03-27T10:55:40.033846+0000","last_undegraded":"2024-03-27T10:55:40.033846+0000","last_fullsized":"2024-03-27T10:55:40.033846+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T20:37:44.390102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.080920+0000","last_change":"2024-03-27T10:54:05.092190+0000","last_active":"2024-03-27T10:55:40.080920+0000","last_peered":"2024-03-27T10:55:40.080920+0000","last_clean":"2024-03-27T10:55:40.080920+0000","last_became_active":"2024-03-27T10:54:05.091827+0000","last_became_peered":"2024-03-27T10:54:05.091827+0000","last_unstale":"2024-03-27T10:55:40.080920+0000","last_undegraded":"2024-03-27T10:55:40.080920+0000","last_fullsized":"2024-03-27T10:55:40.080920+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T21:12:13.452829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.077720+0000","last_change":"2024-03-27T10:54:05.086924+0000","last_active":"2024-03-27T10:55:40.077720+0000","last_peered":"2024-03-27T10:55:40.077720+0000","last_clean":"2024-03-27T10:55:40.077720+0000","last_became_active":"2024-03-27T10:54:05.085701+0000","last_became_peered":"2024-03-27T10:54:05.085701+0000","last_unstale":"2024-03-27T10:55:40.077720+0000","last_undegraded":"2024-03-27T10:55:40.077720+0000","last_fullsized":"2024-03-27T10:55:40.077720+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:43:38.612288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.070593+0000","last_change":"2024-03-27T10:54:05.082532+0000","last_active":"2024-03-27T10:55:40.070593+0000","last_peered":"2024-03-27T10:55:40.070593+0000","last_clean":"2024-03-27T10:55:40.070593+0000","last_became_active":"2024-03-27T10:54:05.082200+0000","last_became_peered":"2024-03-27T10:54:05.082200+0000","last_unstale":"2024-03-27T10:55:40.070593+0000","last_undegraded":"2024-03-27T10:55:40.070593+0000","last_fullsized":"2024-03-27T10:55:40.070593+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T19:07:55.232728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.048428+0000","last_change":"2024-03-27T10:54:05.088204+0000","last_active":"2024-03-27T10:55:40.048428+0000","last_peered":"2024-03-27T10:55:40.048428+0000","last_clean":"2024-03-27T10:55:40.048428+0000","last_became_active":"2024-03-27T10:54:05.087705+0000","last_became_peered":"2024-03-27T10:54:05.087705+0000","last_unstale":"2024-03-27T10:55:40.048428+0000","last_undegraded":"2024-03-27T10:55:40.048428+0000","last_fullsized":"2024-03-27T10:55:40.048428+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T19:24:52.450205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"92'14","reported_seq":170,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:56:25.842018+0000","last_change":"2024-03-27T10:54:04.072182+0000","last_active":"2024-03-27T10:56:25.842018+0000","last_peered":"2024-03-27T10:56:25.842018+0000","last_clean":"2024-03-27T10:56:25.842018+0000","last_became_active":"2024-03-27T10:54:04.071827+0000","last_became_peered":"2024-03-27T10:54:04.071827+0000","last_unstale":"2024-03-27T10:56:25.842018+0000","last_undegraded":"2024-03-27T10:56:25.842018+0000","last_fullsized":"2024-03-27T10:56:25.842018+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T18:40:19.776684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":136,"num_read_kb":136,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.043065+0000","last_change":"2024-03-27T10:54:05.095772+0000","last_active":"2024-03-27T10:55:40.043065+0000","last_peered":"2024-03-27T10:55:40.043065+0000","last_clean":"2024-03-27T10:55:40.043065+0000","last_became_active":"2024-03-27T10:54:05.095385+0000","last_became_peered":"2024-03-27T10:54:05.095385+0000","last_unstale":"2024-03-27T10:55:40.043065+0000","last_undegraded":"2024-03-27T10:55:40.043065+0000","last_fullsized":"2024-03-27T10:55:40.043065+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T12:07:21.744097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.337196+0000","last_change":"2024-03-27T10:54:04.069600+0000","last_active":"2024-03-27T10:54:41.337196+0000","last_peered":"2024-03-27T10:54:41.337196+0000","last_clean":"2024-03-27T10:54:41.337196+0000","last_became_active":"2024-03-27T10:54:04.067770+0000","last_became_peered":"2024-03-27T10:54:04.067770+0000","last_unstale":"2024-03-27T10:54:41.337196+0000","last_undegraded":"2024-03-27T10:54:41.337196+0000","last_fullsized":"2024-03-27T10:54:41.337196+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T12:26:36.930945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.076797+0000","last_change":"2024-03-27T10:54:05.081779+0000","last_active":"2024-03-27T10:55:40.076797+0000","last_peered":"2024-03-27T10:55:40.076797+0000","last_clean":"2024-03-27T10:55:40.076797+0000","last_became_active":"2024-03-27T10:54:05.081492+0000","last_became_peered":"2024-03-27T10:54:05.081492+0000","last_unstale":"2024-03-27T10:55:40.076797+0000","last_undegraded":"2024-03-27T10:55:40.076797+0000","last_fullsized":"2024-03-27T10:55:40.076797+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T19:07:12.838048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"90'12","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.329099+0000","last_change":"2024-03-27T10:54:04.070550+0000","last_active":"2024-03-27T10:54:41.329099+0000","last_peered":"2024-03-27T10:54:41.329099+0000","last_clean":"2024-03-27T10:54:41.329099+0000","last_became_active":"2024-03-27T10:54:04.070010+0000","last_became_peered":"2024-03-27T10:54:04.070010+0000","last_unstale":"2024-03-27T10:54:41.329099+0000","last_undegraded":"2024-03-27T10:54:41.329099+0000","last_fullsized":"2024-03-27T10:54:41.329099+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:46:34.168771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.079329+0000","last_change":"2024-03-27T10:54:05.086372+0000","last_active":"2024-03-27T10:55:40.079329+0000","last_peered":"2024-03-27T10:55:40.079329+0000","last_clean":"2024-03-27T10:55:40.079329+0000","last_became_active":"2024-03-27T10:54:05.086013+0000","last_became_peered":"2024-03-27T10:54:05.086013+0000","last_unstale":"2024-03-27T10:55:40.079329+0000","last_undegraded":"2024-03-27T10:55:40.079329+0000","last_fullsized":"2024-03-27T10:55:40.079329+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:17:39.059071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"90'5","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.347107+0000","last_change":"2024-03-27T10:54:41.346475+0000","last_active":"2024-03-27T10:54:41.347107+0000","last_peered":"2024-03-27T10:54:41.347107+0000","last_clean":"2024-03-27T10:54:41.347107+0000","last_became_active":"2024-03-27T10:54:41.337431+0000","last_became_peered":"2024-03-27T10:54:41.337431+0000","last_unstale":"2024-03-27T10:54:41.347107+0000","last_undegraded":"2024-03-27T10:54:41.347107+0000","last_fullsized":"2024-03-27T10:54:41.347107+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T20:57:10.552047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.075638+0000","last_change":"2024-03-27T10:54:05.087580+0000","last_active":"2024-03-27T10:55:40.075638+0000","last_peered":"2024-03-27T10:55:40.075638+0000","last_clean":"2024-03-27T10:55:40.075638+0000","last_became_active":"2024-03-27T10:54:05.087128+0000","last_became_peered":"2024-03-27T10:54:05.087128+0000","last_unstale":"2024-03-27T10:55:40.075638+0000","last_undegraded":"2024-03-27T10:55:40.075638+0000","last_fullsized":"2024-03-27T10:55:40.075638+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T22:03:46.861258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"92'8","reported_seq":172,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:56:28.513097+0000","last_change":"2024-03-27T10:54:04.066312+0000","last_active":"2024-03-27T10:56:28.513097+0000","last_peered":"2024-03-27T10:56:28.513097+0000","last_clean":"2024-03-27T10:56:28.513097+0000","last_became_active":"2024-03-27T10:54:04.065909+0000","last_became_peered":"2024-03-27T10:54:04.065909+0000","last_unstale":"2024-03-27T10:56:28.513097+0000","last_undegraded":"2024-03-27T10:56:28.513097+0000","last_fullsized":"2024-03-27T10:56:28.513097+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:08:45.257808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1647,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":145,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.040498+0000","last_change":"2024-03-27T10:54:41.370991+0000","last_active":"2024-03-27T10:55:40.040498+0000","last_peered":"2024-03-27T10:55:40.040498+0000","last_clean":"2024-03-27T10:55:40.040498+0000","last_became_active":"2024-03-27T10:54:41.370596+0000","last_became_peered":"2024-03-27T10:54:41.370596+0000","last_unstale":"2024-03-27T10:55:40.040498+0000","last_undegraded":"2024-03-27T10:55:40.040498+0000","last_fullsized":"2024-03-27T10:55:40.040498+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:29:44.187503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"90'7","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.331143+0000","last_change":"2024-03-27T10:54:04.072451+0000","last_active":"2024-03-27T10:54:41.331143+0000","last_peered":"2024-03-27T10:54:41.331143+0000","last_clean":"2024-03-27T10:54:41.331143+0000","last_became_active":"2024-03-27T10:54:04.072042+0000","last_became_peered":"2024-03-27T10:54:04.072042+0000","last_unstale":"2024-03-27T10:54:41.331143+0000","last_undegraded":"2024-03-27T10:54:41.331143+0000","last_fullsized":"2024-03-27T10:54:41.331143+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T12:20:43.231073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.047114+0000","last_change":"2024-03-27T10:54:05.086055+0000","last_active":"2024-03-27T10:55:40.047114+0000","last_peered":"2024-03-27T10:55:40.047114+0000","last_clean":"2024-03-27T10:55:40.047114+0000","last_became_active":"2024-03-27T10:54:05.084429+0000","last_became_peered":"2024-03-27T10:54:05.084429+0000","last_unstale":"2024-03-27T10:55:40.047114+0000","last_undegraded":"2024-03-27T10:55:40.047114+0000","last_fullsized":"2024-03-27T10:55:40.047114+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T12:41:12.868245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"90'3","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.369391+0000","last_change":"2024-03-27T10:54:04.059076+0000","last_active":"2024-03-27T10:54:41.369391+0000","last_peered":"2024-03-27T10:54:41.369391+0000","last_clean":"2024-03-27T10:54:41.369391+0000","last_became_active":"2024-03-27T10:54:04.057614+0000","last_became_peered":"2024-03-27T10:54:04.057614+0000","last_unstale":"2024-03-27T10:54:41.369391+0000","last_undegraded":"2024-03-27T10:54:41.369391+0000","last_fullsized":"2024-03-27T10:54:41.369391+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T22:26:16.299840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.047600+0000","last_change":"2024-03-27T10:54:05.092955+0000","last_active":"2024-03-27T10:55:40.047600+0000","last_peered":"2024-03-27T10:55:40.047600+0000","last_clean":"2024-03-27T10:55:40.047600+0000","last_became_active":"2024-03-27T10:54:05.091867+0000","last_became_peered":"2024-03-27T10:54:05.091867+0000","last_unstale":"2024-03-27T10:55:40.047600+0000","last_undegraded":"2024-03-27T10:55:40.047600+0000","last_fullsized":"2024-03-27T10:55:40.047600+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T21:41:04.636480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"90'3","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.331834+0000","last_change":"2024-03-27T10:54:04.058833+0000","last_active":"2024-03-27T10:54:41.331834+0000","last_peered":"2024-03-27T10:54:41.331834+0000","last_clean":"2024-03-27T10:54:41.331834+0000","last_became_active":"2024-03-27T10:54:04.057750+0000","last_became_peered":"2024-03-27T10:54:04.057750+0000","last_unstale":"2024-03-27T10:54:41.331834+0000","last_undegraded":"2024-03-27T10:54:41.331834+0000","last_fullsized":"2024-03-27T10:54:41.331834+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:03:18.771094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.048019+0000","last_change":"2024-03-27T10:54:05.078933+0000","last_active":"2024-03-27T10:55:40.048019+0000","last_peered":"2024-03-27T10:55:40.048019+0000","last_clean":"2024-03-27T10:55:40.048019+0000","last_became_active":"2024-03-27T10:54:05.078544+0000","last_became_peered":"2024-03-27T10:54:05.078544+0000","last_unstale":"2024-03-27T10:55:40.048019+0000","last_undegraded":"2024-03-27T10:55:40.048019+0000","last_fullsized":"2024-03-27T10:55:40.048019+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T19:40:25.529775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"92'7","reported_seq":301,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:56:27.533739+0000","last_change":"2024-03-27T10:54:04.063974+0000","last_active":"2024-03-27T10:56:27.533739+0000","last_peered":"2024-03-27T10:56:27.533739+0000","last_clean":"2024-03-27T10:56:27.533739+0000","last_became_active":"2024-03-27T10:54:04.058171+0000","last_became_peered":"2024-03-27T10:54:04.058171+0000","last_unstale":"2024-03-27T10:56:27.533739+0000","last_undegraded":"2024-03-27T10:56:27.533739+0000","last_fullsized":"2024-03-27T10:56:27.533739+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T19:14:15.597889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":274,"num_read_kb":274,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.079116+0000","last_change":"2024-03-27T10:54:05.085727+0000","last_active":"2024-03-27T10:55:40.079116+0000","last_peered":"2024-03-27T10:55:40.079116+0000","last_clean":"2024-03-27T10:55:40.079116+0000","last_became_active":"2024-03-27T10:54:05.085354+0000","last_became_peered":"2024-03-27T10:54:05.085354+0000","last_unstale":"2024-03-27T10:55:40.079116+0000","last_undegraded":"2024-03-27T10:55:40.079116+0000","last_fullsized":"2024-03-27T10:55:40.079116+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:33:24.330313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"90'4","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.333606+0000","last_change":"2024-03-27T10:54:04.061757+0000","last_active":"2024-03-27T10:54:41.333606+0000","last_peered":"2024-03-27T10:54:41.333606+0000","last_clean":"2024-03-27T10:54:41.333606+0000","last_became_active":"2024-03-27T10:54:04.059211+0000","last_became_peered":"2024-03-27T10:54:04.059211+0000","last_unstale":"2024-03-27T10:54:41.333606+0000","last_undegraded":"2024-03-27T10:54:41.333606+0000","last_fullsized":"2024-03-27T10:54:41.333606+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T11:20:41.391143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.045487+0000","last_change":"2024-03-27T10:54:05.090919+0000","last_active":"2024-03-27T10:55:40.045487+0000","last_peered":"2024-03-27T10:55:40.045487+0000","last_clean":"2024-03-27T10:55:40.045487+0000","last_became_active":"2024-03-27T10:54:05.087971+0000","last_became_peered":"2024-03-27T10:54:05.087971+0000","last_unstale":"2024-03-27T10:55:40.045487+0000","last_undegraded":"2024-03-27T10:55:40.045487+0000","last_fullsized":"2024-03-27T10:55:40.045487+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:00:45.195957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"90'5","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.334856+0000","last_change":"2024-03-27T10:54:04.064828+0000","last_active":"2024-03-27T10:54:41.334856+0000","last_peered":"2024-03-27T10:54:41.334856+0000","last_clean":"2024-03-27T10:54:41.334856+0000","last_became_active":"2024-03-27T10:54:04.064359+0000","last_became_peered":"2024-03-27T10:54:04.064359+0000","last_unstale":"2024-03-27T10:54:41.334856+0000","last_undegraded":"2024-03-27T10:54:41.334856+0000","last_fullsized":"2024-03-27T10:54:41.334856+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:05:19.905707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.081122+0000","last_change":"2024-03-27T10:54:41.337153+0000","last_active":"2024-03-27T10:55:40.081122+0000","last_peered":"2024-03-27T10:55:40.081122+0000","last_clean":"2024-03-27T10:55:40.081122+0000","last_became_active":"2024-03-27T10:54:41.336433+0000","last_became_peered":"2024-03-27T10:54:41.336433+0000","last_unstale":"2024-03-27T10:55:40.081122+0000","last_undegraded":"2024-03-27T10:55:40.081122+0000","last_fullsized":"2024-03-27T10:55:40.081122+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:00:25.944147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,7],"acting":[1,8,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"92'35","reported_seq":341,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:56:27.489303+0000","last_change":"2024-03-27T10:54:04.061913+0000","last_active":"2024-03-27T10:56:27.489303+0000","last_peered":"2024-03-27T10:56:27.489303+0000","last_clean":"2024-03-27T10:56:27.489303+0000","last_became_active":"2024-03-27T10:54:04.058366+0000","last_became_peered":"2024-03-27T10:54:04.058366+0000","last_unstale":"2024-03-27T10:56:27.489303+0000","last_undegraded":"2024-03-27T10:56:27.489303+0000","last_fullsized":"2024-03-27T10:56:27.489303+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:45:28.365554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61788,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":286,"num_read_kb":203,"num_write":37,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.061525+0000","last_change":"2024-03-27T10:54:05.090840+0000","last_active":"2024-03-27T10:55:40.061525+0000","last_peered":"2024-03-27T10:55:40.061525+0000","last_clean":"2024-03-27T10:55:40.061525+0000","last_became_active":"2024-03-27T10:54:05.090525+0000","last_became_peered":"2024-03-27T10:54:05.090525+0000","last_unstale":"2024-03-27T10:55:40.061525+0000","last_undegraded":"2024-03-27T10:55:40.061525+0000","last_fullsized":"2024-03-27T10:55:40.061525+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T21:26:10.559917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"90'9","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.336322+0000","last_change":"2024-03-27T10:54:04.061205+0000","last_active":"2024-03-27T10:54:41.336322+0000","last_peered":"2024-03-27T10:54:41.336322+0000","last_clean":"2024-03-27T10:54:41.336322+0000","last_became_active":"2024-03-27T10:54:04.059558+0000","last_became_peered":"2024-03-27T10:54:04.059558+0000","last_unstale":"2024-03-27T10:54:41.336322+0000","last_undegraded":"2024-03-27T10:54:41.336322+0000","last_fullsized":"2024-03-27T10:54:41.336322+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T12:00:00.182233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.068118+0000","last_change":"2024-03-27T10:54:05.090149+0000","last_active":"2024-03-27T10:55:40.068118+0000","last_peered":"2024-03-27T10:55:40.068118+0000","last_clean":"2024-03-27T10:55:40.068118+0000","last_became_active":"2024-03-27T10:54:05.089317+0000","last_became_peered":"2024-03-27T10:54:05.089317+0000","last_unstale":"2024-03-27T10:55:40.068118+0000","last_undegraded":"2024-03-27T10:55:40.068118+0000","last_fullsized":"2024-03-27T10:55:40.068118+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T11:50:06.477472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"90'5","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.333600+0000","last_change":"2024-03-27T10:54:04.061797+0000","last_active":"2024-03-27T10:54:41.333600+0000","last_peered":"2024-03-27T10:54:41.333600+0000","last_clean":"2024-03-27T10:54:41.333600+0000","last_became_active":"2024-03-27T10:54:04.060974+0000","last_became_peered":"2024-03-27T10:54:04.060974+0000","last_unstale":"2024-03-27T10:54:41.333600+0000","last_undegraded":"2024-03-27T10:54:41.333600+0000","last_fullsized":"2024-03-27T10:54:41.333600+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T18:34:33.486126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.057613+0000","last_change":"2024-03-27T10:54:41.370832+0000","last_active":"2024-03-27T10:55:40.057613+0000","last_peered":"2024-03-27T10:55:40.057613+0000","last_clean":"2024-03-27T10:55:40.057613+0000","last_became_active":"2024-03-27T10:54:41.370229+0000","last_became_peered":"2024-03-27T10:54:41.370229+0000","last_unstale":"2024-03-27T10:55:40.057613+0000","last_undegraded":"2024-03-27T10:55:40.057613+0000","last_fullsized":"2024-03-27T10:55:40.057613+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:53:19.887501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"92'8","reported_seq":171,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:56:27.517918+0000","last_change":"2024-03-27T10:54:04.060042+0000","last_active":"2024-03-27T10:56:27.517918+0000","last_peered":"2024-03-27T10:56:27.517918+0000","last_clean":"2024-03-27T10:56:27.517918+0000","last_became_active":"2024-03-27T10:54:04.059415+0000","last_became_peered":"2024-03-27T10:54:04.059415+0000","last_unstale":"2024-03-27T10:56:27.517918+0000","last_undegraded":"2024-03-27T10:56:27.517918+0000","last_fullsized":"2024-03-27T10:56:27.517918+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:52:06.846942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1613,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":144,"num_read_kb":6,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.066681+0000","last_change":"2024-03-27T10:54:05.100088+0000","last_active":"2024-03-27T10:55:40.066681+0000","last_peered":"2024-03-27T10:55:40.066681+0000","last_clean":"2024-03-27T10:55:40.066681+0000","last_became_active":"2024-03-27T10:54:05.098709+0000","last_became_peered":"2024-03-27T10:54:05.098709+0000","last_unstale":"2024-03-27T10:55:40.066681+0000","last_undegraded":"2024-03-27T10:55:40.066681+0000","last_fullsized":"2024-03-27T10:55:40.066681+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T21:25:13.329710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"90'17","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.327516+0000","last_change":"2024-03-27T10:54:04.066812+0000","last_active":"2024-03-27T10:54:41.327516+0000","last_peered":"2024-03-27T10:54:41.327516+0000","last_clean":"2024-03-27T10:54:41.327516+0000","last_became_active":"2024-03-27T10:54:04.066364+0000","last_became_peered":"2024-03-27T10:54:04.066364+0000","last_unstale":"2024-03-27T10:54:41.327516+0000","last_undegraded":"2024-03-27T10:54:41.327516+0000","last_fullsized":"2024-03-27T10:54:41.327516+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T19:31:51.616785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.074821+0000","last_change":"2024-03-27T10:54:05.089367+0000","last_active":"2024-03-27T10:55:40.074821+0000","last_peered":"2024-03-27T10:55:40.074821+0000","last_clean":"2024-03-27T10:55:40.074821+0000","last_became_active":"2024-03-27T10:54:05.088718+0000","last_became_peered":"2024-03-27T10:54:05.088718+0000","last_unstale":"2024-03-27T10:55:40.074821+0000","last_undegraded":"2024-03-27T10:55:40.074821+0000","last_fullsized":"2024-03-27T10:55:40.074821+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:56:08.768859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"92'3","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:56:18.706651+0000","last_change":"2024-03-27T10:54:04.059951+0000","last_active":"2024-03-27T10:56:18.706651+0000","last_peered":"2024-03-27T10:56:18.706651+0000","last_clean":"2024-03-27T10:56:18.706651+0000","last_became_active":"2024-03-27T10:54:04.057245+0000","last_became_peered":"2024-03-27T10:54:04.057245+0000","last_unstale":"2024-03-27T10:56:18.706651+0000","last_undegraded":"2024-03-27T10:56:18.706651+0000","last_fullsized":"2024-03-27T10:56:18.706651+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T19:36:48.881920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.051789+0000","last_change":"2024-03-27T10:54:05.089004+0000","last_active":"2024-03-27T10:55:40.051789+0000","last_peered":"2024-03-27T10:55:40.051789+0000","last_clean":"2024-03-27T10:55:40.051789+0000","last_became_active":"2024-03-27T10:54:05.083764+0000","last_became_peered":"2024-03-27T10:54:05.083764+0000","last_unstale":"2024-03-27T10:55:40.051789+0000","last_undegraded":"2024-03-27T10:55:40.051789+0000","last_fullsized":"2024-03-27T10:55:40.051789+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:09:53.883490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"90'1","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.326444+0000","last_change":"2024-03-27T10:54:04.050508+0000","last_active":"2024-03-27T10:54:41.326444+0000","last_peered":"2024-03-27T10:54:41.326444+0000","last_clean":"2024-03-27T10:54:41.326444+0000","last_became_active":"2024-03-27T10:54:04.048877+0000","last_became_peered":"2024-03-27T10:54:04.048877+0000","last_unstale":"2024-03-27T10:54:41.326444+0000","last_undegraded":"2024-03-27T10:54:41.326444+0000","last_fullsized":"2024-03-27T10:54:41.326444+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:44:11.578684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"92'5","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.045703+0000","last_change":"2024-03-27T10:54:05.083931+0000","last_active":"2024-03-27T10:55:40.045703+0000","last_peered":"2024-03-27T10:55:40.045703+0000","last_clean":"2024-03-27T10:55:40.045703+0000","last_became_active":"2024-03-27T10:54:05.083544+0000","last_became_peered":"2024-03-27T10:54:05.083544+0000","last_unstale":"2024-03-27T10:55:40.045703+0000","last_undegraded":"2024-03-27T10:55:40.045703+0000","last_fullsized":"2024-03-27T10:55:40.045703+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T20:54:19.172355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"92'6","reported_seq":166,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:56:28.536227+0000","last_change":"2024-03-27T10:54:04.057428+0000","last_active":"2024-03-27T10:56:28.536227+0000","last_peered":"2024-03-27T10:56:28.536227+0000","last_clean":"2024-03-27T10:56:28.536227+0000","last_became_active":"2024-03-27T10:54:04.052966+0000","last_became_peered":"2024-03-27T10:54:04.052966+0000","last_unstale":"2024-03-27T10:56:28.536227+0000","last_undegraded":"2024-03-27T10:56:28.536227+0000","last_fullsized":"2024-03-27T10:56:28.536227+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T18:50:31.170411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1081,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":140,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.050606+0000","last_change":"2024-03-27T10:54:41.438829+0000","last_active":"2024-03-27T10:55:40.050606+0000","last_peered":"2024-03-27T10:55:40.050606+0000","last_clean":"2024-03-27T10:55:40.050606+0000","last_became_active":"2024-03-27T10:54:41.438031+0000","last_became_peered":"2024-03-27T10:54:41.438031+0000","last_unstale":"2024-03-27T10:55:40.050606+0000","last_undegraded":"2024-03-27T10:55:40.050606+0000","last_fullsized":"2024-03-27T10:55:40.050606+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:33:04.052747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.035927+0000","last_change":"2024-03-27T10:54:05.090255+0000","last_active":"2024-03-27T10:55:40.035927+0000","last_peered":"2024-03-27T10:55:40.035927+0000","last_clean":"2024-03-27T10:55:40.035927+0000","last_became_active":"2024-03-27T10:54:05.088833+0000","last_became_peered":"2024-03-27T10:54:05.088833+0000","last_unstale":"2024-03-27T10:55:40.035927+0000","last_undegraded":"2024-03-27T10:55:40.035927+0000","last_fullsized":"2024-03-27T10:55:40.035927+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:15:55.934698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"92'7","reported_seq":167,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:56:27.516617+0000","last_change":"2024-03-27T10:54:04.059941+0000","last_active":"2024-03-27T10:56:27.516617+0000","last_peered":"2024-03-27T10:56:27.516617+0000","last_clean":"2024-03-27T10:56:27.516617+0000","last_became_active":"2024-03-27T10:54:04.058197+0000","last_became_peered":"2024-03-27T10:54:04.058197+0000","last_unstale":"2024-03-27T10:56:27.516617+0000","last_undegraded":"2024-03-27T10:56:27.516617+0000","last_fullsized":"2024-03-27T10:56:27.516617+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:44:46.374054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":140,"num_read_kb":140,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.078902+0000","last_change":"2024-03-27T10:54:41.337917+0000","last_active":"2024-03-27T10:55:40.078902+0000","last_peered":"2024-03-27T10:55:40.078902+0000","last_clean":"2024-03-27T10:55:40.078902+0000","last_became_active":"2024-03-27T10:54:41.337624+0000","last_became_peered":"2024-03-27T10:54:41.337624+0000","last_unstale":"2024-03-27T10:55:40.078902+0000","last_undegraded":"2024-03-27T10:55:40.078902+0000","last_fullsized":"2024-03-27T10:55:40.078902+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T11:50:48.275234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"90'6","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.327556+0000","last_change":"2024-03-27T10:54:04.064017+0000","last_active":"2024-03-27T10:54:41.327556+0000","last_peered":"2024-03-27T10:54:41.327556+0000","last_clean":"2024-03-27T10:54:41.327556+0000","last_became_active":"2024-03-27T10:54:04.062881+0000","last_became_peered":"2024-03-27T10:54:04.062881+0000","last_unstale":"2024-03-27T10:54:41.327556+0000","last_undegraded":"2024-03-27T10:54:41.327556+0000","last_fullsized":"2024-03-27T10:54:41.327556+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:21:21.638623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":349,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.333171+0000","last_change":"2024-03-27T10:52:31.127495+0000","last_active":"2024-03-27T10:54:41.333171+0000","last_peered":"2024-03-27T10:54:41.333171+0000","last_clean":"2024-03-27T10:54:41.333171+0000","last_became_active":"2024-03-27T10:52:31.125418+0000","last_became_peered":"2024-03-27T10:52:31.125418+0000","last_unstale":"2024-03-27T10:54:41.333171+0000","last_undegraded":"2024-03-27T10:54:41.333171+0000","last_fullsized":"2024-03-27T10:54:41.333171+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:47:54.649615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:47:54.649615+0000","last_clean_scrub_stamp":"2024-03-27T10:47:54.649615+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T18:49:24.591580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.047107+0000","last_change":"2024-03-27T10:54:05.085558+0000","last_active":"2024-03-27T10:55:40.047107+0000","last_peered":"2024-03-27T10:55:40.047107+0000","last_clean":"2024-03-27T10:55:40.047107+0000","last_became_active":"2024-03-27T10:54:05.083896+0000","last_became_peered":"2024-03-27T10:54:05.083896+0000","last_unstale":"2024-03-27T10:55:40.047107+0000","last_undegraded":"2024-03-27T10:55:40.047107+0000","last_fullsized":"2024-03-27T10:55:40.047107+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:15:29.697609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'4","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.333395+0000","last_change":"2024-03-27T10:54:04.072448+0000","last_active":"2024-03-27T10:54:41.333395+0000","last_peered":"2024-03-27T10:54:41.333395+0000","last_clean":"2024-03-27T10:54:41.333395+0000","last_became_active":"2024-03-27T10:54:04.071933+0000","last_became_peered":"2024-03-27T10:54:04.071933+0000","last_unstale":"2024-03-27T10:54:41.333395+0000","last_undegraded":"2024-03-27T10:54:41.333395+0000","last_fullsized":"2024-03-27T10:54:41.333395+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:34:49.204345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"90'4","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.328806+0000","last_change":"2024-03-27T10:54:04.067308+0000","last_active":"2024-03-27T10:54:41.328806+0000","last_peered":"2024-03-27T10:54:41.328806+0000","last_clean":"2024-03-27T10:54:41.328806+0000","last_became_active":"2024-03-27T10:54:04.066927+0000","last_became_peered":"2024-03-27T10:54:04.066927+0000","last_unstale":"2024-03-27T10:54:41.328806+0000","last_undegraded":"2024-03-27T10:54:41.328806+0000","last_fullsized":"2024-03-27T10:54:41.328806+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:08:04.849245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.049057+0000","last_change":"2024-03-27T10:54:05.075154+0000","last_active":"2024-03-27T10:55:40.049057+0000","last_peered":"2024-03-27T10:55:40.049057+0000","last_clean":"2024-03-27T10:55:40.049057+0000","last_became_active":"2024-03-27T10:54:05.074772+0000","last_became_peered":"2024-03-27T10:54:05.074772+0000","last_unstale":"2024-03-27T10:55:40.049057+0000","last_undegraded":"2024-03-27T10:55:40.049057+0000","last_fullsized":"2024-03-27T10:55:40.049057+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T20:49:16.286229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"92'15","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:56:17.327925+0000","last_change":"2024-03-27T10:54:04.066805+0000","last_active":"2024-03-27T10:56:17.327925+0000","last_peered":"2024-03-27T10:56:17.327925+0000","last_clean":"2024-03-27T10:56:17.327925+0000","last_became_active":"2024-03-27T10:54:04.066500+0000","last_became_peered":"2024-03-27T10:54:04.066500+0000","last_unstale":"2024-03-27T10:56:17.327925+0000","last_undegraded":"2024-03-27T10:56:17.327925+0000","last_fullsized":"2024-03-27T10:56:17.327925+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T12:37:44.853575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.070237+0000","last_change":"2024-03-27T10:54:05.094314+0000","last_active":"2024-03-27T10:55:40.070237+0000","last_peered":"2024-03-27T10:55:40.070237+0000","last_clean":"2024-03-27T10:55:40.070237+0000","last_became_active":"2024-03-27T10:54:05.093896+0000","last_became_peered":"2024-03-27T10:54:05.093896+0000","last_unstale":"2024-03-27T10:55:40.070237+0000","last_undegraded":"2024-03-27T10:55:40.070237+0000","last_fullsized":"2024-03-27T10:55:40.070237+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:30:02.441317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"90'7","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.328737+0000","last_change":"2024-03-27T10:54:04.065525+0000","last_active":"2024-03-27T10:54:41.328737+0000","last_peered":"2024-03-27T10:54:41.328737+0000","last_clean":"2024-03-27T10:54:41.328737+0000","last_became_active":"2024-03-27T10:54:04.064979+0000","last_became_peered":"2024-03-27T10:54:04.064979+0000","last_unstale":"2024-03-27T10:54:41.328737+0000","last_undegraded":"2024-03-27T10:54:41.328737+0000","last_fullsized":"2024-03-27T10:54:41.328737+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:58:25.110861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.042948+0000","last_change":"2024-03-27T10:54:05.084038+0000","last_active":"2024-03-27T10:55:40.042948+0000","last_peered":"2024-03-27T10:55:40.042948+0000","last_clean":"2024-03-27T10:55:40.042948+0000","last_became_active":"2024-03-27T10:54:05.083733+0000","last_became_peered":"2024-03-27T10:54:05.083733+0000","last_unstale":"2024-03-27T10:55:40.042948+0000","last_undegraded":"2024-03-27T10:55:40.042948+0000","last_fullsized":"2024-03-27T10:55:40.042948+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T18:51:14.832602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.330550+0000","last_change":"2024-03-27T10:54:04.064506+0000","last_active":"2024-03-27T10:54:41.330550+0000","last_peered":"2024-03-27T10:54:41.330550+0000","last_clean":"2024-03-27T10:54:41.330550+0000","last_became_active":"2024-03-27T10:54:04.064178+0000","last_became_peered":"2024-03-27T10:54:04.064178+0000","last_unstale":"2024-03-27T10:54:41.330550+0000","last_undegraded":"2024-03-27T10:54:41.330550+0000","last_fullsized":"2024-03-27T10:54:41.330550+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:07:19.465782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.038119+0000","last_change":"2024-03-27T10:54:05.088241+0000","last_active":"2024-03-27T10:55:40.038119+0000","last_peered":"2024-03-27T10:55:40.038119+0000","last_clean":"2024-03-27T10:55:40.038119+0000","last_became_active":"2024-03-27T10:54:05.087852+0000","last_became_peered":"2024-03-27T10:54:05.087852+0000","last_unstale":"2024-03-27T10:55:40.038119+0000","last_undegraded":"2024-03-27T10:55:40.038119+0000","last_fullsized":"2024-03-27T10:55:40.038119+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:47:09.633654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"90'7","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.336541+0000","last_change":"2024-03-27T10:54:04.065744+0000","last_active":"2024-03-27T10:54:41.336541+0000","last_peered":"2024-03-27T10:54:41.336541+0000","last_clean":"2024-03-27T10:54:41.336541+0000","last_became_active":"2024-03-27T10:54:04.065155+0000","last_became_peered":"2024-03-27T10:54:04.065155+0000","last_unstale":"2024-03-27T10:54:41.336541+0000","last_undegraded":"2024-03-27T10:54:41.336541+0000","last_fullsized":"2024-03-27T10:54:41.336541+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T12:35:00.556458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.049948+0000","last_change":"2024-03-27T10:54:05.077954+0000","last_active":"2024-03-27T10:55:40.049948+0000","last_peered":"2024-03-27T10:55:40.049948+0000","last_clean":"2024-03-27T10:55:40.049948+0000","last_became_active":"2024-03-27T10:54:05.077589+0000","last_became_peered":"2024-03-27T10:54:05.077589+0000","last_unstale":"2024-03-27T10:55:40.049948+0000","last_undegraded":"2024-03-27T10:55:40.049948+0000","last_fullsized":"2024-03-27T10:55:40.049948+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T20:18:27.411887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"92'4","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.574937+0000","last_change":"2024-03-27T10:54:41.352576+0000","last_active":"2024-03-27T10:54:41.574937+0000","last_peered":"2024-03-27T10:54:41.574937+0000","last_clean":"2024-03-27T10:54:41.574937+0000","last_became_active":"2024-03-27T10:54:41.338775+0000","last_became_peered":"2024-03-27T10:54:41.338775+0000","last_unstale":"2024-03-27T10:54:41.574937+0000","last_undegraded":"2024-03-27T10:54:41.574937+0000","last_fullsized":"2024-03-27T10:54:41.574937+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T18:23:14.194920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.051269+0000","last_change":"2024-03-27T10:54:05.088250+0000","last_active":"2024-03-27T10:55:40.051269+0000","last_peered":"2024-03-27T10:55:40.051269+0000","last_clean":"2024-03-27T10:55:40.051269+0000","last_became_active":"2024-03-27T10:54:05.087942+0000","last_became_peered":"2024-03-27T10:54:05.087942+0000","last_unstale":"2024-03-27T10:55:40.051269+0000","last_undegraded":"2024-03-27T10:55:40.051269+0000","last_fullsized":"2024-03-27T10:55:40.051269+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:57:45.266347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.327767+0000","last_change":"2024-03-27T10:54:04.070513+0000","last_active":"2024-03-27T10:54:41.327767+0000","last_peered":"2024-03-27T10:54:41.327767+0000","last_clean":"2024-03-27T10:54:41.327767+0000","last_became_active":"2024-03-27T10:54:04.070042+0000","last_became_peered":"2024-03-27T10:54:04.070042+0000","last_unstale":"2024-03-27T10:54:41.327767+0000","last_undegraded":"2024-03-27T10:54:41.327767+0000","last_fullsized":"2024-03-27T10:54:41.327767+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:40:25.216899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.080525+0000","last_change":"2024-03-27T10:54:05.095067+0000","last_active":"2024-03-27T10:55:40.080525+0000","last_peered":"2024-03-27T10:55:40.080525+0000","last_clean":"2024-03-27T10:55:40.080525+0000","last_became_active":"2024-03-27T10:54:05.094618+0000","last_became_peered":"2024-03-27T10:54:05.094618+0000","last_unstale":"2024-03-27T10:55:40.080525+0000","last_undegraded":"2024-03-27T10:55:40.080525+0000","last_fullsized":"2024-03-27T10:55:40.080525+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:57:59.557310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"92'6","reported_seq":166,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:56:27.535118+0000","last_change":"2024-03-27T10:54:04.051352+0000","last_active":"2024-03-27T10:56:27.535118+0000","last_peered":"2024-03-27T10:56:27.535118+0000","last_clean":"2024-03-27T10:56:27.535118+0000","last_became_active":"2024-03-27T10:54:04.050850+0000","last_became_peered":"2024-03-27T10:54:04.050850+0000","last_unstale":"2024-03-27T10:56:27.535118+0000","last_undegraded":"2024-03-27T10:56:27.535118+0000","last_fullsized":"2024-03-27T10:56:27.535118+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:50:16.719451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1663,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":140,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.080038+0000","last_change":"2024-03-27T10:54:05.094650+0000","last_active":"2024-03-27T10:55:40.080038+0000","last_peered":"2024-03-27T10:55:40.080038+0000","last_clean":"2024-03-27T10:55:40.080038+0000","last_became_active":"2024-03-27T10:54:05.094168+0000","last_became_peered":"2024-03-27T10:54:05.094168+0000","last_unstale":"2024-03-27T10:55:40.080038+0000","last_undegraded":"2024-03-27T10:55:40.080038+0000","last_fullsized":"2024-03-27T10:55:40.080038+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:29:56.491733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"90'3","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.334649+0000","last_change":"2024-03-27T10:54:04.067755+0000","last_active":"2024-03-27T10:54:41.334649+0000","last_peered":"2024-03-27T10:54:41.334649+0000","last_clean":"2024-03-27T10:54:41.334649+0000","last_became_active":"2024-03-27T10:54:04.067320+0000","last_became_peered":"2024-03-27T10:54:04.067320+0000","last_unstale":"2024-03-27T10:54:41.334649+0000","last_undegraded":"2024-03-27T10:54:41.334649+0000","last_fullsized":"2024-03-27T10:54:41.334649+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T12:16:25.949829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"92'7","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.062408+0000","last_change":"2024-03-27T10:54:05.100715+0000","last_active":"2024-03-27T10:55:40.062408+0000","last_peered":"2024-03-27T10:55:40.062408+0000","last_clean":"2024-03-27T10:55:40.062408+0000","last_became_active":"2024-03-27T10:54:05.097861+0000","last_became_peered":"2024-03-27T10:54:05.097861+0000","last_unstale":"2024-03-27T10:55:40.062408+0000","last_undegraded":"2024-03-27T10:55:40.062408+0000","last_fullsized":"2024-03-27T10:55:40.062408+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:02:39.107232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"90'5","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.369747+0000","last_change":"2024-03-27T10:54:04.063894+0000","last_active":"2024-03-27T10:54:41.369747+0000","last_peered":"2024-03-27T10:54:41.369747+0000","last_clean":"2024-03-27T10:54:41.369747+0000","last_became_active":"2024-03-27T10:54:04.063163+0000","last_became_peered":"2024-03-27T10:54:04.063163+0000","last_unstale":"2024-03-27T10:54:41.369747+0000","last_undegraded":"2024-03-27T10:54:41.369747+0000","last_fullsized":"2024-03-27T10:54:41.369747+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:30:41.025752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.069291+0000","last_change":"2024-03-27T10:54:05.101980+0000","last_active":"2024-03-27T10:55:40.069291+0000","last_peered":"2024-03-27T10:55:40.069291+0000","last_clean":"2024-03-27T10:55:40.069291+0000","last_became_active":"2024-03-27T10:54:05.101448+0000","last_became_peered":"2024-03-27T10:54:05.101448+0000","last_unstale":"2024-03-27T10:55:40.069291+0000","last_undegraded":"2024-03-27T10:55:40.069291+0000","last_fullsized":"2024-03-27T10:55:40.069291+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:45:11.137181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.053342+0000","last_change":"2024-03-27T10:54:05.084026+0000","last_active":"2024-03-27T10:55:40.053342+0000","last_peered":"2024-03-27T10:55:40.053342+0000","last_clean":"2024-03-27T10:55:40.053342+0000","last_became_active":"2024-03-27T10:54:05.083410+0000","last_became_peered":"2024-03-27T10:54:05.083410+0000","last_unstale":"2024-03-27T10:55:40.053342+0000","last_undegraded":"2024-03-27T10:55:40.053342+0000","last_fullsized":"2024-03-27T10:55:40.053342+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T22:39:34.586363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.047447+0000","last_change":"2024-03-27T10:54:05.083592+0000","last_active":"2024-03-27T10:55:40.047447+0000","last_peered":"2024-03-27T10:55:40.047447+0000","last_clean":"2024-03-27T10:55:40.047447+0000","last_became_active":"2024-03-27T10:54:05.082537+0000","last_became_peered":"2024-03-27T10:54:05.082537+0000","last_unstale":"2024-03-27T10:55:40.047447+0000","last_undegraded":"2024-03-27T10:55:40.047447+0000","last_fullsized":"2024-03-27T10:55:40.047447+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T21:15:36.830805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.040301+0000","last_change":"2024-03-27T10:54:05.090226+0000","last_active":"2024-03-27T10:55:40.040301+0000","last_peered":"2024-03-27T10:55:40.040301+0000","last_clean":"2024-03-27T10:55:40.040301+0000","last_became_active":"2024-03-27T10:54:05.089831+0000","last_became_peered":"2024-03-27T10:54:05.089831+0000","last_unstale":"2024-03-27T10:55:40.040301+0000","last_undegraded":"2024-03-27T10:55:40.040301+0000","last_fullsized":"2024-03-27T10:55:40.040301+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T12:45:57.802266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.047183+0000","last_change":"2024-03-27T10:54:05.080196+0000","last_active":"2024-03-27T10:55:40.047183+0000","last_peered":"2024-03-27T10:55:40.047183+0000","last_clean":"2024-03-27T10:55:40.047183+0000","last_became_active":"2024-03-27T10:54:05.079822+0000","last_became_peered":"2024-03-27T10:54:05.079822+0000","last_unstale":"2024-03-27T10:55:40.047183+0000","last_undegraded":"2024-03-27T10:55:40.047183+0000","last_fullsized":"2024-03-27T10:55:40.047183+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:53:11.556503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.040513+0000","last_change":"2024-03-27T10:54:05.087264+0000","last_active":"2024-03-27T10:55:40.040513+0000","last_peered":"2024-03-27T10:55:40.040513+0000","last_clean":"2024-03-27T10:55:40.040513+0000","last_became_active":"2024-03-27T10:54:05.085440+0000","last_became_peered":"2024-03-27T10:54:05.085440+0000","last_unstale":"2024-03-27T10:55:40.040513+0000","last_undegraded":"2024-03-27T10:55:40.040513+0000","last_fullsized":"2024-03-27T10:55:40.040513+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T22:41:40.665643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.050820+0000","last_change":"2024-03-27T10:54:05.094306+0000","last_active":"2024-03-27T10:55:40.050820+0000","last_peered":"2024-03-27T10:55:40.050820+0000","last_clean":"2024-03-27T10:55:40.050820+0000","last_became_active":"2024-03-27T10:54:05.093858+0000","last_became_peered":"2024-03-27T10:54:05.093858+0000","last_unstale":"2024-03-27T10:55:40.050820+0000","last_undegraded":"2024-03-27T10:55:40.050820+0000","last_fullsized":"2024-03-27T10:55:40.050820+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T21:22:57.624833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.077439+0000","last_change":"2024-03-27T10:54:05.094089+0000","last_active":"2024-03-27T10:55:40.077439+0000","last_peered":"2024-03-27T10:55:40.077439+0000","last_clean":"2024-03-27T10:55:40.077439+0000","last_became_active":"2024-03-27T10:54:05.092317+0000","last_became_peered":"2024-03-27T10:54:05.092317+0000","last_unstale":"2024-03-27T10:55:40.077439+0000","last_undegraded":"2024-03-27T10:55:40.077439+0000","last_fullsized":"2024-03-27T10:55:40.077439+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:34:57.606250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.042440+0000","last_change":"2024-03-27T10:54:05.099995+0000","last_active":"2024-03-27T10:55:40.042440+0000","last_peered":"2024-03-27T10:55:40.042440+0000","last_clean":"2024-03-27T10:55:40.042440+0000","last_became_active":"2024-03-27T10:54:05.099641+0000","last_became_peered":"2024-03-27T10:54:05.099641+0000","last_unstale":"2024-03-27T10:55:40.042440+0000","last_undegraded":"2024-03-27T10:55:40.042440+0000","last_fullsized":"2024-03-27T10:55:40.042440+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:01:16.050180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.043273+0000","last_change":"2024-03-27T10:54:41.371472+0000","last_active":"2024-03-27T10:55:40.043273+0000","last_peered":"2024-03-27T10:55:40.043273+0000","last_clean":"2024-03-27T10:55:40.043273+0000","last_became_active":"2024-03-27T10:54:41.370912+0000","last_became_peered":"2024-03-27T10:54:41.370912+0000","last_unstale":"2024-03-27T10:55:40.043273+0000","last_undegraded":"2024-03-27T10:55:40.043273+0000","last_fullsized":"2024-03-27T10:55:40.043273+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T20:21:28.950573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6],"acting":[5,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.046487+0000","last_change":"2024-03-27T10:54:05.096812+0000","last_active":"2024-03-27T10:55:40.046487+0000","last_peered":"2024-03-27T10:55:40.046487+0000","last_clean":"2024-03-27T10:55:40.046487+0000","last_became_active":"2024-03-27T10:54:05.096323+0000","last_became_peered":"2024-03-27T10:54:05.096323+0000","last_unstale":"2024-03-27T10:55:40.046487+0000","last_undegraded":"2024-03-27T10:55:40.046487+0000","last_fullsized":"2024-03-27T10:55:40.046487+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:39:29.827291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.080404+0000","last_change":"2024-03-27T10:54:05.089137+0000","last_active":"2024-03-27T10:55:40.080404+0000","last_peered":"2024-03-27T10:55:40.080404+0000","last_clean":"2024-03-27T10:55:40.080404+0000","last_became_active":"2024-03-27T10:54:05.088795+0000","last_became_peered":"2024-03-27T10:54:05.088795+0000","last_unstale":"2024-03-27T10:55:40.080404+0000","last_undegraded":"2024-03-27T10:55:40.080404+0000","last_fullsized":"2024-03-27T10:55:40.080404+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T19:47:35.231749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.060769+0000","last_change":"2024-03-27T10:54:05.089683+0000","last_active":"2024-03-27T10:55:40.060769+0000","last_peered":"2024-03-27T10:55:40.060769+0000","last_clean":"2024-03-27T10:55:40.060769+0000","last_became_active":"2024-03-27T10:54:05.089235+0000","last_became_peered":"2024-03-27T10:54:05.089235+0000","last_unstale":"2024-03-27T10:55:40.060769+0000","last_undegraded":"2024-03-27T10:55:40.060769+0000","last_fullsized":"2024-03-27T10:55:40.060769+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:43:37.012814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.049661+0000","last_change":"2024-03-27T10:54:05.085043+0000","last_active":"2024-03-27T10:55:40.049661+0000","last_peered":"2024-03-27T10:55:40.049661+0000","last_clean":"2024-03-27T10:55:40.049661+0000","last_became_active":"2024-03-27T10:54:05.084695+0000","last_became_peered":"2024-03-27T10:54:05.084695+0000","last_unstale":"2024-03-27T10:55:40.049661+0000","last_undegraded":"2024-03-27T10:55:40.049661+0000","last_fullsized":"2024-03-27T10:55:40.049661+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T12:07:31.327006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":71246,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1424,"num_read_kb":788,"num_write":264,"num_write_kb":153,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":569344,"data_stored":218052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":228,"ondisk_log_size":228,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":85,"seq":365072220206,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":481518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":77,"seq":330712481845,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37000,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708152,"statfs":{"total":95995035648,"available":95957147648,"internally_reserved":0,"allocated":1703936,"data_stored":1134285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":71,"seq":304942678075,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1028096,"data_stored":480373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":63,"seq":270582939714,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36332,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708820,"statfs":{"total":95995035648,"available":95957831680,"internally_reserved":0,"allocated":1019904,"data_stored":480169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":54,"seq":231928234057,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28600,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716552,"statfs":{"total":95995035648,"available":95965749248,"internally_reserved":0,"allocated":1622016,"data_stored":1075421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":49,"seq":210453397583,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":480583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574962,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1077248,"data_stored":547598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738488,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":481922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411434,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1093632,"data_stored":544529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117093,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1028096,"data_stored":479915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":35,"seq":150323855454,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36324,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708828,"statfs":{"total":95995035648,"available":95957839872,"internally_reserved":0,"allocated":1011712,"data_stored":480976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":41,"seq":176093659223,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36932,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708220,"statfs":{"total":95995035648,"available":95957217280,"internally_reserved":0,"allocated":1634304,"data_stored":1073581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":68347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":65278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":64549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-27T10:56:33.375 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-03-27T10:56:33.376 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-27T10:56:33.565 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:56:33 smithi097 ceph-mon[31859]: pgmap v438: 97 pgs: 97 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-03-27T10:56:33.565 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:56:33 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:56:33.565 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:56:33 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/1532575121' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T10:56:33.565 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:56:33 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:56:33.565 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:56:33 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:56:33.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.634+0000 7f0145200640 1 -- 172.21.15.97:0/2186251643 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f01400780f0 msgr2=0x7f0140078990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:33.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.634+0000 7f0145200640 1 --2- 172.21.15.97:0/2186251643 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f01400780f0 0x7f0140078990 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f0138009ca0 tx=0x7f013802c0f0 comp rx=0 tx=0).stop 2024-03-27T10:56:33.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.634+0000 7f0145200640 1 -- 172.21.15.97:0/2186251643 shutdown_connections 2024-03-27T10:56:33.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.634+0000 7f0145200640 1 --2- 172.21.15.97:0/2186251643 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f01400780f0 0x7f0140078990 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:33.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.634+0000 7f0145200640 1 --2- 172.21.15.97:0/2186251643 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0140077750 0x7f0140077bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:33.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.634+0000 7f0145200640 1 --2- 172.21.15.97:0/2186251643 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0140076510 0x7f0140076930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:33.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.634+0000 7f0145200640 1 -- 172.21.15.97:0/2186251643 >> 172.21.15.97:0/2186251643 conn(0x7f01400ff300 msgr2=0x7f0140101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:33.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.634+0000 7f0145200640 1 -- 172.21.15.97:0/2186251643 shutdown_connections 2024-03-27T10:56:33.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.634+0000 7f0145200640 1 -- 172.21.15.97:0/2186251643 wait complete. 2024-03-27T10:56:33.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.635+0000 7f0145200640 1 Processor -- start 2024-03-27T10:56:33.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.635+0000 7f0145200640 1 -- start start 2024-03-27T10:56:33.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.635+0000 7f0145200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0140076510 0x7f014019e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:33.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.635+0000 7f0145200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0140077750 0x7f014019e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:33.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.635+0000 7f0145200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f01400780f0 0x7f01401a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:33.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.635+0000 7f0145200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0140113cd0 con 0x7f01400780f0 2024-03-27T10:56:33.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.635+0000 7f0145200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f0140113b50 con 0x7f0140076510 2024-03-27T10:56:33.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.635+0000 7f0145200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f0140113e50 con 0x7f0140077750 2024-03-27T10:56:33.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.636+0000 7f013e800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f01400780f0 0x7f01401a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:33.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.636+0000 7f013e800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f01400780f0 0x7f01401a58f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53546/0 (socket says 172.21.15.97:53546) 2024-03-27T10:56:33.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.636+0000 7f013e800640 1 -- 172.21.15.97:0/1138083120 learned_addr learned my addr 172.21.15.97:0/1138083120 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:56:33.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.636+0000 7f013de00640 1 --2- 172.21.15.97:0/1138083120 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0140076510 0x7f014019e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:33.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.636+0000 7f013e800640 1 -- 172.21.15.97:0/1138083120 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0140077750 msgr2=0x7f014019e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:33.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.636+0000 7f013d400640 1 --2- 172.21.15.97:0/1138083120 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0140077750 0x7f014019e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:33.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.636+0000 7f013e800640 1 --2- 172.21.15.97:0/1138083120 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0140077750 0x7f014019e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:33.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.636+0000 7f013e800640 1 -- 172.21.15.97:0/1138083120 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0140076510 msgr2=0x7f014019e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:33.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.636+0000 7f013e800640 1 --2- 172.21.15.97:0/1138083120 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0140076510 0x7f014019e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:33.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.636+0000 7f013e800640 1 -- 172.21.15.97:0/1138083120 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01401a5f60 con 0x7f01400780f0 2024-03-27T10:56:33.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.636+0000 7f013e800640 1 --2- 172.21.15.97:0/1138083120 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f01400780f0 0x7f01401a58f0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f013802c700 tx=0x7f01380328a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:33.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.636+0000 7f0136a00640 1 -- 172.21.15.97:0/1138083120 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0138041020 con 0x7f01400780f0 2024-03-27T10:56:33.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.636+0000 7f0136a00640 1 -- 172.21.15.97:0/1138083120 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0138003e20 con 0x7f01400780f0 2024-03-27T10:56:33.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.637+0000 7f0136a00640 1 -- 172.21.15.97:0/1138083120 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0138039040 con 0x7f01400780f0 2024-03-27T10:56:33.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.637+0000 7f0145200640 1 -- 172.21.15.97:0/1138083120 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01401a61f0 con 0x7f01400780f0 2024-03-27T10:56:33.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.637+0000 7f0145200640 1 -- 172.21.15.97:0/1138083120 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01401a6690 con 0x7f01400780f0 2024-03-27T10:56:33.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.638+0000 7f0145200640 1 -- 172.21.15.97:0/1138083120 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01401108e0 con 0x7f01400780f0 2024-03-27T10:56:33.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.639+0000 7f0136a00640 1 -- 172.21.15.97:0/1138083120 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f0138044020 con 0x7f01400780f0 2024-03-27T10:56:33.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.641+0000 7f0136a00640 1 --2- 172.21.15.97:0/1138083120 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f01240a3690 0x7f01240a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:33.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.641+0000 7f0136a00640 1 -- 172.21.15.97:0/1138083120 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f01380f1f50 con 0x7f01400780f0 2024-03-27T10:56:33.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.641+0000 7f013de00640 1 --2- 172.21.15.97:0/1138083120 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f01240a3690 0x7f01240a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:33.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.642+0000 7f013de00640 1 --2- 172.21.15.97:0/1138083120 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f01240a3690 0x7f01240a5b50 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f013000bbe0 tx=0x7f013000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:33.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.645+0000 7f0136a00640 1 -- 172.21.15.97:0/1138083120 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f013804b020 con 0x7f01400780f0 2024-03-27T10:56:33.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:56:33 smithi177 ceph-mon[35294]: pgmap v438: 97 pgs: 97 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-03-27T10:56:33.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:56:33 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:56:33.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:56:33 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/1532575121' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T10:56:33.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:56:33 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:56:33.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:56:33 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:56:33.766 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:56:33 smithi114 ceph-mon[35338]: pgmap v438: 97 pgs: 97 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-03-27T10:56:33.766 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:56:33 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:56:33.767 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:56:33 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/1532575121' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T10:56:33.767 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:56:33 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:56:33.767 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:56:33 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:56:33.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.861+0000 7f0145200640 1 -- 172.21.15.97:0/1138083120 --> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0140061f90 con 0x7f01240a3690 2024-03-27T10:56:33.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.869+0000 7f0136a00640 1 -- 172.21.15.97:0/1138083120 <== mgr.14152 v2:172.21.15.97:6800/3955463538 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258915 (secure 0 0 0) 0x7f0140061f90 con 0x7f01240a3690 2024-03-27T10:56:33.844 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T10:56:33.845 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-03-27T10:56:33.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.874+0000 7f0145200640 1 -- 172.21.15.97:0/1138083120 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f01240a3690 msgr2=0x7f01240a5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:33.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.874+0000 7f0145200640 1 --2- 172.21.15.97:0/1138083120 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f01240a3690 0x7f01240a5b50 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f013000bbe0 tx=0x7f013000b390 comp rx=0 tx=0).stop 2024-03-27T10:56:33.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.874+0000 7f0145200640 1 -- 172.21.15.97:0/1138083120 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f01400780f0 msgr2=0x7f01401a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:33.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.874+0000 7f0145200640 1 --2- 172.21.15.97:0/1138083120 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f01400780f0 0x7f01401a58f0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f013802c700 tx=0x7f01380328a0 comp rx=0 tx=0).stop 2024-03-27T10:56:33.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.874+0000 7f0145200640 1 -- 172.21.15.97:0/1138083120 shutdown_connections 2024-03-27T10:56:33.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.874+0000 7f0145200640 1 --2- 172.21.15.97:0/1138083120 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f01400780f0 0x7f01401a58f0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:33.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.874+0000 7f0145200640 1 --2- 172.21.15.97:0/1138083120 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0140077750 0x7f014019e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:33.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.874+0000 7f0145200640 1 --2- 172.21.15.97:0/1138083120 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f01240a3690 0x7f01240a5b50 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:33.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.874+0000 7f0145200640 1 --2- 172.21.15.97:0/1138083120 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0140076510 0x7f014019e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:33.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.874+0000 7f0145200640 1 -- 172.21.15.97:0/1138083120 >> 172.21.15.97:0/1138083120 conn(0x7f01400ff300 msgr2=0x7f0140100a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:33.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.874+0000 7f0145200640 1 -- 172.21.15.97:0/1138083120 shutdown_connections 2024-03-27T10:56:33.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:33.874+0000 7f0145200640 1 -- 172.21.15.97:0/1138083120 wait complete. 2024-03-27T10:56:33.861 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":438,"stamp":"2024-03-27T10:56:32.116811+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":661848,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1550,"num_read_kb":897,"num_write":447,"num_write_kb":3093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1181318,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":867,"ondisk_log_size":867,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":379728,"kb_used_data":13936,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124562096,"statfs":{"total":1151940427776,"available":1151551586304,"internally_reserved":0,"allocated":14270464,"data_stored":7740870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":374453561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2465,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":105,"num_read_kb":55,"num_write":16,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005541"},"pg_stats":[{"pgid":"3.3f","version":"92'8","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.077510+0000","last_change":"2024-03-27T10:54:05.077845+0000","last_active":"2024-03-27T10:55:40.077510+0000","last_peered":"2024-03-27T10:55:40.077510+0000","last_clean":"2024-03-27T10:55:40.077510+0000","last_became_active":"2024-03-27T10:54:05.077494+0000","last_became_peered":"2024-03-27T10:54:05.077494+0000","last_unstale":"2024-03-27T10:55:40.077510+0000","last_undegraded":"2024-03-27T10:55:40.077510+0000","last_fullsized":"2024-03-27T10:55:40.077510+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:16:07.094746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.048785+0000","last_change":"2024-03-27T10:54:05.078830+0000","last_active":"2024-03-27T10:55:40.048785+0000","last_peered":"2024-03-27T10:55:40.048785+0000","last_clean":"2024-03-27T10:55:40.048785+0000","last_became_active":"2024-03-27T10:54:05.078474+0000","last_became_peered":"2024-03-27T10:54:05.078474+0000","last_unstale":"2024-03-27T10:55:40.048785+0000","last_undegraded":"2024-03-27T10:55:40.048785+0000","last_fullsized":"2024-03-27T10:55:40.048785+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T20:11:09.900802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.075900+0000","last_change":"2024-03-27T10:54:05.082554+0000","last_active":"2024-03-27T10:55:40.075900+0000","last_peered":"2024-03-27T10:55:40.075900+0000","last_clean":"2024-03-27T10:55:40.075900+0000","last_became_active":"2024-03-27T10:54:05.081395+0000","last_became_peered":"2024-03-27T10:54:05.081395+0000","last_unstale":"2024-03-27T10:55:40.075900+0000","last_undegraded":"2024-03-27T10:55:40.075900+0000","last_fullsized":"2024-03-27T10:55:40.075900+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T20:02:58.866792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.033675+0000","last_change":"2024-03-27T10:54:05.086700+0000","last_active":"2024-03-27T10:55:40.033675+0000","last_peered":"2024-03-27T10:55:40.033675+0000","last_clean":"2024-03-27T10:55:40.033675+0000","last_became_active":"2024-03-27T10:54:05.086347+0000","last_became_peered":"2024-03-27T10:54:05.086347+0000","last_unstale":"2024-03-27T10:55:40.033675+0000","last_undegraded":"2024-03-27T10:55:40.033675+0000","last_fullsized":"2024-03-27T10:55:40.033675+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T22:03:11.336365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.047556+0000","last_change":"2024-03-27T10:54:05.083479+0000","last_active":"2024-03-27T10:55:40.047556+0000","last_peered":"2024-03-27T10:55:40.047556+0000","last_clean":"2024-03-27T10:55:40.047556+0000","last_became_active":"2024-03-27T10:54:05.083162+0000","last_became_peered":"2024-03-27T10:54:05.083162+0000","last_unstale":"2024-03-27T10:55:40.047556+0000","last_undegraded":"2024-03-27T10:55:40.047556+0000","last_fullsized":"2024-03-27T10:55:40.047556+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:23:42.265668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.050257+0000","last_change":"2024-03-27T10:54:05.079096+0000","last_active":"2024-03-27T10:55:40.050257+0000","last_peered":"2024-03-27T10:55:40.050257+0000","last_clean":"2024-03-27T10:55:40.050257+0000","last_became_active":"2024-03-27T10:54:05.078778+0000","last_became_peered":"2024-03-27T10:54:05.078778+0000","last_unstale":"2024-03-27T10:55:40.050257+0000","last_undegraded":"2024-03-27T10:55:40.050257+0000","last_fullsized":"2024-03-27T10:55:40.050257+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T19:26:47.534594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.053852+0000","last_change":"2024-03-27T10:54:05.098926+0000","last_active":"2024-03-27T10:55:40.053852+0000","last_peered":"2024-03-27T10:55:40.053852+0000","last_clean":"2024-03-27T10:55:40.053852+0000","last_became_active":"2024-03-27T10:54:05.098139+0000","last_became_peered":"2024-03-27T10:54:05.098139+0000","last_unstale":"2024-03-27T10:55:40.053852+0000","last_undegraded":"2024-03-27T10:55:40.053852+0000","last_fullsized":"2024-03-27T10:55:40.053852+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:35:14.335947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.050624+0000","last_change":"2024-03-27T10:54:05.093786+0000","last_active":"2024-03-27T10:55:40.050624+0000","last_peered":"2024-03-27T10:55:40.050624+0000","last_clean":"2024-03-27T10:55:40.050624+0000","last_became_active":"2024-03-27T10:54:05.092047+0000","last_became_peered":"2024-03-27T10:54:05.092047+0000","last_unstale":"2024-03-27T10:55:40.050624+0000","last_undegraded":"2024-03-27T10:55:40.050624+0000","last_fullsized":"2024-03-27T10:55:40.050624+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T22:50:30.778358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.073745+0000","last_change":"2024-03-27T10:54:05.081678+0000","last_active":"2024-03-27T10:55:40.073745+0000","last_peered":"2024-03-27T10:55:40.073745+0000","last_clean":"2024-03-27T10:55:40.073745+0000","last_became_active":"2024-03-27T10:54:05.080264+0000","last_became_peered":"2024-03-27T10:54:05.080264+0000","last_unstale":"2024-03-27T10:55:40.073745+0000","last_undegraded":"2024-03-27T10:55:40.073745+0000","last_fullsized":"2024-03-27T10:55:40.073745+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T22:41:04.183282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.063980+0000","last_change":"2024-03-27T10:54:05.095172+0000","last_active":"2024-03-27T10:55:40.063980+0000","last_peered":"2024-03-27T10:55:40.063980+0000","last_clean":"2024-03-27T10:55:40.063980+0000","last_became_active":"2024-03-27T10:54:05.088438+0000","last_became_peered":"2024-03-27T10:54:05.088438+0000","last_unstale":"2024-03-27T10:55:40.063980+0000","last_undegraded":"2024-03-27T10:55:40.063980+0000","last_fullsized":"2024-03-27T10:55:40.063980+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T11:19:50.422788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.076689+0000","last_change":"2024-03-27T10:54:05.093599+0000","last_active":"2024-03-27T10:55:40.076689+0000","last_peered":"2024-03-27T10:55:40.076689+0000","last_clean":"2024-03-27T10:55:40.076689+0000","last_became_active":"2024-03-27T10:54:05.092248+0000","last_became_peered":"2024-03-27T10:54:05.092248+0000","last_unstale":"2024-03-27T10:55:40.076689+0000","last_undegraded":"2024-03-27T10:55:40.076689+0000","last_fullsized":"2024-03-27T10:55:40.076689+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:22:38.712442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.039555+0000","last_change":"2024-03-27T10:54:05.094677+0000","last_active":"2024-03-27T10:55:40.039555+0000","last_peered":"2024-03-27T10:55:40.039555+0000","last_clean":"2024-03-27T10:55:40.039555+0000","last_became_active":"2024-03-27T10:54:05.094326+0000","last_became_peered":"2024-03-27T10:54:05.094326+0000","last_unstale":"2024-03-27T10:55:40.039555+0000","last_undegraded":"2024-03-27T10:55:40.039555+0000","last_fullsized":"2024-03-27T10:55:40.039555+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T21:44:51.923881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.052563+0000","last_change":"2024-03-27T10:54:05.083178+0000","last_active":"2024-03-27T10:55:40.052563+0000","last_peered":"2024-03-27T10:55:40.052563+0000","last_clean":"2024-03-27T10:55:40.052563+0000","last_became_active":"2024-03-27T10:54:05.082805+0000","last_became_peered":"2024-03-27T10:54:05.082805+0000","last_unstale":"2024-03-27T10:55:40.052563+0000","last_undegraded":"2024-03-27T10:55:40.052563+0000","last_fullsized":"2024-03-27T10:55:40.052563+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T19:03:07.558006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.040999+0000","last_change":"2024-03-27T10:54:05.098092+0000","last_active":"2024-03-27T10:55:40.040999+0000","last_peered":"2024-03-27T10:55:40.040999+0000","last_clean":"2024-03-27T10:55:40.040999+0000","last_became_active":"2024-03-27T10:54:05.097759+0000","last_became_peered":"2024-03-27T10:54:05.097759+0000","last_unstale":"2024-03-27T10:55:40.040999+0000","last_undegraded":"2024-03-27T10:55:40.040999+0000","last_fullsized":"2024-03-27T10:55:40.040999+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:14:07.884759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.033846+0000","last_change":"2024-03-27T10:54:05.086089+0000","last_active":"2024-03-27T10:55:40.033846+0000","last_peered":"2024-03-27T10:55:40.033846+0000","last_clean":"2024-03-27T10:55:40.033846+0000","last_became_active":"2024-03-27T10:54:05.084830+0000","last_became_peered":"2024-03-27T10:54:05.084830+0000","last_unstale":"2024-03-27T10:55:40.033846+0000","last_undegraded":"2024-03-27T10:55:40.033846+0000","last_fullsized":"2024-03-27T10:55:40.033846+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T20:37:44.390102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.080920+0000","last_change":"2024-03-27T10:54:05.092190+0000","last_active":"2024-03-27T10:55:40.080920+0000","last_peered":"2024-03-27T10:55:40.080920+0000","last_clean":"2024-03-27T10:55:40.080920+0000","last_became_active":"2024-03-27T10:54:05.091827+0000","last_became_peered":"2024-03-27T10:54:05.091827+0000","last_unstale":"2024-03-27T10:55:40.080920+0000","last_undegraded":"2024-03-27T10:55:40.080920+0000","last_fullsized":"2024-03-27T10:55:40.080920+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T21:12:13.452829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.077720+0000","last_change":"2024-03-27T10:54:05.086924+0000","last_active":"2024-03-27T10:55:40.077720+0000","last_peered":"2024-03-27T10:55:40.077720+0000","last_clean":"2024-03-27T10:55:40.077720+0000","last_became_active":"2024-03-27T10:54:05.085701+0000","last_became_peered":"2024-03-27T10:54:05.085701+0000","last_unstale":"2024-03-27T10:55:40.077720+0000","last_undegraded":"2024-03-27T10:55:40.077720+0000","last_fullsized":"2024-03-27T10:55:40.077720+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:43:38.612288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.070593+0000","last_change":"2024-03-27T10:54:05.082532+0000","last_active":"2024-03-27T10:55:40.070593+0000","last_peered":"2024-03-27T10:55:40.070593+0000","last_clean":"2024-03-27T10:55:40.070593+0000","last_became_active":"2024-03-27T10:54:05.082200+0000","last_became_peered":"2024-03-27T10:54:05.082200+0000","last_unstale":"2024-03-27T10:55:40.070593+0000","last_undegraded":"2024-03-27T10:55:40.070593+0000","last_fullsized":"2024-03-27T10:55:40.070593+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T19:07:55.232728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.048428+0000","last_change":"2024-03-27T10:54:05.088204+0000","last_active":"2024-03-27T10:55:40.048428+0000","last_peered":"2024-03-27T10:55:40.048428+0000","last_clean":"2024-03-27T10:55:40.048428+0000","last_became_active":"2024-03-27T10:54:05.087705+0000","last_became_peered":"2024-03-27T10:54:05.087705+0000","last_unstale":"2024-03-27T10:55:40.048428+0000","last_undegraded":"2024-03-27T10:55:40.048428+0000","last_fullsized":"2024-03-27T10:55:40.048428+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T19:24:52.450205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"92'14","reported_seq":170,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:56:25.842018+0000","last_change":"2024-03-27T10:54:04.072182+0000","last_active":"2024-03-27T10:56:25.842018+0000","last_peered":"2024-03-27T10:56:25.842018+0000","last_clean":"2024-03-27T10:56:25.842018+0000","last_became_active":"2024-03-27T10:54:04.071827+0000","last_became_peered":"2024-03-27T10:54:04.071827+0000","last_unstale":"2024-03-27T10:56:25.842018+0000","last_undegraded":"2024-03-27T10:56:25.842018+0000","last_fullsized":"2024-03-27T10:56:25.842018+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T18:40:19.776684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":136,"num_read_kb":136,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.043065+0000","last_change":"2024-03-27T10:54:05.095772+0000","last_active":"2024-03-27T10:55:40.043065+0000","last_peered":"2024-03-27T10:55:40.043065+0000","last_clean":"2024-03-27T10:55:40.043065+0000","last_became_active":"2024-03-27T10:54:05.095385+0000","last_became_peered":"2024-03-27T10:54:05.095385+0000","last_unstale":"2024-03-27T10:55:40.043065+0000","last_undegraded":"2024-03-27T10:55:40.043065+0000","last_fullsized":"2024-03-27T10:55:40.043065+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T12:07:21.744097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.337196+0000","last_change":"2024-03-27T10:54:04.069600+0000","last_active":"2024-03-27T10:54:41.337196+0000","last_peered":"2024-03-27T10:54:41.337196+0000","last_clean":"2024-03-27T10:54:41.337196+0000","last_became_active":"2024-03-27T10:54:04.067770+0000","last_became_peered":"2024-03-27T10:54:04.067770+0000","last_unstale":"2024-03-27T10:54:41.337196+0000","last_undegraded":"2024-03-27T10:54:41.337196+0000","last_fullsized":"2024-03-27T10:54:41.337196+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T12:26:36.930945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.076797+0000","last_change":"2024-03-27T10:54:05.081779+0000","last_active":"2024-03-27T10:55:40.076797+0000","last_peered":"2024-03-27T10:55:40.076797+0000","last_clean":"2024-03-27T10:55:40.076797+0000","last_became_active":"2024-03-27T10:54:05.081492+0000","last_became_peered":"2024-03-27T10:54:05.081492+0000","last_unstale":"2024-03-27T10:55:40.076797+0000","last_undegraded":"2024-03-27T10:55:40.076797+0000","last_fullsized":"2024-03-27T10:55:40.076797+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T19:07:12.838048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"90'12","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.329099+0000","last_change":"2024-03-27T10:54:04.070550+0000","last_active":"2024-03-27T10:54:41.329099+0000","last_peered":"2024-03-27T10:54:41.329099+0000","last_clean":"2024-03-27T10:54:41.329099+0000","last_became_active":"2024-03-27T10:54:04.070010+0000","last_became_peered":"2024-03-27T10:54:04.070010+0000","last_unstale":"2024-03-27T10:54:41.329099+0000","last_undegraded":"2024-03-27T10:54:41.329099+0000","last_fullsized":"2024-03-27T10:54:41.329099+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:46:34.168771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.079329+0000","last_change":"2024-03-27T10:54:05.086372+0000","last_active":"2024-03-27T10:55:40.079329+0000","last_peered":"2024-03-27T10:55:40.079329+0000","last_clean":"2024-03-27T10:55:40.079329+0000","last_became_active":"2024-03-27T10:54:05.086013+0000","last_became_peered":"2024-03-27T10:54:05.086013+0000","last_unstale":"2024-03-27T10:55:40.079329+0000","last_undegraded":"2024-03-27T10:55:40.079329+0000","last_fullsized":"2024-03-27T10:55:40.079329+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:17:39.059071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"90'5","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.347107+0000","last_change":"2024-03-27T10:54:41.346475+0000","last_active":"2024-03-27T10:54:41.347107+0000","last_peered":"2024-03-27T10:54:41.347107+0000","last_clean":"2024-03-27T10:54:41.347107+0000","last_became_active":"2024-03-27T10:54:41.337431+0000","last_became_peered":"2024-03-27T10:54:41.337431+0000","last_unstale":"2024-03-27T10:54:41.347107+0000","last_undegraded":"2024-03-27T10:54:41.347107+0000","last_fullsized":"2024-03-27T10:54:41.347107+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T20:57:10.552047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.075638+0000","last_change":"2024-03-27T10:54:05.087580+0000","last_active":"2024-03-27T10:55:40.075638+0000","last_peered":"2024-03-27T10:55:40.075638+0000","last_clean":"2024-03-27T10:55:40.075638+0000","last_became_active":"2024-03-27T10:54:05.087128+0000","last_became_peered":"2024-03-27T10:54:05.087128+0000","last_unstale":"2024-03-27T10:55:40.075638+0000","last_undegraded":"2024-03-27T10:55:40.075638+0000","last_fullsized":"2024-03-27T10:55:40.075638+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T22:03:46.861258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"92'8","reported_seq":172,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:56:28.513097+0000","last_change":"2024-03-27T10:54:04.066312+0000","last_active":"2024-03-27T10:56:28.513097+0000","last_peered":"2024-03-27T10:56:28.513097+0000","last_clean":"2024-03-27T10:56:28.513097+0000","last_became_active":"2024-03-27T10:54:04.065909+0000","last_became_peered":"2024-03-27T10:54:04.065909+0000","last_unstale":"2024-03-27T10:56:28.513097+0000","last_undegraded":"2024-03-27T10:56:28.513097+0000","last_fullsized":"2024-03-27T10:56:28.513097+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:08:45.257808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1647,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":145,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.040498+0000","last_change":"2024-03-27T10:54:41.370991+0000","last_active":"2024-03-27T10:55:40.040498+0000","last_peered":"2024-03-27T10:55:40.040498+0000","last_clean":"2024-03-27T10:55:40.040498+0000","last_became_active":"2024-03-27T10:54:41.370596+0000","last_became_peered":"2024-03-27T10:54:41.370596+0000","last_unstale":"2024-03-27T10:55:40.040498+0000","last_undegraded":"2024-03-27T10:55:40.040498+0000","last_fullsized":"2024-03-27T10:55:40.040498+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:29:44.187503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"90'7","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.331143+0000","last_change":"2024-03-27T10:54:04.072451+0000","last_active":"2024-03-27T10:54:41.331143+0000","last_peered":"2024-03-27T10:54:41.331143+0000","last_clean":"2024-03-27T10:54:41.331143+0000","last_became_active":"2024-03-27T10:54:04.072042+0000","last_became_peered":"2024-03-27T10:54:04.072042+0000","last_unstale":"2024-03-27T10:54:41.331143+0000","last_undegraded":"2024-03-27T10:54:41.331143+0000","last_fullsized":"2024-03-27T10:54:41.331143+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T12:20:43.231073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.047114+0000","last_change":"2024-03-27T10:54:05.086055+0000","last_active":"2024-03-27T10:55:40.047114+0000","last_peered":"2024-03-27T10:55:40.047114+0000","last_clean":"2024-03-27T10:55:40.047114+0000","last_became_active":"2024-03-27T10:54:05.084429+0000","last_became_peered":"2024-03-27T10:54:05.084429+0000","last_unstale":"2024-03-27T10:55:40.047114+0000","last_undegraded":"2024-03-27T10:55:40.047114+0000","last_fullsized":"2024-03-27T10:55:40.047114+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T12:41:12.868245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"90'3","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.369391+0000","last_change":"2024-03-27T10:54:04.059076+0000","last_active":"2024-03-27T10:54:41.369391+0000","last_peered":"2024-03-27T10:54:41.369391+0000","last_clean":"2024-03-27T10:54:41.369391+0000","last_became_active":"2024-03-27T10:54:04.057614+0000","last_became_peered":"2024-03-27T10:54:04.057614+0000","last_unstale":"2024-03-27T10:54:41.369391+0000","last_undegraded":"2024-03-27T10:54:41.369391+0000","last_fullsized":"2024-03-27T10:54:41.369391+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T22:26:16.299840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.047600+0000","last_change":"2024-03-27T10:54:05.092955+0000","last_active":"2024-03-27T10:55:40.047600+0000","last_peered":"2024-03-27T10:55:40.047600+0000","last_clean":"2024-03-27T10:55:40.047600+0000","last_became_active":"2024-03-27T10:54:05.091867+0000","last_became_peered":"2024-03-27T10:54:05.091867+0000","last_unstale":"2024-03-27T10:55:40.047600+0000","last_undegraded":"2024-03-27T10:55:40.047600+0000","last_fullsized":"2024-03-27T10:55:40.047600+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T21:41:04.636480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"90'3","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.331834+0000","last_change":"2024-03-27T10:54:04.058833+0000","last_active":"2024-03-27T10:54:41.331834+0000","last_peered":"2024-03-27T10:54:41.331834+0000","last_clean":"2024-03-27T10:54:41.331834+0000","last_became_active":"2024-03-27T10:54:04.057750+0000","last_became_peered":"2024-03-27T10:54:04.057750+0000","last_unstale":"2024-03-27T10:54:41.331834+0000","last_undegraded":"2024-03-27T10:54:41.331834+0000","last_fullsized":"2024-03-27T10:54:41.331834+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:03:18.771094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.048019+0000","last_change":"2024-03-27T10:54:05.078933+0000","last_active":"2024-03-27T10:55:40.048019+0000","last_peered":"2024-03-27T10:55:40.048019+0000","last_clean":"2024-03-27T10:55:40.048019+0000","last_became_active":"2024-03-27T10:54:05.078544+0000","last_became_peered":"2024-03-27T10:54:05.078544+0000","last_unstale":"2024-03-27T10:55:40.048019+0000","last_undegraded":"2024-03-27T10:55:40.048019+0000","last_fullsized":"2024-03-27T10:55:40.048019+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T19:40:25.529775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"92'7","reported_seq":301,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:56:27.533739+0000","last_change":"2024-03-27T10:54:04.063974+0000","last_active":"2024-03-27T10:56:27.533739+0000","last_peered":"2024-03-27T10:56:27.533739+0000","last_clean":"2024-03-27T10:56:27.533739+0000","last_became_active":"2024-03-27T10:54:04.058171+0000","last_became_peered":"2024-03-27T10:54:04.058171+0000","last_unstale":"2024-03-27T10:56:27.533739+0000","last_undegraded":"2024-03-27T10:56:27.533739+0000","last_fullsized":"2024-03-27T10:56:27.533739+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T19:14:15.597889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":274,"num_read_kb":274,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.079116+0000","last_change":"2024-03-27T10:54:05.085727+0000","last_active":"2024-03-27T10:55:40.079116+0000","last_peered":"2024-03-27T10:55:40.079116+0000","last_clean":"2024-03-27T10:55:40.079116+0000","last_became_active":"2024-03-27T10:54:05.085354+0000","last_became_peered":"2024-03-27T10:54:05.085354+0000","last_unstale":"2024-03-27T10:55:40.079116+0000","last_undegraded":"2024-03-27T10:55:40.079116+0000","last_fullsized":"2024-03-27T10:55:40.079116+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:33:24.330313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"90'4","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.333606+0000","last_change":"2024-03-27T10:54:04.061757+0000","last_active":"2024-03-27T10:54:41.333606+0000","last_peered":"2024-03-27T10:54:41.333606+0000","last_clean":"2024-03-27T10:54:41.333606+0000","last_became_active":"2024-03-27T10:54:04.059211+0000","last_became_peered":"2024-03-27T10:54:04.059211+0000","last_unstale":"2024-03-27T10:54:41.333606+0000","last_undegraded":"2024-03-27T10:54:41.333606+0000","last_fullsized":"2024-03-27T10:54:41.333606+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T11:20:41.391143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.045487+0000","last_change":"2024-03-27T10:54:05.090919+0000","last_active":"2024-03-27T10:55:40.045487+0000","last_peered":"2024-03-27T10:55:40.045487+0000","last_clean":"2024-03-27T10:55:40.045487+0000","last_became_active":"2024-03-27T10:54:05.087971+0000","last_became_peered":"2024-03-27T10:54:05.087971+0000","last_unstale":"2024-03-27T10:55:40.045487+0000","last_undegraded":"2024-03-27T10:55:40.045487+0000","last_fullsized":"2024-03-27T10:55:40.045487+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:00:45.195957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"90'5","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.334856+0000","last_change":"2024-03-27T10:54:04.064828+0000","last_active":"2024-03-27T10:54:41.334856+0000","last_peered":"2024-03-27T10:54:41.334856+0000","last_clean":"2024-03-27T10:54:41.334856+0000","last_became_active":"2024-03-27T10:54:04.064359+0000","last_became_peered":"2024-03-27T10:54:04.064359+0000","last_unstale":"2024-03-27T10:54:41.334856+0000","last_undegraded":"2024-03-27T10:54:41.334856+0000","last_fullsized":"2024-03-27T10:54:41.334856+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:05:19.905707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.081122+0000","last_change":"2024-03-27T10:54:41.337153+0000","last_active":"2024-03-27T10:55:40.081122+0000","last_peered":"2024-03-27T10:55:40.081122+0000","last_clean":"2024-03-27T10:55:40.081122+0000","last_became_active":"2024-03-27T10:54:41.336433+0000","last_became_peered":"2024-03-27T10:54:41.336433+0000","last_unstale":"2024-03-27T10:55:40.081122+0000","last_undegraded":"2024-03-27T10:55:40.081122+0000","last_fullsized":"2024-03-27T10:55:40.081122+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:00:25.944147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,7],"acting":[1,8,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"92'35","reported_seq":341,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:56:27.489303+0000","last_change":"2024-03-27T10:54:04.061913+0000","last_active":"2024-03-27T10:56:27.489303+0000","last_peered":"2024-03-27T10:56:27.489303+0000","last_clean":"2024-03-27T10:56:27.489303+0000","last_became_active":"2024-03-27T10:54:04.058366+0000","last_became_peered":"2024-03-27T10:54:04.058366+0000","last_unstale":"2024-03-27T10:56:27.489303+0000","last_undegraded":"2024-03-27T10:56:27.489303+0000","last_fullsized":"2024-03-27T10:56:27.489303+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:45:28.365554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61788,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":286,"num_read_kb":203,"num_write":37,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.061525+0000","last_change":"2024-03-27T10:54:05.090840+0000","last_active":"2024-03-27T10:55:40.061525+0000","last_peered":"2024-03-27T10:55:40.061525+0000","last_clean":"2024-03-27T10:55:40.061525+0000","last_became_active":"2024-03-27T10:54:05.090525+0000","last_became_peered":"2024-03-27T10:54:05.090525+0000","last_unstale":"2024-03-27T10:55:40.061525+0000","last_undegraded":"2024-03-27T10:55:40.061525+0000","last_fullsized":"2024-03-27T10:55:40.061525+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T21:26:10.559917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"90'9","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.336322+0000","last_change":"2024-03-27T10:54:04.061205+0000","last_active":"2024-03-27T10:54:41.336322+0000","last_peered":"2024-03-27T10:54:41.336322+0000","last_clean":"2024-03-27T10:54:41.336322+0000","last_became_active":"2024-03-27T10:54:04.059558+0000","last_became_peered":"2024-03-27T10:54:04.059558+0000","last_unstale":"2024-03-27T10:54:41.336322+0000","last_undegraded":"2024-03-27T10:54:41.336322+0000","last_fullsized":"2024-03-27T10:54:41.336322+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T12:00:00.182233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.068118+0000","last_change":"2024-03-27T10:54:05.090149+0000","last_active":"2024-03-27T10:55:40.068118+0000","last_peered":"2024-03-27T10:55:40.068118+0000","last_clean":"2024-03-27T10:55:40.068118+0000","last_became_active":"2024-03-27T10:54:05.089317+0000","last_became_peered":"2024-03-27T10:54:05.089317+0000","last_unstale":"2024-03-27T10:55:40.068118+0000","last_undegraded":"2024-03-27T10:55:40.068118+0000","last_fullsized":"2024-03-27T10:55:40.068118+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T11:50:06.477472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"90'5","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.333600+0000","last_change":"2024-03-27T10:54:04.061797+0000","last_active":"2024-03-27T10:54:41.333600+0000","last_peered":"2024-03-27T10:54:41.333600+0000","last_clean":"2024-03-27T10:54:41.333600+0000","last_became_active":"2024-03-27T10:54:04.060974+0000","last_became_peered":"2024-03-27T10:54:04.060974+0000","last_unstale":"2024-03-27T10:54:41.333600+0000","last_undegraded":"2024-03-27T10:54:41.333600+0000","last_fullsized":"2024-03-27T10:54:41.333600+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T18:34:33.486126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.057613+0000","last_change":"2024-03-27T10:54:41.370832+0000","last_active":"2024-03-27T10:55:40.057613+0000","last_peered":"2024-03-27T10:55:40.057613+0000","last_clean":"2024-03-27T10:55:40.057613+0000","last_became_active":"2024-03-27T10:54:41.370229+0000","last_became_peered":"2024-03-27T10:54:41.370229+0000","last_unstale":"2024-03-27T10:55:40.057613+0000","last_undegraded":"2024-03-27T10:55:40.057613+0000","last_fullsized":"2024-03-27T10:55:40.057613+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:53:19.887501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"92'8","reported_seq":171,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:56:27.517918+0000","last_change":"2024-03-27T10:54:04.060042+0000","last_active":"2024-03-27T10:56:27.517918+0000","last_peered":"2024-03-27T10:56:27.517918+0000","last_clean":"2024-03-27T10:56:27.517918+0000","last_became_active":"2024-03-27T10:54:04.059415+0000","last_became_peered":"2024-03-27T10:54:04.059415+0000","last_unstale":"2024-03-27T10:56:27.517918+0000","last_undegraded":"2024-03-27T10:56:27.517918+0000","last_fullsized":"2024-03-27T10:56:27.517918+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:52:06.846942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1613,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":144,"num_read_kb":6,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.066681+0000","last_change":"2024-03-27T10:54:05.100088+0000","last_active":"2024-03-27T10:55:40.066681+0000","last_peered":"2024-03-27T10:55:40.066681+0000","last_clean":"2024-03-27T10:55:40.066681+0000","last_became_active":"2024-03-27T10:54:05.098709+0000","last_became_peered":"2024-03-27T10:54:05.098709+0000","last_unstale":"2024-03-27T10:55:40.066681+0000","last_undegraded":"2024-03-27T10:55:40.066681+0000","last_fullsized":"2024-03-27T10:55:40.066681+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T21:25:13.329710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"90'17","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.327516+0000","last_change":"2024-03-27T10:54:04.066812+0000","last_active":"2024-03-27T10:54:41.327516+0000","last_peered":"2024-03-27T10:54:41.327516+0000","last_clean":"2024-03-27T10:54:41.327516+0000","last_became_active":"2024-03-27T10:54:04.066364+0000","last_became_peered":"2024-03-27T10:54:04.066364+0000","last_unstale":"2024-03-27T10:54:41.327516+0000","last_undegraded":"2024-03-27T10:54:41.327516+0000","last_fullsized":"2024-03-27T10:54:41.327516+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T19:31:51.616785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.074821+0000","last_change":"2024-03-27T10:54:05.089367+0000","last_active":"2024-03-27T10:55:40.074821+0000","last_peered":"2024-03-27T10:55:40.074821+0000","last_clean":"2024-03-27T10:55:40.074821+0000","last_became_active":"2024-03-27T10:54:05.088718+0000","last_became_peered":"2024-03-27T10:54:05.088718+0000","last_unstale":"2024-03-27T10:55:40.074821+0000","last_undegraded":"2024-03-27T10:55:40.074821+0000","last_fullsized":"2024-03-27T10:55:40.074821+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:56:08.768859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"92'3","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:56:18.706651+0000","last_change":"2024-03-27T10:54:04.059951+0000","last_active":"2024-03-27T10:56:18.706651+0000","last_peered":"2024-03-27T10:56:18.706651+0000","last_clean":"2024-03-27T10:56:18.706651+0000","last_became_active":"2024-03-27T10:54:04.057245+0000","last_became_peered":"2024-03-27T10:54:04.057245+0000","last_unstale":"2024-03-27T10:56:18.706651+0000","last_undegraded":"2024-03-27T10:56:18.706651+0000","last_fullsized":"2024-03-27T10:56:18.706651+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T19:36:48.881920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.051789+0000","last_change":"2024-03-27T10:54:05.089004+0000","last_active":"2024-03-27T10:55:40.051789+0000","last_peered":"2024-03-27T10:55:40.051789+0000","last_clean":"2024-03-27T10:55:40.051789+0000","last_became_active":"2024-03-27T10:54:05.083764+0000","last_became_peered":"2024-03-27T10:54:05.083764+0000","last_unstale":"2024-03-27T10:55:40.051789+0000","last_undegraded":"2024-03-27T10:55:40.051789+0000","last_fullsized":"2024-03-27T10:55:40.051789+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:09:53.883490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"90'1","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.326444+0000","last_change":"2024-03-27T10:54:04.050508+0000","last_active":"2024-03-27T10:54:41.326444+0000","last_peered":"2024-03-27T10:54:41.326444+0000","last_clean":"2024-03-27T10:54:41.326444+0000","last_became_active":"2024-03-27T10:54:04.048877+0000","last_became_peered":"2024-03-27T10:54:04.048877+0000","last_unstale":"2024-03-27T10:54:41.326444+0000","last_undegraded":"2024-03-27T10:54:41.326444+0000","last_fullsized":"2024-03-27T10:54:41.326444+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:44:11.578684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"92'5","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.045703+0000","last_change":"2024-03-27T10:54:05.083931+0000","last_active":"2024-03-27T10:55:40.045703+0000","last_peered":"2024-03-27T10:55:40.045703+0000","last_clean":"2024-03-27T10:55:40.045703+0000","last_became_active":"2024-03-27T10:54:05.083544+0000","last_became_peered":"2024-03-27T10:54:05.083544+0000","last_unstale":"2024-03-27T10:55:40.045703+0000","last_undegraded":"2024-03-27T10:55:40.045703+0000","last_fullsized":"2024-03-27T10:55:40.045703+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T20:54:19.172355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"92'6","reported_seq":166,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:56:28.536227+0000","last_change":"2024-03-27T10:54:04.057428+0000","last_active":"2024-03-27T10:56:28.536227+0000","last_peered":"2024-03-27T10:56:28.536227+0000","last_clean":"2024-03-27T10:56:28.536227+0000","last_became_active":"2024-03-27T10:54:04.052966+0000","last_became_peered":"2024-03-27T10:54:04.052966+0000","last_unstale":"2024-03-27T10:56:28.536227+0000","last_undegraded":"2024-03-27T10:56:28.536227+0000","last_fullsized":"2024-03-27T10:56:28.536227+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T18:50:31.170411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1081,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":140,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.050606+0000","last_change":"2024-03-27T10:54:41.438829+0000","last_active":"2024-03-27T10:55:40.050606+0000","last_peered":"2024-03-27T10:55:40.050606+0000","last_clean":"2024-03-27T10:55:40.050606+0000","last_became_active":"2024-03-27T10:54:41.438031+0000","last_became_peered":"2024-03-27T10:54:41.438031+0000","last_unstale":"2024-03-27T10:55:40.050606+0000","last_undegraded":"2024-03-27T10:55:40.050606+0000","last_fullsized":"2024-03-27T10:55:40.050606+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:33:04.052747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.035927+0000","last_change":"2024-03-27T10:54:05.090255+0000","last_active":"2024-03-27T10:55:40.035927+0000","last_peered":"2024-03-27T10:55:40.035927+0000","last_clean":"2024-03-27T10:55:40.035927+0000","last_became_active":"2024-03-27T10:54:05.088833+0000","last_became_peered":"2024-03-27T10:54:05.088833+0000","last_unstale":"2024-03-27T10:55:40.035927+0000","last_undegraded":"2024-03-27T10:55:40.035927+0000","last_fullsized":"2024-03-27T10:55:40.035927+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:15:55.934698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"92'7","reported_seq":167,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:56:27.516617+0000","last_change":"2024-03-27T10:54:04.059941+0000","last_active":"2024-03-27T10:56:27.516617+0000","last_peered":"2024-03-27T10:56:27.516617+0000","last_clean":"2024-03-27T10:56:27.516617+0000","last_became_active":"2024-03-27T10:54:04.058197+0000","last_became_peered":"2024-03-27T10:54:04.058197+0000","last_unstale":"2024-03-27T10:56:27.516617+0000","last_undegraded":"2024-03-27T10:56:27.516617+0000","last_fullsized":"2024-03-27T10:56:27.516617+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:44:46.374054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":140,"num_read_kb":140,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.078902+0000","last_change":"2024-03-27T10:54:41.337917+0000","last_active":"2024-03-27T10:55:40.078902+0000","last_peered":"2024-03-27T10:55:40.078902+0000","last_clean":"2024-03-27T10:55:40.078902+0000","last_became_active":"2024-03-27T10:54:41.337624+0000","last_became_peered":"2024-03-27T10:54:41.337624+0000","last_unstale":"2024-03-27T10:55:40.078902+0000","last_undegraded":"2024-03-27T10:55:40.078902+0000","last_fullsized":"2024-03-27T10:55:40.078902+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T11:50:48.275234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"90'6","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.327556+0000","last_change":"2024-03-27T10:54:04.064017+0000","last_active":"2024-03-27T10:54:41.327556+0000","last_peered":"2024-03-27T10:54:41.327556+0000","last_clean":"2024-03-27T10:54:41.327556+0000","last_became_active":"2024-03-27T10:54:04.062881+0000","last_became_peered":"2024-03-27T10:54:04.062881+0000","last_unstale":"2024-03-27T10:54:41.327556+0000","last_undegraded":"2024-03-27T10:54:41.327556+0000","last_fullsized":"2024-03-27T10:54:41.327556+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:21:21.638623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":349,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.333171+0000","last_change":"2024-03-27T10:52:31.127495+0000","last_active":"2024-03-27T10:54:41.333171+0000","last_peered":"2024-03-27T10:54:41.333171+0000","last_clean":"2024-03-27T10:54:41.333171+0000","last_became_active":"2024-03-27T10:52:31.125418+0000","last_became_peered":"2024-03-27T10:52:31.125418+0000","last_unstale":"2024-03-27T10:54:41.333171+0000","last_undegraded":"2024-03-27T10:54:41.333171+0000","last_fullsized":"2024-03-27T10:54:41.333171+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:47:54.649615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:47:54.649615+0000","last_clean_scrub_stamp":"2024-03-27T10:47:54.649615+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T18:49:24.591580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.047107+0000","last_change":"2024-03-27T10:54:05.085558+0000","last_active":"2024-03-27T10:55:40.047107+0000","last_peered":"2024-03-27T10:55:40.047107+0000","last_clean":"2024-03-27T10:55:40.047107+0000","last_became_active":"2024-03-27T10:54:05.083896+0000","last_became_peered":"2024-03-27T10:54:05.083896+0000","last_unstale":"2024-03-27T10:55:40.047107+0000","last_undegraded":"2024-03-27T10:55:40.047107+0000","last_fullsized":"2024-03-27T10:55:40.047107+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:15:29.697609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'4","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.333395+0000","last_change":"2024-03-27T10:54:04.072448+0000","last_active":"2024-03-27T10:54:41.333395+0000","last_peered":"2024-03-27T10:54:41.333395+0000","last_clean":"2024-03-27T10:54:41.333395+0000","last_became_active":"2024-03-27T10:54:04.071933+0000","last_became_peered":"2024-03-27T10:54:04.071933+0000","last_unstale":"2024-03-27T10:54:41.333395+0000","last_undegraded":"2024-03-27T10:54:41.333395+0000","last_fullsized":"2024-03-27T10:54:41.333395+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:34:49.204345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"90'4","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.328806+0000","last_change":"2024-03-27T10:54:04.067308+0000","last_active":"2024-03-27T10:54:41.328806+0000","last_peered":"2024-03-27T10:54:41.328806+0000","last_clean":"2024-03-27T10:54:41.328806+0000","last_became_active":"2024-03-27T10:54:04.066927+0000","last_became_peered":"2024-03-27T10:54:04.066927+0000","last_unstale":"2024-03-27T10:54:41.328806+0000","last_undegraded":"2024-03-27T10:54:41.328806+0000","last_fullsized":"2024-03-27T10:54:41.328806+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:08:04.849245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.049057+0000","last_change":"2024-03-27T10:54:05.075154+0000","last_active":"2024-03-27T10:55:40.049057+0000","last_peered":"2024-03-27T10:55:40.049057+0000","last_clean":"2024-03-27T10:55:40.049057+0000","last_became_active":"2024-03-27T10:54:05.074772+0000","last_became_peered":"2024-03-27T10:54:05.074772+0000","last_unstale":"2024-03-27T10:55:40.049057+0000","last_undegraded":"2024-03-27T10:55:40.049057+0000","last_fullsized":"2024-03-27T10:55:40.049057+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T20:49:16.286229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"92'15","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:56:17.327925+0000","last_change":"2024-03-27T10:54:04.066805+0000","last_active":"2024-03-27T10:56:17.327925+0000","last_peered":"2024-03-27T10:56:17.327925+0000","last_clean":"2024-03-27T10:56:17.327925+0000","last_became_active":"2024-03-27T10:54:04.066500+0000","last_became_peered":"2024-03-27T10:54:04.066500+0000","last_unstale":"2024-03-27T10:56:17.327925+0000","last_undegraded":"2024-03-27T10:56:17.327925+0000","last_fullsized":"2024-03-27T10:56:17.327925+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T12:37:44.853575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.070237+0000","last_change":"2024-03-27T10:54:05.094314+0000","last_active":"2024-03-27T10:55:40.070237+0000","last_peered":"2024-03-27T10:55:40.070237+0000","last_clean":"2024-03-27T10:55:40.070237+0000","last_became_active":"2024-03-27T10:54:05.093896+0000","last_became_peered":"2024-03-27T10:54:05.093896+0000","last_unstale":"2024-03-27T10:55:40.070237+0000","last_undegraded":"2024-03-27T10:55:40.070237+0000","last_fullsized":"2024-03-27T10:55:40.070237+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:30:02.441317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"90'7","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.328737+0000","last_change":"2024-03-27T10:54:04.065525+0000","last_active":"2024-03-27T10:54:41.328737+0000","last_peered":"2024-03-27T10:54:41.328737+0000","last_clean":"2024-03-27T10:54:41.328737+0000","last_became_active":"2024-03-27T10:54:04.064979+0000","last_became_peered":"2024-03-27T10:54:04.064979+0000","last_unstale":"2024-03-27T10:54:41.328737+0000","last_undegraded":"2024-03-27T10:54:41.328737+0000","last_fullsized":"2024-03-27T10:54:41.328737+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:58:25.110861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.042948+0000","last_change":"2024-03-27T10:54:05.084038+0000","last_active":"2024-03-27T10:55:40.042948+0000","last_peered":"2024-03-27T10:55:40.042948+0000","last_clean":"2024-03-27T10:55:40.042948+0000","last_became_active":"2024-03-27T10:54:05.083733+0000","last_became_peered":"2024-03-27T10:54:05.083733+0000","last_unstale":"2024-03-27T10:55:40.042948+0000","last_undegraded":"2024-03-27T10:55:40.042948+0000","last_fullsized":"2024-03-27T10:55:40.042948+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T18:51:14.832602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.330550+0000","last_change":"2024-03-27T10:54:04.064506+0000","last_active":"2024-03-27T10:54:41.330550+0000","last_peered":"2024-03-27T10:54:41.330550+0000","last_clean":"2024-03-27T10:54:41.330550+0000","last_became_active":"2024-03-27T10:54:04.064178+0000","last_became_peered":"2024-03-27T10:54:04.064178+0000","last_unstale":"2024-03-27T10:54:41.330550+0000","last_undegraded":"2024-03-27T10:54:41.330550+0000","last_fullsized":"2024-03-27T10:54:41.330550+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:07:19.465782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.038119+0000","last_change":"2024-03-27T10:54:05.088241+0000","last_active":"2024-03-27T10:55:40.038119+0000","last_peered":"2024-03-27T10:55:40.038119+0000","last_clean":"2024-03-27T10:55:40.038119+0000","last_became_active":"2024-03-27T10:54:05.087852+0000","last_became_peered":"2024-03-27T10:54:05.087852+0000","last_unstale":"2024-03-27T10:55:40.038119+0000","last_undegraded":"2024-03-27T10:55:40.038119+0000","last_fullsized":"2024-03-27T10:55:40.038119+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:47:09.633654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"90'7","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.336541+0000","last_change":"2024-03-27T10:54:04.065744+0000","last_active":"2024-03-27T10:54:41.336541+0000","last_peered":"2024-03-27T10:54:41.336541+0000","last_clean":"2024-03-27T10:54:41.336541+0000","last_became_active":"2024-03-27T10:54:04.065155+0000","last_became_peered":"2024-03-27T10:54:04.065155+0000","last_unstale":"2024-03-27T10:54:41.336541+0000","last_undegraded":"2024-03-27T10:54:41.336541+0000","last_fullsized":"2024-03-27T10:54:41.336541+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T12:35:00.556458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.049948+0000","last_change":"2024-03-27T10:54:05.077954+0000","last_active":"2024-03-27T10:55:40.049948+0000","last_peered":"2024-03-27T10:55:40.049948+0000","last_clean":"2024-03-27T10:55:40.049948+0000","last_became_active":"2024-03-27T10:54:05.077589+0000","last_became_peered":"2024-03-27T10:54:05.077589+0000","last_unstale":"2024-03-27T10:55:40.049948+0000","last_undegraded":"2024-03-27T10:55:40.049948+0000","last_fullsized":"2024-03-27T10:55:40.049948+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T20:18:27.411887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"92'4","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.574937+0000","last_change":"2024-03-27T10:54:41.352576+0000","last_active":"2024-03-27T10:54:41.574937+0000","last_peered":"2024-03-27T10:54:41.574937+0000","last_clean":"2024-03-27T10:54:41.574937+0000","last_became_active":"2024-03-27T10:54:41.338775+0000","last_became_peered":"2024-03-27T10:54:41.338775+0000","last_unstale":"2024-03-27T10:54:41.574937+0000","last_undegraded":"2024-03-27T10:54:41.574937+0000","last_fullsized":"2024-03-27T10:54:41.574937+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T18:23:14.194920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.051269+0000","last_change":"2024-03-27T10:54:05.088250+0000","last_active":"2024-03-27T10:55:40.051269+0000","last_peered":"2024-03-27T10:55:40.051269+0000","last_clean":"2024-03-27T10:55:40.051269+0000","last_became_active":"2024-03-27T10:54:05.087942+0000","last_became_peered":"2024-03-27T10:54:05.087942+0000","last_unstale":"2024-03-27T10:55:40.051269+0000","last_undegraded":"2024-03-27T10:55:40.051269+0000","last_fullsized":"2024-03-27T10:55:40.051269+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:57:45.266347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.327767+0000","last_change":"2024-03-27T10:54:04.070513+0000","last_active":"2024-03-27T10:54:41.327767+0000","last_peered":"2024-03-27T10:54:41.327767+0000","last_clean":"2024-03-27T10:54:41.327767+0000","last_became_active":"2024-03-27T10:54:04.070042+0000","last_became_peered":"2024-03-27T10:54:04.070042+0000","last_unstale":"2024-03-27T10:54:41.327767+0000","last_undegraded":"2024-03-27T10:54:41.327767+0000","last_fullsized":"2024-03-27T10:54:41.327767+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:40:25.216899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.080525+0000","last_change":"2024-03-27T10:54:05.095067+0000","last_active":"2024-03-27T10:55:40.080525+0000","last_peered":"2024-03-27T10:55:40.080525+0000","last_clean":"2024-03-27T10:55:40.080525+0000","last_became_active":"2024-03-27T10:54:05.094618+0000","last_became_peered":"2024-03-27T10:54:05.094618+0000","last_unstale":"2024-03-27T10:55:40.080525+0000","last_undegraded":"2024-03-27T10:55:40.080525+0000","last_fullsized":"2024-03-27T10:55:40.080525+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:57:59.557310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"92'6","reported_seq":166,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:56:27.535118+0000","last_change":"2024-03-27T10:54:04.051352+0000","last_active":"2024-03-27T10:56:27.535118+0000","last_peered":"2024-03-27T10:56:27.535118+0000","last_clean":"2024-03-27T10:56:27.535118+0000","last_became_active":"2024-03-27T10:54:04.050850+0000","last_became_peered":"2024-03-27T10:54:04.050850+0000","last_unstale":"2024-03-27T10:56:27.535118+0000","last_undegraded":"2024-03-27T10:56:27.535118+0000","last_fullsized":"2024-03-27T10:56:27.535118+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:50:16.719451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1663,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":140,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.080038+0000","last_change":"2024-03-27T10:54:05.094650+0000","last_active":"2024-03-27T10:55:40.080038+0000","last_peered":"2024-03-27T10:55:40.080038+0000","last_clean":"2024-03-27T10:55:40.080038+0000","last_became_active":"2024-03-27T10:54:05.094168+0000","last_became_peered":"2024-03-27T10:54:05.094168+0000","last_unstale":"2024-03-27T10:55:40.080038+0000","last_undegraded":"2024-03-27T10:55:40.080038+0000","last_fullsized":"2024-03-27T10:55:40.080038+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:29:56.491733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"90'3","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.334649+0000","last_change":"2024-03-27T10:54:04.067755+0000","last_active":"2024-03-27T10:54:41.334649+0000","last_peered":"2024-03-27T10:54:41.334649+0000","last_clean":"2024-03-27T10:54:41.334649+0000","last_became_active":"2024-03-27T10:54:04.067320+0000","last_became_peered":"2024-03-27T10:54:04.067320+0000","last_unstale":"2024-03-27T10:54:41.334649+0000","last_undegraded":"2024-03-27T10:54:41.334649+0000","last_fullsized":"2024-03-27T10:54:41.334649+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T12:16:25.949829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"92'7","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.062408+0000","last_change":"2024-03-27T10:54:05.100715+0000","last_active":"2024-03-27T10:55:40.062408+0000","last_peered":"2024-03-27T10:55:40.062408+0000","last_clean":"2024-03-27T10:55:40.062408+0000","last_became_active":"2024-03-27T10:54:05.097861+0000","last_became_peered":"2024-03-27T10:54:05.097861+0000","last_unstale":"2024-03-27T10:55:40.062408+0000","last_undegraded":"2024-03-27T10:55:40.062408+0000","last_fullsized":"2024-03-27T10:55:40.062408+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:02:39.107232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"90'5","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:54:41.369747+0000","last_change":"2024-03-27T10:54:04.063894+0000","last_active":"2024-03-27T10:54:41.369747+0000","last_peered":"2024-03-27T10:54:41.369747+0000","last_clean":"2024-03-27T10:54:41.369747+0000","last_became_active":"2024-03-27T10:54:04.063163+0000","last_became_peered":"2024-03-27T10:54:04.063163+0000","last_unstale":"2024-03-27T10:54:41.369747+0000","last_undegraded":"2024-03-27T10:54:41.369747+0000","last_fullsized":"2024-03-27T10:54:41.369747+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:03.008462+0000","last_clean_scrub_stamp":"2024-03-27T10:54:03.008462+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:30:41.025752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.069291+0000","last_change":"2024-03-27T10:54:05.101980+0000","last_active":"2024-03-27T10:55:40.069291+0000","last_peered":"2024-03-27T10:55:40.069291+0000","last_clean":"2024-03-27T10:55:40.069291+0000","last_became_active":"2024-03-27T10:54:05.101448+0000","last_became_peered":"2024-03-27T10:54:05.101448+0000","last_unstale":"2024-03-27T10:55:40.069291+0000","last_undegraded":"2024-03-27T10:55:40.069291+0000","last_fullsized":"2024-03-27T10:55:40.069291+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:45:11.137181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.053342+0000","last_change":"2024-03-27T10:54:05.084026+0000","last_active":"2024-03-27T10:55:40.053342+0000","last_peered":"2024-03-27T10:55:40.053342+0000","last_clean":"2024-03-27T10:55:40.053342+0000","last_became_active":"2024-03-27T10:54:05.083410+0000","last_became_peered":"2024-03-27T10:54:05.083410+0000","last_unstale":"2024-03-27T10:55:40.053342+0000","last_undegraded":"2024-03-27T10:55:40.053342+0000","last_fullsized":"2024-03-27T10:55:40.053342+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T22:39:34.586363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.047447+0000","last_change":"2024-03-27T10:54:05.083592+0000","last_active":"2024-03-27T10:55:40.047447+0000","last_peered":"2024-03-27T10:55:40.047447+0000","last_clean":"2024-03-27T10:55:40.047447+0000","last_became_active":"2024-03-27T10:54:05.082537+0000","last_became_peered":"2024-03-27T10:54:05.082537+0000","last_unstale":"2024-03-27T10:55:40.047447+0000","last_undegraded":"2024-03-27T10:55:40.047447+0000","last_fullsized":"2024-03-27T10:55:40.047447+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T21:15:36.830805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.040301+0000","last_change":"2024-03-27T10:54:05.090226+0000","last_active":"2024-03-27T10:55:40.040301+0000","last_peered":"2024-03-27T10:55:40.040301+0000","last_clean":"2024-03-27T10:55:40.040301+0000","last_became_active":"2024-03-27T10:54:05.089831+0000","last_became_peered":"2024-03-27T10:54:05.089831+0000","last_unstale":"2024-03-27T10:55:40.040301+0000","last_undegraded":"2024-03-27T10:55:40.040301+0000","last_fullsized":"2024-03-27T10:55:40.040301+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T12:45:57.802266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.047183+0000","last_change":"2024-03-27T10:54:05.080196+0000","last_active":"2024-03-27T10:55:40.047183+0000","last_peered":"2024-03-27T10:55:40.047183+0000","last_clean":"2024-03-27T10:55:40.047183+0000","last_became_active":"2024-03-27T10:54:05.079822+0000","last_became_peered":"2024-03-27T10:54:05.079822+0000","last_unstale":"2024-03-27T10:55:40.047183+0000","last_undegraded":"2024-03-27T10:55:40.047183+0000","last_fullsized":"2024-03-27T10:55:40.047183+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:53:11.556503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.040513+0000","last_change":"2024-03-27T10:54:05.087264+0000","last_active":"2024-03-27T10:55:40.040513+0000","last_peered":"2024-03-27T10:55:40.040513+0000","last_clean":"2024-03-27T10:55:40.040513+0000","last_became_active":"2024-03-27T10:54:05.085440+0000","last_became_peered":"2024-03-27T10:54:05.085440+0000","last_unstale":"2024-03-27T10:55:40.040513+0000","last_undegraded":"2024-03-27T10:55:40.040513+0000","last_fullsized":"2024-03-27T10:55:40.040513+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T22:41:40.665643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.050820+0000","last_change":"2024-03-27T10:54:05.094306+0000","last_active":"2024-03-27T10:55:40.050820+0000","last_peered":"2024-03-27T10:55:40.050820+0000","last_clean":"2024-03-27T10:55:40.050820+0000","last_became_active":"2024-03-27T10:54:05.093858+0000","last_became_peered":"2024-03-27T10:54:05.093858+0000","last_unstale":"2024-03-27T10:55:40.050820+0000","last_undegraded":"2024-03-27T10:55:40.050820+0000","last_fullsized":"2024-03-27T10:55:40.050820+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T21:22:57.624833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.077439+0000","last_change":"2024-03-27T10:54:05.094089+0000","last_active":"2024-03-27T10:55:40.077439+0000","last_peered":"2024-03-27T10:55:40.077439+0000","last_clean":"2024-03-27T10:55:40.077439+0000","last_became_active":"2024-03-27T10:54:05.092317+0000","last_became_peered":"2024-03-27T10:54:05.092317+0000","last_unstale":"2024-03-27T10:55:40.077439+0000","last_undegraded":"2024-03-27T10:55:40.077439+0000","last_fullsized":"2024-03-27T10:55:40.077439+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:34:57.606250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.042440+0000","last_change":"2024-03-27T10:54:05.099995+0000","last_active":"2024-03-27T10:55:40.042440+0000","last_peered":"2024-03-27T10:55:40.042440+0000","last_clean":"2024-03-27T10:55:40.042440+0000","last_became_active":"2024-03-27T10:54:05.099641+0000","last_became_peered":"2024-03-27T10:54:05.099641+0000","last_unstale":"2024-03-27T10:55:40.042440+0000","last_undegraded":"2024-03-27T10:55:40.042440+0000","last_fullsized":"2024-03-27T10:55:40.042440+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:01:16.050180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.043273+0000","last_change":"2024-03-27T10:54:41.371472+0000","last_active":"2024-03-27T10:55:40.043273+0000","last_peered":"2024-03-27T10:55:40.043273+0000","last_clean":"2024-03-27T10:55:40.043273+0000","last_became_active":"2024-03-27T10:54:41.370912+0000","last_became_peered":"2024-03-27T10:54:41.370912+0000","last_unstale":"2024-03-27T10:55:40.043273+0000","last_undegraded":"2024-03-27T10:55:40.043273+0000","last_fullsized":"2024-03-27T10:55:40.043273+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T20:21:28.950573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6],"acting":[5,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.046487+0000","last_change":"2024-03-27T10:54:05.096812+0000","last_active":"2024-03-27T10:55:40.046487+0000","last_peered":"2024-03-27T10:55:40.046487+0000","last_clean":"2024-03-27T10:55:40.046487+0000","last_became_active":"2024-03-27T10:54:05.096323+0000","last_became_peered":"2024-03-27T10:54:05.096323+0000","last_unstale":"2024-03-27T10:55:40.046487+0000","last_undegraded":"2024-03-27T10:55:40.046487+0000","last_fullsized":"2024-03-27T10:55:40.046487+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:39:29.827291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.080404+0000","last_change":"2024-03-27T10:54:05.089137+0000","last_active":"2024-03-27T10:55:40.080404+0000","last_peered":"2024-03-27T10:55:40.080404+0000","last_clean":"2024-03-27T10:55:40.080404+0000","last_became_active":"2024-03-27T10:54:05.088795+0000","last_became_peered":"2024-03-27T10:54:05.088795+0000","last_unstale":"2024-03-27T10:55:40.080404+0000","last_undegraded":"2024-03-27T10:55:40.080404+0000","last_fullsized":"2024-03-27T10:55:40.080404+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T19:47:35.231749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.060769+0000","last_change":"2024-03-27T10:54:05.089683+0000","last_active":"2024-03-27T10:55:40.060769+0000","last_peered":"2024-03-27T10:55:40.060769+0000","last_clean":"2024-03-27T10:55:40.060769+0000","last_became_active":"2024-03-27T10:54:05.089235+0000","last_became_peered":"2024-03-27T10:54:05.089235+0000","last_unstale":"2024-03-27T10:55:40.060769+0000","last_undegraded":"2024-03-27T10:55:40.060769+0000","last_fullsized":"2024-03-27T10:55:40.060769+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:43:37.012814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-27T10:55:40.049661+0000","last_change":"2024-03-27T10:54:05.085043+0000","last_active":"2024-03-27T10:55:40.049661+0000","last_peered":"2024-03-27T10:55:40.049661+0000","last_clean":"2024-03-27T10:55:40.049661+0000","last_became_active":"2024-03-27T10:54:05.084695+0000","last_became_peered":"2024-03-27T10:54:05.084695+0000","last_unstale":"2024-03-27T10:55:40.049661+0000","last_undegraded":"2024-03-27T10:55:40.049661+0000","last_fullsized":"2024-03-27T10:55:40.049661+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T10:54:04.017133+0000","last_clean_scrub_stamp":"2024-03-27T10:54:04.017133+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T12:07:31.327006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":71246,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1424,"num_read_kb":788,"num_write":264,"num_write_kb":153,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":569344,"data_stored":218052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":228,"ondisk_log_size":228,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":85,"seq":365072220206,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":481518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":77,"seq":330712481845,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37000,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708152,"statfs":{"total":95995035648,"available":95957147648,"internally_reserved":0,"allocated":1703936,"data_stored":1134285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":71,"seq":304942678075,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1028096,"data_stored":480373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":63,"seq":270582939714,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36332,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708820,"statfs":{"total":95995035648,"available":95957831680,"internally_reserved":0,"allocated":1019904,"data_stored":480169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":54,"seq":231928234057,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28600,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716552,"statfs":{"total":95995035648,"available":95965749248,"internally_reserved":0,"allocated":1622016,"data_stored":1075421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":49,"seq":210453397583,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":480583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574962,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1077248,"data_stored":547598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738488,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":481922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411434,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1093632,"data_stored":544529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117093,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1028096,"data_stored":479915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":35,"seq":150323855454,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36324,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708828,"statfs":{"total":95995035648,"available":95957839872,"internally_reserved":0,"allocated":1011712,"data_stored":480976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":41,"seq":176093659223,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36932,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708220,"statfs":{"total":95995035648,"available":95957217280,"internally_reserved":0,"allocated":1634304,"data_stored":1073581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":68347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":65278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":64549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-27T10:56:33.866 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-27T10:56:34.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.143+0000 7f8c6b200640 1 -- 172.21.15.97:0/358337908 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c64105920 msgr2=0x7f8c6410c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:34.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.143+0000 7f8c6b200640 1 --2- 172.21.15.97:0/358337908 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c64105920 0x7f8c6410c1b0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c009ca0 tx=0x7f8c5c02c0f0 comp rx=0 tx=0).stop 2024-03-27T10:56:34.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.143+0000 7f8c6b200640 1 -- 172.21.15.97:0/358337908 shutdown_connections 2024-03-27T10:56:34.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.143+0000 7f8c6b200640 1 --2- 172.21.15.97:0/358337908 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c64105920 0x7f8c6410c1b0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:34.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.143+0000 7f8c6b200640 1 --2- 172.21.15.97:0/358337908 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c64104f60 0x7f8c641053e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:34.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.143+0000 7f8c6b200640 1 --2- 172.21.15.97:0/358337908 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8c64103d70 0x7f8c64104190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:34.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.143+0000 7f8c6b200640 1 -- 172.21.15.97:0/358337908 >> 172.21.15.97:0/358337908 conn(0x7f8c640ff300 msgr2=0x7f8c64101720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:34.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.143+0000 7f8c6b200640 1 -- 172.21.15.97:0/358337908 shutdown_connections 2024-03-27T10:56:34.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.144+0000 7f8c6b200640 1 -- 172.21.15.97:0/358337908 wait complete. 2024-03-27T10:56:34.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.144+0000 7f8c6b200640 1 Processor -- start 2024-03-27T10:56:34.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.145+0000 7f8c6b200640 1 -- start start 2024-03-27T10:56:34.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.145+0000 7f8c6b200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c64103d70 0x7f8c64199f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:34.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.145+0000 7f8c6b200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8c64104f60 0x7f8c6419a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:34.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.145+0000 7f8c6b200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c64105920 0x7f8c641a14c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:34.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.145+0000 7f8c6b200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8c6410f880 con 0x7f8c64103d70 2024-03-27T10:56:34.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.145+0000 7f8c6b200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f8c6410f700 con 0x7f8c64105920 2024-03-27T10:56:34.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.145+0000 7f8c6b200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f8c6410fa00 con 0x7f8c64104f60 2024-03-27T10:56:34.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.145+0000 7f8c63e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c64103d70 0x7f8c64199f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:34.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.145+0000 7f8c63e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c64103d70 0x7f8c64199f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53560/0 (socket says 172.21.15.97:53560) 2024-03-27T10:56:34.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.145+0000 7f8c63e00640 1 -- 172.21.15.97:0/1446177462 learned_addr learned my addr 172.21.15.97:0/1446177462 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:56:34.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.145+0000 7f8c63400640 1 --2- 172.21.15.97:0/1446177462 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8c64104f60 0x7f8c6419a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:34.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.145+0000 7f8c69200640 1 --2- 172.21.15.97:0/1446177462 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c64105920 0x7f8c641a14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:34.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.146+0000 7f8c63e00640 1 -- 172.21.15.97:0/1446177462 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8c64104f60 msgr2=0x7f8c6419a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:34.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.146+0000 7f8c63e00640 1 --2- 172.21.15.97:0/1446177462 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8c64104f60 0x7f8c6419a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:34.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.146+0000 7f8c63e00640 1 -- 172.21.15.97:0/1446177462 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c64105920 msgr2=0x7f8c641a14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:34.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.146+0000 7f8c63e00640 1 --2- 172.21.15.97:0/1446177462 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c64105920 0x7f8c641a14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:34.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.146+0000 7f8c63e00640 1 -- 172.21.15.97:0/1446177462 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c641a1b30 con 0x7f8c64103d70 2024-03-27T10:56:34.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.146+0000 7f8c63e00640 1 --2- 172.21.15.97:0/1446177462 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c64103d70 0x7f8c64199f10 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f8c5400e9c0 tx=0x7f8c5400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:34.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.146+0000 7f8c60c00640 1 -- 172.21.15.97:0/1446177462 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c5400cd80 con 0x7f8c64103d70 2024-03-27T10:56:34.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.146+0000 7f8c6b200640 1 -- 172.21.15.97:0/1446177462 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c641a1e20 con 0x7f8c64103d70 2024-03-27T10:56:34.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.146+0000 7f8c60c00640 1 -- 172.21.15.97:0/1446177462 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8c540035f0 con 0x7f8c64103d70 2024-03-27T10:56:34.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.146+0000 7f8c60c00640 1 -- 172.21.15.97:0/1446177462 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c54018630 con 0x7f8c64103d70 2024-03-27T10:56:34.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.146+0000 7f8c6b200640 1 -- 172.21.15.97:0/1446177462 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c641a2290 con 0x7f8c64103d70 2024-03-27T10:56:34.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.148+0000 7f8c6b200640 1 -- 172.21.15.97:0/1446177462 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c6410b160 con 0x7f8c64103d70 2024-03-27T10:56:34.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.150+0000 7f8c60c00640 1 -- 172.21.15.97:0/1446177462 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f8c540037a0 con 0x7f8c64103d70 2024-03-27T10:56:34.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.151+0000 7f8c60c00640 1 --2- 172.21.15.97:0/1446177462 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f8c480a3610 0x7f8c480a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:34.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.151+0000 7f8c60c00640 1 -- 172.21.15.97:0/1446177462 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f8c540d01f0 con 0x7f8c64103d70 2024-03-27T10:56:34.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.151+0000 7f8c63400640 1 --2- 172.21.15.97:0/1446177462 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f8c480a3610 0x7f8c480a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:34.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.152+0000 7f8c63400640 1 --2- 172.21.15.97:0/1446177462 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f8c480a3610 0x7f8c480a5ad0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8c6419b420 tx=0x7f8c50005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:34.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.154+0000 7f8c60c00640 1 -- 172.21.15.97:0/1446177462 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f8c5409cdd0 con 0x7f8c64103d70 2024-03-27T10:56:34.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.396+0000 7f8c6b200640 1 -- 172.21.15.97:0/1446177462 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8c640627b0 con 0x7f8c64103d70 2024-03-27T10:56:34.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.397+0000 7f8c60c00640 1 -- 172.21.15.97:0/1446177462 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+22644 (secure 0 0 0) 0x7f8c540870e0 con 0x7f8c64103d70 2024-03-27T10:56:34.369 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T10:56:34.369 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":92,"fsid":"adca88c4-ec26-11ee-95d0-87774f69a715","created":"2024-03-27T10:43:24.680694+0000","modified":"2024-03-27T10:54:41.310812+0000","last_up_change":"2024-03-27T10:53:00.552513+0000","last_in_change":"2024-03-27T10:52:36.302986+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-27T10:47:54.052810+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-27T10:54:02.845060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-27T10:54:03.474151+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b634e5f1-6118-4e2f-95e9-830db390f8d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1746812080},{"type":"v1","addr":"172.21.15.97:6803","nonce":1746812080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1746812080},{"type":"v1","addr":"172.21.15.97:6805","nonce":1746812080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":1746812080},{"type":"v1","addr":"172.21.15.97:6809","nonce":1746812080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1746812080},{"type":"v1","addr":"172.21.15.97:6807","nonce":1746812080}]},"public_addr":"172.21.15.97:6803/1746812080","cluster_addr":"172.21.15.97:6805/1746812080","heartbeat_back_addr":"172.21.15.97:6809/1746812080","heartbeat_front_addr":"172.21.15.97:6807/1746812080","state":["exists","up"]},{"osd":1,"uuid":"441e2770-3284-4388-b0de-c56eb8a373d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":3858821264},{"type":"v1","addr":"172.21.15.114:6801","nonce":3858821264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":3858821264},{"type":"v1","addr":"172.21.15.114:6803","nonce":3858821264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":3858821264},{"type":"v1","addr":"172.21.15.114:6807","nonce":3858821264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":3858821264},{"type":"v1","addr":"172.21.15.114:6805","nonce":3858821264}]},"public_addr":"172.21.15.114:6801/3858821264","cluster_addr":"172.21.15.114:6803/3858821264","heartbeat_back_addr":"172.21.15.114:6807/3858821264","heartbeat_front_addr":"172.21.15.114:6805/3858821264","state":["exists","up"]},{"osd":2,"uuid":"00108864-1aa6-48d6-8398-e8eb79fda123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":124240658},{"type":"v1","addr":"172.21.15.177:6801","nonce":124240658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":124240658},{"type":"v1","addr":"172.21.15.177:6803","nonce":124240658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":124240658},{"type":"v1","addr":"172.21.15.177:6807","nonce":124240658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":124240658},{"type":"v1","addr":"172.21.15.177:6805","nonce":124240658}]},"public_addr":"172.21.15.177:6801/124240658","cluster_addr":"172.21.15.177:6803/124240658","heartbeat_back_addr":"172.21.15.177:6807/124240658","heartbeat_front_addr":"172.21.15.177:6805/124240658","state":["exists","up"]},{"osd":3,"uuid":"1860b690-e2dc-4977-88d7-d1aacf7022ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2478179311},{"type":"v1","addr":"172.21.15.97:6811","nonce":2478179311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2478179311},{"type":"v1","addr":"172.21.15.97:6813","nonce":2478179311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":2478179311},{"type":"v1","addr":"172.21.15.97:6817","nonce":2478179311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2478179311},{"type":"v1","addr":"172.21.15.97:6815","nonce":2478179311}]},"public_addr":"172.21.15.97:6811/2478179311","cluster_addr":"172.21.15.97:6813/2478179311","heartbeat_back_addr":"172.21.15.97:6817/2478179311","heartbeat_front_addr":"172.21.15.97:6815/2478179311","state":["exists","up"]},{"osd":4,"uuid":"9245a3a4-7d49-4fdb-9d55-112e43177050","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":75755945},{"type":"v1","addr":"172.21.15.114:6809","nonce":75755945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":75755945},{"type":"v1","addr":"172.21.15.114:6811","nonce":75755945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":75755945},{"type":"v1","addr":"172.21.15.114:6815","nonce":75755945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":75755945},{"type":"v1","addr":"172.21.15.114:6813","nonce":75755945}]},"public_addr":"172.21.15.114:6809/75755945","cluster_addr":"172.21.15.114:6811/75755945","heartbeat_back_addr":"172.21.15.114:6815/75755945","heartbeat_front_addr":"172.21.15.114:6813/75755945","state":["exists","up"]},{"osd":5,"uuid":"51922f99-e8d3-461b-8615-e9bfdcbf5ed3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":1918493790},{"type":"v1","addr":"172.21.15.177:6809","nonce":1918493790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":1918493790},{"type":"v1","addr":"172.21.15.177:6811","nonce":1918493790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":1918493790},{"type":"v1","addr":"172.21.15.177:6815","nonce":1918493790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":1918493790},{"type":"v1","addr":"172.21.15.177:6813","nonce":1918493790}]},"public_addr":"172.21.15.177:6809/1918493790","cluster_addr":"172.21.15.177:6811/1918493790","heartbeat_back_addr":"172.21.15.177:6815/1918493790","heartbeat_front_addr":"172.21.15.177:6813/1918493790","state":["exists","up"]},{"osd":6,"uuid":"06dffb32-05eb-43b3-852e-915c5cc7f25a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2892389563},{"type":"v1","addr":"172.21.15.97:6819","nonce":2892389563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2892389563},{"type":"v1","addr":"172.21.15.97:6821","nonce":2892389563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2892389563},{"type":"v1","addr":"172.21.15.97:6825","nonce":2892389563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2892389563},{"type":"v1","addr":"172.21.15.97:6823","nonce":2892389563}]},"public_addr":"172.21.15.97:6819/2892389563","cluster_addr":"172.21.15.97:6821/2892389563","heartbeat_back_addr":"172.21.15.97:6825/2892389563","heartbeat_front_addr":"172.21.15.97:6823/2892389563","state":["exists","up"]},{"osd":7,"uuid":"e68e3519-9c1b-43c5-8e06-5385248b19df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":2445491153},{"type":"v1","addr":"172.21.15.114:6817","nonce":2445491153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":2445491153},{"type":"v1","addr":"172.21.15.114:6819","nonce":2445491153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2445491153},{"type":"v1","addr":"172.21.15.114:6823","nonce":2445491153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":2445491153},{"type":"v1","addr":"172.21.15.114:6821","nonce":2445491153}]},"public_addr":"172.21.15.114:6817/2445491153","cluster_addr":"172.21.15.114:6819/2445491153","heartbeat_back_addr":"172.21.15.114:6823/2445491153","heartbeat_front_addr":"172.21.15.114:6821/2445491153","state":["exists","up"]},{"osd":8,"uuid":"31ac592e-1349-49ee-9af9-8ad85d682978","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":2051134191},{"type":"v1","addr":"172.21.15.177:6817","nonce":2051134191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":2051134191},{"type":"v1","addr":"172.21.15.177:6819","nonce":2051134191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":2051134191},{"type":"v1","addr":"172.21.15.177:6823","nonce":2051134191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":2051134191},{"type":"v1","addr":"172.21.15.177:6821","nonce":2051134191}]},"public_addr":"172.21.15.177:6817/2051134191","cluster_addr":"172.21.15.177:6819/2051134191","heartbeat_back_addr":"172.21.15.177:6823/2051134191","heartbeat_front_addr":"172.21.15.177:6821/2051134191","state":["exists","up"]},{"osd":9,"uuid":"0171086a-dd41-4eb4-a389-16f0c46a64db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":1578881271},{"type":"v1","addr":"172.21.15.97:6827","nonce":1578881271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1578881271},{"type":"v1","addr":"172.21.15.97:6829","nonce":1578881271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1578881271},{"type":"v1","addr":"172.21.15.97:6833","nonce":1578881271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1578881271},{"type":"v1","addr":"172.21.15.97:6831","nonce":1578881271}]},"public_addr":"172.21.15.97:6827/1578881271","cluster_addr":"172.21.15.97:6829/1578881271","heartbeat_back_addr":"172.21.15.97:6833/1578881271","heartbeat_front_addr":"172.21.15.97:6831/1578881271","state":["exists","up"]},{"osd":10,"uuid":"64754e31-b124-4601-ad98-2f70b1dd5359","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":2893111461},{"type":"v1","addr":"172.21.15.114:6825","nonce":2893111461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":2893111461},{"type":"v1","addr":"172.21.15.114:6827","nonce":2893111461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2893111461},{"type":"v1","addr":"172.21.15.114:6831","nonce":2893111461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":2893111461},{"type":"v1","addr":"172.21.15.114:6829","nonce":2893111461}]},"public_addr":"172.21.15.114:6825/2893111461","cluster_addr":"172.21.15.114:6827/2893111461","heartbeat_back_addr":"172.21.15.114:6831/2893111461","heartbeat_front_addr":"172.21.15.114:6829/2893111461","state":["exists","up"]},{"osd":11,"uuid":"9ffc752c-adf7-4037-89d3-535787d7475d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":85,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":2333030391},{"type":"v1","addr":"172.21.15.177:6825","nonce":2333030391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":2333030391},{"type":"v1","addr":"172.21.15.177:6827","nonce":2333030391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":2333030391},{"type":"v1","addr":"172.21.15.177:6831","nonce":2333030391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":2333030391},{"type":"v1","addr":"172.21.15.177:6829","nonce":2333030391}]},"public_addr":"172.21.15.177:6825/2333030391","cluster_addr":"172.21.15.177:6827/2333030391","heartbeat_back_addr":"172.21.15.177:6831/2333030391","heartbeat_front_addr":"172.21.15.177:6829/2333030391","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:46:45.143519+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:47:18.656947+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:47:51.316345+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:48:24.575512+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:48:59.113555+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:49:32.103865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:50:06.615539+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:50:41.578341+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:51:15.254011+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:51:49.099165+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:52:24.544821+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:52:58.838484+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.28","mappings":[{"from":3,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/359646":"2024-03-28T10:44:39.889550+0000","172.21.15.97:6801/98682155":"2024-03-28T10:44:06.019075+0000","172.21.15.97:6801/1685921853":"2024-03-28T10:44:39.889550+0000","172.21.15.97:6800/98682155":"2024-03-28T10:44:06.019075+0000","172.21.15.97:0/107690680":"2024-03-28T10:44:06.019075+0000","172.21.15.97:6800/1685921853":"2024-03-28T10:44:39.889550+0000","172.21.15.97:0/437043413":"2024-03-28T10:44:39.889550+0000","172.21.15.97:0/2319706992":"2024-03-28T10:44:39.889550+0000","172.21.15.97:0/3911532654":"2024-03-28T10:44:06.019075+0000","172.21.15.97:0/3347321845":"2024-03-28T10:44:06.019075+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-27T10:56:34.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.400+0000 7f8c6b200640 1 -- 172.21.15.97:0/1446177462 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f8c480a3610 msgr2=0x7f8c480a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:34.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.400+0000 7f8c6b200640 1 --2- 172.21.15.97:0/1446177462 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f8c480a3610 0x7f8c480a5ad0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8c6419b420 tx=0x7f8c50005f90 comp rx=0 tx=0).stop 2024-03-27T10:56:34.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.400+0000 7f8c6b200640 1 -- 172.21.15.97:0/1446177462 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c64103d70 msgr2=0x7f8c64199f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:34.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.400+0000 7f8c6b200640 1 --2- 172.21.15.97:0/1446177462 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c64103d70 0x7f8c64199f10 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f8c5400e9c0 tx=0x7f8c5400ee80 comp rx=0 tx=0).stop 2024-03-27T10:56:34.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.400+0000 7f8c6b200640 1 -- 172.21.15.97:0/1446177462 shutdown_connections 2024-03-27T10:56:34.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.400+0000 7f8c6b200640 1 --2- 172.21.15.97:0/1446177462 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f8c480a3610 0x7f8c480a5ad0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:34.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.400+0000 7f8c6b200640 1 --2- 172.21.15.97:0/1446177462 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c64105920 0x7f8c641a14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:34.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.400+0000 7f8c6b200640 1 --2- 172.21.15.97:0/1446177462 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8c64104f60 0x7f8c6419a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:34.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.400+0000 7f8c6b200640 1 --2- 172.21.15.97:0/1446177462 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c64103d70 0x7f8c64199f10 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:34.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.400+0000 7f8c6b200640 1 -- 172.21.15.97:0/1446177462 >> 172.21.15.97:0/1446177462 conn(0x7f8c640ff300 msgr2=0x7f8c64108330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:34.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.400+0000 7f8c6b200640 1 -- 172.21.15.97:0/1446177462 shutdown_connections 2024-03-27T10:56:34.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.400+0000 7f8c6b200640 1 -- 172.21.15.97:0/1446177462 wait complete. 2024-03-27T10:56:34.389 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-27T10:47:54.052810+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-27T10:54:02.845060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-27T10:54:03.474151+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-27T10:56:34.389 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-27T10:56:34.586 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:56:34 smithi114 ceph-mon[35338]: from='client.15429 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-27T10:56:34.586 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:56:34 smithi114 ceph-mon[35338]: from='client.15435 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-27T10:56:34.586 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:56:34 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:56:34.586 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:56:34 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:56:34.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.623+0000 7f1f5e800640 1 -- 172.21.15.97:0/1117254171 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f58103d90 msgr2=0x7f1f581041b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:34.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.623+0000 7f1f5e800640 1 --2- 172.21.15.97:0/1117254171 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f58103d90 0x7f1f581041b0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f1f48009900 tx=0x7f1f4802bc30 comp rx=0 tx=0).stop 2024-03-27T10:56:34.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.623+0000 7f1f5e800640 1 -- 172.21.15.97:0/1117254171 shutdown_connections 2024-03-27T10:56:34.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.623+0000 7f1f5e800640 1 --2- 172.21.15.97:0/1117254171 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1f58105940 0x7f1f5810c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:34.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.623+0000 7f1f5e800640 1 --2- 172.21.15.97:0/1117254171 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1f58104f80 0x7f1f58105400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:34.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.623+0000 7f1f5e800640 1 --2- 172.21.15.97:0/1117254171 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f58103d90 0x7f1f581041b0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:34.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.623+0000 7f1f5e800640 1 -- 172.21.15.97:0/1117254171 >> 172.21.15.97:0/1117254171 conn(0x7f1f580ff300 msgr2=0x7f1f58101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:34.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.623+0000 7f1f5e800640 1 -- 172.21.15.97:0/1117254171 shutdown_connections 2024-03-27T10:56:34.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.623+0000 7f1f5e800640 1 -- 172.21.15.97:0/1117254171 wait complete. 2024-03-27T10:56:34.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.624+0000 7f1f5e800640 1 Processor -- start 2024-03-27T10:56:34.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.624+0000 7f1f5e800640 1 -- start start 2024-03-27T10:56:34.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.624+0000 7f1f5e800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f58103d90 0x7f1f58199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:34.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.624+0000 7f1f5e800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1f58104f80 0x7f1f5819a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:34.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.624+0000 7f1f5e800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1f58105940 0x7f1f581a1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:34.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.624+0000 7f1f5e800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1f5810f910 con 0x7f1f58103d90 2024-03-27T10:56:34.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.624+0000 7f1f5e800640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f1f5810f790 con 0x7f1f58105940 2024-03-27T10:56:34.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.624+0000 7f1f5e800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1f5810fa90 con 0x7f1f58104f80 2024-03-27T10:56:34.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.625+0000 7f1f57400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f58103d90 0x7f1f58199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:34.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.625+0000 7f1f57400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f58103d90 0x7f1f58199fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53574/0 (socket says 172.21.15.97:53574) 2024-03-27T10:56:34.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.625+0000 7f1f57400640 1 -- 172.21.15.97:0/1257102101 learned_addr learned my addr 172.21.15.97:0/1257102101 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:56:34.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.625+0000 7f1f57e00640 1 --2- 172.21.15.97:0/1257102101 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1f58105940 0x7f1f581a1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:34.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.625+0000 7f1f56a00640 1 --2- 172.21.15.97:0/1257102101 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1f58104f80 0x7f1f5819a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:34.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.625+0000 7f1f57e00640 1 -- 172.21.15.97:0/1257102101 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1f58104f80 msgr2=0x7f1f5819a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:34.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.625+0000 7f1f57e00640 1 --2- 172.21.15.97:0/1257102101 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1f58104f80 0x7f1f5819a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:34.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.625+0000 7f1f57e00640 1 -- 172.21.15.97:0/1257102101 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f58103d90 msgr2=0x7f1f58199fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:34.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.625+0000 7f1f57e00640 1 --2- 172.21.15.97:0/1257102101 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f58103d90 0x7f1f58199fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:34.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.625+0000 7f1f57e00640 1 -- 172.21.15.97:0/1257102101 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f581a1bd0 con 0x7f1f58105940 2024-03-27T10:56:34.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.625+0000 7f1f56a00640 1 --2- 172.21.15.97:0/1257102101 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1f58104f80 0x7f1f5819a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T10:56:34.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.625+0000 7f1f57400640 1 --2- 172.21.15.97:0/1257102101 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f58103d90 0x7f1f58199fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T10:56:34.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.626+0000 7f1f57e00640 1 --2- 172.21.15.97:0/1257102101 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1f58105940 0x7f1f581a1560 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1f50009fd0 tx=0x7f1f5000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:34.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.626+0000 7f1f4fe00640 1 -- 172.21.15.97:0/1257102101 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f5000ce20 con 0x7f1f58105940 2024-03-27T10:56:34.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.626+0000 7f1f4fe00640 1 -- 172.21.15.97:0/1257102101 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1f50005020 con 0x7f1f58105940 2024-03-27T10:56:34.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.626+0000 7f1f5e800640 1 -- 172.21.15.97:0/1257102101 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f581a1ec0 con 0x7f1f58105940 2024-03-27T10:56:34.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.626+0000 7f1f4fe00640 1 -- 172.21.15.97:0/1257102101 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f50010690 con 0x7f1f58105940 2024-03-27T10:56:34.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.626+0000 7f1f5e800640 1 -- 172.21.15.97:0/1257102101 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f581a2380 con 0x7f1f58105940 2024-03-27T10:56:34.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.627+0000 7f1f5e800640 1 -- 172.21.15.97:0/1257102101 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f5810b130 con 0x7f1f58105940 2024-03-27T10:56:34.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.629+0000 7f1f4fe00640 1 -- 172.21.15.97:0/1257102101 <== mon.2 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f1f500051d0 con 0x7f1f58105940 2024-03-27T10:56:34.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.631+0000 7f1f4fe00640 1 --2- 172.21.15.97:0/1257102101 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f1f3c0a3610 0x7f1f3c0a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:34.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.632+0000 7f1f4fe00640 1 -- 172.21.15.97:0/1257102101 <== mon.2 v2:172.21.15.114:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f1f500d04c0 con 0x7f1f58105940 2024-03-27T10:56:34.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.632+0000 7f1f57400640 1 --2- 172.21.15.97:0/1257102101 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f1f3c0a3610 0x7f1f3c0a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:34.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.632+0000 7f1f57400640 1 --2- 172.21.15.97:0/1257102101 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f1f3c0a3610 0x7f1f3c0a5ad0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f1f480096c0 tx=0x7f1f4802c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:34.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.641+0000 7f1f4fe00640 1 -- 172.21.15.97:0/1257102101 <== mon.2 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f1f5009d040 con 0x7f1f58105940 2024-03-27T10:56:34.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:56:34 smithi097 ceph-mon[31859]: from='client.15429 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-27T10:56:34.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:56:34 smithi097 ceph-mon[31859]: from='client.15435 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-27T10:56:34.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:56:34 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:56:34.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:56:34 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:56:34.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:56:34 smithi177 ceph-mon[35294]: from='client.15429 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-27T10:56:34.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:56:34 smithi177 ceph-mon[35294]: from='client.15435 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-27T10:56:34.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:56:34 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:56:34.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:56:34 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:56:34.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.863+0000 7f1f5e800640 1 -- 172.21.15.97:0/1257102101 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1f580627f0 con 0x7f1f58105940 2024-03-27T10:56:34.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.863+0000 7f1f4fe00640 1 -- 172.21.15.97:0/1257102101 <== mon.2 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) ==== 93+0+10 (secure 0 0 0) 0x7f1f5009c940 con 0x7f1f58105940 2024-03-27T10:56:34.835 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-03-27T10:56:34.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.866+0000 7f1f5e800640 1 -- 172.21.15.97:0/1257102101 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f1f3c0a3610 msgr2=0x7f1f3c0a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:34.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.866+0000 7f1f5e800640 1 --2- 172.21.15.97:0/1257102101 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f1f3c0a3610 0x7f1f3c0a5ad0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f1f480096c0 tx=0x7f1f4802c7e0 comp rx=0 tx=0).stop 2024-03-27T10:56:34.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.866+0000 7f1f5e800640 1 -- 172.21.15.97:0/1257102101 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1f58105940 msgr2=0x7f1f581a1560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:34.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.866+0000 7f1f5e800640 1 --2- 172.21.15.97:0/1257102101 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1f58105940 0x7f1f581a1560 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1f50009fd0 tx=0x7f1f5000ef10 comp rx=0 tx=0).stop 2024-03-27T10:56:34.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.867+0000 7f1f5e800640 1 -- 172.21.15.97:0/1257102101 shutdown_connections 2024-03-27T10:56:34.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.867+0000 7f1f5e800640 1 --2- 172.21.15.97:0/1257102101 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f1f3c0a3610 0x7f1f3c0a5ad0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:34.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.867+0000 7f1f5e800640 1 --2- 172.21.15.97:0/1257102101 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1f58105940 0x7f1f581a1560 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:34.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.867+0000 7f1f5e800640 1 --2- 172.21.15.97:0/1257102101 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1f58104f80 0x7f1f5819a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:34.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.867+0000 7f1f5e800640 1 --2- 172.21.15.97:0/1257102101 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f58103d90 0x7f1f58199fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:34.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.867+0000 7f1f5e800640 1 -- 172.21.15.97:0/1257102101 >> 172.21.15.97:0/1257102101 conn(0x7f1f580ff300 msgr2=0x7f1f58108300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:34.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.867+0000 7f1f5e800640 1 -- 172.21.15.97:0/1257102101 shutdown_connections 2024-03-27T10:56:34.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:34.867+0000 7f1f5e800640 1 -- 172.21.15.97:0/1257102101 wait complete. 2024-03-27T10:56:34.863 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-27T10:56:35.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.082+0000 7f1eb9c00640 1 -- 172.21.15.97:0/1734300471 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1eb4102df0 msgr2=0x7f1eb4103270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:35.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.082+0000 7f1eb9c00640 1 --2- 172.21.15.97:0/1734300471 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1eb4102df0 0x7f1eb4103270 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f1ea0009890 tx=0x7f1ea002bb70 comp rx=0 tx=0).stop 2024-03-27T10:56:35.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.082+0000 7f1eb9c00640 1 -- 172.21.15.97:0/1734300471 shutdown_connections 2024-03-27T10:56:35.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.082+0000 7f1eb9c00640 1 --2- 172.21.15.97:0/1734300471 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1eb41037b0 0x7f1eb410a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:35.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.082+0000 7f1eb9c00640 1 --2- 172.21.15.97:0/1734300471 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1eb4102df0 0x7f1eb4103270 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:35.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.082+0000 7f1eb9c00640 1 --2- 172.21.15.97:0/1734300471 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1eb40ffe20 0x7f1eb4100240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:35.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.082+0000 7f1eb9c00640 1 -- 172.21.15.97:0/1734300471 >> 172.21.15.97:0/1734300471 conn(0x7f1eb40fb770 msgr2=0x7f1eb40fdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:35.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.082+0000 7f1eb9c00640 1 -- 172.21.15.97:0/1734300471 shutdown_connections 2024-03-27T10:56:35.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.082+0000 7f1eb9c00640 1 -- 172.21.15.97:0/1734300471 wait complete. 2024-03-27T10:56:35.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.083+0000 7f1eb9c00640 1 Processor -- start 2024-03-27T10:56:35.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.083+0000 7f1eb9c00640 1 -- start start 2024-03-27T10:56:35.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.083+0000 7f1eb9c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1eb40ffe20 0x7f1eb419a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:35.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.084+0000 7f1eb9c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1eb4102df0 0x7f1eb419a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:35.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.084+0000 7f1eb9c00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1eb41037b0 0x7f1eb41a15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:35.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.084+0000 7f1eb9c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb410d6b0 con 0x7f1eb40ffe20 2024-03-27T10:56:35.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.084+0000 7f1eb9c00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb410d530 con 0x7f1eb4102df0 2024-03-27T10:56:35.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.084+0000 7f1eb9c00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb410d830 con 0x7f1eb41037b0 2024-03-27T10:56:35.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.084+0000 7f1eb2a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1eb40ffe20 0x7f1eb419a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:35.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.084+0000 7f1eb2a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1eb40ffe20 0x7f1eb419a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53604/0 (socket says 172.21.15.97:53604) 2024-03-27T10:56:35.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.084+0000 7f1eb2a00640 1 -- 172.21.15.97:0/4093397862 learned_addr learned my addr 172.21.15.97:0/4093397862 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:56:35.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.084+0000 7f1eb2000640 1 --2- 172.21.15.97:0/4093397862 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1eb4102df0 0x7f1eb419a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:35.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.084+0000 7f1eb3400640 1 --2- 172.21.15.97:0/4093397862 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1eb41037b0 0x7f1eb41a15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:35.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.084+0000 7f1eb2a00640 1 -- 172.21.15.97:0/4093397862 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1eb41037b0 msgr2=0x7f1eb41a15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:35.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.084+0000 7f1eb2a00640 1 --2- 172.21.15.97:0/4093397862 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1eb41037b0 0x7f1eb41a15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:35.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.084+0000 7f1eb2a00640 1 -- 172.21.15.97:0/4093397862 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1eb4102df0 msgr2=0x7f1eb419a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:35.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.084+0000 7f1eb2a00640 1 --2- 172.21.15.97:0/4093397862 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1eb4102df0 0x7f1eb419a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:35.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.084+0000 7f1eb2a00640 1 -- 172.21.15.97:0/4093397862 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1eb41a1c30 con 0x7f1eb40ffe20 2024-03-27T10:56:35.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.084+0000 7f1eb2000640 1 --2- 172.21.15.97:0/4093397862 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1eb4102df0 0x7f1eb419a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T10:56:35.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.085+0000 7f1eb2a00640 1 --2- 172.21.15.97:0/4093397862 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1eb40ffe20 0x7f1eb419a010 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f1ea400e9a0 tx=0x7f1ea400ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:35.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.085+0000 7f1eab400640 1 -- 172.21.15.97:0/4093397862 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea400cd80 con 0x7f1eb40ffe20 2024-03-27T10:56:35.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.085+0000 7f1eab400640 1 -- 172.21.15.97:0/4093397862 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1ea40108e0 con 0x7f1eb40ffe20 2024-03-27T10:56:35.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.085+0000 7f1eb9c00640 1 -- 172.21.15.97:0/4093397862 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1eb41a1f20 con 0x7f1eb40ffe20 2024-03-27T10:56:35.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.085+0000 7f1eab400640 1 -- 172.21.15.97:0/4093397862 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea4018950 con 0x7f1eb40ffe20 2024-03-27T10:56:35.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.085+0000 7f1eb9c00640 1 -- 172.21.15.97:0/4093397862 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1eb41a2380 con 0x7f1eb40ffe20 2024-03-27T10:56:35.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.086+0000 7f1eb9c00640 1 -- 172.21.15.97:0/4093397862 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1eb41a2610 con 0x7f1eb40ffe20 2024-03-27T10:56:35.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.088+0000 7f1eab400640 1 -- 172.21.15.97:0/4093397862 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f1ea4010a90 con 0x7f1eb40ffe20 2024-03-27T10:56:35.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.090+0000 7f1eab400640 1 --2- 172.21.15.97:0/4093397862 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f1e980a3610 0x7f1e980a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:35.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.090+0000 7f1eab400640 1 -- 172.21.15.97:0/4093397862 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f1ea40d0180 con 0x7f1eb40ffe20 2024-03-27T10:56:35.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.090+0000 7f1eb2000640 1 --2- 172.21.15.97:0/4093397862 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f1e980a3610 0x7f1e980a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:35.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.091+0000 7f1eb2000640 1 --2- 172.21.15.97:0/4093397862 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f1e980a3610 0x7f1e980a5ad0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f1ea00057c0 tx=0x7f1ea0034980 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:35.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.094+0000 7f1eab400640 1 -- 172.21.15.97:0/4093397862 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f1ea4053cb0 con 0x7f1eb40ffe20 2024-03-27T10:56:35.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.295+0000 7f1eb9c00640 1 -- 172.21.15.97:0/4093397862 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1eb41a2880 con 0x7f1eb40ffe20 2024-03-27T10:56:35.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.295+0000 7f1eab400640 1 -- 172.21.15.97:0/4093397862 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) ==== 104+0+11 (secure 0 0 0) 0x7f1ea409cd00 con 0x7f1eb40ffe20 2024-03-27T10:56:35.267 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-03-27T10:56:35.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.298+0000 7f1eb9c00640 1 -- 172.21.15.97:0/4093397862 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f1e980a3610 msgr2=0x7f1e980a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:35.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.298+0000 7f1eb9c00640 1 --2- 172.21.15.97:0/4093397862 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f1e980a3610 0x7f1e980a5ad0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f1ea00057c0 tx=0x7f1ea0034980 comp rx=0 tx=0).stop 2024-03-27T10:56:35.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.298+0000 7f1eb9c00640 1 -- 172.21.15.97:0/4093397862 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1eb40ffe20 msgr2=0x7f1eb419a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:35.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.298+0000 7f1eb9c00640 1 --2- 172.21.15.97:0/4093397862 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1eb40ffe20 0x7f1eb419a010 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f1ea400e9a0 tx=0x7f1ea400ee60 comp rx=0 tx=0).stop 2024-03-27T10:56:35.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.298+0000 7f1eb9c00640 1 -- 172.21.15.97:0/4093397862 shutdown_connections 2024-03-27T10:56:35.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.298+0000 7f1eb9c00640 1 --2- 172.21.15.97:0/4093397862 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1eb41037b0 0x7f1eb41a15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:35.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.298+0000 7f1eb9c00640 1 --2- 172.21.15.97:0/4093397862 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f1e980a3610 0x7f1e980a5ad0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:35.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.298+0000 7f1eb9c00640 1 --2- 172.21.15.97:0/4093397862 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1eb4102df0 0x7f1eb419a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:35.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.298+0000 7f1eb9c00640 1 --2- 172.21.15.97:0/4093397862 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1eb40ffe20 0x7f1eb419a010 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:35.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.298+0000 7f1eb9c00640 1 -- 172.21.15.97:0/4093397862 >> 172.21.15.97:0/4093397862 conn(0x7f1eb40fb770 msgr2=0x7f1eb4108610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:35.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.299+0000 7f1eb9c00640 1 -- 172.21.15.97:0/4093397862 shutdown_connections 2024-03-27T10:56:35.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.299+0000 7f1eb9c00640 1 -- 172.21.15.97:0/4093397862 wait complete. 2024-03-27T10:56:35.288 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-27T10:56:35.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.522+0000 7f0c01a00640 1 -- 172.21.15.97:0/173514797 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0bfc1025e0 msgr2=0x7f0bfc1049d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:35.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.522+0000 7f0c01a00640 1 --2- 172.21.15.97:0/173514797 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0bfc1025e0 0x7f0bfc1049d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0bec009860 tx=0x7f0bec02bb70 comp rx=0 tx=0).stop 2024-03-27T10:56:35.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.522+0000 7f0c01a00640 1 -- 172.21.15.97:0/173514797 shutdown_connections 2024-03-27T10:56:35.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.522+0000 7f0c01a00640 1 --2- 172.21.15.97:0/173514797 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0bfc107840 0x7f0bfc109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:35.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.522+0000 7f0c01a00640 1 --2- 172.21.15.97:0/173514797 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0bfc104f10 0x7f0bfc107300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:35.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.522+0000 7f0c01a00640 1 --2- 172.21.15.97:0/173514797 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0bfc1025e0 0x7f0bfc1049d0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:35.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.522+0000 7f0c01a00640 1 -- 172.21.15.97:0/173514797 >> 172.21.15.97:0/173514797 conn(0x7f0bfc0fbff0 msgr2=0x7f0bfc0fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:35.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.522+0000 7f0c01a00640 1 -- 172.21.15.97:0/173514797 shutdown_connections 2024-03-27T10:56:35.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.522+0000 7f0c01a00640 1 -- 172.21.15.97:0/173514797 wait complete. 2024-03-27T10:56:35.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.523+0000 7f0c01a00640 1 Processor -- start 2024-03-27T10:56:35.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.523+0000 7f0c01a00640 1 -- start start 2024-03-27T10:56:35.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.524+0000 7f0c01a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0bfc1025e0 0x7f0bfc101f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:35.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.524+0000 7f0c01a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0bfc104f10 0x7f0bfc100590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:35.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.524+0000 7f0c01a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0bfc107840 0x7f0bfc100ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:35.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.524+0000 7f0c01a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0bfc10d690 con 0x7f0bfc1025e0 2024-03-27T10:56:35.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.524+0000 7f0c01a00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f0bfc10d510 con 0x7f0bfc104f10 2024-03-27T10:56:35.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.524+0000 7f0c01a00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f0bfc10d810 con 0x7f0bfc107840 2024-03-27T10:56:35.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.524+0000 7f0bfaa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0bfc1025e0 0x7f0bfc101f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:35.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.524+0000 7f0bfaa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0bfc1025e0 0x7f0bfc101f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53618/0 (socket says 172.21.15.97:53618) 2024-03-27T10:56:35.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.524+0000 7f0bfaa00640 1 -- 172.21.15.97:0/3300118794 learned_addr learned my addr 172.21.15.97:0/3300118794 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:56:35.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.524+0000 7f0bfb400640 1 --2- 172.21.15.97:0/3300118794 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0bfc107840 0x7f0bfc100ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:35.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.524+0000 7f0bfa000640 1 --2- 172.21.15.97:0/3300118794 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0bfc104f10 0x7f0bfc100590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:35.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.524+0000 7f0bfaa00640 1 -- 172.21.15.97:0/3300118794 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0bfc107840 msgr2=0x7f0bfc100ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:35.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.524+0000 7f0bfaa00640 1 --2- 172.21.15.97:0/3300118794 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0bfc107840 0x7f0bfc100ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:35.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.524+0000 7f0bfaa00640 1 -- 172.21.15.97:0/3300118794 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0bfc104f10 msgr2=0x7f0bfc100590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:35.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.524+0000 7f0bfaa00640 1 --2- 172.21.15.97:0/3300118794 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0bfc104f10 0x7f0bfc100590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:35.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.524+0000 7f0bfaa00640 1 -- 172.21.15.97:0/3300118794 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bfc101360 con 0x7f0bfc1025e0 2024-03-27T10:56:35.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.525+0000 7f0bfaa00640 1 --2- 172.21.15.97:0/3300118794 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0bfc1025e0 0x7f0bfc101f50 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f0bec005940 tx=0x7f0bec004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:35.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.525+0000 7f0bf3400640 1 -- 172.21.15.97:0/3300118794 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bec041020 con 0x7f0bfc1025e0 2024-03-27T10:56:35.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.525+0000 7f0bf3400640 1 -- 172.21.15.97:0/3300118794 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0bec02c9c0 con 0x7f0bfc1025e0 2024-03-27T10:56:35.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.525+0000 7f0bf3400640 1 -- 172.21.15.97:0/3300118794 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bec039040 con 0x7f0bfc1025e0 2024-03-27T10:56:35.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.525+0000 7f0c01a00640 1 -- 172.21.15.97:0/3300118794 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bfc1a1ec0 con 0x7f0bfc1025e0 2024-03-27T10:56:35.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.525+0000 7f0c01a00640 1 -- 172.21.15.97:0/3300118794 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bfc1a22d0 con 0x7f0bfc1025e0 2024-03-27T10:56:35.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.527+0000 7f0c01a00640 1 -- 172.21.15.97:0/3300118794 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bfc0fdae0 con 0x7f0bfc1025e0 2024-03-27T10:56:35.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.528+0000 7f0bf3400640 1 -- 172.21.15.97:0/3300118794 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f0bec044020 con 0x7f0bfc1025e0 2024-03-27T10:56:35.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.530+0000 7f0bf3400640 1 --2- 172.21.15.97:0/3300118794 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f0be00a3610 0x7f0be00a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:35.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.530+0000 7f0bf3400640 1 -- 172.21.15.97:0/3300118794 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f0bec0f1630 con 0x7f0bfc1025e0 2024-03-27T10:56:35.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.530+0000 7f0bfa000640 1 --2- 172.21.15.97:0/3300118794 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f0be00a3610 0x7f0be00a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:35.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.530+0000 7f0bfa000640 1 --2- 172.21.15.97:0/3300118794 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f0be00a3610 0x7f0be00a5ad0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc101cc0 tx=0x7f0be8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:35.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.534+0000 7f0bf3400640 1 -- 172.21.15.97:0/3300118794 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f0bec04b020 con 0x7f0bfc1025e0 2024-03-27T10:56:35.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:56:35 smithi097 ceph-mon[31859]: from='client.15441 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-27T10:56:35.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:56:35 smithi097 ceph-mon[31859]: pgmap v439: 97 pgs: 97 active+clean; 651 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.5 KiB/s wr, 13 op/s 2024-03-27T10:56:35.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:56:35 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/1446177462' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T10:56:35.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:56:35 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/1257102101' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-27T10:56:35.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:56:35 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:56:35.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:56:35 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:56:35.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:56:35 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/4093397862' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-27T10:56:35.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:56:35 smithi177 ceph-mon[35294]: from='client.15441 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-27T10:56:35.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:56:35 smithi177 ceph-mon[35294]: pgmap v439: 97 pgs: 97 active+clean; 651 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.5 KiB/s wr, 13 op/s 2024-03-27T10:56:35.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:56:35 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/1446177462' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T10:56:35.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:56:35 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/1257102101' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-27T10:56:35.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:56:35 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:56:35.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:56:35 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:56:35.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:56:35 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/4093397862' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-27T10:56:35.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.736+0000 7f0c01a00640 1 -- 172.21.15.97:0/3300118794 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f0bfc062890 con 0x7f0bfc1025e0 2024-03-27T10:56:35.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.737+0000 7f0bf3400640 1 -- 172.21.15.97:0/3300118794 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) ==== 100+0+11 (secure 0 0 0) 0x7f0bec0be1b0 con 0x7f0bfc1025e0 2024-03-27T10:56:35.709 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-03-27T10:56:35.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.740+0000 7f0c01a00640 1 -- 172.21.15.97:0/3300118794 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f0be00a3610 msgr2=0x7f0be00a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:35.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.740+0000 7f0c01a00640 1 --2- 172.21.15.97:0/3300118794 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f0be00a3610 0x7f0be00a5ad0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc101cc0 tx=0x7f0be8005f90 comp rx=0 tx=0).stop 2024-03-27T10:56:35.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.740+0000 7f0c01a00640 1 -- 172.21.15.97:0/3300118794 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0bfc1025e0 msgr2=0x7f0bfc101f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:35.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.740+0000 7f0c01a00640 1 --2- 172.21.15.97:0/3300118794 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0bfc1025e0 0x7f0bfc101f50 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f0bec005940 tx=0x7f0bec004270 comp rx=0 tx=0).stop 2024-03-27T10:56:35.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.740+0000 7f0c01a00640 1 -- 172.21.15.97:0/3300118794 shutdown_connections 2024-03-27T10:56:35.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.740+0000 7f0c01a00640 1 --2- 172.21.15.97:0/3300118794 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0bfc107840 0x7f0bfc100ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:35.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.740+0000 7f0c01a00640 1 --2- 172.21.15.97:0/3300118794 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f0be00a3610 0x7f0be00a5ad0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:35.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.740+0000 7f0c01a00640 1 --2- 172.21.15.97:0/3300118794 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0bfc104f10 0x7f0bfc100590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:35.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.740+0000 7f0c01a00640 1 --2- 172.21.15.97:0/3300118794 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0bfc1025e0 0x7f0bfc101f50 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:35.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.740+0000 7f0c01a00640 1 -- 172.21.15.97:0/3300118794 >> 172.21.15.97:0/3300118794 conn(0x7f0bfc0fbff0 msgr2=0x7f0bfc1030e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:35.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.740+0000 7f0c01a00640 1 -- 172.21.15.97:0/3300118794 shutdown_connections 2024-03-27T10:56:35.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.740+0000 7f0c01a00640 1 -- 172.21.15.97:0/3300118794 wait complete. 2024-03-27T10:56:35.729 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-27T10:56:35.767 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:56:35 smithi114 ceph-mon[35338]: from='client.15441 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-27T10:56:35.767 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:56:35 smithi114 ceph-mon[35338]: pgmap v439: 97 pgs: 97 active+clean; 651 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.5 KiB/s wr, 13 op/s 2024-03-27T10:56:35.767 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:56:35 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/1446177462' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T10:56:35.767 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:56:35 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/1257102101' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-27T10:56:35.767 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:56:35 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:56:35.767 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:56:35 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:56:35.767 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:56:35 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/4093397862' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-27T10:56:35.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.945+0000 7fb50c600640 1 -- 172.21.15.97:0/1335649217 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb504103d70 msgr2=0x7fb504104190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:35.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.945+0000 7fb50c600640 1 --2- 172.21.15.97:0/1335649217 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb504103d70 0x7fb504104190 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fb4f8009900 tx=0x7fb4f802bc30 comp rx=0 tx=0).stop 2024-03-27T10:56:35.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.945+0000 7fb50c600640 1 -- 172.21.15.97:0/1335649217 shutdown_connections 2024-03-27T10:56:35.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.945+0000 7fb50c600640 1 --2- 172.21.15.97:0/1335649217 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb504105920 0x7fb50410c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:35.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.945+0000 7fb50c600640 1 --2- 172.21.15.97:0/1335649217 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb504104f60 0x7fb5041053e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:35.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.945+0000 7fb50c600640 1 --2- 172.21.15.97:0/1335649217 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb504103d70 0x7fb504104190 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:35.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.945+0000 7fb50c600640 1 -- 172.21.15.97:0/1335649217 >> 172.21.15.97:0/1335649217 conn(0x7fb5040ff300 msgr2=0x7fb504101720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:35.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.945+0000 7fb50c600640 1 -- 172.21.15.97:0/1335649217 shutdown_connections 2024-03-27T10:56:35.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.945+0000 7fb50c600640 1 -- 172.21.15.97:0/1335649217 wait complete. 2024-03-27T10:56:35.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.946+0000 7fb50c600640 1 Processor -- start 2024-03-27T10:56:35.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.947+0000 7fb50c600640 1 -- start start 2024-03-27T10:56:35.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.947+0000 7fb50c600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb504103d70 0x7fb504199fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:35.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.947+0000 7fb50c600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb504104f60 0x7fb50419a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:35.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.947+0000 7fb50c600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb504105920 0x7fb5041a1550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:35.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.947+0000 7fb50c600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb50410f870 con 0x7fb504105920 2024-03-27T10:56:35.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.947+0000 7fb50c600640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb50410f6f0 con 0x7fb504104f60 2024-03-27T10:56:35.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.947+0000 7fb50c600640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb50410f9f0 con 0x7fb504103d70 2024-03-27T10:56:35.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.947+0000 7fb50a600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb504105920 0x7fb5041a1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:35.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.947+0000 7fb50a600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb504105920 0x7fb5041a1550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53630/0 (socket says 172.21.15.97:53630) 2024-03-27T10:56:35.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.947+0000 7fb509200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb504104f60 0x7fb50419a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:35.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.947+0000 7fb50a600640 1 -- 172.21.15.97:0/3043407208 learned_addr learned my addr 172.21.15.97:0/3043407208 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:56:35.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.947+0000 7fb509c00640 1 --2- 172.21.15.97:0/3043407208 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb504103d70 0x7fb504199fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:35.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.948+0000 7fb50a600640 1 -- 172.21.15.97:0/3043407208 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb504103d70 msgr2=0x7fb504199fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:35.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.948+0000 7fb50a600640 1 --2- 172.21.15.97:0/3043407208 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb504103d70 0x7fb504199fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:35.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.948+0000 7fb50a600640 1 -- 172.21.15.97:0/3043407208 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb504104f60 msgr2=0x7fb50419a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:35.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.948+0000 7fb50a600640 1 --2- 172.21.15.97:0/3043407208 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb504104f60 0x7fb50419a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:35.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.948+0000 7fb50a600640 1 -- 172.21.15.97:0/3043407208 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5041a1bc0 con 0x7fb504105920 2024-03-27T10:56:35.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.948+0000 7fb509200640 1 --2- 172.21.15.97:0/3043407208 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb504104f60 0x7fb50419a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T10:56:35.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.948+0000 7fb50a600640 1 --2- 172.21.15.97:0/3043407208 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb504105920 0x7fb5041a1550 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc00eb10 tx=0x7fb4fc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:35.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.948+0000 7fb502000640 1 -- 172.21.15.97:0/3043407208 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc004b20 con 0x7fb504105920 2024-03-27T10:56:35.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.948+0000 7fb502000640 1 -- 172.21.15.97:0/3043407208 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb4fc004cc0 con 0x7fb504105920 2024-03-27T10:56:35.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.948+0000 7fb502000640 1 -- 172.21.15.97:0/3043407208 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc010690 con 0x7fb504105920 2024-03-27T10:56:35.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.948+0000 7fb50c600640 1 -- 172.21.15.97:0/3043407208 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5041a1eb0 con 0x7fb504105920 2024-03-27T10:56:35.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.948+0000 7fb50c600640 1 -- 172.21.15.97:0/3043407208 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5041a2320 con 0x7fb504105920 2024-03-27T10:56:35.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.949+0000 7fb50c600640 1 -- 172.21.15.97:0/3043407208 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb50410b1b0 con 0x7fb504105920 2024-03-27T10:56:35.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.952+0000 7fb502000640 1 -- 172.21.15.97:0/3043407208 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7fb4fc005000 con 0x7fb504105920 2024-03-27T10:56:35.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.954+0000 7fb502000640 1 --2- 172.21.15.97:0/3043407208 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fb4ec0a3610 0x7fb4ec0a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:35.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.954+0000 7fb502000640 1 -- 172.21.15.97:0/3043407208 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fb4fc0d0410 con 0x7fb504105920 2024-03-27T10:56:35.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.954+0000 7fb509c00640 1 --2- 172.21.15.97:0/3043407208 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fb4ec0a3610 0x7fb4ec0a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:35.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.955+0000 7fb509c00640 1 --2- 172.21.15.97:0/3043407208 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fb4ec0a3610 0x7fb4ec0a5ad0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb4f8005ab0 tx=0x7fb4f8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:35.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:35.957+0000 7fb502000640 1 -- 172.21.15.97:0/3043407208 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7fb4fc053cb0 con 0x7fb504105920 2024-03-27T10:56:36.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.209+0000 7fb50c600640 1 -- 172.21.15.97:0/3043407208 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb50402d5c0 con 0x7fb504105920 2024-03-27T10:56:36.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.210+0000 7fb502000640 1 -- 172.21.15.97:0/3043407208 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11944 (secure 0 0 0) 0x7fb4fc085e20 con 0x7fb504105920 2024-03-27T10:56:36.182 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T10:56:36.182 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1009423904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1009423904},{"type":"v1","addr":"172.21.15.97:6835","nonce":1009423904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3251173433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3251173433},{"type":"v1","addr":"172.21.15.97:6837","nonce":3251173433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-27T10:54:04.510178+0000","modified":"2024-03-27T10:56:28.179534+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34508,"mds_1":34478,"mds_2":34445,"mds_3":24454,"mds_4":24400},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14646":{"gid":14646,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6839/1057596821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1057596821},{"type":"v1","addr":"172.21.15.97:6839","nonce":1057596821}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14658":{"gid":14658,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6841/1672464856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6840","nonce":1672464856},{"type":"v1","addr":"172.21.15.97:6841","nonce":1672464856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.177:6833/3787981632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3787981632},{"type":"v1","addr":"172.21.15.177:6833","nonce":3787981632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.177:6835/3349112132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":3349112132},{"type":"v1","addr":"172.21.15.177:6835","nonce":3349112132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6837/3244432640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":3244432640},{"type":"v1","addr":"172.21.15.177:6837","nonce":3244432640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6839/3340454605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":3340454605},{"type":"v1","addr":"172.21.15.177:6839","nonce":3340454605}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34415":{"gid":34415,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6833/4052517902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":4052517902},{"type":"v1","addr":"172.21.15.114:6833","nonce":4052517902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34445":{"gid":34445,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.114:6835/3581918222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3581918222},{"type":"v1","addr":"172.21.15.114:6835","nonce":3581918222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34478":{"gid":34478,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.114:6837/616775588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":616775588},{"type":"v1","addr":"172.21.15.114:6837","nonce":616775588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34508":{"gid":34508,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.114:6839/3937193784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":3937193784},{"type":"v1","addr":"172.21.15.114:6839","nonce":3937193784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34508,"qdb_cluster":[34478,24454,24400,34445,34508]},"id":1}]} 2024-03-27T10:56:36.182 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 28 2024-03-27T10:56:36.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.213+0000 7fb50c600640 1 -- 172.21.15.97:0/3043407208 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fb4ec0a3610 msgr2=0x7fb4ec0a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:36.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.213+0000 7fb50c600640 1 --2- 172.21.15.97:0/3043407208 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fb4ec0a3610 0x7fb4ec0a5ad0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb4f8005ab0 tx=0x7fb4f8005fd0 comp rx=0 tx=0).stop 2024-03-27T10:56:36.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.213+0000 7fb50c600640 1 -- 172.21.15.97:0/3043407208 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb504105920 msgr2=0x7fb5041a1550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:36.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.213+0000 7fb50c600640 1 --2- 172.21.15.97:0/3043407208 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb504105920 0x7fb5041a1550 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc00eb10 tx=0x7fb4fc00c2d0 comp rx=0 tx=0).stop 2024-03-27T10:56:36.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.214+0000 7fb50c600640 1 -- 172.21.15.97:0/3043407208 shutdown_connections 2024-03-27T10:56:36.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.214+0000 7fb50c600640 1 --2- 172.21.15.97:0/3043407208 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb504105920 0x7fb5041a1550 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:36.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.214+0000 7fb50c600640 1 --2- 172.21.15.97:0/3043407208 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fb4ec0a3610 0x7fb4ec0a5ad0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:36.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.214+0000 7fb50c600640 1 --2- 172.21.15.97:0/3043407208 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb504104f60 0x7fb50419a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:36.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.214+0000 7fb50c600640 1 --2- 172.21.15.97:0/3043407208 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb504103d70 0x7fb504199fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:36.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.214+0000 7fb50c600640 1 -- 172.21.15.97:0/3043407208 >> 172.21.15.97:0/3043407208 conn(0x7fb5040ff300 msgr2=0x7fb504108380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:36.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.214+0000 7fb50c600640 1 -- 172.21.15.97:0/3043407208 shutdown_connections 2024-03-27T10:56:36.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.214+0000 7fb50c600640 1 -- 172.21.15.97:0/3043407208 wait complete. 2024-03-27T10:56:36.204 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-27T10:56:36.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.439+0000 7fedeb400640 1 -- 172.21.15.97:0/1577433621 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fede4104f80 msgr2=0x7fede4105400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:36.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.439+0000 7fedeb400640 1 --2- 172.21.15.97:0/1577433621 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fede4104f80 0x7fede4105400 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fedd0009900 tx=0x7fedd002bc30 comp rx=0 tx=0).stop 2024-03-27T10:56:36.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.440+0000 7fedeb400640 1 -- 172.21.15.97:0/1577433621 shutdown_connections 2024-03-27T10:56:36.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.440+0000 7fedeb400640 1 --2- 172.21.15.97:0/1577433621 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fede4105940 0x7fede410c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:36.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.440+0000 7fedeb400640 1 --2- 172.21.15.97:0/1577433621 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fede4104f80 0x7fede4105400 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:36.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.440+0000 7fedeb400640 1 --2- 172.21.15.97:0/1577433621 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fede4103d90 0x7fede41041b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:36.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.440+0000 7fedeb400640 1 -- 172.21.15.97:0/1577433621 >> 172.21.15.97:0/1577433621 conn(0x7fede40ff300 msgr2=0x7fede4101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:36.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.440+0000 7fedeb400640 1 -- 172.21.15.97:0/1577433621 shutdown_connections 2024-03-27T10:56:36.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.440+0000 7fedeb400640 1 -- 172.21.15.97:0/1577433621 wait complete. 2024-03-27T10:56:36.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.441+0000 7fedeb400640 1 Processor -- start 2024-03-27T10:56:36.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.441+0000 7fedeb400640 1 -- start start 2024-03-27T10:56:36.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.441+0000 7fedeb400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fede4103d90 0x7fede4199f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:36.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.441+0000 7fedeb400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fede4104f80 0x7fede419a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:36.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.441+0000 7fedeb400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fede4105940 0x7fede41a1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:36.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.441+0000 7fedeb400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fede410f910 con 0x7fede4103d90 2024-03-27T10:56:36.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.441+0000 7fedeb400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fede410f790 con 0x7fede4104f80 2024-03-27T10:56:36.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.441+0000 7fedeb400640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fede410fa90 con 0x7fede4105940 2024-03-27T10:56:36.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.441+0000 7fede8a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fede4103d90 0x7fede4199f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:36.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.441+0000 7fede8a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fede4103d90 0x7fede4199f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53638/0 (socket says 172.21.15.97:53638) 2024-03-27T10:56:36.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.441+0000 7fede8a00640 1 -- 172.21.15.97:0/2092636601 learned_addr learned my addr 172.21.15.97:0/2092636601 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:56:36.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.441+0000 7fede3e00640 1 --2- 172.21.15.97:0/2092636601 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fede4104f80 0x7fede419a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:36.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.442+0000 7fede9400640 1 --2- 172.21.15.97:0/2092636601 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fede4105940 0x7fede41a1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:36.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.442+0000 7fede8a00640 1 -- 172.21.15.97:0/2092636601 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fede4105940 msgr2=0x7fede41a1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:36.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.442+0000 7fede8a00640 1 --2- 172.21.15.97:0/2092636601 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fede4105940 0x7fede41a1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:36.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.442+0000 7fede8a00640 1 -- 172.21.15.97:0/2092636601 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fede4104f80 msgr2=0x7fede419a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:36.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.442+0000 7fede8a00640 1 --2- 172.21.15.97:0/2092636601 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fede4104f80 0x7fede419a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:36.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.442+0000 7fede8a00640 1 -- 172.21.15.97:0/2092636601 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fede41a1bb0 con 0x7fede4103d90 2024-03-27T10:56:36.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.442+0000 7fede8a00640 1 --2- 172.21.15.97:0/2092636601 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fede4103d90 0x7fede4199f90 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fedd400d950 tx=0x7fedd400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:36.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.442+0000 7fede1600640 1 -- 172.21.15.97:0/2092636601 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd4017070 con 0x7fede4103d90 2024-03-27T10:56:36.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.442+0000 7fede1600640 1 -- 172.21.15.97:0/2092636601 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fedd4009c30 con 0x7fede4103d90 2024-03-27T10:56:36.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.442+0000 7fede1600640 1 -- 172.21.15.97:0/2092636601 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd400f620 con 0x7fede4103d90 2024-03-27T10:56:36.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.442+0000 7fedeb400640 1 -- 172.21.15.97:0/2092636601 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fede41a1ea0 con 0x7fede4103d90 2024-03-27T10:56:36.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.442+0000 7fedeb400640 1 -- 172.21.15.97:0/2092636601 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fede41a22b0 con 0x7fede4103d90 2024-03-27T10:56:36.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.443+0000 7fedeb400640 1 -- 172.21.15.97:0/2092636601 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fede410b0b0 con 0x7fede4103d90 2024-03-27T10:56:36.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.446+0000 7fede1600640 1 -- 172.21.15.97:0/2092636601 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7fedd401a070 con 0x7fede4103d90 2024-03-27T10:56:36.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.447+0000 7fede1600640 1 --2- 172.21.15.97:0/2092636601 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fedc80a3610 0x7fedc80a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:36.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.447+0000 7fede1600640 1 -- 172.21.15.97:0/2092636601 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fedd40cf820 con 0x7fede4103d90 2024-03-27T10:56:36.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.447+0000 7fede3e00640 1 --2- 172.21.15.97:0/2092636601 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fedc80a3610 0x7fedc80a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:36.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.448+0000 7fede3e00640 1 --2- 172.21.15.97:0/2092636601 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fedc80a3610 0x7fedc80a5ad0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fedd0006010 tx=0x7fedd0000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:36.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.449+0000 7fede1600640 1 -- 172.21.15.97:0/2092636601 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7fedd4009de0 con 0x7fede4103d90 2024-03-27T10:56:36.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.650+0000 7fedeb400640 1 -- 172.21.15.97:0/2092636601 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fede402d4f0 con 0x7fede4103d90 2024-03-27T10:56:36.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.653+0000 7fede1600640 1 -- 172.21.15.97:0/2092636601 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+22644 (secure 0 0 0) 0x7fedd409c3a0 con 0x7fede4103d90 2024-03-27T10:56:36.624 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T10:56:36.625 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":92,"fsid":"adca88c4-ec26-11ee-95d0-87774f69a715","created":"2024-03-27T10:43:24.680694+0000","modified":"2024-03-27T10:54:41.310812+0000","last_up_change":"2024-03-27T10:53:00.552513+0000","last_in_change":"2024-03-27T10:52:36.302986+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-27T10:47:54.052810+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-27T10:54:02.845060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-27T10:54:03.474151+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b634e5f1-6118-4e2f-95e9-830db390f8d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1746812080},{"type":"v1","addr":"172.21.15.97:6803","nonce":1746812080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1746812080},{"type":"v1","addr":"172.21.15.97:6805","nonce":1746812080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":1746812080},{"type":"v1","addr":"172.21.15.97:6809","nonce":1746812080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1746812080},{"type":"v1","addr":"172.21.15.97:6807","nonce":1746812080}]},"public_addr":"172.21.15.97:6803/1746812080","cluster_addr":"172.21.15.97:6805/1746812080","heartbeat_back_addr":"172.21.15.97:6809/1746812080","heartbeat_front_addr":"172.21.15.97:6807/1746812080","state":["exists","up"]},{"osd":1,"uuid":"441e2770-3284-4388-b0de-c56eb8a373d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":3858821264},{"type":"v1","addr":"172.21.15.114:6801","nonce":3858821264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":3858821264},{"type":"v1","addr":"172.21.15.114:6803","nonce":3858821264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":3858821264},{"type":"v1","addr":"172.21.15.114:6807","nonce":3858821264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":3858821264},{"type":"v1","addr":"172.21.15.114:6805","nonce":3858821264}]},"public_addr":"172.21.15.114:6801/3858821264","cluster_addr":"172.21.15.114:6803/3858821264","heartbeat_back_addr":"172.21.15.114:6807/3858821264","heartbeat_front_addr":"172.21.15.114:6805/3858821264","state":["exists","up"]},{"osd":2,"uuid":"00108864-1aa6-48d6-8398-e8eb79fda123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":124240658},{"type":"v1","addr":"172.21.15.177:6801","nonce":124240658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":124240658},{"type":"v1","addr":"172.21.15.177:6803","nonce":124240658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":124240658},{"type":"v1","addr":"172.21.15.177:6807","nonce":124240658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":124240658},{"type":"v1","addr":"172.21.15.177:6805","nonce":124240658}]},"public_addr":"172.21.15.177:6801/124240658","cluster_addr":"172.21.15.177:6803/124240658","heartbeat_back_addr":"172.21.15.177:6807/124240658","heartbeat_front_addr":"172.21.15.177:6805/124240658","state":["exists","up"]},{"osd":3,"uuid":"1860b690-e2dc-4977-88d7-d1aacf7022ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2478179311},{"type":"v1","addr":"172.21.15.97:6811","nonce":2478179311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2478179311},{"type":"v1","addr":"172.21.15.97:6813","nonce":2478179311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":2478179311},{"type":"v1","addr":"172.21.15.97:6817","nonce":2478179311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2478179311},{"type":"v1","addr":"172.21.15.97:6815","nonce":2478179311}]},"public_addr":"172.21.15.97:6811/2478179311","cluster_addr":"172.21.15.97:6813/2478179311","heartbeat_back_addr":"172.21.15.97:6817/2478179311","heartbeat_front_addr":"172.21.15.97:6815/2478179311","state":["exists","up"]},{"osd":4,"uuid":"9245a3a4-7d49-4fdb-9d55-112e43177050","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":75755945},{"type":"v1","addr":"172.21.15.114:6809","nonce":75755945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":75755945},{"type":"v1","addr":"172.21.15.114:6811","nonce":75755945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":75755945},{"type":"v1","addr":"172.21.15.114:6815","nonce":75755945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":75755945},{"type":"v1","addr":"172.21.15.114:6813","nonce":75755945}]},"public_addr":"172.21.15.114:6809/75755945","cluster_addr":"172.21.15.114:6811/75755945","heartbeat_back_addr":"172.21.15.114:6815/75755945","heartbeat_front_addr":"172.21.15.114:6813/75755945","state":["exists","up"]},{"osd":5,"uuid":"51922f99-e8d3-461b-8615-e9bfdcbf5ed3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":1918493790},{"type":"v1","addr":"172.21.15.177:6809","nonce":1918493790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":1918493790},{"type":"v1","addr":"172.21.15.177:6811","nonce":1918493790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":1918493790},{"type":"v1","addr":"172.21.15.177:6815","nonce":1918493790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":1918493790},{"type":"v1","addr":"172.21.15.177:6813","nonce":1918493790}]},"public_addr":"172.21.15.177:6809/1918493790","cluster_addr":"172.21.15.177:6811/1918493790","heartbeat_back_addr":"172.21.15.177:6815/1918493790","heartbeat_front_addr":"172.21.15.177:6813/1918493790","state":["exists","up"]},{"osd":6,"uuid":"06dffb32-05eb-43b3-852e-915c5cc7f25a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2892389563},{"type":"v1","addr":"172.21.15.97:6819","nonce":2892389563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2892389563},{"type":"v1","addr":"172.21.15.97:6821","nonce":2892389563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2892389563},{"type":"v1","addr":"172.21.15.97:6825","nonce":2892389563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2892389563},{"type":"v1","addr":"172.21.15.97:6823","nonce":2892389563}]},"public_addr":"172.21.15.97:6819/2892389563","cluster_addr":"172.21.15.97:6821/2892389563","heartbeat_back_addr":"172.21.15.97:6825/2892389563","heartbeat_front_addr":"172.21.15.97:6823/2892389563","state":["exists","up"]},{"osd":7,"uuid":"e68e3519-9c1b-43c5-8e06-5385248b19df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":2445491153},{"type":"v1","addr":"172.21.15.114:6817","nonce":2445491153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":2445491153},{"type":"v1","addr":"172.21.15.114:6819","nonce":2445491153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2445491153},{"type":"v1","addr":"172.21.15.114:6823","nonce":2445491153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":2445491153},{"type":"v1","addr":"172.21.15.114:6821","nonce":2445491153}]},"public_addr":"172.21.15.114:6817/2445491153","cluster_addr":"172.21.15.114:6819/2445491153","heartbeat_back_addr":"172.21.15.114:6823/2445491153","heartbeat_front_addr":"172.21.15.114:6821/2445491153","state":["exists","up"]},{"osd":8,"uuid":"31ac592e-1349-49ee-9af9-8ad85d682978","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":2051134191},{"type":"v1","addr":"172.21.15.177:6817","nonce":2051134191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":2051134191},{"type":"v1","addr":"172.21.15.177:6819","nonce":2051134191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":2051134191},{"type":"v1","addr":"172.21.15.177:6823","nonce":2051134191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":2051134191},{"type":"v1","addr":"172.21.15.177:6821","nonce":2051134191}]},"public_addr":"172.21.15.177:6817/2051134191","cluster_addr":"172.21.15.177:6819/2051134191","heartbeat_back_addr":"172.21.15.177:6823/2051134191","heartbeat_front_addr":"172.21.15.177:6821/2051134191","state":["exists","up"]},{"osd":9,"uuid":"0171086a-dd41-4eb4-a389-16f0c46a64db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":1578881271},{"type":"v1","addr":"172.21.15.97:6827","nonce":1578881271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1578881271},{"type":"v1","addr":"172.21.15.97:6829","nonce":1578881271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1578881271},{"type":"v1","addr":"172.21.15.97:6833","nonce":1578881271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1578881271},{"type":"v1","addr":"172.21.15.97:6831","nonce":1578881271}]},"public_addr":"172.21.15.97:6827/1578881271","cluster_addr":"172.21.15.97:6829/1578881271","heartbeat_back_addr":"172.21.15.97:6833/1578881271","heartbeat_front_addr":"172.21.15.97:6831/1578881271","state":["exists","up"]},{"osd":10,"uuid":"64754e31-b124-4601-ad98-2f70b1dd5359","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":2893111461},{"type":"v1","addr":"172.21.15.114:6825","nonce":2893111461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":2893111461},{"type":"v1","addr":"172.21.15.114:6827","nonce":2893111461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2893111461},{"type":"v1","addr":"172.21.15.114:6831","nonce":2893111461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":2893111461},{"type":"v1","addr":"172.21.15.114:6829","nonce":2893111461}]},"public_addr":"172.21.15.114:6825/2893111461","cluster_addr":"172.21.15.114:6827/2893111461","heartbeat_back_addr":"172.21.15.114:6831/2893111461","heartbeat_front_addr":"172.21.15.114:6829/2893111461","state":["exists","up"]},{"osd":11,"uuid":"9ffc752c-adf7-4037-89d3-535787d7475d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":85,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":2333030391},{"type":"v1","addr":"172.21.15.177:6825","nonce":2333030391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":2333030391},{"type":"v1","addr":"172.21.15.177:6827","nonce":2333030391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":2333030391},{"type":"v1","addr":"172.21.15.177:6831","nonce":2333030391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":2333030391},{"type":"v1","addr":"172.21.15.177:6829","nonce":2333030391}]},"public_addr":"172.21.15.177:6825/2333030391","cluster_addr":"172.21.15.177:6827/2333030391","heartbeat_back_addr":"172.21.15.177:6831/2333030391","heartbeat_front_addr":"172.21.15.177:6829/2333030391","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:46:45.143519+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:47:18.656947+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:47:51.316345+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:48:24.575512+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:48:59.113555+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:49:32.103865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:50:06.615539+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:50:41.578341+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:51:15.254011+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:51:49.099165+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:52:24.544821+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:52:58.838484+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.28","mappings":[{"from":3,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/359646":"2024-03-28T10:44:39.889550+0000","172.21.15.97:6801/98682155":"2024-03-28T10:44:06.019075+0000","172.21.15.97:6801/1685921853":"2024-03-28T10:44:39.889550+0000","172.21.15.97:6800/98682155":"2024-03-28T10:44:06.019075+0000","172.21.15.97:0/107690680":"2024-03-28T10:44:06.019075+0000","172.21.15.97:6800/1685921853":"2024-03-28T10:44:39.889550+0000","172.21.15.97:0/437043413":"2024-03-28T10:44:39.889550+0000","172.21.15.97:0/2319706992":"2024-03-28T10:44:39.889550+0000","172.21.15.97:0/3911532654":"2024-03-28T10:44:06.019075+0000","172.21.15.97:0/3347321845":"2024-03-28T10:44:06.019075+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-27T10:56:36.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.656+0000 7fedeb400640 1 -- 172.21.15.97:0/2092636601 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fedc80a3610 msgr2=0x7fedc80a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:36.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.656+0000 7fedeb400640 1 --2- 172.21.15.97:0/2092636601 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fedc80a3610 0x7fedc80a5ad0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fedd0006010 tx=0x7fedd0000bc0 comp rx=0 tx=0).stop 2024-03-27T10:56:36.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.656+0000 7fedeb400640 1 -- 172.21.15.97:0/2092636601 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fede4103d90 msgr2=0x7fede4199f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:36.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.656+0000 7fedeb400640 1 --2- 172.21.15.97:0/2092636601 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fede4103d90 0x7fede4199f90 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fedd400d950 tx=0x7fedd400de10 comp rx=0 tx=0).stop 2024-03-27T10:56:36.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.656+0000 7fedeb400640 1 -- 172.21.15.97:0/2092636601 shutdown_connections 2024-03-27T10:56:36.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.656+0000 7fedeb400640 1 --2- 172.21.15.97:0/2092636601 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fede4105940 0x7fede41a1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:36.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.656+0000 7fedeb400640 1 --2- 172.21.15.97:0/2092636601 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fedc80a3610 0x7fedc80a5ad0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:36.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.656+0000 7fedeb400640 1 --2- 172.21.15.97:0/2092636601 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fede4104f80 0x7fede419a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:36.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.656+0000 7fedeb400640 1 --2- 172.21.15.97:0/2092636601 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fede4103d90 0x7fede4199f90 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:36.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.656+0000 7fedeb400640 1 -- 172.21.15.97:0/2092636601 >> 172.21.15.97:0/2092636601 conn(0x7fede40ff300 msgr2=0x7fede4108280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:36.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.656+0000 7fedeb400640 1 -- 172.21.15.97:0/2092636601 shutdown_connections 2024-03-27T10:56:36.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.656+0000 7fedeb400640 1 -- 172.21.15.97:0/2092636601 wait complete. 2024-03-27T10:56:36.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:56:36 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/3300118794' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-27T10:56:36.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:56:36 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:56:36.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:56:36 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:56:36.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:56:36 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/3043407208' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T10:56:36.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:56:36 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:56:36.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:56:36 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:56:36.645 DEBUG:tasks.fwd_scrub:Yielding 2024-03-27T10:56:36.645 INFO:teuthology.run_tasks:Running task background_exec... 2024-03-27T10:56:36.656 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi097.front.sepia.ceph.com 2024-03-27T10:56:36.656 DEBUG:teuthology.orchestra.run.smithi097:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-03-27T10:56:36.656 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-03-27T10:56:36.657 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-27T10:56:36.657 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-27T10:56:36.657 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-27T10:56:36.661 INFO:teuthology.run_tasks:Running task workunit... 2024-03-27T10:56:36.674 INFO:tasks.workunit:Pulling workunits from ref 155268c4e432a12433aa833f174f9fe3b1016ae0 2024-03-27T10:56:36.674 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-03-27T10:56:36.674 INFO:tasks.workunit:timeout=3h 2024-03-27T10:56:36.674 INFO:tasks.workunit:cleanup=True 2024-03-27T10:56:36.674 DEBUG:teuthology.orchestra.run.smithi097:> stat -- /home/ubuntu/cephtest/mnt.0 2024-03-27T10:56:36.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:56:36 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/3300118794' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-27T10:56:36.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:56:36 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:56:36.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:56:36 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:56:36.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:56:36 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/3043407208' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T10:56:36.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:56:36 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:56:36.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:56:36 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:56:36.714 INFO:teuthology.orchestra.run.smithi097.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-03-27T10:56:36.714 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-03-27T10:56:36.714 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 76h/118d Inode: 1 Links: 2 2024-03-27T10:56:36.714 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-27T10:56:36.715 INFO:teuthology.orchestra.run.smithi097.stdout:Context: system_u:object_r:fusefs_t:s0 2024-03-27T10:56:36.715 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2024-03-27 10:54:11.129212990 +0000 2024-03-27T10:56:36.715 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2024-03-27 10:54:11.129212990 +0000 2024-03-27T10:56:36.715 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2024-03-27 10:56:27.696141447 +0000 2024-03-27T10:56:36.715 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: - 2024-03-27T10:56:36.715 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-03-27T10:56:36.715 DEBUG:teuthology.orchestra.run.smithi097:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-03-27T10:56:36.766 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:56:36 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/3300118794' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-27T10:56:36.766 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:56:36 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:56:36.767 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:56:36 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:56:36.767 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:56:36 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/3043407208' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T10:56:36.767 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:56:36 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:56:36.767 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:56:36 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:56:36.840 DEBUG:teuthology.orchestra.run.smithi097:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 155268c4e432a12433aa833f174f9fe3b1016ae0 2024-03-27T10:56:36.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.874+0000 7fe703e00640 1 -- 172.21.15.97:0/4261080936 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe704102cc0 msgr2=0x7fe704103140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:36.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.874+0000 7fe703e00640 1 --2- 172.21.15.97:0/4261080936 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe704102cc0 0x7fe704103140 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fe6f0009930 tx=0x7fe6f002bc30 comp rx=0 tx=0).stop 2024-03-27T10:56:36.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.874+0000 7fe703e00640 1 -- 172.21.15.97:0/4261080936 shutdown_connections 2024-03-27T10:56:36.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.874+0000 7fe703e00640 1 --2- 172.21.15.97:0/4261080936 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe704103680 0x7fe704109f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:36.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.874+0000 7fe703e00640 1 --2- 172.21.15.97:0/4261080936 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe704102cc0 0x7fe704103140 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:36.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.874+0000 7fe703e00640 1 --2- 172.21.15.97:0/4261080936 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe7040ffcf0 0x7fe704100110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:36.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.874+0000 7fe703e00640 1 -- 172.21.15.97:0/4261080936 >> 172.21.15.97:0/4261080936 conn(0x7fe7040fb640 msgr2=0x7fe7040fda60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:36.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.874+0000 7fe703e00640 1 -- 172.21.15.97:0/4261080936 shutdown_connections 2024-03-27T10:56:36.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.874+0000 7fe703e00640 1 -- 172.21.15.97:0/4261080936 wait complete. 2024-03-27T10:56:36.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.875+0000 7fe703e00640 1 Processor -- start 2024-03-27T10:56:36.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.875+0000 7fe703e00640 1 -- start start 2024-03-27T10:56:36.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.875+0000 7fe703e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7040ffcf0 0x7fe704199ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:36.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.875+0000 7fe703e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe704102cc0 0x7fe70419a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:36.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.875+0000 7fe703e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe704103680 0x7fe7041a1490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:36.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.875+0000 7fe703e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe70410d5d0 con 0x7fe7040ffcf0 2024-03-27T10:56:36.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.875+0000 7fe703e00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe70410d450 con 0x7fe704103680 2024-03-27T10:56:36.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.875+0000 7fe703e00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe70410d750 con 0x7fe704102cc0 2024-03-27T10:56:36.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.875+0000 7fe702a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7040ffcf0 0x7fe704199ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:36.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.876+0000 7fe702a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7040ffcf0 0x7fe704199ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53652/0 (socket says 172.21.15.97:53652) 2024-03-27T10:56:36.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.876+0000 7fe702a00640 1 -- 172.21.15.97:0/3222409130 learned_addr learned my addr 172.21.15.97:0/3222409130 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:56:36.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.876+0000 7fe703400640 1 --2- 172.21.15.97:0/3222409130 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe704103680 0x7fe7041a1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:36.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.876+0000 7fe702000640 1 --2- 172.21.15.97:0/3222409130 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe704102cc0 0x7fe70419a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:36.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.876+0000 7fe702a00640 1 -- 172.21.15.97:0/3222409130 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe704102cc0 msgr2=0x7fe70419a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:36.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.876+0000 7fe702a00640 1 --2- 172.21.15.97:0/3222409130 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe704102cc0 0x7fe70419a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:36.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.876+0000 7fe702a00640 1 -- 172.21.15.97:0/3222409130 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe704103680 msgr2=0x7fe7041a1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:36.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.876+0000 7fe702a00640 1 --2- 172.21.15.97:0/3222409130 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe704103680 0x7fe7041a1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:36.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.876+0000 7fe702a00640 1 -- 172.21.15.97:0/3222409130 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7041a1b00 con 0x7fe7040ffcf0 2024-03-27T10:56:36.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.876+0000 7fe702000640 1 --2- 172.21.15.97:0/3222409130 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe704102cc0 0x7fe70419a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:56:36.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.876+0000 7fe702a00640 1 --2- 172.21.15.97:0/3222409130 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7040ffcf0 0x7fe704199ee0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fe6f400e9c0 tx=0x7fe6f400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:36.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.876+0000 7fe6fb400640 1 -- 172.21.15.97:0/3222409130 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f400cd80 con 0x7fe7040ffcf0 2024-03-27T10:56:36.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.877+0000 7fe6fb400640 1 -- 172.21.15.97:0/3222409130 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe6f40035f0 con 0x7fe7040ffcf0 2024-03-27T10:56:36.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.877+0000 7fe6fb400640 1 -- 172.21.15.97:0/3222409130 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f4010690 con 0x7fe7040ffcf0 2024-03-27T10:56:36.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.877+0000 7fe703e00640 1 -- 172.21.15.97:0/3222409130 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7041a1df0 con 0x7fe7040ffcf0 2024-03-27T10:56:36.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.877+0000 7fe703e00640 1 -- 172.21.15.97:0/3222409130 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7041a2200 con 0x7fe7040ffcf0 2024-03-27T10:56:36.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.880+0000 7fe6fb400640 1 -- 172.21.15.97:0/3222409130 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7fe6f40037a0 con 0x7fe7040ffcf0 2024-03-27T10:56:36.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.880+0000 7fe70c861740 1 Processor -- start 2024-03-27T10:56:36.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.880+0000 7fe70c861740 1 -- start start 2024-03-27T10:56:36.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.881+0000 7fe70c861740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560ff5b5b0c0 0x560ff5b5b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:36.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.881+0000 7fe70c861740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560ff5b5c290 0x560ff5b5c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:36.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.881+0000 7fe70c861740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560ff5b5cc50 0x560ff5b634e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:36.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.881+0000 7fe70c861740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x560ff5ac0280 con 0x560ff5b5b0c0 2024-03-27T10:56:36.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.881+0000 7fe70c861740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x560ff5a686f0 con 0x560ff5b5cc50 2024-03-27T10:56:36.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.881+0000 7fe70c861740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x560ff5abc230 con 0x560ff5b5c290 2024-03-27T10:56:36.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.881+0000 7fe703400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560ff5b5b0c0 0x560ff5b5b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:36.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.881+0000 7fe703400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560ff5b5b0c0 0x560ff5b5b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53666/0 (socket says 172.21.15.97:53666) 2024-03-27T10:56:36.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.881+0000 7fe703400640 1 -- 172.21.15.97:0/463898091 learned_addr learned my addr 172.21.15.97:0/463898091 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:56:36.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.881+0000 7fe702000640 1 --2- 172.21.15.97:0/463898091 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560ff5b5cc50 0x560ff5b634e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:36.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.881+0000 7fe702a00640 1 --2- 172.21.15.97:0/463898091 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560ff5b5c290 0x560ff5b5c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:36.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.881+0000 7fe703400640 1 -- 172.21.15.97:0/463898091 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560ff5b5c290 msgr2=0x560ff5b5c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:56:36.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.881+0000 7fe703400640 1 --2- 172.21.15.97:0/463898091 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560ff5b5c290 0x560ff5b5c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:36.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.881+0000 7fe703400640 1 -- 172.21.15.97:0/463898091 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560ff5b5cc50 msgr2=0x560ff5b634e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:56:36.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.881+0000 7fe703400640 1 --2- 172.21.15.97:0/463898091 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560ff5b5cc50 0x560ff5b634e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:36.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.882+0000 7fe703400640 1 -- 172.21.15.97:0/463898091 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ff589c7e0 con 0x560ff5b5b0c0 2024-03-27T10:56:36.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.882+0000 7fe6fb400640 1 --2- 172.21.15.97:0/3222409130 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fe6e80a3690 0x7fe6e80a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:36.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.882+0000 7fe703400640 1 --2- 172.21.15.97:0/463898091 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560ff5b5b0c0 0x560ff5b5b4e0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc009fc0 tx=0x7fe6fc010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b50abb98d32e3f9 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:36.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.882+0000 7fe6fb400640 1 -- 172.21.15.97:0/3222409130 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fe6f40d0360 con 0x7fe7040ffcf0 2024-03-27T10:56:36.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.882+0000 7fe702a00640 1 --2- 172.21.15.97:0/3222409130 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fe6e80a3690 0x7fe6e80a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:36.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.882+0000 7fe6efe00640 1 -- 172.21.15.97:0/463898091 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6fc003b70 con 0x560ff5b5b0c0 2024-03-27T10:56:36.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.882+0000 7fe6efe00640 1 -- 172.21.15.97:0/463898091 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe6fc003d10 con 0x560ff5b5b0c0 2024-03-27T10:56:36.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.882+0000 7fe6efe00640 1 -- 172.21.15.97:0/463898091 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6fc011a40 con 0x560ff5b5b0c0 2024-03-27T10:56:36.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.883+0000 7fe702a00640 1 --2- 172.21.15.97:0/3222409130 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fe6e80a3690 0x7fe6e80a5b50 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fe6f40d0600 tx=0x7fe6f4098ad0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:36.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.883+0000 7fe70c861740 1 -- 172.21.15.97:0/463898091 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560ff5b5b0c0 msgr2=0x560ff5b5b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:36.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.883+0000 7fe70c861740 1 --2- 172.21.15.97:0/463898091 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560ff5b5b0c0 0x560ff5b5b4e0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc009fc0 tx=0x7fe6fc010860 comp rx=0 tx=0).stop 2024-03-27T10:56:36.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.883+0000 7fe70c861740 1 -- 172.21.15.97:0/463898091 shutdown_connections 2024-03-27T10:56:36.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.883+0000 7fe70c861740 1 --2- 172.21.15.97:0/463898091 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560ff5b5cc50 0x560ff5b634e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:36.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.883+0000 7fe70c861740 1 --2- 172.21.15.97:0/463898091 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560ff5b5c290 0x560ff5b5c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:36.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.883+0000 7fe70c861740 1 --2- 172.21.15.97:0/463898091 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560ff5b5b0c0 0x560ff5b5b4e0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:36.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.883+0000 7fe70c861740 1 -- 172.21.15.97:0/463898091 >> 172.21.15.97:0/463898091 conn(0x560ff5b564b0 msgr2=0x560ff5b568b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:36.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.884+0000 7fe70c861740 1 -- 172.21.15.97:0/463898091 shutdown_connections 2024-03-27T10:56:36.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.884+0000 7fe70c861740 1 -- 172.21.15.97:0/463898091 wait complete. 2024-03-27T10:56:36.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.884+0000 7fe70c861740 1 -- start start 2024-03-27T10:56:36.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.884+0000 7fe70c861740 1 Processor -- start 2024-03-27T10:56:36.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.885+0000 7fe70c861740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560ff5b5b0c0 0x560ff5b655e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:36.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.885+0000 7fe70c861740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560ff5b5c290 0x560ff5b65b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:36.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.885+0000 7fe70c861740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560ff5b5cc50 0x560ff5b6cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:36.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.885+0000 7fe70c861740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x560ff5b63c60 con 0x560ff5b5cc50 2024-03-27T10:56:36.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.885+0000 7fe70c861740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x560ff5a7baa0 con 0x560ff5b5b0c0 2024-03-27T10:56:36.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.885+0000 7fe70c861740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x560ff5b63de0 con 0x560ff5b5c290 2024-03-27T10:56:36.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.885+0000 7fe70c861740 10 client.0 _ll_register_callbacks cb 0x560ff5a7b490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:56:36.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.885+0000 7fe703400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560ff5b5cc50 0x560ff5b6cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:36.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.885+0000 7fe703400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560ff5b5cc50 0x560ff5b6cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53670/0 (socket says 172.21.15.97:53670) 2024-03-27T10:56:36.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.885+0000 7fe703400640 1 -- 172.21.15.97:0/2109009145 learned_addr learned my addr 172.21.15.97:0/2109009145 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:56:36.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.885+0000 7fe703400640 1 --2- 172.21.15.97:0/2109009145 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560ff5b5b0c0 0x560ff5b655e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:36.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.885+0000 7fe702000640 1 --2- 172.21.15.97:0/2109009145 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560ff5b5c290 0x560ff5b65b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:36.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.886+0000 7fe703400640 1 -- 172.21.15.97:0/2109009145 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560ff5b5c290 msgr2=0x560ff5b65b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:56:36.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.886+0000 7fe703400640 1 --2- 172.21.15.97:0/2109009145 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560ff5b5c290 0x560ff5b65b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:36.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.886+0000 7fe703400640 1 -- 172.21.15.97:0/2109009145 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560ff5b5b0c0 msgr2=0x560ff5b655e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:56:36.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.886+0000 7fe703400640 1 --2- 172.21.15.97:0/2109009145 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560ff5b5b0c0 0x560ff5b655e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:36.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.886+0000 7fe703400640 1 -- 172.21.15.97:0/2109009145 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ff58fcf70 con 0x560ff5b5cc50 2024-03-27T10:56:36.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.886+0000 7fe702000640 1 --2- 172.21.15.97:0/2109009145 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560ff5b5c290 0x560ff5b65b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T10:56:36.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.886+0000 7fe703400640 1 --2- 172.21.15.97:0/2109009145 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560ff5b5cc50 0x560ff5b6cb60 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc009f90 tx=0x7fe6fc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a05c78b3ac817011 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:36.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.886+0000 7fe6ecc00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T10:56:36.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.886+0000 7fe6ecc00640 1 -- 172.21.15.97:0/2109009145 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6fc024040 con 0x560ff5b5cc50 2024-03-27T10:56:36.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.887+0000 7fe70c861740 1 -- 172.21.15.97:0/2109009145 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6fc003d10 con 0x560ff5b5cc50 2024-03-27T10:56:36.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.887+0000 7fe6ecc00640 1 -- 172.21.15.97:0/2109009145 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fe6fc004c40 con 0x560ff5b5cc50 2024-03-27T10:56:36.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.887+0000 7fe6ecc00640 1 -- 172.21.15.97:0/2109009145 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe6fc004e60 con 0x560ff5b5cc50 2024-03-27T10:56:36.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.887+0000 7fe70c861740 10 client.15489 fetch_fsmap learned FSMap version 28 2024-03-27T10:56:36.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.887+0000 7fe70c861740 1 -- 172.21.15.97:0/2109009145 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560ff5794170 con 0x560ff5b5cc50 2024-03-27T10:56:36.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.887+0000 7fe6ecc00640 1 -- 172.21.15.97:0/2109009145 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe6fc003d10 con 0x560ff5b5cc50 2024-03-27T10:56:36.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.887+0000 7fe6ecc00640 1 -- 172.21.15.97:0/2109009145 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fe6fc0050c0 con 0x560ff5b5cc50 2024-03-27T10:56:36.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.887+0000 7fe70c861740 10 client.15489 fetch_fsmap finished waiting for FSMap version 28 2024-03-27T10:56:36.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.887+0000 7fe70c861740 10 client.15489 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-27T10:56:36.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.887+0000 7fe70c861740 20 client.15489 populate_metadata read hostname 'smithi097' 2024-03-27T10:56:36.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.887+0000 7fe70c861740 1 --2- 172.21.15.97:0/2109009145 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x560ff5b71170 0x560ff5b73580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:36.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.887+0000 7fe70c861740 4 client.15489 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T10:56:36.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.887+0000 7fe70c861740 1 -- 172.21.15.97:0/2109009145 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560ff5acd7c0 con 0x560ff5b71170 2024-03-27T10:56:36.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.888+0000 7fe702000640 1 --2- 172.21.15.97:0/2109009145 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x560ff5b71170 0x560ff5b73580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:36.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.888+0000 7fe702000640 1 --2- 172.21.15.97:0/2109009145 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x560ff5b71170 0x560ff5b73580 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb9ed15081094c0 server_cookie=af01f7a4984d83c1 in_seq=0 out_seq=0 2024-03-27T10:56:36.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.888+0000 7fe6ecc00640 10 client.15489 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T10:56:36.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.890+0000 7fe6ecc00640 1 -- 172.21.15.97:0/2109009145 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x560ff5acd7c0 con 0x560ff5b71170 2024-03-27T10:56:36.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.890+0000 7fe6ecc00640 10 client.15489 handle_command_reply: tid=0 2024-03-27T10:56:36.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.890+0000 7fe70c861740 1 client.15489 shutdown 2024-03-27T10:56:36.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.890+0000 7fe70c861740 1 -- 172.21.15.97:0/2109009145 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560ff5b5cc50 msgr2=0x560ff5b6cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:36.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.890+0000 7fe70c861740 1 --2- 172.21.15.97:0/2109009145 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560ff5b5cc50 0x560ff5b6cb60 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc009f90 tx=0x7fe6fc009bd0 comp rx=0 tx=0).stop 2024-03-27T10:56:36.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.890+0000 7fe70c861740 1 -- 172.21.15.97:0/2109009145 shutdown_connections 2024-03-27T10:56:36.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.890+0000 7fe70c861740 1 --2- 172.21.15.97:0/2109009145 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x560ff5b71170 0x560ff5b73580 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:36.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.890+0000 7fe70c861740 1 --2- 172.21.15.97:0/2109009145 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560ff5b5cc50 0x560ff5b6cb60 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:36.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.890+0000 7fe70c861740 1 --2- 172.21.15.97:0/2109009145 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560ff5b5c290 0x560ff5b65b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:36.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.890+0000 7fe70c861740 1 --2- 172.21.15.97:0/2109009145 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560ff5b5b0c0 0x560ff5b655e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:36.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.890+0000 7fe6ecc00640 0 client.15489 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T10:56:36.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.890+0000 7fe70c861740 1 -- 172.21.15.97:0/2109009145 >> 172.21.15.97:0/2109009145 conn(0x560ff5b564b0 msgr2=0x560ff5b557a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:36.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.890+0000 7fe70c861740 1 -- 172.21.15.97:0/2109009145 shutdown_connections 2024-03-27T10:56:36.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.890+0000 7fe70c861740 1 -- 172.21.15.97:0/2109009145 wait complete. 2024-03-27T10:56:36.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.890+0000 7fe70c861740 20 client.15489 trim_cache size 0 max 16384 2024-03-27T10:56:36.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.913+0000 7fe70c861740 1 Processor -- start 2024-03-27T10:56:36.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.913+0000 7fe70c861740 1 -- start start 2024-03-27T10:56:36.885 INFO:tasks.workunit.client.0.smithi097.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-03-27T10:56:36.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.913+0000 7fe70c861740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560ff5b5b0c0 0x560ff5b54f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:36.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.913+0000 7fe70c861740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560ff5b5c290 0x560ff5b6ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:36.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.913+0000 7fe70c861740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560ff5b5cc50 0x560ff5b55450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:36.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.913+0000 7fe70c861740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe7041ab030 con 0x560ff5b5b0c0 2024-03-27T10:56:36.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.913+0000 7fe70c861740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe7041acb70 con 0x560ff5b5cc50 2024-03-27T10:56:36.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.913+0000 7fe70c861740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe7041ab700 con 0x560ff5b5c290 2024-03-27T10:56:36.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.913+0000 7fe703400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560ff5b5b0c0 0x560ff5b54f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:36.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.914+0000 7fe703400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560ff5b5b0c0 0x560ff5b54f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53672/0 (socket says 172.21.15.97:53672) 2024-03-27T10:56:36.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.914+0000 7fe703400640 1 -- 172.21.15.97:0/2643035461 learned_addr learned my addr 172.21.15.97:0/2643035461 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:56:36.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.914+0000 7fe702000640 1 --2- 172.21.15.97:0/2643035461 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560ff5b5c290 0x560ff5b6ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:36.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.914+0000 7fe703400640 1 --2- 172.21.15.97:0/2643035461 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560ff5b5cc50 0x560ff5b55450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:36.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.914+0000 7fe703400640 1 -- 172.21.15.97:0/2643035461 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560ff5b5c290 msgr2=0x560ff5b6ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:56:36.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.914+0000 7fe703400640 1 --2- 172.21.15.97:0/2643035461 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560ff5b5c290 0x560ff5b6ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:36.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.914+0000 7fe703400640 1 -- 172.21.15.97:0/2643035461 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560ff5b5cc50 msgr2=0x560ff5b55450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:56:36.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.914+0000 7fe703400640 1 --2- 172.21.15.97:0/2643035461 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560ff5b5cc50 0x560ff5b55450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:36.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.914+0000 7fe703400640 1 -- 172.21.15.97:0/2643035461 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ff5b63b40 con 0x560ff5b5b0c0 2024-03-27T10:56:36.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.914+0000 7fe703400640 1 --2- 172.21.15.97:0/2643035461 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560ff5b5b0c0 0x560ff5b54f10 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc009560 tx=0x7fe6fc009c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a00cca08cf1f647 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:36.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.914+0000 7fe6ef400640 1 -- 172.21.15.97:0/2643035461 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6fc003a40 con 0x560ff5b5b0c0 2024-03-27T10:56:36.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.915+0000 7fe6ef400640 1 -- 172.21.15.97:0/2643035461 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe6fc005720 con 0x560ff5b5b0c0 2024-03-27T10:56:36.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.915+0000 7fe6ef400640 1 -- 172.21.15.97:0/2643035461 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6fc02baf0 con 0x560ff5b5b0c0 2024-03-27T10:56:36.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.915+0000 7fe70c861740 1 -- 172.21.15.97:0/2643035461 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560ff5b5b0c0 msgr2=0x560ff5b54f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:36.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.915+0000 7fe70c861740 1 --2- 172.21.15.97:0/2643035461 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560ff5b5b0c0 0x560ff5b54f10 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc009560 tx=0x7fe6fc009c40 comp rx=0 tx=0).stop 2024-03-27T10:56:36.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.915+0000 7fe70c861740 1 -- 172.21.15.97:0/2643035461 shutdown_connections 2024-03-27T10:56:36.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.915+0000 7fe70c861740 1 --2- 172.21.15.97:0/2643035461 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560ff5b5cc50 0x560ff5b55450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:36.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.915+0000 7fe70c861740 1 --2- 172.21.15.97:0/2643035461 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560ff5b5c290 0x560ff5b6ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:36.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.915+0000 7fe70c861740 1 --2- 172.21.15.97:0/2643035461 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560ff5b5b0c0 0x560ff5b54f10 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:36.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.915+0000 7fe70c861740 1 -- 172.21.15.97:0/2643035461 >> 172.21.15.97:0/2643035461 conn(0x560ff5b564b0 msgr2=0x560ff5b655e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:36.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.916+0000 7fe70c861740 1 -- 172.21.15.97:0/2643035461 shutdown_connections 2024-03-27T10:56:36.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.916+0000 7fe70c861740 1 -- 172.21.15.97:0/2643035461 wait complete. 2024-03-27T10:56:36.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.916+0000 7fe70c861740 1 -- start start 2024-03-27T10:56:36.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.916+0000 7fe70c861740 1 Processor -- start 2024-03-27T10:56:36.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.916+0000 7fe70c861740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560ff5b5b0c0 0x560ff5b6acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:36.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.916+0000 7fe70c861740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560ff5b5c290 0x560ff5b6b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:36.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.917+0000 7fe70c861740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560ff5b5cc50 0x560ff5b6b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:36.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.917+0000 7fe70c861740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe7041ac4a0 con 0x560ff5b5cc50 2024-03-27T10:56:36.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.917+0000 7fe70c861740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe7041abdd0 con 0x560ff5b5c290 2024-03-27T10:56:36.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.917+0000 7fe70c861740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe7041a9010 con 0x560ff5b5b0c0 2024-03-27T10:56:36.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.917+0000 7fe70c861740 10 client.0 _ll_register_callbacks cb 0x7fe6fc011bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:56:36.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.917+0000 7fe703400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560ff5b5cc50 0x560ff5b6b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:36.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.917+0000 7fe703400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560ff5b5cc50 0x560ff5b6b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53686/0 (socket says 172.21.15.97:53686) 2024-03-27T10:56:36.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.917+0000 7fe703400640 1 -- 172.21.15.97:0/732436701 learned_addr learned my addr 172.21.15.97:0/732436701 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:56:36.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.917+0000 7fe702000640 1 --2- 172.21.15.97:0/732436701 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560ff5b5c290 0x560ff5b6b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:36.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.917+0000 7fe703400640 1 --2- 172.21.15.97:0/732436701 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560ff5b5b0c0 0x560ff5b6acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:36.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.917+0000 7fe703400640 1 -- 172.21.15.97:0/732436701 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560ff5b5b0c0 msgr2=0x560ff5b6acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:56:36.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.917+0000 7fe703400640 1 --2- 172.21.15.97:0/732436701 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560ff5b5b0c0 0x560ff5b6acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:36.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.917+0000 7fe703400640 1 -- 172.21.15.97:0/732436701 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560ff5b5c290 msgr2=0x560ff5b6b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:56:36.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.917+0000 7fe703400640 1 --2- 172.21.15.97:0/732436701 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560ff5b5c290 0x560ff5b6b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:36.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.917+0000 7fe703400640 1 -- 172.21.15.97:0/732436701 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ff5b6c040 con 0x560ff5b5cc50 2024-03-27T10:56:36.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.917+0000 7fe703400640 1 --2- 172.21.15.97:0/732436701 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560ff5b5cc50 0x560ff5b6b810 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x560ff5b62ad0 tx=0x7fe6fc005cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ddcd1c0612bec7e server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:36.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.917+0000 7fe6ecc00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T10:56:36.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.918+0000 7fe6ecc00640 1 -- 172.21.15.97:0/732436701 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6fc049060 con 0x560ff5b5cc50 2024-03-27T10:56:36.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.918+0000 7fe6ecc00640 1 -- 172.21.15.97:0/732436701 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fe6fc009a40 con 0x560ff5b5cc50 2024-03-27T10:56:36.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.918+0000 7fe70c861740 1 -- 172.21.15.97:0/732436701 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6fc005720 con 0x560ff5b5cc50 2024-03-27T10:56:36.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.918+0000 7fe6ecc00640 1 -- 172.21.15.97:0/732436701 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe6fc0058d0 con 0x560ff5b5cc50 2024-03-27T10:56:36.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.918+0000 7fe70c861740 10 client.15495 fetch_fsmap learned FSMap version 28 2024-03-27T10:56:36.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.918+0000 7fe70c861740 1 -- 172.21.15.97:0/732436701 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560ff5bd7780 con 0x560ff5b5cc50 2024-03-27T10:56:36.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.918+0000 7fe6ecc00640 1 -- 172.21.15.97:0/732436701 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe6fc005720 con 0x560ff5b5cc50 2024-03-27T10:56:36.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.918+0000 7fe6ecc00640 1 -- 172.21.15.97:0/732436701 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fe6fc0364b0 con 0x560ff5b5cc50 2024-03-27T10:56:36.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.919+0000 7fe70c861740 10 client.15495 fetch_fsmap finished waiting for FSMap version 28 2024-03-27T10:56:36.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.919+0000 7fe70c861740 10 client.15495 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-27T10:56:36.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.919+0000 7fe70c861740 20 client.15495 populate_metadata read hostname 'smithi097' 2024-03-27T10:56:36.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.919+0000 7fe70c861740 1 --2- 172.21.15.97:0/732436701 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x560ff5bd8160 0x560ff5bda580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:36.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.919+0000 7fe70c861740 4 client.15495 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-27T10:56:36.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.919+0000 7fe70c861740 1 -- 172.21.15.97:0/732436701 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x560ff5ac7b10 con 0x560ff5bd8160 2024-03-27T10:56:36.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.919+0000 7fe702000640 1 --2- 172.21.15.97:0/732436701 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x560ff5bd8160 0x560ff5bda580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:36.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.920+0000 7fe702000640 1 --2- 172.21.15.97:0/732436701 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x560ff5bd8160 0x560ff5bda580 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c10063a57d1dd4cd server_cookie=68c03eb427a9efb9 in_seq=0 out_seq=0 2024-03-27T10:56:36.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.920+0000 7fe6ecc00640 10 client.15495 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T10:56:36.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.921+0000 7fe6ecc00640 1 -- 172.21.15.97:0/732436701 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560ff5ac7b10 con 0x560ff5bd8160 2024-03-27T10:56:36.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.921+0000 7fe6ecc00640 10 client.15495 handle_command_reply: tid=0 2024-03-27T10:56:36.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.921+0000 7fe70c861740 1 client.15495 shutdown 2024-03-27T10:56:36.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.922+0000 7fe70c861740 1 -- 172.21.15.97:0/732436701 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560ff5b5cc50 msgr2=0x560ff5b6b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:36.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.922+0000 7fe70c861740 1 --2- 172.21.15.97:0/732436701 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560ff5b5cc50 0x560ff5b6b810 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x560ff5b62ad0 tx=0x7fe6fc005cf0 comp rx=0 tx=0).stop 2024-03-27T10:56:36.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.922+0000 7fe70c861740 1 -- 172.21.15.97:0/732436701 shutdown_connections 2024-03-27T10:56:36.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.922+0000 7fe70c861740 1 --2- 172.21.15.97:0/732436701 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x560ff5bd8160 0x560ff5bda580 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:36.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.922+0000 7fe70c861740 1 --2- 172.21.15.97:0/732436701 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560ff5b5cc50 0x560ff5b6b810 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:36.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.922+0000 7fe70c861740 1 --2- 172.21.15.97:0/732436701 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560ff5b5c290 0x560ff5b6b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:36.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.922+0000 7fe70c861740 1 --2- 172.21.15.97:0/732436701 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560ff5b5b0c0 0x560ff5b6acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:36.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.922+0000 7fe6ecc00640 0 client.15495 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T10:56:36.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.922+0000 7fe70c861740 1 -- 172.21.15.97:0/732436701 >> 172.21.15.97:0/732436701 conn(0x560ff5b564b0 msgr2=0x560ff5b59710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:36.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.922+0000 7fe70c861740 1 -- 172.21.15.97:0/732436701 shutdown_connections 2024-03-27T10:56:36.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.922+0000 7fe70c861740 1 -- 172.21.15.97:0/732436701 wait complete. 2024-03-27T10:56:36.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.922+0000 7fe70c861740 20 client.15495 trim_cache size 0 max 16384 2024-03-27T10:56:36.897 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-03-27T10:56:36.897 INFO:teuthology.orchestra.run.smithi097.stdout: "return_code": 0, 2024-03-27T10:56:36.897 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_tag": "fc430ced-914b-4e83-85df-1bfef281741a", 2024-03-27T10:56:36.897 INFO:teuthology.orchestra.run.smithi097.stdout: "mode": "asynchronous" 2024-03-27T10:56:36.897 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-03-27T10:56:36.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.923+0000 7fe703e00640 1 -- 172.21.15.97:0/3222409130 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fe6e80a3690 msgr2=0x7fe6e80a5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:36.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.923+0000 7fe703e00640 1 --2- 172.21.15.97:0/3222409130 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fe6e80a3690 0x7fe6e80a5b50 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fe6f40d0600 tx=0x7fe6f4098ad0 comp rx=0 tx=0).stop 2024-03-27T10:56:36.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.923+0000 7fe703e00640 1 -- 172.21.15.97:0/3222409130 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7040ffcf0 msgr2=0x7fe704199ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:36.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.923+0000 7fe703e00640 1 --2- 172.21.15.97:0/3222409130 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7040ffcf0 0x7fe704199ee0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fe6f400e9c0 tx=0x7fe6f400ee80 comp rx=0 tx=0).stop 2024-03-27T10:56:36.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.923+0000 7fe703e00640 1 -- 172.21.15.97:0/3222409130 shutdown_connections 2024-03-27T10:56:36.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.923+0000 7fe703e00640 1 --2- 172.21.15.97:0/3222409130 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fe6e80a3690 0x7fe6e80a5b50 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:36.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.923+0000 7fe703e00640 1 --2- 172.21.15.97:0/3222409130 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe704103680 0x7fe7041a1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:36.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.923+0000 7fe703e00640 1 --2- 172.21.15.97:0/3222409130 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe704102cc0 0x7fe70419a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:36.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.923+0000 7fe703e00640 1 --2- 172.21.15.97:0/3222409130 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7040ffcf0 0x7fe704199ee0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:36.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.923+0000 7fe703e00640 1 -- 172.21.15.97:0/3222409130 >> 172.21.15.97:0/3222409130 conn(0x7fe7040fb640 msgr2=0x7fe704108490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:36.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.923+0000 7fe703e00640 1 -- 172.21.15.97:0/3222409130 shutdown_connections 2024-03-27T10:56:36.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:36.923+0000 7fe703e00640 1 -- 172.21.15.97:0/3222409130 wait complete. 2024-03-27T10:56:36.910 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-27T10:56:36.910 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-27T10:56:37.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.203+0000 7f39e0c00640 1 -- 172.21.15.97:0/75746445 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39dc103600 msgr2=0x7f39dc109e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:37.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.203+0000 7f39e0c00640 1 --2- 172.21.15.97:0/75746445 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39dc103600 0x7f39dc109e90 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f39d4009ca0 tx=0x7f39d402c100 comp rx=0 tx=0).stop 2024-03-27T10:56:37.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.203+0000 7f39e0c00640 1 -- 172.21.15.97:0/75746445 shutdown_connections 2024-03-27T10:56:37.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.203+0000 7f39e0c00640 1 --2- 172.21.15.97:0/75746445 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39dc103600 0x7f39dc109e90 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:37.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.203+0000 7f39e0c00640 1 --2- 172.21.15.97:0/75746445 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f39dc102c40 0x7f39dc1030c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:37.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.203+0000 7f39e0c00640 1 --2- 172.21.15.97:0/75746445 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f39dc101a50 0x7f39dc101e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:37.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.203+0000 7f39e0c00640 1 -- 172.21.15.97:0/75746445 >> 172.21.15.97:0/75746445 conn(0x7f39dc0fcfc0 msgr2=0x7f39dc0ff400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:37.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.203+0000 7f39e0c00640 1 -- 172.21.15.97:0/75746445 shutdown_connections 2024-03-27T10:56:37.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.203+0000 7f39e0c00640 1 -- 172.21.15.97:0/75746445 wait complete. 2024-03-27T10:56:37.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.204+0000 7f39e0c00640 1 Processor -- start 2024-03-27T10:56:37.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.204+0000 7f39e0c00640 1 -- start start 2024-03-27T10:56:37.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.204+0000 7f39e0c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39dc101a50 0x7f39dc075900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:37.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.204+0000 7f39e0c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f39dc102c40 0x7f39dc073f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:37.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.205+0000 7f39e0c00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f39dc103600 0x7f39dc074480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:37.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.205+0000 7f39e0c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f39dc10d580 con 0x7f39dc101a50 2024-03-27T10:56:37.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.205+0000 7f39e0c00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f39dc10d400 con 0x7f39dc102c40 2024-03-27T10:56:37.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.205+0000 7f39e0c00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f39dc10d700 con 0x7f39dc103600 2024-03-27T10:56:37.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.205+0000 7f39db400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39dc101a50 0x7f39dc075900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:37.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.205+0000 7f39db400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39dc101a50 0x7f39dc075900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53702/0 (socket says 172.21.15.97:53702) 2024-03-27T10:56:37.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.205+0000 7f39db400640 1 -- 172.21.15.97:0/3008364435 learned_addr learned my addr 172.21.15.97:0/3008364435 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:56:37.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.205+0000 7f39d3e00640 1 --2- 172.21.15.97:0/3008364435 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f39dc102c40 0x7f39dc073f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:37.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.205+0000 7f39db400640 1 -- 172.21.15.97:0/3008364435 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f39dc103600 msgr2=0x7f39dc074480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:37.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.205+0000 7f39db400640 1 --2- 172.21.15.97:0/3008364435 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f39dc103600 0x7f39dc074480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:37.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.205+0000 7f39db400640 1 -- 172.21.15.97:0/3008364435 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f39dc102c40 msgr2=0x7f39dc073f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:37.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.205+0000 7f39db400640 1 --2- 172.21.15.97:0/3008364435 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f39dc102c40 0x7f39dc073f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:37.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.205+0000 7f39db400640 1 -- 172.21.15.97:0/3008364435 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39dc074d10 con 0x7f39dc101a50 2024-03-27T10:56:37.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.205+0000 7f39d3e00640 1 --2- 172.21.15.97:0/3008364435 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f39dc102c40 0x7f39dc073f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:56:37.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.205+0000 7f39db400640 1 --2- 172.21.15.97:0/3008364435 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39dc101a50 0x7f39dc075900 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f39cc00e9c0 tx=0x7f39cc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:37.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.206+0000 7f39d8c00640 1 -- 172.21.15.97:0/3008364435 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39cc00cd80 con 0x7f39dc101a50 2024-03-27T10:56:37.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.206+0000 7f39d8c00640 1 -- 172.21.15.97:0/3008364435 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f39cc0035f0 con 0x7f39dc101a50 2024-03-27T10:56:37.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.206+0000 7f39e0c00640 1 -- 172.21.15.97:0/3008364435 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39dc1a1d10 con 0x7f39dc101a50 2024-03-27T10:56:37.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.206+0000 7f39d8c00640 1 -- 172.21.15.97:0/3008364435 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39cc010690 con 0x7f39dc101a50 2024-03-27T10:56:37.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.206+0000 7f39e0c00640 1 -- 172.21.15.97:0/3008364435 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39dc1a21b0 con 0x7f39dc101a50 2024-03-27T10:56:37.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.209+0000 7f39e4e54740 1 Processor -- start 2024-03-27T10:56:37.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.209+0000 7f39e4e54740 1 -- start start 2024-03-27T10:56:37.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.209+0000 7f39d8c00640 1 -- 172.21.15.97:0/3008364435 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f39cc0037a0 con 0x7f39dc101a50 2024-03-27T10:56:37.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.209+0000 7f39e4e54740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56323451c670 0x56323451ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:37.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.209+0000 7f39e4e54740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56323451d060 0x563234521b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:37.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.209+0000 7f39e4e54740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5632345221d0 0x563234524660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:37.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.209+0000 7f39e4e54740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5632344826b0 con 0x56323451d060 2024-03-27T10:56:37.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.209+0000 7f39e4e54740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x56323442a750 con 0x56323451c670 2024-03-27T10:56:37.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.209+0000 7f39e4e54740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x56323447e660 con 0x5632345221d0 2024-03-27T10:56:37.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.210+0000 7f39db400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56323451d060 0x563234521b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:37.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.210+0000 7f39db400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56323451d060 0x563234521b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53716/0 (socket says 172.21.15.97:53716) 2024-03-27T10:56:37.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.210+0000 7f39db400640 1 -- 172.21.15.97:0/1978479692 learned_addr learned my addr 172.21.15.97:0/1978479692 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:56:37.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.210+0000 7f39d3e00640 1 --2- 172.21.15.97:0/1978479692 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5632345221d0 0x563234524660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:37.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.210+0000 7f39dbe00640 1 --2- 172.21.15.97:0/1978479692 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56323451c670 0x56323451ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:37.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.210+0000 7f39db400640 1 -- 172.21.15.97:0/1978479692 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5632345221d0 msgr2=0x563234524660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:56:37.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.210+0000 7f39db400640 1 --2- 172.21.15.97:0/1978479692 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5632345221d0 0x563234524660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:37.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.210+0000 7f39db400640 1 -- 172.21.15.97:0/1978479692 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56323451c670 msgr2=0x56323451ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:56:37.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.210+0000 7f39db400640 1 --2- 172.21.15.97:0/1978479692 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56323451c670 0x56323451ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:37.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.210+0000 7f39db400640 1 -- 172.21.15.97:0/1978479692 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632342607d0 con 0x56323451d060 2024-03-27T10:56:37.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.210+0000 7f39d3e00640 1 --2- 172.21.15.97:0/1978479692 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5632345221d0 0x563234524660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T10:56:37.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.210+0000 7f39dbe00640 1 --2- 172.21.15.97:0/1978479692 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56323451c670 0x56323451ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:56:37.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.210+0000 7f39db400640 1 --2- 172.21.15.97:0/1978479692 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56323451d060 0x563234521b60 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f39cc0d8370 tx=0x7f39cc0d8830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1da6c18f6740c20c server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:37.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.211+0000 7f39d8c00640 1 --2- 172.21.15.97:0/3008364435 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f39c00a3690 0x7f39c00a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:37.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.211+0000 7f39cbe00640 1 -- 172.21.15.97:0/1978479692 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39cc0d2520 con 0x56323451d060 2024-03-27T10:56:37.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.211+0000 7f39cbe00640 1 -- 172.21.15.97:0/1978479692 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f39cc09cef0 con 0x56323451d060 2024-03-27T10:56:37.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.211+0000 7f39cbe00640 1 -- 172.21.15.97:0/1978479692 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39cc09c7f0 con 0x56323451d060 2024-03-27T10:56:37.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.211+0000 7f39d8c00640 1 -- 172.21.15.97:0/3008364435 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f39cc0d0310 con 0x7f39dc101a50 2024-03-27T10:56:37.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.211+0000 7f39dbe00640 1 --2- 172.21.15.97:0/3008364435 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f39c00a3690 0x7f39c00a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:37.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.212+0000 7f39e4e54740 1 -- 172.21.15.97:0/1978479692 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56323451d060 msgr2=0x563234521b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:37.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.212+0000 7f39e4e54740 1 --2- 172.21.15.97:0/1978479692 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56323451d060 0x563234521b60 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f39cc0d8370 tx=0x7f39cc0d8830 comp rx=0 tx=0).stop 2024-03-27T10:56:37.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.212+0000 7f39dbe00640 1 --2- 172.21.15.97:0/3008364435 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f39c00a3690 0x7f39c00a5b50 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f39d4009a60 tx=0x7f39d4005d80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:37.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.212+0000 7f39e4e54740 1 -- 172.21.15.97:0/1978479692 shutdown_connections 2024-03-27T10:56:37.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.212+0000 7f39e4e54740 1 --2- 172.21.15.97:0/1978479692 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5632345221d0 0x563234524660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:37.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.212+0000 7f39e4e54740 1 --2- 172.21.15.97:0/1978479692 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56323451d060 0x563234521b60 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:37.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.212+0000 7f39e4e54740 1 --2- 172.21.15.97:0/1978479692 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56323451c670 0x56323451ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:37.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.212+0000 7f39e4e54740 1 -- 172.21.15.97:0/1978479692 >> 172.21.15.97:0/1978479692 conn(0x563234517be0 msgr2=0x563234517fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:37.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.212+0000 7f39e4e54740 1 -- 172.21.15.97:0/1978479692 shutdown_connections 2024-03-27T10:56:37.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.212+0000 7f39e4e54740 1 -- 172.21.15.97:0/1978479692 wait complete. 2024-03-27T10:56:37.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.212+0000 7f39e4e54740 1 -- start start 2024-03-27T10:56:37.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.213+0000 7f39e4e54740 1 Processor -- start 2024-03-27T10:56:37.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.213+0000 7f39e4e54740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56323451c670 0x563234526bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:37.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.213+0000 7f39e4e54740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56323451d060 0x5632345270f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:37.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.213+0000 7f39e4e54740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5632345221d0 0x56323452e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:37.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.213+0000 7f39e4e54740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x563234525180 con 0x5632345221d0 2024-03-27T10:56:37.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.213+0000 7f39e4e54740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x56323443da80 con 0x56323451d060 2024-03-27T10:56:37.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.213+0000 7f39e4e54740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x563234525300 con 0x56323451c670 2024-03-27T10:56:37.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.213+0000 7f39e4e54740 10 client.0 _ll_register_callbacks cb 0x56323443d470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:56:37.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.213+0000 7f39d3e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56323451d060 0x5632345270f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:37.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.213+0000 7f39dbe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5632345221d0 0x56323452e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:37.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.213+0000 7f39d3e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56323451d060 0x5632345270f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:44232/0 (socket says 172.21.15.97:44232) 2024-03-27T10:56:37.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.213+0000 7f39db400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56323451c670 0x563234526bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:37.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.213+0000 7f39d3e00640 1 -- 172.21.15.97:0/4001136006 learned_addr learned my addr 172.21.15.97:0/4001136006 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:56:37.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.214+0000 7f39d3e00640 1 -- 172.21.15.97:0/4001136006 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56323451c670 msgr2=0x563234526bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:56:37.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.214+0000 7f39d3e00640 1 --2- 172.21.15.97:0/4001136006 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56323451c670 0x563234526bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:37.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.214+0000 7f39d3e00640 1 -- 172.21.15.97:0/4001136006 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5632345221d0 msgr2=0x56323452e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:56:37.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.214+0000 7f39d3e00640 1 --2- 172.21.15.97:0/4001136006 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5632345221d0 0x56323452e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:37.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.214+0000 7f39d3e00640 1 -- 172.21.15.97:0/4001136006 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632341963f0 con 0x56323451d060 2024-03-27T10:56:37.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.214+0000 7f39dbe00640 1 --2- 172.21.15.97:0/4001136006 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5632345221d0 0x56323452e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T10:56:37.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.214+0000 7f39d3e00640 1 --2- 172.21.15.97:0/4001136006 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56323451d060 0x5632345270f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f39c4012510 tx=0x7f39c40129d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a952a4843c211d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:37.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.214+0000 7f39c8c00640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-03-27T10:56:37.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.215+0000 7f39c8c00640 1 -- 172.21.15.97:0/4001136006 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c4004220 con 0x56323451d060 2024-03-27T10:56:37.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.215+0000 7f39e4e54740 1 -- 172.21.15.97:0/4001136006 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39cc09cef0 con 0x56323451d060 2024-03-27T10:56:37.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.215+0000 7f39c8c00640 1 -- 172.21.15.97:0/4001136006 <== mon.2 v2:172.21.15.114:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f39c40043c0 con 0x56323451d060 2024-03-27T10:56:37.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.215+0000 7f39c8c00640 1 -- 172.21.15.97:0/4001136006 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f39c4010780 con 0x56323451d060 2024-03-27T10:56:37.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.215+0000 7f39e4e54740 10 client.35198 fetch_fsmap learned FSMap version 28 2024-03-27T10:56:37.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.215+0000 7f39e4e54740 1 -- 172.21.15.97:0/4001136006 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5632341560f0 con 0x56323451d060 2024-03-27T10:56:37.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.215+0000 7f39c8c00640 1 -- 172.21.15.97:0/4001136006 <== mon.2 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f39cc09cef0 con 0x56323451d060 2024-03-27T10:56:37.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.215+0000 7f39c8c00640 1 -- 172.21.15.97:0/4001136006 <== mon.2 v2:172.21.15.114:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f39c400f040 con 0x56323451d060 2024-03-27T10:56:37.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.216+0000 7f39e4e54740 10 client.35198 fetch_fsmap finished waiting for FSMap version 28 2024-03-27T10:56:37.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.216+0000 7f39e4e54740 10 client.35198 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-27T10:56:37.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.216+0000 7f39e4e54740 20 client.35198 populate_metadata read hostname 'smithi097' 2024-03-27T10:56:37.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.216+0000 7f39e4e54740 1 --2- 172.21.15.97:0/4001136006 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x563234532990 0x563234534df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:37.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.216+0000 7f39e4e54740 4 client.35198 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T10:56:37.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.216+0000 7f39e4e54740 1 -- 172.21.15.97:0/4001136006 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56323448fbf0 con 0x563234532990 2024-03-27T10:56:37.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.216+0000 7f39db400640 1 --2- 172.21.15.97:0/4001136006 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x563234532990 0x563234534df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:37.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.217+0000 7f39db400640 1 --2- 172.21.15.97:0/4001136006 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x563234532990 0x563234534df0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=baf9e9516ea4f30a server_cookie=48b3c607024e2432 in_seq=0 out_seq=0 2024-03-27T10:56:37.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.217+0000 7f39c8c00640 10 client.35198 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T10:56:37.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.217+0000 7f39c8c00640 1 -- 172.21.15.97:0/4001136006 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x56323448fbf0 con 0x563234532990 2024-03-27T10:56:37.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.217+0000 7f39c8c00640 10 client.35198 handle_command_reply: tid=0 2024-03-27T10:56:37.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.218+0000 7f39e4e54740 1 client.35198 shutdown 2024-03-27T10:56:37.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.218+0000 7f39e4e54740 1 -- 172.21.15.97:0/4001136006 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56323451d060 msgr2=0x5632345270f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:37.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.218+0000 7f39e4e54740 1 --2- 172.21.15.97:0/4001136006 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56323451d060 0x5632345270f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f39c4012510 tx=0x7f39c40129d0 comp rx=0 tx=0).stop 2024-03-27T10:56:37.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.218+0000 7f39e4e54740 1 -- 172.21.15.97:0/4001136006 shutdown_connections 2024-03-27T10:56:37.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.218+0000 7f39e4e54740 1 --2- 172.21.15.97:0/4001136006 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x563234532990 0x563234534df0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:37.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.218+0000 7f39e4e54740 1 --2- 172.21.15.97:0/4001136006 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5632345221d0 0x56323452e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:37.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.218+0000 7f39c8c00640 0 client.35198 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T10:56:37.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.218+0000 7f39e4e54740 1 --2- 172.21.15.97:0/4001136006 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56323451d060 0x5632345270f0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:37.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.218+0000 7f39e4e54740 1 --2- 172.21.15.97:0/4001136006 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56323451c670 0x563234526bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:37.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.218+0000 7f39e4e54740 1 -- 172.21.15.97:0/4001136006 >> 172.21.15.97:0/4001136006 conn(0x563234517be0 msgr2=0x563234516e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:37.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.218+0000 7f39e4e54740 1 -- 172.21.15.97:0/4001136006 shutdown_connections 2024-03-27T10:56:37.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.218+0000 7f39e4e54740 1 -- 172.21.15.97:0/4001136006 wait complete. 2024-03-27T10:56:37.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.218+0000 7f39e4e54740 20 client.35198 trim_cache size 0 max 16384 2024-03-27T10:56:37.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.238+0000 7f39e4e54740 1 Processor -- start 2024-03-27T10:56:37.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.238+0000 7f39e4e54740 1 -- start start 2024-03-27T10:56:37.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.239+0000 7f39e4e54740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56323451c670 0x56323451e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:37.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.239+0000 7f39e4e54740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56323451d060 0x56323451eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:37.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.239+0000 7f39e4e54740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5632345221d0 0x56323452e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:37.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.239+0000 7f39e4e54740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f39dc1aaf80 con 0x56323451d060 2024-03-27T10:56:37.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.239+0000 7f39e4e54740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f39dc1acac0 con 0x56323451c670 2024-03-27T10:56:37.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.239+0000 7f39e4e54740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f39dc1ab650 con 0x5632345221d0 2024-03-27T10:56:37.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.239+0000 7f39d3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56323451d060 0x56323451eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:37.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.239+0000 7f39d3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56323451d060 0x56323451eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53728/0 (socket says 172.21.15.97:53728) 2024-03-27T10:56:37.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.239+0000 7f39d3e00640 1 -- 172.21.15.97:0/1868511016 learned_addr learned my addr 172.21.15.97:0/1868511016 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:56:37.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.239+0000 7f39db400640 1 --2- 172.21.15.97:0/1868511016 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56323451c670 0x56323451e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:37.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.239+0000 7f39dbe00640 1 --2- 172.21.15.97:0/1868511016 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5632345221d0 0x56323452e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:37.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.239+0000 7f39d3e00640 1 -- 172.21.15.97:0/1868511016 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5632345221d0 msgr2=0x56323452e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:56:37.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.239+0000 7f39d3e00640 1 --2- 172.21.15.97:0/1868511016 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5632345221d0 0x56323452e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:37.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.239+0000 7f39d3e00640 1 -- 172.21.15.97:0/1868511016 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56323451c670 msgr2=0x56323451e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:56:37.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.239+0000 7f39d3e00640 1 --2- 172.21.15.97:0/1868511016 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56323451c670 0x56323451e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:37.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.239+0000 7f39d3e00640 1 -- 172.21.15.97:0/1868511016 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56323451d8f0 con 0x56323451d060 2024-03-27T10:56:37.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.240+0000 7f39d3e00640 1 --2- 172.21.15.97:0/1868511016 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56323451d060 0x56323451eaa0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f39c4001eb0 tx=0x7f39c401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f1864c6f8a16542 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:37.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.240+0000 7f39cb400640 1 -- 172.21.15.97:0/1868511016 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c40120d0 con 0x56323451d060 2024-03-27T10:56:37.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.240+0000 7f39cb400640 1 -- 172.21.15.97:0/1868511016 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f39c402b070 con 0x56323451d060 2024-03-27T10:56:37.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.240+0000 7f39cb400640 1 -- 172.21.15.97:0/1868511016 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c40035f0 con 0x56323451d060 2024-03-27T10:56:37.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.241+0000 7f39e4e54740 1 -- 172.21.15.97:0/1868511016 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56323451d060 msgr2=0x56323451eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:37.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.241+0000 7f39e4e54740 1 --2- 172.21.15.97:0/1868511016 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56323451d060 0x56323451eaa0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f39c4001eb0 tx=0x7f39c401c4d0 comp rx=0 tx=0).stop 2024-03-27T10:56:37.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.241+0000 7f39e4e54740 1 -- 172.21.15.97:0/1868511016 shutdown_connections 2024-03-27T10:56:37.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.241+0000 7f39e4e54740 1 --2- 172.21.15.97:0/1868511016 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5632345221d0 0x56323452e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:37.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.241+0000 7f39e4e54740 1 --2- 172.21.15.97:0/1868511016 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56323451d060 0x56323451eaa0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:37.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.241+0000 7f39e4e54740 1 --2- 172.21.15.97:0/1868511016 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56323451c670 0x56323451e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:37.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.241+0000 7f39e4e54740 1 -- 172.21.15.97:0/1868511016 >> 172.21.15.97:0/1868511016 conn(0x563234517be0 msgr2=0x563234526bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:37.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.241+0000 7f39e4e54740 1 -- 172.21.15.97:0/1868511016 shutdown_connections 2024-03-27T10:56:37.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.241+0000 7f39e4e54740 1 -- 172.21.15.97:0/1868511016 wait complete. 2024-03-27T10:56:37.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.241+0000 7f39e4e54740 1 -- start start 2024-03-27T10:56:37.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.242+0000 7f39e4e54740 1 Processor -- start 2024-03-27T10:56:37.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.242+0000 7f39e4e54740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56323451c670 0x56323452b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:37.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.242+0000 7f39e4e54740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56323451d060 0x56323452be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:37.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.242+0000 7f39e4e54740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5632345221d0 0x56323452ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:37.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.242+0000 7f39e4e54740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f39dc1ac3f0 con 0x5632345221d0 2024-03-27T10:56:37.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.242+0000 7f39e4e54740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f39dc1abd20 con 0x56323451d060 2024-03-27T10:56:37.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.242+0000 7f39e4e54740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f39dc1a8f60 con 0x56323451c670 2024-03-27T10:56:37.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.242+0000 7f39e4e54740 10 client.0 _ll_register_callbacks cb 0x7f39c4009cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:56:37.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.242+0000 7f39dbe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5632345221d0 0x56323452ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:37.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.242+0000 7f39dbe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5632345221d0 0x56323452ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53736/0 (socket says 172.21.15.97:53736) 2024-03-27T10:56:37.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.242+0000 7f39dbe00640 1 -- 172.21.15.97:0/1628733726 learned_addr learned my addr 172.21.15.97:0/1628733726 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:56:37.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.243+0000 7f39d3e00640 1 --2- 172.21.15.97:0/1628733726 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56323451d060 0x56323452be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:37.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.243+0000 7f39db400640 1 --2- 172.21.15.97:0/1628733726 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56323451c670 0x56323452b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:37.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.243+0000 7f39dbe00640 1 -- 172.21.15.97:0/1628733726 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56323451c670 msgr2=0x56323452b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:56:37.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.243+0000 7f39dbe00640 1 --2- 172.21.15.97:0/1628733726 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56323451c670 0x56323452b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:37.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.243+0000 7f39dbe00640 1 -- 172.21.15.97:0/1628733726 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56323451d060 msgr2=0x56323452be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:56:37.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.243+0000 7f39dbe00640 1 --2- 172.21.15.97:0/1628733726 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56323451d060 0x56323452be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:37.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.243+0000 7f39dbe00640 1 -- 172.21.15.97:0/1628733726 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56323452d440 con 0x5632345221d0 2024-03-27T10:56:37.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.243+0000 7f39d3e00640 1 --2- 172.21.15.97:0/1628733726 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56323451d060 0x56323452be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T10:56:37.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.243+0000 7f39dbe00640 1 --2- 172.21.15.97:0/1628733726 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5632345221d0 0x56323452ce50 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f39d4009ca0 tx=0x7f39d4032340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aac39b1375099f3 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:37.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.243+0000 7f39c8c00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T10:56:37.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.243+0000 7f39c8c00640 1 -- 172.21.15.97:0/1628733726 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d4051030 con 0x5632345221d0 2024-03-27T10:56:37.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.244+0000 7f39c8c00640 1 -- 172.21.15.97:0/1628733726 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f39d4003f80 con 0x5632345221d0 2024-03-27T10:56:37.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.244+0000 7f39e4e54740 1 -- 172.21.15.97:0/1628733726 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39c402b070 con 0x5632345221d0 2024-03-27T10:56:37.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.244+0000 7f39c8c00640 1 -- 172.21.15.97:0/1628733726 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f39d4046040 con 0x5632345221d0 2024-03-27T10:56:37.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.244+0000 7f39e4e54740 10 client.15513 fetch_fsmap learned FSMap version 28 2024-03-27T10:56:37.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.244+0000 7f39e4e54740 1 -- 172.21.15.97:0/1628733726 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563234598ec0 con 0x5632345221d0 2024-03-27T10:56:37.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.244+0000 7f39c8c00640 1 -- 172.21.15.97:0/1628733726 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f39c402b070 con 0x5632345221d0 2024-03-27T10:56:37.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.244+0000 7f39c8c00640 1 -- 172.21.15.97:0/1628733726 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f39d404d990 con 0x5632345221d0 2024-03-27T10:56:37.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.244+0000 7f39e4e54740 10 client.15513 fetch_fsmap finished waiting for FSMap version 28 2024-03-27T10:56:37.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.244+0000 7f39e4e54740 10 client.15513 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-27T10:56:37.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.244+0000 7f39e4e54740 20 client.15513 populate_metadata read hostname 'smithi097' 2024-03-27T10:56:37.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.244+0000 7f39e4e54740 1 --2- 172.21.15.97:0/1628733726 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x563234532990 0x56323459b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:37.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.244+0000 7f39e4e54740 4 client.15513 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-27T10:56:37.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.244+0000 7f39e4e54740 1 -- 172.21.15.97:0/1628733726 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563234489f40 con 0x563234532990 2024-03-27T10:56:37.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.245+0000 7f39db400640 1 --2- 172.21.15.97:0/1628733726 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x563234532990 0x56323459b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:37.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.246+0000 7f39db400640 1 --2- 172.21.15.97:0/1628733726 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x563234532990 0x56323459b920 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ee2c81ef9f8247b server_cookie=50cf115751bc2348 in_seq=0 out_seq=0 2024-03-27T10:56:37.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.246+0000 7f39c8c00640 10 client.15513 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T10:56:37.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.246+0000 7f39c8c00640 1 -- 172.21.15.97:0/1628733726 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x563234489f40 con 0x563234532990 2024-03-27T10:56:37.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.246+0000 7f39c8c00640 10 client.15513 handle_command_reply: tid=0 2024-03-27T10:56:37.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.246+0000 7f39e4e54740 1 client.15513 shutdown 2024-03-27T10:56:37.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.246+0000 7f39e4e54740 1 -- 172.21.15.97:0/1628733726 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5632345221d0 msgr2=0x56323452ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:37.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.246+0000 7f39e4e54740 1 --2- 172.21.15.97:0/1628733726 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5632345221d0 0x56323452ce50 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f39d4009ca0 tx=0x7f39d4032340 comp rx=0 tx=0).stop 2024-03-27T10:56:37.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.246+0000 7f39e4e54740 1 -- 172.21.15.97:0/1628733726 shutdown_connections 2024-03-27T10:56:37.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.246+0000 7f39e4e54740 1 --2- 172.21.15.97:0/1628733726 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x563234532990 0x56323459b920 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:37.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.246+0000 7f39e4e54740 1 --2- 172.21.15.97:0/1628733726 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5632345221d0 0x56323452ce50 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:37.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.246+0000 7f39e4e54740 1 --2- 172.21.15.97:0/1628733726 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56323451d060 0x56323452be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:37.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.246+0000 7f39e4e54740 1 --2- 172.21.15.97:0/1628733726 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56323451c670 0x56323452b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:37.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.246+0000 7f39e4e54740 1 -- 172.21.15.97:0/1628733726 >> 172.21.15.97:0/1628733726 conn(0x563234517be0 msgr2=0x56323451b6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:37.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.246+0000 7f39c8c00640 0 client.15513 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T10:56:37.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.246+0000 7f39e4e54740 1 -- 172.21.15.97:0/1628733726 shutdown_connections 2024-03-27T10:56:37.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.246+0000 7f39e4e54740 1 -- 172.21.15.97:0/1628733726 wait complete. 2024-03-27T10:56:37.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.247+0000 7f39e4e54740 20 client.15513 trim_cache size 0 max 16384 2024-03-27T10:56:37.222 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-03-27T10:56:37.223 INFO:teuthology.orchestra.run.smithi097.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-27T10:56:37.223 INFO:teuthology.orchestra.run.smithi097.stdout: "scrubs": { 2024-03-27T10:56:37.223 INFO:teuthology.orchestra.run.smithi097.stdout: "fc430ced-914b-4e83-85df-1bfef281741a": { 2024-03-27T10:56:37.223 INFO:teuthology.orchestra.run.smithi097.stdout: "path": "/", 2024-03-27T10:56:37.223 INFO:teuthology.orchestra.run.smithi097.stdout: "tag": "fc430ced-914b-4e83-85df-1bfef281741a", 2024-03-27T10:56:37.223 INFO:teuthology.orchestra.run.smithi097.stdout: "options": "recursive,force" 2024-03-27T10:56:37.223 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T10:56:37.223 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T10:56:37.223 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-03-27T10:56:37.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.247+0000 7f39e0c00640 1 -- 172.21.15.97:0/3008364435 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f39c00a3690 msgr2=0x7f39c00a5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:37.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.247+0000 7f39e0c00640 1 --2- 172.21.15.97:0/3008364435 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f39c00a3690 0x7f39c00a5b50 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f39d4009a60 tx=0x7f39d4005d80 comp rx=0 tx=0).stop 2024-03-27T10:56:37.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.247+0000 7f39e0c00640 1 -- 172.21.15.97:0/3008364435 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39dc101a50 msgr2=0x7f39dc075900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:37.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.247+0000 7f39e0c00640 1 --2- 172.21.15.97:0/3008364435 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39dc101a50 0x7f39dc075900 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f39cc00e9c0 tx=0x7f39cc00ee80 comp rx=0 tx=0).stop 2024-03-27T10:56:37.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.247+0000 7f39e0c00640 1 -- 172.21.15.97:0/3008364435 shutdown_connections 2024-03-27T10:56:37.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.247+0000 7f39e0c00640 1 --2- 172.21.15.97:0/3008364435 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f39dc103600 0x7f39dc074480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:37.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.247+0000 7f39e0c00640 1 --2- 172.21.15.97:0/3008364435 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f39c00a3690 0x7f39c00a5b50 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:37.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.247+0000 7f39e0c00640 1 --2- 172.21.15.97:0/3008364435 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f39dc102c40 0x7f39dc073f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:37.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.247+0000 7f39e0c00640 1 --2- 172.21.15.97:0/3008364435 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39dc101a50 0x7f39dc075900 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:37.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.247+0000 7f39e0c00640 1 -- 172.21.15.97:0/3008364435 >> 172.21.15.97:0/3008364435 conn(0x7f39dc0fcfc0 msgr2=0x7f39dc105f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:37.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.247+0000 7f39e0c00640 1 -- 172.21.15.97:0/3008364435 shutdown_connections 2024-03-27T10:56:37.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:56:37.247+0000 7f39e0c00640 1 -- 172.21.15.97:0/3008364435 wait complete. 2024-03-27T10:56:37.236 INFO:tasks.cephfs.filesystem:scrub status for tag:fc430ced-914b-4e83-85df-1bfef281741a - {'path': '/', 'tag': 'fc430ced-914b-4e83-85df-1bfef281741a', 'options': 'recursive,force'} 2024-03-27T10:56:37.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:56:37 smithi097 ceph-mon[31859]: pgmap v440: 97 pgs: 97 active+clean; 651 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2024-03-27T10:56:37.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:56:37 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/2092636601' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T10:56:37.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:56:37 smithi177 ceph-mon[35294]: pgmap v440: 97 pgs: 97 active+clean; 651 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2024-03-27T10:56:37.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:56:37 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/2092636601' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T10:56:37.766 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:56:37 smithi114 ceph-mon[35338]: pgmap v440: 97 pgs: 97 active+clean; 651 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2024-03-27T10:56:37.766 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:56:37 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/2092636601' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T10:56:38.641 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:56:38 smithi097 ceph-mon[31859]: scrub queued for path: / 2024-03-27T10:56:38.641 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:56:38 smithi097 ceph-mon[31859]: scrub summary: idle+waiting paths [/] 2024-03-27T10:56:38.641 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:56:38 smithi097 ceph-mon[31859]: scrub summary: active paths [/] 2024-03-27T10:56:38.641 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:56:38 smithi097 ceph-mon[31859]: scrub summary: idle+waiting paths [/] 2024-03-27T10:56:38.641 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:56:38 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:56:38.641 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:56:38 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:56:38.641 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:56:38 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:56:38.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:56:38 smithi177 ceph-mon[35294]: scrub queued for path: / 2024-03-27T10:56:38.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:56:38 smithi177 ceph-mon[35294]: scrub summary: idle+waiting paths [/] 2024-03-27T10:56:38.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:56:38 smithi177 ceph-mon[35294]: scrub summary: active paths [/] 2024-03-27T10:56:38.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:56:38 smithi177 ceph-mon[35294]: scrub summary: idle+waiting paths [/] 2024-03-27T10:56:38.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:56:38 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:56:38.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:56:38 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:56:38.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:56:38 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:56:38.767 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:56:38 smithi114 ceph-mon[35338]: scrub queued for path: / 2024-03-27T10:56:38.767 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:56:38 smithi114 ceph-mon[35338]: scrub summary: idle+waiting paths [/] 2024-03-27T10:56:38.767 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:56:38 smithi114 ceph-mon[35338]: scrub summary: active paths [/] 2024-03-27T10:56:38.767 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:56:38 smithi114 ceph-mon[35338]: scrub summary: idle+waiting paths [/] 2024-03-27T10:56:38.767 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:56:38 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:56:38.767 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:56:38 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:56:38.767 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:56:38 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:56:39.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:56:39 smithi097 ceph-mon[31859]: pgmap v441: 97 pgs: 97 active+clean; 651 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-03-27T10:56:39.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:56:39 smithi177 ceph-mon[35294]: pgmap v441: 97 pgs: 97 active+clean; 651 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-03-27T10:56:39.766 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:56:39 smithi114 ceph-mon[35338]: pgmap v441: 97 pgs: 97 active+clean; 651 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-03-27T10:56:40.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:56:40 smithi097 ceph-mon[31859]: scrub summary: idle 2024-03-27T10:56:40.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:56:40 smithi177 ceph-mon[35294]: scrub summary: idle 2024-03-27T10:56:40.766 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:56:40 smithi114 ceph-mon[35338]: scrub summary: idle 2024-03-27T10:56:41.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:56:41 smithi097 ceph-mon[31859]: pgmap v442: 97 pgs: 97 active+clean; 656 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.8 KiB/s wr, 13 op/s 2024-03-27T10:56:41.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:56:41 smithi177 ceph-mon[35294]: pgmap v442: 97 pgs: 97 active+clean; 656 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.8 KiB/s wr, 13 op/s 2024-03-27T10:56:41.766 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:56:41 smithi114 ceph-mon[35338]: pgmap v442: 97 pgs: 97 active+clean; 656 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.8 KiB/s wr, 13 op/s 2024-03-27T10:56:43.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:56:43 smithi097 ceph-mon[31859]: pgmap v443: 97 pgs: 97 active+clean; 656 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-03-27T10:56:43.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:56:43 smithi177 ceph-mon[35294]: pgmap v443: 97 pgs: 97 active+clean; 656 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-03-27T10:56:43.766 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:56:43 smithi114 ceph-mon[35338]: pgmap v443: 97 pgs: 97 active+clean; 656 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-03-27T10:56:44.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:56:44 smithi097 ceph-mon[31859]: pgmap v444: 97 pgs: 97 active+clean; 656 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1023 B/s wr, 11 op/s 2024-03-27T10:56:44.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:56:44 smithi177 ceph-mon[35294]: pgmap v444: 97 pgs: 97 active+clean; 656 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1023 B/s wr, 11 op/s 2024-03-27T10:56:44.766 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:56:44 smithi114 ceph-mon[35338]: pgmap v444: 97 pgs: 97 active+clean; 656 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1023 B/s wr, 11 op/s 2024-03-27T10:56:47.426 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:56:47 smithi177 ceph-mon[35294]: pgmap v445: 97 pgs: 97 active+clean; 656 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 597 B/s wr, 9 op/s 2024-03-27T10:56:47.516 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:56:47 smithi114 ceph-mon[35338]: pgmap v445: 97 pgs: 97 active+clean; 656 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 597 B/s wr, 9 op/s 2024-03-27T10:56:47.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:56:47 smithi097 ceph-mon[31859]: pgmap v445: 97 pgs: 97 active+clean; 656 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 597 B/s wr, 9 op/s 2024-03-27T10:56:49.427 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:56:49 smithi177 ceph-mon[35294]: pgmap v446: 97 pgs: 97 active+clean; 656 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 9 op/s 2024-03-27T10:56:49.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:56:49 smithi114 ceph-mon[35338]: pgmap v446: 97 pgs: 97 active+clean; 656 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 9 op/s 2024-03-27T10:56:49.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:56:49 smithi097 ceph-mon[31859]: pgmap v446: 97 pgs: 97 active+clean; 656 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 9 op/s 2024-03-27T10:56:49.898 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.925+0000 7f8733200640 1 -- 172.21.15.97:0/118454628 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f872c103da0 msgr2=0x7f872c1041c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:49.898 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.925+0000 7f8733200640 1 --2- 172.21.15.97:0/118454628 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f872c103da0 0x7f872c1041c0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f871c009900 tx=0x7f871c02bc30 comp rx=0 tx=0).stop 2024-03-27T10:56:49.898 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.925+0000 7f8733200640 1 -- 172.21.15.97:0/118454628 shutdown_connections 2024-03-27T10:56:49.898 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.925+0000 7f8733200640 1 --2- 172.21.15.97:0/118454628 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f872c105950 0x7f872c10c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:49.898 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.925+0000 7f8733200640 1 --2- 172.21.15.97:0/118454628 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f872c104f90 0x7f872c105410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:49.899 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.925+0000 7f8733200640 1 --2- 172.21.15.97:0/118454628 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f872c103da0 0x7f872c1041c0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:49.899 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.925+0000 7f8733200640 1 -- 172.21.15.97:0/118454628 >> 172.21.15.97:0/118454628 conn(0x7f872c0ff310 msgr2=0x7f872c101750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:49.899 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.926+0000 7f8733200640 1 -- 172.21.15.97:0/118454628 shutdown_connections 2024-03-27T10:56:49.899 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.926+0000 7f8733200640 1 -- 172.21.15.97:0/118454628 wait complete. 2024-03-27T10:56:49.901 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.926+0000 7f8733200640 1 Processor -- start 2024-03-27T10:56:49.901 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.927+0000 7f8733200640 1 -- start start 2024-03-27T10:56:49.901 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.927+0000 7f8733200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f872c103da0 0x7f872c199f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:49.901 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.927+0000 7f8733200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f872c104f90 0x7f872c19a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:49.901 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.927+0000 7f8733200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f872c105950 0x7f872c1a14b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:49.901 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.927+0000 7f8733200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f872c10f920 con 0x7f872c105950 2024-03-27T10:56:49.901 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.927+0000 7f8733200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f872c10f7a0 con 0x7f872c104f90 2024-03-27T10:56:49.901 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.927+0000 7f8733200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f872c10faa0 con 0x7f872c103da0 2024-03-27T10:56:49.901 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.927+0000 7f8731200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f872c105950 0x7f872c1a14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:49.902 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.927+0000 7f8731200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f872c105950 0x7f872c1a14b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57460/0 (socket says 172.21.15.97:57460) 2024-03-27T10:56:49.902 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.927+0000 7f8731200640 1 -- 172.21.15.97:0/3978668118 learned_addr learned my addr 172.21.15.97:0/3978668118 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:56:49.902 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.927+0000 7f872be00640 1 --2- 172.21.15.97:0/3978668118 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f872c103da0 0x7f872c199f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:49.902 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.927+0000 7f8723e00640 1 --2- 172.21.15.97:0/3978668118 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f872c104f90 0x7f872c19a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:49.902 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.928+0000 7f8731200640 1 -- 172.21.15.97:0/3978668118 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f872c103da0 msgr2=0x7f872c199f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:49.902 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.928+0000 7f8731200640 1 --2- 172.21.15.97:0/3978668118 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f872c103da0 0x7f872c199f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:49.902 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.928+0000 7f8731200640 1 -- 172.21.15.97:0/3978668118 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f872c104f90 msgr2=0x7f872c19a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:49.902 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.928+0000 7f8731200640 1 --2- 172.21.15.97:0/3978668118 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f872c104f90 0x7f872c19a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:49.902 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.928+0000 7f8731200640 1 -- 172.21.15.97:0/3978668118 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f872c1a1b20 con 0x7f872c105950 2024-03-27T10:56:49.902 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.928+0000 7f8723e00640 1 --2- 172.21.15.97:0/3978668118 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f872c104f90 0x7f872c19a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T10:56:49.902 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.928+0000 7f8731200640 1 --2- 172.21.15.97:0/3978668118 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f872c105950 0x7f872c1a14b0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f872400eb10 tx=0x7f872400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:49.902 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.928+0000 7f8729600640 1 -- 172.21.15.97:0/3978668118 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8724004b20 con 0x7f872c105950 2024-03-27T10:56:49.903 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.928+0000 7f8729600640 1 -- 172.21.15.97:0/3978668118 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8724004cc0 con 0x7f872c105950 2024-03-27T10:56:49.903 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.928+0000 7f8729600640 1 -- 172.21.15.97:0/3978668118 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8724010690 con 0x7f872c105950 2024-03-27T10:56:49.903 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.928+0000 7f8733200640 1 -- 172.21.15.97:0/3978668118 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f872c1a1e10 con 0x7f872c105950 2024-03-27T10:56:49.903 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.929+0000 7f8733200640 1 -- 172.21.15.97:0/3978668118 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f872c1a2320 con 0x7f872c105950 2024-03-27T10:56:49.906 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.932+0000 7f8729600640 1 -- 172.21.15.97:0/3978668118 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f8724005020 con 0x7f872c105950 2024-03-27T10:56:49.906 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.932+0000 7f8735fc3740 1 Processor -- start 2024-03-27T10:56:49.906 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.932+0000 7f8735fc3740 1 -- start start 2024-03-27T10:56:49.906 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.932+0000 7f8735fc3740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc40a2d980 0x55cc40a2dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:49.906 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.932+0000 7f8735fc3740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cc40a2e370 0x55cc40a32e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:49.906 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.932+0000 7f8735fc3740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cc40a334e0 0x55cc40a35970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:49.906 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.932+0000 7f8735fc3740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55cc40992d10 con 0x55cc40a2d980 2024-03-27T10:56:49.906 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.932+0000 7f8735fc3740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55cc4093b260 con 0x55cc40a334e0 2024-03-27T10:56:49.906 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.932+0000 7f8735fc3740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55cc4098ecc0 con 0x55cc40a2e370 2024-03-27T10:56:49.906 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.932+0000 7f8723e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc40a2d980 0x55cc40a2dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:49.906 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.932+0000 7f8723e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc40a2d980 0x55cc40a2dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57476/0 (socket says 172.21.15.97:57476) 2024-03-27T10:56:49.907 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.933+0000 7f8723e00640 1 -- 172.21.15.97:0/561040781 learned_addr learned my addr 172.21.15.97:0/561040781 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:56:49.907 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.933+0000 7f872be00640 1 --2- 172.21.15.97:0/561040781 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cc40a2e370 0x55cc40a32e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:49.907 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.933+0000 7f8723e00640 1 -- 172.21.15.97:0/561040781 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cc40a2e370 msgr2=0x55cc40a32e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:56:49.907 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.933+0000 7f8723e00640 1 --2- 172.21.15.97:0/561040781 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cc40a2e370 0x55cc40a32e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:49.907 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.933+0000 7f8723e00640 1 -- 172.21.15.97:0/561040781 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cc40a334e0 msgr2=0x55cc40a35970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:56:49.907 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.933+0000 7f8723e00640 1 --2- 172.21.15.97:0/561040781 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cc40a334e0 0x55cc40a35970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:49.907 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.933+0000 7f8723e00640 1 -- 172.21.15.97:0/561040781 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc40932aa0 con 0x55cc40a2d980 2024-03-27T10:56:49.907 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.933+0000 7f872be00640 1 --2- 172.21.15.97:0/561040781 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cc40a2e370 0x55cc40a32e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T10:56:49.907 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.933+0000 7f8729600640 1 --2- 172.21.15.97:0/3978668118 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f87100a3730 0x7f87100a5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:49.907 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.933+0000 7f8723e00640 1 --2- 172.21.15.97:0/561040781 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc40a2d980 0x55cc40a2dda0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f871400bde0 tx=0x7f8714011750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50f6f36445f02233 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:49.907 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.933+0000 7f8729600640 1 -- 172.21.15.97:0/3978668118 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f87240d03e0 con 0x7f872c105950 2024-03-27T10:56:49.907 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.933+0000 7f8721600640 1 -- 172.21.15.97:0/561040781 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8714016070 con 0x55cc40a2d980 2024-03-27T10:56:49.908 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.933+0000 7f872be00640 1 --2- 172.21.15.97:0/3978668118 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f87100a3730 0x7f87100a5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:49.908 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.934+0000 7f8721600640 1 -- 172.21.15.97:0/561040781 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f87140054d0 con 0x55cc40a2d980 2024-03-27T10:56:49.908 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.934+0000 7f8721600640 1 -- 172.21.15.97:0/561040781 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8714004cb0 con 0x55cc40a2d980 2024-03-27T10:56:49.912 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.934+0000 7f8735fc3740 1 -- 172.21.15.97:0/561040781 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc40a2d980 msgr2=0x55cc40a2dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:49.912 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.934+0000 7f8735fc3740 1 --2- 172.21.15.97:0/561040781 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc40a2d980 0x55cc40a2dda0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f871400bde0 tx=0x7f8714011750 comp rx=0 tx=0).stop 2024-03-27T10:56:49.912 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.934+0000 7f872be00640 1 --2- 172.21.15.97:0/3978668118 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f87100a3730 0x7f87100a5bf0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f871c02c240 tx=0x7f871c02c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:49.912 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.934+0000 7f8735fc3740 1 -- 172.21.15.97:0/561040781 shutdown_connections 2024-03-27T10:56:49.912 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.934+0000 7f8735fc3740 1 --2- 172.21.15.97:0/561040781 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cc40a334e0 0x55cc40a35970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:49.912 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.934+0000 7f8735fc3740 1 --2- 172.21.15.97:0/561040781 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cc40a2e370 0x55cc40a32e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:49.912 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.934+0000 7f8735fc3740 1 --2- 172.21.15.97:0/561040781 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc40a2d980 0x55cc40a2dda0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:49.913 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.934+0000 7f8735fc3740 1 -- 172.21.15.97:0/561040781 >> 172.21.15.97:0/561040781 conn(0x55cc40a28ef0 msgr2=0x55cc40a292f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:49.913 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.934+0000 7f8735fc3740 1 -- 172.21.15.97:0/561040781 shutdown_connections 2024-03-27T10:56:49.913 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.934+0000 7f8735fc3740 1 -- 172.21.15.97:0/561040781 wait complete. 2024-03-27T10:56:49.913 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.935+0000 7f8735fc3740 1 -- start start 2024-03-27T10:56:49.913 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.935+0000 7f8735fc3740 1 Processor -- start 2024-03-27T10:56:49.913 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.935+0000 7f8735fc3740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cc40a2d980 0x55cc40a37c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:49.913 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.935+0000 7f8735fc3740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cc40a2e370 0x55cc40a381d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:49.913 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.935+0000 7f8735fc3740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc40a334e0 0x55cc40a3f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:49.913 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.935+0000 7f8735fc3740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55cc40a36490 con 0x55cc40a334e0 2024-03-27T10:56:49.913 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.936+0000 7f8735fc3740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55cc4094e630 con 0x55cc40a2e370 2024-03-27T10:56:49.913 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.936+0000 7f8735fc3740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55cc40a36610 con 0x55cc40a2d980 2024-03-27T10:56:49.913 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.936+0000 7f8735fc3740 10 client.0 _ll_register_callbacks cb 0x55cc40953c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:56:49.914 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.936+0000 7f8731200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc40a334e0 0x55cc40a3f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:49.914 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.936+0000 7f8731200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc40a334e0 0x55cc40a3f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57492/0 (socket says 172.21.15.97:57492) 2024-03-27T10:56:49.914 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.936+0000 7f8731200640 1 -- 172.21.15.97:0/1151061660 learned_addr learned my addr 172.21.15.97:0/1151061660 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:56:49.914 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.936+0000 7f872be00640 1 --2- 172.21.15.97:0/1151061660 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cc40a2d980 0x55cc40a37c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:49.914 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.936+0000 7f8723e00640 1 --2- 172.21.15.97:0/1151061660 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cc40a2e370 0x55cc40a381d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:49.914 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.936+0000 7f8731200640 1 -- 172.21.15.97:0/1151061660 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cc40a2d980 msgr2=0x55cc40a37c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:56:49.914 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.936+0000 7f8731200640 1 --2- 172.21.15.97:0/1151061660 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cc40a2d980 0x55cc40a37c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:49.914 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.936+0000 7f8731200640 1 -- 172.21.15.97:0/1151061660 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cc40a2e370 msgr2=0x55cc40a381d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:56:49.914 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.936+0000 7f8731200640 1 --2- 172.21.15.97:0/1151061660 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cc40a2e370 0x55cc40a381d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:49.914 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.936+0000 7f8731200640 1 -- 172.21.15.97:0/1151061660 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc4095a5f0 con 0x55cc40a334e0 2024-03-27T10:56:49.914 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.936+0000 7f8731200640 1 --2- 172.21.15.97:0/1151061660 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc40a334e0 0x55cc40a3f210 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f8724000f80 tx=0x7f872409c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa19a7a0ec35aa37 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:49.915 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.936+0000 7f871a000640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T10:56:49.915 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.937+0000 7f871a000640 1 -- 172.21.15.97:0/1151061660 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87240b3070 con 0x55cc40a334e0 2024-03-27T10:56:49.915 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.937+0000 7f871a000640 1 -- 172.21.15.97:0/1151061660 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f87240a5040 con 0x55cc40a334e0 2024-03-27T10:56:49.915 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.937+0000 7f8735fc3740 1 -- 172.21.15.97:0/1151061660 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87140054d0 con 0x55cc40a334e0 2024-03-27T10:56:49.915 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.937+0000 7f871a000640 1 -- 172.21.15.97:0/1151061660 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f872409c860 con 0x55cc40a334e0 2024-03-27T10:56:49.915 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.937+0000 7f8735fc3740 10 client.15525 fetch_fsmap learned FSMap version 28 2024-03-27T10:56:49.915 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.937+0000 7f8735fc3740 1 -- 172.21.15.97:0/1151061660 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55cc406a7210 con 0x55cc40a334e0 2024-03-27T10:56:49.915 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.937+0000 7f871a000640 1 -- 172.21.15.97:0/1151061660 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f87140054d0 con 0x55cc40a334e0 2024-03-27T10:56:49.915 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.937+0000 7f871a000640 1 -- 172.21.15.97:0/1151061660 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f872409db90 con 0x55cc40a334e0 2024-03-27T10:56:49.915 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.937+0000 7f8735fc3740 10 client.15525 fetch_fsmap finished waiting for FSMap version 28 2024-03-27T10:56:49.915 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.937+0000 7f8735fc3740 10 client.15525 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T10:56:49.915 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.937+0000 7f8735fc3740 20 client.15525 populate_metadata read hostname 'smithi097' 2024-03-27T10:56:49.916 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.938+0000 7f8735fc3740 1 --2- 172.21.15.97:0/1151061660 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55cc40a43bc0 0x55cc40a45ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:49.916 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.938+0000 7f8735fc3740 4 client.15525 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T10:56:49.916 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.938+0000 7f8735fc3740 1 -- 172.21.15.97:0/1151061660 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cc409a0250 con 0x55cc40a43bc0 2024-03-27T10:56:49.916 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.938+0000 7f872be00640 1 --2- 172.21.15.97:0/1151061660 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55cc40a43bc0 0x55cc40a45ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:49.916 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.939+0000 7f872be00640 1 --2- 172.21.15.97:0/1151061660 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55cc40a43bc0 0x55cc40a45ff0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79278e71c3554779 server_cookie=933799068162f1ee in_seq=0 out_seq=0 2024-03-27T10:56:49.916 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.939+0000 7f871a000640 10 client.15525 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T10:56:49.917 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.942+0000 7f871a000640 1 -- 172.21.15.97:0/1151061660 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55cc409a0250 con 0x55cc40a43bc0 2024-03-27T10:56:49.917 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.942+0000 7f871a000640 10 client.15525 handle_command_reply: tid=0 2024-03-27T10:56:49.917 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.942+0000 7f8735fc3740 1 client.15525 shutdown 2024-03-27T10:56:49.917 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.942+0000 7f8735fc3740 1 -- 172.21.15.97:0/1151061660 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc40a334e0 msgr2=0x55cc40a3f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:49.918 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.942+0000 7f8735fc3740 1 --2- 172.21.15.97:0/1151061660 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc40a334e0 0x55cc40a3f210 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f8724000f80 tx=0x7f872409c3a0 comp rx=0 tx=0).stop 2024-03-27T10:56:49.918 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.942+0000 7f8735fc3740 1 -- 172.21.15.97:0/1151061660 shutdown_connections 2024-03-27T10:56:49.918 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.942+0000 7f8735fc3740 1 --2- 172.21.15.97:0/1151061660 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55cc40a43bc0 0x55cc40a45ff0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:49.918 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.942+0000 7f8735fc3740 1 --2- 172.21.15.97:0/1151061660 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc40a334e0 0x55cc40a3f210 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:49.918 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.942+0000 7f8735fc3740 1 --2- 172.21.15.97:0/1151061660 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cc40a2e370 0x55cc40a381d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:49.918 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.942+0000 7f871a000640 0 client.15525 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T10:56:49.918 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.942+0000 7f8735fc3740 1 --2- 172.21.15.97:0/1151061660 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cc40a2d980 0x55cc40a37c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:49.918 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.942+0000 7f8735fc3740 1 -- 172.21.15.97:0/1151061660 >> 172.21.15.97:0/1151061660 conn(0x55cc40a28ef0 msgr2=0x55cc40a280f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:49.918 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.943+0000 7f8735fc3740 1 -- 172.21.15.97:0/1151061660 shutdown_connections 2024-03-27T10:56:49.918 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.943+0000 7f8735fc3740 1 -- 172.21.15.97:0/1151061660 wait complete. 2024-03-27T10:56:49.918 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.943+0000 7f8735fc3740 20 client.15525 trim_cache size 0 max 16384 2024-03-27T10:56:49.932 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.959+0000 7f8735fc3740 1 Processor -- start 2024-03-27T10:56:49.932 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.959+0000 7f8735fc3740 1 -- start start 2024-03-27T10:56:49.932 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.959+0000 7f8735fc3740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc40a2d980 0x55cc40a23f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:49.932 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.959+0000 7f8735fc3740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cc40a2e370 0x55cc40a3f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:49.932 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.959+0000 7f8735fc3740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cc40a334e0 0x55cc40a2c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:49.932 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.959+0000 7f8735fc3740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f872c1ab1b0 con 0x55cc40a2d980 2024-03-27T10:56:49.932 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.959+0000 7f8735fc3740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f872c1accf0 con 0x55cc40a334e0 2024-03-27T10:56:49.932 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.959+0000 7f8735fc3740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f872c1ab880 con 0x55cc40a2e370 2024-03-27T10:56:49.932 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.959+0000 7f872be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc40a2d980 0x55cc40a23f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:49.933 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.959+0000 7f872be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc40a2d980 0x55cc40a23f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57502/0 (socket says 172.21.15.97:57502) 2024-03-27T10:56:49.933 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.959+0000 7f872be00640 1 -- 172.21.15.97:0/1535546305 learned_addr learned my addr 172.21.15.97:0/1535546305 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:56:49.933 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.959+0000 7f8723e00640 1 --2- 172.21.15.97:0/1535546305 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cc40a2e370 0x55cc40a3f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:49.933 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.959+0000 7f8731200640 1 --2- 172.21.15.97:0/1535546305 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cc40a334e0 0x55cc40a2c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:49.933 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.960+0000 7f872be00640 1 -- 172.21.15.97:0/1535546305 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cc40a2e370 msgr2=0x55cc40a3f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:56:49.933 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.960+0000 7f872be00640 1 --2- 172.21.15.97:0/1535546305 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cc40a2e370 0x55cc40a3f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:49.933 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.960+0000 7f872be00640 1 -- 172.21.15.97:0/1535546305 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cc40a334e0 msgr2=0x55cc40a2c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:56:49.933 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.960+0000 7f872be00640 1 --2- 172.21.15.97:0/1535546305 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cc40a334e0 0x55cc40a2c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:49.933 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.960+0000 7f872be00640 1 -- 172.21.15.97:0/1535546305 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc40773620 con 0x55cc40a2d980 2024-03-27T10:56:49.933 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.960+0000 7f872be00640 1 --2- 172.21.15.97:0/1535546305 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc40a2d980 0x55cc40a23f50 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f871c006010 tx=0x7f871c044000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=578a41cd0da4d00 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:49.937 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.960+0000 7f8720c00640 1 -- 172.21.15.97:0/1535546305 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f871c05b070 con 0x55cc40a2d980 2024-03-27T10:56:49.937 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.960+0000 7f8720c00640 1 -- 172.21.15.97:0/1535546305 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f871c004260 con 0x55cc40a2d980 2024-03-27T10:56:49.937 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.960+0000 7f8720c00640 1 -- 172.21.15.97:0/1535546305 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f871c005240 con 0x55cc40a2d980 2024-03-27T10:56:49.937 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.961+0000 7f8735fc3740 1 -- 172.21.15.97:0/1535546305 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc40a2d980 msgr2=0x55cc40a23f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:49.937 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.961+0000 7f8735fc3740 1 --2- 172.21.15.97:0/1535546305 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc40a2d980 0x55cc40a23f50 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f871c006010 tx=0x7f871c044000 comp rx=0 tx=0).stop 2024-03-27T10:56:49.937 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.961+0000 7f8735fc3740 1 -- 172.21.15.97:0/1535546305 shutdown_connections 2024-03-27T10:56:49.937 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.961+0000 7f8735fc3740 1 --2- 172.21.15.97:0/1535546305 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cc40a334e0 0x55cc40a2c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:49.937 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.961+0000 7f8735fc3740 1 --2- 172.21.15.97:0/1535546305 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cc40a2e370 0x55cc40a3f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:49.937 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.961+0000 7f8735fc3740 1 --2- 172.21.15.97:0/1535546305 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc40a2d980 0x55cc40a23f50 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:49.937 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.961+0000 7f8735fc3740 1 -- 172.21.15.97:0/1535546305 >> 172.21.15.97:0/1535546305 conn(0x55cc40a28ef0 msgr2=0x55cc40a2c150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:49.937 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.961+0000 7f8735fc3740 1 -- 172.21.15.97:0/1535546305 shutdown_connections 2024-03-27T10:56:49.937 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.961+0000 7f8735fc3740 1 -- 172.21.15.97:0/1535546305 wait complete. 2024-03-27T10:56:49.938 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.962+0000 7f8735fc3740 1 -- start start 2024-03-27T10:56:49.938 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.962+0000 7f8735fc3740 1 Processor -- start 2024-03-27T10:56:49.938 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.962+0000 7f8735fc3740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cc40a2d980 0x55cc40a3ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:49.938 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.962+0000 7f8735fc3740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cc40a2e370 0x55cc40a3efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:49.938 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.962+0000 7f8735fc3740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc40a334e0 0x55cc40a3d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:49.938 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.962+0000 7f8735fc3740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f872c1ac620 con 0x55cc40a334e0 2024-03-27T10:56:49.938 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.962+0000 7f8735fc3740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f872c1abf50 con 0x55cc40a2d980 2024-03-27T10:56:49.938 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.962+0000 7f8735fc3740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f872c1a9190 con 0x55cc40a2e370 2024-03-27T10:56:49.938 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.962+0000 7f8735fc3740 10 client.0 _ll_register_callbacks cb 0x7f8724098b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:56:49.939 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.962+0000 7f8731200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc40a334e0 0x55cc40a3d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:49.939 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.962+0000 7f8731200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc40a334e0 0x55cc40a3d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57512/0 (socket says 172.21.15.97:57512) 2024-03-27T10:56:49.939 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.962+0000 7f8731200640 1 -- 172.21.15.97:0/2652169142 learned_addr learned my addr 172.21.15.97:0/2652169142 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:56:49.939 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.963+0000 7f872be00640 1 --2- 172.21.15.97:0/2652169142 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cc40a2d980 0x55cc40a3ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:49.939 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.963+0000 7f8723e00640 1 --2- 172.21.15.97:0/2652169142 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cc40a2e370 0x55cc40a3efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:49.939 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.963+0000 7f8731200640 1 -- 172.21.15.97:0/2652169142 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cc40a2e370 msgr2=0x55cc40a3efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:56:49.939 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.963+0000 7f8731200640 1 --2- 172.21.15.97:0/2652169142 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cc40a2e370 0x55cc40a3efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:49.939 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.963+0000 7f8731200640 1 -- 172.21.15.97:0/2652169142 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cc40a2d980 msgr2=0x55cc40a3ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:56:49.939 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.963+0000 7f8731200640 1 --2- 172.21.15.97:0/2652169142 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cc40a2d980 0x55cc40a3ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:49.939 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.963+0000 7f8731200640 1 -- 172.21.15.97:0/2652169142 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc40a3d830 con 0x55cc40a334e0 2024-03-27T10:56:49.939 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.963+0000 7f8731200640 1 --2- 172.21.15.97:0/2652169142 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc40a334e0 0x55cc40a3d140 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f872409c7f0 tx=0x7f8724005600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d778c468956dad37 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:56:49.939 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.963+0000 7f871a000640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T10:56:49.939 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.963+0000 7f871a000640 1 -- 172.21.15.97:0/2652169142 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87240b3070 con 0x55cc40a334e0 2024-03-27T10:56:49.939 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.963+0000 7f871a000640 1 -- 172.21.15.97:0/2652169142 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f872409ab60 con 0x55cc40a334e0 2024-03-27T10:56:49.940 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.963+0000 7f8735fc3740 1 -- 172.21.15.97:0/2652169142 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f871c004260 con 0x55cc40a334e0 2024-03-27T10:56:49.940 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.963+0000 7f871a000640 1 -- 172.21.15.97:0/2652169142 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f872409c1c0 con 0x55cc40a334e0 2024-03-27T10:56:49.940 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.963+0000 7f8735fc3740 10 client.15531 fetch_fsmap learned FSMap version 28 2024-03-27T10:56:49.940 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.963+0000 7f8735fc3740 1 -- 172.21.15.97:0/2652169142 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55cc40aaa970 con 0x55cc40a334e0 2024-03-27T10:56:49.940 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.964+0000 7f871a000640 1 -- 172.21.15.97:0/2652169142 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f871c004260 con 0x55cc40a334e0 2024-03-27T10:56:49.940 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.964+0000 7f871a000640 1 -- 172.21.15.97:0/2652169142 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f8724082070 con 0x55cc40a334e0 2024-03-27T10:56:49.940 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.964+0000 7f8735fc3740 10 client.15531 fetch_fsmap finished waiting for FSMap version 28 2024-03-27T10:56:49.940 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.964+0000 7f8735fc3740 10 client.15531 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T10:56:49.940 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.964+0000 7f8735fc3740 20 client.15531 populate_metadata read hostname 'smithi097' 2024-03-27T10:56:49.940 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.964+0000 7f8735fc3740 1 --2- 172.21.15.97:0/2652169142 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55cc40aab190 0x55cc40aad5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:56:49.940 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.964+0000 7f8735fc3740 4 client.15531 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-27T10:56:49.940 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.964+0000 7f8735fc3740 1 -- 172.21.15.97:0/2652169142 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cc4099a5a0 con 0x55cc40aab190 2024-03-27T10:56:49.940 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.965+0000 7f872be00640 1 --2- 172.21.15.97:0/2652169142 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55cc40aab190 0x55cc40aad5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:56:49.941 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.966+0000 7f872be00640 1 --2- 172.21.15.97:0/2652169142 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55cc40aab190 0x55cc40aad5b0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e50b48e8fb960b6e server_cookie=7a566488126c6d0b in_seq=0 out_seq=0 2024-03-27T10:56:49.941 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.966+0000 7f871a000640 10 client.15531 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T10:56:49.960 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.987+0000 7f871a000640 1 -- 172.21.15.97:0/2652169142 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cc4099a5a0 con 0x55cc40aab190 2024-03-27T10:56:49.960 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.987+0000 7f871a000640 10 client.15531 handle_command_reply: tid=0 2024-03-27T10:56:49.961 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.987+0000 7f8735fc3740 1 client.15531 shutdown 2024-03-27T10:56:49.961 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.988+0000 7f8735fc3740 1 -- 172.21.15.97:0/2652169142 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc40a334e0 msgr2=0x55cc40a3d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:49.961 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.988+0000 7f8735fc3740 1 --2- 172.21.15.97:0/2652169142 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc40a334e0 0x55cc40a3d140 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f872409c7f0 tx=0x7f8724005600 comp rx=0 tx=0).stop 2024-03-27T10:56:49.961 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.988+0000 7f8735fc3740 1 -- 172.21.15.97:0/2652169142 shutdown_connections 2024-03-27T10:56:49.961 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.988+0000 7f8735fc3740 1 --2- 172.21.15.97:0/2652169142 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55cc40aab190 0x55cc40aad5b0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:49.961 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.988+0000 7f8735fc3740 1 --2- 172.21.15.97:0/2652169142 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc40a334e0 0x55cc40a3d140 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:49.961 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.988+0000 7f8735fc3740 1 --2- 172.21.15.97:0/2652169142 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cc40a2e370 0x55cc40a3efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:49.961 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.988+0000 7f8735fc3740 1 --2- 172.21.15.97:0/2652169142 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cc40a2d980 0x55cc40a3ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:49.962 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.988+0000 7f871a000640 0 client.15531 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T10:56:49.962 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.988+0000 7f8735fc3740 1 -- 172.21.15.97:0/2652169142 >> 172.21.15.97:0/2652169142 conn(0x55cc40a28ef0 msgr2=0x55cc40a2c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:49.962 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.988+0000 7f8735fc3740 1 -- 172.21.15.97:0/2652169142 shutdown_connections 2024-03-27T10:56:49.962 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.988+0000 7f8735fc3740 1 -- 172.21.15.97:0/2652169142 wait complete. 2024-03-27T10:56:49.962 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.988+0000 7f8735fc3740 20 client.15531 trim_cache size 0 max 16384 2024-03-27T10:56:49.963 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-27T10:56:49.963 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-27T10:56:49.963 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-27T10:56:49.963 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-27T10:56:49.963 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.989+0000 7f8733200640 1 -- 172.21.15.97:0/3978668118 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f87100a3730 msgr2=0x7f87100a5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:49.964 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.989+0000 7f8733200640 1 --2- 172.21.15.97:0/3978668118 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f87100a3730 0x7f87100a5bf0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f871c02c240 tx=0x7f871c02c7e0 comp rx=0 tx=0).stop 2024-03-27T10:56:49.964 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.989+0000 7f8733200640 1 -- 172.21.15.97:0/3978668118 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f872c105950 msgr2=0x7f872c1a14b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:56:49.964 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.989+0000 7f8733200640 1 --2- 172.21.15.97:0/3978668118 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f872c105950 0x7f872c1a14b0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f872400eb10 tx=0x7f872400c2d0 comp rx=0 tx=0).stop 2024-03-27T10:56:49.964 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.989+0000 7f8733200640 1 -- 172.21.15.97:0/3978668118 shutdown_connections 2024-03-27T10:56:49.964 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.989+0000 7f8733200640 1 --2- 172.21.15.97:0/3978668118 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f872c105950 0x7f872c1a14b0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:49.964 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.989+0000 7f8733200640 1 --2- 172.21.15.97:0/3978668118 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f87100a3730 0x7f87100a5bf0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:49.964 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.989+0000 7f8733200640 1 --2- 172.21.15.97:0/3978668118 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f872c104f90 0x7f872c19a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:49.964 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.989+0000 7f8733200640 1 --2- 172.21.15.97:0/3978668118 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f872c103da0 0x7f872c199f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:56:49.964 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.989+0000 7f8733200640 1 -- 172.21.15.97:0/3978668118 >> 172.21.15.97:0/3978668118 conn(0x7f872c0ff310 msgr2=0x7f872c108290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:56:49.964 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.990+0000 7f8733200640 1 -- 172.21.15.97:0/3978668118 shutdown_connections 2024-03-27T10:56:49.964 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:56:49.990+0000 7f8733200640 1 -- 172.21.15.97:0/3978668118 wait complete. 2024-03-27T10:56:51.426 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:56:51 smithi177 ceph-mon[35294]: pgmap v447: 97 pgs: 97 active+clean; 661 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-03-27T10:56:51.516 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:56:51 smithi114 ceph-mon[35338]: pgmap v447: 97 pgs: 97 active+clean; 661 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-03-27T10:56:51.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:56:51 smithi097 ceph-mon[31859]: pgmap v447: 97 pgs: 97 active+clean; 661 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-03-27T10:56:53.426 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:56:53 smithi177 ceph-mon[35294]: pgmap v448: 97 pgs: 97 active+clean; 661 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 8 op/s 2024-03-27T10:56:53.516 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:56:53 smithi114 ceph-mon[35338]: pgmap v448: 97 pgs: 97 active+clean; 661 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 8 op/s 2024-03-27T10:56:53.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:56:53 smithi097 ceph-mon[31859]: pgmap v448: 97 pgs: 97 active+clean; 661 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 8 op/s 2024-03-27T10:56:55.426 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:56:55 smithi177 ceph-mon[35294]: pgmap v449: 97 pgs: 97 active+clean; 662 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.8 KiB/s wr, 14 op/s 2024-03-27T10:56:55.516 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:56:55 smithi114 ceph-mon[35338]: pgmap v449: 97 pgs: 97 active+clean; 662 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.8 KiB/s wr, 14 op/s 2024-03-27T10:56:55.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:56:55 smithi097 ceph-mon[31859]: pgmap v449: 97 pgs: 97 active+clean; 662 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.8 KiB/s wr, 14 op/s 2024-03-27T10:56:57.516 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:56:57 smithi114 ceph-mon[35338]: pgmap v450: 97 pgs: 97 active+clean; 662 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.1 KiB/s wr, 14 op/s 2024-03-27T10:56:57.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:56:57 smithi097 ceph-mon[31859]: pgmap v450: 97 pgs: 97 active+clean; 662 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.1 KiB/s wr, 14 op/s 2024-03-27T10:56:57.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:56:57 smithi177 ceph-mon[35294]: pgmap v450: 97 pgs: 97 active+clean; 662 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.1 KiB/s wr, 14 op/s 2024-03-27T10:56:59.516 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:56:59 smithi114 ceph-mon[35338]: pgmap v451: 97 pgs: 97 active+clean; 662 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.1 KiB/s wr, 13 op/s 2024-03-27T10:56:59.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:56:59 smithi097 ceph-mon[31859]: pgmap v451: 97 pgs: 97 active+clean; 662 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.1 KiB/s wr, 13 op/s 2024-03-27T10:56:59.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:56:59 smithi177 ceph-mon[35294]: pgmap v451: 97 pgs: 97 active+clean; 662 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.1 KiB/s wr, 13 op/s 2024-03-27T10:57:01.445 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:57:01 smithi097 ceph-mon[31859]: pgmap v452: 97 pgs: 97 active+clean; 662 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.0 KiB/s wr, 16 op/s 2024-03-27T10:57:01.516 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:57:01 smithi114 ceph-mon[35338]: pgmap v452: 97 pgs: 97 active+clean; 662 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.0 KiB/s wr, 16 op/s 2024-03-27T10:57:01.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:57:01 smithi177 ceph-mon[35294]: pgmap v452: 97 pgs: 97 active+clean; 662 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.0 KiB/s wr, 16 op/s 2024-03-27T10:57:03.098 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.125+0000 7ff463e00640 1 -- 172.21.15.97:0/3964141289 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff45c1078b0 msgr2=0x7ff45c109ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:03.098 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.125+0000 7ff463e00640 1 --2- 172.21.15.97:0/3964141289 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff45c1078b0 0x7ff45c109ca0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7ff454009ca0 tx=0x7ff45402c0f0 comp rx=0 tx=0).stop 2024-03-27T10:57:03.098 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.125+0000 7ff463e00640 1 -- 172.21.15.97:0/3964141289 shutdown_connections 2024-03-27T10:57:03.098 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.125+0000 7ff463e00640 1 --2- 172.21.15.97:0/3964141289 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff45c1078b0 0x7ff45c109ca0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:03.098 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.125+0000 7ff463e00640 1 --2- 172.21.15.97:0/3964141289 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff45c104f80 0x7ff45c107370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:03.098 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.125+0000 7ff463e00640 1 --2- 172.21.15.97:0/3964141289 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff45c102650 0x7ff45c104a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:03.098 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.125+0000 7ff463e00640 1 -- 172.21.15.97:0/3964141289 >> 172.21.15.97:0/3964141289 conn(0x7ff45c0fbfe0 msgr2=0x7ff45c0fe420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:03.098 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.125+0000 7ff463e00640 1 -- 172.21.15.97:0/3964141289 shutdown_connections 2024-03-27T10:57:03.098 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.125+0000 7ff463e00640 1 -- 172.21.15.97:0/3964141289 wait complete. 2024-03-27T10:57:03.100 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.127+0000 7ff463e00640 1 Processor -- start 2024-03-27T10:57:03.100 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.127+0000 7ff463e00640 1 -- start start 2024-03-27T10:57:03.100 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.127+0000 7ff463e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff45c102650 0x7ff45c199f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:03.100 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.127+0000 7ff463e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff45c104f80 0x7ff45c19a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:03.100 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.127+0000 7ff463e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff45c1078b0 0x7ff45c1a14c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:03.100 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.127+0000 7ff463e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff45c10d6c0 con 0x7ff45c1078b0 2024-03-27T10:57:03.101 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.127+0000 7ff463e00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff45c10d540 con 0x7ff45c104f80 2024-03-27T10:57:03.101 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.127+0000 7ff463e00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff45c10d840 con 0x7ff45c102650 2024-03-27T10:57:03.101 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.127+0000 7ff461e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff45c1078b0 0x7ff45c1a14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:03.101 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.128+0000 7ff461e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff45c1078b0 0x7ff45c1a14c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48342/0 (socket says 172.21.15.97:48342) 2024-03-27T10:57:03.101 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.128+0000 7ff461e00640 1 -- 172.21.15.97:0/1844342004 learned_addr learned my addr 172.21.15.97:0/1844342004 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:03.101 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.128+0000 7ff461400640 1 --2- 172.21.15.97:0/1844342004 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff45c102650 0x7ff45c199f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:03.101 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.128+0000 7ff460a00640 1 --2- 172.21.15.97:0/1844342004 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff45c104f80 0x7ff45c19a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:03.101 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.128+0000 7ff461e00640 1 -- 172.21.15.97:0/1844342004 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff45c102650 msgr2=0x7ff45c199f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:03.101 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.128+0000 7ff461e00640 1 --2- 172.21.15.97:0/1844342004 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff45c102650 0x7ff45c199f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:03.101 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.128+0000 7ff461e00640 1 -- 172.21.15.97:0/1844342004 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff45c104f80 msgr2=0x7ff45c19a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:03.101 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.128+0000 7ff461e00640 1 --2- 172.21.15.97:0/1844342004 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff45c104f80 0x7ff45c19a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:03.101 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.128+0000 7ff461e00640 1 -- 172.21.15.97:0/1844342004 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff45c1a1b30 con 0x7ff45c1078b0 2024-03-27T10:57:03.101 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.128+0000 7ff461400640 1 --2- 172.21.15.97:0/1844342004 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff45c102650 0x7ff45c199f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:57:03.102 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.128+0000 7ff460a00640 1 --2- 172.21.15.97:0/1844342004 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff45c104f80 0x7ff45c19a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:57:03.102 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.128+0000 7ff461e00640 1 --2- 172.21.15.97:0/1844342004 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff45c1078b0 0x7ff45c1a14c0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7ff454032380 tx=0x7ff454032820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:03.102 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.129+0000 7ff45a000640 1 -- 172.21.15.97:0/1844342004 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff454041020 con 0x7ff45c1078b0 2024-03-27T10:57:03.102 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.129+0000 7ff45a000640 1 -- 172.21.15.97:0/1844342004 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff454003e20 con 0x7ff45c1078b0 2024-03-27T10:57:03.103 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.129+0000 7ff463e00640 1 -- 172.21.15.97:0/1844342004 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff45c1a1dc0 con 0x7ff45c1078b0 2024-03-27T10:57:03.103 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.129+0000 7ff45a000640 1 -- 172.21.15.97:0/1844342004 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff454039040 con 0x7ff45c1078b0 2024-03-27T10:57:03.103 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.129+0000 7ff463e00640 1 -- 172.21.15.97:0/1844342004 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff45c1a2270 con 0x7ff45c1078b0 2024-03-27T10:57:03.105 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.133+0000 7ff466b78740 1 Processor -- start 2024-03-27T10:57:03.105 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.133+0000 7ff45a000640 1 -- 172.21.15.97:0/1844342004 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7ff454044020 con 0x7ff45c1078b0 2024-03-27T10:57:03.107 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.133+0000 7ff466b78740 1 -- start start 2024-03-27T10:57:03.107 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.134+0000 7ff466b78740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55914eeb3980 0x55914eeb3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:03.107 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.134+0000 7ff466b78740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55914eeb4370 0x55914eeb8e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:03.107 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.134+0000 7ff466b78740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55914eeb94e0 0x55914eebb970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:03.107 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.134+0000 7ff466b78740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55914ee18d10 con 0x55914eeb94e0 2024-03-27T10:57:03.107 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.134+0000 7ff466b78740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55914edc1260 con 0x55914eeb3980 2024-03-27T10:57:03.107 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.134+0000 7ff466b78740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55914ee14cc0 con 0x55914eeb4370 2024-03-27T10:57:03.108 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.134+0000 7ff460a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55914eeb94e0 0x55914eebb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:03.108 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.134+0000 7ff460a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55914eeb94e0 0x55914eebb970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48358/0 (socket says 172.21.15.97:48358) 2024-03-27T10:57:03.108 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.134+0000 7ff460a00640 1 -- 172.21.15.97:0/2523465625 learned_addr learned my addr 172.21.15.97:0/2523465625 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:03.108 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.134+0000 7ff460a00640 1 --2- 172.21.15.97:0/2523465625 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55914eeb3980 0x55914eeb3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:03.108 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.134+0000 7ff461400640 1 --2- 172.21.15.97:0/2523465625 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55914eeb4370 0x55914eeb8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:03.108 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.135+0000 7ff460a00640 1 -- 172.21.15.97:0/2523465625 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55914eeb4370 msgr2=0x55914eeb8e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:03.108 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.135+0000 7ff460a00640 1 --2- 172.21.15.97:0/2523465625 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55914eeb4370 0x55914eeb8e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:03.108 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.135+0000 7ff460a00640 1 -- 172.21.15.97:0/2523465625 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55914eeb3980 msgr2=0x55914eeb3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:03.108 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.135+0000 7ff460a00640 1 --2- 172.21.15.97:0/2523465625 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55914eeb3980 0x55914eeb3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:03.108 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.135+0000 7ff460a00640 1 -- 172.21.15.97:0/2523465625 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55914edb8aa0 con 0x55914eeb94e0 2024-03-27T10:57:03.108 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.135+0000 7ff45a000640 1 --2- 172.21.15.97:0/1844342004 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7ff4440a3730 0x7ff4440a5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:03.108 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.135+0000 7ff45a000640 1 -- 172.21.15.97:0/1844342004 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7ff4540f1fb0 con 0x7ff45c1078b0 2024-03-27T10:57:03.108 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.135+0000 7ff460a00640 1 --2- 172.21.15.97:0/2523465625 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55914eeb94e0 0x55914eebb970 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7ff448005fd0 tx=0x7ff44800b400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a76f5c546272a6dd server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:03.108 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.135+0000 7ff461400640 1 --2- 172.21.15.97:0/1844342004 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7ff4440a3730 0x7ff4440a5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:03.112 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.135+0000 7ff44e600640 1 -- 172.21.15.97:0/2523465625 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff44800e8f0 con 0x55914eeb94e0 2024-03-27T10:57:03.112 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.136+0000 7ff44e600640 1 -- 172.21.15.97:0/2523465625 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff44801e430 con 0x55914eeb94e0 2024-03-27T10:57:03.112 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.136+0000 7ff44e600640 1 -- 172.21.15.97:0/2523465625 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff448011640 con 0x55914eeb94e0 2024-03-27T10:57:03.112 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.136+0000 7ff466b78740 1 -- 172.21.15.97:0/2523465625 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55914eeb94e0 msgr2=0x55914eebb970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:03.113 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.136+0000 7ff461400640 1 --2- 172.21.15.97:0/1844342004 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7ff4440a3730 0x7ff4440a5bf0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7ff450005fd0 tx=0x7ff45000b650 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:03.113 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.136+0000 7ff466b78740 1 --2- 172.21.15.97:0/2523465625 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55914eeb94e0 0x55914eebb970 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7ff448005fd0 tx=0x7ff44800b400 comp rx=0 tx=0).stop 2024-03-27T10:57:03.113 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.136+0000 7ff466b78740 1 -- 172.21.15.97:0/2523465625 shutdown_connections 2024-03-27T10:57:03.113 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.136+0000 7ff466b78740 1 --2- 172.21.15.97:0/2523465625 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55914eeb94e0 0x55914eebb970 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:03.113 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.136+0000 7ff466b78740 1 --2- 172.21.15.97:0/2523465625 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55914eeb4370 0x55914eeb8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:03.113 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.136+0000 7ff466b78740 1 --2- 172.21.15.97:0/2523465625 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55914eeb3980 0x55914eeb3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:03.113 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.136+0000 7ff466b78740 1 -- 172.21.15.97:0/2523465625 >> 172.21.15.97:0/2523465625 conn(0x55914eeaeef0 msgr2=0x55914eeaf2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:03.113 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.136+0000 7ff466b78740 1 -- 172.21.15.97:0/2523465625 shutdown_connections 2024-03-27T10:57:03.113 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.136+0000 7ff466b78740 1 -- 172.21.15.97:0/2523465625 wait complete. 2024-03-27T10:57:03.113 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.137+0000 7ff466b78740 1 -- start start 2024-03-27T10:57:03.113 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.137+0000 7ff466b78740 1 Processor -- start 2024-03-27T10:57:03.113 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.137+0000 7ff466b78740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55914eeb3980 0x55914eebde40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:03.113 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.137+0000 7ff466b78740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55914eeb4370 0x55914eebe380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:03.114 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.137+0000 7ff466b78740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55914eeb94e0 0x55914eec52e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:03.114 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.137+0000 7ff466b78740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55914eebc490 con 0x55914eeb94e0 2024-03-27T10:57:03.114 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.137+0000 7ff466b78740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55914edd4630 con 0x55914eeb4370 2024-03-27T10:57:03.114 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.137+0000 7ff466b78740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55914eebc610 con 0x55914eeb3980 2024-03-27T10:57:03.114 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.138+0000 7ff466b78740 10 client.0 _ll_register_callbacks cb 0x55914edd9c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:57:03.114 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.138+0000 7ff461e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55914eeb94e0 0x55914eec52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:03.114 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.138+0000 7ff461e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55914eeb94e0 0x55914eec52e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48366/0 (socket says 172.21.15.97:48366) 2024-03-27T10:57:03.114 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.138+0000 7ff461e00640 1 -- 172.21.15.97:0/876496776 learned_addr learned my addr 172.21.15.97:0/876496776 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:03.114 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.138+0000 7ff460a00640 1 --2- 172.21.15.97:0/876496776 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55914eeb4370 0x55914eebe380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:03.114 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.138+0000 7ff461e00640 1 -- 172.21.15.97:0/876496776 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55914eeb3980 msgr2=0x55914eebde40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:03.114 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.138+0000 7ff461400640 1 --2- 172.21.15.97:0/876496776 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55914eeb3980 0x55914eebde40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:03.114 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.138+0000 7ff461e00640 1 --2- 172.21.15.97:0/876496776 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55914eeb3980 0x55914eebde40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:03.114 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.138+0000 7ff461e00640 1 -- 172.21.15.97:0/876496776 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55914eeb4370 msgr2=0x55914eebe380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:03.115 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.138+0000 7ff461e00640 1 --2- 172.21.15.97:0/876496776 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55914eeb4370 0x55914eebe380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:03.115 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.138+0000 7ff461e00640 1 -- 172.21.15.97:0/876496776 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55914ede05f0 con 0x55914eeb94e0 2024-03-27T10:57:03.115 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.138+0000 7ff460a00640 1 --2- 172.21.15.97:0/876496776 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55914eeb4370 0x55914eebe380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:57:03.115 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.138+0000 7ff461400640 1 --2- 172.21.15.97:0/876496776 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55914eeb3980 0x55914eebde40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:57:03.115 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.138+0000 7ff461e00640 1 --2- 172.21.15.97:0/876496776 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55914eeb94e0 0x55914eec52e0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7ff4540c6bd0 tx=0x7ff4540be090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fb7d10f3b1ec5c3 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:03.115 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.139+0000 7ff442a00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T10:57:03.115 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.139+0000 7ff442a00640 1 -- 172.21.15.97:0/876496776 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff454004040 con 0x55914eeb94e0 2024-03-27T10:57:03.115 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.139+0000 7ff442a00640 1 -- 172.21.15.97:0/876496776 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7ff454004680 con 0x55914eeb94e0 2024-03-27T10:57:03.115 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.139+0000 7ff466b78740 1 -- 172.21.15.97:0/876496776 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff44801e430 con 0x55914eeb94e0 2024-03-27T10:57:03.115 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.139+0000 7ff442a00640 1 -- 172.21.15.97:0/876496776 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff45403b2f0 con 0x55914eeb94e0 2024-03-27T10:57:03.115 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.140+0000 7ff466b78740 10 client.15543 fetch_fsmap learned FSMap version 28 2024-03-27T10:57:03.115 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.140+0000 7ff466b78740 1 -- 172.21.15.97:0/876496776 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55914eb2d210 con 0x55914eeb94e0 2024-03-27T10:57:03.115 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.140+0000 7ff442a00640 1 -- 172.21.15.97:0/876496776 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff44801e430 con 0x55914eeb94e0 2024-03-27T10:57:03.117 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.141+0000 7ff442a00640 1 -- 172.21.15.97:0/876496776 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7ff45403c030 con 0x55914eeb94e0 2024-03-27T10:57:03.117 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.141+0000 7ff466b78740 10 client.15543 fetch_fsmap finished waiting for FSMap version 28 2024-03-27T10:57:03.117 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.141+0000 7ff466b78740 10 client.15543 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T10:57:03.118 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.141+0000 7ff466b78740 20 client.15543 populate_metadata read hostname 'smithi097' 2024-03-27T10:57:03.118 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.141+0000 7ff466b78740 1 --2- 172.21.15.97:0/876496776 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55914eec9bb0 0x55914eecc050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:03.118 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.141+0000 7ff466b78740 4 client.15543 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T10:57:03.118 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.141+0000 7ff466b78740 1 -- 172.21.15.97:0/876496776 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55914ee26250 con 0x55914eec9bb0 2024-03-27T10:57:03.118 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.141+0000 7ff461400640 1 --2- 172.21.15.97:0/876496776 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55914eec9bb0 0x55914eecc050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:03.118 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.142+0000 7ff461400640 1 --2- 172.21.15.97:0/876496776 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55914eec9bb0 0x55914eecc050 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=309ebae747873297 server_cookie=2a3a1f84f2eeda0f in_seq=0 out_seq=0 2024-03-27T10:57:03.118 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.142+0000 7ff442a00640 10 client.15543 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T10:57:03.118 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.143+0000 7ff442a00640 1 -- 172.21.15.97:0/876496776 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55914ee26250 con 0x55914eec9bb0 2024-03-27T10:57:03.118 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.143+0000 7ff442a00640 10 client.15543 handle_command_reply: tid=0 2024-03-27T10:57:03.118 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.143+0000 7ff466b78740 1 client.15543 shutdown 2024-03-27T10:57:03.118 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.143+0000 7ff466b78740 1 -- 172.21.15.97:0/876496776 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55914eeb94e0 msgr2=0x55914eec52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:03.118 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.143+0000 7ff466b78740 1 --2- 172.21.15.97:0/876496776 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55914eeb94e0 0x55914eec52e0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7ff4540c6bd0 tx=0x7ff4540be090 comp rx=0 tx=0).stop 2024-03-27T10:57:03.119 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.143+0000 7ff466b78740 1 -- 172.21.15.97:0/876496776 shutdown_connections 2024-03-27T10:57:03.119 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.143+0000 7ff466b78740 1 --2- 172.21.15.97:0/876496776 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55914eec9bb0 0x55914eecc050 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:03.119 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.143+0000 7ff466b78740 1 --2- 172.21.15.97:0/876496776 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55914eeb94e0 0x55914eec52e0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:03.119 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.143+0000 7ff466b78740 1 --2- 172.21.15.97:0/876496776 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55914eeb4370 0x55914eebe380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:03.119 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.144+0000 7ff466b78740 1 --2- 172.21.15.97:0/876496776 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55914eeb3980 0x55914eebde40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:03.119 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.144+0000 7ff466b78740 1 -- 172.21.15.97:0/876496776 >> 172.21.15.97:0/876496776 conn(0x55914eeaeef0 msgr2=0x55914eeae0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:03.119 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.144+0000 7ff442a00640 0 client.15543 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T10:57:03.119 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.144+0000 7ff466b78740 1 -- 172.21.15.97:0/876496776 shutdown_connections 2024-03-27T10:57:03.119 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.144+0000 7ff466b78740 1 -- 172.21.15.97:0/876496776 wait complete. 2024-03-27T10:57:03.119 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.144+0000 7ff466b78740 20 client.15543 trim_cache size 0 max 16384 2024-03-27T10:57:03.132 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.159+0000 7ff466b78740 1 Processor -- start 2024-03-27T10:57:03.132 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.160+0000 7ff466b78740 1 -- start start 2024-03-27T10:57:03.132 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.160+0000 7ff466b78740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55914eeb3980 0x55914eebd460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:03.132 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.160+0000 7ff466b78740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55914eeb4370 0x55914eec52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:03.132 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.160+0000 7ff466b78740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55914eeb94e0 0x55914eec5d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:03.133 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.160+0000 7ff466b78740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff45c1ab070 con 0x55914eeb3980 2024-03-27T10:57:03.133 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.160+0000 7ff466b78740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff45c1acbb0 con 0x55914eeb4370 2024-03-27T10:57:03.133 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.160+0000 7ff466b78740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff45c1ab740 con 0x55914eeb94e0 2024-03-27T10:57:03.133 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.160+0000 7ff461400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55914eeb3980 0x55914eebd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:03.133 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.160+0000 7ff461400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55914eeb3980 0x55914eebd460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48380/0 (socket says 172.21.15.97:48380) 2024-03-27T10:57:03.133 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.160+0000 7ff461400640 1 -- 172.21.15.97:0/3202867887 learned_addr learned my addr 172.21.15.97:0/3202867887 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:03.133 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.160+0000 7ff460a00640 1 --2- 172.21.15.97:0/3202867887 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55914eeb4370 0x55914eec52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:03.133 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.160+0000 7ff461e00640 1 --2- 172.21.15.97:0/3202867887 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55914eeb94e0 0x55914eec5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:03.134 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.161+0000 7ff461400640 1 -- 172.21.15.97:0/3202867887 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55914eeb94e0 msgr2=0x55914eec5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:03.134 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.161+0000 7ff461400640 1 --2- 172.21.15.97:0/3202867887 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55914eeb94e0 0x55914eec5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:03.135 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.161+0000 7ff461400640 1 -- 172.21.15.97:0/3202867887 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55914eeb4370 msgr2=0x55914eec52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:03.135 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.161+0000 7ff461400640 1 --2- 172.21.15.97:0/3202867887 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55914eeb4370 0x55914eec52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:03.135 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.161+0000 7ff461400640 1 -- 172.21.15.97:0/3202867887 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55914ebf9620 con 0x55914eeb3980 2024-03-27T10:57:03.135 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.161+0000 7ff461400640 1 --2- 172.21.15.97:0/3202867887 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55914eeb3980 0x55914eebd460 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7ff4500058c0 tx=0x7ff450012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1445611d5cc1882 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:03.135 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.161+0000 7ff44dc00640 1 -- 172.21.15.97:0/3202867887 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff450010db0 con 0x55914eeb3980 2024-03-27T10:57:03.135 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.161+0000 7ff44dc00640 1 -- 172.21.15.97:0/3202867887 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff45002c070 con 0x55914eeb3980 2024-03-27T10:57:03.135 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.161+0000 7ff44dc00640 1 -- 172.21.15.97:0/3202867887 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff45001dce0 con 0x55914eeb3980 2024-03-27T10:57:03.135 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.162+0000 7ff466b78740 1 -- 172.21.15.97:0/3202867887 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55914eeb3980 msgr2=0x55914eebd460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:03.135 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.162+0000 7ff466b78740 1 --2- 172.21.15.97:0/3202867887 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55914eeb3980 0x55914eebd460 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7ff4500058c0 tx=0x7ff450012000 comp rx=0 tx=0).stop 2024-03-27T10:57:03.135 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.162+0000 7ff466b78740 1 -- 172.21.15.97:0/3202867887 shutdown_connections 2024-03-27T10:57:03.135 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.162+0000 7ff466b78740 1 --2- 172.21.15.97:0/3202867887 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55914eeb94e0 0x55914eec5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:03.135 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.162+0000 7ff466b78740 1 --2- 172.21.15.97:0/3202867887 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55914eeb4370 0x55914eec52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:03.135 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.162+0000 7ff466b78740 1 --2- 172.21.15.97:0/3202867887 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55914eeb3980 0x55914eebd460 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:03.135 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.162+0000 7ff466b78740 1 -- 172.21.15.97:0/3202867887 >> 172.21.15.97:0/3202867887 conn(0x55914eeaeef0 msgr2=0x55914eeb57d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:03.136 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.162+0000 7ff466b78740 1 -- 172.21.15.97:0/3202867887 shutdown_connections 2024-03-27T10:57:03.136 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.162+0000 7ff466b78740 1 -- 172.21.15.97:0/3202867887 wait complete. 2024-03-27T10:57:03.138 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.163+0000 7ff466b78740 1 -- start start 2024-03-27T10:57:03.138 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.163+0000 7ff466b78740 1 Processor -- start 2024-03-27T10:57:03.138 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.164+0000 7ff466b78740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55914eeb3980 0x55914eec2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:03.138 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.164+0000 7ff466b78740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55914eeb4370 0x55914eec2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:03.139 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.164+0000 7ff466b78740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55914eeb94e0 0x55914ef2df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:03.139 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.164+0000 7ff466b78740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff45c1ac4e0 con 0x55914eeb3980 2024-03-27T10:57:03.139 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.164+0000 7ff466b78740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff45c1abe10 con 0x55914eeb4370 2024-03-27T10:57:03.139 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.164+0000 7ff466b78740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff45c1a9050 con 0x55914eeb94e0 2024-03-27T10:57:03.139 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.164+0000 7ff466b78740 10 client.0 _ll_register_callbacks cb 0x7ff454032990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:57:03.139 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.164+0000 7ff461400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55914eeb3980 0x55914eec2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:03.139 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.164+0000 7ff461400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55914eeb3980 0x55914eec2240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48392/0 (socket says 172.21.15.97:48392) 2024-03-27T10:57:03.139 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.164+0000 7ff461400640 1 -- 172.21.15.97:0/367495324 learned_addr learned my addr 172.21.15.97:0/367495324 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:03.139 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.164+0000 7ff460a00640 1 --2- 172.21.15.97:0/367495324 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55914eeb4370 0x55914eec2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:03.139 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.164+0000 7ff461e00640 1 --2- 172.21.15.97:0/367495324 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55914eeb94e0 0x55914ef2df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:03.139 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.164+0000 7ff461400640 1 -- 172.21.15.97:0/367495324 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55914eeb94e0 msgr2=0x55914ef2df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:03.139 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.164+0000 7ff461400640 1 --2- 172.21.15.97:0/367495324 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55914eeb94e0 0x55914ef2df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:03.139 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.164+0000 7ff461400640 1 -- 172.21.15.97:0/367495324 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55914eeb4370 msgr2=0x55914eec2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:03.139 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.164+0000 7ff461400640 1 --2- 172.21.15.97:0/367495324 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55914eeb4370 0x55914eec2780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:03.140 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.164+0000 7ff461400640 1 -- 172.21.15.97:0/367495324 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55914ef2e4c0 con 0x55914eeb3980 2024-03-27T10:57:03.140 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.164+0000 7ff460a00640 1 --2- 172.21.15.97:0/367495324 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55914eeb4370 0x55914eec2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T10:57:03.140 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.165+0000 7ff461400640 1 --2- 172.21.15.97:0/367495324 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55914eeb3980 0x55914eec2240 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7ff450005fa0 tx=0x7ff450010de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b619640fa196196 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:03.140 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.165+0000 7ff442a00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T10:57:03.140 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.165+0000 7ff442a00640 1 -- 172.21.15.97:0/367495324 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff45001d420 con 0x55914eeb3980 2024-03-27T10:57:03.140 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.165+0000 7ff442a00640 1 -- 172.21.15.97:0/367495324 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7ff4500263a0 con 0x55914eeb3980 2024-03-27T10:57:03.140 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.165+0000 7ff466b78740 1 -- 172.21.15.97:0/367495324 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff45002c070 con 0x55914eeb3980 2024-03-27T10:57:03.140 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.165+0000 7ff442a00640 1 -- 172.21.15.97:0/367495324 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff4500265a0 con 0x55914eeb3980 2024-03-27T10:57:03.140 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.165+0000 7ff466b78740 10 client.15549 fetch_fsmap learned FSMap version 28 2024-03-27T10:57:03.140 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.165+0000 7ff466b78740 1 -- 172.21.15.97:0/367495324 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55914ef2f7a0 con 0x55914eeb3980 2024-03-27T10:57:03.140 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.166+0000 7ff442a00640 1 -- 172.21.15.97:0/367495324 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff45002c070 con 0x55914eeb3980 2024-03-27T10:57:03.140 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.166+0000 7ff442a00640 1 -- 172.21.15.97:0/367495324 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7ff450038a40 con 0x55914eeb3980 2024-03-27T10:57:03.140 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.166+0000 7ff466b78740 10 client.15549 fetch_fsmap finished waiting for FSMap version 28 2024-03-27T10:57:03.140 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.166+0000 7ff466b78740 10 client.15549 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T10:57:03.141 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.166+0000 7ff466b78740 20 client.15549 populate_metadata read hostname 'smithi097' 2024-03-27T10:57:03.141 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.166+0000 7ff466b78740 1 --2- 172.21.15.97:0/367495324 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55914ef30150 0x55914ef32570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:03.141 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.166+0000 7ff466b78740 4 client.15549 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-27T10:57:03.141 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.166+0000 7ff466b78740 1 -- 172.21.15.97:0/367495324 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55914eebdc30 con 0x55914ef30150 2024-03-27T10:57:03.141 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.166+0000 7ff460a00640 1 --2- 172.21.15.97:0/367495324 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55914ef30150 0x55914ef32570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:03.141 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.167+0000 7ff460a00640 1 --2- 172.21.15.97:0/367495324 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55914ef30150 0x55914ef32570 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e05fd5b86614a779 server_cookie=c124f0533bab4799 in_seq=0 out_seq=0 2024-03-27T10:57:03.141 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.167+0000 7ff442a00640 10 client.15549 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T10:57:03.148 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.175+0000 7ff442a00640 1 -- 172.21.15.97:0/367495324 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55914eebdc30 con 0x55914ef30150 2024-03-27T10:57:03.148 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.175+0000 7ff442a00640 10 client.15549 handle_command_reply: tid=0 2024-03-27T10:57:03.148 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.175+0000 7ff466b78740 1 client.15549 shutdown 2024-03-27T10:57:03.148 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.175+0000 7ff466b78740 1 -- 172.21.15.97:0/367495324 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55914eeb3980 msgr2=0x55914eec2240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:03.148 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.175+0000 7ff466b78740 1 --2- 172.21.15.97:0/367495324 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55914eeb3980 0x55914eec2240 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7ff450005fa0 tx=0x7ff450010de0 comp rx=0 tx=0).stop 2024-03-27T10:57:03.148 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.175+0000 7ff466b78740 1 -- 172.21.15.97:0/367495324 shutdown_connections 2024-03-27T10:57:03.148 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.175+0000 7ff466b78740 1 --2- 172.21.15.97:0/367495324 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55914ef30150 0x55914ef32570 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:03.148 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.175+0000 7ff466b78740 1 --2- 172.21.15.97:0/367495324 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55914eeb94e0 0x55914ef2df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:03.148 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.175+0000 7ff466b78740 1 --2- 172.21.15.97:0/367495324 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55914eeb4370 0x55914eec2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:03.148 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.175+0000 7ff466b78740 1 --2- 172.21.15.97:0/367495324 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55914eeb3980 0x55914eec2240 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:03.148 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.175+0000 7ff442a00640 0 client.15549 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T10:57:03.148 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.175+0000 7ff466b78740 1 -- 172.21.15.97:0/367495324 >> 172.21.15.97:0/367495324 conn(0x55914eeaeef0 msgr2=0x55914eeb57d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:03.149 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.176+0000 7ff466b78740 1 -- 172.21.15.97:0/367495324 shutdown_connections 2024-03-27T10:57:03.149 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.176+0000 7ff466b78740 1 -- 172.21.15.97:0/367495324 wait complete. 2024-03-27T10:57:03.149 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.176+0000 7ff466b78740 20 client.15549 trim_cache size 0 max 16384 2024-03-27T10:57:03.149 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-27T10:57:03.149 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-27T10:57:03.149 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-27T10:57:03.149 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-27T10:57:03.150 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.176+0000 7ff463e00640 1 -- 172.21.15.97:0/1844342004 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7ff4440a3730 msgr2=0x7ff4440a5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:03.150 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.176+0000 7ff463e00640 1 --2- 172.21.15.97:0/1844342004 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7ff4440a3730 0x7ff4440a5bf0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7ff450005fd0 tx=0x7ff45000b650 comp rx=0 tx=0).stop 2024-03-27T10:57:03.150 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.177+0000 7ff463e00640 1 -- 172.21.15.97:0/1844342004 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff45c1078b0 msgr2=0x7ff45c1a14c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:03.150 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.177+0000 7ff463e00640 1 --2- 172.21.15.97:0/1844342004 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff45c1078b0 0x7ff45c1a14c0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7ff454032380 tx=0x7ff454032820 comp rx=0 tx=0).stop 2024-03-27T10:57:03.150 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.177+0000 7ff463e00640 1 -- 172.21.15.97:0/1844342004 shutdown_connections 2024-03-27T10:57:03.150 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.177+0000 7ff463e00640 1 --2- 172.21.15.97:0/1844342004 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff45c1078b0 0x7ff45c1a14c0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:03.151 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.177+0000 7ff463e00640 1 --2- 172.21.15.97:0/1844342004 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7ff4440a3730 0x7ff4440a5bf0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:03.151 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.177+0000 7ff463e00640 1 --2- 172.21.15.97:0/1844342004 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff45c104f80 0x7ff45c19a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:03.151 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.177+0000 7ff463e00640 1 --2- 172.21.15.97:0/1844342004 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff45c102650 0x7ff45c199f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:03.151 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.177+0000 7ff463e00640 1 -- 172.21.15.97:0/1844342004 >> 172.21.15.97:0/1844342004 conn(0x7ff45c0fbfe0 msgr2=0x7ff45c101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:03.151 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.177+0000 7ff463e00640 1 -- 172.21.15.97:0/1844342004 shutdown_connections 2024-03-27T10:57:03.151 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:03.177+0000 7ff463e00640 1 -- 172.21.15.97:0/1844342004 wait complete. 2024-03-27T10:57:03.390 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:57:03 smithi097 ceph-mon[31859]: pgmap v453: 97 pgs: 97 active+clean; 662 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.4 KiB/s wr, 12 op/s 2024-03-27T10:57:03.516 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:57:03 smithi114 ceph-mon[35338]: pgmap v453: 97 pgs: 97 active+clean; 662 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.4 KiB/s wr, 12 op/s 2024-03-27T10:57:03.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:57:03 smithi177 ceph-mon[35294]: pgmap v453: 97 pgs: 97 active+clean; 662 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.4 KiB/s wr, 12 op/s 2024-03-27T10:57:05.516 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:57:05 smithi114 ceph-mon[35338]: pgmap v454: 97 pgs: 97 active+clean; 663 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.7 KiB/s wr, 15 op/s 2024-03-27T10:57:05.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:57:05 smithi097 ceph-mon[31859]: pgmap v454: 97 pgs: 97 active+clean; 663 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.7 KiB/s wr, 15 op/s 2024-03-27T10:57:05.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:57:05 smithi177 ceph-mon[35294]: pgmap v454: 97 pgs: 97 active+clean; 663 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.7 KiB/s wr, 15 op/s 2024-03-27T10:57:07.237 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-27T10:57:07.238 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-27T10:57:07.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.494+0000 7fb3aaa00640 1 -- 172.21.15.97:0/3551781202 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3a4103d90 msgr2=0x7fb3a41041b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:07.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.494+0000 7fb3aaa00640 1 --2- 172.21.15.97:0/3551781202 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3a4103d90 0x7fb3a41041b0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fb394009900 tx=0x7fb39402bc40 comp rx=0 tx=0).stop 2024-03-27T10:57:07.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.495+0000 7fb3aaa00640 1 -- 172.21.15.97:0/3551781202 shutdown_connections 2024-03-27T10:57:07.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.495+0000 7fb3aaa00640 1 --2- 172.21.15.97:0/3551781202 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb3a4105940 0x7fb3a410c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:07.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.495+0000 7fb3aaa00640 1 --2- 172.21.15.97:0/3551781202 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb3a4104f80 0x7fb3a4105400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:07.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.495+0000 7fb3aaa00640 1 --2- 172.21.15.97:0/3551781202 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3a4103d90 0x7fb3a41041b0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:07.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.495+0000 7fb3aaa00640 1 -- 172.21.15.97:0/3551781202 >> 172.21.15.97:0/3551781202 conn(0x7fb3a40ff300 msgr2=0x7fb3a4101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:07.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.495+0000 7fb3aaa00640 1 -- 172.21.15.97:0/3551781202 shutdown_connections 2024-03-27T10:57:07.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.495+0000 7fb3aaa00640 1 -- 172.21.15.97:0/3551781202 wait complete. 2024-03-27T10:57:07.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.496+0000 7fb3aaa00640 1 Processor -- start 2024-03-27T10:57:07.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.496+0000 7fb3aaa00640 1 -- start start 2024-03-27T10:57:07.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.496+0000 7fb3aaa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3a4103d90 0x7fb3a419a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:07.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.496+0000 7fb3aaa00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb3a4104f80 0x7fb3a419a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:07.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.496+0000 7fb3aaa00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb3a4105940 0x7fb3a41a15f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:07.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.496+0000 7fb3aaa00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a410f8e0 con 0x7fb3a4103d90 2024-03-27T10:57:07.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.496+0000 7fb3aaa00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a410f760 con 0x7fb3a4105940 2024-03-27T10:57:07.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.496+0000 7fb3aaa00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a410fa60 con 0x7fb3a4104f80 2024-03-27T10:57:07.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.496+0000 7fb3a3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3a4103d90 0x7fb3a419a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:07.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.496+0000 7fb3a3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3a4103d90 0x7fb3a419a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48404/0 (socket says 172.21.15.97:48404) 2024-03-27T10:57:07.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.496+0000 7fb3a3e00640 1 -- 172.21.15.97:0/363767757 learned_addr learned my addr 172.21.15.97:0/363767757 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:07.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.497+0000 7fb3a8a00640 1 --2- 172.21.15.97:0/363767757 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb3a4105940 0x7fb3a41a15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:07.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.497+0000 7fb3a3400640 1 --2- 172.21.15.97:0/363767757 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb3a4104f80 0x7fb3a419a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:07.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.497+0000 7fb3a3e00640 1 -- 172.21.15.97:0/363767757 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb3a4104f80 msgr2=0x7fb3a419a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:07.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.497+0000 7fb3a3e00640 1 --2- 172.21.15.97:0/363767757 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb3a4104f80 0x7fb3a419a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:07.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.497+0000 7fb3a3e00640 1 -- 172.21.15.97:0/363767757 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb3a4105940 msgr2=0x7fb3a41a15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:07.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.497+0000 7fb3a3e00640 1 --2- 172.21.15.97:0/363767757 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb3a4105940 0x7fb3a41a15f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:07.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.497+0000 7fb3a3e00640 1 -- 172.21.15.97:0/363767757 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3a41a1c60 con 0x7fb3a4103d90 2024-03-27T10:57:07.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.497+0000 7fb3a3400640 1 --2- 172.21.15.97:0/363767757 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb3a4104f80 0x7fb3a419a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:57:07.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.497+0000 7fb3a3e00640 1 --2- 172.21.15.97:0/363767757 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3a4103d90 0x7fb3a419a040 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fb394005ab0 tx=0x7fb3940042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:07.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.497+0000 7fb3a0c00640 1 -- 172.21.15.97:0/363767757 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb394043020 con 0x7fb3a4103d90 2024-03-27T10:57:07.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.497+0000 7fb3a0c00640 1 -- 172.21.15.97:0/363767757 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb3940054c0 con 0x7fb3a4103d90 2024-03-27T10:57:07.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.497+0000 7fb3a0c00640 1 -- 172.21.15.97:0/363767757 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb394003d10 con 0x7fb3a4103d90 2024-03-27T10:57:07.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.498+0000 7fb3aaa00640 1 -- 172.21.15.97:0/363767757 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3a41a1ef0 con 0x7fb3a4103d90 2024-03-27T10:57:07.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.498+0000 7fb3aaa00640 1 -- 172.21.15.97:0/363767757 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3a41a2350 con 0x7fb3a4103d90 2024-03-27T10:57:07.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.501+0000 7fb3ad7f1740 1 Processor -- start 2024-03-27T10:57:07.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.501+0000 7fb3ad7f1740 1 -- start start 2024-03-27T10:57:07.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.501+0000 7fb3a0c00640 1 -- 172.21.15.97:0/363767757 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7fb394046020 con 0x7fb3a4103d90 2024-03-27T10:57:07.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.501+0000 7fb3ad7f1740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5636322094b0 0x5636322098d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:07.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.501+0000 7fb3ad7f1740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563632209ea0 0x56363220e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:07.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.501+0000 7fb3ad7f1740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56363220f010 0x5636322114a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:07.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.501+0000 7fb3ad7f1740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56363216f6b0 con 0x563632209ea0 2024-03-27T10:57:07.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.501+0000 7fb3ad7f1740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x563632117750 con 0x56363220f010 2024-03-27T10:57:07.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.501+0000 7fb3ad7f1740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x56363216b660 con 0x5636322094b0 2024-03-27T10:57:07.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.501+0000 7fb3a3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563632209ea0 0x56363220e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:07.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.501+0000 7fb3a3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563632209ea0 0x56363220e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48412/0 (socket says 172.21.15.97:48412) 2024-03-27T10:57:07.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.501+0000 7fb3a3e00640 1 -- 172.21.15.97:0/3490790296 learned_addr learned my addr 172.21.15.97:0/3490790296 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:07.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.502+0000 7fb3a3400640 1 --2- 172.21.15.97:0/3490790296 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56363220f010 0x5636322114a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:07.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.502+0000 7fb3a3e00640 1 -- 172.21.15.97:0/3490790296 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5636322094b0 msgr2=0x5636322098d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:07.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.502+0000 7fb3a8a00640 1 --2- 172.21.15.97:0/3490790296 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5636322094b0 0x5636322098d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:07.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.502+0000 7fb3a3e00640 1 --2- 172.21.15.97:0/3490790296 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5636322094b0 0x5636322098d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:07.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.502+0000 7fb3a3e00640 1 -- 172.21.15.97:0/3490790296 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56363220f010 msgr2=0x5636322114a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:07.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.502+0000 7fb3a3e00640 1 --2- 172.21.15.97:0/3490790296 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56363220f010 0x5636322114a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:07.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.502+0000 7fb3a3e00640 1 -- 172.21.15.97:0/3490790296 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563631f4d7d0 con 0x563632209ea0 2024-03-27T10:57:07.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.502+0000 7fb3a3400640 1 --2- 172.21.15.97:0/3490790296 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56363220f010 0x5636322114a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:57:07.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.502+0000 7fb3a3e00640 1 --2- 172.21.15.97:0/3490790296 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563632209ea0 0x56363220e9a0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fb3940fa190 tx=0x7fb3940fa280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67af1dc314db15f7 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:07.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.502+0000 7fb3a0c00640 1 --2- 172.21.15.97:0/363767757 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fb3880a3610 0x7fb3880a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:07.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.502+0000 7fb399200640 1 -- 172.21.15.97:0/3490790296 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3940ba880 con 0x563632209ea0 2024-03-27T10:57:07.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.503+0000 7fb399200640 1 -- 172.21.15.97:0/3490790296 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb3940ff030 con 0x563632209ea0 2024-03-27T10:57:07.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.503+0000 7fb399200640 1 -- 172.21.15.97:0/3490790296 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3940bdd60 con 0x563632209ea0 2024-03-27T10:57:07.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.503+0000 7fb3a0c00640 1 -- 172.21.15.97:0/363767757 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fb3940f2160 con 0x7fb3a4103d90 2024-03-27T10:57:07.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.503+0000 7fb3a8a00640 1 --2- 172.21.15.97:0/363767757 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fb3880a3610 0x7fb3880a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:07.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.503+0000 7fb3ad7f1740 1 -- 172.21.15.97:0/3490790296 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563632209ea0 msgr2=0x56363220e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:07.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.503+0000 7fb3ad7f1740 1 --2- 172.21.15.97:0/3490790296 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563632209ea0 0x56363220e9a0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fb3940fa190 tx=0x7fb3940fa280 comp rx=0 tx=0).stop 2024-03-27T10:57:07.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.503+0000 7fb3a8a00640 1 --2- 172.21.15.97:0/363767757 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fb3880a3610 0x7fb3880a5ad0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x56363220a700 tx=0x7fb39c00a370 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:07.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.503+0000 7fb3ad7f1740 1 -- 172.21.15.97:0/3490790296 shutdown_connections 2024-03-27T10:57:07.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.503+0000 7fb3ad7f1740 1 --2- 172.21.15.97:0/3490790296 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56363220f010 0x5636322114a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:07.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.503+0000 7fb3ad7f1740 1 --2- 172.21.15.97:0/3490790296 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563632209ea0 0x56363220e9a0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:07.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.503+0000 7fb3ad7f1740 1 --2- 172.21.15.97:0/3490790296 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5636322094b0 0x5636322098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:07.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.503+0000 7fb3ad7f1740 1 -- 172.21.15.97:0/3490790296 >> 172.21.15.97:0/3490790296 conn(0x563632204a20 msgr2=0x563632204e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:07.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.504+0000 7fb3ad7f1740 1 -- 172.21.15.97:0/3490790296 shutdown_connections 2024-03-27T10:57:07.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.504+0000 7fb3ad7f1740 1 -- 172.21.15.97:0/3490790296 wait complete. 2024-03-27T10:57:07.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.504+0000 7fb3ad7f1740 1 -- start start 2024-03-27T10:57:07.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.504+0000 7fb3ad7f1740 1 Processor -- start 2024-03-27T10:57:07.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.504+0000 7fb3ad7f1740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5636322094b0 0x563632213650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:07.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.505+0000 7fb3ad7f1740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563632209ea0 0x563632213b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:07.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.505+0000 7fb3ad7f1740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56363220f010 0x56363221abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:07.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.505+0000 7fb3ad7f1740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x563632211d80 con 0x563632209ea0 2024-03-27T10:57:07.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.505+0000 7fb3ad7f1740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x56363212aa80 con 0x5636322094b0 2024-03-27T10:57:07.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.505+0000 7fb3ad7f1740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x563632211f00 con 0x56363220f010 2024-03-27T10:57:07.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.505+0000 7fb3ad7f1740 10 client.0 _ll_register_callbacks cb 0x56363212a470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:57:07.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.505+0000 7fb3a3400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563632209ea0 0x563632213b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:07.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.505+0000 7fb3a3400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563632209ea0 0x563632213b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48414/0 (socket says 172.21.15.97:48414) 2024-03-27T10:57:07.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.505+0000 7fb3a3400640 1 -- 172.21.15.97:0/600292191 learned_addr learned my addr 172.21.15.97:0/600292191 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:07.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.505+0000 7fb3a3e00640 1 --2- 172.21.15.97:0/600292191 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5636322094b0 0x563632213650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:07.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.505+0000 7fb3a8a00640 1 --2- 172.21.15.97:0/600292191 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56363220f010 0x56363221abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:07.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.505+0000 7fb3a3400640 1 -- 172.21.15.97:0/600292191 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56363220f010 msgr2=0x56363221abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:07.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.505+0000 7fb3a3400640 1 --2- 172.21.15.97:0/600292191 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56363220f010 0x56363221abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:07.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.505+0000 7fb3a3400640 1 -- 172.21.15.97:0/600292191 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5636322094b0 msgr2=0x563632213650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:07.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.505+0000 7fb3a3400640 1 --2- 172.21.15.97:0/600292191 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5636322094b0 0x563632213650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:07.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.505+0000 7fb3a3400640 1 -- 172.21.15.97:0/600292191 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563631e833f0 con 0x563632209ea0 2024-03-27T10:57:07.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.505+0000 7fb3a3e00640 1 --2- 172.21.15.97:0/600292191 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5636322094b0 0x563632213650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T10:57:07.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.506+0000 7fb3a3400640 1 --2- 172.21.15.97:0/600292191 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563632209ea0 0x563632213b90 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fb39000de80 tx=0x7fb39000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4213bc11b2f29bef server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:07.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.506+0000 7fb38d600640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T10:57:07.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.506+0000 7fb38d600640 1 -- 172.21.15.97:0/600292191 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb39000fea0 con 0x563632209ea0 2024-03-27T10:57:07.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.506+0000 7fb38d600640 1 -- 172.21.15.97:0/600292191 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fb390004220 con 0x563632209ea0 2024-03-27T10:57:07.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.506+0000 7fb3ad7f1740 1 -- 172.21.15.97:0/600292191 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3940ff030 con 0x563632209ea0 2024-03-27T10:57:07.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.506+0000 7fb38d600640 1 -- 172.21.15.97:0/600292191 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb390004420 con 0x563632209ea0 2024-03-27T10:57:07.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.506+0000 7fb3ad7f1740 10 client.15561 fetch_fsmap learned FSMap version 28 2024-03-27T10:57:07.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.506+0000 7fb3ad7f1740 1 -- 172.21.15.97:0/600292191 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563631e430f0 con 0x563632209ea0 2024-03-27T10:57:07.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.507+0000 7fb38d600640 1 -- 172.21.15.97:0/600292191 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb3940ff030 con 0x563632209ea0 2024-03-27T10:57:07.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.507+0000 7fb38d600640 1 -- 172.21.15.97:0/600292191 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fb390003990 con 0x563632209ea0 2024-03-27T10:57:07.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.507+0000 7fb3ad7f1740 10 client.15561 fetch_fsmap finished waiting for FSMap version 28 2024-03-27T10:57:07.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.507+0000 7fb3ad7f1740 10 client.15561 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-27T10:57:07.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.507+0000 7fb3ad7f1740 20 client.15561 populate_metadata read hostname 'smithi097' 2024-03-27T10:57:07.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.507+0000 7fb3ad7f1740 1 --2- 172.21.15.97:0/600292191 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x56363221f4f0 0x563632221990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:07.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.507+0000 7fb3ad7f1740 4 client.15561 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T10:57:07.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.507+0000 7fb3ad7f1740 1 -- 172.21.15.97:0/600292191 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56363217cbf0 con 0x56363221f4f0 2024-03-27T10:57:07.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.507+0000 7fb3a3e00640 1 --2- 172.21.15.97:0/600292191 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x56363221f4f0 0x563632221990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:07.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.508+0000 7fb3a3e00640 1 --2- 172.21.15.97:0/600292191 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x56363221f4f0 0x563632221990 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71f78d1a9ec5164e server_cookie=844f9965fa83dcbf in_seq=0 out_seq=0 2024-03-27T10:57:07.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.508+0000 7fb38d600640 10 client.15561 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T10:57:07.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.510+0000 7fb38d600640 1 -- 172.21.15.97:0/600292191 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x56363217cbf0 con 0x56363221f4f0 2024-03-27T10:57:07.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.510+0000 7fb38d600640 10 client.15561 handle_command_reply: tid=0 2024-03-27T10:57:07.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.510+0000 7fb3ad7f1740 1 client.15561 shutdown 2024-03-27T10:57:07.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.510+0000 7fb3ad7f1740 1 -- 172.21.15.97:0/600292191 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563632209ea0 msgr2=0x563632213b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:07.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.510+0000 7fb3ad7f1740 1 --2- 172.21.15.97:0/600292191 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563632209ea0 0x563632213b90 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fb39000de80 tx=0x7fb39000f730 comp rx=0 tx=0).stop 2024-03-27T10:57:07.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.510+0000 7fb3ad7f1740 1 -- 172.21.15.97:0/600292191 shutdown_connections 2024-03-27T10:57:07.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.510+0000 7fb3ad7f1740 1 --2- 172.21.15.97:0/600292191 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x56363221f4f0 0x563632221990 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:07.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.510+0000 7fb3ad7f1740 1 --2- 172.21.15.97:0/600292191 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56363220f010 0x56363221abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:07.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.510+0000 7fb3ad7f1740 1 --2- 172.21.15.97:0/600292191 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563632209ea0 0x563632213b90 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:07.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.510+0000 7fb3ad7f1740 1 --2- 172.21.15.97:0/600292191 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5636322094b0 0x563632213650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:07.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.510+0000 7fb38d600640 0 client.15561 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T10:57:07.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.510+0000 7fb3ad7f1740 1 -- 172.21.15.97:0/600292191 >> 172.21.15.97:0/600292191 conn(0x563632204a20 msgr2=0x563632203c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:07.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.511+0000 7fb3ad7f1740 1 -- 172.21.15.97:0/600292191 shutdown_connections 2024-03-27T10:57:07.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.511+0000 7fb3ad7f1740 1 -- 172.21.15.97:0/600292191 wait complete. 2024-03-27T10:57:07.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.511+0000 7fb3ad7f1740 20 client.15561 trim_cache size 0 max 16384 2024-03-27T10:57:07.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.529+0000 7fb3ad7f1740 1 Processor -- start 2024-03-27T10:57:07.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.529+0000 7fb3ad7f1740 1 -- start start 2024-03-27T10:57:07.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.530+0000 7fb3ad7f1740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5636322094b0 0x563632203390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:07.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.530+0000 7fb3ad7f1740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56363220f010 0x56363221ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:07.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.530+0000 7fb3ad7f1740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5636322038d0 0x563632203d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:07.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.530+0000 7fb3ad7f1740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a41ab240 con 0x5636322094b0 2024-03-27T10:57:07.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.530+0000 7fb3ad7f1740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a41acd80 con 0x56363220f010 2024-03-27T10:57:07.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.530+0000 7fb3ad7f1740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a41ab910 con 0x5636322038d0 2024-03-27T10:57:07.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.530+0000 7fb3a3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5636322094b0 0x563632203390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:07.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.530+0000 7fb3a3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5636322094b0 0x563632203390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48420/0 (socket says 172.21.15.97:48420) 2024-03-27T10:57:07.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.530+0000 7fb3a3e00640 1 -- 172.21.15.97:0/220074502 learned_addr learned my addr 172.21.15.97:0/220074502 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:07.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.530+0000 7fb3a3400640 1 --2- 172.21.15.97:0/220074502 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56363220f010 0x56363221ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:07.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.530+0000 7fb3a8a00640 1 --2- 172.21.15.97:0/220074502 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5636322038d0 0x563632203d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:07.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.530+0000 7fb3a3e00640 1 -- 172.21.15.97:0/220074502 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5636322038d0 msgr2=0x563632203d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:07.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.530+0000 7fb3a3e00640 1 --2- 172.21.15.97:0/220074502 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5636322038d0 0x563632203d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:07.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.530+0000 7fb3a3e00640 1 -- 172.21.15.97:0/220074502 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56363220f010 msgr2=0x56363221ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:07.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.530+0000 7fb3a3e00640 1 --2- 172.21.15.97:0/220074502 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56363220f010 0x56363221ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:07.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.530+0000 7fb3a3e00640 1 -- 172.21.15.97:0/220074502 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563632205990 con 0x5636322094b0 2024-03-27T10:57:07.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.530+0000 7fb3a8a00640 1 --2- 172.21.15.97:0/220074502 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5636322038d0 0x563632203d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:57:07.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.531+0000 7fb3a3e00640 1 --2- 172.21.15.97:0/220074502 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5636322094b0 0x563632203390 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fb394003fd0 tx=0x7fb394000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cde3485751bb1379 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:07.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.531+0000 7fb38fe00640 1 -- 172.21.15.97:0/220074502 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3940bdf40 con 0x5636322094b0 2024-03-27T10:57:07.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.531+0000 7fb38fe00640 1 -- 172.21.15.97:0/220074502 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb3940ff040 con 0x5636322094b0 2024-03-27T10:57:07.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.531+0000 7fb38fe00640 1 -- 172.21.15.97:0/220074502 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3940be120 con 0x5636322094b0 2024-03-27T10:57:07.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.532+0000 7fb3ad7f1740 1 -- 172.21.15.97:0/220074502 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5636322094b0 msgr2=0x563632203390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:07.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.532+0000 7fb3ad7f1740 1 --2- 172.21.15.97:0/220074502 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5636322094b0 0x563632203390 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fb394003fd0 tx=0x7fb394000bc0 comp rx=0 tx=0).stop 2024-03-27T10:57:07.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.532+0000 7fb3ad7f1740 1 -- 172.21.15.97:0/220074502 shutdown_connections 2024-03-27T10:57:07.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.532+0000 7fb3ad7f1740 1 --2- 172.21.15.97:0/220074502 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5636322038d0 0x563632203d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:07.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.532+0000 7fb3ad7f1740 1 --2- 172.21.15.97:0/220074502 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56363220f010 0x56363221ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:07.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.532+0000 7fb3ad7f1740 1 --2- 172.21.15.97:0/220074502 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5636322094b0 0x563632203390 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:07.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.532+0000 7fb3ad7f1740 1 -- 172.21.15.97:0/220074502 >> 172.21.15.97:0/220074502 conn(0x563632204a20 msgr2=0x563632204e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:07.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.532+0000 7fb3ad7f1740 1 -- 172.21.15.97:0/220074502 shutdown_connections 2024-03-27T10:57:07.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.532+0000 7fb3ad7f1740 1 -- 172.21.15.97:0/220074502 wait complete. 2024-03-27T10:57:07.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.532+0000 7fb3ad7f1740 1 -- start start 2024-03-27T10:57:07.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.532+0000 7fb3ad7f1740 1 Processor -- start 2024-03-27T10:57:07.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.533+0000 7fb3ad7f1740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5636322038d0 0x56363221a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:07.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.533+0000 7fb3ad7f1740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5636322094b0 0x56363221ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:07.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.533+0000 7fb3ad7f1740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56363220f010 0x563632218d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:07.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.533+0000 7fb3ad7f1740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a41ac6b0 con 0x5636322038d0 2024-03-27T10:57:07.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.533+0000 7fb3ad7f1740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a41abfe0 con 0x5636322094b0 2024-03-27T10:57:07.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.533+0000 7fb3ad7f1740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a41a9220 con 0x56363220f010 2024-03-27T10:57:07.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.533+0000 7fb3ad7f1740 10 client.0 _ll_register_callbacks cb 0x7fb39000da10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:57:07.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.533+0000 7fb3a3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5636322038d0 0x56363221a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:07.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.533+0000 7fb3a3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5636322038d0 0x56363221a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48422/0 (socket says 172.21.15.97:48422) 2024-03-27T10:57:07.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.533+0000 7fb3a3e00640 1 -- 172.21.15.97:0/360245314 learned_addr learned my addr 172.21.15.97:0/360245314 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:07.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.533+0000 7fb3a3400640 1 --2- 172.21.15.97:0/360245314 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5636322094b0 0x56363221ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:07.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.533+0000 7fb3a3e00640 1 -- 172.21.15.97:0/360245314 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56363220f010 msgr2=0x563632218d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:07.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.533+0000 7fb3a8a00640 1 --2- 172.21.15.97:0/360245314 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56363220f010 0x563632218d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:07.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.533+0000 7fb3a3e00640 1 --2- 172.21.15.97:0/360245314 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56363220f010 0x563632218d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:07.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.533+0000 7fb3a3e00640 1 -- 172.21.15.97:0/360245314 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5636322094b0 msgr2=0x56363221ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:07.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.533+0000 7fb3a3e00640 1 --2- 172.21.15.97:0/360245314 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5636322094b0 0x56363221ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:07.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.533+0000 7fb3a3e00640 1 -- 172.21.15.97:0/360245314 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563632219540 con 0x5636322038d0 2024-03-27T10:57:07.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.533+0000 7fb3a3400640 1 --2- 172.21.15.97:0/360245314 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5636322094b0 0x56363221ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:57:07.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.534+0000 7fb3a3e00640 1 --2- 172.21.15.97:0/360245314 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5636322038d0 0x56363221a630 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fb3940baa40 tx=0x7fb394005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a26f0ba3c710bb91 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:07.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.534+0000 7fb38d600640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T10:57:07.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.534+0000 7fb38d600640 1 -- 172.21.15.97:0/360245314 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3940be3b0 con 0x5636322038d0 2024-03-27T10:57:07.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.534+0000 7fb38d600640 1 -- 172.21.15.97:0/360245314 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fb3940bc680 con 0x5636322038d0 2024-03-27T10:57:07.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.534+0000 7fb3ad7f1740 1 -- 172.21.15.97:0/360245314 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3940ff040 con 0x5636322038d0 2024-03-27T10:57:07.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.534+0000 7fb38d600640 1 -- 172.21.15.97:0/360245314 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb3940ba150 con 0x5636322038d0 2024-03-27T10:57:07.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.534+0000 7fb3ad7f1740 10 client.15567 fetch_fsmap learned FSMap version 28 2024-03-27T10:57:07.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.534+0000 7fb3ad7f1740 1 -- 172.21.15.97:0/360245314 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56363228ec60 con 0x5636322038d0 2024-03-27T10:57:07.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.534+0000 7fb38d600640 1 -- 172.21.15.97:0/360245314 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb3940ff040 con 0x5636322038d0 2024-03-27T10:57:07.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.534+0000 7fb38d600640 1 -- 172.21.15.97:0/360245314 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fb3940f5350 con 0x5636322038d0 2024-03-27T10:57:07.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.535+0000 7fb3ad7f1740 10 client.15567 fetch_fsmap finished waiting for FSMap version 28 2024-03-27T10:57:07.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.535+0000 7fb3ad7f1740 10 client.15567 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-27T10:57:07.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.535+0000 7fb3ad7f1740 20 client.15567 populate_metadata read hostname 'smithi097' 2024-03-27T10:57:07.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.535+0000 7fb3ad7f1740 1 --2- 172.21.15.97:0/360245314 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x56363228f3f0 0x563632291810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:07.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.535+0000 7fb3ad7f1740 4 client.15567 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-27T10:57:07.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.535+0000 7fb3ad7f1740 1 -- 172.21.15.97:0/360245314 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563632176f40 con 0x56363228f3f0 2024-03-27T10:57:07.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.535+0000 7fb3a3400640 1 --2- 172.21.15.97:0/360245314 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x56363228f3f0 0x563632291810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:07.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.536+0000 7fb3a3400640 1 --2- 172.21.15.97:0/360245314 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x56363228f3f0 0x563632291810 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8f5a5022a74c523 server_cookie=c57fbc497462c9 in_seq=0 out_seq=0 2024-03-27T10:57:07.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.536+0000 7fb38d600640 10 client.15567 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T10:57:07.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.536+0000 7fb38d600640 1 -- 172.21.15.97:0/360245314 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563632176f40 con 0x56363228f3f0 2024-03-27T10:57:07.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.536+0000 7fb38d600640 10 client.15567 handle_command_reply: tid=0 2024-03-27T10:57:07.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.536+0000 7fb3ad7f1740 1 client.15567 shutdown 2024-03-27T10:57:07.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.537+0000 7fb3ad7f1740 1 -- 172.21.15.97:0/360245314 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5636322038d0 msgr2=0x56363221a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:07.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.537+0000 7fb3ad7f1740 1 --2- 172.21.15.97:0/360245314 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5636322038d0 0x56363221a630 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fb3940baa40 tx=0x7fb394005fd0 comp rx=0 tx=0).stop 2024-03-27T10:57:07.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.537+0000 7fb3ad7f1740 1 -- 172.21.15.97:0/360245314 shutdown_connections 2024-03-27T10:57:07.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.537+0000 7fb3ad7f1740 1 --2- 172.21.15.97:0/360245314 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x56363228f3f0 0x563632291810 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:07.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.537+0000 7fb3ad7f1740 1 --2- 172.21.15.97:0/360245314 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56363220f010 0x563632218d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:07.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.537+0000 7fb38d600640 0 client.15567 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T10:57:07.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.537+0000 7fb3ad7f1740 1 --2- 172.21.15.97:0/360245314 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5636322094b0 0x56363221ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:07.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.537+0000 7fb3ad7f1740 1 --2- 172.21.15.97:0/360245314 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5636322038d0 0x56363221a630 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:07.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.537+0000 7fb3ad7f1740 1 -- 172.21.15.97:0/360245314 >> 172.21.15.97:0/360245314 conn(0x563632204a20 msgr2=0x563632204e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:07.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.537+0000 7fb3ad7f1740 1 -- 172.21.15.97:0/360245314 shutdown_connections 2024-03-27T10:57:07.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.537+0000 7fb3ad7f1740 1 -- 172.21.15.97:0/360245314 wait complete. 2024-03-27T10:57:07.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.537+0000 7fb3ad7f1740 20 client.15567 trim_cache size 0 max 16384 2024-03-27T10:57:07.515 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-03-27T10:57:07.515 INFO:teuthology.orchestra.run.smithi097.stdout: "status": "no active scrubs running", 2024-03-27T10:57:07.515 INFO:teuthology.orchestra.run.smithi097.stdout: "scrubs": {} 2024-03-27T10:57:07.515 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-03-27T10:57:07.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.538+0000 7fb3aaa00640 1 -- 172.21.15.97:0/363767757 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fb3880a3610 msgr2=0x7fb3880a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:07.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.538+0000 7fb3aaa00640 1 --2- 172.21.15.97:0/363767757 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fb3880a3610 0x7fb3880a5ad0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x56363220a700 tx=0x7fb39c00a370 comp rx=0 tx=0).stop 2024-03-27T10:57:07.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.538+0000 7fb3aaa00640 1 -- 172.21.15.97:0/363767757 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3a4103d90 msgr2=0x7fb3a419a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:07.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.538+0000 7fb3aaa00640 1 --2- 172.21.15.97:0/363767757 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3a4103d90 0x7fb3a419a040 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fb394005ab0 tx=0x7fb3940042f0 comp rx=0 tx=0).stop 2024-03-27T10:57:07.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.538+0000 7fb3aaa00640 1 -- 172.21.15.97:0/363767757 shutdown_connections 2024-03-27T10:57:07.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.538+0000 7fb3aaa00640 1 --2- 172.21.15.97:0/363767757 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fb3880a3610 0x7fb3880a5ad0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:07.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.538+0000 7fb3aaa00640 1 --2- 172.21.15.97:0/363767757 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb3a4105940 0x7fb3a41a15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:07.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.538+0000 7fb3aaa00640 1 --2- 172.21.15.97:0/363767757 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb3a4104f80 0x7fb3a419a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:07.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.538+0000 7fb3aaa00640 1 --2- 172.21.15.97:0/363767757 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3a4103d90 0x7fb3a419a040 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:07.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.538+0000 7fb3aaa00640 1 -- 172.21.15.97:0/363767757 >> 172.21.15.97:0/363767757 conn(0x7fb3a40ff300 msgr2=0x7fb3a41083a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:07.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.538+0000 7fb3aaa00640 1 -- 172.21.15.97:0/363767757 shutdown_connections 2024-03-27T10:57:07.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.538+0000 7fb3aaa00640 1 -- 172.21.15.97:0/363767757 wait complete. 2024-03-27T10:57:07.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:57:07 smithi114 ceph-mon[35338]: pgmap v455: 97 pgs: 97 active+clean; 663 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 511 B/s wr, 10 op/s 2024-03-27T10:57:07.524 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:57:07 smithi097 ceph-mon[31859]: pgmap v455: 97 pgs: 97 active+clean; 663 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 511 B/s wr, 10 op/s 2024-03-27T10:57:07.527 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-27T10:57:07.528 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-27T10:57:07.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:57:07 smithi177 ceph-mon[35294]: pgmap v455: 97 pgs: 97 active+clean; 663 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 511 B/s wr, 10 op/s 2024-03-27T10:57:07.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.761+0000 7f6fafa00640 1 -- 172.21.15.97:0/3932787558 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6fa8103d90 msgr2=0x7f6fa81041b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:07.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.761+0000 7f6fafa00640 1 --2- 172.21.15.97:0/3932787558 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6fa8103d90 0x7f6fa81041b0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c009900 tx=0x7f6f9c02bc30 comp rx=0 tx=0).stop 2024-03-27T10:57:07.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.761+0000 7f6fafa00640 1 -- 172.21.15.97:0/3932787558 shutdown_connections 2024-03-27T10:57:07.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.761+0000 7f6fafa00640 1 --2- 172.21.15.97:0/3932787558 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6fa8105940 0x7f6fa810c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:07.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.761+0000 7f6fafa00640 1 --2- 172.21.15.97:0/3932787558 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6fa8104f80 0x7f6fa8105400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:07.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.761+0000 7f6fafa00640 1 --2- 172.21.15.97:0/3932787558 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6fa8103d90 0x7f6fa81041b0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:07.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.761+0000 7f6fafa00640 1 -- 172.21.15.97:0/3932787558 >> 172.21.15.97:0/3932787558 conn(0x7f6fa80ff300 msgr2=0x7f6fa8101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:07.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.761+0000 7f6fafa00640 1 -- 172.21.15.97:0/3932787558 shutdown_connections 2024-03-27T10:57:07.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.761+0000 7f6fafa00640 1 -- 172.21.15.97:0/3932787558 wait complete. 2024-03-27T10:57:07.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.762+0000 7f6fafa00640 1 Processor -- start 2024-03-27T10:57:07.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.762+0000 7f6fafa00640 1 -- start start 2024-03-27T10:57:07.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.763+0000 7f6fafa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6fa8103d90 0x7f6fa8199fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:07.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.763+0000 7f6fafa00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6fa8104f80 0x7f6fa819a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:07.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.763+0000 7f6fafa00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6fa8105940 0x7f6fa81a1550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:07.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.763+0000 7f6fafa00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa810f900 con 0x7f6fa8103d90 2024-03-27T10:57:07.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.763+0000 7f6fafa00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa810f780 con 0x7f6fa8104f80 2024-03-27T10:57:07.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.763+0000 7f6fafa00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa810fa80 con 0x7f6fa8105940 2024-03-27T10:57:07.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.763+0000 7f6fad000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6fa8103d90 0x7f6fa8199fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:07.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.763+0000 7f6fad000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6fa8103d90 0x7f6fa8199fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48434/0 (socket says 172.21.15.97:48434) 2024-03-27T10:57:07.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.763+0000 7f6fad000640 1 -- 172.21.15.97:0/1857115162 learned_addr learned my addr 172.21.15.97:0/1857115162 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:07.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.763+0000 7f6fada00640 1 --2- 172.21.15.97:0/1857115162 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6fa8105940 0x7f6fa81a1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:07.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.763+0000 7f6fa7e00640 1 --2- 172.21.15.97:0/1857115162 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6fa8104f80 0x7f6fa819a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:07.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.763+0000 7f6fad000640 1 -- 172.21.15.97:0/1857115162 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6fa8105940 msgr2=0x7f6fa81a1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:07.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.763+0000 7f6fad000640 1 --2- 172.21.15.97:0/1857115162 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6fa8105940 0x7f6fa81a1550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:07.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.763+0000 7f6fad000640 1 -- 172.21.15.97:0/1857115162 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6fa8104f80 msgr2=0x7f6fa819a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:07.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.763+0000 7f6fad000640 1 --2- 172.21.15.97:0/1857115162 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6fa8104f80 0x7f6fa819a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:07.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.763+0000 7f6fad000640 1 -- 172.21.15.97:0/1857115162 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fa81a1bc0 con 0x7f6fa8103d90 2024-03-27T10:57:07.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.763+0000 7f6fad000640 1 --2- 172.21.15.97:0/1857115162 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6fa8103d90 0x7f6fa8199fa0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c005ab0 tx=0x7f6f9c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:07.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.764+0000 7f6fa5600640 1 -- 172.21.15.97:0/1857115162 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f9c043020 con 0x7f6fa8103d90 2024-03-27T10:57:07.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.764+0000 7f6fa5600640 1 -- 172.21.15.97:0/1857115162 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6f9c0054c0 con 0x7f6fa8103d90 2024-03-27T10:57:07.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.764+0000 7f6fafa00640 1 -- 172.21.15.97:0/1857115162 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fa81a1e50 con 0x7f6fa8103d90 2024-03-27T10:57:07.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.764+0000 7f6fa5600640 1 -- 172.21.15.97:0/1857115162 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f9c003d10 con 0x7f6fa8103d90 2024-03-27T10:57:07.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.764+0000 7f6fafa00640 1 -- 172.21.15.97:0/1857115162 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fa81a22b0 con 0x7f6fa8103d90 2024-03-27T10:57:07.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.765+0000 7f6fafa00640 1 -- 172.21.15.97:0/1857115162 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fa80689b0 con 0x7f6fa8103d90 2024-03-27T10:57:07.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.767+0000 7f6fa5600640 1 -- 172.21.15.97:0/1857115162 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f6f9c046020 con 0x7f6fa8103d90 2024-03-27T10:57:07.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.768+0000 7f6fa5600640 1 --2- 172.21.15.97:0/1857115162 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f6f900a3610 0x7f6f900a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:07.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.769+0000 7f6fa5600640 1 -- 172.21.15.97:0/1857115162 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f6f9c0f2530 con 0x7f6fa8103d90 2024-03-27T10:57:07.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.769+0000 7f6fa7e00640 1 --2- 172.21.15.97:0/1857115162 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f6f900a3610 0x7f6f900a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:07.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.770+0000 7f6fa7e00640 1 --2- 172.21.15.97:0/1857115162 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f6f900a3610 0x7f6f900a5ad0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f6f94005fd0 tx=0x7f6f94005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:07.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:07.772+0000 7f6fa5600640 1 -- 172.21.15.97:0/1857115162 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f6f9c04d020 con 0x7f6fa8103d90 2024-03-27T10:57:07.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.012+0000 7f6fafa00640 1 -- 172.21.15.97:0/1857115162 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6fa810b130 con 0x7f6fa8103d90 2024-03-27T10:57:07.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.014+0000 7f6fa5600640 1 -- 172.21.15.97:0/1857115162 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11944 (secure 0 0 0) 0x7f6f9c0bf0b0 con 0x7f6fa8103d90 2024-03-27T10:57:07.986 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T10:57:07.986 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1009423904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1009423904},{"type":"v1","addr":"172.21.15.97:6835","nonce":1009423904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3251173433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3251173433},{"type":"v1","addr":"172.21.15.97:6837","nonce":3251173433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-27T10:54:04.510178+0000","modified":"2024-03-27T10:56:28.179534+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34508,"mds_1":34478,"mds_2":34445,"mds_3":24454,"mds_4":24400},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14646":{"gid":14646,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6839/1057596821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1057596821},{"type":"v1","addr":"172.21.15.97:6839","nonce":1057596821}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14658":{"gid":14658,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6841/1672464856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6840","nonce":1672464856},{"type":"v1","addr":"172.21.15.97:6841","nonce":1672464856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.177:6833/3787981632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3787981632},{"type":"v1","addr":"172.21.15.177:6833","nonce":3787981632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.177:6835/3349112132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":3349112132},{"type":"v1","addr":"172.21.15.177:6835","nonce":3349112132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6837/3244432640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":3244432640},{"type":"v1","addr":"172.21.15.177:6837","nonce":3244432640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6839/3340454605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":3340454605},{"type":"v1","addr":"172.21.15.177:6839","nonce":3340454605}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34415":{"gid":34415,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6833/4052517902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":4052517902},{"type":"v1","addr":"172.21.15.114:6833","nonce":4052517902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34445":{"gid":34445,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.114:6835/3581918222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3581918222},{"type":"v1","addr":"172.21.15.114:6835","nonce":3581918222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34478":{"gid":34478,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.114:6837/616775588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":616775588},{"type":"v1","addr":"172.21.15.114:6837","nonce":616775588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34508":{"gid":34508,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.114:6839/3937193784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":3937193784},{"type":"v1","addr":"172.21.15.114:6839","nonce":3937193784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34508,"qdb_cluster":[34478,24454,24400,34445,34508]},"id":1}]} 2024-03-27T10:57:07.986 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 28 2024-03-27T10:57:07.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.016+0000 7f6fafa00640 1 -- 172.21.15.97:0/1857115162 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f6f900a3610 msgr2=0x7f6f900a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:07.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.016+0000 7f6fafa00640 1 --2- 172.21.15.97:0/1857115162 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f6f900a3610 0x7f6f900a5ad0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f6f94005fd0 tx=0x7f6f94005850 comp rx=0 tx=0).stop 2024-03-27T10:57:07.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.016+0000 7f6fafa00640 1 -- 172.21.15.97:0/1857115162 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6fa8103d90 msgr2=0x7f6fa8199fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:07.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.016+0000 7f6fafa00640 1 --2- 172.21.15.97:0/1857115162 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6fa8103d90 0x7f6fa8199fa0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c005ab0 tx=0x7f6f9c0042f0 comp rx=0 tx=0).stop 2024-03-27T10:57:07.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.016+0000 7f6fafa00640 1 -- 172.21.15.97:0/1857115162 shutdown_connections 2024-03-27T10:57:07.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.016+0000 7f6fafa00640 1 --2- 172.21.15.97:0/1857115162 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6fa8105940 0x7f6fa81a1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:07.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.016+0000 7f6fafa00640 1 --2- 172.21.15.97:0/1857115162 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f6f900a3610 0x7f6f900a5ad0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:07.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.016+0000 7f6fafa00640 1 --2- 172.21.15.97:0/1857115162 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6fa8104f80 0x7f6fa819a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:07.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.016+0000 7f6fafa00640 1 --2- 172.21.15.97:0/1857115162 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6fa8103d90 0x7f6fa8199fa0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:07.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.016+0000 7f6fafa00640 1 -- 172.21.15.97:0/1857115162 >> 172.21.15.97:0/1857115162 conn(0x7f6fa80ff300 msgr2=0x7f6fa8108300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:07.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.016+0000 7f6fafa00640 1 -- 172.21.15.97:0/1857115162 shutdown_connections 2024-03-27T10:57:07.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.016+0000 7f6fafa00640 1 -- 172.21.15.97:0/1857115162 wait complete. 2024-03-27T10:57:08.007 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-27T10:57:08.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.237+0000 7fdd90400640 1 -- 172.21.15.97:0/4021066660 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd880780f0 msgr2=0x7fdd88078990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:08.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.237+0000 7fdd90400640 1 --2- 172.21.15.97:0/4021066660 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd880780f0 0x7fdd88078990 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fdd80009ca0 tx=0x7fdd8002c0f0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.237+0000 7fdd90400640 1 -- 172.21.15.97:0/4021066660 shutdown_connections 2024-03-27T10:57:08.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.237+0000 7fdd90400640 1 --2- 172.21.15.97:0/4021066660 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd880780f0 0x7fdd88078990 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.237+0000 7fdd90400640 1 --2- 172.21.15.97:0/4021066660 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdd88077750 0x7fdd88077bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.237+0000 7fdd90400640 1 --2- 172.21.15.97:0/4021066660 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdd88076510 0x7fdd88076930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.237+0000 7fdd90400640 1 -- 172.21.15.97:0/4021066660 >> 172.21.15.97:0/4021066660 conn(0x7fdd880ff300 msgr2=0x7fdd88101720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:08.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.237+0000 7fdd90400640 1 -- 172.21.15.97:0/4021066660 shutdown_connections 2024-03-27T10:57:08.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.237+0000 7fdd90400640 1 -- 172.21.15.97:0/4021066660 wait complete. 2024-03-27T10:57:08.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.238+0000 7fdd90400640 1 Processor -- start 2024-03-27T10:57:08.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.238+0000 7fdd90400640 1 -- start start 2024-03-27T10:57:08.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.238+0000 7fdd90400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdd88076510 0x7fdd8819e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:08.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.238+0000 7fdd90400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd88077750 0x7fdd8819e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:08.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.238+0000 7fdd90400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdd880780f0 0x7fdd881a58c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:08.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.238+0000 7fdd90400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdd88113cb0 con 0x7fdd88077750 2024-03-27T10:57:08.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.238+0000 7fdd90400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fdd88113b30 con 0x7fdd880780f0 2024-03-27T10:57:08.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.238+0000 7fdd90400640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fdd88113e30 con 0x7fdd88076510 2024-03-27T10:57:08.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.239+0000 7fdd8d000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd88077750 0x7fdd8819e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:08.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.239+0000 7fdd8d000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd88077750 0x7fdd8819e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48464/0 (socket says 172.21.15.97:48464) 2024-03-27T10:57:08.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.239+0000 7fdd8d000640 1 -- 172.21.15.97:0/1997568173 learned_addr learned my addr 172.21.15.97:0/1997568173 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:08.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.239+0000 7fdd8e400640 1 --2- 172.21.15.97:0/1997568173 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdd880780f0 0x7fdd881a58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:08.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.239+0000 7fdd8da00640 1 --2- 172.21.15.97:0/1997568173 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdd88076510 0x7fdd8819e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:08.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.239+0000 7fdd8d000640 1 -- 172.21.15.97:0/1997568173 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdd88076510 msgr2=0x7fdd8819e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:08.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.239+0000 7fdd8d000640 1 --2- 172.21.15.97:0/1997568173 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdd88076510 0x7fdd8819e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.239+0000 7fdd8d000640 1 -- 172.21.15.97:0/1997568173 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdd880780f0 msgr2=0x7fdd881a58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:08.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.239+0000 7fdd8d000640 1 --2- 172.21.15.97:0/1997568173 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdd880780f0 0x7fdd881a58c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.239+0000 7fdd8d000640 1 -- 172.21.15.97:0/1997568173 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd881a5f30 con 0x7fdd88077750 2024-03-27T10:57:08.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.239+0000 7fdd8d000640 1 --2- 172.21.15.97:0/1997568173 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd88077750 0x7fdd8819e850 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fdd7400d950 tx=0x7fdd7400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:08.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.240+0000 7fdd86000640 1 -- 172.21.15.97:0/1997568173 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd74017030 con 0x7fdd88077750 2024-03-27T10:57:08.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.240+0000 7fdd90400640 1 -- 172.21.15.97:0/1997568173 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd881a6220 con 0x7fdd88077750 2024-03-27T10:57:08.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.240+0000 7fdd86000640 1 -- 172.21.15.97:0/1997568173 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdd74004020 con 0x7fdd88077750 2024-03-27T10:57:08.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.240+0000 7fdd86000640 1 -- 172.21.15.97:0/1997568173 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd740047a0 con 0x7fdd88077750 2024-03-27T10:57:08.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.240+0000 7fdd90400640 1 -- 172.21.15.97:0/1997568173 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd881a6770 con 0x7fdd88077750 2024-03-27T10:57:08.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.244+0000 7fdd86000640 1 -- 172.21.15.97:0/1997568173 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7fdd740041d0 con 0x7fdd88077750 2024-03-27T10:57:08.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.244+0000 7fdd93012740 1 Processor -- start 2024-03-27T10:57:08.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.245+0000 7fdd93012740 1 -- start start 2024-03-27T10:57:08.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.245+0000 7fdd93012740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5613b6af88c0 0x5613b6af8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:08.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.245+0000 7fdd93012740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5613b6af92b0 0x5613b6afddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:08.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.245+0000 7fdd93012740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5613b6afe420 0x5613b6b008b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:08.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.245+0000 7fdd93012740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5613b6ac07b0 con 0x5613b6afe420 2024-03-27T10:57:08.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.245+0000 7fdd93012740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5613b6a5d670 con 0x5613b6af88c0 2024-03-27T10:57:08.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.245+0000 7fdd93012740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5613b6a05970 con 0x5613b6af92b0 2024-03-27T10:57:08.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.245+0000 7fdd8d000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5613b6afe420 0x5613b6b008b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:08.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.245+0000 7fdd8d000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5613b6afe420 0x5613b6b008b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48480/0 (socket says 172.21.15.97:48480) 2024-03-27T10:57:08.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.245+0000 7fdd8d000640 1 -- 172.21.15.97:0/3042543863 learned_addr learned my addr 172.21.15.97:0/3042543863 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:08.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.246+0000 7fdd8e400640 1 --2- 172.21.15.97:0/3042543863 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5613b6af88c0 0x5613b6af8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:08.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.246+0000 7fdd8da00640 1 --2- 172.21.15.97:0/3042543863 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5613b6af92b0 0x5613b6afddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:08.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.246+0000 7fdd86000640 1 --2- 172.21.15.97:0/1997568173 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fdd700a3610 0x7fdd700a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:08.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.246+0000 7fdd86000640 1 -- 172.21.15.97:0/1997568173 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fdd740cf8c0 con 0x7fdd88077750 2024-03-27T10:57:08.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.246+0000 7fdd8d000640 1 -- 172.21.15.97:0/3042543863 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5613b6af92b0 msgr2=0x5613b6afddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:08.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.246+0000 7fdd8d000640 1 --2- 172.21.15.97:0/3042543863 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5613b6af92b0 0x5613b6afddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.246+0000 7fdd8d000640 1 -- 172.21.15.97:0/3042543863 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5613b6af88c0 msgr2=0x5613b6af8ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:08.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.246+0000 7fdd8d000640 1 --2- 172.21.15.97:0/3042543863 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5613b6af88c0 0x5613b6af8ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.246+0000 7fdd8e400640 1 --2- 172.21.15.97:0/1997568173 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fdd700a3610 0x7fdd700a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:08.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.246+0000 7fdd8d000640 1 -- 172.21.15.97:0/3042543863 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613b67713f0 con 0x5613b6afe420 2024-03-27T10:57:08.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.246+0000 7fdd8d000640 1 --2- 172.21.15.97:0/3042543863 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5613b6afe420 0x5613b6b008b0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fdd740cd370 tx=0x7fdd740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3370158d39a6848 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:08.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.247+0000 7fdd7aa00640 1 -- 172.21.15.97:0/3042543863 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd7409bd40 con 0x5613b6afe420 2024-03-27T10:57:08.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.247+0000 7fdd7aa00640 1 -- 172.21.15.97:0/3042543863 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdd740a00f0 con 0x5613b6afe420 2024-03-27T10:57:08.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.247+0000 7fdd7aa00640 1 -- 172.21.15.97:0/3042543863 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd740a4520 con 0x5613b6afe420 2024-03-27T10:57:08.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.248+0000 7fdd93012740 1 -- 172.21.15.97:0/3042543863 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5613b6afe420 msgr2=0x5613b6b008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:08.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.248+0000 7fdd8e400640 1 --2- 172.21.15.97:0/1997568173 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fdd700a3610 0x7fdd700a5ad0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fdd80006010 tx=0x7fdd8002c0f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:08.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.248+0000 7fdd93012740 1 --2- 172.21.15.97:0/3042543863 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5613b6afe420 0x5613b6b008b0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fdd740cd370 tx=0x7fdd740046c0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.248+0000 7fdd93012740 1 -- 172.21.15.97:0/3042543863 shutdown_connections 2024-03-27T10:57:08.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.248+0000 7fdd93012740 1 --2- 172.21.15.97:0/3042543863 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5613b6afe420 0x5613b6b008b0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.248+0000 7fdd93012740 1 --2- 172.21.15.97:0/3042543863 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5613b6af92b0 0x5613b6afddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.248+0000 7fdd93012740 1 --2- 172.21.15.97:0/3042543863 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5613b6af88c0 0x5613b6af8ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.248+0000 7fdd93012740 1 -- 172.21.15.97:0/3042543863 >> 172.21.15.97:0/3042543863 conn(0x5613b6af3e30 msgr2=0x5613b6af4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:08.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.248+0000 7fdd93012740 1 -- 172.21.15.97:0/3042543863 shutdown_connections 2024-03-27T10:57:08.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.248+0000 7fdd93012740 1 -- 172.21.15.97:0/3042543863 wait complete. 2024-03-27T10:57:08.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.248+0000 7fdd93012740 1 -- start start 2024-03-27T10:57:08.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.248+0000 7fdd93012740 1 Processor -- start 2024-03-27T10:57:08.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.249+0000 7fdd93012740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5613b6af88c0 0x5613b6b02a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:08.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.249+0000 7fdd93012740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5613b6af92b0 0x5613b6b02fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:08.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.249+0000 7fdd93012740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5613b6afe420 0x5613b6b09ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:08.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.249+0000 7fdd93012740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5613b6a18ca0 con 0x5613b6af88c0 2024-03-27T10:57:08.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.249+0000 7fdd93012740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5613b6a59620 con 0x5613b6af92b0 2024-03-27T10:57:08.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.249+0000 7fdd93012740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5613b6b01370 con 0x5613b6afe420 2024-03-27T10:57:08.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.249+0000 7fdd93012740 10 client.0 _ll_register_callbacks cb 0x5613b6a55160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:57:08.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.249+0000 7fdd8da00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5613b6af88c0 0x5613b6b02a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:08.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.249+0000 7fdd8da00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5613b6af88c0 0x5613b6b02a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48482/0 (socket says 172.21.15.97:48482) 2024-03-27T10:57:08.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.249+0000 7fdd8da00640 1 -- 172.21.15.97:0/1789034861 learned_addr learned my addr 172.21.15.97:0/1789034861 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:08.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.249+0000 7fdd8d000640 1 --2- 172.21.15.97:0/1789034861 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5613b6af92b0 0x5613b6b02fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:08.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.249+0000 7fdd8e400640 1 --2- 172.21.15.97:0/1789034861 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5613b6afe420 0x5613b6b09ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:08.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.250+0000 7fdd8da00640 1 -- 172.21.15.97:0/1789034861 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5613b6afe420 msgr2=0x5613b6b09ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:08.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.250+0000 7fdd8da00640 1 --2- 172.21.15.97:0/1789034861 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5613b6afe420 0x5613b6b09ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.250+0000 7fdd8da00640 1 -- 172.21.15.97:0/1789034861 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5613b6af92b0 msgr2=0x5613b6b02fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:08.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.250+0000 7fdd8da00640 1 --2- 172.21.15.97:0/1789034861 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5613b6af92b0 0x5613b6b02fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.250+0000 7fdd8da00640 1 -- 172.21.15.97:0/1789034861 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613b683b7d0 con 0x5613b6af88c0 2024-03-27T10:57:08.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.250+0000 7fdd8e400640 1 --2- 172.21.15.97:0/1789034861 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5613b6afe420 0x5613b6b09ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:57:08.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.250+0000 7fdd8da00640 1 --2- 172.21.15.97:0/1789034861 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5613b6af88c0 0x5613b6b02a70 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c00bf00 tx=0x7fdd7c012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93ce3b4e41f8b359 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:08.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.250+0000 7fdd6f400640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T10:57:08.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.252+0000 7fdd6f400640 1 -- 172.21.15.97:0/1789034861 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd7c01f060 con 0x5613b6af88c0 2024-03-27T10:57:08.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.252+0000 7fdd6f400640 1 -- 172.21.15.97:0/1789034861 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fdd7c0108a0 con 0x5613b6af88c0 2024-03-27T10:57:08.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.252+0000 7fdd93012740 1 -- 172.21.15.97:0/1789034861 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd740a00f0 con 0x5613b6af88c0 2024-03-27T10:57:08.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.253+0000 7fdd6f400640 1 -- 172.21.15.97:0/1789034861 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fdd7c010aa0 con 0x5613b6af88c0 2024-03-27T10:57:08.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.253+0000 7fdd93012740 10 client.15585 fetch_fsmap learned FSMap version 28 2024-03-27T10:57:08.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.253+0000 7fdd93012740 1 -- 172.21.15.97:0/1789034861 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5613b6b0d430 con 0x5613b6af88c0 2024-03-27T10:57:08.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.253+0000 7fdd6f400640 1 -- 172.21.15.97:0/1789034861 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdd740a00f0 con 0x5613b6af88c0 2024-03-27T10:57:08.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.253+0000 7fdd6f400640 1 -- 172.21.15.97:0/1789034861 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fdd7c019030 con 0x5613b6af88c0 2024-03-27T10:57:08.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.253+0000 7fdd93012740 10 client.15585 fetch_fsmap finished waiting for FSMap version 28 2024-03-27T10:57:08.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.253+0000 7fdd93012740 10 client.15585 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-03-27T10:57:08.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.253+0000 7fdd93012740 20 client.15585 populate_metadata read hostname 'smithi097' 2024-03-27T10:57:08.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.253+0000 7fdd93012740 1 --2- 172.21.15.97:0/1789034861 >> [v2:172.21.15.177:6832/3787981632,v1:172.21.15.177:6833/3787981632] conn(0x5613b6b0eb30 0x5613b6b10ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:08.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.253+0000 7fdd93012740 4 client.15585 mds_command: new command op to 24400 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T10:57:08.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.253+0000 7fdd93012740 1 -- 172.21.15.97:0/1789034861 --> [v2:172.21.15.177:6832/3787981632,v1:172.21.15.177:6833/3787981632] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5613b6a6abb0 con 0x5613b6b0eb30 2024-03-27T10:57:08.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.254+0000 7fdd8d000640 1 --2- 172.21.15.97:0/1789034861 >> [v2:172.21.15.177:6832/3787981632,v1:172.21.15.177:6833/3787981632] conn(0x5613b6b0eb30 0x5613b6b10ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:08.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.255+0000 7fdd8d000640 1 --2- 172.21.15.97:0/1789034861 >> [v2:172.21.15.177:6832/3787981632,v1:172.21.15.177:6833/3787981632] conn(0x5613b6b0eb30 0x5613b6b10ff0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=27f45d43985d2c4b server_cookie=c6e03da1bc5ed1ab in_seq=0 out_seq=0 2024-03-27T10:57:08.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.255+0000 7fdd6f400640 10 client.15585 ms_handle_connect on v2:172.21.15.177:6832/3787981632 2024-03-27T10:57:08.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.257+0000 7fdd6f400640 1 -- 172.21.15.97:0/1789034861 <== mds.4 v2:172.21.15.177:6832/3787981632 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5613b6a6abb0 con 0x5613b6b0eb30 2024-03-27T10:57:08.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.257+0000 7fdd6f400640 10 client.15585 handle_command_reply: tid=0 2024-03-27T10:57:08.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.258+0000 7fdd93012740 1 client.15585 shutdown 2024-03-27T10:57:08.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.258+0000 7fdd93012740 1 -- 172.21.15.97:0/1789034861 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5613b6af88c0 msgr2=0x5613b6b02a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:08.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.258+0000 7fdd93012740 1 --2- 172.21.15.97:0/1789034861 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5613b6af88c0 0x5613b6b02a70 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c00bf00 tx=0x7fdd7c012820 comp rx=0 tx=0).stop 2024-03-27T10:57:08.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.258+0000 7fdd93012740 1 -- 172.21.15.97:0/1789034861 shutdown_connections 2024-03-27T10:57:08.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.258+0000 7fdd93012740 1 --2- 172.21.15.97:0/1789034861 >> [v2:172.21.15.177:6832/3787981632,v1:172.21.15.177:6833/3787981632] conn(0x5613b6b0eb30 0x5613b6b10ff0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.258+0000 7fdd93012740 1 --2- 172.21.15.97:0/1789034861 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5613b6afe420 0x5613b6b09ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.258+0000 7fdd93012740 1 --2- 172.21.15.97:0/1789034861 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5613b6af92b0 0x5613b6b02fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.258+0000 7fdd93012740 1 --2- 172.21.15.97:0/1789034861 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5613b6af88c0 0x5613b6b02a70 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.258+0000 7fdd6f400640 0 client.15585 ms_handle_reset on v2:172.21.15.177:6832/3787981632 2024-03-27T10:57:08.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.258+0000 7fdd93012740 1 -- 172.21.15.97:0/1789034861 >> 172.21.15.97:0/1789034861 conn(0x5613b6af3e30 msgr2=0x5613b6af3030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:08.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.258+0000 7fdd93012740 1 -- 172.21.15.97:0/1789034861 shutdown_connections 2024-03-27T10:57:08.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.258+0000 7fdd93012740 1 -- 172.21.15.97:0/1789034861 wait complete. 2024-03-27T10:57:08.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.258+0000 7fdd93012740 20 client.15585 trim_cache size 0 max 16384 2024-03-27T10:57:08.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.272+0000 7fdd93012740 1 Processor -- start 2024-03-27T10:57:08.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.272+0000 7fdd93012740 1 -- start start 2024-03-27T10:57:08.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.272+0000 7fdd93012740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5613b6af88c0 0x5613b6af7b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:08.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.272+0000 7fdd93012740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5613b6af92b0 0x5613b6b09fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:08.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.272+0000 7fdd93012740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5613b6afe420 0x5613b6b168c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:08.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.272+0000 7fdd93012740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdd881af5d0 con 0x5613b6afe420 2024-03-27T10:57:08.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.272+0000 7fdd93012740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fdd881b1110 con 0x5613b6af88c0 2024-03-27T10:57:08.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.272+0000 7fdd93012740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fdd881afca0 con 0x5613b6af92b0 2024-03-27T10:57:08.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.273+0000 7fdd8e400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5613b6afe420 0x5613b6b168c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:08.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.273+0000 7fdd8e400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5613b6afe420 0x5613b6b168c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48484/0 (socket says 172.21.15.97:48484) 2024-03-27T10:57:08.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.273+0000 7fdd8e400640 1 -- 172.21.15.97:0/2260958493 learned_addr learned my addr 172.21.15.97:0/2260958493 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:08.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.273+0000 7fdd8da00640 1 --2- 172.21.15.97:0/2260958493 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5613b6af88c0 0x5613b6af7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:08.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.273+0000 7fdd8d000640 1 --2- 172.21.15.97:0/2260958493 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5613b6af92b0 0x5613b6b09fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:08.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.273+0000 7fdd8e400640 1 -- 172.21.15.97:0/2260958493 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5613b6af92b0 msgr2=0x5613b6b09fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:08.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.273+0000 7fdd8e400640 1 --2- 172.21.15.97:0/2260958493 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5613b6af92b0 0x5613b6b09fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.273+0000 7fdd8e400640 1 -- 172.21.15.97:0/2260958493 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5613b6af88c0 msgr2=0x5613b6af7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:08.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.273+0000 7fdd8e400640 1 --2- 172.21.15.97:0/2260958493 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5613b6af88c0 0x5613b6af7b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.273+0000 7fdd8e400640 1 -- 172.21.15.97:0/2260958493 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613b6af4ba0 con 0x5613b6afe420 2024-03-27T10:57:08.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.273+0000 7fdd8e400640 1 --2- 172.21.15.97:0/2260958493 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5613b6afe420 0x5613b6b168c0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fdd80032550 tx=0x7fdd80005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de3bb8fd4cf0a639 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:08.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.274+0000 7fdd7a000640 1 -- 172.21.15.97:0/2260958493 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd80053070 con 0x5613b6afe420 2024-03-27T10:57:08.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.274+0000 7fdd7a000640 1 -- 172.21.15.97:0/2260958493 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdd80003cf0 con 0x5613b6afe420 2024-03-27T10:57:08.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.274+0000 7fdd7a000640 1 -- 172.21.15.97:0/2260958493 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd80039040 con 0x5613b6afe420 2024-03-27T10:57:08.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.274+0000 7fdd93012740 1 -- 172.21.15.97:0/2260958493 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5613b6afe420 msgr2=0x5613b6b168c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:08.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.274+0000 7fdd93012740 1 --2- 172.21.15.97:0/2260958493 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5613b6afe420 0x5613b6b168c0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fdd80032550 tx=0x7fdd80005d80 comp rx=0 tx=0).stop 2024-03-27T10:57:08.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.275+0000 7fdd93012740 1 -- 172.21.15.97:0/2260958493 shutdown_connections 2024-03-27T10:57:08.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.275+0000 7fdd93012740 1 --2- 172.21.15.97:0/2260958493 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5613b6afe420 0x5613b6b168c0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.275+0000 7fdd93012740 1 --2- 172.21.15.97:0/2260958493 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5613b6af92b0 0x5613b6b09fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.275+0000 7fdd93012740 1 --2- 172.21.15.97:0/2260958493 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5613b6af88c0 0x5613b6af7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.275+0000 7fdd93012740 1 -- 172.21.15.97:0/2260958493 >> 172.21.15.97:0/2260958493 conn(0x5613b6af3e30 msgr2=0x5613b6af3030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:08.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.275+0000 7fdd93012740 1 -- 172.21.15.97:0/2260958493 shutdown_connections 2024-03-27T10:57:08.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.275+0000 7fdd93012740 1 -- 172.21.15.97:0/2260958493 wait complete. 2024-03-27T10:57:08.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.275+0000 7fdd93012740 1 -- start start 2024-03-27T10:57:08.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.275+0000 7fdd93012740 1 Processor -- start 2024-03-27T10:57:08.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.276+0000 7fdd93012740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5613b6af88c0 0x5613b6b07a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:08.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.276+0000 7fdd93012740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5613b6af92b0 0x5613b6b07f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:08.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.276+0000 7fdd93012740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5613b6afe420 0x5613b6b08f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:08.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.276+0000 7fdd93012740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdd881b0a40 con 0x5613b6af88c0 2024-03-27T10:57:08.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.276+0000 7fdd93012740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fdd881b0370 con 0x5613b6af92b0 2024-03-27T10:57:08.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.276+0000 7fdd93012740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fdd881ad5b0 con 0x5613b6afe420 2024-03-27T10:57:08.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.276+0000 7fdd93012740 10 client.0 _ll_register_callbacks cb 0x7fdd7c00b730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:57:08.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.276+0000 7fdd8da00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5613b6af88c0 0x5613b6b07a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:08.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.276+0000 7fdd8da00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5613b6af88c0 0x5613b6b07a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48488/0 (socket says 172.21.15.97:48488) 2024-03-27T10:57:08.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.276+0000 7fdd8da00640 1 -- 172.21.15.97:0/2231729103 learned_addr learned my addr 172.21.15.97:0/2231729103 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:08.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.276+0000 7fdd8d000640 1 --2- 172.21.15.97:0/2231729103 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5613b6af92b0 0x5613b6b07f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:08.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.276+0000 7fdd8e400640 1 --2- 172.21.15.97:0/2231729103 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5613b6afe420 0x5613b6b08f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:08.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.276+0000 7fdd8da00640 1 -- 172.21.15.97:0/2231729103 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5613b6afe420 msgr2=0x5613b6b08f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:08.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.276+0000 7fdd8da00640 1 --2- 172.21.15.97:0/2231729103 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5613b6afe420 0x5613b6b08f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.276+0000 7fdd8da00640 1 -- 172.21.15.97:0/2231729103 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5613b6af92b0 msgr2=0x5613b6b07f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:08.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.276+0000 7fdd8da00640 1 --2- 172.21.15.97:0/2231729103 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5613b6af92b0 0x5613b6b07f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.276+0000 7fdd8da00640 1 -- 172.21.15.97:0/2231729103 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613b6b09550 con 0x5613b6af88c0 2024-03-27T10:57:08.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.277+0000 7fdd8da00640 1 --2- 172.21.15.97:0/2231729103 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5613b6af88c0 0x5613b6b07a10 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c025ef0 tx=0x7fdd7c010ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3eb33509d4270a2 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:08.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.277+0000 7fdd6f400640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T10:57:08.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.277+0000 7fdd6f400640 1 -- 172.21.15.97:0/2231729103 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd7c033060 con 0x5613b6af88c0 2024-03-27T10:57:08.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.277+0000 7fdd93012740 1 -- 172.21.15.97:0/2231729103 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd80003cf0 con 0x5613b6af88c0 2024-03-27T10:57:08.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.277+0000 7fdd6f400640 1 -- 172.21.15.97:0/2231729103 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fdd7c0108a0 con 0x5613b6af88c0 2024-03-27T10:57:08.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.277+0000 7fdd6f400640 1 -- 172.21.15.97:0/2231729103 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fdd7c02a040 con 0x5613b6af88c0 2024-03-27T10:57:08.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.277+0000 7fdd93012740 10 client.15591 fetch_fsmap learned FSMap version 28 2024-03-27T10:57:08.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.277+0000 7fdd93012740 1 -- 172.21.15.97:0/2231729103 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5613b6b752c0 con 0x5613b6af88c0 2024-03-27T10:57:08.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.277+0000 7fdd6f400640 1 -- 172.21.15.97:0/2231729103 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdd80003cf0 con 0x5613b6af88c0 2024-03-27T10:57:08.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.278+0000 7fdd6f400640 1 -- 172.21.15.97:0/2231729103 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fdd7c02d030 con 0x5613b6af88c0 2024-03-27T10:57:08.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.278+0000 7fdd93012740 10 client.15591 fetch_fsmap finished waiting for FSMap version 28 2024-03-27T10:57:08.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.278+0000 7fdd93012740 10 client.15591 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-03-27T10:57:08.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.278+0000 7fdd93012740 20 client.15591 populate_metadata read hostname 'smithi097' 2024-03-27T10:57:08.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.278+0000 7fdd93012740 1 --2- 172.21.15.97:0/2231729103 >> [v2:172.21.15.177:6832/3787981632,v1:172.21.15.177:6833/3787981632] conn(0x5613b6b75c60 0x5613b6b78050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:08.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.278+0000 7fdd93012740 4 client.15591 mds_command: new command op to 24400 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-27T10:57:08.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.278+0000 7fdd93012740 1 -- 172.21.15.97:0/2231729103 --> [v2:172.21.15.177:6832/3787981632,v1:172.21.15.177:6833/3787981632] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5613b6a64f00 con 0x5613b6b75c60 2024-03-27T10:57:08.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.278+0000 7fdd8d000640 1 --2- 172.21.15.97:0/2231729103 >> [v2:172.21.15.177:6832/3787981632,v1:172.21.15.177:6833/3787981632] conn(0x5613b6b75c60 0x5613b6b78050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:08.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.279+0000 7fdd8d000640 1 --2- 172.21.15.97:0/2231729103 >> [v2:172.21.15.177:6832/3787981632,v1:172.21.15.177:6833/3787981632] conn(0x5613b6b75c60 0x5613b6b78050 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1a292aa820baeab2 server_cookie=55cb2f84460de867 in_seq=0 out_seq=0 2024-03-27T10:57:08.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.279+0000 7fdd6f400640 10 client.15591 ms_handle_connect on v2:172.21.15.177:6832/3787981632 2024-03-27T10:57:08.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.280+0000 7fdd6f400640 1 -- 172.21.15.97:0/2231729103 <== mds.4 v2:172.21.15.177:6832/3787981632 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5613b6a64f00 con 0x5613b6b75c60 2024-03-27T10:57:08.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.280+0000 7fdd6f400640 10 client.15591 handle_command_reply: tid=0 2024-03-27T10:57:08.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.280+0000 7fdd93012740 1 client.15591 shutdown 2024-03-27T10:57:08.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.280+0000 7fdd93012740 1 -- 172.21.15.97:0/2231729103 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5613b6af88c0 msgr2=0x5613b6b07a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:08.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.280+0000 7fdd93012740 1 --2- 172.21.15.97:0/2231729103 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5613b6af88c0 0x5613b6b07a10 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c025ef0 tx=0x7fdd7c010ef0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.280+0000 7fdd93012740 1 -- 172.21.15.97:0/2231729103 shutdown_connections 2024-03-27T10:57:08.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.280+0000 7fdd93012740 1 --2- 172.21.15.97:0/2231729103 >> [v2:172.21.15.177:6832/3787981632,v1:172.21.15.177:6833/3787981632] conn(0x5613b6b75c60 0x5613b6b78050 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.280+0000 7fdd93012740 1 --2- 172.21.15.97:0/2231729103 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5613b6afe420 0x5613b6b08f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.280+0000 7fdd93012740 1 --2- 172.21.15.97:0/2231729103 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5613b6af92b0 0x5613b6b07f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.280+0000 7fdd6f400640 0 client.15591 ms_handle_reset on v2:172.21.15.177:6832/3787981632 2024-03-27T10:57:08.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.280+0000 7fdd93012740 1 --2- 172.21.15.97:0/2231729103 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5613b6af88c0 0x5613b6b07a10 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.280+0000 7fdd93012740 1 -- 172.21.15.97:0/2231729103 >> 172.21.15.97:0/2231729103 conn(0x5613b6af3e30 msgr2=0x5613b6b158b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:08.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.280+0000 7fdd93012740 1 -- 172.21.15.97:0/2231729103 shutdown_connections 2024-03-27T10:57:08.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.280+0000 7fdd93012740 1 -- 172.21.15.97:0/2231729103 wait complete. 2024-03-27T10:57:08.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.280+0000 7fdd93012740 20 client.15591 trim_cache size 0 max 16384 2024-03-27T10:57:08.256 INFO:teuthology.orchestra.run.smithi097.stdout:[] 2024-03-27T10:57:08.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.281+0000 7fdd90400640 1 -- 172.21.15.97:0/1997568173 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fdd700a3610 msgr2=0x7fdd700a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:08.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.281+0000 7fdd90400640 1 --2- 172.21.15.97:0/1997568173 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fdd700a3610 0x7fdd700a5ad0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fdd80006010 tx=0x7fdd8002c0f0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.281+0000 7fdd90400640 1 -- 172.21.15.97:0/1997568173 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd88077750 msgr2=0x7fdd8819e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:08.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.281+0000 7fdd90400640 1 --2- 172.21.15.97:0/1997568173 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd88077750 0x7fdd8819e850 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fdd7400d950 tx=0x7fdd7400de10 comp rx=0 tx=0).stop 2024-03-27T10:57:08.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.281+0000 7fdd90400640 1 -- 172.21.15.97:0/1997568173 shutdown_connections 2024-03-27T10:57:08.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.281+0000 7fdd90400640 1 --2- 172.21.15.97:0/1997568173 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fdd700a3610 0x7fdd700a5ad0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.281+0000 7fdd90400640 1 --2- 172.21.15.97:0/1997568173 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdd880780f0 0x7fdd881a58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.281+0000 7fdd90400640 1 --2- 172.21.15.97:0/1997568173 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd88077750 0x7fdd8819e850 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.281+0000 7fdd90400640 1 --2- 172.21.15.97:0/1997568173 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdd88076510 0x7fdd8819e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.281+0000 7fdd90400640 1 -- 172.21.15.97:0/1997568173 >> 172.21.15.97:0/1997568173 conn(0x7fdd880ff300 msgr2=0x7fdd88100e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:08.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.281+0000 7fdd90400640 1 -- 172.21.15.97:0/1997568173 shutdown_connections 2024-03-27T10:57:08.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.281+0000 7fdd90400640 1 -- 172.21.15.97:0/1997568173 wait complete. 2024-03-27T10:57:08.267 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:57:08 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/1857115162' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T10:57:08.270 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-27T10:57:08.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.499+0000 7fc47a000640 1 -- 172.21.15.97:0/2566440158 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc474105940 msgr2=0x7fc47410c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:08.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.499+0000 7fc47a000640 1 --2- 172.21.15.97:0/2566440158 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc474105940 0x7fc47410c1d0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fc46c009ca0 tx=0x7fc46c02c0f0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.499+0000 7fc47a000640 1 -- 172.21.15.97:0/2566440158 shutdown_connections 2024-03-27T10:57:08.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.499+0000 7fc47a000640 1 --2- 172.21.15.97:0/2566440158 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc474105940 0x7fc47410c1d0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.499+0000 7fc47a000640 1 --2- 172.21.15.97:0/2566440158 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc474104f80 0x7fc474105400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.499+0000 7fc47a000640 1 --2- 172.21.15.97:0/2566440158 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc474103d90 0x7fc4741041b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.499+0000 7fc47a000640 1 -- 172.21.15.97:0/2566440158 >> 172.21.15.97:0/2566440158 conn(0x7fc4740ff300 msgr2=0x7fc474101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:08.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.499+0000 7fc47a000640 1 -- 172.21.15.97:0/2566440158 shutdown_connections 2024-03-27T10:57:08.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.499+0000 7fc47a000640 1 -- 172.21.15.97:0/2566440158 wait complete. 2024-03-27T10:57:08.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.500+0000 7fc47a000640 1 Processor -- start 2024-03-27T10:57:08.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.500+0000 7fc47a000640 1 -- start start 2024-03-27T10:57:08.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.501+0000 7fc47a000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc474103d90 0x7fc474199f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:08.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.501+0000 7fc47a000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc474104f80 0x7fc47419a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:08.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.501+0000 7fc47a000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc474105940 0x7fc4741a1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:08.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.501+0000 7fc47a000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc47410f8c0 con 0x7fc474104f80 2024-03-27T10:57:08.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.501+0000 7fc47a000640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc47410f740 con 0x7fc474105940 2024-03-27T10:57:08.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.501+0000 7fc47a000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc47410fa40 con 0x7fc474103d90 2024-03-27T10:57:08.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.501+0000 7fc472a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc474104f80 0x7fc47419a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:08.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.501+0000 7fc472a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc474104f80 0x7fc47419a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48506/0 (socket says 172.21.15.97:48506) 2024-03-27T10:57:08.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.501+0000 7fc472a00640 1 -- 172.21.15.97:0/2556847143 learned_addr learned my addr 172.21.15.97:0/2556847143 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:08.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.501+0000 7fc473400640 1 --2- 172.21.15.97:0/2556847143 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc474103d90 0x7fc474199f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:08.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.501+0000 7fc473e00640 1 --2- 172.21.15.97:0/2556847143 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc474105940 0x7fc4741a1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:08.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.501+0000 7fc472a00640 1 -- 172.21.15.97:0/2556847143 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc474103d90 msgr2=0x7fc474199f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:08.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.501+0000 7fc472a00640 1 --2- 172.21.15.97:0/2556847143 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc474103d90 0x7fc474199f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.501+0000 7fc472a00640 1 -- 172.21.15.97:0/2556847143 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc474105940 msgr2=0x7fc4741a1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:08.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.501+0000 7fc472a00640 1 --2- 172.21.15.97:0/2556847143 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc474105940 0x7fc4741a1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.501+0000 7fc472a00640 1 -- 172.21.15.97:0/2556847143 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4741a1b90 con 0x7fc474104f80 2024-03-27T10:57:08.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.502+0000 7fc472a00640 1 --2- 172.21.15.97:0/2556847143 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc474104f80 0x7fc47419a4b0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fc46000d770 tx=0x7fc46000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:08.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.502+0000 7fc46be00640 1 -- 172.21.15.97:0/2556847143 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc460017070 con 0x7fc474104f80 2024-03-27T10:57:08.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.502+0000 7fc46be00640 1 -- 172.21.15.97:0/2556847143 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc4600054d0 con 0x7fc474104f80 2024-03-27T10:57:08.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.502+0000 7fc47a000640 1 -- 172.21.15.97:0/2556847143 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4741a1e80 con 0x7fc474104f80 2024-03-27T10:57:08.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.502+0000 7fc46be00640 1 -- 172.21.15.97:0/2556847143 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc460004cf0 con 0x7fc474104f80 2024-03-27T10:57:08.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.502+0000 7fc47a000640 1 -- 172.21.15.97:0/2556847143 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4741a22f0 con 0x7fc474104f80 2024-03-27T10:57:08.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.505+0000 7fc47cc5c740 1 Processor -- start 2024-03-27T10:57:08.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.505+0000 7fc46be00640 1 -- 172.21.15.97:0/2556847143 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7fc460004e90 con 0x7fc474104f80 2024-03-27T10:57:08.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.506+0000 7fc47cc5c740 1 -- start start 2024-03-27T10:57:08.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.506+0000 7fc47cc5c740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b882ab58c0 0x55b882ab5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:08.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.506+0000 7fc47cc5c740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b882ab62b0 0x55b882abadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:08.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.506+0000 7fc47cc5c740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b882abb420 0x55b882abd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:08.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.506+0000 7fc47cc5c740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55b882a7d7b0 con 0x55b882ab58c0 2024-03-27T10:57:08.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.506+0000 7fc47cc5c740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55b882a1a670 con 0x55b882abb420 2024-03-27T10:57:08.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.506+0000 7fc47cc5c740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55b8829c2970 con 0x55b882ab62b0 2024-03-27T10:57:08.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.506+0000 7fc473e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b882ab58c0 0x55b882ab5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:08.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.506+0000 7fc473e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b882ab58c0 0x55b882ab5ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48522/0 (socket says 172.21.15.97:48522) 2024-03-27T10:57:08.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.506+0000 7fc473e00640 1 -- 172.21.15.97:0/1013137679 learned_addr learned my addr 172.21.15.97:0/1013137679 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:08.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.506+0000 7fc473400640 1 --2- 172.21.15.97:0/1013137679 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b882ab62b0 0x55b882abadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:08.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.506+0000 7fc472a00640 1 --2- 172.21.15.97:0/1013137679 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b882abb420 0x55b882abd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:08.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.507+0000 7fc473e00640 1 -- 172.21.15.97:0/1013137679 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b882ab62b0 msgr2=0x55b882abadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:08.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.507+0000 7fc473e00640 1 --2- 172.21.15.97:0/1013137679 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b882ab62b0 0x55b882abadb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.507+0000 7fc473e00640 1 -- 172.21.15.97:0/1013137679 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b882abb420 msgr2=0x55b882abd8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:08.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.507+0000 7fc473e00640 1 --2- 172.21.15.97:0/1013137679 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b882abb420 0x55b882abd8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.507+0000 7fc473e00640 1 -- 172.21.15.97:0/1013137679 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b88272e3f0 con 0x55b882ab58c0 2024-03-27T10:57:08.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.507+0000 7fc473e00640 1 --2- 172.21.15.97:0/1013137679 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b882ab58c0 0x55b882ab5ce0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fc46c032550 tx=0x7fc46c0093e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb1af1b33f7c3a09 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:08.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.507+0000 7fc46be00640 1 --2- 172.21.15.97:0/2556847143 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fc4580a3610 0x7fc4580a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:08.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.507+0000 7fc45fe00640 1 -- 172.21.15.97:0/1013137679 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc46c043020 con 0x55b882ab58c0 2024-03-27T10:57:08.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.507+0000 7fc46be00640 1 -- 172.21.15.97:0/2556847143 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fc4600cf800 con 0x7fc474104f80 2024-03-27T10:57:08.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.507+0000 7fc45fe00640 1 -- 172.21.15.97:0/1013137679 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc46c032db0 con 0x55b882ab58c0 2024-03-27T10:57:08.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.507+0000 7fc45fe00640 1 -- 172.21.15.97:0/1013137679 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc46c039040 con 0x55b882ab58c0 2024-03-27T10:57:08.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.507+0000 7fc473400640 1 --2- 172.21.15.97:0/2556847143 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fc4580a3610 0x7fc4580a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:08.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.508+0000 7fc47cc5c740 1 -- 172.21.15.97:0/1013137679 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b882ab58c0 msgr2=0x55b882ab5ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:08.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.508+0000 7fc473400640 1 --2- 172.21.15.97:0/2556847143 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fc4580a3610 0x7fc4580a5ad0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc464005fd0 tx=0x7fc464005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:08.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.508+0000 7fc47cc5c740 1 --2- 172.21.15.97:0/1013137679 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b882ab58c0 0x55b882ab5ce0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fc46c032550 tx=0x7fc46c0093e0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.508+0000 7fc47cc5c740 1 -- 172.21.15.97:0/1013137679 shutdown_connections 2024-03-27T10:57:08.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.508+0000 7fc47cc5c740 1 --2- 172.21.15.97:0/1013137679 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b882abb420 0x55b882abd8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.508+0000 7fc47cc5c740 1 --2- 172.21.15.97:0/1013137679 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b882ab62b0 0x55b882abadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.508+0000 7fc47cc5c740 1 --2- 172.21.15.97:0/1013137679 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b882ab58c0 0x55b882ab5ce0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.508+0000 7fc47cc5c740 1 -- 172.21.15.97:0/1013137679 >> 172.21.15.97:0/1013137679 conn(0x55b882ab0e30 msgr2=0x55b882ab1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:08.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.508+0000 7fc47cc5c740 1 -- 172.21.15.97:0/1013137679 shutdown_connections 2024-03-27T10:57:08.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.508+0000 7fc47cc5c740 1 -- 172.21.15.97:0/1013137679 wait complete. 2024-03-27T10:57:08.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.509+0000 7fc47cc5c740 1 -- start start 2024-03-27T10:57:08.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.509+0000 7fc47cc5c740 1 Processor -- start 2024-03-27T10:57:08.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.509+0000 7fc47cc5c740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b882ab58c0 0x55b882abfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:08.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.509+0000 7fc47cc5c740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b882ab62b0 0x55b882abff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:08.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.509+0000 7fc47cc5c740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b882abb420 0x55b882ac6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:08.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.509+0000 7fc47cc5c740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55b8829d5ca0 con 0x55b882ab62b0 2024-03-27T10:57:08.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.509+0000 7fc47cc5c740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55b882a16620 con 0x55b882abb420 2024-03-27T10:57:08.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.509+0000 7fc47cc5c740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55b882abe370 con 0x55b882ab58c0 2024-03-27T10:57:08.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.509+0000 7fc47cc5c740 10 client.0 _ll_register_callbacks cb 0x55b882a12160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:57:08.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.510+0000 7fc472a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b882ab62b0 0x55b882abff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:08.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.510+0000 7fc472a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b882ab62b0 0x55b882abff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48534/0 (socket says 172.21.15.97:48534) 2024-03-27T10:57:08.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.510+0000 7fc472a00640 1 -- 172.21.15.97:0/170694666 learned_addr learned my addr 172.21.15.97:0/170694666 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:08.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.510+0000 7fc473400640 1 --2- 172.21.15.97:0/170694666 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b882ab58c0 0x55b882abfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:08.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.510+0000 7fc473e00640 1 --2- 172.21.15.97:0/170694666 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b882abb420 0x55b882ac6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:08.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.510+0000 7fc472a00640 1 -- 172.21.15.97:0/170694666 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b882ab58c0 msgr2=0x55b882abfa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:08.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.510+0000 7fc472a00640 1 --2- 172.21.15.97:0/170694666 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b882ab58c0 0x55b882abfa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.510+0000 7fc472a00640 1 -- 172.21.15.97:0/170694666 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b882abb420 msgr2=0x55b882ac6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:08.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.510+0000 7fc472a00640 1 --2- 172.21.15.97:0/170694666 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b882abb420 0x55b882ac6fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.510+0000 7fc472a00640 1 -- 172.21.15.97:0/170694666 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8827f87d0 con 0x55b882ab62b0 2024-03-27T10:57:08.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.510+0000 7fc472a00640 1 --2- 172.21.15.97:0/170694666 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b882ab62b0 0x55b882abff60 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fc4600cd310 tx=0x7fc46009b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb89e65ec6c20205 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:08.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.510+0000 7fc45cc00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T10:57:08.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.510+0000 7fc45cc00640 1 -- 172.21.15.97:0/170694666 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc460020070 con 0x55b882ab62b0 2024-03-27T10:57:08.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.510+0000 7fc45cc00640 1 -- 172.21.15.97:0/170694666 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fc460099fe0 con 0x55b882ab62b0 2024-03-27T10:57:08.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.510+0000 7fc47cc5c740 1 -- 172.21.15.97:0/170694666 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc46c032db0 con 0x55b882ab62b0 2024-03-27T10:57:08.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.511+0000 7fc45cc00640 1 -- 172.21.15.97:0/170694666 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc460009c30 con 0x55b882ab62b0 2024-03-27T10:57:08.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.511+0000 7fc47cc5c740 10 client.15603 fetch_fsmap learned FSMap version 28 2024-03-27T10:57:08.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.511+0000 7fc47cc5c740 1 -- 172.21.15.97:0/170694666 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b882aca430 con 0x55b882ab62b0 2024-03-27T10:57:08.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.511+0000 7fc45cc00640 1 -- 172.21.15.97:0/170694666 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc46c032db0 con 0x55b882ab62b0 2024-03-27T10:57:08.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.511+0000 7fc45cc00640 1 -- 172.21.15.97:0/170694666 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fc460098350 con 0x55b882ab62b0 2024-03-27T10:57:08.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.511+0000 7fc47cc5c740 10 client.15603 fetch_fsmap finished waiting for FSMap version 28 2024-03-27T10:57:08.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.511+0000 7fc47cc5c740 10 client.15603 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-03-27T10:57:08.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.511+0000 7fc47cc5c740 20 client.15603 populate_metadata read hostname 'smithi097' 2024-03-27T10:57:08.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.511+0000 7fc47cc5c740 1 --2- 172.21.15.97:0/170694666 >> [v2:172.21.15.177:6834/3349112132,v1:172.21.15.177:6835/3349112132] conn(0x55b882acbbf0 0x55b882ace070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:08.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.511+0000 7fc47cc5c740 4 client.15603 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T10:57:08.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.511+0000 7fc47cc5c740 1 -- 172.21.15.97:0/170694666 --> [v2:172.21.15.177:6834/3349112132,v1:172.21.15.177:6835/3349112132] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b882a27bb0 con 0x55b882acbbf0 2024-03-27T10:57:08.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.512+0000 7fc473400640 1 --2- 172.21.15.97:0/170694666 >> [v2:172.21.15.177:6834/3349112132,v1:172.21.15.177:6835/3349112132] conn(0x55b882acbbf0 0x55b882ace070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:08.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.513+0000 7fc473400640 1 --2- 172.21.15.97:0/170694666 >> [v2:172.21.15.177:6834/3349112132,v1:172.21.15.177:6835/3349112132] conn(0x55b882acbbf0 0x55b882ace070 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=af92311a8f66477 server_cookie=75733edcec7d039a in_seq=0 out_seq=0 2024-03-27T10:57:08.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.513+0000 7fc45cc00640 10 client.15603 ms_handle_connect on v2:172.21.15.177:6834/3349112132 2024-03-27T10:57:08.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.515+0000 7fc45cc00640 1 -- 172.21.15.97:0/170694666 <== mds.3 v2:172.21.15.177:6834/3349112132 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55b882a27bb0 con 0x55b882acbbf0 2024-03-27T10:57:08.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.515+0000 7fc45cc00640 10 client.15603 handle_command_reply: tid=0 2024-03-27T10:57:08.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.515+0000 7fc47cc5c740 1 client.15603 shutdown 2024-03-27T10:57:08.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.516+0000 7fc47cc5c740 1 -- 172.21.15.97:0/170694666 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b882ab62b0 msgr2=0x55b882abff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:08.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.516+0000 7fc47cc5c740 1 --2- 172.21.15.97:0/170694666 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b882ab62b0 0x55b882abff60 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fc4600cd310 tx=0x7fc46009b820 comp rx=0 tx=0).stop 2024-03-27T10:57:08.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.516+0000 7fc47cc5c740 1 -- 172.21.15.97:0/170694666 shutdown_connections 2024-03-27T10:57:08.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.516+0000 7fc47cc5c740 1 --2- 172.21.15.97:0/170694666 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b882abb420 0x55b882ac6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.516+0000 7fc47cc5c740 1 --2- 172.21.15.97:0/170694666 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b882ab62b0 0x55b882abff60 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.516+0000 7fc47cc5c740 1 --2- 172.21.15.97:0/170694666 >> [v2:172.21.15.177:6834/3349112132,v1:172.21.15.177:6835/3349112132] conn(0x55b882acbbf0 0x55b882ace070 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.516+0000 7fc47cc5c740 1 --2- 172.21.15.97:0/170694666 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b882ab58c0 0x55b882abfa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.516+0000 7fc47cc5c740 1 -- 172.21.15.97:0/170694666 >> 172.21.15.97:0/170694666 conn(0x55b882ab0e30 msgr2=0x55b882ab0030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:08.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.516+0000 7fc45cc00640 0 client.15603 ms_handle_reset on v2:172.21.15.177:6834/3349112132 2024-03-27T10:57:08.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.516+0000 7fc47cc5c740 1 -- 172.21.15.97:0/170694666 shutdown_connections 2024-03-27T10:57:08.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.516+0000 7fc47cc5c740 1 -- 172.21.15.97:0/170694666 wait complete. 2024-03-27T10:57:08.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.516+0000 7fc47cc5c740 20 client.15603 trim_cache size 0 max 16384 2024-03-27T10:57:08.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.530+0000 7fc47cc5c740 1 Processor -- start 2024-03-27T10:57:08.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.530+0000 7fc47cc5c740 1 -- start start 2024-03-27T10:57:08.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.531+0000 7fc47cc5c740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b882ab58c0 0x55b882aabdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:08.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.531+0000 7fc47cc5c740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b882ab62b0 0x55b882ac6f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:08.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.531+0000 7fc47cc5c740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b882abb420 0x55b882ab47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:08.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.531+0000 7fc47cc5c740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc4741ab0c0 con 0x55b882ab58c0 2024-03-27T10:57:08.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.531+0000 7fc47cc5c740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc4741acc00 con 0x55b882abb420 2024-03-27T10:57:08.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.531+0000 7fc47cc5c740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc4741ab790 con 0x55b882ab62b0 2024-03-27T10:57:08.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.531+0000 7fc473400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b882ab58c0 0x55b882aabdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:08.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.531+0000 7fc473400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b882ab58c0 0x55b882aabdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48548/0 (socket says 172.21.15.97:48548) 2024-03-27T10:57:08.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.531+0000 7fc473400640 1 -- 172.21.15.97:0/2790559842 learned_addr learned my addr 172.21.15.97:0/2790559842 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:08.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.531+0000 7fc473e00640 1 --2- 172.21.15.97:0/2790559842 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b882abb420 0x55b882ab47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:08.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.531+0000 7fc472a00640 1 --2- 172.21.15.97:0/2790559842 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b882ab62b0 0x55b882ac6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:08.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.531+0000 7fc473400640 1 -- 172.21.15.97:0/2790559842 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b882ab62b0 msgr2=0x55b882ac6f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:08.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.531+0000 7fc473400640 1 --2- 172.21.15.97:0/2790559842 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b882ab62b0 0x55b882ac6f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.531+0000 7fc473400640 1 -- 172.21.15.97:0/2790559842 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b882abb420 msgr2=0x55b882ab47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:08.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.531+0000 7fc473400640 1 --2- 172.21.15.97:0/2790559842 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b882abb420 0x55b882ab47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.531+0000 7fc473400640 1 -- 172.21.15.97:0/2790559842 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b882aca250 con 0x55b882ab58c0 2024-03-27T10:57:08.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.531+0000 7fc473e00640 1 --2- 172.21.15.97:0/2790559842 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b882abb420 0x55b882ab47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T10:57:08.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.532+0000 7fc473400640 1 --2- 172.21.15.97:0/2790559842 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b882ab58c0 0x55b882aabdd0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fc474104de0 tx=0x7fc464016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57f973fd9477a09d server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:08.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.532+0000 7fc45f400640 1 -- 172.21.15.97:0/2790559842 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc46402c070 con 0x55b882ab58c0 2024-03-27T10:57:08.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.532+0000 7fc45f400640 1 -- 172.21.15.97:0/2790559842 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc464010e40 con 0x55b882ab58c0 2024-03-27T10:57:08.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.532+0000 7fc45f400640 1 -- 172.21.15.97:0/2790559842 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc46401f680 con 0x55b882ab58c0 2024-03-27T10:57:08.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.532+0000 7fc47cc5c740 1 -- 172.21.15.97:0/2790559842 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b882ab58c0 msgr2=0x55b882aabdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:08.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.532+0000 7fc47cc5c740 1 --2- 172.21.15.97:0/2790559842 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b882ab58c0 0x55b882aabdd0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fc474104de0 tx=0x7fc464016000 comp rx=0 tx=0).stop 2024-03-27T10:57:08.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.533+0000 7fc47cc5c740 1 -- 172.21.15.97:0/2790559842 shutdown_connections 2024-03-27T10:57:08.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.533+0000 7fc47cc5c740 1 --2- 172.21.15.97:0/2790559842 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b882abb420 0x55b882ab47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.533+0000 7fc47cc5c740 1 --2- 172.21.15.97:0/2790559842 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b882ab62b0 0x55b882ac6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.533+0000 7fc47cc5c740 1 --2- 172.21.15.97:0/2790559842 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b882ab58c0 0x55b882aabdd0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.533+0000 7fc47cc5c740 1 -- 172.21.15.97:0/2790559842 >> 172.21.15.97:0/2790559842 conn(0x55b882ab0e30 msgr2=0x55b882ab4090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:08.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.533+0000 7fc47cc5c740 1 -- 172.21.15.97:0/2790559842 shutdown_connections 2024-03-27T10:57:08.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.533+0000 7fc47cc5c740 1 -- 172.21.15.97:0/2790559842 wait complete. 2024-03-27T10:57:08.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.533+0000 7fc47cc5c740 1 -- start start 2024-03-27T10:57:08.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.533+0000 7fc47cc5c740 1 Processor -- start 2024-03-27T10:57:08.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.534+0000 7fc47cc5c740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b882ab58c0 0x55b882ac4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:08.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.534+0000 7fc47cc5c740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b882ab62b0 0x55b882ac5360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:08.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.534+0000 7fc47cc5c740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b882abb420 0x55b882ac6370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:08.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.534+0000 7fc47cc5c740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc4741ac530 con 0x55b882ab62b0 2024-03-27T10:57:08.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.534+0000 7fc47cc5c740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc4741abe60 con 0x55b882abb420 2024-03-27T10:57:08.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.534+0000 7fc47cc5c740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc4741a90a0 con 0x55b882ab58c0 2024-03-27T10:57:08.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.534+0000 7fc47cc5c740 10 client.0 _ll_register_callbacks cb 0x7fc460009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:57:08.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.534+0000 7fc472a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b882ab62b0 0x55b882ac5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:08.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.534+0000 7fc472a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b882ab62b0 0x55b882ac5360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48550/0 (socket says 172.21.15.97:48550) 2024-03-27T10:57:08.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.534+0000 7fc472a00640 1 -- 172.21.15.97:0/2592597901 learned_addr learned my addr 172.21.15.97:0/2592597901 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:08.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.534+0000 7fc473400640 1 --2- 172.21.15.97:0/2592597901 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b882ab58c0 0x55b882ac4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:08.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.534+0000 7fc473e00640 1 --2- 172.21.15.97:0/2592597901 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b882abb420 0x55b882ac6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:08.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.534+0000 7fc472a00640 1 -- 172.21.15.97:0/2592597901 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b882ab58c0 msgr2=0x55b882ac4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:08.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.534+0000 7fc472a00640 1 --2- 172.21.15.97:0/2592597901 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b882ab58c0 0x55b882ac4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.534+0000 7fc472a00640 1 -- 172.21.15.97:0/2592597901 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b882abb420 msgr2=0x55b882ac6370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:08.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.534+0000 7fc472a00640 1 --2- 172.21.15.97:0/2592597901 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b882abb420 0x55b882ac6370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.534+0000 7fc472a00640 1 -- 172.21.15.97:0/2592597901 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b882ac6960 con 0x55b882ab62b0 2024-03-27T10:57:08.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.534+0000 7fc473400640 1 --2- 172.21.15.97:0/2592597901 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b882ab58c0 0x55b882ac4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:57:08.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.534+0000 7fc472a00640 1 --2- 172.21.15.97:0/2592597901 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b882ab62b0 0x55b882ac5360 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fc46009b9e0 tx=0x7fc4600a4950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=554e80bdd493cf06 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:08.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.534+0000 7fc45cc00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T10:57:08.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.535+0000 7fc45cc00640 1 -- 172.21.15.97:0/2592597901 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc460065060 con 0x55b882ab62b0 2024-03-27T10:57:08.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.535+0000 7fc45cc00640 1 -- 172.21.15.97:0/2592597901 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fc46009b330 con 0x55b882ab62b0 2024-03-27T10:57:08.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.535+0000 7fc47cc5c740 1 -- 172.21.15.97:0/2592597901 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc464010e40 con 0x55b882ab62b0 2024-03-27T10:57:08.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.535+0000 7fc45cc00640 1 -- 172.21.15.97:0/2592597901 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc460020070 con 0x55b882ab62b0 2024-03-27T10:57:08.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.535+0000 7fc47cc5c740 10 client.15609 fetch_fsmap learned FSMap version 28 2024-03-27T10:57:08.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.535+0000 7fc47cc5c740 1 -- 172.21.15.97:0/2592597901 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b882b327e0 con 0x55b882ab62b0 2024-03-27T10:57:08.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.535+0000 7fc45cc00640 1 -- 172.21.15.97:0/2592597901 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc464010e40 con 0x55b882ab62b0 2024-03-27T10:57:08.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.535+0000 7fc45cc00640 1 -- 172.21.15.97:0/2592597901 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fc4600a1220 con 0x55b882ab62b0 2024-03-27T10:57:08.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.535+0000 7fc47cc5c740 10 client.15609 fetch_fsmap finished waiting for FSMap version 28 2024-03-27T10:57:08.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.535+0000 7fc47cc5c740 10 client.15609 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-03-27T10:57:08.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.535+0000 7fc47cc5c740 20 client.15609 populate_metadata read hostname 'smithi097' 2024-03-27T10:57:08.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.535+0000 7fc47cc5c740 1 --2- 172.21.15.97:0/2592597901 >> [v2:172.21.15.177:6834/3349112132,v1:172.21.15.177:6835/3349112132] conn(0x55b882b33030 0x55b882b35450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:08.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.535+0000 7fc47cc5c740 4 client.15609 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-27T10:57:08.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.535+0000 7fc47cc5c740 1 -- 172.21.15.97:0/2592597901 --> [v2:172.21.15.177:6834/3349112132,v1:172.21.15.177:6835/3349112132] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b882a21f00 con 0x55b882b33030 2024-03-27T10:57:08.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.536+0000 7fc473400640 1 --2- 172.21.15.97:0/2592597901 >> [v2:172.21.15.177:6834/3349112132,v1:172.21.15.177:6835/3349112132] conn(0x55b882b33030 0x55b882b35450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:08.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.537+0000 7fc473400640 1 --2- 172.21.15.97:0/2592597901 >> [v2:172.21.15.177:6834/3349112132,v1:172.21.15.177:6835/3349112132] conn(0x55b882b33030 0x55b882b35450 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6a80d200b4b4ab22 server_cookie=1771edf913b58ead in_seq=0 out_seq=0 2024-03-27T10:57:08.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.537+0000 7fc45cc00640 10 client.15609 ms_handle_connect on v2:172.21.15.177:6834/3349112132 2024-03-27T10:57:08.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.537+0000 7fc45cc00640 1 -- 172.21.15.97:0/2592597901 <== mds.3 v2:172.21.15.177:6834/3349112132 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b882a21f00 con 0x55b882b33030 2024-03-27T10:57:08.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.537+0000 7fc45cc00640 10 client.15609 handle_command_reply: tid=0 2024-03-27T10:57:08.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.537+0000 7fc47cc5c740 1 client.15609 shutdown 2024-03-27T10:57:08.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.538+0000 7fc47cc5c740 1 -- 172.21.15.97:0/2592597901 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b882ab62b0 msgr2=0x55b882ac5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:08.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.538+0000 7fc47cc5c740 1 --2- 172.21.15.97:0/2592597901 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b882ab62b0 0x55b882ac5360 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fc46009b9e0 tx=0x7fc4600a4950 comp rx=0 tx=0).stop 2024-03-27T10:57:08.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.538+0000 7fc47cc5c740 1 -- 172.21.15.97:0/2592597901 shutdown_connections 2024-03-27T10:57:08.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.538+0000 7fc47cc5c740 1 --2- 172.21.15.97:0/2592597901 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b882abb420 0x55b882ac6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.538+0000 7fc47cc5c740 1 --2- 172.21.15.97:0/2592597901 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b882ab62b0 0x55b882ac5360 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.538+0000 7fc47cc5c740 1 --2- 172.21.15.97:0/2592597901 >> [v2:172.21.15.177:6834/3349112132,v1:172.21.15.177:6835/3349112132] conn(0x55b882b33030 0x55b882b35450 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.538+0000 7fc47cc5c740 1 --2- 172.21.15.97:0/2592597901 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b882ab58c0 0x55b882ac4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.538+0000 7fc47cc5c740 1 -- 172.21.15.97:0/2592597901 >> 172.21.15.97:0/2592597901 conn(0x55b882ab0e30 msgr2=0x55b882ab4920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:08.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.538+0000 7fc45cc00640 0 client.15609 ms_handle_reset on v2:172.21.15.177:6834/3349112132 2024-03-27T10:57:08.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.538+0000 7fc47cc5c740 1 -- 172.21.15.97:0/2592597901 shutdown_connections 2024-03-27T10:57:08.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.538+0000 7fc47cc5c740 1 -- 172.21.15.97:0/2592597901 wait complete. 2024-03-27T10:57:08.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.538+0000 7fc47cc5c740 20 client.15609 trim_cache size 0 max 16384 2024-03-27T10:57:08.514 INFO:teuthology.orchestra.run.smithi097.stdout:[] 2024-03-27T10:57:08.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.539+0000 7fc47a000640 1 -- 172.21.15.97:0/2556847143 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fc4580a3610 msgr2=0x7fc4580a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:08.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.539+0000 7fc47a000640 1 --2- 172.21.15.97:0/2556847143 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fc4580a3610 0x7fc4580a5ad0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc464005fd0 tx=0x7fc464005850 comp rx=0 tx=0).stop 2024-03-27T10:57:08.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.539+0000 7fc47a000640 1 -- 172.21.15.97:0/2556847143 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc474104f80 msgr2=0x7fc47419a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:08.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.539+0000 7fc47a000640 1 --2- 172.21.15.97:0/2556847143 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc474104f80 0x7fc47419a4b0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fc46000d770 tx=0x7fc46000dc30 comp rx=0 tx=0).stop 2024-03-27T10:57:08.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.539+0000 7fc47a000640 1 -- 172.21.15.97:0/2556847143 shutdown_connections 2024-03-27T10:57:08.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.539+0000 7fc47a000640 1 --2- 172.21.15.97:0/2556847143 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fc4580a3610 0x7fc4580a5ad0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.539+0000 7fc47a000640 1 --2- 172.21.15.97:0/2556847143 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc474105940 0x7fc4741a1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.539+0000 7fc47a000640 1 --2- 172.21.15.97:0/2556847143 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc474104f80 0x7fc47419a4b0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.539+0000 7fc47a000640 1 --2- 172.21.15.97:0/2556847143 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc474103d90 0x7fc474199f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.539+0000 7fc47a000640 1 -- 172.21.15.97:0/2556847143 >> 172.21.15.97:0/2556847143 conn(0x7fc4740ff300 msgr2=0x7fc4741083a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:08.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.539+0000 7fc47a000640 1 -- 172.21.15.97:0/2556847143 shutdown_connections 2024-03-27T10:57:08.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.539+0000 7fc47a000640 1 -- 172.21.15.97:0/2556847143 wait complete. 2024-03-27T10:57:08.516 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:57:08 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/1857115162' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T10:57:08.527 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-27T10:57:08.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:57:08 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/1857115162' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T10:57:08.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.756+0000 7fa225400640 1 -- 172.21.15.97:0/3941882588 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2200780f0 msgr2=0x7fa220078990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:08.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.757+0000 7fa225400640 1 --2- 172.21.15.97:0/3941882588 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2200780f0 0x7fa220078990 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fa218009ca0 tx=0x7fa21802c0f0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.757+0000 7fa225400640 1 -- 172.21.15.97:0/3941882588 shutdown_connections 2024-03-27T10:57:08.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.757+0000 7fa225400640 1 --2- 172.21.15.97:0/3941882588 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2200780f0 0x7fa220078990 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.757+0000 7fa225400640 1 --2- 172.21.15.97:0/3941882588 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa220077750 0x7fa220077bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.757+0000 7fa225400640 1 --2- 172.21.15.97:0/3941882588 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa220076510 0x7fa220076930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.757+0000 7fa225400640 1 -- 172.21.15.97:0/3941882588 >> 172.21.15.97:0/3941882588 conn(0x7fa2200ff300 msgr2=0x7fa220101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:08.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.757+0000 7fa225400640 1 -- 172.21.15.97:0/3941882588 shutdown_connections 2024-03-27T10:57:08.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.757+0000 7fa225400640 1 -- 172.21.15.97:0/3941882588 wait complete. 2024-03-27T10:57:08.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.758+0000 7fa225400640 1 Processor -- start 2024-03-27T10:57:08.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.758+0000 7fa225400640 1 -- start start 2024-03-27T10:57:08.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.758+0000 7fa225400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa220076510 0x7fa22019e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:08.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.758+0000 7fa225400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa220077750 0x7fa22019e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:08.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.758+0000 7fa225400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa2200780f0 0x7fa2201a58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:08.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.758+0000 7fa225400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa220113cd0 con 0x7fa220076510 2024-03-27T10:57:08.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.758+0000 7fa225400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa220113b50 con 0x7fa2200780f0 2024-03-27T10:57:08.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.758+0000 7fa225400640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa220113e50 con 0x7fa220077750 2024-03-27T10:57:08.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.758+0000 7fa21e800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa220076510 0x7fa22019e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:08.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.758+0000 7fa21e800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa220076510 0x7fa22019e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48568/0 (socket says 172.21.15.97:48568) 2024-03-27T10:57:08.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.758+0000 7fa21e800640 1 -- 172.21.15.97:0/2898574926 learned_addr learned my addr 172.21.15.97:0/2898574926 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:08.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.759+0000 7fa21f200640 1 --2- 172.21.15.97:0/2898574926 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa2200780f0 0x7fa2201a58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:08.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.759+0000 7fa215e00640 1 --2- 172.21.15.97:0/2898574926 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa220077750 0x7fa22019e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:08.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.759+0000 7fa21e800640 1 -- 172.21.15.97:0/2898574926 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa220077750 msgr2=0x7fa22019e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:08.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.759+0000 7fa21e800640 1 --2- 172.21.15.97:0/2898574926 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa220077750 0x7fa22019e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.759+0000 7fa21e800640 1 -- 172.21.15.97:0/2898574926 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa2200780f0 msgr2=0x7fa2201a58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:08.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.759+0000 7fa21e800640 1 --2- 172.21.15.97:0/2898574926 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa2200780f0 0x7fa2201a58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.759+0000 7fa21e800640 1 -- 172.21.15.97:0/2898574926 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2201a5f50 con 0x7fa220076510 2024-03-27T10:57:08.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.759+0000 7fa21e800640 1 --2- 172.21.15.97:0/2898574926 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa220076510 0x7fa22019e330 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fa21000e9f0 tx=0x7fa21000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:08.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.759+0000 7fa217e00640 1 -- 172.21.15.97:0/2898574926 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa21000cd80 con 0x7fa220076510 2024-03-27T10:57:08.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.759+0000 7fa225400640 1 -- 172.21.15.97:0/2898574926 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2201a6240 con 0x7fa220076510 2024-03-27T10:57:08.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.759+0000 7fa217e00640 1 -- 172.21.15.97:0/2898574926 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa2100108e0 con 0x7fa220076510 2024-03-27T10:57:08.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.759+0000 7fa217e00640 1 -- 172.21.15.97:0/2898574926 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa210018920 con 0x7fa220076510 2024-03-27T10:57:08.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.759+0000 7fa225400640 1 -- 172.21.15.97:0/2898574926 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2201a6790 con 0x7fa220076510 2024-03-27T10:57:08.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.762+0000 7fa217e00640 1 -- 172.21.15.97:0/2898574926 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7fa210010a90 con 0x7fa220076510 2024-03-27T10:57:08.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.762+0000 7fa2281f7740 1 Processor -- start 2024-03-27T10:57:08.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.763+0000 7fa2281f7740 1 -- start start 2024-03-27T10:57:08.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.763+0000 7fa2281f7740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5646872f28c0 0x5646872f2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:08.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.763+0000 7fa2281f7740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646872f32b0 0x5646872f7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:08.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.763+0000 7fa2281f7740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646872f8420 0x5646872fa8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:08.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.763+0000 7fa2281f7740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5646872ba7b0 con 0x5646872f32b0 2024-03-27T10:57:08.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.763+0000 7fa2281f7740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x564687257670 con 0x5646872f8420 2024-03-27T10:57:08.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.763+0000 7fa2281f7740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5646871ff970 con 0x5646872f28c0 2024-03-27T10:57:08.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.763+0000 7fa21e800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646872f32b0 0x5646872f7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:08.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.763+0000 7fa21e800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646872f32b0 0x5646872f7db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48574/0 (socket says 172.21.15.97:48574) 2024-03-27T10:57:08.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.763+0000 7fa21e800640 1 -- 172.21.15.97:0/3103902708 learned_addr learned my addr 172.21.15.97:0/3103902708 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:08.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.764+0000 7fa215e00640 1 --2- 172.21.15.97:0/3103902708 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646872f8420 0x5646872fa8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:08.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.764+0000 7fa21f200640 1 --2- 172.21.15.97:0/3103902708 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5646872f28c0 0x5646872f2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:08.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.764+0000 7fa21e800640 1 -- 172.21.15.97:0/3103902708 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5646872f28c0 msgr2=0x5646872f2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:08.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.764+0000 7fa21e800640 1 --2- 172.21.15.97:0/3103902708 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5646872f28c0 0x5646872f2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.764+0000 7fa21e800640 1 -- 172.21.15.97:0/3103902708 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646872f8420 msgr2=0x5646872fa8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:08.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.764+0000 7fa21e800640 1 --2- 172.21.15.97:0/3103902708 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646872f8420 0x5646872fa8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.764+0000 7fa21e800640 1 -- 172.21.15.97:0/3103902708 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564686f6b3f0 con 0x5646872f32b0 2024-03-27T10:57:08.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.764+0000 7fa217e00640 1 --2- 172.21.15.97:0/2898574926 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fa2040a3610 0x7fa2040a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:08.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.764+0000 7fa21e800640 1 --2- 172.21.15.97:0/3103902708 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646872f32b0 0x5646872f7db0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fa2100d82a0 tx=0x7fa2100d8760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b23bdfbf28d02e39 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:08.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.764+0000 7fa217e00640 1 -- 172.21.15.97:0/2898574926 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fa2100d00c0 con 0x7fa220076510 2024-03-27T10:57:08.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.764+0000 7fa21f200640 1 --2- 172.21.15.97:0/2898574926 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fa2040a3610 0x7fa2040a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:08.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.764+0000 7fa20f400640 1 -- 172.21.15.97:0/3103902708 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa210098840 con 0x5646872f32b0 2024-03-27T10:57:08.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.764+0000 7fa20f400640 1 -- 172.21.15.97:0/3103902708 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa2100b0e50 con 0x5646872f32b0 2024-03-27T10:57:08.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.765+0000 7fa20f400640 1 -- 172.21.15.97:0/3103902708 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa21009bf00 con 0x5646872f32b0 2024-03-27T10:57:08.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.765+0000 7fa2281f7740 1 -- 172.21.15.97:0/3103902708 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646872f32b0 msgr2=0x5646872f7db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:08.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.765+0000 7fa21f200640 1 --2- 172.21.15.97:0/2898574926 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fa2040a3610 0x7fa2040a5ad0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fa21802c630 tx=0x7fa21802e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:08.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.765+0000 7fa2281f7740 1 --2- 172.21.15.97:0/3103902708 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646872f32b0 0x5646872f7db0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fa2100d82a0 tx=0x7fa2100d8760 comp rx=0 tx=0).stop 2024-03-27T10:57:08.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.765+0000 7fa2281f7740 1 -- 172.21.15.97:0/3103902708 shutdown_connections 2024-03-27T10:57:08.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.765+0000 7fa2281f7740 1 --2- 172.21.15.97:0/3103902708 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646872f8420 0x5646872fa8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.765+0000 7fa2281f7740 1 --2- 172.21.15.97:0/3103902708 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646872f32b0 0x5646872f7db0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.765+0000 7fa2281f7740 1 --2- 172.21.15.97:0/3103902708 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5646872f28c0 0x5646872f2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.765+0000 7fa2281f7740 1 -- 172.21.15.97:0/3103902708 >> 172.21.15.97:0/3103902708 conn(0x5646872ede30 msgr2=0x5646872ee230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:08.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.765+0000 7fa2281f7740 1 -- 172.21.15.97:0/3103902708 shutdown_connections 2024-03-27T10:57:08.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.765+0000 7fa2281f7740 1 -- 172.21.15.97:0/3103902708 wait complete. 2024-03-27T10:57:08.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.766+0000 7fa2281f7740 1 -- start start 2024-03-27T10:57:08.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.766+0000 7fa2281f7740 1 Processor -- start 2024-03-27T10:57:08.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.766+0000 7fa2281f7740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646872f28c0 0x5646872fca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:08.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.766+0000 7fa2281f7740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5646872f32b0 0x5646872fcf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:08.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.766+0000 7fa2281f7740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646872f8420 0x564687303fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:08.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.766+0000 7fa2281f7740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x564687212ca0 con 0x5646872f8420 2024-03-27T10:57:08.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.766+0000 7fa2281f7740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x564687253620 con 0x5646872f28c0 2024-03-27T10:57:08.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.766+0000 7fa2281f7740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5646872fb370 con 0x5646872f32b0 2024-03-27T10:57:08.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.767+0000 7fa2281f7740 10 client.0 _ll_register_callbacks cb 0x56468724f160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:57:08.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.767+0000 7fa21f200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646872f8420 0x564687303fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:08.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.767+0000 7fa21f200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646872f8420 0x564687303fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48590/0 (socket says 172.21.15.97:48590) 2024-03-27T10:57:08.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.767+0000 7fa21f200640 1 -- 172.21.15.97:0/1016962734 learned_addr learned my addr 172.21.15.97:0/1016962734 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:08.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.767+0000 7fa215e00640 1 --2- 172.21.15.97:0/1016962734 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5646872f32b0 0x5646872fcf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:08.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.767+0000 7fa21e800640 1 --2- 172.21.15.97:0/1016962734 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646872f28c0 0x5646872fca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:08.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.767+0000 7fa21f200640 1 -- 172.21.15.97:0/1016962734 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5646872f32b0 msgr2=0x5646872fcf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:08.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.767+0000 7fa21f200640 1 --2- 172.21.15.97:0/1016962734 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5646872f32b0 0x5646872fcf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.767+0000 7fa21f200640 1 -- 172.21.15.97:0/1016962734 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646872f28c0 msgr2=0x5646872fca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:08.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.767+0000 7fa21f200640 1 --2- 172.21.15.97:0/1016962734 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646872f28c0 0x5646872fca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.767+0000 7fa21f200640 1 -- 172.21.15.97:0/1016962734 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646870357d0 con 0x5646872f8420 2024-03-27T10:57:08.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.767+0000 7fa21e800640 1 --2- 172.21.15.97:0/1016962734 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646872f28c0 0x5646872fca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T10:57:08.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.767+0000 7fa21f200640 1 --2- 172.21.15.97:0/1016962734 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646872f8420 0x564687303fd0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fa218032550 tx=0x7fa21802c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f53df15d166e2993 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:08.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.767+0000 7fa203e00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T10:57:08.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.768+0000 7fa203e00640 1 -- 172.21.15.97:0/1016962734 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa218039040 con 0x5646872f8420 2024-03-27T10:57:08.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.768+0000 7fa203e00640 1 -- 172.21.15.97:0/1016962734 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fa21804c660 con 0x5646872f8420 2024-03-27T10:57:08.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.768+0000 7fa2281f7740 1 -- 172.21.15.97:0/1016962734 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2100b0e50 con 0x5646872f8420 2024-03-27T10:57:08.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.768+0000 7fa203e00640 1 -- 172.21.15.97:0/1016962734 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa21804c860 con 0x5646872f8420 2024-03-27T10:57:08.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.768+0000 7fa2281f7740 10 client.15621 fetch_fsmap learned FSMap version 28 2024-03-27T10:57:08.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.768+0000 7fa2281f7740 1 -- 172.21.15.97:0/1016962734 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5646873073b0 con 0x5646872f8420 2024-03-27T10:57:08.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.768+0000 7fa203e00640 1 -- 172.21.15.97:0/1016962734 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa2100b0e50 con 0x5646872f8420 2024-03-27T10:57:08.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.768+0000 7fa203e00640 1 -- 172.21.15.97:0/1016962734 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fa218058c70 con 0x5646872f8420 2024-03-27T10:57:08.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.769+0000 7fa2281f7740 10 client.15621 fetch_fsmap finished waiting for FSMap version 28 2024-03-27T10:57:08.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.769+0000 7fa2281f7740 10 client.15621 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-27T10:57:08.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.769+0000 7fa2281f7740 20 client.15621 populate_metadata read hostname 'smithi097' 2024-03-27T10:57:08.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.769+0000 7fa2281f7740 1 --2- 172.21.15.97:0/1016962734 >> [v2:172.21.15.114:6834/3581918222,v1:172.21.15.114:6835/3581918222] conn(0x564687308b40 0x56468730afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:08.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.769+0000 7fa2281f7740 4 client.15621 mds_command: new command op to 34445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T10:57:08.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.769+0000 7fa2281f7740 1 -- 172.21.15.97:0/1016962734 --> [v2:172.21.15.114:6834/3581918222,v1:172.21.15.114:6835/3581918222] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564687264bb0 con 0x564687308b40 2024-03-27T10:57:08.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.769+0000 7fa21e800640 1 --2- 172.21.15.97:0/1016962734 >> [v2:172.21.15.114:6834/3581918222,v1:172.21.15.114:6835/3581918222] conn(0x564687308b40 0x56468730afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:08.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.770+0000 7fa21e800640 1 --2- 172.21.15.97:0/1016962734 >> [v2:172.21.15.114:6834/3581918222,v1:172.21.15.114:6835/3581918222] conn(0x564687308b40 0x56468730afe0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ea295a5f5936c06f server_cookie=d40b646b1fb131ec in_seq=0 out_seq=0 2024-03-27T10:57:08.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.770+0000 7fa203e00640 10 client.15621 ms_handle_connect on v2:172.21.15.114:6834/3581918222 2024-03-27T10:57:08.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.771+0000 7fa203e00640 1 -- 172.21.15.97:0/1016962734 <== mds.2 v2:172.21.15.114:6834/3581918222 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x564687264bb0 con 0x564687308b40 2024-03-27T10:57:08.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.771+0000 7fa203e00640 10 client.15621 handle_command_reply: tid=0 2024-03-27T10:57:08.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.771+0000 7fa2281f7740 1 client.15621 shutdown 2024-03-27T10:57:08.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.771+0000 7fa2281f7740 1 -- 172.21.15.97:0/1016962734 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646872f8420 msgr2=0x564687303fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:08.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.771+0000 7fa2281f7740 1 --2- 172.21.15.97:0/1016962734 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646872f8420 0x564687303fd0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fa218032550 tx=0x7fa21802c0f0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.771+0000 7fa2281f7740 1 -- 172.21.15.97:0/1016962734 shutdown_connections 2024-03-27T10:57:08.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.771+0000 7fa2281f7740 1 --2- 172.21.15.97:0/1016962734 >> [v2:172.21.15.114:6834/3581918222,v1:172.21.15.114:6835/3581918222] conn(0x564687308b40 0x56468730afe0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.771+0000 7fa2281f7740 1 --2- 172.21.15.97:0/1016962734 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646872f8420 0x564687303fd0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.771+0000 7fa2281f7740 1 --2- 172.21.15.97:0/1016962734 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5646872f32b0 0x5646872fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.771+0000 7fa203e00640 0 client.15621 ms_handle_reset on v2:172.21.15.114:6834/3581918222 2024-03-27T10:57:08.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.771+0000 7fa2281f7740 1 --2- 172.21.15.97:0/1016962734 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646872f28c0 0x5646872fca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.771+0000 7fa2281f7740 1 -- 172.21.15.97:0/1016962734 >> 172.21.15.97:0/1016962734 conn(0x5646872ede30 msgr2=0x5646872ed030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:08.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.771+0000 7fa2281f7740 1 -- 172.21.15.97:0/1016962734 shutdown_connections 2024-03-27T10:57:08.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.772+0000 7fa2281f7740 1 -- 172.21.15.97:0/1016962734 wait complete. 2024-03-27T10:57:08.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.772+0000 7fa2281f7740 20 client.15621 trim_cache size 0 max 16384 2024-03-27T10:57:08.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.785+0000 7fa2281f7740 1 Processor -- start 2024-03-27T10:57:08.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.786+0000 7fa2281f7740 1 -- start start 2024-03-27T10:57:08.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.786+0000 7fa2281f7740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5646872f28c0 0x5646872f17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:08.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.786+0000 7fa2281f7740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646872f32b0 0x564687303f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:08.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.786+0000 7fa2281f7740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646872f8420 0x56468730f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:08.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.786+0000 7fa2281f7740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa2201af500 con 0x5646872f32b0 2024-03-27T10:57:08.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.786+0000 7fa2281f7740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa2201b1040 con 0x5646872f8420 2024-03-27T10:57:08.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.786+0000 7fa2281f7740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa2201afbd0 con 0x5646872f28c0 2024-03-27T10:57:08.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.786+0000 7fa215e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646872f32b0 0x564687303f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:08.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.786+0000 7fa215e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646872f32b0 0x564687303f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48598/0 (socket says 172.21.15.97:48598) 2024-03-27T10:57:08.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.786+0000 7fa215e00640 1 -- 172.21.15.97:0/1564138297 learned_addr learned my addr 172.21.15.97:0/1564138297 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:08.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.786+0000 7fa21f200640 1 --2- 172.21.15.97:0/1564138297 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646872f8420 0x56468730f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:08.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.786+0000 7fa21e800640 1 --2- 172.21.15.97:0/1564138297 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5646872f28c0 0x5646872f17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:08.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.787+0000 7fa215e00640 1 -- 172.21.15.97:0/1564138297 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5646872f28c0 msgr2=0x5646872f17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:08.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.787+0000 7fa215e00640 1 --2- 172.21.15.97:0/1564138297 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5646872f28c0 0x5646872f17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.787+0000 7fa215e00640 1 -- 172.21.15.97:0/1564138297 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646872f8420 msgr2=0x56468730f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:08.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.787+0000 7fa215e00640 1 --2- 172.21.15.97:0/1564138297 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646872f8420 0x56468730f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.787+0000 7fa215e00640 1 -- 172.21.15.97:0/1564138297 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646872f3aa0 con 0x5646872f32b0 2024-03-27T10:57:08.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.787+0000 7fa21e800640 1 --2- 172.21.15.97:0/1564138297 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5646872f28c0 0x5646872f17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T10:57:08.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.787+0000 7fa215e00640 1 --2- 172.21.15.97:0/1564138297 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646872f32b0 0x564687303f90 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fa208009b00 tx=0x7fa2080123f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4e82c1599af8c10 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:08.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.787+0000 7fa20ea00640 1 -- 172.21.15.97:0/1564138297 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa208004220 con 0x5646872f32b0 2024-03-27T10:57:08.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.787+0000 7fa20ea00640 1 -- 172.21.15.97:0/1564138297 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa2080043c0 con 0x5646872f32b0 2024-03-27T10:57:08.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.787+0000 7fa20ea00640 1 -- 172.21.15.97:0/1564138297 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa208003850 con 0x5646872f32b0 2024-03-27T10:57:08.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.788+0000 7fa2281f7740 1 -- 172.21.15.97:0/1564138297 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646872f32b0 msgr2=0x564687303f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:08.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.788+0000 7fa2281f7740 1 --2- 172.21.15.97:0/1564138297 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646872f32b0 0x564687303f90 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fa208009b00 tx=0x7fa2080123f0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.788+0000 7fa2281f7740 1 -- 172.21.15.97:0/1564138297 shutdown_connections 2024-03-27T10:57:08.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.788+0000 7fa2281f7740 1 --2- 172.21.15.97:0/1564138297 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646872f8420 0x56468730f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.788+0000 7fa2281f7740 1 --2- 172.21.15.97:0/1564138297 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646872f32b0 0x564687303f90 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.788+0000 7fa2281f7740 1 --2- 172.21.15.97:0/1564138297 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5646872f28c0 0x5646872f17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.788+0000 7fa2281f7740 1 -- 172.21.15.97:0/1564138297 >> 172.21.15.97:0/1564138297 conn(0x5646872ede30 msgr2=0x5646872f1090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:08.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.788+0000 7fa2281f7740 1 -- 172.21.15.97:0/1564138297 shutdown_connections 2024-03-27T10:57:08.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.788+0000 7fa2281f7740 1 -- 172.21.15.97:0/1564138297 wait complete. 2024-03-27T10:57:08.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.788+0000 7fa2281f7740 1 -- start start 2024-03-27T10:57:08.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.789+0000 7fa2281f7740 1 Processor -- start 2024-03-27T10:57:08.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.789+0000 7fa2281f7740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5646872f28c0 0x564687301a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:08.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.789+0000 7fa2281f7740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646872f32b0 0x564687301f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:08.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.789+0000 7fa2281f7740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646872f8420 0x564687302f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:08.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.789+0000 7fa2281f7740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa2201b0970 con 0x5646872f32b0 2024-03-27T10:57:08.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.789+0000 7fa2281f7740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa2201b02a0 con 0x5646872f8420 2024-03-27T10:57:08.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.789+0000 7fa2281f7740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa2201ad4e0 con 0x5646872f28c0 2024-03-27T10:57:08.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.789+0000 7fa2281f7740 10 client.0 _ll_register_callbacks cb 0x7fa210005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:57:08.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.789+0000 7fa215e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646872f32b0 0x564687301f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:08.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.789+0000 7fa215e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646872f32b0 0x564687301f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48606/0 (socket says 172.21.15.97:48606) 2024-03-27T10:57:08.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.789+0000 7fa215e00640 1 -- 172.21.15.97:0/1230590425 learned_addr learned my addr 172.21.15.97:0/1230590425 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:08.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.789+0000 7fa21f200640 1 --2- 172.21.15.97:0/1230590425 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646872f8420 0x564687302f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:08.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.789+0000 7fa21e800640 1 --2- 172.21.15.97:0/1230590425 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5646872f28c0 0x564687301a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:08.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.790+0000 7fa215e00640 1 -- 172.21.15.97:0/1230590425 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5646872f28c0 msgr2=0x564687301a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:08.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.790+0000 7fa215e00640 1 --2- 172.21.15.97:0/1230590425 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5646872f28c0 0x564687301a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.790+0000 7fa215e00640 1 -- 172.21.15.97:0/1230590425 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646872f8420 msgr2=0x564687302f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:08.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.790+0000 7fa215e00640 1 --2- 172.21.15.97:0/1230590425 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646872f8420 0x564687302f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.790+0000 7fa215e00640 1 -- 172.21.15.97:0/1230590425 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564687303540 con 0x5646872f32b0 2024-03-27T10:57:08.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.790+0000 7fa21f200640 1 --2- 172.21.15.97:0/1230590425 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646872f8420 0x564687302f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T10:57:08.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.790+0000 7fa215e00640 1 --2- 172.21.15.97:0/1230590425 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646872f32b0 0x564687301f40 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fa2080128f0 tx=0x7fa208004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dbb5ad137a33e46 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:08.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.790+0000 7fa203e00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T10:57:08.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.790+0000 7fa203e00640 1 -- 172.21.15.97:0/1230590425 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa208028020 con 0x5646872f32b0 2024-03-27T10:57:08.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.790+0000 7fa203e00640 1 -- 172.21.15.97:0/1230590425 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fa208005010 con 0x5646872f32b0 2024-03-27T10:57:08.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.790+0000 7fa2281f7740 1 -- 172.21.15.97:0/1230590425 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2080043c0 con 0x5646872f32b0 2024-03-27T10:57:08.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.790+0000 7fa203e00640 1 -- 172.21.15.97:0/1230590425 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa208005210 con 0x5646872f32b0 2024-03-27T10:57:08.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.790+0000 7fa2281f7740 10 client.15627 fetch_fsmap learned FSMap version 28 2024-03-27T10:57:08.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.790+0000 7fa2281f7740 1 -- 172.21.15.97:0/1230590425 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56468736f190 con 0x5646872f32b0 2024-03-27T10:57:08.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.791+0000 7fa203e00640 1 -- 172.21.15.97:0/1230590425 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa2080043c0 con 0x5646872f32b0 2024-03-27T10:57:08.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.791+0000 7fa203e00640 1 -- 172.21.15.97:0/1230590425 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fa208019070 con 0x5646872f32b0 2024-03-27T10:57:08.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.791+0000 7fa2281f7740 10 client.15627 fetch_fsmap finished waiting for FSMap version 28 2024-03-27T10:57:08.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.791+0000 7fa2281f7740 10 client.15627 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-27T10:57:08.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.791+0000 7fa2281f7740 20 client.15627 populate_metadata read hostname 'smithi097' 2024-03-27T10:57:08.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.791+0000 7fa2281f7740 1 --2- 172.21.15.97:0/1230590425 >> [v2:172.21.15.114:6834/3581918222,v1:172.21.15.114:6835/3581918222] conn(0x56468736fb20 0x564687371f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:08.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.791+0000 7fa2281f7740 4 client.15627 mds_command: new command op to 34445 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-27T10:57:08.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.791+0000 7fa2281f7740 1 -- 172.21.15.97:0/1230590425 --> [v2:172.21.15.114:6834/3581918222,v1:172.21.15.114:6835/3581918222] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56468725ef00 con 0x56468736fb20 2024-03-27T10:57:08.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.791+0000 7fa21e800640 1 --2- 172.21.15.97:0/1230590425 >> [v2:172.21.15.114:6834/3581918222,v1:172.21.15.114:6835/3581918222] conn(0x56468736fb20 0x564687371f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:08.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.792+0000 7fa21e800640 1 --2- 172.21.15.97:0/1230590425 >> [v2:172.21.15.114:6834/3581918222,v1:172.21.15.114:6835/3581918222] conn(0x56468736fb20 0x564687371f40 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=74eb82ba4ffaf996 server_cookie=116c34e4e858ccb8 in_seq=0 out_seq=0 2024-03-27T10:57:08.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.792+0000 7fa203e00640 10 client.15627 ms_handle_connect on v2:172.21.15.114:6834/3581918222 2024-03-27T10:57:08.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.792+0000 7fa203e00640 1 -- 172.21.15.97:0/1230590425 <== mds.2 v2:172.21.15.114:6834/3581918222 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56468725ef00 con 0x56468736fb20 2024-03-27T10:57:08.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.792+0000 7fa203e00640 10 client.15627 handle_command_reply: tid=0 2024-03-27T10:57:08.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.792+0000 7fa2281f7740 1 client.15627 shutdown 2024-03-27T10:57:08.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.792+0000 7fa2281f7740 1 -- 172.21.15.97:0/1230590425 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646872f32b0 msgr2=0x564687301f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:08.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.792+0000 7fa2281f7740 1 --2- 172.21.15.97:0/1230590425 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646872f32b0 0x564687301f40 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fa2080128f0 tx=0x7fa208004600 comp rx=0 tx=0).stop 2024-03-27T10:57:08.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.792+0000 7fa2281f7740 1 -- 172.21.15.97:0/1230590425 shutdown_connections 2024-03-27T10:57:08.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.792+0000 7fa2281f7740 1 --2- 172.21.15.97:0/1230590425 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646872f8420 0x564687302f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.792+0000 7fa2281f7740 1 --2- 172.21.15.97:0/1230590425 >> [v2:172.21.15.114:6834/3581918222,v1:172.21.15.114:6835/3581918222] conn(0x56468736fb20 0x564687371f40 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.792+0000 7fa2281f7740 1 --2- 172.21.15.97:0/1230590425 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646872f32b0 0x564687301f40 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.792+0000 7fa2281f7740 1 --2- 172.21.15.97:0/1230590425 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5646872f28c0 0x564687301a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.792+0000 7fa203e00640 0 client.15627 ms_handle_reset on v2:172.21.15.114:6834/3581918222 2024-03-27T10:57:08.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.792+0000 7fa2281f7740 1 -- 172.21.15.97:0/1230590425 >> 172.21.15.97:0/1230590425 conn(0x5646872ede30 msgr2=0x5646872f1920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:08.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.792+0000 7fa2281f7740 1 -- 172.21.15.97:0/1230590425 shutdown_connections 2024-03-27T10:57:08.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.793+0000 7fa2281f7740 1 -- 172.21.15.97:0/1230590425 wait complete. 2024-03-27T10:57:08.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.793+0000 7fa2281f7740 20 client.15627 trim_cache size 0 max 16384 2024-03-27T10:57:08.769 INFO:teuthology.orchestra.run.smithi097.stdout:[] 2024-03-27T10:57:08.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.793+0000 7fa225400640 1 -- 172.21.15.97:0/2898574926 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fa2040a3610 msgr2=0x7fa2040a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:08.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.793+0000 7fa225400640 1 --2- 172.21.15.97:0/2898574926 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fa2040a3610 0x7fa2040a5ad0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fa21802c630 tx=0x7fa21802e040 comp rx=0 tx=0).stop 2024-03-27T10:57:08.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.793+0000 7fa225400640 1 -- 172.21.15.97:0/2898574926 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa220076510 msgr2=0x7fa22019e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:08.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.793+0000 7fa225400640 1 --2- 172.21.15.97:0/2898574926 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa220076510 0x7fa22019e330 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fa21000e9f0 tx=0x7fa21000eeb0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.793+0000 7fa225400640 1 -- 172.21.15.97:0/2898574926 shutdown_connections 2024-03-27T10:57:08.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.793+0000 7fa225400640 1 --2- 172.21.15.97:0/2898574926 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fa2040a3610 0x7fa2040a5ad0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.793+0000 7fa225400640 1 --2- 172.21.15.97:0/2898574926 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa2200780f0 0x7fa2201a58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.793+0000 7fa225400640 1 --2- 172.21.15.97:0/2898574926 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa220077750 0x7fa22019e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.793+0000 7fa225400640 1 --2- 172.21.15.97:0/2898574926 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa220076510 0x7fa22019e330 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.793+0000 7fa225400640 1 -- 172.21.15.97:0/2898574926 >> 172.21.15.97:0/2898574926 conn(0x7fa2200ff300 msgr2=0x7fa2201009f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:08.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.793+0000 7fa225400640 1 -- 172.21.15.97:0/2898574926 shutdown_connections 2024-03-27T10:57:08.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:08.794+0000 7fa225400640 1 -- 172.21.15.97:0/2898574926 wait complete. 2024-03-27T10:57:08.781 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-27T10:57:08.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.002+0000 7fe07da00640 1 -- 172.21.15.97:0/2491176380 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe078109260 msgr2=0x7fe0781096e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:08.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.002+0000 7fe07da00640 1 --2- 172.21.15.97:0/2491176380 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe078109260 0x7fe0781096e0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fe060009900 tx=0x7fe06002bc30 comp rx=0 tx=0).stop 2024-03-27T10:57:08.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.002+0000 7fe07da00640 1 -- 172.21.15.97:0/2491176380 shutdown_connections 2024-03-27T10:57:08.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.002+0000 7fe07da00640 1 --2- 172.21.15.97:0/2491176380 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe078109c20 0x7fe0781104b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.002+0000 7fe07da00640 1 --2- 172.21.15.97:0/2491176380 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe078109260 0x7fe0781096e0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.002+0000 7fe07da00640 1 --2- 172.21.15.97:0/2491176380 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe078108070 0x7fe078108490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.002+0000 7fe07da00640 1 -- 172.21.15.97:0/2491176380 >> 172.21.15.97:0/2491176380 conn(0x7fe078077020 msgr2=0x7fe078079440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:08.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.002+0000 7fe07da00640 1 -- 172.21.15.97:0/2491176380 shutdown_connections 2024-03-27T10:57:08.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.002+0000 7fe07da00640 1 -- 172.21.15.97:0/2491176380 wait complete. 2024-03-27T10:57:08.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.003+0000 7fe07da00640 1 Processor -- start 2024-03-27T10:57:08.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.003+0000 7fe07da00640 1 -- start start 2024-03-27T10:57:08.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.004+0000 7fe07da00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe078108070 0x7fe07819e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:08.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.004+0000 7fe07da00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe078109260 0x7fe07819e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:08.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.004+0000 7fe07da00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe078109c20 0x7fe0781a5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:08.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.004+0000 7fe07da00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe078113c00 con 0x7fe078109c20 2024-03-27T10:57:08.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.004+0000 7fe07da00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe078113a80 con 0x7fe078109260 2024-03-27T10:57:08.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.004+0000 7fe07da00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe078113d80 con 0x7fe078108070 2024-03-27T10:57:08.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.004+0000 7fe07d000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe078109c20 0x7fe0781a5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:08.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.004+0000 7fe07d000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe078109c20 0x7fe0781a5840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34640/0 (socket says 172.21.15.97:34640) 2024-03-27T10:57:08.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.004+0000 7fe07d000640 1 -- 172.21.15.97:0/3599358732 learned_addr learned my addr 172.21.15.97:0/3599358732 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:08.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.004+0000 7fe077e00640 1 --2- 172.21.15.97:0/3599358732 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe078108070 0x7fe07819e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:08.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.004+0000 7fe06f400640 1 --2- 172.21.15.97:0/3599358732 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe078109260 0x7fe07819e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:08.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.004+0000 7fe07d000640 1 -- 172.21.15.97:0/3599358732 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe078108070 msgr2=0x7fe07819e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:08.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.004+0000 7fe07d000640 1 --2- 172.21.15.97:0/3599358732 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe078108070 0x7fe07819e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.004+0000 7fe07d000640 1 -- 172.21.15.97:0/3599358732 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe078109260 msgr2=0x7fe07819e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:08.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.004+0000 7fe07d000640 1 --2- 172.21.15.97:0/3599358732 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe078109260 0x7fe07819e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.004+0000 7fe07d000640 1 -- 172.21.15.97:0/3599358732 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0781a5eb0 con 0x7fe078109c20 2024-03-27T10:57:08.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.005+0000 7fe07d000640 1 --2- 172.21.15.97:0/3599358732 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe078109c20 0x7fe0781a5840 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fe07000eb10 tx=0x7fe07000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:08.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.005+0000 7fe075600640 1 -- 172.21.15.97:0/3599358732 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe070004b20 con 0x7fe078109c20 2024-03-27T10:57:08.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.005+0000 7fe075600640 1 -- 172.21.15.97:0/3599358732 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe070004cc0 con 0x7fe078109c20 2024-03-27T10:57:08.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.005+0000 7fe075600640 1 -- 172.21.15.97:0/3599358732 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe070010690 con 0x7fe078109c20 2024-03-27T10:57:08.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.005+0000 7fe07da00640 1 -- 172.21.15.97:0/3599358732 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0781a61a0 con 0x7fe078109c20 2024-03-27T10:57:08.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.005+0000 7fe07da00640 1 -- 172.21.15.97:0/3599358732 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0781a6670 con 0x7fe078109c20 2024-03-27T10:57:08.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.008+0000 7fe075600640 1 -- 172.21.15.97:0/3599358732 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7fe070005020 con 0x7fe078109c20 2024-03-27T10:57:08.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.008+0000 7fe081e26740 1 Processor -- start 2024-03-27T10:57:08.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.008+0000 7fe081e26740 1 -- start start 2024-03-27T10:57:08.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.009+0000 7fe081e26740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e24dfcdab0 0x55e24dfcded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:08.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.009+0000 7fe081e26740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e24dfce4a0 0x55e24dfd2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:08.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.009+0000 7fe081e26740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e24dfd3610 0x55e24dfd5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:08.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.009+0000 7fe081e26740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55e24df957b0 con 0x55e24dfce4a0 2024-03-27T10:57:08.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.009+0000 7fe081e26740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55e24df32670 con 0x55e24dfcdab0 2024-03-27T10:57:08.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.009+0000 7fe081e26740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55e24deda970 con 0x55e24dfd3610 2024-03-27T10:57:08.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.009+0000 7fe077e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e24dfce4a0 0x55e24dfd2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:08.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.009+0000 7fe077e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e24dfce4a0 0x55e24dfd2fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34646/0 (socket says 172.21.15.97:34646) 2024-03-27T10:57:08.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.009+0000 7fe077e00640 1 -- 172.21.15.97:0/153386991 learned_addr learned my addr 172.21.15.97:0/153386991 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:08.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.009+0000 7fe06f400640 1 --2- 172.21.15.97:0/153386991 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e24dfcdab0 0x55e24dfcded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:08.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.009+0000 7fe077e00640 1 -- 172.21.15.97:0/153386991 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e24dfd3610 msgr2=0x55e24dfd5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:08.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.009+0000 7fe077e00640 1 --2- 172.21.15.97:0/153386991 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e24dfd3610 0x55e24dfd5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.009+0000 7fe077e00640 1 -- 172.21.15.97:0/153386991 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e24dfcdab0 msgr2=0x55e24dfcded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:08.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.009+0000 7fe077e00640 1 --2- 172.21.15.97:0/153386991 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e24dfcdab0 0x55e24dfcded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.009+0000 7fe077e00640 1 -- 172.21.15.97:0/153386991 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e24dc463f0 con 0x55e24dfce4a0 2024-03-27T10:57:08.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.010+0000 7fe077e00640 1 --2- 172.21.15.97:0/153386991 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e24dfce4a0 0x55e24dfd2fa0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fe06800bc80 tx=0x7fe06800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9c05cef012cebfe server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:08.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.010+0000 7fe075600640 1 --2- 172.21.15.97:0/3599358732 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fe05c0a3690 0x7fe05c0a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:08.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.010+0000 7fe06d600640 1 -- 172.21.15.97:0/153386991 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe068015070 con 0x55e24dfce4a0 2024-03-27T10:57:08.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.010+0000 7fe06d600640 1 -- 172.21.15.97:0/153386991 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe06801a910 con 0x55e24dfce4a0 2024-03-27T10:57:08.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.010+0000 7fe06d600640 1 -- 172.21.15.97:0/153386991 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe068011860 con 0x55e24dfce4a0 2024-03-27T10:57:08.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.010+0000 7fe075600640 1 -- 172.21.15.97:0/3599358732 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fe0700d03c0 con 0x7fe078109c20 2024-03-27T10:57:08.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.010+0000 7fe06f400640 1 --2- 172.21.15.97:0/3599358732 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fe05c0a3690 0x7fe05c0a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:08.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.011+0000 7fe06f400640 1 --2- 172.21.15.97:0/3599358732 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fe05c0a3690 0x7fe05c0a5b50 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fe060006010 tx=0x7fe060005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:08.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.011+0000 7fe081e26740 1 -- 172.21.15.97:0/153386991 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e24dfce4a0 msgr2=0x55e24dfd2fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:08.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.011+0000 7fe081e26740 1 --2- 172.21.15.97:0/153386991 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e24dfce4a0 0x55e24dfd2fa0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fe06800bc80 tx=0x7fe06800f730 comp rx=0 tx=0).stop 2024-03-27T10:57:08.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.011+0000 7fe081e26740 1 -- 172.21.15.97:0/153386991 shutdown_connections 2024-03-27T10:57:08.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.011+0000 7fe081e26740 1 --2- 172.21.15.97:0/153386991 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e24dfd3610 0x55e24dfd5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.011+0000 7fe081e26740 1 --2- 172.21.15.97:0/153386991 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e24dfce4a0 0x55e24dfd2fa0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.011+0000 7fe081e26740 1 --2- 172.21.15.97:0/153386991 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e24dfcdab0 0x55e24dfcded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.011+0000 7fe081e26740 1 -- 172.21.15.97:0/153386991 >> 172.21.15.97:0/153386991 conn(0x55e24dfc9020 msgr2=0x55e24dfc9420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:08.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.011+0000 7fe081e26740 1 -- 172.21.15.97:0/153386991 shutdown_connections 2024-03-27T10:57:08.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.011+0000 7fe081e26740 1 -- 172.21.15.97:0/153386991 wait complete. 2024-03-27T10:57:08.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.011+0000 7fe081e26740 1 -- start start 2024-03-27T10:57:08.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.012+0000 7fe081e26740 1 Processor -- start 2024-03-27T10:57:08.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.012+0000 7fe081e26740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e24dfcdab0 0x55e24dfd7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:08.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.012+0000 7fe081e26740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e24dfce4a0 0x55e24dfd8110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:08.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.012+0000 7fe081e26740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e24dfd3610 0x55e24dfdf150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:08.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.012+0000 7fe081e26740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55e24deedca0 con 0x55e24dfd3610 2024-03-27T10:57:08.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.012+0000 7fe081e26740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55e24df2e620 con 0x55e24dfcdab0 2024-03-27T10:57:08.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.012+0000 7fe081e26740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55e24dfd65c0 con 0x55e24dfce4a0 2024-03-27T10:57:08.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.012+0000 7fe081e26740 10 client.0 _ll_register_callbacks cb 0x55e24df2a160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:57:08.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.012+0000 7fe07d000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e24dfd3610 0x55e24dfdf150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:08.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.012+0000 7fe07d000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e24dfd3610 0x55e24dfdf150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34662/0 (socket says 172.21.15.97:34662) 2024-03-27T10:57:08.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.012+0000 7fe07d000640 1 -- 172.21.15.97:0/2098209758 learned_addr learned my addr 172.21.15.97:0/2098209758 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:08.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.012+0000 7fe077e00640 1 --2- 172.21.15.97:0/2098209758 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e24dfcdab0 0x55e24dfd7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:08.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.013+0000 7fe06f400640 1 --2- 172.21.15.97:0/2098209758 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e24dfce4a0 0x55e24dfd8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:08.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.013+0000 7fe07d000640 1 -- 172.21.15.97:0/2098209758 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e24dfce4a0 msgr2=0x55e24dfd8110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:08.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.013+0000 7fe07d000640 1 --2- 172.21.15.97:0/2098209758 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e24dfce4a0 0x55e24dfd8110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.013+0000 7fe07d000640 1 -- 172.21.15.97:0/2098209758 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e24dfcdab0 msgr2=0x55e24dfd7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:08.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.013+0000 7fe07d000640 1 --2- 172.21.15.97:0/2098209758 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e24dfcdab0 0x55e24dfd7bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.013+0000 7fe07d000640 1 -- 172.21.15.97:0/2098209758 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e24dd107d0 con 0x55e24dfd3610 2024-03-27T10:57:08.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.013+0000 7fe07d000640 1 --2- 172.21.15.97:0/2098209758 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e24dfd3610 0x55e24dfdf150 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fe070099210 tx=0x7fe07009c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1827c74d56b4037 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:08.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.013+0000 7fe066000640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T10:57:08.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.013+0000 7fe066000640 1 -- 172.21.15.97:0/2098209758 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0700a0bf0 con 0x55e24dfd3610 2024-03-27T10:57:08.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.013+0000 7fe066000640 1 -- 172.21.15.97:0/2098209758 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fe07009caa0 con 0x55e24dfd3610 2024-03-27T10:57:08.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.013+0000 7fe081e26740 1 -- 172.21.15.97:0/2098209758 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe06801a910 con 0x55e24dfd3610 2024-03-27T10:57:08.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.013+0000 7fe066000640 1 -- 172.21.15.97:0/2098209758 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe0700a54b0 con 0x55e24dfd3610 2024-03-27T10:57:08.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.013+0000 7fe081e26740 10 client.15639 fetch_fsmap learned FSMap version 28 2024-03-27T10:57:08.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.013+0000 7fe081e26740 1 -- 172.21.15.97:0/2098209758 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e24dfe2530 con 0x55e24dfd3610 2024-03-27T10:57:08.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.014+0000 7fe066000640 1 -- 172.21.15.97:0/2098209758 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe06801a910 con 0x55e24dfd3610 2024-03-27T10:57:08.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.014+0000 7fe066000640 1 -- 172.21.15.97:0/2098209758 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fe0700053a0 con 0x55e24dfd3610 2024-03-27T10:57:08.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.014+0000 7fe081e26740 10 client.15639 fetch_fsmap finished waiting for FSMap version 28 2024-03-27T10:57:08.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.014+0000 7fe081e26740 10 client.15639 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-27T10:57:08.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.014+0000 7fe081e26740 20 client.15639 populate_metadata read hostname 'smithi097' 2024-03-27T10:57:08.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.014+0000 7fe081e26740 1 --2- 172.21.15.97:0/2098209758 >> [v2:172.21.15.114:6836/616775588,v1:172.21.15.114:6837/616775588] conn(0x55e24dfe3cf0 0x55e24dfe6130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:08.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.014+0000 7fe081e26740 4 client.15639 mds_command: new command op to 34478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T10:57:08.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.014+0000 7fe081e26740 1 -- 172.21.15.97:0/2098209758 --> [v2:172.21.15.114:6836/616775588,v1:172.21.15.114:6837/616775588] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e24df3fbb0 con 0x55e24dfe3cf0 2024-03-27T10:57:08.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.014+0000 7fe077e00640 1 --2- 172.21.15.97:0/2098209758 >> [v2:172.21.15.114:6836/616775588,v1:172.21.15.114:6837/616775588] conn(0x55e24dfe3cf0 0x55e24dfe6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:08.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.015+0000 7fe077e00640 1 --2- 172.21.15.97:0/2098209758 >> [v2:172.21.15.114:6836/616775588,v1:172.21.15.114:6837/616775588] conn(0x55e24dfe3cf0 0x55e24dfe6130 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ef7777c65ad6faa6 server_cookie=20be4a3256f9ad56 in_seq=0 out_seq=0 2024-03-27T10:57:08.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.015+0000 7fe066000640 10 client.15639 ms_handle_connect on v2:172.21.15.114:6836/616775588 2024-03-27T10:57:08.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.017+0000 7fe066000640 1 -- 172.21.15.97:0/2098209758 <== mds.1 v2:172.21.15.114:6836/616775588 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55e24df3fbb0 con 0x55e24dfe3cf0 2024-03-27T10:57:08.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.017+0000 7fe066000640 10 client.15639 handle_command_reply: tid=0 2024-03-27T10:57:08.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.017+0000 7fe081e26740 1 client.15639 shutdown 2024-03-27T10:57:08.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.018+0000 7fe081e26740 1 -- 172.21.15.97:0/2098209758 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e24dfd3610 msgr2=0x55e24dfdf150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:08.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.018+0000 7fe081e26740 1 --2- 172.21.15.97:0/2098209758 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e24dfd3610 0x55e24dfdf150 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fe070099210 tx=0x7fe07009c400 comp rx=0 tx=0).stop 2024-03-27T10:57:08.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.018+0000 7fe081e26740 1 -- 172.21.15.97:0/2098209758 shutdown_connections 2024-03-27T10:57:08.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.018+0000 7fe081e26740 1 --2- 172.21.15.97:0/2098209758 >> [v2:172.21.15.114:6836/616775588,v1:172.21.15.114:6837/616775588] conn(0x55e24dfe3cf0 0x55e24dfe6130 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.018+0000 7fe081e26740 1 --2- 172.21.15.97:0/2098209758 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e24dfd3610 0x55e24dfdf150 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.018+0000 7fe081e26740 1 --2- 172.21.15.97:0/2098209758 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e24dfce4a0 0x55e24dfd8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.018+0000 7fe081e26740 1 --2- 172.21.15.97:0/2098209758 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e24dfcdab0 0x55e24dfd7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:08.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.018+0000 7fe066000640 0 client.15639 ms_handle_reset on v2:172.21.15.114:6836/616775588 2024-03-27T10:57:08.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.018+0000 7fe081e26740 1 -- 172.21.15.97:0/2098209758 >> 172.21.15.97:0/2098209758 conn(0x55e24dfc9020 msgr2=0x55e24dfc8240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:08.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.018+0000 7fe081e26740 1 -- 172.21.15.97:0/2098209758 shutdown_connections 2024-03-27T10:57:08.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.018+0000 7fe081e26740 1 -- 172.21.15.97:0/2098209758 wait complete. 2024-03-27T10:57:08.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.018+0000 7fe081e26740 20 client.15639 trim_cache size 0 max 16384 2024-03-27T10:57:09.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.032+0000 7fe081e26740 1 Processor -- start 2024-03-27T10:57:09.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.032+0000 7fe081e26740 1 -- start start 2024-03-27T10:57:09.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.032+0000 7fe081e26740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e24dfcdab0 0x55e24dfcf780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:09.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.032+0000 7fe081e26740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e24dfce4a0 0x55e24dfcfcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:09.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.032+0000 7fe081e26740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e24dfd3610 0x55e24dfdf110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:09.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.032+0000 7fe081e26740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe0781af2e0 con 0x55e24dfce4a0 2024-03-27T10:57:09.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.032+0000 7fe081e26740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe0781b0e20 con 0x55e24dfd3610 2024-03-27T10:57:09.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.032+0000 7fe081e26740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe0781af9b0 con 0x55e24dfcdab0 2024-03-27T10:57:09.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.033+0000 7fe06f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e24dfce4a0 0x55e24dfcfcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:09.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.033+0000 7fe06f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e24dfce4a0 0x55e24dfcfcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34678/0 (socket says 172.21.15.97:34678) 2024-03-27T10:57:09.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.033+0000 7fe06f400640 1 -- 172.21.15.97:0/3538791470 learned_addr learned my addr 172.21.15.97:0/3538791470 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:09.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.033+0000 7fe07d000640 1 --2- 172.21.15.97:0/3538791470 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e24dfd3610 0x55e24dfdf110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:09.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.033+0000 7fe077e00640 1 --2- 172.21.15.97:0/3538791470 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e24dfcdab0 0x55e24dfcf780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:09.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.033+0000 7fe06f400640 1 -- 172.21.15.97:0/3538791470 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e24dfcdab0 msgr2=0x55e24dfcf780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:09.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.033+0000 7fe06f400640 1 --2- 172.21.15.97:0/3538791470 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e24dfcdab0 0x55e24dfcf780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:09.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.033+0000 7fe06f400640 1 -- 172.21.15.97:0/3538791470 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e24dfd3610 msgr2=0x55e24dfdf110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:09.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.033+0000 7fe06f400640 1 --2- 172.21.15.97:0/3538791470 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e24dfd3610 0x55e24dfdf110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:09.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.033+0000 7fe06f400640 1 -- 172.21.15.97:0/3538791470 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e24dfd29d0 con 0x55e24dfce4a0 2024-03-27T10:57:09.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.033+0000 7fe07d000640 1 --2- 172.21.15.97:0/3538791470 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e24dfd3610 0x55e24dfdf110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T10:57:09.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.034+0000 7fe06f400640 1 --2- 172.21.15.97:0/3538791470 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e24dfce4a0 0x55e24dfcfcc0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fe060009230 tx=0x7fe06003f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be24842fa635961b server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:09.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.034+0000 7fe06cc00640 1 -- 172.21.15.97:0/3538791470 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe060050070 con 0x55e24dfce4a0 2024-03-27T10:57:09.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.034+0000 7fe06cc00640 1 -- 172.21.15.97:0/3538791470 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe060046070 con 0x55e24dfce4a0 2024-03-27T10:57:09.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.034+0000 7fe06cc00640 1 -- 172.21.15.97:0/3538791470 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe060004e00 con 0x55e24dfce4a0 2024-03-27T10:57:09.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.035+0000 7fe081e26740 1 -- 172.21.15.97:0/3538791470 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e24dfce4a0 msgr2=0x55e24dfcfcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:09.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.035+0000 7fe081e26740 1 --2- 172.21.15.97:0/3538791470 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e24dfce4a0 0x55e24dfcfcc0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fe060009230 tx=0x7fe06003f000 comp rx=0 tx=0).stop 2024-03-27T10:57:09.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.035+0000 7fe081e26740 1 -- 172.21.15.97:0/3538791470 shutdown_connections 2024-03-27T10:57:09.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.035+0000 7fe081e26740 1 --2- 172.21.15.97:0/3538791470 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e24dfd3610 0x55e24dfdf110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:09.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.035+0000 7fe081e26740 1 --2- 172.21.15.97:0/3538791470 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e24dfce4a0 0x55e24dfcfcc0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:09.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.035+0000 7fe081e26740 1 --2- 172.21.15.97:0/3538791470 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e24dfcdab0 0x55e24dfcf780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:09.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.035+0000 7fe081e26740 1 -- 172.21.15.97:0/3538791470 >> 172.21.15.97:0/3538791470 conn(0x55e24dfc9020 msgr2=0x55e24dfc9400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:09.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.035+0000 7fe081e26740 1 -- 172.21.15.97:0/3538791470 shutdown_connections 2024-03-27T10:57:09.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.035+0000 7fe081e26740 1 -- 172.21.15.97:0/3538791470 wait complete. 2024-03-27T10:57:09.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.035+0000 7fe081e26740 1 -- start start 2024-03-27T10:57:09.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.035+0000 7fe081e26740 1 Processor -- start 2024-03-27T10:57:09.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.036+0000 7fe081e26740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e24dfcdab0 0x55e24dfdc5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:09.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.036+0000 7fe081e26740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e24dfd3610 0x55e24dfdcae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:09.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.036+0000 7fe081e26740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e24dfddaf0 0x55e24dfddf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:09.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.036+0000 7fe081e26740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe0781b0750 con 0x55e24dfcdab0 2024-03-27T10:57:09.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.036+0000 7fe081e26740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe0781b0080 con 0x55e24dfd3610 2024-03-27T10:57:09.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.036+0000 7fe081e26740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe0781ad2c0 con 0x55e24dfddaf0 2024-03-27T10:57:09.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.036+0000 7fe081e26740 10 client.0 _ll_register_callbacks cb 0x7fe070004a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:57:09.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.036+0000 7fe077e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e24dfcdab0 0x55e24dfdc5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:09.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.036+0000 7fe077e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e24dfcdab0 0x55e24dfdc5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34680/0 (socket says 172.21.15.97:34680) 2024-03-27T10:57:09.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.036+0000 7fe077e00640 1 -- 172.21.15.97:0/2774357223 learned_addr learned my addr 172.21.15.97:0/2774357223 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:09.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.036+0000 7fe06f400640 1 --2- 172.21.15.97:0/2774357223 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e24dfd3610 0x55e24dfdcae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:09.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.036+0000 7fe07d000640 1 --2- 172.21.15.97:0/2774357223 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e24dfddaf0 0x55e24dfddf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:09.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.036+0000 7fe077e00640 1 -- 172.21.15.97:0/2774357223 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e24dfddaf0 msgr2=0x55e24dfddf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:09.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.036+0000 7fe077e00640 1 --2- 172.21.15.97:0/2774357223 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e24dfddaf0 0x55e24dfddf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:09.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.036+0000 7fe077e00640 1 -- 172.21.15.97:0/2774357223 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e24dfd3610 msgr2=0x55e24dfdcae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:09.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.036+0000 7fe077e00640 1 --2- 172.21.15.97:0/2774357223 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e24dfd3610 0x55e24dfdcae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:09.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.036+0000 7fe077e00640 1 -- 172.21.15.97:0/2774357223 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e24dfde4b0 con 0x55e24dfcdab0 2024-03-27T10:57:09.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.037+0000 7fe077e00640 1 --2- 172.21.15.97:0/2774357223 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e24dfcdab0 0x55e24dfdc5a0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fe068005fd0 tx=0x7fe06800bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a490447764bf6563 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:09.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.037+0000 7fe066000640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T10:57:09.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.037+0000 7fe066000640 1 -- 172.21.15.97:0/2774357223 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe06802c060 con 0x55e24dfcdab0 2024-03-27T10:57:09.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.037+0000 7fe066000640 1 -- 172.21.15.97:0/2774357223 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fe068004230 con 0x55e24dfcdab0 2024-03-27T10:57:09.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.037+0000 7fe081e26740 1 -- 172.21.15.97:0/2774357223 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe060046070 con 0x55e24dfcdab0 2024-03-27T10:57:09.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.037+0000 7fe066000640 1 -- 172.21.15.97:0/2774357223 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe068015040 con 0x55e24dfcdab0 2024-03-27T10:57:09.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.037+0000 7fe081e26740 10 client.15645 fetch_fsmap learned FSMap version 28 2024-03-27T10:57:09.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.037+0000 7fe081e26740 1 -- 172.21.15.97:0/2774357223 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e24e04a1a0 con 0x55e24dfcdab0 2024-03-27T10:57:09.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.038+0000 7fe066000640 1 -- 172.21.15.97:0/2774357223 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe060046070 con 0x55e24dfcdab0 2024-03-27T10:57:09.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.038+0000 7fe066000640 1 -- 172.21.15.97:0/2774357223 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fe068026410 con 0x55e24dfcdab0 2024-03-27T10:57:09.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.038+0000 7fe081e26740 10 client.15645 fetch_fsmap finished waiting for FSMap version 28 2024-03-27T10:57:09.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.038+0000 7fe081e26740 10 client.15645 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-27T10:57:09.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.038+0000 7fe081e26740 20 client.15645 populate_metadata read hostname 'smithi097' 2024-03-27T10:57:09.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.038+0000 7fe081e26740 1 --2- 172.21.15.97:0/2774357223 >> [v2:172.21.15.114:6836/616775588,v1:172.21.15.114:6837/616775588] conn(0x55e24e04a870 0x55e24e04cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:09.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.038+0000 7fe081e26740 4 client.15645 mds_command: new command op to 34478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-27T10:57:09.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.038+0000 7fe081e26740 1 -- 172.21.15.97:0/2774357223 --> [v2:172.21.15.114:6836/616775588,v1:172.21.15.114:6837/616775588] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e24df39f00 con 0x55e24e04a870 2024-03-27T10:57:09.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.038+0000 7fe06f400640 1 --2- 172.21.15.97:0/2774357223 >> [v2:172.21.15.114:6836/616775588,v1:172.21.15.114:6837/616775588] conn(0x55e24e04a870 0x55e24e04cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:09.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.039+0000 7fe06f400640 1 --2- 172.21.15.97:0/2774357223 >> [v2:172.21.15.114:6836/616775588,v1:172.21.15.114:6837/616775588] conn(0x55e24e04a870 0x55e24e04cc60 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d7f5bc3a2c5dc66c server_cookie=de59dd1fc224bfd1 in_seq=0 out_seq=0 2024-03-27T10:57:09.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.039+0000 7fe066000640 10 client.15645 ms_handle_connect on v2:172.21.15.114:6836/616775588 2024-03-27T10:57:09.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.039+0000 7fe066000640 1 -- 172.21.15.97:0/2774357223 <== mds.1 v2:172.21.15.114:6836/616775588 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e24df39f00 con 0x55e24e04a870 2024-03-27T10:57:09.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.040+0000 7fe066000640 10 client.15645 handle_command_reply: tid=0 2024-03-27T10:57:09.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.040+0000 7fe081e26740 1 client.15645 shutdown 2024-03-27T10:57:09.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.040+0000 7fe081e26740 1 -- 172.21.15.97:0/2774357223 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e24dfcdab0 msgr2=0x55e24dfdc5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:09.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.040+0000 7fe081e26740 1 --2- 172.21.15.97:0/2774357223 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e24dfcdab0 0x55e24dfdc5a0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fe068005fd0 tx=0x7fe06800bf80 comp rx=0 tx=0).stop 2024-03-27T10:57:09.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.040+0000 7fe081e26740 1 -- 172.21.15.97:0/2774357223 shutdown_connections 2024-03-27T10:57:09.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.040+0000 7fe081e26740 1 --2- 172.21.15.97:0/2774357223 >> [v2:172.21.15.114:6836/616775588,v1:172.21.15.114:6837/616775588] conn(0x55e24e04a870 0x55e24e04cc60 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:09.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.040+0000 7fe081e26740 1 --2- 172.21.15.97:0/2774357223 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e24dfddaf0 0x55e24dfddf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:09.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.040+0000 7fe081e26740 1 --2- 172.21.15.97:0/2774357223 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e24dfd3610 0x55e24dfdcae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:09.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.040+0000 7fe066000640 0 client.15645 ms_handle_reset on v2:172.21.15.114:6836/616775588 2024-03-27T10:57:09.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.040+0000 7fe081e26740 1 --2- 172.21.15.97:0/2774357223 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e24dfcdab0 0x55e24dfdc5a0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:09.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.040+0000 7fe081e26740 1 -- 172.21.15.97:0/2774357223 >> 172.21.15.97:0/2774357223 conn(0x55e24dfc9020 msgr2=0x55e24dfcf780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:09.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.040+0000 7fe081e26740 1 -- 172.21.15.97:0/2774357223 shutdown_connections 2024-03-27T10:57:09.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.040+0000 7fe081e26740 1 -- 172.21.15.97:0/2774357223 wait complete. 2024-03-27T10:57:09.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.040+0000 7fe081e26740 20 client.15645 trim_cache size 0 max 16384 2024-03-27T10:57:09.016 INFO:teuthology.orchestra.run.smithi097.stdout:[] 2024-03-27T10:57:09.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.041+0000 7fe07da00640 1 -- 172.21.15.97:0/3599358732 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fe05c0a3690 msgr2=0x7fe05c0a5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:09.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.041+0000 7fe07da00640 1 --2- 172.21.15.97:0/3599358732 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fe05c0a3690 0x7fe05c0a5b50 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fe060006010 tx=0x7fe060005a20 comp rx=0 tx=0).stop 2024-03-27T10:57:09.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.041+0000 7fe07da00640 1 -- 172.21.15.97:0/3599358732 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe078109c20 msgr2=0x7fe0781a5840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:09.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.041+0000 7fe07da00640 1 --2- 172.21.15.97:0/3599358732 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe078109c20 0x7fe0781a5840 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fe07000eb10 tx=0x7fe07000c2d0 comp rx=0 tx=0).stop 2024-03-27T10:57:09.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.041+0000 7fe07da00640 1 -- 172.21.15.97:0/3599358732 shutdown_connections 2024-03-27T10:57:09.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.041+0000 7fe07da00640 1 --2- 172.21.15.97:0/3599358732 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe078109c20 0x7fe0781a5840 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:09.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.041+0000 7fe07da00640 1 --2- 172.21.15.97:0/3599358732 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fe05c0a3690 0x7fe05c0a5b50 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:09.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.041+0000 7fe07da00640 1 --2- 172.21.15.97:0/3599358732 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe078109260 0x7fe07819e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:09.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.041+0000 7fe07da00640 1 --2- 172.21.15.97:0/3599358732 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe078108070 0x7fe07819e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:09.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.041+0000 7fe07da00640 1 -- 172.21.15.97:0/3599358732 >> 172.21.15.97:0/3599358732 conn(0x7fe078077020 msgr2=0x7fe07810c560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:09.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.041+0000 7fe07da00640 1 -- 172.21.15.97:0/3599358732 shutdown_connections 2024-03-27T10:57:09.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.041+0000 7fe07da00640 1 -- 172.21.15.97:0/3599358732 wait complete. 2024-03-27T10:57:09.029 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-27T10:57:09.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.256+0000 7faebb600640 1 -- 172.21.15.97:0/220704564 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faeb4105940 msgr2=0x7faeb410c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:09.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.256+0000 7faebb600640 1 --2- 172.21.15.97:0/220704564 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faeb4105940 0x7faeb410c1d0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7faeac009ca0 tx=0x7faeac02c0f0 comp rx=0 tx=0).stop 2024-03-27T10:57:09.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.256+0000 7faebb600640 1 -- 172.21.15.97:0/220704564 shutdown_connections 2024-03-27T10:57:09.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.256+0000 7faebb600640 1 --2- 172.21.15.97:0/220704564 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faeb4105940 0x7faeb410c1d0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:09.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.256+0000 7faebb600640 1 --2- 172.21.15.97:0/220704564 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faeb4104f80 0x7faeb4105400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:09.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.256+0000 7faebb600640 1 --2- 172.21.15.97:0/220704564 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faeb4103d90 0x7faeb41041b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:09.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.256+0000 7faebb600640 1 -- 172.21.15.97:0/220704564 >> 172.21.15.97:0/220704564 conn(0x7faeb40ff300 msgr2=0x7faeb4101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:09.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.256+0000 7faebb600640 1 -- 172.21.15.97:0/220704564 shutdown_connections 2024-03-27T10:57:09.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.256+0000 7faebb600640 1 -- 172.21.15.97:0/220704564 wait complete. 2024-03-27T10:57:09.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.257+0000 7faebb600640 1 Processor -- start 2024-03-27T10:57:09.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.257+0000 7faebb600640 1 -- start start 2024-03-27T10:57:09.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.258+0000 7faebb600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faeb4103d90 0x7faeb4199f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:09.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.258+0000 7faebb600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faeb4104f80 0x7faeb419a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:09.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.258+0000 7faebb600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faeb4105940 0x7faeb41a14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:09.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.258+0000 7faebb600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7faeb410f8c0 con 0x7faeb4104f80 2024-03-27T10:57:09.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.258+0000 7faebb600640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7faeb410f740 con 0x7faeb4105940 2024-03-27T10:57:09.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.258+0000 7faebb600640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7faeb410fa40 con 0x7faeb4103d90 2024-03-27T10:57:09.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.258+0000 7faeb3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faeb4104f80 0x7faeb419a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:09.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.258+0000 7faeb3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faeb4104f80 0x7faeb419a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34704/0 (socket says 172.21.15.97:34704) 2024-03-27T10:57:09.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.258+0000 7faeb3e00640 1 -- 172.21.15.97:0/3463160601 learned_addr learned my addr 172.21.15.97:0/3463160601 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:09.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.258+0000 7faeb8c00640 1 --2- 172.21.15.97:0/3463160601 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faeb4103d90 0x7faeb4199f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:09.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.258+0000 7faeb9600640 1 --2- 172.21.15.97:0/3463160601 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faeb4105940 0x7faeb41a14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:09.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.258+0000 7faeb3e00640 1 -- 172.21.15.97:0/3463160601 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faeb4103d90 msgr2=0x7faeb4199f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:09.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.259+0000 7faeb3e00640 1 --2- 172.21.15.97:0/3463160601 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faeb4103d90 0x7faeb4199f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:09.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.259+0000 7faeb3e00640 1 -- 172.21.15.97:0/3463160601 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faeb4105940 msgr2=0x7faeb41a14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:09.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.259+0000 7faeb3e00640 1 --2- 172.21.15.97:0/3463160601 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faeb4105940 0x7faeb41a14f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:09.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.259+0000 7faeb3e00640 1 -- 172.21.15.97:0/3463160601 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faeb41a1b60 con 0x7faeb4104f80 2024-03-27T10:57:09.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.259+0000 7faeb3e00640 1 --2- 172.21.15.97:0/3463160601 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faeb4104f80 0x7faeb419a480 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7faea000e9f0 tx=0x7faea000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:09.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.259+0000 7faeb1600640 1 -- 172.21.15.97:0/3463160601 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea000cd80 con 0x7faeb4104f80 2024-03-27T10:57:09.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.259+0000 7faeb1600640 1 -- 172.21.15.97:0/3463160601 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faea00108e0 con 0x7faeb4104f80 2024-03-27T10:57:09.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.259+0000 7faeb1600640 1 -- 172.21.15.97:0/3463160601 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea0018920 con 0x7faeb4104f80 2024-03-27T10:57:09.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.259+0000 7faebb600640 1 -- 172.21.15.97:0/3463160601 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faeb41a1e50 con 0x7faeb4104f80 2024-03-27T10:57:09.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.259+0000 7faebb600640 1 -- 172.21.15.97:0/3463160601 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faeb41a2360 con 0x7faeb4104f80 2024-03-27T10:57:09.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.262+0000 7faebe3a8740 1 Processor -- start 2024-03-27T10:57:09.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.263+0000 7faebe3a8740 1 -- start start 2024-03-27T10:57:09.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.263+0000 7faebe3a8740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af570198c0 0x55af57019ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:09.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.263+0000 7faebe3a8740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55af5701a2b0 0x55af5701edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:09.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.263+0000 7faebe3a8740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55af5701f420 0x55af570218b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:09.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.263+0000 7faebe3a8740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55af56fe17b0 con 0x55af5701f420 2024-03-27T10:57:09.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.263+0000 7faebe3a8740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55af56f7e670 con 0x55af570198c0 2024-03-27T10:57:09.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.263+0000 7faebe3a8740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55af56f26970 con 0x55af5701a2b0 2024-03-27T10:57:09.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.263+0000 7faeb8c00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55af5701a2b0 0x55af5701edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:09.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.263+0000 7faeb9600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af570198c0 0x55af57019ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:09.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.263+0000 7faeb9600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af570198c0 0x55af57019ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:42562/0 (socket says 172.21.15.97:42562) 2024-03-27T10:57:09.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.263+0000 7faeb9600640 1 -- 172.21.15.97:0/3727580636 learned_addr learned my addr 172.21.15.97:0/3727580636 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:09.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.263+0000 7faeb8c00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55af5701a2b0 0x55af5701edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.97:44262/0 (socket says 172.21.15.97:44262) 2024-03-27T10:57:09.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.264+0000 7faeb1600640 1 -- 172.21.15.97:0/3463160601 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7faea0010a90 con 0x7faeb4104f80 2024-03-27T10:57:09.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.264+0000 7faeb9600640 1 -- 172.21.15.97:0/3727580636 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55af5701a2b0 msgr2=0x55af5701edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:09.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.264+0000 7faeb9600640 1 --2- 172.21.15.97:0/3727580636 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55af5701a2b0 0x55af5701edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:09.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.264+0000 7faeb9600640 1 -- 172.21.15.97:0/3727580636 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55af5701f420 msgr2=0x55af570218b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:09.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.264+0000 7faeb3e00640 1 --2- 172.21.15.97:0/3727580636 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55af5701f420 0x55af570218b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:09.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.264+0000 7faeb9600640 1 --2- 172.21.15.97:0/3727580636 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55af5701f420 0x55af570218b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:09.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.264+0000 7faeb9600640 1 -- 172.21.15.97:0/3727580636 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af56c923f0 con 0x55af570198c0 2024-03-27T10:57:09.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.264+0000 7faeb3e00640 1 --2- 172.21.15.97:0/3727580636 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55af5701f420 0x55af570218b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:57:09.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.264+0000 7faeb9600640 1 --2- 172.21.15.97:0/3727580636 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af570198c0 0x55af57019ce0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7faeac032380 tx=0x7faeac005fa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb0f995d0dffb8f3 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:09.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.265+0000 7faea9c00640 1 -- 172.21.15.97:0/3727580636 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeac043070 con 0x55af570198c0 2024-03-27T10:57:09.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.265+0000 7faea9c00640 1 -- 172.21.15.97:0/3727580636 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faeac032db0 con 0x55af570198c0 2024-03-27T10:57:09.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.265+0000 7faeb1600640 1 --2- 172.21.15.97:0/3463160601 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fae980a3690 0x7fae980a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:09.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.265+0000 7faea9c00640 1 -- 172.21.15.97:0/3727580636 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeac039040 con 0x55af570198c0 2024-03-27T10:57:09.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.265+0000 7faeb1600640 1 -- 172.21.15.97:0/3463160601 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7faea00cffc0 con 0x7faeb4104f80 2024-03-27T10:57:09.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.265+0000 7faeb8c00640 1 --2- 172.21.15.97:0/3463160601 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fae980a3690 0x7fae980a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:09.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.266+0000 7faebe3a8740 1 -- 172.21.15.97:0/3727580636 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af570198c0 msgr2=0x55af57019ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:09.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.266+0000 7faebe3a8740 1 --2- 172.21.15.97:0/3727580636 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af570198c0 0x55af57019ce0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7faeac032380 tx=0x7faeac005fa0 comp rx=0 tx=0).stop 2024-03-27T10:57:09.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.266+0000 7faebe3a8740 1 -- 172.21.15.97:0/3727580636 shutdown_connections 2024-03-27T10:57:09.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.266+0000 7faebe3a8740 1 --2- 172.21.15.97:0/3727580636 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55af5701f420 0x55af570218b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:09.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.266+0000 7faebe3a8740 1 --2- 172.21.15.97:0/3727580636 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55af5701a2b0 0x55af5701edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:09.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.266+0000 7faebe3a8740 1 --2- 172.21.15.97:0/3727580636 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af570198c0 0x55af57019ce0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:09.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.266+0000 7faebe3a8740 1 -- 172.21.15.97:0/3727580636 >> 172.21.15.97:0/3727580636 conn(0x55af57014e30 msgr2=0x55af57015230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:09.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.266+0000 7faeb8c00640 1 --2- 172.21.15.97:0/3463160601 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fae980a3690 0x7fae980a5b50 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x55af5701a6b0 tx=0x7faea4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:09.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.266+0000 7faebe3a8740 1 -- 172.21.15.97:0/3727580636 shutdown_connections 2024-03-27T10:57:09.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.266+0000 7faebe3a8740 1 -- 172.21.15.97:0/3727580636 wait complete. 2024-03-27T10:57:09.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.266+0000 7faebe3a8740 1 -- start start 2024-03-27T10:57:09.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.267+0000 7faebe3a8740 1 Processor -- start 2024-03-27T10:57:09.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.267+0000 7faebe3a8740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55af570198c0 0x55af57023a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:09.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.267+0000 7faebe3a8740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55af5701a2b0 0x55af57023f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:09.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.267+0000 7faebe3a8740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af5701f420 0x55af5702af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:09.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.267+0000 7faebe3a8740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55af56f39ca0 con 0x55af5701a2b0 2024-03-27T10:57:09.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.267+0000 7faebe3a8740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55af56f7a620 con 0x55af5701f420 2024-03-27T10:57:09.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.267+0000 7faebe3a8740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55af57022370 con 0x55af570198c0 2024-03-27T10:57:09.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.267+0000 7faebe3a8740 10 client.0 _ll_register_callbacks cb 0x55af56f76160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:57:09.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.267+0000 7faeb3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55af5701a2b0 0x55af57023f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:09.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.267+0000 7faeb3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55af5701a2b0 0x55af57023f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34716/0 (socket says 172.21.15.97:34716) 2024-03-27T10:57:09.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.267+0000 7faeb3e00640 1 -- 172.21.15.97:0/870768141 learned_addr learned my addr 172.21.15.97:0/870768141 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:09.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.267+0000 7faeb8c00640 1 --2- 172.21.15.97:0/870768141 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55af570198c0 0x55af57023a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:09.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.268+0000 7faeb9600640 1 --2- 172.21.15.97:0/870768141 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af5701f420 0x55af5702af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:09.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.268+0000 7faeb3e00640 1 -- 172.21.15.97:0/870768141 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55af570198c0 msgr2=0x55af57023a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:09.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.268+0000 7faeb3e00640 1 --2- 172.21.15.97:0/870768141 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55af570198c0 0x55af57023a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:09.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.268+0000 7faeb3e00640 1 -- 172.21.15.97:0/870768141 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af5701f420 msgr2=0x55af5702af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:09.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.268+0000 7faeb3e00640 1 --2- 172.21.15.97:0/870768141 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af5701f420 0x55af5702af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:09.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.268+0000 7faeb3e00640 1 -- 172.21.15.97:0/870768141 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af56d5c7d0 con 0x55af5701a2b0 2024-03-27T10:57:09.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.268+0000 7faeb3e00640 1 --2- 172.21.15.97:0/870768141 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55af5701a2b0 0x55af57023f50 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7faea009c480 tx=0x7faea0010de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=673902a402d66188 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:09.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.268+0000 7fae9e000640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T10:57:09.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.268+0000 7fae9e000640 1 -- 172.21.15.97:0/870768141 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea0010630 con 0x55af5701a2b0 2024-03-27T10:57:09.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.268+0000 7fae9e000640 1 -- 172.21.15.97:0/870768141 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7faea009b940 con 0x55af5701a2b0 2024-03-27T10:57:09.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.268+0000 7faebe3a8740 1 -- 172.21.15.97:0/870768141 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faeac032db0 con 0x55af5701a2b0 2024-03-27T10:57:09.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.268+0000 7fae9e000640 1 -- 172.21.15.97:0/870768141 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7faea00a50b0 con 0x55af5701a2b0 2024-03-27T10:57:09.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.268+0000 7faebe3a8740 10 client.15654 fetch_fsmap learned FSMap version 28 2024-03-27T10:57:09.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.268+0000 7faebe3a8740 1 -- 172.21.15.97:0/870768141 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55af5702e270 con 0x55af5701a2b0 2024-03-27T10:57:09.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.269+0000 7fae9e000640 1 -- 172.21.15.97:0/870768141 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faeac032db0 con 0x55af5701a2b0 2024-03-27T10:57:09.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.269+0000 7fae9e000640 1 -- 172.21.15.97:0/870768141 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7faea009b3b0 con 0x55af5701a2b0 2024-03-27T10:57:09.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.269+0000 7faebe3a8740 10 client.15654 fetch_fsmap finished waiting for FSMap version 28 2024-03-27T10:57:09.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.269+0000 7faebe3a8740 10 client.15654 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-27T10:57:09.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.269+0000 7faebe3a8740 20 client.15654 populate_metadata read hostname 'smithi097' 2024-03-27T10:57:09.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.269+0000 7faebe3a8740 1 --2- 172.21.15.97:0/870768141 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55af5702fa50 0x55af57031eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:09.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.269+0000 7faebe3a8740 4 client.15654 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T10:57:09.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.269+0000 7faebe3a8740 1 -- 172.21.15.97:0/870768141 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55af56f8bbb0 con 0x55af5702fa50 2024-03-27T10:57:09.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.270+0000 7faeb8c00640 1 --2- 172.21.15.97:0/870768141 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55af5702fa50 0x55af57031eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:09.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.270+0000 7faeb8c00640 1 --2- 172.21.15.97:0/870768141 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55af5702fa50 0x55af57031eb0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fc466a8af509747 server_cookie=1990a7fc69c7205d in_seq=0 out_seq=0 2024-03-27T10:57:09.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.270+0000 7fae9e000640 10 client.15654 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T10:57:09.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.272+0000 7fae9e000640 1 -- 172.21.15.97:0/870768141 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55af56f8bbb0 con 0x55af5702fa50 2024-03-27T10:57:09.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.272+0000 7fae9e000640 10 client.15654 handle_command_reply: tid=0 2024-03-27T10:57:09.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.272+0000 7faebe3a8740 1 client.15654 shutdown 2024-03-27T10:57:09.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.272+0000 7faebe3a8740 1 -- 172.21.15.97:0/870768141 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55af5701a2b0 msgr2=0x55af57023f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:09.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.272+0000 7faebe3a8740 1 --2- 172.21.15.97:0/870768141 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55af5701a2b0 0x55af57023f50 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7faea009c480 tx=0x7faea0010de0 comp rx=0 tx=0).stop 2024-03-27T10:57:09.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.272+0000 7faebe3a8740 1 -- 172.21.15.97:0/870768141 shutdown_connections 2024-03-27T10:57:09.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.272+0000 7faebe3a8740 1 --2- 172.21.15.97:0/870768141 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55af5702fa50 0x55af57031eb0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:09.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.272+0000 7faebe3a8740 1 --2- 172.21.15.97:0/870768141 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af5701f420 0x55af5702af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:09.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.272+0000 7faebe3a8740 1 --2- 172.21.15.97:0/870768141 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55af5701a2b0 0x55af57023f50 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:09.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.272+0000 7fae9e000640 0 client.15654 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T10:57:09.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.272+0000 7faebe3a8740 1 --2- 172.21.15.97:0/870768141 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55af570198c0 0x55af57023a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:09.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.272+0000 7faebe3a8740 1 -- 172.21.15.97:0/870768141 >> 172.21.15.97:0/870768141 conn(0x55af57014e30 msgr2=0x55af57014030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:09.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.272+0000 7faebe3a8740 1 -- 172.21.15.97:0/870768141 shutdown_connections 2024-03-27T10:57:09.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.272+0000 7faebe3a8740 1 -- 172.21.15.97:0/870768141 wait complete. 2024-03-27T10:57:09.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.272+0000 7faebe3a8740 20 client.15654 trim_cache size 0 max 16384 2024-03-27T10:57:09.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.286+0000 7faebe3a8740 1 Processor -- start 2024-03-27T10:57:09.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.286+0000 7faebe3a8740 1 -- start start 2024-03-27T10:57:09.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.287+0000 7faebe3a8740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55af570198c0 0x55af5700fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:09.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.287+0000 7faebe3a8740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55af5701a2b0 0x55af5702aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:09.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.287+0000 7faebe3a8740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af5701f420 0x55af570187a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:09.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.287+0000 7faebe3a8740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7faeb41ab190 con 0x55af570198c0 2024-03-27T10:57:09.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.287+0000 7faebe3a8740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7faeb41accd0 con 0x55af5701f420 2024-03-27T10:57:09.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.287+0000 7faebe3a8740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7faeb41ab860 con 0x55af5701a2b0 2024-03-27T10:57:09.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.287+0000 7faeb8c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55af570198c0 0x55af5700fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:09.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.287+0000 7faeb8c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55af570198c0 0x55af5700fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34732/0 (socket says 172.21.15.97:34732) 2024-03-27T10:57:09.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.287+0000 7faeb8c00640 1 -- 172.21.15.97:0/866284802 learned_addr learned my addr 172.21.15.97:0/866284802 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:09.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.287+0000 7faeb9600640 1 --2- 172.21.15.97:0/866284802 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af5701f420 0x55af570187a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:09.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.287+0000 7faeb3e00640 1 --2- 172.21.15.97:0/866284802 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55af5701a2b0 0x55af5702aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:09.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.287+0000 7faeb8c00640 1 -- 172.21.15.97:0/866284802 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55af5701a2b0 msgr2=0x55af5702aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:09.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.288+0000 7faeb8c00640 1 --2- 172.21.15.97:0/866284802 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55af5701a2b0 0x55af5702aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:09.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.288+0000 7faeb8c00640 1 -- 172.21.15.97:0/866284802 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af5701f420 msgr2=0x55af570187a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:09.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.288+0000 7faeb8c00640 1 --2- 172.21.15.97:0/866284802 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af5701f420 0x55af570187a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:09.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.288+0000 7faeb8c00640 1 -- 172.21.15.97:0/866284802 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af570158a0 con 0x55af570198c0 2024-03-27T10:57:09.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.288+0000 7faeb8c00640 1 --2- 172.21.15.97:0/866284802 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55af570198c0 0x55af5700fdd0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x55af5701ad20 tx=0x7faea4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79e5a07f3d8368a3 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:09.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.288+0000 7faea9200640 1 -- 172.21.15.97:0/866284802 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea4004410 con 0x55af570198c0 2024-03-27T10:57:09.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.288+0000 7faea9200640 1 -- 172.21.15.97:0/866284802 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faea4030070 con 0x55af570198c0 2024-03-27T10:57:09.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.288+0000 7faea9200640 1 -- 172.21.15.97:0/866284802 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea401f8e0 con 0x55af570198c0 2024-03-27T10:57:09.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.289+0000 7faebe3a8740 1 -- 172.21.15.97:0/866284802 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55af570198c0 msgr2=0x55af5700fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:09.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.289+0000 7faebe3a8740 1 --2- 172.21.15.97:0/866284802 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55af570198c0 0x55af5700fdd0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x55af5701ad20 tx=0x7faea4017000 comp rx=0 tx=0).stop 2024-03-27T10:57:09.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.289+0000 7faebe3a8740 1 -- 172.21.15.97:0/866284802 shutdown_connections 2024-03-27T10:57:09.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.289+0000 7faebe3a8740 1 --2- 172.21.15.97:0/866284802 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af5701f420 0x55af570187a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:09.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.289+0000 7faebe3a8740 1 --2- 172.21.15.97:0/866284802 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55af5701a2b0 0x55af5702aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:09.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.289+0000 7faebe3a8740 1 --2- 172.21.15.97:0/866284802 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55af570198c0 0x55af5700fdd0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:09.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.289+0000 7faebe3a8740 1 -- 172.21.15.97:0/866284802 >> 172.21.15.97:0/866284802 conn(0x55af57014e30 msgr2=0x55af57018090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:09.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.289+0000 7faebe3a8740 1 -- 172.21.15.97:0/866284802 shutdown_connections 2024-03-27T10:57:09.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.289+0000 7faebe3a8740 1 -- 172.21.15.97:0/866284802 wait complete. 2024-03-27T10:57:09.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.289+0000 7faebe3a8740 1 -- start start 2024-03-27T10:57:09.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.289+0000 7faebe3a8740 1 Processor -- start 2024-03-27T10:57:09.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.290+0000 7faebe3a8740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55af570198c0 0x55af57028ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:09.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.290+0000 7faebe3a8740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af5701a2b0 0x55af57029020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:09.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.290+0000 7faebe3a8740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55af5701f420 0x55af5702a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:09.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.290+0000 7faebe3a8740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7faeb41ac600 con 0x55af570198c0 2024-03-27T10:57:09.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.290+0000 7faebe3a8740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7faeb41abf30 con 0x55af5701a2b0 2024-03-27T10:57:09.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.290+0000 7faebe3a8740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7faeb41a9170 con 0x55af5701f420 2024-03-27T10:57:09.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.290+0000 7faebe3a8740 10 client.0 _ll_register_callbacks cb 0x7faea0009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:57:09.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.290+0000 7faeb8c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55af570198c0 0x55af57028ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:09.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.290+0000 7faeb8c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55af570198c0 0x55af57028ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34738/0 (socket says 172.21.15.97:34738) 2024-03-27T10:57:09.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.290+0000 7faeb8c00640 1 -- 172.21.15.97:0/676342858 learned_addr learned my addr 172.21.15.97:0/676342858 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:09.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.290+0000 7faeb3e00640 1 --2- 172.21.15.97:0/676342858 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af5701a2b0 0x55af57029020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:09.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.290+0000 7faeb9600640 1 --2- 172.21.15.97:0/676342858 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55af5701f420 0x55af5702a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:09.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.290+0000 7faeb8c00640 1 -- 172.21.15.97:0/676342858 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55af5701f420 msgr2=0x55af5702a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:09.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.290+0000 7faeb8c00640 1 --2- 172.21.15.97:0/676342858 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55af5701f420 0x55af5702a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:09.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.290+0000 7faeb8c00640 1 -- 172.21.15.97:0/676342858 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af5701a2b0 msgr2=0x55af57029020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:09.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.290+0000 7faeb8c00640 1 --2- 172.21.15.97:0/676342858 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af5701a2b0 0x55af57029020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:09.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.290+0000 7faeb8c00640 1 -- 172.21.15.97:0/676342858 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af5702a620 con 0x55af570198c0 2024-03-27T10:57:09.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.291+0000 7faeb8c00640 1 --2- 172.21.15.97:0/676342858 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55af570198c0 0x55af57028ae0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7faea4009b40 tx=0x7faea4004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bd0dbc6ead7c14d server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:09.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.291+0000 7fae9e000640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T10:57:09.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.291+0000 7fae9e000640 1 -- 172.21.15.97:0/676342858 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea4038070 con 0x55af570198c0 2024-03-27T10:57:09.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.291+0000 7fae9e000640 1 -- 172.21.15.97:0/676342858 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7faea402ce60 con 0x55af570198c0 2024-03-27T10:57:09.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.291+0000 7faebe3a8740 1 -- 172.21.15.97:0/676342858 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faea4030070 con 0x55af570198c0 2024-03-27T10:57:09.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.291+0000 7fae9e000640 1 -- 172.21.15.97:0/676342858 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7faea4044050 con 0x55af570198c0 2024-03-27T10:57:09.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.291+0000 7faebe3a8740 10 client.15660 fetch_fsmap learned FSMap version 28 2024-03-27T10:57:09.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.291+0000 7faebe3a8740 1 -- 172.21.15.97:0/676342858 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55af5709ea20 con 0x55af570198c0 2024-03-27T10:57:09.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.292+0000 7fae9e000640 1 -- 172.21.15.97:0/676342858 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faea4030070 con 0x55af570198c0 2024-03-27T10:57:09.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.292+0000 7fae9e000640 1 -- 172.21.15.97:0/676342858 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7faea40047c0 con 0x55af570198c0 2024-03-27T10:57:09.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.292+0000 7faebe3a8740 10 client.15660 fetch_fsmap finished waiting for FSMap version 28 2024-03-27T10:57:09.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.292+0000 7faebe3a8740 10 client.15660 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-27T10:57:09.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.292+0000 7faebe3a8740 20 client.15660 populate_metadata read hostname 'smithi097' 2024-03-27T10:57:09.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.292+0000 7faebe3a8740 1 --2- 172.21.15.97:0/676342858 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55af5709f3f0 0x55af570a1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:09.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.292+0000 7faebe3a8740 4 client.15660 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-27T10:57:09.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.292+0000 7faebe3a8740 1 -- 172.21.15.97:0/676342858 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55af56f85f00 con 0x55af5709f3f0 2024-03-27T10:57:09.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.292+0000 7faeb3e00640 1 --2- 172.21.15.97:0/676342858 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55af5709f3f0 0x55af570a1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:09.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.293+0000 7faeb3e00640 1 --2- 172.21.15.97:0/676342858 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55af5709f3f0 0x55af570a1810 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d4cdabeac29130c server_cookie=65a88ec5a3ca061c in_seq=0 out_seq=0 2024-03-27T10:57:09.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.293+0000 7fae9e000640 10 client.15660 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T10:57:09.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.294+0000 7fae9e000640 1 -- 172.21.15.97:0/676342858 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55af56f85f00 con 0x55af5709f3f0 2024-03-27T10:57:09.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.294+0000 7fae9e000640 10 client.15660 handle_command_reply: tid=0 2024-03-27T10:57:09.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.294+0000 7faebe3a8740 1 client.15660 shutdown 2024-03-27T10:57:09.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.294+0000 7faebe3a8740 1 -- 172.21.15.97:0/676342858 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55af570198c0 msgr2=0x55af57028ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:09.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.294+0000 7faebe3a8740 1 --2- 172.21.15.97:0/676342858 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55af570198c0 0x55af57028ae0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7faea4009b40 tx=0x7faea4004410 comp rx=0 tx=0).stop 2024-03-27T10:57:09.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.294+0000 7faebe3a8740 1 -- 172.21.15.97:0/676342858 shutdown_connections 2024-03-27T10:57:09.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.294+0000 7faebe3a8740 1 --2- 172.21.15.97:0/676342858 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55af5709f3f0 0x55af570a1810 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:09.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.294+0000 7faebe3a8740 1 --2- 172.21.15.97:0/676342858 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55af5701f420 0x55af5702a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:09.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.294+0000 7faebe3a8740 1 --2- 172.21.15.97:0/676342858 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af5701a2b0 0x55af57029020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:09.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.294+0000 7fae9e000640 0 client.15660 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T10:57:09.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.294+0000 7faebe3a8740 1 --2- 172.21.15.97:0/676342858 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55af570198c0 0x55af57028ae0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:09.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.294+0000 7faebe3a8740 1 -- 172.21.15.97:0/676342858 >> 172.21.15.97:0/676342858 conn(0x55af57014e30 msgr2=0x55af57018920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:09.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.294+0000 7faebe3a8740 1 -- 172.21.15.97:0/676342858 shutdown_connections 2024-03-27T10:57:09.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.294+0000 7faebe3a8740 1 -- 172.21.15.97:0/676342858 wait complete. 2024-03-27T10:57:09.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.294+0000 7faebe3a8740 20 client.15660 trim_cache size 0 max 16384 2024-03-27T10:57:09.270 INFO:teuthology.orchestra.run.smithi097.stdout:[] 2024-03-27T10:57:09.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.295+0000 7faebb600640 1 -- 172.21.15.97:0/3463160601 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fae980a3690 msgr2=0x7fae980a5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:09.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.295+0000 7faebb600640 1 --2- 172.21.15.97:0/3463160601 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fae980a3690 0x7fae980a5b50 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x55af5701a6b0 tx=0x7faea4005f90 comp rx=0 tx=0).stop 2024-03-27T10:57:09.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.295+0000 7faebb600640 1 -- 172.21.15.97:0/3463160601 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faeb4104f80 msgr2=0x7faeb419a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:09.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.295+0000 7faebb600640 1 --2- 172.21.15.97:0/3463160601 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faeb4104f80 0x7faeb419a480 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7faea000e9f0 tx=0x7faea000eeb0 comp rx=0 tx=0).stop 2024-03-27T10:57:09.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.295+0000 7faebb600640 1 -- 172.21.15.97:0/3463160601 shutdown_connections 2024-03-27T10:57:09.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.295+0000 7faebb600640 1 --2- 172.21.15.97:0/3463160601 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fae980a3690 0x7fae980a5b50 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:09.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.295+0000 7faebb600640 1 --2- 172.21.15.97:0/3463160601 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faeb4105940 0x7faeb41a14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:09.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.295+0000 7faebb600640 1 --2- 172.21.15.97:0/3463160601 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faeb4104f80 0x7faeb419a480 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:09.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.295+0000 7faebb600640 1 --2- 172.21.15.97:0/3463160601 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faeb4103d90 0x7faeb4199f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:09.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.295+0000 7faebb600640 1 -- 172.21.15.97:0/3463160601 >> 172.21.15.97:0/3463160601 conn(0x7faeb40ff300 msgr2=0x7faeb4108280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:09.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.295+0000 7faebb600640 1 -- 172.21.15.97:0/3463160601 shutdown_connections 2024-03-27T10:57:09.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:09.295+0000 7faebb600640 1 -- 172.21.15.97:0/3463160601 wait complete. 2024-03-27T10:57:09.390 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:57:09 smithi097 ceph-mon[31859]: pgmap v456: 97 pgs: 97 active+clean; 663 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-27T10:57:09.516 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:57:09 smithi114 ceph-mon[35338]: pgmap v456: 97 pgs: 97 active+clean; 663 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-27T10:57:09.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:57:09 smithi177 ceph-mon[35294]: pgmap v456: 97 pgs: 97 active+clean; 663 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-27T10:57:10.284 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-27T10:57:10.284 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-27T10:57:10.285 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-27T10:57:10.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.538+0000 7f678da00640 1 -- 172.21.15.97:0/1760105815 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6788076510 msgr2=0x7f6788076930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:10.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.538+0000 7f678da00640 1 --2- 172.21.15.97:0/1760105815 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6788076510 0x7f6788076930 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f6778009ca0 tx=0x7f677802c0f0 comp rx=0 tx=0).stop 2024-03-27T10:57:10.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.538+0000 7f678da00640 1 -- 172.21.15.97:0/1760105815 shutdown_connections 2024-03-27T10:57:10.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.538+0000 7f678da00640 1 --2- 172.21.15.97:0/1760105815 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f67880780f0 0x7f6788078990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:10.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.538+0000 7f678da00640 1 --2- 172.21.15.97:0/1760105815 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6788077750 0x7f6788077bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:10.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.538+0000 7f678da00640 1 --2- 172.21.15.97:0/1760105815 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6788076510 0x7f6788076930 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:10.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.538+0000 7f678da00640 1 -- 172.21.15.97:0/1760105815 >> 172.21.15.97:0/1760105815 conn(0x7f67880ff300 msgr2=0x7f6788101720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:10.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.538+0000 7f678da00640 1 -- 172.21.15.97:0/1760105815 shutdown_connections 2024-03-27T10:57:10.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.538+0000 7f678da00640 1 -- 172.21.15.97:0/1760105815 wait complete. 2024-03-27T10:57:10.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.539+0000 7f678da00640 1 Processor -- start 2024-03-27T10:57:10.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.539+0000 7f678da00640 1 -- start start 2024-03-27T10:57:10.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.540+0000 7f678da00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6788076510 0x7f678819e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:10.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.540+0000 7f678da00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6788077750 0x7f678819e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:10.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.540+0000 7f678da00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67880780f0 0x7f67881a5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:10.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.540+0000 7f678da00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6788113cb0 con 0x7f67880780f0 2024-03-27T10:57:10.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.540+0000 7f678da00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f6788113b30 con 0x7f6788076510 2024-03-27T10:57:10.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.540+0000 7f678da00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6788113e30 con 0x7f6788077750 2024-03-27T10:57:10.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.540+0000 7f6787400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67880780f0 0x7f67881a5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:10.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.540+0000 7f6787400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67880780f0 0x7f67881a5950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34752/0 (socket says 172.21.15.97:34752) 2024-03-27T10:57:10.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.540+0000 7f6787400640 1 -- 172.21.15.97:0/3617802387 learned_addr learned my addr 172.21.15.97:0/3617802387 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:10.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.540+0000 7f6786a00640 1 --2- 172.21.15.97:0/3617802387 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6788076510 0x7f678819e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:10.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.540+0000 7f6786000640 1 --2- 172.21.15.97:0/3617802387 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6788077750 0x7f678819e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:10.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.540+0000 7f6787400640 1 -- 172.21.15.97:0/3617802387 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6788077750 msgr2=0x7f678819e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:10.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.540+0000 7f6787400640 1 --2- 172.21.15.97:0/3617802387 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6788077750 0x7f678819e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:10.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.540+0000 7f6787400640 1 -- 172.21.15.97:0/3617802387 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6788076510 msgr2=0x7f678819e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:10.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.540+0000 7f6787400640 1 --2- 172.21.15.97:0/3617802387 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6788076510 0x7f678819e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:10.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.540+0000 7f6787400640 1 -- 172.21.15.97:0/3617802387 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67881a5fc0 con 0x7f67880780f0 2024-03-27T10:57:10.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.541+0000 7f6786a00640 1 --2- 172.21.15.97:0/3617802387 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6788076510 0x7f678819e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:57:10.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.541+0000 7f6787400640 1 --2- 172.21.15.97:0/3617802387 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67880780f0 0x7f67881a5950 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f678000d750 tx=0x7f678000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:10.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.541+0000 7f677f400640 1 -- 172.21.15.97:0/3617802387 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6780017070 con 0x7f67880780f0 2024-03-27T10:57:10.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.541+0000 7f677f400640 1 -- 172.21.15.97:0/3617802387 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f67800054d0 con 0x7f67880780f0 2024-03-27T10:57:10.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.541+0000 7f678da00640 1 -- 172.21.15.97:0/3617802387 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67881a62b0 con 0x7f67880780f0 2024-03-27T10:57:10.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.541+0000 7f677f400640 1 -- 172.21.15.97:0/3617802387 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6780004c60 con 0x7f67880780f0 2024-03-27T10:57:10.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.541+0000 7f678da00640 1 -- 172.21.15.97:0/3617802387 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67881a6800 con 0x7f67880780f0 2024-03-27T10:57:10.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.545+0000 7f679064f740 1 Processor -- start 2024-03-27T10:57:10.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.545+0000 7f677f400640 1 -- 172.21.15.97:0/3617802387 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f6780004e00 con 0x7f67880780f0 2024-03-27T10:57:10.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.545+0000 7f679064f740 1 -- start start 2024-03-27T10:57:10.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.545+0000 7f679064f740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556397077f00 0x556397078320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:10.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.545+0000 7f679064f740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5563970790d0 0x556397079550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:10.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.545+0000 7f679064f740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556397079a90 0x556397080320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:10.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.545+0000 7f679064f740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x556396fdd280 con 0x5563970790d0 2024-03-27T10:57:10.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.545+0000 7f679064f740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x556396f856f0 con 0x556397077f00 2024-03-27T10:57:10.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.545+0000 7f679064f740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x556396fd9230 con 0x556397079a90 2024-03-27T10:57:10.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.546+0000 7f6786a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5563970790d0 0x556397079550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:10.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.546+0000 7f6786a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5563970790d0 0x556397079550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34756/0 (socket says 172.21.15.97:34756) 2024-03-27T10:57:10.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.546+0000 7f6786a00640 1 -- 172.21.15.97:0/2231020509 learned_addr learned my addr 172.21.15.97:0/2231020509 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:10.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.546+0000 7f6786000640 1 --2- 172.21.15.97:0/2231020509 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556397077f00 0x556397078320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:10.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.546+0000 7f6786000640 1 --2- 172.21.15.97:0/2231020509 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556397079a90 0x556397080320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:10.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.546+0000 7f6786a00640 1 -- 172.21.15.97:0/2231020509 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556397079a90 msgr2=0x556397080320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:10.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.546+0000 7f6786a00640 1 --2- 172.21.15.97:0/2231020509 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556397079a90 0x556397080320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:10.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.546+0000 7f6786a00640 1 -- 172.21.15.97:0/2231020509 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556397077f00 msgr2=0x556397078320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:10.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.546+0000 7f6786a00640 1 --2- 172.21.15.97:0/2231020509 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556397077f00 0x556397078320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:10.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.546+0000 7f6786a00640 1 -- 172.21.15.97:0/2231020509 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556396db97e0 con 0x5563970790d0 2024-03-27T10:57:10.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.546+0000 7f6786000640 1 --2- 172.21.15.97:0/2231020509 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556397077f00 0x556397078320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T10:57:10.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.546+0000 7f677f400640 1 --2- 172.21.15.97:0/3617802387 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f676c0a3610 0x7f676c0a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:10.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.547+0000 7f677f400640 1 -- 172.21.15.97:0/3617802387 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f67800cf740 con 0x7f67880780f0 2024-03-27T10:57:10.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.547+0000 7f6786000640 1 --2- 172.21.15.97:0/3617802387 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f676c0a3610 0x7f676c0a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:10.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.547+0000 7f6786a00640 1 --2- 172.21.15.97:0/2231020509 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5563970790d0 0x556397079550 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f6778005e30 tx=0x7f6778005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6bd04d1cc1e4776 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:10.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.547+0000 7f6777e00640 1 -- 172.21.15.97:0/2231020509 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6778039050 con 0x5563970790d0 2024-03-27T10:57:10.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.547+0000 7f6777e00640 1 -- 172.21.15.97:0/2231020509 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6778045070 con 0x5563970790d0 2024-03-27T10:57:10.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.547+0000 7f6777e00640 1 -- 172.21.15.97:0/2231020509 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6778004280 con 0x5563970790d0 2024-03-27T10:57:10.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.548+0000 7f679064f740 1 -- 172.21.15.97:0/2231020509 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5563970790d0 msgr2=0x556397079550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:10.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.548+0000 7f679064f740 1 --2- 172.21.15.97:0/2231020509 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5563970790d0 0x556397079550 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f6778005e30 tx=0x7f6778005fd0 comp rx=0 tx=0).stop 2024-03-27T10:57:10.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.548+0000 7f6786000640 1 --2- 172.21.15.97:0/3617802387 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f676c0a3610 0x7f676c0a5ad0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f6770005d00 tx=0x7f677000e3d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:10.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.548+0000 7f679064f740 1 -- 172.21.15.97:0/2231020509 shutdown_connections 2024-03-27T10:57:10.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.548+0000 7f679064f740 1 --2- 172.21.15.97:0/2231020509 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556397079a90 0x556397080320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:10.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.548+0000 7f679064f740 1 --2- 172.21.15.97:0/2231020509 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5563970790d0 0x556397079550 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:10.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.548+0000 7f679064f740 1 --2- 172.21.15.97:0/2231020509 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556397077f00 0x556397078320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:10.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.548+0000 7f679064f740 1 -- 172.21.15.97:0/2231020509 >> 172.21.15.97:0/2231020509 conn(0x5563970732f0 msgr2=0x5563970736f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:10.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.548+0000 7f679064f740 1 -- 172.21.15.97:0/2231020509 shutdown_connections 2024-03-27T10:57:10.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.548+0000 7f679064f740 1 -- 172.21.15.97:0/2231020509 wait complete. 2024-03-27T10:57:10.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.549+0000 7f679064f740 1 -- start start 2024-03-27T10:57:10.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.549+0000 7f679064f740 1 Processor -- start 2024-03-27T10:57:10.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.549+0000 7f679064f740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556397077f00 0x5563970823b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:10.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.549+0000 7f679064f740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5563970790d0 0x5563970828f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:10.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.549+0000 7f679064f740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556397079a90 0x556397089930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:10.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.549+0000 7f679064f740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x556397080a40 con 0x556397079a90 2024-03-27T10:57:10.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.549+0000 7f679064f740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x556396f98aa0 con 0x5563970790d0 2024-03-27T10:57:10.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.549+0000 7f679064f740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x556397080bc0 con 0x556397077f00 2024-03-27T10:57:10.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.549+0000 7f679064f740 10 client.0 _ll_register_callbacks cb 0x556396f98490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:57:10.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.549+0000 7f6787400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556397079a90 0x556397089930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:10.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.550+0000 7f6787400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556397079a90 0x556397089930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34772/0 (socket says 172.21.15.97:34772) 2024-03-27T10:57:10.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.550+0000 7f6787400640 1 -- 172.21.15.97:0/425874058 learned_addr learned my addr 172.21.15.97:0/425874058 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:10.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.550+0000 7f6786000640 1 --2- 172.21.15.97:0/425874058 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5563970790d0 0x5563970828f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:10.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.550+0000 7f6786a00640 1 --2- 172.21.15.97:0/425874058 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556397077f00 0x5563970823b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:10.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.550+0000 7f6787400640 1 -- 172.21.15.97:0/425874058 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556397077f00 msgr2=0x5563970823b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:10.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.550+0000 7f6787400640 1 --2- 172.21.15.97:0/425874058 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556397077f00 0x5563970823b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:10.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.550+0000 7f6787400640 1 -- 172.21.15.97:0/425874058 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5563970790d0 msgr2=0x5563970828f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:10.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.550+0000 7f6787400640 1 --2- 172.21.15.97:0/425874058 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5563970790d0 0x5563970828f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:10.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.550+0000 7f6787400640 1 -- 172.21.15.97:0/425874058 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556396e19f70 con 0x556397079a90 2024-03-27T10:57:10.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.550+0000 7f6787400640 1 --2- 172.21.15.97:0/425874058 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556397079a90 0x556397089930 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f67800a4300 tx=0x7f6780005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be2e9970d1fffdef server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:10.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.550+0000 7f6774c00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T10:57:10.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.550+0000 7f6774c00640 1 -- 172.21.15.97:0/425874058 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6780020070 con 0x556397079a90 2024-03-27T10:57:10.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.550+0000 7f679064f740 1 -- 172.21.15.97:0/425874058 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6778045070 con 0x556397079a90 2024-03-27T10:57:10.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.550+0000 7f6774c00640 1 -- 172.21.15.97:0/425874058 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f6780098230 con 0x556397079a90 2024-03-27T10:57:10.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.551+0000 7f6774c00640 1 -- 172.21.15.97:0/425874058 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f678009ff70 con 0x556397079a90 2024-03-27T10:57:10.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.551+0000 7f679064f740 10 client.15672 fetch_fsmap learned FSMap version 28 2024-03-27T10:57:10.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.551+0000 7f679064f740 1 -- 172.21.15.97:0/425874058 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556396cb1170 con 0x556397079a90 2024-03-27T10:57:10.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.551+0000 7f6774c00640 1 -- 172.21.15.97:0/425874058 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6778045070 con 0x556397079a90 2024-03-27T10:57:10.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.551+0000 7f6774c00640 1 -- 172.21.15.97:0/425874058 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f67800bab90 con 0x556397079a90 2024-03-27T10:57:10.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.551+0000 7f679064f740 10 client.15672 fetch_fsmap finished waiting for FSMap version 28 2024-03-27T10:57:10.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.551+0000 7f679064f740 10 client.15672 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-27T10:57:10.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.551+0000 7f679064f740 20 client.15672 populate_metadata read hostname 'smithi097' 2024-03-27T10:57:10.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.551+0000 7f679064f740 1 --2- 172.21.15.97:0/425874058 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55639708e3b0 0x556397090830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:10.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.551+0000 7f679064f740 4 client.15672 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T10:57:10.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.551+0000 7f679064f740 1 -- 172.21.15.97:0/425874058 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556396fea7c0 con 0x55639708e3b0 2024-03-27T10:57:10.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.552+0000 7f6786a00640 1 --2- 172.21.15.97:0/425874058 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55639708e3b0 0x556397090830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:10.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.553+0000 7f6786a00640 1 --2- 172.21.15.97:0/425874058 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55639708e3b0 0x556397090830 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d6c7614c1c8a9f8 server_cookie=997b2e0e6632388 in_seq=0 out_seq=0 2024-03-27T10:57:10.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.553+0000 7f6774c00640 10 client.15672 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T10:57:10.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.554+0000 7f6774c00640 1 -- 172.21.15.97:0/425874058 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x556396fea7c0 con 0x55639708e3b0 2024-03-27T10:57:10.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.554+0000 7f6774c00640 10 client.15672 handle_command_reply: tid=0 2024-03-27T10:57:10.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.555+0000 7f679064f740 1 client.15672 shutdown 2024-03-27T10:57:10.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.555+0000 7f679064f740 1 -- 172.21.15.97:0/425874058 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556397079a90 msgr2=0x556397089930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:10.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.555+0000 7f679064f740 1 --2- 172.21.15.97:0/425874058 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556397079a90 0x556397089930 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f67800a4300 tx=0x7f6780005740 comp rx=0 tx=0).stop 2024-03-27T10:57:10.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.555+0000 7f679064f740 1 -- 172.21.15.97:0/425874058 shutdown_connections 2024-03-27T10:57:10.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.555+0000 7f679064f740 1 --2- 172.21.15.97:0/425874058 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55639708e3b0 0x556397090830 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:10.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.555+0000 7f679064f740 1 --2- 172.21.15.97:0/425874058 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556397079a90 0x556397089930 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:10.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.555+0000 7f679064f740 1 --2- 172.21.15.97:0/425874058 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5563970790d0 0x5563970828f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:10.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.555+0000 7f679064f740 1 --2- 172.21.15.97:0/425874058 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556397077f00 0x5563970823b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:10.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.555+0000 7f6774c00640 0 client.15672 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T10:57:10.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.555+0000 7f679064f740 1 -- 172.21.15.97:0/425874058 >> 172.21.15.97:0/425874058 conn(0x5563970732f0 msgr2=0x556397072550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:10.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.555+0000 7f679064f740 1 -- 172.21.15.97:0/425874058 shutdown_connections 2024-03-27T10:57:10.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.555+0000 7f679064f740 1 -- 172.21.15.97:0/425874058 wait complete. 2024-03-27T10:57:10.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.555+0000 7f679064f740 20 client.15672 trim_cache size 0 max 16384 2024-03-27T10:57:10.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.574+0000 7f679064f740 1 Processor -- start 2024-03-27T10:57:10.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.574+0000 7f679064f740 1 -- start start 2024-03-27T10:57:10.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.574+0000 7f679064f740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556397077f00 0x556397076c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:10.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.574+0000 7f679064f740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5563970790d0 0x5563970898f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:10.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.574+0000 7f679064f740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556397079a90 0x5563970950f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:10.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.574+0000 7f679064f740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f67881af570 con 0x556397077f00 2024-03-27T10:57:10.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.574+0000 7f679064f740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f67881b10b0 con 0x556397079a90 2024-03-27T10:57:10.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.574+0000 7f679064f740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f67881afc40 con 0x5563970790d0 2024-03-27T10:57:10.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.575+0000 7f6786a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556397077f00 0x556397076c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:10.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.575+0000 7f6786a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556397077f00 0x556397076c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34774/0 (socket says 172.21.15.97:34774) 2024-03-27T10:57:10.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.575+0000 7f6786a00640 1 -- 172.21.15.97:0/2118980278 learned_addr learned my addr 172.21.15.97:0/2118980278 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:10.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.575+0000 7f6787400640 1 --2- 172.21.15.97:0/2118980278 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556397079a90 0x5563970950f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:10.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.575+0000 7f6786000640 1 --2- 172.21.15.97:0/2118980278 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5563970790d0 0x5563970898f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:10.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.575+0000 7f6786a00640 1 -- 172.21.15.97:0/2118980278 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5563970790d0 msgr2=0x5563970898f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:10.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.575+0000 7f6786a00640 1 --2- 172.21.15.97:0/2118980278 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5563970790d0 0x5563970898f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:10.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.575+0000 7f6786a00640 1 -- 172.21.15.97:0/2118980278 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556397079a90 msgr2=0x5563970950f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:10.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.575+0000 7f6786a00640 1 --2- 172.21.15.97:0/2118980278 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556397079a90 0x5563970950f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:10.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.575+0000 7f6786a00640 1 -- 172.21.15.97:0/2118980278 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556397078b50 con 0x556397077f00 2024-03-27T10:57:10.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.575+0000 7f6787400640 1 --2- 172.21.15.97:0/2118980278 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556397079a90 0x5563970950f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T10:57:10.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.576+0000 7f6786a00640 1 --2- 172.21.15.97:0/2118980278 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556397077f00 0x556397076c60 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f6778009ca0 tx=0x7f6778036040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b4fb6a50f828e8b server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:10.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.576+0000 7f6777400640 1 -- 172.21.15.97:0/2118980278 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6778039040 con 0x556397077f00 2024-03-27T10:57:10.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.576+0000 7f6777400640 1 -- 172.21.15.97:0/2118980278 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6778045040 con 0x556397077f00 2024-03-27T10:57:10.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.576+0000 7f6777400640 1 -- 172.21.15.97:0/2118980278 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6778052350 con 0x556397077f00 2024-03-27T10:57:10.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.577+0000 7f679064f740 1 -- 172.21.15.97:0/2118980278 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556397077f00 msgr2=0x556397076c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:10.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.577+0000 7f679064f740 1 --2- 172.21.15.97:0/2118980278 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556397077f00 0x556397076c60 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f6778009ca0 tx=0x7f6778036040 comp rx=0 tx=0).stop 2024-03-27T10:57:10.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.577+0000 7f679064f740 1 -- 172.21.15.97:0/2118980278 shutdown_connections 2024-03-27T10:57:10.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.577+0000 7f679064f740 1 --2- 172.21.15.97:0/2118980278 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556397079a90 0x5563970950f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:10.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.577+0000 7f679064f740 1 --2- 172.21.15.97:0/2118980278 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5563970790d0 0x5563970898f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:10.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.577+0000 7f679064f740 1 --2- 172.21.15.97:0/2118980278 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556397077f00 0x556397076c60 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:10.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.577+0000 7f679064f740 1 -- 172.21.15.97:0/2118980278 >> 172.21.15.97:0/2118980278 conn(0x5563970732f0 msgr2=0x556397076550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:10.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.577+0000 7f679064f740 1 -- 172.21.15.97:0/2118980278 shutdown_connections 2024-03-27T10:57:10.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.577+0000 7f679064f740 1 -- 172.21.15.97:0/2118980278 wait complete. 2024-03-27T10:57:10.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.577+0000 7f679064f740 1 -- start start 2024-03-27T10:57:10.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.577+0000 7f679064f740 1 Processor -- start 2024-03-27T10:57:10.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.578+0000 7f679064f740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556397077f00 0x5563970891e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:10.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.578+0000 7f679064f740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5563970790d0 0x556397089720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:10.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.578+0000 7f679064f740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556397079a90 0x556397087840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:10.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.578+0000 7f679064f740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f67881b09e0 con 0x556397077f00 2024-03-27T10:57:10.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.578+0000 7f679064f740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f67881b0310 con 0x556397079a90 2024-03-27T10:57:10.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.578+0000 7f679064f740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f67881ad550 con 0x5563970790d0 2024-03-27T10:57:10.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.578+0000 7f679064f740 10 client.0 _ll_register_callbacks cb 0x7f678009b7f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:57:10.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.578+0000 7f6786a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556397077f00 0x5563970891e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:10.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.578+0000 7f6786a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556397077f00 0x5563970891e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34780/0 (socket says 172.21.15.97:34780) 2024-03-27T10:57:10.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.578+0000 7f6786a00640 1 -- 172.21.15.97:0/2945171903 learned_addr learned my addr 172.21.15.97:0/2945171903 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:10.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.578+0000 7f6787400640 1 --2- 172.21.15.97:0/2945171903 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556397079a90 0x556397087840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:10.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.578+0000 7f6786a00640 1 -- 172.21.15.97:0/2945171903 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5563970790d0 msgr2=0x556397089720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:10.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.578+0000 7f6786a00640 1 --2- 172.21.15.97:0/2945171903 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5563970790d0 0x556397089720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:10.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.578+0000 7f6786a00640 1 -- 172.21.15.97:0/2945171903 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556397079a90 msgr2=0x556397087840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:10.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.578+0000 7f6786a00640 1 --2- 172.21.15.97:0/2945171903 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556397079a90 0x556397087840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:10.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.578+0000 7f6786a00640 1 -- 172.21.15.97:0/2945171903 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556397087f30 con 0x556397077f00 2024-03-27T10:57:10.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.579+0000 7f6786a00640 1 --2- 172.21.15.97:0/2945171903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556397077f00 0x5563970891e0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f6778003c10 tx=0x7f67780093f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=370669e6b00c8d84 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:10.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.579+0000 7f6774c00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T10:57:10.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.579+0000 7f6774c00640 1 -- 172.21.15.97:0/2945171903 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f677805e070 con 0x556397077f00 2024-03-27T10:57:10.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.579+0000 7f6774c00640 1 -- 172.21.15.97:0/2945171903 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f6778052b10 con 0x556397077f00 2024-03-27T10:57:10.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.579+0000 7f679064f740 1 -- 172.21.15.97:0/2945171903 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6778045040 con 0x556397077f00 2024-03-27T10:57:10.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.579+0000 7f6774c00640 1 -- 172.21.15.97:0/2945171903 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6778039040 con 0x556397077f00 2024-03-27T10:57:10.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.579+0000 7f679064f740 10 client.15678 fetch_fsmap learned FSMap version 28 2024-03-27T10:57:10.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.579+0000 7f679064f740 1 -- 172.21.15.97:0/2945171903 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5563970f5110 con 0x556397077f00 2024-03-27T10:57:10.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.579+0000 7f6774c00640 1 -- 172.21.15.97:0/2945171903 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6778045040 con 0x556397077f00 2024-03-27T10:57:10.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.580+0000 7f6774c00640 1 -- 172.21.15.97:0/2945171903 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f6778061020 con 0x556397077f00 2024-03-27T10:57:10.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.580+0000 7f679064f740 10 client.15678 fetch_fsmap finished waiting for FSMap version 28 2024-03-27T10:57:10.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.580+0000 7f679064f740 10 client.15678 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-27T10:57:10.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.580+0000 7f679064f740 20 client.15678 populate_metadata read hostname 'smithi097' 2024-03-27T10:57:10.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.580+0000 7f679064f740 1 --2- 172.21.15.97:0/2945171903 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5563970f56d0 0x5563970f7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:10.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.580+0000 7f679064f740 4 client.15678 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-27T10:57:10.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.580+0000 7f679064f740 1 -- 172.21.15.97:0/2945171903 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5563970821a0 con 0x5563970f56d0 2024-03-27T10:57:10.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.580+0000 7f6786000640 1 --2- 172.21.15.97:0/2945171903 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5563970f56d0 0x5563970f7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:10.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.581+0000 7f6774c00640 10 client.15678 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T10:57:10.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.581+0000 7f6786000640 1 --2- 172.21.15.97:0/2945171903 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5563970f56d0 0x5563970f7ac0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55a08e88695a6b63 server_cookie=4382d23f756be63 in_seq=0 out_seq=0 2024-03-27T10:57:10.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.583+0000 7f6774c00640 1 -- 172.21.15.97:0/2945171903 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5563970821a0 con 0x5563970f56d0 2024-03-27T10:57:10.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.583+0000 7f6774c00640 10 client.15678 handle_command_reply: tid=0 2024-03-27T10:57:10.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.583+0000 7f679064f740 1 client.15678 shutdown 2024-03-27T10:57:10.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.584+0000 7f679064f740 1 -- 172.21.15.97:0/2945171903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556397077f00 msgr2=0x5563970891e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:10.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.584+0000 7f679064f740 1 --2- 172.21.15.97:0/2945171903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556397077f00 0x5563970891e0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f6778003c10 tx=0x7f67780093f0 comp rx=0 tx=0).stop 2024-03-27T10:57:10.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.584+0000 7f679064f740 1 -- 172.21.15.97:0/2945171903 shutdown_connections 2024-03-27T10:57:10.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.584+0000 7f679064f740 1 --2- 172.21.15.97:0/2945171903 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5563970f56d0 0x5563970f7ac0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:10.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.584+0000 7f679064f740 1 --2- 172.21.15.97:0/2945171903 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556397079a90 0x556397087840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:10.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.584+0000 7f679064f740 1 --2- 172.21.15.97:0/2945171903 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5563970790d0 0x556397089720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:10.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.584+0000 7f679064f740 1 --2- 172.21.15.97:0/2945171903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556397077f00 0x5563970891e0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:10.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.584+0000 7f679064f740 1 -- 172.21.15.97:0/2945171903 >> 172.21.15.97:0/2945171903 conn(0x5563970732f0 msgr2=0x556397076de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:10.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.584+0000 7f6774c00640 0 client.15678 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T10:57:10.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.584+0000 7f679064f740 1 -- 172.21.15.97:0/2945171903 shutdown_connections 2024-03-27T10:57:10.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.584+0000 7f679064f740 1 -- 172.21.15.97:0/2945171903 wait complete. 2024-03-27T10:57:10.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.584+0000 7f679064f740 20 client.15678 trim_cache size 0 max 16384 2024-03-27T10:57:10.559 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-03-27T10:57:10.559 INFO:teuthology.orchestra.run.smithi097.stdout: "return_code": 0, 2024-03-27T10:57:10.559 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_tag": "68fe59c5-d2da-4677-8403-4ebd300cecb8", 2024-03-27T10:57:10.559 INFO:teuthology.orchestra.run.smithi097.stdout: "mode": "asynchronous" 2024-03-27T10:57:10.559 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-03-27T10:57:10.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.585+0000 7f678da00640 1 -- 172.21.15.97:0/3617802387 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f676c0a3610 msgr2=0x7f676c0a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:10.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.585+0000 7f678da00640 1 --2- 172.21.15.97:0/3617802387 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f676c0a3610 0x7f676c0a5ad0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f6770005d00 tx=0x7f677000e3d0 comp rx=0 tx=0).stop 2024-03-27T10:57:10.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.585+0000 7f678da00640 1 -- 172.21.15.97:0/3617802387 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67880780f0 msgr2=0x7f67881a5950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:10.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.585+0000 7f678da00640 1 --2- 172.21.15.97:0/3617802387 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67880780f0 0x7f67881a5950 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f678000d750 tx=0x7f678000dc10 comp rx=0 tx=0).stop 2024-03-27T10:57:10.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.585+0000 7f678da00640 1 -- 172.21.15.97:0/3617802387 shutdown_connections 2024-03-27T10:57:10.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.585+0000 7f678da00640 1 --2- 172.21.15.97:0/3617802387 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67880780f0 0x7f67881a5950 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:10.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.585+0000 7f678da00640 1 --2- 172.21.15.97:0/3617802387 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6788077750 0x7f678819e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:10.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.585+0000 7f678da00640 1 --2- 172.21.15.97:0/3617802387 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f676c0a3610 0x7f676c0a5ad0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:10.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.585+0000 7f678da00640 1 --2- 172.21.15.97:0/3617802387 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6788076510 0x7f678819e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:10.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.585+0000 7f678da00640 1 -- 172.21.15.97:0/3617802387 >> 172.21.15.97:0/3617802387 conn(0x7f67880ff300 msgr2=0x7f6788100e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:10.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.585+0000 7f678da00640 1 -- 172.21.15.97:0/3617802387 shutdown_connections 2024-03-27T10:57:10.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.585+0000 7f678da00640 1 -- 172.21.15.97:0/3617802387 wait complete. 2024-03-27T10:57:10.575 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-27T10:57:10.575 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-27T10:57:10.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.807+0000 7f76c0600640 1 -- 172.21.15.97:0/2914247786 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76b80780f0 msgr2=0x7f76b8078990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:10.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.807+0000 7f76c0600640 1 --2- 172.21.15.97:0/2914247786 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76b80780f0 0x7f76b8078990 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f76b0009ca0 tx=0x7f76b002c100 comp rx=0 tx=0).stop 2024-03-27T10:57:10.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.807+0000 7f76c0600640 1 -- 172.21.15.97:0/2914247786 shutdown_connections 2024-03-27T10:57:10.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.807+0000 7f76c0600640 1 --2- 172.21.15.97:0/2914247786 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76b80780f0 0x7f76b8078990 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:10.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.807+0000 7f76c0600640 1 --2- 172.21.15.97:0/2914247786 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f76b8077750 0x7f76b8077bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:10.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.807+0000 7f76c0600640 1 --2- 172.21.15.97:0/2914247786 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f76b8076510 0x7f76b8076930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:10.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.807+0000 7f76c0600640 1 -- 172.21.15.97:0/2914247786 >> 172.21.15.97:0/2914247786 conn(0x7f76b80ff300 msgr2=0x7f76b8101720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:10.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.807+0000 7f76c0600640 1 -- 172.21.15.97:0/2914247786 shutdown_connections 2024-03-27T10:57:10.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.807+0000 7f76c0600640 1 -- 172.21.15.97:0/2914247786 wait complete. 2024-03-27T10:57:10.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.808+0000 7f76c0600640 1 Processor -- start 2024-03-27T10:57:10.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.808+0000 7f76c0600640 1 -- start start 2024-03-27T10:57:10.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.808+0000 7f76c0600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76b8076510 0x7f76b819e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:10.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.808+0000 7f76c0600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f76b8077750 0x7f76b819e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:10.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.808+0000 7f76c0600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f76b80780f0 0x7f76b81a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:10.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.808+0000 7f76c0600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f76b8113cb0 con 0x7f76b8076510 2024-03-27T10:57:10.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.808+0000 7f76c0600640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f76b8113b30 con 0x7f76b80780f0 2024-03-27T10:57:10.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.808+0000 7f76c0600640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f76b8113e30 con 0x7f76b8077750 2024-03-27T10:57:10.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.809+0000 7f76bdc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76b8076510 0x7f76b819e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:10.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.809+0000 7f76bdc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76b8076510 0x7f76b819e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34802/0 (socket says 172.21.15.97:34802) 2024-03-27T10:57:10.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.809+0000 7f76bdc00640 1 -- 172.21.15.97:0/3426412475 learned_addr learned my addr 172.21.15.97:0/3426412475 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:10.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.809+0000 7f76be600640 1 --2- 172.21.15.97:0/3426412475 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f76b80780f0 0x7f76b81a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:10.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.809+0000 7f76bd200640 1 --2- 172.21.15.97:0/3426412475 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f76b8077750 0x7f76b819e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:10.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.809+0000 7f76bdc00640 1 -- 172.21.15.97:0/3426412475 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f76b8077750 msgr2=0x7f76b819e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:10.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.809+0000 7f76bdc00640 1 --2- 172.21.15.97:0/3426412475 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f76b8077750 0x7f76b819e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:10.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.809+0000 7f76bdc00640 1 -- 172.21.15.97:0/3426412475 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f76b80780f0 msgr2=0x7f76b81a5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:10.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.809+0000 7f76bdc00640 1 --2- 172.21.15.97:0/3426412475 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f76b80780f0 0x7f76b81a5920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:10.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.809+0000 7f76bdc00640 1 -- 172.21.15.97:0/3426412475 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76b81a5f90 con 0x7f76b8076510 2024-03-27T10:57:10.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.809+0000 7f76bdc00640 1 --2- 172.21.15.97:0/3426412475 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76b8076510 0x7f76b819e370 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f76ac00c970 tx=0x7f76ac00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:10.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.810+0000 7f76b6000640 1 -- 172.21.15.97:0/3426412475 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76ac007bb0 con 0x7f76b8076510 2024-03-27T10:57:10.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.810+0000 7f76b6000640 1 -- 172.21.15.97:0/3426412475 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f76ac007d50 con 0x7f76b8076510 2024-03-27T10:57:10.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.810+0000 7f76c0600640 1 -- 172.21.15.97:0/3426412475 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76b81a6280 con 0x7f76b8076510 2024-03-27T10:57:10.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.810+0000 7f76b6000640 1 -- 172.21.15.97:0/3426412475 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76ac00f450 con 0x7f76b8076510 2024-03-27T10:57:10.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.810+0000 7f76c0600640 1 -- 172.21.15.97:0/3426412475 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76b81a67d0 con 0x7f76b8076510 2024-03-27T10:57:10.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.813+0000 7f76c338f740 1 Processor -- start 2024-03-27T10:57:10.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.813+0000 7f76c338f740 1 -- start start 2024-03-27T10:57:10.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.813+0000 7f76b6000640 1 -- 172.21.15.97:0/3426412475 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f76ac004f20 con 0x7f76b8076510 2024-03-27T10:57:10.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.813+0000 7f76c338f740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc9ef884b0 0x55cc9ef888d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:10.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.813+0000 7f76c338f740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cc9ef88ea0 0x55cc9ef8d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:10.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.813+0000 7f76c338f740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cc9ef8e010 0x55cc9ef904a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:10.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.813+0000 7f76c338f740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55cc9eeee6b0 con 0x55cc9ef884b0 2024-03-27T10:57:10.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.813+0000 7f76c338f740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55cc9ee96750 con 0x55cc9ef8e010 2024-03-27T10:57:10.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.813+0000 7f76c338f740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55cc9eeea660 con 0x55cc9ef88ea0 2024-03-27T10:57:10.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.814+0000 7f76be600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc9ef884b0 0x55cc9ef888d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:10.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.814+0000 7f76be600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc9ef884b0 0x55cc9ef888d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34816/0 (socket says 172.21.15.97:34816) 2024-03-27T10:57:10.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.814+0000 7f76be600640 1 -- 172.21.15.97:0/1217919177 learned_addr learned my addr 172.21.15.97:0/1217919177 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:10.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.814+0000 7f76bd200640 1 --2- 172.21.15.97:0/1217919177 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cc9ef8e010 0x55cc9ef904a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:10.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.814+0000 7f76bdc00640 1 --2- 172.21.15.97:0/1217919177 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cc9ef88ea0 0x55cc9ef8d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:10.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.814+0000 7f76be600640 1 -- 172.21.15.97:0/1217919177 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cc9ef88ea0 msgr2=0x55cc9ef8d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:10.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.814+0000 7f76be600640 1 --2- 172.21.15.97:0/1217919177 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cc9ef88ea0 0x55cc9ef8d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:10.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.814+0000 7f76be600640 1 -- 172.21.15.97:0/1217919177 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cc9ef8e010 msgr2=0x55cc9ef904a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:10.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.814+0000 7f76be600640 1 --2- 172.21.15.97:0/1217919177 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cc9ef8e010 0x55cc9ef904a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:10.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.814+0000 7f76be600640 1 -- 172.21.15.97:0/1217919177 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc9eccc7d0 con 0x55cc9ef884b0 2024-03-27T10:57:10.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.814+0000 7f76bd200640 1 --2- 172.21.15.97:0/1217919177 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cc9ef8e010 0x55cc9ef904a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T10:57:10.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.814+0000 7f76be600640 1 --2- 172.21.15.97:0/1217919177 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc9ef884b0 0x55cc9ef888d0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f76b0032380 tx=0x7f76b00098d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a22f08e7f329bcf7 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:10.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.815+0000 7f76a6a00640 1 -- 172.21.15.97:0/1217919177 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76b0043020 con 0x55cc9ef884b0 2024-03-27T10:57:10.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.815+0000 7f76a6a00640 1 -- 172.21.15.97:0/1217919177 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f76b0003970 con 0x55cc9ef884b0 2024-03-27T10:57:10.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.815+0000 7f76a6a00640 1 -- 172.21.15.97:0/1217919177 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76b0039040 con 0x55cc9ef884b0 2024-03-27T10:57:10.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.815+0000 7f76b6000640 1 --2- 172.21.15.97:0/3426412475 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f76a00a3690 0x7f76a00a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:10.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.815+0000 7f76b6000640 1 -- 172.21.15.97:0/3426412475 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f76ac0cfac0 con 0x7f76b8076510 2024-03-27T10:57:10.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.815+0000 7f76bdc00640 1 --2- 172.21.15.97:0/3426412475 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f76a00a3690 0x7f76a00a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:10.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.816+0000 7f76c338f740 1 -- 172.21.15.97:0/1217919177 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc9ef884b0 msgr2=0x55cc9ef888d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:10.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.816+0000 7f76c338f740 1 --2- 172.21.15.97:0/1217919177 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc9ef884b0 0x55cc9ef888d0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f76b0032380 tx=0x7f76b00098d0 comp rx=0 tx=0).stop 2024-03-27T10:57:10.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.816+0000 7f76c338f740 1 -- 172.21.15.97:0/1217919177 shutdown_connections 2024-03-27T10:57:10.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.816+0000 7f76c338f740 1 --2- 172.21.15.97:0/1217919177 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cc9ef8e010 0x55cc9ef904a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:10.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.816+0000 7f76c338f740 1 --2- 172.21.15.97:0/1217919177 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cc9ef88ea0 0x55cc9ef8d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:10.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.816+0000 7f76c338f740 1 --2- 172.21.15.97:0/1217919177 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc9ef884b0 0x55cc9ef888d0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:10.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.816+0000 7f76c338f740 1 -- 172.21.15.97:0/1217919177 >> 172.21.15.97:0/1217919177 conn(0x55cc9ef83a20 msgr2=0x55cc9ef83e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:10.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.816+0000 7f76bdc00640 1 --2- 172.21.15.97:0/3426412475 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f76a00a3690 0x7f76a00a5b50 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f76ac0cd940 tx=0x7f76ac004920 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:10.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.816+0000 7f76c338f740 1 -- 172.21.15.97:0/1217919177 shutdown_connections 2024-03-27T10:57:10.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.816+0000 7f76c338f740 1 -- 172.21.15.97:0/1217919177 wait complete. 2024-03-27T10:57:10.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.816+0000 7f76c338f740 1 -- start start 2024-03-27T10:57:10.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.817+0000 7f76c338f740 1 Processor -- start 2024-03-27T10:57:10.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.817+0000 7f76c338f740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cc9ef884b0 0x55cc9ef92940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:10.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.817+0000 7f76c338f740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cc9ef88ea0 0x55cc9ef92e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:10.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.817+0000 7f76c338f740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc9ef8e010 0x55cc9ef99ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:10.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.817+0000 7f76c338f740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55cc9ef90f60 con 0x55cc9ef8e010 2024-03-27T10:57:10.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.817+0000 7f76c338f740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55cc9eea9a80 con 0x55cc9ef884b0 2024-03-27T10:57:10.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.817+0000 7f76c338f740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55cc9ef910e0 con 0x55cc9ef88ea0 2024-03-27T10:57:10.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.817+0000 7f76c338f740 10 client.0 _ll_register_callbacks cb 0x55cc9eea9470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:57:10.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.817+0000 7f76be600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc9ef8e010 0x55cc9ef99ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:10.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.817+0000 7f76be600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc9ef8e010 0x55cc9ef99ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34830/0 (socket says 172.21.15.97:34830) 2024-03-27T10:57:10.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.817+0000 7f76be600640 1 -- 172.21.15.97:0/1360181915 learned_addr learned my addr 172.21.15.97:0/1360181915 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:10.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.817+0000 7f76be600640 1 --2- 172.21.15.97:0/1360181915 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cc9ef884b0 0x55cc9ef92940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:10.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.818+0000 7f76bd200640 1 --2- 172.21.15.97:0/1360181915 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cc9ef88ea0 0x55cc9ef92e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:10.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.818+0000 7f76be600640 1 -- 172.21.15.97:0/1360181915 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cc9ef88ea0 msgr2=0x55cc9ef92e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:10.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.818+0000 7f76be600640 1 --2- 172.21.15.97:0/1360181915 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cc9ef88ea0 0x55cc9ef92e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:10.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.818+0000 7f76be600640 1 -- 172.21.15.97:0/1360181915 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cc9ef884b0 msgr2=0x55cc9ef92940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:10.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.818+0000 7f76be600640 1 --2- 172.21.15.97:0/1360181915 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cc9ef884b0 0x55cc9ef92940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:10.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.818+0000 7f76be600640 1 -- 172.21.15.97:0/1360181915 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc9ec023f0 con 0x55cc9ef8e010 2024-03-27T10:57:10.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.818+0000 7f76be600640 1 --2- 172.21.15.97:0/1360181915 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc9ef8e010 0x55cc9ef99ec0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f76b0032520 tx=0x7f76b002e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57a8cd4bb7da9c7c server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:10.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.818+0000 7f769b400640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T10:57:10.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.818+0000 7f769b400640 1 -- 172.21.15.97:0/1360181915 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76b004d020 con 0x55cc9ef8e010 2024-03-27T10:57:10.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.818+0000 7f769b400640 1 -- 172.21.15.97:0/1360181915 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f76b00325c0 con 0x55cc9ef8e010 2024-03-27T10:57:10.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.819+0000 7f76c338f740 1 -- 172.21.15.97:0/1360181915 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76b0003970 con 0x55cc9ef8e010 2024-03-27T10:57:10.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.819+0000 7f769b400640 1 -- 172.21.15.97:0/1360181915 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f76b0043030 con 0x55cc9ef8e010 2024-03-27T10:57:10.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.819+0000 7f76c338f740 10 client.15690 fetch_fsmap learned FSMap version 28 2024-03-27T10:57:10.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.819+0000 7f76c338f740 1 -- 172.21.15.97:0/1360181915 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55cc9ebc20f0 con 0x55cc9ef8e010 2024-03-27T10:57:10.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.819+0000 7f769b400640 1 -- 172.21.15.97:0/1360181915 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f76b0003970 con 0x55cc9ef8e010 2024-03-27T10:57:10.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.819+0000 7f769b400640 1 -- 172.21.15.97:0/1360181915 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f76b003fc20 con 0x55cc9ef8e010 2024-03-27T10:57:10.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.819+0000 7f76c338f740 10 client.15690 fetch_fsmap finished waiting for FSMap version 28 2024-03-27T10:57:10.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.819+0000 7f76c338f740 10 client.15690 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-27T10:57:10.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.819+0000 7f76c338f740 20 client.15690 populate_metadata read hostname 'smithi097' 2024-03-27T10:57:10.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.819+0000 7f76c338f740 1 --2- 172.21.15.97:0/1360181915 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55cc9ef9e3b0 0x55cc9efa07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:10.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.819+0000 7f76c338f740 4 client.15690 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T10:57:10.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.819+0000 7f76c338f740 1 -- 172.21.15.97:0/1360181915 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cc9eefbbf0 con 0x55cc9ef9e3b0 2024-03-27T10:57:10.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.820+0000 7f76bd200640 1 --2- 172.21.15.97:0/1360181915 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55cc9ef9e3b0 0x55cc9efa07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:10.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.821+0000 7f76bd200640 1 --2- 172.21.15.97:0/1360181915 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55cc9ef9e3b0 0x55cc9efa07a0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e37327d5acb2103 server_cookie=2507ff914c17bc32 in_seq=0 out_seq=0 2024-03-27T10:57:10.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.821+0000 7f769b400640 10 client.15690 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T10:57:10.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.822+0000 7f769b400640 1 -- 172.21.15.97:0/1360181915 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55cc9eefbbf0 con 0x55cc9ef9e3b0 2024-03-27T10:57:10.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.822+0000 7f769b400640 10 client.15690 handle_command_reply: tid=0 2024-03-27T10:57:10.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.822+0000 7f76c338f740 1 client.15690 shutdown 2024-03-27T10:57:10.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.822+0000 7f76c338f740 1 -- 172.21.15.97:0/1360181915 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc9ef8e010 msgr2=0x55cc9ef99ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:10.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.822+0000 7f76c338f740 1 --2- 172.21.15.97:0/1360181915 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc9ef8e010 0x55cc9ef99ec0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f76b0032520 tx=0x7f76b002e040 comp rx=0 tx=0).stop 2024-03-27T10:57:10.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.822+0000 7f76c338f740 1 -- 172.21.15.97:0/1360181915 shutdown_connections 2024-03-27T10:57:10.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.822+0000 7f76c338f740 1 --2- 172.21.15.97:0/1360181915 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55cc9ef9e3b0 0x55cc9efa07a0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:10.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.823+0000 7f769b400640 0 client.15690 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T10:57:10.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.822+0000 7f76c338f740 1 --2- 172.21.15.97:0/1360181915 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc9ef8e010 0x55cc9ef99ec0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:10.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.823+0000 7f76c338f740 1 --2- 172.21.15.97:0/1360181915 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cc9ef88ea0 0x55cc9ef92e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:10.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.823+0000 7f76c338f740 1 --2- 172.21.15.97:0/1360181915 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cc9ef884b0 0x55cc9ef92940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:10.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.823+0000 7f76c338f740 1 -- 172.21.15.97:0/1360181915 >> 172.21.15.97:0/1360181915 conn(0x55cc9ef83a20 msgr2=0x55cc9ef82c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:10.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.823+0000 7f76c338f740 1 -- 172.21.15.97:0/1360181915 shutdown_connections 2024-03-27T10:57:10.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.823+0000 7f76c338f740 1 -- 172.21.15.97:0/1360181915 wait complete. 2024-03-27T10:57:10.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.823+0000 7f76c338f740 20 client.15690 trim_cache size 0 max 16384 2024-03-27T10:57:10.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.841+0000 7f76c338f740 1 Processor -- start 2024-03-27T10:57:10.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.842+0000 7f76c338f740 1 -- start start 2024-03-27T10:57:10.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.842+0000 7f76c338f740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc9ef884b0 0x55cc9ef87390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:10.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.842+0000 7f76c338f740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cc9ef88ea0 0x55cc9ef99de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:10.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.842+0000 7f76c338f740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cc9ef8e010 0x55cc9efa4f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:10.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.842+0000 7f76c338f740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f76b81af500 con 0x55cc9ef884b0 2024-03-27T10:57:10.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.842+0000 7f76c338f740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f76b81b1040 con 0x55cc9ef88ea0 2024-03-27T10:57:10.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.842+0000 7f76c338f740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f76b81afbd0 con 0x55cc9ef8e010 2024-03-27T10:57:10.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.842+0000 7f76be600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc9ef884b0 0x55cc9ef87390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:10.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.842+0000 7f76be600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc9ef884b0 0x55cc9ef87390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34846/0 (socket says 172.21.15.97:34846) 2024-03-27T10:57:10.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.842+0000 7f76be600640 1 -- 172.21.15.97:0/3374510885 learned_addr learned my addr 172.21.15.97:0/3374510885 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:10.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.842+0000 7f76bd200640 1 --2- 172.21.15.97:0/3374510885 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cc9ef88ea0 0x55cc9ef99de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:10.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.843+0000 7f76be600640 1 -- 172.21.15.97:0/3374510885 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cc9ef8e010 msgr2=0x55cc9efa4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:10.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.843+0000 7f76be600640 1 --2- 172.21.15.97:0/3374510885 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cc9ef8e010 0x55cc9efa4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:10.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.843+0000 7f76be600640 1 -- 172.21.15.97:0/3374510885 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cc9ef88ea0 msgr2=0x55cc9ef99de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:10.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.843+0000 7f76be600640 1 --2- 172.21.15.97:0/3374510885 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cc9ef88ea0 0x55cc9ef99de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:10.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.843+0000 7f76be600640 1 -- 172.21.15.97:0/3374510885 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc9ef880f0 con 0x55cc9ef884b0 2024-03-27T10:57:10.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.843+0000 7f76be600640 1 --2- 172.21.15.97:0/3374510885 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc9ef884b0 0x55cc9ef87390 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f76b002c640 tx=0x7f76b0032c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5ba6f924efead63 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:10.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.843+0000 7f76a6000640 1 -- 172.21.15.97:0/3374510885 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76b004d070 con 0x55cc9ef884b0 2024-03-27T10:57:10.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.843+0000 7f76a6000640 1 -- 172.21.15.97:0/3374510885 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f76b0005bd0 con 0x55cc9ef884b0 2024-03-27T10:57:10.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.843+0000 7f76a6000640 1 -- 172.21.15.97:0/3374510885 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76b0039070 con 0x55cc9ef884b0 2024-03-27T10:57:10.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.844+0000 7f76c338f740 1 -- 172.21.15.97:0/3374510885 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc9ef884b0 msgr2=0x55cc9ef87390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:10.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.844+0000 7f76c338f740 1 --2- 172.21.15.97:0/3374510885 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc9ef884b0 0x55cc9ef87390 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f76b002c640 tx=0x7f76b0032c90 comp rx=0 tx=0).stop 2024-03-27T10:57:10.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.844+0000 7f76c338f740 1 -- 172.21.15.97:0/3374510885 shutdown_connections 2024-03-27T10:57:10.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.844+0000 7f76c338f740 1 --2- 172.21.15.97:0/3374510885 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cc9ef8e010 0x55cc9efa4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:10.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.844+0000 7f76c338f740 1 --2- 172.21.15.97:0/3374510885 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cc9ef88ea0 0x55cc9ef99de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:10.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.844+0000 7f76c338f740 1 --2- 172.21.15.97:0/3374510885 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc9ef884b0 0x55cc9ef87390 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:10.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.844+0000 7f76c338f740 1 -- 172.21.15.97:0/3374510885 >> 172.21.15.97:0/3374510885 conn(0x55cc9ef83a20 msgr2=0x55cc9ef86c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:10.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.844+0000 7f76c338f740 1 -- 172.21.15.97:0/3374510885 shutdown_connections 2024-03-27T10:57:10.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.844+0000 7f76c338f740 1 -- 172.21.15.97:0/3374510885 wait complete. 2024-03-27T10:57:10.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.845+0000 7f76c338f740 1 -- start start 2024-03-27T10:57:10.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.845+0000 7f76c338f740 1 Processor -- start 2024-03-27T10:57:10.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.845+0000 7f76c338f740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cc9ef884b0 0x55cc9ef97500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:10.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.845+0000 7f76c338f740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc9ef88ea0 0x55cc9ef97a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:10.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.845+0000 7f76c338f740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cc9ef8e010 0x55cc9ef98a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:10.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.845+0000 7f76c338f740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f76b81b0970 con 0x55cc9ef88ea0 2024-03-27T10:57:10.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.845+0000 7f76c338f740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f76b81b02a0 con 0x55cc9ef8e010 2024-03-27T10:57:10.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.845+0000 7f76c338f740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f76b81ad4e0 con 0x55cc9ef884b0 2024-03-27T10:57:10.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.845+0000 7f76c338f740 10 client.0 _ll_register_callbacks cb 0x7f76b0032f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:57:10.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.845+0000 7f76bd200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc9ef88ea0 0x55cc9ef97a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:10.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.846+0000 7f76bd200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc9ef88ea0 0x55cc9ef97a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34858/0 (socket says 172.21.15.97:34858) 2024-03-27T10:57:10.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.846+0000 7f76bd200640 1 -- 172.21.15.97:0/3845631442 learned_addr learned my addr 172.21.15.97:0/3845631442 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:10.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.846+0000 7f76be600640 1 --2- 172.21.15.97:0/3845631442 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cc9ef8e010 0x55cc9ef98a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:10.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.846+0000 7f76be600640 1 --2- 172.21.15.97:0/3845631442 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cc9ef884b0 0x55cc9ef97500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:10.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.846+0000 7f76bd200640 1 -- 172.21.15.97:0/3845631442 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cc9ef884b0 msgr2=0x55cc9ef97500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:10.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.846+0000 7f76bd200640 1 --2- 172.21.15.97:0/3845631442 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cc9ef884b0 0x55cc9ef97500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:10.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.846+0000 7f76bd200640 1 -- 172.21.15.97:0/3845631442 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cc9ef8e010 msgr2=0x55cc9ef98a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:10.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.846+0000 7f76bd200640 1 --2- 172.21.15.97:0/3845631442 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cc9ef8e010 0x55cc9ef98a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:10.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.846+0000 7f76bd200640 1 -- 172.21.15.97:0/3845631442 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc9ef99040 con 0x55cc9ef88ea0 2024-03-27T10:57:10.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.846+0000 7f76be600640 1 --2- 172.21.15.97:0/3845631442 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cc9ef884b0 0x55cc9ef97500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:57:10.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.846+0000 7f76bd200640 1 --2- 172.21.15.97:0/3845631442 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc9ef88ea0 0x55cc9ef97a40 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f76b819f880 tx=0x7f76a801be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53e70cca7c56aeb2 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:10.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.846+0000 7f769b400640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T10:57:10.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.846+0000 7f769b400640 1 -- 172.21.15.97:0/3845631442 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a8012df0 con 0x55cc9ef88ea0 2024-03-27T10:57:10.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.846+0000 7f769b400640 1 -- 172.21.15.97:0/3845631442 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f76a801b290 con 0x55cc9ef88ea0 2024-03-27T10:57:10.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.846+0000 7f76c338f740 1 -- 172.21.15.97:0/3845631442 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76b0005bd0 con 0x55cc9ef88ea0 2024-03-27T10:57:10.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.847+0000 7f769b400640 1 -- 172.21.15.97:0/3845631442 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f76a801b490 con 0x55cc9ef88ea0 2024-03-27T10:57:10.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.847+0000 7f76c338f740 10 client.15696 fetch_fsmap learned FSMap version 28 2024-03-27T10:57:10.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.847+0000 7f76c338f740 1 -- 172.21.15.97:0/3845631442 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55cc9f0045b0 con 0x55cc9ef88ea0 2024-03-27T10:57:10.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.847+0000 7f769b400640 1 -- 172.21.15.97:0/3845631442 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f76b0005bd0 con 0x55cc9ef88ea0 2024-03-27T10:57:10.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.847+0000 7f769b400640 1 -- 172.21.15.97:0/3845631442 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f76a8011440 con 0x55cc9ef88ea0 2024-03-27T10:57:10.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.847+0000 7f76c338f740 10 client.15696 fetch_fsmap finished waiting for FSMap version 28 2024-03-27T10:57:10.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.847+0000 7f76c338f740 10 client.15696 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-27T10:57:10.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.847+0000 7f76c338f740 20 client.15696 populate_metadata read hostname 'smithi097' 2024-03-27T10:57:10.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.847+0000 7f76c338f740 1 --2- 172.21.15.97:0/3845631442 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55cc9f004f30 0x55cc9f007350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:10.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.847+0000 7f76c338f740 4 client.15696 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-27T10:57:10.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.847+0000 7f76c338f740 1 -- 172.21.15.97:0/3845631442 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55cc9eef5f40 con 0x55cc9f004f30 2024-03-27T10:57:10.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.848+0000 7f76be600640 1 --2- 172.21.15.97:0/3845631442 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55cc9f004f30 0x55cc9f007350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:10.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.848+0000 7f76be600640 1 --2- 172.21.15.97:0/3845631442 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55cc9f004f30 0x55cc9f007350 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba724e66632be991 server_cookie=c6dc810451c5306 in_seq=0 out_seq=0 2024-03-27T10:57:10.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.848+0000 7f769b400640 10 client.15696 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T10:57:10.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.849+0000 7f769b400640 1 -- 172.21.15.97:0/3845631442 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55cc9eef5f40 con 0x55cc9f004f30 2024-03-27T10:57:10.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.849+0000 7f769b400640 10 client.15696 handle_command_reply: tid=0 2024-03-27T10:57:10.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.849+0000 7f76c338f740 1 client.15696 shutdown 2024-03-27T10:57:10.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.849+0000 7f76c338f740 1 -- 172.21.15.97:0/3845631442 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc9ef88ea0 msgr2=0x55cc9ef97a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:10.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.849+0000 7f76c338f740 1 --2- 172.21.15.97:0/3845631442 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc9ef88ea0 0x55cc9ef97a40 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f76b819f880 tx=0x7f76a801be70 comp rx=0 tx=0).stop 2024-03-27T10:57:10.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.849+0000 7f76c338f740 1 -- 172.21.15.97:0/3845631442 shutdown_connections 2024-03-27T10:57:10.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.849+0000 7f76c338f740 1 --2- 172.21.15.97:0/3845631442 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55cc9f004f30 0x55cc9f007350 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:10.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.849+0000 7f76c338f740 1 --2- 172.21.15.97:0/3845631442 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cc9ef8e010 0x55cc9ef98a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:10.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.849+0000 7f76c338f740 1 --2- 172.21.15.97:0/3845631442 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc9ef88ea0 0x55cc9ef97a40 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:10.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.849+0000 7f76c338f740 1 --2- 172.21.15.97:0/3845631442 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cc9ef884b0 0x55cc9ef97500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:10.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.849+0000 7f76c338f740 1 -- 172.21.15.97:0/3845631442 >> 172.21.15.97:0/3845631442 conn(0x55cc9ef83a20 msgr2=0x55cc9ef87510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:10.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.849+0000 7f769b400640 0 client.15696 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T10:57:10.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.849+0000 7f76c338f740 1 -- 172.21.15.97:0/3845631442 shutdown_connections 2024-03-27T10:57:10.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.849+0000 7f76c338f740 1 -- 172.21.15.97:0/3845631442 wait complete. 2024-03-27T10:57:10.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.849+0000 7f76c338f740 20 client.15696 trim_cache size 0 max 16384 2024-03-27T10:57:10.825 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-03-27T10:57:10.825 INFO:teuthology.orchestra.run.smithi097.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-27T10:57:10.826 INFO:teuthology.orchestra.run.smithi097.stdout: "scrubs": { 2024-03-27T10:57:10.826 INFO:teuthology.orchestra.run.smithi097.stdout: "68fe59c5-d2da-4677-8403-4ebd300cecb8": { 2024-03-27T10:57:10.826 INFO:teuthology.orchestra.run.smithi097.stdout: "path": "/", 2024-03-27T10:57:10.826 INFO:teuthology.orchestra.run.smithi097.stdout: "tag": "68fe59c5-d2da-4677-8403-4ebd300cecb8", 2024-03-27T10:57:10.826 INFO:teuthology.orchestra.run.smithi097.stdout: "options": "recursive,force" 2024-03-27T10:57:10.826 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T10:57:10.826 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T10:57:10.826 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-03-27T10:57:10.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.850+0000 7f76c0600640 1 -- 172.21.15.97:0/3426412475 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f76a00a3690 msgr2=0x7f76a00a5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:10.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.850+0000 7f76c0600640 1 --2- 172.21.15.97:0/3426412475 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f76a00a3690 0x7f76a00a5b50 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f76ac0cd940 tx=0x7f76ac004920 comp rx=0 tx=0).stop 2024-03-27T10:57:10.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.850+0000 7f76c0600640 1 -- 172.21.15.97:0/3426412475 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76b8076510 msgr2=0x7f76b819e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:10.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.850+0000 7f76c0600640 1 --2- 172.21.15.97:0/3426412475 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76b8076510 0x7f76b819e370 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f76ac00c970 tx=0x7f76ac00ce30 comp rx=0 tx=0).stop 2024-03-27T10:57:10.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.850+0000 7f76c0600640 1 -- 172.21.15.97:0/3426412475 shutdown_connections 2024-03-27T10:57:10.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.850+0000 7f76c0600640 1 --2- 172.21.15.97:0/3426412475 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f76a00a3690 0x7f76a00a5b50 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:10.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.850+0000 7f76c0600640 1 --2- 172.21.15.97:0/3426412475 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f76b80780f0 0x7f76b81a5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:10.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.850+0000 7f76c0600640 1 --2- 172.21.15.97:0/3426412475 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f76b8077750 0x7f76b819e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:10.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.850+0000 7f76c0600640 1 --2- 172.21.15.97:0/3426412475 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76b8076510 0x7f76b819e370 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:10.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.850+0000 7f76c0600640 1 -- 172.21.15.97:0/3426412475 >> 172.21.15.97:0/3426412475 conn(0x7f76b80ff300 msgr2=0x7f76b8100e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:10.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.850+0000 7f76c0600640 1 -- 172.21.15.97:0/3426412475 shutdown_connections 2024-03-27T10:57:10.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:10.850+0000 7f76c0600640 1 -- 172.21.15.97:0/3426412475 wait complete. 2024-03-27T10:57:10.839 INFO:tasks.cephfs.filesystem:scrub status for tag:68fe59c5-d2da-4677-8403-4ebd300cecb8 - {'path': '/', 'tag': '68fe59c5-d2da-4677-8403-4ebd300cecb8', 'options': 'recursive,force'} 2024-03-27T10:57:11.516 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:57:11 smithi114 ceph-mon[35338]: pgmap v457: 97 pgs: 97 active+clean; 663 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-27T10:57:11.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:57:11 smithi114 ceph-mon[35338]: scrub queued for path: / 2024-03-27T10:57:11.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:57:11 smithi114 ceph-mon[35338]: scrub summary: idle+waiting paths [/] 2024-03-27T10:57:11.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:57:11 smithi114 ceph-mon[35338]: scrub summary: active paths [/] 2024-03-27T10:57:11.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:57:11 smithi114 ceph-mon[35338]: scrub summary: idle+waiting paths [/] 2024-03-27T10:57:11.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:57:11 smithi097 ceph-mon[31859]: pgmap v457: 97 pgs: 97 active+clean; 663 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-27T10:57:11.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:57:11 smithi097 ceph-mon[31859]: scrub queued for path: / 2024-03-27T10:57:11.641 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:57:11 smithi097 ceph-mon[31859]: scrub summary: idle+waiting paths [/] 2024-03-27T10:57:11.641 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:57:11 smithi097 ceph-mon[31859]: scrub summary: active paths [/] 2024-03-27T10:57:11.641 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:57:11 smithi097 ceph-mon[31859]: scrub summary: idle+waiting paths [/] 2024-03-27T10:57:11.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:57:11 smithi177 ceph-mon[35294]: pgmap v457: 97 pgs: 97 active+clean; 663 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-27T10:57:11.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:57:11 smithi177 ceph-mon[35294]: scrub queued for path: / 2024-03-27T10:57:11.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:57:11 smithi177 ceph-mon[35294]: scrub summary: idle+waiting paths [/] 2024-03-27T10:57:11.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:57:11 smithi177 ceph-mon[35294]: scrub summary: active paths [/] 2024-03-27T10:57:11.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:57:11 smithi177 ceph-mon[35294]: scrub summary: idle+waiting paths [/] 2024-03-27T10:57:13.516 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:57:13 smithi114 ceph-mon[35338]: pgmap v458: 97 pgs: 97 active+clean; 663 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-27T10:57:13.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:57:13 smithi097 ceph-mon[31859]: pgmap v458: 97 pgs: 97 active+clean; 663 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-27T10:57:13.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:57:13 smithi177 ceph-mon[35294]: pgmap v458: 97 pgs: 97 active+clean; 663 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-27T10:57:15.516 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:57:15 smithi114 ceph-mon[35338]: pgmap v459: 97 pgs: 97 active+clean; 663 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-27T10:57:15.516 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:57:15 smithi114 ceph-mon[35338]: scrub summary: idle 2024-03-27T10:57:15.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:57:15 smithi097 ceph-mon[31859]: pgmap v459: 97 pgs: 97 active+clean; 663 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-27T10:57:15.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:57:15 smithi097 ceph-mon[31859]: scrub summary: idle 2024-03-27T10:57:15.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:57:15 smithi177 ceph-mon[35294]: pgmap v459: 97 pgs: 97 active+clean; 663 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-27T10:57:15.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:57:15 smithi177 ceph-mon[35294]: scrub summary: idle 2024-03-27T10:57:16.296 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.323+0000 7f8d6b200640 1 -- 172.21.15.97:0/400577083 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d6c103bd0 msgr2=0x7f8d6c103ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:16.296 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.323+0000 7f8d6b200640 1 --2- 172.21.15.97:0/400577083 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d6c103bd0 0x7f8d6c103ff0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f8d5c009900 tx=0x7f8d5c02bc30 comp rx=0 tx=0).stop 2024-03-27T10:57:16.296 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.323+0000 7f8d6b200640 1 -- 172.21.15.97:0/400577083 shutdown_connections 2024-03-27T10:57:16.297 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.323+0000 7f8d6b200640 1 --2- 172.21.15.97:0/400577083 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8d6c105780 0x7f8d6c10c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:16.297 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.323+0000 7f8d6b200640 1 --2- 172.21.15.97:0/400577083 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8d6c104dc0 0x7f8d6c105240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:16.297 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.323+0000 7f8d6b200640 1 --2- 172.21.15.97:0/400577083 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d6c103bd0 0x7f8d6c103ff0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:16.297 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.323+0000 7f8d6b200640 1 -- 172.21.15.97:0/400577083 >> 172.21.15.97:0/400577083 conn(0x7f8d6c0ff160 msgr2=0x7f8d6c101580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:16.297 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.323+0000 7f8d6b200640 1 -- 172.21.15.97:0/400577083 shutdown_connections 2024-03-27T10:57:16.297 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.323+0000 7f8d6b200640 1 -- 172.21.15.97:0/400577083 wait complete. 2024-03-27T10:57:16.297 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.324+0000 7f8d6b200640 1 Processor -- start 2024-03-27T10:57:16.297 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.324+0000 7f8d6b200640 1 -- start start 2024-03-27T10:57:16.299 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.325+0000 7f8d6b200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8d6c103bd0 0x7f8d6c199dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:16.299 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.325+0000 7f8d6b200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8d6c104dc0 0x7f8d6c19a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:16.299 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.325+0000 7f8d6b200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d6c105780 0x7f8d6c1a1380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:16.299 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.325+0000 7f8d6b200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8d6c10f6d0 con 0x7f8d6c105780 2024-03-27T10:57:16.299 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.325+0000 7f8d6b200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f8d6c10f550 con 0x7f8d6c104dc0 2024-03-27T10:57:16.300 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.325+0000 7f8d6b200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f8d6c10f850 con 0x7f8d6c103bd0 2024-03-27T10:57:16.300 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.325+0000 7f8d6a800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d6c105780 0x7f8d6c1a1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:16.300 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.325+0000 7f8d6a800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d6c105780 0x7f8d6c1a1380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34878/0 (socket says 172.21.15.97:34878) 2024-03-27T10:57:16.300 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.325+0000 7f8d6a800640 1 -- 172.21.15.97:0/136824068 learned_addr learned my addr 172.21.15.97:0/136824068 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:16.300 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.325+0000 7f8d69400640 1 --2- 172.21.15.97:0/136824068 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8d6c104dc0 0x7f8d6c19a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:16.300 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.325+0000 7f8d69e00640 1 --2- 172.21.15.97:0/136824068 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8d6c103bd0 0x7f8d6c199dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:16.300 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.326+0000 7f8d6a800640 1 -- 172.21.15.97:0/136824068 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8d6c103bd0 msgr2=0x7f8d6c199dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:16.300 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.326+0000 7f8d6a800640 1 --2- 172.21.15.97:0/136824068 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8d6c103bd0 0x7f8d6c199dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:16.300 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.326+0000 7f8d6a800640 1 -- 172.21.15.97:0/136824068 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8d6c104dc0 msgr2=0x7f8d6c19a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:16.300 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.326+0000 7f8d6a800640 1 --2- 172.21.15.97:0/136824068 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8d6c104dc0 0x7f8d6c19a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:16.300 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.326+0000 7f8d6a800640 1 -- 172.21.15.97:0/136824068 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d6c1a19f0 con 0x7f8d6c105780 2024-03-27T10:57:16.300 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.326+0000 7f8d69e00640 1 --2- 172.21.15.97:0/136824068 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8d6c103bd0 0x7f8d6c199dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T10:57:16.300 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.326+0000 7f8d6a800640 1 --2- 172.21.15.97:0/136824068 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d6c105780 0x7f8d6c1a1380 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f8d6400cce0 tx=0x7f8d6400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:16.301 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.326+0000 7f8d62a00640 1 -- 172.21.15.97:0/136824068 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d6400f450 con 0x7f8d6c105780 2024-03-27T10:57:16.301 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.326+0000 7f8d62a00640 1 -- 172.21.15.97:0/136824068 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8d6400faa0 con 0x7f8d6c105780 2024-03-27T10:57:16.301 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.326+0000 7f8d62a00640 1 -- 172.21.15.97:0/136824068 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d64018b70 con 0x7f8d6c105780 2024-03-27T10:57:16.301 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.326+0000 7f8d6b200640 1 -- 172.21.15.97:0/136824068 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d6c1a1ce0 con 0x7f8d6c105780 2024-03-27T10:57:16.301 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.326+0000 7f8d6b200640 1 -- 172.21.15.97:0/136824068 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d6c1a2180 con 0x7f8d6c105780 2024-03-27T10:57:16.301 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.329+0000 7f8d73a7c740 1 Processor -- start 2024-03-27T10:57:16.305 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.331+0000 7f8d62a00640 1 -- 172.21.15.97:0/136824068 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f8d64018d10 con 0x7f8d6c105780 2024-03-27T10:57:16.305 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.331+0000 7f8d73a7c740 1 -- start start 2024-03-27T10:57:16.305 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.331+0000 7f8d73a7c740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f40622cb60 0x55f40622cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:16.305 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.331+0000 7f8d73a7c740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f40622d550 0x55f406232050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:16.305 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.331+0000 7f8d73a7c740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f4062326c0 0x55f406234b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:16.305 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.331+0000 7f8d73a7c740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55f406191d10 con 0x55f4062326c0 2024-03-27T10:57:16.305 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.331+0000 7f8d73a7c740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55f40613a260 con 0x55f40622d550 2024-03-27T10:57:16.305 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.331+0000 7f8d73a7c740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55f40618dcc0 con 0x55f40622cb60 2024-03-27T10:57:16.305 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.332+0000 7f8d69400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f4062326c0 0x55f406234b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:16.305 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.332+0000 7f8d69400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f4062326c0 0x55f406234b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34884/0 (socket says 172.21.15.97:34884) 2024-03-27T10:57:16.305 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.332+0000 7f8d69400640 1 -- 172.21.15.97:0/1121875823 learned_addr learned my addr 172.21.15.97:0/1121875823 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:16.305 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.332+0000 7f8d69e00640 1 --2- 172.21.15.97:0/1121875823 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f40622d550 0x55f406232050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:16.306 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.332+0000 7f8d69400640 1 --2- 172.21.15.97:0/1121875823 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f40622cb60 0x55f40622cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:16.306 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.332+0000 7f8d69400640 1 -- 172.21.15.97:0/1121875823 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f40622cb60 msgr2=0x55f40622cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:16.306 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.332+0000 7f8d69400640 1 --2- 172.21.15.97:0/1121875823 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f40622cb60 0x55f40622cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:16.306 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.332+0000 7f8d69400640 1 -- 172.21.15.97:0/1121875823 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f40622d550 msgr2=0x55f406232050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:16.306 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.332+0000 7f8d69400640 1 --2- 172.21.15.97:0/1121875823 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f40622d550 0x55f406232050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:16.306 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.332+0000 7f8d69400640 1 -- 172.21.15.97:0/1121875823 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f406131aa0 con 0x55f4062326c0 2024-03-27T10:57:16.306 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.333+0000 7f8d62a00640 1 --2- 172.21.15.97:0/136824068 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f8d500a3680 0x7f8d500a5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:16.306 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.333+0000 7f8d69400640 1 --2- 172.21.15.97:0/1121875823 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f4062326c0 0x55f406234b50 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f8d5800bfc0 tx=0x7f8d5800ba70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59bbb5bdfe442590 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:16.306 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.333+0000 7f8d62a00640 1 -- 172.21.15.97:0/136824068 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f8d640cffa0 con 0x7f8d6c105780 2024-03-27T10:57:16.306 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.333+0000 7f8d69e00640 1 --2- 172.21.15.97:0/136824068 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f8d500a3680 0x7f8d500a5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:16.311 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.333+0000 7f8d57400640 1 -- 172.21.15.97:0/1121875823 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d58004220 con 0x55f4062326c0 2024-03-27T10:57:16.311 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.333+0000 7f8d57400640 1 -- 172.21.15.97:0/1121875823 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8d580043c0 con 0x55f4062326c0 2024-03-27T10:57:16.311 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.333+0000 7f8d57400640 1 -- 172.21.15.97:0/1121875823 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d580109f0 con 0x55f4062326c0 2024-03-27T10:57:16.311 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.334+0000 7f8d73a7c740 1 -- 172.21.15.97:0/1121875823 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f4062326c0 msgr2=0x55f406234b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:16.311 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.334+0000 7f8d69e00640 1 --2- 172.21.15.97:0/136824068 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f8d500a3680 0x7f8d500a5b40 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f8d5c006010 tx=0x7f8d5c0058b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:16.312 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.334+0000 7f8d73a7c740 1 --2- 172.21.15.97:0/1121875823 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f4062326c0 0x55f406234b50 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f8d5800bfc0 tx=0x7f8d5800ba70 comp rx=0 tx=0).stop 2024-03-27T10:57:16.312 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.334+0000 7f8d73a7c740 1 -- 172.21.15.97:0/1121875823 shutdown_connections 2024-03-27T10:57:16.312 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.334+0000 7f8d73a7c740 1 --2- 172.21.15.97:0/1121875823 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f4062326c0 0x55f406234b50 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:16.312 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.334+0000 7f8d73a7c740 1 --2- 172.21.15.97:0/1121875823 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f40622d550 0x55f406232050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:16.312 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.334+0000 7f8d73a7c740 1 --2- 172.21.15.97:0/1121875823 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f40622cb60 0x55f40622cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:16.312 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.334+0000 7f8d73a7c740 1 -- 172.21.15.97:0/1121875823 >> 172.21.15.97:0/1121875823 conn(0x55f4062280d0 msgr2=0x55f4062284d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:16.312 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.334+0000 7f8d73a7c740 1 -- 172.21.15.97:0/1121875823 shutdown_connections 2024-03-27T10:57:16.313 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.334+0000 7f8d73a7c740 1 -- 172.21.15.97:0/1121875823 wait complete. 2024-03-27T10:57:16.313 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.335+0000 7f8d73a7c740 1 -- start start 2024-03-27T10:57:16.313 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.335+0000 7f8d73a7c740 1 Processor -- start 2024-03-27T10:57:16.313 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.335+0000 7f8d73a7c740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f40622cb60 0x55f406236ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:16.313 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.335+0000 7f8d73a7c740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f40622d550 0x55f406237400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:16.313 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.335+0000 7f8d73a7c740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f4062326c0 0x55f40623e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:16.313 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.335+0000 7f8d73a7c740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55f406235670 con 0x55f40622d550 2024-03-27T10:57:16.313 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.335+0000 7f8d73a7c740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55f40614d630 con 0x55f4062326c0 2024-03-27T10:57:16.313 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.335+0000 7f8d73a7c740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55f4062357f0 con 0x55f40622cb60 2024-03-27T10:57:16.313 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.335+0000 7f8d73a7c740 10 client.0 _ll_register_callbacks cb 0x55f406152c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:57:16.314 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.335+0000 7f8d69400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f40622d550 0x55f406237400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:16.314 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.335+0000 7f8d69400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f40622d550 0x55f406237400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34896/0 (socket says 172.21.15.97:34896) 2024-03-27T10:57:16.314 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.335+0000 7f8d69400640 1 -- 172.21.15.97:0/1981497034 learned_addr learned my addr 172.21.15.97:0/1981497034 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:16.314 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.336+0000 7f8d6a800640 1 --2- 172.21.15.97:0/1981497034 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f4062326c0 0x55f40623e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:16.314 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.336+0000 7f8d69e00640 1 --2- 172.21.15.97:0/1981497034 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f40622cb60 0x55f406236ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:16.314 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.336+0000 7f8d69400640 1 -- 172.21.15.97:0/1981497034 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f40622cb60 msgr2=0x55f406236ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:16.314 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.336+0000 7f8d69400640 1 --2- 172.21.15.97:0/1981497034 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f40622cb60 0x55f406236ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:16.314 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.336+0000 7f8d69400640 1 -- 172.21.15.97:0/1981497034 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f4062326c0 msgr2=0x55f40623e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:16.314 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.336+0000 7f8d69400640 1 --2- 172.21.15.97:0/1981497034 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f4062326c0 0x55f40623e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:16.314 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.336+0000 7f8d69400640 1 -- 172.21.15.97:0/1981497034 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4061595f0 con 0x55f40622d550 2024-03-27T10:57:16.314 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.336+0000 7f8d69e00640 1 --2- 172.21.15.97:0/1981497034 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f40622cb60 0x55f406236ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:57:16.314 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.336+0000 7f8d69400640 1 --2- 172.21.15.97:0/1981497034 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f40622d550 0x55f406237400 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f8d580104f0 tx=0x7f8d58004250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b34cc36c9c33c7f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:16.315 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.336+0000 7f8d4be00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T10:57:16.315 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.336+0000 7f8d4be00640 1 -- 172.21.15.97:0/1981497034 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d58022040 con 0x55f40622d550 2024-03-27T10:57:16.315 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.336+0000 7f8d4be00640 1 -- 172.21.15.97:0/1981497034 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f8d58005240 con 0x55f40622d550 2024-03-27T10:57:16.315 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.336+0000 7f8d73a7c740 1 -- 172.21.15.97:0/1981497034 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d580043c0 con 0x55f40622d550 2024-03-27T10:57:16.315 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.337+0000 7f8d4be00640 1 -- 172.21.15.97:0/1981497034 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8d58005440 con 0x55f40622d550 2024-03-27T10:57:16.315 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.337+0000 7f8d73a7c740 10 client.15708 fetch_fsmap learned FSMap version 28 2024-03-27T10:57:16.315 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.337+0000 7f8d73a7c740 1 -- 172.21.15.97:0/1981497034 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f405ea6210 con 0x55f40622d550 2024-03-27T10:57:16.315 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.337+0000 7f8d4be00640 1 -- 172.21.15.97:0/1981497034 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8d580043c0 con 0x55f40622d550 2024-03-27T10:57:16.315 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.337+0000 7f8d4be00640 1 -- 172.21.15.97:0/1981497034 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f8d5801c070 con 0x55f40622d550 2024-03-27T10:57:16.315 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.337+0000 7f8d73a7c740 10 client.15708 fetch_fsmap finished waiting for FSMap version 28 2024-03-27T10:57:16.315 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.337+0000 7f8d73a7c740 10 client.15708 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T10:57:16.315 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.337+0000 7f8d73a7c740 20 client.15708 populate_metadata read hostname 'smithi097' 2024-03-27T10:57:16.315 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.337+0000 7f8d73a7c740 1 --2- 172.21.15.97:0/1981497034 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55f406242a80 0x55f406244e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:16.316 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.337+0000 7f8d73a7c740 4 client.15708 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T10:57:16.316 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.337+0000 7f8d73a7c740 1 -- 172.21.15.97:0/1981497034 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f40619f250 con 0x55f406242a80 2024-03-27T10:57:16.316 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.338+0000 7f8d69e00640 1 --2- 172.21.15.97:0/1981497034 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55f406242a80 0x55f406244e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:16.316 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.339+0000 7f8d69e00640 1 --2- 172.21.15.97:0/1981497034 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55f406242a80 0x55f406244e90 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b501b559324dff3b server_cookie=88a957c128dd7883 in_seq=0 out_seq=0 2024-03-27T10:57:16.316 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.339+0000 7f8d4be00640 10 client.15708 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T10:57:16.317 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.340+0000 7f8d4be00640 1 -- 172.21.15.97:0/1981497034 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55f40619f250 con 0x55f406242a80 2024-03-27T10:57:16.317 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.340+0000 7f8d4be00640 10 client.15708 handle_command_reply: tid=0 2024-03-27T10:57:16.317 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.340+0000 7f8d73a7c740 1 client.15708 shutdown 2024-03-27T10:57:16.317 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.340+0000 7f8d73a7c740 1 -- 172.21.15.97:0/1981497034 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f40622d550 msgr2=0x55f406237400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:16.318 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.340+0000 7f8d73a7c740 1 --2- 172.21.15.97:0/1981497034 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f40622d550 0x55f406237400 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f8d580104f0 tx=0x7f8d58004250 comp rx=0 tx=0).stop 2024-03-27T10:57:16.318 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.340+0000 7f8d73a7c740 1 -- 172.21.15.97:0/1981497034 shutdown_connections 2024-03-27T10:57:16.318 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.341+0000 7f8d73a7c740 1 --2- 172.21.15.97:0/1981497034 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55f406242a80 0x55f406244e90 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:16.318 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.341+0000 7f8d73a7c740 1 --2- 172.21.15.97:0/1981497034 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f4062326c0 0x55f40623e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:16.318 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.341+0000 7f8d73a7c740 1 --2- 172.21.15.97:0/1981497034 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f40622d550 0x55f406237400 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:16.318 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.341+0000 7f8d73a7c740 1 --2- 172.21.15.97:0/1981497034 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f40622cb60 0x55f406236ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:16.318 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.341+0000 7f8d4be00640 0 client.15708 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T10:57:16.318 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.341+0000 7f8d73a7c740 1 -- 172.21.15.97:0/1981497034 >> 172.21.15.97:0/1981497034 conn(0x55f4062280d0 msgr2=0x55f406227310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:16.318 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.341+0000 7f8d73a7c740 1 -- 172.21.15.97:0/1981497034 shutdown_connections 2024-03-27T10:57:16.318 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.341+0000 7f8d73a7c740 1 -- 172.21.15.97:0/1981497034 wait complete. 2024-03-27T10:57:16.319 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.341+0000 7f8d73a7c740 20 client.15708 trim_cache size 0 max 16384 2024-03-27T10:57:16.334 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.357+0000 7f8d73a7c740 1 Processor -- start 2024-03-27T10:57:16.334 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.357+0000 7f8d73a7c740 1 -- start start 2024-03-27T10:57:16.334 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.357+0000 7f8d73a7c740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f40622cb60 0x55f4062286e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:16.334 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.357+0000 7f8d73a7c740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f40622d550 0x55f40623e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:16.334 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.357+0000 7f8d73a7c740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f4062326c0 0x55f40622e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:16.334 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.357+0000 7f8d73a7c740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8d6c1aaf30 con 0x55f4062326c0 2024-03-27T10:57:16.334 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.357+0000 7f8d73a7c740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f8d6c1aca70 con 0x55f40622d550 2024-03-27T10:57:16.334 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.357+0000 7f8d73a7c740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f8d6c1ab600 con 0x55f40622cb60 2024-03-27T10:57:16.334 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.357+0000 7f8d6a800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f4062326c0 0x55f40622e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:16.335 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.357+0000 7f8d6a800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f4062326c0 0x55f40622e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34906/0 (socket says 172.21.15.97:34906) 2024-03-27T10:57:16.335 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.357+0000 7f8d6a800640 1 -- 172.21.15.97:0/2728250806 learned_addr learned my addr 172.21.15.97:0/2728250806 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:16.335 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.357+0000 7f8d69e00640 1 --2- 172.21.15.97:0/2728250806 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f40622cb60 0x55f4062286e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:16.335 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.357+0000 7f8d69400640 1 --2- 172.21.15.97:0/2728250806 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f40622d550 0x55f40623e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:16.335 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.358+0000 7f8d6a800640 1 -- 172.21.15.97:0/2728250806 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f40622cb60 msgr2=0x55f4062286e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:16.335 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.358+0000 7f8d6a800640 1 --2- 172.21.15.97:0/2728250806 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f40622cb60 0x55f4062286e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:16.335 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.358+0000 7f8d6a800640 1 -- 172.21.15.97:0/2728250806 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f40622d550 msgr2=0x55f40623e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:16.335 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.358+0000 7f8d6a800640 1 --2- 172.21.15.97:0/2728250806 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f40622d550 0x55f40623e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:16.335 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.358+0000 7f8d6a800640 1 -- 172.21.15.97:0/2728250806 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f405f72620 con 0x55f4062326c0 2024-03-27T10:57:16.335 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.358+0000 7f8d6a800640 1 --2- 172.21.15.97:0/2728250806 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f4062326c0 0x55f40622e870 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f8d640a4c60 tx=0x7f8d6409be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17aaac172d661f39 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:16.335 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.358+0000 7f8d56a00640 1 -- 172.21.15.97:0/2728250806 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d6409c520 con 0x55f4062326c0 2024-03-27T10:57:16.335 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.358+0000 7f8d56a00640 1 -- 172.21.15.97:0/2728250806 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8d640a08d0 con 0x55f4062326c0 2024-03-27T10:57:16.335 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.358+0000 7f8d56a00640 1 -- 172.21.15.97:0/2728250806 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d6409b970 con 0x55f4062326c0 2024-03-27T10:57:16.336 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.359+0000 7f8d73a7c740 1 -- 172.21.15.97:0/2728250806 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f4062326c0 msgr2=0x55f40622e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:16.336 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.359+0000 7f8d73a7c740 1 --2- 172.21.15.97:0/2728250806 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f4062326c0 0x55f40622e870 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f8d640a4c60 tx=0x7f8d6409be80 comp rx=0 tx=0).stop 2024-03-27T10:57:16.336 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.359+0000 7f8d73a7c740 1 -- 172.21.15.97:0/2728250806 shutdown_connections 2024-03-27T10:57:16.337 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.359+0000 7f8d73a7c740 1 --2- 172.21.15.97:0/2728250806 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f4062326c0 0x55f40622e870 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:16.337 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.359+0000 7f8d73a7c740 1 --2- 172.21.15.97:0/2728250806 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f40622d550 0x55f40623e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:16.337 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.359+0000 7f8d73a7c740 1 --2- 172.21.15.97:0/2728250806 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f40622cb60 0x55f4062286e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:16.337 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.359+0000 7f8d73a7c740 1 -- 172.21.15.97:0/2728250806 >> 172.21.15.97:0/2728250806 conn(0x55f4062280d0 msgr2=0x55f406236ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:16.337 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.359+0000 7f8d73a7c740 1 -- 172.21.15.97:0/2728250806 shutdown_connections 2024-03-27T10:57:16.337 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.359+0000 7f8d73a7c740 1 -- 172.21.15.97:0/2728250806 wait complete. 2024-03-27T10:57:16.337 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.360+0000 7f8d73a7c740 1 -- start start 2024-03-27T10:57:16.337 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.360+0000 7f8d73a7c740 1 Processor -- start 2024-03-27T10:57:16.337 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.360+0000 7f8d73a7c740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f40622cb60 0x55f40623b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:16.337 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.360+0000 7f8d73a7c740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f40622d550 0x55f40623ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:16.337 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.360+0000 7f8d73a7c740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f4062326c0 0x55f40623ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:16.337 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.360+0000 7f8d73a7c740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8d6c1ac3a0 con 0x55f40622cb60 2024-03-27T10:57:16.338 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.360+0000 7f8d73a7c740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f8d6c1abcd0 con 0x55f4062326c0 2024-03-27T10:57:16.338 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.360+0000 7f8d73a7c740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f8d6c1a8f10 con 0x55f40622d550 2024-03-27T10:57:16.338 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.360+0000 7f8d73a7c740 10 client.0 _ll_register_callbacks cb 0x7f8d5c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:57:16.340 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.361+0000 7f8d69e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f40622cb60 0x55f40623b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:16.340 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.361+0000 7f8d69e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f40622cb60 0x55f40623b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34916/0 (socket says 172.21.15.97:34916) 2024-03-27T10:57:16.340 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.361+0000 7f8d69e00640 1 -- 172.21.15.97:0/4214241806 learned_addr learned my addr 172.21.15.97:0/4214241806 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:16.340 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.361+0000 7f8d6a800640 1 --2- 172.21.15.97:0/4214241806 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f4062326c0 0x55f40623ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:16.340 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.361+0000 7f8d69400640 1 --2- 172.21.15.97:0/4214241806 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f40622d550 0x55f40623ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:16.340 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.361+0000 7f8d69e00640 1 -- 172.21.15.97:0/4214241806 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f40622d550 msgr2=0x55f40623ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:16.340 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.361+0000 7f8d69e00640 1 --2- 172.21.15.97:0/4214241806 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f40622d550 0x55f40623ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:16.341 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.361+0000 7f8d69e00640 1 -- 172.21.15.97:0/4214241806 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f4062326c0 msgr2=0x55f40623ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:16.341 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.361+0000 7f8d69e00640 1 --2- 172.21.15.97:0/4214241806 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f4062326c0 0x55f40623ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:16.341 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.361+0000 7f8d69e00640 1 -- 172.21.15.97:0/4214241806 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f40623d000 con 0x55f40622cb60 2024-03-27T10:57:16.341 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.361+0000 7f8d6a800640 1 --2- 172.21.15.97:0/4214241806 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f4062326c0 0x55f40623ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:57:16.341 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.361+0000 7f8d69e00640 1 --2- 172.21.15.97:0/4214241806 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f40622cb60 0x55f40623b4c0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f8d5c000c00 tx=0x7f8d5c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f9a612b625b2435 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:16.341 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.361+0000 7f8d4be00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T10:57:16.341 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.361+0000 7f8d4be00640 1 -- 172.21.15.97:0/4214241806 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d5c056070 con 0x55f40622cb60 2024-03-27T10:57:16.341 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.362+0000 7f8d4be00640 1 -- 172.21.15.97:0/4214241806 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f8d5c005040 con 0x55f40622cb60 2024-03-27T10:57:16.341 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.362+0000 7f8d73a7c740 1 -- 172.21.15.97:0/4214241806 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d640a08d0 con 0x55f40622cb60 2024-03-27T10:57:16.341 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.362+0000 7f8d4be00640 1 -- 172.21.15.97:0/4214241806 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8d5c0044e0 con 0x55f40622cb60 2024-03-27T10:57:16.341 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.362+0000 7f8d73a7c740 10 client.15714 fetch_fsmap learned FSMap version 28 2024-03-27T10:57:16.341 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.362+0000 7f8d73a7c740 1 -- 172.21.15.97:0/4214241806 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f40623e380 con 0x55f40622cb60 2024-03-27T10:57:16.341 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.362+0000 7f8d4be00640 1 -- 172.21.15.97:0/4214241806 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8d640a08d0 con 0x55f40622cb60 2024-03-27T10:57:16.342 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.362+0000 7f8d4be00640 1 -- 172.21.15.97:0/4214241806 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f8d5c0046c0 con 0x55f40622cb60 2024-03-27T10:57:16.342 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.362+0000 7f8d73a7c740 10 client.15714 fetch_fsmap finished waiting for FSMap version 28 2024-03-27T10:57:16.342 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.362+0000 7f8d73a7c740 10 client.15714 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T10:57:16.342 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.362+0000 7f8d73a7c740 20 client.15714 populate_metadata read hostname 'smithi097' 2024-03-27T10:57:16.342 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.362+0000 7f8d73a7c740 1 --2- 172.21.15.97:0/4214241806 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55f4062a91b0 0x55f4062ab5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:16.342 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.362+0000 7f8d73a7c740 4 client.15714 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-27T10:57:16.342 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.362+0000 7f8d73a7c740 1 -- 172.21.15.97:0/4214241806 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f4061995a0 con 0x55f4062a91b0 2024-03-27T10:57:16.342 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.363+0000 7f8d69400640 1 --2- 172.21.15.97:0/4214241806 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55f4062a91b0 0x55f4062ab5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:16.342 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.363+0000 7f8d69400640 1 --2- 172.21.15.97:0/4214241806 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55f4062a91b0 0x55f4062ab5d0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c4be1b1426ed98d server_cookie=fb2794a2da8433e0 in_seq=0 out_seq=0 2024-03-27T10:57:16.342 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.363+0000 7f8d4be00640 10 client.15714 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T10:57:16.343 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.371+0000 7f8d4be00640 1 -- 172.21.15.97:0/4214241806 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f4061995a0 con 0x55f4062a91b0 2024-03-27T10:57:16.343 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.371+0000 7f8d4be00640 10 client.15714 handle_command_reply: tid=0 2024-03-27T10:57:16.343 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.371+0000 7f8d73a7c740 1 client.15714 shutdown 2024-03-27T10:57:16.343 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.371+0000 7f8d73a7c740 1 -- 172.21.15.97:0/4214241806 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f40622cb60 msgr2=0x55f40623b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:16.343 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.371+0000 7f8d73a7c740 1 --2- 172.21.15.97:0/4214241806 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f40622cb60 0x55f40623b4c0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f8d5c000c00 tx=0x7f8d5c009260 comp rx=0 tx=0).stop 2024-03-27T10:57:16.344 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.371+0000 7f8d73a7c740 1 -- 172.21.15.97:0/4214241806 shutdown_connections 2024-03-27T10:57:16.344 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.371+0000 7f8d73a7c740 1 --2- 172.21.15.97:0/4214241806 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55f4062a91b0 0x55f4062ab5d0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:16.344 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.371+0000 7f8d73a7c740 1 --2- 172.21.15.97:0/4214241806 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f4062326c0 0x55f40623ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:16.344 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.371+0000 7f8d73a7c740 1 --2- 172.21.15.97:0/4214241806 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f40622d550 0x55f40623ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:16.344 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.371+0000 7f8d4be00640 0 client.15714 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T10:57:16.344 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.371+0000 7f8d73a7c740 1 --2- 172.21.15.97:0/4214241806 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f40622cb60 0x55f40623b4c0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:16.344 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.371+0000 7f8d73a7c740 1 -- 172.21.15.97:0/4214241806 >> 172.21.15.97:0/4214241806 conn(0x55f4062280d0 msgr2=0x55f406236200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:16.345 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.372+0000 7f8d73a7c740 1 -- 172.21.15.97:0/4214241806 shutdown_connections 2024-03-27T10:57:16.345 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.372+0000 7f8d73a7c740 1 -- 172.21.15.97:0/4214241806 wait complete. 2024-03-27T10:57:16.345 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.372+0000 7f8d73a7c740 20 client.15714 trim_cache size 0 max 16384 2024-03-27T10:57:16.345 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-27T10:57:16.346 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-27T10:57:16.346 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-27T10:57:16.346 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-27T10:57:16.346 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.373+0000 7f8d6b200640 1 -- 172.21.15.97:0/136824068 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f8d500a3680 msgr2=0x7f8d500a5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:16.346 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.373+0000 7f8d6b200640 1 --2- 172.21.15.97:0/136824068 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f8d500a3680 0x7f8d500a5b40 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f8d5c006010 tx=0x7f8d5c0058b0 comp rx=0 tx=0).stop 2024-03-27T10:57:16.347 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.373+0000 7f8d6b200640 1 -- 172.21.15.97:0/136824068 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d6c105780 msgr2=0x7f8d6c1a1380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:16.347 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.373+0000 7f8d6b200640 1 --2- 172.21.15.97:0/136824068 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d6c105780 0x7f8d6c1a1380 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f8d6400cce0 tx=0x7f8d6400a620 comp rx=0 tx=0).stop 2024-03-27T10:57:16.347 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.373+0000 7f8d6b200640 1 -- 172.21.15.97:0/136824068 shutdown_connections 2024-03-27T10:57:16.347 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.373+0000 7f8d6b200640 1 --2- 172.21.15.97:0/136824068 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d6c105780 0x7f8d6c1a1380 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:16.347 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.373+0000 7f8d6b200640 1 --2- 172.21.15.97:0/136824068 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f8d500a3680 0x7f8d500a5b40 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:16.347 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.373+0000 7f8d6b200640 1 --2- 172.21.15.97:0/136824068 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8d6c104dc0 0x7f8d6c19a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:16.347 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.373+0000 7f8d6b200640 1 --2- 172.21.15.97:0/136824068 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8d6c103bd0 0x7f8d6c199dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:16.347 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.373+0000 7f8d6b200640 1 -- 172.21.15.97:0/136824068 >> 172.21.15.97:0/136824068 conn(0x7f8d6c0ff160 msgr2=0x7f8d6c108130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:16.347 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.373+0000 7f8d6b200640 1 -- 172.21.15.97:0/136824068 shutdown_connections 2024-03-27T10:57:16.347 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:16.373+0000 7f8d6b200640 1 -- 172.21.15.97:0/136824068 wait complete. 2024-03-27T10:57:17.516 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:57:17 smithi114 ceph-mon[35338]: pgmap v460: 97 pgs: 97 active+clean; 663 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-27T10:57:17.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:57:17 smithi097 ceph-mon[31859]: pgmap v460: 97 pgs: 97 active+clean; 663 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-27T10:57:17.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:57:17 smithi177 ceph-mon[35294]: pgmap v460: 97 pgs: 97 active+clean; 663 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-27T10:57:19.516 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:57:19 smithi114 ceph-mon[35338]: pgmap v461: 97 pgs: 97 active+clean; 663 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-27T10:57:19.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:57:19 smithi097 ceph-mon[31859]: pgmap v461: 97 pgs: 97 active+clean; 663 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-27T10:57:19.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:57:19 smithi177 ceph-mon[35294]: pgmap v461: 97 pgs: 97 active+clean; 663 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-27T10:57:21.516 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:57:21 smithi114 ceph-mon[35338]: pgmap v462: 97 pgs: 97 active+clean; 664 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 255 B/s wr, 13 op/s 2024-03-27T10:57:21.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:57:21 smithi097 ceph-mon[31859]: pgmap v462: 97 pgs: 97 active+clean; 664 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 255 B/s wr, 13 op/s 2024-03-27T10:57:21.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:57:21 smithi177 ceph-mon[35294]: pgmap v462: 97 pgs: 97 active+clean; 664 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 255 B/s wr, 13 op/s 2024-03-27T10:57:23.516 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:57:23 smithi114 ceph-mon[35338]: pgmap v463: 97 pgs: 97 active+clean; 664 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-27T10:57:23.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:57:23 smithi097 ceph-mon[31859]: pgmap v463: 97 pgs: 97 active+clean; 664 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-27T10:57:23.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:57:23 smithi177 ceph-mon[35294]: pgmap v463: 97 pgs: 97 active+clean; 664 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-27T10:57:25.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:57:25 smithi097 ceph-mon[31859]: pgmap v464: 97 pgs: 97 active+clean; 664 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-27T10:57:25.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:57:25 smithi177 ceph-mon[35294]: pgmap v464: 97 pgs: 97 active+clean; 664 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-27T10:57:25.766 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:57:25 smithi114 ceph-mon[35338]: pgmap v464: 97 pgs: 97 active+clean; 664 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-27T10:57:27.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:57:27 smithi097 ceph-mon[31859]: pgmap v465: 97 pgs: 97 active+clean; 664 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-27T10:57:27.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:57:27 smithi177 ceph-mon[35294]: pgmap v465: 97 pgs: 97 active+clean; 664 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-27T10:57:27.766 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:57:27 smithi114 ceph-mon[35338]: pgmap v465: 97 pgs: 97 active+clean; 664 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-27T10:57:29.478 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.506+0000 7f4cff200640 1 -- 172.21.15.97:0/4091484428 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d00103570 msgr2=0x7f4d00109e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:29.478 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.506+0000 7f4cff200640 1 --2- 172.21.15.97:0/4091484428 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d00103570 0x7f4d00109e00 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f4cf8009ca0 tx=0x7f4cf802c120 comp rx=0 tx=0).stop 2024-03-27T10:57:29.478 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.507+0000 7f4cff200640 1 -- 172.21.15.97:0/4091484428 shutdown_connections 2024-03-27T10:57:29.478 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.507+0000 7f4cff200640 1 --2- 172.21.15.97:0/4091484428 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d00103570 0x7f4d00109e00 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:29.478 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.507+0000 7f4cff200640 1 --2- 172.21.15.97:0/4091484428 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4d00102bb0 0x7f4d00103030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:29.478 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.507+0000 7f4cff200640 1 --2- 172.21.15.97:0/4091484428 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4d001019c0 0x7f4d00101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:29.478 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.507+0000 7f4cff200640 1 -- 172.21.15.97:0/4091484428 >> 172.21.15.97:0/4091484428 conn(0x7f4d000fcf30 msgr2=0x7f4d000ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:29.479 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.507+0000 7f4cff200640 1 -- 172.21.15.97:0/4091484428 shutdown_connections 2024-03-27T10:57:29.479 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.507+0000 7f4cff200640 1 -- 172.21.15.97:0/4091484428 wait complete. 2024-03-27T10:57:29.481 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.508+0000 7f4cff200640 1 Processor -- start 2024-03-27T10:57:29.481 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.508+0000 7f4cff200640 1 -- start start 2024-03-27T10:57:29.481 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.508+0000 7f4cff200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4d001019c0 0x7f4d00197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:29.481 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.508+0000 7f4cff200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4d00102bb0 0x7f4d001980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:29.481 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.508+0000 7f4cff200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d00103570 0x7f4d0019f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:29.481 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.508+0000 7f4cff200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4d0010d4f0 con 0x7f4d00103570 2024-03-27T10:57:29.481 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.508+0000 7f4cff200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f4d0010d370 con 0x7f4d001019c0 2024-03-27T10:57:29.481 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.508+0000 7f4cff200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f4d0010d670 con 0x7f4d00102bb0 2024-03-27T10:57:29.482 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.508+0000 7f4cfe800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d00103570 0x7f4d0019f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:29.482 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.508+0000 7f4cfe800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d00103570 0x7f4d0019f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38164/0 (socket says 172.21.15.97:38164) 2024-03-27T10:57:29.482 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.508+0000 7f4cfe800640 1 -- 172.21.15.97:0/2974728876 learned_addr learned my addr 172.21.15.97:0/2974728876 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:29.482 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.508+0000 7f4cfde00640 1 --2- 172.21.15.97:0/2974728876 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4d001019c0 0x7f4d00197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:29.482 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.509+0000 7f4cfd400640 1 --2- 172.21.15.97:0/2974728876 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4d00102bb0 0x7f4d001980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:29.482 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.509+0000 7f4cfe800640 1 -- 172.21.15.97:0/2974728876 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4d00102bb0 msgr2=0x7f4d001980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:29.482 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.509+0000 7f4cfe800640 1 --2- 172.21.15.97:0/2974728876 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4d00102bb0 0x7f4d001980f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:29.482 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.509+0000 7f4cfe800640 1 -- 172.21.15.97:0/2974728876 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4d001019c0 msgr2=0x7f4d00197bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:29.482 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.509+0000 7f4cfe800640 1 --2- 172.21.15.97:0/2974728876 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4d001019c0 0x7f4d00197bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:29.482 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.509+0000 7f4cfe800640 1 -- 172.21.15.97:0/2974728876 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d0019f7d0 con 0x7f4d00103570 2024-03-27T10:57:29.482 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.509+0000 7f4cfe800640 1 --2- 172.21.15.97:0/2974728876 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d00103570 0x7f4d0019f160 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f4cf8009bf0 tx=0x7f4cf8034cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:29.482 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.509+0000 7f4cf6a00640 1 -- 172.21.15.97:0/2974728876 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cf8041020 con 0x7f4d00103570 2024-03-27T10:57:29.483 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.509+0000 7f4cf6a00640 1 -- 172.21.15.97:0/2974728876 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4cf80327e0 con 0x7f4d00103570 2024-03-27T10:57:29.483 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.509+0000 7f4cf6a00640 1 -- 172.21.15.97:0/2974728876 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cf8039040 con 0x7f4d00103570 2024-03-27T10:57:29.483 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.509+0000 7f4cff200640 1 -- 172.21.15.97:0/2974728876 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d0019fa60 con 0x7f4d00103570 2024-03-27T10:57:29.483 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.510+0000 7f4cff200640 1 -- 172.21.15.97:0/2974728876 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d00073f60 con 0x7f4d00103570 2024-03-27T10:57:29.484 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.513+0000 7f4cf6a00640 1 -- 172.21.15.97:0/2974728876 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f4cf8044020 con 0x7f4d00103570 2024-03-27T10:57:29.484 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.513+0000 7f4d079f0740 1 Processor -- start 2024-03-27T10:57:29.484 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.513+0000 7f4d079f0740 1 -- start start 2024-03-27T10:57:29.485 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.513+0000 7f4d079f0740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f72f2bab60 0x55f72f2baf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:29.485 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.514+0000 7f4d079f0740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f72f2bb550 0x55f72f2c0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:29.485 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.514+0000 7f4d079f0740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f72f2c06c0 0x55f72f2c2b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:29.485 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.514+0000 7f4d079f0740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55f72f21fd10 con 0x55f72f2bb550 2024-03-27T10:57:29.485 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.514+0000 7f4d079f0740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55f72f1c8260 con 0x55f72f2bab60 2024-03-27T10:57:29.485 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.514+0000 7f4d079f0740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55f72f21bcc0 con 0x55f72f2c06c0 2024-03-27T10:57:29.486 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.514+0000 7f4cfd400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f72f2bab60 0x55f72f2baf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:29.486 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.514+0000 7f4cfd400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f72f2bab60 0x55f72f2baf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:41546/0 (socket says 172.21.15.97:41546) 2024-03-27T10:57:29.486 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.514+0000 7f4cfd400640 1 -- 172.21.15.97:0/1449248189 learned_addr learned my addr 172.21.15.97:0/1449248189 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:29.486 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.514+0000 7f4cfd400640 1 --2- 172.21.15.97:0/1449248189 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f72f2c06c0 0x55f72f2c2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:29.486 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.514+0000 7f4cf6a00640 1 --2- 172.21.15.97:0/2974728876 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f4ce40a3600 0x7f4ce40a5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:29.487 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.515+0000 7f4cfd400640 1 -- 172.21.15.97:0/1449248189 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f72f2c06c0 msgr2=0x55f72f2c2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:29.487 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.515+0000 7f4cfd400640 1 --2- 172.21.15.97:0/1449248189 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f72f2c06c0 0x55f72f2c2b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:29.488 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.515+0000 7f4cfd400640 1 -- 172.21.15.97:0/1449248189 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f72f2bb550 msgr2=0x55f72f2c0050 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-27T10:57:29.488 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.515+0000 7f4cfd400640 1 --2- 172.21.15.97:0/1449248189 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f72f2bb550 0x55f72f2c0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:29.488 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.515+0000 7f4cfd400640 1 -- 172.21.15.97:0/1449248189 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f72f1bfaa0 con 0x55f72f2bab60 2024-03-27T10:57:29.488 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.515+0000 7f4cf6a00640 1 -- 172.21.15.97:0/2974728876 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f4cf80f2190 con 0x7f4d00103570 2024-03-27T10:57:29.488 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.515+0000 7f4cfe800640 1 --2- 172.21.15.97:0/2974728876 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f4ce40a3600 0x7f4ce40a5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:29.488 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.515+0000 7f4cfd400640 1 --2- 172.21.15.97:0/1449248189 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f72f2bab60 0x55f72f2baf80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4cec012710 tx=0x7f4cec009680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fcb01087a35e34a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:29.488 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.516+0000 7f4ceb400640 1 -- 172.21.15.97:0/1449248189 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cec012c90 con 0x55f72f2bab60 2024-03-27T10:57:29.488 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.516+0000 7f4ceb400640 1 -- 172.21.15.97:0/1449248189 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4cec012e30 con 0x55f72f2bab60 2024-03-27T10:57:29.488 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.516+0000 7f4ceb400640 1 -- 172.21.15.97:0/1449248189 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cec003850 con 0x55f72f2bab60 2024-03-27T10:57:29.488 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.516+0000 7f4d079f0740 1 -- 172.21.15.97:0/1449248189 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f72f2bab60 msgr2=0x55f72f2baf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:29.488 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.516+0000 7f4d079f0740 1 --2- 172.21.15.97:0/1449248189 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f72f2bab60 0x55f72f2baf80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4cec012710 tx=0x7f4cec009680 comp rx=0 tx=0).stop 2024-03-27T10:57:29.488 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.516+0000 7f4cfe800640 1 --2- 172.21.15.97:0/2974728876 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f4ce40a3600 0x7f4ce40a5ac0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f4cf80bafe0 tx=0x7f4cf80095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:29.489 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.516+0000 7f4d079f0740 1 -- 172.21.15.97:0/1449248189 shutdown_connections 2024-03-27T10:57:29.489 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.516+0000 7f4d079f0740 1 --2- 172.21.15.97:0/1449248189 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f72f2c06c0 0x55f72f2c2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:29.489 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.516+0000 7f4d079f0740 1 --2- 172.21.15.97:0/1449248189 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f72f2bb550 0x55f72f2c0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:29.489 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.516+0000 7f4d079f0740 1 --2- 172.21.15.97:0/1449248189 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f72f2bab60 0x55f72f2baf80 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:29.489 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.516+0000 7f4d079f0740 1 -- 172.21.15.97:0/1449248189 >> 172.21.15.97:0/1449248189 conn(0x55f72f2b60d0 msgr2=0x55f72f2b64d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:29.489 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.517+0000 7f4d079f0740 1 -- 172.21.15.97:0/1449248189 shutdown_connections 2024-03-27T10:57:29.489 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.517+0000 7f4d079f0740 1 -- 172.21.15.97:0/1449248189 wait complete. 2024-03-27T10:57:29.491 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.517+0000 7f4d079f0740 1 -- start start 2024-03-27T10:57:29.491 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.517+0000 7f4d079f0740 1 Processor -- start 2024-03-27T10:57:29.491 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.518+0000 7f4d079f0740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f72f2bab60 0x55f72f2c4f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:29.491 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.518+0000 7f4d079f0740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f72f2bb550 0x55f72f2c54c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:29.491 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.518+0000 7f4d079f0740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f72f2c06c0 0x55f72f2cc490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:29.491 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.518+0000 7f4d079f0740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55f72f2c3670 con 0x55f72f2bab60 2024-03-27T10:57:29.491 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.518+0000 7f4d079f0740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55f72f1db630 con 0x55f72f2c06c0 2024-03-27T10:57:29.492 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.518+0000 7f4d079f0740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55f72f2c37f0 con 0x55f72f2bb550 2024-03-27T10:57:29.492 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.518+0000 7f4d079f0740 10 client.0 _ll_register_callbacks cb 0x55f72f1e0c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:57:29.492 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.518+0000 7f4cfd400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f72f2bab60 0x55f72f2c4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:29.492 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.518+0000 7f4cfd400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f72f2bab60 0x55f72f2c4f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38184/0 (socket says 172.21.15.97:38184) 2024-03-27T10:57:29.492 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.518+0000 7f4cfd400640 1 -- 172.21.15.97:0/2745993292 learned_addr learned my addr 172.21.15.97:0/2745993292 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:29.492 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.518+0000 7f4cfde00640 1 --2- 172.21.15.97:0/2745993292 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f72f2bb550 0x55f72f2c54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:29.492 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.518+0000 7f4cfd400640 1 --2- 172.21.15.97:0/2745993292 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f72f2c06c0 0x55f72f2cc490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:29.492 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.519+0000 7f4cfd400640 1 -- 172.21.15.97:0/2745993292 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f72f2bb550 msgr2=0x55f72f2c54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:29.492 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.519+0000 7f4cfd400640 1 --2- 172.21.15.97:0/2745993292 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f72f2bb550 0x55f72f2c54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:29.492 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.519+0000 7f4cfd400640 1 -- 172.21.15.97:0/2745993292 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f72f2c06c0 msgr2=0x55f72f2cc490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:29.492 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.519+0000 7f4cfd400640 1 --2- 172.21.15.97:0/2745993292 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f72f2c06c0 0x55f72f2cc490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:29.492 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.519+0000 7f4cfd400640 1 -- 172.21.15.97:0/2745993292 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f72f1e75f0 con 0x55f72f2bab60 2024-03-27T10:57:29.492 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.519+0000 7f4cfd400640 1 --2- 172.21.15.97:0/2745993292 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f72f2bab60 0x55f72f2c4f80 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f4cec009ed0 tx=0x7f4cec009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74eed9d78685fefb server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:29.492 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.519+0000 7f4cdfe00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T10:57:29.493 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.519+0000 7f4cdfe00640 1 -- 172.21.15.97:0/2745993292 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cec004460 con 0x55f72f2bab60 2024-03-27T10:57:29.493 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.520+0000 7f4d079f0740 1 -- 172.21.15.97:0/2745993292 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4cec012e30 con 0x55f72f2bab60 2024-03-27T10:57:29.493 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.520+0000 7f4cdfe00640 1 -- 172.21.15.97:0/2745993292 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f4cec01f860 con 0x55f72f2bab60 2024-03-27T10:57:29.493 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.520+0000 7f4cdfe00640 1 -- 172.21.15.97:0/2745993292 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4cec01fa60 con 0x55f72f2bab60 2024-03-27T10:57:29.493 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.520+0000 7f4d079f0740 10 client.15723 fetch_fsmap learned FSMap version 28 2024-03-27T10:57:29.493 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.520+0000 7f4d079f0740 1 -- 172.21.15.97:0/2745993292 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f72ef34210 con 0x55f72f2bab60 2024-03-27T10:57:29.494 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.520+0000 7f4cdfe00640 1 -- 172.21.15.97:0/2745993292 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4cec012e30 con 0x55f72f2bab60 2024-03-27T10:57:29.494 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.520+0000 7f4cdfe00640 1 -- 172.21.15.97:0/2745993292 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f4cec02a420 con 0x55f72f2bab60 2024-03-27T10:57:29.494 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.521+0000 7f4d079f0740 10 client.15723 fetch_fsmap finished waiting for FSMap version 28 2024-03-27T10:57:29.494 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.521+0000 7f4d079f0740 10 client.15723 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T10:57:29.494 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.521+0000 7f4d079f0740 20 client.15723 populate_metadata read hostname 'smithi097' 2024-03-27T10:57:29.494 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.521+0000 7f4d079f0740 1 --2- 172.21.15.97:0/2745993292 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55f72f2d09f0 0x55f72f2d2de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:29.494 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.521+0000 7f4d079f0740 4 client.15723 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T10:57:29.494 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.521+0000 7f4d079f0740 1 -- 172.21.15.97:0/2745993292 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f72f22d250 con 0x55f72f2d09f0 2024-03-27T10:57:29.495 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.521+0000 7f4cfde00640 1 --2- 172.21.15.97:0/2745993292 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55f72f2d09f0 0x55f72f2d2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:29.495 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.522+0000 7f4cdfe00640 10 client.15723 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T10:57:29.495 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.522+0000 7f4cfde00640 1 --2- 172.21.15.97:0/2745993292 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55f72f2d09f0 0x55f72f2d2de0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=957144bfa8ee42b0 server_cookie=6ea7ce0e5af4b571 in_seq=0 out_seq=0 2024-03-27T10:57:29.496 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.524+0000 7f4cdfe00640 1 -- 172.21.15.97:0/2745993292 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55f72f22d250 con 0x55f72f2d09f0 2024-03-27T10:57:29.496 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.524+0000 7f4cdfe00640 10 client.15723 handle_command_reply: tid=0 2024-03-27T10:57:29.496 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.524+0000 7f4d079f0740 1 client.15723 shutdown 2024-03-27T10:57:29.496 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.524+0000 7f4d079f0740 1 -- 172.21.15.97:0/2745993292 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f72f2bab60 msgr2=0x55f72f2c4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:29.496 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.524+0000 7f4d079f0740 1 --2- 172.21.15.97:0/2745993292 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f72f2bab60 0x55f72f2c4f80 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f4cec009ed0 tx=0x7f4cec009e60 comp rx=0 tx=0).stop 2024-03-27T10:57:29.496 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.524+0000 7f4d079f0740 1 -- 172.21.15.97:0/2745993292 shutdown_connections 2024-03-27T10:57:29.496 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.524+0000 7f4d079f0740 1 --2- 172.21.15.97:0/2745993292 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55f72f2d09f0 0x55f72f2d2de0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:29.496 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.524+0000 7f4d079f0740 1 --2- 172.21.15.97:0/2745993292 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f72f2c06c0 0x55f72f2cc490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:29.496 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.524+0000 7f4d079f0740 1 --2- 172.21.15.97:0/2745993292 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f72f2bb550 0x55f72f2c54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:29.497 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.524+0000 7f4d079f0740 1 --2- 172.21.15.97:0/2745993292 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f72f2bab60 0x55f72f2c4f80 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:29.497 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.524+0000 7f4d079f0740 1 -- 172.21.15.97:0/2745993292 >> 172.21.15.97:0/2745993292 conn(0x55f72f2b60d0 msgr2=0x55f72f2b5310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:29.497 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.524+0000 7f4cdfe00640 0 client.15723 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T10:57:29.497 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.524+0000 7f4d079f0740 1 -- 172.21.15.97:0/2745993292 shutdown_connections 2024-03-27T10:57:29.497 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.524+0000 7f4d079f0740 1 -- 172.21.15.97:0/2745993292 wait complete. 2024-03-27T10:57:29.497 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.524+0000 7f4d079f0740 20 client.15723 trim_cache size 0 max 16384 2024-03-27T10:57:29.525 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.553+0000 7f4d079f0740 1 Processor -- start 2024-03-27T10:57:29.525 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.554+0000 7f4d079f0740 1 -- start start 2024-03-27T10:57:29.525 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.554+0000 7f4d079f0740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f72f2bab60 0x55f72f2c4b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:29.525 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.554+0000 7f4d079f0740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f72f2bb550 0x55f72f2cc420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:29.525 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.554+0000 7f4d079f0740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f72f2c06c0 0x55f72f2c50d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:29.525 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.554+0000 7f4d079f0740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4d001a6b60 con 0x55f72f2bab60 2024-03-27T10:57:29.525 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.554+0000 7f4d079f0740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f4d001a86a0 con 0x55f72f2bb550 2024-03-27T10:57:29.525 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.554+0000 7f4d079f0740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f4d001a7230 con 0x55f72f2c06c0 2024-03-27T10:57:29.527 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.554+0000 7f4cfd400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f72f2bab60 0x55f72f2c4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:29.527 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.554+0000 7f4cfd400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f72f2bab60 0x55f72f2c4b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38200/0 (socket says 172.21.15.97:38200) 2024-03-27T10:57:29.527 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.554+0000 7f4cfd400640 1 -- 172.21.15.97:0/3893345173 learned_addr learned my addr 172.21.15.97:0/3893345173 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:29.527 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.554+0000 7f4cfde00640 1 --2- 172.21.15.97:0/3893345173 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f72f2bb550 0x55f72f2cc420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:29.527 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.555+0000 7f4cfd400640 1 --2- 172.21.15.97:0/3893345173 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f72f2c06c0 0x55f72f2c50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:29.527 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.555+0000 7f4cfd400640 1 -- 172.21.15.97:0/3893345173 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f72f2c06c0 msgr2=0x55f72f2c50d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:29.527 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.555+0000 7f4cfd400640 1 --2- 172.21.15.97:0/3893345173 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f72f2c06c0 0x55f72f2c50d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:29.527 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.555+0000 7f4cfd400640 1 -- 172.21.15.97:0/3893345173 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f72f2bb550 msgr2=0x55f72f2cc420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:29.527 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.555+0000 7f4cfd400640 1 --2- 172.21.15.97:0/3893345173 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f72f2bb550 0x55f72f2cc420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:29.528 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.555+0000 7f4cfd400640 1 -- 172.21.15.97:0/3893345173 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f72f000620 con 0x55f72f2bab60 2024-03-27T10:57:29.528 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.555+0000 7f4cfd400640 1 --2- 172.21.15.97:0/3893345173 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f72f2bab60 0x55f72f2c4b90 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f4cec000c00 tx=0x7f4cec009270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1608871cbb9b3da3 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:29.528 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.556+0000 7f4ceaa00640 1 -- 172.21.15.97:0/3893345173 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cec039070 con 0x55f72f2bab60 2024-03-27T10:57:29.528 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.556+0000 7f4ceaa00640 1 -- 172.21.15.97:0/3893345173 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4cec02aab0 con 0x55f72f2bab60 2024-03-27T10:57:29.528 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.556+0000 7f4ceaa00640 1 -- 172.21.15.97:0/3893345173 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cec010e50 con 0x55f72f2bab60 2024-03-27T10:57:29.530 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.557+0000 7f4d079f0740 1 -- 172.21.15.97:0/3893345173 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f72f2bab60 msgr2=0x55f72f2c4b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:29.530 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.557+0000 7f4d079f0740 1 --2- 172.21.15.97:0/3893345173 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f72f2bab60 0x55f72f2c4b90 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f4cec000c00 tx=0x7f4cec009270 comp rx=0 tx=0).stop 2024-03-27T10:57:29.530 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.557+0000 7f4d079f0740 1 -- 172.21.15.97:0/3893345173 shutdown_connections 2024-03-27T10:57:29.531 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.557+0000 7f4d079f0740 1 --2- 172.21.15.97:0/3893345173 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f72f2c06c0 0x55f72f2c50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:29.531 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.557+0000 7f4d079f0740 1 --2- 172.21.15.97:0/3893345173 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f72f2bb550 0x55f72f2cc420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:29.531 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.557+0000 7f4d079f0740 1 --2- 172.21.15.97:0/3893345173 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f72f2bab60 0x55f72f2c4b90 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:29.531 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.557+0000 7f4d079f0740 1 -- 172.21.15.97:0/3893345173 >> 172.21.15.97:0/3893345173 conn(0x55f72f2b60d0 msgr2=0x55f72f2bc910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:29.531 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.557+0000 7f4d079f0740 1 -- 172.21.15.97:0/3893345173 shutdown_connections 2024-03-27T10:57:29.531 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.557+0000 7f4d079f0740 1 -- 172.21.15.97:0/3893345173 wait complete. 2024-03-27T10:57:29.531 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.557+0000 7f4d079f0740 1 -- start start 2024-03-27T10:57:29.531 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.557+0000 7f4d079f0740 1 Processor -- start 2024-03-27T10:57:29.531 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.558+0000 7f4d079f0740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f72f2bab60 0x55f72f2c9260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:29.531 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.558+0000 7f4d079f0740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f72f2bb550 0x55f72f2c97a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:29.531 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.558+0000 7f4d079f0740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f72f2c06c0 0x55f72f334ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:29.531 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.558+0000 7f4d079f0740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4d001a7fd0 con 0x55f72f2c06c0 2024-03-27T10:57:29.531 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.558+0000 7f4d079f0740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f4d001a7900 con 0x55f72f2bab60 2024-03-27T10:57:29.532 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.558+0000 7f4d079f0740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f4d001a4b40 con 0x55f72f2bb550 2024-03-27T10:57:29.532 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.558+0000 7f4d079f0740 10 client.0 _ll_register_callbacks cb 0x7f4cec009db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:57:29.532 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.558+0000 7f4cfd400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f72f2c06c0 0x55f72f334ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:29.532 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.558+0000 7f4cfd400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f72f2c06c0 0x55f72f334ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38204/0 (socket says 172.21.15.97:38204) 2024-03-27T10:57:29.532 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.558+0000 7f4cfd400640 1 -- 172.21.15.97:0/3423789915 learned_addr learned my addr 172.21.15.97:0/3423789915 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:29.532 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.558+0000 7f4cfd400640 1 --2- 172.21.15.97:0/3423789915 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f72f2bab60 0x55f72f2c9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:29.532 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.559+0000 7f4cfde00640 1 --2- 172.21.15.97:0/3423789915 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f72f2bb550 0x55f72f2c97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:29.532 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.559+0000 7f4cfd400640 1 -- 172.21.15.97:0/3423789915 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f72f2bb550 msgr2=0x55f72f2c97a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:29.532 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.559+0000 7f4cfd400640 1 --2- 172.21.15.97:0/3423789915 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f72f2bb550 0x55f72f2c97a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:29.532 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.559+0000 7f4cfd400640 1 -- 172.21.15.97:0/3423789915 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f72f2bab60 msgr2=0x55f72f2c9260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:29.532 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.559+0000 7f4cfd400640 1 --2- 172.21.15.97:0/3423789915 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f72f2bab60 0x55f72f2c9260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:29.532 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.559+0000 7f4cfd400640 1 -- 172.21.15.97:0/3423789915 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f72f2c9e10 con 0x55f72f2c06c0 2024-03-27T10:57:29.532 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.559+0000 7f4cfde00640 1 --2- 172.21.15.97:0/3423789915 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f72f2bb550 0x55f72f2c97a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:57:29.533 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.559+0000 7f4cfd400640 1 --2- 172.21.15.97:0/3423789915 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f72f2c06c0 0x55f72f334ca0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f4cec02a480 tx=0x7f4cec02c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afc5e4f3935f8a60 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:29.533 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.559+0000 7f4cdfe00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T10:57:29.533 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.559+0000 7f4cdfe00640 1 -- 172.21.15.97:0/3423789915 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cec039070 con 0x55f72f2c06c0 2024-03-27T10:57:29.533 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.559+0000 7f4d079f0740 1 -- 172.21.15.97:0/3423789915 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4cec02aab0 con 0x55f72f2c06c0 2024-03-27T10:57:29.533 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.559+0000 7f4cdfe00640 1 -- 172.21.15.97:0/3423789915 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f4cec010b10 con 0x55f72f2c06c0 2024-03-27T10:57:29.533 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.559+0000 7f4cdfe00640 1 -- 172.21.15.97:0/3423789915 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4cec010d10 con 0x55f72f2c06c0 2024-03-27T10:57:29.533 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.559+0000 7f4d079f0740 10 client.15729 fetch_fsmap learned FSMap version 28 2024-03-27T10:57:29.533 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.560+0000 7f4d079f0740 1 -- 172.21.15.97:0/3423789915 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f72f3362f0 con 0x55f72f2c06c0 2024-03-27T10:57:29.534 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.560+0000 7f4cdfe00640 1 -- 172.21.15.97:0/3423789915 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4cec02aab0 con 0x55f72f2c06c0 2024-03-27T10:57:29.534 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.560+0000 7f4cdfe00640 1 -- 172.21.15.97:0/3423789915 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f4cec03e030 con 0x55f72f2c06c0 2024-03-27T10:57:29.534 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.560+0000 7f4d079f0740 10 client.15729 fetch_fsmap finished waiting for FSMap version 28 2024-03-27T10:57:29.534 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.560+0000 7f4d079f0740 10 client.15729 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T10:57:29.534 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.560+0000 7f4d079f0740 20 client.15729 populate_metadata read hostname 'smithi097' 2024-03-27T10:57:29.534 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.560+0000 7f4d079f0740 1 --2- 172.21.15.97:0/3423789915 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55f72f336be0 0x55f72f339000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:29.534 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.560+0000 7f4d079f0740 4 client.15729 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-27T10:57:29.534 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.560+0000 7f4d079f0740 1 -- 172.21.15.97:0/3423789915 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f72f2275a0 con 0x55f72f336be0 2024-03-27T10:57:29.535 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.560+0000 7f4cfde00640 1 --2- 172.21.15.97:0/3423789915 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55f72f336be0 0x55f72f339000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:29.535 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.561+0000 7f4cdfe00640 10 client.15729 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T10:57:29.535 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.561+0000 7f4cfde00640 1 --2- 172.21.15.97:0/3423789915 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55f72f336be0 0x55f72f339000 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86568e1c1fbfbd6f server_cookie=4c74bc4956195ec5 in_seq=0 out_seq=0 2024-03-27T10:57:29.539 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.567+0000 7f4cdfe00640 1 -- 172.21.15.97:0/3423789915 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f72f2275a0 con 0x55f72f336be0 2024-03-27T10:57:29.539 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.567+0000 7f4cdfe00640 10 client.15729 handle_command_reply: tid=0 2024-03-27T10:57:29.539 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.568+0000 7f4d079f0740 1 client.15729 shutdown 2024-03-27T10:57:29.539 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.568+0000 7f4d079f0740 1 -- 172.21.15.97:0/3423789915 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f72f2c06c0 msgr2=0x55f72f334ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:29.539 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.568+0000 7f4d079f0740 1 --2- 172.21.15.97:0/3423789915 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f72f2c06c0 0x55f72f334ca0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f4cec02a480 tx=0x7f4cec02c040 comp rx=0 tx=0).stop 2024-03-27T10:57:29.539 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.568+0000 7f4d079f0740 1 -- 172.21.15.97:0/3423789915 shutdown_connections 2024-03-27T10:57:29.539 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.568+0000 7f4d079f0740 1 --2- 172.21.15.97:0/3423789915 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55f72f336be0 0x55f72f339000 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:29.539 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.568+0000 7f4d079f0740 1 --2- 172.21.15.97:0/3423789915 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f72f2c06c0 0x55f72f334ca0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:29.540 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.568+0000 7f4d079f0740 1 --2- 172.21.15.97:0/3423789915 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f72f2bb550 0x55f72f2c97a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:29.540 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.568+0000 7f4d079f0740 1 --2- 172.21.15.97:0/3423789915 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f72f2bab60 0x55f72f2c9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:29.540 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.568+0000 7f4d079f0740 1 -- 172.21.15.97:0/3423789915 >> 172.21.15.97:0/3423789915 conn(0x55f72f2b60d0 msgr2=0x55f72f2c4b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:29.540 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.568+0000 7f4cdfe00640 0 client.15729 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T10:57:29.540 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.568+0000 7f4d079f0740 1 -- 172.21.15.97:0/3423789915 shutdown_connections 2024-03-27T10:57:29.540 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.568+0000 7f4d079f0740 1 -- 172.21.15.97:0/3423789915 wait complete. 2024-03-27T10:57:29.540 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.568+0000 7f4d079f0740 20 client.15729 trim_cache size 0 max 16384 2024-03-27T10:57:29.540 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-27T10:57:29.540 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-27T10:57:29.541 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-27T10:57:29.541 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-27T10:57:29.541 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.569+0000 7f4cff200640 1 -- 172.21.15.97:0/2974728876 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f4ce40a3600 msgr2=0x7f4ce40a5ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:29.541 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.569+0000 7f4cff200640 1 --2- 172.21.15.97:0/2974728876 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f4ce40a3600 0x7f4ce40a5ac0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f4cf80bafe0 tx=0x7f4cf80095c0 comp rx=0 tx=0).stop 2024-03-27T10:57:29.541 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.569+0000 7f4cff200640 1 -- 172.21.15.97:0/2974728876 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d00103570 msgr2=0x7f4d0019f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:29.541 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.569+0000 7f4cff200640 1 --2- 172.21.15.97:0/2974728876 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d00103570 0x7f4d0019f160 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f4cf8009bf0 tx=0x7f4cf8034cd0 comp rx=0 tx=0).stop 2024-03-27T10:57:29.541 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.569+0000 7f4cff200640 1 -- 172.21.15.97:0/2974728876 shutdown_connections 2024-03-27T10:57:29.542 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.569+0000 7f4cff200640 1 --2- 172.21.15.97:0/2974728876 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d00103570 0x7f4d0019f160 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:29.542 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.569+0000 7f4cff200640 1 --2- 172.21.15.97:0/2974728876 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4d00102bb0 0x7f4d001980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:29.542 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.569+0000 7f4cff200640 1 --2- 172.21.15.97:0/2974728876 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f4ce40a3600 0x7f4ce40a5ac0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:29.542 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.569+0000 7f4cff200640 1 --2- 172.21.15.97:0/2974728876 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4d001019c0 0x7f4d00197bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:29.542 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.569+0000 7f4cff200640 1 -- 172.21.15.97:0/2974728876 >> 172.21.15.97:0/2974728876 conn(0x7f4d000fcf30 msgr2=0x7f4d00105eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:29.542 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.569+0000 7f4cff200640 1 -- 172.21.15.97:0/2974728876 shutdown_connections 2024-03-27T10:57:29.542 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:29.570+0000 7f4cff200640 1 -- 172.21.15.97:0/2974728876 wait complete. 2024-03-27T10:57:29.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:57:29 smithi097 ceph-mon[31859]: pgmap v466: 97 pgs: 97 active+clean; 664 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-27T10:57:29.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:57:29 smithi177 ceph-mon[35294]: pgmap v466: 97 pgs: 97 active+clean; 664 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-27T10:57:29.766 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:57:29 smithi114 ceph-mon[35338]: pgmap v466: 97 pgs: 97 active+clean; 664 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-27T10:57:31.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:57:31 smithi097 ceph-mon[31859]: pgmap v467: 97 pgs: 97 active+clean; 664 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-27T10:57:31.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:57:31 smithi177 ceph-mon[35294]: pgmap v467: 97 pgs: 97 active+clean; 664 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-27T10:57:31.766 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:57:31 smithi114 ceph-mon[35338]: pgmap v467: 97 pgs: 97 active+clean; 664 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-27T10:57:33.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:57:33 smithi097 ceph-mon[31859]: pgmap v468: 97 pgs: 97 active+clean; 664 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T10:57:33.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:57:33 smithi177 ceph-mon[35294]: pgmap v468: 97 pgs: 97 active+clean; 664 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T10:57:33.766 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:57:33 smithi114 ceph-mon[35338]: pgmap v468: 97 pgs: 97 active+clean; 664 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T10:57:35.568 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:57:35 smithi114 ceph-mon[35338]: pgmap v469: 97 pgs: 97 active+clean; 664 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-27T10:57:35.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:57:35 smithi097 ceph-mon[31859]: pgmap v469: 97 pgs: 97 active+clean; 664 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-27T10:57:35.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:57:35 smithi177 ceph-mon[35294]: pgmap v469: 97 pgs: 97 active+clean; 664 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-27T10:57:37.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:57:37 smithi097 ceph-mon[31859]: pgmap v470: 97 pgs: 97 active+clean; 664 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-27T10:57:37.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:57:37 smithi177 ceph-mon[35294]: pgmap v470: 97 pgs: 97 active+clean; 664 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-27T10:57:37.766 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:57:37 smithi114 ceph-mon[35338]: pgmap v470: 97 pgs: 97 active+clean; 664 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-27T10:57:38.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:57:38 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:57:38.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:57:38 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:57:38.766 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:57:38 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:57:39.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:57:39 smithi097 ceph-mon[31859]: pgmap v471: 97 pgs: 97 active+clean; 664 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-27T10:57:39.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:57:39 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:57:39.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:57:39 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:57:39.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:57:39 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:57:39.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:57:39 smithi177 ceph-mon[35294]: pgmap v471: 97 pgs: 97 active+clean; 664 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-27T10:57:39.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:57:39 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:57:39.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:57:39 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:57:39.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:57:39 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:57:39.767 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:57:39 smithi114 ceph-mon[35338]: pgmap v471: 97 pgs: 97 active+clean; 664 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-27T10:57:39.767 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:57:39 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:57:39.767 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:57:39 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:57:39.767 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:57:39 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:57:40.840 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-27T10:57:40.841 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-27T10:57:41.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.084+0000 7f66b4e00640 1 -- 172.21.15.97:0/323272832 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66b0100300 msgr2=0x7f66b0100720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:41.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.084+0000 7f66b4e00640 1 --2- 172.21.15.97:0/323272832 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66b0100300 0x7f66b0100720 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f66a0009900 tx=0x7f66a002bc30 comp rx=0 tx=0).stop 2024-03-27T10:57:41.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.084+0000 7f66b4e00640 1 -- 172.21.15.97:0/323272832 shutdown_connections 2024-03-27T10:57:41.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.084+0000 7f66b4e00640 1 --2- 172.21.15.97:0/323272832 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f66b0101600 0x7f66b0101a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:41.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.084+0000 7f66b4e00640 1 --2- 172.21.15.97:0/323272832 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f66b0100c60 0x7f66b01010c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:41.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.084+0000 7f66b4e00640 1 --2- 172.21.15.97:0/323272832 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66b0100300 0x7f66b0100720 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:41.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.084+0000 7f66b4e00640 1 -- 172.21.15.97:0/323272832 >> 172.21.15.97:0/323272832 conn(0x7f66b00fbec0 msgr2=0x7f66b00fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:41.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.084+0000 7f66b4e00640 1 -- 172.21.15.97:0/323272832 shutdown_connections 2024-03-27T10:57:41.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.085+0000 7f66b4e00640 1 -- 172.21.15.97:0/323272832 wait complete. 2024-03-27T10:57:41.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.086+0000 7f66b4e00640 1 Processor -- start 2024-03-27T10:57:41.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.086+0000 7f66b4e00640 1 -- start start 2024-03-27T10:57:41.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.087+0000 7f66b4e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66b0100300 0x7f66b0199e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:41.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.087+0000 7f66b4e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f66b0100c60 0x7f66b019a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:41.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.087+0000 7f66b4e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f66b0101600 0x7f66b01a1440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:41.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.087+0000 7f66b4e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f66b010f7f0 con 0x7f66b0100300 2024-03-27T10:57:41.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.087+0000 7f66b4e00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f66b010f670 con 0x7f66b0100c60 2024-03-27T10:57:41.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.087+0000 7f66b4e00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f66b010f970 con 0x7f66b0101600 2024-03-27T10:57:41.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.087+0000 7f66af400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66b0100300 0x7f66b0199e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:41.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.087+0000 7f66af400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66b0100300 0x7f66b0199e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53296/0 (socket says 172.21.15.97:53296) 2024-03-27T10:57:41.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.087+0000 7f66af400640 1 -- 172.21.15.97:0/3344473127 learned_addr learned my addr 172.21.15.97:0/3344473127 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:41.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.087+0000 7f66afe00640 1 --2- 172.21.15.97:0/3344473127 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f66b0101600 0x7f66b01a1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:41.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.087+0000 7f66aea00640 1 --2- 172.21.15.97:0/3344473127 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f66b0100c60 0x7f66b019a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:41.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.088+0000 7f66af400640 1 -- 172.21.15.97:0/3344473127 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f66b0101600 msgr2=0x7f66b01a1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:41.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.088+0000 7f66af400640 1 --2- 172.21.15.97:0/3344473127 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f66b0101600 0x7f66b01a1440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:41.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.088+0000 7f66af400640 1 -- 172.21.15.97:0/3344473127 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f66b0100c60 msgr2=0x7f66b019a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:41.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.088+0000 7f66af400640 1 --2- 172.21.15.97:0/3344473127 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f66b0100c60 0x7f66b019a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:41.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.088+0000 7f66af400640 1 -- 172.21.15.97:0/3344473127 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66b01a1ab0 con 0x7f66b0100300 2024-03-27T10:57:41.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.089+0000 7f66af400640 1 --2- 172.21.15.97:0/3344473127 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66b0100300 0x7f66b0199e90 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f66a0000c00 tx=0x7f66a00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:41.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.089+0000 7f66a7e00640 1 -- 172.21.15.97:0/3344473127 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a0043020 con 0x7f66b0100300 2024-03-27T10:57:41.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.089+0000 7f66a7e00640 1 -- 172.21.15.97:0/3344473127 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f66a00054c0 con 0x7f66b0100300 2024-03-27T10:57:41.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.089+0000 7f66b4e00640 1 -- 172.21.15.97:0/3344473127 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66b01a1d40 con 0x7f66b0100300 2024-03-27T10:57:41.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.090+0000 7f66b4e00640 1 -- 172.21.15.97:0/3344473127 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66b01a2230 con 0x7f66b0100300 2024-03-27T10:57:41.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.090+0000 7f66a7e00640 1 -- 172.21.15.97:0/3344473127 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a0043020 con 0x7f66b0100300 2024-03-27T10:57:41.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.094+0000 7f66b903d740 1 Processor -- start 2024-03-27T10:57:41.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.095+0000 7f66b903d740 1 -- start start 2024-03-27T10:57:41.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.095+0000 7f66b903d740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55951621f670 0x55951621fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:41.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.095+0000 7f66b903d740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559516220060 0x559516224b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:41.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.095+0000 7f66b903d740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595162251d0 0x559516227660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:41.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.095+0000 7f66b903d740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5595161856b0 con 0x5595162251d0 2024-03-27T10:57:41.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.095+0000 7f66b903d740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55951612d750 con 0x559516220060 2024-03-27T10:57:41.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.095+0000 7f66b903d740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x559516181660 con 0x55951621f670 2024-03-27T10:57:41.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.096+0000 7f66afe00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55951621f670 0x55951621fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:41.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.096+0000 7f66aea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595162251d0 0x559516227660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:41.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.096+0000 7f66aea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595162251d0 0x559516227660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53304/0 (socket says 172.21.15.97:53304) 2024-03-27T10:57:41.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.096+0000 7f66aea00640 1 -- 172.21.15.97:0/1598850236 learned_addr learned my addr 172.21.15.97:0/1598850236 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:41.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.096+0000 7f66afe00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55951621f670 0x55951621fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.97:34878/0 (socket says 172.21.15.97:34878) 2024-03-27T10:57:41.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.096+0000 7f66aea00640 1 -- 172.21.15.97:0/1598850236 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55951621f670 msgr2=0x55951621fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:41.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.096+0000 7f66aea00640 1 --2- 172.21.15.97:0/1598850236 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55951621f670 0x55951621fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:41.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.096+0000 7f66aea00640 1 -- 172.21.15.97:0/1598850236 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559516220060 msgr2=0x559516224b60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-27T10:57:41.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.096+0000 7f66aea00640 1 --2- 172.21.15.97:0/1598850236 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559516220060 0x559516224b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:41.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.096+0000 7f66aea00640 1 -- 172.21.15.97:0/1598850236 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559515f637d0 con 0x5595162251d0 2024-03-27T10:57:41.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.097+0000 7f66aea00640 1 --2- 172.21.15.97:0/1598850236 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595162251d0 0x559516227660 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f669c00bce0 tx=0x7f669c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f129ffb934035ffc server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:41.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.097+0000 7f66a4c00640 1 -- 172.21.15.97:0/1598850236 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f669c015070 con 0x5595162251d0 2024-03-27T10:57:41.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.097+0000 7f66a4c00640 1 -- 172.21.15.97:0/1598850236 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f669c01a910 con 0x5595162251d0 2024-03-27T10:57:41.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.098+0000 7f66a4c00640 1 -- 172.21.15.97:0/1598850236 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f669c011980 con 0x5595162251d0 2024-03-27T10:57:41.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.099+0000 7f66b903d740 1 -- 172.21.15.97:0/1598850236 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595162251d0 msgr2=0x559516227660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:41.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.099+0000 7f66b903d740 1 --2- 172.21.15.97:0/1598850236 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595162251d0 0x559516227660 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f669c00bce0 tx=0x7f669c00f730 comp rx=0 tx=0).stop 2024-03-27T10:57:41.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.099+0000 7f66b903d740 1 -- 172.21.15.97:0/1598850236 shutdown_connections 2024-03-27T10:57:41.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.099+0000 7f66b903d740 1 --2- 172.21.15.97:0/1598850236 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595162251d0 0x559516227660 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:41.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.099+0000 7f66b903d740 1 --2- 172.21.15.97:0/1598850236 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559516220060 0x559516224b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:41.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.099+0000 7f66b903d740 1 --2- 172.21.15.97:0/1598850236 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55951621f670 0x55951621fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:41.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.099+0000 7f66b903d740 1 -- 172.21.15.97:0/1598850236 >> 172.21.15.97:0/1598850236 conn(0x55951621abe0 msgr2=0x55951621afe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:41.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.102+0000 7f66b903d740 1 -- 172.21.15.97:0/1598850236 shutdown_connections 2024-03-27T10:57:41.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.102+0000 7f66a7e00640 1 -- 172.21.15.97:0/3344473127 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f66a0046020 con 0x7f66b0100300 2024-03-27T10:57:41.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.102+0000 7f66b903d740 1 -- 172.21.15.97:0/1598850236 wait complete. 2024-03-27T10:57:41.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.103+0000 7f66b903d740 1 -- start start 2024-03-27T10:57:41.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.103+0000 7f66b903d740 1 Processor -- start 2024-03-27T10:57:41.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.104+0000 7f66b903d740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55951621f670 0x5595162325a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:41.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.104+0000 7f66b903d740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595162251d0 0x559516232ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:41.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.104+0000 7f66b903d740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559516237b10 0x559516239f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:41.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.104+0000 7f66b903d740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x559516228180 con 0x5595162251d0 2024-03-27T10:57:41.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.104+0000 7f66b903d740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x559516140a80 con 0x55951621f670 2024-03-27T10:57:41.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.104+0000 7f66b903d740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x559516228300 con 0x559516237b10 2024-03-27T10:57:41.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.104+0000 7f66b903d740 10 client.0 _ll_register_callbacks cb 0x559516140470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:57:41.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.105+0000 7f66a7e00640 1 --2- 172.21.15.97:0/3344473127 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f66940a37a0 0x7f66940a5c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:41.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.105+0000 7f66afe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55951621f670 0x5595162325a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:41.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.105+0000 7f66a7e00640 1 -- 172.21.15.97:0/3344473127 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f66a00f1a10 con 0x7f66b0100300 2024-03-27T10:57:41.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.105+0000 7f66aea00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559516237b10 0x559516239f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:41.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.105+0000 7f66af400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595162251d0 0x559516232ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:41.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.105+0000 7f66afe00640 1 --2- 172.21.15.97:0/3344473127 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f66940a37a0 0x7f66940a5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:41.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.105+0000 7f66af400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595162251d0 0x559516232ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53310/0 (socket says 172.21.15.97:53310) 2024-03-27T10:57:41.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.105+0000 7f66af400640 1 -- 172.21.15.97:0/2535647730 learned_addr learned my addr 172.21.15.97:0/2535647730 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:41.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.106+0000 7f66afe00640 1 -- 172.21.15.97:0/2535647730 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559516237b10 msgr2=0x559516239f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:41.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.106+0000 7f66afe00640 1 --2- 172.21.15.97:0/2535647730 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559516237b10 0x559516239f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:41.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.106+0000 7f66afe00640 1 -- 172.21.15.97:0/2535647730 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595162251d0 msgr2=0x559516232ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:41.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.106+0000 7f66afe00640 1 --2- 172.21.15.97:0/2535647730 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595162251d0 0x559516232ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:41.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.106+0000 7f66afe00640 1 -- 172.21.15.97:0/2535647730 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559515e993f0 con 0x55951621f670 2024-03-27T10:57:41.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.106+0000 7f66af400640 1 --2- 172.21.15.97:0/2535647730 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595162251d0 0x559516232ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T10:57:41.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.107+0000 7f66afe00640 1 --2- 172.21.15.97:0/3344473127 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f66940a37a0 0x7f66940a5c60 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f66a8009fd0 tx=0x7f66a8009a10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:41.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.107+0000 7f66afe00640 1 --2- 172.21.15.97:0/2535647730 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55951621f670 0x5595162325a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f66a801cc70 tx=0x7f66a800d040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=197cf7d1d0f07480 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:41.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.107+0000 7f6699600640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-03-27T10:57:41.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.108+0000 7f6699600640 1 -- 172.21.15.97:0/2535647730 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a8014040 con 0x55951621f670 2024-03-27T10:57:41.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.108+0000 7f6699600640 1 -- 172.21.15.97:0/2535647730 <== mon.2 v2:172.21.15.114:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f66a801ad60 con 0x55951621f670 2024-03-27T10:57:41.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.108+0000 7f66b903d740 1 -- 172.21.15.97:0/2535647730 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f669c01a910 con 0x55951621f670 2024-03-27T10:57:41.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.108+0000 7f6699600640 1 -- 172.21.15.97:0/2535647730 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f66a800ae30 con 0x55951621f670 2024-03-27T10:57:41.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.108+0000 7f66b903d740 10 client.35402 fetch_fsmap learned FSMap version 28 2024-03-27T10:57:41.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.108+0000 7f66b903d740 1 -- 172.21.15.97:0/2535647730 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559515e590f0 con 0x55951621f670 2024-03-27T10:57:41.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.108+0000 7f6699600640 1 -- 172.21.15.97:0/2535647730 <== mon.2 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f669c01a910 con 0x55951621f670 2024-03-27T10:57:41.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.109+0000 7f6699600640 1 -- 172.21.15.97:0/2535647730 <== mon.2 v2:172.21.15.114:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f66a801dac0 con 0x55951621f670 2024-03-27T10:57:41.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.109+0000 7f66b903d740 10 client.35402 fetch_fsmap finished waiting for FSMap version 28 2024-03-27T10:57:41.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.109+0000 7f66b903d740 10 client.35402 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-27T10:57:41.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.109+0000 7f66b903d740 20 client.35402 populate_metadata read hostname 'smithi097' 2024-03-27T10:57:41.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.109+0000 7f66b903d740 1 --2- 172.21.15.97:0/2535647730 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55951623e790 0x559516240be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:41.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.109+0000 7f66b903d740 4 client.35402 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T10:57:41.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.109+0000 7f66b903d740 1 -- 172.21.15.97:0/2535647730 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559516192bf0 con 0x55951623e790 2024-03-27T10:57:41.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.109+0000 7f66af400640 1 --2- 172.21.15.97:0/2535647730 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55951623e790 0x559516240be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:41.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.110+0000 7f66af400640 1 --2- 172.21.15.97:0/2535647730 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55951623e790 0x559516240be0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd3c907631723ba6 server_cookie=6134195942098aad in_seq=0 out_seq=0 2024-03-27T10:57:41.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.110+0000 7f6699600640 10 client.35402 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T10:57:41.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.111+0000 7f6699600640 1 -- 172.21.15.97:0/2535647730 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x559516192bf0 con 0x55951623e790 2024-03-27T10:57:41.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.111+0000 7f6699600640 10 client.35402 handle_command_reply: tid=0 2024-03-27T10:57:41.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.112+0000 7f66b903d740 1 client.35402 shutdown 2024-03-27T10:57:41.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.112+0000 7f66b903d740 1 -- 172.21.15.97:0/2535647730 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55951621f670 msgr2=0x5595162325a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:41.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.112+0000 7f66b903d740 1 --2- 172.21.15.97:0/2535647730 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55951621f670 0x5595162325a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f66a801cc70 tx=0x7f66a800d040 comp rx=0 tx=0).stop 2024-03-27T10:57:41.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.112+0000 7f66b903d740 1 -- 172.21.15.97:0/2535647730 shutdown_connections 2024-03-27T10:57:41.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.112+0000 7f66b903d740 1 --2- 172.21.15.97:0/2535647730 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55951623e790 0x559516240be0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:41.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.112+0000 7f66b903d740 1 --2- 172.21.15.97:0/2535647730 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559516237b10 0x559516239f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:41.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.112+0000 7f66b903d740 1 --2- 172.21.15.97:0/2535647730 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595162251d0 0x559516232ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:41.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.112+0000 7f66b903d740 1 --2- 172.21.15.97:0/2535647730 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55951621f670 0x5595162325a0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:41.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.112+0000 7f66b903d740 1 -- 172.21.15.97:0/2535647730 >> 172.21.15.97:0/2535647730 conn(0x55951621abe0 msgr2=0x559516219e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:41.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.112+0000 7f6699600640 0 client.35402 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T10:57:41.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.112+0000 7f66b903d740 1 -- 172.21.15.97:0/2535647730 shutdown_connections 2024-03-27T10:57:41.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.112+0000 7f66b903d740 1 -- 172.21.15.97:0/2535647730 wait complete. 2024-03-27T10:57:41.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.113+0000 7f66b903d740 20 client.35402 trim_cache size 0 max 16384 2024-03-27T10:57:41.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.148+0000 7f66b903d740 1 Processor -- start 2024-03-27T10:57:41.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.148+0000 7f66b903d740 1 -- start start 2024-03-27T10:57:41.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.148+0000 7f66b903d740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55951621f670 0x559516221740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:41.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.148+0000 7f66b903d740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5595162251d0 0x559516221c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:41.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.148+0000 7f66b903d740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559516237b10 0x559516232270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:41.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.148+0000 7f66b903d740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f66b01a6bd0 con 0x559516237b10 2024-03-27T10:57:41.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.148+0000 7f66b903d740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f66b01a8710 con 0x55951621f670 2024-03-27T10:57:41.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.148+0000 7f66b903d740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f66b01a72a0 con 0x5595162251d0 2024-03-27T10:57:41.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.149+0000 7f66af400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55951621f670 0x559516221740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:41.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.149+0000 7f66af400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55951621f670 0x559516221740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:33630/0 (socket says 172.21.15.97:33630) 2024-03-27T10:57:41.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.149+0000 7f66af400640 1 -- 172.21.15.97:0/3052175807 learned_addr learned my addr 172.21.15.97:0/3052175807 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:41.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.149+0000 7f66afe00640 1 --2- 172.21.15.97:0/3052175807 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559516237b10 0x559516232270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:41.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.149+0000 7f66aea00640 1 --2- 172.21.15.97:0/3052175807 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5595162251d0 0x559516221c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:41.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.149+0000 7f66afe00640 1 -- 172.21.15.97:0/3052175807 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5595162251d0 msgr2=0x559516221c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:41.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.149+0000 7f66afe00640 1 --2- 172.21.15.97:0/3052175807 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5595162251d0 0x559516221c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:41.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.149+0000 7f66afe00640 1 -- 172.21.15.97:0/3052175807 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55951621f670 msgr2=0x559516221740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:41.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.149+0000 7f66afe00640 1 --2- 172.21.15.97:0/3052175807 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55951621f670 0x559516221740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:41.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.149+0000 7f66afe00640 1 -- 172.21.15.97:0/3052175807 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595162332d0 con 0x559516237b10 2024-03-27T10:57:41.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.150+0000 7f66af400640 1 --2- 172.21.15.97:0/3052175807 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55951621f670 0x559516221740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T10:57:41.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.150+0000 7f66afe00640 1 --2- 172.21.15.97:0/3052175807 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559516237b10 0x559516232270 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f66a8002230 tx=0x7f66a80098b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef2e910ef7b462d0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:41.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.150+0000 7f669be00640 1 -- 172.21.15.97:0/3052175807 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a801cd60 con 0x559516237b10 2024-03-27T10:57:41.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.150+0000 7f669be00640 1 -- 172.21.15.97:0/3052175807 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f66a8026da0 con 0x559516237b10 2024-03-27T10:57:41.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.150+0000 7f669be00640 1 -- 172.21.15.97:0/3052175807 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a801be30 con 0x559516237b10 2024-03-27T10:57:41.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.151+0000 7f66b903d740 1 -- 172.21.15.97:0/3052175807 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559516237b10 msgr2=0x559516232270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:41.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.151+0000 7f66b903d740 1 --2- 172.21.15.97:0/3052175807 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559516237b10 0x559516232270 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f66a8002230 tx=0x7f66a80098b0 comp rx=0 tx=0).stop 2024-03-27T10:57:41.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.151+0000 7f66b903d740 1 -- 172.21.15.97:0/3052175807 shutdown_connections 2024-03-27T10:57:41.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.151+0000 7f66b903d740 1 --2- 172.21.15.97:0/3052175807 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559516237b10 0x559516232270 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:41.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.151+0000 7f66b903d740 1 --2- 172.21.15.97:0/3052175807 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5595162251d0 0x559516221c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:41.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.151+0000 7f66b903d740 1 --2- 172.21.15.97:0/3052175807 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55951621f670 0x559516221740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:41.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.151+0000 7f66b903d740 1 -- 172.21.15.97:0/3052175807 >> 172.21.15.97:0/3052175807 conn(0x55951621abe0 msgr2=0x55951621afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:41.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.152+0000 7f66b903d740 1 -- 172.21.15.97:0/3052175807 shutdown_connections 2024-03-27T10:57:41.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.152+0000 7f66b903d740 1 -- 172.21.15.97:0/3052175807 wait complete. 2024-03-27T10:57:41.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.152+0000 7f66b903d740 1 -- start start 2024-03-27T10:57:41.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.152+0000 7f66b903d740 1 Processor -- start 2024-03-27T10:57:41.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.153+0000 7f66b903d740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55951621f670 0x55951624f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:41.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.153+0000 7f66b903d740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5595162251d0 0x55951624f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:41.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.153+0000 7f66b903d740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559516237b10 0x559516250650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:41.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.153+0000 7f66b903d740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f66b01a8040 con 0x559516237b10 2024-03-27T10:57:41.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.153+0000 7f66b903d740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f66b01a7970 con 0x5595162251d0 2024-03-27T10:57:41.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.153+0000 7f66b903d740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f66b01a4bd0 con 0x55951621f670 2024-03-27T10:57:41.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.153+0000 7f66b903d740 10 client.0 _ll_register_callbacks cb 0x7f66a801c9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:57:41.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.153+0000 7f66aea00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5595162251d0 0x55951624f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:41.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.153+0000 7f66afe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559516237b10 0x559516250650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:41.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.153+0000 7f66aea00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5595162251d0 0x55951624f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:33646/0 (socket says 172.21.15.97:33646) 2024-03-27T10:57:41.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.153+0000 7f66aea00640 1 -- 172.21.15.97:0/3672284493 learned_addr learned my addr 172.21.15.97:0/3672284493 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:41.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.154+0000 7f66af400640 1 --2- 172.21.15.97:0/3672284493 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55951621f670 0x55951624f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:41.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.154+0000 7f66aea00640 1 -- 172.21.15.97:0/3672284493 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55951621f670 msgr2=0x55951624f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:41.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.154+0000 7f66aea00640 1 --2- 172.21.15.97:0/3672284493 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55951621f670 0x55951624f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:41.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.154+0000 7f66aea00640 1 -- 172.21.15.97:0/3672284493 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559516237b10 msgr2=0x559516250650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:41.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.154+0000 7f66aea00640 1 --2- 172.21.15.97:0/3672284493 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559516237b10 0x559516250650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:41.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.154+0000 7f66aea00640 1 -- 172.21.15.97:0/3672284493 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559516250c40 con 0x5595162251d0 2024-03-27T10:57:41.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.154+0000 7f66afe00640 1 --2- 172.21.15.97:0/3672284493 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559516237b10 0x559516250650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T10:57:41.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.154+0000 7f66aea00640 1 --2- 172.21.15.97:0/3672284493 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5595162251d0 0x55951624f640 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f669c00bce0 tx=0x7f669c00ba30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e56c4a743b6a9eff server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:41.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.154+0000 7f6699600640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-03-27T10:57:41.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.155+0000 7f6699600640 1 -- 172.21.15.97:0/3672284493 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f669c015040 con 0x5595162251d0 2024-03-27T10:57:41.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.155+0000 7f66b903d740 1 -- 172.21.15.97:0/3672284493 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66a8026da0 con 0x5595162251d0 2024-03-27T10:57:41.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.155+0000 7f6699600640 1 -- 172.21.15.97:0/3672284493 <== mon.2 v2:172.21.15.114:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f669c004440 con 0x5595162251d0 2024-03-27T10:57:41.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.155+0000 7f6699600640 1 -- 172.21.15.97:0/3672284493 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f669c005140 con 0x5595162251d0 2024-03-27T10:57:41.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.155+0000 7f66b903d740 10 client.35408 fetch_fsmap learned FSMap version 28 2024-03-27T10:57:41.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.155+0000 7f66b903d740 1 -- 172.21.15.97:0/3672284493 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559516251f80 con 0x5595162251d0 2024-03-27T10:57:41.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.156+0000 7f6699600640 1 -- 172.21.15.97:0/3672284493 <== mon.2 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f66a8026da0 con 0x5595162251d0 2024-03-27T10:57:41.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.156+0000 7f6699600640 1 -- 172.21.15.97:0/3672284493 <== mon.2 v2:172.21.15.114:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f669c0035f0 con 0x5595162251d0 2024-03-27T10:57:41.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.156+0000 7f66b903d740 10 client.35408 fetch_fsmap finished waiting for FSMap version 28 2024-03-27T10:57:41.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.156+0000 7f66b903d740 10 client.35408 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-27T10:57:41.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.156+0000 7f66b903d740 20 client.35408 populate_metadata read hostname 'smithi097' 2024-03-27T10:57:41.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.156+0000 7f66b903d740 1 --2- 172.21.15.97:0/3672284493 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5595162a4890 0x5595162a6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:41.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.156+0000 7f66b903d740 4 client.35408 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-27T10:57:41.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.156+0000 7f66b903d740 1 -- 172.21.15.97:0/3672284493 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55951618cf40 con 0x5595162a4890 2024-03-27T10:57:41.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.157+0000 7f66af400640 1 --2- 172.21.15.97:0/3672284493 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5595162a4890 0x5595162a6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:41.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.158+0000 7f66af400640 1 --2- 172.21.15.97:0/3672284493 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5595162a4890 0x5595162a6cb0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ad8e02bdd69cb9 server_cookie=200e0752f1cdf986 in_seq=0 out_seq=0 2024-03-27T10:57:41.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.158+0000 7f6699600640 10 client.35408 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T10:57:41.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.158+0000 7f6699600640 1 -- 172.21.15.97:0/3672284493 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55951618cf40 con 0x5595162a4890 2024-03-27T10:57:41.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.158+0000 7f6699600640 10 client.35408 handle_command_reply: tid=0 2024-03-27T10:57:41.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.158+0000 7f66b903d740 1 client.35408 shutdown 2024-03-27T10:57:41.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.159+0000 7f66b903d740 1 -- 172.21.15.97:0/3672284493 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5595162251d0 msgr2=0x55951624f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:41.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.159+0000 7f66b903d740 1 --2- 172.21.15.97:0/3672284493 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5595162251d0 0x55951624f640 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f669c00bce0 tx=0x7f669c00ba30 comp rx=0 tx=0).stop 2024-03-27T10:57:41.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.159+0000 7f66b903d740 1 -- 172.21.15.97:0/3672284493 shutdown_connections 2024-03-27T10:57:41.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.159+0000 7f66b903d740 1 --2- 172.21.15.97:0/3672284493 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5595162a4890 0x5595162a6cb0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:41.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.159+0000 7f66b903d740 1 --2- 172.21.15.97:0/3672284493 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559516237b10 0x559516250650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:41.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.159+0000 7f66b903d740 1 --2- 172.21.15.97:0/3672284493 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5595162251d0 0x55951624f640 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:41.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.159+0000 7f6699600640 0 client.35408 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T10:57:41.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.159+0000 7f66b903d740 1 --2- 172.21.15.97:0/3672284493 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55951621f670 0x55951624f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:41.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.159+0000 7f66b903d740 1 -- 172.21.15.97:0/3672284493 >> 172.21.15.97:0/3672284493 conn(0x55951621abe0 msgr2=0x559516221740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:41.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.159+0000 7f66b903d740 1 -- 172.21.15.97:0/3672284493 shutdown_connections 2024-03-27T10:57:41.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.159+0000 7f66b903d740 1 -- 172.21.15.97:0/3672284493 wait complete. 2024-03-27T10:57:41.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.159+0000 7f66b903d740 20 client.35408 trim_cache size 0 max 16384 2024-03-27T10:57:41.132 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-03-27T10:57:41.132 INFO:teuthology.orchestra.run.smithi097.stdout: "status": "no active scrubs running", 2024-03-27T10:57:41.132 INFO:teuthology.orchestra.run.smithi097.stdout: "scrubs": {} 2024-03-27T10:57:41.132 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-03-27T10:57:41.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.160+0000 7f66b4e00640 1 -- 172.21.15.97:0/3344473127 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f66940a37a0 msgr2=0x7f66940a5c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:41.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.160+0000 7f66b4e00640 1 --2- 172.21.15.97:0/3344473127 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f66940a37a0 0x7f66940a5c60 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f66a8009fd0 tx=0x7f66a8009a10 comp rx=0 tx=0).stop 2024-03-27T10:57:41.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.160+0000 7f66b4e00640 1 -- 172.21.15.97:0/3344473127 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66b0100300 msgr2=0x7f66b0199e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:41.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.160+0000 7f66b4e00640 1 --2- 172.21.15.97:0/3344473127 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66b0100300 0x7f66b0199e90 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f66a0000c00 tx=0x7f66a00042f0 comp rx=0 tx=0).stop 2024-03-27T10:57:41.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.160+0000 7f66b4e00640 1 -- 172.21.15.97:0/3344473127 shutdown_connections 2024-03-27T10:57:41.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.160+0000 7f66b4e00640 1 --2- 172.21.15.97:0/3344473127 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f66b0101600 0x7f66b01a1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:41.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.160+0000 7f66b4e00640 1 --2- 172.21.15.97:0/3344473127 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f66940a37a0 0x7f66940a5c60 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:41.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.160+0000 7f66b4e00640 1 --2- 172.21.15.97:0/3344473127 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f66b0100c60 0x7f66b019a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:41.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.160+0000 7f66b4e00640 1 --2- 172.21.15.97:0/3344473127 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66b0100300 0x7f66b0199e90 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:41.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.160+0000 7f66b4e00640 1 -- 172.21.15.97:0/3344473127 >> 172.21.15.97:0/3344473127 conn(0x7f66b00fbec0 msgr2=0x7f66b00fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:41.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.160+0000 7f66b4e00640 1 -- 172.21.15.97:0/3344473127 shutdown_connections 2024-03-27T10:57:41.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.161+0000 7f66b4e00640 1 -- 172.21.15.97:0/3344473127 wait complete. 2024-03-27T10:57:41.155 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-27T10:57:41.156 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-27T10:57:41.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.454+0000 7fa47fc00640 1 -- 172.21.15.97:0/2905486781 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa478102d30 msgr2=0x7fa4781051a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:41.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.454+0000 7fa47fc00640 1 --2- 172.21.15.97:0/2905486781 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa478102d30 0x7fa4781051a0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fa464009900 tx=0x7fa46402bc40 comp rx=0 tx=0).stop 2024-03-27T10:57:41.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.454+0000 7fa47fc00640 1 -- 172.21.15.97:0/2905486781 shutdown_connections 2024-03-27T10:57:41.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.454+0000 7fa47fc00640 1 --2- 172.21.15.97:0/2905486781 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa4781056e0 0x7fa478105b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:41.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.454+0000 7fa47fc00640 1 --2- 172.21.15.97:0/2905486781 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa478102d30 0x7fa4781051a0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:41.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.454+0000 7fa47fc00640 1 --2- 172.21.15.97:0/2905486781 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa4781003c0 0x7fa4781027f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:41.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.454+0000 7fa47fc00640 1 -- 172.21.15.97:0/2905486781 >> 172.21.15.97:0/2905486781 conn(0x7fa4780fbfc0 msgr2=0x7fa4780fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:41.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.454+0000 7fa47fc00640 1 -- 172.21.15.97:0/2905486781 shutdown_connections 2024-03-27T10:57:41.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.454+0000 7fa47fc00640 1 -- 172.21.15.97:0/2905486781 wait complete. 2024-03-27T10:57:41.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.455+0000 7fa47fc00640 1 Processor -- start 2024-03-27T10:57:41.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.455+0000 7fa47fc00640 1 -- start start 2024-03-27T10:57:41.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.456+0000 7fa47fc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4781003c0 0x7fa47819e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:41.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.456+0000 7fa47fc00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa478102d30 0x7fa47819e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:41.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.456+0000 7fa47fc00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa4781056e0 0x7fa4781a58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:41.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.456+0000 7fa47fc00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa478113cf0 con 0x7fa4781003c0 2024-03-27T10:57:41.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.456+0000 7fa47fc00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa478113b70 con 0x7fa478102d30 2024-03-27T10:57:41.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.456+0000 7fa47fc00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa478113e70 con 0x7fa4781056e0 2024-03-27T10:57:41.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.456+0000 7fa477e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa478102d30 0x7fa47819e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:41.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.456+0000 7fa47d200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4781003c0 0x7fa47819e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:41.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.456+0000 7fa47d200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4781003c0 0x7fa47819e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53352/0 (socket says 172.21.15.97:53352) 2024-03-27T10:57:41.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.456+0000 7fa477e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa478102d30 0x7fa47819e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:33654/0 (socket says 172.21.15.97:33654) 2024-03-27T10:57:41.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.456+0000 7fa47d200640 1 -- 172.21.15.97:0/1674402243 learned_addr learned my addr 172.21.15.97:0/1674402243 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:41.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.457+0000 7fa47dc00640 1 --2- 172.21.15.97:0/1674402243 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa4781056e0 0x7fa4781a58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:41.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.457+0000 7fa47d200640 1 -- 172.21.15.97:0/1674402243 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa4781056e0 msgr2=0x7fa4781a58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:41.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.457+0000 7fa47d200640 1 --2- 172.21.15.97:0/1674402243 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa4781056e0 0x7fa4781a58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:41.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.457+0000 7fa47d200640 1 -- 172.21.15.97:0/1674402243 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa478102d30 msgr2=0x7fa47819e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:41.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.457+0000 7fa47d200640 1 --2- 172.21.15.97:0/1674402243 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa478102d30 0x7fa47819e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:41.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.457+0000 7fa47d200640 1 -- 172.21.15.97:0/1674402243 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4781a5f50 con 0x7fa4781003c0 2024-03-27T10:57:41.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.457+0000 7fa47d200640 1 --2- 172.21.15.97:0/1674402243 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4781003c0 0x7fa47819e330 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fa46c00e9f0 tx=0x7fa46c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:41.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.457+0000 7fa475600640 1 -- 172.21.15.97:0/1674402243 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa46c00cd80 con 0x7fa4781003c0 2024-03-27T10:57:41.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.457+0000 7fa475600640 1 -- 172.21.15.97:0/1674402243 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa46c0108e0 con 0x7fa4781003c0 2024-03-27T10:57:41.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.457+0000 7fa47fc00640 1 -- 172.21.15.97:0/1674402243 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4781a6240 con 0x7fa4781003c0 2024-03-27T10:57:41.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.457+0000 7fa475600640 1 -- 172.21.15.97:0/1674402243 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa46c018920 con 0x7fa4781003c0 2024-03-27T10:57:41.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.458+0000 7fa47fc00640 1 -- 172.21.15.97:0/1674402243 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4781a67b0 con 0x7fa4781003c0 2024-03-27T10:57:41.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.459+0000 7fa47fc00640 1 -- 172.21.15.97:0/1674402243 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4780fdab0 con 0x7fa4781003c0 2024-03-27T10:57:41.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.461+0000 7fa475600640 1 -- 172.21.15.97:0/1674402243 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7fa46c010a90 con 0x7fa4781003c0 2024-03-27T10:57:41.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.463+0000 7fa475600640 1 --2- 172.21.15.97:0/1674402243 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fa4600a3690 0x7fa4600a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:41.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.463+0000 7fa475600640 1 -- 172.21.15.97:0/1674402243 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fa46c0d0040 con 0x7fa4781003c0 2024-03-27T10:57:41.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.464+0000 7fa477e00640 1 --2- 172.21.15.97:0/1674402243 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fa4600a3690 0x7fa4600a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:41.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.465+0000 7fa477e00640 1 --2- 172.21.15.97:0/1674402243 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fa4600a3690 0x7fa4600a5b50 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa464000c00 tx=0x7fa46402c7f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:41.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.468+0000 7fa475600640 1 -- 172.21.15.97:0/1674402243 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7fa46c09cc20 con 0x7fa4781003c0 2024-03-27T10:57:41.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:57:41 smithi097 ceph-mon[31859]: pgmap v472: 97 pgs: 97 active+clean; 664 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-27T10:57:41.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:57:41 smithi177 ceph-mon[35294]: pgmap v472: 97 pgs: 97 active+clean; 664 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-27T10:57:41.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.735+0000 7fa47fc00640 1 -- 172.21.15.97:0/1674402243 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa478062750 con 0x7fa4781003c0 2024-03-27T10:57:41.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.737+0000 7fa475600640 1 -- 172.21.15.97:0/1674402243 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11944 (secure 0 0 0) 0x7fa46c09c520 con 0x7fa4781003c0 2024-03-27T10:57:41.708 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T10:57:41.708 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1009423904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1009423904},{"type":"v1","addr":"172.21.15.97:6835","nonce":1009423904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3251173433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3251173433},{"type":"v1","addr":"172.21.15.97:6837","nonce":3251173433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-27T10:54:04.510178+0000","modified":"2024-03-27T10:56:28.179534+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34508,"mds_1":34478,"mds_2":34445,"mds_3":24454,"mds_4":24400},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14646":{"gid":14646,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6839/1057596821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1057596821},{"type":"v1","addr":"172.21.15.97:6839","nonce":1057596821}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14658":{"gid":14658,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6841/1672464856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6840","nonce":1672464856},{"type":"v1","addr":"172.21.15.97:6841","nonce":1672464856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.177:6833/3787981632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3787981632},{"type":"v1","addr":"172.21.15.177:6833","nonce":3787981632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.177:6835/3349112132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":3349112132},{"type":"v1","addr":"172.21.15.177:6835","nonce":3349112132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6837/3244432640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":3244432640},{"type":"v1","addr":"172.21.15.177:6837","nonce":3244432640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6839/3340454605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":3340454605},{"type":"v1","addr":"172.21.15.177:6839","nonce":3340454605}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34415":{"gid":34415,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6833/4052517902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":4052517902},{"type":"v1","addr":"172.21.15.114:6833","nonce":4052517902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34445":{"gid":34445,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.114:6835/3581918222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3581918222},{"type":"v1","addr":"172.21.15.114:6835","nonce":3581918222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34478":{"gid":34478,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.114:6837/616775588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":616775588},{"type":"v1","addr":"172.21.15.114:6837","nonce":616775588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34508":{"gid":34508,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.114:6839/3937193784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":3937193784},{"type":"v1","addr":"172.21.15.114:6839","nonce":3937193784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34508,"qdb_cluster":[34478,24454,24400,34445,34508]},"id":1}]} 2024-03-27T10:57:41.708 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 28 2024-03-27T10:57:41.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.739+0000 7fa47fc00640 1 -- 172.21.15.97:0/1674402243 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fa4600a3690 msgr2=0x7fa4600a5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:41.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.739+0000 7fa47fc00640 1 --2- 172.21.15.97:0/1674402243 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fa4600a3690 0x7fa4600a5b50 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa464000c00 tx=0x7fa46402c7f0 comp rx=0 tx=0).stop 2024-03-27T10:57:41.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.739+0000 7fa47fc00640 1 -- 172.21.15.97:0/1674402243 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4781003c0 msgr2=0x7fa47819e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:41.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.739+0000 7fa47fc00640 1 --2- 172.21.15.97:0/1674402243 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4781003c0 0x7fa47819e330 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fa46c00e9f0 tx=0x7fa46c00eeb0 comp rx=0 tx=0).stop 2024-03-27T10:57:41.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.739+0000 7fa47fc00640 1 -- 172.21.15.97:0/1674402243 shutdown_connections 2024-03-27T10:57:41.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.739+0000 7fa47fc00640 1 --2- 172.21.15.97:0/1674402243 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa4781056e0 0x7fa4781a58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:41.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.739+0000 7fa47fc00640 1 --2- 172.21.15.97:0/1674402243 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fa4600a3690 0x7fa4600a5b50 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:41.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.739+0000 7fa47fc00640 1 --2- 172.21.15.97:0/1674402243 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa478102d30 0x7fa47819e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:41.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.739+0000 7fa47fc00640 1 --2- 172.21.15.97:0/1674402243 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4781003c0 0x7fa47819e330 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:41.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.739+0000 7fa47fc00640 1 -- 172.21.15.97:0/1674402243 >> 172.21.15.97:0/1674402243 conn(0x7fa4780fbfc0 msgr2=0x7fa478100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:41.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.740+0000 7fa47fc00640 1 -- 172.21.15.97:0/1674402243 shutdown_connections 2024-03-27T10:57:41.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.740+0000 7fa47fc00640 1 -- 172.21.15.97:0/1674402243 wait complete. 2024-03-27T10:57:41.727 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-27T10:57:41.766 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:57:41 smithi114 ceph-mon[35338]: pgmap v472: 97 pgs: 97 active+clean; 664 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-27T10:57:41.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.932+0000 7f1cc4a00640 1 -- 172.21.15.97:0/3562500659 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1cc0104de0 msgr2=0x7f1cc01071d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:41.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.932+0000 7f1cc4a00640 1 --2- 172.21.15.97:0/3562500659 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1cc0104de0 0x7f1cc01071d0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f1cac009900 tx=0x7f1cac02bc30 comp rx=0 tx=0).stop 2024-03-27T10:57:41.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.932+0000 7f1cc4a00640 1 -- 172.21.15.97:0/3562500659 shutdown_connections 2024-03-27T10:57:41.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.932+0000 7f1cc4a00640 1 --2- 172.21.15.97:0/3562500659 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1cc0107710 0x7f1cc0109b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:41.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.932+0000 7f1cc4a00640 1 --2- 172.21.15.97:0/3562500659 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1cc0104de0 0x7f1cc01071d0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:41.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.932+0000 7f1cc4a00640 1 --2- 172.21.15.97:0/3562500659 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1cc01024b0 0x7f1cc01048a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:41.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.932+0000 7f1cc4a00640 1 -- 172.21.15.97:0/3562500659 >> 172.21.15.97:0/3562500659 conn(0x7f1cc00fbec0 msgr2=0x7f1cc00fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:41.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.932+0000 7f1cc4a00640 1 -- 172.21.15.97:0/3562500659 shutdown_connections 2024-03-27T10:57:41.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.932+0000 7f1cc4a00640 1 -- 172.21.15.97:0/3562500659 wait complete. 2024-03-27T10:57:41.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.934+0000 7f1cc4a00640 1 Processor -- start 2024-03-27T10:57:41.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.934+0000 7f1cc4a00640 1 -- start start 2024-03-27T10:57:41.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.934+0000 7f1cc4a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1cc01024b0 0x7f1cc0197cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:41.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.934+0000 7f1cc4a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1cc0104de0 0x7f1cc0198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:41.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.934+0000 7f1cc4a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1cc0107710 0x7f1cc019f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:41.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.934+0000 7f1cc4a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc010d5a0 con 0x7f1cc0107710 2024-03-27T10:57:41.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.934+0000 7f1cc4a00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc010d420 con 0x7f1cc01024b0 2024-03-27T10:57:41.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.934+0000 7f1cc4a00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc010d720 con 0x7f1cc0104de0 2024-03-27T10:57:41.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.935+0000 7f1cbfe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1cc0107710 0x7f1cc019f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:41.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.935+0000 7f1cbfe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1cc0107710 0x7f1cc019f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53378/0 (socket says 172.21.15.97:53378) 2024-03-27T10:57:41.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.935+0000 7f1cbfe00640 1 -- 172.21.15.97:0/3715321228 learned_addr learned my addr 172.21.15.97:0/3715321228 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:41.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.935+0000 7f1cbf400640 1 --2- 172.21.15.97:0/3715321228 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1cc01024b0 0x7f1cc0197cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:41.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.935+0000 7f1cbea00640 1 --2- 172.21.15.97:0/3715321228 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1cc0104de0 0x7f1cc0198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:41.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.935+0000 7f1cbfe00640 1 -- 172.21.15.97:0/3715321228 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1cc0104de0 msgr2=0x7f1cc0198230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:41.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.935+0000 7f1cbfe00640 1 --2- 172.21.15.97:0/3715321228 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1cc0104de0 0x7f1cc0198230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:41.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.935+0000 7f1cbfe00640 1 -- 172.21.15.97:0/3715321228 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1cc01024b0 msgr2=0x7f1cc0197cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:41.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.935+0000 7f1cbfe00640 1 --2- 172.21.15.97:0/3715321228 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1cc01024b0 0x7f1cc0197cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:41.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.935+0000 7f1cbfe00640 1 -- 172.21.15.97:0/3715321228 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cc01002a0 con 0x7f1cc0107710 2024-03-27T10:57:41.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.936+0000 7f1cbfe00640 1 --2- 172.21.15.97:0/3715321228 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1cc0107710 0x7f1cc019f260 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f1cb800eaa0 tx=0x7f1cb800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:41.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.936+0000 7f1cb7e00640 1 -- 172.21.15.97:0/3715321228 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb800cdb0 con 0x7f1cc0107710 2024-03-27T10:57:41.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.936+0000 7f1cb7e00640 1 -- 172.21.15.97:0/3715321228 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1cb8005020 con 0x7f1cc0107710 2024-03-27T10:57:41.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.936+0000 7f1cb7e00640 1 -- 172.21.15.97:0/3715321228 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb80116d0 con 0x7f1cc0107710 2024-03-27T10:57:41.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.936+0000 7f1cc4a00640 1 -- 172.21.15.97:0/3715321228 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cc0100590 con 0x7f1cc0107710 2024-03-27T10:57:41.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.936+0000 7f1cc4a00640 1 -- 172.21.15.97:0/3715321228 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cc0100ae0 con 0x7f1cc0107710 2024-03-27T10:57:41.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.940+0000 7f1cc8d64740 1 Processor -- start 2024-03-27T10:57:41.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.941+0000 7f1cb7e00640 1 -- 172.21.15.97:0/3715321228 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f1cb80051d0 con 0x7f1cc0107710 2024-03-27T10:57:41.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.941+0000 7f1cc8d64740 1 -- start start 2024-03-27T10:57:41.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.942+0000 7f1cc8d64740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cc247ffab0 0x55cc247ffed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:41.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.942+0000 7f1cc8d64740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc248004a0 0x55cc24804fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:41.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.942+0000 7f1cc8d64740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cc24805610 0x55cc24807aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:41.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.942+0000 7f1cc8d64740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55cc247c77b0 con 0x55cc248004a0 2024-03-27T10:57:41.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.942+0000 7f1cc8d64740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55cc24764670 con 0x55cc247ffab0 2024-03-27T10:57:41.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.942+0000 7f1cc8d64740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55cc2470c970 con 0x55cc24805610 2024-03-27T10:57:41.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.942+0000 7f1cbea00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cc247ffab0 0x55cc247ffed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:41.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.942+0000 7f1cbea00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cc247ffab0 0x55cc247ffed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:33682/0 (socket says 172.21.15.97:33682) 2024-03-27T10:57:41.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.942+0000 7f1cbea00640 1 -- 172.21.15.97:0/706634694 learned_addr learned my addr 172.21.15.97:0/706634694 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:41.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.942+0000 7f1cbea00640 1 --2- 172.21.15.97:0/706634694 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cc24805610 0x55cc24807aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:41.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.943+0000 7f1cbea00640 1 -- 172.21.15.97:0/706634694 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cc24805610 msgr2=0x55cc24807aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:41.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.943+0000 7f1cbea00640 1 --2- 172.21.15.97:0/706634694 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cc24805610 0x55cc24807aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:41.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.943+0000 7f1cbea00640 1 -- 172.21.15.97:0/706634694 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc248004a0 msgr2=0x55cc24804fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:41.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.943+0000 7f1cbea00640 1 --2- 172.21.15.97:0/706634694 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc248004a0 0x55cc24804fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:41.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.943+0000 7f1cbea00640 1 -- 172.21.15.97:0/706634694 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc244783f0 con 0x55cc247ffab0 2024-03-27T10:57:41.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.943+0000 7f1cbea00640 1 --2- 172.21.15.97:0/706634694 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cc247ffab0 0x55cc247ffed0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1cac0096c0 tx=0x7f1cac02c5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e569a0bf5dcdf1aa server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:41.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.944+0000 7f1cabe00640 1 -- 172.21.15.97:0/706634694 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cac004030 con 0x55cc247ffab0 2024-03-27T10:57:41.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.944+0000 7f1cabe00640 1 -- 172.21.15.97:0/706634694 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1cac005040 con 0x55cc247ffab0 2024-03-27T10:57:41.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.944+0000 7f1cabe00640 1 -- 172.21.15.97:0/706634694 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cac004800 con 0x55cc247ffab0 2024-03-27T10:57:41.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.944+0000 7f1cb7e00640 1 --2- 172.21.15.97:0/3715321228 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f1ca40a3760 0x7f1ca40a5c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:41.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.945+0000 7f1cc8d64740 1 -- 172.21.15.97:0/706634694 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cc247ffab0 msgr2=0x55cc247ffed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:41.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.945+0000 7f1cc8d64740 1 --2- 172.21.15.97:0/706634694 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cc247ffab0 0x55cc247ffed0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1cac0096c0 tx=0x7f1cac02c5f0 comp rx=0 tx=0).stop 2024-03-27T10:57:41.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.945+0000 7f1cb7e00640 1 -- 172.21.15.97:0/3715321228 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f1cb80cfe40 con 0x7f1cc0107710 2024-03-27T10:57:41.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.945+0000 7f1cc8d64740 1 -- 172.21.15.97:0/706634694 shutdown_connections 2024-03-27T10:57:41.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.945+0000 7f1cc8d64740 1 --2- 172.21.15.97:0/706634694 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cc24805610 0x55cc24807aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:41.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.945+0000 7f1cc8d64740 1 --2- 172.21.15.97:0/706634694 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc248004a0 0x55cc24804fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:41.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.945+0000 7f1cc8d64740 1 --2- 172.21.15.97:0/706634694 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cc247ffab0 0x55cc247ffed0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:41.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.945+0000 7f1cc8d64740 1 -- 172.21.15.97:0/706634694 >> 172.21.15.97:0/706634694 conn(0x55cc247fb020 msgr2=0x55cc247fb420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:41.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.945+0000 7f1cbf400640 1 --2- 172.21.15.97:0/3715321228 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f1ca40a3760 0x7f1ca40a5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:41.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.946+0000 7f1cc8d64740 1 -- 172.21.15.97:0/706634694 shutdown_connections 2024-03-27T10:57:41.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.946+0000 7f1cc8d64740 1 -- 172.21.15.97:0/706634694 wait complete. 2024-03-27T10:57:41.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.946+0000 7f1cc8d64740 1 -- start start 2024-03-27T10:57:41.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.946+0000 7f1cc8d64740 1 Processor -- start 2024-03-27T10:57:41.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.947+0000 7f1cbf400640 1 --2- 172.21.15.97:0/3715321228 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f1ca40a3760 0x7f1ca40a5c20 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x55cc247fbf90 tx=0x7f1cb00046c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:41.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.947+0000 7f1cc8d64740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cc247ffab0 0x55cc24809c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:41.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.947+0000 7f1cc8d64740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cc248004a0 0x55cc2480a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:41.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.947+0000 7f1cc8d64740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc24805610 0x55cc248111c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:41.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.947+0000 7f1cc8d64740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55cc2471fca0 con 0x55cc24805610 2024-03-27T10:57:41.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.947+0000 7f1cc8d64740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55cc24760620 con 0x55cc248004a0 2024-03-27T10:57:41.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.947+0000 7f1cc8d64740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55cc24808560 con 0x55cc247ffab0 2024-03-27T10:57:41.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.947+0000 7f1cc8d64740 10 client.0 _ll_register_callbacks cb 0x55cc2475c160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:57:41.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.948+0000 7f1cbfe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc24805610 0x55cc248111c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:41.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.948+0000 7f1cbea00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cc248004a0 0x55cc2480a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:41.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.948+0000 7f1cbf400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cc247ffab0 0x55cc24809c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:41.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.948+0000 7f1cbea00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cc248004a0 0x55cc2480a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:33688/0 (socket says 172.21.15.97:33688) 2024-03-27T10:57:41.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.948+0000 7f1cbea00640 1 -- 172.21.15.97:0/1886464297 learned_addr learned my addr 172.21.15.97:0/1886464297 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:41.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.948+0000 7f1cbea00640 1 -- 172.21.15.97:0/1886464297 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cc247ffab0 msgr2=0x55cc24809c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:41.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.948+0000 7f1cbea00640 1 --2- 172.21.15.97:0/1886464297 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cc247ffab0 0x55cc24809c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:41.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.948+0000 7f1cbea00640 1 -- 172.21.15.97:0/1886464297 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc24805610 msgr2=0x55cc248111c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:41.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.948+0000 7f1cbea00640 1 --2- 172.21.15.97:0/1886464297 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc24805610 0x55cc248111c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:41.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.948+0000 7f1cbea00640 1 -- 172.21.15.97:0/1886464297 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc245427d0 con 0x55cc248004a0 2024-03-27T10:57:41.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.948+0000 7f1cbfe00640 1 --2- 172.21.15.97:0/1886464297 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc24805610 0x55cc248111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T10:57:41.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.949+0000 7f1cbea00640 1 --2- 172.21.15.97:0/1886464297 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cc248004a0 0x55cc2480a1c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1cac036040 tx=0x7f1cac005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3f86458412962f1 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:41.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.949+0000 7f1ca8c00640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-03-27T10:57:41.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.949+0000 7f1ca8c00640 1 -- 172.21.15.97:0/1886464297 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cac0047a0 con 0x55cc248004a0 2024-03-27T10:57:41.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.949+0000 7f1cc8d64740 1 -- 172.21.15.97:0/1886464297 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1cac005040 con 0x55cc248004a0 2024-03-27T10:57:41.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.949+0000 7f1ca8c00640 1 -- 172.21.15.97:0/1886464297 <== mon.2 v2:172.21.15.114:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f1cac004940 con 0x55cc248004a0 2024-03-27T10:57:41.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.949+0000 7f1ca8c00640 1 -- 172.21.15.97:0/1886464297 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1cac004b40 con 0x55cc248004a0 2024-03-27T10:57:41.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.949+0000 7f1cc8d64740 10 client.35426 fetch_fsmap learned FSMap version 28 2024-03-27T10:57:41.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.949+0000 7f1cc8d64740 1 -- 172.21.15.97:0/1886464297 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55cc24814550 con 0x55cc248004a0 2024-03-27T10:57:41.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.950+0000 7f1ca8c00640 1 -- 172.21.15.97:0/1886464297 <== mon.2 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1cac005040 con 0x55cc248004a0 2024-03-27T10:57:41.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.950+0000 7f1ca8c00640 1 -- 172.21.15.97:0/1886464297 <== mon.2 v2:172.21.15.114:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f1cac04c070 con 0x55cc248004a0 2024-03-27T10:57:41.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.950+0000 7f1cc8d64740 10 client.35426 fetch_fsmap finished waiting for FSMap version 28 2024-03-27T10:57:41.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.950+0000 7f1cc8d64740 10 client.35426 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-03-27T10:57:41.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.950+0000 7f1cc8d64740 20 client.35426 populate_metadata read hostname 'smithi097' 2024-03-27T10:57:41.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.950+0000 7f1cc8d64740 1 --2- 172.21.15.97:0/1886464297 >> [v2:172.21.15.177:6832/3787981632,v1:172.21.15.177:6833/3787981632] conn(0x55cc248158a0 0x55cc24817c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:41.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.950+0000 7f1cc8d64740 4 client.35426 mds_command: new command op to 24400 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T10:57:41.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.950+0000 7f1cc8d64740 1 -- 172.21.15.97:0/1886464297 --> [v2:172.21.15.177:6832/3787981632,v1:172.21.15.177:6833/3787981632] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cc24771bb0 con 0x55cc248158a0 2024-03-27T10:57:41.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.951+0000 7f1cbf400640 1 --2- 172.21.15.97:0/1886464297 >> [v2:172.21.15.177:6832/3787981632,v1:172.21.15.177:6833/3787981632] conn(0x55cc248158a0 0x55cc24817c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:41.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.952+0000 7f1cbf400640 1 --2- 172.21.15.97:0/1886464297 >> [v2:172.21.15.177:6832/3787981632,v1:172.21.15.177:6833/3787981632] conn(0x55cc248158a0 0x55cc24817c90 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c1f087b3a4fb6bcf server_cookie=509b0bd223df39fb in_seq=0 out_seq=0 2024-03-27T10:57:41.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.952+0000 7f1ca8c00640 10 client.35426 ms_handle_connect on v2:172.21.15.177:6832/3787981632 2024-03-27T10:57:41.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.953+0000 7f1ca8c00640 1 -- 172.21.15.97:0/1886464297 <== mds.4 v2:172.21.15.177:6832/3787981632 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55cc24771bb0 con 0x55cc248158a0 2024-03-27T10:57:41.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.953+0000 7f1ca8c00640 10 client.35426 handle_command_reply: tid=0 2024-03-27T10:57:41.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.954+0000 7f1cc8d64740 1 client.35426 shutdown 2024-03-27T10:57:41.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.954+0000 7f1cc8d64740 1 -- 172.21.15.97:0/1886464297 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cc248004a0 msgr2=0x55cc2480a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:41.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.954+0000 7f1cc8d64740 1 --2- 172.21.15.97:0/1886464297 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cc248004a0 0x55cc2480a1c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1cac036040 tx=0x7f1cac005fd0 comp rx=0 tx=0).stop 2024-03-27T10:57:41.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.954+0000 7f1cc8d64740 1 -- 172.21.15.97:0/1886464297 shutdown_connections 2024-03-27T10:57:41.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.954+0000 7f1cc8d64740 1 --2- 172.21.15.97:0/1886464297 >> [v2:172.21.15.177:6832/3787981632,v1:172.21.15.177:6833/3787981632] conn(0x55cc248158a0 0x55cc24817c90 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:41.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.954+0000 7f1cc8d64740 1 --2- 172.21.15.97:0/1886464297 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc24805610 0x55cc248111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:41.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.954+0000 7f1ca8c00640 0 client.35426 ms_handle_reset on v2:172.21.15.177:6832/3787981632 2024-03-27T10:57:41.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.954+0000 7f1cc8d64740 1 --2- 172.21.15.97:0/1886464297 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cc248004a0 0x55cc2480a1c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:41.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.954+0000 7f1cc8d64740 1 --2- 172.21.15.97:0/1886464297 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cc247ffab0 0x55cc24809c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:41.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.954+0000 7f1cc8d64740 1 -- 172.21.15.97:0/1886464297 >> 172.21.15.97:0/1886464297 conn(0x55cc247fb020 msgr2=0x55cc247fa240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:41.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.954+0000 7f1cc8d64740 1 -- 172.21.15.97:0/1886464297 shutdown_connections 2024-03-27T10:57:41.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.954+0000 7f1cc8d64740 1 -- 172.21.15.97:0/1886464297 wait complete. 2024-03-27T10:57:41.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.954+0000 7f1cc8d64740 20 client.35426 trim_cache size 0 max 16384 2024-03-27T10:57:41.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.968+0000 7f1cc8d64740 1 Processor -- start 2024-03-27T10:57:41.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.968+0000 7f1cc8d64740 1 -- start start 2024-03-27T10:57:41.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.969+0000 7f1cc8d64740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc247fb020 0x55cc247f60d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:41.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.969+0000 7f1cc8d64740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cc247ffab0 0x55cc24811180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:41.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.969+0000 7f1cc8d64740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cc24805610 0x55cc247fe990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:41.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.969+0000 7f1cc8d64740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc01aaec0 con 0x55cc247fb020 2024-03-27T10:57:41.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.969+0000 7f1cc8d64740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc01aca00 con 0x55cc24805610 2024-03-27T10:57:41.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.969+0000 7f1cc8d64740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc01ab590 con 0x55cc247ffab0 2024-03-27T10:57:41.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.969+0000 7f1cbf400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc247fb020 0x55cc247f60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:41.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.969+0000 7f1cbf400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc247fb020 0x55cc247f60d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53402/0 (socket says 172.21.15.97:53402) 2024-03-27T10:57:41.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.969+0000 7f1cbf400640 1 -- 172.21.15.97:0/3028982567 learned_addr learned my addr 172.21.15.97:0/3028982567 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:41.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.969+0000 7f1cbea00640 1 --2- 172.21.15.97:0/3028982567 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cc247ffab0 0x55cc24811180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:41.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.969+0000 7f1cbfe00640 1 --2- 172.21.15.97:0/3028982567 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cc24805610 0x55cc247fe990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:41.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.970+0000 7f1cbf400640 1 -- 172.21.15.97:0/3028982567 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cc247ffab0 msgr2=0x55cc24811180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:41.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.970+0000 7f1cbf400640 1 --2- 172.21.15.97:0/3028982567 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cc247ffab0 0x55cc24811180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:41.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.970+0000 7f1cbf400640 1 -- 172.21.15.97:0/3028982567 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cc24805610 msgr2=0x55cc247fe990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:41.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.970+0000 7f1cbf400640 1 --2- 172.21.15.97:0/3028982567 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cc24805610 0x55cc247fe990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:41.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.970+0000 7f1cbf400640 1 -- 172.21.15.97:0/3028982567 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc247f68a0 con 0x55cc247fb020 2024-03-27T10:57:41.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.970+0000 7f1cbfe00640 1 --2- 172.21.15.97:0/3028982567 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cc24805610 0x55cc247fe990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T10:57:41.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.970+0000 7f1cbf400640 1 --2- 172.21.15.97:0/3028982567 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc247fb020 0x55cc247f60d0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x55cc248008a0 tx=0x7f1cb0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e472594a8cd8360 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:41.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.971+0000 7f1cab400640 1 -- 172.21.15.97:0/3028982567 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb0027260 con 0x55cc247fb020 2024-03-27T10:57:41.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.971+0000 7f1cab400640 1 -- 172.21.15.97:0/3028982567 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1cb002d070 con 0x55cc247fb020 2024-03-27T10:57:41.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.971+0000 7f1cab400640 1 -- 172.21.15.97:0/3028982567 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb001e420 con 0x55cc247fb020 2024-03-27T10:57:41.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.971+0000 7f1cc8d64740 1 -- 172.21.15.97:0/3028982567 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc247fb020 msgr2=0x55cc247f60d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:41.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.972+0000 7f1cc8d64740 1 --2- 172.21.15.97:0/3028982567 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc247fb020 0x55cc247f60d0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x55cc248008a0 tx=0x7f1cb0014000 comp rx=0 tx=0).stop 2024-03-27T10:57:41.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.972+0000 7f1cc8d64740 1 -- 172.21.15.97:0/3028982567 shutdown_connections 2024-03-27T10:57:41.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.972+0000 7f1cc8d64740 1 --2- 172.21.15.97:0/3028982567 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cc24805610 0x55cc247fe990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:41.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.972+0000 7f1cc8d64740 1 --2- 172.21.15.97:0/3028982567 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cc247ffab0 0x55cc24811180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:41.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.972+0000 7f1cc8d64740 1 --2- 172.21.15.97:0/3028982567 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc247fb020 0x55cc247f60d0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:41.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.972+0000 7f1cc8d64740 1 -- 172.21.15.97:0/3028982567 >> 172.21.15.97:0/3028982567 conn(0x55cc24809610 msgr2=0x55cc247fe280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:41.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.972+0000 7f1cc8d64740 1 -- 172.21.15.97:0/3028982567 shutdown_connections 2024-03-27T10:57:41.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.972+0000 7f1cc8d64740 1 -- 172.21.15.97:0/3028982567 wait complete. 2024-03-27T10:57:41.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.972+0000 7f1cc8d64740 1 -- start start 2024-03-27T10:57:41.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.972+0000 7f1cc8d64740 1 Processor -- start 2024-03-27T10:57:41.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.973+0000 7f1cc8d64740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc247fb020 0x55cc2480ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:41.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.973+0000 7f1cc8d64740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cc247ffab0 0x55cc2480f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:41.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.973+0000 7f1cc8d64740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cc24805610 0x55cc24810450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:41.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.973+0000 7f1cc8d64740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc01ac330 con 0x55cc247fb020 2024-03-27T10:57:41.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.973+0000 7f1cc8d64740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc01abc60 con 0x55cc24805610 2024-03-27T10:57:41.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.973+0000 7f1cc8d64740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc01a8ea0 con 0x55cc247ffab0 2024-03-27T10:57:41.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.973+0000 7f1cc8d64740 10 client.0 _ll_register_callbacks cb 0x7f1cac02ca90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:57:41.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.973+0000 7f1cbf400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc247fb020 0x55cc2480ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:41.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.973+0000 7f1cbf400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc247fb020 0x55cc2480ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53410/0 (socket says 172.21.15.97:53410) 2024-03-27T10:57:41.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.973+0000 7f1cbf400640 1 -- 172.21.15.97:0/2564863968 learned_addr learned my addr 172.21.15.97:0/2564863968 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:41.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.974+0000 7f1cbfe00640 1 --2- 172.21.15.97:0/2564863968 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cc24805610 0x55cc24810450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:41.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.974+0000 7f1cbea00640 1 --2- 172.21.15.97:0/2564863968 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cc247ffab0 0x55cc2480f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:41.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.974+0000 7f1cbf400640 1 -- 172.21.15.97:0/2564863968 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cc247ffab0 msgr2=0x55cc2480f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:41.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.974+0000 7f1cbf400640 1 --2- 172.21.15.97:0/2564863968 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cc247ffab0 0x55cc2480f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:41.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.974+0000 7f1cbf400640 1 -- 172.21.15.97:0/2564863968 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cc24805610 msgr2=0x55cc24810450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:41.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.974+0000 7f1cbf400640 1 --2- 172.21.15.97:0/2564863968 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cc24805610 0x55cc24810450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:41.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.974+0000 7f1cbf400640 1 -- 172.21.15.97:0/2564863968 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc24810a40 con 0x55cc247fb020 2024-03-27T10:57:41.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.974+0000 7f1cbf400640 1 --2- 172.21.15.97:0/2564863968 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc247fb020 0x55cc2480ef00 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f1cb0003e20 tx=0x7f1cb0003e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64fd1ab635d68a71 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:41.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.974+0000 7f1ca8c00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T10:57:41.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.975+0000 7f1ca8c00640 1 -- 172.21.15.97:0/2564863968 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb0035020 con 0x55cc247fb020 2024-03-27T10:57:41.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.975+0000 7f1ca8c00640 1 -- 172.21.15.97:0/2564863968 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f1cb00036b0 con 0x55cc247fb020 2024-03-27T10:57:41.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.975+0000 7f1cc8d64740 1 -- 172.21.15.97:0/2564863968 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1cb002d070 con 0x55cc247fb020 2024-03-27T10:57:41.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.975+0000 7f1ca8c00640 1 -- 172.21.15.97:0/2564863968 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1cb00038f0 con 0x55cc247fb020 2024-03-27T10:57:41.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.975+0000 7f1cc8d64740 10 client.15768 fetch_fsmap learned FSMap version 28 2024-03-27T10:57:41.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.975+0000 7f1cc8d64740 1 -- 172.21.15.97:0/2564863968 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55cc2487c0b0 con 0x55cc247fb020 2024-03-27T10:57:41.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.976+0000 7f1ca8c00640 1 -- 172.21.15.97:0/2564863968 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1cb002d070 con 0x55cc247fb020 2024-03-27T10:57:41.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.976+0000 7f1ca8c00640 1 -- 172.21.15.97:0/2564863968 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f1cb0042020 con 0x55cc247fb020 2024-03-27T10:57:41.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.976+0000 7f1cc8d64740 10 client.15768 fetch_fsmap finished waiting for FSMap version 28 2024-03-27T10:57:41.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.976+0000 7f1cc8d64740 10 client.15768 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-03-27T10:57:41.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.976+0000 7f1cc8d64740 20 client.15768 populate_metadata read hostname 'smithi097' 2024-03-27T10:57:41.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.976+0000 7f1cc8d64740 1 --2- 172.21.15.97:0/2564863968 >> [v2:172.21.15.177:6832/3787981632,v1:172.21.15.177:6833/3787981632] conn(0x55cc2487cab0 0x55cc2487eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:41.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.976+0000 7f1cc8d64740 4 client.15768 mds_command: new command op to 24400 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-27T10:57:41.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.976+0000 7f1cc8d64740 1 -- 172.21.15.97:0/2564863968 --> [v2:172.21.15.177:6832/3787981632,v1:172.21.15.177:6833/3787981632] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cc2476bf00 con 0x55cc2487cab0 2024-03-27T10:57:41.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.977+0000 7f1cbea00640 1 --2- 172.21.15.97:0/2564863968 >> [v2:172.21.15.177:6832/3787981632,v1:172.21.15.177:6833/3787981632] conn(0x55cc2487cab0 0x55cc2487eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:41.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.978+0000 7f1cbea00640 1 --2- 172.21.15.97:0/2564863968 >> [v2:172.21.15.177:6832/3787981632,v1:172.21.15.177:6833/3787981632] conn(0x55cc2487cab0 0x55cc2487eed0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=738a0fda0c06e1c6 server_cookie=f0d36a0fcc66a1ca in_seq=0 out_seq=0 2024-03-27T10:57:41.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.978+0000 7f1ca8c00640 10 client.15768 ms_handle_connect on v2:172.21.15.177:6832/3787981632 2024-03-27T10:57:41.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.978+0000 7f1ca8c00640 1 -- 172.21.15.97:0/2564863968 <== mds.4 v2:172.21.15.177:6832/3787981632 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cc2476bf00 con 0x55cc2487cab0 2024-03-27T10:57:41.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.978+0000 7f1ca8c00640 10 client.15768 handle_command_reply: tid=0 2024-03-27T10:57:41.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.978+0000 7f1cc8d64740 1 client.15768 shutdown 2024-03-27T10:57:41.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.978+0000 7f1cc8d64740 1 -- 172.21.15.97:0/2564863968 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc247fb020 msgr2=0x55cc2480ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:41.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.978+0000 7f1cc8d64740 1 --2- 172.21.15.97:0/2564863968 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc247fb020 0x55cc2480ef00 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f1cb0003e20 tx=0x7f1cb0003e50 comp rx=0 tx=0).stop 2024-03-27T10:57:41.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.979+0000 7f1cc8d64740 1 -- 172.21.15.97:0/2564863968 shutdown_connections 2024-03-27T10:57:41.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.979+0000 7f1cc8d64740 1 --2- 172.21.15.97:0/2564863968 >> [v2:172.21.15.177:6832/3787981632,v1:172.21.15.177:6833/3787981632] conn(0x55cc2487cab0 0x55cc2487eed0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:41.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.979+0000 7f1cc8d64740 1 --2- 172.21.15.97:0/2564863968 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cc24805610 0x55cc24810450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:41.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.979+0000 7f1cc8d64740 1 --2- 172.21.15.97:0/2564863968 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cc247ffab0 0x55cc2480f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:41.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.979+0000 7f1cc8d64740 1 --2- 172.21.15.97:0/2564863968 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc247fb020 0x55cc2480ef00 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:41.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.979+0000 7f1cc8d64740 1 -- 172.21.15.97:0/2564863968 >> 172.21.15.97:0/2564863968 conn(0x55cc24809610 msgr2=0x55cc247feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:41.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.979+0000 7f1ca8c00640 0 client.15768 ms_handle_reset on v2:172.21.15.177:6832/3787981632 2024-03-27T10:57:41.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.979+0000 7f1cc8d64740 1 -- 172.21.15.97:0/2564863968 shutdown_connections 2024-03-27T10:57:41.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.979+0000 7f1cc8d64740 1 -- 172.21.15.97:0/2564863968 wait complete. 2024-03-27T10:57:41.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.979+0000 7f1cc8d64740 20 client.15768 trim_cache size 0 max 16384 2024-03-27T10:57:41.952 INFO:teuthology.orchestra.run.smithi097.stdout:[] 2024-03-27T10:57:41.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.980+0000 7f1cc4a00640 1 -- 172.21.15.97:0/3715321228 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f1ca40a3760 msgr2=0x7f1ca40a5c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:41.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.980+0000 7f1cc4a00640 1 --2- 172.21.15.97:0/3715321228 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f1ca40a3760 0x7f1ca40a5c20 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x55cc247fbf90 tx=0x7f1cb00046c0 comp rx=0 tx=0).stop 2024-03-27T10:57:41.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.980+0000 7f1cc4a00640 1 -- 172.21.15.97:0/3715321228 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1cc0107710 msgr2=0x7f1cc019f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:41.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.980+0000 7f1cc4a00640 1 --2- 172.21.15.97:0/3715321228 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1cc0107710 0x7f1cc019f260 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f1cb800eaa0 tx=0x7f1cb800ef60 comp rx=0 tx=0).stop 2024-03-27T10:57:41.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.980+0000 7f1cc4a00640 1 -- 172.21.15.97:0/3715321228 shutdown_connections 2024-03-27T10:57:41.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.980+0000 7f1cc4a00640 1 --2- 172.21.15.97:0/3715321228 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1cc0107710 0x7f1cc019f260 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:41.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.980+0000 7f1cc4a00640 1 --2- 172.21.15.97:0/3715321228 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1cc0104de0 0x7f1cc0198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:41.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.980+0000 7f1cc4a00640 1 --2- 172.21.15.97:0/3715321228 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f1ca40a3760 0x7f1ca40a5c20 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:41.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.980+0000 7f1cc4a00640 1 --2- 172.21.15.97:0/3715321228 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1cc01024b0 0x7f1cc0197cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:41.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.980+0000 7f1cc4a00640 1 -- 172.21.15.97:0/3715321228 >> 172.21.15.97:0/3715321228 conn(0x7f1cc00fbec0 msgr2=0x7f1cc0102f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:41.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.980+0000 7f1cc4a00640 1 -- 172.21.15.97:0/3715321228 shutdown_connections 2024-03-27T10:57:41.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:41.980+0000 7f1cc4a00640 1 -- 172.21.15.97:0/3715321228 wait complete. 2024-03-27T10:57:41.967 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-27T10:57:42.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.173+0000 7f9362200640 1 -- 172.21.15.97:0/324834343 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f935c1024b0 msgr2=0x7f935c1048a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:42.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.173+0000 7f9362200640 1 --2- 172.21.15.97:0/324834343 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f935c1024b0 0x7f935c1048a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9350009860 tx=0x7f935002bb70 comp rx=0 tx=0).stop 2024-03-27T10:57:42.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.174+0000 7f9362200640 1 -- 172.21.15.97:0/324834343 shutdown_connections 2024-03-27T10:57:42.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.174+0000 7f9362200640 1 --2- 172.21.15.97:0/324834343 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f935c107710 0x7f935c109b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.174+0000 7f9362200640 1 --2- 172.21.15.97:0/324834343 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f935c104de0 0x7f935c1071d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.174+0000 7f9362200640 1 --2- 172.21.15.97:0/324834343 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f935c1024b0 0x7f935c1048a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.174+0000 7f9362200640 1 -- 172.21.15.97:0/324834343 >> 172.21.15.97:0/324834343 conn(0x7f935c0fbec0 msgr2=0x7f935c0fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:42.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.174+0000 7f9362200640 1 -- 172.21.15.97:0/324834343 shutdown_connections 2024-03-27T10:57:42.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.174+0000 7f9362200640 1 -- 172.21.15.97:0/324834343 wait complete. 2024-03-27T10:57:42.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.175+0000 7f9362200640 1 Processor -- start 2024-03-27T10:57:42.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.175+0000 7f9362200640 1 -- start start 2024-03-27T10:57:42.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.176+0000 7f9362200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f935c1024b0 0x7f935c197d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:42.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.176+0000 7f9362200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f935c104de0 0x7f935c198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:42.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.176+0000 7f9362200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f935c107710 0x7f935c19f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:42.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.176+0000 7f9362200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f935c10d5e0 con 0x7f935c107710 2024-03-27T10:57:42.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.176+0000 7f9362200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f935c10d460 con 0x7f935c104de0 2024-03-27T10:57:42.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.176+0000 7f9362200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f935c10d760 con 0x7f935c1024b0 2024-03-27T10:57:42.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.176+0000 7f9361800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f935c107710 0x7f935c19f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:42.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.176+0000 7f935be00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f935c104de0 0x7f935c198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:42.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.176+0000 7f935be00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f935c104de0 0x7f935c198270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:33724/0 (socket says 172.21.15.97:33724) 2024-03-27T10:57:42.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.176+0000 7f935be00640 1 -- 172.21.15.97:0/1914087164 learned_addr learned my addr 172.21.15.97:0/1914087164 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:42.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.176+0000 7f9361800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f935c107710 0x7f935c19f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53428/0 (socket says 172.21.15.97:53428) 2024-03-27T10:57:42.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.176+0000 7f9360e00640 1 --2- 172.21.15.97:0/1914087164 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f935c1024b0 0x7f935c197d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:42.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.177+0000 7f9361800640 1 -- 172.21.15.97:0/1914087164 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f935c1024b0 msgr2=0x7f935c197d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:42.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.177+0000 7f9361800640 1 --2- 172.21.15.97:0/1914087164 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f935c1024b0 0x7f935c197d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.177+0000 7f9361800640 1 -- 172.21.15.97:0/1914087164 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f935c104de0 msgr2=0x7f935c198270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:42.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.177+0000 7f9361800640 1 --2- 172.21.15.97:0/1914087164 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f935c104de0 0x7f935c198270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.177+0000 7f9361800640 1 -- 172.21.15.97:0/1914087164 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f935c1002a0 con 0x7f935c107710 2024-03-27T10:57:42.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.177+0000 7f9361800640 1 --2- 172.21.15.97:0/1914087164 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f935c107710 0x7f935c19f2a0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f935400efc0 tx=0x7f935400c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:42.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.177+0000 7f9359600640 1 -- 172.21.15.97:0/1914087164 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9354003970 con 0x7f935c107710 2024-03-27T10:57:42.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.178+0000 7f9359600640 1 -- 172.21.15.97:0/1914087164 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9354004020 con 0x7f935c107710 2024-03-27T10:57:42.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.178+0000 7f9359600640 1 -- 172.21.15.97:0/1914087164 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f935400b940 con 0x7f935c107710 2024-03-27T10:57:42.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.178+0000 7f9362200640 1 -- 172.21.15.97:0/1914087164 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f935c100590 con 0x7f935c107710 2024-03-27T10:57:42.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.178+0000 7f9362200640 1 -- 172.21.15.97:0/1914087164 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f935c100b60 con 0x7f935c107710 2024-03-27T10:57:42.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.181+0000 7f9359600640 1 -- 172.21.15.97:0/1914087164 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f935401d020 con 0x7f935c107710 2024-03-27T10:57:42.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.182+0000 7f9359600640 1 --2- 172.21.15.97:0/1914087164 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f93440a3690 0x7f93440a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:42.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.183+0000 7f9359600640 1 -- 172.21.15.97:0/1914087164 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f93540cfc10 con 0x7f935c107710 2024-03-27T10:57:42.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.183+0000 7f93665b9740 1 Processor -- start 2024-03-27T10:57:42.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.183+0000 7f93665b9740 1 -- start start 2024-03-27T10:57:42.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.183+0000 7f9360e00640 1 --2- 172.21.15.97:0/1914087164 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f93440a3690 0x7f93440a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:42.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.183+0000 7f93665b9740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d126907ab0 0x55d126907ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:42.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.183+0000 7f93665b9740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d1269084a0 0x55d12690cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:42.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.183+0000 7f93665b9740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d12690d610 0x55d12690faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:42.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.183+0000 7f93665b9740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55d1268cf7b0 con 0x55d1269084a0 2024-03-27T10:57:42.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.183+0000 7f93665b9740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55d12686c670 con 0x55d126907ab0 2024-03-27T10:57:42.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.183+0000 7f93665b9740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55d126814970 con 0x55d12690d610 2024-03-27T10:57:42.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.184+0000 7f9360e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d126907ab0 0x55d126907ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:42.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.184+0000 7f9360e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d126907ab0 0x55d126907ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:33726/0 (socket says 172.21.15.97:33726) 2024-03-27T10:57:42.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.184+0000 7f9360e00640 1 -- 172.21.15.97:0/1785064065 learned_addr learned my addr 172.21.15.97:0/1785064065 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:42.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.184+0000 7f9361800640 1 --2- 172.21.15.97:0/1785064065 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d12690d610 0x55d12690faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:42.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.184+0000 7f9360e00640 1 --2- 172.21.15.97:0/1914087164 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f93440a3690 0x7f93440a5b50 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f9350009830 tx=0x7f9350034980 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:42.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.184+0000 7f935be00640 1 --2- 172.21.15.97:0/1785064065 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d1269084a0 0x55d12690cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:42.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.184+0000 7f9360e00640 1 -- 172.21.15.97:0/1785064065 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d12690d610 msgr2=0x55d12690faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:42.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.184+0000 7f9360e00640 1 --2- 172.21.15.97:0/1785064065 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d12690d610 0x55d12690faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.184+0000 7f9360e00640 1 -- 172.21.15.97:0/1785064065 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d1269084a0 msgr2=0x55d12690cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:42.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.184+0000 7f9360e00640 1 --2- 172.21.15.97:0/1785064065 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d1269084a0 0x55d12690cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.184+0000 7f9360e00640 1 -- 172.21.15.97:0/1785064065 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1265803f0 con 0x55d126907ab0 2024-03-27T10:57:42.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.185+0000 7f9360e00640 1 --2- 172.21.15.97:0/1785064065 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d126907ab0 0x55d126907ed0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9350009860 tx=0x7f935003e000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dca5f5d9b4cb11d6 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:42.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.185+0000 7f934e000640 1 -- 172.21.15.97:0/1785064065 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9350039040 con 0x55d126907ab0 2024-03-27T10:57:42.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.185+0000 7f934e000640 1 -- 172.21.15.97:0/1785064065 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f935004f070 con 0x55d126907ab0 2024-03-27T10:57:42.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.185+0000 7f934e000640 1 -- 172.21.15.97:0/1785064065 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9350003ab0 con 0x55d126907ab0 2024-03-27T10:57:42.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.186+0000 7f93665b9740 1 -- 172.21.15.97:0/1785064065 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d126907ab0 msgr2=0x55d126907ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:42.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.186+0000 7f93665b9740 1 --2- 172.21.15.97:0/1785064065 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d126907ab0 0x55d126907ed0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9350009860 tx=0x7f935003e000 comp rx=0 tx=0).stop 2024-03-27T10:57:42.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.186+0000 7f93665b9740 1 -- 172.21.15.97:0/1785064065 shutdown_connections 2024-03-27T10:57:42.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.186+0000 7f93665b9740 1 --2- 172.21.15.97:0/1785064065 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d12690d610 0x55d12690faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.186+0000 7f93665b9740 1 --2- 172.21.15.97:0/1785064065 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d1269084a0 0x55d12690cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.186+0000 7f93665b9740 1 --2- 172.21.15.97:0/1785064065 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d126907ab0 0x55d126907ed0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.186+0000 7f93665b9740 1 -- 172.21.15.97:0/1785064065 >> 172.21.15.97:0/1785064065 conn(0x55d126903020 msgr2=0x55d126903420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:42.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.187+0000 7f93665b9740 1 -- 172.21.15.97:0/1785064065 shutdown_connections 2024-03-27T10:57:42.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.187+0000 7f93665b9740 1 -- 172.21.15.97:0/1785064065 wait complete. 2024-03-27T10:57:42.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.187+0000 7f93665b9740 1 -- start start 2024-03-27T10:57:42.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.187+0000 7f93665b9740 1 Processor -- start 2024-03-27T10:57:42.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.188+0000 7f93665b9740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d126907ab0 0x55d126911c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:42.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.188+0000 7f93665b9740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d1269084a0 0x55d126912150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:42.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.188+0000 7f93665b9740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d12690d610 0x55d126919190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:42.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.188+0000 7f93665b9740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55d126827ca0 con 0x55d1269084a0 2024-03-27T10:57:42.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.188+0000 7f93665b9740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55d126868620 con 0x55d126907ab0 2024-03-27T10:57:42.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.188+0000 7f93665b9740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55d126910560 con 0x55d12690d610 2024-03-27T10:57:42.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.188+0000 7f93665b9740 10 client.0 _ll_register_callbacks cb 0x55d126864160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:57:42.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.188+0000 7f9360e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d126907ab0 0x55d126911c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:42.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.189+0000 7f9360e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d126907ab0 0x55d126911c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:33740/0 (socket says 172.21.15.97:33740) 2024-03-27T10:57:42.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.189+0000 7f9360e00640 1 -- 172.21.15.97:0/399677837 learned_addr learned my addr 172.21.15.97:0/399677837 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:42.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.189+0000 7f935be00640 1 --2- 172.21.15.97:0/399677837 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d1269084a0 0x55d126912150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:42.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.189+0000 7f9361800640 1 --2- 172.21.15.97:0/399677837 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d12690d610 0x55d126919190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:42.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.189+0000 7f9360e00640 1 -- 172.21.15.97:0/399677837 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d12690d610 msgr2=0x55d126919190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:42.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.189+0000 7f9360e00640 1 --2- 172.21.15.97:0/399677837 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d12690d610 0x55d126919190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.189+0000 7f9360e00640 1 -- 172.21.15.97:0/399677837 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d1269084a0 msgr2=0x55d126912150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:42.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.189+0000 7f9360e00640 1 --2- 172.21.15.97:0/399677837 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d1269084a0 0x55d126912150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.189+0000 7f9360e00640 1 -- 172.21.15.97:0/399677837 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d12664a7d0 con 0x55d126907ab0 2024-03-27T10:57:42.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.190+0000 7f9360e00640 1 --2- 172.21.15.97:0/399677837 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d126907ab0 0x55d126911c10 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f935002bbb0 tx=0x7f9350005fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc921ae56602992b server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:42.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.190+0000 7f9342a00640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-03-27T10:57:42.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.190+0000 7f9342a00640 1 -- 172.21.15.97:0/399677837 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9350039070 con 0x55d126907ab0 2024-03-27T10:57:42.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.190+0000 7f9342a00640 1 -- 172.21.15.97:0/399677837 <== mon.2 v2:172.21.15.114:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f935002c950 con 0x55d126907ab0 2024-03-27T10:57:42.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.190+0000 7f93665b9740 1 -- 172.21.15.97:0/399677837 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f935004f070 con 0x55d126907ab0 2024-03-27T10:57:42.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.190+0000 7f9342a00640 1 -- 172.21.15.97:0/399677837 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9350003830 con 0x55d126907ab0 2024-03-27T10:57:42.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.190+0000 7f93665b9740 10 client.35444 fetch_fsmap learned FSMap version 28 2024-03-27T10:57:42.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.190+0000 7f93665b9740 1 -- 172.21.15.97:0/399677837 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d12691c500 con 0x55d126907ab0 2024-03-27T10:57:42.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.191+0000 7f9342a00640 1 -- 172.21.15.97:0/399677837 <== mon.2 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f935004f070 con 0x55d126907ab0 2024-03-27T10:57:42.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.191+0000 7f9342a00640 1 -- 172.21.15.97:0/399677837 <== mon.2 v2:172.21.15.114:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f935004aa10 con 0x55d126907ab0 2024-03-27T10:57:42.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.191+0000 7f93665b9740 10 client.35444 fetch_fsmap finished waiting for FSMap version 28 2024-03-27T10:57:42.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.191+0000 7f93665b9740 10 client.35444 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-03-27T10:57:42.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.191+0000 7f93665b9740 20 client.35444 populate_metadata read hostname 'smithi097' 2024-03-27T10:57:42.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.191+0000 7f93665b9740 1 --2- 172.21.15.97:0/399677837 >> [v2:172.21.15.177:6834/3349112132,v1:172.21.15.177:6835/3349112132] conn(0x55d12691d850 0x55d12691fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:42.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.191+0000 7f93665b9740 4 client.35444 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T10:57:42.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.191+0000 7f93665b9740 1 -- 172.21.15.97:0/399677837 --> [v2:172.21.15.177:6834/3349112132,v1:172.21.15.177:6835/3349112132] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d126879bb0 con 0x55d12691d850 2024-03-27T10:57:42.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.192+0000 7f935be00640 1 --2- 172.21.15.97:0/399677837 >> [v2:172.21.15.177:6834/3349112132,v1:172.21.15.177:6835/3349112132] conn(0x55d12691d850 0x55d12691fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:42.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.193+0000 7f9342a00640 10 client.35444 ms_handle_connect on v2:172.21.15.177:6834/3349112132 2024-03-27T10:57:42.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.193+0000 7f935be00640 1 --2- 172.21.15.97:0/399677837 >> [v2:172.21.15.177:6834/3349112132,v1:172.21.15.177:6835/3349112132] conn(0x55d12691d850 0x55d12691fc40 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b5e8bbc1ad5d53dd server_cookie=ea87f17c5be24632 in_seq=0 out_seq=0 2024-03-27T10:57:42.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.195+0000 7f9342a00640 1 -- 172.21.15.97:0/399677837 <== mds.3 v2:172.21.15.177:6834/3349112132 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55d126879bb0 con 0x55d12691d850 2024-03-27T10:57:42.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.195+0000 7f9342a00640 10 client.35444 handle_command_reply: tid=0 2024-03-27T10:57:42.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.195+0000 7f93665b9740 1 client.35444 shutdown 2024-03-27T10:57:42.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.195+0000 7f93665b9740 1 -- 172.21.15.97:0/399677837 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d126907ab0 msgr2=0x55d126911c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:42.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.195+0000 7f93665b9740 1 --2- 172.21.15.97:0/399677837 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d126907ab0 0x55d126911c10 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f935002bbb0 tx=0x7f9350005fb0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.195+0000 7f93665b9740 1 -- 172.21.15.97:0/399677837 shutdown_connections 2024-03-27T10:57:42.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.195+0000 7f93665b9740 1 --2- 172.21.15.97:0/399677837 >> [v2:172.21.15.177:6834/3349112132,v1:172.21.15.177:6835/3349112132] conn(0x55d12691d850 0x55d12691fc40 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.195+0000 7f93665b9740 1 --2- 172.21.15.97:0/399677837 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d12690d610 0x55d126919190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.195+0000 7f9342a00640 0 client.35444 ms_handle_reset on v2:172.21.15.177:6834/3349112132 2024-03-27T10:57:42.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.195+0000 7f93665b9740 1 --2- 172.21.15.97:0/399677837 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d1269084a0 0x55d126912150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.195+0000 7f93665b9740 1 --2- 172.21.15.97:0/399677837 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d126907ab0 0x55d126911c10 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.195+0000 7f93665b9740 1 -- 172.21.15.97:0/399677837 >> 172.21.15.97:0/399677837 conn(0x55d126903020 msgr2=0x55d126902240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:42.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.196+0000 7f93665b9740 1 -- 172.21.15.97:0/399677837 shutdown_connections 2024-03-27T10:57:42.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.196+0000 7f93665b9740 1 -- 172.21.15.97:0/399677837 wait complete. 2024-03-27T10:57:42.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.196+0000 7f93665b9740 20 client.35444 trim_cache size 0 max 16384 2024-03-27T10:57:42.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.223+0000 7f93665b9740 1 Processor -- start 2024-03-27T10:57:42.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.223+0000 7f93665b9740 1 -- start start 2024-03-27T10:57:42.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.223+0000 7f93665b9740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d126907ab0 0x55d1269019b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:42.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.223+0000 7f93665b9740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d1269084a0 0x55d126919120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:42.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.223+0000 7f93665b9740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d12690d610 0x55d126901ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:42.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.223+0000 7f93665b9740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f935c1aaf90 con 0x55d126907ab0 2024-03-27T10:57:42.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.224+0000 7f93665b9740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f935c1acad0 con 0x55d12690d610 2024-03-27T10:57:42.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.224+0000 7f93665b9740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f935c1ab660 con 0x55d1269084a0 2024-03-27T10:57:42.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.224+0000 7f9360e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d126907ab0 0x55d1269019b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:42.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.224+0000 7f9360e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d126907ab0 0x55d1269019b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53472/0 (socket says 172.21.15.97:53472) 2024-03-27T10:57:42.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.224+0000 7f9360e00640 1 -- 172.21.15.97:0/621456716 learned_addr learned my addr 172.21.15.97:0/621456716 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:42.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.224+0000 7f9361800640 1 --2- 172.21.15.97:0/621456716 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d12690d610 0x55d126901ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:42.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.224+0000 7f935be00640 1 --2- 172.21.15.97:0/621456716 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d1269084a0 0x55d126919120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:42.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.224+0000 7f9360e00640 1 -- 172.21.15.97:0/621456716 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d1269084a0 msgr2=0x55d126919120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:42.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.224+0000 7f9360e00640 1 --2- 172.21.15.97:0/621456716 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d1269084a0 0x55d126919120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.224+0000 7f9360e00640 1 -- 172.21.15.97:0/621456716 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d12690d610 msgr2=0x55d126901ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:42.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.224+0000 7f9360e00640 1 --2- 172.21.15.97:0/621456716 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d12690d610 0x55d126901ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.224+0000 7f9360e00640 1 -- 172.21.15.97:0/621456716 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1269076f0 con 0x55d126907ab0 2024-03-27T10:57:42.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.224+0000 7f935be00640 1 --2- 172.21.15.97:0/621456716 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d1269084a0 0x55d126919120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:57:42.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.225+0000 7f9360e00640 1 --2- 172.21.15.97:0/621456716 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d126907ab0 0x55d1269019b0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f9350009860 tx=0x7f9350000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f15aa29225677de8 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:42.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.225+0000 7f934d600640 1 -- 172.21.15.97:0/621456716 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f935006a070 con 0x55d126907ab0 2024-03-27T10:57:42.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.225+0000 7f934d600640 1 -- 172.21.15.97:0/621456716 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f935004f040 con 0x55d126907ab0 2024-03-27T10:57:42.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.225+0000 7f934d600640 1 -- 172.21.15.97:0/621456716 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9350039070 con 0x55d126907ab0 2024-03-27T10:57:42.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.226+0000 7f93665b9740 1 -- 172.21.15.97:0/621456716 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d126907ab0 msgr2=0x55d1269019b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:42.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.226+0000 7f93665b9740 1 --2- 172.21.15.97:0/621456716 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d126907ab0 0x55d1269019b0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f9350009860 tx=0x7f9350000bc0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.226+0000 7f93665b9740 1 -- 172.21.15.97:0/621456716 shutdown_connections 2024-03-27T10:57:42.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.226+0000 7f93665b9740 1 --2- 172.21.15.97:0/621456716 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d12690d610 0x55d126901ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.226+0000 7f93665b9740 1 --2- 172.21.15.97:0/621456716 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d1269084a0 0x55d126919120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.226+0000 7f93665b9740 1 --2- 172.21.15.97:0/621456716 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d126907ab0 0x55d1269019b0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.226+0000 7f93665b9740 1 -- 172.21.15.97:0/621456716 >> 172.21.15.97:0/621456716 conn(0x55d126903020 msgr2=0x55d126911820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:42.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.226+0000 7f93665b9740 1 -- 172.21.15.97:0/621456716 shutdown_connections 2024-03-27T10:57:42.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.226+0000 7f93665b9740 1 -- 172.21.15.97:0/621456716 wait complete. 2024-03-27T10:57:42.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.227+0000 7f93665b9740 1 -- start start 2024-03-27T10:57:42.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.227+0000 7f93665b9740 1 Processor -- start 2024-03-27T10:57:42.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.227+0000 7f93665b9740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d126907ab0 0x55d126918a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:42.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.227+0000 7f93665b9740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d1269084a0 0x55d126918f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:42.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.227+0000 7f93665b9740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d12690d610 0x55d126917020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:42.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.227+0000 7f93665b9740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f935c1ac400 con 0x55d126907ab0 2024-03-27T10:57:42.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.227+0000 7f93665b9740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f935c1abd30 con 0x55d12690d610 2024-03-27T10:57:42.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.227+0000 7f93665b9740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f935c1a8f70 con 0x55d1269084a0 2024-03-27T10:57:42.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.227+0000 7f93665b9740 10 client.0 _ll_register_callbacks cb 0x7f9348009440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:57:42.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.228+0000 7f9360e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d126907ab0 0x55d126918a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:42.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.228+0000 7f9360e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d126907ab0 0x55d126918a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53486/0 (socket says 172.21.15.97:53486) 2024-03-27T10:57:42.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.228+0000 7f9360e00640 1 -- 172.21.15.97:0/635263672 learned_addr learned my addr 172.21.15.97:0/635263672 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:42.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.228+0000 7f9361800640 1 --2- 172.21.15.97:0/635263672 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d12690d610 0x55d126917020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:42.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.228+0000 7f935be00640 1 --2- 172.21.15.97:0/635263672 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d1269084a0 0x55d126918f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:42.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.228+0000 7f9360e00640 1 -- 172.21.15.97:0/635263672 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d1269084a0 msgr2=0x55d126918f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:42.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.228+0000 7f9360e00640 1 --2- 172.21.15.97:0/635263672 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d1269084a0 0x55d126918f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.228+0000 7f9360e00640 1 -- 172.21.15.97:0/635263672 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d12690d610 msgr2=0x55d126917020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:42.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.228+0000 7f9360e00640 1 --2- 172.21.15.97:0/635263672 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d12690d610 0x55d126917020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.228+0000 7f9360e00640 1 -- 172.21.15.97:0/635263672 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d126912940 con 0x55d126907ab0 2024-03-27T10:57:42.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.229+0000 7f935be00640 1 --2- 172.21.15.97:0/635263672 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d1269084a0 0x55d126918f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T10:57:42.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.229+0000 7f9361800640 1 --2- 172.21.15.97:0/635263672 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d12690d610 0x55d126917020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T10:57:42.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.229+0000 7f9360e00640 1 --2- 172.21.15.97:0/635263672 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d126907ab0 0x55d126918a10 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f935004ad80 tx=0x7f935002c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e202ea07529cf4fd server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:42.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.229+0000 7f9342a00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T10:57:42.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.229+0000 7f9342a00640 1 -- 172.21.15.97:0/635263672 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f935006a070 con 0x55d126907ab0 2024-03-27T10:57:42.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.229+0000 7f9342a00640 1 -- 172.21.15.97:0/635263672 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f935004a440 con 0x55d126907ab0 2024-03-27T10:57:42.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.229+0000 7f93665b9740 1 -- 172.21.15.97:0/635263672 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f935004f040 con 0x55d126907ab0 2024-03-27T10:57:42.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.230+0000 7f9342a00640 1 -- 172.21.15.97:0/635263672 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9350039040 con 0x55d126907ab0 2024-03-27T10:57:42.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.230+0000 7f93665b9740 10 client.15786 fetch_fsmap learned FSMap version 28 2024-03-27T10:57:42.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.230+0000 7f93665b9740 1 -- 172.21.15.97:0/635263672 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d126903d90 con 0x55d126907ab0 2024-03-27T10:57:42.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.230+0000 7f9342a00640 1 -- 172.21.15.97:0/635263672 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f935004f040 con 0x55d126907ab0 2024-03-27T10:57:42.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.230+0000 7f9342a00640 1 -- 172.21.15.97:0/635263672 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f935003f7f0 con 0x55d126907ab0 2024-03-27T10:57:42.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.230+0000 7f93665b9740 10 client.15786 fetch_fsmap finished waiting for FSMap version 28 2024-03-27T10:57:42.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.230+0000 7f93665b9740 10 client.15786 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-03-27T10:57:42.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.230+0000 7f93665b9740 20 client.15786 populate_metadata read hostname 'smithi097' 2024-03-27T10:57:42.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.230+0000 7f93665b9740 1 --2- 172.21.15.97:0/635263672 >> [v2:172.21.15.177:6834/3349112132,v1:172.21.15.177:6835/3349112132] conn(0x55d12691d850 0x55d12698ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:42.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.230+0000 7f93665b9740 4 client.15786 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-27T10:57:42.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.230+0000 7f93665b9740 1 -- 172.21.15.97:0/635263672 --> [v2:172.21.15.177:6834/3349112132,v1:172.21.15.177:6835/3349112132] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d126873f00 con 0x55d12691d850 2024-03-27T10:57:42.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.231+0000 7f935be00640 1 --2- 172.21.15.97:0/635263672 >> [v2:172.21.15.177:6834/3349112132,v1:172.21.15.177:6835/3349112132] conn(0x55d12691d850 0x55d12698ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:42.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.232+0000 7f9342a00640 10 client.15786 ms_handle_connect on v2:172.21.15.177:6834/3349112132 2024-03-27T10:57:42.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.232+0000 7f935be00640 1 --2- 172.21.15.97:0/635263672 >> [v2:172.21.15.177:6834/3349112132,v1:172.21.15.177:6835/3349112132] conn(0x55d12691d850 0x55d12698ef50 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d46dc806951a8fc4 server_cookie=593c700938cea161 in_seq=0 out_seq=0 2024-03-27T10:57:42.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.232+0000 7f9342a00640 1 -- 172.21.15.97:0/635263672 <== mds.3 v2:172.21.15.177:6834/3349112132 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d126873f00 con 0x55d12691d850 2024-03-27T10:57:42.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.232+0000 7f9342a00640 10 client.15786 handle_command_reply: tid=0 2024-03-27T10:57:42.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.232+0000 7f93665b9740 1 client.15786 shutdown 2024-03-27T10:57:42.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.233+0000 7f93665b9740 1 -- 172.21.15.97:0/635263672 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d126907ab0 msgr2=0x55d126918a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:42.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.233+0000 7f93665b9740 1 --2- 172.21.15.97:0/635263672 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d126907ab0 0x55d126918a10 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f935004ad80 tx=0x7f935002c960 comp rx=0 tx=0).stop 2024-03-27T10:57:42.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.233+0000 7f93665b9740 1 -- 172.21.15.97:0/635263672 shutdown_connections 2024-03-27T10:57:42.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.233+0000 7f93665b9740 1 --2- 172.21.15.97:0/635263672 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d12690d610 0x55d126917020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.233+0000 7f93665b9740 1 --2- 172.21.15.97:0/635263672 >> [v2:172.21.15.177:6834/3349112132,v1:172.21.15.177:6835/3349112132] conn(0x55d12691d850 0x55d12698ef50 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.233+0000 7f93665b9740 1 --2- 172.21.15.97:0/635263672 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d1269084a0 0x55d126918f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.233+0000 7f93665b9740 1 --2- 172.21.15.97:0/635263672 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d126907ab0 0x55d126918a10 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.233+0000 7f93665b9740 1 -- 172.21.15.97:0/635263672 >> 172.21.15.97:0/635263672 conn(0x55d126903020 msgr2=0x55d126906280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:42.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.233+0000 7f9342a00640 0 client.15786 ms_handle_reset on v2:172.21.15.177:6834/3349112132 2024-03-27T10:57:42.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.233+0000 7f93665b9740 1 -- 172.21.15.97:0/635263672 shutdown_connections 2024-03-27T10:57:42.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.233+0000 7f93665b9740 1 -- 172.21.15.97:0/635263672 wait complete. 2024-03-27T10:57:42.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.233+0000 7f93665b9740 20 client.15786 trim_cache size 0 max 16384 2024-03-27T10:57:42.206 INFO:teuthology.orchestra.run.smithi097.stdout:[] 2024-03-27T10:57:42.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.234+0000 7f9362200640 1 -- 172.21.15.97:0/1914087164 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f93440a3690 msgr2=0x7f93440a5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:42.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.234+0000 7f9362200640 1 --2- 172.21.15.97:0/1914087164 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f93440a3690 0x7f93440a5b50 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f9350009830 tx=0x7f9350034980 comp rx=0 tx=0).stop 2024-03-27T10:57:42.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.234+0000 7f9362200640 1 -- 172.21.15.97:0/1914087164 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f935c107710 msgr2=0x7f935c19f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:42.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.234+0000 7f9362200640 1 --2- 172.21.15.97:0/1914087164 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f935c107710 0x7f935c19f2a0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f935400efc0 tx=0x7f935400c800 comp rx=0 tx=0).stop 2024-03-27T10:57:42.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.235+0000 7f9362200640 1 -- 172.21.15.97:0/1914087164 shutdown_connections 2024-03-27T10:57:42.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.235+0000 7f9362200640 1 --2- 172.21.15.97:0/1914087164 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f935c107710 0x7f935c19f2a0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.235+0000 7f9362200640 1 --2- 172.21.15.97:0/1914087164 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f93440a3690 0x7f93440a5b50 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.235+0000 7f9362200640 1 --2- 172.21.15.97:0/1914087164 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f935c104de0 0x7f935c198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.235+0000 7f9362200640 1 --2- 172.21.15.97:0/1914087164 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f935c1024b0 0x7f935c197d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.235+0000 7f9362200640 1 -- 172.21.15.97:0/1914087164 >> 172.21.15.97:0/1914087164 conn(0x7f935c0fbec0 msgr2=0x7f935c102ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:42.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.235+0000 7f9362200640 1 -- 172.21.15.97:0/1914087164 shutdown_connections 2024-03-27T10:57:42.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.235+0000 7f9362200640 1 -- 172.21.15.97:0/1914087164 wait complete. 2024-03-27T10:57:42.232 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-27T10:57:42.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.537+0000 7fbf4e600640 1 -- 172.21.15.97:0/2418984916 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf48102c40 msgr2=0x7fbf481030c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:42.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.537+0000 7fbf4e600640 1 --2- 172.21.15.97:0/2418984916 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf48102c40 0x7fbf481030c0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fbf34009900 tx=0x7fbf3402bc30 comp rx=0 tx=0).stop 2024-03-27T10:57:42.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.537+0000 7fbf4e600640 1 -- 172.21.15.97:0/2418984916 shutdown_connections 2024-03-27T10:57:42.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.537+0000 7fbf4e600640 1 --2- 172.21.15.97:0/2418984916 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbf48103600 0x7fbf48109e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.537+0000 7fbf4e600640 1 --2- 172.21.15.97:0/2418984916 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf48102c40 0x7fbf481030c0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.537+0000 7fbf4e600640 1 --2- 172.21.15.97:0/2418984916 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf48101a50 0x7fbf48101e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.537+0000 7fbf4e600640 1 -- 172.21.15.97:0/2418984916 >> 172.21.15.97:0/2418984916 conn(0x7fbf480fcfc0 msgr2=0x7fbf480ff400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:42.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.537+0000 7fbf4e600640 1 -- 172.21.15.97:0/2418984916 shutdown_connections 2024-03-27T10:57:42.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.537+0000 7fbf4e600640 1 -- 172.21.15.97:0/2418984916 wait complete. 2024-03-27T10:57:42.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.538+0000 7fbf4e600640 1 Processor -- start 2024-03-27T10:57:42.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.539+0000 7fbf4e600640 1 -- start start 2024-03-27T10:57:42.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.539+0000 7fbf4e600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf48101a50 0x7fbf48075900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:42.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.539+0000 7fbf4e600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf48102c40 0x7fbf48073f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:42.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.539+0000 7fbf4e600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbf48103600 0x7fbf48074480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:42.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.539+0000 7fbf4e600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fbf4810d5b0 con 0x7fbf48102c40 2024-03-27T10:57:42.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.539+0000 7fbf4e600640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fbf4810d430 con 0x7fbf48101a50 2024-03-27T10:57:42.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.539+0000 7fbf4e600640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fbf4810d730 con 0x7fbf48103600 2024-03-27T10:57:42.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.539+0000 7fbf47e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf48102c40 0x7fbf48073f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:42.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.540+0000 7fbf47e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf48102c40 0x7fbf48073f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53498/0 (socket says 172.21.15.97:53498) 2024-03-27T10:57:42.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.540+0000 7fbf47e00640 1 -- 172.21.15.97:0/704148611 learned_addr learned my addr 172.21.15.97:0/704148611 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:42.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.540+0000 7fbf4d200640 1 --2- 172.21.15.97:0/704148611 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf48101a50 0x7fbf48075900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:42.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.540+0000 7fbf4dc00640 1 --2- 172.21.15.97:0/704148611 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbf48103600 0x7fbf48074480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:42.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.540+0000 7fbf47e00640 1 -- 172.21.15.97:0/704148611 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbf48103600 msgr2=0x7fbf48074480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:42.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.540+0000 7fbf47e00640 1 --2- 172.21.15.97:0/704148611 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbf48103600 0x7fbf48074480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.540+0000 7fbf47e00640 1 -- 172.21.15.97:0/704148611 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf48101a50 msgr2=0x7fbf48075900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:42.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.540+0000 7fbf47e00640 1 --2- 172.21.15.97:0/704148611 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf48101a50 0x7fbf48075900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.540+0000 7fbf47e00640 1 -- 172.21.15.97:0/704148611 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf48074d10 con 0x7fbf48102c40 2024-03-27T10:57:42.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.540+0000 7fbf4dc00640 1 --2- 172.21.15.97:0/704148611 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbf48103600 0x7fbf48074480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T10:57:42.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.541+0000 7fbf47e00640 1 --2- 172.21.15.97:0/704148611 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf48102c40 0x7fbf48073f40 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fbf34000c00 tx=0x7fbf340042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:42.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.541+0000 7fbf45600640 1 -- 172.21.15.97:0/704148611 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf34043020 con 0x7fbf48102c40 2024-03-27T10:57:42.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.541+0000 7fbf45600640 1 -- 172.21.15.97:0/704148611 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbf34004aa0 con 0x7fbf48102c40 2024-03-27T10:57:42.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.541+0000 7fbf4e600640 1 -- 172.21.15.97:0/704148611 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf481a1d80 con 0x7fbf48102c40 2024-03-27T10:57:42.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.541+0000 7fbf45600640 1 -- 172.21.15.97:0/704148611 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf34003d70 con 0x7fbf48102c40 2024-03-27T10:57:42.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.541+0000 7fbf4e600640 1 -- 172.21.15.97:0/704148611 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf481a2210 con 0x7fbf48102c40 2024-03-27T10:57:42.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.545+0000 7fbf45600640 1 -- 172.21.15.97:0/704148611 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7fbf34046020 con 0x7fbf48102c40 2024-03-27T10:57:42.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.545+0000 7fbf5287e740 1 Processor -- start 2024-03-27T10:57:42.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.546+0000 7fbf5287e740 1 -- start start 2024-03-27T10:57:42.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.546+0000 7fbf5287e740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bd168ce2c0 0x55bd168ce6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:42.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.546+0000 7fbf5287e740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd168cecb0 0x55bd168d37b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:42.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.546+0000 7fbf5287e740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd168d3e20 0x55bd168d62b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:42.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.546+0000 7fbf5287e740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55bd1688d7b0 con 0x55bd168d3e20 2024-03-27T10:57:42.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.546+0000 7fbf5287e740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55bd1682a670 con 0x55bd168cecb0 2024-03-27T10:57:42.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.546+0000 7fbf5287e740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55bd167d2970 con 0x55bd168ce2c0 2024-03-27T10:57:42.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.546+0000 7fbf45600640 1 --2- 172.21.15.97:0/704148611 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fbf300a3610 0x7fbf300a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:42.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.547+0000 7fbf45600640 1 -- 172.21.15.97:0/704148611 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fbf340f2070 con 0x7fbf48102c40 2024-03-27T10:57:42.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.547+0000 7fbf4dc00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bd168ce2c0 0x55bd168ce6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:42.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.547+0000 7fbf4d200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd168cecb0 0x55bd168d37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:42.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.547+0000 7fbf4dc00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bd168ce2c0 0x55bd168ce6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.97:35074/0 (socket says 172.21.15.97:35074) 2024-03-27T10:57:42.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.547+0000 7fbf4d200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd168cecb0 0x55bd168d37b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:33800/0 (socket says 172.21.15.97:33800) 2024-03-27T10:57:42.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.547+0000 7fbf4dc00640 1 -- 172.21.15.97:0/926817178 learned_addr learned my addr 172.21.15.97:0/926817178 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:42.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.547+0000 7fbf47e00640 1 --2- 172.21.15.97:0/926817178 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd168d3e20 0x55bd168d62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:42.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.547+0000 7fbf4dc00640 1 --2- 172.21.15.97:0/704148611 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fbf300a3610 0x7fbf300a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:42.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.547+0000 7fbf47e00640 1 -- 172.21.15.97:0/926817178 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bd168ce2c0 msgr2=0x55bd168ce6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:42.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.547+0000 7fbf47e00640 1 --2- 172.21.15.97:0/926817178 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bd168ce2c0 0x55bd168ce6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.548+0000 7fbf47e00640 1 -- 172.21.15.97:0/926817178 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd168cecb0 msgr2=0x55bd168d37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:42.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.548+0000 7fbf47e00640 1 --2- 172.21.15.97:0/926817178 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd168cecb0 0x55bd168d37b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.548+0000 7fbf47e00640 1 -- 172.21.15.97:0/926817178 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd1653e3f0 con 0x55bd168d3e20 2024-03-27T10:57:42.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.548+0000 7fbf47e00640 1 --2- 172.21.15.97:0/926817178 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd168d3e20 0x55bd168d62b0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fbf3402c6a0 tx=0x7fbf340057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=262e58c3c0d25d79 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:42.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.548+0000 7fbf3a000640 1 -- 172.21.15.97:0/926817178 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf340d7070 con 0x55bd168d3e20 2024-03-27T10:57:42.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.548+0000 7fbf3a000640 1 -- 172.21.15.97:0/926817178 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbf34004c50 con 0x55bd168d3e20 2024-03-27T10:57:42.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.548+0000 7fbf3a000640 1 -- 172.21.15.97:0/926817178 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf340bc850 con 0x55bd168d3e20 2024-03-27T10:57:42.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.549+0000 7fbf5287e740 1 -- 172.21.15.97:0/926817178 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd168d3e20 msgr2=0x55bd168d62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:42.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.549+0000 7fbf5287e740 1 --2- 172.21.15.97:0/926817178 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd168d3e20 0x55bd168d62b0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fbf3402c6a0 tx=0x7fbf340057b0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.549+0000 7fbf4dc00640 1 --2- 172.21.15.97:0/704148611 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fbf300a3610 0x7fbf300a5ad0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fbf400128e0 tx=0x7fbf40009ad0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:42.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.549+0000 7fbf5287e740 1 -- 172.21.15.97:0/926817178 shutdown_connections 2024-03-27T10:57:42.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.549+0000 7fbf5287e740 1 --2- 172.21.15.97:0/926817178 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd168d3e20 0x55bd168d62b0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.549+0000 7fbf5287e740 1 --2- 172.21.15.97:0/926817178 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd168cecb0 0x55bd168d37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.549+0000 7fbf5287e740 1 --2- 172.21.15.97:0/926817178 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bd168ce2c0 0x55bd168ce6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.549+0000 7fbf5287e740 1 -- 172.21.15.97:0/926817178 >> 172.21.15.97:0/926817178 conn(0x55bd168c9830 msgr2=0x55bd168c9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:42.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.550+0000 7fbf5287e740 1 -- 172.21.15.97:0/926817178 shutdown_connections 2024-03-27T10:57:42.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.550+0000 7fbf5287e740 1 -- 172.21.15.97:0/926817178 wait complete. 2024-03-27T10:57:42.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.550+0000 7fbf5287e740 1 -- start start 2024-03-27T10:57:42.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.550+0000 7fbf5287e740 1 Processor -- start 2024-03-27T10:57:42.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.551+0000 7fbf5287e740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd168ce2c0 0x55bd168e91a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:42.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.551+0000 7fbf5287e740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd168cecb0 0x55bd168e96e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:42.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.551+0000 7fbf5287e740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bd168d3e20 0x55bd168f0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:42.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.551+0000 7fbf5287e740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55bd167e5ca0 con 0x55bd168ce2c0 2024-03-27T10:57:42.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.551+0000 7fbf5287e740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55bd16826620 con 0x55bd168cecb0 2024-03-27T10:57:42.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.551+0000 7fbf5287e740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55bd168e7ba0 con 0x55bd168d3e20 2024-03-27T10:57:42.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.551+0000 7fbf5287e740 10 client.0 _ll_register_callbacks cb 0x55bd16822160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:57:42.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.551+0000 7fbf4d200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd168ce2c0 0x55bd168e91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:42.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.552+0000 7fbf4d200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd168ce2c0 0x55bd168e91a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53524/0 (socket says 172.21.15.97:53524) 2024-03-27T10:57:42.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.552+0000 7fbf4d200640 1 -- 172.21.15.97:0/138516194 learned_addr learned my addr 172.21.15.97:0/138516194 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:42.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.552+0000 7fbf47e00640 1 --2- 172.21.15.97:0/138516194 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd168cecb0 0x55bd168e96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:42.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.552+0000 7fbf4dc00640 1 --2- 172.21.15.97:0/138516194 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bd168d3e20 0x55bd168f0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:42.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.552+0000 7fbf4d200640 1 -- 172.21.15.97:0/138516194 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bd168d3e20 msgr2=0x55bd168f0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:42.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.552+0000 7fbf4d200640 1 --2- 172.21.15.97:0/138516194 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bd168d3e20 0x55bd168f0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.552+0000 7fbf4d200640 1 -- 172.21.15.97:0/138516194 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd168cecb0 msgr2=0x55bd168e96e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:42.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.552+0000 7fbf4d200640 1 --2- 172.21.15.97:0/138516194 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd168cecb0 0x55bd168e96e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.552+0000 7fbf4d200640 1 -- 172.21.15.97:0/138516194 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd166087d0 con 0x55bd168ce2c0 2024-03-27T10:57:42.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.552+0000 7fbf4dc00640 1 --2- 172.21.15.97:0/138516194 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bd168d3e20 0x55bd168f0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:57:42.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.552+0000 7fbf4d200640 1 --2- 172.21.15.97:0/138516194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd168ce2c0 0x55bd168e91a0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c012700 tx=0x7fbf3c012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3ff1afed291b4a8 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:42.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.552+0000 7fbf2ea00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T10:57:42.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.553+0000 7fbf2ea00640 1 -- 172.21.15.97:0/138516194 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf3c004050 con 0x55bd168ce2c0 2024-03-27T10:57:42.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.553+0000 7fbf2ea00640 1 -- 172.21.15.97:0/138516194 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fbf3c0041f0 con 0x55bd168ce2c0 2024-03-27T10:57:42.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.553+0000 7fbf5287e740 1 -- 172.21.15.97:0/138516194 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf34004c50 con 0x55bd168ce2c0 2024-03-27T10:57:42.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.553+0000 7fbf2ea00640 1 -- 172.21.15.97:0/138516194 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbf3c0043f0 con 0x55bd168ce2c0 2024-03-27T10:57:42.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.553+0000 7fbf5287e740 10 client.15798 fetch_fsmap learned FSMap version 28 2024-03-27T10:57:42.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.553+0000 7fbf5287e740 1 -- 172.21.15.97:0/138516194 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bd168f3b60 con 0x55bd168ce2c0 2024-03-27T10:57:42.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.554+0000 7fbf2ea00640 1 -- 172.21.15.97:0/138516194 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbf34004c50 con 0x55bd168ce2c0 2024-03-27T10:57:42.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.554+0000 7fbf2ea00640 1 -- 172.21.15.97:0/138516194 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fbf3c003ac0 con 0x55bd168ce2c0 2024-03-27T10:57:42.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.554+0000 7fbf5287e740 10 client.15798 fetch_fsmap finished waiting for FSMap version 28 2024-03-27T10:57:42.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.554+0000 7fbf5287e740 10 client.15798 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-27T10:57:42.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.555+0000 7fbf5287e740 20 client.15798 populate_metadata read hostname 'smithi097' 2024-03-27T10:57:42.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.555+0000 7fbf5287e740 1 --2- 172.21.15.97:0/138516194 >> [v2:172.21.15.114:6834/3581918222,v1:172.21.15.114:6835/3581918222] conn(0x55bd168f5260 0x55bd168f7720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:42.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.555+0000 7fbf5287e740 4 client.15798 mds_command: new command op to 34445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T10:57:42.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.555+0000 7fbf5287e740 1 -- 172.21.15.97:0/138516194 --> [v2:172.21.15.114:6834/3581918222,v1:172.21.15.114:6835/3581918222] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bd16837bb0 con 0x55bd168f5260 2024-03-27T10:57:42.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.555+0000 7fbf47e00640 1 --2- 172.21.15.97:0/138516194 >> [v2:172.21.15.114:6834/3581918222,v1:172.21.15.114:6835/3581918222] conn(0x55bd168f5260 0x55bd168f7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:42.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.556+0000 7fbf47e00640 1 --2- 172.21.15.97:0/138516194 >> [v2:172.21.15.114:6834/3581918222,v1:172.21.15.114:6835/3581918222] conn(0x55bd168f5260 0x55bd168f7720 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=26c3dd5c6ee4fd4d server_cookie=91f8ab86f1aa0b4c in_seq=0 out_seq=0 2024-03-27T10:57:42.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.556+0000 7fbf2ea00640 10 client.15798 ms_handle_connect on v2:172.21.15.114:6834/3581918222 2024-03-27T10:57:42.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.558+0000 7fbf2ea00640 1 -- 172.21.15.97:0/138516194 <== mds.2 v2:172.21.15.114:6834/3581918222 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55bd16837bb0 con 0x55bd168f5260 2024-03-27T10:57:42.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.558+0000 7fbf2ea00640 10 client.15798 handle_command_reply: tid=0 2024-03-27T10:57:42.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.558+0000 7fbf5287e740 1 client.15798 shutdown 2024-03-27T10:57:42.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.559+0000 7fbf5287e740 1 -- 172.21.15.97:0/138516194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd168ce2c0 msgr2=0x55bd168e91a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:42.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.559+0000 7fbf5287e740 1 --2- 172.21.15.97:0/138516194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd168ce2c0 0x55bd168e91a0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c012700 tx=0x7fbf3c012bc0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.559+0000 7fbf5287e740 1 -- 172.21.15.97:0/138516194 shutdown_connections 2024-03-27T10:57:42.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.559+0000 7fbf5287e740 1 --2- 172.21.15.97:0/138516194 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bd168d3e20 0x55bd168f0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.559+0000 7fbf5287e740 1 --2- 172.21.15.97:0/138516194 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd168cecb0 0x55bd168e96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.559+0000 7fbf5287e740 1 --2- 172.21.15.97:0/138516194 >> [v2:172.21.15.114:6834/3581918222,v1:172.21.15.114:6835/3581918222] conn(0x55bd168f5260 0x55bd168f7720 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.559+0000 7fbf5287e740 1 --2- 172.21.15.97:0/138516194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd168ce2c0 0x55bd168e91a0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.559+0000 7fbf5287e740 1 -- 172.21.15.97:0/138516194 >> 172.21.15.97:0/138516194 conn(0x55bd168c9830 msgr2=0x55bd168c0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:42.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.559+0000 7fbf2ea00640 0 client.15798 ms_handle_reset on v2:172.21.15.114:6834/3581918222 2024-03-27T10:57:42.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.559+0000 7fbf5287e740 1 -- 172.21.15.97:0/138516194 shutdown_connections 2024-03-27T10:57:42.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.559+0000 7fbf5287e740 1 -- 172.21.15.97:0/138516194 wait complete. 2024-03-27T10:57:42.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.559+0000 7fbf5287e740 20 client.15798 trim_cache size 0 max 16384 2024-03-27T10:57:42.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.586+0000 7fbf5287e740 1 Processor -- start 2024-03-27T10:57:42.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.586+0000 7fbf5287e740 1 -- start start 2024-03-27T10:57:42.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.586+0000 7fbf5287e740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd168ce2c0 0x55bd168e87c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:42.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.586+0000 7fbf5287e740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bd168cecb0 0x55bd168f06e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:42.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.586+0000 7fbf5287e740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd168d3e20 0x55bd168e8d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:42.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.586+0000 7fbf5287e740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fbf4806fbc0 con 0x55bd168ce2c0 2024-03-27T10:57:42.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.586+0000 7fbf5287e740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fbf48071700 con 0x55bd168d3e20 2024-03-27T10:57:42.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.586+0000 7fbf5287e740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fbf48070290 con 0x55bd168cecb0 2024-03-27T10:57:42.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.587+0000 7fbf4d200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd168ce2c0 0x55bd168e87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:42.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.587+0000 7fbf4d200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd168ce2c0 0x55bd168e87c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53538/0 (socket says 172.21.15.97:53538) 2024-03-27T10:57:42.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.587+0000 7fbf4d200640 1 -- 172.21.15.97:0/1901358582 learned_addr learned my addr 172.21.15.97:0/1901358582 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:42.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.587+0000 7fbf4dc00640 1 --2- 172.21.15.97:0/1901358582 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd168d3e20 0x55bd168e8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:42.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.587+0000 7fbf47e00640 1 --2- 172.21.15.97:0/1901358582 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bd168cecb0 0x55bd168f06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:42.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.587+0000 7fbf4d200640 1 -- 172.21.15.97:0/1901358582 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bd168cecb0 msgr2=0x55bd168f06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:42.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.587+0000 7fbf4d200640 1 --2- 172.21.15.97:0/1901358582 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bd168cecb0 0x55bd168f06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.587+0000 7fbf4d200640 1 -- 172.21.15.97:0/1901358582 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd168d3e20 msgr2=0x55bd168e8d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:42.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.587+0000 7fbf4d200640 1 --2- 172.21.15.97:0/1901358582 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd168d3e20 0x55bd168e8d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.587+0000 7fbf4d200640 1 -- 172.21.15.97:0/1901358582 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd168ca5a0 con 0x55bd168ce2c0 2024-03-27T10:57:42.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.587+0000 7fbf47e00640 1 --2- 172.21.15.97:0/1901358582 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bd168cecb0 0x55bd168f06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T10:57:42.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.588+0000 7fbf4d200640 1 --2- 172.21.15.97:0/1901358582 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd168ce2c0 0x55bd168e87c0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c005840 tx=0x7fbf3c010830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcd1ab511f49ad0e server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:42.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.588+0000 7fbf39600640 1 -- 172.21.15.97:0/1901358582 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf3c02b070 con 0x55bd168ce2c0 2024-03-27T10:57:42.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.588+0000 7fbf39600640 1 -- 172.21.15.97:0/1901358582 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbf3c010e50 con 0x55bd168ce2c0 2024-03-27T10:57:42.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.588+0000 7fbf39600640 1 -- 172.21.15.97:0/1901358582 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf3c0055e0 con 0x55bd168ce2c0 2024-03-27T10:57:42.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.589+0000 7fbf5287e740 1 -- 172.21.15.97:0/1901358582 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd168ce2c0 msgr2=0x55bd168e87c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:42.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.589+0000 7fbf5287e740 1 --2- 172.21.15.97:0/1901358582 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd168ce2c0 0x55bd168e87c0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c005840 tx=0x7fbf3c010830 comp rx=0 tx=0).stop 2024-03-27T10:57:42.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.590+0000 7fbf5287e740 1 -- 172.21.15.97:0/1901358582 shutdown_connections 2024-03-27T10:57:42.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.590+0000 7fbf5287e740 1 --2- 172.21.15.97:0/1901358582 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd168d3e20 0x55bd168e8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.590+0000 7fbf5287e740 1 --2- 172.21.15.97:0/1901358582 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bd168cecb0 0x55bd168f06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.590+0000 7fbf5287e740 1 --2- 172.21.15.97:0/1901358582 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd168ce2c0 0x55bd168e87c0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.590+0000 7fbf5287e740 1 -- 172.21.15.97:0/1901358582 >> 172.21.15.97:0/1901358582 conn(0x55bd168c9830 msgr2=0x55bd168cd320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:42.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.590+0000 7fbf5287e740 1 -- 172.21.15.97:0/1901358582 shutdown_connections 2024-03-27T10:57:42.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.590+0000 7fbf5287e740 1 -- 172.21.15.97:0/1901358582 wait complete. 2024-03-27T10:57:42.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.590+0000 7fbf5287e740 1 -- start start 2024-03-27T10:57:42.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.591+0000 7fbf5287e740 1 Processor -- start 2024-03-27T10:57:42.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.591+0000 7fbf5287e740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd168ce2c0 0x55bd168ede00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:42.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.591+0000 7fbf5287e740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd168cecb0 0x55bd168ee340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:42.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.591+0000 7fbf5287e740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bd168d3e20 0x55bd168ef2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:42.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.591+0000 7fbf5287e740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fbf48071030 con 0x55bd168ce2c0 2024-03-27T10:57:42.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.591+0000 7fbf5287e740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fbf48070960 con 0x55bd168cecb0 2024-03-27T10:57:42.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.591+0000 7fbf5287e740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fbf4806dba0 con 0x55bd168d3e20 2024-03-27T10:57:42.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.591+0000 7fbf5287e740 10 client.0 _ll_register_callbacks cb 0x7fbf3c009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:57:42.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.592+0000 7fbf4d200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd168ce2c0 0x55bd168ede00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:42.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.592+0000 7fbf47e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd168cecb0 0x55bd168ee340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:42.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.592+0000 7fbf4d200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd168ce2c0 0x55bd168ede00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53540/0 (socket says 172.21.15.97:53540) 2024-03-27T10:57:42.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.592+0000 7fbf4d200640 1 -- 172.21.15.97:0/1489463601 learned_addr learned my addr 172.21.15.97:0/1489463601 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:42.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.592+0000 7fbf4dc00640 1 --2- 172.21.15.97:0/1489463601 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bd168d3e20 0x55bd168ef2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:42.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.592+0000 7fbf4d200640 1 -- 172.21.15.97:0/1489463601 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bd168d3e20 msgr2=0x55bd168ef2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:42.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.592+0000 7fbf4d200640 1 --2- 172.21.15.97:0/1489463601 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bd168d3e20 0x55bd168ef2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.592+0000 7fbf4d200640 1 -- 172.21.15.97:0/1489463601 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd168cecb0 msgr2=0x55bd168ee340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:42.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.592+0000 7fbf4d200640 1 --2- 172.21.15.97:0/1489463601 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd168cecb0 0x55bd168ee340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.592+0000 7fbf4d200640 1 -- 172.21.15.97:0/1489463601 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd168ca2a0 con 0x55bd168ce2c0 2024-03-27T10:57:42.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.592+0000 7fbf4dc00640 1 --2- 172.21.15.97:0/1489463601 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bd168d3e20 0x55bd168ef2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:57:42.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.593+0000 7fbf4d200640 1 --2- 172.21.15.97:0/1489463601 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd168ce2c0 0x55bd168ede00 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c005840 tx=0x7fbf3c009f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=205689caa1871881 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:42.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.593+0000 7fbf2ea00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T10:57:42.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.593+0000 7fbf2ea00640 1 -- 172.21.15.97:0/1489463601 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf3c02b040 con 0x55bd168ce2c0 2024-03-27T10:57:42.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.593+0000 7fbf2ea00640 1 -- 172.21.15.97:0/1489463601 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fbf3c010920 con 0x55bd168ce2c0 2024-03-27T10:57:42.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.593+0000 7fbf5287e740 1 -- 172.21.15.97:0/1489463601 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf3c010e50 con 0x55bd168ce2c0 2024-03-27T10:57:42.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.594+0000 7fbf2ea00640 1 -- 172.21.15.97:0/1489463601 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbf3c0055f0 con 0x55bd168ce2c0 2024-03-27T10:57:42.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.594+0000 7fbf5287e740 10 client.15804 fetch_fsmap learned FSMap version 28 2024-03-27T10:57:42.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.594+0000 7fbf5287e740 1 -- 172.21.15.97:0/1489463601 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bd168f0a00 con 0x55bd168ce2c0 2024-03-27T10:57:42.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.594+0000 7fbf2ea00640 1 -- 172.21.15.97:0/1489463601 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbf3c010e50 con 0x55bd168ce2c0 2024-03-27T10:57:42.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.594+0000 7fbf2ea00640 1 -- 172.21.15.97:0/1489463601 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fbf3c036030 con 0x55bd168ce2c0 2024-03-27T10:57:42.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.594+0000 7fbf5287e740 10 client.15804 fetch_fsmap finished waiting for FSMap version 28 2024-03-27T10:57:42.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.594+0000 7fbf5287e740 10 client.15804 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-27T10:57:42.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.594+0000 7fbf5287e740 20 client.15804 populate_metadata read hostname 'smithi097' 2024-03-27T10:57:42.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.594+0000 7fbf5287e740 1 --2- 172.21.15.97:0/1489463601 >> [v2:172.21.15.114:6834/3581918222,v1:172.21.15.114:6835/3581918222] conn(0x55bd1695bd10 0x55bd1695e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:42.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.595+0000 7fbf5287e740 4 client.15804 mds_command: new command op to 34445 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-27T10:57:42.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.595+0000 7fbf5287e740 1 -- 172.21.15.97:0/1489463601 --> [v2:172.21.15.114:6834/3581918222,v1:172.21.15.114:6835/3581918222] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bd16831f00 con 0x55bd1695bd10 2024-03-27T10:57:42.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.595+0000 7fbf47e00640 1 --2- 172.21.15.97:0/1489463601 >> [v2:172.21.15.114:6834/3581918222,v1:172.21.15.114:6835/3581918222] conn(0x55bd1695bd10 0x55bd1695e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:42.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.596+0000 7fbf47e00640 1 --2- 172.21.15.97:0/1489463601 >> [v2:172.21.15.114:6834/3581918222,v1:172.21.15.114:6835/3581918222] conn(0x55bd1695bd10 0x55bd1695e130 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a112626bf7b027fb server_cookie=8d57edad5a0cd92e in_seq=0 out_seq=0 2024-03-27T10:57:42.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.596+0000 7fbf2ea00640 10 client.15804 ms_handle_connect on v2:172.21.15.114:6834/3581918222 2024-03-27T10:57:42.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.597+0000 7fbf2ea00640 1 -- 172.21.15.97:0/1489463601 <== mds.2 v2:172.21.15.114:6834/3581918222 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bd16831f00 con 0x55bd1695bd10 2024-03-27T10:57:42.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.597+0000 7fbf2ea00640 10 client.15804 handle_command_reply: tid=0 2024-03-27T10:57:42.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.597+0000 7fbf5287e740 1 client.15804 shutdown 2024-03-27T10:57:42.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.597+0000 7fbf5287e740 1 -- 172.21.15.97:0/1489463601 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd168ce2c0 msgr2=0x55bd168ede00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:42.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.597+0000 7fbf5287e740 1 --2- 172.21.15.97:0/1489463601 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd168ce2c0 0x55bd168ede00 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c005840 tx=0x7fbf3c009f20 comp rx=0 tx=0).stop 2024-03-27T10:57:42.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.598+0000 7fbf5287e740 1 -- 172.21.15.97:0/1489463601 shutdown_connections 2024-03-27T10:57:42.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.598+0000 7fbf5287e740 1 --2- 172.21.15.97:0/1489463601 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bd168d3e20 0x55bd168ef2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.598+0000 7fbf5287e740 1 --2- 172.21.15.97:0/1489463601 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd168cecb0 0x55bd168ee340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.598+0000 7fbf5287e740 1 --2- 172.21.15.97:0/1489463601 >> [v2:172.21.15.114:6834/3581918222,v1:172.21.15.114:6835/3581918222] conn(0x55bd1695bd10 0x55bd1695e130 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.598+0000 7fbf2ea00640 0 client.15804 ms_handle_reset on v2:172.21.15.114:6834/3581918222 2024-03-27T10:57:42.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.598+0000 7fbf5287e740 1 --2- 172.21.15.97:0/1489463601 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd168ce2c0 0x55bd168ede00 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.598+0000 7fbf5287e740 1 -- 172.21.15.97:0/1489463601 >> 172.21.15.97:0/1489463601 conn(0x55bd168c9830 msgr2=0x55bd168bf9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:42.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.598+0000 7fbf5287e740 1 -- 172.21.15.97:0/1489463601 shutdown_connections 2024-03-27T10:57:42.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.598+0000 7fbf5287e740 1 -- 172.21.15.97:0/1489463601 wait complete. 2024-03-27T10:57:42.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.598+0000 7fbf5287e740 20 client.15804 trim_cache size 0 max 16384 2024-03-27T10:57:42.570 INFO:teuthology.orchestra.run.smithi097.stdout:[] 2024-03-27T10:57:42.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.599+0000 7fbf4e600640 1 -- 172.21.15.97:0/704148611 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fbf300a3610 msgr2=0x7fbf300a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:42.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.599+0000 7fbf4e600640 1 --2- 172.21.15.97:0/704148611 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fbf300a3610 0x7fbf300a5ad0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fbf400128e0 tx=0x7fbf40009ad0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.599+0000 7fbf4e600640 1 -- 172.21.15.97:0/704148611 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf48102c40 msgr2=0x7fbf48073f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:42.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.599+0000 7fbf4e600640 1 --2- 172.21.15.97:0/704148611 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf48102c40 0x7fbf48073f40 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fbf34000c00 tx=0x7fbf340042f0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.599+0000 7fbf4e600640 1 -- 172.21.15.97:0/704148611 shutdown_connections 2024-03-27T10:57:42.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.599+0000 7fbf4e600640 1 --2- 172.21.15.97:0/704148611 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbf48103600 0x7fbf48074480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.599+0000 7fbf4e600640 1 --2- 172.21.15.97:0/704148611 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf48102c40 0x7fbf48073f40 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.599+0000 7fbf4e600640 1 --2- 172.21.15.97:0/704148611 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fbf300a3610 0x7fbf300a5ad0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.599+0000 7fbf4e600640 1 --2- 172.21.15.97:0/704148611 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf48101a50 0x7fbf48075900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.599+0000 7fbf4e600640 1 -- 172.21.15.97:0/704148611 >> 172.21.15.97:0/704148611 conn(0x7fbf480fcfc0 msgr2=0x7fbf48105fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:42.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.600+0000 7fbf4e600640 1 -- 172.21.15.97:0/704148611 shutdown_connections 2024-03-27T10:57:42.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.600+0000 7fbf4e600640 1 -- 172.21.15.97:0/704148611 wait complete. 2024-03-27T10:57:42.592 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:57:42 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/1674402243' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T10:57:42.595 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-27T10:57:42.674 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.703+0000 7f256da00640 1 -- 172.21.15.97:0/429427958 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2568100b70 msgr2=0x7f2568100fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:42.674 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.703+0000 7f256da00640 1 --2- 172.21.15.97:0/429427958 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2568100b70 0x7f2568100fd0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f2554009900 tx=0x7f255402bc40 comp rx=0 tx=0).stop 2024-03-27T10:57:42.674 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.703+0000 7f256da00640 1 -- 172.21.15.97:0/429427958 shutdown_connections 2024-03-27T10:57:42.674 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.703+0000 7f256da00640 1 --2- 172.21.15.97:0/429427958 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2568101510 0x7f2568101990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.674 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.703+0000 7f256da00640 1 --2- 172.21.15.97:0/429427958 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2568100b70 0x7f2568100fd0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.675 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.703+0000 7f256da00640 1 --2- 172.21.15.97:0/429427958 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2568100210 0x7f2568100630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.675 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.703+0000 7f256da00640 1 -- 172.21.15.97:0/429427958 >> 172.21.15.97:0/429427958 conn(0x7f25680fbe30 msgr2=0x7f25680fe250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:42.675 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.703+0000 7f256da00640 1 -- 172.21.15.97:0/429427958 shutdown_connections 2024-03-27T10:57:42.675 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.703+0000 7f256da00640 1 -- 172.21.15.97:0/429427958 wait complete. 2024-03-27T10:57:42.676 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.704+0000 7f256da00640 1 Processor -- start 2024-03-27T10:57:42.676 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.704+0000 7f256da00640 1 -- start start 2024-03-27T10:57:42.676 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.704+0000 7f256da00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2568100210 0x7f2568199dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:42.676 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.704+0000 7f256da00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2568100b70 0x7f256819a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:42.676 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.704+0000 7f256da00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2568101510 0x7f25681a1370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:42.676 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.704+0000 7f256da00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f256810f6e0 con 0x7f2568100210 2024-03-27T10:57:42.676 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.704+0000 7f256da00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f256810f560 con 0x7f2568101510 2024-03-27T10:57:42.676 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.704+0000 7f256da00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f256810f860 con 0x7f2568100b70 2024-03-27T10:57:42.677 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.705+0000 7f2567e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2568100210 0x7f2568199dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:42.677 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.705+0000 7f2567e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2568100210 0x7f2568199dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53556/0 (socket says 172.21.15.97:53556) 2024-03-27T10:57:42.677 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.705+0000 7f2567e00640 1 -- 172.21.15.97:0/2990301644 learned_addr learned my addr 172.21.15.97:0/2990301644 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:42.677 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.705+0000 7f256d000640 1 --2- 172.21.15.97:0/2990301644 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2568101510 0x7f25681a1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:42.677 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.705+0000 7f2567400640 1 --2- 172.21.15.97:0/2990301644 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2568100b70 0x7f256819a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:42.677 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.705+0000 7f2567e00640 1 -- 172.21.15.97:0/2990301644 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2568100b70 msgr2=0x7f256819a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:42.677 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.705+0000 7f2567e00640 1 --2- 172.21.15.97:0/2990301644 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2568100b70 0x7f256819a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.677 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.705+0000 7f2567e00640 1 -- 172.21.15.97:0/2990301644 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2568101510 msgr2=0x7f25681a1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:42.677 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.705+0000 7f2567e00640 1 --2- 172.21.15.97:0/2990301644 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2568101510 0x7f25681a1370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.677 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.705+0000 7f2567e00640 1 -- 172.21.15.97:0/2990301644 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25681a19e0 con 0x7f2568100210 2024-03-27T10:57:42.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:57:42 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/1674402243' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T10:57:42.678 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.706+0000 7f2567e00640 1 --2- 172.21.15.97:0/2990301644 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2568100210 0x7f2568199dc0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f255800e9f0 tx=0x7f255800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:42.678 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.706+0000 7f2564c00640 1 -- 172.21.15.97:0/2990301644 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f255800cd80 con 0x7f2568100210 2024-03-27T10:57:42.679 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.706+0000 7f2564c00640 1 -- 172.21.15.97:0/2990301644 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f25580108e0 con 0x7f2568100210 2024-03-27T10:57:42.679 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.706+0000 7f256da00640 1 -- 172.21.15.97:0/2990301644 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25681a1cd0 con 0x7f2568100210 2024-03-27T10:57:42.679 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.706+0000 7f256da00640 1 -- 172.21.15.97:0/2990301644 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25681a21d0 con 0x7f2568100210 2024-03-27T10:57:42.679 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.707+0000 7f2564c00640 1 -- 172.21.15.97:0/2990301644 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2558010630 con 0x7f2568100210 2024-03-27T10:57:42.681 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.710+0000 7f2571c55740 1 Processor -- start 2024-03-27T10:57:42.681 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.711+0000 7f2571c55740 1 -- start start 2024-03-27T10:57:42.682 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.711+0000 7f2571c55740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5565dbba0370 0x5565dbba0790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:42.682 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.711+0000 7f2571c55740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565dbba0d60 0x5565dbba5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:42.682 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.711+0000 7f2571c55740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5565dbba5ed0 0x5565dbba8360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:42.682 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.711+0000 7f2571c55740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5565dbafcd10 con 0x5565dbba0d60 2024-03-27T10:57:42.682 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.711+0000 7f2571c55740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5565dbaa5260 con 0x5565dbba0370 2024-03-27T10:57:42.682 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.711+0000 7f2571c55740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5565dbaf8cc0 con 0x5565dbba5ed0 2024-03-27T10:57:42.683 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.712+0000 7f256d000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5565dbba0370 0x5565dbba0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:42.683 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.712+0000 7f256d000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5565dbba0370 0x5565dbba0790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:33862/0 (socket says 172.21.15.97:33862) 2024-03-27T10:57:42.683 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.712+0000 7f256d000640 1 -- 172.21.15.97:0/1614318223 learned_addr learned my addr 172.21.15.97:0/1614318223 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:42.683 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.712+0000 7f2567400640 1 --2- 172.21.15.97:0/1614318223 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5565dbba5ed0 0x5565dbba8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:42.684 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.712+0000 7f256d000640 1 -- 172.21.15.97:0/1614318223 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5565dbba5ed0 msgr2=0x5565dbba8360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:42.684 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.712+0000 7f256d000640 1 --2- 172.21.15.97:0/1614318223 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5565dbba5ed0 0x5565dbba8360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.684 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.712+0000 7f256d000640 1 -- 172.21.15.97:0/1614318223 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565dbba0d60 msgr2=0x5565dbba5860 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-27T10:57:42.684 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.712+0000 7f256d000640 1 --2- 172.21.15.97:0/1614318223 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565dbba0d60 0x5565dbba5860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.684 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.712+0000 7f256d000640 1 -- 172.21.15.97:0/1614318223 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565dba9caa0 con 0x5565dbba0370 2024-03-27T10:57:42.684 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.712+0000 7f2564c00640 1 -- 172.21.15.97:0/2990301644 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f255801f020 con 0x7f2568100210 2024-03-27T10:57:42.685 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.714+0000 7f2564c00640 1 --2- 172.21.15.97:0/2990301644 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f254c0a3640 0x7f254c0a5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:42.685 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.714+0000 7f256d000640 1 --2- 172.21.15.97:0/1614318223 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5565dbba0370 0x5565dbba0790 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2560009ee0 tx=0x7f2560010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8eb75249901bc3ae server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:42.685 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.714+0000 7f2564c00640 1 -- 172.21.15.97:0/2990301644 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f25580d0090 con 0x7f2568100210 2024-03-27T10:57:42.685 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.714+0000 7f2567e00640 1 --2- 172.21.15.97:0/2990301644 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f254c0a3640 0x7f254c0a5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:42.685 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.715+0000 7f255d600640 1 -- 172.21.15.97:0/1614318223 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2560016070 con 0x5565dbba0370 2024-03-27T10:57:42.686 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.715+0000 7f255d600640 1 -- 172.21.15.97:0/1614318223 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2560004d20 con 0x5565dbba0370 2024-03-27T10:57:42.686 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.715+0000 7f255d600640 1 -- 172.21.15.97:0/1614318223 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2560012740 con 0x5565dbba0370 2024-03-27T10:57:42.687 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.716+0000 7f2571c55740 1 -- 172.21.15.97:0/1614318223 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5565dbba0370 msgr2=0x5565dbba0790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:42.687 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.716+0000 7f2571c55740 1 --2- 172.21.15.97:0/1614318223 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5565dbba0370 0x5565dbba0790 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2560009ee0 tx=0x7f2560010740 comp rx=0 tx=0).stop 2024-03-27T10:57:42.687 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.716+0000 7f2567e00640 1 --2- 172.21.15.97:0/2990301644 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f254c0a3640 0x7f254c0a5b00 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f25580989d0 tx=0x7f2558010d50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:42.687 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.716+0000 7f2571c55740 1 -- 172.21.15.97:0/1614318223 shutdown_connections 2024-03-27T10:57:42.687 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.716+0000 7f2571c55740 1 --2- 172.21.15.97:0/1614318223 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5565dbba5ed0 0x5565dbba8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.687 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.716+0000 7f2571c55740 1 --2- 172.21.15.97:0/1614318223 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565dbba0d60 0x5565dbba5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.687 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.716+0000 7f2571c55740 1 --2- 172.21.15.97:0/1614318223 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5565dbba0370 0x5565dbba0790 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.687 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.716+0000 7f2571c55740 1 -- 172.21.15.97:0/1614318223 >> 172.21.15.97:0/1614318223 conn(0x5565dbb930d0 msgr2=0x5565dbb934d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:42.687 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.716+0000 7f2571c55740 1 -- 172.21.15.97:0/1614318223 shutdown_connections 2024-03-27T10:57:42.688 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.716+0000 7f2571c55740 1 -- 172.21.15.97:0/1614318223 wait complete. 2024-03-27T10:57:42.688 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.717+0000 7f2571c55740 1 -- start start 2024-03-27T10:57:42.688 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.717+0000 7f2571c55740 1 Processor -- start 2024-03-27T10:57:42.689 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.717+0000 7f2571c55740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5565dbba0370 0x5565dbbaa670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:42.689 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.717+0000 7f2571c55740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5565dbba0d60 0x5565dbbaabb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:42.689 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.717+0000 7f2571c55740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565dbba5ed0 0x5565dbbb1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:42.689 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.717+0000 7f2571c55740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5565dbba8e80 con 0x5565dbba5ed0 2024-03-27T10:57:42.690 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.718+0000 7f2571c55740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5565dbab8630 con 0x5565dbba0370 2024-03-27T10:57:42.690 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.718+0000 7f2571c55740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5565dbba9000 con 0x5565dbba0d60 2024-03-27T10:57:42.690 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.718+0000 7f2571c55740 10 client.0 _ll_register_callbacks cb 0x5565dbabdc50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:57:42.690 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.718+0000 7f256d000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5565dbba0370 0x5565dbbaa670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:42.690 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.718+0000 7f256d000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5565dbba0370 0x5565dbbaa670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:33866/0 (socket says 172.21.15.97:33866) 2024-03-27T10:57:42.690 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.718+0000 7f256d000640 1 -- 172.21.15.97:0/1034220750 learned_addr learned my addr 172.21.15.97:0/1034220750 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:42.690 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.718+0000 7f256d000640 1 --2- 172.21.15.97:0/1034220750 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565dbba5ed0 0x5565dbbb1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:42.690 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.718+0000 7f2567400640 1 --2- 172.21.15.97:0/1034220750 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5565dbba0d60 0x5565dbbaabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:42.691 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.719+0000 7f256d000640 1 -- 172.21.15.97:0/1034220750 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5565dbba0d60 msgr2=0x5565dbbaabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:42.691 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.719+0000 7f256d000640 1 --2- 172.21.15.97:0/1034220750 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5565dbba0d60 0x5565dbbaabb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.691 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.719+0000 7f256d000640 1 -- 172.21.15.97:0/1034220750 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565dbba5ed0 msgr2=0x5565dbbb1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:42.691 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.719+0000 7f256d000640 1 --2- 172.21.15.97:0/1034220750 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565dbba5ed0 0x5565dbbb1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.691 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.719+0000 7f256d000640 1 -- 172.21.15.97:0/1034220750 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565dbac45f0 con 0x5565dbba0370 2024-03-27T10:57:42.691 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.719+0000 7f2567400640 1 --2- 172.21.15.97:0/1034220750 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5565dbba0d60 0x5565dbbaabb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T10:57:42.691 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.719+0000 7f256d000640 1 --2- 172.21.15.97:0/1034220750 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5565dbba0370 0x5565dbbaa670 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2560005f40 tx=0x7f2560005c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e01ffd24a7cc9215 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:42.691 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.720+0000 7f2552000640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-03-27T10:57:42.692 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.721+0000 7f2552000640 1 -- 172.21.15.97:0/1034220750 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2560016070 con 0x5565dbba0370 2024-03-27T10:57:42.692 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.721+0000 7f2552000640 1 -- 172.21.15.97:0/1034220750 <== mon.2 v2:172.21.15.114:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f2560013db0 con 0x5565dbba0370 2024-03-27T10:57:42.692 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.721+0000 7f2571c55740 1 -- 172.21.15.97:0/1034220750 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2560004d20 con 0x5565dbba0370 2024-03-27T10:57:42.692 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.721+0000 7f2552000640 1 -- 172.21.15.97:0/1034220750 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2560004ed0 con 0x5565dbba0370 2024-03-27T10:57:42.693 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.721+0000 7f2571c55740 10 client.35480 fetch_fsmap learned FSMap version 28 2024-03-27T10:57:42.693 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.721+0000 7f2571c55740 1 -- 172.21.15.97:0/1034220750 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5565db811210 con 0x5565dbba0370 2024-03-27T10:57:42.693 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.722+0000 7f2552000640 1 -- 172.21.15.97:0/1034220750 <== mon.2 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f256002e030 con 0x5565dbba0370 2024-03-27T10:57:42.693 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.722+0000 7f2552000640 1 -- 172.21.15.97:0/1034220750 <== mon.2 v2:172.21.15.114:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f256000f040 con 0x5565dbba0370 2024-03-27T10:57:42.693 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.722+0000 7f2571c55740 10 client.35480 fetch_fsmap finished waiting for FSMap version 28 2024-03-27T10:57:42.693 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.722+0000 7f2571c55740 10 client.35480 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T10:57:42.694 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.722+0000 7f2571c55740 20 client.35480 populate_metadata read hostname 'smithi097' 2024-03-27T10:57:42.694 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.722+0000 7f2571c55740 1 --2- 172.21.15.97:0/1034220750 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5565dbb9b920 0x5565dbb9dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:42.694 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.722+0000 7f2571c55740 4 client.35480 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T10:57:42.694 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.722+0000 7f2571c55740 1 -- 172.21.15.97:0/1034220750 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f2560004d20 con 0x5565dbb9b920 2024-03-27T10:57:42.694 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.723+0000 7f2567400640 1 --2- 172.21.15.97:0/1034220750 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5565dbb9b920 0x5565dbb9dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:42.694 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.724+0000 7f2567400640 1 --2- 172.21.15.97:0/1034220750 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5565dbb9b920 0x5565dbb9dd30 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68bc147704d88313 server_cookie=746bd01b52a9fa1d in_seq=0 out_seq=0 2024-03-27T10:57:42.694 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.724+0000 7f2552000640 10 client.35480 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T10:57:42.696 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.725+0000 7f2552000640 1 -- 172.21.15.97:0/1034220750 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x7f2560004d20 con 0x5565dbb9b920 2024-03-27T10:57:42.696 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.725+0000 7f2552000640 10 client.35480 handle_command_reply: tid=0 2024-03-27T10:57:42.700 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.726+0000 7f2571c55740 1 client.35480 shutdown 2024-03-27T10:57:42.700 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.726+0000 7f2571c55740 1 -- 172.21.15.97:0/1034220750 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5565dbba0370 msgr2=0x5565dbbaa670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:42.700 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.726+0000 7f2571c55740 1 --2- 172.21.15.97:0/1034220750 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5565dbba0370 0x5565dbbaa670 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2560005f40 tx=0x7f2560005c80 comp rx=0 tx=0).stop 2024-03-27T10:57:42.700 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.726+0000 7f2571c55740 1 -- 172.21.15.97:0/1034220750 shutdown_connections 2024-03-27T10:57:42.700 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.726+0000 7f2571c55740 1 --2- 172.21.15.97:0/1034220750 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5565dbb9b920 0x5565dbb9dd30 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.700 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.726+0000 7f2571c55740 1 --2- 172.21.15.97:0/1034220750 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565dbba5ed0 0x5565dbbb1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.700 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.726+0000 7f2571c55740 1 --2- 172.21.15.97:0/1034220750 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5565dbba0d60 0x5565dbbaabb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.700 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.726+0000 7f2552000640 0 client.35480 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T10:57:42.700 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.726+0000 7f2571c55740 1 --2- 172.21.15.97:0/1034220750 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5565dbba0370 0x5565dbbaa670 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.700 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.726+0000 7f2571c55740 1 -- 172.21.15.97:0/1034220750 >> 172.21.15.97:0/1034220750 conn(0x5565dbb930d0 msgr2=0x5565dbb92310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:42.700 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.726+0000 7f2571c55740 1 -- 172.21.15.97:0/1034220750 shutdown_connections 2024-03-27T10:57:42.700 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.726+0000 7f2571c55740 1 -- 172.21.15.97:0/1034220750 wait complete. 2024-03-27T10:57:42.700 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.729+0000 7f2571c55740 20 client.35480 trim_cache size 0 max 16384 2024-03-27T10:57:42.715 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.744+0000 7f2571c55740 1 Processor -- start 2024-03-27T10:57:42.715 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.744+0000 7f2571c55740 1 -- start start 2024-03-27T10:57:42.715 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.745+0000 7f2571c55740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5565dbba0370 0x5565dbbaa990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:42.716 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.745+0000 7f2571c55740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5565dbba0d60 0x5565dbbb1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:42.716 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.745+0000 7f2571c55740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565dbba5ed0 0x5565dbbc7160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:42.716 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.745+0000 7f2571c55740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f25681a6c30 con 0x5565dbba5ed0 2024-03-27T10:57:42.716 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.745+0000 7f2571c55740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f25681a8770 con 0x5565dbba0370 2024-03-27T10:57:42.716 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.745+0000 7f2571c55740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f25681a7300 con 0x5565dbba0d60 2024-03-27T10:57:42.716 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.745+0000 7f256d000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5565dbba0370 0x5565dbbaa990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:42.716 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.745+0000 7f256d000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565dbba5ed0 0x5565dbbc7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:42.716 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.745+0000 7f256d000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565dbba5ed0 0x5565dbbc7160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53576/0 (socket says 172.21.15.97:53576) 2024-03-27T10:57:42.717 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.745+0000 7f256d000640 1 -- 172.21.15.97:0/577530488 learned_addr learned my addr 172.21.15.97:0/577530488 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:42.717 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.745+0000 7f2567400640 1 --2- 172.21.15.97:0/577530488 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5565dbba0d60 0x5565dbbb1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:42.717 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.746+0000 7f256d000640 1 -- 172.21.15.97:0/577530488 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5565dbba0d60 msgr2=0x5565dbbb1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:42.717 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.746+0000 7f256d000640 1 --2- 172.21.15.97:0/577530488 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5565dbba0d60 0x5565dbbb1bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.717 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.746+0000 7f256d000640 1 -- 172.21.15.97:0/577530488 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5565dbba0370 msgr2=0x5565dbbaa990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:42.717 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.746+0000 7f256d000640 1 --2- 172.21.15.97:0/577530488 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5565dbba0370 0x5565dbbaa990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.717 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.746+0000 7f256d000640 1 -- 172.21.15.97:0/577530488 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565db8dd620 con 0x5565dbba5ed0 2024-03-27T10:57:42.718 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.746+0000 7f256d000640 1 --2- 172.21.15.97:0/577530488 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565dbba5ed0 0x5565dbbc7160 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f2560009ad0 tx=0x7f2560000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caf7458cb41460cb server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:42.718 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.746+0000 7f255cc00640 1 -- 172.21.15.97:0/577530488 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2560016040 con 0x5565dbba5ed0 2024-03-27T10:57:42.719 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.747+0000 7f255cc00640 1 -- 172.21.15.97:0/577530488 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2560004cc0 con 0x5565dbba5ed0 2024-03-27T10:57:42.719 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.747+0000 7f255cc00640 1 -- 172.21.15.97:0/577530488 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2560013320 con 0x5565dbba5ed0 2024-03-27T10:57:42.719 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.747+0000 7f2571c55740 1 -- 172.21.15.97:0/577530488 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565dbba5ed0 msgr2=0x5565dbbc7160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:42.719 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.747+0000 7f2571c55740 1 --2- 172.21.15.97:0/577530488 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565dbba5ed0 0x5565dbbc7160 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f2560009ad0 tx=0x7f2560000f40 comp rx=0 tx=0).stop 2024-03-27T10:57:42.719 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.747+0000 7f2571c55740 1 -- 172.21.15.97:0/577530488 shutdown_connections 2024-03-27T10:57:42.719 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.747+0000 7f2571c55740 1 --2- 172.21.15.97:0/577530488 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565dbba5ed0 0x5565dbbc7160 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.719 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.747+0000 7f2571c55740 1 --2- 172.21.15.97:0/577530488 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5565dbba0d60 0x5565dbbb1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.719 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.747+0000 7f2571c55740 1 --2- 172.21.15.97:0/577530488 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5565dbba0370 0x5565dbbaa990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.719 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.747+0000 7f2571c55740 1 -- 172.21.15.97:0/577530488 >> 172.21.15.97:0/577530488 conn(0x5565dbb930d0 msgr2=0x5565dbbaa280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:42.719 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.748+0000 7f2571c55740 1 -- 172.21.15.97:0/577530488 shutdown_connections 2024-03-27T10:57:42.719 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.748+0000 7f2571c55740 1 -- 172.21.15.97:0/577530488 wait complete. 2024-03-27T10:57:42.720 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.748+0000 7f2571c55740 1 -- start start 2024-03-27T10:57:42.720 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.748+0000 7f2571c55740 1 Processor -- start 2024-03-27T10:57:42.720 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.749+0000 7f2571c55740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5565dbba0370 0x5565dbbadc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:42.720 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.749+0000 7f2571c55740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5565dbba0d60 0x5565dbbae1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:42.720 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.749+0000 7f2571c55740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565dbba5ed0 0x5565dbbb11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:42.721 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.749+0000 7f2571c55740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f25681a80a0 con 0x5565dbba5ed0 2024-03-27T10:57:42.721 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.749+0000 7f2571c55740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f25681a79d0 con 0x5565dbba0370 2024-03-27T10:57:42.721 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.749+0000 7f2571c55740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f25681a4c10 con 0x5565dbba0d60 2024-03-27T10:57:42.721 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.749+0000 7f2571c55740 10 client.0 _ll_register_callbacks cb 0x7f2560009d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:57:42.721 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.749+0000 7f256d000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5565dbba0370 0x5565dbbadc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:42.721 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.749+0000 7f256d000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5565dbba0370 0x5565dbbadc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:33886/0 (socket says 172.21.15.97:33886) 2024-03-27T10:57:42.721 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.749+0000 7f256d000640 1 -- 172.21.15.97:0/1470313684 learned_addr learned my addr 172.21.15.97:0/1470313684 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:42.722 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.749+0000 7f256d000640 1 --2- 172.21.15.97:0/1470313684 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565dbba5ed0 0x5565dbbb11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:42.722 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.749+0000 7f2567400640 1 --2- 172.21.15.97:0/1470313684 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5565dbba0d60 0x5565dbbae1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:42.722 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.750+0000 7f256d000640 1 -- 172.21.15.97:0/1470313684 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5565dbba0d60 msgr2=0x5565dbbae1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:42.722 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.750+0000 7f256d000640 1 --2- 172.21.15.97:0/1470313684 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5565dbba0d60 0x5565dbbae1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.722 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.750+0000 7f256d000640 1 -- 172.21.15.97:0/1470313684 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565dbba5ed0 msgr2=0x5565dbbb11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:42.722 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.750+0000 7f256d000640 1 --2- 172.21.15.97:0/1470313684 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565dbba5ed0 0x5565dbbb11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.723 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.750+0000 7f256d000640 1 -- 172.21.15.97:0/1470313684 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565dbbb17d0 con 0x5565dbba0370 2024-03-27T10:57:42.723 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.750+0000 7f256d000640 1 --2- 172.21.15.97:0/1470313684 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5565dbba0370 0x5565dbbadc80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2560006010 tx=0x7f2560012500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae491475c69aca6d server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:42.723 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.750+0000 7f2552000640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-03-27T10:57:42.723 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.751+0000 7f2552000640 1 -- 172.21.15.97:0/1470313684 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2560016030 con 0x5565dbba0370 2024-03-27T10:57:42.723 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.751+0000 7f2552000640 1 -- 172.21.15.97:0/1470313684 <== mon.2 v2:172.21.15.114:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f2560012660 con 0x5565dbba0370 2024-03-27T10:57:42.723 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.751+0000 7f2571c55740 1 -- 172.21.15.97:0/1470313684 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2560004cc0 con 0x5565dbba0370 2024-03-27T10:57:42.723 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.751+0000 7f2552000640 1 -- 172.21.15.97:0/1470313684 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2560012860 con 0x5565dbba0370 2024-03-27T10:57:42.723 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.751+0000 7f2571c55740 10 client.35486 fetch_fsmap learned FSMap version 28 2024-03-27T10:57:42.723 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.751+0000 7f2571c55740 1 -- 172.21.15.97:0/1470313684 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5565dbc25160 con 0x5565dbba0370 2024-03-27T10:57:42.724 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.752+0000 7f2552000640 1 -- 172.21.15.97:0/1470313684 <== mon.2 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2560004cc0 con 0x5565dbba0370 2024-03-27T10:57:42.724 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.752+0000 7f2552000640 1 -- 172.21.15.97:0/1470313684 <== mon.2 v2:172.21.15.114:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f2560005070 con 0x5565dbba0370 2024-03-27T10:57:42.724 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.752+0000 7f2571c55740 10 client.35486 fetch_fsmap finished waiting for FSMap version 28 2024-03-27T10:57:42.724 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.752+0000 7f2571c55740 10 client.35486 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T10:57:42.724 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.752+0000 7f2571c55740 20 client.35486 populate_metadata read hostname 'smithi097' 2024-03-27T10:57:42.724 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.752+0000 7f2571c55740 1 --2- 172.21.15.97:0/1470313684 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5565dbc25830 0x5565dbc27c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:42.724 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.753+0000 7f2571c55740 4 client.35486 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-27T10:57:42.724 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.753+0000 7f2571c55740 1 -- 172.21.15.97:0/1470313684 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5565dbb0a250 con 0x5565dbc25830 2024-03-27T10:57:42.725 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.753+0000 7f2567400640 1 --2- 172.21.15.97:0/1470313684 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5565dbc25830 0x5565dbc27c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:42.725 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.754+0000 7f2567400640 1 --2- 172.21.15.97:0/1470313684 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5565dbc25830 0x5565dbc27c50 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb33593483810799 server_cookie=fca17cd729943211 in_seq=0 out_seq=0 2024-03-27T10:57:42.725 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.754+0000 7f2552000640 10 client.35486 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T10:57:42.733 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.761+0000 7f2552000640 1 -- 172.21.15.97:0/1470313684 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5565dbb0a250 con 0x5565dbc25830 2024-03-27T10:57:42.733 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.761+0000 7f2552000640 10 client.35486 handle_command_reply: tid=0 2024-03-27T10:57:42.733 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.761+0000 7f2571c55740 1 client.35486 shutdown 2024-03-27T10:57:42.733 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.762+0000 7f2571c55740 1 -- 172.21.15.97:0/1470313684 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5565dbba0370 msgr2=0x5565dbbadc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:42.733 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.762+0000 7f2571c55740 1 --2- 172.21.15.97:0/1470313684 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5565dbba0370 0x5565dbbadc80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2560006010 tx=0x7f2560012500 comp rx=0 tx=0).stop 2024-03-27T10:57:42.733 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.762+0000 7f2571c55740 1 -- 172.21.15.97:0/1470313684 shutdown_connections 2024-03-27T10:57:42.733 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.762+0000 7f2571c55740 1 --2- 172.21.15.97:0/1470313684 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5565dbc25830 0x5565dbc27c50 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.733 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.762+0000 7f2571c55740 1 --2- 172.21.15.97:0/1470313684 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565dbba5ed0 0x5565dbbb11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.733 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.762+0000 7f2571c55740 1 --2- 172.21.15.97:0/1470313684 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5565dbba0d60 0x5565dbbae1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.733 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.762+0000 7f2571c55740 1 --2- 172.21.15.97:0/1470313684 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5565dbba0370 0x5565dbbadc80 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.734 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.762+0000 7f2571c55740 1 -- 172.21.15.97:0/1470313684 >> 172.21.15.97:0/1470313684 conn(0x5565dbb930d0 msgr2=0x5565dbb91a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:42.734 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.762+0000 7f2552000640 0 client.35486 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T10:57:42.734 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.762+0000 7f2571c55740 1 -- 172.21.15.97:0/1470313684 shutdown_connections 2024-03-27T10:57:42.734 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.762+0000 7f2571c55740 1 -- 172.21.15.97:0/1470313684 wait complete. 2024-03-27T10:57:42.734 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.762+0000 7f2571c55740 20 client.35486 trim_cache size 0 max 16384 2024-03-27T10:57:42.734 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-27T10:57:42.734 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-27T10:57:42.734 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-27T10:57:42.734 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-27T10:57:42.735 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.763+0000 7f256da00640 1 -- 172.21.15.97:0/2990301644 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f254c0a3640 msgr2=0x7f254c0a5b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:42.735 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.763+0000 7f256da00640 1 --2- 172.21.15.97:0/2990301644 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f254c0a3640 0x7f254c0a5b00 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f25580989d0 tx=0x7f2558010d50 comp rx=0 tx=0).stop 2024-03-27T10:57:42.735 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.763+0000 7f256da00640 1 -- 172.21.15.97:0/2990301644 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2568100210 msgr2=0x7f2568199dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:42.735 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.763+0000 7f256da00640 1 --2- 172.21.15.97:0/2990301644 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2568100210 0x7f2568199dc0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f255800e9f0 tx=0x7f255800eeb0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.735 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.763+0000 7f256da00640 1 -- 172.21.15.97:0/2990301644 shutdown_connections 2024-03-27T10:57:42.735 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.763+0000 7f256da00640 1 --2- 172.21.15.97:0/2990301644 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f254c0a3640 0x7f254c0a5b00 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.735 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.763+0000 7f256da00640 1 --2- 172.21.15.97:0/2990301644 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2568101510 0x7f25681a1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.736 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.763+0000 7f256da00640 1 --2- 172.21.15.97:0/2990301644 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2568100b70 0x7f256819a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.736 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.763+0000 7f256da00640 1 --2- 172.21.15.97:0/2990301644 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2568100210 0x7f2568199dc0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.736 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.763+0000 7f256da00640 1 -- 172.21.15.97:0/2990301644 >> 172.21.15.97:0/2990301644 conn(0x7f25680fbe30 msgr2=0x7f25680fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:42.736 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.763+0000 7f256da00640 1 -- 172.21.15.97:0/2990301644 shutdown_connections 2024-03-27T10:57:42.736 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:42.764+0000 7f256da00640 1 -- 172.21.15.97:0/2990301644 wait complete. 2024-03-27T10:57:42.766 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:57:42 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/1674402243' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T10:57:42.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.880+0000 7f6e36200640 1 -- 172.21.15.97:0/4158081724 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e301024b0 msgr2=0x7f6e301048a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:42.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.880+0000 7f6e36200640 1 --2- 172.21.15.97:0/4158081724 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e301024b0 0x7f6e301048a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6e24009900 tx=0x7f6e2402bc30 comp rx=0 tx=0).stop 2024-03-27T10:57:42.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.880+0000 7f6e36200640 1 -- 172.21.15.97:0/4158081724 shutdown_connections 2024-03-27T10:57:42.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.880+0000 7f6e36200640 1 --2- 172.21.15.97:0/4158081724 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6e30107710 0x7f6e30109b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.880+0000 7f6e36200640 1 --2- 172.21.15.97:0/4158081724 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e30104de0 0x7f6e301071d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.880+0000 7f6e36200640 1 --2- 172.21.15.97:0/4158081724 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e301024b0 0x7f6e301048a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.880+0000 7f6e36200640 1 -- 172.21.15.97:0/4158081724 >> 172.21.15.97:0/4158081724 conn(0x7f6e300fbec0 msgr2=0x7f6e300fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:42.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.881+0000 7f6e36200640 1 -- 172.21.15.97:0/4158081724 shutdown_connections 2024-03-27T10:57:42.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.881+0000 7f6e36200640 1 -- 172.21.15.97:0/4158081724 wait complete. 2024-03-27T10:57:42.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.881+0000 7f6e36200640 1 Processor -- start 2024-03-27T10:57:42.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.882+0000 7f6e36200640 1 -- start start 2024-03-27T10:57:42.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.882+0000 7f6e36200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e301024b0 0x7f6e30199ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:42.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.882+0000 7f6e36200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e30104de0 0x7f6e3019a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:42.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.882+0000 7f6e36200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6e30107710 0x7f6e301a1450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:42.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.882+0000 7f6e36200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6e3010d5d0 con 0x7f6e30104de0 2024-03-27T10:57:42.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.882+0000 7f6e36200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f6e3010d450 con 0x7f6e301024b0 2024-03-27T10:57:42.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.882+0000 7f6e36200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6e3010d750 con 0x7f6e30107710 2024-03-27T10:57:42.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.882+0000 7f6e34e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e301024b0 0x7f6e30199ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:42.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.882+0000 7f6e34e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e301024b0 0x7f6e30199ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:33902/0 (socket says 172.21.15.97:33902) 2024-03-27T10:57:42.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.882+0000 7f6e34e00640 1 -- 172.21.15.97:0/644544093 learned_addr learned my addr 172.21.15.97:0/644544093 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:42.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.882+0000 7f6e2fe00640 1 --2- 172.21.15.97:0/644544093 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e30104de0 0x7f6e3019a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:42.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.883+0000 7f6e35800640 1 --2- 172.21.15.97:0/644544093 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6e30107710 0x7f6e301a1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:42.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.883+0000 7f6e34e00640 1 -- 172.21.15.97:0/644544093 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6e30107710 msgr2=0x7f6e301a1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:42.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.883+0000 7f6e34e00640 1 --2- 172.21.15.97:0/644544093 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6e30107710 0x7f6e301a1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.883+0000 7f6e34e00640 1 -- 172.21.15.97:0/644544093 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e30104de0 msgr2=0x7f6e3019a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:42.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.883+0000 7f6e34e00640 1 --2- 172.21.15.97:0/644544093 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e30104de0 0x7f6e3019a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.883+0000 7f6e34e00640 1 -- 172.21.15.97:0/644544093 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e301a1ac0 con 0x7f6e301024b0 2024-03-27T10:57:42.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.883+0000 7f6e34e00640 1 --2- 172.21.15.97:0/644544093 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e301024b0 0x7f6e30199ea0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6e24000c00 tx=0x7f6e240042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:42.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.884+0000 7f6e2d600640 1 -- 172.21.15.97:0/644544093 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e24043020 con 0x7f6e301024b0 2024-03-27T10:57:42.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.884+0000 7f6e2d600640 1 -- 172.21.15.97:0/644544093 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6e24004aa0 con 0x7f6e301024b0 2024-03-27T10:57:42.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.884+0000 7f6e2d600640 1 -- 172.21.15.97:0/644544093 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e24003d00 con 0x7f6e301024b0 2024-03-27T10:57:42.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.884+0000 7f6e36200640 1 -- 172.21.15.97:0/644544093 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e301a1d50 con 0x7f6e301024b0 2024-03-27T10:57:42.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.884+0000 7f6e36200640 1 -- 172.21.15.97:0/644544093 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e301a2190 con 0x7f6e301024b0 2024-03-27T10:57:42.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.887+0000 7f6e3a625740 1 Processor -- start 2024-03-27T10:57:42.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.887+0000 7f6e3a625740 1 -- start start 2024-03-27T10:57:42.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.887+0000 7f6e3a625740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c2fb8b2ab0 0x55c2fb8b2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:42.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.887+0000 7f6e3a625740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c2fb8b34a0 0x55c2fb8b7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:42.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.887+0000 7f6e3a625740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c2fb8b8610 0x55c2fb8baaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:42.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.887+0000 7f6e3a625740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55c2fb87a7b0 con 0x55c2fb8b34a0 2024-03-27T10:57:42.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.887+0000 7f6e3a625740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55c2fb817670 con 0x55c2fb8b8610 2024-03-27T10:57:42.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.887+0000 7f6e3a625740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55c2fb7bf970 con 0x55c2fb8b2ab0 2024-03-27T10:57:42.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.888+0000 7f6e2fe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c2fb8b8610 0x55c2fb8baaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:42.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.888+0000 7f6e2fe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c2fb8b8610 0x55c2fb8baaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:33918/0 (socket says 172.21.15.97:33918) 2024-03-27T10:57:42.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.888+0000 7f6e2fe00640 1 -- 172.21.15.97:0/415509171 learned_addr learned my addr 172.21.15.97:0/415509171 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:42.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.888+0000 7f6e35800640 1 --2- 172.21.15.97:0/415509171 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c2fb8b2ab0 0x55c2fb8b2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:42.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.888+0000 7f6e2fe00640 1 -- 172.21.15.97:0/415509171 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c2fb8b2ab0 msgr2=0x55c2fb8b2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:42.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.888+0000 7f6e2fe00640 1 --2- 172.21.15.97:0/415509171 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c2fb8b2ab0 0x55c2fb8b2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.888+0000 7f6e2fe00640 1 -- 172.21.15.97:0/415509171 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c2fb8b34a0 msgr2=0x55c2fb8b7fa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-27T10:57:42.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.888+0000 7f6e2fe00640 1 --2- 172.21.15.97:0/415509171 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c2fb8b34a0 0x55c2fb8b7fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.888+0000 7f6e2fe00640 1 -- 172.21.15.97:0/415509171 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2fb52b3f0 con 0x55c2fb8b8610 2024-03-27T10:57:42.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.889+0000 7f6e2fe00640 1 --2- 172.21.15.97:0/415509171 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c2fb8b8610 0x55c2fb8baaa0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c009c60 tx=0x7f6e1c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=713f9112c39e64e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:42.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.889+0000 7f6e2d600640 1 -- 172.21.15.97:0/644544093 <== mon.2 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f6e24046020 con 0x7f6e301024b0 2024-03-27T10:57:42.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.889+0000 7f6e22000640 1 -- 172.21.15.97:0/415509171 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e1c004230 con 0x55c2fb8b8610 2024-03-27T10:57:42.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.889+0000 7f6e22000640 1 -- 172.21.15.97:0/415509171 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6e1c0043d0 con 0x55c2fb8b8610 2024-03-27T10:57:42.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.889+0000 7f6e22000640 1 -- 172.21.15.97:0/415509171 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e1c004d80 con 0x55c2fb8b8610 2024-03-27T10:57:42.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.890+0000 7f6e3a625740 1 -- 172.21.15.97:0/415509171 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c2fb8b8610 msgr2=0x55c2fb8baaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:42.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.890+0000 7f6e3a625740 1 --2- 172.21.15.97:0/415509171 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c2fb8b8610 0x55c2fb8baaa0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c009c60 tx=0x7f6e1c011810 comp rx=0 tx=0).stop 2024-03-27T10:57:42.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.890+0000 7f6e3a625740 1 -- 172.21.15.97:0/415509171 shutdown_connections 2024-03-27T10:57:42.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.890+0000 7f6e3a625740 1 --2- 172.21.15.97:0/415509171 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c2fb8b8610 0x55c2fb8baaa0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.890+0000 7f6e3a625740 1 --2- 172.21.15.97:0/415509171 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c2fb8b34a0 0x55c2fb8b7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.890+0000 7f6e3a625740 1 --2- 172.21.15.97:0/415509171 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c2fb8b2ab0 0x55c2fb8b2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.890+0000 7f6e3a625740 1 -- 172.21.15.97:0/415509171 >> 172.21.15.97:0/415509171 conn(0x55c2fb8ae020 msgr2=0x55c2fb8ae420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:42.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.890+0000 7f6e3a625740 1 -- 172.21.15.97:0/415509171 shutdown_connections 2024-03-27T10:57:42.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.890+0000 7f6e3a625740 1 -- 172.21.15.97:0/415509171 wait complete. 2024-03-27T10:57:42.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.891+0000 7f6e3a625740 1 -- start start 2024-03-27T10:57:42.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.891+0000 7f6e3a625740 1 Processor -- start 2024-03-27T10:57:42.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.891+0000 7f6e2d600640 1 --2- 172.21.15.97:0/644544093 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f6e180a35a0 0x7f6e180a5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:42.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.892+0000 7f6e2d600640 1 -- 172.21.15.97:0/644544093 <== mon.2 v2:172.21.15.114:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f6e240f2010 con 0x7f6e301024b0 2024-03-27T10:57:42.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.892+0000 7f6e3a625740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c2fb8b2ab0 0x55c2fb8bcc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:42.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.892+0000 7f6e3a625740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c2fb8b34a0 0x55c2fb8bd160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:42.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.892+0000 7f6e3a625740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c2fb8b8610 0x55c2fb8c41a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:42.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.892+0000 7f6e3a625740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55c2fb7d2ca0 con 0x55c2fb8b8610 2024-03-27T10:57:42.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.892+0000 7f6e3a625740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55c2fb813620 con 0x55c2fb8b2ab0 2024-03-27T10:57:42.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.892+0000 7f6e3a625740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55c2fb8bb5c0 con 0x55c2fb8b34a0 2024-03-27T10:57:42.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.892+0000 7f6e3a625740 10 client.0 _ll_register_callbacks cb 0x55c2fb80f160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:57:42.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.892+0000 7f6e34e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c2fb8b2ab0 0x55c2fb8bcc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:42.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.892+0000 7f6e34e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c2fb8b2ab0 0x55c2fb8bcc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:33932/0 (socket says 172.21.15.97:33932) 2024-03-27T10:57:42.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.892+0000 7f6e34e00640 1 -- 172.21.15.97:0/3210089106 learned_addr learned my addr 172.21.15.97:0/3210089106 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:42.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.892+0000 7f6e35800640 1 --2- 172.21.15.97:0/644544093 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f6e180a35a0 0x7f6e180a5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:42.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.892+0000 7f6e2fe00640 1 --2- 172.21.15.97:0/3210089106 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c2fb8b34a0 0x55c2fb8bd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:42.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.892+0000 7f6e34e00640 1 -- 172.21.15.97:0/3210089106 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c2fb8b34a0 msgr2=0x55c2fb8bd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:42.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.892+0000 7f6e34e00640 1 --2- 172.21.15.97:0/3210089106 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c2fb8b34a0 0x55c2fb8bd160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.892+0000 7f6e34e00640 1 -- 172.21.15.97:0/3210089106 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c2fb8b8610 msgr2=0x55c2fb8c41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:42.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.893+0000 7f6e34e00640 1 --2- 172.21.15.97:0/3210089106 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c2fb8b8610 0x55c2fb8c41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.893+0000 7f6e34e00640 1 -- 172.21.15.97:0/3210089106 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2fb5f57d0 con 0x55c2fb8b2ab0 2024-03-27T10:57:42.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.893+0000 7f6e34e00640 1 --2- 172.21.15.97:0/3210089106 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c2fb8b2ab0 0x55c2fb8bcc20 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6e240c6c30 tx=0x7f6e24009650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45d7708eeee10493 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:42.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.893+0000 7f6e16a00640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-03-27T10:57:42.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.893+0000 7f6e16a00640 1 -- 172.21.15.97:0/3210089106 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e240d7060 con 0x55c2fb8b2ab0 2024-03-27T10:57:42.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.893+0000 7f6e3a625740 1 -- 172.21.15.97:0/3210089106 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e1c0043d0 con 0x55c2fb8b2ab0 2024-03-27T10:57:42.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.893+0000 7f6e35800640 1 --2- 172.21.15.97:0/644544093 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f6e180a35a0 0x7f6e180a5a60 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x55c2fb8b3d00 tx=0x7f6e28009640 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:42.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.893+0000 7f6e16a00640 1 -- 172.21.15.97:0/3210089106 <== mon.2 v2:172.21.15.114:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f6e240c6cd0 con 0x55c2fb8b2ab0 2024-03-27T10:57:42.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.894+0000 7f6e16a00640 1 -- 172.21.15.97:0/3210089106 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6e240dc050 con 0x55c2fb8b2ab0 2024-03-27T10:57:42.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.894+0000 7f6e3a625740 10 client.35498 fetch_fsmap learned FSMap version 28 2024-03-27T10:57:42.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.894+0000 7f6e3a625740 1 -- 172.21.15.97:0/3210089106 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c2fb8c7510 con 0x55c2fb8b2ab0 2024-03-27T10:57:42.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.894+0000 7f6e16a00640 1 -- 172.21.15.97:0/3210089106 <== mon.2 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6e1c0043d0 con 0x55c2fb8b2ab0 2024-03-27T10:57:42.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.894+0000 7f6e16a00640 1 -- 172.21.15.97:0/3210089106 <== mon.2 v2:172.21.15.114:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f6e240ba240 con 0x55c2fb8b2ab0 2024-03-27T10:57:42.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.895+0000 7f6e3a625740 10 client.35498 fetch_fsmap finished waiting for FSMap version 28 2024-03-27T10:57:42.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.895+0000 7f6e3a625740 10 client.35498 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-27T10:57:42.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.895+0000 7f6e3a625740 20 client.35498 populate_metadata read hostname 'smithi097' 2024-03-27T10:57:42.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.895+0000 7f6e3a625740 1 --2- 172.21.15.97:0/3210089106 >> [v2:172.21.15.114:6836/616775588,v1:172.21.15.114:6837/616775588] conn(0x55c2fb8c8ba0 0x55c2fb8cb010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:42.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.895+0000 7f6e3a625740 4 client.35498 mds_command: new command op to 34478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T10:57:42.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.895+0000 7f6e3a625740 1 -- 172.21.15.97:0/3210089106 --> [v2:172.21.15.114:6836/616775588,v1:172.21.15.114:6837/616775588] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c2fb824bb0 con 0x55c2fb8c8ba0 2024-03-27T10:57:42.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.895+0000 7f6e2fe00640 1 --2- 172.21.15.97:0/3210089106 >> [v2:172.21.15.114:6836/616775588,v1:172.21.15.114:6837/616775588] conn(0x55c2fb8c8ba0 0x55c2fb8cb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:42.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.896+0000 7f6e2fe00640 1 --2- 172.21.15.97:0/3210089106 >> [v2:172.21.15.114:6836/616775588,v1:172.21.15.114:6837/616775588] conn(0x55c2fb8c8ba0 0x55c2fb8cb010 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d04d1bd2ab8a2880 server_cookie=4542da713b1c454e in_seq=0 out_seq=0 2024-03-27T10:57:42.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.896+0000 7f6e16a00640 10 client.35498 ms_handle_connect on v2:172.21.15.114:6836/616775588 2024-03-27T10:57:42.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.897+0000 7f6e16a00640 1 -- 172.21.15.97:0/3210089106 <== mds.1 v2:172.21.15.114:6836/616775588 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55c2fb824bb0 con 0x55c2fb8c8ba0 2024-03-27T10:57:42.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.897+0000 7f6e16a00640 10 client.35498 handle_command_reply: tid=0 2024-03-27T10:57:42.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.897+0000 7f6e3a625740 1 client.35498 shutdown 2024-03-27T10:57:42.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.897+0000 7f6e3a625740 1 -- 172.21.15.97:0/3210089106 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c2fb8b2ab0 msgr2=0x55c2fb8bcc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:42.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.897+0000 7f6e3a625740 1 --2- 172.21.15.97:0/3210089106 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c2fb8b2ab0 0x55c2fb8bcc20 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6e240c6c30 tx=0x7f6e24009650 comp rx=0 tx=0).stop 2024-03-27T10:57:42.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.897+0000 7f6e3a625740 1 -- 172.21.15.97:0/3210089106 shutdown_connections 2024-03-27T10:57:42.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.897+0000 7f6e3a625740 1 --2- 172.21.15.97:0/3210089106 >> [v2:172.21.15.114:6836/616775588,v1:172.21.15.114:6837/616775588] conn(0x55c2fb8c8ba0 0x55c2fb8cb010 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.897+0000 7f6e3a625740 1 --2- 172.21.15.97:0/3210089106 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c2fb8b8610 0x55c2fb8c41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.897+0000 7f6e3a625740 1 --2- 172.21.15.97:0/3210089106 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c2fb8b34a0 0x55c2fb8bd160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.897+0000 7f6e3a625740 1 --2- 172.21.15.97:0/3210089106 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c2fb8b2ab0 0x55c2fb8bcc20 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.897+0000 7f6e3a625740 1 -- 172.21.15.97:0/3210089106 >> 172.21.15.97:0/3210089106 conn(0x55c2fb8ae020 msgr2=0x55c2fb8ad240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:42.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.897+0000 7f6e16a00640 0 client.35498 ms_handle_reset on v2:172.21.15.114:6836/616775588 2024-03-27T10:57:42.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.897+0000 7f6e3a625740 1 -- 172.21.15.97:0/3210089106 shutdown_connections 2024-03-27T10:57:42.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.897+0000 7f6e3a625740 1 -- 172.21.15.97:0/3210089106 wait complete. 2024-03-27T10:57:42.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.898+0000 7f6e3a625740 20 client.35498 trim_cache size 0 max 16384 2024-03-27T10:57:42.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.912+0000 7f6e3a625740 1 Processor -- start 2024-03-27T10:57:42.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.912+0000 7f6e3a625740 1 -- start start 2024-03-27T10:57:42.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.912+0000 7f6e3a625740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c2fb8b2ab0 0x55c2fb8b47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:42.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.912+0000 7f6e3a625740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c2fb8b34a0 0x55c2fb8c4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:42.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.912+0000 7f6e3a625740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c2fb8b8610 0x55c2fb8b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:42.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.912+0000 7f6e3a625740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6e301aaea0 con 0x55c2fb8b8610 2024-03-27T10:57:42.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.912+0000 7f6e3a625740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f6e301ac9e0 con 0x55c2fb8b34a0 2024-03-27T10:57:42.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.912+0000 7f6e3a625740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6e301ab570 con 0x55c2fb8b2ab0 2024-03-27T10:57:42.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.912+0000 7f6e35800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c2fb8b8610 0x55c2fb8b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:42.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.912+0000 7f6e35800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c2fb8b8610 0x55c2fb8b4d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53624/0 (socket says 172.21.15.97:53624) 2024-03-27T10:57:42.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.912+0000 7f6e35800640 1 -- 172.21.15.97:0/3608339986 learned_addr learned my addr 172.21.15.97:0/3608339986 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:42.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.912+0000 7f6e2fe00640 1 --2- 172.21.15.97:0/3608339986 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c2fb8b34a0 0x55c2fb8c4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:42.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.912+0000 7f6e34e00640 1 --2- 172.21.15.97:0/3608339986 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c2fb8b2ab0 0x55c2fb8b47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:42.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.913+0000 7f6e35800640 1 -- 172.21.15.97:0/3608339986 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c2fb8b2ab0 msgr2=0x55c2fb8b47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:42.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.913+0000 7f6e35800640 1 --2- 172.21.15.97:0/3608339986 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c2fb8b2ab0 0x55c2fb8b47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.913+0000 7f6e35800640 1 -- 172.21.15.97:0/3608339986 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c2fb8b34a0 msgr2=0x55c2fb8c4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:42.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.913+0000 7f6e35800640 1 --2- 172.21.15.97:0/3608339986 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c2fb8b34a0 0x55c2fb8c4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.913+0000 7f6e35800640 1 -- 172.21.15.97:0/3608339986 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2fb8aed90 con 0x55c2fb8b8610 2024-03-27T10:57:42.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.913+0000 7f6e35800640 1 --2- 172.21.15.97:0/3608339986 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c2fb8b8610 0x55c2fb8b4d00 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f6e28011560 tx=0x7f6e28016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75fe47d9ba47144b server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:42.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.914+0000 7f6e21600640 1 -- 172.21.15.97:0/3608339986 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e2801e6c0 con 0x55c2fb8b8610 2024-03-27T10:57:42.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.914+0000 7f6e21600640 1 -- 172.21.15.97:0/3608339986 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6e2801ed10 con 0x55c2fb8b8610 2024-03-27T10:57:42.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.914+0000 7f6e21600640 1 -- 172.21.15.97:0/3608339986 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e2801d400 con 0x55c2fb8b8610 2024-03-27T10:57:42.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.914+0000 7f6e3a625740 1 -- 172.21.15.97:0/3608339986 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c2fb8b8610 msgr2=0x55c2fb8b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:42.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.914+0000 7f6e3a625740 1 --2- 172.21.15.97:0/3608339986 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c2fb8b8610 0x55c2fb8b4d00 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f6e28011560 tx=0x7f6e28016000 comp rx=0 tx=0).stop 2024-03-27T10:57:42.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.914+0000 7f6e3a625740 1 -- 172.21.15.97:0/3608339986 shutdown_connections 2024-03-27T10:57:42.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.914+0000 7f6e3a625740 1 --2- 172.21.15.97:0/3608339986 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c2fb8b8610 0x55c2fb8b4d00 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.914+0000 7f6e3a625740 1 --2- 172.21.15.97:0/3608339986 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c2fb8b34a0 0x55c2fb8c4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.914+0000 7f6e3a625740 1 --2- 172.21.15.97:0/3608339986 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c2fb8b2ab0 0x55c2fb8b47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.914+0000 7f6e3a625740 1 -- 172.21.15.97:0/3608339986 >> 172.21.15.97:0/3608339986 conn(0x55c2fb8ae020 msgr2=0x55c2fb8bb990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:42.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.915+0000 7f6e3a625740 1 -- 172.21.15.97:0/3608339986 shutdown_connections 2024-03-27T10:57:42.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.915+0000 7f6e3a625740 1 -- 172.21.15.97:0/3608339986 wait complete. 2024-03-27T10:57:42.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.915+0000 7f6e3a625740 1 -- start start 2024-03-27T10:57:42.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.915+0000 7f6e3a625740 1 Processor -- start 2024-03-27T10:57:42.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.916+0000 7f6e3a625740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c2fb8b2ab0 0x55c2fb8c11f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:42.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.916+0000 7f6e3a625740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c2fb8b34a0 0x55c2fb8c1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:42.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.916+0000 7f6e3a625740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c2fb8b8610 0x55c2fb8c2740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:42.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.916+0000 7f6e3a625740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6e301ac310 con 0x55c2fb8b34a0 2024-03-27T10:57:42.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.916+0000 7f6e3a625740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f6e301abc40 con 0x55c2fb8b8610 2024-03-27T10:57:42.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.916+0000 7f6e3a625740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6e301a8e80 con 0x55c2fb8b2ab0 2024-03-27T10:57:42.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.916+0000 7f6e3a625740 10 client.0 _ll_register_callbacks cb 0x7f6e240ba790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:57:42.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.916+0000 7f6e2fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c2fb8b34a0 0x55c2fb8c1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:42.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.916+0000 7f6e2fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c2fb8b34a0 0x55c2fb8c1730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53628/0 (socket says 172.21.15.97:53628) 2024-03-27T10:57:42.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.916+0000 7f6e2fe00640 1 -- 172.21.15.97:0/808896647 learned_addr learned my addr 172.21.15.97:0/808896647 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:42.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.916+0000 7f6e34e00640 1 --2- 172.21.15.97:0/808896647 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c2fb8b2ab0 0x55c2fb8c11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:42.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.916+0000 7f6e35800640 1 --2- 172.21.15.97:0/808896647 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c2fb8b8610 0x55c2fb8c2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:42.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.916+0000 7f6e2fe00640 1 -- 172.21.15.97:0/808896647 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c2fb8b2ab0 msgr2=0x55c2fb8c11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:42.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.916+0000 7f6e2fe00640 1 --2- 172.21.15.97:0/808896647 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c2fb8b2ab0 0x55c2fb8c11f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.916+0000 7f6e2fe00640 1 -- 172.21.15.97:0/808896647 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c2fb8b8610 msgr2=0x55c2fb8c2740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:42.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.916+0000 7f6e2fe00640 1 --2- 172.21.15.97:0/808896647 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c2fb8b8610 0x55c2fb8c2740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.916+0000 7f6e2fe00640 1 -- 172.21.15.97:0/808896647 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2fb8c2d30 con 0x55c2fb8b34a0 2024-03-27T10:57:42.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.916+0000 7f6e35800640 1 --2- 172.21.15.97:0/808896647 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c2fb8b8610 0x55c2fb8c2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:57:42.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.917+0000 7f6e2fe00640 1 --2- 172.21.15.97:0/808896647 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c2fb8b34a0 0x55c2fb8c1730 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c0098f0 tx=0x7f6e1c0093c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa37c62bbe8a46dd server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:42.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.917+0000 7f6e16a00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T10:57:42.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.917+0000 7f6e16a00640 1 -- 172.21.15.97:0/808896647 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e1c030060 con 0x55c2fb8b34a0 2024-03-27T10:57:42.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.917+0000 7f6e16a00640 1 -- 172.21.15.97:0/808896647 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f6e1c005100 con 0x55c2fb8b34a0 2024-03-27T10:57:42.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.917+0000 7f6e3a625740 1 -- 172.21.15.97:0/808896647 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e2801ed10 con 0x55c2fb8b34a0 2024-03-27T10:57:42.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.917+0000 7f6e16a00640 1 -- 172.21.15.97:0/808896647 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6e1c005300 con 0x55c2fb8b34a0 2024-03-27T10:57:42.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.917+0000 7f6e3a625740 10 client.15831 fetch_fsmap learned FSMap version 28 2024-03-27T10:57:42.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.917+0000 7f6e3a625740 1 -- 172.21.15.97:0/808896647 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c2fb8c4060 con 0x55c2fb8b34a0 2024-03-27T10:57:42.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.918+0000 7f6e16a00640 1 -- 172.21.15.97:0/808896647 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6e2801ed10 con 0x55c2fb8b34a0 2024-03-27T10:57:42.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.918+0000 7f6e16a00640 1 -- 172.21.15.97:0/808896647 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f6e1c018070 con 0x55c2fb8b34a0 2024-03-27T10:57:42.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.918+0000 7f6e3a625740 10 client.15831 fetch_fsmap finished waiting for FSMap version 28 2024-03-27T10:57:42.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.918+0000 7f6e3a625740 10 client.15831 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-27T10:57:42.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.918+0000 7f6e3a625740 20 client.15831 populate_metadata read hostname 'smithi097' 2024-03-27T10:57:42.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.918+0000 7f6e3a625740 1 --2- 172.21.15.97:0/808896647 >> [v2:172.21.15.114:6836/616775588,v1:172.21.15.114:6837/616775588] conn(0x55c2fb92f120 0x55c2fb931510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:42.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.918+0000 7f6e3a625740 4 client.15831 mds_command: new command op to 34478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-27T10:57:42.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.918+0000 7f6e3a625740 1 -- 172.21.15.97:0/808896647 --> [v2:172.21.15.114:6836/616775588,v1:172.21.15.114:6837/616775588] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c2fb81ef00 con 0x55c2fb92f120 2024-03-27T10:57:42.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.919+0000 7f6e34e00640 1 --2- 172.21.15.97:0/808896647 >> [v2:172.21.15.114:6836/616775588,v1:172.21.15.114:6837/616775588] conn(0x55c2fb92f120 0x55c2fb931510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:42.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.919+0000 7f6e34e00640 1 --2- 172.21.15.97:0/808896647 >> [v2:172.21.15.114:6836/616775588,v1:172.21.15.114:6837/616775588] conn(0x55c2fb92f120 0x55c2fb931510 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=aaffbb74cadb89c4 server_cookie=dcfaf6664ca5204a in_seq=0 out_seq=0 2024-03-27T10:57:42.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.919+0000 7f6e16a00640 10 client.15831 ms_handle_connect on v2:172.21.15.114:6836/616775588 2024-03-27T10:57:42.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.920+0000 7f6e16a00640 1 -- 172.21.15.97:0/808896647 <== mds.1 v2:172.21.15.114:6836/616775588 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c2fb81ef00 con 0x55c2fb92f120 2024-03-27T10:57:42.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.920+0000 7f6e16a00640 10 client.15831 handle_command_reply: tid=0 2024-03-27T10:57:42.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.920+0000 7f6e3a625740 1 client.15831 shutdown 2024-03-27T10:57:42.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.920+0000 7f6e3a625740 1 -- 172.21.15.97:0/808896647 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c2fb8b34a0 msgr2=0x55c2fb8c1730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:42.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.920+0000 7f6e3a625740 1 --2- 172.21.15.97:0/808896647 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c2fb8b34a0 0x55c2fb8c1730 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c0098f0 tx=0x7f6e1c0093c0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.920+0000 7f6e3a625740 1 -- 172.21.15.97:0/808896647 shutdown_connections 2024-03-27T10:57:42.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.920+0000 7f6e3a625740 1 --2- 172.21.15.97:0/808896647 >> [v2:172.21.15.114:6836/616775588,v1:172.21.15.114:6837/616775588] conn(0x55c2fb92f120 0x55c2fb931510 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.920+0000 7f6e3a625740 1 --2- 172.21.15.97:0/808896647 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c2fb8b8610 0x55c2fb8c2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.920+0000 7f6e3a625740 1 --2- 172.21.15.97:0/808896647 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c2fb8b34a0 0x55c2fb8c1730 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.920+0000 7f6e3a625740 1 --2- 172.21.15.97:0/808896647 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c2fb8b2ab0 0x55c2fb8c11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.920+0000 7f6e16a00640 0 client.15831 ms_handle_reset on v2:172.21.15.114:6836/616775588 2024-03-27T10:57:42.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.920+0000 7f6e3a625740 1 -- 172.21.15.97:0/808896647 >> 172.21.15.97:0/808896647 conn(0x55c2fb8ae020 msgr2=0x55c2fb8b47c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:42.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.920+0000 7f6e3a625740 1 -- 172.21.15.97:0/808896647 shutdown_connections 2024-03-27T10:57:42.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.920+0000 7f6e3a625740 1 -- 172.21.15.97:0/808896647 wait complete. 2024-03-27T10:57:42.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.921+0000 7f6e3a625740 20 client.15831 trim_cache size 0 max 16384 2024-03-27T10:57:42.895 INFO:teuthology.orchestra.run.smithi097.stdout:[] 2024-03-27T10:57:42.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.921+0000 7f6e36200640 1 -- 172.21.15.97:0/644544093 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f6e180a35a0 msgr2=0x7f6e180a5a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:42.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.921+0000 7f6e36200640 1 --2- 172.21.15.97:0/644544093 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f6e180a35a0 0x7f6e180a5a60 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x55c2fb8b3d00 tx=0x7f6e28009640 comp rx=0 tx=0).stop 2024-03-27T10:57:42.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.921+0000 7f6e36200640 1 -- 172.21.15.97:0/644544093 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e301024b0 msgr2=0x7f6e30199ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:42.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.921+0000 7f6e36200640 1 --2- 172.21.15.97:0/644544093 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e301024b0 0x7f6e30199ea0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6e24000c00 tx=0x7f6e240042f0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.921+0000 7f6e36200640 1 -- 172.21.15.97:0/644544093 shutdown_connections 2024-03-27T10:57:42.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.921+0000 7f6e36200640 1 --2- 172.21.15.97:0/644544093 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6e30107710 0x7f6e301a1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.921+0000 7f6e36200640 1 --2- 172.21.15.97:0/644544093 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e30104de0 0x7f6e3019a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.921+0000 7f6e36200640 1 --2- 172.21.15.97:0/644544093 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f6e180a35a0 0x7f6e180a5a60 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.921+0000 7f6e36200640 1 --2- 172.21.15.97:0/644544093 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e301024b0 0x7f6e30199ea0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:42.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.921+0000 7f6e36200640 1 -- 172.21.15.97:0/644544093 >> 172.21.15.97:0/644544093 conn(0x7f6e300fbec0 msgr2=0x7f6e30101c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:42.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.922+0000 7f6e36200640 1 -- 172.21.15.97:0/644544093 shutdown_connections 2024-03-27T10:57:42.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:42.922+0000 7f6e36200640 1 -- 172.21.15.97:0/644544093 wait complete. 2024-03-27T10:57:42.908 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-27T10:57:43.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.154+0000 7fbb44e00640 1 -- 172.21.15.97:0/3868100163 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbb401055c0 msgr2=0x7fbb40105a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:43.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.154+0000 7fbb44e00640 1 --2- 172.21.15.97:0/3868100163 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbb401055c0 0x7fbb40105a40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fbb38009ca0 tx=0x7fbb3802c100 comp rx=0 tx=0).stop 2024-03-27T10:57:43.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.154+0000 7fbb44e00640 1 -- 172.21.15.97:0/3868100163 shutdown_connections 2024-03-27T10:57:43.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.154+0000 7fbb44e00640 1 --2- 172.21.15.97:0/3868100163 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbb401055c0 0x7fbb40105a40 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:43.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.154+0000 7fbb44e00640 1 --2- 172.21.15.97:0/3868100163 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb40102c10 0x7fbb40105080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:43.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.154+0000 7fbb44e00640 1 --2- 172.21.15.97:0/3868100163 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbb401002a0 0x7fbb401026d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:43.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.154+0000 7fbb44e00640 1 -- 172.21.15.97:0/3868100163 >> 172.21.15.97:0/3868100163 conn(0x7fbb400fbec0 msgr2=0x7fbb400fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:43.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.154+0000 7fbb44e00640 1 -- 172.21.15.97:0/3868100163 shutdown_connections 2024-03-27T10:57:43.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.154+0000 7fbb44e00640 1 -- 172.21.15.97:0/3868100163 wait complete. 2024-03-27T10:57:43.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.155+0000 7fbb44e00640 1 Processor -- start 2024-03-27T10:57:43.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.156+0000 7fbb44e00640 1 -- start start 2024-03-27T10:57:43.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.156+0000 7fbb44e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb401002a0 0x7fbb4019e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:43.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.156+0000 7fbb44e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbb40102c10 0x7fbb4019e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:43.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.156+0000 7fbb44e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbb401055c0 0x7fbb401a5740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:43.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.156+0000 7fbb44e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fbb40113b80 con 0x7fbb401002a0 2024-03-27T10:57:43.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.156+0000 7fbb44e00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fbb40113a00 con 0x7fbb401055c0 2024-03-27T10:57:43.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.156+0000 7fbb44e00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fbb40113d00 con 0x7fbb40102c10 2024-03-27T10:57:43.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.156+0000 7fbb3f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb401002a0 0x7fbb4019e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:43.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.156+0000 7fbb3f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb401002a0 0x7fbb4019e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53650/0 (socket says 172.21.15.97:53650) 2024-03-27T10:57:43.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.156+0000 7fbb3f400640 1 -- 172.21.15.97:0/1983961920 learned_addr learned my addr 172.21.15.97:0/1983961920 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:43.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.157+0000 7fbb3fe00640 1 --2- 172.21.15.97:0/1983961920 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbb401055c0 0x7fbb401a5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:43.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.157+0000 7fbb3ea00640 1 --2- 172.21.15.97:0/1983961920 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbb40102c10 0x7fbb4019e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:43.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.157+0000 7fbb3f400640 1 -- 172.21.15.97:0/1983961920 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbb40102c10 msgr2=0x7fbb4019e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:43.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.157+0000 7fbb3f400640 1 --2- 172.21.15.97:0/1983961920 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbb40102c10 0x7fbb4019e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:43.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.157+0000 7fbb3f400640 1 -- 172.21.15.97:0/1983961920 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbb401055c0 msgr2=0x7fbb401a5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:43.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.157+0000 7fbb3f400640 1 --2- 172.21.15.97:0/1983961920 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbb401055c0 0x7fbb401a5740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:43.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.157+0000 7fbb3f400640 1 -- 172.21.15.97:0/1983961920 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb401a5db0 con 0x7fbb401002a0 2024-03-27T10:57:43.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.157+0000 7fbb3fe00640 1 --2- 172.21.15.97:0/1983961920 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbb401055c0 0x7fbb401a5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T10:57:43.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.158+0000 7fbb3f400640 1 --2- 172.21.15.97:0/1983961920 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb401002a0 0x7fbb4019e190 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fbb3000d770 tx=0x7fbb3000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:43.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.159+0000 7fbb37e00640 1 -- 172.21.15.97:0/1983961920 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb30017070 con 0x7fbb401002a0 2024-03-27T10:57:43.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.159+0000 7fbb37e00640 1 -- 172.21.15.97:0/1983961920 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbb300054d0 con 0x7fbb401002a0 2024-03-27T10:57:43.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.159+0000 7fbb37e00640 1 -- 172.21.15.97:0/1983961920 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb30004cf0 con 0x7fbb401002a0 2024-03-27T10:57:43.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.160+0000 7fbb44e00640 1 -- 172.21.15.97:0/1983961920 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb401a60a0 con 0x7fbb401002a0 2024-03-27T10:57:43.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.160+0000 7fbb44e00640 1 -- 172.21.15.97:0/1983961920 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb401a65a0 con 0x7fbb401002a0 2024-03-27T10:57:43.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.163+0000 7fbb37e00640 1 -- 172.21.15.97:0/1983961920 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7fbb30004e90 con 0x7fbb401002a0 2024-03-27T10:57:43.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.164+0000 7fbb49033740 1 Processor -- start 2024-03-27T10:57:43.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.164+0000 7fbb49033740 1 -- start start 2024-03-27T10:57:43.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.165+0000 7fbb49033740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55801a4caab0 0x55801a4caed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:43.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.165+0000 7fbb49033740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55801a4cb4a0 0x55801a4cffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:43.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.165+0000 7fbb49033740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55801a4d0610 0x55801a4d2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:43.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.165+0000 7fbb49033740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55801a4927b0 con 0x55801a4caab0 2024-03-27T10:57:43.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.165+0000 7fbb49033740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55801a42f670 con 0x55801a4cb4a0 2024-03-27T10:57:43.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.165+0000 7fbb49033740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55801a3d7970 con 0x55801a4d0610 2024-03-27T10:57:43.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.165+0000 7fbb3f400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55801a4cb4a0 0x55801a4cffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:43.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.165+0000 7fbb3f400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55801a4cb4a0 0x55801a4cffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:33978/0 (socket says 172.21.15.97:33978) 2024-03-27T10:57:43.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.165+0000 7fbb3f400640 1 -- 172.21.15.97:0/1575195253 learned_addr learned my addr 172.21.15.97:0/1575195253 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:43.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.165+0000 7fbb3fe00640 1 --2- 172.21.15.97:0/1575195253 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55801a4caab0 0x55801a4caed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:43.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.166+0000 7fbb3ea00640 1 --2- 172.21.15.97:0/1575195253 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55801a4d0610 0x55801a4d2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:43.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.166+0000 7fbb37e00640 1 --2- 172.21.15.97:0/1983961920 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fbb240a3690 0x7fbb240a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:43.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.166+0000 7fbb3f400640 1 -- 172.21.15.97:0/1575195253 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55801a4d0610 msgr2=0x55801a4d2aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:43.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.166+0000 7fbb3f400640 1 --2- 172.21.15.97:0/1575195253 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55801a4d0610 0x55801a4d2aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:43.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.166+0000 7fbb3f400640 1 -- 172.21.15.97:0/1575195253 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55801a4caab0 msgr2=0x55801a4caed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:43.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.166+0000 7fbb3f400640 1 --2- 172.21.15.97:0/1575195253 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55801a4caab0 0x55801a4caed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:43.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.166+0000 7fbb3f400640 1 -- 172.21.15.97:0/1575195253 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55801a1433f0 con 0x55801a4cb4a0 2024-03-27T10:57:43.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.166+0000 7fbb3ea00640 1 --2- 172.21.15.97:0/1575195253 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55801a4d0610 0x55801a4d2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:57:43.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.166+0000 7fbb3fe00640 1 --2- 172.21.15.97:0/1575195253 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55801a4caab0 0x55801a4caed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T10:57:43.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.166+0000 7fbb37e00640 1 -- 172.21.15.97:0/1983961920 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fbb300cfbc0 con 0x7fbb401002a0 2024-03-27T10:57:43.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.166+0000 7fbb3f400640 1 --2- 172.21.15.97:0/1575195253 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55801a4cb4a0 0x55801a4cffa0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbb30071ab0 tx=0x7fbb30066880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=293500b208b0879 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:43.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.166+0000 7fbb34c00640 1 -- 172.21.15.97:0/1575195253 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb3009bb80 con 0x55801a4cb4a0 2024-03-27T10:57:43.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.166+0000 7fbb3fe00640 1 --2- 172.21.15.97:0/1983961920 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fbb240a3690 0x7fbb240a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:43.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.167+0000 7fbb34c00640 1 -- 172.21.15.97:0/1575195253 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbb30071310 con 0x55801a4cb4a0 2024-03-27T10:57:43.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.167+0000 7fbb34c00640 1 -- 172.21.15.97:0/1575195253 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb30005180 con 0x55801a4cb4a0 2024-03-27T10:57:43.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.167+0000 7fbb49033740 1 -- 172.21.15.97:0/1575195253 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55801a4cb4a0 msgr2=0x55801a4cffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:43.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.167+0000 7fbb49033740 1 --2- 172.21.15.97:0/1575195253 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55801a4cb4a0 0x55801a4cffa0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbb30071ab0 tx=0x7fbb30066880 comp rx=0 tx=0).stop 2024-03-27T10:57:43.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.168+0000 7fbb49033740 1 -- 172.21.15.97:0/1575195253 shutdown_connections 2024-03-27T10:57:43.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.168+0000 7fbb49033740 1 --2- 172.21.15.97:0/1575195253 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55801a4d0610 0x55801a4d2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:43.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.168+0000 7fbb49033740 1 --2- 172.21.15.97:0/1575195253 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55801a4cb4a0 0x55801a4cffa0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:43.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.168+0000 7fbb49033740 1 --2- 172.21.15.97:0/1575195253 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55801a4caab0 0x55801a4caed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:43.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.168+0000 7fbb49033740 1 -- 172.21.15.97:0/1575195253 >> 172.21.15.97:0/1575195253 conn(0x55801a4c6020 msgr2=0x55801a4c6420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:43.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.168+0000 7fbb49033740 1 -- 172.21.15.97:0/1575195253 shutdown_connections 2024-03-27T10:57:43.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.168+0000 7fbb49033740 1 -- 172.21.15.97:0/1575195253 wait complete. 2024-03-27T10:57:43.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.168+0000 7fbb3fe00640 1 --2- 172.21.15.97:0/1983961920 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fbb240a3690 0x7fbb240a5b50 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fbb38032380 tx=0x7fbb3802e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:43.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.169+0000 7fbb49033740 1 -- start start 2024-03-27T10:57:43.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.169+0000 7fbb49033740 1 Processor -- start 2024-03-27T10:57:43.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.169+0000 7fbb49033740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55801a4caab0 0x55801a4d4b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:43.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.169+0000 7fbb49033740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55801a4cb4a0 0x55801a4d5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:43.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.169+0000 7fbb49033740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55801a4d0610 0x55801a4dc0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:43.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.169+0000 7fbb49033740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55801a3eaca0 con 0x55801a4cb4a0 2024-03-27T10:57:43.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.169+0000 7fbb49033740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55801a42b620 con 0x55801a4caab0 2024-03-27T10:57:43.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.169+0000 7fbb49033740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55801a4d3560 con 0x55801a4d0610 2024-03-27T10:57:43.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.170+0000 7fbb49033740 10 client.0 _ll_register_callbacks cb 0x55801a427160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:57:43.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.170+0000 7fbb3ea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55801a4cb4a0 0x55801a4d5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:43.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.170+0000 7fbb3ea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55801a4cb4a0 0x55801a4d5090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53658/0 (socket says 172.21.15.97:53658) 2024-03-27T10:57:43.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.170+0000 7fbb3ea00640 1 -- 172.21.15.97:0/2377476270 learned_addr learned my addr 172.21.15.97:0/2377476270 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:43.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.170+0000 7fbb3fe00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55801a4d0610 0x55801a4dc0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:43.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.170+0000 7fbb3ea00640 1 -- 172.21.15.97:0/2377476270 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55801a4d0610 msgr2=0x55801a4dc0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:43.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.170+0000 7fbb3ea00640 1 --2- 172.21.15.97:0/2377476270 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55801a4d0610 0x55801a4dc0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:43.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.170+0000 7fbb3ea00640 1 -- 172.21.15.97:0/2377476270 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55801a4caab0 msgr2=0x55801a4d4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:43.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.170+0000 7fbb3ea00640 1 --2- 172.21.15.97:0/2377476270 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55801a4caab0 0x55801a4d4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:43.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.170+0000 7fbb3ea00640 1 -- 172.21.15.97:0/2377476270 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55801a20d7d0 con 0x55801a4cb4a0 2024-03-27T10:57:43.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.171+0000 7fbb3ea00640 1 --2- 172.21.15.97:0/2377476270 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55801a4cb4a0 0x55801a4d5090 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c012700 tx=0x7fbb2c012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed6ed66459c6401 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:43.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.171+0000 7fbb29600640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T10:57:43.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.171+0000 7fbb29600640 1 -- 172.21.15.97:0/2377476270 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb2c01f060 con 0x55801a4cb4a0 2024-03-27T10:57:43.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.171+0000 7fbb49033740 1 -- 172.21.15.97:0/2377476270 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55801a43cbb0 con 0x55801a4cb4a0 2024-03-27T10:57:43.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.171+0000 7fbb29600640 1 -- 172.21.15.97:0/2377476270 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fbb2c010d60 con 0x55801a4cb4a0 2024-03-27T10:57:43.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.171+0000 7fbb29600640 1 -- 172.21.15.97:0/2377476270 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbb2c01a430 con 0x55801a4cb4a0 2024-03-27T10:57:43.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.171+0000 7fbb49033740 10 client.15843 fetch_fsmap learned FSMap version 28 2024-03-27T10:57:43.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.171+0000 7fbb49033740 1 -- 172.21.15.97:0/2377476270 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55801a4df4d0 con 0x55801a4cb4a0 2024-03-27T10:57:43.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.172+0000 7fbb29600640 1 -- 172.21.15.97:0/2377476270 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55801a43cbb0 con 0x55801a4cb4a0 2024-03-27T10:57:43.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.172+0000 7fbb29600640 1 -- 172.21.15.97:0/2377476270 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fbb2c00e4d0 con 0x55801a4cb4a0 2024-03-27T10:57:43.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.172+0000 7fbb49033740 10 client.15843 fetch_fsmap finished waiting for FSMap version 28 2024-03-27T10:57:43.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.172+0000 7fbb49033740 10 client.15843 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-27T10:57:43.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.172+0000 7fbb49033740 20 client.15843 populate_metadata read hostname 'smithi097' 2024-03-27T10:57:43.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.172+0000 7fbb49033740 1 --2- 172.21.15.97:0/2377476270 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55801a4e0cc0 0x55801a4e3120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:43.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.172+0000 7fbb49033740 4 client.15843 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T10:57:43.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.172+0000 7fbb49033740 1 -- 172.21.15.97:0/2377476270 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55801a436f00 con 0x55801a4e0cc0 2024-03-27T10:57:43.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.173+0000 7fbb3f400640 1 --2- 172.21.15.97:0/2377476270 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55801a4e0cc0 0x55801a4e3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:43.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.173+0000 7fbb3f400640 1 --2- 172.21.15.97:0/2377476270 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55801a4e0cc0 0x55801a4e3120 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2d141d745321a00 server_cookie=5b284d8ef957fc70 in_seq=0 out_seq=0 2024-03-27T10:57:43.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.173+0000 7fbb29600640 10 client.15843 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T10:57:43.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.175+0000 7fbb29600640 1 -- 172.21.15.97:0/2377476270 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55801a436f00 con 0x55801a4e0cc0 2024-03-27T10:57:43.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.175+0000 7fbb29600640 10 client.15843 handle_command_reply: tid=0 2024-03-27T10:57:43.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.175+0000 7fbb49033740 1 client.15843 shutdown 2024-03-27T10:57:43.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.175+0000 7fbb49033740 1 -- 172.21.15.97:0/2377476270 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55801a4cb4a0 msgr2=0x55801a4d5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:43.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.175+0000 7fbb49033740 1 --2- 172.21.15.97:0/2377476270 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55801a4cb4a0 0x55801a4d5090 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c012700 tx=0x7fbb2c012bc0 comp rx=0 tx=0).stop 2024-03-27T10:57:43.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.175+0000 7fbb49033740 1 -- 172.21.15.97:0/2377476270 shutdown_connections 2024-03-27T10:57:43.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.175+0000 7fbb49033740 1 --2- 172.21.15.97:0/2377476270 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55801a4e0cc0 0x55801a4e3120 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:43.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.175+0000 7fbb49033740 1 --2- 172.21.15.97:0/2377476270 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55801a4d0610 0x55801a4dc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:43.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.175+0000 7fbb49033740 1 --2- 172.21.15.97:0/2377476270 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55801a4cb4a0 0x55801a4d5090 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:43.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.175+0000 7fbb29600640 0 client.15843 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T10:57:43.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.175+0000 7fbb49033740 1 --2- 172.21.15.97:0/2377476270 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55801a4caab0 0x55801a4d4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:43.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.175+0000 7fbb49033740 1 -- 172.21.15.97:0/2377476270 >> 172.21.15.97:0/2377476270 conn(0x55801a4c6020 msgr2=0x55801a4c5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:43.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.175+0000 7fbb49033740 1 -- 172.21.15.97:0/2377476270 shutdown_connections 2024-03-27T10:57:43.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.176+0000 7fbb49033740 1 -- 172.21.15.97:0/2377476270 wait complete. 2024-03-27T10:57:43.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.176+0000 7fbb49033740 20 client.15843 trim_cache size 0 max 16384 2024-03-27T10:57:43.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.203+0000 7fbb49033740 1 Processor -- start 2024-03-27T10:57:43.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.203+0000 7fbb49033740 1 -- start start 2024-03-27T10:57:43.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.203+0000 7fbb49033740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55801a4caab0 0x55801a4c49b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:43.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.203+0000 7fbb49033740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55801a4cb4a0 0x55801a4dc090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:43.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.203+0000 7fbb49033740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55801a4d0610 0x55801a4c4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:43.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.203+0000 7fbb49033740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fbb401af240 con 0x55801a4caab0 2024-03-27T10:57:43.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.203+0000 7fbb49033740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fbb401b0d80 con 0x55801a4d0610 2024-03-27T10:57:43.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.203+0000 7fbb49033740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fbb401af910 con 0x55801a4cb4a0 2024-03-27T10:57:43.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.204+0000 7fbb3f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55801a4caab0 0x55801a4c49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:43.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.204+0000 7fbb3f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55801a4caab0 0x55801a4c49b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53660/0 (socket says 172.21.15.97:53660) 2024-03-27T10:57:43.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.204+0000 7fbb3f400640 1 -- 172.21.15.97:0/3683274959 learned_addr learned my addr 172.21.15.97:0/3683274959 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:43.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.204+0000 7fbb3ea00640 1 --2- 172.21.15.97:0/3683274959 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55801a4cb4a0 0x55801a4dc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:43.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.204+0000 7fbb3fe00640 1 --2- 172.21.15.97:0/3683274959 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55801a4d0610 0x55801a4c4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:43.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.204+0000 7fbb3f400640 1 -- 172.21.15.97:0/3683274959 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55801a4cb4a0 msgr2=0x55801a4dc090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:43.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.204+0000 7fbb3f400640 1 --2- 172.21.15.97:0/3683274959 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55801a4cb4a0 0x55801a4dc090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:43.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.204+0000 7fbb3f400640 1 -- 172.21.15.97:0/3683274959 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55801a4d0610 msgr2=0x55801a4c4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:43.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.204+0000 7fbb3f400640 1 --2- 172.21.15.97:0/3683274959 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55801a4d0610 0x55801a4c4ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:43.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.204+0000 7fbb3f400640 1 -- 172.21.15.97:0/3683274959 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55801a4c6d90 con 0x55801a4caab0 2024-03-27T10:57:43.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.205+0000 7fbb3f400640 1 --2- 172.21.15.97:0/3683274959 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55801a4caab0 0x55801a4c49b0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fbb300714d0 tx=0x7fbb30009f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d81f2f686e6cbdc server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:43.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.205+0000 7fbb2be00640 1 -- 172.21.15.97:0/3683274959 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb300c8040 con 0x55801a4caab0 2024-03-27T10:57:43.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.205+0000 7fbb2be00640 1 -- 172.21.15.97:0/3683274959 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbb300d6070 con 0x55801a4caab0 2024-03-27T10:57:43.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.205+0000 7fbb2be00640 1 -- 172.21.15.97:0/3683274959 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb30071500 con 0x55801a4caab0 2024-03-27T10:57:43.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.206+0000 7fbb49033740 1 -- 172.21.15.97:0/3683274959 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55801a4caab0 msgr2=0x55801a4c49b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:43.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.206+0000 7fbb49033740 1 --2- 172.21.15.97:0/3683274959 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55801a4caab0 0x55801a4c49b0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fbb300714d0 tx=0x7fbb30009f50 comp rx=0 tx=0).stop 2024-03-27T10:57:43.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.206+0000 7fbb49033740 1 -- 172.21.15.97:0/3683274959 shutdown_connections 2024-03-27T10:57:43.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.206+0000 7fbb49033740 1 --2- 172.21.15.97:0/3683274959 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55801a4d0610 0x55801a4c4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:43.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.206+0000 7fbb49033740 1 --2- 172.21.15.97:0/3683274959 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55801a4cb4a0 0x55801a4dc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:43.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.206+0000 7fbb49033740 1 --2- 172.21.15.97:0/3683274959 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55801a4caab0 0x55801a4c49b0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:43.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.206+0000 7fbb49033740 1 -- 172.21.15.97:0/3683274959 >> 172.21.15.97:0/3683274959 conn(0x55801a4c6020 msgr2=0x55801a4cc6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:43.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.206+0000 7fbb49033740 1 -- 172.21.15.97:0/3683274959 shutdown_connections 2024-03-27T10:57:43.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.206+0000 7fbb49033740 1 -- 172.21.15.97:0/3683274959 wait complete. 2024-03-27T10:57:43.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.207+0000 7fbb49033740 1 -- start start 2024-03-27T10:57:43.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.207+0000 7fbb49033740 1 Processor -- start 2024-03-27T10:57:43.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.207+0000 7fbb49033740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55801a4caab0 0x55801a4d9390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:43.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.207+0000 7fbb49033740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55801a4cb4a0 0x55801a4d98d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:43.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.207+0000 7fbb49033740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55801a4d0610 0x55801a4da7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:43.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.207+0000 7fbb49033740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fbb401b06b0 con 0x55801a4cb4a0 2024-03-27T10:57:43.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.207+0000 7fbb49033740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fbb401affe0 con 0x55801a4d0610 2024-03-27T10:57:43.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.207+0000 7fbb49033740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fbb401ad220 con 0x55801a4caab0 2024-03-27T10:57:43.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.207+0000 7fbb49033740 10 client.0 _ll_register_callbacks cb 0x7fbb2c009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:57:43.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.208+0000 7fbb3ea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55801a4cb4a0 0x55801a4d98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:43.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.208+0000 7fbb3ea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55801a4cb4a0 0x55801a4d98d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53670/0 (socket says 172.21.15.97:53670) 2024-03-27T10:57:43.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.208+0000 7fbb3ea00640 1 -- 172.21.15.97:0/130629142 learned_addr learned my addr 172.21.15.97:0/130629142 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:43.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.208+0000 7fbb3f400640 1 --2- 172.21.15.97:0/130629142 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55801a4caab0 0x55801a4d9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:43.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.208+0000 7fbb3fe00640 1 --2- 172.21.15.97:0/130629142 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55801a4d0610 0x55801a4da7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:43.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.208+0000 7fbb3ea00640 1 -- 172.21.15.97:0/130629142 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55801a4caab0 msgr2=0x55801a4d9390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:43.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.208+0000 7fbb3ea00640 1 --2- 172.21.15.97:0/130629142 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55801a4caab0 0x55801a4d9390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:43.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.208+0000 7fbb3ea00640 1 -- 172.21.15.97:0/130629142 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55801a4d0610 msgr2=0x55801a4da7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:43.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.208+0000 7fbb3ea00640 1 --2- 172.21.15.97:0/130629142 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55801a4d0610 0x55801a4da7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:43.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.208+0000 7fbb3ea00640 1 -- 172.21.15.97:0/130629142 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55801a4dadc0 con 0x55801a4cb4a0 2024-03-27T10:57:43.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.209+0000 7fbb3ea00640 1 --2- 172.21.15.97:0/130629142 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55801a4cb4a0 0x55801a4d98d0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x55801a4d6030 tx=0x7fbb2c00edb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14afa3f6dbad2145 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:43.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.209+0000 7fbb29600640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T10:57:43.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.209+0000 7fbb29600640 1 -- 172.21.15.97:0/130629142 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb2c02a050 con 0x55801a4cb4a0 2024-03-27T10:57:43.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.209+0000 7fbb29600640 1 -- 172.21.15.97:0/130629142 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fbb2c00e3b0 con 0x55801a4cb4a0 2024-03-27T10:57:43.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.209+0000 7fbb49033740 1 -- 172.21.15.97:0/130629142 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb300d6070 con 0x55801a4cb4a0 2024-03-27T10:57:43.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.210+0000 7fbb29600640 1 -- 172.21.15.97:0/130629142 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbb2c0107a0 con 0x55801a4cb4a0 2024-03-27T10:57:43.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.210+0000 7fbb49033740 10 client.15849 fetch_fsmap learned FSMap version 28 2024-03-27T10:57:43.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.210+0000 7fbb49033740 1 -- 172.21.15.97:0/130629142 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55801a4dc160 con 0x55801a4cb4a0 2024-03-27T10:57:43.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.210+0000 7fbb29600640 1 -- 172.21.15.97:0/130629142 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbb300d6070 con 0x55801a4cb4a0 2024-03-27T10:57:43.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.210+0000 7fbb29600640 1 -- 172.21.15.97:0/130629142 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fbb2c004780 con 0x55801a4cb4a0 2024-03-27T10:57:43.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.210+0000 7fbb49033740 10 client.15849 fetch_fsmap finished waiting for FSMap version 28 2024-03-27T10:57:43.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.210+0000 7fbb49033740 10 client.15849 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-27T10:57:43.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.210+0000 7fbb49033740 20 client.15849 populate_metadata read hostname 'smithi097' 2024-03-27T10:57:43.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.210+0000 7fbb49033740 1 --2- 172.21.15.97:0/130629142 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55801a5475d0 0x55801a5499f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:43.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.210+0000 7fbb49033740 4 client.15849 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-27T10:57:43.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.210+0000 7fbb49033740 1 -- 172.21.15.97:0/130629142 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55801a3b4560 con 0x55801a5475d0 2024-03-27T10:57:43.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.211+0000 7fbb3f400640 1 --2- 172.21.15.97:0/130629142 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55801a5475d0 0x55801a5499f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:43.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.212+0000 7fbb29600640 10 client.15849 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T10:57:43.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.212+0000 7fbb3f400640 1 --2- 172.21.15.97:0/130629142 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55801a5475d0 0x55801a5499f0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2c793c0ae4eeb4e server_cookie=b0cf34c8a3ea9e02 in_seq=0 out_seq=0 2024-03-27T10:57:43.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.212+0000 7fbb29600640 1 -- 172.21.15.97:0/130629142 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55801a3b4560 con 0x55801a5475d0 2024-03-27T10:57:43.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.212+0000 7fbb29600640 10 client.15849 handle_command_reply: tid=0 2024-03-27T10:57:43.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.212+0000 7fbb49033740 1 client.15849 shutdown 2024-03-27T10:57:43.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.212+0000 7fbb49033740 1 -- 172.21.15.97:0/130629142 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55801a4cb4a0 msgr2=0x55801a4d98d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:43.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.212+0000 7fbb49033740 1 --2- 172.21.15.97:0/130629142 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55801a4cb4a0 0x55801a4d98d0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x55801a4d6030 tx=0x7fbb2c00edb0 comp rx=0 tx=0).stop 2024-03-27T10:57:43.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.212+0000 7fbb49033740 1 -- 172.21.15.97:0/130629142 shutdown_connections 2024-03-27T10:57:43.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.212+0000 7fbb49033740 1 --2- 172.21.15.97:0/130629142 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55801a5475d0 0x55801a5499f0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:43.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.213+0000 7fbb49033740 1 --2- 172.21.15.97:0/130629142 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55801a4d0610 0x55801a4da7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:43.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.213+0000 7fbb49033740 1 --2- 172.21.15.97:0/130629142 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55801a4cb4a0 0x55801a4d98d0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:43.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.213+0000 7fbb49033740 1 --2- 172.21.15.97:0/130629142 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55801a4caab0 0x55801a4d9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:43.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.213+0000 7fbb49033740 1 -- 172.21.15.97:0/130629142 >> 172.21.15.97:0/130629142 conn(0x55801a4c6020 msgr2=0x55801a4c9280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:43.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.213+0000 7fbb29600640 0 client.15849 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T10:57:43.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.213+0000 7fbb49033740 1 -- 172.21.15.97:0/130629142 shutdown_connections 2024-03-27T10:57:43.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.213+0000 7fbb49033740 1 -- 172.21.15.97:0/130629142 wait complete. 2024-03-27T10:57:43.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.213+0000 7fbb49033740 20 client.15849 trim_cache size 0 max 16384 2024-03-27T10:57:43.186 INFO:teuthology.orchestra.run.smithi097.stdout:[] 2024-03-27T10:57:43.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.214+0000 7fbb44e00640 1 -- 172.21.15.97:0/1983961920 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fbb240a3690 msgr2=0x7fbb240a5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:43.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.214+0000 7fbb44e00640 1 --2- 172.21.15.97:0/1983961920 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fbb240a3690 0x7fbb240a5b50 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fbb38032380 tx=0x7fbb3802e040 comp rx=0 tx=0).stop 2024-03-27T10:57:43.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.214+0000 7fbb44e00640 1 -- 172.21.15.97:0/1983961920 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb401002a0 msgr2=0x7fbb4019e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:43.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.214+0000 7fbb44e00640 1 --2- 172.21.15.97:0/1983961920 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb401002a0 0x7fbb4019e190 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fbb3000d770 tx=0x7fbb3000dc30 comp rx=0 tx=0).stop 2024-03-27T10:57:43.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.214+0000 7fbb44e00640 1 -- 172.21.15.97:0/1983961920 shutdown_connections 2024-03-27T10:57:43.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.214+0000 7fbb44e00640 1 --2- 172.21.15.97:0/1983961920 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fbb240a3690 0x7fbb240a5b50 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:43.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.214+0000 7fbb44e00640 1 --2- 172.21.15.97:0/1983961920 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbb401055c0 0x7fbb401a5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:43.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.214+0000 7fbb44e00640 1 --2- 172.21.15.97:0/1983961920 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbb40102c10 0x7fbb4019e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:43.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.214+0000 7fbb44e00640 1 --2- 172.21.15.97:0/1983961920 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb401002a0 0x7fbb4019e190 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:43.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.214+0000 7fbb44e00640 1 -- 172.21.15.97:0/1983961920 >> 172.21.15.97:0/1983961920 conn(0x7fbb400fbec0 msgr2=0x7fbb40100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:43.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.214+0000 7fbb44e00640 1 -- 172.21.15.97:0/1983961920 shutdown_connections 2024-03-27T10:57:43.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:43.214+0000 7fbb44e00640 1 -- 172.21.15.97:0/1983961920 wait complete. 2024-03-27T10:57:43.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:57:43 smithi097 ceph-mon[31859]: pgmap v473: 97 pgs: 97 active+clean; 664 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-27T10:57:43.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:57:43 smithi177 ceph-mon[35294]: pgmap v473: 97 pgs: 97 active+clean; 664 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-27T10:57:43.766 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:57:43 smithi114 ceph-mon[35338]: pgmap v473: 97 pgs: 97 active+clean; 664 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-27T10:57:44.208 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-27T10:57:44.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-27T10:57:44.209 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-27T10:57:44.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.512+0000 7fde43200640 1 -- 172.21.15.97:0/3826969970 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde44101600 msgr2=0x7fde44101a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:44.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.512+0000 7fde43200640 1 --2- 172.21.15.97:0/3826969970 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde44101600 0x7fde44101a80 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fde3c009ca0 tx=0x7fde3c02c0f0 comp rx=0 tx=0).stop 2024-03-27T10:57:44.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.513+0000 7fde43200640 1 -- 172.21.15.97:0/3826969970 shutdown_connections 2024-03-27T10:57:44.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.513+0000 7fde43200640 1 --2- 172.21.15.97:0/3826969970 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde44101600 0x7fde44101a80 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:44.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.513+0000 7fde43200640 1 --2- 172.21.15.97:0/3826969970 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fde44100c60 0x7fde441010c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:44.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.513+0000 7fde43200640 1 --2- 172.21.15.97:0/3826969970 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fde44100300 0x7fde44100720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:44.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.513+0000 7fde43200640 1 -- 172.21.15.97:0/3826969970 >> 172.21.15.97:0/3826969970 conn(0x7fde440fbec0 msgr2=0x7fde440fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:44.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.513+0000 7fde43200640 1 -- 172.21.15.97:0/3826969970 shutdown_connections 2024-03-27T10:57:44.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.513+0000 7fde43200640 1 -- 172.21.15.97:0/3826969970 wait complete. 2024-03-27T10:57:44.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.514+0000 7fde43200640 1 Processor -- start 2024-03-27T10:57:44.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.514+0000 7fde43200640 1 -- start start 2024-03-27T10:57:44.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.515+0000 7fde43200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fde44100300 0x7fde44199e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:44.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.515+0000 7fde43200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde44100c60 0x7fde4419a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:44.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.515+0000 7fde43200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fde44101600 0x7fde441a13d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:44.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.515+0000 7fde43200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fde4410f750 con 0x7fde44100c60 2024-03-27T10:57:44.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.515+0000 7fde43200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fde4410f5d0 con 0x7fde44101600 2024-03-27T10:57:44.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.515+0000 7fde43200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fde4410f8d0 con 0x7fde44100300 2024-03-27T10:57:44.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.516+0000 7fde41e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fde44100300 0x7fde44199e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:44.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.516+0000 7fde41400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde44100c60 0x7fde4419a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:44.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.516+0000 7fde41400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde44100c60 0x7fde4419a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53684/0 (socket says 172.21.15.97:53684) 2024-03-27T10:57:44.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.516+0000 7fde41400640 1 -- 172.21.15.97:0/3684636357 learned_addr learned my addr 172.21.15.97:0/3684636357 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:44.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.516+0000 7fde42800640 1 --2- 172.21.15.97:0/3684636357 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fde44101600 0x7fde441a13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:44.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.516+0000 7fde41400640 1 -- 172.21.15.97:0/3684636357 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fde44100300 msgr2=0x7fde44199e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:44.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.516+0000 7fde41400640 1 --2- 172.21.15.97:0/3684636357 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fde44100300 0x7fde44199e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:44.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.516+0000 7fde41400640 1 -- 172.21.15.97:0/3684636357 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fde44101600 msgr2=0x7fde441a13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:44.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.516+0000 7fde41400640 1 --2- 172.21.15.97:0/3684636357 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fde44101600 0x7fde441a13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:44.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.516+0000 7fde41400640 1 -- 172.21.15.97:0/3684636357 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde441a1a40 con 0x7fde44100c60 2024-03-27T10:57:44.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.517+0000 7fde41400640 1 --2- 172.21.15.97:0/3684636357 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde44100c60 0x7fde4419a360 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fde3000d770 tx=0x7fde3000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:44.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.517+0000 7fde3aa00640 1 -- 172.21.15.97:0/3684636357 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde30017070 con 0x7fde44100c60 2024-03-27T10:57:44.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.517+0000 7fde43200640 1 -- 172.21.15.97:0/3684636357 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde441a1d30 con 0x7fde44100c60 2024-03-27T10:57:44.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.517+0000 7fde3aa00640 1 -- 172.21.15.97:0/3684636357 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fde300054d0 con 0x7fde44100c60 2024-03-27T10:57:44.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.518+0000 7fde3aa00640 1 -- 172.21.15.97:0/3684636357 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde30004cf0 con 0x7fde44100c60 2024-03-27T10:57:44.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.518+0000 7fde43200640 1 -- 172.21.15.97:0/3684636357 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde441a2220 con 0x7fde44100c60 2024-03-27T10:57:44.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.523+0000 7fde4b9dc740 1 Processor -- start 2024-03-27T10:57:44.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.523+0000 7fde4b9dc740 1 -- start start 2024-03-27T10:57:44.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.523+0000 7fde3aa00640 1 -- 172.21.15.97:0/3684636357 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7fde30004e90 con 0x7fde44100c60 2024-03-27T10:57:44.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.524+0000 7fde4b9dc740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fceaa810c0 0x55fceaa814e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:44.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.524+0000 7fde4b9dc740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fceaa82290 0x55fceaa82710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:44.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.525+0000 7fde4b9dc740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fceaa82c50 0x55fceaa894e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:44.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.525+0000 7fde4b9dc740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55fcea9e6280 con 0x55fceaa82c50 2024-03-27T10:57:44.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.525+0000 7fde4b9dc740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55fcea98e6f0 con 0x55fceaa810c0 2024-03-27T10:57:44.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.525+0000 7fde4b9dc740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55fcea9e2230 con 0x55fceaa82290 2024-03-27T10:57:44.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.525+0000 7fde42800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fceaa810c0 0x55fceaa814e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:44.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.525+0000 7fde41400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fceaa82c50 0x55fceaa894e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:44.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.525+0000 7fde41400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fceaa82c50 0x55fceaa894e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53694/0 (socket says 172.21.15.97:53694) 2024-03-27T10:57:44.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.525+0000 7fde41400640 1 -- 172.21.15.97:0/4049877889 learned_addr learned my addr 172.21.15.97:0/4049877889 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:44.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.525+0000 7fde41e00640 1 --2- 172.21.15.97:0/4049877889 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fceaa82290 0x55fceaa82710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:44.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.526+0000 7fde3aa00640 1 --2- 172.21.15.97:0/3684636357 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fde280a3690 0x7fde280a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:44.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.526+0000 7fde3aa00640 1 -- 172.21.15.97:0/3684636357 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fde300cf640 con 0x7fde44100c60 2024-03-27T10:57:44.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.526+0000 7fde41400640 1 -- 172.21.15.97:0/4049877889 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fceaa82290 msgr2=0x55fceaa82710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:44.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.526+0000 7fde41400640 1 --2- 172.21.15.97:0/4049877889 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fceaa82290 0x55fceaa82710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:44.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.526+0000 7fde41400640 1 -- 172.21.15.97:0/4049877889 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fceaa810c0 msgr2=0x55fceaa814e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:44.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.526+0000 7fde41400640 1 --2- 172.21.15.97:0/4049877889 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fceaa810c0 0x55fceaa814e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:44.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.526+0000 7fde41400640 1 -- 172.21.15.97:0/4049877889 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fcea7c27e0 con 0x55fceaa82c50 2024-03-27T10:57:44.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.526+0000 7fde42800640 1 --2- 172.21.15.97:0/3684636357 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fde280a3690 0x7fde280a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:44.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.526+0000 7fde41400640 1 --2- 172.21.15.97:0/4049877889 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fceaa82c50 0x55fceaa894e0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fde3009b8d0 tx=0x7fde30005680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e622b0644668d43 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:44.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.527+0000 7fde2f000640 1 -- 172.21.15.97:0/4049877889 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde30020070 con 0x55fceaa82c50 2024-03-27T10:57:44.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.527+0000 7fde2f000640 1 -- 172.21.15.97:0/4049877889 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fde3009ce50 con 0x55fceaa82c50 2024-03-27T10:57:44.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.527+0000 7fde2f000640 1 -- 172.21.15.97:0/4049877889 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde300a4790 con 0x55fceaa82c50 2024-03-27T10:57:44.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.528+0000 7fde4b9dc740 1 -- 172.21.15.97:0/4049877889 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fceaa82c50 msgr2=0x55fceaa894e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:44.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.528+0000 7fde4b9dc740 1 --2- 172.21.15.97:0/4049877889 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fceaa82c50 0x55fceaa894e0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fde3009b8d0 tx=0x7fde30005680 comp rx=0 tx=0).stop 2024-03-27T10:57:44.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.528+0000 7fde42800640 1 --2- 172.21.15.97:0/3684636357 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fde280a3690 0x7fde280a5b50 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fde3c02c630 tx=0x7fde3c02e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:44.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.528+0000 7fde4b9dc740 1 -- 172.21.15.97:0/4049877889 shutdown_connections 2024-03-27T10:57:44.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.528+0000 7fde4b9dc740 1 --2- 172.21.15.97:0/4049877889 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fceaa82c50 0x55fceaa894e0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:44.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.528+0000 7fde4b9dc740 1 --2- 172.21.15.97:0/4049877889 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fceaa82290 0x55fceaa82710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:44.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.528+0000 7fde4b9dc740 1 --2- 172.21.15.97:0/4049877889 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fceaa810c0 0x55fceaa814e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:44.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.528+0000 7fde4b9dc740 1 -- 172.21.15.97:0/4049877889 >> 172.21.15.97:0/4049877889 conn(0x55fceaa7c4b0 msgr2=0x55fceaa7c8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:44.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.529+0000 7fde4b9dc740 1 -- 172.21.15.97:0/4049877889 shutdown_connections 2024-03-27T10:57:44.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.529+0000 7fde4b9dc740 1 -- 172.21.15.97:0/4049877889 wait complete. 2024-03-27T10:57:44.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.529+0000 7fde4b9dc740 1 -- start start 2024-03-27T10:57:44.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.529+0000 7fde4b9dc740 1 Processor -- start 2024-03-27T10:57:44.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.530+0000 7fde4b9dc740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fceaa810c0 0x55fceaa93c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:44.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.530+0000 7fde4b9dc740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fceaa82290 0x55fceaa941b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:44.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.531+0000 7fde4b9dc740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fceaa82c50 0x55fceaa9b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:44.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.531+0000 7fde4b9dc740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55fceaa89c00 con 0x55fceaa82290 2024-03-27T10:57:44.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.531+0000 7fde4b9dc740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55fcea9a1aa0 con 0x55fceaa82c50 2024-03-27T10:57:44.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.531+0000 7fde4b9dc740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55fceaa89d80 con 0x55fceaa810c0 2024-03-27T10:57:44.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.531+0000 7fde4b9dc740 10 client.0 _ll_register_callbacks cb 0x55fcea9a1490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:57:44.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.531+0000 7fde41e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fceaa810c0 0x55fceaa93c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:44.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.531+0000 7fde41e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fceaa810c0 0x55fceaa93c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.97:35284/0 (socket says 172.21.15.97:35284) 2024-03-27T10:57:44.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.531+0000 7fde41e00640 1 -- 172.21.15.97:0/3987291879 learned_addr learned my addr 172.21.15.97:0/3987291879 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:44.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.531+0000 7fde41400640 1 --2- 172.21.15.97:0/3987291879 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fceaa82290 0x55fceaa941b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:44.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.531+0000 7fde42800640 1 --2- 172.21.15.97:0/3987291879 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fceaa82c50 0x55fceaa9b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:44.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.531+0000 7fde41e00640 1 -- 172.21.15.97:0/3987291879 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fceaa82c50 msgr2=0x55fceaa9b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:44.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.531+0000 7fde41e00640 1 --2- 172.21.15.97:0/3987291879 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fceaa82c50 0x55fceaa9b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:44.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.531+0000 7fde41e00640 1 -- 172.21.15.97:0/3987291879 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fceaa82290 msgr2=0x55fceaa941b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:44.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.531+0000 7fde41e00640 1 --2- 172.21.15.97:0/3987291879 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fceaa82290 0x55fceaa941b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:44.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.531+0000 7fde41e00640 1 -- 172.21.15.97:0/3987291879 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fcea822f70 con 0x55fceaa810c0 2024-03-27T10:57:44.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.532+0000 7fde41e00640 1 --2- 172.21.15.97:0/3987291879 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fceaa810c0 0x55fceaa93c70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fde3400bad0 tx=0x7fde3400bf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ecd8f5cb7c8798b server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:44.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.532+0000 7fde23400640 10 client.0 ms_handle_connect on v2:172.21.15.177:3300/0 2024-03-27T10:57:44.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.532+0000 7fde23400640 1 -- 172.21.15.97:0/3987291879 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde34003dc0 con 0x55fceaa810c0 2024-03-27T10:57:44.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.532+0000 7fde23400640 1 -- 172.21.15.97:0/3987291879 <== mon.1 v2:172.21.15.177:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fde34011440 con 0x55fceaa810c0 2024-03-27T10:57:44.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.532+0000 7fde4b9dc740 1 -- 172.21.15.97:0/3987291879 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde3009ce50 con 0x55fceaa810c0 2024-03-27T10:57:44.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.533+0000 7fde23400640 1 -- 172.21.15.97:0/3987291879 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fde34011640 con 0x55fceaa810c0 2024-03-27T10:57:44.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.533+0000 7fde4b9dc740 10 client.25324 fetch_fsmap learned FSMap version 28 2024-03-27T10:57:44.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.533+0000 7fde4b9dc740 1 -- 172.21.15.97:0/3987291879 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fcea6ba170 con 0x55fceaa810c0 2024-03-27T10:57:44.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.533+0000 7fde23400640 1 -- 172.21.15.97:0/3987291879 <== mon.1 v2:172.21.15.177:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fde3009ce50 con 0x55fceaa810c0 2024-03-27T10:57:44.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.534+0000 7fde23400640 1 -- 172.21.15.97:0/3987291879 <== mon.1 v2:172.21.15.177:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fde3401b870 con 0x55fceaa810c0 2024-03-27T10:57:44.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.534+0000 7fde4b9dc740 10 client.25324 fetch_fsmap finished waiting for FSMap version 28 2024-03-27T10:57:44.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.534+0000 7fde4b9dc740 10 client.25324 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-27T10:57:44.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.534+0000 7fde4b9dc740 20 client.25324 populate_metadata read hostname 'smithi097' 2024-03-27T10:57:44.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.534+0000 7fde4b9dc740 1 --2- 172.21.15.97:0/3987291879 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55fceaa9fad0 0x55fceaaa1f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:44.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.534+0000 7fde4b9dc740 4 client.25324 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T10:57:44.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.534+0000 7fde4b9dc740 1 -- 172.21.15.97:0/3987291879 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fcea9f37c0 con 0x55fceaa9fad0 2024-03-27T10:57:44.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.535+0000 7fde41400640 1 --2- 172.21.15.97:0/3987291879 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55fceaa9fad0 0x55fceaaa1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:44.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.535+0000 7fde41400640 1 --2- 172.21.15.97:0/3987291879 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55fceaa9fad0 0x55fceaaa1f70 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3c31e4fceed7f4d server_cookie=cae5c5ec3ab65ad4 in_seq=0 out_seq=0 2024-03-27T10:57:44.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.535+0000 7fde23400640 10 client.25324 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T10:57:44.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.537+0000 7fde23400640 1 -- 172.21.15.97:0/3987291879 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55fcea9f37c0 con 0x55fceaa9fad0 2024-03-27T10:57:44.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.537+0000 7fde23400640 10 client.25324 handle_command_reply: tid=0 2024-03-27T10:57:44.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.538+0000 7fde4b9dc740 1 client.25324 shutdown 2024-03-27T10:57:44.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.538+0000 7fde4b9dc740 1 -- 172.21.15.97:0/3987291879 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fceaa810c0 msgr2=0x55fceaa93c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:44.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.538+0000 7fde4b9dc740 1 --2- 172.21.15.97:0/3987291879 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fceaa810c0 0x55fceaa93c70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fde3400bad0 tx=0x7fde3400bf90 comp rx=0 tx=0).stop 2024-03-27T10:57:44.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.538+0000 7fde4b9dc740 1 -- 172.21.15.97:0/3987291879 shutdown_connections 2024-03-27T10:57:44.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.538+0000 7fde4b9dc740 1 --2- 172.21.15.97:0/3987291879 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55fceaa9fad0 0x55fceaaa1f70 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:44.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.538+0000 7fde4b9dc740 1 --2- 172.21.15.97:0/3987291879 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fceaa82c50 0x55fceaa9b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:44.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.538+0000 7fde4b9dc740 1 --2- 172.21.15.97:0/3987291879 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fceaa82290 0x55fceaa941b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:44.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.538+0000 7fde4b9dc740 1 --2- 172.21.15.97:0/3987291879 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fceaa810c0 0x55fceaa93c70 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:44.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.538+0000 7fde23400640 0 client.25324 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T10:57:44.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.538+0000 7fde4b9dc740 1 -- 172.21.15.97:0/3987291879 >> 172.21.15.97:0/3987291879 conn(0x55fceaa7c4b0 msgr2=0x55fceaa7b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:44.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.538+0000 7fde4b9dc740 1 -- 172.21.15.97:0/3987291879 shutdown_connections 2024-03-27T10:57:44.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.538+0000 7fde4b9dc740 1 -- 172.21.15.97:0/3987291879 wait complete. 2024-03-27T10:57:44.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.538+0000 7fde4b9dc740 20 client.25324 trim_cache size 0 max 16384 2024-03-27T10:57:44.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.574+0000 7fde4b9dc740 1 Processor -- start 2024-03-27T10:57:44.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.574+0000 7fde4b9dc740 1 -- start start 2024-03-27T10:57:44.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.574+0000 7fde4b9dc740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fceaa810c0 0x55fceaa7b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:44.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.575+0000 7fde4b9dc740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fceaa82290 0x55fceaa9b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:44.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.575+0000 7fde4b9dc740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fceaa82c50 0x55fceaaa7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:44.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.575+0000 7fde4b9dc740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fde441a6b60 con 0x55fceaa810c0 2024-03-27T10:57:44.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.575+0000 7fde4b9dc740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fde441a86a0 con 0x55fceaa82c50 2024-03-27T10:57:44.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.575+0000 7fde4b9dc740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fde441a7230 con 0x55fceaa82290 2024-03-27T10:57:44.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.575+0000 7fde41e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fceaa810c0 0x55fceaa7b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:44.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.575+0000 7fde41e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fceaa810c0 0x55fceaa7b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53720/0 (socket says 172.21.15.97:53720) 2024-03-27T10:57:44.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.575+0000 7fde41e00640 1 -- 172.21.15.97:0/4069509762 learned_addr learned my addr 172.21.15.97:0/4069509762 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:44.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.575+0000 7fde42800640 1 --2- 172.21.15.97:0/4069509762 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fceaa82c50 0x55fceaaa7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:44.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.575+0000 7fde41400640 1 --2- 172.21.15.97:0/4069509762 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fceaa82290 0x55fceaa9b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:44.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.575+0000 7fde41e00640 1 -- 172.21.15.97:0/4069509762 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fceaa82290 msgr2=0x55fceaa9b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:44.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.575+0000 7fde41e00640 1 --2- 172.21.15.97:0/4069509762 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fceaa82290 0x55fceaa9b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:44.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.575+0000 7fde41e00640 1 -- 172.21.15.97:0/4069509762 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fceaa82c50 msgr2=0x55fceaaa7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:44.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.575+0000 7fde41e00640 1 --2- 172.21.15.97:0/4069509762 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fceaa82c50 0x55fceaaa7880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:44.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.575+0000 7fde41e00640 1 -- 172.21.15.97:0/4069509762 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fceaa81d10 con 0x55fceaa810c0 2024-03-27T10:57:44.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.576+0000 7fde41e00640 1 --2- 172.21.15.97:0/4069509762 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fceaa810c0 0x55fceaa7b620 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fde340042e0 tx=0x7fde3400b400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d1e19069183999c server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:44.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.576+0000 7fde2e600640 1 -- 172.21.15.97:0/4069509762 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde34010440 con 0x55fceaa810c0 2024-03-27T10:57:44.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.576+0000 7fde2e600640 1 -- 172.21.15.97:0/4069509762 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fde34021030 con 0x55fceaa810c0 2024-03-27T10:57:44.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.576+0000 7fde2e600640 1 -- 172.21.15.97:0/4069509762 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde34011c40 con 0x55fceaa810c0 2024-03-27T10:57:44.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.577+0000 7fde4b9dc740 1 -- 172.21.15.97:0/4069509762 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fceaa810c0 msgr2=0x55fceaa7b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:44.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.577+0000 7fde4b9dc740 1 --2- 172.21.15.97:0/4069509762 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fceaa810c0 0x55fceaa7b620 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fde340042e0 tx=0x7fde3400b400 comp rx=0 tx=0).stop 2024-03-27T10:57:44.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.577+0000 7fde4b9dc740 1 -- 172.21.15.97:0/4069509762 shutdown_connections 2024-03-27T10:57:44.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.577+0000 7fde4b9dc740 1 --2- 172.21.15.97:0/4069509762 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fceaa82c50 0x55fceaaa7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:44.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.577+0000 7fde4b9dc740 1 --2- 172.21.15.97:0/4069509762 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fceaa82290 0x55fceaa9b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:44.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.577+0000 7fde4b9dc740 1 --2- 172.21.15.97:0/4069509762 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fceaa810c0 0x55fceaa7b620 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:44.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.577+0000 7fde4b9dc740 1 -- 172.21.15.97:0/4069509762 >> 172.21.15.97:0/4069509762 conn(0x55fceaa7c4b0 msgr2=0x55fceaa7af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:44.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.578+0000 7fde4b9dc740 1 -- 172.21.15.97:0/4069509762 shutdown_connections 2024-03-27T10:57:44.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.578+0000 7fde4b9dc740 1 -- 172.21.15.97:0/4069509762 wait complete. 2024-03-27T10:57:44.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.578+0000 7fde4b9dc740 1 -- start start 2024-03-27T10:57:44.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.578+0000 7fde4b9dc740 1 Processor -- start 2024-03-27T10:57:44.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.579+0000 7fde4b9dc740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fceaa810c0 0x55fceaa998d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:44.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.579+0000 7fde4b9dc740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fceaa82290 0x55fceaa99e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:44.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.579+0000 7fde4b9dc740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fceaa82c50 0x55fceaa9a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:44.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.579+0000 7fde4b9dc740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fde441a7fd0 con 0x55fceaa82c50 2024-03-27T10:57:44.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.579+0000 7fde4b9dc740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fde441a7900 con 0x55fceaa82290 2024-03-27T10:57:44.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.579+0000 7fde4b9dc740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fde441a4b40 con 0x55fceaa810c0 2024-03-27T10:57:44.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.579+0000 7fde4b9dc740 10 client.0 _ll_register_callbacks cb 0x7fde34004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:57:44.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.579+0000 7fde42800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fceaa82c50 0x55fceaa9a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:44.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.580+0000 7fde42800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fceaa82c50 0x55fceaa9a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53736/0 (socket says 172.21.15.97:53736) 2024-03-27T10:57:44.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.580+0000 7fde42800640 1 -- 172.21.15.97:0/1294085966 learned_addr learned my addr 172.21.15.97:0/1294085966 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:44.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.580+0000 7fde41400640 1 --2- 172.21.15.97:0/1294085966 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fceaa82290 0x55fceaa99e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:44.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.580+0000 7fde41e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fceaa810c0 0x55fceaa998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:44.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.580+0000 7fde42800640 1 -- 172.21.15.97:0/1294085966 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fceaa810c0 msgr2=0x55fceaa998d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:44.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.580+0000 7fde42800640 1 --2- 172.21.15.97:0/1294085966 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fceaa810c0 0x55fceaa998d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:44.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.580+0000 7fde42800640 1 -- 172.21.15.97:0/1294085966 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fceaa82290 msgr2=0x55fceaa99e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:44.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.580+0000 7fde42800640 1 --2- 172.21.15.97:0/1294085966 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fceaa82290 0x55fceaa99e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:44.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.580+0000 7fde42800640 1 -- 172.21.15.97:0/1294085966 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fceab05eb0 con 0x55fceaa82c50 2024-03-27T10:57:44.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.580+0000 7fde42800640 1 --2- 172.21.15.97:0/1294085966 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fceaa82c50 0x55fceaa9a350 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fde3c009ca0 tx=0x7fde3c0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f4aefb864551695 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:44.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.581+0000 7fde23400640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T10:57:44.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.581+0000 7fde23400640 1 -- 172.21.15.97:0/1294085966 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde3c052070 con 0x55fceaa82c50 2024-03-27T10:57:44.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.581+0000 7fde23400640 1 -- 172.21.15.97:0/1294085966 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fde3c004450 con 0x55fceaa82c50 2024-03-27T10:57:44.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.581+0000 7fde4b9dc740 1 -- 172.21.15.97:0/1294085966 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde34021030 con 0x55fceaa82c50 2024-03-27T10:57:44.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.581+0000 7fde23400640 1 -- 172.21.15.97:0/1294085966 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fde3c05c050 con 0x55fceaa82c50 2024-03-27T10:57:44.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.581+0000 7fde4b9dc740 10 client.15867 fetch_fsmap learned FSMap version 28 2024-03-27T10:57:44.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.581+0000 7fde4b9dc740 1 -- 172.21.15.97:0/1294085966 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fceab07070 con 0x55fceaa82c50 2024-03-27T10:57:44.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.582+0000 7fde23400640 1 -- 172.21.15.97:0/1294085966 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fde34021030 con 0x55fceaa82c50 2024-03-27T10:57:44.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.582+0000 7fde23400640 1 -- 172.21.15.97:0/1294085966 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fde3c034440 con 0x55fceaa82c50 2024-03-27T10:57:44.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.583+0000 7fde4b9dc740 10 client.15867 fetch_fsmap finished waiting for FSMap version 28 2024-03-27T10:57:44.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.583+0000 7fde4b9dc740 10 client.15867 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-27T10:57:44.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.583+0000 7fde4b9dc740 20 client.15867 populate_metadata read hostname 'smithi097' 2024-03-27T10:57:44.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.583+0000 7fde4b9dc740 1 --2- 172.21.15.97:0/1294085966 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55fceab07a50 0x55fceab09e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:44.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.583+0000 7fde4b9dc740 4 client.15867 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-27T10:57:44.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.583+0000 7fde4b9dc740 1 -- 172.21.15.97:0/1294085966 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55fcea9edb10 con 0x55fceab07a50 2024-03-27T10:57:44.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.583+0000 7fde41e00640 1 --2- 172.21.15.97:0/1294085966 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55fceab07a50 0x55fceab09e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:44.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.584+0000 7fde41e00640 1 --2- 172.21.15.97:0/1294085966 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55fceab07a50 0x55fceab09e70 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c73ef258ce5bd5d server_cookie=b2eb46011cd78401 in_seq=0 out_seq=0 2024-03-27T10:57:44.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.584+0000 7fde23400640 10 client.15867 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T10:57:44.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.587+0000 7fde23400640 1 -- 172.21.15.97:0/1294085966 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55fcea9edb10 con 0x55fceab07a50 2024-03-27T10:57:44.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.587+0000 7fde23400640 10 client.15867 handle_command_reply: tid=0 2024-03-27T10:57:44.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.587+0000 7fde4b9dc740 1 client.15867 shutdown 2024-03-27T10:57:44.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.587+0000 7fde4b9dc740 1 -- 172.21.15.97:0/1294085966 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fceaa82c50 msgr2=0x55fceaa9a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:44.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.587+0000 7fde4b9dc740 1 --2- 172.21.15.97:0/1294085966 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fceaa82c50 0x55fceaa9a350 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fde3c009ca0 tx=0x7fde3c0095c0 comp rx=0 tx=0).stop 2024-03-27T10:57:44.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.587+0000 7fde4b9dc740 1 -- 172.21.15.97:0/1294085966 shutdown_connections 2024-03-27T10:57:44.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.587+0000 7fde4b9dc740 1 --2- 172.21.15.97:0/1294085966 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55fceab07a50 0x55fceab09e70 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:44.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.587+0000 7fde4b9dc740 1 --2- 172.21.15.97:0/1294085966 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fceaa82c50 0x55fceaa9a350 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:44.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.587+0000 7fde4b9dc740 1 --2- 172.21.15.97:0/1294085966 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fceaa82290 0x55fceaa99e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:44.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.587+0000 7fde4b9dc740 1 --2- 172.21.15.97:0/1294085966 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fceaa810c0 0x55fceaa998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:44.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.587+0000 7fde4b9dc740 1 -- 172.21.15.97:0/1294085966 >> 172.21.15.97:0/1294085966 conn(0x55fceaa7c4b0 msgr2=0x55fceaa7b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:44.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.587+0000 7fde23400640 0 client.15867 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T10:57:44.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.588+0000 7fde4b9dc740 1 -- 172.21.15.97:0/1294085966 shutdown_connections 2024-03-27T10:57:44.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.588+0000 7fde4b9dc740 1 -- 172.21.15.97:0/1294085966 wait complete. 2024-03-27T10:57:44.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.588+0000 7fde4b9dc740 20 client.15867 trim_cache size 0 max 16384 2024-03-27T10:57:44.560 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-03-27T10:57:44.560 INFO:teuthology.orchestra.run.smithi097.stdout: "return_code": 0, 2024-03-27T10:57:44.560 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_tag": "b627adc9-39c1-41c6-a089-d8c6608244c4", 2024-03-27T10:57:44.560 INFO:teuthology.orchestra.run.smithi097.stdout: "mode": "asynchronous" 2024-03-27T10:57:44.560 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-03-27T10:57:44.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.590+0000 7fde43200640 1 -- 172.21.15.97:0/3684636357 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fde280a3690 msgr2=0x7fde280a5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:44.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.590+0000 7fde43200640 1 --2- 172.21.15.97:0/3684636357 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fde280a3690 0x7fde280a5b50 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fde3c02c630 tx=0x7fde3c02e040 comp rx=0 tx=0).stop 2024-03-27T10:57:44.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.590+0000 7fde43200640 1 -- 172.21.15.97:0/3684636357 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde44100c60 msgr2=0x7fde4419a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:44.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.590+0000 7fde43200640 1 --2- 172.21.15.97:0/3684636357 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde44100c60 0x7fde4419a360 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fde3000d770 tx=0x7fde3000dc30 comp rx=0 tx=0).stop 2024-03-27T10:57:44.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.590+0000 7fde43200640 1 -- 172.21.15.97:0/3684636357 shutdown_connections 2024-03-27T10:57:44.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.590+0000 7fde43200640 1 --2- 172.21.15.97:0/3684636357 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fde280a3690 0x7fde280a5b50 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:44.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.590+0000 7fde43200640 1 --2- 172.21.15.97:0/3684636357 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fde44101600 0x7fde441a13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:44.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.590+0000 7fde43200640 1 --2- 172.21.15.97:0/3684636357 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde44100c60 0x7fde4419a360 secure :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fde3000d770 tx=0x7fde3000dc30 comp rx=0 tx=0).stop 2024-03-27T10:57:44.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.590+0000 7fde43200640 1 --2- 172.21.15.97:0/3684636357 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fde44100300 0x7fde44199e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:44.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.590+0000 7fde43200640 1 -- 172.21.15.97:0/3684636357 >> 172.21.15.97:0/3684636357 conn(0x7fde440fbec0 msgr2=0x7fde440fd650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:44.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.590+0000 7fde43200640 1 -- 172.21.15.97:0/3684636357 shutdown_connections 2024-03-27T10:57:44.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.591+0000 7fde43200640 1 -- 172.21.15.97:0/3684636357 wait complete. 2024-03-27T10:57:44.585 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-27T10:57:44.585 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-27T10:57:44.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.809+0000 7fd1a2a00640 1 -- 172.21.15.97:0/560229816 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd19c1024b0 msgr2=0x7fd19c1048a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:44.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.809+0000 7fd1a2a00640 1 --2- 172.21.15.97:0/560229816 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd19c1024b0 0x7fd19c1048a0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fd190009900 tx=0x7fd19002bc40 comp rx=0 tx=0).stop 2024-03-27T10:57:44.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.809+0000 7fd1a2a00640 1 -- 172.21.15.97:0/560229816 shutdown_connections 2024-03-27T10:57:44.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.809+0000 7fd1a2a00640 1 --2- 172.21.15.97:0/560229816 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd19c107710 0x7fd19c109b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:44.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.809+0000 7fd1a2a00640 1 --2- 172.21.15.97:0/560229816 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd19c104de0 0x7fd19c1071d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:44.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.809+0000 7fd1a2a00640 1 --2- 172.21.15.97:0/560229816 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd19c1024b0 0x7fd19c1048a0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:44.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.809+0000 7fd1a2a00640 1 -- 172.21.15.97:0/560229816 >> 172.21.15.97:0/560229816 conn(0x7fd19c0fbec0 msgr2=0x7fd19c0fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:44.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.809+0000 7fd1a2a00640 1 -- 172.21.15.97:0/560229816 shutdown_connections 2024-03-27T10:57:44.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.809+0000 7fd1a2a00640 1 -- 172.21.15.97:0/560229816 wait complete. 2024-03-27T10:57:44.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.810+0000 7fd1a2a00640 1 Processor -- start 2024-03-27T10:57:44.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.810+0000 7fd1a2a00640 1 -- start start 2024-03-27T10:57:44.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.810+0000 7fd1a2a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd19c1024b0 0x7fd19c101e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:44.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.810+0000 7fd1a2a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd19c104de0 0x7fd19c100460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:44.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.811+0000 7fd1a2a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd19c107710 0x7fd19c1009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:44.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.811+0000 7fd1a2a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd19c10d590 con 0x7fd19c107710 2024-03-27T10:57:44.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.811+0000 7fd1a2a00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd19c10d410 con 0x7fd19c104de0 2024-03-27T10:57:44.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.811+0000 7fd1a2a00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd19c10d710 con 0x7fd19c1024b0 2024-03-27T10:57:44.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.811+0000 7fd1a2000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd19c107710 0x7fd19c1009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:44.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.811+0000 7fd1a2000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd19c107710 0x7fd19c1009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53758/0 (socket says 172.21.15.97:53758) 2024-03-27T10:57:44.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.811+0000 7fd1a2000640 1 -- 172.21.15.97:0/1812299188 learned_addr learned my addr 172.21.15.97:0/1812299188 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:44.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.811+0000 7fd1a0c00640 1 --2- 172.21.15.97:0/1812299188 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd19c104de0 0x7fd19c100460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:44.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.811+0000 7fd1a1600640 1 --2- 172.21.15.97:0/1812299188 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd19c1024b0 0x7fd19c101e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:44.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.811+0000 7fd1a2000640 1 -- 172.21.15.97:0/1812299188 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd19c1024b0 msgr2=0x7fd19c101e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:44.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.811+0000 7fd1a2000640 1 --2- 172.21.15.97:0/1812299188 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd19c1024b0 0x7fd19c101e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:44.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.811+0000 7fd1a2000640 1 -- 172.21.15.97:0/1812299188 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd19c104de0 msgr2=0x7fd19c100460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:44.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.811+0000 7fd1a2000640 1 --2- 172.21.15.97:0/1812299188 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd19c104de0 0x7fd19c100460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:44.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.811+0000 7fd1a2000640 1 -- 172.21.15.97:0/1812299188 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd19c101230 con 0x7fd19c107710 2024-03-27T10:57:44.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.812+0000 7fd1a2000640 1 --2- 172.21.15.97:0/1812299188 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd19c107710 0x7fd19c1009a0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fd19400eb10 tx=0x7fd19400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:44.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.812+0000 7fd19a000640 1 -- 172.21.15.97:0/1812299188 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd194004b20 con 0x7fd19c107710 2024-03-27T10:57:44.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.812+0000 7fd19a000640 1 -- 172.21.15.97:0/1812299188 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd194004cc0 con 0x7fd19c107710 2024-03-27T10:57:44.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.812+0000 7fd19a000640 1 -- 172.21.15.97:0/1812299188 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd194010690 con 0x7fd19c107710 2024-03-27T10:57:44.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.812+0000 7fd1a2a00640 1 -- 172.21.15.97:0/1812299188 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd19c1a1dc0 con 0x7fd19c107710 2024-03-27T10:57:44.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.812+0000 7fd1a2a00640 1 -- 172.21.15.97:0/1812299188 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd19c1a2230 con 0x7fd19c107710 2024-03-27T10:57:44.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.816+0000 7fd1a6d88740 1 Processor -- start 2024-03-27T10:57:44.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.816+0000 7fd19a000640 1 -- 172.21.15.97:0/1812299188 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7fd194005020 con 0x7fd19c107710 2024-03-27T10:57:44.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.816+0000 7fd1a6d88740 1 -- start start 2024-03-27T10:57:44.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.816+0000 7fd1a6d88740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cba1fca670 0x55cba1fcaa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:44.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.816+0000 7fd1a6d88740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cba1fcb060 0x55cba1fcfb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:44.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.816+0000 7fd1a6d88740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cba1fd01d0 0x55cba1fd2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:44.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.816+0000 7fd1a6d88740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55cba1f306b0 con 0x55cba1fcb060 2024-03-27T10:57:44.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.816+0000 7fd1a6d88740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55cba1ed8750 con 0x55cba1fd01d0 2024-03-27T10:57:44.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.817+0000 7fd1a6d88740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55cba1f2c660 con 0x55cba1fca670 2024-03-27T10:57:44.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.817+0000 7fd1a1600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cba1fcb060 0x55cba1fcfb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:44.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.817+0000 7fd1a1600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cba1fcb060 0x55cba1fcfb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53764/0 (socket says 172.21.15.97:53764) 2024-03-27T10:57:44.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.817+0000 7fd1a1600640 1 -- 172.21.15.97:0/2835687169 learned_addr learned my addr 172.21.15.97:0/2835687169 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:44.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.817+0000 7fd1a0c00640 1 --2- 172.21.15.97:0/2835687169 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cba1fca670 0x55cba1fcaa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:44.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.817+0000 7fd1a0c00640 1 --2- 172.21.15.97:0/2835687169 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cba1fd01d0 0x55cba1fd2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:44.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.818+0000 7fd1a0c00640 1 -- 172.21.15.97:0/2835687169 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cba1fd01d0 msgr2=0x55cba1fd2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:44.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.818+0000 7fd1a0c00640 1 --2- 172.21.15.97:0/2835687169 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cba1fd01d0 0x55cba1fd2660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:44.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.818+0000 7fd1a0c00640 1 -- 172.21.15.97:0/2835687169 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cba1fcb060 msgr2=0x55cba1fcfb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:44.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.818+0000 7fd1a0c00640 1 --2- 172.21.15.97:0/2835687169 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cba1fcb060 0x55cba1fcfb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:44.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.818+0000 7fd19a000640 1 --2- 172.21.15.97:0/1812299188 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fd1840a3690 0x7fd1840a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:44.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.818+0000 7fd1a0c00640 1 -- 172.21.15.97:0/2835687169 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cba1d0e7d0 con 0x55cba1fca670 2024-03-27T10:57:44.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.818+0000 7fd19a000640 1 -- 172.21.15.97:0/1812299188 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fd1940d0350 con 0x7fd19c107710 2024-03-27T10:57:44.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.818+0000 7fd1a0c00640 1 --2- 172.21.15.97:0/1812299188 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fd1840a3690 0x7fd1840a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:44.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.818+0000 7fd1a0c00640 1 --2- 172.21.15.97:0/2835687169 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cba1fca670 0x55cba1fcaa90 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd188005d00 tx=0x7fd188005c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c944df2b6c1f2404 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:44.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.819+0000 7fd18e600640 1 -- 172.21.15.97:0/2835687169 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd188026070 con 0x55cba1fca670 2024-03-27T10:57:44.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.819+0000 7fd18e600640 1 -- 172.21.15.97:0/2835687169 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd18801f870 con 0x55cba1fca670 2024-03-27T10:57:44.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.819+0000 7fd18e600640 1 -- 172.21.15.97:0/2835687169 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd18801e5d0 con 0x55cba1fca670 2024-03-27T10:57:44.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.820+0000 7fd1a6d88740 1 -- 172.21.15.97:0/2835687169 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cba1fca670 msgr2=0x55cba1fcaa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:44.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.820+0000 7fd1a0c00640 1 --2- 172.21.15.97:0/1812299188 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fd1840a3690 0x7fd1840a5b50 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd188023000 tx=0x7fd188020000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:44.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.820+0000 7fd1a6d88740 1 --2- 172.21.15.97:0/2835687169 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cba1fca670 0x55cba1fcaa90 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd188005d00 tx=0x7fd188005c90 comp rx=0 tx=0).stop 2024-03-27T10:57:44.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.820+0000 7fd1a6d88740 1 -- 172.21.15.97:0/2835687169 shutdown_connections 2024-03-27T10:57:44.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.820+0000 7fd1a6d88740 1 --2- 172.21.15.97:0/2835687169 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cba1fd01d0 0x55cba1fd2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:44.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.820+0000 7fd1a6d88740 1 --2- 172.21.15.97:0/2835687169 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cba1fcb060 0x55cba1fcfb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:44.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.820+0000 7fd1a6d88740 1 --2- 172.21.15.97:0/2835687169 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cba1fca670 0x55cba1fcaa90 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:44.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.820+0000 7fd1a6d88740 1 -- 172.21.15.97:0/2835687169 >> 172.21.15.97:0/2835687169 conn(0x55cba1fc5be0 msgr2=0x55cba1fc5fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:44.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.820+0000 7fd1a6d88740 1 -- 172.21.15.97:0/2835687169 shutdown_connections 2024-03-27T10:57:44.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.820+0000 7fd1a6d88740 1 -- 172.21.15.97:0/2835687169 wait complete. 2024-03-27T10:57:44.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.821+0000 7fd1a6d88740 1 -- start start 2024-03-27T10:57:44.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.821+0000 7fd1a6d88740 1 Processor -- start 2024-03-27T10:57:44.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.821+0000 7fd1a6d88740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cba1fca670 0x55cba1fe5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:44.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.821+0000 7fd1a6d88740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cba1fcb060 0x55cba1fe60e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:44.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.821+0000 7fd1a6d88740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cba1fd01d0 0x55cba1fed120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:44.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.821+0000 7fd1a6d88740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55cba1fe4190 con 0x55cba1fd01d0 2024-03-27T10:57:44.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.822+0000 7fd1a6d88740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55cba1eeba80 con 0x55cba1fca670 2024-03-27T10:57:44.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.822+0000 7fd1a6d88740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55cba1fe4310 con 0x55cba1fcb060 2024-03-27T10:57:44.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.822+0000 7fd1a6d88740 10 client.0 _ll_register_callbacks cb 0x55cba1eeb470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:57:44.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.822+0000 7fd1a2000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cba1fd01d0 0x55cba1fed120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:44.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.822+0000 7fd1a2000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cba1fd01d0 0x55cba1fed120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53774/0 (socket says 172.21.15.97:53774) 2024-03-27T10:57:44.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.822+0000 7fd1a2000640 1 -- 172.21.15.97:0/3050529333 learned_addr learned my addr 172.21.15.97:0/3050529333 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:44.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.822+0000 7fd1a1600640 1 --2- 172.21.15.97:0/3050529333 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cba1fca670 0x55cba1fe5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:44.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.822+0000 7fd1a0c00640 1 --2- 172.21.15.97:0/3050529333 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cba1fcb060 0x55cba1fe60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:44.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.822+0000 7fd1a2000640 1 -- 172.21.15.97:0/3050529333 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cba1fcb060 msgr2=0x55cba1fe60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:44.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.822+0000 7fd1a2000640 1 --2- 172.21.15.97:0/3050529333 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cba1fcb060 0x55cba1fe60e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:44.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.822+0000 7fd1a2000640 1 -- 172.21.15.97:0/3050529333 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cba1fca670 msgr2=0x55cba1fe5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:44.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.822+0000 7fd1a2000640 1 --2- 172.21.15.97:0/3050529333 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cba1fca670 0x55cba1fe5ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:44.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.822+0000 7fd1a2000640 1 -- 172.21.15.97:0/3050529333 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cba1c443f0 con 0x55cba1fd01d0 2024-03-27T10:57:44.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.822+0000 7fd1a1600640 1 --2- 172.21.15.97:0/3050529333 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cba1fca670 0x55cba1fe5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T10:57:44.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.823+0000 7fd1a2000640 1 --2- 172.21.15.97:0/3050529333 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cba1fd01d0 0x55cba1fed120 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fd1940a4f10 tx=0x7fd19409c310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aaf8b788b1f648b server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:44.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.823+0000 7fd182a00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T10:57:44.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.823+0000 7fd182a00640 1 -- 172.21.15.97:0/3050529333 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd19409c7d0 con 0x55cba1fd01d0 2024-03-27T10:57:44.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.823+0000 7fd1a6d88740 1 -- 172.21.15.97:0/3050529333 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd18801f870 con 0x55cba1fd01d0 2024-03-27T10:57:44.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.824+0000 7fd182a00640 1 -- 172.21.15.97:0/3050529333 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fd1940a4fb0 con 0x55cba1fd01d0 2024-03-27T10:57:44.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.824+0000 7fd182a00640 1 -- 172.21.15.97:0/3050529333 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd1940a51d0 con 0x55cba1fd01d0 2024-03-27T10:57:44.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.824+0000 7fd1a6d88740 10 client.15879 fetch_fsmap learned FSMap version 28 2024-03-27T10:57:44.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.824+0000 7fd1a6d88740 1 -- 172.21.15.97:0/3050529333 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55cba1c040f0 con 0x55cba1fd01d0 2024-03-27T10:57:44.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.824+0000 7fd182a00640 1 -- 172.21.15.97:0/3050529333 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd18801f870 con 0x55cba1fd01d0 2024-03-27T10:57:44.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.824+0000 7fd182a00640 1 -- 172.21.15.97:0/3050529333 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fd1940bb280 con 0x55cba1fd01d0 2024-03-27T10:57:44.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.824+0000 7fd1a6d88740 10 client.15879 fetch_fsmap finished waiting for FSMap version 28 2024-03-27T10:57:44.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.824+0000 7fd1a6d88740 10 client.15879 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-27T10:57:44.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.824+0000 7fd1a6d88740 20 client.15879 populate_metadata read hostname 'smithi097' 2024-03-27T10:57:44.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.825+0000 7fd1a6d88740 1 --2- 172.21.15.97:0/3050529333 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55cba1ff1ad0 0x55cba1ff3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:44.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.825+0000 7fd1a6d88740 4 client.15879 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T10:57:44.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.825+0000 7fd1a6d88740 1 -- 172.21.15.97:0/3050529333 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cba1f3dbf0 con 0x55cba1ff1ad0 2024-03-27T10:57:44.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.825+0000 7fd1a1600640 1 --2- 172.21.15.97:0/3050529333 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55cba1ff1ad0 0x55cba1ff3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:44.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.826+0000 7fd1a1600640 1 --2- 172.21.15.97:0/3050529333 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55cba1ff1ad0 0x55cba1ff3f50 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c91862ad9b97a72 server_cookie=f13de8b3e9fd299f in_seq=0 out_seq=0 2024-03-27T10:57:44.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.826+0000 7fd182a00640 10 client.15879 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T10:57:44.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.828+0000 7fd182a00640 1 -- 172.21.15.97:0/3050529333 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55cba1f3dbf0 con 0x55cba1ff1ad0 2024-03-27T10:57:44.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.828+0000 7fd182a00640 10 client.15879 handle_command_reply: tid=0 2024-03-27T10:57:44.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.828+0000 7fd1a6d88740 1 client.15879 shutdown 2024-03-27T10:57:44.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.828+0000 7fd1a6d88740 1 -- 172.21.15.97:0/3050529333 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cba1fd01d0 msgr2=0x55cba1fed120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:44.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.828+0000 7fd1a6d88740 1 --2- 172.21.15.97:0/3050529333 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cba1fd01d0 0x55cba1fed120 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fd1940a4f10 tx=0x7fd19409c310 comp rx=0 tx=0).stop 2024-03-27T10:57:44.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.828+0000 7fd1a6d88740 1 -- 172.21.15.97:0/3050529333 shutdown_connections 2024-03-27T10:57:44.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.828+0000 7fd1a6d88740 1 --2- 172.21.15.97:0/3050529333 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55cba1ff1ad0 0x55cba1ff3f50 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:44.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.828+0000 7fd1a6d88740 1 --2- 172.21.15.97:0/3050529333 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cba1fd01d0 0x55cba1fed120 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:44.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.828+0000 7fd182a00640 0 client.15879 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T10:57:44.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.828+0000 7fd1a6d88740 1 --2- 172.21.15.97:0/3050529333 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cba1fcb060 0x55cba1fe60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:44.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.828+0000 7fd1a6d88740 1 --2- 172.21.15.97:0/3050529333 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cba1fca670 0x55cba1fe5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:44.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.828+0000 7fd1a6d88740 1 -- 172.21.15.97:0/3050529333 >> 172.21.15.97:0/3050529333 conn(0x55cba1fc5be0 msgr2=0x55cba1fc4e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:44.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.828+0000 7fd1a6d88740 1 -- 172.21.15.97:0/3050529333 shutdown_connections 2024-03-27T10:57:44.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.828+0000 7fd1a6d88740 1 -- 172.21.15.97:0/3050529333 wait complete. 2024-03-27T10:57:44.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.829+0000 7fd1a6d88740 20 client.15879 trim_cache size 0 max 16384 2024-03-27T10:57:44.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.847+0000 7fd1a6d88740 1 Processor -- start 2024-03-27T10:57:44.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.847+0000 7fd1a6d88740 1 -- start start 2024-03-27T10:57:44.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.847+0000 7fd1a6d88740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cba1fca670 0x55cba1fe51c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:44.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.847+0000 7fd1a6d88740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cba1fcb060 0x55cba1fed0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:44.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.847+0000 7fd1a6d88740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cba1fd01d0 0x55cba1fc45b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:44.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.847+0000 7fd1a6d88740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd19c1a2710 con 0x55cba1fd01d0 2024-03-27T10:57:44.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.847+0000 7fd1a6d88740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd19c1a4250 con 0x55cba1fca670 2024-03-27T10:57:44.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.847+0000 7fd1a6d88740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd19c1a2de0 con 0x55cba1fcb060 2024-03-27T10:57:44.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.848+0000 7fd1a2000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cba1fd01d0 0x55cba1fc45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:44.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.848+0000 7fd1a2000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cba1fd01d0 0x55cba1fc45b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53784/0 (socket says 172.21.15.97:53784) 2024-03-27T10:57:44.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.848+0000 7fd1a2000640 1 -- 172.21.15.97:0/33617067 learned_addr learned my addr 172.21.15.97:0/33617067 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:44.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.848+0000 7fd1a1600640 1 --2- 172.21.15.97:0/33617067 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cba1fca670 0x55cba1fe51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:44.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.848+0000 7fd1a0c00640 1 --2- 172.21.15.97:0/33617067 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cba1fcb060 0x55cba1fed0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:44.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.848+0000 7fd1a2000640 1 -- 172.21.15.97:0/33617067 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cba1fcb060 msgr2=0x55cba1fed0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:44.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.848+0000 7fd1a2000640 1 --2- 172.21.15.97:0/33617067 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cba1fcb060 0x55cba1fed0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:44.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.848+0000 7fd1a2000640 1 -- 172.21.15.97:0/33617067 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cba1fca670 msgr2=0x55cba1fe51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:44.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.848+0000 7fd1a2000640 1 --2- 172.21.15.97:0/33617067 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cba1fca670 0x55cba1fe51c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:44.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.848+0000 7fd1a2000640 1 -- 172.21.15.97:0/33617067 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cba1fcb8f0 con 0x55cba1fd01d0 2024-03-27T10:57:44.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.849+0000 7fd1a2000640 1 --2- 172.21.15.97:0/33617067 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cba1fd01d0 0x55cba1fc45b0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fd19409aa80 tx=0x7fd1940a5370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d027ce168a03f90 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:44.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.849+0000 7fd18dc00640 1 -- 172.21.15.97:0/33617067 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd194080080 con 0x55cba1fd01d0 2024-03-27T10:57:44.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.849+0000 7fd18dc00640 1 -- 172.21.15.97:0/33617067 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd1940b4070 con 0x55cba1fd01d0 2024-03-27T10:57:44.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.849+0000 7fd18dc00640 1 -- 172.21.15.97:0/33617067 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1940bb280 con 0x55cba1fd01d0 2024-03-27T10:57:44.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.850+0000 7fd1a6d88740 1 -- 172.21.15.97:0/33617067 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cba1fd01d0 msgr2=0x55cba1fc45b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:44.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.850+0000 7fd1a6d88740 1 --2- 172.21.15.97:0/33617067 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cba1fd01d0 0x55cba1fc45b0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fd19409aa80 tx=0x7fd1940a5370 comp rx=0 tx=0).stop 2024-03-27T10:57:44.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.850+0000 7fd1a6d88740 1 -- 172.21.15.97:0/33617067 shutdown_connections 2024-03-27T10:57:44.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.850+0000 7fd1a6d88740 1 --2- 172.21.15.97:0/33617067 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cba1fd01d0 0x55cba1fc45b0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:44.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.850+0000 7fd1a6d88740 1 --2- 172.21.15.97:0/33617067 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cba1fcb060 0x55cba1fed0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:44.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.850+0000 7fd1a6d88740 1 --2- 172.21.15.97:0/33617067 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cba1fca670 0x55cba1fe51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:44.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.850+0000 7fd1a6d88740 1 -- 172.21.15.97:0/33617067 >> 172.21.15.97:0/33617067 conn(0x55cba1fc5be0 msgr2=0x55cba1fcc560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:44.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.850+0000 7fd1a6d88740 1 -- 172.21.15.97:0/33617067 shutdown_connections 2024-03-27T10:57:44.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.850+0000 7fd1a6d88740 1 -- 172.21.15.97:0/33617067 wait complete. 2024-03-27T10:57:44.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.850+0000 7fd1a6d88740 1 -- start start 2024-03-27T10:57:44.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.851+0000 7fd1a6d88740 1 Processor -- start 2024-03-27T10:57:44.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.851+0000 7fd1a6d88740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cba1fca670 0x55cba1fead40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:44.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.851+0000 7fd1a6d88740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cba1fcb060 0x55cba1feb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:44.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.851+0000 7fd1a6d88740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cba1fd01d0 0x55cba1fec290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:44.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.851+0000 7fd1a6d88740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd19c1a3b80 con 0x55cba1fd01d0 2024-03-27T10:57:44.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.851+0000 7fd1a6d88740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd19c1a34b0 con 0x55cba1fca670 2024-03-27T10:57:44.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.851+0000 7fd1a6d88740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd19c071f50 con 0x55cba1fcb060 2024-03-27T10:57:44.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.851+0000 7fd1a6d88740 10 client.0 _ll_register_callbacks cb 0x7fd19409c5a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:57:44.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.851+0000 7fd1a2000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cba1fd01d0 0x55cba1fec290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:44.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.851+0000 7fd1a2000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cba1fd01d0 0x55cba1fec290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53800/0 (socket says 172.21.15.97:53800) 2024-03-27T10:57:44.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.851+0000 7fd1a2000640 1 -- 172.21.15.97:0/2546343941 learned_addr learned my addr 172.21.15.97:0/2546343941 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:44.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.852+0000 7fd1a1600640 1 --2- 172.21.15.97:0/2546343941 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cba1fca670 0x55cba1fead40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:44.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.852+0000 7fd1a2000640 1 -- 172.21.15.97:0/2546343941 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cba1fcb060 msgr2=0x55cba1feb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:44.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.852+0000 7fd1a0c00640 1 --2- 172.21.15.97:0/2546343941 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cba1fcb060 0x55cba1feb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:44.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.852+0000 7fd1a2000640 1 --2- 172.21.15.97:0/2546343941 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cba1fcb060 0x55cba1feb280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:44.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.852+0000 7fd1a2000640 1 -- 172.21.15.97:0/2546343941 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cba1fca670 msgr2=0x55cba1fead40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:44.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.852+0000 7fd1a2000640 1 --2- 172.21.15.97:0/2546343941 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cba1fca670 0x55cba1fead40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:44.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.852+0000 7fd1a2000640 1 -- 172.21.15.97:0/2546343941 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cba1fec880 con 0x55cba1fd01d0 2024-03-27T10:57:44.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.852+0000 7fd1a1600640 1 --2- 172.21.15.97:0/2546343941 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cba1fca670 0x55cba1fead40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:57:44.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.852+0000 7fd1a2000640 1 --2- 172.21.15.97:0/2546343941 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cba1fd01d0 0x55cba1fec290 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fd19409c760 tx=0x7fd1940057e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95e33655987dda81 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:44.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.852+0000 7fd182a00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T10:57:44.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.852+0000 7fd182a00640 1 -- 172.21.15.97:0/2546343941 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd19409c130 con 0x55cba1fd01d0 2024-03-27T10:57:44.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.853+0000 7fd182a00640 1 -- 172.21.15.97:0/2546343941 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fd19409db00 con 0x55cba1fd01d0 2024-03-27T10:57:44.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.853+0000 7fd1a6d88740 1 -- 172.21.15.97:0/2546343941 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1940b4070 con 0x55cba1fd01d0 2024-03-27T10:57:44.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.853+0000 7fd182a00640 1 -- 172.21.15.97:0/2546343941 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd19409dd00 con 0x55cba1fd01d0 2024-03-27T10:57:44.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.853+0000 7fd1a6d88740 10 client.15885 fetch_fsmap learned FSMap version 28 2024-03-27T10:57:44.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.853+0000 7fd1a6d88740 1 -- 172.21.15.97:0/2546343941 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55cba2060b50 con 0x55cba1fd01d0 2024-03-27T10:57:44.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.853+0000 7fd182a00640 1 -- 172.21.15.97:0/2546343941 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd1940b4070 con 0x55cba1fd01d0 2024-03-27T10:57:44.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.853+0000 7fd182a00640 1 -- 172.21.15.97:0/2546343941 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fd19409b690 con 0x55cba1fd01d0 2024-03-27T10:57:44.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.853+0000 7fd1a6d88740 10 client.15885 fetch_fsmap finished waiting for FSMap version 28 2024-03-27T10:57:44.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.853+0000 7fd1a6d88740 10 client.15885 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-27T10:57:44.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.853+0000 7fd1a6d88740 20 client.15885 populate_metadata read hostname 'smithi097' 2024-03-27T10:57:44.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.853+0000 7fd1a6d88740 1 --2- 172.21.15.97:0/2546343941 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55cba2061460 0x55cba2063880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:44.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.853+0000 7fd1a6d88740 4 client.15885 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-27T10:57:44.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.853+0000 7fd1a6d88740 1 -- 172.21.15.97:0/2546343941 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55cba1fe5990 con 0x55cba2061460 2024-03-27T10:57:44.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.854+0000 7fd1a1600640 1 --2- 172.21.15.97:0/2546343941 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55cba2061460 0x55cba2063880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:44.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.855+0000 7fd1a1600640 1 --2- 172.21.15.97:0/2546343941 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55cba2061460 0x55cba2063880 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bcbc82b40562b2d1 server_cookie=b8ba020b94d3447f in_seq=0 out_seq=0 2024-03-27T10:57:44.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.855+0000 7fd182a00640 10 client.15885 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T10:57:44.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.855+0000 7fd182a00640 1 -- 172.21.15.97:0/2546343941 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55cba1fe5990 con 0x55cba2061460 2024-03-27T10:57:44.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.855+0000 7fd182a00640 10 client.15885 handle_command_reply: tid=0 2024-03-27T10:57:44.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.855+0000 7fd1a6d88740 1 client.15885 shutdown 2024-03-27T10:57:44.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.855+0000 7fd1a6d88740 1 -- 172.21.15.97:0/2546343941 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cba1fd01d0 msgr2=0x55cba1fec290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:44.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.855+0000 7fd1a6d88740 1 --2- 172.21.15.97:0/2546343941 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cba1fd01d0 0x55cba1fec290 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fd19409c760 tx=0x7fd1940057e0 comp rx=0 tx=0).stop 2024-03-27T10:57:44.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.855+0000 7fd1a6d88740 1 -- 172.21.15.97:0/2546343941 shutdown_connections 2024-03-27T10:57:44.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.855+0000 7fd1a6d88740 1 --2- 172.21.15.97:0/2546343941 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55cba2061460 0x55cba2063880 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:44.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.855+0000 7fd1a6d88740 1 --2- 172.21.15.97:0/2546343941 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cba1fd01d0 0x55cba1fec290 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:44.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.855+0000 7fd1a6d88740 1 --2- 172.21.15.97:0/2546343941 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cba1fcb060 0x55cba1feb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:44.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.855+0000 7fd1a6d88740 1 --2- 172.21.15.97:0/2546343941 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cba1fca670 0x55cba1fead40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:44.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.855+0000 7fd1a6d88740 1 -- 172.21.15.97:0/2546343941 >> 172.21.15.97:0/2546343941 conn(0x55cba1fc5be0 msgr2=0x55cba1fc45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:44.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.855+0000 7fd182a00640 0 client.15885 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T10:57:44.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.856+0000 7fd1a6d88740 1 -- 172.21.15.97:0/2546343941 shutdown_connections 2024-03-27T10:57:44.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.856+0000 7fd1a6d88740 1 -- 172.21.15.97:0/2546343941 wait complete. 2024-03-27T10:57:44.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.856+0000 7fd1a6d88740 20 client.15885 trim_cache size 0 max 16384 2024-03-27T10:57:44.830 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-03-27T10:57:44.830 INFO:teuthology.orchestra.run.smithi097.stdout: "status": "no active scrubs running", 2024-03-27T10:57:44.830 INFO:teuthology.orchestra.run.smithi097.stdout: "scrubs": {} 2024-03-27T10:57:44.830 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-03-27T10:57:44.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.856+0000 7fd1a2a00640 1 -- 172.21.15.97:0/1812299188 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fd1840a3690 msgr2=0x7fd1840a5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:44.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.856+0000 7fd1a2a00640 1 --2- 172.21.15.97:0/1812299188 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fd1840a3690 0x7fd1840a5b50 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd188023000 tx=0x7fd188020000 comp rx=0 tx=0).stop 2024-03-27T10:57:44.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.857+0000 7fd1a2a00640 1 -- 172.21.15.97:0/1812299188 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd19c107710 msgr2=0x7fd19c1009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:44.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.857+0000 7fd1a2a00640 1 --2- 172.21.15.97:0/1812299188 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd19c107710 0x7fd19c1009a0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fd19400eb10 tx=0x7fd19400c2d0 comp rx=0 tx=0).stop 2024-03-27T10:57:44.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.857+0000 7fd1a2a00640 1 -- 172.21.15.97:0/1812299188 shutdown_connections 2024-03-27T10:57:44.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.857+0000 7fd1a2a00640 1 --2- 172.21.15.97:0/1812299188 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd19c107710 0x7fd19c1009a0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:44.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.857+0000 7fd1a2a00640 1 --2- 172.21.15.97:0/1812299188 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fd1840a3690 0x7fd1840a5b50 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:44.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.857+0000 7fd1a2a00640 1 --2- 172.21.15.97:0/1812299188 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd19c104de0 0x7fd19c100460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:44.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.857+0000 7fd1a2a00640 1 --2- 172.21.15.97:0/1812299188 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd19c1024b0 0x7fd19c101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:44.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.857+0000 7fd1a2a00640 1 -- 172.21.15.97:0/1812299188 >> 172.21.15.97:0/1812299188 conn(0x7fd19c0fbec0 msgr2=0x7fd19c102ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:44.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.857+0000 7fd1a2a00640 1 -- 172.21.15.97:0/1812299188 shutdown_connections 2024-03-27T10:57:44.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:44.857+0000 7fd1a2a00640 1 -- 172.21.15.97:0/1812299188 wait complete. 2024-03-27T10:57:44.843 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-27T10:57:44.843 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-27T10:57:45.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.044+0000 7f7efdc00640 1 -- 172.21.15.97:0/3597161107 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ef8109850 msgr2=0x7f7ef810bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:45.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.044+0000 7f7efdc00640 1 --2- 172.21.15.97:0/3597161107 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ef8109850 0x7f7ef810bce0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f7ef0009ca0 tx=0x7f7ef002c190 comp rx=0 tx=0).stop 2024-03-27T10:57:45.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.045+0000 7f7efdc00640 1 -- 172.21.15.97:0/3597161107 shutdown_connections 2024-03-27T10:57:45.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.045+0000 7f7efdc00640 1 --2- 172.21.15.97:0/3597161107 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ef8109850 0x7f7ef810bce0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:45.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.045+0000 7f7efdc00640 1 --2- 172.21.15.97:0/3597161107 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7ef8074530 0x7f7ef8074990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:45.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.045+0000 7f7efdc00640 1 --2- 172.21.15.97:0/3597161107 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7ef8075b10 0x7f7ef8073f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:45.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.045+0000 7f7efdc00640 1 -- 172.21.15.97:0/3597161107 >> 172.21.15.97:0/3597161107 conn(0x7f7ef80fcf30 msgr2=0x7f7ef80ff350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:45.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.045+0000 7f7efdc00640 1 -- 172.21.15.97:0/3597161107 shutdown_connections 2024-03-27T10:57:45.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.045+0000 7f7efdc00640 1 -- 172.21.15.97:0/3597161107 wait complete. 2024-03-27T10:57:45.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.046+0000 7f7efdc00640 1 Processor -- start 2024-03-27T10:57:45.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.046+0000 7f7efdc00640 1 -- start start 2024-03-27T10:57:45.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.046+0000 7f7efdc00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7ef8074530 0x7f7ef8199d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:45.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.046+0000 7f7efdc00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7ef8075b10 0x7f7ef819a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:45.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.046+0000 7f7efdc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ef8109850 0x7f7ef81a12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:45.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.046+0000 7f7efdc00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef810f6e0 con 0x7f7ef8109850 2024-03-27T10:57:45.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.046+0000 7f7efdc00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef810f560 con 0x7f7ef8074530 2024-03-27T10:57:45.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.046+0000 7f7efdc00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef810f860 con 0x7f7ef8075b10 2024-03-27T10:57:45.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.047+0000 7f7efd200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ef8109850 0x7f7ef81a12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:45.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.047+0000 7f7efd200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ef8109850 0x7f7ef81a12a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53816/0 (socket says 172.21.15.97:53816) 2024-03-27T10:57:45.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.047+0000 7f7efd200640 1 -- 172.21.15.97:0/4242056364 learned_addr learned my addr 172.21.15.97:0/4242056364 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:45.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.047+0000 7f7ef7e00640 1 --2- 172.21.15.97:0/4242056364 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7ef8074530 0x7f7ef8199d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:45.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.047+0000 7f7ef7400640 1 --2- 172.21.15.97:0/4242056364 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7ef8075b10 0x7f7ef819a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:45.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.047+0000 7f7ef7e00640 1 -- 172.21.15.97:0/4242056364 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7ef8075b10 msgr2=0x7f7ef819a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:45.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.047+0000 7f7ef7e00640 1 --2- 172.21.15.97:0/4242056364 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7ef8075b10 0x7f7ef819a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:45.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.047+0000 7f7ef7e00640 1 -- 172.21.15.97:0/4242056364 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ef8109850 msgr2=0x7f7ef81a12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:45.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.047+0000 7f7ef7e00640 1 --2- 172.21.15.97:0/4242056364 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ef8109850 0x7f7ef81a12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:45.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.047+0000 7f7ef7e00640 1 -- 172.21.15.97:0/4242056364 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ef81a1940 con 0x7f7ef8074530 2024-03-27T10:57:45.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.047+0000 7f7efd200640 1 --2- 172.21.15.97:0/4242056364 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ef8109850 0x7f7ef81a12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T10:57:45.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.048+0000 7f7ef7e00640 1 --2- 172.21.15.97:0/4242056364 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7ef8074530 0x7f7ef8199d30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7ee800e9f0 tx=0x7f7ee800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:45.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.048+0000 7f7ef4c00640 1 -- 172.21.15.97:0/4242056364 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee800ccc0 con 0x7f7ef8074530 2024-03-27T10:57:45.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.048+0000 7f7efdc00640 1 -- 172.21.15.97:0/4242056364 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ef81a1c30 con 0x7f7ef8074530 2024-03-27T10:57:45.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.048+0000 7f7efdc00640 1 -- 172.21.15.97:0/4242056364 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ef81a21d0 con 0x7f7ef8074530 2024-03-27T10:57:45.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.048+0000 7f7ef4c00640 1 -- 172.21.15.97:0/4242056364 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7ee80108e0 con 0x7f7ef8074530 2024-03-27T10:57:45.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.048+0000 7f7ef4c00640 1 -- 172.21.15.97:0/4242056364 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee8018750 con 0x7f7ef8074530 2024-03-27T10:57:45.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.050+0000 7f7efdc00640 1 -- 172.21.15.97:0/4242056364 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ef8108610 con 0x7f7ef8074530 2024-03-27T10:57:45.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.053+0000 7f7ef4c00640 1 -- 172.21.15.97:0/4242056364 <== mon.2 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f7ee8010a90 con 0x7f7ef8074530 2024-03-27T10:57:45.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.054+0000 7f7ef4c00640 1 --2- 172.21.15.97:0/4242056364 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f7edc0a3610 0x7f7edc0a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:45.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.054+0000 7f7ef4c00640 1 -- 172.21.15.97:0/4242056364 <== mon.2 v2:172.21.15.114:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f7ee80d0270 con 0x7f7ef8074530 2024-03-27T10:57:45.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.055+0000 7f7ef7400640 1 --2- 172.21.15.97:0/4242056364 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f7edc0a3610 0x7f7edc0a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:45.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.055+0000 7f7ef7400640 1 --2- 172.21.15.97:0/4242056364 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f7edc0a3610 0x7f7edc0a5ad0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f7ee4005fd0 tx=0x7f7ee400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:45.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.072+0000 7f7ef4c00640 1 -- 172.21.15.97:0/4242056364 <== mon.2 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f7ee809c800 con 0x7f7ef8074530 2024-03-27T10:57:45.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.307+0000 7f7efdc00640 1 -- 172.21.15.97:0/4242056364 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7ef802d580 con 0x7f7ef8074530 2024-03-27T10:57:45.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.308+0000 7f7ef4c00640 1 -- 172.21.15.97:0/4242056364 <== mon.2 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11944 (secure 0 0 0) 0x7f7ee80a0b00 con 0x7f7ef8074530 2024-03-27T10:57:45.280 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T10:57:45.280 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1009423904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1009423904},{"type":"v1","addr":"172.21.15.97:6835","nonce":1009423904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3251173433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3251173433},{"type":"v1","addr":"172.21.15.97:6837","nonce":3251173433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-27T10:54:04.510178+0000","modified":"2024-03-27T10:56:28.179534+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34508,"mds_1":34478,"mds_2":34445,"mds_3":24454,"mds_4":24400},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14646":{"gid":14646,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6839/1057596821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1057596821},{"type":"v1","addr":"172.21.15.97:6839","nonce":1057596821}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14658":{"gid":14658,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6841/1672464856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6840","nonce":1672464856},{"type":"v1","addr":"172.21.15.97:6841","nonce":1672464856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.177:6833/3787981632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3787981632},{"type":"v1","addr":"172.21.15.177:6833","nonce":3787981632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.177:6835/3349112132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":3349112132},{"type":"v1","addr":"172.21.15.177:6835","nonce":3349112132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6837/3244432640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":3244432640},{"type":"v1","addr":"172.21.15.177:6837","nonce":3244432640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6839/3340454605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":3340454605},{"type":"v1","addr":"172.21.15.177:6839","nonce":3340454605}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34415":{"gid":34415,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6833/4052517902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":4052517902},{"type":"v1","addr":"172.21.15.114:6833","nonce":4052517902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34445":{"gid":34445,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.114:6835/3581918222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3581918222},{"type":"v1","addr":"172.21.15.114:6835","nonce":3581918222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34478":{"gid":34478,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.114:6837/616775588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":616775588},{"type":"v1","addr":"172.21.15.114:6837","nonce":616775588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34508":{"gid":34508,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.114:6839/3937193784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":3937193784},{"type":"v1","addr":"172.21.15.114:6839","nonce":3937193784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34508,"qdb_cluster":[34478,24454,24400,34445,34508]},"id":1}]} 2024-03-27T10:57:45.280 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 28 2024-03-27T10:57:45.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.311+0000 7f7efdc00640 1 -- 172.21.15.97:0/4242056364 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f7edc0a3610 msgr2=0x7f7edc0a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:45.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.311+0000 7f7efdc00640 1 --2- 172.21.15.97:0/4242056364 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f7edc0a3610 0x7f7edc0a5ad0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f7ee4005fd0 tx=0x7f7ee400b390 comp rx=0 tx=0).stop 2024-03-27T10:57:45.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.311+0000 7f7efdc00640 1 -- 172.21.15.97:0/4242056364 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7ef8074530 msgr2=0x7f7ef8199d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:45.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.311+0000 7f7efdc00640 1 --2- 172.21.15.97:0/4242056364 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7ef8074530 0x7f7ef8199d30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7ee800e9f0 tx=0x7f7ee800eeb0 comp rx=0 tx=0).stop 2024-03-27T10:57:45.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.311+0000 7f7efdc00640 1 -- 172.21.15.97:0/4242056364 shutdown_connections 2024-03-27T10:57:45.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.311+0000 7f7efdc00640 1 --2- 172.21.15.97:0/4242056364 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ef8109850 0x7f7ef81a12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:45.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.311+0000 7f7efdc00640 1 --2- 172.21.15.97:0/4242056364 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7ef8075b10 0x7f7ef819a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:45.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.311+0000 7f7efdc00640 1 --2- 172.21.15.97:0/4242056364 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f7edc0a3610 0x7f7edc0a5ad0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:45.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.311+0000 7f7efdc00640 1 --2- 172.21.15.97:0/4242056364 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7ef8074530 0x7f7ef8199d30 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:45.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.311+0000 7f7efdc00640 1 -- 172.21.15.97:0/4242056364 >> 172.21.15.97:0/4242056364 conn(0x7f7ef80fcf30 msgr2=0x7f7ef81057e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:45.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.311+0000 7f7efdc00640 1 -- 172.21.15.97:0/4242056364 shutdown_connections 2024-03-27T10:57:45.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.311+0000 7f7efdc00640 1 -- 172.21.15.97:0/4242056364 wait complete. 2024-03-27T10:57:45.304 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-27T10:57:45.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.510+0000 7f0695600640 1 -- 172.21.15.97:0/3600241331 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0690105800 msgr2=0x7f069010c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:45.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.510+0000 7f0695600640 1 --2- 172.21.15.97:0/3600241331 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0690105800 0x7f069010c090 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f0688009ca0 tx=0x7f068802c0f0 comp rx=0 tx=0).stop 2024-03-27T10:57:45.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.510+0000 7f0695600640 1 -- 172.21.15.97:0/3600241331 shutdown_connections 2024-03-27T10:57:45.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.510+0000 7f0695600640 1 --2- 172.21.15.97:0/3600241331 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0690105800 0x7f069010c090 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:45.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.510+0000 7f0695600640 1 --2- 172.21.15.97:0/3600241331 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0690104e40 0x7f06901052c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:45.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.510+0000 7f0695600640 1 --2- 172.21.15.97:0/3600241331 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0690103c50 0x7f0690104070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:45.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.510+0000 7f0695600640 1 -- 172.21.15.97:0/3600241331 >> 172.21.15.97:0/3600241331 conn(0x7f06900ff1c0 msgr2=0x7f0690101600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:45.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.510+0000 7f0695600640 1 -- 172.21.15.97:0/3600241331 shutdown_connections 2024-03-27T10:57:45.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.510+0000 7f0695600640 1 -- 172.21.15.97:0/3600241331 wait complete. 2024-03-27T10:57:45.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.511+0000 7f0695600640 1 Processor -- start 2024-03-27T10:57:45.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.511+0000 7f0695600640 1 -- start start 2024-03-27T10:57:45.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.512+0000 7f0695600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0690103c50 0x7f0690199df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:45.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.512+0000 7f0695600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0690104e40 0x7f069019a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:45.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.512+0000 7f0695600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0690105800 0x7f06901a13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:45.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.512+0000 7f0695600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f069010f780 con 0x7f0690105800 2024-03-27T10:57:45.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.512+0000 7f0695600640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f069010f600 con 0x7f0690103c50 2024-03-27T10:57:45.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.512+0000 7f0695600640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f069010f900 con 0x7f0690104e40 2024-03-27T10:57:45.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.512+0000 7f0694c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0690105800 0x7f06901a13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:45.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.512+0000 7f0694c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0690105800 0x7f06901a13a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53842/0 (socket says 172.21.15.97:53842) 2024-03-27T10:57:45.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.512+0000 7f068fe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0690103c50 0x7f0690199df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:45.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.512+0000 7f0694c00640 1 -- 172.21.15.97:0/1194109221 learned_addr learned my addr 172.21.15.97:0/1194109221 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:45.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.513+0000 7f068f400640 1 --2- 172.21.15.97:0/1194109221 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0690104e40 0x7f069019a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:45.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.513+0000 7f0694c00640 1 -- 172.21.15.97:0/1194109221 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0690104e40 msgr2=0x7f069019a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:45.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.513+0000 7f0694c00640 1 --2- 172.21.15.97:0/1194109221 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0690104e40 0x7f069019a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:45.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.513+0000 7f0694c00640 1 -- 172.21.15.97:0/1194109221 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0690103c50 msgr2=0x7f0690199df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:45.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.513+0000 7f0694c00640 1 --2- 172.21.15.97:0/1194109221 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0690103c50 0x7f0690199df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:45.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.513+0000 7f0694c00640 1 -- 172.21.15.97:0/1194109221 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06901a1a10 con 0x7f0690105800 2024-03-27T10:57:45.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.514+0000 7f0694c00640 1 --2- 172.21.15.97:0/1194109221 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0690105800 0x7f06901a13a0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f068802c700 tx=0x7f06880328a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:45.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.514+0000 7f068cc00640 1 -- 172.21.15.97:0/1194109221 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0688041020 con 0x7f0690105800 2024-03-27T10:57:45.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.514+0000 7f068cc00640 1 -- 172.21.15.97:0/1194109221 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0688003e20 con 0x7f0690105800 2024-03-27T10:57:45.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.514+0000 7f068cc00640 1 -- 172.21.15.97:0/1194109221 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0688039040 con 0x7f0690105800 2024-03-27T10:57:45.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.514+0000 7f0695600640 1 -- 172.21.15.97:0/1194109221 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06901a1ca0 con 0x7f0690105800 2024-03-27T10:57:45.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.514+0000 7f0695600640 1 -- 172.21.15.97:0/1194109221 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06901a2190 con 0x7f0690105800 2024-03-27T10:57:45.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.518+0000 7f06999fa740 1 Processor -- start 2024-03-27T10:57:45.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.519+0000 7f068cc00640 1 -- 172.21.15.97:0/1194109221 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f0688044020 con 0x7f0690105800 2024-03-27T10:57:45.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.519+0000 7f06999fa740 1 -- start start 2024-03-27T10:57:45.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.520+0000 7f06999fa740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561b3bda82c0 0x561b3bda86e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:45.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.520+0000 7f06999fa740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561b3bda8cb0 0x561b3bdad7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:45.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.520+0000 7f06999fa740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561b3bdade20 0x561b3bdb02b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:45.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.520+0000 7f06999fa740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x561b3bd677b0 con 0x561b3bda82c0 2024-03-27T10:57:45.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.520+0000 7f06999fa740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x561b3bd04670 con 0x561b3bda8cb0 2024-03-27T10:57:45.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.520+0000 7f06999fa740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x561b3bcac970 con 0x561b3bdade20 2024-03-27T10:57:45.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.520+0000 7f068f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561b3bda82c0 0x561b3bda86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:45.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.520+0000 7f068fe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561b3bda8cb0 0x561b3bdad7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:45.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.520+0000 7f068fe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561b3bda8cb0 0x561b3bdad7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:34170/0 (socket says 172.21.15.97:34170) 2024-03-27T10:57:45.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.520+0000 7f068fe00640 1 -- 172.21.15.97:0/1584957894 learned_addr learned my addr 172.21.15.97:0/1584957894 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:45.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.521+0000 7f068fe00640 1 -- 172.21.15.97:0/1584957894 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561b3bdade20 msgr2=0x561b3bdb02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:45.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.521+0000 7f068fe00640 1 --2- 172.21.15.97:0/1584957894 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561b3bdade20 0x561b3bdb02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:45.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.521+0000 7f068fe00640 1 -- 172.21.15.97:0/1584957894 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561b3bda82c0 msgr2=0x561b3bda86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:45.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.521+0000 7f068fe00640 1 --2- 172.21.15.97:0/1584957894 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561b3bda82c0 0x561b3bda86e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:45.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.521+0000 7f068fe00640 1 -- 172.21.15.97:0/1584957894 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b3ba183f0 con 0x561b3bda8cb0 2024-03-27T10:57:45.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.521+0000 7f068f400640 1 --2- 172.21.15.97:0/1584957894 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561b3bda82c0 0x561b3bda86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T10:57:45.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.521+0000 7f068cc00640 1 --2- 172.21.15.97:0/1194109221 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f06740a3610 0x7f06740a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:45.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.521+0000 7f068cc00640 1 -- 172.21.15.97:0/1194109221 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f06880f2120 con 0x7f0690105800 2024-03-27T10:57:45.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.522+0000 7f068fe00640 1 --2- 172.21.15.97:0/1584957894 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561b3bda8cb0 0x561b3bdad7b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f068000b7d0 tx=0x7f0680004610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d435394e93cce55 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:45.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.522+0000 7f068f400640 1 --2- 172.21.15.97:0/1194109221 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f06740a3610 0x7f06740a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:45.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.522+0000 7f0685200640 1 -- 172.21.15.97:0/1584957894 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0680003dd0 con 0x561b3bda8cb0 2024-03-27T10:57:45.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.522+0000 7f0685200640 1 -- 172.21.15.97:0/1584957894 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0680015070 con 0x561b3bda8cb0 2024-03-27T10:57:45.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.522+0000 7f0685200640 1 -- 172.21.15.97:0/1584957894 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0680011680 con 0x561b3bda8cb0 2024-03-27T10:57:45.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.523+0000 7f06999fa740 1 -- 172.21.15.97:0/1584957894 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561b3bda8cb0 msgr2=0x561b3bdad7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:45.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.523+0000 7f06999fa740 1 --2- 172.21.15.97:0/1584957894 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561b3bda8cb0 0x561b3bdad7b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f068000b7d0 tx=0x7f0680004610 comp rx=0 tx=0).stop 2024-03-27T10:57:45.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.523+0000 7f068f400640 1 --2- 172.21.15.97:0/1194109221 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f06740a3610 0x7f06740a5ad0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f067c004650 tx=0x7f067c004170 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:45.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.523+0000 7f06999fa740 1 -- 172.21.15.97:0/1584957894 shutdown_connections 2024-03-27T10:57:45.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.523+0000 7f06999fa740 1 --2- 172.21.15.97:0/1584957894 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561b3bdade20 0x561b3bdb02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:45.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.523+0000 7f06999fa740 1 --2- 172.21.15.97:0/1584957894 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561b3bda8cb0 0x561b3bdad7b0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:45.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.523+0000 7f06999fa740 1 --2- 172.21.15.97:0/1584957894 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561b3bda82c0 0x561b3bda86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:45.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.523+0000 7f06999fa740 1 -- 172.21.15.97:0/1584957894 >> 172.21.15.97:0/1584957894 conn(0x561b3bda3830 msgr2=0x561b3bda3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:45.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.523+0000 7f06999fa740 1 -- 172.21.15.97:0/1584957894 shutdown_connections 2024-03-27T10:57:45.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.523+0000 7f06999fa740 1 -- 172.21.15.97:0/1584957894 wait complete. 2024-03-27T10:57:45.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.524+0000 7f06999fa740 1 -- start start 2024-03-27T10:57:45.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.524+0000 7f06999fa740 1 Processor -- start 2024-03-27T10:57:45.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.525+0000 7f06999fa740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561b3bda82c0 0x561b3bdb23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:45.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.525+0000 7f06999fa740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561b3bda8cb0 0x561b3bdb2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:45.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.525+0000 7f06999fa740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561b3bdade20 0x561b3bdb9970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:45.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.525+0000 7f06999fa740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x561b3bcbfca0 con 0x561b3bda8cb0 2024-03-27T10:57:45.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.525+0000 7f06999fa740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x561b3bd00620 con 0x561b3bdade20 2024-03-27T10:57:45.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.525+0000 7f06999fa740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x561b3bdb0dd0 con 0x561b3bda82c0 2024-03-27T10:57:45.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.525+0000 7f06999fa740 10 client.0 _ll_register_callbacks cb 0x561b3bcfc160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:57:45.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.525+0000 7f0694c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561b3bdade20 0x561b3bdb9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:45.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.525+0000 7f068fe00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561b3bda82c0 0x561b3bdb23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:45.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.525+0000 7f0694c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561b3bdade20 0x561b3bdb9970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:34184/0 (socket says 172.21.15.97:34184) 2024-03-27T10:57:45.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.525+0000 7f0694c00640 1 -- 172.21.15.97:0/2434561281 learned_addr learned my addr 172.21.15.97:0/2434561281 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:45.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.525+0000 7f068f400640 1 --2- 172.21.15.97:0/2434561281 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561b3bda8cb0 0x561b3bdb2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:45.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.526+0000 7f0694c00640 1 -- 172.21.15.97:0/2434561281 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561b3bda82c0 msgr2=0x561b3bdb23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:45.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.526+0000 7f0694c00640 1 --2- 172.21.15.97:0/2434561281 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561b3bda82c0 0x561b3bdb23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:45.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.526+0000 7f0694c00640 1 -- 172.21.15.97:0/2434561281 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561b3bda8cb0 msgr2=0x561b3bdb2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:45.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.526+0000 7f0694c00640 1 --2- 172.21.15.97:0/2434561281 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561b3bda8cb0 0x561b3bdb2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:45.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.526+0000 7f0694c00640 1 -- 172.21.15.97:0/2434561281 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b3bae27d0 con 0x561b3bdade20 2024-03-27T10:57:45.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.526+0000 7f0694c00640 1 --2- 172.21.15.97:0/2434561281 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561b3bdade20 0x561b3bdb9970 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f068802c6d0 tx=0x7f06880bdf00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a43fd0762235ddb server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:45.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.526+0000 7f0679600640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-03-27T10:57:45.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.527+0000 7f0679600640 1 -- 172.21.15.97:0/2434561281 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06880d7060 con 0x561b3bdade20 2024-03-27T10:57:45.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.527+0000 7f06999fa740 1 -- 172.21.15.97:0/2434561281 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0680015070 con 0x561b3bdade20 2024-03-27T10:57:45.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.527+0000 7f0679600640 1 -- 172.21.15.97:0/2434561281 <== mon.2 v2:172.21.15.114:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f0688004650 con 0x561b3bdade20 2024-03-27T10:57:45.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.527+0000 7f0679600640 1 -- 172.21.15.97:0/2434561281 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f06880dc050 con 0x561b3bdade20 2024-03-27T10:57:45.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.527+0000 7f06999fa740 10 client.35567 fetch_fsmap learned FSMap version 28 2024-03-27T10:57:45.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.527+0000 7f06999fa740 1 -- 172.21.15.97:0/2434561281 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561b3bdbcce0 con 0x561b3bdade20 2024-03-27T10:57:45.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.528+0000 7f0679600640 1 -- 172.21.15.97:0/2434561281 <== mon.2 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0680015070 con 0x561b3bdade20 2024-03-27T10:57:45.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.528+0000 7f0679600640 1 -- 172.21.15.97:0/2434561281 <== mon.2 v2:172.21.15.114:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f068803c030 con 0x561b3bdade20 2024-03-27T10:57:45.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.528+0000 7f06999fa740 10 client.35567 fetch_fsmap finished waiting for FSMap version 28 2024-03-27T10:57:45.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.528+0000 7f06999fa740 10 client.35567 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-03-27T10:57:45.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.528+0000 7f06999fa740 20 client.35567 populate_metadata read hostname 'smithi097' 2024-03-27T10:57:45.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.528+0000 7f06999fa740 1 --2- 172.21.15.97:0/2434561281 >> [v2:172.21.15.177:6832/3787981632,v1:172.21.15.177:6833/3787981632] conn(0x561b3bdbe410 0x561b3bdc0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:45.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.528+0000 7f06999fa740 4 client.35567 mds_command: new command op to 24400 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T10:57:45.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.528+0000 7f06999fa740 1 -- 172.21.15.97:0/2434561281 --> [v2:172.21.15.177:6832/3787981632,v1:172.21.15.177:6833/3787981632] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561b3bd11bb0 con 0x561b3bdbe410 2024-03-27T10:57:45.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.529+0000 7f068fe00640 1 --2- 172.21.15.97:0/2434561281 >> [v2:172.21.15.177:6832/3787981632,v1:172.21.15.177:6833/3787981632] conn(0x561b3bdbe410 0x561b3bdc0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:45.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.529+0000 7f068fe00640 1 --2- 172.21.15.97:0/2434561281 >> [v2:172.21.15.177:6832/3787981632,v1:172.21.15.177:6833/3787981632] conn(0x561b3bdbe410 0x561b3bdc0870 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3fbd9ccff40e88ce server_cookie=9d20140cd58a9de in_seq=0 out_seq=0 2024-03-27T10:57:45.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.529+0000 7f0679600640 10 client.35567 ms_handle_connect on v2:172.21.15.177:6832/3787981632 2024-03-27T10:57:45.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.531+0000 7f0679600640 1 -- 172.21.15.97:0/2434561281 <== mds.4 v2:172.21.15.177:6832/3787981632 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x561b3bd11bb0 con 0x561b3bdbe410 2024-03-27T10:57:45.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.531+0000 7f0679600640 10 client.35567 handle_command_reply: tid=0 2024-03-27T10:57:45.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.531+0000 7f06999fa740 1 client.35567 shutdown 2024-03-27T10:57:45.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.531+0000 7f06999fa740 1 -- 172.21.15.97:0/2434561281 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561b3bdade20 msgr2=0x561b3bdb9970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:45.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.531+0000 7f06999fa740 1 --2- 172.21.15.97:0/2434561281 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561b3bdade20 0x561b3bdb9970 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f068802c6d0 tx=0x7f06880bdf00 comp rx=0 tx=0).stop 2024-03-27T10:57:45.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.532+0000 7f06999fa740 1 -- 172.21.15.97:0/2434561281 shutdown_connections 2024-03-27T10:57:45.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.532+0000 7f06999fa740 1 --2- 172.21.15.97:0/2434561281 >> [v2:172.21.15.177:6832/3787981632,v1:172.21.15.177:6833/3787981632] conn(0x561b3bdbe410 0x561b3bdc0870 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:45.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.532+0000 7f06999fa740 1 --2- 172.21.15.97:0/2434561281 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561b3bdade20 0x561b3bdb9970 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:45.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.532+0000 7f06999fa740 1 --2- 172.21.15.97:0/2434561281 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561b3bda8cb0 0x561b3bdb2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:45.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.532+0000 7f06999fa740 1 --2- 172.21.15.97:0/2434561281 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561b3bda82c0 0x561b3bdb23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:45.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.532+0000 7f0679600640 0 client.35567 ms_handle_reset on v2:172.21.15.177:6832/3787981632 2024-03-27T10:57:45.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.532+0000 7f06999fa740 1 -- 172.21.15.97:0/2434561281 >> 172.21.15.97:0/2434561281 conn(0x561b3bda3830 msgr2=0x561b3bd9a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:45.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.532+0000 7f06999fa740 1 -- 172.21.15.97:0/2434561281 shutdown_connections 2024-03-27T10:57:45.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.532+0000 7f06999fa740 1 -- 172.21.15.97:0/2434561281 wait complete. 2024-03-27T10:57:45.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.532+0000 7f06999fa740 20 client.35567 trim_cache size 0 max 16384 2024-03-27T10:57:45.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.546+0000 7f06999fa740 1 Processor -- start 2024-03-27T10:57:45.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.546+0000 7f06999fa740 1 -- start start 2024-03-27T10:57:45.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.546+0000 7f06999fa740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561b3bda82c0 0x561b3bda9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:45.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.546+0000 7f06999fa740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561b3bda8cb0 0x561b3bdb9930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:45.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.546+0000 7f06999fa740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561b3bdade20 0x561b3bdaa430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:45.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.546+0000 7f06999fa740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f06901a6ab0 con 0x561b3bda8cb0 2024-03-27T10:57:45.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.546+0000 7f06999fa740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f06901a85f0 con 0x561b3bdade20 2024-03-27T10:57:45.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.546+0000 7f06999fa740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f06901a7180 con 0x561b3bda82c0 2024-03-27T10:57:45.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.547+0000 7f068f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561b3bda8cb0 0x561b3bdb9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:45.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.547+0000 7f0694c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561b3bdade20 0x561b3bdaa430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:45.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.547+0000 7f068f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561b3bda8cb0 0x561b3bdb9930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53870/0 (socket says 172.21.15.97:53870) 2024-03-27T10:57:45.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.547+0000 7f068f400640 1 -- 172.21.15.97:0/1861313978 learned_addr learned my addr 172.21.15.97:0/1861313978 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:45.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.547+0000 7f068fe00640 1 --2- 172.21.15.97:0/1861313978 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561b3bda82c0 0x561b3bda9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:45.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.547+0000 7f0694c00640 1 -- 172.21.15.97:0/1861313978 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561b3bda82c0 msgr2=0x561b3bda9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:45.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.547+0000 7f0694c00640 1 --2- 172.21.15.97:0/1861313978 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561b3bda82c0 0x561b3bda9ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:45.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.547+0000 7f0694c00640 1 -- 172.21.15.97:0/1861313978 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561b3bda8cb0 msgr2=0x561b3bdb9930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:45.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.547+0000 7f0694c00640 1 --2- 172.21.15.97:0/1861313978 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561b3bda8cb0 0x561b3bdb9930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:45.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.547+0000 7f0694c00640 1 -- 172.21.15.97:0/1861313978 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b3bda4280 con 0x561b3bdade20 2024-03-27T10:57:45.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.548+0000 7f068f400640 1 --2- 172.21.15.97:0/1861313978 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561b3bda8cb0 0x561b3bdb9930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T10:57:45.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.548+0000 7f0694c00640 1 --2- 172.21.15.97:0/1861313978 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561b3bdade20 0x561b3bdaa430 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f06880baf70 tx=0x7f06880bcab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76947b3b41fcd394 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:45.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.548+0000 7f067be00640 1 -- 172.21.15.97:0/1861313978 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06880d7070 con 0x561b3bdade20 2024-03-27T10:57:45.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.548+0000 7f067be00640 1 -- 172.21.15.97:0/1861313978 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f06880d2e50 con 0x561b3bdade20 2024-03-27T10:57:45.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.548+0000 7f067be00640 1 -- 172.21.15.97:0/1861313978 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0688004da0 con 0x561b3bdade20 2024-03-27T10:57:45.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.549+0000 7f06999fa740 1 -- 172.21.15.97:0/1861313978 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561b3bdade20 msgr2=0x561b3bdaa430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:45.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.549+0000 7f06999fa740 1 --2- 172.21.15.97:0/1861313978 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561b3bdade20 0x561b3bdaa430 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f06880baf70 tx=0x7f06880bcab0 comp rx=0 tx=0).stop 2024-03-27T10:57:45.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.549+0000 7f06999fa740 1 -- 172.21.15.97:0/1861313978 shutdown_connections 2024-03-27T10:57:45.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.549+0000 7f06999fa740 1 --2- 172.21.15.97:0/1861313978 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561b3bdade20 0x561b3bdaa430 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:45.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.549+0000 7f06999fa740 1 --2- 172.21.15.97:0/1861313978 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561b3bda8cb0 0x561b3bdb9930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:45.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.549+0000 7f06999fa740 1 --2- 172.21.15.97:0/1861313978 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561b3bda82c0 0x561b3bda9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:45.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.549+0000 7f06999fa740 1 -- 172.21.15.97:0/1861313978 >> 172.21.15.97:0/1861313978 conn(0x561b3bda3830 msgr2=0x561b3bdb23f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:45.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.549+0000 7f06999fa740 1 -- 172.21.15.97:0/1861313978 shutdown_connections 2024-03-27T10:57:45.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.549+0000 7f06999fa740 1 -- 172.21.15.97:0/1861313978 wait complete. 2024-03-27T10:57:45.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.550+0000 7f06999fa740 1 -- start start 2024-03-27T10:57:45.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.550+0000 7f06999fa740 1 Processor -- start 2024-03-27T10:57:45.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.550+0000 7f06999fa740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561b3bda82c0 0x561b3bdb6e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:45.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.550+0000 7f06999fa740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561b3bda8cb0 0x561b3bdb73b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:45.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.550+0000 7f06999fa740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561b3bdade20 0x561b3bdb83c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:45.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.550+0000 7f06999fa740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f06901a7f20 con 0x561b3bda8cb0 2024-03-27T10:57:45.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.550+0000 7f06999fa740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f06901a7850 con 0x561b3bda82c0 2024-03-27T10:57:45.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.550+0000 7f06999fa740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f06901a4a90 con 0x561b3bdade20 2024-03-27T10:57:45.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.550+0000 7f06999fa740 10 client.0 _ll_register_callbacks cb 0x7f0688005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:57:45.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.551+0000 7f068f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561b3bda8cb0 0x561b3bdb73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:45.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.551+0000 7f068f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561b3bda8cb0 0x561b3bdb73b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53872/0 (socket says 172.21.15.97:53872) 2024-03-27T10:57:45.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.551+0000 7f068f400640 1 -- 172.21.15.97:0/3214258535 learned_addr learned my addr 172.21.15.97:0/3214258535 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:45.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.551+0000 7f068fe00640 1 --2- 172.21.15.97:0/3214258535 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561b3bda82c0 0x561b3bdb6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:45.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.551+0000 7f0694c00640 1 --2- 172.21.15.97:0/3214258535 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561b3bdade20 0x561b3bdb83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:45.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.551+0000 7f068f400640 1 -- 172.21.15.97:0/3214258535 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561b3bdade20 msgr2=0x561b3bdb83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:45.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.551+0000 7f068f400640 1 --2- 172.21.15.97:0/3214258535 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561b3bdade20 0x561b3bdb83c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:45.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.551+0000 7f068f400640 1 -- 172.21.15.97:0/3214258535 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561b3bda82c0 msgr2=0x561b3bdb6e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:45.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.551+0000 7f068f400640 1 --2- 172.21.15.97:0/3214258535 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561b3bda82c0 0x561b3bdb6e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:45.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.551+0000 7f068f400640 1 -- 172.21.15.97:0/3214258535 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b3bdb89b0 con 0x561b3bda8cb0 2024-03-27T10:57:45.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.552+0000 7f068f400640 1 --2- 172.21.15.97:0/3214258535 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561b3bda8cb0 0x561b3bdb73b0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f067c003f80 tx=0x7f067c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7151d0f29f943a14 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:45.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.552+0000 7f0679600640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T10:57:45.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.552+0000 7f0679600640 1 -- 172.21.15.97:0/3214258535 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f067c02c060 con 0x561b3bda8cb0 2024-03-27T10:57:45.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.552+0000 7f0679600640 1 -- 172.21.15.97:0/3214258535 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f067c01b650 con 0x561b3bda8cb0 2024-03-27T10:57:45.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.552+0000 7f06999fa740 1 -- 172.21.15.97:0/3214258535 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06880d2e50 con 0x561b3bda8cb0 2024-03-27T10:57:45.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.552+0000 7f0679600640 1 -- 172.21.15.97:0/3214258535 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f067c01b850 con 0x561b3bda8cb0 2024-03-27T10:57:45.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.552+0000 7f06999fa740 10 client.15909 fetch_fsmap learned FSMap version 28 2024-03-27T10:57:45.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.552+0000 7f06999fa740 1 -- 172.21.15.97:0/3214258535 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561b3be24770 con 0x561b3bda8cb0 2024-03-27T10:57:45.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.553+0000 7f0679600640 1 -- 172.21.15.97:0/3214258535 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f06880d2e50 con 0x561b3bda8cb0 2024-03-27T10:57:45.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.553+0000 7f0679600640 1 -- 172.21.15.97:0/3214258535 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f067c026070 con 0x561b3bda8cb0 2024-03-27T10:57:45.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.553+0000 7f06999fa740 10 client.15909 fetch_fsmap finished waiting for FSMap version 28 2024-03-27T10:57:45.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.553+0000 7f06999fa740 10 client.15909 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-03-27T10:57:45.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.553+0000 7f06999fa740 20 client.15909 populate_metadata read hostname 'smithi097' 2024-03-27T10:57:45.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.553+0000 7f06999fa740 1 --2- 172.21.15.97:0/3214258535 >> [v2:172.21.15.177:6832/3787981632,v1:172.21.15.177:6833/3787981632] conn(0x561b3be250e0 0x561b3be27500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:45.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.553+0000 7f06999fa740 4 client.15909 mds_command: new command op to 24400 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-27T10:57:45.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.553+0000 7f06999fa740 1 -- 172.21.15.97:0/3214258535 --> [v2:172.21.15.177:6832/3787981632,v1:172.21.15.177:6833/3787981632] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561b3bd0bf00 con 0x561b3be250e0 2024-03-27T10:57:45.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.554+0000 7f068fe00640 1 --2- 172.21.15.97:0/3214258535 >> [v2:172.21.15.177:6832/3787981632,v1:172.21.15.177:6833/3787981632] conn(0x561b3be250e0 0x561b3be27500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:45.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.555+0000 7f0679600640 10 client.15909 ms_handle_connect on v2:172.21.15.177:6832/3787981632 2024-03-27T10:57:45.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.555+0000 7f068fe00640 1 --2- 172.21.15.97:0/3214258535 >> [v2:172.21.15.177:6832/3787981632,v1:172.21.15.177:6833/3787981632] conn(0x561b3be250e0 0x561b3be27500 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7a19e88d0d7f253f server_cookie=f8bd5482c8fd4826 in_seq=0 out_seq=0 2024-03-27T10:57:45.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.555+0000 7f0679600640 1 -- 172.21.15.97:0/3214258535 <== mds.4 v2:172.21.15.177:6832/3787981632 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561b3bd0bf00 con 0x561b3be250e0 2024-03-27T10:57:45.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.555+0000 7f0679600640 10 client.15909 handle_command_reply: tid=0 2024-03-27T10:57:45.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.555+0000 7f06999fa740 1 client.15909 shutdown 2024-03-27T10:57:45.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.555+0000 7f06999fa740 1 -- 172.21.15.97:0/3214258535 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561b3bda8cb0 msgr2=0x561b3bdb73b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:45.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.555+0000 7f06999fa740 1 --2- 172.21.15.97:0/3214258535 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561b3bda8cb0 0x561b3bdb73b0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f067c003f80 tx=0x7f067c0046c0 comp rx=0 tx=0).stop 2024-03-27T10:57:45.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.555+0000 7f06999fa740 1 -- 172.21.15.97:0/3214258535 shutdown_connections 2024-03-27T10:57:45.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.555+0000 7f06999fa740 1 --2- 172.21.15.97:0/3214258535 >> [v2:172.21.15.177:6832/3787981632,v1:172.21.15.177:6833/3787981632] conn(0x561b3be250e0 0x561b3be27500 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:45.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.555+0000 7f06999fa740 1 --2- 172.21.15.97:0/3214258535 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561b3bdade20 0x561b3bdb83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:45.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.555+0000 7f06999fa740 1 --2- 172.21.15.97:0/3214258535 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561b3bda8cb0 0x561b3bdb73b0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:45.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.555+0000 7f06999fa740 1 --2- 172.21.15.97:0/3214258535 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561b3bda82c0 0x561b3bdb6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:45.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.555+0000 7f0679600640 0 client.15909 ms_handle_reset on v2:172.21.15.177:6832/3787981632 2024-03-27T10:57:45.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.555+0000 7f06999fa740 1 -- 172.21.15.97:0/3214258535 >> 172.21.15.97:0/3214258535 conn(0x561b3bda3830 msgr2=0x561b3bdb23f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:45.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.556+0000 7f06999fa740 1 -- 172.21.15.97:0/3214258535 shutdown_connections 2024-03-27T10:57:45.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.556+0000 7f06999fa740 1 -- 172.21.15.97:0/3214258535 wait complete. 2024-03-27T10:57:45.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.556+0000 7f06999fa740 20 client.15909 trim_cache size 0 max 16384 2024-03-27T10:57:45.529 INFO:teuthology.orchestra.run.smithi097.stdout:[] 2024-03-27T10:57:45.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.556+0000 7f0695600640 1 -- 172.21.15.97:0/1194109221 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f06740a3610 msgr2=0x7f06740a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:45.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.556+0000 7f0695600640 1 --2- 172.21.15.97:0/1194109221 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f06740a3610 0x7f06740a5ad0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f067c004650 tx=0x7f067c004170 comp rx=0 tx=0).stop 2024-03-27T10:57:45.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.556+0000 7f0695600640 1 -- 172.21.15.97:0/1194109221 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0690105800 msgr2=0x7f06901a13a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:45.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.556+0000 7f0695600640 1 --2- 172.21.15.97:0/1194109221 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0690105800 0x7f06901a13a0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f068802c700 tx=0x7f06880328a0 comp rx=0 tx=0).stop 2024-03-27T10:57:45.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.557+0000 7f0695600640 1 -- 172.21.15.97:0/1194109221 shutdown_connections 2024-03-27T10:57:45.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.557+0000 7f0695600640 1 --2- 172.21.15.97:0/1194109221 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0690105800 0x7f06901a13a0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:45.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.557+0000 7f0695600640 1 --2- 172.21.15.97:0/1194109221 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0690104e40 0x7f069019a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:45.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.557+0000 7f0695600640 1 --2- 172.21.15.97:0/1194109221 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f06740a3610 0x7f06740a5ad0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:45.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.557+0000 7f0695600640 1 --2- 172.21.15.97:0/1194109221 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0690103c50 0x7f0690199df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:45.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.557+0000 7f0695600640 1 -- 172.21.15.97:0/1194109221 >> 172.21.15.97:0/1194109221 conn(0x7f06900ff1c0 msgr2=0x7f0690108190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:45.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.557+0000 7f0695600640 1 -- 172.21.15.97:0/1194109221 shutdown_connections 2024-03-27T10:57:45.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.557+0000 7f0695600640 1 -- 172.21.15.97:0/1194109221 wait complete. 2024-03-27T10:57:45.548 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:57:45 smithi097 ceph-mon[31859]: pgmap v474: 97 pgs: 97 active+clean; 665 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 511 B/s wr, 11 op/s 2024-03-27T10:57:45.548 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:57:45 smithi097 ceph-mon[31859]: scrub queued for path: / 2024-03-27T10:57:45.548 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:57:45 smithi097 ceph-mon[31859]: scrub summary: idle+waiting paths [/] 2024-03-27T10:57:45.548 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:57:45 smithi097 ceph-mon[31859]: scrub summary: active paths [/] 2024-03-27T10:57:45.548 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:57:45 smithi097 ceph-mon[31859]: scrub summary: idle+waiting paths [/] 2024-03-27T10:57:45.548 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:57:45 smithi097 ceph-mon[31859]: scrub summary: idle 2024-03-27T10:57:45.548 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:57:45 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/4242056364' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T10:57:45.551 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-27T10:57:45.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:57:45 smithi177 ceph-mon[35294]: pgmap v474: 97 pgs: 97 active+clean; 665 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 511 B/s wr, 11 op/s 2024-03-27T10:57:45.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:57:45 smithi177 ceph-mon[35294]: scrub queued for path: / 2024-03-27T10:57:45.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:57:45 smithi177 ceph-mon[35294]: scrub summary: idle+waiting paths [/] 2024-03-27T10:57:45.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:57:45 smithi177 ceph-mon[35294]: scrub summary: active paths [/] 2024-03-27T10:57:45.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:57:45 smithi177 ceph-mon[35294]: scrub summary: idle+waiting paths [/] 2024-03-27T10:57:45.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:57:45 smithi177 ceph-mon[35294]: scrub summary: idle 2024-03-27T10:57:45.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:57:45 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/4242056364' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T10:57:45.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.756+0000 7fc3f2200640 1 -- 172.21.15.97:0/1056983512 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3ec0745b0 msgr2=0x7fc3ec074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:45.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.756+0000 7fc3f2200640 1 --2- 172.21.15.97:0/1056983512 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3ec0745b0 0x7fc3ec074a30 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fc3d8009930 tx=0x7fc3d802bc30 comp rx=0 tx=0).stop 2024-03-27T10:57:45.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.756+0000 7fc3f2200640 1 -- 172.21.15.97:0/1056983512 shutdown_connections 2024-03-27T10:57:45.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.756+0000 7fc3f2200640 1 --2- 172.21.15.97:0/1056983512 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc3ec10e020 0x7fc3ec110410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:45.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.756+0000 7fc3f2200640 1 --2- 172.21.15.97:0/1056983512 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3ec0745b0 0x7fc3ec074a30 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:45.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.756+0000 7fc3f2200640 1 --2- 172.21.15.97:0/1056983512 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc3ec075b90 0x7fc3ec073fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:45.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.756+0000 7fc3f2200640 1 -- 172.21.15.97:0/1056983512 >> 172.21.15.97:0/1056983512 conn(0x7fc3ec0fcfc0 msgr2=0x7fc3ec0ff400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:45.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.757+0000 7fc3f2200640 1 -- 172.21.15.97:0/1056983512 shutdown_connections 2024-03-27T10:57:45.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.757+0000 7fc3f2200640 1 -- 172.21.15.97:0/1056983512 wait complete. 2024-03-27T10:57:45.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.758+0000 7fc3f2200640 1 Processor -- start 2024-03-27T10:57:45.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.758+0000 7fc3f2200640 1 -- start start 2024-03-27T10:57:45.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.759+0000 7fc3f2200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3ec075b90 0x7fc3ec19e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:45.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.759+0000 7fc3f2200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc3ec10e020 0x7fc3ec19ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:45.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.759+0000 7fc3f2200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc3ec1a3a80 0x7fc3ec1a5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:45.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.759+0000 7fc3f2200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ec113ba0 con 0x7fc3ec075b90 2024-03-27T10:57:45.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.759+0000 7fc3f2200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ec113a20 con 0x7fc3ec10e020 2024-03-27T10:57:45.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.759+0000 7fc3f2200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ec113d20 con 0x7fc3ec1a3a80 2024-03-27T10:57:45.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.759+0000 7fc3f0e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3ec075b90 0x7fc3ec19e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:45.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.759+0000 7fc3f0e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3ec075b90 0x7fc3ec19e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53896/0 (socket says 172.21.15.97:53896) 2024-03-27T10:57:45.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.759+0000 7fc3f0e00640 1 -- 172.21.15.97:0/2972101156 learned_addr learned my addr 172.21.15.97:0/2972101156 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:45.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.759+0000 7fc3ebe00640 1 --2- 172.21.15.97:0/2972101156 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc3ec10e020 0x7fc3ec19ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:45.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.759+0000 7fc3f1800640 1 --2- 172.21.15.97:0/2972101156 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc3ec1a3a80 0x7fc3ec1a5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:45.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.760+0000 7fc3f0e00640 1 -- 172.21.15.97:0/2972101156 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc3ec1a3a80 msgr2=0x7fc3ec1a5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:45.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.760+0000 7fc3f0e00640 1 --2- 172.21.15.97:0/2972101156 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc3ec1a3a80 0x7fc3ec1a5e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:45.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.760+0000 7fc3f0e00640 1 -- 172.21.15.97:0/2972101156 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc3ec10e020 msgr2=0x7fc3ec19ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:45.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.760+0000 7fc3f0e00640 1 --2- 172.21.15.97:0/2972101156 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc3ec10e020 0x7fc3ec19ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:45.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.760+0000 7fc3f0e00640 1 -- 172.21.15.97:0/2972101156 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3ec1a64e0 con 0x7fc3ec075b90 2024-03-27T10:57:45.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.760+0000 7fc3f1800640 1 --2- 172.21.15.97:0/2972101156 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc3ec1a3a80 0x7fc3ec1a5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T10:57:45.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.760+0000 7fc3f0e00640 1 --2- 172.21.15.97:0/2972101156 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3ec075b90 0x7fc3ec19e4e0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fc3e000e9f0 tx=0x7fc3e000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:45.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.761+0000 7fc3e9600640 1 -- 172.21.15.97:0/2972101156 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e000cd80 con 0x7fc3ec075b90 2024-03-27T10:57:45.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.761+0000 7fc3e9600640 1 -- 172.21.15.97:0/2972101156 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc3e00108e0 con 0x7fc3ec075b90 2024-03-27T10:57:45.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.761+0000 7fc3e9600640 1 -- 172.21.15.97:0/2972101156 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e0018920 con 0x7fc3ec075b90 2024-03-27T10:57:45.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.761+0000 7fc3f2200640 1 -- 172.21.15.97:0/2972101156 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3ec1a67d0 con 0x7fc3ec075b90 2024-03-27T10:57:45.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.761+0000 7fc3f2200640 1 -- 172.21.15.97:0/2972101156 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3ec1a6ca0 con 0x7fc3ec075b90 2024-03-27T10:57:45.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.765+0000 7fc3f6579740 1 Processor -- start 2024-03-27T10:57:45.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.765+0000 7fc3f6579740 1 -- start start 2024-03-27T10:57:45.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.765+0000 7fc3f6579740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e4aff86ab0 0x55e4aff86ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:45.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.765+0000 7fc3f6579740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e4aff874a0 0x55e4aff8bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:45.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.765+0000 7fc3f6579740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e4aff8c610 0x55e4aff8eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:45.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.765+0000 7fc3f6579740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55e4aff4e7b0 con 0x55e4aff86ab0 2024-03-27T10:57:45.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.765+0000 7fc3f6579740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55e4afeeb670 con 0x55e4aff8c610 2024-03-27T10:57:45.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.765+0000 7fc3f6579740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55e4afe93970 con 0x55e4aff874a0 2024-03-27T10:57:45.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.766+0000 7fc3f1800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e4aff86ab0 0x55e4aff86ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:45.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.766+0000 7fc3f1800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e4aff86ab0 0x55e4aff86ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53902/0 (socket says 172.21.15.97:53902) 2024-03-27T10:57:45.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.766+0000 7fc3f1800640 1 -- 172.21.15.97:0/979009532 learned_addr learned my addr 172.21.15.97:0/979009532 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:45.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.766+0000 7fc3ebe00640 1 --2- 172.21.15.97:0/979009532 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e4aff8c610 0x55e4aff8eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:45.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.766+0000 7fc3ebe00640 1 -- 172.21.15.97:0/979009532 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e4aff874a0 msgr2=0x55e4aff8bfa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-27T10:57:45.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.766+0000 7fc3ebe00640 1 --2- 172.21.15.97:0/979009532 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e4aff874a0 0x55e4aff8bfa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:45.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.766+0000 7fc3ebe00640 1 -- 172.21.15.97:0/979009532 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e4aff86ab0 msgr2=0x55e4aff86ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:45.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.766+0000 7fc3ebe00640 1 --2- 172.21.15.97:0/979009532 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e4aff86ab0 0x55e4aff86ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:45.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.766+0000 7fc3ebe00640 1 -- 172.21.15.97:0/979009532 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4afbff3f0 con 0x55e4aff8c610 2024-03-27T10:57:45.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.766+0000 7fc3f1800640 1 --2- 172.21.15.97:0/979009532 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e4aff86ab0 0x55e4aff86ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T10:57:45.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.767+0000 7fc3ebe00640 1 --2- 172.21.15.97:0/979009532 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e4aff8c610 0x55e4aff8eaa0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc3d8008000 tx=0x7fc3d8005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb66d4de4aab37d4 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:45.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.767+0000 7fc3e9600640 1 -- 172.21.15.97:0/2972101156 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7fc3e0010a90 con 0x7fc3ec075b90 2024-03-27T10:57:45.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.768+0000 7fc3de000640 1 -- 172.21.15.97:0/979009532 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d8055070 con 0x55e4aff8c610 2024-03-27T10:57:45.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.768+0000 7fc3de000640 1 -- 172.21.15.97:0/979009532 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc3d8004040 con 0x55e4aff8c610 2024-03-27T10:57:45.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.768+0000 7fc3de000640 1 -- 172.21.15.97:0/979009532 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d8005610 con 0x55e4aff8c610 2024-03-27T10:57:45.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.768+0000 7fc3f6579740 1 -- 172.21.15.97:0/979009532 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e4aff8c610 msgr2=0x55e4aff8eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:45.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.768+0000 7fc3f6579740 1 --2- 172.21.15.97:0/979009532 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e4aff8c610 0x55e4aff8eaa0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc3d8008000 tx=0x7fc3d8005d50 comp rx=0 tx=0).stop 2024-03-27T10:57:45.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.769+0000 7fc3f6579740 1 -- 172.21.15.97:0/979009532 shutdown_connections 2024-03-27T10:57:45.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.769+0000 7fc3f6579740 1 --2- 172.21.15.97:0/979009532 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e4aff8c610 0x55e4aff8eaa0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:45.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.769+0000 7fc3f6579740 1 --2- 172.21.15.97:0/979009532 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e4aff874a0 0x55e4aff8bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:45.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.769+0000 7fc3f6579740 1 --2- 172.21.15.97:0/979009532 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e4aff86ab0 0x55e4aff86ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:45.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.769+0000 7fc3f6579740 1 -- 172.21.15.97:0/979009532 >> 172.21.15.97:0/979009532 conn(0x55e4aff82020 msgr2=0x55e4aff82420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:45.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.769+0000 7fc3f6579740 1 -- 172.21.15.97:0/979009532 shutdown_connections 2024-03-27T10:57:45.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.769+0000 7fc3f6579740 1 -- 172.21.15.97:0/979009532 wait complete. 2024-03-27T10:57:45.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.769+0000 7fc3e9600640 1 --2- 172.21.15.97:0/2972101156 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fc3d40a3670 0x7fc3d40a5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:45.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.769+0000 7fc3e9600640 1 -- 172.21.15.97:0/2972101156 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fc3e00d0010 con 0x7fc3ec075b90 2024-03-27T10:57:45.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.769+0000 7fc3f6579740 1 -- start start 2024-03-27T10:57:45.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.770+0000 7fc3ebe00640 1 --2- 172.21.15.97:0/2972101156 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fc3d40a3670 0x7fc3d40a5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:45.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.770+0000 7fc3f6579740 1 Processor -- start 2024-03-27T10:57:45.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.770+0000 7fc3f6579740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e4aff86ab0 0x55e4aff90be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:45.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.770+0000 7fc3f6579740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e4aff874a0 0x55e4aff91120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:45.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.770+0000 7fc3f6579740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e4aff8c610 0x55e4aff98160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:45.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.770+0000 7fc3f6579740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55e4afea6ca0 con 0x55e4aff8c610 2024-03-27T10:57:45.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.770+0000 7fc3f6579740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55e4afee7620 con 0x55e4aff874a0 2024-03-27T10:57:45.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.770+0000 7fc3f6579740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55e4aff8f560 con 0x55e4aff86ab0 2024-03-27T10:57:45.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.770+0000 7fc3f6579740 10 client.0 _ll_register_callbacks cb 0x55e4afee3160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:57:45.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.771+0000 7fc3ebe00640 1 --2- 172.21.15.97:0/2972101156 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fc3d40a3670 0x7fc3d40a5b30 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc3d8008000 tx=0x7fc3d8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:45.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.771+0000 7fc3ebe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e4aff874a0 0x55e4aff91120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:45.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.771+0000 7fc3ebe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e4aff874a0 0x55e4aff91120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:34230/0 (socket says 172.21.15.97:34230) 2024-03-27T10:57:45.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.771+0000 7fc3ebe00640 1 -- 172.21.15.97:0/3322608494 learned_addr learned my addr 172.21.15.97:0/3322608494 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:45.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.771+0000 7fc3f0e00640 1 --2- 172.21.15.97:0/3322608494 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e4aff86ab0 0x55e4aff90be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:45.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.771+0000 7fc3f1800640 1 --2- 172.21.15.97:0/3322608494 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e4aff8c610 0x55e4aff98160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:45.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.771+0000 7fc3ebe00640 1 -- 172.21.15.97:0/3322608494 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e4aff86ab0 msgr2=0x55e4aff90be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:45.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.771+0000 7fc3ebe00640 1 --2- 172.21.15.97:0/3322608494 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e4aff86ab0 0x55e4aff90be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:45.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.771+0000 7fc3ebe00640 1 -- 172.21.15.97:0/3322608494 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e4aff8c610 msgr2=0x55e4aff98160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:45.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.771+0000 7fc3ebe00640 1 --2- 172.21.15.97:0/3322608494 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e4aff8c610 0x55e4aff98160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:45.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.771+0000 7fc3ebe00640 1 -- 172.21.15.97:0/3322608494 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4afcc97d0 con 0x55e4aff874a0 2024-03-27T10:57:45.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.771+0000 7fc3f1800640 1 --2- 172.21.15.97:0/3322608494 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e4aff8c610 0x55e4aff98160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T10:57:45.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.772+0000 7fc3ebe00640 1 --2- 172.21.15.97:0/3322608494 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e4aff874a0 0x55e4aff91120 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc3d8005ab0 tx=0x7fc3d802c5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e046f377231ecfc server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:45.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.772+0000 7fc3d2a00640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-03-27T10:57:45.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.772+0000 7fc3d2a00640 1 -- 172.21.15.97:0/3322608494 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d806a030 con 0x55e4aff874a0 2024-03-27T10:57:45.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.772+0000 7fc3f6579740 1 -- 172.21.15.97:0/3322608494 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3d8004040 con 0x55e4aff874a0 2024-03-27T10:57:45.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.772+0000 7fc3d2a00640 1 -- 172.21.15.97:0/3322608494 <== mon.2 v2:172.21.15.114:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fc3d8003d80 con 0x55e4aff874a0 2024-03-27T10:57:45.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.773+0000 7fc3d2a00640 1 -- 172.21.15.97:0/3322608494 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc3d8055040 con 0x55e4aff874a0 2024-03-27T10:57:45.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.773+0000 7fc3f6579740 10 client.35585 fetch_fsmap learned FSMap version 28 2024-03-27T10:57:45.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.773+0000 7fc3f6579740 1 -- 172.21.15.97:0/3322608494 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e4aff9b4b0 con 0x55e4aff874a0 2024-03-27T10:57:45.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.774+0000 7fc3d2a00640 1 -- 172.21.15.97:0/3322608494 <== mon.2 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc3d8004040 con 0x55e4aff874a0 2024-03-27T10:57:45.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.774+0000 7fc3d2a00640 1 -- 172.21.15.97:0/3322608494 <== mon.2 v2:172.21.15.114:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fc3d80714d0 con 0x55e4aff874a0 2024-03-27T10:57:45.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.774+0000 7fc3f6579740 10 client.35585 fetch_fsmap finished waiting for FSMap version 28 2024-03-27T10:57:45.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.774+0000 7fc3f6579740 10 client.35585 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-03-27T10:57:45.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.774+0000 7fc3f6579740 20 client.35585 populate_metadata read hostname 'smithi097' 2024-03-27T10:57:45.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.774+0000 7fc3f6579740 1 --2- 172.21.15.97:0/3322608494 >> [v2:172.21.15.177:6834/3349112132,v1:172.21.15.177:6835/3349112132] conn(0x55e4aff9c850 0x55e4aff9ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:45.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.774+0000 7fc3f6579740 4 client.35585 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T10:57:45.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.774+0000 7fc3f6579740 1 -- 172.21.15.97:0/3322608494 --> [v2:172.21.15.177:6834/3349112132,v1:172.21.15.177:6835/3349112132] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e4afef8bb0 con 0x55e4aff9c850 2024-03-27T10:57:45.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.775+0000 7fc3f0e00640 1 --2- 172.21.15.97:0/3322608494 >> [v2:172.21.15.177:6834/3349112132,v1:172.21.15.177:6835/3349112132] conn(0x55e4aff9c850 0x55e4aff9ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:45.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.775+0000 7fc3f0e00640 1 --2- 172.21.15.97:0/3322608494 >> [v2:172.21.15.177:6834/3349112132,v1:172.21.15.177:6835/3349112132] conn(0x55e4aff9c850 0x55e4aff9ec80 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c6771ac8d125b385 server_cookie=ab2894ac3dd1ba79 in_seq=0 out_seq=0 2024-03-27T10:57:45.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.775+0000 7fc3d2a00640 10 client.35585 ms_handle_connect on v2:172.21.15.177:6834/3349112132 2024-03-27T10:57:45.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.778+0000 7fc3d2a00640 1 -- 172.21.15.97:0/3322608494 <== mds.3 v2:172.21.15.177:6834/3349112132 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55e4afef8bb0 con 0x55e4aff9c850 2024-03-27T10:57:45.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.778+0000 7fc3d2a00640 10 client.35585 handle_command_reply: tid=0 2024-03-27T10:57:45.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.778+0000 7fc3f6579740 1 client.35585 shutdown 2024-03-27T10:57:45.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.778+0000 7fc3f6579740 1 -- 172.21.15.97:0/3322608494 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e4aff874a0 msgr2=0x55e4aff91120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:45.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.778+0000 7fc3f6579740 1 --2- 172.21.15.97:0/3322608494 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e4aff874a0 0x55e4aff91120 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc3d8005ab0 tx=0x7fc3d802c5f0 comp rx=0 tx=0).stop 2024-03-27T10:57:45.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.778+0000 7fc3f6579740 1 -- 172.21.15.97:0/3322608494 shutdown_connections 2024-03-27T10:57:45.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.778+0000 7fc3f6579740 1 --2- 172.21.15.97:0/3322608494 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e4aff8c610 0x55e4aff98160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:45.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.778+0000 7fc3f6579740 1 --2- 172.21.15.97:0/3322608494 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e4aff874a0 0x55e4aff91120 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:45.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.778+0000 7fc3f6579740 1 --2- 172.21.15.97:0/3322608494 >> [v2:172.21.15.177:6834/3349112132,v1:172.21.15.177:6835/3349112132] conn(0x55e4aff9c850 0x55e4aff9ec80 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:45.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.778+0000 7fc3f6579740 1 --2- 172.21.15.97:0/3322608494 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e4aff86ab0 0x55e4aff90be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:45.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.778+0000 7fc3f6579740 1 -- 172.21.15.97:0/3322608494 >> 172.21.15.97:0/3322608494 conn(0x55e4aff82020 msgr2=0x55e4aff81240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:45.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.778+0000 7fc3d2a00640 0 client.35585 ms_handle_reset on v2:172.21.15.177:6834/3349112132 2024-03-27T10:57:45.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.778+0000 7fc3f6579740 1 -- 172.21.15.97:0/3322608494 shutdown_connections 2024-03-27T10:57:45.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.779+0000 7fc3f6579740 1 -- 172.21.15.97:0/3322608494 wait complete. 2024-03-27T10:57:45.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.779+0000 7fc3f6579740 20 client.35585 trim_cache size 0 max 16384 2024-03-27T10:57:45.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.792+0000 7fc3f6579740 1 Processor -- start 2024-03-27T10:57:45.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.793+0000 7fc3f6579740 1 -- start start 2024-03-27T10:57:45.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.793+0000 7fc3f6579740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e4aff86ab0 0x55e4aff887c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:45.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.793+0000 7fc3f6579740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e4aff874a0 0x55e4aff88d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:45.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.793+0000 7fc3f6579740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e4aff8c610 0x55e4aff98120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:45.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.793+0000 7fc3f6579740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ec1a70b0 con 0x55e4aff86ab0 2024-03-27T10:57:45.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.793+0000 7fc3f6579740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ec1a8bf0 con 0x55e4aff874a0 2024-03-27T10:57:45.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.793+0000 7fc3f6579740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ec1a7780 con 0x55e4aff8c610 2024-03-27T10:57:45.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.793+0000 7fc3f0e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e4aff86ab0 0x55e4aff887c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:45.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.793+0000 7fc3f0e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e4aff86ab0 0x55e4aff887c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53912/0 (socket says 172.21.15.97:53912) 2024-03-27T10:57:45.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.793+0000 7fc3f0e00640 1 -- 172.21.15.97:0/269752909 learned_addr learned my addr 172.21.15.97:0/269752909 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:45.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.793+0000 7fc3ebe00640 1 --2- 172.21.15.97:0/269752909 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e4aff874a0 0x55e4aff88d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:45.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.793+0000 7fc3f1800640 1 --2- 172.21.15.97:0/269752909 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e4aff8c610 0x55e4aff98120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:45.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.793+0000 7fc3f0e00640 1 -- 172.21.15.97:0/269752909 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e4aff8c610 msgr2=0x55e4aff98120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:45.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.793+0000 7fc3f0e00640 1 --2- 172.21.15.97:0/269752909 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e4aff8c610 0x55e4aff98120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:45.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.794+0000 7fc3f0e00640 1 -- 172.21.15.97:0/269752909 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e4aff874a0 msgr2=0x55e4aff88d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:45.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.794+0000 7fc3f0e00640 1 --2- 172.21.15.97:0/269752909 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e4aff874a0 0x55e4aff88d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:45.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.794+0000 7fc3f0e00640 1 -- 172.21.15.97:0/269752909 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4aff9eed0 con 0x55e4aff86ab0 2024-03-27T10:57:45.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.794+0000 7fc3f1800640 1 --2- 172.21.15.97:0/269752909 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e4aff8c610 0x55e4aff98120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:57:45.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.794+0000 7fc3f0e00640 1 --2- 172.21.15.97:0/269752909 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e4aff86ab0 0x55e4aff887c0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fc3e009c190 tx=0x7fc3e001f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40259e354ad622b7 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:45.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.794+0000 7fc3dd600640 1 -- 172.21.15.97:0/269752909 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e00a5100 con 0x55e4aff86ab0 2024-03-27T10:57:45.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.794+0000 7fc3dd600640 1 -- 172.21.15.97:0/269752909 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc3e00cdbd0 con 0x55e4aff86ab0 2024-03-27T10:57:45.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.794+0000 7fc3dd600640 1 -- 172.21.15.97:0/269752909 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e00981e0 con 0x55e4aff86ab0 2024-03-27T10:57:45.767 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:57:45 smithi114 ceph-mon[35338]: pgmap v474: 97 pgs: 97 active+clean; 665 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 511 B/s wr, 11 op/s 2024-03-27T10:57:45.767 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:57:45 smithi114 ceph-mon[35338]: scrub queued for path: / 2024-03-27T10:57:45.767 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:57:45 smithi114 ceph-mon[35338]: scrub summary: idle+waiting paths [/] 2024-03-27T10:57:45.767 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:57:45 smithi114 ceph-mon[35338]: scrub summary: active paths [/] 2024-03-27T10:57:45.767 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:57:45 smithi114 ceph-mon[35338]: scrub summary: idle+waiting paths [/] 2024-03-27T10:57:45.767 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:57:45 smithi114 ceph-mon[35338]: scrub summary: idle 2024-03-27T10:57:45.767 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:57:45 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/4242056364' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T10:57:45.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.795+0000 7fc3f6579740 1 -- 172.21.15.97:0/269752909 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e4aff86ab0 msgr2=0x55e4aff887c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:45.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.795+0000 7fc3f6579740 1 --2- 172.21.15.97:0/269752909 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e4aff86ab0 0x55e4aff887c0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fc3e009c190 tx=0x7fc3e001f000 comp rx=0 tx=0).stop 2024-03-27T10:57:45.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.795+0000 7fc3f6579740 1 -- 172.21.15.97:0/269752909 shutdown_connections 2024-03-27T10:57:45.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.795+0000 7fc3f6579740 1 --2- 172.21.15.97:0/269752909 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e4aff8c610 0x55e4aff98120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:45.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.795+0000 7fc3f6579740 1 --2- 172.21.15.97:0/269752909 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e4aff874a0 0x55e4aff88d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:45.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.795+0000 7fc3f6579740 1 --2- 172.21.15.97:0/269752909 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e4aff86ab0 0x55e4aff887c0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:45.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.795+0000 7fc3f6579740 1 -- 172.21.15.97:0/269752909 >> 172.21.15.97:0/269752909 conn(0x55e4aff82020 msgr2=0x55e4aff90be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:45.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.795+0000 7fc3f6579740 1 -- 172.21.15.97:0/269752909 shutdown_connections 2024-03-27T10:57:45.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.795+0000 7fc3f6579740 1 -- 172.21.15.97:0/269752909 wait complete. 2024-03-27T10:57:45.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.796+0000 7fc3f6579740 1 -- start start 2024-03-27T10:57:45.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.796+0000 7fc3f6579740 1 Processor -- start 2024-03-27T10:57:45.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.796+0000 7fc3f6579740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e4aff86ab0 0x55e4aff95160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:45.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.796+0000 7fc3f6579740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e4aff874a0 0x55e4aff956a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:45.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.796+0000 7fc3f6579740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e4aff8c610 0x55e4b0000b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:45.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.796+0000 7fc3f6579740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ec1a8520 con 0x55e4aff874a0 2024-03-27T10:57:45.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.796+0000 7fc3f6579740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ec1a7e50 con 0x55e4aff86ab0 2024-03-27T10:57:45.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.796+0000 7fc3f6579740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ec071e40 con 0x55e4aff8c610 2024-03-27T10:57:45.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.796+0000 7fc3f6579740 10 client.0 _ll_register_callbacks cb 0x7fc3d805e070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:57:45.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.797+0000 7fc3ebe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e4aff874a0 0x55e4aff956a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:45.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.797+0000 7fc3f0e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e4aff86ab0 0x55e4aff95160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:45.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.797+0000 7fc3ebe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e4aff874a0 0x55e4aff956a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53928/0 (socket says 172.21.15.97:53928) 2024-03-27T10:57:45.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.797+0000 7fc3ebe00640 1 -- 172.21.15.97:0/4069746280 learned_addr learned my addr 172.21.15.97:0/4069746280 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:45.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.797+0000 7fc3f1800640 1 --2- 172.21.15.97:0/4069746280 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e4aff8c610 0x55e4b0000b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:45.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.797+0000 7fc3ebe00640 1 -- 172.21.15.97:0/4069746280 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e4aff8c610 msgr2=0x55e4b0000b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:45.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.797+0000 7fc3ebe00640 1 --2- 172.21.15.97:0/4069746280 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e4aff8c610 0x55e4b0000b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:45.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.797+0000 7fc3ebe00640 1 -- 172.21.15.97:0/4069746280 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e4aff86ab0 msgr2=0x55e4aff95160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:45.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.797+0000 7fc3ebe00640 1 --2- 172.21.15.97:0/4069746280 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e4aff86ab0 0x55e4aff95160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:45.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.797+0000 7fc3ebe00640 1 -- 172.21.15.97:0/4069746280 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4b0001130 con 0x55e4aff874a0 2024-03-27T10:57:45.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.797+0000 7fc3ebe00640 1 --2- 172.21.15.97:0/4069746280 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e4aff874a0 0x55e4aff956a0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fc3d8071ab0 tx=0x7fc3d8005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc0f27921da435f1 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:45.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.797+0000 7fc3d2a00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T10:57:45.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.798+0000 7fc3d2a00640 1 -- 172.21.15.97:0/4069746280 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d806a070 con 0x55e4aff874a0 2024-03-27T10:57:45.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.798+0000 7fc3f6579740 1 -- 172.21.15.97:0/4069746280 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3e00cdbd0 con 0x55e4aff874a0 2024-03-27T10:57:45.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.798+0000 7fc3d2a00640 1 -- 172.21.15.97:0/4069746280 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fc3d8003f90 con 0x55e4aff874a0 2024-03-27T10:57:45.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.798+0000 7fc3d2a00640 1 -- 172.21.15.97:0/4069746280 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc3d80039c0 con 0x55e4aff874a0 2024-03-27T10:57:45.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.798+0000 7fc3f6579740 10 client.15927 fetch_fsmap learned FSMap version 28 2024-03-27T10:57:45.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.798+0000 7fc3f6579740 1 -- 172.21.15.97:0/4069746280 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e4b00023a0 con 0x55e4aff874a0 2024-03-27T10:57:45.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.798+0000 7fc3d2a00640 1 -- 172.21.15.97:0/4069746280 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc3e00cdbd0 con 0x55e4aff874a0 2024-03-27T10:57:45.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.799+0000 7fc3d2a00640 1 -- 172.21.15.97:0/4069746280 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fc3d805a840 con 0x55e4aff874a0 2024-03-27T10:57:45.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.799+0000 7fc3f6579740 10 client.15927 fetch_fsmap finished waiting for FSMap version 28 2024-03-27T10:57:45.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.799+0000 7fc3f6579740 10 client.15927 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-03-27T10:57:45.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.799+0000 7fc3f6579740 20 client.15927 populate_metadata read hostname 'smithi097' 2024-03-27T10:57:45.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.799+0000 7fc3f6579740 1 --2- 172.21.15.97:0/4069746280 >> [v2:172.21.15.177:6834/3349112132,v1:172.21.15.177:6835/3349112132] conn(0x55e4b0002d20 0x55e4b0005140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:45.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.799+0000 7fc3f6579740 4 client.15927 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-27T10:57:45.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.799+0000 7fc3f6579740 1 -- 172.21.15.97:0/4069746280 --> [v2:172.21.15.177:6834/3349112132,v1:172.21.15.177:6835/3349112132] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e4afef2f00 con 0x55e4b0002d20 2024-03-27T10:57:45.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.799+0000 7fc3f0e00640 1 --2- 172.21.15.97:0/4069746280 >> [v2:172.21.15.177:6834/3349112132,v1:172.21.15.177:6835/3349112132] conn(0x55e4b0002d20 0x55e4b0005140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:45.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.800+0000 7fc3f0e00640 1 --2- 172.21.15.97:0/4069746280 >> [v2:172.21.15.177:6834/3349112132,v1:172.21.15.177:6835/3349112132] conn(0x55e4b0002d20 0x55e4b0005140 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a4a1f828f8d16cea server_cookie=3dc55b4758c2bbd4 in_seq=0 out_seq=0 2024-03-27T10:57:45.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.800+0000 7fc3d2a00640 10 client.15927 ms_handle_connect on v2:172.21.15.177:6834/3349112132 2024-03-27T10:57:45.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.801+0000 7fc3d2a00640 1 -- 172.21.15.97:0/4069746280 <== mds.3 v2:172.21.15.177:6834/3349112132 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e4afef2f00 con 0x55e4b0002d20 2024-03-27T10:57:45.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.801+0000 7fc3d2a00640 10 client.15927 handle_command_reply: tid=0 2024-03-27T10:57:45.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.801+0000 7fc3f6579740 1 client.15927 shutdown 2024-03-27T10:57:45.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.801+0000 7fc3f6579740 1 -- 172.21.15.97:0/4069746280 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e4aff874a0 msgr2=0x55e4aff956a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:45.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.801+0000 7fc3f6579740 1 --2- 172.21.15.97:0/4069746280 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e4aff874a0 0x55e4aff956a0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fc3d8071ab0 tx=0x7fc3d8005d50 comp rx=0 tx=0).stop 2024-03-27T10:57:45.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.801+0000 7fc3f6579740 1 -- 172.21.15.97:0/4069746280 shutdown_connections 2024-03-27T10:57:45.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.801+0000 7fc3f6579740 1 --2- 172.21.15.97:0/4069746280 >> [v2:172.21.15.177:6834/3349112132,v1:172.21.15.177:6835/3349112132] conn(0x55e4b0002d20 0x55e4b0005140 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:45.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.801+0000 7fc3f6579740 1 --2- 172.21.15.97:0/4069746280 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e4aff8c610 0x55e4b0000b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:45.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.801+0000 7fc3d2a00640 0 client.15927 ms_handle_reset on v2:172.21.15.177:6834/3349112132 2024-03-27T10:57:45.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.801+0000 7fc3f6579740 1 --2- 172.21.15.97:0/4069746280 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e4aff874a0 0x55e4aff956a0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:45.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.801+0000 7fc3f6579740 1 --2- 172.21.15.97:0/4069746280 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e4aff86ab0 0x55e4aff95160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:45.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.801+0000 7fc3f6579740 1 -- 172.21.15.97:0/4069746280 >> 172.21.15.97:0/4069746280 conn(0x55e4aff82020 msgr2=0x55e4aff887c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:45.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.802+0000 7fc3f6579740 1 -- 172.21.15.97:0/4069746280 shutdown_connections 2024-03-27T10:57:45.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.802+0000 7fc3f6579740 1 -- 172.21.15.97:0/4069746280 wait complete. 2024-03-27T10:57:45.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.802+0000 7fc3f6579740 20 client.15927 trim_cache size 0 max 16384 2024-03-27T10:57:45.776 INFO:teuthology.orchestra.run.smithi097.stdout:[] 2024-03-27T10:57:45.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.803+0000 7fc3f2200640 1 -- 172.21.15.97:0/2972101156 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fc3d40a3670 msgr2=0x7fc3d40a5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:45.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.803+0000 7fc3f2200640 1 --2- 172.21.15.97:0/2972101156 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fc3d40a3670 0x7fc3d40a5b30 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc3d8008000 tx=0x7fc3d8005fd0 comp rx=0 tx=0).stop 2024-03-27T10:57:45.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.803+0000 7fc3f2200640 1 -- 172.21.15.97:0/2972101156 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3ec075b90 msgr2=0x7fc3ec19e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:45.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.803+0000 7fc3f2200640 1 --2- 172.21.15.97:0/2972101156 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3ec075b90 0x7fc3ec19e4e0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fc3e000e9f0 tx=0x7fc3e000eeb0 comp rx=0 tx=0).stop 2024-03-27T10:57:45.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.803+0000 7fc3f2200640 1 -- 172.21.15.97:0/2972101156 shutdown_connections 2024-03-27T10:57:45.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.803+0000 7fc3f2200640 1 --2- 172.21.15.97:0/2972101156 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc3ec1a3a80 0x7fc3ec1a5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:45.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.803+0000 7fc3f2200640 1 --2- 172.21.15.97:0/2972101156 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fc3d40a3670 0x7fc3d40a5b30 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:45.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.803+0000 7fc3f2200640 1 --2- 172.21.15.97:0/2972101156 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc3ec10e020 0x7fc3ec19ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:45.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.803+0000 7fc3f2200640 1 --2- 172.21.15.97:0/2972101156 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3ec075b90 0x7fc3ec19e4e0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:45.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.803+0000 7fc3f2200640 1 -- 172.21.15.97:0/2972101156 >> 172.21.15.97:0/2972101156 conn(0x7fc3ec0fcfc0 msgr2=0x7fc3ec1013c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:45.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.803+0000 7fc3f2200640 1 -- 172.21.15.97:0/2972101156 shutdown_connections 2024-03-27T10:57:45.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.803+0000 7fc3f2200640 1 -- 172.21.15.97:0/2972101156 wait complete. 2024-03-27T10:57:45.789 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-27T10:57:45.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.993+0000 7f87ebe00640 1 -- 172.21.15.97:0/2792155435 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87e4105940 msgr2=0x7f87e410c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:45.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.993+0000 7f87ebe00640 1 --2- 172.21.15.97:0/2792155435 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87e4105940 0x7f87e410c1d0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f87dc009be0 tx=0x7f87dc02bef0 comp rx=0 tx=0).stop 2024-03-27T10:57:45.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.993+0000 7f87ebe00640 1 -- 172.21.15.97:0/2792155435 shutdown_connections 2024-03-27T10:57:45.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.993+0000 7f87ebe00640 1 --2- 172.21.15.97:0/2792155435 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87e4105940 0x7f87e410c1d0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:45.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.993+0000 7f87ebe00640 1 --2- 172.21.15.97:0/2792155435 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f87e4104f80 0x7f87e4105400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:45.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.993+0000 7f87ebe00640 1 --2- 172.21.15.97:0/2792155435 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f87e4103d90 0x7f87e41041b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:45.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.993+0000 7f87ebe00640 1 -- 172.21.15.97:0/2792155435 >> 172.21.15.97:0/2792155435 conn(0x7f87e40ff300 msgr2=0x7f87e4101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:45.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.993+0000 7f87ebe00640 1 -- 172.21.15.97:0/2792155435 shutdown_connections 2024-03-27T10:57:45.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.993+0000 7f87ebe00640 1 -- 172.21.15.97:0/2792155435 wait complete. 2024-03-27T10:57:45.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.994+0000 7f87ebe00640 1 Processor -- start 2024-03-27T10:57:45.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.994+0000 7f87ebe00640 1 -- start start 2024-03-27T10:57:45.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.995+0000 7f87ebe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87e4103d90 0x7f87e4199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:45.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.995+0000 7f87ebe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f87e4104f80 0x7f87e419a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:45.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.995+0000 7f87ebe00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f87e4105940 0x7f87e41a1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:45.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.995+0000 7f87ebe00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f87e410f910 con 0x7f87e4103d90 2024-03-27T10:57:45.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.995+0000 7f87ebe00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f87e410f790 con 0x7f87e4104f80 2024-03-27T10:57:45.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.995+0000 7f87ebe00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f87e410fa90 con 0x7f87e4105940 2024-03-27T10:57:45.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.995+0000 7f87e9400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87e4103d90 0x7f87e4199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:45.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.995+0000 7f87e8a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f87e4104f80 0x7f87e419a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:45.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.995+0000 7f87e9400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87e4103d90 0x7f87e4199f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53944/0 (socket says 172.21.15.97:53944) 2024-03-27T10:57:45.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.995+0000 7f87e9400640 1 -- 172.21.15.97:0/3228341471 learned_addr learned my addr 172.21.15.97:0/3228341471 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:45.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.996+0000 7f87e9e00640 1 --2- 172.21.15.97:0/3228341471 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f87e4105940 0x7f87e41a1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:45.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.996+0000 7f87e9400640 1 -- 172.21.15.97:0/3228341471 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f87e4105940 msgr2=0x7f87e41a1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:45.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.996+0000 7f87e9400640 1 --2- 172.21.15.97:0/3228341471 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f87e4105940 0x7f87e41a1500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:45.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.996+0000 7f87e9400640 1 -- 172.21.15.97:0/3228341471 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f87e4104f80 msgr2=0x7f87e419a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:45.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.996+0000 7f87e9400640 1 --2- 172.21.15.97:0/3228341471 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f87e4104f80 0x7f87e419a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:45.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.996+0000 7f87e9400640 1 -- 172.21.15.97:0/3228341471 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87e41a1b70 con 0x7f87e4103d90 2024-03-27T10:57:45.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.996+0000 7f87e9400640 1 --2- 172.21.15.97:0/3228341471 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87e4103d90 0x7f87e4199f50 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f87d800e9c0 tx=0x7f87d800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:45.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.996+0000 7f87e2000640 1 -- 172.21.15.97:0/3228341471 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d800cd80 con 0x7f87e4103d90 2024-03-27T10:57:45.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.997+0000 7f87e2000640 1 -- 172.21.15.97:0/3228341471 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f87d80035f0 con 0x7f87e4103d90 2024-03-27T10:57:45.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.997+0000 7f87e2000640 1 -- 172.21.15.97:0/3228341471 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d8010690 con 0x7f87e4103d90 2024-03-27T10:57:45.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.997+0000 7f87ebe00640 1 -- 172.21.15.97:0/3228341471 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87e41a1e60 con 0x7f87e4103d90 2024-03-27T10:57:45.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:45.997+0000 7f87ebe00640 1 -- 172.21.15.97:0/3228341471 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87e41a2270 con 0x7f87e4103d90 2024-03-27T10:57:45.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.000+0000 7f87e2000640 1 -- 172.21.15.97:0/3228341471 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f87d80037a0 con 0x7f87e4103d90 2024-03-27T10:57:45.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.001+0000 7f87eeb8b740 1 Processor -- start 2024-03-27T10:57:45.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.002+0000 7f87eeb8b740 1 -- start start 2024-03-27T10:57:45.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.002+0000 7f87eeb8b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f6bb4068c0 0x55f6bb406ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:45.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.002+0000 7f87eeb8b740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f6bb4072b0 0x55f6bb40bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:45.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.002+0000 7f87eeb8b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f6bb40c420 0x55f6bb40e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:45.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.002+0000 7f87eeb8b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55f6bb3ce7b0 con 0x55f6bb40c420 2024-03-27T10:57:45.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.002+0000 7f87eeb8b740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55f6bb36b670 con 0x55f6bb4068c0 2024-03-27T10:57:45.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.002+0000 7f87eeb8b740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55f6bb313970 con 0x55f6bb4072b0 2024-03-27T10:57:45.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.002+0000 7f87e2000640 1 --2- 172.21.15.97:0/3228341471 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f87cc0a3710 0x7f87cc0a5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:45.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.002+0000 7f87e8a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f6bb40c420 0x55f6bb40e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:45.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.002+0000 7f87e8a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f6bb40c420 0x55f6bb40e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53958/0 (socket says 172.21.15.97:53958) 2024-03-27T10:57:45.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.002+0000 7f87e8a00640 1 -- 172.21.15.97:0/443515928 learned_addr learned my addr 172.21.15.97:0/443515928 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:45.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.003+0000 7f87e2000640 1 -- 172.21.15.97:0/3228341471 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f87d80d0320 con 0x7f87e4103d90 2024-03-27T10:57:45.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.003+0000 7f87e9e00640 1 --2- 172.21.15.97:0/443515928 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f6bb4068c0 0x55f6bb406ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:45.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.003+0000 7f87e9e00640 1 --2- 172.21.15.97:0/3228341471 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f87cc0a3710 0x7f87cc0a5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:45.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.003+0000 7f87e9400640 1 --2- 172.21.15.97:0/443515928 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f6bb4072b0 0x55f6bb40bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:45.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.003+0000 7f87e8a00640 1 -- 172.21.15.97:0/443515928 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f6bb4072b0 msgr2=0x55f6bb40bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:45.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.003+0000 7f87e8a00640 1 --2- 172.21.15.97:0/443515928 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f6bb4072b0 0x55f6bb40bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:45.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.003+0000 7f87e8a00640 1 -- 172.21.15.97:0/443515928 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f6bb4068c0 msgr2=0x55f6bb406ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:45.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.003+0000 7f87e8a00640 1 --2- 172.21.15.97:0/443515928 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f6bb4068c0 0x55f6bb406ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:45.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.003+0000 7f87e8a00640 1 -- 172.21.15.97:0/443515928 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6bb07f3f0 con 0x55f6bb40c420 2024-03-27T10:57:45.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.004+0000 7f87e8a00640 1 --2- 172.21.15.97:0/443515928 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f6bb40c420 0x55f6bb40e8b0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f87d0009c10 tx=0x7f87d00117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50091d344c95a75f server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:45.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.004+0000 7f87e9e00640 1 --2- 172.21.15.97:0/3228341471 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f87cc0a3710 0x7f87cc0a5bd0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f87dc006010 tx=0x7f87dc034f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:45.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.004+0000 7f87d6600640 1 -- 172.21.15.97:0/443515928 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d0012450 con 0x55f6bb40c420 2024-03-27T10:57:45.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.004+0000 7f87d6600640 1 -- 172.21.15.97:0/443515928 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f87d0012aa0 con 0x55f6bb40c420 2024-03-27T10:57:45.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.004+0000 7f87d6600640 1 -- 172.21.15.97:0/443515928 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d000faf0 con 0x55f6bb40c420 2024-03-27T10:57:45.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.005+0000 7f87eeb8b740 1 -- 172.21.15.97:0/443515928 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f6bb40c420 msgr2=0x55f6bb40e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:45.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.005+0000 7f87eeb8b740 1 --2- 172.21.15.97:0/443515928 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f6bb40c420 0x55f6bb40e8b0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f87d0009c10 tx=0x7f87d00117b0 comp rx=0 tx=0).stop 2024-03-27T10:57:45.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.005+0000 7f87eeb8b740 1 -- 172.21.15.97:0/443515928 shutdown_connections 2024-03-27T10:57:45.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.005+0000 7f87eeb8b740 1 --2- 172.21.15.97:0/443515928 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f6bb40c420 0x55f6bb40e8b0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:45.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.005+0000 7f87eeb8b740 1 --2- 172.21.15.97:0/443515928 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f6bb4072b0 0x55f6bb40bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:45.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.005+0000 7f87eeb8b740 1 --2- 172.21.15.97:0/443515928 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f6bb4068c0 0x55f6bb406ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:45.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.005+0000 7f87eeb8b740 1 -- 172.21.15.97:0/443515928 >> 172.21.15.97:0/443515928 conn(0x55f6bb401e30 msgr2=0x55f6bb402230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:45.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.005+0000 7f87eeb8b740 1 -- 172.21.15.97:0/443515928 shutdown_connections 2024-03-27T10:57:45.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.005+0000 7f87eeb8b740 1 -- 172.21.15.97:0/443515928 wait complete. 2024-03-27T10:57:45.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.006+0000 7f87eeb8b740 1 -- start start 2024-03-27T10:57:45.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.006+0000 7f87eeb8b740 1 Processor -- start 2024-03-27T10:57:45.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.006+0000 7f87eeb8b740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f6bb4068c0 0x55f6bb410ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:45.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.006+0000 7f87eeb8b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f6bb4072b0 0x55f6bb410ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:45.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.007+0000 7f87eeb8b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f6bb40c420 0x55f6bb417fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:45.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.007+0000 7f87eeb8b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55f6bb326ca0 con 0x55f6bb40c420 2024-03-27T10:57:45.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.007+0000 7f87eeb8b740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55f6bb367620 con 0x55f6bb4072b0 2024-03-27T10:57:45.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.007+0000 7f87eeb8b740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55f6bb40f3d0 con 0x55f6bb4068c0 2024-03-27T10:57:45.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.007+0000 7f87eeb8b740 10 client.0 _ll_register_callbacks cb 0x55f6bb363160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:57:45.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.007+0000 7f87e8a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f6bb4072b0 0x55f6bb410ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:45.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.007+0000 7f87e9e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f6bb40c420 0x55f6bb417fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:45.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.007+0000 7f87e9400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f6bb4068c0 0x55f6bb410ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:45.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.007+0000 7f87e9400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f6bb4068c0 0x55f6bb410ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.97:35548/0 (socket says 172.21.15.97:35548) 2024-03-27T10:57:45.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.007+0000 7f87e9400640 1 -- 172.21.15.97:0/1830085506 learned_addr learned my addr 172.21.15.97:0/1830085506 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:45.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.008+0000 7f87e9e00640 1 -- 172.21.15.97:0/1830085506 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f6bb4068c0 msgr2=0x55f6bb410ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:45.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.008+0000 7f87e9e00640 1 --2- 172.21.15.97:0/1830085506 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f6bb4068c0 0x55f6bb410ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:45.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.008+0000 7f87e9e00640 1 -- 172.21.15.97:0/1830085506 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f6bb4072b0 msgr2=0x55f6bb410ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:45.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.008+0000 7f87e9e00640 1 --2- 172.21.15.97:0/1830085506 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f6bb4072b0 0x55f6bb410ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:45.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.008+0000 7f87e9e00640 1 -- 172.21.15.97:0/1830085506 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6bb1497d0 con 0x55f6bb40c420 2024-03-27T10:57:45.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.008+0000 7f87e9e00640 1 --2- 172.21.15.97:0/1830085506 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f6bb40c420 0x55f6bb417fc0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f87dc036040 tx=0x7f87dc009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c22bc53486a3daf server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:45.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.008+0000 7f87caa00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T10:57:45.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.009+0000 7f87caa00640 1 -- 172.21.15.97:0/1830085506 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87dc033e80 con 0x55f6bb40c420 2024-03-27T10:57:45.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.009+0000 7f87caa00640 1 -- 172.21.15.97:0/1830085506 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f87dc003e00 con 0x55f6bb40c420 2024-03-27T10:57:45.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.009+0000 7f87eeb8b740 1 -- 172.21.15.97:0/1830085506 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87d0012aa0 con 0x55f6bb40c420 2024-03-27T10:57:45.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.009+0000 7f87caa00640 1 -- 172.21.15.97:0/1830085506 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f87dc004720 con 0x55f6bb40c420 2024-03-27T10:57:45.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.009+0000 7f87eeb8b740 10 client.15939 fetch_fsmap learned FSMap version 28 2024-03-27T10:57:45.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.009+0000 7f87eeb8b740 1 -- 172.21.15.97:0/1830085506 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f6bb41b3a0 con 0x55f6bb40c420 2024-03-27T10:57:45.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.009+0000 7f87caa00640 1 -- 172.21.15.97:0/1830085506 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f87d0012aa0 con 0x55f6bb40c420 2024-03-27T10:57:45.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.010+0000 7f87caa00640 1 -- 172.21.15.97:0/1830085506 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f87dc04f460 con 0x55f6bb40c420 2024-03-27T10:57:45.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.010+0000 7f87eeb8b740 10 client.15939 fetch_fsmap finished waiting for FSMap version 28 2024-03-27T10:57:45.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.010+0000 7f87eeb8b740 10 client.15939 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-27T10:57:45.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.010+0000 7f87eeb8b740 20 client.15939 populate_metadata read hostname 'smithi097' 2024-03-27T10:57:45.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.010+0000 7f87eeb8b740 1 --2- 172.21.15.97:0/1830085506 >> [v2:172.21.15.114:6834/3581918222,v1:172.21.15.114:6835/3581918222] conn(0x55f6bb41cb10 0x55f6bb41ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:45.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.010+0000 7f87eeb8b740 4 client.15939 mds_command: new command op to 34445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T10:57:45.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.010+0000 7f87eeb8b740 1 -- 172.21.15.97:0/1830085506 --> [v2:172.21.15.114:6834/3581918222,v1:172.21.15.114:6835/3581918222] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f6bb378bb0 con 0x55f6bb41cb10 2024-03-27T10:57:45.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.010+0000 7f87e9400640 1 --2- 172.21.15.97:0/1830085506 >> [v2:172.21.15.114:6834/3581918222,v1:172.21.15.114:6835/3581918222] conn(0x55f6bb41cb10 0x55f6bb41ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:45.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.011+0000 7f87e9400640 1 --2- 172.21.15.97:0/1830085506 >> [v2:172.21.15.114:6834/3581918222,v1:172.21.15.114:6835/3581918222] conn(0x55f6bb41cb10 0x55f6bb41ef20 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=aac273925d5c1c32 server_cookie=578f2bdf92d76d84 in_seq=0 out_seq=0 2024-03-27T10:57:45.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.011+0000 7f87caa00640 10 client.15939 ms_handle_connect on v2:172.21.15.114:6834/3581918222 2024-03-27T10:57:45.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.013+0000 7f87caa00640 1 -- 172.21.15.97:0/1830085506 <== mds.2 v2:172.21.15.114:6834/3581918222 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55f6bb378bb0 con 0x55f6bb41cb10 2024-03-27T10:57:45.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.013+0000 7f87caa00640 10 client.15939 handle_command_reply: tid=0 2024-03-27T10:57:45.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.013+0000 7f87eeb8b740 1 client.15939 shutdown 2024-03-27T10:57:45.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.013+0000 7f87eeb8b740 1 -- 172.21.15.97:0/1830085506 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f6bb40c420 msgr2=0x55f6bb417fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:45.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.013+0000 7f87eeb8b740 1 --2- 172.21.15.97:0/1830085506 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f6bb40c420 0x55f6bb417fc0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f87dc036040 tx=0x7f87dc009500 comp rx=0 tx=0).stop 2024-03-27T10:57:45.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.013+0000 7f87eeb8b740 1 -- 172.21.15.97:0/1830085506 shutdown_connections 2024-03-27T10:57:45.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.013+0000 7f87eeb8b740 1 --2- 172.21.15.97:0/1830085506 >> [v2:172.21.15.114:6834/3581918222,v1:172.21.15.114:6835/3581918222] conn(0x55f6bb41cb10 0x55f6bb41ef20 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:45.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.013+0000 7f87caa00640 0 client.15939 ms_handle_reset on v2:172.21.15.114:6834/3581918222 2024-03-27T10:57:45.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.013+0000 7f87eeb8b740 1 --2- 172.21.15.97:0/1830085506 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f6bb40c420 0x55f6bb417fc0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:45.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.013+0000 7f87eeb8b740 1 --2- 172.21.15.97:0/1830085506 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f6bb4072b0 0x55f6bb410ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:45.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.013+0000 7f87eeb8b740 1 --2- 172.21.15.97:0/1830085506 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f6bb4068c0 0x55f6bb410ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:45.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.013+0000 7f87eeb8b740 1 -- 172.21.15.97:0/1830085506 >> 172.21.15.97:0/1830085506 conn(0x55f6bb401e30 msgr2=0x55f6bb401030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:45.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.013+0000 7f87eeb8b740 1 -- 172.21.15.97:0/1830085506 shutdown_connections 2024-03-27T10:57:45.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.013+0000 7f87eeb8b740 1 -- 172.21.15.97:0/1830085506 wait complete. 2024-03-27T10:57:45.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.014+0000 7f87eeb8b740 20 client.15939 trim_cache size 0 max 16384 2024-03-27T10:57:45.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.028+0000 7f87eeb8b740 1 Processor -- start 2024-03-27T10:57:45.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.028+0000 7f87eeb8b740 1 -- start start 2024-03-27T10:57:46.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.028+0000 7f87eeb8b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f6bb4068c0 0x55f6bb4007a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:46.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.028+0000 7f87eeb8b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f6bb4072b0 0x55f6bb417f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:46.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.028+0000 7f87eeb8b740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f6bb40c420 0x55f6bb400ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:46.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.028+0000 7f87eeb8b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f87e41ab170 con 0x55f6bb4072b0 2024-03-27T10:57:46.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.028+0000 7f87eeb8b740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f87e41accb0 con 0x55f6bb4068c0 2024-03-27T10:57:46.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.028+0000 7f87eeb8b740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f87e41ab840 con 0x55f6bb40c420 2024-03-27T10:57:46.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.028+0000 7f87e9400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f6bb4068c0 0x55f6bb4007a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:46.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.028+0000 7f87e8a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f6bb4072b0 0x55f6bb417f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:46.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.028+0000 7f87e9400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f6bb4068c0 0x55f6bb4007a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:34284/0 (socket says 172.21.15.97:34284) 2024-03-27T10:57:46.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.028+0000 7f87e9400640 1 -- 172.21.15.97:0/1872516594 learned_addr learned my addr 172.21.15.97:0/1872516594 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:46.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.029+0000 7f87e9e00640 1 --2- 172.21.15.97:0/1872516594 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f6bb40c420 0x55f6bb400ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:46.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.029+0000 7f87e9400640 1 -- 172.21.15.97:0/1872516594 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f6bb40c420 msgr2=0x55f6bb400ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:46.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.029+0000 7f87e9400640 1 --2- 172.21.15.97:0/1872516594 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f6bb40c420 0x55f6bb400ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:46.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.029+0000 7f87e9400640 1 -- 172.21.15.97:0/1872516594 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f6bb4072b0 msgr2=0x55f6bb417f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:46.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.029+0000 7f87e9400640 1 --2- 172.21.15.97:0/1872516594 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f6bb4072b0 0x55f6bb417f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:46.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.029+0000 7f87e9400640 1 -- 172.21.15.97:0/1872516594 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6bb407690 con 0x55f6bb4068c0 2024-03-27T10:57:46.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.029+0000 7f87e9e00640 1 --2- 172.21.15.97:0/1872516594 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f6bb40c420 0x55f6bb400ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T10:57:46.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.029+0000 7f87e9400640 1 --2- 172.21.15.97:0/1872516594 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f6bb4068c0 0x55f6bb4007a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f87d809c4a0 tx=0x7f87d8087040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf4324d1c5437fcb server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:46.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.030+0000 7f87d5c00640 1 -- 172.21.15.97:0/1872516594 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d8016e30 con 0x55f6bb4068c0 2024-03-27T10:57:46.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.030+0000 7f87d5c00640 1 -- 172.21.15.97:0/1872516594 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f87d809c7a0 con 0x55f6bb4068c0 2024-03-27T10:57:46.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.031+0000 7f87eeb8b740 1 -- 172.21.15.97:0/1872516594 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f6bb4068c0 msgr2=0x55f6bb4007a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:46.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.031+0000 7f87eeb8b740 1 --2- 172.21.15.97:0/1872516594 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f6bb4068c0 0x55f6bb4007a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f87d809c4a0 tx=0x7f87d8087040 comp rx=0 tx=0).stop 2024-03-27T10:57:46.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.031+0000 7f87eeb8b740 1 -- 172.21.15.97:0/1872516594 shutdown_connections 2024-03-27T10:57:46.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.031+0000 7f87eeb8b740 1 --2- 172.21.15.97:0/1872516594 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f6bb40c420 0x55f6bb400ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:46.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.031+0000 7f87eeb8b740 1 --2- 172.21.15.97:0/1872516594 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f6bb4072b0 0x55f6bb417f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:46.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.031+0000 7f87eeb8b740 1 --2- 172.21.15.97:0/1872516594 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f6bb4068c0 0x55f6bb4007a0 secure :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f87d809c4a0 tx=0x7f87d8087040 comp rx=0 tx=0).stop 2024-03-27T10:57:46.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.031+0000 7f87eeb8b740 1 -- 172.21.15.97:0/1872516594 >> 172.21.15.97:0/1872516594 conn(0x55f6bb401e30 msgr2=0x55f6bb408610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:46.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.031+0000 7f87eeb8b740 1 -- 172.21.15.97:0/1872516594 shutdown_connections 2024-03-27T10:57:46.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.031+0000 7f87eeb8b740 1 -- 172.21.15.97:0/1872516594 wait complete. 2024-03-27T10:57:46.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.031+0000 7f87eeb8b740 1 -- start start 2024-03-27T10:57:46.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.032+0000 7f87eeb8b740 1 Processor -- start 2024-03-27T10:57:46.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.032+0000 7f87eeb8b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f6bb4072b0 0x55f6bb415710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:46.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.032+0000 7f87eeb8b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f6bb40c420 0x55f6bb415c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:46.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.032+0000 7f87eeb8b740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f6bb416c10 0x55f6bb417090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:46.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.032+0000 7f87eeb8b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f87e41ac5e0 con 0x55f6bb4072b0 2024-03-27T10:57:46.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.032+0000 7f87eeb8b740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f87e41abf10 con 0x55f6bb40c420 2024-03-27T10:57:46.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.032+0000 7f87eeb8b740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f87e41a9150 con 0x55f6bb416c10 2024-03-27T10:57:46.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.032+0000 7f87eeb8b740 10 client.0 _ll_register_callbacks cb 0x7f87d8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:57:46.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.032+0000 7f87e9400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f6bb4072b0 0x55f6bb415710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:46.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.032+0000 7f87e9400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f6bb4072b0 0x55f6bb415710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53970/0 (socket says 172.21.15.97:53970) 2024-03-27T10:57:46.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.032+0000 7f87e9400640 1 -- 172.21.15.97:0/2127804958 learned_addr learned my addr 172.21.15.97:0/2127804958 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:46.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.033+0000 7f87e9e00640 1 --2- 172.21.15.97:0/2127804958 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f6bb416c10 0x55f6bb417090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:46.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.033+0000 7f87e9400640 1 -- 172.21.15.97:0/2127804958 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f6bb416c10 msgr2=0x55f6bb417090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:46.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.033+0000 7f87e9400640 1 --2- 172.21.15.97:0/2127804958 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f6bb416c10 0x55f6bb417090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:46.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.033+0000 7f87e9400640 1 -- 172.21.15.97:0/2127804958 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f6bb40c420 msgr2=0x55f6bb415c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:46.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.033+0000 7f87e9400640 1 --2- 172.21.15.97:0/2127804958 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f6bb40c420 0x55f6bb415c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:46.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.033+0000 7f87e9400640 1 -- 172.21.15.97:0/2127804958 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6bb4175d0 con 0x55f6bb4072b0 2024-03-27T10:57:46.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.033+0000 7f87e9400640 1 --2- 172.21.15.97:0/2127804958 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f6bb4072b0 0x55f6bb415710 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f87d80058c0 tx=0x7f87d809c230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=683fa76c35145c65 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:46.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.033+0000 7f87caa00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T10:57:46.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.033+0000 7f87caa00640 1 -- 172.21.15.97:0/2127804958 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d80a5410 con 0x55f6bb4072b0 2024-03-27T10:57:46.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.033+0000 7f87caa00640 1 -- 172.21.15.97:0/2127804958 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f87d8098e10 con 0x55f6bb4072b0 2024-03-27T10:57:46.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.033+0000 7f87eeb8b740 1 -- 172.21.15.97:0/2127804958 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87d809c7a0 con 0x55f6bb4072b0 2024-03-27T10:57:46.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.033+0000 7f87caa00640 1 -- 172.21.15.97:0/2127804958 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f87d8016e30 con 0x55f6bb4072b0 2024-03-27T10:57:46.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.033+0000 7f87eeb8b740 10 client.15945 fetch_fsmap learned FSMap version 28 2024-03-27T10:57:46.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.033+0000 7f87eeb8b740 1 -- 172.21.15.97:0/2127804958 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f6bb48ba10 con 0x55f6bb4072b0 2024-03-27T10:57:46.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.034+0000 7f87caa00640 1 -- 172.21.15.97:0/2127804958 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f87d809c7a0 con 0x55f6bb4072b0 2024-03-27T10:57:46.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.034+0000 7f87caa00640 1 -- 172.21.15.97:0/2127804958 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f87d8098610 con 0x55f6bb4072b0 2024-03-27T10:57:46.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.034+0000 7f87eeb8b740 10 client.15945 fetch_fsmap finished waiting for FSMap version 28 2024-03-27T10:57:46.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.034+0000 7f87eeb8b740 10 client.15945 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-27T10:57:46.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.034+0000 7f87eeb8b740 20 client.15945 populate_metadata read hostname 'smithi097' 2024-03-27T10:57:46.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.034+0000 7f87eeb8b740 1 --2- 172.21.15.97:0/2127804958 >> [v2:172.21.15.114:6834/3581918222,v1:172.21.15.114:6835/3581918222] conn(0x55f6bb48c2f0 0x55f6bb48e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:46.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.034+0000 7f87eeb8b740 4 client.15945 mds_command: new command op to 34445 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-27T10:57:46.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.034+0000 7f87eeb8b740 1 -- 172.21.15.97:0/2127804958 --> [v2:172.21.15.114:6834/3581918222,v1:172.21.15.114:6835/3581918222] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f6bb4108a0 con 0x55f6bb48c2f0 2024-03-27T10:57:46.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.035+0000 7f87e8a00640 1 --2- 172.21.15.97:0/2127804958 >> [v2:172.21.15.114:6834/3581918222,v1:172.21.15.114:6835/3581918222] conn(0x55f6bb48c2f0 0x55f6bb48e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:46.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.036+0000 7f87caa00640 10 client.15945 ms_handle_connect on v2:172.21.15.114:6834/3581918222 2024-03-27T10:57:46.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.036+0000 7f87e8a00640 1 --2- 172.21.15.97:0/2127804958 >> [v2:172.21.15.114:6834/3581918222,v1:172.21.15.114:6835/3581918222] conn(0x55f6bb48c2f0 0x55f6bb48e710 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=80cc832de240834f server_cookie=da3339f5c97b136b in_seq=0 out_seq=0 2024-03-27T10:57:46.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.036+0000 7f87caa00640 1 -- 172.21.15.97:0/2127804958 <== mds.2 v2:172.21.15.114:6834/3581918222 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f6bb4108a0 con 0x55f6bb48c2f0 2024-03-27T10:57:46.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.036+0000 7f87caa00640 10 client.15945 handle_command_reply: tid=0 2024-03-27T10:57:46.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.036+0000 7f87eeb8b740 1 client.15945 shutdown 2024-03-27T10:57:46.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.036+0000 7f87eeb8b740 1 -- 172.21.15.97:0/2127804958 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f6bb4072b0 msgr2=0x55f6bb415710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:46.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.036+0000 7f87eeb8b740 1 --2- 172.21.15.97:0/2127804958 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f6bb4072b0 0x55f6bb415710 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f87d80058c0 tx=0x7f87d809c230 comp rx=0 tx=0).stop 2024-03-27T10:57:46.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.036+0000 7f87eeb8b740 1 -- 172.21.15.97:0/2127804958 shutdown_connections 2024-03-27T10:57:46.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.036+0000 7f87eeb8b740 1 --2- 172.21.15.97:0/2127804958 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f6bb416c10 0x55f6bb417090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:46.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.036+0000 7f87eeb8b740 1 --2- 172.21.15.97:0/2127804958 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f6bb40c420 0x55f6bb415c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:46.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.036+0000 7f87eeb8b740 1 --2- 172.21.15.97:0/2127804958 >> [v2:172.21.15.114:6834/3581918222,v1:172.21.15.114:6835/3581918222] conn(0x55f6bb48c2f0 0x55f6bb48e710 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:46.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.036+0000 7f87eeb8b740 1 --2- 172.21.15.97:0/2127804958 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f6bb4072b0 0x55f6bb415710 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:46.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.036+0000 7f87eeb8b740 1 -- 172.21.15.97:0/2127804958 >> 172.21.15.97:0/2127804958 conn(0x55f6bb401e30 msgr2=0x55f6bb405090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:46.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.036+0000 7f87caa00640 0 client.15945 ms_handle_reset on v2:172.21.15.114:6834/3581918222 2024-03-27T10:57:46.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.037+0000 7f87eeb8b740 1 -- 172.21.15.97:0/2127804958 shutdown_connections 2024-03-27T10:57:46.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.037+0000 7f87eeb8b740 1 -- 172.21.15.97:0/2127804958 wait complete. 2024-03-27T10:57:46.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.037+0000 7f87eeb8b740 20 client.15945 trim_cache size 0 max 16384 2024-03-27T10:57:46.010 INFO:teuthology.orchestra.run.smithi097.stdout:[] 2024-03-27T10:57:46.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.037+0000 7f87ebe00640 1 -- 172.21.15.97:0/3228341471 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f87cc0a3710 msgr2=0x7f87cc0a5bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:46.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.037+0000 7f87ebe00640 1 --2- 172.21.15.97:0/3228341471 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f87cc0a3710 0x7f87cc0a5bd0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f87dc006010 tx=0x7f87dc034f90 comp rx=0 tx=0).stop 2024-03-27T10:57:46.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.037+0000 7f87ebe00640 1 -- 172.21.15.97:0/3228341471 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87e4103d90 msgr2=0x7f87e4199f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:46.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.037+0000 7f87ebe00640 1 --2- 172.21.15.97:0/3228341471 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87e4103d90 0x7f87e4199f50 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f87d800e9c0 tx=0x7f87d800ee80 comp rx=0 tx=0).stop 2024-03-27T10:57:46.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.038+0000 7f87ebe00640 1 -- 172.21.15.97:0/3228341471 shutdown_connections 2024-03-27T10:57:46.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.038+0000 7f87ebe00640 1 --2- 172.21.15.97:0/3228341471 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f87e4105940 0x7f87e41a1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:46.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.038+0000 7f87ebe00640 1 --2- 172.21.15.97:0/3228341471 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f87cc0a3710 0x7f87cc0a5bd0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:46.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.038+0000 7f87ebe00640 1 --2- 172.21.15.97:0/3228341471 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f87e4104f80 0x7f87e419a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:46.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.038+0000 7f87ebe00640 1 --2- 172.21.15.97:0/3228341471 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87e4103d90 0x7f87e4199f50 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:46.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.038+0000 7f87ebe00640 1 -- 172.21.15.97:0/3228341471 >> 172.21.15.97:0/3228341471 conn(0x7f87e40ff300 msgr2=0x7f87e4108280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:46.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.038+0000 7f87ebe00640 1 -- 172.21.15.97:0/3228341471 shutdown_connections 2024-03-27T10:57:46.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.038+0000 7f87ebe00640 1 -- 172.21.15.97:0/3228341471 wait complete. 2024-03-27T10:57:46.025 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-27T10:57:46.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.245+0000 7fd874c00640 1 -- 172.21.15.97:0/1432055030 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd870102c10 msgr2=0x7fd870105080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:46.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.245+0000 7fd874c00640 1 --2- 172.21.15.97:0/1432055030 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd870102c10 0x7fd870105080 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fd85c009900 tx=0x7fd85c02bc40 comp rx=0 tx=0).stop 2024-03-27T10:57:46.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.245+0000 7fd874c00640 1 -- 172.21.15.97:0/1432055030 shutdown_connections 2024-03-27T10:57:46.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.245+0000 7fd874c00640 1 --2- 172.21.15.97:0/1432055030 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd8701055c0 0x7fd870105a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:46.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.245+0000 7fd874c00640 1 --2- 172.21.15.97:0/1432055030 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd870102c10 0x7fd870105080 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:46.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.245+0000 7fd874c00640 1 --2- 172.21.15.97:0/1432055030 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd8701002a0 0x7fd8701026d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:46.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.245+0000 7fd874c00640 1 -- 172.21.15.97:0/1432055030 >> 172.21.15.97:0/1432055030 conn(0x7fd8700fbec0 msgr2=0x7fd8700fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:46.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.246+0000 7fd874c00640 1 -- 172.21.15.97:0/1432055030 shutdown_connections 2024-03-27T10:57:46.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.246+0000 7fd874c00640 1 -- 172.21.15.97:0/1432055030 wait complete. 2024-03-27T10:57:46.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.246+0000 7fd874c00640 1 Processor -- start 2024-03-27T10:57:46.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.247+0000 7fd874c00640 1 -- start start 2024-03-27T10:57:46.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.247+0000 7fd874c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd8701002a0 0x7fd87019e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:46.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.247+0000 7fd874c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd870102c10 0x7fd87019e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:46.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.247+0000 7fd874c00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd8701055c0 0x7fd8701a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:46.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.247+0000 7fd874c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd870113b90 con 0x7fd870102c10 2024-03-27T10:57:46.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.247+0000 7fd874c00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd870113a10 con 0x7fd8701002a0 2024-03-27T10:57:46.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.247+0000 7fd874c00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd870113d10 con 0x7fd8701055c0 2024-03-27T10:57:46.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.247+0000 7fd86ea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd870102c10 0x7fd87019e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:46.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.247+0000 7fd86ea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd870102c10 0x7fd87019e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53996/0 (socket says 172.21.15.97:53996) 2024-03-27T10:57:46.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.247+0000 7fd86f400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd8701002a0 0x7fd87019e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:46.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.248+0000 7fd86ea00640 1 -- 172.21.15.97:0/2169147767 learned_addr learned my addr 172.21.15.97:0/2169147767 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:46.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.248+0000 7fd86fe00640 1 --2- 172.21.15.97:0/2169147767 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd8701055c0 0x7fd8701a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:46.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.248+0000 7fd86ea00640 1 -- 172.21.15.97:0/2169147767 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd8701055c0 msgr2=0x7fd8701a57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:46.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.248+0000 7fd86ea00640 1 --2- 172.21.15.97:0/2169147767 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd8701055c0 0x7fd8701a57b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:46.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.248+0000 7fd86ea00640 1 -- 172.21.15.97:0/2169147767 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd8701002a0 msgr2=0x7fd87019e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:46.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.248+0000 7fd86ea00640 1 --2- 172.21.15.97:0/2169147767 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd8701002a0 0x7fd87019e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:46.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.248+0000 7fd86ea00640 1 -- 172.21.15.97:0/2169147767 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8701a5e20 con 0x7fd870102c10 2024-03-27T10:57:46.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.248+0000 7fd86fe00640 1 --2- 172.21.15.97:0/2169147767 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd8701055c0 0x7fd8701a57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T10:57:46.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.249+0000 7fd86ea00640 1 --2- 172.21.15.97:0/2169147767 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd870102c10 0x7fd87019e790 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fd85c000c00 tx=0x7fd85c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:46.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.249+0000 7fd867e00640 1 -- 172.21.15.97:0/2169147767 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd85c043020 con 0x7fd870102c10 2024-03-27T10:57:46.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.249+0000 7fd874c00640 1 -- 172.21.15.97:0/2169147767 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8701a60b0 con 0x7fd870102c10 2024-03-27T10:57:46.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.249+0000 7fd867e00640 1 -- 172.21.15.97:0/2169147767 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd85c0054c0 con 0x7fd870102c10 2024-03-27T10:57:46.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.249+0000 7fd867e00640 1 -- 172.21.15.97:0/2169147767 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd85c003d70 con 0x7fd870102c10 2024-03-27T10:57:46.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.249+0000 7fd874c00640 1 -- 172.21.15.97:0/2169147767 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8701a6650 con 0x7fd870102c10 2024-03-27T10:57:46.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.253+0000 7fd867e00640 1 -- 172.21.15.97:0/2169147767 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7fd85c046020 con 0x7fd870102c10 2024-03-27T10:57:46.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.254+0000 7fd878e52740 1 Processor -- start 2024-03-27T10:57:46.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.254+0000 7fd878e52740 1 -- start start 2024-03-27T10:57:46.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.254+0000 7fd878e52740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c735cdfab0 0x55c735cdfed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:46.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.254+0000 7fd878e52740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c735ce04a0 0x55c735ce4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:46.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.254+0000 7fd878e52740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c735ce5610 0x55c735ce7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:46.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.254+0000 7fd878e52740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55c735ca77b0 con 0x55c735cdfab0 2024-03-27T10:57:46.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.254+0000 7fd878e52740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55c735c44670 con 0x55c735ce04a0 2024-03-27T10:57:46.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.254+0000 7fd878e52740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55c735bec970 con 0x55c735ce5610 2024-03-27T10:57:46.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.255+0000 7fd86fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c735cdfab0 0x55c735cdfed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:46.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.255+0000 7fd867e00640 1 --2- 172.21.15.97:0/2169147767 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fd8540a3690 0x7fd8540a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:46.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.255+0000 7fd86fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c735cdfab0 0x55c735cdfed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54000/0 (socket says 172.21.15.97:54000) 2024-03-27T10:57:46.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.255+0000 7fd86fe00640 1 -- 172.21.15.97:0/1801698794 learned_addr learned my addr 172.21.15.97:0/1801698794 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:46.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.255+0000 7fd86f400640 1 --2- 172.21.15.97:0/1801698794 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c735ce04a0 0x55c735ce4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:46.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.255+0000 7fd867e00640 1 -- 172.21.15.97:0/2169147767 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fd85c0f2160 con 0x7fd870102c10 2024-03-27T10:57:46.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.255+0000 7fd86ea00640 1 --2- 172.21.15.97:0/1801698794 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c735ce5610 0x55c735ce7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:46.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.255+0000 7fd86fe00640 1 --2- 172.21.15.97:0/2169147767 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fd8540a3690 0x7fd8540a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:46.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.256+0000 7fd86fe00640 1 -- 172.21.15.97:0/1801698794 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c735ce5610 msgr2=0x55c735ce7aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:46.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.256+0000 7fd86fe00640 1 --2- 172.21.15.97:0/1801698794 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c735ce5610 0x55c735ce7aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:46.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.256+0000 7fd86fe00640 1 -- 172.21.15.97:0/1801698794 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c735ce04a0 msgr2=0x55c735ce4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:46.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.256+0000 7fd86fe00640 1 --2- 172.21.15.97:0/1801698794 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c735ce04a0 0x55c735ce4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:46.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.256+0000 7fd86fe00640 1 -- 172.21.15.97:0/1801698794 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7359583f0 con 0x55c735cdfab0 2024-03-27T10:57:46.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.256+0000 7fd86fe00640 1 --2- 172.21.15.97:0/1801698794 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c735cdfab0 0x55c735cdfed0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fd868012560 tx=0x7fd868012a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f52e69d59fba096 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:46.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.257+0000 7fd85be00640 1 -- 172.21.15.97:0/1801698794 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd868003e30 con 0x55c735cdfab0 2024-03-27T10:57:46.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.257+0000 7fd85be00640 1 -- 172.21.15.97:0/1801698794 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd868010c90 con 0x55c735cdfab0 2024-03-27T10:57:46.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.257+0000 7fd85be00640 1 -- 172.21.15.97:0/1801698794 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd86801ab00 con 0x55c735cdfab0 2024-03-27T10:57:46.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.257+0000 7fd878e52740 1 -- 172.21.15.97:0/1801698794 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c735cdfab0 msgr2=0x55c735cdfed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:46.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.257+0000 7fd86fe00640 1 --2- 172.21.15.97:0/2169147767 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fd8540a3690 0x7fd8540a5b50 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fd868012fd0 tx=0x7fd868018000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:46.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.257+0000 7fd878e52740 1 --2- 172.21.15.97:0/1801698794 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c735cdfab0 0x55c735cdfed0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fd868012560 tx=0x7fd868012a20 comp rx=0 tx=0).stop 2024-03-27T10:57:46.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.258+0000 7fd878e52740 1 -- 172.21.15.97:0/1801698794 shutdown_connections 2024-03-27T10:57:46.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.258+0000 7fd878e52740 1 --2- 172.21.15.97:0/1801698794 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c735ce5610 0x55c735ce7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:46.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.258+0000 7fd878e52740 1 --2- 172.21.15.97:0/1801698794 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c735ce04a0 0x55c735ce4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:46.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.258+0000 7fd878e52740 1 --2- 172.21.15.97:0/1801698794 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c735cdfab0 0x55c735cdfed0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:46.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.258+0000 7fd878e52740 1 -- 172.21.15.97:0/1801698794 >> 172.21.15.97:0/1801698794 conn(0x55c735cdb020 msgr2=0x55c735cdb420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:46.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.258+0000 7fd878e52740 1 -- 172.21.15.97:0/1801698794 shutdown_connections 2024-03-27T10:57:46.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.258+0000 7fd878e52740 1 -- 172.21.15.97:0/1801698794 wait complete. 2024-03-27T10:57:46.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.258+0000 7fd878e52740 1 -- start start 2024-03-27T10:57:46.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.259+0000 7fd878e52740 1 Processor -- start 2024-03-27T10:57:46.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.259+0000 7fd878e52740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c735cdfab0 0x55c735ce9c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:46.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.259+0000 7fd878e52740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c735ce04a0 0x55c735cea140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:46.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.259+0000 7fd878e52740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c735ce5610 0x55c735cf1180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:46.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.259+0000 7fd878e52740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55c735bffca0 con 0x55c735cdfab0 2024-03-27T10:57:46.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.259+0000 7fd878e52740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55c735c40620 con 0x55c735ce5610 2024-03-27T10:57:46.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.259+0000 7fd878e52740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55c735ce8560 con 0x55c735ce04a0 2024-03-27T10:57:46.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.259+0000 7fd878e52740 10 client.0 _ll_register_callbacks cb 0x55c735c3c160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:57:46.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.260+0000 7fd86f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c735cdfab0 0x55c735ce9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:46.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.260+0000 7fd86f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c735cdfab0 0x55c735ce9c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54002/0 (socket says 172.21.15.97:54002) 2024-03-27T10:57:46.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.260+0000 7fd86f400640 1 -- 172.21.15.97:0/869071532 learned_addr learned my addr 172.21.15.97:0/869071532 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:46.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.260+0000 7fd86fe00640 1 --2- 172.21.15.97:0/869071532 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c735ce5610 0x55c735cf1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:46.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.260+0000 7fd86ea00640 1 --2- 172.21.15.97:0/869071532 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c735ce04a0 0x55c735cea140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:46.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.260+0000 7fd86f400640 1 -- 172.21.15.97:0/869071532 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c735ce04a0 msgr2=0x55c735cea140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:46.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.260+0000 7fd86f400640 1 --2- 172.21.15.97:0/869071532 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c735ce04a0 0x55c735cea140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:46.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.260+0000 7fd86f400640 1 -- 172.21.15.97:0/869071532 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c735ce5610 msgr2=0x55c735cf1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:46.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.260+0000 7fd86f400640 1 --2- 172.21.15.97:0/869071532 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c735ce5610 0x55c735cf1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:46.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.260+0000 7fd86f400640 1 -- 172.21.15.97:0/869071532 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c735a227d0 con 0x55c735cdfab0 2024-03-27T10:57:46.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.260+0000 7fd86f400640 1 --2- 172.21.15.97:0/869071532 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c735cdfab0 0x55c735ce9c00 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fd86000bfd0 tx=0x7fd8600118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe9af300e3279896 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:46.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.260+0000 7fd858c00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T10:57:46.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.261+0000 7fd858c00640 1 -- 172.21.15.97:0/869071532 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd860004020 con 0x55c735cdfab0 2024-03-27T10:57:46.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.261+0000 7fd858c00640 1 -- 172.21.15.97:0/869071532 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fd8600041c0 con 0x55c735cdfab0 2024-03-27T10:57:46.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.261+0000 7fd878e52740 1 -- 172.21.15.97:0/869071532 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd868010c90 con 0x55c735cdfab0 2024-03-27T10:57:46.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.261+0000 7fd858c00640 1 -- 172.21.15.97:0/869071532 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd860004400 con 0x55c735cdfab0 2024-03-27T10:57:46.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.262+0000 7fd878e52740 10 client.15957 fetch_fsmap learned FSMap version 28 2024-03-27T10:57:46.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.262+0000 7fd878e52740 1 -- 172.21.15.97:0/869071532 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c735cf4560 con 0x55c735cdfab0 2024-03-27T10:57:46.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.262+0000 7fd858c00640 1 -- 172.21.15.97:0/869071532 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd868010c90 con 0x55c735cdfab0 2024-03-27T10:57:46.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.263+0000 7fd858c00640 1 -- 172.21.15.97:0/869071532 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fd860003af0 con 0x55c735cdfab0 2024-03-27T10:57:46.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.263+0000 7fd878e52740 10 client.15957 fetch_fsmap finished waiting for FSMap version 28 2024-03-27T10:57:46.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.263+0000 7fd878e52740 10 client.15957 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-27T10:57:46.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.263+0000 7fd878e52740 20 client.15957 populate_metadata read hostname 'smithi097' 2024-03-27T10:57:46.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.263+0000 7fd878e52740 1 --2- 172.21.15.97:0/869071532 >> [v2:172.21.15.114:6836/616775588,v1:172.21.15.114:6837/616775588] conn(0x55c735cf5c80 0x55c735cf8140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:46.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.263+0000 7fd878e52740 4 client.15957 mds_command: new command op to 34478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T10:57:46.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.263+0000 7fd878e52740 1 -- 172.21.15.97:0/869071532 --> [v2:172.21.15.114:6836/616775588,v1:172.21.15.114:6837/616775588] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c735c51bb0 con 0x55c735cf5c80 2024-03-27T10:57:46.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.263+0000 7fd86ea00640 1 --2- 172.21.15.97:0/869071532 >> [v2:172.21.15.114:6836/616775588,v1:172.21.15.114:6837/616775588] conn(0x55c735cf5c80 0x55c735cf8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:46.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.264+0000 7fd858c00640 10 client.15957 ms_handle_connect on v2:172.21.15.114:6836/616775588 2024-03-27T10:57:46.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.264+0000 7fd86ea00640 1 --2- 172.21.15.97:0/869071532 >> [v2:172.21.15.114:6836/616775588,v1:172.21.15.114:6837/616775588] conn(0x55c735cf5c80 0x55c735cf8140 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=63d331a66d75fa9f server_cookie=47ee67a7c9f5fd4c in_seq=0 out_seq=0 2024-03-27T10:57:46.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.266+0000 7fd858c00640 1 -- 172.21.15.97:0/869071532 <== mds.1 v2:172.21.15.114:6836/616775588 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55c735c51bb0 con 0x55c735cf5c80 2024-03-27T10:57:46.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.266+0000 7fd858c00640 10 client.15957 handle_command_reply: tid=0 2024-03-27T10:57:46.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.266+0000 7fd878e52740 1 client.15957 shutdown 2024-03-27T10:57:46.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.267+0000 7fd878e52740 1 -- 172.21.15.97:0/869071532 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c735cdfab0 msgr2=0x55c735ce9c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:46.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.267+0000 7fd878e52740 1 --2- 172.21.15.97:0/869071532 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c735cdfab0 0x55c735ce9c00 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fd86000bfd0 tx=0x7fd8600118d0 comp rx=0 tx=0).stop 2024-03-27T10:57:46.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.267+0000 7fd878e52740 1 -- 172.21.15.97:0/869071532 shutdown_connections 2024-03-27T10:57:46.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.267+0000 7fd878e52740 1 --2- 172.21.15.97:0/869071532 >> [v2:172.21.15.114:6836/616775588,v1:172.21.15.114:6837/616775588] conn(0x55c735cf5c80 0x55c735cf8140 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:46.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.267+0000 7fd878e52740 1 --2- 172.21.15.97:0/869071532 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c735ce5610 0x55c735cf1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:46.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.267+0000 7fd878e52740 1 --2- 172.21.15.97:0/869071532 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c735ce04a0 0x55c735cea140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:46.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.267+0000 7fd878e52740 1 --2- 172.21.15.97:0/869071532 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c735cdfab0 0x55c735ce9c00 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:46.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.267+0000 7fd878e52740 1 -- 172.21.15.97:0/869071532 >> 172.21.15.97:0/869071532 conn(0x55c735cdb020 msgr2=0x55c735cda240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:46.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.267+0000 7fd858c00640 0 client.15957 ms_handle_reset on v2:172.21.15.114:6836/616775588 2024-03-27T10:57:46.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.267+0000 7fd878e52740 1 -- 172.21.15.97:0/869071532 shutdown_connections 2024-03-27T10:57:46.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.267+0000 7fd878e52740 1 -- 172.21.15.97:0/869071532 wait complete. 2024-03-27T10:57:46.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.267+0000 7fd878e52740 20 client.15957 trim_cache size 0 max 16384 2024-03-27T10:57:46.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.294+0000 7fd878e52740 1 Processor -- start 2024-03-27T10:57:46.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.294+0000 7fd878e52740 1 -- start start 2024-03-27T10:57:46.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.295+0000 7fd878e52740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c735cdb020 0x55c735cde990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:46.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.295+0000 7fd878e52740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c735cdfab0 0x55c735cf1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:46.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.295+0000 7fd878e52740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c735ce04a0 0x55c735ce9220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:46.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.295+0000 7fd878e52740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd8701af360 con 0x55c735cdb020 2024-03-27T10:57:46.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.295+0000 7fd878e52740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd8701b0ea0 con 0x55c735ce04a0 2024-03-27T10:57:46.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.295+0000 7fd878e52740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd8701afa30 con 0x55c735cdfab0 2024-03-27T10:57:46.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.295+0000 7fd86f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c735cdb020 0x55c735cde990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:46.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.295+0000 7fd86f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c735cdb020 0x55c735cde990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54018/0 (socket says 172.21.15.97:54018) 2024-03-27T10:57:46.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.295+0000 7fd86f400640 1 -- 172.21.15.97:0/3560858332 learned_addr learned my addr 172.21.15.97:0/3560858332 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:46.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.295+0000 7fd86ea00640 1 --2- 172.21.15.97:0/3560858332 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c735cdfab0 0x55c735cf1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:46.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.295+0000 7fd86fe00640 1 --2- 172.21.15.97:0/3560858332 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c735ce04a0 0x55c735ce9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:46.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.296+0000 7fd86f400640 1 -- 172.21.15.97:0/3560858332 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c735cdfab0 msgr2=0x55c735cf1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:46.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.296+0000 7fd86f400640 1 --2- 172.21.15.97:0/3560858332 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c735cdfab0 0x55c735cf1140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:46.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.296+0000 7fd86f400640 1 -- 172.21.15.97:0/3560858332 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c735ce04a0 msgr2=0x55c735ce9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:46.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.296+0000 7fd86f400640 1 --2- 172.21.15.97:0/3560858332 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c735ce04a0 0x55c735ce9220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:46.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.296+0000 7fd86f400640 1 -- 172.21.15.97:0/3560858332 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c735cdbd90 con 0x55c735cdb020 2024-03-27T10:57:46.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.296+0000 7fd86f400640 1 --2- 172.21.15.97:0/3560858332 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c735cdb020 0x55c735cde990 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fd86001a3e0 tx=0x7fd86000b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a40b7ff6cdd0d636 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:46.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.296+0000 7fd85b400640 1 -- 172.21.15.97:0/3560858332 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd860030020 con 0x55c735cdb020 2024-03-27T10:57:46.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.296+0000 7fd85b400640 1 -- 172.21.15.97:0/3560858332 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd86000b480 con 0x55c735cdb020 2024-03-27T10:57:46.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.296+0000 7fd85b400640 1 -- 172.21.15.97:0/3560858332 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd860000e60 con 0x55c735cdb020 2024-03-27T10:57:46.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.297+0000 7fd878e52740 1 -- 172.21.15.97:0/3560858332 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c735cdb020 msgr2=0x55c735cde990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:46.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.297+0000 7fd878e52740 1 --2- 172.21.15.97:0/3560858332 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c735cdb020 0x55c735cde990 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fd86001a3e0 tx=0x7fd86000b940 comp rx=0 tx=0).stop 2024-03-27T10:57:46.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.298+0000 7fd878e52740 1 -- 172.21.15.97:0/3560858332 shutdown_connections 2024-03-27T10:57:46.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.298+0000 7fd878e52740 1 --2- 172.21.15.97:0/3560858332 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c735ce04a0 0x55c735ce9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:46.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.298+0000 7fd878e52740 1 --2- 172.21.15.97:0/3560858332 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c735cdfab0 0x55c735cf1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:46.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.298+0000 7fd878e52740 1 --2- 172.21.15.97:0/3560858332 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c735cdb020 0x55c735cde990 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:46.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.298+0000 7fd878e52740 1 -- 172.21.15.97:0/3560858332 >> 172.21.15.97:0/3560858332 conn(0x55c735ce10a0 msgr2=0x55c735cde280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:46.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.298+0000 7fd878e52740 1 -- 172.21.15.97:0/3560858332 shutdown_connections 2024-03-27T10:57:46.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.298+0000 7fd878e52740 1 -- 172.21.15.97:0/3560858332 wait complete. 2024-03-27T10:57:46.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.298+0000 7fd878e52740 1 -- start start 2024-03-27T10:57:46.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.299+0000 7fd878e52740 1 Processor -- start 2024-03-27T10:57:46.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.299+0000 7fd878e52740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c735cdb020 0x55c735cee360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:46.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.299+0000 7fd878e52740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c735cdfab0 0x55c735cee8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:46.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.299+0000 7fd878e52740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c735ce04a0 0x55c735cef8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:46.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.299+0000 7fd878e52740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd8701b07d0 con 0x55c735cdb020 2024-03-27T10:57:46.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.299+0000 7fd878e52740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd8701b0100 con 0x55c735ce04a0 2024-03-27T10:57:46.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.299+0000 7fd878e52740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd8701ad340 con 0x55c735cdfab0 2024-03-27T10:57:46.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.299+0000 7fd878e52740 10 client.0 _ll_register_callbacks cb 0x7fd86000bc10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:57:46.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.299+0000 7fd86f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c735cdb020 0x55c735cee360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:46.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.300+0000 7fd86f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c735cdb020 0x55c735cee360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54034/0 (socket says 172.21.15.97:54034) 2024-03-27T10:57:46.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.300+0000 7fd86f400640 1 -- 172.21.15.97:0/4241092592 learned_addr learned my addr 172.21.15.97:0/4241092592 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:46.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.300+0000 7fd86ea00640 1 --2- 172.21.15.97:0/4241092592 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c735cdfab0 0x55c735cee8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:46.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.300+0000 7fd86fe00640 1 --2- 172.21.15.97:0/4241092592 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c735ce04a0 0x55c735cef8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:46.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.300+0000 7fd86f400640 1 -- 172.21.15.97:0/4241092592 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c735cdfab0 msgr2=0x55c735cee8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:46.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.300+0000 7fd86f400640 1 --2- 172.21.15.97:0/4241092592 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c735cdfab0 0x55c735cee8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:46.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.300+0000 7fd86f400640 1 -- 172.21.15.97:0/4241092592 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c735ce04a0 msgr2=0x55c735cef8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:46.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.300+0000 7fd86f400640 1 --2- 172.21.15.97:0/4241092592 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c735ce04a0 0x55c735cef8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:46.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.300+0000 7fd86f400640 1 -- 172.21.15.97:0/4241092592 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c735cd68a0 con 0x55c735cdb020 2024-03-27T10:57:46.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.300+0000 7fd86f400640 1 --2- 172.21.15.97:0/4241092592 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c735cdb020 0x55c735cee360 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fd860005d00 tx=0x7fd860010e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdc199996b93c94c server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:46.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.300+0000 7fd858c00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T10:57:46.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.301+0000 7fd858c00640 1 -- 172.21.15.97:0/4241092592 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd860030070 con 0x55c735cdb020 2024-03-27T10:57:46.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.301+0000 7fd858c00640 1 -- 172.21.15.97:0/4241092592 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fd860003800 con 0x55c735cdb020 2024-03-27T10:57:46.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.301+0000 7fd878e52740 1 -- 172.21.15.97:0/4241092592 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd86000b480 con 0x55c735cdb020 2024-03-27T10:57:46.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.301+0000 7fd858c00640 1 -- 172.21.15.97:0/4241092592 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd860028040 con 0x55c735cdb020 2024-03-27T10:57:46.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.301+0000 7fd878e52740 10 client.15963 fetch_fsmap learned FSMap version 28 2024-03-27T10:57:46.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.301+0000 7fd878e52740 1 -- 172.21.15.97:0/4241092592 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c735cf0fd0 con 0x55c735cdb020 2024-03-27T10:57:46.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.301+0000 7fd858c00640 1 -- 172.21.15.97:0/4241092592 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd86000b480 con 0x55c735cdb020 2024-03-27T10:57:46.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.302+0000 7fd858c00640 1 -- 172.21.15.97:0/4241092592 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fd86003d030 con 0x55c735cdb020 2024-03-27T10:57:46.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.302+0000 7fd878e52740 10 client.15963 fetch_fsmap finished waiting for FSMap version 28 2024-03-27T10:57:46.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.302+0000 7fd878e52740 10 client.15963 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-27T10:57:46.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.302+0000 7fd878e52740 20 client.15963 populate_metadata read hostname 'smithi097' 2024-03-27T10:57:46.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.302+0000 7fd878e52740 1 --2- 172.21.15.97:0/4241092592 >> [v2:172.21.15.114:6836/616775588,v1:172.21.15.114:6837/616775588] conn(0x55c735ce5610 0x55c735d5e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:46.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.302+0000 7fd878e52740 4 client.15963 mds_command: new command op to 34478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-27T10:57:46.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.302+0000 7fd878e52740 1 -- 172.21.15.97:0/4241092592 --> [v2:172.21.15.114:6836/616775588,v1:172.21.15.114:6837/616775588] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c735c4bf00 con 0x55c735ce5610 2024-03-27T10:57:46.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.302+0000 7fd86ea00640 1 --2- 172.21.15.97:0/4241092592 >> [v2:172.21.15.114:6836/616775588,v1:172.21.15.114:6837/616775588] conn(0x55c735ce5610 0x55c735d5e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:46.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.303+0000 7fd858c00640 10 client.15963 ms_handle_connect on v2:172.21.15.114:6836/616775588 2024-03-27T10:57:46.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.303+0000 7fd86ea00640 1 --2- 172.21.15.97:0/4241092592 >> [v2:172.21.15.114:6836/616775588,v1:172.21.15.114:6837/616775588] conn(0x55c735ce5610 0x55c735d5e3c0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fe4371ca02bd1956 server_cookie=7404666fc9b57dd in_seq=0 out_seq=0 2024-03-27T10:57:46.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.304+0000 7fd858c00640 1 -- 172.21.15.97:0/4241092592 <== mds.1 v2:172.21.15.114:6836/616775588 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c735c4bf00 con 0x55c735ce5610 2024-03-27T10:57:46.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.304+0000 7fd858c00640 10 client.15963 handle_command_reply: tid=0 2024-03-27T10:57:46.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.304+0000 7fd878e52740 1 client.15963 shutdown 2024-03-27T10:57:46.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.304+0000 7fd878e52740 1 -- 172.21.15.97:0/4241092592 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c735cdb020 msgr2=0x55c735cee360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:46.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.304+0000 7fd878e52740 1 --2- 172.21.15.97:0/4241092592 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c735cdb020 0x55c735cee360 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fd860005d00 tx=0x7fd860010e10 comp rx=0 tx=0).stop 2024-03-27T10:57:46.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.304+0000 7fd878e52740 1 -- 172.21.15.97:0/4241092592 shutdown_connections 2024-03-27T10:57:46.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.304+0000 7fd878e52740 1 --2- 172.21.15.97:0/4241092592 >> [v2:172.21.15.114:6836/616775588,v1:172.21.15.114:6837/616775588] conn(0x55c735ce5610 0x55c735d5e3c0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:46.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.304+0000 7fd878e52740 1 --2- 172.21.15.97:0/4241092592 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c735ce04a0 0x55c735cef8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:46.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.304+0000 7fd878e52740 1 --2- 172.21.15.97:0/4241092592 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c735cdfab0 0x55c735cee8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:46.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.304+0000 7fd878e52740 1 --2- 172.21.15.97:0/4241092592 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c735cdb020 0x55c735cee360 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:46.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.304+0000 7fd878e52740 1 -- 172.21.15.97:0/4241092592 >> 172.21.15.97:0/4241092592 conn(0x55c735ce10a0 msgr2=0x55c735cdeb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:46.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.304+0000 7fd858c00640 0 client.15963 ms_handle_reset on v2:172.21.15.114:6836/616775588 2024-03-27T10:57:46.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.304+0000 7fd878e52740 1 -- 172.21.15.97:0/4241092592 shutdown_connections 2024-03-27T10:57:46.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.304+0000 7fd878e52740 1 -- 172.21.15.97:0/4241092592 wait complete. 2024-03-27T10:57:46.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.304+0000 7fd878e52740 20 client.15963 trim_cache size 0 max 16384 2024-03-27T10:57:46.278 INFO:teuthology.orchestra.run.smithi097.stdout:[] 2024-03-27T10:57:46.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.305+0000 7fd874c00640 1 -- 172.21.15.97:0/2169147767 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fd8540a3690 msgr2=0x7fd8540a5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:46.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.305+0000 7fd874c00640 1 --2- 172.21.15.97:0/2169147767 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fd8540a3690 0x7fd8540a5b50 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fd868012fd0 tx=0x7fd868018000 comp rx=0 tx=0).stop 2024-03-27T10:57:46.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.305+0000 7fd874c00640 1 -- 172.21.15.97:0/2169147767 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd870102c10 msgr2=0x7fd87019e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:46.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.305+0000 7fd874c00640 1 --2- 172.21.15.97:0/2169147767 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd870102c10 0x7fd87019e790 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fd85c000c00 tx=0x7fd85c0042f0 comp rx=0 tx=0).stop 2024-03-27T10:57:46.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.305+0000 7fd874c00640 1 -- 172.21.15.97:0/2169147767 shutdown_connections 2024-03-27T10:57:46.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.305+0000 7fd874c00640 1 --2- 172.21.15.97:0/2169147767 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd8701055c0 0x7fd8701a57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:46.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.305+0000 7fd874c00640 1 --2- 172.21.15.97:0/2169147767 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd870102c10 0x7fd87019e790 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:46.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.305+0000 7fd874c00640 1 --2- 172.21.15.97:0/2169147767 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fd8540a3690 0x7fd8540a5b50 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:46.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.305+0000 7fd874c00640 1 --2- 172.21.15.97:0/2169147767 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd8701002a0 0x7fd87019e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:46.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.305+0000 7fd874c00640 1 -- 172.21.15.97:0/2169147767 >> 172.21.15.97:0/2169147767 conn(0x7fd8700fbec0 msgr2=0x7fd870100ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:46.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.306+0000 7fd874c00640 1 -- 172.21.15.97:0/2169147767 shutdown_connections 2024-03-27T10:57:46.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.306+0000 7fd874c00640 1 -- 172.21.15.97:0/2169147767 wait complete. 2024-03-27T10:57:46.301 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-27T10:57:46.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.516+0000 7f5e17e00640 1 -- 172.21.15.97:0/3572808045 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e181002a0 msgr2=0x7f5e181026d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:46.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.516+0000 7f5e17e00640 1 --2- 172.21.15.97:0/3572808045 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e181002a0 0x7f5e181026d0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f5e08009900 tx=0x7f5e0802bc30 comp rx=0 tx=0).stop 2024-03-27T10:57:46.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.516+0000 7f5e17e00640 1 -- 172.21.15.97:0/3572808045 shutdown_connections 2024-03-27T10:57:46.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.516+0000 7f5e17e00640 1 --2- 172.21.15.97:0/3572808045 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e181055c0 0x7f5e18105a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:46.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.516+0000 7f5e17e00640 1 --2- 172.21.15.97:0/3572808045 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5e18102c10 0x7f5e18105080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:46.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.516+0000 7f5e17e00640 1 --2- 172.21.15.97:0/3572808045 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e181002a0 0x7f5e181026d0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:46.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.516+0000 7f5e17e00640 1 -- 172.21.15.97:0/3572808045 >> 172.21.15.97:0/3572808045 conn(0x7f5e180fbec0 msgr2=0x7f5e180fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:46.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.517+0000 7f5e17e00640 1 -- 172.21.15.97:0/3572808045 shutdown_connections 2024-03-27T10:57:46.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.517+0000 7f5e17e00640 1 -- 172.21.15.97:0/3572808045 wait complete. 2024-03-27T10:57:46.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.518+0000 7f5e17e00640 1 Processor -- start 2024-03-27T10:57:46.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.518+0000 7f5e17e00640 1 -- start start 2024-03-27T10:57:46.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.519+0000 7f5e17e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e181002a0 0x7f5e1819e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:46.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.519+0000 7f5e17e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5e18102c10 0x7f5e1819e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:46.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.519+0000 7f5e17e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e181055c0 0x7f5e181a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:46.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.519+0000 7f5e17e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5e18113bd0 con 0x7f5e181002a0 2024-03-27T10:57:46.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.519+0000 7f5e17e00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f5e18113a50 con 0x7f5e18102c10 2024-03-27T10:57:46.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.519+0000 7f5e17e00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f5e18113d50 con 0x7f5e181055c0 2024-03-27T10:57:46.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.519+0000 7f5e16a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e181002a0 0x7f5e1819e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:46.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.519+0000 7f5e16a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e181002a0 0x7f5e1819e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54058/0 (socket says 172.21.15.97:54058) 2024-03-27T10:57:46.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.519+0000 7f5e16a00640 1 -- 172.21.15.97:0/4066360401 learned_addr learned my addr 172.21.15.97:0/4066360401 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:46.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.519+0000 7f5e17400640 1 --2- 172.21.15.97:0/4066360401 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e181055c0 0x7f5e181a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:46.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.519+0000 7f5e16000640 1 --2- 172.21.15.97:0/4066360401 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5e18102c10 0x7f5e1819e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:46.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.520+0000 7f5e16a00640 1 -- 172.21.15.97:0/4066360401 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e181055c0 msgr2=0x7f5e181a57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:46.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.520+0000 7f5e16a00640 1 --2- 172.21.15.97:0/4066360401 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e181055c0 0x7f5e181a57c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:46.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.520+0000 7f5e16a00640 1 -- 172.21.15.97:0/4066360401 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5e18102c10 msgr2=0x7f5e1819e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:46.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.520+0000 7f5e16a00640 1 --2- 172.21.15.97:0/4066360401 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5e18102c10 0x7f5e1819e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:46.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.520+0000 7f5e16a00640 1 -- 172.21.15.97:0/4066360401 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e181a5e30 con 0x7f5e181002a0 2024-03-27T10:57:46.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.520+0000 7f5e16a00640 1 --2- 172.21.15.97:0/4066360401 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e181002a0 0x7f5e1819e210 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f5e08000c00 tx=0x7f5e080042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:46.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.520+0000 7f5e0f400640 1 -- 172.21.15.97:0/4066360401 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e08043020 con 0x7f5e181002a0 2024-03-27T10:57:46.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.520+0000 7f5e0f400640 1 -- 172.21.15.97:0/4066360401 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5e08004aa0 con 0x7f5e181002a0 2024-03-27T10:57:46.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.520+0000 7f5e17e00640 1 -- 172.21.15.97:0/4066360401 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e181a60c0 con 0x7f5e181002a0 2024-03-27T10:57:46.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.520+0000 7f5e0f400640 1 -- 172.21.15.97:0/4066360401 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e08003d70 con 0x7f5e181002a0 2024-03-27T10:57:46.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.520+0000 7f5e17e00640 1 -- 172.21.15.97:0/4066360401 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e181a6660 con 0x7f5e181002a0 2024-03-27T10:57:46.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.525+0000 7f5e20569740 1 Processor -- start 2024-03-27T10:57:46.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.525+0000 7f5e0f400640 1 -- 172.21.15.97:0/4066360401 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f5e08046020 con 0x7f5e181002a0 2024-03-27T10:57:46.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.525+0000 7f5e20569740 1 -- start start 2024-03-27T10:57:46.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.526+0000 7f5e20569740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ec808f6ab0 0x55ec808f6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:46.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.526+0000 7f5e20569740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ec808f74a0 0x55ec808fbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:46.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.526+0000 7f5e20569740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ec808fc610 0x55ec808feaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:46.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.526+0000 7f5e20569740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ec808be7b0 con 0x55ec808fc610 2024-03-27T10:57:46.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.526+0000 7f5e20569740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55ec8085b670 con 0x55ec808f6ab0 2024-03-27T10:57:46.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.526+0000 7f5e20569740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55ec80803970 con 0x55ec808f74a0 2024-03-27T10:57:46.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.526+0000 7f5e16000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ec808fc610 0x55ec808feaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:46.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.526+0000 7f5e16a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ec808f74a0 0x55ec808fbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:46.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.526+0000 7f5e16000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ec808fc610 0x55ec808feaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54068/0 (socket says 172.21.15.97:54068) 2024-03-27T10:57:46.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.526+0000 7f5e16000640 1 -- 172.21.15.97:0/3282288171 learned_addr learned my addr 172.21.15.97:0/3282288171 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:46.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.527+0000 7f5e17400640 1 --2- 172.21.15.97:0/3282288171 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ec808f6ab0 0x55ec808f6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:46.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.527+0000 7f5e16000640 1 -- 172.21.15.97:0/3282288171 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ec808f74a0 msgr2=0x55ec808fbfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:46.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.527+0000 7f5e16000640 1 --2- 172.21.15.97:0/3282288171 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ec808f74a0 0x55ec808fbfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:46.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.527+0000 7f5e16000640 1 -- 172.21.15.97:0/3282288171 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ec808f6ab0 msgr2=0x55ec808f6ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:46.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.527+0000 7f5e16000640 1 --2- 172.21.15.97:0/3282288171 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ec808f6ab0 0x55ec808f6ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:46.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.527+0000 7f5e16000640 1 -- 172.21.15.97:0/3282288171 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec8056f3f0 con 0x55ec808fc610 2024-03-27T10:57:46.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.527+0000 7f5e16000640 1 --2- 172.21.15.97:0/3282288171 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ec808fc610 0x55ec808feaa0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f5e04009c20 tx=0x7f5e040117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1603f560e3b5e946 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:46.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.527+0000 7f5e0f400640 1 --2- 172.21.15.97:0/4066360401 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f5dfc0a3710 0x7f5dfc0a5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:46.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.528+0000 7f5e0f400640 1 -- 172.21.15.97:0/4066360401 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f5e080f2110 con 0x7f5e181002a0 2024-03-27T10:57:46.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.528+0000 7f5e17400640 1 --2- 172.21.15.97:0/4066360401 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f5dfc0a3710 0x7f5dfc0a5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:46.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.528+0000 7f5e03e00640 1 -- 172.21.15.97:0/3282288171 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e04012450 con 0x55ec808fc610 2024-03-27T10:57:46.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.528+0000 7f5e03e00640 1 -- 172.21.15.97:0/3282288171 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5e04012aa0 con 0x55ec808fc610 2024-03-27T10:57:46.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.528+0000 7f5e03e00640 1 -- 172.21.15.97:0/3282288171 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e0400fb10 con 0x55ec808fc610 2024-03-27T10:57:46.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.529+0000 7f5e20569740 1 -- 172.21.15.97:0/3282288171 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ec808fc610 msgr2=0x55ec808feaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:46.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.529+0000 7f5e17400640 1 --2- 172.21.15.97:0/4066360401 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f5dfc0a3710 0x7f5dfc0a5bd0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x55ec808f7d00 tx=0x7f5e10009580 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:46.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.529+0000 7f5e20569740 1 --2- 172.21.15.97:0/3282288171 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ec808fc610 0x55ec808feaa0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f5e04009c20 tx=0x7f5e040117b0 comp rx=0 tx=0).stop 2024-03-27T10:57:46.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.529+0000 7f5e20569740 1 -- 172.21.15.97:0/3282288171 shutdown_connections 2024-03-27T10:57:46.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.529+0000 7f5e20569740 1 --2- 172.21.15.97:0/3282288171 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ec808fc610 0x55ec808feaa0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:46.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.529+0000 7f5e20569740 1 --2- 172.21.15.97:0/3282288171 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ec808f74a0 0x55ec808fbfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:46.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.529+0000 7f5e20569740 1 --2- 172.21.15.97:0/3282288171 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ec808f6ab0 0x55ec808f6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:46.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.529+0000 7f5e20569740 1 -- 172.21.15.97:0/3282288171 >> 172.21.15.97:0/3282288171 conn(0x55ec808f2020 msgr2=0x55ec808f2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:46.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.529+0000 7f5e20569740 1 -- 172.21.15.97:0/3282288171 shutdown_connections 2024-03-27T10:57:46.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.529+0000 7f5e20569740 1 -- 172.21.15.97:0/3282288171 wait complete. 2024-03-27T10:57:46.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.530+0000 7f5e20569740 1 -- start start 2024-03-27T10:57:46.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.530+0000 7f5e20569740 1 Processor -- start 2024-03-27T10:57:46.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.531+0000 7f5e20569740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ec808f6ab0 0x55ec80900c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:46.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.531+0000 7f5e20569740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ec808f74a0 0x55ec809011a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:46.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.531+0000 7f5e20569740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ec808fc610 0x55ec809081a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:46.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.531+0000 7f5e20569740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ec80816ca0 con 0x55ec808fc610 2024-03-27T10:57:46.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.531+0000 7f5e20569740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55ec80857620 con 0x55ec808f74a0 2024-03-27T10:57:46.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.531+0000 7f5e20569740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55ec808ff5c0 con 0x55ec808f6ab0 2024-03-27T10:57:46.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.531+0000 7f5e20569740 10 client.0 _ll_register_callbacks cb 0x55ec80853160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:57:46.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.531+0000 7f5e17400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ec808fc610 0x55ec809081a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:46.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.531+0000 7f5e17400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ec808fc610 0x55ec809081a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54080/0 (socket says 172.21.15.97:54080) 2024-03-27T10:57:46.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.531+0000 7f5e17400640 1 -- 172.21.15.97:0/1971685163 learned_addr learned my addr 172.21.15.97:0/1971685163 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:46.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.531+0000 7f5e16a00640 1 --2- 172.21.15.97:0/1971685163 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ec808f6ab0 0x55ec80900c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:46.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.531+0000 7f5e16000640 1 --2- 172.21.15.97:0/1971685163 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ec808f74a0 0x55ec809011a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:46.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.532+0000 7f5e17400640 1 -- 172.21.15.97:0/1971685163 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ec808f6ab0 msgr2=0x55ec80900c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:46.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.532+0000 7f5e17400640 1 --2- 172.21.15.97:0/1971685163 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ec808f6ab0 0x55ec80900c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:46.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.532+0000 7f5e17400640 1 -- 172.21.15.97:0/1971685163 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ec808f74a0 msgr2=0x55ec809011a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:46.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.532+0000 7f5e17400640 1 --2- 172.21.15.97:0/1971685163 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ec808f74a0 0x55ec809011a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:46.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.532+0000 7f5e17400640 1 -- 172.21.15.97:0/1971685163 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec806397d0 con 0x55ec808fc610 2024-03-27T10:57:46.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.532+0000 7f5e17400640 1 --2- 172.21.15.97:0/1971685163 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ec808fc610 0x55ec809081a0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f5e10012930 tx=0x7f5e10004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eacd5ad674dbc87d server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:46.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.532+0000 7f5e00c00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T10:57:46.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.533+0000 7f5e00c00640 1 -- 172.21.15.97:0/1971685163 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e10004d20 con 0x55ec808fc610 2024-03-27T10:57:46.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.533+0000 7f5e00c00640 1 -- 172.21.15.97:0/1971685163 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f5e1000eae0 con 0x55ec808fc610 2024-03-27T10:57:46.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.533+0000 7f5e20569740 1 -- 172.21.15.97:0/1971685163 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e04012aa0 con 0x55ec808fc610 2024-03-27T10:57:46.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.533+0000 7f5e00c00640 1 -- 172.21.15.97:0/1971685163 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5e1000ece0 con 0x55ec808fc610 2024-03-27T10:57:46.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.533+0000 7f5e20569740 10 client.15975 fetch_fsmap learned FSMap version 28 2024-03-27T10:57:46.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.533+0000 7f5e20569740 1 -- 172.21.15.97:0/1971685163 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ec8090b600 con 0x55ec808fc610 2024-03-27T10:57:46.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.533+0000 7f5e00c00640 1 -- 172.21.15.97:0/1971685163 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5e04012aa0 con 0x55ec808fc610 2024-03-27T10:57:46.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.534+0000 7f5e00c00640 1 -- 172.21.15.97:0/1971685163 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f5e1000f040 con 0x55ec808fc610 2024-03-27T10:57:46.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.534+0000 7f5e20569740 10 client.15975 fetch_fsmap finished waiting for FSMap version 28 2024-03-27T10:57:46.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.534+0000 7f5e20569740 10 client.15975 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-27T10:57:46.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.534+0000 7f5e20569740 20 client.15975 populate_metadata read hostname 'smithi097' 2024-03-27T10:57:46.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.534+0000 7f5e20569740 1 --2- 172.21.15.97:0/1971685163 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55ec8090cd50 0x55ec8090f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:46.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.534+0000 7f5e20569740 4 client.15975 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T10:57:46.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.534+0000 7f5e20569740 1 -- 172.21.15.97:0/1971685163 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ec80868bb0 con 0x55ec8090cd50 2024-03-27T10:57:46.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.535+0000 7f5e16a00640 1 --2- 172.21.15.97:0/1971685163 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55ec8090cd50 0x55ec8090f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:46.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.537+0000 7f5e00c00640 10 client.15975 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T10:57:46.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.537+0000 7f5e16a00640 1 --2- 172.21.15.97:0/1971685163 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55ec8090cd50 0x55ec8090f1f0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2373c7ac7ea5a507 server_cookie=63b73e0f17bb4bdd in_seq=0 out_seq=0 2024-03-27T10:57:46.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.538+0000 7f5e00c00640 1 -- 172.21.15.97:0/1971685163 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55ec80868bb0 con 0x55ec8090cd50 2024-03-27T10:57:46.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.538+0000 7f5e00c00640 10 client.15975 handle_command_reply: tid=0 2024-03-27T10:57:46.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.538+0000 7f5e20569740 1 client.15975 shutdown 2024-03-27T10:57:46.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.540+0000 7f5e20569740 1 -- 172.21.15.97:0/1971685163 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ec808fc610 msgr2=0x55ec809081a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:46.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.540+0000 7f5e20569740 1 --2- 172.21.15.97:0/1971685163 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ec808fc610 0x55ec809081a0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f5e10012930 tx=0x7f5e10004a40 comp rx=0 tx=0).stop 2024-03-27T10:57:46.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.540+0000 7f5e20569740 1 -- 172.21.15.97:0/1971685163 shutdown_connections 2024-03-27T10:57:46.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.540+0000 7f5e20569740 1 --2- 172.21.15.97:0/1971685163 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55ec8090cd50 0x55ec8090f1f0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:46.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.540+0000 7f5e20569740 1 --2- 172.21.15.97:0/1971685163 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ec808fc610 0x55ec809081a0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:46.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.540+0000 7f5e20569740 1 --2- 172.21.15.97:0/1971685163 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ec808f74a0 0x55ec809011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:46.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.540+0000 7f5e00c00640 0 client.15975 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T10:57:46.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.540+0000 7f5e20569740 1 --2- 172.21.15.97:0/1971685163 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ec808f6ab0 0x55ec80900c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:46.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.540+0000 7f5e20569740 1 -- 172.21.15.97:0/1971685163 >> 172.21.15.97:0/1971685163 conn(0x55ec808f2020 msgr2=0x55ec808f1240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:46.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.541+0000 7f5e20569740 1 -- 172.21.15.97:0/1971685163 shutdown_connections 2024-03-27T10:57:46.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.541+0000 7f5e20569740 1 -- 172.21.15.97:0/1971685163 wait complete. 2024-03-27T10:57:46.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.541+0000 7f5e20569740 20 client.15975 trim_cache size 0 max 16384 2024-03-27T10:57:46.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.568+0000 7f5e20569740 1 Processor -- start 2024-03-27T10:57:46.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.568+0000 7f5e20569740 1 -- start start 2024-03-27T10:57:46.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.568+0000 7f5e20569740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ec808f6ab0 0x55ec808f87c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:46.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.568+0000 7f5e20569740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ec808f74a0 0x55ec80908160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:46.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.568+0000 7f5e20569740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ec808fc610 0x55ec808f8d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:46.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.568+0000 7f5e20569740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5e181af2b0 con 0x55ec808fc610 2024-03-27T10:57:46.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.568+0000 7f5e20569740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f5e181b0df0 con 0x55ec808f74a0 2024-03-27T10:57:46.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.568+0000 7f5e20569740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f5e181af980 con 0x55ec808f6ab0 2024-03-27T10:57:46.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.569+0000 7f5e17400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ec808fc610 0x55ec808f8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:46.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.569+0000 7f5e17400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ec808fc610 0x55ec808f8d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54084/0 (socket says 172.21.15.97:54084) 2024-03-27T10:57:46.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.569+0000 7f5e17400640 1 -- 172.21.15.97:0/1588645816 learned_addr learned my addr 172.21.15.97:0/1588645816 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:46.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.569+0000 7f5e16000640 1 --2- 172.21.15.97:0/1588645816 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ec808f74a0 0x55ec80908160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:46.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.569+0000 7f5e17400640 1 -- 172.21.15.97:0/1588645816 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ec808f6ab0 msgr2=0x55ec808f87c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:46.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.569+0000 7f5e16a00640 1 --2- 172.21.15.97:0/1588645816 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ec808f6ab0 0x55ec808f87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:46.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.569+0000 7f5e17400640 1 --2- 172.21.15.97:0/1588645816 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ec808f6ab0 0x55ec808f87c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:46.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.569+0000 7f5e17400640 1 -- 172.21.15.97:0/1588645816 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ec808f74a0 msgr2=0x55ec80908160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:46.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.569+0000 7f5e17400640 1 --2- 172.21.15.97:0/1588645816 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ec808f74a0 0x55ec80908160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:46.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.569+0000 7f5e17400640 1 -- 172.21.15.97:0/1588645816 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec808fc360 con 0x55ec808fc610 2024-03-27T10:57:46.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.570+0000 7f5e17400640 1 --2- 172.21.15.97:0/1588645816 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ec808fc610 0x55ec808f8d00 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f5e100124d0 tx=0x7f5e10000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=745fd2b22c85bd11 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:46.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.570+0000 7f5e03400640 1 -- 172.21.15.97:0/1588645816 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e1000eae0 con 0x55ec808fc610 2024-03-27T10:57:46.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.570+0000 7f5e03400640 1 -- 172.21.15.97:0/1588645816 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5e1001f6e0 con 0x55ec808fc610 2024-03-27T10:57:46.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.570+0000 7f5e03400640 1 -- 172.21.15.97:0/1588645816 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e10032690 con 0x55ec808fc610 2024-03-27T10:57:46.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.571+0000 7f5e20569740 1 -- 172.21.15.97:0/1588645816 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ec808fc610 msgr2=0x55ec808f8d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:46.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.571+0000 7f5e20569740 1 --2- 172.21.15.97:0/1588645816 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ec808fc610 0x55ec808f8d00 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f5e100124d0 tx=0x7f5e10000f40 comp rx=0 tx=0).stop 2024-03-27T10:57:46.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.571+0000 7f5e20569740 1 -- 172.21.15.97:0/1588645816 shutdown_connections 2024-03-27T10:57:46.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.571+0000 7f5e20569740 1 --2- 172.21.15.97:0/1588645816 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ec808fc610 0x55ec808f8d00 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:46.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.571+0000 7f5e20569740 1 --2- 172.21.15.97:0/1588645816 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ec808f74a0 0x55ec80908160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:46.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.571+0000 7f5e20569740 1 --2- 172.21.15.97:0/1588645816 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ec808f6ab0 0x55ec808f87c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:46.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.571+0000 7f5e20569740 1 -- 172.21.15.97:0/1588645816 >> 172.21.15.97:0/1588645816 conn(0x55ec808f2020 msgr2=0x55ec80900c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:46.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.571+0000 7f5e20569740 1 -- 172.21.15.97:0/1588645816 shutdown_connections 2024-03-27T10:57:46.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.571+0000 7f5e20569740 1 -- 172.21.15.97:0/1588645816 wait complete. 2024-03-27T10:57:46.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.572+0000 7f5e20569740 1 -- start start 2024-03-27T10:57:46.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.572+0000 7f5e20569740 1 Processor -- start 2024-03-27T10:57:46.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.573+0000 7f5e20569740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ec808f6ab0 0x55ec80905720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:46.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.573+0000 7f5e20569740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ec808f74a0 0x55ec80905c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:46.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.573+0000 7f5e20569740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ec808fc610 0x55ec80906c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:46.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.573+0000 7f5e20569740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5e181b0720 con 0x55ec808f6ab0 2024-03-27T10:57:46.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.573+0000 7f5e20569740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f5e181b0050 con 0x55ec808fc610 2024-03-27T10:57:46.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.573+0000 7f5e20569740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f5e181ad290 con 0x55ec808f74a0 2024-03-27T10:57:46.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.573+0000 7f5e20569740 10 client.0 _ll_register_callbacks cb 0x7f5e10009630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:57:46.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.573+0000 7f5e16a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ec808f6ab0 0x55ec80905720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:46.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.573+0000 7f5e16a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ec808f6ab0 0x55ec80905720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54086/0 (socket says 172.21.15.97:54086) 2024-03-27T10:57:46.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.573+0000 7f5e16a00640 1 -- 172.21.15.97:0/1926005394 learned_addr learned my addr 172.21.15.97:0/1926005394 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:46.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.573+0000 7f5e17400640 1 --2- 172.21.15.97:0/1926005394 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ec808fc610 0x55ec80906c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:46.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.573+0000 7f5e16000640 1 --2- 172.21.15.97:0/1926005394 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ec808f74a0 0x55ec80905c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:46.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.573+0000 7f5e16a00640 1 -- 172.21.15.97:0/1926005394 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ec808f74a0 msgr2=0x55ec80905c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:46.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.574+0000 7f5e16a00640 1 --2- 172.21.15.97:0/1926005394 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ec808f74a0 0x55ec80905c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:46.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.574+0000 7f5e16a00640 1 -- 172.21.15.97:0/1926005394 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ec808fc610 msgr2=0x55ec80906c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:46.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.574+0000 7f5e16a00640 1 --2- 172.21.15.97:0/1926005394 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ec808fc610 0x55ec80906c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:46.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.574+0000 7f5e16a00640 1 -- 172.21.15.97:0/1926005394 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec80907260 con 0x55ec808f6ab0 2024-03-27T10:57:46.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.574+0000 7f5e16000640 1 --2- 172.21.15.97:0/1926005394 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ec808f74a0 0x55ec80905c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:57:46.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.574+0000 7f5e16a00640 1 --2- 172.21.15.97:0/1926005394 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ec808f6ab0 0x55ec80905720 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f5e0802ce10 tx=0x7f5e08009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba51f83a4dba7cb8 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:46.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.574+0000 7f5e00c00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T10:57:46.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.574+0000 7f5e00c00640 1 -- 172.21.15.97:0/1926005394 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e080be360 con 0x55ec808f6ab0 2024-03-27T10:57:46.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.574+0000 7f5e00c00640 1 -- 172.21.15.97:0/1926005394 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f5e080035f0 con 0x55ec808f6ab0 2024-03-27T10:57:46.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.575+0000 7f5e20569740 1 -- 172.21.15.97:0/1926005394 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e1001f6e0 con 0x55ec808f6ab0 2024-03-27T10:57:46.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.575+0000 7f5e00c00640 1 -- 172.21.15.97:0/1926005394 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5e080bdef0 con 0x55ec808f6ab0 2024-03-27T10:57:46.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.575+0000 7f5e20569740 10 client.15981 fetch_fsmap learned FSMap version 28 2024-03-27T10:57:46.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.575+0000 7f5e20569740 1 -- 172.21.15.97:0/1926005394 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ec809730f0 con 0x55ec808f6ab0 2024-03-27T10:57:46.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.575+0000 7f5e00c00640 1 -- 172.21.15.97:0/1926005394 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5e1001f6e0 con 0x55ec808f6ab0 2024-03-27T10:57:46.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.575+0000 7f5e00c00640 1 -- 172.21.15.97:0/1926005394 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f5e080bac00 con 0x55ec808f6ab0 2024-03-27T10:57:46.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.575+0000 7f5e20569740 10 client.15981 fetch_fsmap finished waiting for FSMap version 28 2024-03-27T10:57:46.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.575+0000 7f5e20569740 10 client.15981 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-27T10:57:46.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.575+0000 7f5e20569740 20 client.15981 populate_metadata read hostname 'smithi097' 2024-03-27T10:57:46.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.575+0000 7f5e20569740 1 --2- 172.21.15.97:0/1926005394 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55ec809739a0 0x55ec80975dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:46.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.575+0000 7f5e20569740 4 client.15981 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-27T10:57:46.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.575+0000 7f5e20569740 1 -- 172.21.15.97:0/1926005394 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ec80862f00 con 0x55ec809739a0 2024-03-27T10:57:46.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.576+0000 7f5e16000640 1 --2- 172.21.15.97:0/1926005394 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55ec809739a0 0x55ec80975dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:46.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.577+0000 7f5e16000640 1 --2- 172.21.15.97:0/1926005394 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55ec809739a0 0x55ec80975dc0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38971b852ec40270 server_cookie=dde263fe7c9f9727 in_seq=0 out_seq=0 2024-03-27T10:57:46.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.577+0000 7f5e00c00640 10 client.15981 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T10:57:46.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.577+0000 7f5e00c00640 1 -- 172.21.15.97:0/1926005394 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ec80862f00 con 0x55ec809739a0 2024-03-27T10:57:46.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.577+0000 7f5e00c00640 10 client.15981 handle_command_reply: tid=0 2024-03-27T10:57:46.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.577+0000 7f5e20569740 1 client.15981 shutdown 2024-03-27T10:57:46.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.577+0000 7f5e20569740 1 -- 172.21.15.97:0/1926005394 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ec808f6ab0 msgr2=0x55ec80905720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:46.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.577+0000 7f5e20569740 1 --2- 172.21.15.97:0/1926005394 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ec808f6ab0 0x55ec80905720 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f5e0802ce10 tx=0x7f5e08009620 comp rx=0 tx=0).stop 2024-03-27T10:57:46.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.577+0000 7f5e20569740 1 -- 172.21.15.97:0/1926005394 shutdown_connections 2024-03-27T10:57:46.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.578+0000 7f5e20569740 1 --2- 172.21.15.97:0/1926005394 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55ec809739a0 0x55ec80975dc0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:46.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.578+0000 7f5e00c00640 0 client.15981 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T10:57:46.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.578+0000 7f5e20569740 1 --2- 172.21.15.97:0/1926005394 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ec808fc610 0x55ec80906c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:46.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.578+0000 7f5e20569740 1 --2- 172.21.15.97:0/1926005394 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ec808f74a0 0x55ec80905c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:46.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.578+0000 7f5e20569740 1 --2- 172.21.15.97:0/1926005394 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ec808f6ab0 0x55ec80905720 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:46.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.578+0000 7f5e20569740 1 -- 172.21.15.97:0/1926005394 >> 172.21.15.97:0/1926005394 conn(0x55ec808f2020 msgr2=0x55ec80900c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:46.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.578+0000 7f5e20569740 1 -- 172.21.15.97:0/1926005394 shutdown_connections 2024-03-27T10:57:46.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.578+0000 7f5e20569740 1 -- 172.21.15.97:0/1926005394 wait complete. 2024-03-27T10:57:46.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.578+0000 7f5e20569740 20 client.15981 trim_cache size 0 max 16384 2024-03-27T10:57:46.554 INFO:teuthology.orchestra.run.smithi097.stdout:[] 2024-03-27T10:57:46.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.579+0000 7f5e17e00640 1 -- 172.21.15.97:0/4066360401 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f5dfc0a3710 msgr2=0x7f5dfc0a5bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:46.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.579+0000 7f5e17e00640 1 --2- 172.21.15.97:0/4066360401 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f5dfc0a3710 0x7f5dfc0a5bd0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x55ec808f7d00 tx=0x7f5e10009580 comp rx=0 tx=0).stop 2024-03-27T10:57:46.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.579+0000 7f5e17e00640 1 -- 172.21.15.97:0/4066360401 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e181002a0 msgr2=0x7f5e1819e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:46.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.579+0000 7f5e17e00640 1 --2- 172.21.15.97:0/4066360401 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e181002a0 0x7f5e1819e210 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f5e08000c00 tx=0x7f5e080042f0 comp rx=0 tx=0).stop 2024-03-27T10:57:46.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.580+0000 7f5e17e00640 1 -- 172.21.15.97:0/4066360401 shutdown_connections 2024-03-27T10:57:46.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.580+0000 7f5e17e00640 1 --2- 172.21.15.97:0/4066360401 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e181055c0 0x7f5e181a57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:46.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.580+0000 7f5e17e00640 1 --2- 172.21.15.97:0/4066360401 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f5dfc0a3710 0x7f5dfc0a5bd0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:46.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.580+0000 7f5e17e00640 1 --2- 172.21.15.97:0/4066360401 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5e18102c10 0x7f5e1819e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:46.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.580+0000 7f5e17e00640 1 --2- 172.21.15.97:0/4066360401 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e181002a0 0x7f5e1819e210 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:46.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.580+0000 7f5e17e00640 1 -- 172.21.15.97:0/4066360401 >> 172.21.15.97:0/4066360401 conn(0x7f5e180fbec0 msgr2=0x7f5e181035d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:46.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.580+0000 7f5e17e00640 1 -- 172.21.15.97:0/4066360401 shutdown_connections 2024-03-27T10:57:46.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:46.580+0000 7f5e17e00640 1 -- 172.21.15.97:0/4066360401 wait complete. 2024-03-27T10:57:47.575 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-27T10:57:47.575 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-27T10:57:47.575 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-27T10:57:47.636 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:57:47 smithi097 ceph-mon[31859]: pgmap v475: 97 pgs: 97 active+clean; 665 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-27T10:57:47.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:57:47 smithi177 ceph-mon[35294]: pgmap v475: 97 pgs: 97 active+clean; 665 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-27T10:57:47.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.787+0000 7f4a6d800640 1 -- 172.21.15.97:0/2925971804 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a68107840 msgr2=0x7f4a68109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:47.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.787+0000 7f4a6d800640 1 --2- 172.21.15.97:0/2925971804 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a68107840 0x7f4a68109c90 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f4a60009ca0 tx=0x7f4a6002c0f0 comp rx=0 tx=0).stop 2024-03-27T10:57:47.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.787+0000 7f4a6d800640 1 -- 172.21.15.97:0/2925971804 shutdown_connections 2024-03-27T10:57:47.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.787+0000 7f4a6d800640 1 --2- 172.21.15.97:0/2925971804 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a68107840 0x7f4a68109c90 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:47.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.787+0000 7f4a6d800640 1 --2- 172.21.15.97:0/2925971804 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4a68104f10 0x7f4a68107300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:47.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.787+0000 7f4a6d800640 1 --2- 172.21.15.97:0/2925971804 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4a681025e0 0x7f4a681049d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:47.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.787+0000 7f4a6d800640 1 -- 172.21.15.97:0/2925971804 >> 172.21.15.97:0/2925971804 conn(0x7f4a680fbff0 msgr2=0x7f4a680fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:47.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.787+0000 7f4a6d800640 1 -- 172.21.15.97:0/2925971804 shutdown_connections 2024-03-27T10:57:47.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.787+0000 7f4a6d800640 1 -- 172.21.15.97:0/2925971804 wait complete. 2024-03-27T10:57:47.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.788+0000 7f4a6d800640 1 Processor -- start 2024-03-27T10:57:47.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.788+0000 7f4a6d800640 1 -- start start 2024-03-27T10:57:47.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.789+0000 7f4a6d800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4a681025e0 0x7f4a68197d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:47.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.789+0000 7f4a6d800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4a68104f10 0x7f4a68198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:47.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.789+0000 7f4a6d800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a68107840 0x7f4a6819f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:47.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.789+0000 7f4a6d800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4a6810d690 con 0x7f4a68107840 2024-03-27T10:57:47.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.789+0000 7f4a6d800640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f4a6810d510 con 0x7f4a681025e0 2024-03-27T10:57:47.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.789+0000 7f4a6d800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f4a6810d810 con 0x7f4a68104f10 2024-03-27T10:57:47.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.789+0000 7f4a67400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a68107840 0x7f4a6819f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:47.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.789+0000 7f4a66a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4a681025e0 0x7f4a68197d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:47.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.789+0000 7f4a67400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a68107840 0x7f4a6819f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54110/0 (socket says 172.21.15.97:54110) 2024-03-27T10:57:47.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.789+0000 7f4a67400640 1 -- 172.21.15.97:0/3018858117 learned_addr learned my addr 172.21.15.97:0/3018858117 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:47.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.789+0000 7f4a66000640 1 --2- 172.21.15.97:0/3018858117 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4a68104f10 0x7f4a68198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:47.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.790+0000 7f4a67400640 1 -- 172.21.15.97:0/3018858117 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4a68104f10 msgr2=0x7f4a68198260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:47.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.790+0000 7f4a67400640 1 --2- 172.21.15.97:0/3018858117 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4a68104f10 0x7f4a68198260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:47.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.790+0000 7f4a67400640 1 -- 172.21.15.97:0/3018858117 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4a681025e0 msgr2=0x7f4a68197d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:47.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.790+0000 7f4a67400640 1 --2- 172.21.15.97:0/3018858117 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4a681025e0 0x7f4a68197d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:47.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.790+0000 7f4a67400640 1 -- 172.21.15.97:0/3018858117 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a6819f930 con 0x7f4a68107840 2024-03-27T10:57:47.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.790+0000 7f4a67400640 1 --2- 172.21.15.97:0/3018858117 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a68107840 0x7f4a6819f290 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f4a60032380 tx=0x7f4a60032820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:47.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.790+0000 7f4a5f400640 1 -- 172.21.15.97:0/3018858117 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a60041020 con 0x7f4a68107840 2024-03-27T10:57:47.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.791+0000 7f4a5f400640 1 -- 172.21.15.97:0/3018858117 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4a60003e20 con 0x7f4a68107840 2024-03-27T10:57:47.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.791+0000 7f4a6d800640 1 -- 172.21.15.97:0/3018858117 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a6819fbc0 con 0x7f4a68107840 2024-03-27T10:57:47.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.791+0000 7f4a5f400640 1 -- 172.21.15.97:0/3018858117 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a60039040 con 0x7f4a68107840 2024-03-27T10:57:47.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.791+0000 7f4a6d800640 1 -- 172.21.15.97:0/3018858117 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a681005b0 con 0x7f4a68107840 2024-03-27T10:57:47.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.794+0000 7f4a7042a740 1 Processor -- start 2024-03-27T10:57:47.766 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:57:47 smithi114 ceph-mon[35338]: pgmap v475: 97 pgs: 97 active+clean; 665 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-27T10:57:47.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.795+0000 7f4a5f400640 1 -- 172.21.15.97:0/3018858117 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f4a60044020 con 0x7f4a68107840 2024-03-27T10:57:47.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.796+0000 7f4a7042a740 1 -- start start 2024-03-27T10:57:47.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.796+0000 7f4a7042a740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5581dd842f00 0x5581dd843320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:47.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.796+0000 7f4a7042a740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581dd8440d0 0x5581dd844550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:47.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.796+0000 7f4a7042a740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5581dd844a90 0x5581dd84b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:47.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.796+0000 7f4a7042a740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5581dd7a8280 con 0x5581dd8440d0 2024-03-27T10:57:47.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.796+0000 7f4a7042a740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5581dd7506f0 con 0x5581dd844a90 2024-03-27T10:57:47.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.796+0000 7f4a7042a740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5581dd7a4230 con 0x5581dd842f00 2024-03-27T10:57:47.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.797+0000 7f4a66a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581dd8440d0 0x5581dd844550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:47.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.797+0000 7f4a66000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5581dd842f00 0x5581dd843320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:47.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.797+0000 7f4a66000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5581dd842f00 0x5581dd843320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.97:35696/0 (socket says 172.21.15.97:35696) 2024-03-27T10:57:47.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.797+0000 7f4a66000640 1 -- 172.21.15.97:0/3813863164 learned_addr learned my addr 172.21.15.97:0/3813863164 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:47.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.797+0000 7f4a66000640 1 --2- 172.21.15.97:0/3813863164 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5581dd844a90 0x5581dd84b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:47.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.797+0000 7f4a5f400640 1 --2- 172.21.15.97:0/3018858117 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f4a4c0a7aa0 0x7f4a4c0a9f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:47.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.797+0000 7f4a66a00640 1 -- 172.21.15.97:0/3813863164 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5581dd842f00 msgr2=0x5581dd843320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:47.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.797+0000 7f4a66a00640 1 --2- 172.21.15.97:0/3813863164 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5581dd842f00 0x5581dd843320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:47.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.798+0000 7f4a66a00640 1 -- 172.21.15.97:0/3813863164 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5581dd844a90 msgr2=0x5581dd84b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:47.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.798+0000 7f4a66a00640 1 --2- 172.21.15.97:0/3813863164 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5581dd844a90 0x5581dd84b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:47.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.798+0000 7f4a66a00640 1 -- 172.21.15.97:0/3813863164 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581dd5847e0 con 0x5581dd8440d0 2024-03-27T10:57:47.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.798+0000 7f4a5f400640 1 -- 172.21.15.97:0/3018858117 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f4a600f18d0 con 0x7f4a68107840 2024-03-27T10:57:47.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.798+0000 7f4a66000640 1 --2- 172.21.15.97:0/3813863164 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5581dd844a90 0x5581dd84b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T10:57:47.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.798+0000 7f4a66a00640 1 --2- 172.21.15.97:0/3813863164 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581dd8440d0 0x5581dd844550 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f4a58009bf0 tx=0x7f4a58011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eec265c77e4a4936 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:47.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.798+0000 7f4a67400640 1 --2- 172.21.15.97:0/3018858117 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f4a4c0a7aa0 0x7f4a4c0a9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:47.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.798+0000 7f4a53e00640 1 -- 172.21.15.97:0/3813863164 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a58004050 con 0x5581dd8440d0 2024-03-27T10:57:47.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.798+0000 7f4a53e00640 1 -- 172.21.15.97:0/3813863164 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4a580054d0 con 0x5581dd8440d0 2024-03-27T10:57:47.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.798+0000 7f4a53e00640 1 -- 172.21.15.97:0/3813863164 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a58004ca0 con 0x5581dd8440d0 2024-03-27T10:57:47.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.799+0000 7f4a7042a740 1 -- 172.21.15.97:0/3813863164 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581dd8440d0 msgr2=0x5581dd844550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:47.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.799+0000 7f4a67400640 1 --2- 172.21.15.97:0/3018858117 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f4a4c0a7aa0 0x7f4a4c0a9f60 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f4a60006010 tx=0x7f4a600093d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:47.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.799+0000 7f4a7042a740 1 --2- 172.21.15.97:0/3813863164 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581dd8440d0 0x5581dd844550 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f4a58009bf0 tx=0x7f4a58011810 comp rx=0 tx=0).stop 2024-03-27T10:57:47.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.799+0000 7f4a7042a740 1 -- 172.21.15.97:0/3813863164 shutdown_connections 2024-03-27T10:57:47.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.799+0000 7f4a7042a740 1 --2- 172.21.15.97:0/3813863164 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5581dd844a90 0x5581dd84b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:47.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.799+0000 7f4a7042a740 1 --2- 172.21.15.97:0/3813863164 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581dd8440d0 0x5581dd844550 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:47.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.799+0000 7f4a7042a740 1 --2- 172.21.15.97:0/3813863164 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5581dd842f00 0x5581dd843320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:47.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.799+0000 7f4a7042a740 1 -- 172.21.15.97:0/3813863164 >> 172.21.15.97:0/3813863164 conn(0x5581dd83e2f0 msgr2=0x5581dd83e6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:47.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.800+0000 7f4a7042a740 1 -- 172.21.15.97:0/3813863164 shutdown_connections 2024-03-27T10:57:47.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.800+0000 7f4a7042a740 1 -- 172.21.15.97:0/3813863164 wait complete. 2024-03-27T10:57:47.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.800+0000 7f4a7042a740 1 -- start start 2024-03-27T10:57:47.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.800+0000 7f4a7042a740 1 Processor -- start 2024-03-27T10:57:47.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.801+0000 7f4a7042a740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5581dd842f00 0x5581dd84d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:47.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.801+0000 7f4a7042a740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581dd8440d0 0x5581dd84da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:47.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.801+0000 7f4a7042a740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5581dd844a90 0x5581dd854a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:47.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.801+0000 7f4a7042a740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5581dd84baa0 con 0x5581dd8440d0 2024-03-27T10:57:47.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.801+0000 7f4a7042a740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5581dd763aa0 con 0x5581dd842f00 2024-03-27T10:57:47.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.801+0000 7f4a7042a740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5581dd84bc20 con 0x5581dd844a90 2024-03-27T10:57:47.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.801+0000 7f4a7042a740 10 client.0 _ll_register_callbacks cb 0x5581dd763490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:57:47.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.801+0000 7f4a66a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581dd8440d0 0x5581dd84da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:47.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.801+0000 7f4a66a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581dd8440d0 0x5581dd84da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54128/0 (socket says 172.21.15.97:54128) 2024-03-27T10:57:47.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.801+0000 7f4a66a00640 1 -- 172.21.15.97:0/3153770501 learned_addr learned my addr 172.21.15.97:0/3153770501 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:47.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.801+0000 7f4a66000640 1 --2- 172.21.15.97:0/3153770501 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5581dd842f00 0x5581dd84d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:47.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.801+0000 7f4a66000640 1 --2- 172.21.15.97:0/3153770501 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5581dd844a90 0x5581dd854a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:47.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.802+0000 7f4a66a00640 1 -- 172.21.15.97:0/3153770501 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5581dd844a90 msgr2=0x5581dd854a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:47.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.802+0000 7f4a66a00640 1 --2- 172.21.15.97:0/3153770501 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5581dd844a90 0x5581dd854a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:47.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.802+0000 7f4a66a00640 1 -- 172.21.15.97:0/3153770501 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5581dd842f00 msgr2=0x5581dd84d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:47.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.802+0000 7f4a66a00640 1 --2- 172.21.15.97:0/3153770501 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5581dd842f00 0x5581dd84d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:47.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.802+0000 7f4a66a00640 1 -- 172.21.15.97:0/3153770501 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581dd5e4f70 con 0x5581dd8440d0 2024-03-27T10:57:47.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.802+0000 7f4a66000640 1 --2- 172.21.15.97:0/3153770501 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5581dd842f00 0x5581dd84d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T10:57:47.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.802+0000 7f4a66a00640 1 --2- 172.21.15.97:0/3153770501 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581dd8440d0 0x5581dd84da40 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f4a5801b040 tx=0x7f4a58009fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95a5699d66de8512 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:47.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.802+0000 7f4a50c00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T10:57:47.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.802+0000 7f4a50c00640 1 -- 172.21.15.97:0/3153770501 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a58029070 con 0x5581dd8440d0 2024-03-27T10:57:47.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.802+0000 7f4a7042a740 1 -- 172.21.15.97:0/3153770501 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a580054d0 con 0x5581dd8440d0 2024-03-27T10:57:47.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.802+0000 7f4a50c00640 1 -- 172.21.15.97:0/3153770501 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f4a58003a00 con 0x5581dd8440d0 2024-03-27T10:57:47.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.803+0000 7f4a50c00640 1 -- 172.21.15.97:0/3153770501 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4a58033050 con 0x5581dd8440d0 2024-03-27T10:57:47.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.803+0000 7f4a7042a740 10 client.15993 fetch_fsmap learned FSMap version 28 2024-03-27T10:57:47.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.803+0000 7f4a7042a740 1 -- 172.21.15.97:0/3153770501 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5581dd47c170 con 0x5581dd8440d0 2024-03-27T10:57:47.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.803+0000 7f4a50c00640 1 -- 172.21.15.97:0/3153770501 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4a580054d0 con 0x5581dd8440d0 2024-03-27T10:57:47.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.803+0000 7f4a50c00640 1 -- 172.21.15.97:0/3153770501 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f4a58036020 con 0x5581dd8440d0 2024-03-27T10:57:47.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.803+0000 7f4a7042a740 10 client.15993 fetch_fsmap finished waiting for FSMap version 28 2024-03-27T10:57:47.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.803+0000 7f4a7042a740 10 client.15993 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-27T10:57:47.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.804+0000 7f4a7042a740 20 client.15993 populate_metadata read hostname 'smithi097' 2024-03-27T10:57:47.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.804+0000 7f4a7042a740 1 --2- 172.21.15.97:0/3153770501 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5581dd859170 0x5581dd85b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:47.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.804+0000 7f4a7042a740 4 client.15993 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T10:57:47.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.804+0000 7f4a7042a740 1 -- 172.21.15.97:0/3153770501 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5581dd7b57c0 con 0x5581dd859170 2024-03-27T10:57:47.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.804+0000 7f4a66000640 1 --2- 172.21.15.97:0/3153770501 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5581dd859170 0x5581dd85b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:47.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.805+0000 7f4a66000640 1 --2- 172.21.15.97:0/3153770501 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5581dd859170 0x5581dd85b5a0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8eb41681679523db server_cookie=a641d141bd1db929 in_seq=0 out_seq=0 2024-03-27T10:57:47.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.805+0000 7f4a50c00640 10 client.15993 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T10:57:47.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.806+0000 7f4a50c00640 1 -- 172.21.15.97:0/3153770501 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5581dd7b57c0 con 0x5581dd859170 2024-03-27T10:57:47.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.806+0000 7f4a50c00640 10 client.15993 handle_command_reply: tid=0 2024-03-27T10:57:47.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.806+0000 7f4a7042a740 1 client.15993 shutdown 2024-03-27T10:57:47.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.807+0000 7f4a7042a740 1 -- 172.21.15.97:0/3153770501 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581dd8440d0 msgr2=0x5581dd84da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:47.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.807+0000 7f4a7042a740 1 --2- 172.21.15.97:0/3153770501 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581dd8440d0 0x5581dd84da40 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f4a5801b040 tx=0x7f4a58009fd0 comp rx=0 tx=0).stop 2024-03-27T10:57:47.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.807+0000 7f4a7042a740 1 -- 172.21.15.97:0/3153770501 shutdown_connections 2024-03-27T10:57:47.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.807+0000 7f4a7042a740 1 --2- 172.21.15.97:0/3153770501 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5581dd859170 0x5581dd85b5a0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:47.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.807+0000 7f4a7042a740 1 --2- 172.21.15.97:0/3153770501 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5581dd844a90 0x5581dd854a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:47.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.807+0000 7f4a7042a740 1 --2- 172.21.15.97:0/3153770501 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581dd8440d0 0x5581dd84da40 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:47.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.807+0000 7f4a7042a740 1 --2- 172.21.15.97:0/3153770501 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5581dd842f00 0x5581dd84d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:47.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.807+0000 7f4a7042a740 1 -- 172.21.15.97:0/3153770501 >> 172.21.15.97:0/3153770501 conn(0x5581dd83e2f0 msgr2=0x5581dd83d550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:47.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.807+0000 7f4a50c00640 0 client.15993 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T10:57:47.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.807+0000 7f4a7042a740 1 -- 172.21.15.97:0/3153770501 shutdown_connections 2024-03-27T10:57:47.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.807+0000 7f4a7042a740 1 -- 172.21.15.97:0/3153770501 wait complete. 2024-03-27T10:57:47.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.807+0000 7f4a7042a740 20 client.15993 trim_cache size 0 max 16384 2024-03-27T10:57:47.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.825+0000 7f4a7042a740 1 Processor -- start 2024-03-27T10:57:47.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.825+0000 7f4a7042a740 1 -- start start 2024-03-27T10:57:47.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.825+0000 7f4a7042a740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5581dd842f00 0x5581dd847d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:47.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.825+0000 7f4a7042a740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581dd8440d0 0x5581dd848260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:47.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.825+0000 7f4a7042a740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5581dd844a90 0x5581dd854a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:47.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.826+0000 7f4a7042a740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4a681a6bf0 con 0x5581dd8440d0 2024-03-27T10:57:47.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.826+0000 7f4a7042a740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f4a681a8730 con 0x5581dd842f00 2024-03-27T10:57:47.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.826+0000 7f4a7042a740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f4a681a72c0 con 0x5581dd844a90 2024-03-27T10:57:47.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.826+0000 7f4a66a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581dd8440d0 0x5581dd848260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:47.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.826+0000 7f4a66a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581dd8440d0 0x5581dd848260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54142/0 (socket says 172.21.15.97:54142) 2024-03-27T10:57:47.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.826+0000 7f4a66a00640 1 -- 172.21.15.97:0/7869304 learned_addr learned my addr 172.21.15.97:0/7869304 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:47.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.826+0000 7f4a66000640 1 --2- 172.21.15.97:0/7869304 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5581dd842f00 0x5581dd847d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:47.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.826+0000 7f4a66000640 1 --2- 172.21.15.97:0/7869304 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5581dd844a90 0x5581dd854a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:47.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.826+0000 7f4a66a00640 1 -- 172.21.15.97:0/7869304 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5581dd844a90 msgr2=0x5581dd854a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:47.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.826+0000 7f4a66a00640 1 --2- 172.21.15.97:0/7869304 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5581dd844a90 0x5581dd854a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:47.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.826+0000 7f4a66a00640 1 -- 172.21.15.97:0/7869304 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5581dd842f00 msgr2=0x5581dd847d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:47.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.826+0000 7f4a66a00640 1 --2- 172.21.15.97:0/7869304 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5581dd842f00 0x5581dd847d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:47.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.826+0000 7f4a66a00640 1 -- 172.21.15.97:0/7869304 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581dd84af70 con 0x5581dd8440d0 2024-03-27T10:57:47.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.826+0000 7f4a66000640 1 --2- 172.21.15.97:0/7869304 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5581dd842f00 0x5581dd847d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:57:47.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.827+0000 7f4a66a00640 1 --2- 172.21.15.97:0/7869304 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581dd8440d0 0x5581dd848260 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f4a58009730 tx=0x7f4a58005970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bb7d26a53d012d7 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:47.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.827+0000 7f4a53400640 1 -- 172.21.15.97:0/7869304 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a58029070 con 0x5581dd8440d0 2024-03-27T10:57:47.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.827+0000 7f4a53400640 1 -- 172.21.15.97:0/7869304 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4a58023430 con 0x5581dd8440d0 2024-03-27T10:57:47.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.827+0000 7f4a53400640 1 -- 172.21.15.97:0/7869304 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a580237a0 con 0x5581dd8440d0 2024-03-27T10:57:47.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.828+0000 7f4a7042a740 1 -- 172.21.15.97:0/7869304 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581dd8440d0 msgr2=0x5581dd848260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:47.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.828+0000 7f4a7042a740 1 --2- 172.21.15.97:0/7869304 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581dd8440d0 0x5581dd848260 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f4a58009730 tx=0x7f4a58005970 comp rx=0 tx=0).stop 2024-03-27T10:57:47.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.828+0000 7f4a7042a740 1 -- 172.21.15.97:0/7869304 shutdown_connections 2024-03-27T10:57:47.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.828+0000 7f4a7042a740 1 --2- 172.21.15.97:0/7869304 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5581dd844a90 0x5581dd854a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:47.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.828+0000 7f4a7042a740 1 --2- 172.21.15.97:0/7869304 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581dd8440d0 0x5581dd848260 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:47.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.828+0000 7f4a7042a740 1 --2- 172.21.15.97:0/7869304 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5581dd842f00 0x5581dd847d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:47.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.828+0000 7f4a7042a740 1 -- 172.21.15.97:0/7869304 >> 172.21.15.97:0/7869304 conn(0x5581dd83e2f0 msgr2=0x5581dd84d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:47.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.828+0000 7f4a7042a740 1 -- 172.21.15.97:0/7869304 shutdown_connections 2024-03-27T10:57:47.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.828+0000 7f4a7042a740 1 -- 172.21.15.97:0/7869304 wait complete. 2024-03-27T10:57:47.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.828+0000 7f4a7042a740 1 -- start start 2024-03-27T10:57:47.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.828+0000 7f4a7042a740 1 Processor -- start 2024-03-27T10:57:47.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.829+0000 7f4a7042a740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581dd842f00 0x5581dd850bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:47.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.829+0000 7f4a7042a740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5581dd8440d0 0x5581dd851110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:47.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.829+0000 7f4a7042a740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5581dd844a90 0x5581dd853ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:47.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.829+0000 7f4a7042a740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4a681a8060 con 0x5581dd842f00 2024-03-27T10:57:47.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.829+0000 7f4a7042a740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f4a681a7990 con 0x5581dd8440d0 2024-03-27T10:57:47.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.829+0000 7f4a7042a740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f4a681a4bd0 con 0x5581dd844a90 2024-03-27T10:57:47.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.829+0000 7f4a7042a740 10 client.0 _ll_register_callbacks cb 0x7f4a58009a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:57:47.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.829+0000 7f4a66000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581dd842f00 0x5581dd850bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:47.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.829+0000 7f4a66000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581dd842f00 0x5581dd850bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54150/0 (socket says 172.21.15.97:54150) 2024-03-27T10:57:47.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.829+0000 7f4a66000640 1 -- 172.21.15.97:0/3115437702 learned_addr learned my addr 172.21.15.97:0/3115437702 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:47.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.830+0000 7f4a66000640 1 -- 172.21.15.97:0/3115437702 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5581dd844a90 msgr2=0x5581dd853ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:47.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.830+0000 7f4a66000640 1 --2- 172.21.15.97:0/3115437702 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5581dd844a90 0x5581dd853ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:47.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.830+0000 7f4a66000640 1 -- 172.21.15.97:0/3115437702 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5581dd8440d0 msgr2=0x5581dd851110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:47.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.830+0000 7f4a66a00640 1 --2- 172.21.15.97:0/3115437702 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5581dd8440d0 0x5581dd851110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:47.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.830+0000 7f4a66000640 1 --2- 172.21.15.97:0/3115437702 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5581dd8440d0 0x5581dd851110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:47.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.830+0000 7f4a66000640 1 -- 172.21.15.97:0/3115437702 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581dd854090 con 0x5581dd842f00 2024-03-27T10:57:47.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.830+0000 7f4a66000640 1 --2- 172.21.15.97:0/3115437702 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581dd842f00 0x5581dd850bd0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x5581dd85be50 tx=0x7f4a54013f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a3be32b9a914669 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:47.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.830+0000 7f4a50c00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T10:57:47.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.830+0000 7f4a50c00640 1 -- 172.21.15.97:0/3115437702 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a54010b10 con 0x5581dd842f00 2024-03-27T10:57:47.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.830+0000 7f4a7042a740 1 -- 172.21.15.97:0/3115437702 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a58023430 con 0x5581dd842f00 2024-03-27T10:57:47.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.830+0000 7f4a50c00640 1 -- 172.21.15.97:0/3115437702 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f4a54010cb0 con 0x5581dd842f00 2024-03-27T10:57:47.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.830+0000 7f4a50c00640 1 -- 172.21.15.97:0/3115437702 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4a54030050 con 0x5581dd842f00 2024-03-27T10:57:47.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.830+0000 7f4a7042a740 10 client.15999 fetch_fsmap learned FSMap version 28 2024-03-27T10:57:47.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.830+0000 7f4a7042a740 1 -- 172.21.15.97:0/3115437702 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5581dd8bf940 con 0x5581dd842f00 2024-03-27T10:57:47.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.831+0000 7f4a50c00640 1 -- 172.21.15.97:0/3115437702 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4a58023430 con 0x5581dd842f00 2024-03-27T10:57:47.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.831+0000 7f4a50c00640 1 -- 172.21.15.97:0/3115437702 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f4a54011560 con 0x5581dd842f00 2024-03-27T10:57:47.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.831+0000 7f4a7042a740 10 client.15999 fetch_fsmap finished waiting for FSMap version 28 2024-03-27T10:57:47.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.831+0000 7f4a7042a740 10 client.15999 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-27T10:57:47.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.831+0000 7f4a7042a740 20 client.15999 populate_metadata read hostname 'smithi097' 2024-03-27T10:57:47.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.831+0000 7f4a7042a740 1 --2- 172.21.15.97:0/3115437702 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5581dd8c02d0 0x5581dd8c26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:47.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.831+0000 7f4a7042a740 4 client.15999 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-27T10:57:47.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.831+0000 7f4a7042a740 1 -- 172.21.15.97:0/3115437702 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5581dd7afb10 con 0x5581dd8c02d0 2024-03-27T10:57:47.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.832+0000 7f4a66a00640 1 --2- 172.21.15.97:0/3115437702 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5581dd8c02d0 0x5581dd8c26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:47.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.832+0000 7f4a66a00640 1 --2- 172.21.15.97:0/3115437702 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5581dd8c02d0 0x5581dd8c26f0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=829a6632c2f0ba4 server_cookie=cb385798932104d in_seq=0 out_seq=0 2024-03-27T10:57:47.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.832+0000 7f4a50c00640 10 client.15999 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T10:57:47.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.834+0000 7f4a50c00640 1 -- 172.21.15.97:0/3115437702 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5581dd7afb10 con 0x5581dd8c02d0 2024-03-27T10:57:47.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.834+0000 7f4a50c00640 10 client.15999 handle_command_reply: tid=0 2024-03-27T10:57:47.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.834+0000 7f4a7042a740 1 client.15999 shutdown 2024-03-27T10:57:47.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.835+0000 7f4a7042a740 1 -- 172.21.15.97:0/3115437702 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581dd842f00 msgr2=0x5581dd850bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:47.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.835+0000 7f4a7042a740 1 --2- 172.21.15.97:0/3115437702 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581dd842f00 0x5581dd850bd0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x5581dd85be50 tx=0x7f4a54013f20 comp rx=0 tx=0).stop 2024-03-27T10:57:47.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.835+0000 7f4a7042a740 1 -- 172.21.15.97:0/3115437702 shutdown_connections 2024-03-27T10:57:47.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.835+0000 7f4a7042a740 1 --2- 172.21.15.97:0/3115437702 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5581dd8c02d0 0x5581dd8c26f0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:47.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.835+0000 7f4a7042a740 1 --2- 172.21.15.97:0/3115437702 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5581dd844a90 0x5581dd853ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:47.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.835+0000 7f4a50c00640 0 client.15999 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T10:57:47.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.835+0000 7f4a7042a740 1 --2- 172.21.15.97:0/3115437702 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5581dd8440d0 0x5581dd851110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:47.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.835+0000 7f4a7042a740 1 --2- 172.21.15.97:0/3115437702 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581dd842f00 0x5581dd850bd0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:47.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.835+0000 7f4a7042a740 1 -- 172.21.15.97:0/3115437702 >> 172.21.15.97:0/3115437702 conn(0x5581dd83e2f0 msgr2=0x5581dd84d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:47.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.835+0000 7f4a7042a740 1 -- 172.21.15.97:0/3115437702 shutdown_connections 2024-03-27T10:57:47.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.835+0000 7f4a7042a740 1 -- 172.21.15.97:0/3115437702 wait complete. 2024-03-27T10:57:47.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.835+0000 7f4a7042a740 20 client.15999 trim_cache size 0 max 16384 2024-03-27T10:57:47.812 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-03-27T10:57:47.812 INFO:teuthology.orchestra.run.smithi097.stdout: "return_code": 0, 2024-03-27T10:57:47.812 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_tag": "e38c1f2b-c3d2-4bc7-a454-b354b322dbb0", 2024-03-27T10:57:47.813 INFO:teuthology.orchestra.run.smithi097.stdout: "mode": "asynchronous" 2024-03-27T10:57:47.813 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-03-27T10:57:47.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.836+0000 7f4a6d800640 1 -- 172.21.15.97:0/3018858117 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f4a4c0a7aa0 msgr2=0x7f4a4c0a9f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:47.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.836+0000 7f4a6d800640 1 --2- 172.21.15.97:0/3018858117 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f4a4c0a7aa0 0x7f4a4c0a9f60 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f4a60006010 tx=0x7f4a600093d0 comp rx=0 tx=0).stop 2024-03-27T10:57:47.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.836+0000 7f4a6d800640 1 -- 172.21.15.97:0/3018858117 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a68107840 msgr2=0x7f4a6819f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:47.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.836+0000 7f4a6d800640 1 --2- 172.21.15.97:0/3018858117 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a68107840 0x7f4a6819f290 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f4a60032380 tx=0x7f4a60032820 comp rx=0 tx=0).stop 2024-03-27T10:57:47.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.836+0000 7f4a6d800640 1 -- 172.21.15.97:0/3018858117 shutdown_connections 2024-03-27T10:57:47.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.836+0000 7f4a6d800640 1 --2- 172.21.15.97:0/3018858117 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a68107840 0x7f4a6819f290 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:47.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.836+0000 7f4a6d800640 1 --2- 172.21.15.97:0/3018858117 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4a68104f10 0x7f4a68198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:47.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.836+0000 7f4a6d800640 1 --2- 172.21.15.97:0/3018858117 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f4a4c0a7aa0 0x7f4a4c0a9f60 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:47.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.836+0000 7f4a6d800640 1 --2- 172.21.15.97:0/3018858117 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4a681025e0 0x7f4a68197d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:47.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.836+0000 7f4a6d800640 1 -- 172.21.15.97:0/3018858117 >> 172.21.15.97:0/3018858117 conn(0x7f4a680fbff0 msgr2=0x7f4a68102f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:47.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.836+0000 7f4a6d800640 1 -- 172.21.15.97:0/3018858117 shutdown_connections 2024-03-27T10:57:47.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:47.836+0000 7f4a6d800640 1 -- 172.21.15.97:0/3018858117 wait complete. 2024-03-27T10:57:47.825 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-27T10:57:47.825 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-27T10:57:48.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.036+0000 7faac6600640 1 -- 172.21.15.97:0/2444502354 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faac0104f10 msgr2=0x7faac0107300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:48.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.036+0000 7faac6600640 1 --2- 172.21.15.97:0/2444502354 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faac0104f10 0x7faac0107300 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7faaac009900 tx=0x7faaac02bc30 comp rx=0 tx=0).stop 2024-03-27T10:57:48.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.036+0000 7faac6600640 1 -- 172.21.15.97:0/2444502354 shutdown_connections 2024-03-27T10:57:48.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.036+0000 7faac6600640 1 --2- 172.21.15.97:0/2444502354 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faac0107840 0x7faac0109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:48.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.036+0000 7faac6600640 1 --2- 172.21.15.97:0/2444502354 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faac0104f10 0x7faac0107300 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:48.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.036+0000 7faac6600640 1 --2- 172.21.15.97:0/2444502354 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faac01025e0 0x7faac01049d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:48.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.036+0000 7faac6600640 1 -- 172.21.15.97:0/2444502354 >> 172.21.15.97:0/2444502354 conn(0x7faac00fbff0 msgr2=0x7faac00fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:48.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.036+0000 7faac6600640 1 -- 172.21.15.97:0/2444502354 shutdown_connections 2024-03-27T10:57:48.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.036+0000 7faac6600640 1 -- 172.21.15.97:0/2444502354 wait complete. 2024-03-27T10:57:48.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.037+0000 7faac6600640 1 Processor -- start 2024-03-27T10:57:48.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.038+0000 7faac6600640 1 -- start start 2024-03-27T10:57:48.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.038+0000 7faac6600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faac01025e0 0x7faac0199fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:48.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.038+0000 7faac6600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faac0104f10 0x7faac019a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:48.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.038+0000 7faac6600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faac0107840 0x7faac01a1570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:48.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.038+0000 7faac6600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7faac010d6d0 con 0x7faac0104f10 2024-03-27T10:57:48.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.038+0000 7faac6600640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7faac010d550 con 0x7faac0107840 2024-03-27T10:57:48.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.038+0000 7faac6600640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7faac010d850 con 0x7faac01025e0 2024-03-27T10:57:48.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.038+0000 7faabea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faac0104f10 0x7faac019a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:48.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.038+0000 7faabea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faac0104f10 0x7faac019a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54170/0 (socket says 172.21.15.97:54170) 2024-03-27T10:57:48.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.038+0000 7faabea00640 1 -- 172.21.15.97:0/2720980418 learned_addr learned my addr 172.21.15.97:0/2720980418 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:48.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.039+0000 7faabf400640 1 --2- 172.21.15.97:0/2720980418 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faac01025e0 0x7faac0199fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:48.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.039+0000 7faabea00640 1 -- 172.21.15.97:0/2720980418 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faac01025e0 msgr2=0x7faac0199fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:48.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.039+0000 7faabea00640 1 --2- 172.21.15.97:0/2720980418 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faac01025e0 0x7faac0199fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:48.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.039+0000 7faabea00640 1 -- 172.21.15.97:0/2720980418 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faac0107840 msgr2=0x7faac01a1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:48.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.039+0000 7faabfe00640 1 --2- 172.21.15.97:0/2720980418 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faac0107840 0x7faac01a1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:48.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.039+0000 7faabea00640 1 --2- 172.21.15.97:0/2720980418 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faac0107840 0x7faac01a1570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:48.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.039+0000 7faabea00640 1 -- 172.21.15.97:0/2720980418 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faac01a1be0 con 0x7faac0104f10 2024-03-27T10:57:48.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.039+0000 7faabfe00640 1 --2- 172.21.15.97:0/2720980418 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faac0107840 0x7faac01a1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:57:48.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.039+0000 7faabea00640 1 --2- 172.21.15.97:0/2720980418 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faac0104f10 0x7faac019a500 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7faaac02c840 tx=0x7faaac004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:48.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.039+0000 7faab7e00640 1 -- 172.21.15.97:0/2720980418 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaac043020 con 0x7faac0104f10 2024-03-27T10:57:48.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.039+0000 7faac6600640 1 -- 172.21.15.97:0/2720980418 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faac01a1e70 con 0x7faac0104f10 2024-03-27T10:57:48.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.040+0000 7faac6600640 1 -- 172.21.15.97:0/2720980418 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faac01a2380 con 0x7faac0104f10 2024-03-27T10:57:48.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.040+0000 7faab7e00640 1 -- 172.21.15.97:0/2720980418 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faaac0052b0 con 0x7faac0104f10 2024-03-27T10:57:48.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.040+0000 7faab7e00640 1 -- 172.21.15.97:0/2720980418 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaac003d90 con 0x7faac0104f10 2024-03-27T10:57:48.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.044+0000 7faac9415740 1 Processor -- start 2024-03-27T10:57:48.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.044+0000 7faac9415740 1 -- start start 2024-03-27T10:57:48.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.044+0000 7faab7e00640 1 -- 172.21.15.97:0/2720980418 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7faaac046020 con 0x7faac0104f10 2024-03-27T10:57:48.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.045+0000 7faac9415740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5603922fecc0 0x5603922ff0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:48.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.045+0000 7faac9415740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5603922ff6b0 0x5603923041b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:48.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.045+0000 7faac9415740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560392304820 0x560392306cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:48.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.045+0000 7faac9415740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56039225c6b0 con 0x560392304820 2024-03-27T10:57:48.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.045+0000 7faac9415740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x560392204750 con 0x5603922ff6b0 2024-03-27T10:57:48.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.045+0000 7faac9415740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x560392258660 con 0x5603922fecc0 2024-03-27T10:57:48.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.045+0000 7faabea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560392304820 0x560392306cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:48.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.045+0000 7faabf400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5603922ff6b0 0x5603923041b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:48.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.045+0000 7faabea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560392304820 0x560392306cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54178/0 (socket says 172.21.15.97:54178) 2024-03-27T10:57:48.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.045+0000 7faabea00640 1 -- 172.21.15.97:0/3990157266 learned_addr learned my addr 172.21.15.97:0/3990157266 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:48.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.045+0000 7faabfe00640 1 --2- 172.21.15.97:0/3990157266 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5603922fecc0 0x5603922ff0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:48.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.046+0000 7faabf400640 1 -- 172.21.15.97:0/3990157266 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5603922fecc0 msgr2=0x5603922ff0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:48.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.046+0000 7faabf400640 1 --2- 172.21.15.97:0/3990157266 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5603922fecc0 0x5603922ff0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:48.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.046+0000 7faabf400640 1 -- 172.21.15.97:0/3990157266 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560392304820 msgr2=0x560392306cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:48.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.046+0000 7faabf400640 1 --2- 172.21.15.97:0/3990157266 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560392304820 0x560392306cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:48.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.046+0000 7faabf400640 1 -- 172.21.15.97:0/3990157266 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56039203a7d0 con 0x5603922ff6b0 2024-03-27T10:57:48.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.046+0000 7faab7e00640 1 --2- 172.21.15.97:0/2720980418 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7faaa40a36e0 0x7faaa40a5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:48.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.046+0000 7faab7e00640 1 -- 172.21.15.97:0/2720980418 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7faaac0f1b10 con 0x7faac0104f10 2024-03-27T10:57:48.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.047+0000 7faabfe00640 1 --2- 172.21.15.97:0/2720980418 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7faaa40a36e0 0x7faaa40a5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:48.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.047+0000 7faabf400640 1 --2- 172.21.15.97:0/3990157266 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5603922ff6b0 0x5603923041b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7faab000bbe0 tx=0x7faab000f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9cccc20c9a7f2b24 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:48.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.047+0000 7faab4c00640 1 -- 172.21.15.97:0/3990157266 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab0015070 con 0x5603922ff6b0 2024-03-27T10:57:48.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.047+0000 7faab4c00640 1 -- 172.21.15.97:0/3990157266 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faab00035f0 con 0x5603922ff6b0 2024-03-27T10:57:48.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.047+0000 7faab4c00640 1 -- 172.21.15.97:0/3990157266 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab00116a0 con 0x5603922ff6b0 2024-03-27T10:57:48.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.048+0000 7faac9415740 1 -- 172.21.15.97:0/3990157266 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5603922ff6b0 msgr2=0x5603923041b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:48.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.048+0000 7faabfe00640 1 --2- 172.21.15.97:0/2720980418 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7faaa40a36e0 0x7faaa40a5ba0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x5603922fff10 tx=0x7faab8009700 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:48.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.048+0000 7faac9415740 1 --2- 172.21.15.97:0/3990157266 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5603922ff6b0 0x5603923041b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7faab000bbe0 tx=0x7faab000f730 comp rx=0 tx=0).stop 2024-03-27T10:57:48.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.048+0000 7faac9415740 1 -- 172.21.15.97:0/3990157266 shutdown_connections 2024-03-27T10:57:48.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.048+0000 7faac9415740 1 --2- 172.21.15.97:0/3990157266 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560392304820 0x560392306cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:48.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.048+0000 7faac9415740 1 --2- 172.21.15.97:0/3990157266 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5603922ff6b0 0x5603923041b0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:48.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.048+0000 7faac9415740 1 --2- 172.21.15.97:0/3990157266 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5603922fecc0 0x5603922ff0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:48.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.048+0000 7faac9415740 1 -- 172.21.15.97:0/3990157266 >> 172.21.15.97:0/3990157266 conn(0x5603922f1a20 msgr2=0x5603922f1e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:48.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.048+0000 7faac9415740 1 -- 172.21.15.97:0/3990157266 shutdown_connections 2024-03-27T10:57:48.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.048+0000 7faac9415740 1 -- 172.21.15.97:0/3990157266 wait complete. 2024-03-27T10:57:48.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.049+0000 7faac9415740 1 -- start start 2024-03-27T10:57:48.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.049+0000 7faac9415740 1 Processor -- start 2024-03-27T10:57:48.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.049+0000 7faac9415740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5603922fecc0 0x560392309210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:48.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.049+0000 7faac9415740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5603922ff6b0 0x560392309750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:48.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.049+0000 7faac9415740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560392304820 0x560392310790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:48.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.050+0000 7faac9415740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5603923077d0 con 0x560392304820 2024-03-27T10:57:48.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.050+0000 7faac9415740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x560392217a80 con 0x5603922ff6b0 2024-03-27T10:57:48.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.050+0000 7faac9415740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x560392307950 con 0x5603922fecc0 2024-03-27T10:57:48.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.050+0000 7faac9415740 10 client.0 _ll_register_callbacks cb 0x560392217470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:57:48.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.050+0000 7faabfe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560392304820 0x560392310790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:48.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.050+0000 7faabea00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5603922ff6b0 0x560392309750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:48.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.050+0000 7faabfe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560392304820 0x560392310790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54182/0 (socket says 172.21.15.97:54182) 2024-03-27T10:57:48.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.050+0000 7faabfe00640 1 -- 172.21.15.97:0/3210697969 learned_addr learned my addr 172.21.15.97:0/3210697969 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:48.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.050+0000 7faabf400640 1 --2- 172.21.15.97:0/3210697969 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5603922fecc0 0x560392309210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:48.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.050+0000 7faabfe00640 1 -- 172.21.15.97:0/3210697969 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5603922fecc0 msgr2=0x560392309210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:48.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.050+0000 7faabfe00640 1 --2- 172.21.15.97:0/3210697969 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5603922fecc0 0x560392309210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:48.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.050+0000 7faabfe00640 1 -- 172.21.15.97:0/3210697969 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5603922ff6b0 msgr2=0x560392309750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:48.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.050+0000 7faabfe00640 1 --2- 172.21.15.97:0/3210697969 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5603922ff6b0 0x560392309750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:48.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.050+0000 7faabfe00640 1 -- 172.21.15.97:0/3210697969 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560391f703f0 con 0x560392304820 2024-03-27T10:57:48.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.051+0000 7faabfe00640 1 --2- 172.21.15.97:0/3210697969 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560392304820 0x560392310790 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7faab80109f0 tx=0x7faab8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf8243e384681d8a server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:48.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.051+0000 7faaa9600640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T10:57:48.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.051+0000 7faaa9600640 1 -- 172.21.15.97:0/3210697969 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab800fd50 con 0x560392304820 2024-03-27T10:57:48.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.051+0000 7faaa9600640 1 -- 172.21.15.97:0/3210697969 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7faab8004b20 con 0x560392304820 2024-03-27T10:57:48.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.051+0000 7faac9415740 1 -- 172.21.15.97:0/3210697969 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faab00035f0 con 0x560392304820 2024-03-27T10:57:48.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.051+0000 7faaa9600640 1 -- 172.21.15.97:0/3210697969 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7faab8004d20 con 0x560392304820 2024-03-27T10:57:48.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.051+0000 7faac9415740 10 client.16011 fetch_fsmap learned FSMap version 28 2024-03-27T10:57:48.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.051+0000 7faac9415740 1 -- 172.21.15.97:0/3210697969 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560391f300f0 con 0x560392304820 2024-03-27T10:57:48.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.052+0000 7faaa9600640 1 -- 172.21.15.97:0/3210697969 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faab00035f0 con 0x560392304820 2024-03-27T10:57:48.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.052+0000 7faaa9600640 1 -- 172.21.15.97:0/3210697969 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7faab8027400 con 0x560392304820 2024-03-27T10:57:48.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.052+0000 7faac9415740 10 client.16011 fetch_fsmap finished waiting for FSMap version 28 2024-03-27T10:57:48.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.052+0000 7faac9415740 10 client.16011 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-27T10:57:48.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.052+0000 7faac9415740 20 client.16011 populate_metadata read hostname 'smithi097' 2024-03-27T10:57:48.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.052+0000 7faac9415740 1 --2- 172.21.15.97:0/3210697969 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5603923151c0 0x560392317630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:48.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.052+0000 7faac9415740 4 client.16011 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T10:57:48.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.052+0000 7faac9415740 1 -- 172.21.15.97:0/3210697969 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560392269bf0 con 0x5603923151c0 2024-03-27T10:57:48.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.053+0000 7faabf400640 1 --2- 172.21.15.97:0/3210697969 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5603923151c0 0x560392317630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:48.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.054+0000 7faabf400640 1 --2- 172.21.15.97:0/3210697969 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5603923151c0 0x560392317630 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59f1d9efc270323c server_cookie=55a8fa523af1f1d0 in_seq=0 out_seq=0 2024-03-27T10:57:48.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.054+0000 7faaa9600640 10 client.16011 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T10:57:48.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.055+0000 7faaa9600640 1 -- 172.21.15.97:0/3210697969 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x560392269bf0 con 0x5603923151c0 2024-03-27T10:57:48.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.055+0000 7faaa9600640 10 client.16011 handle_command_reply: tid=0 2024-03-27T10:57:48.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.055+0000 7faac9415740 1 client.16011 shutdown 2024-03-27T10:57:48.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.056+0000 7faac9415740 1 -- 172.21.15.97:0/3210697969 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560392304820 msgr2=0x560392310790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:48.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.056+0000 7faac9415740 1 --2- 172.21.15.97:0/3210697969 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560392304820 0x560392310790 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7faab80109f0 tx=0x7faab8004a40 comp rx=0 tx=0).stop 2024-03-27T10:57:48.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.056+0000 7faac9415740 1 -- 172.21.15.97:0/3210697969 shutdown_connections 2024-03-27T10:57:48.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.056+0000 7faac9415740 1 --2- 172.21.15.97:0/3210697969 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5603923151c0 0x560392317630 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:48.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.056+0000 7faac9415740 1 --2- 172.21.15.97:0/3210697969 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560392304820 0x560392310790 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:48.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.056+0000 7faac9415740 1 --2- 172.21.15.97:0/3210697969 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5603922ff6b0 0x560392309750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:48.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.056+0000 7faaa9600640 0 client.16011 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T10:57:48.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.056+0000 7faac9415740 1 --2- 172.21.15.97:0/3210697969 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5603922fecc0 0x560392309210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:48.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.056+0000 7faac9415740 1 -- 172.21.15.97:0/3210697969 >> 172.21.15.97:0/3210697969 conn(0x5603922f1a20 msgr2=0x5603922f0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:48.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.056+0000 7faac9415740 1 -- 172.21.15.97:0/3210697969 shutdown_connections 2024-03-27T10:57:48.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.056+0000 7faac9415740 1 -- 172.21.15.97:0/3210697969 wait complete. 2024-03-27T10:57:48.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.056+0000 7faac9415740 20 client.16011 trim_cache size 0 max 16384 2024-03-27T10:57:48.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.081+0000 7faac9415740 1 Processor -- start 2024-03-27T10:57:48.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.082+0000 7faac9415740 1 -- start start 2024-03-27T10:57:48.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.082+0000 7faac9415740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5603922fecc0 0x5603922f0390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:48.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.082+0000 7faac9415740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5603922ff6b0 0x560392310750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:48.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.082+0000 7faac9415740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560392304820 0x5603922f08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:48.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.082+0000 7faac9415740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7faac01a6da0 con 0x5603922fecc0 2024-03-27T10:57:48.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.082+0000 7faac9415740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7faac01a88e0 con 0x5603922ff6b0 2024-03-27T10:57:48.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.082+0000 7faac9415740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7faac01a7470 con 0x560392304820 2024-03-27T10:57:48.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.082+0000 7faabf400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5603922fecc0 0x5603922f0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:48.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.082+0000 7faabf400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5603922fecc0 0x5603922f0390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54190/0 (socket says 172.21.15.97:54190) 2024-03-27T10:57:48.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.082+0000 7faabf400640 1 -- 172.21.15.97:0/3824948806 learned_addr learned my addr 172.21.15.97:0/3824948806 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:48.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.083+0000 7faabea00640 1 --2- 172.21.15.97:0/3824948806 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5603922ff6b0 0x560392310750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:48.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.083+0000 7faabfe00640 1 --2- 172.21.15.97:0/3824948806 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560392304820 0x5603922f08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:48.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.083+0000 7faabf400640 1 -- 172.21.15.97:0/3824948806 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560392304820 msgr2=0x5603922f08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:48.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.083+0000 7faabf400640 1 --2- 172.21.15.97:0/3824948806 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560392304820 0x5603922f08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:48.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.083+0000 7faabf400640 1 -- 172.21.15.97:0/3824948806 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5603922ff6b0 msgr2=0x560392310750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:48.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.083+0000 7faabf400640 1 --2- 172.21.15.97:0/3824948806 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5603922ff6b0 0x560392310750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:48.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.083+0000 7faabf400640 1 -- 172.21.15.97:0/3824948806 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560392313a40 con 0x5603922fecc0 2024-03-27T10:57:48.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.083+0000 7faabea00640 1 --2- 172.21.15.97:0/3824948806 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5603922ff6b0 0x560392310750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T10:57:48.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.083+0000 7faabf400640 1 --2- 172.21.15.97:0/3824948806 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5603922fecc0 0x5603922f0390 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7faab000bb00 tx=0x7faab0005f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86406f4a47955ea4 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:48.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.083+0000 7faaabe00640 1 -- 172.21.15.97:0/3824948806 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab0015040 con 0x5603922fecc0 2024-03-27T10:57:48.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.083+0000 7faaabe00640 1 -- 172.21.15.97:0/3824948806 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faab0003aa0 con 0x5603922fecc0 2024-03-27T10:57:48.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.083+0000 7faaabe00640 1 -- 172.21.15.97:0/3824948806 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab0004030 con 0x5603922fecc0 2024-03-27T10:57:48.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.084+0000 7faac9415740 1 -- 172.21.15.97:0/3824948806 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5603922fecc0 msgr2=0x5603922f0390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:48.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.084+0000 7faac9415740 1 --2- 172.21.15.97:0/3824948806 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5603922fecc0 0x5603922f0390 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7faab000bb00 tx=0x7faab0005f30 comp rx=0 tx=0).stop 2024-03-27T10:57:48.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.085+0000 7faac9415740 1 -- 172.21.15.97:0/3824948806 shutdown_connections 2024-03-27T10:57:48.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.085+0000 7faac9415740 1 --2- 172.21.15.97:0/3824948806 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560392304820 0x5603922f08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:48.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.085+0000 7faac9415740 1 --2- 172.21.15.97:0/3824948806 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5603922ff6b0 0x560392310750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:48.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.085+0000 7faac9415740 1 --2- 172.21.15.97:0/3824948806 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5603922fecc0 0x5603922f0390 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:48.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.085+0000 7faac9415740 1 -- 172.21.15.97:0/3824948806 >> 172.21.15.97:0/3824948806 conn(0x5603922f1a20 msgr2=0x560392309210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:48.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.085+0000 7faac9415740 1 -- 172.21.15.97:0/3824948806 shutdown_connections 2024-03-27T10:57:48.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.085+0000 7faac9415740 1 -- 172.21.15.97:0/3824948806 wait complete. 2024-03-27T10:57:48.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.085+0000 7faac9415740 1 -- start start 2024-03-27T10:57:48.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.086+0000 7faac9415740 1 Processor -- start 2024-03-27T10:57:48.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.086+0000 7faac9415740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5603922fecc0 0x56039230e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:48.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.086+0000 7faac9415740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5603922ff6b0 0x56039230e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:48.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.086+0000 7faac9415740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560392304820 0x56039230f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:48.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.086+0000 7faac9415740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7faac01a8210 con 0x560392304820 2024-03-27T10:57:48.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.086+0000 7faac9415740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7faac01a7b40 con 0x5603922fecc0 2024-03-27T10:57:48.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.086+0000 7faac9415740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7faac01a4d80 con 0x5603922ff6b0 2024-03-27T10:57:48.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.086+0000 7faac9415740 10 client.0 _ll_register_callbacks cb 0x7faab8010670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:57:48.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.086+0000 7faabfe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560392304820 0x56039230f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:48.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.086+0000 7faabfe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560392304820 0x56039230f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54196/0 (socket says 172.21.15.97:54196) 2024-03-27T10:57:48.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.087+0000 7faabfe00640 1 -- 172.21.15.97:0/3538839133 learned_addr learned my addr 172.21.15.97:0/3538839133 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:48.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.087+0000 7faabf400640 1 --2- 172.21.15.97:0/3538839133 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5603922fecc0 0x56039230e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:48.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.087+0000 7faabea00640 1 --2- 172.21.15.97:0/3538839133 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5603922ff6b0 0x56039230e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:48.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.087+0000 7faabfe00640 1 -- 172.21.15.97:0/3538839133 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5603922ff6b0 msgr2=0x56039230e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:48.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.087+0000 7faabfe00640 1 --2- 172.21.15.97:0/3538839133 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5603922ff6b0 0x56039230e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:48.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.087+0000 7faabfe00640 1 -- 172.21.15.97:0/3538839133 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5603922fecc0 msgr2=0x56039230e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:48.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.087+0000 7faabfe00640 1 --2- 172.21.15.97:0/3538839133 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5603922fecc0 0x56039230e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:48.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.087+0000 7faabfe00640 1 -- 172.21.15.97:0/3538839133 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56039230fba0 con 0x560392304820 2024-03-27T10:57:48.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.087+0000 7faabfe00640 1 --2- 172.21.15.97:0/3538839133 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560392304820 0x56039230f5b0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7faab8012fd0 tx=0x7faab8009440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acf75d86145f5958 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:48.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.087+0000 7faaa9600640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T10:57:48.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.087+0000 7faaa9600640 1 -- 172.21.15.97:0/3538839133 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab800fd50 con 0x560392304820 2024-03-27T10:57:48.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.088+0000 7faac9415740 1 -- 172.21.15.97:0/3538839133 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faab0003aa0 con 0x560392304820 2024-03-27T10:57:48.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.088+0000 7faaa9600640 1 -- 172.21.15.97:0/3538839133 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7faab8027ae0 con 0x560392304820 2024-03-27T10:57:48.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.088+0000 7faaa9600640 1 -- 172.21.15.97:0/3538839133 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7faab800f790 con 0x560392304820 2024-03-27T10:57:48.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.088+0000 7faac9415740 10 client.16017 fetch_fsmap learned FSMap version 28 2024-03-27T10:57:48.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.088+0000 7faac9415740 1 -- 172.21.15.97:0/3538839133 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56039237b7e0 con 0x560392304820 2024-03-27T10:57:48.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.088+0000 7faaa9600640 1 -- 172.21.15.97:0/3538839133 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faab0003aa0 con 0x560392304820 2024-03-27T10:57:48.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.089+0000 7faaa9600640 1 -- 172.21.15.97:0/3538839133 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7faab8032c00 con 0x560392304820 2024-03-27T10:57:48.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.089+0000 7faac9415740 10 client.16017 fetch_fsmap finished waiting for FSMap version 28 2024-03-27T10:57:48.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.089+0000 7faac9415740 10 client.16017 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-27T10:57:48.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.089+0000 7faac9415740 20 client.16017 populate_metadata read hostname 'smithi097' 2024-03-27T10:57:48.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.089+0000 7faac9415740 1 --2- 172.21.15.97:0/3538839133 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x56039237c1e0 0x56039237e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:48.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.089+0000 7faac9415740 4 client.16017 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-27T10:57:48.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.089+0000 7faac9415740 1 -- 172.21.15.97:0/3538839133 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560392263f40 con 0x56039237c1e0 2024-03-27T10:57:48.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.089+0000 7faabf400640 1 --2- 172.21.15.97:0/3538839133 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x56039237c1e0 0x56039237e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:48.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.090+0000 7faabf400640 1 --2- 172.21.15.97:0/3538839133 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x56039237c1e0 0x56039237e600 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd3de38e9f5967c4 server_cookie=78e1ea1e8c20a3b in_seq=0 out_seq=0 2024-03-27T10:57:48.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.090+0000 7faaa9600640 10 client.16017 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T10:57:48.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.090+0000 7faaa9600640 1 -- 172.21.15.97:0/3538839133 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x560392263f40 con 0x56039237c1e0 2024-03-27T10:57:48.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.090+0000 7faaa9600640 10 client.16017 handle_command_reply: tid=0 2024-03-27T10:57:48.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.090+0000 7faac9415740 1 client.16017 shutdown 2024-03-27T10:57:48.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.090+0000 7faac9415740 1 -- 172.21.15.97:0/3538839133 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560392304820 msgr2=0x56039230f5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:48.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.090+0000 7faac9415740 1 --2- 172.21.15.97:0/3538839133 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560392304820 0x56039230f5b0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7faab8012fd0 tx=0x7faab8009440 comp rx=0 tx=0).stop 2024-03-27T10:57:48.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.090+0000 7faac9415740 1 -- 172.21.15.97:0/3538839133 shutdown_connections 2024-03-27T10:57:48.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.090+0000 7faac9415740 1 --2- 172.21.15.97:0/3538839133 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x56039237c1e0 0x56039237e600 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:48.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.090+0000 7faac9415740 1 --2- 172.21.15.97:0/3538839133 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560392304820 0x56039230f5b0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:48.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.090+0000 7faac9415740 1 --2- 172.21.15.97:0/3538839133 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5603922ff6b0 0x56039230e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:48.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.090+0000 7faaa9600640 0 client.16017 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T10:57:48.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.090+0000 7faac9415740 1 --2- 172.21.15.97:0/3538839133 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5603922fecc0 0x56039230e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:48.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.091+0000 7faac9415740 1 -- 172.21.15.97:0/3538839133 >> 172.21.15.97:0/3538839133 conn(0x5603922f1a20 msgr2=0x5603922f4c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:48.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.091+0000 7faac9415740 1 -- 172.21.15.97:0/3538839133 shutdown_connections 2024-03-27T10:57:48.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.091+0000 7faac9415740 1 -- 172.21.15.97:0/3538839133 wait complete. 2024-03-27T10:57:48.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.091+0000 7faac9415740 20 client.16017 trim_cache size 0 max 16384 2024-03-27T10:57:48.070 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-03-27T10:57:48.070 INFO:teuthology.orchestra.run.smithi097.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-27T10:57:48.070 INFO:teuthology.orchestra.run.smithi097.stdout: "scrubs": { 2024-03-27T10:57:48.070 INFO:teuthology.orchestra.run.smithi097.stdout: "e38c1f2b-c3d2-4bc7-a454-b354b322dbb0": { 2024-03-27T10:57:48.070 INFO:teuthology.orchestra.run.smithi097.stdout: "path": "/", 2024-03-27T10:57:48.070 INFO:teuthology.orchestra.run.smithi097.stdout: "tag": "e38c1f2b-c3d2-4bc7-a454-b354b322dbb0", 2024-03-27T10:57:48.071 INFO:teuthology.orchestra.run.smithi097.stdout: "options": "recursive,force" 2024-03-27T10:57:48.071 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T10:57:48.071 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T10:57:48.071 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-03-27T10:57:48.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.092+0000 7faac6600640 1 -- 172.21.15.97:0/2720980418 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7faaa40a36e0 msgr2=0x7faaa40a5ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:48.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.092+0000 7faac6600640 1 --2- 172.21.15.97:0/2720980418 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7faaa40a36e0 0x7faaa40a5ba0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x5603922fff10 tx=0x7faab8009700 comp rx=0 tx=0).stop 2024-03-27T10:57:48.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.092+0000 7faac6600640 1 -- 172.21.15.97:0/2720980418 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faac0104f10 msgr2=0x7faac019a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:48.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.092+0000 7faac6600640 1 --2- 172.21.15.97:0/2720980418 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faac0104f10 0x7faac019a500 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7faaac02c840 tx=0x7faaac004270 comp rx=0 tx=0).stop 2024-03-27T10:57:48.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.092+0000 7faac6600640 1 -- 172.21.15.97:0/2720980418 shutdown_connections 2024-03-27T10:57:48.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.092+0000 7faac6600640 1 --2- 172.21.15.97:0/2720980418 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7faaa40a36e0 0x7faaa40a5ba0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:48.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.092+0000 7faac6600640 1 --2- 172.21.15.97:0/2720980418 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faac0107840 0x7faac01a1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:48.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.092+0000 7faac6600640 1 --2- 172.21.15.97:0/2720980418 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faac0104f10 0x7faac019a500 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:48.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.092+0000 7faac6600640 1 --2- 172.21.15.97:0/2720980418 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faac01025e0 0x7faac0199fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:48.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.092+0000 7faac6600640 1 -- 172.21.15.97:0/2720980418 >> 172.21.15.97:0/2720980418 conn(0x7faac00fbff0 msgr2=0x7faac0101bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:48.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.092+0000 7faac6600640 1 -- 172.21.15.97:0/2720980418 shutdown_connections 2024-03-27T10:57:48.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:57:48.092+0000 7faac6600640 1 -- 172.21.15.97:0/2720980418 wait complete. 2024-03-27T10:57:48.087 INFO:tasks.cephfs.filesystem:scrub status for tag:e38c1f2b-c3d2-4bc7-a454-b354b322dbb0 - {'path': '/', 'tag': 'e38c1f2b-c3d2-4bc7-a454-b354b322dbb0', 'options': 'recursive,force'} 2024-03-27T10:57:49.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:57:49 smithi097 ceph-mon[31859]: scrub queued for path: / 2024-03-27T10:57:49.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:57:49 smithi097 ceph-mon[31859]: scrub summary: idle+waiting paths [/] 2024-03-27T10:57:49.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:57:49 smithi097 ceph-mon[31859]: scrub summary: active paths [/] 2024-03-27T10:57:49.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:57:49 smithi097 ceph-mon[31859]: scrub summary: idle+waiting paths [/] 2024-03-27T10:57:49.641 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:57:49 smithi097 ceph-mon[31859]: pgmap v476: 97 pgs: 97 active+clean; 665 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-27T10:57:49.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:57:49 smithi177 ceph-mon[35294]: scrub queued for path: / 2024-03-27T10:57:49.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:57:49 smithi177 ceph-mon[35294]: scrub summary: idle+waiting paths [/] 2024-03-27T10:57:49.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:57:49 smithi177 ceph-mon[35294]: scrub summary: active paths [/] 2024-03-27T10:57:49.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:57:49 smithi177 ceph-mon[35294]: scrub summary: idle+waiting paths [/] 2024-03-27T10:57:49.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:57:49 smithi177 ceph-mon[35294]: pgmap v476: 97 pgs: 97 active+clean; 665 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-27T10:57:49.766 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:57:49 smithi114 ceph-mon[35338]: scrub queued for path: / 2024-03-27T10:57:49.766 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:57:49 smithi114 ceph-mon[35338]: scrub summary: idle+waiting paths [/] 2024-03-27T10:57:49.767 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:57:49 smithi114 ceph-mon[35338]: scrub summary: active paths [/] 2024-03-27T10:57:49.767 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:57:49 smithi114 ceph-mon[35338]: scrub summary: idle+waiting paths [/] 2024-03-27T10:57:49.767 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:57:49 smithi114 ceph-mon[35338]: pgmap v476: 97 pgs: 97 active+clean; 665 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-27T10:57:50.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:57:50 smithi097 ceph-mon[31859]: scrub summary: idle 2024-03-27T10:57:50.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:57:50 smithi177 ceph-mon[35294]: scrub summary: idle 2024-03-27T10:57:50.766 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:57:50 smithi114 ceph-mon[35338]: scrub summary: idle 2024-03-27T10:57:51.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:57:51 smithi097 ceph-mon[31859]: pgmap v477: 97 pgs: 97 active+clean; 665 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 255 B/s wr, 13 op/s 2024-03-27T10:57:51.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:57:51 smithi177 ceph-mon[35294]: pgmap v477: 97 pgs: 97 active+clean; 665 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 255 B/s wr, 13 op/s 2024-03-27T10:57:51.766 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:57:51 smithi114 ceph-mon[35338]: pgmap v477: 97 pgs: 97 active+clean; 665 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 255 B/s wr, 13 op/s 2024-03-27T10:57:53.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:57:53 smithi097 ceph-mon[31859]: pgmap v478: 97 pgs: 97 active+clean; 665 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-27T10:57:53.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:57:53 smithi177 ceph-mon[35294]: pgmap v478: 97 pgs: 97 active+clean; 665 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-27T10:57:53.766 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:57:53 smithi114 ceph-mon[35338]: pgmap v478: 97 pgs: 97 active+clean; 665 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-27T10:57:55.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:57:55 smithi097 ceph-mon[31859]: pgmap v479: 97 pgs: 97 active+clean; 665 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 255 B/s wr, 13 op/s 2024-03-27T10:57:55.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:57:55 smithi177 ceph-mon[35294]: pgmap v479: 97 pgs: 97 active+clean; 665 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 255 B/s wr, 13 op/s 2024-03-27T10:57:55.766 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:57:55 smithi114 ceph-mon[35338]: pgmap v479: 97 pgs: 97 active+clean; 665 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 255 B/s wr, 13 op/s 2024-03-27T10:57:55.877 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.905+0000 7fad5e200640 1 -- 172.21.15.97:0/590885049 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad58103570 msgr2=0x7fad58109e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:55.877 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.905+0000 7fad5e200640 1 --2- 172.21.15.97:0/590885049 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad58103570 0x7fad58109e00 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fad50009ca0 tx=0x7fad5002c0f0 comp rx=0 tx=0).stop 2024-03-27T10:57:55.877 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.905+0000 7fad5e200640 1 -- 172.21.15.97:0/590885049 shutdown_connections 2024-03-27T10:57:55.877 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.905+0000 7fad5e200640 1 --2- 172.21.15.97:0/590885049 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad58103570 0x7fad58109e00 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:55.877 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.905+0000 7fad5e200640 1 --2- 172.21.15.97:0/590885049 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fad58102bb0 0x7fad58103030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:55.877 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.905+0000 7fad5e200640 1 --2- 172.21.15.97:0/590885049 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fad581019c0 0x7fad58101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:55.878 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.905+0000 7fad5e200640 1 -- 172.21.15.97:0/590885049 >> 172.21.15.97:0/590885049 conn(0x7fad580fcf30 msgr2=0x7fad580ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:55.878 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.905+0000 7fad5e200640 1 -- 172.21.15.97:0/590885049 shutdown_connections 2024-03-27T10:57:55.878 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.905+0000 7fad5e200640 1 -- 172.21.15.97:0/590885049 wait complete. 2024-03-27T10:57:55.880 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.906+0000 7fad5e200640 1 Processor -- start 2024-03-27T10:57:55.880 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.907+0000 7fad5e200640 1 -- start start 2024-03-27T10:57:55.880 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.907+0000 7fad5e200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad581019c0 0x7fad5819e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:55.880 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.907+0000 7fad5e200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fad58102bb0 0x7fad5819e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:55.880 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.907+0000 7fad5e200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fad58103570 0x7fad581a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:55.880 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.907+0000 7fad5e200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fad58075890 con 0x7fad581019c0 2024-03-27T10:57:55.880 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.907+0000 7fad5e200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fad58075710 con 0x7fad58102bb0 2024-03-27T10:57:55.880 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.907+0000 7fad5e200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fad58075a10 con 0x7fad58103570 2024-03-27T10:57:55.880 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.907+0000 7fad5ce00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad581019c0 0x7fad5819e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:55.880 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.907+0000 7fad5ce00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad581019c0 0x7fad5819e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54590/0 (socket says 172.21.15.97:54590) 2024-03-27T10:57:55.881 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.907+0000 7fad5ce00640 1 -- 172.21.15.97:0/111339274 learned_addr learned my addr 172.21.15.97:0/111339274 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:55.881 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.908+0000 7fad57e00640 1 --2- 172.21.15.97:0/111339274 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fad58102bb0 0x7fad5819e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:55.881 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.908+0000 7fad5d800640 1 --2- 172.21.15.97:0/111339274 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fad58103570 0x7fad581a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:55.881 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.908+0000 7fad5ce00640 1 -- 172.21.15.97:0/111339274 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fad58103570 msgr2=0x7fad581a5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:55.881 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.908+0000 7fad5ce00640 1 --2- 172.21.15.97:0/111339274 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fad58103570 0x7fad581a5700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:55.881 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.908+0000 7fad5ce00640 1 -- 172.21.15.97:0/111339274 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fad58102bb0 msgr2=0x7fad5819e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:55.881 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.908+0000 7fad5ce00640 1 --2- 172.21.15.97:0/111339274 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fad58102bb0 0x7fad5819e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:55.881 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.908+0000 7fad5ce00640 1 -- 172.21.15.97:0/111339274 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad581a5d70 con 0x7fad581019c0 2024-03-27T10:57:55.882 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.909+0000 7fad5ce00640 1 --2- 172.21.15.97:0/111339274 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad581019c0 0x7fad5819e150 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fad4c00e9c0 tx=0x7fad4c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:55.882 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.909+0000 7fad55600640 1 -- 172.21.15.97:0/111339274 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad4c00cd80 con 0x7fad581019c0 2024-03-27T10:57:55.882 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.909+0000 7fad5e200640 1 -- 172.21.15.97:0/111339274 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad581a6060 con 0x7fad581019c0 2024-03-27T10:57:55.882 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.909+0000 7fad5e200640 1 -- 172.21.15.97:0/111339274 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad581a65b0 con 0x7fad581019c0 2024-03-27T10:57:55.882 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.909+0000 7fad55600640 1 -- 172.21.15.97:0/111339274 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fad4c0035f0 con 0x7fad581019c0 2024-03-27T10:57:55.882 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.909+0000 7fad55600640 1 -- 172.21.15.97:0/111339274 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad4c010690 con 0x7fad581019c0 2024-03-27T10:57:55.885 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.913+0000 7fad62401740 1 Processor -- start 2024-03-27T10:57:55.886 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.914+0000 7fad62401740 1 -- start start 2024-03-27T10:57:55.886 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.914+0000 7fad62401740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b335e18b60 0x55b335e18f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:55.886 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.914+0000 7fad62401740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b335e19550 0x55b335e1e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:55.886 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.914+0000 7fad55600640 1 -- 172.21.15.97:0/111339274 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7fad4c0037a0 con 0x7fad581019c0 2024-03-27T10:57:55.886 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.914+0000 7fad62401740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b335e1e6c0 0x55b335e20b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:55.886 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.914+0000 7fad62401740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55b335d7dd10 con 0x55b335e1e6c0 2024-03-27T10:57:55.886 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.914+0000 7fad62401740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55b335d26260 con 0x55b335e19550 2024-03-27T10:57:55.886 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.914+0000 7fad62401740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55b335d79cc0 con 0x55b335e18b60 2024-03-27T10:57:55.887 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.915+0000 7fad57e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b335e1e6c0 0x55b335e20b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:55.887 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.915+0000 7fad5d800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b335e18b60 0x55b335e18f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:55.888 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.915+0000 7fad57e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b335e1e6c0 0x55b335e20b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54600/0 (socket says 172.21.15.97:54600) 2024-03-27T10:57:55.888 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.915+0000 7fad57e00640 1 -- 172.21.15.97:0/1887854512 learned_addr learned my addr 172.21.15.97:0/1887854512 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:55.888 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.915+0000 7fad5ce00640 1 --2- 172.21.15.97:0/1887854512 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b335e19550 0x55b335e1e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:55.888 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.915+0000 7fad57e00640 1 -- 172.21.15.97:0/1887854512 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b335e18b60 msgr2=0x55b335e18f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:55.888 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.915+0000 7fad57e00640 1 --2- 172.21.15.97:0/1887854512 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b335e18b60 0x55b335e18f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:55.888 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.916+0000 7fad57e00640 1 -- 172.21.15.97:0/1887854512 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b335e19550 msgr2=0x55b335e1e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:55.888 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.916+0000 7fad57e00640 1 --2- 172.21.15.97:0/1887854512 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b335e19550 0x55b335e1e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:55.888 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.916+0000 7fad57e00640 1 -- 172.21.15.97:0/1887854512 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b335d1daa0 con 0x55b335e1e6c0 2024-03-27T10:57:55.888 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.916+0000 7fad5ce00640 1 --2- 172.21.15.97:0/1887854512 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b335e19550 0x55b335e1e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T10:57:55.889 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.916+0000 7fad57e00640 1 --2- 172.21.15.97:0/1887854512 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b335e1e6c0 0x55b335e20b50 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fad44009c90 tx=0x7fad44011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec6b82b05af7dd2a server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:55.889 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.916+0000 7fad55600640 1 --2- 172.21.15.97:0/111339274 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fad400a3680 0x7fad400a5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:55.889 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.917+0000 7fad55600640 1 -- 172.21.15.97:0/111339274 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fad4c0d0210 con 0x7fad581019c0 2024-03-27T10:57:55.889 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.917+0000 7fad4a000640 1 -- 172.21.15.97:0/1887854512 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad4401e070 con 0x55b335e1e6c0 2024-03-27T10:57:55.889 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.917+0000 7fad4a000640 1 -- 172.21.15.97:0/1887854512 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fad44019ae0 con 0x55b335e1e6c0 2024-03-27T10:57:55.889 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.917+0000 7fad4a000640 1 -- 172.21.15.97:0/1887854512 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad4400ea40 con 0x55b335e1e6c0 2024-03-27T10:57:55.889 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.917+0000 7fad5d800640 1 --2- 172.21.15.97:0/111339274 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fad400a3680 0x7fad400a5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:55.890 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.918+0000 7fad62401740 1 -- 172.21.15.97:0/1887854512 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b335e1e6c0 msgr2=0x55b335e20b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:55.890 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.918+0000 7fad62401740 1 --2- 172.21.15.97:0/1887854512 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b335e1e6c0 0x55b335e20b50 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fad44009c90 tx=0x7fad44011810 comp rx=0 tx=0).stop 2024-03-27T10:57:55.890 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.918+0000 7fad62401740 1 -- 172.21.15.97:0/1887854512 shutdown_connections 2024-03-27T10:57:55.891 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.918+0000 7fad62401740 1 --2- 172.21.15.97:0/1887854512 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b335e1e6c0 0x55b335e20b50 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:55.891 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.918+0000 7fad62401740 1 --2- 172.21.15.97:0/1887854512 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b335e19550 0x55b335e1e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:55.891 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.918+0000 7fad62401740 1 --2- 172.21.15.97:0/1887854512 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b335e18b60 0x55b335e18f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:55.891 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.918+0000 7fad62401740 1 -- 172.21.15.97:0/1887854512 >> 172.21.15.97:0/1887854512 conn(0x55b335e140d0 msgr2=0x55b335e144d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:55.891 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.919+0000 7fad5d800640 1 --2- 172.21.15.97:0/111339274 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fad400a3680 0x7fad400a5b40 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fad5002c630 tx=0x7fad5002e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:55.891 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.919+0000 7fad62401740 1 -- 172.21.15.97:0/1887854512 shutdown_connections 2024-03-27T10:57:55.891 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.919+0000 7fad62401740 1 -- 172.21.15.97:0/1887854512 wait complete. 2024-03-27T10:57:55.892 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.919+0000 7fad62401740 1 -- start start 2024-03-27T10:57:55.892 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.920+0000 7fad62401740 1 Processor -- start 2024-03-27T10:57:55.892 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.920+0000 7fad62401740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b335e18b60 0x55b335e22e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:55.892 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.920+0000 7fad62401740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b335e19550 0x55b335e23380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:55.892 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.920+0000 7fad62401740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b335e1e6c0 0x55b335e2a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:55.892 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.920+0000 7fad62401740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55b335e21670 con 0x55b335e19550 2024-03-27T10:57:55.893 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.920+0000 7fad62401740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55b335d39630 con 0x55b335e18b60 2024-03-27T10:57:55.893 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.920+0000 7fad62401740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55b335e217f0 con 0x55b335e1e6c0 2024-03-27T10:57:55.893 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.920+0000 7fad62401740 10 client.0 _ll_register_callbacks cb 0x55b335d3ec50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:57:55.894 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.921+0000 7fad5d800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b335e1e6c0 0x55b335e2a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:55.894 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.921+0000 7fad57e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b335e19550 0x55b335e23380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:55.894 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.921+0000 7fad5d800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b335e1e6c0 0x55b335e2a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.97:47950/0 (socket says 172.21.15.97:47950) 2024-03-27T10:57:55.894 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.921+0000 7fad5d800640 1 -- 172.21.15.97:0/994343749 learned_addr learned my addr 172.21.15.97:0/994343749 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:55.894 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.921+0000 7fad5ce00640 1 --2- 172.21.15.97:0/994343749 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b335e18b60 0x55b335e22e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:55.894 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.922+0000 7fad5ce00640 1 -- 172.21.15.97:0/994343749 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b335e1e6c0 msgr2=0x55b335e2a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:55.894 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.922+0000 7fad5ce00640 1 --2- 172.21.15.97:0/994343749 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b335e1e6c0 0x55b335e2a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:55.894 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.922+0000 7fad5ce00640 1 -- 172.21.15.97:0/994343749 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b335e19550 msgr2=0x55b335e23380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:55.894 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.922+0000 7fad5ce00640 1 --2- 172.21.15.97:0/994343749 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b335e19550 0x55b335e23380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:55.894 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.922+0000 7fad5ce00640 1 -- 172.21.15.97:0/994343749 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b335d455f0 con 0x55b335e18b60 2024-03-27T10:57:55.895 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.922+0000 7fad57e00640 1 --2- 172.21.15.97:0/994343749 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b335e19550 0x55b335e23380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T10:57:55.895 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.922+0000 7fad5d800640 1 --2- 172.21.15.97:0/994343749 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b335e1e6c0 0x55b335e2a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T10:57:55.895 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.922+0000 7fad5ce00640 1 --2- 172.21.15.97:0/994343749 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b335e18b60 0x55b335e22e40 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x55b335e19fc0 tx=0x7fad4c098310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b721804d147eb70 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:55.895 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.922+0000 7fad3ea00640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-03-27T10:57:55.896 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.922+0000 7fad3ea00640 1 -- 172.21.15.97:0/994343749 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad4c09bff0 con 0x55b335e18b60 2024-03-27T10:57:55.896 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.923+0000 7fad62401740 1 -- 172.21.15.97:0/994343749 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad44019ae0 con 0x55b335e18b60 2024-03-27T10:57:55.896 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.923+0000 7fad3ea00640 1 -- 172.21.15.97:0/994343749 <== mon.2 v2:172.21.15.114:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fad4c0cddd0 con 0x55b335e18b60 2024-03-27T10:57:55.896 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.923+0000 7fad3ea00640 1 -- 172.21.15.97:0/994343749 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fad4c09cd90 con 0x55b335e18b60 2024-03-27T10:57:55.896 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.923+0000 7fad62401740 10 client.35681 fetch_fsmap learned FSMap version 28 2024-03-27T10:57:55.896 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.923+0000 7fad62401740 1 -- 172.21.15.97:0/994343749 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b335a92210 con 0x55b335e18b60 2024-03-27T10:57:55.896 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.923+0000 7fad3ea00640 1 -- 172.21.15.97:0/994343749 <== mon.2 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fad44019ae0 con 0x55b335e18b60 2024-03-27T10:57:55.897 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.924+0000 7fad3ea00640 1 -- 172.21.15.97:0/994343749 <== mon.2 v2:172.21.15.114:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fad4c0983e0 con 0x55b335e18b60 2024-03-27T10:57:55.897 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.924+0000 7fad62401740 10 client.35681 fetch_fsmap finished waiting for FSMap version 28 2024-03-27T10:57:55.897 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.924+0000 7fad62401740 10 client.35681 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T10:57:55.897 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.924+0000 7fad62401740 20 client.35681 populate_metadata read hostname 'smithi097' 2024-03-27T10:57:55.897 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.924+0000 7fad62401740 1 --2- 172.21.15.97:0/994343749 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55b335e2ed40 0x55b335e31200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:55.897 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.924+0000 7fad62401740 4 client.35681 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T10:57:55.897 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.924+0000 7fad62401740 1 -- 172.21.15.97:0/994343749 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b335d8b250 con 0x55b335e2ed40 2024-03-27T10:57:55.897 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.924+0000 7fad57e00640 1 --2- 172.21.15.97:0/994343749 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55b335e2ed40 0x55b335e31200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:55.898 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.925+0000 7fad57e00640 1 --2- 172.21.15.97:0/994343749 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55b335e2ed40 0x55b335e31200 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d5547e9be73f7cb server_cookie=7fec14184888240b in_seq=0 out_seq=0 2024-03-27T10:57:55.898 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.925+0000 7fad3ea00640 10 client.35681 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T10:57:55.899 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.927+0000 7fad3ea00640 1 -- 172.21.15.97:0/994343749 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55b335d8b250 con 0x55b335e2ed40 2024-03-27T10:57:55.899 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.927+0000 7fad3ea00640 10 client.35681 handle_command_reply: tid=0 2024-03-27T10:57:55.899 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.928+0000 7fad62401740 1 client.35681 shutdown 2024-03-27T10:57:55.900 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.928+0000 7fad62401740 1 -- 172.21.15.97:0/994343749 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b335e18b60 msgr2=0x55b335e22e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:55.900 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.928+0000 7fad62401740 1 --2- 172.21.15.97:0/994343749 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b335e18b60 0x55b335e22e40 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x55b335e19fc0 tx=0x7fad4c098310 comp rx=0 tx=0).stop 2024-03-27T10:57:55.900 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.928+0000 7fad62401740 1 -- 172.21.15.97:0/994343749 shutdown_connections 2024-03-27T10:57:55.901 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.928+0000 7fad62401740 1 --2- 172.21.15.97:0/994343749 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55b335e2ed40 0x55b335e31200 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:55.901 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.928+0000 7fad62401740 1 --2- 172.21.15.97:0/994343749 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b335e1e6c0 0x55b335e2a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:55.901 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.928+0000 7fad62401740 1 --2- 172.21.15.97:0/994343749 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b335e19550 0x55b335e23380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:55.901 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.928+0000 7fad62401740 1 --2- 172.21.15.97:0/994343749 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b335e18b60 0x55b335e22e40 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:55.901 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.928+0000 7fad62401740 1 -- 172.21.15.97:0/994343749 >> 172.21.15.97:0/994343749 conn(0x55b335e140d0 msgr2=0x55b335e13310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:55.901 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.928+0000 7fad3ea00640 0 client.35681 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T10:57:55.901 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.928+0000 7fad62401740 1 -- 172.21.15.97:0/994343749 shutdown_connections 2024-03-27T10:57:55.901 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.928+0000 7fad62401740 1 -- 172.21.15.97:0/994343749 wait complete. 2024-03-27T10:57:55.901 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.929+0000 7fad62401740 20 client.35681 trim_cache size 0 max 16384 2024-03-27T10:57:55.931 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.959+0000 7fad62401740 1 Processor -- start 2024-03-27T10:57:55.931 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.959+0000 7fad62401740 1 -- start start 2024-03-27T10:57:55.931 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.959+0000 7fad62401740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b335e18b60 0x55b335e36b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:55.931 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.959+0000 7fad62401740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b335e19550 0x55b335e2a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:55.931 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.959+0000 7fad62401740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b335e1e6c0 0x55b335e17330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:55.931 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.959+0000 7fad62401740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fad3800e430 con 0x55b335e1e6c0 2024-03-27T10:57:55.932 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.959+0000 7fad62401740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fad3800ff70 con 0x55b335e19550 2024-03-27T10:57:55.932 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.959+0000 7fad62401740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fad3800eb00 con 0x55b335e18b60 2024-03-27T10:57:55.933 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.960+0000 7fad5d800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b335e1e6c0 0x55b335e17330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:55.933 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.960+0000 7fad57e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b335e19550 0x55b335e2a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:55.933 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.960+0000 7fad5d800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b335e1e6c0 0x55b335e17330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54616/0 (socket says 172.21.15.97:54616) 2024-03-27T10:57:55.933 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.960+0000 7fad5d800640 1 -- 172.21.15.97:0/4114244690 learned_addr learned my addr 172.21.15.97:0/4114244690 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:55.933 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.960+0000 7fad5ce00640 1 --2- 172.21.15.97:0/4114244690 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b335e18b60 0x55b335e36b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:55.933 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.960+0000 7fad5d800640 1 -- 172.21.15.97:0/4114244690 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b335e18b60 msgr2=0x55b335e36b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:55.933 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.960+0000 7fad5d800640 1 --2- 172.21.15.97:0/4114244690 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b335e18b60 0x55b335e36b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:55.933 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.960+0000 7fad5d800640 1 -- 172.21.15.97:0/4114244690 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b335e19550 msgr2=0x55b335e2a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:55.933 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.960+0000 7fad5d800640 1 --2- 172.21.15.97:0/4114244690 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b335e19550 0x55b335e2a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:55.933 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.960+0000 7fad5d800640 1 -- 172.21.15.97:0/4114244690 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b335b5e620 con 0x55b335e1e6c0 2024-03-27T10:57:55.933 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.961+0000 7fad5d800640 1 --2- 172.21.15.97:0/4114244690 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b335e1e6c0 0x55b335e17330 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fad5002c700 tx=0x7fad5003f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7524cfa7b2e8bef8 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:55.934 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.961+0000 7fad49600640 1 -- 172.21.15.97:0/4114244690 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad50039040 con 0x55b335e1e6c0 2024-03-27T10:57:55.934 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.961+0000 7fad49600640 1 -- 172.21.15.97:0/4114244690 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fad50051070 con 0x55b335e1e6c0 2024-03-27T10:57:55.935 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.961+0000 7fad49600640 1 -- 172.21.15.97:0/4114244690 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad5004c6a0 con 0x55b335e1e6c0 2024-03-27T10:57:55.935 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.962+0000 7fad62401740 1 -- 172.21.15.97:0/4114244690 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b335e1e6c0 msgr2=0x55b335e17330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:55.935 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.962+0000 7fad62401740 1 --2- 172.21.15.97:0/4114244690 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b335e1e6c0 0x55b335e17330 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fad5002c700 tx=0x7fad5003f000 comp rx=0 tx=0).stop 2024-03-27T10:57:55.935 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.962+0000 7fad62401740 1 -- 172.21.15.97:0/4114244690 shutdown_connections 2024-03-27T10:57:55.935 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.963+0000 7fad62401740 1 --2- 172.21.15.97:0/4114244690 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b335e1e6c0 0x55b335e17330 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:55.935 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.963+0000 7fad62401740 1 --2- 172.21.15.97:0/4114244690 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b335e19550 0x55b335e2a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:55.935 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.963+0000 7fad62401740 1 --2- 172.21.15.97:0/4114244690 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b335e18b60 0x55b335e36b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:55.935 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.963+0000 7fad62401740 1 -- 172.21.15.97:0/4114244690 >> 172.21.15.97:0/4114244690 conn(0x55b335e140d0 msgr2=0x55b335e1a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:55.935 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.963+0000 7fad62401740 1 -- 172.21.15.97:0/4114244690 shutdown_connections 2024-03-27T10:57:55.935 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.963+0000 7fad62401740 1 -- 172.21.15.97:0/4114244690 wait complete. 2024-03-27T10:57:55.939 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.963+0000 7fad62401740 1 -- start start 2024-03-27T10:57:55.939 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.963+0000 7fad62401740 1 Processor -- start 2024-03-27T10:57:55.939 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.964+0000 7fad62401740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b335e18b60 0x55b335e2a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:55.939 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.964+0000 7fad62401740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b335e19550 0x55b335e26b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:55.939 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.964+0000 7fad62401740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b335e1e6c0 0x55b335e270c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:55.939 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.964+0000 7fad62401740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fad3800f8a0 con 0x55b335e18b60 2024-03-27T10:57:55.939 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.964+0000 7fad62401740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fad3800f1d0 con 0x55b335e19550 2024-03-27T10:57:55.939 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.964+0000 7fad62401740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fad3800c410 con 0x55b335e1e6c0 2024-03-27T10:57:55.939 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.964+0000 7fad62401740 10 client.0 _ll_register_callbacks cb 0x7fad4c0d3940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:57:55.939 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.964+0000 7fad5ce00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b335e18b60 0x55b335e2a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:55.939 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.964+0000 7fad5ce00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b335e18b60 0x55b335e2a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54628/0 (socket says 172.21.15.97:54628) 2024-03-27T10:57:55.940 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.964+0000 7fad5ce00640 1 -- 172.21.15.97:0/3241391920 learned_addr learned my addr 172.21.15.97:0/3241391920 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:57:55.940 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.965+0000 7fad57e00640 1 --2- 172.21.15.97:0/3241391920 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b335e19550 0x55b335e26b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:55.940 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.965+0000 7fad5d800640 1 --2- 172.21.15.97:0/3241391920 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b335e1e6c0 0x55b335e270c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:55.940 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.965+0000 7fad5ce00640 1 -- 172.21.15.97:0/3241391920 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b335e1e6c0 msgr2=0x55b335e270c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:55.940 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.965+0000 7fad5ce00640 1 --2- 172.21.15.97:0/3241391920 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b335e1e6c0 0x55b335e270c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:55.940 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.965+0000 7fad5ce00640 1 -- 172.21.15.97:0/3241391920 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b335e19550 msgr2=0x55b335e26b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:57:55.940 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.965+0000 7fad5ce00640 1 --2- 172.21.15.97:0/3241391920 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b335e19550 0x55b335e26b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:55.940 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.965+0000 7fad5ce00640 1 -- 172.21.15.97:0/3241391920 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b335e276d0 con 0x55b335e18b60 2024-03-27T10:57:55.940 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.965+0000 7fad5ce00640 1 --2- 172.21.15.97:0/3241391920 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b335e18b60 0x55b335e2a090 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fad4c0d39f0 tx=0x7fad4c09aba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9d64be8e0cdfec2 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:57:55.940 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.965+0000 7fad3ea00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T10:57:55.940 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.966+0000 7fad3ea00640 1 -- 172.21.15.97:0/3241391920 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad4c098930 con 0x55b335e18b60 2024-03-27T10:57:55.940 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.966+0000 7fad3ea00640 1 -- 172.21.15.97:0/3241391920 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fad4c09bae0 con 0x55b335e18b60 2024-03-27T10:57:55.941 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.966+0000 7fad62401740 1 -- 172.21.15.97:0/3241391920 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad50051070 con 0x55b335e18b60 2024-03-27T10:57:55.941 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.966+0000 7fad3ea00640 1 -- 172.21.15.97:0/3241391920 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fad4c09c690 con 0x55b335e18b60 2024-03-27T10:57:55.941 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.966+0000 7fad62401740 10 client.16035 fetch_fsmap learned FSMap version 28 2024-03-27T10:57:55.941 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.966+0000 7fad62401740 1 -- 172.21.15.97:0/3241391920 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b335e941d0 con 0x55b335e18b60 2024-03-27T10:57:55.941 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.967+0000 7fad3ea00640 1 -- 172.21.15.97:0/3241391920 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fad50051070 con 0x55b335e18b60 2024-03-27T10:57:55.942 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.967+0000 7fad3ea00640 1 -- 172.21.15.97:0/3241391920 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fad4c0bb020 con 0x55b335e18b60 2024-03-27T10:57:55.942 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.968+0000 7fad62401740 10 client.16035 fetch_fsmap finished waiting for FSMap version 28 2024-03-27T10:57:55.942 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.968+0000 7fad62401740 10 client.16035 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T10:57:55.942 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.968+0000 7fad62401740 20 client.16035 populate_metadata read hostname 'smithi097' 2024-03-27T10:57:55.942 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.968+0000 7fad62401740 1 --2- 172.21.15.97:0/3241391920 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55b335e94a10 0x55b335e96e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:57:55.942 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.968+0000 7fad62401740 4 client.16035 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-27T10:57:55.942 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.968+0000 7fad62401740 1 -- 172.21.15.97:0/3241391920 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b335d855a0 con 0x55b335e94a10 2024-03-27T10:57:55.942 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.968+0000 7fad57e00640 1 --2- 172.21.15.97:0/3241391920 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55b335e94a10 0x55b335e96e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:57:55.942 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.969+0000 7fad57e00640 1 --2- 172.21.15.97:0/3241391920 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55b335e94a10 0x55b335e96e30 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e88de74d566503ef server_cookie=81939d257e17edcc in_seq=0 out_seq=0 2024-03-27T10:57:55.942 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.969+0000 7fad3ea00640 10 client.16035 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T10:57:55.948 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.976+0000 7fad3ea00640 1 -- 172.21.15.97:0/3241391920 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b335d855a0 con 0x55b335e94a10 2024-03-27T10:57:55.948 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.976+0000 7fad3ea00640 10 client.16035 handle_command_reply: tid=0 2024-03-27T10:57:55.948 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.976+0000 7fad62401740 1 client.16035 shutdown 2024-03-27T10:57:55.948 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.977+0000 7fad62401740 1 -- 172.21.15.97:0/3241391920 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b335e18b60 msgr2=0x55b335e2a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:55.949 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.977+0000 7fad62401740 1 --2- 172.21.15.97:0/3241391920 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b335e18b60 0x55b335e2a090 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fad4c0d39f0 tx=0x7fad4c09aba0 comp rx=0 tx=0).stop 2024-03-27T10:57:55.949 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.977+0000 7fad62401740 1 -- 172.21.15.97:0/3241391920 shutdown_connections 2024-03-27T10:57:55.949 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.977+0000 7fad62401740 1 --2- 172.21.15.97:0/3241391920 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55b335e94a10 0x55b335e96e30 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:55.949 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.977+0000 7fad62401740 1 --2- 172.21.15.97:0/3241391920 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b335e1e6c0 0x55b335e270c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:55.949 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.977+0000 7fad62401740 1 --2- 172.21.15.97:0/3241391920 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b335e19550 0x55b335e26b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:55.949 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.977+0000 7fad62401740 1 --2- 172.21.15.97:0/3241391920 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b335e18b60 0x55b335e2a090 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:55.949 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.977+0000 7fad62401740 1 -- 172.21.15.97:0/3241391920 >> 172.21.15.97:0/3241391920 conn(0x55b335e140d0 msgr2=0x55b335e2ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:55.949 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.977+0000 7fad3ea00640 0 client.16035 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T10:57:55.951 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.980+0000 7fad62401740 1 -- 172.21.15.97:0/3241391920 shutdown_connections 2024-03-27T10:57:55.951 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.980+0000 7fad62401740 1 -- 172.21.15.97:0/3241391920 wait complete. 2024-03-27T10:57:55.952 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.980+0000 7fad62401740 20 client.16035 trim_cache size 0 max 16384 2024-03-27T10:57:55.952 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-27T10:57:55.952 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-27T10:57:55.952 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-27T10:57:55.952 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-27T10:57:55.953 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.981+0000 7fad5e200640 1 -- 172.21.15.97:0/111339274 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fad400a3680 msgr2=0x7fad400a5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:55.954 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.981+0000 7fad5e200640 1 --2- 172.21.15.97:0/111339274 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fad400a3680 0x7fad400a5b40 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fad5002c630 tx=0x7fad5002e040 comp rx=0 tx=0).stop 2024-03-27T10:57:55.954 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.981+0000 7fad5e200640 1 -- 172.21.15.97:0/111339274 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad581019c0 msgr2=0x7fad5819e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:57:55.954 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.981+0000 7fad5e200640 1 --2- 172.21.15.97:0/111339274 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad581019c0 0x7fad5819e150 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fad4c00e9c0 tx=0x7fad4c00ee80 comp rx=0 tx=0).stop 2024-03-27T10:57:55.954 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.981+0000 7fad5e200640 1 -- 172.21.15.97:0/111339274 shutdown_connections 2024-03-27T10:57:55.954 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.981+0000 7fad5e200640 1 --2- 172.21.15.97:0/111339274 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fad58103570 0x7fad581a5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:55.954 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.981+0000 7fad5e200640 1 --2- 172.21.15.97:0/111339274 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fad400a3680 0x7fad400a5b40 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:55.954 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.981+0000 7fad5e200640 1 --2- 172.21.15.97:0/111339274 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fad58102bb0 0x7fad5819e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:55.954 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.981+0000 7fad5e200640 1 --2- 172.21.15.97:0/111339274 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad581019c0 0x7fad5819e150 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:57:55.954 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.981+0000 7fad5e200640 1 -- 172.21.15.97:0/111339274 >> 172.21.15.97:0/111339274 conn(0x7fad580fcf30 msgr2=0x7fad5810ded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:57:55.954 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.981+0000 7fad5e200640 1 -- 172.21.15.97:0/111339274 shutdown_connections 2024-03-27T10:57:55.954 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:57:55.981+0000 7fad5e200640 1 -- 172.21.15.97:0/111339274 wait complete. 2024-03-27T10:57:56.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:57:56 smithi097 ceph-mon[31859]: pgmap v480: 97 pgs: 97 active+clean; 665 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 0 B/s wr, 11 op/s 2024-03-27T10:57:56.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:57:56 smithi177 ceph-mon[35294]: pgmap v480: 97 pgs: 97 active+clean; 665 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 0 B/s wr, 11 op/s 2024-03-27T10:57:56.766 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:57:56 smithi114 ceph-mon[35338]: pgmap v480: 97 pgs: 97 active+clean; 665 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 0 B/s wr, 11 op/s 2024-03-27T10:57:59.516 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:57:59 smithi114 ceph-mon[35338]: pgmap v481: 97 pgs: 97 active+clean; 665 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 0 B/s wr, 11 op/s 2024-03-27T10:57:59.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:57:59 smithi097 ceph-mon[31859]: pgmap v481: 97 pgs: 97 active+clean; 665 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 0 B/s wr, 11 op/s 2024-03-27T10:57:59.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:57:59 smithi177 ceph-mon[35294]: pgmap v481: 97 pgs: 97 active+clean; 665 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 0 B/s wr, 11 op/s 2024-03-27T10:58:01.443 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:58:01 smithi097 ceph-mon[31859]: pgmap v482: 97 pgs: 97 active+clean; 666 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 255 B/s wr, 13 op/s 2024-03-27T10:58:01.516 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:58:01 smithi114 ceph-mon[35338]: pgmap v482: 97 pgs: 97 active+clean; 666 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 255 B/s wr, 13 op/s 2024-03-27T10:58:01.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:58:01 smithi177 ceph-mon[35294]: pgmap v482: 97 pgs: 97 active+clean; 666 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 255 B/s wr, 13 op/s 2024-03-27T10:58:01.810 INFO:tasks.workunit.client.0.smithi097.stderr:Updating files: 63% (7845/12271) Updating files: 64% (7854/12271) Updating files: 65% (7977/12271) Updating files: 66% (8099/12271) Updating files: 67% (8222/12271) Updating files: 68% (8345/12271) Updating files: 69% (8467/12271) Updating files: 70% (8590/12271) Updating files: 71% (8713/12271) Updating files: 72% (8836/12271) Updating files: 73% (8958/12271) Updating files: 74% (9081/12271) Updating files: 75% (9204/12271) Updating files: 76% (9326/12271) Updating files: 77% (9449/12271) Updating files: 78% (9572/12271) Updating files: 79% (9695/12271) Updating files: 80% (9817/12271) Updating files: 81% (9940/12271) Updating files: 82% (10063/12271) Updating files: 83% (10185/12271) Updating files: 84% (10308/12271) Updating files: 85% (10431/12271) Updating files: 86% (10554/12271) Updating files: 87% (10676/12271) Updating files: 88% (10799/12271) Updating files: 89% (10922/12271) Updating files: 90% (11044/12271) Updating files: 91% (11167/12271) Updating files: 92% (11290/12271) Updating files: 93% (11413/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-03-27T10:58:02.665 INFO:tasks.workunit.client.0.smithi097.stderr:Note: switching to '155268c4e432a12433aa833f174f9fe3b1016ae0'. 2024-03-27T10:58:02.665 INFO:tasks.workunit.client.0.smithi097.stderr: 2024-03-27T10:58:02.666 INFO:tasks.workunit.client.0.smithi097.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-03-27T10:58:02.666 INFO:tasks.workunit.client.0.smithi097.stderr:changes and commit them, and you can discard any commits you make in this 2024-03-27T10:58:02.666 INFO:tasks.workunit.client.0.smithi097.stderr:state without impacting any branches by switching back to a branch. 2024-03-27T10:58:02.666 INFO:tasks.workunit.client.0.smithi097.stderr: 2024-03-27T10:58:02.666 INFO:tasks.workunit.client.0.smithi097.stderr:If you want to create a new branch to retain commits you create, you may 2024-03-27T10:58:02.666 INFO:tasks.workunit.client.0.smithi097.stderr:do so (now or later) by using -c with the switch command. Example: 2024-03-27T10:58:02.666 INFO:tasks.workunit.client.0.smithi097.stderr: 2024-03-27T10:58:02.666 INFO:tasks.workunit.client.0.smithi097.stderr: git switch -c 2024-03-27T10:58:02.666 INFO:tasks.workunit.client.0.smithi097.stderr: 2024-03-27T10:58:02.666 INFO:tasks.workunit.client.0.smithi097.stderr:Or undo this operation with: 2024-03-27T10:58:02.667 INFO:tasks.workunit.client.0.smithi097.stderr: 2024-03-27T10:58:02.667 INFO:tasks.workunit.client.0.smithi097.stderr: git switch - 2024-03-27T10:58:02.667 INFO:tasks.workunit.client.0.smithi097.stderr: 2024-03-27T10:58:02.667 INFO:tasks.workunit.client.0.smithi097.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-03-27T10:58:02.667 INFO:tasks.workunit.client.0.smithi097.stderr: 2024-03-27T10:58:02.667 INFO:tasks.workunit.client.0.smithi097.stderr:HEAD is now at 155268c4e43 Merge PR #51993 into wip-rishabh-testing-20240326.131558 2024-03-27T10:58:02.677 DEBUG:teuthology.orchestra.run.smithi097:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-03-27T10:58:02.732 INFO:tasks.workunit.client.0.smithi097.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-03-27T10:58:02.740 INFO:tasks.workunit.client.0.smithi097.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-27T10:58:02.740 INFO:tasks.workunit.client.0.smithi097.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-03-27T10:58:03.516 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:58:03 smithi114 ceph-mon[35338]: pgmap v483: 97 pgs: 97 active+clean; 666 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-27T10:58:03.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:58:03 smithi097 ceph-mon[31859]: pgmap v483: 97 pgs: 97 active+clean; 666 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-27T10:58:03.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:58:03 smithi177 ceph-mon[35294]: pgmap v483: 97 pgs: 97 active+clean; 666 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-27T10:58:03.741 INFO:tasks.workunit.client.0.smithi097.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-03-27T10:58:03.875 INFO:tasks.workunit.client.0.smithi097.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-03-27T10:58:03.960 INFO:tasks.workunit.client.0.smithi097.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-27T10:58:03.966 INFO:tasks.workunit.client.0.smithi097.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-27T10:58:03.966 INFO:tasks.workunit.client.0.smithi097.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-03-27T10:58:04.047 INFO:tasks.workunit.client.0.smithi097.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-27T10:58:04.056 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-27T10:58:04.056 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-03-27T10:58:04.103 INFO:tasks.workunit:Running workunits matching suites/iogen.sh on client.0... 2024-03-27T10:58:04.104 INFO:tasks.workunit:Running workunit suites/iogen.sh... 2024-03-27T10:58:04.105 DEBUG:teuthology.orchestra.run.smithi097:workunit test suites/iogen.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=155268c4e432a12433aa833f174f9fe3b1016ae0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/iogen.sh 2024-03-27T10:58:04.243 INFO:tasks.workunit.client.0.smithi097.stdout:getting iogen 2024-03-27T10:58:04.243 INFO:tasks.workunit.client.0.smithi097.stderr:+ echo 'getting iogen' 2024-03-27T10:58:04.243 INFO:tasks.workunit.client.0.smithi097.stderr:+ wget http://download.ceph.com/qa/iogen_3.1p0.tar 2024-03-27T10:58:04.277 INFO:tasks.workunit.client.0.smithi097.stderr:--2024-03-27 10:58:04-- http://download.ceph.com/qa/iogen_3.1p0.tar 2024-03-27T10:58:04.288 INFO:tasks.workunit.client.0.smithi097.stderr:Resolving download.ceph.com (download.ceph.com)... 158.69.68.124, 2607:5300:201:2000::3:58a1 2024-03-27T10:58:04.309 INFO:tasks.workunit.client.0.smithi097.stderr:Connecting to download.ceph.com (download.ceph.com)|158.69.68.124|:80... connected. 2024-03-27T10:58:04.330 INFO:tasks.workunit.client.0.smithi097.stderr:HTTP request sent, awaiting response... 200 OK 2024-03-27T10:58:04.330 INFO:tasks.workunit.client.0.smithi097.stderr:Length: 8071 (7.9K) [application/octet-stream] 2024-03-27T10:58:04.337 INFO:tasks.workunit.client.0.smithi097.stderr:Saving to: ‘iogen_3.1p0.tar’ 2024-03-27T10:58:04.348 INFO:tasks.workunit.client.0.smithi097.stderr: 2024-03-27T10:58:04.348 INFO:tasks.workunit.client.0.smithi097.stderr: 0K ....... 100% 270M=0s 2024-03-27T10:58:04.348 INFO:tasks.workunit.client.0.smithi097.stderr: 2024-03-27T10:58:04.350 INFO:tasks.workunit.client.0.smithi097.stderr:2024-03-27 10:58:04 (270 MB/s) - ‘iogen_3.1p0.tar’ saved [8071/8071] 2024-03-27T10:58:04.350 INFO:tasks.workunit.client.0.smithi097.stderr: 2024-03-27T10:58:04.352 INFO:tasks.workunit.client.0.smithi097.stderr:+ tar -xvzf iogen_3.1p0.tar 2024-03-27T10:58:04.377 INFO:tasks.workunit.client.0.smithi097.stdout:iogen_3.1p0 2024-03-27T10:58:04.381 INFO:tasks.workunit.client.0.smithi097.stdout:iogen_3.1p0/src 2024-03-27T10:58:04.387 INFO:tasks.workunit.client.0.smithi097.stdout:iogen_3.1p0/src/iogen.8 2024-03-27T10:58:04.395 INFO:tasks.workunit.client.0.smithi097.stdout:iogen_3.1p0/src/iogen.c 2024-03-27T10:58:04.407 INFO:tasks.workunit.client.0.smithi097.stdout:iogen_3.1p0/README 2024-03-27T10:58:04.417 INFO:tasks.workunit.client.0.smithi097.stdout:iogen_3.1p0/Makefile.bsd 2024-03-27T10:58:04.425 INFO:tasks.workunit.client.0.smithi097.stdout:iogen_3.1p0/Makefile 2024-03-27T10:58:04.434 INFO:tasks.workunit.client.0.smithi097.stderr:+ cd iogen_3.1p0 2024-03-27T10:58:04.441 INFO:tasks.workunit.client.0.smithi097.stdout:making iogen 2024-03-27T10:58:04.441 INFO:tasks.workunit.client.0.smithi097.stderr:+ echo 'making iogen' 2024-03-27T10:58:04.441 INFO:tasks.workunit.client.0.smithi097.stderr:+ make 2024-03-27T10:58:04.477 INFO:tasks.workunit.client.0.smithi097.stdout:gcc -D_FILE_OFFSET_BITS=64 -DVERSION=\"3.1p0\" -Dstrlcpy=strncpy -Darc4random=rand -O2 src/iogen.c -o iogen 2024-03-27T10:58:05.162 INFO:tasks.workunit.client.0.smithi097.stdout:nroff -mandoc -Tascii src/iogen.8 > iogen.cat8 2024-03-27T10:58:05.346 INFO:tasks.workunit.client.0.smithi097.stdout:running iogen 2024-03-27T10:58:05.347 INFO:tasks.workunit.client.0.smithi097.stderr:+ echo 'running iogen' 2024-03-27T10:58:05.347 INFO:tasks.workunit.client.0.smithi097.stderr:+ ./iogen -n 5 -s 2g 2024-03-27T10:58:05.351 INFO:tasks.workunit.client.0.smithi097.stderr:child spawned 88206 2024-03-27T10:58:05.351 INFO:tasks.workunit.client.0.smithi097.stderr:child spawned 88207 2024-03-27T10:58:05.351 INFO:tasks.workunit.client.0.smithi097.stderr:child spawned 88208 2024-03-27T10:58:05.352 INFO:tasks.workunit.client.0.smithi097.stderr:child spawned 88209 2024-03-27T10:58:05.352 INFO:tasks.workunit.client.0.smithi097.stderr:child spawned 88210 2024-03-27T10:58:05.353 INFO:tasks.workunit.client.0.smithi097.stderr:+ echo 'sleep for 10 min' 2024-03-27T10:58:05.354 INFO:tasks.workunit.client.0.smithi097.stdout:sleep for 10 min 2024-03-27T10:58:05.354 INFO:tasks.workunit.client.0.smithi097.stderr:+ sleep 600 2024-03-27T10:58:05.516 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:58:05 smithi114 ceph-mon[35338]: pgmap v484: 97 pgs: 97 active+clean; 666 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-27T10:58:05.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:58:05 smithi097 ceph-mon[31859]: pgmap v484: 97 pgs: 97 active+clean; 666 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-27T10:58:05.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:58:05 smithi177 ceph-mon[35294]: pgmap v484: 97 pgs: 97 active+clean; 666 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-27T10:58:07.516 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:58:07 smithi114 ceph-mon[35338]: pgmap v485: 97 pgs: 97 active+clean; 675 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1023 B/s wr, 10 op/s 2024-03-27T10:58:07.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:58:07 smithi097 ceph-mon[31859]: pgmap v485: 97 pgs: 97 active+clean; 675 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1023 B/s wr, 10 op/s 2024-03-27T10:58:07.689 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:58:07 smithi177 ceph-mon[35294]: pgmap v485: 97 pgs: 97 active+clean; 675 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1023 B/s wr, 10 op/s 2024-03-27T10:58:09.135 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.163+0000 7fb2ec200640 1 -- 172.21.15.97:0/3620788931 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2e41026b0 msgr2=0x7fb2e4104aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:09.135 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.163+0000 7fb2ec200640 1 --2- 172.21.15.97:0/3620788931 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2e41026b0 0x7fb2e4104aa0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fb2d8009900 tx=0x7fb2d802bc30 comp rx=0 tx=0).stop 2024-03-27T10:58:09.135 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.163+0000 7fb2ec200640 1 -- 172.21.15.97:0/3620788931 shutdown_connections 2024-03-27T10:58:09.136 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.163+0000 7fb2ec200640 1 --2- 172.21.15.97:0/3620788931 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2e4107910 0x7fb2e4109d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:09.136 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.163+0000 7fb2ec200640 1 --2- 172.21.15.97:0/3620788931 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2e4104fe0 0x7fb2e41073d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:09.136 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.163+0000 7fb2ec200640 1 --2- 172.21.15.97:0/3620788931 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2e41026b0 0x7fb2e4104aa0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:09.136 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.163+0000 7fb2ec200640 1 -- 172.21.15.97:0/3620788931 >> 172.21.15.97:0/3620788931 conn(0x7fb2e40fbfe0 msgr2=0x7fb2e40fe420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:09.136 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.163+0000 7fb2ec200640 1 -- 172.21.15.97:0/3620788931 shutdown_connections 2024-03-27T10:58:09.136 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.163+0000 7fb2ec200640 1 -- 172.21.15.97:0/3620788931 wait complete. 2024-03-27T10:58:09.136 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.165+0000 7fb2ec200640 1 Processor -- start 2024-03-27T10:58:09.137 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.165+0000 7fb2ec200640 1 -- start start 2024-03-27T10:58:09.138 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.166+0000 7fb2ec200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2e41026b0 0x7fb2e419e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:09.138 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.166+0000 7fb2ec200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2e4104fe0 0x7fb2e419e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:09.138 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.166+0000 7fb2ec200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2e4107910 0x7fb2e41a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:09.138 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.166+0000 7fb2ec200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e410b500 con 0x7fb2e4104fe0 2024-03-27T10:58:09.138 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.166+0000 7fb2ec200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e410b380 con 0x7fb2e41026b0 2024-03-27T10:58:09.138 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.166+0000 7fb2ec200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e410b680 con 0x7fb2e4107910 2024-03-27T10:58:09.138 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.166+0000 7fb2e9800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2e41026b0 0x7fb2e419e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:09.138 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.166+0000 7fb2e8e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2e4104fe0 0x7fb2e419e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:09.138 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.166+0000 7fb2e9800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2e41026b0 0x7fb2e419e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:34468/0 (socket says 172.21.15.97:34468) 2024-03-27T10:58:09.139 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.166+0000 7fb2e9800640 1 -- 172.21.15.97:0/1066883607 learned_addr learned my addr 172.21.15.97:0/1066883607 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:58:09.140 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.167+0000 7fb2ea200640 1 --2- 172.21.15.97:0/1066883607 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2e4107910 0x7fb2e41a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:09.140 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.167+0000 7fb2e9800640 1 -- 172.21.15.97:0/1066883607 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2e4107910 msgr2=0x7fb2e41a58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:09.140 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.167+0000 7fb2e9800640 1 --2- 172.21.15.97:0/1066883607 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2e4107910 0x7fb2e41a58f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:09.140 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.167+0000 7fb2e9800640 1 -- 172.21.15.97:0/1066883607 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2e4104fe0 msgr2=0x7fb2e419e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:09.140 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.167+0000 7fb2e9800640 1 --2- 172.21.15.97:0/1066883607 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2e4104fe0 0x7fb2e419e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:09.140 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.167+0000 7fb2e9800640 1 -- 172.21.15.97:0/1066883607 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2e41a5f60 con 0x7fb2e41026b0 2024-03-27T10:58:09.140 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.167+0000 7fb2e9800640 1 --2- 172.21.15.97:0/1066883607 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2e41026b0 0x7fb2e419e340 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb2d8000c00 tx=0x7fb2d80042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:09.140 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.168+0000 7fb2e2000640 1 -- 172.21.15.97:0/1066883607 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d8043020 con 0x7fb2e41026b0 2024-03-27T10:58:09.140 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.168+0000 7fb2e2000640 1 -- 172.21.15.97:0/1066883607 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb2d8004aa0 con 0x7fb2e41026b0 2024-03-27T10:58:09.140 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.168+0000 7fb2e2000640 1 -- 172.21.15.97:0/1066883607 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d8003d00 con 0x7fb2e41026b0 2024-03-27T10:58:09.140 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.168+0000 7fb2ec200640 1 -- 172.21.15.97:0/1066883607 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2e41a61f0 con 0x7fb2e41026b0 2024-03-27T10:58:09.140 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.168+0000 7fb2ec200640 1 -- 172.21.15.97:0/1066883607 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2e41a66f0 con 0x7fb2e41026b0 2024-03-27T10:58:09.144 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.172+0000 7fb2eede1740 1 Processor -- start 2024-03-27T10:58:09.144 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.172+0000 7fb2eede1740 1 -- start start 2024-03-27T10:58:09.144 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.173+0000 7fb2eede1740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560441ac8980 0x560441ac8da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:09.145 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.173+0000 7fb2eede1740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560441ac9370 0x560441acde70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:09.146 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.173+0000 7fb2eede1740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560441ace4e0 0x560441ad0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:09.146 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.173+0000 7fb2e2000640 1 -- 172.21.15.97:0/1066883607 <== mon.2 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7fb2d8046020 con 0x7fb2e41026b0 2024-03-27T10:58:09.146 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.173+0000 7fb2eede1740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x560441a2dd10 con 0x560441ace4e0 2024-03-27T10:58:09.146 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.173+0000 7fb2eede1740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5604419d6260 con 0x560441ac9370 2024-03-27T10:58:09.146 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.173+0000 7fb2eede1740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x560441a29cc0 con 0x560441ac8980 2024-03-27T10:58:09.146 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.174+0000 7fb2ea200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560441ac8980 0x560441ac8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:09.146 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.174+0000 7fb2ea200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560441ac8980 0x560441ac8da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.97:53124/0 (socket says 172.21.15.97:53124) 2024-03-27T10:58:09.146 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.174+0000 7fb2ea200640 1 -- 172.21.15.97:0/2901033950 learned_addr learned my addr 172.21.15.97:0/2901033950 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:58:09.146 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.174+0000 7fb2e9800640 1 --2- 172.21.15.97:0/2901033950 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560441ac9370 0x560441acde70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:09.147 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.174+0000 7fb2e8e00640 1 --2- 172.21.15.97:0/2901033950 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560441ace4e0 0x560441ad0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:09.147 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.174+0000 7fb2ea200640 1 -- 172.21.15.97:0/2901033950 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560441ac9370 msgr2=0x560441acde70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:09.147 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.174+0000 7fb2ea200640 1 --2- 172.21.15.97:0/2901033950 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560441ac9370 0x560441acde70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:09.148 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.174+0000 7fb2ea200640 1 -- 172.21.15.97:0/2901033950 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560441ace4e0 msgr2=0x560441ad0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:09.148 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.175+0000 7fb2ea200640 1 --2- 172.21.15.97:0/2901033950 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560441ace4e0 0x560441ad0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:09.148 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.175+0000 7fb2ea200640 1 -- 172.21.15.97:0/2901033950 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604419cdaa0 con 0x560441ac8980 2024-03-27T10:58:09.148 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.175+0000 7fb2e9800640 1 --2- 172.21.15.97:0/2901033950 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560441ac9370 0x560441acde70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T10:58:09.148 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.175+0000 7fb2e2000640 1 --2- 172.21.15.97:0/1066883607 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fb2cc0a3750 0x7fb2cc0a5c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:09.148 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.175+0000 7fb2e8e00640 1 --2- 172.21.15.97:0/2901033950 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560441ace4e0 0x560441ad0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:58:09.148 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.175+0000 7fb2ea200640 1 --2- 172.21.15.97:0/2901033950 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560441ac8980 0x560441ac8da0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc009ee0 tx=0x7fb2dc010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1248a406752c1a2 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:09.148 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.175+0000 7fb2e2000640 1 -- 172.21.15.97:0/1066883607 <== mon.2 v2:172.21.15.114:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fb2d80f1fa0 con 0x7fb2e41026b0 2024-03-27T10:58:09.149 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.176+0000 7fb2d6a00640 1 -- 172.21.15.97:0/2901033950 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc016070 con 0x560441ac8980 2024-03-27T10:58:09.149 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.176+0000 7fb2e9800640 1 --2- 172.21.15.97:0/1066883607 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fb2cc0a3750 0x7fb2cc0a5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:09.149 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.176+0000 7fb2d6a00640 1 -- 172.21.15.97:0/2901033950 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb2dc004d20 con 0x560441ac8980 2024-03-27T10:58:09.149 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.176+0000 7fb2d6a00640 1 -- 172.21.15.97:0/2901033950 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc0126b0 con 0x560441ac8980 2024-03-27T10:58:09.149 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.177+0000 7fb2eede1740 1 -- 172.21.15.97:0/2901033950 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560441ac8980 msgr2=0x560441ac8da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:09.149 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.177+0000 7fb2eede1740 1 --2- 172.21.15.97:0/2901033950 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560441ac8980 0x560441ac8da0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc009ee0 tx=0x7fb2dc010740 comp rx=0 tx=0).stop 2024-03-27T10:58:09.150 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.178+0000 7fb2e9800640 1 --2- 172.21.15.97:0/1066883607 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fb2cc0a3750 0x7fb2cc0a5c10 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x560441ac9770 tx=0x7fb2d8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:09.150 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.178+0000 7fb2eede1740 1 -- 172.21.15.97:0/2901033950 shutdown_connections 2024-03-27T10:58:09.150 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.178+0000 7fb2eede1740 1 --2- 172.21.15.97:0/2901033950 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560441ace4e0 0x560441ad0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:09.150 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.178+0000 7fb2eede1740 1 --2- 172.21.15.97:0/2901033950 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560441ac9370 0x560441acde70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:09.150 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.178+0000 7fb2eede1740 1 --2- 172.21.15.97:0/2901033950 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560441ac8980 0x560441ac8da0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:09.150 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.178+0000 7fb2eede1740 1 -- 172.21.15.97:0/2901033950 >> 172.21.15.97:0/2901033950 conn(0x560441ac3ef0 msgr2=0x560441ac42f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:09.150 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.178+0000 7fb2eede1740 1 -- 172.21.15.97:0/2901033950 shutdown_connections 2024-03-27T10:58:09.150 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.178+0000 7fb2eede1740 1 -- 172.21.15.97:0/2901033950 wait complete. 2024-03-27T10:58:09.151 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.179+0000 7fb2eede1740 1 -- start start 2024-03-27T10:58:09.151 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.179+0000 7fb2eede1740 1 Processor -- start 2024-03-27T10:58:09.152 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.180+0000 7fb2eede1740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560441ac8980 0x560441ad2d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:09.152 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.180+0000 7fb2eede1740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560441ac9370 0x560441ad3250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:09.152 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.180+0000 7fb2eede1740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560441ace4e0 0x560441ada290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:09.152 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.180+0000 7fb2eede1740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x560441ad1490 con 0x560441ac9370 2024-03-27T10:58:09.152 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.180+0000 7fb2eede1740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5604419e9630 con 0x560441ace4e0 2024-03-27T10:58:09.152 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.180+0000 7fb2eede1740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x560441ad1610 con 0x560441ac8980 2024-03-27T10:58:09.152 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.180+0000 7fb2eede1740 10 client.0 _ll_register_callbacks cb 0x5604419eec50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:58:09.153 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.181+0000 7fb2e8e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560441ac9370 0x560441ad3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:09.153 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.181+0000 7fb2e8e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560441ac9370 0x560441ad3250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46828/0 (socket says 172.21.15.97:46828) 2024-03-27T10:58:09.153 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.181+0000 7fb2e8e00640 1 -- 172.21.15.97:0/246699867 learned_addr learned my addr 172.21.15.97:0/246699867 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:58:09.153 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.181+0000 7fb2ea200640 1 --2- 172.21.15.97:0/246699867 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560441ace4e0 0x560441ada290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:09.153 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.181+0000 7fb2ea200640 1 --2- 172.21.15.97:0/246699867 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560441ac8980 0x560441ad2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:09.154 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.182+0000 7fb2e8e00640 1 -- 172.21.15.97:0/246699867 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560441ac8980 msgr2=0x560441ad2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:09.154 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.182+0000 7fb2e8e00640 1 --2- 172.21.15.97:0/246699867 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560441ac8980 0x560441ad2d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:09.154 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.182+0000 7fb2e8e00640 1 -- 172.21.15.97:0/246699867 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560441ace4e0 msgr2=0x560441ada290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:09.154 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.182+0000 7fb2e8e00640 1 --2- 172.21.15.97:0/246699867 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560441ace4e0 0x560441ada290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:09.154 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.182+0000 7fb2e8e00640 1 -- 172.21.15.97:0/246699867 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604419f55f0 con 0x560441ac9370 2024-03-27T10:58:09.154 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.182+0000 7fb2ea200640 1 --2- 172.21.15.97:0/246699867 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560441ace4e0 0x560441ada290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T10:58:09.155 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.183+0000 7fb2e8e00640 1 --2- 172.21.15.97:0/246699867 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560441ac9370 0x560441ad3250 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fb2d0013a50 tx=0x7fb2d0013f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb81856c7abaeafb server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:09.155 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.183+0000 7fb2cb400640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T10:58:09.155 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.184+0000 7fb2cb400640 1 -- 172.21.15.97:0/246699867 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d0020060 con 0x560441ac9370 2024-03-27T10:58:09.156 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.184+0000 7fb2cb400640 1 -- 172.21.15.97:0/246699867 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fb2d0004310 con 0x560441ac9370 2024-03-27T10:58:09.156 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.184+0000 7fb2eede1740 1 -- 172.21.15.97:0/246699867 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2dc004d20 con 0x560441ac9370 2024-03-27T10:58:09.156 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.184+0000 7fb2cb400640 1 -- 172.21.15.97:0/246699867 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb2d00118e0 con 0x560441ac9370 2024-03-27T10:58:09.156 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.184+0000 7fb2eede1740 10 client.16044 fetch_fsmap learned FSMap version 28 2024-03-27T10:58:09.156 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.184+0000 7fb2eede1740 1 -- 172.21.15.97:0/246699867 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560441742210 con 0x560441ac9370 2024-03-27T10:58:09.156 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.185+0000 7fb2cb400640 1 -- 172.21.15.97:0/246699867 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb2dc004d20 con 0x560441ac9370 2024-03-27T10:58:09.157 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.185+0000 7fb2cb400640 1 -- 172.21.15.97:0/246699867 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fb2d0003c30 con 0x560441ac9370 2024-03-27T10:58:09.157 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.185+0000 7fb2eede1740 10 client.16044 fetch_fsmap finished waiting for FSMap version 28 2024-03-27T10:58:09.157 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.185+0000 7fb2eede1740 10 client.16044 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T10:58:09.157 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.185+0000 7fb2eede1740 20 client.16044 populate_metadata read hostname 'smithi097' 2024-03-27T10:58:09.157 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.185+0000 7fb2eede1740 1 --2- 172.21.15.97:0/246699867 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x560441aefb00 0x560441af1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:09.157 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.185+0000 7fb2eede1740 4 client.16044 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T10:58:09.157 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.185+0000 7fb2eede1740 1 -- 172.21.15.97:0/246699867 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560441a3b250 con 0x560441aefb00 2024-03-27T10:58:09.158 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.185+0000 7fb2ea200640 1 --2- 172.21.15.97:0/246699867 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x560441aefb00 0x560441af1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:09.159 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.187+0000 7fb2ea200640 1 --2- 172.21.15.97:0/246699867 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x560441aefb00 0x560441af1ef0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=831face71ca481d1 server_cookie=ddef699e2a441775 in_seq=0 out_seq=0 2024-03-27T10:58:09.159 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.187+0000 7fb2cb400640 10 client.16044 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T10:58:09.159 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.188+0000 7fb2cb400640 1 -- 172.21.15.97:0/246699867 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x560441a3b250 con 0x560441aefb00 2024-03-27T10:58:09.159 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.188+0000 7fb2cb400640 10 client.16044 handle_command_reply: tid=0 2024-03-27T10:58:09.160 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.188+0000 7fb2eede1740 1 client.16044 shutdown 2024-03-27T10:58:09.160 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.188+0000 7fb2eede1740 1 -- 172.21.15.97:0/246699867 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560441ac9370 msgr2=0x560441ad3250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:09.160 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.188+0000 7fb2eede1740 1 --2- 172.21.15.97:0/246699867 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560441ac9370 0x560441ad3250 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fb2d0013a50 tx=0x7fb2d0013f10 comp rx=0 tx=0).stop 2024-03-27T10:58:09.160 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.188+0000 7fb2eede1740 1 -- 172.21.15.97:0/246699867 shutdown_connections 2024-03-27T10:58:09.160 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.188+0000 7fb2eede1740 1 --2- 172.21.15.97:0/246699867 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x560441aefb00 0x560441af1ef0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:09.161 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.188+0000 7fb2eede1740 1 --2- 172.21.15.97:0/246699867 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560441ace4e0 0x560441ada290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:09.161 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.188+0000 7fb2eede1740 1 --2- 172.21.15.97:0/246699867 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560441ac9370 0x560441ad3250 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:09.161 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.188+0000 7fb2eede1740 1 --2- 172.21.15.97:0/246699867 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560441ac8980 0x560441ad2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:09.161 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.188+0000 7fb2eede1740 1 -- 172.21.15.97:0/246699867 >> 172.21.15.97:0/246699867 conn(0x560441ac3ef0 msgr2=0x560441ac30f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:09.161 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.188+0000 7fb2cb400640 0 client.16044 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T10:58:09.161 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.188+0000 7fb2eede1740 1 -- 172.21.15.97:0/246699867 shutdown_connections 2024-03-27T10:58:09.161 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.189+0000 7fb2eede1740 1 -- 172.21.15.97:0/246699867 wait complete. 2024-03-27T10:58:09.161 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.189+0000 7fb2eede1740 20 client.16044 trim_cache size 0 max 16384 2024-03-27T10:58:09.182 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.210+0000 7fb2eede1740 1 Processor -- start 2024-03-27T10:58:09.182 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.210+0000 7fb2eede1740 1 -- start start 2024-03-27T10:58:09.182 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.211+0000 7fb2eede1740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560441ac8980 0x560441ad2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:09.183 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.211+0000 7fb2eede1740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560441ace4e0 0x560441ada250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:09.183 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.211+0000 7fb2eede1740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560441acada0 0x560441ac2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:09.183 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.211+0000 7fb2eede1740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e41a6e40 con 0x560441acada0 2024-03-27T10:58:09.183 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.211+0000 7fb2eede1740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e41a8980 con 0x560441ace4e0 2024-03-27T10:58:09.183 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.211+0000 7fb2eede1740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e41a7510 con 0x560441ac8980 2024-03-27T10:58:09.183 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.211+0000 7fb2ea200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560441acada0 0x560441ac2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:09.183 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.211+0000 7fb2ea200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560441ac8980 0x560441ad2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:09.184 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.211+0000 7fb2e8e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560441ace4e0 0x560441ada250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:09.184 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.211+0000 7fb2ea200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560441ac8980 0x560441ad2330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.97:53140/0 (socket says 172.21.15.97:53140) 2024-03-27T10:58:09.184 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.211+0000 7fb2ea200640 1 -- 172.21.15.97:0/3724598915 learned_addr learned my addr 172.21.15.97:0/3724598915 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:58:09.184 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.212+0000 7fb2e8e00640 1 -- 172.21.15.97:0/3724598915 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560441ac8980 msgr2=0x560441ad2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:09.184 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.212+0000 7fb2e8e00640 1 --2- 172.21.15.97:0/3724598915 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560441ac8980 0x560441ad2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:09.184 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.212+0000 7fb2e8e00640 1 -- 172.21.15.97:0/3724598915 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560441acada0 msgr2=0x560441ac2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:09.185 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.212+0000 7fb2e8e00640 1 --2- 172.21.15.97:0/3724598915 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560441acada0 0x560441ac2860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:09.185 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.212+0000 7fb2e8e00640 1 -- 172.21.15.97:0/3724598915 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560441aca180 con 0x560441ace4e0 2024-03-27T10:58:09.185 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.212+0000 7fb2ea200640 1 --2- 172.21.15.97:0/3724598915 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560441ac8980 0x560441ad2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T10:58:09.185 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.213+0000 7fb2e8e00640 1 --2- 172.21.15.97:0/3724598915 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560441ace4e0 0x560441ada250 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb2d0010000 tx=0x7fb2d001b880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d891222e5e751b93 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:09.185 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.213+0000 7fb2d6000640 1 -- 172.21.15.97:0/3724598915 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d0000e60 con 0x560441ace4e0 2024-03-27T10:58:09.185 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.213+0000 7fb2d6000640 1 -- 172.21.15.97:0/3724598915 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb2d0011a80 con 0x560441ace4e0 2024-03-27T10:58:09.185 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.213+0000 7fb2d6000640 1 -- 172.21.15.97:0/3724598915 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d002a080 con 0x560441ace4e0 2024-03-27T10:58:09.186 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.214+0000 7fb2eede1740 1 -- 172.21.15.97:0/3724598915 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560441ace4e0 msgr2=0x560441ada250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:09.186 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.214+0000 7fb2eede1740 1 --2- 172.21.15.97:0/3724598915 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560441ace4e0 0x560441ada250 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb2d0010000 tx=0x7fb2d001b880 comp rx=0 tx=0).stop 2024-03-27T10:58:09.186 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.214+0000 7fb2eede1740 1 -- 172.21.15.97:0/3724598915 shutdown_connections 2024-03-27T10:58:09.186 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.214+0000 7fb2eede1740 1 --2- 172.21.15.97:0/3724598915 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560441acada0 0x560441ac2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:09.186 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.214+0000 7fb2eede1740 1 --2- 172.21.15.97:0/3724598915 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560441ace4e0 0x560441ada250 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:09.186 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.214+0000 7fb2eede1740 1 --2- 172.21.15.97:0/3724598915 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560441ac8980 0x560441ad2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:09.186 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.214+0000 7fb2eede1740 1 -- 172.21.15.97:0/3724598915 >> 172.21.15.97:0/3724598915 conn(0x560441ac3ef0 msgr2=0x560441aca690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:09.187 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.215+0000 7fb2eede1740 1 -- 172.21.15.97:0/3724598915 shutdown_connections 2024-03-27T10:58:09.187 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.215+0000 7fb2eede1740 1 -- 172.21.15.97:0/3724598915 wait complete. 2024-03-27T10:58:09.187 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.215+0000 7fb2eede1740 1 -- start start 2024-03-27T10:58:09.187 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.215+0000 7fb2eede1740 1 Processor -- start 2024-03-27T10:58:09.188 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.216+0000 7fb2eede1740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560441ac8980 0x560441ad7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:09.188 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.216+0000 7fb2eede1740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560441acada0 0x560441ad8420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:09.188 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.216+0000 7fb2eede1740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560441ace4e0 0x560441ad9430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:09.188 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.216+0000 7fb2eede1740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e41a82b0 con 0x560441ac8980 2024-03-27T10:58:09.188 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.216+0000 7fb2eede1740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e41a7be0 con 0x560441ace4e0 2024-03-27T10:58:09.188 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.216+0000 7fb2eede1740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e4072140 con 0x560441acada0 2024-03-27T10:58:09.189 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.216+0000 7fb2eede1740 10 client.0 _ll_register_callbacks cb 0x7fb2d0009490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:58:09.189 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.216+0000 7fb2ea200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560441ac8980 0x560441ad7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:09.189 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.216+0000 7fb2e8e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560441acada0 0x560441ad8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:09.189 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.216+0000 7fb2ea200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560441ac8980 0x560441ad7ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46858/0 (socket says 172.21.15.97:46858) 2024-03-27T10:58:09.189 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.216+0000 7fb2ea200640 1 -- 172.21.15.97:0/1989877507 learned_addr learned my addr 172.21.15.97:0/1989877507 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:58:09.190 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.217+0000 7fb2ea200640 1 -- 172.21.15.97:0/1989877507 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560441acada0 msgr2=0x560441ad8420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:09.190 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.217+0000 7fb2ea200640 1 --2- 172.21.15.97:0/1989877507 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560441acada0 0x560441ad8420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:09.190 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.217+0000 7fb2ea200640 1 -- 172.21.15.97:0/1989877507 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560441ace4e0 msgr2=0x560441ad9430 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-27T10:58:09.190 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.217+0000 7fb2ea200640 1 --2- 172.21.15.97:0/1989877507 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560441ace4e0 0x560441ad9430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:09.190 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.217+0000 7fb2ea200640 1 -- 172.21.15.97:0/1989877507 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56044180e620 con 0x560441ac8980 2024-03-27T10:58:09.190 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.217+0000 7fb2ea200640 1 --2- 172.21.15.97:0/1989877507 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560441ac8980 0x560441ad7ee0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc009ee0 tx=0x7fb2dc009690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0bcf97dca4e5ae7 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:09.190 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.217+0000 7fb2cb400640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T10:58:09.190 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.217+0000 7fb2cb400640 1 -- 172.21.15.97:0/1989877507 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc016040 con 0x560441ac8980 2024-03-27T10:58:09.190 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.217+0000 7fb2cb400640 1 -- 172.21.15.97:0/1989877507 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fb2dc029bc0 con 0x560441ac8980 2024-03-27T10:58:09.190 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.217+0000 7fb2eede1740 1 -- 172.21.15.97:0/1989877507 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2d0011a80 con 0x560441ac8980 2024-03-27T10:58:09.191 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.218+0000 7fb2cb400640 1 -- 172.21.15.97:0/1989877507 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb2dc029350 con 0x560441ac8980 2024-03-27T10:58:09.191 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.218+0000 7fb2eede1740 10 client.16050 fetch_fsmap learned FSMap version 28 2024-03-27T10:58:09.191 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.218+0000 7fb2eede1740 1 -- 172.21.15.97:0/1989877507 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560441b671d0 con 0x560441ac8980 2024-03-27T10:58:09.191 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.218+0000 7fb2cb400640 1 -- 172.21.15.97:0/1989877507 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb2d0011a80 con 0x560441ac8980 2024-03-27T10:58:09.191 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.218+0000 7fb2cb400640 1 -- 172.21.15.97:0/1989877507 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fb2dc027070 con 0x560441ac8980 2024-03-27T10:58:09.192 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.218+0000 7fb2eede1740 10 client.16050 fetch_fsmap finished waiting for FSMap version 28 2024-03-27T10:58:09.192 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.218+0000 7fb2eede1740 10 client.16050 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T10:58:09.192 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.218+0000 7fb2eede1740 20 client.16050 populate_metadata read hostname 'smithi097' 2024-03-27T10:58:09.192 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.218+0000 7fb2eede1740 1 --2- 172.21.15.97:0/1989877507 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x560441b67a70 0x560441b69e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:09.192 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.218+0000 7fb2eede1740 4 client.16050 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-27T10:58:09.192 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.218+0000 7fb2eede1740 1 -- 172.21.15.97:0/1989877507 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560441ad2b00 con 0x560441b67a70 2024-03-27T10:58:09.192 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.219+0000 7fb2e8e00640 1 --2- 172.21.15.97:0/1989877507 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x560441b67a70 0x560441b69e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:09.192 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.220+0000 7fb2e8e00640 1 --2- 172.21.15.97:0/1989877507 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x560441b67a70 0x560441b69e90 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a722eba9aeb29420 server_cookie=767ceccaad4a1577 in_seq=0 out_seq=0 2024-03-27T10:58:09.192 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.220+0000 7fb2cb400640 10 client.16050 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T10:58:09.217 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.245+0000 7fb2cb400640 1 -- 172.21.15.97:0/1989877507 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560441ad2b00 con 0x560441b67a70 2024-03-27T10:58:09.218 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.245+0000 7fb2cb400640 10 client.16050 handle_command_reply: tid=0 2024-03-27T10:58:09.218 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.245+0000 7fb2eede1740 1 client.16050 shutdown 2024-03-27T10:58:09.218 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.245+0000 7fb2eede1740 1 -- 172.21.15.97:0/1989877507 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560441ac8980 msgr2=0x560441ad7ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:09.218 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.245+0000 7fb2eede1740 1 --2- 172.21.15.97:0/1989877507 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560441ac8980 0x560441ad7ee0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc009ee0 tx=0x7fb2dc009690 comp rx=0 tx=0).stop 2024-03-27T10:58:09.218 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.245+0000 7fb2eede1740 1 -- 172.21.15.97:0/1989877507 shutdown_connections 2024-03-27T10:58:09.218 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.245+0000 7fb2eede1740 1 --2- 172.21.15.97:0/1989877507 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x560441b67a70 0x560441b69e90 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:09.218 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.245+0000 7fb2eede1740 1 --2- 172.21.15.97:0/1989877507 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560441ace4e0 0x560441ad9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:09.218 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.245+0000 7fb2eede1740 1 --2- 172.21.15.97:0/1989877507 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560441acada0 0x560441ad8420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:09.218 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.245+0000 7fb2eede1740 1 --2- 172.21.15.97:0/1989877507 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560441ac8980 0x560441ad7ee0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:09.218 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.245+0000 7fb2eede1740 1 -- 172.21.15.97:0/1989877507 >> 172.21.15.97:0/1989877507 conn(0x560441ac3ef0 msgr2=0x560441ad22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:09.218 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.245+0000 7fb2cb400640 0 client.16050 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T10:58:09.218 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.246+0000 7fb2eede1740 1 -- 172.21.15.97:0/1989877507 shutdown_connections 2024-03-27T10:58:09.219 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.246+0000 7fb2eede1740 1 -- 172.21.15.97:0/1989877507 wait complete. 2024-03-27T10:58:09.219 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.246+0000 7fb2eede1740 20 client.16050 trim_cache size 0 max 16384 2024-03-27T10:58:09.219 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-27T10:58:09.219 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-27T10:58:09.219 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-27T10:58:09.219 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-27T10:58:09.220 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.247+0000 7fb2ec200640 1 -- 172.21.15.97:0/1066883607 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fb2cc0a3750 msgr2=0x7fb2cc0a5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:09.220 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.247+0000 7fb2ec200640 1 --2- 172.21.15.97:0/1066883607 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fb2cc0a3750 0x7fb2cc0a5c10 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x560441ac9770 tx=0x7fb2d8005fd0 comp rx=0 tx=0).stop 2024-03-27T10:58:09.220 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.247+0000 7fb2ec200640 1 -- 172.21.15.97:0/1066883607 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2e41026b0 msgr2=0x7fb2e419e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:09.220 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.247+0000 7fb2ec200640 1 --2- 172.21.15.97:0/1066883607 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2e41026b0 0x7fb2e419e340 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb2d8000c00 tx=0x7fb2d80042f0 comp rx=0 tx=0).stop 2024-03-27T10:58:09.220 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.247+0000 7fb2ec200640 1 -- 172.21.15.97:0/1066883607 shutdown_connections 2024-03-27T10:58:09.220 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.247+0000 7fb2ec200640 1 --2- 172.21.15.97:0/1066883607 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2e4107910 0x7fb2e41a58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:09.220 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.247+0000 7fb2ec200640 1 --2- 172.21.15.97:0/1066883607 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2e4104fe0 0x7fb2e419e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:09.220 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.247+0000 7fb2ec200640 1 --2- 172.21.15.97:0/1066883607 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fb2cc0a3750 0x7fb2cc0a5c10 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:09.220 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.247+0000 7fb2ec200640 1 --2- 172.21.15.97:0/1066883607 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2e41026b0 0x7fb2e419e340 secure :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb2d8000c00 tx=0x7fb2d80042f0 comp rx=0 tx=0).stop 2024-03-27T10:58:09.221 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.247+0000 7fb2ec200640 1 -- 172.21.15.97:0/1066883607 >> 172.21.15.97:0/1066883607 conn(0x7fb2e40fbfe0 msgr2=0x7fb2e4103030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:09.221 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.247+0000 7fb2ec200640 1 -- 172.21.15.97:0/1066883607 shutdown_connections 2024-03-27T10:58:09.221 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:09.247+0000 7fb2ec200640 1 -- 172.21.15.97:0/1066883607 wait complete. 2024-03-27T10:58:09.516 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:58:09 smithi114 ceph-mon[35338]: pgmap v486: 97 pgs: 97 active+clean; 61 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 5.0 MiB/s wr, 12 op/s 2024-03-27T10:58:09.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:58:09 smithi097 ceph-mon[31859]: pgmap v486: 97 pgs: 97 active+clean; 61 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 5.0 MiB/s wr, 12 op/s 2024-03-27T10:58:09.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:58:09 smithi177 ceph-mon[35294]: pgmap v486: 97 pgs: 97 active+clean; 61 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 5.0 MiB/s wr, 12 op/s 2024-03-27T10:58:11.516 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:58:11 smithi114 ceph-mon[35338]: pgmap v487: 97 pgs: 97 active+clean; 264 MiB data, 1002 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 MiB/s wr, 29 op/s 2024-03-27T10:58:11.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:58:11 smithi097 ceph-mon[31859]: pgmap v487: 97 pgs: 97 active+clean; 264 MiB data, 1002 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 MiB/s wr, 29 op/s 2024-03-27T10:58:11.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:58:11 smithi177 ceph-mon[35294]: pgmap v487: 97 pgs: 97 active+clean; 264 MiB data, 1002 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 MiB/s wr, 29 op/s 2024-03-27T10:58:13.516 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:58:13 smithi114 ceph-mon[35338]: pgmap v488: 97 pgs: 97 active+clean; 514 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 43 MiB/s wr, 38 op/s 2024-03-27T10:58:13.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:58:13 smithi097 ceph-mon[31859]: pgmap v488: 97 pgs: 97 active+clean; 514 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 43 MiB/s wr, 38 op/s 2024-03-27T10:58:13.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:58:13 smithi177 ceph-mon[35294]: pgmap v488: 97 pgs: 97 active+clean; 514 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 43 MiB/s wr, 38 op/s 2024-03-27T10:58:15.516 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:58:15 smithi114 ceph-mon[35338]: pgmap v489: 97 pgs: 97 active+clean; 779 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 65 MiB/s wr, 57 op/s 2024-03-27T10:58:15.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:58:15 smithi097 ceph-mon[31859]: pgmap v489: 97 pgs: 97 active+clean; 779 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 65 MiB/s wr, 57 op/s 2024-03-27T10:58:15.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:58:15 smithi177 ceph-mon[35294]: pgmap v489: 97 pgs: 97 active+clean; 779 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 65 MiB/s wr, 57 op/s 2024-03-27T10:58:16.516 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:58:16 smithi114 ceph-mon[35338]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T10:58:16.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:58:16 smithi097 ceph-mon[31859]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T10:58:16.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:58:16 smithi177 ceph-mon[35294]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T10:58:17.516 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:58:17 smithi114 ceph-mon[35338]: pgmap v490: 97 pgs: 97 active+clean; 926 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 77 MiB/s wr, 62 op/s 2024-03-27T10:58:17.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:58:17 smithi097 ceph-mon[31859]: pgmap v490: 97 pgs: 97 active+clean; 926 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 77 MiB/s wr, 62 op/s 2024-03-27T10:58:17.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:58:17 smithi177 ceph-mon[35294]: pgmap v490: 97 pgs: 97 active+clean; 926 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 77 MiB/s wr, 62 op/s 2024-03-27T10:58:18.088 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-27T10:58:18.089 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-27T10:58:18.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.379+0000 7f939f800640 1 -- 172.21.15.97:0/3805541418 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9398101b60 msgr2=0x7f9398101f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:18.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.379+0000 7f939f800640 1 --2- 172.21.15.97:0/3805541418 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9398101b60 0x7f9398101f80 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f938c009900 tx=0x7f938c02bc30 comp rx=0 tx=0).stop 2024-03-27T10:58:18.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.379+0000 7f939f800640 1 -- 172.21.15.97:0/3805541418 shutdown_connections 2024-03-27T10:58:18.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.379+0000 7f939f800640 1 --2- 172.21.15.97:0/3805541418 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9398103710 0x7f9398109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:18.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.379+0000 7f939f800640 1 --2- 172.21.15.97:0/3805541418 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9398102d50 0x7f93981031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:18.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.379+0000 7f939f800640 1 --2- 172.21.15.97:0/3805541418 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9398101b60 0x7f9398101f80 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:18.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.379+0000 7f939f800640 1 -- 172.21.15.97:0/3805541418 >> 172.21.15.97:0/3805541418 conn(0x7f93980fd0f0 msgr2=0x7f93980ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:18.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.380+0000 7f939f800640 1 -- 172.21.15.97:0/3805541418 shutdown_connections 2024-03-27T10:58:18.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.380+0000 7f939f800640 1 -- 172.21.15.97:0/3805541418 wait complete. 2024-03-27T10:58:18.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.381+0000 7f939f800640 1 Processor -- start 2024-03-27T10:58:18.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.381+0000 7f939f800640 1 -- start start 2024-03-27T10:58:18.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.381+0000 7f939f800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9398101b60 0x7f9398199fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:18.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.381+0000 7f939f800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9398102d50 0x7f939819a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:18.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.381+0000 7f939f800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9398103710 0x7f93981a1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:18.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.382+0000 7f939f800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f939810f910 con 0x7f9398101b60 2024-03-27T10:58:18.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.382+0000 7f939f800640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f939810f790 con 0x7f9398102d50 2024-03-27T10:58:18.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.382+0000 7f939f800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f939810fa90 con 0x7f9398103710 2024-03-27T10:58:18.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.382+0000 7f939ce00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9398101b60 0x7f9398199fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:18.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.382+0000 7f939ce00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9398101b60 0x7f9398199fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46880/0 (socket says 172.21.15.97:46880) 2024-03-27T10:58:18.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.382+0000 7f939ce00640 1 -- 172.21.15.97:0/4215027385 learned_addr learned my addr 172.21.15.97:0/4215027385 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:58:18.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.382+0000 7f9397e00640 1 --2- 172.21.15.97:0/4215027385 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9398102d50 0x7f939819a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:18.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.382+0000 7f939d800640 1 --2- 172.21.15.97:0/4215027385 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9398103710 0x7f93981a1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:18.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.383+0000 7f939ce00640 1 -- 172.21.15.97:0/4215027385 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9398103710 msgr2=0x7f93981a1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:18.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.383+0000 7f939ce00640 1 --2- 172.21.15.97:0/4215027385 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9398103710 0x7f93981a1590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:18.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.383+0000 7f939ce00640 1 -- 172.21.15.97:0/4215027385 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9398102d50 msgr2=0x7f939819a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:18.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.383+0000 7f939ce00640 1 --2- 172.21.15.97:0/4215027385 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9398102d50 0x7f939819a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:18.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.383+0000 7f939ce00640 1 -- 172.21.15.97:0/4215027385 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93981a1c00 con 0x7f9398101b60 2024-03-27T10:58:18.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.383+0000 7f9397e00640 1 --2- 172.21.15.97:0/4215027385 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9398102d50 0x7f939819a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T10:58:18.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.383+0000 7f939ce00640 1 --2- 172.21.15.97:0/4215027385 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9398101b60 0x7f9398199fe0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f938c000c00 tx=0x7f938c0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:18.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.383+0000 7f9395600640 1 -- 172.21.15.97:0/4215027385 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f938c043020 con 0x7f9398101b60 2024-03-27T10:58:18.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.384+0000 7f939f800640 1 -- 172.21.15.97:0/4215027385 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93981a1e90 con 0x7f9398101b60 2024-03-27T10:58:18.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.384+0000 7f9395600640 1 -- 172.21.15.97:0/4215027385 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f938c0044c0 con 0x7f9398101b60 2024-03-27T10:58:18.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.384+0000 7f9395600640 1 -- 172.21.15.97:0/4215027385 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f938c003c60 con 0x7f9398101b60 2024-03-27T10:58:18.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.384+0000 7f939f800640 1 -- 172.21.15.97:0/4215027385 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93981a2350 con 0x7f9398101b60 2024-03-27T10:58:18.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.389+0000 7f9395600640 1 -- 172.21.15.97:0/4215027385 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f938c046020 con 0x7f9398101b60 2024-03-27T10:58:18.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.390+0000 7f93a258c740 1 Processor -- start 2024-03-27T10:58:18.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.391+0000 7f93a258c740 1 -- start start 2024-03-27T10:58:18.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.391+0000 7f93a258c740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559a26280cc0 0x559a262810e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:18.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.391+0000 7f93a258c740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559a262816b0 0x559a262861b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:18.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.391+0000 7f93a258c740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a26286820 0x559a26288cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:18.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.391+0000 7f93a258c740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x559a261de6b0 con 0x559a26286820 2024-03-27T10:58:18.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.391+0000 7f93a258c740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x559a26186750 con 0x559a26280cc0 2024-03-27T10:58:18.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.391+0000 7f93a258c740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x559a261da660 con 0x559a262816b0 2024-03-27T10:58:18.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.391+0000 7f9397e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a26286820 0x559a26288cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:18.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.391+0000 7f9397e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a26286820 0x559a26288cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46882/0 (socket says 172.21.15.97:46882) 2024-03-27T10:58:18.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.391+0000 7f9397e00640 1 -- 172.21.15.97:0/750274601 learned_addr learned my addr 172.21.15.97:0/750274601 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:58:18.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.391+0000 7f939d800640 1 --2- 172.21.15.97:0/750274601 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559a26280cc0 0x559a262810e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:18.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.392+0000 7f939ce00640 1 --2- 172.21.15.97:0/750274601 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559a262816b0 0x559a262861b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:18.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.392+0000 7f939d800640 1 -- 172.21.15.97:0/750274601 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559a262816b0 msgr2=0x559a262861b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:18.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.392+0000 7f9395600640 1 --2- 172.21.15.97:0/4215027385 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f93800a3690 0x7f93800a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:18.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.392+0000 7f939d800640 1 --2- 172.21.15.97:0/750274601 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559a262816b0 0x559a262861b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:18.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.392+0000 7f939d800640 1 -- 172.21.15.97:0/750274601 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a26286820 msgr2=0x559a26288cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:18.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.392+0000 7f939d800640 1 --2- 172.21.15.97:0/750274601 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a26286820 0x559a26288cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:18.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.392+0000 7f939d800640 1 -- 172.21.15.97:0/750274601 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a25fbc7d0 con 0x559a26280cc0 2024-03-27T10:58:18.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.392+0000 7f9395600640 1 -- 172.21.15.97:0/4215027385 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f938c0f2080 con 0x7f9398101b60 2024-03-27T10:58:18.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.392+0000 7f9397e00640 1 --2- 172.21.15.97:0/750274601 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a26286820 0x559a26288cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T10:58:18.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.393+0000 7f939d800640 1 --2- 172.21.15.97:0/4215027385 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f93800a3690 0x7f93800a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:18.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.393+0000 7f939d800640 1 --2- 172.21.15.97:0/750274601 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559a26280cc0 0x559a262810e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9390009e10 tx=0x7f939000f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45384be6d11e31ba server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:18.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.393+0000 7f938a000640 1 -- 172.21.15.97:0/750274601 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f939001c980 con 0x559a26280cc0 2024-03-27T10:58:18.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.393+0000 7f938a000640 1 -- 172.21.15.97:0/750274601 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f939001cba0 con 0x559a26280cc0 2024-03-27T10:58:18.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.393+0000 7f938a000640 1 -- 172.21.15.97:0/750274601 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f939001b6a0 con 0x559a26280cc0 2024-03-27T10:58:18.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.394+0000 7f93a258c740 1 -- 172.21.15.97:0/750274601 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559a26280cc0 msgr2=0x559a262810e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:18.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.394+0000 7f939d800640 1 --2- 172.21.15.97:0/4215027385 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f93800a3690 0x7f93800a5b50 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f939001b000 tx=0x7f939001bdd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:18.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.394+0000 7f93a258c740 1 --2- 172.21.15.97:0/750274601 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559a26280cc0 0x559a262810e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9390009e10 tx=0x7f939000f730 comp rx=0 tx=0).stop 2024-03-27T10:58:18.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.394+0000 7f93a258c740 1 -- 172.21.15.97:0/750274601 shutdown_connections 2024-03-27T10:58:18.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.394+0000 7f93a258c740 1 --2- 172.21.15.97:0/750274601 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a26286820 0x559a26288cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:18.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.394+0000 7f93a258c740 1 --2- 172.21.15.97:0/750274601 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559a262816b0 0x559a262861b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:18.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.394+0000 7f93a258c740 1 --2- 172.21.15.97:0/750274601 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559a26280cc0 0x559a262810e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:18.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.394+0000 7f93a258c740 1 -- 172.21.15.97:0/750274601 >> 172.21.15.97:0/750274601 conn(0x559a2627c230 msgr2=0x559a2627c630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:18.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.394+0000 7f93a258c740 1 -- 172.21.15.97:0/750274601 shutdown_connections 2024-03-27T10:58:18.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.395+0000 7f93a258c740 1 -- 172.21.15.97:0/750274601 wait complete. 2024-03-27T10:58:18.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.395+0000 7f93a258c740 1 -- start start 2024-03-27T10:58:18.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.395+0000 7f93a258c740 1 Processor -- start 2024-03-27T10:58:18.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.396+0000 7f93a258c740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559a26280cc0 0x559a2628b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:18.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.396+0000 7f93a258c740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559a262816b0 0x559a2628b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:18.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.396+0000 7f93a258c740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a26286820 0x559a26292730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:18.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.396+0000 7f93a258c740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x559a262897d0 con 0x559a26286820 2024-03-27T10:58:18.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.396+0000 7f93a258c740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x559a26199a80 con 0x559a262816b0 2024-03-27T10:58:18.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.396+0000 7f93a258c740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x559a26289950 con 0x559a26280cc0 2024-03-27T10:58:18.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.396+0000 7f93a258c740 10 client.0 _ll_register_callbacks cb 0x559a26199470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:58:18.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.396+0000 7f9397e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559a262816b0 0x559a2628b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:18.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.396+0000 7f9397e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559a262816b0 0x559a2628b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:34524/0 (socket says 172.21.15.97:34524) 2024-03-27T10:58:18.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.396+0000 7f9397e00640 1 -- 172.21.15.97:0/621213698 learned_addr learned my addr 172.21.15.97:0/621213698 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:58:18.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.396+0000 7f939d800640 1 --2- 172.21.15.97:0/621213698 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a26286820 0x559a26292730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:18.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.397+0000 7f939ce00640 1 --2- 172.21.15.97:0/621213698 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559a26280cc0 0x559a2628b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:18.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.397+0000 7f939d800640 1 -- 172.21.15.97:0/621213698 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559a26280cc0 msgr2=0x559a2628b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:18.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.397+0000 7f939d800640 1 --2- 172.21.15.97:0/621213698 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559a26280cc0 0x559a2628b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:18.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.397+0000 7f939d800640 1 -- 172.21.15.97:0/621213698 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559a262816b0 msgr2=0x559a2628b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:18.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.397+0000 7f939d800640 1 --2- 172.21.15.97:0/621213698 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559a262816b0 0x559a2628b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:18.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.397+0000 7f939d800640 1 -- 172.21.15.97:0/621213698 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a25ef23f0 con 0x559a26286820 2024-03-27T10:58:18.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.397+0000 7f939d800640 1 --2- 172.21.15.97:0/621213698 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a26286820 0x559a26292730 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f9390009e10 tx=0x7f9390016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=493a503001e50f96 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:18.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.397+0000 7f937ea00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T10:58:18.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.398+0000 7f937ea00640 1 -- 172.21.15.97:0/621213698 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9390035060 con 0x559a26286820 2024-03-27T10:58:18.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.398+0000 7f937ea00640 1 -- 172.21.15.97:0/621213698 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f939001c990 con 0x559a26286820 2024-03-27T10:58:18.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.398+0000 7f93a258c740 1 -- 172.21.15.97:0/621213698 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f939001cba0 con 0x559a26286820 2024-03-27T10:58:18.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.398+0000 7f937ea00640 1 -- 172.21.15.97:0/621213698 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f939001b3e0 con 0x559a26286820 2024-03-27T10:58:18.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.398+0000 7f93a258c740 10 client.16062 fetch_fsmap learned FSMap version 29 2024-03-27T10:58:18.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.398+0000 7f93a258c740 1 -- 172.21.15.97:0/621213698 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559a25eb20f0 con 0x559a26286820 2024-03-27T10:58:18.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.398+0000 7f937ea00640 1 -- 172.21.15.97:0/621213698 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f939001cba0 con 0x559a26286820 2024-03-27T10:58:18.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.399+0000 7f937ea00640 1 -- 172.21.15.97:0/621213698 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f9390010970 con 0x559a26286820 2024-03-27T10:58:18.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.399+0000 7f93a258c740 10 client.16062 fetch_fsmap finished waiting for FSMap version 29 2024-03-27T10:58:18.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.399+0000 7f93a258c740 10 client.16062 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-27T10:58:18.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.399+0000 7f93a258c740 20 client.16062 populate_metadata read hostname 'smithi097' 2024-03-27T10:58:18.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.399+0000 7f93a258c740 1 --2- 172.21.15.97:0/621213698 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x559a26296e00 0x559a262991f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:18.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.399+0000 7f93a258c740 4 client.16062 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T10:58:18.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.399+0000 7f93a258c740 1 -- 172.21.15.97:0/621213698 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a261ebbf0 con 0x559a26296e00 2024-03-27T10:58:18.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.399+0000 7f939ce00640 1 --2- 172.21.15.97:0/621213698 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x559a26296e00 0x559a262991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:18.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.400+0000 7f939ce00640 1 --2- 172.21.15.97:0/621213698 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x559a26296e00 0x559a262991f0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b712212732400b5 server_cookie=7b584a95d5b21820 in_seq=0 out_seq=0 2024-03-27T10:58:18.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.400+0000 7f937ea00640 10 client.16062 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T10:58:18.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.401+0000 7f937ea00640 1 -- 172.21.15.97:0/621213698 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x559a261ebbf0 con 0x559a26296e00 2024-03-27T10:58:18.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.401+0000 7f937ea00640 10 client.16062 handle_command_reply: tid=0 2024-03-27T10:58:18.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.401+0000 7f93a258c740 1 client.16062 shutdown 2024-03-27T10:58:18.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.402+0000 7f93a258c740 1 -- 172.21.15.97:0/621213698 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a26286820 msgr2=0x559a26292730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:18.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.402+0000 7f93a258c740 1 --2- 172.21.15.97:0/621213698 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a26286820 0x559a26292730 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f9390009e10 tx=0x7f9390016000 comp rx=0 tx=0).stop 2024-03-27T10:58:18.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.402+0000 7f93a258c740 1 -- 172.21.15.97:0/621213698 shutdown_connections 2024-03-27T10:58:18.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.402+0000 7f93a258c740 1 --2- 172.21.15.97:0/621213698 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x559a26296e00 0x559a262991f0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:18.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.402+0000 7f93a258c740 1 --2- 172.21.15.97:0/621213698 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a26286820 0x559a26292730 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:18.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.402+0000 7f93a258c740 1 --2- 172.21.15.97:0/621213698 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559a262816b0 0x559a2628b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:18.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.402+0000 7f93a258c740 1 --2- 172.21.15.97:0/621213698 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559a26280cc0 0x559a2628b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:18.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.402+0000 7f937ea00640 0 client.16062 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T10:58:18.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.402+0000 7f93a258c740 1 -- 172.21.15.97:0/621213698 >> 172.21.15.97:0/621213698 conn(0x559a2627c230 msgr2=0x559a26272c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:18.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.402+0000 7f93a258c740 1 -- 172.21.15.97:0/621213698 shutdown_connections 2024-03-27T10:58:18.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.402+0000 7f93a258c740 1 -- 172.21.15.97:0/621213698 wait complete. 2024-03-27T10:58:18.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.402+0000 7f93a258c740 20 client.16062 trim_cache size 0 max 16384 2024-03-27T10:58:18.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.427+0000 7f93a258c740 1 Processor -- start 2024-03-27T10:58:18.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.427+0000 7f93a258c740 1 -- start start 2024-03-27T10:58:18.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.428+0000 7f93a258c740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559a26280cc0 0x559a26272aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:18.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.428+0000 7f93a258c740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a262816b0 0x559a262926f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:18.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.428+0000 7f93a258c740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559a26286820 0x559a2629dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:18.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.428+0000 7f93a258c740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f93981a6e90 con 0x559a262816b0 2024-03-27T10:58:18.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.428+0000 7f93a258c740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f93981a8a00 con 0x559a26286820 2024-03-27T10:58:18.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.428+0000 7f93a258c740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f93981a7560 con 0x559a26280cc0 2024-03-27T10:58:18.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.428+0000 7f9397e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a262816b0 0x559a262926f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:18.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.428+0000 7f939ce00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559a26280cc0 0x559a26272aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:18.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.428+0000 7f9397e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a262816b0 0x559a262926f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46912/0 (socket says 172.21.15.97:46912) 2024-03-27T10:58:18.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.428+0000 7f9397e00640 1 -- 172.21.15.97:0/1875193139 learned_addr learned my addr 172.21.15.97:0/1875193139 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:58:18.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.429+0000 7f939d800640 1 --2- 172.21.15.97:0/1875193139 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559a26286820 0x559a2629dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:18.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.429+0000 7f939ce00640 1 -- 172.21.15.97:0/1875193139 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559a26286820 msgr2=0x559a2629dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:18.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.429+0000 7f939ce00640 1 --2- 172.21.15.97:0/1875193139 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559a26286820 0x559a2629dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:18.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.429+0000 7f939ce00640 1 -- 172.21.15.97:0/1875193139 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a262816b0 msgr2=0x559a262926f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:18.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.430+0000 7f939ce00640 1 --2- 172.21.15.97:0/1875193139 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a262816b0 0x559a262926f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:18.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.430+0000 7f939ce00640 1 -- 172.21.15.97:0/1875193139 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a26299850 con 0x559a26280cc0 2024-03-27T10:58:18.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.430+0000 7f9397e00640 1 --2- 172.21.15.97:0/1875193139 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a262816b0 0x559a262926f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T10:58:18.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.430+0000 7f939ce00640 1 --2- 172.21.15.97:0/1875193139 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559a26280cc0 0x559a26272aa0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x559a26282120 tx=0x7f938c0efa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a61bd87e43e207f server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:18.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.431+0000 7f9389600640 1 -- 172.21.15.97:0/1875193139 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f938c0be040 con 0x559a26280cc0 2024-03-27T10:58:18.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.431+0000 7f9389600640 1 -- 172.21.15.97:0/1875193139 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f938c0035f0 con 0x559a26280cc0 2024-03-27T10:58:18.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.431+0000 7f9389600640 1 -- 172.21.15.97:0/1875193139 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f938c0bde60 con 0x559a26280cc0 2024-03-27T10:58:18.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.432+0000 7f93a258c740 1 -- 172.21.15.97:0/1875193139 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559a26280cc0 msgr2=0x559a26272aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:18.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.432+0000 7f93a258c740 1 --2- 172.21.15.97:0/1875193139 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559a26280cc0 0x559a26272aa0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x559a26282120 tx=0x7f938c0efa00 comp rx=0 tx=0).stop 2024-03-27T10:58:18.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.432+0000 7f93a258c740 1 -- 172.21.15.97:0/1875193139 shutdown_connections 2024-03-27T10:58:18.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.432+0000 7f93a258c740 1 --2- 172.21.15.97:0/1875193139 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559a26286820 0x559a2629dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:18.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.432+0000 7f93a258c740 1 --2- 172.21.15.97:0/1875193139 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a262816b0 0x559a262926f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:18.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.432+0000 7f93a258c740 1 --2- 172.21.15.97:0/1875193139 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559a26280cc0 0x559a26272aa0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:18.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.432+0000 7f93a258c740 1 -- 172.21.15.97:0/1875193139 >> 172.21.15.97:0/1875193139 conn(0x559a2627c230 msgr2=0x559a26272390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:18.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.432+0000 7f93a258c740 1 -- 172.21.15.97:0/1875193139 shutdown_connections 2024-03-27T10:58:18.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.432+0000 7f93a258c740 1 -- 172.21.15.97:0/1875193139 wait complete. 2024-03-27T10:58:18.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.432+0000 7f93a258c740 1 -- start start 2024-03-27T10:58:18.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.433+0000 7f93a258c740 1 Processor -- start 2024-03-27T10:58:18.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.433+0000 7f93a258c740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559a26280cc0 0x559a2628f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:18.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.433+0000 7f93a258c740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a262816b0 0x559a2628f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:18.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.433+0000 7f93a258c740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559a26286820 0x559a262763b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:18.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.433+0000 7f93a258c740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f93981a8300 con 0x559a262816b0 2024-03-27T10:58:18.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.433+0000 7f93a258c740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f93981a7c30 con 0x559a26286820 2024-03-27T10:58:18.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.433+0000 7f93a258c740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f93981a4e70 con 0x559a26280cc0 2024-03-27T10:58:18.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.433+0000 7f93a258c740 10 client.0 _ll_register_callbacks cb 0x7f9390009d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:58:18.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.434+0000 7f939d800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559a26286820 0x559a262763b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:18.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.434+0000 7f939d800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559a26286820 0x559a262763b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:34550/0 (socket says 172.21.15.97:34550) 2024-03-27T10:58:18.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.434+0000 7f939d800640 1 -- 172.21.15.97:0/3565579166 learned_addr learned my addr 172.21.15.97:0/3565579166 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:58:18.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.434+0000 7f939ce00640 1 --2- 172.21.15.97:0/3565579166 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559a26280cc0 0x559a2628f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:18.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.434+0000 7f9397e00640 1 --2- 172.21.15.97:0/3565579166 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a262816b0 0x559a2628f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:18.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.435+0000 7f939d800640 1 -- 172.21.15.97:0/3565579166 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559a26280cc0 msgr2=0x559a2628f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:18.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.435+0000 7f939d800640 1 --2- 172.21.15.97:0/3565579166 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559a26280cc0 0x559a2628f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:18.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.435+0000 7f939d800640 1 -- 172.21.15.97:0/3565579166 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a262816b0 msgr2=0x559a2628f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:18.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.435+0000 7f939d800640 1 --2- 172.21.15.97:0/3565579166 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a262816b0 0x559a2628f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:18.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.435+0000 7f939d800640 1 -- 172.21.15.97:0/3565579166 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a262768f0 con 0x559a26286820 2024-03-27T10:58:18.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.435+0000 7f939ce00640 1 --2- 172.21.15.97:0/3565579166 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559a26280cc0 0x559a2628f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:58:18.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.435+0000 7f939d800640 1 --2- 172.21.15.97:0/3565579166 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559a26286820 0x559a262763b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f939000fc30 tx=0x7f9390010c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc3078ae33c802c3 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:18.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.435+0000 7f937ea00640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-03-27T10:58:18.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.435+0000 7f937ea00640 1 -- 172.21.15.97:0/3565579166 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9390040070 con 0x559a26286820 2024-03-27T10:58:18.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.436+0000 7f937ea00640 1 -- 172.21.15.97:0/3565579166 <== mon.2 v2:172.21.15.114:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f939001b580 con 0x559a26286820 2024-03-27T10:58:18.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.436+0000 7f93a258c740 1 -- 172.21.15.97:0/3565579166 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559a261e5f40 con 0x559a26286820 2024-03-27T10:58:18.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.436+0000 7f937ea00640 1 -- 172.21.15.97:0/3565579166 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f939004a050 con 0x559a26286820 2024-03-27T10:58:18.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.436+0000 7f93a258c740 10 client.35720 fetch_fsmap learned FSMap version 29 2024-03-27T10:58:18.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.437+0000 7f93a258c740 1 -- 172.21.15.97:0/3565579166 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559a26277bc0 con 0x559a26286820 2024-03-27T10:58:18.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.437+0000 7f937ea00640 1 -- 172.21.15.97:0/3565579166 <== mon.2 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x559a261e5f40 con 0x559a26286820 2024-03-27T10:58:18.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.437+0000 7f937ea00640 1 -- 172.21.15.97:0/3565579166 <== mon.2 v2:172.21.15.114:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f9390005150 con 0x559a26286820 2024-03-27T10:58:18.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.437+0000 7f93a258c740 10 client.35720 fetch_fsmap finished waiting for FSMap version 29 2024-03-27T10:58:18.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.437+0000 7f93a258c740 10 client.35720 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-27T10:58:18.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.437+0000 7f93a258c740 20 client.35720 populate_metadata read hostname 'smithi097' 2024-03-27T10:58:18.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.437+0000 7f93a258c740 1 --2- 172.21.15.97:0/3565579166 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x559a26278620 0x559a2627aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:18.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.437+0000 7f93a258c740 4 client.35720 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-27T10:58:18.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.437+0000 7f93a258c740 1 -- 172.21.15.97:0/3565579166 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559a26163400 con 0x559a26278620 2024-03-27T10:58:18.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.438+0000 7f939ce00640 1 --2- 172.21.15.97:0/3565579166 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x559a26278620 0x559a2627aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:18.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.439+0000 7f937ea00640 10 client.35720 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T10:58:18.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.439+0000 7f939ce00640 1 --2- 172.21.15.97:0/3565579166 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x559a26278620 0x559a2627aa40 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b13dc9d5da279cdb server_cookie=402c67ffc60f16c7 in_seq=0 out_seq=0 2024-03-27T10:58:18.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.440+0000 7f937ea00640 1 -- 172.21.15.97:0/3565579166 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x559a26163400 con 0x559a26278620 2024-03-27T10:58:18.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.440+0000 7f937ea00640 10 client.35720 handle_command_reply: tid=0 2024-03-27T10:58:18.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.440+0000 7f93a258c740 1 client.35720 shutdown 2024-03-27T10:58:18.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.440+0000 7f93a258c740 1 -- 172.21.15.97:0/3565579166 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559a26286820 msgr2=0x559a262763b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:18.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.440+0000 7f93a258c740 1 --2- 172.21.15.97:0/3565579166 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559a26286820 0x559a262763b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f939000fc30 tx=0x7f9390010c20 comp rx=0 tx=0).stop 2024-03-27T10:58:18.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.440+0000 7f93a258c740 1 -- 172.21.15.97:0/3565579166 shutdown_connections 2024-03-27T10:58:18.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.440+0000 7f93a258c740 1 --2- 172.21.15.97:0/3565579166 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x559a26278620 0x559a2627aa40 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:18.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.440+0000 7f93a258c740 1 --2- 172.21.15.97:0/3565579166 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559a26286820 0x559a262763b0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:18.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.440+0000 7f93a258c740 1 --2- 172.21.15.97:0/3565579166 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a262816b0 0x559a2628f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:18.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.440+0000 7f93a258c740 1 --2- 172.21.15.97:0/3565579166 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559a26280cc0 0x559a2628f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:18.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.440+0000 7f93a258c740 1 -- 172.21.15.97:0/3565579166 >> 172.21.15.97:0/3565579166 conn(0x559a2627c230 msgr2=0x559a2627fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:18.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.440+0000 7f937ea00640 0 client.35720 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T10:58:18.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.441+0000 7f93a258c740 1 -- 172.21.15.97:0/3565579166 shutdown_connections 2024-03-27T10:58:18.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.441+0000 7f93a258c740 1 -- 172.21.15.97:0/3565579166 wait complete. 2024-03-27T10:58:18.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.441+0000 7f93a258c740 20 client.35720 trim_cache size 0 max 16384 2024-03-27T10:58:18.414 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-03-27T10:58:18.414 INFO:teuthology.orchestra.run.smithi097.stdout: "status": "no active scrubs running", 2024-03-27T10:58:18.414 INFO:teuthology.orchestra.run.smithi097.stdout: "scrubs": {} 2024-03-27T10:58:18.414 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-03-27T10:58:18.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.442+0000 7f939f800640 1 -- 172.21.15.97:0/4215027385 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f93800a3690 msgr2=0x7f93800a5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:18.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.442+0000 7f939f800640 1 --2- 172.21.15.97:0/4215027385 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f93800a3690 0x7f93800a5b50 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f939001b000 tx=0x7f939001bdd0 comp rx=0 tx=0).stop 2024-03-27T10:58:18.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.442+0000 7f939f800640 1 -- 172.21.15.97:0/4215027385 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9398101b60 msgr2=0x7f9398199fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:18.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.442+0000 7f939f800640 1 --2- 172.21.15.97:0/4215027385 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9398101b60 0x7f9398199fe0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f938c000c00 tx=0x7f938c0040c0 comp rx=0 tx=0).stop 2024-03-27T10:58:18.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.443+0000 7f939f800640 1 -- 172.21.15.97:0/4215027385 shutdown_connections 2024-03-27T10:58:18.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.443+0000 7f939f800640 1 --2- 172.21.15.97:0/4215027385 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9398103710 0x7f93981a1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:18.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.443+0000 7f939f800640 1 --2- 172.21.15.97:0/4215027385 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f93800a3690 0x7f93800a5b50 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:18.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.443+0000 7f939f800640 1 --2- 172.21.15.97:0/4215027385 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9398102d50 0x7f939819a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:18.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.443+0000 7f939f800640 1 --2- 172.21.15.97:0/4215027385 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9398101b60 0x7f9398199fe0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:18.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.443+0000 7f939f800640 1 -- 172.21.15.97:0/4215027385 >> 172.21.15.97:0/4215027385 conn(0x7f93980fd0f0 msgr2=0x7f93981060c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:18.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.443+0000 7f939f800640 1 -- 172.21.15.97:0/4215027385 shutdown_connections 2024-03-27T10:58:18.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.443+0000 7f939f800640 1 -- 172.21.15.97:0/4215027385 wait complete. 2024-03-27T10:58:18.437 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-27T10:58:18.437 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-27T10:58:18.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.724+0000 7f9932200640 1 -- 172.21.15.97:0/3290437956 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f992c1091a0 msgr2=0x7f992c109620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:18.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.724+0000 7f9932200640 1 --2- 172.21.15.97:0/3290437956 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f992c1091a0 0x7f992c109620 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9918009860 tx=0x7f991802bb90 comp rx=0 tx=0).stop 2024-03-27T10:58:18.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.724+0000 7f9932200640 1 -- 172.21.15.97:0/3290437956 shutdown_connections 2024-03-27T10:58:18.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.725+0000 7f9932200640 1 --2- 172.21.15.97:0/3290437956 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f992c109b60 0x7f992c1103f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:18.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.725+0000 7f9932200640 1 --2- 172.21.15.97:0/3290437956 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f992c1091a0 0x7f992c109620 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:18.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.725+0000 7f9932200640 1 --2- 172.21.15.97:0/3290437956 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f992c107fb0 0x7f992c1083d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:18.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.725+0000 7f9932200640 1 -- 172.21.15.97:0/3290437956 >> 172.21.15.97:0/3290437956 conn(0x7f992c076f60 msgr2=0x7f992c079380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:18.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.725+0000 7f9932200640 1 -- 172.21.15.97:0/3290437956 shutdown_connections 2024-03-27T10:58:18.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.725+0000 7f9932200640 1 -- 172.21.15.97:0/3290437956 wait complete. 2024-03-27T10:58:18.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.726+0000 7f9932200640 1 Processor -- start 2024-03-27T10:58:18.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.726+0000 7f9932200640 1 -- start start 2024-03-27T10:58:18.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.728+0000 7f9932200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f992c107fb0 0x7f992c19e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:18.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.728+0000 7f9932200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f992c1091a0 0x7f992c19e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:18.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.728+0000 7f9932200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f992c109b60 0x7f992c1a56d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:18.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.728+0000 7f9932200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f992c113af0 con 0x7f992c107fb0 2024-03-27T10:58:18.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.728+0000 7f9932200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f992c113970 con 0x7f992c109b60 2024-03-27T10:58:18.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.728+0000 7f9932200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f992c113c70 con 0x7f992c1091a0 2024-03-27T10:58:18.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.729+0000 7f9931800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f992c109b60 0x7f992c1a56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:18.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.729+0000 7f9931800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f992c109b60 0x7f992c1a56d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:34562/0 (socket says 172.21.15.97:34562) 2024-03-27T10:58:18.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.729+0000 7f9931800640 1 -- 172.21.15.97:0/2794000730 learned_addr learned my addr 172.21.15.97:0/2794000730 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:58:18.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.729+0000 7f992be00640 1 --2- 172.21.15.97:0/2794000730 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f992c1091a0 0x7f992c19e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:18.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.729+0000 7f9931800640 1 -- 172.21.15.97:0/2794000730 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f992c1091a0 msgr2=0x7f992c19e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:18.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.729+0000 7f9931800640 1 --2- 172.21.15.97:0/2794000730 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f992c1091a0 0x7f992c19e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:18.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.729+0000 7f9931800640 1 -- 172.21.15.97:0/2794000730 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f992c107fb0 msgr2=0x7f992c19e120 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-27T10:58:18.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.730+0000 7f9930e00640 1 --2- 172.21.15.97:0/2794000730 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f992c107fb0 0x7f992c19e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:18.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.730+0000 7f9931800640 1 --2- 172.21.15.97:0/2794000730 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f992c107fb0 0x7f992c19e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:18.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.731+0000 7f9931800640 1 -- 172.21.15.97:0/2794000730 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f992c1a5d40 con 0x7f992c109b60 2024-03-27T10:58:18.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.731+0000 7f9930e00640 1 --2- 172.21.15.97:0/2794000730 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f992c107fb0 0x7f992c19e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:58:18.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.731+0000 7f9931800640 1 --2- 172.21.15.97:0/2794000730 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f992c109b60 0x7f992c1a56d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f991c00d750 tx=0x7f991c00dc10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:18.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.732+0000 7f9929600640 1 -- 172.21.15.97:0/2794000730 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f991c017070 con 0x7f992c109b60 2024-03-27T10:58:18.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.732+0000 7f9929600640 1 -- 172.21.15.97:0/2794000730 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f991c0054d0 con 0x7f992c109b60 2024-03-27T10:58:18.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.732+0000 7f9929600640 1 -- 172.21.15.97:0/2794000730 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f991c004c70 con 0x7f992c109b60 2024-03-27T10:58:18.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.732+0000 7f9932200640 1 -- 172.21.15.97:0/2794000730 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f992c1a5fd0 con 0x7f992c109b60 2024-03-27T10:58:18.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.732+0000 7f9932200640 1 -- 172.21.15.97:0/2794000730 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f992c1a64a0 con 0x7f992c109b60 2024-03-27T10:58:18.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.734+0000 7f9932200640 1 -- 172.21.15.97:0/2794000730 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f992c068860 con 0x7f992c109b60 2024-03-27T10:58:18.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.736+0000 7f9929600640 1 -- 172.21.15.97:0/2794000730 <== mon.2 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f991c004e10 con 0x7f992c109b60 2024-03-27T10:58:18.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.737+0000 7f9929600640 1 --2- 172.21.15.97:0/2794000730 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f99100a3710 0x7f99100a5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:18.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.738+0000 7f9929600640 1 -- 172.21.15.97:0/2794000730 <== mon.2 v2:172.21.15.114:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f991c0cfb70 con 0x7f992c109b60 2024-03-27T10:58:18.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.738+0000 7f9930e00640 1 --2- 172.21.15.97:0/2794000730 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f99100a3710 0x7f99100a5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:18.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.739+0000 7f9930e00640 1 --2- 172.21.15.97:0/2794000730 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f99100a3710 0x7f99100a5bd0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f992c109000 tx=0x7f99240095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:18.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:18.749+0000 7f9929600640 1 -- 172.21.15.97:0/2794000730 <== mon.2 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f991c09c770 con 0x7f992c109b60 2024-03-27T10:58:19.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.160+0000 7f9932200640 1 -- 172.21.15.97:0/2794000730 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f992c10f2d0 con 0x7f992c109b60 2024-03-27T10:58:19.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.162+0000 7f9929600640 1 -- 172.21.15.97:0/2794000730 <== mon.2 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11945 (secure 0 0 0) 0x7f991c09c070 con 0x7f992c109b60 2024-03-27T10:58:19.135 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T10:58:19.135 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1009423904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1009423904},{"type":"v1","addr":"172.21.15.97:6835","nonce":1009423904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3251173433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3251173433},{"type":"v1","addr":"172.21.15.97:6837","nonce":3251173433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-27T10:54:04.510178+0000","modified":"2024-03-27T10:58:14.689996+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34508,"mds_1":34478,"mds_2":34445,"mds_3":24454,"mds_4":24400},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14646":{"gid":14646,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6839/1057596821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1057596821},{"type":"v1","addr":"172.21.15.97:6839","nonce":1057596821}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14658":{"gid":14658,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6841/1672464856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6840","nonce":1672464856},{"type":"v1","addr":"172.21.15.97:6841","nonce":1672464856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.177:6833/3787981632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3787981632},{"type":"v1","addr":"172.21.15.177:6833","nonce":3787981632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.177:6835/3349112132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":3349112132},{"type":"v1","addr":"172.21.15.177:6835","nonce":3349112132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6837/3244432640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":3244432640},{"type":"v1","addr":"172.21.15.177:6837","nonce":3244432640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6839/3340454605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":3340454605},{"type":"v1","addr":"172.21.15.177:6839","nonce":3340454605}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34415":{"gid":34415,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6833/4052517902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":4052517902},{"type":"v1","addr":"172.21.15.114:6833","nonce":4052517902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34445":{"gid":34445,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.114:6835/3581918222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3581918222},{"type":"v1","addr":"172.21.15.114:6835","nonce":3581918222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34478":{"gid":34478,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.114:6837/616775588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":616775588},{"type":"v1","addr":"172.21.15.114:6837","nonce":616775588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34508":{"gid":34508,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.114:6839/3937193784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":3937193784},{"type":"v1","addr":"172.21.15.114:6839","nonce":3937193784}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34508,"qdb_cluster":[34508,34445,24400,34478,24454]},"id":1}]} 2024-03-27T10:58:19.135 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 29 2024-03-27T10:58:19.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.166+0000 7f9932200640 1 -- 172.21.15.97:0/2794000730 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f99100a3710 msgr2=0x7f99100a5bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:19.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.166+0000 7f9932200640 1 --2- 172.21.15.97:0/2794000730 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f99100a3710 0x7f99100a5bd0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f992c109000 tx=0x7f99240095c0 comp rx=0 tx=0).stop 2024-03-27T10:58:19.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.166+0000 7f9932200640 1 -- 172.21.15.97:0/2794000730 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f992c109b60 msgr2=0x7f992c1a56d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:19.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.166+0000 7f9932200640 1 --2- 172.21.15.97:0/2794000730 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f992c109b60 0x7f992c1a56d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f991c00d750 tx=0x7f991c00dc10 comp rx=0 tx=0).stop 2024-03-27T10:58:19.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.166+0000 7f9932200640 1 -- 172.21.15.97:0/2794000730 shutdown_connections 2024-03-27T10:58:19.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.166+0000 7f9932200640 1 --2- 172.21.15.97:0/2794000730 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f99100a3710 0x7f99100a5bd0 secure :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f992c109000 tx=0x7f99240095c0 comp rx=0 tx=0).stop 2024-03-27T10:58:19.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.166+0000 7f9932200640 1 --2- 172.21.15.97:0/2794000730 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f992c109b60 0x7f992c1a56d0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:19.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.166+0000 7f9932200640 1 --2- 172.21.15.97:0/2794000730 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f992c1091a0 0x7f992c19e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:19.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.166+0000 7f9932200640 1 --2- 172.21.15.97:0/2794000730 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f992c107fb0 0x7f992c19e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:19.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.166+0000 7f9932200640 1 -- 172.21.15.97:0/2794000730 >> 172.21.15.97:0/2794000730 conn(0x7f992c076f60 msgr2=0x7f992c10c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:19.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.167+0000 7f9932200640 1 -- 172.21.15.97:0/2794000730 shutdown_connections 2024-03-27T10:58:19.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.167+0000 7f9932200640 1 -- 172.21.15.97:0/2794000730 wait complete. 2024-03-27T10:58:19.166 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-27T10:58:19.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.461+0000 7f5982c00640 1 -- 172.21.15.97:0/933081582 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f597c102c10 msgr2=0x7f597c105080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:19.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.461+0000 7f5982c00640 1 --2- 172.21.15.97:0/933081582 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f597c102c10 0x7f597c105080 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f5968009900 tx=0x7f596802bc30 comp rx=0 tx=0).stop 2024-03-27T10:58:19.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.461+0000 7f5982c00640 1 -- 172.21.15.97:0/933081582 shutdown_connections 2024-03-27T10:58:19.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.461+0000 7f5982c00640 1 --2- 172.21.15.97:0/933081582 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f597c1055c0 0x7f597c105a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:19.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.461+0000 7f5982c00640 1 --2- 172.21.15.97:0/933081582 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f597c102c10 0x7f597c105080 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:19.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.461+0000 7f5982c00640 1 --2- 172.21.15.97:0/933081582 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f597c1002a0 0x7f597c1026d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:19.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.461+0000 7f5982c00640 1 -- 172.21.15.97:0/933081582 >> 172.21.15.97:0/933081582 conn(0x7f597c0fbec0 msgr2=0x7f597c0fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:19.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.461+0000 7f5982c00640 1 -- 172.21.15.97:0/933081582 shutdown_connections 2024-03-27T10:58:19.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.461+0000 7f5982c00640 1 -- 172.21.15.97:0/933081582 wait complete. 2024-03-27T10:58:19.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.462+0000 7f5982c00640 1 Processor -- start 2024-03-27T10:58:19.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.462+0000 7f5982c00640 1 -- start start 2024-03-27T10:58:19.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.463+0000 7f5982c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f597c1002a0 0x7f597c19e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:19.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.463+0000 7f5982c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f597c102c10 0x7f597c19e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:19.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.463+0000 7f5982c00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f597c1055c0 0x7f597c1a57f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:19.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.463+0000 7f5982c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f597c113bd0 con 0x7f597c102c10 2024-03-27T10:58:19.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.463+0000 7f5982c00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f597c113a50 con 0x7f597c1002a0 2024-03-27T10:58:19.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.463+0000 7f5982c00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f597c113d50 con 0x7f597c1055c0 2024-03-27T10:58:19.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.463+0000 7f5980e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f597c102c10 0x7f597c19e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:19.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.463+0000 7f5980e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f597c102c10 0x7f597c19e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53082/0 (socket says 172.21.15.97:53082) 2024-03-27T10:58:19.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.463+0000 7f5980e00640 1 -- 172.21.15.97:0/1410428548 learned_addr learned my addr 172.21.15.97:0/1410428548 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:58:19.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.463+0000 7f5981800640 1 --2- 172.21.15.97:0/1410428548 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f597c1002a0 0x7f597c19e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:19.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.463+0000 7f5982200640 1 --2- 172.21.15.97:0/1410428548 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f597c1055c0 0x7f597c1a57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:19.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.464+0000 7f5980e00640 1 -- 172.21.15.97:0/1410428548 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f597c1055c0 msgr2=0x7f597c1a57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:19.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.464+0000 7f5980e00640 1 --2- 172.21.15.97:0/1410428548 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f597c1055c0 0x7f597c1a57f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:19.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.464+0000 7f5980e00640 1 -- 172.21.15.97:0/1410428548 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f597c1002a0 msgr2=0x7f597c19e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:19.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.464+0000 7f5980e00640 1 --2- 172.21.15.97:0/1410428548 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f597c1002a0 0x7f597c19e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:19.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.464+0000 7f5980e00640 1 -- 172.21.15.97:0/1410428548 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f597c1a5e60 con 0x7f597c102c10 2024-03-27T10:58:19.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.464+0000 7f5981800640 1 --2- 172.21.15.97:0/1410428548 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f597c1002a0 0x7f597c19e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T10:58:19.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.465+0000 7f5980e00640 1 --2- 172.21.15.97:0/1410428548 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f597c102c10 0x7f597c19e780 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f5968000c00 tx=0x7f59680042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:19.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.465+0000 7f597a000640 1 -- 172.21.15.97:0/1410428548 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5968043020 con 0x7f597c102c10 2024-03-27T10:58:19.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.465+0000 7f597a000640 1 -- 172.21.15.97:0/1410428548 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5968004aa0 con 0x7f597c102c10 2024-03-27T10:58:19.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.465+0000 7f597a000640 1 -- 172.21.15.97:0/1410428548 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5968043020 con 0x7f597c102c10 2024-03-27T10:58:19.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.466+0000 7f5982c00640 1 -- 172.21.15.97:0/1410428548 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f597c1a60f0 con 0x7f597c102c10 2024-03-27T10:58:19.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.467+0000 7f5982c00640 1 -- 172.21.15.97:0/1410428548 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f597c1a65c0 con 0x7f597c102c10 2024-03-27T10:58:19.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.473+0000 7f5987003740 1 Processor -- start 2024-03-27T10:58:19.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.473+0000 7f5987003740 1 -- start start 2024-03-27T10:58:19.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.473+0000 7f5987003740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a2c2f8dab0 0x55a2c2f8ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:19.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.474+0000 7f5987003740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a2c2f8e4a0 0x55a2c2f92fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:19.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.474+0000 7f5987003740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a2c2f93610 0x55a2c2f95aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:19.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.474+0000 7f5987003740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55a2c2f557b0 con 0x55a2c2f8e4a0 2024-03-27T10:58:19.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.474+0000 7f5987003740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55a2c2ef2670 con 0x55a2c2f8dab0 2024-03-27T10:58:19.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.474+0000 7f5987003740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55a2c2e9a970 con 0x55a2c2f93610 2024-03-27T10:58:19.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.474+0000 7f5981800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a2c2f8e4a0 0x55a2c2f92fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:19.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.474+0000 7f597a000640 1 -- 172.21.15.97:0/1410428548 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f5968046020 con 0x7f597c102c10 2024-03-27T10:58:19.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.474+0000 7f5982200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a2c2f8dab0 0x55a2c2f8ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:19.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.474+0000 7f5981800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a2c2f8e4a0 0x55a2c2f92fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53088/0 (socket says 172.21.15.97:53088) 2024-03-27T10:58:19.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.474+0000 7f5981800640 1 -- 172.21.15.97:0/692515290 learned_addr learned my addr 172.21.15.97:0/692515290 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:58:19.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.475+0000 7f5982200640 1 -- 172.21.15.97:0/692515290 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a2c2f93610 msgr2=0x55a2c2f95aa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-27T10:58:19.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.475+0000 7f5982200640 1 --2- 172.21.15.97:0/692515290 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a2c2f93610 0x55a2c2f95aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:19.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.475+0000 7f5982200640 1 -- 172.21.15.97:0/692515290 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a2c2f8e4a0 msgr2=0x55a2c2f92fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:19.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.475+0000 7f5982200640 1 --2- 172.21.15.97:0/692515290 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a2c2f8e4a0 0x55a2c2f92fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:19.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.475+0000 7f5982200640 1 -- 172.21.15.97:0/692515290 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2c2c063f0 con 0x55a2c2f8dab0 2024-03-27T10:58:19.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.475+0000 7f5981800640 1 --2- 172.21.15.97:0/692515290 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a2c2f8e4a0 0x55a2c2f92fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T10:58:19.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.476+0000 7f5982200640 1 --2- 172.21.15.97:0/692515290 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a2c2f8dab0 0x55a2c2f8ded0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5974009f30 tx=0x7f5974010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f45e75f479ae02f9 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:19.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.476+0000 7f596ea00640 1 -- 172.21.15.97:0/692515290 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5974016070 con 0x55a2c2f8dab0 2024-03-27T10:58:19.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.476+0000 7f597a000640 1 --2- 172.21.15.97:0/1410428548 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f59640a3a00 0x7f59640a5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:19.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.476+0000 7f596ea00640 1 -- 172.21.15.97:0/692515290 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5974004d20 con 0x55a2c2f8dab0 2024-03-27T10:58:19.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.476+0000 7f596ea00640 1 -- 172.21.15.97:0/692515290 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5974012740 con 0x55a2c2f8dab0 2024-03-27T10:58:19.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.476+0000 7f597a000640 1 -- 172.21.15.97:0/1410428548 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f596803a040 con 0x7f597c102c10 2024-03-27T10:58:19.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.477+0000 7f5981800640 1 --2- 172.21.15.97:0/1410428548 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f59640a3a00 0x7f59640a5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:19.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.477+0000 7f5987003740 1 -- 172.21.15.97:0/692515290 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a2c2f8dab0 msgr2=0x55a2c2f8ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:19.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.477+0000 7f5987003740 1 --2- 172.21.15.97:0/692515290 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a2c2f8dab0 0x55a2c2f8ded0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5974009f30 tx=0x7f5974010800 comp rx=0 tx=0).stop 2024-03-27T10:58:19.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.478+0000 7f5981800640 1 --2- 172.21.15.97:0/1410428548 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f59640a3a00 0x7f59640a5ec0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x55a2c2f8e8a0 tx=0x7f5970005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:19.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.478+0000 7f5987003740 1 -- 172.21.15.97:0/692515290 shutdown_connections 2024-03-27T10:58:19.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.478+0000 7f5987003740 1 --2- 172.21.15.97:0/692515290 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a2c2f93610 0x55a2c2f95aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:19.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.478+0000 7f5987003740 1 --2- 172.21.15.97:0/692515290 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a2c2f8e4a0 0x55a2c2f92fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:19.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.478+0000 7f5987003740 1 --2- 172.21.15.97:0/692515290 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a2c2f8dab0 0x55a2c2f8ded0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:19.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.478+0000 7f5987003740 1 -- 172.21.15.97:0/692515290 >> 172.21.15.97:0/692515290 conn(0x55a2c2f89020 msgr2=0x55a2c2f89420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:19.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.478+0000 7f5987003740 1 -- 172.21.15.97:0/692515290 shutdown_connections 2024-03-27T10:58:19.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.478+0000 7f5987003740 1 -- 172.21.15.97:0/692515290 wait complete. 2024-03-27T10:58:19.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.479+0000 7f5987003740 1 -- start start 2024-03-27T10:58:19.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.479+0000 7f5987003740 1 Processor -- start 2024-03-27T10:58:19.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.479+0000 7f5987003740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a2c2f8dab0 0x55a2c2f97d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:19.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.479+0000 7f5987003740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a2c2f8e4a0 0x55a2c2f98290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:19.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.479+0000 7f5987003740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a2c2f93610 0x55a2c2f9f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:19.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.479+0000 7f5987003740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55a2c2eadca0 con 0x55a2c2f93610 2024-03-27T10:58:19.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.479+0000 7f5987003740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55a2c2eee620 con 0x55a2c2f8dab0 2024-03-27T10:58:19.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.479+0000 7f5987003740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55a2c2f965c0 con 0x55a2c2f8e4a0 2024-03-27T10:58:19.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.479+0000 7f5987003740 10 client.0 _ll_register_callbacks cb 0x55a2c2eea160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:58:19.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.480+0000 7f5981800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a2c2f8dab0 0x55a2c2f97d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:19.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.480+0000 7f5981800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a2c2f8dab0 0x55a2c2f97d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:52378/0 (socket says 172.21.15.97:52378) 2024-03-27T10:58:19.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.480+0000 7f5981800640 1 -- 172.21.15.97:0/3202664857 learned_addr learned my addr 172.21.15.97:0/3202664857 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:58:19.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.480+0000 7f5982200640 1 --2- 172.21.15.97:0/3202664857 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a2c2f93610 0x55a2c2f9f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:19.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.480+0000 7f5980e00640 1 --2- 172.21.15.97:0/3202664857 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a2c2f8e4a0 0x55a2c2f98290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:19.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.480+0000 7f5981800640 1 -- 172.21.15.97:0/3202664857 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a2c2f8e4a0 msgr2=0x55a2c2f98290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:19.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.480+0000 7f5981800640 1 --2- 172.21.15.97:0/3202664857 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a2c2f8e4a0 0x55a2c2f98290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:19.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.480+0000 7f5981800640 1 -- 172.21.15.97:0/3202664857 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a2c2f93610 msgr2=0x55a2c2f9f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:19.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.480+0000 7f5981800640 1 --2- 172.21.15.97:0/3202664857 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a2c2f93610 0x55a2c2f9f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:19.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.480+0000 7f5981800640 1 -- 172.21.15.97:0/3202664857 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2c2cd07d0 con 0x55a2c2f8dab0 2024-03-27T10:58:19.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.481+0000 7f5981800640 1 --2- 172.21.15.97:0/3202664857 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a2c2f8dab0 0x55a2c2f97d50 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f59700045b0 tx=0x7f5970004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69018a4ea439da25 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:19.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.481+0000 7f5963400640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-03-27T10:58:19.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.481+0000 7f5963400640 1 -- 172.21.15.97:0/3202664857 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5970009cf0 con 0x55a2c2f8dab0 2024-03-27T10:58:19.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.481+0000 7f5963400640 1 -- 172.21.15.97:0/3202664857 <== mon.2 v2:172.21.15.114:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f5970003af0 con 0x55a2c2f8dab0 2024-03-27T10:58:19.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.481+0000 7f5987003740 1 -- 172.21.15.97:0/3202664857 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5974004d20 con 0x55a2c2f8dab0 2024-03-27T10:58:19.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.481+0000 7f5963400640 1 -- 172.21.15.97:0/3202664857 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5970003cf0 con 0x55a2c2f8dab0 2024-03-27T10:58:19.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.481+0000 7f5987003740 10 client.35738 fetch_fsmap learned FSMap version 29 2024-03-27T10:58:19.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.481+0000 7f5987003740 1 -- 172.21.15.97:0/3202664857 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a2c2fa26f0 con 0x55a2c2f8dab0 2024-03-27T10:58:19.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.482+0000 7f5963400640 1 -- 172.21.15.97:0/3202664857 <== mon.2 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5974004d20 con 0x55a2c2f8dab0 2024-03-27T10:58:19.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.482+0000 7f5963400640 1 -- 172.21.15.97:0/3202664857 <== mon.2 v2:172.21.15.114:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f5970027070 con 0x55a2c2f8dab0 2024-03-27T10:58:19.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.482+0000 7f5987003740 10 client.35738 fetch_fsmap finished waiting for FSMap version 29 2024-03-27T10:58:19.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.482+0000 7f5987003740 10 client.35738 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-03-27T10:58:19.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.482+0000 7f5987003740 20 client.35738 populate_metadata read hostname 'smithi097' 2024-03-27T10:58:19.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.482+0000 7f5987003740 1 --2- 172.21.15.97:0/3202664857 >> [v2:172.21.15.177:6832/3787981632,v1:172.21.15.177:6833/3787981632] conn(0x55a2c2fa3d10 0x55a2c2fa6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:19.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.482+0000 7f5987003740 4 client.35738 mds_command: new command op to 24400 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T10:58:19.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.482+0000 7f5987003740 1 -- 172.21.15.97:0/3202664857 --> [v2:172.21.15.177:6832/3787981632,v1:172.21.15.177:6833/3787981632] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a2c2effbb0 con 0x55a2c2fa3d10 2024-03-27T10:58:19.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.483+0000 7f5980e00640 1 --2- 172.21.15.97:0/3202664857 >> [v2:172.21.15.177:6832/3787981632,v1:172.21.15.177:6833/3787981632] conn(0x55a2c2fa3d10 0x55a2c2fa6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:19.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.483+0000 7f5980e00640 1 --2- 172.21.15.97:0/3202664857 >> [v2:172.21.15.177:6832/3787981632,v1:172.21.15.177:6833/3787981632] conn(0x55a2c2fa3d10 0x55a2c2fa6170 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9ffddf7aa8c0fa36 server_cookie=a6cecbaa0a01956f in_seq=0 out_seq=0 2024-03-27T10:58:19.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.483+0000 7f5963400640 10 client.35738 ms_handle_connect on v2:172.21.15.177:6832/3787981632 2024-03-27T10:58:19.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.484+0000 7f5963400640 1 -- 172.21.15.97:0/3202664857 <== mds.4 v2:172.21.15.177:6832/3787981632 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55a2c2effbb0 con 0x55a2c2fa3d10 2024-03-27T10:58:19.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.484+0000 7f5963400640 10 client.35738 handle_command_reply: tid=0 2024-03-27T10:58:19.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.484+0000 7f5987003740 1 client.35738 shutdown 2024-03-27T10:58:19.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.485+0000 7f5987003740 1 -- 172.21.15.97:0/3202664857 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a2c2f8dab0 msgr2=0x55a2c2f97d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:19.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.485+0000 7f5987003740 1 --2- 172.21.15.97:0/3202664857 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a2c2f8dab0 0x55a2c2f97d50 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f59700045b0 tx=0x7f5970004730 comp rx=0 tx=0).stop 2024-03-27T10:58:19.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.485+0000 7f5987003740 1 -- 172.21.15.97:0/3202664857 shutdown_connections 2024-03-27T10:58:19.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.485+0000 7f5987003740 1 --2- 172.21.15.97:0/3202664857 >> [v2:172.21.15.177:6832/3787981632,v1:172.21.15.177:6833/3787981632] conn(0x55a2c2fa3d10 0x55a2c2fa6170 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:19.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.485+0000 7f5987003740 1 --2- 172.21.15.97:0/3202664857 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a2c2f93610 0x55a2c2f9f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:19.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.485+0000 7f5987003740 1 --2- 172.21.15.97:0/3202664857 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a2c2f8e4a0 0x55a2c2f98290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:19.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.485+0000 7f5963400640 0 client.35738 ms_handle_reset on v2:172.21.15.177:6832/3787981632 2024-03-27T10:58:19.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.485+0000 7f5987003740 1 --2- 172.21.15.97:0/3202664857 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a2c2f8dab0 0x55a2c2f97d50 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:19.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.485+0000 7f5987003740 1 -- 172.21.15.97:0/3202664857 >> 172.21.15.97:0/3202664857 conn(0x55a2c2f89020 msgr2=0x55a2c2f88240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:19.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.485+0000 7f5987003740 1 -- 172.21.15.97:0/3202664857 shutdown_connections 2024-03-27T10:58:19.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.485+0000 7f5987003740 1 -- 172.21.15.97:0/3202664857 wait complete. 2024-03-27T10:58:19.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.485+0000 7f5987003740 20 client.35738 trim_cache size 0 max 16384 2024-03-27T10:58:19.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.502+0000 7f5987003740 1 Processor -- start 2024-03-27T10:58:19.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.502+0000 7f5987003740 1 -- start start 2024-03-27T10:58:19.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.502+0000 7f5987003740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a2c2f8dab0 0x55a2c2f92ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:19.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.502+0000 7f5987003740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a2c2f8e4a0 0x55a2c2f9f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:19.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.502+0000 7f5987003740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a2c2f93610 0x55a2c2f879b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:19.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.502+0000 7f5987003740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f597c1af330 con 0x55a2c2f8dab0 2024-03-27T10:58:19.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.502+0000 7f5987003740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f597c1b0ea0 con 0x55a2c2f93610 2024-03-27T10:58:19.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.502+0000 7f5987003740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f597c1afa00 con 0x55a2c2f8e4a0 2024-03-27T10:58:19.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.503+0000 7f5981800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a2c2f8dab0 0x55a2c2f92ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:19.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.503+0000 7f5981800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a2c2f8dab0 0x55a2c2f92ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53104/0 (socket says 172.21.15.97:53104) 2024-03-27T10:58:19.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.503+0000 7f5981800640 1 -- 172.21.15.97:0/287276899 learned_addr learned my addr 172.21.15.97:0/287276899 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:58:19.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.503+0000 7f5982200640 1 --2- 172.21.15.97:0/287276899 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a2c2f93610 0x55a2c2f879b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:19.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.503+0000 7f5981800640 1 -- 172.21.15.97:0/287276899 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a2c2f8e4a0 msgr2=0x55a2c2f9f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:19.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.503+0000 7f5981800640 1 --2- 172.21.15.97:0/287276899 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a2c2f8e4a0 0x55a2c2f9f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:19.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.503+0000 7f5981800640 1 -- 172.21.15.97:0/287276899 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a2c2f93610 msgr2=0x55a2c2f879b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:19.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.503+0000 7f5981800640 1 --2- 172.21.15.97:0/287276899 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a2c2f93610 0x55a2c2f879b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:19.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.503+0000 7f5981800640 1 -- 172.21.15.97:0/287276899 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2c2fa67f0 con 0x55a2c2f8dab0 2024-03-27T10:58:19.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.504+0000 7f5981800640 1 --2- 172.21.15.97:0/287276899 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a2c2f8dab0 0x55a2c2f92ed0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f5970019040 tx=0x7f59700043e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89e0ff127d928bd2 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:19.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.504+0000 7f596e000640 1 -- 172.21.15.97:0/287276899 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f597001f770 con 0x55a2c2f8dab0 2024-03-27T10:58:19.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.505+0000 7f596e000640 1 -- 172.21.15.97:0/287276899 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5970038070 con 0x55a2c2f8dab0 2024-03-27T10:58:19.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.505+0000 7f596e000640 1 -- 172.21.15.97:0/287276899 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59700047a0 con 0x55a2c2f8dab0 2024-03-27T10:58:19.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.506+0000 7f5987003740 1 -- 172.21.15.97:0/287276899 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a2c2f8dab0 msgr2=0x55a2c2f92ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:19.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.506+0000 7f5987003740 1 --2- 172.21.15.97:0/287276899 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a2c2f8dab0 0x55a2c2f92ed0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f5970019040 tx=0x7f59700043e0 comp rx=0 tx=0).stop 2024-03-27T10:58:19.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.506+0000 7f5987003740 1 -- 172.21.15.97:0/287276899 shutdown_connections 2024-03-27T10:58:19.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.506+0000 7f5987003740 1 --2- 172.21.15.97:0/287276899 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a2c2f93610 0x55a2c2f879b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:19.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.506+0000 7f5987003740 1 --2- 172.21.15.97:0/287276899 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a2c2f8e4a0 0x55a2c2f9f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:19.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.506+0000 7f5987003740 1 --2- 172.21.15.97:0/287276899 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a2c2f8dab0 0x55a2c2f92ed0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:19.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.506+0000 7f5987003740 1 -- 172.21.15.97:0/287276899 >> 172.21.15.97:0/287276899 conn(0x55a2c2f89020 msgr2=0x55a2c2f97d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:19.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.506+0000 7f5987003740 1 -- 172.21.15.97:0/287276899 shutdown_connections 2024-03-27T10:58:19.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.506+0000 7f5987003740 1 -- 172.21.15.97:0/287276899 wait complete. 2024-03-27T10:58:19.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.507+0000 7f5987003740 1 -- start start 2024-03-27T10:58:19.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.507+0000 7f5987003740 1 Processor -- start 2024-03-27T10:58:19.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.508+0000 7f5987003740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a2c2f8dab0 0x55a2c2f9ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:19.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.508+0000 7f5987003740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a2c2f8e4a0 0x55a2c2f9d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:19.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.508+0000 7f5987003740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a2c2f93610 0x55a2c2f9e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:19.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.508+0000 7f5987003740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f597c1b07a0 con 0x55a2c2f8e4a0 2024-03-27T10:58:19.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.508+0000 7f5987003740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f597c1b00d0 con 0x55a2c2f93610 2024-03-27T10:58:19.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.508+0000 7f5987003740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f597c1ad310 con 0x55a2c2f8dab0 2024-03-27T10:58:19.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.508+0000 7f5987003740 10 client.0 _ll_register_callbacks cb 0x7f5970009c40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:58:19.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.508+0000 7f5980e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a2c2f8e4a0 0x55a2c2f9d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:19.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.508+0000 7f5982200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a2c2f93610 0x55a2c2f9e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:19.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.508+0000 7f5982200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a2c2f93610 0x55a2c2f9e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:52396/0 (socket says 172.21.15.97:52396) 2024-03-27T10:58:19.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.508+0000 7f5982200640 1 -- 172.21.15.97:0/632333192 learned_addr learned my addr 172.21.15.97:0/632333192 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:58:19.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.508+0000 7f5981800640 1 --2- 172.21.15.97:0/632333192 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a2c2f8dab0 0x55a2c2f9ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:19.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.509+0000 7f5982200640 1 -- 172.21.15.97:0/632333192 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a2c2f8dab0 msgr2=0x55a2c2f9ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:19.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.509+0000 7f5982200640 1 --2- 172.21.15.97:0/632333192 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a2c2f8dab0 0x55a2c2f9ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:19.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.509+0000 7f5982200640 1 -- 172.21.15.97:0/632333192 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a2c2f8e4a0 msgr2=0x55a2c2f9d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:19.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.509+0000 7f5982200640 1 --2- 172.21.15.97:0/632333192 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a2c2f8e4a0 0x55a2c2f9d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:19.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.509+0000 7f5982200640 1 -- 172.21.15.97:0/632333192 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2c2f9e910 con 0x55a2c2f93610 2024-03-27T10:58:19.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.509+0000 7f5980e00640 1 --2- 172.21.15.97:0/632333192 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a2c2f8e4a0 0x55a2c2f9d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T10:58:19.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.510+0000 7f5982200640 1 --2- 172.21.15.97:0/632333192 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a2c2f93610 0x55a2c2f9e320 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5974009f30 tx=0x7f5974004e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f09a787e98566cd server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:19.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.510+0000 7f5963400640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-03-27T10:58:19.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.510+0000 7f5963400640 1 -- 172.21.15.97:0/632333192 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5974016070 con 0x55a2c2f93610 2024-03-27T10:58:19.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.510+0000 7f5963400640 1 -- 172.21.15.97:0/632333192 <== mon.2 v2:172.21.15.114:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f5974005510 con 0x55a2c2f93610 2024-03-27T10:58:19.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.510+0000 7f5987003740 1 -- 172.21.15.97:0/632333192 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5970038070 con 0x55a2c2f93610 2024-03-27T10:58:19.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.511+0000 7f5963400640 1 -- 172.21.15.97:0/632333192 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5974005710 con 0x55a2c2f93610 2024-03-27T10:58:19.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.511+0000 7f5987003740 10 client.35744 fetch_fsmap learned FSMap version 29 2024-03-27T10:58:19.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.511+0000 7f5987003740 1 -- 172.21.15.97:0/632333192 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a2c300a4a0 con 0x55a2c2f93610 2024-03-27T10:58:19.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.512+0000 7f5963400640 1 -- 172.21.15.97:0/632333192 <== mon.2 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5970038070 con 0x55a2c2f93610 2024-03-27T10:58:19.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.512+0000 7f5963400640 1 -- 172.21.15.97:0/632333192 <== mon.2 v2:172.21.15.114:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f597400f040 con 0x55a2c2f93610 2024-03-27T10:58:19.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.512+0000 7f5987003740 10 client.35744 fetch_fsmap finished waiting for FSMap version 29 2024-03-27T10:58:19.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.512+0000 7f5987003740 10 client.35744 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-03-27T10:58:19.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.512+0000 7f5987003740 20 client.35744 populate_metadata read hostname 'smithi097' 2024-03-27T10:58:19.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.512+0000 7f5987003740 1 --2- 172.21.15.97:0/632333192 >> [v2:172.21.15.177:6832/3787981632,v1:172.21.15.177:6833/3787981632] conn(0x55a2c300ab00 0x55a2c300cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:19.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.512+0000 7f5987003740 4 client.35744 mds_command: new command op to 24400 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-27T10:58:19.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.512+0000 7f5987003740 1 -- 172.21.15.97:0/632333192 --> [v2:172.21.15.177:6832/3787981632,v1:172.21.15.177:6833/3787981632] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a2c2ef9f00 con 0x55a2c300ab00 2024-03-27T10:58:19.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.513+0000 7f5981800640 1 --2- 172.21.15.97:0/632333192 >> [v2:172.21.15.177:6832/3787981632,v1:172.21.15.177:6833/3787981632] conn(0x55a2c300ab00 0x55a2c300cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:19.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.514+0000 7f5981800640 1 --2- 172.21.15.97:0/632333192 >> [v2:172.21.15.177:6832/3787981632,v1:172.21.15.177:6833/3787981632] conn(0x55a2c300ab00 0x55a2c300cef0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e67cc07bf692d667 server_cookie=ad900423ddc25da1 in_seq=0 out_seq=0 2024-03-27T10:58:19.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.514+0000 7f5963400640 10 client.35744 ms_handle_connect on v2:172.21.15.177:6832/3787981632 2024-03-27T10:58:19.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.514+0000 7f5963400640 1 -- 172.21.15.97:0/632333192 <== mds.4 v2:172.21.15.177:6832/3787981632 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a2c2ef9f00 con 0x55a2c300ab00 2024-03-27T10:58:19.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.514+0000 7f5963400640 10 client.35744 handle_command_reply: tid=0 2024-03-27T10:58:19.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.514+0000 7f5987003740 1 client.35744 shutdown 2024-03-27T10:58:19.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.515+0000 7f5987003740 1 -- 172.21.15.97:0/632333192 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a2c2f93610 msgr2=0x55a2c2f9e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:19.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.515+0000 7f5987003740 1 --2- 172.21.15.97:0/632333192 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a2c2f93610 0x55a2c2f9e320 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5974009f30 tx=0x7f5974004e00 comp rx=0 tx=0).stop 2024-03-27T10:58:19.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.515+0000 7f5987003740 1 -- 172.21.15.97:0/632333192 shutdown_connections 2024-03-27T10:58:19.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.515+0000 7f5987003740 1 --2- 172.21.15.97:0/632333192 >> [v2:172.21.15.177:6832/3787981632,v1:172.21.15.177:6833/3787981632] conn(0x55a2c300ab00 0x55a2c300cef0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:19.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.515+0000 7f5987003740 1 --2- 172.21.15.97:0/632333192 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a2c2f93610 0x55a2c2f9e320 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:19.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.515+0000 7f5987003740 1 --2- 172.21.15.97:0/632333192 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a2c2f8e4a0 0x55a2c2f9d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:19.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.515+0000 7f5987003740 1 --2- 172.21.15.97:0/632333192 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a2c2f8dab0 0x55a2c2f9ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:19.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.515+0000 7f5963400640 0 client.35744 ms_handle_reset on v2:172.21.15.177:6832/3787981632 2024-03-27T10:58:19.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.515+0000 7f5987003740 1 -- 172.21.15.97:0/632333192 >> 172.21.15.97:0/632333192 conn(0x55a2c2f89020 msgr2=0x55a2c2f97cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:19.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.516+0000 7f5987003740 1 -- 172.21.15.97:0/632333192 shutdown_connections 2024-03-27T10:58:19.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.516+0000 7f5987003740 1 -- 172.21.15.97:0/632333192 wait complete. 2024-03-27T10:58:19.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.516+0000 7f5987003740 20 client.35744 trim_cache size 0 max 16384 2024-03-27T10:58:19.489 INFO:teuthology.orchestra.run.smithi097.stdout:[] 2024-03-27T10:58:19.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.517+0000 7f5982c00640 1 -- 172.21.15.97:0/1410428548 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f59640a3a00 msgr2=0x7f59640a5ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:19.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.517+0000 7f5982c00640 1 --2- 172.21.15.97:0/1410428548 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f59640a3a00 0x7f59640a5ec0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x55a2c2f8e8a0 tx=0x7f5970005f90 comp rx=0 tx=0).stop 2024-03-27T10:58:19.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.517+0000 7f5982c00640 1 -- 172.21.15.97:0/1410428548 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f597c102c10 msgr2=0x7f597c19e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:19.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.517+0000 7f5982c00640 1 --2- 172.21.15.97:0/1410428548 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f597c102c10 0x7f597c19e780 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f5968000c00 tx=0x7f59680042f0 comp rx=0 tx=0).stop 2024-03-27T10:58:19.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.517+0000 7f5982c00640 1 -- 172.21.15.97:0/1410428548 shutdown_connections 2024-03-27T10:58:19.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.517+0000 7f5982c00640 1 --2- 172.21.15.97:0/1410428548 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f597c1055c0 0x7f597c1a57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:19.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.517+0000 7f5982c00640 1 --2- 172.21.15.97:0/1410428548 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f597c102c10 0x7f597c19e780 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:19.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.517+0000 7f5982c00640 1 --2- 172.21.15.97:0/1410428548 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f59640a3a00 0x7f59640a5ec0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:19.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.517+0000 7f5982c00640 1 --2- 172.21.15.97:0/1410428548 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f597c1002a0 0x7f597c19e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:19.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.517+0000 7f5982c00640 1 -- 172.21.15.97:0/1410428548 >> 172.21.15.97:0/1410428548 conn(0x7f597c0fbec0 msgr2=0x7f597c101050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:19.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.517+0000 7f5982c00640 1 -- 172.21.15.97:0/1410428548 shutdown_connections 2024-03-27T10:58:19.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.518+0000 7f5982c00640 1 -- 172.21.15.97:0/1410428548 wait complete. 2024-03-27T10:58:19.511 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:58:19 smithi097 ceph-mon[31859]: pgmap v491: 97 pgs: 97 active+clean; 1.2 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 99 MiB/s wr, 73 op/s 2024-03-27T10:58:19.511 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:58:19 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/2794000730' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T10:58:19.514 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-27T10:58:19.516 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:58:19 smithi114 ceph-mon[35338]: pgmap v491: 97 pgs: 97 active+clean; 1.2 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 99 MiB/s wr, 73 op/s 2024-03-27T10:58:19.516 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:58:19 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/2794000730' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T10:58:19.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:58:19 smithi177 ceph-mon[35294]: pgmap v491: 97 pgs: 97 active+clean; 1.2 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 99 MiB/s wr, 73 op/s 2024-03-27T10:58:19.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:58:19 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/2794000730' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T10:58:19.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.728+0000 7f26a4a00640 1 -- 172.21.15.97:0/2085154360 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26a0107840 msgr2=0x7f26a0109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:19.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.728+0000 7f26a4a00640 1 --2- 172.21.15.97:0/2085154360 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26a0107840 0x7f26a0109c90 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f2698009ca0 tx=0x7f269802c0f0 comp rx=0 tx=0).stop 2024-03-27T10:58:19.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.728+0000 7f26a4a00640 1 -- 172.21.15.97:0/2085154360 shutdown_connections 2024-03-27T10:58:19.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.728+0000 7f26a4a00640 1 --2- 172.21.15.97:0/2085154360 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26a0107840 0x7f26a0109c90 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:19.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.728+0000 7f26a4a00640 1 --2- 172.21.15.97:0/2085154360 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f26a0104f10 0x7f26a0107300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:19.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.728+0000 7f26a4a00640 1 --2- 172.21.15.97:0/2085154360 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f26a01025e0 0x7f26a01049d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:19.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.728+0000 7f26a4a00640 1 -- 172.21.15.97:0/2085154360 >> 172.21.15.97:0/2085154360 conn(0x7f26a00fbff0 msgr2=0x7f26a00fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:19.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.728+0000 7f26a4a00640 1 -- 172.21.15.97:0/2085154360 shutdown_connections 2024-03-27T10:58:19.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.728+0000 7f26a4a00640 1 -- 172.21.15.97:0/2085154360 wait complete. 2024-03-27T10:58:19.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.729+0000 7f26a4a00640 1 Processor -- start 2024-03-27T10:58:19.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.729+0000 7f26a4a00640 1 -- start start 2024-03-27T10:58:19.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.729+0000 7f26a4a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26a01025e0 0x7f26a0197cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:19.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.729+0000 7f26a4a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f26a0104f10 0x7f26a0198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:19.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.729+0000 7f26a4a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f26a0107840 0x7f26a019f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:19.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.729+0000 7f26a4a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f26a010d690 con 0x7f26a01025e0 2024-03-27T10:58:19.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.729+0000 7f26a4a00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f26a010d510 con 0x7f26a0104f10 2024-03-27T10:58:19.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.729+0000 7f26a4a00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f26a010d810 con 0x7f26a0107840 2024-03-27T10:58:19.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.729+0000 7f269de00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26a01025e0 0x7f26a0197cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:19.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.729+0000 7f269de00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26a01025e0 0x7f26a0197cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53140/0 (socket says 172.21.15.97:53140) 2024-03-27T10:58:19.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.729+0000 7f269de00640 1 -- 172.21.15.97:0/983816129 learned_addr learned my addr 172.21.15.97:0/983816129 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:58:19.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.730+0000 7f269d400640 1 --2- 172.21.15.97:0/983816129 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f26a0104f10 0x7f26a0198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:19.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.730+0000 7f269e800640 1 --2- 172.21.15.97:0/983816129 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f26a0107840 0x7f26a019f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:19.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.730+0000 7f269de00640 1 -- 172.21.15.97:0/983816129 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f26a0107840 msgr2=0x7f26a019f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:19.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.730+0000 7f269de00640 1 --2- 172.21.15.97:0/983816129 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f26a0107840 0x7f26a019f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:19.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.730+0000 7f269de00640 1 -- 172.21.15.97:0/983816129 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f26a0104f10 msgr2=0x7f26a0198230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:19.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.730+0000 7f269de00640 1 --2- 172.21.15.97:0/983816129 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f26a0104f10 0x7f26a0198230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:19.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.730+0000 7f269de00640 1 -- 172.21.15.97:0/983816129 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26a019f900 con 0x7f26a01025e0 2024-03-27T10:58:19.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.730+0000 7f269d400640 1 --2- 172.21.15.97:0/983816129 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f26a0104f10 0x7f26a0198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T10:58:19.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.730+0000 7f269de00640 1 --2- 172.21.15.97:0/983816129 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26a01025e0 0x7f26a0197cf0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f269000c890 tx=0x7f269000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:19.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.730+0000 7f2696a00640 1 -- 172.21.15.97:0/983816129 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2690004220 con 0x7f26a01025e0 2024-03-27T10:58:19.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.730+0000 7f2696a00640 1 -- 172.21.15.97:0/983816129 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f26900043c0 con 0x7f26a01025e0 2024-03-27T10:58:19.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.730+0000 7f2696a00640 1 -- 172.21.15.97:0/983816129 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2690004a00 con 0x7f26a01025e0 2024-03-27T10:58:19.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.730+0000 7f26a4a00640 1 -- 172.21.15.97:0/983816129 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26a019fbf0 con 0x7f26a01025e0 2024-03-27T10:58:19.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.731+0000 7f26a4a00640 1 -- 172.21.15.97:0/983816129 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26a01a00f0 con 0x7f26a01025e0 2024-03-27T10:58:19.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.734+0000 7f26a77f5740 1 Processor -- start 2024-03-27T10:58:19.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.734+0000 7f26a77f5740 1 -- start start 2024-03-27T10:58:19.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.735+0000 7f26a77f5740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e0096a48c0 0x55e0096a4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:19.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.735+0000 7f26a77f5740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e0096a52b0 0x55e0096a9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:19.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.735+0000 7f26a77f5740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e0096aa420 0x55e0096ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:19.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.735+0000 7f26a77f5740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55e00966c7b0 con 0x55e0096a52b0 2024-03-27T10:58:19.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.735+0000 7f26a77f5740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55e009609670 con 0x55e0096aa420 2024-03-27T10:58:19.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.735+0000 7f26a77f5740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55e0095b1970 con 0x55e0096a48c0 2024-03-27T10:58:19.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.735+0000 7f2696a00640 1 -- 172.21.15.97:0/983816129 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f2690004ba0 con 0x7f26a01025e0 2024-03-27T10:58:19.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.735+0000 7f269de00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e0096a52b0 0x55e0096a9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:19.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.735+0000 7f269d400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e0096aa420 0x55e0096ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:19.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.735+0000 7f269de00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e0096a52b0 0x55e0096a9db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53150/0 (socket says 172.21.15.97:53150) 2024-03-27T10:58:19.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.735+0000 7f269de00640 1 -- 172.21.15.97:0/4214193700 learned_addr learned my addr 172.21.15.97:0/4214193700 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:58:19.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.735+0000 7f269e800640 1 --2- 172.21.15.97:0/4214193700 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e0096a48c0 0x55e0096a4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:19.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.736+0000 7f269de00640 1 -- 172.21.15.97:0/4214193700 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e0096a48c0 msgr2=0x55e0096a4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:19.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.736+0000 7f269de00640 1 --2- 172.21.15.97:0/4214193700 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e0096a48c0 0x55e0096a4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:19.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.736+0000 7f269de00640 1 -- 172.21.15.97:0/4214193700 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e0096aa420 msgr2=0x55e0096ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:19.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.736+0000 7f269de00640 1 --2- 172.21.15.97:0/4214193700 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e0096aa420 0x55e0096ac8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:19.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.736+0000 7f269de00640 1 -- 172.21.15.97:0/4214193700 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e00931d3f0 con 0x55e0096a52b0 2024-03-27T10:58:19.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.736+0000 7f269de00640 1 --2- 172.21.15.97:0/4214193700 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e0096a52b0 0x55e0096a9db0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f26900d3b00 tx=0x7f26900d81e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b40fa8320fa121f server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:19.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.736+0000 7f268b000640 1 -- 172.21.15.97:0/4214193700 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26900d8c10 con 0x55e0096a52b0 2024-03-27T10:58:19.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.736+0000 7f268b000640 1 -- 172.21.15.97:0/4214193700 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f26900e1070 con 0x55e0096a52b0 2024-03-27T10:58:19.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.736+0000 7f268b000640 1 -- 172.21.15.97:0/4214193700 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26900db5c0 con 0x55e0096a52b0 2024-03-27T10:58:19.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.737+0000 7f2696a00640 1 --2- 172.21.15.97:0/983816129 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f26840a3610 0x7f26840a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:19.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.737+0000 7f2696a00640 1 -- 172.21.15.97:0/983816129 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f26900cfb60 con 0x7f26a01025e0 2024-03-27T10:58:19.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.737+0000 7f269e800640 1 --2- 172.21.15.97:0/983816129 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f26840a3610 0x7f26840a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:19.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.737+0000 7f26a77f5740 1 -- 172.21.15.97:0/4214193700 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e0096a52b0 msgr2=0x55e0096a9db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:19.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.737+0000 7f26a77f5740 1 --2- 172.21.15.97:0/4214193700 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e0096a52b0 0x55e0096a9db0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f26900d3b00 tx=0x7f26900d81e0 comp rx=0 tx=0).stop 2024-03-27T10:58:19.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.737+0000 7f26a77f5740 1 -- 172.21.15.97:0/4214193700 shutdown_connections 2024-03-27T10:58:19.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.737+0000 7f26a77f5740 1 --2- 172.21.15.97:0/4214193700 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e0096aa420 0x55e0096ac8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:19.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.737+0000 7f26a77f5740 1 --2- 172.21.15.97:0/4214193700 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e0096a52b0 0x55e0096a9db0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:19.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.737+0000 7f26a77f5740 1 --2- 172.21.15.97:0/4214193700 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e0096a48c0 0x55e0096a4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:19.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.737+0000 7f26a77f5740 1 -- 172.21.15.97:0/4214193700 >> 172.21.15.97:0/4214193700 conn(0x55e00969fe30 msgr2=0x55e0096a0230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:19.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.737+0000 7f26a77f5740 1 -- 172.21.15.97:0/4214193700 shutdown_connections 2024-03-27T10:58:19.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.738+0000 7f26a77f5740 1 -- 172.21.15.97:0/4214193700 wait complete. 2024-03-27T10:58:19.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.738+0000 7f269e800640 1 --2- 172.21.15.97:0/983816129 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f26840a3610 0x7f26840a5ad0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f269802c700 tx=0x7f2698032510 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:19.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.738+0000 7f26a77f5740 1 -- start start 2024-03-27T10:58:19.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.738+0000 7f26a77f5740 1 Processor -- start 2024-03-27T10:58:19.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.738+0000 7f26a77f5740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e0096a48c0 0x55e0096ae9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:19.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.738+0000 7f26a77f5740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e0096a52b0 0x55e0096aef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:19.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.739+0000 7f26a77f5740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e0096aa420 0x55e0096b5f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:19.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.739+0000 7f26a77f5740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55e0095c4ca0 con 0x55e0096a48c0 2024-03-27T10:58:19.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.739+0000 7f26a77f5740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55e009605620 con 0x55e0096a52b0 2024-03-27T10:58:19.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.739+0000 7f26a77f5740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55e0096ad370 con 0x55e0096aa420 2024-03-27T10:58:19.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.739+0000 7f26a77f5740 10 client.0 _ll_register_callbacks cb 0x55e009601160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:58:19.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.739+0000 7f269de00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e0096a48c0 0x55e0096ae9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:19.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.739+0000 7f269de00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e0096a48c0 0x55e0096ae9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53152/0 (socket says 172.21.15.97:53152) 2024-03-27T10:58:19.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.739+0000 7f269de00640 1 -- 172.21.15.97:0/1024906488 learned_addr learned my addr 172.21.15.97:0/1024906488 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:58:19.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.739+0000 7f269d400640 1 --2- 172.21.15.97:0/1024906488 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e0096a52b0 0x55e0096aef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:19.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.739+0000 7f269e800640 1 --2- 172.21.15.97:0/1024906488 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e0096aa420 0x55e0096b5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:19.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.739+0000 7f269de00640 1 -- 172.21.15.97:0/1024906488 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e0096aa420 msgr2=0x55e0096b5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:19.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.739+0000 7f269de00640 1 --2- 172.21.15.97:0/1024906488 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e0096aa420 0x55e0096b5f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:19.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.739+0000 7f269de00640 1 -- 172.21.15.97:0/1024906488 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e0096a52b0 msgr2=0x55e0096aef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:19.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.739+0000 7f269de00640 1 --2- 172.21.15.97:0/1024906488 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e0096a52b0 0x55e0096aef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:19.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.739+0000 7f269de00640 1 -- 172.21.15.97:0/1024906488 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0093e77d0 con 0x55e0096a48c0 2024-03-27T10:58:19.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.739+0000 7f269d400640 1 --2- 172.21.15.97:0/1024906488 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e0096a52b0 0x55e0096aef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:58:19.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.739+0000 7f269de00640 1 --2- 172.21.15.97:0/1024906488 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e0096a48c0 0x55e0096ae9e0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f2690098920 tx=0x7f26900053f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b527cb883aaa75f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:19.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.739+0000 7f267f400640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T10:58:19.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.740+0000 7f267f400640 1 -- 172.21.15.97:0/1024906488 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f269001d070 con 0x55e0096a48c0 2024-03-27T10:58:19.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.740+0000 7f267f400640 1 -- 172.21.15.97:0/1024906488 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f26900050b0 con 0x55e0096a48c0 2024-03-27T10:58:19.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.740+0000 7f26a77f5740 1 -- 172.21.15.97:0/1024906488 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26900e1070 con 0x55e0096a48c0 2024-03-27T10:58:19.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.740+0000 7f267f400640 1 -- 172.21.15.97:0/1024906488 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f26900d3020 con 0x55e0096a48c0 2024-03-27T10:58:19.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.740+0000 7f26a77f5740 10 client.16101 fetch_fsmap learned FSMap version 29 2024-03-27T10:58:19.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.740+0000 7f26a77f5740 1 -- 172.21.15.97:0/1024906488 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e0096b92d0 con 0x55e0096a48c0 2024-03-27T10:58:19.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.740+0000 7f267f400640 1 -- 172.21.15.97:0/1024906488 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f26900e1070 con 0x55e0096a48c0 2024-03-27T10:58:19.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.740+0000 7f267f400640 1 -- 172.21.15.97:0/1024906488 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f26900b50f0 con 0x55e0096a48c0 2024-03-27T10:58:19.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.740+0000 7f26a77f5740 10 client.16101 fetch_fsmap finished waiting for FSMap version 29 2024-03-27T10:58:19.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.740+0000 7f26a77f5740 10 client.16101 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-03-27T10:58:19.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.740+0000 7f26a77f5740 20 client.16101 populate_metadata read hostname 'smithi097' 2024-03-27T10:58:19.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.740+0000 7f26a77f5740 1 --2- 172.21.15.97:0/1024906488 >> [v2:172.21.15.177:6834/3349112132,v1:172.21.15.177:6835/3349112132] conn(0x55e0096ba620 0x55e0096bca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:19.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.740+0000 7f26a77f5740 4 client.16101 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T10:58:19.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.740+0000 7f26a77f5740 1 -- 172.21.15.97:0/1024906488 --> [v2:172.21.15.177:6834/3349112132,v1:172.21.15.177:6835/3349112132] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e009616bb0 con 0x55e0096ba620 2024-03-27T10:58:19.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.741+0000 7f269d400640 1 --2- 172.21.15.97:0/1024906488 >> [v2:172.21.15.177:6834/3349112132,v1:172.21.15.177:6835/3349112132] conn(0x55e0096ba620 0x55e0096bca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:19.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.742+0000 7f269d400640 1 --2- 172.21.15.97:0/1024906488 >> [v2:172.21.15.177:6834/3349112132,v1:172.21.15.177:6835/3349112132] conn(0x55e0096ba620 0x55e0096bca10 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c148cc949fbe772d server_cookie=d579ab1177b8d305 in_seq=0 out_seq=0 2024-03-27T10:58:19.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.742+0000 7f267f400640 10 client.16101 ms_handle_connect on v2:172.21.15.177:6834/3349112132 2024-03-27T10:58:19.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.744+0000 7f267f400640 1 -- 172.21.15.97:0/1024906488 <== mds.3 v2:172.21.15.177:6834/3349112132 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55e009616bb0 con 0x55e0096ba620 2024-03-27T10:58:19.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.744+0000 7f267f400640 10 client.16101 handle_command_reply: tid=0 2024-03-27T10:58:19.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.744+0000 7f26a77f5740 1 client.16101 shutdown 2024-03-27T10:58:19.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.744+0000 7f26a77f5740 1 -- 172.21.15.97:0/1024906488 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e0096a48c0 msgr2=0x55e0096ae9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:19.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.744+0000 7f26a77f5740 1 --2- 172.21.15.97:0/1024906488 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e0096a48c0 0x55e0096ae9e0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f2690098920 tx=0x7f26900053f0 comp rx=0 tx=0).stop 2024-03-27T10:58:19.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.744+0000 7f26a77f5740 1 -- 172.21.15.97:0/1024906488 shutdown_connections 2024-03-27T10:58:19.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.744+0000 7f26a77f5740 1 --2- 172.21.15.97:0/1024906488 >> [v2:172.21.15.177:6834/3349112132,v1:172.21.15.177:6835/3349112132] conn(0x55e0096ba620 0x55e0096bca10 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:19.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.744+0000 7f26a77f5740 1 --2- 172.21.15.97:0/1024906488 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e0096aa420 0x55e0096b5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:19.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.744+0000 7f267f400640 0 client.16101 ms_handle_reset on v2:172.21.15.177:6834/3349112132 2024-03-27T10:58:19.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.744+0000 7f26a77f5740 1 --2- 172.21.15.97:0/1024906488 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e0096a52b0 0x55e0096aef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:19.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.744+0000 7f26a77f5740 1 --2- 172.21.15.97:0/1024906488 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e0096a48c0 0x55e0096ae9e0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:19.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.744+0000 7f26a77f5740 1 -- 172.21.15.97:0/1024906488 >> 172.21.15.97:0/1024906488 conn(0x55e00969fe30 msgr2=0x55e00969f030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:19.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.744+0000 7f26a77f5740 1 -- 172.21.15.97:0/1024906488 shutdown_connections 2024-03-27T10:58:19.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.744+0000 7f26a77f5740 1 -- 172.21.15.97:0/1024906488 wait complete. 2024-03-27T10:58:19.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.744+0000 7f26a77f5740 20 client.16101 trim_cache size 0 max 16384 2024-03-27T10:58:19.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.758+0000 7f26a77f5740 1 Processor -- start 2024-03-27T10:58:19.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.758+0000 7f26a77f5740 1 -- start start 2024-03-27T10:58:19.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.758+0000 7f26a77f5740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e0096a48c0 0x55e0096ae5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:19.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.758+0000 7f26a77f5740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e0096a52b0 0x55e0096b5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:19.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.758+0000 7f26a77f5740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e0096aa420 0x55e0096aeb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:19.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.758+0000 7f26a77f5740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f26a01a8eb0 con 0x55e0096aa420 2024-03-27T10:58:19.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.758+0000 7f26a77f5740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f26a01aaa20 con 0x55e0096a52b0 2024-03-27T10:58:19.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.758+0000 7f26a77f5740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f26a01a9580 con 0x55e0096a48c0 2024-03-27T10:58:19.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.758+0000 7f269e800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e0096aa420 0x55e0096aeb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:19.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.759+0000 7f269e800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e0096aa420 0x55e0096aeb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53166/0 (socket says 172.21.15.97:53166) 2024-03-27T10:58:19.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.759+0000 7f269e800640 1 -- 172.21.15.97:0/3856413991 learned_addr learned my addr 172.21.15.97:0/3856413991 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:58:19.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.759+0000 7f269d400640 1 --2- 172.21.15.97:0/3856413991 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e0096a52b0 0x55e0096b5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:19.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.759+0000 7f269de00640 1 --2- 172.21.15.97:0/3856413991 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e0096a48c0 0x55e0096ae5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:19.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.759+0000 7f269e800640 1 -- 172.21.15.97:0/3856413991 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e0096a48c0 msgr2=0x55e0096ae5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:19.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.759+0000 7f269e800640 1 --2- 172.21.15.97:0/3856413991 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e0096a48c0 0x55e0096ae5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:19.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.759+0000 7f269e800640 1 -- 172.21.15.97:0/3856413991 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e0096a52b0 msgr2=0x55e0096b5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:19.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.759+0000 7f269e800640 1 --2- 172.21.15.97:0/3856413991 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e0096a52b0 0x55e0096b5ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:19.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.759+0000 7f269e800640 1 -- 172.21.15.97:0/3856413991 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0096a0ba0 con 0x55e0096aa420 2024-03-27T10:58:19.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.759+0000 7f269e800640 1 --2- 172.21.15.97:0/3856413991 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e0096aa420 0x55e0096aeb30 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f2698009ca0 tx=0x7f269803f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5457e920ddb6a3a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:19.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.760+0000 7f268a600640 1 -- 172.21.15.97:0/3856413991 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2698039040 con 0x55e0096aa420 2024-03-27T10:58:19.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.760+0000 7f268a600640 1 -- 172.21.15.97:0/3856413991 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2698046070 con 0x55e0096aa420 2024-03-27T10:58:19.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.760+0000 7f268a600640 1 -- 172.21.15.97:0/3856413991 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2698034770 con 0x55e0096aa420 2024-03-27T10:58:19.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.761+0000 7f26a77f5740 1 -- 172.21.15.97:0/3856413991 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e0096aa420 msgr2=0x55e0096aeb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:19.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.761+0000 7f26a77f5740 1 --2- 172.21.15.97:0/3856413991 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e0096aa420 0x55e0096aeb30 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f2698009ca0 tx=0x7f269803f000 comp rx=0 tx=0).stop 2024-03-27T10:58:19.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.761+0000 7f26a77f5740 1 -- 172.21.15.97:0/3856413991 shutdown_connections 2024-03-27T10:58:19.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.761+0000 7f26a77f5740 1 --2- 172.21.15.97:0/3856413991 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e0096aa420 0x55e0096aeb30 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:19.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.761+0000 7f26a77f5740 1 --2- 172.21.15.97:0/3856413991 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e0096a52b0 0x55e0096b5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:19.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.761+0000 7f26a77f5740 1 --2- 172.21.15.97:0/3856413991 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e0096a48c0 0x55e0096ae5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:19.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.761+0000 7f26a77f5740 1 -- 172.21.15.97:0/3856413991 >> 172.21.15.97:0/3856413991 conn(0x55e00969fe30 msgr2=0x55e0096a0210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:19.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.761+0000 7f26a77f5740 1 -- 172.21.15.97:0/3856413991 shutdown_connections 2024-03-27T10:58:19.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.761+0000 7f26a77f5740 1 -- 172.21.15.97:0/3856413991 wait complete. 2024-03-27T10:58:19.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.761+0000 7f26a77f5740 1 -- start start 2024-03-27T10:58:19.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.761+0000 7f26a77f5740 1 Processor -- start 2024-03-27T10:58:19.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.762+0000 7f26a77f5740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e0096a48c0 0x55e0096b36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:19.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.762+0000 7f26a77f5740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e0096a52b0 0x55e0096b3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:19.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.762+0000 7f26a77f5740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e0096aa420 0x55e0096b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:19.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.762+0000 7f26a77f5740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f26a01aa320 con 0x55e0096aa420 2024-03-27T10:58:19.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.762+0000 7f26a77f5740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f26a01a9c50 con 0x55e0096a48c0 2024-03-27T10:58:19.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.762+0000 7f26a77f5740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f26a01a6e90 con 0x55e0096a52b0 2024-03-27T10:58:19.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.762+0000 7f26a77f5740 10 client.0 _ll_register_callbacks cb 0x7f26900cff50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:58:19.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.762+0000 7f269e800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e0096aa420 0x55e0096b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:19.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.762+0000 7f269e800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e0096aa420 0x55e0096b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53168/0 (socket says 172.21.15.97:53168) 2024-03-27T10:58:19.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.762+0000 7f269e800640 1 -- 172.21.15.97:0/305257051 learned_addr learned my addr 172.21.15.97:0/305257051 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:58:19.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.762+0000 7f269de00640 1 --2- 172.21.15.97:0/305257051 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e0096a48c0 0x55e0096b36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:19.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.762+0000 7f269d400640 1 --2- 172.21.15.97:0/305257051 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e0096a52b0 0x55e0096b3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:19.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.762+0000 7f269e800640 1 -- 172.21.15.97:0/305257051 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e0096a52b0 msgr2=0x55e0096b3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:19.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.762+0000 7f269e800640 1 --2- 172.21.15.97:0/305257051 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e0096a52b0 0x55e0096b3be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:19.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.762+0000 7f269e800640 1 -- 172.21.15.97:0/305257051 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e0096a48c0 msgr2=0x55e0096b36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:19.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.762+0000 7f269e800640 1 --2- 172.21.15.97:0/305257051 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e0096a48c0 0x55e0096b36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:19.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.762+0000 7f269e800640 1 -- 172.21.15.97:0/305257051 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0096b51e0 con 0x55e0096aa420 2024-03-27T10:58:19.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.763+0000 7f269e800640 1 --2- 172.21.15.97:0/305257051 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e0096aa420 0x55e0096b4bf0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f2698032ae0 tx=0x7f2698003d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e7fb09f284b0d8b server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:19.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.763+0000 7f267f400640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T10:58:19.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.763+0000 7f267f400640 1 -- 172.21.15.97:0/305257051 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2698058020 con 0x55e0096aa420 2024-03-27T10:58:19.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.763+0000 7f267f400640 1 -- 172.21.15.97:0/305257051 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f2698004450 con 0x55e0096aa420 2024-03-27T10:58:19.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.763+0000 7f26a77f5740 1 -- 172.21.15.97:0/305257051 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2698046070 con 0x55e0096aa420 2024-03-27T10:58:19.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.763+0000 7f267f400640 1 -- 172.21.15.97:0/305257051 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2698039030 con 0x55e0096aa420 2024-03-27T10:58:19.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.763+0000 7f26a77f5740 10 client.16107 fetch_fsmap learned FSMap version 29 2024-03-27T10:58:19.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.763+0000 7f26a77f5740 1 -- 172.21.15.97:0/305257051 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e009720800 con 0x55e0096aa420 2024-03-27T10:58:19.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.763+0000 7f267f400640 1 -- 172.21.15.97:0/305257051 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2698046070 con 0x55e0096aa420 2024-03-27T10:58:19.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.763+0000 7f267f400640 1 -- 172.21.15.97:0/305257051 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f269804b070 con 0x55e0096aa420 2024-03-27T10:58:19.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.764+0000 7f26a77f5740 10 client.16107 fetch_fsmap finished waiting for FSMap version 29 2024-03-27T10:58:19.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.764+0000 7f26a77f5740 10 client.16107 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-03-27T10:58:19.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.764+0000 7f26a77f5740 20 client.16107 populate_metadata read hostname 'smithi097' 2024-03-27T10:58:19.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.764+0000 7f26a77f5740 1 --2- 172.21.15.97:0/305257051 >> [v2:172.21.15.177:6834/3349112132,v1:172.21.15.177:6835/3349112132] conn(0x55e0097211d0 0x55e0097235f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:19.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.764+0000 7f26a77f5740 4 client.16107 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-27T10:58:19.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.764+0000 7f26a77f5740 1 -- 172.21.15.97:0/305257051 --> [v2:172.21.15.177:6834/3349112132,v1:172.21.15.177:6835/3349112132] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e009610f00 con 0x55e0097211d0 2024-03-27T10:58:19.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.764+0000 7f269de00640 1 --2- 172.21.15.97:0/305257051 >> [v2:172.21.15.177:6834/3349112132,v1:172.21.15.177:6835/3349112132] conn(0x55e0097211d0 0x55e0097235f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:19.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.765+0000 7f269de00640 1 --2- 172.21.15.97:0/305257051 >> [v2:172.21.15.177:6834/3349112132,v1:172.21.15.177:6835/3349112132] conn(0x55e0097211d0 0x55e0097235f0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=756212c81df78676 server_cookie=5178389a11b884ae in_seq=0 out_seq=0 2024-03-27T10:58:19.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.765+0000 7f267f400640 10 client.16107 ms_handle_connect on v2:172.21.15.177:6834/3349112132 2024-03-27T10:58:19.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.765+0000 7f267f400640 1 -- 172.21.15.97:0/305257051 <== mds.3 v2:172.21.15.177:6834/3349112132 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e009610f00 con 0x55e0097211d0 2024-03-27T10:58:19.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.765+0000 7f267f400640 10 client.16107 handle_command_reply: tid=0 2024-03-27T10:58:19.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.765+0000 7f26a77f5740 1 client.16107 shutdown 2024-03-27T10:58:19.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.765+0000 7f26a77f5740 1 -- 172.21.15.97:0/305257051 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e0096aa420 msgr2=0x55e0096b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:19.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.765+0000 7f26a77f5740 1 --2- 172.21.15.97:0/305257051 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e0096aa420 0x55e0096b4bf0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f2698032ae0 tx=0x7f2698003d20 comp rx=0 tx=0).stop 2024-03-27T10:58:19.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.765+0000 7f26a77f5740 1 -- 172.21.15.97:0/305257051 shutdown_connections 2024-03-27T10:58:19.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.765+0000 7f26a77f5740 1 --2- 172.21.15.97:0/305257051 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e0096aa420 0x55e0096b4bf0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:19.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.765+0000 7f26a77f5740 1 --2- 172.21.15.97:0/305257051 >> [v2:172.21.15.177:6834/3349112132,v1:172.21.15.177:6835/3349112132] conn(0x55e0097211d0 0x55e0097235f0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:19.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.765+0000 7f26a77f5740 1 --2- 172.21.15.97:0/305257051 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e0096a52b0 0x55e0096b3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:19.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.765+0000 7f26a77f5740 1 --2- 172.21.15.97:0/305257051 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e0096a48c0 0x55e0096b36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:19.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.765+0000 7f267f400640 0 client.16107 ms_handle_reset on v2:172.21.15.177:6834/3349112132 2024-03-27T10:58:19.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.765+0000 7f26a77f5740 1 -- 172.21.15.97:0/305257051 >> 172.21.15.97:0/305257051 conn(0x55e00969fe30 msgr2=0x55e0096ae560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:19.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.766+0000 7f26a77f5740 1 -- 172.21.15.97:0/305257051 shutdown_connections 2024-03-27T10:58:19.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.766+0000 7f26a77f5740 1 -- 172.21.15.97:0/305257051 wait complete. 2024-03-27T10:58:19.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.766+0000 7f26a77f5740 20 client.16107 trim_cache size 0 max 16384 2024-03-27T10:58:19.742 INFO:teuthology.orchestra.run.smithi097.stdout:[] 2024-03-27T10:58:19.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.766+0000 7f26a4a00640 1 -- 172.21.15.97:0/983816129 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f26840a3610 msgr2=0x7f26840a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:19.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.766+0000 7f26a4a00640 1 --2- 172.21.15.97:0/983816129 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f26840a3610 0x7f26840a5ad0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f269802c700 tx=0x7f2698032510 comp rx=0 tx=0).stop 2024-03-27T10:58:19.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.766+0000 7f26a4a00640 1 -- 172.21.15.97:0/983816129 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26a01025e0 msgr2=0x7f26a0197cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:19.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.766+0000 7f26a4a00640 1 --2- 172.21.15.97:0/983816129 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26a01025e0 0x7f26a0197cf0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f269000c890 tx=0x7f269000cd50 comp rx=0 tx=0).stop 2024-03-27T10:58:19.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.766+0000 7f26a4a00640 1 -- 172.21.15.97:0/983816129 shutdown_connections 2024-03-27T10:58:19.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.766+0000 7f26a4a00640 1 --2- 172.21.15.97:0/983816129 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f26a0107840 0x7f26a019f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:19.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.766+0000 7f26a4a00640 1 --2- 172.21.15.97:0/983816129 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f26840a3610 0x7f26840a5ad0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:19.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.766+0000 7f26a4a00640 1 --2- 172.21.15.97:0/983816129 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f26a0104f10 0x7f26a0198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:19.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.766+0000 7f26a4a00640 1 --2- 172.21.15.97:0/983816129 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26a01025e0 0x7f26a0197cf0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:19.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.766+0000 7f26a4a00640 1 -- 172.21.15.97:0/983816129 >> 172.21.15.97:0/983816129 conn(0x7f26a00fbff0 msgr2=0x7f26a0102f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:19.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.766+0000 7f26a4a00640 1 -- 172.21.15.97:0/983816129 shutdown_connections 2024-03-27T10:58:19.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:19.767+0000 7f26a4a00640 1 -- 172.21.15.97:0/983816129 wait complete. 2024-03-27T10:58:19.753 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-27T10:58:20.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.046+0000 7f6bc0c00640 1 -- 172.21.15.97:0/2452397955 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6bbc073190 msgr2=0x7f6bbc073610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:20.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.046+0000 7f6bc0c00640 1 --2- 172.21.15.97:0/2452397955 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6bbc073190 0x7f6bbc073610 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f6bb4009ca0 tx=0x7f6bb402c0f0 comp rx=0 tx=0).stop 2024-03-27T10:58:20.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.046+0000 7f6bc0c00640 1 -- 172.21.15.97:0/2452397955 shutdown_connections 2024-03-27T10:58:20.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.046+0000 7f6bc0c00640 1 --2- 172.21.15.97:0/2452397955 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6bbc10ff20 0x7f6bbc112370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:20.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.046+0000 7f6bc0c00640 1 --2- 172.21.15.97:0/2452397955 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6bbc073190 0x7f6bbc073610 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:20.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.046+0000 7f6bc0c00640 1 --2- 172.21.15.97:0/2452397955 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6bbc1097f0 0x7f6bbc072bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:20.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.046+0000 7f6bc0c00640 1 -- 172.21.15.97:0/2452397955 >> 172.21.15.97:0/2452397955 conn(0x7f6bbc06ea30 msgr2=0x7f6bbc070e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:20.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.046+0000 7f6bc0c00640 1 -- 172.21.15.97:0/2452397955 shutdown_connections 2024-03-27T10:58:20.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.046+0000 7f6bc0c00640 1 -- 172.21.15.97:0/2452397955 wait complete. 2024-03-27T10:58:20.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.049+0000 7f6bc0c00640 1 Processor -- start 2024-03-27T10:58:20.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.049+0000 7f6bc0c00640 1 -- start start 2024-03-27T10:58:20.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.049+0000 7f6bc0c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6bbc073190 0x7f6bbc10b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:20.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.049+0000 7f6bc0c00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6bbc1097f0 0x7f6bbc109e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:20.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.050+0000 7f6bc0c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6bbc10ff20 0x7f6bbc10a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:20.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.050+0000 7f6bc0c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6bbc115dc0 con 0x7f6bbc073190 2024-03-27T10:58:20.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.050+0000 7f6bc0c00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f6bbc115c40 con 0x7f6bbc10ff20 2024-03-27T10:58:20.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.050+0000 7f6bc0c00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6bbc115f40 con 0x7f6bbc1097f0 2024-03-27T10:58:20.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.050+0000 7f6bbb400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6bbc073190 0x7f6bbc10b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:20.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.051+0000 7f6bbb400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6bbc073190 0x7f6bbc10b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53196/0 (socket says 172.21.15.97:53196) 2024-03-27T10:58:20.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.051+0000 7f6bbaa00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6bbc1097f0 0x7f6bbc109e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:20.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.051+0000 7f6bbaa00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6bbc1097f0 0x7f6bbc109e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.97:45384/0 (socket says 172.21.15.97:45384) 2024-03-27T10:58:20.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.051+0000 7f6bbaa00640 1 -- 172.21.15.97:0/1699159129 learned_addr learned my addr 172.21.15.97:0/1699159129 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:58:20.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.051+0000 7f6bbbe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6bbc10ff20 0x7f6bbc10a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:20.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.052+0000 7f6bbb400640 1 -- 172.21.15.97:0/1699159129 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6bbc1097f0 msgr2=0x7f6bbc109e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:20.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.052+0000 7f6bbb400640 1 --2- 172.21.15.97:0/1699159129 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6bbc1097f0 0x7f6bbc109e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:20.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.052+0000 7f6bbb400640 1 -- 172.21.15.97:0/1699159129 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6bbc10ff20 msgr2=0x7f6bbc10a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:20.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.052+0000 7f6bbb400640 1 --2- 172.21.15.97:0/1699159129 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6bbc10ff20 0x7f6bbc10a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:20.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.052+0000 7f6bbb400640 1 -- 172.21.15.97:0/1699159129 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bbc1a4450 con 0x7f6bbc073190 2024-03-27T10:58:20.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.052+0000 7f6bbbe00640 1 --2- 172.21.15.97:0/1699159129 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6bbc10ff20 0x7f6bbc10a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T10:58:20.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.053+0000 7f6bbb400640 1 --2- 172.21.15.97:0/1699159129 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6bbc073190 0x7f6bbc10b7e0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f6ba800e9c0 tx=0x7f6ba800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:20.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.053+0000 7f6bb3e00640 1 -- 172.21.15.97:0/1699159129 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba800ccc0 con 0x7f6bbc073190 2024-03-27T10:58:20.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.053+0000 7f6bb3e00640 1 -- 172.21.15.97:0/1699159129 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6ba80035f0 con 0x7f6bbc073190 2024-03-27T10:58:20.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.053+0000 7f6bb3e00640 1 -- 172.21.15.97:0/1699159129 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba800ce60 con 0x7f6bbc073190 2024-03-27T10:58:20.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.053+0000 7f6bc0c00640 1 -- 172.21.15.97:0/1699159129 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bbc1a4740 con 0x7f6bbc073190 2024-03-27T10:58:20.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.053+0000 7f6bc0c00640 1 -- 172.21.15.97:0/1699159129 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bbc1a4c40 con 0x7f6bbc073190 2024-03-27T10:58:20.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.056+0000 7f6bc4f6a740 1 Processor -- start 2024-03-27T10:58:20.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.056+0000 7f6bc4f6a740 1 -- start start 2024-03-27T10:58:20.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.056+0000 7f6bc4f6a740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560834152ab0 0x560834152ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:20.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.056+0000 7f6bc4f6a740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5608341534a0 0x560834157fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:20.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.056+0000 7f6bc4f6a740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560834158610 0x56083415aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:20.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.056+0000 7f6bc4f6a740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56083411a7b0 con 0x560834158610 2024-03-27T10:58:20.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.056+0000 7f6bc4f6a740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5608340b7670 con 0x560834152ab0 2024-03-27T10:58:20.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.056+0000 7f6bc4f6a740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x56083405f970 con 0x5608341534a0 2024-03-27T10:58:20.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.058+0000 7f6bbbe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560834152ab0 0x560834152ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:20.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.058+0000 7f6bbbe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560834152ab0 0x560834152ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:52470/0 (socket says 172.21.15.97:52470) 2024-03-27T10:58:20.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.058+0000 7f6bbbe00640 1 -- 172.21.15.97:0/1263973784 learned_addr learned my addr 172.21.15.97:0/1263973784 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:58:20.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.058+0000 7f6bbbe00640 1 -- 172.21.15.97:0/1263973784 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5608341534a0 msgr2=0x560834157fa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-27T10:58:20.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.058+0000 7f6bbbe00640 1 --2- 172.21.15.97:0/1263973784 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5608341534a0 0x560834157fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:20.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.058+0000 7f6bbbe00640 1 -- 172.21.15.97:0/1263973784 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560834158610 msgr2=0x56083415aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:20.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.058+0000 7f6bbbe00640 1 --2- 172.21.15.97:0/1263973784 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560834158610 0x56083415aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:20.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.058+0000 7f6bbbe00640 1 -- 172.21.15.97:0/1263973784 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560833dcb3f0 con 0x560834152ab0 2024-03-27T10:58:20.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.058+0000 7f6bbbe00640 1 --2- 172.21.15.97:0/1263973784 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560834152ab0 0x560834152ed0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6bac010400 tx=0x7f6bac0108c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=afb3c254c66a6cfd server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:20.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.059+0000 7f6ba7e00640 1 -- 172.21.15.97:0/1263973784 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bac011700 con 0x560834152ab0 2024-03-27T10:58:20.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.059+0000 7f6ba7e00640 1 -- 172.21.15.97:0/1263973784 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6bac011d50 con 0x560834152ab0 2024-03-27T10:58:20.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.059+0000 7f6ba7e00640 1 -- 172.21.15.97:0/1263973784 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bac01a380 con 0x560834152ab0 2024-03-27T10:58:20.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.059+0000 7f6bb3e00640 1 -- 172.21.15.97:0/1699159129 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f6ba80037a0 con 0x7f6bbc073190 2024-03-27T10:58:20.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.060+0000 7f6bc4f6a740 1 -- 172.21.15.97:0/1263973784 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560834152ab0 msgr2=0x560834152ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:20.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.060+0000 7f6bc4f6a740 1 --2- 172.21.15.97:0/1263973784 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560834152ab0 0x560834152ed0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6bac010400 tx=0x7f6bac0108c0 comp rx=0 tx=0).stop 2024-03-27T10:58:20.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.060+0000 7f6bc4f6a740 1 -- 172.21.15.97:0/1263973784 shutdown_connections 2024-03-27T10:58:20.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.060+0000 7f6bc4f6a740 1 --2- 172.21.15.97:0/1263973784 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560834158610 0x56083415aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:20.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.060+0000 7f6bc4f6a740 1 --2- 172.21.15.97:0/1263973784 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5608341534a0 0x560834157fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:20.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.060+0000 7f6bc4f6a740 1 --2- 172.21.15.97:0/1263973784 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560834152ab0 0x560834152ed0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:20.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.060+0000 7f6bc4f6a740 1 -- 172.21.15.97:0/1263973784 >> 172.21.15.97:0/1263973784 conn(0x56083414e020 msgr2=0x56083414e420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:20.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.060+0000 7f6bc4f6a740 1 -- 172.21.15.97:0/1263973784 shutdown_connections 2024-03-27T10:58:20.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.061+0000 7f6bc4f6a740 1 -- 172.21.15.97:0/1263973784 wait complete. 2024-03-27T10:58:20.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.061+0000 7f6bc4f6a740 1 -- start start 2024-03-27T10:58:20.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.061+0000 7f6bc4f6a740 1 Processor -- start 2024-03-27T10:58:20.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.062+0000 7f6bc4f6a740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560834152ab0 0x56083415ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:20.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.062+0000 7f6bc4f6a740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5608341534a0 0x56083415d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:20.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.062+0000 7f6bc4f6a740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560834158610 0x560834164230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:20.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.062+0000 7f6bc4f6a740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x560834072ca0 con 0x5608341534a0 2024-03-27T10:58:20.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.062+0000 7f6bc4f6a740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5608340b3620 con 0x560834152ab0 2024-03-27T10:58:20.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.062+0000 7f6bc4f6a740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x56083415b5c0 con 0x560834158610 2024-03-27T10:58:20.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.062+0000 7f6bc4f6a740 10 client.0 _ll_register_callbacks cb 0x5608340af160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:58:20.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.062+0000 7f6bbbe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560834152ab0 0x56083415ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:20.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.062+0000 7f6bb3e00640 1 --2- 172.21.15.97:0/1699159129 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f6ba00a35f0 0x7f6ba00a5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:20.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.064+0000 7f6bbb400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5608341534a0 0x56083415d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:20.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.064+0000 7f6bbb400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5608341534a0 0x56083415d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53200/0 (socket says 172.21.15.97:53200) 2024-03-27T10:58:20.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.064+0000 7f6bbb400640 1 -- 172.21.15.97:0/1561050583 learned_addr learned my addr 172.21.15.97:0/1561050583 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:58:20.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.064+0000 7f6bbaa00640 1 --2- 172.21.15.97:0/1561050583 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560834158610 0x560834164230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:20.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.064+0000 7f6bbbe00640 1 --2- 172.21.15.97:0/1699159129 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f6ba00a35f0 0x7f6ba00a5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:20.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.064+0000 7f6bb3e00640 1 -- 172.21.15.97:0/1699159129 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f6ba80d0110 con 0x7f6bbc073190 2024-03-27T10:58:20.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.064+0000 7f6bbaa00640 1 -- 172.21.15.97:0/1561050583 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560834152ab0 msgr2=0x56083415ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:20.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.064+0000 7f6bbaa00640 1 --2- 172.21.15.97:0/1561050583 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560834152ab0 0x56083415ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:20.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.064+0000 7f6bbaa00640 1 -- 172.21.15.97:0/1561050583 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5608341534a0 msgr2=0x56083415d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:20.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.064+0000 7f6bbaa00640 1 --2- 172.21.15.97:0/1561050583 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5608341534a0 0x56083415d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:20.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.064+0000 7f6bbaa00640 1 -- 172.21.15.97:0/1561050583 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560833e957d0 con 0x560834158610 2024-03-27T10:58:20.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.064+0000 7f6bbb400640 1 --2- 172.21.15.97:0/1561050583 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5608341534a0 0x56083415d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T10:58:20.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.065+0000 7f6bbbe00640 1 --2- 172.21.15.97:0/1699159129 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f6ba00a35f0 0x7f6ba00a5ab0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f6bac011010 tx=0x7f6bac01c880 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:20.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.065+0000 7f6bbaa00640 1 --2- 172.21.15.97:0/1561050583 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560834158610 0x560834164230 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6bb4000f80 tx=0x7f6bb4009650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d337f94a32c25689 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:20.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.065+0000 7f6ba4c00640 10 client.0 ms_handle_connect on v2:172.21.15.177:3300/0 2024-03-27T10:58:20.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.065+0000 7f6ba4c00640 1 -- 172.21.15.97:0/1561050583 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb4043070 con 0x560834158610 2024-03-27T10:58:20.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.065+0000 7f6bc4f6a740 1 -- 172.21.15.97:0/1561050583 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6bac011d50 con 0x560834158610 2024-03-27T10:58:20.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.067+0000 7f6ba4c00640 1 -- 172.21.15.97:0/1561050583 <== mon.1 v2:172.21.15.177:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f6bb4003970 con 0x560834158610 2024-03-27T10:58:20.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.069+0000 7f6ba4c00640 1 -- 172.21.15.97:0/1561050583 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6bb404f050 con 0x560834158610 2024-03-27T10:58:20.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.069+0000 7f6bc4f6a740 10 client.25519 fetch_fsmap learned FSMap version 29 2024-03-27T10:58:20.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.069+0000 7f6bc4f6a740 1 -- 172.21.15.97:0/1561050583 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560834167660 con 0x560834158610 2024-03-27T10:58:20.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.070+0000 7f6ba4c00640 1 -- 172.21.15.97:0/1561050583 <== mon.1 v2:172.21.15.177:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6bac011d50 con 0x560834158610 2024-03-27T10:58:20.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.070+0000 7f6ba4c00640 1 -- 172.21.15.97:0/1561050583 <== mon.1 v2:172.21.15.177:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f6bb403f440 con 0x560834158610 2024-03-27T10:58:20.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.070+0000 7f6bc4f6a740 10 client.25519 fetch_fsmap finished waiting for FSMap version 29 2024-03-27T10:58:20.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.070+0000 7f6bc4f6a740 10 client.25519 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-27T10:58:20.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.070+0000 7f6bc4f6a740 20 client.25519 populate_metadata read hostname 'smithi097' 2024-03-27T10:58:20.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.070+0000 7f6bc4f6a740 1 --2- 172.21.15.97:0/1561050583 >> [v2:172.21.15.114:6834/3581918222,v1:172.21.15.114:6835/3581918222] conn(0x560834168c40 0x56083416b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:20.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.070+0000 7f6bc4f6a740 4 client.25519 mds_command: new command op to 34445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T10:58:20.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.070+0000 7f6bc4f6a740 1 -- 172.21.15.97:0/1561050583 --> [v2:172.21.15.114:6834/3581918222,v1:172.21.15.114:6835/3581918222] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5608340c4bb0 con 0x560834168c40 2024-03-27T10:58:20.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.071+0000 7f6bbb400640 1 --2- 172.21.15.97:0/1561050583 >> [v2:172.21.15.114:6834/3581918222,v1:172.21.15.114:6835/3581918222] conn(0x560834168c40 0x56083416b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:20.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.071+0000 7f6bbb400640 1 --2- 172.21.15.97:0/1561050583 >> [v2:172.21.15.114:6834/3581918222,v1:172.21.15.114:6835/3581918222] conn(0x560834168c40 0x56083416b030 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=63fe178eed826980 server_cookie=17ad5123889ef8b3 in_seq=0 out_seq=0 2024-03-27T10:58:20.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.071+0000 7f6ba4c00640 10 client.25519 ms_handle_connect on v2:172.21.15.114:6834/3581918222 2024-03-27T10:58:20.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.073+0000 7f6ba4c00640 1 -- 172.21.15.97:0/1561050583 <== mds.2 v2:172.21.15.114:6834/3581918222 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5608340c4bb0 con 0x560834168c40 2024-03-27T10:58:20.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.073+0000 7f6ba4c00640 10 client.25519 handle_command_reply: tid=0 2024-03-27T10:58:20.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.073+0000 7f6bc4f6a740 1 client.25519 shutdown 2024-03-27T10:58:20.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.073+0000 7f6bc4f6a740 1 -- 172.21.15.97:0/1561050583 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560834158610 msgr2=0x560834164230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:20.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.073+0000 7f6bc4f6a740 1 --2- 172.21.15.97:0/1561050583 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560834158610 0x560834164230 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6bb4000f80 tx=0x7f6bb4009650 comp rx=0 tx=0).stop 2024-03-27T10:58:20.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.073+0000 7f6bc4f6a740 1 -- 172.21.15.97:0/1561050583 shutdown_connections 2024-03-27T10:58:20.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.073+0000 7f6bc4f6a740 1 --2- 172.21.15.97:0/1561050583 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560834158610 0x560834164230 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:20.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.073+0000 7f6bc4f6a740 1 --2- 172.21.15.97:0/1561050583 >> [v2:172.21.15.114:6834/3581918222,v1:172.21.15.114:6835/3581918222] conn(0x560834168c40 0x56083416b030 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:20.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.073+0000 7f6bc4f6a740 1 --2- 172.21.15.97:0/1561050583 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5608341534a0 0x56083415d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:20.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.073+0000 7f6bc4f6a740 1 --2- 172.21.15.97:0/1561050583 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560834152ab0 0x56083415ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:20.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.073+0000 7f6bc4f6a740 1 -- 172.21.15.97:0/1561050583 >> 172.21.15.97:0/1561050583 conn(0x56083414e020 msgr2=0x56083414d240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:20.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.073+0000 7f6ba4c00640 0 client.25519 ms_handle_reset on v2:172.21.15.114:6834/3581918222 2024-03-27T10:58:20.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.073+0000 7f6bc4f6a740 1 -- 172.21.15.97:0/1561050583 shutdown_connections 2024-03-27T10:58:20.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.073+0000 7f6bc4f6a740 1 -- 172.21.15.97:0/1561050583 wait complete. 2024-03-27T10:58:20.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.073+0000 7f6bc4f6a740 20 client.25519 trim_cache size 0 max 16384 2024-03-27T10:58:20.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.098+0000 7f6bc4f6a740 1 Processor -- start 2024-03-27T10:58:20.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.098+0000 7f6bc4f6a740 1 -- start start 2024-03-27T10:58:20.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.098+0000 7f6bc4f6a740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560834152ab0 0x56083415ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:20.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.098+0000 7f6bc4f6a740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5608341534a0 0x5608341641f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:20.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.098+0000 7f6bc4f6a740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560834158610 0x56083415d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:20.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.099+0000 7f6bc4f6a740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6bbc1c25b0 con 0x560834158610 2024-03-27T10:58:20.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.099+0000 7f6bc4f6a740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f6bbc1c0370 con 0x5608341534a0 2024-03-27T10:58:20.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.099+0000 7f6bc4f6a740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6bbc1c1ee0 con 0x560834152ab0 2024-03-27T10:58:20.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.099+0000 7f6bbbe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560834158610 0x56083415d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:20.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.099+0000 7f6bbbe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560834158610 0x56083415d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53208/0 (socket says 172.21.15.97:53208) 2024-03-27T10:58:20.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.099+0000 7f6bbbe00640 1 -- 172.21.15.97:0/4037810723 learned_addr learned my addr 172.21.15.97:0/4037810723 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:58:20.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.099+0000 7f6bbb400640 1 --2- 172.21.15.97:0/4037810723 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560834152ab0 0x56083415ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:20.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.099+0000 7f6bbaa00640 1 --2- 172.21.15.97:0/4037810723 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5608341534a0 0x5608341641f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:20.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.100+0000 7f6bbb400640 1 -- 172.21.15.97:0/4037810723 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5608341534a0 msgr2=0x5608341641f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:20.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.100+0000 7f6bbb400640 1 --2- 172.21.15.97:0/4037810723 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5608341534a0 0x5608341641f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:20.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.100+0000 7f6bbb400640 1 -- 172.21.15.97:0/4037810723 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560834158610 msgr2=0x56083415d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:20.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.100+0000 7f6bbb400640 1 --2- 172.21.15.97:0/4037810723 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560834158610 0x56083415d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:20.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.100+0000 7f6bbb400640 1 -- 172.21.15.97:0/4037810723 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56083415b6d0 con 0x560834152ab0 2024-03-27T10:58:20.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.100+0000 7f6bbbe00640 1 --2- 172.21.15.97:0/4037810723 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560834158610 0x56083415d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T10:58:20.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.100+0000 7f6bbb400640 1 --2- 172.21.15.97:0/4037810723 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560834152ab0 0x56083415ccc0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6ba8098f60 tx=0x7f6ba8003d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bdded33c6883d4f0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:20.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.100+0000 7f6ba7400640 1 -- 172.21.15.97:0/4037810723 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba80a0940 con 0x560834152ab0 2024-03-27T10:58:20.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.100+0000 7f6ba7400640 1 -- 172.21.15.97:0/4037810723 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6ba80a5200 con 0x560834152ab0 2024-03-27T10:58:20.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.100+0000 7f6ba7400640 1 -- 172.21.15.97:0/4037810723 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba8085b20 con 0x560834152ab0 2024-03-27T10:58:20.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.101+0000 7f6bc4f6a740 1 -- 172.21.15.97:0/4037810723 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560834152ab0 msgr2=0x56083415ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:20.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.101+0000 7f6bc4f6a740 1 --2- 172.21.15.97:0/4037810723 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560834152ab0 0x56083415ccc0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6ba8098f60 tx=0x7f6ba8003d00 comp rx=0 tx=0).stop 2024-03-27T10:58:20.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.101+0000 7f6bc4f6a740 1 -- 172.21.15.97:0/4037810723 shutdown_connections 2024-03-27T10:58:20.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.102+0000 7f6bc4f6a740 1 --2- 172.21.15.97:0/4037810723 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560834158610 0x56083415d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:20.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.102+0000 7f6bc4f6a740 1 --2- 172.21.15.97:0/4037810723 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5608341534a0 0x5608341641f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:20.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.102+0000 7f6bc4f6a740 1 --2- 172.21.15.97:0/4037810723 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560834152ab0 0x56083415ccc0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:20.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.102+0000 7f6bc4f6a740 1 -- 172.21.15.97:0/4037810723 >> 172.21.15.97:0/4037810723 conn(0x56083414e020 msgr2=0x560834154800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:20.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.102+0000 7f6bc4f6a740 1 -- 172.21.15.97:0/4037810723 shutdown_connections 2024-03-27T10:58:20.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.102+0000 7f6bc4f6a740 1 -- 172.21.15.97:0/4037810723 wait complete. 2024-03-27T10:58:20.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.102+0000 7f6bc4f6a740 1 -- start start 2024-03-27T10:58:20.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.102+0000 7f6bc4f6a740 1 Processor -- start 2024-03-27T10:58:20.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.103+0000 7f6bc4f6a740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560834152ab0 0x560834161390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:20.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.103+0000 7f6bc4f6a740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5608341534a0 0x5608341618d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:20.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.103+0000 7f6bc4f6a740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560834158610 0x560834162890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:20.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.103+0000 7f6bc4f6a740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6bbc1c1110 con 0x560834152ab0 2024-03-27T10:58:20.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.103+0000 7f6bc4f6a740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f6bbc1c0a40 con 0x560834158610 2024-03-27T10:58:20.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.103+0000 7f6bc4f6a740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6bbc1c17e0 con 0x5608341534a0 2024-03-27T10:58:20.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.103+0000 7f6bc4f6a740 10 client.0 _ll_register_callbacks cb 0x7f6ba8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:58:20.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.103+0000 7f6bbb400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560834152ab0 0x560834161390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:20.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.103+0000 7f6bbb400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560834152ab0 0x560834161390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53212/0 (socket says 172.21.15.97:53212) 2024-03-27T10:58:20.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.103+0000 7f6bbb400640 1 -- 172.21.15.97:0/2631290053 learned_addr learned my addr 172.21.15.97:0/2631290053 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:58:20.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.104+0000 7f6bbaa00640 1 --2- 172.21.15.97:0/2631290053 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5608341534a0 0x5608341618d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:20.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.104+0000 7f6bbbe00640 1 --2- 172.21.15.97:0/2631290053 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560834158610 0x560834162890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:20.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.104+0000 7f6bbb400640 1 -- 172.21.15.97:0/2631290053 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5608341534a0 msgr2=0x5608341618d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:20.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.104+0000 7f6bbb400640 1 --2- 172.21.15.97:0/2631290053 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5608341534a0 0x5608341618d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:20.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.104+0000 7f6bbb400640 1 -- 172.21.15.97:0/2631290053 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560834158610 msgr2=0x560834162890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:20.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.104+0000 7f6bbb400640 1 --2- 172.21.15.97:0/2631290053 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560834158610 0x560834162890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:20.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.104+0000 7f6bbb400640 1 -- 172.21.15.97:0/2631290053 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560834162e80 con 0x560834152ab0 2024-03-27T10:58:20.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.104+0000 7f6bbb400640 1 --2- 172.21.15.97:0/2631290053 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560834152ab0 0x560834161390 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f6ba809f040 tx=0x7f6ba8005b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=199d00acbebb14ef server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:20.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.104+0000 7f6ba4c00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T10:58:20.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.105+0000 7f6ba4c00640 1 -- 172.21.15.97:0/2631290053 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba80b9070 con 0x560834152ab0 2024-03-27T10:58:20.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.105+0000 7f6ba4c00640 1 -- 172.21.15.97:0/2631290053 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f6ba809bbe0 con 0x560834152ab0 2024-03-27T10:58:20.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.106+0000 7f6bc4f6a740 1 -- 172.21.15.97:0/2631290053 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ba80a5200 con 0x560834152ab0 2024-03-27T10:58:20.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.106+0000 7f6ba4c00640 1 -- 172.21.15.97:0/2631290053 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6ba80bb310 con 0x560834152ab0 2024-03-27T10:58:20.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.106+0000 7f6bc4f6a740 10 client.16122 fetch_fsmap learned FSMap version 29 2024-03-27T10:58:20.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.106+0000 7f6bc4f6a740 1 -- 172.21.15.97:0/2631290053 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560834164180 con 0x560834152ab0 2024-03-27T10:58:20.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.107+0000 7f6ba4c00640 1 -- 172.21.15.97:0/2631290053 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6ba80a5200 con 0x560834152ab0 2024-03-27T10:58:20.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.107+0000 7f6ba4c00640 1 -- 172.21.15.97:0/2631290053 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f6ba80983e0 con 0x560834152ab0 2024-03-27T10:58:20.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.108+0000 7f6bc4f6a740 10 client.16122 fetch_fsmap finished waiting for FSMap version 29 2024-03-27T10:58:20.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.108+0000 7f6bc4f6a740 10 client.16122 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-27T10:58:20.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.108+0000 7f6bc4f6a740 20 client.16122 populate_metadata read hostname 'smithi097' 2024-03-27T10:58:20.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.108+0000 7f6bc4f6a740 1 --2- 172.21.15.97:0/2631290053 >> [v2:172.21.15.114:6834/3581918222,v1:172.21.15.114:6835/3581918222] conn(0x5608341cf2e0 0x5608341d1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:20.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.108+0000 7f6bc4f6a740 4 client.16122 mds_command: new command op to 34445 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-27T10:58:20.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.108+0000 7f6bc4f6a740 1 -- 172.21.15.97:0/2631290053 --> [v2:172.21.15.114:6834/3581918222,v1:172.21.15.114:6835/3581918222] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5608340bef00 con 0x5608341cf2e0 2024-03-27T10:58:20.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.108+0000 7f6bbaa00640 1 --2- 172.21.15.97:0/2631290053 >> [v2:172.21.15.114:6834/3581918222,v1:172.21.15.114:6835/3581918222] conn(0x5608341cf2e0 0x5608341d1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:20.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.109+0000 7f6bbaa00640 1 --2- 172.21.15.97:0/2631290053 >> [v2:172.21.15.114:6834/3581918222,v1:172.21.15.114:6835/3581918222] conn(0x5608341cf2e0 0x5608341d1700 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6a4ad9cb8717cc99 server_cookie=970dee1d4ae25384 in_seq=0 out_seq=0 2024-03-27T10:58:20.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.109+0000 7f6ba4c00640 10 client.16122 ms_handle_connect on v2:172.21.15.114:6834/3581918222 2024-03-27T10:58:20.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.110+0000 7f6ba4c00640 1 -- 172.21.15.97:0/2631290053 <== mds.2 v2:172.21.15.114:6834/3581918222 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5608340bef00 con 0x5608341cf2e0 2024-03-27T10:58:20.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.110+0000 7f6ba4c00640 10 client.16122 handle_command_reply: tid=0 2024-03-27T10:58:20.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.110+0000 7f6bc4f6a740 1 client.16122 shutdown 2024-03-27T10:58:20.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.110+0000 7f6bc4f6a740 1 -- 172.21.15.97:0/2631290053 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560834152ab0 msgr2=0x560834161390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:20.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.110+0000 7f6bc4f6a740 1 --2- 172.21.15.97:0/2631290053 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560834152ab0 0x560834161390 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f6ba809f040 tx=0x7f6ba8005b20 comp rx=0 tx=0).stop 2024-03-27T10:58:20.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.111+0000 7f6bc4f6a740 1 -- 172.21.15.97:0/2631290053 shutdown_connections 2024-03-27T10:58:20.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.111+0000 7f6bc4f6a740 1 --2- 172.21.15.97:0/2631290053 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560834158610 0x560834162890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:20.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.111+0000 7f6bc4f6a740 1 --2- 172.21.15.97:0/2631290053 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5608341534a0 0x5608341618d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:20.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.111+0000 7f6bc4f6a740 1 --2- 172.21.15.97:0/2631290053 >> [v2:172.21.15.114:6834/3581918222,v1:172.21.15.114:6835/3581918222] conn(0x5608341cf2e0 0x5608341d1700 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:20.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.111+0000 7f6bc4f6a740 1 --2- 172.21.15.97:0/2631290053 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560834152ab0 0x560834161390 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:20.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.111+0000 7f6bc4f6a740 1 -- 172.21.15.97:0/2631290053 >> 172.21.15.97:0/2631290053 conn(0x56083414e020 msgr2=0x56083415ccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:20.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.111+0000 7f6ba4c00640 0 client.16122 ms_handle_reset on v2:172.21.15.114:6834/3581918222 2024-03-27T10:58:20.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.112+0000 7f6bc4f6a740 1 -- 172.21.15.97:0/2631290053 shutdown_connections 2024-03-27T10:58:20.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.112+0000 7f6bc4f6a740 1 -- 172.21.15.97:0/2631290053 wait complete. 2024-03-27T10:58:20.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.112+0000 7f6bc4f6a740 20 client.16122 trim_cache size 0 max 16384 2024-03-27T10:58:20.084 INFO:teuthology.orchestra.run.smithi097.stdout:[] 2024-03-27T10:58:20.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.113+0000 7f6bc0c00640 1 -- 172.21.15.97:0/1699159129 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f6ba00a35f0 msgr2=0x7f6ba00a5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:20.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.113+0000 7f6bc0c00640 1 --2- 172.21.15.97:0/1699159129 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f6ba00a35f0 0x7f6ba00a5ab0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f6bac011010 tx=0x7f6bac01c880 comp rx=0 tx=0).stop 2024-03-27T10:58:20.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.113+0000 7f6bc0c00640 1 -- 172.21.15.97:0/1699159129 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6bbc073190 msgr2=0x7f6bbc10b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:20.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.113+0000 7f6bc0c00640 1 --2- 172.21.15.97:0/1699159129 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6bbc073190 0x7f6bbc10b7e0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f6ba800e9c0 tx=0x7f6ba800ee80 comp rx=0 tx=0).stop 2024-03-27T10:58:20.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.114+0000 7f6bc0c00640 1 -- 172.21.15.97:0/1699159129 shutdown_connections 2024-03-27T10:58:20.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.114+0000 7f6bc0c00640 1 --2- 172.21.15.97:0/1699159129 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f6ba00a35f0 0x7f6ba00a5ab0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:20.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.114+0000 7f6bc0c00640 1 --2- 172.21.15.97:0/1699159129 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6bbc10ff20 0x7f6bbc10a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:20.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.114+0000 7f6bc0c00640 1 --2- 172.21.15.97:0/1699159129 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6bbc1097f0 0x7f6bbc109e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:20.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.114+0000 7f6bc0c00640 1 --2- 172.21.15.97:0/1699159129 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6bbc073190 0x7f6bbc10b7e0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:20.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.114+0000 7f6bc0c00640 1 -- 172.21.15.97:0/1699159129 >> 172.21.15.97:0/1699159129 conn(0x7f6bbc06ea30 msgr2=0x7f6bbc10be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:20.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.114+0000 7f6bc0c00640 1 -- 172.21.15.97:0/1699159129 shutdown_connections 2024-03-27T10:58:20.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.114+0000 7f6bc0c00640 1 -- 172.21.15.97:0/1699159129 wait complete. 2024-03-27T10:58:20.105 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-27T10:58:20.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.324+0000 7f979b200640 1 -- 172.21.15.97:0/3851283323 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f979c102c40 msgr2=0x7f979c1030c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:20.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.324+0000 7f979b200640 1 --2- 172.21.15.97:0/3851283323 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f979c102c40 0x7f979c1030c0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f9788009860 tx=0x7f978802bb70 comp rx=0 tx=0).stop 2024-03-27T10:58:20.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.324+0000 7f979b200640 1 -- 172.21.15.97:0/3851283323 shutdown_connections 2024-03-27T10:58:20.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.324+0000 7f979b200640 1 --2- 172.21.15.97:0/3851283323 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f979c103600 0x7f979c109e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:20.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.324+0000 7f979b200640 1 --2- 172.21.15.97:0/3851283323 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f979c102c40 0x7f979c1030c0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:20.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.324+0000 7f979b200640 1 --2- 172.21.15.97:0/3851283323 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f979c101a50 0x7f979c101e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:20.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.325+0000 7f979b200640 1 -- 172.21.15.97:0/3851283323 >> 172.21.15.97:0/3851283323 conn(0x7f979c0fcfc0 msgr2=0x7f979c0ff400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:20.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.325+0000 7f979b200640 1 -- 172.21.15.97:0/3851283323 shutdown_connections 2024-03-27T10:58:20.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.325+0000 7f979b200640 1 -- 172.21.15.97:0/3851283323 wait complete. 2024-03-27T10:58:20.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.326+0000 7f979b200640 1 Processor -- start 2024-03-27T10:58:20.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.326+0000 7f979b200640 1 -- start start 2024-03-27T10:58:20.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.326+0000 7f979b200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f979c101a50 0x7f979c199ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:20.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.326+0000 7f979b200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f979c102c40 0x7f979c19a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:20.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.326+0000 7f979b200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f979c103600 0x7f979c1a1440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:20.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.326+0000 7f979b200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f979c10d580 con 0x7f979c102c40 2024-03-27T10:58:20.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.326+0000 7f979b200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f979c10d400 con 0x7f979c103600 2024-03-27T10:58:20.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.326+0000 7f979b200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f979c10d700 con 0x7f979c101a50 2024-03-27T10:58:20.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.326+0000 7f9799400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f979c102c40 0x7f979c19a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:20.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.326+0000 7f9799400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f979c102c40 0x7f979c19a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53242/0 (socket says 172.21.15.97:53242) 2024-03-27T10:58:20.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.326+0000 7f9799400640 1 -- 172.21.15.97:0/3925846378 learned_addr learned my addr 172.21.15.97:0/3925846378 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:58:20.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.327+0000 7f9799e00640 1 --2- 172.21.15.97:0/3925846378 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f979c101a50 0x7f979c199ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:20.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.327+0000 7f979a800640 1 --2- 172.21.15.97:0/3925846378 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f979c103600 0x7f979c1a1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:20.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.327+0000 7f9799400640 1 -- 172.21.15.97:0/3925846378 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f979c101a50 msgr2=0x7f979c199ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:20.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.327+0000 7f9799400640 1 --2- 172.21.15.97:0/3925846378 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f979c101a50 0x7f979c199ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:20.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.327+0000 7f9799400640 1 -- 172.21.15.97:0/3925846378 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f979c103600 msgr2=0x7f979c1a1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:20.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.327+0000 7f9799400640 1 --2- 172.21.15.97:0/3925846378 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f979c103600 0x7f979c1a1440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:20.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.327+0000 7f9799400640 1 -- 172.21.15.97:0/3925846378 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f979c1a1ab0 con 0x7f979c102c40 2024-03-27T10:58:20.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.327+0000 7f9799400640 1 --2- 172.21.15.97:0/3925846378 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f979c102c40 0x7f979c19a420 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f97880057c0 tx=0x7f97880042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:20.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.327+0000 7f9792a00640 1 -- 172.21.15.97:0/3925846378 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9788041030 con 0x7f979c102c40 2024-03-27T10:58:20.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.327+0000 7f9792a00640 1 -- 172.21.15.97:0/3925846378 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f978802c9c0 con 0x7f979c102c40 2024-03-27T10:58:20.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.327+0000 7f979b200640 1 -- 172.21.15.97:0/3925846378 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f979c1a1d40 con 0x7f979c102c40 2024-03-27T10:58:20.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.327+0000 7f9792a00640 1 -- 172.21.15.97:0/3925846378 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9788039040 con 0x7f979c102c40 2024-03-27T10:58:20.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.327+0000 7f979b200640 1 -- 172.21.15.97:0/3925846378 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f979c1a21d0 con 0x7f979c102c40 2024-03-27T10:58:20.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.331+0000 7f9792a00640 1 -- 172.21.15.97:0/3925846378 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f9788044070 con 0x7f979c102c40 2024-03-27T10:58:20.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.331+0000 7f97a3de5740 1 Processor -- start 2024-03-27T10:58:20.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.331+0000 7f97a3de5740 1 -- start start 2024-03-27T10:58:20.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.331+0000 7f97a3de5740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557d6dcb6ab0 0x557d6dcb6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:20.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.331+0000 7f97a3de5740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557d6dcb74a0 0x557d6dcbbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:20.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.331+0000 7f97a3de5740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557d6dcbc610 0x557d6dcbeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:20.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.331+0000 7f97a3de5740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x557d6dc7e7b0 con 0x557d6dcb74a0 2024-03-27T10:58:20.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.331+0000 7f97a3de5740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x557d6dc1b670 con 0x557d6dcb6ab0 2024-03-27T10:58:20.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.331+0000 7f97a3de5740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x557d6dbc3970 con 0x557d6dcbc610 2024-03-27T10:58:20.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.331+0000 7f9799e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557d6dcb74a0 0x557d6dcbbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:20.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.331+0000 7f9799e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557d6dcb74a0 0x557d6dcbbfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53252/0 (socket says 172.21.15.97:53252) 2024-03-27T10:58:20.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.331+0000 7f9799e00640 1 -- 172.21.15.97:0/2277025019 learned_addr learned my addr 172.21.15.97:0/2277025019 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:58:20.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.331+0000 7f979a800640 1 --2- 172.21.15.97:0/2277025019 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557d6dcb6ab0 0x557d6dcb6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:20.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.332+0000 7f9799400640 1 --2- 172.21.15.97:0/2277025019 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557d6dcbc610 0x557d6dcbeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:20.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.332+0000 7f9799e00640 1 -- 172.21.15.97:0/2277025019 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557d6dcbc610 msgr2=0x557d6dcbeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:20.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.332+0000 7f9799e00640 1 --2- 172.21.15.97:0/2277025019 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557d6dcbc610 0x557d6dcbeaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:20.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.332+0000 7f9799e00640 1 -- 172.21.15.97:0/2277025019 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557d6dcb6ab0 msgr2=0x557d6dcb6ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:20.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.332+0000 7f9799e00640 1 --2- 172.21.15.97:0/2277025019 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557d6dcb6ab0 0x557d6dcb6ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:20.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.332+0000 7f9799e00640 1 -- 172.21.15.97:0/2277025019 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d6d92f3f0 con 0x557d6dcb74a0 2024-03-27T10:58:20.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.332+0000 7f9799400640 1 --2- 172.21.15.97:0/2277025019 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557d6dcbc610 0x557d6dcbeaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:58:20.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.332+0000 7f9792a00640 1 --2- 172.21.15.97:0/3925846378 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f97800a3610 0x7f97800a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:20.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.332+0000 7f9799e00640 1 --2- 172.21.15.97:0/2277025019 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557d6dcb74a0 0x557d6dcbbfa0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f978c009c20 tx=0x7f978c0117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8be312ed22551a8a server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:20.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.332+0000 7f9792a00640 1 -- 172.21.15.97:0/3925846378 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f97880f14e0 con 0x7f979c102c40 2024-03-27T10:58:20.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.332+0000 7f9787400640 1 -- 172.21.15.97:0/2277025019 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f978c004030 con 0x557d6dcb74a0 2024-03-27T10:58:20.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.332+0000 7f979a800640 1 --2- 172.21.15.97:0/3925846378 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f97800a3610 0x7f97800a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:20.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.332+0000 7f9787400640 1 -- 172.21.15.97:0/2277025019 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f978c0041d0 con 0x557d6dcb74a0 2024-03-27T10:58:20.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.332+0000 7f9787400640 1 -- 172.21.15.97:0/2277025019 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f978c0126b0 con 0x557d6dcb74a0 2024-03-27T10:58:20.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.333+0000 7f97a3de5740 1 -- 172.21.15.97:0/2277025019 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557d6dcb74a0 msgr2=0x557d6dcbbfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:20.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.333+0000 7f979a800640 1 --2- 172.21.15.97:0/3925846378 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f97800a3610 0x7f97800a5ad0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x557d6dcb7d00 tx=0x7f9794009740 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:20.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.333+0000 7f97a3de5740 1 --2- 172.21.15.97:0/2277025019 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557d6dcb74a0 0x557d6dcbbfa0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f978c009c20 tx=0x7f978c0117b0 comp rx=0 tx=0).stop 2024-03-27T10:58:20.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.333+0000 7f97a3de5740 1 -- 172.21.15.97:0/2277025019 shutdown_connections 2024-03-27T10:58:20.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.333+0000 7f97a3de5740 1 --2- 172.21.15.97:0/2277025019 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557d6dcbc610 0x557d6dcbeaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:20.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.333+0000 7f97a3de5740 1 --2- 172.21.15.97:0/2277025019 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557d6dcb74a0 0x557d6dcbbfa0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:20.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.333+0000 7f97a3de5740 1 --2- 172.21.15.97:0/2277025019 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557d6dcb6ab0 0x557d6dcb6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:20.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.333+0000 7f97a3de5740 1 -- 172.21.15.97:0/2277025019 >> 172.21.15.97:0/2277025019 conn(0x557d6dcb2020 msgr2=0x557d6dcb2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:20.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.333+0000 7f97a3de5740 1 -- 172.21.15.97:0/2277025019 shutdown_connections 2024-03-27T10:58:20.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.333+0000 7f97a3de5740 1 -- 172.21.15.97:0/2277025019 wait complete. 2024-03-27T10:58:20.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.334+0000 7f97a3de5740 1 -- start start 2024-03-27T10:58:20.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.334+0000 7f97a3de5740 1 Processor -- start 2024-03-27T10:58:20.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.334+0000 7f97a3de5740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557d6dcb6ab0 0x557d6dcc0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:20.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.334+0000 7f97a3de5740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557d6dcb74a0 0x557d6dcc1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:20.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.334+0000 7f97a3de5740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557d6dcbc610 0x557d6dcc8180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:20.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.334+0000 7f97a3de5740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x557d6dbd6ca0 con 0x557d6dcb74a0 2024-03-27T10:58:20.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.334+0000 7f97a3de5740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x557d6dc17620 con 0x557d6dcbc610 2024-03-27T10:58:20.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.334+0000 7f97a3de5740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x557d6dcbf5c0 con 0x557d6dcb6ab0 2024-03-27T10:58:20.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.334+0000 7f97a3de5740 10 client.0 _ll_register_callbacks cb 0x557d6dc13160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:58:20.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.335+0000 7f9799400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557d6dcb74a0 0x557d6dcc1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:20.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.335+0000 7f9799400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557d6dcb74a0 0x557d6dcc1140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53264/0 (socket says 172.21.15.97:53264) 2024-03-27T10:58:20.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.335+0000 7f9799400640 1 -- 172.21.15.97:0/1179338214 learned_addr learned my addr 172.21.15.97:0/1179338214 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:58:20.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.335+0000 7f9799e00640 1 --2- 172.21.15.97:0/1179338214 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557d6dcb6ab0 0x557d6dcc0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:20.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.335+0000 7f979a800640 1 --2- 172.21.15.97:0/1179338214 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557d6dcbc610 0x557d6dcc8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:20.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.335+0000 7f9799400640 1 -- 172.21.15.97:0/1179338214 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557d6dcb6ab0 msgr2=0x557d6dcc0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:20.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.335+0000 7f9799400640 1 --2- 172.21.15.97:0/1179338214 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557d6dcb6ab0 0x557d6dcc0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:20.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.335+0000 7f9799400640 1 -- 172.21.15.97:0/1179338214 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557d6dcbc610 msgr2=0x557d6dcc8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:20.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.335+0000 7f9799400640 1 --2- 172.21.15.97:0/1179338214 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557d6dcbc610 0x557d6dcc8180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:20.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.335+0000 7f9799400640 1 -- 172.21.15.97:0/1179338214 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d6d9f97d0 con 0x557d6dcb74a0 2024-03-27T10:58:20.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.335+0000 7f9799400640 1 --2- 172.21.15.97:0/1179338214 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557d6dcb74a0 0x557d6dcc1140 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f9788005d40 tx=0x7f97880bd2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=842d150c46cf8c97 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:20.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.335+0000 7f977be00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T10:58:20.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.335+0000 7f977be00640 1 -- 172.21.15.97:0/1179338214 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97880f12d0 con 0x557d6dcb74a0 2024-03-27T10:58:20.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.336+0000 7f977be00640 1 -- 172.21.15.97:0/1179338214 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f97880c6140 con 0x557d6dcb74a0 2024-03-27T10:58:20.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.336+0000 7f97a3de5740 1 -- 172.21.15.97:0/1179338214 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f978c0041d0 con 0x557d6dcb74a0 2024-03-27T10:58:20.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.336+0000 7f977be00640 1 -- 172.21.15.97:0/1179338214 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f97880bd960 con 0x557d6dcb74a0 2024-03-27T10:58:20.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.336+0000 7f97a3de5740 10 client.16134 fetch_fsmap learned FSMap version 30 2024-03-27T10:58:20.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.336+0000 7f97a3de5740 1 -- 172.21.15.97:0/1179338214 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557d6dccb460 con 0x557d6dcb74a0 2024-03-27T10:58:20.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.336+0000 7f977be00640 1 -- 172.21.15.97:0/1179338214 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f978c0041d0 con 0x557d6dcb74a0 2024-03-27T10:58:20.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.336+0000 7f977be00640 1 -- 172.21.15.97:0/1179338214 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f978804e060 con 0x557d6dcb74a0 2024-03-27T10:58:20.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.336+0000 7f97a3de5740 10 client.16134 fetch_fsmap finished waiting for FSMap version 30 2024-03-27T10:58:20.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.336+0000 7f97a3de5740 10 client.16134 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-27T10:58:20.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.336+0000 7f97a3de5740 20 client.16134 populate_metadata read hostname 'smithi097' 2024-03-27T10:58:20.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.336+0000 7f97a3de5740 1 --2- 172.21.15.97:0/1179338214 >> [v2:172.21.15.114:6836/616775588,v1:172.21.15.114:6837/616775588] conn(0x557d6dcccbe0 0x557d6dccf060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:20.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.336+0000 7f97a3de5740 4 client.16134 mds_command: new command op to 34478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T10:58:20.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.336+0000 7f97a3de5740 1 -- 172.21.15.97:0/1179338214 --> [v2:172.21.15.114:6836/616775588,v1:172.21.15.114:6837/616775588] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557d6dc28bb0 con 0x557d6dcccbe0 2024-03-27T10:58:20.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.337+0000 7f9799e00640 1 --2- 172.21.15.97:0/1179338214 >> [v2:172.21.15.114:6836/616775588,v1:172.21.15.114:6837/616775588] conn(0x557d6dcccbe0 0x557d6dccf060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:20.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.337+0000 7f9799e00640 1 --2- 172.21.15.97:0/1179338214 >> [v2:172.21.15.114:6836/616775588,v1:172.21.15.114:6837/616775588] conn(0x557d6dcccbe0 0x557d6dccf060 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c5cbc2c071922886 server_cookie=dca4336f4dcc5f8b in_seq=0 out_seq=0 2024-03-27T10:58:20.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.337+0000 7f977be00640 10 client.16134 ms_handle_connect on v2:172.21.15.114:6836/616775588 2024-03-27T10:58:20.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.339+0000 7f977be00640 1 -- 172.21.15.97:0/1179338214 <== mds.1 v2:172.21.15.114:6836/616775588 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x557d6dc28bb0 con 0x557d6dcccbe0 2024-03-27T10:58:20.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.339+0000 7f977be00640 10 client.16134 handle_command_reply: tid=0 2024-03-27T10:58:20.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.339+0000 7f97a3de5740 1 client.16134 shutdown 2024-03-27T10:58:20.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.339+0000 7f97a3de5740 1 -- 172.21.15.97:0/1179338214 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557d6dcb74a0 msgr2=0x557d6dcc1140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:20.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.339+0000 7f97a3de5740 1 --2- 172.21.15.97:0/1179338214 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557d6dcb74a0 0x557d6dcc1140 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f9788005d40 tx=0x7f97880bd2c0 comp rx=0 tx=0).stop 2024-03-27T10:58:20.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.339+0000 7f97a3de5740 1 -- 172.21.15.97:0/1179338214 shutdown_connections 2024-03-27T10:58:20.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.339+0000 7f97a3de5740 1 --2- 172.21.15.97:0/1179338214 >> [v2:172.21.15.114:6836/616775588,v1:172.21.15.114:6837/616775588] conn(0x557d6dcccbe0 0x557d6dccf060 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:20.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.339+0000 7f97a3de5740 1 --2- 172.21.15.97:0/1179338214 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557d6dcbc610 0x557d6dcc8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:20.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.339+0000 7f97a3de5740 1 --2- 172.21.15.97:0/1179338214 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557d6dcb74a0 0x557d6dcc1140 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:20.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.339+0000 7f97a3de5740 1 --2- 172.21.15.97:0/1179338214 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557d6dcb6ab0 0x557d6dcc0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:20.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.339+0000 7f977be00640 0 client.16134 ms_handle_reset on v2:172.21.15.114:6836/616775588 2024-03-27T10:58:20.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.339+0000 7f97a3de5740 1 -- 172.21.15.97:0/1179338214 >> 172.21.15.97:0/1179338214 conn(0x557d6dcb2020 msgr2=0x557d6dcb1240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:20.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.340+0000 7f97a3de5740 1 -- 172.21.15.97:0/1179338214 shutdown_connections 2024-03-27T10:58:20.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.340+0000 7f97a3de5740 1 -- 172.21.15.97:0/1179338214 wait complete. 2024-03-27T10:58:20.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.340+0000 7f97a3de5740 20 client.16134 trim_cache size 0 max 16384 2024-03-27T10:58:20.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.353+0000 7f97a3de5740 1 Processor -- start 2024-03-27T10:58:20.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.353+0000 7f97a3de5740 1 -- start start 2024-03-27T10:58:20.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.354+0000 7f97a3de5740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557d6dcb6ab0 0x557d6dcb86c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:20.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.354+0000 7f97a3de5740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557d6dcb74a0 0x557d6dcb8c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:20.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.354+0000 7f97a3de5740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557d6dcbc610 0x557d6dcc8140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:20.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.354+0000 7f97a3de5740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f979c1aaee0 con 0x557d6dcb74a0 2024-03-27T10:58:20.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.354+0000 7f97a3de5740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f979c1acae0 con 0x557d6dcb6ab0 2024-03-27T10:58:20.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.354+0000 7f97a3de5740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f979c1ab5b0 con 0x557d6dcbc610 2024-03-27T10:58:20.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.354+0000 7f9799400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557d6dcb74a0 0x557d6dcb8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:20.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.354+0000 7f9799400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557d6dcb74a0 0x557d6dcb8c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53274/0 (socket says 172.21.15.97:53274) 2024-03-27T10:58:20.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.354+0000 7f9799400640 1 -- 172.21.15.97:0/2885178349 learned_addr learned my addr 172.21.15.97:0/2885178349 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:58:20.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.354+0000 7f979a800640 1 --2- 172.21.15.97:0/2885178349 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557d6dcbc610 0x557d6dcc8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:20.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.354+0000 7f9799e00640 1 --2- 172.21.15.97:0/2885178349 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557d6dcb6ab0 0x557d6dcb86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:20.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.355+0000 7f9799400640 1 -- 172.21.15.97:0/2885178349 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557d6dcbc610 msgr2=0x557d6dcc8140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:20.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.355+0000 7f9799400640 1 --2- 172.21.15.97:0/2885178349 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557d6dcbc610 0x557d6dcc8140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:20.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.355+0000 7f9799400640 1 -- 172.21.15.97:0/2885178349 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557d6dcb6ab0 msgr2=0x557d6dcb86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:20.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.355+0000 7f9799400640 1 --2- 172.21.15.97:0/2885178349 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557d6dcb6ab0 0x557d6dcb86c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:20.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.355+0000 7f9799400640 1 -- 172.21.15.97:0/2885178349 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d6dcb7d30 con 0x557d6dcb74a0 2024-03-27T10:58:20.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.355+0000 7f9799e00640 1 --2- 172.21.15.97:0/2885178349 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557d6dcb6ab0 0x557d6dcb86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T10:58:20.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.355+0000 7f9799400640 1 --2- 172.21.15.97:0/2885178349 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557d6dcb74a0 0x557d6dcb8c00 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f978802c420 tx=0x7f97880bd3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=347597be56a511ad server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:20.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.355+0000 7f9786a00640 1 -- 172.21.15.97:0/2885178349 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97880c1d10 con 0x557d6dcb74a0 2024-03-27T10:58:20.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.355+0000 7f9786a00640 1 -- 172.21.15.97:0/2885178349 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f97880c65d0 con 0x557d6dcb74a0 2024-03-27T10:58:20.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.355+0000 7f9786a00640 1 -- 172.21.15.97:0/2885178349 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97880b96b0 con 0x557d6dcb74a0 2024-03-27T10:58:20.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.356+0000 7f97a3de5740 1 -- 172.21.15.97:0/2885178349 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557d6dcb74a0 msgr2=0x557d6dcb8c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:20.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.356+0000 7f97a3de5740 1 --2- 172.21.15.97:0/2885178349 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557d6dcb74a0 0x557d6dcb8c00 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f978802c420 tx=0x7f97880bd3f0 comp rx=0 tx=0).stop 2024-03-27T10:58:20.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.356+0000 7f97a3de5740 1 -- 172.21.15.97:0/2885178349 shutdown_connections 2024-03-27T10:58:20.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.356+0000 7f97a3de5740 1 --2- 172.21.15.97:0/2885178349 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557d6dcbc610 0x557d6dcc8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:20.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.356+0000 7f97a3de5740 1 --2- 172.21.15.97:0/2885178349 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557d6dcb74a0 0x557d6dcb8c00 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:20.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.356+0000 7f97a3de5740 1 --2- 172.21.15.97:0/2885178349 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557d6dcb6ab0 0x557d6dcb86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:20.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.356+0000 7f97a3de5740 1 -- 172.21.15.97:0/2885178349 >> 172.21.15.97:0/2885178349 conn(0x557d6dcb2020 msgr2=0x557d6dcc0c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:20.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.356+0000 7f97a3de5740 1 -- 172.21.15.97:0/2885178349 shutdown_connections 2024-03-27T10:58:20.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.356+0000 7f97a3de5740 1 -- 172.21.15.97:0/2885178349 wait complete. 2024-03-27T10:58:20.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.357+0000 7f97a3de5740 1 -- start start 2024-03-27T10:58:20.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.357+0000 7f97a3de5740 1 Processor -- start 2024-03-27T10:58:20.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.357+0000 7f97a3de5740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557d6dcb6ab0 0x557d6dcc53e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:20.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.357+0000 7f97a3de5740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557d6dcb74a0 0x557d6dcc5920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:20.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.357+0000 7f97a3de5740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557d6dcbc610 0x557d6dcc6930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:20.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.357+0000 7f97a3de5740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f979c1ac350 con 0x557d6dcbc610 2024-03-27T10:58:20.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.357+0000 7f97a3de5740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f979c1abc80 con 0x557d6dcb6ab0 2024-03-27T10:58:20.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.357+0000 7f97a3de5740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f979c1a8ec0 con 0x557d6dcb74a0 2024-03-27T10:58:20.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.357+0000 7f97a3de5740 10 client.0 _ll_register_callbacks cb 0x7f9788005c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:58:20.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.357+0000 7f979a800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557d6dcbc610 0x557d6dcc6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:20.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.357+0000 7f979a800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557d6dcbc610 0x557d6dcc6930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53280/0 (socket says 172.21.15.97:53280) 2024-03-27T10:58:20.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.357+0000 7f979a800640 1 -- 172.21.15.97:0/1962494317 learned_addr learned my addr 172.21.15.97:0/1962494317 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:58:20.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.358+0000 7f9799e00640 1 --2- 172.21.15.97:0/1962494317 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557d6dcb6ab0 0x557d6dcc53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:20.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.358+0000 7f9799400640 1 --2- 172.21.15.97:0/1962494317 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557d6dcb74a0 0x557d6dcc5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:20.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.358+0000 7f979a800640 1 -- 172.21.15.97:0/1962494317 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557d6dcb74a0 msgr2=0x557d6dcc5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:20.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.358+0000 7f979a800640 1 --2- 172.21.15.97:0/1962494317 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557d6dcb74a0 0x557d6dcc5920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:20.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.358+0000 7f979a800640 1 -- 172.21.15.97:0/1962494317 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557d6dcb6ab0 msgr2=0x557d6dcc53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:20.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.358+0000 7f979a800640 1 --2- 172.21.15.97:0/1962494317 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557d6dcb6ab0 0x557d6dcc53e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:20.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.358+0000 7f979a800640 1 -- 172.21.15.97:0/1962494317 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d6dcc6f20 con 0x557d6dcbc610 2024-03-27T10:58:20.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.358+0000 7f9799e00640 1 --2- 172.21.15.97:0/1962494317 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557d6dcb6ab0 0x557d6dcc53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T10:58:20.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.358+0000 7f979a800640 1 --2- 172.21.15.97:0/1962494317 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557d6dcbc610 0x557d6dcc6930 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f979401ea50 tx=0x7f9794004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6ed7e94686f5d41 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:20.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.358+0000 7f977be00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T10:58:20.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.358+0000 7f977be00640 1 -- 172.21.15.97:0/1962494317 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9794004d20 con 0x557d6dcbc610 2024-03-27T10:58:20.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.358+0000 7f977be00640 1 -- 172.21.15.97:0/1962494317 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f979401f7b0 con 0x557d6dcbc610 2024-03-27T10:58:20.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.358+0000 7f97a3de5740 1 -- 172.21.15.97:0/1962494317 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97880c65d0 con 0x557d6dcbc610 2024-03-27T10:58:20.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.359+0000 7f977be00640 1 -- 172.21.15.97:0/1962494317 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f979401f9b0 con 0x557d6dcbc610 2024-03-27T10:58:20.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.359+0000 7f97a3de5740 10 client.16140 fetch_fsmap learned FSMap version 30 2024-03-27T10:58:20.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.359+0000 7f97a3de5740 1 -- 172.21.15.97:0/1962494317 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557d6dcc8290 con 0x557d6dcbc610 2024-03-27T10:58:20.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.359+0000 7f977be00640 1 -- 172.21.15.97:0/1962494317 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f97880c65d0 con 0x557d6dcbc610 2024-03-27T10:58:20.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.359+0000 7f977be00640 1 -- 172.21.15.97:0/1962494317 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f9794028070 con 0x557d6dcbc610 2024-03-27T10:58:20.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.359+0000 7f97a3de5740 10 client.16140 fetch_fsmap finished waiting for FSMap version 30 2024-03-27T10:58:20.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.359+0000 7f97a3de5740 10 client.16140 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-27T10:58:20.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.359+0000 7f97a3de5740 20 client.16140 populate_metadata read hostname 'smithi097' 2024-03-27T10:58:20.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.359+0000 7f97a3de5740 1 --2- 172.21.15.97:0/1962494317 >> [v2:172.21.15.114:6836/616775588,v1:172.21.15.114:6837/616775588] conn(0x557d6dd33620 0x557d6dd35a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:20.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.359+0000 7f97a3de5740 4 client.16140 mds_command: new command op to 34478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-27T10:58:20.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.359+0000 7f97a3de5740 1 -- 172.21.15.97:0/1962494317 --> [v2:172.21.15.114:6836/616775588,v1:172.21.15.114:6837/616775588] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557d6dc22f00 con 0x557d6dd33620 2024-03-27T10:58:20.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.360+0000 7f9799e00640 1 --2- 172.21.15.97:0/1962494317 >> [v2:172.21.15.114:6836/616775588,v1:172.21.15.114:6837/616775588] conn(0x557d6dd33620 0x557d6dd35a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:20.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.360+0000 7f9799e00640 1 --2- 172.21.15.97:0/1962494317 >> [v2:172.21.15.114:6836/616775588,v1:172.21.15.114:6837/616775588] conn(0x557d6dd33620 0x557d6dd35a40 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b137f1e2daf5d797 server_cookie=78c12709bed12b00 in_seq=0 out_seq=0 2024-03-27T10:58:20.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.360+0000 7f977be00640 10 client.16140 ms_handle_connect on v2:172.21.15.114:6836/616775588 2024-03-27T10:58:20.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.360+0000 7f977be00640 1 -- 172.21.15.97:0/1962494317 <== mds.1 v2:172.21.15.114:6836/616775588 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557d6dc22f00 con 0x557d6dd33620 2024-03-27T10:58:20.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.360+0000 7f977be00640 10 client.16140 handle_command_reply: tid=0 2024-03-27T10:58:20.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.360+0000 7f97a3de5740 1 client.16140 shutdown 2024-03-27T10:58:20.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.361+0000 7f97a3de5740 1 -- 172.21.15.97:0/1962494317 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557d6dcbc610 msgr2=0x557d6dcc6930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:20.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.361+0000 7f97a3de5740 1 --2- 172.21.15.97:0/1962494317 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557d6dcbc610 0x557d6dcc6930 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f979401ea50 tx=0x7f9794004a40 comp rx=0 tx=0).stop 2024-03-27T10:58:20.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.361+0000 7f97a3de5740 1 -- 172.21.15.97:0/1962494317 shutdown_connections 2024-03-27T10:58:20.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.361+0000 7f97a3de5740 1 --2- 172.21.15.97:0/1962494317 >> [v2:172.21.15.114:6836/616775588,v1:172.21.15.114:6837/616775588] conn(0x557d6dd33620 0x557d6dd35a40 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:20.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.361+0000 7f97a3de5740 1 --2- 172.21.15.97:0/1962494317 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557d6dcbc610 0x557d6dcc6930 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:20.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.361+0000 7f97a3de5740 1 --2- 172.21.15.97:0/1962494317 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557d6dcb74a0 0x557d6dcc5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:20.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.361+0000 7f97a3de5740 1 --2- 172.21.15.97:0/1962494317 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557d6dcb6ab0 0x557d6dcc53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:20.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.361+0000 7f977be00640 0 client.16140 ms_handle_reset on v2:172.21.15.114:6836/616775588 2024-03-27T10:58:20.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.361+0000 7f97a3de5740 1 -- 172.21.15.97:0/1962494317 >> 172.21.15.97:0/1962494317 conn(0x557d6dcb2020 msgr2=0x557d6dcb5b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:20.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.361+0000 7f97a3de5740 1 -- 172.21.15.97:0/1962494317 shutdown_connections 2024-03-27T10:58:20.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.361+0000 7f97a3de5740 1 -- 172.21.15.97:0/1962494317 wait complete. 2024-03-27T10:58:20.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.361+0000 7f97a3de5740 20 client.16140 trim_cache size 0 max 16384 2024-03-27T10:58:20.337 INFO:teuthology.orchestra.run.smithi097.stdout:[] 2024-03-27T10:58:20.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.361+0000 7f979b200640 1 -- 172.21.15.97:0/3925846378 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f97800a3610 msgr2=0x7f97800a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:20.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.361+0000 7f979b200640 1 --2- 172.21.15.97:0/3925846378 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f97800a3610 0x7f97800a5ad0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x557d6dcb7d00 tx=0x7f9794009740 comp rx=0 tx=0).stop 2024-03-27T10:58:20.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.361+0000 7f979b200640 1 -- 172.21.15.97:0/3925846378 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f979c102c40 msgr2=0x7f979c19a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:20.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.361+0000 7f979b200640 1 --2- 172.21.15.97:0/3925846378 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f979c102c40 0x7f979c19a420 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f97880057c0 tx=0x7f97880042f0 comp rx=0 tx=0).stop 2024-03-27T10:58:20.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.362+0000 7f979b200640 1 -- 172.21.15.97:0/3925846378 shutdown_connections 2024-03-27T10:58:20.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.362+0000 7f979b200640 1 --2- 172.21.15.97:0/3925846378 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f97800a3610 0x7f97800a5ad0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:20.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.362+0000 7f979b200640 1 --2- 172.21.15.97:0/3925846378 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f979c103600 0x7f979c1a1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:20.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.362+0000 7f979b200640 1 --2- 172.21.15.97:0/3925846378 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f979c102c40 0x7f979c19a420 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:20.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.362+0000 7f979b200640 1 --2- 172.21.15.97:0/3925846378 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f979c101a50 0x7f979c199ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:20.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.362+0000 7f979b200640 1 -- 172.21.15.97:0/3925846378 >> 172.21.15.97:0/3925846378 conn(0x7f979c0fcfc0 msgr2=0x7f979c073d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:20.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.362+0000 7f979b200640 1 -- 172.21.15.97:0/3925846378 shutdown_connections 2024-03-27T10:58:20.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.362+0000 7f979b200640 1 -- 172.21.15.97:0/3925846378 wait complete. 2024-03-27T10:58:20.347 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-27T10:58:20.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.608+0000 7f3afc400640 1 -- 172.21.15.97:0/2210751575 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3af4102d70 msgr2=0x7f3af41031f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:20.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.608+0000 7f3afc400640 1 --2- 172.21.15.97:0/2210751575 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3af4102d70 0x7f3af41031f0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f3ae0009900 tx=0x7f3ae002bc30 comp rx=0 tx=0).stop 2024-03-27T10:58:20.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.609+0000 7f3afc400640 1 -- 172.21.15.97:0/2210751575 shutdown_connections 2024-03-27T10:58:20.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.609+0000 7f3afc400640 1 --2- 172.21.15.97:0/2210751575 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3af4103730 0x7f3af4109fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:20.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.609+0000 7f3afc400640 1 --2- 172.21.15.97:0/2210751575 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3af4102d70 0x7f3af41031f0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:20.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.609+0000 7f3afc400640 1 --2- 172.21.15.97:0/2210751575 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3af4101b80 0x7f3af4101fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:20.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.609+0000 7f3afc400640 1 -- 172.21.15.97:0/2210751575 >> 172.21.15.97:0/2210751575 conn(0x7f3af40fd0f0 msgr2=0x7f3af40ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:20.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.609+0000 7f3afc400640 1 -- 172.21.15.97:0/2210751575 shutdown_connections 2024-03-27T10:58:20.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.609+0000 7f3afc400640 1 -- 172.21.15.97:0/2210751575 wait complete. 2024-03-27T10:58:20.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.611+0000 7f3afc400640 1 Processor -- start 2024-03-27T10:58:20.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.611+0000 7f3afc400640 1 -- start start 2024-03-27T10:58:20.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.611+0000 7f3afc400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3af4101b80 0x7f3af41078b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:20.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.611+0000 7f3afc400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3af4102d70 0x7f3af4107df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:20.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.611+0000 7f3afc400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3af4103730 0x7f3af4104420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:20.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.611+0000 7f3afc400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3af410d700 con 0x7f3af4101b80 2024-03-27T10:58:20.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.611+0000 7f3afc400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f3af410d580 con 0x7f3af4103730 2024-03-27T10:58:20.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.611+0000 7f3afc400640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f3af410d880 con 0x7f3af4102d70 2024-03-27T10:58:20.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.611+0000 7f3af9a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3af4101b80 0x7f3af41078b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:20.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.612+0000 7f3af9a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3af4101b80 0x7f3af41078b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53300/0 (socket says 172.21.15.97:53300) 2024-03-27T10:58:20.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.612+0000 7f3af9a00640 1 -- 172.21.15.97:0/1738159564 learned_addr learned my addr 172.21.15.97:0/1738159564 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:58:20.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.612+0000 7f3afa400640 1 --2- 172.21.15.97:0/1738159564 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3af4103730 0x7f3af4104420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:20.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.612+0000 7f3af9000640 1 --2- 172.21.15.97:0/1738159564 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3af4102d70 0x7f3af4107df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:20.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.612+0000 7f3af9a00640 1 -- 172.21.15.97:0/1738159564 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3af4102d70 msgr2=0x7f3af4107df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:20.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.612+0000 7f3af9a00640 1 --2- 172.21.15.97:0/1738159564 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3af4102d70 0x7f3af4107df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:20.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.612+0000 7f3af9a00640 1 -- 172.21.15.97:0/1738159564 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3af4103730 msgr2=0x7f3af4104420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:20.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.612+0000 7f3af9a00640 1 --2- 172.21.15.97:0/1738159564 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3af4103730 0x7f3af4104420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:20.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.612+0000 7f3af9a00640 1 -- 172.21.15.97:0/1738159564 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3af4104ce0 con 0x7f3af4101b80 2024-03-27T10:58:20.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.612+0000 7f3af9a00640 1 --2- 172.21.15.97:0/1738159564 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3af4101b80 0x7f3af41078b0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f3ae800e9c0 tx=0x7f3ae800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:20.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.613+0000 7f3af2000640 1 -- 172.21.15.97:0/1738159564 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae800cd80 con 0x7f3af4101b80 2024-03-27T10:58:20.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.613+0000 7f3af2000640 1 -- 172.21.15.97:0/1738159564 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3ae80035f0 con 0x7f3af4101b80 2024-03-27T10:58:20.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.613+0000 7f3af2000640 1 -- 172.21.15.97:0/1738159564 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae8010690 con 0x7f3af4101b80 2024-03-27T10:58:20.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.613+0000 7f3afc400640 1 -- 172.21.15.97:0/1738159564 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3af41a6460 con 0x7f3af4101b80 2024-03-27T10:58:20.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.613+0000 7f3afc400640 1 -- 172.21.15.97:0/1738159564 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3af41a6810 con 0x7f3af4101b80 2024-03-27T10:58:20.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.617+0000 7f3aff016740 1 Processor -- start 2024-03-27T10:58:20.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.617+0000 7f3aff016740 1 -- start start 2024-03-27T10:58:20.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.617+0000 7f3aff016740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562ebc60f8c0 0x562ebc60fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:20.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.617+0000 7f3aff016740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562ebc6102b0 0x562ebc614db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:20.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.617+0000 7f3aff016740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562ebc615420 0x562ebc6178b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:20.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.617+0000 7f3aff016740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x562ebc5d77b0 con 0x562ebc615420 2024-03-27T10:58:20.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.617+0000 7f3aff016740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x562ebc574670 con 0x562ebc60f8c0 2024-03-27T10:58:20.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.617+0000 7f3aff016740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x562ebc51c970 con 0x562ebc6102b0 2024-03-27T10:58:20.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.618+0000 7f3af9000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562ebc615420 0x562ebc6178b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:20.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.618+0000 7f3af9000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562ebc615420 0x562ebc6178b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53308/0 (socket says 172.21.15.97:53308) 2024-03-27T10:58:20.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.618+0000 7f3af9000640 1 -- 172.21.15.97:0/2209079546 learned_addr learned my addr 172.21.15.97:0/2209079546 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:58:20.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.618+0000 7f3afa400640 1 --2- 172.21.15.97:0/2209079546 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562ebc60f8c0 0x562ebc60fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:20.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.618+0000 7f3af9000640 1 -- 172.21.15.97:0/2209079546 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562ebc6102b0 msgr2=0x562ebc614db0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-27T10:58:20.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.618+0000 7f3af9000640 1 --2- 172.21.15.97:0/2209079546 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562ebc6102b0 0x562ebc614db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:20.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.618+0000 7f3af9000640 1 -- 172.21.15.97:0/2209079546 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562ebc60f8c0 msgr2=0x562ebc60fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:20.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.618+0000 7f3af9000640 1 --2- 172.21.15.97:0/2209079546 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562ebc60f8c0 0x562ebc60fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:20.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.618+0000 7f3af9000640 1 -- 172.21.15.97:0/2209079546 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ebc2883f0 con 0x562ebc615420 2024-03-27T10:58:20.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.618+0000 7f3afa400640 1 --2- 172.21.15.97:0/2209079546 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562ebc60f8c0 0x562ebc60fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T10:58:20.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.619+0000 7f3af9000640 1 --2- 172.21.15.97:0/2209079546 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562ebc615420 0x562ebc6178b0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f3ae002c840 tx=0x7f3ae0009670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a5c606a840b0d09 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:20.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.619+0000 7f3ae6a00640 1 -- 172.21.15.97:0/2209079546 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae0044020 con 0x562ebc615420 2024-03-27T10:58:20.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.619+0000 7f3af2000640 1 -- 172.21.15.97:0/1738159564 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f3ae80037a0 con 0x7f3af4101b80 2024-03-27T10:58:20.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.620+0000 7f3af2000640 1 --2- 172.21.15.97:0/1738159564 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f3adc0a3690 0x7f3adc0a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:20.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.621+0000 7f3af2000640 1 -- 172.21.15.97:0/1738159564 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f3ae80d04c0 con 0x7f3af4101b80 2024-03-27T10:58:20.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.621+0000 7f3ae6a00640 1 -- 172.21.15.97:0/2209079546 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3ae002cae0 con 0x562ebc615420 2024-03-27T10:58:20.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.621+0000 7f3ae6a00640 1 -- 172.21.15.97:0/2209079546 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae0004920 con 0x562ebc615420 2024-03-27T10:58:20.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.621+0000 7f3afa400640 1 --2- 172.21.15.97:0/1738159564 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f3adc0a3690 0x7f3adc0a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:20.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.622+0000 7f3aff016740 1 -- 172.21.15.97:0/2209079546 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562ebc615420 msgr2=0x562ebc6178b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:20.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.622+0000 7f3aff016740 1 --2- 172.21.15.97:0/2209079546 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562ebc615420 0x562ebc6178b0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f3ae002c840 tx=0x7f3ae0009670 comp rx=0 tx=0).stop 2024-03-27T10:58:20.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.622+0000 7f3afa400640 1 --2- 172.21.15.97:0/1738159564 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f3adc0a3690 0x7f3adc0a5b50 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x562ebc610b10 tx=0x7f3aec009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:20.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.622+0000 7f3aff016740 1 -- 172.21.15.97:0/2209079546 shutdown_connections 2024-03-27T10:58:20.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.622+0000 7f3aff016740 1 --2- 172.21.15.97:0/2209079546 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562ebc615420 0x562ebc6178b0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:20.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.622+0000 7f3aff016740 1 --2- 172.21.15.97:0/2209079546 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562ebc6102b0 0x562ebc614db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:20.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.622+0000 7f3aff016740 1 --2- 172.21.15.97:0/2209079546 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562ebc60f8c0 0x562ebc60fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:20.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.622+0000 7f3aff016740 1 -- 172.21.15.97:0/2209079546 >> 172.21.15.97:0/2209079546 conn(0x562ebc60ae30 msgr2=0x562ebc60b230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:20.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.623+0000 7f3aff016740 1 -- 172.21.15.97:0/2209079546 shutdown_connections 2024-03-27T10:58:20.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.623+0000 7f3aff016740 1 -- 172.21.15.97:0/2209079546 wait complete. 2024-03-27T10:58:20.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.623+0000 7f3aff016740 1 -- start start 2024-03-27T10:58:20.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.624+0000 7f3aff016740 1 Processor -- start 2024-03-27T10:58:20.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.624+0000 7f3aff016740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562ebc60f8c0 0x562ebc619a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:20.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.625+0000 7f3aff016740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562ebc6102b0 0x562ebc619f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:20.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.625+0000 7f3aff016740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562ebc615420 0x562ebc620f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:20.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.625+0000 7f3aff016740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x562ebc52fca0 con 0x562ebc615420 2024-03-27T10:58:20.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.625+0000 7f3aff016740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x562ebc570620 con 0x562ebc60f8c0 2024-03-27T10:58:20.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.625+0000 7f3aff016740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x562ebc618370 con 0x562ebc6102b0 2024-03-27T10:58:20.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.625+0000 7f3aff016740 10 client.0 _ll_register_callbacks cb 0x562ebc56c160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:58:20.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.625+0000 7f3af9a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562ebc60f8c0 0x562ebc619a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:20.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.625+0000 7f3af9a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562ebc60f8c0 0x562ebc619a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:52608/0 (socket says 172.21.15.97:52608) 2024-03-27T10:58:20.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.626+0000 7f3af9000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562ebc6102b0 0x562ebc619f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:20.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.626+0000 7f3af9a00640 1 -- 172.21.15.97:0/3909602378 learned_addr learned my addr 172.21.15.97:0/3909602378 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:58:20.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.626+0000 7f3afa400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562ebc615420 0x562ebc620f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:20.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.627+0000 7f3afa400640 1 -- 172.21.15.97:0/3909602378 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562ebc6102b0 msgr2=0x562ebc619f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:20.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.627+0000 7f3afa400640 1 --2- 172.21.15.97:0/3909602378 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562ebc6102b0 0x562ebc619f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:20.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.627+0000 7f3afa400640 1 -- 172.21.15.97:0/3909602378 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562ebc60f8c0 msgr2=0x562ebc619a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:20.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.627+0000 7f3afa400640 1 --2- 172.21.15.97:0/3909602378 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562ebc60f8c0 0x562ebc619a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:20.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.627+0000 7f3afa400640 1 -- 172.21.15.97:0/3909602378 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ebc3527d0 con 0x562ebc615420 2024-03-27T10:58:20.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.627+0000 7f3af9000640 1 --2- 172.21.15.97:0/3909602378 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562ebc6102b0 0x562ebc619f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T10:58:20.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.628+0000 7f3afa400640 1 --2- 172.21.15.97:0/3909602378 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562ebc615420 0x562ebc620f90 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f3aec005fd0 tx=0x7f3aec004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65f3a760a7167437 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:20.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.628+0000 7f3adb400640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T10:58:20.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.628+0000 7f3adb400640 1 -- 172.21.15.97:0/3909602378 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aec02c060 con 0x562ebc615420 2024-03-27T10:58:20.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.629+0000 7f3aff016740 1 -- 172.21.15.97:0/3909602378 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ae002cae0 con 0x562ebc615420 2024-03-27T10:58:20.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.629+0000 7f3adb400640 1 -- 172.21.15.97:0/3909602378 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f3aec01db90 con 0x562ebc615420 2024-03-27T10:58:20.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.629+0000 7f3adb400640 1 -- 172.21.15.97:0/3909602378 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3aec01dd90 con 0x562ebc615420 2024-03-27T10:58:20.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.629+0000 7f3aff016740 10 client.16152 fetch_fsmap learned FSMap version 30 2024-03-27T10:58:20.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.629+0000 7f3aff016740 1 -- 172.21.15.97:0/3909602378 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562ebc624420 con 0x562ebc615420 2024-03-27T10:58:20.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.630+0000 7f3adb400640 1 -- 172.21.15.97:0/3909602378 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3ae002cae0 con 0x562ebc615420 2024-03-27T10:58:20.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.630+0000 7f3adb400640 1 -- 172.21.15.97:0/3909602378 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f3aec032550 con 0x562ebc615420 2024-03-27T10:58:20.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.630+0000 7f3aff016740 10 client.16152 fetch_fsmap finished waiting for FSMap version 30 2024-03-27T10:58:20.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.630+0000 7f3aff016740 10 client.16152 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-27T10:58:20.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.630+0000 7f3aff016740 20 client.16152 populate_metadata read hostname 'smithi097' 2024-03-27T10:58:20.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.631+0000 7f3aff016740 1 --2- 172.21.15.97:0/3909602378 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x562ebc625b60 0x562ebc627fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:20.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.631+0000 7f3aff016740 4 client.16152 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T10:58:20.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.631+0000 7f3aff016740 1 -- 172.21.15.97:0/3909602378 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562ebc581bb0 con 0x562ebc625b60 2024-03-27T10:58:20.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.631+0000 7f3af9a00640 1 --2- 172.21.15.97:0/3909602378 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x562ebc625b60 0x562ebc627fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:20.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.632+0000 7f3adb400640 10 client.16152 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T10:58:20.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.632+0000 7f3af9a00640 1 --2- 172.21.15.97:0/3909602378 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x562ebc625b60 0x562ebc627fa0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20aafc12706ab802 server_cookie=15150f6b5efaceda in_seq=0 out_seq=0 2024-03-27T10:58:20.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.633+0000 7f3adb400640 1 -- 172.21.15.97:0/3909602378 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x562ebc581bb0 con 0x562ebc625b60 2024-03-27T10:58:20.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.633+0000 7f3adb400640 10 client.16152 handle_command_reply: tid=0 2024-03-27T10:58:20.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.633+0000 7f3aff016740 1 client.16152 shutdown 2024-03-27T10:58:20.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.634+0000 7f3aff016740 1 -- 172.21.15.97:0/3909602378 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562ebc615420 msgr2=0x562ebc620f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:20.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.634+0000 7f3aff016740 1 --2- 172.21.15.97:0/3909602378 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562ebc615420 0x562ebc620f90 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f3aec005fd0 tx=0x7f3aec004a40 comp rx=0 tx=0).stop 2024-03-27T10:58:20.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.634+0000 7f3aff016740 1 -- 172.21.15.97:0/3909602378 shutdown_connections 2024-03-27T10:58:20.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.634+0000 7f3aff016740 1 --2- 172.21.15.97:0/3909602378 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x562ebc625b60 0x562ebc627fa0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:20.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.634+0000 7f3aff016740 1 --2- 172.21.15.97:0/3909602378 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562ebc615420 0x562ebc620f90 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:20.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.634+0000 7f3aff016740 1 --2- 172.21.15.97:0/3909602378 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562ebc6102b0 0x562ebc619f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:20.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.634+0000 7f3aff016740 1 --2- 172.21.15.97:0/3909602378 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562ebc60f8c0 0x562ebc619a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:20.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.634+0000 7f3adb400640 0 client.16152 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T10:58:20.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.634+0000 7f3aff016740 1 -- 172.21.15.97:0/3909602378 >> 172.21.15.97:0/3909602378 conn(0x562ebc60ae30 msgr2=0x562ebc60a030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:20.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.634+0000 7f3aff016740 1 -- 172.21.15.97:0/3909602378 shutdown_connections 2024-03-27T10:58:20.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.635+0000 7f3aff016740 1 -- 172.21.15.97:0/3909602378 wait complete. 2024-03-27T10:58:20.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.635+0000 7f3aff016740 20 client.16152 trim_cache size 0 max 16384 2024-03-27T10:58:20.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.657+0000 7f3aff016740 1 Processor -- start 2024-03-27T10:58:20.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.658+0000 7f3aff016740 1 -- start start 2024-03-27T10:58:20.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.658+0000 7f3aff016740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562ebc60f8c0 0x562ebc611570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:20.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.658+0000 7f3aff016740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562ebc6102b0 0x562ebc620f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:20.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.658+0000 7f3aff016740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562ebc615420 0x562ebc611ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:20.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.658+0000 7f3aff016740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3af41af570 con 0x562ebc615420 2024-03-27T10:58:20.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.658+0000 7f3aff016740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f3af41b1170 con 0x562ebc6102b0 2024-03-27T10:58:20.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.658+0000 7f3aff016740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f3af41afc40 con 0x562ebc60f8c0 2024-03-27T10:58:20.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.658+0000 7f3afa400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562ebc615420 0x562ebc611ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:20.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.659+0000 7f3afa400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562ebc615420 0x562ebc611ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53332/0 (socket says 172.21.15.97:53332) 2024-03-27T10:58:20.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.659+0000 7f3afa400640 1 -- 172.21.15.97:0/844976696 learned_addr learned my addr 172.21.15.97:0/844976696 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:58:20.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.659+0000 7f3af9000640 1 --2- 172.21.15.97:0/844976696 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562ebc6102b0 0x562ebc620f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:20.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.659+0000 7f3afa400640 1 -- 172.21.15.97:0/844976696 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562ebc60f8c0 msgr2=0x562ebc611570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:20.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.659+0000 7f3af9a00640 1 --2- 172.21.15.97:0/844976696 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562ebc60f8c0 0x562ebc611570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:20.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.659+0000 7f3afa400640 1 --2- 172.21.15.97:0/844976696 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562ebc60f8c0 0x562ebc611570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:20.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.659+0000 7f3afa400640 1 -- 172.21.15.97:0/844976696 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562ebc6102b0 msgr2=0x562ebc620f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:20.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.659+0000 7f3afa400640 1 --2- 172.21.15.97:0/844976696 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562ebc6102b0 0x562ebc620f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:20.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.659+0000 7f3afa400640 1 -- 172.21.15.97:0/844976696 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ebc6184f0 con 0x562ebc615420 2024-03-27T10:58:20.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.660+0000 7f3afa400640 1 --2- 172.21.15.97:0/844976696 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562ebc615420 0x562ebc611ab0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f3aec012e30 tx=0x7f3aec009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b67b18e3bd5feed server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:20.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.660+0000 7f3ae6000640 1 -- 172.21.15.97:0/844976696 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aec040070 con 0x562ebc615420 2024-03-27T10:58:20.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.660+0000 7f3ae6000640 1 -- 172.21.15.97:0/844976696 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3aec032e30 con 0x562ebc615420 2024-03-27T10:58:20.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.660+0000 7f3ae6000640 1 -- 172.21.15.97:0/844976696 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aec037050 con 0x562ebc615420 2024-03-27T10:58:20.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.661+0000 7f3aff016740 1 -- 172.21.15.97:0/844976696 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562ebc615420 msgr2=0x562ebc611ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:20.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.661+0000 7f3aff016740 1 --2- 172.21.15.97:0/844976696 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562ebc615420 0x562ebc611ab0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f3aec012e30 tx=0x7f3aec009650 comp rx=0 tx=0).stop 2024-03-27T10:58:20.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.661+0000 7f3aff016740 1 -- 172.21.15.97:0/844976696 shutdown_connections 2024-03-27T10:58:20.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.661+0000 7f3aff016740 1 --2- 172.21.15.97:0/844976696 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562ebc615420 0x562ebc611ab0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:20.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.661+0000 7f3aff016740 1 --2- 172.21.15.97:0/844976696 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562ebc6102b0 0x562ebc620f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:20.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.661+0000 7f3aff016740 1 --2- 172.21.15.97:0/844976696 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562ebc60f8c0 0x562ebc611570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:20.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.661+0000 7f3aff016740 1 -- 172.21.15.97:0/844976696 >> 172.21.15.97:0/844976696 conn(0x562ebc60ae30 msgr2=0x562ebc60b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:20.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.661+0000 7f3aff016740 1 -- 172.21.15.97:0/844976696 shutdown_connections 2024-03-27T10:58:20.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.661+0000 7f3aff016740 1 -- 172.21.15.97:0/844976696 wait complete. 2024-03-27T10:58:20.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.662+0000 7f3aff016740 1 -- start start 2024-03-27T10:58:20.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.662+0000 7f3aff016740 1 Processor -- start 2024-03-27T10:58:20.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.662+0000 7f3aff016740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562ebc60f8c0 0x562ebc61e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:20.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.662+0000 7f3aff016740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562ebc6102b0 0x562ebc61e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:20.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.662+0000 7f3aff016740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562ebc615420 0x562ebc61f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:20.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.662+0000 7f3aff016740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3af41b09e0 con 0x562ebc615420 2024-03-27T10:58:20.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.662+0000 7f3aff016740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f3af41b0310 con 0x562ebc60f8c0 2024-03-27T10:58:20.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.662+0000 7f3aff016740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f3af41ad550 con 0x562ebc6102b0 2024-03-27T10:58:20.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.662+0000 7f3aff016740 10 client.0 _ll_register_callbacks cb 0x7f3aec010960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:58:20.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.663+0000 7f3af9a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562ebc60f8c0 0x562ebc61e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:20.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.663+0000 7f3af9a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562ebc60f8c0 0x562ebc61e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:52616/0 (socket says 172.21.15.97:52616) 2024-03-27T10:58:20.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.663+0000 7f3afa400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562ebc615420 0x562ebc61f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:20.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.663+0000 7f3af9a00640 1 -- 172.21.15.97:0/107159733 learned_addr learned my addr 172.21.15.97:0/107159733 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:58:20.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.663+0000 7f3af9000640 1 --2- 172.21.15.97:0/107159733 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562ebc6102b0 0x562ebc61e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:20.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.663+0000 7f3af9a00640 1 -- 172.21.15.97:0/107159733 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562ebc6102b0 msgr2=0x562ebc61e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:20.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.663+0000 7f3af9a00640 1 --2- 172.21.15.97:0/107159733 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562ebc6102b0 0x562ebc61e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:20.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.663+0000 7f3af9a00640 1 -- 172.21.15.97:0/107159733 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562ebc615420 msgr2=0x562ebc61f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:20.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.663+0000 7f3af9a00640 1 --2- 172.21.15.97:0/107159733 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562ebc615420 0x562ebc61f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:20.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.663+0000 7f3af9a00640 1 -- 172.21.15.97:0/107159733 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ebc61fcc0 con 0x562ebc60f8c0 2024-03-27T10:58:20.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.664+0000 7f3af9a00640 1 --2- 172.21.15.97:0/107159733 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562ebc60f8c0 0x562ebc61e180 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3ae8099370 tx=0x7f3ae809d210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3d643cf00a642ad server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:20.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.664+0000 7f3adb400640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-03-27T10:58:20.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.664+0000 7f3adb400640 1 -- 172.21.15.97:0/107159733 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae80a5610 con 0x562ebc60f8c0 2024-03-27T10:58:20.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.664+0000 7f3aff016740 1 -- 172.21.15.97:0/107159733 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562ebc57bf00 con 0x562ebc60f8c0 2024-03-27T10:58:20.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.664+0000 7f3adb400640 1 -- 172.21.15.97:0/107159733 <== mon.2 v2:172.21.15.114:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f3ae809aca0 con 0x562ebc60f8c0 2024-03-27T10:58:20.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.665+0000 7f3adb400640 1 -- 172.21.15.97:0/107159733 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3ae8099010 con 0x562ebc60f8c0 2024-03-27T10:58:20.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.665+0000 7f3aff016740 10 client.35810 fetch_fsmap learned FSMap version 30 2024-03-27T10:58:20.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.665+0000 7f3aff016740 1 -- 172.21.15.97:0/107159733 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562ebc621030 con 0x562ebc60f8c0 2024-03-27T10:58:20.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.665+0000 7f3adb400640 1 -- 172.21.15.97:0/107159733 <== mon.2 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x562ebc57bf00 con 0x562ebc60f8c0 2024-03-27T10:58:20.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.665+0000 7f3adb400640 1 -- 172.21.15.97:0/107159733 <== mon.2 v2:172.21.15.114:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f3ae80b3020 con 0x562ebc60f8c0 2024-03-27T10:58:20.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.665+0000 7f3aff016740 10 client.35810 fetch_fsmap finished waiting for FSMap version 30 2024-03-27T10:58:20.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.665+0000 7f3aff016740 10 client.35810 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-27T10:58:20.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.665+0000 7f3aff016740 20 client.35810 populate_metadata read hostname 'smithi097' 2024-03-27T10:58:20.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.666+0000 7f3aff016740 1 --2- 172.21.15.97:0/107159733 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x562ebc68c420 0x562ebc68e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:20.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.666+0000 7f3aff016740 4 client.35810 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-27T10:58:20.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.666+0000 7f3aff016740 1 -- 172.21.15.97:0/107159733 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562ebc4f9560 con 0x562ebc68c420 2024-03-27T10:58:20.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.666+0000 7f3af9000640 1 --2- 172.21.15.97:0/107159733 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x562ebc68c420 0x562ebc68e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:20.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.667+0000 7f3af9000640 1 --2- 172.21.15.97:0/107159733 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x562ebc68c420 0x562ebc68e840 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=901258cb08bfeb01 server_cookie=1b58aeb049072e39 in_seq=0 out_seq=0 2024-03-27T10:58:20.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.667+0000 7f3adb400640 10 client.35810 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T10:58:20.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.667+0000 7f3adb400640 1 -- 172.21.15.97:0/107159733 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562ebc4f9560 con 0x562ebc68c420 2024-03-27T10:58:20.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.667+0000 7f3adb400640 10 client.35810 handle_command_reply: tid=0 2024-03-27T10:58:20.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.668+0000 7f3aff016740 1 client.35810 shutdown 2024-03-27T10:58:20.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.668+0000 7f3aff016740 1 -- 172.21.15.97:0/107159733 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562ebc60f8c0 msgr2=0x562ebc61e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:20.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.668+0000 7f3aff016740 1 --2- 172.21.15.97:0/107159733 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562ebc60f8c0 0x562ebc61e180 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3ae8099370 tx=0x7f3ae809d210 comp rx=0 tx=0).stop 2024-03-27T10:58:20.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.668+0000 7f3aff016740 1 -- 172.21.15.97:0/107159733 shutdown_connections 2024-03-27T10:58:20.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.668+0000 7f3aff016740 1 --2- 172.21.15.97:0/107159733 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x562ebc68c420 0x562ebc68e840 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:20.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.668+0000 7f3aff016740 1 --2- 172.21.15.97:0/107159733 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562ebc615420 0x562ebc61f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:20.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.668+0000 7f3adb400640 0 client.35810 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T10:58:20.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.668+0000 7f3aff016740 1 --2- 172.21.15.97:0/107159733 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562ebc6102b0 0x562ebc61e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:20.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.668+0000 7f3aff016740 1 --2- 172.21.15.97:0/107159733 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562ebc60f8c0 0x562ebc61e180 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:20.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.668+0000 7f3aff016740 1 -- 172.21.15.97:0/107159733 >> 172.21.15.97:0/107159733 conn(0x562ebc60ae30 msgr2=0x562ebc60b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:20.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.669+0000 7f3aff016740 1 -- 172.21.15.97:0/107159733 shutdown_connections 2024-03-27T10:58:20.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.669+0000 7f3aff016740 1 -- 172.21.15.97:0/107159733 wait complete. 2024-03-27T10:58:20.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.669+0000 7f3aff016740 20 client.35810 trim_cache size 0 max 16384 2024-03-27T10:58:20.642 INFO:teuthology.orchestra.run.smithi097.stdout:[] 2024-03-27T10:58:20.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.670+0000 7f3afc400640 1 -- 172.21.15.97:0/1738159564 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f3adc0a3690 msgr2=0x7f3adc0a5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:20.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.670+0000 7f3afc400640 1 --2- 172.21.15.97:0/1738159564 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f3adc0a3690 0x7f3adc0a5b50 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x562ebc610b10 tx=0x7f3aec009500 comp rx=0 tx=0).stop 2024-03-27T10:58:20.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.670+0000 7f3afc400640 1 -- 172.21.15.97:0/1738159564 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3af4101b80 msgr2=0x7f3af41078b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:20.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.670+0000 7f3afc400640 1 --2- 172.21.15.97:0/1738159564 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3af4101b80 0x7f3af41078b0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f3ae800e9c0 tx=0x7f3ae800ee80 comp rx=0 tx=0).stop 2024-03-27T10:58:20.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.670+0000 7f3afc400640 1 -- 172.21.15.97:0/1738159564 shutdown_connections 2024-03-27T10:58:20.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.670+0000 7f3afc400640 1 --2- 172.21.15.97:0/1738159564 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f3adc0a3690 0x7f3adc0a5b50 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:20.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.670+0000 7f3afc400640 1 --2- 172.21.15.97:0/1738159564 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3af4103730 0x7f3af4104420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:20.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.670+0000 7f3afc400640 1 --2- 172.21.15.97:0/1738159564 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3af4102d70 0x7f3af4107df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:20.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.670+0000 7f3afc400640 1 --2- 172.21.15.97:0/1738159564 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3af4101b80 0x7f3af41078b0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:20.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.670+0000 7f3afc400640 1 -- 172.21.15.97:0/1738159564 >> 172.21.15.97:0/1738159564 conn(0x7f3af40fd0f0 msgr2=0x7f3af4073d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:20.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.671+0000 7f3afc400640 1 -- 172.21.15.97:0/1738159564 shutdown_connections 2024-03-27T10:58:20.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:20.671+0000 7f3afc400640 1 -- 172.21.15.97:0/1738159564 wait complete. 2024-03-27T10:58:21.516 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:58:21 smithi114 ceph-mon[35338]: pgmap v492: 97 pgs: 97 active+clean; 1.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 126 MiB/s wr, 89 op/s 2024-03-27T10:58:21.516 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:58:21 smithi114 ceph-mon[35338]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T10:58:21.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:58:21 smithi097 ceph-mon[31859]: pgmap v492: 97 pgs: 97 active+clean; 1.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 126 MiB/s wr, 89 op/s 2024-03-27T10:58:21.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:58:21 smithi097 ceph-mon[31859]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T10:58:21.667 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-27T10:58:21.667 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-27T10:58:21.667 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-27T10:58:21.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:58:21 smithi177 ceph-mon[35294]: pgmap v492: 97 pgs: 97 active+clean; 1.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 126 MiB/s wr, 89 op/s 2024-03-27T10:58:21.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:58:21 smithi177 ceph-mon[35294]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T10:58:21.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.969+0000 7f78afe00640 1 -- 172.21.15.97:0/360574497 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78a8103d70 msgr2=0x7f78a8104190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:21.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.969+0000 7f78afe00640 1 --2- 172.21.15.97:0/360574497 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78a8103d70 0x7f78a8104190 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f78a0009ca0 tx=0x7f78a002c100 comp rx=0 tx=0).stop 2024-03-27T10:58:21.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.969+0000 7f78afe00640 1 -- 172.21.15.97:0/360574497 shutdown_connections 2024-03-27T10:58:21.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.969+0000 7f78afe00640 1 --2- 172.21.15.97:0/360574497 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f78a8105920 0x7f78a810c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:21.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.969+0000 7f78afe00640 1 --2- 172.21.15.97:0/360574497 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f78a8104f60 0x7f78a81053e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:21.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.969+0000 7f78afe00640 1 --2- 172.21.15.97:0/360574497 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78a8103d70 0x7f78a8104190 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:21.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.969+0000 7f78afe00640 1 -- 172.21.15.97:0/360574497 >> 172.21.15.97:0/360574497 conn(0x7f78a80ff2c0 msgr2=0x7f78a8101720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:21.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.969+0000 7f78afe00640 1 -- 172.21.15.97:0/360574497 shutdown_connections 2024-03-27T10:58:21.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.969+0000 7f78afe00640 1 -- 172.21.15.97:0/360574497 wait complete. 2024-03-27T10:58:21.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.970+0000 7f78afe00640 1 Processor -- start 2024-03-27T10:58:21.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.971+0000 7f78afe00640 1 -- start start 2024-03-27T10:58:21.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.971+0000 7f78afe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78a8103d70 0x7f78a819e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:21.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.971+0000 7f78afe00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f78a8104f60 0x7f78a819e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:21.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.971+0000 7f78afe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f78a8105920 0x7f78a81a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:21.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.971+0000 7f78afe00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f78a810f870 con 0x7f78a8103d70 2024-03-27T10:58:21.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.971+0000 7f78afe00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f78a810f6f0 con 0x7f78a8105920 2024-03-27T10:58:21.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.971+0000 7f78afe00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f78a810f9f0 con 0x7f78a8104f60 2024-03-27T10:58:21.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.971+0000 7f78ade00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f78a8105920 0x7f78a81a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:21.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.971+0000 7f78ad400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78a8103d70 0x7f78a819e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:21.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.971+0000 7f78ade00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f78a8105920 0x7f78a81a5990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:52634/0 (socket says 172.21.15.97:52634) 2024-03-27T10:58:21.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.971+0000 7f78ade00640 1 -- 172.21.15.97:0/1008858621 learned_addr learned my addr 172.21.15.97:0/1008858621 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:58:21.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.972+0000 7f78aca00640 1 --2- 172.21.15.97:0/1008858621 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f78a8104f60 0x7f78a819e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:21.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.972+0000 7f78ad400640 1 -- 172.21.15.97:0/1008858621 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f78a8104f60 msgr2=0x7f78a819e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:21.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.972+0000 7f78ad400640 1 --2- 172.21.15.97:0/1008858621 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f78a8104f60 0x7f78a819e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:21.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.972+0000 7f78ad400640 1 -- 172.21.15.97:0/1008858621 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f78a8105920 msgr2=0x7f78a81a5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:21.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.972+0000 7f78ad400640 1 --2- 172.21.15.97:0/1008858621 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f78a8105920 0x7f78a81a5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:21.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.972+0000 7f78ad400640 1 -- 172.21.15.97:0/1008858621 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78a81a6000 con 0x7f78a8103d70 2024-03-27T10:58:21.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.972+0000 7f78ad400640 1 --2- 172.21.15.97:0/1008858621 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78a8103d70 0x7f78a819e3e0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f78a0009bf0 tx=0x7f78a0035d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:21.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.973+0000 7f78a6000640 1 -- 172.21.15.97:0/1008858621 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78a0039040 con 0x7f78a8103d70 2024-03-27T10:58:21.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.973+0000 7f78afe00640 1 -- 172.21.15.97:0/1008858621 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78a81a6290 con 0x7f78a8103d70 2024-03-27T10:58:21.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.973+0000 7f78a6000640 1 -- 172.21.15.97:0/1008858621 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f78a0044070 con 0x7f78a8103d70 2024-03-27T10:58:21.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.973+0000 7f78afe00640 1 -- 172.21.15.97:0/1008858621 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78a81a67a0 con 0x7f78a8103d70 2024-03-27T10:58:21.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.973+0000 7f78a6000640 1 -- 172.21.15.97:0/1008858621 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78a00043a0 con 0x7f78a8103d70 2024-03-27T10:58:21.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.977+0000 7f78a6000640 1 -- 172.21.15.97:0/1008858621 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f78a0004540 con 0x7f78a8103d70 2024-03-27T10:58:21.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.977+0000 7f78b2c07740 1 Processor -- start 2024-03-27T10:58:21.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.977+0000 7f78b2c07740 1 -- start start 2024-03-27T10:58:21.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.978+0000 7f78b2c07740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56194d940f00 0x56194d941320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:21.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.978+0000 7f78b2c07740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56194d9420d0 0x56194d942550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:21.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.978+0000 7f78b2c07740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56194d942a90 0x56194d949320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:21.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.978+0000 7f78b2c07740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56194d8a6280 con 0x56194d9420d0 2024-03-27T10:58:21.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.978+0000 7f78b2c07740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x56194d84e6f0 con 0x56194d942a90 2024-03-27T10:58:21.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.978+0000 7f78b2c07740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x56194d8a2230 con 0x56194d940f00 2024-03-27T10:58:21.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.978+0000 7f78ad400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56194d9420d0 0x56194d942550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:21.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.978+0000 7f78ad400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56194d9420d0 0x56194d942550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53376/0 (socket says 172.21.15.97:53376) 2024-03-27T10:58:21.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.978+0000 7f78ad400640 1 -- 172.21.15.97:0/2488831641 learned_addr learned my addr 172.21.15.97:0/2488831641 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:58:21.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.979+0000 7f78ade00640 1 --2- 172.21.15.97:0/2488831641 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56194d940f00 0x56194d941320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:21.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.979+0000 7f78ad400640 1 -- 172.21.15.97:0/2488831641 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56194d940f00 msgr2=0x56194d941320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:21.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.979+0000 7f78ad400640 1 --2- 172.21.15.97:0/2488831641 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56194d940f00 0x56194d941320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:21.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.979+0000 7f78ad400640 1 -- 172.21.15.97:0/2488831641 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56194d942a90 msgr2=0x56194d949320 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-27T10:58:21.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.979+0000 7f78ad400640 1 --2- 172.21.15.97:0/2488831641 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56194d942a90 0x56194d949320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:21.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.979+0000 7f78ad400640 1 -- 172.21.15.97:0/2488831641 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56194d6827e0 con 0x56194d9420d0 2024-03-27T10:58:21.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.979+0000 7f78a6000640 1 --2- 172.21.15.97:0/1008858621 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f78900a3810 0x7f78900a5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:21.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.979+0000 7f78ad400640 1 --2- 172.21.15.97:0/2488831641 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56194d9420d0 0x56194d942550 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f78a00f5bf0 tx=0x7f78a00f5ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6553d9cf32b39304 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:21.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.980+0000 7f78a6000640 1 -- 172.21.15.97:0/1008858621 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f78a0004140 con 0x7f78a8103d70 2024-03-27T10:58:21.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.980+0000 7f789a600640 1 -- 172.21.15.97:0/2488831641 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78a00ba570 con 0x56194d9420d0 2024-03-27T10:58:21.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.980+0000 7f789a600640 1 -- 172.21.15.97:0/2488831641 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f78a0004380 con 0x56194d9420d0 2024-03-27T10:58:21.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.980+0000 7f789a600640 1 -- 172.21.15.97:0/2488831641 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78a00bdc30 con 0x56194d9420d0 2024-03-27T10:58:21.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.980+0000 7f78ade00640 1 --2- 172.21.15.97:0/1008858621 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f78900a3810 0x7f78900a5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:21.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.981+0000 7f78b2c07740 1 -- 172.21.15.97:0/2488831641 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56194d9420d0 msgr2=0x56194d942550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:21.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.981+0000 7f78ade00640 1 --2- 172.21.15.97:0/1008858621 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f78900a3810 0x7f78900a5cd0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x56194d941fd0 tx=0x7f789c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:21.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.981+0000 7f78b2c07740 1 --2- 172.21.15.97:0/2488831641 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56194d9420d0 0x56194d942550 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f78a00f5bf0 tx=0x7f78a00f5ce0 comp rx=0 tx=0).stop 2024-03-27T10:58:21.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.981+0000 7f78b2c07740 1 -- 172.21.15.97:0/2488831641 shutdown_connections 2024-03-27T10:58:21.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.981+0000 7f78b2c07740 1 --2- 172.21.15.97:0/2488831641 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56194d942a90 0x56194d949320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:21.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.981+0000 7f78b2c07740 1 --2- 172.21.15.97:0/2488831641 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56194d9420d0 0x56194d942550 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:21.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.981+0000 7f78b2c07740 1 --2- 172.21.15.97:0/2488831641 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56194d940f00 0x56194d941320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:21.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.981+0000 7f78b2c07740 1 -- 172.21.15.97:0/2488831641 >> 172.21.15.97:0/2488831641 conn(0x56194d93c2f0 msgr2=0x56194d93c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:21.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.981+0000 7f78b2c07740 1 -- 172.21.15.97:0/2488831641 shutdown_connections 2024-03-27T10:58:21.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.982+0000 7f78b2c07740 1 -- 172.21.15.97:0/2488831641 wait complete. 2024-03-27T10:58:21.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.982+0000 7f78b2c07740 1 -- start start 2024-03-27T10:58:21.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.982+0000 7f78b2c07740 1 Processor -- start 2024-03-27T10:58:21.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.982+0000 7f78b2c07740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56194d940f00 0x56194d94b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:21.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.983+0000 7f78b2c07740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56194d9420d0 0x56194d94b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:21.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.983+0000 7f78b2c07740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56194d942a90 0x56194d952750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:21.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.983+0000 7f78b2c07740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56194d949860 con 0x56194d942a90 2024-03-27T10:58:21.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.983+0000 7f78b2c07740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x56194d861aa0 con 0x56194d9420d0 2024-03-27T10:58:21.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.983+0000 7f78aca00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56194d9420d0 0x56194d94b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:21.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.983+0000 7f78aca00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56194d9420d0 0x56194d94b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:52656/0 (socket says 172.21.15.97:52656) 2024-03-27T10:58:21.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.983+0000 7f78aca00640 1 -- 172.21.15.97:0/3985351555 learned_addr learned my addr 172.21.15.97:0/3985351555 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:58:21.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.983+0000 7f78b2c07740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x56194d9499e0 con 0x56194d940f00 2024-03-27T10:58:21.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.983+0000 7f78b2c07740 10 client.0 _ll_register_callbacks cb 0x56194d861490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:58:21.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.983+0000 7f78ad400640 1 --2- 172.21.15.97:0/3985351555 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56194d940f00 0x56194d94b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:21.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.984+0000 7f78aca00640 1 -- 172.21.15.97:0/3985351555 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56194d940f00 msgr2=0x56194d94b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:21.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.984+0000 7f78aca00640 1 --2- 172.21.15.97:0/3985351555 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56194d940f00 0x56194d94b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:21.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.984+0000 7f78aca00640 1 -- 172.21.15.97:0/3985351555 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56194d942a90 msgr2=0x56194d952750 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-27T10:58:21.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.984+0000 7f78aca00640 1 --2- 172.21.15.97:0/3985351555 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56194d942a90 0x56194d952750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:21.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.984+0000 7f78aca00640 1 -- 172.21.15.97:0/3985351555 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56194d6e2f70 con 0x56194d9420d0 2024-03-27T10:58:21.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.985+0000 7f78aca00640 1 --2- 172.21.15.97:0/3985351555 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56194d9420d0 0x56194d94b710 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f789400bd90 tx=0x7f789400f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=627424486425020b server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:21.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.985+0000 7f788ea00640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-03-27T10:58:21.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.985+0000 7f788ea00640 1 -- 172.21.15.97:0/3985351555 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7894015070 con 0x56194d9420d0 2024-03-27T10:58:21.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.985+0000 7f788ea00640 1 -- 172.21.15.97:0/3985351555 <== mon.2 v2:172.21.15.114:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f7894004030 con 0x56194d9420d0 2024-03-27T10:58:21.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.985+0000 7f78b2c07740 1 -- 172.21.15.97:0/3985351555 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56194d8b37c0 con 0x56194d9420d0 2024-03-27T10:58:21.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.986+0000 7f788ea00640 1 -- 172.21.15.97:0/3985351555 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7894004230 con 0x56194d9420d0 2024-03-27T10:58:21.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.986+0000 7f78b2c07740 10 client.35819 fetch_fsmap learned FSMap version 30 2024-03-27T10:58:21.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.986+0000 7f78b2c07740 1 -- 172.21.15.97:0/3985351555 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56194d57a170 con 0x56194d9420d0 2024-03-27T10:58:21.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.986+0000 7f788ea00640 1 -- 172.21.15.97:0/3985351555 <== mon.2 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x56194d8b37c0 con 0x56194d9420d0 2024-03-27T10:58:21.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.987+0000 7f788ea00640 1 -- 172.21.15.97:0/3985351555 <== mon.2 v2:172.21.15.114:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f7894003850 con 0x56194d9420d0 2024-03-27T10:58:21.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.987+0000 7f78b2c07740 10 client.35819 fetch_fsmap finished waiting for FSMap version 30 2024-03-27T10:58:21.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.987+0000 7f78b2c07740 10 client.35819 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-27T10:58:21.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.987+0000 7f78b2c07740 20 client.35819 populate_metadata read hostname 'smithi097' 2024-03-27T10:58:21.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.987+0000 7f78b2c07740 1 --2- 172.21.15.97:0/3985351555 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x56194d957180 0x56194d9595e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:21.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.987+0000 7f78b2c07740 4 client.35819 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T10:58:21.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.987+0000 7f78b2c07740 1 -- 172.21.15.97:0/3985351555 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56194d8adb10 con 0x56194d957180 2024-03-27T10:58:21.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.988+0000 7f78ad400640 1 --2- 172.21.15.97:0/3985351555 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x56194d957180 0x56194d9595e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:21.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.989+0000 7f78ad400640 1 --2- 172.21.15.97:0/3985351555 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x56194d957180 0x56194d9595e0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8bbf2ea4ba2403f0 server_cookie=d8bbc0be39eaa15f in_seq=0 out_seq=0 2024-03-27T10:58:21.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.990+0000 7f788ea00640 10 client.35819 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T10:58:21.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.991+0000 7f788ea00640 1 -- 172.21.15.97:0/3985351555 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x56194d8adb10 con 0x56194d957180 2024-03-27T10:58:21.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.991+0000 7f788ea00640 10 client.35819 handle_command_reply: tid=0 2024-03-27T10:58:21.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.991+0000 7f78b2c07740 1 client.35819 shutdown 2024-03-27T10:58:21.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.992+0000 7f78b2c07740 1 -- 172.21.15.97:0/3985351555 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56194d9420d0 msgr2=0x56194d94b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:21.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.992+0000 7f78b2c07740 1 --2- 172.21.15.97:0/3985351555 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56194d9420d0 0x56194d94b710 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f789400bd90 tx=0x7f789400f730 comp rx=0 tx=0).stop 2024-03-27T10:58:21.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.992+0000 7f78b2c07740 1 -- 172.21.15.97:0/3985351555 shutdown_connections 2024-03-27T10:58:21.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.992+0000 7f78b2c07740 1 --2- 172.21.15.97:0/3985351555 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x56194d957180 0x56194d9595e0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:21.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.992+0000 7f78b2c07740 1 --2- 172.21.15.97:0/3985351555 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56194d942a90 0x56194d952750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:21.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.992+0000 7f78b2c07740 1 --2- 172.21.15.97:0/3985351555 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56194d9420d0 0x56194d94b710 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:21.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.992+0000 7f78b2c07740 1 --2- 172.21.15.97:0/3985351555 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56194d940f00 0x56194d94b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:21.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.992+0000 7f78b2c07740 1 -- 172.21.15.97:0/3985351555 >> 172.21.15.97:0/3985351555 conn(0x56194d93c2f0 msgr2=0x56194d93b550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:21.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.992+0000 7f788ea00640 0 client.35819 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T10:58:21.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.992+0000 7f78b2c07740 1 -- 172.21.15.97:0/3985351555 shutdown_connections 2024-03-27T10:58:21.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.992+0000 7f78b2c07740 1 -- 172.21.15.97:0/3985351555 wait complete. 2024-03-27T10:58:21.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:21.993+0000 7f78b2c07740 20 client.35819 trim_cache size 0 max 16384 2024-03-27T10:58:21.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.018+0000 7f78b2c07740 1 Processor -- start 2024-03-27T10:58:21.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.018+0000 7f78b2c07740 1 -- start start 2024-03-27T10:58:21.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.018+0000 7f78b2c07740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56194d940f00 0x56194d93fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:21.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.018+0000 7f78b2c07740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56194d9420d0 0x56194d952710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:21.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.018+0000 7f78b2c07740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56194d942a90 0x56194d949d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:21.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.018+0000 7f78b2c07740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f788800e430 con 0x56194d9420d0 2024-03-27T10:58:21.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.018+0000 7f78b2c07740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f7888010030 con 0x56194d942a90 2024-03-27T10:58:21.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.018+0000 7f78b2c07740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f788800eb00 con 0x56194d940f00 2024-03-27T10:58:21.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.019+0000 7f78aca00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56194d9420d0 0x56194d952710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:21.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.019+0000 7f78ade00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56194d942a90 0x56194d949d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:21.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.019+0000 7f78aca00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56194d9420d0 0x56194d952710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53390/0 (socket says 172.21.15.97:53390) 2024-03-27T10:58:21.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.019+0000 7f78aca00640 1 -- 172.21.15.97:0/1618838456 learned_addr learned my addr 172.21.15.97:0/1618838456 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:58:21.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.019+0000 7f78ad400640 1 --2- 172.21.15.97:0/1618838456 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56194d940f00 0x56194d93fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:21.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.020+0000 7f78aca00640 1 -- 172.21.15.97:0/1618838456 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56194d940f00 msgr2=0x56194d93fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:21.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.020+0000 7f78aca00640 1 --2- 172.21.15.97:0/1618838456 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56194d940f00 0x56194d93fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:21.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.020+0000 7f78aca00640 1 -- 172.21.15.97:0/1618838456 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56194d942a90 msgr2=0x56194d949d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:21.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.020+0000 7f78aca00640 1 --2- 172.21.15.97:0/1618838456 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56194d942a90 0x56194d949d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:21.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.020+0000 7f78aca00640 1 -- 172.21.15.97:0/1618838456 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56194d948f00 con 0x56194d9420d0 2024-03-27T10:58:21.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.020+0000 7f78ad400640 1 --2- 172.21.15.97:0/1618838456 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56194d940f00 0x56194d93fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T10:58:21.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.020+0000 7f78aca00640 1 --2- 172.21.15.97:0/1618838456 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56194d9420d0 0x56194d952710 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f7894001eb0 tx=0x7f7894005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ac03640b75465e3 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:21.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.020+0000 7f7899c00640 1 -- 172.21.15.97:0/1618838456 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7894032070 con 0x56194d9420d0 2024-03-27T10:58:21.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.020+0000 7f7899c00640 1 -- 172.21.15.97:0/1618838456 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7894003aa0 con 0x56194d9420d0 2024-03-27T10:58:21.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.020+0000 7f7899c00640 1 -- 172.21.15.97:0/1618838456 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7894015070 con 0x56194d9420d0 2024-03-27T10:58:21.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.021+0000 7f78b2c07740 1 -- 172.21.15.97:0/1618838456 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56194d9420d0 msgr2=0x56194d952710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:21.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.021+0000 7f78b2c07740 1 --2- 172.21.15.97:0/1618838456 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56194d9420d0 0x56194d952710 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f7894001eb0 tx=0x7f7894005710 comp rx=0 tx=0).stop 2024-03-27T10:58:21.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.021+0000 7f78b2c07740 1 -- 172.21.15.97:0/1618838456 shutdown_connections 2024-03-27T10:58:21.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.021+0000 7f78b2c07740 1 --2- 172.21.15.97:0/1618838456 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56194d942a90 0x56194d949d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:21.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.021+0000 7f78b2c07740 1 --2- 172.21.15.97:0/1618838456 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56194d9420d0 0x56194d952710 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:21.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.021+0000 7f78b2c07740 1 --2- 172.21.15.97:0/1618838456 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56194d940f00 0x56194d93fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:21.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.021+0000 7f78b2c07740 1 -- 172.21.15.97:0/1618838456 >> 172.21.15.97:0/1618838456 conn(0x56194d93c2f0 msgr2=0x56194d93f550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:21.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.022+0000 7f78b2c07740 1 -- 172.21.15.97:0/1618838456 shutdown_connections 2024-03-27T10:58:21.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.022+0000 7f78b2c07740 1 -- 172.21.15.97:0/1618838456 wait complete. 2024-03-27T10:58:21.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.022+0000 7f78b2c07740 1 -- start start 2024-03-27T10:58:21.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.023+0000 7f78b2c07740 1 Processor -- start 2024-03-27T10:58:21.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.023+0000 7f78b2c07740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56194d940f00 0x56194d950400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:21.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.023+0000 7f78b2c07740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56194d942a90 0x56194d950940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:21.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.023+0000 7f78b2c07740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56194d951950 0x56194d951dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:21.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.023+0000 7f78b2c07740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f788800f8a0 con 0x56194d940f00 2024-03-27T10:58:21.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.023+0000 7f78b2c07740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f788800f1d0 con 0x56194d942a90 2024-03-27T10:58:21.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.023+0000 7f78b2c07740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f788800c410 con 0x56194d951950 2024-03-27T10:58:21.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.023+0000 7f78b2c07740 10 client.0 _ll_register_callbacks cb 0x7f789400b920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:58:21.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.024+0000 7f78aca00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56194d942a90 0x56194d950940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:21.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.024+0000 7f78ad400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56194d940f00 0x56194d950400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:21.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.024+0000 7f78aca00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56194d942a90 0x56194d950940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:52670/0 (socket says 172.21.15.97:52670) 2024-03-27T10:58:21.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.024+0000 7f78aca00640 1 -- 172.21.15.97:0/1800349074 learned_addr learned my addr 172.21.15.97:0/1800349074 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:58:21.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.024+0000 7f78ade00640 1 --2- 172.21.15.97:0/1800349074 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56194d951950 0x56194d951dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:21.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.024+0000 7f78aca00640 1 -- 172.21.15.97:0/1800349074 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56194d951950 msgr2=0x56194d951dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:21.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.024+0000 7f78aca00640 1 --2- 172.21.15.97:0/1800349074 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56194d951950 0x56194d951dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:21.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.025+0000 7f78aca00640 1 -- 172.21.15.97:0/1800349074 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56194d940f00 msgr2=0x56194d950400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:21.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.025+0000 7f78aca00640 1 --2- 172.21.15.97:0/1800349074 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56194d940f00 0x56194d950400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:21.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.025+0000 7f78aca00640 1 -- 172.21.15.97:0/1800349074 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56194d952310 con 0x56194d942a90 2024-03-27T10:58:21.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.025+0000 7f78ad400640 1 --2- 172.21.15.97:0/1800349074 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56194d940f00 0x56194d950400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T10:58:21.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.025+0000 7f78aca00640 1 --2- 172.21.15.97:0/1800349074 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56194d942a90 0x56194d950940 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7894005780 tx=0x7f789400b450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55135af17f258b1 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:21.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.025+0000 7f788ea00640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-03-27T10:58:21.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.026+0000 7f788ea00640 1 -- 172.21.15.97:0/1800349074 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7894015070 con 0x56194d942a90 2024-03-27T10:58:21.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.026+0000 7f788ea00640 1 -- 172.21.15.97:0/1800349074 <== mon.2 v2:172.21.15.114:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f7894003f30 con 0x56194d942a90 2024-03-27T10:58:21.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.026+0000 7f78b2c07740 1 -- 172.21.15.97:0/1800349074 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7894003aa0 con 0x56194d942a90 2024-03-27T10:58:21.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.026+0000 7f788ea00640 1 -- 172.21.15.97:0/1800349074 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7894032040 con 0x56194d942a90 2024-03-27T10:58:21.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.026+0000 7f78b2c07740 10 client.35825 fetch_fsmap learned FSMap version 30 2024-03-27T10:58:21.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.026+0000 7f78b2c07740 1 -- 172.21.15.97:0/1800349074 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56194d9bdf20 con 0x56194d942a90 2024-03-27T10:58:21.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.027+0000 7f788ea00640 1 -- 172.21.15.97:0/1800349074 <== mon.2 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7894003aa0 con 0x56194d942a90 2024-03-27T10:58:21.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.027+0000 7f788ea00640 1 -- 172.21.15.97:0/1800349074 <== mon.2 v2:172.21.15.114:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f789403e030 con 0x56194d942a90 2024-03-27T10:58:21.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.027+0000 7f78b2c07740 10 client.35825 fetch_fsmap finished waiting for FSMap version 30 2024-03-27T10:58:21.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.027+0000 7f78b2c07740 10 client.35825 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-27T10:58:21.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.027+0000 7f78b2c07740 20 client.35825 populate_metadata read hostname 'smithi097' 2024-03-27T10:58:21.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.027+0000 7f78b2c07740 1 --2- 172.21.15.97:0/1800349074 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x56194d9be9c0 0x56194d9c0de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:21.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.027+0000 7f78b2c07740 4 client.35825 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-27T10:58:21.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.027+0000 7f78b2c07740 1 -- 172.21.15.97:0/1800349074 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56194d82baa0 con 0x56194d9be9c0 2024-03-27T10:58:21.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.027+0000 7f78ad400640 1 --2- 172.21.15.97:0/1800349074 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x56194d9be9c0 0x56194d9c0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:22.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.028+0000 7f788ea00640 10 client.35825 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T10:58:22.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.028+0000 7f78ad400640 1 --2- 172.21.15.97:0/1800349074 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x56194d9be9c0 0x56194d9c0de0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec17576a307e46f2 server_cookie=1c2e1163b8379177 in_seq=0 out_seq=0 2024-03-27T10:58:22.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.031+0000 7f788ea00640 1 -- 172.21.15.97:0/1800349074 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56194d82baa0 con 0x56194d9be9c0 2024-03-27T10:58:22.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.031+0000 7f788ea00640 10 client.35825 handle_command_reply: tid=0 2024-03-27T10:58:22.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.031+0000 7f78b2c07740 1 client.35825 shutdown 2024-03-27T10:58:22.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.032+0000 7f78b2c07740 1 -- 172.21.15.97:0/1800349074 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56194d942a90 msgr2=0x56194d950940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:22.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.032+0000 7f78b2c07740 1 --2- 172.21.15.97:0/1800349074 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56194d942a90 0x56194d950940 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7894005780 tx=0x7f789400b450 comp rx=0 tx=0).stop 2024-03-27T10:58:22.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.032+0000 7f78b2c07740 1 -- 172.21.15.97:0/1800349074 shutdown_connections 2024-03-27T10:58:22.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.032+0000 7f78b2c07740 1 --2- 172.21.15.97:0/1800349074 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x56194d9be9c0 0x56194d9c0de0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:22.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.032+0000 7f78b2c07740 1 --2- 172.21.15.97:0/1800349074 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56194d951950 0x56194d951dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:22.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.032+0000 7f788ea00640 0 client.35825 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T10:58:22.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.032+0000 7f78b2c07740 1 --2- 172.21.15.97:0/1800349074 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56194d942a90 0x56194d950940 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:22.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.032+0000 7f78b2c07740 1 --2- 172.21.15.97:0/1800349074 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56194d940f00 0x56194d950400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:22.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.032+0000 7f78b2c07740 1 -- 172.21.15.97:0/1800349074 >> 172.21.15.97:0/1800349074 conn(0x56194d93c2f0 msgr2=0x56194d93fde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:22.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.032+0000 7f78b2c07740 1 -- 172.21.15.97:0/1800349074 shutdown_connections 2024-03-27T10:58:22.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.033+0000 7f78b2c07740 1 -- 172.21.15.97:0/1800349074 wait complete. 2024-03-27T10:58:22.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.033+0000 7f78b2c07740 20 client.35825 trim_cache size 0 max 16384 2024-03-27T10:58:22.004 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-03-27T10:58:22.004 INFO:teuthology.orchestra.run.smithi097.stdout: "return_code": 0, 2024-03-27T10:58:22.004 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_tag": "a0f9dce1-c2cf-4f12-a3e5-272846ff044f", 2024-03-27T10:58:22.004 INFO:teuthology.orchestra.run.smithi097.stdout: "mode": "asynchronous" 2024-03-27T10:58:22.005 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-03-27T10:58:22.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.035+0000 7f78afe00640 1 -- 172.21.15.97:0/1008858621 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f78900a3810 msgr2=0x7f78900a5cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:22.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.035+0000 7f78afe00640 1 --2- 172.21.15.97:0/1008858621 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f78900a3810 0x7f78900a5cd0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x56194d941fd0 tx=0x7f789c005f90 comp rx=0 tx=0).stop 2024-03-27T10:58:22.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.035+0000 7f78afe00640 1 -- 172.21.15.97:0/1008858621 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78a8103d70 msgr2=0x7f78a819e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:22.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.035+0000 7f78afe00640 1 --2- 172.21.15.97:0/1008858621 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78a8103d70 0x7f78a819e3e0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f78a0009bf0 tx=0x7f78a0035d60 comp rx=0 tx=0).stop 2024-03-27T10:58:22.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.035+0000 7f78afe00640 1 -- 172.21.15.97:0/1008858621 shutdown_connections 2024-03-27T10:58:22.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.035+0000 7f78afe00640 1 --2- 172.21.15.97:0/1008858621 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f78900a3810 0x7f78900a5cd0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:22.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.035+0000 7f78afe00640 1 --2- 172.21.15.97:0/1008858621 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f78a8105920 0x7f78a81a5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:22.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.035+0000 7f78afe00640 1 --2- 172.21.15.97:0/1008858621 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f78a8104f60 0x7f78a819e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:22.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.035+0000 7f78afe00640 1 --2- 172.21.15.97:0/1008858621 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78a8103d70 0x7f78a819e3e0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:22.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.035+0000 7f78afe00640 1 -- 172.21.15.97:0/1008858621 >> 172.21.15.97:0/1008858621 conn(0x7f78a80ff2c0 msgr2=0x7f78a8077fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:22.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.035+0000 7f78afe00640 1 -- 172.21.15.97:0/1008858621 shutdown_connections 2024-03-27T10:58:22.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.035+0000 7f78afe00640 1 -- 172.21.15.97:0/1008858621 wait complete. 2024-03-27T10:58:22.032 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-27T10:58:22.032 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-27T10:58:22.266 INFO:journalctl@ceph.mds.h.smithi114.stdout:Mar 27 10:58:22 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-h[65555]: 2024-03-27T10:58:22.036+0000 7fd7ef800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000204 [2,head] /volumes/qa/sv_0/22bc04cd-8106-4d11-83f1-2aead5019d07/client.0/tmp/iogen_3.1p0/src/iogen.c auth v8 ap=1 s=19072 n(v0 rc2024-03-27T10:58:04.437264+0000 b19072 1=1+0) caps={15363=pAsLsXsFscr/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x563b8a1a5180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000204:[<0x10000000202/iogen.c v8>,<0x10000000201/src v58>,<0x10000000006/iogen_3.1p0 v128>,<0x10000000005/tmp v20>,<0x10000000003/client.0 v14>,<0x10000000002/22bc04cd-8106-4d11-83f1-2aead5019d07 v30>,<0x10000000001/sv_0 v53>,<0x10000000000/qa v41>,<0x1/volumes v39>]//[]","memoryvalue":"(3)0x10000000204:[<0x10000000202/iogen.c v8>,<0x10000000201/src v133>,<0x10000000006/iogen_3.1p0 v130>,<0x10000000005/tmp v32>,<0x10000000003/client.0 v22>,<0x10000000002/22bc04cd-8106-4d11-83f1-2aead5019d07 v24>,<0x10000000001/sv_0 v59>,<0x10000000000/qa v25>,<0x1/volumes v23>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-27T10:58:22.266 INFO:journalctl@ceph.mds.h.smithi114.stdout:Mar 27 10:58:22 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-h[65555]: 2024-03-27T10:58:22.036+0000 7fd7ef800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000203 [2,head] /volumes/qa/sv_0/22bc04cd-8106-4d11-83f1-2aead5019d07/client.0/tmp/iogen_3.1p0/src/iogen.8 auth v12 ap=1 s=4783 n(v0 rc2024-03-27T10:58:04.425106+0000 b4783 1=1+0) caps={15363=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x563b8a1a5700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000203:[<0x10000000202/iogen.8 v12>,<0x10000000201/src v58>,<0x10000000006/iogen_3.1p0 v128>,<0x10000000005/tmp v20>,<0x10000000003/client.0 v14>,<0x10000000002/22bc04cd-8106-4d11-83f1-2aead5019d07 v30>,<0x10000000001/sv_0 v53>,<0x10000000000/qa v41>,<0x1/volumes v39>]//[]","memoryvalue":"(3)0x10000000203:[<0x10000000202/iogen.8 v12>,<0x10000000201/src v133>,<0x10000000006/iogen_3.1p0 v130>,<0x10000000005/tmp v32>,<0x10000000003/client.0 v22>,<0x10000000002/22bc04cd-8106-4d11-83f1-2aead5019d07 v24>,<0x10000000001/sv_0 v59>,<0x10000000000/qa v25>,<0x1/volumes v23>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-27T10:58:22.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.337+0000 7ffa86400640 1 -- 172.21.15.97:0/322063862 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffa80100c60 msgr2=0x7ffa801010c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:22.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.337+0000 7ffa86400640 1 --2- 172.21.15.97:0/322063862 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffa80100c60 0x7ffa801010c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c009860 tx=0x7ffa6c02bb70 comp rx=0 tx=0).stop 2024-03-27T10:58:22.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.337+0000 7ffa86400640 1 -- 172.21.15.97:0/322063862 shutdown_connections 2024-03-27T10:58:22.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.337+0000 7ffa86400640 1 --2- 172.21.15.97:0/322063862 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa80101600 0x7ffa80101a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:22.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.337+0000 7ffa86400640 1 --2- 172.21.15.97:0/322063862 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffa80100c60 0x7ffa801010c0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:22.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.337+0000 7ffa86400640 1 --2- 172.21.15.97:0/322063862 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffa80100300 0x7ffa80100720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:22.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.337+0000 7ffa86400640 1 -- 172.21.15.97:0/322063862 >> 172.21.15.97:0/322063862 conn(0x7ffa800fbec0 msgr2=0x7ffa800fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:22.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.337+0000 7ffa86400640 1 -- 172.21.15.97:0/322063862 shutdown_connections 2024-03-27T10:58:22.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.337+0000 7ffa86400640 1 -- 172.21.15.97:0/322063862 wait complete. 2024-03-27T10:58:22.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.339+0000 7ffa86400640 1 Processor -- start 2024-03-27T10:58:22.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.339+0000 7ffa86400640 1 -- start start 2024-03-27T10:58:22.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.340+0000 7ffa86400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa80100300 0x7ffa8019e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:22.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.340+0000 7ffa86400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffa80100c60 0x7ffa8019e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:22.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.340+0000 7ffa86400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffa80101600 0x7ffa801a5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:22.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.340+0000 7ffa86400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ffa801041b0 con 0x7ffa80100300 2024-03-27T10:58:22.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.340+0000 7ffa86400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ffa80104030 con 0x7ffa80100c60 2024-03-27T10:58:22.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.340+0000 7ffa86400640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ffa80104330 con 0x7ffa80101600 2024-03-27T10:58:22.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.340+0000 7ffa7fe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffa80100c60 0x7ffa8019e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:22.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.340+0000 7ffa7fe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffa80100c60 0x7ffa8019e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:52696/0 (socket says 172.21.15.97:52696) 2024-03-27T10:58:22.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.340+0000 7ffa7fe00640 1 -- 172.21.15.97:0/3083486108 learned_addr learned my addr 172.21.15.97:0/3083486108 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:58:22.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.341+0000 7ffa85a00640 1 --2- 172.21.15.97:0/3083486108 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffa80101600 0x7ffa801a5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:22.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.341+0000 7ffa85000640 1 --2- 172.21.15.97:0/3083486108 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa80100300 0x7ffa8019e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:22.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.341+0000 7ffa7fe00640 1 -- 172.21.15.97:0/3083486108 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffa80101600 msgr2=0x7ffa801a5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:22.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.341+0000 7ffa7fe00640 1 --2- 172.21.15.97:0/3083486108 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffa80101600 0x7ffa801a5800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:22.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.341+0000 7ffa7fe00640 1 -- 172.21.15.97:0/3083486108 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa80100300 msgr2=0x7ffa8019e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:22.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.341+0000 7ffa7fe00640 1 --2- 172.21.15.97:0/3083486108 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa80100300 0x7ffa8019e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:22.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.341+0000 7ffa7fe00640 1 -- 172.21.15.97:0/3083486108 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa801a5e70 con 0x7ffa80100c60 2024-03-27T10:58:22.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.342+0000 7ffa7fe00640 1 --2- 172.21.15.97:0/3083486108 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffa80100c60 0x7ffa8019e790 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c005940 tx=0x7ffa6c004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:22.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.343+0000 7ffa7d600640 1 -- 172.21.15.97:0/3083486108 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6c041020 con 0x7ffa80100c60 2024-03-27T10:58:22.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.343+0000 7ffa86400640 1 -- 172.21.15.97:0/3083486108 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa801a6100 con 0x7ffa80100c60 2024-03-27T10:58:22.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.343+0000 7ffa7d600640 1 -- 172.21.15.97:0/3083486108 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffa6c02c9c0 con 0x7ffa80100c60 2024-03-27T10:58:22.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.344+0000 7ffa7d600640 1 -- 172.21.15.97:0/3083486108 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6c039040 con 0x7ffa80100c60 2024-03-27T10:58:22.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.344+0000 7ffa86400640 1 -- 172.21.15.97:0/3083486108 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa801a6640 con 0x7ffa80100c60 2024-03-27T10:58:22.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.350+0000 7ffa8a7c6740 1 Processor -- start 2024-03-27T10:58:22.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.351+0000 7ffa7d600640 1 -- 172.21.15.97:0/3083486108 <== mon.2 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7ffa6c044020 con 0x7ffa80100c60 2024-03-27T10:58:22.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.351+0000 7ffa8a7c6740 1 -- start start 2024-03-27T10:58:22.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.351+0000 7ffa8a7c6740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bd80213670 0x55bd80213a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:22.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.351+0000 7ffa8a7c6740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd80214060 0x55bd80218b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:22.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.352+0000 7ffa8a7c6740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd802191d0 0x55bd8021b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:22.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.352+0000 7ffa8a7c6740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55bd801796b0 con 0x55bd80214060 2024-03-27T10:58:22.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.352+0000 7ffa8a7c6740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55bd80121750 con 0x55bd802191d0 2024-03-27T10:58:22.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.352+0000 7ffa8a7c6740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55bd80175660 con 0x55bd80213670 2024-03-27T10:58:22.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.352+0000 7ffa85a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bd80213670 0x55bd80213a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:22.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.353+0000 7ffa85a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bd80213670 0x55bd80213a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.97:45612/0 (socket says 172.21.15.97:45612) 2024-03-27T10:58:22.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.353+0000 7ffa85a00640 1 -- 172.21.15.97:0/2128504625 learned_addr learned my addr 172.21.15.97:0/2128504625 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:58:22.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.353+0000 7ffa7d600640 1 --2- 172.21.15.97:0/3083486108 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7ffa680a3690 0x7ffa680a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:22.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.353+0000 7ffa7fe00640 1 --2- 172.21.15.97:0/2128504625 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd802191d0 0x55bd8021b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:22.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.353+0000 7ffa7d600640 1 -- 172.21.15.97:0/3083486108 <== mon.2 v2:172.21.15.114:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7ffa6c0f1130 con 0x7ffa80100c60 2024-03-27T10:58:22.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.354+0000 7ffa85000640 1 --2- 172.21.15.97:0/2128504625 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd80214060 0x55bd80218b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:22.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.354+0000 7ffa7fe00640 1 -- 172.21.15.97:0/2128504625 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bd80213670 msgr2=0x55bd80213a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:22.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.354+0000 7ffa7fe00640 1 --2- 172.21.15.97:0/2128504625 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bd80213670 0x55bd80213a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:22.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.354+0000 7ffa7fe00640 1 -- 172.21.15.97:0/2128504625 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd80214060 msgr2=0x55bd80218b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:22.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.355+0000 7ffa7fe00640 1 --2- 172.21.15.97:0/2128504625 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd80214060 0x55bd80218b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:22.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.355+0000 7ffa7fe00640 1 -- 172.21.15.97:0/2128504625 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd7ff577d0 con 0x55bd802191d0 2024-03-27T10:58:22.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.355+0000 7ffa85000640 1 --2- 172.21.15.97:0/2128504625 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd80214060 0x55bd80218b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T10:58:22.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.355+0000 7ffa85a00640 1 --2- 172.21.15.97:0/3083486108 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7ffa680a3690 0x7ffa680a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:22.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.355+0000 7ffa7fe00640 1 --2- 172.21.15.97:0/2128504625 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd802191d0 0x55bd8021b660 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c0eec90 tx=0x7ffa6c005750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7ca07a462521c5a server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:22.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.355+0000 7ffa72000640 1 -- 172.21.15.97:0/2128504625 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6c003e60 con 0x55bd802191d0 2024-03-27T10:58:22.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.355+0000 7ffa72000640 1 -- 172.21.15.97:0/2128504625 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffa6c0b9040 con 0x55bd802191d0 2024-03-27T10:58:22.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.355+0000 7ffa72000640 1 -- 172.21.15.97:0/2128504625 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6c0bb910 con 0x55bd802191d0 2024-03-27T10:58:22.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.356+0000 7ffa8a7c6740 1 -- 172.21.15.97:0/2128504625 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd802191d0 msgr2=0x55bd8021b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:22.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.356+0000 7ffa8a7c6740 1 --2- 172.21.15.97:0/2128504625 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd802191d0 0x55bd8021b660 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c0eec90 tx=0x7ffa6c005750 comp rx=0 tx=0).stop 2024-03-27T10:58:22.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.356+0000 7ffa85a00640 1 --2- 172.21.15.97:0/3083486108 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7ffa680a3690 0x7ffa680a5b50 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x55bd802148c0 tx=0x7ffa78004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:22.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.357+0000 7ffa8a7c6740 1 -- 172.21.15.97:0/2128504625 shutdown_connections 2024-03-27T10:58:22.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.357+0000 7ffa8a7c6740 1 --2- 172.21.15.97:0/2128504625 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd802191d0 0x55bd8021b660 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:22.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.357+0000 7ffa8a7c6740 1 --2- 172.21.15.97:0/2128504625 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd80214060 0x55bd80218b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:22.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.357+0000 7ffa8a7c6740 1 --2- 172.21.15.97:0/2128504625 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bd80213670 0x55bd80213a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:22.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.357+0000 7ffa8a7c6740 1 -- 172.21.15.97:0/2128504625 >> 172.21.15.97:0/2128504625 conn(0x55bd8020ebe0 msgr2=0x55bd8020efe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:22.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.357+0000 7ffa8a7c6740 1 -- 172.21.15.97:0/2128504625 shutdown_connections 2024-03-27T10:58:22.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.357+0000 7ffa8a7c6740 1 -- 172.21.15.97:0/2128504625 wait complete. 2024-03-27T10:58:22.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.357+0000 7ffa8a7c6740 1 -- start start 2024-03-27T10:58:22.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.358+0000 7ffa8a7c6740 1 Processor -- start 2024-03-27T10:58:22.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.358+0000 7ffa8a7c6740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd80213670 0x55bd8021d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:22.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.358+0000 7ffa8a7c6740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bd80214060 0x55bd8021dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:22.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.358+0000 7ffa8a7c6740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd802191d0 0x55bd80224da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:22.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.358+0000 7ffa8a7c6740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55bd8021bf40 con 0x55bd80213670 2024-03-27T10:58:22.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.358+0000 7ffa8a7c6740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55bd80134a80 con 0x55bd802191d0 2024-03-27T10:58:22.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.358+0000 7ffa8a7c6740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55bd8021c0c0 con 0x55bd80214060 2024-03-27T10:58:22.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.359+0000 7ffa85000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd80213670 0x55bd8021d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:22.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.359+0000 7ffa85a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd802191d0 0x55bd80224da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:22.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.359+0000 7ffa85a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd802191d0 0x55bd80224da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:52710/0 (socket says 172.21.15.97:52710) 2024-03-27T10:58:22.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.359+0000 7ffa85a00640 1 -- 172.21.15.97:0/1406783975 learned_addr learned my addr 172.21.15.97:0/1406783975 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:58:22.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.359+0000 7ffa7fe00640 1 --2- 172.21.15.97:0/1406783975 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bd80214060 0x55bd8021dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:22.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.359+0000 7ffa8a7c6740 10 client.0 _ll_register_callbacks cb 0x55bd80134470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:58:22.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.360+0000 7ffa85a00640 1 -- 172.21.15.97:0/1406783975 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bd80214060 msgr2=0x55bd8021dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:22.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.360+0000 7ffa85a00640 1 --2- 172.21.15.97:0/1406783975 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bd80214060 0x55bd8021dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:22.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.360+0000 7ffa85a00640 1 -- 172.21.15.97:0/1406783975 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd80213670 msgr2=0x55bd8021d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:22.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.360+0000 7ffa85a00640 1 --2- 172.21.15.97:0/1406783975 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd80213670 0x55bd8021d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:22.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.360+0000 7ffa85a00640 1 -- 172.21.15.97:0/1406783975 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd7fe8d3f0 con 0x55bd802191d0 2024-03-27T10:58:22.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.360+0000 7ffa85000640 1 --2- 172.21.15.97:0/1406783975 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd80213670 0x55bd8021d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T10:58:22.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.360+0000 7ffa7fe00640 1 --2- 172.21.15.97:0/1406783975 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bd80214060 0x55bd8021dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T10:58:22.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.360+0000 7ffa85a00640 1 --2- 172.21.15.97:0/1406783975 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd802191d0 0x55bd80224da0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ffa78009880 tx=0x7ffa78004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9b270b53d44e4c2 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:22.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.360+0000 7ffa66a00640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-03-27T10:58:22.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.361+0000 7ffa66a00640 1 -- 172.21.15.97:0/1406783975 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa78018050 con 0x55bd802191d0 2024-03-27T10:58:22.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.361+0000 7ffa66a00640 1 -- 172.21.15.97:0/1406783975 <== mon.2 v2:172.21.15.114:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7ffa7800f030 con 0x55bd802191d0 2024-03-27T10:58:22.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.361+0000 7ffa8a7c6740 1 -- 172.21.15.97:0/1406783975 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa6c0b9040 con 0x55bd802191d0 2024-03-27T10:58:22.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.362+0000 7ffa66a00640 1 -- 172.21.15.97:0/1406783975 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ffa78020e20 con 0x55bd802191d0 2024-03-27T10:58:22.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.362+0000 7ffa8a7c6740 10 client.35837 fetch_fsmap learned FSMap version 30 2024-03-27T10:58:22.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.362+0000 7ffa8a7c6740 1 -- 172.21.15.97:0/1406783975 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bd7fe4d0f0 con 0x55bd802191d0 2024-03-27T10:58:22.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.362+0000 7ffa66a00640 1 -- 172.21.15.97:0/1406783975 <== mon.2 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffa6c0b9040 con 0x55bd802191d0 2024-03-27T10:58:22.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.363+0000 7ffa66a00640 1 -- 172.21.15.97:0/1406783975 <== mon.2 v2:172.21.15.114:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7ffa7800b040 con 0x55bd802191d0 2024-03-27T10:58:22.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.363+0000 7ffa8a7c6740 10 client.35837 fetch_fsmap finished waiting for FSMap version 30 2024-03-27T10:58:22.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.363+0000 7ffa8a7c6740 10 client.35837 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-27T10:58:22.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.363+0000 7ffa8a7c6740 20 client.35837 populate_metadata read hostname 'smithi097' 2024-03-27T10:58:22.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.363+0000 7ffa8a7c6740 1 --2- 172.21.15.97:0/1406783975 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55bd8023a740 0x55bd8023cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:22.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.363+0000 7ffa8a7c6740 4 client.35837 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T10:58:22.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.363+0000 7ffa8a7c6740 1 -- 172.21.15.97:0/1406783975 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bd80186bf0 con 0x55bd8023a740 2024-03-27T10:58:22.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.364+0000 7ffa85000640 1 --2- 172.21.15.97:0/1406783975 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55bd8023a740 0x55bd8023cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:22.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.365+0000 7ffa85000640 1 --2- 172.21.15.97:0/1406783975 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55bd8023a740 0x55bd8023cbe0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a464d1871c7255e server_cookie=4adec124213a93cd in_seq=0 out_seq=0 2024-03-27T10:58:22.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.365+0000 7ffa66a00640 10 client.35837 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T10:58:22.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.366+0000 7ffa66a00640 1 -- 172.21.15.97:0/1406783975 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55bd80186bf0 con 0x55bd8023a740 2024-03-27T10:58:22.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.366+0000 7ffa66a00640 10 client.35837 handle_command_reply: tid=0 2024-03-27T10:58:22.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.366+0000 7ffa8a7c6740 1 client.35837 shutdown 2024-03-27T10:58:22.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.367+0000 7ffa8a7c6740 1 -- 172.21.15.97:0/1406783975 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd802191d0 msgr2=0x55bd80224da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:22.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.367+0000 7ffa8a7c6740 1 --2- 172.21.15.97:0/1406783975 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd802191d0 0x55bd80224da0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ffa78009880 tx=0x7ffa78004a40 comp rx=0 tx=0).stop 2024-03-27T10:58:22.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.367+0000 7ffa8a7c6740 1 -- 172.21.15.97:0/1406783975 shutdown_connections 2024-03-27T10:58:22.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.367+0000 7ffa8a7c6740 1 --2- 172.21.15.97:0/1406783975 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55bd8023a740 0x55bd8023cbe0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:22.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.367+0000 7ffa8a7c6740 1 --2- 172.21.15.97:0/1406783975 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd802191d0 0x55bd80224da0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:22.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.367+0000 7ffa8a7c6740 1 --2- 172.21.15.97:0/1406783975 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bd80214060 0x55bd8021dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:22.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.367+0000 7ffa8a7c6740 1 --2- 172.21.15.97:0/1406783975 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd80213670 0x55bd8021d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:22.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.367+0000 7ffa8a7c6740 1 -- 172.21.15.97:0/1406783975 >> 172.21.15.97:0/1406783975 conn(0x55bd8020ebe0 msgr2=0x55bd8020de40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:22.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.367+0000 7ffa66a00640 0 client.35837 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T10:58:22.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.367+0000 7ffa8a7c6740 1 -- 172.21.15.97:0/1406783975 shutdown_connections 2024-03-27T10:58:22.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.368+0000 7ffa8a7c6740 1 -- 172.21.15.97:0/1406783975 wait complete. 2024-03-27T10:58:22.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.368+0000 7ffa8a7c6740 20 client.35837 trim_cache size 0 max 16384 2024-03-27T10:58:22.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.402+0000 7ffa8a7c6740 1 Processor -- start 2024-03-27T10:58:22.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.402+0000 7ffa8a7c6740 1 -- start start 2024-03-27T10:58:22.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.402+0000 7ffa8a7c6740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd80213670 0x55bd80215500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:22.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.402+0000 7ffa8a7c6740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd80214060 0x55bd80215a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:22.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.402+0000 7ffa8a7c6740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bd802191d0 0x55bd80224d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:22.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.403+0000 7ffa8a7c6740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ffa801a6af0 con 0x55bd80213670 2024-03-27T10:58:22.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.403+0000 7ffa8a7c6740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ffa801a86f0 con 0x55bd80214060 2024-03-27T10:58:22.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.403+0000 7ffa8a7c6740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ffa801a71c0 con 0x55bd802191d0 2024-03-27T10:58:22.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.403+0000 7ffa85a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bd802191d0 0x55bd80224d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:22.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.403+0000 7ffa7fe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd80214060 0x55bd80215a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:22.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.403+0000 7ffa85a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bd802191d0 0x55bd80224d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.97:45618/0 (socket says 172.21.15.97:45618) 2024-03-27T10:58:22.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.403+0000 7ffa85a00640 1 -- 172.21.15.97:0/292101843 learned_addr learned my addr 172.21.15.97:0/292101843 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:58:22.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.404+0000 7ffa85000640 1 --2- 172.21.15.97:0/292101843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd80213670 0x55bd80215500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:22.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.404+0000 7ffa85a00640 1 -- 172.21.15.97:0/292101843 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd80214060 msgr2=0x55bd80215a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:22.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.404+0000 7ffa85a00640 1 --2- 172.21.15.97:0/292101843 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd80214060 0x55bd80215a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:22.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.404+0000 7ffa85a00640 1 -- 172.21.15.97:0/292101843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd80213670 msgr2=0x55bd80215500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:22.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.404+0000 7ffa85a00640 1 --2- 172.21.15.97:0/292101843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd80213670 0x55bd80215500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:22.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.404+0000 7ffa85a00640 1 -- 172.21.15.97:0/292101843 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd8021cc40 con 0x55bd802191d0 2024-03-27T10:58:22.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.405+0000 7ffa85a00640 1 --2- 172.21.15.97:0/292101843 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bd802191d0 0x55bd80224d60 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ffa7800b7a0 tx=0x7ffa7800b250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c17093cc062f73e1 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:22.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.405+0000 7ffa71600640 1 -- 172.21.15.97:0/292101843 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa78037070 con 0x55bd802191d0 2024-03-27T10:58:22.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.405+0000 7ffa71600640 1 -- 172.21.15.97:0/292101843 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffa78019040 con 0x55bd802191d0 2024-03-27T10:58:22.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.405+0000 7ffa71600640 1 -- 172.21.15.97:0/292101843 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa78018070 con 0x55bd802191d0 2024-03-27T10:58:22.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.406+0000 7ffa8a7c6740 1 -- 172.21.15.97:0/292101843 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bd802191d0 msgr2=0x55bd80224d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:22.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.406+0000 7ffa8a7c6740 1 --2- 172.21.15.97:0/292101843 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bd802191d0 0x55bd80224d60 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ffa7800b7a0 tx=0x7ffa7800b250 comp rx=0 tx=0).stop 2024-03-27T10:58:22.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.406+0000 7ffa8a7c6740 1 -- 172.21.15.97:0/292101843 shutdown_connections 2024-03-27T10:58:22.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.406+0000 7ffa8a7c6740 1 --2- 172.21.15.97:0/292101843 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bd802191d0 0x55bd80224d60 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:22.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.406+0000 7ffa8a7c6740 1 --2- 172.21.15.97:0/292101843 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd80214060 0x55bd80215a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:22.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.406+0000 7ffa8a7c6740 1 --2- 172.21.15.97:0/292101843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd80213670 0x55bd80215500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:22.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.406+0000 7ffa8a7c6740 1 -- 172.21.15.97:0/292101843 >> 172.21.15.97:0/292101843 conn(0x55bd8020ebe0 msgr2=0x55bd8021d820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:22.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.407+0000 7ffa8a7c6740 1 -- 172.21.15.97:0/292101843 shutdown_connections 2024-03-27T10:58:22.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.407+0000 7ffa8a7c6740 1 -- 172.21.15.97:0/292101843 wait complete. 2024-03-27T10:58:22.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.407+0000 7ffa8a7c6740 1 -- start start 2024-03-27T10:58:22.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.408+0000 7ffa8a7c6740 1 Processor -- start 2024-03-27T10:58:22.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.408+0000 7ffa8a7c6740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd80213670 0x55bd80222030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:22.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.408+0000 7ffa8a7c6740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bd80214060 0x55bd80222570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:22.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.408+0000 7ffa8a7c6740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd802191d0 0x55bd80223580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:22.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.408+0000 7ffa8a7c6740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ffa801a7f60 con 0x55bd802191d0 2024-03-27T10:58:22.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.408+0000 7ffa8a7c6740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ffa801a7890 con 0x55bd80213670 2024-03-27T10:58:22.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.408+0000 7ffa8a7c6740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ffa80114510 con 0x55bd80214060 2024-03-27T10:58:22.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.408+0000 7ffa8a7c6740 10 client.0 _ll_register_callbacks cb 0x7ffa7800bae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:58:22.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.409+0000 7ffa85000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd80213670 0x55bd80222030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:22.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.409+0000 7ffa85000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd80213670 0x55bd80222030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:52722/0 (socket says 172.21.15.97:52722) 2024-03-27T10:58:22.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.409+0000 7ffa85000640 1 -- 172.21.15.97:0/892819346 learned_addr learned my addr 172.21.15.97:0/892819346 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:58:22.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.409+0000 7ffa7fe00640 1 --2- 172.21.15.97:0/892819346 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bd80214060 0x55bd80222570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:22.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.409+0000 7ffa85a00640 1 --2- 172.21.15.97:0/892819346 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd802191d0 0x55bd80223580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:22.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.409+0000 7ffa85000640 1 -- 172.21.15.97:0/892819346 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bd80214060 msgr2=0x55bd80222570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:22.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.409+0000 7ffa85000640 1 --2- 172.21.15.97:0/892819346 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bd80214060 0x55bd80222570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:22.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.409+0000 7ffa85000640 1 -- 172.21.15.97:0/892819346 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd802191d0 msgr2=0x55bd80223580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:22.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.409+0000 7ffa85000640 1 --2- 172.21.15.97:0/892819346 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd802191d0 0x55bd80223580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:22.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.409+0000 7ffa85000640 1 -- 172.21.15.97:0/892819346 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd80223b70 con 0x55bd80213670 2024-03-27T10:58:22.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.409+0000 7ffa85000640 1 --2- 172.21.15.97:0/892819346 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd80213670 0x55bd80222030 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x55bd80214460 tx=0x7ffa74011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c971227d1f77f9b server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:22.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.409+0000 7ffa66a00640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-03-27T10:58:22.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.410+0000 7ffa66a00640 1 -- 172.21.15.97:0/892819346 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa7401d5b0 con 0x55bd80213670 2024-03-27T10:58:22.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.410+0000 7ffa66a00640 1 -- 172.21.15.97:0/892819346 <== mon.2 v2:172.21.15.114:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7ffa7401d750 con 0x55bd80213670 2024-03-27T10:58:22.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.410+0000 7ffa8a7c6740 1 -- 172.21.15.97:0/892819346 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa78019040 con 0x55bd80213670 2024-03-27T10:58:22.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.410+0000 7ffa66a00640 1 -- 172.21.15.97:0/892819346 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ffa7402d050 con 0x55bd80213670 2024-03-27T10:58:22.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.410+0000 7ffa8a7c6740 10 client.35843 fetch_fsmap learned FSMap version 30 2024-03-27T10:58:22.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.410+0000 7ffa8a7c6740 1 -- 172.21.15.97:0/892819346 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bd80224ef0 con 0x55bd80213670 2024-03-27T10:58:22.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.411+0000 7ffa66a00640 1 -- 172.21.15.97:0/892819346 <== mon.2 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffa78019040 con 0x55bd80213670 2024-03-27T10:58:22.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.411+0000 7ffa66a00640 1 -- 172.21.15.97:0/892819346 <== mon.2 v2:172.21.15.114:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7ffa74025450 con 0x55bd80213670 2024-03-27T10:58:22.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.411+0000 7ffa8a7c6740 10 client.35843 fetch_fsmap finished waiting for FSMap version 30 2024-03-27T10:58:22.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.411+0000 7ffa8a7c6740 10 client.35843 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-27T10:58:22.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.411+0000 7ffa8a7c6740 20 client.35843 populate_metadata read hostname 'smithi097' 2024-03-27T10:58:22.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.411+0000 7ffa8a7c6740 1 --2- 172.21.15.97:0/892819346 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55bd802a1260 0x55bd802a3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:22.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.411+0000 7ffa8a7c6740 4 client.35843 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-27T10:58:22.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.411+0000 7ffa8a7c6740 1 -- 172.21.15.97:0/892819346 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55bd80180f40 con 0x55bd802a1260 2024-03-27T10:58:22.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.412+0000 7ffa7fe00640 1 --2- 172.21.15.97:0/892819346 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55bd802a1260 0x55bd802a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:22.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.413+0000 7ffa7fe00640 1 --2- 172.21.15.97:0/892819346 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55bd802a1260 0x55bd802a3680 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2b65701f73df115 server_cookie=6b53c1a1e24d952a in_seq=0 out_seq=0 2024-03-27T10:58:22.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.413+0000 7ffa66a00640 10 client.35843 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T10:58:22.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.413+0000 7ffa66a00640 1 -- 172.21.15.97:0/892819346 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55bd80180f40 con 0x55bd802a1260 2024-03-27T10:58:22.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.413+0000 7ffa66a00640 10 client.35843 handle_command_reply: tid=0 2024-03-27T10:58:22.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.413+0000 7ffa8a7c6740 1 client.35843 shutdown 2024-03-27T10:58:22.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.413+0000 7ffa8a7c6740 1 -- 172.21.15.97:0/892819346 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd80213670 msgr2=0x55bd80222030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:22.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.413+0000 7ffa8a7c6740 1 --2- 172.21.15.97:0/892819346 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd80213670 0x55bd80222030 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x55bd80214460 tx=0x7ffa74011810 comp rx=0 tx=0).stop 2024-03-27T10:58:22.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.413+0000 7ffa8a7c6740 1 -- 172.21.15.97:0/892819346 shutdown_connections 2024-03-27T10:58:22.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.413+0000 7ffa8a7c6740 1 --2- 172.21.15.97:0/892819346 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55bd802a1260 0x55bd802a3680 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:22.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.413+0000 7ffa8a7c6740 1 --2- 172.21.15.97:0/892819346 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd802191d0 0x55bd80223580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:22.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.413+0000 7ffa8a7c6740 1 --2- 172.21.15.97:0/892819346 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bd80214060 0x55bd80222570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:22.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.413+0000 7ffa8a7c6740 1 --2- 172.21.15.97:0/892819346 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd80213670 0x55bd80222030 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:22.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.414+0000 7ffa8a7c6740 1 -- 172.21.15.97:0/892819346 >> 172.21.15.97:0/892819346 conn(0x55bd8020ebe0 msgr2=0x55bd8021d790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:22.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.414+0000 7ffa66a00640 0 client.35843 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T10:58:22.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.414+0000 7ffa8a7c6740 1 -- 172.21.15.97:0/892819346 shutdown_connections 2024-03-27T10:58:22.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.414+0000 7ffa8a7c6740 1 -- 172.21.15.97:0/892819346 wait complete. 2024-03-27T10:58:22.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.414+0000 7ffa8a7c6740 20 client.35843 trim_cache size 0 max 16384 2024-03-27T10:58:22.387 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-03-27T10:58:22.387 INFO:teuthology.orchestra.run.smithi097.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-27T10:58:22.387 INFO:teuthology.orchestra.run.smithi097.stdout: "scrubs": { 2024-03-27T10:58:22.387 INFO:teuthology.orchestra.run.smithi097.stdout: "a0f9dce1-c2cf-4f12-a3e5-272846ff044f": { 2024-03-27T10:58:22.387 INFO:teuthology.orchestra.run.smithi097.stdout: "path": "/", 2024-03-27T10:58:22.387 INFO:teuthology.orchestra.run.smithi097.stdout: "tag": "a0f9dce1-c2cf-4f12-a3e5-272846ff044f", 2024-03-27T10:58:22.387 INFO:teuthology.orchestra.run.smithi097.stdout: "options": "recursive,force" 2024-03-27T10:58:22.387 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T10:58:22.387 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T10:58:22.387 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-03-27T10:58:22.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.415+0000 7ffa86400640 1 -- 172.21.15.97:0/3083486108 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7ffa680a3690 msgr2=0x7ffa680a5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:22.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.415+0000 7ffa86400640 1 --2- 172.21.15.97:0/3083486108 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7ffa680a3690 0x7ffa680a5b50 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x55bd802148c0 tx=0x7ffa78004ab0 comp rx=0 tx=0).stop 2024-03-27T10:58:22.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.416+0000 7ffa86400640 1 -- 172.21.15.97:0/3083486108 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffa80100c60 msgr2=0x7ffa8019e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:22.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.416+0000 7ffa86400640 1 --2- 172.21.15.97:0/3083486108 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffa80100c60 0x7ffa8019e790 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c005940 tx=0x7ffa6c004270 comp rx=0 tx=0).stop 2024-03-27T10:58:22.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.416+0000 7ffa86400640 1 -- 172.21.15.97:0/3083486108 shutdown_connections 2024-03-27T10:58:22.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.416+0000 7ffa86400640 1 --2- 172.21.15.97:0/3083486108 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffa80101600 0x7ffa801a5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:22.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.416+0000 7ffa86400640 1 --2- 172.21.15.97:0/3083486108 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7ffa680a3690 0x7ffa680a5b50 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:22.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.416+0000 7ffa86400640 1 --2- 172.21.15.97:0/3083486108 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffa80100c60 0x7ffa8019e790 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:22.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.416+0000 7ffa86400640 1 --2- 172.21.15.97:0/3083486108 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa80100300 0x7ffa8019e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:22.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.416+0000 7ffa86400640 1 -- 172.21.15.97:0/3083486108 >> 172.21.15.97:0/3083486108 conn(0x7ffa800fbec0 msgr2=0x7ffa800fd620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:22.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.416+0000 7ffa86400640 1 -- 172.21.15.97:0/3083486108 shutdown_connections 2024-03-27T10:58:22.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:22.416+0000 7ffa86400640 1 -- 172.21.15.97:0/3083486108 wait complete. 2024-03-27T10:58:22.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:58:22 smithi097 ceph-mon[31859]: scrub summary: active paths [/] 2024-03-27T10:58:22.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:58:22 smithi097 ceph-mon[31859]: Scrub error on inode 0x10000000204 (/volumes/qa/sv_0/22bc04cd-8106-4d11-83f1-2aead5019d07/client.0/tmp/iogen_3.1p0/src/iogen.c) see mds.h log and `damage ls` output for details 2024-03-27T10:58:22.392 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:58:22 smithi097 ceph-mon[31859]: Scrub error on inode 0x10000000203 (/volumes/qa/sv_0/22bc04cd-8106-4d11-83f1-2aead5019d07/client.0/tmp/iogen_3.1p0/src/iogen.8) see mds.h log and `damage ls` output for details 2024-03-27T10:58:22.393 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:58:22 smithi097 ceph-mon[31859]: scrub summary: idle+waiting paths [/] 2024-03-27T10:58:22.401 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.430+0000 7ff57d000640 1 -- 172.21.15.97:0/2129514707 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff57810c590 msgr2=0x7ff57810e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:22.402 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.431+0000 7ff57d000640 1 --2- 172.21.15.97:0/2129514707 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff57810c590 0x7ff57810e980 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff568009900 tx=0x7ff56802bc60 comp rx=0 tx=0).stop 2024-03-27T10:58:22.402 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.431+0000 7ff57d000640 1 -- 172.21.15.97:0/2129514707 shutdown_connections 2024-03-27T10:58:22.402 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.431+0000 7ff57d000640 1 --2- 172.21.15.97:0/2129514707 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff57810ef10 0x7ff5781113a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:22.402 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.431+0000 7ff57d000640 1 --2- 172.21.15.97:0/2129514707 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff57810c590 0x7ff57810e980 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:22.402 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.431+0000 7ff57d000640 1 --2- 172.21.15.97:0/2129514707 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff578109c60 0x7ff57810c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:22.402 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.431+0000 7ff57d000640 1 -- 172.21.15.97:0/2129514707 >> 172.21.15.97:0/2129514707 conn(0x7ff57806eb90 msgr2=0x7ff578070fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:22.402 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.431+0000 7ff57d000640 1 -- 172.21.15.97:0/2129514707 shutdown_connections 2024-03-27T10:58:22.402 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.431+0000 7ff57d000640 1 -- 172.21.15.97:0/2129514707 wait complete. 2024-03-27T10:58:22.403 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.433+0000 7ff57d000640 1 Processor -- start 2024-03-27T10:58:22.403 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.433+0000 7ff57d000640 1 -- start start 2024-03-27T10:58:22.404 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.433+0000 7ff57d000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff578109c60 0x7ff5781a2df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:22.404 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.434+0000 7ff57d000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff57810c590 0x7ff5781a5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:22.405 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.434+0000 7ff57d000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff57810ef10 0x7ff5781a7a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:22.405 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.434+0000 7ff575400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff57810c590 0x7ff5781a5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:22.405 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.434+0000 7ff575400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff57810c590 0x7ff5781a5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:52732/0 (socket says 172.21.15.97:52732) 2024-03-27T10:58:22.405 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.434+0000 7ff575400640 1 -- 172.21.15.97:0/1605468048 learned_addr learned my addr 172.21.15.97:0/1605468048 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:58:22.406 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.434+0000 7ff576800640 1 --2- 172.21.15.97:0/1605468048 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff57810ef10 0x7ff5781a7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:22.406 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.434+0000 7ff575e00640 1 --2- 172.21.15.97:0/1605468048 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff578109c60 0x7ff5781a2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:22.406 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.435+0000 7ff57d000640 1 -- 172.21.15.97:0/1605468048 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff578114ee0 con 0x7ff578109c60 2024-03-27T10:58:22.406 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.435+0000 7ff57d000640 1 -- 172.21.15.97:0/1605468048 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff578114d60 con 0x7ff57810c590 2024-03-27T10:58:22.406 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.435+0000 7ff57d000640 1 -- 172.21.15.97:0/1605468048 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff578115060 con 0x7ff57810ef10 2024-03-27T10:58:22.407 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.435+0000 7ff575e00640 1 -- 172.21.15.97:0/1605468048 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff57810ef10 msgr2=0x7ff5781a7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:22.407 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.435+0000 7ff575e00640 1 --2- 172.21.15.97:0/1605468048 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff57810ef10 0x7ff5781a7a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:22.407 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.435+0000 7ff575e00640 1 -- 172.21.15.97:0/1605468048 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff57810c590 msgr2=0x7ff5781a5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:22.407 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.435+0000 7ff575e00640 1 --2- 172.21.15.97:0/1605468048 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff57810c590 0x7ff5781a5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:22.407 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.435+0000 7ff575e00640 1 -- 172.21.15.97:0/1605468048 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5781a82c0 con 0x7ff578109c60 2024-03-27T10:58:22.407 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.436+0000 7ff575e00640 1 --2- 172.21.15.97:0/1605468048 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff578109c60 0x7ff5781a2df0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7ff564009fd0 tx=0x7ff56400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:22.407 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.436+0000 7ff56ea00640 1 -- 172.21.15.97:0/1605468048 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff564015020 con 0x7ff578109c60 2024-03-27T10:58:22.408 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.436+0000 7ff56ea00640 1 -- 172.21.15.97:0/1605468048 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff564011ce0 con 0x7ff578109c60 2024-03-27T10:58:22.408 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.436+0000 7ff56ea00640 1 -- 172.21.15.97:0/1605468048 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff564010de0 con 0x7ff578109c60 2024-03-27T10:58:22.408 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.436+0000 7ff57d000640 1 -- 172.21.15.97:0/1605468048 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5781a85b0 con 0x7ff578109c60 2024-03-27T10:58:22.408 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.436+0000 7ff57d000640 1 -- 172.21.15.97:0/1605468048 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5781a8bd0 con 0x7ff578109c60 2024-03-27T10:58:22.409 INFO:tasks.cephfs.filesystem:scrub status for tag:a0f9dce1-c2cf-4f12-a3e5-272846ff044f - {'path': '/', 'tag': 'a0f9dce1-c2cf-4f12-a3e5-272846ff044f', 'options': 'recursive,force'} 2024-03-27T10:58:22.411 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.440+0000 7ff57fd85740 1 Processor -- start 2024-03-27T10:58:22.411 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.441+0000 7ff57fd85740 1 -- start start 2024-03-27T10:58:22.413 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.441+0000 7ff57fd85740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bdd3ca3980 0x55bdd3ca3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:22.413 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.441+0000 7ff57fd85740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bdd3ca4370 0x55bdd3ca8e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:22.413 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.441+0000 7ff57fd85740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bdd3ca94e0 0x55bdd3cab970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:22.413 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.441+0000 7ff57fd85740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55bdd3c08d10 con 0x55bdd3ca4370 2024-03-27T10:58:22.413 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.441+0000 7ff57fd85740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55bdd3bb1260 con 0x55bdd3ca94e0 2024-03-27T10:58:22.413 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.441+0000 7ff57fd85740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55bdd3c04cc0 con 0x55bdd3ca3980 2024-03-27T10:58:22.414 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.441+0000 7ff56ea00640 1 -- 172.21.15.97:0/1605468048 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7ff564018020 con 0x7ff578109c60 2024-03-27T10:58:22.414 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.441+0000 7ff575e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bdd3ca4370 0x55bdd3ca8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:22.414 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.442+0000 7ff575400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bdd3ca94e0 0x55bdd3cab970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:22.414 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.442+0000 7ff575400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bdd3ca94e0 0x55bdd3cab970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:52736/0 (socket says 172.21.15.97:52736) 2024-03-27T10:58:22.414 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.442+0000 7ff575400640 1 -- 172.21.15.97:0/1365807795 learned_addr learned my addr 172.21.15.97:0/1365807795 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:58:22.414 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.442+0000 7ff576800640 1 --2- 172.21.15.97:0/1365807795 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bdd3ca3980 0x55bdd3ca3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:22.414 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.442+0000 7ff576800640 1 -- 172.21.15.97:0/1365807795 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bdd3ca94e0 msgr2=0x55bdd3cab970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:22.414 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.442+0000 7ff576800640 1 --2- 172.21.15.97:0/1365807795 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bdd3ca94e0 0x55bdd3cab970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:22.414 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.442+0000 7ff576800640 1 -- 172.21.15.97:0/1365807795 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bdd3ca4370 msgr2=0x55bdd3ca8e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:22.414 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.442+0000 7ff576800640 1 --2- 172.21.15.97:0/1365807795 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bdd3ca4370 0x55bdd3ca8e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:22.414 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.443+0000 7ff576800640 1 -- 172.21.15.97:0/1365807795 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bdd3ba8aa0 con 0x55bdd3ca3980 2024-03-27T10:58:22.414 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.443+0000 7ff575e00640 1 --2- 172.21.15.97:0/1365807795 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bdd3ca4370 0x55bdd3ca8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T10:58:22.415 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.443+0000 7ff576800640 1 --2- 172.21.15.97:0/1365807795 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bdd3ca3980 0x55bdd3ca3da0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff570009f70 tx=0x7ff570009650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b95168c5ba286684 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:22.416 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.443+0000 7ff563400640 1 -- 172.21.15.97:0/1365807795 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff570013850 con 0x55bdd3ca3980 2024-03-27T10:58:22.416 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.444+0000 7ff563400640 1 -- 172.21.15.97:0/1365807795 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff570019070 con 0x55bdd3ca3980 2024-03-27T10:58:22.416 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.444+0000 7ff563400640 1 -- 172.21.15.97:0/1365807795 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff570014d50 con 0x55bdd3ca3980 2024-03-27T10:58:22.416 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.444+0000 7ff56ea00640 1 --2- 172.21.15.97:0/1605468048 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7ff55c0abf40 0x7ff55c0ae400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:22.416 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.444+0000 7ff56ea00640 1 -- 172.21.15.97:0/1605468048 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7ff5640cf420 con 0x7ff578109c60 2024-03-27T10:58:22.416 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.445+0000 7ff575e00640 1 --2- 172.21.15.97:0/1605468048 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7ff55c0abf40 0x7ff55c0ae400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:22.416 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.445+0000 7ff57fd85740 1 -- 172.21.15.97:0/1365807795 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bdd3ca3980 msgr2=0x55bdd3ca3da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:22.416 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.445+0000 7ff57fd85740 1 --2- 172.21.15.97:0/1365807795 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bdd3ca3980 0x55bdd3ca3da0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff570009f70 tx=0x7ff570009650 comp rx=0 tx=0).stop 2024-03-27T10:58:22.416 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.445+0000 7ff57fd85740 1 -- 172.21.15.97:0/1365807795 shutdown_connections 2024-03-27T10:58:22.416 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.445+0000 7ff57fd85740 1 --2- 172.21.15.97:0/1365807795 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bdd3ca94e0 0x55bdd3cab970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:22.416 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.445+0000 7ff57fd85740 1 --2- 172.21.15.97:0/1365807795 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bdd3ca4370 0x55bdd3ca8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:22.416 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.445+0000 7ff57fd85740 1 --2- 172.21.15.97:0/1365807795 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bdd3ca3980 0x55bdd3ca3da0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:22.417 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.445+0000 7ff57fd85740 1 -- 172.21.15.97:0/1365807795 >> 172.21.15.97:0/1365807795 conn(0x55bdd3c9eef0 msgr2=0x55bdd3c9f2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:22.417 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.445+0000 7ff57fd85740 1 -- 172.21.15.97:0/1365807795 shutdown_connections 2024-03-27T10:58:22.417 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.446+0000 7ff57fd85740 1 -- 172.21.15.97:0/1365807795 wait complete. 2024-03-27T10:58:22.417 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.446+0000 7ff575e00640 1 --2- 172.21.15.97:0/1605468048 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7ff55c0abf40 0x7ff55c0ae400 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff5640cd2a0 tx=0x7ff5640d2f00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:22.418 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.446+0000 7ff57fd85740 1 -- start start 2024-03-27T10:58:22.418 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.447+0000 7ff57fd85740 1 Processor -- start 2024-03-27T10:58:22.418 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.447+0000 7ff57fd85740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bdd3ca3980 0x55bdd3ca6a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:22.418 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.447+0000 7ff57fd85740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bdd3ca4370 0x55bdd3ca6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:22.419 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.448+0000 7ff57fd85740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bdd3ca94e0 0x55bdd3ca7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:22.420 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.448+0000 7ff57fd85740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55bdd3cac270 con 0x55bdd3ca94e0 2024-03-27T10:58:22.420 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.448+0000 7ff57fd85740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55bdd3bc4630 con 0x55bdd3ca4370 2024-03-27T10:58:22.420 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.448+0000 7ff57fd85740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55bdd3cac3f0 con 0x55bdd3ca3980 2024-03-27T10:58:22.420 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.448+0000 7ff57fd85740 10 client.0 _ll_register_callbacks cb 0x55bdd3bc9c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:58:22.420 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.448+0000 7ff576800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bdd3ca94e0 0x55bdd3ca7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:22.420 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.448+0000 7ff576800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bdd3ca94e0 0x55bdd3ca7fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53488/0 (socket says 172.21.15.97:53488) 2024-03-27T10:58:22.420 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.448+0000 7ff576800640 1 -- 172.21.15.97:0/3047254350 learned_addr learned my addr 172.21.15.97:0/3047254350 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:58:22.420 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.449+0000 7ff575400640 1 --2- 172.21.15.97:0/3047254350 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bdd3ca4370 0x55bdd3ca6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:22.421 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.449+0000 7ff576800640 1 --2- 172.21.15.97:0/3047254350 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bdd3ca3980 0x55bdd3ca6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:22.421 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.449+0000 7ff575400640 1 -- 172.21.15.97:0/3047254350 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bdd3ca3980 msgr2=0x55bdd3ca6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:22.421 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.449+0000 7ff575400640 1 --2- 172.21.15.97:0/3047254350 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bdd3ca3980 0x55bdd3ca6a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:22.421 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.449+0000 7ff575400640 1 -- 172.21.15.97:0/3047254350 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bdd3ca94e0 msgr2=0x55bdd3ca7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:22.421 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.449+0000 7ff575400640 1 --2- 172.21.15.97:0/3047254350 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bdd3ca94e0 0x55bdd3ca7fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:22.421 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.449+0000 7ff575400640 1 -- 172.21.15.97:0/3047254350 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bdd3bd05f0 con 0x55bdd3ca4370 2024-03-27T10:58:22.421 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.450+0000 7ff575400640 1 --2- 172.21.15.97:0/3047254350 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bdd3ca4370 0x55bdd3ca6fa0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x55bdd3c9fe60 tx=0x7ff568004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f6b7904d3ff4ecc server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:22.421 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.450+0000 7ff557e00640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-03-27T10:58:22.422 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.451+0000 7ff557e00640 1 -- 172.21.15.97:0/3047254350 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff568005250 con 0x55bdd3ca4370 2024-03-27T10:58:22.422 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.451+0000 7ff557e00640 1 -- 172.21.15.97:0/3047254350 <== mon.2 v2:172.21.15.114:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7ff5680053f0 con 0x55bdd3ca4370 2024-03-27T10:58:22.422 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.451+0000 7ff57fd85740 1 -- 172.21.15.97:0/3047254350 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff570019070 con 0x55bdd3ca4370 2024-03-27T10:58:22.423 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.452+0000 7ff557e00640 1 -- 172.21.15.97:0/3047254350 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff5680055f0 con 0x55bdd3ca4370 2024-03-27T10:58:22.423 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.452+0000 7ff57fd85740 10 client.35852 fetch_fsmap learned FSMap version 30 2024-03-27T10:58:22.423 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.452+0000 7ff57fd85740 1 -- 172.21.15.97:0/3047254350 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bdd391d210 con 0x55bdd3ca4370 2024-03-27T10:58:22.423 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.453+0000 7ff557e00640 1 -- 172.21.15.97:0/3047254350 <== mon.2 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff570019070 con 0x55bdd3ca4370 2024-03-27T10:58:22.424 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.453+0000 7ff557e00640 1 -- 172.21.15.97:0/3047254350 <== mon.2 v2:172.21.15.114:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7ff568003920 con 0x55bdd3ca4370 2024-03-27T10:58:22.425 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.454+0000 7ff57fd85740 10 client.35852 fetch_fsmap finished waiting for FSMap version 30 2024-03-27T10:58:22.425 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.454+0000 7ff57fd85740 10 client.35852 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T10:58:22.425 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.454+0000 7ff57fd85740 20 client.35852 populate_metadata read hostname 'smithi097' 2024-03-27T10:58:22.425 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.454+0000 7ff57fd85740 1 --2- 172.21.15.97:0/3047254350 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55bdd3cb9b90 0x55bdd3cbbfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:22.425 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.454+0000 7ff57fd85740 4 client.35852 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T10:58:22.425 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.455+0000 7ff57fd85740 1 -- 172.21.15.97:0/3047254350 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bdd3c16250 con 0x55bdd3cb9b90 2024-03-27T10:58:22.426 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.455+0000 7ff576800640 1 --2- 172.21.15.97:0/3047254350 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55bdd3cb9b90 0x55bdd3cbbfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:22.427 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.456+0000 7ff576800640 1 --2- 172.21.15.97:0/3047254350 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55bdd3cb9b90 0x55bdd3cbbfd0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51f55eeec08318c1 server_cookie=db9f9bd0a8e26f2f in_seq=0 out_seq=0 2024-03-27T10:58:22.427 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.456+0000 7ff557e00640 10 client.35852 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T10:58:22.428 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.458+0000 7ff557e00640 1 -- 172.21.15.97:0/3047254350 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55bdd3c16250 con 0x55bdd3cb9b90 2024-03-27T10:58:22.428 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.458+0000 7ff557e00640 10 client.35852 handle_command_reply: tid=0 2024-03-27T10:58:22.429 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.458+0000 7ff57fd85740 1 client.35852 shutdown 2024-03-27T10:58:22.429 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.458+0000 7ff57fd85740 1 -- 172.21.15.97:0/3047254350 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bdd3ca4370 msgr2=0x55bdd3ca6fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:22.429 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.458+0000 7ff57fd85740 1 --2- 172.21.15.97:0/3047254350 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bdd3ca4370 0x55bdd3ca6fa0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x55bdd3c9fe60 tx=0x7ff568004270 comp rx=0 tx=0).stop 2024-03-27T10:58:22.430 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.458+0000 7ff57fd85740 1 -- 172.21.15.97:0/3047254350 shutdown_connections 2024-03-27T10:58:22.430 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.458+0000 7ff57fd85740 1 --2- 172.21.15.97:0/3047254350 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55bdd3cb9b90 0x55bdd3cbbfd0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:22.430 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.458+0000 7ff57fd85740 1 --2- 172.21.15.97:0/3047254350 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bdd3ca94e0 0x55bdd3ca7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:22.430 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.458+0000 7ff57fd85740 1 --2- 172.21.15.97:0/3047254350 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bdd3ca4370 0x55bdd3ca6fa0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:22.430 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.458+0000 7ff57fd85740 1 --2- 172.21.15.97:0/3047254350 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bdd3ca3980 0x55bdd3ca6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:22.430 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.458+0000 7ff57fd85740 1 -- 172.21.15.97:0/3047254350 >> 172.21.15.97:0/3047254350 conn(0x55bdd3c9eef0 msgr2=0x55bdd3c9e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:22.430 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.458+0000 7ff557e00640 0 client.35852 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T10:58:22.430 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.458+0000 7ff57fd85740 1 -- 172.21.15.97:0/3047254350 shutdown_connections 2024-03-27T10:58:22.430 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.459+0000 7ff57fd85740 1 -- 172.21.15.97:0/3047254350 wait complete. 2024-03-27T10:58:22.430 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.459+0000 7ff57fd85740 20 client.35852 trim_cache size 0 max 16384 2024-03-27T10:58:22.456 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.485+0000 7ff57fd85740 1 Processor -- start 2024-03-27T10:58:22.456 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.486+0000 7ff57fd85740 1 -- start start 2024-03-27T10:58:22.456 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.486+0000 7ff57fd85740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bdd3ca3980 0x55bdd3c9a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:22.457 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.486+0000 7ff57fd85740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bdd3ca4370 0x55bdd3ca2150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:22.457 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.486+0000 7ff57fd85740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bdd3ca94e0 0x55bdd3ca2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:22.457 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.486+0000 7ff57fd85740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff5781af720 con 0x55bdd3ca4370 2024-03-27T10:58:22.457 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.486+0000 7ff57fd85740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff5781b1320 con 0x55bdd3ca94e0 2024-03-27T10:58:22.457 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.486+0000 7ff57fd85740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff5781afdf0 con 0x55bdd3ca3980 2024-03-27T10:58:22.458 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.486+0000 7ff575400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bdd3ca4370 0x55bdd3ca2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:22.458 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.486+0000 7ff575400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bdd3ca4370 0x55bdd3ca2150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53494/0 (socket says 172.21.15.97:53494) 2024-03-27T10:58:22.458 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.486+0000 7ff575400640 1 -- 172.21.15.97:0/3024375570 learned_addr learned my addr 172.21.15.97:0/3024375570 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:58:22.458 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.487+0000 7ff576800640 1 --2- 172.21.15.97:0/3024375570 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bdd3ca3980 0x55bdd3c9a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:22.458 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.487+0000 7ff576800640 1 --2- 172.21.15.97:0/3024375570 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bdd3ca94e0 0x55bdd3ca2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:22.458 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.487+0000 7ff575400640 1 -- 172.21.15.97:0/3024375570 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bdd3ca3980 msgr2=0x55bdd3c9a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:22.459 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.487+0000 7ff575400640 1 --2- 172.21.15.97:0/3024375570 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bdd3ca3980 0x55bdd3c9a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:22.459 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.487+0000 7ff575400640 1 -- 172.21.15.97:0/3024375570 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bdd3ca94e0 msgr2=0x55bdd3ca2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:22.459 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.487+0000 7ff575400640 1 --2- 172.21.15.97:0/3024375570 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bdd3ca94e0 0x55bdd3ca2690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:22.459 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.487+0000 7ff575400640 1 -- 172.21.15.97:0/3024375570 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bdd39e9620 con 0x55bdd3ca4370 2024-03-27T10:58:22.459 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.487+0000 7ff576800640 1 --2- 172.21.15.97:0/3024375570 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bdd3ca3980 0x55bdd3c9a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T10:58:22.459 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.487+0000 7ff575400640 1 --2- 172.21.15.97:0/3024375570 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bdd3ca4370 0x55bdd3ca2150 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7ff568005c40 tx=0x7ff56802bc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd0c909253a03514 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:22.459 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.488+0000 7ff562a00640 1 -- 172.21.15.97:0/3024375570 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff568005250 con 0x55bdd3ca4370 2024-03-27T10:58:22.459 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.488+0000 7ff562a00640 1 -- 172.21.15.97:0/3024375570 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff5680053f0 con 0x55bdd3ca4370 2024-03-27T10:58:22.459 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.488+0000 7ff562a00640 1 -- 172.21.15.97:0/3024375570 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff568004920 con 0x55bdd3ca4370 2024-03-27T10:58:22.460 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.489+0000 7ff57fd85740 1 -- 172.21.15.97:0/3024375570 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bdd3ca4370 msgr2=0x55bdd3ca2150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:22.460 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.489+0000 7ff57fd85740 1 --2- 172.21.15.97:0/3024375570 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bdd3ca4370 0x55bdd3ca2150 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7ff568005c40 tx=0x7ff56802bc60 comp rx=0 tx=0).stop 2024-03-27T10:58:22.460 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.489+0000 7ff57fd85740 1 -- 172.21.15.97:0/3024375570 shutdown_connections 2024-03-27T10:58:22.460 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.489+0000 7ff57fd85740 1 --2- 172.21.15.97:0/3024375570 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bdd3ca94e0 0x55bdd3ca2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:22.460 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.490+0000 7ff57fd85740 1 --2- 172.21.15.97:0/3024375570 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bdd3ca4370 0x55bdd3ca2150 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:22.461 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.490+0000 7ff57fd85740 1 --2- 172.21.15.97:0/3024375570 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bdd3ca3980 0x55bdd3c9a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:22.461 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.490+0000 7ff57fd85740 1 -- 172.21.15.97:0/3024375570 >> 172.21.15.97:0/3024375570 conn(0x55bdd3c9eef0 msgr2=0x55bdd3ca50c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:22.461 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.490+0000 7ff57fd85740 1 -- 172.21.15.97:0/3024375570 shutdown_connections 2024-03-27T10:58:22.461 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.490+0000 7ff57fd85740 1 -- 172.21.15.97:0/3024375570 wait complete. 2024-03-27T10:58:22.462 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.491+0000 7ff57fd85740 1 -- start start 2024-03-27T10:58:22.462 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.491+0000 7ff57fd85740 1 Processor -- start 2024-03-27T10:58:22.462 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.491+0000 7ff57fd85740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bdd3ca3980 0x55bdd3cb37f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:22.462 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.491+0000 7ff57fd85740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bdd3ca4370 0x55bdd3cb3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:22.462 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.491+0000 7ff57fd85740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bdd3ca94e0 0x55bdd3cb4d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:22.462 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.491+0000 7ff57fd85740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff5781b0b90 con 0x55bdd3ca94e0 2024-03-27T10:58:22.462 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.491+0000 7ff57fd85740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff5781b04c0 con 0x55bdd3ca4370 2024-03-27T10:58:22.463 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.491+0000 7ff57fd85740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff5781ad700 con 0x55bdd3ca3980 2024-03-27T10:58:22.463 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.491+0000 7ff57fd85740 10 client.0 _ll_register_callbacks cb 0x7ff568009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:58:22.464 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.492+0000 7ff575400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bdd3ca4370 0x55bdd3cb3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:22.464 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.492+0000 7ff575400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bdd3ca4370 0x55bdd3cb3d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:52768/0 (socket says 172.21.15.97:52768) 2024-03-27T10:58:22.464 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.492+0000 7ff575400640 1 -- 172.21.15.97:0/1496750426 learned_addr learned my addr 172.21.15.97:0/1496750426 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:58:22.464 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.492+0000 7ff576800640 1 --2- 172.21.15.97:0/1496750426 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bdd3ca3980 0x55bdd3cb37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:22.464 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.492+0000 7ff576800640 1 --2- 172.21.15.97:0/1496750426 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bdd3ca94e0 0x55bdd3cb4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:22.464 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.493+0000 7ff575400640 1 -- 172.21.15.97:0/1496750426 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bdd3ca3980 msgr2=0x55bdd3cb37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:22.464 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.493+0000 7ff575400640 1 --2- 172.21.15.97:0/1496750426 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bdd3ca3980 0x55bdd3cb37f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:22.464 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.493+0000 7ff575400640 1 -- 172.21.15.97:0/1496750426 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bdd3ca94e0 msgr2=0x55bdd3cb4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:22.464 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.493+0000 7ff575400640 1 --2- 172.21.15.97:0/1496750426 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bdd3ca94e0 0x55bdd3cb4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:22.464 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.493+0000 7ff575400640 1 -- 172.21.15.97:0/1496750426 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bdd3cb5330 con 0x55bdd3ca4370 2024-03-27T10:58:22.464 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.493+0000 7ff575400640 1 --2- 172.21.15.97:0/1496750426 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bdd3ca4370 0x55bdd3cb3d30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff568000c00 tx=0x7ff568005e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ae58f629ebd33fd server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:22.465 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.493+0000 7ff557e00640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-03-27T10:58:22.465 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.494+0000 7ff557e00640 1 -- 172.21.15.97:0/1496750426 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff568005250 con 0x55bdd3ca4370 2024-03-27T10:58:22.465 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.494+0000 7ff557e00640 1 -- 172.21.15.97:0/1496750426 <== mon.2 v2:172.21.15.114:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7ff5680048d0 con 0x55bdd3ca4370 2024-03-27T10:58:22.466 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.494+0000 7ff57fd85740 1 -- 172.21.15.97:0/1496750426 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5680053f0 con 0x55bdd3ca4370 2024-03-27T10:58:22.466 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.494+0000 7ff557e00640 1 -- 172.21.15.97:0/1496750426 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff56806e050 con 0x55bdd3ca4370 2024-03-27T10:58:22.466 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.495+0000 7ff57fd85740 10 client.35858 fetch_fsmap learned FSMap version 30 2024-03-27T10:58:22.466 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.495+0000 7ff57fd85740 1 -- 172.21.15.97:0/1496750426 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bdd3cb6670 con 0x55bdd3ca4370 2024-03-27T10:58:22.467 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.496+0000 7ff557e00640 1 -- 172.21.15.97:0/1496750426 <== mon.2 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff5680053f0 con 0x55bdd3ca4370 2024-03-27T10:58:22.468 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.497+0000 7ff557e00640 1 -- 172.21.15.97:0/1496750426 <== mon.2 v2:172.21.15.114:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7ff568034aa0 con 0x55bdd3ca4370 2024-03-27T10:58:22.468 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.497+0000 7ff57fd85740 10 client.35858 fetch_fsmap finished waiting for FSMap version 30 2024-03-27T10:58:22.468 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.497+0000 7ff57fd85740 10 client.35858 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T10:58:22.468 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.497+0000 7ff57fd85740 20 client.35858 populate_metadata read hostname 'smithi097' 2024-03-27T10:58:22.468 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.497+0000 7ff57fd85740 1 --2- 172.21.15.97:0/1496750426 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55bdd3d206a0 0x55bdd3d22ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:22.469 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.497+0000 7ff57fd85740 4 client.35858 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-27T10:58:22.469 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.497+0000 7ff57fd85740 1 -- 172.21.15.97:0/1496750426 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bdd3c105a0 con 0x55bdd3d206a0 2024-03-27T10:58:22.469 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.498+0000 7ff576800640 1 --2- 172.21.15.97:0/1496750426 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55bdd3d206a0 0x55bdd3d22ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:22.470 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.499+0000 7ff576800640 1 --2- 172.21.15.97:0/1496750426 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55bdd3d206a0 0x55bdd3d22ac0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6143ad3dbf38384a server_cookie=f2ebbc03d95e3078 in_seq=0 out_seq=0 2024-03-27T10:58:22.470 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.499+0000 7ff557e00640 10 client.35858 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T10:58:22.485 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.514+0000 7ff557e00640 1 -- 172.21.15.97:0/1496750426 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bdd3c105a0 con 0x55bdd3d206a0 2024-03-27T10:58:22.485 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.514+0000 7ff557e00640 10 client.35858 handle_command_reply: tid=0 2024-03-27T10:58:22.485 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.514+0000 7ff57fd85740 1 client.35858 shutdown 2024-03-27T10:58:22.485 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.514+0000 7ff57fd85740 1 -- 172.21.15.97:0/1496750426 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bdd3ca4370 msgr2=0x55bdd3cb3d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:22.485 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.514+0000 7ff57fd85740 1 --2- 172.21.15.97:0/1496750426 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bdd3ca4370 0x55bdd3cb3d30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff568000c00 tx=0x7ff568005e30 comp rx=0 tx=0).stop 2024-03-27T10:58:22.485 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.514+0000 7ff57fd85740 1 -- 172.21.15.97:0/1496750426 shutdown_connections 2024-03-27T10:58:22.485 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.514+0000 7ff57fd85740 1 --2- 172.21.15.97:0/1496750426 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55bdd3d206a0 0x55bdd3d22ac0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:22.485 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.514+0000 7ff557e00640 0 client.35858 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T10:58:22.486 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.514+0000 7ff57fd85740 1 --2- 172.21.15.97:0/1496750426 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bdd3ca94e0 0x55bdd3cb4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:22.486 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.514+0000 7ff57fd85740 1 --2- 172.21.15.97:0/1496750426 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bdd3ca4370 0x55bdd3cb3d30 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:22.486 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.514+0000 7ff57fd85740 1 --2- 172.21.15.97:0/1496750426 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bdd3ca3980 0x55bdd3cb37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:22.486 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.514+0000 7ff57fd85740 1 -- 172.21.15.97:0/1496750426 >> 172.21.15.97:0/1496750426 conn(0x55bdd3c9eef0 msgr2=0x55bdd3ca2150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:22.486 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.514+0000 7ff57fd85740 1 -- 172.21.15.97:0/1496750426 shutdown_connections 2024-03-27T10:58:22.486 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.514+0000 7ff57fd85740 1 -- 172.21.15.97:0/1496750426 wait complete. 2024-03-27T10:58:22.486 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.515+0000 7ff57fd85740 20 client.35858 trim_cache size 0 max 16384 2024-03-27T10:58:22.486 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-27T10:58:22.486 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-27T10:58:22.487 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-27T10:58:22.487 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-27T10:58:22.487 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.515+0000 7ff57d000640 1 -- 172.21.15.97:0/1605468048 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7ff55c0abf40 msgr2=0x7ff55c0ae400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:22.487 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.516+0000 7ff57d000640 1 --2- 172.21.15.97:0/1605468048 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7ff55c0abf40 0x7ff55c0ae400 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff5640cd2a0 tx=0x7ff5640d2f00 comp rx=0 tx=0).stop 2024-03-27T10:58:22.487 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.516+0000 7ff57d000640 1 -- 172.21.15.97:0/1605468048 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff578109c60 msgr2=0x7ff5781a2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:22.488 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.516+0000 7ff57d000640 1 --2- 172.21.15.97:0/1605468048 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff578109c60 0x7ff5781a2df0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7ff564009fd0 tx=0x7ff56400b870 comp rx=0 tx=0).stop 2024-03-27T10:58:22.488 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.516+0000 7ff57d000640 1 -- 172.21.15.97:0/1605468048 shutdown_connections 2024-03-27T10:58:22.488 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.516+0000 7ff57d000640 1 --2- 172.21.15.97:0/1605468048 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff57810ef10 0x7ff5781a7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:22.488 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.516+0000 7ff57d000640 1 --2- 172.21.15.97:0/1605468048 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7ff55c0abf40 0x7ff55c0ae400 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:22.488 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.516+0000 7ff57d000640 1 --2- 172.21.15.97:0/1605468048 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff57810c590 0x7ff5781a5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:22.488 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.516+0000 7ff57d000640 1 --2- 172.21.15.97:0/1605468048 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff578109c60 0x7ff5781a2df0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:22.488 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.516+0000 7ff57d000640 1 -- 172.21.15.97:0/1605468048 >> 172.21.15.97:0/1605468048 conn(0x7ff57806eb90 msgr2=0x7ff57810a040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:22.488 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.516+0000 7ff57d000640 1 -- 172.21.15.97:0/1605468048 shutdown_connections 2024-03-27T10:58:22.488 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:22.516+0000 7ff57d000640 1 -- 172.21.15.97:0/1605468048 wait complete. 2024-03-27T10:58:22.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:58:22 smithi177 ceph-mon[35294]: scrub summary: active paths [/] 2024-03-27T10:58:22.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:58:22 smithi177 ceph-mon[35294]: Scrub error on inode 0x10000000204 (/volumes/qa/sv_0/22bc04cd-8106-4d11-83f1-2aead5019d07/client.0/tmp/iogen_3.1p0/src/iogen.c) see mds.h log and `damage ls` output for details 2024-03-27T10:58:22.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:58:22 smithi177 ceph-mon[35294]: Scrub error on inode 0x10000000203 (/volumes/qa/sv_0/22bc04cd-8106-4d11-83f1-2aead5019d07/client.0/tmp/iogen_3.1p0/src/iogen.8) see mds.h log and `damage ls` output for details 2024-03-27T10:58:22.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:58:22 smithi177 ceph-mon[35294]: scrub summary: idle+waiting paths [/] 2024-03-27T10:58:22.766 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:58:22 smithi114 ceph-mon[35338]: scrub summary: active paths [/] 2024-03-27T10:58:22.766 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:58:22 smithi114 ceph-mon[35338]: Scrub error on inode 0x10000000204 (/volumes/qa/sv_0/22bc04cd-8106-4d11-83f1-2aead5019d07/client.0/tmp/iogen_3.1p0/src/iogen.c) see mds.h log and `damage ls` output for details 2024-03-27T10:58:22.767 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:58:22 smithi114 ceph-mon[35338]: Scrub error on inode 0x10000000203 (/volumes/qa/sv_0/22bc04cd-8106-4d11-83f1-2aead5019d07/client.0/tmp/iogen_3.1p0/src/iogen.8) see mds.h log and `damage ls` output for details 2024-03-27T10:58:22.767 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:58:22 smithi114 ceph-mon[35338]: scrub summary: idle+waiting paths [/] 2024-03-27T10:58:23.577 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:58:23 smithi177 ceph-mon[35294]: scrub queued for path: / 2024-03-27T10:58:23.578 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:58:23 smithi177 ceph-mon[35294]: scrub summary: idle+waiting paths [/] 2024-03-27T10:58:23.578 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:58:23 smithi177 ceph-mon[35294]: scrub summary: active paths [/] 2024-03-27T10:58:23.578 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:58:23 smithi177 ceph-mon[35294]: scrub summary: idle+waiting paths [/] 2024-03-27T10:58:23.578 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:58:23 smithi177 ceph-mon[35294]: pgmap v493: 97 pgs: 97 active+clean; 1.7 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 124 MiB/s wr, 79 op/s 2024-03-27T10:58:23.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:58:23 smithi097 ceph-mon[31859]: scrub queued for path: / 2024-03-27T10:58:23.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:58:23 smithi097 ceph-mon[31859]: scrub summary: idle+waiting paths [/] 2024-03-27T10:58:23.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:58:23 smithi097 ceph-mon[31859]: scrub summary: active paths [/] 2024-03-27T10:58:23.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:58:23 smithi097 ceph-mon[31859]: scrub summary: idle+waiting paths [/] 2024-03-27T10:58:23.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:58:23 smithi097 ceph-mon[31859]: pgmap v493: 97 pgs: 97 active+clean; 1.7 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 124 MiB/s wr, 79 op/s 2024-03-27T10:58:23.766 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:58:23 smithi114 ceph-mon[35338]: scrub queued for path: / 2024-03-27T10:58:23.766 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:58:23 smithi114 ceph-mon[35338]: scrub summary: idle+waiting paths [/] 2024-03-27T10:58:23.766 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:58:23 smithi114 ceph-mon[35338]: scrub summary: active paths [/] 2024-03-27T10:58:23.766 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:58:23 smithi114 ceph-mon[35338]: scrub summary: idle+waiting paths [/] 2024-03-27T10:58:23.766 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:58:23 smithi114 ceph-mon[35338]: pgmap v493: 97 pgs: 97 active+clean; 1.7 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 124 MiB/s wr, 79 op/s 2024-03-27T10:58:25.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:58:25 smithi097 ceph-mon[31859]: pgmap v494: 97 pgs: 97 active+clean; 1.9 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 116 MiB/s wr, 79 op/s 2024-03-27T10:58:25.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:58:25 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mon-a[31855]: 2024-03-27T10:58:25.306+0000 7fd0ea600700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-03-27T10:58:25.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:58:25 smithi177 ceph-mon[35294]: pgmap v494: 97 pgs: 97 active+clean; 1.9 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 116 MiB/s wr, 79 op/s 2024-03-27T10:58:25.766 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:58:25 smithi114 ceph-mon[35338]: pgmap v494: 97 pgs: 97 active+clean; 1.9 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 116 MiB/s wr, 79 op/s 2024-03-27T10:58:26.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:58:26 smithi097 ceph-mon[31859]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-03-27T10:58:26.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:58:26 smithi097 ceph-mon[31859]: mds.? [v2:172.21.15.114:6836/616775588,v1:172.21.15.114:6837/616775588] up:active 2024-03-27T10:58:26.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:58:26 smithi097 ceph-mon[31859]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T10:58:26.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:58:26 smithi177 ceph-mon[35294]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-03-27T10:58:26.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:58:26 smithi177 ceph-mon[35294]: mds.? [v2:172.21.15.114:6836/616775588,v1:172.21.15.114:6837/616775588] up:active 2024-03-27T10:58:26.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:58:26 smithi177 ceph-mon[35294]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T10:58:26.766 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:58:26 smithi114 ceph-mon[35338]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-03-27T10:58:26.766 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:58:26 smithi114 ceph-mon[35338]: mds.? [v2:172.21.15.114:6836/616775588,v1:172.21.15.114:6837/616775588] up:active 2024-03-27T10:58:26.766 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:58:26 smithi114 ceph-mon[35338]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T10:58:27.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:58:27 smithi097 ceph-mon[31859]: pgmap v495: 97 pgs: 97 active+clean; 2.0 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 108 MiB/s wr, 70 op/s 2024-03-27T10:58:27.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:58:27 smithi177 ceph-mon[35294]: pgmap v495: 97 pgs: 97 active+clean; 2.0 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 108 MiB/s wr, 70 op/s 2024-03-27T10:58:27.766 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:58:27 smithi114 ceph-mon[35338]: pgmap v495: 97 pgs: 97 active+clean; 2.0 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 108 MiB/s wr, 70 op/s 2024-03-27T10:58:29.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:58:29 smithi097 ceph-mon[31859]: pgmap v496: 97 pgs: 97 active+clean; 2.3 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 117 MiB/s wr, 75 op/s 2024-03-27T10:58:29.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:58:29 smithi177 ceph-mon[35294]: pgmap v496: 97 pgs: 97 active+clean; 2.3 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 117 MiB/s wr, 75 op/s 2024-03-27T10:58:29.766 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:58:29 smithi114 ceph-mon[35338]: pgmap v496: 97 pgs: 97 active+clean; 2.3 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 117 MiB/s wr, 75 op/s 2024-03-27T10:58:30.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:58:30 smithi097 ceph-mon[31859]: scrub summary: idle 2024-03-27T10:58:30.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:58:30 smithi097 ceph-mon[31859]: scrub summary: idle 2024-03-27T10:58:30.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:58:30 smithi177 ceph-mon[35294]: scrub summary: idle 2024-03-27T10:58:30.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:58:30 smithi177 ceph-mon[35294]: scrub summary: idle 2024-03-27T10:58:30.766 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:58:30 smithi114 ceph-mon[35338]: scrub summary: idle 2024-03-27T10:58:30.766 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:58:30 smithi114 ceph-mon[35338]: scrub summary: idle 2024-03-27T10:58:31.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:58:31 smithi097 ceph-mon[31859]: pgmap v497: 97 pgs: 97 active+clean; 2.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 128 MiB/s wr, 83 op/s 2024-03-27T10:58:31.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:58:31 smithi177 ceph-mon[35294]: pgmap v497: 97 pgs: 97 active+clean; 2.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 128 MiB/s wr, 83 op/s 2024-03-27T10:58:31.766 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:58:31 smithi114 ceph-mon[35338]: pgmap v497: 97 pgs: 97 active+clean; 2.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 128 MiB/s wr, 83 op/s 2024-03-27T10:58:33.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:58:33 smithi097 ceph-mon[31859]: pgmap v498: 97 pgs: 97 active+clean; 2.9 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 116 MiB/s wr, 72 op/s 2024-03-27T10:58:33.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:58:33 smithi177 ceph-mon[35294]: pgmap v498: 97 pgs: 97 active+clean; 2.9 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 116 MiB/s wr, 72 op/s 2024-03-27T10:58:33.766 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:58:33 smithi114 ceph-mon[35338]: pgmap v498: 97 pgs: 97 active+clean; 2.9 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 116 MiB/s wr, 72 op/s 2024-03-27T10:58:35.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:58:35 smithi097 ceph-mon[31859]: pgmap v499: 97 pgs: 97 active+clean; 3.1 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 119 MiB/s wr, 77 op/s 2024-03-27T10:58:35.664 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.693+0000 7f7f33200640 1 -- 172.21.15.97:0/306903018 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f2c105700 msgr2=0x7f7f2c105b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:35.664 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.693+0000 7f7f33200640 1 --2- 172.21.15.97:0/306903018 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f2c105700 0x7f7f2c105b80 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f7f24009ca0 tx=0x7f7f2402c100 comp rx=0 tx=0).stop 2024-03-27T10:58:35.665 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.694+0000 7f7f33200640 1 -- 172.21.15.97:0/306903018 shutdown_connections 2024-03-27T10:58:35.665 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.694+0000 7f7f33200640 1 --2- 172.21.15.97:0/306903018 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f2c105700 0x7f7f2c105b80 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:35.665 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.694+0000 7f7f33200640 1 --2- 172.21.15.97:0/306903018 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7f2c102d50 0x7f7f2c1051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:35.665 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.694+0000 7f7f33200640 1 --2- 172.21.15.97:0/306903018 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7f2c1003e0 0x7f7f2c102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:35.665 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.694+0000 7f7f33200640 1 -- 172.21.15.97:0/306903018 >> 172.21.15.97:0/306903018 conn(0x7f7f2c0fbfe0 msgr2=0x7f7f2c0fe420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:35.665 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.694+0000 7f7f33200640 1 -- 172.21.15.97:0/306903018 shutdown_connections 2024-03-27T10:58:35.665 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.694+0000 7f7f33200640 1 -- 172.21.15.97:0/306903018 wait complete. 2024-03-27T10:58:35.666 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.695+0000 7f7f33200640 1 Processor -- start 2024-03-27T10:58:35.666 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.696+0000 7f7f33200640 1 -- start start 2024-03-27T10:58:35.667 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.696+0000 7f7f33200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7f2c1003e0 0x7f7f2c19e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:35.667 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.696+0000 7f7f33200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f2c102d50 0x7f7f2c19e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:35.667 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.696+0000 7f7f33200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7f2c105700 0x7f7f2c1a5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:35.667 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.696+0000 7f7f33200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7f2c113cc0 con 0x7f7f2c102d50 2024-03-27T10:58:35.667 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.696+0000 7f7f33200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f7f2c113b40 con 0x7f7f2c1003e0 2024-03-27T10:58:35.667 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.696+0000 7f7f33200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f7f2c113e40 con 0x7f7f2c105700 2024-03-27T10:58:35.668 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.697+0000 7f7f2be00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7f2c1003e0 0x7f7f2c19e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:35.668 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.697+0000 7f7f2b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f2c102d50 0x7f7f2c19e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:35.668 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.697+0000 7f7f2b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f2c102d50 0x7f7f2c19e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37362/0 (socket says 172.21.15.97:37362) 2024-03-27T10:58:35.668 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.697+0000 7f7f2b400640 1 -- 172.21.15.97:0/568838687 learned_addr learned my addr 172.21.15.97:0/568838687 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:58:35.668 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.697+0000 7f7f31200640 1 --2- 172.21.15.97:0/568838687 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7f2c105700 0x7f7f2c1a5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:35.668 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.697+0000 7f7f2b400640 1 -- 172.21.15.97:0/568838687 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7f2c105700 msgr2=0x7f7f2c1a5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:35.668 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.697+0000 7f7f2b400640 1 --2- 172.21.15.97:0/568838687 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7f2c105700 0x7f7f2c1a5880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:35.668 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.697+0000 7f7f2b400640 1 -- 172.21.15.97:0/568838687 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7f2c1003e0 msgr2=0x7f7f2c19e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:35.668 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.697+0000 7f7f2b400640 1 --2- 172.21.15.97:0/568838687 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7f2c1003e0 0x7f7f2c19e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:35.668 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.697+0000 7f7f2b400640 1 -- 172.21.15.97:0/568838687 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f2c1a5ef0 con 0x7f7f2c102d50 2024-03-27T10:58:35.669 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.697+0000 7f7f31200640 1 --2- 172.21.15.97:0/568838687 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7f2c105700 0x7f7f2c1a5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T10:58:35.669 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.698+0000 7f7f2b400640 1 --2- 172.21.15.97:0/568838687 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f2c102d50 0x7f7f2c19e810 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f7f1800e9f0 tx=0x7f7f1800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:35.669 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.699+0000 7f7f28c00640 1 -- 172.21.15.97:0/568838687 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f1800cd80 con 0x7f7f2c102d50 2024-03-27T10:58:35.670 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.699+0000 7f7f28c00640 1 -- 172.21.15.97:0/568838687 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7f180108e0 con 0x7f7f2c102d50 2024-03-27T10:58:35.670 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.699+0000 7f7f28c00640 1 -- 172.21.15.97:0/568838687 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f18017890 con 0x7f7f2c102d50 2024-03-27T10:58:35.670 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.699+0000 7f7f33200640 1 -- 172.21.15.97:0/568838687 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f2c1a61e0 con 0x7f7f2c102d50 2024-03-27T10:58:35.670 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.699+0000 7f7f33200640 1 -- 172.21.15.97:0/568838687 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f2c1a66e0 con 0x7f7f2c102d50 2024-03-27T10:58:35.674 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.702+0000 7f7f35ddf740 1 Processor -- start 2024-03-27T10:58:35.674 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.703+0000 7f7f35ddf740 1 -- start start 2024-03-27T10:58:35.674 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.703+0000 7f7f35ddf740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5613c3914980 0x5613c3914da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:35.674 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.703+0000 7f7f35ddf740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5613c3915370 0x5613c3919e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:35.675 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.703+0000 7f7f35ddf740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5613c391a4e0 0x5613c391c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:35.675 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.703+0000 7f7f35ddf740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5613c3879d10 con 0x5613c3915370 2024-03-27T10:58:35.675 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.703+0000 7f7f35ddf740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5613c3822260 con 0x5613c391a4e0 2024-03-27T10:58:35.675 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.703+0000 7f7f35ddf740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5613c3875cc0 con 0x5613c3914980 2024-03-27T10:58:35.675 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.703+0000 7f7f2be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5613c3915370 0x5613c3919e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:35.675 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.704+0000 7f7f31200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5613c3914980 0x5613c3914da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:35.675 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.704+0000 7f7f2be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5613c3915370 0x5613c3919e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37372/0 (socket says 172.21.15.97:37372) 2024-03-27T10:58:35.675 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.704+0000 7f7f2be00640 1 -- 172.21.15.97:0/3333967979 learned_addr learned my addr 172.21.15.97:0/3333967979 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:58:35.676 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.704+0000 7f7f31200640 1 -- 172.21.15.97:0/3333967979 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5613c391a4e0 msgr2=0x5613c391c970 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-27T10:58:35.676 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.704+0000 7f7f31200640 1 --2- 172.21.15.97:0/3333967979 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5613c391a4e0 0x5613c391c970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:35.676 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.704+0000 7f7f31200640 1 -- 172.21.15.97:0/3333967979 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5613c3915370 msgr2=0x5613c3919e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:35.676 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.704+0000 7f7f31200640 1 --2- 172.21.15.97:0/3333967979 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5613c3915370 0x5613c3919e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:35.676 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.704+0000 7f7f31200640 1 -- 172.21.15.97:0/3333967979 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613c3819aa0 con 0x5613c3914980 2024-03-27T10:58:35.676 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.704+0000 7f7f2be00640 1 --2- 172.21.15.97:0/3333967979 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5613c3915370 0x5613c3919e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T10:58:35.676 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.705+0000 7f7f31200640 1 --2- 172.21.15.97:0/3333967979 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5613c3914980 0x5613c3914da0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7f24009bf0 tx=0x7f7f24005f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=242598912ff0f54a server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:35.677 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.705+0000 7f7f21600640 1 -- 172.21.15.97:0/3333967979 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f24043070 con 0x5613c3914980 2024-03-27T10:58:35.677 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.705+0000 7f7f28c00640 1 -- 172.21.15.97:0/568838687 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f7f18010a90 con 0x7f7f2c102d50 2024-03-27T10:58:35.677 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.705+0000 7f7f21600640 1 -- 172.21.15.97:0/3333967979 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7f24003970 con 0x5613c3914980 2024-03-27T10:58:35.677 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.705+0000 7f7f21600640 1 -- 172.21.15.97:0/3333967979 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f24039040 con 0x5613c3914980 2024-03-27T10:58:35.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:58:35 smithi177 ceph-mon[35294]: pgmap v499: 97 pgs: 97 active+clean; 3.1 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 119 MiB/s wr, 77 op/s 2024-03-27T10:58:35.678 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.706+0000 7f7f35ddf740 1 -- 172.21.15.97:0/3333967979 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5613c3914980 msgr2=0x5613c3914da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:35.679 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.706+0000 7f7f35ddf740 1 --2- 172.21.15.97:0/3333967979 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5613c3914980 0x5613c3914da0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7f24009bf0 tx=0x7f7f24005f60 comp rx=0 tx=0).stop 2024-03-27T10:58:35.679 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.706+0000 7f7f35ddf740 1 -- 172.21.15.97:0/3333967979 shutdown_connections 2024-03-27T10:58:35.679 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.706+0000 7f7f35ddf740 1 --2- 172.21.15.97:0/3333967979 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5613c391a4e0 0x5613c391c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:35.679 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.706+0000 7f7f35ddf740 1 --2- 172.21.15.97:0/3333967979 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5613c3915370 0x5613c3919e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:35.679 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.706+0000 7f7f35ddf740 1 --2- 172.21.15.97:0/3333967979 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5613c3914980 0x5613c3914da0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:35.679 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.706+0000 7f7f35ddf740 1 -- 172.21.15.97:0/3333967979 >> 172.21.15.97:0/3333967979 conn(0x5613c390fef0 msgr2=0x5613c39102f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:35.679 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.707+0000 7f7f35ddf740 1 -- 172.21.15.97:0/3333967979 shutdown_connections 2024-03-27T10:58:35.679 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.707+0000 7f7f35ddf740 1 -- 172.21.15.97:0/3333967979 wait complete. 2024-03-27T10:58:35.679 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.707+0000 7f7f28c00640 1 --2- 172.21.15.97:0/568838687 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f7f100a3780 0x7f7f100a5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:35.679 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.707+0000 7f7f28c00640 1 -- 172.21.15.97:0/568838687 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f7f180d0180 con 0x7f7f2c102d50 2024-03-27T10:58:35.680 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.707+0000 7f7f35ddf740 1 -- start start 2024-03-27T10:58:35.680 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.708+0000 7f7f2be00640 1 --2- 172.21.15.97:0/568838687 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f7f100a3780 0x7f7f100a5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:35.680 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.708+0000 7f7f35ddf740 1 Processor -- start 2024-03-27T10:58:35.683 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.708+0000 7f7f35ddf740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5613c3914980 0x5613c391ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:35.683 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.708+0000 7f7f35ddf740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5613c3915370 0x5613c391f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:35.683 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.708+0000 7f7f35ddf740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5613c391a4e0 0x5613c3926290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:35.683 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.708+0000 7f7f35ddf740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5613c391d430 con 0x5613c3915370 2024-03-27T10:58:35.684 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.708+0000 7f7f35ddf740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5613c3835630 con 0x5613c3914980 2024-03-27T10:58:35.684 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.708+0000 7f7f35ddf740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5613c391d5b0 con 0x5613c391a4e0 2024-03-27T10:58:35.684 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.708+0000 7f7f35ddf740 10 client.0 _ll_register_callbacks cb 0x5613c383ac50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:58:35.684 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.709+0000 7f7f2b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5613c3915370 0x5613c391f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:35.684 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.709+0000 7f7f2b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5613c3915370 0x5613c391f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37374/0 (socket says 172.21.15.97:37374) 2024-03-27T10:58:35.684 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.709+0000 7f7f2b400640 1 -- 172.21.15.97:0/2001505014 learned_addr learned my addr 172.21.15.97:0/2001505014 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:58:35.684 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.709+0000 7f7f31200640 1 --2- 172.21.15.97:0/2001505014 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5613c391a4e0 0x5613c3926290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:35.684 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.709+0000 7f7f2be00640 1 --2- 172.21.15.97:0/568838687 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f7f100a3780 0x7f7f100a5c40 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x5613c3910e60 tx=0x7f7f1c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:35.684 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.709+0000 7f7f2be00640 1 --2- 172.21.15.97:0/2001505014 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5613c3914980 0x5613c391ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:35.684 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.710+0000 7f7f2b400640 1 -- 172.21.15.97:0/2001505014 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5613c391a4e0 msgr2=0x5613c3926290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:35.685 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.710+0000 7f7f2b400640 1 --2- 172.21.15.97:0/2001505014 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5613c391a4e0 0x5613c3926290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:35.685 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.710+0000 7f7f2b400640 1 -- 172.21.15.97:0/2001505014 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5613c3914980 msgr2=0x5613c391ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:35.685 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.710+0000 7f7f2b400640 1 --2- 172.21.15.97:0/2001505014 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5613c3914980 0x5613c391ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:35.685 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.710+0000 7f7f2b400640 1 -- 172.21.15.97:0/2001505014 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613c38415f0 con 0x5613c3915370 2024-03-27T10:58:35.685 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.710+0000 7f7f2be00640 1 --2- 172.21.15.97:0/2001505014 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5613c3914980 0x5613c391ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:58:35.685 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.710+0000 7f7f2b400640 1 --2- 172.21.15.97:0/2001505014 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5613c3915370 0x5613c391f250 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f7f1809c590 tx=0x7f7f1809cd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6d88f1a2bd6b5e8 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:35.685 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.710+0000 7f7f16000640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T10:58:35.685 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.710+0000 7f7f16000640 1 -- 172.21.15.97:0/2001505014 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f180b4070 con 0x5613c3915370 2024-03-27T10:58:35.685 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.710+0000 7f7f16000640 1 -- 172.21.15.97:0/2001505014 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f7f180a4fe0 con 0x5613c3915370 2024-03-27T10:58:35.686 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.710+0000 7f7f35ddf740 1 -- 172.21.15.97:0/2001505014 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f24003970 con 0x5613c3915370 2024-03-27T10:58:35.686 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.711+0000 7f7f16000640 1 -- 172.21.15.97:0/2001505014 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f7f18098040 con 0x5613c3915370 2024-03-27T10:58:35.686 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.711+0000 7f7f35ddf740 10 client.16218 fetch_fsmap learned FSMap version 31 2024-03-27T10:58:35.686 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.711+0000 7f7f35ddf740 1 -- 172.21.15.97:0/2001505014 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5613c358e210 con 0x5613c3915370 2024-03-27T10:58:35.686 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.711+0000 7f7f16000640 1 -- 172.21.15.97:0/2001505014 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7f24003970 con 0x5613c3915370 2024-03-27T10:58:35.686 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.712+0000 7f7f16000640 1 -- 172.21.15.97:0/2001505014 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 31) ==== 8370+0+0 (secure 0 0 0) 0x7f7f18098350 con 0x5613c3915370 2024-03-27T10:58:35.686 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.712+0000 7f7f35ddf740 10 client.16218 fetch_fsmap finished waiting for FSMap version 31 2024-03-27T10:58:35.686 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.712+0000 7f7f35ddf740 10 client.16218 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T10:58:35.686 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.712+0000 7f7f35ddf740 20 client.16218 populate_metadata read hostname 'smithi097' 2024-03-27T10:58:35.687 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.712+0000 7f7f35ddf740 1 --2- 172.21.15.97:0/2001505014 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5613c392ac60 0x5613c392d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:35.687 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.712+0000 7f7f35ddf740 4 client.16218 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T10:58:35.687 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.712+0000 7f7f35ddf740 1 -- 172.21.15.97:0/2001505014 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5613c3887250 con 0x5613c392ac60 2024-03-27T10:58:35.687 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.712+0000 7f7f2be00640 1 --2- 172.21.15.97:0/2001505014 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5613c392ac60 0x5613c392d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:35.688 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.713+0000 7f7f2be00640 1 --2- 172.21.15.97:0/2001505014 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5613c392ac60 0x5613c392d080 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b6ba889845d7206 server_cookie=3e80a823e73bc9d1 in_seq=0 out_seq=0 2024-03-27T10:58:35.688 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.713+0000 7f7f16000640 10 client.16218 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T10:58:35.689 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.714+0000 7f7f16000640 1 -- 172.21.15.97:0/2001505014 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5613c3887250 con 0x5613c392ac60 2024-03-27T10:58:35.689 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.714+0000 7f7f16000640 10 client.16218 handle_command_reply: tid=0 2024-03-27T10:58:35.689 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.714+0000 7f7f35ddf740 1 client.16218 shutdown 2024-03-27T10:58:35.689 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.714+0000 7f7f35ddf740 1 -- 172.21.15.97:0/2001505014 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5613c3915370 msgr2=0x5613c391f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:35.689 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.714+0000 7f7f35ddf740 1 --2- 172.21.15.97:0/2001505014 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5613c3915370 0x5613c391f250 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f7f1809c590 tx=0x7f7f1809cd00 comp rx=0 tx=0).stop 2024-03-27T10:58:35.689 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.715+0000 7f7f35ddf740 1 -- 172.21.15.97:0/2001505014 shutdown_connections 2024-03-27T10:58:35.689 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.715+0000 7f7f35ddf740 1 --2- 172.21.15.97:0/2001505014 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5613c392ac60 0x5613c392d080 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:35.689 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.715+0000 7f7f35ddf740 1 --2- 172.21.15.97:0/2001505014 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5613c391a4e0 0x5613c3926290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:35.689 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.715+0000 7f7f35ddf740 1 --2- 172.21.15.97:0/2001505014 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5613c3915370 0x5613c391f250 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:35.689 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.715+0000 7f7f16000640 0 client.16218 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T10:58:35.690 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.715+0000 7f7f35ddf740 1 --2- 172.21.15.97:0/2001505014 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5613c3914980 0x5613c391ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:35.690 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.715+0000 7f7f35ddf740 1 -- 172.21.15.97:0/2001505014 >> 172.21.15.97:0/2001505014 conn(0x5613c390fef0 msgr2=0x5613c390f0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:35.690 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.715+0000 7f7f35ddf740 1 -- 172.21.15.97:0/2001505014 shutdown_connections 2024-03-27T10:58:35.690 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.715+0000 7f7f35ddf740 1 -- 172.21.15.97:0/2001505014 wait complete. 2024-03-27T10:58:35.690 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.716+0000 7f7f35ddf740 20 client.16218 trim_cache size 0 max 16384 2024-03-27T10:58:35.712 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.741+0000 7f7f35ddf740 1 Processor -- start 2024-03-27T10:58:35.713 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.741+0000 7f7f35ddf740 1 -- start start 2024-03-27T10:58:35.713 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.741+0000 7f7f35ddf740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5613c3914980 0x5613c3919da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:35.713 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.741+0000 7f7f35ddf740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5613c3915370 0x5613c3926250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:35.713 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.741+0000 7f7f35ddf740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5613c391a4e0 0x5613c390e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:35.713 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.741+0000 7f7f35ddf740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7f2c1af410 con 0x5613c3915370 2024-03-27T10:58:35.713 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.741+0000 7f7f35ddf740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f7f2c1b1010 con 0x5613c3914980 2024-03-27T10:58:35.713 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.741+0000 7f7f35ddf740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f7f2c1afae0 con 0x5613c391a4e0 2024-03-27T10:58:35.713 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.742+0000 7f7f2b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5613c3915370 0x5613c3926250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:35.713 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.742+0000 7f7f2b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5613c3915370 0x5613c3926250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37378/0 (socket says 172.21.15.97:37378) 2024-03-27T10:58:35.713 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.742+0000 7f7f2b400640 1 -- 172.21.15.97:0/2112858740 learned_addr learned my addr 172.21.15.97:0/2112858740 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:58:35.713 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.742+0000 7f7f2be00640 1 --2- 172.21.15.97:0/2112858740 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5613c3914980 0x5613c3919da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:35.713 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.742+0000 7f7f31200640 1 --2- 172.21.15.97:0/2112858740 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5613c391a4e0 0x5613c390e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:35.714 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.742+0000 7f7f2b400640 1 -- 172.21.15.97:0/2112858740 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5613c391a4e0 msgr2=0x5613c390e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:35.714 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.742+0000 7f7f2b400640 1 --2- 172.21.15.97:0/2112858740 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5613c391a4e0 0x5613c390e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:35.714 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.742+0000 7f7f2b400640 1 -- 172.21.15.97:0/2112858740 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5613c3914980 msgr2=0x5613c3919da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:35.714 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.742+0000 7f7f2b400640 1 --2- 172.21.15.97:0/2112858740 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5613c3914980 0x5613c3919da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:35.715 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.742+0000 7f7f2b400640 1 -- 172.21.15.97:0/2112858740 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613c365a620 con 0x5613c3915370 2024-03-27T10:58:35.715 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.742+0000 7f7f2be00640 1 --2- 172.21.15.97:0/2112858740 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5613c3914980 0x5613c3919da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T10:58:35.715 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.743+0000 7f7f2b400640 1 --2- 172.21.15.97:0/2112858740 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5613c3915370 0x5613c3926250 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f7f18098fd0 tx=0x7f7f1809ab10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bcff2cf735cbca9 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:35.715 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.743+0000 7f7f20c00640 1 -- 172.21.15.97:0/2112858740 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f180b4070 con 0x5613c3915370 2024-03-27T10:58:35.715 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.743+0000 7f7f20c00640 1 -- 172.21.15.97:0/2112858740 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7f180bb020 con 0x5613c3915370 2024-03-27T10:58:35.715 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.743+0000 7f7f20c00640 1 -- 172.21.15.97:0/2112858740 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f180bb1d0 con 0x5613c3915370 2024-03-27T10:58:35.716 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.744+0000 7f7f35ddf740 1 -- 172.21.15.97:0/2112858740 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5613c3915370 msgr2=0x5613c3926250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:35.716 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.744+0000 7f7f35ddf740 1 --2- 172.21.15.97:0/2112858740 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5613c3915370 0x5613c3926250 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f7f18098fd0 tx=0x7f7f1809ab10 comp rx=0 tx=0).stop 2024-03-27T10:58:35.716 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.744+0000 7f7f35ddf740 1 -- 172.21.15.97:0/2112858740 shutdown_connections 2024-03-27T10:58:35.716 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.744+0000 7f7f35ddf740 1 --2- 172.21.15.97:0/2112858740 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5613c391a4e0 0x5613c390e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:35.716 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.744+0000 7f7f35ddf740 1 --2- 172.21.15.97:0/2112858740 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5613c3915370 0x5613c3926250 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:35.716 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.744+0000 7f7f35ddf740 1 --2- 172.21.15.97:0/2112858740 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5613c3914980 0x5613c3919da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:35.716 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.744+0000 7f7f35ddf740 1 -- 172.21.15.97:0/2112858740 >> 172.21.15.97:0/2112858740 conn(0x5613c390fef0 msgr2=0x5613c3916730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:35.716 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.744+0000 7f7f35ddf740 1 -- 172.21.15.97:0/2112858740 shutdown_connections 2024-03-27T10:58:35.716 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.745+0000 7f7f35ddf740 1 -- 172.21.15.97:0/2112858740 wait complete. 2024-03-27T10:58:35.717 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.745+0000 7f7f35ddf740 1 -- start start 2024-03-27T10:58:35.717 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.745+0000 7f7f35ddf740 1 Processor -- start 2024-03-27T10:58:35.718 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.746+0000 7f7f35ddf740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5613c3914980 0x5613c3923620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:35.718 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.746+0000 7f7f35ddf740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5613c3915370 0x5613c3923b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:35.718 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.746+0000 7f7f35ddf740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5613c391a4e0 0x5613c3924b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:35.718 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.746+0000 7f7f35ddf740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7f2c1b0880 con 0x5613c391a4e0 2024-03-27T10:58:35.718 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.746+0000 7f7f35ddf740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f7f2c1b01b0 con 0x5613c3914980 2024-03-27T10:58:35.718 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.746+0000 7f7f35ddf740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f7f2c1ad3f0 con 0x5613c3915370 2024-03-27T10:58:35.718 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.746+0000 7f7f35ddf740 10 client.0 _ll_register_callbacks cb 0x7f7f180981e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:58:35.718 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.746+0000 7f7f2b400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5613c3915370 0x5613c3923b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:35.718 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.746+0000 7f7f2b400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5613c3915370 0x5613c3923b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.97:35222/0 (socket says 172.21.15.97:35222) 2024-03-27T10:58:35.718 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.746+0000 7f7f2b400640 1 -- 172.21.15.97:0/980669899 learned_addr learned my addr 172.21.15.97:0/980669899 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:58:35.718 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.746+0000 7f7f2be00640 1 --2- 172.21.15.97:0/980669899 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5613c3914980 0x5613c3923620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:35.719 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.746+0000 7f7f31200640 1 --2- 172.21.15.97:0/980669899 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5613c391a4e0 0x5613c3924b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:35.719 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.747+0000 7f7f2be00640 1 -- 172.21.15.97:0/980669899 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5613c3915370 msgr2=0x5613c3923b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:35.719 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.747+0000 7f7f2be00640 1 --2- 172.21.15.97:0/980669899 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5613c3915370 0x5613c3923b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:35.719 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.747+0000 7f7f2be00640 1 -- 172.21.15.97:0/980669899 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5613c391a4e0 msgr2=0x5613c3924b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:35.719 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.747+0000 7f7f2be00640 1 --2- 172.21.15.97:0/980669899 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5613c391a4e0 0x5613c3924b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:35.719 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.747+0000 7f7f2be00640 1 -- 172.21.15.97:0/980669899 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613c3925160 con 0x5613c3914980 2024-03-27T10:58:35.719 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.747+0000 7f7f2be00640 1 --2- 172.21.15.97:0/980669899 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5613c3914980 0x5613c3923620 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x5613c392d8f0 tx=0x7f7f1c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a45e0fa125329d54 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:35.719 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.747+0000 7f7f16000640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-03-27T10:58:35.720 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.747+0000 7f7f16000640 1 -- 172.21.15.97:0/980669899 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f1c004460 con 0x5613c3914980 2024-03-27T10:58:35.720 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.747+0000 7f7f16000640 1 -- 172.21.15.97:0/980669899 <== mon.2 v2:172.21.15.114:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f7f1c02a3b0 con 0x5613c3914980 2024-03-27T10:58:35.720 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.747+0000 7f7f35ddf740 1 -- 172.21.15.97:0/980669899 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f180bb020 con 0x5613c3914980 2024-03-27T10:58:35.720 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.748+0000 7f7f16000640 1 -- 172.21.15.97:0/980669899 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f7f1c02a5b0 con 0x5613c3914980 2024-03-27T10:58:35.720 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.748+0000 7f7f35ddf740 10 client.35870 fetch_fsmap learned FSMap version 31 2024-03-27T10:58:35.721 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.748+0000 7f7f35ddf740 1 -- 172.21.15.97:0/980669899 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5613c39264e0 con 0x5613c3914980 2024-03-27T10:58:35.721 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.748+0000 7f7f16000640 1 -- 172.21.15.97:0/980669899 <== mon.2 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7f180bb020 con 0x5613c3914980 2024-03-27T10:58:35.721 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.749+0000 7f7f16000640 1 -- 172.21.15.97:0/980669899 <== mon.2 v2:172.21.15.114:3300/0 5 ==== fsmap(e 31) ==== 8370+0+0 (secure 0 0 0) 0x7f7f1c033420 con 0x5613c3914980 2024-03-27T10:58:35.721 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.749+0000 7f7f35ddf740 10 client.35870 fetch_fsmap finished waiting for FSMap version 31 2024-03-27T10:58:35.721 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.749+0000 7f7f35ddf740 10 client.35870 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T10:58:35.721 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.749+0000 7f7f35ddf740 20 client.35870 populate_metadata read hostname 'smithi097' 2024-03-27T10:58:35.721 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.749+0000 7f7f35ddf740 1 --2- 172.21.15.97:0/980669899 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5613c3991780 0x5613c3993ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:35.721 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.749+0000 7f7f35ddf740 4 client.35870 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-27T10:58:35.721 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.749+0000 7f7f35ddf740 1 -- 172.21.15.97:0/980669899 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5613c38815a0 con 0x5613c3991780 2024-03-27T10:58:35.722 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.750+0000 7f7f2b400640 1 --2- 172.21.15.97:0/980669899 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5613c3991780 0x5613c3993ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:35.722 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.751+0000 7f7f2b400640 1 --2- 172.21.15.97:0/980669899 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5613c3991780 0x5613c3993ba0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4631cce4463f3481 server_cookie=adb491cb1ccfc50c in_seq=0 out_seq=0 2024-03-27T10:58:35.722 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.751+0000 7f7f16000640 10 client.35870 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T10:58:35.742 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.771+0000 7f7f16000640 1 -- 172.21.15.97:0/980669899 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5613c38815a0 con 0x5613c3991780 2024-03-27T10:58:35.742 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.771+0000 7f7f16000640 10 client.35870 handle_command_reply: tid=0 2024-03-27T10:58:35.742 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.772+0000 7f7f35ddf740 1 client.35870 shutdown 2024-03-27T10:58:35.743 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.772+0000 7f7f35ddf740 1 -- 172.21.15.97:0/980669899 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5613c3914980 msgr2=0x5613c3923620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:35.743 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.772+0000 7f7f35ddf740 1 --2- 172.21.15.97:0/980669899 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5613c3914980 0x5613c3923620 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x5613c392d8f0 tx=0x7f7f1c004730 comp rx=0 tx=0).stop 2024-03-27T10:58:35.744 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.772+0000 7f7f35ddf740 1 -- 172.21.15.97:0/980669899 shutdown_connections 2024-03-27T10:58:35.744 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.772+0000 7f7f35ddf740 1 --2- 172.21.15.97:0/980669899 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5613c3991780 0x5613c3993ba0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:35.744 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.772+0000 7f7f35ddf740 1 --2- 172.21.15.97:0/980669899 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5613c391a4e0 0x5613c3924b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:35.744 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.772+0000 7f7f35ddf740 1 --2- 172.21.15.97:0/980669899 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5613c3915370 0x5613c3923b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:35.744 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.772+0000 7f7f35ddf740 1 --2- 172.21.15.97:0/980669899 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5613c3914980 0x5613c3923620 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:35.744 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.772+0000 7f7f35ddf740 1 -- 172.21.15.97:0/980669899 >> 172.21.15.97:0/980669899 conn(0x5613c390fef0 msgr2=0x5613c3913150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:35.744 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.772+0000 7f7f16000640 0 client.35870 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T10:58:35.744 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.772+0000 7f7f35ddf740 1 -- 172.21.15.97:0/980669899 shutdown_connections 2024-03-27T10:58:35.744 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.772+0000 7f7f35ddf740 1 -- 172.21.15.97:0/980669899 wait complete. 2024-03-27T10:58:35.744 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.772+0000 7f7f35ddf740 20 client.35870 trim_cache size 0 max 16384 2024-03-27T10:58:35.745 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-27T10:58:35.745 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-27T10:58:35.745 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-27T10:58:35.745 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-27T10:58:35.746 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.773+0000 7f7f33200640 1 -- 172.21.15.97:0/568838687 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f7f100a3780 msgr2=0x7f7f100a5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:35.746 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.773+0000 7f7f33200640 1 --2- 172.21.15.97:0/568838687 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f7f100a3780 0x7f7f100a5c40 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x5613c3910e60 tx=0x7f7f1c005f90 comp rx=0 tx=0).stop 2024-03-27T10:58:35.746 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.773+0000 7f7f33200640 1 -- 172.21.15.97:0/568838687 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f2c102d50 msgr2=0x7f7f2c19e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:35.746 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.773+0000 7f7f33200640 1 --2- 172.21.15.97:0/568838687 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f2c102d50 0x7f7f2c19e810 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f7f1800e9f0 tx=0x7f7f1800eeb0 comp rx=0 tx=0).stop 2024-03-27T10:58:35.746 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.774+0000 7f7f33200640 1 -- 172.21.15.97:0/568838687 shutdown_connections 2024-03-27T10:58:35.746 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.774+0000 7f7f33200640 1 --2- 172.21.15.97:0/568838687 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7f2c105700 0x7f7f2c1a5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:35.746 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.774+0000 7f7f33200640 1 --2- 172.21.15.97:0/568838687 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f2c102d50 0x7f7f2c19e810 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:35.746 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.774+0000 7f7f33200640 1 --2- 172.21.15.97:0/568838687 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f7f100a3780 0x7f7f100a5c40 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:35.746 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.774+0000 7f7f33200640 1 --2- 172.21.15.97:0/568838687 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7f2c1003e0 0x7f7f2c19e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:35.746 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.774+0000 7f7f33200640 1 -- 172.21.15.97:0/568838687 >> 172.21.15.97:0/568838687 conn(0x7f7f2c0fbfe0 msgr2=0x7f7f2c100d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:35.746 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.774+0000 7f7f33200640 1 -- 172.21.15.97:0/568838687 shutdown_connections 2024-03-27T10:58:35.746 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:35.774+0000 7f7f33200640 1 -- 172.21.15.97:0/568838687 wait complete. 2024-03-27T10:58:35.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:58:35 smithi114 ceph-mon[35338]: pgmap v499: 97 pgs: 97 active+clean; 3.1 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 119 MiB/s wr, 77 op/s 2024-03-27T10:58:37.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:58:37 smithi097 ceph-mon[31859]: pgmap v500: 97 pgs: 97 active+clean; 3.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 120 MiB/s wr, 72 op/s 2024-03-27T10:58:37.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:58:37 smithi177 ceph-mon[35294]: pgmap v500: 97 pgs: 97 active+clean; 3.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 120 MiB/s wr, 72 op/s 2024-03-27T10:58:37.766 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:58:37 smithi114 ceph-mon[35338]: pgmap v500: 97 pgs: 97 active+clean; 3.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 120 MiB/s wr, 72 op/s 2024-03-27T10:58:39.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:58:39 smithi097 ceph-mon[31859]: pgmap v501: 97 pgs: 97 active+clean; 3.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 129 MiB/s wr, 76 op/s 2024-03-27T10:58:39.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:58:39 smithi177 ceph-mon[35294]: pgmap v501: 97 pgs: 97 active+clean; 3.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 129 MiB/s wr, 76 op/s 2024-03-27T10:58:39.766 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:58:39 smithi114 ceph-mon[35338]: pgmap v501: 97 pgs: 97 active+clean; 3.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 129 MiB/s wr, 76 op/s 2024-03-27T10:58:40.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:58:40 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:58:40.679 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:58:40 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:58:40.766 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:58:40 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:58:41.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:58:41 smithi097 ceph-mon[31859]: pgmap v502: 97 pgs: 97 active+clean; 3.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 139 MiB/s wr, 84 op/s 2024-03-27T10:58:41.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:58:41 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:58:41.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:58:41 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:58:41.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:58:41 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:58:41.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:58:41 smithi177 ceph-mon[35294]: pgmap v502: 97 pgs: 97 active+clean; 3.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 139 MiB/s wr, 84 op/s 2024-03-27T10:58:41.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:58:41 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:58:41.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:58:41 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:58:41.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:58:41 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:58:41.766 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:58:41 smithi114 ceph-mon[35338]: pgmap v502: 97 pgs: 97 active+clean; 3.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 139 MiB/s wr, 84 op/s 2024-03-27T10:58:41.766 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:58:41 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:58:41.766 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:58:41 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:58:41.766 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:58:41 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:58:43.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:58:43 smithi097 ceph-mon[31859]: pgmap v503: 97 pgs: 97 active+clean; 4.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 123 MiB/s wr, 73 op/s 2024-03-27T10:58:43.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:58:43 smithi177 ceph-mon[35294]: pgmap v503: 97 pgs: 97 active+clean; 4.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 123 MiB/s wr, 73 op/s 2024-03-27T10:58:43.766 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:58:43 smithi114 ceph-mon[35338]: pgmap v503: 97 pgs: 97 active+clean; 4.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 123 MiB/s wr, 73 op/s 2024-03-27T10:58:45.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:58:45 smithi097 ceph-mon[31859]: pgmap v504: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 124 MiB/s wr, 76 op/s 2024-03-27T10:58:45.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:58:45 smithi177 ceph-mon[35294]: pgmap v504: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 124 MiB/s wr, 76 op/s 2024-03-27T10:58:45.766 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:58:45 smithi114 ceph-mon[35338]: pgmap v504: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 124 MiB/s wr, 76 op/s 2024-03-27T10:58:47.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:58:47 smithi097 ceph-mon[31859]: pgmap v505: 97 pgs: 97 active+clean; 4.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 122 MiB/s wr, 73 op/s 2024-03-27T10:58:47.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:58:47 smithi177 ceph-mon[35294]: pgmap v505: 97 pgs: 97 active+clean; 4.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 122 MiB/s wr, 73 op/s 2024-03-27T10:58:47.766 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:58:47 smithi114 ceph-mon[35338]: pgmap v505: 97 pgs: 97 active+clean; 4.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 122 MiB/s wr, 73 op/s 2024-03-27T10:58:48.933 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.961+0000 7f7f30a00640 1 -- 172.21.15.97:0/3913632832 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f2c103da0 msgr2=0x7f7f2c1041c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:48.933 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.961+0000 7f7f30a00640 1 --2- 172.21.15.97:0/3913632832 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f2c103da0 0x7f7f2c1041c0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c009900 tx=0x7f7f1c02bc40 comp rx=0 tx=0).stop 2024-03-27T10:58:48.933 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.961+0000 7f7f30a00640 1 -- 172.21.15.97:0/3913632832 shutdown_connections 2024-03-27T10:58:48.933 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.961+0000 7f7f30a00640 1 --2- 172.21.15.97:0/3913632832 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7f2c105950 0x7f7f2c10c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:48.933 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.961+0000 7f7f30a00640 1 --2- 172.21.15.97:0/3913632832 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7f2c104f90 0x7f7f2c105410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:48.933 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.961+0000 7f7f30a00640 1 --2- 172.21.15.97:0/3913632832 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f2c103da0 0x7f7f2c1041c0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:48.933 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.961+0000 7f7f30a00640 1 -- 172.21.15.97:0/3913632832 >> 172.21.15.97:0/3913632832 conn(0x7f7f2c0ff310 msgr2=0x7f7f2c101750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:48.933 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.961+0000 7f7f30a00640 1 -- 172.21.15.97:0/3913632832 shutdown_connections 2024-03-27T10:58:48.933 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.961+0000 7f7f30a00640 1 -- 172.21.15.97:0/3913632832 wait complete. 2024-03-27T10:58:48.934 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.962+0000 7f7f30a00640 1 Processor -- start 2024-03-27T10:58:48.934 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.963+0000 7f7f30a00640 1 -- start start 2024-03-27T10:58:48.935 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.963+0000 7f7f30a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7f2c103da0 0x7f7f2c19e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:48.935 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.963+0000 7f7f30a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7f2c104f90 0x7f7f2c19e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:48.935 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.963+0000 7f7f30a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f2c105950 0x7f7f2c1a5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:48.935 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.963+0000 7f7f30a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7f2c10f970 con 0x7f7f2c105950 2024-03-27T10:58:48.935 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.963+0000 7f7f30a00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f7f2c10f7f0 con 0x7f7f2c104f90 2024-03-27T10:58:48.935 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.963+0000 7f7f30a00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f7f2c10faf0 con 0x7f7f2c103da0 2024-03-27T10:58:48.935 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.964+0000 7f7f29e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7f2c103da0 0x7f7f2c19e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:48.936 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.964+0000 7f7f29e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7f2c103da0 0x7f7f2c19e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.97:56710/0 (socket says 172.21.15.97:56710) 2024-03-27T10:58:48.936 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.964+0000 7f7f29e00640 1 -- 172.21.15.97:0/1475714463 learned_addr learned my addr 172.21.15.97:0/1475714463 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:58:48.936 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.964+0000 7f7f29400640 1 --2- 172.21.15.97:0/1475714463 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7f2c104f90 0x7f7f2c19e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:48.936 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.964+0000 7f7f2a800640 1 --2- 172.21.15.97:0/1475714463 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f2c105950 0x7f7f2c1a5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:48.936 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.964+0000 7f7f29e00640 1 -- 172.21.15.97:0/1475714463 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7f2c104f90 msgr2=0x7f7f2c19e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:48.936 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.964+0000 7f7f29e00640 1 --2- 172.21.15.97:0/1475714463 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7f2c104f90 0x7f7f2c19e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:48.936 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.964+0000 7f7f29e00640 1 -- 172.21.15.97:0/1475714463 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f2c105950 msgr2=0x7f7f2c1a5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:48.937 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.964+0000 7f7f29e00640 1 --2- 172.21.15.97:0/1475714463 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f2c105950 0x7f7f2c1a5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:48.937 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.964+0000 7f7f29e00640 1 -- 172.21.15.97:0/1475714463 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f2c1a5fc0 con 0x7f7f2c103da0 2024-03-27T10:58:48.937 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.965+0000 7f7f29e00640 1 --2- 172.21.15.97:0/1475714463 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7f2c103da0 0x7f7f2c19e3a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c005ab0 tx=0x7f7f1c0042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:48.937 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.965+0000 7f7f22a00640 1 -- 172.21.15.97:0/1475714463 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f1c043020 con 0x7f7f2c103da0 2024-03-27T10:58:48.937 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.965+0000 7f7f22a00640 1 -- 172.21.15.97:0/1475714463 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7f1c0054c0 con 0x7f7f2c103da0 2024-03-27T10:58:48.938 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.965+0000 7f7f22a00640 1 -- 172.21.15.97:0/1475714463 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f1c003d10 con 0x7f7f2c103da0 2024-03-27T10:58:48.938 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.966+0000 7f7f30a00640 1 -- 172.21.15.97:0/1475714463 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f2c1a6250 con 0x7f7f2c103da0 2024-03-27T10:58:48.938 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.966+0000 7f7f30a00640 1 -- 172.21.15.97:0/1475714463 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f2c1a67d0 con 0x7f7f2c103da0 2024-03-27T10:58:48.941 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.970+0000 7f7f337cf740 1 Processor -- start 2024-03-27T10:58:48.942 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.970+0000 7f7f337cf740 1 -- start start 2024-03-27T10:58:48.945 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.972+0000 7f7f22a00640 1 -- 172.21.15.97:0/1475714463 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f7f1c046020 con 0x7f7f2c103da0 2024-03-27T10:58:48.945 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.972+0000 7f7f337cf740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5569676dc980 0x5569676dcda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:48.945 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.972+0000 7f7f337cf740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5569676dd370 0x5569676e1e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:48.945 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.972+0000 7f7f337cf740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5569676e24e0 0x5569676e4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:48.945 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.972+0000 7f7f337cf740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x556967641d10 con 0x5569676e24e0 2024-03-27T10:58:48.945 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.972+0000 7f7f337cf740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5569675ea260 con 0x5569676dc980 2024-03-27T10:58:48.945 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.972+0000 7f7f337cf740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55696763dcc0 con 0x5569676dd370 2024-03-27T10:58:48.945 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.973+0000 7f7f29e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5569676dd370 0x5569676e1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:48.945 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.973+0000 7f7f29e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5569676dd370 0x5569676e1e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.97:56712/0 (socket says 172.21.15.97:56712) 2024-03-27T10:58:48.945 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.973+0000 7f7f29e00640 1 -- 172.21.15.97:0/1062236969 learned_addr learned my addr 172.21.15.97:0/1062236969 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:58:48.946 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.973+0000 7f7f29400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5569676e24e0 0x5569676e4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:48.946 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.973+0000 7f7f2a800640 1 --2- 172.21.15.97:0/1062236969 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5569676dc980 0x5569676dcda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:48.946 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.974+0000 7f7f22a00640 1 --2- 172.21.15.97:0/1475714463 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f7f100a36b0 0x7f7f100a5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:48.946 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.975+0000 7f7f29e00640 1 -- 172.21.15.97:0/1062236969 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5569676dc980 msgr2=0x5569676dcda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:48.946 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.975+0000 7f7f29e00640 1 --2- 172.21.15.97:0/1062236969 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5569676dc980 0x5569676dcda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:48.947 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.975+0000 7f7f29e00640 1 -- 172.21.15.97:0/1062236969 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5569676e24e0 msgr2=0x5569676e4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:48.947 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.975+0000 7f7f29e00640 1 --2- 172.21.15.97:0/1062236969 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5569676e24e0 0x5569676e4970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:48.947 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.975+0000 7f7f29e00640 1 -- 172.21.15.97:0/1062236969 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569675e1aa0 con 0x5569676dd370 2024-03-27T10:58:48.947 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.975+0000 7f7f29400640 1 --2- 172.21.15.97:0/1062236969 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5569676e24e0 0x5569676e4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T10:58:48.948 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.975+0000 7f7f2a800640 1 --2- 172.21.15.97:0/1062236969 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5569676dc980 0x5569676dcda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T10:58:48.948 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.975+0000 7f7f2a800640 1 --2- 172.21.15.97:0/1475714463 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f7f100a36b0 0x7f7f100a5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:48.948 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.975+0000 7f7f22a00640 1 -- 172.21.15.97:0/1475714463 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f7f1c0f1fd0 con 0x7f7f2c103da0 2024-03-27T10:58:48.948 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.976+0000 7f7f29e00640 1 --2- 172.21.15.97:0/1062236969 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5569676dd370 0x5569676e1e70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c0f5eb0 tx=0x7f7f1c0fa0e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee438a98ef7621c3 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:48.948 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.976+0000 7f7f17000640 1 -- 172.21.15.97:0/1062236969 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f1c0ba810 con 0x5569676dd370 2024-03-27T10:58:48.948 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.976+0000 7f7f2a800640 1 --2- 172.21.15.97:0/1475714463 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f7f100a36b0 0x7f7f100a5b70 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x5569676ddbd0 tx=0x7f7f24004b50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:48.948 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.976+0000 7f7f17000640 1 -- 172.21.15.97:0/1062236969 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7f1c004c50 con 0x5569676dd370 2024-03-27T10:58:48.948 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.976+0000 7f7f17000640 1 -- 172.21.15.97:0/1062236969 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f1c0044e0 con 0x5569676dd370 2024-03-27T10:58:48.949 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.978+0000 7f7f337cf740 1 -- 172.21.15.97:0/1062236969 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5569676dd370 msgr2=0x5569676e1e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:48.949 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.978+0000 7f7f337cf740 1 --2- 172.21.15.97:0/1062236969 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5569676dd370 0x5569676e1e70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c0f5eb0 tx=0x7f7f1c0fa0e0 comp rx=0 tx=0).stop 2024-03-27T10:58:48.950 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.978+0000 7f7f337cf740 1 -- 172.21.15.97:0/1062236969 shutdown_connections 2024-03-27T10:58:48.950 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.978+0000 7f7f337cf740 1 --2- 172.21.15.97:0/1062236969 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5569676e24e0 0x5569676e4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:48.951 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.979+0000 7f7f337cf740 1 --2- 172.21.15.97:0/1062236969 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5569676dd370 0x5569676e1e70 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:48.951 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.979+0000 7f7f337cf740 1 --2- 172.21.15.97:0/1062236969 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5569676dc980 0x5569676dcda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:48.951 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.979+0000 7f7f337cf740 1 -- 172.21.15.97:0/1062236969 >> 172.21.15.97:0/1062236969 conn(0x5569676d7ef0 msgr2=0x5569676d82f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:48.951 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.979+0000 7f7f337cf740 1 -- 172.21.15.97:0/1062236969 shutdown_connections 2024-03-27T10:58:48.951 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.979+0000 7f7f337cf740 1 -- 172.21.15.97:0/1062236969 wait complete. 2024-03-27T10:58:48.952 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.979+0000 7f7f337cf740 1 -- start start 2024-03-27T10:58:48.952 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.980+0000 7f7f337cf740 1 Processor -- start 2024-03-27T10:58:48.952 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.980+0000 7f7f337cf740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5569676dc980 0x5569676e6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:48.952 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.980+0000 7f7f337cf740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5569676dd370 0x5569676e7010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:48.953 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.980+0000 7f7f337cf740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5569676e24e0 0x5569676edfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:48.953 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.980+0000 7f7f337cf740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5569676e5250 con 0x5569676e24e0 2024-03-27T10:58:48.953 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.980+0000 7f7f337cf740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5569675fd630 con 0x5569676dc980 2024-03-27T10:58:48.953 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.980+0000 7f7f337cf740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5569676e53d0 con 0x5569676dd370 2024-03-27T10:58:48.953 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.980+0000 7f7f337cf740 10 client.0 _ll_register_callbacks cb 0x556967602c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:58:48.953 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.981+0000 7f7f29400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5569676dd370 0x5569676e7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:48.953 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.981+0000 7f7f29e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5569676dc980 0x5569676e6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:48.954 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.981+0000 7f7f29400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5569676dd370 0x5569676e7010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.97:56720/0 (socket says 172.21.15.97:56720) 2024-03-27T10:58:48.954 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.981+0000 7f7f29400640 1 -- 172.21.15.97:0/2590148075 learned_addr learned my addr 172.21.15.97:0/2590148075 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:58:48.954 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.981+0000 7f7f2a800640 1 --2- 172.21.15.97:0/2590148075 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5569676e24e0 0x5569676edfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:48.954 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.981+0000 7f7f29400640 1 -- 172.21.15.97:0/2590148075 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5569676dc980 msgr2=0x5569676e6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:48.954 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.981+0000 7f7f29400640 1 --2- 172.21.15.97:0/2590148075 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5569676dc980 0x5569676e6ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:48.954 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.981+0000 7f7f29400640 1 -- 172.21.15.97:0/2590148075 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5569676e24e0 msgr2=0x5569676edfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:48.954 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.981+0000 7f7f29400640 1 --2- 172.21.15.97:0/2590148075 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5569676e24e0 0x5569676edfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:48.954 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.981+0000 7f7f29400640 1 -- 172.21.15.97:0/2590148075 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569676095f0 con 0x5569676dd370 2024-03-27T10:58:48.954 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.982+0000 7f7f29400640 1 --2- 172.21.15.97:0/2590148075 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5569676dd370 0x5569676e7010 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7f1800bad0 tx=0x7f7f1800bf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3f0d221ff278db7 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:48.956 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.982+0000 7f7f0b400640 10 client.0 ms_handle_connect on v2:172.21.15.177:3300/0 2024-03-27T10:58:48.956 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.982+0000 7f7f0b400640 1 -- 172.21.15.97:0/2590148075 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f18003e20 con 0x5569676dd370 2024-03-27T10:58:48.956 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.982+0000 7f7f337cf740 1 -- 172.21.15.97:0/2590148075 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f1c004c50 con 0x5569676dd370 2024-03-27T10:58:48.956 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.982+0000 7f7f0b400640 1 -- 172.21.15.97:0/2590148075 <== mon.1 v2:172.21.15.177:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f7f18011440 con 0x5569676dd370 2024-03-27T10:58:48.956 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.983+0000 7f7f0b400640 1 -- 172.21.15.97:0/2590148075 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f7f18011640 con 0x5569676dd370 2024-03-27T10:58:48.956 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.983+0000 7f7f337cf740 10 client.25630 fetch_fsmap learned FSMap version 31 2024-03-27T10:58:48.956 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.983+0000 7f7f337cf740 1 -- 172.21.15.97:0/2590148075 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x556967356210 con 0x5569676dd370 2024-03-27T10:58:48.956 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.983+0000 7f7f0b400640 1 -- 172.21.15.97:0/2590148075 <== mon.1 v2:172.21.15.177:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7f1c004c50 con 0x5569676dd370 2024-03-27T10:58:48.956 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.984+0000 7f7f0b400640 1 -- 172.21.15.97:0/2590148075 <== mon.1 v2:172.21.15.177:3300/0 5 ==== fsmap(e 31) ==== 8370+0+0 (secure 0 0 0) 0x7f7f1801b8d0 con 0x5569676dd370 2024-03-27T10:58:48.956 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.984+0000 7f7f337cf740 10 client.25630 fetch_fsmap finished waiting for FSMap version 31 2024-03-27T10:58:48.956 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.984+0000 7f7f337cf740 10 client.25630 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T10:58:48.957 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.984+0000 7f7f337cf740 20 client.25630 populate_metadata read hostname 'smithi097' 2024-03-27T10:58:48.957 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.984+0000 7f7f337cf740 1 --2- 172.21.15.97:0/2590148075 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5569676f2a00 0x5569676f4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:48.957 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.984+0000 7f7f337cf740 4 client.25630 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T10:58:48.957 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.984+0000 7f7f337cf740 1 -- 172.21.15.97:0/2590148075 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55696764f250 con 0x5569676f2a00 2024-03-27T10:58:48.957 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.985+0000 7f7f29e00640 1 --2- 172.21.15.97:0/2590148075 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5569676f2a00 0x5569676f4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:48.957 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.985+0000 7f7f29e00640 1 --2- 172.21.15.97:0/2590148075 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5569676f2a00 0x5569676f4e80 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2b1cd89a3969153 server_cookie=ae405c874df6d599 in_seq=0 out_seq=0 2024-03-27T10:58:48.957 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.985+0000 7f7f0b400640 10 client.25630 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T10:58:48.958 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.987+0000 7f7f0b400640 1 -- 172.21.15.97:0/2590148075 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55696764f250 con 0x5569676f2a00 2024-03-27T10:58:48.958 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.987+0000 7f7f0b400640 10 client.25630 handle_command_reply: tid=0 2024-03-27T10:58:48.958 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.987+0000 7f7f337cf740 1 client.25630 shutdown 2024-03-27T10:58:48.959 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.987+0000 7f7f337cf740 1 -- 172.21.15.97:0/2590148075 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5569676dd370 msgr2=0x5569676e7010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:48.959 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.987+0000 7f7f337cf740 1 --2- 172.21.15.97:0/2590148075 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5569676dd370 0x5569676e7010 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7f1800bad0 tx=0x7f7f1800bf90 comp rx=0 tx=0).stop 2024-03-27T10:58:48.959 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.987+0000 7f7f337cf740 1 -- 172.21.15.97:0/2590148075 shutdown_connections 2024-03-27T10:58:48.959 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.987+0000 7f7f337cf740 1 --2- 172.21.15.97:0/2590148075 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5569676f2a00 0x5569676f4e80 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:48.959 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.987+0000 7f7f337cf740 1 --2- 172.21.15.97:0/2590148075 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5569676e24e0 0x5569676edfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:48.959 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.987+0000 7f7f337cf740 1 --2- 172.21.15.97:0/2590148075 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5569676dd370 0x5569676e7010 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:48.960 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.987+0000 7f7f337cf740 1 --2- 172.21.15.97:0/2590148075 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5569676dc980 0x5569676e6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:48.960 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.987+0000 7f7f337cf740 1 -- 172.21.15.97:0/2590148075 >> 172.21.15.97:0/2590148075 conn(0x5569676d7ef0 msgr2=0x5569676d70f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:48.960 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.988+0000 7f7f0b400640 0 client.25630 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T10:58:48.960 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.988+0000 7f7f337cf740 1 -- 172.21.15.97:0/2590148075 shutdown_connections 2024-03-27T10:58:48.960 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.988+0000 7f7f337cf740 1 -- 172.21.15.97:0/2590148075 wait complete. 2024-03-27T10:58:48.960 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:48.988+0000 7f7f337cf740 20 client.25630 trim_cache size 0 max 16384 2024-03-27T10:58:48.983 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:49.011+0000 7f7f337cf740 1 Processor -- start 2024-03-27T10:58:48.983 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:49.012+0000 7f7f337cf740 1 -- start start 2024-03-27T10:58:48.984 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:49.012+0000 7f7f337cf740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5569676dc980 0x5569676de770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:48.984 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:49.012+0000 7f7f337cf740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5569676dd370 0x5569676decb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:48.984 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:49.012+0000 7f7f337cf740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5569676e24e0 0x5569676edfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:48.984 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:49.012+0000 7f7f337cf740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7f2c1af580 con 0x5569676dc980 2024-03-27T10:58:48.984 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:49.012+0000 7f7f337cf740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f7f2c1b1180 con 0x5569676e24e0 2024-03-27T10:58:48.984 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:49.012+0000 7f7f337cf740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f7f2c1afc50 con 0x5569676dd370 2024-03-27T10:58:48.984 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:49.012+0000 7f7f29e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5569676dc980 0x5569676de770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:48.984 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:49.012+0000 7f7f29e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5569676dc980 0x5569676de770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49282/0 (socket says 172.21.15.97:49282) 2024-03-27T10:58:48.985 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:49.013+0000 7f7f29e00640 1 -- 172.21.15.97:0/401937786 learned_addr learned my addr 172.21.15.97:0/401937786 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:58:48.985 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:49.013+0000 7f7f29e00640 1 -- 172.21.15.97:0/401937786 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5569676dd370 msgr2=0x5569676decb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:48.986 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:49.013+0000 7f7f29e00640 1 --2- 172.21.15.97:0/401937786 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5569676dd370 0x5569676decb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:48.986 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:49.013+0000 7f7f29e00640 1 -- 172.21.15.97:0/401937786 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5569676e24e0 msgr2=0x5569676edfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:48.986 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:49.013+0000 7f7f29e00640 1 --2- 172.21.15.97:0/401937786 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5569676e24e0 0x5569676edfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:48.986 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:49.013+0000 7f7f29e00640 1 -- 172.21.15.97:0/401937786 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556967422620 con 0x5569676dc980 2024-03-27T10:58:48.986 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:49.013+0000 7f7f29e00640 1 --2- 172.21.15.97:0/401937786 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5569676dc980 0x5569676de770 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c0ba9d0 tx=0x7f7f1c041040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee7241cfe4b83cd9 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:48.986 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:49.014+0000 7f7f16600640 1 -- 172.21.15.97:0/401937786 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f1c0bded0 con 0x5569676dc980 2024-03-27T10:58:48.986 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:49.014+0000 7f7f16600640 1 -- 172.21.15.97:0/401937786 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7f1c0d7070 con 0x5569676dc980 2024-03-27T10:58:48.986 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:49.014+0000 7f7f16600640 1 -- 172.21.15.97:0/401937786 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f1c0be0b0 con 0x5569676dc980 2024-03-27T10:58:48.987 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:49.014+0000 7f7f337cf740 1 -- 172.21.15.97:0/401937786 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5569676dc980 msgr2=0x5569676de770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:48.987 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:49.014+0000 7f7f337cf740 1 --2- 172.21.15.97:0/401937786 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5569676dc980 0x5569676de770 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c0ba9d0 tx=0x7f7f1c041040 comp rx=0 tx=0).stop 2024-03-27T10:58:48.987 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:49.015+0000 7f7f337cf740 1 -- 172.21.15.97:0/401937786 shutdown_connections 2024-03-27T10:58:48.987 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:49.015+0000 7f7f337cf740 1 --2- 172.21.15.97:0/401937786 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5569676e24e0 0x5569676edfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:48.987 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:49.015+0000 7f7f337cf740 1 --2- 172.21.15.97:0/401937786 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5569676dd370 0x5569676decb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:48.987 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:49.015+0000 7f7f337cf740 1 --2- 172.21.15.97:0/401937786 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5569676dc980 0x5569676de770 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:48.988 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:49.015+0000 7f7f337cf740 1 -- 172.21.15.97:0/401937786 >> 172.21.15.97:0/401937786 conn(0x5569676d7ef0 msgr2=0x5569676e6ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:48.988 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:49.015+0000 7f7f337cf740 1 -- 172.21.15.97:0/401937786 shutdown_connections 2024-03-27T10:58:48.988 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:49.015+0000 7f7f337cf740 1 -- 172.21.15.97:0/401937786 wait complete. 2024-03-27T10:58:48.988 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:49.015+0000 7f7f337cf740 1 -- start start 2024-03-27T10:58:48.988 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:49.016+0000 7f7f337cf740 1 Processor -- start 2024-03-27T10:58:48.989 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:49.016+0000 7f7f337cf740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5569676dc980 0x5569676eb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:48.989 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:49.016+0000 7f7f337cf740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5569676dd370 0x5569676eb6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:48.989 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:49.016+0000 7f7f337cf740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5569676e24e0 0x5569676ec700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:48.989 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:49.016+0000 7f7f337cf740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7f2c1b09f0 con 0x5569676dc980 2024-03-27T10:58:48.989 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:49.016+0000 7f7f337cf740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f7f2c1b0320 con 0x5569676e24e0 2024-03-27T10:58:48.989 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:49.016+0000 7f7f337cf740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f7f2c1ad560 con 0x5569676dd370 2024-03-27T10:58:48.989 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:49.016+0000 7f7f337cf740 10 client.0 _ll_register_callbacks cb 0x7f7f18004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:58:48.989 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:49.017+0000 7f7f2a800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5569676e24e0 0x5569676ec700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:48.989 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:49.017+0000 7f7f2a800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5569676e24e0 0x5569676ec700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:59944/0 (socket says 172.21.15.97:59944) 2024-03-27T10:58:48.989 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:49.017+0000 7f7f2a800640 1 -- 172.21.15.97:0/1678640027 learned_addr learned my addr 172.21.15.97:0/1678640027 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:58:48.989 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:49.017+0000 7f7f29400640 1 --2- 172.21.15.97:0/1678640027 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5569676dd370 0x5569676eb6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:48.989 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:49.017+0000 7f7f29e00640 1 --2- 172.21.15.97:0/1678640027 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5569676dc980 0x5569676eb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:48.990 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:49.018+0000 7f7f2a800640 1 -- 172.21.15.97:0/1678640027 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5569676dd370 msgr2=0x5569676eb6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:48.990 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:49.018+0000 7f7f2a800640 1 --2- 172.21.15.97:0/1678640027 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5569676dd370 0x5569676eb6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:48.991 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:49.018+0000 7f7f2a800640 1 -- 172.21.15.97:0/1678640027 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5569676dc980 msgr2=0x5569676eb1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:48.991 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:49.018+0000 7f7f2a800640 1 --2- 172.21.15.97:0/1678640027 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5569676dc980 0x5569676eb1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:48.991 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:49.018+0000 7f7f2a800640 1 -- 172.21.15.97:0/1678640027 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569676eccf0 con 0x5569676e24e0 2024-03-27T10:58:48.991 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:49.018+0000 7f7f29e00640 1 --2- 172.21.15.97:0/1678640027 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5569676dc980 0x5569676eb1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T10:58:48.991 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:49.018+0000 7f7f2a800640 1 --2- 172.21.15.97:0/1678640027 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5569676e24e0 0x5569676ec700 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7f24007e40 tx=0x7f7f24009650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e1c7cc328153016 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:48.991 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:49.018+0000 7f7f0b400640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-03-27T10:58:48.991 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:49.019+0000 7f7f0b400640 1 -- 172.21.15.97:0/1678640027 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f24026070 con 0x5569676e24e0 2024-03-27T10:58:48.991 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:49.019+0000 7f7f337cf740 1 -- 172.21.15.97:0/1678640027 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f1c0d7070 con 0x5569676e24e0 2024-03-27T10:58:48.992 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:49.021+0000 7f7f0b400640 1 -- 172.21.15.97:0/1678640027 <== mon.2 v2:172.21.15.114:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f7f240225f0 con 0x5569676e24e0 2024-03-27T10:58:48.994 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:49.021+0000 7f7f0b400640 1 -- 172.21.15.97:0/1678640027 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f7f24030050 con 0x5569676e24e0 2024-03-27T10:58:48.994 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:49.022+0000 7f7f337cf740 10 client.35882 fetch_fsmap learned FSMap version 31 2024-03-27T10:58:48.994 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:49.022+0000 7f7f337cf740 1 -- 172.21.15.97:0/1678640027 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5569676ee0c0 con 0x5569676e24e0 2024-03-27T10:58:48.994 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:49.022+0000 7f7f0b400640 1 -- 172.21.15.97:0/1678640027 <== mon.2 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7f1c0d7070 con 0x5569676e24e0 2024-03-27T10:58:48.995 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:49.023+0000 7f7f0b400640 1 -- 172.21.15.97:0/1678640027 <== mon.2 v2:172.21.15.114:3300/0 5 ==== fsmap(e 31) ==== 8370+0+0 (secure 0 0 0) 0x7f7f24033020 con 0x5569676e24e0 2024-03-27T10:58:48.995 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:49.023+0000 7f7f337cf740 10 client.35882 fetch_fsmap finished waiting for FSMap version 31 2024-03-27T10:58:48.995 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:49.023+0000 7f7f337cf740 10 client.35882 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T10:58:48.995 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:49.023+0000 7f7f337cf740 20 client.35882 populate_metadata read hostname 'smithi097' 2024-03-27T10:58:48.995 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:49.023+0000 7f7f337cf740 1 --2- 172.21.15.97:0/1678640027 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5569677593d0 0x55696775b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:48.995 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:49.023+0000 7f7f337cf740 4 client.35882 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-27T10:58:48.996 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:49.024+0000 7f7f337cf740 1 -- 172.21.15.97:0/1678640027 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5569676495a0 con 0x5569677593d0 2024-03-27T10:58:48.996 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:49.024+0000 7f7f29e00640 1 --2- 172.21.15.97:0/1678640027 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5569677593d0 0x55696775b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:48.996 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:49.025+0000 7f7f29e00640 1 --2- 172.21.15.97:0/1678640027 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5569677593d0 0x55696775b7f0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e8a011a12765925 server_cookie=2e1efcfa3069c0df in_seq=0 out_seq=0 2024-03-27T10:58:48.996 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:49.025+0000 7f7f0b400640 10 client.35882 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T10:58:49.014 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:49.043+0000 7f7f0b400640 1 -- 172.21.15.97:0/1678640027 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5569676495a0 con 0x5569677593d0 2024-03-27T10:58:49.014 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:49.043+0000 7f7f0b400640 10 client.35882 handle_command_reply: tid=0 2024-03-27T10:58:49.015 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:49.043+0000 7f7f337cf740 1 client.35882 shutdown 2024-03-27T10:58:49.015 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:49.043+0000 7f7f337cf740 1 -- 172.21.15.97:0/1678640027 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5569676e24e0 msgr2=0x5569676ec700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:49.015 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:49.043+0000 7f7f337cf740 1 --2- 172.21.15.97:0/1678640027 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5569676e24e0 0x5569676ec700 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7f24007e40 tx=0x7f7f24009650 comp rx=0 tx=0).stop 2024-03-27T10:58:49.015 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:49.043+0000 7f7f337cf740 1 -- 172.21.15.97:0/1678640027 shutdown_connections 2024-03-27T10:58:49.015 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:49.043+0000 7f7f337cf740 1 --2- 172.21.15.97:0/1678640027 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5569677593d0 0x55696775b7f0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:49.016 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:49.043+0000 7f7f337cf740 1 --2- 172.21.15.97:0/1678640027 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5569676e24e0 0x5569676ec700 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:49.016 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:49.043+0000 7f7f337cf740 1 --2- 172.21.15.97:0/1678640027 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5569676dd370 0x5569676eb6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:49.016 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:49.043+0000 7f7f337cf740 1 --2- 172.21.15.97:0/1678640027 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5569676dc980 0x5569676eb1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:49.016 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:49.043+0000 7f7f337cf740 1 -- 172.21.15.97:0/1678640027 >> 172.21.15.97:0/1678640027 conn(0x5569676d7ef0 msgr2=0x5569676e6a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:49.016 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:49.044+0000 7f7f0b400640 0 client.35882 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T10:58:49.016 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:49.044+0000 7f7f337cf740 1 -- 172.21.15.97:0/1678640027 shutdown_connections 2024-03-27T10:58:49.016 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:49.044+0000 7f7f337cf740 1 -- 172.21.15.97:0/1678640027 wait complete. 2024-03-27T10:58:49.016 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:49.044+0000 7f7f337cf740 20 client.35882 trim_cache size 0 max 16384 2024-03-27T10:58:49.017 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-27T10:58:49.017 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-27T10:58:49.017 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-27T10:58:49.017 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-27T10:58:49.018 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:49.045+0000 7f7f30a00640 1 -- 172.21.15.97:0/1475714463 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f7f100a36b0 msgr2=0x7f7f100a5b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:49.018 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:49.045+0000 7f7f30a00640 1 --2- 172.21.15.97:0/1475714463 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f7f100a36b0 0x7f7f100a5b70 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x5569676ddbd0 tx=0x7f7f24004b50 comp rx=0 tx=0).stop 2024-03-27T10:58:49.018 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:49.045+0000 7f7f30a00640 1 -- 172.21.15.97:0/1475714463 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7f2c103da0 msgr2=0x7f7f2c19e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:49.018 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:49.045+0000 7f7f30a00640 1 --2- 172.21.15.97:0/1475714463 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7f2c103da0 0x7f7f2c19e3a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c005ab0 tx=0x7f7f1c0042f0 comp rx=0 tx=0).stop 2024-03-27T10:58:49.018 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:49.046+0000 7f7f30a00640 1 -- 172.21.15.97:0/1475714463 shutdown_connections 2024-03-27T10:58:49.018 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:49.046+0000 7f7f30a00640 1 --2- 172.21.15.97:0/1475714463 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f2c105950 0x7f7f2c1a5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:49.018 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:49.046+0000 7f7f30a00640 1 --2- 172.21.15.97:0/1475714463 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f7f100a36b0 0x7f7f100a5b70 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:49.018 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:49.046+0000 7f7f30a00640 1 --2- 172.21.15.97:0/1475714463 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7f2c104f90 0x7f7f2c19e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:49.018 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:49.046+0000 7f7f30a00640 1 --2- 172.21.15.97:0/1475714463 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7f2c103da0 0x7f7f2c19e3a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:49.018 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:49.046+0000 7f7f30a00640 1 -- 172.21.15.97:0/1475714463 >> 172.21.15.97:0/1475714463 conn(0x7f7f2c0ff310 msgr2=0x7f7f2c077fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:49.018 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:49.046+0000 7f7f30a00640 1 -- 172.21.15.97:0/1475714463 shutdown_connections 2024-03-27T10:58:49.019 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:58:49.046+0000 7f7f30a00640 1 -- 172.21.15.97:0/1475714463 wait complete. 2024-03-27T10:58:49.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:58:49 smithi097 ceph-mon[31859]: pgmap v506: 97 pgs: 97 active+clean; 4.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 128 MiB/s wr, 75 op/s 2024-03-27T10:58:49.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:58:49 smithi177 ceph-mon[35294]: pgmap v506: 97 pgs: 97 active+clean; 4.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 128 MiB/s wr, 75 op/s 2024-03-27T10:58:49.766 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:58:49 smithi114 ceph-mon[35338]: pgmap v506: 97 pgs: 97 active+clean; 4.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 128 MiB/s wr, 75 op/s 2024-03-27T10:58:51.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:58:51 smithi097 ceph-mon[31859]: pgmap v507: 97 pgs: 97 active+clean; 5.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 137 MiB/s wr, 83 op/s 2024-03-27T10:58:51.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:58:51 smithi177 ceph-mon[35294]: pgmap v507: 97 pgs: 97 active+clean; 5.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 137 MiB/s wr, 83 op/s 2024-03-27T10:58:51.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:58:51 smithi114 ceph-mon[35338]: pgmap v507: 97 pgs: 97 active+clean; 5.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 137 MiB/s wr, 83 op/s 2024-03-27T10:58:52.409 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-27T10:58:52.410 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-27T10:58:52.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.698+0000 7f970f200640 1 -- 172.21.15.97:0/1004518719 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9710103c50 msgr2=0x7f9710104070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:52.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.698+0000 7f970f200640 1 --2- 172.21.15.97:0/1004518719 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9710103c50 0x7f9710104070 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f96fc009900 tx=0x7f96fc02bc30 comp rx=0 tx=0).stop 2024-03-27T10:58:52.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.698+0000 7f970f200640 1 -- 172.21.15.97:0/1004518719 shutdown_connections 2024-03-27T10:58:52.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.698+0000 7f970f200640 1 --2- 172.21.15.97:0/1004518719 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9710105800 0x7f971010c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:52.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.698+0000 7f970f200640 1 --2- 172.21.15.97:0/1004518719 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9710104e40 0x7f97101052c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:52.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.698+0000 7f970f200640 1 --2- 172.21.15.97:0/1004518719 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9710103c50 0x7f9710104070 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:52.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.698+0000 7f970f200640 1 -- 172.21.15.97:0/1004518719 >> 172.21.15.97:0/1004518719 conn(0x7f97100ff1c0 msgr2=0x7f9710101600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:52.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.699+0000 7f970f200640 1 -- 172.21.15.97:0/1004518719 shutdown_connections 2024-03-27T10:58:52.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.699+0000 7f970f200640 1 -- 172.21.15.97:0/1004518719 wait complete. 2024-03-27T10:58:52.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.700+0000 7f970f200640 1 Processor -- start 2024-03-27T10:58:52.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.701+0000 7f970f200640 1 -- start start 2024-03-27T10:58:52.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.701+0000 7f970f200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9710103c50 0x7f9710199e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:52.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.701+0000 7f970f200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9710104e40 0x7f971019a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:52.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.701+0000 7f970f200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9710105800 0x7f97101a1420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:52.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.701+0000 7f970f200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f971010f7d0 con 0x7f9710103c50 2024-03-27T10:58:52.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.701+0000 7f970f200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f971010f650 con 0x7f9710105800 2024-03-27T10:58:52.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.701+0000 7f970f200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f971010f950 con 0x7f9710104e40 2024-03-27T10:58:52.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.701+0000 7f970de00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9710103c50 0x7f9710199e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:52.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.701+0000 7f970de00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9710103c50 0x7f9710199e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49322/0 (socket says 172.21.15.97:49322) 2024-03-27T10:58:52.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.701+0000 7f970de00640 1 -- 172.21.15.97:0/3886357855 learned_addr learned my addr 172.21.15.97:0/3886357855 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:58:52.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.702+0000 7f970e800640 1 --2- 172.21.15.97:0/3886357855 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9710105800 0x7f97101a1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:52.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.702+0000 7f970d400640 1 --2- 172.21.15.97:0/3886357855 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9710104e40 0x7f971019a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:52.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.702+0000 7f970de00640 1 -- 172.21.15.97:0/3886357855 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9710104e40 msgr2=0x7f971019a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:52.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.702+0000 7f970de00640 1 --2- 172.21.15.97:0/3886357855 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9710104e40 0x7f971019a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:52.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.702+0000 7f970de00640 1 -- 172.21.15.97:0/3886357855 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9710105800 msgr2=0x7f97101a1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:52.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.702+0000 7f970de00640 1 --2- 172.21.15.97:0/3886357855 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9710105800 0x7f97101a1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:52.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.702+0000 7f970de00640 1 -- 172.21.15.97:0/3886357855 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97101a1a90 con 0x7f9710103c50 2024-03-27T10:58:52.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.702+0000 7f970de00640 1 --2- 172.21.15.97:0/3886357855 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9710103c50 0x7f9710199e70 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f96fc000c00 tx=0x7f96fc004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:52.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.703+0000 7f9706a00640 1 -- 172.21.15.97:0/3886357855 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96fc043020 con 0x7f9710103c50 2024-03-27T10:58:52.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.703+0000 7f9706a00640 1 -- 172.21.15.97:0/3886357855 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f96fc0043c0 con 0x7f9710103c50 2024-03-27T10:58:52.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.703+0000 7f9706a00640 1 -- 172.21.15.97:0/3886357855 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96fc003b80 con 0x7f9710103c50 2024-03-27T10:58:52.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.704+0000 7f970f200640 1 -- 172.21.15.97:0/3886357855 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97101a1d20 con 0x7f9710103c50 2024-03-27T10:58:52.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.704+0000 7f970f200640 1 -- 172.21.15.97:0/3886357855 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97101a21c0 con 0x7f9710103c50 2024-03-27T10:58:52.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.707+0000 7f9717db6740 1 Processor -- start 2024-03-27T10:58:52.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.708+0000 7f9717db6740 1 -- start start 2024-03-27T10:58:52.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.708+0000 7f9717db6740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b1c4a21e80 0x55b1c4a222a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:52.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.708+0000 7f9717db6740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b1c4a22870 0x55b1c4a27370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:52.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.708+0000 7f9717db6740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b1c4a279e0 0x55b1c4a29e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:52.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.708+0000 7f9717db6740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55b1c497f6b0 con 0x55b1c4a279e0 2024-03-27T10:58:52.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.708+0000 7f9717db6740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55b1c4927750 con 0x55b1c4a21e80 2024-03-27T10:58:52.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.708+0000 7f9717db6740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55b1c497b660 con 0x55b1c4a22870 2024-03-27T10:58:52.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.708+0000 7f9706a00640 1 -- 172.21.15.97:0/3886357855 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f96fc046020 con 0x7f9710103c50 2024-03-27T10:58:52.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.709+0000 7f970de00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b1c4a22870 0x55b1c4a27370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:52.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.709+0000 7f970de00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b1c4a22870 0x55b1c4a27370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.97:56764/0 (socket says 172.21.15.97:56764) 2024-03-27T10:58:52.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.709+0000 7f970de00640 1 -- 172.21.15.97:0/431968199 learned_addr learned my addr 172.21.15.97:0/431968199 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:58:52.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.709+0000 7f970e800640 1 --2- 172.21.15.97:0/431968199 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b1c4a21e80 0x55b1c4a222a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:52.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.709+0000 7f970d400640 1 --2- 172.21.15.97:0/431968199 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b1c4a279e0 0x55b1c4a29e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:52.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.709+0000 7f970de00640 1 -- 172.21.15.97:0/431968199 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b1c4a21e80 msgr2=0x55b1c4a222a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:52.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.709+0000 7f970de00640 1 --2- 172.21.15.97:0/431968199 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b1c4a21e80 0x55b1c4a222a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:52.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.709+0000 7f970de00640 1 -- 172.21.15.97:0/431968199 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b1c4a279e0 msgr2=0x55b1c4a29e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:52.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.710+0000 7f970de00640 1 --2- 172.21.15.97:0/431968199 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b1c4a279e0 0x55b1c4a29e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:52.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.710+0000 7f970de00640 1 -- 172.21.15.97:0/431968199 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1c475d7d0 con 0x55b1c4a22870 2024-03-27T10:58:52.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.710+0000 7f970d400640 1 --2- 172.21.15.97:0/431968199 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b1c4a279e0 0x55b1c4a29e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T10:58:52.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.710+0000 7f9706a00640 1 --2- 172.21.15.97:0/3886357855 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f96f40a3760 0x7f96f40a5c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:52.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.711+0000 7f970de00640 1 --2- 172.21.15.97:0/431968199 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b1c4a22870 0x55b1c4a27370 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f96fc0f7a90 tx=0x7f96fc0f7b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=298ccc238f29daae server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:52.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.711+0000 7f9706a00640 1 -- 172.21.15.97:0/3886357855 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f96fc0f1a00 con 0x7f9710103c50 2024-03-27T10:58:52.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.711+0000 7f970e800640 1 --2- 172.21.15.97:0/3886357855 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f96f40a3760 0x7f96f40a5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:52.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.712+0000 7f96fb400640 1 -- 172.21.15.97:0/431968199 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96fc0ba090 con 0x55b1c4a22870 2024-03-27T10:58:52.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.712+0000 7f96fb400640 1 -- 172.21.15.97:0/431968199 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f96fc0dce50 con 0x55b1c4a22870 2024-03-27T10:58:52.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.712+0000 7f96fb400640 1 -- 172.21.15.97:0/431968199 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96fc0bbfc0 con 0x55b1c4a22870 2024-03-27T10:58:52.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.713+0000 7f9717db6740 1 -- 172.21.15.97:0/431968199 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b1c4a22870 msgr2=0x55b1c4a27370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:52.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.713+0000 7f9717db6740 1 --2- 172.21.15.97:0/431968199 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b1c4a22870 0x55b1c4a27370 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f96fc0f7a90 tx=0x7f96fc0f7b80 comp rx=0 tx=0).stop 2024-03-27T10:58:52.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.714+0000 7f970e800640 1 --2- 172.21.15.97:0/3886357855 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f96f40a3760 0x7f96f40a5c20 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x55b1c4a230d0 tx=0x7f9708009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:52.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.714+0000 7f9717db6740 1 -- 172.21.15.97:0/431968199 shutdown_connections 2024-03-27T10:58:52.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.714+0000 7f9717db6740 1 --2- 172.21.15.97:0/431968199 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b1c4a279e0 0x55b1c4a29e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:52.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.714+0000 7f9717db6740 1 --2- 172.21.15.97:0/431968199 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b1c4a22870 0x55b1c4a27370 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:52.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.714+0000 7f9717db6740 1 --2- 172.21.15.97:0/431968199 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b1c4a21e80 0x55b1c4a222a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:52.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.714+0000 7f9717db6740 1 -- 172.21.15.97:0/431968199 >> 172.21.15.97:0/431968199 conn(0x55b1c4a14be0 msgr2=0x55b1c4a14fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:52.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.714+0000 7f9717db6740 1 -- 172.21.15.97:0/431968199 shutdown_connections 2024-03-27T10:58:52.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.714+0000 7f9717db6740 1 -- 172.21.15.97:0/431968199 wait complete. 2024-03-27T10:58:52.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.715+0000 7f9717db6740 1 -- start start 2024-03-27T10:58:52.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.715+0000 7f9717db6740 1 Processor -- start 2024-03-27T10:58:52.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.716+0000 7f9717db6740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b1c4a21e80 0x55b1c4a2bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:52.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.716+0000 7f9717db6740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b1c4a22870 0x55b1c4a2c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:52.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.716+0000 7f9717db6740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b1c4a279e0 0x55b1c4a33560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:52.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.716+0000 7f9717db6740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55b1c497b660 con 0x55b1c4a22870 2024-03-27T10:58:52.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.716+0000 7f9717db6740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55b1c493aa80 con 0x55b1c4a21e80 2024-03-27T10:58:52.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.716+0000 7f9717db6740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55b1c4a2a750 con 0x55b1c4a279e0 2024-03-27T10:58:52.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.716+0000 7f9717db6740 10 client.0 _ll_register_callbacks cb 0x55b1c493a470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:58:52.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.716+0000 7f970de00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b1c4a21e80 0x55b1c4a2bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:52.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.716+0000 7f970de00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b1c4a21e80 0x55b1c4a2bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:59986/0 (socket says 172.21.15.97:59986) 2024-03-27T10:58:52.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.716+0000 7f970de00640 1 -- 172.21.15.97:0/1229217139 learned_addr learned my addr 172.21.15.97:0/1229217139 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:58:52.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.717+0000 7f970e800640 1 --2- 172.21.15.97:0/1229217139 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b1c4a279e0 0x55b1c4a33560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:52.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.717+0000 7f970d400640 1 --2- 172.21.15.97:0/1229217139 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b1c4a22870 0x55b1c4a2c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:52.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.717+0000 7f970de00640 1 -- 172.21.15.97:0/1229217139 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b1c4a279e0 msgr2=0x55b1c4a33560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:52.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.717+0000 7f970de00640 1 --2- 172.21.15.97:0/1229217139 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b1c4a279e0 0x55b1c4a33560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:52.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.717+0000 7f970de00640 1 -- 172.21.15.97:0/1229217139 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b1c4a22870 msgr2=0x55b1c4a2c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:52.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.717+0000 7f970de00640 1 --2- 172.21.15.97:0/1229217139 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b1c4a22870 0x55b1c4a2c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:52.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.717+0000 7f970de00640 1 -- 172.21.15.97:0/1229217139 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1c46933f0 con 0x55b1c4a21e80 2024-03-27T10:58:52.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.717+0000 7f970d400640 1 --2- 172.21.15.97:0/1229217139 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b1c4a22870 0x55b1c4a2c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:58:52.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.718+0000 7f970de00640 1 --2- 172.21.15.97:0/1229217139 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b1c4a21e80 0x55b1c4a2bfe0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f96fc0f7a60 tx=0x7f96fc005b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28bb47021b7939a8 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:52.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.718+0000 7f96efe00640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-03-27T10:58:52.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.718+0000 7f96efe00640 1 -- 172.21.15.97:0/1229217139 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96fc0bd7e0 con 0x55b1c4a21e80 2024-03-27T10:58:52.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.718+0000 7f96efe00640 1 -- 172.21.15.97:0/1229217139 <== mon.2 v2:172.21.15.114:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f96fc003d20 con 0x55b1c4a21e80 2024-03-27T10:58:52.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.718+0000 7f9717db6740 1 -- 172.21.15.97:0/1229217139 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96fc0dce50 con 0x55b1c4a21e80 2024-03-27T10:58:52.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.719+0000 7f96efe00640 1 -- 172.21.15.97:0/1229217139 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f96fc0bd9c0 con 0x55b1c4a21e80 2024-03-27T10:58:52.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.719+0000 7f9717db6740 10 client.35894 fetch_fsmap learned FSMap version 31 2024-03-27T10:58:52.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.719+0000 7f9717db6740 1 -- 172.21.15.97:0/1229217139 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b1c46530f0 con 0x55b1c4a21e80 2024-03-27T10:58:52.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.719+0000 7f96efe00640 1 -- 172.21.15.97:0/1229217139 <== mon.2 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f96fc0dce50 con 0x55b1c4a21e80 2024-03-27T10:58:52.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.719+0000 7f96efe00640 1 -- 172.21.15.97:0/1229217139 <== mon.2 v2:172.21.15.114:3300/0 5 ==== fsmap(e 31) ==== 8370+0+0 (secure 0 0 0) 0x7f96fc005500 con 0x55b1c4a21e80 2024-03-27T10:58:52.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.720+0000 7f9717db6740 10 client.35894 fetch_fsmap finished waiting for FSMap version 31 2024-03-27T10:58:52.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.720+0000 7f9717db6740 10 client.35894 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-27T10:58:52.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.720+0000 7f9717db6740 20 client.35894 populate_metadata read hostname 'smithi097' 2024-03-27T10:58:52.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.720+0000 7f9717db6740 1 --2- 172.21.15.97:0/1229217139 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55b1c4a37b60 0x55b1c4a39fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:52.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.720+0000 7f9717db6740 4 client.35894 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T10:58:52.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.720+0000 7f9717db6740 1 -- 172.21.15.97:0/1229217139 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b1c498cbf0 con 0x55b1c4a37b60 2024-03-27T10:58:52.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.720+0000 7f970d400640 1 --2- 172.21.15.97:0/1229217139 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55b1c4a37b60 0x55b1c4a39fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:52.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.721+0000 7f970d400640 1 --2- 172.21.15.97:0/1229217139 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55b1c4a37b60 0x55b1c4a39fe0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21b7a33a379c82a6 server_cookie=84838110991113bd in_seq=0 out_seq=0 2024-03-27T10:58:52.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.722+0000 7f96efe00640 10 client.35894 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T10:58:52.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.723+0000 7f96efe00640 1 -- 172.21.15.97:0/1229217139 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55b1c498cbf0 con 0x55b1c4a37b60 2024-03-27T10:58:52.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.723+0000 7f96efe00640 10 client.35894 handle_command_reply: tid=0 2024-03-27T10:58:52.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.723+0000 7f9717db6740 1 client.35894 shutdown 2024-03-27T10:58:52.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.723+0000 7f9717db6740 1 -- 172.21.15.97:0/1229217139 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b1c4a21e80 msgr2=0x55b1c4a2bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:52.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.723+0000 7f9717db6740 1 --2- 172.21.15.97:0/1229217139 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b1c4a21e80 0x55b1c4a2bfe0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f96fc0f7a60 tx=0x7f96fc005b50 comp rx=0 tx=0).stop 2024-03-27T10:58:52.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.723+0000 7f9717db6740 1 -- 172.21.15.97:0/1229217139 shutdown_connections 2024-03-27T10:58:52.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.723+0000 7f9717db6740 1 --2- 172.21.15.97:0/1229217139 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55b1c4a37b60 0x55b1c4a39fe0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:52.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.723+0000 7f9717db6740 1 --2- 172.21.15.97:0/1229217139 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b1c4a279e0 0x55b1c4a33560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:52.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.723+0000 7f9717db6740 1 --2- 172.21.15.97:0/1229217139 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b1c4a22870 0x55b1c4a2c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:52.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.723+0000 7f9717db6740 1 --2- 172.21.15.97:0/1229217139 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b1c4a21e80 0x55b1c4a2bfe0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:52.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.723+0000 7f9717db6740 1 -- 172.21.15.97:0/1229217139 >> 172.21.15.97:0/1229217139 conn(0x55b1c4a14be0 msgr2=0x55b1c4a13e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:52.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.723+0000 7f96efe00640 0 client.35894 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T10:58:52.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.724+0000 7f9717db6740 1 -- 172.21.15.97:0/1229217139 shutdown_connections 2024-03-27T10:58:52.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.724+0000 7f9717db6740 1 -- 172.21.15.97:0/1229217139 wait complete. 2024-03-27T10:58:52.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.724+0000 7f9717db6740 20 client.35894 trim_cache size 0 max 16384 2024-03-27T10:58:52.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.753+0000 7f9717db6740 1 Processor -- start 2024-03-27T10:58:52.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.753+0000 7f9717db6740 1 -- start start 2024-03-27T10:58:52.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.754+0000 7f9717db6740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b1c4a21e80 0x55b1c4a2c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:52.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.754+0000 7f9717db6740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b1c4a22870 0x55b1c4a334f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:52.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.754+0000 7f9717db6740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b1c4a279e0 0x55b1c4a135b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:52.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.754+0000 7f9717db6740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f97101a6c70 con 0x55b1c4a21e80 2024-03-27T10:58:52.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.754+0000 7f9717db6740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f97101a8870 con 0x55b1c4a279e0 2024-03-27T10:58:52.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.754+0000 7f9717db6740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f97101a7340 con 0x55b1c4a22870 2024-03-27T10:58:52.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.754+0000 7f970de00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b1c4a21e80 0x55b1c4a2c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:52.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.754+0000 7f970de00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b1c4a21e80 0x55b1c4a2c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49344/0 (socket says 172.21.15.97:49344) 2024-03-27T10:58:52.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.754+0000 7f970de00640 1 -- 172.21.15.97:0/3449173607 learned_addr learned my addr 172.21.15.97:0/3449173607 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:58:52.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.754+0000 7f970d400640 1 --2- 172.21.15.97:0/3449173607 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b1c4a22870 0x55b1c4a334f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:52.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.754+0000 7f970e800640 1 --2- 172.21.15.97:0/3449173607 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b1c4a279e0 0x55b1c4a135b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:52.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.755+0000 7f970de00640 1 -- 172.21.15.97:0/3449173607 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b1c4a22870 msgr2=0x55b1c4a334f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:52.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.755+0000 7f970de00640 1 --2- 172.21.15.97:0/3449173607 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b1c4a22870 0x55b1c4a334f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:52.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.755+0000 7f970de00640 1 -- 172.21.15.97:0/3449173607 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b1c4a279e0 msgr2=0x55b1c4a135b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:52.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.755+0000 7f970de00640 1 --2- 172.21.15.97:0/3449173607 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b1c4a279e0 0x55b1c4a135b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:52.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.755+0000 7f970de00640 1 -- 172.21.15.97:0/3449173607 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1c46530f0 con 0x55b1c4a21e80 2024-03-27T10:58:52.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.755+0000 7f970de00640 1 --2- 172.21.15.97:0/3449173607 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b1c4a21e80 0x55b1c4a2c260 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f96fc02c240 tx=0x7f96fc0bc390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13c52e461dad882a server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:52.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.755+0000 7f96faa00640 1 -- 172.21.15.97:0/3449173607 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96fc0bdc50 con 0x55b1c4a21e80 2024-03-27T10:58:52.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.755+0000 7f96faa00640 1 -- 172.21.15.97:0/3449173607 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f96fc0dce50 con 0x55b1c4a21e80 2024-03-27T10:58:52.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.756+0000 7f96faa00640 1 -- 172.21.15.97:0/3449173607 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96fc0de2d0 con 0x55b1c4a21e80 2024-03-27T10:58:52.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.756+0000 7f9717db6740 1 -- 172.21.15.97:0/3449173607 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b1c4a21e80 msgr2=0x55b1c4a2c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:52.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.757+0000 7f9717db6740 1 --2- 172.21.15.97:0/3449173607 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b1c4a21e80 0x55b1c4a2c260 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f96fc02c240 tx=0x7f96fc0bc390 comp rx=0 tx=0).stop 2024-03-27T10:58:52.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.757+0000 7f9717db6740 1 -- 172.21.15.97:0/3449173607 shutdown_connections 2024-03-27T10:58:52.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.757+0000 7f9717db6740 1 --2- 172.21.15.97:0/3449173607 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b1c4a279e0 0x55b1c4a135b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:52.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.757+0000 7f9717db6740 1 --2- 172.21.15.97:0/3449173607 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b1c4a22870 0x55b1c4a334f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:52.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.757+0000 7f9717db6740 1 --2- 172.21.15.97:0/3449173607 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b1c4a21e80 0x55b1c4a2c260 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:52.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.757+0000 7f9717db6740 1 -- 172.21.15.97:0/3449173607 >> 172.21.15.97:0/3449173607 conn(0x55b1c4a14be0 msgr2=0x55b1c4a186d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:52.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.757+0000 7f9717db6740 1 -- 172.21.15.97:0/3449173607 shutdown_connections 2024-03-27T10:58:52.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.757+0000 7f9717db6740 1 -- 172.21.15.97:0/3449173607 wait complete. 2024-03-27T10:58:52.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.757+0000 7f9717db6740 1 -- start start 2024-03-27T10:58:52.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.758+0000 7f9717db6740 1 Processor -- start 2024-03-27T10:58:52.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.758+0000 7f9717db6740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b1c4a21e80 0x55b1c4a306c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:52.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.758+0000 7f9717db6740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b1c4a22870 0x55b1c4a30c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:52.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.758+0000 7f9717db6740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b1c4a279e0 0x55b1c4a31ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:52.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.758+0000 7f9717db6740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f97101a80e0 con 0x55b1c4a21e80 2024-03-27T10:58:52.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.758+0000 7f9717db6740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f97101a7a10 con 0x55b1c4a279e0 2024-03-27T10:58:52.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.758+0000 7f9717db6740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f97101a4c50 con 0x55b1c4a22870 2024-03-27T10:58:52.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.758+0000 7f9717db6740 10 client.0 _ll_register_callbacks cb 0x7f96fc0f78e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:58:52.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.758+0000 7f970de00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b1c4a21e80 0x55b1c4a306c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:52.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.758+0000 7f970de00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b1c4a21e80 0x55b1c4a306c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49348/0 (socket says 172.21.15.97:49348) 2024-03-27T10:58:52.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.758+0000 7f970de00640 1 -- 172.21.15.97:0/2914186749 learned_addr learned my addr 172.21.15.97:0/2914186749 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:58:52.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.759+0000 7f970d400640 1 --2- 172.21.15.97:0/2914186749 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b1c4a22870 0x55b1c4a30c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:52.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.759+0000 7f970e800640 1 --2- 172.21.15.97:0/2914186749 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b1c4a279e0 0x55b1c4a31ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:52.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.759+0000 7f970d400640 1 -- 172.21.15.97:0/2914186749 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b1c4a279e0 msgr2=0x55b1c4a31ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:52.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.759+0000 7f970d400640 1 --2- 172.21.15.97:0/2914186749 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b1c4a279e0 0x55b1c4a31ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:52.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.759+0000 7f970d400640 1 -- 172.21.15.97:0/2914186749 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b1c4a21e80 msgr2=0x55b1c4a306c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:52.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.759+0000 7f970d400640 1 --2- 172.21.15.97:0/2914186749 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b1c4a21e80 0x55b1c4a306c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:52.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.759+0000 7f970d400640 1 -- 172.21.15.97:0/2914186749 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1c4a32190 con 0x55b1c4a22870 2024-03-27T10:58:52.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.759+0000 7f970e800640 1 --2- 172.21.15.97:0/2914186749 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b1c4a279e0 0x55b1c4a31ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T10:58:52.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.759+0000 7f970de00640 1 --2- 172.21.15.97:0/2914186749 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b1c4a21e80 0x55b1c4a306c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T10:58:52.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.760+0000 7f970d400640 1 --2- 172.21.15.97:0/2914186749 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b1c4a22870 0x55b1c4a30c00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x55b1c4a3a7f0 tx=0x7f970001ef60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63effe2cb271ed1e server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:52.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.760+0000 7f96efe00640 10 client.0 ms_handle_connect on v2:172.21.15.177:3300/0 2024-03-27T10:58:52.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.760+0000 7f96efe00640 1 -- 172.21.15.97:0/2914186749 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f970001e6a0 con 0x55b1c4a22870 2024-03-27T10:58:52.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.760+0000 7f96efe00640 1 -- 172.21.15.97:0/2914186749 <== mon.1 v2:172.21.15.177:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f9700004050 con 0x55b1c4a22870 2024-03-27T10:58:52.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.760+0000 7f9717db6740 1 -- 172.21.15.97:0/2914186749 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96fc0dce50 con 0x55b1c4a22870 2024-03-27T10:58:52.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.760+0000 7f96efe00640 1 -- 172.21.15.97:0/2914186749 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f9700004250 con 0x55b1c4a22870 2024-03-27T10:58:52.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.761+0000 7f9717db6740 10 client.25651 fetch_fsmap learned FSMap version 31 2024-03-27T10:58:52.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.761+0000 7f9717db6740 1 -- 172.21.15.97:0/2914186749 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b1c4a33570 con 0x55b1c4a22870 2024-03-27T10:58:52.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.761+0000 7f96efe00640 1 -- 172.21.15.97:0/2914186749 <== mon.1 v2:172.21.15.177:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f96fc0dce50 con 0x55b1c4a22870 2024-03-27T10:58:52.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.761+0000 7f96efe00640 1 -- 172.21.15.97:0/2914186749 <== mon.1 v2:172.21.15.177:3300/0 5 ==== fsmap(e 31) ==== 8370+0+0 (secure 0 0 0) 0x7f97000037f0 con 0x55b1c4a22870 2024-03-27T10:58:52.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.761+0000 7f9717db6740 10 client.25651 fetch_fsmap finished waiting for FSMap version 31 2024-03-27T10:58:52.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.761+0000 7f9717db6740 10 client.25651 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-27T10:58:52.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.761+0000 7f9717db6740 20 client.25651 populate_metadata read hostname 'smithi097' 2024-03-27T10:58:52.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.761+0000 7f9717db6740 1 --2- 172.21.15.97:0/2914186749 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55b1c4a9e410 0x55b1c4aa0830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:52.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.761+0000 7f9717db6740 4 client.25651 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-27T10:58:52.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.761+0000 7f9717db6740 1 -- 172.21.15.97:0/2914186749 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b1c4986f40 con 0x55b1c4a9e410 2024-03-27T10:58:52.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.762+0000 7f970de00640 1 --2- 172.21.15.97:0/2914186749 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55b1c4a9e410 0x55b1c4aa0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:52.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.763+0000 7f970de00640 1 --2- 172.21.15.97:0/2914186749 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55b1c4a9e410 0x55b1c4aa0830 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab6f61370cf3fbf0 server_cookie=179ea1b7009bbdc7 in_seq=0 out_seq=0 2024-03-27T10:58:52.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.763+0000 7f96efe00640 10 client.25651 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T10:58:52.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.764+0000 7f96efe00640 1 -- 172.21.15.97:0/2914186749 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b1c4986f40 con 0x55b1c4a9e410 2024-03-27T10:58:52.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.764+0000 7f96efe00640 10 client.25651 handle_command_reply: tid=0 2024-03-27T10:58:52.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.764+0000 7f9717db6740 1 client.25651 shutdown 2024-03-27T10:58:52.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.764+0000 7f9717db6740 1 -- 172.21.15.97:0/2914186749 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b1c4a22870 msgr2=0x55b1c4a30c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:52.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.764+0000 7f9717db6740 1 --2- 172.21.15.97:0/2914186749 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b1c4a22870 0x55b1c4a30c00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x55b1c4a3a7f0 tx=0x7f970001ef60 comp rx=0 tx=0).stop 2024-03-27T10:58:52.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.764+0000 7f9717db6740 1 -- 172.21.15.97:0/2914186749 shutdown_connections 2024-03-27T10:58:52.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.764+0000 7f9717db6740 1 --2- 172.21.15.97:0/2914186749 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55b1c4a9e410 0x55b1c4aa0830 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:52.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.765+0000 7f9717db6740 1 --2- 172.21.15.97:0/2914186749 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b1c4a279e0 0x55b1c4a31ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:52.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.765+0000 7f9717db6740 1 --2- 172.21.15.97:0/2914186749 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b1c4a22870 0x55b1c4a30c00 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:52.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.765+0000 7f9717db6740 1 --2- 172.21.15.97:0/2914186749 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b1c4a21e80 0x55b1c4a306c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:52.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.765+0000 7f9717db6740 1 -- 172.21.15.97:0/2914186749 >> 172.21.15.97:0/2914186749 conn(0x55b1c4a14be0 msgr2=0x55b1c4a186d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:52.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.765+0000 7f96efe00640 0 client.25651 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T10:58:52.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.765+0000 7f9717db6740 1 -- 172.21.15.97:0/2914186749 shutdown_connections 2024-03-27T10:58:52.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.765+0000 7f9717db6740 1 -- 172.21.15.97:0/2914186749 wait complete. 2024-03-27T10:58:52.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.765+0000 7f9717db6740 20 client.25651 trim_cache size 0 max 16384 2024-03-27T10:58:52.738 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-03-27T10:58:52.738 INFO:teuthology.orchestra.run.smithi097.stdout: "status": "no active scrubs running", 2024-03-27T10:58:52.738 INFO:teuthology.orchestra.run.smithi097.stdout: "scrubs": {} 2024-03-27T10:58:52.738 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-03-27T10:58:52.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.766+0000 7f970f200640 1 -- 172.21.15.97:0/3886357855 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f96f40a3760 msgr2=0x7f96f40a5c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:52.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.766+0000 7f970f200640 1 --2- 172.21.15.97:0/3886357855 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f96f40a3760 0x7f96f40a5c20 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x55b1c4a230d0 tx=0x7f9708009500 comp rx=0 tx=0).stop 2024-03-27T10:58:52.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.766+0000 7f970f200640 1 -- 172.21.15.97:0/3886357855 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9710103c50 msgr2=0x7f9710199e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:52.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.766+0000 7f970f200640 1 --2- 172.21.15.97:0/3886357855 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9710103c50 0x7f9710199e70 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f96fc000c00 tx=0x7f96fc004040 comp rx=0 tx=0).stop 2024-03-27T10:58:52.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.766+0000 7f970f200640 1 -- 172.21.15.97:0/3886357855 shutdown_connections 2024-03-27T10:58:52.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.766+0000 7f970f200640 1 --2- 172.21.15.97:0/3886357855 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f96f40a3760 0x7f96f40a5c20 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:52.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.766+0000 7f970f200640 1 --2- 172.21.15.97:0/3886357855 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9710105800 0x7f97101a1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:52.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.766+0000 7f970f200640 1 --2- 172.21.15.97:0/3886357855 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9710104e40 0x7f971019a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:52.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.767+0000 7f970f200640 1 --2- 172.21.15.97:0/3886357855 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9710103c50 0x7f9710199e70 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:52.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.767+0000 7f970f200640 1 -- 172.21.15.97:0/3886357855 >> 172.21.15.97:0/3886357855 conn(0x7f97100ff1c0 msgr2=0x7f9710108140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:52.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.767+0000 7f970f200640 1 -- 172.21.15.97:0/3886357855 shutdown_connections 2024-03-27T10:58:52.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:52.767+0000 7f970f200640 1 -- 172.21.15.97:0/3886357855 wait complete. 2024-03-27T10:58:52.762 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-27T10:58:52.762 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-27T10:58:53.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.050+0000 7efed2a00640 1 -- 172.21.15.97:0/3188857897 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efecc105750 msgr2=0x7efecc10bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:53.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.050+0000 7efed2a00640 1 --2- 172.21.15.97:0/3188857897 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efecc105750 0x7efecc10bfe0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7efec0009900 tx=0x7efec002bc30 comp rx=0 tx=0).stop 2024-03-27T10:58:53.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.050+0000 7efed2a00640 1 -- 172.21.15.97:0/3188857897 shutdown_connections 2024-03-27T10:58:53.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.050+0000 7efed2a00640 1 --2- 172.21.15.97:0/3188857897 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efecc105750 0x7efecc10bfe0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:53.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.050+0000 7efed2a00640 1 --2- 172.21.15.97:0/3188857897 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efecc104d90 0x7efecc105210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:53.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.050+0000 7efed2a00640 1 --2- 172.21.15.97:0/3188857897 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efecc103ba0 0x7efecc103fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:53.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.050+0000 7efed2a00640 1 -- 172.21.15.97:0/3188857897 >> 172.21.15.97:0/3188857897 conn(0x7efecc0ff130 msgr2=0x7efecc101550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:53.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.050+0000 7efed2a00640 1 -- 172.21.15.97:0/3188857897 shutdown_connections 2024-03-27T10:58:53.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.050+0000 7efed2a00640 1 -- 172.21.15.97:0/3188857897 wait complete. 2024-03-27T10:58:53.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.051+0000 7efed2a00640 1 Processor -- start 2024-03-27T10:58:53.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.052+0000 7efed2a00640 1 -- start start 2024-03-27T10:58:53.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.052+0000 7efed2a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efecc103ba0 0x7efecc199d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:53.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.052+0000 7efed2a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efecc104d90 0x7efecc19a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:53.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.052+0000 7efed2a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efecc105750 0x7efecc1a12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:53.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.052+0000 7efed2a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7efecc10f720 con 0x7efecc103ba0 2024-03-27T10:58:53.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.052+0000 7efed2a00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7efecc10f5a0 con 0x7efecc105750 2024-03-27T10:58:53.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.052+0000 7efed2a00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7efecc10f8a0 con 0x7efecc104d90 2024-03-27T10:58:53.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.052+0000 7efed1600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efecc103ba0 0x7efecc199d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:53.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.052+0000 7efed1600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efecc103ba0 0x7efecc199d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49372/0 (socket says 172.21.15.97:49372) 2024-03-27T10:58:53.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.053+0000 7efed1600640 1 -- 172.21.15.97:0/1384333746 learned_addr learned my addr 172.21.15.97:0/1384333746 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:58:53.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.053+0000 7efed2000640 1 --2- 172.21.15.97:0/1384333746 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efecc105750 0x7efecc1a12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:53.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.053+0000 7efed0c00640 1 --2- 172.21.15.97:0/1384333746 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efecc104d90 0x7efecc19a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:53.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.053+0000 7efed2000640 1 -- 172.21.15.97:0/1384333746 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efecc104d90 msgr2=0x7efecc19a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:53.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.053+0000 7efed2000640 1 --2- 172.21.15.97:0/1384333746 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efecc104d90 0x7efecc19a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:53.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.053+0000 7efed2000640 1 -- 172.21.15.97:0/1384333746 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efecc103ba0 msgr2=0x7efecc199d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:53.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.053+0000 7efed2000640 1 --2- 172.21.15.97:0/1384333746 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efecc103ba0 0x7efecc199d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:53.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.053+0000 7efed2000640 1 -- 172.21.15.97:0/1384333746 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efecc1a1940 con 0x7efecc105750 2024-03-27T10:58:53.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.053+0000 7efed1600640 1 --2- 172.21.15.97:0/1384333746 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efecc103ba0 0x7efecc199d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T10:58:53.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.054+0000 7efed2000640 1 --2- 172.21.15.97:0/1384333746 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efecc105750 0x7efecc1a12a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7efec0036040 tx=0x7efec0005dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:53.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.054+0000 7efeca000640 1 -- 172.21.15.97:0/1384333746 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec0043020 con 0x7efecc105750 2024-03-27T10:58:53.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.054+0000 7efed2a00640 1 -- 172.21.15.97:0/1384333746 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efecc1a1bd0 con 0x7efecc105750 2024-03-27T10:58:53.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.054+0000 7efed2a00640 1 -- 172.21.15.97:0/1384333746 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efecc1a2170 con 0x7efecc105750 2024-03-27T10:58:53.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.054+0000 7efeca000640 1 -- 172.21.15.97:0/1384333746 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efec0004410 con 0x7efecc105750 2024-03-27T10:58:53.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.054+0000 7efeca000640 1 -- 172.21.15.97:0/1384333746 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec0003b50 con 0x7efecc105750 2024-03-27T10:58:53.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.056+0000 7efed2a00640 1 -- 172.21.15.97:0/1384333746 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efeac0052c0 con 0x7efecc105750 2024-03-27T10:58:53.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.058+0000 7efeca000640 1 -- 172.21.15.97:0/1384333746 <== mon.2 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7efec0046020 con 0x7efecc105750 2024-03-27T10:58:53.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.060+0000 7efeca000640 1 --2- 172.21.15.97:0/1384333746 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7efeb40a3710 0x7efeb40a5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:53.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.060+0000 7efeca000640 1 -- 172.21.15.97:0/1384333746 <== mon.2 v2:172.21.15.114:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7efec00f2420 con 0x7efecc105750 2024-03-27T10:58:53.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.060+0000 7efed1600640 1 --2- 172.21.15.97:0/1384333746 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7efeb40a3710 0x7efeb40a5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:53.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.061+0000 7efed1600640 1 --2- 172.21.15.97:0/1384333746 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7efeb40a3710 0x7efeb40a5bd0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7efecc19b200 tx=0x7efec4009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:53.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.064+0000 7efeca000640 1 -- 172.21.15.97:0/1384333746 <== mon.2 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7efec004d020 con 0x7efecc105750 2024-03-27T10:58:53.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.478+0000 7efed2a00640 1 -- 172.21.15.97:0/1384333746 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efeac0055b0 con 0x7efecc105750 2024-03-27T10:58:53.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.480+0000 7efeca000640 1 -- 172.21.15.97:0/1384333746 <== mon.2 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11952 (secure 0 0 0) 0x7efec00befa0 con 0x7efecc105750 2024-03-27T10:58:53.453 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T10:58:53.453 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1009423904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1009423904},{"type":"v1","addr":"172.21.15.97:6835","nonce":1009423904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3251173433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3251173433},{"type":"v1","addr":"172.21.15.97:6837","nonce":3251173433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-27T10:54:04.510178+0000","modified":"2024-03-27T10:58:24.628614+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34508,"mds_1":34478,"mds_2":34445,"mds_3":24454,"mds_4":24400},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14646":{"gid":14646,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6839/1057596821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1057596821},{"type":"v1","addr":"172.21.15.97:6839","nonce":1057596821}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14658":{"gid":14658,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6841/1672464856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6840","nonce":1672464856},{"type":"v1","addr":"172.21.15.97:6841","nonce":1672464856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.177:6833/3787981632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3787981632},{"type":"v1","addr":"172.21.15.177:6833","nonce":3787981632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.177:6835/3349112132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":3349112132},{"type":"v1","addr":"172.21.15.177:6835","nonce":3349112132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6837/3244432640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":3244432640},{"type":"v1","addr":"172.21.15.177:6837","nonce":3244432640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6839/3340454605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":3340454605},{"type":"v1","addr":"172.21.15.177:6839","nonce":3340454605}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34415":{"gid":34415,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6833/4052517902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":4052517902},{"type":"v1","addr":"172.21.15.114:6833","nonce":4052517902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34445":{"gid":34445,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.114:6835/3581918222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3581918222},{"type":"v1","addr":"172.21.15.114:6835","nonce":3581918222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34478":{"gid":34478,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":72,"addr":"172.21.15.114:6837/616775588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":616775588},{"type":"v1","addr":"172.21.15.114:6837","nonce":616775588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34508":{"gid":34508,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.114:6839/3937193784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":3937193784},{"type":"v1","addr":"172.21.15.114:6839","nonce":3937193784}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34508,"qdb_cluster":[34445,34508,24400,24454,34478]},"id":1}]} 2024-03-27T10:58:53.454 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 31 2024-03-27T10:58:53.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.483+0000 7efed2a00640 1 -- 172.21.15.97:0/1384333746 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7efeb40a3710 msgr2=0x7efeb40a5bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:53.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.483+0000 7efed2a00640 1 --2- 172.21.15.97:0/1384333746 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7efeb40a3710 0x7efeb40a5bd0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7efecc19b200 tx=0x7efec4009500 comp rx=0 tx=0).stop 2024-03-27T10:58:53.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.483+0000 7efed2a00640 1 -- 172.21.15.97:0/1384333746 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efecc105750 msgr2=0x7efecc1a12a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:53.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.483+0000 7efed2a00640 1 --2- 172.21.15.97:0/1384333746 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efecc105750 0x7efecc1a12a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7efec0036040 tx=0x7efec0005dd0 comp rx=0 tx=0).stop 2024-03-27T10:58:53.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.483+0000 7efed2a00640 1 -- 172.21.15.97:0/1384333746 shutdown_connections 2024-03-27T10:58:53.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.483+0000 7efed2a00640 1 --2- 172.21.15.97:0/1384333746 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7efeb40a3710 0x7efeb40a5bd0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:53.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.483+0000 7efed2a00640 1 --2- 172.21.15.97:0/1384333746 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efecc105750 0x7efecc1a12a0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:53.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.483+0000 7efed2a00640 1 --2- 172.21.15.97:0/1384333746 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efecc104d90 0x7efecc19a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:53.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.483+0000 7efed2a00640 1 --2- 172.21.15.97:0/1384333746 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efecc103ba0 0x7efecc199d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:53.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.483+0000 7efed2a00640 1 -- 172.21.15.97:0/1384333746 >> 172.21.15.97:0/1384333746 conn(0x7efecc0ff130 msgr2=0x7efecc108030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:53.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.484+0000 7efed2a00640 1 -- 172.21.15.97:0/1384333746 shutdown_connections 2024-03-27T10:58:53.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.484+0000 7efed2a00640 1 -- 172.21.15.97:0/1384333746 wait complete. 2024-03-27T10:58:53.480 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-27T10:58:53.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:58:53 smithi097 ceph-mon[31859]: pgmap v508: 97 pgs: 97 active+clean; 5.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 122 MiB/s wr, 70 op/s 2024-03-27T10:58:53.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:58:53 smithi177 ceph-mon[35294]: pgmap v508: 97 pgs: 97 active+clean; 5.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 122 MiB/s wr, 70 op/s 2024-03-27T10:58:53.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.773+0000 7ff498e00640 1 -- 172.21.15.97:0/1876253196 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4940680b0 msgr2=0x7ff494100f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:53.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.773+0000 7ff498e00640 1 --2- 172.21.15.97:0/1876253196 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4940680b0 0x7ff494100f90 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7ff484009860 tx=0x7ff48402bb70 comp rx=0 tx=0).stop 2024-03-27T10:58:53.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.773+0000 7ff498e00640 1 -- 172.21.15.97:0/1876253196 shutdown_connections 2024-03-27T10:58:53.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.773+0000 7ff498e00640 1 --2- 172.21.15.97:0/1876253196 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff4941061d0 0x7ff4941085c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:53.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.773+0000 7ff498e00640 1 --2- 172.21.15.97:0/1876253196 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff4941014d0 0x7ff494101950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:53.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.773+0000 7ff498e00640 1 --2- 172.21.15.97:0/1876253196 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4940680b0 0x7ff494100f90 secure :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7ff484009860 tx=0x7ff48402bb70 comp rx=0 tx=0).stop 2024-03-27T10:58:53.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.773+0000 7ff498e00640 1 -- 172.21.15.97:0/1876253196 >> 172.21.15.97:0/1876253196 conn(0x7ff4940fb310 msgr2=0x7ff4940fd750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:53.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.773+0000 7ff498e00640 1 -- 172.21.15.97:0/1876253196 shutdown_connections 2024-03-27T10:58:53.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.773+0000 7ff498e00640 1 -- 172.21.15.97:0/1876253196 wait complete. 2024-03-27T10:58:53.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.774+0000 7ff498e00640 1 Processor -- start 2024-03-27T10:58:53.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.775+0000 7ff498e00640 1 -- start start 2024-03-27T10:58:53.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.775+0000 7ff498e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff4941014d0 0x7ff49419e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:53.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.775+0000 7ff498e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff4941061d0 0x7ff49419eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:53.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.775+0000 7ff498e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4941a3bd0 0x7ff4941a5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:53.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.775+0000 7ff498e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff49410b4c0 con 0x7ff4941a3bd0 2024-03-27T10:58:53.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.775+0000 7ff498e00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff49410b340 con 0x7ff4941014d0 2024-03-27T10:58:53.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.775+0000 7ff498e00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff49410b640 con 0x7ff4941061d0 2024-03-27T10:58:53.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.775+0000 7ff491e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff4941014d0 0x7ff49419e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:53.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.775+0000 7ff491e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff4941014d0 0x7ff49419e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:60032/0 (socket says 172.21.15.97:60032) 2024-03-27T10:58:53.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.775+0000 7ff491e00640 1 -- 172.21.15.97:0/2993170395 learned_addr learned my addr 172.21.15.97:0/2993170395 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:58:53.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.775+0000 7ff491400640 1 --2- 172.21.15.97:0/2993170395 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff4941061d0 0x7ff49419eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:53.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.776+0000 7ff492800640 1 --2- 172.21.15.97:0/2993170395 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4941a3bd0 0x7ff4941a5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:53.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.776+0000 7ff491e00640 1 -- 172.21.15.97:0/2993170395 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff4941061d0 msgr2=0x7ff49419eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:53.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.776+0000 7ff491e00640 1 --2- 172.21.15.97:0/2993170395 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff4941061d0 0x7ff49419eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:53.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.776+0000 7ff491e00640 1 -- 172.21.15.97:0/2993170395 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4941a3bd0 msgr2=0x7ff4941a5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:53.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.776+0000 7ff491e00640 1 --2- 172.21.15.97:0/2993170395 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4941a3bd0 0x7ff4941a5fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:53.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.776+0000 7ff491e00640 1 -- 172.21.15.97:0/2993170395 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff49406b670 con 0x7ff4941014d0 2024-03-27T10:58:53.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.776+0000 7ff491e00640 1 --2- 172.21.15.97:0/2993170395 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff4941014d0 0x7ff49419e630 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff484037040 tx=0x7ff484035d40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:53.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.777+0000 7ff48aa00640 1 -- 172.21.15.97:0/2993170395 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff484039040 con 0x7ff4941014d0 2024-03-27T10:58:53.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.777+0000 7ff48aa00640 1 -- 172.21.15.97:0/2993170395 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff4840054e0 con 0x7ff4941014d0 2024-03-27T10:58:53.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.777+0000 7ff498e00640 1 -- 172.21.15.97:0/2993170395 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff49406b900 con 0x7ff4941014d0 2024-03-27T10:58:53.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.777+0000 7ff48aa00640 1 -- 172.21.15.97:0/2993170395 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff484004b60 con 0x7ff4941014d0 2024-03-27T10:58:53.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.777+0000 7ff498e00640 1 -- 172.21.15.97:0/2993170395 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff49406bde0 con 0x7ff4941014d0 2024-03-27T10:58:53.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.781+0000 7ff49bbd0740 1 Processor -- start 2024-03-27T10:58:53.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.781+0000 7ff49bbd0740 1 -- start start 2024-03-27T10:58:53.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.782+0000 7ff48aa00640 1 -- 172.21.15.97:0/2993170395 <== mon.2 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7ff484044060 con 0x7ff4941014d0 2024-03-27T10:58:53.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.782+0000 7ff49bbd0740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56506f8868c0 0x56506f886ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:53.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.782+0000 7ff49bbd0740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56506f8872b0 0x56506f88bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:53.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.782+0000 7ff49bbd0740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56506f88c420 0x56506f88e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:53.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.782+0000 7ff49bbd0740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56506f84e7b0 con 0x56506f88c420 2024-03-27T10:58:53.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.782+0000 7ff49bbd0740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x56506f7eb670 con 0x56506f8868c0 2024-03-27T10:58:53.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.782+0000 7ff49bbd0740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x56506f793970 con 0x56506f8872b0 2024-03-27T10:58:53.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.782+0000 7ff491400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56506f88c420 0x56506f88e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:53.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.782+0000 7ff491400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56506f88c420 0x56506f88e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49400/0 (socket says 172.21.15.97:49400) 2024-03-27T10:58:53.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.782+0000 7ff491400640 1 -- 172.21.15.97:0/4290547637 learned_addr learned my addr 172.21.15.97:0/4290547637 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:58:53.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.782+0000 7ff492800640 1 --2- 172.21.15.97:0/4290547637 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56506f8868c0 0x56506f886ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:53.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.782+0000 7ff491e00640 1 --2- 172.21.15.97:0/4290547637 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56506f8872b0 0x56506f88bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:53.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.783+0000 7ff491400640 1 -- 172.21.15.97:0/4290547637 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56506f8872b0 msgr2=0x56506f88bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:53.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.783+0000 7ff491400640 1 --2- 172.21.15.97:0/4290547637 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56506f8872b0 0x56506f88bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:53.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.783+0000 7ff491400640 1 -- 172.21.15.97:0/4290547637 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56506f8868c0 msgr2=0x56506f886ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:53.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.783+0000 7ff491400640 1 --2- 172.21.15.97:0/4290547637 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56506f8868c0 0x56506f886ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:53.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.783+0000 7ff491400640 1 -- 172.21.15.97:0/4290547637 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56506f4ff3f0 con 0x56506f88c420 2024-03-27T10:58:53.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.783+0000 7ff491400640 1 --2- 172.21.15.97:0/4290547637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56506f88c420 0x56506f88e8b0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7ff480009c60 tx=0x7ff480011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5901f26fb9d61d3 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:53.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.784+0000 7ff48aa00640 1 --2- 172.21.15.97:0/2993170395 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7ff4780a3680 0x7ff4780a5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:53.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.784+0000 7ff47f400640 1 -- 172.21.15.97:0/4290547637 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff480004220 con 0x56506f88c420 2024-03-27T10:58:53.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.784+0000 7ff47f400640 1 -- 172.21.15.97:0/4290547637 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff4800043c0 con 0x56506f88c420 2024-03-27T10:58:53.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.784+0000 7ff47f400640 1 -- 172.21.15.97:0/4290547637 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff480004c70 con 0x56506f88c420 2024-03-27T10:58:53.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.784+0000 7ff48aa00640 1 -- 172.21.15.97:0/2993170395 <== mon.2 v2:172.21.15.114:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7ff4840f0a10 con 0x7ff4941014d0 2024-03-27T10:58:53.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.785+0000 7ff49bbd0740 1 -- 172.21.15.97:0/4290547637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56506f88c420 msgr2=0x56506f88e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:53.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.785+0000 7ff49bbd0740 1 --2- 172.21.15.97:0/4290547637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56506f88c420 0x56506f88e8b0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7ff480009c60 tx=0x7ff480011810 comp rx=0 tx=0).stop 2024-03-27T10:58:53.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.785+0000 7ff492800640 1 --2- 172.21.15.97:0/2993170395 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7ff4780a3680 0x7ff4780a5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:53.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.786+0000 7ff49bbd0740 1 -- 172.21.15.97:0/4290547637 shutdown_connections 2024-03-27T10:58:53.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.786+0000 7ff49bbd0740 1 --2- 172.21.15.97:0/4290547637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56506f88c420 0x56506f88e8b0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:53.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.786+0000 7ff49bbd0740 1 --2- 172.21.15.97:0/4290547637 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56506f8872b0 0x56506f88bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:53.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.786+0000 7ff49bbd0740 1 --2- 172.21.15.97:0/4290547637 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56506f8868c0 0x56506f886ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:53.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.786+0000 7ff49bbd0740 1 -- 172.21.15.97:0/4290547637 >> 172.21.15.97:0/4290547637 conn(0x56506f881e30 msgr2=0x56506f882230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:53.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.786+0000 7ff492800640 1 --2- 172.21.15.97:0/2993170395 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7ff4780a3680 0x7ff4780a5b40 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x56506f887b10 tx=0x7ff48c009740 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:53.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.788+0000 7ff49bbd0740 1 -- 172.21.15.97:0/4290547637 shutdown_connections 2024-03-27T10:58:53.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.788+0000 7ff49bbd0740 1 -- 172.21.15.97:0/4290547637 wait complete. 2024-03-27T10:58:53.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.789+0000 7ff49bbd0740 1 -- start start 2024-03-27T10:58:53.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.789+0000 7ff49bbd0740 1 Processor -- start 2024-03-27T10:58:53.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.790+0000 7ff49bbd0740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56506f8868c0 0x56506f8909f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:53.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.790+0000 7ff49bbd0740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56506f8872b0 0x56506f890f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:53.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.790+0000 7ff49bbd0740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56506f88c420 0x56506f897f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:53.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.790+0000 7ff49bbd0740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56506f7a6ca0 con 0x56506f8872b0 2024-03-27T10:58:53.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.790+0000 7ff49bbd0740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x56506f7e7620 con 0x56506f88c420 2024-03-27T10:58:53.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.790+0000 7ff49bbd0740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x56506f88f3d0 con 0x56506f8868c0 2024-03-27T10:58:53.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.790+0000 7ff49bbd0740 10 client.0 _ll_register_callbacks cb 0x56506f7e3160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:58:53.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.790+0000 7ff492800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56506f88c420 0x56506f897f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:53.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.790+0000 7ff491400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56506f8872b0 0x56506f890f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:53.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.790+0000 7ff492800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56506f88c420 0x56506f897f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:60056/0 (socket says 172.21.15.97:60056) 2024-03-27T10:58:53.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.790+0000 7ff492800640 1 -- 172.21.15.97:0/1200096148 learned_addr learned my addr 172.21.15.97:0/1200096148 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:58:53.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.791+0000 7ff491e00640 1 --2- 172.21.15.97:0/1200096148 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56506f8868c0 0x56506f8909f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:53.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.791+0000 7ff492800640 1 -- 172.21.15.97:0/1200096148 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56506f8868c0 msgr2=0x56506f8909f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:53.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.791+0000 7ff492800640 1 --2- 172.21.15.97:0/1200096148 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56506f8868c0 0x56506f8909f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:53.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.791+0000 7ff492800640 1 -- 172.21.15.97:0/1200096148 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56506f8872b0 msgr2=0x56506f890f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:53.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.791+0000 7ff492800640 1 --2- 172.21.15.97:0/1200096148 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56506f8872b0 0x56506f890f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:53.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.791+0000 7ff492800640 1 -- 172.21.15.97:0/1200096148 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56506f5c97d0 con 0x56506f88c420 2024-03-27T10:58:53.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.791+0000 7ff491e00640 1 --2- 172.21.15.97:0/1200096148 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56506f8868c0 0x56506f8909f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T10:58:53.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.791+0000 7ff492800640 1 --2- 172.21.15.97:0/1200096148 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56506f88c420 0x56506f897f70 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff48c009fd0 tx=0x7ff48c004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6515e09d6a71518 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:53.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.792+0000 7ff473e00640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-03-27T10:58:53.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.792+0000 7ff473e00640 1 -- 172.21.15.97:0/1200096148 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff48c010d30 con 0x56506f88c420 2024-03-27T10:58:53.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.792+0000 7ff473e00640 1 -- 172.21.15.97:0/1200096148 <== mon.2 v2:172.21.15.114:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7ff48c028470 con 0x56506f88c420 2024-03-27T10:58:53.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.792+0000 7ff49bbd0740 1 -- 172.21.15.97:0/1200096148 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4800043c0 con 0x56506f88c420 2024-03-27T10:58:53.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.792+0000 7ff473e00640 1 -- 172.21.15.97:0/1200096148 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff48c028670 con 0x56506f88c420 2024-03-27T10:58:53.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.792+0000 7ff49bbd0740 10 client.35918 fetch_fsmap learned FSMap version 31 2024-03-27T10:58:53.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.793+0000 7ff49bbd0740 1 -- 172.21.15.97:0/1200096148 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56506f89b3a0 con 0x56506f88c420 2024-03-27T10:58:53.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.793+0000 7ff473e00640 1 -- 172.21.15.97:0/1200096148 <== mon.2 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff4800043c0 con 0x56506f88c420 2024-03-27T10:58:53.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.793+0000 7ff473e00640 1 -- 172.21.15.97:0/1200096148 <== mon.2 v2:172.21.15.114:3300/0 5 ==== fsmap(e 31) ==== 8370+0+0 (secure 0 0 0) 0x7ff48c00f040 con 0x56506f88c420 2024-03-27T10:58:53.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.793+0000 7ff49bbd0740 10 client.35918 fetch_fsmap finished waiting for FSMap version 31 2024-03-27T10:58:53.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.793+0000 7ff49bbd0740 10 client.35918 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-03-27T10:58:53.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.793+0000 7ff49bbd0740 20 client.35918 populate_metadata read hostname 'smithi097' 2024-03-27T10:58:53.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.793+0000 7ff49bbd0740 1 --2- 172.21.15.97:0/1200096148 >> [v2:172.21.15.177:6832/3787981632,v1:172.21.15.177:6833/3787981632] conn(0x56506f89ca50 0x56506f89eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:53.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.793+0000 7ff49bbd0740 4 client.35918 mds_command: new command op to 24400 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T10:58:53.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.793+0000 7ff49bbd0740 1 -- 172.21.15.97:0/1200096148 --> [v2:172.21.15.177:6832/3787981632,v1:172.21.15.177:6833/3787981632] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56506f7f8bb0 con 0x56506f89ca50 2024-03-27T10:58:53.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.794+0000 7ff491e00640 1 --2- 172.21.15.97:0/1200096148 >> [v2:172.21.15.177:6832/3787981632,v1:172.21.15.177:6833/3787981632] conn(0x56506f89ca50 0x56506f89eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:53.768 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:58:53 smithi114 ceph-mon[35338]: pgmap v508: 97 pgs: 97 active+clean; 5.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 122 MiB/s wr, 70 op/s 2024-03-27T10:58:53.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.796+0000 7ff491e00640 1 --2- 172.21.15.97:0/1200096148 >> [v2:172.21.15.177:6832/3787981632,v1:172.21.15.177:6833/3787981632] conn(0x56506f89ca50 0x56506f89eeb0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9d8397975e9ec568 server_cookie=5d9b77499cfcc002 in_seq=0 out_seq=0 2024-03-27T10:58:53.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.796+0000 7ff473e00640 10 client.35918 ms_handle_connect on v2:172.21.15.177:6832/3787981632 2024-03-27T10:58:53.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.797+0000 7ff473e00640 1 -- 172.21.15.97:0/1200096148 <== mds.4 v2:172.21.15.177:6832/3787981632 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x56506f7f8bb0 con 0x56506f89ca50 2024-03-27T10:58:53.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.797+0000 7ff473e00640 10 client.35918 handle_command_reply: tid=0 2024-03-27T10:58:53.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.797+0000 7ff49bbd0740 1 client.35918 shutdown 2024-03-27T10:58:53.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.797+0000 7ff49bbd0740 1 -- 172.21.15.97:0/1200096148 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56506f88c420 msgr2=0x56506f897f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:53.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.797+0000 7ff49bbd0740 1 --2- 172.21.15.97:0/1200096148 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56506f88c420 0x56506f897f70 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff48c009fd0 tx=0x7ff48c004a40 comp rx=0 tx=0).stop 2024-03-27T10:58:53.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.797+0000 7ff49bbd0740 1 -- 172.21.15.97:0/1200096148 shutdown_connections 2024-03-27T10:58:53.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.797+0000 7ff49bbd0740 1 --2- 172.21.15.97:0/1200096148 >> [v2:172.21.15.177:6832/3787981632,v1:172.21.15.177:6833/3787981632] conn(0x56506f89ca50 0x56506f89eeb0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:53.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.798+0000 7ff49bbd0740 1 --2- 172.21.15.97:0/1200096148 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56506f88c420 0x56506f897f70 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:53.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.798+0000 7ff49bbd0740 1 --2- 172.21.15.97:0/1200096148 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56506f8872b0 0x56506f890f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:53.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.798+0000 7ff49bbd0740 1 --2- 172.21.15.97:0/1200096148 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56506f8868c0 0x56506f8909f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:53.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.798+0000 7ff49bbd0740 1 -- 172.21.15.97:0/1200096148 >> 172.21.15.97:0/1200096148 conn(0x56506f881e30 msgr2=0x56506f881030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:53.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.798+0000 7ff473e00640 0 client.35918 ms_handle_reset on v2:172.21.15.177:6832/3787981632 2024-03-27T10:58:53.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.799+0000 7ff49bbd0740 1 -- 172.21.15.97:0/1200096148 shutdown_connections 2024-03-27T10:58:53.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.799+0000 7ff49bbd0740 1 -- 172.21.15.97:0/1200096148 wait complete. 2024-03-27T10:58:53.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.799+0000 7ff49bbd0740 20 client.35918 trim_cache size 0 max 16384 2024-03-27T10:58:53.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.821+0000 7ff49bbd0740 1 Processor -- start 2024-03-27T10:58:53.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.821+0000 7ff49bbd0740 1 -- start start 2024-03-27T10:58:53.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.821+0000 7ff49bbd0740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56506f8868c0 0x56506f8807a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:53.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.821+0000 7ff49bbd0740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56506f8872b0 0x56506f897f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:53.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.821+0000 7ff49bbd0740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56506f88c420 0x56506f880ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:53.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.821+0000 7ff49bbd0740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff4941b82f0 con 0x56506f8872b0 2024-03-27T10:58:53.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.821+0000 7ff49bbd0740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff4941b9ef0 con 0x56506f8868c0 2024-03-27T10:58:53.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.821+0000 7ff49bbd0740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff4941b89c0 con 0x56506f88c420 2024-03-27T10:58:53.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.822+0000 7ff491e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56506f8868c0 0x56506f8807a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:53.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.822+0000 7ff491e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56506f8868c0 0x56506f8807a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:60064/0 (socket says 172.21.15.97:60064) 2024-03-27T10:58:53.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.822+0000 7ff491e00640 1 -- 172.21.15.97:0/2139073621 learned_addr learned my addr 172.21.15.97:0/2139073621 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:58:53.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.822+0000 7ff491400640 1 --2- 172.21.15.97:0/2139073621 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56506f8872b0 0x56506f897f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:53.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.822+0000 7ff492800640 1 --2- 172.21.15.97:0/2139073621 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56506f88c420 0x56506f880ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:53.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.823+0000 7ff491400640 1 -- 172.21.15.97:0/2139073621 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56506f88c420 msgr2=0x56506f880ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:53.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.823+0000 7ff491400640 1 --2- 172.21.15.97:0/2139073621 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56506f88c420 0x56506f880ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:53.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.823+0000 7ff491400640 1 -- 172.21.15.97:0/2139073621 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56506f8868c0 msgr2=0x56506f8807a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:53.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.823+0000 7ff491400640 1 --2- 172.21.15.97:0/2139073621 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56506f8868c0 0x56506f8807a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:53.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.823+0000 7ff491400640 1 -- 172.21.15.97:0/2139073621 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56506f887b40 con 0x56506f8872b0 2024-03-27T10:58:53.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.823+0000 7ff491400640 1 --2- 172.21.15.97:0/2139073621 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56506f8872b0 0x56506f897f30 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7ff48001b040 tx=0x7ff4800099b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca47cfe638282a76 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:53.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.824+0000 7ff47ea00640 1 -- 172.21.15.97:0/2139073621 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff48001d040 con 0x56506f8872b0 2024-03-27T10:58:53.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.824+0000 7ff47ea00640 1 -- 172.21.15.97:0/2139073621 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff480004240 con 0x56506f8872b0 2024-03-27T10:58:53.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.824+0000 7ff47ea00640 1 -- 172.21.15.97:0/2139073621 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4800246e0 con 0x56506f8872b0 2024-03-27T10:58:53.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.825+0000 7ff49bbd0740 1 -- 172.21.15.97:0/2139073621 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56506f8872b0 msgr2=0x56506f897f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:53.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.825+0000 7ff49bbd0740 1 --2- 172.21.15.97:0/2139073621 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56506f8872b0 0x56506f897f30 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7ff48001b040 tx=0x7ff4800099b0 comp rx=0 tx=0).stop 2024-03-27T10:58:53.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.825+0000 7ff49bbd0740 1 -- 172.21.15.97:0/2139073621 shutdown_connections 2024-03-27T10:58:53.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.825+0000 7ff49bbd0740 1 --2- 172.21.15.97:0/2139073621 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56506f88c420 0x56506f880ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:53.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.825+0000 7ff49bbd0740 1 --2- 172.21.15.97:0/2139073621 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56506f8872b0 0x56506f897f30 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:53.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.825+0000 7ff49bbd0740 1 --2- 172.21.15.97:0/2139073621 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56506f8868c0 0x56506f8807a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:53.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.825+0000 7ff49bbd0740 1 -- 172.21.15.97:0/2139073621 >> 172.21.15.97:0/2139073621 conn(0x56506f881e30 msgr2=0x56506f89e7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:53.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.825+0000 7ff49bbd0740 1 -- 172.21.15.97:0/2139073621 shutdown_connections 2024-03-27T10:58:53.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.826+0000 7ff49bbd0740 1 -- 172.21.15.97:0/2139073621 wait complete. 2024-03-27T10:58:53.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.826+0000 7ff49bbd0740 1 -- start start 2024-03-27T10:58:53.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.826+0000 7ff49bbd0740 1 Processor -- start 2024-03-27T10:58:53.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.827+0000 7ff49bbd0740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56506f8868c0 0x56506f895370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:53.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.827+0000 7ff49bbd0740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56506f8872b0 0x56506f8958b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:53.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.827+0000 7ff49bbd0740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56506f88c420 0x56506f896870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:53.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.827+0000 7ff49bbd0740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff4941b9760 con 0x56506f8872b0 2024-03-27T10:58:53.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.827+0000 7ff49bbd0740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff4941b9090 con 0x56506f88c420 2024-03-27T10:58:53.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.827+0000 7ff49bbd0740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff494072b10 con 0x56506f8868c0 2024-03-27T10:58:53.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.827+0000 7ff49bbd0740 10 client.0 _ll_register_callbacks cb 0x7ff48c009a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:58:53.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.827+0000 7ff491e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56506f8868c0 0x56506f895370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:53.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.828+0000 7ff491400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56506f8872b0 0x56506f8958b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:53.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.828+0000 7ff491e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56506f8868c0 0x56506f895370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.97:56874/0 (socket says 172.21.15.97:56874) 2024-03-27T10:58:53.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.828+0000 7ff491e00640 1 -- 172.21.15.97:0/34632227 learned_addr learned my addr 172.21.15.97:0/34632227 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:58:53.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.828+0000 7ff492800640 1 --2- 172.21.15.97:0/34632227 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56506f88c420 0x56506f896870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:53.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.828+0000 7ff491400640 1 -- 172.21.15.97:0/34632227 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56506f8868c0 msgr2=0x56506f895370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:53.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.828+0000 7ff491400640 1 --2- 172.21.15.97:0/34632227 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56506f8868c0 0x56506f895370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:53.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.828+0000 7ff491400640 1 -- 172.21.15.97:0/34632227 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56506f88c420 msgr2=0x56506f896870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:53.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.828+0000 7ff491400640 1 --2- 172.21.15.97:0/34632227 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56506f88c420 0x56506f896870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:53.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.828+0000 7ff491400640 1 -- 172.21.15.97:0/34632227 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56506f896e60 con 0x56506f8872b0 2024-03-27T10:58:53.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.828+0000 7ff492800640 1 --2- 172.21.15.97:0/34632227 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56506f88c420 0x56506f896870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T10:58:53.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.829+0000 7ff491400640 1 --2- 172.21.15.97:0/34632227 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56506f8872b0 0x56506f8958b0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7ff480000c00 tx=0x7ff4800097e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eef518287e397c74 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:53.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.829+0000 7ff473e00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T10:58:53.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.829+0000 7ff473e00640 1 -- 172.21.15.97:0/34632227 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff480032070 con 0x56506f8872b0 2024-03-27T10:58:53.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.829+0000 7ff473e00640 1 -- 172.21.15.97:0/34632227 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7ff480003fd0 con 0x56506f8872b0 2024-03-27T10:58:53.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.829+0000 7ff49bbd0740 1 -- 172.21.15.97:0/34632227 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff480004240 con 0x56506f8872b0 2024-03-27T10:58:53.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.830+0000 7ff473e00640 1 -- 172.21.15.97:0/34632227 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff48001d040 con 0x56506f8872b0 2024-03-27T10:58:53.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.830+0000 7ff49bbd0740 10 client.16281 fetch_fsmap learned FSMap version 31 2024-03-27T10:58:53.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.830+0000 7ff49bbd0740 1 -- 172.21.15.97:0/34632227 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56506f898160 con 0x56506f8872b0 2024-03-27T10:58:53.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.831+0000 7ff473e00640 1 -- 172.21.15.97:0/34632227 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff480004240 con 0x56506f8872b0 2024-03-27T10:58:53.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.831+0000 7ff473e00640 1 -- 172.21.15.97:0/34632227 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 31) ==== 8370+0+0 (secure 0 0 0) 0x7ff480024b40 con 0x56506f8872b0 2024-03-27T10:58:53.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.831+0000 7ff49bbd0740 10 client.16281 fetch_fsmap finished waiting for FSMap version 31 2024-03-27T10:58:53.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.831+0000 7ff49bbd0740 10 client.16281 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-03-27T10:58:53.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.831+0000 7ff49bbd0740 20 client.16281 populate_metadata read hostname 'smithi097' 2024-03-27T10:58:53.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.831+0000 7ff49bbd0740 1 --2- 172.21.15.97:0/34632227 >> [v2:172.21.15.177:6832/3787981632,v1:172.21.15.177:6833/3787981632] conn(0x56506f903190 0x56506f9055b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:53.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.831+0000 7ff49bbd0740 4 client.16281 mds_command: new command op to 24400 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-27T10:58:53.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.831+0000 7ff49bbd0740 1 -- 172.21.15.97:0/34632227 --> [v2:172.21.15.177:6832/3787981632,v1:172.21.15.177:6833/3787981632] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56506f7f2f00 con 0x56506f903190 2024-03-27T10:58:53.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.832+0000 7ff491e00640 1 --2- 172.21.15.97:0/34632227 >> [v2:172.21.15.177:6832/3787981632,v1:172.21.15.177:6833/3787981632] conn(0x56506f903190 0x56506f9055b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:53.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.834+0000 7ff473e00640 10 client.16281 ms_handle_connect on v2:172.21.15.177:6832/3787981632 2024-03-27T10:58:53.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.835+0000 7ff491e00640 1 --2- 172.21.15.97:0/34632227 >> [v2:172.21.15.177:6832/3787981632,v1:172.21.15.177:6833/3787981632] conn(0x56506f903190 0x56506f9055b0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=766ce683f572b4e server_cookie=9afe1b8cd58f8849 in_seq=0 out_seq=0 2024-03-27T10:58:53.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.835+0000 7ff473e00640 1 -- 172.21.15.97:0/34632227 <== mds.4 v2:172.21.15.177:6832/3787981632 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56506f7f2f00 con 0x56506f903190 2024-03-27T10:58:53.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.835+0000 7ff473e00640 10 client.16281 handle_command_reply: tid=0 2024-03-27T10:58:53.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.835+0000 7ff49bbd0740 1 client.16281 shutdown 2024-03-27T10:58:53.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.836+0000 7ff49bbd0740 1 -- 172.21.15.97:0/34632227 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56506f8872b0 msgr2=0x56506f8958b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:53.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.836+0000 7ff49bbd0740 1 --2- 172.21.15.97:0/34632227 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56506f8872b0 0x56506f8958b0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7ff480000c00 tx=0x7ff4800097e0 comp rx=0 tx=0).stop 2024-03-27T10:58:53.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.838+0000 7ff49bbd0740 1 -- 172.21.15.97:0/34632227 shutdown_connections 2024-03-27T10:58:53.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.838+0000 7ff49bbd0740 1 --2- 172.21.15.97:0/34632227 >> [v2:172.21.15.177:6832/3787981632,v1:172.21.15.177:6833/3787981632] conn(0x56506f903190 0x56506f9055b0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:53.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.838+0000 7ff49bbd0740 1 --2- 172.21.15.97:0/34632227 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56506f88c420 0x56506f896870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:53.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.838+0000 7ff49bbd0740 1 --2- 172.21.15.97:0/34632227 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56506f8872b0 0x56506f8958b0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:53.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.838+0000 7ff49bbd0740 1 --2- 172.21.15.97:0/34632227 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56506f8868c0 0x56506f895370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:53.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.838+0000 7ff49bbd0740 1 -- 172.21.15.97:0/34632227 >> 172.21.15.97:0/34632227 conn(0x56506f881e30 msgr2=0x56506f89e7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:53.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.838+0000 7ff473e00640 0 client.16281 ms_handle_reset on v2:172.21.15.177:6832/3787981632 2024-03-27T10:58:53.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.839+0000 7ff49bbd0740 1 -- 172.21.15.97:0/34632227 shutdown_connections 2024-03-27T10:58:53.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.839+0000 7ff49bbd0740 1 -- 172.21.15.97:0/34632227 wait complete. 2024-03-27T10:58:53.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.839+0000 7ff49bbd0740 20 client.16281 trim_cache size 0 max 16384 2024-03-27T10:58:53.812 INFO:teuthology.orchestra.run.smithi097.stdout:[] 2024-03-27T10:58:53.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.840+0000 7ff498e00640 1 -- 172.21.15.97:0/2993170395 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7ff4780a3680 msgr2=0x7ff4780a5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:53.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.840+0000 7ff498e00640 1 --2- 172.21.15.97:0/2993170395 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7ff4780a3680 0x7ff4780a5b40 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x56506f887b10 tx=0x7ff48c009740 comp rx=0 tx=0).stop 2024-03-27T10:58:53.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.840+0000 7ff498e00640 1 -- 172.21.15.97:0/2993170395 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff4941014d0 msgr2=0x7ff49419e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:53.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.840+0000 7ff498e00640 1 --2- 172.21.15.97:0/2993170395 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff4941014d0 0x7ff49419e630 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff484037040 tx=0x7ff484035d40 comp rx=0 tx=0).stop 2024-03-27T10:58:53.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.840+0000 7ff498e00640 1 -- 172.21.15.97:0/2993170395 shutdown_connections 2024-03-27T10:58:53.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.840+0000 7ff498e00640 1 --2- 172.21.15.97:0/2993170395 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4941a3bd0 0x7ff4941a5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:53.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.840+0000 7ff498e00640 1 --2- 172.21.15.97:0/2993170395 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff4941061d0 0x7ff49419eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:53.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.840+0000 7ff498e00640 1 --2- 172.21.15.97:0/2993170395 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7ff4780a3680 0x7ff4780a5b40 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:53.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.840+0000 7ff498e00640 1 --2- 172.21.15.97:0/2993170395 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff4941014d0 0x7ff49419e630 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:53.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.840+0000 7ff498e00640 1 -- 172.21.15.97:0/2993170395 >> 172.21.15.97:0/2993170395 conn(0x7ff4940fb310 msgr2=0x7ff4941021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:53.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.841+0000 7ff498e00640 1 -- 172.21.15.97:0/2993170395 shutdown_connections 2024-03-27T10:58:53.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:53.841+0000 7ff498e00640 1 -- 172.21.15.97:0/2993170395 wait complete. 2024-03-27T10:58:53.835 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-27T10:58:54.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.127+0000 7f0aa4800640 1 -- 172.21.15.97:0/4090258827 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0a9c1003c0 msgr2=0x7f0a9c1007e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:54.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.127+0000 7f0aa4800640 1 --2- 172.21.15.97:0/4090258827 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0a9c1003c0 0x7f0a9c1007e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c009900 tx=0x7f0a8c02bc50 comp rx=0 tx=0).stop 2024-03-27T10:58:54.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.127+0000 7f0aa4800640 1 -- 172.21.15.97:0/4090258827 shutdown_connections 2024-03-27T10:58:54.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.127+0000 7f0aa4800640 1 --2- 172.21.15.97:0/4090258827 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0a9c1016c0 0x7f0a9c101b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:54.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.127+0000 7f0aa4800640 1 --2- 172.21.15.97:0/4090258827 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0a9c100d20 0x7f0a9c101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:54.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.127+0000 7f0aa4800640 1 --2- 172.21.15.97:0/4090258827 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0a9c1003c0 0x7f0a9c1007e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:54.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.127+0000 7f0aa4800640 1 -- 172.21.15.97:0/4090258827 >> 172.21.15.97:0/4090258827 conn(0x7f0a9c0fbfe0 msgr2=0x7f0a9c0fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:54.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.127+0000 7f0aa4800640 1 -- 172.21.15.97:0/4090258827 shutdown_connections 2024-03-27T10:58:54.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.127+0000 7f0aa4800640 1 -- 172.21.15.97:0/4090258827 wait complete. 2024-03-27T10:58:54.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.128+0000 7f0aa4800640 1 Processor -- start 2024-03-27T10:58:54.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.129+0000 7f0aa4800640 1 -- start start 2024-03-27T10:58:54.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.129+0000 7f0aa4800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0a9c1003c0 0x7f0a9c19e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:54.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.129+0000 7f0aa4800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0a9c100d20 0x7f0a9c19e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:54.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.129+0000 7f0aa4800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0a9c1016c0 0x7f0a9c1a5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:54.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.129+0000 7f0aa4800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0a9c1037e0 con 0x7f0a9c100d20 2024-03-27T10:58:54.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.129+0000 7f0aa4800640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f0a9c103660 con 0x7f0a9c1016c0 2024-03-27T10:58:54.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.129+0000 7f0aa4800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f0a9c103960 con 0x7f0a9c1003c0 2024-03-27T10:58:54.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.130+0000 7f0aa1400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0a9c100d20 0x7f0a9c19e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:54.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.130+0000 7f0aa1400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0a9c100d20 0x7f0a9c19e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49440/0 (socket says 172.21.15.97:49440) 2024-03-27T10:58:54.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.130+0000 7f0aa1400640 1 -- 172.21.15.97:0/2107573406 learned_addr learned my addr 172.21.15.97:0/2107573406 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:58:54.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.130+0000 7f0aa2800640 1 --2- 172.21.15.97:0/2107573406 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0a9c1016c0 0x7f0a9c1a5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:54.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.130+0000 7f0aa1e00640 1 --2- 172.21.15.97:0/2107573406 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0a9c1003c0 0x7f0a9c19e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:54.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.130+0000 7f0aa1400640 1 -- 172.21.15.97:0/2107573406 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0a9c1003c0 msgr2=0x7f0a9c19e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:54.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.130+0000 7f0aa1400640 1 --2- 172.21.15.97:0/2107573406 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0a9c1003c0 0x7f0a9c19e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:54.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.130+0000 7f0aa1400640 1 -- 172.21.15.97:0/2107573406 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0a9c1016c0 msgr2=0x7f0a9c1a5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:54.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.130+0000 7f0aa1400640 1 --2- 172.21.15.97:0/2107573406 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0a9c1016c0 0x7f0a9c1a5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:54.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.130+0000 7f0aa1400640 1 -- 172.21.15.97:0/2107573406 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a9c1a5fd0 con 0x7f0a9c100d20 2024-03-27T10:58:54.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.131+0000 7f0aa1400640 1 --2- 172.21.15.97:0/2107573406 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0a9c100d20 0x7f0a9c19e8f0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f0a9000e9f0 tx=0x7f0a9000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:54.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.131+0000 7f0a96a00640 1 -- 172.21.15.97:0/2107573406 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a9000cd80 con 0x7f0a9c100d20 2024-03-27T10:58:54.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.131+0000 7f0a96a00640 1 -- 172.21.15.97:0/2107573406 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0a900108e0 con 0x7f0a9c100d20 2024-03-27T10:58:54.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.131+0000 7f0aa4800640 1 -- 172.21.15.97:0/2107573406 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a9c1a62c0 con 0x7f0a9c100d20 2024-03-27T10:58:54.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.131+0000 7f0a96a00640 1 -- 172.21.15.97:0/2107573406 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a90018920 con 0x7f0a9c100d20 2024-03-27T10:58:54.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.131+0000 7f0aa4800640 1 -- 172.21.15.97:0/2107573406 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a9c1a6860 con 0x7f0a9c100d20 2024-03-27T10:58:54.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.135+0000 7f0a96a00640 1 -- 172.21.15.97:0/2107573406 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f0a90010a90 con 0x7f0a9c100d20 2024-03-27T10:58:54.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.135+0000 7f0aa7446740 1 Processor -- start 2024-03-27T10:58:54.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.136+0000 7f0aa7446740 1 -- start start 2024-03-27T10:58:54.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.136+0000 7f0aa7446740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559fc2e328c0 0x559fc2e32ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:54.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.136+0000 7f0aa7446740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559fc2e332b0 0x559fc2e37db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:54.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.136+0000 7f0aa7446740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559fc2e38420 0x559fc2e3a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:54.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.136+0000 7f0aa7446740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x559fc2dfa7b0 con 0x559fc2e332b0 2024-03-27T10:58:54.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.136+0000 7f0aa7446740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x559fc2d97670 con 0x559fc2e38420 2024-03-27T10:58:54.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.136+0000 7f0aa7446740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x559fc2d3f970 con 0x559fc2e328c0 2024-03-27T10:58:54.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.137+0000 7f0aa1400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559fc2e38420 0x559fc2e3a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:54.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.137+0000 7f0aa1e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559fc2e332b0 0x559fc2e37db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:54.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.137+0000 7f0aa1400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559fc2e38420 0x559fc2e3a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:60098/0 (socket says 172.21.15.97:60098) 2024-03-27T10:58:54.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.137+0000 7f0aa1400640 1 -- 172.21.15.97:0/2460528576 learned_addr learned my addr 172.21.15.97:0/2460528576 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:58:54.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.137+0000 7f0a96a00640 1 --2- 172.21.15.97:0/2107573406 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f0a840a3690 0x7f0a840a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:54.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.137+0000 7f0aa1e00640 1 --2- 172.21.15.97:0/2460528576 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559fc2e332b0 0x559fc2e37db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49456/0 (socket says 172.21.15.97:49456) 2024-03-27T10:58:54.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.137+0000 7f0aa2800640 1 --2- 172.21.15.97:0/2460528576 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559fc2e328c0 0x559fc2e32ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:54.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.137+0000 7f0a96a00640 1 -- 172.21.15.97:0/2107573406 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f0a900cfff0 con 0x7f0a9c100d20 2024-03-27T10:58:54.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.137+0000 7f0aa1e00640 1 -- 172.21.15.97:0/2460528576 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559fc2e328c0 msgr2=0x559fc2e32ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:54.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.138+0000 7f0aa1e00640 1 --2- 172.21.15.97:0/2460528576 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559fc2e328c0 0x559fc2e32ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:54.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.138+0000 7f0aa1e00640 1 -- 172.21.15.97:0/2460528576 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559fc2e38420 msgr2=0x559fc2e3a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:54.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.138+0000 7f0aa1e00640 1 --2- 172.21.15.97:0/2460528576 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559fc2e38420 0x559fc2e3a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:54.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.138+0000 7f0aa1e00640 1 -- 172.21.15.97:0/2460528576 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559fc2aab3f0 con 0x559fc2e332b0 2024-03-27T10:58:54.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.138+0000 7f0aa2800640 1 --2- 172.21.15.97:0/2107573406 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f0a840a3690 0x7f0a840a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:54.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.138+0000 7f0aa1400640 1 --2- 172.21.15.97:0/2460528576 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559fc2e38420 0x559fc2e3a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T10:58:54.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.139+0000 7f0aa1e00640 1 --2- 172.21.15.97:0/2460528576 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559fc2e332b0 0x559fc2e37db0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c02c190 tx=0x7f0a8c009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ec39ba85d349631 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:54.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.140+0000 7f0a8b000640 1 -- 172.21.15.97:0/2460528576 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a8c045020 con 0x559fc2e332b0 2024-03-27T10:58:54.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.140+0000 7f0a8b000640 1 -- 172.21.15.97:0/2460528576 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0a8c02ca40 con 0x559fc2e332b0 2024-03-27T10:58:54.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.140+0000 7f0a8b000640 1 -- 172.21.15.97:0/2460528576 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a8c0046c0 con 0x559fc2e332b0 2024-03-27T10:58:54.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.141+0000 7f0aa2800640 1 --2- 172.21.15.97:0/2107573406 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f0a840a3690 0x7f0a840a5b50 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f0a980098b0 tx=0x7f0a98004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:54.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.141+0000 7f0aa7446740 1 -- 172.21.15.97:0/2460528576 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559fc2e332b0 msgr2=0x559fc2e37db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:54.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.141+0000 7f0aa7446740 1 --2- 172.21.15.97:0/2460528576 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559fc2e332b0 0x559fc2e37db0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c02c190 tx=0x7f0a8c009560 comp rx=0 tx=0).stop 2024-03-27T10:58:54.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.141+0000 7f0aa7446740 1 -- 172.21.15.97:0/2460528576 shutdown_connections 2024-03-27T10:58:54.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.141+0000 7f0aa7446740 1 --2- 172.21.15.97:0/2460528576 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559fc2e38420 0x559fc2e3a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:54.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.141+0000 7f0aa7446740 1 --2- 172.21.15.97:0/2460528576 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559fc2e332b0 0x559fc2e37db0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:54.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.141+0000 7f0aa7446740 1 --2- 172.21.15.97:0/2460528576 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559fc2e328c0 0x559fc2e32ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:54.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.141+0000 7f0aa7446740 1 -- 172.21.15.97:0/2460528576 >> 172.21.15.97:0/2460528576 conn(0x559fc2e2de30 msgr2=0x559fc2e2e230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:54.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.141+0000 7f0aa7446740 1 -- 172.21.15.97:0/2460528576 shutdown_connections 2024-03-27T10:58:54.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.141+0000 7f0aa7446740 1 -- 172.21.15.97:0/2460528576 wait complete. 2024-03-27T10:58:54.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.142+0000 7f0aa7446740 1 -- start start 2024-03-27T10:58:54.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.142+0000 7f0aa7446740 1 Processor -- start 2024-03-27T10:58:54.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.143+0000 7f0aa7446740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559fc2e328c0 0x559fc2e3ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:54.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.143+0000 7f0aa7446740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559fc2e332b0 0x559fc2e3cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:54.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.143+0000 7f0aa7446740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559fc2e38420 0x559fc2e43fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:54.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.143+0000 7f0aa7446740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x559fc2d52ca0 con 0x559fc2e332b0 2024-03-27T10:58:54.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.143+0000 7f0aa7446740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x559fc2d93620 con 0x559fc2e38420 2024-03-27T10:58:54.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.143+0000 7f0aa7446740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x559fc2e3b370 con 0x559fc2e328c0 2024-03-27T10:58:54.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.143+0000 7f0aa7446740 10 client.0 _ll_register_callbacks cb 0x559fc2d8f160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:58:54.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.143+0000 7f0aa2800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559fc2e38420 0x559fc2e43fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:54.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.143+0000 7f0aa1400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559fc2e332b0 0x559fc2e3cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:54.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.144+0000 7f0aa2800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559fc2e38420 0x559fc2e43fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:60112/0 (socket says 172.21.15.97:60112) 2024-03-27T10:58:54.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.144+0000 7f0aa2800640 1 -- 172.21.15.97:0/4138964765 learned_addr learned my addr 172.21.15.97:0/4138964765 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:58:54.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.144+0000 7f0aa1e00640 1 --2- 172.21.15.97:0/4138964765 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559fc2e328c0 0x559fc2e3ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:54.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.144+0000 7f0aa1400640 1 -- 172.21.15.97:0/4138964765 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559fc2e328c0 msgr2=0x559fc2e3ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:54.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.144+0000 7f0aa1400640 1 --2- 172.21.15.97:0/4138964765 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559fc2e328c0 0x559fc2e3ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:54.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.144+0000 7f0aa1400640 1 -- 172.21.15.97:0/4138964765 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559fc2e38420 msgr2=0x559fc2e43fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:54.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.144+0000 7f0aa1400640 1 --2- 172.21.15.97:0/4138964765 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559fc2e38420 0x559fc2e43fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:54.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.144+0000 7f0aa1400640 1 -- 172.21.15.97:0/4138964765 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559fc2b757d0 con 0x559fc2e332b0 2024-03-27T10:58:54.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.144+0000 7f0aa1400640 1 --2- 172.21.15.97:0/4138964765 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559fc2e332b0 0x559fc2e3cf90 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f0a9009c4b0 tx=0x7f0a9009c150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7643dc6ce9a806c server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:54.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.145+0000 7f0a7f400640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T10:58:54.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.145+0000 7f0a7f400640 1 -- 172.21.15.97:0/4138964765 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a90010630 con 0x559fc2e332b0 2024-03-27T10:58:54.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.145+0000 7f0a7f400640 1 -- 172.21.15.97:0/4138964765 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f0a9009b970 con 0x559fc2e332b0 2024-03-27T10:58:54.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.145+0000 7f0aa7446740 1 -- 172.21.15.97:0/4138964765 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a8c02ca40 con 0x559fc2e332b0 2024-03-27T10:58:54.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.145+0000 7f0a7f400640 1 -- 172.21.15.97:0/4138964765 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f0a900a0820 con 0x559fc2e332b0 2024-03-27T10:58:54.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.145+0000 7f0aa7446740 10 client.16293 fetch_fsmap learned FSMap version 31 2024-03-27T10:58:54.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.145+0000 7f0aa7446740 1 -- 172.21.15.97:0/4138964765 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x559fc2e473b0 con 0x559fc2e332b0 2024-03-27T10:58:54.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.145+0000 7f0a7f400640 1 -- 172.21.15.97:0/4138964765 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0a8c02ca40 con 0x559fc2e332b0 2024-03-27T10:58:54.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.146+0000 7f0a7f400640 1 -- 172.21.15.97:0/4138964765 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 31) ==== 8370+0+0 (secure 0 0 0) 0x7f0a900d30b0 con 0x559fc2e332b0 2024-03-27T10:58:54.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.146+0000 7f0aa7446740 10 client.16293 fetch_fsmap finished waiting for FSMap version 31 2024-03-27T10:58:54.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.146+0000 7f0aa7446740 10 client.16293 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-03-27T10:58:54.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.146+0000 7f0aa7446740 20 client.16293 populate_metadata read hostname 'smithi097' 2024-03-27T10:58:54.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.146+0000 7f0aa7446740 1 --2- 172.21.15.97:0/4138964765 >> [v2:172.21.15.177:6834/3349112132,v1:172.21.15.177:6835/3349112132] conn(0x559fc2e48b20 0x559fc2e4af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:54.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.146+0000 7f0aa7446740 4 client.16293 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T10:58:54.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.146+0000 7f0aa7446740 1 -- 172.21.15.97:0/4138964765 --> [v2:172.21.15.177:6834/3349112132,v1:172.21.15.177:6835/3349112132] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559fc2da4bb0 con 0x559fc2e48b20 2024-03-27T10:58:54.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.146+0000 7f0aa1e00640 1 --2- 172.21.15.97:0/4138964765 >> [v2:172.21.15.177:6834/3349112132,v1:172.21.15.177:6835/3349112132] conn(0x559fc2e48b20 0x559fc2e4af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:54.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.147+0000 7f0aa1e00640 1 --2- 172.21.15.97:0/4138964765 >> [v2:172.21.15.177:6834/3349112132,v1:172.21.15.177:6835/3349112132] conn(0x559fc2e48b20 0x559fc2e4af70 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e43a15f8320bfded server_cookie=a03de95600523a4 in_seq=0 out_seq=0 2024-03-27T10:58:54.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.147+0000 7f0a7f400640 10 client.16293 ms_handle_connect on v2:172.21.15.177:6834/3349112132 2024-03-27T10:58:54.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.149+0000 7f0a7f400640 1 -- 172.21.15.97:0/4138964765 <== mds.3 v2:172.21.15.177:6834/3349112132 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x559fc2da4bb0 con 0x559fc2e48b20 2024-03-27T10:58:54.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.149+0000 7f0a7f400640 10 client.16293 handle_command_reply: tid=0 2024-03-27T10:58:54.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.149+0000 7f0aa7446740 1 client.16293 shutdown 2024-03-27T10:58:54.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.150+0000 7f0aa7446740 1 -- 172.21.15.97:0/4138964765 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559fc2e332b0 msgr2=0x559fc2e3cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:54.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.150+0000 7f0aa7446740 1 --2- 172.21.15.97:0/4138964765 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559fc2e332b0 0x559fc2e3cf90 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f0a9009c4b0 tx=0x7f0a9009c150 comp rx=0 tx=0).stop 2024-03-27T10:58:54.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.150+0000 7f0aa7446740 1 -- 172.21.15.97:0/4138964765 shutdown_connections 2024-03-27T10:58:54.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.150+0000 7f0aa7446740 1 --2- 172.21.15.97:0/4138964765 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559fc2e38420 0x559fc2e43fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:54.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.150+0000 7f0aa7446740 1 --2- 172.21.15.97:0/4138964765 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559fc2e332b0 0x559fc2e3cf90 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:54.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.150+0000 7f0aa7446740 1 --2- 172.21.15.97:0/4138964765 >> [v2:172.21.15.177:6834/3349112132,v1:172.21.15.177:6835/3349112132] conn(0x559fc2e48b20 0x559fc2e4af70 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:54.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.150+0000 7f0a7f400640 0 client.16293 ms_handle_reset on v2:172.21.15.177:6834/3349112132 2024-03-27T10:58:54.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.150+0000 7f0aa7446740 1 --2- 172.21.15.97:0/4138964765 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559fc2e328c0 0x559fc2e3ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:54.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.150+0000 7f0aa7446740 1 -- 172.21.15.97:0/4138964765 >> 172.21.15.97:0/4138964765 conn(0x559fc2e2de30 msgr2=0x559fc2e2d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:54.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.150+0000 7f0aa7446740 1 -- 172.21.15.97:0/4138964765 shutdown_connections 2024-03-27T10:58:54.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.150+0000 7f0aa7446740 1 -- 172.21.15.97:0/4138964765 wait complete. 2024-03-27T10:58:54.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.151+0000 7f0aa7446740 20 client.16293 trim_cache size 0 max 16384 2024-03-27T10:58:54.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.174+0000 7f0aa7446740 1 Processor -- start 2024-03-27T10:58:54.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.174+0000 7f0aa7446740 1 -- start start 2024-03-27T10:58:54.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.174+0000 7f0aa7446740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559fc2e328c0 0x559fc2e3bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:54.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.174+0000 7f0aa7446740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559fc2e332b0 0x559fc2e43f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:54.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.174+0000 7f0aa7446740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559fc2e38420 0x559fc2e317a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:54.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.174+0000 7f0aa7446740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0a9c1af510 con 0x559fc2e332b0 2024-03-27T10:58:54.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.174+0000 7f0aa7446740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f0a9c1b1110 con 0x559fc2e328c0 2024-03-27T10:58:54.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.174+0000 7f0aa7446740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f0a9c1afbe0 con 0x559fc2e38420 2024-03-27T10:58:54.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.175+0000 7f0aa1400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559fc2e332b0 0x559fc2e43f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:54.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.175+0000 7f0aa1400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559fc2e332b0 0x559fc2e43f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49476/0 (socket says 172.21.15.97:49476) 2024-03-27T10:58:54.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.175+0000 7f0aa1400640 1 -- 172.21.15.97:0/3221346684 learned_addr learned my addr 172.21.15.97:0/3221346684 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:58:54.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.175+0000 7f0aa1e00640 1 --2- 172.21.15.97:0/3221346684 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559fc2e328c0 0x559fc2e3bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:54.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.175+0000 7f0aa2800640 1 --2- 172.21.15.97:0/3221346684 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559fc2e38420 0x559fc2e317a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:54.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.175+0000 7f0aa1400640 1 -- 172.21.15.97:0/3221346684 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559fc2e38420 msgr2=0x559fc2e317a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:54.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.175+0000 7f0aa1400640 1 --2- 172.21.15.97:0/3221346684 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559fc2e38420 0x559fc2e317a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:54.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.175+0000 7f0aa1400640 1 -- 172.21.15.97:0/3221346684 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559fc2e328c0 msgr2=0x559fc2e3bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:54.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.175+0000 7f0aa1400640 1 --2- 172.21.15.97:0/3221346684 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559fc2e328c0 0x559fc2e3bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:54.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.175+0000 7f0aa1400640 1 -- 172.21.15.97:0/3221346684 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559fc2e29680 con 0x559fc2e332b0 2024-03-27T10:58:54.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.175+0000 7f0aa1400640 1 --2- 172.21.15.97:0/3221346684 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559fc2e332b0 0x559fc2e43f90 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f0a9000e9c0 tx=0x7f0a9009bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9e699ce1d60a19b server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:54.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.176+0000 7f0a8a600640 1 -- 172.21.15.97:0/3221346684 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a900d30d0 con 0x559fc2e332b0 2024-03-27T10:58:54.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.176+0000 7f0a8a600640 1 -- 172.21.15.97:0/3221346684 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0a9009b970 con 0x559fc2e332b0 2024-03-27T10:58:54.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.176+0000 7f0a8a600640 1 -- 172.21.15.97:0/3221346684 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a90085ab0 con 0x559fc2e332b0 2024-03-27T10:58:54.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.177+0000 7f0aa7446740 1 -- 172.21.15.97:0/3221346684 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559fc2e332b0 msgr2=0x559fc2e43f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:54.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.177+0000 7f0aa7446740 1 --2- 172.21.15.97:0/3221346684 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559fc2e332b0 0x559fc2e43f90 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f0a9000e9c0 tx=0x7f0a9009bf40 comp rx=0 tx=0).stop 2024-03-27T10:58:54.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.177+0000 7f0aa7446740 1 -- 172.21.15.97:0/3221346684 shutdown_connections 2024-03-27T10:58:54.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.177+0000 7f0aa7446740 1 --2- 172.21.15.97:0/3221346684 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559fc2e38420 0x559fc2e317a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:54.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.177+0000 7f0aa7446740 1 --2- 172.21.15.97:0/3221346684 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559fc2e332b0 0x559fc2e43f90 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:54.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.177+0000 7f0aa7446740 1 --2- 172.21.15.97:0/3221346684 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559fc2e328c0 0x559fc2e3bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:54.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.177+0000 7f0aa7446740 1 -- 172.21.15.97:0/3221346684 >> 172.21.15.97:0/3221346684 conn(0x559fc2e2de30 msgr2=0x559fc2e31090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:54.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.178+0000 7f0aa7446740 1 -- 172.21.15.97:0/3221346684 shutdown_connections 2024-03-27T10:58:54.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.178+0000 7f0aa7446740 1 -- 172.21.15.97:0/3221346684 wait complete. 2024-03-27T10:58:54.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.178+0000 7f0aa7446740 1 -- start start 2024-03-27T10:58:54.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.178+0000 7f0aa7446740 1 Processor -- start 2024-03-27T10:58:54.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.179+0000 7f0aa7446740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559fc2e328c0 0x559fc2e41710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:54.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.180+0000 7f0aa7446740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559fc2e332b0 0x559fc2e41c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:54.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.180+0000 7f0aa7446740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559fc2e38420 0x559fc2e42c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:54.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.180+0000 7f0aa7446740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0a9c1b0980 con 0x559fc2e328c0 2024-03-27T10:58:54.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.180+0000 7f0aa7446740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f0a9c1b02b0 con 0x559fc2e332b0 2024-03-27T10:58:54.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.180+0000 7f0aa7446740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f0a9c1ad4f0 con 0x559fc2e38420 2024-03-27T10:58:54.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.180+0000 7f0aa7446740 10 client.0 _ll_register_callbacks cb 0x7f0a9000cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:58:54.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.180+0000 7f0aa1e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559fc2e328c0 0x559fc2e41710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:54.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.180+0000 7f0aa1e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559fc2e328c0 0x559fc2e41710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49492/0 (socket says 172.21.15.97:49492) 2024-03-27T10:58:54.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.180+0000 7f0aa1e00640 1 -- 172.21.15.97:0/1667488056 learned_addr learned my addr 172.21.15.97:0/1667488056 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:58:54.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.180+0000 7f0aa1400640 1 --2- 172.21.15.97:0/1667488056 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559fc2e332b0 0x559fc2e41c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:54.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.181+0000 7f0aa1e00640 1 -- 172.21.15.97:0/1667488056 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559fc2e38420 msgr2=0x559fc2e42c60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-27T10:58:54.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.181+0000 7f0aa1e00640 1 --2- 172.21.15.97:0/1667488056 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559fc2e38420 0x559fc2e42c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:54.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.181+0000 7f0aa1e00640 1 -- 172.21.15.97:0/1667488056 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559fc2e332b0 msgr2=0x559fc2e41c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:54.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.181+0000 7f0aa1e00640 1 --2- 172.21.15.97:0/1667488056 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559fc2e332b0 0x559fc2e41c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:54.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.181+0000 7f0aa1e00640 1 -- 172.21.15.97:0/1667488056 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559fc2e43250 con 0x559fc2e328c0 2024-03-27T10:58:54.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.181+0000 7f0aa1e00640 1 --2- 172.21.15.97:0/1667488056 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559fc2e328c0 0x559fc2e41710 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c005ab0 tx=0x7f0a8c02bc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8e66d23fc7856f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:54.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.182+0000 7f0a7f400640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T10:58:54.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.182+0000 7f0a7f400640 1 -- 172.21.15.97:0/1667488056 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a8c045040 con 0x559fc2e328c0 2024-03-27T10:58:54.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.182+0000 7f0a7f400640 1 -- 172.21.15.97:0/1667488056 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f0a8c005690 con 0x559fc2e328c0 2024-03-27T10:58:54.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.182+0000 7f0aa7446740 1 -- 172.21.15.97:0/1667488056 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a9009b970 con 0x559fc2e328c0 2024-03-27T10:58:54.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.183+0000 7f0a7f400640 1 -- 172.21.15.97:0/1667488056 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f0a8c004240 con 0x559fc2e328c0 2024-03-27T10:58:54.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.183+0000 7f0aa7446740 10 client.16299 fetch_fsmap learned FSMap version 31 2024-03-27T10:58:54.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.183+0000 7f0aa7446740 1 -- 172.21.15.97:0/1667488056 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x559fc2eaee70 con 0x559fc2e328c0 2024-03-27T10:58:54.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.183+0000 7f0a7f400640 1 -- 172.21.15.97:0/1667488056 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0a9009b970 con 0x559fc2e328c0 2024-03-27T10:58:54.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.184+0000 7f0a7f400640 1 -- 172.21.15.97:0/1667488056 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 31) ==== 8370+0+0 (secure 0 0 0) 0x7f0a8c041480 con 0x559fc2e328c0 2024-03-27T10:58:54.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.184+0000 7f0aa7446740 10 client.16299 fetch_fsmap finished waiting for FSMap version 31 2024-03-27T10:58:54.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.184+0000 7f0aa7446740 10 client.16299 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-03-27T10:58:54.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.184+0000 7f0aa7446740 20 client.16299 populate_metadata read hostname 'smithi097' 2024-03-27T10:58:54.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.184+0000 7f0aa7446740 1 --2- 172.21.15.97:0/1667488056 >> [v2:172.21.15.177:6834/3349112132,v1:172.21.15.177:6835/3349112132] conn(0x559fc2eaf3d0 0x559fc2eb17c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:54.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.184+0000 7f0aa7446740 4 client.16299 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-27T10:58:54.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.184+0000 7f0aa7446740 1 -- 172.21.15.97:0/1667488056 --> [v2:172.21.15.177:6834/3349112132,v1:172.21.15.177:6835/3349112132] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559fc2d9ef00 con 0x559fc2eaf3d0 2024-03-27T10:58:54.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.184+0000 7f0aa1400640 1 --2- 172.21.15.97:0/1667488056 >> [v2:172.21.15.177:6834/3349112132,v1:172.21.15.177:6835/3349112132] conn(0x559fc2eaf3d0 0x559fc2eb17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:54.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.185+0000 7f0aa1400640 1 --2- 172.21.15.97:0/1667488056 >> [v2:172.21.15.177:6834/3349112132,v1:172.21.15.177:6835/3349112132] conn(0x559fc2eaf3d0 0x559fc2eb17c0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=aabf952c64b9a0b3 server_cookie=3b9fbb2378f00338 in_seq=0 out_seq=0 2024-03-27T10:58:54.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.185+0000 7f0a7f400640 10 client.16299 ms_handle_connect on v2:172.21.15.177:6834/3349112132 2024-03-27T10:58:54.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.186+0000 7f0a7f400640 1 -- 172.21.15.97:0/1667488056 <== mds.3 v2:172.21.15.177:6834/3349112132 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559fc2d9ef00 con 0x559fc2eaf3d0 2024-03-27T10:58:54.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.186+0000 7f0a7f400640 10 client.16299 handle_command_reply: tid=0 2024-03-27T10:58:54.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.186+0000 7f0aa7446740 1 client.16299 shutdown 2024-03-27T10:58:54.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.187+0000 7f0aa7446740 1 -- 172.21.15.97:0/1667488056 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559fc2e328c0 msgr2=0x559fc2e41710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:54.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.187+0000 7f0aa7446740 1 --2- 172.21.15.97:0/1667488056 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559fc2e328c0 0x559fc2e41710 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c005ab0 tx=0x7f0a8c02bc50 comp rx=0 tx=0).stop 2024-03-27T10:58:54.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.187+0000 7f0aa7446740 1 -- 172.21.15.97:0/1667488056 shutdown_connections 2024-03-27T10:58:54.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.187+0000 7f0aa7446740 1 --2- 172.21.15.97:0/1667488056 >> [v2:172.21.15.177:6834/3349112132,v1:172.21.15.177:6835/3349112132] conn(0x559fc2eaf3d0 0x559fc2eb17c0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:54.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.187+0000 7f0aa7446740 1 --2- 172.21.15.97:0/1667488056 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559fc2e38420 0x559fc2e42c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:54.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.187+0000 7f0aa7446740 1 --2- 172.21.15.97:0/1667488056 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559fc2e332b0 0x559fc2e41c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:54.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.187+0000 7f0aa7446740 1 --2- 172.21.15.97:0/1667488056 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559fc2e328c0 0x559fc2e41710 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:54.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.187+0000 7f0a7f400640 0 client.16299 ms_handle_reset on v2:172.21.15.177:6834/3349112132 2024-03-27T10:58:54.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.187+0000 7f0aa7446740 1 -- 172.21.15.97:0/1667488056 >> 172.21.15.97:0/1667488056 conn(0x559fc2e2de30 msgr2=0x559fc2e31920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:54.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.187+0000 7f0aa7446740 1 -- 172.21.15.97:0/1667488056 shutdown_connections 2024-03-27T10:58:54.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.187+0000 7f0aa7446740 1 -- 172.21.15.97:0/1667488056 wait complete. 2024-03-27T10:58:54.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.187+0000 7f0aa7446740 20 client.16299 trim_cache size 0 max 16384 2024-03-27T10:58:54.160 INFO:teuthology.orchestra.run.smithi097.stdout:[] 2024-03-27T10:58:54.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.188+0000 7f0aa4800640 1 -- 172.21.15.97:0/2107573406 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f0a840a3690 msgr2=0x7f0a840a5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:54.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.188+0000 7f0aa4800640 1 --2- 172.21.15.97:0/2107573406 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f0a840a3690 0x7f0a840a5b50 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f0a980098b0 tx=0x7f0a98004a40 comp rx=0 tx=0).stop 2024-03-27T10:58:54.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.188+0000 7f0aa4800640 1 -- 172.21.15.97:0/2107573406 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0a9c100d20 msgr2=0x7f0a9c19e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:54.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.188+0000 7f0aa4800640 1 --2- 172.21.15.97:0/2107573406 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0a9c100d20 0x7f0a9c19e8f0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f0a9000e9f0 tx=0x7f0a9000eeb0 comp rx=0 tx=0).stop 2024-03-27T10:58:54.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.189+0000 7f0aa4800640 1 -- 172.21.15.97:0/2107573406 shutdown_connections 2024-03-27T10:58:54.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.189+0000 7f0aa4800640 1 --2- 172.21.15.97:0/2107573406 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f0a840a3690 0x7f0a840a5b50 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:54.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.189+0000 7f0aa4800640 1 --2- 172.21.15.97:0/2107573406 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0a9c1016c0 0x7f0a9c1a5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:54.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.189+0000 7f0aa4800640 1 --2- 172.21.15.97:0/2107573406 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0a9c100d20 0x7f0a9c19e8f0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:54.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.189+0000 7f0aa4800640 1 --2- 172.21.15.97:0/2107573406 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0a9c1003c0 0x7f0a9c19e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:54.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.189+0000 7f0aa4800640 1 -- 172.21.15.97:0/2107573406 >> 172.21.15.97:0/2107573406 conn(0x7f0a9c0fbfe0 msgr2=0x7f0a9c0fd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:54.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.189+0000 7f0aa4800640 1 -- 172.21.15.97:0/2107573406 shutdown_connections 2024-03-27T10:58:54.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.189+0000 7f0aa4800640 1 -- 172.21.15.97:0/2107573406 wait complete. 2024-03-27T10:58:54.186 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-27T10:58:54.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.494+0000 7f5abaa00640 1 -- 172.21.15.97:0/3114155049 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5ab4107840 msgr2=0x7f5ab4109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:54.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.494+0000 7f5abaa00640 1 --2- 172.21.15.97:0/3114155049 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5ab4107840 0x7f5ab4109c90 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c009860 tx=0x7f5a9c02bb70 comp rx=0 tx=0).stop 2024-03-27T10:58:54.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.494+0000 7f5abaa00640 1 -- 172.21.15.97:0/3114155049 shutdown_connections 2024-03-27T10:58:54.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.494+0000 7f5abaa00640 1 --2- 172.21.15.97:0/3114155049 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5ab4107840 0x7f5ab4109c90 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:54.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.494+0000 7f5abaa00640 1 --2- 172.21.15.97:0/3114155049 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5ab4104f10 0x7f5ab4107300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:54.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.494+0000 7f5abaa00640 1 --2- 172.21.15.97:0/3114155049 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ab41025e0 0x7f5ab41049d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:54.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.494+0000 7f5abaa00640 1 -- 172.21.15.97:0/3114155049 >> 172.21.15.97:0/3114155049 conn(0x7f5ab40fbff0 msgr2=0x7f5ab40fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:54.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.495+0000 7f5abaa00640 1 -- 172.21.15.97:0/3114155049 shutdown_connections 2024-03-27T10:58:54.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.495+0000 7f5abaa00640 1 -- 172.21.15.97:0/3114155049 wait complete. 2024-03-27T10:58:54.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.496+0000 7f5abaa00640 1 Processor -- start 2024-03-27T10:58:54.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.497+0000 7f5abaa00640 1 -- start start 2024-03-27T10:58:54.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.497+0000 7f5abaa00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5ab41025e0 0x7f5ab419e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:54.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.497+0000 7f5abaa00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5ab4104f10 0x7f5ab419e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:54.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.497+0000 7f5abaa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ab4107840 0x7f5ab41a5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:54.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.497+0000 7f5abaa00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab4101ec0 con 0x7f5ab4107840 2024-03-27T10:58:54.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.497+0000 7f5abaa00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab4101d40 con 0x7f5ab41025e0 2024-03-27T10:58:54.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.497+0000 7f5abaa00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab4102040 con 0x7f5ab4104f10 2024-03-27T10:58:54.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.497+0000 7f5ab8a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ab4107840 0x7f5ab41a5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:54.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.498+0000 7f5ab8a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ab4107840 0x7f5ab41a5940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49504/0 (socket says 172.21.15.97:49504) 2024-03-27T10:58:54.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.498+0000 7f5ab8a00640 1 -- 172.21.15.97:0/1928694847 learned_addr learned my addr 172.21.15.97:0/1928694847 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:58:54.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.498+0000 7f5ab3e00640 1 --2- 172.21.15.97:0/1928694847 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5ab41025e0 0x7f5ab419e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:54.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.498+0000 7f5ab3e00640 1 -- 172.21.15.97:0/1928694847 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5ab4104f10 msgr2=0x7f5ab419e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:54.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.498+0000 7f5ab3e00640 1 --2- 172.21.15.97:0/1928694847 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5ab4104f10 0x7f5ab419e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:54.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.498+0000 7f5ab3e00640 1 -- 172.21.15.97:0/1928694847 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ab4107840 msgr2=0x7f5ab41a5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:54.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.498+0000 7f5ab3e00640 1 --2- 172.21.15.97:0/1928694847 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ab4107840 0x7f5ab41a5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:54.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.498+0000 7f5ab3e00640 1 -- 172.21.15.97:0/1928694847 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ab41a5fb0 con 0x7f5ab41025e0 2024-03-27T10:58:54.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.499+0000 7f5ab3e00640 1 --2- 172.21.15.97:0/1928694847 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5ab41025e0 0x7f5ab419e390 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5aac00eb00 tx=0x7f5aac00c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:54.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.499+0000 7f5ab0c00640 1 -- 172.21.15.97:0/1928694847 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aac010430 con 0x7f5ab41025e0 2024-03-27T10:58:54.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.500+0000 7f5abaa00640 1 -- 172.21.15.97:0/1928694847 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ab41a62a0 con 0x7f5ab41025e0 2024-03-27T10:58:54.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.500+0000 7f5abaa00640 1 -- 172.21.15.97:0/1928694847 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ab41a6840 con 0x7f5ab41025e0 2024-03-27T10:58:54.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.500+0000 7f5ab0c00640 1 -- 172.21.15.97:0/1928694847 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5aac010a80 con 0x7f5ab41025e0 2024-03-27T10:58:54.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.500+0000 7f5ab0c00640 1 -- 172.21.15.97:0/1928694847 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aac018af0 con 0x7f5ab41025e0 2024-03-27T10:58:54.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.504+0000 7f5abd7c1740 1 Processor -- start 2024-03-27T10:58:54.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.504+0000 7f5abd7c1740 1 -- start start 2024-03-27T10:58:54.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.504+0000 7f5ab0c00640 1 -- 172.21.15.97:0/1928694847 <== mon.2 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f5aac018c90 con 0x7f5ab41025e0 2024-03-27T10:58:54.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.504+0000 7f5abd7c1740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c50ead78c0 0x55c50ead7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:54.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.504+0000 7f5abd7c1740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c50ead82b0 0x55c50eadcdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:54.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.504+0000 7f5abd7c1740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c50eadd420 0x55c50eadf8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:54.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.504+0000 7f5abd7c1740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55c50ea9f7b0 con 0x55c50ead82b0 2024-03-27T10:58:54.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.504+0000 7f5abd7c1740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55c50ea3c670 con 0x55c50ead78c0 2024-03-27T10:58:54.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.504+0000 7f5abd7c1740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55c50e9e4970 con 0x55c50eadd420 2024-03-27T10:58:54.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.505+0000 7f5ab3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c50ead82b0 0x55c50eadcdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:54.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.505+0000 7f5ab3400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c50eadd420 0x55c50eadf8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:54.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.505+0000 7f5ab8a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c50ead78c0 0x55c50ead7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:54.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.505+0000 7f5ab3400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c50eadd420 0x55c50eadf8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.97:56938/0 (socket says 172.21.15.97:56938) 2024-03-27T10:58:54.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.505+0000 7f5ab3400640 1 -- 172.21.15.97:0/3995897933 learned_addr learned my addr 172.21.15.97:0/3995897933 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:58:54.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.505+0000 7f5ab3e00640 1 -- 172.21.15.97:0/3995897933 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c50eadd420 msgr2=0x55c50eadf8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:54.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.505+0000 7f5ab3e00640 1 --2- 172.21.15.97:0/3995897933 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c50eadd420 0x55c50eadf8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:54.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.505+0000 7f5ab3e00640 1 -- 172.21.15.97:0/3995897933 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c50ead78c0 msgr2=0x55c50ead7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:54.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.505+0000 7f5ab3e00640 1 --2- 172.21.15.97:0/3995897933 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c50ead78c0 0x55c50ead7ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:54.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.505+0000 7f5ab3e00640 1 -- 172.21.15.97:0/3995897933 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c50e7503f0 con 0x55c50ead82b0 2024-03-27T10:58:54.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.505+0000 7f5ab3400640 1 --2- 172.21.15.97:0/3995897933 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c50eadd420 0x55c50eadf8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T10:58:54.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.506+0000 7f5ab3e00640 1 --2- 172.21.15.97:0/3995897933 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c50ead82b0 0x55c50eadcdb0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f5aac0ce4a0 tx=0x7f5aac0d82a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29f0f6f96e09bef1 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:54.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.506+0000 7f5ab0c00640 1 --2- 172.21.15.97:0/1928694847 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f5a980a3690 0x7f5a980a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:54.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.506+0000 7f5aa9200640 1 -- 172.21.15.97:0/3995897933 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aac0df070 con 0x55c50ead82b0 2024-03-27T10:58:54.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.506+0000 7f5ab0c00640 1 -- 172.21.15.97:0/1928694847 <== mon.2 v2:172.21.15.114:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f5aac0d0a50 con 0x7f5ab41025e0 2024-03-27T10:58:54.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.506+0000 7f5ab8a00640 1 --2- 172.21.15.97:0/1928694847 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f5a980a3690 0x7f5a980a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:54.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.506+0000 7f5aa9200640 1 -- 172.21.15.97:0/3995897933 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5aac09d5d0 con 0x55c50ead82b0 2024-03-27T10:58:54.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.507+0000 7f5aa9200640 1 -- 172.21.15.97:0/3995897933 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aac09ca10 con 0x55c50ead82b0 2024-03-27T10:58:54.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.507+0000 7f5abd7c1740 1 -- 172.21.15.97:0/3995897933 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c50ead82b0 msgr2=0x55c50eadcdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:54.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.507+0000 7f5ab8a00640 1 --2- 172.21.15.97:0/1928694847 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f5a980a3690 0x7f5a980a5b50 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c005e00 tx=0x7f5a9c0056d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:54.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.507+0000 7f5abd7c1740 1 --2- 172.21.15.97:0/3995897933 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c50ead82b0 0x55c50eadcdb0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f5aac0ce4a0 tx=0x7f5aac0d82a0 comp rx=0 tx=0).stop 2024-03-27T10:58:54.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.507+0000 7f5abd7c1740 1 -- 172.21.15.97:0/3995897933 shutdown_connections 2024-03-27T10:58:54.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.507+0000 7f5abd7c1740 1 --2- 172.21.15.97:0/3995897933 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c50eadd420 0x55c50eadf8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:54.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.507+0000 7f5abd7c1740 1 --2- 172.21.15.97:0/3995897933 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c50ead82b0 0x55c50eadcdb0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:54.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.508+0000 7f5abd7c1740 1 --2- 172.21.15.97:0/3995897933 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c50ead78c0 0x55c50ead7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:54.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.508+0000 7f5abd7c1740 1 -- 172.21.15.97:0/3995897933 >> 172.21.15.97:0/3995897933 conn(0x55c50ead2e30 msgr2=0x55c50ead3230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:54.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.508+0000 7f5abd7c1740 1 -- 172.21.15.97:0/3995897933 shutdown_connections 2024-03-27T10:58:54.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.508+0000 7f5abd7c1740 1 -- 172.21.15.97:0/3995897933 wait complete. 2024-03-27T10:58:54.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.508+0000 7f5abd7c1740 1 -- start start 2024-03-27T10:58:54.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.509+0000 7f5abd7c1740 1 Processor -- start 2024-03-27T10:58:54.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.509+0000 7f5abd7c1740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c50ead78c0 0x55c50eae1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:54.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.509+0000 7f5abd7c1740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c50ead82b0 0x55c50eae1f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:54.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.509+0000 7f5abd7c1740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c50eadd420 0x55c50eae8f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:54.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.509+0000 7f5abd7c1740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55c50e9f7ca0 con 0x55c50eadd420 2024-03-27T10:58:54.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.509+0000 7f5abd7c1740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55c50ea38620 con 0x55c50ead78c0 2024-03-27T10:58:54.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.509+0000 7f5abd7c1740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55c50eae0420 con 0x55c50ead82b0 2024-03-27T10:58:54.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.509+0000 7f5abd7c1740 10 client.0 _ll_register_callbacks cb 0x55c50ea34160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:58:54.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.509+0000 7f5ab8a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c50eadd420 0x55c50eae8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:54.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.509+0000 7f5ab8a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c50eadd420 0x55c50eae8f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49534/0 (socket says 172.21.15.97:49534) 2024-03-27T10:58:54.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.510+0000 7f5ab8a00640 1 -- 172.21.15.97:0/1790346479 learned_addr learned my addr 172.21.15.97:0/1790346479 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:58:54.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.510+0000 7f5ab3e00640 1 --2- 172.21.15.97:0/1790346479 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c50ead78c0 0x55c50eae1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:54.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.510+0000 7f5ab3400640 1 --2- 172.21.15.97:0/1790346479 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c50ead82b0 0x55c50eae1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:54.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.510+0000 7f5ab8a00640 1 -- 172.21.15.97:0/1790346479 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c50ead82b0 msgr2=0x55c50eae1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:54.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.510+0000 7f5ab8a00640 1 --2- 172.21.15.97:0/1790346479 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c50ead82b0 0x55c50eae1f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:54.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.510+0000 7f5ab8a00640 1 -- 172.21.15.97:0/1790346479 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c50ead78c0 msgr2=0x55c50eae1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:54.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.510+0000 7f5ab8a00640 1 --2- 172.21.15.97:0/1790346479 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c50ead78c0 0x55c50eae1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:54.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.510+0000 7f5ab8a00640 1 -- 172.21.15.97:0/1790346479 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c50e81a7d0 con 0x55c50eadd420 2024-03-27T10:58:54.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.510+0000 7f5ab3400640 1 --2- 172.21.15.97:0/1790346479 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c50ead82b0 0x55c50eae1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:58:54.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.510+0000 7f5ab8a00640 1 --2- 172.21.15.97:0/1790346479 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c50eadd420 0x55c50eae8f90 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c009780 tx=0x7f5a9c02bb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55191629ef994407 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:54.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.511+0000 7f5aa1600640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T10:58:54.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.511+0000 7f5aa1600640 1 -- 172.21.15.97:0/1790346479 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a9c04f020 con 0x55c50eadd420 2024-03-27T10:58:54.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.511+0000 7f5aa1600640 1 -- 172.21.15.97:0/1790346479 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f5a9c02c9e0 con 0x55c50eadd420 2024-03-27T10:58:54.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.511+0000 7f5abd7c1740 1 -- 172.21.15.97:0/1790346479 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5aac09d5d0 con 0x55c50eadd420 2024-03-27T10:58:54.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.511+0000 7f5aa1600640 1 -- 172.21.15.97:0/1790346479 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f5a9c039040 con 0x55c50eadd420 2024-03-27T10:58:54.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.511+0000 7f5abd7c1740 10 client.16311 fetch_fsmap learned FSMap version 31 2024-03-27T10:58:54.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.511+0000 7f5abd7c1740 1 -- 172.21.15.97:0/1790346479 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c50eaec3a0 con 0x55c50eadd420 2024-03-27T10:58:54.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.512+0000 7f5aa1600640 1 -- 172.21.15.97:0/1790346479 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5aac09d5d0 con 0x55c50eadd420 2024-03-27T10:58:54.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.512+0000 7f5aa1600640 1 -- 172.21.15.97:0/1790346479 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 31) ==== 8370+0+0 (secure 0 0 0) 0x7f5a9c03f550 con 0x55c50eadd420 2024-03-27T10:58:54.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.512+0000 7f5abd7c1740 10 client.16311 fetch_fsmap finished waiting for FSMap version 31 2024-03-27T10:58:54.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.512+0000 7f5abd7c1740 10 client.16311 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-27T10:58:54.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.512+0000 7f5abd7c1740 20 client.16311 populate_metadata read hostname 'smithi097' 2024-03-27T10:58:54.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.512+0000 7f5abd7c1740 1 --2- 172.21.15.97:0/1790346479 >> [v2:172.21.15.114:6834/3581918222,v1:172.21.15.114:6835/3581918222] conn(0x55c50eaedaf0 0x55c50eaeff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:54.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.512+0000 7f5abd7c1740 4 client.16311 mds_command: new command op to 34445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T10:58:54.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.512+0000 7f5abd7c1740 1 -- 172.21.15.97:0/1790346479 --> [v2:172.21.15.114:6834/3581918222,v1:172.21.15.114:6835/3581918222] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c50ea49bb0 con 0x55c50eaedaf0 2024-03-27T10:58:54.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.512+0000 7f5ab3e00640 1 --2- 172.21.15.97:0/1790346479 >> [v2:172.21.15.114:6834/3581918222,v1:172.21.15.114:6835/3581918222] conn(0x55c50eaedaf0 0x55c50eaeff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:54.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.513+0000 7f5ab3e00640 1 --2- 172.21.15.97:0/1790346479 >> [v2:172.21.15.114:6834/3581918222,v1:172.21.15.114:6835/3581918222] conn(0x55c50eaedaf0 0x55c50eaeff70 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7514d5b38163676 server_cookie=72873a744e0240e6 in_seq=0 out_seq=0 2024-03-27T10:58:54.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.513+0000 7f5aa1600640 10 client.16311 ms_handle_connect on v2:172.21.15.114:6834/3581918222 2024-03-27T10:58:54.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.514+0000 7f5aa1600640 1 -- 172.21.15.97:0/1790346479 <== mds.2 v2:172.21.15.114:6834/3581918222 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55c50ea49bb0 con 0x55c50eaedaf0 2024-03-27T10:58:54.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.514+0000 7f5aa1600640 10 client.16311 handle_command_reply: tid=0 2024-03-27T10:58:54.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.514+0000 7f5abd7c1740 1 client.16311 shutdown 2024-03-27T10:58:54.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.515+0000 7f5abd7c1740 1 -- 172.21.15.97:0/1790346479 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c50eadd420 msgr2=0x55c50eae8f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:54.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.515+0000 7f5abd7c1740 1 --2- 172.21.15.97:0/1790346479 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c50eadd420 0x55c50eae8f90 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c009780 tx=0x7f5a9c02bb70 comp rx=0 tx=0).stop 2024-03-27T10:58:54.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.515+0000 7f5abd7c1740 1 -- 172.21.15.97:0/1790346479 shutdown_connections 2024-03-27T10:58:54.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.515+0000 7f5abd7c1740 1 --2- 172.21.15.97:0/1790346479 >> [v2:172.21.15.114:6834/3581918222,v1:172.21.15.114:6835/3581918222] conn(0x55c50eaedaf0 0x55c50eaeff70 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:54.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.515+0000 7f5abd7c1740 1 --2- 172.21.15.97:0/1790346479 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c50eadd420 0x55c50eae8f90 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:54.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.515+0000 7f5aa1600640 0 client.16311 ms_handle_reset on v2:172.21.15.114:6834/3581918222 2024-03-27T10:58:54.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.515+0000 7f5abd7c1740 1 --2- 172.21.15.97:0/1790346479 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c50ead82b0 0x55c50eae1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:54.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.515+0000 7f5abd7c1740 1 --2- 172.21.15.97:0/1790346479 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c50ead78c0 0x55c50eae1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:54.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.515+0000 7f5abd7c1740 1 -- 172.21.15.97:0/1790346479 >> 172.21.15.97:0/1790346479 conn(0x55c50ead2e30 msgr2=0x55c50ead2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:54.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.515+0000 7f5abd7c1740 1 -- 172.21.15.97:0/1790346479 shutdown_connections 2024-03-27T10:58:54.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.515+0000 7f5abd7c1740 1 -- 172.21.15.97:0/1790346479 wait complete. 2024-03-27T10:58:54.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.515+0000 7f5abd7c1740 20 client.16311 trim_cache size 0 max 16384 2024-03-27T10:58:54.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.538+0000 7f5abd7c1740 1 Processor -- start 2024-03-27T10:58:54.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.539+0000 7f5abd7c1740 1 -- start start 2024-03-27T10:58:54.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.539+0000 7f5abd7c1740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c50ead78c0 0x55c50ead17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:54.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.539+0000 7f5abd7c1740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c50ead82b0 0x55c50eae8f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:54.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.539+0000 7f5abd7c1740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c50eadd420 0x55c50ead1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:54.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.539+0000 7f5abd7c1740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5a9000e430 con 0x55c50eadd420 2024-03-27T10:58:54.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.539+0000 7f5abd7c1740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f5a90010030 con 0x55c50ead78c0 2024-03-27T10:58:54.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.539+0000 7f5abd7c1740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f5a9000eb00 con 0x55c50ead82b0 2024-03-27T10:58:54.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.540+0000 7f5ab3400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c50ead82b0 0x55c50eae8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:54.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.540+0000 7f5ab3e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c50ead78c0 0x55c50ead17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:54.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.540+0000 7f5ab3400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c50ead82b0 0x55c50eae8f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.97:56942/0 (socket says 172.21.15.97:56942) 2024-03-27T10:58:54.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.540+0000 7f5ab3400640 1 -- 172.21.15.97:0/1131860860 learned_addr learned my addr 172.21.15.97:0/1131860860 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:58:54.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.540+0000 7f5ab8a00640 1 --2- 172.21.15.97:0/1131860860 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c50eadd420 0x55c50ead1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:54.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.541+0000 7f5ab3400640 1 -- 172.21.15.97:0/1131860860 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c50ead78c0 msgr2=0x55c50ead17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:54.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.541+0000 7f5ab3400640 1 --2- 172.21.15.97:0/1131860860 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c50ead78c0 0x55c50ead17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:54.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.541+0000 7f5ab3400640 1 -- 172.21.15.97:0/1131860860 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c50eadd420 msgr2=0x55c50ead1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:54.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.541+0000 7f5ab3400640 1 --2- 172.21.15.97:0/1131860860 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c50eadd420 0x55c50ead1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:54.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.541+0000 7f5ab3400640 1 -- 172.21.15.97:0/1131860860 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c50ead8690 con 0x55c50ead82b0 2024-03-27T10:58:54.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.541+0000 7f5ab3e00640 1 --2- 172.21.15.97:0/1131860860 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c50ead78c0 0x55c50ead17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T10:58:54.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.541+0000 7f5ab8a00640 1 --2- 172.21.15.97:0/1131860860 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c50eadd420 0x55c50ead1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T10:58:54.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.541+0000 7f5ab3400640 1 --2- 172.21.15.97:0/1131860860 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c50ead82b0 0x55c50eae8f50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5aa4009c80 tx=0x7f5aa4013830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=706eae27231e07d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:54.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.542+0000 7f5aa3e00640 1 -- 172.21.15.97:0/1131860860 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa4020070 con 0x55c50ead82b0 2024-03-27T10:58:54.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.542+0000 7f5aa3e00640 1 -- 172.21.15.97:0/1131860860 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5aa4009e30 con 0x55c50ead82b0 2024-03-27T10:58:54.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.542+0000 7f5aa3e00640 1 -- 172.21.15.97:0/1131860860 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa401b780 con 0x55c50ead82b0 2024-03-27T10:58:54.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.543+0000 7f5abd7c1740 1 -- 172.21.15.97:0/1131860860 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c50ead82b0 msgr2=0x55c50eae8f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:54.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.543+0000 7f5abd7c1740 1 --2- 172.21.15.97:0/1131860860 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c50ead82b0 0x55c50eae8f50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5aa4009c80 tx=0x7f5aa4013830 comp rx=0 tx=0).stop 2024-03-27T10:58:54.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.543+0000 7f5abd7c1740 1 -- 172.21.15.97:0/1131860860 shutdown_connections 2024-03-27T10:58:54.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.543+0000 7f5abd7c1740 1 --2- 172.21.15.97:0/1131860860 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c50eadd420 0x55c50ead1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:54.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.543+0000 7f5abd7c1740 1 --2- 172.21.15.97:0/1131860860 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c50ead82b0 0x55c50eae8f50 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:54.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.543+0000 7f5abd7c1740 1 --2- 172.21.15.97:0/1131860860 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c50ead78c0 0x55c50ead17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:54.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.543+0000 7f5abd7c1740 1 -- 172.21.15.97:0/1131860860 >> 172.21.15.97:0/1131860860 conn(0x55c50ead2e30 msgr2=0x55c50ead6920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:54.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.543+0000 7f5abd7c1740 1 -- 172.21.15.97:0/1131860860 shutdown_connections 2024-03-27T10:58:54.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.543+0000 7f5abd7c1740 1 -- 172.21.15.97:0/1131860860 wait complete. 2024-03-27T10:58:54.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.543+0000 7f5abd7c1740 1 -- start start 2024-03-27T10:58:54.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.544+0000 7f5abd7c1740 1 Processor -- start 2024-03-27T10:58:54.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.544+0000 7f5abd7c1740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c50ead78c0 0x55c50eae67b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:54.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.544+0000 7f5abd7c1740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c50ead82b0 0x55c50eae6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:54.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.544+0000 7f5abd7c1740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c50eadd420 0x55c50eae7d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:54.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.544+0000 7f5abd7c1740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5a9000f8a0 con 0x55c50eadd420 2024-03-27T10:58:54.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.544+0000 7f5abd7c1740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f5a9000f1d0 con 0x55c50ead82b0 2024-03-27T10:58:54.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.544+0000 7f5abd7c1740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f5a9000c410 con 0x55c50ead78c0 2024-03-27T10:58:54.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.544+0000 7f5abd7c1740 10 client.0 _ll_register_callbacks cb 0x7f5aa4005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:58:54.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.545+0000 7f5ab8a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c50eadd420 0x55c50eae7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:54.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.545+0000 7f5ab3e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c50ead78c0 0x55c50eae67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:54.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.545+0000 7f5ab8a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c50eadd420 0x55c50eae7d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49552/0 (socket says 172.21.15.97:49552) 2024-03-27T10:58:54.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.545+0000 7f5ab8a00640 1 -- 172.21.15.97:0/476253654 learned_addr learned my addr 172.21.15.97:0/476253654 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:58:54.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.545+0000 7f5ab3400640 1 --2- 172.21.15.97:0/476253654 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c50ead82b0 0x55c50eae6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:54.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.545+0000 7f5ab8a00640 1 -- 172.21.15.97:0/476253654 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c50ead78c0 msgr2=0x55c50eae67b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:54.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.545+0000 7f5ab8a00640 1 --2- 172.21.15.97:0/476253654 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c50ead78c0 0x55c50eae67b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:54.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.545+0000 7f5ab8a00640 1 -- 172.21.15.97:0/476253654 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c50ead82b0 msgr2=0x55c50eae6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:54.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.545+0000 7f5ab8a00640 1 --2- 172.21.15.97:0/476253654 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c50ead82b0 0x55c50eae6cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:54.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.545+0000 7f5ab8a00640 1 -- 172.21.15.97:0/476253654 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c50eae82f0 con 0x55c50eadd420 2024-03-27T10:58:54.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.546+0000 7f5ab8a00640 1 --2- 172.21.15.97:0/476253654 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c50eadd420 0x55c50eae7d00 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c034840 tx=0x7f5a9c03f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee35696eb067cbeb server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:54.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.546+0000 7f5aa1600640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T10:58:54.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.546+0000 7f5aa1600640 1 -- 172.21.15.97:0/476253654 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a9c063060 con 0x55c50eadd420 2024-03-27T10:58:54.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.546+0000 7f5aa1600640 1 -- 172.21.15.97:0/476253654 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f5a9c0035f0 con 0x55c50eadd420 2024-03-27T10:58:54.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.546+0000 7f5abd7c1740 1 -- 172.21.15.97:0/476253654 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5aa4009e30 con 0x55c50eadd420 2024-03-27T10:58:54.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.547+0000 7f5aa1600640 1 -- 172.21.15.97:0/476253654 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f5a9c04f040 con 0x55c50eadd420 2024-03-27T10:58:54.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.547+0000 7f5abd7c1740 10 client.16317 fetch_fsmap learned FSMap version 31 2024-03-27T10:58:54.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.547+0000 7f5abd7c1740 1 -- 172.21.15.97:0/476253654 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c50eb54030 con 0x55c50eadd420 2024-03-27T10:58:54.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.547+0000 7f5aa1600640 1 -- 172.21.15.97:0/476253654 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5aa4009e30 con 0x55c50eadd420 2024-03-27T10:58:54.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.547+0000 7f5aa1600640 1 -- 172.21.15.97:0/476253654 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 31) ==== 8370+0+0 (secure 0 0 0) 0x7f5a9c05d030 con 0x55c50eadd420 2024-03-27T10:58:54.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.547+0000 7f5abd7c1740 10 client.16317 fetch_fsmap finished waiting for FSMap version 31 2024-03-27T10:58:54.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.547+0000 7f5abd7c1740 10 client.16317 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-27T10:58:54.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.547+0000 7f5abd7c1740 20 client.16317 populate_metadata read hostname 'smithi097' 2024-03-27T10:58:54.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.547+0000 7f5abd7c1740 1 --2- 172.21.15.97:0/476253654 >> [v2:172.21.15.114:6834/3581918222,v1:172.21.15.114:6835/3581918222] conn(0x55c50eb54a60 0x55c50eb56e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:54.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.547+0000 7f5abd7c1740 4 client.16317 mds_command: new command op to 34445 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-27T10:58:54.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.547+0000 7f5abd7c1740 1 -- 172.21.15.97:0/476253654 --> [v2:172.21.15.114:6834/3581918222,v1:172.21.15.114:6835/3581918222] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c50ea43f00 con 0x55c50eb54a60 2024-03-27T10:58:54.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.548+0000 7f5ab3e00640 1 --2- 172.21.15.97:0/476253654 >> [v2:172.21.15.114:6834/3581918222,v1:172.21.15.114:6835/3581918222] conn(0x55c50eb54a60 0x55c50eb56e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:54.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.549+0000 7f5ab3e00640 1 --2- 172.21.15.97:0/476253654 >> [v2:172.21.15.114:6834/3581918222,v1:172.21.15.114:6835/3581918222] conn(0x55c50eb54a60 0x55c50eb56e80 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d334304241cb63db server_cookie=718346b2fd21c56c in_seq=0 out_seq=0 2024-03-27T10:58:54.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.549+0000 7f5aa1600640 10 client.16317 ms_handle_connect on v2:172.21.15.114:6834/3581918222 2024-03-27T10:58:54.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.549+0000 7f5aa1600640 1 -- 172.21.15.97:0/476253654 <== mds.2 v2:172.21.15.114:6834/3581918222 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c50ea43f00 con 0x55c50eb54a60 2024-03-27T10:58:54.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.549+0000 7f5aa1600640 10 client.16317 handle_command_reply: tid=0 2024-03-27T10:58:54.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.549+0000 7f5abd7c1740 1 client.16317 shutdown 2024-03-27T10:58:54.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.549+0000 7f5abd7c1740 1 -- 172.21.15.97:0/476253654 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c50eadd420 msgr2=0x55c50eae7d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:54.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.549+0000 7f5abd7c1740 1 --2- 172.21.15.97:0/476253654 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c50eadd420 0x55c50eae7d00 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c034840 tx=0x7f5a9c03f000 comp rx=0 tx=0).stop 2024-03-27T10:58:54.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.550+0000 7f5abd7c1740 1 -- 172.21.15.97:0/476253654 shutdown_connections 2024-03-27T10:58:54.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.550+0000 7f5abd7c1740 1 --2- 172.21.15.97:0/476253654 >> [v2:172.21.15.114:6834/3581918222,v1:172.21.15.114:6835/3581918222] conn(0x55c50eb54a60 0x55c50eb56e80 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:54.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.550+0000 7f5abd7c1740 1 --2- 172.21.15.97:0/476253654 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c50eadd420 0x55c50eae7d00 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:54.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.550+0000 7f5abd7c1740 1 --2- 172.21.15.97:0/476253654 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c50ead82b0 0x55c50eae6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:54.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.550+0000 7f5abd7c1740 1 --2- 172.21.15.97:0/476253654 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c50ead78c0 0x55c50eae67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:54.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.550+0000 7f5abd7c1740 1 -- 172.21.15.97:0/476253654 >> 172.21.15.97:0/476253654 conn(0x55c50ead2e30 msgr2=0x55c50eae0850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:54.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.550+0000 7f5aa1600640 0 client.16317 ms_handle_reset on v2:172.21.15.114:6834/3581918222 2024-03-27T10:58:54.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.550+0000 7f5abd7c1740 1 -- 172.21.15.97:0/476253654 shutdown_connections 2024-03-27T10:58:54.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.550+0000 7f5abd7c1740 1 -- 172.21.15.97:0/476253654 wait complete. 2024-03-27T10:58:54.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.550+0000 7f5abd7c1740 20 client.16317 trim_cache size 0 max 16384 2024-03-27T10:58:54.525 INFO:teuthology.orchestra.run.smithi097.stdout:[] 2024-03-27T10:58:54.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.551+0000 7f5abaa00640 1 -- 172.21.15.97:0/1928694847 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f5a980a3690 msgr2=0x7f5a980a5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:54.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.551+0000 7f5abaa00640 1 --2- 172.21.15.97:0/1928694847 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f5a980a3690 0x7f5a980a5b50 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c005e00 tx=0x7f5a9c0056d0 comp rx=0 tx=0).stop 2024-03-27T10:58:54.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.551+0000 7f5abaa00640 1 -- 172.21.15.97:0/1928694847 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5ab41025e0 msgr2=0x7f5ab419e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:54.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.551+0000 7f5abaa00640 1 --2- 172.21.15.97:0/1928694847 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5ab41025e0 0x7f5ab419e390 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5aac00eb00 tx=0x7f5aac00c2d0 comp rx=0 tx=0).stop 2024-03-27T10:58:54.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.551+0000 7f5abaa00640 1 -- 172.21.15.97:0/1928694847 shutdown_connections 2024-03-27T10:58:54.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.551+0000 7f5abaa00640 1 --2- 172.21.15.97:0/1928694847 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ab4107840 0x7f5ab41a5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:54.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.551+0000 7f5abaa00640 1 --2- 172.21.15.97:0/1928694847 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5ab4104f10 0x7f5ab419e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:54.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.551+0000 7f5abaa00640 1 --2- 172.21.15.97:0/1928694847 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f5a980a3690 0x7f5a980a5b50 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:54.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.551+0000 7f5abaa00640 1 --2- 172.21.15.97:0/1928694847 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5ab41025e0 0x7f5ab419e390 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:54.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.551+0000 7f5abaa00640 1 -- 172.21.15.97:0/1928694847 >> 172.21.15.97:0/1928694847 conn(0x7f5ab40fbff0 msgr2=0x7f5ab4102f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:54.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.551+0000 7f5abaa00640 1 -- 172.21.15.97:0/1928694847 shutdown_connections 2024-03-27T10:58:54.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.551+0000 7f5abaa00640 1 -- 172.21.15.97:0/1928694847 wait complete. 2024-03-27T10:58:54.540 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:58:54 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/1384333746' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T10:58:54.540 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:58:54 smithi097 ceph-mon[31859]: pgmap v509: 97 pgs: 97 active+clean; 5.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 125 MiB/s wr, 76 op/s 2024-03-27T10:58:54.542 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-27T10:58:54.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:58:54 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/1384333746' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T10:58:54.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:58:54 smithi177 ceph-mon[35294]: pgmap v509: 97 pgs: 97 active+clean; 5.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 125 MiB/s wr, 76 op/s 2024-03-27T10:58:54.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.742+0000 7f7913e00640 1 -- 172.21.15.97:0/4186129863 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7914109c20 msgr2=0x7f79141104b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:54.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.742+0000 7f7913e00640 1 --2- 172.21.15.97:0/4186129863 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7914109c20 0x7f79141104b0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f7904009900 tx=0x7f790402bc30 comp rx=0 tx=0).stop 2024-03-27T10:58:54.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.742+0000 7f7913e00640 1 -- 172.21.15.97:0/4186129863 shutdown_connections 2024-03-27T10:58:54.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.742+0000 7f7913e00640 1 --2- 172.21.15.97:0/4186129863 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7914109c20 0x7f79141104b0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:54.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.742+0000 7f7913e00640 1 --2- 172.21.15.97:0/4186129863 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7914109260 0x7f79141096e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:54.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.742+0000 7f7913e00640 1 --2- 172.21.15.97:0/4186129863 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7914108070 0x7f7914108490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:54.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.742+0000 7f7913e00640 1 -- 172.21.15.97:0/4186129863 >> 172.21.15.97:0/4186129863 conn(0x7f7914077020 msgr2=0x7f7914079440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:54.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.742+0000 7f7913e00640 1 -- 172.21.15.97:0/4186129863 shutdown_connections 2024-03-27T10:58:54.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.742+0000 7f7913e00640 1 -- 172.21.15.97:0/4186129863 wait complete. 2024-03-27T10:58:54.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.743+0000 7f7913e00640 1 Processor -- start 2024-03-27T10:58:54.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.743+0000 7f7913e00640 1 -- start start 2024-03-27T10:58:54.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.743+0000 7f7913e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7914108070 0x7f791419e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:54.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.743+0000 7f7913e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7914109260 0x7f791419e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:54.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.743+0000 7f7913e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7914109c20 0x7f79141a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:54.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.743+0000 7f7913e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7914113c00 con 0x7f7914108070 2024-03-27T10:58:54.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.743+0000 7f7913e00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f7914113a80 con 0x7f7914109c20 2024-03-27T10:58:54.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.743+0000 7f7913e00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f7914113d80 con 0x7f7914109260 2024-03-27T10:58:54.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.744+0000 7f7912a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7914108070 0x7f791419e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:54.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.744+0000 7f7912a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7914108070 0x7f791419e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49568/0 (socket says 172.21.15.97:49568) 2024-03-27T10:58:54.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.744+0000 7f7912a00640 1 -- 172.21.15.97:0/3897900656 learned_addr learned my addr 172.21.15.97:0/3897900656 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:58:54.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.744+0000 7f7913400640 1 --2- 172.21.15.97:0/3897900656 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7914109c20 0x7f79141a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:54.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.744+0000 7f7912000640 1 --2- 172.21.15.97:0/3897900656 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7914109260 0x7f791419e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:54.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.744+0000 7f7912a00640 1 -- 172.21.15.97:0/3897900656 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7914109260 msgr2=0x7f791419e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:54.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.744+0000 7f7912a00640 1 --2- 172.21.15.97:0/3897900656 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7914109260 0x7f791419e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:54.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.744+0000 7f7912a00640 1 -- 172.21.15.97:0/3897900656 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7914109c20 msgr2=0x7f79141a5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:54.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.744+0000 7f7912a00640 1 --2- 172.21.15.97:0/3897900656 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7914109c20 0x7f79141a5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:54.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.744+0000 7f7912a00640 1 -- 172.21.15.97:0/3897900656 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79141a5e00 con 0x7f7914108070 2024-03-27T10:58:54.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.744+0000 7f7913400640 1 --2- 172.21.15.97:0/3897900656 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7914109c20 0x7f79141a5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T10:58:54.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.744+0000 7f7912a00640 1 --2- 172.21.15.97:0/3897900656 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7914108070 0x7f791419e1e0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f790c00ea80 tx=0x7f790c00ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:54.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.745+0000 7f790b400640 1 -- 172.21.15.97:0/3897900656 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f790c00ce40 con 0x7f7914108070 2024-03-27T10:58:54.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.745+0000 7f790b400640 1 -- 172.21.15.97:0/3897900656 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f790c011920 con 0x7f7914108070 2024-03-27T10:58:54.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.745+0000 7f790b400640 1 -- 172.21.15.97:0/3897900656 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f790c01aaa0 con 0x7f7914108070 2024-03-27T10:58:54.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.745+0000 7f7913e00640 1 -- 172.21.15.97:0/3897900656 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79141a60f0 con 0x7f7914108070 2024-03-27T10:58:54.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.745+0000 7f7913e00640 1 -- 172.21.15.97:0/3897900656 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79141a6690 con 0x7f7914108070 2024-03-27T10:58:54.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.748+0000 7f791c430740 1 Processor -- start 2024-03-27T10:58:54.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.748+0000 7f790b400640 1 -- 172.21.15.97:0/3897900656 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f790c011ad0 con 0x7f7914108070 2024-03-27T10:58:54.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.748+0000 7f791c430740 1 -- start start 2024-03-27T10:58:54.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.748+0000 7f791c430740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55893f3a0ab0 0x55893f3a0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:54.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.748+0000 7f791c430740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55893f3a14a0 0x55893f3a5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:54.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.748+0000 7f791c430740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55893f3a6610 0x55893f3a8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:54.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.748+0000 7f791c430740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55893f3687b0 con 0x55893f3a14a0 2024-03-27T10:58:54.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.748+0000 7f791c430740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55893f305670 con 0x55893f3a0ab0 2024-03-27T10:58:54.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.748+0000 7f791c430740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55893f2ad970 con 0x55893f3a6610 2024-03-27T10:58:54.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.749+0000 7f7912a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55893f3a14a0 0x55893f3a5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:54.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.749+0000 7f7912a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55893f3a14a0 0x55893f3a5fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49572/0 (socket says 172.21.15.97:49572) 2024-03-27T10:58:54.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.749+0000 7f7912a00640 1 -- 172.21.15.97:0/1641455351 learned_addr learned my addr 172.21.15.97:0/1641455351 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:58:54.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.749+0000 7f7913400640 1 --2- 172.21.15.97:0/1641455351 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55893f3a0ab0 0x55893f3a0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:54.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.749+0000 7f7912000640 1 --2- 172.21.15.97:0/1641455351 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55893f3a6610 0x55893f3a8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:54.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.749+0000 7f7912a00640 1 -- 172.21.15.97:0/1641455351 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55893f3a6610 msgr2=0x55893f3a8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:54.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.749+0000 7f7912a00640 1 --2- 172.21.15.97:0/1641455351 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55893f3a6610 0x55893f3a8aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:54.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.749+0000 7f7912a00640 1 -- 172.21.15.97:0/1641455351 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55893f3a0ab0 msgr2=0x55893f3a0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:54.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.749+0000 7f7912a00640 1 --2- 172.21.15.97:0/1641455351 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55893f3a0ab0 0x55893f3a0ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:54.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.749+0000 7f7912a00640 1 -- 172.21.15.97:0/1641455351 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55893f0193f0 con 0x55893f3a14a0 2024-03-27T10:58:54.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.749+0000 7f790b400640 1 --2- 172.21.15.97:0/3897900656 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f78f80a3690 0x7f78f80a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:54.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.750+0000 7f7913400640 1 --2- 172.21.15.97:0/3897900656 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f78f80a3690 0x7f78f80a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:54.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.750+0000 7f790b400640 1 -- 172.21.15.97:0/3897900656 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f790c0d0340 con 0x7f7914108070 2024-03-27T10:58:54.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.750+0000 7f7912a00640 1 --2- 172.21.15.97:0/1641455351 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55893f3a14a0 0x55893f3a5fa0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f790c0d8510 tx=0x7f790c0d89d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7088817862095f server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:54.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.750+0000 7f7903e00640 1 -- 172.21.15.97:0/1641455351 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f790c09c5d0 con 0x55893f3a14a0 2024-03-27T10:58:54.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.750+0000 7f7903e00640 1 -- 172.21.15.97:0/1641455351 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f790c011e00 con 0x55893f3a14a0 2024-03-27T10:58:54.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.750+0000 7f7903e00640 1 -- 172.21.15.97:0/1641455351 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f790c09c160 con 0x55893f3a14a0 2024-03-27T10:58:54.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.751+0000 7f791c430740 1 -- 172.21.15.97:0/1641455351 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55893f3a14a0 msgr2=0x55893f3a5fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:54.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.751+0000 7f791c430740 1 --2- 172.21.15.97:0/1641455351 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55893f3a14a0 0x55893f3a5fa0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f790c0d8510 tx=0x7f790c0d89d0 comp rx=0 tx=0).stop 2024-03-27T10:58:54.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.751+0000 7f7913400640 1 --2- 172.21.15.97:0/3897900656 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f78f80a3690 0x7f78f80a5b50 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f7904006010 tx=0x7f7904005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:54.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.751+0000 7f791c430740 1 -- 172.21.15.97:0/1641455351 shutdown_connections 2024-03-27T10:58:54.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.751+0000 7f791c430740 1 --2- 172.21.15.97:0/1641455351 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55893f3a6610 0x55893f3a8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:54.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.751+0000 7f791c430740 1 --2- 172.21.15.97:0/1641455351 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55893f3a14a0 0x55893f3a5fa0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:54.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.751+0000 7f791c430740 1 --2- 172.21.15.97:0/1641455351 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55893f3a0ab0 0x55893f3a0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:54.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.751+0000 7f791c430740 1 -- 172.21.15.97:0/1641455351 >> 172.21.15.97:0/1641455351 conn(0x55893f39c020 msgr2=0x55893f39c420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:54.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.751+0000 7f791c430740 1 -- 172.21.15.97:0/1641455351 shutdown_connections 2024-03-27T10:58:54.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.751+0000 7f791c430740 1 -- 172.21.15.97:0/1641455351 wait complete. 2024-03-27T10:58:54.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.751+0000 7f791c430740 1 -- start start 2024-03-27T10:58:54.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.751+0000 7f791c430740 1 Processor -- start 2024-03-27T10:58:54.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.752+0000 7f791c430740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55893f3a0ab0 0x55893f3aac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:54.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.752+0000 7f791c430740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55893f3a14a0 0x55893f3ab1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:54.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.752+0000 7f791c430740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55893f3a6610 0x55893f3b21d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:54.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.752+0000 7f791c430740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55893f2c0ca0 con 0x55893f3a14a0 2024-03-27T10:58:54.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.752+0000 7f791c430740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55893f301620 con 0x55893f3a0ab0 2024-03-27T10:58:54.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.752+0000 7f791c430740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55893f3a95c0 con 0x55893f3a6610 2024-03-27T10:58:54.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.752+0000 7f791c430740 10 client.0 _ll_register_callbacks cb 0x55893f2fd160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:58:54.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.752+0000 7f7912000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55893f3a14a0 0x55893f3ab1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:54.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.752+0000 7f7912000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55893f3a14a0 0x55893f3ab1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49584/0 (socket says 172.21.15.97:49584) 2024-03-27T10:58:54.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.752+0000 7f7912000640 1 -- 172.21.15.97:0/3483557971 learned_addr learned my addr 172.21.15.97:0/3483557971 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:58:54.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.752+0000 7f7912a00640 1 --2- 172.21.15.97:0/3483557971 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55893f3a0ab0 0x55893f3aac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:54.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.752+0000 7f7913400640 1 --2- 172.21.15.97:0/3483557971 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55893f3a6610 0x55893f3b21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:54.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.752+0000 7f7912000640 1 -- 172.21.15.97:0/3483557971 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55893f3a6610 msgr2=0x55893f3b21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:54.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.752+0000 7f7912000640 1 --2- 172.21.15.97:0/3483557971 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55893f3a6610 0x55893f3b21d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:54.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.752+0000 7f7912000640 1 -- 172.21.15.97:0/3483557971 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55893f3a0ab0 msgr2=0x55893f3aac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:54.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.752+0000 7f7912000640 1 --2- 172.21.15.97:0/3483557971 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55893f3a0ab0 0x55893f3aac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:54.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.753+0000 7f7912000640 1 -- 172.21.15.97:0/3483557971 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55893f0e37d0 con 0x55893f3a14a0 2024-03-27T10:58:54.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.753+0000 7f7912000640 1 --2- 172.21.15.97:0/3483557971 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55893f3a14a0 0x55893f3ab1d0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f78fc010870 tx=0x7f78fc010d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d70e0664575b0fb server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:54.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.753+0000 7f7900c00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T10:58:54.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.753+0000 7f7900c00640 1 -- 172.21.15.97:0/3483557971 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78fc00cd20 con 0x55893f3a14a0 2024-03-27T10:58:54.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.753+0000 7f7900c00640 1 -- 172.21.15.97:0/3483557971 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f78fc01c480 con 0x55893f3a14a0 2024-03-27T10:58:54.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.753+0000 7f791c430740 1 -- 172.21.15.97:0/3483557971 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55893f312bb0 con 0x55893f3a14a0 2024-03-27T10:58:54.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.753+0000 7f7900c00640 1 -- 172.21.15.97:0/3483557971 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f78fc01c680 con 0x55893f3a14a0 2024-03-27T10:58:54.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.753+0000 7f791c430740 10 client.16329 fetch_fsmap learned FSMap version 31 2024-03-27T10:58:54.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.753+0000 7f791c430740 1 -- 172.21.15.97:0/3483557971 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55893f3b5630 con 0x55893f3a14a0 2024-03-27T10:58:54.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.754+0000 7f7900c00640 1 -- 172.21.15.97:0/3483557971 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55893f312bb0 con 0x55893f3a14a0 2024-03-27T10:58:54.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.754+0000 7f7900c00640 1 -- 172.21.15.97:0/3483557971 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 31) ==== 8370+0+0 (secure 0 0 0) 0x7f78fc011890 con 0x55893f3a14a0 2024-03-27T10:58:54.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.754+0000 7f791c430740 10 client.16329 fetch_fsmap finished waiting for FSMap version 31 2024-03-27T10:58:54.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.754+0000 7f791c430740 10 client.16329 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-27T10:58:54.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.754+0000 7f791c430740 20 client.16329 populate_metadata read hostname 'smithi097' 2024-03-27T10:58:54.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.754+0000 7f791c430740 1 --2- 172.21.15.97:0/3483557971 >> [v2:172.21.15.114:6836/616775588,v1:172.21.15.114:6837/616775588] conn(0x55893f3b6cc0 0x55893f3b9150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:54.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.754+0000 7f791c430740 4 client.16329 mds_command: new command op to 34478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T10:58:54.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.754+0000 7f791c430740 1 -- 172.21.15.97:0/3483557971 --> [v2:172.21.15.114:6836/616775588,v1:172.21.15.114:6837/616775588] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55893f30cf00 con 0x55893f3b6cc0 2024-03-27T10:58:54.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.754+0000 7f7912a00640 1 --2- 172.21.15.97:0/3483557971 >> [v2:172.21.15.114:6836/616775588,v1:172.21.15.114:6837/616775588] conn(0x55893f3b6cc0 0x55893f3b9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:54.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.755+0000 7f7900c00640 10 client.16329 ms_handle_connect on v2:172.21.15.114:6836/616775588 2024-03-27T10:58:54.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.755+0000 7f7912a00640 1 --2- 172.21.15.97:0/3483557971 >> [v2:172.21.15.114:6836/616775588,v1:172.21.15.114:6837/616775588] conn(0x55893f3b6cc0 0x55893f3b9150 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c04edce0f1d61024 server_cookie=d7242411aeeb1060 in_seq=0 out_seq=0 2024-03-27T10:58:54.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.757+0000 7f7900c00640 1 -- 172.21.15.97:0/3483557971 <== mds.1 v2:172.21.15.114:6836/616775588 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55893f30cf00 con 0x55893f3b6cc0 2024-03-27T10:58:54.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.757+0000 7f7900c00640 10 client.16329 handle_command_reply: tid=0 2024-03-27T10:58:54.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.757+0000 7f791c430740 1 client.16329 shutdown 2024-03-27T10:58:54.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.757+0000 7f791c430740 1 -- 172.21.15.97:0/3483557971 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55893f3a14a0 msgr2=0x55893f3ab1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:54.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.757+0000 7f791c430740 1 --2- 172.21.15.97:0/3483557971 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55893f3a14a0 0x55893f3ab1d0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f78fc010870 tx=0x7f78fc010d30 comp rx=0 tx=0).stop 2024-03-27T10:58:54.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.757+0000 7f791c430740 1 -- 172.21.15.97:0/3483557971 shutdown_connections 2024-03-27T10:58:54.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.757+0000 7f791c430740 1 --2- 172.21.15.97:0/3483557971 >> [v2:172.21.15.114:6836/616775588,v1:172.21.15.114:6837/616775588] conn(0x55893f3b6cc0 0x55893f3b9150 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:54.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.757+0000 7f791c430740 1 --2- 172.21.15.97:0/3483557971 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55893f3a6610 0x55893f3b21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:54.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.757+0000 7f7900c00640 0 client.16329 ms_handle_reset on v2:172.21.15.114:6836/616775588 2024-03-27T10:58:54.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.757+0000 7f791c430740 1 --2- 172.21.15.97:0/3483557971 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55893f3a14a0 0x55893f3ab1d0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:54.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.757+0000 7f791c430740 1 --2- 172.21.15.97:0/3483557971 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55893f3a0ab0 0x55893f3aac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:54.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.757+0000 7f791c430740 1 -- 172.21.15.97:0/3483557971 >> 172.21.15.97:0/3483557971 conn(0x55893f39c020 msgr2=0x55893f39b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:54.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.757+0000 7f791c430740 1 -- 172.21.15.97:0/3483557971 shutdown_connections 2024-03-27T10:58:54.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.757+0000 7f791c430740 1 -- 172.21.15.97:0/3483557971 wait complete. 2024-03-27T10:58:54.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.758+0000 7f791c430740 20 client.16329 trim_cache size 0 max 16384 2024-03-27T10:58:54.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.771+0000 7f791c430740 1 Processor -- start 2024-03-27T10:58:54.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.771+0000 7f791c430740 1 -- start start 2024-03-27T10:58:54.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.772+0000 7f791c430740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55893f3a0ab0 0x55893f3a2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:54.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.772+0000 7f791c430740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55893f3a14a0 0x55893f3a2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:54.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.772+0000 7f791c430740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55893f3a6610 0x55893f3b2190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:54.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.772+0000 7f791c430740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f79141af360 con 0x55893f3a6610 2024-03-27T10:58:54.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.772+0000 7f791c430740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f79141b0f60 con 0x55893f3a0ab0 2024-03-27T10:58:54.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.772+0000 7f791c430740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f79141afa30 con 0x55893f3a14a0 2024-03-27T10:58:54.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.772+0000 7f7913400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55893f3a6610 0x55893f3b2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:54.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.772+0000 7f7913400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55893f3a6610 0x55893f3b2190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49598/0 (socket says 172.21.15.97:49598) 2024-03-27T10:58:54.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.772+0000 7f7913400640 1 -- 172.21.15.97:0/3759970432 learned_addr learned my addr 172.21.15.97:0/3759970432 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:58:54.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.772+0000 7f7912a00640 1 --2- 172.21.15.97:0/3759970432 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55893f3a0ab0 0x55893f3a2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:54.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.772+0000 7f7912000640 1 --2- 172.21.15.97:0/3759970432 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55893f3a14a0 0x55893f3a2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:54.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.772+0000 7f7913400640 1 -- 172.21.15.97:0/3759970432 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55893f3a14a0 msgr2=0x55893f3a2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:54.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.772+0000 7f7913400640 1 --2- 172.21.15.97:0/3759970432 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55893f3a14a0 0x55893f3a2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:54.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.772+0000 7f7913400640 1 -- 172.21.15.97:0/3759970432 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55893f3a0ab0 msgr2=0x55893f3a2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:54.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.772+0000 7f7913400640 1 --2- 172.21.15.97:0/3759970432 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55893f3a0ab0 0x55893f3a2860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:54.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.772+0000 7f7913400640 1 -- 172.21.15.97:0/3759970432 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55893f3aa830 con 0x55893f3a6610 2024-03-27T10:58:54.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.773+0000 7f7913400640 1 --2- 172.21.15.97:0/3759970432 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55893f3a6610 0x55893f3b2190 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f790402c240 tx=0x7f7904040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52ceb7cf37d1ac05 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:54.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.773+0000 7f7903400640 1 -- 172.21.15.97:0/3759970432 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7904050070 con 0x55893f3a6610 2024-03-27T10:58:54.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.773+0000 7f7903400640 1 -- 172.21.15.97:0/3759970432 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f79040054f0 con 0x55893f3a6610 2024-03-27T10:58:54.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.773+0000 7f7903400640 1 -- 172.21.15.97:0/3759970432 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79040035f0 con 0x55893f3a6610 2024-03-27T10:58:54.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.773+0000 7f791c430740 1 -- 172.21.15.97:0/3759970432 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55893f3a6610 msgr2=0x55893f3b2190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:54.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.773+0000 7f791c430740 1 --2- 172.21.15.97:0/3759970432 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55893f3a6610 0x55893f3b2190 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f790402c240 tx=0x7f7904040000 comp rx=0 tx=0).stop 2024-03-27T10:58:54.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.774+0000 7f791c430740 1 -- 172.21.15.97:0/3759970432 shutdown_connections 2024-03-27T10:58:54.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.774+0000 7f791c430740 1 --2- 172.21.15.97:0/3759970432 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55893f3a6610 0x55893f3b2190 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:54.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.774+0000 7f791c430740 1 --2- 172.21.15.97:0/3759970432 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55893f3a14a0 0x55893f3a2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:54.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.774+0000 7f791c430740 1 --2- 172.21.15.97:0/3759970432 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55893f3a0ab0 0x55893f3a2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:54.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.774+0000 7f791c430740 1 -- 172.21.15.97:0/3759970432 >> 172.21.15.97:0/3759970432 conn(0x55893f39c020 msgr2=0x55893f3aac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:54.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.774+0000 7f791c430740 1 -- 172.21.15.97:0/3759970432 shutdown_connections 2024-03-27T10:58:54.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.774+0000 7f791c430740 1 -- 172.21.15.97:0/3759970432 wait complete. 2024-03-27T10:58:54.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.774+0000 7f791c430740 1 -- start start 2024-03-27T10:58:54.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.774+0000 7f791c430740 1 Processor -- start 2024-03-27T10:58:54.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.775+0000 7f791c430740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55893f3a0ab0 0x55893f3af650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:54.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.775+0000 7f791c430740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55893f3a14a0 0x55893f3afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:54.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.775+0000 7f791c430740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55893f3a6610 0x55893f3b0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:54.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.775+0000 7f791c430740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f79141b07d0 con 0x55893f3a0ab0 2024-03-27T10:58:54.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.775+0000 7f791c430740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f79141b0100 con 0x55893f3a14a0 2024-03-27T10:58:54.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.775+0000 7f791c430740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f79141ad340 con 0x55893f3a6610 2024-03-27T10:58:54.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.775+0000 7f791c430740 10 client.0 _ll_register_callbacks cb 0x7f78fc00e430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:58:54.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.775+0000 7f7912a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55893f3a0ab0 0x55893f3af650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:54.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.775+0000 7f7912a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55893f3a0ab0 0x55893f3af650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49610/0 (socket says 172.21.15.97:49610) 2024-03-27T10:58:54.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.775+0000 7f7912a00640 1 -- 172.21.15.97:0/3110291426 learned_addr learned my addr 172.21.15.97:0/3110291426 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:58:54.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.775+0000 7f7912000640 1 --2- 172.21.15.97:0/3110291426 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55893f3a14a0 0x55893f3afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:54.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.775+0000 7f7913400640 1 --2- 172.21.15.97:0/3110291426 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55893f3a6610 0x55893f3b0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:54.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.775+0000 7f7912a00640 1 -- 172.21.15.97:0/3110291426 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55893f3a6610 msgr2=0x55893f3b0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:54.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.775+0000 7f7912a00640 1 --2- 172.21.15.97:0/3110291426 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55893f3a6610 0x55893f3b0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:54.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.775+0000 7f7912a00640 1 -- 172.21.15.97:0/3110291426 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55893f3a14a0 msgr2=0x55893f3afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:54.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.775+0000 7f7912a00640 1 --2- 172.21.15.97:0/3110291426 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55893f3a14a0 0x55893f3afb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:54.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.775+0000 7f7912a00640 1 -- 172.21.15.97:0/3110291426 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55893f3b10f0 con 0x55893f3a0ab0 2024-03-27T10:58:54.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.775+0000 7f7912a00640 1 --2- 172.21.15.97:0/3110291426 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55893f3a0ab0 0x55893f3af650 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f790c0d8ed0 tx=0x7f790c0d5040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18accbb40e3058fe server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:54.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.775+0000 7f7900c00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T10:58:54.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.776+0000 7f7900c00640 1 -- 172.21.15.97:0/3110291426 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f790c09c340 con 0x55893f3a0ab0 2024-03-27T10:58:54.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.776+0000 7f7900c00640 1 -- 172.21.15.97:0/3110291426 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f790c0870d0 con 0x55893f3a0ab0 2024-03-27T10:58:54.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.776+0000 7f791c430740 1 -- 172.21.15.97:0/3110291426 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79040054f0 con 0x55893f3a0ab0 2024-03-27T10:58:54.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.776+0000 7f7900c00640 1 -- 172.21.15.97:0/3110291426 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f790c09c160 con 0x55893f3a0ab0 2024-03-27T10:58:54.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.776+0000 7f791c430740 10 client.16335 fetch_fsmap learned FSMap version 31 2024-03-27T10:58:54.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.776+0000 7f791c430740 1 -- 172.21.15.97:0/3110291426 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55893f3b2460 con 0x55893f3a0ab0 2024-03-27T10:58:54.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.776+0000 7f7900c00640 1 -- 172.21.15.97:0/3110291426 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f79040054f0 con 0x55893f3a0ab0 2024-03-27T10:58:54.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.776+0000 7f7900c00640 1 -- 172.21.15.97:0/3110291426 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 31) ==== 8370+0+0 (secure 0 0 0) 0x7f790c0cf040 con 0x55893f3a0ab0 2024-03-27T10:58:54.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.776+0000 7f791c430740 10 client.16335 fetch_fsmap finished waiting for FSMap version 31 2024-03-27T10:58:54.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.776+0000 7f791c430740 10 client.16335 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-27T10:58:54.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.776+0000 7f791c430740 20 client.16335 populate_metadata read hostname 'smithi097' 2024-03-27T10:58:54.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.776+0000 7f791c430740 1 --2- 172.21.15.97:0/3110291426 >> [v2:172.21.15.114:6836/616775588,v1:172.21.15.114:6837/616775588] conn(0x55893f41d2d0 0x55893f41f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:54.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.776+0000 7f791c430740 4 client.16335 mds_command: new command op to 34478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-27T10:58:54.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.776+0000 7f791c430740 1 -- 172.21.15.97:0/3110291426 --> [v2:172.21.15.114:6836/616775588,v1:172.21.15.114:6837/616775588] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55893f28a560 con 0x55893f41d2d0 2024-03-27T10:58:54.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.777+0000 7f7912000640 1 --2- 172.21.15.97:0/3110291426 >> [v2:172.21.15.114:6836/616775588,v1:172.21.15.114:6837/616775588] conn(0x55893f41d2d0 0x55893f41f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:54.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.777+0000 7f7912000640 1 --2- 172.21.15.97:0/3110291426 >> [v2:172.21.15.114:6836/616775588,v1:172.21.15.114:6837/616775588] conn(0x55893f41d2d0 0x55893f41f6c0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6b5352bba96d9308 server_cookie=f4e2debae191e062 in_seq=0 out_seq=0 2024-03-27T10:58:54.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.777+0000 7f7900c00640 10 client.16335 ms_handle_connect on v2:172.21.15.114:6836/616775588 2024-03-27T10:58:54.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.778+0000 7f7900c00640 1 -- 172.21.15.97:0/3110291426 <== mds.1 v2:172.21.15.114:6836/616775588 1 ==== command_reply(tid 0: 0 ) ==== 8+0+431 (crc 0 0 0) 0x55893f28a560 con 0x55893f41d2d0 2024-03-27T10:58:54.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.778+0000 7f7900c00640 10 client.16335 handle_command_reply: tid=0 2024-03-27T10:58:54.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.778+0000 7f791c430740 1 client.16335 shutdown 2024-03-27T10:58:54.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.778+0000 7f791c430740 1 -- 172.21.15.97:0/3110291426 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55893f3a0ab0 msgr2=0x55893f3af650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:54.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.778+0000 7f791c430740 1 --2- 172.21.15.97:0/3110291426 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55893f3a0ab0 0x55893f3af650 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f790c0d8ed0 tx=0x7f790c0d5040 comp rx=0 tx=0).stop 2024-03-27T10:58:54.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.778+0000 7f791c430740 1 -- 172.21.15.97:0/3110291426 shutdown_connections 2024-03-27T10:58:54.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.778+0000 7f791c430740 1 --2- 172.21.15.97:0/3110291426 >> [v2:172.21.15.114:6836/616775588,v1:172.21.15.114:6837/616775588] conn(0x55893f41d2d0 0x55893f41f6c0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:54.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.778+0000 7f791c430740 1 --2- 172.21.15.97:0/3110291426 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55893f3a6610 0x55893f3b0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:54.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.778+0000 7f7900c00640 0 client.16335 ms_handle_reset on v2:172.21.15.114:6836/616775588 2024-03-27T10:58:54.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.778+0000 7f791c430740 1 --2- 172.21.15.97:0/3110291426 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55893f3a14a0 0x55893f3afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:54.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.778+0000 7f791c430740 1 --2- 172.21.15.97:0/3110291426 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55893f3a0ab0 0x55893f3af650 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:54.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.778+0000 7f791c430740 1 -- 172.21.15.97:0/3110291426 >> 172.21.15.97:0/3110291426 conn(0x55893f39c020 msgr2=0x55893f3aac70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:54.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.778+0000 7f791c430740 1 -- 172.21.15.97:0/3110291426 shutdown_connections 2024-03-27T10:58:54.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.778+0000 7f791c430740 1 -- 172.21.15.97:0/3110291426 wait complete. 2024-03-27T10:58:54.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.778+0000 7f791c430740 20 client.16335 trim_cache size 0 max 16384 2024-03-27T10:58:54.758 INFO:teuthology.orchestra.run.smithi097.stdout:[ 2024-03-27T10:58:54.759 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T10:58:54.759 INFO:teuthology.orchestra.run.smithi097.stdout: "damage_type": "backtrace", 2024-03-27T10:58:54.759 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 2849809169, 2024-03-27T10:58:54.759 INFO:teuthology.orchestra.run.smithi097.stdout: "ino": 1099511628292, 2024-03-27T10:58:54.759 INFO:teuthology.orchestra.run.smithi097.stdout: "path": "/volumes/qa/sv_0/22bc04cd-8106-4d11-83f1-2aead5019d07/client.0/tmp/iogen_3.1p0/src/iogen.c" 2024-03-27T10:58:54.759 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T10:58:54.759 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T10:58:54.759 INFO:teuthology.orchestra.run.smithi097.stdout: "damage_type": "backtrace", 2024-03-27T10:58:54.759 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 4259426906, 2024-03-27T10:58:54.759 INFO:teuthology.orchestra.run.smithi097.stdout: "ino": 1099511628291, 2024-03-27T10:58:54.759 INFO:teuthology.orchestra.run.smithi097.stdout: "path": "/volumes/qa/sv_0/22bc04cd-8106-4d11-83f1-2aead5019d07/client.0/tmp/iogen_3.1p0/src/iogen.8" 2024-03-27T10:58:54.759 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T10:58:54.759 INFO:teuthology.orchestra.run.smithi097.stdout:] 2024-03-27T10:58:54.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.779+0000 7f7913e00640 1 -- 172.21.15.97:0/3897900656 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f78f80a3690 msgr2=0x7f78f80a5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:54.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.779+0000 7f7913e00640 1 --2- 172.21.15.97:0/3897900656 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f78f80a3690 0x7f78f80a5b50 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f7904006010 tx=0x7f7904005a20 comp rx=0 tx=0).stop 2024-03-27T10:58:54.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.779+0000 7f7913e00640 1 -- 172.21.15.97:0/3897900656 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7914108070 msgr2=0x7f791419e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:54.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.779+0000 7f7913e00640 1 --2- 172.21.15.97:0/3897900656 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7914108070 0x7f791419e1e0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f790c00ea80 tx=0x7f790c00ef40 comp rx=0 tx=0).stop 2024-03-27T10:58:54.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.779+0000 7f7913e00640 1 -- 172.21.15.97:0/3897900656 shutdown_connections 2024-03-27T10:58:54.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.779+0000 7f7913e00640 1 --2- 172.21.15.97:0/3897900656 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f78f80a3690 0x7f78f80a5b50 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:54.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.779+0000 7f7913e00640 1 --2- 172.21.15.97:0/3897900656 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7914109c20 0x7f79141a5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:54.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.779+0000 7f7913e00640 1 --2- 172.21.15.97:0/3897900656 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7914109260 0x7f791419e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:54.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.779+0000 7f7913e00640 1 --2- 172.21.15.97:0/3897900656 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7914108070 0x7f791419e1e0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:54.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.779+0000 7f7913e00640 1 -- 172.21.15.97:0/3897900656 >> 172.21.15.97:0/3897900656 conn(0x7f7914077020 msgr2=0x7f791410c500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:54.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.779+0000 7f7913e00640 1 -- 172.21.15.97:0/3897900656 shutdown_connections 2024-03-27T10:58:54.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:54.779+0000 7f7913e00640 1 -- 172.21.15.97:0/3897900656 wait complete. 2024-03-27T10:58:54.765 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-27T10:58:54.766 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:58:54 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/1384333746' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T10:58:54.766 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:58:54 smithi114 ceph-mon[35338]: pgmap v509: 97 pgs: 97 active+clean; 5.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 125 MiB/s wr, 76 op/s 2024-03-27T10:58:55.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.052+0000 7fd1bfe00640 1 -- 172.21.15.97:0/3181974029 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd1c01024e0 msgr2=0x7fd1c0108d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:55.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.052+0000 7fd1bfe00640 1 --2- 172.21.15.97:0/3181974029 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd1c01024e0 0x7fd1c0108d70 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd1b8009ca0 tx=0x7fd1b802c100 comp rx=0 tx=0).stop 2024-03-27T10:58:55.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.052+0000 7fd1bfe00640 1 -- 172.21.15.97:0/3181974029 shutdown_connections 2024-03-27T10:58:55.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.052+0000 7fd1bfe00640 1 --2- 172.21.15.97:0/3181974029 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd1c01024e0 0x7fd1c0108d70 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:55.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.052+0000 7fd1bfe00640 1 --2- 172.21.15.97:0/3181974029 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd1c0101b20 0x7fd1c0101fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:55.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.052+0000 7fd1bfe00640 1 --2- 172.21.15.97:0/3181974029 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1c0100930 0x7fd1c0100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:55.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.052+0000 7fd1bfe00640 1 -- 172.21.15.97:0/3181974029 >> 172.21.15.97:0/3181974029 conn(0x7fd1c00fbec0 msgr2=0x7fd1c00fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:55.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.052+0000 7fd1bfe00640 1 -- 172.21.15.97:0/3181974029 shutdown_connections 2024-03-27T10:58:55.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.052+0000 7fd1bfe00640 1 -- 172.21.15.97:0/3181974029 wait complete. 2024-03-27T10:58:55.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.053+0000 7fd1bfe00640 1 Processor -- start 2024-03-27T10:58:55.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.054+0000 7fd1bfe00640 1 -- start start 2024-03-27T10:58:55.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.054+0000 7fd1bfe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd1c0100930 0x7fd1c0199e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:55.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.054+0000 7fd1bfe00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd1c0101b20 0x7fd1c019a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:55.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.054+0000 7fd1bfe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1c01024e0 0x7fd1c01a13b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:55.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.054+0000 7fd1bfe00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c010c440 con 0x7fd1c01024e0 2024-03-27T10:58:55.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.054+0000 7fd1bfe00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c010c2c0 con 0x7fd1c0100930 2024-03-27T10:58:55.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.054+0000 7fd1bfe00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c010c5c0 con 0x7fd1c0101b20 2024-03-27T10:58:55.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.054+0000 7fd1bea00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd1c0100930 0x7fd1c0199e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:55.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.055+0000 7fd1bea00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd1c0100930 0x7fd1c0199e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:60256/0 (socket says 172.21.15.97:60256) 2024-03-27T10:58:55.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.055+0000 7fd1bea00640 1 -- 172.21.15.97:0/4005359956 learned_addr learned my addr 172.21.15.97:0/4005359956 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:58:55.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.055+0000 7fd1be000640 1 --2- 172.21.15.97:0/4005359956 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd1c0101b20 0x7fd1c019a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:55.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.055+0000 7fd1bf400640 1 --2- 172.21.15.97:0/4005359956 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1c01024e0 0x7fd1c01a13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:55.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.055+0000 7fd1bea00640 1 -- 172.21.15.97:0/4005359956 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd1c0101b20 msgr2=0x7fd1c019a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:55.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.055+0000 7fd1bea00640 1 --2- 172.21.15.97:0/4005359956 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd1c0101b20 0x7fd1c019a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:55.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.055+0000 7fd1bea00640 1 -- 172.21.15.97:0/4005359956 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1c01024e0 msgr2=0x7fd1c01a13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:55.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.055+0000 7fd1bea00640 1 --2- 172.21.15.97:0/4005359956 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1c01024e0 0x7fd1c01a13b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:55.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.055+0000 7fd1bea00640 1 -- 172.21.15.97:0/4005359956 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1c01a1a20 con 0x7fd1c0100930 2024-03-27T10:58:55.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.056+0000 7fd1bea00640 1 --2- 172.21.15.97:0/4005359956 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd1c0100930 0x7fd1c0199e00 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd1b000e9c0 tx=0x7fd1b000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:55.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.056+0000 7fd1b7400640 1 -- 172.21.15.97:0/4005359956 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b000cd80 con 0x7fd1c0100930 2024-03-27T10:58:55.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.056+0000 7fd1b7400640 1 -- 172.21.15.97:0/4005359956 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd1b00035f0 con 0x7fd1c0100930 2024-03-27T10:58:55.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.056+0000 7fd1b7400640 1 -- 172.21.15.97:0/4005359956 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b0010690 con 0x7fd1c0100930 2024-03-27T10:58:55.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.056+0000 7fd1bfe00640 1 -- 172.21.15.97:0/4005359956 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1c01a1d10 con 0x7fd1c0100930 2024-03-27T10:58:55.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.056+0000 7fd1bfe00640 1 -- 172.21.15.97:0/4005359956 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1c01a21b0 con 0x7fd1c0100930 2024-03-27T10:58:55.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.061+0000 7fd1c89bc740 1 Processor -- start 2024-03-27T10:58:55.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.062+0000 7fd1c89bc740 1 -- start start 2024-03-27T10:58:55.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.062+0000 7fd1c89bc740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556e64cf4ab0 0x556e64cf4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:55.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.062+0000 7fd1c89bc740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556e64cf54a0 0x556e64cf9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:55.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.062+0000 7fd1c89bc740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556e64cfa610 0x556e64cfcaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:55.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.062+0000 7fd1c89bc740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x556e64cbc7b0 con 0x556e64cf54a0 2024-03-27T10:58:55.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.062+0000 7fd1c89bc740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x556e64c59670 con 0x556e64cf4ab0 2024-03-27T10:58:55.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.062+0000 7fd1c89bc740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x556e64c01970 con 0x556e64cfa610 2024-03-27T10:58:55.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.063+0000 7fd1bf400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556e64cf4ab0 0x556e64cf4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:55.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.063+0000 7fd1b7400640 1 -- 172.21.15.97:0/4005359956 <== mon.2 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7fd1b00037a0 con 0x7fd1c0100930 2024-03-27T10:58:55.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.063+0000 7fd1bf400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556e64cf4ab0 0x556e64cf4ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:60266/0 (socket says 172.21.15.97:60266) 2024-03-27T10:58:55.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.063+0000 7fd1bf400640 1 -- 172.21.15.97:0/1482907858 learned_addr learned my addr 172.21.15.97:0/1482907858 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:58:55.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.063+0000 7fd1be000640 1 --2- 172.21.15.97:0/1482907858 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556e64cfa610 0x556e64cfcaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:55.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.063+0000 7fd1bea00640 1 --2- 172.21.15.97:0/1482907858 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556e64cf54a0 0x556e64cf9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:55.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.064+0000 7fd1bf400640 1 -- 172.21.15.97:0/1482907858 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556e64cfa610 msgr2=0x556e64cfcaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:55.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.064+0000 7fd1bf400640 1 --2- 172.21.15.97:0/1482907858 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556e64cfa610 0x556e64cfcaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:55.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.064+0000 7fd1bf400640 1 -- 172.21.15.97:0/1482907858 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556e64cf54a0 msgr2=0x556e64cf9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:55.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.064+0000 7fd1bf400640 1 --2- 172.21.15.97:0/1482907858 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556e64cf54a0 0x556e64cf9fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:55.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.064+0000 7fd1bf400640 1 -- 172.21.15.97:0/1482907858 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e6496d3f0 con 0x556e64cf4ab0 2024-03-27T10:58:55.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.064+0000 7fd1bea00640 1 --2- 172.21.15.97:0/1482907858 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556e64cf54a0 0x556e64cf9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T10:58:55.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.064+0000 7fd1b7400640 1 --2- 172.21.15.97:0/4005359956 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fd1a40a3690 0x7fd1a40a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:55.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.065+0000 7fd1b7400640 1 -- 172.21.15.97:0/4005359956 <== mon.2 v2:172.21.15.114:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fd1b00d0170 con 0x7fd1c0100930 2024-03-27T10:58:55.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.065+0000 7fd1bea00640 1 --2- 172.21.15.97:0/4005359956 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fd1a40a3690 0x7fd1a40a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:55.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.065+0000 7fd1bf400640 1 --2- 172.21.15.97:0/1482907858 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556e64cf4ab0 0x556e64cf4ed0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd1b802c710 tx=0x7fd1b8009800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e774336c5deae0c4 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:55.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.065+0000 7fd1abe00640 1 -- 172.21.15.97:0/1482907858 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b8043070 con 0x556e64cf4ab0 2024-03-27T10:58:55.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.066+0000 7fd1abe00640 1 -- 172.21.15.97:0/1482907858 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd1b8032dc0 con 0x556e64cf4ab0 2024-03-27T10:58:55.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.066+0000 7fd1abe00640 1 -- 172.21.15.97:0/1482907858 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b8039040 con 0x556e64cf4ab0 2024-03-27T10:58:55.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.066+0000 7fd1c89bc740 1 -- 172.21.15.97:0/1482907858 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556e64cf4ab0 msgr2=0x556e64cf4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:55.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.066+0000 7fd1bea00640 1 --2- 172.21.15.97:0/4005359956 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fd1a40a3690 0x7fd1a40a5b50 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fd1b0098a50 tx=0x7fd1b0003e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:55.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.066+0000 7fd1c89bc740 1 --2- 172.21.15.97:0/1482907858 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556e64cf4ab0 0x556e64cf4ed0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd1b802c710 tx=0x7fd1b8009800 comp rx=0 tx=0).stop 2024-03-27T10:58:55.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.066+0000 7fd1c89bc740 1 -- 172.21.15.97:0/1482907858 shutdown_connections 2024-03-27T10:58:55.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.066+0000 7fd1c89bc740 1 --2- 172.21.15.97:0/1482907858 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556e64cfa610 0x556e64cfcaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:55.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.066+0000 7fd1c89bc740 1 --2- 172.21.15.97:0/1482907858 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556e64cf54a0 0x556e64cf9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:55.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.066+0000 7fd1c89bc740 1 --2- 172.21.15.97:0/1482907858 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556e64cf4ab0 0x556e64cf4ed0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:55.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.066+0000 7fd1c89bc740 1 -- 172.21.15.97:0/1482907858 >> 172.21.15.97:0/1482907858 conn(0x556e64cf0020 msgr2=0x556e64cf0420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:55.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.067+0000 7fd1c89bc740 1 -- 172.21.15.97:0/1482907858 shutdown_connections 2024-03-27T10:58:55.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.067+0000 7fd1c89bc740 1 -- 172.21.15.97:0/1482907858 wait complete. 2024-03-27T10:58:55.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.067+0000 7fd1c89bc740 1 -- start start 2024-03-27T10:58:55.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.068+0000 7fd1c89bc740 1 Processor -- start 2024-03-27T10:58:55.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.068+0000 7fd1c89bc740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556e64cf4ab0 0x556e64cfeb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:55.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.068+0000 7fd1c89bc740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556e64cf54a0 0x556e64cff0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:55.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.068+0000 7fd1c89bc740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556e64cfa610 0x556e64d060f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:55.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.068+0000 7fd1c89bc740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x556e64c14ca0 con 0x556e64cf54a0 2024-03-27T10:58:55.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.068+0000 7fd1c89bc740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x556e64c55620 con 0x556e64cf4ab0 2024-03-27T10:58:55.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.068+0000 7fd1c89bc740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x556e64cfd5b0 con 0x556e64cfa610 2024-03-27T10:58:55.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.068+0000 7fd1c89bc740 10 client.0 _ll_register_callbacks cb 0x556e64c51160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:58:55.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.069+0000 7fd1be000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556e64cf54a0 0x556e64cff0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:55.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.069+0000 7fd1be000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556e64cf54a0 0x556e64cff0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49656/0 (socket says 172.21.15.97:49656) 2024-03-27T10:58:55.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.069+0000 7fd1be000640 1 -- 172.21.15.97:0/110831576 learned_addr learned my addr 172.21.15.97:0/110831576 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:58:55.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.069+0000 7fd1bf400640 1 --2- 172.21.15.97:0/110831576 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556e64cf4ab0 0x556e64cfeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:55.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.069+0000 7fd1bf400640 1 --2- 172.21.15.97:0/110831576 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556e64cfa610 0x556e64d060f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:55.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.070+0000 7fd1be000640 1 -- 172.21.15.97:0/110831576 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556e64cfa610 msgr2=0x556e64d060f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:55.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.070+0000 7fd1be000640 1 --2- 172.21.15.97:0/110831576 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556e64cfa610 0x556e64d060f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:55.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.070+0000 7fd1be000640 1 -- 172.21.15.97:0/110831576 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556e64cf4ab0 msgr2=0x556e64cfeb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:55.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.070+0000 7fd1be000640 1 --2- 172.21.15.97:0/110831576 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556e64cf4ab0 0x556e64cfeb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:55.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.070+0000 7fd1be000640 1 -- 172.21.15.97:0/110831576 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e64a377d0 con 0x556e64cf54a0 2024-03-27T10:58:55.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.070+0000 7fd1be000640 1 --2- 172.21.15.97:0/110831576 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556e64cf54a0 0x556e64cff0b0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac00bf30 tx=0x7fd1ac0128e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9efe0355712425d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:55.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.070+0000 7fd1a8c00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T10:58:55.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.071+0000 7fd1a8c00640 1 -- 172.21.15.97:0/110831576 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1ac01f060 con 0x556e64cf54a0 2024-03-27T10:58:55.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.071+0000 7fd1c89bc740 1 -- 172.21.15.97:0/110831576 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1b8032dc0 con 0x556e64cf54a0 2024-03-27T10:58:55.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.071+0000 7fd1a8c00640 1 -- 172.21.15.97:0/110831576 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fd1ac0108a0 con 0x556e64cf54a0 2024-03-27T10:58:55.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.071+0000 7fd1a8c00640 1 -- 172.21.15.97:0/110831576 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fd1ac010aa0 con 0x556e64cf54a0 2024-03-27T10:58:55.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.071+0000 7fd1c89bc740 10 client.16347 fetch_fsmap learned FSMap version 31 2024-03-27T10:58:55.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.071+0000 7fd1c89bc740 1 -- 172.21.15.97:0/110831576 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x556e64d11d90 con 0x556e64cf54a0 2024-03-27T10:58:55.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.072+0000 7fd1a8c00640 1 -- 172.21.15.97:0/110831576 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd1b8032dc0 con 0x556e64cf54a0 2024-03-27T10:58:55.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.072+0000 7fd1a8c00640 1 -- 172.21.15.97:0/110831576 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 31) ==== 8370+0+0 (secure 0 0 0) 0x7fd1ac019030 con 0x556e64cf54a0 2024-03-27T10:58:55.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.072+0000 7fd1c89bc740 10 client.16347 fetch_fsmap finished waiting for FSMap version 31 2024-03-27T10:58:55.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.072+0000 7fd1c89bc740 10 client.16347 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-27T10:58:55.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.072+0000 7fd1c89bc740 20 client.16347 populate_metadata read hostname 'smithi097' 2024-03-27T10:58:55.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.072+0000 7fd1c89bc740 1 --2- 172.21.15.97:0/110831576 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x556e64d13470 0x556e64d158f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:55.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.072+0000 7fd1c89bc740 4 client.16347 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T10:58:55.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.072+0000 7fd1c89bc740 1 -- 172.21.15.97:0/110831576 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556e64c66bb0 con 0x556e64d13470 2024-03-27T10:58:55.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.073+0000 7fd1bf400640 1 --2- 172.21.15.97:0/110831576 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x556e64d13470 0x556e64d158f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:55.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.073+0000 7fd1bf400640 1 --2- 172.21.15.97:0/110831576 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x556e64d13470 0x556e64d158f0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57eda7f887a8181f server_cookie=5623bc82afcce5d3 in_seq=0 out_seq=0 2024-03-27T10:58:55.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.073+0000 7fd1a8c00640 10 client.16347 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T10:58:55.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.074+0000 7fd1a8c00640 1 -- 172.21.15.97:0/110831576 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x556e64c66bb0 con 0x556e64d13470 2024-03-27T10:58:55.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.074+0000 7fd1a8c00640 10 client.16347 handle_command_reply: tid=0 2024-03-27T10:58:55.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.075+0000 7fd1c89bc740 1 client.16347 shutdown 2024-03-27T10:58:55.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.075+0000 7fd1c89bc740 1 -- 172.21.15.97:0/110831576 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556e64cf54a0 msgr2=0x556e64cff0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:55.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.075+0000 7fd1c89bc740 1 --2- 172.21.15.97:0/110831576 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556e64cf54a0 0x556e64cff0b0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac00bf30 tx=0x7fd1ac0128e0 comp rx=0 tx=0).stop 2024-03-27T10:58:55.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.075+0000 7fd1c89bc740 1 -- 172.21.15.97:0/110831576 shutdown_connections 2024-03-27T10:58:55.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.075+0000 7fd1c89bc740 1 --2- 172.21.15.97:0/110831576 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x556e64d13470 0x556e64d158f0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:55.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.075+0000 7fd1c89bc740 1 --2- 172.21.15.97:0/110831576 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556e64cfa610 0x556e64d060f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:55.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.075+0000 7fd1c89bc740 1 --2- 172.21.15.97:0/110831576 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556e64cf54a0 0x556e64cff0b0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:55.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.075+0000 7fd1c89bc740 1 --2- 172.21.15.97:0/110831576 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556e64cf4ab0 0x556e64cfeb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:55.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.075+0000 7fd1c89bc740 1 -- 172.21.15.97:0/110831576 >> 172.21.15.97:0/110831576 conn(0x556e64cf0020 msgr2=0x556e64cef240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:55.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.075+0000 7fd1a8c00640 0 client.16347 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T10:58:55.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.075+0000 7fd1c89bc740 1 -- 172.21.15.97:0/110831576 shutdown_connections 2024-03-27T10:58:55.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.075+0000 7fd1c89bc740 1 -- 172.21.15.97:0/110831576 wait complete. 2024-03-27T10:58:55.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.075+0000 7fd1c89bc740 20 client.16347 trim_cache size 0 max 16384 2024-03-27T10:58:55.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.099+0000 7fd1c89bc740 1 Processor -- start 2024-03-27T10:58:55.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.099+0000 7fd1c89bc740 1 -- start start 2024-03-27T10:58:55.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.100+0000 7fd1c89bc740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556e64cf4ab0 0x556e64cf66e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:55.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.100+0000 7fd1c89bc740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556e64cf54a0 0x556e64d060b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:55.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.100+0000 7fd1c89bc740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556e64cfa610 0x556e64cf6c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:55.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.100+0000 7fd1c89bc740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c01a6bb0 con 0x556e64cfa610 2024-03-27T10:58:55.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.100+0000 7fd1c89bc740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c01a87b0 con 0x556e64cf4ab0 2024-03-27T10:58:55.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.100+0000 7fd1c89bc740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c01a7280 con 0x556e64cf54a0 2024-03-27T10:58:55.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.100+0000 7fd1bf400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556e64cfa610 0x556e64cf6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:55.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.100+0000 7fd1bf400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556e64cfa610 0x556e64cf6c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49672/0 (socket says 172.21.15.97:49672) 2024-03-27T10:58:55.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.100+0000 7fd1bf400640 1 -- 172.21.15.97:0/4063931162 learned_addr learned my addr 172.21.15.97:0/4063931162 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:58:55.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.100+0000 7fd1be000640 1 --2- 172.21.15.97:0/4063931162 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556e64cf54a0 0x556e64d060b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:55.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.100+0000 7fd1bf400640 1 --2- 172.21.15.97:0/4063931162 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556e64cf4ab0 0x556e64cf66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:55.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.101+0000 7fd1bf400640 1 -- 172.21.15.97:0/4063931162 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556e64cf54a0 msgr2=0x556e64d060b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:55.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.101+0000 7fd1bf400640 1 --2- 172.21.15.97:0/4063931162 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556e64cf54a0 0x556e64d060b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:55.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.101+0000 7fd1bf400640 1 -- 172.21.15.97:0/4063931162 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556e64cf4ab0 msgr2=0x556e64cf66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:55.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.101+0000 7fd1bf400640 1 --2- 172.21.15.97:0/4063931162 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556e64cf4ab0 0x556e64cf66e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:55.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.101+0000 7fd1bf400640 1 -- 172.21.15.97:0/4063931162 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e64cf46f0 con 0x556e64cfa610 2024-03-27T10:58:55.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.101+0000 7fd1bf400640 1 --2- 172.21.15.97:0/4063931162 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556e64cfa610 0x556e64cf6c20 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fd1b8006010 tx=0x7fd1b8005fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dd9a2d2bf3e1ba server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:55.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.101+0000 7fd1ab400640 1 -- 172.21.15.97:0/4063931162 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b805a070 con 0x556e64cfa610 2024-03-27T10:58:55.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.102+0000 7fd1ab400640 1 -- 172.21.15.97:0/4063931162 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd1b80325c0 con 0x556e64cfa610 2024-03-27T10:58:55.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.102+0000 7fd1ab400640 1 -- 172.21.15.97:0/4063931162 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b8039040 con 0x556e64cfa610 2024-03-27T10:58:55.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.103+0000 7fd1c89bc740 1 -- 172.21.15.97:0/4063931162 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556e64cfa610 msgr2=0x556e64cf6c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:55.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.103+0000 7fd1c89bc740 1 --2- 172.21.15.97:0/4063931162 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556e64cfa610 0x556e64cf6c20 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fd1b8006010 tx=0x7fd1b8005fa0 comp rx=0 tx=0).stop 2024-03-27T10:58:55.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.103+0000 7fd1c89bc740 1 -- 172.21.15.97:0/4063931162 shutdown_connections 2024-03-27T10:58:55.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.103+0000 7fd1c89bc740 1 --2- 172.21.15.97:0/4063931162 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556e64cfa610 0x556e64cf6c20 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:55.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.103+0000 7fd1c89bc740 1 --2- 172.21.15.97:0/4063931162 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556e64cf54a0 0x556e64d060b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:55.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.103+0000 7fd1c89bc740 1 --2- 172.21.15.97:0/4063931162 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556e64cf4ab0 0x556e64cf66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:55.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.103+0000 7fd1c89bc740 1 -- 172.21.15.97:0/4063931162 >> 172.21.15.97:0/4063931162 conn(0x556e64cf0020 msgr2=0x556e64cf9d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:55.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.103+0000 7fd1c89bc740 1 -- 172.21.15.97:0/4063931162 shutdown_connections 2024-03-27T10:58:55.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.103+0000 7fd1c89bc740 1 -- 172.21.15.97:0/4063931162 wait complete. 2024-03-27T10:58:55.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.103+0000 7fd1c89bc740 1 -- start start 2024-03-27T10:58:55.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.104+0000 7fd1c89bc740 1 Processor -- start 2024-03-27T10:58:55.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.104+0000 7fd1c89bc740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556e64cf4ab0 0x556e64d03c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:55.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.104+0000 7fd1c89bc740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556e64cf54a0 0x556e64d041b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:55.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.104+0000 7fd1c89bc740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556e64cfa610 0x556e64d051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:55.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.104+0000 7fd1c89bc740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c01a8020 con 0x556e64cf54a0 2024-03-27T10:58:55.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.104+0000 7fd1c89bc740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c01a7950 con 0x556e64cf4ab0 2024-03-27T10:58:55.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.104+0000 7fd1c89bc740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c01a4bb0 con 0x556e64cfa610 2024-03-27T10:58:55.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.104+0000 7fd1c89bc740 10 client.0 _ll_register_callbacks cb 0x7fd1ac00b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:58:55.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.104+0000 7fd1be000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556e64cf54a0 0x556e64d041b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:55.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.104+0000 7fd1be000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556e64cf54a0 0x556e64d041b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49676/0 (socket says 172.21.15.97:49676) 2024-03-27T10:58:55.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.104+0000 7fd1be000640 1 -- 172.21.15.97:0/1567206144 learned_addr learned my addr 172.21.15.97:0/1567206144 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:58:55.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.104+0000 7fd1bf400640 1 --2- 172.21.15.97:0/1567206144 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556e64cf4ab0 0x556e64d03c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:55.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.105+0000 7fd1bf400640 1 --2- 172.21.15.97:0/1567206144 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556e64cfa610 0x556e64d051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:55.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.105+0000 7fd1be000640 1 -- 172.21.15.97:0/1567206144 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556e64cfa610 msgr2=0x556e64d051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:55.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.105+0000 7fd1be000640 1 --2- 172.21.15.97:0/1567206144 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556e64cfa610 0x556e64d051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:55.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.105+0000 7fd1be000640 1 -- 172.21.15.97:0/1567206144 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556e64cf4ab0 msgr2=0x556e64d03c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:58:55.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.105+0000 7fd1be000640 1 --2- 172.21.15.97:0/1567206144 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556e64cf4ab0 0x556e64d03c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:55.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.105+0000 7fd1be000640 1 -- 172.21.15.97:0/1567206144 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e64d057b0 con 0x556e64cf54a0 2024-03-27T10:58:55.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.105+0000 7fd1be000640 1 --2- 172.21.15.97:0/1567206144 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556e64cf54a0 0x556e64d041b0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac01aee0 tx=0x7fd1ac00bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8327be594021190 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:58:55.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.105+0000 7fd1a8c00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T10:58:55.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.105+0000 7fd1a8c00640 1 -- 172.21.15.97:0/1567206144 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1ac02a040 con 0x556e64cf54a0 2024-03-27T10:58:55.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.105+0000 7fd1a8c00640 1 -- 172.21.15.97:0/1567206144 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fd1ac010cc0 con 0x556e64cf54a0 2024-03-27T10:58:55.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.105+0000 7fd1c89bc740 1 -- 172.21.15.97:0/1567206144 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1b80325c0 con 0x556e64cf54a0 2024-03-27T10:58:55.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.105+0000 7fd1a8c00640 1 -- 172.21.15.97:0/1567206144 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fd1ac01f030 con 0x556e64cf54a0 2024-03-27T10:58:55.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.105+0000 7fd1c89bc740 10 client.16353 fetch_fsmap learned FSMap version 31 2024-03-27T10:58:55.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.105+0000 7fd1c89bc740 1 -- 172.21.15.97:0/1567206144 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x556e64d79c90 con 0x556e64cf54a0 2024-03-27T10:58:55.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.106+0000 7fd1a8c00640 1 -- 172.21.15.97:0/1567206144 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd1b80325c0 con 0x556e64cf54a0 2024-03-27T10:58:55.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.106+0000 7fd1a8c00640 1 -- 172.21.15.97:0/1567206144 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 31) ==== 8370+0+0 (secure 0 0 0) 0x7fd1ac02d070 con 0x556e64cf54a0 2024-03-27T10:58:55.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.106+0000 7fd1c89bc740 10 client.16353 fetch_fsmap finished waiting for FSMap version 31 2024-03-27T10:58:55.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.106+0000 7fd1c89bc740 10 client.16353 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-27T10:58:55.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.106+0000 7fd1c89bc740 20 client.16353 populate_metadata read hostname 'smithi097' 2024-03-27T10:58:55.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.106+0000 7fd1c89bc740 1 --2- 172.21.15.97:0/1567206144 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x556e64d7a770 0x556e64d7cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:58:55.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.106+0000 7fd1c89bc740 4 client.16353 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-27T10:58:55.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.106+0000 7fd1c89bc740 1 -- 172.21.15.97:0/1567206144 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556e64c60f00 con 0x556e64d7a770 2024-03-27T10:58:55.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.106+0000 7fd1bf400640 1 --2- 172.21.15.97:0/1567206144 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x556e64d7a770 0x556e64d7cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:58:55.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.107+0000 7fd1bf400640 1 --2- 172.21.15.97:0/1567206144 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x556e64d7a770 0x556e64d7cb90 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3335181978aebbb2 server_cookie=ab1d918598061dcd in_seq=0 out_seq=0 2024-03-27T10:58:55.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.107+0000 7fd1a8c00640 10 client.16353 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T10:58:55.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.107+0000 7fd1a8c00640 1 -- 172.21.15.97:0/1567206144 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556e64c60f00 con 0x556e64d7a770 2024-03-27T10:58:55.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.107+0000 7fd1a8c00640 10 client.16353 handle_command_reply: tid=0 2024-03-27T10:58:55.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.107+0000 7fd1c89bc740 1 client.16353 shutdown 2024-03-27T10:58:55.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.107+0000 7fd1c89bc740 1 -- 172.21.15.97:0/1567206144 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556e64cf54a0 msgr2=0x556e64d041b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:55.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.107+0000 7fd1c89bc740 1 --2- 172.21.15.97:0/1567206144 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556e64cf54a0 0x556e64d041b0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac01aee0 tx=0x7fd1ac00bab0 comp rx=0 tx=0).stop 2024-03-27T10:58:55.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.107+0000 7fd1c89bc740 1 -- 172.21.15.97:0/1567206144 shutdown_connections 2024-03-27T10:58:55.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.107+0000 7fd1c89bc740 1 --2- 172.21.15.97:0/1567206144 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x556e64d7a770 0x556e64d7cb90 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:55.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.107+0000 7fd1c89bc740 1 --2- 172.21.15.97:0/1567206144 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556e64cfa610 0x556e64d051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:55.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.107+0000 7fd1c89bc740 1 --2- 172.21.15.97:0/1567206144 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556e64cf54a0 0x556e64d041b0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:55.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.107+0000 7fd1c89bc740 1 --2- 172.21.15.97:0/1567206144 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556e64cf4ab0 0x556e64d03c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:55.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.107+0000 7fd1c89bc740 1 -- 172.21.15.97:0/1567206144 >> 172.21.15.97:0/1567206144 conn(0x556e64cf0020 msgr2=0x556e64cfeb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:55.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.107+0000 7fd1a8c00640 0 client.16353 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T10:58:55.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.108+0000 7fd1c89bc740 1 -- 172.21.15.97:0/1567206144 shutdown_connections 2024-03-27T10:58:55.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.108+0000 7fd1c89bc740 1 -- 172.21.15.97:0/1567206144 wait complete. 2024-03-27T10:58:55.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.108+0000 7fd1c89bc740 20 client.16353 trim_cache size 0 max 16384 2024-03-27T10:58:55.084 INFO:teuthology.orchestra.run.smithi097.stdout:[] 2024-03-27T10:58:55.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.108+0000 7fd1bfe00640 1 -- 172.21.15.97:0/4005359956 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fd1a40a3690 msgr2=0x7fd1a40a5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:55.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.108+0000 7fd1bfe00640 1 --2- 172.21.15.97:0/4005359956 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fd1a40a3690 0x7fd1a40a5b50 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fd1b0098a50 tx=0x7fd1b0003e30 comp rx=0 tx=0).stop 2024-03-27T10:58:55.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.108+0000 7fd1bfe00640 1 -- 172.21.15.97:0/4005359956 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd1c0100930 msgr2=0x7fd1c0199e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:58:55.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.108+0000 7fd1bfe00640 1 --2- 172.21.15.97:0/4005359956 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd1c0100930 0x7fd1c0199e00 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd1b000e9c0 tx=0x7fd1b000ee80 comp rx=0 tx=0).stop 2024-03-27T10:58:55.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.108+0000 7fd1bfe00640 1 -- 172.21.15.97:0/4005359956 shutdown_connections 2024-03-27T10:58:55.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.108+0000 7fd1bfe00640 1 --2- 172.21.15.97:0/4005359956 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1c01024e0 0x7fd1c01a13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:55.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.108+0000 7fd1bfe00640 1 --2- 172.21.15.97:0/4005359956 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd1c0101b20 0x7fd1c019a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:55.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.108+0000 7fd1bfe00640 1 --2- 172.21.15.97:0/4005359956 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fd1a40a3690 0x7fd1a40a5b50 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:55.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.108+0000 7fd1bfe00640 1 --2- 172.21.15.97:0/4005359956 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd1c0100930 0x7fd1c0199e00 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:58:55.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.108+0000 7fd1bfe00640 1 -- 172.21.15.97:0/4005359956 >> 172.21.15.97:0/4005359956 conn(0x7fd1c00fbec0 msgr2=0x7fd1c00fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:58:55.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.108+0000 7fd1bfe00640 1 -- 172.21.15.97:0/4005359956 shutdown_connections 2024-03-27T10:58:55.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T10:58:55.109+0000 7fd1bfe00640 1 -- 172.21.15.97:0/4005359956 wait complete. 2024-03-27T10:58:55.097 ERROR:tasks.fwd_scrub:rank 1 damaged: [{'damage_type': 'backtrace', 'id': 2849809169, 'ino': 1099511628292, 'path': '/volumes/qa/sv_0/22bc04cd-8106-4d11-83f1-2aead5019d07/client.0/tmp/iogen_3.1p0/src/iogen.c'}, {'damage_type': 'backtrace', 'id': 4259426906, 'ino': 1099511628291, 'path': '/volumes/qa/sv_0/22bc04cd-8106-4d11-83f1-2aead5019d07/client.0/tmp/iogen_3.1p0/src/iogen.8'}] 2024-03-27T10:58:55.097 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_155268c4e432a12433aa833f174f9fe3b1016ae0/qa/tasks/fwd_scrub.py", line 38, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_155268c4e432a12433aa833f174f9fe3b1016ae0/qa/tasks/fwd_scrub.py", line 55, in do_scrub self._scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_155268c4e432a12433aa833f174f9fe3b1016ae0/qa/tasks/fwd_scrub.py", line 79, in _scrub self._check_damage() File "/home/teuthworker/src/git.ceph.com_ceph-c_155268c4e432a12433aa833f174f9fe3b1016ae0/qa/tasks/fwd_scrub.py", line 90, in _check_damage raise RuntimeError(f"rank damage found: {types}") RuntimeError: rank damage found: {'backtrace'} 2024-03-27T10:58:57.516 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:58:57 smithi114 ceph-mon[35338]: pgmap v510: 97 pgs: 97 active+clean; 5.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 122 MiB/s wr, 73 op/s 2024-03-27T10:58:57.642 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:58:57 smithi097 ceph-mon[31859]: pgmap v510: 97 pgs: 97 active+clean; 5.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 122 MiB/s wr, 73 op/s 2024-03-27T10:58:57.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:58:57 smithi177 ceph-mon[35294]: pgmap v510: 97 pgs: 97 active+clean; 5.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 122 MiB/s wr, 73 op/s 2024-03-27T10:58:59.516 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:58:59 smithi114 ceph-mon[35338]: pgmap v511: 97 pgs: 97 active+clean; 6.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 123 MiB/s wr, 74 op/s 2024-03-27T10:58:59.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:58:59 smithi097 ceph-mon[31859]: pgmap v511: 97 pgs: 97 active+clean; 6.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 123 MiB/s wr, 74 op/s 2024-03-27T10:58:59.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:58:59 smithi177 ceph-mon[35294]: pgmap v511: 97 pgs: 97 active+clean; 6.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 123 MiB/s wr, 74 op/s 2024-03-27T10:59:01.464 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:59:01 smithi097 ceph-mon[31859]: pgmap v512: 97 pgs: 97 active+clean; 6.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 132 MiB/s wr, 80 op/s 2024-03-27T10:59:01.516 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:59:01 smithi114 ceph-mon[35338]: pgmap v512: 97 pgs: 97 active+clean; 6.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 132 MiB/s wr, 80 op/s 2024-03-27T10:59:01.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:59:01 smithi177 ceph-mon[35294]: pgmap v512: 97 pgs: 97 active+clean; 6.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 132 MiB/s wr, 80 op/s 2024-03-27T10:59:02.204 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.232+0000 7f219ba00640 1 -- 172.21.15.97:0/240727394 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2194100a70 msgr2=0x7f2194100e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:59:02.204 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.232+0000 7f219ba00640 1 --2- 172.21.15.97:0/240727394 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2194100a70 0x7f2194100e90 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f2188009900 tx=0x7f218802bc40 comp rx=0 tx=0).stop 2024-03-27T10:59:02.205 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.232+0000 7f219ba00640 1 -- 172.21.15.97:0/240727394 shutdown_connections 2024-03-27T10:59:02.205 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.232+0000 7f219ba00640 1 --2- 172.21.15.97:0/240727394 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2194102620 0x7f2194108eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:02.205 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.232+0000 7f219ba00640 1 --2- 172.21.15.97:0/240727394 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2194101c60 0x7f21941020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:02.205 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.232+0000 7f219ba00640 1 --2- 172.21.15.97:0/240727394 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2194100a70 0x7f2194100e90 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:02.205 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.232+0000 7f219ba00640 1 -- 172.21.15.97:0/240727394 >> 172.21.15.97:0/240727394 conn(0x7f21940fbfe0 msgr2=0x7f21940fe420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:59:02.205 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.232+0000 7f219ba00640 1 -- 172.21.15.97:0/240727394 shutdown_connections 2024-03-27T10:59:02.205 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.232+0000 7f219ba00640 1 -- 172.21.15.97:0/240727394 wait complete. 2024-03-27T10:59:02.206 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.234+0000 7f219ba00640 1 Processor -- start 2024-03-27T10:59:02.206 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.234+0000 7f219ba00640 1 -- start start 2024-03-27T10:59:02.206 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.234+0000 7f219ba00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2194101c60 0x7f219419e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:59:02.206 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.234+0000 7f219ba00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2194102620 0x7f219419eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:59:02.207 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.235+0000 7f219ba00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21941a3b20 0x7f21941a5f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:59:02.207 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.235+0000 7f219ba00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f219410b470 con 0x7f21941a3b20 2024-03-27T10:59:02.207 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.235+0000 7f219ba00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f219410b2f0 con 0x7f2194102620 2024-03-27T10:59:02.207 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.235+0000 7f219ba00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f219410b5f0 con 0x7f2194101c60 2024-03-27T10:59:02.207 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.235+0000 7f2193e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2194102620 0x7f219419eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:59:02.207 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.235+0000 7f2199000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2194101c60 0x7f219419e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:59:02.207 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.235+0000 7f2193e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2194102620 0x7f219419eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:41776/0 (socket says 172.21.15.97:41776) 2024-03-27T10:59:02.208 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.235+0000 7f2193e00640 1 -- 172.21.15.97:0/2896816689 learned_addr learned my addr 172.21.15.97:0/2896816689 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:59:02.208 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.235+0000 7f2199a00640 1 --2- 172.21.15.97:0/2896816689 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21941a3b20 0x7f21941a5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:59:02.209 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.236+0000 7f2193e00640 1 -- 172.21.15.97:0/2896816689 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2194101c60 msgr2=0x7f219419e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:59:02.209 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.236+0000 7f2193e00640 1 --2- 172.21.15.97:0/2896816689 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2194101c60 0x7f219419e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:02.209 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.236+0000 7f2193e00640 1 -- 172.21.15.97:0/2896816689 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21941a3b20 msgr2=0x7f21941a5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:59:02.209 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.236+0000 7f2193e00640 1 --2- 172.21.15.97:0/2896816689 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21941a3b20 0x7f21941a5f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:02.209 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.236+0000 7f2193e00640 1 -- 172.21.15.97:0/2896816689 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21941a6580 con 0x7f2194102620 2024-03-27T10:59:02.209 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.236+0000 7f2199a00640 1 --2- 172.21.15.97:0/2896816689 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21941a3b20 0x7f21941a5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T10:59:02.209 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.236+0000 7f2193e00640 1 --2- 172.21.15.97:0/2896816689 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2194102620 0x7f219419eac0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f218000e9f0 tx=0x7f218000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:59:02.209 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.237+0000 7f2191600640 1 -- 172.21.15.97:0/2896816689 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f218000ccc0 con 0x7f2194102620 2024-03-27T10:59:02.209 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.237+0000 7f219ba00640 1 -- 172.21.15.97:0/2896816689 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21941a6870 con 0x7f2194102620 2024-03-27T10:59:02.210 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.237+0000 7f219ba00640 1 -- 172.21.15.97:0/2896816689 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21941a6e10 con 0x7f2194102620 2024-03-27T10:59:02.210 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.237+0000 7f2191600640 1 -- 172.21.15.97:0/2896816689 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f21800108e0 con 0x7f2194102620 2024-03-27T10:59:02.210 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.237+0000 7f2191600640 1 -- 172.21.15.97:0/2896816689 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21800187c0 con 0x7f2194102620 2024-03-27T10:59:02.213 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.242+0000 7f219e657740 1 Processor -- start 2024-03-27T10:59:02.214 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.242+0000 7f219e657740 1 -- start start 2024-03-27T10:59:02.214 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.242+0000 7f219e657740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b4f4cbe980 0x55b4f4cbeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:59:02.214 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.242+0000 7f219e657740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b4f4cbf370 0x55b4f4cc3e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:59:02.214 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.242+0000 7f219e657740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b4f4cc44e0 0x55b4f4cc6970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:59:02.214 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.242+0000 7f219e657740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55b4f4c23d10 con 0x55b4f4cc44e0 2024-03-27T10:59:02.214 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.242+0000 7f219e657740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55b4f4bcc260 con 0x55b4f4cbe980 2024-03-27T10:59:02.214 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.242+0000 7f219e657740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55b4f4c1fcc0 con 0x55b4f4cbf370 2024-03-27T10:59:02.215 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.243+0000 7f2191600640 1 -- 172.21.15.97:0/2896816689 <== mon.2 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f2180018960 con 0x7f2194102620 2024-03-27T10:59:02.215 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.243+0000 7f2199a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b4f4cbe980 0x55b4f4cbeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:59:02.215 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.243+0000 7f2199a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b4f4cbe980 0x55b4f4cbeda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:41790/0 (socket says 172.21.15.97:41790) 2024-03-27T10:59:02.216 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.243+0000 7f2199a00640 1 -- 172.21.15.97:0/2532298280 learned_addr learned my addr 172.21.15.97:0/2532298280 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:59:02.216 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.243+0000 7f2199000640 1 --2- 172.21.15.97:0/2532298280 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b4f4cbf370 0x55b4f4cc3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:59:02.216 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.244+0000 7f2199a00640 1 -- 172.21.15.97:0/2532298280 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b4f4cbf370 msgr2=0x55b4f4cc3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:59:02.216 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.244+0000 7f2199a00640 1 --2- 172.21.15.97:0/2532298280 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b4f4cbf370 0x55b4f4cc3e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:02.216 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.244+0000 7f2199a00640 1 -- 172.21.15.97:0/2532298280 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b4f4cc44e0 msgr2=0x55b4f4cc6970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:59:02.216 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.244+0000 7f2199a00640 1 --2- 172.21.15.97:0/2532298280 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b4f4cc44e0 0x55b4f4cc6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:02.216 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.244+0000 7f2199a00640 1 -- 172.21.15.97:0/2532298280 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4f4bc3aa0 con 0x55b4f4cbe980 2024-03-27T10:59:02.217 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.244+0000 7f2199a00640 1 --2- 172.21.15.97:0/2532298280 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b4f4cbe980 0x55b4f4cbeda0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f218c011810 tx=0x7f218c011cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a8d4a5459e31397 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:59:02.217 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.245+0000 7f2191600640 1 --2- 172.21.15.97:0/2896816689 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f217c0a37b0 0x7f217c0a5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:59:02.217 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.245+0000 7f2191600640 1 -- 172.21.15.97:0/2896816689 <== mon.2 v2:172.21.15.114:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f21800cffc0 con 0x7f2194102620 2024-03-27T10:59:02.217 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.246+0000 7f2199000640 1 --2- 172.21.15.97:0/2896816689 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f217c0a37b0 0x7f217c0a5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:59:02.218 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.246+0000 7f2186000640 1 -- 172.21.15.97:0/2532298280 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f218c011310 con 0x55b4f4cbe980 2024-03-27T10:59:02.218 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.246+0000 7f2186000640 1 -- 172.21.15.97:0/2532298280 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f218c00f9a0 con 0x55b4f4cbe980 2024-03-27T10:59:02.218 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.246+0000 7f2186000640 1 -- 172.21.15.97:0/2532298280 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f218c005610 con 0x55b4f4cbe980 2024-03-27T10:59:02.219 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.247+0000 7f219e657740 1 -- 172.21.15.97:0/2532298280 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b4f4cbe980 msgr2=0x55b4f4cbeda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:59:02.219 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.247+0000 7f219e657740 1 --2- 172.21.15.97:0/2532298280 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b4f4cbe980 0x55b4f4cbeda0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f218c011810 tx=0x7f218c011cd0 comp rx=0 tx=0).stop 2024-03-27T10:59:02.220 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.247+0000 7f219e657740 1 -- 172.21.15.97:0/2532298280 shutdown_connections 2024-03-27T10:59:02.220 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.247+0000 7f219e657740 1 --2- 172.21.15.97:0/2532298280 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b4f4cc44e0 0x55b4f4cc6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:02.220 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.247+0000 7f219e657740 1 --2- 172.21.15.97:0/2532298280 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b4f4cbf370 0x55b4f4cc3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:02.220 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.247+0000 7f219e657740 1 --2- 172.21.15.97:0/2532298280 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b4f4cbe980 0x55b4f4cbeda0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:02.220 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.247+0000 7f2199000640 1 --2- 172.21.15.97:0/2896816689 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f217c0a37b0 0x7f217c0a5c70 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f2188036040 tx=0x7f2188000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:59:02.220 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.247+0000 7f219e657740 1 -- 172.21.15.97:0/2532298280 >> 172.21.15.97:0/2532298280 conn(0x55b4f4cb9ef0 msgr2=0x55b4f4cba2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:59:02.220 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.247+0000 7f219e657740 1 -- 172.21.15.97:0/2532298280 shutdown_connections 2024-03-27T10:59:02.220 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.248+0000 7f219e657740 1 -- 172.21.15.97:0/2532298280 wait complete. 2024-03-27T10:59:02.220 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.248+0000 7f219e657740 1 -- start start 2024-03-27T10:59:02.220 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.248+0000 7f219e657740 1 Processor -- start 2024-03-27T10:59:02.222 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.249+0000 7f219e657740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b4f4cbe980 0x55b4f4cc8d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:59:02.222 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.249+0000 7f219e657740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b4f4cbf370 0x55b4f4cc9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:59:02.223 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.249+0000 7f219e657740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b4f4cc44e0 0x55b4f4cd02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:59:02.223 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.249+0000 7f219e657740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55b4f4cc7490 con 0x55b4f4cc44e0 2024-03-27T10:59:02.223 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.249+0000 7f219e657740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55b4f4bdf630 con 0x55b4f4cbe980 2024-03-27T10:59:02.223 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.249+0000 7f219e657740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55b4f4cc7610 con 0x55b4f4cbf370 2024-03-27T10:59:02.223 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.249+0000 7f219e657740 10 client.0 _ll_register_callbacks cb 0x55b4f4be4c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:59:02.223 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.249+0000 7f2199000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b4f4cbe980 0x55b4f4cc8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:59:02.223 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.249+0000 7f2199000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b4f4cbe980 0x55b4f4cc8d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:41802/0 (socket says 172.21.15.97:41802) 2024-03-27T10:59:02.223 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.249+0000 7f2199000640 1 -- 172.21.15.97:0/1876156171 learned_addr learned my addr 172.21.15.97:0/1876156171 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:59:02.223 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.250+0000 7f2199a00640 1 --2- 172.21.15.97:0/1876156171 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b4f4cc44e0 0x55b4f4cd02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:59:02.223 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.250+0000 7f2193e00640 1 --2- 172.21.15.97:0/1876156171 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b4f4cbf370 0x55b4f4cc9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:59:02.223 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.250+0000 7f2199000640 1 -- 172.21.15.97:0/1876156171 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b4f4cbf370 msgr2=0x55b4f4cc9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:59:02.224 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.250+0000 7f2199000640 1 --2- 172.21.15.97:0/1876156171 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b4f4cbf370 0x55b4f4cc9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:02.224 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.250+0000 7f2199000640 1 -- 172.21.15.97:0/1876156171 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b4f4cc44e0 msgr2=0x55b4f4cd02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:59:02.224 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.250+0000 7f2199000640 1 --2- 172.21.15.97:0/1876156171 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b4f4cc44e0 0x55b4f4cd02c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:02.224 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.250+0000 7f2199000640 1 -- 172.21.15.97:0/1876156171 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4f4beb5f0 con 0x55b4f4cbe980 2024-03-27T10:59:02.224 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.250+0000 7f2193e00640 1 --2- 172.21.15.97:0/1876156171 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b4f4cbf370 0x55b4f4cc9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:59:02.224 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.250+0000 7f2199a00640 1 --2- 172.21.15.97:0/1876156171 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b4f4cc44e0 0x55b4f4cd02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T10:59:02.224 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.251+0000 7f2199000640 1 --2- 172.21.15.97:0/1876156171 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b4f4cbe980 0x55b4f4cc8d40 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f218802c850 tx=0x7f218802bc40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91ddb1221bbbfafe server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:59:02.224 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.251+0000 7f217aa00640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-03-27T10:59:02.226 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.251+0000 7f217aa00640 1 -- 172.21.15.97:0/1876156171 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f218804f030 con 0x55b4f4cbe980 2024-03-27T10:59:02.226 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.251+0000 7f217aa00640 1 -- 172.21.15.97:0/1876156171 <== mon.2 v2:172.21.15.114:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f218803a040 con 0x55b4f4cbe980 2024-03-27T10:59:02.226 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.252+0000 7f219e657740 1 -- 172.21.15.97:0/1876156171 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f218c00f9a0 con 0x55b4f4cbe980 2024-03-27T10:59:02.226 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.252+0000 7f217aa00640 1 -- 172.21.15.97:0/1876156171 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f2188005040 con 0x55b4f4cbe980 2024-03-27T10:59:02.226 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.253+0000 7f219e657740 10 client.36005 fetch_fsmap learned FSMap version 31 2024-03-27T10:59:02.226 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.253+0000 7f219e657740 1 -- 172.21.15.97:0/1876156171 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b4f4938210 con 0x55b4f4cbe980 2024-03-27T10:59:02.226 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.253+0000 7f217aa00640 1 -- 172.21.15.97:0/1876156171 <== mon.2 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f218c00f9a0 con 0x55b4f4cbe980 2024-03-27T10:59:02.226 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.253+0000 7f217aa00640 1 -- 172.21.15.97:0/1876156171 <== mon.2 v2:172.21.15.114:3300/0 5 ==== fsmap(e 31) ==== 8370+0+0 (secure 0 0 0) 0x7f21880035f0 con 0x55b4f4cbe980 2024-03-27T10:59:02.226 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.254+0000 7f219e657740 10 client.36005 fetch_fsmap finished waiting for FSMap version 31 2024-03-27T10:59:02.227 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.254+0000 7f219e657740 10 client.36005 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T10:59:02.227 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.254+0000 7f219e657740 20 client.36005 populate_metadata read hostname 'smithi097' 2024-03-27T10:59:02.227 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.254+0000 7f219e657740 1 --2- 172.21.15.97:0/1876156171 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55b4f4cd4ac0 0x55b4f4cd6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:59:02.227 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.254+0000 7f219e657740 4 client.36005 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T10:59:02.227 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.254+0000 7f219e657740 1 -- 172.21.15.97:0/1876156171 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b4f4c31250 con 0x55b4f4cd4ac0 2024-03-27T10:59:02.227 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.254+0000 7f2193e00640 1 --2- 172.21.15.97:0/1876156171 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55b4f4cd4ac0 0x55b4f4cd6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:59:02.227 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.255+0000 7f2193e00640 1 --2- 172.21.15.97:0/1876156171 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55b4f4cd4ac0 0x55b4f4cd6ee0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed5c532e5b449c8d server_cookie=846ef8091102a0fd in_seq=0 out_seq=0 2024-03-27T10:59:02.228 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.255+0000 7f217aa00640 10 client.36005 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T10:59:02.228 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.257+0000 7f217aa00640 1 -- 172.21.15.97:0/1876156171 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55b4f4c31250 con 0x55b4f4cd4ac0 2024-03-27T10:59:02.228 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.257+0000 7f217aa00640 10 client.36005 handle_command_reply: tid=0 2024-03-27T10:59:02.228 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.257+0000 7f219e657740 1 client.36005 shutdown 2024-03-27T10:59:02.229 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.257+0000 7f219e657740 1 -- 172.21.15.97:0/1876156171 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b4f4cbe980 msgr2=0x55b4f4cc8d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:59:02.230 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.257+0000 7f219e657740 1 --2- 172.21.15.97:0/1876156171 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b4f4cbe980 0x55b4f4cc8d40 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f218802c850 tx=0x7f218802bc40 comp rx=0 tx=0).stop 2024-03-27T10:59:02.230 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.257+0000 7f219e657740 1 -- 172.21.15.97:0/1876156171 shutdown_connections 2024-03-27T10:59:02.230 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.257+0000 7f219e657740 1 --2- 172.21.15.97:0/1876156171 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55b4f4cd4ac0 0x55b4f4cd6ee0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:02.230 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.258+0000 7f217aa00640 0 client.36005 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T10:59:02.230 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.258+0000 7f219e657740 1 --2- 172.21.15.97:0/1876156171 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b4f4cc44e0 0x55b4f4cd02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:02.230 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.258+0000 7f219e657740 1 --2- 172.21.15.97:0/1876156171 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b4f4cbf370 0x55b4f4cc9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:02.230 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.258+0000 7f219e657740 1 --2- 172.21.15.97:0/1876156171 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b4f4cbe980 0x55b4f4cc8d40 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:02.230 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.258+0000 7f219e657740 1 -- 172.21.15.97:0/1876156171 >> 172.21.15.97:0/1876156171 conn(0x55b4f4cb9ef0 msgr2=0x55b4f4cb90f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:59:02.230 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.258+0000 7f219e657740 1 -- 172.21.15.97:0/1876156171 shutdown_connections 2024-03-27T10:59:02.230 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.258+0000 7f219e657740 1 -- 172.21.15.97:0/1876156171 wait complete. 2024-03-27T10:59:02.231 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.258+0000 7f219e657740 20 client.36005 trim_cache size 0 max 16384 2024-03-27T10:59:02.256 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.285+0000 7f219e657740 1 Processor -- start 2024-03-27T10:59:02.256 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.285+0000 7f219e657740 1 -- start start 2024-03-27T10:59:02.257 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.285+0000 7f219e657740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b4f4cbe980 0x55b4f4cc91e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:59:02.257 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.285+0000 7f219e657740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b4f4cbf370 0x55b4f4cd0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:59:02.257 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.285+0000 7f219e657740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b4f4cc44e0 0x55b4f4cc80f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:59:02.257 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.285+0000 7f219e657740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f217400e430 con 0x55b4f4cbe980 2024-03-27T10:59:02.257 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.285+0000 7f219e657740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f2174010030 con 0x55b4f4cbf370 2024-03-27T10:59:02.257 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.285+0000 7f219e657740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f217400eb00 con 0x55b4f4cc44e0 2024-03-27T10:59:02.258 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.286+0000 7f2193e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b4f4cbf370 0x55b4f4cd0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:59:02.258 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.286+0000 7f2193e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b4f4cbf370 0x55b4f4cd0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:41812/0 (socket says 172.21.15.97:41812) 2024-03-27T10:59:02.258 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.286+0000 7f2193e00640 1 -- 172.21.15.97:0/3265467144 learned_addr learned my addr 172.21.15.97:0/3265467144 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:59:02.258 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.286+0000 7f2199a00640 1 --2- 172.21.15.97:0/3265467144 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b4f4cc44e0 0x55b4f4cc80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:59:02.259 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.286+0000 7f2199000640 1 --2- 172.21.15.97:0/3265467144 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b4f4cbe980 0x55b4f4cc91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:59:02.259 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.287+0000 7f2193e00640 1 -- 172.21.15.97:0/3265467144 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b4f4cc44e0 msgr2=0x55b4f4cc80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:59:02.259 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.287+0000 7f2193e00640 1 --2- 172.21.15.97:0/3265467144 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b4f4cc44e0 0x55b4f4cc80f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:02.259 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.287+0000 7f2193e00640 1 -- 172.21.15.97:0/3265467144 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b4f4cbe980 msgr2=0x55b4f4cc91e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:59:02.259 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.287+0000 7f2193e00640 1 --2- 172.21.15.97:0/3265467144 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b4f4cbe980 0x55b4f4cc91e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:02.259 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.287+0000 7f2193e00640 1 -- 172.21.15.97:0/3265467144 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4f4a04620 con 0x55b4f4cbf370 2024-03-27T10:59:02.259 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.287+0000 7f2199000640 1 --2- 172.21.15.97:0/3265467144 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b4f4cbe980 0x55b4f4cc91e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:59:02.259 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.287+0000 7f2199a00640 1 --2- 172.21.15.97:0/3265467144 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b4f4cc44e0 0x55b4f4cc80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T10:59:02.260 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.288+0000 7f2193e00640 1 --2- 172.21.15.97:0/3265467144 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b4f4cbf370 0x55b4f4cd0280 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2180009830 tx=0x7f218000ce60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=262c30daa5c6991d server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:59:02.260 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.288+0000 7f2185600640 1 -- 172.21.15.97:0/3265467144 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f218009aa30 con 0x55b4f4cbf370 2024-03-27T10:59:02.260 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.288+0000 7f2185600640 1 -- 172.21.15.97:0/3265467144 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f21800cdb80 con 0x55b4f4cbf370 2024-03-27T10:59:02.260 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.288+0000 7f2185600640 1 -- 172.21.15.97:0/3265467144 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2180098620 con 0x55b4f4cbf370 2024-03-27T10:59:02.261 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.289+0000 7f219e657740 1 -- 172.21.15.97:0/3265467144 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b4f4cbf370 msgr2=0x55b4f4cd0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:59:02.261 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.289+0000 7f219e657740 1 --2- 172.21.15.97:0/3265467144 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b4f4cbf370 0x55b4f4cd0280 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2180009830 tx=0x7f218000ce60 comp rx=0 tx=0).stop 2024-03-27T10:59:02.261 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.290+0000 7f219e657740 1 -- 172.21.15.97:0/3265467144 shutdown_connections 2024-03-27T10:59:02.262 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.290+0000 7f219e657740 1 --2- 172.21.15.97:0/3265467144 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b4f4cc44e0 0x55b4f4cc80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:02.262 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.290+0000 7f219e657740 1 --2- 172.21.15.97:0/3265467144 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b4f4cbf370 0x55b4f4cd0280 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:02.262 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.290+0000 7f219e657740 1 --2- 172.21.15.97:0/3265467144 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b4f4cbe980 0x55b4f4cc91e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:02.262 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.290+0000 7f219e657740 1 -- 172.21.15.97:0/3265467144 >> 172.21.15.97:0/3265467144 conn(0x55b4f4cb9ef0 msgr2=0x55b4f4cc79e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:59:02.262 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.290+0000 7f219e657740 1 -- 172.21.15.97:0/3265467144 shutdown_connections 2024-03-27T10:59:02.262 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.290+0000 7f219e657740 1 -- 172.21.15.97:0/3265467144 wait complete. 2024-03-27T10:59:02.262 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.290+0000 7f219e657740 1 -- start start 2024-03-27T10:59:02.262 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.291+0000 7f219e657740 1 Processor -- start 2024-03-27T10:59:02.263 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.292+0000 7f219e657740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b4f4cbe980 0x55b4f4ccd120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:59:02.263 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.292+0000 7f219e657740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b4f4cbf370 0x55b4f4ccd660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:59:02.264 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.292+0000 7f219e657740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b4f4cc44e0 0x55b4f4d38e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:59:02.264 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.292+0000 7f219e657740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f217400f8a0 con 0x55b4f4cc44e0 2024-03-27T10:59:02.264 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.292+0000 7f219e657740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f217400f1d0 con 0x55b4f4cbf370 2024-03-27T10:59:02.264 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.292+0000 7f219e657740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f217400c410 con 0x55b4f4cbe980 2024-03-27T10:59:02.264 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.292+0000 7f219e657740 10 client.0 _ll_register_callbacks cb 0x7f2188043050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:59:02.264 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.292+0000 7f2199000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b4f4cbe980 0x55b4f4ccd120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:59:02.265 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.292+0000 7f2193e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b4f4cbf370 0x55b4f4ccd660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:59:02.265 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.293+0000 7f2193e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b4f4cbf370 0x55b4f4ccd660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:41818/0 (socket says 172.21.15.97:41818) 2024-03-27T10:59:02.265 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.293+0000 7f2193e00640 1 -- 172.21.15.97:0/938436771 learned_addr learned my addr 172.21.15.97:0/938436771 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:59:02.265 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.293+0000 7f2199a00640 1 --2- 172.21.15.97:0/938436771 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b4f4cc44e0 0x55b4f4d38e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:59:02.266 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.293+0000 7f2193e00640 1 -- 172.21.15.97:0/938436771 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b4f4cbe980 msgr2=0x55b4f4ccd120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:59:02.266 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.293+0000 7f2193e00640 1 --2- 172.21.15.97:0/938436771 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b4f4cbe980 0x55b4f4ccd120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:02.266 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.293+0000 7f2193e00640 1 -- 172.21.15.97:0/938436771 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b4f4cc44e0 msgr2=0x55b4f4d38e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:59:02.266 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.293+0000 7f2193e00640 1 --2- 172.21.15.97:0/938436771 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b4f4cc44e0 0x55b4f4d38e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:02.266 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.293+0000 7f2193e00640 1 -- 172.21.15.97:0/938436771 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4f4ccdd00 con 0x55b4f4cbf370 2024-03-27T10:59:02.266 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.294+0000 7f2193e00640 1 --2- 172.21.15.97:0/938436771 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b4f4cbf370 0x55b4f4ccd660 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f218009c320 tx=0x7f21800a50b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78d3b0234054d8f2 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:59:02.266 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.294+0000 7f217aa00640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-03-27T10:59:02.267 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.295+0000 7f217aa00640 1 -- 172.21.15.97:0/938436771 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21800b4060 con 0x55b4f4cbf370 2024-03-27T10:59:02.267 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.295+0000 7f217aa00640 1 -- 172.21.15.97:0/938436771 <== mon.2 v2:172.21.15.114:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f2180010430 con 0x55b4f4cbf370 2024-03-27T10:59:02.267 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.295+0000 7f219e657740 1 -- 172.21.15.97:0/938436771 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21800cdb80 con 0x55b4f4cbf370 2024-03-27T10:59:02.268 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.296+0000 7f217aa00640 1 -- 172.21.15.97:0/938436771 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f218009bac0 con 0x55b4f4cbf370 2024-03-27T10:59:02.268 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.297+0000 7f219e657740 10 client.36011 fetch_fsmap learned FSMap version 31 2024-03-27T10:59:02.269 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.297+0000 7f219e657740 1 -- 172.21.15.97:0/938436771 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b4f4d3a4a0 con 0x55b4f4cbf370 2024-03-27T10:59:02.269 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.298+0000 7f217aa00640 1 -- 172.21.15.97:0/938436771 <== mon.2 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f21800cdb80 con 0x55b4f4cbf370 2024-03-27T10:59:02.270 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.298+0000 7f217aa00640 1 -- 172.21.15.97:0/938436771 <== mon.2 v2:172.21.15.114:3300/0 5 ==== fsmap(e 31) ==== 8370+0+0 (secure 0 0 0) 0x7f2180085ce0 con 0x55b4f4cbf370 2024-03-27T10:59:02.270 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.298+0000 7f219e657740 10 client.36011 fetch_fsmap finished waiting for FSMap version 31 2024-03-27T10:59:02.270 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.298+0000 7f219e657740 10 client.36011 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T10:59:02.270 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.298+0000 7f219e657740 20 client.36011 populate_metadata read hostname 'smithi097' 2024-03-27T10:59:02.270 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.298+0000 7f219e657740 1 --2- 172.21.15.97:0/938436771 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55b4f4d3aec0 0x55b4f4d3d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:59:02.270 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.298+0000 7f219e657740 4 client.36011 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-27T10:59:02.270 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.298+0000 7f219e657740 1 -- 172.21.15.97:0/938436771 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b4f4c2b5a0 con 0x55b4f4d3aec0 2024-03-27T10:59:02.271 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.299+0000 7f2199000640 1 --2- 172.21.15.97:0/938436771 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55b4f4d3aec0 0x55b4f4d3d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:59:02.271 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.300+0000 7f2199000640 1 --2- 172.21.15.97:0/938436771 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55b4f4d3aec0 0x55b4f4d3d2e0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7c189b5276d7f3d server_cookie=35b96c1d69ed136f in_seq=0 out_seq=0 2024-03-27T10:59:02.271 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.300+0000 7f217aa00640 10 client.36011 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T10:59:02.286 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.315+0000 7f217aa00640 1 -- 172.21.15.97:0/938436771 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b4f4c2b5a0 con 0x55b4f4d3aec0 2024-03-27T10:59:02.287 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.315+0000 7f217aa00640 10 client.36011 handle_command_reply: tid=0 2024-03-27T10:59:02.287 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.315+0000 7f219e657740 1 client.36011 shutdown 2024-03-27T10:59:02.288 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.315+0000 7f219e657740 1 -- 172.21.15.97:0/938436771 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b4f4cbf370 msgr2=0x55b4f4ccd660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:59:02.288 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.315+0000 7f219e657740 1 --2- 172.21.15.97:0/938436771 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b4f4cbf370 0x55b4f4ccd660 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f218009c320 tx=0x7f21800a50b0 comp rx=0 tx=0).stop 2024-03-27T10:59:02.288 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.316+0000 7f219e657740 1 -- 172.21.15.97:0/938436771 shutdown_connections 2024-03-27T10:59:02.288 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.316+0000 7f219e657740 1 --2- 172.21.15.97:0/938436771 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55b4f4d3aec0 0x55b4f4d3d2e0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:02.288 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.316+0000 7f219e657740 1 --2- 172.21.15.97:0/938436771 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b4f4cc44e0 0x55b4f4d38e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:02.288 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.316+0000 7f219e657740 1 --2- 172.21.15.97:0/938436771 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b4f4cbf370 0x55b4f4ccd660 secure :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f218009c320 tx=0x7f21800a50b0 comp rx=0 tx=0).stop 2024-03-27T10:59:02.288 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.316+0000 7f219e657740 1 --2- 172.21.15.97:0/938436771 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b4f4cbe980 0x55b4f4ccd120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:02.288 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.316+0000 7f219e657740 1 -- 172.21.15.97:0/938436771 >> 172.21.15.97:0/938436771 conn(0x55b4f4cb9ef0 msgr2=0x55b4f4cb8860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:59:02.288 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.316+0000 7f217aa00640 0 client.36011 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T10:59:02.288 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.316+0000 7f219e657740 1 -- 172.21.15.97:0/938436771 shutdown_connections 2024-03-27T10:59:02.288 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.316+0000 7f219e657740 1 -- 172.21.15.97:0/938436771 wait complete. 2024-03-27T10:59:02.288 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.316+0000 7f219e657740 20 client.36011 trim_cache size 0 max 16384 2024-03-27T10:59:02.289 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-27T10:59:02.289 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-27T10:59:02.289 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-27T10:59:02.289 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-27T10:59:02.290 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.317+0000 7f219ba00640 1 -- 172.21.15.97:0/2896816689 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f217c0a37b0 msgr2=0x7f217c0a5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:59:02.290 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.317+0000 7f219ba00640 1 --2- 172.21.15.97:0/2896816689 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f217c0a37b0 0x7f217c0a5c70 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f2188036040 tx=0x7f2188000bc0 comp rx=0 tx=0).stop 2024-03-27T10:59:02.290 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.317+0000 7f219ba00640 1 -- 172.21.15.97:0/2896816689 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2194102620 msgr2=0x7f219419eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:59:02.290 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.317+0000 7f219ba00640 1 --2- 172.21.15.97:0/2896816689 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2194102620 0x7f219419eac0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f218000e9f0 tx=0x7f218000edf0 comp rx=0 tx=0).stop 2024-03-27T10:59:02.290 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.317+0000 7f219ba00640 1 -- 172.21.15.97:0/2896816689 shutdown_connections 2024-03-27T10:59:02.290 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.317+0000 7f219ba00640 1 --2- 172.21.15.97:0/2896816689 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21941a3b20 0x7f21941a5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:02.290 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.317+0000 7f219ba00640 1 --2- 172.21.15.97:0/2896816689 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f217c0a37b0 0x7f217c0a5c70 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:02.290 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.317+0000 7f219ba00640 1 --2- 172.21.15.97:0/2896816689 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2194102620 0x7f219419eac0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:02.290 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.317+0000 7f219ba00640 1 --2- 172.21.15.97:0/2896816689 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2194101c60 0x7f219419e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:02.291 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.317+0000 7f219ba00640 1 -- 172.21.15.97:0/2896816689 >> 172.21.15.97:0/2896816689 conn(0x7f21940fbfe0 msgr2=0x7f219410e0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:59:02.291 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.318+0000 7f219ba00640 1 -- 172.21.15.97:0/2896816689 shutdown_connections 2024-03-27T10:59:02.291 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:02.318+0000 7f219ba00640 1 -- 172.21.15.97:0/2896816689 wait complete. 2024-03-27T10:59:03.516 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:59:03 smithi114 ceph-mon[35338]: pgmap v513: 97 pgs: 97 active+clean; 6.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 117 MiB/s wr, 69 op/s 2024-03-27T10:59:03.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:59:03 smithi097 ceph-mon[31859]: pgmap v513: 97 pgs: 97 active+clean; 6.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 117 MiB/s wr, 69 op/s 2024-03-27T10:59:03.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:59:03 smithi177 ceph-mon[35294]: pgmap v513: 97 pgs: 97 active+clean; 6.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 117 MiB/s wr, 69 op/s 2024-03-27T10:59:05.515 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:59:05 smithi114 ceph-mon[35338]: pgmap v514: 97 pgs: 97 active+clean; 6.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 116 MiB/s wr, 72 op/s 2024-03-27T10:59:05.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:59:05 smithi097 ceph-mon[31859]: pgmap v514: 97 pgs: 97 active+clean; 6.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 116 MiB/s wr, 72 op/s 2024-03-27T10:59:05.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:59:05 smithi177 ceph-mon[35294]: pgmap v514: 97 pgs: 97 active+clean; 6.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 116 MiB/s wr, 72 op/s 2024-03-27T10:59:07.516 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:59:07 smithi114 ceph-mon[35338]: pgmap v515: 97 pgs: 97 active+clean; 6.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 112 MiB/s wr, 68 op/s 2024-03-27T10:59:07.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:59:07 smithi097 ceph-mon[31859]: pgmap v515: 97 pgs: 97 active+clean; 6.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 112 MiB/s wr, 68 op/s 2024-03-27T10:59:07.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:59:07 smithi177 ceph-mon[35294]: pgmap v515: 97 pgs: 97 active+clean; 6.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 112 MiB/s wr, 68 op/s 2024-03-27T10:59:09.516 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:59:09 smithi114 ceph-mon[35338]: pgmap v516: 97 pgs: 97 active+clean; 7.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 116 MiB/s wr, 69 op/s 2024-03-27T10:59:09.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:59:09 smithi097 ceph-mon[31859]: pgmap v516: 97 pgs: 97 active+clean; 7.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 116 MiB/s wr, 69 op/s 2024-03-27T10:59:09.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:59:09 smithi177 ceph-mon[35294]: pgmap v516: 97 pgs: 97 active+clean; 7.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 116 MiB/s wr, 69 op/s 2024-03-27T10:59:11.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:59:11 smithi114 ceph-mon[35338]: pgmap v517: 97 pgs: 97 active+clean; 7.5 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 130 MiB/s wr, 79 op/s 2024-03-27T10:59:11.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:59:11 smithi097 ceph-mon[31859]: pgmap v517: 97 pgs: 97 active+clean; 7.5 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 130 MiB/s wr, 79 op/s 2024-03-27T10:59:11.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:59:11 smithi177 ceph-mon[35294]: pgmap v517: 97 pgs: 97 active+clean; 7.5 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 130 MiB/s wr, 79 op/s 2024-03-27T10:59:13.516 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:59:13 smithi114 ceph-mon[35338]: pgmap v518: 97 pgs: 97 active+clean; 7.7 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 118 MiB/s wr, 70 op/s 2024-03-27T10:59:13.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:59:13 smithi097 ceph-mon[31859]: pgmap v518: 97 pgs: 97 active+clean; 7.7 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 118 MiB/s wr, 70 op/s 2024-03-27T10:59:13.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:59:13 smithi177 ceph-mon[35294]: pgmap v518: 97 pgs: 97 active+clean; 7.7 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 118 MiB/s wr, 70 op/s 2024-03-27T10:59:15.471 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.499+0000 7f2806000640 1 -- 172.21.15.97:0/511375979 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2800077750 msgr2=0x7f2800077bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:59:15.471 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.499+0000 7f2806000640 1 --2- 172.21.15.97:0/511375979 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2800077750 0x7f2800077bb0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f27ec009860 tx=0x7f27ec02bb70 comp rx=0 tx=0).stop 2024-03-27T10:59:15.471 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.499+0000 7f2806000640 1 -- 172.21.15.97:0/511375979 shutdown_connections 2024-03-27T10:59:15.471 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.499+0000 7f2806000640 1 --2- 172.21.15.97:0/511375979 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f28000780f0 0x7f2800078990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:15.472 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.499+0000 7f2806000640 1 --2- 172.21.15.97:0/511375979 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2800077750 0x7f2800077bb0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:15.472 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.499+0000 7f2806000640 1 --2- 172.21.15.97:0/511375979 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2800076510 0x7f2800076930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:15.472 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.499+0000 7f2806000640 1 -- 172.21.15.97:0/511375979 >> 172.21.15.97:0/511375979 conn(0x7f28000ff310 msgr2=0x7f2800101750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:59:15.472 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.499+0000 7f2806000640 1 -- 172.21.15.97:0/511375979 shutdown_connections 2024-03-27T10:59:15.472 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.499+0000 7f2806000640 1 -- 172.21.15.97:0/511375979 wait complete. 2024-03-27T10:59:15.472 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.500+0000 7f2806000640 1 Processor -- start 2024-03-27T10:59:15.473 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.501+0000 7f2806000640 1 -- start start 2024-03-27T10:59:15.473 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.501+0000 7f2806000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2800076510 0x7f280019e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:59:15.473 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.501+0000 7f2806000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2800077750 0x7f280019e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:59:15.473 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.501+0000 7f2806000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f28000780f0 0x7f28001a5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:59:15.474 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.501+0000 7f2806000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2800113c90 con 0x7f2800077750 2024-03-27T10:59:15.474 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.501+0000 7f2806000640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f2800113b10 con 0x7f2800076510 2024-03-27T10:59:15.474 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.501+0000 7f2806000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2800113e10 con 0x7f28000780f0 2024-03-27T10:59:15.474 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.502+0000 7f27ff400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2800076510 0x7f280019e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:59:15.474 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.502+0000 7f27fea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2800077750 0x7f280019e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:59:15.474 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.502+0000 7f27ff400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2800076510 0x7f280019e2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:36776/0 (socket says 172.21.15.97:36776) 2024-03-27T10:59:15.474 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.502+0000 7f27ff400640 1 -- 172.21.15.97:0/3429679434 learned_addr learned my addr 172.21.15.97:0/3429679434 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:59:15.474 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.502+0000 7f27ffe00640 1 --2- 172.21.15.97:0/3429679434 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f28000780f0 0x7f28001a5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:59:15.475 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.503+0000 7f27fea00640 1 -- 172.21.15.97:0/3429679434 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f28000780f0 msgr2=0x7f28001a5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:59:15.475 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.503+0000 7f27fea00640 1 --2- 172.21.15.97:0/3429679434 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f28000780f0 0x7f28001a5880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:15.475 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.503+0000 7f27fea00640 1 -- 172.21.15.97:0/3429679434 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2800076510 msgr2=0x7f280019e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:59:15.475 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.503+0000 7f27fea00640 1 --2- 172.21.15.97:0/3429679434 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2800076510 0x7f280019e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:15.475 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.503+0000 7f27fea00640 1 -- 172.21.15.97:0/3429679434 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28001a5ef0 con 0x7f2800077750 2024-03-27T10:59:15.475 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.503+0000 7f27ff400640 1 --2- 172.21.15.97:0/3429679434 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2800076510 0x7f280019e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T10:59:15.475 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.503+0000 7f27fea00640 1 --2- 172.21.15.97:0/3429679434 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2800077750 0x7f280019e810 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f27ec02c0b0 tx=0x7f27ec0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:59:15.476 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.504+0000 7f27f7e00640 1 -- 172.21.15.97:0/3429679434 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27ec041020 con 0x7f2800077750 2024-03-27T10:59:15.476 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.504+0000 7f27f7e00640 1 -- 172.21.15.97:0/3429679434 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f27ec0035f0 con 0x7f2800077750 2024-03-27T10:59:15.476 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.504+0000 7f2806000640 1 -- 172.21.15.97:0/3429679434 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28001a6180 con 0x7f2800077750 2024-03-27T10:59:15.476 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.504+0000 7f27f7e00640 1 -- 172.21.15.97:0/3429679434 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27ec039040 con 0x7f2800077750 2024-03-27T10:59:15.477 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.504+0000 7f2806000640 1 -- 172.21.15.97:0/3429679434 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28001a66c0 con 0x7f2800077750 2024-03-27T10:59:15.481 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.510+0000 7f27f7e00640 1 -- 172.21.15.97:0/3429679434 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f27ec044020 con 0x7f2800077750 2024-03-27T10:59:15.482 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.510+0000 7f2808d84740 1 Processor -- start 2024-03-27T10:59:15.483 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.511+0000 7f2808d84740 1 -- start start 2024-03-27T10:59:15.483 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.511+0000 7f2808d84740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5654321e2980 0x5654321e2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:59:15.483 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.511+0000 7f2808d84740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5654321e3370 0x5654321e7e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:59:15.483 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.511+0000 7f2808d84740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5654321e84e0 0x5654321ea970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:59:15.483 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.511+0000 7f2808d84740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x565432147d10 con 0x5654321e3370 2024-03-27T10:59:15.483 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.511+0000 7f2808d84740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5654320f0260 con 0x5654321e2980 2024-03-27T10:59:15.483 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.511+0000 7f2808d84740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x565432143cc0 con 0x5654321e84e0 2024-03-27T10:59:15.485 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.512+0000 7f27ff400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5654321e3370 0x5654321e7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:59:15.485 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.512+0000 7f27ff400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5654321e3370 0x5654321e7e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43850/0 (socket says 172.21.15.97:43850) 2024-03-27T10:59:15.485 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.512+0000 7f27ff400640 1 -- 172.21.15.97:0/2185253935 learned_addr learned my addr 172.21.15.97:0/2185253935 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:59:15.485 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.512+0000 7f27ffe00640 1 --2- 172.21.15.97:0/2185253935 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5654321e2980 0x5654321e2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:59:15.485 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.512+0000 7f27fea00640 1 --2- 172.21.15.97:0/2185253935 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5654321e84e0 0x5654321ea970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:59:15.485 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.512+0000 7f27f7e00640 1 --2- 172.21.15.97:0/3429679434 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f27e40a3730 0x7f27e40a5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:59:15.485 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.513+0000 7f27ff400640 1 -- 172.21.15.97:0/2185253935 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5654321e84e0 msgr2=0x5654321ea970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:59:15.485 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.513+0000 7f27ff400640 1 --2- 172.21.15.97:0/2185253935 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5654321e84e0 0x5654321ea970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:15.485 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.513+0000 7f27f7e00640 1 -- 172.21.15.97:0/3429679434 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f27ec0f14b0 con 0x7f2800077750 2024-03-27T10:59:15.485 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.513+0000 7f27ff400640 1 -- 172.21.15.97:0/2185253935 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5654321e2980 msgr2=0x5654321e2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:59:15.486 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.513+0000 7f27ff400640 1 --2- 172.21.15.97:0/2185253935 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5654321e2980 0x5654321e2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:15.486 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.513+0000 7f27ff400640 1 -- 172.21.15.97:0/2185253935 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654320e7aa0 con 0x5654321e3370 2024-03-27T10:59:15.486 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.513+0000 7f27ffe00640 1 --2- 172.21.15.97:0/3429679434 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f27e40a3730 0x7f27e40a5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:59:15.486 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.513+0000 7f27ff400640 1 --2- 172.21.15.97:0/2185253935 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5654321e3370 0x5654321e7e70 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f27f0010580 tx=0x7f27f0010a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb049cc38b90bbbf server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:59:15.487 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.514+0000 7f27ebe00640 1 -- 172.21.15.97:0/2185253935 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27f000e890 con 0x5654321e3370 2024-03-27T10:59:15.487 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.514+0000 7f27ebe00640 1 -- 172.21.15.97:0/2185253935 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f27f001c480 con 0x5654321e3370 2024-03-27T10:59:15.487 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.514+0000 7f27ebe00640 1 -- 172.21.15.97:0/2185253935 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27f0011680 con 0x5654321e3370 2024-03-27T10:59:15.487 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.515+0000 7f2808d84740 1 -- 172.21.15.97:0/2185253935 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5654321e3370 msgr2=0x5654321e7e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:59:15.487 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.515+0000 7f2808d84740 1 --2- 172.21.15.97:0/2185253935 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5654321e3370 0x5654321e7e70 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f27f0010580 tx=0x7f27f0010a40 comp rx=0 tx=0).stop 2024-03-27T10:59:15.487 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.515+0000 7f27ffe00640 1 --2- 172.21.15.97:0/3429679434 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f27e40a3730 0x7f27e40a5bf0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x5654321e3bd0 tx=0x7f27f8004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:59:15.487 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.515+0000 7f2808d84740 1 -- 172.21.15.97:0/2185253935 shutdown_connections 2024-03-27T10:59:15.487 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.515+0000 7f2808d84740 1 --2- 172.21.15.97:0/2185253935 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5654321e84e0 0x5654321ea970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:15.488 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.515+0000 7f2808d84740 1 --2- 172.21.15.97:0/2185253935 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5654321e3370 0x5654321e7e70 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:15.488 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.515+0000 7f2808d84740 1 --2- 172.21.15.97:0/2185253935 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5654321e2980 0x5654321e2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:15.488 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.515+0000 7f2808d84740 1 -- 172.21.15.97:0/2185253935 >> 172.21.15.97:0/2185253935 conn(0x5654321ddef0 msgr2=0x5654321de2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:59:15.488 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.516+0000 7f2808d84740 1 -- 172.21.15.97:0/2185253935 shutdown_connections 2024-03-27T10:59:15.489 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.516+0000 7f2808d84740 1 -- 172.21.15.97:0/2185253935 wait complete. 2024-03-27T10:59:15.489 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.516+0000 7f2808d84740 1 -- start start 2024-03-27T10:59:15.489 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.516+0000 7f2808d84740 1 Processor -- start 2024-03-27T10:59:15.489 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.517+0000 7f2808d84740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5654321e2980 0x5654321fdcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:59:15.489 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.517+0000 7f2808d84740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5654321e3370 0x5654321fe200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:59:15.489 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.517+0000 7f2808d84740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5654321e84e0 0x565432205240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:59:15.489 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.517+0000 7f2808d84740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5654321fc4a0 con 0x5654321e2980 2024-03-27T10:59:15.489 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.517+0000 7f2808d84740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x565432103630 con 0x5654321e3370 2024-03-27T10:59:15.489 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.517+0000 7f2808d84740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5654321fc620 con 0x5654321e84e0 2024-03-27T10:59:15.490 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.517+0000 7f2808d84740 10 client.0 _ll_register_callbacks cb 0x565432108c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:59:15.490 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.517+0000 7f27fea00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5654321e3370 0x5654321fe200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:59:15.490 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.517+0000 7f27ffe00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5654321e84e0 0x565432205240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:59:15.490 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.517+0000 7f27fea00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5654321e3370 0x5654321fe200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:36790/0 (socket says 172.21.15.97:36790) 2024-03-27T10:59:15.490 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.517+0000 7f27fea00640 1 -- 172.21.15.97:0/2314098267 learned_addr learned my addr 172.21.15.97:0/2314098267 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:59:15.490 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.518+0000 7f27ff400640 1 --2- 172.21.15.97:0/2314098267 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5654321e2980 0x5654321fdcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:59:15.491 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.518+0000 7f27ffe00640 1 -- 172.21.15.97:0/2314098267 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5654321e3370 msgr2=0x5654321fe200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:59:15.491 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.518+0000 7f27ffe00640 1 --2- 172.21.15.97:0/2314098267 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5654321e3370 0x5654321fe200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:15.491 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.518+0000 7f27ffe00640 1 -- 172.21.15.97:0/2314098267 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5654321e2980 msgr2=0x5654321fdcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:59:15.491 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.518+0000 7f27ffe00640 1 --2- 172.21.15.97:0/2314098267 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5654321e2980 0x5654321fdcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:15.492 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.518+0000 7f27ffe00640 1 -- 172.21.15.97:0/2314098267 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56543210f5f0 con 0x5654321e84e0 2024-03-27T10:59:15.492 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.518+0000 7f27ff400640 1 --2- 172.21.15.97:0/2314098267 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5654321e2980 0x5654321fdcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T10:59:15.492 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.519+0000 7f27ffe00640 1 --2- 172.21.15.97:0/2314098267 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5654321e84e0 0x565432205240 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f27f8009fd0 tx=0x7f27f8004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b73eeb3e2a9cbe7 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:59:15.492 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.519+0000 7f27e8c00640 10 client.0 ms_handle_connect on v2:172.21.15.177:3300/0 2024-03-27T10:59:15.492 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.520+0000 7f27e8c00640 1 -- 172.21.15.97:0/2314098267 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27f8020b70 con 0x5654321e84e0 2024-03-27T10:59:15.492 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.520+0000 7f2808d84740 1 -- 172.21.15.97:0/2314098267 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27f001c480 con 0x5654321e84e0 2024-03-27T10:59:15.492 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.520+0000 7f27e8c00640 1 -- 172.21.15.97:0/2314098267 <== mon.1 v2:172.21.15.177:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f27f8020d10 con 0x5654321e84e0 2024-03-27T10:59:15.493 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.521+0000 7f27e8c00640 1 -- 172.21.15.97:0/2314098267 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f27f801e480 con 0x5654321e84e0 2024-03-27T10:59:15.493 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.521+0000 7f2808d84740 10 client.25756 fetch_fsmap learned FSMap version 31 2024-03-27T10:59:15.493 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.521+0000 7f2808d84740 1 -- 172.21.15.97:0/2314098267 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x565431e5c210 con 0x5654321e84e0 2024-03-27T10:59:15.493 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.521+0000 7f27e8c00640 1 -- 172.21.15.97:0/2314098267 <== mon.1 v2:172.21.15.177:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f27f001c480 con 0x5654321e84e0 2024-03-27T10:59:15.493 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.521+0000 7f27e8c00640 1 -- 172.21.15.97:0/2314098267 <== mon.1 v2:172.21.15.177:3300/0 5 ==== fsmap(e 31) ==== 8370+0+0 (secure 0 0 0) 0x7f27f8027070 con 0x5654321e84e0 2024-03-27T10:59:15.494 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.522+0000 7f2808d84740 10 client.25756 fetch_fsmap finished waiting for FSMap version 31 2024-03-27T10:59:15.494 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.522+0000 7f2808d84740 10 client.25756 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T10:59:15.494 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.522+0000 7f2808d84740 20 client.25756 populate_metadata read hostname 'smithi097' 2024-03-27T10:59:15.494 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.522+0000 7f2808d84740 1 --2- 172.21.15.97:0/2314098267 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x565432209cb0 0x56543220c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:59:15.494 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.522+0000 7f2808d84740 4 client.25756 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T10:59:15.494 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.522+0000 7f2808d84740 1 -- 172.21.15.97:0/2314098267 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565432155250 con 0x565432209cb0 2024-03-27T10:59:15.494 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.522+0000 7f27ff400640 1 --2- 172.21.15.97:0/2314098267 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x565432209cb0 0x56543220c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:59:15.494 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.523+0000 7f27ff400640 1 --2- 172.21.15.97:0/2314098267 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x565432209cb0 0x56543220c140 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50441b260d95a3ea server_cookie=c136cc906488284d in_seq=0 out_seq=0 2024-03-27T10:59:15.494 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.523+0000 7f27e8c00640 10 client.25756 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T10:59:15.495 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.524+0000 7f27e8c00640 1 -- 172.21.15.97:0/2314098267 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x565432155250 con 0x565432209cb0 2024-03-27T10:59:15.495 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.524+0000 7f27e8c00640 10 client.25756 handle_command_reply: tid=0 2024-03-27T10:59:15.495 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.524+0000 7f2808d84740 1 client.25756 shutdown 2024-03-27T10:59:15.497 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.524+0000 7f2808d84740 1 -- 172.21.15.97:0/2314098267 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5654321e84e0 msgr2=0x565432205240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:59:15.497 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.524+0000 7f2808d84740 1 --2- 172.21.15.97:0/2314098267 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5654321e84e0 0x565432205240 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f27f8009fd0 tx=0x7f27f8004a40 comp rx=0 tx=0).stop 2024-03-27T10:59:15.497 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.524+0000 7f2808d84740 1 -- 172.21.15.97:0/2314098267 shutdown_connections 2024-03-27T10:59:15.497 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.524+0000 7f2808d84740 1 --2- 172.21.15.97:0/2314098267 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x565432209cb0 0x56543220c140 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:15.497 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.524+0000 7f2808d84740 1 --2- 172.21.15.97:0/2314098267 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5654321e84e0 0x565432205240 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:15.497 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.524+0000 7f2808d84740 1 --2- 172.21.15.97:0/2314098267 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5654321e3370 0x5654321fe200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:15.497 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.524+0000 7f2808d84740 1 --2- 172.21.15.97:0/2314098267 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5654321e2980 0x5654321fdcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:15.497 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.524+0000 7f2808d84740 1 -- 172.21.15.97:0/2314098267 >> 172.21.15.97:0/2314098267 conn(0x5654321ddef0 msgr2=0x5654321df140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:59:15.497 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.525+0000 7f27e8c00640 0 client.25756 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T10:59:15.497 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.525+0000 7f2808d84740 1 -- 172.21.15.97:0/2314098267 shutdown_connections 2024-03-27T10:59:15.497 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.525+0000 7f2808d84740 1 -- 172.21.15.97:0/2314098267 wait complete. 2024-03-27T10:59:15.497 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.525+0000 7f2808d84740 20 client.25756 trim_cache size 0 max 16384 2024-03-27T10:59:15.516 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:59:15 smithi114 ceph-mon[35338]: pgmap v519: 97 pgs: 97 active+clean; 7.9 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 119 MiB/s wr, 74 op/s 2024-03-27T10:59:15.521 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.550+0000 7f2808d84740 1 Processor -- start 2024-03-27T10:59:15.521 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.550+0000 7f2808d84740 1 -- start start 2024-03-27T10:59:15.523 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.550+0000 7f2808d84740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5654321e2980 0x5654321fc9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:59:15.523 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.550+0000 7f2808d84740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5654321e3370 0x565432205200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:59:15.523 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.550+0000 7f2808d84740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5654321e84e0 0x5654321fcf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:59:15.523 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.550+0000 7f2808d84740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f28001a6c00 con 0x5654321e3370 2024-03-27T10:59:15.523 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.550+0000 7f2808d84740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f28001a8800 con 0x5654321e84e0 2024-03-27T10:59:15.523 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.550+0000 7f2808d84740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f28001a72d0 con 0x5654321e2980 2024-03-27T10:59:15.523 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.551+0000 7f27ff400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5654321e2980 0x5654321fc9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:59:15.523 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.551+0000 7f27fea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5654321e3370 0x565432205200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:59:15.523 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.551+0000 7f27fea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5654321e3370 0x565432205200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43858/0 (socket says 172.21.15.97:43858) 2024-03-27T10:59:15.523 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.551+0000 7f27fea00640 1 -- 172.21.15.97:0/2873903871 learned_addr learned my addr 172.21.15.97:0/2873903871 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:59:15.524 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.551+0000 7f27ff400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5654321e2980 0x5654321fc9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.97:49942/0 (socket says 172.21.15.97:49942) 2024-03-27T10:59:15.524 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.551+0000 7f27ffe00640 1 --2- 172.21.15.97:0/2873903871 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5654321e84e0 0x5654321fcf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:59:15.524 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.551+0000 7f27fea00640 1 -- 172.21.15.97:0/2873903871 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5654321e2980 msgr2=0x5654321fc9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:59:15.524 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.551+0000 7f27fea00640 1 --2- 172.21.15.97:0/2873903871 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5654321e2980 0x5654321fc9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:15.524 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.551+0000 7f27fea00640 1 -- 172.21.15.97:0/2873903871 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5654321e84e0 msgr2=0x5654321fcf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:59:15.524 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.551+0000 7f27fea00640 1 --2- 172.21.15.97:0/2873903871 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5654321e84e0 0x5654321fcf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:15.524 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.551+0000 7f27fea00640 1 -- 172.21.15.97:0/2873903871 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565431f28620 con 0x5654321e3370 2024-03-27T10:59:15.525 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.552+0000 7f27fea00640 1 --2- 172.21.15.97:0/2873903871 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5654321e3370 0x565432205200 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f27ec0bd690 tx=0x7f27ec005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c89276a8ed122614 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:59:15.525 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.552+0000 7f27eb400640 1 -- 172.21.15.97:0/2873903871 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27ec0044e0 con 0x5654321e3370 2024-03-27T10:59:15.525 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.552+0000 7f27eb400640 1 -- 172.21.15.97:0/2873903871 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f27ec0be090 con 0x5654321e3370 2024-03-27T10:59:15.525 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.552+0000 7f27eb400640 1 -- 172.21.15.97:0/2873903871 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27ec003b10 con 0x5654321e3370 2024-03-27T10:59:15.525 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.553+0000 7f2808d84740 1 -- 172.21.15.97:0/2873903871 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5654321e3370 msgr2=0x565432205200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:59:15.526 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.553+0000 7f2808d84740 1 --2- 172.21.15.97:0/2873903871 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5654321e3370 0x565432205200 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f27ec0bd690 tx=0x7f27ec005f90 comp rx=0 tx=0).stop 2024-03-27T10:59:15.526 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.554+0000 7f2808d84740 1 -- 172.21.15.97:0/2873903871 shutdown_connections 2024-03-27T10:59:15.526 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.554+0000 7f2808d84740 1 --2- 172.21.15.97:0/2873903871 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5654321e84e0 0x5654321fcf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:15.526 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.554+0000 7f2808d84740 1 --2- 172.21.15.97:0/2873903871 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5654321e3370 0x565432205200 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:15.526 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.554+0000 7f2808d84740 1 --2- 172.21.15.97:0/2873903871 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5654321e2980 0x5654321fc9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:15.526 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.554+0000 7f2808d84740 1 -- 172.21.15.97:0/2873903871 >> 172.21.15.97:0/2873903871 conn(0x5654321ddef0 msgr2=0x5654321de2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:59:15.526 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.554+0000 7f2808d84740 1 -- 172.21.15.97:0/2873903871 shutdown_connections 2024-03-27T10:59:15.526 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.554+0000 7f2808d84740 1 -- 172.21.15.97:0/2873903871 wait complete. 2024-03-27T10:59:15.527 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.554+0000 7f2808d84740 1 -- start start 2024-03-27T10:59:15.527 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.555+0000 7f2808d84740 1 Processor -- start 2024-03-27T10:59:15.527 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.555+0000 7f2808d84740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5654321e2980 0x5654322026c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:59:15.527 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.555+0000 7f2808d84740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5654321e3370 0x565432202c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:59:15.527 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.555+0000 7f2808d84740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5654321e84e0 0x565432203bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:59:15.527 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.555+0000 7f2808d84740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f28001a8070 con 0x5654321e2980 2024-03-27T10:59:15.528 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.555+0000 7f2808d84740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f28001a79a0 con 0x5654321e3370 2024-03-27T10:59:15.528 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.555+0000 7f2808d84740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f280010a2a0 con 0x5654321e84e0 2024-03-27T10:59:15.528 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.555+0000 7f2808d84740 10 client.0 _ll_register_callbacks cb 0x7f27f800e5f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:59:15.528 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.556+0000 7f27fea00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5654321e3370 0x565432202c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:59:15.528 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.556+0000 7f27fea00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5654321e3370 0x565432202c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:36808/0 (socket says 172.21.15.97:36808) 2024-03-27T10:59:15.528 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.556+0000 7f27fea00640 1 -- 172.21.15.97:0/2428869013 learned_addr learned my addr 172.21.15.97:0/2428869013 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:59:15.528 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.556+0000 7f27ff400640 1 --2- 172.21.15.97:0/2428869013 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5654321e2980 0x5654322026c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:59:15.530 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.556+0000 7f27ffe00640 1 --2- 172.21.15.97:0/2428869013 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5654321e84e0 0x565432203bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:59:15.530 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.556+0000 7f27fea00640 1 -- 172.21.15.97:0/2428869013 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5654321e84e0 msgr2=0x565432203bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:59:15.530 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.556+0000 7f27fea00640 1 --2- 172.21.15.97:0/2428869013 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5654321e84e0 0x565432203bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:15.530 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.556+0000 7f27fea00640 1 -- 172.21.15.97:0/2428869013 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5654321e2980 msgr2=0x5654322026c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:59:15.530 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.556+0000 7f27fea00640 1 --2- 172.21.15.97:0/2428869013 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5654321e2980 0x5654322026c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:15.530 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.556+0000 7f27fea00640 1 -- 172.21.15.97:0/2428869013 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654322041b0 con 0x5654321e3370 2024-03-27T10:59:15.530 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.556+0000 7f27ffe00640 1 --2- 172.21.15.97:0/2428869013 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5654321e84e0 0x565432203bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:59:15.531 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.556+0000 7f27ff400640 1 --2- 172.21.15.97:0/2428869013 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5654321e2980 0x5654322026c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T10:59:15.531 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.557+0000 7f27fea00640 1 --2- 172.21.15.97:0/2428869013 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5654321e3370 0x565432202c00 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f27ec02c180 tx=0x7f27ec0b9d40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e77bab8fb28dc58f server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:59:15.531 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.557+0000 7f27e8c00640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-03-27T10:59:15.531 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.557+0000 7f27e8c00640 1 -- 172.21.15.97:0/2428869013 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27ec0da070 con 0x5654321e3370 2024-03-27T10:59:15.531 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.557+0000 7f2808d84740 1 -- 172.21.15.97:0/2428869013 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27ec0be090 con 0x5654321e3370 2024-03-27T10:59:15.531 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.557+0000 7f27e8c00640 1 -- 172.21.15.97:0/2428869013 <== mon.2 v2:172.21.15.114:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f27ec0f10d0 con 0x5654321e3370 2024-03-27T10:59:15.531 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.557+0000 7f27e8c00640 1 -- 172.21.15.97:0/2428869013 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f27ec0f12d0 con 0x5654321e3370 2024-03-27T10:59:15.531 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.557+0000 7f2808d84740 10 client.36029 fetch_fsmap learned FSMap version 31 2024-03-27T10:59:15.531 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.557+0000 7f2808d84740 1 -- 172.21.15.97:0/2428869013 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5654322054c0 con 0x5654321e3370 2024-03-27T10:59:15.531 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.558+0000 7f27e8c00640 1 -- 172.21.15.97:0/2428869013 <== mon.2 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f27ec0be090 con 0x5654321e3370 2024-03-27T10:59:15.531 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.558+0000 7f27e8c00640 1 -- 172.21.15.97:0/2428869013 <== mon.2 v2:172.21.15.114:3300/0 5 ==== fsmap(e 31) ==== 8370+0+0 (secure 0 0 0) 0x7f27ec03c030 con 0x5654321e3370 2024-03-27T10:59:15.531 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.558+0000 7f2808d84740 10 client.36029 fetch_fsmap finished waiting for FSMap version 31 2024-03-27T10:59:15.531 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.558+0000 7f2808d84740 10 client.36029 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T10:59:15.532 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.558+0000 7f2808d84740 20 client.36029 populate_metadata read hostname 'smithi097' 2024-03-27T10:59:15.532 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.558+0000 7f2808d84740 1 --2- 172.21.15.97:0/2428869013 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x565432270610 0x565432272a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:59:15.532 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.558+0000 7f2808d84740 4 client.36029 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-27T10:59:15.532 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.558+0000 7f2808d84740 1 -- 172.21.15.97:0/2428869013 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56543214f5a0 con 0x565432270610 2024-03-27T10:59:15.532 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.558+0000 7f27ff400640 1 --2- 172.21.15.97:0/2428869013 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x565432270610 0x565432272a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:59:15.532 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.559+0000 7f27ff400640 1 --2- 172.21.15.97:0/2428869013 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x565432270610 0x565432272a30 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd8ef63573546230 server_cookie=2d878f538787daa3 in_seq=0 out_seq=0 2024-03-27T10:59:15.532 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.559+0000 7f27e8c00640 10 client.36029 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T10:59:15.559 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.587+0000 7f27e8c00640 1 -- 172.21.15.97:0/2428869013 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56543214f5a0 con 0x565432270610 2024-03-27T10:59:15.559 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.587+0000 7f27e8c00640 10 client.36029 handle_command_reply: tid=0 2024-03-27T10:59:15.559 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.587+0000 7f2808d84740 1 client.36029 shutdown 2024-03-27T10:59:15.559 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.587+0000 7f2808d84740 1 -- 172.21.15.97:0/2428869013 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5654321e3370 msgr2=0x565432202c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:59:15.559 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.587+0000 7f2808d84740 1 --2- 172.21.15.97:0/2428869013 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5654321e3370 0x565432202c00 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f27ec02c180 tx=0x7f27ec0b9d40 comp rx=0 tx=0).stop 2024-03-27T10:59:15.560 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.588+0000 7f2808d84740 1 -- 172.21.15.97:0/2428869013 shutdown_connections 2024-03-27T10:59:15.560 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.588+0000 7f2808d84740 1 --2- 172.21.15.97:0/2428869013 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x565432270610 0x565432272a30 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:15.560 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.588+0000 7f2808d84740 1 --2- 172.21.15.97:0/2428869013 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5654321e84e0 0x565432203bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:15.560 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.588+0000 7f2808d84740 1 --2- 172.21.15.97:0/2428869013 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5654321e3370 0x565432202c00 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:15.560 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.588+0000 7f2808d84740 1 --2- 172.21.15.97:0/2428869013 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5654321e2980 0x5654322026c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:15.561 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.588+0000 7f2808d84740 1 -- 172.21.15.97:0/2428869013 >> 172.21.15.97:0/2428869013 conn(0x5654321ddef0 msgr2=0x5654321fc9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:59:15.561 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.588+0000 7f27e8c00640 0 client.36029 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T10:59:15.561 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.588+0000 7f2808d84740 1 -- 172.21.15.97:0/2428869013 shutdown_connections 2024-03-27T10:59:15.561 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.588+0000 7f2808d84740 1 -- 172.21.15.97:0/2428869013 wait complete. 2024-03-27T10:59:15.561 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.588+0000 7f2808d84740 20 client.36029 trim_cache size 0 max 16384 2024-03-27T10:59:15.561 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-27T10:59:15.561 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-27T10:59:15.561 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-27T10:59:15.561 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-27T10:59:15.562 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.589+0000 7f27f5200640 1 -- 172.21.15.97:0/3429679434 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f27e40a3730 msgr2=0x7f27e40a5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:59:15.562 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.589+0000 7f27f5200640 1 --2- 172.21.15.97:0/3429679434 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f27e40a3730 0x7f27e40a5bf0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x5654321e3bd0 tx=0x7f27f8004ab0 comp rx=0 tx=0).stop 2024-03-27T10:59:15.562 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.589+0000 7f27f5200640 1 -- 172.21.15.97:0/3429679434 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2800077750 msgr2=0x7f280019e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:59:15.562 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.589+0000 7f27f5200640 1 --2- 172.21.15.97:0/3429679434 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2800077750 0x7f280019e810 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f27ec02c0b0 tx=0x7f27ec0042f0 comp rx=0 tx=0).stop 2024-03-27T10:59:15.562 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.590+0000 7f27f5200640 1 -- 172.21.15.97:0/3429679434 shutdown_connections 2024-03-27T10:59:15.562 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.590+0000 7f27f5200640 1 --2- 172.21.15.97:0/3429679434 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f28000780f0 0x7f28001a5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:15.562 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.590+0000 7f27f5200640 1 --2- 172.21.15.97:0/3429679434 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2800077750 0x7f280019e810 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:15.563 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.590+0000 7f27f5200640 1 --2- 172.21.15.97:0/3429679434 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f27e40a3730 0x7f27e40a5bf0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:15.563 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.590+0000 7f27f5200640 1 --2- 172.21.15.97:0/3429679434 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2800076510 0x7f280019e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:15.563 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.590+0000 7f27f5200640 1 -- 172.21.15.97:0/3429679434 >> 172.21.15.97:0/3429679434 conn(0x7f28000ff310 msgr2=0x7f2800100e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:59:15.563 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.590+0000 7f27f5200640 1 -- 172.21.15.97:0/3429679434 shutdown_connections 2024-03-27T10:59:15.563 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:15.590+0000 7f27f5200640 1 -- 172.21.15.97:0/3429679434 wait complete. 2024-03-27T10:59:15.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:59:15 smithi097 ceph-mon[31859]: pgmap v519: 97 pgs: 97 active+clean; 7.9 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 119 MiB/s wr, 74 op/s 2024-03-27T10:59:15.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:59:15 smithi177 ceph-mon[35294]: pgmap v519: 97 pgs: 97 active+clean; 7.9 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 119 MiB/s wr, 74 op/s 2024-03-27T10:59:17.516 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:59:17 smithi114 ceph-mon[35338]: pgmap v520: 97 pgs: 97 active+clean; 8.1 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 122 MiB/s wr, 72 op/s 2024-03-27T10:59:17.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:59:17 smithi097 ceph-mon[31859]: pgmap v520: 97 pgs: 97 active+clean; 8.1 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 122 MiB/s wr, 72 op/s 2024-03-27T10:59:17.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:59:17 smithi177 ceph-mon[35294]: pgmap v520: 97 pgs: 97 active+clean; 8.1 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 122 MiB/s wr, 72 op/s 2024-03-27T10:59:19.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:59:19 smithi097 ceph-mon[31859]: pgmap v521: 97 pgs: 97 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 127 MiB/s wr, 74 op/s 2024-03-27T10:59:19.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:59:19 smithi177 ceph-mon[35294]: pgmap v521: 97 pgs: 97 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 127 MiB/s wr, 74 op/s 2024-03-27T10:59:19.766 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:59:19 smithi114 ceph-mon[35338]: pgmap v521: 97 pgs: 97 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 127 MiB/s wr, 74 op/s 2024-03-27T10:59:21.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:59:21 smithi097 ceph-mon[31859]: pgmap v522: 97 pgs: 97 active+clean; 8.7 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 135 MiB/s wr, 82 op/s 2024-03-27T10:59:21.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:59:21 smithi177 ceph-mon[35294]: pgmap v522: 97 pgs: 97 active+clean; 8.7 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 135 MiB/s wr, 82 op/s 2024-03-27T10:59:21.766 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:59:21 smithi114 ceph-mon[35338]: pgmap v522: 97 pgs: 97 active+clean; 8.7 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 135 MiB/s wr, 82 op/s 2024-03-27T10:59:23.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:59:23 smithi097 ceph-mon[31859]: pgmap v523: 97 pgs: 97 active+clean; 9.0 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 123 MiB/s wr, 72 op/s 2024-03-27T10:59:23.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:59:23 smithi177 ceph-mon[35294]: pgmap v523: 97 pgs: 97 active+clean; 9.0 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 123 MiB/s wr, 72 op/s 2024-03-27T10:59:23.767 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:59:23 smithi114 ceph-mon[35338]: pgmap v523: 97 pgs: 97 active+clean; 9.0 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 123 MiB/s wr, 72 op/s 2024-03-27T10:59:25.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:59:25 smithi097 ceph-mon[31859]: pgmap v524: 97 pgs: 97 active+clean; 9.2 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 131 MiB/s wr, 79 op/s 2024-03-27T10:59:25.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:59:25 smithi177 ceph-mon[35294]: pgmap v524: 97 pgs: 97 active+clean; 9.2 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 131 MiB/s wr, 79 op/s 2024-03-27T10:59:25.766 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:59:25 smithi114 ceph-mon[35338]: pgmap v524: 97 pgs: 97 active+clean; 9.2 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 131 MiB/s wr, 79 op/s 2024-03-27T10:59:27.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:59:27 smithi097 ceph-mon[31859]: pgmap v525: 97 pgs: 97 active+clean; 9.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 128 MiB/s wr, 76 op/s 2024-03-27T10:59:27.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:59:27 smithi177 ceph-mon[35294]: pgmap v525: 97 pgs: 97 active+clean; 9.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 128 MiB/s wr, 76 op/s 2024-03-27T10:59:27.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:59:27 smithi114 ceph-mon[35338]: pgmap v525: 97 pgs: 97 active+clean; 9.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 128 MiB/s wr, 76 op/s 2024-03-27T10:59:28.775 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.803+0000 7f5e6e800640 1 -- 172.21.15.97:0/4184624797 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5e68109840 msgr2=0x7f5e6810e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:59:28.775 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.803+0000 7f5e6e800640 1 --2- 172.21.15.97:0/4184624797 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5e68109840 0x7f5e6810e4b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5e54009860 tx=0x7f5e5402bb70 comp rx=0 tx=0).stop 2024-03-27T10:59:28.775 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.803+0000 7f5e6e800640 1 -- 172.21.15.97:0/4184624797 shutdown_connections 2024-03-27T10:59:28.775 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.803+0000 7f5e6e800640 1 --2- 172.21.15.97:0/4184624797 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5e68109840 0x7f5e6810e4b0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:28.775 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.803+0000 7f5e6e800640 1 --2- 172.21.15.97:0/4184624797 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e68073760 0x7f5e68109180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:28.775 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.803+0000 7f5e6e800640 1 --2- 172.21.15.97:0/4184624797 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e68072e00 0x7f5e68073220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:28.775 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.803+0000 7f5e6e800640 1 -- 172.21.15.97:0/4184624797 >> 172.21.15.97:0/4184624797 conn(0x7f5e6806e9c0 msgr2=0x7f5e68070e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:59:28.775 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.804+0000 7f5e6e800640 1 -- 172.21.15.97:0/4184624797 shutdown_connections 2024-03-27T10:59:28.775 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.804+0000 7f5e6e800640 1 -- 172.21.15.97:0/4184624797 wait complete. 2024-03-27T10:59:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.806+0000 7f5e6e800640 1 Processor -- start 2024-03-27T10:59:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.806+0000 7f5e6e800640 1 -- start start 2024-03-27T10:59:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.806+0000 7f5e6e800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e68072e00 0x7f5e681a6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:59:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.806+0000 7f5e6e800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e68073760 0x7f5e681a6e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:59:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.806+0000 7f5e6e800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5e68109840 0x7f5e681aded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:59:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.806+0000 7f5e6e800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5e6811c2e0 con 0x7f5e68072e00 2024-03-27T10:59:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.806+0000 7f5e6e800640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f5e6811c160 con 0x7f5e68109840 2024-03-27T10:59:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.806+0000 7f5e6e800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f5e6811c460 con 0x7f5e68073760 2024-03-27T10:59:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.807+0000 7f5e6ca00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e68073760 0x7f5e681a6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:59:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.807+0000 7f5e6ca00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e68073760 0x7f5e681a6e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.97:57750/0 (socket says 172.21.15.97:57750) 2024-03-27T10:59:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.807+0000 7f5e6ca00640 1 -- 172.21.15.97:0/667744121 learned_addr learned my addr 172.21.15.97:0/667744121 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:59:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.807+0000 7f5e6ca00640 1 -- 172.21.15.97:0/667744121 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5e68109840 msgr2=0x7f5e681aded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:59:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.807+0000 7f5e6ca00640 1 --2- 172.21.15.97:0/667744121 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5e68109840 0x7f5e681aded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.807+0000 7f5e6ca00640 1 -- 172.21.15.97:0/667744121 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e68072e00 msgr2=0x7f5e681a6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:59:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.807+0000 7f5e6ca00640 1 --2- 172.21.15.97:0/667744121 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e68072e00 0x7f5e681a6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.807+0000 7f5e6ca00640 1 -- 172.21.15.97:0/667744121 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e681ae540 con 0x7f5e68073760 2024-03-27T10:59:28.781 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.808+0000 7f5e6ca00640 1 --2- 172.21.15.97:0/667744121 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e68073760 0x7f5e681a6e60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5e5c00e9c0 tx=0x7f5e5c00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:59:28.781 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.808+0000 7f5e66000640 1 -- 172.21.15.97:0/667744121 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e5c00ccc0 con 0x7f5e68073760 2024-03-27T10:59:28.781 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.808+0000 7f5e6e800640 1 -- 172.21.15.97:0/667744121 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e681ae830 con 0x7f5e68073760 2024-03-27T10:59:28.781 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.808+0000 7f5e66000640 1 -- 172.21.15.97:0/667744121 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5e5c0054d0 con 0x7f5e68073760 2024-03-27T10:59:28.781 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.808+0000 7f5e66000640 1 -- 172.21.15.97:0/667744121 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e5c010430 con 0x7f5e68073760 2024-03-27T10:59:28.781 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.808+0000 7f5e6e800640 1 -- 172.21.15.97:0/667744121 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e681aed80 con 0x7f5e68073760 2024-03-27T10:59:28.783 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.813+0000 7f5e72c2d740 1 Processor -- start 2024-03-27T10:59:28.784 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.813+0000 7f5e72c2d740 1 -- start start 2024-03-27T10:59:28.784 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.814+0000 7f5e66000640 1 -- 172.21.15.97:0/667744121 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f5e5c0105d0 con 0x7f5e68073760 2024-03-27T10:59:28.785 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.814+0000 7f5e72c2d740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55726fd6ab60 0x55726fd6af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:59:28.785 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.814+0000 7f5e72c2d740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55726fd6b550 0x55726fd70050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:59:28.785 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.814+0000 7f5e72c2d740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55726fd706c0 0x55726fd72b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:59:28.785 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.814+0000 7f5e72c2d740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55726fccfd10 con 0x55726fd6ab60 2024-03-27T10:59:28.785 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.814+0000 7f5e72c2d740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55726fc78260 con 0x55726fd706c0 2024-03-27T10:59:28.785 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.814+0000 7f5e72c2d740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55726fccbcc0 con 0x55726fd6b550 2024-03-27T10:59:28.786 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.815+0000 7f5e6de00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55726fd6ab60 0x55726fd6af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:59:28.786 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.815+0000 7f5e6de00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55726fd6ab60 0x55726fd6af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33724/0 (socket says 172.21.15.97:33724) 2024-03-27T10:59:28.786 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.815+0000 7f5e6de00640 1 -- 172.21.15.97:0/1918173469 learned_addr learned my addr 172.21.15.97:0/1918173469 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:59:28.786 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.815+0000 7f5e6d400640 1 --2- 172.21.15.97:0/1918173469 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55726fd6b550 0x55726fd70050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:59:28.786 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.815+0000 7f5e6ca00640 1 --2- 172.21.15.97:0/1918173469 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55726fd706c0 0x55726fd72b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:59:28.787 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.815+0000 7f5e6de00640 1 -- 172.21.15.97:0/1918173469 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55726fd6b550 msgr2=0x55726fd70050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:59:28.787 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.815+0000 7f5e6de00640 1 --2- 172.21.15.97:0/1918173469 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55726fd6b550 0x55726fd70050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:28.787 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.815+0000 7f5e6de00640 1 -- 172.21.15.97:0/1918173469 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55726fd706c0 msgr2=0x55726fd72b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:59:28.787 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.815+0000 7f5e6de00640 1 --2- 172.21.15.97:0/1918173469 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55726fd706c0 0x55726fd72b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:28.787 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.815+0000 7f5e6de00640 1 -- 172.21.15.97:0/1918173469 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55726fc6faa0 con 0x55726fd6ab60 2024-03-27T10:59:28.787 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.816+0000 7f5e6de00640 1 --2- 172.21.15.97:0/1918173469 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55726fd6ab60 0x55726fd6af80 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f5e54005b60 tx=0x7f5e54005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efdd8e5735e57253 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:59:28.787 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.816+0000 7f5e66000640 1 --2- 172.21.15.97:0/667744121 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f5e500a3680 0x7f5e500a5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:59:28.787 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.816+0000 7f5e66000640 1 -- 172.21.15.97:0/667744121 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f5e5c0cffa0 con 0x7f5e68073760 2024-03-27T10:59:28.787 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.816+0000 7f5e5a600640 1 -- 172.21.15.97:0/1918173469 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e54039040 con 0x55726fd6ab60 2024-03-27T10:59:28.788 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.816+0000 7f5e5a600640 1 -- 172.21.15.97:0/1918173469 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5e54034440 con 0x55726fd6ab60 2024-03-27T10:59:28.789 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.816+0000 7f5e5a600640 1 -- 172.21.15.97:0/1918173469 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e54039040 con 0x55726fd6ab60 2024-03-27T10:59:28.789 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.816+0000 7f5e6d400640 1 --2- 172.21.15.97:0/667744121 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f5e500a3680 0x7f5e500a5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:59:28.789 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.817+0000 7f5e72c2d740 1 -- 172.21.15.97:0/1918173469 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55726fd6ab60 msgr2=0x55726fd6af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:59:28.789 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.817+0000 7f5e6d400640 1 --2- 172.21.15.97:0/667744121 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f5e500a3680 0x7f5e500a5b40 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x55726fd6b950 tx=0x7f5e600075c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:59:28.789 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.817+0000 7f5e72c2d740 1 --2- 172.21.15.97:0/1918173469 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55726fd6ab60 0x55726fd6af80 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f5e54005b60 tx=0x7f5e54005c90 comp rx=0 tx=0).stop 2024-03-27T10:59:28.789 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.817+0000 7f5e72c2d740 1 -- 172.21.15.97:0/1918173469 shutdown_connections 2024-03-27T10:59:28.789 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.817+0000 7f5e72c2d740 1 --2- 172.21.15.97:0/1918173469 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55726fd706c0 0x55726fd72b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:28.789 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.817+0000 7f5e72c2d740 1 --2- 172.21.15.97:0/1918173469 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55726fd6b550 0x55726fd70050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:28.789 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.817+0000 7f5e72c2d740 1 --2- 172.21.15.97:0/1918173469 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55726fd6ab60 0x55726fd6af80 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:28.789 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.817+0000 7f5e72c2d740 1 -- 172.21.15.97:0/1918173469 >> 172.21.15.97:0/1918173469 conn(0x55726fd660d0 msgr2=0x55726fd664d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:59:28.789 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.818+0000 7f5e72c2d740 1 -- 172.21.15.97:0/1918173469 shutdown_connections 2024-03-27T10:59:28.789 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.818+0000 7f5e72c2d740 1 -- 172.21.15.97:0/1918173469 wait complete. 2024-03-27T10:59:28.789 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.818+0000 7f5e72c2d740 1 -- start start 2024-03-27T10:59:28.790 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.818+0000 7f5e72c2d740 1 Processor -- start 2024-03-27T10:59:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.819+0000 7f5e72c2d740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55726fd6ab60 0x55726fd74f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:59:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.819+0000 7f5e72c2d740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55726fd6b550 0x55726fd75490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:59:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.819+0000 7f5e72c2d740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55726fd706c0 0x55726fd7c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:59:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.819+0000 7f5e72c2d740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55726fd73610 con 0x55726fd6b550 2024-03-27T10:59:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.819+0000 7f5e72c2d740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55726fc8b630 con 0x55726fd6ab60 2024-03-27T10:59:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.819+0000 7f5e72c2d740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55726fd73790 con 0x55726fd706c0 2024-03-27T10:59:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.819+0000 7f5e72c2d740 10 client.0 _ll_register_callbacks cb 0x55726fc90c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:59:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.819+0000 7f5e6d400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55726fd6ab60 0x55726fd74f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:59:28.792 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.819+0000 7f5e6d400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55726fd6ab60 0x55726fd74f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:42828/0 (socket says 172.21.15.97:42828) 2024-03-27T10:59:28.792 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.819+0000 7f5e6d400640 1 -- 172.21.15.97:0/4060704207 learned_addr learned my addr 172.21.15.97:0/4060704207 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:59:28.792 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.820+0000 7f5e6ca00640 1 --2- 172.21.15.97:0/4060704207 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55726fd6b550 0x55726fd75490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:59:28.792 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.820+0000 7f5e6de00640 1 --2- 172.21.15.97:0/4060704207 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55726fd706c0 0x55726fd7c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:59:28.792 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.820+0000 7f5e6d400640 1 -- 172.21.15.97:0/4060704207 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55726fd706c0 msgr2=0x55726fd7c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:59:28.792 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.820+0000 7f5e6d400640 1 --2- 172.21.15.97:0/4060704207 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55726fd706c0 0x55726fd7c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:28.792 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.820+0000 7f5e6d400640 1 -- 172.21.15.97:0/4060704207 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55726fd6b550 msgr2=0x55726fd75490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:59:28.792 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.820+0000 7f5e6d400640 1 --2- 172.21.15.97:0/4060704207 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55726fd6b550 0x55726fd75490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:28.792 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.820+0000 7f5e6d400640 1 -- 172.21.15.97:0/4060704207 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55726fc975f0 con 0x55726fd6ab60 2024-03-27T10:59:28.792 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.820+0000 7f5e6ca00640 1 --2- 172.21.15.97:0/4060704207 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55726fd6b550 0x55726fd75490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T10:59:28.792 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.820+0000 7f5e6de00640 1 --2- 172.21.15.97:0/4060704207 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55726fd706c0 0x55726fd7c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:59:28.792 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.820+0000 7f5e6d400640 1 --2- 172.21.15.97:0/4060704207 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55726fd6ab60 0x55726fd74f50 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5e60010e30 tx=0x7f5e60004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47caee1f18840233 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:59:28.792 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.820+0000 7f5e4ea00640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-03-27T10:59:28.793 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.821+0000 7f5e4ea00640 1 -- 172.21.15.97:0/4060704207 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e60010e60 con 0x55726fd6ab60 2024-03-27T10:59:28.793 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.821+0000 7f5e4ea00640 1 -- 172.21.15.97:0/4060704207 <== mon.2 v2:172.21.15.114:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f5e60026680 con 0x55726fd6ab60 2024-03-27T10:59:28.793 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.822+0000 7f5e72c2d740 1 -- 172.21.15.97:0/4060704207 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e54034440 con 0x55726fd6ab60 2024-03-27T10:59:28.793 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.822+0000 7f5e4ea00640 1 -- 172.21.15.97:0/4060704207 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f5e60007880 con 0x55726fd6ab60 2024-03-27T10:59:28.793 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.822+0000 7f5e72c2d740 10 client.36038 fetch_fsmap learned FSMap version 31 2024-03-27T10:59:28.793 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.822+0000 7f5e72c2d740 1 -- 172.21.15.97:0/4060704207 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55726f9e4210 con 0x55726fd6ab60 2024-03-27T10:59:28.793 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.823+0000 7f5e4ea00640 1 -- 172.21.15.97:0/4060704207 <== mon.2 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5e54034440 con 0x55726fd6ab60 2024-03-27T10:59:28.794 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.823+0000 7f5e4ea00640 1 -- 172.21.15.97:0/4060704207 <== mon.2 v2:172.21.15.114:3300/0 5 ==== fsmap(e 31) ==== 8370+0+0 (secure 0 0 0) 0x7f5e600309c0 con 0x55726fd6ab60 2024-03-27T10:59:28.794 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.823+0000 7f5e72c2d740 10 client.36038 fetch_fsmap finished waiting for FSMap version 31 2024-03-27T10:59:28.794 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.823+0000 7f5e72c2d740 10 client.36038 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T10:59:28.794 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.823+0000 7f5e72c2d740 20 client.36038 populate_metadata read hostname 'smithi097' 2024-03-27T10:59:28.794 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.823+0000 7f5e72c2d740 1 --2- 172.21.15.97:0/4060704207 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55726fd80cb0 0x55726fd83170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:59:28.795 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.823+0000 7f5e72c2d740 4 client.36038 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T10:59:28.795 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.823+0000 7f5e72c2d740 1 -- 172.21.15.97:0/4060704207 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55726fcdd250 con 0x55726fd80cb0 2024-03-27T10:59:28.795 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.824+0000 7f5e6ca00640 1 --2- 172.21.15.97:0/4060704207 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55726fd80cb0 0x55726fd83170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:59:28.795 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.824+0000 7f5e6ca00640 1 --2- 172.21.15.97:0/4060704207 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55726fd80cb0 0x55726fd83170 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b4bfc0245ef501 server_cookie=f403c562f71c840d in_seq=0 out_seq=0 2024-03-27T10:59:28.795 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.825+0000 7f5e4ea00640 10 client.36038 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T10:59:28.796 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.826+0000 7f5e4ea00640 1 -- 172.21.15.97:0/4060704207 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55726fcdd250 con 0x55726fd80cb0 2024-03-27T10:59:28.796 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.826+0000 7f5e4ea00640 10 client.36038 handle_command_reply: tid=0 2024-03-27T10:59:28.796 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.826+0000 7f5e72c2d740 1 client.36038 shutdown 2024-03-27T10:59:28.797 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.827+0000 7f5e72c2d740 1 -- 172.21.15.97:0/4060704207 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55726fd6ab60 msgr2=0x55726fd74f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:59:28.797 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.827+0000 7f5e72c2d740 1 --2- 172.21.15.97:0/4060704207 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55726fd6ab60 0x55726fd74f50 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5e60010e30 tx=0x7f5e60004a40 comp rx=0 tx=0).stop 2024-03-27T10:59:28.798 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.827+0000 7f5e72c2d740 1 -- 172.21.15.97:0/4060704207 shutdown_connections 2024-03-27T10:59:28.798 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.827+0000 7f5e72c2d740 1 --2- 172.21.15.97:0/4060704207 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55726fd80cb0 0x55726fd83170 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:28.798 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.827+0000 7f5e72c2d740 1 --2- 172.21.15.97:0/4060704207 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55726fd706c0 0x55726fd7c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:28.798 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.827+0000 7f5e72c2d740 1 --2- 172.21.15.97:0/4060704207 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55726fd6b550 0x55726fd75490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:28.798 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.827+0000 7f5e72c2d740 1 --2- 172.21.15.97:0/4060704207 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55726fd6ab60 0x55726fd74f50 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:28.798 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.827+0000 7f5e72c2d740 1 -- 172.21.15.97:0/4060704207 >> 172.21.15.97:0/4060704207 conn(0x55726fd660d0 msgr2=0x55726fd65310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:59:28.798 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.827+0000 7f5e4ea00640 0 client.36038 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T10:59:28.798 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.827+0000 7f5e72c2d740 1 -- 172.21.15.97:0/4060704207 shutdown_connections 2024-03-27T10:59:28.798 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.827+0000 7f5e72c2d740 1 -- 172.21.15.97:0/4060704207 wait complete. 2024-03-27T10:59:28.799 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.828+0000 7f5e72c2d740 20 client.36038 trim_cache size 0 max 16384 2024-03-27T10:59:28.820 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.849+0000 7f5e72c2d740 1 Processor -- start 2024-03-27T10:59:28.820 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.849+0000 7f5e72c2d740 1 -- start start 2024-03-27T10:59:28.820 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.850+0000 7f5e72c2d740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55726fd6ab60 0x55726fd74b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:59:28.820 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.850+0000 7f5e72c2d740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55726fd6b550 0x55726fd7c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:59:28.820 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.850+0000 7f5e72c2d740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55726fd706c0 0x55726fd750a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:59:28.820 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.850+0000 7f5e72c2d740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5e4800e4f0 con 0x55726fd6b550 2024-03-27T10:59:28.821 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.850+0000 7f5e72c2d740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f5e480100f0 con 0x55726fd6ab60 2024-03-27T10:59:28.821 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.850+0000 7f5e72c2d740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f5e4800ebc0 con 0x55726fd706c0 2024-03-27T10:59:28.821 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.850+0000 7f5e6de00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55726fd706c0 0x55726fd750a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:59:28.822 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.850+0000 7f5e6de00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55726fd706c0 0x55726fd750a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.97:57790/0 (socket says 172.21.15.97:57790) 2024-03-27T10:59:28.822 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.850+0000 7f5e6de00640 1 -- 172.21.15.97:0/1243987342 learned_addr learned my addr 172.21.15.97:0/1243987342 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:59:28.822 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.850+0000 7f5e6d400640 1 --2- 172.21.15.97:0/1243987342 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55726fd6ab60 0x55726fd74b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:59:28.822 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.850+0000 7f5e6ca00640 1 --2- 172.21.15.97:0/1243987342 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55726fd6b550 0x55726fd7c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:59:28.822 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.851+0000 7f5e6de00640 1 -- 172.21.15.97:0/1243987342 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55726fd6ab60 msgr2=0x55726fd74b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:59:28.822 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.851+0000 7f5e6de00640 1 --2- 172.21.15.97:0/1243987342 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55726fd6ab60 0x55726fd74b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:28.822 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.851+0000 7f5e6de00640 1 -- 172.21.15.97:0/1243987342 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55726fd6b550 msgr2=0x55726fd7c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:59:28.822 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.851+0000 7f5e6de00640 1 --2- 172.21.15.97:0/1243987342 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55726fd6b550 0x55726fd7c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:28.822 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.851+0000 7f5e6de00640 1 -- 172.21.15.97:0/1243987342 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55726fab0620 con 0x55726fd706c0 2024-03-27T10:59:28.822 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.851+0000 7f5e6de00640 1 --2- 172.21.15.97:0/1243987342 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55726fd706c0 0x55726fd750a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5e5402c180 tx=0x7f5e540345d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39024cc5a7f44103 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:59:28.823 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.851+0000 7f5e59c00640 1 -- 172.21.15.97:0/1243987342 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e54039040 con 0x55726fd706c0 2024-03-27T10:59:28.823 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.852+0000 7f5e59c00640 1 -- 172.21.15.97:0/1243987342 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5e540346c0 con 0x55726fd706c0 2024-03-27T10:59:28.823 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.852+0000 7f5e59c00640 1 -- 172.21.15.97:0/1243987342 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e5402c510 con 0x55726fd706c0 2024-03-27T10:59:28.823 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.852+0000 7f5e72c2d740 1 -- 172.21.15.97:0/1243987342 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55726fd706c0 msgr2=0x55726fd750a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:59:28.823 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.852+0000 7f5e72c2d740 1 --2- 172.21.15.97:0/1243987342 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55726fd706c0 0x55726fd750a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5e5402c180 tx=0x7f5e540345d0 comp rx=0 tx=0).stop 2024-03-27T10:59:28.823 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.853+0000 7f5e72c2d740 1 -- 172.21.15.97:0/1243987342 shutdown_connections 2024-03-27T10:59:28.823 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.853+0000 7f5e72c2d740 1 --2- 172.21.15.97:0/1243987342 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55726fd706c0 0x55726fd750a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:28.824 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.853+0000 7f5e72c2d740 1 --2- 172.21.15.97:0/1243987342 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55726fd6b550 0x55726fd7c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:28.824 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.853+0000 7f5e72c2d740 1 --2- 172.21.15.97:0/1243987342 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55726fd6ab60 0x55726fd74b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:28.824 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.853+0000 7f5e72c2d740 1 -- 172.21.15.97:0/1243987342 >> 172.21.15.97:0/1243987342 conn(0x55726fd660d0 msgr2=0x55726fd6c910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:59:28.825 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.853+0000 7f5e72c2d740 1 -- 172.21.15.97:0/1243987342 shutdown_connections 2024-03-27T10:59:28.825 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.853+0000 7f5e72c2d740 1 -- 172.21.15.97:0/1243987342 wait complete. 2024-03-27T10:59:28.825 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.853+0000 7f5e72c2d740 1 -- start start 2024-03-27T10:59:28.825 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.853+0000 7f5e72c2d740 1 Processor -- start 2024-03-27T10:59:28.825 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.854+0000 7f5e72c2d740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55726fd6ab60 0x55726fd79830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:59:28.826 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.854+0000 7f5e72c2d740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55726fd6b550 0x55726fd79d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:59:28.826 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.854+0000 7f5e72c2d740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55726fd706c0 0x55726fd7ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:59:28.826 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.855+0000 7f5e72c2d740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5e4800f960 con 0x55726fd6ab60 2024-03-27T10:59:28.826 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.855+0000 7f5e72c2d740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f5e4800f290 con 0x55726fd6b550 2024-03-27T10:59:28.826 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.855+0000 7f5e72c2d740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f5e4800c4d0 con 0x55726fd706c0 2024-03-27T10:59:28.826 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.855+0000 7f5e72c2d740 10 client.0 _ll_register_callbacks cb 0x7f5e60007f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:59:28.826 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.855+0000 7f5e6ca00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55726fd6b550 0x55726fd79d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:59:28.826 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.855+0000 7f5e6ca00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55726fd6b550 0x55726fd79d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:42844/0 (socket says 172.21.15.97:42844) 2024-03-27T10:59:28.826 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.855+0000 7f5e6ca00640 1 -- 172.21.15.97:0/3393939806 learned_addr learned my addr 172.21.15.97:0/3393939806 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:59:28.826 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.855+0000 7f5e6d400640 1 --2- 172.21.15.97:0/3393939806 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55726fd6ab60 0x55726fd79830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:59:28.826 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.855+0000 7f5e6de00640 1 --2- 172.21.15.97:0/3393939806 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55726fd706c0 0x55726fd7ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:59:28.827 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.855+0000 7f5e6ca00640 1 -- 172.21.15.97:0/3393939806 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55726fd706c0 msgr2=0x55726fd7ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:59:28.827 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.855+0000 7f5e6ca00640 1 --2- 172.21.15.97:0/3393939806 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55726fd706c0 0x55726fd7ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:28.827 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.855+0000 7f5e6ca00640 1 -- 172.21.15.97:0/3393939806 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55726fd6ab60 msgr2=0x55726fd79830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:59:28.827 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.855+0000 7f5e6ca00640 1 --2- 172.21.15.97:0/3393939806 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55726fd6ab60 0x55726fd79830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:28.828 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.855+0000 7f5e6ca00640 1 -- 172.21.15.97:0/3393939806 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55726fd7b300 con 0x55726fd6b550 2024-03-27T10:59:28.828 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.855+0000 7f5e6ca00640 1 --2- 172.21.15.97:0/3393939806 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55726fd6b550 0x55726fd79d70 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5e5c004ad0 tx=0x7f5e5c09bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a262c21d79fed5e server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:59:28.828 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.855+0000 7f5e4ea00640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-03-27T10:59:28.828 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.856+0000 7f5e4ea00640 1 -- 172.21.15.97:0/3393939806 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e5c09cb80 con 0x55726fd6b550 2024-03-27T10:59:28.828 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.856+0000 7f5e72c2d740 1 -- 172.21.15.97:0/3393939806 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e540346c0 con 0x55726fd6b550 2024-03-27T10:59:28.828 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.856+0000 7f5e4ea00640 1 -- 172.21.15.97:0/3393939806 <== mon.2 v2:172.21.15.114:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f5e5c09a780 con 0x55726fd6b550 2024-03-27T10:59:28.828 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.856+0000 7f5e4ea00640 1 -- 172.21.15.97:0/3393939806 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f5e5c09c480 con 0x55726fd6b550 2024-03-27T10:59:28.828 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.856+0000 7f5e72c2d740 10 client.36044 fetch_fsmap learned FSMap version 31 2024-03-27T10:59:28.828 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.856+0000 7f5e72c2d740 1 -- 172.21.15.97:0/3393939806 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55726fd7c6d0 con 0x55726fd6b550 2024-03-27T10:59:28.829 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.856+0000 7f5e4ea00640 1 -- 172.21.15.97:0/3393939806 <== mon.2 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5e540346c0 con 0x55726fd6b550 2024-03-27T10:59:28.829 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.857+0000 7f5e4ea00640 1 -- 172.21.15.97:0/3393939806 <== mon.2 v2:172.21.15.114:3300/0 5 ==== fsmap(e 31) ==== 8370+0+0 (secure 0 0 0) 0x7f5e5c098af0 con 0x55726fd6b550 2024-03-27T10:59:28.829 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.857+0000 7f5e72c2d740 10 client.36044 fetch_fsmap finished waiting for FSMap version 31 2024-03-27T10:59:28.829 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.857+0000 7f5e72c2d740 10 client.36044 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T10:59:28.829 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.857+0000 7f5e72c2d740 20 client.36044 populate_metadata read hostname 'smithi097' 2024-03-27T10:59:28.829 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.857+0000 7f5e72c2d740 1 --2- 172.21.15.97:0/3393939806 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55726fde77d0 0x55726fde9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:59:28.829 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.857+0000 7f5e72c2d740 4 client.36044 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-27T10:59:28.829 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.857+0000 7f5e72c2d740 1 -- 172.21.15.97:0/3393939806 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55726fcd75a0 con 0x55726fde77d0 2024-03-27T10:59:28.829 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.857+0000 7f5e6d400640 1 --2- 172.21.15.97:0/3393939806 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55726fde77d0 0x55726fde9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:59:28.829 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.858+0000 7f5e6d400640 1 --2- 172.21.15.97:0/3393939806 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55726fde77d0 0x55726fde9bf0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d734dfbc43ecb2 server_cookie=a9093ab315b74b52 in_seq=0 out_seq=0 2024-03-27T10:59:28.829 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.858+0000 7f5e4ea00640 10 client.36044 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T10:59:28.840 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.869+0000 7f5e4ea00640 1 -- 172.21.15.97:0/3393939806 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55726fcd75a0 con 0x55726fde77d0 2024-03-27T10:59:28.840 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.869+0000 7f5e4ea00640 10 client.36044 handle_command_reply: tid=0 2024-03-27T10:59:28.840 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.870+0000 7f5e72c2d740 1 client.36044 shutdown 2024-03-27T10:59:28.840 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.870+0000 7f5e72c2d740 1 -- 172.21.15.97:0/3393939806 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55726fd6b550 msgr2=0x55726fd79d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:59:28.840 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.870+0000 7f5e72c2d740 1 --2- 172.21.15.97:0/3393939806 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55726fd6b550 0x55726fd79d70 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5e5c004ad0 tx=0x7f5e5c09bea0 comp rx=0 tx=0).stop 2024-03-27T10:59:28.841 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.870+0000 7f5e72c2d740 1 -- 172.21.15.97:0/3393939806 shutdown_connections 2024-03-27T10:59:28.841 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.870+0000 7f5e72c2d740 1 --2- 172.21.15.97:0/3393939806 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55726fde77d0 0x55726fde9bf0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:28.841 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.870+0000 7f5e72c2d740 1 --2- 172.21.15.97:0/3393939806 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55726fd706c0 0x55726fd7ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:28.842 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.870+0000 7f5e72c2d740 1 --2- 172.21.15.97:0/3393939806 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55726fd6b550 0x55726fd79d70 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:28.842 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.870+0000 7f5e72c2d740 1 --2- 172.21.15.97:0/3393939806 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55726fd6ab60 0x55726fd79830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:28.842 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.870+0000 7f5e72c2d740 1 -- 172.21.15.97:0/3393939806 >> 172.21.15.97:0/3393939806 conn(0x55726fd660d0 msgr2=0x55726fd69bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:59:28.842 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.870+0000 7f5e4ea00640 0 client.36044 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T10:59:28.842 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.871+0000 7f5e72c2d740 1 -- 172.21.15.97:0/3393939806 shutdown_connections 2024-03-27T10:59:28.842 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.871+0000 7f5e72c2d740 1 -- 172.21.15.97:0/3393939806 wait complete. 2024-03-27T10:59:28.842 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.871+0000 7f5e72c2d740 20 client.36044 trim_cache size 0 max 16384 2024-03-27T10:59:28.842 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-27T10:59:28.842 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-27T10:59:28.842 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-27T10:59:28.843 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-27T10:59:28.843 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.872+0000 7f5e6e800640 1 -- 172.21.15.97:0/667744121 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f5e500a3680 msgr2=0x7f5e500a5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:59:28.843 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.872+0000 7f5e6e800640 1 --2- 172.21.15.97:0/667744121 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f5e500a3680 0x7f5e500a5b40 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x55726fd6b950 tx=0x7f5e600075c0 comp rx=0 tx=0).stop 2024-03-27T10:59:28.843 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.872+0000 7f5e6e800640 1 -- 172.21.15.97:0/667744121 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e68073760 msgr2=0x7f5e681a6e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:59:28.843 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.872+0000 7f5e6e800640 1 --2- 172.21.15.97:0/667744121 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e68073760 0x7f5e681a6e60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5e5c00e9c0 tx=0x7f5e5c00ee80 comp rx=0 tx=0).stop 2024-03-27T10:59:28.843 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.872+0000 7f5e6e800640 1 -- 172.21.15.97:0/667744121 shutdown_connections 2024-03-27T10:59:28.843 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.872+0000 7f5e6e800640 1 --2- 172.21.15.97:0/667744121 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f5e500a3680 0x7f5e500a5b40 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:28.843 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.872+0000 7f5e6e800640 1 --2- 172.21.15.97:0/667744121 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5e68109840 0x7f5e681aded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:28.844 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.872+0000 7f5e6e800640 1 --2- 172.21.15.97:0/667744121 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e68073760 0x7f5e681a6e60 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:28.844 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.872+0000 7f5e6e800640 1 --2- 172.21.15.97:0/667744121 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e68072e00 0x7f5e681a6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:28.844 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.872+0000 7f5e6e800640 1 -- 172.21.15.97:0/667744121 >> 172.21.15.97:0/667744121 conn(0x7f5e6806e9c0 msgr2=0x7f5e680705e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:59:28.844 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.872+0000 7f5e6e800640 1 -- 172.21.15.97:0/667744121 shutdown_connections 2024-03-27T10:59:28.844 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:28.873+0000 7f5e6e800640 1 -- 172.21.15.97:0/667744121 wait complete. 2024-03-27T10:59:29.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:59:29 smithi097 ceph-mon[31859]: pgmap v526: 97 pgs: 97 active+clean; 9.7 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 131 MiB/s wr, 77 op/s 2024-03-27T10:59:29.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:59:29 smithi177 ceph-mon[35294]: pgmap v526: 97 pgs: 97 active+clean; 9.7 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 131 MiB/s wr, 77 op/s 2024-03-27T10:59:29.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:59:29 smithi114 ceph-mon[35338]: pgmap v526: 97 pgs: 97 active+clean; 9.7 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 131 MiB/s wr, 77 op/s 2024-03-27T10:59:31.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:59:31 smithi097 ceph-mon[31859]: pgmap v527: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 138 MiB/s wr, 140 op/s 2024-03-27T10:59:31.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:59:31 smithi177 ceph-mon[35294]: pgmap v527: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 138 MiB/s wr, 140 op/s 2024-03-27T10:59:31.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:59:31 smithi114 ceph-mon[35338]: pgmap v527: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 138 MiB/s wr, 140 op/s 2024-03-27T10:59:33.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:59:33 smithi097 ceph-mon[31859]: pgmap v528: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 119 MiB/s wr, 228 op/s 2024-03-27T10:59:33.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:59:33 smithi177 ceph-mon[35294]: pgmap v528: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 119 MiB/s wr, 228 op/s 2024-03-27T10:59:33.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:59:33 smithi114 ceph-mon[35338]: pgmap v528: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 119 MiB/s wr, 228 op/s 2024-03-27T10:59:35.568 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:59:35 smithi114 ceph-mon[35338]: pgmap v529: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 109 MiB/s wr, 375 op/s 2024-03-27T10:59:35.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:59:35 smithi097 ceph-mon[31859]: pgmap v529: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 109 MiB/s wr, 375 op/s 2024-03-27T10:59:35.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:59:35 smithi177 ceph-mon[35294]: pgmap v529: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 109 MiB/s wr, 375 op/s 2024-03-27T10:59:37.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:59:37 smithi097 ceph-mon[31859]: pgmap v530: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 93 MiB/s wr, 478 op/s 2024-03-27T10:59:37.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:59:37 smithi177 ceph-mon[35294]: pgmap v530: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 93 MiB/s wr, 478 op/s 2024-03-27T10:59:37.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:59:37 smithi114 ceph-mon[35338]: pgmap v530: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 93 MiB/s wr, 478 op/s 2024-03-27T10:59:39.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:59:39 smithi097 ceph-mon[31859]: pgmap v531: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 87 MiB/s wr, 630 op/s 2024-03-27T10:59:39.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:59:39 smithi177 ceph-mon[35294]: pgmap v531: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 87 MiB/s wr, 630 op/s 2024-03-27T10:59:39.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:59:39 smithi114 ceph-mon[35338]: pgmap v531: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 87 MiB/s wr, 630 op/s 2024-03-27T10:59:41.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:59:41 smithi097 ceph-mon[31859]: pgmap v532: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 80 MiB/s wr, 850 op/s 2024-03-27T10:59:41.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:59:41 smithi177 ceph-mon[35294]: pgmap v532: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 80 MiB/s wr, 850 op/s 2024-03-27T10:59:41.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:59:41 smithi114 ceph-mon[35338]: pgmap v532: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 80 MiB/s wr, 850 op/s 2024-03-27T10:59:42.048 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.077+0000 7f6540600640 1 -- 172.21.15.97:0/3950316412 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6538100b60 msgr2=0x7f6538100fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:59:42.048 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.077+0000 7f6540600640 1 --2- 172.21.15.97:0/3950316412 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6538100b60 0x7f6538100fc0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f6524009900 tx=0x7f652402bc40 comp rx=0 tx=0).stop 2024-03-27T10:59:42.048 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.077+0000 7f6540600640 1 -- 172.21.15.97:0/3950316412 shutdown_connections 2024-03-27T10:59:42.048 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.077+0000 7f6540600640 1 --2- 172.21.15.97:0/3950316412 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6538101500 0x7f6538101da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:42.048 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.077+0000 7f6540600640 1 --2- 172.21.15.97:0/3950316412 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6538100b60 0x7f6538100fc0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:42.048 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.077+0000 7f6540600640 1 --2- 172.21.15.97:0/3950316412 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f65380ffa30 0x7f65380ffe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:42.048 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.077+0000 7f6540600640 1 -- 172.21.15.97:0/3950316412 >> 172.21.15.97:0/3950316412 conn(0x7f65380fb330 msgr2=0x7f65380fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:59:42.049 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.078+0000 7f6540600640 1 -- 172.21.15.97:0/3950316412 shutdown_connections 2024-03-27T10:59:42.049 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.078+0000 7f6540600640 1 -- 172.21.15.97:0/3950316412 wait complete. 2024-03-27T10:59:42.050 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.080+0000 7f6540600640 1 Processor -- start 2024-03-27T10:59:42.051 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.080+0000 7f6540600640 1 -- start start 2024-03-27T10:59:42.051 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.080+0000 7f6540600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f65380ffa30 0x7f6538199f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:59:42.051 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.080+0000 7f6540600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6538100b60 0x7f653819a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:59:42.051 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.080+0000 7f6540600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6538101500 0x7f65381a14d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:59:42.051 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.081+0000 7f653e600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6538101500 0x7f65381a14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:59:42.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.081+0000 7f653e600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6538101500 0x7f65381a14d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:48558/0 (socket says 172.21.15.97:48558) 2024-03-27T10:59:42.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.082+0000 7f653dc00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f65380ffa30 0x7f6538199f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:59:42.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.082+0000 7f653dc00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f65380ffa30 0x7f6538199f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.97:59994/0 (socket says 172.21.15.97:59994) 2024-03-27T10:59:42.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.082+0000 7f653e600640 1 -- 172.21.15.97:0/4283739525 learned_addr learned my addr 172.21.15.97:0/4283739525 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:59:42.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.082+0000 7f6540600640 1 -- 172.21.15.97:0/4283739525 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f653810f940 con 0x7f6538100b60 2024-03-27T10:59:42.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.082+0000 7f6540600640 1 -- 172.21.15.97:0/4283739525 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f653810f7c0 con 0x7f6538101500 2024-03-27T10:59:42.053 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.082+0000 7f6540600640 1 -- 172.21.15.97:0/4283739525 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f653810fac0 con 0x7f65380ffa30 2024-03-27T10:59:42.053 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.082+0000 7f653d200640 1 --2- 172.21.15.97:0/4283739525 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6538100b60 0x7f653819a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:59:42.053 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.083+0000 7f653e600640 1 -- 172.21.15.97:0/4283739525 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f65380ffa30 msgr2=0x7f6538199f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:59:42.053 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.083+0000 7f653e600640 1 --2- 172.21.15.97:0/4283739525 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f65380ffa30 0x7f6538199f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:42.053 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.083+0000 7f653e600640 1 -- 172.21.15.97:0/4283739525 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6538100b60 msgr2=0x7f653819a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:59:42.054 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.083+0000 7f653e600640 1 --2- 172.21.15.97:0/4283739525 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6538100b60 0x7f653819a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:42.054 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.083+0000 7f653e600640 1 -- 172.21.15.97:0/4283739525 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65381a1b40 con 0x7f6538101500 2024-03-27T10:59:42.054 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.084+0000 7f653e600640 1 --2- 172.21.15.97:0/4283739525 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6538101500 0x7f65381a14d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f653000b970 tx=0x7f653000be30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:59:42.055 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.085+0000 7f6536000640 1 -- 172.21.15.97:0/4283739525 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6530019070 con 0x7f6538101500 2024-03-27T10:59:42.055 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.085+0000 7f6536000640 1 -- 172.21.15.97:0/4283739525 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6530014440 con 0x7f6538101500 2024-03-27T10:59:42.055 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.085+0000 7f6536000640 1 -- 172.21.15.97:0/4283739525 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65300075d0 con 0x7f6538101500 2024-03-27T10:59:42.055 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.085+0000 7f6540600640 1 -- 172.21.15.97:0/4283739525 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65381a1e30 con 0x7f6538101500 2024-03-27T10:59:42.056 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.085+0000 7f6540600640 1 -- 172.21.15.97:0/4283739525 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65381a22c0 con 0x7f6538101500 2024-03-27T10:59:42.061 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.090+0000 7f6543372740 1 Processor -- start 2024-03-27T10:59:42.061 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.090+0000 7f6536000640 1 -- 172.21.15.97:0/4283739525 <== mon.2 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f653001c070 con 0x7f6538101500 2024-03-27T10:59:42.064 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.091+0000 7f6543372740 1 -- start start 2024-03-27T10:59:42.064 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.091+0000 7f6543372740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a84af0a980 0x55a84af0ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:59:42.064 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.091+0000 7f6543372740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a84af0b370 0x55a84af0fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:59:42.064 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.091+0000 7f6543372740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a84af104e0 0x55a84af12970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:59:42.064 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.091+0000 7f6543372740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55a84ae6fd10 con 0x55a84af0b370 2024-03-27T10:59:42.064 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.091+0000 7f6543372740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55a84ae18260 con 0x55a84af0a980 2024-03-27T10:59:42.065 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.091+0000 7f6543372740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55a84ae6bcc0 con 0x55a84af104e0 2024-03-27T10:59:42.065 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.093+0000 7f653dc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a84af0b370 0x55a84af0fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:59:42.065 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.093+0000 7f6536000640 1 --2- 172.21.15.97:0/4283739525 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f65200a38a0 0x7f65200a5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:59:42.065 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.093+0000 7f653dc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a84af0b370 0x55a84af0fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58352/0 (socket says 172.21.15.97:58352) 2024-03-27T10:59:42.065 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.093+0000 7f6536000640 1 -- 172.21.15.97:0/4283739525 <== mon.2 v2:172.21.15.114:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f65300cf1b0 con 0x7f6538101500 2024-03-27T10:59:42.065 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.093+0000 7f653dc00640 1 -- 172.21.15.97:0/1826897876 learned_addr learned my addr 172.21.15.97:0/1826897876 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:59:42.065 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.094+0000 7f653dc00640 1 -- 172.21.15.97:0/1826897876 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a84af104e0 msgr2=0x55a84af12970 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-27T10:59:42.065 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.094+0000 7f653dc00640 1 --2- 172.21.15.97:0/1826897876 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a84af104e0 0x55a84af12970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:42.065 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.094+0000 7f653dc00640 1 -- 172.21.15.97:0/1826897876 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a84af0a980 msgr2=0x55a84af0ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:59:42.066 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.094+0000 7f653dc00640 1 --2- 172.21.15.97:0/1826897876 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a84af0a980 0x55a84af0ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:42.066 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.094+0000 7f653dc00640 1 -- 172.21.15.97:0/1826897876 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a84ae0faa0 con 0x55a84af0b370 2024-03-27T10:59:42.066 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.095+0000 7f653dc00640 1 --2- 172.21.15.97:0/1826897876 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a84af0b370 0x55a84af0fe70 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f652c009cf0 tx=0x7f652c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfb1f4faf9d01360 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:59:42.067 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.096+0000 7f653e600640 1 --2- 172.21.15.97:0/4283739525 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f65200a38a0 0x7f65200a5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:59:42.067 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.097+0000 7f652aa00640 1 -- 172.21.15.97:0/1826897876 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f652c01e070 con 0x55a84af0b370 2024-03-27T10:59:42.068 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.097+0000 7f652aa00640 1 -- 172.21.15.97:0/1826897876 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f652c019ae0 con 0x55a84af0b370 2024-03-27T10:59:42.068 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.097+0000 7f652aa00640 1 -- 172.21.15.97:0/1826897876 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f652c00e990 con 0x55a84af0b370 2024-03-27T10:59:42.068 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.097+0000 7f653e600640 1 --2- 172.21.15.97:0/4283739525 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f65200a38a0 0x7f65200a5d60 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f653009b670 tx=0x7f653000e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:59:42.069 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.098+0000 7f6543372740 1 -- 172.21.15.97:0/1826897876 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a84af0b370 msgr2=0x55a84af0fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:59:42.069 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.098+0000 7f6543372740 1 --2- 172.21.15.97:0/1826897876 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a84af0b370 0x55a84af0fe70 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f652c009cf0 tx=0x7f652c011810 comp rx=0 tx=0).stop 2024-03-27T10:59:42.070 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.099+0000 7f6543372740 1 -- 172.21.15.97:0/1826897876 shutdown_connections 2024-03-27T10:59:42.070 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.099+0000 7f6543372740 1 --2- 172.21.15.97:0/1826897876 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a84af104e0 0x55a84af12970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:42.070 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.099+0000 7f6543372740 1 --2- 172.21.15.97:0/1826897876 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a84af0b370 0x55a84af0fe70 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:42.070 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.099+0000 7f6543372740 1 --2- 172.21.15.97:0/1826897876 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a84af0a980 0x55a84af0ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:42.070 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.099+0000 7f6543372740 1 -- 172.21.15.97:0/1826897876 >> 172.21.15.97:0/1826897876 conn(0x55a84af05ef0 msgr2=0x55a84af062f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:59:42.071 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.100+0000 7f6543372740 1 -- 172.21.15.97:0/1826897876 shutdown_connections 2024-03-27T10:59:42.071 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.100+0000 7f6543372740 1 -- 172.21.15.97:0/1826897876 wait complete. 2024-03-27T10:59:42.071 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.100+0000 7f6543372740 1 -- start start 2024-03-27T10:59:42.071 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.101+0000 7f6543372740 1 Processor -- start 2024-03-27T10:59:42.072 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.101+0000 7f6543372740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a84af0a980 0x55a84af2e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:59:42.072 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.101+0000 7f6543372740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a84af0b370 0x55a84af2eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:59:42.072 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.101+0000 7f6543372740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a84af104e0 0x55a84af35ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:59:42.072 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.101+0000 7f6543372740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55a84af244a0 con 0x55a84af0b370 2024-03-27T10:59:42.072 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.101+0000 7f6543372740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55a84ae2b630 con 0x55a84af0a980 2024-03-27T10:59:42.072 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.101+0000 7f6543372740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55a84af24620 con 0x55a84af104e0 2024-03-27T10:59:42.072 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.101+0000 7f6543372740 10 client.0 _ll_register_callbacks cb 0x55a84ae30c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:59:42.072 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.102+0000 7f653d200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a84af104e0 0x55a84af35ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:59:42.073 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.102+0000 7f653d200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a84af104e0 0x55a84af35ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.97:60012/0 (socket says 172.21.15.97:60012) 2024-03-27T10:59:42.073 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.102+0000 7f653d200640 1 -- 172.21.15.97:0/1361348620 learned_addr learned my addr 172.21.15.97:0/1361348620 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:59:42.073 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.102+0000 7f653d200640 1 --2- 172.21.15.97:0/1361348620 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a84af0a980 0x55a84af2e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:59:42.073 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.102+0000 7f653dc00640 1 --2- 172.21.15.97:0/1361348620 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a84af0b370 0x55a84af2eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:59:42.074 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.102+0000 7f653d200640 1 -- 172.21.15.97:0/1361348620 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a84af0a980 msgr2=0x55a84af2e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:59:42.074 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.102+0000 7f653d200640 1 --2- 172.21.15.97:0/1361348620 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a84af0a980 0x55a84af2e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:42.074 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.102+0000 7f653d200640 1 -- 172.21.15.97:0/1361348620 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a84af0b370 msgr2=0x55a84af2eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:59:42.074 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.102+0000 7f653d200640 1 --2- 172.21.15.97:0/1361348620 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a84af0b370 0x55a84af2eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:42.074 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.102+0000 7f653d200640 1 -- 172.21.15.97:0/1361348620 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a84ae375f0 con 0x55a84af104e0 2024-03-27T10:59:42.074 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.102+0000 7f653dc00640 1 --2- 172.21.15.97:0/1361348620 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a84af0b370 0x55a84af2eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:59:42.074 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.103+0000 7f653d200640 1 --2- 172.21.15.97:0/1361348620 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a84af104e0 0x55a84af35ae0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6524009230 tx=0x7f6524009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8b198c14483f713 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:59:42.074 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.103+0000 7f651f400640 10 client.0 ms_handle_connect on v2:172.21.15.177:3300/0 2024-03-27T10:59:42.075 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.103+0000 7f651f400640 1 -- 172.21.15.97:0/1361348620 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f652402cc90 con 0x55a84af104e0 2024-03-27T10:59:42.075 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.103+0000 7f651f400640 1 -- 172.21.15.97:0/1361348620 <== mon.1 v2:172.21.15.177:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f6524005460 con 0x55a84af104e0 2024-03-27T10:59:42.075 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.103+0000 7f6543372740 1 -- 172.21.15.97:0/1361348620 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f652c019ae0 con 0x55a84af104e0 2024-03-27T10:59:42.075 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.103+0000 7f651f400640 1 -- 172.21.15.97:0/1361348620 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f6524005660 con 0x55a84af104e0 2024-03-27T10:59:42.075 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.103+0000 7f6543372740 10 client.25780 fetch_fsmap learned FSMap version 31 2024-03-27T10:59:42.075 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.104+0000 7f6543372740 1 -- 172.21.15.97:0/1361348620 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55a84ab84210 con 0x55a84af104e0 2024-03-27T10:59:42.075 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.104+0000 7f651f400640 1 -- 172.21.15.97:0/1361348620 <== mon.1 v2:172.21.15.177:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f652c019ae0 con 0x55a84af104e0 2024-03-27T10:59:42.075 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.104+0000 7f651f400640 1 -- 172.21.15.97:0/1361348620 <== mon.1 v2:172.21.15.177:3300/0 5 ==== fsmap(e 31) ==== 8370+0+0 (secure 0 0 0) 0x7f65240038f0 con 0x55a84af104e0 2024-03-27T10:59:42.076 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.104+0000 7f6543372740 10 client.25780 fetch_fsmap finished waiting for FSMap version 31 2024-03-27T10:59:42.076 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.104+0000 7f6543372740 10 client.25780 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T10:59:42.076 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.104+0000 7f6543372740 20 client.25780 populate_metadata read hostname 'smithi097' 2024-03-27T10:59:42.076 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.104+0000 7f6543372740 1 --2- 172.21.15.97:0/1361348620 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55a84af3a3e0 0x55a84af3c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:59:42.076 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.104+0000 7f6543372740 4 client.25780 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T10:59:42.076 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.104+0000 7f6543372740 1 -- 172.21.15.97:0/1361348620 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a84ae7d250 con 0x55a84af3a3e0 2024-03-27T10:59:42.076 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.105+0000 7f653dc00640 1 --2- 172.21.15.97:0/1361348620 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55a84af3a3e0 0x55a84af3c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:59:42.077 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.106+0000 7f653dc00640 1 --2- 172.21.15.97:0/1361348620 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55a84af3a3e0 0x55a84af3c8a0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1958b2abb73acb8 server_cookie=54d0349b8f1c250b in_seq=0 out_seq=0 2024-03-27T10:59:42.077 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.106+0000 7f651f400640 10 client.25780 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T10:59:42.078 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.107+0000 7f651f400640 1 -- 172.21.15.97:0/1361348620 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55a84ae7d250 con 0x55a84af3a3e0 2024-03-27T10:59:42.078 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.107+0000 7f651f400640 10 client.25780 handle_command_reply: tid=0 2024-03-27T10:59:42.079 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.107+0000 7f6543372740 1 client.25780 shutdown 2024-03-27T10:59:42.079 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.108+0000 7f6543372740 1 -- 172.21.15.97:0/1361348620 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a84af104e0 msgr2=0x55a84af35ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:59:42.079 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.108+0000 7f6543372740 1 --2- 172.21.15.97:0/1361348620 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a84af104e0 0x55a84af35ae0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6524009230 tx=0x7f6524009260 comp rx=0 tx=0).stop 2024-03-27T10:59:42.080 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.109+0000 7f6543372740 1 -- 172.21.15.97:0/1361348620 shutdown_connections 2024-03-27T10:59:42.080 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.109+0000 7f6543372740 1 --2- 172.21.15.97:0/1361348620 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55a84af3a3e0 0x55a84af3c8a0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:42.080 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.109+0000 7f6543372740 1 --2- 172.21.15.97:0/1361348620 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a84af104e0 0x55a84af35ae0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:42.080 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.109+0000 7f6543372740 1 --2- 172.21.15.97:0/1361348620 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a84af0b370 0x55a84af2eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:42.080 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.109+0000 7f6543372740 1 --2- 172.21.15.97:0/1361348620 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a84af0a980 0x55a84af2e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:42.080 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.109+0000 7f6543372740 1 -- 172.21.15.97:0/1361348620 >> 172.21.15.97:0/1361348620 conn(0x55a84af05ef0 msgr2=0x55a84af050f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:59:42.080 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.109+0000 7f651f400640 0 client.25780 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T10:59:42.080 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.109+0000 7f6543372740 1 -- 172.21.15.97:0/1361348620 shutdown_connections 2024-03-27T10:59:42.080 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.109+0000 7f6543372740 1 -- 172.21.15.97:0/1361348620 wait complete. 2024-03-27T10:59:42.080 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.109+0000 7f6543372740 20 client.25780 trim_cache size 0 max 16384 2024-03-27T10:59:42.110 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.140+0000 7f6543372740 1 Processor -- start 2024-03-27T10:59:42.110 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.140+0000 7f6543372740 1 -- start start 2024-03-27T10:59:42.111 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.141+0000 7f6543372740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a84af0a980 0x55a84af249c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:59:42.111 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.141+0000 7f6543372740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a84af0b370 0x55a84af35aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:59:42.111 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.141+0000 7f6543372740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a84af104e0 0x55a84af04860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:59:42.111 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.141+0000 7f6543372740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f653806fc40 con 0x55a84af0b370 2024-03-27T10:59:42.111 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.141+0000 7f6543372740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f6538071840 con 0x55a84af104e0 2024-03-27T10:59:42.111 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.141+0000 7f6543372740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6538070310 con 0x55a84af0a980 2024-03-27T10:59:42.112 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.142+0000 7f653dc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a84af0b370 0x55a84af35aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:59:42.112 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.142+0000 7f653dc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a84af0b370 0x55a84af35aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58364/0 (socket says 172.21.15.97:58364) 2024-03-27T10:59:42.112 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.142+0000 7f653dc00640 1 -- 172.21.15.97:0/2937860478 learned_addr learned my addr 172.21.15.97:0/2937860478 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:59:42.112 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.142+0000 7f653d200640 1 --2- 172.21.15.97:0/2937860478 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a84af0a980 0x55a84af249c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:59:42.112 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.142+0000 7f653d200640 1 --2- 172.21.15.97:0/2937860478 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a84af104e0 0x55a84af04860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:59:42.113 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.142+0000 7f653dc00640 1 -- 172.21.15.97:0/2937860478 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a84af0a980 msgr2=0x55a84af249c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:59:42.113 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.142+0000 7f653dc00640 1 --2- 172.21.15.97:0/2937860478 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a84af0a980 0x55a84af249c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:42.113 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.142+0000 7f653dc00640 1 -- 172.21.15.97:0/2937860478 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a84af104e0 msgr2=0x55a84af04860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:59:42.113 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.142+0000 7f653dc00640 1 --2- 172.21.15.97:0/2937860478 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a84af104e0 0x55a84af04860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:42.114 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.143+0000 7f653dc00640 1 -- 172.21.15.97:0/2937860478 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a84ac50620 con 0x55a84af0b370 2024-03-27T10:59:42.114 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.143+0000 7f653d200640 1 --2- 172.21.15.97:0/2937860478 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a84af104e0 0x55a84af04860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T10:59:42.114 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.143+0000 7f653dc00640 1 --2- 172.21.15.97:0/2937860478 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a84af0b370 0x55a84af35aa0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f652c0058c0 tx=0x7f652c0195c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11d2b2f7787cb5fa server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:59:42.114 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.143+0000 7f652a000640 1 -- 172.21.15.97:0/2937860478 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f652c01e040 con 0x55a84af0b370 2024-03-27T10:59:42.114 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.144+0000 7f652a000640 1 -- 172.21.15.97:0/2937860478 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f652c0054f0 con 0x55a84af0b370 2024-03-27T10:59:42.114 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.144+0000 7f652a000640 1 -- 172.21.15.97:0/2937860478 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f652c02a4f0 con 0x55a84af0b370 2024-03-27T10:59:42.115 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.145+0000 7f6543372740 1 -- 172.21.15.97:0/2937860478 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a84af0b370 msgr2=0x55a84af35aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:59:42.115 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.145+0000 7f6543372740 1 --2- 172.21.15.97:0/2937860478 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a84af0b370 0x55a84af35aa0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f652c0058c0 tx=0x7f652c0195c0 comp rx=0 tx=0).stop 2024-03-27T10:59:42.116 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.145+0000 7f6543372740 1 -- 172.21.15.97:0/2937860478 shutdown_connections 2024-03-27T10:59:42.116 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.145+0000 7f6543372740 1 --2- 172.21.15.97:0/2937860478 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a84af104e0 0x55a84af04860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:42.116 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.145+0000 7f6543372740 1 --2- 172.21.15.97:0/2937860478 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a84af0b370 0x55a84af35aa0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:42.116 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.145+0000 7f6543372740 1 --2- 172.21.15.97:0/2937860478 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a84af0a980 0x55a84af249c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:42.116 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.145+0000 7f6543372740 1 -- 172.21.15.97:0/2937860478 >> 172.21.15.97:0/2937860478 conn(0x55a84af05ef0 msgr2=0x55a84af3c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:59:42.116 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.145+0000 7f6543372740 1 -- 172.21.15.97:0/2937860478 shutdown_connections 2024-03-27T10:59:42.116 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.145+0000 7f6543372740 1 -- 172.21.15.97:0/2937860478 wait complete. 2024-03-27T10:59:42.117 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.146+0000 7f6543372740 1 -- start start 2024-03-27T10:59:42.117 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.146+0000 7f6543372740 1 Processor -- start 2024-03-27T10:59:42.117 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.147+0000 7f6543372740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a84af0a980 0x55a84af331c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:59:42.117 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.147+0000 7f6543372740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a84af0b370 0x55a84af33700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:59:42.117 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.147+0000 7f6543372740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a84af104e0 0x55a84af34670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:59:42.117 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.147+0000 7f6543372740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f65380710b0 con 0x55a84af0a980 2024-03-27T10:59:42.118 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.147+0000 7f6543372740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f65380709e0 con 0x55a84af104e0 2024-03-27T10:59:42.118 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.147+0000 7f6543372740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f653806dc20 con 0x55a84af0b370 2024-03-27T10:59:42.118 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.147+0000 7f6543372740 10 client.0 _ll_register_callbacks cb 0x7f6524009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:59:42.119 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.147+0000 7f653d200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a84af0a980 0x55a84af331c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:59:42.119 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.147+0000 7f653d200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a84af0a980 0x55a84af331c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58376/0 (socket says 172.21.15.97:58376) 2024-03-27T10:59:42.119 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.147+0000 7f653d200640 1 -- 172.21.15.97:0/3089762062 learned_addr learned my addr 172.21.15.97:0/3089762062 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:59:42.119 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.147+0000 7f653d200640 1 --2- 172.21.15.97:0/3089762062 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a84af104e0 0x55a84af34670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:59:42.119 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.148+0000 7f653dc00640 1 --2- 172.21.15.97:0/3089762062 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a84af0b370 0x55a84af33700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:59:42.119 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.148+0000 7f653d200640 1 -- 172.21.15.97:0/3089762062 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a84af0b370 msgr2=0x55a84af33700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:59:42.119 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.148+0000 7f653d200640 1 --2- 172.21.15.97:0/3089762062 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a84af0b370 0x55a84af33700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:42.119 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.148+0000 7f653d200640 1 -- 172.21.15.97:0/3089762062 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a84af104e0 msgr2=0x55a84af34670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:59:42.119 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.148+0000 7f653d200640 1 --2- 172.21.15.97:0/3089762062 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a84af104e0 0x55a84af34670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:42.119 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.148+0000 7f653d200640 1 -- 172.21.15.97:0/3089762062 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a84af34c60 con 0x55a84af0a980 2024-03-27T10:59:42.119 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.148+0000 7f653d200640 1 --2- 172.21.15.97:0/3089762062 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a84af0a980 0x55a84af331c0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f6524003d60 tx=0x7f6524009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a2e72bbb62575fb server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:59:42.120 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.149+0000 7f651f400640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T10:59:42.121 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.149+0000 7f651f400640 1 -- 172.21.15.97:0/3089762062 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65240048d0 con 0x55a84af0a980 2024-03-27T10:59:42.121 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.149+0000 7f651f400640 1 -- 172.21.15.97:0/3089762062 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f6524004a70 con 0x55a84af0a980 2024-03-27T10:59:42.121 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.149+0000 7f6543372740 1 -- 172.21.15.97:0/3089762062 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f652c0054f0 con 0x55a84af0a980 2024-03-27T10:59:42.121 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.149+0000 7f651f400640 1 -- 172.21.15.97:0/3089762062 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f6524004c70 con 0x55a84af0a980 2024-03-27T10:59:42.121 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.149+0000 7f6543372740 10 client.16410 fetch_fsmap learned FSMap version 31 2024-03-27T10:59:42.121 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.149+0000 7f6543372740 1 -- 172.21.15.97:0/3089762062 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55a84afa07e0 con 0x55a84af0a980 2024-03-27T10:59:42.121 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.150+0000 7f651f400640 1 -- 172.21.15.97:0/3089762062 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f652c0054f0 con 0x55a84af0a980 2024-03-27T10:59:42.121 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.150+0000 7f651f400640 1 -- 172.21.15.97:0/3089762062 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 31) ==== 8370+0+0 (secure 0 0 0) 0x7f6524051480 con 0x55a84af0a980 2024-03-27T10:59:42.121 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.150+0000 7f6543372740 10 client.16410 fetch_fsmap finished waiting for FSMap version 31 2024-03-27T10:59:42.121 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.150+0000 7f6543372740 10 client.16410 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T10:59:42.121 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.150+0000 7f6543372740 20 client.16410 populate_metadata read hostname 'smithi097' 2024-03-27T10:59:42.121 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.150+0000 7f6543372740 1 --2- 172.21.15.97:0/3089762062 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55a84afa10a0 0x55a84afa34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:59:42.121 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.150+0000 7f6543372740 4 client.16410 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-27T10:59:42.121 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.150+0000 7f6543372740 1 -- 172.21.15.97:0/3089762062 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a84af25190 con 0x55a84afa10a0 2024-03-27T10:59:42.122 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.152+0000 7f653dc00640 1 --2- 172.21.15.97:0/3089762062 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55a84afa10a0 0x55a84afa34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:59:42.123 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.152+0000 7f653dc00640 1 --2- 172.21.15.97:0/3089762062 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55a84afa10a0 0x55a84afa34c0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6eba0d3a368e3a96 server_cookie=b482c72599b887b0 in_seq=0 out_seq=0 2024-03-27T10:59:42.123 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.152+0000 7f651f400640 10 client.16410 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T10:59:42.129 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.158+0000 7f651f400640 1 -- 172.21.15.97:0/3089762062 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a84af25190 con 0x55a84afa10a0 2024-03-27T10:59:42.129 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.158+0000 7f651f400640 10 client.16410 handle_command_reply: tid=0 2024-03-27T10:59:42.129 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.158+0000 7f6543372740 1 client.16410 shutdown 2024-03-27T10:59:42.129 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.159+0000 7f6543372740 1 -- 172.21.15.97:0/3089762062 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a84af0a980 msgr2=0x55a84af331c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:59:42.129 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.159+0000 7f6543372740 1 --2- 172.21.15.97:0/3089762062 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a84af0a980 0x55a84af331c0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f6524003d60 tx=0x7f6524009260 comp rx=0 tx=0).stop 2024-03-27T10:59:42.130 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.159+0000 7f6543372740 1 -- 172.21.15.97:0/3089762062 shutdown_connections 2024-03-27T10:59:42.130 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.159+0000 7f6543372740 1 --2- 172.21.15.97:0/3089762062 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55a84afa10a0 0x55a84afa34c0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:42.130 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.159+0000 7f6543372740 1 --2- 172.21.15.97:0/3089762062 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a84af104e0 0x55a84af34670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:42.130 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.159+0000 7f6543372740 1 --2- 172.21.15.97:0/3089762062 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a84af0b370 0x55a84af33700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:42.130 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.159+0000 7f6543372740 1 --2- 172.21.15.97:0/3089762062 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a84af0a980 0x55a84af331c0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:42.130 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.159+0000 7f651f400640 0 client.16410 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T10:59:42.130 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.159+0000 7f6543372740 1 -- 172.21.15.97:0/3089762062 >> 172.21.15.97:0/3089762062 conn(0x55a84af05ef0 msgr2=0x55a84af09150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:59:42.131 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.160+0000 7f6543372740 1 -- 172.21.15.97:0/3089762062 shutdown_connections 2024-03-27T10:59:42.131 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.160+0000 7f6543372740 1 -- 172.21.15.97:0/3089762062 wait complete. 2024-03-27T10:59:42.131 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.160+0000 7f6543372740 20 client.16410 trim_cache size 0 max 16384 2024-03-27T10:59:42.131 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-27T10:59:42.131 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-27T10:59:42.131 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-27T10:59:42.131 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-27T10:59:42.132 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.162+0000 7f6540600640 1 -- 172.21.15.97:0/4283739525 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f65200a38a0 msgr2=0x7f65200a5d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:59:42.132 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.162+0000 7f6540600640 1 --2- 172.21.15.97:0/4283739525 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f65200a38a0 0x7f65200a5d60 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f653009b670 tx=0x7f653000e040 comp rx=0 tx=0).stop 2024-03-27T10:59:42.132 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.162+0000 7f6540600640 1 -- 172.21.15.97:0/4283739525 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6538101500 msgr2=0x7f65381a14d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:59:42.133 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.162+0000 7f6540600640 1 --2- 172.21.15.97:0/4283739525 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6538101500 0x7f65381a14d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f653000b970 tx=0x7f653000be30 comp rx=0 tx=0).stop 2024-03-27T10:59:42.133 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.162+0000 7f6540600640 1 -- 172.21.15.97:0/4283739525 shutdown_connections 2024-03-27T10:59:42.133 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.162+0000 7f6540600640 1 --2- 172.21.15.97:0/4283739525 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f65200a38a0 0x7f65200a5d60 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:42.133 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.162+0000 7f6540600640 1 --2- 172.21.15.97:0/4283739525 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6538101500 0x7f65381a14d0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:42.133 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.162+0000 7f6540600640 1 --2- 172.21.15.97:0/4283739525 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6538100b60 0x7f653819a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:42.133 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.162+0000 7f6540600640 1 --2- 172.21.15.97:0/4283739525 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f65380ffa30 0x7f6538199f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:42.133 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.162+0000 7f6540600640 1 -- 172.21.15.97:0/4283739525 >> 172.21.15.97:0/4283739525 conn(0x7f65380fb330 msgr2=0x7f65380fc9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:59:42.134 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.163+0000 7f6540600640 1 -- 172.21.15.97:0/4283739525 shutdown_connections 2024-03-27T10:59:42.134 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:42.164+0000 7f6540600640 1 -- 172.21.15.97:0/4283739525 wait complete. 2024-03-27T10:59:42.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:59:42 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:59:42.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:59:42 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:59:42.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:59:42 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T10:59:43.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:59:43 smithi097 ceph-mon[31859]: pgmap v533: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 61 MiB/s wr, 911 op/s 2024-03-27T10:59:43.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:59:43 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:59:43.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:59:43 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:59:43.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:59:43 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:59:43.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:59:43 smithi177 ceph-mon[35294]: pgmap v533: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 61 MiB/s wr, 911 op/s 2024-03-27T10:59:43.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:59:43 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:59:43.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:59:43 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:59:43.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:59:43 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:59:43.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:59:43 smithi114 ceph-mon[35338]: pgmap v533: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 61 MiB/s wr, 911 op/s 2024-03-27T10:59:43.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:59:43 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T10:59:43.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:59:43 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T10:59:43.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:59:43 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T10:59:45.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:59:45 smithi097 ceph-mon[31859]: pgmap v534: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 59 MiB/s wr, 953 op/s 2024-03-27T10:59:45.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:59:45 smithi177 ceph-mon[35294]: pgmap v534: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 59 MiB/s wr, 953 op/s 2024-03-27T10:59:45.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:59:45 smithi114 ceph-mon[35338]: pgmap v534: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 59 MiB/s wr, 953 op/s 2024-03-27T10:59:47.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:59:47 smithi097 ceph-mon[31859]: pgmap v535: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 56 MiB/s wr, 906 op/s 2024-03-27T10:59:47.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:59:47 smithi177 ceph-mon[35294]: pgmap v535: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 56 MiB/s wr, 906 op/s 2024-03-27T10:59:47.766 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:59:47 smithi114 ceph-mon[35338]: pgmap v535: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 56 MiB/s wr, 906 op/s 2024-03-27T10:59:49.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:59:49 smithi097 ceph-mon[31859]: pgmap v536: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 59 MiB/s wr, 949 op/s 2024-03-27T10:59:49.677 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:59:49 smithi177 ceph-mon[35294]: pgmap v536: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 59 MiB/s wr, 949 op/s 2024-03-27T10:59:49.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:59:49 smithi114 ceph-mon[35338]: pgmap v536: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 59 MiB/s wr, 949 op/s 2024-03-27T10:59:51.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:59:51 smithi097 ceph-mon[31859]: pgmap v537: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 64 MiB/s wr, 1.04k op/s 2024-03-27T10:59:51.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:59:51 smithi177 ceph-mon[35294]: pgmap v537: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 64 MiB/s wr, 1.04k op/s 2024-03-27T10:59:51.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:59:51 smithi114 ceph-mon[35338]: pgmap v537: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 64 MiB/s wr, 1.04k op/s 2024-03-27T10:59:53.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:59:53 smithi097 ceph-mon[31859]: pgmap v538: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 59 MiB/s wr, 946 op/s 2024-03-27T10:59:53.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:59:53 smithi177 ceph-mon[35294]: pgmap v538: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 59 MiB/s wr, 946 op/s 2024-03-27T10:59:53.766 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:59:53 smithi114 ceph-mon[35338]: pgmap v538: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 59 MiB/s wr, 946 op/s 2024-03-27T10:59:55.326 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.354+0000 7fb2eb000640 1 -- 172.21.15.97:0/2741595551 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2e40ffa30 msgr2=0x7fb2e40ffe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:59:55.326 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.354+0000 7fb2eb000640 1 --2- 172.21.15.97:0/2741595551 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2e40ffa30 0x7fb2e40ffe50 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc009ca0 tx=0x7fb2dc02c100 comp rx=0 tx=0).stop 2024-03-27T10:59:55.327 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.355+0000 7fb2eb000640 1 -- 172.21.15.97:0/2741595551 shutdown_connections 2024-03-27T10:59:55.327 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.355+0000 7fb2eb000640 1 --2- 172.21.15.97:0/2741595551 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2e4101500 0x7fb2e4105dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:55.327 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.355+0000 7fb2eb000640 1 --2- 172.21.15.97:0/2741595551 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2e4100b60 0x7fb2e4100fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:55.327 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.355+0000 7fb2eb000640 1 --2- 172.21.15.97:0/2741595551 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2e40ffa30 0x7fb2e40ffe50 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:55.327 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.355+0000 7fb2eb000640 1 -- 172.21.15.97:0/2741595551 >> 172.21.15.97:0/2741595551 conn(0x7fb2e40fb330 msgr2=0x7fb2e40fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:59:55.327 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.355+0000 7fb2eb000640 1 -- 172.21.15.97:0/2741595551 shutdown_connections 2024-03-27T10:59:55.327 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.356+0000 7fb2eb000640 1 -- 172.21.15.97:0/2741595551 wait complete. 2024-03-27T10:59:55.328 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.356+0000 7fb2eb000640 1 Processor -- start 2024-03-27T10:59:55.328 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.357+0000 7fb2eb000640 1 -- start start 2024-03-27T10:59:55.329 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.357+0000 7fb2eb000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2e40ffa30 0x7fb2e419e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:59:55.329 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.357+0000 7fb2eb000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2e4100b60 0x7fb2e419e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:59:55.329 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.357+0000 7fb2eb000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2e4101500 0x7fb2e41a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:59:55.329 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.357+0000 7fb2eb000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e4113cc0 con 0x7fb2e4101500 2024-03-27T10:59:55.329 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.357+0000 7fb2eb000640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e4113b40 con 0x7fb2e40ffa30 2024-03-27T10:59:55.329 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.357+0000 7fb2eb000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e4113e40 con 0x7fb2e4100b60 2024-03-27T10:59:55.330 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.358+0000 7fb2e3e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2e40ffa30 0x7fb2e419e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:59:55.330 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.358+0000 7fb2e3e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2e40ffa30 0x7fb2e419e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:35550/0 (socket says 172.21.15.97:35550) 2024-03-27T10:59:55.330 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.358+0000 7fb2e3e00640 1 -- 172.21.15.97:0/805325017 learned_addr learned my addr 172.21.15.97:0/805325017 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:59:55.330 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.358+0000 7fb2e9000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2e4101500 0x7fb2e41a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:59:55.330 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.358+0000 7fb2e3400640 1 --2- 172.21.15.97:0/805325017 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2e4100b60 0x7fb2e419e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:59:55.330 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.358+0000 7fb2e3e00640 1 -- 172.21.15.97:0/805325017 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2e4100b60 msgr2=0x7fb2e419e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:59:55.331 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.358+0000 7fb2e3e00640 1 --2- 172.21.15.97:0/805325017 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2e4100b60 0x7fb2e419e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:55.331 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.358+0000 7fb2e3e00640 1 -- 172.21.15.97:0/805325017 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2e4101500 msgr2=0x7fb2e41a5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:59:55.331 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.359+0000 7fb2e3e00640 1 --2- 172.21.15.97:0/805325017 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2e4101500 0x7fb2e41a5870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:55.331 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.359+0000 7fb2e3e00640 1 -- 172.21.15.97:0/805325017 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2e41a5ee0 con 0x7fb2e40ffa30 2024-03-27T10:59:55.331 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.359+0000 7fb2e9000640 1 --2- 172.21.15.97:0/805325017 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2e4101500 0x7fb2e41a5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T10:59:55.331 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.359+0000 7fb2e3e00640 1 --2- 172.21.15.97:0/805325017 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2e40ffa30 0x7fb2e419e2c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc032380 tx=0x7fb2dc0093f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:59:55.332 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.359+0000 7fb2e0c00640 1 -- 172.21.15.97:0/805325017 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc039040 con 0x7fb2e40ffa30 2024-03-27T10:59:55.332 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.360+0000 7fb2e0c00640 1 -- 172.21.15.97:0/805325017 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb2dc044070 con 0x7fb2e40ffa30 2024-03-27T10:59:55.332 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.360+0000 7fb2e0c00640 1 -- 172.21.15.97:0/805325017 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc004330 con 0x7fb2e40ffa30 2024-03-27T10:59:55.332 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.360+0000 7fb2eb000640 1 -- 172.21.15.97:0/805325017 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2e41a6170 con 0x7fb2e40ffa30 2024-03-27T10:59:55.332 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.360+0000 7fb2eb000640 1 -- 172.21.15.97:0/805325017 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2e41a6610 con 0x7fb2e40ffa30 2024-03-27T10:59:55.338 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.365+0000 7fb2eddcd740 1 Processor -- start 2024-03-27T10:59:55.338 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.365+0000 7fb2eddcd740 1 -- start start 2024-03-27T10:59:55.338 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.366+0000 7fb2eddcd740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5653c0eff990 0x5653c0effdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:59:55.338 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.366+0000 7fb2eddcd740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5653c0f00380 0x5653c0f04e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:59:55.338 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.366+0000 7fb2eddcd740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5653c0f054f0 0x5653c0f07980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:59:55.338 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.366+0000 7fb2eddcd740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5653c0e53d10 con 0x5653c0f054f0 2024-03-27T10:59:55.338 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.366+0000 7fb2eddcd740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5653c0dfc260 con 0x5653c0eff990 2024-03-27T10:59:55.338 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.366+0000 7fb2eddcd740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5653c0e4fcc0 con 0x5653c0f00380 2024-03-27T10:59:55.338 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.366+0000 7fb2e9000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5653c0eff990 0x5653c0effdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:59:55.339 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.367+0000 7fb2e9000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5653c0eff990 0x5653c0effdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:35564/0 (socket says 172.21.15.97:35564) 2024-03-27T10:59:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.367+0000 7fb2e9000640 1 -- 172.21.15.97:0/1777060782 learned_addr learned my addr 172.21.15.97:0/1777060782 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:59:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.367+0000 7fb2e3400640 1 --2- 172.21.15.97:0/1777060782 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5653c0f054f0 0x5653c0f07980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:59:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.367+0000 7fb2e9000640 1 -- 172.21.15.97:0/1777060782 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5653c0f00380 msgr2=0x5653c0f04e80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-27T10:59:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.367+0000 7fb2e9000640 1 --2- 172.21.15.97:0/1777060782 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5653c0f00380 0x5653c0f04e80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.367+0000 7fb2e9000640 1 -- 172.21.15.97:0/1777060782 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5653c0f054f0 msgr2=0x5653c0f07980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:59:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.368+0000 7fb2e9000640 1 --2- 172.21.15.97:0/1777060782 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5653c0f054f0 0x5653c0f07980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.368+0000 7fb2e9000640 1 -- 172.21.15.97:0/1777060782 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653c0df3aa0 con 0x5653c0eff990 2024-03-27T10:59:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.368+0000 7fb2e3400640 1 --2- 172.21.15.97:0/1777060782 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5653c0f054f0 0x5653c0f07980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T10:59:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.368+0000 7fb2e0c00640 1 -- 172.21.15.97:0/805325017 <== mon.2 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7fb2dc003b00 con 0x7fb2e40ffa30 2024-03-27T10:59:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.368+0000 7fb2e9000640 1 --2- 172.21.15.97:0/1777060782 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5653c0eff990 0x5653c0effdb0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb2d4009cc0 tx=0x7fb2d40117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d66cf3f2bc5749ae server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:59:55.341 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.369+0000 7fb2d9600640 1 -- 172.21.15.97:0/1777060782 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d4012450 con 0x5653c0eff990 2024-03-27T10:59:55.341 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.369+0000 7fb2d9600640 1 -- 172.21.15.97:0/1777060782 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb2d4012aa0 con 0x5653c0eff990 2024-03-27T10:59:55.341 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.369+0000 7fb2d9600640 1 -- 172.21.15.97:0/1777060782 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d400fbf0 con 0x5653c0eff990 2024-03-27T10:59:55.342 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.370+0000 7fb2eddcd740 1 -- 172.21.15.97:0/1777060782 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5653c0eff990 msgr2=0x5653c0effdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:59:55.342 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.370+0000 7fb2eddcd740 1 --2- 172.21.15.97:0/1777060782 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5653c0eff990 0x5653c0effdb0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb2d4009cc0 tx=0x7fb2d40117b0 comp rx=0 tx=0).stop 2024-03-27T10:59:55.342 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.371+0000 7fb2eddcd740 1 -- 172.21.15.97:0/1777060782 shutdown_connections 2024-03-27T10:59:55.342 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.371+0000 7fb2eddcd740 1 --2- 172.21.15.97:0/1777060782 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5653c0f054f0 0x5653c0f07980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:55.343 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.371+0000 7fb2eddcd740 1 --2- 172.21.15.97:0/1777060782 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5653c0f00380 0x5653c0f04e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:55.343 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.371+0000 7fb2eddcd740 1 --2- 172.21.15.97:0/1777060782 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5653c0eff990 0x5653c0effdb0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:55.343 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.371+0000 7fb2e0c00640 1 --2- 172.21.15.97:0/805325017 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fb2c80a38a0 0x7fb2c80a5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:59:55.343 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.371+0000 7fb2eddcd740 1 -- 172.21.15.97:0/1777060782 >> 172.21.15.97:0/1777060782 conn(0x5653c0ee9ef0 msgr2=0x5653c0eea2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:59:55.344 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.372+0000 7fb2e0c00640 1 -- 172.21.15.97:0/805325017 <== mon.2 v2:172.21.15.114:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fb2dc0f1d20 con 0x7fb2e40ffa30 2024-03-27T10:59:55.344 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.372+0000 7fb2e9000640 1 --2- 172.21.15.97:0/805325017 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fb2c80a38a0 0x7fb2c80a5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:59:55.344 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.373+0000 7fb2eddcd740 1 -- 172.21.15.97:0/1777060782 shutdown_connections 2024-03-27T10:59:55.344 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.373+0000 7fb2e9000640 1 --2- 172.21.15.97:0/805325017 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fb2c80a38a0 0x7fb2c80a5d60 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fb2d4009a50 tx=0x7fb2d4009850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:59:55.345 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.373+0000 7fb2eddcd740 1 -- 172.21.15.97:0/1777060782 wait complete. 2024-03-27T10:59:55.345 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.373+0000 7fb2eddcd740 1 -- start start 2024-03-27T10:59:55.345 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.374+0000 7fb2eddcd740 1 Processor -- start 2024-03-27T10:59:55.347 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.376+0000 7fb2eddcd740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5653c0eff990 0x5653c0f09d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:59:55.347 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.376+0000 7fb2eddcd740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5653c0f00380 0x5653c0f0a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:59:55.347 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.376+0000 7fb2eddcd740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5653c0f054f0 0x5653c0f112d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:59:55.348 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.377+0000 7fb2e3400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5653c0f00380 0x5653c0f0a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:59:55.348 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.377+0000 7fb2e3e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5653c0eff990 0x5653c0f09d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:59:55.349 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.377+0000 7fb2e3e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5653c0eff990 0x5653c0f09d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.97:43048/0 (socket says 172.21.15.97:43048) 2024-03-27T10:59:55.349 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.377+0000 7fb2e3e00640 1 -- 172.21.15.97:0/878724270 learned_addr learned my addr 172.21.15.97:0/878724270 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:59:55.349 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.377+0000 7fb2eddcd740 1 -- 172.21.15.97:0/878724270 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5653c0f084a0 con 0x5653c0f00380 2024-03-27T10:59:55.349 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.377+0000 7fb2eddcd740 1 -- 172.21.15.97:0/878724270 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5653c0e0f630 con 0x5653c0f054f0 2024-03-27T10:59:55.349 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.377+0000 7fb2eddcd740 1 -- 172.21.15.97:0/878724270 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5653c0f08620 con 0x5653c0eff990 2024-03-27T10:59:55.349 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.377+0000 7fb2e9000640 1 --2- 172.21.15.97:0/878724270 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5653c0f054f0 0x5653c0f112d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:59:55.349 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.377+0000 7fb2eddcd740 10 client.0 _ll_register_callbacks cb 0x5653c0e14c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:59:55.350 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.378+0000 7fb2e3e00640 1 -- 172.21.15.97:0/878724270 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5653c0f054f0 msgr2=0x5653c0f112d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:59:55.350 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.378+0000 7fb2e3e00640 1 --2- 172.21.15.97:0/878724270 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5653c0f054f0 0x5653c0f112d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:55.350 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.378+0000 7fb2e3e00640 1 -- 172.21.15.97:0/878724270 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5653c0f00380 msgr2=0x5653c0f0a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:59:55.350 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.378+0000 7fb2e3e00640 1 --2- 172.21.15.97:0/878724270 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5653c0f00380 0x5653c0f0a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:55.350 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.378+0000 7fb2e3e00640 1 -- 172.21.15.97:0/878724270 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653c0e1b5f0 con 0x5653c0eff990 2024-03-27T10:59:55.350 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.378+0000 7fb2e9000640 1 --2- 172.21.15.97:0/878724270 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5653c0f054f0 0x5653c0f112d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T10:59:55.350 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.378+0000 7fb2e3400640 1 --2- 172.21.15.97:0/878724270 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5653c0f00380 0x5653c0f0a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T10:59:55.351 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.378+0000 7fb2e3e00640 1 --2- 172.21.15.97:0/878724270 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5653c0eff990 0x5653c0f09d50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc0ba600 tx=0x7fb2dc0bdf70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae37af11dab19119 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:59:55.351 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.379+0000 7fb2ce000640 10 client.0 ms_handle_connect on v2:172.21.15.177:3300/0 2024-03-27T10:59:55.351 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.379+0000 7fb2ce000640 1 -- 172.21.15.97:0/878724270 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc0de040 con 0x5653c0eff990 2024-03-27T10:59:55.351 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.379+0000 7fb2ce000640 1 -- 172.21.15.97:0/878724270 <== mon.1 v2:172.21.15.177:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fb2dc0c2120 con 0x5653c0eff990 2024-03-27T10:59:55.351 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.379+0000 7fb2eddcd740 1 -- 172.21.15.97:0/878724270 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2d4012aa0 con 0x5653c0eff990 2024-03-27T10:59:55.352 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.379+0000 7fb2ce000640 1 -- 172.21.15.97:0/878724270 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fb2dc0335a0 con 0x5653c0eff990 2024-03-27T10:59:55.352 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.380+0000 7fb2eddcd740 10 client.25795 fetch_fsmap learned FSMap version 31 2024-03-27T10:59:55.352 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.380+0000 7fb2eddcd740 1 -- 172.21.15.97:0/878724270 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5653c0b68210 con 0x5653c0eff990 2024-03-27T10:59:55.352 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.380+0000 7fb2ce000640 1 -- 172.21.15.97:0/878724270 <== mon.1 v2:172.21.15.177:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb2d4012aa0 con 0x5653c0eff990 2024-03-27T10:59:55.352 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.380+0000 7fb2ce000640 1 -- 172.21.15.97:0/878724270 <== mon.1 v2:172.21.15.177:3300/0 5 ==== fsmap(e 31) ==== 8370+0+0 (secure 0 0 0) 0x7fb2dc03c030 con 0x5653c0eff990 2024-03-27T10:59:55.352 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.381+0000 7fb2eddcd740 10 client.25795 fetch_fsmap finished waiting for FSMap version 31 2024-03-27T10:59:55.352 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.381+0000 7fb2eddcd740 10 client.25795 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T10:59:55.352 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.381+0000 7fb2eddcd740 20 client.25795 populate_metadata read hostname 'smithi097' 2024-03-27T10:59:55.353 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.381+0000 7fb2eddcd740 1 --2- 172.21.15.97:0/878724270 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5653c0f15bf0 0x5653c0f18050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:59:55.353 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.381+0000 7fb2eddcd740 4 client.25795 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T10:59:55.353 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.381+0000 7fb2eddcd740 1 -- 172.21.15.97:0/878724270 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5653c0e61250 con 0x5653c0f15bf0 2024-03-27T10:59:55.353 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.381+0000 7fb2e3400640 1 --2- 172.21.15.97:0/878724270 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5653c0f15bf0 0x5653c0f18050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:59:55.353 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.382+0000 7fb2e3400640 1 --2- 172.21.15.97:0/878724270 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5653c0f15bf0 0x5653c0f18050 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4a63b0e2847f220 server_cookie=2be4e43ad0f4da61 in_seq=0 out_seq=0 2024-03-27T10:59:55.353 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.382+0000 7fb2ce000640 10 client.25795 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T10:59:55.355 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.384+0000 7fb2ce000640 1 -- 172.21.15.97:0/878724270 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5653c0e61250 con 0x5653c0f15bf0 2024-03-27T10:59:55.355 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.384+0000 7fb2ce000640 10 client.25795 handle_command_reply: tid=0 2024-03-27T10:59:55.356 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.384+0000 7fb2eddcd740 1 client.25795 shutdown 2024-03-27T10:59:55.358 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.385+0000 7fb2eddcd740 1 -- 172.21.15.97:0/878724270 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5653c0eff990 msgr2=0x5653c0f09d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:59:55.358 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.385+0000 7fb2eddcd740 1 --2- 172.21.15.97:0/878724270 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5653c0eff990 0x5653c0f09d50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc0ba600 tx=0x7fb2dc0bdf70 comp rx=0 tx=0).stop 2024-03-27T10:59:55.359 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.386+0000 7fb2eddcd740 1 -- 172.21.15.97:0/878724270 shutdown_connections 2024-03-27T10:59:55.359 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.386+0000 7fb2eddcd740 1 --2- 172.21.15.97:0/878724270 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5653c0f15bf0 0x5653c0f18050 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:55.359 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.386+0000 7fb2eddcd740 1 --2- 172.21.15.97:0/878724270 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5653c0f054f0 0x5653c0f112d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:55.359 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.386+0000 7fb2eddcd740 1 --2- 172.21.15.97:0/878724270 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5653c0f00380 0x5653c0f0a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:55.359 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.386+0000 7fb2eddcd740 1 --2- 172.21.15.97:0/878724270 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5653c0eff990 0x5653c0f09d50 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:55.359 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.386+0000 7fb2eddcd740 1 -- 172.21.15.97:0/878724270 >> 172.21.15.97:0/878724270 conn(0x5653c0ee9ef0 msgr2=0x5653c0ee90f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:59:55.359 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.386+0000 7fb2ce000640 0 client.25795 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T10:59:55.359 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.387+0000 7fb2eddcd740 1 -- 172.21.15.97:0/878724270 shutdown_connections 2024-03-27T10:59:55.359 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.387+0000 7fb2eddcd740 1 -- 172.21.15.97:0/878724270 wait complete. 2024-03-27T10:59:55.359 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.387+0000 7fb2eddcd740 20 client.25795 trim_cache size 0 max 16384 2024-03-27T10:59:55.385 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.414+0000 7fb2eddcd740 1 Processor -- start 2024-03-27T10:59:55.386 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.414+0000 7fb2eddcd740 1 -- start start 2024-03-27T10:59:55.386 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.414+0000 7fb2eddcd740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5653c0eff990 0x5653c0f0a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:59:55.386 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.414+0000 7fb2eddcd740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5653c0f00380 0x5653c0f11260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:59:55.386 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.414+0000 7fb2eddcd740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5653c0f054f0 0x5653c0ee8860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:59:55.387 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.414+0000 7fb2eddcd740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e41a6c10 con 0x5653c0f054f0 2024-03-27T10:59:55.387 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.414+0000 7fb2eddcd740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e41a8810 con 0x5653c0eff990 2024-03-27T10:59:55.387 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.414+0000 7fb2eddcd740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e41a72e0 con 0x5653c0f00380 2024-03-27T10:59:55.387 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.415+0000 7fb2e3e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5653c0eff990 0x5653c0f0a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:59:55.387 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.415+0000 7fb2e3e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5653c0eff990 0x5653c0f0a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:35584/0 (socket says 172.21.15.97:35584) 2024-03-27T10:59:55.387 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.415+0000 7fb2e3e00640 1 -- 172.21.15.97:0/3431447261 learned_addr learned my addr 172.21.15.97:0/3431447261 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:59:55.387 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.415+0000 7fb2e9000640 1 --2- 172.21.15.97:0/3431447261 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5653c0f054f0 0x5653c0ee8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:59:55.388 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.415+0000 7fb2e3e00640 1 -- 172.21.15.97:0/3431447261 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5653c0f00380 msgr2=0x5653c0f11260 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-27T10:59:55.388 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.415+0000 7fb2e3e00640 1 --2- 172.21.15.97:0/3431447261 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5653c0f00380 0x5653c0f11260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:55.388 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.415+0000 7fb2e3e00640 1 -- 172.21.15.97:0/3431447261 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5653c0f054f0 msgr2=0x5653c0ee8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:59:55.388 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.415+0000 7fb2e3e00640 1 --2- 172.21.15.97:0/3431447261 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5653c0f054f0 0x5653c0ee8860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:55.388 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.415+0000 7fb2e3e00640 1 -- 172.21.15.97:0/3431447261 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653c0c34620 con 0x5653c0eff990 2024-03-27T10:59:55.389 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.416+0000 7fb2e3e00640 1 --2- 172.21.15.97:0/3431447261 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5653c0eff990 0x5653c0f0a070 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc0be130 tx=0x7fb2dc0be0a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=895aa4ac1679f627 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:59:55.389 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.416+0000 7fb2d8c00640 1 -- 172.21.15.97:0/3431447261 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc0de040 con 0x5653c0eff990 2024-03-27T10:59:55.389 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.416+0000 7fb2d8c00640 1 -- 172.21.15.97:0/3431447261 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb2dc0be8a0 con 0x5653c0eff990 2024-03-27T10:59:55.389 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.416+0000 7fb2d8c00640 1 -- 172.21.15.97:0/3431447261 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc0c6be0 con 0x5653c0eff990 2024-03-27T10:59:55.390 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.418+0000 7fb2eddcd740 1 -- 172.21.15.97:0/3431447261 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5653c0eff990 msgr2=0x5653c0f0a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:59:55.390 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.418+0000 7fb2eddcd740 1 --2- 172.21.15.97:0/3431447261 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5653c0eff990 0x5653c0f0a070 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc0be130 tx=0x7fb2dc0be0a0 comp rx=0 tx=0).stop 2024-03-27T10:59:55.390 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.418+0000 7fb2eddcd740 1 -- 172.21.15.97:0/3431447261 shutdown_connections 2024-03-27T10:59:55.390 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.418+0000 7fb2eddcd740 1 --2- 172.21.15.97:0/3431447261 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5653c0f054f0 0x5653c0ee8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:55.390 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.418+0000 7fb2eddcd740 1 --2- 172.21.15.97:0/3431447261 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5653c0f00380 0x5653c0f11260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:55.390 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.418+0000 7fb2eddcd740 1 --2- 172.21.15.97:0/3431447261 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5653c0eff990 0x5653c0f0a070 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:55.390 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.418+0000 7fb2eddcd740 1 -- 172.21.15.97:0/3431447261 >> 172.21.15.97:0/3431447261 conn(0x5653c0ee9ef0 msgr2=0x5653c0f09960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:59:55.391 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.419+0000 7fb2eddcd740 1 -- 172.21.15.97:0/3431447261 shutdown_connections 2024-03-27T10:59:55.391 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.419+0000 7fb2eddcd740 1 -- 172.21.15.97:0/3431447261 wait complete. 2024-03-27T10:59:55.391 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.419+0000 7fb2eddcd740 1 -- start start 2024-03-27T10:59:55.391 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.420+0000 7fb2eddcd740 1 Processor -- start 2024-03-27T10:59:55.393 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.420+0000 7fb2eddcd740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5653c0eff990 0x5653c0f0e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:59:55.393 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.420+0000 7fb2eddcd740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5653c0f00380 0x5653c0f0ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:59:55.393 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.420+0000 7fb2eddcd740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5653c0f054f0 0x5653c0f0fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:59:55.393 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.420+0000 7fb2eddcd740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e41a8080 con 0x5653c0f00380 2024-03-27T10:59:55.393 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.420+0000 7fb2eddcd740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e41a79b0 con 0x5653c0eff990 2024-03-27T10:59:55.393 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.420+0000 7fb2eddcd740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e410e120 con 0x5653c0f054f0 2024-03-27T10:59:55.393 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.420+0000 7fb2eddcd740 10 client.0 _ll_register_callbacks cb 0x7fb2dc035ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T10:59:55.393 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.421+0000 7fb2e3e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5653c0eff990 0x5653c0f0e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:59:55.394 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.421+0000 7fb2e3e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5653c0eff990 0x5653c0f0e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:35600/0 (socket says 172.21.15.97:35600) 2024-03-27T10:59:55.394 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.421+0000 7fb2e3e00640 1 -- 172.21.15.97:0/4244186415 learned_addr learned my addr 172.21.15.97:0/4244186415 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T10:59:55.394 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.421+0000 7fb2e3400640 1 --2- 172.21.15.97:0/4244186415 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5653c0f00380 0x5653c0f0ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:59:55.394 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.421+0000 7fb2e9000640 1 --2- 172.21.15.97:0/4244186415 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5653c0f054f0 0x5653c0f0fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:59:55.394 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.421+0000 7fb2e3e00640 1 -- 172.21.15.97:0/4244186415 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5653c0f054f0 msgr2=0x5653c0f0fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:59:55.394 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.421+0000 7fb2e3e00640 1 --2- 172.21.15.97:0/4244186415 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5653c0f054f0 0x5653c0f0fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:55.394 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.421+0000 7fb2e3e00640 1 -- 172.21.15.97:0/4244186415 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5653c0f00380 msgr2=0x5653c0f0ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T10:59:55.394 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.421+0000 7fb2e3e00640 1 --2- 172.21.15.97:0/4244186415 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5653c0f00380 0x5653c0f0ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:55.394 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.421+0000 7fb2e3e00640 1 -- 172.21.15.97:0/4244186415 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653c0f10370 con 0x5653c0eff990 2024-03-27T10:59:55.394 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.421+0000 7fb2e3400640 1 --2- 172.21.15.97:0/4244186415 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5653c0f00380 0x5653c0f0ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T10:59:55.399 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.422+0000 7fb2e3e00640 1 --2- 172.21.15.97:0/4244186415 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5653c0eff990 0x5653c0f0e830 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc0bab70 tx=0x7fb2dc0bdf70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1af5bf2a8bdc5f17 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T10:59:55.399 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.422+0000 7fb2ce000640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-03-27T10:59:55.399 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.422+0000 7fb2ce000640 1 -- 172.21.15.97:0/4244186415 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc0de040 con 0x5653c0eff990 2024-03-27T10:59:55.399 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.422+0000 7fb2ce000640 1 -- 172.21.15.97:0/4244186415 <== mon.2 v2:172.21.15.114:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fb2dc004d50 con 0x5653c0eff990 2024-03-27T10:59:55.399 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.423+0000 7fb2eddcd740 1 -- 172.21.15.97:0/4244186415 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2dc0be8a0 con 0x5653c0eff990 2024-03-27T10:59:55.399 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.423+0000 7fb2ce000640 1 -- 172.21.15.97:0/4244186415 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fb2dc0be4f0 con 0x5653c0eff990 2024-03-27T10:59:55.400 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.423+0000 7fb2eddcd740 10 client.36077 fetch_fsmap learned FSMap version 31 2024-03-27T10:59:55.400 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.423+0000 7fb2eddcd740 1 -- 172.21.15.97:0/4244186415 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5653c0f10370 con 0x5653c0eff990 2024-03-27T10:59:55.400 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.423+0000 7fb2ce000640 1 -- 172.21.15.97:0/4244186415 <== mon.2 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb2dc0be8a0 con 0x5653c0eff990 2024-03-27T10:59:55.400 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.424+0000 7fb2ce000640 1 -- 172.21.15.97:0/4244186415 <== mon.2 v2:172.21.15.114:3300/0 5 ==== fsmap(e 31) ==== 8370+0+0 (secure 0 0 0) 0x7fb2dc0d2540 con 0x5653c0eff990 2024-03-27T10:59:55.400 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.424+0000 7fb2eddcd740 10 client.36077 fetch_fsmap finished waiting for FSMap version 31 2024-03-27T10:59:55.400 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.424+0000 7fb2eddcd740 10 client.36077 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T10:59:55.400 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.424+0000 7fb2eddcd740 20 client.36077 populate_metadata read hostname 'smithi097' 2024-03-27T10:59:55.400 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.424+0000 7fb2eddcd740 1 --2- 172.21.15.97:0/4244186415 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5653c0f7c460 0x5653c0f7e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T10:59:55.400 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.424+0000 7fb2eddcd740 4 client.36077 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-27T10:59:55.400 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.424+0000 7fb2eddcd740 1 -- 172.21.15.97:0/4244186415 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5653c0e5b5a0 con 0x5653c0f7c460 2024-03-27T10:59:55.400 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.424+0000 7fb2e3400640 1 --2- 172.21.15.97:0/4244186415 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5653c0f7c460 0x5653c0f7e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T10:59:55.401 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.425+0000 7fb2e3400640 1 --2- 172.21.15.97:0/4244186415 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5653c0f7c460 0x5653c0f7e880 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36df1a8e071512a1 server_cookie=def67d389557d2ac in_seq=0 out_seq=0 2024-03-27T10:59:55.401 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.425+0000 7fb2ce000640 10 client.36077 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T10:59:55.402 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.431+0000 7fb2ce000640 1 -- 172.21.15.97:0/4244186415 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5653c0e5b5a0 con 0x5653c0f7c460 2024-03-27T10:59:55.402 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.431+0000 7fb2ce000640 10 client.36077 handle_command_reply: tid=0 2024-03-27T10:59:55.402 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.431+0000 7fb2eddcd740 1 client.36077 shutdown 2024-03-27T10:59:55.402 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.431+0000 7fb2eddcd740 1 -- 172.21.15.97:0/4244186415 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5653c0eff990 msgr2=0x5653c0f0e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:59:55.403 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.431+0000 7fb2eddcd740 1 --2- 172.21.15.97:0/4244186415 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5653c0eff990 0x5653c0f0e830 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc0bab70 tx=0x7fb2dc0bdf70 comp rx=0 tx=0).stop 2024-03-27T10:59:55.403 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.432+0000 7fb2eddcd740 1 -- 172.21.15.97:0/4244186415 shutdown_connections 2024-03-27T10:59:55.403 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.432+0000 7fb2eddcd740 1 --2- 172.21.15.97:0/4244186415 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5653c0f7c460 0x5653c0f7e880 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:55.403 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.432+0000 7fb2eddcd740 1 --2- 172.21.15.97:0/4244186415 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5653c0f054f0 0x5653c0f0fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:55.404 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.432+0000 7fb2eddcd740 1 --2- 172.21.15.97:0/4244186415 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5653c0f00380 0x5653c0f0ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:55.404 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.432+0000 7fb2eddcd740 1 --2- 172.21.15.97:0/4244186415 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5653c0eff990 0x5653c0f0e830 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:55.404 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.432+0000 7fb2eddcd740 1 -- 172.21.15.97:0/4244186415 >> 172.21.15.97:0/4244186415 conn(0x5653c0ee9ef0 msgr2=0x5653c0f0a160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:59:55.404 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.433+0000 7fb2ce000640 0 client.36077 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T10:59:55.405 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.433+0000 7fb2eddcd740 1 -- 172.21.15.97:0/4244186415 shutdown_connections 2024-03-27T10:59:55.405 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.433+0000 7fb2eddcd740 1 -- 172.21.15.97:0/4244186415 wait complete. 2024-03-27T10:59:55.405 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.433+0000 7fb2eddcd740 20 client.36077 trim_cache size 0 max 16384 2024-03-27T10:59:55.405 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-27T10:59:55.405 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-27T10:59:55.406 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-27T10:59:55.406 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-27T10:59:55.407 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.435+0000 7fb2eb000640 1 -- 172.21.15.97:0/805325017 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fb2c80a38a0 msgr2=0x7fb2c80a5d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:59:55.407 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.435+0000 7fb2eb000640 1 --2- 172.21.15.97:0/805325017 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fb2c80a38a0 0x7fb2c80a5d60 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fb2d4009a50 tx=0x7fb2d4009850 comp rx=0 tx=0).stop 2024-03-27T10:59:55.407 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.435+0000 7fb2eb000640 1 -- 172.21.15.97:0/805325017 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2e40ffa30 msgr2=0x7fb2e419e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T10:59:55.407 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.435+0000 7fb2eb000640 1 --2- 172.21.15.97:0/805325017 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2e40ffa30 0x7fb2e419e2c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc032380 tx=0x7fb2dc0093f0 comp rx=0 tx=0).stop 2024-03-27T10:59:55.407 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.435+0000 7fb2eb000640 1 -- 172.21.15.97:0/805325017 shutdown_connections 2024-03-27T10:59:55.407 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.435+0000 7fb2eb000640 1 --2- 172.21.15.97:0/805325017 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2e4101500 0x7fb2e41a5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:55.407 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.435+0000 7fb2eb000640 1 --2- 172.21.15.97:0/805325017 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2e4100b60 0x7fb2e419e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:55.407 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.435+0000 7fb2eb000640 1 --2- 172.21.15.97:0/805325017 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fb2c80a38a0 0x7fb2c80a5d60 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:55.407 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.435+0000 7fb2eb000640 1 --2- 172.21.15.97:0/805325017 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2e40ffa30 0x7fb2e419e2c0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T10:59:55.408 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.435+0000 7fb2eb000640 1 -- 172.21.15.97:0/805325017 >> 172.21.15.97:0/805325017 conn(0x7fb2e40fb330 msgr2=0x7fb2e4101da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T10:59:55.408 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.435+0000 7fb2eb000640 1 -- 172.21.15.97:0/805325017 shutdown_connections 2024-03-27T10:59:55.408 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T10:59:55.436+0000 7fb2eb000640 1 -- 172.21.15.97:0/805325017 wait complete. 2024-03-27T10:59:55.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:59:55 smithi097 ceph-mon[31859]: pgmap v539: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 60 MiB/s wr, 967 op/s 2024-03-27T10:59:55.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:59:55 smithi177 ceph-mon[35294]: pgmap v539: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 60 MiB/s wr, 967 op/s 2024-03-27T10:59:55.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:59:55 smithi114 ceph-mon[35338]: pgmap v539: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 60 MiB/s wr, 967 op/s 2024-03-27T10:59:57.641 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:59:57 smithi097 ceph-mon[31859]: pgmap v540: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 58 MiB/s wr, 940 op/s 2024-03-27T10:59:57.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:59:57 smithi177 ceph-mon[35294]: pgmap v540: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 58 MiB/s wr, 940 op/s 2024-03-27T10:59:57.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:59:57 smithi114 ceph-mon[35338]: pgmap v540: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 58 MiB/s wr, 940 op/s 2024-03-27T10:59:59.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 10:59:59 smithi097 ceph-mon[31859]: pgmap v541: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 62 MiB/s wr, 994 op/s 2024-03-27T10:59:59.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 10:59:59 smithi177 ceph-mon[35294]: pgmap v541: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 62 MiB/s wr, 994 op/s 2024-03-27T10:59:59.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 10:59:59 smithi114 ceph-mon[35338]: pgmap v541: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 62 MiB/s wr, 994 op/s 2024-03-27T11:00:00.376 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:00:00 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mon-a[31855]: 2024-03-27T10:59:59.998+0000 7fd0ea600700 -1 log_channel(cluster) log [ERR] : Health detail: HEALTH_ERR 1 MDSs report damaged metadata 2024-03-27T11:00:00.376 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:00:00 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mon-a[31855]: 2024-03-27T10:59:59.998+0000 7fd0ea600700 -1 log_channel(cluster) log [ERR] : [ERR] MDS_DAMAGE: 1 MDSs report damaged metadata 2024-03-27T11:00:00.376 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:00:00 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mon-a[31855]: 2024-03-27T10:59:59.998+0000 7fd0ea600700 -1 log_channel(cluster) log [ERR] : mds.h(mds.1): Metadata damage detected 2024-03-27T11:00:00.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:00:00 smithi097 ceph-mon[31859]: Health detail: HEALTH_ERR 1 MDSs report damaged metadata 2024-03-27T11:00:00.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:00:00 smithi097 ceph-mon[31859]: [ERR] MDS_DAMAGE: 1 MDSs report damaged metadata 2024-03-27T11:00:00.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:00:00 smithi097 ceph-mon[31859]: mds.h(mds.1): Metadata damage detected 2024-03-27T11:00:00.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:00:00 smithi177 ceph-mon[35294]: Health detail: HEALTH_ERR 1 MDSs report damaged metadata 2024-03-27T11:00:00.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:00:00 smithi177 ceph-mon[35294]: [ERR] MDS_DAMAGE: 1 MDSs report damaged metadata 2024-03-27T11:00:00.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:00:00 smithi177 ceph-mon[35294]: mds.h(mds.1): Metadata damage detected 2024-03-27T11:00:00.766 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:00:00 smithi114 ceph-mon[35338]: Health detail: HEALTH_ERR 1 MDSs report damaged metadata 2024-03-27T11:00:00.766 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:00:00 smithi114 ceph-mon[35338]: [ERR] MDS_DAMAGE: 1 MDSs report damaged metadata 2024-03-27T11:00:00.766 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:00:00 smithi114 ceph-mon[35338]: mds.h(mds.1): Metadata damage detected 2024-03-27T11:00:01.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:00:01 smithi097 ceph-mon[31859]: pgmap v542: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 67 MiB/s wr, 1.08k op/s 2024-03-27T11:00:01.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:00:01 smithi177 ceph-mon[35294]: pgmap v542: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 67 MiB/s wr, 1.08k op/s 2024-03-27T11:00:01.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:00:01 smithi114 ceph-mon[35338]: pgmap v542: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 67 MiB/s wr, 1.08k op/s 2024-03-27T11:00:02.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:00:02 smithi114 ceph-mon[35338]: pgmap v543: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 60 MiB/s wr, 961 op/s 2024-03-27T11:00:02.889 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:00:02 smithi097 ceph-mon[31859]: pgmap v543: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 60 MiB/s wr, 961 op/s 2024-03-27T11:00:02.925 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:00:02 smithi177 ceph-mon[35294]: pgmap v543: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 60 MiB/s wr, 961 op/s 2024-03-27T11:00:05.467 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:00:05 smithi114 ceph-mon[35338]: pgmap v544: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 61 MiB/s wr, 978 op/s 2024-03-27T11:00:05.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:00:05 smithi097 ceph-mon[31859]: pgmap v544: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 61 MiB/s wr, 978 op/s 2024-03-27T11:00:05.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:00:05 smithi177 ceph-mon[35294]: pgmap v544: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 61 MiB/s wr, 978 op/s 2024-03-27T11:00:07.515 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:00:07 smithi114 ceph-mon[35338]: pgmap v545: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 58 MiB/s wr, 942 op/s 2024-03-27T11:00:07.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:00:07 smithi097 ceph-mon[31859]: pgmap v545: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 58 MiB/s wr, 942 op/s 2024-03-27T11:00:07.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:00:07 smithi177 ceph-mon[35294]: pgmap v545: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 58 MiB/s wr, 942 op/s 2024-03-27T11:00:08.617 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.645+0000 7ff1c6400640 1 -- 172.21.15.97:0/4177584398 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff1c0109d60 msgr2=0x7ff1c01105f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:00:08.617 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.645+0000 7ff1c6400640 1 --2- 172.21.15.97:0/4177584398 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff1c0109d60 0x7ff1c01105f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff1b8009be0 tx=0x7ff1b802bef0 comp rx=0 tx=0).stop 2024-03-27T11:00:08.617 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.645+0000 7ff1c6400640 1 -- 172.21.15.97:0/4177584398 shutdown_connections 2024-03-27T11:00:08.618 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.646+0000 7ff1c6400640 1 --2- 172.21.15.97:0/4177584398 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff1c0109d60 0x7ff1c01105f0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:08.618 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.646+0000 7ff1c6400640 1 --2- 172.21.15.97:0/4177584398 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1c01093a0 0x7ff1c0109820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:08.618 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.646+0000 7ff1c6400640 1 --2- 172.21.15.97:0/4177584398 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff1c01081b0 0x7ff1c01085d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:08.618 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.646+0000 7ff1c6400640 1 -- 172.21.15.97:0/4177584398 >> 172.21.15.97:0/4177584398 conn(0x7ff1c0077160 msgr2=0x7ff1c0079580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:00:08.618 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.646+0000 7ff1c6400640 1 -- 172.21.15.97:0/4177584398 shutdown_connections 2024-03-27T11:00:08.619 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.646+0000 7ff1c6400640 1 -- 172.21.15.97:0/4177584398 wait complete. 2024-03-27T11:00:08.619 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.648+0000 7ff1c6400640 1 Processor -- start 2024-03-27T11:00:08.619 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.648+0000 7ff1c6400640 1 -- start start 2024-03-27T11:00:08.620 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.648+0000 7ff1c6400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff1c01081b0 0x7ff1c007f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:00:08.620 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.648+0000 7ff1c6400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff1c01093a0 0x7ff1c007f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:00:08.620 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.648+0000 7ff1c6400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1c0109d60 0x7ff1c01ae430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:00:08.621 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.648+0000 7ff1c6400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c0113d40 con 0x7ff1c0109d60 2024-03-27T11:00:08.621 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.648+0000 7ff1c6400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c0113bc0 con 0x7ff1c01093a0 2024-03-27T11:00:08.621 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.648+0000 7ff1c6400640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c0113ec0 con 0x7ff1c01081b0 2024-03-27T11:00:08.621 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.649+0000 7ff1bea00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff1c01093a0 0x7ff1c007f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:00:08.621 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.649+0000 7ff1bea00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff1c01093a0 0x7ff1c007f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:44846/0 (socket says 172.21.15.97:44846) 2024-03-27T11:00:08.621 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.649+0000 7ff1bea00640 1 -- 172.21.15.97:0/4253027068 learned_addr learned my addr 172.21.15.97:0/4253027068 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:00:08.621 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.649+0000 7ff1bfe00640 1 --2- 172.21.15.97:0/4253027068 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1c0109d60 0x7ff1c01ae430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:00:08.621 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.649+0000 7ff1bf400640 1 --2- 172.21.15.97:0/4253027068 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff1c01081b0 0x7ff1c007f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:00:08.621 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.649+0000 7ff1bea00640 1 -- 172.21.15.97:0/4253027068 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff1c01081b0 msgr2=0x7ff1c007f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:00:08.621 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.649+0000 7ff1bea00640 1 --2- 172.21.15.97:0/4253027068 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff1c01081b0 0x7ff1c007f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:08.621 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.649+0000 7ff1bea00640 1 -- 172.21.15.97:0/4253027068 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1c0109d60 msgr2=0x7ff1c01ae430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:00:08.621 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.649+0000 7ff1bea00640 1 --2- 172.21.15.97:0/4253027068 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1c0109d60 0x7ff1c01ae430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:08.621 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.649+0000 7ff1bea00640 1 -- 172.21.15.97:0/4253027068 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1c0082930 con 0x7ff1c01093a0 2024-03-27T11:00:08.622 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.650+0000 7ff1bea00640 1 --2- 172.21.15.97:0/4253027068 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff1c01093a0 0x7ff1c007f800 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff1b000e9c0 tx=0x7ff1b000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:00:08.622 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.650+0000 7ff1b7e00640 1 -- 172.21.15.97:0/4253027068 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b000cde0 con 0x7ff1c01093a0 2024-03-27T11:00:08.622 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.650+0000 7ff1b7e00640 1 -- 172.21.15.97:0/4253027068 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff1b00035f0 con 0x7ff1c01093a0 2024-03-27T11:00:08.622 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.650+0000 7ff1c6400640 1 -- 172.21.15.97:0/4253027068 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1c01aea30 con 0x7ff1c01093a0 2024-03-27T11:00:08.623 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.650+0000 7ff1b7e00640 1 -- 172.21.15.97:0/4253027068 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b0010690 con 0x7ff1c01093a0 2024-03-27T11:00:08.623 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.650+0000 7ff1c6400640 1 -- 172.21.15.97:0/4253027068 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1c01aef80 con 0x7ff1c01093a0 2024-03-27T11:00:08.626 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.655+0000 7ff1c91ac740 1 Processor -- start 2024-03-27T11:00:08.627 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.656+0000 7ff1c91ac740 1 -- start start 2024-03-27T11:00:08.627 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.656+0000 7ff1c91ac740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562e1de45980 0x562e1de45da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:00:08.628 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.656+0000 7ff1c91ac740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562e1de46370 0x562e1de4ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:00:08.628 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.657+0000 7ff1c91ac740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562e1de4b4e0 0x562e1de4d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:00:08.628 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.657+0000 7ff1c91ac740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x562e1ddaad10 con 0x562e1de4b4e0 2024-03-27T11:00:08.628 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.657+0000 7ff1c91ac740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x562e1dd53260 con 0x562e1de45980 2024-03-27T11:00:08.628 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.657+0000 7ff1c91ac740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x562e1dda6cc0 con 0x562e1de46370 2024-03-27T11:00:08.628 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.657+0000 7ff1bfe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562e1de45980 0x562e1de45da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:00:08.628 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.657+0000 7ff1bfe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562e1de45980 0x562e1de45da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:44852/0 (socket says 172.21.15.97:44852) 2024-03-27T11:00:08.629 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.657+0000 7ff1bfe00640 1 -- 172.21.15.97:0/2741071731 learned_addr learned my addr 172.21.15.97:0/2741071731 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:00:08.629 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.657+0000 7ff1b7e00640 1 -- 172.21.15.97:0/4253027068 <== mon.2 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7ff1b00037a0 con 0x7ff1c01093a0 2024-03-27T11:00:08.629 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.657+0000 7ff1bea00640 1 --2- 172.21.15.97:0/2741071731 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562e1de4b4e0 0x562e1de4d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:00:08.629 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.657+0000 7ff1bf400640 1 --2- 172.21.15.97:0/2741071731 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562e1de46370 0x562e1de4ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:00:08.629 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.658+0000 7ff1bf400640 1 -- 172.21.15.97:0/2741071731 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562e1de45980 msgr2=0x562e1de45da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:00:08.630 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.658+0000 7ff1bf400640 1 --2- 172.21.15.97:0/2741071731 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562e1de45980 0x562e1de45da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:08.630 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.658+0000 7ff1bf400640 1 -- 172.21.15.97:0/2741071731 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562e1de4b4e0 msgr2=0x562e1de4d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:00:08.630 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.658+0000 7ff1bf400640 1 --2- 172.21.15.97:0/2741071731 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562e1de4b4e0 0x562e1de4d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:08.630 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.658+0000 7ff1bf400640 1 -- 172.21.15.97:0/2741071731 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e1dd4aaa0 con 0x562e1de46370 2024-03-27T11:00:08.631 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.658+0000 7ff1bea00640 1 --2- 172.21.15.97:0/2741071731 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562e1de4b4e0 0x562e1de4d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T11:00:08.631 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.659+0000 7ff1bf400640 1 --2- 172.21.15.97:0/2741071731 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562e1de46370 0x562e1de4ae70 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff1a8009bd0 tx=0x7ff1a8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad26dceef04ce8e3 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:00:08.631 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.659+0000 7ff1b4c00640 1 -- 172.21.15.97:0/2741071731 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a8004230 con 0x562e1de46370 2024-03-27T11:00:08.631 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.659+0000 7ff1b4c00640 1 -- 172.21.15.97:0/2741071731 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff1a8009e40 con 0x562e1de46370 2024-03-27T11:00:08.631 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.659+0000 7ff1b4c00640 1 -- 172.21.15.97:0/2741071731 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a8019640 con 0x562e1de46370 2024-03-27T11:00:08.631 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.660+0000 7ff1c91ac740 1 -- 172.21.15.97:0/2741071731 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562e1de46370 msgr2=0x562e1de4ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:00:08.631 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.660+0000 7ff1c91ac740 1 --2- 172.21.15.97:0/2741071731 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562e1de46370 0x562e1de4ae70 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff1a8009bd0 tx=0x7ff1a8011810 comp rx=0 tx=0).stop 2024-03-27T11:00:08.631 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.660+0000 7ff1b7e00640 1 --2- 172.21.15.97:0/4253027068 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7ff1a40a37b0 0x7ff1a40a5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:00:08.632 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.661+0000 7ff1b7e00640 1 -- 172.21.15.97:0/4253027068 <== mon.2 v2:172.21.15.114:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7ff1b00cfff0 con 0x7ff1c01093a0 2024-03-27T11:00:08.632 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.661+0000 7ff1c91ac740 1 -- 172.21.15.97:0/2741071731 shutdown_connections 2024-03-27T11:00:08.632 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.661+0000 7ff1c91ac740 1 --2- 172.21.15.97:0/2741071731 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562e1de4b4e0 0x562e1de4d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:08.632 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.661+0000 7ff1c91ac740 1 --2- 172.21.15.97:0/2741071731 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562e1de46370 0x562e1de4ae70 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:08.632 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.661+0000 7ff1c91ac740 1 --2- 172.21.15.97:0/2741071731 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562e1de45980 0x562e1de45da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:08.633 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.661+0000 7ff1c91ac740 1 -- 172.21.15.97:0/2741071731 >> 172.21.15.97:0/2741071731 conn(0x562e1de40ef0 msgr2=0x562e1de412f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:00:08.633 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.661+0000 7ff1bfe00640 1 --2- 172.21.15.97:0/4253027068 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7ff1a40a37b0 0x7ff1a40a5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:00:08.633 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.662+0000 7ff1c91ac740 1 -- 172.21.15.97:0/2741071731 shutdown_connections 2024-03-27T11:00:08.633 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.662+0000 7ff1c91ac740 1 -- 172.21.15.97:0/2741071731 wait complete. 2024-03-27T11:00:08.634 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.662+0000 7ff1bfe00640 1 --2- 172.21.15.97:0/4253027068 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7ff1a40a37b0 0x7ff1a40a5c70 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7ff1b80099a0 tx=0x7ff1b8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:00:08.634 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.662+0000 7ff1c91ac740 1 -- start start 2024-03-27T11:00:08.634 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.663+0000 7ff1c91ac740 1 Processor -- start 2024-03-27T11:00:08.635 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.664+0000 7ff1c91ac740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562e1de45980 0x562e1de4fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:00:08.635 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.664+0000 7ff1c91ac740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562e1de46370 0x562e1de50220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:00:08.636 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.664+0000 7ff1c91ac740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562e1de4b4e0 0x562e1de571f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:00:08.636 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.664+0000 7ff1bf400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562e1de45980 0x562e1de4fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:00:08.636 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.664+0000 7ff1bf400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562e1de45980 0x562e1de4fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38320/0 (socket says 172.21.15.97:38320) 2024-03-27T11:00:08.636 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.664+0000 7ff1bea00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562e1de46370 0x562e1de50220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:00:08.636 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.665+0000 7ff1bea00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562e1de46370 0x562e1de50220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.97:42228/0 (socket says 172.21.15.97:42228) 2024-03-27T11:00:08.636 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.665+0000 7ff1c91ac740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x562e1de4e490 con 0x562e1de45980 2024-03-27T11:00:08.637 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.665+0000 7ff1bf400640 1 -- 172.21.15.97:0/419052411 learned_addr learned my addr 172.21.15.97:0/419052411 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:00:08.637 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.665+0000 7ff1c91ac740 1 -- 172.21.15.97:0/419052411 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x562e1dd66630 con 0x562e1de4b4e0 2024-03-27T11:00:08.637 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.665+0000 7ff1c91ac740 1 -- 172.21.15.97:0/419052411 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x562e1de4e610 con 0x562e1de46370 2024-03-27T11:00:08.637 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.665+0000 7ff1c91ac740 10 client.0 _ll_register_callbacks cb 0x562e1dd6bc50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T11:00:08.637 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.665+0000 7ff1bfe00640 1 --2- 172.21.15.97:0/419052411 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562e1de4b4e0 0x562e1de571f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:00:08.637 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.665+0000 7ff1bf400640 1 -- 172.21.15.97:0/419052411 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562e1de46370 msgr2=0x562e1de50220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:00:08.637 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.665+0000 7ff1bf400640 1 --2- 172.21.15.97:0/419052411 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562e1de46370 0x562e1de50220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:08.637 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.665+0000 7ff1bf400640 1 -- 172.21.15.97:0/419052411 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562e1de4b4e0 msgr2=0x562e1de571f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:00:08.637 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.665+0000 7ff1bf400640 1 --2- 172.21.15.97:0/419052411 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562e1de4b4e0 0x562e1de571f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:08.637 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.665+0000 7ff1bf400640 1 -- 172.21.15.97:0/419052411 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e1dd725f0 con 0x562e1de45980 2024-03-27T11:00:08.638 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.666+0000 7ff1bf400640 1 --2- 172.21.15.97:0/419052411 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562e1de45980 0x562e1de4fce0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7ff1a801a000 tx=0x7ff1a80042b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c61f51ee4bfdd482 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:00:08.638 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.666+0000 7ff1ad600640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T11:00:08.638 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.666+0000 7ff1ad600640 1 -- 172.21.15.97:0/419052411 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a8028070 con 0x562e1de45980 2024-03-27T11:00:08.638 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.666+0000 7ff1ad600640 1 -- 172.21.15.97:0/419052411 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7ff1a8005250 con 0x562e1de45980 2024-03-27T11:00:08.638 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.666+0000 7ff1c91ac740 1 -- 172.21.15.97:0/419052411 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562e1ddb8250 con 0x562e1de45980 2024-03-27T11:00:08.639 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.667+0000 7ff1ad600640 1 -- 172.21.15.97:0/419052411 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff1a8032050 con 0x562e1de45980 2024-03-27T11:00:08.639 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.667+0000 7ff1c91ac740 10 client.16431 fetch_fsmap learned FSMap version 31 2024-03-27T11:00:08.639 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.667+0000 7ff1c91ac740 1 -- 172.21.15.97:0/419052411 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x562e1dabf210 con 0x562e1de45980 2024-03-27T11:00:08.639 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.668+0000 7ff1ad600640 1 -- 172.21.15.97:0/419052411 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x562e1ddb8250 con 0x562e1de45980 2024-03-27T11:00:08.640 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.668+0000 7ff1ad600640 1 -- 172.21.15.97:0/419052411 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 31) ==== 8370+0+0 (secure 0 0 0) 0x7ff1a8004b70 con 0x562e1de45980 2024-03-27T11:00:08.640 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.668+0000 7ff1c91ac740 10 client.16431 fetch_fsmap finished waiting for FSMap version 31 2024-03-27T11:00:08.640 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.668+0000 7ff1c91ac740 10 client.16431 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T11:00:08.640 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.668+0000 7ff1c91ac740 20 client.16431 populate_metadata read hostname 'smithi097' 2024-03-27T11:00:08.640 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.669+0000 7ff1c91ac740 1 --2- 172.21.15.97:0/419052411 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x562e1de5ba00 0x562e1de5ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:00:08.640 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.669+0000 7ff1c91ac740 4 client.16431 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T11:00:08.640 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.669+0000 7ff1c91ac740 1 -- 172.21.15.97:0/419052411 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562e1ddb25a0 con 0x562e1de5ba00 2024-03-27T11:00:08.640 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.669+0000 7ff1bea00640 1 --2- 172.21.15.97:0/419052411 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x562e1de5ba00 0x562e1de5ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:00:08.641 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.670+0000 7ff1bea00640 1 --2- 172.21.15.97:0/419052411 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x562e1de5ba00 0x562e1de5ddf0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3da3528b64740159 server_cookie=cae0a0f3974e817a in_seq=0 out_seq=0 2024-03-27T11:00:08.641 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.670+0000 7ff1ad600640 10 client.16431 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T11:00:08.642 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.671+0000 7ff1ad600640 1 -- 172.21.15.97:0/419052411 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x562e1ddb25a0 con 0x562e1de5ba00 2024-03-27T11:00:08.642 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.671+0000 7ff1ad600640 10 client.16431 handle_command_reply: tid=0 2024-03-27T11:00:08.642 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.671+0000 7ff1c91ac740 1 client.16431 shutdown 2024-03-27T11:00:08.643 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.671+0000 7ff1c91ac740 1 -- 172.21.15.97:0/419052411 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562e1de45980 msgr2=0x562e1de4fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:00:08.643 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.671+0000 7ff1c91ac740 1 --2- 172.21.15.97:0/419052411 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562e1de45980 0x562e1de4fce0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7ff1a801a000 tx=0x7ff1a80042b0 comp rx=0 tx=0).stop 2024-03-27T11:00:08.643 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.672+0000 7ff1c91ac740 1 -- 172.21.15.97:0/419052411 shutdown_connections 2024-03-27T11:00:08.643 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.672+0000 7ff1c91ac740 1 --2- 172.21.15.97:0/419052411 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x562e1de5ba00 0x562e1de5ddf0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:08.644 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.672+0000 7ff1c91ac740 1 --2- 172.21.15.97:0/419052411 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562e1de4b4e0 0x562e1de571f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:08.644 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.672+0000 7ff1c91ac740 1 --2- 172.21.15.97:0/419052411 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562e1de46370 0x562e1de50220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:08.644 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.672+0000 7ff1c91ac740 1 --2- 172.21.15.97:0/419052411 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562e1de45980 0x562e1de4fce0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:08.644 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.672+0000 7ff1c91ac740 1 -- 172.21.15.97:0/419052411 >> 172.21.15.97:0/419052411 conn(0x562e1de40ef0 msgr2=0x562e1de400f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:00:08.644 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.672+0000 7ff1ad600640 0 client.16431 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T11:00:08.644 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.672+0000 7ff1c91ac740 1 -- 172.21.15.97:0/419052411 shutdown_connections 2024-03-27T11:00:08.644 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.672+0000 7ff1c91ac740 1 -- 172.21.15.97:0/419052411 wait complete. 2024-03-27T11:00:08.644 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.672+0000 7ff1c91ac740 20 client.16431 trim_cache size 0 max 16384 2024-03-27T11:00:08.671 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.700+0000 7ff1c91ac740 1 Processor -- start 2024-03-27T11:00:08.671 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.700+0000 7ff1c91ac740 1 -- start start 2024-03-27T11:00:08.671 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.700+0000 7ff1c91ac740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562e1de45980 0x562e1de4f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:00:08.672 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.700+0000 7ff1c91ac740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562e1de46370 0x562e1de571b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:00:08.672 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.700+0000 7ff1c91ac740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562e1de4b4e0 0x562e1de4f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:00:08.672 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.700+0000 7ff1c91ac740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c01b7c80 con 0x562e1de46370 2024-03-27T11:00:08.672 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.700+0000 7ff1c91ac740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c01b9880 con 0x562e1de45980 2024-03-27T11:00:08.672 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.700+0000 7ff1c91ac740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c01b8350 con 0x562e1de4b4e0 2024-03-27T11:00:08.672 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.701+0000 7ff1bf400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562e1de45980 0x562e1de4f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:00:08.672 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.701+0000 7ff1bea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562e1de46370 0x562e1de571b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:00:08.673 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.701+0000 7ff1bf400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562e1de45980 0x562e1de4f300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:44878/0 (socket says 172.21.15.97:44878) 2024-03-27T11:00:08.673 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.701+0000 7ff1bf400640 1 -- 172.21.15.97:0/493518780 learned_addr learned my addr 172.21.15.97:0/493518780 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:00:08.673 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.701+0000 7ff1bfe00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562e1de4b4e0 0x562e1de4f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:00:08.673 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.701+0000 7ff1bf400640 1 -- 172.21.15.97:0/493518780 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562e1de4b4e0 msgr2=0x562e1de4f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:00:08.673 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.701+0000 7ff1bf400640 1 --2- 172.21.15.97:0/493518780 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562e1de4b4e0 0x562e1de4f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:08.673 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.701+0000 7ff1bf400640 1 -- 172.21.15.97:0/493518780 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562e1de46370 msgr2=0x562e1de571b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:00:08.673 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.701+0000 7ff1bf400640 1 --2- 172.21.15.97:0/493518780 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562e1de46370 0x562e1de571b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:08.673 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.701+0000 7ff1bf400640 1 -- 172.21.15.97:0/493518780 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e1db8b620 con 0x562e1de45980 2024-03-27T11:00:08.674 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.701+0000 7ff1bea00640 1 --2- 172.21.15.97:0/493518780 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562e1de46370 0x562e1de571b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T11:00:08.674 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.702+0000 7ff1bf400640 1 --2- 172.21.15.97:0/493518780 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562e1de45980 0x562e1de4f300 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff1a802d630 tx=0x7ff1a80096c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f5469db40945aed server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:00:08.674 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.703+0000 7ff1afe00640 1 -- 172.21.15.97:0/493518780 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a8028040 con 0x562e1de45980 2024-03-27T11:00:08.674 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.703+0000 7ff1afe00640 1 -- 172.21.15.97:0/493518780 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff1a8005700 con 0x562e1de45980 2024-03-27T11:00:08.674 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.703+0000 7ff1afe00640 1 -- 172.21.15.97:0/493518780 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a8037440 con 0x562e1de45980 2024-03-27T11:00:08.675 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.704+0000 7ff1c91ac740 1 -- 172.21.15.97:0/493518780 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562e1de45980 msgr2=0x562e1de4f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:00:08.675 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.704+0000 7ff1c91ac740 1 --2- 172.21.15.97:0/493518780 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562e1de45980 0x562e1de4f300 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff1a802d630 tx=0x7ff1a80096c0 comp rx=0 tx=0).stop 2024-03-27T11:00:08.675 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.704+0000 7ff1c91ac740 1 -- 172.21.15.97:0/493518780 shutdown_connections 2024-03-27T11:00:08.675 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.704+0000 7ff1c91ac740 1 --2- 172.21.15.97:0/493518780 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562e1de4b4e0 0x562e1de4f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:08.675 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.704+0000 7ff1c91ac740 1 --2- 172.21.15.97:0/493518780 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562e1de46370 0x562e1de571b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:08.675 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.704+0000 7ff1c91ac740 1 --2- 172.21.15.97:0/493518780 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562e1de45980 0x562e1de4f300 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:08.676 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.704+0000 7ff1c91ac740 1 -- 172.21.15.97:0/493518780 >> 172.21.15.97:0/493518780 conn(0x562e1de40ef0 msgr2=0x562e1de449e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:00:08.676 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.704+0000 7ff1c91ac740 1 -- 172.21.15.97:0/493518780 shutdown_connections 2024-03-27T11:00:08.676 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.704+0000 7ff1c91ac740 1 -- 172.21.15.97:0/493518780 wait complete. 2024-03-27T11:00:08.676 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.704+0000 7ff1c91ac740 1 -- start start 2024-03-27T11:00:08.676 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.705+0000 7ff1c91ac740 1 Processor -- start 2024-03-27T11:00:08.676 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.705+0000 7ff1c91ac740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562e1de45980 0x562e1de53d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:00:08.676 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.705+0000 7ff1c91ac740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562e1de46370 0x562e1de54260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:00:08.676 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.705+0000 7ff1c91ac740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562e1de4b4e0 0x562e1ded0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:00:08.677 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.705+0000 7ff1c91ac740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c01b90f0 con 0x562e1de4b4e0 2024-03-27T11:00:08.677 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.705+0000 7ff1c91ac740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c01b8a20 con 0x562e1de45980 2024-03-27T11:00:08.677 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.705+0000 7ff1c91ac740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c01b5c60 con 0x562e1de46370 2024-03-27T11:00:08.677 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.705+0000 7ff1c91ac740 10 client.0 _ll_register_callbacks cb 0x7ff1a8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T11:00:08.677 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.706+0000 7ff1bea00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562e1de46370 0x562e1de54260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:00:08.677 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.706+0000 7ff1bfe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562e1de4b4e0 0x562e1ded0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:00:08.678 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.706+0000 7ff1bfe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562e1de4b4e0 0x562e1ded0cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38324/0 (socket says 172.21.15.97:38324) 2024-03-27T11:00:08.678 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.706+0000 7ff1bfe00640 1 -- 172.21.15.97:0/2655833776 learned_addr learned my addr 172.21.15.97:0/2655833776 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:00:08.678 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.706+0000 7ff1bfe00640 1 -- 172.21.15.97:0/2655833776 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562e1de46370 msgr2=0x562e1de54260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:00:08.678 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.706+0000 7ff1bfe00640 1 --2- 172.21.15.97:0/2655833776 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562e1de46370 0x562e1de54260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:08.678 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.706+0000 7ff1bfe00640 1 -- 172.21.15.97:0/2655833776 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562e1de45980 msgr2=0x562e1de53d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:00:08.678 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.706+0000 7ff1bfe00640 1 --2- 172.21.15.97:0/2655833776 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562e1de45980 0x562e1de53d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:08.678 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.706+0000 7ff1bfe00640 1 -- 172.21.15.97:0/2655833776 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e1de57330 con 0x562e1de4b4e0 2024-03-27T11:00:08.678 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.707+0000 7ff1bfe00640 1 --2- 172.21.15.97:0/2655833776 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562e1de4b4e0 0x562e1ded0cf0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7ff1b802c500 tx=0x7ff1b8009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4232e5c0ae4935a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:00:08.679 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.707+0000 7ff1ad600640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T11:00:08.682 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.708+0000 7ff1ad600640 1 -- 172.21.15.97:0/2655833776 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b8004040 con 0x562e1de4b4e0 2024-03-27T11:00:08.682 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.708+0000 7ff1ad600640 1 -- 172.21.15.97:0/2655833776 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7ff1b80041e0 con 0x562e1de4b4e0 2024-03-27T11:00:08.682 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.708+0000 7ff1c91ac740 1 -- 172.21.15.97:0/2655833776 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1a8005700 con 0x562e1de4b4e0 2024-03-27T11:00:08.682 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.708+0000 7ff1ad600640 1 -- 172.21.15.97:0/2655833776 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff1b80043e0 con 0x562e1de4b4e0 2024-03-27T11:00:08.683 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.708+0000 7ff1c91ac740 10 client.16437 fetch_fsmap learned FSMap version 31 2024-03-27T11:00:08.683 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.708+0000 7ff1c91ac740 1 -- 172.21.15.97:0/2655833776 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x562e1ded2160 con 0x562e1de4b4e0 2024-03-27T11:00:08.683 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.709+0000 7ff1ad600640 1 -- 172.21.15.97:0/2655833776 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff1a8005700 con 0x562e1de4b4e0 2024-03-27T11:00:08.683 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.710+0000 7ff1ad600640 1 -- 172.21.15.97:0/2655833776 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 31) ==== 8370+0+0 (secure 0 0 0) 0x7ff1b8034430 con 0x562e1de4b4e0 2024-03-27T11:00:08.683 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.710+0000 7ff1c91ac740 10 client.16437 fetch_fsmap finished waiting for FSMap version 31 2024-03-27T11:00:08.683 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.710+0000 7ff1c91ac740 10 client.16437 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T11:00:08.683 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.710+0000 7ff1c91ac740 20 client.16437 populate_metadata read hostname 'smithi097' 2024-03-27T11:00:08.683 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.710+0000 7ff1c91ac740 1 --2- 172.21.15.97:0/2655833776 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x562e1ded2bb0 0x562e1ded4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:00:08.683 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.710+0000 7ff1c91ac740 4 client.16437 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-27T11:00:08.683 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.710+0000 7ff1c91ac740 1 -- 172.21.15.97:0/2655833776 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562e1dd30630 con 0x562e1ded2bb0 2024-03-27T11:00:08.684 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.711+0000 7ff1bf400640 1 --2- 172.21.15.97:0/2655833776 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x562e1ded2bb0 0x562e1ded4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:00:08.684 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.711+0000 7ff1bf400640 1 --2- 172.21.15.97:0/2655833776 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x562e1ded2bb0 0x562e1ded4fd0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c7c8341532ad115 server_cookie=7f7e80781650975b in_seq=0 out_seq=0 2024-03-27T11:00:08.684 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.711+0000 7ff1ad600640 10 client.16437 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T11:00:08.689 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.718+0000 7ff1ad600640 1 -- 172.21.15.97:0/2655833776 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562e1dd30630 con 0x562e1ded2bb0 2024-03-27T11:00:08.689 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.718+0000 7ff1ad600640 10 client.16437 handle_command_reply: tid=0 2024-03-27T11:00:08.689 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.718+0000 7ff1c91ac740 1 client.16437 shutdown 2024-03-27T11:00:08.689 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.718+0000 7ff1c91ac740 1 -- 172.21.15.97:0/2655833776 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562e1de4b4e0 msgr2=0x562e1ded0cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:00:08.689 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.718+0000 7ff1c91ac740 1 --2- 172.21.15.97:0/2655833776 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562e1de4b4e0 0x562e1ded0cf0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7ff1b802c500 tx=0x7ff1b8009500 comp rx=0 tx=0).stop 2024-03-27T11:00:08.690 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.719+0000 7ff1c91ac740 1 -- 172.21.15.97:0/2655833776 shutdown_connections 2024-03-27T11:00:08.690 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.719+0000 7ff1c91ac740 1 --2- 172.21.15.97:0/2655833776 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x562e1ded2bb0 0x562e1ded4fd0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:08.690 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.719+0000 7ff1c91ac740 1 --2- 172.21.15.97:0/2655833776 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562e1de4b4e0 0x562e1ded0cf0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:08.691 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.719+0000 7ff1ad600640 0 client.16437 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T11:00:08.691 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.719+0000 7ff1c91ac740 1 --2- 172.21.15.97:0/2655833776 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562e1de46370 0x562e1de54260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:08.691 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.719+0000 7ff1c91ac740 1 --2- 172.21.15.97:0/2655833776 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562e1de45980 0x562e1de53d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:08.691 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.719+0000 7ff1c91ac740 1 -- 172.21.15.97:0/2655833776 >> 172.21.15.97:0/2655833776 conn(0x562e1de40ef0 msgr2=0x562e1de449e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:00:08.691 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.719+0000 7ff1c91ac740 1 -- 172.21.15.97:0/2655833776 shutdown_connections 2024-03-27T11:00:08.691 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.719+0000 7ff1c91ac740 1 -- 172.21.15.97:0/2655833776 wait complete. 2024-03-27T11:00:08.691 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.720+0000 7ff1c91ac740 20 client.16437 trim_cache size 0 max 16384 2024-03-27T11:00:08.692 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-27T11:00:08.692 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-27T11:00:08.692 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-27T11:00:08.692 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-27T11:00:08.693 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.721+0000 7ff1c6400640 1 -- 172.21.15.97:0/4253027068 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7ff1a40a37b0 msgr2=0x7ff1a40a5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:00:08.693 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.721+0000 7ff1c6400640 1 --2- 172.21.15.97:0/4253027068 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7ff1a40a37b0 0x7ff1a40a5c70 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7ff1b80099a0 tx=0x7ff1b8005fd0 comp rx=0 tx=0).stop 2024-03-27T11:00:08.693 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.721+0000 7ff1c6400640 1 -- 172.21.15.97:0/4253027068 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff1c01093a0 msgr2=0x7ff1c007f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:00:08.693 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.721+0000 7ff1c6400640 1 --2- 172.21.15.97:0/4253027068 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff1c01093a0 0x7ff1c007f800 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff1b000e9c0 tx=0x7ff1b000ee80 comp rx=0 tx=0).stop 2024-03-27T11:00:08.693 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.722+0000 7ff1c6400640 1 -- 172.21.15.97:0/4253027068 shutdown_connections 2024-03-27T11:00:08.693 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.722+0000 7ff1c6400640 1 --2- 172.21.15.97:0/4253027068 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1c0109d60 0x7ff1c01ae430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:08.693 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.722+0000 7ff1c6400640 1 --2- 172.21.15.97:0/4253027068 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7ff1a40a37b0 0x7ff1a40a5c70 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:08.694 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.722+0000 7ff1c6400640 1 --2- 172.21.15.97:0/4253027068 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff1c01093a0 0x7ff1c007f800 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:08.694 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.722+0000 7ff1c6400640 1 --2- 172.21.15.97:0/4253027068 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff1c01081b0 0x7ff1c007f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:08.694 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.722+0000 7ff1c6400640 1 -- 172.21.15.97:0/4253027068 >> 172.21.15.97:0/4253027068 conn(0x7ff1c0077160 msgr2=0x7ff1c010c5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:00:08.694 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.723+0000 7ff1c6400640 1 -- 172.21.15.97:0/4253027068 shutdown_connections 2024-03-27T11:00:08.694 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:08.723+0000 7ff1c6400640 1 -- 172.21.15.97:0/4253027068 wait complete. 2024-03-27T11:00:09.516 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:00:09 smithi114 ceph-mon[35338]: pgmap v546: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 61 MiB/s wr, 987 op/s 2024-03-27T11:00:09.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:00:09 smithi097 ceph-mon[31859]: pgmap v546: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 61 MiB/s wr, 987 op/s 2024-03-27T11:00:09.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:00:09 smithi177 ceph-mon[35294]: pgmap v546: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 61 MiB/s wr, 987 op/s 2024-03-27T11:00:11.515 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:00:11 smithi114 ceph-mon[35338]: pgmap v547: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 65 MiB/s wr, 1.04k op/s 2024-03-27T11:00:11.516 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:00:11 smithi114 ceph-mon[35338]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T11:00:11.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:00:11 smithi097 ceph-mon[31859]: pgmap v547: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 65 MiB/s wr, 1.04k op/s 2024-03-27T11:00:11.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:00:11 smithi097 ceph-mon[31859]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T11:00:11.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:00:11 smithi177 ceph-mon[35294]: pgmap v547: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 65 MiB/s wr, 1.04k op/s 2024-03-27T11:00:11.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:00:11 smithi177 ceph-mon[35294]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T11:00:13.516 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:00:13 smithi114 ceph-mon[35338]: pgmap v548: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 57 MiB/s wr, 926 op/s 2024-03-27T11:00:13.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:00:13 smithi097 ceph-mon[31859]: pgmap v548: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 57 MiB/s wr, 926 op/s 2024-03-27T11:00:13.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:00:13 smithi177 ceph-mon[35294]: pgmap v548: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 57 MiB/s wr, 926 op/s 2024-03-27T11:00:15.515 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:00:15 smithi114 ceph-mon[35338]: pgmap v549: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 58 MiB/s wr, 941 op/s 2024-03-27T11:00:15.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:00:15 smithi097 ceph-mon[31859]: pgmap v549: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 58 MiB/s wr, 941 op/s 2024-03-27T11:00:15.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:00:15 smithi177 ceph-mon[35294]: pgmap v549: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 58 MiB/s wr, 941 op/s 2024-03-27T11:00:17.515 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:00:17 smithi114 ceph-mon[35338]: pgmap v550: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 56 MiB/s wr, 900 op/s 2024-03-27T11:00:17.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:00:17 smithi097 ceph-mon[31859]: pgmap v550: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 56 MiB/s wr, 900 op/s 2024-03-27T11:00:17.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:00:17 smithi177 ceph-mon[35294]: pgmap v550: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 56 MiB/s wr, 900 op/s 2024-03-27T11:00:19.515 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:00:19 smithi114 ceph-mon[35338]: pgmap v551: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 59 MiB/s wr, 947 op/s 2024-03-27T11:00:19.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:00:19 smithi097 ceph-mon[31859]: pgmap v551: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 59 MiB/s wr, 947 op/s 2024-03-27T11:00:19.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:00:19 smithi177 ceph-mon[35294]: pgmap v551: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 59 MiB/s wr, 947 op/s 2024-03-27T11:00:21.515 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:00:21 smithi114 ceph-mon[35338]: pgmap v552: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 63 MiB/s wr, 1.02k op/s 2024-03-27T11:00:21.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:00:21 smithi097 ceph-mon[31859]: pgmap v552: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 63 MiB/s wr, 1.02k op/s 2024-03-27T11:00:21.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:00:21 smithi177 ceph-mon[35294]: pgmap v552: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 63 MiB/s wr, 1.02k op/s 2024-03-27T11:00:21.891 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.921+0000 7f5256c00640 1 -- 172.21.15.97:0/3270587802 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f52500780f0 msgr2=0x7f5250078990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:00:21.891 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.921+0000 7f5256c00640 1 --2- 172.21.15.97:0/3270587802 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f52500780f0 0x7f5250078990 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f5248009be0 tx=0x7f524802bf10 comp rx=0 tx=0).stop 2024-03-27T11:00:21.892 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.921+0000 7f5256c00640 1 -- 172.21.15.97:0/3270587802 shutdown_connections 2024-03-27T11:00:21.892 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.921+0000 7f5256c00640 1 --2- 172.21.15.97:0/3270587802 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f52500780f0 0x7f5250078990 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:21.892 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.921+0000 7f5256c00640 1 --2- 172.21.15.97:0/3270587802 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5250077750 0x7f5250077bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:21.892 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.921+0000 7f5256c00640 1 --2- 172.21.15.97:0/3270587802 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5250076510 0x7f5250076930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:21.892 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.921+0000 7f5256c00640 1 -- 172.21.15.97:0/3270587802 >> 172.21.15.97:0/3270587802 conn(0x7f52500ff310 msgr2=0x7f5250101750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:00:21.892 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.921+0000 7f5256c00640 1 -- 172.21.15.97:0/3270587802 shutdown_connections 2024-03-27T11:00:21.892 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.922+0000 7f5256c00640 1 -- 172.21.15.97:0/3270587802 wait complete. 2024-03-27T11:00:21.893 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.923+0000 7f5256c00640 1 Processor -- start 2024-03-27T11:00:21.893 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.923+0000 7f5256c00640 1 -- start start 2024-03-27T11:00:21.894 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.924+0000 7f5256c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5250076510 0x7f5250070430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:00:21.894 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.924+0000 7f5256c00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5250077750 0x7f5250070970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:00:21.894 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.924+0000 7f5256c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f52500780f0 0x7f52501ae490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:00:21.894 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.924+0000 7f5256c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5250113d80 con 0x7f5250076510 2024-03-27T11:00:21.894 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.924+0000 7f5256c00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f5250113c00 con 0x7f52500780f0 2024-03-27T11:00:21.894 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.924+0000 7f5256c00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f5250113f00 con 0x7f5250077750 2024-03-27T11:00:21.895 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.924+0000 7f524fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5250076510 0x7f5250070430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:00:21.895 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.924+0000 7f524fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5250076510 0x7f5250070430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58244/0 (socket says 172.21.15.97:58244) 2024-03-27T11:00:21.896 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.924+0000 7f524fe00640 1 -- 172.21.15.97:0/1734458949 learned_addr learned my addr 172.21.15.97:0/1734458949 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:00:21.896 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.924+0000 7f524f400640 1 --2- 172.21.15.97:0/1734458949 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5250077750 0x7f5250070970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:00:21.896 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.924+0000 7f5254c00640 1 --2- 172.21.15.97:0/1734458949 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f52500780f0 0x7f52501ae490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:00:21.896 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.925+0000 7f524fe00640 1 -- 172.21.15.97:0/1734458949 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5250077750 msgr2=0x7f5250070970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:00:21.896 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.925+0000 7f524fe00640 1 --2- 172.21.15.97:0/1734458949 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5250077750 0x7f5250070970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:21.896 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.925+0000 7f524fe00640 1 -- 172.21.15.97:0/1734458949 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f52500780f0 msgr2=0x7f52501ae490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:00:21.896 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.925+0000 7f524fe00640 1 --2- 172.21.15.97:0/1734458949 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f52500780f0 0x7f52501ae490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:21.896 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.925+0000 7f524fe00640 1 -- 172.21.15.97:0/1734458949 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5250073af0 con 0x7f5250076510 2024-03-27T11:00:21.896 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.926+0000 7f524fe00640 1 --2- 172.21.15.97:0/1734458949 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5250076510 0x7f5250070430 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f524000e9c0 tx=0x7f524000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:00:21.897 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.927+0000 7f524cc00640 1 -- 172.21.15.97:0/1734458949 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f524000ccc0 con 0x7f5250076510 2024-03-27T11:00:21.897 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.927+0000 7f524cc00640 1 -- 172.21.15.97:0/1734458949 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f52400054d0 con 0x7f5250076510 2024-03-27T11:00:21.898 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.927+0000 7f5256c00640 1 -- 172.21.15.97:0/1734458949 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52501aea30 con 0x7f5250076510 2024-03-27T11:00:21.898 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.927+0000 7f524cc00640 1 -- 172.21.15.97:0/1734458949 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5240010430 con 0x7f5250076510 2024-03-27T11:00:21.898 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.927+0000 7f5256c00640 1 -- 172.21.15.97:0/1734458949 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52501af000 con 0x7f5250076510 2024-03-27T11:00:21.903 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.933+0000 7f5259845740 1 Processor -- start 2024-03-27T11:00:21.903 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.933+0000 7f5259845740 1 -- start start 2024-03-27T11:00:21.904 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.933+0000 7f5259845740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5606f0811980 0x5606f0811da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:00:21.904 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.934+0000 7f5259845740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5606f0812370 0x5606f0816e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:00:21.904 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.934+0000 7f5259845740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5606f08174e0 0x5606f0819970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:00:21.904 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.934+0000 7f5259845740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5606f0776d10 con 0x5606f08174e0 2024-03-27T11:00:21.904 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.934+0000 7f5259845740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5606f071f260 con 0x5606f0811980 2024-03-27T11:00:21.905 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.934+0000 7f5259845740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5606f0772cc0 con 0x5606f0812370 2024-03-27T11:00:21.905 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.934+0000 7f5254c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5606f0811980 0x5606f0811da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:00:21.905 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.934+0000 7f5254c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5606f0811980 0x5606f0811da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:33850/0 (socket says 172.21.15.97:33850) 2024-03-27T11:00:21.905 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.934+0000 7f5254c00640 1 -- 172.21.15.97:0/1869682686 learned_addr learned my addr 172.21.15.97:0/1869682686 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:00:21.905 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.934+0000 7f524cc00640 1 -- 172.21.15.97:0/1734458949 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f52400046c0 con 0x7f5250076510 2024-03-27T11:00:21.905 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.935+0000 7f524fe00640 1 --2- 172.21.15.97:0/1869682686 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5606f0812370 0x5606f0816e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:00:21.906 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.935+0000 7f524f400640 1 --2- 172.21.15.97:0/1869682686 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5606f08174e0 0x5606f0819970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:00:21.906 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.935+0000 7f524fe00640 1 -- 172.21.15.97:0/1869682686 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5606f0811980 msgr2=0x5606f0811da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:00:21.906 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.935+0000 7f524fe00640 1 --2- 172.21.15.97:0/1869682686 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5606f0811980 0x5606f0811da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:21.906 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.935+0000 7f524fe00640 1 -- 172.21.15.97:0/1869682686 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5606f08174e0 msgr2=0x5606f0819970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:00:21.906 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.935+0000 7f524fe00640 1 --2- 172.21.15.97:0/1869682686 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5606f08174e0 0x5606f0819970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:21.906 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.935+0000 7f524fe00640 1 -- 172.21.15.97:0/1869682686 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606f0716aa0 con 0x5606f0812370 2024-03-27T11:00:21.906 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.935+0000 7f524f400640 1 --2- 172.21.15.97:0/1869682686 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5606f08174e0 0x5606f0819970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T11:00:21.906 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.936+0000 7f524fe00640 1 --2- 172.21.15.97:0/1869682686 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5606f0812370 0x5606f0816e70 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f52400cd8f0 tx=0x7f52400d3e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b831b3aead55d84 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:00:21.908 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.937+0000 7f524cc00640 1 --2- 172.21.15.97:0/1734458949 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f52340a3820 0x7f52340a5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:00:21.908 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.937+0000 7f5245200640 1 -- 172.21.15.97:0/1869682686 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52400e3070 con 0x5606f0812370 2024-03-27T11:00:21.908 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.937+0000 7f5245200640 1 -- 172.21.15.97:0/1869682686 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f524009ca20 con 0x5606f0812370 2024-03-27T11:00:21.908 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.937+0000 7f5245200640 1 -- 172.21.15.97:0/1869682686 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f524009c340 con 0x5606f0812370 2024-03-27T11:00:21.908 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.937+0000 7f524cc00640 1 -- 172.21.15.97:0/1734458949 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f52400cfea0 con 0x7f5250076510 2024-03-27T11:00:21.908 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.937+0000 7f5254c00640 1 --2- 172.21.15.97:0/1734458949 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f52340a3820 0x7f52340a5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:00:21.909 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.938+0000 7f5259845740 1 -- 172.21.15.97:0/1869682686 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5606f0812370 msgr2=0x5606f0816e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:00:21.909 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.938+0000 7f5259845740 1 --2- 172.21.15.97:0/1869682686 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5606f0812370 0x5606f0816e70 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f52400cd8f0 tx=0x7f52400d3e40 comp rx=0 tx=0).stop 2024-03-27T11:00:21.909 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.938+0000 7f5254c00640 1 --2- 172.21.15.97:0/1734458949 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f52340a3820 0x7f52340a5ce0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f52480092f0 tx=0x7f5248009280 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:00:21.909 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.939+0000 7f5259845740 1 -- 172.21.15.97:0/1869682686 shutdown_connections 2024-03-27T11:00:21.910 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.939+0000 7f5259845740 1 --2- 172.21.15.97:0/1869682686 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5606f08174e0 0x5606f0819970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:21.910 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.939+0000 7f5259845740 1 --2- 172.21.15.97:0/1869682686 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5606f0812370 0x5606f0816e70 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:21.910 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.939+0000 7f5259845740 1 --2- 172.21.15.97:0/1869682686 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5606f0811980 0x5606f0811da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:21.910 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.939+0000 7f5259845740 1 -- 172.21.15.97:0/1869682686 >> 172.21.15.97:0/1869682686 conn(0x5606f080cef0 msgr2=0x5606f080d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:00:21.910 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.939+0000 7f5259845740 1 -- 172.21.15.97:0/1869682686 shutdown_connections 2024-03-27T11:00:21.910 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.939+0000 7f5259845740 1 -- 172.21.15.97:0/1869682686 wait complete. 2024-03-27T11:00:21.910 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.940+0000 7f5259845740 1 -- start start 2024-03-27T11:00:21.911 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.940+0000 7f5259845740 1 Processor -- start 2024-03-27T11:00:21.911 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.940+0000 7f5259845740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5606f0811980 0x5606f081bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:00:21.911 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.941+0000 7f5259845740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5606f0812370 0x5606f081c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:00:21.911 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.941+0000 7f5259845740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5606f08174e0 0x5606f08232c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:00:21.911 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.941+0000 7f5259845740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5606f081a490 con 0x5606f08174e0 2024-03-27T11:00:21.911 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.941+0000 7f5259845740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5606f0732630 con 0x5606f0812370 2024-03-27T11:00:21.911 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.941+0000 7f5259845740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5606f081a610 con 0x5606f0811980 2024-03-27T11:00:21.911 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.941+0000 7f5259845740 10 client.0 _ll_register_callbacks cb 0x5606f0737c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T11:00:21.912 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.941+0000 7f524f400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5606f0812370 0x5606f081c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:00:21.912 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.941+0000 7f524f400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5606f0812370 0x5606f081c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:33852/0 (socket says 172.21.15.97:33852) 2024-03-27T11:00:21.912 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.941+0000 7f524f400640 1 -- 172.21.15.97:0/1721104894 learned_addr learned my addr 172.21.15.97:0/1721104894 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:00:21.912 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.941+0000 7f524fe00640 1 --2- 172.21.15.97:0/1721104894 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5606f0811980 0x5606f081bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:00:21.913 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.942+0000 7f524f400640 1 -- 172.21.15.97:0/1721104894 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5606f0811980 msgr2=0x5606f081bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:00:21.913 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.942+0000 7f524f400640 1 --2- 172.21.15.97:0/1721104894 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5606f0811980 0x5606f081bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:21.913 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.942+0000 7f524f400640 1 -- 172.21.15.97:0/1721104894 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5606f08174e0 msgr2=0x5606f08232c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:00:21.913 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.942+0000 7f524f400640 1 --2- 172.21.15.97:0/1721104894 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5606f08174e0 0x5606f08232c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:21.913 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.942+0000 7f524f400640 1 -- 172.21.15.97:0/1721104894 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606f073e5f0 con 0x5606f0812370 2024-03-27T11:00:21.913 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.942+0000 7f524f400640 1 --2- 172.21.15.97:0/1721104894 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5606f0812370 0x5606f081c280 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f523c009f10 tx=0x7f523c011630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=695ce92180293c81 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:00:21.913 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.942+0000 7f5239600640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-03-27T11:00:21.913 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.943+0000 7f5239600640 1 -- 172.21.15.97:0/1721104894 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f523c019070 con 0x5606f0812370 2024-03-27T11:00:21.914 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.943+0000 7f5239600640 1 -- 172.21.15.97:0/1721104894 <== mon.2 v2:172.21.15.114:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f523c015870 con 0x5606f0812370 2024-03-27T11:00:21.914 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.943+0000 7f5259845740 1 -- 172.21.15.97:0/1721104894 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f524009ca20 con 0x5606f0812370 2024-03-27T11:00:21.914 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.944+0000 7f5239600640 1 -- 172.21.15.97:0/1721104894 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f523c025050 con 0x5606f0812370 2024-03-27T11:00:21.914 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.944+0000 7f5259845740 10 client.36104 fetch_fsmap learned FSMap version 32 2024-03-27T11:00:21.914 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.944+0000 7f5259845740 1 -- 172.21.15.97:0/1721104894 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5606f048b210 con 0x5606f0812370 2024-03-27T11:00:21.915 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.945+0000 7f5239600640 1 -- 172.21.15.97:0/1721104894 <== mon.2 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f524009ca20 con 0x5606f0812370 2024-03-27T11:00:21.915 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.945+0000 7f5239600640 1 -- 172.21.15.97:0/1721104894 <== mon.2 v2:172.21.15.114:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f523c014b00 con 0x5606f0812370 2024-03-27T11:00:21.915 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.945+0000 7f5259845740 10 client.36104 fetch_fsmap finished waiting for FSMap version 32 2024-03-27T11:00:21.915 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.945+0000 7f5259845740 10 client.36104 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T11:00:21.915 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.945+0000 7f5259845740 20 client.36104 populate_metadata read hostname 'smithi097' 2024-03-27T11:00:21.916 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.945+0000 7f5259845740 1 --2- 172.21.15.97:0/1721104894 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5606f08280e0 0x5606f082a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:00:21.916 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.945+0000 7f5259845740 4 client.36104 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T11:00:21.916 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.945+0000 7f5259845740 1 -- 172.21.15.97:0/1721104894 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5606f0784250 con 0x5606f08280e0 2024-03-27T11:00:21.916 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.946+0000 7f524fe00640 1 --2- 172.21.15.97:0/1721104894 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5606f08280e0 0x5606f082a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:00:21.916 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.946+0000 7f524fe00640 1 --2- 172.21.15.97:0/1721104894 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5606f08280e0 0x5606f082a500 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b883d3f0cbb4be1 server_cookie=68d05fa26d2d803b in_seq=0 out_seq=0 2024-03-27T11:00:21.917 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.947+0000 7f5239600640 10 client.36104 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T11:00:21.918 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.947+0000 7f5239600640 1 -- 172.21.15.97:0/1721104894 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5606f0784250 con 0x5606f08280e0 2024-03-27T11:00:21.918 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.947+0000 7f5239600640 10 client.36104 handle_command_reply: tid=0 2024-03-27T11:00:21.918 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.947+0000 7f5259845740 1 client.36104 shutdown 2024-03-27T11:00:21.919 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.948+0000 7f5259845740 1 -- 172.21.15.97:0/1721104894 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5606f0812370 msgr2=0x5606f081c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:00:21.919 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.948+0000 7f5259845740 1 --2- 172.21.15.97:0/1721104894 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5606f0812370 0x5606f081c280 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f523c009f10 tx=0x7f523c011630 comp rx=0 tx=0).stop 2024-03-27T11:00:21.919 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.948+0000 7f5259845740 1 -- 172.21.15.97:0/1721104894 shutdown_connections 2024-03-27T11:00:21.919 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.948+0000 7f5259845740 1 --2- 172.21.15.97:0/1721104894 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5606f08280e0 0x5606f082a500 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:21.919 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.948+0000 7f5259845740 1 --2- 172.21.15.97:0/1721104894 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5606f08174e0 0x5606f08232c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:21.919 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.948+0000 7f5259845740 1 --2- 172.21.15.97:0/1721104894 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5606f0812370 0x5606f081c280 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:21.919 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.948+0000 7f5259845740 1 --2- 172.21.15.97:0/1721104894 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5606f0811980 0x5606f081bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:21.919 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.948+0000 7f5259845740 1 -- 172.21.15.97:0/1721104894 >> 172.21.15.97:0/1721104894 conn(0x5606f080cef0 msgr2=0x5606f080c0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:00:21.919 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.948+0000 7f5239600640 0 client.36104 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T11:00:21.919 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.949+0000 7f5259845740 1 -- 172.21.15.97:0/1721104894 shutdown_connections 2024-03-27T11:00:21.919 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.949+0000 7f5259845740 1 -- 172.21.15.97:0/1721104894 wait complete. 2024-03-27T11:00:21.920 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.949+0000 7f5259845740 20 client.36104 trim_cache size 0 max 16384 2024-03-27T11:00:21.947 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.977+0000 7f5259845740 1 Processor -- start 2024-03-27T11:00:21.948 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.977+0000 7f5259845740 1 -- start start 2024-03-27T11:00:21.948 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.978+0000 7f5259845740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5606f0811980 0x5606f0813730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:00:21.948 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.978+0000 7f5259845740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5606f0812370 0x5606f0813c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:00:21.949 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.978+0000 7f5259845740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5606f08174e0 0x5606f0823280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:00:21.949 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.978+0000 7f5259845740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f52501b7da0 con 0x5606f0811980 2024-03-27T11:00:21.949 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.978+0000 7f5259845740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f52501b9940 con 0x5606f08174e0 2024-03-27T11:00:21.949 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.978+0000 7f5259845740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f52501b8470 con 0x5606f0812370 2024-03-27T11:00:21.949 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.978+0000 7f524fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5606f0811980 0x5606f0813730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:00:21.949 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.979+0000 7f524fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5606f0811980 0x5606f0813730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58268/0 (socket says 172.21.15.97:58268) 2024-03-27T11:00:21.949 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.979+0000 7f524fe00640 1 -- 172.21.15.97:0/1022934341 learned_addr learned my addr 172.21.15.97:0/1022934341 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:00:21.949 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.979+0000 7f5254c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5606f08174e0 0x5606f0823280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:00:21.950 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.979+0000 7f524f400640 1 --2- 172.21.15.97:0/1022934341 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5606f0812370 0x5606f0813c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:00:21.950 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.979+0000 7f524fe00640 1 -- 172.21.15.97:0/1022934341 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5606f0812370 msgr2=0x5606f0813c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:00:21.950 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.979+0000 7f524fe00640 1 --2- 172.21.15.97:0/1022934341 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5606f0812370 0x5606f0813c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:21.950 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.979+0000 7f524fe00640 1 -- 172.21.15.97:0/1022934341 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5606f08174e0 msgr2=0x5606f0823280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:00:21.950 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.980+0000 7f524fe00640 1 --2- 172.21.15.97:0/1022934341 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5606f08174e0 0x5606f0823280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:21.951 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.980+0000 7f524fe00640 1 -- 172.21.15.97:0/1022934341 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606f0557620 con 0x5606f0811980 2024-03-27T11:00:21.951 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.980+0000 7f5254c00640 1 --2- 172.21.15.97:0/1022934341 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5606f08174e0 0x5606f0823280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T11:00:21.951 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.980+0000 7f524fe00640 1 --2- 172.21.15.97:0/1022934341 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5606f0811980 0x5606f0813730 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f524000c6a0 tx=0x7f5240009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dab6c334bebbfd28 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:00:21.951 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.980+0000 7f523be00640 1 -- 172.21.15.97:0/1022934341 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f524009c0f0 con 0x5606f0811980 2024-03-27T11:00:21.951 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.981+0000 7f523be00640 1 -- 172.21.15.97:0/1022934341 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f524009ca20 con 0x5606f0811980 2024-03-27T11:00:21.951 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.981+0000 7f523be00640 1 -- 172.21.15.97:0/1022934341 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f524009be60 con 0x5606f0811980 2024-03-27T11:00:21.952 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.981+0000 7f5259845740 1 -- 172.21.15.97:0/1022934341 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5606f0811980 msgr2=0x5606f0813730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:00:21.952 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.981+0000 7f5259845740 1 --2- 172.21.15.97:0/1022934341 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5606f0811980 0x5606f0813730 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f524000c6a0 tx=0x7f5240009f90 comp rx=0 tx=0).stop 2024-03-27T11:00:21.952 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.982+0000 7f5259845740 1 -- 172.21.15.97:0/1022934341 shutdown_connections 2024-03-27T11:00:21.952 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.982+0000 7f5259845740 1 --2- 172.21.15.97:0/1022934341 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5606f08174e0 0x5606f0823280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:21.952 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.982+0000 7f5259845740 1 --2- 172.21.15.97:0/1022934341 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5606f0812370 0x5606f0813c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:21.953 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.982+0000 7f5259845740 1 --2- 172.21.15.97:0/1022934341 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5606f0811980 0x5606f0813730 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:21.953 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.982+0000 7f5259845740 1 -- 172.21.15.97:0/1022934341 >> 172.21.15.97:0/1022934341 conn(0x5606f080cef0 msgr2=0x5606f081bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:00:21.953 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.982+0000 7f5259845740 1 -- 172.21.15.97:0/1022934341 shutdown_connections 2024-03-27T11:00:21.953 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.982+0000 7f5259845740 1 -- 172.21.15.97:0/1022934341 wait complete. 2024-03-27T11:00:21.953 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.983+0000 7f5259845740 1 -- start start 2024-03-27T11:00:21.954 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.983+0000 7f5259845740 1 Processor -- start 2024-03-27T11:00:21.954 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.983+0000 7f5259845740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5606f0811980 0x5606f0820080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:00:21.954 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.983+0000 7f5259845740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5606f0812370 0x5606f08205c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:00:21.954 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.984+0000 7f5259845740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5606f08174e0 0x5606f088c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:00:21.954 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.984+0000 7f5259845740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f52501b9210 con 0x5606f08174e0 2024-03-27T11:00:21.955 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.984+0000 7f5259845740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f52501b8b40 con 0x5606f0812370 2024-03-27T11:00:21.955 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.984+0000 7f5259845740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f52501b5d80 con 0x5606f0811980 2024-03-27T11:00:21.955 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.984+0000 7f5259845740 10 client.0 _ll_register_callbacks cb 0x7f523c009450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T11:00:21.956 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.984+0000 7f524fe00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5606f0811980 0x5606f0820080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:00:21.956 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.984+0000 7f524f400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5606f0812370 0x5606f08205c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:00:21.956 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.985+0000 7f524f400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5606f0812370 0x5606f08205c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:33872/0 (socket says 172.21.15.97:33872) 2024-03-27T11:00:21.956 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.985+0000 7f524f400640 1 -- 172.21.15.97:0/1058279521 learned_addr learned my addr 172.21.15.97:0/1058279521 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:00:21.956 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.985+0000 7f5254c00640 1 --2- 172.21.15.97:0/1058279521 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5606f08174e0 0x5606f088c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:00:21.956 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.985+0000 7f524f400640 1 -- 172.21.15.97:0/1058279521 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5606f0811980 msgr2=0x5606f0820080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:00:21.956 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.985+0000 7f524f400640 1 --2- 172.21.15.97:0/1058279521 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5606f0811980 0x5606f0820080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:21.956 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.985+0000 7f524f400640 1 -- 172.21.15.97:0/1058279521 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5606f08174e0 msgr2=0x5606f088c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:00:21.956 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.986+0000 7f524f400640 1 --2- 172.21.15.97:0/1058279521 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5606f08174e0 0x5606f088c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:21.957 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.986+0000 7f524f400640 1 -- 172.21.15.97:0/1058279521 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606f088c970 con 0x5606f0812370 2024-03-27T11:00:21.957 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.986+0000 7f5254c00640 1 --2- 172.21.15.97:0/1058279521 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5606f08174e0 0x5606f088c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T11:00:21.957 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.986+0000 7f524f400640 1 --2- 172.21.15.97:0/1058279521 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5606f0812370 0x5606f08205c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f523c009f10 tx=0x7f523c011b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f17f82f20f028e8 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:00:21.958 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.986+0000 7f5239600640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-03-27T11:00:21.958 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.986+0000 7f5239600640 1 -- 172.21.15.97:0/1058279521 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f523c02d070 con 0x5606f0812370 2024-03-27T11:00:21.958 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.987+0000 7f5239600640 1 -- 172.21.15.97:0/1058279521 <== mon.2 v2:172.21.15.114:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f523c004450 con 0x5606f0812370 2024-03-27T11:00:21.958 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.987+0000 7f5259845740 1 -- 172.21.15.97:0/1058279521 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f524009ca20 con 0x5606f0812370 2024-03-27T11:00:21.958 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.987+0000 7f5239600640 1 -- 172.21.15.97:0/1058279521 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f523c039050 con 0x5606f0812370 2024-03-27T11:00:21.958 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.987+0000 7f5259845740 10 client.36110 fetch_fsmap learned FSMap version 32 2024-03-27T11:00:21.958 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.987+0000 7f5259845740 1 -- 172.21.15.97:0/1058279521 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5606f088db90 con 0x5606f0812370 2024-03-27T11:00:21.958 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.987+0000 7f5239600640 1 -- 172.21.15.97:0/1058279521 <== mon.2 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f523c003b40 con 0x5606f0812370 2024-03-27T11:00:21.959 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.988+0000 7f5239600640 1 -- 172.21.15.97:0/1058279521 <== mon.2 v2:172.21.15.114:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f523c015a90 con 0x5606f0812370 2024-03-27T11:00:21.959 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.988+0000 7f5259845740 10 client.36110 fetch_fsmap finished waiting for FSMap version 32 2024-03-27T11:00:21.959 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.988+0000 7f5259845740 10 client.36110 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T11:00:21.959 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.988+0000 7f5259845740 20 client.36110 populate_metadata read hostname 'smithi097' 2024-03-27T11:00:21.959 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.988+0000 7f5259845740 1 --2- 172.21.15.97:0/1058279521 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5606f088e5a0 0x5606f08909c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:00:21.959 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.988+0000 7f5259845740 4 client.36110 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-27T11:00:21.959 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.988+0000 7f5259845740 1 -- 172.21.15.97:0/1058279521 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f524009ca20 con 0x5606f088e5a0 2024-03-27T11:00:21.959 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.989+0000 7f524fe00640 1 --2- 172.21.15.97:0/1058279521 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5606f088e5a0 0x5606f08909c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:00:21.960 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.990+0000 7f524fe00640 1 --2- 172.21.15.97:0/1058279521 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5606f088e5a0 0x5606f08909c0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a18ee4bf525e8f1e server_cookie=c89bed9cc7ad2300 in_seq=0 out_seq=0 2024-03-27T11:00:21.960 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:21.990+0000 7f5239600640 10 client.36110 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T11:00:21.975 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:22.001+0000 7f5239600640 1 -- 172.21.15.97:0/1058279521 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f524009ca20 con 0x5606f088e5a0 2024-03-27T11:00:21.975 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:22.001+0000 7f5239600640 10 client.36110 handle_command_reply: tid=0 2024-03-27T11:00:21.975 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:22.001+0000 7f5259845740 1 client.36110 shutdown 2024-03-27T11:00:21.975 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:22.001+0000 7f5259845740 1 -- 172.21.15.97:0/1058279521 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5606f0812370 msgr2=0x5606f08205c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:00:21.975 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:22.001+0000 7f5259845740 1 --2- 172.21.15.97:0/1058279521 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5606f0812370 0x5606f08205c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f523c009f10 tx=0x7f523c011b80 comp rx=0 tx=0).stop 2024-03-27T11:00:21.975 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:22.001+0000 7f5259845740 1 -- 172.21.15.97:0/1058279521 shutdown_connections 2024-03-27T11:00:21.975 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:22.001+0000 7f5259845740 1 --2- 172.21.15.97:0/1058279521 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5606f088e5a0 0x5606f08909c0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:21.975 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:22.001+0000 7f5259845740 1 --2- 172.21.15.97:0/1058279521 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5606f08174e0 0x5606f088c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:21.975 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:22.001+0000 7f5259845740 1 --2- 172.21.15.97:0/1058279521 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5606f0812370 0x5606f08205c0 secure :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f523c009f10 tx=0x7f523c011b80 comp rx=0 tx=0).stop 2024-03-27T11:00:21.975 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:22.001+0000 7f5259845740 1 --2- 172.21.15.97:0/1058279521 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5606f0811980 0x5606f0820080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:21.976 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:22.001+0000 7f5259845740 1 -- 172.21.15.97:0/1058279521 >> 172.21.15.97:0/1058279521 conn(0x5606f080cef0 msgr2=0x5606f081bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:00:21.976 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:22.002+0000 7f5239600640 0 client.36110 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T11:00:21.976 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:22.002+0000 7f5259845740 1 -- 172.21.15.97:0/1058279521 shutdown_connections 2024-03-27T11:00:21.976 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:22.002+0000 7f5259845740 1 -- 172.21.15.97:0/1058279521 wait complete. 2024-03-27T11:00:21.976 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:22.002+0000 7f5259845740 20 client.36110 trim_cache size 0 max 16384 2024-03-27T11:00:21.976 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-27T11:00:21.976 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-27T11:00:21.976 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-27T11:00:21.976 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-27T11:00:21.977 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:22.003+0000 7f5256c00640 1 -- 172.21.15.97:0/1734458949 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f52340a3820 msgr2=0x7f52340a5ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:00:21.977 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:22.003+0000 7f5256c00640 1 --2- 172.21.15.97:0/1734458949 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f52340a3820 0x7f52340a5ce0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f52480092f0 tx=0x7f5248009280 comp rx=0 tx=0).stop 2024-03-27T11:00:21.977 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:22.003+0000 7f5256c00640 1 -- 172.21.15.97:0/1734458949 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5250076510 msgr2=0x7f5250070430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:00:21.977 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:22.003+0000 7f5256c00640 1 --2- 172.21.15.97:0/1734458949 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5250076510 0x7f5250070430 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f524000e9c0 tx=0x7f524000ee80 comp rx=0 tx=0).stop 2024-03-27T11:00:21.977 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:22.004+0000 7f5256c00640 1 -- 172.21.15.97:0/1734458949 shutdown_connections 2024-03-27T11:00:21.977 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:22.004+0000 7f5256c00640 1 --2- 172.21.15.97:0/1734458949 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f52340a3820 0x7f52340a5ce0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:21.977 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:22.004+0000 7f5256c00640 1 --2- 172.21.15.97:0/1734458949 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f52500780f0 0x7f52501ae490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:21.978 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:22.004+0000 7f5256c00640 1 --2- 172.21.15.97:0/1734458949 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5250077750 0x7f5250070970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:21.978 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:22.004+0000 7f5256c00640 1 --2- 172.21.15.97:0/1734458949 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5250076510 0x7f5250070430 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:21.978 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:22.004+0000 7f5256c00640 1 -- 172.21.15.97:0/1734458949 >> 172.21.15.97:0/1734458949 conn(0x7f52500ff310 msgr2=0x7f5250100e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:00:21.978 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:22.004+0000 7f5256c00640 1 -- 172.21.15.97:0/1734458949 shutdown_connections 2024-03-27T11:00:21.978 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:22.004+0000 7f5256c00640 1 -- 172.21.15.97:0/1734458949 wait complete. 2024-03-27T11:00:23.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:00:23 smithi097 ceph-mon[31859]: pgmap v553: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 58 MiB/s wr, 937 op/s 2024-03-27T11:00:23.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:00:23 smithi177 ceph-mon[35294]: pgmap v553: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 58 MiB/s wr, 937 op/s 2024-03-27T11:00:24.044 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:00:23 smithi114 ceph-mon[35338]: pgmap v553: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 58 MiB/s wr, 937 op/s 2024-03-27T11:00:25.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:00:25 smithi097 ceph-mon[31859]: pgmap v554: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 60 MiB/s wr, 965 op/s 2024-03-27T11:00:25.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:00:25 smithi177 ceph-mon[35294]: pgmap v554: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 60 MiB/s wr, 965 op/s 2024-03-27T11:00:25.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:00:25 smithi114 ceph-mon[35338]: pgmap v554: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 60 MiB/s wr, 965 op/s 2024-03-27T11:00:27.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:00:27 smithi097 ceph-mon[31859]: pgmap v555: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 58 MiB/s wr, 931 op/s 2024-03-27T11:00:27.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:00:27 smithi177 ceph-mon[35294]: pgmap v555: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 58 MiB/s wr, 931 op/s 2024-03-27T11:00:27.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:00:27 smithi114 ceph-mon[35338]: pgmap v555: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 58 MiB/s wr, 931 op/s 2024-03-27T11:00:29.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:00:29 smithi097 ceph-mon[31859]: pgmap v556: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 61 MiB/s wr, 986 op/s 2024-03-27T11:00:29.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:00:29 smithi177 ceph-mon[35294]: pgmap v556: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 61 MiB/s wr, 986 op/s 2024-03-27T11:00:29.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:00:29 smithi114 ceph-mon[35338]: pgmap v556: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 61 MiB/s wr, 986 op/s 2024-03-27T11:00:31.648 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:00:31 smithi097 ceph-mon[31859]: pgmap v557: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 65 MiB/s wr, 1.06k op/s 2024-03-27T11:00:31.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:00:31 smithi177 ceph-mon[35294]: pgmap v557: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 65 MiB/s wr, 1.06k op/s 2024-03-27T11:00:31.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:00:31 smithi114 ceph-mon[35338]: pgmap v557: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 65 MiB/s wr, 1.06k op/s 2024-03-27T11:00:33.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:00:33 smithi097 ceph-mon[31859]: pgmap v558: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 59 MiB/s wr, 948 op/s 2024-03-27T11:00:33.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:00:33 smithi177 ceph-mon[35294]: pgmap v558: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 59 MiB/s wr, 948 op/s 2024-03-27T11:00:33.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:00:33 smithi114 ceph-mon[35338]: pgmap v558: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 59 MiB/s wr, 948 op/s 2024-03-27T11:00:35.179 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.208+0000 7fbaa3e00640 1 -- 172.21.15.97:0/2403700160 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fba9c1013e0 msgr2=0x7fba9c101860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:00:35.179 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.208+0000 7fbaa3e00640 1 --2- 172.21.15.97:0/2403700160 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fba9c1013e0 0x7fba9c101860 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fba8800a910 tx=0x7fba8802be50 comp rx=0 tx=0).stop 2024-03-27T11:00:35.179 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.209+0000 7fbaa3e00640 1 -- 172.21.15.97:0/2403700160 shutdown_connections 2024-03-27T11:00:35.179 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.209+0000 7fbaa3e00640 1 --2- 172.21.15.97:0/2403700160 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fba9c101da0 0x7fba9c108630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:35.179 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.209+0000 7fbaa3e00640 1 --2- 172.21.15.97:0/2403700160 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fba9c1013e0 0x7fba9c101860 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:35.180 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.209+0000 7fbaa3e00640 1 --2- 172.21.15.97:0/2403700160 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba9c1001f0 0x7fba9c100610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:35.180 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.209+0000 7fbaa3e00640 1 -- 172.21.15.97:0/2403700160 >> 172.21.15.97:0/2403700160 conn(0x7fba9c0fb7a0 msgr2=0x7fba9c0fdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:00:35.180 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.209+0000 7fbaa3e00640 1 -- 172.21.15.97:0/2403700160 shutdown_connections 2024-03-27T11:00:35.180 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.210+0000 7fbaa3e00640 1 -- 172.21.15.97:0/2403700160 wait complete. 2024-03-27T11:00:35.182 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.212+0000 7fbaa3e00640 1 Processor -- start 2024-03-27T11:00:35.182 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.212+0000 7fbaa3e00640 1 -- start start 2024-03-27T11:00:35.183 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.213+0000 7fbaa3e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fba9c1001f0 0x7fba9c195b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:00:35.183 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.213+0000 7fbaa3e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fba9c1013e0 0x7fba9c196040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:00:35.183 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.213+0000 7fbaa3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba9c101da0 0x7fba9c19d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:00:35.183 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.213+0000 7fbaa3e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fba9c10b500 con 0x7fba9c101da0 2024-03-27T11:00:35.183 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.213+0000 7fbaa3e00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fba9c10b380 con 0x7fba9c1013e0 2024-03-27T11:00:35.183 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.213+0000 7fbaa3e00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fba9c10b680 con 0x7fba9c1001f0 2024-03-27T11:00:35.183 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.213+0000 7fbaa1400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fba9c1001f0 0x7fba9c195b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:00:35.184 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.213+0000 7fbaa1400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fba9c1001f0 0x7fba9c195b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.97:41242/0 (socket says 172.21.15.97:41242) 2024-03-27T11:00:35.184 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.213+0000 7fbaa1400640 1 -- 172.21.15.97:0/3366858511 learned_addr learned my addr 172.21.15.97:0/3366858511 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:00:35.184 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.213+0000 7fbaa1e00640 1 --2- 172.21.15.97:0/3366858511 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba9c101da0 0x7fba9c19d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:00:35.184 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.213+0000 7fbaa1400640 1 -- 172.21.15.97:0/3366858511 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fba9c1013e0 msgr2=0x7fba9c196040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:00:35.184 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.213+0000 7fbaa0a00640 1 --2- 172.21.15.97:0/3366858511 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fba9c1013e0 0x7fba9c196040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:00:35.185 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.214+0000 7fbaa1400640 1 --2- 172.21.15.97:0/3366858511 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fba9c1013e0 0x7fba9c196040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:35.185 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.214+0000 7fbaa1400640 1 -- 172.21.15.97:0/3366858511 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba9c101da0 msgr2=0x7fba9c19d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:00:35.185 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.214+0000 7fbaa1400640 1 --2- 172.21.15.97:0/3366858511 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba9c101da0 0x7fba9c19d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:35.185 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.214+0000 7fbaa1400640 1 -- 172.21.15.97:0/3366858511 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba9c1a1b30 con 0x7fba9c1001f0 2024-03-27T11:00:35.185 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.214+0000 7fbaa1e00640 1 --2- 172.21.15.97:0/3366858511 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba9c101da0 0x7fba9c19d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T11:00:35.185 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.214+0000 7fbaa1400640 1 --2- 172.21.15.97:0/3366858511 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fba9c1001f0 0x7fba9c195b00 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fba9000a8d0 tx=0x7fba9000ad90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:00:35.185 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.214+0000 7fbaa0a00640 1 --2- 172.21.15.97:0/3366858511 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fba9c1013e0 0x7fba9c196040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T11:00:35.185 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.214+0000 7fba9a000640 1 -- 172.21.15.97:0/3366858511 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba90004020 con 0x7fba9c1001f0 2024-03-27T11:00:35.186 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.215+0000 7fba9a000640 1 -- 172.21.15.97:0/3366858511 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fba900041c0 con 0x7fba9c1001f0 2024-03-27T11:00:35.186 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.215+0000 7fba9a000640 1 -- 172.21.15.97:0/3366858511 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba90003d60 con 0x7fba9c1001f0 2024-03-27T11:00:35.186 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.215+0000 7fbaa3e00640 1 -- 172.21.15.97:0/3366858511 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba9c1a1e20 con 0x7fba9c1001f0 2024-03-27T11:00:35.186 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.215+0000 7fbaa3e00640 1 -- 172.21.15.97:0/3366858511 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba9c1a2230 con 0x7fba9c1001f0 2024-03-27T11:00:35.190 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.219+0000 7fbaa67fd740 1 Processor -- start 2024-03-27T11:00:35.190 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.220+0000 7fbaa67fd740 1 -- start start 2024-03-27T11:00:35.190 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.220+0000 7fbaa67fd740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561ab5afb980 0x561ab5afbda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:00:35.190 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.220+0000 7fbaa67fd740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561ab5afc370 0x561ab5b00e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:00:35.191 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.220+0000 7fbaa67fd740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561ab5b014e0 0x561ab5b03970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:00:35.191 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.220+0000 7fbaa67fd740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x561ab5a60d10 con 0x561ab5afc370 2024-03-27T11:00:35.191 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.220+0000 7fbaa67fd740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x561ab5a09260 con 0x561ab5b014e0 2024-03-27T11:00:35.191 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.220+0000 7fbaa67fd740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x561ab5a5ccc0 con 0x561ab5afb980 2024-03-27T11:00:35.191 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.220+0000 7fbaa1e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561ab5afb980 0x561ab5afbda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:00:35.191 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.220+0000 7fbaa1e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561ab5afb980 0x561ab5afbda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.97:41258/0 (socket says 172.21.15.97:41258) 2024-03-27T11:00:35.191 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.220+0000 7fbaa1e00640 1 -- 172.21.15.97:0/723858716 learned_addr learned my addr 172.21.15.97:0/723858716 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:00:35.192 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.221+0000 7fba9a000640 1 -- 172.21.15.97:0/3366858511 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7fba900079b0 con 0x7fba9c1001f0 2024-03-27T11:00:35.192 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.221+0000 7fbaa1e00640 1 -- 172.21.15.97:0/723858716 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561ab5b014e0 msgr2=0x561ab5b03970 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-27T11:00:35.192 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.221+0000 7fbaa1e00640 1 --2- 172.21.15.97:0/723858716 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561ab5b014e0 0x561ab5b03970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:35.192 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.221+0000 7fbaa1e00640 1 -- 172.21.15.97:0/723858716 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561ab5afc370 msgr2=0x561ab5b00e70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-27T11:00:35.193 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.221+0000 7fbaa1400640 1 --2- 172.21.15.97:0/723858716 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561ab5afc370 0x561ab5b00e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:00:35.193 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.222+0000 7fbaa1e00640 1 --2- 172.21.15.97:0/723858716 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561ab5afc370 0x561ab5b00e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:35.193 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.222+0000 7fbaa1e00640 1 -- 172.21.15.97:0/723858716 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ab5a00aa0 con 0x561ab5afb980 2024-03-27T11:00:35.193 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.222+0000 7fbaa1400640 1 --2- 172.21.15.97:0/723858716 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561ab5afc370 0x561ab5b00e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T11:00:35.193 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.222+0000 7fbaa1e00640 1 --2- 172.21.15.97:0/723858716 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561ab5afb980 0x561ab5afbda0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fba9400f410 tx=0x7fba9400f8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c61e7816dc5e75a server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:00:35.193 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.223+0000 7fba8e600640 1 -- 172.21.15.97:0/723858716 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba94015070 con 0x561ab5afb980 2024-03-27T11:00:35.193 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.223+0000 7fba8e600640 1 -- 172.21.15.97:0/723858716 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fba94011bb0 con 0x561ab5afb980 2024-03-27T11:00:35.194 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.223+0000 7fba8e600640 1 -- 172.21.15.97:0/723858716 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba94010c20 con 0x561ab5afb980 2024-03-27T11:00:35.194 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.224+0000 7fba9a000640 1 --2- 172.21.15.97:0/3366858511 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fba840a38a0 0x7fba840a5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:00:35.194 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.224+0000 7fba9a000640 1 -- 172.21.15.97:0/3366858511 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fba900cf140 con 0x7fba9c1001f0 2024-03-27T11:00:35.194 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.224+0000 7fbaa67fd740 1 -- 172.21.15.97:0/723858716 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561ab5afb980 msgr2=0x561ab5afbda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:00:35.195 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.224+0000 7fbaa67fd740 1 --2- 172.21.15.97:0/723858716 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561ab5afb980 0x561ab5afbda0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fba9400f410 tx=0x7fba9400f8d0 comp rx=0 tx=0).stop 2024-03-27T11:00:35.195 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.225+0000 7fbaa67fd740 1 -- 172.21.15.97:0/723858716 shutdown_connections 2024-03-27T11:00:35.195 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.225+0000 7fbaa67fd740 1 --2- 172.21.15.97:0/723858716 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561ab5b014e0 0x561ab5b03970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:35.195 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.225+0000 7fbaa67fd740 1 --2- 172.21.15.97:0/723858716 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561ab5afc370 0x561ab5b00e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:35.195 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.225+0000 7fbaa67fd740 1 --2- 172.21.15.97:0/723858716 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561ab5afb980 0x561ab5afbda0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:35.195 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.225+0000 7fbaa67fd740 1 -- 172.21.15.97:0/723858716 >> 172.21.15.97:0/723858716 conn(0x561ab5af6ef0 msgr2=0x561ab5af72f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:00:35.196 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.225+0000 7fbaa1400640 1 --2- 172.21.15.97:0/3366858511 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fba840a38a0 0x7fba840a5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:00:35.196 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.226+0000 7fbaa67fd740 1 -- 172.21.15.97:0/723858716 shutdown_connections 2024-03-27T11:00:35.196 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.226+0000 7fbaa67fd740 1 -- 172.21.15.97:0/723858716 wait complete. 2024-03-27T11:00:35.197 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.226+0000 7fbaa67fd740 1 -- start start 2024-03-27T11:00:35.197 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.227+0000 7fbaa67fd740 1 Processor -- start 2024-03-27T11:00:35.197 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.227+0000 7fbaa1400640 1 --2- 172.21.15.97:0/3366858511 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fba840a38a0 0x7fba840a5d60 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fba900a3d00 tx=0x7fba90000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:00:35.198 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.228+0000 7fbaa67fd740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561ab5afb980 0x561ab5b05d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:00:35.198 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.228+0000 7fbaa67fd740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561ab5afc370 0x561ab5b06250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:00:35.198 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.228+0000 7fbaa1e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561ab5afb980 0x561ab5b05d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:00:35.198 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.228+0000 7fbaa1e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561ab5afb980 0x561ab5b05d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.97:41264/0 (socket says 172.21.15.97:41264) 2024-03-27T11:00:35.199 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.228+0000 7fbaa67fd740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561ab5b014e0 0x561ab5b0d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:00:35.199 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.228+0000 7fbaa1e00640 1 -- 172.21.15.97:0/3326100939 learned_addr learned my addr 172.21.15.97:0/3326100939 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:00:35.199 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.228+0000 7fbaa67fd740 1 -- 172.21.15.97:0/3326100939 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x561ab5b04490 con 0x561ab5afc370 2024-03-27T11:00:35.199 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.228+0000 7fbaa67fd740 1 -- 172.21.15.97:0/3326100939 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x561ab5a1c630 con 0x561ab5b014e0 2024-03-27T11:00:35.199 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.228+0000 7fbaa67fd740 1 -- 172.21.15.97:0/3326100939 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x561ab5b04610 con 0x561ab5afb980 2024-03-27T11:00:35.199 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.229+0000 7fbaa67fd740 10 client.0 _ll_register_callbacks cb 0x561ab5a21c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T11:00:35.200 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.229+0000 7fbaa1e00640 1 --2- 172.21.15.97:0/3326100939 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561ab5b014e0 0x561ab5b0d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:00:35.200 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.229+0000 7fbaa0a00640 1 --2- 172.21.15.97:0/3326100939 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561ab5afc370 0x561ab5b06250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:00:35.200 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.229+0000 7fbaa1e00640 1 -- 172.21.15.97:0/3326100939 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561ab5b014e0 msgr2=0x561ab5b0d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:00:35.200 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.229+0000 7fbaa1e00640 1 --2- 172.21.15.97:0/3326100939 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561ab5b014e0 0x561ab5b0d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:35.200 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.229+0000 7fbaa1e00640 1 -- 172.21.15.97:0/3326100939 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561ab5afc370 msgr2=0x561ab5b06250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:00:35.201 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.231+0000 7fbaa1e00640 1 --2- 172.21.15.97:0/3326100939 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561ab5afc370 0x561ab5b06250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:35.202 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.231+0000 7fbaa1e00640 1 -- 172.21.15.97:0/3326100939 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ab5a285f0 con 0x561ab5afb980 2024-03-27T11:00:35.203 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.232+0000 7fbaa0a00640 1 --2- 172.21.15.97:0/3326100939 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561ab5afc370 0x561ab5b06250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T11:00:35.204 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.233+0000 7fbaa1e00640 1 --2- 172.21.15.97:0/3326100939 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561ab5afb980 0x561ab5b05d10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fba94012fa0 tx=0x7fba940103d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=80767f97c58db79b server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:00:35.204 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.233+0000 7fba82a00640 10 client.0 ms_handle_connect on v2:172.21.15.177:3300/0 2024-03-27T11:00:35.204 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.234+0000 7fba82a00640 1 -- 172.21.15.97:0/3326100939 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba94022040 con 0x561ab5afb980 2024-03-27T11:00:35.204 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.234+0000 7fba82a00640 1 -- 172.21.15.97:0/3326100939 <== mon.1 v2:172.21.15.177:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fba94010ba0 con 0x561ab5afb980 2024-03-27T11:00:35.205 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.234+0000 7fbaa67fd740 1 -- 172.21.15.97:0/3326100939 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba94011bb0 con 0x561ab5afb980 2024-03-27T11:00:35.205 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.234+0000 7fba82a00640 1 -- 172.21.15.97:0/3326100939 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fba94015040 con 0x561ab5afb980 2024-03-27T11:00:35.205 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.234+0000 7fbaa67fd740 10 client.25843 fetch_fsmap learned FSMap version 32 2024-03-27T11:00:35.205 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.234+0000 7fbaa67fd740 1 -- 172.21.15.97:0/3326100939 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x561ab5775210 con 0x561ab5afb980 2024-03-27T11:00:35.205 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.235+0000 7fba82a00640 1 -- 172.21.15.97:0/3326100939 <== mon.1 v2:172.21.15.177:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fba94011bb0 con 0x561ab5afb980 2024-03-27T11:00:35.206 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.236+0000 7fba82a00640 1 -- 172.21.15.97:0/3326100939 <== mon.1 v2:172.21.15.177:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fba94011700 con 0x561ab5afb980 2024-03-27T11:00:35.206 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.236+0000 7fbaa67fd740 10 client.25843 fetch_fsmap finished waiting for FSMap version 32 2024-03-27T11:00:35.206 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.236+0000 7fbaa67fd740 10 client.25843 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T11:00:35.206 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.236+0000 7fbaa67fd740 20 client.25843 populate_metadata read hostname 'smithi097' 2024-03-27T11:00:35.206 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.236+0000 7fbaa67fd740 1 --2- 172.21.15.97:0/3326100939 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x561ab5b117d0 0x561ab5b13bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:00:35.206 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.236+0000 7fbaa67fd740 4 client.25843 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T11:00:35.206 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.236+0000 7fbaa67fd740 1 -- 172.21.15.97:0/3326100939 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561ab5a6e250 con 0x561ab5b117d0 2024-03-27T11:00:35.207 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.237+0000 7fbaa0a00640 1 --2- 172.21.15.97:0/3326100939 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x561ab5b117d0 0x561ab5b13bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:00:35.208 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.238+0000 7fbaa0a00640 1 --2- 172.21.15.97:0/3326100939 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x561ab5b117d0 0x561ab5b13bc0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea1565911d2c5129 server_cookie=15dc3ff28663ebac in_seq=0 out_seq=0 2024-03-27T11:00:35.209 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.238+0000 7fba82a00640 10 client.25843 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T11:00:35.209 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.239+0000 7fba82a00640 1 -- 172.21.15.97:0/3326100939 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x561ab5a6e250 con 0x561ab5b117d0 2024-03-27T11:00:35.209 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.239+0000 7fba82a00640 10 client.25843 handle_command_reply: tid=0 2024-03-27T11:00:35.209 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.239+0000 7fbaa67fd740 1 client.25843 shutdown 2024-03-27T11:00:35.209 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.239+0000 7fbaa67fd740 1 -- 172.21.15.97:0/3326100939 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561ab5afb980 msgr2=0x561ab5b05d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:00:35.210 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.239+0000 7fbaa67fd740 1 --2- 172.21.15.97:0/3326100939 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561ab5afb980 0x561ab5b05d10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fba94012fa0 tx=0x7fba940103d0 comp rx=0 tx=0).stop 2024-03-27T11:00:35.211 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.241+0000 7fbaa67fd740 1 -- 172.21.15.97:0/3326100939 shutdown_connections 2024-03-27T11:00:35.212 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.241+0000 7fbaa67fd740 1 --2- 172.21.15.97:0/3326100939 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x561ab5b117d0 0x561ab5b13bc0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:35.212 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.241+0000 7fbaa67fd740 1 --2- 172.21.15.97:0/3326100939 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561ab5b014e0 0x561ab5b0d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:35.212 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.241+0000 7fbaa67fd740 1 --2- 172.21.15.97:0/3326100939 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561ab5afc370 0x561ab5b06250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:35.212 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.241+0000 7fbaa67fd740 1 --2- 172.21.15.97:0/3326100939 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561ab5afb980 0x561ab5b05d10 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:35.212 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.241+0000 7fbaa67fd740 1 -- 172.21.15.97:0/3326100939 >> 172.21.15.97:0/3326100939 conn(0x561ab5af6ef0 msgr2=0x561ab5af60f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:00:35.212 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.241+0000 7fba82a00640 0 client.25843 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T11:00:35.213 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.242+0000 7fbaa67fd740 1 -- 172.21.15.97:0/3326100939 shutdown_connections 2024-03-27T11:00:35.213 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.242+0000 7fbaa67fd740 1 -- 172.21.15.97:0/3326100939 wait complete. 2024-03-27T11:00:35.213 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.243+0000 7fbaa67fd740 20 client.25843 trim_cache size 0 max 16384 2024-03-27T11:00:35.247 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.276+0000 7fbaa67fd740 1 Processor -- start 2024-03-27T11:00:35.247 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.276+0000 7fbaa67fd740 1 -- start start 2024-03-27T11:00:35.247 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.277+0000 7fbaa67fd740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561ab5afb980 0x561ab5afa150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:00:35.247 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.277+0000 7fbaa67fd740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561ab5afc370 0x561ab5b0d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:00:35.247 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.277+0000 7fbaa67fd740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561ab5b014e0 0x561ab5afa690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:00:35.248 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.277+0000 7fbaa67fd740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fba9c1ab0f0 con 0x561ab5b014e0 2024-03-27T11:00:35.248 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.277+0000 7fbaa67fd740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fba9c1acc90 con 0x561ab5afc370 2024-03-27T11:00:35.248 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.277+0000 7fbaa67fd740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fba9c1ab7c0 con 0x561ab5afb980 2024-03-27T11:00:35.248 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.277+0000 7fbaa1e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561ab5afb980 0x561ab5afa150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:00:35.249 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.277+0000 7fbaa1e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561ab5afb980 0x561ab5afa150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.97:41268/0 (socket says 172.21.15.97:41268) 2024-03-27T11:00:35.249 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.277+0000 7fbaa1e00640 1 -- 172.21.15.97:0/809730844 learned_addr learned my addr 172.21.15.97:0/809730844 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:00:35.249 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.277+0000 7fbaa1e00640 1 --2- 172.21.15.97:0/809730844 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561ab5b014e0 0x561ab5afa690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:00:35.249 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.278+0000 7fbaa0a00640 1 --2- 172.21.15.97:0/809730844 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561ab5afc370 0x561ab5b0d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:00:35.249 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.278+0000 7fbaa1e00640 1 -- 172.21.15.97:0/809730844 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561ab5afc370 msgr2=0x561ab5b0d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:00:35.249 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.278+0000 7fbaa1e00640 1 --2- 172.21.15.97:0/809730844 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561ab5afc370 0x561ab5b0d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:35.249 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.278+0000 7fbaa1e00640 1 -- 172.21.15.97:0/809730844 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561ab5b014e0 msgr2=0x561ab5afa690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:00:35.249 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.278+0000 7fbaa1e00640 1 --2- 172.21.15.97:0/809730844 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561ab5b014e0 0x561ab5afa690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:35.249 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.278+0000 7fbaa1e00640 1 -- 172.21.15.97:0/809730844 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ab5841620 con 0x561ab5afb980 2024-03-27T11:00:35.249 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.278+0000 7fbaa1e00640 1 --2- 172.21.15.97:0/809730844 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561ab5afb980 0x561ab5afa150 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fba94004420 tx=0x7fba9400f8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35b5d66393459da7 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:00:35.250 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.279+0000 7fba8dc00640 1 -- 172.21.15.97:0/809730844 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba94004740 con 0x561ab5afb980 2024-03-27T11:00:35.250 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.279+0000 7fba8dc00640 1 -- 172.21.15.97:0/809730844 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fba94011500 con 0x561ab5afb980 2024-03-27T11:00:35.251 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.279+0000 7fba8dc00640 1 -- 172.21.15.97:0/809730844 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba94003cd0 con 0x561ab5afb980 2024-03-27T11:00:35.251 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.280+0000 7fbaa67fd740 1 -- 172.21.15.97:0/809730844 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561ab5afb980 msgr2=0x561ab5afa150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:00:35.251 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.280+0000 7fbaa67fd740 1 --2- 172.21.15.97:0/809730844 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561ab5afb980 0x561ab5afa150 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fba94004420 tx=0x7fba9400f8d0 comp rx=0 tx=0).stop 2024-03-27T11:00:35.251 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.280+0000 7fbaa67fd740 1 -- 172.21.15.97:0/809730844 shutdown_connections 2024-03-27T11:00:35.251 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.280+0000 7fbaa67fd740 1 --2- 172.21.15.97:0/809730844 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561ab5b014e0 0x561ab5afa690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:35.251 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.280+0000 7fbaa67fd740 1 --2- 172.21.15.97:0/809730844 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561ab5afc370 0x561ab5b0d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:35.251 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.280+0000 7fbaa67fd740 1 --2- 172.21.15.97:0/809730844 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561ab5afb980 0x561ab5afa150 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:35.251 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.280+0000 7fbaa67fd740 1 -- 172.21.15.97:0/809730844 >> 172.21.15.97:0/809730844 conn(0x561ab5af6ef0 msgr2=0x561ab5af72d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:00:35.251 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.280+0000 7fbaa67fd740 1 -- 172.21.15.97:0/809730844 shutdown_connections 2024-03-27T11:00:35.252 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.280+0000 7fbaa67fd740 1 -- 172.21.15.97:0/809730844 wait complete. 2024-03-27T11:00:35.252 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.281+0000 7fbaa67fd740 1 -- start start 2024-03-27T11:00:35.252 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.281+0000 7fbaa67fd740 1 Processor -- start 2024-03-27T11:00:35.252 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.282+0000 7fbaa67fd740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561ab5afb980 0x561ab5b0a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:00:35.252 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.282+0000 7fbaa67fd740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561ab5afc370 0x561ab5b0a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:00:35.252 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.282+0000 7fbaa67fd740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561ab5b014e0 0x561ab5b7e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:00:35.252 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.282+0000 7fbaa67fd740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fba9c1ac560 con 0x561ab5afb980 2024-03-27T11:00:35.253 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.282+0000 7fbaa67fd740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fba9c1abe90 con 0x561ab5b014e0 2024-03-27T11:00:35.253 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.282+0000 7fbaa67fd740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fba9c1a90d0 con 0x561ab5afc370 2024-03-27T11:00:35.253 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.282+0000 7fbaa67fd740 10 client.0 _ll_register_callbacks cb 0x7fba94009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T11:00:35.254 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.282+0000 7fbaa0a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561ab5afc370 0x561ab5b0a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:00:35.254 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.282+0000 7fbaa1e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561ab5afb980 0x561ab5b0a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:00:35.254 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.282+0000 7fbaa0a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561ab5afc370 0x561ab5b0a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.97:41276/0 (socket says 172.21.15.97:41276) 2024-03-27T11:00:35.254 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.282+0000 7fbaa0a00640 1 -- 172.21.15.97:0/1537218124 learned_addr learned my addr 172.21.15.97:0/1537218124 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:00:35.254 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.283+0000 7fbaa1e00640 1 --2- 172.21.15.97:0/1537218124 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561ab5b014e0 0x561ab5b7e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:00:35.254 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.283+0000 7fbaa1e00640 1 -- 172.21.15.97:0/1537218124 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561ab5afc370 msgr2=0x561ab5b0a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:00:35.254 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.283+0000 7fbaa1e00640 1 --2- 172.21.15.97:0/1537218124 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561ab5afc370 0x561ab5b0a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:35.254 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.283+0000 7fbaa1e00640 1 -- 172.21.15.97:0/1537218124 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561ab5b014e0 msgr2=0x561ab5b7e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:00:35.255 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.283+0000 7fbaa1e00640 1 --2- 172.21.15.97:0/1537218124 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561ab5b014e0 0x561ab5b7e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:35.255 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.283+0000 7fbaa1e00640 1 -- 172.21.15.97:0/1537218124 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ab5b7e8b0 con 0x561ab5afb980 2024-03-27T11:00:35.255 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.283+0000 7fbaa0a00640 1 --2- 172.21.15.97:0/1537218124 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561ab5afc370 0x561ab5b0a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T11:00:35.255 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.283+0000 7fbaa1e00640 1 --2- 172.21.15.97:0/1537218124 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561ab5afb980 0x561ab5b0a220 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fba9400ff50 tx=0x7fba94004450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99d546de1aecc128 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:00:35.255 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.283+0000 7fba82a00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T11:00:35.255 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.284+0000 7fba82a00640 1 -- 172.21.15.97:0/1537218124 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba94004740 con 0x561ab5afb980 2024-03-27T11:00:35.255 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.284+0000 7fba82a00640 1 -- 172.21.15.97:0/1537218124 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fba94003a50 con 0x561ab5afb980 2024-03-27T11:00:35.255 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.284+0000 7fbaa67fd740 1 -- 172.21.15.97:0/1537218124 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba94011500 con 0x561ab5afb980 2024-03-27T11:00:35.256 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.284+0000 7fba82a00640 1 -- 172.21.15.97:0/1537218124 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fba94003c50 con 0x561ab5afb980 2024-03-27T11:00:35.256 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.284+0000 7fbaa67fd740 10 client.16467 fetch_fsmap learned FSMap version 32 2024-03-27T11:00:35.256 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.284+0000 7fbaa67fd740 1 -- 172.21.15.97:0/1537218124 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x561ab5b7fbb0 con 0x561ab5afb980 2024-03-27T11:00:35.256 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.285+0000 7fba82a00640 1 -- 172.21.15.97:0/1537218124 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fba94011500 con 0x561ab5afb980 2024-03-27T11:00:35.256 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.285+0000 7fba82a00640 1 -- 172.21.15.97:0/1537218124 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fba94029b20 con 0x561ab5afb980 2024-03-27T11:00:35.256 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.285+0000 7fbaa67fd740 10 client.16467 fetch_fsmap finished waiting for FSMap version 32 2024-03-27T11:00:35.256 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.285+0000 7fbaa67fd740 10 client.16467 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T11:00:35.257 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.285+0000 7fbaa67fd740 20 client.16467 populate_metadata read hostname 'smithi097' 2024-03-27T11:00:35.257 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.285+0000 7fbaa67fd740 1 --2- 172.21.15.97:0/1537218124 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x561ab5b80590 0x561ab5b829b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:00:35.257 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.285+0000 7fbaa67fd740 4 client.16467 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-27T11:00:35.257 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.285+0000 7fbaa67fd740 1 -- 172.21.15.97:0/1537218124 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561ab5a685a0 con 0x561ab5b80590 2024-03-27T11:00:35.257 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.286+0000 7fbaa0a00640 1 --2- 172.21.15.97:0/1537218124 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x561ab5b80590 0x561ab5b829b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:00:35.257 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.286+0000 7fbaa0a00640 1 --2- 172.21.15.97:0/1537218124 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x561ab5b80590 0x561ab5b829b0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d1ed33090fc3339 server_cookie=a9e30754c8631760 in_seq=0 out_seq=0 2024-03-27T11:00:35.257 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.286+0000 7fba82a00640 10 client.16467 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T11:00:35.262 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.292+0000 7fba82a00640 1 -- 172.21.15.97:0/1537218124 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561ab5a685a0 con 0x561ab5b80590 2024-03-27T11:00:35.262 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.292+0000 7fba82a00640 10 client.16467 handle_command_reply: tid=0 2024-03-27T11:00:35.262 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.292+0000 7fbaa67fd740 1 client.16467 shutdown 2024-03-27T11:00:35.263 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.292+0000 7fbaa67fd740 1 -- 172.21.15.97:0/1537218124 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561ab5afb980 msgr2=0x561ab5b0a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:00:35.263 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.292+0000 7fbaa67fd740 1 --2- 172.21.15.97:0/1537218124 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561ab5afb980 0x561ab5b0a220 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fba9400ff50 tx=0x7fba94004450 comp rx=0 tx=0).stop 2024-03-27T11:00:35.263 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.293+0000 7fbaa67fd740 1 -- 172.21.15.97:0/1537218124 shutdown_connections 2024-03-27T11:00:35.263 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.293+0000 7fbaa67fd740 1 --2- 172.21.15.97:0/1537218124 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x561ab5b80590 0x561ab5b829b0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:35.263 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.293+0000 7fbaa67fd740 1 --2- 172.21.15.97:0/1537218124 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561ab5b014e0 0x561ab5b7e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:35.264 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.293+0000 7fbaa67fd740 1 --2- 172.21.15.97:0/1537218124 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561ab5afc370 0x561ab5b0a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:35.264 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.293+0000 7fbaa67fd740 1 --2- 172.21.15.97:0/1537218124 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561ab5afb980 0x561ab5b0a220 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:35.264 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.293+0000 7fbaa67fd740 1 -- 172.21.15.97:0/1537218124 >> 172.21.15.97:0/1537218124 conn(0x561ab5af6ef0 msgr2=0x561ab5afa150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:00:35.264 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.293+0000 7fba82a00640 0 client.16467 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T11:00:35.264 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.294+0000 7fbaa67fd740 1 -- 172.21.15.97:0/1537218124 shutdown_connections 2024-03-27T11:00:35.264 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.294+0000 7fbaa67fd740 1 -- 172.21.15.97:0/1537218124 wait complete. 2024-03-27T11:00:35.265 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.294+0000 7fbaa67fd740 20 client.16467 trim_cache size 0 max 16384 2024-03-27T11:00:35.265 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-27T11:00:35.265 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-27T11:00:35.265 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-27T11:00:35.265 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-27T11:00:35.266 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.295+0000 7fbaa3e00640 1 -- 172.21.15.97:0/3366858511 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fba840a38a0 msgr2=0x7fba840a5d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:00:35.266 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.295+0000 7fbaa3e00640 1 --2- 172.21.15.97:0/3366858511 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fba840a38a0 0x7fba840a5d60 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fba900a3d00 tx=0x7fba90000bc0 comp rx=0 tx=0).stop 2024-03-27T11:00:35.266 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.295+0000 7fbaa3e00640 1 -- 172.21.15.97:0/3366858511 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fba9c1001f0 msgr2=0x7fba9c195b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:00:35.266 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.295+0000 7fbaa3e00640 1 --2- 172.21.15.97:0/3366858511 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fba9c1001f0 0x7fba9c195b00 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fba9000a8d0 tx=0x7fba9000ad90 comp rx=0 tx=0).stop 2024-03-27T11:00:35.266 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.295+0000 7fbaa3e00640 1 -- 172.21.15.97:0/3366858511 shutdown_connections 2024-03-27T11:00:35.266 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.295+0000 7fbaa3e00640 1 --2- 172.21.15.97:0/3366858511 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba9c101da0 0x7fba9c19d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:35.266 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.295+0000 7fbaa3e00640 1 --2- 172.21.15.97:0/3366858511 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fba840a38a0 0x7fba840a5d60 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:35.266 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.295+0000 7fbaa3e00640 1 --2- 172.21.15.97:0/3366858511 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fba9c1013e0 0x7fba9c196040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:35.266 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.295+0000 7fbaa3e00640 1 --2- 172.21.15.97:0/3366858511 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fba9c1001f0 0x7fba9c195b00 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:35.266 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.295+0000 7fbaa3e00640 1 -- 172.21.15.97:0/3366858511 >> 172.21.15.97:0/3366858511 conn(0x7fba9c0fb7a0 msgr2=0x7fba9c104760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:00:35.266 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.295+0000 7fbaa3e00640 1 -- 172.21.15.97:0/3366858511 shutdown_connections 2024-03-27T11:00:35.266 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:35.295+0000 7fbaa3e00640 1 -- 172.21.15.97:0/3366858511 wait complete. 2024-03-27T11:00:35.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:00:35 smithi097 ceph-mon[31859]: pgmap v559: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 59 MiB/s wr, 959 op/s 2024-03-27T11:00:35.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:00:35 smithi177 ceph-mon[35294]: pgmap v559: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 59 MiB/s wr, 959 op/s 2024-03-27T11:00:35.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:00:35 smithi114 ceph-mon[35338]: pgmap v559: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 59 MiB/s wr, 959 op/s 2024-03-27T11:00:37.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:00:37 smithi097 ceph-mon[31859]: pgmap v560: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 57 MiB/s wr, 923 op/s 2024-03-27T11:00:37.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:00:37 smithi177 ceph-mon[35294]: pgmap v560: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 57 MiB/s wr, 923 op/s 2024-03-27T11:00:37.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:00:37 smithi114 ceph-mon[35338]: pgmap v560: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 57 MiB/s wr, 923 op/s 2024-03-27T11:00:39.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:00:39 smithi097 ceph-mon[31859]: pgmap v561: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 60 MiB/s wr, 965 op/s 2024-03-27T11:00:39.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:00:39 smithi177 ceph-mon[35294]: pgmap v561: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 60 MiB/s wr, 965 op/s 2024-03-27T11:00:39.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:00:39 smithi114 ceph-mon[35338]: pgmap v561: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 60 MiB/s wr, 965 op/s 2024-03-27T11:00:41.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:00:41 smithi097 ceph-mon[31859]: pgmap v562: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 63 MiB/s wr, 1.02k op/s 2024-03-27T11:00:41.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:00:41 smithi177 ceph-mon[35294]: pgmap v562: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 63 MiB/s wr, 1.02k op/s 2024-03-27T11:00:41.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:00:41 smithi114 ceph-mon[35338]: pgmap v562: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 63 MiB/s wr, 1.02k op/s 2024-03-27T11:00:43.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:00:43 smithi097 ceph-mon[31859]: pgmap v563: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 56 MiB/s wr, 906 op/s 2024-03-27T11:00:43.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:00:43 smithi177 ceph-mon[35294]: pgmap v563: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 56 MiB/s wr, 906 op/s 2024-03-27T11:00:43.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:00:43 smithi114 ceph-mon[35338]: pgmap v563: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 56 MiB/s wr, 906 op/s 2024-03-27T11:00:44.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:00:44 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T11:00:44.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:00:44 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T11:00:44.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:00:44 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T11:00:45.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:00:45 smithi097 ceph-mon[31859]: pgmap v564: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 57 MiB/s wr, 914 op/s 2024-03-27T11:00:45.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:00:45 smithi177 ceph-mon[35294]: pgmap v564: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 57 MiB/s wr, 914 op/s 2024-03-27T11:00:45.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:00:45 smithi114 ceph-mon[35338]: pgmap v564: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 57 MiB/s wr, 914 op/s 2024-03-27T11:00:46.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:00:46 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T11:00:46.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:00:46 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T11:00:46.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:00:46 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T11:00:46.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:00:46 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T11:00:46.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:00:46 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T11:00:46.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:00:46 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T11:00:46.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:00:46 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T11:00:46.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:00:46 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T11:00:46.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:00:46 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T11:00:47.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:00:47 smithi097 ceph-mon[31859]: pgmap v565: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 54 MiB/s wr, 872 op/s 2024-03-27T11:00:47.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:00:47 smithi177 ceph-mon[35294]: pgmap v565: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 54 MiB/s wr, 872 op/s 2024-03-27T11:00:47.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:00:47 smithi114 ceph-mon[35338]: pgmap v565: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 54 MiB/s wr, 872 op/s 2024-03-27T11:00:48.474 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.502+0000 7fd8cbc00640 1 -- 172.21.15.97:0/3626690511 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd8c4075b90 msgr2=0x7fd8c4073fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:00:48.474 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.502+0000 7fd8cbc00640 1 --2- 172.21.15.97:0/3626690511 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd8c4075b90 0x7fd8c4073fe0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac009930 tx=0x7fd8ac02bc20 comp rx=0 tx=0).stop 2024-03-27T11:00:48.475 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.503+0000 7fd8cbc00640 1 -- 172.21.15.97:0/3626690511 shutdown_connections 2024-03-27T11:00:48.475 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.503+0000 7fd8cbc00640 1 --2- 172.21.15.97:0/3626690511 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd8c410e150 0x7fd8c4110540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:48.475 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.503+0000 7fd8cbc00640 1 --2- 172.21.15.97:0/3626690511 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8c40745b0 0x7fd8c4074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:48.475 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.503+0000 7fd8cbc00640 1 --2- 172.21.15.97:0/3626690511 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd8c4075b90 0x7fd8c4073fe0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:48.475 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.503+0000 7fd8cbc00640 1 -- 172.21.15.97:0/3626690511 >> 172.21.15.97:0/3626690511 conn(0x7fd8c40fd110 msgr2=0x7fd8c40ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:00:48.475 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.503+0000 7fd8cbc00640 1 -- 172.21.15.97:0/3626690511 shutdown_connections 2024-03-27T11:00:48.475 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.503+0000 7fd8cbc00640 1 -- 172.21.15.97:0/3626690511 wait complete. 2024-03-27T11:00:48.476 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.505+0000 7fd8cbc00640 1 Processor -- start 2024-03-27T11:00:48.477 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.506+0000 7fd8cbc00640 1 -- start start 2024-03-27T11:00:48.477 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.506+0000 7fd8cbc00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd8c40745b0 0x7fd8c419e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:00:48.478 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.506+0000 7fd8cbc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8c4075b90 0x7fd8c419e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:00:48.478 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.506+0000 7fd8cbc00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd8c410e150 0x7fd8c41a58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:00:48.478 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.506+0000 7fd8cbc00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c4113ce0 con 0x7fd8c4075b90 2024-03-27T11:00:48.478 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.506+0000 7fd8cbc00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c4113b60 con 0x7fd8c40745b0 2024-03-27T11:00:48.478 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.507+0000 7fd8cbc00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c4113e60 con 0x7fd8c410e150 2024-03-27T11:00:48.478 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.507+0000 7fd8c9200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd8c40745b0 0x7fd8c419e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:00:48.479 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.507+0000 7fd8c9200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd8c40745b0 0x7fd8c419e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:50064/0 (socket says 172.21.15.97:50064) 2024-03-27T11:00:48.479 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.507+0000 7fd8c9200640 1 -- 172.21.15.97:0/2144393059 learned_addr learned my addr 172.21.15.97:0/2144393059 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:00:48.479 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.507+0000 7fd8c9c00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd8c410e150 0x7fd8c41a58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:00:48.479 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.508+0000 7fd8c9200640 1 -- 172.21.15.97:0/2144393059 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd8c410e150 msgr2=0x7fd8c41a58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:00:48.479 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.508+0000 7fd8c9200640 1 --2- 172.21.15.97:0/2144393059 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd8c410e150 0x7fd8c41a58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:48.479 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.508+0000 7fd8c3e00640 1 --2- 172.21.15.97:0/2144393059 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8c4075b90 0x7fd8c419e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:00:48.479 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.508+0000 7fd8c9200640 1 -- 172.21.15.97:0/2144393059 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8c4075b90 msgr2=0x7fd8c419e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:00:48.481 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.509+0000 7fd8c9200640 1 --2- 172.21.15.97:0/2144393059 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8c4075b90 0x7fd8c419e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:48.481 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.509+0000 7fd8c9200640 1 -- 172.21.15.97:0/2144393059 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8c41a5f50 con 0x7fd8c40745b0 2024-03-27T11:00:48.481 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.509+0000 7fd8c3e00640 1 --2- 172.21.15.97:0/2144393059 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8c4075b90 0x7fd8c419e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T11:00:48.482 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.510+0000 7fd8c9200640 1 --2- 172.21.15.97:0/2144393059 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd8c40745b0 0x7fd8c419e330 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac000c00 tx=0x7fd8ac005e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:00:48.482 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.511+0000 7fd8c1600640 1 -- 172.21.15.97:0/2144393059 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8ac043020 con 0x7fd8c40745b0 2024-03-27T11:00:48.483 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.511+0000 7fd8c1600640 1 -- 172.21.15.97:0/2144393059 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd8ac0043d0 con 0x7fd8c40745b0 2024-03-27T11:00:48.483 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.511+0000 7fd8c1600640 1 -- 172.21.15.97:0/2144393059 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8ac0039a0 con 0x7fd8c40745b0 2024-03-27T11:00:48.483 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.511+0000 7fd8cbc00640 1 -- 172.21.15.97:0/2144393059 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8c41a61e0 con 0x7fd8c40745b0 2024-03-27T11:00:48.483 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.511+0000 7fd8cbc00640 1 -- 172.21.15.97:0/2144393059 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8c41a66e0 con 0x7fd8c40745b0 2024-03-27T11:00:48.487 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.516+0000 7fd8ce986740 1 Processor -- start 2024-03-27T11:00:48.487 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.516+0000 7fd8ce986740 1 -- start start 2024-03-27T11:00:48.487 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.516+0000 7fd8ce986740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fc0e6b980 0x560fc0e6bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:00:48.488 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.516+0000 7fd8ce986740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560fc0e6c370 0x560fc0e70e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:00:48.488 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.516+0000 7fd8ce986740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fc0e714e0 0x560fc0e73970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:00:48.488 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.516+0000 7fd8ce986740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x560fc0dd0d10 con 0x560fc0e714e0 2024-03-27T11:00:48.489 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.516+0000 7fd8ce986740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x560fc0d79260 con 0x560fc0e6b980 2024-03-27T11:00:48.489 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.517+0000 7fd8ce986740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x560fc0dcccc0 con 0x560fc0e6c370 2024-03-27T11:00:48.489 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.517+0000 7fd8c1600640 1 -- 172.21.15.97:0/2144393059 <== mon.2 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7fd8ac046020 con 0x7fd8c40745b0 2024-03-27T11:00:48.489 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.517+0000 7fd8c9c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fc0e6b980 0x560fc0e6bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:00:48.489 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.517+0000 7fd8c9c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fc0e6b980 0x560fc0e6bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:50066/0 (socket says 172.21.15.97:50066) 2024-03-27T11:00:48.489 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.517+0000 7fd8c9c00640 1 -- 172.21.15.97:0/2731171342 learned_addr learned my addr 172.21.15.97:0/2731171342 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:00:48.489 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.517+0000 7fd8c9200640 1 --2- 172.21.15.97:0/2731171342 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560fc0e6c370 0x560fc0e70e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:00:48.489 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.517+0000 7fd8c3e00640 1 --2- 172.21.15.97:0/2731171342 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fc0e714e0 0x560fc0e73970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:00:48.490 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.518+0000 7fd8c9c00640 1 -- 172.21.15.97:0/2731171342 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560fc0e6c370 msgr2=0x560fc0e70e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:00:48.490 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.518+0000 7fd8c9c00640 1 --2- 172.21.15.97:0/2731171342 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560fc0e6c370 0x560fc0e70e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:48.490 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.518+0000 7fd8c9c00640 1 -- 172.21.15.97:0/2731171342 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fc0e714e0 msgr2=0x560fc0e73970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:00:48.490 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.518+0000 7fd8c9c00640 1 --2- 172.21.15.97:0/2731171342 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fc0e714e0 0x560fc0e73970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:48.490 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.518+0000 7fd8c9c00640 1 -- 172.21.15.97:0/2731171342 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fc0d70aa0 con 0x560fc0e6b980 2024-03-27T11:00:48.490 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.518+0000 7fd8c3e00640 1 --2- 172.21.15.97:0/2731171342 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fc0e714e0 0x560fc0e73970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T11:00:48.490 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.519+0000 7fd8c9c00640 1 --2- 172.21.15.97:0/2731171342 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fc0e6b980 0x560fc0e6bda0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc009c90 tx=0x7fd8bc011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f4e4d00be0a3777 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:00:48.490 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.519+0000 7fd8c1600640 1 --2- 172.21.15.97:0/2144393059 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fd8a80a37b0 0x7fd8a80a5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:00:48.491 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.519+0000 7fd8c1600640 1 -- 172.21.15.97:0/2144393059 <== mon.2 v2:172.21.15.114:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fd8ac0f1da0 con 0x7fd8c40745b0 2024-03-27T11:00:48.491 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.519+0000 7fd8c9200640 1 --2- 172.21.15.97:0/2144393059 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fd8a80a37b0 0x7fd8a80a5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:00:48.491 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.520+0000 7fd8ba000640 1 -- 172.21.15.97:0/2731171342 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8bc004230 con 0x560fc0e6b980 2024-03-27T11:00:48.491 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.520+0000 7fd8ba000640 1 -- 172.21.15.97:0/2731171342 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd8bc009e40 con 0x560fc0e6b980 2024-03-27T11:00:48.491 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.520+0000 7fd8ba000640 1 -- 172.21.15.97:0/2731171342 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8bc019760 con 0x560fc0e6b980 2024-03-27T11:00:48.492 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.521+0000 7fd8c9200640 1 --2- 172.21.15.97:0/2144393059 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fd8a80a37b0 0x7fd8a80a5c70 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac0bdf20 tx=0x7fd8ac009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:00:48.492 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.521+0000 7fd8ce986740 1 -- 172.21.15.97:0/2731171342 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fc0e6b980 msgr2=0x560fc0e6bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:00:48.492 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.521+0000 7fd8ce986740 1 --2- 172.21.15.97:0/2731171342 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fc0e6b980 0x560fc0e6bda0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc009c90 tx=0x7fd8bc011810 comp rx=0 tx=0).stop 2024-03-27T11:00:48.493 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.522+0000 7fd8ce986740 1 -- 172.21.15.97:0/2731171342 shutdown_connections 2024-03-27T11:00:48.493 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.522+0000 7fd8ce986740 1 --2- 172.21.15.97:0/2731171342 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fc0e714e0 0x560fc0e73970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:48.493 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.522+0000 7fd8ce986740 1 --2- 172.21.15.97:0/2731171342 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560fc0e6c370 0x560fc0e70e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:48.493 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.522+0000 7fd8ce986740 1 --2- 172.21.15.97:0/2731171342 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fc0e6b980 0x560fc0e6bda0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:48.493 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.522+0000 7fd8ce986740 1 -- 172.21.15.97:0/2731171342 >> 172.21.15.97:0/2731171342 conn(0x560fc0e66ef0 msgr2=0x560fc0e672f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:00:48.493 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.522+0000 7fd8ce986740 1 -- 172.21.15.97:0/2731171342 shutdown_connections 2024-03-27T11:00:48.494 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.522+0000 7fd8ce986740 1 -- 172.21.15.97:0/2731171342 wait complete. 2024-03-27T11:00:48.494 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.523+0000 7fd8ce986740 1 -- start start 2024-03-27T11:00:48.494 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.523+0000 7fd8ce986740 1 Processor -- start 2024-03-27T11:00:48.495 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.524+0000 7fd8ce986740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fc0e6b980 0x560fc0e75dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:00:48.495 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.524+0000 7fd8ce986740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fc0e6c370 0x560fc0e76310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:00:48.495 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.524+0000 7fd8ce986740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560fc0e714e0 0x560fc0e7d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:00:48.495 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.524+0000 7fd8ce986740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x560fc0e74490 con 0x560fc0e6c370 2024-03-27T11:00:48.495 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.524+0000 7fd8ce986740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x560fc0d8c630 con 0x560fc0e6b980 2024-03-27T11:00:48.495 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.524+0000 7fd8ce986740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x560fc0e74610 con 0x560fc0e714e0 2024-03-27T11:00:48.495 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.524+0000 7fd8ce986740 10 client.0 _ll_register_callbacks cb 0x560fc0d91c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T11:00:48.495 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.524+0000 7fd8c9c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fc0e6b980 0x560fc0e75dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:00:48.496 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.524+0000 7fd8c9c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fc0e6b980 0x560fc0e75dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:50080/0 (socket says 172.21.15.97:50080) 2024-03-27T11:00:48.496 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.524+0000 7fd8c9c00640 1 -- 172.21.15.97:0/3858832075 learned_addr learned my addr 172.21.15.97:0/3858832075 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:00:48.496 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.524+0000 7fd8c9c00640 1 --2- 172.21.15.97:0/3858832075 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560fc0e714e0 0x560fc0e7d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:00:48.496 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.524+0000 7fd8c3e00640 1 --2- 172.21.15.97:0/3858832075 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fc0e6c370 0x560fc0e76310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:00:48.496 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.525+0000 7fd8c9c00640 1 -- 172.21.15.97:0/3858832075 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560fc0e714e0 msgr2=0x560fc0e7d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:00:48.497 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.525+0000 7fd8c9c00640 1 --2- 172.21.15.97:0/3858832075 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560fc0e714e0 0x560fc0e7d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:48.497 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.525+0000 7fd8c9c00640 1 -- 172.21.15.97:0/3858832075 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fc0e6c370 msgr2=0x560fc0e76310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:00:48.497 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.525+0000 7fd8c9c00640 1 --2- 172.21.15.97:0/3858832075 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fc0e6c370 0x560fc0e76310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:48.497 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.525+0000 7fd8c9c00640 1 -- 172.21.15.97:0/3858832075 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560fc0d985f0 con 0x560fc0e6b980 2024-03-27T11:00:48.497 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.526+0000 7fd8c3e00640 1 --2- 172.21.15.97:0/3858832075 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fc0e6c370 0x560fc0e76310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T11:00:48.498 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.526+0000 7fd8c9c00640 1 --2- 172.21.15.97:0/3858832075 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fc0e6b980 0x560fc0e75dd0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc003cb0 tx=0x7fd8bc003d70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cdfbf03276c0a9e0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:00:48.498 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.526+0000 7fd8b2a00640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-03-27T11:00:48.498 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.527+0000 7fd8b2a00640 1 -- 172.21.15.97:0/3858832075 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8bc0196f0 con 0x560fc0e6b980 2024-03-27T11:00:48.498 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.527+0000 7fd8b2a00640 1 -- 172.21.15.97:0/3858832075 <== mon.2 v2:172.21.15.114:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fd8bc019890 con 0x560fc0e6b980 2024-03-27T11:00:48.498 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.527+0000 7fd8ce986740 1 -- 172.21.15.97:0/3858832075 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560fc0dde250 con 0x560fc0e6b980 2024-03-27T11:00:48.499 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.528+0000 7fd8b2a00640 1 -- 172.21.15.97:0/3858832075 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd8bc019a90 con 0x560fc0e6b980 2024-03-27T11:00:48.499 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.528+0000 7fd8ce986740 10 client.36134 fetch_fsmap learned FSMap version 32 2024-03-27T11:00:48.499 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.528+0000 7fd8ce986740 1 -- 172.21.15.97:0/3858832075 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x560fc0ae5210 con 0x560fc0e6b980 2024-03-27T11:00:48.499 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.528+0000 7fd8b2a00640 1 -- 172.21.15.97:0/3858832075 <== mon.2 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x560fc0dde250 con 0x560fc0e6b980 2024-03-27T11:00:48.500 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.529+0000 7fd8b2a00640 1 -- 172.21.15.97:0/3858832075 <== mon.2 v2:172.21.15.114:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fd8bc018070 con 0x560fc0e6b980 2024-03-27T11:00:48.500 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.529+0000 7fd8ce986740 10 client.36134 fetch_fsmap finished waiting for FSMap version 32 2024-03-27T11:00:48.500 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.529+0000 7fd8ce986740 10 client.36134 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T11:00:48.501 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.529+0000 7fd8ce986740 20 client.36134 populate_metadata read hostname 'smithi097' 2024-03-27T11:00:48.501 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.529+0000 7fd8ce986740 1 --2- 172.21.15.97:0/3858832075 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x560fc0e81800 0x560fc0e83bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:00:48.501 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.529+0000 7fd8ce986740 4 client.36134 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T11:00:48.501 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.529+0000 7fd8ce986740 1 -- 172.21.15.97:0/3858832075 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560fc0dd85a0 con 0x560fc0e81800 2024-03-27T11:00:48.501 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.530+0000 7fd8c3e00640 1 --2- 172.21.15.97:0/3858832075 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x560fc0e81800 0x560fc0e83bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:00:48.501 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.530+0000 7fd8c3e00640 1 --2- 172.21.15.97:0/3858832075 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x560fc0e81800 0x560fc0e83bf0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf7d0eb4f8b044a4 server_cookie=bb023d7fb4211404 in_seq=0 out_seq=0 2024-03-27T11:00:48.501 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.530+0000 7fd8b2a00640 10 client.36134 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T11:00:48.502 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.531+0000 7fd8b2a00640 1 -- 172.21.15.97:0/3858832075 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x560fc0dd85a0 con 0x560fc0e81800 2024-03-27T11:00:48.502 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.531+0000 7fd8b2a00640 10 client.36134 handle_command_reply: tid=0 2024-03-27T11:00:48.503 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.531+0000 7fd8ce986740 1 client.36134 shutdown 2024-03-27T11:00:48.503 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.532+0000 7fd8ce986740 1 -- 172.21.15.97:0/3858832075 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fc0e6b980 msgr2=0x560fc0e75dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:00:48.504 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.532+0000 7fd8ce986740 1 --2- 172.21.15.97:0/3858832075 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fc0e6b980 0x560fc0e75dd0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc003cb0 tx=0x7fd8bc003d70 comp rx=0 tx=0).stop 2024-03-27T11:00:48.504 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.532+0000 7fd8ce986740 1 -- 172.21.15.97:0/3858832075 shutdown_connections 2024-03-27T11:00:48.504 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.532+0000 7fd8ce986740 1 --2- 172.21.15.97:0/3858832075 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x560fc0e81800 0x560fc0e83bf0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:48.504 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.532+0000 7fd8ce986740 1 --2- 172.21.15.97:0/3858832075 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560fc0e714e0 0x560fc0e7d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:48.504 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.532+0000 7fd8ce986740 1 --2- 172.21.15.97:0/3858832075 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fc0e6c370 0x560fc0e76310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:48.504 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.532+0000 7fd8ce986740 1 --2- 172.21.15.97:0/3858832075 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fc0e6b980 0x560fc0e75dd0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:48.504 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.532+0000 7fd8ce986740 1 -- 172.21.15.97:0/3858832075 >> 172.21.15.97:0/3858832075 conn(0x560fc0e66ef0 msgr2=0x560fc0e660f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:00:48.504 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.532+0000 7fd8b2a00640 0 client.36134 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T11:00:48.504 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.532+0000 7fd8ce986740 1 -- 172.21.15.97:0/3858832075 shutdown_connections 2024-03-27T11:00:48.504 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.532+0000 7fd8ce986740 1 -- 172.21.15.97:0/3858832075 wait complete. 2024-03-27T11:00:48.504 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.532+0000 7fd8ce986740 20 client.36134 trim_cache size 0 max 16384 2024-03-27T11:00:48.534 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.563+0000 7fd8ce986740 1 Processor -- start 2024-03-27T11:00:48.534 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.563+0000 7fd8ce986740 1 -- start start 2024-03-27T11:00:48.534 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.563+0000 7fd8ce986740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fc0e6b980 0x560fc0e61f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:00:48.534 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.563+0000 7fd8ce986740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fc0e6c370 0x560fc0e7d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:00:48.534 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.563+0000 7fd8ce986740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560fc0e714e0 0x560fc0e760f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:00:48.535 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.563+0000 7fd8ce986740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c41af540 con 0x560fc0e6b980 2024-03-27T11:00:48.535 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.564+0000 7fd8ce986740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c41b10e0 con 0x560fc0e6c370 2024-03-27T11:00:48.535 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.564+0000 7fd8ce986740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c41afc10 con 0x560fc0e714e0 2024-03-27T11:00:48.535 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.564+0000 7fd8c3e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fc0e6c370 0x560fc0e7d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:00:48.535 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.564+0000 7fd8c3e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fc0e6c370 0x560fc0e7d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:50088/0 (socket says 172.21.15.97:50088) 2024-03-27T11:00:48.536 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.564+0000 7fd8c3e00640 1 -- 172.21.15.97:0/2180222219 learned_addr learned my addr 172.21.15.97:0/2180222219 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:00:48.536 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.564+0000 7fd8c9c00640 1 --2- 172.21.15.97:0/2180222219 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fc0e6b980 0x560fc0e61f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:00:48.536 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.564+0000 7fd8c3e00640 1 -- 172.21.15.97:0/2180222219 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560fc0e714e0 msgr2=0x560fc0e760f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-27T11:00:48.537 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.564+0000 7fd8c3e00640 1 --2- 172.21.15.97:0/2180222219 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560fc0e714e0 0x560fc0e760f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:48.537 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.564+0000 7fd8c3e00640 1 -- 172.21.15.97:0/2180222219 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fc0e6b980 msgr2=0x560fc0e61f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:00:48.537 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.564+0000 7fd8c3e00640 1 --2- 172.21.15.97:0/2180222219 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fc0e6b980 0x560fc0e61f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:48.537 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.564+0000 7fd8c3e00640 1 -- 172.21.15.97:0/2180222219 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fc0bb1620 con 0x560fc0e6c370 2024-03-27T11:00:48.537 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.565+0000 7fd8c9c00640 1 --2- 172.21.15.97:0/2180222219 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fc0e6b980 0x560fc0e61f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T11:00:48.537 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.565+0000 7fd8c3e00640 1 --2- 172.21.15.97:0/2180222219 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fc0e6c370 0x560fc0e7d2a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd8b40092f0 tx=0x7fd8b4003b00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8d6725b2a84abbd server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:00:48.537 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.565+0000 7fd8b9600640 1 -- 172.21.15.97:0/2180222219 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b401dd90 con 0x560fc0e6c370 2024-03-27T11:00:48.537 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.565+0000 7fd8b9600640 1 -- 172.21.15.97:0/2180222219 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd8b4022070 con 0x560fc0e6c370 2024-03-27T11:00:48.537 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.565+0000 7fd8b9600640 1 -- 172.21.15.97:0/2180222219 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b401ce50 con 0x560fc0e6c370 2024-03-27T11:00:48.538 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.567+0000 7fd8ce986740 1 -- 172.21.15.97:0/2180222219 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fc0e6c370 msgr2=0x560fc0e7d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:00:48.538 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.567+0000 7fd8ce986740 1 --2- 172.21.15.97:0/2180222219 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fc0e6c370 0x560fc0e7d2a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd8b40092f0 tx=0x7fd8b4003b00 comp rx=0 tx=0).stop 2024-03-27T11:00:48.538 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.567+0000 7fd8ce986740 1 -- 172.21.15.97:0/2180222219 shutdown_connections 2024-03-27T11:00:48.538 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.567+0000 7fd8ce986740 1 --2- 172.21.15.97:0/2180222219 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560fc0e714e0 0x560fc0e760f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:48.538 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.567+0000 7fd8ce986740 1 --2- 172.21.15.97:0/2180222219 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fc0e6c370 0x560fc0e7d2a0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:48.538 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.567+0000 7fd8ce986740 1 --2- 172.21.15.97:0/2180222219 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fc0e6b980 0x560fc0e61f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:48.539 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.567+0000 7fd8ce986740 1 -- 172.21.15.97:0/2180222219 >> 172.21.15.97:0/2180222219 conn(0x560fc0e66ef0 msgr2=0x560fc0e759e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:00:48.539 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.568+0000 7fd8ce986740 1 -- 172.21.15.97:0/2180222219 shutdown_connections 2024-03-27T11:00:48.539 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.568+0000 7fd8ce986740 1 -- 172.21.15.97:0/2180222219 wait complete. 2024-03-27T11:00:48.540 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.568+0000 7fd8ce986740 1 -- start start 2024-03-27T11:00:48.540 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.569+0000 7fd8ce986740 1 Processor -- start 2024-03-27T11:00:48.541 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.570+0000 7fd8ce986740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fc0e6b980 0x560fc0e7a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:00:48.542 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.571+0000 7fd8ce986740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fc0e714e0 0x560fc0e7ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:00:48.542 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.571+0000 7fd8c9c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fc0e6b980 0x560fc0e7a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:00:48.542 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.571+0000 7fd8ce986740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560fc0e7bab0 0x560fc0e7bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:00:48.543 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.571+0000 7fd8ce986740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c41b09b0 con 0x560fc0e6b980 2024-03-27T11:00:48.543 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.571+0000 7fd8ce986740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c41b02e0 con 0x560fc0e714e0 2024-03-27T11:00:48.543 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.571+0000 7fd8ce986740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c41ad520 con 0x560fc0e7bab0 2024-03-27T11:00:48.543 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.571+0000 7fd8ce986740 10 client.0 _ll_register_callbacks cb 0x7fd8b4009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T11:00:48.543 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.572+0000 7fd8c3e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fc0e714e0 0x560fc0e7ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:00:48.543 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.572+0000 7fd8c3e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fc0e714e0 0x560fc0e7ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:50104/0 (socket says 172.21.15.97:50104) 2024-03-27T11:00:48.543 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.572+0000 7fd8c3e00640 1 -- 172.21.15.97:0/1954608407 learned_addr learned my addr 172.21.15.97:0/1954608407 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:00:48.543 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.571+0000 7fd8c9c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fc0e6b980 0x560fc0e7a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57124/0 (socket says 172.21.15.97:57124) 2024-03-27T11:00:48.544 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.573+0000 7fd8c9c00640 1 --2- 172.21.15.97:0/1954608407 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560fc0e7bab0 0x560fc0e7bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:00:48.544 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.573+0000 7fd8c3e00640 1 -- 172.21.15.97:0/1954608407 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560fc0e7bab0 msgr2=0x560fc0e7bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:00:48.544 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.573+0000 7fd8c3e00640 1 --2- 172.21.15.97:0/1954608407 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560fc0e7bab0 0x560fc0e7bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:48.545 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.573+0000 7fd8c3e00640 1 -- 172.21.15.97:0/1954608407 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fc0e6b980 msgr2=0x560fc0e7a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:00:48.545 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.573+0000 7fd8c3e00640 1 --2- 172.21.15.97:0/1954608407 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fc0e6b980 0x560fc0e7a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:48.545 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.573+0000 7fd8c3e00640 1 -- 172.21.15.97:0/1954608407 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560fc0e7c470 con 0x560fc0e714e0 2024-03-27T11:00:48.545 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.573+0000 7fd8c9c00640 1 --2- 172.21.15.97:0/1954608407 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560fc0e7bab0 0x560fc0e7bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T11:00:48.546 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.574+0000 7fd8c3e00640 1 --2- 172.21.15.97:0/1954608407 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fc0e714e0 0x560fc0e7ab10 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd8b4005cc0 tx=0x7fd8b401dc20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4da47c9a1d5028c1 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:00:48.546 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.575+0000 7fd8b2a00640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-03-27T11:00:48.546 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.575+0000 7fd8b2a00640 1 -- 172.21.15.97:0/1954608407 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b401d730 con 0x560fc0e714e0 2024-03-27T11:00:48.546 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.575+0000 7fd8b2a00640 1 -- 172.21.15.97:0/1954608407 <== mon.2 v2:172.21.15.114:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fd8b401cb50 con 0x560fc0e714e0 2024-03-27T11:00:48.546 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.575+0000 7fd8ce986740 1 -- 172.21.15.97:0/1954608407 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8b4022070 con 0x560fc0e714e0 2024-03-27T11:00:48.547 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.576+0000 7fd8b2a00640 1 -- 172.21.15.97:0/1954608407 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd8b401cd50 con 0x560fc0e714e0 2024-03-27T11:00:48.547 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.576+0000 7fd8ce986740 10 client.36140 fetch_fsmap learned FSMap version 32 2024-03-27T11:00:48.547 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.576+0000 7fd8ce986740 1 -- 172.21.15.97:0/1954608407 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x560fc0ee7aa0 con 0x560fc0e714e0 2024-03-27T11:00:48.548 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.577+0000 7fd8b2a00640 1 -- 172.21.15.97:0/1954608407 <== mon.2 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd8b4022070 con 0x560fc0e714e0 2024-03-27T11:00:48.549 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.578+0000 7fd8b2a00640 1 -- 172.21.15.97:0/1954608407 <== mon.2 v2:172.21.15.114:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fd8b40304f0 con 0x560fc0e714e0 2024-03-27T11:00:48.549 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.578+0000 7fd8ce986740 10 client.36140 fetch_fsmap finished waiting for FSMap version 32 2024-03-27T11:00:48.549 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.578+0000 7fd8ce986740 10 client.36140 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T11:00:48.549 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.578+0000 7fd8ce986740 20 client.36140 populate_metadata read hostname 'smithi097' 2024-03-27T11:00:48.549 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.578+0000 7fd8ce986740 1 --2- 172.21.15.97:0/1954608407 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x560fc0ee82e0 0x560fc0eea700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:00:48.549 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.578+0000 7fd8ce986740 4 client.36140 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-27T11:00:48.549 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.578+0000 7fd8ce986740 1 -- 172.21.15.97:0/1954608407 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560fc0d56630 con 0x560fc0ee82e0 2024-03-27T11:00:48.550 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.578+0000 7fd8c9c00640 1 --2- 172.21.15.97:0/1954608407 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x560fc0ee82e0 0x560fc0eea700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:00:48.550 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.579+0000 7fd8c9c00640 1 --2- 172.21.15.97:0/1954608407 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x560fc0ee82e0 0x560fc0eea700 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5fa52cc22ba7e06e server_cookie=4ce4dcc1572029e8 in_seq=0 out_seq=0 2024-03-27T11:00:48.550 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.579+0000 7fd8b2a00640 10 client.36140 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T11:00:48.555 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.584+0000 7fd8b2a00640 1 -- 172.21.15.97:0/1954608407 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560fc0d56630 con 0x560fc0ee82e0 2024-03-27T11:00:48.555 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.584+0000 7fd8b2a00640 10 client.36140 handle_command_reply: tid=0 2024-03-27T11:00:48.555 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.584+0000 7fd8ce986740 1 client.36140 shutdown 2024-03-27T11:00:48.557 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.585+0000 7fd8ce986740 1 -- 172.21.15.97:0/1954608407 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fc0e714e0 msgr2=0x560fc0e7ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:00:48.557 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.585+0000 7fd8ce986740 1 --2- 172.21.15.97:0/1954608407 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fc0e714e0 0x560fc0e7ab10 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd8b4005cc0 tx=0x7fd8b401dc20 comp rx=0 tx=0).stop 2024-03-27T11:00:48.557 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.585+0000 7fd8ce986740 1 -- 172.21.15.97:0/1954608407 shutdown_connections 2024-03-27T11:00:48.557 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.585+0000 7fd8ce986740 1 --2- 172.21.15.97:0/1954608407 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x560fc0ee82e0 0x560fc0eea700 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:48.557 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.585+0000 7fd8ce986740 1 --2- 172.21.15.97:0/1954608407 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560fc0e7bab0 0x560fc0e7bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:48.557 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.585+0000 7fd8ce986740 1 --2- 172.21.15.97:0/1954608407 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fc0e714e0 0x560fc0e7ab10 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:48.557 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.585+0000 7fd8ce986740 1 --2- 172.21.15.97:0/1954608407 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fc0e6b980 0x560fc0e7a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:48.557 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.585+0000 7fd8ce986740 1 -- 172.21.15.97:0/1954608407 >> 172.21.15.97:0/1954608407 conn(0x560fc0e66ef0 msgr2=0x560fc0e6a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:00:48.557 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.585+0000 7fd8b2a00640 0 client.36140 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T11:00:48.557 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.585+0000 7fd8ce986740 1 -- 172.21.15.97:0/1954608407 shutdown_connections 2024-03-27T11:00:48.557 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.585+0000 7fd8ce986740 1 -- 172.21.15.97:0/1954608407 wait complete. 2024-03-27T11:00:48.558 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.586+0000 7fd8ce986740 20 client.36140 trim_cache size 0 max 16384 2024-03-27T11:00:48.558 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-27T11:00:48.558 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-27T11:00:48.558 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-27T11:00:48.558 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-27T11:00:48.559 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.587+0000 7fd8cbc00640 1 -- 172.21.15.97:0/2144393059 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fd8a80a37b0 msgr2=0x7fd8a80a5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:00:48.559 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.587+0000 7fd8cbc00640 1 --2- 172.21.15.97:0/2144393059 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fd8a80a37b0 0x7fd8a80a5c70 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac0bdf20 tx=0x7fd8ac009260 comp rx=0 tx=0).stop 2024-03-27T11:00:48.559 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.587+0000 7fd8cbc00640 1 -- 172.21.15.97:0/2144393059 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd8c40745b0 msgr2=0x7fd8c419e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:00:48.559 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.587+0000 7fd8cbc00640 1 --2- 172.21.15.97:0/2144393059 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd8c40745b0 0x7fd8c419e330 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac000c00 tx=0x7fd8ac005e60 comp rx=0 tx=0).stop 2024-03-27T11:00:48.559 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.587+0000 7fd8cbc00640 1 -- 172.21.15.97:0/2144393059 shutdown_connections 2024-03-27T11:00:48.559 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.587+0000 7fd8cbc00640 1 --2- 172.21.15.97:0/2144393059 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd8c410e150 0x7fd8c41a58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:48.559 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.587+0000 7fd8cbc00640 1 --2- 172.21.15.97:0/2144393059 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8c4075b90 0x7fd8c419e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:48.560 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.587+0000 7fd8cbc00640 1 --2- 172.21.15.97:0/2144393059 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fd8a80a37b0 0x7fd8a80a5c70 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:48.560 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.587+0000 7fd8cbc00640 1 --2- 172.21.15.97:0/2144393059 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd8c40745b0 0x7fd8c419e330 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:00:48.560 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.587+0000 7fd8cbc00640 1 -- 172.21.15.97:0/2144393059 >> 172.21.15.97:0/2144393059 conn(0x7fd8c40fd110 msgr2=0x7fd8c41014f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:00:48.560 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.587+0000 7fd8cbc00640 1 -- 172.21.15.97:0/2144393059 shutdown_connections 2024-03-27T11:00:48.560 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:00:48.587+0000 7fd8cbc00640 1 -- 172.21.15.97:0/2144393059 wait complete. 2024-03-27T11:00:49.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:00:49 smithi097 ceph-mon[31859]: pgmap v566: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 56 MiB/s wr, 906 op/s 2024-03-27T11:00:49.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:00:49 smithi177 ceph-mon[35294]: pgmap v566: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 56 MiB/s wr, 906 op/s 2024-03-27T11:00:49.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:00:49 smithi114 ceph-mon[35338]: pgmap v566: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 56 MiB/s wr, 906 op/s 2024-03-27T11:00:51.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:00:51 smithi097 ceph-mon[31859]: pgmap v567: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 61 MiB/s wr, 980 op/s 2024-03-27T11:00:51.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:00:51 smithi177 ceph-mon[35294]: pgmap v567: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 61 MiB/s wr, 980 op/s 2024-03-27T11:00:51.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:00:51 smithi114 ceph-mon[35338]: pgmap v567: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 61 MiB/s wr, 980 op/s 2024-03-27T11:00:53.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:00:53 smithi097 ceph-mon[31859]: pgmap v568: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 55 MiB/s wr, 892 op/s 2024-03-27T11:00:53.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:00:53 smithi177 ceph-mon[35294]: pgmap v568: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 55 MiB/s wr, 892 op/s 2024-03-27T11:00:53.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:00:53 smithi114 ceph-mon[35338]: pgmap v568: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 55 MiB/s wr, 892 op/s 2024-03-27T11:00:55.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:00:55 smithi097 ceph-mon[31859]: pgmap v569: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 57 MiB/s wr, 916 op/s 2024-03-27T11:00:55.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:00:55 smithi177 ceph-mon[35294]: pgmap v569: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 57 MiB/s wr, 916 op/s 2024-03-27T11:00:55.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:00:55 smithi114 ceph-mon[35338]: pgmap v569: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 57 MiB/s wr, 916 op/s 2024-03-27T11:00:57.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:00:57 smithi097 ceph-mon[31859]: pgmap v570: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 55 MiB/s wr, 893 op/s 2024-03-27T11:00:57.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:00:57 smithi177 ceph-mon[35294]: pgmap v570: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 55 MiB/s wr, 893 op/s 2024-03-27T11:00:57.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:00:57 smithi114 ceph-mon[35338]: pgmap v570: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 55 MiB/s wr, 893 op/s 2024-03-27T11:00:59.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:00:59 smithi097 ceph-mon[31859]: pgmap v571: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 59 MiB/s wr, 948 op/s 2024-03-27T11:00:59.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:00:59 smithi177 ceph-mon[35294]: pgmap v571: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 59 MiB/s wr, 948 op/s 2024-03-27T11:00:59.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:00:59 smithi114 ceph-mon[35338]: pgmap v571: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 59 MiB/s wr, 948 op/s 2024-03-27T11:01:01.393 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:01:01 smithi114 ceph-mon[35338]: pgmap v572: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 64 MiB/s wr, 1.03k op/s 2024-03-27T11:01:01.393 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:01:01 smithi114 ceph-mon[35338]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T11:01:01.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:01:01 smithi177 ceph-mon[35294]: pgmap v572: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 64 MiB/s wr, 1.03k op/s 2024-03-27T11:01:01.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:01:01 smithi177 ceph-mon[35294]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T11:01:01.758 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.787+0000 7fdcd8200640 1 -- 172.21.15.97:0/3456754832 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdcd0109d60 msgr2=0x7fdcd01105f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:01:01.759 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.787+0000 7fdcd8200640 1 --2- 172.21.15.97:0/3456754832 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdcd0109d60 0x7fdcd01105f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fdcc8009ca0 tx=0x7fdcc802c0f0 comp rx=0 tx=0).stop 2024-03-27T11:01:01.759 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.787+0000 7fdcd8200640 1 -- 172.21.15.97:0/3456754832 shutdown_connections 2024-03-27T11:01:01.759 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.787+0000 7fdcd8200640 1 --2- 172.21.15.97:0/3456754832 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdcd0109d60 0x7fdcd01105f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:01.759 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.787+0000 7fdcd8200640 1 --2- 172.21.15.97:0/3456754832 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcd01093a0 0x7fdcd0109820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:01.759 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.787+0000 7fdcd8200640 1 --2- 172.21.15.97:0/3456754832 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdcd01081b0 0x7fdcd01085d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:01.759 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.787+0000 7fdcd8200640 1 -- 172.21.15.97:0/3456754832 >> 172.21.15.97:0/3456754832 conn(0x7fdcd0077160 msgr2=0x7fdcd0079580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:01:01.760 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.788+0000 7fdcd8200640 1 -- 172.21.15.97:0/3456754832 shutdown_connections 2024-03-27T11:01:01.760 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.788+0000 7fdcd8200640 1 -- 172.21.15.97:0/3456754832 wait complete. 2024-03-27T11:01:01.760 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.789+0000 7fdcd8200640 1 Processor -- start 2024-03-27T11:01:01.760 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.789+0000 7fdcd8200640 1 -- start start 2024-03-27T11:01:01.761 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.790+0000 7fdcd8200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdcd01081b0 0x7fdcd019e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:01:01.762 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.790+0000 7fdcd8200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdcd01093a0 0x7fdcd019e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:01:01.762 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.790+0000 7fdcd8200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcd0109d60 0x7fdcd01a5860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:01:01.762 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.790+0000 7fdcd8200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd0113cf0 con 0x7fdcd0109d60 2024-03-27T11:01:01.762 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.790+0000 7fdcd8200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd0113b70 con 0x7fdcd01081b0 2024-03-27T11:01:01.762 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.790+0000 7fdcd8200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd0113e70 con 0x7fdcd01093a0 2024-03-27T11:01:01.762 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.790+0000 7fdcd6200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcd0109d60 0x7fdcd01a5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:01:01.762 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.790+0000 7fdcd6200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcd0109d60 0x7fdcd01a5860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55364/0 (socket says 172.21.15.97:55364) 2024-03-27T11:01:01.762 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.790+0000 7fdcd6200640 1 -- 172.21.15.97:0/629987309 learned_addr learned my addr 172.21.15.97:0/629987309 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:01:01.763 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.790+0000 7fdcd5800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdcd01081b0 0x7fdcd019e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:01:01.763 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.791+0000 7fdcd4e00640 1 --2- 172.21.15.97:0/629987309 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdcd01093a0 0x7fdcd019e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:01:01.763 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.791+0000 7fdcd6200640 1 -- 172.21.15.97:0/629987309 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdcd01093a0 msgr2=0x7fdcd019e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:01:01.763 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.791+0000 7fdcd6200640 1 --2- 172.21.15.97:0/629987309 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdcd01093a0 0x7fdcd019e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:01.763 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.791+0000 7fdcd6200640 1 -- 172.21.15.97:0/629987309 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdcd01081b0 msgr2=0x7fdcd019e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:01:01.763 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.791+0000 7fdcd6200640 1 --2- 172.21.15.97:0/629987309 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdcd01081b0 0x7fdcd019e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:01.763 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.791+0000 7fdcd6200640 1 -- 172.21.15.97:0/629987309 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcd01a5ed0 con 0x7fdcd0109d60 2024-03-27T11:01:01.763 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.791+0000 7fdcd5800640 1 --2- 172.21.15.97:0/629987309 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdcd01081b0 0x7fdcd019e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T11:01:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.791+0000 7fdcd6200640 1 --2- 172.21.15.97:0/629987309 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcd0109d60 0x7fdcd01a5860 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fdcc8032380 tx=0x7fdcc8032820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:01:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.792+0000 7fdcce000640 1 -- 172.21.15.97:0/629987309 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc8041020 con 0x7fdcd0109d60 2024-03-27T11:01:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.792+0000 7fdcce000640 1 -- 172.21.15.97:0/629987309 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdcc8003e20 con 0x7fdcd0109d60 2024-03-27T11:01:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.792+0000 7fdcce000640 1 -- 172.21.15.97:0/629987309 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc8039040 con 0x7fdcd0109d60 2024-03-27T11:01:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.792+0000 7fdcd8200640 1 -- 172.21.15.97:0/629987309 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcd01a6160 con 0x7fdcd0109d60 2024-03-27T11:01:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.792+0000 7fdcd8200640 1 -- 172.21.15.97:0/629987309 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcd01a6600 con 0x7fdcd0109d60 2024-03-27T11:01:01.769 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.797+0000 7fdcce000640 1 -- 172.21.15.97:0/629987309 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7fdcc8044020 con 0x7fdcd0109d60 2024-03-27T11:01:01.770 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.799+0000 7fdcdb032740 1 Processor -- start 2024-03-27T11:01:01.771 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.800+0000 7fdcce000640 1 --2- 172.21.15.97:0/629987309 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fdcb80a37b0 0x7fdcb80a5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:01:01.771 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.800+0000 7fdcdb032740 1 -- start start 2024-03-27T11:01:01.772 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.800+0000 7fdcce000640 1 -- 172.21.15.97:0/629987309 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fdcc80f1ea0 con 0x7fdcd0109d60 2024-03-27T11:01:01.772 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.800+0000 7fdcd4e00640 1 --2- 172.21.15.97:0/629987309 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fdcb80a37b0 0x7fdcb80a5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:01:01.772 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.801+0000 7fdcdb032740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55796279e980 0x55796279eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:01:01.772 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.801+0000 7fdcd4e00640 1 --2- 172.21.15.97:0/629987309 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fdcb80a37b0 0x7fdcb80a5c70 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc0095f0 tx=0x7fdcbc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:01:01.773 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.802+0000 7fdcdb032740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55796279f370 0x5579627a3e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:01:01.773 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.802+0000 7fdcd5800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55796279e980 0x55796279eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:01:01.773 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.802+0000 7fdcd5800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55796279e980 0x55796279eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:43572/0 (socket says 172.21.15.97:43572) 2024-03-27T11:01:01.774 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.803+0000 7fdcdb032740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5579627a44e0 0x5579627a6970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:01:01.774 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.803+0000 7fdcdb032740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x557962703d10 con 0x55796279f370 2024-03-27T11:01:01.774 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.803+0000 7fdcdb032740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5579626ac260 con 0x55796279e980 2024-03-27T11:01:01.774 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.803+0000 7fdcdb032740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5579626ffcc0 con 0x5579627a44e0 2024-03-27T11:01:01.774 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.803+0000 7fdcd5800640 1 -- 172.21.15.97:0/780377562 learned_addr learned my addr 172.21.15.97:0/780377562 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:01:01.775 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.803+0000 7fdcd6200640 1 --2- 172.21.15.97:0/780377562 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5579627a44e0 0x5579627a6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:01:01.775 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.803+0000 7fdcd4e00640 1 --2- 172.21.15.97:0/780377562 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55796279f370 0x5579627a3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:01:01.775 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.803+0000 7fdcd5800640 1 -- 172.21.15.97:0/780377562 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5579627a44e0 msgr2=0x5579627a6970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:01:01.775 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.803+0000 7fdcd5800640 1 --2- 172.21.15.97:0/780377562 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5579627a44e0 0x5579627a6970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:01.775 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.803+0000 7fdcd5800640 1 -- 172.21.15.97:0/780377562 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55796279f370 msgr2=0x5579627a3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:01:01.775 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.803+0000 7fdcd5800640 1 --2- 172.21.15.97:0/780377562 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55796279f370 0x5579627a3e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:01.775 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.803+0000 7fdcd5800640 1 -- 172.21.15.97:0/780377562 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579626a3aa0 con 0x55796279e980 2024-03-27T11:01:01.775 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.804+0000 7fdcd4e00640 1 --2- 172.21.15.97:0/780377562 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55796279f370 0x5579627a3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T11:01:01.776 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.804+0000 7fdcd5800640 1 --2- 172.21.15.97:0/780377562 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55796279e980 0x55796279eda0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fdcc40114f0 tx=0x7fdcc40119b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c4a187757f0fa2f server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:01:01.776 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.804+0000 7fdcc2a00640 1 -- 172.21.15.97:0/780377562 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc401e070 con 0x55796279e980 2024-03-27T11:01:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.804+0000 7fdcc2a00640 1 -- 172.21.15.97:0/780377562 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdcc40035f0 con 0x55796279e980 2024-03-27T11:01:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.804+0000 7fdcc2a00640 1 -- 172.21.15.97:0/780377562 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc400f820 con 0x55796279e980 2024-03-27T11:01:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.805+0000 7fdcdb032740 1 -- 172.21.15.97:0/780377562 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55796279e980 msgr2=0x55796279eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:01:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.805+0000 7fdcdb032740 1 --2- 172.21.15.97:0/780377562 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55796279e980 0x55796279eda0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fdcc40114f0 tx=0x7fdcc40119b0 comp rx=0 tx=0).stop 2024-03-27T11:01:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.805+0000 7fdcdb032740 1 -- 172.21.15.97:0/780377562 shutdown_connections 2024-03-27T11:01:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.805+0000 7fdcdb032740 1 --2- 172.21.15.97:0/780377562 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5579627a44e0 0x5579627a6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.805+0000 7fdcdb032740 1 --2- 172.21.15.97:0/780377562 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55796279f370 0x5579627a3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.805+0000 7fdcdb032740 1 --2- 172.21.15.97:0/780377562 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55796279e980 0x55796279eda0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.805+0000 7fdcdb032740 1 -- 172.21.15.97:0/780377562 >> 172.21.15.97:0/780377562 conn(0x557962799ef0 msgr2=0x55796279a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:01:01.778 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.806+0000 7fdcdb032740 1 -- 172.21.15.97:0/780377562 shutdown_connections 2024-03-27T11:01:01.778 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.806+0000 7fdcdb032740 1 -- 172.21.15.97:0/780377562 wait complete. 2024-03-27T11:01:01.778 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.807+0000 7fdcdb032740 1 -- start start 2024-03-27T11:01:01.778 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.807+0000 7fdcdb032740 1 Processor -- start 2024-03-27T11:01:01.779 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.808+0000 7fdcdb032740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55796279e980 0x5579627a8d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:01:01.779 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.808+0000 7fdcdb032740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55796279f370 0x5579627a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:01:01.779 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.808+0000 7fdcdb032740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5579627a44e0 0x5579627b02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:01:01.779 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.808+0000 7fdcdb032740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5579627a7490 con 0x55796279e980 2024-03-27T11:01:01.779 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.808+0000 7fdcdb032740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5579626bf630 con 0x5579627a44e0 2024-03-27T11:01:01.779 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.808+0000 7fdcdb032740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5579627a7610 con 0x55796279f370 2024-03-27T11:01:01.780 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.808+0000 7fdcdb032740 10 client.0 _ll_register_callbacks cb 0x5579626c4c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T11:01:01.780 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.808+0000 7fdcd5800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55796279e980 0x5579627a8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:01:01.780 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.808+0000 7fdcd4e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55796279f370 0x5579627a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:01:01.780 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.808+0000 7fdcd5800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55796279e980 0x5579627a8d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55382/0 (socket says 172.21.15.97:55382) 2024-03-27T11:01:01.780 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.808+0000 7fdcd5800640 1 -- 172.21.15.97:0/3821767842 learned_addr learned my addr 172.21.15.97:0/3821767842 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:01:01.780 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.808+0000 7fdcd6200640 1 --2- 172.21.15.97:0/3821767842 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5579627a44e0 0x5579627b02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:01:01.781 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.809+0000 7fdcd4e00640 1 -- 172.21.15.97:0/3821767842 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5579627a44e0 msgr2=0x5579627b02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:01:01.781 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.809+0000 7fdcd4e00640 1 --2- 172.21.15.97:0/3821767842 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5579627a44e0 0x5579627b02c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:01.781 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.809+0000 7fdcd4e00640 1 -- 172.21.15.97:0/3821767842 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55796279e980 msgr2=0x5579627a8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:01:01.781 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.809+0000 7fdcd4e00640 1 --2- 172.21.15.97:0/3821767842 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55796279e980 0x5579627a8d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:01.781 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.809+0000 7fdcd4e00640 1 -- 172.21.15.97:0/3821767842 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579626cb5f0 con 0x55796279f370 2024-03-27T11:01:01.781 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.809+0000 7fdcd5800640 1 --2- 172.21.15.97:0/3821767842 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55796279e980 0x5579627a8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T11:01:01.781 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.809+0000 7fdcd4e00640 1 --2- 172.21.15.97:0/3821767842 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55796279f370 0x5579627a9280 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x55796279fde0 tx=0x7fdcbc004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e6222b2402128fb server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:01:01.781 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.809+0000 7fdcb7400640 10 client.0 ms_handle_connect on v2:172.21.15.177:3300/0 2024-03-27T11:01:01.782 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.810+0000 7fdcb7400640 1 -- 172.21.15.97:0/3821767842 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc01fb10 con 0x55796279f370 2024-03-27T11:01:01.782 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.810+0000 7fdcb7400640 1 -- 172.21.15.97:0/3821767842 <== mon.1 v2:172.21.15.177:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fdcbc01fcb0 con 0x55796279f370 2024-03-27T11:01:01.782 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.810+0000 7fdcdb032740 1 -- 172.21.15.97:0/3821767842 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdcc40035f0 con 0x55796279f370 2024-03-27T11:01:01.782 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.811+0000 7fdcb7400640 1 -- 172.21.15.97:0/3821767842 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fdcbc029430 con 0x55796279f370 2024-03-27T11:01:01.783 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.811+0000 7fdcdb032740 10 client.25873 fetch_fsmap learned FSMap version 33 2024-03-27T11:01:01.783 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.811+0000 7fdcdb032740 1 -- 172.21.15.97:0/3821767842 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x557962418210 con 0x55796279f370 2024-03-27T11:01:01.783 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.811+0000 7fdcb7400640 1 -- 172.21.15.97:0/3821767842 <== mon.1 v2:172.21.15.177:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdcc40035f0 con 0x55796279f370 2024-03-27T11:01:01.783 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.811+0000 7fdcb7400640 1 -- 172.21.15.97:0/3821767842 <== mon.1 v2:172.21.15.177:3300/0 5 ==== fsmap(e 33) ==== 8358+0+0 (secure 0 0 0) 0x7fdcbc0205b0 con 0x55796279f370 2024-03-27T11:01:01.783 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.811+0000 7fdcdb032740 10 client.25873 fetch_fsmap finished waiting for FSMap version 33 2024-03-27T11:01:01.783 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.811+0000 7fdcdb032740 10 client.25873 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T11:01:01.784 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.811+0000 7fdcdb032740 20 client.25873 populate_metadata read hostname 'smithi097' 2024-03-27T11:01:01.784 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.811+0000 7fdcdb032740 1 --2- 172.21.15.97:0/3821767842 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5579627b4c20 0x5579627b70e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:01:01.784 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.811+0000 7fdcdb032740 4 client.25873 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T11:01:01.784 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.811+0000 7fdcdb032740 1 -- 172.21.15.97:0/3821767842 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557962711250 con 0x5579627b4c20 2024-03-27T11:01:01.784 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.812+0000 7fdcd5800640 1 --2- 172.21.15.97:0/3821767842 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5579627b4c20 0x5579627b70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:01:01.784 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.813+0000 7fdcb7400640 10 client.25873 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T11:01:01.784 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.813+0000 7fdcd5800640 1 --2- 172.21.15.97:0/3821767842 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5579627b4c20 0x5579627b70e0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80c02aff8d8f7bf7 server_cookie=c2cf8538812d0c7 in_seq=0 out_seq=0 2024-03-27T11:01:01.785 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.814+0000 7fdcb7400640 1 -- 172.21.15.97:0/3821767842 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x557962711250 con 0x5579627b4c20 2024-03-27T11:01:01.785 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.814+0000 7fdcb7400640 10 client.25873 handle_command_reply: tid=0 2024-03-27T11:01:01.785 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.814+0000 7fdcdb032740 1 client.25873 shutdown 2024-03-27T11:01:01.785 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.814+0000 7fdcdb032740 1 -- 172.21.15.97:0/3821767842 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55796279f370 msgr2=0x5579627a9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:01:01.785 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.814+0000 7fdcdb032740 1 --2- 172.21.15.97:0/3821767842 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55796279f370 0x5579627a9280 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x55796279fde0 tx=0x7fdcbc004730 comp rx=0 tx=0).stop 2024-03-27T11:01:01.786 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.815+0000 7fdcdb032740 1 -- 172.21.15.97:0/3821767842 shutdown_connections 2024-03-27T11:01:01.786 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.815+0000 7fdcdb032740 1 --2- 172.21.15.97:0/3821767842 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5579627b4c20 0x5579627b70e0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:01.786 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.815+0000 7fdcdb032740 1 --2- 172.21.15.97:0/3821767842 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5579627a44e0 0x5579627b02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:01.786 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.815+0000 7fdcb7400640 0 client.25873 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T11:01:01.786 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.815+0000 7fdcdb032740 1 --2- 172.21.15.97:0/3821767842 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55796279f370 0x5579627a9280 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:01.786 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.815+0000 7fdcdb032740 1 --2- 172.21.15.97:0/3821767842 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55796279e980 0x5579627a8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:01.787 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.815+0000 7fdcdb032740 1 -- 172.21.15.97:0/3821767842 >> 172.21.15.97:0/3821767842 conn(0x557962799ef0 msgr2=0x5579627990f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:01:01.787 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.815+0000 7fdcdb032740 1 -- 172.21.15.97:0/3821767842 shutdown_connections 2024-03-27T11:01:01.787 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.815+0000 7fdcdb032740 1 -- 172.21.15.97:0/3821767842 wait complete. 2024-03-27T11:01:01.787 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.816+0000 7fdcdb032740 20 client.25873 trim_cache size 0 max 16384 2024-03-27T11:01:01.812 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:01:01 smithi097 ceph-mon[31859]: pgmap v572: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 64 MiB/s wr, 1.03k op/s 2024-03-27T11:01:01.812 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:01:01 smithi097 ceph-mon[31859]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T11:01:01.820 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.847+0000 7fdcdb032740 1 Processor -- start 2024-03-27T11:01:01.820 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.847+0000 7fdcdb032740 1 -- start start 2024-03-27T11:01:01.820 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.847+0000 7fdcdb032740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55796279e980 0x5579627a3d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:01:01.820 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.847+0000 7fdcdb032740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55796279f370 0x5579627b0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:01:01.821 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.847+0000 7fdcdb032740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5579627a44e0 0x5579627a06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:01:01.821 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.847+0000 7fdcdb032740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd01af490 con 0x5579627a44e0 2024-03-27T11:01:01.821 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.847+0000 7fdcdb032740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd01b1000 con 0x55796279e980 2024-03-27T11:01:01.821 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.847+0000 7fdcdb032740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd01afb60 con 0x55796279f370 2024-03-27T11:01:01.821 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.848+0000 7fdcd5800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55796279e980 0x5579627a3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:01:01.821 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.848+0000 7fdcd4e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55796279f370 0x5579627b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:01:01.821 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.848+0000 7fdcd5800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55796279e980 0x5579627a3d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:43592/0 (socket says 172.21.15.97:43592) 2024-03-27T11:01:01.821 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.848+0000 7fdcd5800640 1 -- 172.21.15.97:0/686742904 learned_addr learned my addr 172.21.15.97:0/686742904 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:01:01.821 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.848+0000 7fdcd6200640 1 --2- 172.21.15.97:0/686742904 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5579627a44e0 0x5579627a06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:01:01.821 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.848+0000 7fdcd5800640 1 -- 172.21.15.97:0/686742904 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55796279f370 msgr2=0x5579627b0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:01:01.822 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.848+0000 7fdcd5800640 1 --2- 172.21.15.97:0/686742904 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55796279f370 0x5579627b0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:01.822 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.848+0000 7fdcd5800640 1 -- 172.21.15.97:0/686742904 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5579627a44e0 msgr2=0x5579627a06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:01:01.822 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.848+0000 7fdcd5800640 1 --2- 172.21.15.97:0/686742904 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5579627a44e0 0x5579627a06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:01.822 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.848+0000 7fdcd5800640 1 -- 172.21.15.97:0/686742904 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579624e4620 con 0x55796279e980 2024-03-27T11:01:01.822 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.848+0000 7fdcd6200640 1 --2- 172.21.15.97:0/686742904 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5579627a44e0 0x5579627a06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T11:01:01.822 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.848+0000 7fdcd5800640 1 --2- 172.21.15.97:0/686742904 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55796279e980 0x5579627a3d60 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fdcc40114f0 tx=0x7fdcc4005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5bc9e67ed98bfda server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:01:01.822 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.849+0000 7fdcc2000640 1 -- 172.21.15.97:0/686742904 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc401e040 con 0x55796279e980 2024-03-27T11:01:01.822 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.849+0000 7fdcc2000640 1 -- 172.21.15.97:0/686742904 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdcc402b6f0 con 0x55796279e980 2024-03-27T11:01:01.822 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.849+0000 7fdcc2000640 1 -- 172.21.15.97:0/686742904 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc400f5c0 con 0x55796279e980 2024-03-27T11:01:01.824 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.850+0000 7fdcdb032740 1 -- 172.21.15.97:0/686742904 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55796279e980 msgr2=0x5579627a3d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:01:01.824 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.850+0000 7fdcdb032740 1 --2- 172.21.15.97:0/686742904 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55796279e980 0x5579627a3d60 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fdcc40114f0 tx=0x7fdcc4005850 comp rx=0 tx=0).stop 2024-03-27T11:01:01.824 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.850+0000 7fdcdb032740 1 -- 172.21.15.97:0/686742904 shutdown_connections 2024-03-27T11:01:01.824 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.850+0000 7fdcdb032740 1 --2- 172.21.15.97:0/686742904 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5579627a44e0 0x5579627a06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:01.824 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.850+0000 7fdcdb032740 1 --2- 172.21.15.97:0/686742904 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55796279f370 0x5579627b0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:01.824 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.850+0000 7fdcdb032740 1 --2- 172.21.15.97:0/686742904 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55796279e980 0x5579627a3d60 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:01.824 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.850+0000 7fdcdb032740 1 -- 172.21.15.97:0/686742904 >> 172.21.15.97:0/686742904 conn(0x557962799ef0 msgr2=0x5579627a8d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:01:01.824 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.850+0000 7fdcdb032740 1 -- 172.21.15.97:0/686742904 shutdown_connections 2024-03-27T11:01:01.825 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.850+0000 7fdcdb032740 1 -- 172.21.15.97:0/686742904 wait complete. 2024-03-27T11:01:01.825 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.851+0000 7fdcdb032740 1 -- start start 2024-03-27T11:01:01.825 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.851+0000 7fdcdb032740 1 Processor -- start 2024-03-27T11:01:01.825 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.851+0000 7fdcdb032740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55796279e980 0x5579627ab300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:01:01.825 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.851+0000 7fdcdb032740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55796279f370 0x5579627ab840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:01:01.825 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.851+0000 7fdcdb032740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5579627a44e0 0x55796282a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:01:01.825 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.851+0000 7fdcdb032740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd01b0900 con 0x55796279f370 2024-03-27T11:01:01.825 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.851+0000 7fdcdb032740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd01b0230 con 0x55796279e980 2024-03-27T11:01:01.825 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.851+0000 7fdcdb032740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd01ad470 con 0x5579627a44e0 2024-03-27T11:01:01.826 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.852+0000 7fdcdb032740 10 client.0 _ll_register_callbacks cb 0x7fdcbc0125b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T11:01:01.827 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.852+0000 7fdcd4e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55796279f370 0x5579627ab840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:01:01.827 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.852+0000 7fdcd5800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55796279e980 0x5579627ab300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:01:01.827 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.852+0000 7fdcd5800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55796279e980 0x5579627ab300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:43608/0 (socket says 172.21.15.97:43608) 2024-03-27T11:01:01.827 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.852+0000 7fdcd5800640 1 -- 172.21.15.97:0/1044680905 learned_addr learned my addr 172.21.15.97:0/1044680905 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:01:01.827 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.852+0000 7fdcd6200640 1 --2- 172.21.15.97:0/1044680905 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5579627a44e0 0x55796282a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:01:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.853+0000 7fdcd4e00640 1 -- 172.21.15.97:0/1044680905 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5579627a44e0 msgr2=0x55796282a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:01:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.853+0000 7fdcd4e00640 1 --2- 172.21.15.97:0/1044680905 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5579627a44e0 0x55796282a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.853+0000 7fdcd4e00640 1 -- 172.21.15.97:0/1044680905 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55796279e980 msgr2=0x5579627ab300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:01:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.853+0000 7fdcd4e00640 1 --2- 172.21.15.97:0/1044680905 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55796279e980 0x5579627ab300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.853+0000 7fdcd4e00640 1 -- 172.21.15.97:0/1044680905 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55796282a5e0 con 0x55796279f370 2024-03-27T11:01:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.853+0000 7fdcd4e00640 1 --2- 172.21.15.97:0/1044680905 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55796279f370 0x5579627ab840 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc011820 tx=0x7fdcbc020d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46237b9b6e839bfe server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:01:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.853+0000 7fdcb7400640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T11:01:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.854+0000 7fdcb7400640 1 -- 172.21.15.97:0/1044680905 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc020630 con 0x55796279f370 2024-03-27T11:01:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.854+0000 7fdcdb032740 1 -- 172.21.15.97:0/1044680905 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdcc402b6f0 con 0x55796279f370 2024-03-27T11:01:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.854+0000 7fdcb7400640 1 -- 172.21.15.97:0/1044680905 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fdcbc0207d0 con 0x55796279f370 2024-03-27T11:01:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.855+0000 7fdcb7400640 1 -- 172.21.15.97:0/1044680905 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fdcbc01f790 con 0x55796279f370 2024-03-27T11:01:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.855+0000 7fdcdb032740 10 client.16500 fetch_fsmap learned FSMap version 33 2024-03-27T11:01:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.855+0000 7fdcdb032740 1 -- 172.21.15.97:0/1044680905 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55796282b940 con 0x55796279f370 2024-03-27T11:01:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.855+0000 7fdcb7400640 1 -- 172.21.15.97:0/1044680905 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdcc402b6f0 con 0x55796279f370 2024-03-27T11:01:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.855+0000 7fdcb7400640 1 -- 172.21.15.97:0/1044680905 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 33) ==== 8358+0+0 (secure 0 0 0) 0x7fdcbc029a90 con 0x55796279f370 2024-03-27T11:01:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.855+0000 7fdcdb032740 10 client.16500 fetch_fsmap finished waiting for FSMap version 33 2024-03-27T11:01:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.855+0000 7fdcdb032740 10 client.16500 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T11:01:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.855+0000 7fdcdb032740 20 client.16500 populate_metadata read hostname 'smithi097' 2024-03-27T11:01:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.855+0000 7fdcdb032740 1 --2- 172.21.15.97:0/1044680905 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5579627b4c20 0x55796282e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:01:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.856+0000 7fdcdb032740 4 client.16500 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-27T11:01:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.856+0000 7fdcdb032740 1 -- 172.21.15.97:0/1044680905 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55796270b5a0 con 0x5579627b4c20 2024-03-27T11:01:01.830 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.856+0000 7fdcd5800640 1 --2- 172.21.15.97:0/1044680905 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5579627b4c20 0x55796282e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:01:01.830 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.857+0000 7fdcb7400640 10 client.16500 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T11:01:01.830 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.857+0000 7fdcd5800640 1 --2- 172.21.15.97:0/1044680905 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5579627b4c20 0x55796282e360 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd2394b26163224c server_cookie=49e6c1d971e7a7c6 in_seq=0 out_seq=0 2024-03-27T11:01:01.834 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.863+0000 7fdcb7400640 1 -- 172.21.15.97:0/1044680905 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55796270b5a0 con 0x5579627b4c20 2024-03-27T11:01:01.834 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.863+0000 7fdcb7400640 10 client.16500 handle_command_reply: tid=0 2024-03-27T11:01:01.834 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.863+0000 7fdcdb032740 1 client.16500 shutdown 2024-03-27T11:01:01.836 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.864+0000 7fdcdb032740 1 -- 172.21.15.97:0/1044680905 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55796279f370 msgr2=0x5579627ab840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:01:01.836 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.864+0000 7fdcdb032740 1 --2- 172.21.15.97:0/1044680905 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55796279f370 0x5579627ab840 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc011820 tx=0x7fdcbc020d10 comp rx=0 tx=0).stop 2024-03-27T11:01:01.836 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.864+0000 7fdcdb032740 1 -- 172.21.15.97:0/1044680905 shutdown_connections 2024-03-27T11:01:01.836 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.864+0000 7fdcdb032740 1 --2- 172.21.15.97:0/1044680905 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5579627b4c20 0x55796282e360 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:01.836 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.864+0000 7fdcdb032740 1 --2- 172.21.15.97:0/1044680905 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5579627a44e0 0x55796282a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:01.836 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.864+0000 7fdcdb032740 1 --2- 172.21.15.97:0/1044680905 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55796279f370 0x5579627ab840 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:01.836 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.864+0000 7fdcdb032740 1 --2- 172.21.15.97:0/1044680905 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55796279e980 0x5579627ab300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:01.836 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.864+0000 7fdcdb032740 1 -- 172.21.15.97:0/1044680905 >> 172.21.15.97:0/1044680905 conn(0x557962799ef0 msgr2=0x5579627a06f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:01:01.836 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.864+0000 7fdcb7400640 0 client.16500 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T11:01:01.836 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.864+0000 7fdcdb032740 1 -- 172.21.15.97:0/1044680905 shutdown_connections 2024-03-27T11:01:01.837 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.864+0000 7fdcdb032740 1 -- 172.21.15.97:0/1044680905 wait complete. 2024-03-27T11:01:01.837 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.865+0000 7fdcdb032740 20 client.16500 trim_cache size 0 max 16384 2024-03-27T11:01:01.837 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-27T11:01:01.837 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-27T11:01:01.837 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-27T11:01:01.837 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-27T11:01:01.838 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.866+0000 7fdcd8200640 1 -- 172.21.15.97:0/629987309 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fdcb80a37b0 msgr2=0x7fdcb80a5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:01:01.838 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.866+0000 7fdcd8200640 1 --2- 172.21.15.97:0/629987309 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fdcb80a37b0 0x7fdcb80a5c70 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc0095f0 tx=0x7fdcbc005f90 comp rx=0 tx=0).stop 2024-03-27T11:01:01.838 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.866+0000 7fdcd8200640 1 -- 172.21.15.97:0/629987309 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcd0109d60 msgr2=0x7fdcd01a5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:01:01.838 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.866+0000 7fdcd8200640 1 --2- 172.21.15.97:0/629987309 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcd0109d60 0x7fdcd01a5860 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fdcc8032380 tx=0x7fdcc8032820 comp rx=0 tx=0).stop 2024-03-27T11:01:01.838 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.866+0000 7fdcd8200640 1 -- 172.21.15.97:0/629987309 shutdown_connections 2024-03-27T11:01:01.838 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.866+0000 7fdcd8200640 1 --2- 172.21.15.97:0/629987309 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcd0109d60 0x7fdcd01a5860 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:01.838 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.866+0000 7fdcd8200640 1 --2- 172.21.15.97:0/629987309 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdcd01093a0 0x7fdcd019e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:01.838 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.866+0000 7fdcd8200640 1 --2- 172.21.15.97:0/629987309 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fdcb80a37b0 0x7fdcb80a5c70 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:01.838 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.866+0000 7fdcd8200640 1 --2- 172.21.15.97:0/629987309 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdcd01081b0 0x7fdcd019e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:01.839 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.866+0000 7fdcd8200640 1 -- 172.21.15.97:0/629987309 >> 172.21.15.97:0/629987309 conn(0x7fdcd0077160 msgr2=0x7fdcd010c6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:01:01.839 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.866+0000 7fdcd8200640 1 -- 172.21.15.97:0/629987309 shutdown_connections 2024-03-27T11:01:01.839 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:01.866+0000 7fdcd8200640 1 -- 172.21.15.97:0/629987309 wait complete. 2024-03-27T11:01:02.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:01:02 smithi097 ceph-mon[31859]: pgmap v573: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 58 MiB/s wr, 933 op/s 2024-03-27T11:01:02.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:01:02 smithi177 ceph-mon[35294]: pgmap v573: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 58 MiB/s wr, 933 op/s 2024-03-27T11:01:02.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:01:02 smithi114 ceph-mon[35338]: pgmap v573: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 58 MiB/s wr, 933 op/s 2024-03-27T11:01:05.515 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:01:05 smithi114 ceph-mon[35338]: pgmap v574: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 58 MiB/s wr, 939 op/s 2024-03-27T11:01:05.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:01:05 smithi097 ceph-mon[31859]: pgmap v574: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 58 MiB/s wr, 939 op/s 2024-03-27T11:01:05.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:01:05 smithi177 ceph-mon[35294]: pgmap v574: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 58 MiB/s wr, 939 op/s 2024-03-27T11:01:07.515 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:01:07 smithi114 ceph-mon[35338]: pgmap v575: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 56 MiB/s wr, 905 op/s 2024-03-27T11:01:07.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:01:07 smithi097 ceph-mon[31859]: pgmap v575: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 56 MiB/s wr, 905 op/s 2024-03-27T11:01:07.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:01:07 smithi177 ceph-mon[35294]: pgmap v575: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 56 MiB/s wr, 905 op/s 2024-03-27T11:01:09.516 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:01:09 smithi114 ceph-mon[35338]: pgmap v576: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 59 MiB/s wr, 949 op/s 2024-03-27T11:01:09.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:01:09 smithi097 ceph-mon[31859]: pgmap v576: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 59 MiB/s wr, 949 op/s 2024-03-27T11:01:09.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:01:09 smithi177 ceph-mon[35294]: pgmap v576: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 59 MiB/s wr, 949 op/s 2024-03-27T11:01:11.515 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:01:11 smithi114 ceph-mon[35338]: pgmap v577: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 62 MiB/s wr, 1.01k op/s 2024-03-27T11:01:11.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:01:11 smithi097 ceph-mon[31859]: pgmap v577: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 62 MiB/s wr, 1.01k op/s 2024-03-27T11:01:11.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:01:11 smithi177 ceph-mon[35294]: pgmap v577: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 62 MiB/s wr, 1.01k op/s 2024-03-27T11:01:13.515 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:01:13 smithi114 ceph-mon[35338]: pgmap v578: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 55 MiB/s wr, 892 op/s 2024-03-27T11:01:13.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:01:13 smithi097 ceph-mon[31859]: pgmap v578: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 55 MiB/s wr, 892 op/s 2024-03-27T11:01:13.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:01:13 smithi177 ceph-mon[35294]: pgmap v578: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 55 MiB/s wr, 892 op/s 2024-03-27T11:01:15.039 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.069+0000 7f37b2800640 1 -- 172.21.15.97:0/2197029112 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f37ac103db0 msgr2=0x7f37ac1041d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:01:15.039 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.069+0000 7f37b2800640 1 --2- 172.21.15.97:0/2197029112 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f37ac103db0 0x7f37ac1041d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3794009900 tx=0x7f379402bc30 comp rx=0 tx=0).stop 2024-03-27T11:01:15.040 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.070+0000 7f37b2800640 1 -- 172.21.15.97:0/2197029112 shutdown_connections 2024-03-27T11:01:15.040 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.070+0000 7f37b2800640 1 --2- 172.21.15.97:0/2197029112 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37ac105960 0x7f37ac10c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:15.040 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.070+0000 7f37b2800640 1 --2- 172.21.15.97:0/2197029112 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37ac104fa0 0x7f37ac105420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:15.041 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.070+0000 7f37b2800640 1 --2- 172.21.15.97:0/2197029112 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f37ac103db0 0x7f37ac1041d0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:15.041 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.070+0000 7f37b2800640 1 -- 172.21.15.97:0/2197029112 >> 172.21.15.97:0/2197029112 conn(0x7f37ac0ff340 msgr2=0x7f37ac101760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:01:15.041 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.070+0000 7f37b2800640 1 -- 172.21.15.97:0/2197029112 shutdown_connections 2024-03-27T11:01:15.041 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.070+0000 7f37b2800640 1 -- 172.21.15.97:0/2197029112 wait complete. 2024-03-27T11:01:15.041 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.071+0000 7f37b2800640 1 Processor -- start 2024-03-27T11:01:15.042 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.071+0000 7f37b2800640 1 -- start start 2024-03-27T11:01:15.042 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.071+0000 7f37b2800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f37ac103db0 0x7f37ac199f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:01:15.042 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.071+0000 7f37b2800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37ac104fa0 0x7f37ac19a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:01:15.042 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.071+0000 7f37b2800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37ac105960 0x7f37ac1a14c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:01:15.042 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.071+0000 7f37b2800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f37ac10f900 con 0x7f37ac105960 2024-03-27T11:01:15.042 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.071+0000 7f37b2800640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f37ac10f780 con 0x7f37ac104fa0 2024-03-27T11:01:15.042 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.071+0000 7f37b2800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f37ac10fa80 con 0x7f37ac103db0 2024-03-27T11:01:15.043 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.072+0000 7f37a2a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37ac104fa0 0x7f37ac19a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:01:15.044 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.072+0000 7f37a2a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37ac104fa0 0x7f37ac19a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:58566/0 (socket says 172.21.15.97:58566) 2024-03-27T11:01:15.044 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.072+0000 7f37a2a00640 1 -- 172.21.15.97:0/3148540090 learned_addr learned my addr 172.21.15.97:0/3148540090 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:01:15.044 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.072+0000 7f37ab400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f37ac103db0 0x7f37ac199f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:01:15.044 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.072+0000 7f37abe00640 1 --2- 172.21.15.97:0/3148540090 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37ac105960 0x7f37ac1a14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:01:15.044 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.072+0000 7f37a2a00640 1 -- 172.21.15.97:0/3148540090 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f37ac103db0 msgr2=0x7f37ac199f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:01:15.044 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.072+0000 7f37a2a00640 1 --2- 172.21.15.97:0/3148540090 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f37ac103db0 0x7f37ac199f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:15.044 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.072+0000 7f37a2a00640 1 -- 172.21.15.97:0/3148540090 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37ac105960 msgr2=0x7f37ac1a14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:01:15.044 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.072+0000 7f37a2a00640 1 --2- 172.21.15.97:0/3148540090 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37ac105960 0x7f37ac1a14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:15.044 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.072+0000 7f37a2a00640 1 -- 172.21.15.97:0/3148540090 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37ac1a1b30 con 0x7f37ac104fa0 2024-03-27T11:01:15.044 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.073+0000 7f37a2a00640 1 --2- 172.21.15.97:0/3148540090 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37ac104fa0 0x7f37ac19a450 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f379c00e9f0 tx=0x7f379c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:01:15.044 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.073+0000 7f37a8c00640 1 -- 172.21.15.97:0/3148540090 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f379c00cd80 con 0x7f37ac104fa0 2024-03-27T11:01:15.044 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.073+0000 7f37b2800640 1 -- 172.21.15.97:0/3148540090 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37ac1a1e20 con 0x7f37ac104fa0 2024-03-27T11:01:15.044 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.073+0000 7f37a8c00640 1 -- 172.21.15.97:0/3148540090 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f379c0108e0 con 0x7f37ac104fa0 2024-03-27T11:01:15.045 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.073+0000 7f37b2800640 1 -- 172.21.15.97:0/3148540090 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37ac1a2330 con 0x7f37ac104fa0 2024-03-27T11:01:15.045 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.073+0000 7f37a8c00640 1 -- 172.21.15.97:0/3148540090 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f379c018920 con 0x7f37ac104fa0 2024-03-27T11:01:15.048 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.077+0000 7f37b5434740 1 Processor -- start 2024-03-27T11:01:15.048 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.078+0000 7f37b5434740 1 -- start start 2024-03-27T11:01:15.049 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.078+0000 7f37a8c00640 1 -- 172.21.15.97:0/3148540090 <== mon.2 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f379c010430 con 0x7f37ac104fa0 2024-03-27T11:01:15.049 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.078+0000 7f37b5434740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56006ff91190 0x56006ff915b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:01:15.049 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.078+0000 7f37b5434740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56006ff91b80 0x56006ff96680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:01:15.049 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.079+0000 7f37b5434740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56006ff96cf0 0x56006ff99180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:01:15.049 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.079+0000 7f37b5434740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56006feedd10 con 0x56006ff96cf0 2024-03-27T11:01:15.049 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.079+0000 7f37b5434740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x56006fe96260 con 0x56006ff91b80 2024-03-27T11:01:15.049 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.079+0000 7f37b5434740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x56006fee9cc0 con 0x56006ff91190 2024-03-27T11:01:15.050 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.079+0000 7f37abe00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56006ff91190 0x56006ff915b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:01:15.050 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.079+0000 7f37abe00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56006ff91190 0x56006ff915b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.97:43870/0 (socket says 172.21.15.97:43870) 2024-03-27T11:01:15.050 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.079+0000 7f37abe00640 1 -- 172.21.15.97:0/344081368 learned_addr learned my addr 172.21.15.97:0/344081368 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:01:15.050 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.079+0000 7f37a2a00640 1 --2- 172.21.15.97:0/344081368 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56006ff96cf0 0x56006ff99180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:01:15.050 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.079+0000 7f37ab400640 1 --2- 172.21.15.97:0/344081368 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56006ff91b80 0x56006ff96680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:01:15.050 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.079+0000 7f37abe00640 1 -- 172.21.15.97:0/344081368 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56006ff91b80 msgr2=0x56006ff96680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:01:15.050 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.079+0000 7f37abe00640 1 --2- 172.21.15.97:0/344081368 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56006ff91b80 0x56006ff96680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:15.050 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.079+0000 7f37abe00640 1 -- 172.21.15.97:0/344081368 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56006ff96cf0 msgr2=0x56006ff99180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:01:15.050 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.080+0000 7f37abe00640 1 --2- 172.21.15.97:0/344081368 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56006ff96cf0 0x56006ff99180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:15.051 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.080+0000 7f37abe00640 1 -- 172.21.15.97:0/344081368 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56006fe8daa0 con 0x56006ff91190 2024-03-27T11:01:15.051 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.080+0000 7f37a2a00640 1 --2- 172.21.15.97:0/344081368 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56006ff96cf0 0x56006ff99180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T11:01:15.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.080+0000 7f37abe00640 1 --2- 172.21.15.97:0/344081368 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56006ff91190 0x56006ff915b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f37a4009fc0 tx=0x7f37a4010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4de2e74c9ff4581a server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:01:15.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.080+0000 7f37a0c00640 1 -- 172.21.15.97:0/344081368 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a4003b70 con 0x56006ff91190 2024-03-27T11:01:15.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.081+0000 7f37a0c00640 1 -- 172.21.15.97:0/344081368 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f37a4003d10 con 0x56006ff91190 2024-03-27T11:01:15.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.081+0000 7f37a0c00640 1 -- 172.21.15.97:0/344081368 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a4011a40 con 0x56006ff91190 2024-03-27T11:01:15.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.081+0000 7f37a8c00640 1 --2- 172.21.15.97:0/3148540090 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f37900a3730 0x7f37900a5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:01:15.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.081+0000 7f37a8c00640 1 -- 172.21.15.97:0/3148540090 <== mon.2 v2:172.21.15.114:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f379c0cff50 con 0x7f37ac104fa0 2024-03-27T11:01:15.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.082+0000 7f37ab400640 1 --2- 172.21.15.97:0/3148540090 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f37900a3730 0x7f37900a5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:01:15.053 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.082+0000 7f37b5434740 1 -- 172.21.15.97:0/344081368 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56006ff91190 msgr2=0x56006ff915b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:01:15.053 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.082+0000 7f37b5434740 1 --2- 172.21.15.97:0/344081368 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56006ff91190 0x56006ff915b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f37a4009fc0 tx=0x7f37a4010860 comp rx=0 tx=0).stop 2024-03-27T11:01:15.053 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.083+0000 7f37b5434740 1 -- 172.21.15.97:0/344081368 shutdown_connections 2024-03-27T11:01:15.053 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.083+0000 7f37b5434740 1 --2- 172.21.15.97:0/344081368 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56006ff96cf0 0x56006ff99180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:15.053 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.083+0000 7f37b5434740 1 --2- 172.21.15.97:0/344081368 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56006ff91b80 0x56006ff96680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:15.053 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.083+0000 7f37b5434740 1 --2- 172.21.15.97:0/344081368 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56006ff91190 0x56006ff915b0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:15.053 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.083+0000 7f37b5434740 1 -- 172.21.15.97:0/344081368 >> 172.21.15.97:0/344081368 conn(0x56006ff83ef0 msgr2=0x56006ff842f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:01:15.054 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.083+0000 7f37ab400640 1 --2- 172.21.15.97:0/3148540090 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f37900a3730 0x7f37900a5bf0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f379402c240 tx=0x7f379402c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:01:15.054 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.083+0000 7f37b5434740 1 -- 172.21.15.97:0/344081368 shutdown_connections 2024-03-27T11:01:15.054 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.083+0000 7f37b5434740 1 -- 172.21.15.97:0/344081368 wait complete. 2024-03-27T11:01:15.054 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.084+0000 7f37b5434740 1 -- start start 2024-03-27T11:01:15.055 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.084+0000 7f37b5434740 1 Processor -- start 2024-03-27T11:01:15.055 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.085+0000 7f37b5434740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56006ff91190 0x56006ff9b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:01:15.055 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.085+0000 7f37b5434740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56006ff91b80 0x56006ff9ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:01:15.055 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.085+0000 7f37b5434740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56006ff96cf0 0x56006ffa2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:01:15.055 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.085+0000 7f37b5434740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56006ff99ca0 con 0x56006ff96cf0 2024-03-27T11:01:15.055 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.085+0000 7f37b5434740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x56006fea9630 con 0x56006ff91b80 2024-03-27T11:01:15.056 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.085+0000 7f37b5434740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x56006ff99e20 con 0x56006ff91190 2024-03-27T11:01:15.056 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.085+0000 7f37b5434740 10 client.0 _ll_register_callbacks cb 0x56006feaec50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T11:01:15.057 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.086+0000 7f37abe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56006ff96cf0 0x56006ffa2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:01:15.057 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.086+0000 7f37abe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56006ff96cf0 0x56006ffa2a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41470/0 (socket says 172.21.15.97:41470) 2024-03-27T11:01:15.057 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.086+0000 7f37abe00640 1 -- 172.21.15.97:0/800785004 learned_addr learned my addr 172.21.15.97:0/800785004 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:01:15.057 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.086+0000 7f37a2a00640 1 --2- 172.21.15.97:0/800785004 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56006ff91b80 0x56006ff9ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:01:15.057 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.086+0000 7f37ab400640 1 --2- 172.21.15.97:0/800785004 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56006ff91190 0x56006ff9b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:01:15.057 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.086+0000 7f37abe00640 1 -- 172.21.15.97:0/800785004 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56006ff91190 msgr2=0x56006ff9b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:01:15.057 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.086+0000 7f37abe00640 1 --2- 172.21.15.97:0/800785004 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56006ff91190 0x56006ff9b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:15.057 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.086+0000 7f37abe00640 1 -- 172.21.15.97:0/800785004 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56006ff91b80 msgr2=0x56006ff9ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:01:15.057 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.086+0000 7f37abe00640 1 --2- 172.21.15.97:0/800785004 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56006ff91b80 0x56006ff9ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:15.057 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.086+0000 7f37abe00640 1 -- 172.21.15.97:0/800785004 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56006feb55f0 con 0x56006ff96cf0 2024-03-27T11:01:15.057 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.086+0000 7f37a2a00640 1 --2- 172.21.15.97:0/800785004 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56006ff91b80 0x56006ff9ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T11:01:15.057 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.087+0000 7f37abe00640 1 --2- 172.21.15.97:0/800785004 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56006ff96cf0 0x56006ffa2a50 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f37a4004cd0 tx=0x7f37a4004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25225caf515d00f9 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:01:15.058 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.087+0000 7f3799600640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T11:01:15.058 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.087+0000 7f3799600640 1 -- 172.21.15.97:0/800785004 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a4029060 con 0x56006ff96cf0 2024-03-27T11:01:15.058 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.087+0000 7f3799600640 1 -- 172.21.15.97:0/800785004 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f37a4011b50 con 0x56006ff96cf0 2024-03-27T11:01:15.059 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.087+0000 7f37b5434740 1 -- 172.21.15.97:0/800785004 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37a4003d10 con 0x56006ff96cf0 2024-03-27T11:01:15.059 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.088+0000 7f3799600640 1 -- 172.21.15.97:0/800785004 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f37a4011d50 con 0x56006ff96cf0 2024-03-27T11:01:15.059 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.088+0000 7f37b5434740 10 client.16509 fetch_fsmap learned FSMap version 33 2024-03-27T11:01:15.059 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.088+0000 7f37b5434740 1 -- 172.21.15.97:0/800785004 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56006fc02210 con 0x56006ff96cf0 2024-03-27T11:01:15.059 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.089+0000 7f3799600640 1 -- 172.21.15.97:0/800785004 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f37a4003d10 con 0x56006ff96cf0 2024-03-27T11:01:15.059 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.089+0000 7f3799600640 1 -- 172.21.15.97:0/800785004 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 33) ==== 8358+0+0 (secure 0 0 0) 0x7f37a4005380 con 0x56006ff96cf0 2024-03-27T11:01:15.060 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.089+0000 7f37b5434740 10 client.16509 fetch_fsmap finished waiting for FSMap version 33 2024-03-27T11:01:15.060 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.089+0000 7f37b5434740 10 client.16509 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T11:01:15.060 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.089+0000 7f37b5434740 20 client.16509 populate_metadata read hostname 'smithi097' 2024-03-27T11:01:15.060 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.089+0000 7f37b5434740 1 --2- 172.21.15.97:0/800785004 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x56006ffa6f10 0x56006ffa9300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:01:15.060 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.089+0000 7f37b5434740 4 client.16509 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T11:01:15.060 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.089+0000 7f37b5434740 1 -- 172.21.15.97:0/800785004 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56006fefb250 con 0x56006ffa6f10 2024-03-27T11:01:15.060 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.089+0000 7f37ab400640 1 --2- 172.21.15.97:0/800785004 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x56006ffa6f10 0x56006ffa9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:01:15.060 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.091+0000 7f37ab400640 1 --2- 172.21.15.97:0/800785004 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x56006ffa6f10 0x56006ffa9300 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4752a48a6817bf4b server_cookie=ca996a2291abb5c7 in_seq=0 out_seq=0 2024-03-27T11:01:15.061 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.091+0000 7f3799600640 10 client.16509 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T11:01:15.063 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.092+0000 7f3799600640 1 -- 172.21.15.97:0/800785004 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x56006fefb250 con 0x56006ffa6f10 2024-03-27T11:01:15.063 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.093+0000 7f3799600640 10 client.16509 handle_command_reply: tid=0 2024-03-27T11:01:15.063 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.093+0000 7f37b5434740 1 client.16509 shutdown 2024-03-27T11:01:15.063 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.093+0000 7f37b5434740 1 -- 172.21.15.97:0/800785004 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56006ff96cf0 msgr2=0x56006ffa2a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:01:15.064 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.093+0000 7f37b5434740 1 --2- 172.21.15.97:0/800785004 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56006ff96cf0 0x56006ffa2a50 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f37a4004cd0 tx=0x7f37a4004d00 comp rx=0 tx=0).stop 2024-03-27T11:01:15.064 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.093+0000 7f37b5434740 1 -- 172.21.15.97:0/800785004 shutdown_connections 2024-03-27T11:01:15.064 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.093+0000 7f37b5434740 1 --2- 172.21.15.97:0/800785004 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x56006ffa6f10 0x56006ffa9300 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:15.064 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.093+0000 7f37b5434740 1 --2- 172.21.15.97:0/800785004 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56006ff96cf0 0x56006ffa2a50 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:15.064 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.094+0000 7f37b5434740 1 --2- 172.21.15.97:0/800785004 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56006ff91b80 0x56006ff9ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:15.064 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.094+0000 7f37b5434740 1 --2- 172.21.15.97:0/800785004 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56006ff91190 0x56006ff9b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:15.064 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.094+0000 7f37b5434740 1 -- 172.21.15.97:0/800785004 >> 172.21.15.97:0/800785004 conn(0x56006ff83ef0 msgr2=0x56006ff830f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:01:15.064 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.094+0000 7f3799600640 0 client.16509 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T11:01:15.065 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.094+0000 7f37b5434740 1 -- 172.21.15.97:0/800785004 shutdown_connections 2024-03-27T11:01:15.065 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.094+0000 7f37b5434740 1 -- 172.21.15.97:0/800785004 wait complete. 2024-03-27T11:01:15.065 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.094+0000 7f37b5434740 20 client.16509 trim_cache size 0 max 16384 2024-03-27T11:01:15.094 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.124+0000 7f37b5434740 1 Processor -- start 2024-03-27T11:01:15.094 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.124+0000 7f37b5434740 1 -- start start 2024-03-27T11:01:15.095 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.125+0000 7f37b5434740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56006ff91190 0x56006ffa8f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:01:15.095 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.125+0000 7f37b5434740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56006ff91b80 0x56006ffa2970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:01:15.095 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.125+0000 7f37b5434740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56006ff96cf0 0x56006ff9a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:01:15.095 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.125+0000 7f37b5434740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f378800e430 con 0x56006ff91190 2024-03-27T11:01:15.095 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.125+0000 7f37b5434740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f378800ffa0 con 0x56006ff91b80 2024-03-27T11:01:15.096 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.125+0000 7f37b5434740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f378800eb00 con 0x56006ff96cf0 2024-03-27T11:01:15.096 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.126+0000 7f37a2a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56006ff91b80 0x56006ffa2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:01:15.096 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.126+0000 7f37a2a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56006ff91b80 0x56006ffa2970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:58604/0 (socket says 172.21.15.97:58604) 2024-03-27T11:01:15.096 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.126+0000 7f37a2a00640 1 -- 172.21.15.97:0/3984924965 learned_addr learned my addr 172.21.15.97:0/3984924965 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:01:15.096 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.126+0000 7f37abe00640 1 --2- 172.21.15.97:0/3984924965 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56006ff96cf0 0x56006ff9a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:01:15.097 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.126+0000 7f37ab400640 1 --2- 172.21.15.97:0/3984924965 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56006ff91190 0x56006ffa8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:01:15.097 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.127+0000 7f37a2a00640 1 -- 172.21.15.97:0/3984924965 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56006ff96cf0 msgr2=0x56006ff9a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:01:15.097 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.127+0000 7f37a2a00640 1 --2- 172.21.15.97:0/3984924965 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56006ff96cf0 0x56006ff9a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:15.097 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.127+0000 7f37a2a00640 1 -- 172.21.15.97:0/3984924965 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56006ff91190 msgr2=0x56006ffa8f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:01:15.098 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.127+0000 7f37a2a00640 1 --2- 172.21.15.97:0/3984924965 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56006ff91190 0x56006ffa8f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:15.098 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.127+0000 7f37a2a00640 1 -- 172.21.15.97:0/3984924965 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56006fcce620 con 0x56006ff91b80 2024-03-27T11:01:15.098 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.127+0000 7f37abe00640 1 --2- 172.21.15.97:0/3984924965 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56006ff96cf0 0x56006ff9a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T11:01:15.098 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.127+0000 7f37ab400640 1 --2- 172.21.15.97:0/3984924965 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56006ff91190 0x56006ffa8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T11:01:15.098 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.128+0000 7f37a2a00640 1 --2- 172.21.15.97:0/3984924965 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56006ff91b80 0x56006ffa2970 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f379c098e00 tx=0x7f379c010720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a23be3258a8e8cf1 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:01:15.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.128+0000 7f379be00640 1 -- 172.21.15.97:0/3984924965 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f379c09c430 con 0x56006ff91b80 2024-03-27T11:01:15.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.128+0000 7f379be00640 1 -- 172.21.15.97:0/3984924965 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f379c0a07e0 con 0x56006ff91b80 2024-03-27T11:01:15.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.128+0000 7f379be00640 1 -- 172.21.15.97:0/3984924965 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f379c098450 con 0x56006ff91b80 2024-03-27T11:01:15.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.129+0000 7f37b5434740 1 -- 172.21.15.97:0/3984924965 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56006ff91b80 msgr2=0x56006ffa2970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:01:15.100 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.129+0000 7f37b5434740 1 --2- 172.21.15.97:0/3984924965 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56006ff91b80 0x56006ffa2970 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f379c098e00 tx=0x7f379c010720 comp rx=0 tx=0).stop 2024-03-27T11:01:15.100 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.130+0000 7f37b5434740 1 -- 172.21.15.97:0/3984924965 shutdown_connections 2024-03-27T11:01:15.100 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.130+0000 7f37b5434740 1 --2- 172.21.15.97:0/3984924965 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56006ff96cf0 0x56006ff9a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:15.100 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.130+0000 7f37b5434740 1 --2- 172.21.15.97:0/3984924965 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56006ff91b80 0x56006ffa2970 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:15.100 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.130+0000 7f37b5434740 1 --2- 172.21.15.97:0/3984924965 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56006ff91190 0x56006ffa8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:15.100 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.130+0000 7f37b5434740 1 -- 172.21.15.97:0/3984924965 >> 172.21.15.97:0/3984924965 conn(0x56006ff83ef0 msgr2=0x56006ff9b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:01:15.101 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.130+0000 7f37b5434740 1 -- 172.21.15.97:0/3984924965 shutdown_connections 2024-03-27T11:01:15.101 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.130+0000 7f37b5434740 1 -- 172.21.15.97:0/3984924965 wait complete. 2024-03-27T11:01:15.101 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.131+0000 7f37b5434740 1 -- start start 2024-03-27T11:01:15.101 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.131+0000 7f37b5434740 1 Processor -- start 2024-03-27T11:01:15.102 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.132+0000 7f37b5434740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56006ff91190 0x56006ff9d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:01:15.103 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.132+0000 7f37b5434740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56006ff91b80 0x56006ff9ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:01:15.103 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.132+0000 7f37b5434740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56006ff96cf0 0x56007001c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:01:15.104 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.133+0000 7f37b5434740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f378800f8a0 con 0x56006ff91b80 2024-03-27T11:01:15.104 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.133+0000 7f37b5434740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f378800f1d0 con 0x56006ff96cf0 2024-03-27T11:01:15.104 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.133+0000 7f37b5434740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f378800c410 con 0x56006ff91190 2024-03-27T11:01:15.104 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.133+0000 7f37b5434740 10 client.0 _ll_register_callbacks cb 0x7f37a4009e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T11:01:15.104 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.133+0000 7f37abe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56006ff96cf0 0x56007001c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:01:15.104 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.133+0000 7f37abe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56006ff96cf0 0x56007001c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:58608/0 (socket says 172.21.15.97:58608) 2024-03-27T11:01:15.104 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.133+0000 7f37abe00640 1 -- 172.21.15.97:0/80306260 learned_addr learned my addr 172.21.15.97:0/80306260 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:01:15.104 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.133+0000 7f37ab400640 1 --2- 172.21.15.97:0/80306260 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56006ff91190 0x56006ff9d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:01:15.105 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.134+0000 7f37a2a00640 1 --2- 172.21.15.97:0/80306260 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56006ff91b80 0x56006ff9ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:01:15.105 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.134+0000 7f37abe00640 1 -- 172.21.15.97:0/80306260 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56006ff91190 msgr2=0x56006ff9d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:01:15.105 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.134+0000 7f37abe00640 1 --2- 172.21.15.97:0/80306260 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56006ff91190 0x56006ff9d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:15.105 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.134+0000 7f37abe00640 1 -- 172.21.15.97:0/80306260 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56006ff91b80 msgr2=0x56006ff9ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:01:15.106 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.134+0000 7f37abe00640 1 --2- 172.21.15.97:0/80306260 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56006ff91b80 0x56006ff9ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:15.106 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.134+0000 7f37abe00640 1 -- 172.21.15.97:0/80306260 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56007001c740 con 0x56006ff96cf0 2024-03-27T11:01:15.106 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.134+0000 7f37ab400640 1 --2- 172.21.15.97:0/80306260 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56006ff91190 0x56006ff9d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T11:01:15.106 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.135+0000 7f37abe00640 1 --2- 172.21.15.97:0/80306260 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56006ff96cf0 0x56007001c200 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f37a4011010 tx=0x7f37a4009f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6fffaf433699ad7e server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:01:15.106 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.135+0000 7f3799600640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-03-27T11:01:15.106 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.135+0000 7f3799600640 1 -- 172.21.15.97:0/80306260 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a403c060 con 0x56006ff96cf0 2024-03-27T11:01:15.107 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.136+0000 7f3799600640 1 -- 172.21.15.97:0/80306260 <== mon.2 v2:172.21.15.114:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f37a4009500 con 0x56006ff96cf0 2024-03-27T11:01:15.107 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.136+0000 7f37b5434740 1 -- 172.21.15.97:0/80306260 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f379c0a07e0 con 0x56006ff96cf0 2024-03-27T11:01:15.107 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.136+0000 7f3799600640 1 -- 172.21.15.97:0/80306260 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f37a4029050 con 0x56006ff96cf0 2024-03-27T11:01:15.107 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.137+0000 7f37b5434740 10 client.36173 fetch_fsmap learned FSMap version 33 2024-03-27T11:01:15.108 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.137+0000 7f37b5434740 1 -- 172.21.15.97:0/80306260 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56007001da60 con 0x56006ff96cf0 2024-03-27T11:01:15.108 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.137+0000 7f3799600640 1 -- 172.21.15.97:0/80306260 <== mon.2 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f379c0a07e0 con 0x56006ff96cf0 2024-03-27T11:01:15.108 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.138+0000 7f3799600640 1 -- 172.21.15.97:0/80306260 <== mon.2 v2:172.21.15.114:3300/0 5 ==== fsmap(e 33) ==== 8358+0+0 (secure 0 0 0) 0x7f37a401b750 con 0x56006ff96cf0 2024-03-27T11:01:15.108 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.138+0000 7f37b5434740 10 client.36173 fetch_fsmap finished waiting for FSMap version 33 2024-03-27T11:01:15.109 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.138+0000 7f37b5434740 10 client.36173 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T11:01:15.109 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.138+0000 7f37b5434740 20 client.36173 populate_metadata read hostname 'smithi097' 2024-03-27T11:01:15.109 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.138+0000 7f37b5434740 1 --2- 172.21.15.97:0/80306260 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x56007001e3c0 0x5600700207e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:01:15.109 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.138+0000 7f37b5434740 4 client.36173 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-27T11:01:15.109 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.138+0000 7f37b5434740 1 -- 172.21.15.97:0/80306260 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56006fef55a0 con 0x56007001e3c0 2024-03-27T11:01:15.109 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.138+0000 7f37ab400640 1 --2- 172.21.15.97:0/80306260 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x56007001e3c0 0x5600700207e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:01:15.109 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.139+0000 7f37ab400640 1 --2- 172.21.15.97:0/80306260 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x56007001e3c0 0x5600700207e0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1d9e153c76098 server_cookie=1aafce04ad3e7979 in_seq=0 out_seq=0 2024-03-27T11:01:15.109 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.139+0000 7f3799600640 10 client.36173 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T11:01:15.124 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.154+0000 7f3799600640 1 -- 172.21.15.97:0/80306260 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56006fef55a0 con 0x56007001e3c0 2024-03-27T11:01:15.124 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.154+0000 7f3799600640 10 client.36173 handle_command_reply: tid=0 2024-03-27T11:01:15.124 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.154+0000 7f37b5434740 1 client.36173 shutdown 2024-03-27T11:01:15.125 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.155+0000 7f37b5434740 1 -- 172.21.15.97:0/80306260 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56006ff96cf0 msgr2=0x56007001c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:01:15.125 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.155+0000 7f37b5434740 1 --2- 172.21.15.97:0/80306260 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56006ff96cf0 0x56007001c200 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f37a4011010 tx=0x7f37a4009f40 comp rx=0 tx=0).stop 2024-03-27T11:01:15.125 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.155+0000 7f37b5434740 1 -- 172.21.15.97:0/80306260 shutdown_connections 2024-03-27T11:01:15.125 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.155+0000 7f37b5434740 1 --2- 172.21.15.97:0/80306260 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x56007001e3c0 0x5600700207e0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:15.125 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.155+0000 7f37b5434740 1 --2- 172.21.15.97:0/80306260 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56006ff96cf0 0x56007001c200 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:15.126 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.155+0000 7f37b5434740 1 --2- 172.21.15.97:0/80306260 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56006ff91b80 0x56006ff9ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:15.126 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.155+0000 7f37b5434740 1 --2- 172.21.15.97:0/80306260 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56006ff91190 0x56006ff9d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:15.126 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.155+0000 7f3799600640 0 client.36173 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T11:01:15.126 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.155+0000 7f37b5434740 1 -- 172.21.15.97:0/80306260 >> 172.21.15.97:0/80306260 conn(0x56006ff83ef0 msgr2=0x56006ff879e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:01:15.126 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.155+0000 7f37b5434740 1 -- 172.21.15.97:0/80306260 shutdown_connections 2024-03-27T11:01:15.126 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.155+0000 7f37b5434740 1 -- 172.21.15.97:0/80306260 wait complete. 2024-03-27T11:01:15.127 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.155+0000 7f37b5434740 20 client.36173 trim_cache size 0 max 16384 2024-03-27T11:01:15.127 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-27T11:01:15.127 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-27T11:01:15.127 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-27T11:01:15.127 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-27T11:01:15.128 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.156+0000 7f37b2800640 1 -- 172.21.15.97:0/3148540090 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f37900a3730 msgr2=0x7f37900a5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:01:15.128 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.156+0000 7f37b2800640 1 --2- 172.21.15.97:0/3148540090 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f37900a3730 0x7f37900a5bf0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f379402c240 tx=0x7f379402c7e0 comp rx=0 tx=0).stop 2024-03-27T11:01:15.128 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.156+0000 7f37b2800640 1 -- 172.21.15.97:0/3148540090 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37ac104fa0 msgr2=0x7f37ac19a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:01:15.128 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.156+0000 7f37b2800640 1 --2- 172.21.15.97:0/3148540090 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37ac104fa0 0x7f37ac19a450 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f379c00e9f0 tx=0x7f379c00eeb0 comp rx=0 tx=0).stop 2024-03-27T11:01:15.128 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.156+0000 7f37b2800640 1 -- 172.21.15.97:0/3148540090 shutdown_connections 2024-03-27T11:01:15.128 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.156+0000 7f37b2800640 1 --2- 172.21.15.97:0/3148540090 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37ac105960 0x7f37ac1a14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:15.128 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.156+0000 7f37b2800640 1 --2- 172.21.15.97:0/3148540090 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f37900a3730 0x7f37900a5bf0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:15.128 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.157+0000 7f37b2800640 1 --2- 172.21.15.97:0/3148540090 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37ac104fa0 0x7f37ac19a450 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:15.129 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.157+0000 7f37b2800640 1 --2- 172.21.15.97:0/3148540090 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f37ac103db0 0x7f37ac199f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:15.129 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.157+0000 7f37b2800640 1 -- 172.21.15.97:0/3148540090 >> 172.21.15.97:0/3148540090 conn(0x7f37ac0ff340 msgr2=0x7f37ac1082a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:01:15.129 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.157+0000 7f37b2800640 1 -- 172.21.15.97:0/3148540090 shutdown_connections 2024-03-27T11:01:15.129 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:15.157+0000 7f37b2800640 1 -- 172.21.15.97:0/3148540090 wait complete. 2024-03-27T11:01:15.515 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:01:15 smithi114 ceph-mon[35338]: pgmap v579: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 55 MiB/s wr, 890 op/s 2024-03-27T11:01:15.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:01:15 smithi097 ceph-mon[31859]: pgmap v579: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 55 MiB/s wr, 890 op/s 2024-03-27T11:01:15.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:01:15 smithi177 ceph-mon[35294]: pgmap v579: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 55 MiB/s wr, 890 op/s 2024-03-27T11:01:17.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:01:17 smithi097 ceph-mon[31859]: pgmap v580: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 53 MiB/s wr, 860 op/s 2024-03-27T11:01:17.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:01:17 smithi177 ceph-mon[35294]: pgmap v580: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 53 MiB/s wr, 860 op/s 2024-03-27T11:01:17.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:01:17 smithi114 ceph-mon[35338]: pgmap v580: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 53 MiB/s wr, 860 op/s 2024-03-27T11:01:19.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:01:19 smithi097 ceph-mon[31859]: pgmap v581: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 56 MiB/s wr, 905 op/s 2024-03-27T11:01:19.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:01:19 smithi177 ceph-mon[35294]: pgmap v581: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 56 MiB/s wr, 905 op/s 2024-03-27T11:01:19.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:01:19 smithi114 ceph-mon[35338]: pgmap v581: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 56 MiB/s wr, 905 op/s 2024-03-27T11:01:21.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:01:21 smithi097 ceph-mon[31859]: pgmap v582: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 61 MiB/s wr, 982 op/s 2024-03-27T11:01:21.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:01:21 smithi097 ceph-mon[31859]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T11:01:21.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:01:21 smithi177 ceph-mon[35294]: pgmap v582: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 61 MiB/s wr, 982 op/s 2024-03-27T11:01:21.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:01:21 smithi177 ceph-mon[35294]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T11:01:21.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:01:21 smithi114 ceph-mon[35338]: pgmap v582: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 61 MiB/s wr, 982 op/s 2024-03-27T11:01:21.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:01:21 smithi114 ceph-mon[35338]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T11:01:23.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:01:23 smithi097 ceph-mon[31859]: pgmap v583: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 55 MiB/s wr, 891 op/s 2024-03-27T11:01:23.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:01:23 smithi177 ceph-mon[35294]: pgmap v583: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 55 MiB/s wr, 891 op/s 2024-03-27T11:01:23.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:01:23 smithi114 ceph-mon[35338]: pgmap v583: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 55 MiB/s wr, 891 op/s 2024-03-27T11:01:25.641 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:01:25 smithi097 ceph-mon[31859]: pgmap v584: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 57 MiB/s wr, 918 op/s 2024-03-27T11:01:25.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:01:25 smithi177 ceph-mon[35294]: pgmap v584: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 57 MiB/s wr, 918 op/s 2024-03-27T11:01:26.511 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:01:25 smithi114 ceph-mon[35338]: pgmap v584: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 57 MiB/s wr, 918 op/s 2024-03-27T11:01:27.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:01:27 smithi097 ceph-mon[31859]: pgmap v585: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 56 MiB/s wr, 901 op/s 2024-03-27T11:01:27.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:01:27 smithi177 ceph-mon[35294]: pgmap v585: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 56 MiB/s wr, 901 op/s 2024-03-27T11:01:27.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:01:27 smithi114 ceph-mon[35338]: pgmap v585: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 56 MiB/s wr, 901 op/s 2024-03-27T11:01:28.333 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.363+0000 7f3e12c00640 1 -- 172.21.15.97:0/3573412230 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3e0c102ba0 msgr2=0x7f3e0c105010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:01:28.334 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.363+0000 7f3e12c00640 1 --2- 172.21.15.97:0/3573412230 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3e0c102ba0 0x7f3e0c105010 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3df8009900 tx=0x7f3df802bc30 comp rx=0 tx=0).stop 2024-03-27T11:01:28.334 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.364+0000 7f3e12c00640 1 -- 172.21.15.97:0/3573412230 shutdown_connections 2024-03-27T11:01:28.334 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.364+0000 7f3e12c00640 1 --2- 172.21.15.97:0/3573412230 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3e0c105550 0x7f3e0c1059d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:28.335 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.364+0000 7f3e12c00640 1 --2- 172.21.15.97:0/3573412230 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3e0c102ba0 0x7f3e0c105010 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:28.335 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.364+0000 7f3e12c00640 1 --2- 172.21.15.97:0/3573412230 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e0c100230 0x7f3e0c102660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:28.335 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.364+0000 7f3e12c00640 1 -- 172.21.15.97:0/3573412230 >> 172.21.15.97:0/3573412230 conn(0x7f3e0c0fbe50 msgr2=0x7f3e0c0fe270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:01:28.335 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.364+0000 7f3e12c00640 1 -- 172.21.15.97:0/3573412230 shutdown_connections 2024-03-27T11:01:28.335 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.364+0000 7f3e12c00640 1 -- 172.21.15.97:0/3573412230 wait complete. 2024-03-27T11:01:28.335 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.366+0000 7f3e12c00640 1 Processor -- start 2024-03-27T11:01:28.336 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.366+0000 7f3e12c00640 1 -- start start 2024-03-27T11:01:28.336 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.366+0000 7f3e12c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e0c100230 0x7f3e0c19e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:01:28.336 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.366+0000 7f3e12c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3e0c102ba0 0x7f3e0c19e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:01:28.337 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.366+0000 7f3e12c00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3e0c105550 0x7f3e0c1a5750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:01:28.337 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.366+0000 7f3e12c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3e0c113b60 con 0x7f3e0c100230 2024-03-27T11:01:28.337 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.366+0000 7f3e12c00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f3e0c1139e0 con 0x7f3e0c102ba0 2024-03-27T11:01:28.337 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.366+0000 7f3e12c00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f3e0c113ce0 con 0x7f3e0c105550 2024-03-27T11:01:28.338 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.367+0000 7f3e11800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e0c100230 0x7f3e0c19e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:01:28.338 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.367+0000 7f3e11800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e0c100230 0x7f3e0c19e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52580/0 (socket says 172.21.15.97:52580) 2024-03-27T11:01:28.338 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.367+0000 7f3e10e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3e0c102ba0 0x7f3e0c19e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:01:28.338 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.367+0000 7f3e12200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3e0c105550 0x7f3e0c1a5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:01:28.338 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.367+0000 7f3e11800640 1 -- 172.21.15.97:0/3879102089 learned_addr learned my addr 172.21.15.97:0/3879102089 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:01:28.338 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.367+0000 7f3e12200640 1 --2- 172.21.15.97:0/3879102089 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3e0c105550 0x7f3e0c1a5750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.97:33590/0 (socket says 172.21.15.97:33590) 2024-03-27T11:01:28.339 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.368+0000 7f3e11800640 1 -- 172.21.15.97:0/3879102089 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3e0c105550 msgr2=0x7f3e0c1a5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:01:28.339 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.368+0000 7f3e11800640 1 --2- 172.21.15.97:0/3879102089 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3e0c105550 0x7f3e0c1a5750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:28.339 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.368+0000 7f3e11800640 1 -- 172.21.15.97:0/3879102089 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3e0c102ba0 msgr2=0x7f3e0c19e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:01:28.339 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.368+0000 7f3e11800640 1 --2- 172.21.15.97:0/3879102089 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3e0c102ba0 0x7f3e0c19e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:28.339 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.368+0000 7f3e11800640 1 -- 172.21.15.97:0/3879102089 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e0c06b6a0 con 0x7f3e0c100230 2024-03-27T11:01:28.339 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.368+0000 7f3e10e00640 1 --2- 172.21.15.97:0/3879102089 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3e0c102ba0 0x7f3e0c19e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T11:01:28.340 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.368+0000 7f3e11800640 1 --2- 172.21.15.97:0/3879102089 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e0c100230 0x7f3e0c19e1a0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f3e0000e9f0 tx=0x7f3e0000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:01:28.340 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.369+0000 7f3e0a000640 1 -- 172.21.15.97:0/3879102089 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e0000cd80 con 0x7f3e0c100230 2024-03-27T11:01:28.340 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.369+0000 7f3e0a000640 1 -- 172.21.15.97:0/3879102089 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e000108e0 con 0x7f3e0c100230 2024-03-27T11:01:28.340 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.369+0000 7f3e12c00640 1 -- 172.21.15.97:0/3879102089 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e0c06b990 con 0x7f3e0c100230 2024-03-27T11:01:28.340 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.369+0000 7f3e0a000640 1 -- 172.21.15.97:0/3879102089 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e00018920 con 0x7f3e0c100230 2024-03-27T11:01:28.340 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.369+0000 7f3e12c00640 1 -- 172.21.15.97:0/3879102089 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e0c06bf00 con 0x7f3e0c100230 2024-03-27T11:01:28.344 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.373+0000 7f3e0a000640 1 -- 172.21.15.97:0/3879102089 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f3e00010a90 con 0x7f3e0c100230 2024-03-27T11:01:28.345 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.375+0000 7f3e16e4b740 1 Processor -- start 2024-03-27T11:01:28.345 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.375+0000 7f3e16e4b740 1 -- start start 2024-03-27T11:01:28.345 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.375+0000 7f3e16e4b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8c0ac4b60 0x55d8c0ac4f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:01:28.345 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.375+0000 7f3e16e4b740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d8c0ac5550 0x55d8c0aca050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:01:28.345 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.375+0000 7f3e16e4b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d8c0aca6c0 0x55d8c0accb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:01:28.346 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.375+0000 7f3e16e4b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55d8c0a29d10 con 0x55d8c0ac4b60 2024-03-27T11:01:28.346 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.375+0000 7f3e16e4b740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55d8c09d2260 con 0x55d8c0aca6c0 2024-03-27T11:01:28.346 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.375+0000 7f3e16e4b740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55d8c0a25cc0 con 0x55d8c0ac5550 2024-03-27T11:01:28.346 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.376+0000 7f3e11800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d8c0ac5550 0x55d8c0aca050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:01:28.346 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.376+0000 7f3e0a000640 1 --2- 172.21.15.97:0/3879102089 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f3df40a3700 0x7f3df40a5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:01:28.346 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.376+0000 7f3e12200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8c0ac4b60 0x55d8c0ac4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:01:28.347 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.376+0000 7f3e11800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d8c0ac5550 0x55d8c0aca050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.97:33600/0 (socket says 172.21.15.97:33600) 2024-03-27T11:01:28.347 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.376+0000 7f3e11800640 1 -- 172.21.15.97:0/3630879953 learned_addr learned my addr 172.21.15.97:0/3630879953 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:01:28.347 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.376+0000 7f3e10e00640 1 --2- 172.21.15.97:0/3630879953 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d8c0aca6c0 0x55d8c0accb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:01:28.347 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.377+0000 7f3e0a000640 1 -- 172.21.15.97:0/3879102089 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f3e000cfff0 con 0x7f3e0c100230 2024-03-27T11:01:28.347 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.377+0000 7f3e10e00640 1 -- 172.21.15.97:0/3630879953 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d8c0ac5550 msgr2=0x55d8c0aca050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:01:28.348 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.377+0000 7f3e10e00640 1 --2- 172.21.15.97:0/3630879953 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d8c0ac5550 0x55d8c0aca050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:28.348 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.377+0000 7f3e10e00640 1 -- 172.21.15.97:0/3630879953 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8c0ac4b60 msgr2=0x55d8c0ac4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:01:28.348 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.377+0000 7f3e10e00640 1 --2- 172.21.15.97:0/3630879953 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8c0ac4b60 0x55d8c0ac4f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:28.348 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.377+0000 7f3e12200640 1 --2- 172.21.15.97:0/3879102089 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f3df40a3700 0x7f3df40a5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:01:28.348 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.377+0000 7f3e10e00640 1 -- 172.21.15.97:0/3630879953 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8c09c9aa0 con 0x55d8c0aca6c0 2024-03-27T11:01:28.348 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.378+0000 7f3e10e00640 1 --2- 172.21.15.97:0/3630879953 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d8c0aca6c0 0x55d8c0accb50 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3df802c170 tx=0x7f3df8009610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a9b38fd7f0984e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:01:28.349 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.378+0000 7f3dfea00640 1 -- 172.21.15.97:0/3630879953 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df8045020 con 0x55d8c0aca6c0 2024-03-27T11:01:28.349 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.379+0000 7f3dfea00640 1 -- 172.21.15.97:0/3630879953 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3df8005040 con 0x55d8c0aca6c0 2024-03-27T11:01:28.349 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.379+0000 7f3dfea00640 1 -- 172.21.15.97:0/3630879953 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df8004920 con 0x55d8c0aca6c0 2024-03-27T11:01:28.349 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.380+0000 7f3e16e4b740 1 -- 172.21.15.97:0/3630879953 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d8c0aca6c0 msgr2=0x55d8c0accb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:01:28.350 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.380+0000 7f3e16e4b740 1 --2- 172.21.15.97:0/3630879953 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d8c0aca6c0 0x55d8c0accb50 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3df802c170 tx=0x7f3df8009610 comp rx=0 tx=0).stop 2024-03-27T11:01:28.350 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.380+0000 7f3e12200640 1 --2- 172.21.15.97:0/3879102089 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f3df40a3700 0x7f3df40a5bc0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x55d8c0ac5db0 tx=0x7f3e04004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:01:28.350 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.380+0000 7f3e16e4b740 1 -- 172.21.15.97:0/3630879953 shutdown_connections 2024-03-27T11:01:28.350 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.380+0000 7f3e16e4b740 1 --2- 172.21.15.97:0/3630879953 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d8c0aca6c0 0x55d8c0accb50 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:28.350 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.380+0000 7f3e16e4b740 1 --2- 172.21.15.97:0/3630879953 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d8c0ac5550 0x55d8c0aca050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:28.350 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.380+0000 7f3e16e4b740 1 --2- 172.21.15.97:0/3630879953 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8c0ac4b60 0x55d8c0ac4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:28.351 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.380+0000 7f3e16e4b740 1 -- 172.21.15.97:0/3630879953 >> 172.21.15.97:0/3630879953 conn(0x55d8c0ac00d0 msgr2=0x55d8c0ac04d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:01:28.351 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.380+0000 7f3e16e4b740 1 -- 172.21.15.97:0/3630879953 shutdown_connections 2024-03-27T11:01:28.351 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.380+0000 7f3e16e4b740 1 -- 172.21.15.97:0/3630879953 wait complete. 2024-03-27T11:01:28.351 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.381+0000 7f3e16e4b740 1 -- start start 2024-03-27T11:01:28.351 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.381+0000 7f3e16e4b740 1 Processor -- start 2024-03-27T11:01:28.352 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.382+0000 7f3e16e4b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8c0ac4b60 0x55d8c0acee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:01:28.352 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.382+0000 7f3e16e4b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d8c0ac5550 0x55d8c0acf340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:01:28.352 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.382+0000 7f3e16e4b740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d8c0aca6c0 0x55d8c0ad6380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:01:28.352 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.382+0000 7f3e16e4b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55d8c0acd610 con 0x55d8c0ac4b60 2024-03-27T11:01:28.353 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.382+0000 7f3e16e4b740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55d8c09e5630 con 0x55d8c0ac5550 2024-03-27T11:01:28.353 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.382+0000 7f3e16e4b740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55d8c0acd790 con 0x55d8c0aca6c0 2024-03-27T11:01:28.353 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.382+0000 7f3e16e4b740 10 client.0 _ll_register_callbacks cb 0x55d8c09eac50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T11:01:28.353 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.383+0000 7f3e11800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8c0ac4b60 0x55d8c0acee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:01:28.354 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.383+0000 7f3e11800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8c0ac4b60 0x55d8c0acee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52596/0 (socket says 172.21.15.97:52596) 2024-03-27T11:01:28.354 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.383+0000 7f3e11800640 1 -- 172.21.15.97:0/842384070 learned_addr learned my addr 172.21.15.97:0/842384070 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:01:28.354 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.383+0000 7f3e12200640 1 --2- 172.21.15.97:0/842384070 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d8c0aca6c0 0x55d8c0ad6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:01:28.354 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.383+0000 7f3e10e00640 1 --2- 172.21.15.97:0/842384070 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d8c0ac5550 0x55d8c0acf340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:01:28.354 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.383+0000 7f3e11800640 1 -- 172.21.15.97:0/842384070 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d8c0aca6c0 msgr2=0x55d8c0ad6380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:01:28.354 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.383+0000 7f3e11800640 1 --2- 172.21.15.97:0/842384070 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d8c0aca6c0 0x55d8c0ad6380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:28.354 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.383+0000 7f3e11800640 1 -- 172.21.15.97:0/842384070 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d8c0ac5550 msgr2=0x55d8c0acf340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:01:28.355 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.383+0000 7f3e11800640 1 --2- 172.21.15.97:0/842384070 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d8c0ac5550 0x55d8c0acf340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:28.355 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.383+0000 7f3e11800640 1 -- 172.21.15.97:0/842384070 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8c09f15f0 con 0x55d8c0ac4b60 2024-03-27T11:01:28.355 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.383+0000 7f3e12200640 1 --2- 172.21.15.97:0/842384070 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d8c0aca6c0 0x55d8c0ad6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T11:01:28.355 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.384+0000 7f3e11800640 1 --2- 172.21.15.97:0/842384070 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8c0ac4b60 0x55d8c0acee00 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f3e000a4bb0 tx=0x7f3e000cd9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c68f0604156f2551 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:01:28.355 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.384+0000 7f3df3400640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T11:01:28.355 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.384+0000 7f3df3400640 1 -- 172.21.15.97:0/842384070 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e00026040 con 0x55d8c0ac4b60 2024-03-27T11:01:28.355 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.384+0000 7f3df3400640 1 -- 172.21.15.97:0/842384070 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f3e00010430 con 0x55d8c0ac4b60 2024-03-27T11:01:28.355 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.384+0000 7f3e16e4b740 1 -- 172.21.15.97:0/842384070 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3df8005040 con 0x55d8c0ac4b60 2024-03-27T11:01:28.356 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.385+0000 7f3df3400640 1 -- 172.21.15.97:0/842384070 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f3e0009be80 con 0x55d8c0ac4b60 2024-03-27T11:01:28.357 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.385+0000 7f3e16e4b740 10 client.16524 fetch_fsmap learned FSMap version 34 2024-03-27T11:01:28.357 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.385+0000 7f3e16e4b740 1 -- 172.21.15.97:0/842384070 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55d8c073e210 con 0x55d8c0ac4b60 2024-03-27T11:01:28.357 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.385+0000 7f3df3400640 1 -- 172.21.15.97:0/842384070 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3df8005040 con 0x55d8c0ac4b60 2024-03-27T11:01:28.357 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.386+0000 7f3df3400640 1 -- 172.21.15.97:0/842384070 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f3e00018ac0 con 0x55d8c0ac4b60 2024-03-27T11:01:28.357 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.386+0000 7f3e16e4b740 10 client.16524 fetch_fsmap finished waiting for FSMap version 34 2024-03-27T11:01:28.357 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.386+0000 7f3e16e4b740 10 client.16524 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T11:01:28.357 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.386+0000 7f3e16e4b740 20 client.16524 populate_metadata read hostname 'smithi097' 2024-03-27T11:01:28.357 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.386+0000 7f3e16e4b740 1 --2- 172.21.15.97:0/842384070 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55d8c0adacc0 0x55d8c0add110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:01:28.357 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.386+0000 7f3e16e4b740 4 client.16524 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T11:01:28.357 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.386+0000 7f3e16e4b740 1 -- 172.21.15.97:0/842384070 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d8c0a37250 con 0x55d8c0adacc0 2024-03-27T11:01:28.357 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.387+0000 7f3e10e00640 1 --2- 172.21.15.97:0/842384070 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55d8c0adacc0 0x55d8c0add110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:01:28.358 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.387+0000 7f3e10e00640 1 --2- 172.21.15.97:0/842384070 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55d8c0adacc0 0x55d8c0add110 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10210ddf13a771a8 server_cookie=2789fa8e891a048c in_seq=0 out_seq=0 2024-03-27T11:01:28.358 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.387+0000 7f3df3400640 10 client.16524 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T11:01:28.359 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.389+0000 7f3df3400640 1 -- 172.21.15.97:0/842384070 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55d8c0a37250 con 0x55d8c0adacc0 2024-03-27T11:01:28.359 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.389+0000 7f3df3400640 10 client.16524 handle_command_reply: tid=0 2024-03-27T11:01:28.359 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.389+0000 7f3e16e4b740 1 client.16524 shutdown 2024-03-27T11:01:28.359 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.390+0000 7f3e16e4b740 1 -- 172.21.15.97:0/842384070 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8c0ac4b60 msgr2=0x55d8c0acee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:01:28.360 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.390+0000 7f3e16e4b740 1 --2- 172.21.15.97:0/842384070 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8c0ac4b60 0x55d8c0acee00 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f3e000a4bb0 tx=0x7f3e000cd9e0 comp rx=0 tx=0).stop 2024-03-27T11:01:28.361 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.390+0000 7f3e16e4b740 1 -- 172.21.15.97:0/842384070 shutdown_connections 2024-03-27T11:01:28.361 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.390+0000 7f3e16e4b740 1 --2- 172.21.15.97:0/842384070 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55d8c0adacc0 0x55d8c0add110 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:28.361 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.390+0000 7f3e16e4b740 1 --2- 172.21.15.97:0/842384070 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d8c0aca6c0 0x55d8c0ad6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:28.361 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.390+0000 7f3e16e4b740 1 --2- 172.21.15.97:0/842384070 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d8c0ac5550 0x55d8c0acf340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:28.361 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.390+0000 7f3e16e4b740 1 --2- 172.21.15.97:0/842384070 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8c0ac4b60 0x55d8c0acee00 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:28.362 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.390+0000 7f3e16e4b740 1 -- 172.21.15.97:0/842384070 >> 172.21.15.97:0/842384070 conn(0x55d8c0ac00d0 msgr2=0x55d8c0abf310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:01:28.362 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.390+0000 7f3df3400640 0 client.16524 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T11:01:28.362 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.391+0000 7f3e16e4b740 1 -- 172.21.15.97:0/842384070 shutdown_connections 2024-03-27T11:01:28.362 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.391+0000 7f3e16e4b740 1 -- 172.21.15.97:0/842384070 wait complete. 2024-03-27T11:01:28.362 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.391+0000 7f3e16e4b740 20 client.16524 trim_cache size 0 max 16384 2024-03-27T11:01:28.388 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.417+0000 7f3e16e4b740 1 Processor -- start 2024-03-27T11:01:28.388 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.418+0000 7f3e16e4b740 1 -- start start 2024-03-27T11:01:28.388 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.418+0000 7f3e16e4b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d8c0ac00d0 0x55d8c0ac3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:01:28.388 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.418+0000 7f3e16e4b740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d8c0ac4b60 0x55d8c0ad6310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:01:28.388 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.418+0000 7f3e16e4b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8c0ac5550 0x55d8c0acea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:01:28.388 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.418+0000 7f3e16e4b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3e0c1b7aa0 con 0x55d8c0ac5550 2024-03-27T11:01:28.389 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.418+0000 7f3e16e4b740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f3e0c1b95e0 con 0x55d8c0ac00d0 2024-03-27T11:01:28.389 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.418+0000 7f3e16e4b740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f3e0c1b8170 con 0x55d8c0ac4b60 2024-03-27T11:01:28.389 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.419+0000 7f3e12200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8c0ac5550 0x55d8c0acea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:01:28.389 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.419+0000 7f3e10e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d8c0ac4b60 0x55d8c0ad6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:01:28.389 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.419+0000 7f3e10e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d8c0ac4b60 0x55d8c0ad6310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.97:33612/0 (socket says 172.21.15.97:33612) 2024-03-27T11:01:28.389 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.419+0000 7f3e10e00640 1 -- 172.21.15.97:0/3868572040 learned_addr learned my addr 172.21.15.97:0/3868572040 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:01:28.390 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.419+0000 7f3e11800640 1 --2- 172.21.15.97:0/3868572040 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d8c0ac00d0 0x55d8c0ac3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:01:28.390 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.419+0000 7f3e10e00640 1 -- 172.21.15.97:0/3868572040 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d8c0ac00d0 msgr2=0x55d8c0ac3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:01:28.390 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.419+0000 7f3e10e00640 1 --2- 172.21.15.97:0/3868572040 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d8c0ac00d0 0x55d8c0ac3a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:28.390 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.419+0000 7f3e10e00640 1 -- 172.21.15.97:0/3868572040 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8c0ac5550 msgr2=0x55d8c0acea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:01:28.390 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.419+0000 7f3e10e00640 1 --2- 172.21.15.97:0/3868572040 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8c0ac5550 0x55d8c0acea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:28.390 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.419+0000 7f3e10e00640 1 -- 172.21.15.97:0/3868572040 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8c080a620 con 0x55d8c0ac4b60 2024-03-27T11:01:28.390 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.420+0000 7f3e10e00640 1 --2- 172.21.15.97:0/3868572040 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d8c0ac4b60 0x55d8c0ad6310 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3df8005ab0 tx=0x7f3df802bc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74ea100091daa1d0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:01:28.390 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.420+0000 7f3e12200640 1 --2- 172.21.15.97:0/3868572040 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8c0ac5550 0x55d8c0acea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T11:01:28.391 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.420+0000 7f3dfe000640 1 -- 172.21.15.97:0/3868572040 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df8045040 con 0x55d8c0ac4b60 2024-03-27T11:01:28.391 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.420+0000 7f3dfe000640 1 -- 172.21.15.97:0/3868572040 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3df80510d0 con 0x55d8c0ac4b60 2024-03-27T11:01:28.391 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.420+0000 7f3dfe000640 1 -- 172.21.15.97:0/3868572040 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df8003830 con 0x55d8c0ac4b60 2024-03-27T11:01:28.391 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.421+0000 7f3e16e4b740 1 -- 172.21.15.97:0/3868572040 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d8c0ac4b60 msgr2=0x55d8c0ad6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:01:28.391 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.421+0000 7f3e16e4b740 1 --2- 172.21.15.97:0/3868572040 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d8c0ac4b60 0x55d8c0ad6310 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3df8005ab0 tx=0x7f3df802bc30 comp rx=0 tx=0).stop 2024-03-27T11:01:28.391 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.421+0000 7f3e16e4b740 1 -- 172.21.15.97:0/3868572040 shutdown_connections 2024-03-27T11:01:28.391 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.421+0000 7f3e16e4b740 1 --2- 172.21.15.97:0/3868572040 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8c0ac5550 0x55d8c0acea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:28.392 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.421+0000 7f3e16e4b740 1 --2- 172.21.15.97:0/3868572040 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d8c0ac4b60 0x55d8c0ad6310 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:28.392 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.421+0000 7f3e16e4b740 1 --2- 172.21.15.97:0/3868572040 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d8c0ac00d0 0x55d8c0ac3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:28.392 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.421+0000 7f3e16e4b740 1 -- 172.21.15.97:0/3868572040 >> 172.21.15.97:0/3868572040 conn(0x55d8c0aca200 msgr2=0x55d8c0ac3330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:01:28.392 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.421+0000 7f3e16e4b740 1 -- 172.21.15.97:0/3868572040 shutdown_connections 2024-03-27T11:01:28.392 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.422+0000 7f3e16e4b740 1 -- 172.21.15.97:0/3868572040 wait complete. 2024-03-27T11:01:28.393 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.423+0000 7f3e16e4b740 1 -- start start 2024-03-27T11:01:28.393 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.423+0000 7f3e16e4b740 1 Processor -- start 2024-03-27T11:01:28.394 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.424+0000 7f3e16e4b740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d8c0ac00d0 0x55d8c0ad32f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:01:28.395 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.424+0000 7f3e16e4b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d8c0ac4b60 0x55d8c0ad3830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:01:28.395 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.424+0000 7f3e16e4b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8c0ac5550 0x55d8c0b3f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:01:28.395 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.424+0000 7f3e16e4b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3e0c1b8f10 con 0x55d8c0ac5550 2024-03-27T11:01:28.395 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.424+0000 7f3e16e4b740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f3e0c1b8840 con 0x55d8c0ac4b60 2024-03-27T11:01:28.395 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.424+0000 7f3e16e4b740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f3e0c072b60 con 0x55d8c0ac00d0 2024-03-27T11:01:28.395 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.424+0000 7f3e16e4b740 10 client.0 _ll_register_callbacks cb 0x7f3e00098100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T11:01:28.395 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.425+0000 7f3e12200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8c0ac5550 0x55d8c0b3f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:01:28.395 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.425+0000 7f3e12200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8c0ac5550 0x55d8c0b3f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52620/0 (socket says 172.21.15.97:52620) 2024-03-27T11:01:28.396 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.425+0000 7f3e12200640 1 -- 172.21.15.97:0/1495670146 learned_addr learned my addr 172.21.15.97:0/1495670146 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:01:28.396 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.425+0000 7f3e10e00640 1 --2- 172.21.15.97:0/1495670146 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d8c0ac4b60 0x55d8c0ad3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:01:28.396 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.425+0000 7f3e11800640 1 --2- 172.21.15.97:0/1495670146 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d8c0ac00d0 0x55d8c0ad32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:01:28.397 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.426+0000 7f3e10e00640 1 -- 172.21.15.97:0/1495670146 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d8c0ac00d0 msgr2=0x55d8c0ad32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:01:28.397 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.426+0000 7f3e10e00640 1 --2- 172.21.15.97:0/1495670146 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d8c0ac00d0 0x55d8c0ad32f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:28.397 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.426+0000 7f3e10e00640 1 -- 172.21.15.97:0/1495670146 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8c0ac5550 msgr2=0x55d8c0b3f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:01:28.397 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.426+0000 7f3e10e00640 1 --2- 172.21.15.97:0/1495670146 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8c0ac5550 0x55d8c0b3f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:28.397 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.426+0000 7f3e10e00640 1 -- 172.21.15.97:0/1495670146 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8c0b3f630 con 0x55d8c0ac4b60 2024-03-27T11:01:28.397 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.426+0000 7f3e12200640 1 --2- 172.21.15.97:0/1495670146 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8c0ac5550 0x55d8c0b3f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T11:01:28.397 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.426+0000 7f3e10e00640 1 --2- 172.21.15.97:0/1495670146 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d8c0ac4b60 0x55d8c0ad3830 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3df8005ab0 tx=0x7f3df80055d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ed97498b1886db6 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:01:28.397 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.426+0000 7f3df3400640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-03-27T11:01:28.398 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.427+0000 7f3df3400640 1 -- 172.21.15.97:0/1495670146 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df8045040 con 0x55d8c0ac4b60 2024-03-27T11:01:28.398 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.427+0000 7f3e16e4b740 1 -- 172.21.15.97:0/1495670146 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3df80510d0 con 0x55d8c0ac4b60 2024-03-27T11:01:28.398 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.427+0000 7f3df3400640 1 -- 172.21.15.97:0/1495670146 <== mon.2 v2:172.21.15.114:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f3df80519c0 con 0x55d8c0ac4b60 2024-03-27T11:01:28.398 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.427+0000 7f3df3400640 1 -- 172.21.15.97:0/1495670146 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f3df8051bc0 con 0x55d8c0ac4b60 2024-03-27T11:01:28.399 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.427+0000 7f3e16e4b740 10 client.36188 fetch_fsmap learned FSMap version 34 2024-03-27T11:01:28.399 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.427+0000 7f3e16e4b740 1 -- 172.21.15.97:0/1495670146 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55d8c0b40940 con 0x55d8c0ac4b60 2024-03-27T11:01:28.399 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.428+0000 7f3df3400640 1 -- 172.21.15.97:0/1495670146 <== mon.2 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3df80510d0 con 0x55d8c0ac4b60 2024-03-27T11:01:28.399 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.428+0000 7f3df3400640 1 -- 172.21.15.97:0/1495670146 <== mon.2 v2:172.21.15.114:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f3df8003a70 con 0x55d8c0ac4b60 2024-03-27T11:01:28.399 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.428+0000 7f3e16e4b740 10 client.36188 fetch_fsmap finished waiting for FSMap version 34 2024-03-27T11:01:28.399 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.429+0000 7f3e16e4b740 10 client.36188 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T11:01:28.399 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.429+0000 7f3e16e4b740 20 client.36188 populate_metadata read hostname 'smithi097' 2024-03-27T11:01:28.399 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.429+0000 7f3e16e4b740 1 --2- 172.21.15.97:0/1495670146 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55d8c0aca6c0 0x55d8c0b43250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:01:28.400 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.429+0000 7f3e16e4b740 4 client.36188 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-27T11:01:28.400 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.429+0000 7f3e16e4b740 1 -- 172.21.15.97:0/1495670146 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d8c0ac6380 con 0x55d8c0aca6c0 2024-03-27T11:01:28.400 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.429+0000 7f3e11800640 1 --2- 172.21.15.97:0/1495670146 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55d8c0aca6c0 0x55d8c0b43250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:01:28.400 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.430+0000 7f3e11800640 1 --2- 172.21.15.97:0/1495670146 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55d8c0aca6c0 0x55d8c0b43250 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36904c6f1cd3a05a server_cookie=c2c28631d6e97712 in_seq=0 out_seq=0 2024-03-27T11:01:28.400 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.430+0000 7f3df3400640 10 client.36188 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T11:01:28.413 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.443+0000 7f3df3400640 1 -- 172.21.15.97:0/1495670146 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d8c0ac6380 con 0x55d8c0aca6c0 2024-03-27T11:01:28.413 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.443+0000 7f3df3400640 10 client.36188 handle_command_reply: tid=0 2024-03-27T11:01:28.413 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.443+0000 7f3e16e4b740 1 client.36188 shutdown 2024-03-27T11:01:28.413 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.443+0000 7f3e16e4b740 1 -- 172.21.15.97:0/1495670146 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d8c0ac4b60 msgr2=0x55d8c0ad3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:01:28.413 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.443+0000 7f3e16e4b740 1 --2- 172.21.15.97:0/1495670146 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d8c0ac4b60 0x55d8c0ad3830 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3df8005ab0 tx=0x7f3df80055d0 comp rx=0 tx=0).stop 2024-03-27T11:01:28.414 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.443+0000 7f3e16e4b740 1 -- 172.21.15.97:0/1495670146 shutdown_connections 2024-03-27T11:01:28.414 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.443+0000 7f3e16e4b740 1 --2- 172.21.15.97:0/1495670146 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55d8c0aca6c0 0x55d8c0b43250 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:28.414 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.443+0000 7f3e16e4b740 1 --2- 172.21.15.97:0/1495670146 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8c0ac5550 0x55d8c0b3f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:28.414 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.443+0000 7f3e16e4b740 1 --2- 172.21.15.97:0/1495670146 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d8c0ac4b60 0x55d8c0ad3830 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:28.415 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.444+0000 7f3e16e4b740 1 --2- 172.21.15.97:0/1495670146 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d8c0ac00d0 0x55d8c0ad32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:28.415 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.444+0000 7f3df3400640 0 client.36188 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T11:01:28.415 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.444+0000 7f3e16e4b740 1 -- 172.21.15.97:0/1495670146 >> 172.21.15.97:0/1495670146 conn(0x55d8c0aca200 msgr2=0x55d8c0ac3330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:01:28.415 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.444+0000 7f3e16e4b740 1 -- 172.21.15.97:0/1495670146 shutdown_connections 2024-03-27T11:01:28.415 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.444+0000 7f3e16e4b740 1 -- 172.21.15.97:0/1495670146 wait complete. 2024-03-27T11:01:28.415 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.444+0000 7f3e16e4b740 20 client.36188 trim_cache size 0 max 16384 2024-03-27T11:01:28.415 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-27T11:01:28.415 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-27T11:01:28.415 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-27T11:01:28.416 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-27T11:01:28.416 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.445+0000 7f3e12c00640 1 -- 172.21.15.97:0/3879102089 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f3df40a3700 msgr2=0x7f3df40a5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:01:28.416 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.445+0000 7f3e12c00640 1 --2- 172.21.15.97:0/3879102089 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f3df40a3700 0x7f3df40a5bc0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x55d8c0ac5db0 tx=0x7f3e04004a40 comp rx=0 tx=0).stop 2024-03-27T11:01:28.416 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.445+0000 7f3e12c00640 1 -- 172.21.15.97:0/3879102089 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e0c100230 msgr2=0x7f3e0c19e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:01:28.416 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.445+0000 7f3e12c00640 1 --2- 172.21.15.97:0/3879102089 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e0c100230 0x7f3e0c19e1a0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f3e0000e9f0 tx=0x7f3e0000eeb0 comp rx=0 tx=0).stop 2024-03-27T11:01:28.416 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.446+0000 7f3e12c00640 1 -- 172.21.15.97:0/3879102089 shutdown_connections 2024-03-27T11:01:28.417 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.446+0000 7f3e12c00640 1 --2- 172.21.15.97:0/3879102089 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3e0c105550 0x7f3e0c1a5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:28.417 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.446+0000 7f3e12c00640 1 --2- 172.21.15.97:0/3879102089 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f3df40a3700 0x7f3df40a5bc0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:28.417 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.446+0000 7f3e12c00640 1 --2- 172.21.15.97:0/3879102089 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3e0c102ba0 0x7f3e0c19e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:28.417 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.446+0000 7f3e12c00640 1 --2- 172.21.15.97:0/3879102089 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e0c100230 0x7f3e0c19e1a0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:28.417 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.446+0000 7f3e12c00640 1 -- 172.21.15.97:0/3879102089 >> 172.21.15.97:0/3879102089 conn(0x7f3e0c0fbe50 msgr2=0x7f3e0c100fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:01:28.417 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.446+0000 7f3e12c00640 1 -- 172.21.15.97:0/3879102089 shutdown_connections 2024-03-27T11:01:28.417 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:28.446+0000 7f3e12c00640 1 -- 172.21.15.97:0/3879102089 wait complete. 2024-03-27T11:01:29.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:01:29 smithi097 ceph-mon[31859]: pgmap v586: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 58 MiB/s wr, 940 op/s 2024-03-27T11:01:29.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:01:29 smithi177 ceph-mon[35294]: pgmap v586: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 58 MiB/s wr, 940 op/s 2024-03-27T11:01:29.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:01:29 smithi114 ceph-mon[35338]: pgmap v586: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 58 MiB/s wr, 940 op/s 2024-03-27T11:01:31.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:01:31 smithi097 ceph-mon[31859]: pgmap v587: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 62 MiB/s wr, 1.01k op/s 2024-03-27T11:01:31.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:01:31 smithi177 ceph-mon[35294]: pgmap v587: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 62 MiB/s wr, 1.01k op/s 2024-03-27T11:01:31.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:01:31 smithi114 ceph-mon[35338]: pgmap v587: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 62 MiB/s wr, 1.01k op/s 2024-03-27T11:01:33.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:01:33 smithi097 ceph-mon[31859]: pgmap v588: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 56 MiB/s wr, 902 op/s 2024-03-27T11:01:33.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:01:33 smithi177 ceph-mon[35294]: pgmap v588: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 56 MiB/s wr, 902 op/s 2024-03-27T11:01:33.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:01:33 smithi114 ceph-mon[35338]: pgmap v588: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 56 MiB/s wr, 902 op/s 2024-03-27T11:01:35.644 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:01:35 smithi097 ceph-mon[31859]: pgmap v589: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 57 MiB/s wr, 923 op/s 2024-03-27T11:01:35.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:01:35 smithi177 ceph-mon[35294]: pgmap v589: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 57 MiB/s wr, 923 op/s 2024-03-27T11:01:35.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:01:35 smithi114 ceph-mon[35338]: pgmap v589: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 57 MiB/s wr, 923 op/s 2024-03-27T11:01:37.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:01:37 smithi097 ceph-mon[31859]: pgmap v590: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 55 MiB/s wr, 883 op/s 2024-03-27T11:01:37.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:01:37 smithi177 ceph-mon[35294]: pgmap v590: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 55 MiB/s wr, 883 op/s 2024-03-27T11:01:37.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:01:37 smithi114 ceph-mon[35338]: pgmap v590: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 55 MiB/s wr, 883 op/s 2024-03-27T11:01:39.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:01:39 smithi097 ceph-mon[31859]: pgmap v591: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 56 MiB/s wr, 909 op/s 2024-03-27T11:01:39.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:01:39 smithi177 ceph-mon[35294]: pgmap v591: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 56 MiB/s wr, 909 op/s 2024-03-27T11:01:39.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:01:39 smithi114 ceph-mon[35338]: pgmap v591: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 56 MiB/s wr, 909 op/s 2024-03-27T11:01:41.630 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.658+0000 7f3759800640 1 -- 172.21.15.97:0/1583171564 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3754075b30 msgr2=0x7f3754073f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:01:41.630 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.658+0000 7f3759800640 1 --2- 172.21.15.97:0/1583171564 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3754075b30 0x7f3754073f80 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f3744009860 tx=0x7f374402bc00 comp rx=0 tx=0).stop 2024-03-27T11:01:41.630 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.659+0000 7f3759800640 1 -- 172.21.15.97:0/1583171564 shutdown_connections 2024-03-27T11:01:41.630 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.659+0000 7f3759800640 1 --2- 172.21.15.97:0/1583171564 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f37541035f0 0x7f3754103a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:41.630 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.659+0000 7f3759800640 1 --2- 172.21.15.97:0/1583171564 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3754074550 0x7f37540749b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:41.630 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.659+0000 7f3759800640 1 --2- 172.21.15.97:0/1583171564 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3754075b30 0x7f3754073f80 secure :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f3744009860 tx=0x7f374402bc00 comp rx=0 tx=0).stop 2024-03-27T11:01:41.630 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.659+0000 7f3759800640 1 -- 172.21.15.97:0/1583171564 >> 172.21.15.97:0/1583171564 conn(0x7f37540fd110 msgr2=0x7f37540ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:01:41.631 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.659+0000 7f3759800640 1 -- 172.21.15.97:0/1583171564 shutdown_connections 2024-03-27T11:01:41.631 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.659+0000 7f3759800640 1 -- 172.21.15.97:0/1583171564 wait complete. 2024-03-27T11:01:41.631 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.660+0000 7f3759800640 1 Processor -- start 2024-03-27T11:01:41.632 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.661+0000 7f3759800640 1 -- start start 2024-03-27T11:01:41.632 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.661+0000 7f3759800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3754074550 0x7f375419e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:01:41.632 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.661+0000 7f3759800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f37541035f0 0x7f375419ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:01:41.632 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.661+0000 7f3759800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37541a3af0 0x7f37541a5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:01:41.632 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.661+0000 7f3759800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3754113cd0 con 0x7f37541a3af0 2024-03-27T11:01:41.633 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.661+0000 7f3759800640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f3754113b50 con 0x7f3754074550 2024-03-27T11:01:41.633 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.661+0000 7f3759800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f3754113e50 con 0x7f37541035f0 2024-03-27T11:01:41.634 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.662+0000 7f3752a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3754074550 0x7f375419e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:01:41.634 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.662+0000 7f3752000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f37541035f0 0x7f375419ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:01:41.634 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.662+0000 7f3752000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f37541035f0 0x7f375419ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.97:34214/0 (socket says 172.21.15.97:34214) 2024-03-27T11:01:41.634 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.662+0000 7f3752000640 1 -- 172.21.15.97:0/3091128136 learned_addr learned my addr 172.21.15.97:0/3091128136 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:01:41.634 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.662+0000 7f3753400640 1 --2- 172.21.15.97:0/3091128136 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37541a3af0 0x7f37541a5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:01:41.634 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.662+0000 7f3752000640 1 -- 172.21.15.97:0/3091128136 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3754074550 msgr2=0x7f375419e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:01:41.634 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.662+0000 7f3752000640 1 --2- 172.21.15.97:0/3091128136 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3754074550 0x7f375419e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:41.635 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.662+0000 7f3752000640 1 -- 172.21.15.97:0/3091128136 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37541a3af0 msgr2=0x7f37541a5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:01:41.635 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.662+0000 7f3752000640 1 --2- 172.21.15.97:0/3091128136 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37541a3af0 0x7f37541a5ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:41.635 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.662+0000 7f3752000640 1 -- 172.21.15.97:0/3091128136 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f375406b6a0 con 0x7f37541035f0 2024-03-27T11:01:41.635 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.663+0000 7f3752a00640 1 --2- 172.21.15.97:0/3091128136 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3754074550 0x7f375419e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T11:01:41.636 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.663+0000 7f3752000640 1 --2- 172.21.15.97:0/3091128136 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f37541035f0 0x7f375419ea90 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f374000e9f0 tx=0x7f374000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:01:41.636 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.663+0000 7f374b400640 1 -- 172.21.15.97:0/3091128136 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f374000cd80 con 0x7f37541035f0 2024-03-27T11:01:41.636 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.664+0000 7f374b400640 1 -- 172.21.15.97:0/3091128136 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f37400108e0 con 0x7f37541035f0 2024-03-27T11:01:41.636 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.664+0000 7f374b400640 1 -- 172.21.15.97:0/3091128136 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3740018920 con 0x7f37541035f0 2024-03-27T11:01:41.636 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.664+0000 7f3759800640 1 -- 172.21.15.97:0/3091128136 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f375406b990 con 0x7f37541035f0 2024-03-27T11:01:41.636 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.664+0000 7f3759800640 1 -- 172.21.15.97:0/3091128136 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f375406be90 con 0x7f37541035f0 2024-03-27T11:01:41.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:01:41 smithi097 ceph-mon[31859]: pgmap v592: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 61 MiB/s wr, 980 op/s 2024-03-27T11:01:41.640 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.669+0000 7f375c5b0740 1 Processor -- start 2024-03-27T11:01:41.640 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.669+0000 7f375c5b0740 1 -- start start 2024-03-27T11:01:41.641 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.670+0000 7f375c5b0740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5582a96a3980 0x5582a96a3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:01:41.641 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.670+0000 7f375c5b0740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5582a96a4370 0x5582a96a8e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:01:41.641 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.670+0000 7f375c5b0740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5582a96a94e0 0x5582a96ab970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:01:41.641 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.670+0000 7f375c5b0740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5582a9608d10 con 0x5582a96a94e0 2024-03-27T11:01:41.641 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.670+0000 7f375c5b0740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5582a95b1260 con 0x5582a96a3980 2024-03-27T11:01:41.641 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.670+0000 7f375c5b0740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5582a9604cc0 con 0x5582a96a4370 2024-03-27T11:01:41.643 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.671+0000 7f3753400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5582a96a3980 0x5582a96a3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:01:41.643 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.671+0000 7f3752a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5582a96a4370 0x5582a96a8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:01:41.643 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.671+0000 7f3753400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5582a96a3980 0x5582a96a3da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:47206/0 (socket says 172.21.15.97:47206) 2024-03-27T11:01:41.643 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.671+0000 7f3752a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5582a96a4370 0x5582a96a8e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.97:34216/0 (socket says 172.21.15.97:34216) 2024-03-27T11:01:41.643 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.671+0000 7f3753400640 1 -- 172.21.15.97:0/1546000702 learned_addr learned my addr 172.21.15.97:0/1546000702 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:01:41.643 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.671+0000 7f374b400640 1 -- 172.21.15.97:0/3091128136 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f3740010430 con 0x7f37541035f0 2024-03-27T11:01:41.644 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.671+0000 7f3753400640 1 -- 172.21.15.97:0/1546000702 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5582a96a4370 msgr2=0x5582a96a8e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:01:41.644 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.671+0000 7f3753400640 1 --2- 172.21.15.97:0/1546000702 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5582a96a4370 0x5582a96a8e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:41.644 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.671+0000 7f3753400640 1 -- 172.21.15.97:0/1546000702 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5582a96a94e0 msgr2=0x5582a96ab970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:01:41.644 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.671+0000 7f3753400640 1 --2- 172.21.15.97:0/1546000702 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5582a96a94e0 0x5582a96ab970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:41.644 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.671+0000 7f3753400640 1 -- 172.21.15.97:0/1546000702 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582a95a8aa0 con 0x5582a96a3980 2024-03-27T11:01:41.645 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.672+0000 7f3753400640 1 --2- 172.21.15.97:0/1546000702 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5582a96a3980 0x5582a96a3da0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f374c00e580 tx=0x7f374c0095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa4a254b2a86e48b server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:01:41.646 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.672+0000 7f373fe00640 1 -- 172.21.15.97:0/1546000702 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f374c003970 con 0x5582a96a3980 2024-03-27T11:01:41.646 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.672+0000 7f373fe00640 1 -- 172.21.15.97:0/1546000702 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f374c00d830 con 0x5582a96a3980 2024-03-27T11:01:41.646 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.672+0000 7f373fe00640 1 -- 172.21.15.97:0/1546000702 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f374c004500 con 0x5582a96a3980 2024-03-27T11:01:41.646 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.673+0000 7f374b400640 1 --2- 172.21.15.97:0/3091128136 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f37380a3780 0x7f37380a5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:01:41.646 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.673+0000 7f375c5b0740 1 -- 172.21.15.97:0/1546000702 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5582a96a3980 msgr2=0x5582a96a3da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:01:41.646 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.673+0000 7f375c5b0740 1 --2- 172.21.15.97:0/1546000702 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5582a96a3980 0x5582a96a3da0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f374c00e580 tx=0x7f374c0095c0 comp rx=0 tx=0).stop 2024-03-27T11:01:41.646 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.674+0000 7f374b400640 1 -- 172.21.15.97:0/3091128136 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f37400cffb0 con 0x7f37541035f0 2024-03-27T11:01:41.647 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.674+0000 7f375c5b0740 1 -- 172.21.15.97:0/1546000702 shutdown_connections 2024-03-27T11:01:41.647 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.674+0000 7f375c5b0740 1 --2- 172.21.15.97:0/1546000702 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5582a96a94e0 0x5582a96ab970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:41.647 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.674+0000 7f375c5b0740 1 --2- 172.21.15.97:0/1546000702 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5582a96a4370 0x5582a96a8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:41.647 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.674+0000 7f375c5b0740 1 --2- 172.21.15.97:0/1546000702 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5582a96a3980 0x5582a96a3da0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:41.647 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.674+0000 7f375c5b0740 1 -- 172.21.15.97:0/1546000702 >> 172.21.15.97:0/1546000702 conn(0x5582a969eef0 msgr2=0x5582a969f2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:01:41.647 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.674+0000 7f3752a00640 1 --2- 172.21.15.97:0/3091128136 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f37380a3780 0x7f37380a5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:01:41.647 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.674+0000 7f375c5b0740 1 -- 172.21.15.97:0/1546000702 shutdown_connections 2024-03-27T11:01:41.650 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.675+0000 7f375c5b0740 1 -- 172.21.15.97:0/1546000702 wait complete. 2024-03-27T11:01:41.650 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.675+0000 7f375c5b0740 1 -- start start 2024-03-27T11:01:41.650 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.675+0000 7f375c5b0740 1 Processor -- start 2024-03-27T11:01:41.650 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.676+0000 7f375c5b0740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5582a96a3980 0x5582a96add40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:01:41.650 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.676+0000 7f375c5b0740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5582a96a4370 0x5582a96ae280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:01:41.650 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.676+0000 7f375c5b0740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5582a96a94e0 0x5582a96b52c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:01:41.651 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.676+0000 7f375c5b0740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5582a96ac490 con 0x5582a96a4370 2024-03-27T11:01:41.651 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.676+0000 7f375c5b0740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5582a95c4630 con 0x5582a96a3980 2024-03-27T11:01:41.651 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.676+0000 7f375c5b0740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5582a96ac610 con 0x5582a96a94e0 2024-03-27T11:01:41.651 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.676+0000 7f3752a00640 1 --2- 172.21.15.97:0/3091128136 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f37380a3780 0x7f37380a5c40 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f374402c190 tx=0x7f374402e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:01:41.651 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.677+0000 7f375c5b0740 10 client.0 _ll_register_callbacks cb 0x5582a95c9c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T11:01:41.651 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.677+0000 7f3752a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5582a96a3980 0x5582a96add40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:01:41.651 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.677+0000 7f3752a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5582a96a3980 0x5582a96add40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:47222/0 (socket says 172.21.15.97:47222) 2024-03-27T11:01:41.651 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.677+0000 7f3752000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5582a96a4370 0x5582a96ae280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:01:41.652 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.677+0000 7f3752a00640 1 -- 172.21.15.97:0/31837776 learned_addr learned my addr 172.21.15.97:0/31837776 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:01:41.652 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.677+0000 7f3753400640 1 --2- 172.21.15.97:0/31837776 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5582a96a94e0 0x5582a96b52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:01:41.652 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.678+0000 7f3752a00640 1 -- 172.21.15.97:0/31837776 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5582a96a94e0 msgr2=0x5582a96b52c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:01:41.653 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.678+0000 7f3752a00640 1 --2- 172.21.15.97:0/31837776 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5582a96a94e0 0x5582a96b52c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:41.653 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.678+0000 7f3752a00640 1 -- 172.21.15.97:0/31837776 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5582a96a4370 msgr2=0x5582a96ae280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:01:41.653 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.678+0000 7f3752a00640 1 --2- 172.21.15.97:0/31837776 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5582a96a4370 0x5582a96ae280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:41.653 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.678+0000 7f3752a00640 1 -- 172.21.15.97:0/31837776 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582a95d05f0 con 0x5582a96a3980 2024-03-27T11:01:41.653 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.678+0000 7f3752000640 1 --2- 172.21.15.97:0/31837776 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5582a96a4370 0x5582a96ae280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T11:01:41.653 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.678+0000 7f3752a00640 1 --2- 172.21.15.97:0/31837776 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5582a96a3980 0x5582a96add40 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3744009250 tx=0x7f374402bc00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f36c8c5ce941f60 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:01:41.653 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.678+0000 7f373cc00640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-03-27T11:01:41.653 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.679+0000 7f373cc00640 1 -- 172.21.15.97:0/31837776 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3744039040 con 0x5582a96a3980 2024-03-27T11:01:41.653 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.679+0000 7f373cc00640 1 -- 172.21.15.97:0/31837776 <== mon.2 v2:172.21.15.114:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f3744046070 con 0x5582a96a3980 2024-03-27T11:01:41.653 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.679+0000 7f375c5b0740 1 -- 172.21.15.97:0/31837776 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f374c00d830 con 0x5582a96a3980 2024-03-27T11:01:41.653 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.679+0000 7f373cc00640 1 -- 172.21.15.97:0/31837776 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f3744005260 con 0x5582a96a3980 2024-03-27T11:01:41.653 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.679+0000 7f375c5b0740 10 client.36200 fetch_fsmap learned FSMap version 34 2024-03-27T11:01:41.653 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.679+0000 7f375c5b0740 1 -- 172.21.15.97:0/31837776 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5582a931d210 con 0x5582a96a3980 2024-03-27T11:01:41.654 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.680+0000 7f373cc00640 1 -- 172.21.15.97:0/31837776 <== mon.2 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f374c00d830 con 0x5582a96a3980 2024-03-27T11:01:41.655 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.680+0000 7f373cc00640 1 -- 172.21.15.97:0/31837776 <== mon.2 v2:172.21.15.114:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f37440046c0 con 0x5582a96a3980 2024-03-27T11:01:41.655 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.681+0000 7f375c5b0740 10 client.36200 fetch_fsmap finished waiting for FSMap version 34 2024-03-27T11:01:41.655 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.681+0000 7f375c5b0740 10 client.36200 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T11:01:41.655 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.681+0000 7f375c5b0740 20 client.36200 populate_metadata read hostname 'smithi097' 2024-03-27T11:01:41.655 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.681+0000 7f375c5b0740 1 --2- 172.21.15.97:0/31837776 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5582a96cab00 0x5582a96ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:01:41.655 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.681+0000 7f375c5b0740 4 client.36200 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T11:01:41.655 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.681+0000 7f375c5b0740 1 -- 172.21.15.97:0/31837776 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5582a9616250 con 0x5582a96cab00 2024-03-27T11:01:41.655 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.682+0000 7f3752000640 1 --2- 172.21.15.97:0/31837776 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5582a96cab00 0x5582a96ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:01:41.655 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.683+0000 7f373cc00640 10 client.36200 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T11:01:41.655 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.683+0000 7f3752000640 1 --2- 172.21.15.97:0/31837776 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5582a96cab00 0x5582a96ccf30 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6727f7e745761b server_cookie=bd2f3ff25d6462d8 in_seq=0 out_seq=0 2024-03-27T11:01:41.656 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.685+0000 7f373cc00640 1 -- 172.21.15.97:0/31837776 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5582a9616250 con 0x5582a96cab00 2024-03-27T11:01:41.656 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.685+0000 7f373cc00640 10 client.36200 handle_command_reply: tid=0 2024-03-27T11:01:41.656 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.685+0000 7f375c5b0740 1 client.36200 shutdown 2024-03-27T11:01:41.657 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.686+0000 7f375c5b0740 1 -- 172.21.15.97:0/31837776 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5582a96a3980 msgr2=0x5582a96add40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:01:41.657 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.686+0000 7f375c5b0740 1 --2- 172.21.15.97:0/31837776 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5582a96a3980 0x5582a96add40 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3744009250 tx=0x7f374402bc00 comp rx=0 tx=0).stop 2024-03-27T11:01:41.658 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.686+0000 7f375c5b0740 1 -- 172.21.15.97:0/31837776 shutdown_connections 2024-03-27T11:01:41.658 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.686+0000 7f375c5b0740 1 --2- 172.21.15.97:0/31837776 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5582a96cab00 0x5582a96ccf30 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:41.658 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.686+0000 7f375c5b0740 1 --2- 172.21.15.97:0/31837776 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5582a96a94e0 0x5582a96b52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:41.658 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.686+0000 7f375c5b0740 1 --2- 172.21.15.97:0/31837776 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5582a96a4370 0x5582a96ae280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:41.658 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.686+0000 7f375c5b0740 1 --2- 172.21.15.97:0/31837776 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5582a96a3980 0x5582a96add40 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:41.658 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.686+0000 7f375c5b0740 1 -- 172.21.15.97:0/31837776 >> 172.21.15.97:0/31837776 conn(0x5582a969eef0 msgr2=0x5582a969e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:01:41.658 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.686+0000 7f373cc00640 0 client.36200 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T11:01:41.658 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.687+0000 7f375c5b0740 1 -- 172.21.15.97:0/31837776 shutdown_connections 2024-03-27T11:01:41.658 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.687+0000 7f375c5b0740 1 -- 172.21.15.97:0/31837776 wait complete. 2024-03-27T11:01:41.659 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.687+0000 7f375c5b0740 20 client.36200 trim_cache size 0 max 16384 2024-03-27T11:01:41.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:01:41 smithi177 ceph-mon[35294]: pgmap v592: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 61 MiB/s wr, 980 op/s 2024-03-27T11:01:41.687 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.717+0000 7f375c5b0740 1 Processor -- start 2024-03-27T11:01:41.689 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.717+0000 7f375c5b0740 1 -- start start 2024-03-27T11:01:41.689 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.717+0000 7f375c5b0740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5582a96a3980 0x5582a969d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:01:41.689 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.717+0000 7f375c5b0740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5582a96a4370 0x5582a96b5280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:01:41.689 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.717+0000 7f375c5b0740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5582a96a94e0 0x5582a969dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:01:41.689 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.717+0000 7f375c5b0740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3754108c10 con 0x5582a96a3980 2024-03-27T11:01:41.689 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.717+0000 7f375c5b0740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f375410a750 con 0x5582a96a94e0 2024-03-27T11:01:41.689 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.717+0000 7f375c5b0740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f37541092e0 con 0x5582a96a4370 2024-03-27T11:01:41.689 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.718+0000 7f3753400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5582a96a94e0 0x5582a969dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:01:41.689 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.718+0000 7f3752000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5582a96a4370 0x5582a96b5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:01:41.689 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.718+0000 7f3752000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5582a96a4370 0x5582a96b5280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.97:34240/0 (socket says 172.21.15.97:34240) 2024-03-27T11:01:41.689 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.718+0000 7f3752000640 1 -- 172.21.15.97:0/985311380 learned_addr learned my addr 172.21.15.97:0/985311380 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:01:41.690 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.718+0000 7f3752a00640 1 --2- 172.21.15.97:0/985311380 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5582a96a3980 0x5582a969d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:01:41.691 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.718+0000 7f3753400640 1 -- 172.21.15.97:0/985311380 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5582a96a4370 msgr2=0x5582a96b5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:01:41.691 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.718+0000 7f3753400640 1 --2- 172.21.15.97:0/985311380 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5582a96a4370 0x5582a96b5280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:41.691 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.718+0000 7f3753400640 1 -- 172.21.15.97:0/985311380 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5582a96a3980 msgr2=0x5582a969d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:01:41.691 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.718+0000 7f3753400640 1 --2- 172.21.15.97:0/985311380 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5582a96a3980 0x5582a969d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:41.691 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.718+0000 7f3753400640 1 -- 172.21.15.97:0/985311380 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582a93e9620 con 0x5582a96a94e0 2024-03-27T11:01:41.691 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.719+0000 7f3752000640 1 --2- 172.21.15.97:0/985311380 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5582a96a4370 0x5582a96b5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T11:01:41.691 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.719+0000 7f3753400640 1 --2- 172.21.15.97:0/985311380 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5582a96a94e0 0x5582a969dda0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f374c00e580 tx=0x7f374c009c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b74580ec0b12d166 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:01:41.691 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.719+0000 7f373f400640 1 -- 172.21.15.97:0/985311380 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f374c003970 con 0x5582a96a94e0 2024-03-27T11:01:41.691 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.719+0000 7f373f400640 1 -- 172.21.15.97:0/985311380 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f374c027030 con 0x5582a96a94e0 2024-03-27T11:01:41.691 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.719+0000 7f373f400640 1 -- 172.21.15.97:0/985311380 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f374c005370 con 0x5582a96a94e0 2024-03-27T11:01:41.692 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.720+0000 7f375c5b0740 1 -- 172.21.15.97:0/985311380 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5582a96a94e0 msgr2=0x5582a969dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:01:41.692 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.720+0000 7f375c5b0740 1 --2- 172.21.15.97:0/985311380 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5582a96a94e0 0x5582a969dda0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f374c00e580 tx=0x7f374c009c10 comp rx=0 tx=0).stop 2024-03-27T11:01:41.692 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.720+0000 7f375c5b0740 1 -- 172.21.15.97:0/985311380 shutdown_connections 2024-03-27T11:01:41.693 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.720+0000 7f375c5b0740 1 --2- 172.21.15.97:0/985311380 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5582a96a94e0 0x5582a969dda0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:41.693 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.720+0000 7f375c5b0740 1 --2- 172.21.15.97:0/985311380 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5582a96a4370 0x5582a96b5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:41.693 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.720+0000 7f375c5b0740 1 --2- 172.21.15.97:0/985311380 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5582a96a3980 0x5582a969d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:41.693 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.720+0000 7f375c5b0740 1 -- 172.21.15.97:0/985311380 >> 172.21.15.97:0/985311380 conn(0x5582a969eef0 msgr2=0x5582a96ad950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:01:41.693 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.720+0000 7f375c5b0740 1 -- 172.21.15.97:0/985311380 shutdown_connections 2024-03-27T11:01:41.693 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.721+0000 7f375c5b0740 1 -- 172.21.15.97:0/985311380 wait complete. 2024-03-27T11:01:41.693 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.721+0000 7f375c5b0740 1 -- start start 2024-03-27T11:01:41.693 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.721+0000 7f375c5b0740 1 Processor -- start 2024-03-27T11:01:41.694 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.722+0000 7f375c5b0740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5582a96a3980 0x5582a96b2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:01:41.694 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.722+0000 7f375c5b0740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5582a96a4370 0x5582a96b32e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:01:41.694 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.722+0000 7f375c5b0740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5582a96a94e0 0x5582a96b42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:01:41.695 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.722+0000 7f375c5b0740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f375410a080 con 0x5582a96a4370 2024-03-27T11:01:41.695 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.722+0000 7f375c5b0740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f37541099b0 con 0x5582a96a3980 2024-03-27T11:01:41.695 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.722+0000 7f375c5b0740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f3754072d20 con 0x5582a96a94e0 2024-03-27T11:01:41.695 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.722+0000 7f375c5b0740 10 client.0 _ll_register_callbacks cb 0x7f3740005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T11:01:41.695 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.722+0000 7f3752a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5582a96a3980 0x5582a96b2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:01:41.695 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.722+0000 7f3752a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5582a96a3980 0x5582a96b2da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:47254/0 (socket says 172.21.15.97:47254) 2024-03-27T11:01:41.695 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.722+0000 7f3752a00640 1 -- 172.21.15.97:0/458919158 learned_addr learned my addr 172.21.15.97:0/458919158 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:01:41.695 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.722+0000 7f3752000640 1 --2- 172.21.15.97:0/458919158 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5582a96a4370 0x5582a96b32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:01:41.695 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.722+0000 7f3753400640 1 --2- 172.21.15.97:0/458919158 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5582a96a94e0 0x5582a96b42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:01:41.695 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.723+0000 7f3752000640 1 -- 172.21.15.97:0/458919158 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5582a96a94e0 msgr2=0x5582a96b42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:01:41.695 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.723+0000 7f3752000640 1 --2- 172.21.15.97:0/458919158 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5582a96a94e0 0x5582a96b42a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:41.695 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.723+0000 7f3752000640 1 -- 172.21.15.97:0/458919158 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5582a96a3980 msgr2=0x5582a96b2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:01:41.696 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.724+0000 7f3752000640 1 --2- 172.21.15.97:0/458919158 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5582a96a3980 0x5582a96b2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:41.696 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.724+0000 7f3752000640 1 -- 172.21.15.97:0/458919158 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582a96b4890 con 0x5582a96a4370 2024-03-27T11:01:41.696 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.724+0000 7f3752a00640 1 --2- 172.21.15.97:0/458919158 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5582a96a3980 0x5582a96b2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T11:01:41.696 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.724+0000 7f3752000640 1 --2- 172.21.15.97:0/458919158 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5582a96a4370 0x5582a96b32e0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f3740098990 tx=0x7f3740000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4728abda61dc926 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:01:41.696 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.724+0000 7f373cc00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T11:01:41.697 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.725+0000 7f373cc00640 1 -- 172.21.15.97:0/458919158 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f374009cc30 con 0x5582a96a4370 2024-03-27T11:01:41.697 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.725+0000 7f375c5b0740 1 -- 172.21.15.97:0/458919158 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f374c027030 con 0x5582a96a4370 2024-03-27T11:01:41.697 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.725+0000 7f373cc00640 1 -- 172.21.15.97:0/458919158 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f3740010a90 con 0x5582a96a4370 2024-03-27T11:01:41.697 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.726+0000 7f373cc00640 1 -- 172.21.15.97:0/458919158 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f374009c530 con 0x5582a96a4370 2024-03-27T11:01:41.697 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.727+0000 7f375c5b0740 10 client.16545 fetch_fsmap learned FSMap version 34 2024-03-27T11:01:41.697 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.727+0000 7f375c5b0740 1 -- 172.21.15.97:0/458919158 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5582a9731360 con 0x5582a96a4370 2024-03-27T11:01:41.698 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.727+0000 7f373cc00640 1 -- 172.21.15.97:0/458919158 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f374c027030 con 0x5582a96a4370 2024-03-27T11:01:41.698 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.727+0000 7f373cc00640 1 -- 172.21.15.97:0/458919158 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f37400d20b0 con 0x5582a96a4370 2024-03-27T11:01:41.699 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.728+0000 7f375c5b0740 10 client.16545 fetch_fsmap finished waiting for FSMap version 34 2024-03-27T11:01:41.699 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.728+0000 7f375c5b0740 10 client.16545 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T11:01:41.699 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.728+0000 7f375c5b0740 20 client.16545 populate_metadata read hostname 'smithi097' 2024-03-27T11:01:41.699 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.728+0000 7f375c5b0740 1 --2- 172.21.15.97:0/458919158 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5582a9731d60 0x5582a9734180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:01:41.699 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.728+0000 7f375c5b0740 4 client.16545 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-27T11:01:41.699 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.728+0000 7f375c5b0740 1 -- 172.21.15.97:0/458919158 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5582a96105a0 con 0x5582a9731d60 2024-03-27T11:01:41.700 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.728+0000 7f3752a00640 1 --2- 172.21.15.97:0/458919158 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5582a9731d60 0x5582a9734180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:01:41.700 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.729+0000 7f373cc00640 10 client.16545 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T11:01:41.700 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.729+0000 7f3752a00640 1 --2- 172.21.15.97:0/458919158 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5582a9731d60 0x5582a9734180 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4457204856b57982 server_cookie=142f3ef182de1caa in_seq=0 out_seq=0 2024-03-27T11:01:41.704 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.734+0000 7f373cc00640 1 -- 172.21.15.97:0/458919158 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5582a96105a0 con 0x5582a9731d60 2024-03-27T11:01:41.704 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.734+0000 7f373cc00640 10 client.16545 handle_command_reply: tid=0 2024-03-27T11:01:41.705 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.734+0000 7f375c5b0740 1 client.16545 shutdown 2024-03-27T11:01:41.705 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.734+0000 7f375c5b0740 1 -- 172.21.15.97:0/458919158 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5582a96a4370 msgr2=0x5582a96b32e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:01:41.705 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.734+0000 7f375c5b0740 1 --2- 172.21.15.97:0/458919158 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5582a96a4370 0x5582a96b32e0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f3740098990 tx=0x7f3740000bc0 comp rx=0 tx=0).stop 2024-03-27T11:01:41.705 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.734+0000 7f375c5b0740 1 -- 172.21.15.97:0/458919158 shutdown_connections 2024-03-27T11:01:41.706 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.734+0000 7f375c5b0740 1 --2- 172.21.15.97:0/458919158 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5582a9731d60 0x5582a9734180 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:41.707 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.734+0000 7f375c5b0740 1 --2- 172.21.15.97:0/458919158 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5582a96a94e0 0x5582a96b42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:41.707 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.734+0000 7f375c5b0740 1 --2- 172.21.15.97:0/458919158 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5582a96a4370 0x5582a96b32e0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:41.707 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.734+0000 7f375c5b0740 1 --2- 172.21.15.97:0/458919158 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5582a96a3980 0x5582a96b2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:41.707 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.734+0000 7f375c5b0740 1 -- 172.21.15.97:0/458919158 >> 172.21.15.97:0/458919158 conn(0x5582a969eef0 msgr2=0x5582a96a2150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:01:41.707 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.734+0000 7f373cc00640 0 client.16545 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T11:01:41.707 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.735+0000 7f375c5b0740 1 -- 172.21.15.97:0/458919158 shutdown_connections 2024-03-27T11:01:41.707 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.736+0000 7f375c5b0740 1 -- 172.21.15.97:0/458919158 wait complete. 2024-03-27T11:01:41.707 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.736+0000 7f375c5b0740 20 client.16545 trim_cache size 0 max 16384 2024-03-27T11:01:41.708 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-27T11:01:41.708 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-27T11:01:41.708 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-27T11:01:41.708 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-27T11:01:41.708 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.737+0000 7f3759800640 1 -- 172.21.15.97:0/3091128136 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f37380a3780 msgr2=0x7f37380a5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:01:41.709 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.737+0000 7f3759800640 1 --2- 172.21.15.97:0/3091128136 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f37380a3780 0x7f37380a5c40 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f374402c190 tx=0x7f374402e040 comp rx=0 tx=0).stop 2024-03-27T11:01:41.709 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.737+0000 7f3759800640 1 -- 172.21.15.97:0/3091128136 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f37541035f0 msgr2=0x7f375419ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:01:41.709 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.737+0000 7f3759800640 1 --2- 172.21.15.97:0/3091128136 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f37541035f0 0x7f375419ea90 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f374000e9f0 tx=0x7f374000eeb0 comp rx=0 tx=0).stop 2024-03-27T11:01:41.709 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.737+0000 7f3759800640 1 -- 172.21.15.97:0/3091128136 shutdown_connections 2024-03-27T11:01:41.709 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.737+0000 7f3759800640 1 --2- 172.21.15.97:0/3091128136 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37541a3af0 0x7f37541a5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:41.709 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.737+0000 7f3759800640 1 --2- 172.21.15.97:0/3091128136 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f37541035f0 0x7f375419ea90 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:41.709 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.737+0000 7f3759800640 1 --2- 172.21.15.97:0/3091128136 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f37380a3780 0x7f37380a5c40 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:41.709 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.737+0000 7f3759800640 1 --2- 172.21.15.97:0/3091128136 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3754074550 0x7f375419e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:41.709 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.737+0000 7f3759800640 1 -- 172.21.15.97:0/3091128136 >> 172.21.15.97:0/3091128136 conn(0x7f37540fd110 msgr2=0x7f3754106be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:01:41.709 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.738+0000 7f3759800640 1 -- 172.21.15.97:0/3091128136 shutdown_connections 2024-03-27T11:01:41.710 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:41.738+0000 7f3759800640 1 -- 172.21.15.97:0/3091128136 wait complete. 2024-03-27T11:01:41.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:01:41 smithi114 ceph-mon[35338]: pgmap v592: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 61 MiB/s wr, 980 op/s 2024-03-27T11:01:43.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:01:43 smithi097 ceph-mon[31859]: pgmap v593: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 55 MiB/s wr, 881 op/s 2024-03-27T11:01:43.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:01:43 smithi177 ceph-mon[35294]: pgmap v593: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 55 MiB/s wr, 881 op/s 2024-03-27T11:01:43.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:01:43 smithi114 ceph-mon[35338]: pgmap v593: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 55 MiB/s wr, 881 op/s 2024-03-27T11:01:45.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:01:45 smithi097 ceph-mon[31859]: pgmap v594: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 55 MiB/s wr, 895 op/s 2024-03-27T11:01:45.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:01:45 smithi177 ceph-mon[35294]: pgmap v594: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 55 MiB/s wr, 895 op/s 2024-03-27T11:01:45.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:01:45 smithi114 ceph-mon[35338]: pgmap v594: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 55 MiB/s wr, 895 op/s 2024-03-27T11:01:46.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:01:46 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T11:01:46.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:01:46 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T11:01:46.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:01:46 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T11:01:47.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:01:47 smithi097 ceph-mon[31859]: pgmap v595: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 53 MiB/s wr, 854 op/s 2024-03-27T11:01:47.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:01:47 smithi177 ceph-mon[35294]: pgmap v595: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 53 MiB/s wr, 854 op/s 2024-03-27T11:01:47.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:01:47 smithi114 ceph-mon[35338]: pgmap v595: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 53 MiB/s wr, 854 op/s 2024-03-27T11:01:48.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:01:48 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T11:01:48.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:01:48 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T11:01:48.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:01:48 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T11:01:48.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:01:48 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T11:01:48.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:01:48 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T11:01:48.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:01:48 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T11:01:48.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:01:48 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T11:01:48.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:01:48 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T11:01:48.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:01:48 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T11:01:49.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:01:49 smithi097 ceph-mon[31859]: pgmap v596: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 56 MiB/s wr, 906 op/s 2024-03-27T11:01:49.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:01:49 smithi177 ceph-mon[35294]: pgmap v596: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 56 MiB/s wr, 906 op/s 2024-03-27T11:01:49.766 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:01:49 smithi114 ceph-mon[35338]: pgmap v596: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 56 MiB/s wr, 906 op/s 2024-03-27T11:01:51.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:01:51 smithi097 ceph-mon[31859]: pgmap v597: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 61 MiB/s wr, 982 op/s 2024-03-27T11:01:51.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:01:51 smithi177 ceph-mon[35294]: pgmap v597: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 61 MiB/s wr, 982 op/s 2024-03-27T11:01:51.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:01:51 smithi114 ceph-mon[35338]: pgmap v597: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 61 MiB/s wr, 982 op/s 2024-03-27T11:01:53.641 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:01:53 smithi097 ceph-mon[31859]: pgmap v598: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 55 MiB/s wr, 882 op/s 2024-03-27T11:01:53.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:01:53 smithi177 ceph-mon[35294]: pgmap v598: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 55 MiB/s wr, 882 op/s 2024-03-27T11:01:53.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:01:53 smithi114 ceph-mon[35338]: pgmap v598: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 55 MiB/s wr, 882 op/s 2024-03-27T11:01:54.915 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.943+0000 7fd0bc200640 1 -- 172.21.15.97:0/892778306 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd0b41094c0 msgr2=0x7fd0b4109950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:01:54.915 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.943+0000 7fd0bc200640 1 --2- 172.21.15.97:0/892778306 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd0b41094c0 0x7fd0b4109950 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac009ca0 tx=0x7fd0ac02c100 comp rx=0 tx=0).stop 2024-03-27T11:01:54.915 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.944+0000 7fd0bc200640 1 -- 172.21.15.97:0/892778306 shutdown_connections 2024-03-27T11:01:54.915 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.944+0000 7fd0bc200640 1 --2- 172.21.15.97:0/892778306 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd0b41094c0 0x7fd0b4109950 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:54.915 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.944+0000 7fd0bc200640 1 --2- 172.21.15.97:0/892778306 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0b410c770 0x7fd0b410eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:54.916 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.944+0000 7fd0bc200640 1 --2- 172.21.15.97:0/892778306 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd0b4072d20 0x7fd0b4108dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:54.916 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.944+0000 7fd0bc200640 1 -- 172.21.15.97:0/892778306 >> 172.21.15.97:0/892778306 conn(0x7fd0b406eb90 msgr2=0x7fd0b4070fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:01:54.916 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.944+0000 7fd0bc200640 1 -- 172.21.15.97:0/892778306 shutdown_connections 2024-03-27T11:01:54.916 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.944+0000 7fd0bc200640 1 -- 172.21.15.97:0/892778306 wait complete. 2024-03-27T11:01:54.916 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.946+0000 7fd0bc200640 1 Processor -- start 2024-03-27T11:01:54.916 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.946+0000 7fd0bc200640 1 -- start start 2024-03-27T11:01:54.917 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.946+0000 7fd0bc200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd0b4072d20 0x7fd0b419e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:01:54.917 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.946+0000 7fd0bc200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd0b41094c0 0x7fd0b419e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:01:54.918 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.946+0000 7fd0bc200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0b410c770 0x7fd0b41a5890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:01:54.918 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.947+0000 7fd0bc200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd0b4113cb0 con 0x7fd0b410c770 2024-03-27T11:01:54.918 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.947+0000 7fd0bc200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd0b4113b30 con 0x7fd0b4072d20 2024-03-27T11:01:54.918 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.947+0000 7fd0bc200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd0b4113e30 con 0x7fd0b41094c0 2024-03-27T11:01:54.919 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.947+0000 7fd0b9800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd0b4072d20 0x7fd0b419e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:01:54.919 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.947+0000 7fd0b9800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd0b4072d20 0x7fd0b419e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:53294/0 (socket says 172.21.15.97:53294) 2024-03-27T11:01:54.919 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.947+0000 7fd0b9800640 1 -- 172.21.15.97:0/3717287280 learned_addr learned my addr 172.21.15.97:0/3717287280 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:01:54.919 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.947+0000 7fd0b8e00640 1 --2- 172.21.15.97:0/3717287280 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd0b41094c0 0x7fd0b419e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:01:54.919 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.947+0000 7fd0ba200640 1 --2- 172.21.15.97:0/3717287280 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0b410c770 0x7fd0b41a5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:01:54.920 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.948+0000 7fd0b9800640 1 -- 172.21.15.97:0/3717287280 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd0b41094c0 msgr2=0x7fd0b419e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:01:54.920 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.948+0000 7fd0b9800640 1 --2- 172.21.15.97:0/3717287280 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd0b41094c0 0x7fd0b419e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:54.920 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.948+0000 7fd0b9800640 1 -- 172.21.15.97:0/3717287280 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0b410c770 msgr2=0x7fd0b41a5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:01:54.920 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.948+0000 7fd0b9800640 1 --2- 172.21.15.97:0/3717287280 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0b410c770 0x7fd0b41a5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:54.920 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.948+0000 7fd0b9800640 1 -- 172.21.15.97:0/3717287280 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0b41a5f00 con 0x7fd0b4072d20 2024-03-27T11:01:54.920 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.948+0000 7fd0ba200640 1 --2- 172.21.15.97:0/3717287280 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0b410c770 0x7fd0b41a5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T11:01:54.920 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.949+0000 7fd0b9800640 1 --2- 172.21.15.97:0/3717287280 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd0b4072d20 0x7fd0b419e2e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd0a8011a60 tx=0x7fd0a8011f20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:01:54.921 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.949+0000 7fd0b2000640 1 -- 172.21.15.97:0/3717287280 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a8004220 con 0x7fd0b4072d20 2024-03-27T11:01:54.921 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.949+0000 7fd0bc200640 1 -- 172.21.15.97:0/3717287280 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0b41a61f0 con 0x7fd0b4072d20 2024-03-27T11:01:54.921 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.950+0000 7fd0b2000640 1 -- 172.21.15.97:0/3717287280 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd0a80043c0 con 0x7fd0b4072d20 2024-03-27T11:01:54.921 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.950+0000 7fd0b2000640 1 -- 172.21.15.97:0/3717287280 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a8019400 con 0x7fd0b4072d20 2024-03-27T11:01:54.921 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.950+0000 7fd0bc200640 1 -- 172.21.15.97:0/3717287280 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0b41a6790 con 0x7fd0b4072d20 2024-03-27T11:01:54.930 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.958+0000 7fd0bee42740 1 Processor -- start 2024-03-27T11:01:54.930 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.958+0000 7fd0b2000640 1 -- 172.21.15.97:0/3717287280 <== mon.2 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7fd0a800f9c0 con 0x7fd0b4072d20 2024-03-27T11:01:54.930 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.959+0000 7fd0bee42740 1 -- start start 2024-03-27T11:01:54.931 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.960+0000 7fd0bee42740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c0445da190 0x55c0445da5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:01:54.931 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.960+0000 7fd0bee42740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c0445dab80 0x55c0445df680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:01:54.931 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.960+0000 7fd0bee42740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c0445dfcf0 0x55c0445e2180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:01:54.931 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.960+0000 7fd0bee42740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55c044536d10 con 0x55c0445dab80 2024-03-27T11:01:54.931 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.960+0000 7fd0bee42740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55c0444df260 con 0x55c0445dfcf0 2024-03-27T11:01:54.931 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.960+0000 7fd0bee42740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55c044532cc0 con 0x55c0445da190 2024-03-27T11:01:54.933 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.960+0000 7fd0b9800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c0445dab80 0x55c0445df680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:01:54.933 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.960+0000 7fd0b9800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c0445dab80 0x55c0445df680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37884/0 (socket says 172.21.15.97:37884) 2024-03-27T11:01:54.933 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.960+0000 7fd0b9800640 1 -- 172.21.15.97:0/3506991877 learned_addr learned my addr 172.21.15.97:0/3506991877 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:01:54.933 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.960+0000 7fd0ba200640 1 --2- 172.21.15.97:0/3506991877 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c0445da190 0x55c0445da5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:01:54.933 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.960+0000 7fd0b2000640 1 --2- 172.21.15.97:0/3717287280 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fd09c0a3800 0x7fd09c0a5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:01:54.933 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.961+0000 7fd0b8e00640 1 --2- 172.21.15.97:0/3506991877 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c0445dfcf0 0x55c0445e2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:01:54.933 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.961+0000 7fd0b2000640 1 -- 172.21.15.97:0/3717287280 <== mon.2 v2:172.21.15.114:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fd0a80d34b0 con 0x7fd0b4072d20 2024-03-27T11:01:54.933 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.961+0000 7fd0ba200640 1 --2- 172.21.15.97:0/3717287280 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fd09c0a3800 0x7fd09c0a5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:01:54.933 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.961+0000 7fd0b9800640 1 -- 172.21.15.97:0/3506991877 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c0445da190 msgr2=0x55c0445da5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:01:54.933 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.961+0000 7fd0b9800640 1 --2- 172.21.15.97:0/3506991877 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c0445da190 0x55c0445da5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:54.933 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.961+0000 7fd0b9800640 1 -- 172.21.15.97:0/3506991877 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c0445dfcf0 msgr2=0x55c0445e2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:01:54.933 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.961+0000 7fd0b9800640 1 --2- 172.21.15.97:0/3506991877 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c0445dfcf0 0x55c0445e2180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:54.934 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.961+0000 7fd0b9800640 1 -- 172.21.15.97:0/3506991877 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0444d6aa0 con 0x55c0445dab80 2024-03-27T11:01:54.934 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.961+0000 7fd0b9800640 1 --2- 172.21.15.97:0/3506991877 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c0445dab80 0x55c0445df680 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fd0a809f970 tx=0x7fd0a8025000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ba583fed0641322 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:01:54.934 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.962+0000 7fd0ba200640 1 --2- 172.21.15.97:0/3717287280 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fd09c0a3800 0x7fd09c0a5cc0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac032380 tx=0x7fd0ac02c100 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:01:54.934 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.962+0000 7fd0a6a00640 1 -- 172.21.15.97:0/3506991877 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a809f340 con 0x55c0445dab80 2024-03-27T11:01:54.934 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.962+0000 7fd0a6a00640 1 -- 172.21.15.97:0/3506991877 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd0a8003680 con 0x55c0445dab80 2024-03-27T11:01:54.934 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.962+0000 7fd0a6a00640 1 -- 172.21.15.97:0/3506991877 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a800fdd0 con 0x55c0445dab80 2024-03-27T11:01:54.935 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.963+0000 7fd0bee42740 1 -- 172.21.15.97:0/3506991877 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c0445dab80 msgr2=0x55c0445df680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:01:54.935 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.963+0000 7fd0bee42740 1 --2- 172.21.15.97:0/3506991877 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c0445dab80 0x55c0445df680 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fd0a809f970 tx=0x7fd0a8025000 comp rx=0 tx=0).stop 2024-03-27T11:01:54.935 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.963+0000 7fd0bee42740 1 -- 172.21.15.97:0/3506991877 shutdown_connections 2024-03-27T11:01:54.935 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.963+0000 7fd0bee42740 1 --2- 172.21.15.97:0/3506991877 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c0445dfcf0 0x55c0445e2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:54.935 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.963+0000 7fd0bee42740 1 --2- 172.21.15.97:0/3506991877 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c0445dab80 0x55c0445df680 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:54.935 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.963+0000 7fd0bee42740 1 --2- 172.21.15.97:0/3506991877 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c0445da190 0x55c0445da5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:54.935 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.963+0000 7fd0bee42740 1 -- 172.21.15.97:0/3506991877 >> 172.21.15.97:0/3506991877 conn(0x55c0445d5700 msgr2=0x55c0445d5b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:01:54.935 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.963+0000 7fd0bee42740 1 -- 172.21.15.97:0/3506991877 shutdown_connections 2024-03-27T11:01:54.935 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.963+0000 7fd0bee42740 1 -- 172.21.15.97:0/3506991877 wait complete. 2024-03-27T11:01:54.936 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.964+0000 7fd0bee42740 1 -- start start 2024-03-27T11:01:54.936 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.964+0000 7fd0bee42740 1 Processor -- start 2024-03-27T11:01:54.939 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.966+0000 7fd0bee42740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c0445da190 0x55c0445e44d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:01:54.939 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.966+0000 7fd0bee42740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c0445dab80 0x55c0445e4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:01:54.939 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.966+0000 7fd0bee42740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c0445dfcf0 0x55c0445eb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:01:54.939 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.966+0000 7fd0bee42740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55c0445e2be0 con 0x55c0445dab80 2024-03-27T11:01:54.939 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.966+0000 7fd0bee42740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55c0444f2630 con 0x55c0445dfcf0 2024-03-27T11:01:54.939 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.966+0000 7fd0bee42740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55c0445e2d60 con 0x55c0445da190 2024-03-27T11:01:54.939 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.966+0000 7fd0bee42740 10 client.0 _ll_register_callbacks cb 0x55c0444f7c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T11:01:54.939 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.966+0000 7fd0b8e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c0445dab80 0x55c0445e4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:01:54.939 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.966+0000 7fd0ba200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c0445dfcf0 0x55c0445eb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:01:54.939 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.966+0000 7fd0b8e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c0445dab80 0x55c0445e4a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37898/0 (socket says 172.21.15.97:37898) 2024-03-27T11:01:54.940 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.966+0000 7fd0b8e00640 1 -- 172.21.15.97:0/1861250026 learned_addr learned my addr 172.21.15.97:0/1861250026 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:01:54.940 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.966+0000 7fd0b9800640 1 --2- 172.21.15.97:0/1861250026 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c0445da190 0x55c0445e44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:01:54.940 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.967+0000 7fd0ba200640 1 -- 172.21.15.97:0/1861250026 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c0445da190 msgr2=0x55c0445e44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:01:54.940 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.967+0000 7fd0ba200640 1 --2- 172.21.15.97:0/1861250026 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c0445da190 0x55c0445e44d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:54.940 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.967+0000 7fd0ba200640 1 -- 172.21.15.97:0/1861250026 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c0445dab80 msgr2=0x55c0445e4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:01:54.940 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.967+0000 7fd0ba200640 1 --2- 172.21.15.97:0/1861250026 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c0445dab80 0x55c0445e4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:54.940 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.967+0000 7fd0ba200640 1 -- 172.21.15.97:0/1861250026 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0444fe5f0 con 0x55c0445dfcf0 2024-03-27T11:01:54.940 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.967+0000 7fd0b8e00640 1 --2- 172.21.15.97:0/1861250026 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c0445dab80 0x55c0445e4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T11:01:54.940 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.968+0000 7fd0ba200640 1 --2- 172.21.15.97:0/1861250026 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c0445dfcf0 0x55c0445eb9e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac009bf0 tx=0x7fd0ac0095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=837616a46dbac9cb server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:01:54.940 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.968+0000 7fd09b400640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-03-27T11:01:54.940 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.968+0000 7fd09b400640 1 -- 172.21.15.97:0/1861250026 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0ac053060 con 0x55c0445dfcf0 2024-03-27T11:01:54.941 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.968+0000 7fd09b400640 1 -- 172.21.15.97:0/1861250026 <== mon.2 v2:172.21.15.114:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fd0ac044610 con 0x55c0445dfcf0 2024-03-27T11:01:54.942 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.968+0000 7fd0bee42740 1 -- 172.21.15.97:0/1861250026 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c044544250 con 0x55c0445dfcf0 2024-03-27T11:01:54.942 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.968+0000 7fd09b400640 1 -- 172.21.15.97:0/1861250026 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fd0ac039040 con 0x55c0445dfcf0 2024-03-27T11:01:54.942 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.969+0000 7fd0bee42740 10 client.36218 fetch_fsmap learned FSMap version 34 2024-03-27T11:01:54.942 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.969+0000 7fd0bee42740 1 -- 172.21.15.97:0/1861250026 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55c04424b210 con 0x55c0445dfcf0 2024-03-27T11:01:54.942 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.969+0000 7fd09b400640 1 -- 172.21.15.97:0/1861250026 <== mon.2 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55c044544250 con 0x55c0445dfcf0 2024-03-27T11:01:54.942 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.969+0000 7fd09b400640 1 -- 172.21.15.97:0/1861250026 <== mon.2 v2:172.21.15.114:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7fd0ac03f070 con 0x55c0445dfcf0 2024-03-27T11:01:54.942 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.969+0000 7fd0bee42740 10 client.36218 fetch_fsmap finished waiting for FSMap version 34 2024-03-27T11:01:54.942 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.969+0000 7fd0bee42740 10 client.36218 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T11:01:54.942 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.969+0000 7fd0bee42740 20 client.36218 populate_metadata read hostname 'smithi097' 2024-03-27T11:01:54.942 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.969+0000 7fd0bee42740 1 --2- 172.21.15.97:0/1861250026 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55c0445f0210 0x55c0445f2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:01:54.942 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.969+0000 7fd0bee42740 4 client.36218 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T11:01:54.942 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.969+0000 7fd0bee42740 1 -- 172.21.15.97:0/1861250026 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c04453e5a0 con 0x55c0445f0210 2024-03-27T11:01:54.943 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.970+0000 7fd0b9800640 1 --2- 172.21.15.97:0/1861250026 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55c0445f0210 0x55c0445f2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:01:54.943 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.970+0000 7fd0b9800640 1 --2- 172.21.15.97:0/1861250026 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55c0445f0210 0x55c0445f2620 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c09bd2206994734e server_cookie=a50fe555d8f5c1b7 in_seq=0 out_seq=0 2024-03-27T11:01:54.943 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.970+0000 7fd09b400640 10 client.36218 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T11:01:54.943 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.972+0000 7fd09b400640 1 -- 172.21.15.97:0/1861250026 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55c04453e5a0 con 0x55c0445f0210 2024-03-27T11:01:54.943 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.972+0000 7fd09b400640 10 client.36218 handle_command_reply: tid=0 2024-03-27T11:01:54.944 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.972+0000 7fd0bee42740 1 client.36218 shutdown 2024-03-27T11:01:54.944 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.973+0000 7fd0bee42740 1 -- 172.21.15.97:0/1861250026 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c0445dfcf0 msgr2=0x55c0445eb9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:01:54.944 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.973+0000 7fd0bee42740 1 --2- 172.21.15.97:0/1861250026 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c0445dfcf0 0x55c0445eb9e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac009bf0 tx=0x7fd0ac0095c0 comp rx=0 tx=0).stop 2024-03-27T11:01:54.944 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.973+0000 7fd0bee42740 1 -- 172.21.15.97:0/1861250026 shutdown_connections 2024-03-27T11:01:54.944 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.973+0000 7fd0bee42740 1 --2- 172.21.15.97:0/1861250026 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55c0445f0210 0x55c0445f2620 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:54.944 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.973+0000 7fd0bee42740 1 --2- 172.21.15.97:0/1861250026 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c0445dfcf0 0x55c0445eb9e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:54.944 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.973+0000 7fd0bee42740 1 --2- 172.21.15.97:0/1861250026 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c0445dab80 0x55c0445e4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:54.944 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.973+0000 7fd0bee42740 1 --2- 172.21.15.97:0/1861250026 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c0445da190 0x55c0445e44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:54.944 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.973+0000 7fd0bee42740 1 -- 172.21.15.97:0/1861250026 >> 172.21.15.97:0/1861250026 conn(0x55c0445d5700 msgr2=0x55c0445cc0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:01:54.945 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.973+0000 7fd09b400640 0 client.36218 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T11:01:54.945 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.974+0000 7fd0bee42740 1 -- 172.21.15.97:0/1861250026 shutdown_connections 2024-03-27T11:01:54.945 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.974+0000 7fd0bee42740 1 -- 172.21.15.97:0/1861250026 wait complete. 2024-03-27T11:01:54.945 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:54.974+0000 7fd0bee42740 20 client.36218 trim_cache size 0 max 16384 2024-03-27T11:01:54.971 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:55.000+0000 7fd0bee42740 1 Processor -- start 2024-03-27T11:01:54.971 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:55.000+0000 7fd0bee42740 1 -- start start 2024-03-27T11:01:54.971 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:55.000+0000 7fd0bee42740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c0445da190 0x55c0445dbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:01:54.971 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:55.000+0000 7fd0bee42740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c0445dab80 0x55c0445eb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:01:54.971 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:55.000+0000 7fd0bee42740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c0445dfcf0 0x55c0445dc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:01:54.972 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:55.001+0000 7fd0bee42740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd0b41af5c0 con 0x55c0445da190 2024-03-27T11:01:54.972 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:55.001+0000 7fd0bee42740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd0b41b1100 con 0x55c0445dab80 2024-03-27T11:01:54.973 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:55.001+0000 7fd0bee42740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd0b41afc90 con 0x55c0445dfcf0 2024-03-27T11:01:54.973 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:55.001+0000 7fd0b9800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c0445da190 0x55c0445dbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:01:54.973 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:55.001+0000 7fd0b9800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c0445da190 0x55c0445dbf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37902/0 (socket says 172.21.15.97:37902) 2024-03-27T11:01:54.973 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:55.001+0000 7fd0b9800640 1 -- 172.21.15.97:0/675229004 learned_addr learned my addr 172.21.15.97:0/675229004 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:01:54.973 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:55.001+0000 7fd0b8e00640 1 --2- 172.21.15.97:0/675229004 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c0445dab80 0x55c0445eb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:01:54.973 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:55.001+0000 7fd0b9800640 1 -- 172.21.15.97:0/675229004 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c0445dfcf0 msgr2=0x55c0445dc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:01:54.973 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:55.001+0000 7fd0b9800640 1 --2- 172.21.15.97:0/675229004 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c0445dfcf0 0x55c0445dc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:54.973 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:55.001+0000 7fd0b9800640 1 -- 172.21.15.97:0/675229004 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c0445dab80 msgr2=0x55c0445eb9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:01:54.973 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:55.001+0000 7fd0b9800640 1 --2- 172.21.15.97:0/675229004 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c0445dab80 0x55c0445eb9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:54.973 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:55.001+0000 7fd0b9800640 1 -- 172.21.15.97:0/675229004 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c044317620 con 0x55c0445da190 2024-03-27T11:01:54.974 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:55.002+0000 7fd0b9800640 1 --2- 172.21.15.97:0/675229004 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c0445da190 0x55c0445dbf80 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fd0a809c3b0 tx=0x7fd0a8005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4e274978b22f735 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:01:54.974 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:55.002+0000 7fd0a6000640 1 -- 172.21.15.97:0/675229004 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a809f9e0 con 0x55c0445da190 2024-03-27T11:01:54.974 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:55.002+0000 7fd0a6000640 1 -- 172.21.15.97:0/675229004 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd0a80a85a0 con 0x55c0445da190 2024-03-27T11:01:54.974 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:55.002+0000 7fd0a6000640 1 -- 172.21.15.97:0/675229004 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a809fc40 con 0x55c0445da190 2024-03-27T11:01:54.974 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:55.003+0000 7fd0bee42740 1 -- 172.21.15.97:0/675229004 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c0445da190 msgr2=0x55c0445dbf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:01:54.974 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:55.003+0000 7fd0bee42740 1 --2- 172.21.15.97:0/675229004 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c0445da190 0x55c0445dbf80 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fd0a809c3b0 tx=0x7fd0a8005d30 comp rx=0 tx=0).stop 2024-03-27T11:01:54.975 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:55.003+0000 7fd0bee42740 1 -- 172.21.15.97:0/675229004 shutdown_connections 2024-03-27T11:01:54.975 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:55.003+0000 7fd0bee42740 1 --2- 172.21.15.97:0/675229004 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c0445dfcf0 0x55c0445dc4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:54.975 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:55.004+0000 7fd0bee42740 1 --2- 172.21.15.97:0/675229004 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c0445dab80 0x55c0445eb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:54.975 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:55.004+0000 7fd0bee42740 1 --2- 172.21.15.97:0/675229004 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c0445da190 0x55c0445dbf80 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:54.975 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:55.004+0000 7fd0bee42740 1 -- 172.21.15.97:0/675229004 >> 172.21.15.97:0/675229004 conn(0x55c0445d5700 msgr2=0x55c0445d5ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:01:54.975 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:55.004+0000 7fd0bee42740 1 -- 172.21.15.97:0/675229004 shutdown_connections 2024-03-27T11:01:54.976 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:55.005+0000 7fd0bee42740 1 -- 172.21.15.97:0/675229004 wait complete. 2024-03-27T11:01:54.976 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:55.005+0000 7fd0bee42740 1 -- start start 2024-03-27T11:01:54.976 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:55.005+0000 7fd0bee42740 1 Processor -- start 2024-03-27T11:01:54.978 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:55.007+0000 7fd0bee42740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c0445da190 0x55c0445e9020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:01:54.978 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:55.007+0000 7fd0bee42740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c0445dab80 0x55c0445e9560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:01:54.978 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:55.007+0000 7fd0bee42740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c0445dfcf0 0x55c0445ea500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:01:54.978 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:55.007+0000 7fd0b8e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c0445dab80 0x55c0445e9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:01:54.979 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:55.008+0000 7fd0b8e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c0445dab80 0x55c0445e9560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.97:33314/0 (socket says 172.21.15.97:33314) 2024-03-27T11:01:54.979 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:55.008+0000 7fd0ba200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c0445dfcf0 0x55c0445ea500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:01:54.979 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:55.008+0000 7fd0b9800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c0445da190 0x55c0445e9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:01:54.980 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:55.008+0000 7fd0ba200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c0445dfcf0 0x55c0445ea500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37910/0 (socket says 172.21.15.97:37910) 2024-03-27T11:01:54.980 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:55.008+0000 7fd0bee42740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd0b41b0a30 con 0x55c0445dfcf0 2024-03-27T11:01:54.980 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:55.008+0000 7fd0bee42740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd0b41b0360 con 0x55c0445da190 2024-03-27T11:01:54.980 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:55.008+0000 7fd0bee42740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd0b41ad590 con 0x55c0445dab80 2024-03-27T11:01:54.980 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:55.008+0000 7fd0b8e00640 1 -- 172.21.15.97:0/1265339137 learned_addr learned my addr 172.21.15.97:0/1265339137 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:01:54.980 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:55.008+0000 7fd0b9800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c0445da190 0x55c0445e9020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:53336/0 (socket says 172.21.15.97:53336) 2024-03-27T11:01:54.980 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:55.008+0000 7fd0bee42740 10 client.0 _ll_register_callbacks cb 0x7fd0ac034820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T11:01:54.980 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:55.009+0000 7fd0b8e00640 1 -- 172.21.15.97:0/1265339137 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c0445da190 msgr2=0x55c0445e9020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:01:54.980 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:55.009+0000 7fd0b8e00640 1 --2- 172.21.15.97:0/1265339137 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c0445da190 0x55c0445e9020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:54.980 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:55.009+0000 7fd0b8e00640 1 -- 172.21.15.97:0/1265339137 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c0445dfcf0 msgr2=0x55c0445ea500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:01:54.981 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:55.010+0000 7fd0b8e00640 1 --2- 172.21.15.97:0/1265339137 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c0445dfcf0 0x55c0445ea500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:54.981 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:55.010+0000 7fd0b8e00640 1 -- 172.21.15.97:0/1265339137 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0445eaaf0 con 0x55c0445dab80 2024-03-27T11:01:54.981 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:55.010+0000 7fd0b9800640 1 --2- 172.21.15.97:0/1265339137 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c0445da190 0x55c0445e9020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T11:01:54.981 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:55.010+0000 7fd0ba200640 1 --2- 172.21.15.97:0/1265339137 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c0445dfcf0 0x55c0445ea500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T11:01:54.982 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:55.011+0000 7fd09b400640 10 client.0 ms_handle_connect on v2:172.21.15.177:3300/0 2024-03-27T11:01:54.982 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:55.011+0000 7fd0b8e00640 1 --2- 172.21.15.97:0/1265339137 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c0445dab80 0x55c0445e9560 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x55c0445e5940 tx=0x7fd0a0011d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b901168d08dd76b7 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:01:54.983 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:55.011+0000 7fd09b400640 1 -- 172.21.15.97:0/1265339137 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a0020030 con 0x55c0445dab80 2024-03-27T11:01:54.983 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:55.011+0000 7fd0bee42740 1 -- 172.21.15.97:0/1265339137 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0a80a85a0 con 0x55c0445dab80 2024-03-27T11:01:54.983 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:55.012+0000 7fd09b400640 1 -- 172.21.15.97:0/1265339137 <== mon.1 v2:172.21.15.177:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fd0a0021070 con 0x55c0445dab80 2024-03-27T11:01:54.983 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:55.012+0000 7fd09b400640 1 -- 172.21.15.97:0/1265339137 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fd0a0020030 con 0x55c0445dab80 2024-03-27T11:01:54.983 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:55.012+0000 7fd0bee42740 10 client.25945 fetch_fsmap learned FSMap version 34 2024-03-27T11:01:54.983 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:55.012+0000 7fd0bee42740 1 -- 172.21.15.97:0/1265339137 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55c0446565c0 con 0x55c0445dab80 2024-03-27T11:01:54.983 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:55.012+0000 7fd09b400640 1 -- 172.21.15.97:0/1265339137 <== mon.1 v2:172.21.15.177:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd0a80a85a0 con 0x55c0445dab80 2024-03-27T11:01:54.984 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:55.012+0000 7fd09b400640 1 -- 172.21.15.97:0/1265339137 <== mon.1 v2:172.21.15.177:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7fd0a00048c0 con 0x55c0445dab80 2024-03-27T11:01:54.984 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:55.012+0000 7fd0bee42740 10 client.25945 fetch_fsmap finished waiting for FSMap version 34 2024-03-27T11:01:54.984 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:55.012+0000 7fd0bee42740 10 client.25945 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T11:01:54.984 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:55.012+0000 7fd0bee42740 20 client.25945 populate_metadata read hostname 'smithi097' 2024-03-27T11:01:54.984 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:55.013+0000 7fd0bee42740 1 --2- 172.21.15.97:0/1265339137 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55c044656ff0 0x55c044659410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:01:54.985 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:55.013+0000 7fd0bee42740 4 client.25945 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-27T11:01:54.985 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:55.013+0000 7fd0bee42740 1 -- 172.21.15.97:0/1265339137 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c0444bc630 con 0x55c044656ff0 2024-03-27T11:01:54.985 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:55.013+0000 7fd0b9800640 1 --2- 172.21.15.97:0/1265339137 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55c044656ff0 0x55c044659410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:01:54.985 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:55.014+0000 7fd0b9800640 1 --2- 172.21.15.97:0/1265339137 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55c044656ff0 0x55c044659410 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=820497fb6f1bb076 server_cookie=9844d27bbb9cb394 in_seq=0 out_seq=0 2024-03-27T11:01:54.985 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:55.014+0000 7fd09b400640 10 client.25945 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T11:01:54.991 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:55.020+0000 7fd09b400640 1 -- 172.21.15.97:0/1265339137 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c0444bc630 con 0x55c044656ff0 2024-03-27T11:01:54.991 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:55.020+0000 7fd09b400640 10 client.25945 handle_command_reply: tid=0 2024-03-27T11:01:54.991 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:55.020+0000 7fd0bee42740 1 client.25945 shutdown 2024-03-27T11:01:54.992 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:55.021+0000 7fd0bee42740 1 -- 172.21.15.97:0/1265339137 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c0445dab80 msgr2=0x55c0445e9560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:01:54.992 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:55.021+0000 7fd0bee42740 1 --2- 172.21.15.97:0/1265339137 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c0445dab80 0x55c0445e9560 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x55c0445e5940 tx=0x7fd0a0011d00 comp rx=0 tx=0).stop 2024-03-27T11:01:54.992 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:55.021+0000 7fd0bee42740 1 -- 172.21.15.97:0/1265339137 shutdown_connections 2024-03-27T11:01:54.992 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:55.021+0000 7fd0bee42740 1 --2- 172.21.15.97:0/1265339137 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55c044656ff0 0x55c044659410 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:54.992 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:55.021+0000 7fd0bee42740 1 --2- 172.21.15.97:0/1265339137 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c0445dfcf0 0x55c0445ea500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:54.993 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:55.021+0000 7fd0bee42740 1 --2- 172.21.15.97:0/1265339137 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c0445dab80 0x55c0445e9560 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:54.993 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:55.021+0000 7fd0bee42740 1 --2- 172.21.15.97:0/1265339137 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c0445da190 0x55c0445e9020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:54.993 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:55.021+0000 7fd0bee42740 1 -- 172.21.15.97:0/1265339137 >> 172.21.15.97:0/1265339137 conn(0x55c0445d5700 msgr2=0x55c0445e44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:01:54.993 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:55.021+0000 7fd09b400640 0 client.25945 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T11:01:54.993 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:55.021+0000 7fd0bee42740 1 -- 172.21.15.97:0/1265339137 shutdown_connections 2024-03-27T11:01:54.993 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:55.021+0000 7fd0bee42740 1 -- 172.21.15.97:0/1265339137 wait complete. 2024-03-27T11:01:54.993 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:55.021+0000 7fd0bee42740 20 client.25945 trim_cache size 0 max 16384 2024-03-27T11:01:54.993 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-27T11:01:54.993 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-27T11:01:54.994 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-27T11:01:54.994 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-27T11:01:54.994 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:55.023+0000 7fd0bc200640 1 -- 172.21.15.97:0/3717287280 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fd09c0a3800 msgr2=0x7fd09c0a5cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:01:54.994 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:55.023+0000 7fd0bc200640 1 --2- 172.21.15.97:0/3717287280 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fd09c0a3800 0x7fd09c0a5cc0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac032380 tx=0x7fd0ac02c100 comp rx=0 tx=0).stop 2024-03-27T11:01:54.994 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:55.023+0000 7fd0bc200640 1 -- 172.21.15.97:0/3717287280 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd0b4072d20 msgr2=0x7fd0b419e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:01:54.994 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:55.023+0000 7fd0bc200640 1 --2- 172.21.15.97:0/3717287280 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd0b4072d20 0x7fd0b419e2e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd0a8011a60 tx=0x7fd0a8011f20 comp rx=0 tx=0).stop 2024-03-27T11:01:54.994 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:55.023+0000 7fd0bc200640 1 -- 172.21.15.97:0/3717287280 shutdown_connections 2024-03-27T11:01:54.995 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:55.023+0000 7fd0bc200640 1 --2- 172.21.15.97:0/3717287280 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0b410c770 0x7fd0b41a5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:54.995 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:55.023+0000 7fd0bc200640 1 --2- 172.21.15.97:0/3717287280 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd0b41094c0 0x7fd0b419e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:54.995 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:55.023+0000 7fd0bc200640 1 --2- 172.21.15.97:0/3717287280 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fd09c0a3800 0x7fd09c0a5cc0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:54.995 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:55.023+0000 7fd0bc200640 1 --2- 172.21.15.97:0/3717287280 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd0b4072d20 0x7fd0b419e2e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:01:54.995 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:55.023+0000 7fd0bc200640 1 -- 172.21.15.97:0/3717287280 >> 172.21.15.97:0/3717287280 conn(0x7fd0b406eb90 msgr2=0x7fd0b41119b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:01:54.995 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:55.023+0000 7fd0bc200640 1 -- 172.21.15.97:0/3717287280 shutdown_connections 2024-03-27T11:01:54.995 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:01:55.023+0000 7fd0bc200640 1 -- 172.21.15.97:0/3717287280 wait complete. 2024-03-27T11:01:55.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:01:55 smithi097 ceph-mon[31859]: pgmap v599: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 56 MiB/s wr, 897 op/s 2024-03-27T11:01:55.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:01:55 smithi177 ceph-mon[35294]: pgmap v599: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 56 MiB/s wr, 897 op/s 2024-03-27T11:01:55.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:01:55 smithi114 ceph-mon[35338]: pgmap v599: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 56 MiB/s wr, 897 op/s 2024-03-27T11:01:57.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:01:57 smithi097 ceph-mon[31859]: pgmap v600: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 54 MiB/s wr, 868 op/s 2024-03-27T11:01:57.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:01:57 smithi177 ceph-mon[35294]: pgmap v600: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 54 MiB/s wr, 868 op/s 2024-03-27T11:01:57.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:01:57 smithi114 ceph-mon[35338]: pgmap v600: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 54 MiB/s wr, 868 op/s 2024-03-27T11:01:59.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:01:59 smithi177 ceph-mon[35294]: pgmap v601: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 57 MiB/s wr, 914 op/s 2024-03-27T11:01:59.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:01:59 smithi114 ceph-mon[35338]: pgmap v601: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 57 MiB/s wr, 914 op/s 2024-03-27T11:01:59.890 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:01:59 smithi097 ceph-mon[31859]: pgmap v601: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 57 MiB/s wr, 914 op/s 2024-03-27T11:02:01.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:02:01 smithi177 ceph-mon[35294]: pgmap v602: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 61 MiB/s wr, 986 op/s 2024-03-27T11:02:01.693 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:02:01 smithi097 ceph-mon[31859]: pgmap v602: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 61 MiB/s wr, 986 op/s 2024-03-27T11:02:01.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:02:01 smithi114 ceph-mon[35338]: pgmap v602: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 61 MiB/s wr, 986 op/s 2024-03-27T11:02:03.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:02:03 smithi177 ceph-mon[35294]: pgmap v603: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 56 MiB/s wr, 894 op/s 2024-03-27T11:02:03.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:02:03 smithi114 ceph-mon[35338]: pgmap v603: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 56 MiB/s wr, 894 op/s 2024-03-27T11:02:03.890 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:02:03 smithi097 ceph-mon[31859]: pgmap v603: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 56 MiB/s wr, 894 op/s 2024-03-27T11:02:04.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:02:04 smithi177 ceph-mon[35294]: pgmap v604: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 56 MiB/s wr, 909 op/s 2024-03-27T11:02:04.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:02:04 smithi114 ceph-mon[35338]: pgmap v604: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 56 MiB/s wr, 909 op/s 2024-03-27T11:02:04.890 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:02:04 smithi097 ceph-mon[31859]: pgmap v604: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 56 MiB/s wr, 909 op/s 2024-03-27T11:02:07.515 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:02:07 smithi114 ceph-mon[35338]: pgmap v605: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 54 MiB/s wr, 869 op/s 2024-03-27T11:02:07.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:02:07 smithi097 ceph-mon[31859]: pgmap v605: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 54 MiB/s wr, 869 op/s 2024-03-27T11:02:07.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:02:07 smithi177 ceph-mon[35294]: pgmap v605: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 54 MiB/s wr, 869 op/s 2024-03-27T11:02:08.203 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.232+0000 7f86ff000640 1 -- 172.21.15.97:0/1489310286 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f86f8101b80 msgr2=0x7f86f8101fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:02:08.203 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.232+0000 7f86ff000640 1 --2- 172.21.15.97:0/1489310286 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f86f8101b80 0x7f86f8101fa0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f86e8009900 tx=0x7f86e802bc30 comp rx=0 tx=0).stop 2024-03-27T11:02:08.203 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.232+0000 7f86ff000640 1 -- 172.21.15.97:0/1489310286 shutdown_connections 2024-03-27T11:02:08.203 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.232+0000 7f86ff000640 1 --2- 172.21.15.97:0/1489310286 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f86f8103730 0x7f86f8109fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:08.204 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.232+0000 7f86ff000640 1 --2- 172.21.15.97:0/1489310286 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86f8102d70 0x7f86f81031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:08.204 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.232+0000 7f86ff000640 1 --2- 172.21.15.97:0/1489310286 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f86f8101b80 0x7f86f8101fa0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:08.204 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.232+0000 7f86ff000640 1 -- 172.21.15.97:0/1489310286 >> 172.21.15.97:0/1489310286 conn(0x7f86f80fd110 msgr2=0x7f86f80ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:02:08.204 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.233+0000 7f86ff000640 1 -- 172.21.15.97:0/1489310286 shutdown_connections 2024-03-27T11:02:08.204 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.233+0000 7f86ff000640 1 -- 172.21.15.97:0/1489310286 wait complete. 2024-03-27T11:02:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.234+0000 7f86ff000640 1 Processor -- start 2024-03-27T11:02:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.234+0000 7f86ff000640 1 -- start start 2024-03-27T11:02:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.235+0000 7f86ff000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f86f8102d70 0x7f86f819e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:02:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.235+0000 7f86ff000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86f8103730 0x7f86f819ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:02:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.235+0000 7f86ff000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f86f81a3a80 0x7f86f81a5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:02:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.235+0000 7f86ff000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f86f810d6d0 con 0x7f86f8103730 2024-03-27T11:02:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.235+0000 7f86ff000640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f86f810d550 con 0x7f86f81a3a80 2024-03-27T11:02:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.235+0000 7f86ff000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f86f810d850 con 0x7f86f8102d70 2024-03-27T11:02:08.207 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.235+0000 7f86f7e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f86f8102d70 0x7f86f819e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:02:08.207 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.235+0000 7f86f7400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86f8103730 0x7f86f819ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:02:08.207 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.235+0000 7f86f7e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f86f8102d70 0x7f86f819e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.97:46244/0 (socket says 172.21.15.97:46244) 2024-03-27T11:02:08.207 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.235+0000 7f86f7e00640 1 -- 172.21.15.97:0/503780597 learned_addr learned my addr 172.21.15.97:0/503780597 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:02:08.207 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.235+0000 7f86fd000640 1 --2- 172.21.15.97:0/503780597 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f86f81a3a80 0x7f86f81a5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:02:08.207 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.236+0000 7f86f7e00640 1 -- 172.21.15.97:0/503780597 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f86f81a3a80 msgr2=0x7f86f81a5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:02:08.208 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.236+0000 7f86f7e00640 1 --2- 172.21.15.97:0/503780597 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f86f81a3a80 0x7f86f81a5e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:08.208 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.236+0000 7f86f7e00640 1 -- 172.21.15.97:0/503780597 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86f8103730 msgr2=0x7f86f819ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:02:08.208 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.236+0000 7f86f7e00640 1 --2- 172.21.15.97:0/503780597 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86f8103730 0x7f86f819ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:08.208 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.237+0000 7f86f7e00640 1 -- 172.21.15.97:0/503780597 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86f81a64e0 con 0x7f86f8102d70 2024-03-27T11:02:08.208 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.237+0000 7f86fd000640 1 --2- 172.21.15.97:0/503780597 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f86f81a3a80 0x7f86f81a5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T11:02:08.208 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.237+0000 7f86f7400640 1 --2- 172.21.15.97:0/503780597 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86f8103730 0x7f86f819ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T11:02:08.208 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.237+0000 7f86f7e00640 1 --2- 172.21.15.97:0/503780597 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f86f8102d70 0x7f86f819e4e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f86e8009850 tx=0x7f86e8004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:02:08.210 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.238+0000 7f86f4c00640 1 -- 172.21.15.97:0/503780597 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e8043020 con 0x7f86f8102d70 2024-03-27T11:02:08.210 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.238+0000 7f86f4c00640 1 -- 172.21.15.97:0/503780597 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f86e80052b0 con 0x7f86f8102d70 2024-03-27T11:02:08.210 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.238+0000 7f86f4c00640 1 -- 172.21.15.97:0/503780597 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e8003b80 con 0x7f86f8102d70 2024-03-27T11:02:08.210 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.238+0000 7f86ff000640 1 -- 172.21.15.97:0/503780597 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86f81a6770 con 0x7f86f8102d70 2024-03-27T11:02:08.210 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.239+0000 7f86ff000640 1 -- 172.21.15.97:0/503780597 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86f81a6c40 con 0x7f86f8102d70 2024-03-27T11:02:08.216 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.245+0000 7f86f4c00640 1 -- 172.21.15.97:0/503780597 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f86e8046020 con 0x7f86f8102d70 2024-03-27T11:02:08.217 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.246+0000 7f8701e17740 1 Processor -- start 2024-03-27T11:02:08.217 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.246+0000 7f8701e17740 1 -- start start 2024-03-27T11:02:08.218 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.247+0000 7f8701e17740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5653328fd980 0x5653328fdda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:02:08.218 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.247+0000 7f86f4c00640 1 --2- 172.21.15.97:0/503780597 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f86dc0a3870 0x7f86dc0a5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:02:08.218 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.247+0000 7f8701e17740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5653328fe370 0x565332902e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:02:08.218 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.247+0000 7f8701e17740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5653329034e0 0x565332905970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:02:08.218 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.247+0000 7f8701e17740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x565332862d10 con 0x5653329034e0 2024-03-27T11:02:08.218 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.247+0000 7f8701e17740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x56533280b260 con 0x5653328fe370 2024-03-27T11:02:08.218 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.247+0000 7f8701e17740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x56533285ecc0 con 0x5653328fd980 2024-03-27T11:02:08.219 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.247+0000 7f86f4c00640 1 -- 172.21.15.97:0/503780597 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f86e80f1ee0 con 0x7f86f8102d70 2024-03-27T11:02:08.219 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.247+0000 7f86fd000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5653328fd980 0x5653328fdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:02:08.219 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.247+0000 7f86f7e00640 1 --2- 172.21.15.97:0/503780597 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f86dc0a3870 0x7f86dc0a5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:02:08.219 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.247+0000 7f86fd000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5653328fd980 0x5653328fdda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.97:46260/0 (socket says 172.21.15.97:46260) 2024-03-27T11:02:08.219 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.248+0000 7f86fd000640 1 -- 172.21.15.97:0/1802037433 learned_addr learned my addr 172.21.15.97:0/1802037433 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:02:08.219 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.248+0000 7f86f7400640 1 --2- 172.21.15.97:0/1802037433 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5653328fe370 0x565332902e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:02:08.220 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.248+0000 7f86fd000640 1 --2- 172.21.15.97:0/1802037433 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5653329034e0 0x565332905970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:02:08.220 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.248+0000 7f86fd000640 1 -- 172.21.15.97:0/1802037433 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5653328fd980 msgr2=0x5653328fdda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:02:08.220 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.248+0000 7f86fd000640 1 --2- 172.21.15.97:0/1802037433 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5653328fd980 0x5653328fdda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:08.220 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.248+0000 7f86fd000640 1 -- 172.21.15.97:0/1802037433 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5653328fe370 msgr2=0x565332902e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:02:08.220 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.248+0000 7f86fd000640 1 --2- 172.21.15.97:0/1802037433 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5653328fe370 0x565332902e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:08.220 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.248+0000 7f86fd000640 1 -- 172.21.15.97:0/1802037433 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565332802aa0 con 0x5653329034e0 2024-03-27T11:02:08.220 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.248+0000 7f86f7e00640 1 --2- 172.21.15.97:0/503780597 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f86dc0a3870 0x7f86dc0a5d30 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f86e80be2f0 tx=0x7f86e8005d50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:02:08.221 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.249+0000 7f86fd000640 1 --2- 172.21.15.97:0/1802037433 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5653329034e0 0x565332905970 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f86f00138a0 tx=0x7f86f0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bd9b5c254f36cb8 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:02:08.221 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.249+0000 7f86ed600640 1 -- 172.21.15.97:0/1802037433 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f0004d20 con 0x5653329034e0 2024-03-27T11:02:08.221 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.249+0000 7f86ed600640 1 -- 172.21.15.97:0/1802037433 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f86f0017030 con 0x5653329034e0 2024-03-27T11:02:08.221 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.249+0000 7f86ed600640 1 -- 172.21.15.97:0/1802037433 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f0005740 con 0x5653329034e0 2024-03-27T11:02:08.221 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.250+0000 7f8701e17740 1 -- 172.21.15.97:0/1802037433 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5653329034e0 msgr2=0x565332905970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:02:08.221 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.250+0000 7f8701e17740 1 --2- 172.21.15.97:0/1802037433 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5653329034e0 0x565332905970 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f86f00138a0 tx=0x7f86f0004ab0 comp rx=0 tx=0).stop 2024-03-27T11:02:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.251+0000 7f8701e17740 1 -- 172.21.15.97:0/1802037433 shutdown_connections 2024-03-27T11:02:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.251+0000 7f8701e17740 1 --2- 172.21.15.97:0/1802037433 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5653329034e0 0x565332905970 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.251+0000 7f8701e17740 1 --2- 172.21.15.97:0/1802037433 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5653328fe370 0x565332902e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.251+0000 7f8701e17740 1 --2- 172.21.15.97:0/1802037433 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5653328fd980 0x5653328fdda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.251+0000 7f8701e17740 1 -- 172.21.15.97:0/1802037433 >> 172.21.15.97:0/1802037433 conn(0x5653328f8ef0 msgr2=0x5653328f92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:02:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.251+0000 7f8701e17740 1 -- 172.21.15.97:0/1802037433 shutdown_connections 2024-03-27T11:02:08.223 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.251+0000 7f8701e17740 1 -- 172.21.15.97:0/1802037433 wait complete. 2024-03-27T11:02:08.223 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.252+0000 7f8701e17740 1 -- start start 2024-03-27T11:02:08.223 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.252+0000 7f8701e17740 1 Processor -- start 2024-03-27T11:02:08.224 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.253+0000 7f8701e17740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5653328fd980 0x565332907d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:02:08.228 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.253+0000 7f8701e17740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5653328fe370 0x565332908280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:02:08.229 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.253+0000 7f8701e17740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5653329034e0 0x56533290f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:02:08.229 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.253+0000 7f8701e17740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x565332906490 con 0x5653329034e0 2024-03-27T11:02:08.229 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.253+0000 7f8701e17740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x56533281e630 con 0x5653328fd980 2024-03-27T11:02:08.229 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.253+0000 7f8701e17740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x565332906610 con 0x5653328fe370 2024-03-27T11:02:08.229 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.253+0000 7f8701e17740 10 client.0 _ll_register_callbacks cb 0x565332823c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T11:02:08.229 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.254+0000 7f86fd000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5653328fd980 0x565332907d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:02:08.229 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.254+0000 7f86f7400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5653328fe370 0x565332908280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:02:08.229 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.254+0000 7f86fd000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5653328fd980 0x565332907d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:60362/0 (socket says 172.21.15.97:60362) 2024-03-27T11:02:08.229 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.254+0000 7f86fd000640 1 -- 172.21.15.97:0/1269165186 learned_addr learned my addr 172.21.15.97:0/1269165186 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:02:08.229 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.254+0000 7f86f7400640 1 -- 172.21.15.97:0/1269165186 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5653328fd980 msgr2=0x565332907d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:02:08.229 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.254+0000 7f86f7400640 1 --2- 172.21.15.97:0/1269165186 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5653328fd980 0x565332907d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:08.229 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.254+0000 7f86f7400640 1 -- 172.21.15.97:0/1269165186 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5653329034e0 msgr2=0x56533290f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:02:08.230 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.254+0000 7f86fd000640 1 --2- 172.21.15.97:0/1269165186 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5653329034e0 0x56533290f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:02:08.230 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.255+0000 7f86f7400640 1 --2- 172.21.15.97:0/1269165186 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5653329034e0 0x56533290f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:08.230 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.255+0000 7f86f7400640 1 -- 172.21.15.97:0/1269165186 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56533282a5f0 con 0x5653328fe370 2024-03-27T11:02:08.230 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.255+0000 7f86fd000640 1 --2- 172.21.15.97:0/1269165186 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5653329034e0 0x56533290f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T11:02:08.230 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.255+0000 7f86f7400640 1 --2- 172.21.15.97:0/1269165186 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5653328fe370 0x565332908280 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x5653328fe770 tx=0x7f86e4011b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bcee7b4b260a1550 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:02:08.230 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.255+0000 7f86e2000640 10 client.0 ms_handle_connect on v2:172.21.15.177:3300/0 2024-03-27T11:02:08.230 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.255+0000 7f86e2000640 1 -- 172.21.15.97:0/1269165186 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e4010bd0 con 0x5653328fe370 2024-03-27T11:02:08.230 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.255+0000 7f86e2000640 1 -- 172.21.15.97:0/1269165186 <== mon.1 v2:172.21.15.177:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f86e4010d70 con 0x5653328fe370 2024-03-27T11:02:08.230 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.255+0000 7f8701e17740 1 -- 172.21.15.97:0/1269165186 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86f0017030 con 0x5653328fe370 2024-03-27T11:02:08.230 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.256+0000 7f86e2000640 1 -- 172.21.15.97:0/1269165186 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f86e4004020 con 0x5653328fe370 2024-03-27T11:02:08.230 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.256+0000 7f8701e17740 10 client.25954 fetch_fsmap learned FSMap version 34 2024-03-27T11:02:08.230 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.256+0000 7f8701e17740 1 -- 172.21.15.97:0/1269165186 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x565332577210 con 0x5653328fe370 2024-03-27T11:02:08.230 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.256+0000 7f86e2000640 1 -- 172.21.15.97:0/1269165186 <== mon.1 v2:172.21.15.177:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f86f0017030 con 0x5653328fe370 2024-03-27T11:02:08.231 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.256+0000 7f86e2000640 1 -- 172.21.15.97:0/1269165186 <== mon.1 v2:172.21.15.177:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f86e4004830 con 0x5653328fe370 2024-03-27T11:02:08.231 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.257+0000 7f8701e17740 10 client.25954 fetch_fsmap finished waiting for FSMap version 34 2024-03-27T11:02:08.231 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.257+0000 7f8701e17740 10 client.25954 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T11:02:08.231 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.257+0000 7f8701e17740 20 client.25954 populate_metadata read hostname 'smithi097' 2024-03-27T11:02:08.231 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.257+0000 7f8701e17740 1 --2- 172.21.15.97:0/1269165186 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x565332913ba0 0x565332915ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:02:08.231 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.257+0000 7f8701e17740 4 client.25954 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T11:02:08.231 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.257+0000 7f8701e17740 1 -- 172.21.15.97:0/1269165186 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565332870250 con 0x565332913ba0 2024-03-27T11:02:08.231 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.257+0000 7f86fd000640 1 --2- 172.21.15.97:0/1269165186 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x565332913ba0 0x565332915ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:02:08.232 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.259+0000 7f86fd000640 1 --2- 172.21.15.97:0/1269165186 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x565332913ba0 0x565332915ff0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=abe213415f36d2ef server_cookie=cbf6b5f1346e4c5e in_seq=0 out_seq=0 2024-03-27T11:02:08.232 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.259+0000 7f86e2000640 10 client.25954 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T11:02:08.232 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.260+0000 7f86e2000640 1 -- 172.21.15.97:0/1269165186 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x565332870250 con 0x565332913ba0 2024-03-27T11:02:08.232 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.260+0000 7f86e2000640 10 client.25954 handle_command_reply: tid=0 2024-03-27T11:02:08.232 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.260+0000 7f8701e17740 1 client.25954 shutdown 2024-03-27T11:02:08.233 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.260+0000 7f8701e17740 1 -- 172.21.15.97:0/1269165186 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5653328fe370 msgr2=0x565332908280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:02:08.233 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.260+0000 7f8701e17740 1 --2- 172.21.15.97:0/1269165186 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5653328fe370 0x565332908280 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x5653328fe770 tx=0x7f86e4011b50 comp rx=0 tx=0).stop 2024-03-27T11:02:08.233 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.261+0000 7f8701e17740 1 -- 172.21.15.97:0/1269165186 shutdown_connections 2024-03-27T11:02:08.233 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.261+0000 7f8701e17740 1 --2- 172.21.15.97:0/1269165186 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x565332913ba0 0x565332915ff0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:08.233 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.261+0000 7f8701e17740 1 --2- 172.21.15.97:0/1269165186 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5653329034e0 0x56533290f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:08.233 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.261+0000 7f8701e17740 1 --2- 172.21.15.97:0/1269165186 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5653328fe370 0x565332908280 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:08.233 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.261+0000 7f8701e17740 1 --2- 172.21.15.97:0/1269165186 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5653328fd980 0x565332907d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:08.233 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.261+0000 7f8701e17740 1 -- 172.21.15.97:0/1269165186 >> 172.21.15.97:0/1269165186 conn(0x5653328f8ef0 msgr2=0x5653328f80f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:02:08.233 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.261+0000 7f86e2000640 0 client.25954 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T11:02:08.233 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.261+0000 7f8701e17740 1 -- 172.21.15.97:0/1269165186 shutdown_connections 2024-03-27T11:02:08.233 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.261+0000 7f8701e17740 1 -- 172.21.15.97:0/1269165186 wait complete. 2024-03-27T11:02:08.233 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.261+0000 7f8701e17740 20 client.25954 trim_cache size 0 max 16384 2024-03-27T11:02:08.258 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.287+0000 7f8701e17740 1 Processor -- start 2024-03-27T11:02:08.258 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.287+0000 7f8701e17740 1 -- start start 2024-03-27T11:02:08.259 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.288+0000 7f8701e17740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5653328fd980 0x5653328ff6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:02:08.259 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.288+0000 7f8701e17740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5653328fe370 0x5653328ffc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:02:08.259 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.288+0000 7f8701e17740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5653329034e0 0x56533290f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:02:08.259 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.288+0000 7f8701e17740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f86f81af9b0 con 0x5653328fe370 2024-03-27T11:02:08.259 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.288+0000 7f8701e17740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f86f81b14f0 con 0x5653329034e0 2024-03-27T11:02:08.259 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.288+0000 7f8701e17740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f86f81b0080 con 0x5653328fd980 2024-03-27T11:02:08.260 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.288+0000 7f86f7400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5653328fe370 0x5653328ffc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:02:08.260 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.289+0000 7f86fd000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5653328fd980 0x5653328ff6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:02:08.260 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.289+0000 7f86fd000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5653328fd980 0x5653328ff6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.97:46272/0 (socket says 172.21.15.97:46272) 2024-03-27T11:02:08.260 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.289+0000 7f86fd000640 1 -- 172.21.15.97:0/3777089991 learned_addr learned my addr 172.21.15.97:0/3777089991 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:02:08.260 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.289+0000 7f86fd000640 1 --2- 172.21.15.97:0/3777089991 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5653329034e0 0x56533290f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:02:08.261 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.289+0000 7f86f7400640 1 -- 172.21.15.97:0/3777089991 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5653328fd980 msgr2=0x5653328ff6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:02:08.261 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.289+0000 7f86f7400640 1 --2- 172.21.15.97:0/3777089991 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5653328fd980 0x5653328ff6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:08.261 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.290+0000 7f86f7400640 1 -- 172.21.15.97:0/3777089991 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5653329034e0 msgr2=0x56533290f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:02:08.261 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.290+0000 7f86f7400640 1 --2- 172.21.15.97:0/3777089991 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5653329034e0 0x56533290f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:08.261 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.290+0000 7f86f7400640 1 -- 172.21.15.97:0/3777089991 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565332643620 con 0x5653328fe370 2024-03-27T11:02:08.261 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.291+0000 7f86f7400640 1 --2- 172.21.15.97:0/3777089991 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5653328fe370 0x5653328ffc30 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f86e400c010 tx=0x7f86e400b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e8d522f45e8c348 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:02:08.262 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.291+0000 7f86ecc00640 1 -- 172.21.15.97:0/3777089991 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e402b040 con 0x5653328fe370 2024-03-27T11:02:08.262 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.291+0000 7f86ecc00640 1 -- 172.21.15.97:0/3777089991 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f86e4005bb0 con 0x5653328fe370 2024-03-27T11:02:08.262 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.291+0000 7f86ecc00640 1 -- 172.21.15.97:0/3777089991 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e40039d0 con 0x5653328fe370 2024-03-27T11:02:08.263 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.292+0000 7f8701e17740 1 -- 172.21.15.97:0/3777089991 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5653328fe370 msgr2=0x5653328ffc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:02:08.263 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.292+0000 7f8701e17740 1 --2- 172.21.15.97:0/3777089991 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5653328fe370 0x5653328ffc30 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f86e400c010 tx=0x7f86e400b480 comp rx=0 tx=0).stop 2024-03-27T11:02:08.264 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.292+0000 7f8701e17740 1 -- 172.21.15.97:0/3777089991 shutdown_connections 2024-03-27T11:02:08.264 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.292+0000 7f8701e17740 1 --2- 172.21.15.97:0/3777089991 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5653329034e0 0x56533290f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:08.264 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.293+0000 7f8701e17740 1 --2- 172.21.15.97:0/3777089991 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5653328fe370 0x5653328ffc30 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:08.264 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.293+0000 7f8701e17740 1 --2- 172.21.15.97:0/3777089991 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5653328fd980 0x5653328ff6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:08.264 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.293+0000 7f8701e17740 1 -- 172.21.15.97:0/3777089991 >> 172.21.15.97:0/3777089991 conn(0x5653328f8ef0 msgr2=0x565332907d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:02:08.264 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.293+0000 7f8701e17740 1 -- 172.21.15.97:0/3777089991 shutdown_connections 2024-03-27T11:02:08.265 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.293+0000 7f8701e17740 1 -- 172.21.15.97:0/3777089991 wait complete. 2024-03-27T11:02:08.265 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.294+0000 7f8701e17740 1 -- start start 2024-03-27T11:02:08.265 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.294+0000 7f8701e17740 1 Processor -- start 2024-03-27T11:02:08.266 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.295+0000 7f8701e17740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5653328fd980 0x56533290c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:02:08.266 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.296+0000 7f8701e17740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5653328fe370 0x56533290cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:02:08.266 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.296+0000 7f8701e17740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5653329034e0 0x56533290db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:02:08.267 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.296+0000 7f86fd000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5653328fd980 0x56533290c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:02:08.267 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.296+0000 7f86fd000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5653328fd980 0x56533290c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.97:46276/0 (socket says 172.21.15.97:46276) 2024-03-27T11:02:08.267 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.296+0000 7f86fd000640 1 -- 172.21.15.97:0/3407560192 learned_addr learned my addr 172.21.15.97:0/3407560192 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:02:08.267 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.296+0000 7f8701e17740 1 -- 172.21.15.97:0/3407560192 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f86f81b0e20 con 0x5653328fe370 2024-03-27T11:02:08.267 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.296+0000 7f8701e17740 1 -- 172.21.15.97:0/3407560192 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f86f81b0750 con 0x5653329034e0 2024-03-27T11:02:08.268 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.296+0000 7f8701e17740 1 -- 172.21.15.97:0/3407560192 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f86f81ad990 con 0x5653328fd980 2024-03-27T11:02:08.268 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.296+0000 7f8701e17740 10 client.0 _ll_register_callbacks cb 0x7f86e400bbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T11:02:08.268 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.296+0000 7f86fd000640 1 --2- 172.21.15.97:0/3407560192 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5653329034e0 0x56533290db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:02:08.268 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.296+0000 7f86f7400640 1 --2- 172.21.15.97:0/3407560192 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5653328fe370 0x56533290cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:02:08.268 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.296+0000 7f86fd000640 1 -- 172.21.15.97:0/3407560192 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5653329034e0 msgr2=0x56533290db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:02:08.268 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.296+0000 7f86fd000640 1 --2- 172.21.15.97:0/3407560192 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5653329034e0 0x56533290db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:08.268 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.296+0000 7f86fd000640 1 -- 172.21.15.97:0/3407560192 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5653328fe370 msgr2=0x56533290cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:02:08.269 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.296+0000 7f86fd000640 1 --2- 172.21.15.97:0/3407560192 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5653328fe370 0x56533290cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:08.269 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.296+0000 7f86fd000640 1 -- 172.21.15.97:0/3407560192 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56533290e110 con 0x5653328fd980 2024-03-27T11:02:08.269 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.297+0000 7f86f7400640 1 --2- 172.21.15.97:0/3407560192 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5653328fe370 0x56533290cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T11:02:08.269 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.297+0000 7f86fd000640 1 --2- 172.21.15.97:0/3407560192 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5653328fd980 0x56533290c700 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f86f00136e0 tx=0x7f86e400be00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=28861609d409e81c server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:02:08.269 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.297+0000 7f86e2000640 10 client.0 ms_handle_connect on v2:172.21.15.177:3300/0 2024-03-27T11:02:08.270 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.298+0000 7f86e2000640 1 -- 172.21.15.97:0/3407560192 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f002c5a0 con 0x5653328fd980 2024-03-27T11:02:08.270 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.298+0000 7f86e2000640 1 -- 172.21.15.97:0/3407560192 <== mon.1 v2:172.21.15.177:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f86f002c740 con 0x5653328fd980 2024-03-27T11:02:08.270 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.299+0000 7f8701e17740 1 -- 172.21.15.97:0/3407560192 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86e4005bb0 con 0x5653328fd980 2024-03-27T11:02:08.270 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.299+0000 7f86e2000640 1 -- 172.21.15.97:0/3407560192 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f86f0010a00 con 0x5653328fd980 2024-03-27T11:02:08.270 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.299+0000 7f8701e17740 10 client.25960 fetch_fsmap learned FSMap version 34 2024-03-27T11:02:08.270 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.299+0000 7f8701e17740 1 -- 172.21.15.97:0/3407560192 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56533290f420 con 0x5653328fd980 2024-03-27T11:02:08.271 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.299+0000 7f86e2000640 1 -- 172.21.15.97:0/3407560192 <== mon.1 v2:172.21.15.177:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f86e4005bb0 con 0x5653328fd980 2024-03-27T11:02:08.271 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.300+0000 7f86e2000640 1 -- 172.21.15.97:0/3407560192 <== mon.1 v2:172.21.15.177:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f86f000f040 con 0x5653328fd980 2024-03-27T11:02:08.271 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.300+0000 7f8701e17740 10 client.25960 fetch_fsmap finished waiting for FSMap version 34 2024-03-27T11:02:08.271 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.300+0000 7f8701e17740 10 client.25960 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T11:02:08.271 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.300+0000 7f8701e17740 20 client.25960 populate_metadata read hostname 'smithi097' 2024-03-27T11:02:08.271 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.300+0000 7f8701e17740 1 --2- 172.21.15.97:0/3407560192 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x56533297a1b0 0x56533297c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:02:08.271 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.300+0000 7f8701e17740 4 client.25960 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-27T11:02:08.271 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.300+0000 7f8701e17740 1 -- 172.21.15.97:0/3407560192 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56533286a5a0 con 0x56533297a1b0 2024-03-27T11:02:08.272 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.301+0000 7f86f7400640 1 --2- 172.21.15.97:0/3407560192 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x56533297a1b0 0x56533297c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:02:08.272 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.302+0000 7f86f7400640 1 --2- 172.21.15.97:0/3407560192 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x56533297a1b0 0x56533297c5a0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bcdb8ba3efee1851 server_cookie=81b8a726fcee691 in_seq=0 out_seq=0 2024-03-27T11:02:08.272 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.302+0000 7f86e2000640 10 client.25960 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T11:02:08.279 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.308+0000 7f86e2000640 1 -- 172.21.15.97:0/3407560192 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56533286a5a0 con 0x56533297a1b0 2024-03-27T11:02:08.279 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.308+0000 7f86e2000640 10 client.25960 handle_command_reply: tid=0 2024-03-27T11:02:08.279 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.308+0000 7f8701e17740 1 client.25960 shutdown 2024-03-27T11:02:08.279 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.309+0000 7f8701e17740 1 -- 172.21.15.97:0/3407560192 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5653328fd980 msgr2=0x56533290c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:02:08.279 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.309+0000 7f8701e17740 1 --2- 172.21.15.97:0/3407560192 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5653328fd980 0x56533290c700 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f86f00136e0 tx=0x7f86e400be00 comp rx=0 tx=0).stop 2024-03-27T11:02:08.280 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.310+0000 7f8701e17740 1 -- 172.21.15.97:0/3407560192 shutdown_connections 2024-03-27T11:02:08.281 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.310+0000 7f8701e17740 1 --2- 172.21.15.97:0/3407560192 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x56533297a1b0 0x56533297c5a0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:08.281 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.310+0000 7f8701e17740 1 --2- 172.21.15.97:0/3407560192 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5653329034e0 0x56533290db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:08.281 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.310+0000 7f8701e17740 1 --2- 172.21.15.97:0/3407560192 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5653328fe370 0x56533290cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:08.281 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.310+0000 7f8701e17740 1 --2- 172.21.15.97:0/3407560192 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5653328fd980 0x56533290c700 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:08.281 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.310+0000 7f8701e17740 1 -- 172.21.15.97:0/3407560192 >> 172.21.15.97:0/3407560192 conn(0x5653328f8ef0 msgr2=0x565332907ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:02:08.281 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.310+0000 7f86e2000640 0 client.25960 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T11:02:08.282 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.311+0000 7f8701e17740 1 -- 172.21.15.97:0/3407560192 shutdown_connections 2024-03-27T11:02:08.282 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.311+0000 7f8701e17740 1 -- 172.21.15.97:0/3407560192 wait complete. 2024-03-27T11:02:08.282 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.312+0000 7f8701e17740 20 client.25960 trim_cache size 0 max 16384 2024-03-27T11:02:08.283 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-27T11:02:08.283 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-27T11:02:08.283 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-27T11:02:08.283 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-27T11:02:08.284 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.313+0000 7f86ff000640 1 -- 172.21.15.97:0/503780597 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f86dc0a3870 msgr2=0x7f86dc0a5d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:02:08.285 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.313+0000 7f86ff000640 1 --2- 172.21.15.97:0/503780597 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f86dc0a3870 0x7f86dc0a5d30 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f86e80be2f0 tx=0x7f86e8005d50 comp rx=0 tx=0).stop 2024-03-27T11:02:08.285 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.314+0000 7f86ff000640 1 -- 172.21.15.97:0/503780597 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f86f8102d70 msgr2=0x7f86f819e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:02:08.285 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.314+0000 7f86ff000640 1 --2- 172.21.15.97:0/503780597 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f86f8102d70 0x7f86f819e4e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f86e8009850 tx=0x7f86e8004270 comp rx=0 tx=0).stop 2024-03-27T11:02:08.285 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.314+0000 7f86ff000640 1 -- 172.21.15.97:0/503780597 shutdown_connections 2024-03-27T11:02:08.285 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.314+0000 7f86ff000640 1 --2- 172.21.15.97:0/503780597 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f86dc0a3870 0x7f86dc0a5d30 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:08.285 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.314+0000 7f86ff000640 1 --2- 172.21.15.97:0/503780597 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f86f81a3a80 0x7f86f81a5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:08.285 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.314+0000 7f86ff000640 1 --2- 172.21.15.97:0/503780597 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86f8103730 0x7f86f819ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:08.285 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.314+0000 7f86ff000640 1 --2- 172.21.15.97:0/503780597 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f86f8102d70 0x7f86f819e4e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:08.285 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.314+0000 7f86ff000640 1 -- 172.21.15.97:0/503780597 >> 172.21.15.97:0/503780597 conn(0x7f86f80fd110 msgr2=0x7f86f8073d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:02:08.286 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.314+0000 7f86ff000640 1 -- 172.21.15.97:0/503780597 shutdown_connections 2024-03-27T11:02:08.286 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:08.314+0000 7f86ff000640 1 -- 172.21.15.97:0/503780597 wait complete. 2024-03-27T11:02:09.515 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:02:09 smithi114 ceph-mon[35338]: pgmap v606: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 56 MiB/s wr, 902 op/s 2024-03-27T11:02:09.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:02:09 smithi097 ceph-mon[31859]: pgmap v606: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 56 MiB/s wr, 902 op/s 2024-03-27T11:02:09.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:02:09 smithi177 ceph-mon[35294]: pgmap v606: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 56 MiB/s wr, 902 op/s 2024-03-27T11:02:11.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:02:11 smithi097 ceph-mon[31859]: pgmap v607: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 60 MiB/s wr, 969 op/s 2024-03-27T11:02:11.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:02:11 smithi177 ceph-mon[35294]: pgmap v607: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 60 MiB/s wr, 969 op/s 2024-03-27T11:02:11.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:02:11 smithi114 ceph-mon[35338]: pgmap v607: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 60 MiB/s wr, 969 op/s 2024-03-27T11:02:13.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:02:13 smithi097 ceph-mon[31859]: pgmap v608: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 54 MiB/s wr, 867 op/s 2024-03-27T11:02:13.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:02:13 smithi177 ceph-mon[35294]: pgmap v608: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 54 MiB/s wr, 867 op/s 2024-03-27T11:02:13.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:02:13 smithi114 ceph-mon[35338]: pgmap v608: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 54 MiB/s wr, 867 op/s 2024-03-27T11:02:15.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:02:15 smithi097 ceph-mon[31859]: pgmap v609: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 54 MiB/s wr, 882 op/s 2024-03-27T11:02:15.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:02:15 smithi177 ceph-mon[35294]: pgmap v609: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 54 MiB/s wr, 882 op/s 2024-03-27T11:02:15.766 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:02:15 smithi114 ceph-mon[35338]: pgmap v609: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 54 MiB/s wr, 882 op/s 2024-03-27T11:02:17.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:02:17 smithi097 ceph-mon[31859]: pgmap v610: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 53 MiB/s wr, 853 op/s 2024-03-27T11:02:17.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:02:17 smithi177 ceph-mon[35294]: pgmap v610: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 53 MiB/s wr, 853 op/s 2024-03-27T11:02:17.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:02:17 smithi114 ceph-mon[35338]: pgmap v610: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 53 MiB/s wr, 853 op/s 2024-03-27T11:02:19.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:02:19 smithi097 ceph-mon[31859]: pgmap v611: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 56 MiB/s wr, 905 op/s 2024-03-27T11:02:19.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:02:19 smithi177 ceph-mon[35294]: pgmap v611: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 56 MiB/s wr, 905 op/s 2024-03-27T11:02:19.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:02:19 smithi114 ceph-mon[35338]: pgmap v611: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 56 MiB/s wr, 905 op/s 2024-03-27T11:02:21.488 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.518+0000 7fc03c400640 1 -- 172.21.15.97:0/2059654781 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc034075b90 msgr2=0x7fc034073fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:02:21.488 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.518+0000 7fc03c400640 1 --2- 172.21.15.97:0/2059654781 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc034075b90 0x7fc034073fe0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fc028009930 tx=0x7fc02802bc30 comp rx=0 tx=0).stop 2024-03-27T11:02:21.489 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.518+0000 7fc03c400640 1 -- 172.21.15.97:0/2059654781 shutdown_connections 2024-03-27T11:02:21.489 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.518+0000 7fc03c400640 1 --2- 172.21.15.97:0/2059654781 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc03410e170 0x7fc034110560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:21.489 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.518+0000 7fc03c400640 1 --2- 172.21.15.97:0/2059654781 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc0340745b0 0x7fc034074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:21.489 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.518+0000 7fc03c400640 1 --2- 172.21.15.97:0/2059654781 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc034075b90 0x7fc034073fe0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:21.489 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.519+0000 7fc03c400640 1 -- 172.21.15.97:0/2059654781 >> 172.21.15.97:0/2059654781 conn(0x7fc0340fd110 msgr2=0x7fc0340ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:02:21.489 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.519+0000 7fc03c400640 1 -- 172.21.15.97:0/2059654781 shutdown_connections 2024-03-27T11:02:21.489 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.519+0000 7fc03c400640 1 -- 172.21.15.97:0/2059654781 wait complete. 2024-03-27T11:02:21.490 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.520+0000 7fc03c400640 1 Processor -- start 2024-03-27T11:02:21.491 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.521+0000 7fc03c400640 1 -- start start 2024-03-27T11:02:21.492 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.522+0000 7fc03c400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc0340745b0 0x7fc03419e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:02:21.492 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.522+0000 7fc03c400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc034075b90 0x7fc03419e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:02:21.492 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.522+0000 7fc03c400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc03410e170 0x7fc0341a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:02:21.492 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.522+0000 7fc03c400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc034113d30 con 0x7fc03410e170 2024-03-27T11:02:21.492 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.522+0000 7fc03c400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc034113bb0 con 0x7fc034075b90 2024-03-27T11:02:21.492 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.522+0000 7fc03c400640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc034113eb0 con 0x7fc0340745b0 2024-03-27T11:02:21.493 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.523+0000 7fc039000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc034075b90 0x7fc03419e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:02:21.493 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.523+0000 7fc03a400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc03410e170 0x7fc0341a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:02:21.493 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.523+0000 7fc039000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc034075b90 0x7fc03419e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:60762/0 (socket says 172.21.15.97:60762) 2024-03-27T11:02:21.493 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.523+0000 7fc039000640 1 -- 172.21.15.97:0/158840283 learned_addr learned my addr 172.21.15.97:0/158840283 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:02:21.493 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.523+0000 7fc039000640 1 -- 172.21.15.97:0/158840283 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc0340745b0 msgr2=0x7fc03419e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:02:21.494 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.523+0000 7fc039000640 1 --2- 172.21.15.97:0/158840283 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc0340745b0 0x7fc03419e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:21.494 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.523+0000 7fc039000640 1 -- 172.21.15.97:0/158840283 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc03410e170 msgr2=0x7fc0341a5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:02:21.494 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.523+0000 7fc039000640 1 --2- 172.21.15.97:0/158840283 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc03410e170 0x7fc0341a5900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:21.494 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.523+0000 7fc039000640 1 -- 172.21.15.97:0/158840283 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0341a5f70 con 0x7fc034075b90 2024-03-27T11:02:21.494 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.524+0000 7fc039000640 1 --2- 172.21.15.97:0/158840283 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc034075b90 0x7fc03419e8e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc02000d950 tx=0x7fc02000de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:02:21.495 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.524+0000 7fc032000640 1 -- 172.21.15.97:0/158840283 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc020017030 con 0x7fc034075b90 2024-03-27T11:02:21.495 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.525+0000 7fc032000640 1 -- 172.21.15.97:0/158840283 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc020004020 con 0x7fc034075b90 2024-03-27T11:02:21.495 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.525+0000 7fc032000640 1 -- 172.21.15.97:0/158840283 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0200047a0 con 0x7fc034075b90 2024-03-27T11:02:21.495 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.525+0000 7fc03c400640 1 -- 172.21.15.97:0/158840283 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0341a6260 con 0x7fc034075b90 2024-03-27T11:02:21.495 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.525+0000 7fc03c400640 1 -- 172.21.15.97:0/158840283 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0341a6760 con 0x7fc034075b90 2024-03-27T11:02:21.500 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.530+0000 7fc032000640 1 -- 172.21.15.97:0/158840283 <== mon.2 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7fc0200041d0 con 0x7fc034075b90 2024-03-27T11:02:21.502 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.532+0000 7fc03effc740 1 Processor -- start 2024-03-27T11:02:21.503 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.532+0000 7fc03effc740 1 -- start start 2024-03-27T11:02:21.503 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.532+0000 7fc032000640 1 --2- 172.21.15.97:0/158840283 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fc01c0a37b0 0x7fc01c0a5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:02:21.503 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.532+0000 7fc03effc740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55eb35043980 0x55eb35043da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:02:21.503 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.532+0000 7fc03effc740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eb35044370 0x55eb35048e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:02:21.503 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.532+0000 7fc03effc740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55eb350494e0 0x55eb3504b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:02:21.503 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.532+0000 7fc03effc740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55eb34fa8d10 con 0x55eb35044370 2024-03-27T11:02:21.503 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.532+0000 7fc03effc740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55eb34f51260 con 0x55eb350494e0 2024-03-27T11:02:21.503 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.532+0000 7fc03effc740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55eb34fa4cc0 con 0x55eb35043980 2024-03-27T11:02:21.503 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.532+0000 7fc032000640 1 -- 172.21.15.97:0/158840283 <== mon.2 v2:172.21.15.114:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fc0200cf7f0 con 0x7fc034075b90 2024-03-27T11:02:21.503 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.533+0000 7fc03a400640 1 --2- 172.21.15.97:0/158840283 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fc01c0a37b0 0x7fc01c0a5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:02:21.503 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.533+0000 7fc039a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55eb35043980 0x55eb35043da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:02:21.503 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.533+0000 7fc039a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55eb35043980 0x55eb35043da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.97:46046/0 (socket says 172.21.15.97:46046) 2024-03-27T11:02:21.503 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.533+0000 7fc039a00640 1 -- 172.21.15.97:0/412997854 learned_addr learned my addr 172.21.15.97:0/412997854 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:02:21.504 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.533+0000 7fc03a400640 1 --2- 172.21.15.97:0/412997854 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55eb350494e0 0x55eb3504b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:02:21.505 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.533+0000 7fc039000640 1 --2- 172.21.15.97:0/412997854 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eb35044370 0x55eb35048e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:02:21.505 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.533+0000 7fc039a00640 1 -- 172.21.15.97:0/412997854 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55eb350494e0 msgr2=0x55eb3504b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:02:21.505 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.533+0000 7fc039a00640 1 --2- 172.21.15.97:0/412997854 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55eb350494e0 0x55eb3504b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:21.505 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.533+0000 7fc039a00640 1 -- 172.21.15.97:0/412997854 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eb35044370 msgr2=0x55eb35048e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:02:21.505 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.534+0000 7fc039a00640 1 --2- 172.21.15.97:0/412997854 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eb35044370 0x55eb35048e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:21.505 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.534+0000 7fc039a00640 1 -- 172.21.15.97:0/412997854 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb34f48aa0 con 0x55eb35043980 2024-03-27T11:02:21.505 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.534+0000 7fc039000640 1 --2- 172.21.15.97:0/412997854 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eb35044370 0x55eb35048e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T11:02:21.505 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.534+0000 7fc03a400640 1 --2- 172.21.15.97:0/158840283 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fc01c0a37b0 0x7fc01c0a5c70 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fc02c009fd0 tx=0x7fc02c009d80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:02:21.505 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.534+0000 7fc039a00640 1 --2- 172.21.15.97:0/412997854 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55eb35043980 0x55eb35043da0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc028036040 tx=0x7fc028005bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf7cbdf34f45e835 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:02:21.505 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.535+0000 7fc026a00640 1 -- 172.21.15.97:0/412997854 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc028042020 con 0x55eb35043980 2024-03-27T11:02:21.505 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.535+0000 7fc026a00640 1 -- 172.21.15.97:0/412997854 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc02802cae0 con 0x55eb35043980 2024-03-27T11:02:21.506 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.535+0000 7fc026a00640 1 -- 172.21.15.97:0/412997854 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc028003c90 con 0x55eb35043980 2024-03-27T11:02:21.506 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.535+0000 7fc03effc740 1 -- 172.21.15.97:0/412997854 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55eb35043980 msgr2=0x55eb35043da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:02:21.506 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.535+0000 7fc03effc740 1 --2- 172.21.15.97:0/412997854 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55eb35043980 0x55eb35043da0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc028036040 tx=0x7fc028005bd0 comp rx=0 tx=0).stop 2024-03-27T11:02:21.507 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.536+0000 7fc03effc740 1 -- 172.21.15.97:0/412997854 shutdown_connections 2024-03-27T11:02:21.507 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.536+0000 7fc03effc740 1 --2- 172.21.15.97:0/412997854 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55eb350494e0 0x55eb3504b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:21.507 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.536+0000 7fc03effc740 1 --2- 172.21.15.97:0/412997854 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eb35044370 0x55eb35048e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:21.507 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.536+0000 7fc03effc740 1 --2- 172.21.15.97:0/412997854 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55eb35043980 0x55eb35043da0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:21.507 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.536+0000 7fc03effc740 1 -- 172.21.15.97:0/412997854 >> 172.21.15.97:0/412997854 conn(0x55eb3503eef0 msgr2=0x55eb3503f2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:02:21.507 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.536+0000 7fc03effc740 1 -- 172.21.15.97:0/412997854 shutdown_connections 2024-03-27T11:02:21.507 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.536+0000 7fc03effc740 1 -- 172.21.15.97:0/412997854 wait complete. 2024-03-27T11:02:21.507 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.537+0000 7fc03effc740 1 -- start start 2024-03-27T11:02:21.508 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.538+0000 7fc03effc740 1 Processor -- start 2024-03-27T11:02:21.509 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.538+0000 7fc03effc740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eb35043980 0x55eb3504dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:02:21.509 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.538+0000 7fc03effc740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55eb35044370 0x55eb3504e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:02:21.509 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.538+0000 7fc03effc740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55eb350494e0 0x55eb35055180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:02:21.509 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.538+0000 7fc03effc740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55eb3504c480 con 0x55eb35043980 2024-03-27T11:02:21.509 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.538+0000 7fc03effc740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55eb34f64630 con 0x55eb350494e0 2024-03-27T11:02:21.509 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.538+0000 7fc03effc740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55eb3504c600 con 0x55eb35044370 2024-03-27T11:02:21.509 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.539+0000 7fc03effc740 10 client.0 _ll_register_callbacks cb 0x55eb34f69c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T11:02:21.509 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.539+0000 7fc039a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eb35043980 0x55eb3504dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:02:21.510 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.539+0000 7fc039a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eb35043980 0x55eb3504dc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41056/0 (socket says 172.21.15.97:41056) 2024-03-27T11:02:21.510 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.539+0000 7fc039a00640 1 -- 172.21.15.97:0/2728831806 learned_addr learned my addr 172.21.15.97:0/2728831806 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:02:21.511 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.539+0000 7fc039000640 1 --2- 172.21.15.97:0/2728831806 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55eb35044370 0x55eb3504e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:02:21.511 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.539+0000 7fc03a400640 1 --2- 172.21.15.97:0/2728831806 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55eb350494e0 0x55eb35055180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:02:21.511 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.539+0000 7fc039a00640 1 -- 172.21.15.97:0/2728831806 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55eb35044370 msgr2=0x55eb3504e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:02:21.511 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.540+0000 7fc039a00640 1 --2- 172.21.15.97:0/2728831806 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55eb35044370 0x55eb3504e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:21.511 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.540+0000 7fc039a00640 1 -- 172.21.15.97:0/2728831806 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55eb350494e0 msgr2=0x55eb35055180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:02:21.511 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.540+0000 7fc039a00640 1 --2- 172.21.15.97:0/2728831806 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55eb350494e0 0x55eb35055180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:21.511 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.540+0000 7fc039a00640 1 -- 172.21.15.97:0/2728831806 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb34f705f0 con 0x55eb35043980 2024-03-27T11:02:21.511 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.540+0000 7fc03a400640 1 --2- 172.21.15.97:0/2728831806 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55eb350494e0 0x55eb35055180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T11:02:21.512 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.540+0000 7fc039000640 1 --2- 172.21.15.97:0/2728831806 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55eb35044370 0x55eb3504e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T11:02:21.512 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.540+0000 7fc039a00640 1 --2- 172.21.15.97:0/2728831806 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eb35043980 0x55eb3504dc70 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fc028009930 tx=0x7fc028004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=627b6d2d208d4345 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:02:21.512 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.540+0000 7fc01b400640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T11:02:21.512 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.540+0000 7fc01b400640 1 -- 172.21.15.97:0/2728831806 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc028042030 con 0x55eb35043980 2024-03-27T11:02:21.512 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.540+0000 7fc01b400640 1 -- 172.21.15.97:0/2728831806 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fc028034440 con 0x55eb35043980 2024-03-27T11:02:21.512 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.541+0000 7fc03effc740 1 -- 172.21.15.97:0/2728831806 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc02802cae0 con 0x55eb35043980 2024-03-27T11:02:21.513 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.542+0000 7fc01b400640 1 -- 172.21.15.97:0/2728831806 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fc02802cc90 con 0x55eb35043980 2024-03-27T11:02:21.513 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.542+0000 7fc03effc740 10 client.16590 fetch_fsmap learned FSMap version 34 2024-03-27T11:02:21.513 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.542+0000 7fc03effc740 1 -- 172.21.15.97:0/2728831806 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55eb34cbd210 con 0x55eb35043980 2024-03-27T11:02:21.513 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.543+0000 7fc01b400640 1 -- 172.21.15.97:0/2728831806 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc02802cae0 con 0x55eb35043980 2024-03-27T11:02:21.513 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.543+0000 7fc01b400640 1 -- 172.21.15.97:0/2728831806 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7fc028047530 con 0x55eb35043980 2024-03-27T11:02:21.513 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.543+0000 7fc03effc740 10 client.16590 fetch_fsmap finished waiting for FSMap version 34 2024-03-27T11:02:21.513 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.543+0000 7fc03effc740 10 client.16590 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T11:02:21.513 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.543+0000 7fc03effc740 20 client.16590 populate_metadata read hostname 'smithi097' 2024-03-27T11:02:21.513 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.543+0000 7fc03effc740 1 --2- 172.21.15.97:0/2728831806 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55eb35059670 0x55eb3505ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:02:21.513 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.543+0000 7fc03effc740 4 client.16590 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T11:02:21.514 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.543+0000 7fc03effc740 1 -- 172.21.15.97:0/2728831806 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eb34fb6250 con 0x55eb35059670 2024-03-27T11:02:21.514 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.544+0000 7fc039000640 1 --2- 172.21.15.97:0/2728831806 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55eb35059670 0x55eb3505ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:02:21.514 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.545+0000 7fc039000640 1 --2- 172.21.15.97:0/2728831806 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55eb35059670 0x55eb3505ba60 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc2787939dd0baf9 server_cookie=3eb2e9a22d467416 in_seq=0 out_seq=0 2024-03-27T11:02:21.514 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.545+0000 7fc01b400640 10 client.16590 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T11:02:21.516 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.546+0000 7fc01b400640 1 -- 172.21.15.97:0/2728831806 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55eb34fb6250 con 0x55eb35059670 2024-03-27T11:02:21.516 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.546+0000 7fc01b400640 10 client.16590 handle_command_reply: tid=0 2024-03-27T11:02:21.516 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.546+0000 7fc03effc740 1 client.16590 shutdown 2024-03-27T11:02:21.517 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.546+0000 7fc03effc740 1 -- 172.21.15.97:0/2728831806 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eb35043980 msgr2=0x55eb3504dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:02:21.517 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.546+0000 7fc03effc740 1 --2- 172.21.15.97:0/2728831806 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eb35043980 0x55eb3504dc70 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fc028009930 tx=0x7fc028004270 comp rx=0 tx=0).stop 2024-03-27T11:02:21.517 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.547+0000 7fc03effc740 1 -- 172.21.15.97:0/2728831806 shutdown_connections 2024-03-27T11:02:21.517 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.547+0000 7fc03effc740 1 --2- 172.21.15.97:0/2728831806 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55eb35059670 0x55eb3505ba60 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:21.517 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.547+0000 7fc03effc740 1 --2- 172.21.15.97:0/2728831806 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55eb350494e0 0x55eb35055180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:21.517 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.547+0000 7fc03effc740 1 --2- 172.21.15.97:0/2728831806 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55eb35044370 0x55eb3504e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:21.517 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.547+0000 7fc03effc740 1 --2- 172.21.15.97:0/2728831806 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eb35043980 0x55eb3504dc70 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:21.517 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.547+0000 7fc03effc740 1 -- 172.21.15.97:0/2728831806 >> 172.21.15.97:0/2728831806 conn(0x55eb3503eef0 msgr2=0x55eb3503e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:02:21.517 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.547+0000 7fc01b400640 0 client.16590 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T11:02:21.517 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.547+0000 7fc03effc740 1 -- 172.21.15.97:0/2728831806 shutdown_connections 2024-03-27T11:02:21.518 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.547+0000 7fc03effc740 1 -- 172.21.15.97:0/2728831806 wait complete. 2024-03-27T11:02:21.518 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.548+0000 7fc03effc740 20 client.16590 trim_cache size 0 max 16384 2024-03-27T11:02:21.545 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.576+0000 7fc03effc740 1 Processor -- start 2024-03-27T11:02:21.546 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.576+0000 7fc03effc740 1 -- start start 2024-03-27T11:02:21.546 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.576+0000 7fc03effc740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55eb35044370 0x55eb3503d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:02:21.546 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.576+0000 7fc03effc740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eb350494e0 0x55eb35055140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:02:21.546 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.576+0000 7fc03effc740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55eb3505bd90 0x55eb3503dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:02:21.546 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.576+0000 7fc03effc740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc0341af620 con 0x55eb350494e0 2024-03-27T11:02:21.546 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.576+0000 7fc03effc740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc0341b1160 con 0x55eb35044370 2024-03-27T11:02:21.547 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.576+0000 7fc03effc740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc0341afcf0 con 0x55eb3505bd90 2024-03-27T11:02:21.547 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.577+0000 7fc039a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55eb35044370 0x55eb3503d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:02:21.547 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.577+0000 7fc039a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55eb35044370 0x55eb3503d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:60790/0 (socket says 172.21.15.97:60790) 2024-03-27T11:02:21.547 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.577+0000 7fc039a00640 1 -- 172.21.15.97:0/2547643985 learned_addr learned my addr 172.21.15.97:0/2547643985 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:02:21.547 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.577+0000 7fc039000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eb350494e0 0x55eb35055140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:02:21.548 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.577+0000 7fc039a00640 1 -- 172.21.15.97:0/2547643985 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55eb3505bd90 msgr2=0x55eb3503dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:02:21.548 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.577+0000 7fc039a00640 1 --2- 172.21.15.97:0/2547643985 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55eb3505bd90 0x55eb3503dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:21.548 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.577+0000 7fc039a00640 1 -- 172.21.15.97:0/2547643985 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eb350494e0 msgr2=0x55eb35055140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:02:21.548 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.577+0000 7fc039a00640 1 --2- 172.21.15.97:0/2547643985 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eb350494e0 0x55eb35055140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:21.548 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.577+0000 7fc039a00640 1 -- 172.21.15.97:0/2547643985 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb34d89620 con 0x55eb35044370 2024-03-27T11:02:21.548 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.577+0000 7fc039a00640 1 --2- 172.21.15.97:0/2547643985 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55eb35044370 0x55eb3503d860 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc028005780 tx=0x7fc028005c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70a5a67b54150dfe server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:02:21.549 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.578+0000 7fc026000640 1 -- 172.21.15.97:0/2547643985 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc02805e070 con 0x55eb35044370 2024-03-27T11:02:21.549 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.578+0000 7fc026000640 1 -- 172.21.15.97:0/2547643985 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc02804b070 con 0x55eb35044370 2024-03-27T11:02:21.549 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.578+0000 7fc026000640 1 -- 172.21.15.97:0/2547643985 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc028047420 con 0x55eb35044370 2024-03-27T11:02:21.549 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.578+0000 7fc03effc740 1 -- 172.21.15.97:0/2547643985 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55eb35044370 msgr2=0x55eb3503d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:02:21.549 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.578+0000 7fc03effc740 1 --2- 172.21.15.97:0/2547643985 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55eb35044370 0x55eb3503d860 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc028005780 tx=0x7fc028005c90 comp rx=0 tx=0).stop 2024-03-27T11:02:21.549 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.579+0000 7fc03effc740 1 -- 172.21.15.97:0/2547643985 shutdown_connections 2024-03-27T11:02:21.549 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.579+0000 7fc03effc740 1 --2- 172.21.15.97:0/2547643985 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55eb3505bd90 0x55eb3503dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:21.549 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.579+0000 7fc03effc740 1 --2- 172.21.15.97:0/2547643985 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eb350494e0 0x55eb35055140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:21.549 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.579+0000 7fc03effc740 1 --2- 172.21.15.97:0/2547643985 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55eb35044370 0x55eb3503d860 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:21.549 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.579+0000 7fc03effc740 1 -- 172.21.15.97:0/2547643985 >> 172.21.15.97:0/2547643985 conn(0x55eb3503eef0 msgr2=0x55eb3505b680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:02:21.550 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.579+0000 7fc03effc740 1 -- 172.21.15.97:0/2547643985 shutdown_connections 2024-03-27T11:02:21.550 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.579+0000 7fc03effc740 1 -- 172.21.15.97:0/2547643985 wait complete. 2024-03-27T11:02:21.551 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.579+0000 7fc03effc740 1 -- start start 2024-03-27T11:02:21.551 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.580+0000 7fc03effc740 1 Processor -- start 2024-03-27T11:02:21.551 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.580+0000 7fc03effc740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55eb35044370 0x55eb350500b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:02:21.551 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.580+0000 7fc03effc740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eb350494e0 0x55eb350505f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:02:21.551 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.580+0000 7fc03effc740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55eb3505bd90 0x55eb350ce900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:02:21.551 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.580+0000 7fc03effc740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc0341b0a90 con 0x55eb350494e0 2024-03-27T11:02:21.551 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.580+0000 7fc03effc740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc0341b03c0 con 0x55eb3505bd90 2024-03-27T11:02:21.551 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.580+0000 7fc03effc740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc0341ad600 con 0x55eb35044370 2024-03-27T11:02:21.551 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.580+0000 7fc03effc740 10 client.0 _ll_register_callbacks cb 0x7fc02c013740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T11:02:21.551 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.580+0000 7fc039a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55eb35044370 0x55eb350500b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:02:21.551 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.580+0000 7fc039a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55eb35044370 0x55eb350500b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.97:46072/0 (socket says 172.21.15.97:46072) 2024-03-27T11:02:21.552 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.580+0000 7fc039a00640 1 -- 172.21.15.97:0/429235103 learned_addr learned my addr 172.21.15.97:0/429235103 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:02:21.552 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.580+0000 7fc03a400640 1 --2- 172.21.15.97:0/429235103 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55eb3505bd90 0x55eb350ce900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:02:21.552 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.580+0000 7fc039000640 1 --2- 172.21.15.97:0/429235103 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eb350494e0 0x55eb350505f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:02:21.552 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.581+0000 7fc039a00640 1 -- 172.21.15.97:0/429235103 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55eb3505bd90 msgr2=0x55eb350ce900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:02:21.552 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.581+0000 7fc039a00640 1 --2- 172.21.15.97:0/429235103 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55eb3505bd90 0x55eb350ce900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:21.552 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.581+0000 7fc039a00640 1 -- 172.21.15.97:0/429235103 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eb350494e0 msgr2=0x55eb350505f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:02:21.552 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.581+0000 7fc039a00640 1 --2- 172.21.15.97:0/429235103 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eb350494e0 0x55eb350505f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:21.552 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.581+0000 7fc039a00640 1 -- 172.21.15.97:0/429235103 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb350cee40 con 0x55eb35044370 2024-03-27T11:02:21.553 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.581+0000 7fc039a00640 1 --2- 172.21.15.97:0/429235103 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55eb35044370 0x55eb350500b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc02802c840 tx=0x7fc02802c650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d8cfa322ec838000 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:02:21.553 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.581+0000 7fc01b400640 10 client.0 ms_handle_connect on v2:172.21.15.177:3300/0 2024-03-27T11:02:21.553 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.582+0000 7fc01b400640 1 -- 172.21.15.97:0/429235103 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc028067070 con 0x55eb35044370 2024-03-27T11:02:21.553 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.582+0000 7fc03effc740 1 -- 172.21.15.97:0/429235103 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc02804b070 con 0x55eb35044370 2024-03-27T11:02:21.553 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.582+0000 7fc01b400640 1 -- 172.21.15.97:0/429235103 <== mon.1 v2:172.21.15.177:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fc028047620 con 0x55eb35044370 2024-03-27T11:02:21.554 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.582+0000 7fc01b400640 1 -- 172.21.15.97:0/429235103 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fc028073050 con 0x55eb35044370 2024-03-27T11:02:21.554 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.582+0000 7fc03effc740 10 client.25972 fetch_fsmap learned FSMap version 34 2024-03-27T11:02:21.554 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.582+0000 7fc03effc740 1 -- 172.21.15.97:0/429235103 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55eb350d0060 con 0x55eb35044370 2024-03-27T11:02:21.554 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.583+0000 7fc01b400640 1 -- 172.21.15.97:0/429235103 <== mon.1 v2:172.21.15.177:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc02804b070 con 0x55eb35044370 2024-03-27T11:02:21.554 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.583+0000 7fc01b400640 1 -- 172.21.15.97:0/429235103 <== mon.1 v2:172.21.15.177:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7fc02805a800 con 0x55eb35044370 2024-03-27T11:02:21.554 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.583+0000 7fc03effc740 10 client.25972 fetch_fsmap finished waiting for FSMap version 34 2024-03-27T11:02:21.554 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.583+0000 7fc03effc740 10 client.25972 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T11:02:21.554 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.583+0000 7fc03effc740 20 client.25972 populate_metadata read hostname 'smithi097' 2024-03-27T11:02:21.554 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.583+0000 7fc03effc740 1 --2- 172.21.15.97:0/429235103 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55eb350d0960 0x55eb350d2d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:02:21.554 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.583+0000 7fc03effc740 4 client.25972 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-27T11:02:21.554 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.583+0000 7fc03effc740 1 -- 172.21.15.97:0/429235103 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55eb34fb05a0 con 0x55eb350d0960 2024-03-27T11:02:21.555 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.584+0000 7fc039000640 1 --2- 172.21.15.97:0/429235103 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55eb350d0960 0x55eb350d2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:02:21.555 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.585+0000 7fc01b400640 10 client.25972 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T11:02:21.555 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.585+0000 7fc039000640 1 --2- 172.21.15.97:0/429235103 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55eb350d0960 0x55eb350d2d80 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f1c17dc5e551ccd server_cookie=9de28d4211c84a38 in_seq=0 out_seq=0 2024-03-27T11:02:21.568 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.598+0000 7fc01b400640 1 -- 172.21.15.97:0/429235103 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55eb34fb05a0 con 0x55eb350d0960 2024-03-27T11:02:21.568 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.598+0000 7fc01b400640 10 client.25972 handle_command_reply: tid=0 2024-03-27T11:02:21.568 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.599+0000 7fc03effc740 1 client.25972 shutdown 2024-03-27T11:02:21.569 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.599+0000 7fc03effc740 1 -- 172.21.15.97:0/429235103 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55eb35044370 msgr2=0x55eb350500b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:02:21.569 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.599+0000 7fc03effc740 1 --2- 172.21.15.97:0/429235103 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55eb35044370 0x55eb350500b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc02802c840 tx=0x7fc02802c650 comp rx=0 tx=0).stop 2024-03-27T11:02:21.569 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.599+0000 7fc03effc740 1 -- 172.21.15.97:0/429235103 shutdown_connections 2024-03-27T11:02:21.569 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.599+0000 7fc03effc740 1 --2- 172.21.15.97:0/429235103 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55eb350d0960 0x55eb350d2d80 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:21.569 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.599+0000 7fc03effc740 1 --2- 172.21.15.97:0/429235103 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55eb3505bd90 0x55eb350ce900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:21.569 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.599+0000 7fc03effc740 1 --2- 172.21.15.97:0/429235103 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eb350494e0 0x55eb350505f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:21.570 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.599+0000 7fc03effc740 1 --2- 172.21.15.97:0/429235103 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55eb35044370 0x55eb350500b0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:21.570 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.599+0000 7fc03effc740 1 -- 172.21.15.97:0/429235103 >> 172.21.15.97:0/429235103 conn(0x55eb3503eef0 msgr2=0x55eb35042150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:02:21.570 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.599+0000 7fc01b400640 0 client.25972 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T11:02:21.570 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.600+0000 7fc03effc740 1 -- 172.21.15.97:0/429235103 shutdown_connections 2024-03-27T11:02:21.570 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.600+0000 7fc03effc740 1 -- 172.21.15.97:0/429235103 wait complete. 2024-03-27T11:02:21.570 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.600+0000 7fc03effc740 20 client.25972 trim_cache size 0 max 16384 2024-03-27T11:02:21.571 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-27T11:02:21.571 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-27T11:02:21.571 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-27T11:02:21.571 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-27T11:02:21.572 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.601+0000 7fc03c400640 1 -- 172.21.15.97:0/158840283 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fc01c0a37b0 msgr2=0x7fc01c0a5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:02:21.572 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.601+0000 7fc03c400640 1 --2- 172.21.15.97:0/158840283 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fc01c0a37b0 0x7fc01c0a5c70 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fc02c009fd0 tx=0x7fc02c009d80 comp rx=0 tx=0).stop 2024-03-27T11:02:21.572 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.601+0000 7fc03c400640 1 -- 172.21.15.97:0/158840283 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc034075b90 msgr2=0x7fc03419e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:02:21.572 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.601+0000 7fc03c400640 1 --2- 172.21.15.97:0/158840283 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc034075b90 0x7fc03419e8e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc02000d950 tx=0x7fc02000de10 comp rx=0 tx=0).stop 2024-03-27T11:02:21.572 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.601+0000 7fc03c400640 1 -- 172.21.15.97:0/158840283 shutdown_connections 2024-03-27T11:02:21.572 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.601+0000 7fc03c400640 1 --2- 172.21.15.97:0/158840283 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc03410e170 0x7fc0341a5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:21.572 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.601+0000 7fc03c400640 1 --2- 172.21.15.97:0/158840283 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fc01c0a37b0 0x7fc01c0a5c70 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:21.572 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.601+0000 7fc03c400640 1 --2- 172.21.15.97:0/158840283 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc034075b90 0x7fc03419e8e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:21.572 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.601+0000 7fc03c400640 1 --2- 172.21.15.97:0/158840283 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc0340745b0 0x7fc03419e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:21.572 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.601+0000 7fc03c400640 1 -- 172.21.15.97:0/158840283 >> 172.21.15.97:0/158840283 conn(0x7fc0340fd110 msgr2=0x7fc034101510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:02:21.572 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.601+0000 7fc03c400640 1 -- 172.21.15.97:0/158840283 shutdown_connections 2024-03-27T11:02:21.573 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:21.601+0000 7fc03c400640 1 -- 172.21.15.97:0/158840283 wait complete. 2024-03-27T11:02:21.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:02:21 smithi097 ceph-mon[31859]: pgmap v612: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 60 MiB/s wr, 979 op/s 2024-03-27T11:02:21.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:02:21 smithi177 ceph-mon[35294]: pgmap v612: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 60 MiB/s wr, 979 op/s 2024-03-27T11:02:21.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:02:21 smithi114 ceph-mon[35338]: pgmap v612: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 60 MiB/s wr, 979 op/s 2024-03-27T11:02:23.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:02:23 smithi097 ceph-mon[31859]: pgmap v613: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 55 MiB/s wr, 887 op/s 2024-03-27T11:02:23.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:02:23 smithi177 ceph-mon[35294]: pgmap v613: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 55 MiB/s wr, 887 op/s 2024-03-27T11:02:23.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:02:23 smithi114 ceph-mon[35338]: pgmap v613: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 55 MiB/s wr, 887 op/s 2024-03-27T11:02:25.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:02:25 smithi097 ceph-mon[31859]: pgmap v614: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 56 MiB/s wr, 903 op/s 2024-03-27T11:02:25.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:02:25 smithi177 ceph-mon[35294]: pgmap v614: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 56 MiB/s wr, 903 op/s 2024-03-27T11:02:25.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:02:25 smithi114 ceph-mon[35338]: pgmap v614: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 56 MiB/s wr, 903 op/s 2024-03-27T11:02:27.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:02:27 smithi097 ceph-mon[31859]: pgmap v615: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 54 MiB/s wr, 878 op/s 2024-03-27T11:02:27.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:02:27 smithi177 ceph-mon[35294]: pgmap v615: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 54 MiB/s wr, 878 op/s 2024-03-27T11:02:27.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:02:27 smithi114 ceph-mon[35338]: pgmap v615: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 54 MiB/s wr, 878 op/s 2024-03-27T11:02:29.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:02:29 smithi097 ceph-mon[31859]: pgmap v616: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 57 MiB/s wr, 915 op/s 2024-03-27T11:02:29.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:02:29 smithi177 ceph-mon[35294]: pgmap v616: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 57 MiB/s wr, 915 op/s 2024-03-27T11:02:29.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:02:29 smithi114 ceph-mon[35338]: pgmap v616: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 57 MiB/s wr, 915 op/s 2024-03-27T11:02:31.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:02:31 smithi097 ceph-mon[31859]: pgmap v617: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 61 MiB/s wr, 989 op/s 2024-03-27T11:02:31.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:02:31 smithi177 ceph-mon[35294]: pgmap v617: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 61 MiB/s wr, 989 op/s 2024-03-27T11:02:31.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:02:31 smithi114 ceph-mon[35338]: pgmap v617: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 61 MiB/s wr, 989 op/s 2024-03-27T11:02:33.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:02:33 smithi097 ceph-mon[31859]: pgmap v618: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 55 MiB/s wr, 894 op/s 2024-03-27T11:02:33.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:02:33 smithi177 ceph-mon[35294]: pgmap v618: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 55 MiB/s wr, 894 op/s 2024-03-27T11:02:33.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:02:33 smithi114 ceph-mon[35338]: pgmap v618: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 55 MiB/s wr, 894 op/s 2024-03-27T11:02:34.780 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.809+0000 7f8fc7a00640 1 -- 172.21.15.97:0/2001737791 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8fc010b780 msgr2=0x7f8fc010db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:02:34.780 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.809+0000 7f8fc7a00640 1 --2- 172.21.15.97:0/2001737791 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8fc010b780 0x7f8fc010db90 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4009860 tx=0x7f8fb402bb70 comp rx=0 tx=0).stop 2024-03-27T11:02:34.780 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.810+0000 7f8fc7a00640 1 -- 172.21.15.97:0/2001737791 shutdown_connections 2024-03-27T11:02:34.780 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.810+0000 7f8fc7a00640 1 --2- 172.21.15.97:0/2001737791 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8fc006a9a0 0x7f8fc0110b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:34.780 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.810+0000 7f8fc7a00640 1 --2- 172.21.15.97:0/2001737791 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8fc010b780 0x7f8fc010db90 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:34.780 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.810+0000 7f8fc7a00640 1 --2- 172.21.15.97:0/2001737791 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fc0108dc0 0x7f8fc010b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:34.780 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.810+0000 7f8fc7a00640 1 -- 172.21.15.97:0/2001737791 >> 172.21.15.97:0/2001737791 conn(0x7f8fc006ed50 msgr2=0x7f8fc0071190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:02:34.781 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.810+0000 7f8fc7a00640 1 -- 172.21.15.97:0/2001737791 shutdown_connections 2024-03-27T11:02:34.781 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.810+0000 7f8fc7a00640 1 -- 172.21.15.97:0/2001737791 wait complete. 2024-03-27T11:02:34.782 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.812+0000 7f8fc7a00640 1 Processor -- start 2024-03-27T11:02:34.782 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.812+0000 7f8fc7a00640 1 -- start start 2024-03-27T11:02:34.784 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.812+0000 7f8fc7a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8fc006a9a0 0x7f8fc01a6b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:02:34.784 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.812+0000 7f8fc7a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fc0108dc0 0x7f8fc01a7070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:02:34.784 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.812+0000 7f8fc7a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8fc010b780 0x7f8fc01ae0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:02:34.784 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.812+0000 7f8fc7a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc011c520 con 0x7f8fc0108dc0 2024-03-27T11:02:34.784 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.812+0000 7f8fc7a00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc011c3a0 con 0x7f8fc006a9a0 2024-03-27T11:02:34.784 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.812+0000 7f8fc7a00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc011c6a0 con 0x7f8fc010b780 2024-03-27T11:02:34.784 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.813+0000 7f8fbfe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fc0108dc0 0x7f8fc01a7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:02:34.784 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.813+0000 7f8fc5a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8fc010b780 0x7f8fc01ae0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:02:34.784 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.813+0000 7f8fc5a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8fc010b780 0x7f8fc01ae0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.97:39234/0 (socket says 172.21.15.97:39234) 2024-03-27T11:02:34.785 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.813+0000 7f8fc5a00640 1 -- 172.21.15.97:0/3737700604 learned_addr learned my addr 172.21.15.97:0/3737700604 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:02:34.785 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.813+0000 7f8fc5000640 1 --2- 172.21.15.97:0/3737700604 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8fc006a9a0 0x7f8fc01a6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:02:34.785 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.814+0000 7f8fc5a00640 1 -- 172.21.15.97:0/3737700604 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8fc006a9a0 msgr2=0x7f8fc01a6b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:02:34.785 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.814+0000 7f8fc5a00640 1 --2- 172.21.15.97:0/3737700604 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8fc006a9a0 0x7f8fc01a6b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:34.785 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.814+0000 7f8fc5a00640 1 -- 172.21.15.97:0/3737700604 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fc0108dc0 msgr2=0x7f8fc01a7070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:02:34.785 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.814+0000 7f8fc5a00640 1 --2- 172.21.15.97:0/3737700604 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fc0108dc0 0x7f8fc01a7070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:34.785 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.814+0000 7f8fc5a00640 1 -- 172.21.15.97:0/3737700604 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fc01ae750 con 0x7f8fc010b780 2024-03-27T11:02:34.786 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.814+0000 7f8fc5a00640 1 --2- 172.21.15.97:0/3737700604 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8fc010b780 0x7f8fc01ae0e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8fb800eb10 tx=0x7f8fb800ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:02:34.786 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.815+0000 7f8fbd600640 1 -- 172.21.15.97:0/3737700604 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb800ce20 con 0x7f8fc010b780 2024-03-27T11:02:34.786 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.815+0000 7f8fbd600640 1 -- 172.21.15.97:0/3737700604 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8fb8005020 con 0x7f8fc010b780 2024-03-27T11:02:34.786 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.815+0000 7f8fbd600640 1 -- 172.21.15.97:0/3737700604 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb8010690 con 0x7f8fc010b780 2024-03-27T11:02:34.786 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.815+0000 7f8fc7a00640 1 -- 172.21.15.97:0/3737700604 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fc01aea40 con 0x7f8fc010b780 2024-03-27T11:02:34.786 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.816+0000 7f8fc7a00640 1 -- 172.21.15.97:0/3737700604 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fc01aef10 con 0x7f8fc010b780 2024-03-27T11:02:34.790 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.820+0000 7f8fca7e5740 1 Processor -- start 2024-03-27T11:02:34.791 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.821+0000 7f8fbd600640 1 -- 172.21.15.97:0/3737700604 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f8fb80051d0 con 0x7f8fc010b780 2024-03-27T11:02:34.791 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.822+0000 7f8fca7e5740 1 -- start start 2024-03-27T11:02:34.792 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.822+0000 7f8fca7e5740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5565df4e1980 0x5565df4e1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:02:34.792 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.822+0000 7f8fca7e5740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5565df4e2370 0x5565df4e6e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:02:34.793 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.823+0000 7f8fca7e5740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565df4e74e0 0x5565df4e9970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:02:34.793 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.823+0000 7f8fca7e5740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5565df446d10 con 0x5565df4e74e0 2024-03-27T11:02:34.793 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.823+0000 7f8fca7e5740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5565df3ef260 con 0x5565df4e1980 2024-03-27T11:02:34.793 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.823+0000 7f8fca7e5740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5565df442cc0 con 0x5565df4e2370 2024-03-27T11:02:34.793 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.823+0000 7f8fbfe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5565df4e1980 0x5565df4e1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:02:34.793 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.823+0000 7f8fbfe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5565df4e1980 0x5565df4e1da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:52940/0 (socket says 172.21.15.97:52940) 2024-03-27T11:02:34.793 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.823+0000 7f8fbfe00640 1 -- 172.21.15.97:0/1212622987 learned_addr learned my addr 172.21.15.97:0/1212622987 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:02:34.794 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.823+0000 7f8fbfe00640 1 -- 172.21.15.97:0/1212622987 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5565df4e2370 msgr2=0x5565df4e6e70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-27T11:02:34.794 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.823+0000 7f8fbfe00640 1 --2- 172.21.15.97:0/1212622987 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5565df4e2370 0x5565df4e6e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:34.794 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.823+0000 7f8fbfe00640 1 -- 172.21.15.97:0/1212622987 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565df4e74e0 msgr2=0x5565df4e9970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:02:34.794 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.823+0000 7f8fbfe00640 1 --2- 172.21.15.97:0/1212622987 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565df4e74e0 0x5565df4e9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:34.794 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.823+0000 7f8fbfe00640 1 -- 172.21.15.97:0/1212622987 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565df3e6aa0 con 0x5565df4e1980 2024-03-27T11:02:34.794 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.824+0000 7f8fbd600640 1 --2- 172.21.15.97:0/3737700604 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f8fa80a3710 0x7f8fa80a5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:02:34.794 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.824+0000 7f8fbfe00640 1 --2- 172.21.15.97:0/1212622987 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5565df4e1980 0x5565df4e1da0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8fc0073cb0 tx=0x7f8fb4009520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=231a2d430abb05ac server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:02:34.794 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.824+0000 7f8fbd600640 1 -- 172.21.15.97:0/3737700604 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f8fb80cfe00 con 0x7f8fc010b780 2024-03-27T11:02:34.795 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.824+0000 7f8fae000640 1 -- 172.21.15.97:0/1212622987 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb4045070 con 0x5565df4e1980 2024-03-27T11:02:34.795 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.825+0000 7f8fae000640 1 -- 172.21.15.97:0/1212622987 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8fb403a040 con 0x5565df4e1980 2024-03-27T11:02:34.795 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.825+0000 7f8fae000640 1 -- 172.21.15.97:0/1212622987 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb4039070 con 0x5565df4e1980 2024-03-27T11:02:34.796 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.826+0000 7f8fc5000640 1 --2- 172.21.15.97:0/3737700604 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f8fa80a3710 0x7f8fa80a5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:02:34.796 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.826+0000 7f8fca7e5740 1 -- 172.21.15.97:0/1212622987 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5565df4e1980 msgr2=0x5565df4e1da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:02:34.796 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.826+0000 7f8fca7e5740 1 --2- 172.21.15.97:0/1212622987 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5565df4e1980 0x5565df4e1da0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8fc0073cb0 tx=0x7f8fb4009520 comp rx=0 tx=0).stop 2024-03-27T11:02:34.796 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.827+0000 7f8fca7e5740 1 -- 172.21.15.97:0/1212622987 shutdown_connections 2024-03-27T11:02:34.796 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.827+0000 7f8fca7e5740 1 --2- 172.21.15.97:0/1212622987 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565df4e74e0 0x5565df4e9970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:34.797 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.827+0000 7f8fca7e5740 1 --2- 172.21.15.97:0/1212622987 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5565df4e2370 0x5565df4e6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:34.797 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.827+0000 7f8fca7e5740 1 --2- 172.21.15.97:0/1212622987 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5565df4e1980 0x5565df4e1da0 secure :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8fc0073cb0 tx=0x7f8fb4009520 comp rx=0 tx=0).stop 2024-03-27T11:02:34.797 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.827+0000 7f8fca7e5740 1 -- 172.21.15.97:0/1212622987 >> 172.21.15.97:0/1212622987 conn(0x5565df4dcef0 msgr2=0x5565df4dd2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:02:34.797 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.827+0000 7f8fc5000640 1 --2- 172.21.15.97:0/3737700604 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f8fa80a3710 0x7f8fa80a5bd0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x5565df4e2770 tx=0x7f8fb0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:02:34.797 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.828+0000 7f8fca7e5740 1 -- 172.21.15.97:0/1212622987 shutdown_connections 2024-03-27T11:02:34.798 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.828+0000 7f8fca7e5740 1 -- 172.21.15.97:0/1212622987 wait complete. 2024-03-27T11:02:34.799 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.830+0000 7f8fca7e5740 1 -- start start 2024-03-27T11:02:34.800 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.830+0000 7f8fca7e5740 1 Processor -- start 2024-03-27T11:02:34.802 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.831+0000 7f8fca7e5740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5565df4e2370 0x5565df4ebe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:02:34.802 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.832+0000 7f8fca7e5740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565df4e74e0 0x5565df4ec3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:02:34.802 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.832+0000 7f8fca7e5740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5565df4f1390 0x5565df4f3780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:02:34.802 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.832+0000 7f8fca7e5740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5565df4ea430 con 0x5565df4e74e0 2024-03-27T11:02:34.802 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.832+0000 7f8fca7e5740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5565df402630 con 0x5565df4e2370 2024-03-27T11:02:34.802 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.832+0000 7f8fca7e5740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5565df4ea5b0 con 0x5565df4f1390 2024-03-27T11:02:34.802 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.832+0000 7f8fca7e5740 10 client.0 _ll_register_callbacks cb 0x5565df407c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T11:02:34.803 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.833+0000 7f8fc5a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5565df4f1390 0x5565df4f3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:02:34.803 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.833+0000 7f8fc5000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5565df4e2370 0x5565df4ebe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:02:34.803 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.833+0000 7f8fc5000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5565df4e2370 0x5565df4ebe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:52946/0 (socket says 172.21.15.97:52946) 2024-03-27T11:02:34.803 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.833+0000 7f8fc5000640 1 -- 172.21.15.97:0/1082334626 learned_addr learned my addr 172.21.15.97:0/1082334626 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:02:34.803 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.833+0000 7f8fbfe00640 1 --2- 172.21.15.97:0/1082334626 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565df4e74e0 0x5565df4ec3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:02:34.804 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.834+0000 7f8fc5a00640 1 -- 172.21.15.97:0/1082334626 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5565df4e2370 msgr2=0x5565df4ebe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:02:34.804 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.834+0000 7f8fc5a00640 1 --2- 172.21.15.97:0/1082334626 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5565df4e2370 0x5565df4ebe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:34.804 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.834+0000 7f8fc5a00640 1 -- 172.21.15.97:0/1082334626 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565df4e74e0 msgr2=0x5565df4ec3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:02:34.805 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.835+0000 7f8fc5a00640 1 --2- 172.21.15.97:0/1082334626 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565df4e74e0 0x5565df4ec3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:34.805 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.835+0000 7f8fc5a00640 1 -- 172.21.15.97:0/1082334626 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565df40e5f0 con 0x5565df4f1390 2024-03-27T11:02:34.805 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.835+0000 7f8fbfe00640 1 --2- 172.21.15.97:0/1082334626 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565df4e74e0 0x5565df4ec3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T11:02:34.806 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.835+0000 7f8fc5a00640 1 --2- 172.21.15.97:0/1082334626 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5565df4f1390 0x5565df4f3780 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8fb80cdc80 tx=0x7f8fb809cb20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd272b65788aef2d server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:02:34.806 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.836+0000 7f8fa2a00640 10 client.0 ms_handle_connect on v2:172.21.15.177:3300/0 2024-03-27T11:02:34.806 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.836+0000 7f8fa2a00640 1 -- 172.21.15.97:0/1082334626 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb80b0b90 con 0x5565df4f1390 2024-03-27T11:02:34.806 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.836+0000 7f8fa2a00640 1 -- 172.21.15.97:0/1082334626 <== mon.1 v2:172.21.15.177:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f8fb809b730 con 0x5565df4f1390 2024-03-27T11:02:34.806 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.836+0000 7f8fca7e5740 1 -- 172.21.15.97:0/1082334626 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8fb403a040 con 0x5565df4f1390 2024-03-27T11:02:34.806 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.836+0000 7f8fa2a00640 1 -- 172.21.15.97:0/1082334626 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f8fb809b950 con 0x5565df4f1390 2024-03-27T11:02:34.807 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.837+0000 7f8fca7e5740 10 client.25981 fetch_fsmap learned FSMap version 34 2024-03-27T11:02:34.807 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.837+0000 7f8fca7e5740 1 -- 172.21.15.97:0/1082334626 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5565df15b210 con 0x5565df4f1390 2024-03-27T11:02:34.807 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.837+0000 7f8fa2a00640 1 -- 172.21.15.97:0/1082334626 <== mon.1 v2:172.21.15.177:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8fb403a040 con 0x5565df4f1390 2024-03-27T11:02:34.808 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.838+0000 7f8fa2a00640 1 -- 172.21.15.97:0/1082334626 <== mon.1 v2:172.21.15.177:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f8fb809b1a0 con 0x5565df4f1390 2024-03-27T11:02:34.808 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.838+0000 7f8fca7e5740 10 client.25981 fetch_fsmap finished waiting for FSMap version 34 2024-03-27T11:02:34.808 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.838+0000 7f8fca7e5740 10 client.25981 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T11:02:34.808 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.838+0000 7f8fca7e5740 20 client.25981 populate_metadata read hostname 'smithi097' 2024-03-27T11:02:34.808 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.838+0000 7f8fca7e5740 1 --2- 172.21.15.97:0/1082334626 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5565df4f80d0 0x5565df4fa4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:02:34.808 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.838+0000 7f8fca7e5740 4 client.25981 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T11:02:34.808 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.838+0000 7f8fca7e5740 1 -- 172.21.15.97:0/1082334626 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5565df454250 con 0x5565df4f80d0 2024-03-27T11:02:34.809 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.839+0000 7f8fc5000640 1 --2- 172.21.15.97:0/1082334626 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5565df4f80d0 0x5565df4fa4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:02:34.809 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.840+0000 7f8fc5000640 1 --2- 172.21.15.97:0/1082334626 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5565df4f80d0 0x5565df4fa4c0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df79801e45b21a09 server_cookie=4b41651837d826 in_seq=0 out_seq=0 2024-03-27T11:02:34.809 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.840+0000 7f8fa2a00640 10 client.25981 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T11:02:34.812 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.842+0000 7f8fa2a00640 1 -- 172.21.15.97:0/1082334626 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5565df454250 con 0x5565df4f80d0 2024-03-27T11:02:34.812 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.842+0000 7f8fa2a00640 10 client.25981 handle_command_reply: tid=0 2024-03-27T11:02:34.812 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.842+0000 7f8fca7e5740 1 client.25981 shutdown 2024-03-27T11:02:34.813 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.843+0000 7f8fca7e5740 1 -- 172.21.15.97:0/1082334626 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5565df4f1390 msgr2=0x5565df4f3780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:02:34.813 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.843+0000 7f8fca7e5740 1 --2- 172.21.15.97:0/1082334626 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5565df4f1390 0x5565df4f3780 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8fb80cdc80 tx=0x7f8fb809cb20 comp rx=0 tx=0).stop 2024-03-27T11:02:34.813 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.843+0000 7f8fca7e5740 1 -- 172.21.15.97:0/1082334626 shutdown_connections 2024-03-27T11:02:34.813 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.843+0000 7f8fca7e5740 1 --2- 172.21.15.97:0/1082334626 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5565df4f80d0 0x5565df4fa4c0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:34.813 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.843+0000 7f8fca7e5740 1 --2- 172.21.15.97:0/1082334626 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5565df4f1390 0x5565df4f3780 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:34.814 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.843+0000 7f8fca7e5740 1 --2- 172.21.15.97:0/1082334626 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565df4e74e0 0x5565df4ec3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:34.814 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.843+0000 7f8fca7e5740 1 --2- 172.21.15.97:0/1082334626 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5565df4e2370 0x5565df4ebe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:34.814 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.843+0000 7f8fca7e5740 1 -- 172.21.15.97:0/1082334626 >> 172.21.15.97:0/1082334626 conn(0x5565df4dcef0 msgr2=0x5565df4dc0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:02:34.814 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.843+0000 7f8fa2a00640 0 client.25981 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T11:02:34.814 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.844+0000 7f8fca7e5740 1 -- 172.21.15.97:0/1082334626 shutdown_connections 2024-03-27T11:02:34.814 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.844+0000 7f8fca7e5740 1 -- 172.21.15.97:0/1082334626 wait complete. 2024-03-27T11:02:34.815 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.844+0000 7f8fca7e5740 20 client.25981 trim_cache size 0 max 16384 2024-03-27T11:02:34.849 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.880+0000 7f8fca7e5740 1 Processor -- start 2024-03-27T11:02:34.850 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.880+0000 7f8fca7e5740 1 -- start start 2024-03-27T11:02:34.850 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.880+0000 7f8fca7e5740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5565df4e2370 0x5565df4e38b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:02:34.851 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.880+0000 7f8fca7e5740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565df4e74e0 0x5565df4e3df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:02:34.851 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.880+0000 7f8fca7e5740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5565df4f1390 0x5565df4db860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:02:34.851 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.881+0000 7f8fca7e5740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc01b7cc0 con 0x5565df4e74e0 2024-03-27T11:02:34.851 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.881+0000 7f8fca7e5740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc01b9800 con 0x5565df4f1390 2024-03-27T11:02:34.851 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.881+0000 7f8fca7e5740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc01b8390 con 0x5565df4e2370 2024-03-27T11:02:34.851 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.881+0000 7f8fc5a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5565df4f1390 0x5565df4db860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:02:34.852 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.882+0000 7f8fc5a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5565df4f1390 0x5565df4db860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:52948/0 (socket says 172.21.15.97:52948) 2024-03-27T11:02:34.852 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.881+0000 7f8fbfe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565df4e74e0 0x5565df4e3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:02:34.852 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.882+0000 7f8fc5000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5565df4e2370 0x5565df4e38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:02:34.852 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.882+0000 7f8fbfe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565df4e74e0 0x5565df4e3df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55436/0 (socket says 172.21.15.97:55436) 2024-03-27T11:02:34.852 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.882+0000 7f8fbfe00640 1 -- 172.21.15.97:0/1848116478 learned_addr learned my addr 172.21.15.97:0/1848116478 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:02:34.853 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.883+0000 7f8fbfe00640 1 -- 172.21.15.97:0/1848116478 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5565df4e2370 msgr2=0x5565df4e38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:02:34.853 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.883+0000 7f8fbfe00640 1 --2- 172.21.15.97:0/1848116478 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5565df4e2370 0x5565df4e38b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:34.853 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.883+0000 7f8fbfe00640 1 -- 172.21.15.97:0/1848116478 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5565df4f1390 msgr2=0x5565df4db860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:02:34.853 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.883+0000 7f8fbfe00640 1 --2- 172.21.15.97:0/1848116478 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5565df4f1390 0x5565df4db860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:34.853 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.883+0000 7f8fbfe00640 1 -- 172.21.15.97:0/1848116478 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565df227620 con 0x5565df4e74e0 2024-03-27T11:02:34.854 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.884+0000 7f8fbfe00640 1 --2- 172.21.15.97:0/1848116478 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565df4e74e0 0x5565df4e3df0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4005b60 tx=0x7f8fb4034780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e46e03cc53dd268 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:02:34.855 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.885+0000 7f8fad600640 1 -- 172.21.15.97:0/1848116478 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb4045070 con 0x5565df4e74e0 2024-03-27T11:02:34.855 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.885+0000 7f8fad600640 1 -- 172.21.15.97:0/1848116478 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8fb40054f0 con 0x5565df4e74e0 2024-03-27T11:02:34.856 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.886+0000 7f8fad600640 1 -- 172.21.15.97:0/1848116478 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb4039040 con 0x5565df4e74e0 2024-03-27T11:02:34.856 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.886+0000 7f8fca7e5740 1 -- 172.21.15.97:0/1848116478 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565df4e74e0 msgr2=0x5565df4e3df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:02:34.856 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.886+0000 7f8fca7e5740 1 --2- 172.21.15.97:0/1848116478 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565df4e74e0 0x5565df4e3df0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4005b60 tx=0x7f8fb4034780 comp rx=0 tx=0).stop 2024-03-27T11:02:34.856 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.887+0000 7f8fca7e5740 1 -- 172.21.15.97:0/1848116478 shutdown_connections 2024-03-27T11:02:34.857 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.887+0000 7f8fca7e5740 1 --2- 172.21.15.97:0/1848116478 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5565df4f1390 0x5565df4db860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:34.857 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.887+0000 7f8fca7e5740 1 --2- 172.21.15.97:0/1848116478 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565df4e74e0 0x5565df4e3df0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:34.857 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.887+0000 7f8fca7e5740 1 --2- 172.21.15.97:0/1848116478 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5565df4e2370 0x5565df4e38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:34.857 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.887+0000 7f8fca7e5740 1 -- 172.21.15.97:0/1848116478 >> 172.21.15.97:0/1848116478 conn(0x5565df4dcef0 msgr2=0x5565df4ebe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:02:34.858 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.888+0000 7f8fca7e5740 1 -- 172.21.15.97:0/1848116478 shutdown_connections 2024-03-27T11:02:34.858 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.888+0000 7f8fca7e5740 1 -- 172.21.15.97:0/1848116478 wait complete. 2024-03-27T11:02:34.859 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.889+0000 7f8fca7e5740 1 -- start start 2024-03-27T11:02:34.859 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.890+0000 7f8fca7e5740 1 Processor -- start 2024-03-27T11:02:34.860 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.891+0000 7f8fca7e5740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5565df4e2370 0x5565df508bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:02:34.861 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.891+0000 7f8fca7e5740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565df4e74e0 0x5565df509110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:02:34.861 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.891+0000 7f8fca7e5740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5565df4f1390 0x5565df50a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:02:34.861 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.891+0000 7f8fca7e5740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc01b9130 con 0x5565df4e74e0 2024-03-27T11:02:34.861 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.891+0000 7f8fca7e5740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc01b8a60 con 0x5565df4f1390 2024-03-27T11:02:34.861 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.891+0000 7f8fca7e5740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc01b5ca0 con 0x5565df4e2370 2024-03-27T11:02:34.861 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.891+0000 7f8fca7e5740 10 client.0 _ll_register_callbacks cb 0x7f8fb8005850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T11:02:34.862 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.891+0000 7f8fc5000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5565df4e2370 0x5565df508bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:02:34.862 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.891+0000 7f8fc5000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5565df4e2370 0x5565df508bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.97:39278/0 (socket says 172.21.15.97:39278) 2024-03-27T11:02:34.862 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.892+0000 7f8fc5000640 1 -- 172.21.15.97:0/2747722736 learned_addr learned my addr 172.21.15.97:0/2747722736 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:02:34.862 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.892+0000 7f8fbfe00640 1 --2- 172.21.15.97:0/2747722736 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565df4e74e0 0x5565df509110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:02:34.862 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.892+0000 7f8fc5000640 1 -- 172.21.15.97:0/2747722736 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5565df4f1390 msgr2=0x5565df50a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:02:34.862 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.892+0000 7f8fc5000640 1 --2- 172.21.15.97:0/2747722736 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5565df4f1390 0x5565df50a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:34.862 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.892+0000 7f8fc5000640 1 -- 172.21.15.97:0/2747722736 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565df4e74e0 msgr2=0x5565df509110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:02:34.863 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.892+0000 7f8fc5000640 1 --2- 172.21.15.97:0/2747722736 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565df4e74e0 0x5565df509110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:34.863 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.893+0000 7f8fc5000640 1 -- 172.21.15.97:0/2747722736 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565df50a650 con 0x5565df4e2370 2024-03-27T11:02:34.863 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.893+0000 7f8fbfe00640 1 --2- 172.21.15.97:0/2747722736 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565df4e74e0 0x5565df509110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T11:02:34.863 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.893+0000 7f8fc5000640 1 --2- 172.21.15.97:0/2747722736 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5565df4e2370 0x5565df508bd0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8fb0009480 tx=0x7f8fb0004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=21cde26d2061e07b server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:02:34.863 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.893+0000 7f8fa2a00640 10 client.0 ms_handle_connect on v2:172.21.15.177:3300/0 2024-03-27T11:02:34.864 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.894+0000 7f8fa2a00640 1 -- 172.21.15.97:0/2747722736 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb00039f0 con 0x5565df4e2370 2024-03-27T11:02:34.864 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.894+0000 7f8fa2a00640 1 -- 172.21.15.97:0/2747722736 <== mon.1 v2:172.21.15.177:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f8fb0003b90 con 0x5565df4e2370 2024-03-27T11:02:34.864 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.895+0000 7f8fca7e5740 1 -- 172.21.15.97:0/2747722736 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8fb40054f0 con 0x5565df4e2370 2024-03-27T11:02:34.865 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.895+0000 7f8fa2a00640 1 -- 172.21.15.97:0/2747722736 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f8fb0037050 con 0x5565df4e2370 2024-03-27T11:02:34.865 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.895+0000 7f8fca7e5740 10 client.25987 fetch_fsmap learned FSMap version 34 2024-03-27T11:02:34.865 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.895+0000 7f8fca7e5740 1 -- 172.21.15.97:0/2747722736 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5565df55dd80 con 0x5565df4e2370 2024-03-27T11:02:34.866 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.896+0000 7f8fa2a00640 1 -- 172.21.15.97:0/2747722736 <== mon.1 v2:172.21.15.177:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8fb40054f0 con 0x5565df4e2370 2024-03-27T11:02:34.866 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.896+0000 7f8fa2a00640 1 -- 172.21.15.97:0/2747722736 <== mon.1 v2:172.21.15.177:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f8fb00325f0 con 0x5565df4e2370 2024-03-27T11:02:34.866 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.896+0000 7f8fca7e5740 10 client.25987 fetch_fsmap finished waiting for FSMap version 34 2024-03-27T11:02:34.867 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.896+0000 7f8fca7e5740 10 client.25987 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T11:02:34.867 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.896+0000 7f8fca7e5740 20 client.25987 populate_metadata read hostname 'smithi097' 2024-03-27T11:02:34.867 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.896+0000 7f8fca7e5740 1 --2- 172.21.15.97:0/2747722736 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5565df55e6f0 0x5565df560b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:02:34.867 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.896+0000 7f8fca7e5740 4 client.25987 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-27T11:02:34.867 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.897+0000 7f8fca7e5740 1 -- 172.21.15.97:0/2747722736 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5565df44e5a0 con 0x5565df55e6f0 2024-03-27T11:02:34.867 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.897+0000 7f8fbfe00640 1 --2- 172.21.15.97:0/2747722736 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5565df55e6f0 0x5565df560b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:02:34.869 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.899+0000 7f8fbfe00640 1 --2- 172.21.15.97:0/2747722736 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5565df55e6f0 0x5565df560b10 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=517888cfbadb564b server_cookie=5f81fe0fa0970931 in_seq=0 out_seq=0 2024-03-27T11:02:34.869 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.899+0000 7f8fa2a00640 10 client.25987 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T11:02:34.877 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.907+0000 7f8fa2a00640 1 -- 172.21.15.97:0/2747722736 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5565df44e5a0 con 0x5565df55e6f0 2024-03-27T11:02:34.878 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.908+0000 7f8fa2a00640 10 client.25987 handle_command_reply: tid=0 2024-03-27T11:02:34.878 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.908+0000 7f8fca7e5740 1 client.25987 shutdown 2024-03-27T11:02:34.879 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.908+0000 7f8fca7e5740 1 -- 172.21.15.97:0/2747722736 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5565df4e2370 msgr2=0x5565df508bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:02:34.879 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.908+0000 7f8fca7e5740 1 --2- 172.21.15.97:0/2747722736 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5565df4e2370 0x5565df508bd0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8fb0009480 tx=0x7f8fb0004730 comp rx=0 tx=0).stop 2024-03-27T11:02:34.879 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.909+0000 7f8fca7e5740 1 -- 172.21.15.97:0/2747722736 shutdown_connections 2024-03-27T11:02:34.879 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.909+0000 7f8fca7e5740 1 --2- 172.21.15.97:0/2747722736 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5565df55e6f0 0x5565df560b10 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:34.879 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.909+0000 7f8fa2a00640 0 client.25987 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T11:02:34.879 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.909+0000 7f8fca7e5740 1 --2- 172.21.15.97:0/2747722736 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5565df4f1390 0x5565df50a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:34.880 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.910+0000 7f8fca7e5740 1 --2- 172.21.15.97:0/2747722736 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565df4e74e0 0x5565df509110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:34.880 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.910+0000 7f8fca7e5740 1 --2- 172.21.15.97:0/2747722736 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5565df4e2370 0x5565df508bd0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:34.880 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.910+0000 7f8fca7e5740 1 -- 172.21.15.97:0/2747722736 >> 172.21.15.97:0/2747722736 conn(0x5565df4dcef0 msgr2=0x5565df4ebe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:02:34.881 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.912+0000 7f8fca7e5740 1 -- 172.21.15.97:0/2747722736 shutdown_connections 2024-03-27T11:02:34.882 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.912+0000 7f8fca7e5740 1 -- 172.21.15.97:0/2747722736 wait complete. 2024-03-27T11:02:34.882 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.912+0000 7f8fca7e5740 20 client.25987 trim_cache size 0 max 16384 2024-03-27T11:02:34.883 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-27T11:02:34.883 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-27T11:02:34.883 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-27T11:02:34.883 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-27T11:02:34.885 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.914+0000 7f8fc7a00640 1 -- 172.21.15.97:0/3737700604 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f8fa80a3710 msgr2=0x7f8fa80a5bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:02:34.885 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.914+0000 7f8fc7a00640 1 --2- 172.21.15.97:0/3737700604 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f8fa80a3710 0x7f8fa80a5bd0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x5565df4e2770 tx=0x7f8fb0009260 comp rx=0 tx=0).stop 2024-03-27T11:02:34.885 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.914+0000 7f8fc7a00640 1 -- 172.21.15.97:0/3737700604 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8fc010b780 msgr2=0x7f8fc01ae0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:02:34.885 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.914+0000 7f8fc7a00640 1 --2- 172.21.15.97:0/3737700604 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8fc010b780 0x7f8fc01ae0e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8fb800eb10 tx=0x7f8fb800ef10 comp rx=0 tx=0).stop 2024-03-27T11:02:34.885 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.915+0000 7f8fc7a00640 1 -- 172.21.15.97:0/3737700604 shutdown_connections 2024-03-27T11:02:34.885 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.915+0000 7f8fc7a00640 1 --2- 172.21.15.97:0/3737700604 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8fc010b780 0x7f8fc01ae0e0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:34.885 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.915+0000 7f8fc7a00640 1 --2- 172.21.15.97:0/3737700604 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fc0108dc0 0x7f8fc01a7070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:34.885 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.915+0000 7f8fc7a00640 1 --2- 172.21.15.97:0/3737700604 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f8fa80a3710 0x7f8fa80a5bd0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:34.885 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.915+0000 7f8fc7a00640 1 --2- 172.21.15.97:0/3737700604 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8fc006a9a0 0x7f8fc01a6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:34.885 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.915+0000 7f8fc7a00640 1 -- 172.21.15.97:0/3737700604 >> 172.21.15.97:0/3737700604 conn(0x7f8fc006ed50 msgr2=0x7f8fc010c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:02:34.885 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.915+0000 7f8fc7a00640 1 -- 172.21.15.97:0/3737700604 shutdown_connections 2024-03-27T11:02:34.885 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:34.915+0000 7f8fc7a00640 1 -- 172.21.15.97:0/3737700604 wait complete. 2024-03-27T11:02:35.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:02:35 smithi097 ceph-mon[31859]: pgmap v619: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 55 MiB/s wr, 894 op/s 2024-03-27T11:02:35.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:02:35 smithi177 ceph-mon[35294]: pgmap v619: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 55 MiB/s wr, 894 op/s 2024-03-27T11:02:35.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:02:35 smithi114 ceph-mon[35338]: pgmap v619: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 55 MiB/s wr, 894 op/s 2024-03-27T11:02:37.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:02:37 smithi097 ceph-mon[31859]: pgmap v620: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 53 MiB/s wr, 852 op/s 2024-03-27T11:02:37.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:02:37 smithi177 ceph-mon[35294]: pgmap v620: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 53 MiB/s wr, 852 op/s 2024-03-27T11:02:37.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:02:37 smithi114 ceph-mon[35338]: pgmap v620: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 53 MiB/s wr, 852 op/s 2024-03-27T11:02:39.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:02:39 smithi097 ceph-mon[31859]: pgmap v621: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 54 MiB/s wr, 878 op/s 2024-03-27T11:02:39.678 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:02:39 smithi177 ceph-mon[35294]: pgmap v621: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 54 MiB/s wr, 878 op/s 2024-03-27T11:02:39.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:02:39 smithi114 ceph-mon[35338]: pgmap v621: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 54 MiB/s wr, 878 op/s 2024-03-27T11:02:41.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:02:41 smithi097 ceph-mon[31859]: pgmap v622: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 59 MiB/s wr, 951 op/s 2024-03-27T11:02:41.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:02:41 smithi177 ceph-mon[35294]: pgmap v622: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 59 MiB/s wr, 951 op/s 2024-03-27T11:02:41.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:02:41 smithi114 ceph-mon[35338]: pgmap v622: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 59 MiB/s wr, 951 op/s 2024-03-27T11:02:43.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:02:43 smithi097 ceph-mon[31859]: pgmap v623: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 52 MiB/s wr, 846 op/s 2024-03-27T11:02:43.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:02:43 smithi177 ceph-mon[35294]: pgmap v623: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 52 MiB/s wr, 846 op/s 2024-03-27T11:02:43.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:02:43 smithi114 ceph-mon[35338]: pgmap v623: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 52 MiB/s wr, 846 op/s 2024-03-27T11:02:45.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:02:45 smithi097 ceph-mon[31859]: pgmap v624: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 53 MiB/s wr, 861 op/s 2024-03-27T11:02:45.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:02:45 smithi177 ceph-mon[35294]: pgmap v624: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 53 MiB/s wr, 861 op/s 2024-03-27T11:02:45.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:02:45 smithi114 ceph-mon[35338]: pgmap v624: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 53 MiB/s wr, 861 op/s 2024-03-27T11:02:47.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:02:47 smithi097 ceph-mon[31859]: pgmap v625: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 52 MiB/s wr, 839 op/s 2024-03-27T11:02:47.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:02:47 smithi177 ceph-mon[35294]: pgmap v625: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 52 MiB/s wr, 839 op/s 2024-03-27T11:02:47.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:02:47 smithi114 ceph-mon[35338]: pgmap v625: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 52 MiB/s wr, 839 op/s 2024-03-27T11:02:48.098 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.127+0000 7f4835000640 1 -- 172.21.15.97:0/3280572349 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48301093a0 msgr2=0x7f4830109820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:02:48.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.127+0000 7f4835000640 1 --2- 172.21.15.97:0/3280572349 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48301093a0 0x7f4830109820 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f4818009900 tx=0x7f481802bc40 comp rx=0 tx=0).stop 2024-03-27T11:02:48.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.128+0000 7f4835000640 1 -- 172.21.15.97:0/3280572349 shutdown_connections 2024-03-27T11:02:48.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.128+0000 7f4835000640 1 --2- 172.21.15.97:0/3280572349 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4830109d60 0x7f48301105f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:48.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.128+0000 7f4835000640 1 --2- 172.21.15.97:0/3280572349 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48301093a0 0x7f4830109820 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:48.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.128+0000 7f4835000640 1 --2- 172.21.15.97:0/3280572349 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f48301081b0 0x7f48301085d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:48.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.128+0000 7f4835000640 1 -- 172.21.15.97:0/3280572349 >> 172.21.15.97:0/3280572349 conn(0x7f4830077160 msgr2=0x7f4830079580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:02:48.100 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.129+0000 7f4835000640 1 -- 172.21.15.97:0/3280572349 shutdown_connections 2024-03-27T11:02:48.100 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.129+0000 7f4835000640 1 -- 172.21.15.97:0/3280572349 wait complete. 2024-03-27T11:02:48.101 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.130+0000 7f4835000640 1 Processor -- start 2024-03-27T11:02:48.101 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.131+0000 7f4835000640 1 -- start start 2024-03-27T11:02:48.102 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.131+0000 7f4835000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f48301081b0 0x7f48300704a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:02:48.102 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.131+0000 7f4835000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f48301093a0 0x7f48300709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:02:48.102 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.131+0000 7f4835000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4830109d60 0x7f48301ae410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:02:48.103 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.132+0000 7f4835000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4830113d40 con 0x7f4830109d60 2024-03-27T11:02:48.103 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.132+0000 7f4835000640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f4830113bc0 con 0x7f48301081b0 2024-03-27T11:02:48.103 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.132+0000 7f4835000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f4830113ec0 con 0x7f48301093a0 2024-03-27T11:02:48.103 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.132+0000 7f482de00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f48301081b0 0x7f48300704a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:02:48.103 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.132+0000 7f482de00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f48301081b0 0x7f48300704a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:46040/0 (socket says 172.21.15.97:46040) 2024-03-27T11:02:48.103 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.132+0000 7f482de00640 1 -- 172.21.15.97:0/513358870 learned_addr learned my addr 172.21.15.97:0/513358870 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:02:48.103 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.132+0000 7f4825400640 1 --2- 172.21.15.97:0/513358870 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f48301093a0 0x7f48300709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:02:48.104 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.132+0000 7f482e800640 1 --2- 172.21.15.97:0/513358870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4830109d60 0x7f48301ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:02:48.104 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.133+0000 7f482de00640 1 -- 172.21.15.97:0/513358870 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f48301093a0 msgr2=0x7f48300709e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:02:48.104 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.133+0000 7f482de00640 1 --2- 172.21.15.97:0/513358870 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f48301093a0 0x7f48300709e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:48.104 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.133+0000 7f482de00640 1 -- 172.21.15.97:0/513358870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4830109d60 msgr2=0x7f48301ae410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:02:48.105 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.133+0000 7f482de00640 1 --2- 172.21.15.97:0/513358870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4830109d60 0x7f48301ae410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:48.105 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.133+0000 7f482de00640 1 -- 172.21.15.97:0/513358870 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4830071080 con 0x7f48301081b0 2024-03-27T11:02:48.105 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.133+0000 7f482e800640 1 --2- 172.21.15.97:0/513358870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4830109d60 0x7f48301ae410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T11:02:48.105 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.133+0000 7f482de00640 1 --2- 172.21.15.97:0/513358870 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f48301081b0 0x7f48300704a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4820009d20 tx=0x7f482000c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:02:48.105 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.134+0000 7f4827400640 1 -- 172.21.15.97:0/513358870 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4820010630 con 0x7f48301081b0 2024-03-27T11:02:48.105 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.134+0000 7f4827400640 1 -- 172.21.15.97:0/513358870 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4820010c80 con 0x7f48301081b0 2024-03-27T11:02:48.105 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.134+0000 7f4827400640 1 -- 172.21.15.97:0/513358870 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4820018d70 con 0x7f48301081b0 2024-03-27T11:02:48.105 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.134+0000 7f4835000640 1 -- 172.21.15.97:0/513358870 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48301aea70 con 0x7f48301081b0 2024-03-27T11:02:48.105 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.134+0000 7f4835000640 1 -- 172.21.15.97:0/513358870 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48301aef40 con 0x7f48301081b0 2024-03-27T11:02:48.114 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.142+0000 7f4837c4f740 1 Processor -- start 2024-03-27T11:02:48.114 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.142+0000 7f4827400640 1 -- 172.21.15.97:0/513358870 <== mon.2 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f482001f020 con 0x7f48301081b0 2024-03-27T11:02:48.114 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.143+0000 7f4837c4f740 1 -- start start 2024-03-27T11:02:48.114 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.144+0000 7f4837c4f740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cd89b55980 0x55cd89b55da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:02:48.115 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.144+0000 7f4837c4f740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cd89b56370 0x55cd89b5ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:02:48.115 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.144+0000 7f4837c4f740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cd89b5b4e0 0x55cd89b5d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:02:48.115 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.144+0000 7f4837c4f740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55cd89abad10 con 0x55cd89b5b4e0 2024-03-27T11:02:48.115 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.144+0000 7f4837c4f740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55cd89a63260 con 0x55cd89b56370 2024-03-27T11:02:48.115 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.144+0000 7f4837c4f740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55cd89ab6cc0 con 0x55cd89b55980 2024-03-27T11:02:48.115 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.145+0000 7f4825400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cd89b5b4e0 0x55cd89b5d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:02:48.116 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.145+0000 7f4825400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cd89b5b4e0 0x55cd89b5d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35096/0 (socket says 172.21.15.97:35096) 2024-03-27T11:02:48.116 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.145+0000 7f4825400640 1 -- 172.21.15.97:0/66655454 learned_addr learned my addr 172.21.15.97:0/66655454 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:02:48.117 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.145+0000 7f4825400640 1 -- 172.21.15.97:0/66655454 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cd89b55980 msgr2=0x55cd89b55da0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-27T11:02:48.117 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.146+0000 7f4825400640 1 --2- 172.21.15.97:0/66655454 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cd89b55980 0x55cd89b55da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:48.117 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.146+0000 7f4825400640 1 -- 172.21.15.97:0/66655454 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cd89b56370 msgr2=0x55cd89b5ae70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-27T11:02:48.117 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.146+0000 7f4825400640 1 --2- 172.21.15.97:0/66655454 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cd89b56370 0x55cd89b5ae70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:48.117 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.146+0000 7f4825400640 1 -- 172.21.15.97:0/66655454 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd89a5aaa0 con 0x55cd89b5b4e0 2024-03-27T11:02:48.117 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.146+0000 7f4825400640 1 --2- 172.21.15.97:0/66655454 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cd89b5b4e0 0x55cd89b5d970 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f481802c850 tx=0x7f4818009610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f04a17cfdf9e1abd server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:02:48.118 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.145+0000 7f4827400640 1 --2- 172.21.15.97:0/513358870 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f48140a3800 0x7f48140a5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:02:48.118 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.147+0000 7f481f400640 1 -- 172.21.15.97:0/66655454 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4818045020 con 0x55cd89b5b4e0 2024-03-27T11:02:48.119 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.148+0000 7f4827400640 1 -- 172.21.15.97:0/513358870 <== mon.2 v2:172.21.15.114:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f48200d0230 con 0x7f48301081b0 2024-03-27T11:02:48.119 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.148+0000 7f482e800640 1 --2- 172.21.15.97:0/513358870 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f48140a3800 0x7f48140a5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:02:48.119 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.149+0000 7f481f400640 1 -- 172.21.15.97:0/66655454 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f481802cab0 con 0x55cd89b5b4e0 2024-03-27T11:02:48.120 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.149+0000 7f481f400640 1 -- 172.21.15.97:0/66655454 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48180046c0 con 0x55cd89b5b4e0 2024-03-27T11:02:48.121 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.150+0000 7f4837c4f740 1 -- 172.21.15.97:0/66655454 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cd89b5b4e0 msgr2=0x55cd89b5d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:02:48.121 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.150+0000 7f482e800640 1 --2- 172.21.15.97:0/513358870 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f48140a3800 0x7f48140a5cc0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x55cd89b56bd0 tx=0x7f4828005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:02:48.122 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.150+0000 7f4837c4f740 1 --2- 172.21.15.97:0/66655454 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cd89b5b4e0 0x55cd89b5d970 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f481802c850 tx=0x7f4818009610 comp rx=0 tx=0).stop 2024-03-27T11:02:48.122 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.151+0000 7f4837c4f740 1 -- 172.21.15.97:0/66655454 shutdown_connections 2024-03-27T11:02:48.122 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.151+0000 7f4837c4f740 1 --2- 172.21.15.97:0/66655454 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cd89b5b4e0 0x55cd89b5d970 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:48.122 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.151+0000 7f4837c4f740 1 --2- 172.21.15.97:0/66655454 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cd89b56370 0x55cd89b5ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:48.122 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.151+0000 7f4837c4f740 1 --2- 172.21.15.97:0/66655454 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cd89b55980 0x55cd89b55da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:48.122 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.151+0000 7f4837c4f740 1 -- 172.21.15.97:0/66655454 >> 172.21.15.97:0/66655454 conn(0x55cd89b50ef0 msgr2=0x55cd89b512f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:02:48.123 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.151+0000 7f4837c4f740 1 -- 172.21.15.97:0/66655454 shutdown_connections 2024-03-27T11:02:48.123 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.151+0000 7f4837c4f740 1 -- 172.21.15.97:0/66655454 wait complete. 2024-03-27T11:02:48.123 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.152+0000 7f4837c4f740 1 -- start start 2024-03-27T11:02:48.123 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.153+0000 7f4837c4f740 1 Processor -- start 2024-03-27T11:02:48.124 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.153+0000 7f4837c4f740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cd89b55980 0x55cd89b5fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:02:48.124 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.153+0000 7f4837c4f740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cd89b56370 0x55cd89b60230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:02:48.125 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.153+0000 7f4837c4f740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cd89b5b4e0 0x55cd89b67270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:02:48.125 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.153+0000 7f4837c4f740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55cd89b5e430 con 0x55cd89b55980 2024-03-27T11:02:48.125 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.153+0000 7f4837c4f740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55cd89a76630 con 0x55cd89b56370 2024-03-27T11:02:48.125 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.153+0000 7f4837c4f740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55cd89b5e5b0 con 0x55cd89b5b4e0 2024-03-27T11:02:48.125 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.153+0000 7f4837c4f740 10 client.0 _ll_register_callbacks cb 0x55cd89a7bc50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T11:02:48.125 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.154+0000 7f4825400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cd89b56370 0x55cd89b60230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:02:48.125 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.154+0000 7f4825400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cd89b56370 0x55cd89b60230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:46058/0 (socket says 172.21.15.97:46058) 2024-03-27T11:02:48.125 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.154+0000 7f4825400640 1 -- 172.21.15.97:0/38009623 learned_addr learned my addr 172.21.15.97:0/38009623 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:02:48.126 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.154+0000 7f482e800640 1 --2- 172.21.15.97:0/38009623 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cd89b5b4e0 0x55cd89b67270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:02:48.126 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.154+0000 7f482de00640 1 --2- 172.21.15.97:0/38009623 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cd89b55980 0x55cd89b5fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:02:48.126 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.155+0000 7f4825400640 1 -- 172.21.15.97:0/38009623 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cd89b5b4e0 msgr2=0x55cd89b67270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:02:48.126 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.155+0000 7f4825400640 1 --2- 172.21.15.97:0/38009623 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cd89b5b4e0 0x55cd89b67270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:48.126 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.155+0000 7f4825400640 1 -- 172.21.15.97:0/38009623 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cd89b55980 msgr2=0x55cd89b5fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:02:48.126 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.155+0000 7f4825400640 1 --2- 172.21.15.97:0/38009623 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cd89b55980 0x55cd89b5fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:48.126 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.155+0000 7f4825400640 1 -- 172.21.15.97:0/38009623 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd89a825f0 con 0x55cd89b56370 2024-03-27T11:02:48.126 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.155+0000 7f482de00640 1 --2- 172.21.15.97:0/38009623 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cd89b55980 0x55cd89b5fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T11:02:48.126 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.155+0000 7f4825400640 1 --2- 172.21.15.97:0/38009623 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cd89b56370 0x55cd89b60230 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4818036040 tx=0x7f4818009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2fcc81307739ddd9 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:02:48.127 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.155+0000 7f4813e00640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-03-27T11:02:48.127 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.156+0000 7f4813e00640 1 -- 172.21.15.97:0/38009623 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4818051070 con 0x55cd89b56370 2024-03-27T11:02:48.127 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.156+0000 7f4813e00640 1 -- 172.21.15.97:0/38009623 <== mon.2 v2:172.21.15.114:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f4818003a00 con 0x55cd89b56370 2024-03-27T11:02:48.127 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.156+0000 7f4837c4f740 1 -- 172.21.15.97:0/38009623 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f481802cab0 con 0x55cd89b56370 2024-03-27T11:02:48.127 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.156+0000 7f4813e00640 1 -- 172.21.15.97:0/38009623 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f4818045030 con 0x55cd89b56370 2024-03-27T11:02:48.128 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.156+0000 7f4837c4f740 10 client.36281 fetch_fsmap learned FSMap version 34 2024-03-27T11:02:48.128 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.156+0000 7f4837c4f740 1 -- 172.21.15.97:0/38009623 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55cd897cf210 con 0x55cd89b56370 2024-03-27T11:02:48.128 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.157+0000 7f4813e00640 1 -- 172.21.15.97:0/38009623 <== mon.2 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4818003c00 con 0x55cd89b56370 2024-03-27T11:02:48.128 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.157+0000 7f4813e00640 1 -- 172.21.15.97:0/38009623 <== mon.2 v2:172.21.15.114:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f4818054020 con 0x55cd89b56370 2024-03-27T11:02:48.128 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.158+0000 7f4837c4f740 10 client.36281 fetch_fsmap finished waiting for FSMap version 34 2024-03-27T11:02:48.128 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.158+0000 7f4837c4f740 10 client.36281 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T11:02:48.129 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.158+0000 7f4837c4f740 20 client.36281 populate_metadata read hostname 'smithi097' 2024-03-27T11:02:48.129 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.158+0000 7f4837c4f740 1 --2- 172.21.15.97:0/38009623 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55cd89b6ba10 0x55cd89b6de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:02:48.129 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.158+0000 7f4837c4f740 4 client.36281 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T11:02:48.129 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.158+0000 7f4837c4f740 1 -- 172.21.15.97:0/38009623 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f481802cab0 con 0x55cd89b6ba10 2024-03-27T11:02:48.130 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.159+0000 7f482de00640 1 --2- 172.21.15.97:0/38009623 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55cd89b6ba10 0x55cd89b6de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:02:48.130 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.160+0000 7f482de00640 1 --2- 172.21.15.97:0/38009623 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55cd89b6ba10 0x55cd89b6de00 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6992dd162b97dee5 server_cookie=f2ed009a7fd5077a in_seq=0 out_seq=0 2024-03-27T11:02:48.130 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.160+0000 7f4813e00640 10 client.36281 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T11:02:48.131 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.161+0000 7f4813e00640 1 -- 172.21.15.97:0/38009623 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x7f481802cab0 con 0x55cd89b6ba10 2024-03-27T11:02:48.131 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.161+0000 7f4813e00640 10 client.36281 handle_command_reply: tid=0 2024-03-27T11:02:48.132 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.161+0000 7f4837c4f740 1 client.36281 shutdown 2024-03-27T11:02:48.132 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.162+0000 7f4837c4f740 1 -- 172.21.15.97:0/38009623 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cd89b56370 msgr2=0x55cd89b60230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:02:48.132 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.162+0000 7f4837c4f740 1 --2- 172.21.15.97:0/38009623 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cd89b56370 0x55cd89b60230 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4818036040 tx=0x7f4818009260 comp rx=0 tx=0).stop 2024-03-27T11:02:48.133 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.162+0000 7f4837c4f740 1 -- 172.21.15.97:0/38009623 shutdown_connections 2024-03-27T11:02:48.133 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.162+0000 7f4837c4f740 1 --2- 172.21.15.97:0/38009623 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55cd89b6ba10 0x55cd89b6de00 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:48.133 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.162+0000 7f4837c4f740 1 --2- 172.21.15.97:0/38009623 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cd89b5b4e0 0x55cd89b67270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:48.133 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.162+0000 7f4837c4f740 1 --2- 172.21.15.97:0/38009623 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cd89b56370 0x55cd89b60230 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:48.133 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.162+0000 7f4837c4f740 1 --2- 172.21.15.97:0/38009623 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cd89b55980 0x55cd89b5fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:48.133 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.162+0000 7f4837c4f740 1 -- 172.21.15.97:0/38009623 >> 172.21.15.97:0/38009623 conn(0x55cd89b50ef0 msgr2=0x55cd89b500f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:02:48.134 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.162+0000 7f4813e00640 0 client.36281 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T11:02:48.134 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.162+0000 7f4837c4f740 1 -- 172.21.15.97:0/38009623 shutdown_connections 2024-03-27T11:02:48.134 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.163+0000 7f4837c4f740 1 -- 172.21.15.97:0/38009623 wait complete. 2024-03-27T11:02:48.134 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.163+0000 7f4837c4f740 20 client.36281 trim_cache size 0 max 16384 2024-03-27T11:02:48.157 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.187+0000 7f4837c4f740 1 Processor -- start 2024-03-27T11:02:48.157 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.187+0000 7f4837c4f740 1 -- start start 2024-03-27T11:02:48.158 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.188+0000 7f4837c4f740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cd89b55980 0x55cd89b512d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:02:48.158 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.188+0000 7f4837c4f740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cd89b5b4e0 0x55cd89b67230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:02:48.159 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.188+0000 7f4837c4f740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cd89b54150 0x55cd89b545d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:02:48.159 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.188+0000 7f4837c4f740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f480c00e420 con 0x55cd89b55980 2024-03-27T11:02:48.159 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.188+0000 7f4837c4f740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f480c00ff60 con 0x55cd89b5b4e0 2024-03-27T11:02:48.159 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.188+0000 7f4837c4f740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f480c00eaf0 con 0x55cd89b54150 2024-03-27T11:02:48.159 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.188+0000 7f482e800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cd89b54150 0x55cd89b545d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:02:48.160 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.188+0000 7f4825400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cd89b5b4e0 0x55cd89b67230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:02:48.160 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.189+0000 7f482e800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cd89b54150 0x55cd89b545d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.97:33486/0 (socket says 172.21.15.97:33486) 2024-03-27T11:02:48.160 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.189+0000 7f482e800640 1 -- 172.21.15.97:0/1884145304 learned_addr learned my addr 172.21.15.97:0/1884145304 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:02:48.160 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.189+0000 7f482de00640 1 --2- 172.21.15.97:0/1884145304 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cd89b55980 0x55cd89b512d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:02:48.160 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.189+0000 7f4825400640 1 -- 172.21.15.97:0/1884145304 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cd89b54150 msgr2=0x55cd89b545d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:02:48.160 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.189+0000 7f4825400640 1 --2- 172.21.15.97:0/1884145304 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cd89b54150 0x55cd89b545d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:48.160 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.189+0000 7f4825400640 1 -- 172.21.15.97:0/1884145304 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cd89b55980 msgr2=0x55cd89b512d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:02:48.160 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.189+0000 7f4825400640 1 --2- 172.21.15.97:0/1884145304 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cd89b55980 0x55cd89b512d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:48.161 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.189+0000 7f4825400640 1 -- 172.21.15.97:0/1884145304 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd8989b620 con 0x55cd89b5b4e0 2024-03-27T11:02:48.161 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.190+0000 7f4825400640 1 --2- 172.21.15.97:0/1884145304 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cd89b5b4e0 0x55cd89b67230 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f481804b000 tx=0x7f4818005780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f96961308535bb7d server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:02:48.162 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.191+0000 7f481ea00640 1 -- 172.21.15.97:0/1884145304 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f481804e070 con 0x55cd89b5b4e0 2024-03-27T11:02:48.162 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.191+0000 7f481ea00640 1 -- 172.21.15.97:0/1884145304 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f481802cab0 con 0x55cd89b5b4e0 2024-03-27T11:02:48.162 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.191+0000 7f481ea00640 1 -- 172.21.15.97:0/1884145304 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f481804e070 con 0x55cd89b5b4e0 2024-03-27T11:02:48.163 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.192+0000 7f4837c4f740 1 -- 172.21.15.97:0/1884145304 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cd89b5b4e0 msgr2=0x55cd89b67230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:02:48.163 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.192+0000 7f4837c4f740 1 --2- 172.21.15.97:0/1884145304 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cd89b5b4e0 0x55cd89b67230 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f481804b000 tx=0x7f4818005780 comp rx=0 tx=0).stop 2024-03-27T11:02:48.164 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.193+0000 7f4837c4f740 1 -- 172.21.15.97:0/1884145304 shutdown_connections 2024-03-27T11:02:48.164 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.193+0000 7f4837c4f740 1 --2- 172.21.15.97:0/1884145304 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cd89b54150 0x55cd89b545d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:48.164 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.193+0000 7f4837c4f740 1 --2- 172.21.15.97:0/1884145304 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cd89b5b4e0 0x55cd89b67230 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:48.164 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.193+0000 7f4837c4f740 1 --2- 172.21.15.97:0/1884145304 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cd89b55980 0x55cd89b512d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:48.164 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.193+0000 7f4837c4f740 1 -- 172.21.15.97:0/1884145304 >> 172.21.15.97:0/1884145304 conn(0x55cd89b50ef0 msgr2=0x55cd89b4c0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:02:48.165 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.194+0000 7f4837c4f740 1 -- 172.21.15.97:0/1884145304 shutdown_connections 2024-03-27T11:02:48.165 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.194+0000 7f4837c4f740 1 -- 172.21.15.97:0/1884145304 wait complete. 2024-03-27T11:02:48.166 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.195+0000 7f4837c4f740 1 -- start start 2024-03-27T11:02:48.166 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.195+0000 7f4837c4f740 1 Processor -- start 2024-03-27T11:02:48.168 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.196+0000 7f4837c4f740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cd89b54150 0x55cd89b650b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:02:48.168 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.196+0000 7f4837c4f740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cd89b55980 0x55cd89b655f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:02:48.168 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.196+0000 7f4837c4f740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cd89b5b4e0 0x55cd89b66540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:02:48.168 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.196+0000 7f4837c4f740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f480c00f890 con 0x55cd89b55980 2024-03-27T11:02:48.168 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.196+0000 7f4837c4f740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f480c00f1c0 con 0x55cd89b5b4e0 2024-03-27T11:02:48.168 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.196+0000 7f4837c4f740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f480c00c400 con 0x55cd89b54150 2024-03-27T11:02:48.168 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.196+0000 7f4837c4f740 10 client.0 _ll_register_callbacks cb 0x7f4818009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T11:02:48.168 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.197+0000 7f482de00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cd89b54150 0x55cd89b650b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:02:48.168 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.197+0000 7f482de00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cd89b54150 0x55cd89b650b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.97:33500/0 (socket says 172.21.15.97:33500) 2024-03-27T11:02:48.168 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.197+0000 7f482de00640 1 -- 172.21.15.97:0/2008166960 learned_addr learned my addr 172.21.15.97:0/2008166960 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:02:48.168 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.197+0000 7f482e800640 1 --2- 172.21.15.97:0/2008166960 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cd89b5b4e0 0x55cd89b66540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:02:48.170 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.198+0000 7f4825400640 1 --2- 172.21.15.97:0/2008166960 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cd89b55980 0x55cd89b655f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:02:48.170 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.198+0000 7f482e800640 1 -- 172.21.15.97:0/2008166960 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cd89b54150 msgr2=0x55cd89b650b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:02:48.170 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.198+0000 7f482e800640 1 --2- 172.21.15.97:0/2008166960 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cd89b54150 0x55cd89b650b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:48.170 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.198+0000 7f482e800640 1 -- 172.21.15.97:0/2008166960 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cd89b55980 msgr2=0x55cd89b655f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:02:48.170 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.198+0000 7f482e800640 1 --2- 172.21.15.97:0/2008166960 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cd89b55980 0x55cd89b655f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:48.170 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.198+0000 7f482e800640 1 -- 172.21.15.97:0/2008166960 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd89b66b30 con 0x55cd89b5b4e0 2024-03-27T11:02:48.170 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.199+0000 7f482de00640 1 --2- 172.21.15.97:0/2008166960 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cd89b54150 0x55cd89b650b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T11:02:48.170 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.199+0000 7f482e800640 1 --2- 172.21.15.97:0/2008166960 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cd89b5b4e0 0x55cd89b66540 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f482800e6c0 tx=0x7f48280046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34a8b4178cecce9b server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:02:48.170 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.199+0000 7f4813e00640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-03-27T11:02:48.171 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.199+0000 7f4813e00640 1 -- 172.21.15.97:0/2008166960 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4828020b50 con 0x55cd89b5b4e0 2024-03-27T11:02:48.171 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.200+0000 7f4813e00640 1 -- 172.21.15.97:0/2008166960 <== mon.2 v2:172.21.15.114:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f4828020cf0 con 0x55cd89b5b4e0 2024-03-27T11:02:48.171 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.200+0000 7f4837c4f740 1 -- 172.21.15.97:0/2008166960 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f481802cab0 con 0x55cd89b5b4e0 2024-03-27T11:02:48.172 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.200+0000 7f4813e00640 1 -- 172.21.15.97:0/2008166960 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f4828029440 con 0x55cd89b5b4e0 2024-03-27T11:02:48.172 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.200+0000 7f4837c4f740 10 client.36287 fetch_fsmap learned FSMap version 34 2024-03-27T11:02:48.172 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.200+0000 7f4837c4f740 1 -- 172.21.15.97:0/2008166960 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55cd89be3030 con 0x55cd89b5b4e0 2024-03-27T11:02:48.172 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.201+0000 7f4813e00640 1 -- 172.21.15.97:0/2008166960 <== mon.2 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f481802cab0 con 0x55cd89b5b4e0 2024-03-27T11:02:48.172 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.201+0000 7f4813e00640 1 -- 172.21.15.97:0/2008166960 <== mon.2 v2:172.21.15.114:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f4828027070 con 0x55cd89b5b4e0 2024-03-27T11:02:48.173 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.201+0000 7f4837c4f740 10 client.36287 fetch_fsmap finished waiting for FSMap version 34 2024-03-27T11:02:48.173 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.201+0000 7f4837c4f740 10 client.36287 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T11:02:48.173 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.201+0000 7f4837c4f740 20 client.36287 populate_metadata read hostname 'smithi097' 2024-03-27T11:02:48.173 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.201+0000 7f4837c4f740 1 --2- 172.21.15.97:0/2008166960 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55cd89be3a00 0x55cd89be5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:02:48.173 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.201+0000 7f4837c4f740 4 client.36287 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-27T11:02:48.173 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.201+0000 7f4837c4f740 1 -- 172.21.15.97:0/2008166960 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cd89ac8250 con 0x55cd89be3a00 2024-03-27T11:02:48.173 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.202+0000 7f482de00640 1 --2- 172.21.15.97:0/2008166960 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55cd89be3a00 0x55cd89be5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:02:48.174 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.203+0000 7f482de00640 1 --2- 172.21.15.97:0/2008166960 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55cd89be3a00 0x55cd89be5e20 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=beb45fd16d2eb7dd server_cookie=113f5ef9e9ebb59c in_seq=0 out_seq=0 2024-03-27T11:02:48.174 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.203+0000 7f4813e00640 10 client.36287 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T11:02:48.182 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.211+0000 7f4813e00640 1 -- 172.21.15.97:0/2008166960 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cd89ac8250 con 0x55cd89be3a00 2024-03-27T11:02:48.182 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.211+0000 7f4813e00640 10 client.36287 handle_command_reply: tid=0 2024-03-27T11:02:48.184 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.212+0000 7f4837c4f740 1 client.36287 shutdown 2024-03-27T11:02:48.185 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.215+0000 7f4837c4f740 1 -- 172.21.15.97:0/2008166960 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cd89b5b4e0 msgr2=0x55cd89b66540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:02:48.186 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.215+0000 7f4837c4f740 1 --2- 172.21.15.97:0/2008166960 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cd89b5b4e0 0x55cd89b66540 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f482800e6c0 tx=0x7f48280046c0 comp rx=0 tx=0).stop 2024-03-27T11:02:48.186 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.215+0000 7f4837c4f740 1 -- 172.21.15.97:0/2008166960 shutdown_connections 2024-03-27T11:02:48.186 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.215+0000 7f4837c4f740 1 --2- 172.21.15.97:0/2008166960 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55cd89be3a00 0x55cd89be5e20 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:48.186 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.215+0000 7f4837c4f740 1 --2- 172.21.15.97:0/2008166960 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cd89b5b4e0 0x55cd89b66540 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:48.186 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.215+0000 7f4837c4f740 1 --2- 172.21.15.97:0/2008166960 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cd89b55980 0x55cd89b655f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:48.187 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.215+0000 7f4837c4f740 1 --2- 172.21.15.97:0/2008166960 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cd89b54150 0x55cd89b650b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:48.187 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.215+0000 7f4837c4f740 1 -- 172.21.15.97:0/2008166960 >> 172.21.15.97:0/2008166960 conn(0x55cd89b50ef0 msgr2=0x55cd89b512d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:02:48.187 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.215+0000 7f4813e00640 0 client.36287 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T11:02:48.187 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.216+0000 7f4837c4f740 1 -- 172.21.15.97:0/2008166960 shutdown_connections 2024-03-27T11:02:48.188 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-27T11:02:48.188 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-27T11:02:48.188 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-27T11:02:48.188 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-27T11:02:48.188 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.216+0000 7f4837c4f740 1 -- 172.21.15.97:0/2008166960 wait complete. 2024-03-27T11:02:48.188 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.216+0000 7f4837c4f740 20 client.36287 trim_cache size 0 max 16384 2024-03-27T11:02:48.189 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.218+0000 7f4835000640 1 -- 172.21.15.97:0/513358870 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f48140a3800 msgr2=0x7f48140a5cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:02:48.189 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.218+0000 7f4835000640 1 --2- 172.21.15.97:0/513358870 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f48140a3800 0x7f48140a5cc0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x55cd89b56bd0 tx=0x7f4828005e50 comp rx=0 tx=0).stop 2024-03-27T11:02:48.189 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.218+0000 7f4835000640 1 -- 172.21.15.97:0/513358870 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f48301081b0 msgr2=0x7f48300704a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:02:48.189 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.218+0000 7f4835000640 1 --2- 172.21.15.97:0/513358870 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f48301081b0 0x7f48300704a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4820009d20 tx=0x7f482000c6a0 comp rx=0 tx=0).stop 2024-03-27T11:02:48.191 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.220+0000 7f4835000640 1 -- 172.21.15.97:0/513358870 shutdown_connections 2024-03-27T11:02:48.191 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.220+0000 7f4835000640 1 --2- 172.21.15.97:0/513358870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4830109d60 0x7f48301ae410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:48.191 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.220+0000 7f4835000640 1 --2- 172.21.15.97:0/513358870 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f48301093a0 0x7f48300709e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:48.191 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.220+0000 7f4835000640 1 --2- 172.21.15.97:0/513358870 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f48140a3800 0x7f48140a5cc0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:48.191 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.220+0000 7f4835000640 1 --2- 172.21.15.97:0/513358870 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f48301081b0 0x7f48300704a0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:02:48.191 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.220+0000 7f4835000640 1 -- 172.21.15.97:0/513358870 >> 172.21.15.97:0/513358870 conn(0x7f4830077160 msgr2=0x7f483010c640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:02:48.191 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.220+0000 7f4835000640 1 -- 172.21.15.97:0/513358870 shutdown_connections 2024-03-27T11:02:48.191 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:02:48.220+0000 7f4835000640 1 -- 172.21.15.97:0/513358870 wait complete. 2024-03-27T11:02:48.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:02:48 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T11:02:48.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:02:48 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T11:02:48.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:02:48 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T11:02:49.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:02:49 smithi097 ceph-mon[31859]: pgmap v626: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 55 MiB/s wr, 893 op/s 2024-03-27T11:02:49.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:02:49 smithi177 ceph-mon[35294]: pgmap v626: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 55 MiB/s wr, 893 op/s 2024-03-27T11:02:49.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:02:49 smithi114 ceph-mon[35338]: pgmap v626: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 55 MiB/s wr, 893 op/s 2024-03-27T11:02:50.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:02:50 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-27T11:02:50.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:02:50 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-27T11:02:50.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:02:50 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-27T11:02:50.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:02:50 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-27T11:02:50.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:02:50 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-27T11:02:50.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:02:50 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-27T11:02:50.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:02:50 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-27T11:02:50.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:02:50 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-27T11:02:50.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:02:50 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T11:02:50.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:02:50 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T11:02:50.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:02:50 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T11:02:50.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:02:50 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-27T11:02:50.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:02:50 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-27T11:02:50.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:02:50 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-27T11:02:50.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:02:50 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-27T11:02:50.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:02:50 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-27T11:02:50.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:02:50 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-27T11:02:50.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:02:50 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-27T11:02:50.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:02:50 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-27T11:02:50.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:02:50 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T11:02:50.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:02:50 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T11:02:50.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:02:50 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T11:02:50.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:02:50 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-27T11:02:50.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:02:50 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-27T11:02:50.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:02:50 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-27T11:02:50.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:02:50 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-27T11:02:50.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:02:50 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-27T11:02:50.766 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:02:50 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-27T11:02:50.766 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:02:50 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-27T11:02:50.766 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:02:50 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-27T11:02:50.766 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:02:50 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T11:02:50.766 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:02:50 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T11:02:50.766 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:02:50 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T11:02:51.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:02:51 smithi097 ceph-mon[31859]: Adjusting osd_memory_target on smithi114 to 176.7M 2024-03-27T11:02:51.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:02:51 smithi097 ceph-mon[31859]: Unable to set osd_memory_target on smithi114 to 185372364: error parsing value: Value '185372364' is below minimum 939524096 2024-03-27T11:02:51.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:02:51 smithi097 ceph-mon[31859]: Adjusting osd_memory_target on smithi097 to 176.7M 2024-03-27T11:02:51.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:02:51 smithi097 ceph-mon[31859]: Unable to set osd_memory_target on smithi097 to 185386700: error parsing value: Value '185386700' is below minimum 939524096 2024-03-27T11:02:51.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:02:51 smithi097 ceph-mon[31859]: pgmap v627: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 60 MiB/s wr, 969 op/s 2024-03-27T11:02:51.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:02:51 smithi177 ceph-mon[35294]: Adjusting osd_memory_target on smithi114 to 176.7M 2024-03-27T11:02:51.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:02:51 smithi177 ceph-mon[35294]: Unable to set osd_memory_target on smithi114 to 185372364: error parsing value: Value '185372364' is below minimum 939524096 2024-03-27T11:02:51.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:02:51 smithi177 ceph-mon[35294]: Adjusting osd_memory_target on smithi097 to 176.7M 2024-03-27T11:02:51.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:02:51 smithi177 ceph-mon[35294]: Unable to set osd_memory_target on smithi097 to 185386700: error parsing value: Value '185386700' is below minimum 939524096 2024-03-27T11:02:51.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:02:51 smithi177 ceph-mon[35294]: pgmap v627: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 60 MiB/s wr, 969 op/s 2024-03-27T11:02:51.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:02:51 smithi114 ceph-mon[35338]: Adjusting osd_memory_target on smithi114 to 176.7M 2024-03-27T11:02:51.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:02:51 smithi114 ceph-mon[35338]: Unable to set osd_memory_target on smithi114 to 185372364: error parsing value: Value '185372364' is below minimum 939524096 2024-03-27T11:02:51.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:02:51 smithi114 ceph-mon[35338]: Adjusting osd_memory_target on smithi097 to 176.7M 2024-03-27T11:02:51.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:02:51 smithi114 ceph-mon[35338]: Unable to set osd_memory_target on smithi097 to 185386700: error parsing value: Value '185386700' is below minimum 939524096 2024-03-27T11:02:51.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:02:51 smithi114 ceph-mon[35338]: pgmap v627: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 60 MiB/s wr, 969 op/s 2024-03-27T11:02:53.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:02:53 smithi097 ceph-mon[31859]: pgmap v628: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 54 MiB/s wr, 876 op/s 2024-03-27T11:02:53.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:02:53 smithi177 ceph-mon[35294]: pgmap v628: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 54 MiB/s wr, 876 op/s 2024-03-27T11:02:53.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:02:53 smithi114 ceph-mon[35338]: pgmap v628: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 54 MiB/s wr, 876 op/s 2024-03-27T11:02:55.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:02:55 smithi097 ceph-mon[31859]: pgmap v629: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 54 MiB/s wr, 880 op/s 2024-03-27T11:02:55.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:02:55 smithi177 ceph-mon[35294]: pgmap v629: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 54 MiB/s wr, 880 op/s 2024-03-27T11:02:55.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:02:55 smithi114 ceph-mon[35338]: pgmap v629: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 54 MiB/s wr, 880 op/s 2024-03-27T11:02:57.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:02:57 smithi097 ceph-mon[31859]: pgmap v630: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 52 MiB/s wr, 845 op/s 2024-03-27T11:02:57.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:02:57 smithi177 ceph-mon[35294]: pgmap v630: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 52 MiB/s wr, 845 op/s 2024-03-27T11:02:57.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:02:57 smithi114 ceph-mon[35338]: pgmap v630: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 52 MiB/s wr, 845 op/s 2024-03-27T11:02:59.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:02:59 smithi177 ceph-mon[35294]: pgmap v631: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 55 MiB/s wr, 886 op/s 2024-03-27T11:02:59.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:02:59 smithi114 ceph-mon[35338]: pgmap v631: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 55 MiB/s wr, 886 op/s 2024-03-27T11:02:59.889 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:02:59 smithi097 ceph-mon[31859]: pgmap v631: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 55 MiB/s wr, 886 op/s 2024-03-27T11:03:01.402 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.431+0000 7f0888000640 1 -- 172.21.15.97:0/2084648634 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0880103da0 msgr2=0x7f08801041c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:03:01.403 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.431+0000 7f0888000640 1 --2- 172.21.15.97:0/2084648634 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0880103da0 0x7f08801041c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0878009ca0 tx=0x7f087802c100 comp rx=0 tx=0).stop 2024-03-27T11:03:01.403 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.431+0000 7f0888000640 1 -- 172.21.15.97:0/2084648634 shutdown_connections 2024-03-27T11:03:01.403 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.431+0000 7f0888000640 1 --2- 172.21.15.97:0/2084648634 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0880105950 0x7f088010c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:01.403 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.431+0000 7f0888000640 1 --2- 172.21.15.97:0/2084648634 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0880104f90 0x7f0880105410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:01.403 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.431+0000 7f0888000640 1 --2- 172.21.15.97:0/2084648634 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0880103da0 0x7f08801041c0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:01.403 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.431+0000 7f0888000640 1 -- 172.21.15.97:0/2084648634 >> 172.21.15.97:0/2084648634 conn(0x7f08800ff310 msgr2=0x7f0880101750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:03:01.403 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.432+0000 7f0888000640 1 -- 172.21.15.97:0/2084648634 shutdown_connections 2024-03-27T11:03:01.403 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.432+0000 7f0888000640 1 -- 172.21.15.97:0/2084648634 wait complete. 2024-03-27T11:03:01.404 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.433+0000 7f0888000640 1 Processor -- start 2024-03-27T11:03:01.404 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.433+0000 7f0888000640 1 -- start start 2024-03-27T11:03:01.405 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.434+0000 7f0888000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0880103da0 0x7f088019e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:03:01.405 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.434+0000 7f0888000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0880104f90 0x7f088019e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:03:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.434+0000 7f0888000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0880105950 0x7f08801a5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:03:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.434+0000 7f0888000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f088010f8d0 con 0x7f0880103da0 2024-03-27T11:03:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.434+0000 7f0888000640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f088010f750 con 0x7f0880105950 2024-03-27T11:03:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.434+0000 7f0888000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f088010fa50 con 0x7f0880104f90 2024-03-27T11:03:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.434+0000 7f0886000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0880105950 0x7f08801a5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:03:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.434+0000 7f0884c00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0880104f90 0x7f088019e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:03:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.434+0000 7f0884c00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0880104f90 0x7f088019e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.97:53222/0 (socket says 172.21.15.97:53222) 2024-03-27T11:03:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.434+0000 7f0884c00640 1 -- 172.21.15.97:0/3974546963 learned_addr learned my addr 172.21.15.97:0/3974546963 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:03:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.435+0000 7f0885600640 1 --2- 172.21.15.97:0/3974546963 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0880103da0 0x7f088019e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:03:01.407 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.435+0000 7f0884c00640 1 -- 172.21.15.97:0/3974546963 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0880105950 msgr2=0x7f08801a5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:03:01.407 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.435+0000 7f0884c00640 1 --2- 172.21.15.97:0/3974546963 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0880105950 0x7f08801a5910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:01.408 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.435+0000 7f0884c00640 1 -- 172.21.15.97:0/3974546963 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0880103da0 msgr2=0x7f088019e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:03:01.408 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.435+0000 7f0884c00640 1 --2- 172.21.15.97:0/3974546963 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0880103da0 0x7f088019e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:01.408 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.435+0000 7f0884c00640 1 -- 172.21.15.97:0/3974546963 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08801a5f80 con 0x7f0880104f90 2024-03-27T11:03:01.408 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.436+0000 7f0884c00640 1 --2- 172.21.15.97:0/3974546963 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0880104f90 0x7f088019e8a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f087000d950 tx=0x7f087000de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:03:01.408 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.436+0000 7f087e000640 1 -- 172.21.15.97:0/3974546963 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0870017030 con 0x7f0880104f90 2024-03-27T11:03:01.408 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.436+0000 7f087e000640 1 -- 172.21.15.97:0/3974546963 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0870004020 con 0x7f0880104f90 2024-03-27T11:03:01.408 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.436+0000 7f087e000640 1 -- 172.21.15.97:0/3974546963 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08700047a0 con 0x7f0880104f90 2024-03-27T11:03:01.408 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.436+0000 7f0888000640 1 -- 172.21.15.97:0/3974546963 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08801a6270 con 0x7f0880104f90 2024-03-27T11:03:01.409 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.437+0000 7f0888000640 1 -- 172.21.15.97:0/3974546963 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08801a6770 con 0x7f0880104f90 2024-03-27T11:03:01.413 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.443+0000 7f087e000640 1 -- 172.21.15.97:0/3974546963 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f08700041d0 con 0x7f0880104f90 2024-03-27T11:03:01.414 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.443+0000 7f088add4740 1 Processor -- start 2024-03-27T11:03:01.414 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.444+0000 7f088add4740 1 -- start start 2024-03-27T11:03:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.444+0000 7f088add4740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555ce6d1c980 0x555ce6d1cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:03:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.444+0000 7f088add4740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555ce6d1d370 0x555ce6d21e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:03:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.444+0000 7f088add4740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ce6d224e0 0x555ce6d24970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:03:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.444+0000 7f088add4740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x555ce6c81d10 con 0x555ce6d224e0 2024-03-27T11:03:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.444+0000 7f088add4740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x555ce6c2a260 con 0x555ce6d1c980 2024-03-27T11:03:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.444+0000 7f088add4740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x555ce6c7dcc0 con 0x555ce6d1d370 2024-03-27T11:03:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.445+0000 7f0886000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555ce6d1c980 0x555ce6d1cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:03:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.445+0000 7f0886000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555ce6d1c980 0x555ce6d1cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:35148/0 (socket says 172.21.15.97:35148) 2024-03-27T11:03:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.445+0000 7f0886000640 1 -- 172.21.15.97:0/1314282378 learned_addr learned my addr 172.21.15.97:0/1314282378 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:03:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.445+0000 7f087e000640 1 --2- 172.21.15.97:0/3974546963 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f08680a37b0 0x7f08680a5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:03:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.445+0000 7f0884c00640 1 --2- 172.21.15.97:0/1314282378 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ce6d224e0 0x555ce6d24970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:03:01.417 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.445+0000 7f087e000640 1 -- 172.21.15.97:0/3974546963 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f08700cf740 con 0x7f0880104f90 2024-03-27T11:03:01.417 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.446+0000 7f0885600640 1 --2- 172.21.15.97:0/1314282378 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555ce6d1d370 0x555ce6d21e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:03:01.417 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.446+0000 7f0886000640 1 -- 172.21.15.97:0/1314282378 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555ce6d1d370 msgr2=0x555ce6d21e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:03:01.417 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.446+0000 7f0886000640 1 --2- 172.21.15.97:0/1314282378 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555ce6d1d370 0x555ce6d21e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:01.417 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.446+0000 7f0886000640 1 -- 172.21.15.97:0/1314282378 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ce6d224e0 msgr2=0x555ce6d24970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:03:01.417 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.446+0000 7f0886000640 1 --2- 172.21.15.97:0/1314282378 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ce6d224e0 0x555ce6d24970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:01.418 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.446+0000 7f0886000640 1 -- 172.21.15.97:0/1314282378 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ce6c21aa0 con 0x555ce6d1c980 2024-03-27T11:03:01.418 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.446+0000 7f0886000640 1 --2- 172.21.15.97:0/3974546963 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f08680a37b0 0x7f08680a5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:03:01.418 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.447+0000 7f0884c00640 1 --2- 172.21.15.97:0/1314282378 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ce6d224e0 0x555ce6d24970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T11:03:01.419 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.448+0000 7f0886000640 1 --2- 172.21.15.97:0/3974546963 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f08680a37b0 0x7f08680a5c70 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f0874009fd0 tx=0x7f0874016000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:03:01.419 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.448+0000 7f0886000640 1 --2- 172.21.15.97:0/1314282378 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555ce6d1c980 0x555ce6d1cda0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0874009da0 tx=0x7f0874011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d3b8d2c6c770130 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:03:01.419 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.449+0000 7f086e600640 1 -- 172.21.15.97:0/1314282378 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0874021070 con 0x555ce6d1c980 2024-03-27T11:03:01.419 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.449+0000 7f086e600640 1 -- 172.21.15.97:0/1314282378 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f087401cb30 con 0x555ce6d1c980 2024-03-27T11:03:01.420 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.449+0000 7f086e600640 1 -- 172.21.15.97:0/1314282378 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0874017050 con 0x555ce6d1c980 2024-03-27T11:03:01.420 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.450+0000 7f088add4740 1 -- 172.21.15.97:0/1314282378 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555ce6d1c980 msgr2=0x555ce6d1cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:03:01.420 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.450+0000 7f088add4740 1 --2- 172.21.15.97:0/1314282378 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555ce6d1c980 0x555ce6d1cda0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0874009da0 tx=0x7f0874011810 comp rx=0 tx=0).stop 2024-03-27T11:03:01.421 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.450+0000 7f088add4740 1 -- 172.21.15.97:0/1314282378 shutdown_connections 2024-03-27T11:03:01.421 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.450+0000 7f088add4740 1 --2- 172.21.15.97:0/1314282378 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ce6d224e0 0x555ce6d24970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:01.421 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.450+0000 7f088add4740 1 --2- 172.21.15.97:0/1314282378 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555ce6d1d370 0x555ce6d21e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:01.421 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.450+0000 7f088add4740 1 --2- 172.21.15.97:0/1314282378 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555ce6d1c980 0x555ce6d1cda0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:01.421 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.450+0000 7f088add4740 1 -- 172.21.15.97:0/1314282378 >> 172.21.15.97:0/1314282378 conn(0x555ce6d17ef0 msgr2=0x555ce6d182f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:03:01.422 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.450+0000 7f088add4740 1 -- 172.21.15.97:0/1314282378 shutdown_connections 2024-03-27T11:03:01.423 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.450+0000 7f088add4740 1 -- 172.21.15.97:0/1314282378 wait complete. 2024-03-27T11:03:01.423 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.451+0000 7f088add4740 1 -- start start 2024-03-27T11:03:01.423 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.451+0000 7f088add4740 1 Processor -- start 2024-03-27T11:03:01.423 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.452+0000 7f088add4740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ce6d1c980 0x555ce6d26d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:03:01.423 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.452+0000 7f088add4740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555ce6d1d370 0x555ce6d27250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:03:01.423 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.452+0000 7f088add4740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555ce6d224e0 0x555ce6d2e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:03:01.424 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.452+0000 7f088add4740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x555ce6d25490 con 0x555ce6d1c980 2024-03-27T11:03:01.424 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.452+0000 7f088add4740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x555ce6c3d630 con 0x555ce6d1d370 2024-03-27T11:03:01.424 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.453+0000 7f0886000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555ce6d224e0 0x555ce6d2e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:03:01.424 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.453+0000 7f0885600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ce6d1c980 0x555ce6d26d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:03:01.425 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.453+0000 7f0886000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555ce6d224e0 0x555ce6d2e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.97:53236/0 (socket says 172.21.15.97:53236) 2024-03-27T11:03:01.425 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.453+0000 7f0886000640 1 -- 172.21.15.97:0/3996628700 learned_addr learned my addr 172.21.15.97:0/3996628700 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:03:01.425 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.453+0000 7f088add4740 1 -- 172.21.15.97:0/3996628700 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x555ce6d25610 con 0x555ce6d224e0 2024-03-27T11:03:01.425 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.453+0000 7f088add4740 10 client.0 _ll_register_callbacks cb 0x555ce6c42c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T11:03:01.425 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.453+0000 7f0884c00640 1 --2- 172.21.15.97:0/3996628700 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555ce6d1d370 0x555ce6d27250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:03:01.425 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.454+0000 7f0886000640 1 -- 172.21.15.97:0/3996628700 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555ce6d1d370 msgr2=0x555ce6d27250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:03:01.426 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.454+0000 7f0886000640 1 --2- 172.21.15.97:0/3996628700 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555ce6d1d370 0x555ce6d27250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:01.426 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.454+0000 7f0886000640 1 -- 172.21.15.97:0/3996628700 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ce6d1c980 msgr2=0x555ce6d26d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:03:01.426 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.454+0000 7f0886000640 1 --2- 172.21.15.97:0/3996628700 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ce6d1c980 0x555ce6d26d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:01.426 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.454+0000 7f0886000640 1 -- 172.21.15.97:0/3996628700 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ce6c495f0 con 0x555ce6d224e0 2024-03-27T11:03:01.426 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.454+0000 7f088add4740 1 -- 172.21.15.97:0/3996628700 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f087401cb30 con 0x555ce6d224e0 2024-03-27T11:03:01.426 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.455+0000 7f0886000640 1 --2- 172.21.15.97:0/3996628700 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555ce6d224e0 0x555ce6d2e290 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f08740098a0 tx=0x7f0874025400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e4e9feabe903e24 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:03:01.426 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.455+0000 7f0862a00640 10 client.26014 ms_handle_connect on v2:172.21.15.177:3300/0 2024-03-27T11:03:01.427 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.455+0000 7f0862a00640 1 -- 172.21.15.97:0/3996628700 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0874034020 con 0x555ce6d224e0 2024-03-27T11:03:01.427 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.455+0000 7f0862a00640 1 -- 172.21.15.97:0/3996628700 <== mon.1 v2:172.21.15.177:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f08740256d0 con 0x555ce6d224e0 2024-03-27T11:03:01.427 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.456+0000 7f0862a00640 1 -- 172.21.15.97:0/3996628700 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f0874021040 con 0x555ce6d224e0 2024-03-27T11:03:01.427 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.456+0000 7f088add4740 10 client.26014 fetch_fsmap learned FSMap version 34 2024-03-27T11:03:01.427 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.456+0000 7f088add4740 1 -- 172.21.15.97:0/3996628700 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({fsmap=34}) -- 0x555ce6996210 con 0x555ce6d224e0 2024-03-27T11:03:01.428 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.456+0000 7f0862a00640 1 -- 172.21.15.97:0/3996628700 <== mon.1 v2:172.21.15.177:3300/0 4 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f0874025930 con 0x555ce6d224e0 2024-03-27T11:03:01.428 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.456+0000 7f088add4740 10 client.26014 fetch_fsmap finished waiting for FSMap version 34 2024-03-27T11:03:01.428 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.456+0000 7f088add4740 10 client.26014 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T11:03:01.428 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.457+0000 7f088add4740 20 client.26014 populate_metadata read hostname 'smithi097' 2024-03-27T11:03:01.428 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.457+0000 7f088add4740 1 --2- 172.21.15.97:0/3996628700 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x555ce6d32da0 0x555ce6d35220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:03:01.428 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.457+0000 7f088add4740 4 client.26014 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T11:03:01.428 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.457+0000 7f088add4740 1 -- 172.21.15.97:0/3996628700 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555ce6c8f250 con 0x555ce6d32da0 2024-03-27T11:03:01.428 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.457+0000 7f0885600640 1 --2- 172.21.15.97:0/3996628700 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x555ce6d32da0 0x555ce6d35220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:03:01.429 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.458+0000 7f0885600640 1 --2- 172.21.15.97:0/3996628700 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x555ce6d32da0 0x555ce6d35220 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=708688c9dee4a40f server_cookie=d4a19770273780f8 in_seq=0 out_seq=0 2024-03-27T11:03:01.429 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.458+0000 7f0862a00640 10 client.26014 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T11:03:01.430 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.459+0000 7f0862a00640 1 -- 172.21.15.97:0/3996628700 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x555ce6c8f250 con 0x555ce6d32da0 2024-03-27T11:03:01.430 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.459+0000 7f0862a00640 10 client.26014 handle_command_reply: tid=0 2024-03-27T11:03:01.430 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.459+0000 7f088add4740 1 client.26014 shutdown 2024-03-27T11:03:01.431 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.459+0000 7f088add4740 1 -- 172.21.15.97:0/3996628700 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555ce6d224e0 msgr2=0x555ce6d2e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:03:01.431 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.459+0000 7f088add4740 1 --2- 172.21.15.97:0/3996628700 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555ce6d224e0 0x555ce6d2e290 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f08740098a0 tx=0x7f0874025400 comp rx=0 tx=0).stop 2024-03-27T11:03:01.431 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.460+0000 7f088add4740 1 -- 172.21.15.97:0/3996628700 shutdown_connections 2024-03-27T11:03:01.431 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.460+0000 7f088add4740 1 --2- 172.21.15.97:0/3996628700 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x555ce6d32da0 0x555ce6d35220 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:01.431 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.460+0000 7f0862a00640 0 client.26014 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T11:03:01.431 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.460+0000 7f088add4740 1 --2- 172.21.15.97:0/3996628700 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555ce6d224e0 0x555ce6d2e290 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:01.431 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.460+0000 7f088add4740 1 --2- 172.21.15.97:0/3996628700 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555ce6d1d370 0x555ce6d27250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:01.431 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.460+0000 7f088add4740 1 --2- 172.21.15.97:0/3996628700 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ce6d1c980 0x555ce6d26d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:01.431 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.460+0000 7f088add4740 1 -- 172.21.15.97:0/3996628700 >> 172.21.15.97:0/3996628700 conn(0x555ce6d17ef0 msgr2=0x555ce6d170f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:03:01.431 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.460+0000 7f088add4740 1 -- 172.21.15.97:0/3996628700 shutdown_connections 2024-03-27T11:03:01.432 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.460+0000 7f088add4740 1 -- 172.21.15.97:0/3996628700 wait complete. 2024-03-27T11:03:01.432 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.460+0000 7f088add4740 20 client.26014 trim_cache size 0 max 16384 2024-03-27T11:03:01.459 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.489+0000 7f088add4740 1 Processor -- start 2024-03-27T11:03:01.460 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.489+0000 7f088add4740 1 -- start start 2024-03-27T11:03:01.460 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.489+0000 7f088add4740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555ce6d1c980 0x555ce6d16860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:03:01.460 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.489+0000 7f088add4740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ce6d1d370 0x555ce6d2e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:03:01.460 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.489+0000 7f088add4740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555ce6d224e0 0x555ce6d16da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:03:01.461 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.489+0000 7f088add4740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f08801af680 con 0x555ce6d1d370 2024-03-27T11:03:01.461 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.489+0000 7f088add4740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f08801b11c0 con 0x555ce6d224e0 2024-03-27T11:03:01.461 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.489+0000 7f088add4740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f08801afd50 con 0x555ce6d1c980 2024-03-27T11:03:01.461 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.490+0000 7f0886000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555ce6d224e0 0x555ce6d16da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:03:01.461 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.490+0000 7f0886000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555ce6d224e0 0x555ce6d16da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:35162/0 (socket says 172.21.15.97:35162) 2024-03-27T11:03:01.461 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.490+0000 7f0886000640 1 -- 172.21.15.97:0/2675245475 learned_addr learned my addr 172.21.15.97:0/2675245475 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:03:01.462 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.490+0000 7f0885600640 1 --2- 172.21.15.97:0/2675245475 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555ce6d1c980 0x555ce6d16860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:03:01.462 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.490+0000 7f0884c00640 1 --2- 172.21.15.97:0/2675245475 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ce6d1d370 0x555ce6d2e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:03:01.462 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.490+0000 7f0886000640 1 -- 172.21.15.97:0/2675245475 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555ce6d1c980 msgr2=0x555ce6d16860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:03:01.462 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.491+0000 7f0886000640 1 --2- 172.21.15.97:0/2675245475 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555ce6d1c980 0x555ce6d16860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:01.462 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.491+0000 7f0886000640 1 -- 172.21.15.97:0/2675245475 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ce6d1d370 msgr2=0x555ce6d2e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:03:01.462 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.491+0000 7f0886000640 1 --2- 172.21.15.97:0/2675245475 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ce6d1d370 0x555ce6d2e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:01.463 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.491+0000 7f0886000640 1 -- 172.21.15.97:0/2675245475 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ce6a62620 con 0x555ce6d224e0 2024-03-27T11:03:01.463 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.491+0000 7f0885600640 1 --2- 172.21.15.97:0/2675245475 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555ce6d1c980 0x555ce6d16860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T11:03:01.463 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.491+0000 7f0884c00640 1 --2- 172.21.15.97:0/2675245475 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ce6d1d370 0x555ce6d2e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T11:03:01.463 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.491+0000 7f0886000640 1 --2- 172.21.15.97:0/2675245475 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555ce6d224e0 0x555ce6d16da0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0874009ab0 tx=0x7f0874030000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ec87188b80ab04d server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:03:01.463 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.492+0000 7f086dc00640 1 -- 172.21.15.97:0/2675245475 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0874034040 con 0x555ce6d224e0 2024-03-27T11:03:01.463 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.492+0000 7f086dc00640 1 -- 172.21.15.97:0/2675245475 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f087401cb30 con 0x555ce6d224e0 2024-03-27T11:03:01.463 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.492+0000 7f086dc00640 1 -- 172.21.15.97:0/2675245475 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f087402f920 con 0x555ce6d224e0 2024-03-27T11:03:01.464 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.493+0000 7f088add4740 1 -- 172.21.15.97:0/2675245475 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555ce6d224e0 msgr2=0x555ce6d16da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:03:01.465 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.493+0000 7f088add4740 1 --2- 172.21.15.97:0/2675245475 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555ce6d224e0 0x555ce6d16da0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0874009ab0 tx=0x7f0874030000 comp rx=0 tx=0).stop 2024-03-27T11:03:01.465 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.493+0000 7f088add4740 1 -- 172.21.15.97:0/2675245475 shutdown_connections 2024-03-27T11:03:01.465 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.493+0000 7f088add4740 1 --2- 172.21.15.97:0/2675245475 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555ce6d224e0 0x555ce6d16da0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:01.465 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.493+0000 7f088add4740 1 --2- 172.21.15.97:0/2675245475 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ce6d1d370 0x555ce6d2e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:01.465 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.493+0000 7f088add4740 1 --2- 172.21.15.97:0/2675245475 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555ce6d1c980 0x555ce6d16860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:01.465 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.493+0000 7f088add4740 1 -- 172.21.15.97:0/2675245475 >> 172.21.15.97:0/2675245475 conn(0x555ce6d17ef0 msgr2=0x555ce6d1e6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:03:01.465 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.494+0000 7f088add4740 1 -- 172.21.15.97:0/2675245475 shutdown_connections 2024-03-27T11:03:01.465 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.494+0000 7f088add4740 1 -- 172.21.15.97:0/2675245475 wait complete. 2024-03-27T11:03:01.466 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.494+0000 7f088add4740 1 -- start start 2024-03-27T11:03:01.466 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.494+0000 7f088add4740 1 Processor -- start 2024-03-27T11:03:01.466 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.495+0000 7f088add4740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555ce6d1c980 0x555ce6d2b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:03:01.467 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.495+0000 7f088add4740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ce6d1d370 0x555ce6d2b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:03:01.467 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.495+0000 7f088add4740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555ce6d224e0 0x555ce6d2c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:03:01.467 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.495+0000 7f088add4740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f08801b0af0 con 0x555ce6d1d370 2024-03-27T11:03:01.467 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.495+0000 7f088add4740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f08801b0420 con 0x555ce6d224e0 2024-03-27T11:03:01.467 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.495+0000 7f088add4740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f08801ad660 con 0x555ce6d1c980 2024-03-27T11:03:01.467 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.495+0000 7f088add4740 10 client.0 _ll_register_callbacks cb 0x7f0874014070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T11:03:01.467 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.495+0000 7f0884c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ce6d1d370 0x555ce6d2b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:03:01.467 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.495+0000 7f0885600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555ce6d1c980 0x555ce6d2b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:03:01.467 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.495+0000 7f0885600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555ce6d1c980 0x555ce6d2b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.97:53254/0 (socket says 172.21.15.97:53254) 2024-03-27T11:03:01.467 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.495+0000 7f0885600640 1 -- 172.21.15.97:0/1715715856 learned_addr learned my addr 172.21.15.97:0/1715715856 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:03:01.467 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.495+0000 7f0886000640 1 --2- 172.21.15.97:0/1715715856 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555ce6d224e0 0x555ce6d2c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:03:01.469 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.496+0000 7f0885600640 1 -- 172.21.15.97:0/1715715856 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555ce6d224e0 msgr2=0x555ce6d2c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:03:01.469 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.496+0000 7f0885600640 1 --2- 172.21.15.97:0/1715715856 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555ce6d224e0 0x555ce6d2c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:01.469 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.496+0000 7f0885600640 1 -- 172.21.15.97:0/1715715856 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ce6d1d370 msgr2=0x555ce6d2b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:03:01.469 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.496+0000 7f0885600640 1 --2- 172.21.15.97:0/1715715856 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ce6d1d370 0x555ce6d2b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:01.469 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.496+0000 7f0885600640 1 -- 172.21.15.97:0/1715715856 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ce6d2ce90 con 0x555ce6d1c980 2024-03-27T11:03:01.469 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.496+0000 7f0884c00640 1 --2- 172.21.15.97:0/1715715856 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ce6d1d370 0x555ce6d2b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T11:03:01.469 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.496+0000 7f0886000640 1 --2- 172.21.15.97:0/1715715856 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555ce6d224e0 0x555ce6d2c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T11:03:01.470 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.496+0000 7f0885600640 1 --2- 172.21.15.97:0/1715715856 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555ce6d1c980 0x555ce6d2b350 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f087802c710 tx=0x7f0878032db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65077951073f344a server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:03:01.470 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.496+0000 7f0862a00640 10 client.0 ms_handle_connect on v2:172.21.15.177:3300/0 2024-03-27T11:03:01.470 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.497+0000 7f0862a00640 1 -- 172.21.15.97:0/1715715856 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0878039040 con 0x555ce6d1c980 2024-03-27T11:03:01.470 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.497+0000 7f0862a00640 1 -- 172.21.15.97:0/1715715856 <== mon.1 v2:172.21.15.177:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f0878048ca0 con 0x555ce6d1c980 2024-03-27T11:03:01.470 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.497+0000 7f088add4740 1 -- 172.21.15.97:0/1715715856 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f087401cb30 con 0x555ce6d1c980 2024-03-27T11:03:01.470 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.497+0000 7f0862a00640 1 -- 172.21.15.97:0/1715715856 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f0878057050 con 0x555ce6d1c980 2024-03-27T11:03:01.470 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.497+0000 7f088add4740 10 client.26020 fetch_fsmap learned FSMap version 34 2024-03-27T11:03:01.470 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.497+0000 7f088add4740 1 -- 172.21.15.97:0/1715715856 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x555ce6d2e180 con 0x555ce6d1c980 2024-03-27T11:03:01.470 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.497+0000 7f0862a00640 1 -- 172.21.15.97:0/1715715856 <== mon.1 v2:172.21.15.177:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f087401cb30 con 0x555ce6d1c980 2024-03-27T11:03:01.470 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.498+0000 7f0862a00640 1 -- 172.21.15.97:0/1715715856 <== mon.1 v2:172.21.15.177:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f08780482d0 con 0x555ce6d1c980 2024-03-27T11:03:01.470 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.498+0000 7f088add4740 10 client.26020 fetch_fsmap finished waiting for FSMap version 34 2024-03-27T11:03:01.471 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.498+0000 7f088add4740 10 client.26020 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T11:03:01.471 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.498+0000 7f088add4740 20 client.26020 populate_metadata read hostname 'smithi097' 2024-03-27T11:03:01.471 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.498+0000 7f088add4740 1 --2- 172.21.15.97:0/1715715856 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x555ce6d32da0 0x555ce6d9b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:03:01.471 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.498+0000 7f088add4740 4 client.26020 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-27T11:03:01.471 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.498+0000 7f088add4740 1 -- 172.21.15.97:0/1715715856 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555ce6c895a0 con 0x555ce6d32da0 2024-03-27T11:03:01.471 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.498+0000 7f0884c00640 1 --2- 172.21.15.97:0/1715715856 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x555ce6d32da0 0x555ce6d9b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:03:01.471 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.499+0000 7f0884c00640 1 --2- 172.21.15.97:0/1715715856 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x555ce6d32da0 0x555ce6d9b500 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4dc8ac0d2c0db924 server_cookie=54094deb8d91655f in_seq=0 out_seq=0 2024-03-27T11:03:01.471 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.499+0000 7f0862a00640 10 client.26020 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T11:03:01.477 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.506+0000 7f0862a00640 1 -- 172.21.15.97:0/1715715856 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555ce6c895a0 con 0x555ce6d32da0 2024-03-27T11:03:01.477 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.506+0000 7f0862a00640 10 client.26020 handle_command_reply: tid=0 2024-03-27T11:03:01.477 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.506+0000 7f088add4740 1 client.26020 shutdown 2024-03-27T11:03:01.478 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.506+0000 7f088add4740 1 -- 172.21.15.97:0/1715715856 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555ce6d1c980 msgr2=0x555ce6d2b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:03:01.478 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.506+0000 7f088add4740 1 --2- 172.21.15.97:0/1715715856 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555ce6d1c980 0x555ce6d2b350 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f087802c710 tx=0x7f0878032db0 comp rx=0 tx=0).stop 2024-03-27T11:03:01.478 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.507+0000 7f088add4740 1 -- 172.21.15.97:0/1715715856 shutdown_connections 2024-03-27T11:03:01.478 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.507+0000 7f088add4740 1 --2- 172.21.15.97:0/1715715856 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x555ce6d32da0 0x555ce6d9b500 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:01.478 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.507+0000 7f088add4740 1 --2- 172.21.15.97:0/1715715856 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555ce6d224e0 0x555ce6d2c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:01.478 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.507+0000 7f088add4740 1 --2- 172.21.15.97:0/1715715856 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ce6d1d370 0x555ce6d2b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:01.478 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.507+0000 7f088add4740 1 --2- 172.21.15.97:0/1715715856 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555ce6d1c980 0x555ce6d2b350 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:01.478 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.507+0000 7f088add4740 1 -- 172.21.15.97:0/1715715856 >> 172.21.15.97:0/1715715856 conn(0x555ce6d17ef0 msgr2=0x555ce6d26cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:03:01.478 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.507+0000 7f0862a00640 0 client.26020 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T11:03:01.479 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.507+0000 7f088add4740 1 -- 172.21.15.97:0/1715715856 shutdown_connections 2024-03-27T11:03:01.479 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.507+0000 7f088add4740 1 -- 172.21.15.97:0/1715715856 wait complete. 2024-03-27T11:03:01.479 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.507+0000 7f088add4740 20 client.26020 trim_cache size 0 max 16384 2024-03-27T11:03:01.479 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-27T11:03:01.480 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-27T11:03:01.480 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-27T11:03:01.480 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-27T11:03:01.481 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.509+0000 7f0888000640 1 -- 172.21.15.97:0/3974546963 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f08680a37b0 msgr2=0x7f08680a5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:03:01.481 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.509+0000 7f0888000640 1 --2- 172.21.15.97:0/3974546963 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f08680a37b0 0x7f08680a5c70 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f0874009fd0 tx=0x7f0874016000 comp rx=0 tx=0).stop 2024-03-27T11:03:01.481 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.509+0000 7f0888000640 1 -- 172.21.15.97:0/3974546963 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0880104f90 msgr2=0x7f088019e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:03:01.481 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.509+0000 7f0888000640 1 --2- 172.21.15.97:0/3974546963 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0880104f90 0x7f088019e8a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f087000d950 tx=0x7f087000de10 comp rx=0 tx=0).stop 2024-03-27T11:03:01.481 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.509+0000 7f0888000640 1 -- 172.21.15.97:0/3974546963 shutdown_connections 2024-03-27T11:03:01.481 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.509+0000 7f0888000640 1 --2- 172.21.15.97:0/3974546963 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f08680a37b0 0x7f08680a5c70 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:01.481 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.509+0000 7f0888000640 1 --2- 172.21.15.97:0/3974546963 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0880105950 0x7f08801a5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:01.481 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.509+0000 7f0888000640 1 --2- 172.21.15.97:0/3974546963 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0880104f90 0x7f088019e8a0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:01.481 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.509+0000 7f0888000640 1 --2- 172.21.15.97:0/3974546963 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0880103da0 0x7f088019e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:01.481 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.509+0000 7f0888000640 1 -- 172.21.15.97:0/3974546963 >> 172.21.15.97:0/3974546963 conn(0x7f08800ff310 msgr2=0x7f0880077fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:03:01.481 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.510+0000 7f0888000640 1 -- 172.21.15.97:0/3974546963 shutdown_connections 2024-03-27T11:03:01.482 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:01.511+0000 7f0888000640 1 -- 172.21.15.97:0/3974546963 wait complete. 2024-03-27T11:03:01.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:03:01 smithi177 ceph-mon[35294]: pgmap v632: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 58 MiB/s wr, 944 op/s 2024-03-27T11:03:01.692 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:03:01 smithi097 ceph-mon[31859]: pgmap v632: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 58 MiB/s wr, 944 op/s 2024-03-27T11:03:01.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:03:01 smithi114 ceph-mon[35338]: pgmap v632: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 58 MiB/s wr, 944 op/s 2024-03-27T11:03:03.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:03:03 smithi177 ceph-mon[35294]: pgmap v633: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 53 MiB/s wr, 857 op/s 2024-03-27T11:03:03.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:03:03 smithi114 ceph-mon[35338]: pgmap v633: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 53 MiB/s wr, 857 op/s 2024-03-27T11:03:03.889 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:03:03 smithi097 ceph-mon[31859]: pgmap v633: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 53 MiB/s wr, 857 op/s 2024-03-27T11:03:05.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:03:05 smithi177 ceph-mon[35294]: pgmap v634: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 53 MiB/s wr, 861 op/s 2024-03-27T11:03:05.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:03:05 smithi114 ceph-mon[35338]: pgmap v634: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 53 MiB/s wr, 861 op/s 2024-03-27T11:03:05.889 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:03:05 smithi097 ceph-mon[31859]: pgmap v634: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 53 MiB/s wr, 861 op/s 2024-03-27T11:03:06.767 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:03:06 smithi114 ceph-mon[35338]: pgmap v635: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 51 MiB/s wr, 829 op/s 2024-03-27T11:03:06.888 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:03:06 smithi097 ceph-mon[31859]: pgmap v635: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 51 MiB/s wr, 829 op/s 2024-03-27T11:03:06.924 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:03:06 smithi177 ceph-mon[35294]: pgmap v635: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 51 MiB/s wr, 829 op/s 2024-03-27T11:03:09.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:03:09 smithi097 ceph-mon[31859]: pgmap v636: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 54 MiB/s wr, 866 op/s 2024-03-27T11:03:09.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:03:09 smithi177 ceph-mon[35294]: pgmap v636: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 54 MiB/s wr, 866 op/s 2024-03-27T11:03:09.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:03:09 smithi114 ceph-mon[35338]: pgmap v636: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 54 MiB/s wr, 866 op/s 2024-03-27T11:03:11.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:03:11 smithi097 ceph-mon[31859]: pgmap v637: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 58 MiB/s wr, 934 op/s 2024-03-27T11:03:11.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:03:11 smithi177 ceph-mon[35294]: pgmap v637: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 58 MiB/s wr, 934 op/s 2024-03-27T11:03:11.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:03:11 smithi114 ceph-mon[35338]: pgmap v637: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 58 MiB/s wr, 934 op/s 2024-03-27T11:03:13.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:03:13 smithi097 ceph-mon[31859]: pgmap v638: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 52 MiB/s wr, 842 op/s 2024-03-27T11:03:13.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:03:13 smithi177 ceph-mon[35294]: pgmap v638: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 52 MiB/s wr, 842 op/s 2024-03-27T11:03:13.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:03:13 smithi114 ceph-mon[35338]: pgmap v638: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 52 MiB/s wr, 842 op/s 2024-03-27T11:03:14.704 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.734+0000 7fa046e00640 1 -- 172.21.15.97:0/4054027877 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa040103750 msgr2=0x7fa040109fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:03:14.704 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.734+0000 7fa046e00640 1 --2- 172.21.15.97:0/4054027877 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa040103750 0x7fa040109fe0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa038009ca0 tx=0x7fa03802c100 comp rx=0 tx=0).stop 2024-03-27T11:03:14.705 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.734+0000 7fa046e00640 1 -- 172.21.15.97:0/4054027877 shutdown_connections 2024-03-27T11:03:14.705 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.734+0000 7fa046e00640 1 --2- 172.21.15.97:0/4054027877 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa040103750 0x7fa040109fe0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:14.705 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.734+0000 7fa046e00640 1 --2- 172.21.15.97:0/4054027877 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa040102d90 0x7fa040103210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:14.705 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.734+0000 7fa046e00640 1 --2- 172.21.15.97:0/4054027877 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa040101ba0 0x7fa040101fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:14.705 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.734+0000 7fa046e00640 1 -- 172.21.15.97:0/4054027877 >> 172.21.15.97:0/4054027877 conn(0x7fa0400fd110 msgr2=0x7fa0400ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:03:14.705 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.734+0000 7fa046e00640 1 -- 172.21.15.97:0/4054027877 shutdown_connections 2024-03-27T11:03:14.705 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.734+0000 7fa046e00640 1 -- 172.21.15.97:0/4054027877 wait complete. 2024-03-27T11:03:14.706 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.735+0000 7fa046e00640 1 Processor -- start 2024-03-27T11:03:14.706 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.736+0000 7fa046e00640 1 -- start start 2024-03-27T11:03:14.707 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.736+0000 7fa046e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa040101ba0 0x7fa040197cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:03:14.707 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.736+0000 7fa046e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa040102d90 0x7fa040198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:03:14.707 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.736+0000 7fa046e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa040103750 0x7fa04019f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:03:14.707 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.736+0000 7fa046e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa04010d6b0 con 0x7fa040102d90 2024-03-27T11:03:14.708 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.737+0000 7fa046e00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa04010d530 con 0x7fa040103750 2024-03-27T11:03:14.708 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.737+0000 7fa046e00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa04010d830 con 0x7fa040101ba0 2024-03-27T11:03:14.708 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.737+0000 7fa03fe00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa040101ba0 0x7fa040197cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:03:14.708 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.737+0000 7fa03fe00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa040101ba0 0x7fa040197cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.97:32996/0 (socket says 172.21.15.97:32996) 2024-03-27T11:03:14.708 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.737+0000 7fa03fe00640 1 -- 172.21.15.97:0/4078059162 learned_addr learned my addr 172.21.15.97:0/4078059162 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:03:14.708 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.737+0000 7fa03f400640 1 --2- 172.21.15.97:0/4078059162 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa040102d90 0x7fa040198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:03:14.708 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.737+0000 7fa044e00640 1 --2- 172.21.15.97:0/4078059162 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa040103750 0x7fa04019f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:03:14.708 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.737+0000 7fa03fe00640 1 -- 172.21.15.97:0/4078059162 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa040103750 msgr2=0x7fa04019f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:03:14.708 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.737+0000 7fa03fe00640 1 --2- 172.21.15.97:0/4078059162 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa040103750 0x7fa04019f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:14.708 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.737+0000 7fa03fe00640 1 -- 172.21.15.97:0/4078059162 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa040102d90 msgr2=0x7fa040198200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:03:14.708 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.737+0000 7fa03fe00640 1 --2- 172.21.15.97:0/4078059162 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa040102d90 0x7fa040198200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:14.709 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.737+0000 7fa03fe00640 1 -- 172.21.15.97:0/4078059162 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa04019f900 con 0x7fa040101ba0 2024-03-27T11:03:14.709 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.738+0000 7fa03f400640 1 --2- 172.21.15.97:0/4078059162 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa040102d90 0x7fa040198200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T11:03:14.709 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.738+0000 7fa03fe00640 1 --2- 172.21.15.97:0/4078059162 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa040101ba0 0x7fa040197cc0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa03000e9c0 tx=0x7fa03000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:03:14.709 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.738+0000 7fa03cc00640 1 -- 172.21.15.97:0/4078059162 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa03000cd80 con 0x7fa040101ba0 2024-03-27T11:03:14.709 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.738+0000 7fa03cc00640 1 -- 172.21.15.97:0/4078059162 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa0300035f0 con 0x7fa040101ba0 2024-03-27T11:03:14.709 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.738+0000 7fa03cc00640 1 -- 172.21.15.97:0/4078059162 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa030010690 con 0x7fa040101ba0 2024-03-27T11:03:14.710 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.739+0000 7fa046e00640 1 -- 172.21.15.97:0/4078059162 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa040073d80 con 0x7fa040101ba0 2024-03-27T11:03:14.710 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.739+0000 7fa046e00640 1 -- 172.21.15.97:0/4078059162 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa040074320 con 0x7fa040101ba0 2024-03-27T11:03:14.713 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.744+0000 7fa049be5740 1 Processor -- start 2024-03-27T11:03:14.715 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.744+0000 7fa049be5740 1 -- start start 2024-03-27T11:03:14.715 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.745+0000 7fa049be5740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d31ce51190 0x55d31ce515b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:03:14.715 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.745+0000 7fa049be5740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d31ce51b80 0x55d31ce56680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:03:14.716 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.745+0000 7fa03cc00640 1 -- 172.21.15.97:0/4078059162 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7fa0300037a0 con 0x7fa040101ba0 2024-03-27T11:03:14.716 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.746+0000 7fa049be5740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d31ce56cf0 0x55d31ce59180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:03:14.716 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.746+0000 7fa049be5740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55d31cdadd10 con 0x55d31ce51b80 2024-03-27T11:03:14.716 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.746+0000 7fa049be5740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55d31cd56260 con 0x55d31ce51190 2024-03-27T11:03:14.716 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.746+0000 7fa049be5740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55d31cda9cc0 con 0x55d31ce56cf0 2024-03-27T11:03:14.717 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.746+0000 7fa044e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d31ce51190 0x55d31ce515b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:03:14.717 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.746+0000 7fa044e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d31ce51190 0x55d31ce515b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:43734/0 (socket says 172.21.15.97:43734) 2024-03-27T11:03:14.717 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.746+0000 7fa044e00640 1 -- 172.21.15.97:0/3968006585 learned_addr learned my addr 172.21.15.97:0/3968006585 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:03:14.717 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.746+0000 7fa03f400640 1 --2- 172.21.15.97:0/3968006585 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d31ce56cf0 0x55d31ce59180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:03:14.717 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.746+0000 7fa03fe00640 1 --2- 172.21.15.97:0/3968006585 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d31ce51b80 0x55d31ce56680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:03:14.718 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.747+0000 7fa03cc00640 1 --2- 172.21.15.97:0/4078059162 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fa0240a3730 0x7fa0240a5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:03:14.718 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.747+0000 7fa03fe00640 1 -- 172.21.15.97:0/3968006585 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d31ce56cf0 msgr2=0x55d31ce59180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:03:14.718 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.747+0000 7fa03fe00640 1 --2- 172.21.15.97:0/3968006585 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d31ce56cf0 0x55d31ce59180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:14.718 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.747+0000 7fa03fe00640 1 -- 172.21.15.97:0/3968006585 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d31ce51190 msgr2=0x55d31ce515b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:03:14.718 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.747+0000 7fa03fe00640 1 --2- 172.21.15.97:0/3968006585 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d31ce51190 0x55d31ce515b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:14.718 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.747+0000 7fa03fe00640 1 -- 172.21.15.97:0/3968006585 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d31cd4daa0 con 0x55d31ce51b80 2024-03-27T11:03:14.718 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.748+0000 7fa03cc00640 1 -- 172.21.15.97:0/4078059162 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fa0300d0160 con 0x7fa040101ba0 2024-03-27T11:03:14.718 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.748+0000 7fa03fe00640 1 --2- 172.21.15.97:0/3968006585 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d31ce51b80 0x55d31ce56680 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fa0300cdc40 tx=0x7fa030021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d33c3594ded9391 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:03:14.719 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.749+0000 7fa035600640 1 -- 172.21.15.97:0/3968006585 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa03009c410 con 0x55d31ce51b80 2024-03-27T11:03:14.719 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.749+0000 7fa035600640 1 -- 172.21.15.97:0/3968006585 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa030003b30 con 0x55d31ce51b80 2024-03-27T11:03:14.719 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.749+0000 7fa035600640 1 -- 172.21.15.97:0/3968006585 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa03009bfa0 con 0x55d31ce51b80 2024-03-27T11:03:14.720 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.750+0000 7fa049be5740 1 -- 172.21.15.97:0/3968006585 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d31ce51b80 msgr2=0x55d31ce56680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:03:14.720 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.750+0000 7fa049be5740 1 --2- 172.21.15.97:0/3968006585 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d31ce51b80 0x55d31ce56680 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fa0300cdc40 tx=0x7fa030021000 comp rx=0 tx=0).stop 2024-03-27T11:03:14.720 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.750+0000 7fa044e00640 1 --2- 172.21.15.97:0/4078059162 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fa0240a3730 0x7fa0240a5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:03:14.720 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.750+0000 7fa049be5740 1 -- 172.21.15.97:0/3968006585 shutdown_connections 2024-03-27T11:03:14.721 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.750+0000 7fa049be5740 1 --2- 172.21.15.97:0/3968006585 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d31ce56cf0 0x55d31ce59180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:14.721 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.751+0000 7fa049be5740 1 --2- 172.21.15.97:0/3968006585 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d31ce51b80 0x55d31ce56680 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:14.721 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.751+0000 7fa049be5740 1 --2- 172.21.15.97:0/3968006585 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d31ce51190 0x55d31ce515b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:14.721 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.751+0000 7fa049be5740 1 -- 172.21.15.97:0/3968006585 >> 172.21.15.97:0/3968006585 conn(0x55d31ce43ef0 msgr2=0x55d31ce442f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:03:14.722 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.751+0000 7fa049be5740 1 -- 172.21.15.97:0/3968006585 shutdown_connections 2024-03-27T11:03:14.722 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.751+0000 7fa044e00640 1 --2- 172.21.15.97:0/4078059162 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fa0240a3730 0x7fa0240a5bf0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x55d31ce523e0 tx=0x7fa03802e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:03:14.722 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.751+0000 7fa049be5740 1 -- 172.21.15.97:0/3968006585 wait complete. 2024-03-27T11:03:14.722 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.752+0000 7fa049be5740 1 -- start start 2024-03-27T11:03:14.722 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.752+0000 7fa049be5740 1 Processor -- start 2024-03-27T11:03:14.723 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.753+0000 7fa049be5740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d31ce51190 0x55d31ce5b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:03:14.723 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.753+0000 7fa049be5740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d31ce51b80 0x55d31ce5b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:03:14.724 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.753+0000 7fa049be5740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d31ce56cf0 0x55d31ce62920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:03:14.724 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.753+0000 7fa049be5740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55d31ce59be0 con 0x55d31ce51190 2024-03-27T11:03:14.724 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.753+0000 7fa049be5740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55d31cd69630 con 0x55d31ce51b80 2024-03-27T11:03:14.724 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.753+0000 7fa049be5740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55d31ce59d60 con 0x55d31ce56cf0 2024-03-27T11:03:14.724 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.753+0000 7fa049be5740 10 client.0 _ll_register_callbacks cb 0x55d31cd6ec50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T11:03:14.724 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.753+0000 7fa03fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d31ce51190 0x55d31ce5b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:03:14.724 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.753+0000 7fa03fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d31ce51190 0x55d31ce5b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50116/0 (socket says 172.21.15.97:50116) 2024-03-27T11:03:14.724 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.753+0000 7fa03fe00640 1 -- 172.21.15.97:0/184889599 learned_addr learned my addr 172.21.15.97:0/184889599 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:03:14.724 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.753+0000 7fa044e00640 1 --2- 172.21.15.97:0/184889599 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d31ce56cf0 0x55d31ce62920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:03:14.724 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.753+0000 7fa03f400640 1 --2- 172.21.15.97:0/184889599 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d31ce51b80 0x55d31ce5b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:03:14.724 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.754+0000 7fa03fe00640 1 -- 172.21.15.97:0/184889599 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d31ce56cf0 msgr2=0x55d31ce62920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:03:14.725 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.754+0000 7fa03fe00640 1 --2- 172.21.15.97:0/184889599 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d31ce56cf0 0x55d31ce62920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:14.725 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.754+0000 7fa03fe00640 1 -- 172.21.15.97:0/184889599 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d31ce51b80 msgr2=0x55d31ce5b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:03:14.725 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.754+0000 7fa03fe00640 1 --2- 172.21.15.97:0/184889599 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d31ce51b80 0x55d31ce5b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:14.725 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.754+0000 7fa03fe00640 1 -- 172.21.15.97:0/184889599 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d31cd755f0 con 0x55d31ce51190 2024-03-27T11:03:14.725 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.754+0000 7fa03f400640 1 --2- 172.21.15.97:0/184889599 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d31ce51b80 0x55d31ce5b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T11:03:14.726 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.754+0000 7fa03fe00640 1 --2- 172.21.15.97:0/184889599 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d31ce51190 0x55d31ce5b3a0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fa0300cdfe0 tx=0x7fa0300d0460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13cb4a08bfa6578f server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:03:14.726 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.754+0000 7fa02a000640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T11:03:14.727 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.755+0000 7fa02a000640 1 -- 172.21.15.97:0/184889599 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa03009c180 con 0x55d31ce51190 2024-03-27T11:03:14.727 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.755+0000 7fa049be5740 1 -- 172.21.15.97:0/184889599 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d31cdbb250 con 0x55d31ce51190 2024-03-27T11:03:14.727 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.755+0000 7fa02a000640 1 -- 172.21.15.97:0/184889599 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fa0300b0bb0 con 0x55d31ce51190 2024-03-27T11:03:14.727 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.755+0000 7fa02a000640 1 -- 172.21.15.97:0/184889599 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fa03009bfa0 con 0x55d31ce51190 2024-03-27T11:03:14.727 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.755+0000 7fa049be5740 10 client.16650 fetch_fsmap learned FSMap version 34 2024-03-27T11:03:14.727 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.755+0000 7fa049be5740 1 -- 172.21.15.97:0/184889599 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55d31cac2210 con 0x55d31ce51190 2024-03-27T11:03:14.727 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.756+0000 7fa02a000640 1 -- 172.21.15.97:0/184889599 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55d31cdbb250 con 0x55d31ce51190 2024-03-27T11:03:14.727 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.756+0000 7fa02a000640 1 -- 172.21.15.97:0/184889599 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7fa030016330 con 0x55d31ce51190 2024-03-27T11:03:14.727 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.756+0000 7fa049be5740 10 client.16650 fetch_fsmap finished waiting for FSMap version 34 2024-03-27T11:03:14.727 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.757+0000 7fa049be5740 10 client.16650 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T11:03:14.727 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.757+0000 7fa049be5740 20 client.16650 populate_metadata read hostname 'smithi097' 2024-03-27T11:03:14.728 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.757+0000 7fa049be5740 1 --2- 172.21.15.97:0/184889599 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55d31ce671a0 0x55d31ce695f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:03:14.728 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.757+0000 7fa049be5740 4 client.16650 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T11:03:14.728 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.757+0000 7fa049be5740 1 -- 172.21.15.97:0/184889599 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d31cdb55a0 con 0x55d31ce671a0 2024-03-27T11:03:14.729 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.757+0000 7fa03f400640 1 --2- 172.21.15.97:0/184889599 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55d31ce671a0 0x55d31ce695f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:03:14.729 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.758+0000 7fa03f400640 1 --2- 172.21.15.97:0/184889599 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55d31ce671a0 0x55d31ce695f0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22a42cc5065e66c7 server_cookie=84b18c0d3638a7 in_seq=0 out_seq=0 2024-03-27T11:03:14.729 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.758+0000 7fa02a000640 10 client.16650 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T11:03:14.729 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.759+0000 7fa02a000640 1 -- 172.21.15.97:0/184889599 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55d31cdb55a0 con 0x55d31ce671a0 2024-03-27T11:03:14.730 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.760+0000 7fa02a000640 10 client.16650 handle_command_reply: tid=0 2024-03-27T11:03:14.730 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.760+0000 7fa049be5740 1 client.16650 shutdown 2024-03-27T11:03:14.731 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.760+0000 7fa049be5740 1 -- 172.21.15.97:0/184889599 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d31ce51190 msgr2=0x55d31ce5b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:03:14.731 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.760+0000 7fa049be5740 1 --2- 172.21.15.97:0/184889599 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d31ce51190 0x55d31ce5b3a0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fa0300cdfe0 tx=0x7fa0300d0460 comp rx=0 tx=0).stop 2024-03-27T11:03:14.731 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.760+0000 7fa049be5740 1 -- 172.21.15.97:0/184889599 shutdown_connections 2024-03-27T11:03:14.731 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.760+0000 7fa049be5740 1 --2- 172.21.15.97:0/184889599 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55d31ce671a0 0x55d31ce695f0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:14.731 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.761+0000 7fa02a000640 0 client.16650 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T11:03:14.731 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.761+0000 7fa049be5740 1 --2- 172.21.15.97:0/184889599 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d31ce56cf0 0x55d31ce62920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:14.731 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.761+0000 7fa049be5740 1 --2- 172.21.15.97:0/184889599 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d31ce51b80 0x55d31ce5b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:14.731 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.761+0000 7fa049be5740 1 --2- 172.21.15.97:0/184889599 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d31ce51190 0x55d31ce5b3a0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:14.731 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.761+0000 7fa049be5740 1 -- 172.21.15.97:0/184889599 >> 172.21.15.97:0/184889599 conn(0x55d31ce43ef0 msgr2=0x55d31ce430f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:03:14.731 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.761+0000 7fa049be5740 1 -- 172.21.15.97:0/184889599 shutdown_connections 2024-03-27T11:03:14.732 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.761+0000 7fa049be5740 1 -- 172.21.15.97:0/184889599 wait complete. 2024-03-27T11:03:14.732 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.762+0000 7fa049be5740 20 client.16650 trim_cache size 0 max 16384 2024-03-27T11:03:14.766 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.796+0000 7fa049be5740 1 Processor -- start 2024-03-27T11:03:14.766 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.796+0000 7fa049be5740 1 -- start start 2024-03-27T11:03:14.767 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.797+0000 7fa049be5740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d31ce51190 0x55d31ce442d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:03:14.767 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.797+0000 7fa049be5740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d31ce51b80 0x55d31ce628b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:03:14.767 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.797+0000 7fa049be5740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d31ce56cf0 0x55d31ce5afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:03:14.767 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.797+0000 7fa049be5740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa0401a6ef0 con 0x55d31ce56cf0 2024-03-27T11:03:14.767 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.797+0000 7fa049be5740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa0401a8a30 con 0x55d31ce51b80 2024-03-27T11:03:14.767 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.797+0000 7fa049be5740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa0401a75c0 con 0x55d31ce51190 2024-03-27T11:03:14.768 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.798+0000 7fa03f400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d31ce51b80 0x55d31ce628b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:03:14.768 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.798+0000 7fa03f400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d31ce51b80 0x55d31ce628b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:43758/0 (socket says 172.21.15.97:43758) 2024-03-27T11:03:14.768 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.798+0000 7fa03f400640 1 -- 172.21.15.97:0/916400944 learned_addr learned my addr 172.21.15.97:0/916400944 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:03:14.768 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.798+0000 7fa03fe00640 1 --2- 172.21.15.97:0/916400944 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d31ce51190 0x55d31ce442d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:03:14.768 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.798+0000 7fa044e00640 1 --2- 172.21.15.97:0/916400944 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d31ce56cf0 0x55d31ce5afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:03:14.769 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.799+0000 7fa03f400640 1 -- 172.21.15.97:0/916400944 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d31ce51190 msgr2=0x55d31ce442d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:03:14.769 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.799+0000 7fa03f400640 1 --2- 172.21.15.97:0/916400944 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d31ce51190 0x55d31ce442d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:14.769 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.799+0000 7fa03f400640 1 -- 172.21.15.97:0/916400944 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d31ce56cf0 msgr2=0x55d31ce5afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:03:14.769 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.799+0000 7fa03f400640 1 --2- 172.21.15.97:0/916400944 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d31ce56cf0 0x55d31ce5afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:14.769 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.799+0000 7fa03f400640 1 -- 172.21.15.97:0/916400944 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d31cb8e620 con 0x55d31ce51b80 2024-03-27T11:03:14.769 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.799+0000 7fa044e00640 1 --2- 172.21.15.97:0/916400944 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d31ce56cf0 0x55d31ce5afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T11:03:14.770 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.801+0000 7fa03f400640 1 --2- 172.21.15.97:0/916400944 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d31ce51b80 0x55d31ce628b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa0401991c0 tx=0x7fa02c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da9d5b5c37402ef9 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:03:14.772 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.801+0000 7fa034c00640 1 -- 172.21.15.97:0/916400944 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa02c021030 con 0x55d31ce51b80 2024-03-27T11:03:14.773 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.803+0000 7fa034c00640 1 -- 172.21.15.97:0/916400944 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa02c01dc50 con 0x55d31ce51b80 2024-03-27T11:03:14.773 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.803+0000 7fa034c00640 1 -- 172.21.15.97:0/916400944 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa02c005350 con 0x55d31ce51b80 2024-03-27T11:03:14.774 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.805+0000 7fa049be5740 1 -- 172.21.15.97:0/916400944 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d31ce51b80 msgr2=0x55d31ce628b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:03:14.774 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.805+0000 7fa049be5740 1 --2- 172.21.15.97:0/916400944 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d31ce51b80 0x55d31ce628b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa0401991c0 tx=0x7fa02c0046c0 comp rx=0 tx=0).stop 2024-03-27T11:03:14.775 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.805+0000 7fa049be5740 1 -- 172.21.15.97:0/916400944 shutdown_connections 2024-03-27T11:03:14.775 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.805+0000 7fa049be5740 1 --2- 172.21.15.97:0/916400944 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d31ce56cf0 0x55d31ce5afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:14.775 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.805+0000 7fa049be5740 1 --2- 172.21.15.97:0/916400944 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d31ce51b80 0x55d31ce628b0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:14.775 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.805+0000 7fa049be5740 1 --2- 172.21.15.97:0/916400944 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d31ce51190 0x55d31ce442d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:14.775 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.805+0000 7fa049be5740 1 -- 172.21.15.97:0/916400944 >> 172.21.15.97:0/916400944 conn(0x55d31ce43ef0 msgr2=0x55d31ce56460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:03:14.776 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.805+0000 7fa049be5740 1 -- 172.21.15.97:0/916400944 shutdown_connections 2024-03-27T11:03:14.776 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.806+0000 7fa049be5740 1 -- 172.21.15.97:0/916400944 wait complete. 2024-03-27T11:03:14.776 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.806+0000 7fa049be5740 1 -- start start 2024-03-27T11:03:14.776 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.807+0000 7fa049be5740 1 Processor -- start 2024-03-27T11:03:14.779 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.809+0000 7fa049be5740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d31ce51190 0x55d31ce5fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:03:14.779 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.809+0000 7fa049be5740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d31ce51b80 0x55d31ce602e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:03:14.779 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.809+0000 7fa049be5740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d31ce56cf0 0x55d31ce61280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:03:14.780 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.809+0000 7fa049be5740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa0401a8360 con 0x55d31ce51190 2024-03-27T11:03:14.780 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.810+0000 7fa049be5740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa0401a7c90 con 0x55d31ce56cf0 2024-03-27T11:03:14.780 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.810+0000 7fa049be5740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa0401a4ed0 con 0x55d31ce51b80 2024-03-27T11:03:14.780 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.810+0000 7fa049be5740 10 client.0 _ll_register_callbacks cb 0x7fa038037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T11:03:14.780 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.810+0000 7fa03fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d31ce51190 0x55d31ce5fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:03:14.780 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.810+0000 7fa044e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d31ce56cf0 0x55d31ce61280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:03:14.780 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.810+0000 7fa044e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d31ce56cf0 0x55d31ce61280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:43764/0 (socket says 172.21.15.97:43764) 2024-03-27T11:03:14.780 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.810+0000 7fa03fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d31ce51190 0x55d31ce5fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50130/0 (socket says 172.21.15.97:50130) 2024-03-27T11:03:14.780 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.810+0000 7fa044e00640 1 -- 172.21.15.97:0/1359245071 learned_addr learned my addr 172.21.15.97:0/1359245071 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:03:14.781 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.810+0000 7fa03f400640 1 --2- 172.21.15.97:0/1359245071 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d31ce51b80 0x55d31ce602e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:03:14.782 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.811+0000 7fa03fe00640 1 -- 172.21.15.97:0/1359245071 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d31ce51b80 msgr2=0x55d31ce602e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:03:14.782 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.811+0000 7fa03fe00640 1 --2- 172.21.15.97:0/1359245071 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d31ce51b80 0x55d31ce602e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:14.782 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.811+0000 7fa03fe00640 1 -- 172.21.15.97:0/1359245071 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d31ce56cf0 msgr2=0x55d31ce61280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:03:14.782 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.811+0000 7fa03fe00640 1 --2- 172.21.15.97:0/1359245071 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d31ce56cf0 0x55d31ce61280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:14.782 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.812+0000 7fa03fe00640 1 -- 172.21.15.97:0/1359245071 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d31ce61870 con 0x55d31ce51190 2024-03-27T11:03:14.783 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.813+0000 7fa03fe00640 1 --2- 172.21.15.97:0/1359245071 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d31ce51190 0x55d31ce5fda0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fa030000c00 tx=0x7fa030009700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=642cff7ffbc4dc3 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:03:14.783 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.813+0000 7fa02a000640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T11:03:14.783 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.813+0000 7fa02a000640 1 -- 172.21.15.97:0/1359245071 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa03009c410 con 0x55d31ce51190 2024-03-27T11:03:14.784 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.813+0000 7fa02a000640 1 -- 172.21.15.97:0/1359245071 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fa030085bd0 con 0x55d31ce51190 2024-03-27T11:03:14.784 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.813+0000 7fa049be5740 1 -- 172.21.15.97:0/1359245071 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa02c01dc50 con 0x55d31ce51190 2024-03-27T11:03:14.784 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.814+0000 7fa02a000640 1 -- 172.21.15.97:0/1359245071 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fa03009c180 con 0x55d31ce51190 2024-03-27T11:03:14.784 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.814+0000 7fa049be5740 10 client.16656 fetch_fsmap learned FSMap version 34 2024-03-27T11:03:14.784 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.814+0000 7fa049be5740 1 -- 172.21.15.97:0/1359245071 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55d31ce62b80 con 0x55d31ce51190 2024-03-27T11:03:14.785 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.814+0000 7fa02a000640 1 -- 172.21.15.97:0/1359245071 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa02c01dc50 con 0x55d31ce51190 2024-03-27T11:03:14.785 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.815+0000 7fa02a000640 1 -- 172.21.15.97:0/1359245071 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7fa0300b5bd0 con 0x55d31ce51190 2024-03-27T11:03:14.785 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.815+0000 7fa049be5740 10 client.16656 fetch_fsmap finished waiting for FSMap version 34 2024-03-27T11:03:14.785 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.815+0000 7fa049be5740 10 client.16656 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T11:03:14.785 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.815+0000 7fa049be5740 20 client.16656 populate_metadata read hostname 'smithi097' 2024-03-27T11:03:14.785 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.815+0000 7fa049be5740 1 --2- 172.21.15.97:0/1359245071 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55d31ce671a0 0x55d31cecf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:03:14.785 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.815+0000 7fa049be5740 4 client.16656 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-27T11:03:14.785 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.815+0000 7fa049be5740 1 -- 172.21.15.97:0/1359245071 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d31cd33630 con 0x55d31ce671a0 2024-03-27T11:03:14.785 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.815+0000 7fa03f400640 1 --2- 172.21.15.97:0/1359245071 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55d31ce671a0 0x55d31cecf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:03:14.786 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.816+0000 7fa03f400640 1 --2- 172.21.15.97:0/1359245071 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55d31ce671a0 0x55d31cecf9f0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48e78c310facea8b server_cookie=3398a9a988f37fb8 in_seq=0 out_seq=0 2024-03-27T11:03:14.786 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.816+0000 7fa02a000640 10 client.16656 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T11:03:14.801 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.831+0000 7fa02a000640 1 -- 172.21.15.97:0/1359245071 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d31cd33630 con 0x55d31ce671a0 2024-03-27T11:03:14.801 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.831+0000 7fa02a000640 10 client.16656 handle_command_reply: tid=0 2024-03-27T11:03:14.801 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.831+0000 7fa049be5740 1 client.16656 shutdown 2024-03-27T11:03:14.802 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.832+0000 7fa049be5740 1 -- 172.21.15.97:0/1359245071 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d31ce51190 msgr2=0x55d31ce5fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:03:14.802 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.832+0000 7fa049be5740 1 --2- 172.21.15.97:0/1359245071 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d31ce51190 0x55d31ce5fda0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fa030000c00 tx=0x7fa030009700 comp rx=0 tx=0).stop 2024-03-27T11:03:14.803 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.832+0000 7fa049be5740 1 -- 172.21.15.97:0/1359245071 shutdown_connections 2024-03-27T11:03:14.803 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.832+0000 7fa049be5740 1 --2- 172.21.15.97:0/1359245071 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55d31ce671a0 0x55d31cecf9f0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:14.803 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.832+0000 7fa049be5740 1 --2- 172.21.15.97:0/1359245071 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d31ce56cf0 0x55d31ce61280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:14.803 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.832+0000 7fa049be5740 1 --2- 172.21.15.97:0/1359245071 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d31ce51b80 0x55d31ce602e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:14.803 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.832+0000 7fa049be5740 1 --2- 172.21.15.97:0/1359245071 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d31ce51190 0x55d31ce5fda0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:14.803 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.832+0000 7fa049be5740 1 -- 172.21.15.97:0/1359245071 >> 172.21.15.97:0/1359245071 conn(0x55d31ce43ef0 msgr2=0x55d31ce4f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:03:14.803 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.832+0000 7fa02a000640 0 client.16656 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T11:03:14.803 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.833+0000 7fa049be5740 1 -- 172.21.15.97:0/1359245071 shutdown_connections 2024-03-27T11:03:14.803 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.833+0000 7fa049be5740 1 -- 172.21.15.97:0/1359245071 wait complete. 2024-03-27T11:03:14.803 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.833+0000 7fa049be5740 20 client.16656 trim_cache size 0 max 16384 2024-03-27T11:03:14.804 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-27T11:03:14.804 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-27T11:03:14.804 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-27T11:03:14.804 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-27T11:03:14.805 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.834+0000 7fa046e00640 1 -- 172.21.15.97:0/4078059162 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fa0240a3730 msgr2=0x7fa0240a5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:03:14.805 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.834+0000 7fa046e00640 1 --2- 172.21.15.97:0/4078059162 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fa0240a3730 0x7fa0240a5bf0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x55d31ce523e0 tx=0x7fa03802e040 comp rx=0 tx=0).stop 2024-03-27T11:03:14.805 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.834+0000 7fa046e00640 1 -- 172.21.15.97:0/4078059162 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa040101ba0 msgr2=0x7fa040197cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:03:14.805 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.834+0000 7fa046e00640 1 --2- 172.21.15.97:0/4078059162 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa040101ba0 0x7fa040197cc0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa03000e9c0 tx=0x7fa03000ee80 comp rx=0 tx=0).stop 2024-03-27T11:03:14.805 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.835+0000 7fa046e00640 1 -- 172.21.15.97:0/4078059162 shutdown_connections 2024-03-27T11:03:14.806 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.835+0000 7fa046e00640 1 --2- 172.21.15.97:0/4078059162 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fa0240a3730 0x7fa0240a5bf0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:14.806 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.835+0000 7fa046e00640 1 --2- 172.21.15.97:0/4078059162 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa040103750 0x7fa04019f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:14.806 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.835+0000 7fa046e00640 1 --2- 172.21.15.97:0/4078059162 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa040102d90 0x7fa040198200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:14.806 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.835+0000 7fa046e00640 1 --2- 172.21.15.97:0/4078059162 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa040101ba0 0x7fa040197cc0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:14.806 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.835+0000 7fa046e00640 1 -- 172.21.15.97:0/4078059162 >> 172.21.15.97:0/4078059162 conn(0x7fa0400fd110 msgr2=0x7fa040106090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:03:14.806 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.835+0000 7fa046e00640 1 -- 172.21.15.97:0/4078059162 shutdown_connections 2024-03-27T11:03:14.806 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:14.835+0000 7fa046e00640 1 -- 172.21.15.97:0/4078059162 wait complete. 2024-03-27T11:03:15.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:03:15 smithi097 ceph-mon[31859]: pgmap v639: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 53 MiB/s wr, 851 op/s 2024-03-27T11:03:15.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:03:15 smithi177 ceph-mon[35294]: pgmap v639: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 53 MiB/s wr, 851 op/s 2024-03-27T11:03:15.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:03:15 smithi114 ceph-mon[35338]: pgmap v639: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 53 MiB/s wr, 851 op/s 2024-03-27T11:03:17.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:03:17 smithi097 ceph-mon[31859]: pgmap v640: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 51 MiB/s wr, 829 op/s 2024-03-27T11:03:17.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:03:17 smithi177 ceph-mon[35294]: pgmap v640: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 51 MiB/s wr, 829 op/s 2024-03-27T11:03:17.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:03:17 smithi114 ceph-mon[35338]: pgmap v640: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 51 MiB/s wr, 829 op/s 2024-03-27T11:03:19.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:03:19 smithi097 ceph-mon[31859]: pgmap v641: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 55 MiB/s wr, 882 op/s 2024-03-27T11:03:19.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:03:19 smithi177 ceph-mon[35294]: pgmap v641: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 55 MiB/s wr, 882 op/s 2024-03-27T11:03:19.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:03:19 smithi114 ceph-mon[35338]: pgmap v641: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 55 MiB/s wr, 882 op/s 2024-03-27T11:03:21.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:03:21 smithi097 ceph-mon[31859]: pgmap v642: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 59 MiB/s wr, 954 op/s 2024-03-27T11:03:21.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:03:21 smithi177 ceph-mon[35294]: pgmap v642: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 59 MiB/s wr, 954 op/s 2024-03-27T11:03:21.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:03:21 smithi114 ceph-mon[35338]: pgmap v642: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 59 MiB/s wr, 954 op/s 2024-03-27T11:03:23.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:03:23 smithi097 ceph-mon[31859]: pgmap v643: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 53 MiB/s wr, 857 op/s 2024-03-27T11:03:23.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:03:23 smithi177 ceph-mon[35294]: pgmap v643: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 53 MiB/s wr, 857 op/s 2024-03-27T11:03:23.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:03:23 smithi114 ceph-mon[35338]: pgmap v643: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 53 MiB/s wr, 857 op/s 2024-03-27T11:03:25.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:03:25 smithi097 ceph-mon[31859]: pgmap v644: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 54 MiB/s wr, 875 op/s 2024-03-27T11:03:25.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:03:25 smithi177 ceph-mon[35294]: pgmap v644: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 54 MiB/s wr, 875 op/s 2024-03-27T11:03:25.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:03:25 smithi114 ceph-mon[35338]: pgmap v644: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 54 MiB/s wr, 875 op/s 2024-03-27T11:03:27.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:03:27 smithi097 ceph-mon[31859]: pgmap v645: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 52 MiB/s wr, 845 op/s 2024-03-27T11:03:27.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:03:27 smithi177 ceph-mon[35294]: pgmap v645: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 52 MiB/s wr, 845 op/s 2024-03-27T11:03:27.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:03:27 smithi114 ceph-mon[35338]: pgmap v645: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 52 MiB/s wr, 845 op/s 2024-03-27T11:03:28.010 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.040+0000 7fba4fa00640 1 -- 172.21.15.97:0/489695315 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fba48103db0 msgr2=0x7fba481041d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:03:28.010 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.040+0000 7fba4fa00640 1 --2- 172.21.15.97:0/489695315 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fba48103db0 0x7fba481041d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fba3c009900 tx=0x7fba3c02bc30 comp rx=0 tx=0).stop 2024-03-27T11:03:28.011 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.041+0000 7fba4fa00640 1 -- 172.21.15.97:0/489695315 shutdown_connections 2024-03-27T11:03:28.011 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.041+0000 7fba4fa00640 1 --2- 172.21.15.97:0/489695315 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba48105960 0x7fba4810c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:28.011 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.041+0000 7fba4fa00640 1 --2- 172.21.15.97:0/489695315 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fba48104fa0 0x7fba48105420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:28.011 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.041+0000 7fba4fa00640 1 --2- 172.21.15.97:0/489695315 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fba48103db0 0x7fba481041d0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:28.011 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.041+0000 7fba4fa00640 1 -- 172.21.15.97:0/489695315 >> 172.21.15.97:0/489695315 conn(0x7fba480ff340 msgr2=0x7fba48101760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:03:28.011 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.041+0000 7fba4fa00640 1 -- 172.21.15.97:0/489695315 shutdown_connections 2024-03-27T11:03:28.011 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.041+0000 7fba4fa00640 1 -- 172.21.15.97:0/489695315 wait complete. 2024-03-27T11:03:28.012 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.042+0000 7fba4fa00640 1 Processor -- start 2024-03-27T11:03:28.012 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.043+0000 7fba4fa00640 1 -- start start 2024-03-27T11:03:28.013 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.043+0000 7fba4fa00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fba48103db0 0x7fba4806d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:03:28.013 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.043+0000 7fba4fa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba48104fa0 0x7fba4806d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:03:28.013 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.043+0000 7fba4fa00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fba48105960 0x7fba48072700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:03:28.013 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.043+0000 7fba4fa00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fba4806bcd0 con 0x7fba48104fa0 2024-03-27T11:03:28.013 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.043+0000 7fba4fa00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fba4806bb50 con 0x7fba48105960 2024-03-27T11:03:28.013 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.043+0000 7fba4fa00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fba4806be50 con 0x7fba48103db0 2024-03-27T11:03:28.013 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.043+0000 7fba4d000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fba48103db0 0x7fba4806d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:03:28.014 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.043+0000 7fba4d000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fba48103db0 0x7fba4806d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.97:50944/0 (socket says 172.21.15.97:50944) 2024-03-27T11:03:28.014 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.043+0000 7fba4d000640 1 -- 172.21.15.97:0/1757843394 learned_addr learned my addr 172.21.15.97:0/1757843394 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:03:28.014 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.043+0000 7fba4da00640 1 --2- 172.21.15.97:0/1757843394 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fba48105960 0x7fba48072700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:03:28.014 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.043+0000 7fba47e00640 1 --2- 172.21.15.97:0/1757843394 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba48104fa0 0x7fba4806d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:03:28.014 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.044+0000 7fba4d000640 1 -- 172.21.15.97:0/1757843394 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fba48105960 msgr2=0x7fba48072700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:03:28.014 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.044+0000 7fba4d000640 1 --2- 172.21.15.97:0/1757843394 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fba48105960 0x7fba48072700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:28.014 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.044+0000 7fba4d000640 1 -- 172.21.15.97:0/1757843394 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba48104fa0 msgr2=0x7fba4806d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:03:28.014 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.044+0000 7fba4d000640 1 --2- 172.21.15.97:0/1757843394 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba48104fa0 0x7fba4806d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:28.014 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.044+0000 7fba4d000640 1 -- 172.21.15.97:0/1757843394 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba48072d70 con 0x7fba48103db0 2024-03-27T11:03:28.015 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.044+0000 7fba47e00640 1 --2- 172.21.15.97:0/1757843394 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba48104fa0 0x7fba4806d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T11:03:28.015 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.044+0000 7fba4da00640 1 --2- 172.21.15.97:0/1757843394 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fba48105960 0x7fba48072700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T11:03:28.015 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.045+0000 7fba4d000640 1 --2- 172.21.15.97:0/1757843394 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fba48103db0 0x7fba4806d160 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fba3c000c00 tx=0x7fba3c0040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:03:28.015 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.045+0000 7fba45600640 1 -- 172.21.15.97:0/1757843394 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba3c043020 con 0x7fba48103db0 2024-03-27T11:03:28.015 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.045+0000 7fba45600640 1 -- 172.21.15.97:0/1757843394 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fba3c0044c0 con 0x7fba48103db0 2024-03-27T11:03:28.016 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.045+0000 7fba45600640 1 -- 172.21.15.97:0/1757843394 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba3c003c90 con 0x7fba48103db0 2024-03-27T11:03:28.016 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.045+0000 7fba4fa00640 1 -- 172.21.15.97:0/1757843394 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba48073000 con 0x7fba48103db0 2024-03-27T11:03:28.016 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.045+0000 7fba4fa00640 1 -- 172.21.15.97:0/1757843394 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba48073500 con 0x7fba48103db0 2024-03-27T11:03:28.019 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.049+0000 7fba52817740 1 Processor -- start 2024-03-27T11:03:28.019 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.050+0000 7fba45600640 1 -- 172.21.15.97:0/1757843394 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7fba3c046020 con 0x7fba48103db0 2024-03-27T11:03:28.020 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.050+0000 7fba52817740 1 -- start start 2024-03-27T11:03:28.021 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.050+0000 7fba52817740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e17148a980 0x55e17148ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:03:28.021 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.050+0000 7fba52817740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e17148b370 0x55e17148fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:03:28.021 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.050+0000 7fba52817740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e1714904e0 0x55e171492970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:03:28.021 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.050+0000 7fba52817740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55e1713efd10 con 0x55e1714904e0 2024-03-27T11:03:28.021 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.050+0000 7fba52817740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55e171398260 con 0x55e17148b370 2024-03-27T11:03:28.022 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.050+0000 7fba52817740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55e1713ebcc0 con 0x55e17148a980 2024-03-27T11:03:28.022 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.051+0000 7fba4da00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e17148a980 0x55e17148ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:03:28.022 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.051+0000 7fba4d000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e17148b370 0x55e17148fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:03:28.022 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.051+0000 7fba4d000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e17148b370 0x55e17148fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:50394/0 (socket says 172.21.15.97:50394) 2024-03-27T11:03:28.022 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.051+0000 7fba4d000640 1 -- 172.21.15.97:0/925681364 learned_addr learned my addr 172.21.15.97:0/925681364 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:03:28.022 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.051+0000 7fba4da00640 1 --2- 172.21.15.97:0/925681364 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e17148a980 0x55e17148ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.97:50954/0 (socket says 172.21.15.97:50954) 2024-03-27T11:03:28.023 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.051+0000 7fba47e00640 1 --2- 172.21.15.97:0/925681364 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e1714904e0 0x55e171492970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:03:28.025 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.052+0000 7fba4d000640 1 -- 172.21.15.97:0/925681364 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e17148a980 msgr2=0x55e17148ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:03:28.025 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.052+0000 7fba4d000640 1 --2- 172.21.15.97:0/925681364 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e17148a980 0x55e17148ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:28.025 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.052+0000 7fba4d000640 1 -- 172.21.15.97:0/925681364 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e1714904e0 msgr2=0x55e171492970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:03:28.025 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.052+0000 7fba4d000640 1 --2- 172.21.15.97:0/925681364 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e1714904e0 0x55e171492970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:28.025 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.052+0000 7fba4d000640 1 -- 172.21.15.97:0/925681364 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e17138faa0 con 0x55e17148b370 2024-03-27T11:03:28.025 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.052+0000 7fba45600640 1 --2- 172.21.15.97:0/1757843394 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fba300a3730 0x7fba300a5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:03:28.025 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.052+0000 7fba4d000640 1 --2- 172.21.15.97:0/925681364 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e17148b370 0x55e17148fe70 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fba3c0f5f90 tx=0x7fba3c0fa1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2a7be24bacd4e44 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:03:28.025 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.053+0000 7fba45600640 1 -- 172.21.15.97:0/1757843394 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fba3c0f2010 con 0x7fba48103db0 2024-03-27T11:03:28.025 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.053+0000 7fba3a000640 1 -- 172.21.15.97:0/925681364 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba3c0ba790 con 0x55e17148b370 2024-03-27T11:03:28.026 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.053+0000 7fba3a000640 1 -- 172.21.15.97:0/925681364 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fba3c0035f0 con 0x55e17148b370 2024-03-27T11:03:28.026 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.054+0000 7fba3a000640 1 -- 172.21.15.97:0/925681364 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba3c0bde50 con 0x55e17148b370 2024-03-27T11:03:28.026 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.054+0000 7fba4da00640 1 --2- 172.21.15.97:0/1757843394 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fba300a3730 0x7fba300a5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:03:28.026 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.054+0000 7fba52817740 1 -- 172.21.15.97:0/925681364 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e17148b370 msgr2=0x55e17148fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:03:28.026 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.054+0000 7fba52817740 1 --2- 172.21.15.97:0/925681364 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e17148b370 0x55e17148fe70 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fba3c0f5f90 tx=0x7fba3c0fa1a0 comp rx=0 tx=0).stop 2024-03-27T11:03:28.026 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.055+0000 7fba52817740 1 -- 172.21.15.97:0/925681364 shutdown_connections 2024-03-27T11:03:28.026 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.055+0000 7fba52817740 1 --2- 172.21.15.97:0/925681364 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e1714904e0 0x55e171492970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:28.026 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.055+0000 7fba52817740 1 --2- 172.21.15.97:0/925681364 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e17148b370 0x55e17148fe70 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:28.026 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.055+0000 7fba52817740 1 --2- 172.21.15.97:0/925681364 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e17148a980 0x55e17148ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:28.026 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.055+0000 7fba52817740 1 -- 172.21.15.97:0/925681364 >> 172.21.15.97:0/925681364 conn(0x55e171485ef0 msgr2=0x55e1714862f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:03:28.026 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.055+0000 7fba4da00640 1 --2- 172.21.15.97:0/1757843394 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fba300a3730 0x7fba300a5bf0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x55e17148bbd0 tx=0x7fba400086a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:03:28.026 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.055+0000 7fba52817740 1 -- 172.21.15.97:0/925681364 shutdown_connections 2024-03-27T11:03:28.026 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.055+0000 7fba52817740 1 -- 172.21.15.97:0/925681364 wait complete. 2024-03-27T11:03:28.028 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.056+0000 7fba52817740 1 -- start start 2024-03-27T11:03:28.028 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.056+0000 7fba52817740 1 Processor -- start 2024-03-27T11:03:28.028 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.057+0000 7fba52817740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e17148a980 0x55e171494a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:03:28.028 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.057+0000 7fba52817740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e17148b370 0x55e171494f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:03:28.028 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.057+0000 7fba52817740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e1714904e0 0x55e17149bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:03:28.029 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.057+0000 7fba52817740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55e171493250 con 0x55e17148b370 2024-03-27T11:03:28.029 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.057+0000 7fba52817740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55e1713ab630 con 0x55e17148a980 2024-03-27T11:03:28.029 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.057+0000 7fba52817740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55e1714933d0 con 0x55e1714904e0 2024-03-27T11:03:28.029 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.057+0000 7fba52817740 10 client.0 _ll_register_callbacks cb 0x55e1713b0c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T11:03:28.029 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.057+0000 7fba4d000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e17148a980 0x55e171494a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:03:28.029 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.057+0000 7fba4d000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e17148a980 0x55e171494a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:50398/0 (socket says 172.21.15.97:50398) 2024-03-27T11:03:28.029 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.057+0000 7fba4d000640 1 -- 172.21.15.97:0/2208347341 learned_addr learned my addr 172.21.15.97:0/2208347341 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:03:28.029 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.057+0000 7fba4da00640 1 --2- 172.21.15.97:0/2208347341 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e1714904e0 0x55e17149bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:03:28.029 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.058+0000 7fba4d000640 1 -- 172.21.15.97:0/2208347341 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e1714904e0 msgr2=0x55e17149bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:03:28.029 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.058+0000 7fba4d000640 1 --2- 172.21.15.97:0/2208347341 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e1714904e0 0x55e17149bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:28.029 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.058+0000 7fba4d000640 1 -- 172.21.15.97:0/2208347341 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e17148b370 msgr2=0x55e171494f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:03:28.029 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.058+0000 7fba4d000640 1 --2- 172.21.15.97:0/2208347341 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e17148b370 0x55e171494f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:28.029 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.058+0000 7fba4d000640 1 -- 172.21.15.97:0/2208347341 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1713b75f0 con 0x55e17148a980 2024-03-27T11:03:28.030 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.058+0000 7fba4d000640 1 --2- 172.21.15.97:0/2208347341 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e17148a980 0x55e171494a30 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fba3c0efb70 tx=0x7fba3c02c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6184e4ff31c4ab0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:03:28.030 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.058+0000 7fba2ea00640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-03-27T11:03:28.030 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.058+0000 7fba2ea00640 1 -- 172.21.15.97:0/2208347341 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba3c0be030 con 0x55e17148a980 2024-03-27T11:03:28.030 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.058+0000 7fba52817740 1 -- 172.21.15.97:0/2208347341 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e1713fd250 con 0x55e17148a980 2024-03-27T11:03:28.030 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.058+0000 7fba2ea00640 1 -- 172.21.15.97:0/2208347341 <== mon.2 v2:172.21.15.114:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fba3c0bc7f0 con 0x55e17148a980 2024-03-27T11:03:28.030 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.059+0000 7fba2ea00640 1 -- 172.21.15.97:0/2208347341 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fba3c0be2c0 con 0x55e17148a980 2024-03-27T11:03:28.030 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.059+0000 7fba52817740 10 client.36332 fetch_fsmap learned FSMap version 34 2024-03-27T11:03:28.030 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.059+0000 7fba52817740 1 -- 172.21.15.97:0/2208347341 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55e171104210 con 0x55e17148a980 2024-03-27T11:03:28.031 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.059+0000 7fba2ea00640 1 -- 172.21.15.97:0/2208347341 <== mon.2 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55e1713fd250 con 0x55e17148a980 2024-03-27T11:03:28.031 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.059+0000 7fba2ea00640 1 -- 172.21.15.97:0/2208347341 <== mon.2 v2:172.21.15.114:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7fba3c0c6cd0 con 0x55e17148a980 2024-03-27T11:03:28.031 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.059+0000 7fba52817740 10 client.36332 fetch_fsmap finished waiting for FSMap version 34 2024-03-27T11:03:28.031 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.059+0000 7fba52817740 10 client.36332 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T11:03:28.031 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.059+0000 7fba52817740 20 client.36332 populate_metadata read hostname 'smithi097' 2024-03-27T11:03:28.031 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.059+0000 7fba52817740 1 --2- 172.21.15.97:0/2208347341 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55e1714a0690 0x55e1714a2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:03:28.031 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.059+0000 7fba52817740 4 client.36332 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T11:03:28.031 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.059+0000 7fba52817740 1 -- 172.21.15.97:0/2208347341 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e1713f75a0 con 0x55e1714a0690 2024-03-27T11:03:28.031 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.060+0000 7fba47e00640 1 --2- 172.21.15.97:0/2208347341 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55e1714a0690 0x55e1714a2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:03:28.031 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.060+0000 7fba47e00640 1 --2- 172.21.15.97:0/2208347341 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55e1714a0690 0x55e1714a2ac0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be6877abff726c38 server_cookie=ee4468ae615b547b in_seq=0 out_seq=0 2024-03-27T11:03:28.032 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.060+0000 7fba2ea00640 10 client.36332 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T11:03:28.032 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.061+0000 7fba2ea00640 1 -- 172.21.15.97:0/2208347341 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55e1713f75a0 con 0x55e1714a0690 2024-03-27T11:03:28.032 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.061+0000 7fba2ea00640 10 client.36332 handle_command_reply: tid=0 2024-03-27T11:03:28.032 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.062+0000 7fba52817740 1 client.36332 shutdown 2024-03-27T11:03:28.032 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.062+0000 7fba52817740 1 -- 172.21.15.97:0/2208347341 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e17148a980 msgr2=0x55e171494a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:03:28.032 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.062+0000 7fba52817740 1 --2- 172.21.15.97:0/2208347341 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e17148a980 0x55e171494a30 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fba3c0efb70 tx=0x7fba3c02c450 comp rx=0 tx=0).stop 2024-03-27T11:03:28.032 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.062+0000 7fba52817740 1 -- 172.21.15.97:0/2208347341 shutdown_connections 2024-03-27T11:03:28.033 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.063+0000 7fba52817740 1 --2- 172.21.15.97:0/2208347341 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55e1714a0690 0x55e1714a2ac0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:28.033 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.063+0000 7fba52817740 1 --2- 172.21.15.97:0/2208347341 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e1714904e0 0x55e17149bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:28.033 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.063+0000 7fba52817740 1 --2- 172.21.15.97:0/2208347341 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e17148b370 0x55e171494f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:28.033 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.063+0000 7fba52817740 1 --2- 172.21.15.97:0/2208347341 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e17148a980 0x55e171494a30 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:28.033 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.063+0000 7fba52817740 1 -- 172.21.15.97:0/2208347341 >> 172.21.15.97:0/2208347341 conn(0x55e171485ef0 msgr2=0x55e1714850f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:03:28.033 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.063+0000 7fba2ea00640 0 client.36332 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T11:03:28.034 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.064+0000 7fba52817740 1 -- 172.21.15.97:0/2208347341 shutdown_connections 2024-03-27T11:03:28.034 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.065+0000 7fba52817740 1 -- 172.21.15.97:0/2208347341 wait complete. 2024-03-27T11:03:28.034 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.065+0000 7fba52817740 20 client.36332 trim_cache size 0 max 16384 2024-03-27T11:03:28.069 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.100+0000 7fba52817740 1 Processor -- start 2024-03-27T11:03:28.070 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.100+0000 7fba52817740 1 -- start start 2024-03-27T11:03:28.071 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.101+0000 7fba52817740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e17148a980 0x55e171489860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:03:28.071 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.101+0000 7fba52817740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e17148b370 0x55e17149bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:03:28.071 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.101+0000 7fba52817740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e1714904e0 0x55e171494640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:03:28.071 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.101+0000 7fba52817740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fba481c0490 con 0x55e17148b370 2024-03-27T11:03:28.071 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.101+0000 7fba52817740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fba481c1fd0 con 0x55e17148a980 2024-03-27T11:03:28.072 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.101+0000 7fba52817740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fba481c0b60 con 0x55e1714904e0 2024-03-27T11:03:28.074 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.101+0000 7fba4d000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e17148a980 0x55e171489860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:03:28.074 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.101+0000 7fba47e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e17148b370 0x55e17149bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:03:28.074 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.101+0000 7fba47e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e17148b370 0x55e17149bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33728/0 (socket says 172.21.15.97:33728) 2024-03-27T11:03:28.074 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.101+0000 7fba47e00640 1 -- 172.21.15.97:0/1296305463 learned_addr learned my addr 172.21.15.97:0/1296305463 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:03:28.074 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.102+0000 7fba4da00640 1 --2- 172.21.15.97:0/1296305463 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e1714904e0 0x55e171494640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:03:28.074 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.102+0000 7fba4d000640 1 -- 172.21.15.97:0/1296305463 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e1714904e0 msgr2=0x55e171494640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:03:28.074 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.102+0000 7fba4d000640 1 --2- 172.21.15.97:0/1296305463 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e1714904e0 0x55e171494640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:28.075 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.102+0000 7fba4d000640 1 -- 172.21.15.97:0/1296305463 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e17148b370 msgr2=0x55e17149bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:03:28.075 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.102+0000 7fba4d000640 1 --2- 172.21.15.97:0/1296305463 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e17148b370 0x55e17149bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:28.075 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.102+0000 7fba4d000640 1 -- 172.21.15.97:0/1296305463 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1711d0620 con 0x55e17148a980 2024-03-27T11:03:28.075 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.102+0000 7fba47e00640 1 --2- 172.21.15.97:0/1296305463 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e17148b370 0x55e17149bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T11:03:28.075 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.103+0000 7fba4d000640 1 --2- 172.21.15.97:0/1296305463 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e17148a980 0x55e171489860 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fba3c0dd330 tx=0x7fba3c03c350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a006efc64eb65292 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:03:28.075 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.103+0000 7fba39600640 1 -- 172.21.15.97:0/1296305463 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba3c0be030 con 0x55e17148a980 2024-03-27T11:03:28.075 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.104+0000 7fba39600640 1 -- 172.21.15.97:0/1296305463 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fba3c003e30 con 0x55e17148a980 2024-03-27T11:03:28.075 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.104+0000 7fba39600640 1 -- 172.21.15.97:0/1296305463 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba3c0ba790 con 0x55e17148a980 2024-03-27T11:03:28.077 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.105+0000 7fba52817740 1 -- 172.21.15.97:0/1296305463 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e17148a980 msgr2=0x55e171489860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:03:28.077 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.105+0000 7fba52817740 1 --2- 172.21.15.97:0/1296305463 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e17148a980 0x55e171489860 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fba3c0dd330 tx=0x7fba3c03c350 comp rx=0 tx=0).stop 2024-03-27T11:03:28.077 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.105+0000 7fba52817740 1 -- 172.21.15.97:0/1296305463 shutdown_connections 2024-03-27T11:03:28.077 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.105+0000 7fba52817740 1 --2- 172.21.15.97:0/1296305463 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e1714904e0 0x55e171494640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:28.077 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.105+0000 7fba52817740 1 --2- 172.21.15.97:0/1296305463 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e17148b370 0x55e17149bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:28.077 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.105+0000 7fba52817740 1 --2- 172.21.15.97:0/1296305463 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e17148a980 0x55e171489860 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:28.077 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.105+0000 7fba52817740 1 -- 172.21.15.97:0/1296305463 >> 172.21.15.97:0/1296305463 conn(0x55e171485ef0 msgr2=0x55e171489150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:03:28.078 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.105+0000 7fba52817740 1 -- 172.21.15.97:0/1296305463 shutdown_connections 2024-03-27T11:03:28.078 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.105+0000 7fba52817740 1 -- 172.21.15.97:0/1296305463 wait complete. 2024-03-27T11:03:28.078 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.106+0000 7fba52817740 1 -- start start 2024-03-27T11:03:28.078 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.106+0000 7fba52817740 1 Processor -- start 2024-03-27T11:03:28.078 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.107+0000 7fba52817740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e17148a980 0x55e1714993e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:03:28.078 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.107+0000 7fba52817740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e17148b370 0x55e171499920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:03:28.078 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.107+0000 7fba52817740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e1714904e0 0x55e17149a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:03:28.078 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.107+0000 7fba52817740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fba481c1900 con 0x55e17148a980 2024-03-27T11:03:28.078 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.107+0000 7fba52817740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fba481c1230 con 0x55e1714904e0 2024-03-27T11:03:28.078 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.107+0000 7fba52817740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fba4804f8b0 con 0x55e17148b370 2024-03-27T11:03:28.079 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.107+0000 7fba52817740 10 client.0 _ll_register_callbacks cb 0x7fba40008570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T11:03:28.081 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.108+0000 7fba4d000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e17148a980 0x55e1714993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:03:28.081 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.108+0000 7fba4d000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e17148a980 0x55e1714993e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33744/0 (socket says 172.21.15.97:33744) 2024-03-27T11:03:28.081 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.108+0000 7fba4d000640 1 -- 172.21.15.97:0/2227242419 learned_addr learned my addr 172.21.15.97:0/2227242419 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:03:28.082 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.108+0000 7fba47e00640 1 --2- 172.21.15.97:0/2227242419 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e17148b370 0x55e171499920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:03:28.082 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.108+0000 7fba4da00640 1 --2- 172.21.15.97:0/2227242419 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e1714904e0 0x55e17149a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:03:28.082 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.108+0000 7fba4d000640 1 -- 172.21.15.97:0/2227242419 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e17148b370 msgr2=0x55e171499920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:03:28.082 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.108+0000 7fba4d000640 1 --2- 172.21.15.97:0/2227242419 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e17148b370 0x55e171499920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:28.082 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.108+0000 7fba4d000640 1 -- 172.21.15.97:0/2227242419 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e1714904e0 msgr2=0x55e17149a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:03:28.082 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.109+0000 7fba4d000640 1 --2- 172.21.15.97:0/2227242419 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e1714904e0 0x55e17149a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:28.082 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.109+0000 7fba4d000640 1 -- 172.21.15.97:0/2227242419 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e17149af20 con 0x55e17148a980 2024-03-27T11:03:28.082 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.109+0000 7fba4d000640 1 --2- 172.21.15.97:0/2227242419 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e17148a980 0x55e1714993e0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fba3c02ce10 tx=0x7fba3c0f5c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87637019051d88a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:03:28.083 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.109+0000 7fba2ea00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T11:03:28.083 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.110+0000 7fba2ea00640 1 -- 172.21.15.97:0/2227242419 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba3c0bde50 con 0x55e17148a980 2024-03-27T11:03:28.083 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.110+0000 7fba2ea00640 1 -- 172.21.15.97:0/2227242419 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fba3c0f5140 con 0x55e17148a980 2024-03-27T11:03:28.083 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.110+0000 7fba52817740 1 -- 172.21.15.97:0/2227242419 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba3c003e30 con 0x55e17148a980 2024-03-27T11:03:28.083 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.110+0000 7fba2ea00640 1 -- 172.21.15.97:0/2227242419 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fba3c0ba790 con 0x55e17148a980 2024-03-27T11:03:28.083 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.111+0000 7fba52817740 10 client.16668 fetch_fsmap learned FSMap version 34 2024-03-27T11:03:28.083 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.111+0000 7fba52817740 1 -- 172.21.15.97:0/2227242419 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55e17149af20 con 0x55e17148a980 2024-03-27T11:03:28.083 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.111+0000 7fba2ea00640 1 -- 172.21.15.97:0/2227242419 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fba3c003e30 con 0x55e17148a980 2024-03-27T11:03:28.083 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.111+0000 7fba2ea00640 1 -- 172.21.15.97:0/2227242419 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7fba3c0ba240 con 0x55e17148a980 2024-03-27T11:03:28.084 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.111+0000 7fba52817740 10 client.16668 fetch_fsmap finished waiting for FSMap version 34 2024-03-27T11:03:28.084 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.111+0000 7fba52817740 10 client.16668 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T11:03:28.084 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.111+0000 7fba52817740 20 client.16668 populate_metadata read hostname 'smithi097' 2024-03-27T11:03:28.084 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.112+0000 7fba52817740 1 --2- 172.21.15.97:0/2227242419 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55e171506b10 0x55e171508f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:03:28.084 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.112+0000 7fba52817740 4 client.16668 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-27T11:03:28.084 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.112+0000 7fba52817740 1 -- 172.21.15.97:0/2227242419 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e171375630 con 0x55e171506b10 2024-03-27T11:03:28.085 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.113+0000 7fba47e00640 1 --2- 172.21.15.97:0/2227242419 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55e171506b10 0x55e171508f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:03:28.085 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.114+0000 7fba47e00640 1 --2- 172.21.15.97:0/2227242419 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55e171506b10 0x55e171508f00 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8941c2925667c server_cookie=3efabacf0212b0bc in_seq=0 out_seq=0 2024-03-27T11:03:28.085 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.114+0000 7fba2ea00640 10 client.16668 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T11:03:28.097 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.126+0000 7fba2ea00640 1 -- 172.21.15.97:0/2227242419 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e171375630 con 0x55e171506b10 2024-03-27T11:03:28.097 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.126+0000 7fba2ea00640 10 client.16668 handle_command_reply: tid=0 2024-03-27T11:03:28.098 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.126+0000 7fba52817740 1 client.16668 shutdown 2024-03-27T11:03:28.098 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.126+0000 7fba52817740 1 -- 172.21.15.97:0/2227242419 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e17148a980 msgr2=0x55e1714993e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:03:28.098 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.126+0000 7fba52817740 1 --2- 172.21.15.97:0/2227242419 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e17148a980 0x55e1714993e0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fba3c02ce10 tx=0x7fba3c0f5c90 comp rx=0 tx=0).stop 2024-03-27T11:03:28.098 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.127+0000 7fba52817740 1 -- 172.21.15.97:0/2227242419 shutdown_connections 2024-03-27T11:03:28.098 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.127+0000 7fba52817740 1 --2- 172.21.15.97:0/2227242419 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55e171506b10 0x55e171508f00 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:28.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.127+0000 7fba52817740 1 --2- 172.21.15.97:0/2227242419 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e1714904e0 0x55e17149a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:28.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.127+0000 7fba52817740 1 --2- 172.21.15.97:0/2227242419 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e17148b370 0x55e171499920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:28.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.127+0000 7fba52817740 1 --2- 172.21.15.97:0/2227242419 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e17148a980 0x55e1714993e0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:28.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.127+0000 7fba52817740 1 -- 172.21.15.97:0/2227242419 >> 172.21.15.97:0/2227242419 conn(0x55e171485ef0 msgr2=0x55e1714945b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:03:28.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.127+0000 7fba2ea00640 0 client.16668 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T11:03:28.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.127+0000 7fba52817740 1 -- 172.21.15.97:0/2227242419 shutdown_connections 2024-03-27T11:03:28.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.127+0000 7fba52817740 1 -- 172.21.15.97:0/2227242419 wait complete. 2024-03-27T11:03:28.100 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.128+0000 7fba52817740 20 client.16668 trim_cache size 0 max 16384 2024-03-27T11:03:28.100 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-27T11:03:28.100 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-27T11:03:28.101 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-27T11:03:28.101 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-27T11:03:28.101 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.129+0000 7fba4fa00640 1 -- 172.21.15.97:0/1757843394 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fba300a3730 msgr2=0x7fba300a5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:03:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.129+0000 7fba4fa00640 1 --2- 172.21.15.97:0/1757843394 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fba300a3730 0x7fba300a5bf0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x55e17148bbd0 tx=0x7fba400086a0 comp rx=0 tx=0).stop 2024-03-27T11:03:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.129+0000 7fba4fa00640 1 -- 172.21.15.97:0/1757843394 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fba48103db0 msgr2=0x7fba4806d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:03:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.129+0000 7fba4fa00640 1 --2- 172.21.15.97:0/1757843394 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fba48103db0 0x7fba4806d160 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fba3c000c00 tx=0x7fba3c0040c0 comp rx=0 tx=0).stop 2024-03-27T11:03:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.129+0000 7fba4fa00640 1 -- 172.21.15.97:0/1757843394 shutdown_connections 2024-03-27T11:03:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.129+0000 7fba4fa00640 1 --2- 172.21.15.97:0/1757843394 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fba300a3730 0x7fba300a5bf0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.129+0000 7fba4fa00640 1 --2- 172.21.15.97:0/1757843394 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fba48105960 0x7fba48072700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.129+0000 7fba4fa00640 1 --2- 172.21.15.97:0/1757843394 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba48104fa0 0x7fba4806d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.129+0000 7fba4fa00640 1 --2- 172.21.15.97:0/1757843394 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fba48103db0 0x7fba4806d160 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.129+0000 7fba4fa00640 1 -- 172.21.15.97:0/1757843394 >> 172.21.15.97:0/1757843394 conn(0x7fba480ff340 msgr2=0x7fba48077fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:03:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.130+0000 7fba4fa00640 1 -- 172.21.15.97:0/1757843394 shutdown_connections 2024-03-27T11:03:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:28.130+0000 7fba4fa00640 1 -- 172.21.15.97:0/1757843394 wait complete. 2024-03-27T11:03:29.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:03:29 smithi097 ceph-mon[31859]: pgmap v646: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 54 MiB/s wr, 881 op/s 2024-03-27T11:03:29.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:03:29 smithi177 ceph-mon[35294]: pgmap v646: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 54 MiB/s wr, 881 op/s 2024-03-27T11:03:29.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:03:29 smithi114 ceph-mon[35338]: pgmap v646: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 54 MiB/s wr, 881 op/s 2024-03-27T11:03:31.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:03:31 smithi097 ceph-mon[31859]: pgmap v647: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 59 MiB/s wr, 951 op/s 2024-03-27T11:03:31.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:03:31 smithi177 ceph-mon[35294]: pgmap v647: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 59 MiB/s wr, 951 op/s 2024-03-27T11:03:31.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:03:31 smithi114 ceph-mon[35338]: pgmap v647: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 59 MiB/s wr, 951 op/s 2024-03-27T11:03:33.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:03:33 smithi097 ceph-mon[31859]: pgmap v648: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 53 MiB/s wr, 853 op/s 2024-03-27T11:03:33.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:03:33 smithi177 ceph-mon[35294]: pgmap v648: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 53 MiB/s wr, 853 op/s 2024-03-27T11:03:33.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:03:33 smithi114 ceph-mon[35338]: pgmap v648: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 53 MiB/s wr, 853 op/s 2024-03-27T11:03:35.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:03:35 smithi097 ceph-mon[31859]: pgmap v649: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 53 MiB/s wr, 866 op/s 2024-03-27T11:03:35.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:03:35 smithi177 ceph-mon[35294]: pgmap v649: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 53 MiB/s wr, 866 op/s 2024-03-27T11:03:35.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:03:35 smithi114 ceph-mon[35338]: pgmap v649: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 53 MiB/s wr, 866 op/s 2024-03-27T11:03:37.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:03:37 smithi097 ceph-mon[31859]: pgmap v650: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 51 MiB/s wr, 827 op/s 2024-03-27T11:03:37.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:03:37 smithi177 ceph-mon[35294]: pgmap v650: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 51 MiB/s wr, 827 op/s 2024-03-27T11:03:37.764 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:03:37 smithi114 ceph-mon[35338]: pgmap v650: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 51 MiB/s wr, 827 op/s 2024-03-27T11:03:39.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:03:39 smithi097 ceph-mon[31859]: pgmap v651: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 53 MiB/s wr, 856 op/s 2024-03-27T11:03:39.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:03:39 smithi177 ceph-mon[35294]: pgmap v651: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 53 MiB/s wr, 856 op/s 2024-03-27T11:03:39.764 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:03:39 smithi114 ceph-mon[35338]: pgmap v651: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 53 MiB/s wr, 856 op/s 2024-03-27T11:03:41.309 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.339+0000 7f1ddb200640 1 -- 172.21.15.97:0/1504391543 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1dd410b580 msgr2=0x7f1dd410ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:03:41.310 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.339+0000 7f1ddb200640 1 --2- 172.21.15.97:0/1504391543 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1dd410b580 0x7f1dd410ba00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc009860 tx=0x7f1dbc02bb70 comp rx=0 tx=0).stop 2024-03-27T11:03:41.310 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.339+0000 7f1ddb200640 1 -- 172.21.15.97:0/1504391543 shutdown_connections 2024-03-27T11:03:41.310 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.339+0000 7f1ddb200640 1 --2- 172.21.15.97:0/1504391543 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1dd410bf40 0x7f1dd41127d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:41.310 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.339+0000 7f1ddb200640 1 --2- 172.21.15.97:0/1504391543 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1dd410b580 0x7f1dd410ba00 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:41.310 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.339+0000 7f1ddb200640 1 --2- 172.21.15.97:0/1504391543 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1dd410a390 0x7f1dd410a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:41.310 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.339+0000 7f1ddb200640 1 -- 172.21.15.97:0/1504391543 >> 172.21.15.97:0/1504391543 conn(0x7f1dd4070b40 msgr2=0x7f1dd4072f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:03:41.311 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.339+0000 7f1ddb200640 1 -- 172.21.15.97:0/1504391543 shutdown_connections 2024-03-27T11:03:41.311 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.340+0000 7f1ddb200640 1 -- 172.21.15.97:0/1504391543 wait complete. 2024-03-27T11:03:41.312 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.342+0000 7f1ddb200640 1 Processor -- start 2024-03-27T11:03:41.312 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.342+0000 7f1ddb200640 1 -- start start 2024-03-27T11:03:41.314 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.343+0000 7f1ddb200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1dd410a390 0x7f1dd41a6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:03:41.314 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.343+0000 7f1ddb200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1dd410b580 0x7f1dd41a7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:03:41.314 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.343+0000 7f1ddb200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1dd410bf40 0x7f1dd41ae0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:03:41.315 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.343+0000 7f1ddb200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd411c540 con 0x7f1dd410bf40 2024-03-27T11:03:41.315 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.343+0000 7f1ddb200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd411c3c0 con 0x7f1dd410b580 2024-03-27T11:03:41.315 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.343+0000 7f1ddb200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd411c6c0 con 0x7f1dd410a390 2024-03-27T11:03:41.315 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.343+0000 7f1dd3e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1dd410a390 0x7f1dd41a6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:03:41.315 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.343+0000 7f1dd3e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1dd410a390 0x7f1dd41a6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.97:33578/0 (socket says 172.21.15.97:33578) 2024-03-27T11:03:41.315 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.343+0000 7f1dd3e00640 1 -- 172.21.15.97:0/2243664169 learned_addr learned my addr 172.21.15.97:0/2243664169 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:03:41.315 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.344+0000 7f1dd9200640 1 --2- 172.21.15.97:0/2243664169 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1dd410bf40 0x7f1dd41ae0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:03:41.315 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.344+0000 7f1dd3400640 1 --2- 172.21.15.97:0/2243664169 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1dd410b580 0x7f1dd41a7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:03:41.315 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.344+0000 7f1dd3e00640 1 -- 172.21.15.97:0/2243664169 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1dd410b580 msgr2=0x7f1dd41a7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:03:41.316 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.344+0000 7f1dd3e00640 1 --2- 172.21.15.97:0/2243664169 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1dd410b580 0x7f1dd41a7040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:41.316 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.344+0000 7f1dd3e00640 1 -- 172.21.15.97:0/2243664169 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1dd410bf40 msgr2=0x7f1dd41ae0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:03:41.317 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.344+0000 7f1dd3e00640 1 --2- 172.21.15.97:0/2243664169 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1dd410bf40 0x7f1dd41ae0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:41.317 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.344+0000 7f1dd3e00640 1 -- 172.21.15.97:0/2243664169 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dd41ae720 con 0x7f1dd410a390 2024-03-27T11:03:41.317 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.344+0000 7f1dd9200640 1 --2- 172.21.15.97:0/2243664169 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1dd410bf40 0x7f1dd41ae0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T11:03:41.317 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.345+0000 7f1dd3e00640 1 --2- 172.21.15.97:0/2243664169 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1dd410a390 0x7f1dd41a6b00 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc00c3f0 tx=0x7f1dcc009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:03:41.317 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.345+0000 7f1dd0c00640 1 -- 172.21.15.97:0/2243664169 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dcc01b070 con 0x7f1dd410a390 2024-03-27T11:03:41.317 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.345+0000 7f1dd0c00640 1 -- 172.21.15.97:0/2243664169 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1dcc00bae0 con 0x7f1dd410a390 2024-03-27T11:03:41.317 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.345+0000 7f1dd0c00640 1 -- 172.21.15.97:0/2243664169 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dcc016c50 con 0x7f1dd410a390 2024-03-27T11:03:41.317 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.345+0000 7f1ddb200640 1 -- 172.21.15.97:0/2243664169 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dd41aea10 con 0x7f1dd410a390 2024-03-27T11:03:41.317 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.346+0000 7f1ddb200640 1 -- 172.21.15.97:0/2243664169 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dd41aefb0 con 0x7f1dd410a390 2024-03-27T11:03:41.322 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.351+0000 7f1dddfe8740 1 Processor -- start 2024-03-27T11:03:41.323 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.352+0000 7f1dddfe8740 1 -- start start 2024-03-27T11:03:41.323 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.352+0000 7f1dd0c00640 1 -- 172.21.15.97:0/2243664169 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f1dcc01e070 con 0x7f1dd410a390 2024-03-27T11:03:41.323 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.352+0000 7f1dddfe8740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558573540980 0x558573540da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:03:41.323 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.352+0000 7f1dddfe8740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558573541370 0x558573545e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:03:41.324 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.352+0000 7f1dddfe8740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5585735464e0 0x558573548970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:03:41.324 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.352+0000 7f1dddfe8740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5585734a5d10 con 0x558573540980 2024-03-27T11:03:41.324 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.352+0000 7f1dddfe8740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55857344e260 con 0x5585735464e0 2024-03-27T11:03:41.324 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.352+0000 7f1dddfe8740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5585734a1cc0 con 0x558573541370 2024-03-27T11:03:41.324 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.353+0000 7f1dd3400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5585735464e0 0x558573548970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:03:41.324 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.353+0000 7f1dd3400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5585735464e0 0x558573548970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:51338/0 (socket says 172.21.15.97:51338) 2024-03-27T11:03:41.324 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.353+0000 7f1dd3400640 1 -- 172.21.15.97:0/3544977431 learned_addr learned my addr 172.21.15.97:0/3544977431 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:03:41.325 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.353+0000 7f1dd9200640 1 --2- 172.21.15.97:0/3544977431 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558573540980 0x558573540da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:03:41.325 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.353+0000 7f1dd3e00640 1 --2- 172.21.15.97:0/3544977431 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558573541370 0x558573545e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:03:41.325 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.353+0000 7f1dd3400640 1 -- 172.21.15.97:0/3544977431 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558573541370 msgr2=0x558573545e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:03:41.325 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.353+0000 7f1dd3400640 1 --2- 172.21.15.97:0/3544977431 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558573541370 0x558573545e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:41.325 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.353+0000 7f1dd3400640 1 -- 172.21.15.97:0/3544977431 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558573540980 msgr2=0x558573540da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:03:41.325 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.353+0000 7f1dd3400640 1 --2- 172.21.15.97:0/3544977431 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558573540980 0x558573540da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:41.325 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.353+0000 7f1dd3400640 1 -- 172.21.15.97:0/3544977431 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558573445aa0 con 0x5585735464e0 2024-03-27T11:03:41.326 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.354+0000 7f1dd3e00640 1 --2- 172.21.15.97:0/3544977431 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558573541370 0x558573545e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T11:03:41.326 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.354+0000 7f1dd3400640 1 --2- 172.21.15.97:0/3544977431 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5585735464e0 0x558573548970 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc000c00 tx=0x7f1dbc009540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aae03975a23a78ee server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:03:41.326 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.354+0000 7f1dd0c00640 1 --2- 172.21.15.97:0/2243664169 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f1db80a3790 0x7f1db80a5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:03:41.327 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.355+0000 7f1dd0c00640 1 -- 172.21.15.97:0/2243664169 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f1dcc0cfc50 con 0x7f1dd410a390 2024-03-27T11:03:41.327 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.355+0000 7f1dd9200640 1 --2- 172.21.15.97:0/2243664169 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f1db80a3790 0x7f1db80a5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:03:41.327 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.355+0000 7f1dc9600640 1 -- 172.21.15.97:0/3544977431 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dbc039070 con 0x5585735464e0 2024-03-27T11:03:41.327 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.355+0000 7f1dc9600640 1 -- 172.21.15.97:0/3544977431 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1dbc03a040 con 0x5585735464e0 2024-03-27T11:03:41.327 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.355+0000 7f1dc9600640 1 -- 172.21.15.97:0/3544977431 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dbc0053c0 con 0x5585735464e0 2024-03-27T11:03:41.327 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.356+0000 7f1dddfe8740 1 -- 172.21.15.97:0/3544977431 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5585735464e0 msgr2=0x558573548970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:03:41.327 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.356+0000 7f1dddfe8740 1 --2- 172.21.15.97:0/3544977431 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5585735464e0 0x558573548970 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc000c00 tx=0x7f1dbc009540 comp rx=0 tx=0).stop 2024-03-27T11:03:41.327 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.356+0000 7f1dd9200640 1 --2- 172.21.15.97:0/2243664169 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f1db80a3790 0x7f1db80a5c50 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x558573541bd0 tx=0x7f1dc4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:03:41.327 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.356+0000 7f1dddfe8740 1 -- 172.21.15.97:0/3544977431 shutdown_connections 2024-03-27T11:03:41.328 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.356+0000 7f1dddfe8740 1 --2- 172.21.15.97:0/3544977431 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5585735464e0 0x558573548970 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:41.328 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.356+0000 7f1dddfe8740 1 --2- 172.21.15.97:0/3544977431 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558573541370 0x558573545e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:41.328 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.356+0000 7f1dddfe8740 1 --2- 172.21.15.97:0/3544977431 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558573540980 0x558573540da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:41.328 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.356+0000 7f1dddfe8740 1 -- 172.21.15.97:0/3544977431 >> 172.21.15.97:0/3544977431 conn(0x55857353bef0 msgr2=0x55857353c2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:03:41.328 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.357+0000 7f1dddfe8740 1 -- 172.21.15.97:0/3544977431 shutdown_connections 2024-03-27T11:03:41.328 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.357+0000 7f1dddfe8740 1 -- 172.21.15.97:0/3544977431 wait complete. 2024-03-27T11:03:41.329 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.358+0000 7f1dddfe8740 1 -- start start 2024-03-27T11:03:41.329 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.358+0000 7f1dddfe8740 1 Processor -- start 2024-03-27T11:03:41.330 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.359+0000 7f1dddfe8740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558573540980 0x55857354ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:03:41.330 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.359+0000 7f1dddfe8740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558573541370 0x55857354b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:03:41.331 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.359+0000 7f1dddfe8740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5585735464e0 0x5585735521d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:03:41.331 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.359+0000 7f1dddfe8740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x558573549430 con 0x558573540980 2024-03-27T11:03:41.331 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.359+0000 7f1dd3400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558573541370 0x55857354b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:03:41.331 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.359+0000 7f1dd3400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558573541370 0x55857354b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.97:33590/0 (socket says 172.21.15.97:33590) 2024-03-27T11:03:41.331 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.359+0000 7f1dd3400640 1 -- 172.21.15.97:0/4037341401 learned_addr learned my addr 172.21.15.97:0/4037341401 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:03:41.331 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.359+0000 7f1dd9200640 1 --2- 172.21.15.97:0/4037341401 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5585735464e0 0x5585735521d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:03:41.331 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.359+0000 7f1dddfe8740 1 -- 172.21.15.97:0/4037341401 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x558573461630 con 0x5585735464e0 2024-03-27T11:03:41.331 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.359+0000 7f1dddfe8740 1 -- 172.21.15.97:0/4037341401 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5585735495b0 con 0x558573541370 2024-03-27T11:03:41.331 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.360+0000 7f1dd3e00640 1 --2- 172.21.15.97:0/4037341401 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558573540980 0x55857354ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:03:41.331 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.360+0000 7f1dddfe8740 10 client.0 _ll_register_callbacks cb 0x558573466c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T11:03:41.332 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.360+0000 7f1dd9200640 1 -- 172.21.15.97:0/4037341401 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558573541370 msgr2=0x55857354b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:03:41.332 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.360+0000 7f1dd9200640 1 --2- 172.21.15.97:0/4037341401 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558573541370 0x55857354b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:41.332 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.360+0000 7f1dd9200640 1 -- 172.21.15.97:0/4037341401 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558573540980 msgr2=0x55857354ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:03:41.332 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.360+0000 7f1dd9200640 1 --2- 172.21.15.97:0/4037341401 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558573540980 0x55857354ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:41.333 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.360+0000 7f1dd9200640 1 -- 172.21.15.97:0/4037341401 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55857346d5f0 con 0x5585735464e0 2024-03-27T11:03:41.333 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.361+0000 7f1dd9200640 1 --2- 172.21.15.97:0/4037341401 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5585735464e0 0x5585735521d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1dc4011bc0 tx=0x7f1dc40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=582db2775f8a1e17 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:03:41.333 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.361+0000 7f1dc2000640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-03-27T11:03:41.333 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.361+0000 7f1dc2000640 1 -- 172.21.15.97:0/4037341401 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc4011cb0 con 0x5585735464e0 2024-03-27T11:03:41.333 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.361+0000 7f1dc2000640 1 -- 172.21.15.97:0/4037341401 <== mon.2 v2:172.21.15.114:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f1dc4010950 con 0x5585735464e0 2024-03-27T11:03:41.333 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.361+0000 7f1dddfe8740 1 -- 172.21.15.97:0/4037341401 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1dbc03a040 con 0x5585735464e0 2024-03-27T11:03:41.334 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.362+0000 7f1dc2000640 1 -- 172.21.15.97:0/4037341401 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f1dc4011e50 con 0x5585735464e0 2024-03-27T11:03:41.334 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.362+0000 7f1dddfe8740 10 client.36347 fetch_fsmap learned FSMap version 34 2024-03-27T11:03:41.334 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.362+0000 7f1dddfe8740 1 -- 172.21.15.97:0/4037341401 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5585731ba210 con 0x5585735464e0 2024-03-27T11:03:41.334 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.363+0000 7f1dc2000640 1 -- 172.21.15.97:0/4037341401 <== mon.2 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1dbc03a040 con 0x5585735464e0 2024-03-27T11:03:41.334 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.363+0000 7f1dc2000640 1 -- 172.21.15.97:0/4037341401 <== mon.2 v2:172.21.15.114:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f1dc401e410 con 0x5585735464e0 2024-03-27T11:03:41.334 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.363+0000 7f1dddfe8740 10 client.36347 fetch_fsmap finished waiting for FSMap version 34 2024-03-27T11:03:41.334 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.363+0000 7f1dddfe8740 10 client.36347 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T11:03:41.335 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.363+0000 7f1dddfe8740 20 client.36347 populate_metadata read hostname 'smithi097' 2024-03-27T11:03:41.335 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.364+0000 7f1dddfe8740 1 --2- 172.21.15.97:0/4037341401 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x558573556ab0 0x558573558ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:03:41.335 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.364+0000 7f1dddfe8740 4 client.36347 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T11:03:41.335 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.364+0000 7f1dddfe8740 1 -- 172.21.15.97:0/4037341401 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5585734b3250 con 0x558573556ab0 2024-03-27T11:03:41.335 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.364+0000 7f1dd3e00640 1 --2- 172.21.15.97:0/4037341401 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x558573556ab0 0x558573558ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:03:41.336 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.365+0000 7f1dd3e00640 1 --2- 172.21.15.97:0/4037341401 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x558573556ab0 0x558573558ef0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6cf20c415a3ec76e server_cookie=125285ee866416e3 in_seq=0 out_seq=0 2024-03-27T11:03:41.336 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.365+0000 7f1dc2000640 10 client.36347 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T11:03:41.336 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.366+0000 7f1dc2000640 1 -- 172.21.15.97:0/4037341401 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5585734b3250 con 0x558573556ab0 2024-03-27T11:03:41.336 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.366+0000 7f1dc2000640 10 client.36347 handle_command_reply: tid=0 2024-03-27T11:03:41.336 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.366+0000 7f1dddfe8740 1 client.36347 shutdown 2024-03-27T11:03:41.337 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.366+0000 7f1dddfe8740 1 -- 172.21.15.97:0/4037341401 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5585735464e0 msgr2=0x5585735521d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:03:41.337 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.366+0000 7f1dddfe8740 1 --2- 172.21.15.97:0/4037341401 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5585735464e0 0x5585735521d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1dc4011bc0 tx=0x7f1dc40046c0 comp rx=0 tx=0).stop 2024-03-27T11:03:41.337 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.367+0000 7f1dddfe8740 1 -- 172.21.15.97:0/4037341401 shutdown_connections 2024-03-27T11:03:41.338 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.367+0000 7f1dddfe8740 1 --2- 172.21.15.97:0/4037341401 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x558573556ab0 0x558573558ef0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:41.338 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.367+0000 7f1dddfe8740 1 --2- 172.21.15.97:0/4037341401 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5585735464e0 0x5585735521d0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:41.338 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.367+0000 7f1dddfe8740 1 --2- 172.21.15.97:0/4037341401 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558573541370 0x55857354b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:41.338 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.367+0000 7f1dddfe8740 1 --2- 172.21.15.97:0/4037341401 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558573540980 0x55857354ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:41.338 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.367+0000 7f1dddfe8740 1 -- 172.21.15.97:0/4037341401 >> 172.21.15.97:0/4037341401 conn(0x55857353bef0 msgr2=0x55857353b0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:03:41.338 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.367+0000 7f1dc2000640 0 client.36347 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T11:03:41.339 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.367+0000 7f1dddfe8740 1 -- 172.21.15.97:0/4037341401 shutdown_connections 2024-03-27T11:03:41.339 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.367+0000 7f1dddfe8740 1 -- 172.21.15.97:0/4037341401 wait complete. 2024-03-27T11:03:41.340 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.368+0000 7f1dddfe8740 20 client.36347 trim_cache size 0 max 16384 2024-03-27T11:03:41.371 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.400+0000 7f1dddfe8740 1 Processor -- start 2024-03-27T11:03:41.371 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.400+0000 7f1dddfe8740 1 -- start start 2024-03-27T11:03:41.371 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.401+0000 7f1dddfe8740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558573540980 0x55857353f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:03:41.371 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.401+0000 7f1dddfe8740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558573541370 0x558573552190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:03:41.371 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.401+0000 7f1dddfe8740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5585735464e0 0x558573549f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:03:41.372 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.401+0000 7f1dddfe8740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd41b7c70 con 0x558573541370 2024-03-27T11:03:41.372 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.401+0000 7f1dddfe8740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd41b97b0 con 0x5585735464e0 2024-03-27T11:03:41.372 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.401+0000 7f1dddfe8740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd41b8340 con 0x558573540980 2024-03-27T11:03:41.373 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.401+0000 7f1dd3400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558573541370 0x558573552190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:03:41.373 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.401+0000 7f1dd3e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558573540980 0x55857353f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:03:41.373 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.401+0000 7f1dd3e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558573540980 0x55857353f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.97:33594/0 (socket says 172.21.15.97:33594) 2024-03-27T11:03:41.373 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.401+0000 7f1dd3e00640 1 -- 172.21.15.97:0/3370428238 learned_addr learned my addr 172.21.15.97:0/3370428238 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:03:41.373 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.401+0000 7f1dd9200640 1 --2- 172.21.15.97:0/3370428238 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5585735464e0 0x558573549f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:03:41.373 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.402+0000 7f1dd9200640 1 -- 172.21.15.97:0/3370428238 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558573540980 msgr2=0x55857353f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:03:41.373 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.402+0000 7f1dd9200640 1 --2- 172.21.15.97:0/3370428238 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558573540980 0x55857353f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:41.373 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.402+0000 7f1dd9200640 1 -- 172.21.15.97:0/3370428238 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558573541370 msgr2=0x558573552190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:03:41.373 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.402+0000 7f1dd9200640 1 --2- 172.21.15.97:0/3370428238 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558573541370 0x558573552190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:41.374 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.402+0000 7f1dd9200640 1 -- 172.21.15.97:0/3370428238 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558573286620 con 0x5585735464e0 2024-03-27T11:03:41.374 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.402+0000 7f1dd3400640 1 --2- 172.21.15.97:0/3370428238 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558573541370 0x558573552190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T11:03:41.374 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.402+0000 7f1dd9200640 1 --2- 172.21.15.97:0/3370428238 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5585735464e0 0x558573549f20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1dc4011870 tx=0x7f1dc4011920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59d69dcfcacc70e0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:03:41.374 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.403+0000 7f1dc8c00640 1 -- 172.21.15.97:0/3370428238 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc4011cb0 con 0x5585735464e0 2024-03-27T11:03:41.374 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.403+0000 7f1dc8c00640 1 -- 172.21.15.97:0/3370428238 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1dc402e030 con 0x5585735464e0 2024-03-27T11:03:41.375 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.403+0000 7f1dc8c00640 1 -- 172.21.15.97:0/3370428238 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc4010850 con 0x5585735464e0 2024-03-27T11:03:41.375 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.404+0000 7f1dddfe8740 1 -- 172.21.15.97:0/3370428238 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5585735464e0 msgr2=0x558573549f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:03:41.375 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.404+0000 7f1dddfe8740 1 --2- 172.21.15.97:0/3370428238 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5585735464e0 0x558573549f20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1dc4011870 tx=0x7f1dc4011920 comp rx=0 tx=0).stop 2024-03-27T11:03:41.375 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.404+0000 7f1dddfe8740 1 -- 172.21.15.97:0/3370428238 shutdown_connections 2024-03-27T11:03:41.375 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.404+0000 7f1dddfe8740 1 --2- 172.21.15.97:0/3370428238 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5585735464e0 0x558573549f20 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:41.376 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.404+0000 7f1dddfe8740 1 --2- 172.21.15.97:0/3370428238 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558573541370 0x558573552190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:41.376 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.404+0000 7f1dddfe8740 1 --2- 172.21.15.97:0/3370428238 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558573540980 0x55857353f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:41.376 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.404+0000 7f1dddfe8740 1 -- 172.21.15.97:0/3370428238 >> 172.21.15.97:0/3370428238 conn(0x55857353bef0 msgr2=0x55857353f150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:03:41.376 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.404+0000 7f1dddfe8740 1 -- 172.21.15.97:0/3370428238 shutdown_connections 2024-03-27T11:03:41.376 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.404+0000 7f1dddfe8740 1 -- 172.21.15.97:0/3370428238 wait complete. 2024-03-27T11:03:41.376 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.405+0000 7f1dddfe8740 1 -- start start 2024-03-27T11:03:41.376 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.405+0000 7f1dddfe8740 1 Processor -- start 2024-03-27T11:03:41.377 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.406+0000 7f1dddfe8740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558573540980 0x55857354fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:03:41.377 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.406+0000 7f1dddfe8740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558573541370 0x558573550430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:03:41.377 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.406+0000 7f1dddfe8740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5585735464e0 0x5585735513f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:03:41.377 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.406+0000 7f1dddfe8740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd41b90e0 con 0x558573541370 2024-03-27T11:03:41.377 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.406+0000 7f1dddfe8740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd41b8a10 con 0x5585735464e0 2024-03-27T11:03:41.377 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.406+0000 7f1dddfe8740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd41b5c50 con 0x558573540980 2024-03-27T11:03:41.377 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.406+0000 7f1dddfe8740 10 client.0 _ll_register_callbacks cb 0x7f1dc40098f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T11:03:41.378 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.406+0000 7f1dd9200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5585735464e0 0x5585735513f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:03:41.378 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.406+0000 7f1dd3400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558573541370 0x558573550430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:03:41.378 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.407+0000 7f1dd3400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558573541370 0x558573550430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36872/0 (socket says 172.21.15.97:36872) 2024-03-27T11:03:41.378 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.407+0000 7f1dd3400640 1 -- 172.21.15.97:0/805624840 learned_addr learned my addr 172.21.15.97:0/805624840 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:03:41.378 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.407+0000 7f1dd3e00640 1 --2- 172.21.15.97:0/805624840 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558573540980 0x55857354fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:03:41.380 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.407+0000 7f1dd9200640 1 -- 172.21.15.97:0/805624840 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558573540980 msgr2=0x55857354fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:03:41.380 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.407+0000 7f1dd9200640 1 --2- 172.21.15.97:0/805624840 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558573540980 0x55857354fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:41.380 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.407+0000 7f1dd9200640 1 -- 172.21.15.97:0/805624840 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558573541370 msgr2=0x558573550430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:03:41.380 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.407+0000 7f1dd9200640 1 --2- 172.21.15.97:0/805624840 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558573541370 0x558573550430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:41.380 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.407+0000 7f1dd9200640 1 -- 172.21.15.97:0/805624840 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585735519e0 con 0x5585735464e0 2024-03-27T11:03:41.380 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.407+0000 7f1dd3400640 1 --2- 172.21.15.97:0/805624840 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558573541370 0x558573550430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T11:03:41.380 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.408+0000 7f1dd9200640 1 --2- 172.21.15.97:0/805624840 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5585735464e0 0x5585735513f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1dc401e880 tx=0x7f1dc4009e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a453eb59f1b3cbc4 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:03:41.380 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.408+0000 7f1dc2000640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-03-27T11:03:41.380 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.408+0000 7f1dc2000640 1 -- 172.21.15.97:0/805624840 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc4005040 con 0x5585735464e0 2024-03-27T11:03:41.380 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.408+0000 7f1dc2000640 1 -- 172.21.15.97:0/805624840 <== mon.2 v2:172.21.15.114:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f1dc40051e0 con 0x5585735464e0 2024-03-27T11:03:41.381 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.408+0000 7f1dddfe8740 1 -- 172.21.15.97:0/805624840 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1dc402e030 con 0x5585735464e0 2024-03-27T11:03:41.381 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.408+0000 7f1dc2000640 1 -- 172.21.15.97:0/805624840 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f1dc4005460 con 0x5585735464e0 2024-03-27T11:03:41.381 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.409+0000 7f1dddfe8740 10 client.36353 fetch_fsmap learned FSMap version 34 2024-03-27T11:03:41.381 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.409+0000 7f1dddfe8740 1 -- 172.21.15.97:0/805624840 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5585735bd400 con 0x5585735464e0 2024-03-27T11:03:41.381 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.409+0000 7f1dc2000640 1 -- 172.21.15.97:0/805624840 <== mon.2 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1dc402e030 con 0x5585735464e0 2024-03-27T11:03:41.381 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.409+0000 7f1dc2000640 1 -- 172.21.15.97:0/805624840 <== mon.2 v2:172.21.15.114:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f1dc40047a0 con 0x5585735464e0 2024-03-27T11:03:41.381 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.409+0000 7f1dddfe8740 10 client.36353 fetch_fsmap finished waiting for FSMap version 34 2024-03-27T11:03:41.381 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.409+0000 7f1dddfe8740 10 client.36353 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T11:03:41.381 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.409+0000 7f1dddfe8740 20 client.36353 populate_metadata read hostname 'smithi097' 2024-03-27T11:03:41.381 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.409+0000 7f1dddfe8740 1 --2- 172.21.15.97:0/805624840 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5585735bdd10 0x5585735c0130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:03:41.382 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.409+0000 7f1dddfe8740 4 client.36353 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-27T11:03:41.382 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.409+0000 7f1dddfe8740 1 -- 172.21.15.97:0/805624840 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5585734ad5a0 con 0x5585735bdd10 2024-03-27T11:03:41.382 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.410+0000 7f1dd3e00640 1 --2- 172.21.15.97:0/805624840 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5585735bdd10 0x5585735c0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:03:41.382 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.412+0000 7f1dc2000640 10 client.36353 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T11:03:41.382 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.412+0000 7f1dd3e00640 1 --2- 172.21.15.97:0/805624840 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5585735bdd10 0x5585735c0130 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6938a18be50b0aa5 server_cookie=a5a14abe69e1cda7 in_seq=0 out_seq=0 2024-03-27T11:03:41.391 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.421+0000 7f1dc2000640 1 -- 172.21.15.97:0/805624840 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5585734ad5a0 con 0x5585735bdd10 2024-03-27T11:03:41.391 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.421+0000 7f1dc2000640 10 client.36353 handle_command_reply: tid=0 2024-03-27T11:03:41.391 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.421+0000 7f1dddfe8740 1 client.36353 shutdown 2024-03-27T11:03:41.393 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.422+0000 7f1dddfe8740 1 -- 172.21.15.97:0/805624840 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5585735464e0 msgr2=0x5585735513f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:03:41.393 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.422+0000 7f1dddfe8740 1 --2- 172.21.15.97:0/805624840 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5585735464e0 0x5585735513f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1dc401e880 tx=0x7f1dc4009e00 comp rx=0 tx=0).stop 2024-03-27T11:03:41.393 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.422+0000 7f1dddfe8740 1 -- 172.21.15.97:0/805624840 shutdown_connections 2024-03-27T11:03:41.393 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.422+0000 7f1dddfe8740 1 --2- 172.21.15.97:0/805624840 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5585735bdd10 0x5585735c0130 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:41.393 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.422+0000 7f1dddfe8740 1 --2- 172.21.15.97:0/805624840 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5585735464e0 0x5585735513f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:41.393 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.422+0000 7f1dc2000640 0 client.36353 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T11:03:41.393 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.422+0000 7f1dddfe8740 1 --2- 172.21.15.97:0/805624840 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558573541370 0x558573550430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:41.393 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.422+0000 7f1dddfe8740 1 --2- 172.21.15.97:0/805624840 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558573540980 0x55857354fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:41.393 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.422+0000 7f1dddfe8740 1 -- 172.21.15.97:0/805624840 >> 172.21.15.97:0/805624840 conn(0x55857353bef0 msgr2=0x558573549f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:03:41.393 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.422+0000 7f1dddfe8740 1 -- 172.21.15.97:0/805624840 shutdown_connections 2024-03-27T11:03:41.394 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.422+0000 7f1dddfe8740 1 -- 172.21.15.97:0/805624840 wait complete. 2024-03-27T11:03:41.394 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.422+0000 7f1dddfe8740 20 client.36353 trim_cache size 0 max 16384 2024-03-27T11:03:41.394 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-27T11:03:41.394 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-27T11:03:41.394 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-27T11:03:41.394 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-27T11:03:41.395 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.423+0000 7f1ddb200640 1 -- 172.21.15.97:0/2243664169 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f1db80a3790 msgr2=0x7f1db80a5c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:03:41.395 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.423+0000 7f1ddb200640 1 --2- 172.21.15.97:0/2243664169 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f1db80a3790 0x7f1db80a5c50 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x558573541bd0 tx=0x7f1dc4005f90 comp rx=0 tx=0).stop 2024-03-27T11:03:41.395 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.423+0000 7f1ddb200640 1 -- 172.21.15.97:0/2243664169 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1dd410a390 msgr2=0x7f1dd41a6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:03:41.395 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.423+0000 7f1ddb200640 1 --2- 172.21.15.97:0/2243664169 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1dd410a390 0x7f1dd41a6b00 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc00c3f0 tx=0x7f1dcc009f90 comp rx=0 tx=0).stop 2024-03-27T11:03:41.395 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.423+0000 7f1ddb200640 1 -- 172.21.15.97:0/2243664169 shutdown_connections 2024-03-27T11:03:41.395 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.423+0000 7f1ddb200640 1 --2- 172.21.15.97:0/2243664169 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1dd410bf40 0x7f1dd41ae0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:41.396 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.423+0000 7f1ddb200640 1 --2- 172.21.15.97:0/2243664169 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f1db80a3790 0x7f1db80a5c50 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:41.396 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.424+0000 7f1ddb200640 1 --2- 172.21.15.97:0/2243664169 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1dd410b580 0x7f1dd41a7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:41.396 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.424+0000 7f1ddb200640 1 --2- 172.21.15.97:0/2243664169 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1dd410a390 0x7f1dd41a6b00 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:41.396 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.424+0000 7f1ddb200640 1 -- 172.21.15.97:0/2243664169 >> 172.21.15.97:0/2243664169 conn(0x7f1dd4070b40 msgr2=0x7f1dd410e880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:03:41.396 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.424+0000 7f1ddb200640 1 -- 172.21.15.97:0/2243664169 shutdown_connections 2024-03-27T11:03:41.396 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:41.424+0000 7f1ddb200640 1 -- 172.21.15.97:0/2243664169 wait complete. 2024-03-27T11:03:41.638 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:03:41 smithi097 ceph-mon[31859]: pgmap v652: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 57 MiB/s wr, 926 op/s 2024-03-27T11:03:41.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:03:41 smithi177 ceph-mon[35294]: pgmap v652: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 57 MiB/s wr, 926 op/s 2024-03-27T11:03:41.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:03:41 smithi114 ceph-mon[35338]: pgmap v652: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 57 MiB/s wr, 926 op/s 2024-03-27T11:03:43.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:03:43 smithi097 ceph-mon[31859]: pgmap v653: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 51 MiB/s wr, 824 op/s 2024-03-27T11:03:43.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:03:43 smithi177 ceph-mon[35294]: pgmap v653: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 51 MiB/s wr, 824 op/s 2024-03-27T11:03:43.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:03:43 smithi114 ceph-mon[35338]: pgmap v653: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 51 MiB/s wr, 824 op/s 2024-03-27T11:03:45.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:03:45 smithi097 ceph-mon[31859]: pgmap v654: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 52 MiB/s wr, 842 op/s 2024-03-27T11:03:45.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:03:45 smithi177 ceph-mon[35294]: pgmap v654: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 52 MiB/s wr, 842 op/s 2024-03-27T11:03:45.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:03:45 smithi114 ceph-mon[35338]: pgmap v654: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 52 MiB/s wr, 842 op/s 2024-03-27T11:03:47.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:03:47 smithi097 ceph-mon[31859]: pgmap v655: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 50 MiB/s wr, 813 op/s 2024-03-27T11:03:47.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:03:47 smithi177 ceph-mon[35294]: pgmap v655: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 50 MiB/s wr, 813 op/s 2024-03-27T11:03:47.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:03:47 smithi114 ceph-mon[35338]: pgmap v655: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 50 MiB/s wr, 813 op/s 2024-03-27T11:03:49.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:03:49 smithi097 ceph-mon[31859]: pgmap v656: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 54 MiB/s wr, 867 op/s 2024-03-27T11:03:49.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:03:49 smithi177 ceph-mon[35294]: pgmap v656: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 54 MiB/s wr, 867 op/s 2024-03-27T11:03:49.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:03:49 smithi114 ceph-mon[35338]: pgmap v656: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 54 MiB/s wr, 867 op/s 2024-03-27T11:03:50.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:03:50 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T11:03:50.764 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:03:50 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T11:03:50.890 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:03:50 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T11:03:51.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:03:51 smithi177 ceph-mon[35294]: pgmap v657: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 58 MiB/s wr, 940 op/s 2024-03-27T11:03:51.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:03:51 smithi114 ceph-mon[35338]: pgmap v657: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 58 MiB/s wr, 940 op/s 2024-03-27T11:03:51.889 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:03:51 smithi097 ceph-mon[31859]: pgmap v657: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 58 MiB/s wr, 940 op/s 2024-03-27T11:03:52.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:03:52 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-27T11:03:52.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:03:52 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-27T11:03:52.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:03:52 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-27T11:03:52.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:03:52 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-27T11:03:52.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:03:52 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T11:03:52.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:03:52 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T11:03:52.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:03:52 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T11:03:52.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:03:52 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-27T11:03:52.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:03:52 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-27T11:03:52.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:03:52 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-27T11:03:52.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:03:52 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-27T11:03:52.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:03:52 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T11:03:52.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:03:52 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T11:03:52.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:03:52 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T11:03:52.889 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:03:52 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-27T11:03:52.889 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:03:52 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-27T11:03:52.889 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:03:52 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-27T11:03:52.889 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:03:52 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-27T11:03:52.889 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:03:52 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T11:03:52.889 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:03:52 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T11:03:52.889 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:03:52 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' 2024-03-27T11:03:53.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:03:53 smithi177 ceph-mon[35294]: Adjusting osd_memory_target on smithi177 to 176.7M 2024-03-27T11:03:53.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:03:53 smithi177 ceph-mon[35294]: Unable to set osd_memory_target on smithi177 to 185329356: error parsing value: Value '185329356' is below minimum 939524096 2024-03-27T11:03:53.676 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:03:53 smithi177 ceph-mon[35294]: pgmap v658: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 52 MiB/s wr, 845 op/s 2024-03-27T11:03:53.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:03:53 smithi114 ceph-mon[35338]: Adjusting osd_memory_target on smithi177 to 176.7M 2024-03-27T11:03:53.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:03:53 smithi114 ceph-mon[35338]: Unable to set osd_memory_target on smithi177 to 185329356: error parsing value: Value '185329356' is below minimum 939524096 2024-03-27T11:03:53.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:03:53 smithi114 ceph-mon[35338]: pgmap v658: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 52 MiB/s wr, 845 op/s 2024-03-27T11:03:53.889 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:03:53 smithi097 ceph-mon[31859]: Adjusting osd_memory_target on smithi177 to 176.7M 2024-03-27T11:03:53.889 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:03:53 smithi097 ceph-mon[31859]: Unable to set osd_memory_target on smithi177 to 185329356: error parsing value: Value '185329356' is below minimum 939524096 2024-03-27T11:03:53.889 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:03:53 smithi097 ceph-mon[31859]: pgmap v658: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 52 MiB/s wr, 845 op/s 2024-03-27T11:03:54.606 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.635+0000 7f5e58c00640 1 -- 172.21.15.97:0/3075591877 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5e540780f0 msgr2=0x7f5e54078990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:03:54.606 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.635+0000 7f5e58c00640 1 --2- 172.21.15.97:0/3075591877 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5e540780f0 0x7f5e54078990 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c009be0 tx=0x7f5e4c02bef0 comp rx=0 tx=0).stop 2024-03-27T11:03:54.606 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.635+0000 7f5e58c00640 1 -- 172.21.15.97:0/3075591877 shutdown_connections 2024-03-27T11:03:54.606 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.635+0000 7f5e58c00640 1 --2- 172.21.15.97:0/3075591877 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5e540780f0 0x7f5e54078990 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:54.606 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.635+0000 7f5e58c00640 1 --2- 172.21.15.97:0/3075591877 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e54077750 0x7f5e54077bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:54.607 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.635+0000 7f5e58c00640 1 --2- 172.21.15.97:0/3075591877 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e54076510 0x7f5e54076930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:54.607 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.635+0000 7f5e58c00640 1 -- 172.21.15.97:0/3075591877 >> 172.21.15.97:0/3075591877 conn(0x7f5e540ff160 msgr2=0x7f5e54101580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:03:54.607 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.635+0000 7f5e58c00640 1 -- 172.21.15.97:0/3075591877 shutdown_connections 2024-03-27T11:03:54.607 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.635+0000 7f5e58c00640 1 -- 172.21.15.97:0/3075591877 wait complete. 2024-03-27T11:03:54.607 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.637+0000 7f5e58c00640 1 Processor -- start 2024-03-27T11:03:54.608 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.637+0000 7f5e58c00640 1 -- start start 2024-03-27T11:03:54.608 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.637+0000 7f5e58c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5e54076510 0x7f5e5419e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:03:54.608 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.637+0000 7f5e58c00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e54077750 0x7f5e5419e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:03:54.609 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.638+0000 7f5e58c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e540780f0 0x7f5e541a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:03:54.609 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.638+0000 7f5e58c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5e54113b50 con 0x7f5e540780f0 2024-03-27T11:03:54.609 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.638+0000 7f5e58c00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f5e541139d0 con 0x7f5e54076510 2024-03-27T11:03:54.609 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.638+0000 7f5e58c00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f5e54113cd0 con 0x7f5e54077750 2024-03-27T11:03:54.609 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.638+0000 7f5e53e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e540780f0 0x7f5e541a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:03:54.609 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.638+0000 7f5e53400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5e54076510 0x7f5e5419e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:03:54.609 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.638+0000 7f5e52a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e54077750 0x7f5e5419e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:03:54.609 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.638+0000 7f5e53400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5e54076510 0x7f5e5419e1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:34972/0 (socket says 172.21.15.97:34972) 2024-03-27T11:03:54.609 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.638+0000 7f5e53400640 1 -- 172.21.15.97:0/1250989745 learned_addr learned my addr 172.21.15.97:0/1250989745 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:03:54.610 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.639+0000 7f5e52a00640 1 -- 172.21.15.97:0/1250989745 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5e54076510 msgr2=0x7f5e5419e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:03:54.610 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.639+0000 7f5e52a00640 1 --2- 172.21.15.97:0/1250989745 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5e54076510 0x7f5e5419e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:54.610 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.639+0000 7f5e52a00640 1 -- 172.21.15.97:0/1250989745 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e540780f0 msgr2=0x7f5e541a5780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:03:54.610 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.639+0000 7f5e52a00640 1 --2- 172.21.15.97:0/1250989745 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e540780f0 0x7f5e541a5780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:54.610 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.639+0000 7f5e52a00640 1 -- 172.21.15.97:0/1250989745 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e541a5df0 con 0x7f5e54077750 2024-03-27T11:03:54.610 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.639+0000 7f5e52a00640 1 --2- 172.21.15.97:0/1250989745 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e54077750 0x7f5e5419e710 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5e4000e790 tx=0x7f5e4000ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:03:54.611 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.639+0000 7f5e4be00640 1 -- 172.21.15.97:0/1250989745 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e4000eea0 con 0x7f5e54077750 2024-03-27T11:03:54.611 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.639+0000 7f5e4be00640 1 -- 172.21.15.97:0/1250989745 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5e4000cc40 con 0x7f5e54077750 2024-03-27T11:03:54.611 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.639+0000 7f5e4be00640 1 -- 172.21.15.97:0/1250989745 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e40018660 con 0x7f5e54077750 2024-03-27T11:03:54.611 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.640+0000 7f5e58c00640 1 -- 172.21.15.97:0/1250989745 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e541a60e0 con 0x7f5e54077750 2024-03-27T11:03:54.611 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.640+0000 7f5e58c00640 1 -- 172.21.15.97:0/1250989745 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e541a6660 con 0x7f5e54077750 2024-03-27T11:03:54.615 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.644+0000 7f5e5cfa6740 1 Processor -- start 2024-03-27T11:03:54.615 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.645+0000 7f5e5cfa6740 1 -- start start 2024-03-27T11:03:54.616 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.645+0000 7f5e5cfa6740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b22c291b60 0x55b22c291f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:03:54.616 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.645+0000 7f5e5cfa6740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b22c292550 0x55b22c297050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:03:54.616 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.645+0000 7f5e5cfa6740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b22c2976c0 0x55b22c299b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:03:54.616 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.645+0000 7f5e5cfa6740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55b22c1f6d10 con 0x55b22c291b60 2024-03-27T11:03:54.616 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.645+0000 7f5e5cfa6740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55b22c19f260 con 0x55b22c2976c0 2024-03-27T11:03:54.616 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.645+0000 7f5e5cfa6740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55b22c1f2cc0 con 0x55b22c292550 2024-03-27T11:03:54.617 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.646+0000 7f5e53400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b22c292550 0x55b22c297050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:03:54.617 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.646+0000 7f5e53400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b22c292550 0x55b22c297050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.97:33546/0 (socket says 172.21.15.97:33546) 2024-03-27T11:03:54.617 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.646+0000 7f5e53e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b22c291b60 0x55b22c291f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:03:54.617 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.646+0000 7f5e53400640 1 -- 172.21.15.97:0/1614037867 learned_addr learned my addr 172.21.15.97:0/1614037867 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:03:54.618 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.647+0000 7f5e53e00640 1 -- 172.21.15.97:0/1614037867 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b22c292550 msgr2=0x55b22c297050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:03:54.618 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.647+0000 7f5e53e00640 1 --2- 172.21.15.97:0/1614037867 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b22c292550 0x55b22c297050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:54.618 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.647+0000 7f5e4be00640 1 -- 172.21.15.97:0/1250989745 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f5e40010430 con 0x7f5e54077750 2024-03-27T11:03:54.618 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.647+0000 7f5e53e00640 1 -- 172.21.15.97:0/1614037867 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b22c2976c0 msgr2=0x55b22c299b50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-27T11:03:54.618 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.647+0000 7f5e53e00640 1 --2- 172.21.15.97:0/1614037867 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b22c2976c0 0x55b22c299b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:54.618 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.647+0000 7f5e53e00640 1 -- 172.21.15.97:0/1614037867 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b22c196aa0 con 0x55b22c291b60 2024-03-27T11:03:54.619 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.648+0000 7f5e53400640 1 --2- 172.21.15.97:0/1614037867 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b22c292550 0x55b22c297050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T11:03:54.619 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.648+0000 7f5e53e00640 1 --2- 172.21.15.97:0/1614037867 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b22c291b60 0x55b22c291f80 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c036040 tx=0x7f5e4c033500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94b6a516f348e1bc server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:03:54.619 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.649+0000 7f5e3fe00640 1 -- 172.21.15.97:0/1614037867 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e4c043030 con 0x55b22c291b60 2024-03-27T11:03:54.619 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.649+0000 7f5e3fe00640 1 -- 172.21.15.97:0/1614037867 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5e4c0336b0 con 0x55b22c291b60 2024-03-27T11:03:54.619 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.649+0000 7f5e3fe00640 1 -- 172.21.15.97:0/1614037867 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e4c02c890 con 0x55b22c291b60 2024-03-27T11:03:54.620 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.649+0000 7f5e4be00640 1 --2- 172.21.15.97:0/1250989745 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f5e380a36c0 0x7f5e380a5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:03:54.620 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.649+0000 7f5e4be00640 1 -- 172.21.15.97:0/1250989745 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f5e400cfe10 con 0x7f5e54077750 2024-03-27T11:03:54.620 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.650+0000 7f5e5cfa6740 1 -- 172.21.15.97:0/1614037867 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b22c291b60 msgr2=0x55b22c291f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:03:54.620 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.650+0000 7f5e5cfa6740 1 --2- 172.21.15.97:0/1614037867 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b22c291b60 0x55b22c291f80 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c036040 tx=0x7f5e4c033500 comp rx=0 tx=0).stop 2024-03-27T11:03:54.621 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.650+0000 7f5e5cfa6740 1 -- 172.21.15.97:0/1614037867 shutdown_connections 2024-03-27T11:03:54.621 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.650+0000 7f5e5cfa6740 1 --2- 172.21.15.97:0/1614037867 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b22c2976c0 0x55b22c299b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:54.621 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.650+0000 7f5e5cfa6740 1 --2- 172.21.15.97:0/1614037867 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b22c292550 0x55b22c297050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:54.621 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.650+0000 7f5e5cfa6740 1 --2- 172.21.15.97:0/1614037867 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b22c291b60 0x55b22c291f80 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:54.621 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.650+0000 7f5e5cfa6740 1 -- 172.21.15.97:0/1614037867 >> 172.21.15.97:0/1614037867 conn(0x55b22c28d0d0 msgr2=0x55b22c28d4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:03:54.622 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.651+0000 7f5e5cfa6740 1 -- 172.21.15.97:0/1614037867 shutdown_connections 2024-03-27T11:03:54.622 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.651+0000 7f5e53400640 1 --2- 172.21.15.97:0/1250989745 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f5e380a36c0 0x7f5e380a5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:03:54.622 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.651+0000 7f5e5cfa6740 1 -- 172.21.15.97:0/1614037867 wait complete. 2024-03-27T11:03:54.623 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.652+0000 7f5e5cfa6740 1 -- start start 2024-03-27T11:03:54.623 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.652+0000 7f5e5cfa6740 1 Processor -- start 2024-03-27T11:03:54.623 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.653+0000 7f5e53400640 1 --2- 172.21.15.97:0/1250989745 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f5e380a36c0 0x7f5e380a5b80 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f5e44006a70 tx=0x7f5e440091c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:03:54.624 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.653+0000 7f5e5cfa6740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b22c291b60 0x55b22c29bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:03:54.624 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.653+0000 7f5e5cfa6740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b22c292550 0x55b22c29c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:03:54.624 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.653+0000 7f5e5cfa6740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b22c2976c0 0x55b22c2a3480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:03:54.624 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.653+0000 7f5e5cfa6740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55b22c29a610 con 0x55b22c2976c0 2024-03-27T11:03:54.624 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.653+0000 7f5e5cfa6740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55b22c1b2630 con 0x55b22c291b60 2024-03-27T11:03:54.624 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.653+0000 7f5e5cfa6740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55b22c29a790 con 0x55b22c292550 2024-03-27T11:03:54.624 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.653+0000 7f5e5cfa6740 10 client.0 _ll_register_callbacks cb 0x55b22c1b7c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T11:03:54.625 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.653+0000 7f5e53400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b22c291b60 0x55b22c29bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:03:54.625 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.653+0000 7f5e53400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b22c291b60 0x55b22c29bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:34998/0 (socket says 172.21.15.97:34998) 2024-03-27T11:03:54.625 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.653+0000 7f5e53400640 1 -- 172.21.15.97:0/2567186473 learned_addr learned my addr 172.21.15.97:0/2567186473 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:03:54.625 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.653+0000 7f5e52a00640 1 --2- 172.21.15.97:0/2567186473 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b22c292550 0x55b22c29c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:03:54.625 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.654+0000 7f5e53e00640 1 --2- 172.21.15.97:0/2567186473 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b22c2976c0 0x55b22c2a3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:03:54.625 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.654+0000 7f5e53400640 1 -- 172.21.15.97:0/2567186473 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b22c292550 msgr2=0x55b22c29c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:03:54.625 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.654+0000 7f5e53400640 1 --2- 172.21.15.97:0/2567186473 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b22c292550 0x55b22c29c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:54.625 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.654+0000 7f5e53400640 1 -- 172.21.15.97:0/2567186473 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b22c2976c0 msgr2=0x55b22c2a3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:03:54.625 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.654+0000 7f5e53400640 1 --2- 172.21.15.97:0/2567186473 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b22c2976c0 0x55b22c2a3480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:54.626 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.654+0000 7f5e53400640 1 -- 172.21.15.97:0/2567186473 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b22c1be5f0 con 0x55b22c291b60 2024-03-27T11:03:54.626 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.654+0000 7f5e53400640 1 --2- 172.21.15.97:0/2567186473 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b22c291b60 0x55b22c29bf00 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5e440149f0 tx=0x7f5e440046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a429fd22f0aecab3 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:03:54.626 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.654+0000 7f5e3cc00640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-03-27T11:03:54.627 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.655+0000 7f5e3cc00640 1 -- 172.21.15.97:0/2567186473 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e44025070 con 0x55b22c291b60 2024-03-27T11:03:54.627 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.655+0000 7f5e3cc00640 1 -- 172.21.15.97:0/2567186473 <== mon.2 v2:172.21.15.114:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f5e44021600 con 0x55b22c291b60 2024-03-27T11:03:54.627 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.655+0000 7f5e5cfa6740 1 -- 172.21.15.97:0/2567186473 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e4c0336b0 con 0x55b22c291b60 2024-03-27T11:03:54.627 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.656+0000 7f5e3cc00640 1 -- 172.21.15.97:0/2567186473 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f5e44021800 con 0x55b22c291b60 2024-03-27T11:03:54.627 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.656+0000 7f5e5cfa6740 10 client.36362 fetch_fsmap learned FSMap version 34 2024-03-27T11:03:54.627 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.656+0000 7f5e5cfa6740 1 -- 172.21.15.97:0/2567186473 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55b22bf0b210 con 0x55b22c291b60 2024-03-27T11:03:54.627 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.656+0000 7f5e3cc00640 1 -- 172.21.15.97:0/2567186473 <== mon.2 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5e4c0336b0 con 0x55b22c291b60 2024-03-27T11:03:54.627 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.656+0000 7f5e3cc00640 1 -- 172.21.15.97:0/2567186473 <== mon.2 v2:172.21.15.114:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f5e44020070 con 0x55b22c291b60 2024-03-27T11:03:54.627 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.656+0000 7f5e5cfa6740 10 client.36362 fetch_fsmap finished waiting for FSMap version 34 2024-03-27T11:03:54.628 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.656+0000 7f5e5cfa6740 10 client.36362 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T11:03:54.628 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.656+0000 7f5e5cfa6740 20 client.36362 populate_metadata read hostname 'smithi097' 2024-03-27T11:03:54.628 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.657+0000 7f5e5cfa6740 1 --2- 172.21.15.97:0/2567186473 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55b22c2b8d30 0x55b22c2bb120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:03:54.628 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.657+0000 7f5e5cfa6740 4 client.36362 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T11:03:54.628 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.657+0000 7f5e5cfa6740 1 -- 172.21.15.97:0/2567186473 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b22c204250 con 0x55b22c2b8d30 2024-03-27T11:03:54.628 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.657+0000 7f5e52a00640 1 --2- 172.21.15.97:0/2567186473 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55b22c2b8d30 0x55b22c2bb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:03:54.628 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.658+0000 7f5e52a00640 1 --2- 172.21.15.97:0/2567186473 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55b22c2b8d30 0x55b22c2bb120 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d627c60041ecc32 server_cookie=c42637ba4193a374 in_seq=0 out_seq=0 2024-03-27T11:03:54.628 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.658+0000 7f5e3cc00640 10 client.36362 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T11:03:54.630 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.659+0000 7f5e3cc00640 1 -- 172.21.15.97:0/2567186473 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55b22c204250 con 0x55b22c2b8d30 2024-03-27T11:03:54.630 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.659+0000 7f5e3cc00640 10 client.36362 handle_command_reply: tid=0 2024-03-27T11:03:54.630 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.659+0000 7f5e5cfa6740 1 client.36362 shutdown 2024-03-27T11:03:54.630 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.660+0000 7f5e5cfa6740 1 -- 172.21.15.97:0/2567186473 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b22c291b60 msgr2=0x55b22c29bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:03:54.631 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.660+0000 7f5e5cfa6740 1 --2- 172.21.15.97:0/2567186473 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b22c291b60 0x55b22c29bf00 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5e440149f0 tx=0x7f5e440046c0 comp rx=0 tx=0).stop 2024-03-27T11:03:54.631 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.660+0000 7f5e5cfa6740 1 -- 172.21.15.97:0/2567186473 shutdown_connections 2024-03-27T11:03:54.631 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.660+0000 7f5e5cfa6740 1 --2- 172.21.15.97:0/2567186473 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55b22c2b8d30 0x55b22c2bb120 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:54.631 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.660+0000 7f5e5cfa6740 1 --2- 172.21.15.97:0/2567186473 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b22c2976c0 0x55b22c2a3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:54.631 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.660+0000 7f5e5cfa6740 1 --2- 172.21.15.97:0/2567186473 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b22c292550 0x55b22c29c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:54.631 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.660+0000 7f5e5cfa6740 1 --2- 172.21.15.97:0/2567186473 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b22c291b60 0x55b22c29bf00 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:54.631 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.660+0000 7f5e5cfa6740 1 -- 172.21.15.97:0/2567186473 >> 172.21.15.97:0/2567186473 conn(0x55b22c28d0d0 msgr2=0x55b22c28c310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:03:54.631 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.660+0000 7f5e3cc00640 0 client.36362 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T11:03:54.631 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.660+0000 7f5e5cfa6740 1 -- 172.21.15.97:0/2567186473 shutdown_connections 2024-03-27T11:03:54.631 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.660+0000 7f5e5cfa6740 1 -- 172.21.15.97:0/2567186473 wait complete. 2024-03-27T11:03:54.632 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.660+0000 7f5e5cfa6740 20 client.36362 trim_cache size 0 max 16384 2024-03-27T11:03:54.658 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.688+0000 7f5e5cfa6740 1 Processor -- start 2024-03-27T11:03:54.658 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.688+0000 7f5e5cfa6740 1 -- start start 2024-03-27T11:03:54.658 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.688+0000 7f5e5cfa6740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b22c291b60 0x55b22c28d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:03:54.659 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.688+0000 7f5e5cfa6740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b22c292550 0x55b22c2a3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:03:54.659 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.688+0000 7f5e5cfa6740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b22c2976c0 0x55b22c29c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:03:54.659 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.688+0000 7f5e5cfa6740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5e541a6c30 con 0x55b22c292550 2024-03-27T11:03:54.659 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.688+0000 7f5e5cfa6740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f5e541a8770 con 0x55b22c291b60 2024-03-27T11:03:54.659 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.688+0000 7f5e5cfa6740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f5e541a7300 con 0x55b22c2976c0 2024-03-27T11:03:54.659 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.689+0000 7f5e53400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b22c291b60 0x55b22c28d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:03:54.659 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.689+0000 7f5e53400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b22c291b60 0x55b22c28d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:35014/0 (socket says 172.21.15.97:35014) 2024-03-27T11:03:54.660 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.689+0000 7f5e53e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b22c2976c0 0x55b22c29c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:03:54.660 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.689+0000 7f5e53e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b22c2976c0 0x55b22c29c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.97:33574/0 (socket says 172.21.15.97:33574) 2024-03-27T11:03:54.660 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.689+0000 7f5e53e00640 1 -- 172.21.15.97:0/1667861899 learned_addr learned my addr 172.21.15.97:0/1667861899 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:03:54.660 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.689+0000 7f5e52a00640 1 --2- 172.21.15.97:0/1667861899 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b22c292550 0x55b22c2a3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:03:54.660 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.690+0000 7f5e53e00640 1 -- 172.21.15.97:0/1667861899 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b22c291b60 msgr2=0x55b22c28d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:03:54.660 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.690+0000 7f5e53e00640 1 --2- 172.21.15.97:0/1667861899 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b22c291b60 0x55b22c28d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:54.660 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.690+0000 7f5e53e00640 1 -- 172.21.15.97:0/1667861899 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b22c292550 msgr2=0x55b22c2a3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:03:54.661 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.690+0000 7f5e53e00640 1 --2- 172.21.15.97:0/1667861899 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b22c292550 0x55b22c2a3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:54.661 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.690+0000 7f5e53e00640 1 -- 172.21.15.97:0/1667861899 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b22bfd7620 con 0x55b22c2976c0 2024-03-27T11:03:54.661 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.690+0000 7f5e53400640 1 --2- 172.21.15.97:0/1667861899 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b22c291b60 0x55b22c28d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T11:03:54.661 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.690+0000 7f5e52a00640 1 --2- 172.21.15.97:0/1667861899 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b22c292550 0x55b22c2a3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T11:03:54.661 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.691+0000 7f5e53e00640 1 --2- 172.21.15.97:0/1667861899 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b22c2976c0 0x55b22c29c220 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c009be0 tx=0x7f5e4c009a60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c78789060852a20 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:03:54.662 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.691+0000 7f5e3f400640 1 -- 172.21.15.97:0/1667861899 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e4c04c070 con 0x55b22c2976c0 2024-03-27T11:03:54.662 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.691+0000 7f5e3f400640 1 -- 172.21.15.97:0/1667861899 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5e4c003e20 con 0x55b22c2976c0 2024-03-27T11:03:54.662 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.691+0000 7f5e3f400640 1 -- 172.21.15.97:0/1667861899 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e4c043040 con 0x55b22c2976c0 2024-03-27T11:03:54.663 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.692+0000 7f5e5cfa6740 1 -- 172.21.15.97:0/1667861899 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b22c2976c0 msgr2=0x55b22c29c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:03:54.663 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.692+0000 7f5e5cfa6740 1 --2- 172.21.15.97:0/1667861899 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b22c2976c0 0x55b22c29c220 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c009be0 tx=0x7f5e4c009a60 comp rx=0 tx=0).stop 2024-03-27T11:03:54.663 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.692+0000 7f5e5cfa6740 1 -- 172.21.15.97:0/1667861899 shutdown_connections 2024-03-27T11:03:54.663 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.692+0000 7f5e5cfa6740 1 --2- 172.21.15.97:0/1667861899 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b22c2976c0 0x55b22c29c220 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:54.663 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.692+0000 7f5e5cfa6740 1 --2- 172.21.15.97:0/1667861899 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b22c292550 0x55b22c2a3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:54.663 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.692+0000 7f5e5cfa6740 1 --2- 172.21.15.97:0/1667861899 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b22c291b60 0x55b22c28d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:54.663 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.692+0000 7f5e5cfa6740 1 -- 172.21.15.97:0/1667861899 >> 172.21.15.97:0/1667861899 conn(0x55b22c28d0d0 msgr2=0x55b22c29bb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:03:54.664 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.692+0000 7f5e5cfa6740 1 -- 172.21.15.97:0/1667861899 shutdown_connections 2024-03-27T11:03:54.664 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.693+0000 7f5e5cfa6740 1 -- 172.21.15.97:0/1667861899 wait complete. 2024-03-27T11:03:54.664 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.693+0000 7f5e5cfa6740 1 -- start start 2024-03-27T11:03:54.664 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.693+0000 7f5e5cfa6740 1 Processor -- start 2024-03-27T11:03:54.664 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.694+0000 7f5e5cfa6740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b22c291b60 0x55b22c2a0570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:03:54.665 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.694+0000 7f5e5cfa6740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b22c292550 0x55b22c2a0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:03:54.665 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.694+0000 7f5e5cfa6740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b22c2976c0 0x55b22c2a1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:03:54.665 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.694+0000 7f5e5cfa6740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5e541a80a0 con 0x55b22c291b60 2024-03-27T11:03:54.665 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.694+0000 7f5e5cfa6740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f5e541a79d0 con 0x55b22c2976c0 2024-03-27T11:03:54.665 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.694+0000 7f5e5cfa6740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f5e5410a090 con 0x55b22c292550 2024-03-27T11:03:54.665 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.694+0000 7f5e5cfa6740 10 client.0 _ll_register_callbacks cb 0x7f5e440097c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T11:03:54.665 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.694+0000 7f5e53400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b22c291b60 0x55b22c2a0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:03:54.665 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.694+0000 7f5e53400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b22c291b60 0x55b22c2a0570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55924/0 (socket says 172.21.15.97:55924) 2024-03-27T11:03:54.665 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.694+0000 7f5e53400640 1 -- 172.21.15.97:0/835951646 learned_addr learned my addr 172.21.15.97:0/835951646 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:03:54.667 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.694+0000 7f5e53e00640 1 --2- 172.21.15.97:0/835951646 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b22c2976c0 0x55b22c2a1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:03:54.667 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.694+0000 7f5e52a00640 1 --2- 172.21.15.97:0/835951646 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b22c292550 0x55b22c2a0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:03:54.667 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.695+0000 7f5e53400640 1 -- 172.21.15.97:0/835951646 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b22c292550 msgr2=0x55b22c2a0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:03:54.667 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.695+0000 7f5e53400640 1 --2- 172.21.15.97:0/835951646 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b22c292550 0x55b22c2a0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:54.667 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.695+0000 7f5e53400640 1 -- 172.21.15.97:0/835951646 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b22c2976c0 msgr2=0x55b22c2a1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:03:54.667 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.695+0000 7f5e53400640 1 --2- 172.21.15.97:0/835951646 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b22c2976c0 0x55b22c2a1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:54.667 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.695+0000 7f5e53400640 1 -- 172.21.15.97:0/835951646 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b22c2a20b0 con 0x55b22c291b60 2024-03-27T11:03:54.667 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.695+0000 7f5e52a00640 1 --2- 172.21.15.97:0/835951646 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b22c292550 0x55b22c2a0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T11:03:54.667 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.695+0000 7f5e53e00640 1 --2- 172.21.15.97:0/835951646 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b22c2976c0 0x55b22c2a1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T11:03:54.667 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.695+0000 7f5e53400640 1 --2- 172.21.15.97:0/835951646 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b22c291b60 0x55b22c2a0570 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f5e44016040 tx=0x7f5e44021f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4200849dd8043853 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:03:54.668 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.695+0000 7f5e3cc00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T11:03:54.668 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.695+0000 7f5e3cc00640 1 -- 172.21.15.97:0/835951646 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e44025040 con 0x55b22c291b60 2024-03-27T11:03:54.668 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.695+0000 7f5e5cfa6740 1 -- 172.21.15.97:0/835951646 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e4c003e20 con 0x55b22c291b60 2024-03-27T11:03:54.668 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.695+0000 7f5e3cc00640 1 -- 172.21.15.97:0/835951646 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f5e44021600 con 0x55b22c291b60 2024-03-27T11:03:54.668 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.696+0000 7f5e3cc00640 1 -- 172.21.15.97:0/835951646 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f5e440133a0 con 0x55b22c291b60 2024-03-27T11:03:54.668 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.696+0000 7f5e5cfa6740 10 client.16701 fetch_fsmap learned FSMap version 34 2024-03-27T11:03:54.668 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.696+0000 7f5e5cfa6740 1 -- 172.21.15.97:0/835951646 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55b22c2a3330 con 0x55b22c291b60 2024-03-27T11:03:54.668 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.696+0000 7f5e3cc00640 1 -- 172.21.15.97:0/835951646 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5e4c003e20 con 0x55b22c291b60 2024-03-27T11:03:54.669 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.697+0000 7f5e3cc00640 1 -- 172.21.15.97:0/835951646 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f5e44020040 con 0x55b22c291b60 2024-03-27T11:03:54.669 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.697+0000 7f5e5cfa6740 10 client.16701 fetch_fsmap finished waiting for FSMap version 34 2024-03-27T11:03:54.669 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.697+0000 7f5e5cfa6740 10 client.16701 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T11:03:54.669 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.697+0000 7f5e5cfa6740 20 client.16701 populate_metadata read hostname 'smithi097' 2024-03-27T11:03:54.669 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.697+0000 7f5e5cfa6740 1 --2- 172.21.15.97:0/835951646 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55b22c3304e0 0x55b22c332900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:03:54.669 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.697+0000 7f5e5cfa6740 4 client.16701 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-27T11:03:54.669 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.697+0000 7f5e5cfa6740 1 -- 172.21.15.97:0/835951646 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b22c1fe5a0 con 0x55b22c3304e0 2024-03-27T11:03:54.669 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.698+0000 7f5e52a00640 1 --2- 172.21.15.97:0/835951646 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55b22c3304e0 0x55b22c332900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:03:54.669 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.698+0000 7f5e52a00640 1 --2- 172.21.15.97:0/835951646 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55b22c3304e0 0x55b22c332900 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=260ee21e1e036102 server_cookie=3d515571d83ccb26 in_seq=0 out_seq=0 2024-03-27T11:03:54.670 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.698+0000 7f5e3cc00640 10 client.16701 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T11:03:54.677 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.706+0000 7f5e3cc00640 1 -- 172.21.15.97:0/835951646 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b22c1fe5a0 con 0x55b22c3304e0 2024-03-27T11:03:54.677 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.706+0000 7f5e3cc00640 10 client.16701 handle_command_reply: tid=0 2024-03-27T11:03:54.677 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.706+0000 7f5e5cfa6740 1 client.16701 shutdown 2024-03-27T11:03:54.677 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.706+0000 7f5e5cfa6740 1 -- 172.21.15.97:0/835951646 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b22c291b60 msgr2=0x55b22c2a0570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:03:54.677 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.706+0000 7f5e5cfa6740 1 --2- 172.21.15.97:0/835951646 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b22c291b60 0x55b22c2a0570 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f5e44016040 tx=0x7f5e44021f60 comp rx=0 tx=0).stop 2024-03-27T11:03:54.677 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.706+0000 7f5e5cfa6740 1 -- 172.21.15.97:0/835951646 shutdown_connections 2024-03-27T11:03:54.677 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.706+0000 7f5e5cfa6740 1 --2- 172.21.15.97:0/835951646 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55b22c3304e0 0x55b22c332900 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:54.677 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.706+0000 7f5e5cfa6740 1 --2- 172.21.15.97:0/835951646 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b22c2976c0 0x55b22c2a1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:54.677 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.706+0000 7f5e5cfa6740 1 --2- 172.21.15.97:0/835951646 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b22c292550 0x55b22c2a0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:54.677 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.706+0000 7f5e5cfa6740 1 --2- 172.21.15.97:0/835951646 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b22c291b60 0x55b22c2a0570 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:54.677 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.706+0000 7f5e5cfa6740 1 -- 172.21.15.97:0/835951646 >> 172.21.15.97:0/835951646 conn(0x55b22c28d0d0 msgr2=0x55b22c290330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:03:54.677 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.706+0000 7f5e3cc00640 0 client.16701 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T11:03:54.678 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.707+0000 7f5e5cfa6740 1 -- 172.21.15.97:0/835951646 shutdown_connections 2024-03-27T11:03:54.678 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.707+0000 7f5e5cfa6740 1 -- 172.21.15.97:0/835951646 wait complete. 2024-03-27T11:03:54.678 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.707+0000 7f5e5cfa6740 20 client.16701 trim_cache size 0 max 16384 2024-03-27T11:03:54.679 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-27T11:03:54.679 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-27T11:03:54.679 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-27T11:03:54.679 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-27T11:03:54.680 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.708+0000 7f5e58c00640 1 -- 172.21.15.97:0/1250989745 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f5e380a36c0 msgr2=0x7f5e380a5b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:03:54.680 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.708+0000 7f5e58c00640 1 --2- 172.21.15.97:0/1250989745 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f5e380a36c0 0x7f5e380a5b80 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f5e44006a70 tx=0x7f5e440091c0 comp rx=0 tx=0).stop 2024-03-27T11:03:54.680 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.708+0000 7f5e58c00640 1 -- 172.21.15.97:0/1250989745 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e54077750 msgr2=0x7f5e5419e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:03:54.680 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.708+0000 7f5e58c00640 1 --2- 172.21.15.97:0/1250989745 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e54077750 0x7f5e5419e710 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5e4000e790 tx=0x7f5e4000ec50 comp rx=0 tx=0).stop 2024-03-27T11:03:54.680 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.709+0000 7f5e58c00640 1 -- 172.21.15.97:0/1250989745 shutdown_connections 2024-03-27T11:03:54.680 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.709+0000 7f5e58c00640 1 --2- 172.21.15.97:0/1250989745 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e540780f0 0x7f5e541a5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:54.680 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.709+0000 7f5e58c00640 1 --2- 172.21.15.97:0/1250989745 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e54077750 0x7f5e5419e710 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:54.680 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.709+0000 7f5e58c00640 1 --2- 172.21.15.97:0/1250989745 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f5e380a36c0 0x7f5e380a5b80 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:54.680 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.709+0000 7f5e58c00640 1 --2- 172.21.15.97:0/1250989745 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5e54076510 0x7f5e5419e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:03:54.680 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.709+0000 7f5e58c00640 1 -- 172.21.15.97:0/1250989745 >> 172.21.15.97:0/1250989745 conn(0x7f5e540ff160 msgr2=0x7f5e54100830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:03:54.680 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.709+0000 7f5e58c00640 1 -- 172.21.15.97:0/1250989745 shutdown_connections 2024-03-27T11:03:54.681 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:03:54.710+0000 7f5e58c00640 1 -- 172.21.15.97:0/1250989745 wait complete. 2024-03-27T11:03:55.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:03:55 smithi177 ceph-mon[35294]: pgmap v659: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 53 MiB/s wr, 852 op/s 2024-03-27T11:03:55.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:03:55 smithi114 ceph-mon[35338]: pgmap v659: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 53 MiB/s wr, 852 op/s 2024-03-27T11:03:55.889 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:03:55 smithi097 ceph-mon[31859]: pgmap v659: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 53 MiB/s wr, 852 op/s 2024-03-27T11:03:57.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:03:57 smithi177 ceph-mon[35294]: pgmap v660: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 51 MiB/s wr, 817 op/s 2024-03-27T11:03:57.764 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:03:57 smithi114 ceph-mon[35338]: pgmap v660: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 51 MiB/s wr, 817 op/s 2024-03-27T11:03:57.889 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:03:57 smithi097 ceph-mon[31859]: pgmap v660: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 51 MiB/s wr, 817 op/s 2024-03-27T11:03:59.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:03:59 smithi177 ceph-mon[35294]: pgmap v661: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 53 MiB/s wr, 856 op/s 2024-03-27T11:03:59.764 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:03:59 smithi114 ceph-mon[35338]: pgmap v661: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 53 MiB/s wr, 856 op/s 2024-03-27T11:03:59.889 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:03:59 smithi097 ceph-mon[31859]: pgmap v661: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 53 MiB/s wr, 856 op/s 2024-03-27T11:04:01.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:04:01 smithi177 ceph-mon[35294]: pgmap v662: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 57 MiB/s wr, 915 op/s 2024-03-27T11:04:01.690 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:04:01 smithi097 ceph-mon[31859]: pgmap v662: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 57 MiB/s wr, 915 op/s 2024-03-27T11:04:01.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:04:01 smithi114 ceph-mon[35338]: pgmap v662: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 57 MiB/s wr, 915 op/s 2024-03-27T11:04:03.764 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:04:03 smithi114 ceph-mon[35338]: pgmap v663: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 51 MiB/s wr, 820 op/s 2024-03-27T11:04:03.889 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:04:03 smithi097 ceph-mon[31859]: pgmap v663: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 51 MiB/s wr, 820 op/s 2024-03-27T11:04:03.924 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:04:03 smithi177 ceph-mon[35294]: pgmap v663: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 51 MiB/s wr, 820 op/s 2024-03-27T11:04:04.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:04:04 smithi114 ceph-mon[35338]: pgmap v664: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 52 MiB/s wr, 836 op/s 2024-03-27T11:04:04.889 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:04:04 smithi097 ceph-mon[31859]: pgmap v664: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 52 MiB/s wr, 836 op/s 2024-03-27T11:04:04.924 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:04:04 smithi177 ceph-mon[35294]: pgmap v664: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 52 MiB/s wr, 836 op/s 2024-03-27T11:04:07.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:04:07 smithi097 ceph-mon[31859]: pgmap v665: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 50 MiB/s wr, 807 op/s 2024-03-27T11:04:07.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:04:07 smithi177 ceph-mon[35294]: pgmap v665: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 50 MiB/s wr, 807 op/s 2024-03-27T11:04:07.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:04:07 smithi114 ceph-mon[35338]: pgmap v665: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 50 MiB/s wr, 807 op/s 2024-03-27T11:04:08.060 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.089+0000 7f615e400640 1 -- 172.21.15.97:0/3794204127 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6158103db0 msgr2=0x7f61581041d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:04:08.060 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.089+0000 7f615e400640 1 --2- 172.21.15.97:0/3794204127 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6158103db0 0x7f61581041d0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f6148009900 tx=0x7f614802bc40 comp rx=0 tx=0).stop 2024-03-27T11:04:08.061 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.090+0000 7f615e400640 1 -- 172.21.15.97:0/3794204127 shutdown_connections 2024-03-27T11:04:08.061 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.090+0000 7f615e400640 1 --2- 172.21.15.97:0/3794204127 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6158105960 0x7f615810c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:08.061 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.090+0000 7f615e400640 1 --2- 172.21.15.97:0/3794204127 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6158104fa0 0x7f6158105420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:08.061 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.090+0000 7f615e400640 1 --2- 172.21.15.97:0/3794204127 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6158103db0 0x7f61581041d0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:08.061 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.090+0000 7f615e400640 1 -- 172.21.15.97:0/3794204127 >> 172.21.15.97:0/3794204127 conn(0x7f61580ff340 msgr2=0x7f6158101760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:04:08.062 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.091+0000 7f615e400640 1 -- 172.21.15.97:0/3794204127 shutdown_connections 2024-03-27T11:04:08.063 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.091+0000 7f615e400640 1 -- 172.21.15.97:0/3794204127 wait complete. 2024-03-27T11:04:08.063 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.093+0000 7f615e400640 1 Processor -- start 2024-03-27T11:04:08.064 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.093+0000 7f615e400640 1 -- start start 2024-03-27T11:04:08.065 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.094+0000 7f615e400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6158103db0 0x7f6158195b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:04:08.065 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.094+0000 7f615e400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6158104fa0 0x7f6158196090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:04:08.065 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.094+0000 7f615e400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6158105960 0x7f615819d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:04:08.065 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.094+0000 7f615e400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f615810f930 con 0x7f6158104fa0 2024-03-27T11:04:08.065 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.094+0000 7f615e400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f615810f7b0 con 0x7f6158105960 2024-03-27T11:04:08.065 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.094+0000 7f615e400640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f615810fab0 con 0x7f6158103db0 2024-03-27T11:04:08.065 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.094+0000 7f6157e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6158105960 0x7f615819d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:04:08.066 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.094+0000 7f6157400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6158103db0 0x7f6158195b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:04:08.066 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.094+0000 7f6156a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6158104fa0 0x7f6158196090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:04:08.066 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.094+0000 7f6157e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6158105960 0x7f615819d100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:46428/0 (socket says 172.21.15.97:46428) 2024-03-27T11:04:08.066 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.094+0000 7f6157e00640 1 -- 172.21.15.97:0/2838818422 learned_addr learned my addr 172.21.15.97:0/2838818422 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:04:08.066 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.095+0000 7f6157400640 1 -- 172.21.15.97:0/2838818422 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6158105960 msgr2=0x7f615819d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:04:08.067 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.095+0000 7f6157400640 1 --2- 172.21.15.97:0/2838818422 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6158105960 0x7f615819d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:08.067 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.095+0000 7f6157400640 1 -- 172.21.15.97:0/2838818422 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6158104fa0 msgr2=0x7f6158196090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:04:08.067 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.095+0000 7f6157400640 1 --2- 172.21.15.97:0/2838818422 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6158104fa0 0x7f6158196090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:08.067 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.095+0000 7f6157400640 1 -- 172.21.15.97:0/2838818422 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61581ae830 con 0x7f6158103db0 2024-03-27T11:04:08.067 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.095+0000 7f6157400640 1 --2- 172.21.15.97:0/2838818422 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6158103db0 0x7f6158195b50 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6148036040 tx=0x7f6148004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:04:08.068 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.096+0000 7f614fe00640 1 -- 172.21.15.97:0/2838818422 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6148043020 con 0x7f6158103db0 2024-03-27T11:04:08.068 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.096+0000 7f614fe00640 1 -- 172.21.15.97:0/2838818422 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f61480043c0 con 0x7f6158103db0 2024-03-27T11:04:08.068 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.096+0000 7f614fe00640 1 -- 172.21.15.97:0/2838818422 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6148003b10 con 0x7f6158103db0 2024-03-27T11:04:08.068 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.096+0000 7f615e400640 1 -- 172.21.15.97:0/2838818422 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61581aea60 con 0x7f6158103db0 2024-03-27T11:04:08.068 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.096+0000 7f615e400640 1 -- 172.21.15.97:0/2838818422 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61581aef30 con 0x7f6158103db0 2024-03-27T11:04:08.071 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.101+0000 7f614fe00640 1 -- 172.21.15.97:0/2838818422 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f6148046020 con 0x7f6158103db0 2024-03-27T11:04:08.074 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.103+0000 7f614fe00640 1 --2- 172.21.15.97:0/2838818422 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f613c0a3870 0x7f613c0a5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:04:08.075 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.103+0000 7f614fe00640 1 -- 172.21.15.97:0/2838818422 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f61480f1e80 con 0x7f6158103db0 2024-03-27T11:04:08.075 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.104+0000 7f6156a00640 1 --2- 172.21.15.97:0/2838818422 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f613c0a3870 0x7f613c0a5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:04:08.075 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.105+0000 7f6156a00640 1 --2- 172.21.15.97:0/2838818422 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f613c0a3870 0x7f613c0a5d30 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f6158197060 tx=0x7f6144005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:04:08.088 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.117+0000 7f616116a740 1 Processor -- start 2024-03-27T11:04:08.089 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.117+0000 7f616116a740 1 -- start start 2024-03-27T11:04:08.089 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.118+0000 7f616116a740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5623b7753980 0x5623b7753da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:04:08.089 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.118+0000 7f616116a740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5623b7754370 0x5623b7758e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:04:08.089 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.118+0000 7f616116a740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5623b77594e0 0x5623b775b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:04:08.089 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.118+0000 7f616116a740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5623b76b8d10 con 0x5623b7753980 2024-03-27T11:04:08.090 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.118+0000 7f616116a740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5623b7661260 con 0x5623b77594e0 2024-03-27T11:04:08.090 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.118+0000 7f616116a740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5623b76b4cc0 con 0x5623b7754370 2024-03-27T11:04:08.090 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.118+0000 7f6157400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5623b7753980 0x5623b7753da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:04:08.090 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.118+0000 7f6156a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5623b7754370 0x5623b7758e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:04:08.090 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.118+0000 7f6156a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5623b7754370 0x5623b7758e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.97:37110/0 (socket says 172.21.15.97:37110) 2024-03-27T11:04:08.090 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.118+0000 7f6156a00640 1 -- 172.21.15.97:0/2332337199 learned_addr learned my addr 172.21.15.97:0/2332337199 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:04:08.092 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.119+0000 7f6156a00640 1 -- 172.21.15.97:0/2332337199 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5623b77594e0 msgr2=0x5623b775b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:04:08.092 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.119+0000 7f6156a00640 1 --2- 172.21.15.97:0/2332337199 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5623b77594e0 0x5623b775b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:08.092 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.119+0000 7f6156a00640 1 -- 172.21.15.97:0/2332337199 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5623b7753980 msgr2=0x5623b7753da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:04:08.092 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.119+0000 7f6156a00640 1 --2- 172.21.15.97:0/2332337199 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5623b7753980 0x5623b7753da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:08.092 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.119+0000 7f6156a00640 1 -- 172.21.15.97:0/2332337199 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623b7658aa0 con 0x5623b7754370 2024-03-27T11:04:08.092 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.119+0000 7f6157400640 1 --2- 172.21.15.97:0/2332337199 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5623b7753980 0x5623b7753da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T11:04:08.092 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.119+0000 7f6156a00640 1 --2- 172.21.15.97:0/2332337199 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5623b7754370 0x5623b7758e70 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f614400e630 tx=0x7f6144014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=532cde7400cf0572 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:04:08.092 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.120+0000 7f614cc00640 1 -- 172.21.15.97:0/2332337199 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6144004020 con 0x5623b7754370 2024-03-27T11:04:08.093 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.120+0000 7f614cc00640 1 -- 172.21.15.97:0/2332337199 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f61440041c0 con 0x5623b7754370 2024-03-27T11:04:08.093 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.120+0000 7f614cc00640 1 -- 172.21.15.97:0/2332337199 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f614401c6f0 con 0x5623b7754370 2024-03-27T11:04:08.093 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.121+0000 7f616116a740 1 -- 172.21.15.97:0/2332337199 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5623b7754370 msgr2=0x5623b7758e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:04:08.093 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.121+0000 7f616116a740 1 --2- 172.21.15.97:0/2332337199 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5623b7754370 0x5623b7758e70 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f614400e630 tx=0x7f6144014000 comp rx=0 tx=0).stop 2024-03-27T11:04:08.094 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.121+0000 7f616116a740 1 -- 172.21.15.97:0/2332337199 shutdown_connections 2024-03-27T11:04:08.094 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.121+0000 7f616116a740 1 --2- 172.21.15.97:0/2332337199 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5623b77594e0 0x5623b775b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:08.094 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.121+0000 7f616116a740 1 --2- 172.21.15.97:0/2332337199 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5623b7754370 0x5623b7758e70 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:08.094 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.121+0000 7f616116a740 1 --2- 172.21.15.97:0/2332337199 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5623b7753980 0x5623b7753da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:08.095 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.121+0000 7f616116a740 1 -- 172.21.15.97:0/2332337199 >> 172.21.15.97:0/2332337199 conn(0x5623b774eef0 msgr2=0x5623b774f2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:04:08.095 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.122+0000 7f616116a740 1 -- 172.21.15.97:0/2332337199 shutdown_connections 2024-03-27T11:04:08.095 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.122+0000 7f616116a740 1 -- 172.21.15.97:0/2332337199 wait complete. 2024-03-27T11:04:08.095 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.122+0000 7f616116a740 1 -- start start 2024-03-27T11:04:08.095 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.124+0000 7f616116a740 1 Processor -- start 2024-03-27T11:04:08.096 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.125+0000 7f616116a740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5623b7753980 0x5623b775dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:04:08.096 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.125+0000 7f616116a740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5623b7754370 0x5623b775e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:04:08.096 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.125+0000 7f616116a740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5623b77594e0 0x5623b7765290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:04:08.097 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.125+0000 7f616116a740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5623b775c490 con 0x5623b7753980 2024-03-27T11:04:08.097 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.125+0000 7f616116a740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5623b7674630 con 0x5623b7754370 2024-03-27T11:04:08.097 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.125+0000 7f616116a740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5623b775c610 con 0x5623b77594e0 2024-03-27T11:04:08.097 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.125+0000 7f616116a740 10 client.0 _ll_register_callbacks cb 0x5623b7679c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T11:04:08.097 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.126+0000 7f6157400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5623b7753980 0x5623b775dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:04:08.098 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.126+0000 7f6157400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5623b7753980 0x5623b775dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58554/0 (socket says 172.21.15.97:58554) 2024-03-27T11:04:08.098 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.126+0000 7f6157400640 1 -- 172.21.15.97:0/395618882 learned_addr learned my addr 172.21.15.97:0/395618882 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:04:08.098 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.126+0000 7f6156a00640 1 --2- 172.21.15.97:0/395618882 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5623b7754370 0x5623b775e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:04:08.098 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.126+0000 7f6157e00640 1 --2- 172.21.15.97:0/395618882 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5623b77594e0 0x5623b7765290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:04:08.098 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.127+0000 7f6157400640 1 -- 172.21.15.97:0/395618882 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5623b77594e0 msgr2=0x5623b7765290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:04:08.098 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.127+0000 7f6157400640 1 --2- 172.21.15.97:0/395618882 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5623b77594e0 0x5623b7765290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:08.098 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.127+0000 7f6157400640 1 -- 172.21.15.97:0/395618882 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5623b7754370 msgr2=0x5623b775e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:04:08.098 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.127+0000 7f6157400640 1 --2- 172.21.15.97:0/395618882 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5623b7754370 0x5623b775e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:08.098 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.127+0000 7f6157400640 1 -- 172.21.15.97:0/395618882 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623b76805f0 con 0x5623b7753980 2024-03-27T11:04:08.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.129+0000 7f6157400640 1 --2- 172.21.15.97:0/395618882 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5623b7753980 0x5623b775dd10 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f61480baca0 tx=0x7f61480bdd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3127182f36e41dd server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:04:08.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.129+0000 7f6141600640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T11:04:08.102 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.131+0000 7f6141600640 1 -- 172.21.15.97:0/395618882 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61480c2710 con 0x5623b7753980 2024-03-27T11:04:08.102 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.131+0000 7f6141600640 1 -- 172.21.15.97:0/395618882 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f6148003cb0 con 0x5623b7753980 2024-03-27T11:04:08.102 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.131+0000 7f616116a740 1 -- 172.21.15.97:0/395618882 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61440041c0 con 0x5623b7753980 2024-03-27T11:04:08.102 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.131+0000 7f6141600640 1 -- 172.21.15.97:0/395618882 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f61480c6fd0 con 0x5623b7753980 2024-03-27T11:04:08.104 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.132+0000 7f616116a740 10 client.16713 fetch_fsmap learned FSMap version 34 2024-03-27T11:04:08.104 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.132+0000 7f616116a740 1 -- 172.21.15.97:0/395618882 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5623b73cd210 con 0x5623b7753980 2024-03-27T11:04:08.104 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.132+0000 7f6141600640 1 -- 172.21.15.97:0/395618882 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f61440041c0 con 0x5623b7753980 2024-03-27T11:04:08.104 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.132+0000 7f6141600640 1 -- 172.21.15.97:0/395618882 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f61480ba020 con 0x5623b7753980 2024-03-27T11:04:08.104 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.133+0000 7f616116a740 10 client.16713 fetch_fsmap finished waiting for FSMap version 34 2024-03-27T11:04:08.104 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.133+0000 7f616116a740 10 client.16713 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T11:04:08.104 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.133+0000 7f616116a740 20 client.16713 populate_metadata read hostname 'smithi097' 2024-03-27T11:04:08.104 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.133+0000 7f616116a740 1 --2- 172.21.15.97:0/395618882 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5623b7769af0 0x5623b776bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:04:08.104 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.133+0000 7f616116a740 4 client.16713 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T11:04:08.104 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.133+0000 7f616116a740 1 -- 172.21.15.97:0/395618882 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5623b76c6250 con 0x5623b7769af0 2024-03-27T11:04:08.104 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.133+0000 7f6156a00640 1 --2- 172.21.15.97:0/395618882 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5623b7769af0 0x5623b776bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:04:08.105 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.134+0000 7f6156a00640 1 --2- 172.21.15.97:0/395618882 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5623b7769af0 0x5623b776bf20 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1788d17dd0f88b4 server_cookie=42e4afa4dbfd3e43 in_seq=0 out_seq=0 2024-03-27T11:04:08.105 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.134+0000 7f6141600640 10 client.16713 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T11:04:08.106 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.135+0000 7f6141600640 1 -- 172.21.15.97:0/395618882 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5623b76c6250 con 0x5623b7769af0 2024-03-27T11:04:08.106 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.135+0000 7f6141600640 10 client.16713 handle_command_reply: tid=0 2024-03-27T11:04:08.106 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.136+0000 7f616116a740 1 client.16713 shutdown 2024-03-27T11:04:08.107 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.136+0000 7f616116a740 1 -- 172.21.15.97:0/395618882 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5623b7753980 msgr2=0x5623b775dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:04:08.107 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.136+0000 7f616116a740 1 --2- 172.21.15.97:0/395618882 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5623b7753980 0x5623b775dd10 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f61480baca0 tx=0x7f61480bdd40 comp rx=0 tx=0).stop 2024-03-27T11:04:08.107 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.136+0000 7f616116a740 1 -- 172.21.15.97:0/395618882 shutdown_connections 2024-03-27T11:04:08.107 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.136+0000 7f616116a740 1 --2- 172.21.15.97:0/395618882 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5623b7769af0 0x5623b776bf20 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:08.107 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.136+0000 7f616116a740 1 --2- 172.21.15.97:0/395618882 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5623b77594e0 0x5623b7765290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:08.107 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.136+0000 7f616116a740 1 --2- 172.21.15.97:0/395618882 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5623b7754370 0x5623b775e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:08.107 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.136+0000 7f616116a740 1 --2- 172.21.15.97:0/395618882 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5623b7753980 0x5623b775dd10 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:08.107 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.136+0000 7f616116a740 1 -- 172.21.15.97:0/395618882 >> 172.21.15.97:0/395618882 conn(0x5623b774eef0 msgr2=0x5623b774e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:04:08.107 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.136+0000 7f6141600640 0 client.16713 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T11:04:08.108 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.137+0000 7f616116a740 1 -- 172.21.15.97:0/395618882 shutdown_connections 2024-03-27T11:04:08.108 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.137+0000 7f616116a740 1 -- 172.21.15.97:0/395618882 wait complete. 2024-03-27T11:04:08.108 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.137+0000 7f616116a740 20 client.16713 trim_cache size 0 max 16384 2024-03-27T11:04:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.162+0000 7f616116a740 1 Processor -- start 2024-03-27T11:04:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.162+0000 7f616116a740 1 -- start start 2024-03-27T11:04:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.162+0000 7f616116a740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5623b7753980 0x5623b776bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:04:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.162+0000 7f616116a740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5623b7754370 0x5623b7765250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:04:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.162+0000 7f616116a740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5623b77594e0 0x5623b775e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:04:08.134 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.162+0000 7f616116a740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f61581b7c60 con 0x5623b7754370 2024-03-27T11:04:08.134 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.162+0000 7f616116a740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f61581b97a0 con 0x5623b7753980 2024-03-27T11:04:08.134 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.162+0000 7f616116a740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f61581b8330 con 0x5623b77594e0 2024-03-27T11:04:08.135 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.163+0000 7f6157400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5623b7753980 0x5623b776bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:04:08.135 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.163+0000 7f6157400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5623b7753980 0x5623b776bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:46458/0 (socket says 172.21.15.97:46458) 2024-03-27T11:04:08.135 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.163+0000 7f6157400640 1 -- 172.21.15.97:0/2194776019 learned_addr learned my addr 172.21.15.97:0/2194776019 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:04:08.135 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.163+0000 7f6157e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5623b77594e0 0x5623b775e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:04:08.135 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.163+0000 7f6156a00640 1 --2- 172.21.15.97:0/2194776019 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5623b7754370 0x5623b7765250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:04:08.135 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.164+0000 7f6157e00640 1 -- 172.21.15.97:0/2194776019 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5623b7753980 msgr2=0x5623b776bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:04:08.135 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.164+0000 7f6157e00640 1 --2- 172.21.15.97:0/2194776019 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5623b7753980 0x5623b776bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:08.135 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.164+0000 7f6157e00640 1 -- 172.21.15.97:0/2194776019 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5623b7754370 msgr2=0x5623b7765250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:04:08.135 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.164+0000 7f6157e00640 1 --2- 172.21.15.97:0/2194776019 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5623b7754370 0x5623b7765250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:08.135 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.164+0000 7f6157e00640 1 -- 172.21.15.97:0/2194776019 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623b7499620 con 0x5623b77594e0 2024-03-27T11:04:08.135 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.164+0000 7f6157e00640 1 --2- 172.21.15.97:0/2194776019 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5623b77594e0 0x5623b775e030 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x5623b774fe60 tx=0x7f61500129e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=abac2b53b9f2fdeb server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:04:08.135 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.164+0000 7f6156a00640 1 --2- 172.21.15.97:0/2194776019 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5623b7754370 0x5623b7765250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T11:04:08.136 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.165+0000 7f6143e00640 1 -- 172.21.15.97:0/2194776019 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61500120d0 con 0x5623b77594e0 2024-03-27T11:04:08.136 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.165+0000 7f6143e00640 1 -- 172.21.15.97:0/2194776019 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6150012270 con 0x5623b77594e0 2024-03-27T11:04:08.136 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.165+0000 7f6143e00640 1 -- 172.21.15.97:0/2194776019 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f615001c4d0 con 0x5623b77594e0 2024-03-27T11:04:08.137 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.166+0000 7f616116a740 1 -- 172.21.15.97:0/2194776019 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5623b77594e0 msgr2=0x5623b775e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:04:08.137 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.166+0000 7f616116a740 1 --2- 172.21.15.97:0/2194776019 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5623b77594e0 0x5623b775e030 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x5623b774fe60 tx=0x7f61500129e0 comp rx=0 tx=0).stop 2024-03-27T11:04:08.137 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.166+0000 7f616116a740 1 -- 172.21.15.97:0/2194776019 shutdown_connections 2024-03-27T11:04:08.137 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.166+0000 7f616116a740 1 --2- 172.21.15.97:0/2194776019 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5623b77594e0 0x5623b775e030 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:08.137 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.166+0000 7f616116a740 1 --2- 172.21.15.97:0/2194776019 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5623b7754370 0x5623b7765250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:08.137 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.166+0000 7f616116a740 1 --2- 172.21.15.97:0/2194776019 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5623b7753980 0x5623b776bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:08.137 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.166+0000 7f616116a740 1 -- 172.21.15.97:0/2194776019 >> 172.21.15.97:0/2194776019 conn(0x5623b774eef0 msgr2=0x5623b775d920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:04:08.138 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.166+0000 7f616116a740 1 -- 172.21.15.97:0/2194776019 shutdown_connections 2024-03-27T11:04:08.138 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.166+0000 7f616116a740 1 -- 172.21.15.97:0/2194776019 wait complete. 2024-03-27T11:04:08.138 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.166+0000 7f616116a740 1 -- start start 2024-03-27T11:04:08.139 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.167+0000 7f616116a740 1 Processor -- start 2024-03-27T11:04:08.139 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.167+0000 7f616116a740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5623b7753980 0x5623b77608f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:04:08.139 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.167+0000 7f616116a740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5623b7754370 0x5623b7760e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:04:08.139 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.167+0000 7f616116a740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5623b77594e0 0x5623b7763e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:04:08.139 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.167+0000 7f616116a740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f61581b90d0 con 0x5623b77594e0 2024-03-27T11:04:08.139 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.167+0000 7f616116a740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f61581b8a00 con 0x5623b7753980 2024-03-27T11:04:08.139 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.167+0000 7f616116a740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f61581b5c40 con 0x5623b7754370 2024-03-27T11:04:08.139 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.167+0000 7f616116a740 10 client.0 _ll_register_callbacks cb 0x7f61480ba660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T11:04:08.139 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.167+0000 7f6157400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5623b7753980 0x5623b77608f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:04:08.139 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.167+0000 7f6157400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5623b7753980 0x5623b77608f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:46464/0 (socket says 172.21.15.97:46464) 2024-03-27T11:04:08.139 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.167+0000 7f6157400640 1 -- 172.21.15.97:0/867482157 learned_addr learned my addr 172.21.15.97:0/867482157 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:04:08.139 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.168+0000 7f6157e00640 1 --2- 172.21.15.97:0/867482157 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5623b77594e0 0x5623b7763e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:04:08.139 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.168+0000 7f6156a00640 1 --2- 172.21.15.97:0/867482157 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5623b7754370 0x5623b7760e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:04:08.140 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.168+0000 7f6157400640 1 -- 172.21.15.97:0/867482157 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5623b7754370 msgr2=0x5623b7760e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:04:08.140 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.168+0000 7f6157400640 1 --2- 172.21.15.97:0/867482157 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5623b7754370 0x5623b7760e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:08.140 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.168+0000 7f6157400640 1 -- 172.21.15.97:0/867482157 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5623b77594e0 msgr2=0x5623b7763e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:04:08.140 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.168+0000 7f6157400640 1 --2- 172.21.15.97:0/867482157 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5623b77594e0 0x5623b7763e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:08.140 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.168+0000 7f6157400640 1 -- 172.21.15.97:0/867482157 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623b7764440 con 0x5623b7753980 2024-03-27T11:04:08.141 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.168+0000 7f6157400640 1 --2- 172.21.15.97:0/867482157 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5623b7753980 0x5623b77608f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f614802c850 tx=0x7f61480be240 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a42515ce6ce36d1b server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:04:08.141 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.168+0000 7f6141600640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-03-27T11:04:08.141 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.169+0000 7f6141600640 1 -- 172.21.15.97:0/867482157 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61480bdea0 con 0x5623b7753980 2024-03-27T11:04:08.141 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.169+0000 7f6141600640 1 -- 172.21.15.97:0/867482157 <== mon.2 v2:172.21.15.114:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f61480c6b40 con 0x5623b7753980 2024-03-27T11:04:08.141 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.169+0000 7f616116a740 1 -- 172.21.15.97:0/867482157 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6150012270 con 0x5623b7753980 2024-03-27T11:04:08.141 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.169+0000 7f6141600640 1 -- 172.21.15.97:0/867482157 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f61480c2710 con 0x5623b7753980 2024-03-27T11:04:08.141 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.169+0000 7f616116a740 10 client.36383 fetch_fsmap learned FSMap version 34 2024-03-27T11:04:08.142 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.169+0000 7f616116a740 1 -- 172.21.15.97:0/867482157 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5623b77e0e10 con 0x5623b7753980 2024-03-27T11:04:08.142 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.169+0000 7f6141600640 1 -- 172.21.15.97:0/867482157 <== mon.2 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6150012270 con 0x5623b7753980 2024-03-27T11:04:08.142 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.170+0000 7f6141600640 1 -- 172.21.15.97:0/867482157 <== mon.2 v2:172.21.15.114:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f61480c38a0 con 0x5623b7753980 2024-03-27T11:04:08.142 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.170+0000 7f616116a740 10 client.36383 fetch_fsmap finished waiting for FSMap version 34 2024-03-27T11:04:08.142 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.170+0000 7f616116a740 10 client.36383 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T11:04:08.142 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.170+0000 7f616116a740 20 client.36383 populate_metadata read hostname 'smithi097' 2024-03-27T11:04:08.142 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.170+0000 7f616116a740 1 --2- 172.21.15.97:0/867482157 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5623b77e1540 0x5623b77e3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:04:08.142 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.170+0000 7f616116a740 4 client.36383 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-27T11:04:08.142 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.170+0000 7f616116a740 1 -- 172.21.15.97:0/867482157 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5623b76c05a0 con 0x5623b77e1540 2024-03-27T11:04:08.142 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.170+0000 7f6156a00640 1 --2- 172.21.15.97:0/867482157 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5623b77e1540 0x5623b77e3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:04:08.143 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.171+0000 7f6156a00640 1 --2- 172.21.15.97:0/867482157 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5623b77e1540 0x5623b77e3960 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ac4021309babcc9 server_cookie=4c6dc3254297acb0 in_seq=0 out_seq=0 2024-03-27T11:04:08.143 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.171+0000 7f6141600640 10 client.36383 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T11:04:08.149 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.178+0000 7f6141600640 1 -- 172.21.15.97:0/867482157 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5623b76c05a0 con 0x5623b77e1540 2024-03-27T11:04:08.149 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.178+0000 7f6141600640 10 client.36383 handle_command_reply: tid=0 2024-03-27T11:04:08.149 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.178+0000 7f616116a740 1 client.36383 shutdown 2024-03-27T11:04:08.149 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.178+0000 7f616116a740 1 -- 172.21.15.97:0/867482157 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5623b7753980 msgr2=0x5623b77608f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:04:08.149 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.178+0000 7f616116a740 1 --2- 172.21.15.97:0/867482157 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5623b7753980 0x5623b77608f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f614802c850 tx=0x7f61480be240 comp rx=0 tx=0).stop 2024-03-27T11:04:08.150 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.179+0000 7f616116a740 1 -- 172.21.15.97:0/867482157 shutdown_connections 2024-03-27T11:04:08.150 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.179+0000 7f616116a740 1 --2- 172.21.15.97:0/867482157 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5623b77e1540 0x5623b77e3960 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:08.150 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.179+0000 7f616116a740 1 --2- 172.21.15.97:0/867482157 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5623b77594e0 0x5623b7763e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:08.150 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.179+0000 7f616116a740 1 --2- 172.21.15.97:0/867482157 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5623b7754370 0x5623b7760e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:08.150 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.179+0000 7f616116a740 1 --2- 172.21.15.97:0/867482157 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5623b7753980 0x5623b77608f0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:08.150 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.179+0000 7f616116a740 1 -- 172.21.15.97:0/867482157 >> 172.21.15.97:0/867482157 conn(0x5623b774eef0 msgr2=0x5623b775e120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:04:08.151 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.179+0000 7f6141600640 0 client.36383 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T11:04:08.151 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.180+0000 7f616116a740 1 -- 172.21.15.97:0/867482157 shutdown_connections 2024-03-27T11:04:08.151 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.180+0000 7f616116a740 1 -- 172.21.15.97:0/867482157 wait complete. 2024-03-27T11:04:08.151 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.180+0000 7f616116a740 20 client.36383 trim_cache size 0 max 16384 2024-03-27T11:04:08.151 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-27T11:04:08.152 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-27T11:04:08.152 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-27T11:04:08.152 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-27T11:04:08.153 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.182+0000 7f615e400640 1 -- 172.21.15.97:0/2838818422 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f613c0a3870 msgr2=0x7f613c0a5d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:04:08.153 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.182+0000 7f615e400640 1 --2- 172.21.15.97:0/2838818422 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f613c0a3870 0x7f613c0a5d30 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f6158197060 tx=0x7f6144005f90 comp rx=0 tx=0).stop 2024-03-27T11:04:08.153 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.182+0000 7f615e400640 1 -- 172.21.15.97:0/2838818422 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6158103db0 msgr2=0x7f6158195b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:04:08.153 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.182+0000 7f615e400640 1 --2- 172.21.15.97:0/2838818422 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6158103db0 0x7f6158195b50 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6148036040 tx=0x7f6148004040 comp rx=0 tx=0).stop 2024-03-27T11:04:08.153 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.182+0000 7f615e400640 1 -- 172.21.15.97:0/2838818422 shutdown_connections 2024-03-27T11:04:08.154 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.182+0000 7f615e400640 1 --2- 172.21.15.97:0/2838818422 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f613c0a3870 0x7f613c0a5d30 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:08.154 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.182+0000 7f615e400640 1 --2- 172.21.15.97:0/2838818422 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6158105960 0x7f615819d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:08.154 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.182+0000 7f615e400640 1 --2- 172.21.15.97:0/2838818422 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6158104fa0 0x7f6158196090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:08.154 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.182+0000 7f615e400640 1 --2- 172.21.15.97:0/2838818422 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6158103db0 0x7f6158195b50 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:08.154 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.182+0000 7f615e400640 1 -- 172.21.15.97:0/2838818422 >> 172.21.15.97:0/2838818422 conn(0x7f61580ff340 msgr2=0x7f6158078010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:04:08.154 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.183+0000 7f615e400640 1 -- 172.21.15.97:0/2838818422 shutdown_connections 2024-03-27T11:04:08.155 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:08.183+0000 7f615e400640 1 -- 172.21.15.97:0/2838818422 wait complete. 2024-03-27T11:04:09.567 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:04:09 smithi114 ceph-mon[35338]: pgmap v666: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 53 MiB/s wr, 850 op/s 2024-03-27T11:04:09.653 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:04:09 smithi097 ceph-mon[31859]: pgmap v666: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 53 MiB/s wr, 850 op/s 2024-03-27T11:04:09.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:04:09 smithi177 ceph-mon[35294]: pgmap v666: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 53 MiB/s wr, 850 op/s 2024-03-27T11:04:11.657 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:04:11 smithi097 ceph-mon[31859]: pgmap v667: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 56 MiB/s wr, 911 op/s 2024-03-27T11:04:11.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:04:11 smithi177 ceph-mon[35294]: pgmap v667: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 56 MiB/s wr, 911 op/s 2024-03-27T11:04:11.764 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:04:11 smithi114 ceph-mon[35338]: pgmap v667: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 56 MiB/s wr, 911 op/s 2024-03-27T11:04:13.658 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:04:13 smithi097 ceph-mon[31859]: pgmap v668: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 50 MiB/s wr, 815 op/s 2024-03-27T11:04:13.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:04:13 smithi177 ceph-mon[35294]: pgmap v668: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 50 MiB/s wr, 815 op/s 2024-03-27T11:04:13.764 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:04:13 smithi114 ceph-mon[35338]: pgmap v668: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 50 MiB/s wr, 815 op/s 2024-03-27T11:04:15.638 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:04:15 smithi097 ceph-mon[31859]: pgmap v669: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 51 MiB/s wr, 830 op/s 2024-03-27T11:04:15.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:04:15 smithi177 ceph-mon[35294]: pgmap v669: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 51 MiB/s wr, 830 op/s 2024-03-27T11:04:15.764 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:04:15 smithi114 ceph-mon[35338]: pgmap v669: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 51 MiB/s wr, 830 op/s 2024-03-27T11:04:17.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:04:17 smithi097 ceph-mon[31859]: pgmap v670: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 49 MiB/s wr, 796 op/s 2024-03-27T11:04:17.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:04:17 smithi177 ceph-mon[35294]: pgmap v670: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 49 MiB/s wr, 796 op/s 2024-03-27T11:04:17.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:04:17 smithi114 ceph-mon[35338]: pgmap v670: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 49 MiB/s wr, 796 op/s 2024-03-27T11:04:19.638 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:04:19 smithi097 ceph-mon[31859]: pgmap v671: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 52 MiB/s wr, 841 op/s 2024-03-27T11:04:19.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:04:19 smithi177 ceph-mon[35294]: pgmap v671: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 52 MiB/s wr, 841 op/s 2024-03-27T11:04:19.764 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:04:19 smithi114 ceph-mon[35338]: pgmap v671: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 52 MiB/s wr, 841 op/s 2024-03-27T11:04:21.477 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.507+0000 7fd1e7e00640 1 -- 172.21.15.97:0/1096968289 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd1e0094650 msgr2=0x7fd1e0094a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:04:21.477 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.507+0000 7fd1e7e00640 1 --2- 172.21.15.97:0/1096968289 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd1e0094650 0x7fd1e0094a70 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd1d8009900 tx=0x7fd1d802bc50 comp rx=0 tx=0).stop 2024-03-27T11:04:21.478 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.508+0000 7fd1e7e00640 1 -- 172.21.15.97:0/1096968289 shutdown_connections 2024-03-27T11:04:21.478 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.508+0000 7fd1e7e00640 1 --2- 172.21.15.97:0/1096968289 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd1e0095950 0x7fd1e0095dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:21.478 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.508+0000 7fd1e7e00640 1 --2- 172.21.15.97:0/1096968289 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1e0094fb0 0x7fd1e0095410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:21.478 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.508+0000 7fd1e7e00640 1 --2- 172.21.15.97:0/1096968289 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd1e0094650 0x7fd1e0094a70 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:21.478 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.508+0000 7fd1e7e00640 1 -- 172.21.15.97:0/1096968289 >> 172.21.15.97:0/1096968289 conn(0x7fd1e00901d0 msgr2=0x7fd1e0092630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:04:21.478 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.508+0000 7fd1e7e00640 1 -- 172.21.15.97:0/1096968289 shutdown_connections 2024-03-27T11:04:21.478 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.508+0000 7fd1e7e00640 1 -- 172.21.15.97:0/1096968289 wait complete. 2024-03-27T11:04:21.547 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.578+0000 7fd1e7e00640 1 Processor -- start 2024-03-27T11:04:21.548 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.579+0000 7fd1e7e00640 1 -- start start 2024-03-27T11:04:21.550 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.580+0000 7fd1e7e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1e0094650 0x7fd1e009a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:04:21.550 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.580+0000 7fd1e7e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd1e0094fb0 0x7fd1e009a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:04:21.550 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.580+0000 7fd1e7e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd1e0095950 0x7fd1e0096d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:04:21.550 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.580+0000 7fd1e7e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd1e00a3b30 con 0x7fd1e0094650 2024-03-27T11:04:21.550 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.580+0000 7fd1e7e00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd1e00a39b0 con 0x7fd1e0095950 2024-03-27T11:04:21.550 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.580+0000 7fd1e7e00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd1e00a3cb0 con 0x7fd1e0094fb0 2024-03-27T11:04:21.551 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.581+0000 7fd1e7400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd1e0095950 0x7fd1e0096d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:04:21.551 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.581+0000 7fd1e6000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd1e0094fb0 0x7fd1e009a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:04:21.552 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.581+0000 7fd1e7400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd1e0095950 0x7fd1e0096d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:55592/0 (socket says 172.21.15.97:55592) 2024-03-27T11:04:21.552 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.581+0000 7fd1e7400640 1 -- 172.21.15.97:0/4110170736 learned_addr learned my addr 172.21.15.97:0/4110170736 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:04:21.553 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.582+0000 7fd1e7400640 1 -- 172.21.15.97:0/4110170736 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd1e0094fb0 msgr2=0x7fd1e009a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:04:21.553 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.582+0000 7fd1e7400640 1 --2- 172.21.15.97:0/4110170736 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd1e0094fb0 0x7fd1e009a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:21.553 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.582+0000 7fd1e7400640 1 -- 172.21.15.97:0/4110170736 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1e0094650 msgr2=0x7fd1e009a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:04:21.553 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.582+0000 7fd1e7400640 1 --2- 172.21.15.97:0/4110170736 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1e0094650 0x7fd1e009a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:21.554 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.582+0000 7fd1e7400640 1 -- 172.21.15.97:0/4110170736 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1e0097600 con 0x7fd1e0095950 2024-03-27T11:04:21.554 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.582+0000 7fd1e7400640 1 --2- 172.21.15.97:0/4110170736 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd1e0095950 0x7fd1e0096d40 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd1e804fb80 tx=0x7fd1e806aa50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:04:21.554 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.583+0000 7fd1df400640 1 -- 172.21.15.97:0/4110170736 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1e8050100 con 0x7fd1e0095950 2024-03-27T11:04:21.554 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.583+0000 7fd1df400640 1 -- 172.21.15.97:0/4110170736 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd1e807f420 con 0x7fd1e0095950 2024-03-27T11:04:21.554 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.583+0000 7fd1df400640 1 -- 172.21.15.97:0/4110170736 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1e807e380 con 0x7fd1e0095950 2024-03-27T11:04:21.554 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.583+0000 7fd1e7e00640 1 -- 172.21.15.97:0/4110170736 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1e013a560 con 0x7fd1e0095950 2024-03-27T11:04:21.554 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.583+0000 7fd1e7e00640 1 -- 172.21.15.97:0/4110170736 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1e013a910 con 0x7fd1e0095950 2024-03-27T11:04:21.558 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.589+0000 7fd1f0791740 1 Processor -- start 2024-03-27T11:04:21.560 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.591+0000 7fd1df400640 1 -- 172.21.15.97:0/4110170736 <== mon.2 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7fd1e8086030 con 0x7fd1e0095950 2024-03-27T11:04:21.562 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.591+0000 7fd1f0791740 1 -- start start 2024-03-27T11:04:21.562 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.591+0000 7fd1f0791740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55afd3ab0b60 0x55afd3ab0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:04:21.562 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.591+0000 7fd1f0791740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55afd3ab1550 0x55afd3ab6050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:04:21.562 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.591+0000 7fd1f0791740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55afd3ab66c0 0x55afd3ab8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:04:21.562 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.591+0000 7fd1f0791740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55afd3a15d10 con 0x55afd3ab66c0 2024-03-27T11:04:21.563 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.591+0000 7fd1f0791740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55afd39be260 con 0x55afd3ab1550 2024-03-27T11:04:21.563 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.591+0000 7fd1f0791740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55afd3a11cc0 con 0x55afd3ab0b60 2024-03-27T11:04:21.563 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.592+0000 7fd1e6a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55afd3ab1550 0x55afd3ab6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:04:21.563 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.592+0000 7fd1e6a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55afd3ab1550 0x55afd3ab6050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:55604/0 (socket says 172.21.15.97:55604) 2024-03-27T11:04:21.563 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.592+0000 7fd1e6a00640 1 -- 172.21.15.97:0/566483747 learned_addr learned my addr 172.21.15.97:0/566483747 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:04:21.563 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.593+0000 7fd1e6a00640 1 -- 172.21.15.97:0/566483747 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55afd3ab0b60 msgr2=0x55afd3ab0f80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-27T11:04:21.564 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.593+0000 7fd1e6a00640 1 --2- 172.21.15.97:0/566483747 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55afd3ab0b60 0x55afd3ab0f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:21.564 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.593+0000 7fd1e6a00640 1 -- 172.21.15.97:0/566483747 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55afd3ab66c0 msgr2=0x55afd3ab8b50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-27T11:04:21.564 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.593+0000 7fd1df400640 1 --2- 172.21.15.97:0/4110170736 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fd1cc0a3760 0x7fd1cc0a5c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:04:21.564 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.593+0000 7fd1df400640 1 -- 172.21.15.97:0/4110170736 <== mon.2 v2:172.21.15.114:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fd1e813ac80 con 0x7fd1e0095950 2024-03-27T11:04:21.565 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.594+0000 7fd1e6a00640 1 --2- 172.21.15.97:0/566483747 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55afd3ab66c0 0x55afd3ab8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:21.565 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.594+0000 7fd1e6a00640 1 -- 172.21.15.97:0/566483747 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55afd39b5aa0 con 0x55afd3ab1550 2024-03-27T11:04:21.565 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.594+0000 7fd1e6000640 1 --2- 172.21.15.97:0/4110170736 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fd1cc0a3760 0x7fd1cc0a5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:04:21.566 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.596+0000 7fd1e6a00640 1 --2- 172.21.15.97:0/566483747 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55afd3ab1550 0x55afd3ab6050 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd1d8036040 tx=0x7fd1d8000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf6f04ae257f777f server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:04:21.567 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.597+0000 7fd1d7e00640 1 -- 172.21.15.97:0/566483747 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d8041020 con 0x55afd3ab1550 2024-03-27T11:04:21.567 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.598+0000 7fd1d7e00640 1 -- 172.21.15.97:0/566483747 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd1d802cb60 con 0x55afd3ab1550 2024-03-27T11:04:21.567 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.598+0000 7fd1d7e00640 1 -- 172.21.15.97:0/566483747 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d8004920 con 0x55afd3ab1550 2024-03-27T11:04:21.568 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.599+0000 7fd1f0791740 1 -- 172.21.15.97:0/566483747 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55afd3ab1550 msgr2=0x55afd3ab6050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:04:21.568 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.599+0000 7fd1f0791740 1 --2- 172.21.15.97:0/566483747 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55afd3ab1550 0x55afd3ab6050 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd1d8036040 tx=0x7fd1d8000bc0 comp rx=0 tx=0).stop 2024-03-27T11:04:21.569 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.599+0000 7fd1f0791740 1 -- 172.21.15.97:0/566483747 shutdown_connections 2024-03-27T11:04:21.569 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.599+0000 7fd1f0791740 1 --2- 172.21.15.97:0/566483747 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55afd3ab66c0 0x55afd3ab8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:21.569 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.599+0000 7fd1f0791740 1 --2- 172.21.15.97:0/566483747 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55afd3ab1550 0x55afd3ab6050 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:21.569 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.599+0000 7fd1f0791740 1 --2- 172.21.15.97:0/566483747 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55afd3ab0b60 0x55afd3ab0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:21.569 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.599+0000 7fd1f0791740 1 -- 172.21.15.97:0/566483747 >> 172.21.15.97:0/566483747 conn(0x55afd3aac0d0 msgr2=0x55afd3aac4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:04:21.569 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.599+0000 7fd1e6000640 1 --2- 172.21.15.97:0/4110170736 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fd1cc0a3760 0x7fd1cc0a5c20 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x55afd3ab1db0 tx=0x7fd1d000a860 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:04:21.569 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.599+0000 7fd1f0791740 1 -- 172.21.15.97:0/566483747 shutdown_connections 2024-03-27T11:04:21.569 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.600+0000 7fd1f0791740 1 -- 172.21.15.97:0/566483747 wait complete. 2024-03-27T11:04:21.570 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.600+0000 7fd1f0791740 1 -- start start 2024-03-27T11:04:21.571 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.602+0000 7fd1f0791740 1 Processor -- start 2024-03-27T11:04:21.573 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.603+0000 7fd1f0791740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55afd3ab0b60 0x55afd3acbf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:04:21.573 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.603+0000 7fd1f0791740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55afd3ab1550 0x55afd3acc450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:04:21.573 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.603+0000 7fd1f0791740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55afd3ab66c0 0x55afd3ad3420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:04:21.573 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.603+0000 7fd1f0791740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55afd3aca620 con 0x55afd3ab66c0 2024-03-27T11:04:21.573 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.603+0000 7fd1f0791740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55afd39d1630 con 0x55afd3ab0b60 2024-03-27T11:04:21.573 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.603+0000 7fd1f0791740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55afd3aca7a0 con 0x55afd3ab1550 2024-03-27T11:04:21.573 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.603+0000 7fd1f0791740 10 client.0 _ll_register_callbacks cb 0x55afd39d6c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T11:04:21.574 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.604+0000 7fd1e7400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55afd3ab66c0 0x55afd3ad3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:04:21.574 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.605+0000 7fd1e7400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55afd3ab66c0 0x55afd3ad3420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52020/0 (socket says 172.21.15.97:52020) 2024-03-27T11:04:21.574 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.605+0000 7fd1e7400640 1 -- 172.21.15.97:0/1318837290 learned_addr learned my addr 172.21.15.97:0/1318837290 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:04:21.574 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.604+0000 7fd1e6a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55afd3ab0b60 0x55afd3acbf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:04:21.575 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.605+0000 7fd1e6000640 1 --2- 172.21.15.97:0/1318837290 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55afd3ab1550 0x55afd3acc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:04:21.575 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.606+0000 7fd1e6a00640 1 -- 172.21.15.97:0/1318837290 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55afd3ab1550 msgr2=0x55afd3acc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:04:21.575 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.606+0000 7fd1e6a00640 1 --2- 172.21.15.97:0/1318837290 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55afd3ab1550 0x55afd3acc450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:21.575 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.606+0000 7fd1e6a00640 1 -- 172.21.15.97:0/1318837290 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55afd3ab66c0 msgr2=0x55afd3ad3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:04:21.576 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.606+0000 7fd1e6a00640 1 --2- 172.21.15.97:0/1318837290 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55afd3ab66c0 0x55afd3ad3420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:21.576 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.606+0000 7fd1e6a00640 1 -- 172.21.15.97:0/1318837290 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55afd39dd5f0 con 0x55afd3ab0b60 2024-03-27T11:04:21.576 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.606+0000 7fd1e7400640 1 --2- 172.21.15.97:0/1318837290 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55afd3ab66c0 0x55afd3ad3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T11:04:21.576 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.607+0000 7fd1d4c00640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-03-27T11:04:21.576 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.607+0000 7fd1e6a00640 1 --2- 172.21.15.97:0/1318837290 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55afd3ab0b60 0x55afd3acbf10 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x55afd3aad040 tx=0x7fd1d8004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24d535e2c035240d server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:04:21.577 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.607+0000 7fd1d4c00640 1 -- 172.21.15.97:0/1318837290 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d80046c0 con 0x55afd3ab0b60 2024-03-27T11:04:21.577 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.607+0000 7fd1d4c00640 1 -- 172.21.15.97:0/1318837290 <== mon.2 v2:172.21.15.114:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fd1d8004860 con 0x55afd3ab0b60 2024-03-27T11:04:21.577 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.608+0000 7fd1f0791740 1 -- 172.21.15.97:0/1318837290 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1d802cb60 con 0x55afd3ab0b60 2024-03-27T11:04:21.578 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.608+0000 7fd1d4c00640 1 -- 172.21.15.97:0/1318837290 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fd1d8004a60 con 0x55afd3ab0b60 2024-03-27T11:04:21.578 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.608+0000 7fd1f0791740 10 client.36392 fetch_fsmap learned FSMap version 34 2024-03-27T11:04:21.578 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.608+0000 7fd1f0791740 1 -- 172.21.15.97:0/1318837290 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55afd372a210 con 0x55afd3ab0b60 2024-03-27T11:04:21.578 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.609+0000 7fd1d4c00640 1 -- 172.21.15.97:0/1318837290 <== mon.2 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd1d802cb60 con 0x55afd3ab0b60 2024-03-27T11:04:21.579 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.609+0000 7fd1d4c00640 1 -- 172.21.15.97:0/1318837290 <== mon.2 v2:172.21.15.114:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7fd1d803d440 con 0x55afd3ab0b60 2024-03-27T11:04:21.579 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.609+0000 7fd1f0791740 10 client.36392 fetch_fsmap finished waiting for FSMap version 34 2024-03-27T11:04:21.579 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.609+0000 7fd1f0791740 10 client.36392 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T11:04:21.579 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.609+0000 7fd1f0791740 20 client.36392 populate_metadata read hostname 'smithi097' 2024-03-27T11:04:21.579 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.609+0000 7fd1f0791740 1 --2- 172.21.15.97:0/1318837290 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55afd3ae8920 0x55afd3aead10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:04:21.579 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.609+0000 7fd1f0791740 4 client.36392 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T11:04:21.579 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.609+0000 7fd1f0791740 1 -- 172.21.15.97:0/1318837290 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55afd3a23250 con 0x55afd3ae8920 2024-03-27T11:04:21.579 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.610+0000 7fd1e6000640 1 --2- 172.21.15.97:0/1318837290 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55afd3ae8920 0x55afd3aead10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:04:21.580 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.611+0000 7fd1e6000640 1 --2- 172.21.15.97:0/1318837290 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55afd3ae8920 0x55afd3aead10 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=593838c18f4f1d7 server_cookie=e2c99e87ba6ae46e in_seq=0 out_seq=0 2024-03-27T11:04:21.580 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.611+0000 7fd1d4c00640 10 client.36392 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T11:04:21.583 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.614+0000 7fd1d4c00640 1 -- 172.21.15.97:0/1318837290 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55afd3a23250 con 0x55afd3ae8920 2024-03-27T11:04:21.584 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.614+0000 7fd1d4c00640 10 client.36392 handle_command_reply: tid=0 2024-03-27T11:04:21.584 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.614+0000 7fd1f0791740 1 client.36392 shutdown 2024-03-27T11:04:21.584 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.615+0000 7fd1f0791740 1 -- 172.21.15.97:0/1318837290 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55afd3ab0b60 msgr2=0x55afd3acbf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:04:21.584 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.615+0000 7fd1f0791740 1 --2- 172.21.15.97:0/1318837290 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55afd3ab0b60 0x55afd3acbf10 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x55afd3aad040 tx=0x7fd1d8004270 comp rx=0 tx=0).stop 2024-03-27T11:04:21.585 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.615+0000 7fd1f0791740 1 -- 172.21.15.97:0/1318837290 shutdown_connections 2024-03-27T11:04:21.585 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.615+0000 7fd1f0791740 1 --2- 172.21.15.97:0/1318837290 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55afd3ae8920 0x55afd3aead10 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:21.585 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.615+0000 7fd1f0791740 1 --2- 172.21.15.97:0/1318837290 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55afd3ab66c0 0x55afd3ad3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:21.585 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.615+0000 7fd1f0791740 1 --2- 172.21.15.97:0/1318837290 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55afd3ab1550 0x55afd3acc450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:21.585 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.615+0000 7fd1f0791740 1 --2- 172.21.15.97:0/1318837290 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55afd3ab0b60 0x55afd3acbf10 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:21.585 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.615+0000 7fd1f0791740 1 -- 172.21.15.97:0/1318837290 >> 172.21.15.97:0/1318837290 conn(0x55afd3aac0d0 msgr2=0x55afd3aab310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:04:21.585 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.615+0000 7fd1d4c00640 0 client.36392 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T11:04:21.586 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.616+0000 7fd1f0791740 1 -- 172.21.15.97:0/1318837290 shutdown_connections 2024-03-27T11:04:21.586 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.616+0000 7fd1f0791740 1 -- 172.21.15.97:0/1318837290 wait complete. 2024-03-27T11:04:21.586 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.616+0000 7fd1f0791740 20 client.36392 trim_cache size 0 max 16384 2024-03-27T11:04:21.615 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.645+0000 7fd1f0791740 1 Processor -- start 2024-03-27T11:04:21.615 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.645+0000 7fd1f0791740 1 -- start start 2024-03-27T11:04:21.616 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.646+0000 7fd1f0791740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55afd3ab0b60 0x55afd3aafa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:04:21.616 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.646+0000 7fd1f0791740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55afd3ab1550 0x55afd3ad33e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:04:21.616 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.646+0000 7fd1f0791740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55afd3ab66c0 0x55afd3aef490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:04:21.616 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.646+0000 7fd1f0791740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd1e013aee0 con 0x55afd3ab0b60 2024-03-27T11:04:21.616 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.646+0000 7fd1f0791740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd1e013ca20 con 0x55afd3ab1550 2024-03-27T11:04:21.616 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.646+0000 7fd1f0791740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd1e013b5b0 con 0x55afd3ab66c0 2024-03-27T11:04:21.617 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.647+0000 7fd1e7400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55afd3ab66c0 0x55afd3aef490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:04:21.617 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.647+0000 7fd1e7400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55afd3ab66c0 0x55afd3aef490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.97:57412/0 (socket says 172.21.15.97:57412) 2024-03-27T11:04:21.617 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.647+0000 7fd1e7400640 1 -- 172.21.15.97:0/2408651868 learned_addr learned my addr 172.21.15.97:0/2408651868 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:04:21.618 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.647+0000 7fd1e6a00640 1 --2- 172.21.15.97:0/2408651868 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55afd3ab0b60 0x55afd3aafa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:04:21.618 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.648+0000 7fd1e7400640 1 -- 172.21.15.97:0/2408651868 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55afd3ab1550 msgr2=0x55afd3ad33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:04:21.618 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.648+0000 7fd1e7400640 1 --2- 172.21.15.97:0/2408651868 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55afd3ab1550 0x55afd3ad33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:21.618 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.648+0000 7fd1e7400640 1 -- 172.21.15.97:0/2408651868 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55afd3ab0b60 msgr2=0x55afd3aafa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:04:21.618 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.648+0000 7fd1e7400640 1 --2- 172.21.15.97:0/2408651868 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55afd3ab0b60 0x55afd3aafa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:21.618 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.648+0000 7fd1e7400640 1 -- 172.21.15.97:0/2408651868 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55afd37f6620 con 0x55afd3ab66c0 2024-03-27T11:04:21.619 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.648+0000 7fd1e7400640 1 --2- 172.21.15.97:0/2408651868 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55afd3ab66c0 0x55afd3aef490 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd1e8064ef0 tx=0x7fd1e8106e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4fe27a20fbf78914 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:04:21.619 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.648+0000 7fd1d7400640 1 -- 172.21.15.97:0/2408651868 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1e8125070 con 0x55afd3ab66c0 2024-03-27T11:04:21.620 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.649+0000 7fd1d7400640 1 -- 172.21.15.97:0/2408651868 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd1e810b5c0 con 0x55afd3ab66c0 2024-03-27T11:04:21.620 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.649+0000 7fd1d7400640 1 -- 172.21.15.97:0/2408651868 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1e811f030 con 0x55afd3ab66c0 2024-03-27T11:04:21.620 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.650+0000 7fd1f0791740 1 -- 172.21.15.97:0/2408651868 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55afd3ab66c0 msgr2=0x55afd3aef490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:04:21.620 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.650+0000 7fd1f0791740 1 --2- 172.21.15.97:0/2408651868 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55afd3ab66c0 0x55afd3aef490 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd1e8064ef0 tx=0x7fd1e8106e80 comp rx=0 tx=0).stop 2024-03-27T11:04:21.620 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.650+0000 7fd1f0791740 1 -- 172.21.15.97:0/2408651868 shutdown_connections 2024-03-27T11:04:21.620 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.650+0000 7fd1f0791740 1 --2- 172.21.15.97:0/2408651868 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55afd3ab66c0 0x55afd3aef490 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:21.621 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.650+0000 7fd1f0791740 1 --2- 172.21.15.97:0/2408651868 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55afd3ab1550 0x55afd3ad33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:21.621 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.651+0000 7fd1f0791740 1 --2- 172.21.15.97:0/2408651868 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55afd3ab0b60 0x55afd3aafa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:21.621 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.651+0000 7fd1f0791740 1 -- 172.21.15.97:0/2408651868 >> 172.21.15.97:0/2408651868 conn(0x55afd3aac0d0 msgr2=0x55afd3aaf330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:04:21.621 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.652+0000 7fd1f0791740 1 -- 172.21.15.97:0/2408651868 shutdown_connections 2024-03-27T11:04:21.621 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.652+0000 7fd1f0791740 1 -- 172.21.15.97:0/2408651868 wait complete. 2024-03-27T11:04:21.623 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.653+0000 7fd1f0791740 1 -- start start 2024-03-27T11:04:21.623 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.654+0000 7fd1f0791740 1 Processor -- start 2024-03-27T11:04:21.625 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.656+0000 7fd1f0791740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55afd3ab0b60 0x55afd3ad3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:04:21.625 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.656+0000 7fd1f0791740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55afd3ab1550 0x55afd3acfc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:04:21.625 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.656+0000 7fd1f0791740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55afd3ab66c0 0x55afd3ad0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:04:21.626 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.656+0000 7fd1f0791740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd1e013c350 con 0x55afd3ab66c0 2024-03-27T11:04:21.626 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.656+0000 7fd1f0791740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd1e013bc80 con 0x55afd3ab0b60 2024-03-27T11:04:21.626 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.656+0000 7fd1f0791740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd1e0134820 con 0x55afd3ab1550 2024-03-27T11:04:21.626 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.656+0000 7fd1f0791740 10 client.0 _ll_register_callbacks cb 0x7fd1d8009490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T11:04:21.627 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.656+0000 7fd1e6000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55afd3ab1550 0x55afd3acfc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:04:21.627 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.656+0000 7fd1e6000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55afd3ab1550 0x55afd3acfc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.97:57414/0 (socket says 172.21.15.97:57414) 2024-03-27T11:04:21.627 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.656+0000 7fd1e6000640 1 -- 172.21.15.97:0/1252479581 learned_addr learned my addr 172.21.15.97:0/1252479581 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:04:21.627 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.656+0000 7fd1e6a00640 1 --2- 172.21.15.97:0/1252479581 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55afd3ab0b60 0x55afd3ad3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:04:21.627 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.657+0000 7fd1e6000640 1 -- 172.21.15.97:0/1252479581 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55afd3ab0b60 msgr2=0x55afd3ad3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:04:21.627 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.657+0000 7fd1e6000640 1 --2- 172.21.15.97:0/1252479581 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55afd3ab0b60 0x55afd3ad3140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:21.627 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.657+0000 7fd1e6000640 1 -- 172.21.15.97:0/1252479581 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55afd3ab66c0 msgr2=0x55afd3ad0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:04:21.627 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.657+0000 7fd1e7400640 1 --2- 172.21.15.97:0/1252479581 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55afd3ab66c0 0x55afd3ad0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:04:21.628 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.657+0000 7fd1e6000640 1 --2- 172.21.15.97:0/1252479581 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55afd3ab66c0 0x55afd3ad0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:21.628 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.658+0000 7fd1e6000640 1 -- 172.21.15.97:0/1252479581 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55afd3ad0810 con 0x55afd3ab1550 2024-03-27T11:04:21.628 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.658+0000 7fd1e7400640 1 --2- 172.21.15.97:0/1252479581 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55afd3ab66c0 0x55afd3ad0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T11:04:21.629 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.658+0000 7fd1e6000640 1 --2- 172.21.15.97:0/1252479581 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55afd3ab1550 0x55afd3acfc30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd1d0004700 tx=0x7fd1d0004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2dc7fbfb91c13b75 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:04:21.629 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.659+0000 7fd1d4c00640 10 client.0 ms_handle_connect on v2:172.21.15.177:3300/0 2024-03-27T11:04:21.629 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.660+0000 7fd1d4c00640 1 -- 172.21.15.97:0/1252479581 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d001d310 con 0x55afd3ab1550 2024-03-27T11:04:21.629 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.660+0000 7fd1d4c00640 1 -- 172.21.15.97:0/1252479581 <== mon.1 v2:172.21.15.177:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fd1d0027030 con 0x55afd3ab1550 2024-03-27T11:04:21.629 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.660+0000 7fd1f0791740 1 -- 172.21.15.97:0/1252479581 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1e810b5c0 con 0x55afd3ab1550 2024-03-27T11:04:21.630 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.660+0000 7fd1d4c00640 1 -- 172.21.15.97:0/1252479581 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fd1d0027230 con 0x55afd3ab1550 2024-03-27T11:04:21.630 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.660+0000 7fd1f0791740 10 client.26116 fetch_fsmap learned FSMap version 34 2024-03-27T11:04:21.630 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.660+0000 7fd1f0791740 1 -- 172.21.15.97:0/1252479581 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55afd3b4dce0 con 0x55afd3ab1550 2024-03-27T11:04:21.630 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.661+0000 7fd1d4c00640 1 -- 172.21.15.97:0/1252479581 <== mon.1 v2:172.21.15.177:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd1e810b5c0 con 0x55afd3ab1550 2024-03-27T11:04:21.631 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.661+0000 7fd1d4c00640 1 -- 172.21.15.97:0/1252479581 <== mon.1 v2:172.21.15.177:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7fd1d001bba0 con 0x55afd3ab1550 2024-03-27T11:04:21.631 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.662+0000 7fd1f0791740 10 client.26116 fetch_fsmap finished waiting for FSMap version 34 2024-03-27T11:04:21.631 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.662+0000 7fd1f0791740 10 client.26116 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T11:04:21.631 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.662+0000 7fd1f0791740 20 client.26116 populate_metadata read hostname 'smithi097' 2024-03-27T11:04:21.631 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.662+0000 7fd1f0791740 1 --2- 172.21.15.97:0/1252479581 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55afd3b4e6f0 0x55afd3b50b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:04:21.632 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.662+0000 7fd1f0791740 4 client.26116 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-27T11:04:21.632 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.662+0000 7fd1f0791740 1 -- 172.21.15.97:0/1252479581 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55afd3a1d5a0 con 0x55afd3b4e6f0 2024-03-27T11:04:21.632 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.662+0000 7fd1e6a00640 1 --2- 172.21.15.97:0/1252479581 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55afd3b4e6f0 0x55afd3b50b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:04:21.633 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.664+0000 7fd1e6a00640 1 --2- 172.21.15.97:0/1252479581 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55afd3b4e6f0 0x55afd3b50b10 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3db7172e87fbbfc server_cookie=9595c6eb6218678e in_seq=0 out_seq=0 2024-03-27T11:04:21.633 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.664+0000 7fd1d4c00640 10 client.26116 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T11:04:21.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:04:21 smithi097 ceph-mon[31859]: pgmap v672: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 56 MiB/s wr, 906 op/s 2024-03-27T11:04:21.653 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.683+0000 7fd1d4c00640 1 -- 172.21.15.97:0/1252479581 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55afd3a1d5a0 con 0x55afd3b4e6f0 2024-03-27T11:04:21.653 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.683+0000 7fd1d4c00640 10 client.26116 handle_command_reply: tid=0 2024-03-27T11:04:21.653 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.684+0000 7fd1f0791740 1 client.26116 shutdown 2024-03-27T11:04:21.654 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.684+0000 7fd1f0791740 1 -- 172.21.15.97:0/1252479581 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55afd3ab1550 msgr2=0x55afd3acfc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:04:21.654 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.684+0000 7fd1f0791740 1 --2- 172.21.15.97:0/1252479581 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55afd3ab1550 0x55afd3acfc30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd1d0004700 tx=0x7fd1d0004730 comp rx=0 tx=0).stop 2024-03-27T11:04:21.655 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.685+0000 7fd1f0791740 1 -- 172.21.15.97:0/1252479581 shutdown_connections 2024-03-27T11:04:21.655 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.685+0000 7fd1f0791740 1 --2- 172.21.15.97:0/1252479581 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55afd3b4e6f0 0x55afd3b50b10 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:21.655 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.685+0000 7fd1f0791740 1 --2- 172.21.15.97:0/1252479581 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55afd3ab66c0 0x55afd3ad0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:21.655 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.685+0000 7fd1f0791740 1 --2- 172.21.15.97:0/1252479581 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55afd3ab1550 0x55afd3acfc30 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:21.655 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.685+0000 7fd1f0791740 1 --2- 172.21.15.97:0/1252479581 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55afd3ab0b60 0x55afd3ad3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:21.655 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.685+0000 7fd1f0791740 1 -- 172.21.15.97:0/1252479581 >> 172.21.15.97:0/1252479581 conn(0x55afd3aac0d0 msgr2=0x55afd3aafbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:04:21.656 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.686+0000 7fd1d4c00640 0 client.26116 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T11:04:21.656 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.686+0000 7fd1f0791740 1 -- 172.21.15.97:0/1252479581 shutdown_connections 2024-03-27T11:04:21.656 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.686+0000 7fd1f0791740 1 -- 172.21.15.97:0/1252479581 wait complete. 2024-03-27T11:04:21.656 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.687+0000 7fd1f0791740 20 client.26116 trim_cache size 0 max 16384 2024-03-27T11:04:21.656 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-27T11:04:21.656 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-27T11:04:21.656 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-27T11:04:21.657 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-27T11:04:21.657 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.687+0000 7fd1e7e00640 1 -- 172.21.15.97:0/4110170736 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fd1cc0a3760 msgr2=0x7fd1cc0a5c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:04:21.657 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.688+0000 7fd1e7e00640 1 --2- 172.21.15.97:0/4110170736 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fd1cc0a3760 0x7fd1cc0a5c20 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x55afd3ab1db0 tx=0x7fd1d000a860 comp rx=0 tx=0).stop 2024-03-27T11:04:21.658 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.688+0000 7fd1e7e00640 1 -- 172.21.15.97:0/4110170736 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd1e0095950 msgr2=0x7fd1e0096d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:04:21.658 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.688+0000 7fd1e7e00640 1 --2- 172.21.15.97:0/4110170736 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd1e0095950 0x7fd1e0096d40 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd1e804fb80 tx=0x7fd1e806aa50 comp rx=0 tx=0).stop 2024-03-27T11:04:21.658 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.688+0000 7fd1e7e00640 1 -- 172.21.15.97:0/4110170736 shutdown_connections 2024-03-27T11:04:21.658 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.688+0000 7fd1e7e00640 1 --2- 172.21.15.97:0/4110170736 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fd1cc0a3760 0x7fd1cc0a5c20 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:21.658 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.688+0000 7fd1e7e00640 1 --2- 172.21.15.97:0/4110170736 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd1e0095950 0x7fd1e0096d40 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:21.658 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.688+0000 7fd1e7e00640 1 --2- 172.21.15.97:0/4110170736 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd1e0094fb0 0x7fd1e009a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:21.658 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.688+0000 7fd1e7e00640 1 --2- 172.21.15.97:0/4110170736 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1e0094650 0x7fd1e009a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:21.658 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.688+0000 7fd1e7e00640 1 -- 172.21.15.97:0/4110170736 >> 172.21.15.97:0/4110170736 conn(0x7fd1e00901d0 msgr2=0x7fd1e0091cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:04:21.658 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.688+0000 7fd1e7e00640 1 -- 172.21.15.97:0/4110170736 shutdown_connections 2024-03-27T11:04:21.658 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:21.688+0000 7fd1e7e00640 1 -- 172.21.15.97:0/4110170736 wait complete. 2024-03-27T11:04:21.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:04:21 smithi177 ceph-mon[35294]: pgmap v672: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 56 MiB/s wr, 906 op/s 2024-03-27T11:04:21.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:04:21 smithi114 ceph-mon[35338]: pgmap v672: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 56 MiB/s wr, 906 op/s 2024-03-27T11:04:23.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:04:23 smithi097 ceph-mon[31859]: pgmap v673: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 51 MiB/s wr, 818 op/s 2024-03-27T11:04:23.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:04:23 smithi177 ceph-mon[35294]: pgmap v673: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 51 MiB/s wr, 818 op/s 2024-03-27T11:04:23.764 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:04:23 smithi114 ceph-mon[35338]: pgmap v673: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 51 MiB/s wr, 818 op/s 2024-03-27T11:04:25.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:04:25 smithi097 ceph-mon[31859]: pgmap v674: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 51 MiB/s wr, 832 op/s 2024-03-27T11:04:25.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:04:25 smithi177 ceph-mon[35294]: pgmap v674: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 51 MiB/s wr, 832 op/s 2024-03-27T11:04:25.764 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:04:25 smithi114 ceph-mon[35338]: pgmap v674: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 51 MiB/s wr, 832 op/s 2024-03-27T11:04:27.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:04:27 smithi097 ceph-mon[31859]: pgmap v675: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 49 MiB/s wr, 798 op/s 2024-03-27T11:04:27.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:04:27 smithi177 ceph-mon[35294]: pgmap v675: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 49 MiB/s wr, 798 op/s 2024-03-27T11:04:27.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:04:27 smithi114 ceph-mon[35338]: pgmap v675: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 49 MiB/s wr, 798 op/s 2024-03-27T11:04:29.641 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:04:29 smithi097 ceph-mon[31859]: pgmap v676: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 52 MiB/s wr, 834 op/s 2024-03-27T11:04:29.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:04:29 smithi177 ceph-mon[35294]: pgmap v676: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 52 MiB/s wr, 834 op/s 2024-03-27T11:04:29.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:04:29 smithi114 ceph-mon[35338]: pgmap v676: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 52 MiB/s wr, 834 op/s 2024-03-27T11:04:31.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:04:31 smithi097 ceph-mon[31859]: pgmap v677: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 55 MiB/s wr, 897 op/s 2024-03-27T11:04:31.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:04:31 smithi177 ceph-mon[35294]: pgmap v677: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 55 MiB/s wr, 897 op/s 2024-03-27T11:04:31.766 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:04:31 smithi114 ceph-mon[35338]: pgmap v677: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 55 MiB/s wr, 897 op/s 2024-03-27T11:04:33.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:04:33 smithi097 ceph-mon[31859]: pgmap v678: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 50 MiB/s wr, 808 op/s 2024-03-27T11:04:33.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:04:33 smithi177 ceph-mon[35294]: pgmap v678: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 50 MiB/s wr, 808 op/s 2024-03-27T11:04:33.764 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:04:33 smithi114 ceph-mon[35338]: pgmap v678: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 50 MiB/s wr, 808 op/s 2024-03-27T11:04:35.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:04:35 smithi097 ceph-mon[31859]: pgmap v679: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 50 MiB/s wr, 819 op/s 2024-03-27T11:04:35.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:04:35 smithi177 ceph-mon[35294]: pgmap v679: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 50 MiB/s wr, 819 op/s 2024-03-27T11:04:35.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:04:35 smithi114 ceph-mon[35338]: pgmap v679: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 50 MiB/s wr, 819 op/s 2024-03-27T11:04:36.040 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.069+0000 7f4f22200640 1 -- 172.21.15.97:0/2242507223 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4f1c105720 msgr2=0x7f4f1c105ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:04:36.040 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.069+0000 7f4f22200640 1 --2- 172.21.15.97:0/2242507223 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4f1c105720 0x7f4f1c105ba0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4f14009be0 tx=0x7f4f1402bef0 comp rx=0 tx=0).stop 2024-03-27T11:04:36.040 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.069+0000 7f4f22200640 1 -- 172.21.15.97:0/2242507223 shutdown_connections 2024-03-27T11:04:36.040 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.069+0000 7f4f22200640 1 --2- 172.21.15.97:0/2242507223 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4f1c105720 0x7f4f1c105ba0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:36.040 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.069+0000 7f4f22200640 1 --2- 172.21.15.97:0/2242507223 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4f1c102d70 0x7f4f1c1051e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:36.040 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.069+0000 7f4f22200640 1 --2- 172.21.15.97:0/2242507223 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4f1c100400 0x7f4f1c102830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:36.040 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.069+0000 7f4f22200640 1 -- 172.21.15.97:0/2242507223 >> 172.21.15.97:0/2242507223 conn(0x7f4f1c0fc000 msgr2=0x7f4f1c0fe440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:04:36.040 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.069+0000 7f4f22200640 1 -- 172.21.15.97:0/2242507223 shutdown_connections 2024-03-27T11:04:36.041 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.069+0000 7f4f22200640 1 -- 172.21.15.97:0/2242507223 wait complete. 2024-03-27T11:04:36.041 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.071+0000 7f4f22200640 1 Processor -- start 2024-03-27T11:04:36.041 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.071+0000 7f4f22200640 1 -- start start 2024-03-27T11:04:36.041 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.071+0000 7f4f22200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4f1c100400 0x7f4f1c19e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:04:36.042 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.071+0000 7f4f22200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4f1c102d70 0x7f4f1c19e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:04:36.042 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.071+0000 7f4f22200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4f1c105720 0x7f4f1c1a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:04:36.042 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.071+0000 7f4f22200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4f1c113d30 con 0x7f4f1c105720 2024-03-27T11:04:36.042 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.071+0000 7f4f22200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f4f1c113bb0 con 0x7f4f1c100400 2024-03-27T11:04:36.042 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.071+0000 7f4f22200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f4f1c113eb0 con 0x7f4f1c102d70 2024-03-27T11:04:36.043 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.072+0000 7f4f1b400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4f1c100400 0x7f4f1c19e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:04:36.043 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.072+0000 7f4f1b400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4f1c100400 0x7f4f1c19e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:53530/0 (socket says 172.21.15.97:53530) 2024-03-27T11:04:36.043 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.072+0000 7f4f1b400640 1 -- 172.21.15.97:0/1058254743 learned_addr learned my addr 172.21.15.97:0/1058254743 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:04:36.043 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.072+0000 7f4f1be00640 1 --2- 172.21.15.97:0/1058254743 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4f1c105720 0x7f4f1c1a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:04:36.043 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.072+0000 7f4f1aa00640 1 --2- 172.21.15.97:0/1058254743 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4f1c102d70 0x7f4f1c19e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:04:36.044 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.072+0000 7f4f1b400640 1 -- 172.21.15.97:0/1058254743 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4f1c102d70 msgr2=0x7f4f1c19e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:04:36.044 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.072+0000 7f4f1b400640 1 --2- 172.21.15.97:0/1058254743 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4f1c102d70 0x7f4f1c19e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:36.044 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.072+0000 7f4f1b400640 1 -- 172.21.15.97:0/1058254743 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4f1c105720 msgr2=0x7f4f1c1a5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:04:36.044 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.073+0000 7f4f1b400640 1 --2- 172.21.15.97:0/1058254743 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4f1c105720 0x7f4f1c1a5810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:36.044 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.073+0000 7f4f1b400640 1 -- 172.21.15.97:0/1058254743 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f1c1a5e80 con 0x7f4f1c100400 2024-03-27T11:04:36.044 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.073+0000 7f4f1be00640 1 --2- 172.21.15.97:0/1058254743 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4f1c105720 0x7f4f1c1a5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T11:04:36.044 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.073+0000 7f4f1b400640 1 --2- 172.21.15.97:0/1058254743 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4f1c100400 0x7f4f1c19e260 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c00d950 tx=0x7f4f0c00de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:04:36.044 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.073+0000 7f4f13e00640 1 -- 172.21.15.97:0/1058254743 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f0c017070 con 0x7f4f1c100400 2024-03-27T11:04:36.045 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.074+0000 7f4f13e00640 1 -- 172.21.15.97:0/1058254743 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4f0c009c30 con 0x7f4f1c100400 2024-03-27T11:04:36.045 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.074+0000 7f4f13e00640 1 -- 172.21.15.97:0/1058254743 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f0c00f620 con 0x7f4f1c100400 2024-03-27T11:04:36.045 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.074+0000 7f4f22200640 1 -- 172.21.15.97:0/1058254743 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f1c1a6170 con 0x7f4f1c100400 2024-03-27T11:04:36.045 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.074+0000 7f4f22200640 1 -- 172.21.15.97:0/1058254743 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f1c1a6670 con 0x7f4f1c100400 2024-03-27T11:04:36.048 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.078+0000 7f4f2502b740 1 Processor -- start 2024-03-27T11:04:36.049 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.078+0000 7f4f13e00640 1 -- 172.21.15.97:0/1058254743 <== mon.2 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f4f0c0109a0 con 0x7f4f1c100400 2024-03-27T11:04:36.049 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.078+0000 7f4f2502b740 1 -- start start 2024-03-27T11:04:36.049 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.079+0000 7f4f2502b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c13bfab980 0x55c13bfabda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:04:36.049 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.079+0000 7f4f2502b740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c13bfac370 0x55c13bfb0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:04:36.050 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.079+0000 7f4f2502b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c13bfb14e0 0x55c13bfb3970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:04:36.050 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.079+0000 7f4f2502b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55c13bf10d10 con 0x55c13bfb14e0 2024-03-27T11:04:36.050 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.079+0000 7f4f2502b740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55c13beb9260 con 0x55c13bfab980 2024-03-27T11:04:36.050 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.079+0000 7f4f2502b740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55c13bf0ccc0 con 0x55c13bfac370 2024-03-27T11:04:36.051 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.079+0000 7f4f1b400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c13bfac370 0x55c13bfb0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:04:36.051 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.080+0000 7f4f1b400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c13bfac370 0x55c13bfb0e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.97:49932/0 (socket says 172.21.15.97:49932) 2024-03-27T11:04:36.051 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.080+0000 7f4f1b400640 1 -- 172.21.15.97:0/1706922907 learned_addr learned my addr 172.21.15.97:0/1706922907 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:04:36.051 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.080+0000 7f4f1aa00640 1 --2- 172.21.15.97:0/1706922907 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c13bfb14e0 0x55c13bfb3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:04:36.051 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.080+0000 7f4f1be00640 1 --2- 172.21.15.97:0/1706922907 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c13bfab980 0x55c13bfabda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:04:36.051 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.080+0000 7f4f1b400640 1 -- 172.21.15.97:0/1706922907 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c13bfab980 msgr2=0x55c13bfabda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:04:36.051 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.080+0000 7f4f1b400640 1 --2- 172.21.15.97:0/1706922907 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c13bfab980 0x55c13bfabda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:36.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.080+0000 7f4f1b400640 1 -- 172.21.15.97:0/1706922907 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c13bfb14e0 msgr2=0x55c13bfb3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:04:36.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.080+0000 7f4f13e00640 1 --2- 172.21.15.97:0/1058254743 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f4f000a3730 0x7f4f000a5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:04:36.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.081+0000 7f4f1b400640 1 --2- 172.21.15.97:0/1706922907 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c13bfb14e0 0x55c13bfb3970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:36.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.081+0000 7f4f13e00640 1 -- 172.21.15.97:0/1058254743 <== mon.2 v2:172.21.15.114:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f4f0c0cf880 con 0x7f4f1c100400 2024-03-27T11:04:36.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.081+0000 7f4f1b400640 1 -- 172.21.15.97:0/1706922907 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c13beb0aa0 con 0x55c13bfac370 2024-03-27T11:04:36.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.081+0000 7f4f1be00640 1 --2- 172.21.15.97:0/1058254743 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f4f000a3730 0x7f4f000a5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:04:36.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.081+0000 7f4f1aa00640 1 --2- 172.21.15.97:0/1706922907 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c13bfb14e0 0x55c13bfb3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T11:04:36.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.081+0000 7f4f1b400640 1 --2- 172.21.15.97:0/1706922907 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c13bfac370 0x55c13bfb0e70 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c098780 tx=0x7f4f0c00bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3acf9ad93c43a13 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:04:36.053 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.082+0000 7f4f07e00640 1 -- 172.21.15.97:0/1706922907 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f0c020030 con 0x55c13bfac370 2024-03-27T11:04:36.053 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.082+0000 7f4f07e00640 1 -- 172.21.15.97:0/1706922907 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4f0c009de0 con 0x55c13bfac370 2024-03-27T11:04:36.053 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.082+0000 7f4f07e00640 1 -- 172.21.15.97:0/1706922907 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f0c0bc0d0 con 0x55c13bfac370 2024-03-27T11:04:36.053 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.083+0000 7f4f2502b740 1 -- 172.21.15.97:0/1706922907 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c13bfac370 msgr2=0x55c13bfb0e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:04:36.053 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.083+0000 7f4f1be00640 1 --2- 172.21.15.97:0/1058254743 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f4f000a3730 0x7f4f000a5bf0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f4f14034fd0 tx=0x7f4f14005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:04:36.053 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.083+0000 7f4f2502b740 1 --2- 172.21.15.97:0/1706922907 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c13bfac370 0x55c13bfb0e70 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c098780 tx=0x7f4f0c00bf60 comp rx=0 tx=0).stop 2024-03-27T11:04:36.054 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.083+0000 7f4f2502b740 1 -- 172.21.15.97:0/1706922907 shutdown_connections 2024-03-27T11:04:36.054 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.083+0000 7f4f2502b740 1 --2- 172.21.15.97:0/1706922907 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c13bfb14e0 0x55c13bfb3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:36.054 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.083+0000 7f4f2502b740 1 --2- 172.21.15.97:0/1706922907 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c13bfac370 0x55c13bfb0e70 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:36.054 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.083+0000 7f4f2502b740 1 --2- 172.21.15.97:0/1706922907 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c13bfab980 0x55c13bfabda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:36.054 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.083+0000 7f4f2502b740 1 -- 172.21.15.97:0/1706922907 >> 172.21.15.97:0/1706922907 conn(0x55c13bfa6ef0 msgr2=0x55c13bfa72f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:04:36.055 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.083+0000 7f4f2502b740 1 -- 172.21.15.97:0/1706922907 shutdown_connections 2024-03-27T11:04:36.055 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.083+0000 7f4f2502b740 1 -- 172.21.15.97:0/1706922907 wait complete. 2024-03-27T11:04:36.056 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.084+0000 7f4f2502b740 1 -- start start 2024-03-27T11:04:36.056 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.084+0000 7f4f2502b740 1 Processor -- start 2024-03-27T11:04:36.056 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.085+0000 7f4f2502b740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c13bfab980 0x55c13bfb5c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:04:36.056 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.085+0000 7f4f2502b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c13bfac370 0x55c13bfb6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:04:36.056 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.085+0000 7f4f2502b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c13bfb14e0 0x55c13bfbd1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:04:36.056 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.085+0000 7f4f2502b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55c13bfb43d0 con 0x55c13bfac370 2024-03-27T11:04:36.057 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.085+0000 7f4f2502b740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55c13becc630 con 0x55c13bfb14e0 2024-03-27T11:04:36.057 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.085+0000 7f4f2502b740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55c13bfb4550 con 0x55c13bfab980 2024-03-27T11:04:36.057 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.085+0000 7f4f2502b740 10 client.0 _ll_register_callbacks cb 0x55c13bed1c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T11:04:36.057 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.086+0000 7f4f1be00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c13bfb14e0 0x55c13bfbd1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:04:36.057 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.086+0000 7f4f1be00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c13bfb14e0 0x55c13bfbd1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:53538/0 (socket says 172.21.15.97:53538) 2024-03-27T11:04:36.057 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.086+0000 7f4f1be00640 1 -- 172.21.15.97:0/2899559097 learned_addr learned my addr 172.21.15.97:0/2899559097 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:04:36.057 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.087+0000 7f4f1b400640 1 --2- 172.21.15.97:0/2899559097 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c13bfab980 0x55c13bfb5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:04:36.058 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.087+0000 7f4f1aa00640 1 --2- 172.21.15.97:0/2899559097 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c13bfac370 0x55c13bfb6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:04:36.058 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.088+0000 7f4f1be00640 1 -- 172.21.15.97:0/2899559097 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c13bfab980 msgr2=0x55c13bfb5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:04:36.063 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.088+0000 7f4f1be00640 1 --2- 172.21.15.97:0/2899559097 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c13bfab980 0x55c13bfb5c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:36.063 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.088+0000 7f4f1be00640 1 -- 172.21.15.97:0/2899559097 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c13bfac370 msgr2=0x55c13bfb6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:04:36.063 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.088+0000 7f4f1be00640 1 --2- 172.21.15.97:0/2899559097 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c13bfac370 0x55c13bfb6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:36.063 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.088+0000 7f4f1be00640 1 -- 172.21.15.97:0/2899559097 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c13bed85f0 con 0x55c13bfb14e0 2024-03-27T11:04:36.063 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.088+0000 7f4f1b400640 1 --2- 172.21.15.97:0/2899559097 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c13bfab980 0x55c13bfb5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T11:04:36.063 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.088+0000 7f4f1aa00640 1 --2- 172.21.15.97:0/2899559097 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c13bfac370 0x55c13bfb6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T11:04:36.064 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.089+0000 7f4f1be00640 1 --2- 172.21.15.97:0/2899559097 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c13bfb14e0 0x55c13bfbd1a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4f14035000 tx=0x7f4f1402bef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d90465497b1d52d7 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:04:36.064 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.089+0000 7f4f04c00640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-03-27T11:04:36.064 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.090+0000 7f4f04c00640 1 -- 172.21.15.97:0/2899559097 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f14051020 con 0x55c13bfb14e0 2024-03-27T11:04:36.064 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.090+0000 7f4f04c00640 1 -- 172.21.15.97:0/2899559097 <== mon.2 v2:172.21.15.114:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f4f1402cd90 con 0x55c13bfb14e0 2024-03-27T11:04:36.064 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.090+0000 7f4f2502b740 1 -- 172.21.15.97:0/2899559097 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f0c009de0 con 0x55c13bfb14e0 2024-03-27T11:04:36.064 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.091+0000 7f4f04c00640 1 -- 172.21.15.97:0/2899559097 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f4f14034430 con 0x55c13bfb14e0 2024-03-27T11:04:36.064 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.091+0000 7f4f2502b740 10 client.36407 fetch_fsmap learned FSMap version 34 2024-03-27T11:04:36.064 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.091+0000 7f4f2502b740 1 -- 172.21.15.97:0/2899559097 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55c13bc25210 con 0x55c13bfb14e0 2024-03-27T11:04:36.064 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.091+0000 7f4f04c00640 1 -- 172.21.15.97:0/2899559097 <== mon.2 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4f0c009de0 con 0x55c13bfb14e0 2024-03-27T11:04:36.064 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.091+0000 7f4f04c00640 1 -- 172.21.15.97:0/2899559097 <== mon.2 v2:172.21.15.114:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f4f1405e020 con 0x55c13bfb14e0 2024-03-27T11:04:36.064 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.092+0000 7f4f2502b740 10 client.36407 fetch_fsmap finished waiting for FSMap version 34 2024-03-27T11:04:36.064 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.092+0000 7f4f2502b740 10 client.36407 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T11:04:36.064 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.092+0000 7f4f2502b740 20 client.36407 populate_metadata read hostname 'smithi097' 2024-03-27T11:04:36.064 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.092+0000 7f4f2502b740 1 --2- 172.21.15.97:0/2899559097 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55c13bfd2af0 0x55c13bfd4f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:04:36.065 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.092+0000 7f4f2502b740 4 client.36407 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T11:04:36.065 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.092+0000 7f4f2502b740 1 -- 172.21.15.97:0/2899559097 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c13bf1e250 con 0x55c13bfd2af0 2024-03-27T11:04:36.065 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.093+0000 7f4f1b400640 1 --2- 172.21.15.97:0/2899559097 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55c13bfd2af0 0x55c13bfd4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:04:36.065 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.094+0000 7f4f1b400640 1 --2- 172.21.15.97:0/2899559097 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55c13bfd2af0 0x55c13bfd4f10 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98ba424b75c6b7e server_cookie=737e2fe8f4ffd310 in_seq=0 out_seq=0 2024-03-27T11:04:36.065 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.094+0000 7f4f04c00640 10 client.36407 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T11:04:36.068 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.097+0000 7f4f04c00640 1 -- 172.21.15.97:0/2899559097 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55c13bf1e250 con 0x55c13bfd2af0 2024-03-27T11:04:36.068 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.097+0000 7f4f04c00640 10 client.36407 handle_command_reply: tid=0 2024-03-27T11:04:36.068 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.097+0000 7f4f2502b740 1 client.36407 shutdown 2024-03-27T11:04:36.068 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.098+0000 7f4f2502b740 1 -- 172.21.15.97:0/2899559097 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c13bfb14e0 msgr2=0x55c13bfbd1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:04:36.068 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.098+0000 7f4f2502b740 1 --2- 172.21.15.97:0/2899559097 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c13bfb14e0 0x55c13bfbd1a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4f14035000 tx=0x7f4f1402bef0 comp rx=0 tx=0).stop 2024-03-27T11:04:36.070 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.100+0000 7f4f2502b740 1 -- 172.21.15.97:0/2899559097 shutdown_connections 2024-03-27T11:04:36.071 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.100+0000 7f4f2502b740 1 --2- 172.21.15.97:0/2899559097 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55c13bfd2af0 0x55c13bfd4f10 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:36.071 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.100+0000 7f4f2502b740 1 --2- 172.21.15.97:0/2899559097 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c13bfb14e0 0x55c13bfbd1a0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:36.071 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.100+0000 7f4f2502b740 1 --2- 172.21.15.97:0/2899559097 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c13bfac370 0x55c13bfb6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:36.071 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.100+0000 7f4f04c00640 0 client.36407 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T11:04:36.071 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.100+0000 7f4f2502b740 1 --2- 172.21.15.97:0/2899559097 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c13bfab980 0x55c13bfb5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:36.071 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.100+0000 7f4f2502b740 1 -- 172.21.15.97:0/2899559097 >> 172.21.15.97:0/2899559097 conn(0x55c13bfa6ef0 msgr2=0x55c13bfa60f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:04:36.071 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.100+0000 7f4f2502b740 1 -- 172.21.15.97:0/2899559097 shutdown_connections 2024-03-27T11:04:36.071 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.100+0000 7f4f2502b740 1 -- 172.21.15.97:0/2899559097 wait complete. 2024-03-27T11:04:36.071 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.101+0000 7f4f2502b740 20 client.36407 trim_cache size 0 max 16384 2024-03-27T11:04:36.110 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.139+0000 7f4f2502b740 1 Processor -- start 2024-03-27T11:04:36.110 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.139+0000 7f4f2502b740 1 -- start start 2024-03-27T11:04:36.110 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.139+0000 7f4f2502b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c13bfab980 0x55c13bfcec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:04:36.110 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.139+0000 7f4f2502b740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c13bfac370 0x55c13bfbd160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:04:36.111 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.140+0000 7f4f2502b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c13bfb14e0 0x55c13bfb48f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:04:36.111 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.140+0000 7f4f2502b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4f1c1a6e10 con 0x55c13bfab980 2024-03-27T11:04:36.111 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.140+0000 7f4f2502b740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f4f1c1a8950 con 0x55c13bfb14e0 2024-03-27T11:04:36.111 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.140+0000 7f4f2502b740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f4f1c1a74e0 con 0x55c13bfac370 2024-03-27T11:04:36.111 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.140+0000 7f4f1aa00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c13bfac370 0x55c13bfbd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:04:36.111 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.140+0000 7f4f1aa00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c13bfac370 0x55c13bfbd160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.97:49948/0 (socket says 172.21.15.97:49948) 2024-03-27T11:04:36.111 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.140+0000 7f4f1aa00640 1 -- 172.21.15.97:0/3484376932 learned_addr learned my addr 172.21.15.97:0/3484376932 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:04:36.111 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.140+0000 7f4f1b400640 1 --2- 172.21.15.97:0/3484376932 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c13bfab980 0x55c13bfcec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:04:36.112 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.140+0000 7f4f1be00640 1 --2- 172.21.15.97:0/3484376932 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c13bfb14e0 0x55c13bfb48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:04:36.112 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.141+0000 7f4f1aa00640 1 -- 172.21.15.97:0/3484376932 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c13bfb14e0 msgr2=0x55c13bfb48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:04:36.112 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.141+0000 7f4f1aa00640 1 --2- 172.21.15.97:0/3484376932 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c13bfb14e0 0x55c13bfb48f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:36.112 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.141+0000 7f4f1aa00640 1 -- 172.21.15.97:0/3484376932 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c13bfab980 msgr2=0x55c13bfcec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:04:36.112 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.141+0000 7f4f1aa00640 1 --2- 172.21.15.97:0/3484376932 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c13bfab980 0x55c13bfcec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:36.112 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.141+0000 7f4f1aa00640 1 -- 172.21.15.97:0/3484376932 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c13bfd5280 con 0x55c13bfac370 2024-03-27T11:04:36.113 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.141+0000 7f4f1b400640 1 --2- 172.21.15.97:0/3484376932 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c13bfab980 0x55c13bfcec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T11:04:36.113 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.141+0000 7f4f1aa00640 1 --2- 172.21.15.97:0/3484376932 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c13bfac370 0x55c13bfbd160 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4f080125c0 tx=0x7f4f08005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac4e7b7ff7e4f0ef server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:04:36.113 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.142+0000 7f4f07400640 1 -- 172.21.15.97:0/3484376932 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f08012190 con 0x55c13bfac370 2024-03-27T11:04:36.113 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.142+0000 7f4f07400640 1 -- 172.21.15.97:0/3484376932 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4f080035f0 con 0x55c13bfac370 2024-03-27T11:04:36.113 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.142+0000 7f4f07400640 1 -- 172.21.15.97:0/3484376932 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f0800f820 con 0x55c13bfac370 2024-03-27T11:04:36.114 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.143+0000 7f4f2502b740 1 -- 172.21.15.97:0/3484376932 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c13bfac370 msgr2=0x55c13bfbd160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:04:36.114 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.143+0000 7f4f2502b740 1 --2- 172.21.15.97:0/3484376932 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c13bfac370 0x55c13bfbd160 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4f080125c0 tx=0x7f4f08005f90 comp rx=0 tx=0).stop 2024-03-27T11:04:36.114 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.143+0000 7f4f2502b740 1 -- 172.21.15.97:0/3484376932 shutdown_connections 2024-03-27T11:04:36.114 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.143+0000 7f4f2502b740 1 --2- 172.21.15.97:0/3484376932 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c13bfb14e0 0x55c13bfb48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:36.114 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.143+0000 7f4f2502b740 1 --2- 172.21.15.97:0/3484376932 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c13bfac370 0x55c13bfbd160 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:36.114 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.143+0000 7f4f2502b740 1 --2- 172.21.15.97:0/3484376932 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c13bfab980 0x55c13bfcec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:36.114 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.143+0000 7f4f2502b740 1 -- 172.21.15.97:0/3484376932 >> 172.21.15.97:0/3484376932 conn(0x55c13bfa6ef0 msgr2=0x55c13bfb5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:04:36.114 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.143+0000 7f4f2502b740 1 -- 172.21.15.97:0/3484376932 shutdown_connections 2024-03-27T11:04:36.115 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.144+0000 7f4f2502b740 1 -- 172.21.15.97:0/3484376932 wait complete. 2024-03-27T11:04:36.115 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.144+0000 7f4f2502b740 1 -- start start 2024-03-27T11:04:36.115 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.144+0000 7f4f2502b740 1 Processor -- start 2024-03-27T11:04:36.116 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.145+0000 7f4f2502b740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c13bfab980 0x55c13bfba000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:04:36.116 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.145+0000 7f4f2502b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c13bfac370 0x55c13bfba540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:04:36.116 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.145+0000 7f4f2502b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c13bfb14e0 0x55c13c036e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:04:36.116 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.145+0000 7f4f2502b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4f1c1a8280 con 0x55c13bfac370 2024-03-27T11:04:36.116 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.145+0000 7f4f2502b740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f4f1c1a7bb0 con 0x55c13bfb14e0 2024-03-27T11:04:36.116 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.145+0000 7f4f2502b740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f4f1c072190 con 0x55c13bfab980 2024-03-27T11:04:36.116 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.145+0000 7f4f2502b740 10 client.0 _ll_register_callbacks cb 0x7f4f14044070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T11:04:36.116 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.145+0000 7f4f1be00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c13bfb14e0 0x55c13c036e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:04:36.116 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.145+0000 7f4f1be00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c13bfb14e0 0x55c13c036e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:53558/0 (socket says 172.21.15.97:53558) 2024-03-27T11:04:36.116 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.145+0000 7f4f1be00640 1 -- 172.21.15.97:0/184652274 learned_addr learned my addr 172.21.15.97:0/184652274 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:04:36.117 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.146+0000 7f4f1b400640 1 --2- 172.21.15.97:0/184652274 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c13bfab980 0x55c13bfba000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:04:36.117 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.146+0000 7f4f1aa00640 1 --2- 172.21.15.97:0/184652274 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c13bfac370 0x55c13bfba540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:04:36.118 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.146+0000 7f4f1be00640 1 -- 172.21.15.97:0/184652274 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c13bfab980 msgr2=0x55c13bfba000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:04:36.118 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.146+0000 7f4f1be00640 1 --2- 172.21.15.97:0/184652274 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c13bfab980 0x55c13bfba000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:36.118 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.146+0000 7f4f1be00640 1 -- 172.21.15.97:0/184652274 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c13bfac370 msgr2=0x55c13bfba540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:04:36.118 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.146+0000 7f4f1be00640 1 --2- 172.21.15.97:0/184652274 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c13bfac370 0x55c13bfba540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.146+0000 7f4f1be00640 1 -- 172.21.15.97:0/184652274 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c13bcf1620 con 0x55c13bfb14e0 2024-03-27T11:04:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.146+0000 7f4f1b400640 1 --2- 172.21.15.97:0/184652274 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c13bfab980 0x55c13bfba000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T11:04:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.146+0000 7f4f1be00640 1 --2- 172.21.15.97:0/184652274 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c13bfb14e0 0x55c13c036e20 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4f140099a0 tx=0x7f4f140041b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f0f939f930bfe2d server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:04:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.146+0000 7f4f04c00640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-03-27T11:04:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.147+0000 7f4f04c00640 1 -- 172.21.15.97:0/184652274 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f14034430 con 0x55c13bfb14e0 2024-03-27T11:04:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.147+0000 7f4f04c00640 1 -- 172.21.15.97:0/184652274 <== mon.2 v2:172.21.15.114:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f4f140347e0 con 0x55c13bfb14e0 2024-03-27T11:04:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.147+0000 7f4f2502b740 1 -- 172.21.15.97:0/184652274 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f080035f0 con 0x55c13bfb14e0 2024-03-27T11:04:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.147+0000 7f4f04c00640 1 -- 172.21.15.97:0/184652274 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f4f14033a10 con 0x55c13bfb14e0 2024-03-27T11:04:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.147+0000 7f4f2502b740 10 client.36413 fetch_fsmap learned FSMap version 34 2024-03-27T11:04:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.147+0000 7f4f2502b740 1 -- 172.21.15.97:0/184652274 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55c13c0383c0 con 0x55c13bfb14e0 2024-03-27T11:04:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.148+0000 7f4f04c00640 1 -- 172.21.15.97:0/184652274 <== mon.2 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4f080035f0 con 0x55c13bfb14e0 2024-03-27T11:04:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.148+0000 7f4f04c00640 1 -- 172.21.15.97:0/184652274 <== mon.2 v2:172.21.15.114:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f4f1405e040 con 0x55c13bfb14e0 2024-03-27T11:04:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.148+0000 7f4f2502b740 10 client.36413 fetch_fsmap finished waiting for FSMap version 34 2024-03-27T11:04:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.148+0000 7f4f2502b740 10 client.36413 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T11:04:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.148+0000 7f4f2502b740 20 client.36413 populate_metadata read hostname 'smithi097' 2024-03-27T11:04:36.120 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.148+0000 7f4f2502b740 1 --2- 172.21.15.97:0/184652274 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55c13bfd2af0 0x55c13c03ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:04:36.120 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.148+0000 7f4f2502b740 4 client.36413 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-27T11:04:36.120 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.148+0000 7f4f2502b740 1 -- 172.21.15.97:0/184652274 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c13bf185a0 con 0x55c13bfd2af0 2024-03-27T11:04:36.120 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.148+0000 7f4f1b400640 1 --2- 172.21.15.97:0/184652274 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55c13bfd2af0 0x55c13c03ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:04:36.120 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.149+0000 7f4f1b400640 1 --2- 172.21.15.97:0/184652274 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55c13bfd2af0 0x55c13c03ade0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e649b590c93b2b9f server_cookie=d6c5b6e7c1c303db in_seq=0 out_seq=0 2024-03-27T11:04:36.120 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.149+0000 7f4f04c00640 10 client.36413 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T11:04:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.155+0000 7f4f04c00640 1 -- 172.21.15.97:0/184652274 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c13bf185a0 con 0x55c13bfd2af0 2024-03-27T11:04:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.155+0000 7f4f04c00640 10 client.36413 handle_command_reply: tid=0 2024-03-27T11:04:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.155+0000 7f4f2502b740 1 client.36413 shutdown 2024-03-27T11:04:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.156+0000 7f4f2502b740 1 -- 172.21.15.97:0/184652274 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c13bfb14e0 msgr2=0x55c13c036e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:04:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.156+0000 7f4f2502b740 1 --2- 172.21.15.97:0/184652274 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c13bfb14e0 0x55c13c036e20 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4f140099a0 tx=0x7f4f140041b0 comp rx=0 tx=0).stop 2024-03-27T11:04:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.156+0000 7f4f2502b740 1 -- 172.21.15.97:0/184652274 shutdown_connections 2024-03-27T11:04:36.128 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.156+0000 7f4f2502b740 1 --2- 172.21.15.97:0/184652274 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55c13bfd2af0 0x55c13c03ade0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:36.128 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.156+0000 7f4f2502b740 1 --2- 172.21.15.97:0/184652274 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c13bfb14e0 0x55c13c036e20 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:36.128 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.156+0000 7f4f2502b740 1 --2- 172.21.15.97:0/184652274 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c13bfac370 0x55c13bfba540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:36.128 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.156+0000 7f4f2502b740 1 --2- 172.21.15.97:0/184652274 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c13bfab980 0x55c13bfba000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:36.128 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.156+0000 7f4f2502b740 1 -- 172.21.15.97:0/184652274 >> 172.21.15.97:0/184652274 conn(0x55c13bfa6ef0 msgr2=0x55c13bfad6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:04:36.128 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.156+0000 7f4f04c00640 0 client.36413 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T11:04:36.129 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.156+0000 7f4f2502b740 1 -- 172.21.15.97:0/184652274 shutdown_connections 2024-03-27T11:04:36.129 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.156+0000 7f4f2502b740 1 -- 172.21.15.97:0/184652274 wait complete. 2024-03-27T11:04:36.129 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.157+0000 7f4f2502b740 20 client.36413 trim_cache size 0 max 16384 2024-03-27T11:04:36.129 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-27T11:04:36.129 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-27T11:04:36.129 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-27T11:04:36.129 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-27T11:04:36.130 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.157+0000 7f4f22200640 1 -- 172.21.15.97:0/1058254743 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f4f000a3730 msgr2=0x7f4f000a5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:04:36.130 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.157+0000 7f4f22200640 1 --2- 172.21.15.97:0/1058254743 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f4f000a3730 0x7f4f000a5bf0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f4f14034fd0 tx=0x7f4f14005fd0 comp rx=0 tx=0).stop 2024-03-27T11:04:36.130 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.157+0000 7f4f22200640 1 -- 172.21.15.97:0/1058254743 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4f1c100400 msgr2=0x7f4f1c19e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:04:36.130 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.157+0000 7f4f22200640 1 --2- 172.21.15.97:0/1058254743 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4f1c100400 0x7f4f1c19e260 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c00d950 tx=0x7f4f0c00de10 comp rx=0 tx=0).stop 2024-03-27T11:04:36.130 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.158+0000 7f4f22200640 1 -- 172.21.15.97:0/1058254743 shutdown_connections 2024-03-27T11:04:36.130 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.158+0000 7f4f22200640 1 --2- 172.21.15.97:0/1058254743 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4f1c105720 0x7f4f1c1a5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:36.130 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.158+0000 7f4f22200640 1 --2- 172.21.15.97:0/1058254743 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4f1c102d70 0x7f4f1c19e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:36.131 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.158+0000 7f4f22200640 1 --2- 172.21.15.97:0/1058254743 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f4f000a3730 0x7f4f000a5bf0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:36.131 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.158+0000 7f4f22200640 1 --2- 172.21.15.97:0/1058254743 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4f1c100400 0x7f4f1c19e260 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:36.131 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.158+0000 7f4f22200640 1 -- 172.21.15.97:0/1058254743 >> 172.21.15.97:0/1058254743 conn(0x7f4f1c0fc000 msgr2=0x7f4f1c103a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:04:36.131 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.158+0000 7f4f22200640 1 -- 172.21.15.97:0/1058254743 shutdown_connections 2024-03-27T11:04:36.131 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:36.158+0000 7f4f22200640 1 -- 172.21.15.97:0/1058254743 wait complete. 2024-03-27T11:04:37.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:04:37 smithi177 ceph-mon[35294]: pgmap v680: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 49 MiB/s wr, 784 op/s 2024-03-27T11:04:37.764 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:04:37 smithi114 ceph-mon[35338]: pgmap v680: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 49 MiB/s wr, 784 op/s 2024-03-27T11:04:37.889 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:04:37 smithi097 ceph-mon[31859]: pgmap v680: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 49 MiB/s wr, 784 op/s 2024-03-27T11:04:39.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:04:39 smithi177 ceph-mon[35294]: pgmap v681: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 51 MiB/s wr, 824 op/s 2024-03-27T11:04:39.764 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:04:39 smithi114 ceph-mon[35338]: pgmap v681: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 51 MiB/s wr, 824 op/s 2024-03-27T11:04:39.890 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:04:39 smithi097 ceph-mon[31859]: pgmap v681: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 51 MiB/s wr, 824 op/s 2024-03-27T11:04:41.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:04:41 smithi177 ceph-mon[35294]: pgmap v682: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 55 MiB/s wr, 891 op/s 2024-03-27T11:04:41.764 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:04:41 smithi114 ceph-mon[35338]: pgmap v682: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 55 MiB/s wr, 891 op/s 2024-03-27T11:04:41.890 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:04:41 smithi097 ceph-mon[31859]: pgmap v682: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 55 MiB/s wr, 891 op/s 2024-03-27T11:04:43.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:04:43 smithi177 ceph-mon[35294]: pgmap v683: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 50 MiB/s wr, 805 op/s 2024-03-27T11:04:43.764 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:04:43 smithi114 ceph-mon[35338]: pgmap v683: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 50 MiB/s wr, 805 op/s 2024-03-27T11:04:43.888 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:04:43 smithi097 ceph-mon[31859]: pgmap v683: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 50 MiB/s wr, 805 op/s 2024-03-27T11:04:45.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:04:45 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-27T11:04:45.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:04:45 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-27T11:04:45.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:04:45 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-27T11:04:45.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:04:45 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T11:04:45.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:04:45 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T11:04:45.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:04:45 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T11:04:45.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:04:45 smithi177 ceph-mon[35294]: from='mgr.14152 ' entity='mgr.x' 2024-03-27T11:04:45.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:04:45 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-27T11:04:45.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:04:45 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-27T11:04:45.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:04:45 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-27T11:04:45.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:04:45 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T11:04:45.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:04:45 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T11:04:45.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:04:45 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T11:04:45.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:04:45 smithi114 ceph-mon[35338]: from='mgr.14152 ' entity='mgr.x' 2024-03-27T11:04:45.890 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:04:45 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-27T11:04:45.890 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:04:45 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-27T11:04:45.890 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:04:45 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-27T11:04:45.890 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:04:45 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T11:04:45.893 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:04:45 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T11:04:45.894 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:04:45 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T11:04:45.894 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:04:45 smithi097 ceph-mon[31859]: from='mgr.14152 ' entity='mgr.x' 2024-03-27T11:04:49.382 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.411+0000 7fd475c00640 1 -- 172.21.15.97:0/959935391 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd470108b50 msgr2=0x7fd47010af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:04:49.383 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.411+0000 7fd475c00640 1 --2- 172.21.15.97:0/959935391 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd470108b50 0x7fd47010af80 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd460009ca0 tx=0x7fd46002c100 comp rx=0 tx=0).stop 2024-03-27T11:04:49.383 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.411+0000 7fd475c00640 1 -- 172.21.15.97:0/959935391 shutdown_connections 2024-03-27T11:04:49.383 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.411+0000 7fd475c00640 1 --2- 172.21.15.97:0/959935391 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd47010b940 0x7fd4701147e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:49.383 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.411+0000 7fd475c00640 1 --2- 172.21.15.97:0/959935391 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd47010b4c0 0x7fd470112290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:49.383 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.411+0000 7fd475c00640 1 --2- 172.21.15.97:0/959935391 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd470108b50 0x7fd47010af80 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:49.383 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.411+0000 7fd475c00640 1 -- 172.21.15.97:0/959935391 >> 172.21.15.97:0/959935391 conn(0x7fd47006ed50 msgr2=0x7fd470071190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:04:49.383 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.412+0000 7fd475c00640 1 -- 172.21.15.97:0/959935391 shutdown_connections 2024-03-27T11:04:49.383 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.412+0000 7fd475c00640 1 -- 172.21.15.97:0/959935391 wait complete. 2024-03-27T11:04:49.383 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.413+0000 7fd475c00640 1 Processor -- start 2024-03-27T11:04:49.384 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.414+0000 7fd475c00640 1 -- start start 2024-03-27T11:04:49.385 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.415+0000 7fd475c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd470108b50 0x7fd4701a2700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:04:49.385 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.415+0000 7fd475c00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd47010b4c0 0x7fd4701a2c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:04:49.386 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.415+0000 7fd475c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd47010b940 0x7fd4701a9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:04:49.386 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.415+0000 7fd475c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd470118090 con 0x7fd470108b50 2024-03-27T11:04:49.387 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.415+0000 7fd475c00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd470117f10 con 0x7fd47010b940 2024-03-27T11:04:49.387 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.415+0000 7fd475c00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd470118210 con 0x7fd47010b4c0 2024-03-27T11:04:49.387 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.415+0000 7fd46e000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd47010b4c0 0x7fd4701a2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:04:49.387 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.415+0000 7fd46e000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd47010b4c0 0x7fd4701a2c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.97:49780/0 (socket says 172.21.15.97:49780) 2024-03-27T11:04:49.387 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.415+0000 7fd46e000640 1 -- 172.21.15.97:0/2439872638 learned_addr learned my addr 172.21.15.97:0/2439872638 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:04:49.387 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.416+0000 7fd46f400640 1 --2- 172.21.15.97:0/2439872638 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd47010b940 0x7fd4701a9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:04:49.388 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.416+0000 7fd46f400640 1 -- 172.21.15.97:0/2439872638 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd47010b4c0 msgr2=0x7fd4701a2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:04:49.388 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.416+0000 7fd46f400640 1 --2- 172.21.15.97:0/2439872638 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd47010b4c0 0x7fd4701a2c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:49.388 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.416+0000 7fd46f400640 1 -- 172.21.15.97:0/2439872638 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd470108b50 msgr2=0x7fd4701a2700 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-27T11:04:49.388 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.416+0000 7fd46ea00640 1 --2- 172.21.15.97:0/2439872638 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd470108b50 0x7fd4701a2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:04:49.388 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.416+0000 7fd46f400640 1 --2- 172.21.15.97:0/2439872638 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd470108b50 0x7fd4701a2700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:49.388 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.416+0000 7fd46f400640 1 -- 172.21.15.97:0/2439872638 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4701aa320 con 0x7fd47010b940 2024-03-27T11:04:49.388 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.417+0000 7fd46f400640 1 --2- 172.21.15.97:0/2439872638 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd47010b940 0x7fd4701a9cb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd46800d680 tx=0x7fd46800db40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:04:49.388 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.417+0000 7fd467400640 1 -- 172.21.15.97:0/2439872638 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd468017070 con 0x7fd47010b940 2024-03-27T11:04:49.389 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.418+0000 7fd467400640 1 -- 172.21.15.97:0/2439872638 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd468009820 con 0x7fd47010b940 2024-03-27T11:04:49.389 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.418+0000 7fd467400640 1 -- 172.21.15.97:0/2439872638 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd468004cf0 con 0x7fd47010b940 2024-03-27T11:04:49.389 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.418+0000 7fd475c00640 1 -- 172.21.15.97:0/2439872638 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4701aa610 con 0x7fd47010b940 2024-03-27T11:04:49.389 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.418+0000 7fd475c00640 1 -- 172.21.15.97:0/2439872638 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4701aab10 con 0x7fd47010b940 2024-03-27T11:04:49.391 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.421+0000 7fd4789e5740 1 Processor -- start 2024-03-27T11:04:49.392 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.422+0000 7fd467400640 1 -- 172.21.15.97:0/2439872638 <== mon.2 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7fd468010460 con 0x7fd47010b940 2024-03-27T11:04:49.393 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.422+0000 7fd4789e5740 1 -- start start 2024-03-27T11:04:49.393 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.423+0000 7fd4789e5740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ddfac6a190 0x55ddfac6a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:04:49.394 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.423+0000 7fd4789e5740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ddfac6ab80 0x55ddfac6f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:04:49.394 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.423+0000 7fd4789e5740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ddfac6fcf0 0x55ddfac72180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:04:49.394 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.423+0000 7fd4789e5740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ddfabc6d10 con 0x55ddfac6fcf0 2024-03-27T11:04:49.394 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.423+0000 7fd4789e5740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55ddfab6f260 con 0x55ddfac6a190 2024-03-27T11:04:49.394 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.423+0000 7fd4789e5740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55ddfabc2cc0 con 0x55ddfac6ab80 2024-03-27T11:04:49.395 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.424+0000 7fd46e000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ddfac6a190 0x55ddfac6a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:04:49.395 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.424+0000 7fd46e000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ddfac6a190 0x55ddfac6a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:47006/0 (socket says 172.21.15.97:47006) 2024-03-27T11:04:49.395 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.424+0000 7fd46e000640 1 -- 172.21.15.97:0/3707612981 learned_addr learned my addr 172.21.15.97:0/3707612981 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:04:49.395 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.424+0000 7fd46e000640 1 --2- 172.21.15.97:0/3707612981 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ddfac6fcf0 0x55ddfac72180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:04:49.395 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.424+0000 7fd46ea00640 1 --2- 172.21.15.97:0/3707612981 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ddfac6ab80 0x55ddfac6f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:04:49.395 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.425+0000 7fd467400640 1 --2- 172.21.15.97:0/2439872638 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fd4540a3730 0x7fd4540a5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:04:49.396 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.426+0000 7fd46ea00640 1 -- 172.21.15.97:0/3707612981 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ddfac6a190 msgr2=0x55ddfac6a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:04:49.396 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.426+0000 7fd46ea00640 1 --2- 172.21.15.97:0/3707612981 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ddfac6a190 0x55ddfac6a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:49.396 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.426+0000 7fd46ea00640 1 -- 172.21.15.97:0/3707612981 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ddfac6fcf0 msgr2=0x55ddfac72180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:04:49.396 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.426+0000 7fd46ea00640 1 --2- 172.21.15.97:0/3707612981 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ddfac6fcf0 0x55ddfac72180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:49.396 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.426+0000 7fd46ea00640 1 -- 172.21.15.97:0/3707612981 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddfab66aa0 con 0x55ddfac6ab80 2024-03-27T11:04:49.397 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.426+0000 7fd467400640 1 -- 172.21.15.97:0/2439872638 <== mon.2 v2:172.21.15.114:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fd4680cf570 con 0x7fd47010b940 2024-03-27T11:04:49.397 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.426+0000 7fd46f400640 1 --2- 172.21.15.97:0/2439872638 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fd4540a3730 0x7fd4540a5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:04:49.397 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.426+0000 7fd46ea00640 1 --2- 172.21.15.97:0/3707612981 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ddfac6ab80 0x55ddfac6f680 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd460009bf0 tx=0x7fd4600093f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c02b157f18778951 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:04:49.397 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.427+0000 7fd45fe00640 1 -- 172.21.15.97:0/3707612981 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd460039050 con 0x55ddfac6ab80 2024-03-27T11:04:49.397 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.427+0000 7fd45fe00640 1 -- 172.21.15.97:0/3707612981 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd46003a040 con 0x55ddfac6ab80 2024-03-27T11:04:49.397 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.427+0000 7fd45fe00640 1 -- 172.21.15.97:0/3707612981 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd460004040 con 0x55ddfac6ab80 2024-03-27T11:04:49.398 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.428+0000 7fd4789e5740 1 -- 172.21.15.97:0/3707612981 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ddfac6ab80 msgr2=0x55ddfac6f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:04:49.398 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.428+0000 7fd46f400640 1 --2- 172.21.15.97:0/2439872638 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fd4540a3730 0x7fd4540a5bf0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fd468005d80 tx=0x7fd468005cd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:04:49.398 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.428+0000 7fd4789e5740 1 --2- 172.21.15.97:0/3707612981 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ddfac6ab80 0x55ddfac6f680 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd460009bf0 tx=0x7fd4600093f0 comp rx=0 tx=0).stop 2024-03-27T11:04:49.399 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.428+0000 7fd4789e5740 1 -- 172.21.15.97:0/3707612981 shutdown_connections 2024-03-27T11:04:49.399 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.428+0000 7fd4789e5740 1 --2- 172.21.15.97:0/3707612981 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ddfac6fcf0 0x55ddfac72180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:49.399 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.428+0000 7fd4789e5740 1 --2- 172.21.15.97:0/3707612981 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ddfac6ab80 0x55ddfac6f680 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:49.399 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.428+0000 7fd4789e5740 1 --2- 172.21.15.97:0/3707612981 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ddfac6a190 0x55ddfac6a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:49.399 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.428+0000 7fd4789e5740 1 -- 172.21.15.97:0/3707612981 >> 172.21.15.97:0/3707612981 conn(0x55ddfac65700 msgr2=0x55ddfac65b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:04:49.399 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.428+0000 7fd4789e5740 1 -- 172.21.15.97:0/3707612981 shutdown_connections 2024-03-27T11:04:49.399 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.428+0000 7fd4789e5740 1 -- 172.21.15.97:0/3707612981 wait complete. 2024-03-27T11:04:49.401 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.429+0000 7fd4789e5740 1 -- start start 2024-03-27T11:04:49.401 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.430+0000 7fd4789e5740 1 Processor -- start 2024-03-27T11:04:49.401 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.430+0000 7fd4789e5740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ddfac6a190 0x55ddfac5e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:04:49.401 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.430+0000 7fd4789e5740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ddfac6ab80 0x55ddfac5ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:04:49.401 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.430+0000 7fd4789e5740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ddfac6fcf0 0x55ddfac63be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:04:49.401 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.430+0000 7fd4789e5740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ddfac5cd00 con 0x55ddfac6a190 2024-03-27T11:04:49.401 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.430+0000 7fd4789e5740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55ddfab82630 con 0x55ddfac6fcf0 2024-03-27T11:04:49.401 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.430+0000 7fd4789e5740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55ddfac5ce80 con 0x55ddfac6ab80 2024-03-27T11:04:49.401 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.430+0000 7fd4789e5740 10 client.0 _ll_register_callbacks cb 0x55ddfab87c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T11:04:49.402 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.431+0000 7fd46e000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ddfac6a190 0x55ddfac5e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:04:49.402 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.431+0000 7fd46e000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ddfac6a190 0x55ddfac5e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55046/0 (socket says 172.21.15.97:55046) 2024-03-27T11:04:49.402 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.431+0000 7fd46e000640 1 -- 172.21.15.97:0/2133111512 learned_addr learned my addr 172.21.15.97:0/2133111512 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:04:49.402 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.431+0000 7fd46e000640 1 --2- 172.21.15.97:0/2133111512 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ddfac6fcf0 0x55ddfac63be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:04:49.403 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.431+0000 7fd46ea00640 1 --2- 172.21.15.97:0/2133111512 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ddfac6ab80 0x55ddfac5ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:04:49.403 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.432+0000 7fd46ea00640 1 -- 172.21.15.97:0/2133111512 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ddfac6fcf0 msgr2=0x55ddfac63be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:04:49.404 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.432+0000 7fd46ea00640 1 --2- 172.21.15.97:0/2133111512 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ddfac6fcf0 0x55ddfac63be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:49.404 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.432+0000 7fd46ea00640 1 -- 172.21.15.97:0/2133111512 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ddfac6a190 msgr2=0x55ddfac5e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:04:49.404 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.432+0000 7fd46ea00640 1 --2- 172.21.15.97:0/2133111512 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ddfac6a190 0x55ddfac5e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:49.404 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.432+0000 7fd46ea00640 1 -- 172.21.15.97:0/2133111512 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ddfab8e5f0 con 0x55ddfac6ab80 2024-03-27T11:04:49.404 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.433+0000 7fd46ea00640 1 --2- 172.21.15.97:0/2133111512 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ddfac6ab80 0x55ddfac5ebb0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x55ddfac66670 tx=0x7fd4600327f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4809cb8055f8a37d server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:04:49.404 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.433+0000 7fd45cc00640 10 client.0 ms_handle_connect on v2:172.21.15.177:3300/0 2024-03-27T11:04:49.404 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.433+0000 7fd45cc00640 1 -- 172.21.15.97:0/2133111512 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd46004f070 con 0x55ddfac6ab80 2024-03-27T11:04:49.405 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.434+0000 7fd45cc00640 1 -- 172.21.15.97:0/2133111512 <== mon.1 v2:172.21.15.177:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fd46004b670 con 0x55ddfac6ab80 2024-03-27T11:04:49.405 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.434+0000 7fd4789e5740 1 -- 172.21.15.97:0/2133111512 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd46003a040 con 0x55ddfac6ab80 2024-03-27T11:04:49.405 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.434+0000 7fd45cc00640 1 -- 172.21.15.97:0/2133111512 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fd460032960 con 0x55ddfac6ab80 2024-03-27T11:04:49.405 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.434+0000 7fd4789e5740 10 client.26146 fetch_fsmap learned FSMap version 34 2024-03-27T11:04:49.405 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.435+0000 7fd4789e5740 1 -- 172.21.15.97:0/2133111512 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55ddfa8db210 con 0x55ddfac6ab80 2024-03-27T11:04:49.406 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.435+0000 7fd45cc00640 1 -- 172.21.15.97:0/2133111512 <== mon.1 v2:172.21.15.177:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd46003a040 con 0x55ddfac6ab80 2024-03-27T11:04:49.406 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.435+0000 7fd45cc00640 1 -- 172.21.15.97:0/2133111512 <== mon.1 v2:172.21.15.177:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7fd460040930 con 0x55ddfac6ab80 2024-03-27T11:04:49.406 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.435+0000 7fd4789e5740 10 client.26146 fetch_fsmap finished waiting for FSMap version 34 2024-03-27T11:04:49.406 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.435+0000 7fd4789e5740 10 client.26146 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T11:04:49.406 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.435+0000 7fd4789e5740 20 client.26146 populate_metadata read hostname 'smithi097' 2024-03-27T11:04:49.406 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.436+0000 7fd4789e5740 1 --2- 172.21.15.97:0/2133111512 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55ddfac88770 0x55ddfac8ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:04:49.406 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.436+0000 7fd4789e5740 4 client.26146 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T11:04:49.406 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.436+0000 7fd4789e5740 1 -- 172.21.15.97:0/2133111512 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ddfabd4250 con 0x55ddfac88770 2024-03-27T11:04:49.407 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.436+0000 7fd46e000640 1 --2- 172.21.15.97:0/2133111512 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55ddfac88770 0x55ddfac8ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:04:49.407 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.437+0000 7fd45cc00640 10 client.26146 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T11:04:49.407 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.437+0000 7fd46e000640 1 --2- 172.21.15.97:0/2133111512 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55ddfac88770 0x55ddfac8ab60 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9c32d25d288eed8 server_cookie=6a32c4c62d5fe869 in_seq=0 out_seq=0 2024-03-27T11:04:49.409 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.439+0000 7fd45cc00640 1 -- 172.21.15.97:0/2133111512 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55ddfabd4250 con 0x55ddfac88770 2024-03-27T11:04:49.409 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.439+0000 7fd45cc00640 10 client.26146 handle_command_reply: tid=0 2024-03-27T11:04:49.409 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.439+0000 7fd4789e5740 1 client.26146 shutdown 2024-03-27T11:04:49.410 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.439+0000 7fd4789e5740 1 -- 172.21.15.97:0/2133111512 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ddfac6ab80 msgr2=0x55ddfac5ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:04:49.410 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.439+0000 7fd4789e5740 1 --2- 172.21.15.97:0/2133111512 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ddfac6ab80 0x55ddfac5ebb0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x55ddfac66670 tx=0x7fd4600327f0 comp rx=0 tx=0).stop 2024-03-27T11:04:49.410 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.439+0000 7fd4789e5740 1 -- 172.21.15.97:0/2133111512 shutdown_connections 2024-03-27T11:04:49.410 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.439+0000 7fd4789e5740 1 --2- 172.21.15.97:0/2133111512 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55ddfac88770 0x55ddfac8ab60 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:49.410 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.439+0000 7fd4789e5740 1 --2- 172.21.15.97:0/2133111512 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ddfac6fcf0 0x55ddfac63be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:49.410 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.439+0000 7fd4789e5740 1 --2- 172.21.15.97:0/2133111512 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ddfac6ab80 0x55ddfac5ebb0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:49.410 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.439+0000 7fd4789e5740 1 --2- 172.21.15.97:0/2133111512 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ddfac6a190 0x55ddfac5e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:49.410 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.439+0000 7fd4789e5740 1 -- 172.21.15.97:0/2133111512 >> 172.21.15.97:0/2133111512 conn(0x55ddfac65700 msgr2=0x55ddfac5c0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:04:49.410 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.439+0000 7fd45cc00640 0 client.26146 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T11:04:49.411 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.440+0000 7fd4789e5740 1 -- 172.21.15.97:0/2133111512 shutdown_connections 2024-03-27T11:04:49.411 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.440+0000 7fd4789e5740 1 -- 172.21.15.97:0/2133111512 wait complete. 2024-03-27T11:04:49.411 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.441+0000 7fd4789e5740 20 client.26146 trim_cache size 0 max 16384 2024-03-27T11:04:49.446 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.475+0000 7fd4789e5740 1 Processor -- start 2024-03-27T11:04:49.446 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.476+0000 7fd4789e5740 1 -- start start 2024-03-27T11:04:49.447 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.476+0000 7fd4789e5740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ddfac6a190 0x55ddfac5b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:04:49.447 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.476+0000 7fd4789e5740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ddfac6fcf0 0x55ddfac5bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:04:49.447 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.476+0000 7fd4789e5740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ddfac6c6f0 0x55ddfac5c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:04:49.447 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.476+0000 7fd4789e5740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd4701af3b0 con 0x55ddfac6fcf0 2024-03-27T11:04:49.447 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.476+0000 7fd4789e5740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd4701b0ef0 con 0x55ddfac6a190 2024-03-27T11:04:49.447 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.476+0000 7fd4789e5740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd4701afa80 con 0x55ddfac6c6f0 2024-03-27T11:04:49.447 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.477+0000 7fd46ea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ddfac6fcf0 0x55ddfac5bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:04:49.448 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.477+0000 7fd46ea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ddfac6fcf0 0x55ddfac5bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55048/0 (socket says 172.21.15.97:55048) 2024-03-27T11:04:49.448 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.477+0000 7fd46ea00640 1 -- 172.21.15.97:0/2224532192 learned_addr learned my addr 172.21.15.97:0/2224532192 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:04:49.449 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.477+0000 7fd46e000640 1 --2- 172.21.15.97:0/2224532192 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ddfac6a190 0x55ddfac5b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:04:49.449 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.477+0000 7fd46ea00640 1 -- 172.21.15.97:0/2224532192 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ddfac6c6f0 msgr2=0x55ddfac5c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:04:49.449 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.477+0000 7fd46ea00640 1 --2- 172.21.15.97:0/2224532192 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ddfac6c6f0 0x55ddfac5c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:49.449 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.477+0000 7fd46ea00640 1 -- 172.21.15.97:0/2224532192 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ddfac6a190 msgr2=0x55ddfac5b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:04:49.449 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.478+0000 7fd46ea00640 1 --2- 172.21.15.97:0/2224532192 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ddfac6a190 0x55ddfac5b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:49.449 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.478+0000 7fd46ea00640 1 -- 172.21.15.97:0/2224532192 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddfa9a7620 con 0x55ddfac6fcf0 2024-03-27T11:04:49.449 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.478+0000 7fd46e000640 1 --2- 172.21.15.97:0/2224532192 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ddfac6a190 0x55ddfac5b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T11:04:49.450 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.479+0000 7fd46ea00640 1 --2- 172.21.15.97:0/2224532192 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ddfac6fcf0 0x55ddfac5bda0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fd460005e30 tx=0x7fd4600099c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a10f8dbf8414274 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:04:49.450 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.479+0000 7fd45f400640 1 -- 172.21.15.97:0/2224532192 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd460061070 con 0x55ddfac6fcf0 2024-03-27T11:04:49.450 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.479+0000 7fd45f400640 1 -- 172.21.15.97:0/2224532192 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd460009b20 con 0x55ddfac6fcf0 2024-03-27T11:04:49.450 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.479+0000 7fd45f400640 1 -- 172.21.15.97:0/2224532192 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd460039070 con 0x55ddfac6fcf0 2024-03-27T11:04:49.451 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.481+0000 7fd4789e5740 1 -- 172.21.15.97:0/2224532192 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ddfac6fcf0 msgr2=0x55ddfac5bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:04:49.451 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.481+0000 7fd4789e5740 1 --2- 172.21.15.97:0/2224532192 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ddfac6fcf0 0x55ddfac5bda0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fd460005e30 tx=0x7fd4600099c0 comp rx=0 tx=0).stop 2024-03-27T11:04:49.451 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.481+0000 7fd4789e5740 1 -- 172.21.15.97:0/2224532192 shutdown_connections 2024-03-27T11:04:49.451 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.481+0000 7fd4789e5740 1 --2- 172.21.15.97:0/2224532192 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ddfac6c6f0 0x55ddfac5c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:49.451 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.481+0000 7fd4789e5740 1 --2- 172.21.15.97:0/2224532192 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ddfac6fcf0 0x55ddfac5bda0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:49.451 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.481+0000 7fd4789e5740 1 --2- 172.21.15.97:0/2224532192 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ddfac6a190 0x55ddfac5b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:49.452 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.481+0000 7fd4789e5740 1 -- 172.21.15.97:0/2224532192 >> 172.21.15.97:0/2224532192 conn(0x55ddfac65700 msgr2=0x55ddfac6bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:04:49.452 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.481+0000 7fd4789e5740 1 -- 172.21.15.97:0/2224532192 shutdown_connections 2024-03-27T11:04:49.452 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.481+0000 7fd4789e5740 1 -- 172.21.15.97:0/2224532192 wait complete. 2024-03-27T11:04:49.452 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.482+0000 7fd4789e5740 1 -- start start 2024-03-27T11:04:49.452 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.482+0000 7fd4789e5740 1 Processor -- start 2024-03-27T11:04:49.453 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.482+0000 7fd4789e5740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ddfac6a190 0x55ddfac9b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:04:49.453 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.482+0000 7fd4789e5740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ddfac6fcf0 0x55ddfac99e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:04:49.453 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.483+0000 7fd4789e5740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ddfac9a3b0 0x55ddfac9a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:04:49.453 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.483+0000 7fd4789e5740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd4701b0820 con 0x55ddfac6fcf0 2024-03-27T11:04:49.453 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.483+0000 7fd4789e5740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd4701b0150 con 0x55ddfac6a190 2024-03-27T11:04:49.453 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.483+0000 7fd4789e5740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd4701ad350 con 0x55ddfac9a3b0 2024-03-27T11:04:49.453 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.483+0000 7fd4789e5740 10 client.0 _ll_register_callbacks cb 0x7fd45800bbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T11:04:49.454 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.484+0000 7fd46e000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ddfac6a190 0x55ddfac9b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:04:49.454 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.484+0000 7fd46e000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ddfac6a190 0x55ddfac9b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:47050/0 (socket says 172.21.15.97:47050) 2024-03-27T11:04:49.454 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.484+0000 7fd46e000640 1 -- 172.21.15.97:0/2915692178 learned_addr learned my addr 172.21.15.97:0/2915692178 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:04:49.454 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.484+0000 7fd46e000640 1 --2- 172.21.15.97:0/2915692178 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ddfac9a3b0 0x55ddfac9a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:04:49.454 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.484+0000 7fd46ea00640 1 --2- 172.21.15.97:0/2915692178 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ddfac6fcf0 0x55ddfac99e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:04:49.455 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.484+0000 7fd46e000640 1 -- 172.21.15.97:0/2915692178 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ddfac9a3b0 msgr2=0x55ddfac9a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:04:49.455 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.484+0000 7fd46e000640 1 --2- 172.21.15.97:0/2915692178 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ddfac9a3b0 0x55ddfac9a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:49.455 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.484+0000 7fd46e000640 1 -- 172.21.15.97:0/2915692178 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ddfac6fcf0 msgr2=0x55ddfac99e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:04:49.455 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.484+0000 7fd46e000640 1 --2- 172.21.15.97:0/2915692178 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ddfac6fcf0 0x55ddfac99e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:49.455 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.484+0000 7fd46e000640 1 -- 172.21.15.97:0/2915692178 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ddfacee2c0 con 0x55ddfac6a190 2024-03-27T11:04:49.455 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.485+0000 7fd46e000640 1 --2- 172.21.15.97:0/2915692178 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ddfac6a190 0x55ddfac9b860 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd458004700 tx=0x7fd458014040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=791d9286a4893481 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:04:49.456 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.485+0000 7fd45cc00640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-03-27T11:04:49.456 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.485+0000 7fd45cc00640 1 -- 172.21.15.97:0/2915692178 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd458004220 con 0x55ddfac6a190 2024-03-27T11:04:49.456 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.485+0000 7fd45cc00640 1 -- 172.21.15.97:0/2915692178 <== mon.2 v2:172.21.15.114:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fd4580043c0 con 0x55ddfac6a190 2024-03-27T11:04:49.456 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.485+0000 7fd4789e5740 1 -- 172.21.15.97:0/2915692178 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd460009b20 con 0x55ddfac6a190 2024-03-27T11:04:49.456 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.486+0000 7fd45cc00640 1 -- 172.21.15.97:0/2915692178 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fd458010e90 con 0x55ddfac6a190 2024-03-27T11:04:49.457 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.486+0000 7fd4789e5740 10 client.36425 fetch_fsmap learned FSMap version 34 2024-03-27T11:04:49.457 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.486+0000 7fd4789e5740 1 -- 172.21.15.97:0/2915692178 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55ddfacef5c0 con 0x55ddfac6a190 2024-03-27T11:04:49.457 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.486+0000 7fd45cc00640 1 -- 172.21.15.97:0/2915692178 <== mon.2 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd460009b20 con 0x55ddfac6a190 2024-03-27T11:04:49.457 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.487+0000 7fd45cc00640 1 -- 172.21.15.97:0/2915692178 <== mon.2 v2:172.21.15.114:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7fd4580054d0 con 0x55ddfac6a190 2024-03-27T11:04:49.458 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.487+0000 7fd4789e5740 10 client.36425 fetch_fsmap finished waiting for FSMap version 34 2024-03-27T11:04:49.458 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.487+0000 7fd4789e5740 10 client.36425 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T11:04:49.458 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.487+0000 7fd4789e5740 20 client.36425 populate_metadata read hostname 'smithi097' 2024-03-27T11:04:49.458 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.488+0000 7fd4789e5740 1 --2- 172.21.15.97:0/2915692178 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55ddfacefec0 0x55ddfacf22b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:04:49.458 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.488+0000 7fd4789e5740 4 client.36425 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-27T11:04:49.458 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.488+0000 7fd4789e5740 1 -- 172.21.15.97:0/2915692178 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ddfabce5a0 con 0x55ddfacefec0 2024-03-27T11:04:49.458 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.488+0000 7fd46ea00640 1 --2- 172.21.15.97:0/2915692178 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55ddfacefec0 0x55ddfacf22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:04:49.459 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.489+0000 7fd45cc00640 10 client.36425 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T11:04:49.459 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.489+0000 7fd46ea00640 1 --2- 172.21.15.97:0/2915692178 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55ddfacefec0 0x55ddfacf22b0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91727ca3a0e2127f server_cookie=4cd17b8666c4ddc6 in_seq=0 out_seq=0 2024-03-27T11:04:49.464 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.493+0000 7fd45cc00640 1 -- 172.21.15.97:0/2915692178 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ddfabce5a0 con 0x55ddfacefec0 2024-03-27T11:04:49.464 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.493+0000 7fd45cc00640 10 client.36425 handle_command_reply: tid=0 2024-03-27T11:04:49.464 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.493+0000 7fd4789e5740 1 client.36425 shutdown 2024-03-27T11:04:49.464 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.494+0000 7fd4789e5740 1 -- 172.21.15.97:0/2915692178 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ddfac6a190 msgr2=0x55ddfac9b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:04:49.464 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.494+0000 7fd4789e5740 1 --2- 172.21.15.97:0/2915692178 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ddfac6a190 0x55ddfac9b860 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd458004700 tx=0x7fd458014040 comp rx=0 tx=0).stop 2024-03-27T11:04:49.465 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.494+0000 7fd4789e5740 1 -- 172.21.15.97:0/2915692178 shutdown_connections 2024-03-27T11:04:49.465 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.494+0000 7fd4789e5740 1 --2- 172.21.15.97:0/2915692178 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55ddfacefec0 0x55ddfacf22b0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:49.465 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.494+0000 7fd4789e5740 1 --2- 172.21.15.97:0/2915692178 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ddfac9a3b0 0x55ddfac9a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:49.465 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.494+0000 7fd4789e5740 1 --2- 172.21.15.97:0/2915692178 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ddfac6fcf0 0x55ddfac99e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:49.465 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.494+0000 7fd4789e5740 1 --2- 172.21.15.97:0/2915692178 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ddfac6a190 0x55ddfac9b860 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:49.465 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.494+0000 7fd4789e5740 1 -- 172.21.15.97:0/2915692178 >> 172.21.15.97:0/2915692178 conn(0x55ddfac65700 msgr2=0x55ddfac68960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:04:49.465 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.494+0000 7fd45cc00640 0 client.36425 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T11:04:49.466 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.495+0000 7fd4789e5740 1 -- 172.21.15.97:0/2915692178 shutdown_connections 2024-03-27T11:04:49.466 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.495+0000 7fd4789e5740 1 -- 172.21.15.97:0/2915692178 wait complete. 2024-03-27T11:04:49.466 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.495+0000 7fd4789e5740 20 client.36425 trim_cache size 0 max 16384 2024-03-27T11:04:49.466 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-27T11:04:49.466 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-27T11:04:49.466 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-27T11:04:49.467 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-27T11:04:49.467 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.497+0000 7fd475c00640 1 -- 172.21.15.97:0/2439872638 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fd4540a3730 msgr2=0x7fd4540a5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:04:49.468 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.497+0000 7fd475c00640 1 --2- 172.21.15.97:0/2439872638 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fd4540a3730 0x7fd4540a5bf0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fd468005d80 tx=0x7fd468005cd0 comp rx=0 tx=0).stop 2024-03-27T11:04:49.468 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.498+0000 7fd475c00640 1 -- 172.21.15.97:0/2439872638 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd47010b940 msgr2=0x7fd4701a9cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:04:49.469 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.498+0000 7fd475c00640 1 --2- 172.21.15.97:0/2439872638 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd47010b940 0x7fd4701a9cb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd46800d680 tx=0x7fd46800db40 comp rx=0 tx=0).stop 2024-03-27T11:04:49.469 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.498+0000 7fd475c00640 1 -- 172.21.15.97:0/2439872638 shutdown_connections 2024-03-27T11:04:49.469 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.498+0000 7fd475c00640 1 --2- 172.21.15.97:0/2439872638 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fd4540a3730 0x7fd4540a5bf0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:49.469 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.498+0000 7fd475c00640 1 --2- 172.21.15.97:0/2439872638 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd47010b940 0x7fd4701a9cb0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:49.469 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.498+0000 7fd475c00640 1 --2- 172.21.15.97:0/2439872638 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd47010b4c0 0x7fd4701a2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:49.469 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.498+0000 7fd475c00640 1 --2- 172.21.15.97:0/2439872638 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd470108b50 0x7fd4701a2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:04:49.469 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.498+0000 7fd475c00640 1 -- 172.21.15.97:0/2439872638 >> 172.21.15.97:0/2439872638 conn(0x7fd47006ed50 msgr2=0x7fd47006f690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:04:49.469 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.499+0000 7fd475c00640 1 -- 172.21.15.97:0/2439872638 shutdown_connections 2024-03-27T11:04:49.470 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:04:49.499+0000 7fd475c00640 1 -- 172.21.15.97:0/2439872638 wait complete. 2024-03-27T11:04:49.638 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:04:49 smithi097 ceph-mon[31859]: pgmap v685: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 49 MiB/s wr, 792 op/s 2024-03-27T11:04:49.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:04:49 smithi097 ceph-mon[31859]: pgmap v686: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 52 MiB/s wr, 840 op/s 2024-03-27T11:04:49.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:04:49 smithi177 ceph-mon[35294]: pgmap v685: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 49 MiB/s wr, 792 op/s 2024-03-27T11:04:49.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:04:49 smithi177 ceph-mon[35294]: pgmap v686: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 52 MiB/s wr, 840 op/s 2024-03-27T11:04:49.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:04:49 smithi114 ceph-mon[35338]: pgmap v685: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 49 MiB/s wr, 792 op/s 2024-03-27T11:04:49.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:04:49 smithi114 ceph-mon[35338]: pgmap v686: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 52 MiB/s wr, 840 op/s 2024-03-27T11:04:51.638 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:04:51 smithi097 ceph-mon[31859]: pgmap v687: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 56 MiB/s wr, 901 op/s 2024-03-27T11:04:51.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:04:51 smithi177 ceph-mon[35294]: pgmap v687: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 56 MiB/s wr, 901 op/s 2024-03-27T11:04:51.764 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:04:51 smithi114 ceph-mon[35338]: pgmap v687: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 56 MiB/s wr, 901 op/s 2024-03-27T11:04:53.640 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:04:53 smithi097 ceph-mon[31859]: pgmap v688: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 51 MiB/s wr, 815 op/s 2024-03-27T11:04:53.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:04:53 smithi177 ceph-mon[35294]: pgmap v688: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 51 MiB/s wr, 815 op/s 2024-03-27T11:04:53.764 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:04:53 smithi114 ceph-mon[35338]: pgmap v688: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 51 MiB/s wr, 815 op/s 2024-03-27T11:04:55.638 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:04:55 smithi097 ceph-mon[31859]: pgmap v689: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 51 MiB/s wr, 824 op/s 2024-03-27T11:04:55.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:04:55 smithi177 ceph-mon[35294]: pgmap v689: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 51 MiB/s wr, 824 op/s 2024-03-27T11:04:55.764 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:04:55 smithi114 ceph-mon[35338]: pgmap v689: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 51 MiB/s wr, 824 op/s 2024-03-27T11:04:57.638 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:04:57 smithi097 ceph-mon[31859]: pgmap v690: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 49 MiB/s wr, 795 op/s 2024-03-27T11:04:57.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:04:57 smithi177 ceph-mon[35294]: pgmap v690: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 49 MiB/s wr, 795 op/s 2024-03-27T11:04:57.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:04:57 smithi114 ceph-mon[35338]: pgmap v690: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 49 MiB/s wr, 795 op/s 2024-03-27T11:04:59.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:04:59 smithi097 ceph-mon[31859]: pgmap v691: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 52 MiB/s wr, 833 op/s 2024-03-27T11:04:59.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:04:59 smithi177 ceph-mon[35294]: pgmap v691: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 52 MiB/s wr, 833 op/s 2024-03-27T11:04:59.764 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:04:59 smithi114 ceph-mon[35338]: pgmap v691: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 52 MiB/s wr, 833 op/s 2024-03-27T11:05:01.638 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:05:01 smithi097 ceph-mon[31859]: pgmap v692: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 55 MiB/s wr, 895 op/s 2024-03-27T11:05:01.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:05:01 smithi177 ceph-mon[35294]: pgmap v692: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 55 MiB/s wr, 895 op/s 2024-03-27T11:05:01.764 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:05:01 smithi114 ceph-mon[35338]: pgmap v692: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 55 MiB/s wr, 895 op/s 2024-03-27T11:05:02.698 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.727+0000 7f7a15800640 1 -- 172.21.15.97:0/1470159179 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7a10107680 msgr2=0x7f7a10109ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:05:02.698 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.727+0000 7f7a15800640 1 --2- 172.21.15.97:0/1470159179 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7a10107680 0x7f7a10109ad0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7a08009be0 tx=0x7f7a0802bf10 comp rx=0 tx=0).stop 2024-03-27T11:05:02.698 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.727+0000 7f7a15800640 1 -- 172.21.15.97:0/1470159179 shutdown_connections 2024-03-27T11:05:02.698 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.727+0000 7f7a15800640 1 --2- 172.21.15.97:0/1470159179 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7a10107680 0x7f7a10109ad0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:02.698 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.727+0000 7f7a15800640 1 --2- 172.21.15.97:0/1470159179 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7a10104d50 0x7f7a10107140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:02.698 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.727+0000 7f7a15800640 1 --2- 172.21.15.97:0/1470159179 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a10102420 0x7f7a10104810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:02.698 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.727+0000 7f7a15800640 1 -- 172.21.15.97:0/1470159179 >> 172.21.15.97:0/1470159179 conn(0x7f7a100fbe30 msgr2=0x7f7a100fe250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:05:02.698 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.728+0000 7f7a15800640 1 -- 172.21.15.97:0/1470159179 shutdown_connections 2024-03-27T11:05:02.699 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.728+0000 7f7a15800640 1 -- 172.21.15.97:0/1470159179 wait complete. 2024-03-27T11:05:02.699 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.729+0000 7f7a15800640 1 Processor -- start 2024-03-27T11:05:02.699 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.729+0000 7f7a15800640 1 -- start start 2024-03-27T11:05:02.699 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.729+0000 7f7a15800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7a10102420 0x7f7a10196570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:05:02.700 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.729+0000 7f7a15800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7a10104d50 0x7f7a10196ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:05:02.700 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.730+0000 7f7a15800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a10107680 0x7f7a101930e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:05:02.700 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.730+0000 7f7a15800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7a1010d520 con 0x7f7a10107680 2024-03-27T11:05:02.700 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.730+0000 7f7a15800640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f7a1010d3a0 con 0x7f7a10104d50 2024-03-27T11:05:02.700 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.730+0000 7f7a15800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f7a1010d6a0 con 0x7f7a10102420 2024-03-27T11:05:02.701 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.730+0000 7f7a0fe00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7a10102420 0x7f7a10196570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:05:02.701 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.730+0000 7f7a0fe00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7a10102420 0x7f7a10196570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.97:51496/0 (socket says 172.21.15.97:51496) 2024-03-27T11:05:02.701 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.730+0000 7f7a0fe00640 1 -- 172.21.15.97:0/3372037475 learned_addr learned my addr 172.21.15.97:0/3372037475 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:05:02.701 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.730+0000 7f7a0f400640 1 --2- 172.21.15.97:0/3372037475 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7a10104d50 0x7f7a10196ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:05:02.701 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.730+0000 7f7a14e00640 1 --2- 172.21.15.97:0/3372037475 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a10107680 0x7f7a101930e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:05:02.702 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.730+0000 7f7a0f400640 1 -- 172.21.15.97:0/3372037475 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7a10102420 msgr2=0x7f7a10196570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:05:02.702 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.730+0000 7f7a0f400640 1 --2- 172.21.15.97:0/3372037475 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7a10102420 0x7f7a10196570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:02.702 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.731+0000 7f7a0f400640 1 -- 172.21.15.97:0/3372037475 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a10107680 msgr2=0x7f7a101930e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:05:02.702 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.731+0000 7f7a0f400640 1 --2- 172.21.15.97:0/3372037475 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a10107680 0x7f7a101930e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:02.702 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.731+0000 7f7a0f400640 1 -- 172.21.15.97:0/3372037475 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a101939a0 con 0x7f7a10104d50 2024-03-27T11:05:02.702 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.731+0000 7f7a14e00640 1 --2- 172.21.15.97:0/3372037475 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a10107680 0x7f7a101930e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T11:05:02.702 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.731+0000 7f7a0f400640 1 --2- 172.21.15.97:0/3372037475 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7a10104d50 0x7f7a10196ab0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f79fc005fd0 tx=0x7f79fc00ec20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:05:02.702 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.731+0000 7f7a0cc00640 1 -- 172.21.15.97:0/3372037475 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79fc00ccc0 con 0x7f7a10104d50 2024-03-27T11:05:02.702 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.732+0000 7f7a0cc00640 1 -- 172.21.15.97:0/3372037475 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f79fc00ee70 con 0x7f7a10104d50 2024-03-27T11:05:02.703 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.732+0000 7f7a15800640 1 -- 172.21.15.97:0/3372037475 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a101a61e0 con 0x7f7a10104d50 2024-03-27T11:05:02.703 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.732+0000 7f7a0cc00640 1 -- 172.21.15.97:0/3372037475 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79fc018750 con 0x7f7a10104d50 2024-03-27T11:05:02.703 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.732+0000 7f7a15800640 1 -- 172.21.15.97:0/3372037475 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a101a6640 con 0x7f7a10104d50 2024-03-27T11:05:02.707 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.736+0000 7f7a0cc00640 1 -- 172.21.15.97:0/3372037475 <== mon.2 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f79fc010430 con 0x7f7a10104d50 2024-03-27T11:05:02.707 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.737+0000 7f7a19a3b740 1 Processor -- start 2024-03-27T11:05:02.707 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.737+0000 7f7a19a3b740 1 -- start start 2024-03-27T11:05:02.708 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.738+0000 7f7a19a3b740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b81fdfdb60 0x55b81fdfdf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:05:02.708 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.738+0000 7f7a19a3b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b81fdfe550 0x55b81fe03050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:05:02.708 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.738+0000 7f7a19a3b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b81fe036c0 0x55b81fe05b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:05:02.708 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.738+0000 7f7a19a3b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55b81fd62d10 con 0x55b81fe036c0 2024-03-27T11:05:02.708 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.738+0000 7f7a19a3b740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55b81fd0b260 con 0x55b81fdfe550 2024-03-27T11:05:02.708 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.738+0000 7f7a19a3b740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55b81fd5ecc0 con 0x55b81fdfdb60 2024-03-27T11:05:02.709 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.738+0000 7f7a0cc00640 1 --2- 172.21.15.97:0/3372037475 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f79f40a3680 0x7f79f40a5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:05:02.709 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.738+0000 7f7a0cc00640 1 -- 172.21.15.97:0/3372037475 <== mon.2 v2:172.21.15.114:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f79fc0cfec0 con 0x7f7a10104d50 2024-03-27T11:05:02.709 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.739+0000 7f7a14e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b81fdfdb60 0x55b81fdfdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:05:02.709 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.739+0000 7f7a0fe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b81fdfe550 0x55b81fe03050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:05:02.709 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.739+0000 7f7a0fe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b81fdfe550 0x55b81fe03050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:33954/0 (socket says 172.21.15.97:33954) 2024-03-27T11:05:02.710 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.739+0000 7f7a0fe00640 1 -- 172.21.15.97:0/3671531666 learned_addr learned my addr 172.21.15.97:0/3671531666 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:05:02.710 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.739+0000 7f7a0f400640 1 --2- 172.21.15.97:0/3671531666 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b81fe036c0 0x55b81fe05b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:05:02.710 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.739+0000 7f7a14e00640 1 --2- 172.21.15.97:0/3372037475 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f79f40a3680 0x7f79f40a5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:05:02.710 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.739+0000 7f7a0f400640 1 -- 172.21.15.97:0/3671531666 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b81fdfdb60 msgr2=0x55b81fdfdf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:05:02.710 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.739+0000 7f7a0f400640 1 --2- 172.21.15.97:0/3671531666 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b81fdfdb60 0x55b81fdfdf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:02.711 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.739+0000 7f7a0f400640 1 -- 172.21.15.97:0/3671531666 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b81fdfe550 msgr2=0x55b81fe03050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:05:02.711 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.739+0000 7f7a0f400640 1 --2- 172.21.15.97:0/3671531666 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b81fdfe550 0x55b81fe03050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:02.711 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.739+0000 7f7a0f400640 1 -- 172.21.15.97:0/3671531666 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b81fd02aa0 con 0x55b81fe036c0 2024-03-27T11:05:02.711 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.739+0000 7f7a14e00640 1 --2- 172.21.15.97:0/3671531666 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b81fdfdb60 0x55b81fdfdf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T11:05:02.711 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.740+0000 7f7a0f400640 1 --2- 172.21.15.97:0/3671531666 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b81fe036c0 0x55b81fe05b50 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f79fc00c6a0 tx=0x7f79fc00ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90261397270a4e0d server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:05:02.711 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.740+0000 7f7a14e00640 1 --2- 172.21.15.97:0/3372037475 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f79f40a3680 0x7f79f40a5b40 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f7a08009370 tx=0x7f7a08005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:05:02.711 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.740+0000 7f7a05600640 1 -- 172.21.15.97:0/3671531666 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79fc09c3a0 con 0x55b81fe036c0 2024-03-27T11:05:02.711 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.740+0000 7f7a05600640 1 -- 172.21.15.97:0/3671531666 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f79fc0a0750 con 0x55b81fe036c0 2024-03-27T11:05:02.711 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.740+0000 7f7a05600640 1 -- 172.21.15.97:0/3671531666 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79fc0980f0 con 0x55b81fe036c0 2024-03-27T11:05:02.711 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.741+0000 7f7a19a3b740 1 -- 172.21.15.97:0/3671531666 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b81fe036c0 msgr2=0x55b81fe05b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:05:02.712 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.741+0000 7f7a19a3b740 1 --2- 172.21.15.97:0/3671531666 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b81fe036c0 0x55b81fe05b50 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f79fc00c6a0 tx=0x7f79fc00ce60 comp rx=0 tx=0).stop 2024-03-27T11:05:02.712 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.742+0000 7f7a19a3b740 1 -- 172.21.15.97:0/3671531666 shutdown_connections 2024-03-27T11:05:02.712 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.742+0000 7f7a19a3b740 1 --2- 172.21.15.97:0/3671531666 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b81fe036c0 0x55b81fe05b50 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:02.712 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.742+0000 7f7a19a3b740 1 --2- 172.21.15.97:0/3671531666 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b81fdfe550 0x55b81fe03050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:02.712 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.742+0000 7f7a19a3b740 1 --2- 172.21.15.97:0/3671531666 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b81fdfdb60 0x55b81fdfdf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:02.712 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.742+0000 7f7a19a3b740 1 -- 172.21.15.97:0/3671531666 >> 172.21.15.97:0/3671531666 conn(0x55b81fdf90d0 msgr2=0x55b81fdf94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:05:02.713 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.742+0000 7f7a19a3b740 1 -- 172.21.15.97:0/3671531666 shutdown_connections 2024-03-27T11:05:02.713 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.742+0000 7f7a19a3b740 1 -- 172.21.15.97:0/3671531666 wait complete. 2024-03-27T11:05:02.713 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.743+0000 7f7a19a3b740 1 -- start start 2024-03-27T11:05:02.713 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.743+0000 7f7a19a3b740 1 Processor -- start 2024-03-27T11:05:02.714 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.744+0000 7f7a19a3b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b81fdfdb60 0x55b81fe07e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:05:02.715 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.744+0000 7f7a19a3b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b81fdfe550 0x55b81fe083b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:05:02.715 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.744+0000 7f7a19a3b740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b81fe036c0 0x55b81fe0f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:05:02.715 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.744+0000 7f7a19a3b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55b81fe06610 con 0x55b81fdfe550 2024-03-27T11:05:02.715 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.744+0000 7f7a19a3b740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55b81fd1e630 con 0x55b81fdfdb60 2024-03-27T11:05:02.715 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.744+0000 7f7a19a3b740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55b81fe06790 con 0x55b81fe036c0 2024-03-27T11:05:02.715 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.744+0000 7f7a19a3b740 10 client.0 _ll_register_callbacks cb 0x55b81fd23c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T11:05:02.715 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.744+0000 7f7a0fe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b81fdfdb60 0x55b81fe07e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:05:02.715 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.744+0000 7f7a0fe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b81fdfdb60 0x55b81fe07e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:33970/0 (socket says 172.21.15.97:33970) 2024-03-27T11:05:02.715 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.744+0000 7f7a0fe00640 1 -- 172.21.15.97:0/3901957491 learned_addr learned my addr 172.21.15.97:0/3901957491 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:05:02.716 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.745+0000 7f7a14e00640 1 --2- 172.21.15.97:0/3901957491 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b81fe036c0 0x55b81fe0f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:05:02.716 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.745+0000 7f7a0fe00640 1 -- 172.21.15.97:0/3901957491 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b81fe036c0 msgr2=0x55b81fe0f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:05:02.716 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.745+0000 7f7a0f400640 1 --2- 172.21.15.97:0/3901957491 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b81fdfe550 0x55b81fe083b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:05:02.716 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.745+0000 7f7a0fe00640 1 --2- 172.21.15.97:0/3901957491 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b81fe036c0 0x55b81fe0f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:02.717 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.745+0000 7f7a0fe00640 1 -- 172.21.15.97:0/3901957491 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b81fdfe550 msgr2=0x55b81fe083b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:05:02.717 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.745+0000 7f7a0fe00640 1 --2- 172.21.15.97:0/3901957491 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b81fdfe550 0x55b81fe083b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:02.717 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.745+0000 7f7a0fe00640 1 -- 172.21.15.97:0/3901957491 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b81fd2a5f0 con 0x55b81fdfdb60 2024-03-27T11:05:02.717 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.745+0000 7f7a0f400640 1 --2- 172.21.15.97:0/3901957491 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b81fdfe550 0x55b81fe083b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T11:05:02.717 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.745+0000 7f7a14e00640 1 --2- 172.21.15.97:0/3901957491 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b81fe036c0 0x55b81fe0f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T11:05:02.717 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.745+0000 7f7a0fe00640 1 --2- 172.21.15.97:0/3901957491 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b81fdfdb60 0x55b81fe07e70 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7a00012700 tx=0x7f7a00012bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae11efcf7ee94039 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:05:02.717 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.745+0000 7f79fa000640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-03-27T11:05:02.717 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.746+0000 7f79fa000640 1 -- 172.21.15.97:0/3901957491 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a00004050 con 0x55b81fdfdb60 2024-03-27T11:05:02.717 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.746+0000 7f79fa000640 1 -- 172.21.15.97:0/3901957491 <== mon.2 v2:172.21.15.114:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f7a000041f0 con 0x55b81fdfdb60 2024-03-27T11:05:02.717 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.746+0000 7f7a19a3b740 1 -- 172.21.15.97:0/3901957491 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79fc0a0750 con 0x55b81fdfdb60 2024-03-27T11:05:02.717 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.746+0000 7f79fa000640 1 -- 172.21.15.97:0/3901957491 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f7a000043f0 con 0x55b81fdfdb60 2024-03-27T11:05:02.718 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.747+0000 7f7a19a3b740 10 client.36437 fetch_fsmap learned FSMap version 34 2024-03-27T11:05:02.718 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.747+0000 7f7a19a3b740 1 -- 172.21.15.97:0/3901957491 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55b81fa77210 con 0x55b81fdfdb60 2024-03-27T11:05:02.718 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.747+0000 7f79fa000640 1 -- 172.21.15.97:0/3901957491 <== mon.2 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f79fc0a0750 con 0x55b81fdfdb60 2024-03-27T11:05:02.719 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.747+0000 7f79fa000640 1 -- 172.21.15.97:0/3901957491 <== mon.2 v2:172.21.15.114:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f7a00003ab0 con 0x55b81fdfdb60 2024-03-27T11:05:02.719 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.747+0000 7f7a19a3b740 10 client.36437 fetch_fsmap finished waiting for FSMap version 34 2024-03-27T11:05:02.719 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.747+0000 7f7a19a3b740 10 client.36437 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T11:05:02.719 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.747+0000 7f7a19a3b740 20 client.36437 populate_metadata read hostname 'smithi097' 2024-03-27T11:05:02.719 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.747+0000 7f7a19a3b740 1 --2- 172.21.15.97:0/3901957491 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55b81fe24cb0 0x55b81fe27140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:05:02.719 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.747+0000 7f7a19a3b740 4 client.36437 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T11:05:02.719 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.747+0000 7f7a19a3b740 1 -- 172.21.15.97:0/3901957491 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b81fd70250 con 0x55b81fe24cb0 2024-03-27T11:05:02.719 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.748+0000 7f7a0f400640 1 --2- 172.21.15.97:0/3901957491 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55b81fe24cb0 0x55b81fe27140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:05:02.719 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.748+0000 7f79fa000640 10 client.36437 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T11:05:02.719 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.749+0000 7f7a0f400640 1 --2- 172.21.15.97:0/3901957491 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55b81fe24cb0 0x55b81fe27140 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5cf53bb85cae490 server_cookie=95304490d30c697d in_seq=0 out_seq=0 2024-03-27T11:05:02.720 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.750+0000 7f79fa000640 1 -- 172.21.15.97:0/3901957491 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55b81fd70250 con 0x55b81fe24cb0 2024-03-27T11:05:02.720 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.750+0000 7f79fa000640 10 client.36437 handle_command_reply: tid=0 2024-03-27T11:05:02.721 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.750+0000 7f7a19a3b740 1 client.36437 shutdown 2024-03-27T11:05:02.721 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.751+0000 7f7a19a3b740 1 -- 172.21.15.97:0/3901957491 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b81fdfdb60 msgr2=0x55b81fe07e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:05:02.721 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.751+0000 7f7a19a3b740 1 --2- 172.21.15.97:0/3901957491 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b81fdfdb60 0x55b81fe07e70 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7a00012700 tx=0x7f7a00012bc0 comp rx=0 tx=0).stop 2024-03-27T11:05:02.722 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.751+0000 7f7a19a3b740 1 -- 172.21.15.97:0/3901957491 shutdown_connections 2024-03-27T11:05:02.722 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.751+0000 7f7a19a3b740 1 --2- 172.21.15.97:0/3901957491 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55b81fe24cb0 0x55b81fe27140 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:02.722 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.751+0000 7f7a19a3b740 1 --2- 172.21.15.97:0/3901957491 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b81fe036c0 0x55b81fe0f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:02.722 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.751+0000 7f7a19a3b740 1 --2- 172.21.15.97:0/3901957491 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b81fdfe550 0x55b81fe083b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:02.722 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.751+0000 7f7a19a3b740 1 --2- 172.21.15.97:0/3901957491 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b81fdfdb60 0x55b81fe07e70 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:02.722 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.751+0000 7f7a19a3b740 1 -- 172.21.15.97:0/3901957491 >> 172.21.15.97:0/3901957491 conn(0x55b81fdf90d0 msgr2=0x55b81fdf8310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:05:02.722 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.751+0000 7f79fa000640 0 client.36437 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T11:05:02.722 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.751+0000 7f7a19a3b740 1 -- 172.21.15.97:0/3901957491 shutdown_connections 2024-03-27T11:05:02.722 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.752+0000 7f7a19a3b740 1 -- 172.21.15.97:0/3901957491 wait complete. 2024-03-27T11:05:02.723 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.752+0000 7f7a19a3b740 20 client.36437 trim_cache size 0 max 16384 2024-03-27T11:05:02.745 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.775+0000 7f7a19a3b740 1 Processor -- start 2024-03-27T11:05:02.746 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.775+0000 7f7a19a3b740 1 -- start start 2024-03-27T11:05:02.746 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.776+0000 7f7a19a3b740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b81fdfdb60 0x55b81fdfca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:05:02.746 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.776+0000 7f7a19a3b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b81fdfe550 0x55b81fe0f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:05:02.746 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.776+0000 7f7a19a3b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b81fe036c0 0x55b81fe07490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:05:02.746 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.776+0000 7f7a19a3b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7a101a6c40 con 0x55b81fdfe550 2024-03-27T11:05:02.746 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.776+0000 7f7a19a3b740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f7a101a8780 con 0x55b81fe036c0 2024-03-27T11:05:02.746 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.776+0000 7f7a19a3b740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f7a101a7310 con 0x55b81fdfdb60 2024-03-27T11:05:02.749 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.777+0000 7f7a0fe00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b81fdfdb60 0x55b81fdfca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:05:02.749 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.777+0000 7f7a0fe00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b81fdfdb60 0x55b81fdfca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.97:51512/0 (socket says 172.21.15.97:51512) 2024-03-27T11:05:02.749 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.777+0000 7f7a0fe00640 1 -- 172.21.15.97:0/1909416216 learned_addr learned my addr 172.21.15.97:0/1909416216 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:05:02.749 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.777+0000 7f7a14e00640 1 --2- 172.21.15.97:0/1909416216 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b81fe036c0 0x55b81fe07490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:05:02.749 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.778+0000 7f7a0f400640 1 --2- 172.21.15.97:0/1909416216 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b81fdfe550 0x55b81fe0f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:05:02.749 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.778+0000 7f7a0fe00640 1 -- 172.21.15.97:0/1909416216 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b81fe036c0 msgr2=0x55b81fe07490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:05:02.749 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.778+0000 7f7a0fe00640 1 --2- 172.21.15.97:0/1909416216 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b81fe036c0 0x55b81fe07490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:02.749 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.778+0000 7f7a0fe00640 1 -- 172.21.15.97:0/1909416216 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b81fdfe550 msgr2=0x55b81fe0f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:05:02.749 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.778+0000 7f7a0fe00640 1 --2- 172.21.15.97:0/1909416216 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b81fdfe550 0x55b81fe0f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:02.749 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.778+0000 7f7a0fe00640 1 -- 172.21.15.97:0/1909416216 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b81fb43620 con 0x55b81fdfdb60 2024-03-27T11:05:02.749 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.778+0000 7f7a0f400640 1 --2- 172.21.15.97:0/1909416216 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b81fdfe550 0x55b81fe0f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T11:05:02.753 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.783+0000 7f7a0fe00640 1 --2- 172.21.15.97:0/1909416216 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b81fdfdb60 0x55b81fdfca40 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7a00003f20 tx=0x7f7a000092a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39d62a684f9c7a55 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:05:02.754 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.784+0000 7f7a04c00640 1 -- 172.21.15.97:0/1909416216 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a0002c070 con 0x55b81fdfdb60 2024-03-27T11:05:02.754 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.784+0000 7f7a04c00640 1 -- 172.21.15.97:0/1909416216 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7a00010e00 con 0x55b81fdfdb60 2024-03-27T11:05:02.754 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.784+0000 7f7a04c00640 1 -- 172.21.15.97:0/1909416216 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a0000fca0 con 0x55b81fdfdb60 2024-03-27T11:05:02.756 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.785+0000 7f7a19a3b740 1 -- 172.21.15.97:0/1909416216 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b81fdfdb60 msgr2=0x55b81fdfca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:05:02.756 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.785+0000 7f7a19a3b740 1 --2- 172.21.15.97:0/1909416216 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b81fdfdb60 0x55b81fdfca40 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7a00003f20 tx=0x7f7a000092a0 comp rx=0 tx=0).stop 2024-03-27T11:05:02.756 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.786+0000 7f7a19a3b740 1 -- 172.21.15.97:0/1909416216 shutdown_connections 2024-03-27T11:05:02.756 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.786+0000 7f7a19a3b740 1 --2- 172.21.15.97:0/1909416216 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b81fe036c0 0x55b81fe07490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:02.756 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.786+0000 7f7a19a3b740 1 --2- 172.21.15.97:0/1909416216 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b81fdfe550 0x55b81fe0f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:02.756 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.786+0000 7f7a19a3b740 1 --2- 172.21.15.97:0/1909416216 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b81fdfdb60 0x55b81fdfca40 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:02.756 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.786+0000 7f7a19a3b740 1 -- 172.21.15.97:0/1909416216 >> 172.21.15.97:0/1909416216 conn(0x55b81fdf90d0 msgr2=0x55b81fdfc330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:05:02.757 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.786+0000 7f7a19a3b740 1 -- 172.21.15.97:0/1909416216 shutdown_connections 2024-03-27T11:05:02.757 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.786+0000 7f7a19a3b740 1 -- 172.21.15.97:0/1909416216 wait complete. 2024-03-27T11:05:02.757 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.787+0000 7f7a19a3b740 1 -- start start 2024-03-27T11:05:02.757 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.787+0000 7f7a19a3b740 1 Processor -- start 2024-03-27T11:05:02.758 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.788+0000 7f7a19a3b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b81fdfdb60 0x55b81fe0c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:05:02.759 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.788+0000 7f7a19a3b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b81fdfe550 0x55b81fe0c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:05:02.759 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.789+0000 7f7a19a3b740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b81fe036c0 0x55b81fe0d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:05:02.759 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.789+0000 7f7a19a3b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7a101a80b0 con 0x55b81fdfe550 2024-03-27T11:05:02.759 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.789+0000 7f7a19a3b740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f7a101a79e0 con 0x55b81fdfdb60 2024-03-27T11:05:02.759 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.789+0000 7f7a19a3b740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f7a101a0480 con 0x55b81fe036c0 2024-03-27T11:05:02.759 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.789+0000 7f7a19a3b740 10 client.0 _ll_register_callbacks cb 0x7f7a00009860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T11:05:02.759 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.789+0000 7f7a0fe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b81fdfdb60 0x55b81fe0c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:05:02.760 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.789+0000 7f7a0fe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b81fdfdb60 0x55b81fe0c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:33994/0 (socket says 172.21.15.97:33994) 2024-03-27T11:05:02.760 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.789+0000 7f7a0fe00640 1 -- 172.21.15.97:0/1682430213 learned_addr learned my addr 172.21.15.97:0/1682430213 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:05:02.760 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.789+0000 7f7a0fe00640 1 -- 172.21.15.97:0/1682430213 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b81fe036c0 msgr2=0x55b81fe0d9b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-27T11:05:02.761 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.789+0000 7f7a0fe00640 1 --2- 172.21.15.97:0/1682430213 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b81fe036c0 0x55b81fe0d9b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:02.761 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.789+0000 7f7a0fe00640 1 -- 172.21.15.97:0/1682430213 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b81fdfe550 msgr2=0x55b81fe0c9a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-27T11:05:02.761 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.789+0000 7f7a0fe00640 1 --2- 172.21.15.97:0/1682430213 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b81fdfe550 0x55b81fe0c9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:02.761 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.789+0000 7f7a0fe00640 1 -- 172.21.15.97:0/1682430213 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b81fe0dfa0 con 0x55b81fdfdb60 2024-03-27T11:05:02.761 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.790+0000 7f7a0fe00640 1 --2- 172.21.15.97:0/1682430213 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b81fdfdb60 0x55b81fe0c460 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7a00012c00 tx=0x7f7a00004230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f29eac50540ecbc6 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:05:02.761 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.790+0000 7f79fa000640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-03-27T11:05:02.761 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.790+0000 7f79fa000640 1 -- 172.21.15.97:0/1682430213 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a00035070 con 0x55b81fdfdb60 2024-03-27T11:05:02.761 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.790+0000 7f79fa000640 1 -- 172.21.15.97:0/1682430213 <== mon.2 v2:172.21.15.114:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f7a0000fc30 con 0x55b81fdfdb60 2024-03-27T11:05:02.761 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.791+0000 7f7a19a3b740 1 -- 172.21.15.97:0/1682430213 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a00010e00 con 0x55b81fdfdb60 2024-03-27T11:05:02.762 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.791+0000 7f79fa000640 1 -- 172.21.15.97:0/1682430213 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f7a0002c030 con 0x55b81fdfdb60 2024-03-27T11:05:02.762 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.791+0000 7f7a19a3b740 10 client.36443 fetch_fsmap learned FSMap version 34 2024-03-27T11:05:02.762 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.791+0000 7f7a19a3b740 1 -- 172.21.15.97:0/1682430213 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55b81fe0f2b0 con 0x55b81fdfdb60 2024-03-27T11:05:02.762 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.791+0000 7f79fa000640 1 -- 172.21.15.97:0/1682430213 <== mon.2 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7a00010e00 con 0x55b81fdfdb60 2024-03-27T11:05:02.763 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.792+0000 7f79fa000640 1 -- 172.21.15.97:0/1682430213 <== mon.2 v2:172.21.15.114:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f7a00038020 con 0x55b81fdfdb60 2024-03-27T11:05:02.763 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.792+0000 7f7a19a3b740 10 client.36443 fetch_fsmap finished waiting for FSMap version 34 2024-03-27T11:05:02.763 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.792+0000 7f7a19a3b740 10 client.36443 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T11:05:02.763 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.792+0000 7f7a19a3b740 20 client.36443 populate_metadata read hostname 'smithi097' 2024-03-27T11:05:02.763 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.792+0000 7f7a19a3b740 1 --2- 172.21.15.97:0/1682430213 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55b81fe8b1b0 0x55b81fe8d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:05:02.763 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.792+0000 7f7a19a3b740 4 client.36443 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-27T11:05:02.763 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.792+0000 7f7a19a3b740 1 -- 172.21.15.97:0/1682430213 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b81fd6a5a0 con 0x55b81fe8b1b0 2024-03-27T11:05:02.764 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.794+0000 7f7a0f400640 1 --2- 172.21.15.97:0/1682430213 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55b81fe8b1b0 0x55b81fe8d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:05:02.766 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.796+0000 7f7a0f400640 1 --2- 172.21.15.97:0/1682430213 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55b81fe8b1b0 0x55b81fe8d5d0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59fee287719d8a1 server_cookie=4adfafc61b9c74a2 in_seq=0 out_seq=0 2024-03-27T11:05:02.766 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.796+0000 7f79fa000640 10 client.36443 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T11:05:02.777 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.807+0000 7f79fa000640 1 -- 172.21.15.97:0/1682430213 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b81fd6a5a0 con 0x55b81fe8b1b0 2024-03-27T11:05:02.777 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.807+0000 7f79fa000640 10 client.36443 handle_command_reply: tid=0 2024-03-27T11:05:02.777 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.807+0000 7f7a19a3b740 1 client.36443 shutdown 2024-03-27T11:05:02.777 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.807+0000 7f7a19a3b740 1 -- 172.21.15.97:0/1682430213 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b81fdfdb60 msgr2=0x55b81fe0c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:05:02.777 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.807+0000 7f7a19a3b740 1 --2- 172.21.15.97:0/1682430213 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b81fdfdb60 0x55b81fe0c460 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7a00012c00 tx=0x7f7a00004230 comp rx=0 tx=0).stop 2024-03-27T11:05:02.778 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.807+0000 7f7a19a3b740 1 -- 172.21.15.97:0/1682430213 shutdown_connections 2024-03-27T11:05:02.778 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.807+0000 7f7a19a3b740 1 --2- 172.21.15.97:0/1682430213 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55b81fe8b1b0 0x55b81fe8d5d0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:02.778 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.807+0000 7f7a19a3b740 1 --2- 172.21.15.97:0/1682430213 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b81fe036c0 0x55b81fe0d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:02.778 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.807+0000 7f7a19a3b740 1 --2- 172.21.15.97:0/1682430213 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b81fdfe550 0x55b81fe0c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:02.778 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.807+0000 7f79fa000640 0 client.36443 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T11:05:02.778 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.807+0000 7f7a19a3b740 1 --2- 172.21.15.97:0/1682430213 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b81fdfdb60 0x55b81fe0c460 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:02.778 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.807+0000 7f7a19a3b740 1 -- 172.21.15.97:0/1682430213 >> 172.21.15.97:0/1682430213 conn(0x55b81fdf90d0 msgr2=0x55b81fdf7a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:05:02.779 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.808+0000 7f7a19a3b740 1 -- 172.21.15.97:0/1682430213 shutdown_connections 2024-03-27T11:05:02.779 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.808+0000 7f7a19a3b740 1 -- 172.21.15.97:0/1682430213 wait complete. 2024-03-27T11:05:02.779 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.808+0000 7f7a19a3b740 20 client.36443 trim_cache size 0 max 16384 2024-03-27T11:05:02.779 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-27T11:05:02.779 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-27T11:05:02.779 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-27T11:05:02.779 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-27T11:05:02.780 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.810+0000 7f7a15800640 1 -- 172.21.15.97:0/3372037475 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f79f40a3680 msgr2=0x7f79f40a5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:05:02.780 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.810+0000 7f7a15800640 1 --2- 172.21.15.97:0/3372037475 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f79f40a3680 0x7f79f40a5b40 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f7a08009370 tx=0x7f7a08005fd0 comp rx=0 tx=0).stop 2024-03-27T11:05:02.780 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.810+0000 7f7a15800640 1 -- 172.21.15.97:0/3372037475 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7a10104d50 msgr2=0x7f7a10196ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:05:02.780 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.810+0000 7f7a15800640 1 --2- 172.21.15.97:0/3372037475 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7a10104d50 0x7f7a10196ab0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f79fc005fd0 tx=0x7f79fc00ec20 comp rx=0 tx=0).stop 2024-03-27T11:05:02.780 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.810+0000 7f7a15800640 1 -- 172.21.15.97:0/3372037475 shutdown_connections 2024-03-27T11:05:02.781 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.810+0000 7f7a15800640 1 --2- 172.21.15.97:0/3372037475 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a10107680 0x7f7a101930e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:02.781 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.810+0000 7f7a15800640 1 --2- 172.21.15.97:0/3372037475 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f79f40a3680 0x7f79f40a5b40 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:02.781 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.810+0000 7f7a15800640 1 --2- 172.21.15.97:0/3372037475 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7a10104d50 0x7f7a10196ab0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:02.781 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.810+0000 7f7a15800640 1 --2- 172.21.15.97:0/3372037475 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7a10102420 0x7f7a10196570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:02.781 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.810+0000 7f7a15800640 1 -- 172.21.15.97:0/3372037475 >> 172.21.15.97:0/3372037475 conn(0x7f7a100fbe30 msgr2=0x7f7a10101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:05:02.782 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.812+0000 7f7a15800640 1 -- 172.21.15.97:0/3372037475 shutdown_connections 2024-03-27T11:05:02.783 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:02.813+0000 7f7a15800640 1 -- 172.21.15.97:0/3372037475 wait complete. 2024-03-27T11:05:03.641 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:05:03 smithi097 ceph-mon[31859]: pgmap v693: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 51 MiB/s wr, 815 op/s 2024-03-27T11:05:03.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:05:03 smithi177 ceph-mon[35294]: pgmap v693: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 51 MiB/s wr, 815 op/s 2024-03-27T11:05:03.764 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:05:03 smithi114 ceph-mon[35338]: pgmap v693: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 51 MiB/s wr, 815 op/s 2024-03-27T11:05:05.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:05:05 smithi097 ceph-mon[31859]: pgmap v694: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 51 MiB/s wr, 823 op/s 2024-03-27T11:05:05.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:05:05 smithi177 ceph-mon[35294]: pgmap v694: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 51 MiB/s wr, 823 op/s 2024-03-27T11:05:05.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:05:05 smithi114 ceph-mon[35338]: pgmap v694: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 51 MiB/s wr, 823 op/s 2024-03-27T11:05:07.638 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:05:07 smithi097 ceph-mon[31859]: pgmap v695: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 49 MiB/s wr, 797 op/s 2024-03-27T11:05:07.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:05:07 smithi177 ceph-mon[35294]: pgmap v695: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 49 MiB/s wr, 797 op/s 2024-03-27T11:05:07.766 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:05:07 smithi114 ceph-mon[35338]: pgmap v695: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 49 MiB/s wr, 797 op/s 2024-03-27T11:05:09.639 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:05:09 smithi097 ceph-mon[31859]: pgmap v696: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 52 MiB/s wr, 833 op/s 2024-03-27T11:05:09.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:05:09 smithi177 ceph-mon[35294]: pgmap v696: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 52 MiB/s wr, 833 op/s 2024-03-27T11:05:09.764 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:05:09 smithi114 ceph-mon[35338]: pgmap v696: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 52 MiB/s wr, 833 op/s 2024-03-27T11:05:11.675 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:05:11 smithi177 ceph-mon[35294]: pgmap v697: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 54 MiB/s wr, 879 op/s 2024-03-27T11:05:11.852 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:05:11 smithi114 ceph-mon[35338]: pgmap v697: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 54 MiB/s wr, 879 op/s 2024-03-27T11:05:11.889 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:05:11 smithi097 ceph-mon[31859]: pgmap v697: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 54 MiB/s wr, 879 op/s 2024-03-27T11:05:13.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:05:13 smithi177 ceph-mon[35294]: pgmap v698: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 48 MiB/s wr, 781 op/s 2024-03-27T11:05:13.764 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:05:13 smithi114 ceph-mon[35338]: pgmap v698: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 48 MiB/s wr, 781 op/s 2024-03-27T11:05:13.888 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:05:13 smithi097 ceph-mon[31859]: pgmap v698: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 48 MiB/s wr, 781 op/s 2024-03-27T11:05:15.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:05:15 smithi177 ceph-mon[35294]: pgmap v699: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 49 MiB/s wr, 795 op/s 2024-03-27T11:05:15.764 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:05:15 smithi114 ceph-mon[35338]: pgmap v699: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 49 MiB/s wr, 795 op/s 2024-03-27T11:05:15.889 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:05:15 smithi097 ceph-mon[31859]: pgmap v699: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 49 MiB/s wr, 795 op/s 2024-03-27T11:05:16.006 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.037+0000 7f2998600640 1 -- 172.21.15.97:0/2511437893 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2990077750 msgr2=0x7f2990077bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:05:16.006 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.037+0000 7f2998600640 1 --2- 172.21.15.97:0/2511437893 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2990077750 0x7f2990077bb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f297c009900 tx=0x7f297c02bc50 comp rx=0 tx=0).stop 2024-03-27T11:05:16.006 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.037+0000 7f2998600640 1 -- 172.21.15.97:0/2511437893 shutdown_connections 2024-03-27T11:05:16.006 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.037+0000 7f2998600640 1 --2- 172.21.15.97:0/2511437893 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f29900780f0 0x7f2990078990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:16.007 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.037+0000 7f2998600640 1 --2- 172.21.15.97:0/2511437893 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2990077750 0x7f2990077bb0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:16.007 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.037+0000 7f2998600640 1 --2- 172.21.15.97:0/2511437893 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2990076510 0x7f2990076930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:16.007 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.037+0000 7f2998600640 1 -- 172.21.15.97:0/2511437893 >> 172.21.15.97:0/2511437893 conn(0x7f29900ff340 msgr2=0x7f2990101760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:05:16.007 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.037+0000 7f2998600640 1 -- 172.21.15.97:0/2511437893 shutdown_connections 2024-03-27T11:05:16.007 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.037+0000 7f2998600640 1 -- 172.21.15.97:0/2511437893 wait complete. 2024-03-27T11:05:16.008 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.039+0000 7f2998600640 1 Processor -- start 2024-03-27T11:05:16.008 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.039+0000 7f2998600640 1 -- start start 2024-03-27T11:05:16.008 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.039+0000 7f2998600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2990076510 0x7f299019e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:05:16.009 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.039+0000 7f2998600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2990077750 0x7f299019e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:05:16.009 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.039+0000 7f2998600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29900780f0 0x7f29901a5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:05:16.009 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.039+0000 7f2998600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2990113d30 con 0x7f29900780f0 2024-03-27T11:05:16.009 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.039+0000 7f2998600640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f2990113bb0 con 0x7f2990077750 2024-03-27T11:05:16.009 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.039+0000 7f2998600640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2990113eb0 con 0x7f2990076510 2024-03-27T11:05:16.009 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.040+0000 7f2995c00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2990076510 0x7f299019e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:05:16.010 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.040+0000 7f2995c00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2990076510 0x7f299019e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.97:45450/0 (socket says 172.21.15.97:45450) 2024-03-27T11:05:16.010 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.040+0000 7f2995200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2990077750 0x7f299019e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:05:16.010 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.040+0000 7f2995c00640 1 -- 172.21.15.97:0/624185665 learned_addr learned my addr 172.21.15.97:0/624185665 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:05:16.010 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.040+0000 7f2996600640 1 --2- 172.21.15.97:0/624185665 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29900780f0 0x7f29901a5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:05:16.010 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.040+0000 7f2995c00640 1 -- 172.21.15.97:0/624185665 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2990077750 msgr2=0x7f299019e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:05:16.010 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.040+0000 7f2995c00640 1 --2- 172.21.15.97:0/624185665 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2990077750 0x7f299019e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:16.011 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.040+0000 7f2995c00640 1 -- 172.21.15.97:0/624185665 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29900780f0 msgr2=0x7f29901a5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:05:16.011 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.040+0000 7f2995c00640 1 --2- 172.21.15.97:0/624185665 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29900780f0 0x7f29901a5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:16.011 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.040+0000 7f2995c00640 1 -- 172.21.15.97:0/624185665 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29901a5ec0 con 0x7f2990076510 2024-03-27T11:05:16.011 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.041+0000 7f2995c00640 1 --2- 172.21.15.97:0/624185665 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2990076510 0x7f299019e2f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f298400d950 tx=0x7f298400de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:05:16.011 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.041+0000 7f298e000640 1 -- 172.21.15.97:0/624185665 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2984017070 con 0x7f2990076510 2024-03-27T11:05:16.012 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.041+0000 7f298e000640 1 -- 172.21.15.97:0/624185665 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2984009c30 con 0x7f2990076510 2024-03-27T11:05:16.012 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.042+0000 7f298e000640 1 -- 172.21.15.97:0/624185665 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f298400f620 con 0x7f2990076510 2024-03-27T11:05:16.012 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.043+0000 7f2998600640 1 -- 172.21.15.97:0/624185665 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29901a61b0 con 0x7f2990076510 2024-03-27T11:05:16.012 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.043+0000 7f2998600640 1 -- 172.21.15.97:0/624185665 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29901a6680 con 0x7f2990076510 2024-03-27T11:05:16.016 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.047+0000 7f298e000640 1 -- 172.21.15.97:0/624185665 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f29840109a0 con 0x7f2990076510 2024-03-27T11:05:16.018 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.049+0000 7f298e000640 1 --2- 172.21.15.97:0/624185665 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f29780a3870 0x7f29780a5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:05:16.019 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.049+0000 7f298e000640 1 -- 172.21.15.97:0/624185665 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f29840cf3f0 con 0x7f2990076510 2024-03-27T11:05:16.021 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.050+0000 7f299b435740 1 Processor -- start 2024-03-27T11:05:16.021 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.050+0000 7f299b435740 1 -- start start 2024-03-27T11:05:16.021 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.051+0000 7f299b435740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5579fcfb4990 0x5579fcfb4db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:05:16.021 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.051+0000 7f299b435740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5579fcfb5380 0x5579fcfb9e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:05:16.021 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.051+0000 7f299b435740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5579fcfba4f0 0x5579fcfbc980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:05:16.021 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.051+0000 7f2995200640 1 --2- 172.21.15.97:0/624185665 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f29780a3870 0x7f29780a5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:05:16.021 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.051+0000 7f2995c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5579fcfb4990 0x5579fcfb4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:05:16.022 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.051+0000 7f2995c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5579fcfb4990 0x5579fcfb4db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47922/0 (socket says 172.21.15.97:47922) 2024-03-27T11:05:16.022 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.051+0000 7f2995200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5579fcfb5380 0x5579fcfb9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:05:16.022 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.052+0000 7f2995200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5579fcfb5380 0x5579fcfb9e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.97:45466/0 (socket says 172.21.15.97:45466) 2024-03-27T11:05:16.022 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.052+0000 7f2995c00640 1 -- 172.21.15.97:0/238103733 learned_addr learned my addr 172.21.15.97:0/238103733 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:05:16.022 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.052+0000 7f299b435740 1 -- 172.21.15.97:0/238103733 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5579fcf08d10 con 0x5579fcfb4990 2024-03-27T11:05:16.022 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.053+0000 7f299b435740 1 -- 172.21.15.97:0/238103733 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5579fceb1260 con 0x5579fcfba4f0 2024-03-27T11:05:16.023 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.053+0000 7f299b435740 1 -- 172.21.15.97:0/238103733 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5579fcf04cc0 con 0x5579fcfb5380 2024-03-27T11:05:16.023 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.053+0000 7f2996600640 1 --2- 172.21.15.97:0/238103733 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5579fcfba4f0 0x5579fcfbc980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:05:16.023 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.054+0000 7f2995200640 1 --2- 172.21.15.97:0/624185665 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f29780a3870 0x7f29780a5d30 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f297c000c00 tx=0x7f297c02cd00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:05:16.023 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.054+0000 7f2995c00640 1 -- 172.21.15.97:0/238103733 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5579fcfb5380 msgr2=0x5579fcfb9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:05:16.023 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.054+0000 7f2995c00640 1 --2- 172.21.15.97:0/238103733 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5579fcfb5380 0x5579fcfb9e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.054+0000 7f2995c00640 1 -- 172.21.15.97:0/238103733 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5579fcfba4f0 msgr2=0x5579fcfbc980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:05:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.054+0000 7f2995c00640 1 --2- 172.21.15.97:0/238103733 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5579fcfba4f0 0x5579fcfbc980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.054+0000 7f2995c00640 1 -- 172.21.15.97:0/238103733 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579fcea8aa0 con 0x5579fcfb4990 2024-03-27T11:05:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.054+0000 7f2995200640 1 --2- 172.21.15.97:0/238103733 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5579fcfb5380 0x5579fcfb9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T11:05:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.055+0000 7f2995c00640 1 --2- 172.21.15.97:0/238103733 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5579fcfb4990 0x5579fcfb4db0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f2984010cf0 tx=0x7f298409b2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c64a00a92fdfe16 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:05:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.055+0000 7f2982a00640 1 -- 172.21.15.97:0/238103733 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f298409fc20 con 0x5579fcfb4990 2024-03-27T11:05:16.025 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.055+0000 7f2982a00640 1 -- 172.21.15.97:0/238103733 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f29840a44e0 con 0x5579fcfb4990 2024-03-27T11:05:16.025 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.055+0000 7f2982a00640 1 -- 172.21.15.97:0/238103733 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29840106c0 con 0x5579fcfb4990 2024-03-27T11:05:16.027 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.058+0000 7f299b435740 1 -- 172.21.15.97:0/238103733 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5579fcfb4990 msgr2=0x5579fcfb4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:05:16.027 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.058+0000 7f299b435740 1 --2- 172.21.15.97:0/238103733 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5579fcfb4990 0x5579fcfb4db0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f2984010cf0 tx=0x7f298409b2d0 comp rx=0 tx=0).stop 2024-03-27T11:05:16.028 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.058+0000 7f299b435740 1 -- 172.21.15.97:0/238103733 shutdown_connections 2024-03-27T11:05:16.028 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.058+0000 7f299b435740 1 --2- 172.21.15.97:0/238103733 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5579fcfba4f0 0x5579fcfbc980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:16.028 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.058+0000 7f299b435740 1 --2- 172.21.15.97:0/238103733 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5579fcfb5380 0x5579fcfb9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:16.028 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.058+0000 7f299b435740 1 --2- 172.21.15.97:0/238103733 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5579fcfb4990 0x5579fcfb4db0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:16.028 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.058+0000 7f299b435740 1 -- 172.21.15.97:0/238103733 >> 172.21.15.97:0/238103733 conn(0x5579fcf9eef0 msgr2=0x5579fcf9f2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:05:16.028 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.059+0000 7f299b435740 1 -- 172.21.15.97:0/238103733 shutdown_connections 2024-03-27T11:05:16.029 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.059+0000 7f299b435740 1 -- 172.21.15.97:0/238103733 wait complete. 2024-03-27T11:05:16.029 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.059+0000 7f299b435740 1 -- start start 2024-03-27T11:05:16.029 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.060+0000 7f299b435740 1 Processor -- start 2024-03-27T11:05:16.030 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.060+0000 7f299b435740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5579fcfb4990 0x5579fcfbecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:05:16.030 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.060+0000 7f299b435740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5579fcfb5380 0x5579fcfbf200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:05:16.031 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.061+0000 7f299b435740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5579fcfba4f0 0x5579fcfc6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:05:16.031 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.062+0000 7f2995200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5579fcfb5380 0x5579fcfbf200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:05:16.032 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.062+0000 7f299b435740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5579fcfbd4a0 con 0x5579fcfb5380 2024-03-27T11:05:16.032 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.062+0000 7f299b435740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5579fcec4630 con 0x5579fcfba4f0 2024-03-27T11:05:16.032 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.062+0000 7f299b435740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5579fcfbd620 con 0x5579fcfb4990 2024-03-27T11:05:16.032 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.062+0000 7f299b435740 10 client.0 _ll_register_callbacks cb 0x5579fcec9c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T11:05:16.032 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.062+0000 7f2995c00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5579fcfb4990 0x5579fcfbecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:05:16.032 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.062+0000 7f2995c00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5579fcfb4990 0x5579fcfbecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.97:45478/0 (socket says 172.21.15.97:45478) 2024-03-27T11:05:16.032 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.062+0000 7f2995c00640 1 -- 172.21.15.97:0/2386452529 learned_addr learned my addr 172.21.15.97:0/2386452529 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:05:16.032 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.062+0000 7f2995200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5579fcfb5380 0x5579fcfbf200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47936/0 (socket says 172.21.15.97:47936) 2024-03-27T11:05:16.033 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.063+0000 7f2996600640 1 --2- 172.21.15.97:0/2386452529 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5579fcfba4f0 0x5579fcfc6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:05:16.033 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.063+0000 7f2995200640 1 -- 172.21.15.97:0/2386452529 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5579fcfb4990 msgr2=0x5579fcfbecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:05:16.033 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.063+0000 7f2995200640 1 --2- 172.21.15.97:0/2386452529 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5579fcfb4990 0x5579fcfbecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:16.033 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.063+0000 7f2995200640 1 -- 172.21.15.97:0/2386452529 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5579fcfba4f0 msgr2=0x5579fcfc6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:05:16.033 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.063+0000 7f2995200640 1 --2- 172.21.15.97:0/2386452529 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5579fcfba4f0 0x5579fcfc6240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:16.033 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.063+0000 7f2995200640 1 -- 172.21.15.97:0/2386452529 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579fced05f0 con 0x5579fcfb5380 2024-03-27T11:05:16.033 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.064+0000 7f2995200640 1 --2- 172.21.15.97:0/2386452529 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5579fcfb5380 0x5579fcfbf200 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f297c009850 tx=0x7f297c02bc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=800e3a21cbde4114 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:05:16.033 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.064+0000 7f2977400640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T11:05:16.034 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.065+0000 7f2977400640 1 -- 172.21.15.97:0/2386452529 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f297c051030 con 0x5579fcfb5380 2024-03-27T11:05:16.034 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.065+0000 7f2977400640 1 -- 172.21.15.97:0/2386452529 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f297c03a040 con 0x5579fcfb5380 2024-03-27T11:05:16.034 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.065+0000 7f299b435740 1 -- 172.21.15.97:0/2386452529 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29840a44e0 con 0x5579fcfb5380 2024-03-27T11:05:16.035 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.066+0000 7f2977400640 1 -- 172.21.15.97:0/2386452529 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f297c049070 con 0x5579fcfb5380 2024-03-27T11:05:16.035 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.066+0000 7f299b435740 10 client.16779 fetch_fsmap learned FSMap version 34 2024-03-27T11:05:16.035 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.066+0000 7f299b435740 1 -- 172.21.15.97:0/2386452529 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5579fcc1d210 con 0x5579fcfb5380 2024-03-27T11:05:16.037 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.067+0000 7f2977400640 1 -- 172.21.15.97:0/2386452529 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f29840a44e0 con 0x5579fcfb5380 2024-03-27T11:05:16.037 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.067+0000 7f2977400640 1 -- 172.21.15.97:0/2386452529 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f297c040040 con 0x5579fcfb5380 2024-03-27T11:05:16.037 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.067+0000 7f299b435740 10 client.16779 fetch_fsmap finished waiting for FSMap version 34 2024-03-27T11:05:16.037 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.067+0000 7f299b435740 10 client.16779 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T11:05:16.037 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.067+0000 7f299b435740 20 client.16779 populate_metadata read hostname 'smithi097' 2024-03-27T11:05:16.037 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.067+0000 7f299b435740 1 --2- 172.21.15.97:0/2386452529 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5579fcfcab20 0x5579fcfccfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:05:16.037 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.068+0000 7f299b435740 4 client.16779 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T11:05:16.037 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.068+0000 7f299b435740 1 -- 172.21.15.97:0/2386452529 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5579fcf16250 con 0x5579fcfcab20 2024-03-27T11:05:16.037 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.068+0000 7f2995c00640 1 --2- 172.21.15.97:0/2386452529 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5579fcfcab20 0x5579fcfccfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:05:16.038 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.069+0000 7f2995c00640 1 --2- 172.21.15.97:0/2386452529 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5579fcfcab20 0x5579fcfccfb0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=154c1e699b911e7e server_cookie=a241954276589eb3 in_seq=0 out_seq=0 2024-03-27T11:05:16.038 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.069+0000 7f2977400640 10 client.16779 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T11:05:16.040 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.070+0000 7f2977400640 1 -- 172.21.15.97:0/2386452529 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5579fcf16250 con 0x5579fcfcab20 2024-03-27T11:05:16.040 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.070+0000 7f2977400640 10 client.16779 handle_command_reply: tid=0 2024-03-27T11:05:16.041 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.071+0000 7f299b435740 1 client.16779 shutdown 2024-03-27T11:05:16.041 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.072+0000 7f299b435740 1 -- 172.21.15.97:0/2386452529 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5579fcfb5380 msgr2=0x5579fcfbf200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:05:16.041 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.072+0000 7f299b435740 1 --2- 172.21.15.97:0/2386452529 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5579fcfb5380 0x5579fcfbf200 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f297c009850 tx=0x7f297c02bc50 comp rx=0 tx=0).stop 2024-03-27T11:05:16.042 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.073+0000 7f299b435740 1 -- 172.21.15.97:0/2386452529 shutdown_connections 2024-03-27T11:05:16.042 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.073+0000 7f299b435740 1 --2- 172.21.15.97:0/2386452529 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5579fcfcab20 0x5579fcfccfb0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:16.042 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.073+0000 7f299b435740 1 --2- 172.21.15.97:0/2386452529 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5579fcfba4f0 0x5579fcfc6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:16.042 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.073+0000 7f299b435740 1 --2- 172.21.15.97:0/2386452529 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5579fcfb5380 0x5579fcfbf200 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:16.042 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.073+0000 7f299b435740 1 --2- 172.21.15.97:0/2386452529 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5579fcfb4990 0x5579fcfbecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:16.042 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.073+0000 7f299b435740 1 -- 172.21.15.97:0/2386452529 >> 172.21.15.97:0/2386452529 conn(0x5579fcf9eef0 msgr2=0x5579fcf9e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:05:16.043 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.073+0000 7f2977400640 0 client.16779 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T11:05:16.043 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.073+0000 7f299b435740 1 -- 172.21.15.97:0/2386452529 shutdown_connections 2024-03-27T11:05:16.043 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.074+0000 7f299b435740 1 -- 172.21.15.97:0/2386452529 wait complete. 2024-03-27T11:05:16.044 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.075+0000 7f299b435740 20 client.16779 trim_cache size 0 max 16384 2024-03-27T11:05:16.079 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.110+0000 7f299b435740 1 Processor -- start 2024-03-27T11:05:16.079 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.110+0000 7f299b435740 1 -- start start 2024-03-27T11:05:16.079 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.110+0000 7f299b435740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5579fcfb4990 0x5579fcfd18b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:05:16.080 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.110+0000 7f299b435740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5579fcfb5380 0x5579fcfc6200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:05:16.080 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.110+0000 7f299b435740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5579fcfba4f0 0x5579fcfd1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:05:16.080 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.110+0000 7f299b435740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f29901a6c40 con 0x5579fcfba4f0 2024-03-27T11:05:16.080 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.110+0000 7f299b435740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f29901a8780 con 0x5579fcfb4990 2024-03-27T11:05:16.080 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.110+0000 7f299b435740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f29901a7310 con 0x5579fcfb5380 2024-03-27T11:05:16.080 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.111+0000 7f2996600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5579fcfba4f0 0x5579fcfd1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:05:16.080 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.111+0000 7f2996600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5579fcfba4f0 0x5579fcfd1df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47952/0 (socket says 172.21.15.97:47952) 2024-03-27T11:05:16.080 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.111+0000 7f2996600640 1 -- 172.21.15.97:0/1169444320 learned_addr learned my addr 172.21.15.97:0/1169444320 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:05:16.081 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.111+0000 7f2995200640 1 --2- 172.21.15.97:0/1169444320 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5579fcfb5380 0x5579fcfc6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:05:16.081 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.112+0000 7f2996600640 1 -- 172.21.15.97:0/1169444320 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5579fcfb5380 msgr2=0x5579fcfc6200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:05:16.081 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.112+0000 7f2996600640 1 --2- 172.21.15.97:0/1169444320 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5579fcfb5380 0x5579fcfc6200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:16.081 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.112+0000 7f2996600640 1 -- 172.21.15.97:0/1169444320 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5579fcfb4990 msgr2=0x5579fcfd18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:05:16.081 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.112+0000 7f2996600640 1 --2- 172.21.15.97:0/1169444320 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5579fcfb4990 0x5579fcfd18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:16.081 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.112+0000 7f2996600640 1 -- 172.21.15.97:0/1169444320 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579fcce9620 con 0x5579fcfba4f0 2024-03-27T11:05:16.081 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.112+0000 7f2996600640 1 --2- 172.21.15.97:0/1169444320 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5579fcfba4f0 0x5579fcfd1df0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f2988003a40 tx=0x7f2988003f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b7fc569b4f56109 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:05:16.082 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.113+0000 7f2982000640 1 -- 172.21.15.97:0/1169444320 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2988004640 con 0x5579fcfba4f0 2024-03-27T11:05:16.082 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.113+0000 7f2982000640 1 -- 172.21.15.97:0/1169444320 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f29880047e0 con 0x5579fcfba4f0 2024-03-27T11:05:16.082 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.113+0000 7f2982000640 1 -- 172.21.15.97:0/1169444320 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f298801c8e0 con 0x5579fcfba4f0 2024-03-27T11:05:16.084 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.114+0000 7f299b435740 1 -- 172.21.15.97:0/1169444320 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5579fcfba4f0 msgr2=0x5579fcfd1df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:05:16.084 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.114+0000 7f299b435740 1 --2- 172.21.15.97:0/1169444320 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5579fcfba4f0 0x5579fcfd1df0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f2988003a40 tx=0x7f2988003f00 comp rx=0 tx=0).stop 2024-03-27T11:05:16.084 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.114+0000 7f299b435740 1 -- 172.21.15.97:0/1169444320 shutdown_connections 2024-03-27T11:05:16.084 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.114+0000 7f299b435740 1 --2- 172.21.15.97:0/1169444320 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5579fcfba4f0 0x5579fcfd1df0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:16.084 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.114+0000 7f299b435740 1 --2- 172.21.15.97:0/1169444320 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5579fcfb5380 0x5579fcfc6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:16.084 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.114+0000 7f299b435740 1 --2- 172.21.15.97:0/1169444320 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5579fcfb4990 0x5579fcfd18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:16.084 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.114+0000 7f299b435740 1 -- 172.21.15.97:0/1169444320 >> 172.21.15.97:0/1169444320 conn(0x5579fcf9eef0 msgr2=0x5579fcfa2150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:05:16.084 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.115+0000 7f299b435740 1 -- 172.21.15.97:0/1169444320 shutdown_connections 2024-03-27T11:05:16.084 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.115+0000 7f299b435740 1 -- 172.21.15.97:0/1169444320 wait complete. 2024-03-27T11:05:16.085 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.115+0000 7f299b435740 1 -- start start 2024-03-27T11:05:16.085 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.116+0000 7f299b435740 1 Processor -- start 2024-03-27T11:05:16.085 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.116+0000 7f299b435740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5579fcfb4990 0x5579fcfc41a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:05:16.085 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.116+0000 7f299b435740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5579fcfb5380 0x5579fcfc46e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:05:16.086 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.116+0000 7f299b435740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5579fcfba4f0 0x5579fcfc56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:05:16.086 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.116+0000 7f299b435740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f29901a80b0 con 0x5579fcfb5380 2024-03-27T11:05:16.086 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.116+0000 7f299b435740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f29901a79e0 con 0x5579fcfb4990 2024-03-27T11:05:16.086 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.116+0000 7f299b435740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f299010a350 con 0x5579fcfba4f0 2024-03-27T11:05:16.086 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.116+0000 7f299b435740 10 client.0 _ll_register_callbacks cb 0x7f297c02c610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T11:05:16.087 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.116+0000 7f2995200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5579fcfb5380 0x5579fcfc46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:05:16.087 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.117+0000 7f2995200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5579fcfb5380 0x5579fcfc46e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47960/0 (socket says 172.21.15.97:47960) 2024-03-27T11:05:16.087 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.117+0000 7f2995200640 1 -- 172.21.15.97:0/3536257724 learned_addr learned my addr 172.21.15.97:0/3536257724 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:05:16.087 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.117+0000 7f2996600640 1 --2- 172.21.15.97:0/3536257724 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5579fcfba4f0 0x5579fcfc56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:05:16.088 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.117+0000 7f2995c00640 1 --2- 172.21.15.97:0/3536257724 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5579fcfb4990 0x5579fcfc41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:05:16.088 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.117+0000 7f2995200640 1 -- 172.21.15.97:0/3536257724 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5579fcfba4f0 msgr2=0x5579fcfc56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:05:16.088 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.117+0000 7f2995200640 1 --2- 172.21.15.97:0/3536257724 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5579fcfba4f0 0x5579fcfc56f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:16.088 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.117+0000 7f2995200640 1 -- 172.21.15.97:0/3536257724 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5579fcfb4990 msgr2=0x5579fcfc41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:05:16.088 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.117+0000 7f2995200640 1 --2- 172.21.15.97:0/3536257724 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5579fcfb4990 0x5579fcfc41a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:16.088 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.117+0000 7f2995200640 1 -- 172.21.15.97:0/3536257724 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579fcfc5ce0 con 0x5579fcfb5380 2024-03-27T11:05:16.088 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.117+0000 7f2995c00640 1 --2- 172.21.15.97:0/3536257724 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5579fcfb4990 0x5579fcfc41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T11:05:16.088 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.117+0000 7f2995200640 1 --2- 172.21.15.97:0/3536257724 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5579fcfb5380 0x5579fcfc46e0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f297c003a40 tx=0x7f297c0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed88e72437de0231 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:05:16.089 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.117+0000 7f2977400640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T11:05:16.089 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.118+0000 7f2977400640 1 -- 172.21.15.97:0/3536257724 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f297c051070 con 0x5579fcfb5380 2024-03-27T11:05:16.089 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.118+0000 7f2977400640 1 -- 172.21.15.97:0/3536257724 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f297c004ae0 con 0x5579fcfb5380 2024-03-27T11:05:16.089 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.118+0000 7f299b435740 1 -- 172.21.15.97:0/3536257724 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29880047e0 con 0x5579fcfb5380 2024-03-27T11:05:16.089 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.118+0000 7f2977400640 1 -- 172.21.15.97:0/3536257724 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f297c02cac0 con 0x5579fcfb5380 2024-03-27T11:05:16.090 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.118+0000 7f299b435740 10 client.16785 fetch_fsmap learned FSMap version 34 2024-03-27T11:05:16.090 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.118+0000 7f299b435740 1 -- 172.21.15.97:0/3536257724 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5579fd0427b0 con 0x5579fcfb5380 2024-03-27T11:05:16.090 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.119+0000 7f2977400640 1 -- 172.21.15.97:0/3536257724 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f29880047e0 con 0x5579fcfb5380 2024-03-27T11:05:16.090 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.119+0000 7f2977400640 1 -- 172.21.15.97:0/3536257724 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f297c04dc30 con 0x5579fcfb5380 2024-03-27T11:05:16.090 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.120+0000 7f299b435740 10 client.16785 fetch_fsmap finished waiting for FSMap version 34 2024-03-27T11:05:16.090 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.120+0000 7f299b435740 10 client.16785 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T11:05:16.090 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.120+0000 7f299b435740 20 client.16785 populate_metadata read hostname 'smithi097' 2024-03-27T11:05:16.090 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.120+0000 7f299b435740 1 --2- 172.21.15.97:0/3536257724 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5579fd0431d0 0x5579fd0455f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:05:16.090 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.120+0000 7f299b435740 4 client.16785 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-27T11:05:16.091 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.120+0000 7f299b435740 1 -- 172.21.15.97:0/3536257724 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5579fcf105a0 con 0x5579fd0431d0 2024-03-27T11:05:16.091 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.120+0000 7f2995c00640 1 --2- 172.21.15.97:0/3536257724 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5579fd0431d0 0x5579fd0455f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:05:16.091 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.122+0000 7f2995c00640 1 --2- 172.21.15.97:0/3536257724 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5579fd0431d0 0x5579fd0455f0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfd2a639bdf68cb8 server_cookie=498695279f3ba5ef in_seq=0 out_seq=0 2024-03-27T11:05:16.091 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.122+0000 7f2977400640 10 client.16785 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T11:05:16.119 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.149+0000 7f2977400640 1 -- 172.21.15.97:0/3536257724 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5579fcf105a0 con 0x5579fd0431d0 2024-03-27T11:05:16.119 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.149+0000 7f2977400640 10 client.16785 handle_command_reply: tid=0 2024-03-27T11:05:16.119 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.150+0000 7f299b435740 1 client.16785 shutdown 2024-03-27T11:05:16.120 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.150+0000 7f299b435740 1 -- 172.21.15.97:0/3536257724 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5579fcfb5380 msgr2=0x5579fcfc46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:05:16.120 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.150+0000 7f299b435740 1 --2- 172.21.15.97:0/3536257724 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5579fcfb5380 0x5579fcfc46e0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f297c003a40 tx=0x7f297c0048f0 comp rx=0 tx=0).stop 2024-03-27T11:05:16.120 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.150+0000 7f299b435740 1 -- 172.21.15.97:0/3536257724 shutdown_connections 2024-03-27T11:05:16.120 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.150+0000 7f299b435740 1 --2- 172.21.15.97:0/3536257724 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5579fd0431d0 0x5579fd0455f0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:16.120 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.150+0000 7f299b435740 1 --2- 172.21.15.97:0/3536257724 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5579fcfba4f0 0x5579fcfc56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:16.120 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.150+0000 7f299b435740 1 --2- 172.21.15.97:0/3536257724 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5579fcfb5380 0x5579fcfc46e0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:16.120 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.150+0000 7f299b435740 1 --2- 172.21.15.97:0/3536257724 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5579fcfb4990 0x5579fcfc41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:16.120 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.150+0000 7f299b435740 1 -- 172.21.15.97:0/3536257724 >> 172.21.15.97:0/3536257724 conn(0x5579fcf9eef0 msgr2=0x5579fcfa2150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:05:16.120 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.151+0000 7f2977400640 0 client.16785 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T11:05:16.120 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.151+0000 7f299b435740 1 -- 172.21.15.97:0/3536257724 shutdown_connections 2024-03-27T11:05:16.121 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.151+0000 7f299b435740 1 -- 172.21.15.97:0/3536257724 wait complete. 2024-03-27T11:05:16.121 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.152+0000 7f299b435740 20 client.16785 trim_cache size 0 max 16384 2024-03-27T11:05:16.123 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-27T11:05:16.123 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-27T11:05:16.123 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-27T11:05:16.123 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-27T11:05:16.124 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.155+0000 7f2998600640 1 -- 172.21.15.97:0/624185665 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f29780a3870 msgr2=0x7f29780a5d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:05:16.124 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.155+0000 7f2998600640 1 --2- 172.21.15.97:0/624185665 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f29780a3870 0x7f29780a5d30 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f297c000c00 tx=0x7f297c02cd00 comp rx=0 tx=0).stop 2024-03-27T11:05:16.124 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.155+0000 7f2998600640 1 -- 172.21.15.97:0/624185665 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2990076510 msgr2=0x7f299019e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:05:16.124 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.155+0000 7f2998600640 1 --2- 172.21.15.97:0/624185665 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2990076510 0x7f299019e2f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f298400d950 tx=0x7f298400de10 comp rx=0 tx=0).stop 2024-03-27T11:05:16.124 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.155+0000 7f2998600640 1 -- 172.21.15.97:0/624185665 shutdown_connections 2024-03-27T11:05:16.124 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.155+0000 7f2998600640 1 --2- 172.21.15.97:0/624185665 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29900780f0 0x7f29901a5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:16.124 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.155+0000 7f2998600640 1 --2- 172.21.15.97:0/624185665 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f29780a3870 0x7f29780a5d30 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:16.125 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.155+0000 7f2998600640 1 --2- 172.21.15.97:0/624185665 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2990077750 0x7f299019e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:16.125 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.155+0000 7f2998600640 1 --2- 172.21.15.97:0/624185665 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2990076510 0x7f299019e2f0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:16.125 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.155+0000 7f2998600640 1 -- 172.21.15.97:0/624185665 >> 172.21.15.97:0/624185665 conn(0x7f29900ff340 msgr2=0x7f2990100e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:05:16.125 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.155+0000 7f2998600640 1 -- 172.21.15.97:0/624185665 shutdown_connections 2024-03-27T11:05:16.125 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:16.156+0000 7f2998600640 1 -- 172.21.15.97:0/624185665 wait complete. 2024-03-27T11:05:17.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:05:17 smithi177 ceph-mon[35294]: pgmap v700: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 48 MiB/s wr, 770 op/s 2024-03-27T11:05:17.764 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:05:17 smithi114 ceph-mon[35338]: pgmap v700: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 48 MiB/s wr, 770 op/s 2024-03-27T11:05:17.888 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:05:17 smithi097 ceph-mon[31859]: pgmap v700: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 48 MiB/s wr, 770 op/s 2024-03-27T11:05:19.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:05:19 smithi177 ceph-mon[35294]: pgmap v701: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 50 MiB/s wr, 805 op/s 2024-03-27T11:05:19.764 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:05:19 smithi114 ceph-mon[35338]: pgmap v701: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 50 MiB/s wr, 805 op/s 2024-03-27T11:05:19.892 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:05:19 smithi097 ceph-mon[31859]: pgmap v701: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 50 MiB/s wr, 805 op/s 2024-03-27T11:05:21.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:05:21 smithi177 ceph-mon[35294]: pgmap v702: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 53 MiB/s wr, 867 op/s 2024-03-27T11:05:21.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:05:21 smithi114 ceph-mon[35338]: pgmap v702: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 53 MiB/s wr, 867 op/s 2024-03-27T11:05:21.888 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:05:21 smithi097 ceph-mon[31859]: pgmap v702: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 53 MiB/s wr, 867 op/s 2024-03-27T11:05:23.764 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:05:23 smithi114 ceph-mon[35338]: pgmap v703: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 49 MiB/s wr, 796 op/s 2024-03-27T11:05:23.888 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:05:23 smithi097 ceph-mon[31859]: pgmap v703: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 49 MiB/s wr, 796 op/s 2024-03-27T11:05:23.924 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:05:23 smithi177 ceph-mon[35294]: pgmap v703: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 49 MiB/s wr, 796 op/s 2024-03-27T11:05:25.766 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:05:25 smithi114 ceph-mon[35338]: pgmap v704: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 50 MiB/s wr, 817 op/s 2024-03-27T11:05:25.888 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:05:25 smithi097 ceph-mon[31859]: pgmap v704: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 50 MiB/s wr, 817 op/s 2024-03-27T11:05:25.924 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:05:25 smithi177 ceph-mon[35294]: pgmap v704: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 50 MiB/s wr, 817 op/s 2024-03-27T11:05:27.764 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:05:27 smithi114 ceph-mon[35338]: pgmap v705: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 48 MiB/s wr, 784 op/s 2024-03-27T11:05:27.889 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:05:27 smithi097 ceph-mon[31859]: pgmap v705: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 48 MiB/s wr, 784 op/s 2024-03-27T11:05:27.924 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:05:27 smithi177 ceph-mon[35294]: pgmap v705: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 48 MiB/s wr, 784 op/s 2024-03-27T11:05:29.363 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.393+0000 7f67f0600640 1 -- 172.21.15.97:0/4086816060 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67e8102d90 msgr2=0x7f67e8103210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:05:29.363 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.393+0000 7f67f0600640 1 --2- 172.21.15.97:0/4086816060 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67e8102d90 0x7f67e8103210 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f67d4009900 tx=0x7f67d402bc50 comp rx=0 tx=0).stop 2024-03-27T11:05:29.363 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.393+0000 7f67f0600640 1 -- 172.21.15.97:0/4086816060 shutdown_connections 2024-03-27T11:05:29.363 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.393+0000 7f67f0600640 1 --2- 172.21.15.97:0/4086816060 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f67e8103750 0x7f67e8109fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:29.363 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.393+0000 7f67f0600640 1 --2- 172.21.15.97:0/4086816060 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67e8102d90 0x7f67e8103210 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:29.363 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.393+0000 7f67f0600640 1 --2- 172.21.15.97:0/4086816060 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f67e8101ba0 0x7f67e8101fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:29.363 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.393+0000 7f67f0600640 1 -- 172.21.15.97:0/4086816060 >> 172.21.15.97:0/4086816060 conn(0x7f67e80fd110 msgr2=0x7f67e80ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:05:29.364 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.394+0000 7f67f0600640 1 -- 172.21.15.97:0/4086816060 shutdown_connections 2024-03-27T11:05:29.364 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.394+0000 7f67f0600640 1 -- 172.21.15.97:0/4086816060 wait complete. 2024-03-27T11:05:29.364 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.395+0000 7f67f0600640 1 Processor -- start 2024-03-27T11:05:29.366 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.395+0000 7f67f0600640 1 -- start start 2024-03-27T11:05:29.366 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.397+0000 7f67f0600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f67e8101ba0 0x7f67e810e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:05:29.366 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.397+0000 7f67f0600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f67e8102d90 0x7f67e810e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:05:29.368 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.397+0000 7f67f0600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67e8103750 0x7f67e810ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:05:29.368 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.397+0000 7f67f0600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f67e810f960 con 0x7f67e8103750 2024-03-27T11:05:29.368 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.397+0000 7f67f0600640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f67e810f7e0 con 0x7f67e8101ba0 2024-03-27T11:05:29.368 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.397+0000 7f67f0600640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f67e810fae0 con 0x7f67e8102d90 2024-03-27T11:05:29.368 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.398+0000 7f67edc00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f67e8101ba0 0x7f67e810e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:05:29.368 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.399+0000 7f67edc00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f67e8101ba0 0x7f67e810e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:51640/0 (socket says 172.21.15.97:51640) 2024-03-27T11:05:29.368 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.399+0000 7f67edc00640 1 -- 172.21.15.97:0/3700744982 learned_addr learned my addr 172.21.15.97:0/3700744982 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:05:29.369 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.399+0000 7f67ed200640 1 --2- 172.21.15.97:0/3700744982 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f67e8102d90 0x7f67e810e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:05:29.369 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.399+0000 7f67ee600640 1 --2- 172.21.15.97:0/3700744982 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67e8103750 0x7f67e810ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:05:29.370 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.400+0000 7f67edc00640 1 -- 172.21.15.97:0/3700744982 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f67e8102d90 msgr2=0x7f67e810e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:05:29.370 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.400+0000 7f67edc00640 1 --2- 172.21.15.97:0/3700744982 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f67e8102d90 0x7f67e810e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:29.370 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.400+0000 7f67edc00640 1 -- 172.21.15.97:0/3700744982 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67e8103750 msgr2=0x7f67e810ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:05:29.370 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.400+0000 7f67edc00640 1 --2- 172.21.15.97:0/3700744982 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67e8103750 0x7f67e810ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:29.370 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.400+0000 7f67edc00640 1 -- 172.21.15.97:0/3700744982 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67e810b440 con 0x7f67e8101ba0 2024-03-27T11:05:29.370 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.400+0000 7f67ee600640 1 --2- 172.21.15.97:0/3700744982 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67e8103750 0x7f67e810ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T11:05:29.370 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.400+0000 7f67edc00640 1 --2- 172.21.15.97:0/3700744982 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f67e8101ba0 0x7f67e810e010 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f67e000eb70 tx=0x7f67e000c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:05:29.371 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.401+0000 7f67e6000640 1 -- 172.21.15.97:0/3700744982 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e0003b70 con 0x7f67e8101ba0 2024-03-27T11:05:29.371 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.401+0000 7f67e6000640 1 -- 172.21.15.97:0/3700744982 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f67e000ee70 con 0x7f67e8101ba0 2024-03-27T11:05:29.371 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.401+0000 7f67e6000640 1 -- 172.21.15.97:0/3700744982 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e0010650 con 0x7f67e8101ba0 2024-03-27T11:05:29.371 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.401+0000 7f67f0600640 1 -- 172.21.15.97:0/3700744982 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67e81a6430 con 0x7f67e8101ba0 2024-03-27T11:05:29.371 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.401+0000 7f67f0600640 1 -- 172.21.15.97:0/3700744982 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67e81a6860 con 0x7f67e8101ba0 2024-03-27T11:05:29.377 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.407+0000 7f67f3384740 1 Processor -- start 2024-03-27T11:05:29.377 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.407+0000 7f67f3384740 1 -- start start 2024-03-27T11:05:29.377 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.407+0000 7f67e6000640 1 -- 172.21.15.97:0/3700744982 <== mon.2 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f67e0004020 con 0x7f67e8101ba0 2024-03-27T11:05:29.377 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.408+0000 7f67f3384740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558f83a67990 0x558f83a67db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:05:29.377 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.408+0000 7f67f3384740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558f83a68380 0x558f83a6ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:05:29.378 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.408+0000 7f67f3384740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558f83a6d4f0 0x558f83a6f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:05:29.378 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.408+0000 7f67f3384740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x558f839bbd10 con 0x558f83a6d4f0 2024-03-27T11:05:29.378 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.408+0000 7f67f3384740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x558f83964260 con 0x558f83a67990 2024-03-27T11:05:29.378 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.408+0000 7f67f3384740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x558f839b7cc0 con 0x558f83a68380 2024-03-27T11:05:29.378 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.409+0000 7f67ee600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558f83a67990 0x558f83a67db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:05:29.378 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.409+0000 7f67ee600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558f83a67990 0x558f83a67db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:51650/0 (socket says 172.21.15.97:51650) 2024-03-27T11:05:29.378 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.409+0000 7f67ee600640 1 -- 172.21.15.97:0/1648268409 learned_addr learned my addr 172.21.15.97:0/1648268409 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:05:29.379 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.410+0000 7f67e6000640 1 --2- 172.21.15.97:0/3700744982 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f67cc0a3800 0x7f67cc0a5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:05:29.380 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.410+0000 7f67ed200640 1 --2- 172.21.15.97:0/1648268409 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558f83a6d4f0 0x558f83a6f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:05:29.380 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.410+0000 7f67e6000640 1 -- 172.21.15.97:0/3700744982 <== mon.2 v2:172.21.15.114:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f67e00d0380 con 0x7f67e8101ba0 2024-03-27T11:05:29.381 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.410+0000 7f67edc00640 1 --2- 172.21.15.97:0/1648268409 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558f83a68380 0x558f83a6ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:05:29.383 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.412+0000 7f67edc00640 1 -- 172.21.15.97:0/1648268409 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558f83a67990 msgr2=0x558f83a67db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:05:29.383 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.412+0000 7f67edc00640 1 --2- 172.21.15.97:0/1648268409 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558f83a67990 0x558f83a67db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:29.383 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.412+0000 7f67edc00640 1 -- 172.21.15.97:0/1648268409 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558f83a6d4f0 msgr2=0x558f83a6f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:05:29.384 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.412+0000 7f67edc00640 1 --2- 172.21.15.97:0/1648268409 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558f83a6d4f0 0x558f83a6f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:29.384 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.412+0000 7f67edc00640 1 -- 172.21.15.97:0/1648268409 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f8395baa0 con 0x558f83a68380 2024-03-27T11:05:29.384 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.412+0000 7f67ee600640 1 --2- 172.21.15.97:0/1648268409 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558f83a67990 0x558f83a67db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T11:05:29.384 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.412+0000 7f67ee600640 1 --2- 172.21.15.97:0/3700744982 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f67cc0a3800 0x7f67cc0a5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:05:29.384 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.413+0000 7f67ee600640 1 --2- 172.21.15.97:0/3700744982 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f67cc0a3800 0x7f67cc0a5cc0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f67d8009fd0 tx=0x7f67d8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:05:29.384 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.414+0000 7f67ed200640 1 --2- 172.21.15.97:0/1648268409 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558f83a6d4f0 0x558f83a6f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T11:05:29.384 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.414+0000 7f67edc00640 1 --2- 172.21.15.97:0/1648268409 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558f83a68380 0x558f83a6ce80 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f67e00991d0 tx=0x7f67e009c5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ddf0dea469b72c4 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:05:29.386 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.414+0000 7f67dea00640 1 -- 172.21.15.97:0/1648268409 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e009cf00 con 0x558f83a68380 2024-03-27T11:05:29.386 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.414+0000 7f67dea00640 1 -- 172.21.15.97:0/1648268409 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f67e009c800 con 0x558f83a68380 2024-03-27T11:05:29.386 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.414+0000 7f67dea00640 1 -- 172.21.15.97:0/1648268409 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e0087070 con 0x558f83a68380 2024-03-27T11:05:29.387 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.416+0000 7f67f3384740 1 -- 172.21.15.97:0/1648268409 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558f83a68380 msgr2=0x558f83a6ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:05:29.387 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.416+0000 7f67f3384740 1 --2- 172.21.15.97:0/1648268409 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558f83a68380 0x558f83a6ce80 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f67e00991d0 tx=0x7f67e009c5d0 comp rx=0 tx=0).stop 2024-03-27T11:05:29.387 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.416+0000 7f67f3384740 1 -- 172.21.15.97:0/1648268409 shutdown_connections 2024-03-27T11:05:29.387 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.416+0000 7f67f3384740 1 --2- 172.21.15.97:0/1648268409 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558f83a6d4f0 0x558f83a6f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:29.387 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.416+0000 7f67f3384740 1 --2- 172.21.15.97:0/1648268409 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558f83a68380 0x558f83a6ce80 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:29.387 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.416+0000 7f67f3384740 1 --2- 172.21.15.97:0/1648268409 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558f83a67990 0x558f83a67db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:29.387 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.416+0000 7f67f3384740 1 -- 172.21.15.97:0/1648268409 >> 172.21.15.97:0/1648268409 conn(0x558f83a51ef0 msgr2=0x558f83a522f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:05:29.387 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.416+0000 7f67f3384740 1 -- 172.21.15.97:0/1648268409 shutdown_connections 2024-03-27T11:05:29.388 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.416+0000 7f67f3384740 1 -- 172.21.15.97:0/1648268409 wait complete. 2024-03-27T11:05:29.388 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.417+0000 7f67f3384740 1 -- start start 2024-03-27T11:05:29.388 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.417+0000 7f67f3384740 1 Processor -- start 2024-03-27T11:05:29.390 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.417+0000 7f67f3384740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558f83a67990 0x558f83a71c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:05:29.390 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.417+0000 7f67f3384740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558f83a68380 0x558f83a721b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:05:29.390 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.417+0000 7f67f3384740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558f83a6d4f0 0x558f83a791f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:05:29.390 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.417+0000 7f67f3384740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x558f83a704a0 con 0x558f83a67990 2024-03-27T11:05:29.391 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.417+0000 7f67f3384740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x558f83977630 con 0x558f83a6d4f0 2024-03-27T11:05:29.391 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.417+0000 7f67f3384740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x558f83a70620 con 0x558f83a68380 2024-03-27T11:05:29.391 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.418+0000 7f67f3384740 10 client.0 _ll_register_callbacks cb 0x558f8397cc50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T11:05:29.391 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.418+0000 7f67ed200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558f83a68380 0x558f83a721b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:05:29.391 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.418+0000 7f67ee600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558f83a6d4f0 0x558f83a791f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:05:29.391 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.418+0000 7f67ee600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558f83a6d4f0 0x558f83a791f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:51662/0 (socket says 172.21.15.97:51662) 2024-03-27T11:05:29.391 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.418+0000 7f67ed200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558f83a68380 0x558f83a721b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.97:49670/0 (socket says 172.21.15.97:49670) 2024-03-27T11:05:29.392 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.418+0000 7f67ee600640 1 -- 172.21.15.97:0/3629862803 learned_addr learned my addr 172.21.15.97:0/3629862803 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:05:29.392 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.418+0000 7f67edc00640 1 --2- 172.21.15.97:0/3629862803 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558f83a67990 0x558f83a71c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:05:29.392 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.419+0000 7f67ee600640 1 -- 172.21.15.97:0/3629862803 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558f83a68380 msgr2=0x558f83a721b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:05:29.392 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.419+0000 7f67ee600640 1 --2- 172.21.15.97:0/3629862803 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558f83a68380 0x558f83a721b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:29.392 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.419+0000 7f67ee600640 1 -- 172.21.15.97:0/3629862803 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558f83a67990 msgr2=0x558f83a71c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:05:29.392 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.419+0000 7f67ee600640 1 --2- 172.21.15.97:0/3629862803 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558f83a67990 0x558f83a71c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:29.392 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.419+0000 7f67ee600640 1 -- 172.21.15.97:0/3629862803 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f839835f0 con 0x558f83a6d4f0 2024-03-27T11:05:29.393 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.419+0000 7f67edc00640 1 --2- 172.21.15.97:0/3629862803 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558f83a67990 0x558f83a71c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T11:05:29.393 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.419+0000 7f67ee600640 1 --2- 172.21.15.97:0/3629862803 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558f83a6d4f0 0x558f83a791f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f67d80149c0 tx=0x7f67d8009060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91c49bb1be84c0d server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:05:29.393 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.419+0000 7f67d3400640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-03-27T11:05:29.393 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.420+0000 7f67d3400640 1 -- 172.21.15.97:0/3629862803 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d8026020 con 0x558f83a6d4f0 2024-03-27T11:05:29.393 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.420+0000 7f67d3400640 1 -- 172.21.15.97:0/3629862803 <== mon.2 v2:172.21.15.114:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f67d8015b00 con 0x558f83a6d4f0 2024-03-27T11:05:29.393 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.420+0000 7f67f3384740 1 -- 172.21.15.97:0/3629862803 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67e009c800 con 0x558f83a6d4f0 2024-03-27T11:05:29.393 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.420+0000 7f67d3400640 1 -- 172.21.15.97:0/3629862803 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f67d8009b60 con 0x558f83a6d4f0 2024-03-27T11:05:29.393 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.421+0000 7f67f3384740 10 client.36461 fetch_fsmap learned FSMap version 34 2024-03-27T11:05:29.393 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.421+0000 7f67f3384740 1 -- 172.21.15.97:0/3629862803 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x558f836d0210 con 0x558f83a6d4f0 2024-03-27T11:05:29.396 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.421+0000 7f67d3400640 1 -- 172.21.15.97:0/3629862803 <== mon.2 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f67e009c800 con 0x558f83a6d4f0 2024-03-27T11:05:29.396 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.422+0000 7f67d3400640 1 -- 172.21.15.97:0/3629862803 <== mon.2 v2:172.21.15.114:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f67d8021590 con 0x558f83a6d4f0 2024-03-27T11:05:29.396 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.422+0000 7f67f3384740 10 client.36461 fetch_fsmap finished waiting for FSMap version 34 2024-03-27T11:05:29.396 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.422+0000 7f67f3384740 10 client.36461 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T11:05:29.396 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.422+0000 7f67f3384740 20 client.36461 populate_metadata read hostname 'smithi097' 2024-03-27T11:05:29.396 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.422+0000 7f67f3384740 1 --2- 172.21.15.97:0/3629862803 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x558f83a7db40 0x558f83a7ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:05:29.396 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.422+0000 7f67f3384740 4 client.36461 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T11:05:29.396 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.422+0000 7f67f3384740 1 -- 172.21.15.97:0/3629862803 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558f839c9250 con 0x558f83a7db40 2024-03-27T11:05:29.396 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.422+0000 7f67edc00640 1 --2- 172.21.15.97:0/3629862803 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x558f83a7db40 0x558f83a7ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:05:29.396 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.423+0000 7f67edc00640 1 --2- 172.21.15.97:0/3629862803 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x558f83a7db40 0x558f83a7ff50 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1bda8b45e2717275 server_cookie=278543be9e30bc67 in_seq=0 out_seq=0 2024-03-27T11:05:29.397 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.423+0000 7f67d3400640 10 client.36461 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T11:05:29.397 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.424+0000 7f67d3400640 1 -- 172.21.15.97:0/3629862803 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x558f839c9250 con 0x558f83a7db40 2024-03-27T11:05:29.397 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.424+0000 7f67d3400640 10 client.36461 handle_command_reply: tid=0 2024-03-27T11:05:29.397 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.424+0000 7f67f3384740 1 client.36461 shutdown 2024-03-27T11:05:29.397 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.425+0000 7f67f3384740 1 -- 172.21.15.97:0/3629862803 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558f83a6d4f0 msgr2=0x558f83a791f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:05:29.397 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.425+0000 7f67f3384740 1 --2- 172.21.15.97:0/3629862803 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558f83a6d4f0 0x558f83a791f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f67d80149c0 tx=0x7f67d8009060 comp rx=0 tx=0).stop 2024-03-27T11:05:29.397 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.425+0000 7f67f3384740 1 -- 172.21.15.97:0/3629862803 shutdown_connections 2024-03-27T11:05:29.397 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.425+0000 7f67f3384740 1 --2- 172.21.15.97:0/3629862803 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x558f83a7db40 0x558f83a7ff50 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:29.397 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.425+0000 7f67f3384740 1 --2- 172.21.15.97:0/3629862803 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558f83a6d4f0 0x558f83a791f0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:29.397 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.425+0000 7f67f3384740 1 --2- 172.21.15.97:0/3629862803 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558f83a68380 0x558f83a721b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:29.397 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.425+0000 7f67f3384740 1 --2- 172.21.15.97:0/3629862803 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558f83a67990 0x558f83a71c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:29.397 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.425+0000 7f67f3384740 1 -- 172.21.15.97:0/3629862803 >> 172.21.15.97:0/3629862803 conn(0x558f83a51ef0 msgr2=0x558f83a510f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:05:29.397 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.425+0000 7f67d3400640 0 client.36461 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T11:05:29.397 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.426+0000 7f67f3384740 1 -- 172.21.15.97:0/3629862803 shutdown_connections 2024-03-27T11:05:29.398 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.426+0000 7f67f3384740 1 -- 172.21.15.97:0/3629862803 wait complete. 2024-03-27T11:05:29.398 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.426+0000 7f67f3384740 20 client.36461 trim_cache size 0 max 16384 2024-03-27T11:05:29.426 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.456+0000 7f67f3384740 1 Processor -- start 2024-03-27T11:05:29.426 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.457+0000 7f67f3384740 1 -- start start 2024-03-27T11:05:29.427 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.457+0000 7f67f3384740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558f83a67990 0x558f83a522d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:05:29.427 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.457+0000 7f67f3384740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558f83a68380 0x558f83a79110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:05:29.427 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.457+0000 7f67f3384740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558f83a6d4f0 0x558f83a50860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:05:29.427 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.457+0000 7f67f3384740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f67e81a6e10 con 0x558f83a67990 2024-03-27T11:05:29.427 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.457+0000 7f67f3384740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f67e81a8950 con 0x558f83a68380 2024-03-27T11:05:29.427 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.457+0000 7f67f3384740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f67e81a74e0 con 0x558f83a6d4f0 2024-03-27T11:05:29.428 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.458+0000 7f67ed200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558f83a68380 0x558f83a79110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:05:29.428 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.458+0000 7f67ee600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558f83a6d4f0 0x558f83a50860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:05:29.428 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.458+0000 7f67ed200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558f83a68380 0x558f83a79110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:51676/0 (socket says 172.21.15.97:51676) 2024-03-27T11:05:29.428 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.458+0000 7f67ed200640 1 -- 172.21.15.97:0/3211227162 learned_addr learned my addr 172.21.15.97:0/3211227162 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:05:29.428 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.458+0000 7f67edc00640 1 --2- 172.21.15.97:0/3211227162 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558f83a67990 0x558f83a522d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:05:29.428 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.458+0000 7f67ed200640 1 -- 172.21.15.97:0/3211227162 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558f83a6d4f0 msgr2=0x558f83a50860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:05:29.428 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.458+0000 7f67ed200640 1 --2- 172.21.15.97:0/3211227162 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558f83a6d4f0 0x558f83a50860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:29.428 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.458+0000 7f67ed200640 1 -- 172.21.15.97:0/3211227162 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558f83a67990 msgr2=0x558f83a522d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:05:29.428 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.458+0000 7f67ed200640 1 --2- 172.21.15.97:0/3211227162 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558f83a67990 0x558f83a522d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:29.428 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.458+0000 7f67ed200640 1 -- 172.21.15.97:0/3211227162 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f8379c620 con 0x558f83a68380 2024-03-27T11:05:29.429 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.459+0000 7f67ed200640 1 --2- 172.21.15.97:0/3211227162 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558f83a68380 0x558f83a79110 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f67d4005ec0 tx=0x7f67d402ca40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69233b3e6b81cfa8 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:05:29.429 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.459+0000 7f67de000640 1 -- 172.21.15.97:0/3211227162 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d402cc70 con 0x558f83a68380 2024-03-27T11:05:29.429 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.460+0000 7f67de000640 1 -- 172.21.15.97:0/3211227162 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f67d4005040 con 0x558f83a68380 2024-03-27T11:05:29.429 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.460+0000 7f67de000640 1 -- 172.21.15.97:0/3211227162 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d402cc70 con 0x558f83a68380 2024-03-27T11:05:29.430 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.461+0000 7f67f3384740 1 -- 172.21.15.97:0/3211227162 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558f83a68380 msgr2=0x558f83a79110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:05:29.430 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.461+0000 7f67f3384740 1 --2- 172.21.15.97:0/3211227162 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558f83a68380 0x558f83a79110 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f67d4005ec0 tx=0x7f67d402ca40 comp rx=0 tx=0).stop 2024-03-27T11:05:29.430 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.461+0000 7f67f3384740 1 -- 172.21.15.97:0/3211227162 shutdown_connections 2024-03-27T11:05:29.430 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.461+0000 7f67f3384740 1 --2- 172.21.15.97:0/3211227162 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558f83a6d4f0 0x558f83a50860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:29.430 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.461+0000 7f67f3384740 1 --2- 172.21.15.97:0/3211227162 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558f83a68380 0x558f83a79110 secure :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f67d4005ec0 tx=0x7f67d402ca40 comp rx=0 tx=0).stop 2024-03-27T11:05:29.430 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.461+0000 7f67f3384740 1 --2- 172.21.15.97:0/3211227162 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558f83a67990 0x558f83a522d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:29.430 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.461+0000 7f67f3384740 1 -- 172.21.15.97:0/3211227162 >> 172.21.15.97:0/3211227162 conn(0x558f83a51ef0 msgr2=0x558f83a6cca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:05:29.431 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.462+0000 7f67f3384740 1 -- 172.21.15.97:0/3211227162 shutdown_connections 2024-03-27T11:05:29.431 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.462+0000 7f67f3384740 1 -- 172.21.15.97:0/3211227162 wait complete. 2024-03-27T11:05:29.431 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.462+0000 7f67f3384740 1 -- start start 2024-03-27T11:05:29.432 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.463+0000 7f67f3384740 1 Processor -- start 2024-03-27T11:05:29.434 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.463+0000 7f67f3384740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558f83a67990 0x558f83a74820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:05:29.434 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.463+0000 7f67f3384740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558f83a6d4f0 0x558f83a74d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:05:29.434 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.463+0000 7f67f3384740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558f83a77d80 0x558f83a78200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:05:29.435 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.463+0000 7f67f3384740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f67e81a8280 con 0x558f83a6d4f0 2024-03-27T11:05:29.435 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.463+0000 7f67f3384740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f67e81a7bb0 con 0x558f83a67990 2024-03-27T11:05:29.435 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.463+0000 7f67f3384740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f67e81a0680 con 0x558f83a77d80 2024-03-27T11:05:29.435 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.464+0000 7f67ee600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558f83a77d80 0x558f83a78200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:05:29.435 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.465+0000 7f67edc00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558f83a67990 0x558f83a74820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:05:29.435 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.465+0000 7f67f3384740 10 client.0 _ll_register_callbacks cb 0x7f67d8014640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T11:05:29.435 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.465+0000 7f67ee600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558f83a77d80 0x558f83a78200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.97:49700/0 (socket says 172.21.15.97:49700) 2024-03-27T11:05:29.435 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.465+0000 7f67ee600640 1 -- 172.21.15.97:0/2743162537 learned_addr learned my addr 172.21.15.97:0/2743162537 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:05:29.437 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.466+0000 7f67edc00640 1 -- 172.21.15.97:0/2743162537 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558f83a77d80 msgr2=0x558f83a78200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:05:29.437 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.466+0000 7f67edc00640 1 --2- 172.21.15.97:0/2743162537 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558f83a77d80 0x558f83a78200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:29.437 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.466+0000 7f67edc00640 1 -- 172.21.15.97:0/2743162537 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558f83a6d4f0 msgr2=0x558f83a74d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:05:29.437 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.466+0000 7f67edc00640 1 --2- 172.21.15.97:0/2743162537 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558f83a6d4f0 0x558f83a74d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:29.437 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.467+0000 7f67edc00640 1 -- 172.21.15.97:0/2743162537 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f83a78740 con 0x558f83a67990 2024-03-27T11:05:29.437 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.468+0000 7f67edc00640 1 --2- 172.21.15.97:0/2743162537 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558f83a67990 0x558f83a74820 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x558f83a68780 tx=0x7f67e009c700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d172462c05093b4 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:05:29.437 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.468+0000 7f67d3400640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-03-27T11:05:29.438 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.469+0000 7f67d3400640 1 -- 172.21.15.97:0/2743162537 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e0004310 con 0x558f83a67990 2024-03-27T11:05:29.439 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.469+0000 7f67d3400640 1 -- 172.21.15.97:0/2743162537 <== mon.2 v2:172.21.15.114:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f67e009ab00 con 0x558f83a67990 2024-03-27T11:05:29.439 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.469+0000 7f67f3384740 1 -- 172.21.15.97:0/2743162537 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67d4005040 con 0x558f83a67990 2024-03-27T11:05:29.439 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.469+0000 7f67d3400640 1 -- 172.21.15.97:0/2743162537 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f67e009cf00 con 0x558f83a67990 2024-03-27T11:05:29.439 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.470+0000 7f67f3384740 10 client.36467 fetch_fsmap learned FSMap version 34 2024-03-27T11:05:29.439 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.470+0000 7f67f3384740 1 -- 172.21.15.97:0/2743162537 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x558f83af5310 con 0x558f83a67990 2024-03-27T11:05:29.439 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.470+0000 7f67d3400640 1 -- 172.21.15.97:0/2743162537 <== mon.2 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f67d4005040 con 0x558f83a67990 2024-03-27T11:05:29.441 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.471+0000 7f67d3400640 1 -- 172.21.15.97:0/2743162537 <== mon.2 v2:172.21.15.114:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f67e00cf040 con 0x558f83a67990 2024-03-27T11:05:29.441 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.471+0000 7f67f3384740 10 client.36467 fetch_fsmap finished waiting for FSMap version 34 2024-03-27T11:05:29.441 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.471+0000 7f67f3384740 10 client.36467 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T11:05:29.441 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.471+0000 7f67f3384740 20 client.36467 populate_metadata read hostname 'smithi097' 2024-03-27T11:05:29.441 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.471+0000 7f67f3384740 1 --2- 172.21.15.97:0/2743162537 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x558f83af5880 0x558f83af7c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:05:29.441 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.471+0000 7f67f3384740 4 client.36467 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-27T11:05:29.441 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.471+0000 7f67f3384740 1 -- 172.21.15.97:0/2743162537 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558f839c35a0 con 0x558f83af5880 2024-03-27T11:05:29.441 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.471+0000 7f67ed200640 1 --2- 172.21.15.97:0/2743162537 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x558f83af5880 0x558f83af7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:05:29.442 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.473+0000 7f67ed200640 1 --2- 172.21.15.97:0/2743162537 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x558f83af5880 0x558f83af7c70 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=959530de15241f26 server_cookie=253c4007182ec084 in_seq=0 out_seq=0 2024-03-27T11:05:29.442 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.473+0000 7f67d3400640 10 client.36467 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T11:05:29.459 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.490+0000 7f67d3400640 1 -- 172.21.15.97:0/2743162537 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558f839c35a0 con 0x558f83af5880 2024-03-27T11:05:29.459 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.490+0000 7f67d3400640 10 client.36467 handle_command_reply: tid=0 2024-03-27T11:05:29.459 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.490+0000 7f67f3384740 1 client.36467 shutdown 2024-03-27T11:05:29.460 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.491+0000 7f67f3384740 1 -- 172.21.15.97:0/2743162537 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558f83a67990 msgr2=0x558f83a74820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:05:29.460 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.491+0000 7f67f3384740 1 --2- 172.21.15.97:0/2743162537 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558f83a67990 0x558f83a74820 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x558f83a68780 tx=0x7f67e009c700 comp rx=0 tx=0).stop 2024-03-27T11:05:29.460 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.491+0000 7f67f3384740 1 -- 172.21.15.97:0/2743162537 shutdown_connections 2024-03-27T11:05:29.460 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.491+0000 7f67f3384740 1 --2- 172.21.15.97:0/2743162537 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x558f83af5880 0x558f83af7c70 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:29.460 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.491+0000 7f67d3400640 0 client.36467 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T11:05:29.460 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.491+0000 7f67f3384740 1 --2- 172.21.15.97:0/2743162537 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558f83a77d80 0x558f83a78200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:29.460 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.491+0000 7f67f3384740 1 --2- 172.21.15.97:0/2743162537 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558f83a6d4f0 0x558f83a74d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:29.460 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.491+0000 7f67f3384740 1 --2- 172.21.15.97:0/2743162537 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558f83a67990 0x558f83a74820 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:29.460 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.491+0000 7f67f3384740 1 -- 172.21.15.97:0/2743162537 >> 172.21.15.97:0/2743162537 conn(0x558f83a51ef0 msgr2=0x558f83a6c870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:05:29.461 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.491+0000 7f67f3384740 1 -- 172.21.15.97:0/2743162537 shutdown_connections 2024-03-27T11:05:29.461 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.492+0000 7f67f3384740 1 -- 172.21.15.97:0/2743162537 wait complete. 2024-03-27T11:05:29.461 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.492+0000 7f67f3384740 20 client.36467 trim_cache size 0 max 16384 2024-03-27T11:05:29.461 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-27T11:05:29.462 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-27T11:05:29.462 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-27T11:05:29.462 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-27T11:05:29.462 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.493+0000 7f67f0600640 1 -- 172.21.15.97:0/3700744982 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f67cc0a3800 msgr2=0x7f67cc0a5cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:05:29.462 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.493+0000 7f67f0600640 1 --2- 172.21.15.97:0/3700744982 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f67cc0a3800 0x7f67cc0a5cc0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f67d8009fd0 tx=0x7f67d8008040 comp rx=0 tx=0).stop 2024-03-27T11:05:29.462 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.493+0000 7f67f0600640 1 -- 172.21.15.97:0/3700744982 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f67e8101ba0 msgr2=0x7f67e810e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:05:29.462 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.493+0000 7f67f0600640 1 --2- 172.21.15.97:0/3700744982 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f67e8101ba0 0x7f67e810e010 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f67e000eb70 tx=0x7f67e000c6a0 comp rx=0 tx=0).stop 2024-03-27T11:05:29.463 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.494+0000 7f67f0600640 1 -- 172.21.15.97:0/3700744982 shutdown_connections 2024-03-27T11:05:29.463 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.494+0000 7f67f0600640 1 --2- 172.21.15.97:0/3700744982 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67e8103750 0x7f67e810ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:29.463 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.494+0000 7f67f0600640 1 --2- 172.21.15.97:0/3700744982 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f67e8102d90 0x7f67e810e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:29.463 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.494+0000 7f67f0600640 1 --2- 172.21.15.97:0/3700744982 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f67cc0a3800 0x7f67cc0a5cc0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:29.463 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.494+0000 7f67f0600640 1 --2- 172.21.15.97:0/3700744982 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f67e8101ba0 0x7f67e810e010 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:29.464 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.494+0000 7f67f0600640 1 -- 172.21.15.97:0/3700744982 >> 172.21.15.97:0/3700744982 conn(0x7f67e80fd110 msgr2=0x7f67e8106090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:05:29.464 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.494+0000 7f67f0600640 1 -- 172.21.15.97:0/3700744982 shutdown_connections 2024-03-27T11:05:29.464 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:29.494+0000 7f67f0600640 1 -- 172.21.15.97:0/3700744982 wait complete. 2024-03-27T11:05:29.764 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:05:29 smithi114 ceph-mon[35338]: pgmap v706: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 51 MiB/s wr, 818 op/s 2024-03-27T11:05:29.888 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:05:29 smithi097 ceph-mon[31859]: pgmap v706: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 51 MiB/s wr, 818 op/s 2024-03-27T11:05:29.924 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:05:29 smithi177 ceph-mon[35294]: pgmap v706: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 51 MiB/s wr, 818 op/s 2024-03-27T11:05:31.764 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:05:31 smithi114 ceph-mon[35338]: pgmap v707: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 55 MiB/s wr, 883 op/s 2024-03-27T11:05:31.888 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:05:31 smithi097 ceph-mon[31859]: pgmap v707: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 55 MiB/s wr, 883 op/s 2024-03-27T11:05:31.924 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:05:31 smithi177 ceph-mon[35294]: pgmap v707: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 55 MiB/s wr, 883 op/s 2024-03-27T11:05:33.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:05:33 smithi114 ceph-mon[35338]: pgmap v708: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 50 MiB/s wr, 798 op/s 2024-03-27T11:05:33.889 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:05:33 smithi097 ceph-mon[31859]: pgmap v708: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 50 MiB/s wr, 798 op/s 2024-03-27T11:05:33.924 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:05:33 smithi177 ceph-mon[35294]: pgmap v708: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 50 MiB/s wr, 798 op/s 2024-03-27T11:05:35.764 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:05:35 smithi114 ceph-mon[35338]: pgmap v709: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 50 MiB/s wr, 816 op/s 2024-03-27T11:05:35.888 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:05:35 smithi097 ceph-mon[31859]: pgmap v709: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 50 MiB/s wr, 816 op/s 2024-03-27T11:05:35.924 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:05:35 smithi177 ceph-mon[35294]: pgmap v709: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 50 MiB/s wr, 816 op/s 2024-03-27T11:05:37.764 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:05:37 smithi114 ceph-mon[35338]: pgmap v710: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 48 MiB/s wr, 781 op/s 2024-03-27T11:05:37.889 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:05:37 smithi097 ceph-mon[31859]: pgmap v710: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 48 MiB/s wr, 781 op/s 2024-03-27T11:05:37.924 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:05:37 smithi177 ceph-mon[35294]: pgmap v710: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 48 MiB/s wr, 781 op/s 2024-03-27T11:05:39.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:05:39 smithi114 ceph-mon[35338]: pgmap v711: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 51 MiB/s wr, 824 op/s 2024-03-27T11:05:39.889 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:05:39 smithi097 ceph-mon[31859]: pgmap v711: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 51 MiB/s wr, 824 op/s 2024-03-27T11:05:39.924 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:05:39 smithi177 ceph-mon[35294]: pgmap v711: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 51 MiB/s wr, 824 op/s 2024-03-27T11:05:41.764 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:05:41 smithi114 ceph-mon[35338]: pgmap v712: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 55 MiB/s wr, 886 op/s 2024-03-27T11:05:41.889 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:05:41 smithi097 ceph-mon[31859]: pgmap v712: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 55 MiB/s wr, 886 op/s 2024-03-27T11:05:41.924 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:05:41 smithi177 ceph-mon[35294]: pgmap v712: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 55 MiB/s wr, 886 op/s 2024-03-27T11:05:42.696 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.725+0000 7f9da5c00640 1 -- 172.21.15.97:0/40302555 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9da0103bd0 msgr2=0x7f9da0103ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:05:42.696 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.725+0000 7f9da5c00640 1 --2- 172.21.15.97:0/40302555 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9da0103bd0 0x7f9da0103ff0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f9d90009900 tx=0x7f9d9002bc30 comp rx=0 tx=0).stop 2024-03-27T11:05:42.697 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.726+0000 7f9da5c00640 1 -- 172.21.15.97:0/40302555 shutdown_connections 2024-03-27T11:05:42.697 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.726+0000 7f9da5c00640 1 --2- 172.21.15.97:0/40302555 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9da0105780 0x7f9da010c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:42.697 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.726+0000 7f9da5c00640 1 --2- 172.21.15.97:0/40302555 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9da0104dc0 0x7f9da0105240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:42.697 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.726+0000 7f9da5c00640 1 --2- 172.21.15.97:0/40302555 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9da0103bd0 0x7f9da0103ff0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:42.697 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.726+0000 7f9da5c00640 1 -- 172.21.15.97:0/40302555 >> 172.21.15.97:0/40302555 conn(0x7f9da00ff160 msgr2=0x7f9da0101580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:05:42.697 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.726+0000 7f9da5c00640 1 -- 172.21.15.97:0/40302555 shutdown_connections 2024-03-27T11:05:42.697 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.727+0000 7f9da5c00640 1 -- 172.21.15.97:0/40302555 wait complete. 2024-03-27T11:05:42.697 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.728+0000 7f9da5c00640 1 Processor -- start 2024-03-27T11:05:42.698 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.728+0000 7f9da5c00640 1 -- start start 2024-03-27T11:05:42.699 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.728+0000 7f9da5c00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9da0103bd0 0x7f9da0199e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:05:42.699 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.728+0000 7f9da5c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9da0104dc0 0x7f9da019a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:05:42.699 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.728+0000 7f9da5c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9da0105780 0x7f9da01a13d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:05:42.699 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.729+0000 7f9da5c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9da010f720 con 0x7f9da0104dc0 2024-03-27T11:05:42.699 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.729+0000 7f9da5c00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9da010f5a0 con 0x7f9da0105780 2024-03-27T11:05:42.699 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.729+0000 7f9da5c00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f9da010f8a0 con 0x7f9da0103bd0 2024-03-27T11:05:42.699 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.729+0000 7f9da5200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9da0105780 0x7f9da01a13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:05:42.700 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.729+0000 7f9da5200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9da0105780 0x7f9da01a13d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:35768/0 (socket says 172.21.15.97:35768) 2024-03-27T11:05:42.700 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.729+0000 7f9da5200640 1 -- 172.21.15.97:0/2347407667 learned_addr learned my addr 172.21.15.97:0/2347407667 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:05:42.700 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.729+0000 7f9d9f400640 1 --2- 172.21.15.97:0/2347407667 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9da0104dc0 0x7f9da019a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:05:42.700 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.730+0000 7f9da5200640 1 -- 172.21.15.97:0/2347407667 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9da0103bd0 msgr2=0x7f9da0199e20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-27T11:05:42.700 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.730+0000 7f9da5200640 1 --2- 172.21.15.97:0/2347407667 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9da0103bd0 0x7f9da0199e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:42.700 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.730+0000 7f9da5200640 1 -- 172.21.15.97:0/2347407667 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9da0104dc0 msgr2=0x7f9da019a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:05:42.701 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.730+0000 7f9da5200640 1 --2- 172.21.15.97:0/2347407667 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9da0104dc0 0x7f9da019a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:42.701 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.730+0000 7f9da5200640 1 -- 172.21.15.97:0/2347407667 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9da01a1a40 con 0x7f9da0105780 2024-03-27T11:05:42.701 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.730+0000 7f9d9f400640 1 --2- 172.21.15.97:0/2347407667 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9da0104dc0 0x7f9da019a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T11:05:42.701 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.730+0000 7f9da5200640 1 --2- 172.21.15.97:0/2347407667 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9da0105780 0x7f9da01a13d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f9d98009fd0 tx=0x7f9d9800b260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:05:42.701 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.731+0000 7f9d9cc00640 1 -- 172.21.15.97:0/2347407667 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d98017070 con 0x7f9da0105780 2024-03-27T11:05:42.704 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.731+0000 7f9da5c00640 1 -- 172.21.15.97:0/2347407667 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9da01a1d30 con 0x7f9da0105780 2024-03-27T11:05:42.704 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.731+0000 7f9da5c00640 1 -- 172.21.15.97:0/2347407667 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9da01a2220 con 0x7f9da0105780 2024-03-27T11:05:42.704 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.731+0000 7f9d9cc00640 1 -- 172.21.15.97:0/2347407667 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d98003e20 con 0x7f9da0105780 2024-03-27T11:05:42.704 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.731+0000 7f9d9cc00640 1 -- 172.21.15.97:0/2347407667 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d98010460 con 0x7f9da0105780 2024-03-27T11:05:42.705 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.735+0000 7f9daa037740 1 Processor -- start 2024-03-27T11:05:42.707 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.736+0000 7f9daa037740 1 -- start start 2024-03-27T11:05:42.707 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.736+0000 7f9daa037740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559dd4be0370 0x559dd4be0790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:05:42.707 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.736+0000 7f9daa037740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559dd4be0d60 0x559dd4be5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:05:42.707 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.736+0000 7f9daa037740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559dd4be5ed0 0x559dd4be8360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:05:42.707 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.736+0000 7f9daa037740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x559dd4b3cd10 con 0x559dd4be0d60 2024-03-27T11:05:42.707 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.736+0000 7f9daa037740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x559dd4ae5260 con 0x559dd4be5ed0 2024-03-27T11:05:42.708 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.736+0000 7f9daa037740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x559dd4b38cc0 con 0x559dd4be0370 2024-03-27T11:05:42.710 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.739+0000 7f9d9cc00640 1 -- 172.21.15.97:0/2347407667 <== mon.2 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f9d980106c0 con 0x7f9da0105780 2024-03-27T11:05:42.710 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.739+0000 7f9d9f400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559dd4be0370 0x559dd4be0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:05:42.710 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.739+0000 7f9d9f400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559dd4be0370 0x559dd4be0790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.97:54974/0 (socket says 172.21.15.97:54974) 2024-03-27T11:05:42.710 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.739+0000 7f9d9f400640 1 -- 172.21.15.97:0/877565820 learned_addr learned my addr 172.21.15.97:0/877565820 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:05:42.710 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.739+0000 7f9d9f400640 1 --2- 172.21.15.97:0/877565820 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559dd4be5ed0 0x559dd4be8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:05:42.710 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.739+0000 7f9d9f400640 1 -- 172.21.15.97:0/877565820 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559dd4be5ed0 msgr2=0x559dd4be8360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:05:42.710 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.739+0000 7f9d9f400640 1 --2- 172.21.15.97:0/877565820 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559dd4be5ed0 0x559dd4be8360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:42.710 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.739+0000 7f9d9f400640 1 -- 172.21.15.97:0/877565820 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559dd4be0d60 msgr2=0x559dd4be5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:05:42.710 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.739+0000 7f9d9f400640 1 --2- 172.21.15.97:0/877565820 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559dd4be0d60 0x559dd4be5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:42.710 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.739+0000 7f9d9f400640 1 -- 172.21.15.97:0/877565820 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559dd4adcaa0 con 0x559dd4be0370 2024-03-27T11:05:42.711 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.740+0000 7f9d9f400640 1 --2- 172.21.15.97:0/877565820 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559dd4be0370 0x559dd4be0790 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c012820 tx=0x7f9d8c009720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=849b1478b04fe7e5 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:05:42.711 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.741+0000 7f9d9cc00640 1 --2- 172.21.15.97:0/2347407667 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f9d840a3740 0x7f9d840a5c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:05:42.711 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.742+0000 7f9d9cc00640 1 -- 172.21.15.97:0/2347407667 <== mon.2 v2:172.21.15.114:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f9d980d0a50 con 0x7f9da0105780 2024-03-27T11:05:42.712 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.742+0000 7f9d95600640 1 -- 172.21.15.97:0/877565820 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d8c012e60 con 0x559dd4be0370 2024-03-27T11:05:42.712 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.742+0000 7f9d95600640 1 -- 172.21.15.97:0/877565820 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d8c0107d0 con 0x559dd4be0370 2024-03-27T11:05:42.712 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.742+0000 7f9d95600640 1 -- 172.21.15.97:0/877565820 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d8c003850 con 0x559dd4be0370 2024-03-27T11:05:42.712 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.742+0000 7f9d9fe00640 1 --2- 172.21.15.97:0/2347407667 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f9d840a3740 0x7f9d840a5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:05:42.713 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.743+0000 7f9daa037740 1 -- 172.21.15.97:0/877565820 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559dd4be0370 msgr2=0x559dd4be0790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:05:42.713 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.743+0000 7f9daa037740 1 --2- 172.21.15.97:0/877565820 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559dd4be0370 0x559dd4be0790 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c012820 tx=0x7f9d8c009720 comp rx=0 tx=0).stop 2024-03-27T11:05:42.714 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.744+0000 7f9daa037740 1 -- 172.21.15.97:0/877565820 shutdown_connections 2024-03-27T11:05:42.714 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.744+0000 7f9daa037740 1 --2- 172.21.15.97:0/877565820 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559dd4be5ed0 0x559dd4be8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:42.717 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.744+0000 7f9daa037740 1 --2- 172.21.15.97:0/877565820 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559dd4be0d60 0x559dd4be5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:42.717 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.744+0000 7f9daa037740 1 --2- 172.21.15.97:0/877565820 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559dd4be0370 0x559dd4be0790 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:42.717 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.744+0000 7f9daa037740 1 -- 172.21.15.97:0/877565820 >> 172.21.15.97:0/877565820 conn(0x559dd4bd30d0 msgr2=0x559dd4bd34d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:05:42.717 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.744+0000 7f9d9fe00640 1 --2- 172.21.15.97:0/2347407667 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f9d840a3740 0x7f9d840a5c00 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f9d90009230 tx=0x7f9d90000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:05:42.717 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.747+0000 7f9daa037740 1 -- 172.21.15.97:0/877565820 shutdown_connections 2024-03-27T11:05:42.717 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.747+0000 7f9daa037740 1 -- 172.21.15.97:0/877565820 wait complete. 2024-03-27T11:05:42.718 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.748+0000 7f9daa037740 1 -- start start 2024-03-27T11:05:42.718 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.748+0000 7f9daa037740 1 Processor -- start 2024-03-27T11:05:42.719 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.749+0000 7f9daa037740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559dd4be0370 0x559dd4bd9750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:05:42.719 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.749+0000 7f9daa037740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559dd4be0d60 0x559dd4bd9c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:05:42.720 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.750+0000 7f9d9f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559dd4be0d60 0x559dd4bd9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:05:42.720 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.750+0000 7f9d9fe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559dd4be0370 0x559dd4bd9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:05:42.721 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.750+0000 7f9d9fe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559dd4be0370 0x559dd4bd9750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:35784/0 (socket says 172.21.15.97:35784) 2024-03-27T11:05:42.721 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.750+0000 7f9d9f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559dd4be0d60 0x559dd4bd9c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47196/0 (socket says 172.21.15.97:47196) 2024-03-27T11:05:42.721 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.751+0000 7f9d9fe00640 1 -- 172.21.15.97:0/3228119946 learned_addr learned my addr 172.21.15.97:0/3228119946 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:05:42.721 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.751+0000 7f9daa037740 1 --2- 172.21.15.97:0/3228119946 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559dd4be5ed0 0x559dd4bdecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:05:42.721 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.751+0000 7f9daa037740 1 -- 172.21.15.97:0/3228119946 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x559dd4bd7c60 con 0x559dd4be0d60 2024-03-27T11:05:42.724 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.751+0000 7f9da5200640 1 --2- 172.21.15.97:0/3228119946 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559dd4be5ed0 0x559dd4bdecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:05:42.724 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.751+0000 7f9daa037740 1 -- 172.21.15.97:0/3228119946 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x559dd4af8630 con 0x559dd4be0370 2024-03-27T11:05:42.724 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.751+0000 7f9daa037740 1 -- 172.21.15.97:0/3228119946 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x559dd4bd7de0 con 0x559dd4be5ed0 2024-03-27T11:05:42.724 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.752+0000 7f9daa037740 10 client.0 _ll_register_callbacks cb 0x559dd4afdc50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T11:05:42.724 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.754+0000 7f9da5200640 1 -- 172.21.15.97:0/3228119946 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559dd4be0370 msgr2=0x559dd4bd9750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:05:42.725 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.754+0000 7f9da5200640 1 --2- 172.21.15.97:0/3228119946 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559dd4be0370 0x559dd4bd9750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:42.725 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.754+0000 7f9da5200640 1 -- 172.21.15.97:0/3228119946 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559dd4be0d60 msgr2=0x559dd4bd9c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:05:42.725 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.754+0000 7f9da5200640 1 --2- 172.21.15.97:0/3228119946 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559dd4be0d60 0x559dd4bd9c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:42.725 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.754+0000 7f9da5200640 1 -- 172.21.15.97:0/3228119946 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559dd4b045f0 con 0x559dd4be5ed0 2024-03-27T11:05:42.725 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.754+0000 7f9d9f400640 1 --2- 172.21.15.97:0/3228119946 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559dd4be0d60 0x559dd4bd9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T11:05:42.725 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.755+0000 7f9da5200640 1 --2- 172.21.15.97:0/3228119946 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559dd4be5ed0 0x559dd4bdecc0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9d9800dfd0 tx=0x7f9d9809d710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7445d26756db87ee server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:05:42.726 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.755+0000 7f9d8a000640 10 client.0 ms_handle_connect on v2:172.21.15.177:3300/0 2024-03-27T11:05:42.726 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.756+0000 7f9d8a000640 1 -- 172.21.15.97:0/3228119946 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d98020040 con 0x559dd4be5ed0 2024-03-27T11:05:42.727 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.757+0000 7f9d8a000640 1 -- 172.21.15.97:0/3228119946 <== mon.1 v2:172.21.15.177:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f9d980bd070 con 0x559dd4be5ed0 2024-03-27T11:05:42.727 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.757+0000 7f9daa037740 1 -- 172.21.15.97:0/3228119946 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d8c0107d0 con 0x559dd4be5ed0 2024-03-27T11:05:42.728 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.758+0000 7f9d8a000640 1 -- 172.21.15.97:0/3228119946 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f9d980b6200 con 0x559dd4be5ed0 2024-03-27T11:05:42.728 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.758+0000 7f9daa037740 10 client.26203 fetch_fsmap learned FSMap version 34 2024-03-27T11:05:42.728 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.758+0000 7f9daa037740 1 -- 172.21.15.97:0/3228119946 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x559dd4851210 con 0x559dd4be5ed0 2024-03-27T11:05:42.729 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.758+0000 7f9d8a000640 1 -- 172.21.15.97:0/3228119946 <== mon.1 v2:172.21.15.177:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d8c0107d0 con 0x559dd4be5ed0 2024-03-27T11:05:42.729 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.758+0000 7f9d8a000640 1 -- 172.21.15.97:0/3228119946 <== mon.1 v2:172.21.15.177:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f9d98098c20 con 0x559dd4be5ed0 2024-03-27T11:05:42.729 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.759+0000 7f9daa037740 10 client.26203 fetch_fsmap finished waiting for FSMap version 34 2024-03-27T11:05:42.729 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.759+0000 7f9daa037740 10 client.26203 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T11:05:42.729 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.759+0000 7f9daa037740 20 client.26203 populate_metadata read hostname 'smithi097' 2024-03-27T11:05:42.729 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.759+0000 7f9daa037740 1 --2- 172.21.15.97:0/3228119946 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x559dd4bfef70 0x559dd4c013b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:05:42.729 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.759+0000 7f9daa037740 4 client.26203 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T11:05:42.729 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.759+0000 7f9daa037740 1 -- 172.21.15.97:0/3228119946 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559dd4b4a250 con 0x559dd4bfef70 2024-03-27T11:05:42.730 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.760+0000 7f9d9fe00640 1 --2- 172.21.15.97:0/3228119946 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x559dd4bfef70 0x559dd4c013b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:05:42.731 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.761+0000 7f9d9fe00640 1 --2- 172.21.15.97:0/3228119946 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x559dd4bfef70 0x559dd4c013b0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b7ed0e1378ddeab server_cookie=6fbcb4835ced335f in_seq=0 out_seq=0 2024-03-27T11:05:42.731 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.761+0000 7f9d8a000640 10 client.26203 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T11:05:42.732 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.762+0000 7f9d8a000640 1 -- 172.21.15.97:0/3228119946 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x559dd4b4a250 con 0x559dd4bfef70 2024-03-27T11:05:42.732 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.762+0000 7f9d8a000640 10 client.26203 handle_command_reply: tid=0 2024-03-27T11:05:42.732 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.762+0000 7f9daa037740 1 client.26203 shutdown 2024-03-27T11:05:42.733 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.763+0000 7f9daa037740 1 -- 172.21.15.97:0/3228119946 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559dd4be5ed0 msgr2=0x559dd4bdecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:05:42.733 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.763+0000 7f9daa037740 1 --2- 172.21.15.97:0/3228119946 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559dd4be5ed0 0x559dd4bdecc0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9d9800dfd0 tx=0x7f9d9809d710 comp rx=0 tx=0).stop 2024-03-27T11:05:42.733 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.763+0000 7f9daa037740 1 -- 172.21.15.97:0/3228119946 shutdown_connections 2024-03-27T11:05:42.733 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.763+0000 7f9daa037740 1 --2- 172.21.15.97:0/3228119946 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x559dd4bfef70 0x559dd4c013b0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:42.733 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.763+0000 7f9daa037740 1 --2- 172.21.15.97:0/3228119946 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559dd4be5ed0 0x559dd4bdecc0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:42.733 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.763+0000 7f9daa037740 1 --2- 172.21.15.97:0/3228119946 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559dd4be0d60 0x559dd4bd9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:42.733 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.763+0000 7f9daa037740 1 --2- 172.21.15.97:0/3228119946 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559dd4be0370 0x559dd4bd9750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:42.734 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.763+0000 7f9daa037740 1 -- 172.21.15.97:0/3228119946 >> 172.21.15.97:0/3228119946 conn(0x559dd4bd30d0 msgr2=0x559dd4bd2310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:05:42.734 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.763+0000 7f9d8a000640 0 client.26203 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T11:05:42.734 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.764+0000 7f9daa037740 1 -- 172.21.15.97:0/3228119946 shutdown_connections 2024-03-27T11:05:42.734 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.764+0000 7f9daa037740 1 -- 172.21.15.97:0/3228119946 wait complete. 2024-03-27T11:05:42.735 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.764+0000 7f9daa037740 20 client.26203 trim_cache size 0 max 16384 2024-03-27T11:05:42.759 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.789+0000 7f9daa037740 1 Processor -- start 2024-03-27T11:05:42.759 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.789+0000 7f9daa037740 1 -- start start 2024-03-27T11:05:42.759 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.789+0000 7f9daa037740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559dd4be0370 0x559dd4be15f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:05:42.759 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.789+0000 7f9daa037740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559dd4be0d60 0x559dd4bd6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:05:42.760 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.789+0000 7f9daa037740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559dd4be5ed0 0x559dd4bd8230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:05:42.760 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.789+0000 7f9daa037740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9d8000e380 con 0x559dd4be5ed0 2024-03-27T11:05:42.760 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.789+0000 7f9daa037740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9d8000fec0 con 0x559dd4be0d60 2024-03-27T11:05:42.760 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.789+0000 7f9daa037740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f9d8000ea50 con 0x559dd4be0370 2024-03-27T11:05:42.760 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.790+0000 7f9da5200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559dd4be5ed0 0x559dd4bd8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:05:42.760 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.790+0000 7f9d9fe00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559dd4be0370 0x559dd4be15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:05:42.760 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.790+0000 7f9da5200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559dd4be5ed0 0x559dd4bd8230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47204/0 (socket says 172.21.15.97:47204) 2024-03-27T11:05:42.761 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.790+0000 7f9da5200640 1 -- 172.21.15.97:0/3876425479 learned_addr learned my addr 172.21.15.97:0/3876425479 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:05:42.761 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.790+0000 7f9d9f400640 1 --2- 172.21.15.97:0/3876425479 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559dd4be0d60 0x559dd4bd6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:05:42.761 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.790+0000 7f9d9fe00640 1 -- 172.21.15.97:0/3876425479 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559dd4be0d60 msgr2=0x559dd4bd6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:05:42.761 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.790+0000 7f9d9fe00640 1 --2- 172.21.15.97:0/3876425479 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559dd4be0d60 0x559dd4bd6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:42.761 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.790+0000 7f9d9fe00640 1 -- 172.21.15.97:0/3876425479 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559dd4be5ed0 msgr2=0x559dd4bd8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:05:42.761 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.791+0000 7f9d9fe00640 1 --2- 172.21.15.97:0/3876425479 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559dd4be5ed0 0x559dd4bd8230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:42.762 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.791+0000 7f9d9fe00640 1 -- 172.21.15.97:0/3876425479 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559dd491d620 con 0x559dd4be0370 2024-03-27T11:05:42.762 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.791+0000 7f9da5200640 1 --2- 172.21.15.97:0/3876425479 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559dd4be5ed0 0x559dd4bd8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T11:05:42.762 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.792+0000 7f9d9fe00640 1 --2- 172.21.15.97:0/3876425479 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559dd4be0370 0x559dd4be15f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f9d9002bc70 tx=0x7f9d9003b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b449b55fd52467b server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:05:42.762 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.792+0000 7f9d94c00640 1 -- 172.21.15.97:0/3876425479 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d90045040 con 0x559dd4be0370 2024-03-27T11:05:42.763 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.792+0000 7f9d94c00640 1 -- 172.21.15.97:0/3876425479 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d90055070 con 0x559dd4be0370 2024-03-27T11:05:42.764 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.793+0000 7f9daa037740 1 -- 172.21.15.97:0/3876425479 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559dd4be0370 msgr2=0x559dd4be15f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:05:42.764 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.793+0000 7f9d94c00640 1 -- 172.21.15.97:0/3876425479 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d9003f470 con 0x559dd4be0370 2024-03-27T11:05:42.764 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.794+0000 7f9daa037740 1 --2- 172.21.15.97:0/3876425479 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559dd4be0370 0x559dd4be15f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f9d9002bc70 tx=0x7f9d9003b000 comp rx=0 tx=0).stop 2024-03-27T11:05:42.764 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.794+0000 7f9daa037740 1 -- 172.21.15.97:0/3876425479 shutdown_connections 2024-03-27T11:05:42.765 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.794+0000 7f9daa037740 1 --2- 172.21.15.97:0/3876425479 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559dd4be5ed0 0x559dd4bd8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:42.765 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.794+0000 7f9daa037740 1 --2- 172.21.15.97:0/3876425479 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559dd4be0d60 0x559dd4bd6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:42.765 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.794+0000 7f9daa037740 1 --2- 172.21.15.97:0/3876425479 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559dd4be0370 0x559dd4be15f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:42.765 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.794+0000 7f9daa037740 1 -- 172.21.15.97:0/3876425479 >> 172.21.15.97:0/3876425479 conn(0x559dd4bd30d0 msgr2=0x559dd4bd6330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:05:42.766 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.795+0000 7f9daa037740 1 -- 172.21.15.97:0/3876425479 shutdown_connections 2024-03-27T11:05:42.766 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.795+0000 7f9daa037740 1 -- 172.21.15.97:0/3876425479 wait complete. 2024-03-27T11:05:42.766 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.796+0000 7f9daa037740 1 -- start start 2024-03-27T11:05:42.766 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.796+0000 7f9daa037740 1 Processor -- start 2024-03-27T11:05:42.767 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.796+0000 7f9daa037740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559dd4be0370 0x559dd4c10080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:05:42.767 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.797+0000 7f9daa037740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559dd4be0d60 0x559dd4c105c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:05:42.767 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.797+0000 7f9d9fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559dd4be0370 0x559dd4c10080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:05:42.768 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.797+0000 7f9d9fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559dd4be0370 0x559dd4c10080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47220/0 (socket says 172.21.15.97:47220) 2024-03-27T11:05:42.768 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.797+0000 7f9d9fe00640 1 -- 172.21.15.97:0/1483376865 learned_addr learned my addr 172.21.15.97:0/1483376865 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:05:42.768 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.797+0000 7f9daa037740 1 --2- 172.21.15.97:0/1483376865 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559dd4be5ed0 0x559dd4c115d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:05:42.768 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.797+0000 7f9daa037740 1 -- 172.21.15.97:0/1483376865 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9d8000f7f0 con 0x559dd4be0370 2024-03-27T11:05:42.768 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.797+0000 7f9daa037740 1 -- 172.21.15.97:0/1483376865 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9d8000f120 con 0x559dd4be5ed0 2024-03-27T11:05:42.768 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.797+0000 7f9daa037740 1 -- 172.21.15.97:0/1483376865 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f9d8000c360 con 0x559dd4be0d60 2024-03-27T11:05:42.768 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.797+0000 7f9d9f400640 1 --2- 172.21.15.97:0/1483376865 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559dd4be0d60 0x559dd4c105c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:05:42.769 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.798+0000 7f9da5200640 1 --2- 172.21.15.97:0/1483376865 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559dd4be5ed0 0x559dd4c115d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:05:42.769 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.798+0000 7f9daa037740 10 client.0 _ll_register_callbacks cb 0x7f9d9800bea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T11:05:42.770 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.798+0000 7f9da5200640 1 -- 172.21.15.97:0/1483376865 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559dd4be0d60 msgr2=0x559dd4c105c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:05:42.770 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.798+0000 7f9da5200640 1 --2- 172.21.15.97:0/1483376865 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559dd4be0d60 0x559dd4c105c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:42.770 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.798+0000 7f9da5200640 1 -- 172.21.15.97:0/1483376865 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559dd4be0370 msgr2=0x559dd4c10080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:05:42.770 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.798+0000 7f9da5200640 1 --2- 172.21.15.97:0/1483376865 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559dd4be0370 0x559dd4c10080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:42.770 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.798+0000 7f9da5200640 1 -- 172.21.15.97:0/1483376865 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559dd4c11bc0 con 0x559dd4be5ed0 2024-03-27T11:05:42.770 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.798+0000 7f9d9fe00640 1 --2- 172.21.15.97:0/1483376865 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559dd4be0370 0x559dd4c10080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T11:05:42.770 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.799+0000 7f9da5200640 1 --2- 172.21.15.97:0/1483376865 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559dd4be5ed0 0x559dd4c115d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9d98099330 tx=0x7f9d98099750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a172017c01c9fe3 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:05:42.770 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.799+0000 7f9d8a000640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-03-27T11:05:42.770 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.799+0000 7f9d8a000640 1 -- 172.21.15.97:0/1483376865 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d98020070 con 0x559dd4be5ed0 2024-03-27T11:05:42.770 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.799+0000 7f9d8a000640 1 -- 172.21.15.97:0/1483376865 <== mon.2 v2:172.21.15.114:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f9d980bd270 con 0x559dd4be5ed0 2024-03-27T11:05:42.770 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.799+0000 7f9daa037740 1 -- 172.21.15.97:0/1483376865 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d90055070 con 0x559dd4be5ed0 2024-03-27T11:05:42.771 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.800+0000 7f9d8a000640 1 -- 172.21.15.97:0/1483376865 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f9d9809cca0 con 0x559dd4be5ed0 2024-03-27T11:05:42.771 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.800+0000 7f9daa037740 10 client.36485 fetch_fsmap learned FSMap version 34 2024-03-27T11:05:42.772 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.800+0000 7f9daa037740 1 -- 172.21.15.97:0/1483376865 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x559dd4c651d0 con 0x559dd4be5ed0 2024-03-27T11:05:42.772 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.800+0000 7f9d8a000640 1 -- 172.21.15.97:0/1483376865 <== mon.2 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d90055070 con 0x559dd4be5ed0 2024-03-27T11:05:42.772 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.800+0000 7f9d8a000640 1 -- 172.21.15.97:0/1483376865 <== mon.2 v2:172.21.15.114:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f9d98083020 con 0x559dd4be5ed0 2024-03-27T11:05:42.772 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.801+0000 7f9daa037740 10 client.36485 fetch_fsmap finished waiting for FSMap version 34 2024-03-27T11:05:42.772 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.801+0000 7f9daa037740 10 client.36485 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T11:05:42.772 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.801+0000 7f9daa037740 20 client.36485 populate_metadata read hostname 'smithi097' 2024-03-27T11:05:42.772 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.801+0000 7f9daa037740 1 --2- 172.21.15.97:0/1483376865 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x559dd4c65b40 0x559dd4c67f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:05:42.772 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.801+0000 7f9daa037740 4 client.36485 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-27T11:05:42.772 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.801+0000 7f9daa037740 1 -- 172.21.15.97:0/1483376865 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559dd4b445a0 con 0x559dd4c65b40 2024-03-27T11:05:42.772 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.802+0000 7f9d9fe00640 1 --2- 172.21.15.97:0/1483376865 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x559dd4c65b40 0x559dd4c67f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:05:42.773 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.802+0000 7f9d9fe00640 1 --2- 172.21.15.97:0/1483376865 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x559dd4c65b40 0x559dd4c67f60 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d6b8d8b6a87a96e server_cookie=2076eba73cf737df in_seq=0 out_seq=0 2024-03-27T11:05:42.773 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.802+0000 7f9d8a000640 10 client.36485 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T11:05:42.780 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.810+0000 7f9d8a000640 1 -- 172.21.15.97:0/1483376865 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559dd4b445a0 con 0x559dd4c65b40 2024-03-27T11:05:42.780 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.810+0000 7f9d8a000640 10 client.36485 handle_command_reply: tid=0 2024-03-27T11:05:42.780 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.810+0000 7f9daa037740 1 client.36485 shutdown 2024-03-27T11:05:42.781 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.811+0000 7f9daa037740 1 -- 172.21.15.97:0/1483376865 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559dd4be5ed0 msgr2=0x559dd4c115d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:05:42.781 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.811+0000 7f9daa037740 1 --2- 172.21.15.97:0/1483376865 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559dd4be5ed0 0x559dd4c115d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9d98099330 tx=0x7f9d98099750 comp rx=0 tx=0).stop 2024-03-27T11:05:42.782 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.811+0000 7f9daa037740 1 -- 172.21.15.97:0/1483376865 shutdown_connections 2024-03-27T11:05:42.782 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.812+0000 7f9daa037740 1 --2- 172.21.15.97:0/1483376865 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x559dd4c65b40 0x559dd4c67f60 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:42.782 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.812+0000 7f9daa037740 1 --2- 172.21.15.97:0/1483376865 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559dd4be5ed0 0x559dd4c115d0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:42.782 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.812+0000 7f9daa037740 1 --2- 172.21.15.97:0/1483376865 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559dd4be0d60 0x559dd4c105c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:42.782 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.812+0000 7f9daa037740 1 --2- 172.21.15.97:0/1483376865 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559dd4be0370 0x559dd4c10080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:42.782 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.812+0000 7f9daa037740 1 -- 172.21.15.97:0/1483376865 >> 172.21.15.97:0/1483376865 conn(0x559dd4bd30d0 msgr2=0x559dd4bd6bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:05:42.782 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.812+0000 7f9d8a000640 0 client.36485 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T11:05:42.783 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.812+0000 7f9daa037740 1 -- 172.21.15.97:0/1483376865 shutdown_connections 2024-03-27T11:05:42.783 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.813+0000 7f9daa037740 1 -- 172.21.15.97:0/1483376865 wait complete. 2024-03-27T11:05:42.784 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.814+0000 7f9daa037740 20 client.36485 trim_cache size 0 max 16384 2024-03-27T11:05:42.784 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-27T11:05:42.784 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-27T11:05:42.785 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-27T11:05:42.785 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-27T11:05:42.785 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.814+0000 7f9da5c00640 1 -- 172.21.15.97:0/2347407667 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f9d840a3740 msgr2=0x7f9d840a5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:05:42.785 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.814+0000 7f9da5c00640 1 --2- 172.21.15.97:0/2347407667 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f9d840a3740 0x7f9d840a5c00 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f9d90009230 tx=0x7f9d90000bc0 comp rx=0 tx=0).stop 2024-03-27T11:05:42.785 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.814+0000 7f9da5c00640 1 -- 172.21.15.97:0/2347407667 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9da0105780 msgr2=0x7f9da01a13d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:05:42.785 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.814+0000 7f9da5c00640 1 --2- 172.21.15.97:0/2347407667 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9da0105780 0x7f9da01a13d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f9d98009fd0 tx=0x7f9d9800b260 comp rx=0 tx=0).stop 2024-03-27T11:05:42.787 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.817+0000 7f9da5c00640 1 -- 172.21.15.97:0/2347407667 shutdown_connections 2024-03-27T11:05:42.787 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.817+0000 7f9da5c00640 1 --2- 172.21.15.97:0/2347407667 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f9d840a3740 0x7f9d840a5c00 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:42.787 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.817+0000 7f9da5c00640 1 --2- 172.21.15.97:0/2347407667 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9da0105780 0x7f9da01a13d0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:42.788 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.817+0000 7f9da5c00640 1 --2- 172.21.15.97:0/2347407667 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9da0104dc0 0x7f9da019a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:42.788 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.817+0000 7f9da5c00640 1 --2- 172.21.15.97:0/2347407667 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9da0103bd0 0x7f9da0199e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:42.788 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.817+0000 7f9da5c00640 1 -- 172.21.15.97:0/2347407667 >> 172.21.15.97:0/2347407667 conn(0x7f9da00ff160 msgr2=0x7f9da0108190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:05:42.788 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.818+0000 7f9da5c00640 1 -- 172.21.15.97:0/2347407667 shutdown_connections 2024-03-27T11:05:42.789 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:42.818+0000 7f9da5c00640 1 -- 172.21.15.97:0/2347407667 wait complete. 2024-03-27T11:05:43.764 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:05:43 smithi114 ceph-mon[35338]: pgmap v713: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 49 MiB/s wr, 799 op/s 2024-03-27T11:05:43.889 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:05:43 smithi097 ceph-mon[31859]: pgmap v713: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 49 MiB/s wr, 799 op/s 2024-03-27T11:05:43.924 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:05:43 smithi177 ceph-mon[35294]: pgmap v713: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 49 MiB/s wr, 799 op/s 2024-03-27T11:05:45.764 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:05:45 smithi114 ceph-mon[35338]: pgmap v714: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 50 MiB/s wr, 810 op/s 2024-03-27T11:05:45.764 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:05:45 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T11:05:45.888 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:05:45 smithi097 ceph-mon[31859]: pgmap v714: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 50 MiB/s wr, 810 op/s 2024-03-27T11:05:45.888 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:05:45 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T11:05:45.924 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:05:45 smithi177 ceph-mon[35294]: pgmap v714: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 50 MiB/s wr, 810 op/s 2024-03-27T11:05:45.924 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:05:45 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T11:05:48.138 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:05:47 smithi097 ceph-mon[31859]: pgmap v715: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 48 MiB/s wr, 772 op/s 2024-03-27T11:05:48.139 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:05:47 smithi097 ceph-mon[31859]: from='mgr.14152 ' entity='mgr.x' 2024-03-27T11:05:48.139 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:05:47 smithi097 ceph-mon[31859]: from='mgr.14152 ' entity='mgr.x' 2024-03-27T11:05:48.174 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:05:47 smithi177 ceph-mon[35294]: pgmap v715: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 48 MiB/s wr, 772 op/s 2024-03-27T11:05:48.174 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:05:47 smithi177 ceph-mon[35294]: from='mgr.14152 ' entity='mgr.x' 2024-03-27T11:05:48.174 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:05:47 smithi177 ceph-mon[35294]: from='mgr.14152 ' entity='mgr.x' 2024-03-27T11:05:48.264 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:05:47 smithi114 ceph-mon[35338]: pgmap v715: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 48 MiB/s wr, 772 op/s 2024-03-27T11:05:48.264 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:05:47 smithi114 ceph-mon[35338]: from='mgr.14152 ' entity='mgr.x' 2024-03-27T11:05:48.264 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:05:47 smithi114 ceph-mon[35338]: from='mgr.14152 ' entity='mgr.x' 2024-03-27T11:05:50.139 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:05:49 smithi097 ceph-mon[31859]: pgmap v716: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 50 MiB/s wr, 803 op/s 2024-03-27T11:05:50.139 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:05:49 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T11:05:50.139 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:05:49 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T11:05:50.139 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:05:49 smithi097 ceph-mon[31859]: from='mgr.14152 ' entity='mgr.x' 2024-03-27T11:05:50.174 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:05:49 smithi177 ceph-mon[35294]: pgmap v716: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 50 MiB/s wr, 803 op/s 2024-03-27T11:05:50.174 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:05:49 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T11:05:50.174 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:05:49 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T11:05:50.174 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:05:49 smithi177 ceph-mon[35294]: from='mgr.14152 ' entity='mgr.x' 2024-03-27T11:05:50.264 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:05:49 smithi114 ceph-mon[35338]: pgmap v716: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 50 MiB/s wr, 803 op/s 2024-03-27T11:05:50.264 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:05:49 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T11:05:50.264 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:05:49 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T11:05:50.264 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:05:49 smithi114 ceph-mon[35338]: from='mgr.14152 ' entity='mgr.x' 2024-03-27T11:05:52.140 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:05:51 smithi097 ceph-mon[31859]: pgmap v717: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 52 MiB/s wr, 849 op/s 2024-03-27T11:05:52.174 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:05:51 smithi177 ceph-mon[35294]: pgmap v717: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 52 MiB/s wr, 849 op/s 2024-03-27T11:05:52.264 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:05:51 smithi114 ceph-mon[35338]: pgmap v717: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 52 MiB/s wr, 849 op/s 2024-03-27T11:05:54.139 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:05:53 smithi097 ceph-mon[31859]: pgmap v718: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 47 MiB/s wr, 755 op/s 2024-03-27T11:05:54.174 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:05:53 smithi177 ceph-mon[35294]: pgmap v718: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 47 MiB/s wr, 755 op/s 2024-03-27T11:05:54.265 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:05:53 smithi114 ceph-mon[35338]: pgmap v718: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 47 MiB/s wr, 755 op/s 2024-03-27T11:05:56.014 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.043+0000 7f174ba00640 1 -- 172.21.15.97:0/71066760 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1744105700 msgr2=0x7f1744105b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:05:56.014 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.043+0000 7f174ba00640 1 --2- 172.21.15.97:0/71066760 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1744105700 0x7f1744105b80 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f173c009ca0 tx=0x7f173c02c100 comp rx=0 tx=0).stop 2024-03-27T11:05:56.015 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.044+0000 7f174ba00640 1 -- 172.21.15.97:0/71066760 shutdown_connections 2024-03-27T11:05:56.015 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.044+0000 7f174ba00640 1 --2- 172.21.15.97:0/71066760 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1744105700 0x7f1744105b80 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:56.015 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.044+0000 7f174ba00640 1 --2- 172.21.15.97:0/71066760 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1744102d50 0x7f17441051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:56.015 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.044+0000 7f174ba00640 1 --2- 172.21.15.97:0/71066760 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f17441003e0 0x7f1744102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:56.015 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.044+0000 7f174ba00640 1 -- 172.21.15.97:0/71066760 >> 172.21.15.97:0/71066760 conn(0x7f17440fbfe0 msgr2=0x7f17440fe420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:05:56.015 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.044+0000 7f174ba00640 1 -- 172.21.15.97:0/71066760 shutdown_connections 2024-03-27T11:05:56.016 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.044+0000 7f174ba00640 1 -- 172.21.15.97:0/71066760 wait complete. 2024-03-27T11:05:56.016 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.045+0000 7f174ba00640 1 Processor -- start 2024-03-27T11:05:56.016 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.046+0000 7f174ba00640 1 -- start start 2024-03-27T11:05:56.016 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.046+0000 7f174ba00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f17441003e0 0x7f174419e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:05:56.016 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.046+0000 7f174ba00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1744102d50 0x7f174419e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:05:56.016 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.046+0000 7f174ba00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1744105700 0x7f17441a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:05:56.017 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.046+0000 7f174ba00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1744113cc0 con 0x7f1744102d50 2024-03-27T11:05:56.017 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.046+0000 7f174ba00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f1744113b40 con 0x7f1744105700 2024-03-27T11:05:56.017 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.046+0000 7f174ba00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1744113e40 con 0x7f17441003e0 2024-03-27T11:05:56.017 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.047+0000 7f1749a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1744105700 0x7f17441a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:05:56.017 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.047+0000 7f1749a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1744105700 0x7f17441a5790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:48926/0 (socket says 172.21.15.97:48926) 2024-03-27T11:05:56.018 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.047+0000 7f1749a00640 1 -- 172.21.15.97:0/3336169896 learned_addr learned my addr 172.21.15.97:0/3336169896 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:05:56.018 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.047+0000 7f1743e00640 1 --2- 172.21.15.97:0/3336169896 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1744102d50 0x7f174419e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:05:56.018 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.047+0000 7f1749000640 1 --2- 172.21.15.97:0/3336169896 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f17441003e0 0x7f174419e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:05:56.018 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.047+0000 7f1749a00640 1 -- 172.21.15.97:0/3336169896 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f17441003e0 msgr2=0x7f174419e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:05:56.018 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.047+0000 7f1749a00640 1 --2- 172.21.15.97:0/3336169896 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f17441003e0 0x7f174419e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:56.018 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.047+0000 7f1749a00640 1 -- 172.21.15.97:0/3336169896 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1744102d50 msgr2=0x7f174419e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:05:56.018 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.048+0000 7f1749a00640 1 --2- 172.21.15.97:0/3336169896 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1744102d50 0x7f174419e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:56.019 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.048+0000 7f1749a00640 1 -- 172.21.15.97:0/3336169896 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f174406b6a0 con 0x7f1744105700 2024-03-27T11:05:56.019 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.048+0000 7f1743e00640 1 --2- 172.21.15.97:0/3336169896 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1744102d50 0x7f174419e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T11:05:56.019 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.049+0000 7f1749a00640 1 --2- 172.21.15.97:0/3336169896 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1744105700 0x7f17441a5790 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f173c032520 tx=0x7f173c035ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:05:56.020 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.050+0000 7f1741600640 1 -- 172.21.15.97:0/3336169896 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f173c039040 con 0x7f1744105700 2024-03-27T11:05:56.020 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.050+0000 7f1741600640 1 -- 172.21.15.97:0/3336169896 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f173c044070 con 0x7f1744105700 2024-03-27T11:05:56.020 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.050+0000 7f1741600640 1 -- 172.21.15.97:0/3336169896 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f173c0043c0 con 0x7f1744105700 2024-03-27T11:05:56.021 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.051+0000 7f174ba00640 1 -- 172.21.15.97:0/3336169896 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f174406b930 con 0x7f1744105700 2024-03-27T11:05:56.022 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.051+0000 7f174ba00640 1 -- 172.21.15.97:0/3336169896 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f174406bdd0 con 0x7f1744105700 2024-03-27T11:05:56.027 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.057+0000 7f174e79f740 1 Processor -- start 2024-03-27T11:05:56.031 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.059+0000 7f1741600640 1 -- 172.21.15.97:0/3336169896 <== mon.2 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f173c004560 con 0x7f1744105700 2024-03-27T11:05:56.031 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.061+0000 7f1741600640 1 --2- 172.21.15.97:0/3336169896 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f172c0a3820 0x7f172c0a5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:05:56.032 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.062+0000 7f1743e00640 1 --2- 172.21.15.97:0/3336169896 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f172c0a3820 0x7f172c0a5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:05:56.032 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.062+0000 7f174e79f740 1 -- start start 2024-03-27T11:05:56.033 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.062+0000 7f174e79f740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5595876ef980 0x5595876efda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:05:56.033 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.062+0000 7f174e79f740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595876f0370 0x5595876f4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:05:56.033 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.062+0000 7f174e79f740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5595876f54e0 0x5595876f7970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:05:56.033 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.062+0000 7f174e79f740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x559587654d10 con 0x5595876f0370 2024-03-27T11:05:56.033 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.062+0000 7f174e79f740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5595875fd260 con 0x5595876ef980 2024-03-27T11:05:56.033 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.062+0000 7f174e79f740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x559587650cc0 con 0x5595876f54e0 2024-03-27T11:05:56.033 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.063+0000 7f1749a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5595876f54e0 0x5595876f7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:05:56.033 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.063+0000 7f1749a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5595876f54e0 0x5595876f7970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.97:60220/0 (socket says 172.21.15.97:60220) 2024-03-27T11:05:56.033 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.063+0000 7f1749a00640 1 -- 172.21.15.97:0/3237368462 learned_addr learned my addr 172.21.15.97:0/3237368462 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:05:56.034 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.064+0000 7f1743e00640 1 --2- 172.21.15.97:0/3237368462 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595876f0370 0x5595876f4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:05:56.034 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.064+0000 7f1749a00640 1 -- 172.21.15.97:0/3237368462 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5595876ef980 msgr2=0x5595876efda0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-27T11:05:56.034 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.064+0000 7f1749a00640 1 --2- 172.21.15.97:0/3237368462 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5595876ef980 0x5595876efda0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:56.034 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.064+0000 7f1749a00640 1 -- 172.21.15.97:0/3237368462 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595876f0370 msgr2=0x5595876f4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:05:56.035 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.064+0000 7f1741600640 1 -- 172.21.15.97:0/3336169896 <== mon.2 v2:172.21.15.114:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f173c009890 con 0x7f1744105700 2024-03-27T11:05:56.035 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.064+0000 7f1749a00640 1 --2- 172.21.15.97:0/3237368462 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595876f0370 0x5595876f4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:56.035 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.064+0000 7f1749a00640 1 -- 172.21.15.97:0/3237368462 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595875f4aa0 con 0x5595876f54e0 2024-03-27T11:05:56.035 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.064+0000 7f1743e00640 1 --2- 172.21.15.97:0/3237368462 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595876f0370 0x5595876f4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T11:05:56.035 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.065+0000 7f1749a00640 1 --2- 172.21.15.97:0/3237368462 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5595876f54e0 0x5595876f7970 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f173c006010 tx=0x7f173c0c5d20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b31184a1b9f201b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:05:56.036 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.065+0000 7f1743e00640 1 --2- 172.21.15.97:0/3336169896 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f172c0a3820 0x7f172c0a5ce0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f174419f6f0 tx=0x7f17300127c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:05:56.036 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.065+0000 7f1736000640 1 -- 172.21.15.97:0/3237368462 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f173c0bb090 con 0x5595876f54e0 2024-03-27T11:05:56.036 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.065+0000 7f1736000640 1 -- 172.21.15.97:0/3237368462 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f173c004850 con 0x5595876f54e0 2024-03-27T11:05:56.037 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.066+0000 7f1736000640 1 -- 172.21.15.97:0/3237368462 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f173c0bb090 con 0x5595876f54e0 2024-03-27T11:05:56.037 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.067+0000 7f174e79f740 1 -- 172.21.15.97:0/3237368462 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5595876f54e0 msgr2=0x5595876f7970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:05:56.037 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.067+0000 7f174e79f740 1 --2- 172.21.15.97:0/3237368462 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5595876f54e0 0x5595876f7970 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f173c006010 tx=0x7f173c0c5d20 comp rx=0 tx=0).stop 2024-03-27T11:05:56.038 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.067+0000 7f174e79f740 1 -- 172.21.15.97:0/3237368462 shutdown_connections 2024-03-27T11:05:56.038 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.068+0000 7f174e79f740 1 --2- 172.21.15.97:0/3237368462 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5595876f54e0 0x5595876f7970 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:56.038 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.068+0000 7f174e79f740 1 --2- 172.21.15.97:0/3237368462 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595876f0370 0x5595876f4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:56.038 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.068+0000 7f174e79f740 1 --2- 172.21.15.97:0/3237368462 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5595876ef980 0x5595876efda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:56.038 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.068+0000 7f174e79f740 1 -- 172.21.15.97:0/3237368462 >> 172.21.15.97:0/3237368462 conn(0x5595876eaef0 msgr2=0x5595876eb2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:05:56.039 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.068+0000 7f174e79f740 1 -- 172.21.15.97:0/3237368462 shutdown_connections 2024-03-27T11:05:56.039 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.069+0000 7f174e79f740 1 -- 172.21.15.97:0/3237368462 wait complete. 2024-03-27T11:05:56.040 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.070+0000 7f174e79f740 1 -- start start 2024-03-27T11:05:56.040 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.070+0000 7f174e79f740 1 Processor -- start 2024-03-27T11:05:56.042 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.072+0000 7f174e79f740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5595876ef980 0x55958770a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:05:56.042 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.072+0000 7f174e79f740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5595876f0370 0x55958770ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:05:56.042 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.072+0000 7f174e79f740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595876f54e0 0x559587711ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:05:56.043 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.073+0000 7f174e79f740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x559587709260 con 0x5595876f54e0 2024-03-27T11:05:56.043 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.073+0000 7f174e79f740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x559587610630 con 0x5595876ef980 2024-03-27T11:05:56.043 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.073+0000 7f174e79f740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5595877093e0 con 0x5595876f0370 2024-03-27T11:05:56.043 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.073+0000 7f1749000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5595876ef980 0x55958770a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:05:56.043 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.073+0000 7f1749000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5595876ef980 0x55958770a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:48946/0 (socket says 172.21.15.97:48946) 2024-03-27T11:05:56.043 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.073+0000 7f1749000640 1 -- 172.21.15.97:0/461869355 learned_addr learned my addr 172.21.15.97:0/461869355 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:05:56.044 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.073+0000 7f174e79f740 10 client.0 _ll_register_callbacks cb 0x559587615c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T11:05:56.044 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.073+0000 7f1749a00640 1 --2- 172.21.15.97:0/461869355 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595876f54e0 0x559587711ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:05:56.044 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.074+0000 7f1743e00640 1 --2- 172.21.15.97:0/461869355 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5595876f0370 0x55958770ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:05:56.044 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.074+0000 7f1749000640 1 -- 172.21.15.97:0/461869355 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5595876f0370 msgr2=0x55958770ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:05:56.045 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.074+0000 7f1749000640 1 --2- 172.21.15.97:0/461869355 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5595876f0370 0x55958770ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:56.045 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.074+0000 7f1749000640 1 -- 172.21.15.97:0/461869355 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595876f54e0 msgr2=0x559587711ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:05:56.045 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.074+0000 7f1749000640 1 --2- 172.21.15.97:0/461869355 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595876f54e0 0x559587711ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:56.045 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.074+0000 7f1749000640 1 -- 172.21.15.97:0/461869355 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55958761c5f0 con 0x5595876ef980 2024-03-27T11:05:56.046 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.076+0000 7f1749000640 1 --2- 172.21.15.97:0/461869355 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5595876ef980 0x55958770a940 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f17380116f0 tx=0x7f1738011bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1a22808335e5d4c server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:05:56.046 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.076+0000 7f172aa00640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-03-27T11:05:56.046 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.076+0000 7f172aa00640 1 -- 172.21.15.97:0/461869355 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f173800f8c0 con 0x5595876ef980 2024-03-27T11:05:56.046 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.076+0000 7f172aa00640 1 -- 172.21.15.97:0/461869355 <== mon.2 v2:172.21.15.114:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f173800fa60 con 0x5595876ef980 2024-03-27T11:05:56.047 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.076+0000 7f174e79f740 1 -- 172.21.15.97:0/461869355 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f173c004850 con 0x5595876ef980 2024-03-27T11:05:56.047 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.077+0000 7f172aa00640 1 -- 172.21.15.97:0/461869355 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f173800fc60 con 0x5595876ef980 2024-03-27T11:05:56.047 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.077+0000 7f174e79f740 10 client.36494 fetch_fsmap learned FSMap version 34 2024-03-27T11:05:56.047 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.077+0000 7f174e79f740 1 -- 172.21.15.97:0/461869355 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x559587369210 con 0x5595876ef980 2024-03-27T11:05:56.048 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.077+0000 7f172aa00640 1 -- 172.21.15.97:0/461869355 <== mon.2 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f173c004850 con 0x5595876ef980 2024-03-27T11:05:56.048 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.077+0000 7f172aa00640 1 -- 172.21.15.97:0/461869355 <== mon.2 v2:172.21.15.114:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f1738018070 con 0x5595876ef980 2024-03-27T11:05:56.049 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.079+0000 7f174e79f740 10 client.36494 fetch_fsmap finished waiting for FSMap version 34 2024-03-27T11:05:56.049 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.079+0000 7f174e79f740 10 client.36494 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T11:05:56.050 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.079+0000 7f174e79f740 20 client.36494 populate_metadata read hostname 'smithi097' 2024-03-27T11:05:56.050 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.079+0000 7f174e79f740 1 --2- 172.21.15.97:0/461869355 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x559587716760 0x559587718c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:05:56.050 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.079+0000 7f174e79f740 4 client.36494 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T11:05:56.050 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.079+0000 7f174e79f740 1 -- 172.21.15.97:0/461869355 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559587662250 con 0x559587716760 2024-03-27T11:05:56.050 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.080+0000 7f1743e00640 1 --2- 172.21.15.97:0/461869355 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x559587716760 0x559587718c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:05:56.050 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.080+0000 7f1743e00640 1 --2- 172.21.15.97:0/461869355 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x559587716760 0x559587718c00 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b00fb87deff69035 server_cookie=38c26800636b0f2a in_seq=0 out_seq=0 2024-03-27T11:05:56.051 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.081+0000 7f172aa00640 10 client.36494 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T11:05:56.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.082+0000 7f172aa00640 1 -- 172.21.15.97:0/461869355 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x559587662250 con 0x559587716760 2024-03-27T11:05:56.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.082+0000 7f172aa00640 10 client.36494 handle_command_reply: tid=0 2024-03-27T11:05:56.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.082+0000 7f174e79f740 1 client.36494 shutdown 2024-03-27T11:05:56.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.082+0000 7f174e79f740 1 -- 172.21.15.97:0/461869355 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5595876ef980 msgr2=0x55958770a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:05:56.053 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.082+0000 7f174e79f740 1 --2- 172.21.15.97:0/461869355 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5595876ef980 0x55958770a940 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f17380116f0 tx=0x7f1738011bb0 comp rx=0 tx=0).stop 2024-03-27T11:05:56.053 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.083+0000 7f174e79f740 1 -- 172.21.15.97:0/461869355 shutdown_connections 2024-03-27T11:05:56.053 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.083+0000 7f174e79f740 1 --2- 172.21.15.97:0/461869355 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x559587716760 0x559587718c00 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:56.053 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.083+0000 7f174e79f740 1 --2- 172.21.15.97:0/461869355 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595876f54e0 0x559587711ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:56.053 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.083+0000 7f174e79f740 1 --2- 172.21.15.97:0/461869355 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5595876f0370 0x55958770ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:56.053 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.083+0000 7f174e79f740 1 --2- 172.21.15.97:0/461869355 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5595876ef980 0x55958770a940 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:56.053 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.083+0000 7f174e79f740 1 -- 172.21.15.97:0/461869355 >> 172.21.15.97:0/461869355 conn(0x5595876eaef0 msgr2=0x5595876ea0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:05:56.053 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.083+0000 7f172aa00640 0 client.36494 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T11:05:56.054 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.083+0000 7f174e79f740 1 -- 172.21.15.97:0/461869355 shutdown_connections 2024-03-27T11:05:56.054 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.084+0000 7f174e79f740 1 -- 172.21.15.97:0/461869355 wait complete. 2024-03-27T11:05:56.054 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.084+0000 7f174e79f740 20 client.36494 trim_cache size 0 max 16384 2024-03-27T11:05:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.115+0000 7f174e79f740 1 Processor -- start 2024-03-27T11:05:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.115+0000 7f174e79f740 1 -- start start 2024-03-27T11:05:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.115+0000 7f174e79f740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5595876ef980 0x5595876f1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:05:56.086 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.115+0000 7f174e79f740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595876f0370 0x5595876f1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:05:56.086 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.115+0000 7f174e79f740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5595876f54e0 0x559587711de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:05:56.086 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.115+0000 7f174e79f740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1744107870 con 0x5595876f0370 2024-03-27T11:05:56.086 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.115+0000 7f174e79f740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f17441093b0 con 0x5595876f54e0 2024-03-27T11:05:56.086 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.115+0000 7f174e79f740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1744107f40 con 0x5595876ef980 2024-03-27T11:05:56.086 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.116+0000 7f1749a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5595876f54e0 0x559587711de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:05:56.086 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.116+0000 7f1749a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5595876f54e0 0x559587711de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:48962/0 (socket says 172.21.15.97:48962) 2024-03-27T11:05:56.087 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.116+0000 7f1749a00640 1 -- 172.21.15.97:0/468822891 learned_addr learned my addr 172.21.15.97:0/468822891 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:05:56.087 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.116+0000 7f1743e00640 1 --2- 172.21.15.97:0/468822891 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595876f0370 0x5595876f1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:05:56.087 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.117+0000 7f1743e00640 1 -- 172.21.15.97:0/468822891 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5595876ef980 msgr2=0x5595876f1630 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-27T11:05:56.087 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.117+0000 7f1743e00640 1 --2- 172.21.15.97:0/468822891 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5595876ef980 0x5595876f1630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:56.088 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.117+0000 7f1743e00640 1 -- 172.21.15.97:0/468822891 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5595876f54e0 msgr2=0x559587711de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:05:56.088 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.117+0000 7f1743e00640 1 --2- 172.21.15.97:0/468822891 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5595876f54e0 0x559587711de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:56.088 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.117+0000 7f1743e00640 1 -- 172.21.15.97:0/468822891 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559587435620 con 0x5595876f0370 2024-03-27T11:05:56.088 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.117+0000 7f1743e00640 1 --2- 172.21.15.97:0/468822891 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595876f0370 0x5595876f1b70 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x559587719480 tx=0x7f1730015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0ffb79264dd7889 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:05:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.118+0000 7f1735600640 1 -- 172.21.15.97:0/468822891 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f173002e070 con 0x5595876f0370 2024-03-27T11:05:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.118+0000 7f1735600640 1 -- 172.21.15.97:0/468822891 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1730028350 con 0x5595876f0370 2024-03-27T11:05:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.118+0000 7f1735600640 1 -- 172.21.15.97:0/468822891 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f173001f910 con 0x5595876f0370 2024-03-27T11:05:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.119+0000 7f174e79f740 1 -- 172.21.15.97:0/468822891 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595876f0370 msgr2=0x5595876f1b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:05:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.119+0000 7f174e79f740 1 --2- 172.21.15.97:0/468822891 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595876f0370 0x5595876f1b70 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x559587719480 tx=0x7f1730015000 comp rx=0 tx=0).stop 2024-03-27T11:05:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.119+0000 7f174e79f740 1 -- 172.21.15.97:0/468822891 shutdown_connections 2024-03-27T11:05:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.119+0000 7f174e79f740 1 --2- 172.21.15.97:0/468822891 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5595876f54e0 0x559587711de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.120+0000 7f174e79f740 1 --2- 172.21.15.97:0/468822891 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595876f0370 0x5595876f1b70 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.120+0000 7f174e79f740 1 --2- 172.21.15.97:0/468822891 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5595876ef980 0x5595876f1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.120+0000 7f174e79f740 1 -- 172.21.15.97:0/468822891 >> 172.21.15.97:0/468822891 conn(0x5595876eaef0 msgr2=0x5595877096b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:05:56.091 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.120+0000 7f174e79f740 1 -- 172.21.15.97:0/468822891 shutdown_connections 2024-03-27T11:05:56.091 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.120+0000 7f174e79f740 1 -- 172.21.15.97:0/468822891 wait complete. 2024-03-27T11:05:56.094 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.121+0000 7f174e79f740 1 -- start start 2024-03-27T11:05:56.094 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.121+0000 7f174e79f740 1 Processor -- start 2024-03-27T11:05:56.094 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.122+0000 7f174e79f740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5595876ef980 0x559587711e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:05:56.094 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.122+0000 7f174e79f740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595876f0370 0x55958770e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:05:56.094 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.122+0000 7f174e79f740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5595876f54e0 0x55958770ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:05:56.094 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.122+0000 7f174e79f740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1744108ce0 con 0x5595876f0370 2024-03-27T11:05:56.094 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.122+0000 7f174e79f740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f1744108610 con 0x5595876f54e0 2024-03-27T11:05:56.094 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.122+0000 7f174e79f740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1744072bd0 con 0x5595876ef980 2024-03-27T11:05:56.094 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.122+0000 7f174e79f740 10 client.0 _ll_register_callbacks cb 0x7f17380097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T11:05:56.095 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.123+0000 7f1749000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5595876ef980 0x559587711e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:05:56.095 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.123+0000 7f1749000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5595876ef980 0x559587711e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.97:60254/0 (socket says 172.21.15.97:60254) 2024-03-27T11:05:56.095 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.123+0000 7f1749000640 1 -- 172.21.15.97:0/876435448 learned_addr learned my addr 172.21.15.97:0/876435448 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:05:56.095 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.123+0000 7f1743e00640 1 --2- 172.21.15.97:0/876435448 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595876f0370 0x55958770e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:05:56.095 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.123+0000 7f1749a00640 1 --2- 172.21.15.97:0/876435448 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5595876f54e0 0x55958770ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:05:56.095 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.124+0000 7f1743e00640 1 -- 172.21.15.97:0/876435448 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5595876ef980 msgr2=0x559587711e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:05:56.095 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.124+0000 7f1743e00640 1 --2- 172.21.15.97:0/876435448 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5595876ef980 0x559587711e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:56.096 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.124+0000 7f1743e00640 1 -- 172.21.15.97:0/876435448 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5595876f54e0 msgr2=0x55958770ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:05:56.096 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.124+0000 7f1743e00640 1 --2- 172.21.15.97:0/876435448 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5595876f54e0 0x55958770ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:56.096 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.124+0000 7f1743e00640 1 -- 172.21.15.97:0/876435448 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55958777ab30 con 0x5595876f0370 2024-03-27T11:05:56.096 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.125+0000 7f1749000640 1 --2- 172.21.15.97:0/876435448 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5595876ef980 0x559587711e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T11:05:56.096 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.126+0000 7f1743e00640 1 --2- 172.21.15.97:0/876435448 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595876f0370 0x55958770e910 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f17300123c0 tx=0x7f173001f460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2e1b87f96ab479f server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:05:56.096 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.126+0000 7f172aa00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T11:05:56.097 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.127+0000 7f172aa00640 1 -- 172.21.15.97:0/876435448 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1730039060 con 0x5595876f0370 2024-03-27T11:05:56.097 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.127+0000 7f172aa00640 1 -- 172.21.15.97:0/876435448 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f1730010a30 con 0x5595876f0370 2024-03-27T11:05:56.097 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.127+0000 7f174e79f740 1 -- 172.21.15.97:0/876435448 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1730028350 con 0x5595876f0370 2024-03-27T11:05:56.097 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.127+0000 7f172aa00640 1 -- 172.21.15.97:0/876435448 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f173002e040 con 0x5595876f0370 2024-03-27T11:05:56.097 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.127+0000 7f174e79f740 10 client.16824 fetch_fsmap learned FSMap version 34 2024-03-27T11:05:56.098 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.127+0000 7f174e79f740 1 -- 172.21.15.97:0/876435448 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55958777be30 con 0x5595876f0370 2024-03-27T11:05:56.098 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.128+0000 7f172aa00640 1 -- 172.21.15.97:0/876435448 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1730028350 con 0x5595876f0370 2024-03-27T11:05:56.098 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.128+0000 7f172aa00640 1 -- 172.21.15.97:0/876435448 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f173003f3d0 con 0x5595876f0370 2024-03-27T11:05:56.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.128+0000 7f174e79f740 10 client.16824 fetch_fsmap finished waiting for FSMap version 34 2024-03-27T11:05:56.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.128+0000 7f174e79f740 10 client.16824 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T11:05:56.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.128+0000 7f174e79f740 20 client.16824 populate_metadata read hostname 'smithi097' 2024-03-27T11:05:56.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.128+0000 7f174e79f740 1 --2- 172.21.15.97:0/876435448 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55958777c710 0x55958777eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:05:56.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.129+0000 7f174e79f740 4 client.16824 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-27T11:05:56.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.129+0000 7f174e79f740 1 -- 172.21.15.97:0/876435448 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55958765c5a0 con 0x55958777c710 2024-03-27T11:05:56.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.129+0000 7f1749000640 1 --2- 172.21.15.97:0/876435448 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55958777c710 0x55958777eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:05:56.101 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.131+0000 7f1749000640 1 --2- 172.21.15.97:0/876435448 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55958777c710 0x55958777eb30 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dac25383acec210a server_cookie=ea0ebdc933653e5 in_seq=0 out_seq=0 2024-03-27T11:05:56.102 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.132+0000 7f172aa00640 10 client.16824 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T11:05:56.108 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.137+0000 7f172aa00640 1 -- 172.21.15.97:0/876435448 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55958765c5a0 con 0x55958777c710 2024-03-27T11:05:56.108 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.137+0000 7f172aa00640 10 client.16824 handle_command_reply: tid=0 2024-03-27T11:05:56.108 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.138+0000 7f174e79f740 1 client.16824 shutdown 2024-03-27T11:05:56.109 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.138+0000 7f174e79f740 1 -- 172.21.15.97:0/876435448 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595876f0370 msgr2=0x55958770e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:05:56.109 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.138+0000 7f174e79f740 1 --2- 172.21.15.97:0/876435448 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595876f0370 0x55958770e910 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f17300123c0 tx=0x7f173001f460 comp rx=0 tx=0).stop 2024-03-27T11:05:56.109 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.139+0000 7f174e79f740 1 -- 172.21.15.97:0/876435448 shutdown_connections 2024-03-27T11:05:56.109 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.139+0000 7f174e79f740 1 --2- 172.21.15.97:0/876435448 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55958777c710 0x55958777eb30 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:56.109 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.139+0000 7f174e79f740 1 --2- 172.21.15.97:0/876435448 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5595876f54e0 0x55958770ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:56.109 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.139+0000 7f174e79f740 1 --2- 172.21.15.97:0/876435448 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595876f0370 0x55958770e910 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:56.109 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.139+0000 7f174e79f740 1 --2- 172.21.15.97:0/876435448 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5595876ef980 0x559587711e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:56.109 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.139+0000 7f174e79f740 1 -- 172.21.15.97:0/876435448 >> 172.21.15.97:0/876435448 conn(0x5595876eaef0 msgr2=0x5595876f1630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:05:56.109 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.139+0000 7f172aa00640 0 client.16824 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T11:05:56.110 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.139+0000 7f174e79f740 1 -- 172.21.15.97:0/876435448 shutdown_connections 2024-03-27T11:05:56.110 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.139+0000 7f174e79f740 1 -- 172.21.15.97:0/876435448 wait complete. 2024-03-27T11:05:56.110 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.140+0000 7f174e79f740 20 client.16824 trim_cache size 0 max 16384 2024-03-27T11:05:56.111 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-27T11:05:56.111 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-27T11:05:56.111 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-27T11:05:56.111 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-27T11:05:56.112 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.142+0000 7f174ba00640 1 -- 172.21.15.97:0/3336169896 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f172c0a3820 msgr2=0x7f172c0a5ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:05:56.112 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.142+0000 7f174ba00640 1 --2- 172.21.15.97:0/3336169896 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f172c0a3820 0x7f172c0a5ce0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f174419f6f0 tx=0x7f17300127c0 comp rx=0 tx=0).stop 2024-03-27T11:05:56.113 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.143+0000 7f174ba00640 1 -- 172.21.15.97:0/3336169896 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1744105700 msgr2=0x7f17441a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:05:56.113 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.143+0000 7f174ba00640 1 --2- 172.21.15.97:0/3336169896 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1744105700 0x7f17441a5790 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f173c032520 tx=0x7f173c035ce0 comp rx=0 tx=0).stop 2024-03-27T11:05:56.113 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.143+0000 7f174ba00640 1 -- 172.21.15.97:0/3336169896 shutdown_connections 2024-03-27T11:05:56.113 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.143+0000 7f174ba00640 1 --2- 172.21.15.97:0/3336169896 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f172c0a3820 0x7f172c0a5ce0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:56.114 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.143+0000 7f174ba00640 1 --2- 172.21.15.97:0/3336169896 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1744105700 0x7f17441a5790 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:56.114 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.143+0000 7f174ba00640 1 --2- 172.21.15.97:0/3336169896 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1744102d50 0x7f174419e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:56.114 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.143+0000 7f174ba00640 1 --2- 172.21.15.97:0/3336169896 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f17441003e0 0x7f174419e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:05:56.114 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.143+0000 7f174ba00640 1 -- 172.21.15.97:0/3336169896 >> 172.21.15.97:0/3336169896 conn(0x7f17440fbfe0 msgr2=0x7f1744101090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:05:56.114 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.143+0000 7f174ba00640 1 -- 172.21.15.97:0/3336169896 shutdown_connections 2024-03-27T11:05:56.114 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:05:56.143+0000 7f174ba00640 1 -- 172.21.15.97:0/3336169896 wait complete. 2024-03-27T11:05:56.138 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:05:55 smithi097 ceph-mon[31859]: pgmap v719: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 47 MiB/s wr, 767 op/s 2024-03-27T11:05:56.174 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:05:55 smithi177 ceph-mon[35294]: pgmap v719: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 47 MiB/s wr, 767 op/s 2024-03-27T11:05:56.266 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:05:55 smithi114 ceph-mon[35338]: pgmap v719: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 47 MiB/s wr, 767 op/s 2024-03-27T11:05:58.138 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:05:57 smithi097 ceph-mon[31859]: pgmap v720: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 46 MiB/s wr, 743 op/s 2024-03-27T11:05:58.174 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:05:57 smithi177 ceph-mon[35294]: pgmap v720: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 46 MiB/s wr, 743 op/s 2024-03-27T11:05:58.264 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:05:57 smithi114 ceph-mon[35338]: pgmap v720: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 46 MiB/s wr, 743 op/s 2024-03-27T11:06:00.138 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:05:59 smithi097 ceph-mon[31859]: pgmap v721: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 48 MiB/s wr, 783 op/s 2024-03-27T11:06:00.174 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:05:59 smithi177 ceph-mon[35294]: pgmap v721: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 48 MiB/s wr, 783 op/s 2024-03-27T11:06:00.264 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:05:59 smithi114 ceph-mon[35338]: pgmap v721: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 48 MiB/s wr, 783 op/s 2024-03-27T11:06:02.140 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:06:01 smithi097 ceph-mon[31859]: pgmap v722: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 636 KiB/s rd, 53 MiB/s wr, 861 op/s 2024-03-27T11:06:02.174 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:06:01 smithi177 ceph-mon[35294]: pgmap v722: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 636 KiB/s rd, 53 MiB/s wr, 861 op/s 2024-03-27T11:06:02.264 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:06:01 smithi114 ceph-mon[35338]: pgmap v722: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 636 KiB/s rd, 53 MiB/s wr, 861 op/s 2024-03-27T11:06:04.140 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:06:03 smithi097 ceph-mon[31859]: pgmap v723: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 687 KiB/s rd, 48 MiB/s wr, 782 op/s 2024-03-27T11:06:04.174 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:06:03 smithi177 ceph-mon[35294]: pgmap v723: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 687 KiB/s rd, 48 MiB/s wr, 782 op/s 2024-03-27T11:06:04.264 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:06:03 smithi114 ceph-mon[35338]: pgmap v723: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 687 KiB/s rd, 48 MiB/s wr, 782 op/s 2024-03-27T11:06:06.139 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:06:05 smithi097 ceph-mon[31859]: pgmap v724: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 721 KiB/s rd, 49 MiB/s wr, 809 op/s 2024-03-27T11:06:06.174 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:06:05 smithi177 ceph-mon[35294]: pgmap v724: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 721 KiB/s rd, 49 MiB/s wr, 809 op/s 2024-03-27T11:06:06.264 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:06:05 smithi114 ceph-mon[35338]: pgmap v724: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 721 KiB/s rd, 49 MiB/s wr, 809 op/s 2024-03-27T11:06:08.174 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:06:07 smithi177 ceph-mon[35294]: pgmap v725: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 837 KiB/s rd, 48 MiB/s wr, 781 op/s 2024-03-27T11:06:08.264 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:06:07 smithi114 ceph-mon[35338]: pgmap v725: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 837 KiB/s rd, 48 MiB/s wr, 781 op/s 2024-03-27T11:06:08.389 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:06:07 smithi097 ceph-mon[31859]: pgmap v725: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 837 KiB/s rd, 48 MiB/s wr, 781 op/s 2024-03-27T11:06:09.312 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.343+0000 7f7562c00640 1 -- 172.21.15.97:0/2057495405 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f755c0ff7e0 msgr2=0x7f755c0ffc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:06:09.312 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.343+0000 7f7562c00640 1 --2- 172.21.15.97:0/2057495405 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f755c0ff7e0 0x7f755c0ffc00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f754c009900 tx=0x7f754c02bc50 comp rx=0 tx=0).stop 2024-03-27T11:06:09.315 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.346+0000 7f7562c00640 1 -- 172.21.15.97:0/2057495405 shutdown_connections 2024-03-27T11:06:09.315 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.346+0000 7f7562c00640 1 --2- 172.21.15.97:0/2057495405 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f755c107860 0x7f755c109cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:09.315 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.346+0000 7f7562c00640 1 --2- 172.21.15.97:0/2057495405 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f755c1001d0 0x7f755c107320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:09.315 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.346+0000 7f7562c00640 1 --2- 172.21.15.97:0/2057495405 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f755c0ff7e0 0x7f755c0ffc00 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:09.315 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.346+0000 7f7562c00640 1 -- 172.21.15.97:0/2057495405 >> 172.21.15.97:0/2057495405 conn(0x7f755c0fb330 msgr2=0x7f755c0fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:06:09.316 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.346+0000 7f7562c00640 1 -- 172.21.15.97:0/2057495405 shutdown_connections 2024-03-27T11:06:09.316 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.346+0000 7f7562c00640 1 -- 172.21.15.97:0/2057495405 wait complete. 2024-03-27T11:06:09.317 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.348+0000 7f7562c00640 1 Processor -- start 2024-03-27T11:06:09.317 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.348+0000 7f7562c00640 1 -- start start 2024-03-27T11:06:09.317 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.348+0000 7f7562c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f755c0ff7e0 0x7f755c19e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:06:09.318 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.349+0000 7f7562c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f755c1001d0 0x7f755c19e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:06:09.318 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.349+0000 7f7562c00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f755c107860 0x7f755c1a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:06:09.318 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.349+0000 7f7562c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f755c10d780 con 0x7f755c0ff7e0 2024-03-27T11:06:09.318 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.349+0000 7f7562c00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f755c10d600 con 0x7f755c1001d0 2024-03-27T11:06:09.319 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.349+0000 7f7562c00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f755c10d900 con 0x7f755c107860 2024-03-27T11:06:09.319 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.349+0000 7f755be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f755c0ff7e0 0x7f755c19e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:06:09.319 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.349+0000 7f755be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f755c0ff7e0 0x7f755c19e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43060/0 (socket says 172.21.15.97:43060) 2024-03-27T11:06:09.319 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.349+0000 7f755be00640 1 -- 172.21.15.97:0/210811686 learned_addr learned my addr 172.21.15.97:0/210811686 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:06:09.319 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.350+0000 7f755be00640 1 -- 172.21.15.97:0/210811686 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f755c107860 msgr2=0x7f755c1a5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:06:09.320 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.350+0000 7f755be00640 1 --2- 172.21.15.97:0/210811686 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f755c107860 0x7f755c1a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:09.320 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.350+0000 7f755b400640 1 --2- 172.21.15.97:0/210811686 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f755c1001d0 0x7f755c19e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:06:09.320 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.350+0000 7f755be00640 1 -- 172.21.15.97:0/210811686 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f755c1001d0 msgr2=0x7f755c19e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:06:09.320 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.350+0000 7f755be00640 1 --2- 172.21.15.97:0/210811686 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f755c1001d0 0x7f755c19e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:09.320 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.350+0000 7f755be00640 1 -- 172.21.15.97:0/210811686 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f755c1a5f90 con 0x7f755c0ff7e0 2024-03-27T11:06:09.320 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.350+0000 7f755b400640 1 --2- 172.21.15.97:0/210811686 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f755c1001d0 0x7f755c19e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T11:06:09.320 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.351+0000 7f755be00640 1 --2- 172.21.15.97:0/210811686 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f755c0ff7e0 0x7f755c19e370 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f754c000c00 tx=0x7f754c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:06:09.320 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.351+0000 7f7558c00640 1 -- 172.21.15.97:0/210811686 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f754c043020 con 0x7f755c0ff7e0 2024-03-27T11:06:09.321 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.352+0000 7f7558c00640 1 -- 172.21.15.97:0/210811686 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f754c0054c0 con 0x7f755c0ff7e0 2024-03-27T11:06:09.321 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.352+0000 7f7558c00640 1 -- 172.21.15.97:0/210811686 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f754c003d40 con 0x7f755c0ff7e0 2024-03-27T11:06:09.322 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.352+0000 7f7562c00640 1 -- 172.21.15.97:0/210811686 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f755c1a6220 con 0x7f755c0ff7e0 2024-03-27T11:06:09.322 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.352+0000 7f7562c00640 1 -- 172.21.15.97:0/210811686 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f755c1a67a0 con 0x7f755c0ff7e0 2024-03-27T11:06:09.328 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.359+0000 7f7558c00640 1 -- 172.21.15.97:0/210811686 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f754c046020 con 0x7f755c0ff7e0 2024-03-27T11:06:09.329 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.360+0000 7f7565a11740 1 Processor -- start 2024-03-27T11:06:09.330 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.361+0000 7f7558c00640 1 --2- 172.21.15.97:0/210811686 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f75400a38a0 0x7f75400a5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:06:09.331 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.361+0000 7f7558c00640 1 -- 172.21.15.97:0/210811686 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f754c0f2080 con 0x7f755c0ff7e0 2024-03-27T11:06:09.331 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.362+0000 7f7560c00640 1 --2- 172.21.15.97:0/210811686 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f75400a38a0 0x7f75400a5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:06:09.331 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.362+0000 7f7565a11740 1 -- start start 2024-03-27T11:06:09.332 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.362+0000 7f7565a11740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561a0f28f980 0x561a0f28fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:06:09.332 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.362+0000 7f7565a11740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561a0f290370 0x561a0f294e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:06:09.332 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.362+0000 7f7565a11740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561a0f2954e0 0x561a0f297970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:06:09.332 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.362+0000 7f7565a11740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x561a0f1f4d10 con 0x561a0f28f980 2024-03-27T11:06:09.332 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.362+0000 7f7565a11740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x561a0f19d260 con 0x561a0f290370 2024-03-27T11:06:09.332 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.362+0000 7f7565a11740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x561a0f1f0cc0 con 0x561a0f2954e0 2024-03-27T11:06:09.333 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.363+0000 7f755be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561a0f28f980 0x561a0f28fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:06:09.333 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.363+0000 7f755be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561a0f28f980 0x561a0f28fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43072/0 (socket says 172.21.15.97:43072) 2024-03-27T11:06:09.333 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.363+0000 7f755be00640 1 -- 172.21.15.97:0/3697166074 learned_addr learned my addr 172.21.15.97:0/3697166074 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:06:09.333 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.363+0000 7f7560c00640 1 --2- 172.21.15.97:0/3697166074 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561a0f2954e0 0x561a0f297970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:06:09.333 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.363+0000 7f755be00640 1 -- 172.21.15.97:0/3697166074 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561a0f2954e0 msgr2=0x561a0f297970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:06:09.333 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.363+0000 7f755be00640 1 --2- 172.21.15.97:0/3697166074 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561a0f2954e0 0x561a0f297970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:09.333 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.363+0000 7f755be00640 1 -- 172.21.15.97:0/3697166074 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561a0f290370 msgr2=0x561a0f294e70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-27T11:06:09.334 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.364+0000 7f755be00640 1 --2- 172.21.15.97:0/3697166074 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561a0f290370 0x561a0f294e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:09.334 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.364+0000 7f755be00640 1 -- 172.21.15.97:0/3697166074 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a0f194aa0 con 0x561a0f28f980 2024-03-27T11:06:09.334 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.365+0000 7f7560c00640 1 --2- 172.21.15.97:0/210811686 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f75400a38a0 0x7f75400a5d60 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f755c100730 tx=0x7f7554012760 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:06:09.334 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.365+0000 7f755be00640 1 --2- 172.21.15.97:0/3697166074 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561a0f28f980 0x561a0f28fda0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f754c0be320 tx=0x7f754c02cdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=196f7f61e86a5412 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:06:09.335 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.366+0000 7f7551200640 1 -- 172.21.15.97:0/3697166074 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f754c0d6020 con 0x561a0f28f980 2024-03-27T11:06:09.335 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.366+0000 7f7551200640 1 -- 172.21.15.97:0/3697166074 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f754c004c50 con 0x561a0f28f980 2024-03-27T11:06:09.335 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.366+0000 7f7551200640 1 -- 172.21.15.97:0/3697166074 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f754c0c29d0 con 0x561a0f28f980 2024-03-27T11:06:09.336 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.367+0000 7f7565a11740 1 -- 172.21.15.97:0/3697166074 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561a0f28f980 msgr2=0x561a0f28fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:06:09.336 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.367+0000 7f7565a11740 1 --2- 172.21.15.97:0/3697166074 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561a0f28f980 0x561a0f28fda0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f754c0be320 tx=0x7f754c02cdf0 comp rx=0 tx=0).stop 2024-03-27T11:06:09.336 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.367+0000 7f7565a11740 1 -- 172.21.15.97:0/3697166074 shutdown_connections 2024-03-27T11:06:09.336 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.367+0000 7f7565a11740 1 --2- 172.21.15.97:0/3697166074 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561a0f2954e0 0x561a0f297970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:09.337 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.367+0000 7f7565a11740 1 --2- 172.21.15.97:0/3697166074 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561a0f290370 0x561a0f294e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:09.337 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.367+0000 7f7565a11740 1 --2- 172.21.15.97:0/3697166074 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561a0f28f980 0x561a0f28fda0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:09.337 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.367+0000 7f7565a11740 1 -- 172.21.15.97:0/3697166074 >> 172.21.15.97:0/3697166074 conn(0x561a0f28aef0 msgr2=0x561a0f28b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:06:09.337 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.367+0000 7f7565a11740 1 -- 172.21.15.97:0/3697166074 shutdown_connections 2024-03-27T11:06:09.337 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.367+0000 7f7565a11740 1 -- 172.21.15.97:0/3697166074 wait complete. 2024-03-27T11:06:09.337 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.368+0000 7f7565a11740 1 -- start start 2024-03-27T11:06:09.337 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.368+0000 7f7565a11740 1 Processor -- start 2024-03-27T11:06:09.338 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.369+0000 7f7565a11740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561a0f28f980 0x561a0f2999d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:06:09.339 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.369+0000 7f7565a11740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561a0f290370 0x561a0f299f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:06:09.339 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.369+0000 7f7565a11740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561a0f2954e0 0x561a0f2a0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:06:09.339 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.369+0000 7f755be00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561a0f28f980 0x561a0f2999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:06:09.339 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.369+0000 7f755be00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561a0f28f980 0x561a0f2999d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:36826/0 (socket says 172.21.15.97:36826) 2024-03-27T11:06:09.339 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.370+0000 7f755b400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561a0f290370 0x561a0f299f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:06:09.339 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.370+0000 7f755be00640 1 -- 172.21.15.97:0/567104925 learned_addr learned my addr 172.21.15.97:0/567104925 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:06:09.339 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.370+0000 7f7565a11740 1 -- 172.21.15.97:0/567104925 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x561a0f298250 con 0x561a0f2954e0 2024-03-27T11:06:09.339 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.370+0000 7f7560c00640 1 --2- 172.21.15.97:0/567104925 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561a0f2954e0 0x561a0f2a0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:06:09.340 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.370+0000 7f7565a11740 1 -- 172.21.15.97:0/567104925 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x561a0f1b0630 con 0x561a0f28f980 2024-03-27T11:06:09.340 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.370+0000 7f7565a11740 1 -- 172.21.15.97:0/567104925 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x561a0f2983d0 con 0x561a0f290370 2024-03-27T11:06:09.340 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.370+0000 7f7565a11740 10 client.0 _ll_register_callbacks cb 0x561a0f1b5c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T11:06:09.340 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.371+0000 7f755be00640 1 -- 172.21.15.97:0/567104925 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561a0f290370 msgr2=0x561a0f299f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:06:09.340 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.371+0000 7f755be00640 1 --2- 172.21.15.97:0/567104925 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561a0f290370 0x561a0f299f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:09.341 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.371+0000 7f755be00640 1 -- 172.21.15.97:0/567104925 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561a0f2954e0 msgr2=0x561a0f2a0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:06:09.341 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.371+0000 7f755be00640 1 --2- 172.21.15.97:0/567104925 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561a0f2954e0 0x561a0f2a0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:09.341 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.371+0000 7f755be00640 1 -- 172.21.15.97:0/567104925 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a0f1bc5f0 con 0x561a0f28f980 2024-03-27T11:06:09.341 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.371+0000 7f755b400640 1 --2- 172.21.15.97:0/567104925 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561a0f290370 0x561a0f299f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T11:06:09.341 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.371+0000 7f755be00640 1 --2- 172.21.15.97:0/567104925 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561a0f28f980 0x561a0f2999d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f754c0ba280 tx=0x7f754c0ba2b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=837bc74358c134c7 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:06:09.341 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.371+0000 7f7545600640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-03-27T11:06:09.341 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.372+0000 7f7545600640 1 -- 172.21.15.97:0/567104925 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f754c0cb070 con 0x561a0f28f980 2024-03-27T11:06:09.341 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.372+0000 7f7545600640 1 -- 172.21.15.97:0/567104925 <== mon.2 v2:172.21.15.114:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f754c0dd1f0 con 0x561a0f28f980 2024-03-27T11:06:09.341 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.373+0000 7f7565a11740 1 -- 172.21.15.97:0/567104925 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f754c004c50 con 0x561a0f28f980 2024-03-27T11:06:09.342 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.373+0000 7f7545600640 1 -- 172.21.15.97:0/567104925 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f754c0dd050 con 0x561a0f28f980 2024-03-27T11:06:09.342 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.373+0000 7f7565a11740 10 client.36506 fetch_fsmap learned FSMap version 34 2024-03-27T11:06:09.342 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.373+0000 7f7565a11740 1 -- 172.21.15.97:0/567104925 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x561a0ef09210 con 0x561a0f28f980 2024-03-27T11:06:09.342 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.373+0000 7f7545600640 1 -- 172.21.15.97:0/567104925 <== mon.2 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f754c004c50 con 0x561a0f28f980 2024-03-27T11:06:09.343 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.374+0000 7f7545600640 1 -- 172.21.15.97:0/567104925 <== mon.2 v2:172.21.15.114:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f754c0d7bb0 con 0x561a0f28f980 2024-03-27T11:06:09.343 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.374+0000 7f7565a11740 10 client.36506 fetch_fsmap finished waiting for FSMap version 34 2024-03-27T11:06:09.343 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.374+0000 7f7565a11740 10 client.36506 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T11:06:09.343 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.374+0000 7f7565a11740 20 client.36506 populate_metadata read hostname 'smithi097' 2024-03-27T11:06:09.343 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.374+0000 7f7565a11740 1 --2- 172.21.15.97:0/567104925 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x561a0f2a5630 0x561a0f2a7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:06:09.343 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.374+0000 7f7565a11740 4 client.36506 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T11:06:09.344 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.374+0000 7f7565a11740 1 -- 172.21.15.97:0/567104925 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561a0f202250 con 0x561a0f2a5630 2024-03-27T11:06:09.344 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.375+0000 7f755b400640 1 --2- 172.21.15.97:0/567104925 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x561a0f2a5630 0x561a0f2a7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:06:09.345 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.376+0000 7f755b400640 1 --2- 172.21.15.97:0/567104925 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x561a0f2a5630 0x561a0f2a7ad0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91c607737910a2f5 server_cookie=576aed17fc3d3134 in_seq=0 out_seq=0 2024-03-27T11:06:09.345 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.376+0000 7f7545600640 10 client.36506 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T11:06:09.346 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.377+0000 7f7545600640 1 -- 172.21.15.97:0/567104925 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x561a0f202250 con 0x561a0f2a5630 2024-03-27T11:06:09.346 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.377+0000 7f7545600640 10 client.36506 handle_command_reply: tid=0 2024-03-27T11:06:09.347 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.378+0000 7f7565a11740 1 client.36506 shutdown 2024-03-27T11:06:09.347 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.378+0000 7f7565a11740 1 -- 172.21.15.97:0/567104925 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561a0f28f980 msgr2=0x561a0f2999d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:06:09.347 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.378+0000 7f7565a11740 1 --2- 172.21.15.97:0/567104925 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561a0f28f980 0x561a0f2999d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f754c0ba280 tx=0x7f754c0ba2b0 comp rx=0 tx=0).stop 2024-03-27T11:06:09.348 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.379+0000 7f7565a11740 1 -- 172.21.15.97:0/567104925 shutdown_connections 2024-03-27T11:06:09.348 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.379+0000 7f7565a11740 1 --2- 172.21.15.97:0/567104925 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x561a0f2a5630 0x561a0f2a7ad0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:09.348 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.379+0000 7f7565a11740 1 --2- 172.21.15.97:0/567104925 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561a0f2954e0 0x561a0f2a0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:09.348 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.379+0000 7f7565a11740 1 --2- 172.21.15.97:0/567104925 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561a0f290370 0x561a0f299f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:09.348 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.379+0000 7f7565a11740 1 --2- 172.21.15.97:0/567104925 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561a0f28f980 0x561a0f2999d0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:09.348 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.379+0000 7f7565a11740 1 -- 172.21.15.97:0/567104925 >> 172.21.15.97:0/567104925 conn(0x561a0f28aef0 msgr2=0x561a0f28a0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:06:09.349 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.379+0000 7f7545600640 0 client.36506 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T11:06:09.349 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.379+0000 7f7565a11740 1 -- 172.21.15.97:0/567104925 shutdown_connections 2024-03-27T11:06:09.349 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.379+0000 7f7565a11740 1 -- 172.21.15.97:0/567104925 wait complete. 2024-03-27T11:06:09.349 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.379+0000 7f7565a11740 20 client.36506 trim_cache size 0 max 16384 2024-03-27T11:06:09.375 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.405+0000 7f7565a11740 1 Processor -- start 2024-03-27T11:06:09.375 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.406+0000 7f7565a11740 1 -- start start 2024-03-27T11:06:09.375 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.406+0000 7f7565a11740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561a0f28f980 0x561a0f289860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:06:09.375 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.406+0000 7f7565a11740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561a0f290370 0x561a0f2a0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:06:09.375 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.406+0000 7f7565a11740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561a0f2954e0 0x561a0f289da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:06:09.375 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.406+0000 7f7565a11740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f753c00e430 con 0x561a0f290370 2024-03-27T11:06:09.376 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.406+0000 7f7565a11740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f753c00ff70 con 0x561a0f2954e0 2024-03-27T11:06:09.376 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.406+0000 7f7565a11740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f753c00eb00 con 0x561a0f28f980 2024-03-27T11:06:09.377 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.407+0000 7f755be00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561a0f28f980 0x561a0f289860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:06:09.377 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.407+0000 7f755be00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561a0f28f980 0x561a0f289860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.97:48370/0 (socket says 172.21.15.97:48370) 2024-03-27T11:06:09.378 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.407+0000 7f755be00640 1 -- 172.21.15.97:0/1610926206 learned_addr learned my addr 172.21.15.97:0/1610926206 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:06:09.378 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.407+0000 7f7560c00640 1 --2- 172.21.15.97:0/1610926206 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561a0f2954e0 0x561a0f289da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:06:09.378 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.407+0000 7f755b400640 1 --2- 172.21.15.97:0/1610926206 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561a0f290370 0x561a0f2a0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:06:09.378 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.407+0000 7f755be00640 1 -- 172.21.15.97:0/1610926206 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561a0f2954e0 msgr2=0x561a0f289da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:06:09.378 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.407+0000 7f755be00640 1 --2- 172.21.15.97:0/1610926206 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561a0f2954e0 0x561a0f289da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:09.378 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.407+0000 7f755be00640 1 -- 172.21.15.97:0/1610926206 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561a0f290370 msgr2=0x561a0f2a0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:06:09.378 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.407+0000 7f755be00640 1 --2- 172.21.15.97:0/1610926206 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561a0f290370 0x561a0f2a0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:09.378 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.407+0000 7f755be00640 1 -- 172.21.15.97:0/1610926206 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a0efd5620 con 0x561a0f28f980 2024-03-27T11:06:09.378 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.408+0000 7f755be00640 1 --2- 172.21.15.97:0/1610926206 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561a0f28f980 0x561a0f289860 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f754c0be5b0 tx=0x7f754c02bc50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ed6d63059b729fe server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:06:09.378 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.408+0000 7f7547e00640 1 -- 172.21.15.97:0/1610926206 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f754c0cb070 con 0x561a0f28f980 2024-03-27T11:06:09.378 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.408+0000 7f7547e00640 1 -- 172.21.15.97:0/1610926206 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f754c004c50 con 0x561a0f28f980 2024-03-27T11:06:09.378 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.408+0000 7f7547e00640 1 -- 172.21.15.97:0/1610926206 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f754c0d6070 con 0x561a0f28f980 2024-03-27T11:06:09.379 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.409+0000 7f7565a11740 1 -- 172.21.15.97:0/1610926206 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561a0f28f980 msgr2=0x561a0f289860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:06:09.379 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.409+0000 7f7565a11740 1 --2- 172.21.15.97:0/1610926206 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561a0f28f980 0x561a0f289860 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f754c0be5b0 tx=0x7f754c02bc50 comp rx=0 tx=0).stop 2024-03-27T11:06:09.380 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.409+0000 7f7565a11740 1 -- 172.21.15.97:0/1610926206 shutdown_connections 2024-03-27T11:06:09.380 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.409+0000 7f7565a11740 1 --2- 172.21.15.97:0/1610926206 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561a0f2954e0 0x561a0f289da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:09.380 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.409+0000 7f7565a11740 1 --2- 172.21.15.97:0/1610926206 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561a0f290370 0x561a0f2a0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:09.380 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.409+0000 7f7565a11740 1 --2- 172.21.15.97:0/1610926206 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561a0f28f980 0x561a0f289860 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:09.380 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.409+0000 7f7565a11740 1 -- 172.21.15.97:0/1610926206 >> 172.21.15.97:0/1610926206 conn(0x561a0f28aef0 msgr2=0x561a0f2995e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:06:09.380 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.410+0000 7f7565a11740 1 -- 172.21.15.97:0/1610926206 shutdown_connections 2024-03-27T11:06:09.380 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.410+0000 7f7565a11740 1 -- 172.21.15.97:0/1610926206 wait complete. 2024-03-27T11:06:09.380 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.410+0000 7f7565a11740 1 -- start start 2024-03-27T11:06:09.380 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.411+0000 7f7565a11740 1 Processor -- start 2024-03-27T11:06:09.381 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.411+0000 7f7565a11740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561a0f28f980 0x561a0f29e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:06:09.381 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.411+0000 7f7565a11740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561a0f290370 0x561a0f29ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:06:09.381 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.411+0000 7f7565a11740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561a0f2954e0 0x561a0f29fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:06:09.381 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.411+0000 7f7565a11740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f753c00f8a0 con 0x561a0f28f980 2024-03-27T11:06:09.381 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.411+0000 7f7565a11740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f753c00f1d0 con 0x561a0f290370 2024-03-27T11:06:09.381 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.411+0000 7f7565a11740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f753c00c410 con 0x561a0f2954e0 2024-03-27T11:06:09.381 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.411+0000 7f7565a11740 10 client.0 _ll_register_callbacks cb 0x7f754c0ba8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T11:06:09.381 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.412+0000 7f7560c00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561a0f2954e0 0x561a0f29fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:06:09.382 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.412+0000 7f7560c00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561a0f2954e0 0x561a0f29fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.97:48386/0 (socket says 172.21.15.97:48386) 2024-03-27T11:06:09.382 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.412+0000 7f7560c00640 1 -- 172.21.15.97:0/2349740708 learned_addr learned my addr 172.21.15.97:0/2349740708 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:06:09.382 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.412+0000 7f755be00640 1 --2- 172.21.15.97:0/2349740708 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561a0f28f980 0x561a0f29e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:06:09.383 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.412+0000 7f755b400640 1 --2- 172.21.15.97:0/2349740708 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561a0f290370 0x561a0f29ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:06:09.383 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.412+0000 7f7560c00640 1 -- 172.21.15.97:0/2349740708 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561a0f290370 msgr2=0x561a0f29ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:06:09.383 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.412+0000 7f7560c00640 1 --2- 172.21.15.97:0/2349740708 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561a0f290370 0x561a0f29ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:09.383 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.412+0000 7f7560c00640 1 -- 172.21.15.97:0/2349740708 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561a0f28f980 msgr2=0x561a0f29e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:06:09.383 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.412+0000 7f7560c00640 1 --2- 172.21.15.97:0/2349740708 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561a0f28f980 0x561a0f29e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:09.383 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.412+0000 7f7560c00640 1 -- 172.21.15.97:0/2349740708 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a0f2a01b0 con 0x561a0f2954e0 2024-03-27T11:06:09.383 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.413+0000 7f7560c00640 1 --2- 172.21.15.97:0/2349740708 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561a0f2954e0 0x561a0f29fbc0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7554010ba0 tx=0x7f7554009200 comp rx=0 tx=0).ready entity=mon.1 client_cookie=927890d688fa034a server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:06:09.383 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.413+0000 7f7545600640 10 client.0 ms_handle_connect on v2:172.21.15.177:3300/0 2024-03-27T11:06:09.383 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.413+0000 7f7545600640 1 -- 172.21.15.97:0/2349740708 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7554013b00 con 0x561a0f2954e0 2024-03-27T11:06:09.384 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.413+0000 7f7545600640 1 -- 172.21.15.97:0/2349740708 <== mon.1 v2:172.21.15.177:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f7554013ca0 con 0x561a0f2954e0 2024-03-27T11:06:09.384 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.413+0000 7f7565a11740 1 -- 172.21.15.97:0/2349740708 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f754c004c50 con 0x561a0f2954e0 2024-03-27T11:06:09.384 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.414+0000 7f7545600640 1 -- 172.21.15.97:0/2349740708 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f755402a470 con 0x561a0f2954e0 2024-03-27T11:06:09.384 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.414+0000 7f7565a11740 10 client.26236 fetch_fsmap learned FSMap version 34 2024-03-27T11:06:09.384 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.414+0000 7f7565a11740 1 -- 172.21.15.97:0/2349740708 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x561a0f30baa0 con 0x561a0f2954e0 2024-03-27T11:06:09.384 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.414+0000 7f7545600640 1 -- 172.21.15.97:0/2349740708 <== mon.1 v2:172.21.15.177:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f754c004c50 con 0x561a0f2954e0 2024-03-27T11:06:09.385 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.414+0000 7f7545600640 1 -- 172.21.15.97:0/2349740708 <== mon.1 v2:172.21.15.177:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f755400f040 con 0x561a0f2954e0 2024-03-27T11:06:09.385 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.414+0000 7f7565a11740 10 client.26236 fetch_fsmap finished waiting for FSMap version 34 2024-03-27T11:06:09.385 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.414+0000 7f7565a11740 10 client.26236 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T11:06:09.385 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.414+0000 7f7565a11740 20 client.26236 populate_metadata read hostname 'smithi097' 2024-03-27T11:06:09.385 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.414+0000 7f7565a11740 1 --2- 172.21.15.97:0/2349740708 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x561a0f30c500 0x561a0f30e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:06:09.385 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.414+0000 7f7565a11740 4 client.26236 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-27T11:06:09.385 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.414+0000 7f7565a11740 1 -- 172.21.15.97:0/2349740708 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561a0f1fc5a0 con 0x561a0f30c500 2024-03-27T11:06:09.385 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.415+0000 7f755be00640 1 --2- 172.21.15.97:0/2349740708 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x561a0f30c500 0x561a0f30e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:06:09.385 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.416+0000 7f755be00640 1 --2- 172.21.15.97:0/2349740708 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x561a0f30c500 0x561a0f30e920 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5337905d7cf0ef7c server_cookie=b09415c351f41754 in_seq=0 out_seq=0 2024-03-27T11:06:09.385 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.416+0000 7f7545600640 10 client.26236 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T11:06:09.390 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.421+0000 7f7545600640 1 -- 172.21.15.97:0/2349740708 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561a0f1fc5a0 con 0x561a0f30c500 2024-03-27T11:06:09.391 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.421+0000 7f7545600640 10 client.26236 handle_command_reply: tid=0 2024-03-27T11:06:09.391 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.421+0000 7f7565a11740 1 client.26236 shutdown 2024-03-27T11:06:09.391 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.422+0000 7f7565a11740 1 -- 172.21.15.97:0/2349740708 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561a0f2954e0 msgr2=0x561a0f29fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:06:09.391 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.422+0000 7f7565a11740 1 --2- 172.21.15.97:0/2349740708 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561a0f2954e0 0x561a0f29fbc0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7554010ba0 tx=0x7f7554009200 comp rx=0 tx=0).stop 2024-03-27T11:06:09.392 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.422+0000 7f7565a11740 1 -- 172.21.15.97:0/2349740708 shutdown_connections 2024-03-27T11:06:09.392 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.422+0000 7f7565a11740 1 --2- 172.21.15.97:0/2349740708 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x561a0f30c500 0x561a0f30e920 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:09.392 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.422+0000 7f7565a11740 1 --2- 172.21.15.97:0/2349740708 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561a0f2954e0 0x561a0f29fbc0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:09.392 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.422+0000 7f7565a11740 1 --2- 172.21.15.97:0/2349740708 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561a0f290370 0x561a0f29ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:09.392 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.422+0000 7f7565a11740 1 --2- 172.21.15.97:0/2349740708 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561a0f28f980 0x561a0f29e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:09.392 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.422+0000 7f7565a11740 1 -- 172.21.15.97:0/2349740708 >> 172.21.15.97:0/2349740708 conn(0x561a0f28aef0 msgr2=0x561a0f28e150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:06:09.392 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.422+0000 7f7545600640 0 client.26236 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T11:06:09.393 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-27T11:06:09.393 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-27T11:06:09.393 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-27T11:06:09.393 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-27T11:06:09.393 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.423+0000 7f7565a11740 1 -- 172.21.15.97:0/2349740708 shutdown_connections 2024-03-27T11:06:09.393 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.423+0000 7f7565a11740 1 -- 172.21.15.97:0/2349740708 wait complete. 2024-03-27T11:06:09.394 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.423+0000 7f7565a11740 20 client.26236 trim_cache size 0 max 16384 2024-03-27T11:06:09.394 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.424+0000 7f7562c00640 1 -- 172.21.15.97:0/210811686 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f75400a38a0 msgr2=0x7f75400a5d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:06:09.394 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.424+0000 7f7562c00640 1 --2- 172.21.15.97:0/210811686 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f75400a38a0 0x7f75400a5d60 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f755c100730 tx=0x7f7554012760 comp rx=0 tx=0).stop 2024-03-27T11:06:09.394 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.424+0000 7f7562c00640 1 -- 172.21.15.97:0/210811686 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f755c0ff7e0 msgr2=0x7f755c19e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:06:09.394 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.424+0000 7f7562c00640 1 --2- 172.21.15.97:0/210811686 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f755c0ff7e0 0x7f755c19e370 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f754c000c00 tx=0x7f754c0042f0 comp rx=0 tx=0).stop 2024-03-27T11:06:09.394 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.424+0000 7f7562c00640 1 -- 172.21.15.97:0/210811686 shutdown_connections 2024-03-27T11:06:09.394 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.424+0000 7f7562c00640 1 --2- 172.21.15.97:0/210811686 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f755c107860 0x7f755c1a5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:09.394 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.424+0000 7f7562c00640 1 --2- 172.21.15.97:0/210811686 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f75400a38a0 0x7f75400a5d60 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:09.394 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.424+0000 7f7562c00640 1 --2- 172.21.15.97:0/210811686 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f755c1001d0 0x7f755c19e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:09.394 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.424+0000 7f7562c00640 1 --2- 172.21.15.97:0/210811686 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f755c0ff7e0 0x7f755c19e370 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:09.394 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.424+0000 7f7562c00640 1 -- 172.21.15.97:0/210811686 >> 172.21.15.97:0/210811686 conn(0x7f755c0fb330 msgr2=0x7f755c1010f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:06:09.395 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.425+0000 7f7562c00640 1 -- 172.21.15.97:0/210811686 shutdown_connections 2024-03-27T11:06:09.395 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:09.425+0000 7f7562c00640 1 -- 172.21.15.97:0/210811686 wait complete. 2024-03-27T11:06:10.174 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:06:09 smithi177 ceph-mon[35294]: pgmap v726: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 50 MiB/s wr, 824 op/s 2024-03-27T11:06:10.264 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:06:09 smithi114 ceph-mon[35338]: pgmap v726: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 50 MiB/s wr, 824 op/s 2024-03-27T11:06:10.388 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:06:09 smithi097 ceph-mon[31859]: pgmap v726: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 50 MiB/s wr, 824 op/s 2024-03-27T11:06:12.174 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:06:11 smithi177 ceph-mon[35294]: pgmap v727: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 54 MiB/s wr, 890 op/s 2024-03-27T11:06:12.264 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:06:11 smithi114 ceph-mon[35338]: pgmap v727: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 54 MiB/s wr, 890 op/s 2024-03-27T11:06:12.388 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:06:11 smithi097 ceph-mon[31859]: pgmap v727: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 54 MiB/s wr, 890 op/s 2024-03-27T11:06:14.174 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:06:13 smithi177 ceph-mon[35294]: pgmap v728: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 48 MiB/s wr, 792 op/s 2024-03-27T11:06:14.264 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:06:13 smithi114 ceph-mon[35338]: pgmap v728: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 48 MiB/s wr, 792 op/s 2024-03-27T11:06:14.388 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:06:13 smithi097 ceph-mon[31859]: pgmap v728: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 48 MiB/s wr, 792 op/s 2024-03-27T11:06:16.174 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:06:15 smithi177 ceph-mon[35294]: pgmap v729: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 49 MiB/s wr, 814 op/s 2024-03-27T11:06:16.264 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:06:15 smithi114 ceph-mon[35338]: pgmap v729: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 49 MiB/s wr, 814 op/s 2024-03-27T11:06:16.390 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:06:15 smithi097 ceph-mon[31859]: pgmap v729: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 49 MiB/s wr, 814 op/s 2024-03-27T11:06:18.174 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:06:17 smithi177 ceph-mon[35294]: pgmap v730: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 47 MiB/s wr, 782 op/s 2024-03-27T11:06:18.264 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:06:17 smithi114 ceph-mon[35338]: pgmap v730: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 47 MiB/s wr, 782 op/s 2024-03-27T11:06:18.389 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:06:17 smithi097 ceph-mon[31859]: pgmap v730: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 47 MiB/s wr, 782 op/s 2024-03-27T11:06:20.265 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:06:19 smithi114 ceph-mon[35338]: pgmap v731: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 50 MiB/s wr, 825 op/s 2024-03-27T11:06:20.389 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:06:19 smithi097 ceph-mon[31859]: pgmap v731: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 50 MiB/s wr, 825 op/s 2024-03-27T11:06:20.424 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:06:19 smithi177 ceph-mon[35294]: pgmap v731: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 50 MiB/s wr, 825 op/s 2024-03-27T11:06:22.264 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:06:21 smithi114 ceph-mon[35338]: pgmap v732: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 53 MiB/s wr, 880 op/s 2024-03-27T11:06:22.388 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:06:21 smithi097 ceph-mon[31859]: pgmap v732: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 53 MiB/s wr, 880 op/s 2024-03-27T11:06:22.424 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:06:21 smithi177 ceph-mon[35294]: pgmap v732: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 53 MiB/s wr, 880 op/s 2024-03-27T11:06:22.635 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.666+0000 7ff946e00640 1 -- 172.21.15.97:0/1012482248 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9401094b0 msgr2=0x7ff94010fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:06:22.636 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.666+0000 7ff946e00640 1 --2- 172.21.15.97:0/1012482248 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9401094b0 0x7ff94010fe90 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7ff930009860 tx=0x7ff93002bb90 comp rx=0 tx=0).stop 2024-03-27T11:06:22.636 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.667+0000 7ff946e00640 1 -- 172.21.15.97:0/1012482248 shutdown_connections 2024-03-27T11:06:22.636 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.667+0000 7ff946e00640 1 --2- 172.21.15.97:0/1012482248 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff9401103d0 0x7ff9401127c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:22.636 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.667+0000 7ff946e00640 1 --2- 172.21.15.97:0/1012482248 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9401094b0 0x7ff94010fe90 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:22.637 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.667+0000 7ff946e00640 1 --2- 172.21.15.97:0/1012482248 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff940108b50 0x7ff940108f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:22.637 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.667+0000 7ff946e00640 1 -- 172.21.15.97:0/1012482248 >> 172.21.15.97:0/1012482248 conn(0x7ff94006eb90 msgr2=0x7ff940070fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:06:22.637 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.667+0000 7ff946e00640 1 -- 172.21.15.97:0/1012482248 shutdown_connections 2024-03-27T11:06:22.637 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.667+0000 7ff946e00640 1 -- 172.21.15.97:0/1012482248 wait complete. 2024-03-27T11:06:22.638 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.669+0000 7ff946e00640 1 Processor -- start 2024-03-27T11:06:22.639 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.669+0000 7ff946e00640 1 -- start start 2024-03-27T11:06:22.639 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.670+0000 7ff946e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff940108b50 0x7ff9401a2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:06:22.639 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.670+0000 7ff946e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff9401094b0 0x7ff9401a2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:06:22.639 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.670+0000 7ff946e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9401103d0 0x7ff9401a9c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:06:22.639 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.670+0000 7ff946e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff9401180a0 con 0x7ff9401103d0 2024-03-27T11:06:22.639 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.670+0000 7ff946e00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff940117f20 con 0x7ff940108b50 2024-03-27T11:06:22.640 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.670+0000 7ff946e00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff940118220 con 0x7ff9401094b0 2024-03-27T11:06:22.640 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.671+0000 7ff93fe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff940108b50 0x7ff9401a2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:06:22.641 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.671+0000 7ff93fe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff940108b50 0x7ff9401a2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:40116/0 (socket says 172.21.15.97:40116) 2024-03-27T11:06:22.641 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.671+0000 7ff93fe00640 1 -- 172.21.15.97:0/1582870408 learned_addr learned my addr 172.21.15.97:0/1582870408 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:06:22.641 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.671+0000 7ff944e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9401103d0 0x7ff9401a9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:06:22.641 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.671+0000 7ff93f400640 1 --2- 172.21.15.97:0/1582870408 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff9401094b0 0x7ff9401a2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:06:22.642 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.672+0000 7ff93fe00640 1 -- 172.21.15.97:0/1582870408 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff9401094b0 msgr2=0x7ff9401a2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:06:22.642 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.672+0000 7ff93fe00640 1 --2- 172.21.15.97:0/1582870408 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff9401094b0 0x7ff9401a2bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:22.642 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.672+0000 7ff93fe00640 1 -- 172.21.15.97:0/1582870408 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9401103d0 msgr2=0x7ff9401a9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:06:22.642 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.672+0000 7ff93fe00640 1 --2- 172.21.15.97:0/1582870408 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9401103d0 0x7ff9401a9c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:22.642 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.672+0000 7ff93fe00640 1 -- 172.21.15.97:0/1582870408 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9401aa2a0 con 0x7ff940108b50 2024-03-27T11:06:22.642 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.673+0000 7ff93fe00640 1 --2- 172.21.15.97:0/1582870408 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff940108b50 0x7ff9401a2680 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff93800ebc0 tx=0x7ff93800c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:06:22.643 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.674+0000 7ff93cc00640 1 -- 172.21.15.97:0/1582870408 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff938010630 con 0x7ff940108b50 2024-03-27T11:06:22.643 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.674+0000 7ff93cc00640 1 -- 172.21.15.97:0/1582870408 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff938010c80 con 0x7ff940108b50 2024-03-27T11:06:22.643 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.674+0000 7ff93cc00640 1 -- 172.21.15.97:0/1582870408 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff938018b30 con 0x7ff940108b50 2024-03-27T11:06:22.644 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.674+0000 7ff946e00640 1 -- 172.21.15.97:0/1582870408 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9401aa590 con 0x7ff940108b50 2024-03-27T11:06:22.644 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.675+0000 7ff946e00640 1 -- 172.21.15.97:0/1582870408 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9401aa9a0 con 0x7ff940108b50 2024-03-27T11:06:22.649 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.680+0000 7ff949a74740 1 Processor -- start 2024-03-27T11:06:22.649 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.680+0000 7ff93cc00640 1 -- 172.21.15.97:0/1582870408 <== mon.2 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7ff938018cd0 con 0x7ff940108b50 2024-03-27T11:06:22.649 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.680+0000 7ff949a74740 1 -- start start 2024-03-27T11:06:22.651 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.681+0000 7ff949a74740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555a813ab980 0x555a813abda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:06:22.651 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.681+0000 7ff949a74740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555a813ac370 0x555a813b0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:06:22.651 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.681+0000 7ff949a74740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555a813b14e0 0x555a813b3970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:06:22.651 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.681+0000 7ff949a74740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x555a81310d10 con 0x555a813ab980 2024-03-27T11:06:22.651 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.681+0000 7ff949a74740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x555a812b9260 con 0x555a813b14e0 2024-03-27T11:06:22.651 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.681+0000 7ff949a74740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x555a8130ccc0 con 0x555a813ac370 2024-03-27T11:06:22.651 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.682+0000 7ff93fe00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555a813ac370 0x555a813b0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:06:22.651 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.682+0000 7ff93fe00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555a813ac370 0x555a813b0e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.97:59672/0 (socket says 172.21.15.97:59672) 2024-03-27T11:06:22.651 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.682+0000 7ff93fe00640 1 -- 172.21.15.97:0/2058709347 learned_addr learned my addr 172.21.15.97:0/2058709347 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:06:22.652 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.682+0000 7ff944e00640 1 --2- 172.21.15.97:0/2058709347 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555a813ab980 0x555a813abda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:06:22.653 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.682+0000 7ff93f400640 1 --2- 172.21.15.97:0/2058709347 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555a813b14e0 0x555a813b3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:06:22.653 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.683+0000 7ff93fe00640 1 -- 172.21.15.97:0/2058709347 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555a813b14e0 msgr2=0x555a813b3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:06:22.653 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.683+0000 7ff93fe00640 1 --2- 172.21.15.97:0/2058709347 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555a813b14e0 0x555a813b3970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:22.653 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.683+0000 7ff93fe00640 1 -- 172.21.15.97:0/2058709347 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555a813ab980 msgr2=0x555a813abda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:06:22.653 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.683+0000 7ff93fe00640 1 --2- 172.21.15.97:0/2058709347 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555a813ab980 0x555a813abda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:22.653 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.683+0000 7ff93fe00640 1 -- 172.21.15.97:0/2058709347 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a812b0aa0 con 0x555a813ac370 2024-03-27T11:06:22.653 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.683+0000 7ff944e00640 1 --2- 172.21.15.97:0/2058709347 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555a813ab980 0x555a813abda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T11:06:22.653 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.683+0000 7ff93cc00640 1 --2- 172.21.15.97:0/1582870408 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7ff9240a37b0 0x7ff9240a5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:06:22.653 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.683+0000 7ff93fe00640 1 --2- 172.21.15.97:0/2058709347 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555a813ac370 0x555a813b0e70 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff9380d84b0 tx=0x7ff9380d8970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3a79e1cb4605773 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:06:22.653 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.684+0000 7ff944e00640 1 --2- 172.21.15.97:0/1582870408 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7ff9240a37b0 0x7ff9240a5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:06:22.654 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.684+0000 7ff93cc00640 1 -- 172.21.15.97:0/1582870408 <== mon.2 v2:172.21.15.114:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7ff9380d0280 con 0x7ff940108b50 2024-03-27T11:06:22.654 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.684+0000 7ff935600640 1 -- 172.21.15.97:0/2058709347 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff938026020 con 0x555a813ac370 2024-03-27T11:06:22.654 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.684+0000 7ff935600640 1 -- 172.21.15.97:0/2058709347 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff93809c7b0 con 0x555a813ac370 2024-03-27T11:06:22.654 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.684+0000 7ff935600640 1 -- 172.21.15.97:0/2058709347 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff938026020 con 0x555a813ac370 2024-03-27T11:06:22.655 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.685+0000 7ff949a74740 1 -- 172.21.15.97:0/2058709347 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555a813ac370 msgr2=0x555a813b0e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:06:22.655 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.685+0000 7ff949a74740 1 --2- 172.21.15.97:0/2058709347 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555a813ac370 0x555a813b0e70 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff9380d84b0 tx=0x7ff9380d8970 comp rx=0 tx=0).stop 2024-03-27T11:06:22.655 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.685+0000 7ff944e00640 1 --2- 172.21.15.97:0/1582870408 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7ff9240a37b0 0x7ff9240a5c70 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x555a813acbd0 tx=0x7ff92800b2e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:06:22.655 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.686+0000 7ff949a74740 1 -- 172.21.15.97:0/2058709347 shutdown_connections 2024-03-27T11:06:22.655 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.686+0000 7ff949a74740 1 --2- 172.21.15.97:0/2058709347 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555a813b14e0 0x555a813b3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:22.655 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.686+0000 7ff949a74740 1 --2- 172.21.15.97:0/2058709347 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555a813ac370 0x555a813b0e70 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:22.655 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.686+0000 7ff949a74740 1 --2- 172.21.15.97:0/2058709347 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555a813ab980 0x555a813abda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:22.655 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.686+0000 7ff949a74740 1 -- 172.21.15.97:0/2058709347 >> 172.21.15.97:0/2058709347 conn(0x555a813a6ef0 msgr2=0x555a813a72f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:06:22.655 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.686+0000 7ff949a74740 1 -- 172.21.15.97:0/2058709347 shutdown_connections 2024-03-27T11:06:22.656 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.686+0000 7ff949a74740 1 -- 172.21.15.97:0/2058709347 wait complete. 2024-03-27T11:06:22.656 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.686+0000 7ff949a74740 1 -- start start 2024-03-27T11:06:22.656 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.687+0000 7ff949a74740 1 Processor -- start 2024-03-27T11:06:22.657 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.688+0000 7ff949a74740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555a813ab980 0x555a813b5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:06:22.657 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.688+0000 7ff949a74740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555a813ac370 0x555a813b61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:06:22.657 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.688+0000 7ff949a74740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555a813b14e0 0x555a813bd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:06:22.657 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.688+0000 7ff93fe00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555a813ab980 0x555a813b5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:06:22.658 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.688+0000 7ff93fe00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555a813ab980 0x555a813b5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.97:59674/0 (socket says 172.21.15.97:59674) 2024-03-27T11:06:22.658 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.689+0000 7ff944e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555a813b14e0 0x555a813bd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:06:22.658 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.689+0000 7ff93f400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555a813ac370 0x555a813b61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:06:22.658 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.689+0000 7ff944e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555a813b14e0 0x555a813bd1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47276/0 (socket says 172.21.15.97:47276) 2024-03-27T11:06:22.658 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.689+0000 7ff93f400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555a813ac370 0x555a813b61b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:40124/0 (socket says 172.21.15.97:40124) 2024-03-27T11:06:22.659 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.689+0000 7ff949a74740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x555a813b4490 con 0x555a813b14e0 2024-03-27T11:06:22.659 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.689+0000 7ff949a74740 1 -- 172.21.15.97:0/2531980923 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x555a812cc630 con 0x555a813ac370 2024-03-27T11:06:22.659 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.689+0000 7ff949a74740 1 -- 172.21.15.97:0/2531980923 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x555a813b4610 con 0x555a813ab980 2024-03-27T11:06:22.659 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.689+0000 7ff93fe00640 1 -- 172.21.15.97:0/2531980923 learned_addr learned my addr 172.21.15.97:0/2531980923 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:06:22.659 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.689+0000 7ff949a74740 10 client.0 _ll_register_callbacks cb 0x555a812d1c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T11:06:22.660 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.690+0000 7ff944e00640 1 -- 172.21.15.97:0/2531980923 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555a813ab980 msgr2=0x555a813b5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:06:22.660 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.690+0000 7ff944e00640 1 --2- 172.21.15.97:0/2531980923 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555a813ab980 0x555a813b5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:22.660 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.690+0000 7ff944e00640 1 -- 172.21.15.97:0/2531980923 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555a813ac370 msgr2=0x555a813b61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:06:22.660 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.690+0000 7ff944e00640 1 --2- 172.21.15.97:0/2531980923 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555a813ac370 0x555a813b61b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:22.660 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.690+0000 7ff944e00640 1 -- 172.21.15.97:0/2531980923 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a812d85f0 con 0x555a813b14e0 2024-03-27T11:06:22.660 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.690+0000 7ff93fe00640 1 --2- 172.21.15.97:0/2531980923 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555a813ab980 0x555a813b5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T11:06:22.660 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.690+0000 7ff944e00640 1 --2- 172.21.15.97:0/2531980923 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555a813b14e0 0x555a813bd1f0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7ff928012010 tx=0x7ff928004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a8c0ca976f81b08 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:06:22.660 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.690+0000 7ff92e000640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T11:06:22.660 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.691+0000 7ff92e000640 1 -- 172.21.15.97:0/2531980923 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff92802c060 con 0x555a813b14e0 2024-03-27T11:06:22.660 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.691+0000 7ff949a74740 1 -- 172.21.15.97:0/2531980923 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff93809c7b0 con 0x555a813b14e0 2024-03-27T11:06:22.661 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.691+0000 7ff92e000640 1 -- 172.21.15.97:0/2531980923 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7ff928017030 con 0x555a813b14e0 2024-03-27T11:06:22.661 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.691+0000 7ff92e000640 1 -- 172.21.15.97:0/2531980923 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7ff92801c600 con 0x555a813b14e0 2024-03-27T11:06:22.662 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.691+0000 7ff949a74740 10 client.16851 fetch_fsmap learned FSMap version 34 2024-03-27T11:06:22.662 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.691+0000 7ff949a74740 1 -- 172.21.15.97:0/2531980923 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x555a81025210 con 0x555a813b14e0 2024-03-27T11:06:22.662 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.692+0000 7ff92e000640 1 -- 172.21.15.97:0/2531980923 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff93809c7b0 con 0x555a813b14e0 2024-03-27T11:06:22.662 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.692+0000 7ff92e000640 1 -- 172.21.15.97:0/2531980923 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7ff9280278b0 con 0x555a813b14e0 2024-03-27T11:06:22.662 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.692+0000 7ff949a74740 10 client.16851 fetch_fsmap finished waiting for FSMap version 34 2024-03-27T11:06:22.662 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.692+0000 7ff949a74740 10 client.16851 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T11:06:22.662 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.692+0000 7ff949a74740 20 client.16851 populate_metadata read hostname 'smithi097' 2024-03-27T11:06:22.662 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.692+0000 7ff949a74740 1 --2- 172.21.15.97:0/2531980923 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x555a813c1b30 0x555a813c3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:06:22.662 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.692+0000 7ff949a74740 4 client.16851 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T11:06:22.662 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.692+0000 7ff949a74740 1 -- 172.21.15.97:0/2531980923 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555a8131e250 con 0x555a813c1b30 2024-03-27T11:06:22.663 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.694+0000 7ff93fe00640 1 --2- 172.21.15.97:0/2531980923 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x555a813c1b30 0x555a813c3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:06:22.664 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.695+0000 7ff93fe00640 1 --2- 172.21.15.97:0/2531980923 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x555a813c1b30 0x555a813c3f60 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a1912815fb900f3 server_cookie=1331729ad9f0c725 in_seq=0 out_seq=0 2024-03-27T11:06:22.664 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.695+0000 7ff92e000640 10 client.16851 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T11:06:22.665 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.696+0000 7ff92e000640 1 -- 172.21.15.97:0/2531980923 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x555a8131e250 con 0x555a813c1b30 2024-03-27T11:06:22.665 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.696+0000 7ff92e000640 10 client.16851 handle_command_reply: tid=0 2024-03-27T11:06:22.665 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.696+0000 7ff949a74740 1 client.16851 shutdown 2024-03-27T11:06:22.666 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.697+0000 7ff949a74740 1 -- 172.21.15.97:0/2531980923 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555a813b14e0 msgr2=0x555a813bd1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:06:22.666 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.697+0000 7ff949a74740 1 --2- 172.21.15.97:0/2531980923 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555a813b14e0 0x555a813bd1f0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7ff928012010 tx=0x7ff928004730 comp rx=0 tx=0).stop 2024-03-27T11:06:22.666 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.697+0000 7ff949a74740 1 -- 172.21.15.97:0/2531980923 shutdown_connections 2024-03-27T11:06:22.667 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.697+0000 7ff949a74740 1 --2- 172.21.15.97:0/2531980923 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x555a813c1b30 0x555a813c3f60 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:22.667 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.697+0000 7ff949a74740 1 --2- 172.21.15.97:0/2531980923 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555a813b14e0 0x555a813bd1f0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:22.667 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.697+0000 7ff949a74740 1 --2- 172.21.15.97:0/2531980923 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555a813ac370 0x555a813b61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:22.667 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.697+0000 7ff92e000640 0 client.16851 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T11:06:22.667 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.697+0000 7ff949a74740 1 --2- 172.21.15.97:0/2531980923 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555a813ab980 0x555a813b5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:22.667 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.697+0000 7ff949a74740 1 -- 172.21.15.97:0/2531980923 >> 172.21.15.97:0/2531980923 conn(0x555a813a6ef0 msgr2=0x555a813a60f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:06:22.667 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.697+0000 7ff949a74740 1 -- 172.21.15.97:0/2531980923 shutdown_connections 2024-03-27T11:06:22.667 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.697+0000 7ff949a74740 1 -- 172.21.15.97:0/2531980923 wait complete. 2024-03-27T11:06:22.667 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.697+0000 7ff949a74740 20 client.16851 trim_cache size 0 max 16384 2024-03-27T11:06:22.693 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.724+0000 7ff949a74740 1 Processor -- start 2024-03-27T11:06:22.694 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.725+0000 7ff949a74740 1 -- start start 2024-03-27T11:06:22.694 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.725+0000 7ff949a74740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555a813ab980 0x555a813ad690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:06:22.694 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.725+0000 7ff949a74740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555a813ac370 0x555a813adbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:06:22.694 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.725+0000 7ff949a74740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555a813b14e0 0x555a813bd110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:06:22.694 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.725+0000 7ff949a74740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff9401b3830 con 0x555a813ab980 2024-03-27T11:06:22.695 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.725+0000 7ff949a74740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff9401b5370 con 0x555a813ac370 2024-03-27T11:06:22.695 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.725+0000 7ff949a74740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff9401b3f00 con 0x555a813b14e0 2024-03-27T11:06:22.695 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.725+0000 7ff93f400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555a813ac370 0x555a813adbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:06:22.695 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.726+0000 7ff944e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555a813b14e0 0x555a813bd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:06:22.695 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.726+0000 7ff944e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555a813b14e0 0x555a813bd110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.97:59678/0 (socket says 172.21.15.97:59678) 2024-03-27T11:06:22.696 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.726+0000 7ff944e00640 1 -- 172.21.15.97:0/3969143896 learned_addr learned my addr 172.21.15.97:0/3969143896 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:06:22.696 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.726+0000 7ff93fe00640 1 --2- 172.21.15.97:0/3969143896 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555a813ab980 0x555a813ad690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:06:22.696 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.726+0000 7ff944e00640 1 -- 172.21.15.97:0/3969143896 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555a813ac370 msgr2=0x555a813adbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:06:22.696 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.726+0000 7ff944e00640 1 --2- 172.21.15.97:0/3969143896 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555a813ac370 0x555a813adbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:22.696 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.726+0000 7ff944e00640 1 -- 172.21.15.97:0/3969143896 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555a813ab980 msgr2=0x555a813ad690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:06:22.696 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.726+0000 7ff944e00640 1 --2- 172.21.15.97:0/3969143896 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555a813ab980 0x555a813ad690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:22.696 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.726+0000 7ff944e00640 1 -- 172.21.15.97:0/3969143896 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a810f1620 con 0x555a813b14e0 2024-03-27T11:06:22.696 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.726+0000 7ff93fe00640 1 --2- 172.21.15.97:0/3969143896 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555a813ab980 0x555a813ad690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T11:06:22.697 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.727+0000 7ff944e00640 1 --2- 172.21.15.97:0/3969143896 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555a813b14e0 0x555a813bd110 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff928018040 tx=0x7ff928010640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7390b5cff4dfbce9 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:06:22.697 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.727+0000 7ff93f400640 1 --2- 172.21.15.97:0/3969143896 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555a813ac370 0x555a813adbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T11:06:22.697 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.727+0000 7ff934c00640 1 -- 172.21.15.97:0/3969143896 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff928037070 con 0x555a813b14e0 2024-03-27T11:06:22.697 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.728+0000 7ff934c00640 1 -- 172.21.15.97:0/3969143896 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff92801ce50 con 0x555a813b14e0 2024-03-27T11:06:22.697 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.728+0000 7ff934c00640 1 -- 172.21.15.97:0/3969143896 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9280109f0 con 0x555a813b14e0 2024-03-27T11:06:22.698 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.729+0000 7ff949a74740 1 -- 172.21.15.97:0/3969143896 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555a813b14e0 msgr2=0x555a813bd110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:06:22.698 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.729+0000 7ff949a74740 1 --2- 172.21.15.97:0/3969143896 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555a813b14e0 0x555a813bd110 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff928018040 tx=0x7ff928010640 comp rx=0 tx=0).stop 2024-03-27T11:06:22.698 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.729+0000 7ff949a74740 1 -- 172.21.15.97:0/3969143896 shutdown_connections 2024-03-27T11:06:22.698 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.729+0000 7ff949a74740 1 --2- 172.21.15.97:0/3969143896 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555a813b14e0 0x555a813bd110 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:22.698 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.729+0000 7ff949a74740 1 --2- 172.21.15.97:0/3969143896 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555a813ac370 0x555a813adbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:22.698 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.729+0000 7ff949a74740 1 --2- 172.21.15.97:0/3969143896 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555a813ab980 0x555a813ad690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:22.698 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.729+0000 7ff949a74740 1 -- 172.21.15.97:0/3969143896 >> 172.21.15.97:0/3969143896 conn(0x555a813a6ef0 msgr2=0x555a813a72d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:06:22.698 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.729+0000 7ff949a74740 1 -- 172.21.15.97:0/3969143896 shutdown_connections 2024-03-27T11:06:22.699 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.729+0000 7ff949a74740 1 -- 172.21.15.97:0/3969143896 wait complete. 2024-03-27T11:06:22.699 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.730+0000 7ff949a74740 1 -- start start 2024-03-27T11:06:22.699 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.730+0000 7ff949a74740 1 Processor -- start 2024-03-27T11:06:22.700 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.730+0000 7ff949a74740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555a813ab980 0x555a813ba6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:06:22.700 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.730+0000 7ff949a74740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555a813ac370 0x555a813bac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:06:22.700 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.731+0000 7ff949a74740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555a813b14e0 0x555a813bbc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:06:22.700 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.731+0000 7ff949a74740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff9401b4ca0 con 0x555a813ab980 2024-03-27T11:06:22.700 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.731+0000 7ff949a74740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff9401b45d0 con 0x555a813b14e0 2024-03-27T11:06:22.700 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.731+0000 7ff949a74740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff9401b1810 con 0x555a813ac370 2024-03-27T11:06:22.700 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.731+0000 7ff949a74740 10 client.0 _ll_register_callbacks cb 0x7ff928005d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T11:06:22.701 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.731+0000 7ff93fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555a813ab980 0x555a813ba6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:06:22.701 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.731+0000 7ff93fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555a813ab980 0x555a813ba6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47288/0 (socket says 172.21.15.97:47288) 2024-03-27T11:06:22.701 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.731+0000 7ff93fe00640 1 -- 172.21.15.97:0/89012389 learned_addr learned my addr 172.21.15.97:0/89012389 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:06:22.701 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.731+0000 7ff944e00640 1 --2- 172.21.15.97:0/89012389 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555a813b14e0 0x555a813bbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:06:22.701 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.731+0000 7ff93f400640 1 --2- 172.21.15.97:0/89012389 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555a813ac370 0x555a813bac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:06:22.701 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.732+0000 7ff93fe00640 1 -- 172.21.15.97:0/89012389 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555a813ac370 msgr2=0x555a813bac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:06:22.701 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.732+0000 7ff93fe00640 1 --2- 172.21.15.97:0/89012389 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555a813ac370 0x555a813bac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:22.701 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.732+0000 7ff93fe00640 1 -- 172.21.15.97:0/89012389 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555a813b14e0 msgr2=0x555a813bbc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:06:22.701 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.732+0000 7ff93fe00640 1 --2- 172.21.15.97:0/89012389 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555a813b14e0 0x555a813bbc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:22.701 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.732+0000 7ff93fe00640 1 -- 172.21.15.97:0/89012389 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a813bc210 con 0x555a813ab980 2024-03-27T11:06:22.702 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.732+0000 7ff93fe00640 1 --2- 172.21.15.97:0/89012389 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555a813ab980 0x555a813ba6d0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7ff938099180 tx=0x7ff93801f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80528e4956aecb45 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:06:22.702 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.732+0000 7ff92e000640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T11:06:22.702 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.733+0000 7ff92e000640 1 -- 172.21.15.97:0/89012389 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff938098a50 con 0x555a813ab980 2024-03-27T11:06:22.702 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.733+0000 7ff92e000640 1 -- 172.21.15.97:0/89012389 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7ff93809a850 con 0x555a813ab980 2024-03-27T11:06:22.702 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.733+0000 7ff949a74740 1 -- 172.21.15.97:0/89012389 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff92801ce50 con 0x555a813ab980 2024-03-27T11:06:22.703 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.733+0000 7ff92e000640 1 -- 172.21.15.97:0/89012389 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7ff93809c110 con 0x555a813ab980 2024-03-27T11:06:22.703 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.733+0000 7ff949a74740 10 client.16857 fetch_fsmap learned FSMap version 34 2024-03-27T11:06:22.703 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.734+0000 7ff949a74740 1 -- 172.21.15.97:0/89012389 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x555a813bd500 con 0x555a813ab980 2024-03-27T11:06:22.703 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.734+0000 7ff92e000640 1 -- 172.21.15.97:0/89012389 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff92801ce50 con 0x555a813ab980 2024-03-27T11:06:22.703 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.734+0000 7ff92e000640 1 -- 172.21.15.97:0/89012389 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7ff9380bb2a0 con 0x555a813ab980 2024-03-27T11:06:22.704 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.734+0000 7ff949a74740 10 client.16857 fetch_fsmap finished waiting for FSMap version 34 2024-03-27T11:06:22.704 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.734+0000 7ff949a74740 10 client.16857 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T11:06:22.704 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.734+0000 7ff949a74740 20 client.16857 populate_metadata read hostname 'smithi097' 2024-03-27T11:06:22.704 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.734+0000 7ff949a74740 1 --2- 172.21.15.97:0/89012389 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x555a81430970 0x555a81432d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:06:22.704 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.734+0000 7ff949a74740 4 client.16857 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-27T11:06:22.704 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.734+0000 7ff949a74740 1 -- 172.21.15.97:0/89012389 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555a813185a0 con 0x555a81430970 2024-03-27T11:06:22.704 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.735+0000 7ff93f400640 1 --2- 172.21.15.97:0/89012389 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x555a81430970 0x555a81432d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:06:22.705 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.736+0000 7ff93f400640 1 --2- 172.21.15.97:0/89012389 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x555a81430970 0x555a81432d90 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8ba444ab6cce95b server_cookie=4993c05e602748e3 in_seq=0 out_seq=0 2024-03-27T11:06:22.705 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.736+0000 7ff92e000640 10 client.16857 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T11:06:22.725 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.756+0000 7ff92e000640 1 -- 172.21.15.97:0/89012389 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555a813185a0 con 0x555a81430970 2024-03-27T11:06:22.725 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.756+0000 7ff92e000640 10 client.16857 handle_command_reply: tid=0 2024-03-27T11:06:22.725 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.756+0000 7ff949a74740 1 client.16857 shutdown 2024-03-27T11:06:22.726 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.756+0000 7ff949a74740 1 -- 172.21.15.97:0/89012389 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555a813ab980 msgr2=0x555a813ba6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:06:22.726 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.756+0000 7ff949a74740 1 --2- 172.21.15.97:0/89012389 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555a813ab980 0x555a813ba6d0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7ff938099180 tx=0x7ff93801f000 comp rx=0 tx=0).stop 2024-03-27T11:06:22.726 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.757+0000 7ff949a74740 1 -- 172.21.15.97:0/89012389 shutdown_connections 2024-03-27T11:06:22.727 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.757+0000 7ff949a74740 1 --2- 172.21.15.97:0/89012389 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x555a81430970 0x555a81432d90 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:22.727 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.757+0000 7ff949a74740 1 --2- 172.21.15.97:0/89012389 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555a813b14e0 0x555a813bbc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:22.727 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.757+0000 7ff949a74740 1 --2- 172.21.15.97:0/89012389 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555a813ac370 0x555a813bac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:22.727 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.757+0000 7ff949a74740 1 --2- 172.21.15.97:0/89012389 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555a813ab980 0x555a813ba6d0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:22.727 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.757+0000 7ff949a74740 1 -- 172.21.15.97:0/89012389 >> 172.21.15.97:0/89012389 conn(0x555a813a6ef0 msgr2=0x555a813b0880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:06:22.727 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.758+0000 7ff92e000640 0 client.16857 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T11:06:22.727 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.758+0000 7ff949a74740 1 -- 172.21.15.97:0/89012389 shutdown_connections 2024-03-27T11:06:22.728 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.758+0000 7ff949a74740 1 -- 172.21.15.97:0/89012389 wait complete. 2024-03-27T11:06:22.728 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.758+0000 7ff949a74740 20 client.16857 trim_cache size 0 max 16384 2024-03-27T11:06:22.728 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-27T11:06:22.728 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-27T11:06:22.728 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-27T11:06:22.728 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-27T11:06:22.730 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.761+0000 7ff946e00640 1 -- 172.21.15.97:0/1582870408 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7ff9240a37b0 msgr2=0x7ff9240a5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:06:22.730 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.761+0000 7ff946e00640 1 --2- 172.21.15.97:0/1582870408 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7ff9240a37b0 0x7ff9240a5c70 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x555a813acbd0 tx=0x7ff92800b2e0 comp rx=0 tx=0).stop 2024-03-27T11:06:22.730 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.761+0000 7ff946e00640 1 -- 172.21.15.97:0/1582870408 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff940108b50 msgr2=0x7ff9401a2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:06:22.730 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.761+0000 7ff946e00640 1 --2- 172.21.15.97:0/1582870408 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff940108b50 0x7ff9401a2680 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff93800ebc0 tx=0x7ff93800c6a0 comp rx=0 tx=0).stop 2024-03-27T11:06:22.730 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.761+0000 7ff946e00640 1 -- 172.21.15.97:0/1582870408 shutdown_connections 2024-03-27T11:06:22.730 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.761+0000 7ff946e00640 1 --2- 172.21.15.97:0/1582870408 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9401103d0 0x7ff9401a9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:22.730 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.761+0000 7ff946e00640 1 --2- 172.21.15.97:0/1582870408 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff9401094b0 0x7ff9401a2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:22.730 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.761+0000 7ff946e00640 1 --2- 172.21.15.97:0/1582870408 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7ff9240a37b0 0x7ff9240a5c70 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:22.731 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.761+0000 7ff946e00640 1 --2- 172.21.15.97:0/1582870408 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff940108b50 0x7ff9401a2680 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:22.731 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.761+0000 7ff946e00640 1 -- 172.21.15.97:0/1582870408 >> 172.21.15.97:0/1582870408 conn(0x7ff94006eb90 msgr2=0x7ff9400708b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:06:22.731 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.762+0000 7ff946e00640 1 -- 172.21.15.97:0/1582870408 shutdown_connections 2024-03-27T11:06:22.731 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:22.762+0000 7ff946e00640 1 -- 172.21.15.97:0/1582870408 wait complete. 2024-03-27T11:06:24.228 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:06:23 smithi177 ceph-mon[35294]: pgmap v733: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 48 MiB/s wr, 800 op/s 2024-03-27T11:06:24.264 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:06:23 smithi114 ceph-mon[35338]: pgmap v733: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 48 MiB/s wr, 800 op/s 2024-03-27T11:06:24.388 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:06:23 smithi097 ceph-mon[31859]: pgmap v733: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 48 MiB/s wr, 800 op/s 2024-03-27T11:06:26.264 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:06:25 smithi114 ceph-mon[35338]: pgmap v734: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 49 MiB/s wr, 821 op/s 2024-03-27T11:06:26.390 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:06:25 smithi097 ceph-mon[31859]: pgmap v734: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 49 MiB/s wr, 821 op/s 2024-03-27T11:06:26.424 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:06:25 smithi177 ceph-mon[35294]: pgmap v734: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 49 MiB/s wr, 821 op/s 2024-03-27T11:06:28.264 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:06:27 smithi114 ceph-mon[35338]: pgmap v735: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 47 MiB/s wr, 791 op/s 2024-03-27T11:06:28.388 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:06:27 smithi097 ceph-mon[31859]: pgmap v735: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 47 MiB/s wr, 791 op/s 2024-03-27T11:06:28.424 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:06:27 smithi177 ceph-mon[35294]: pgmap v735: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 47 MiB/s wr, 791 op/s 2024-03-27T11:06:30.265 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:06:29 smithi114 ceph-mon[35338]: pgmap v736: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 50 MiB/s wr, 828 op/s 2024-03-27T11:06:30.388 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:06:29 smithi097 ceph-mon[31859]: pgmap v736: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 50 MiB/s wr, 828 op/s 2024-03-27T11:06:30.423 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:06:29 smithi177 ceph-mon[35294]: pgmap v736: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 50 MiB/s wr, 828 op/s 2024-03-27T11:06:32.264 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:06:31 smithi114 ceph-mon[35338]: pgmap v737: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 53 MiB/s wr, 895 op/s 2024-03-27T11:06:32.388 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:06:31 smithi097 ceph-mon[31859]: pgmap v737: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 53 MiB/s wr, 895 op/s 2024-03-27T11:06:32.424 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:06:31 smithi177 ceph-mon[35294]: pgmap v737: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 53 MiB/s wr, 895 op/s 2024-03-27T11:06:34.264 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:06:34 smithi114 ceph-mon[35338]: pgmap v738: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 48 MiB/s wr, 808 op/s 2024-03-27T11:06:34.389 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:06:33 smithi097 ceph-mon[31859]: pgmap v738: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 48 MiB/s wr, 808 op/s 2024-03-27T11:06:34.424 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:06:33 smithi177 ceph-mon[35294]: pgmap v738: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 48 MiB/s wr, 808 op/s 2024-03-27T11:06:35.389 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:06:35 smithi097 ceph-mon[31859]: pgmap v739: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 48 MiB/s wr, 814 op/s 2024-03-27T11:06:35.424 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:06:35 smithi177 ceph-mon[35294]: pgmap v739: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 48 MiB/s wr, 814 op/s 2024-03-27T11:06:35.514 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:06:35 smithi114 ceph-mon[35338]: pgmap v739: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 48 MiB/s wr, 814 op/s 2024-03-27T11:06:35.965 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:35.994+0000 7f15a4400640 1 -- 172.21.15.97:0/1822611648 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f159c075b90 msgr2=0x7f159c073fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:06:35.965 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:35.994+0000 7f15a4400640 1 --2- 172.21.15.97:0/1822611648 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f159c075b90 0x7f159c073fe0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1588009900 tx=0x7f158802bc40 comp rx=0 tx=0).stop 2024-03-27T11:06:35.965 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:35.994+0000 7f15a4400640 1 -- 172.21.15.97:0/1822611648 shutdown_connections 2024-03-27T11:06:35.965 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:35.994+0000 7f15a4400640 1 --2- 172.21.15.97:0/1822611648 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f159c109a40 0x7f159c10bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:35.965 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:35.994+0000 7f15a4400640 1 --2- 172.21.15.97:0/1822611648 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f159c0745b0 0x7f159c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:35.965 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:35.994+0000 7f15a4400640 1 --2- 172.21.15.97:0/1822611648 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f159c075b90 0x7f159c073fe0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:35.965 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:35.994+0000 7f15a4400640 1 -- 172.21.15.97:0/1822611648 >> 172.21.15.97:0/1822611648 conn(0x7f159c0fd110 msgr2=0x7f159c0ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:06:35.965 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:35.995+0000 7f15a4400640 1 -- 172.21.15.97:0/1822611648 shutdown_connections 2024-03-27T11:06:35.966 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:35.995+0000 7f15a4400640 1 -- 172.21.15.97:0/1822611648 wait complete. 2024-03-27T11:06:35.966 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:35.996+0000 7f15a4400640 1 Processor -- start 2024-03-27T11:06:35.966 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:35.996+0000 7f15a4400640 1 -- start start 2024-03-27T11:06:35.967 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:35.997+0000 7f15a4400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f159c0745b0 0x7f159c1958a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:06:35.967 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:35.997+0000 7f15a4400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f159c075b90 0x7f159c195de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:06:35.967 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:35.997+0000 7f15a4400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f159c109a40 0x7f159c19ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:06:35.967 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:35.997+0000 7f15a4400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f159c10f8c0 con 0x7f159c0745b0 2024-03-27T11:06:35.967 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:35.997+0000 7f15a4400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f159c10f740 con 0x7f159c109a40 2024-03-27T11:06:35.967 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:35.997+0000 7f15a4400640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f159c10fa40 con 0x7f159c075b90 2024-03-27T11:06:35.968 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:35.998+0000 7f15a2400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f159c109a40 0x7f159c19ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:06:35.968 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:35.998+0000 7f15a1a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f159c0745b0 0x7f159c1958a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:06:35.968 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:35.998+0000 7f15a2400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f159c109a40 0x7f159c19ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:41822/0 (socket says 172.21.15.97:41822) 2024-03-27T11:06:35.969 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:35.998+0000 7f15a2400640 1 -- 172.21.15.97:0/1264207388 learned_addr learned my addr 172.21.15.97:0/1264207388 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:06:35.969 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:35.998+0000 7f15a1000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f159c075b90 0x7f159c195de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:06:35.969 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:35.998+0000 7f15a2400640 1 -- 172.21.15.97:0/1264207388 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f159c075b90 msgr2=0x7f159c195de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:06:35.969 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:35.998+0000 7f15a2400640 1 --2- 172.21.15.97:0/1264207388 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f159c075b90 0x7f159c195de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:35.969 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:35.998+0000 7f15a2400640 1 -- 172.21.15.97:0/1264207388 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f159c0745b0 msgr2=0x7f159c1958a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:06:35.969 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:35.998+0000 7f15a2400640 1 --2- 172.21.15.97:0/1264207388 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f159c0745b0 0x7f159c1958a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:35.969 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:35.998+0000 7f15a2400640 1 -- 172.21.15.97:0/1264207388 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f159c19d4c0 con 0x7f159c109a40 2024-03-27T11:06:35.970 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:35.999+0000 7f15a2400640 1 --2- 172.21.15.97:0/1264207388 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f159c109a40 0x7f159c19ce50 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f159400eaa0 tx=0x7f159400ef60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:06:35.970 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:35.999+0000 7f159a000640 1 -- 172.21.15.97:0/1264207388 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f159400cdb0 con 0x7f159c109a40 2024-03-27T11:06:35.970 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:35.999+0000 7f159a000640 1 -- 172.21.15.97:0/1264207388 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1594005020 con 0x7f159c109a40 2024-03-27T11:06:35.970 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:35.999+0000 7f159a000640 1 -- 172.21.15.97:0/1264207388 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15940116d0 con 0x7f159c109a40 2024-03-27T11:06:35.970 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:35.999+0000 7f15a4400640 1 -- 172.21.15.97:0/1264207388 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f159c1ae7c0 con 0x7f159c109a40 2024-03-27T11:06:35.970 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:35.999+0000 7f15a4400640 1 -- 172.21.15.97:0/1264207388 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f159c1aecc0 con 0x7f159c109a40 2024-03-27T11:06:35.974 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.004+0000 7f15a71c0740 1 Processor -- start 2024-03-27T11:06:35.974 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.004+0000 7f159a000640 1 -- 172.21.15.97:0/1264207388 <== mon.2 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f15940051d0 con 0x7f159c109a40 2024-03-27T11:06:35.974 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.004+0000 7f15a71c0740 1 -- start start 2024-03-27T11:06:35.975 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.005+0000 7f15a71c0740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556e9291b980 0x556e9291bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:06:35.975 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.005+0000 7f15a71c0740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556e9291c370 0x556e92920e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:06:35.975 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.005+0000 7f15a71c0740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556e929214e0 0x556e92923970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:06:35.975 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.005+0000 7f15a71c0740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x556e92880d10 con 0x556e9291b980 2024-03-27T11:06:35.975 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.005+0000 7f15a71c0740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x556e92829260 con 0x556e929214e0 2024-03-27T11:06:35.975 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.005+0000 7f15a71c0740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x556e9287ccc0 con 0x556e9291c370 2024-03-27T11:06:35.976 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.006+0000 7f15a1000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556e9291b980 0x556e9291bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:06:35.977 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.006+0000 7f15a1000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556e9291b980 0x556e9291bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60166/0 (socket says 172.21.15.97:60166) 2024-03-27T11:06:35.977 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.006+0000 7f15a1a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556e9291c370 0x556e92920e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:06:35.977 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.006+0000 7f15a1a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556e9291c370 0x556e92920e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.97:58540/0 (socket says 172.21.15.97:58540) 2024-03-27T11:06:35.977 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.006+0000 7f15a1a00640 1 -- 172.21.15.97:0/2290706099 learned_addr learned my addr 172.21.15.97:0/2290706099 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:06:35.977 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.006+0000 7f159a000640 1 --2- 172.21.15.97:0/1264207388 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f15840a36b0 0x7f15840a5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:06:35.977 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.006+0000 7f159a000640 1 -- 172.21.15.97:0/1264207388 <== mon.2 v2:172.21.15.114:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f15940cff10 con 0x7f159c109a40 2024-03-27T11:06:35.977 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.007+0000 7f15a2400640 1 --2- 172.21.15.97:0/1264207388 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f15840a36b0 0x7f15840a5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:06:35.978 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.007+0000 7f15a2400640 1 --2- 172.21.15.97:0/1264207388 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f15840a36b0 0x7f15840a5b70 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f1594098850 tx=0x7f159400cf50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:06:35.978 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.008+0000 7f15a1000640 1 --2- 172.21.15.97:0/2290706099 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556e929214e0 0x556e92923970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:06:35.978 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.008+0000 7f15a1a00640 1 -- 172.21.15.97:0/2290706099 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556e929214e0 msgr2=0x556e92923970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:06:35.978 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.008+0000 7f15a1a00640 1 --2- 172.21.15.97:0/2290706099 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556e929214e0 0x556e92923970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:35.978 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.008+0000 7f15a1a00640 1 -- 172.21.15.97:0/2290706099 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556e9291b980 msgr2=0x556e9291bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:06:35.978 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.008+0000 7f15a1a00640 1 --2- 172.21.15.97:0/2290706099 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556e9291b980 0x556e9291bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:35.978 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.008+0000 7f15a1a00640 1 -- 172.21.15.97:0/2290706099 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e92820aa0 con 0x556e9291c370 2024-03-27T11:06:35.978 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.008+0000 7f15a1000640 1 --2- 172.21.15.97:0/2290706099 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556e9291b980 0x556e9291bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T11:06:35.979 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.010+0000 7f15a1a00640 1 --2- 172.21.15.97:0/2290706099 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556e9291c370 0x556e92920e70 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1588009900 tx=0x7f1588009560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a6d1f215a7a7720 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:06:35.980 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.010+0000 7f158ea00640 1 -- 172.21.15.97:0/2290706099 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1588045020 con 0x556e9291c370 2024-03-27T11:06:35.980 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.010+0000 7f158ea00640 1 -- 172.21.15.97:0/2290706099 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f158802cbb0 con 0x556e9291c370 2024-03-27T11:06:35.980 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.010+0000 7f158ea00640 1 -- 172.21.15.97:0/2290706099 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1588004920 con 0x556e9291c370 2024-03-27T11:06:35.981 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.011+0000 7f15a71c0740 1 -- 172.21.15.97:0/2290706099 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556e9291c370 msgr2=0x556e92920e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:06:35.981 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.011+0000 7f15a71c0740 1 --2- 172.21.15.97:0/2290706099 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556e9291c370 0x556e92920e70 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1588009900 tx=0x7f1588009560 comp rx=0 tx=0).stop 2024-03-27T11:06:35.981 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.011+0000 7f15a71c0740 1 -- 172.21.15.97:0/2290706099 shutdown_connections 2024-03-27T11:06:35.982 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.011+0000 7f15a71c0740 1 --2- 172.21.15.97:0/2290706099 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556e929214e0 0x556e92923970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:35.982 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.011+0000 7f15a71c0740 1 --2- 172.21.15.97:0/2290706099 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556e9291c370 0x556e92920e70 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:35.982 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.011+0000 7f15a71c0740 1 --2- 172.21.15.97:0/2290706099 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556e9291b980 0x556e9291bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:35.982 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.011+0000 7f15a71c0740 1 -- 172.21.15.97:0/2290706099 >> 172.21.15.97:0/2290706099 conn(0x556e92916ef0 msgr2=0x556e929172f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:06:35.982 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.011+0000 7f15a71c0740 1 -- 172.21.15.97:0/2290706099 shutdown_connections 2024-03-27T11:06:35.982 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.012+0000 7f15a71c0740 1 -- 172.21.15.97:0/2290706099 wait complete. 2024-03-27T11:06:35.983 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.012+0000 7f15a71c0740 1 -- start start 2024-03-27T11:06:35.983 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.012+0000 7f15a71c0740 1 Processor -- start 2024-03-27T11:06:35.983 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.013+0000 7f15a71c0740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556e9291b980 0x556e92925c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:06:35.984 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.013+0000 7f15a71c0740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556e9291c370 0x556e929261c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:06:35.984 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.013+0000 7f15a71c0740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556e929214e0 0x556e9292d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:06:35.984 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.013+0000 7f15a71c0740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x556e92924430 con 0x556e9291c370 2024-03-27T11:06:35.984 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.013+0000 7f15a71c0740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x556e9283c630 con 0x556e929214e0 2024-03-27T11:06:35.984 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.013+0000 7f15a71c0740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x556e929245b0 con 0x556e9291b980 2024-03-27T11:06:35.984 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.013+0000 7f15a71c0740 10 client.0 _ll_register_callbacks cb 0x556e92841c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T11:06:35.984 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.014+0000 7f15a1a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556e9291c370 0x556e929261c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:06:35.984 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.014+0000 7f15a1000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556e9291b980 0x556e92925c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:06:35.984 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.014+0000 7f15a1a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556e9291c370 0x556e929261c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60176/0 (socket says 172.21.15.97:60176) 2024-03-27T11:06:35.984 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.014+0000 7f15a1a00640 1 -- 172.21.15.97:0/1026093537 learned_addr learned my addr 172.21.15.97:0/1026093537 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:06:35.984 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.014+0000 7f15a1000640 1 --2- 172.21.15.97:0/1026093537 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556e929214e0 0x556e9292d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:06:35.985 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.014+0000 7f15a1a00640 1 -- 172.21.15.97:0/1026093537 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556e9291b980 msgr2=0x556e92925c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:06:35.985 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.014+0000 7f15a1a00640 1 --2- 172.21.15.97:0/1026093537 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556e9291b980 0x556e92925c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:35.985 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.014+0000 7f15a1a00640 1 -- 172.21.15.97:0/1026093537 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556e929214e0 msgr2=0x556e9292d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:06:35.985 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.014+0000 7f15a1a00640 1 --2- 172.21.15.97:0/1026093537 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556e929214e0 0x556e9292d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:35.986 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.014+0000 7f15a1a00640 1 -- 172.21.15.97:0/1026093537 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e928485f0 con 0x556e9291c370 2024-03-27T11:06:35.986 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.015+0000 7f15a1a00640 1 --2- 172.21.15.97:0/1026093537 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556e9291c370 0x556e929261c0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f1588036040 tx=0x7f158802c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ee569db937df801 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:06:35.986 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.015+0000 7f1583400640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T11:06:35.986 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.015+0000 7f1583400640 1 -- 172.21.15.97:0/1026093537 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1588045070 con 0x556e9291c370 2024-03-27T11:06:35.986 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.015+0000 7f1583400640 1 -- 172.21.15.97:0/1026093537 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f1588003a00 con 0x556e9291c370 2024-03-27T11:06:35.986 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.015+0000 7f15a71c0740 1 -- 172.21.15.97:0/1026093537 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f158802cbb0 con 0x556e9291c370 2024-03-27T11:06:35.987 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.016+0000 7f1583400640 1 -- 172.21.15.97:0/1026093537 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f158805b050 con 0x556e9291c370 2024-03-27T11:06:35.987 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.016+0000 7f15a71c0740 10 client.16869 fetch_fsmap learned FSMap version 34 2024-03-27T11:06:35.987 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.016+0000 7f15a71c0740 1 -- 172.21.15.97:0/1026093537 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x556e92595210 con 0x556e9291c370 2024-03-27T11:06:35.987 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.016+0000 7f1583400640 1 -- 172.21.15.97:0/1026093537 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f158802cbb0 con 0x556e9291c370 2024-03-27T11:06:35.987 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.017+0000 7f1583400640 1 -- 172.21.15.97:0/1026093537 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f1588041cc0 con 0x556e9291c370 2024-03-27T11:06:35.987 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.017+0000 7f15a71c0740 10 client.16869 fetch_fsmap finished waiting for FSMap version 34 2024-03-27T11:06:35.987 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.017+0000 7f15a71c0740 10 client.16869 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T11:06:35.987 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.017+0000 7f15a71c0740 20 client.16869 populate_metadata read hostname 'smithi097' 2024-03-27T11:06:35.987 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.017+0000 7f15a71c0740 1 --2- 172.21.15.97:0/1026093537 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x556e929316f0 0x556e92933ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:06:35.987 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.017+0000 7f15a71c0740 4 client.16869 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T11:06:35.988 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.017+0000 7f15a71c0740 1 -- 172.21.15.97:0/1026093537 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556e9288e250 con 0x556e929316f0 2024-03-27T11:06:35.988 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.018+0000 7f15a1000640 1 --2- 172.21.15.97:0/1026093537 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x556e929316f0 0x556e92933ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:06:35.990 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.019+0000 7f15a1000640 1 --2- 172.21.15.97:0/1026093537 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x556e929316f0 0x556e92933ae0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ed9caa09da6bfe7 server_cookie=9012cca8c42c032e in_seq=0 out_seq=0 2024-03-27T11:06:35.991 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.019+0000 7f1583400640 10 client.16869 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T11:06:35.991 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.020+0000 7f1583400640 1 -- 172.21.15.97:0/1026093537 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x556e9288e250 con 0x556e929316f0 2024-03-27T11:06:35.991 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.020+0000 7f1583400640 10 client.16869 handle_command_reply: tid=0 2024-03-27T11:06:35.991 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.020+0000 7f15a71c0740 1 client.16869 shutdown 2024-03-27T11:06:35.991 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.020+0000 7f15a71c0740 1 -- 172.21.15.97:0/1026093537 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556e9291c370 msgr2=0x556e929261c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:06:35.991 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.020+0000 7f15a71c0740 1 --2- 172.21.15.97:0/1026093537 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556e9291c370 0x556e929261c0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f1588036040 tx=0x7f158802c670 comp rx=0 tx=0).stop 2024-03-27T11:06:35.991 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.020+0000 7f15a71c0740 1 -- 172.21.15.97:0/1026093537 shutdown_connections 2024-03-27T11:06:35.991 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.020+0000 7f15a71c0740 1 --2- 172.21.15.97:0/1026093537 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x556e929316f0 0x556e92933ae0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:35.991 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.020+0000 7f15a71c0740 1 --2- 172.21.15.97:0/1026093537 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556e929214e0 0x556e9292d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:35.991 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.020+0000 7f15a71c0740 1 --2- 172.21.15.97:0/1026093537 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556e9291c370 0x556e929261c0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:35.991 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.020+0000 7f15a71c0740 1 --2- 172.21.15.97:0/1026093537 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556e9291b980 0x556e92925c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:35.991 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.020+0000 7f1583400640 0 client.16869 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T11:06:35.992 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.020+0000 7f15a71c0740 1 -- 172.21.15.97:0/1026093537 >> 172.21.15.97:0/1026093537 conn(0x556e92916ef0 msgr2=0x556e929160f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:06:35.992 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.022+0000 7f15a71c0740 1 -- 172.21.15.97:0/1026093537 shutdown_connections 2024-03-27T11:06:35.992 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.022+0000 7f15a71c0740 1 -- 172.21.15.97:0/1026093537 wait complete. 2024-03-27T11:06:35.992 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.022+0000 7f15a71c0740 20 client.16869 trim_cache size 0 max 16384 2024-03-27T11:06:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.052+0000 7f15a71c0740 1 Processor -- start 2024-03-27T11:06:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.052+0000 7f15a71c0740 1 -- start start 2024-03-27T11:06:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.052+0000 7f15a71c0740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556e9291b980 0x556e9291d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:06:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.053+0000 7f15a71c0740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556e929214e0 0x556e9291dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:06:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.053+0000 7f15a71c0740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556e9292d1c0 0x556e92915860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:06:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.053+0000 7f15a71c0740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f159c1b7970 con 0x556e9291b980 2024-03-27T11:06:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.053+0000 7f15a71c0740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f159c1b94b0 con 0x556e9292d1c0 2024-03-27T11:06:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.053+0000 7f15a71c0740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f159c1b8040 con 0x556e929214e0 2024-03-27T11:06:36.024 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.053+0000 7f15a1a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556e929214e0 0x556e9291dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:06:36.024 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.053+0000 7f15a1a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556e929214e0 0x556e9291dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.97:58546/0 (socket says 172.21.15.97:58546) 2024-03-27T11:06:36.024 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.053+0000 7f15a1a00640 1 -- 172.21.15.97:0/248926877 learned_addr learned my addr 172.21.15.97:0/248926877 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:06:36.024 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.053+0000 7f15a1000640 1 --2- 172.21.15.97:0/248926877 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556e9291b980 0x556e9291d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:06:36.024 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.053+0000 7f15a1000640 1 --2- 172.21.15.97:0/248926877 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556e9292d1c0 0x556e92915860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:06:36.024 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.054+0000 7f15a1a00640 1 -- 172.21.15.97:0/248926877 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556e9292d1c0 msgr2=0x556e92915860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:06:36.024 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.054+0000 7f15a1a00640 1 --2- 172.21.15.97:0/248926877 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556e9292d1c0 0x556e92915860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:36.025 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.054+0000 7f15a1a00640 1 -- 172.21.15.97:0/248926877 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556e9291b980 msgr2=0x556e9291d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:06:36.025 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.054+0000 7f15a1a00640 1 --2- 172.21.15.97:0/248926877 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556e9291b980 0x556e9291d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:36.025 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.054+0000 7f15a1a00640 1 -- 172.21.15.97:0/248926877 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e92661620 con 0x556e929214e0 2024-03-27T11:06:36.025 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.054+0000 7f15a1a00640 1 --2- 172.21.15.97:0/248926877 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556e929214e0 0x556e9291dc10 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1588005ec0 tx=0x7f1588005a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c62299f8fae5e06 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:06:36.025 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.055+0000 7f158e000640 1 -- 172.21.15.97:0/248926877 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1588058070 con 0x556e929214e0 2024-03-27T11:06:36.026 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.055+0000 7f158e000640 1 -- 172.21.15.97:0/248926877 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f158802cbb0 con 0x556e929214e0 2024-03-27T11:06:36.026 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.055+0000 7f158e000640 1 -- 172.21.15.97:0/248926877 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1588045040 con 0x556e929214e0 2024-03-27T11:06:36.026 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.055+0000 7f15a71c0740 1 -- 172.21.15.97:0/248926877 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556e929214e0 msgr2=0x556e9291dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:06:36.026 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.055+0000 7f15a71c0740 1 --2- 172.21.15.97:0/248926877 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556e929214e0 0x556e9291dc10 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1588005ec0 tx=0x7f1588005a20 comp rx=0 tx=0).stop 2024-03-27T11:06:36.026 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.056+0000 7f15a71c0740 1 -- 172.21.15.97:0/248926877 shutdown_connections 2024-03-27T11:06:36.026 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.056+0000 7f15a71c0740 1 --2- 172.21.15.97:0/248926877 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556e9292d1c0 0x556e92915860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:36.026 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.056+0000 7f15a71c0740 1 --2- 172.21.15.97:0/248926877 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556e929214e0 0x556e9291dc10 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:36.026 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.056+0000 7f15a71c0740 1 --2- 172.21.15.97:0/248926877 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556e9291b980 0x556e9291d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:36.026 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.056+0000 7f15a71c0740 1 -- 172.21.15.97:0/248926877 >> 172.21.15.97:0/248926877 conn(0x556e92916ef0 msgr2=0x556e92925c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:06:36.027 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.056+0000 7f15a71c0740 1 -- 172.21.15.97:0/248926877 shutdown_connections 2024-03-27T11:06:36.027 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.056+0000 7f15a71c0740 1 -- 172.21.15.97:0/248926877 wait complete. 2024-03-27T11:06:36.027 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.056+0000 7f15a71c0740 1 -- start start 2024-03-27T11:06:36.027 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.057+0000 7f15a71c0740 1 Processor -- start 2024-03-27T11:06:36.029 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.057+0000 7f15a71c0740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556e9291b980 0x556e9292a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:06:36.029 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.057+0000 7f15a71c0740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556e929214e0 0x556e9292ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:06:36.029 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.057+0000 7f15a71c0740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556e9292d1c0 0x556e9292bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:06:36.029 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.057+0000 7f15a71c0740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f159c1b8de0 con 0x556e929214e0 2024-03-27T11:06:36.029 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.057+0000 7f15a71c0740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f159c1b8710 con 0x556e9291b980 2024-03-27T11:06:36.029 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.057+0000 7f15a71c0740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f159c1b5950 con 0x556e9292d1c0 2024-03-27T11:06:36.029 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.058+0000 7f15a71c0740 10 client.0 _ll_register_callbacks cb 0x7f1590009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T11:06:36.031 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.059+0000 7f15a1a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556e929214e0 0x556e9292ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:06:36.031 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.060+0000 7f15a1a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556e929214e0 0x556e9292ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60184/0 (socket says 172.21.15.97:60184) 2024-03-27T11:06:36.031 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.060+0000 7f15a1a00640 1 -- 172.21.15.97:0/1116936091 learned_addr learned my addr 172.21.15.97:0/1116936091 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:06:36.031 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.060+0000 7f15a1000640 1 --2- 172.21.15.97:0/1116936091 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556e9291b980 0x556e9292a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:06:36.031 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.060+0000 7f15a1000640 1 --2- 172.21.15.97:0/1116936091 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556e9292d1c0 0x556e9292bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:06:36.031 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.060+0000 7f15a1a00640 1 -- 172.21.15.97:0/1116936091 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556e9292d1c0 msgr2=0x556e9292bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:06:36.032 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.061+0000 7f15a1a00640 1 --2- 172.21.15.97:0/1116936091 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556e9292d1c0 0x556e9292bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:36.033 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.062+0000 7f15a1a00640 1 -- 172.21.15.97:0/1116936091 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556e9291b980 msgr2=0x556e9292a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:06:36.033 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.062+0000 7f15a1a00640 1 --2- 172.21.15.97:0/1116936091 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556e9291b980 0x556e9292a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:36.033 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.062+0000 7f15a1a00640 1 -- 172.21.15.97:0/1116936091 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e9292c160 con 0x556e929214e0 2024-03-27T11:06:36.033 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.062+0000 7f15a1000640 1 --2- 172.21.15.97:0/1116936091 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556e9292d1c0 0x556e9292bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T11:06:36.034 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.064+0000 7f15a1a00640 1 --2- 172.21.15.97:0/1116936091 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556e929214e0 0x556e9292ab60 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f1588005ab0 tx=0x7f158802c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdabc17f57ad90d0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:06:36.034 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.064+0000 7f1583400640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T11:06:36.035 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.065+0000 7f1583400640 1 -- 172.21.15.97:0/1116936091 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1588058070 con 0x556e929214e0 2024-03-27T11:06:36.035 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.065+0000 7f1583400640 1 -- 172.21.15.97:0/1116936091 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f1588005040 con 0x556e929214e0 2024-03-27T11:06:36.036 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.066+0000 7f15a71c0740 1 -- 172.21.15.97:0/1116936091 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f158802cbb0 con 0x556e929214e0 2024-03-27T11:06:36.036 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.066+0000 7f1583400640 1 -- 172.21.15.97:0/1116936091 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f1588045040 con 0x556e929214e0 2024-03-27T11:06:36.037 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.066+0000 7f15a71c0740 10 client.16875 fetch_fsmap learned FSMap version 34 2024-03-27T11:06:36.037 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.066+0000 7f15a71c0740 1 -- 172.21.15.97:0/1116936091 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x556e92997c80 con 0x556e929214e0 2024-03-27T11:06:36.037 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.067+0000 7f1583400640 1 -- 172.21.15.97:0/1116936091 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f158802cbb0 con 0x556e929214e0 2024-03-27T11:06:36.037 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.067+0000 7f1583400640 1 -- 172.21.15.97:0/1116936091 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f1588067030 con 0x556e929214e0 2024-03-27T11:06:36.037 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.067+0000 7f15a71c0740 10 client.16875 fetch_fsmap finished waiting for FSMap version 34 2024-03-27T11:06:36.038 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.067+0000 7f15a71c0740 10 client.16875 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T11:06:36.038 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.067+0000 7f15a71c0740 20 client.16875 populate_metadata read hostname 'smithi097' 2024-03-27T11:06:36.038 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.067+0000 7f15a71c0740 1 --2- 172.21.15.97:0/1116936091 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x556e92998600 0x556e9299aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:06:36.038 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.067+0000 7f15a71c0740 4 client.16875 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-27T11:06:36.038 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.067+0000 7f15a71c0740 1 -- 172.21.15.97:0/1116936091 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556e928885a0 con 0x556e92998600 2024-03-27T11:06:36.038 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.068+0000 7f15a1000640 1 --2- 172.21.15.97:0/1116936091 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x556e92998600 0x556e9299aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:06:36.039 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.069+0000 7f15a1000640 1 --2- 172.21.15.97:0/1116936091 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x556e92998600 0x556e9299aa20 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e57c7d370a13146e server_cookie=6d9fbb256cf20208 in_seq=0 out_seq=0 2024-03-27T11:06:36.040 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.071+0000 7f1583400640 10 client.16875 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T11:06:36.047 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.077+0000 7f1583400640 1 -- 172.21.15.97:0/1116936091 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556e928885a0 con 0x556e92998600 2024-03-27T11:06:36.048 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.077+0000 7f1583400640 10 client.16875 handle_command_reply: tid=0 2024-03-27T11:06:36.048 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.077+0000 7f15a71c0740 1 client.16875 shutdown 2024-03-27T11:06:36.048 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.078+0000 7f15a71c0740 1 -- 172.21.15.97:0/1116936091 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556e929214e0 msgr2=0x556e9292ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:06:36.048 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.078+0000 7f15a71c0740 1 --2- 172.21.15.97:0/1116936091 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556e929214e0 0x556e9292ab60 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f1588005ab0 tx=0x7f158802c900 comp rx=0 tx=0).stop 2024-03-27T11:06:36.048 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.078+0000 7f15a71c0740 1 -- 172.21.15.97:0/1116936091 shutdown_connections 2024-03-27T11:06:36.048 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.078+0000 7f15a71c0740 1 --2- 172.21.15.97:0/1116936091 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x556e92998600 0x556e9299aa20 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:36.048 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.078+0000 7f15a71c0740 1 --2- 172.21.15.97:0/1116936091 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556e9292d1c0 0x556e9292bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:36.049 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.078+0000 7f15a71c0740 1 --2- 172.21.15.97:0/1116936091 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556e929214e0 0x556e9292ab60 secure :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f1588005ab0 tx=0x7f158802c900 comp rx=0 tx=0).stop 2024-03-27T11:06:36.049 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.078+0000 7f15a71c0740 1 --2- 172.21.15.97:0/1116936091 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556e9291b980 0x556e9292a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:36.049 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.078+0000 7f1583400640 0 client.16875 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T11:06:36.049 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.078+0000 7f15a71c0740 1 -- 172.21.15.97:0/1116936091 >> 172.21.15.97:0/1116936091 conn(0x556e92916ef0 msgr2=0x556e92925c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:06:36.049 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.079+0000 7f15a71c0740 1 -- 172.21.15.97:0/1116936091 shutdown_connections 2024-03-27T11:06:36.050 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.079+0000 7f15a71c0740 1 -- 172.21.15.97:0/1116936091 wait complete. 2024-03-27T11:06:36.050 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.079+0000 7f15a71c0740 20 client.16875 trim_cache size 0 max 16384 2024-03-27T11:06:36.050 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-27T11:06:36.050 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-27T11:06:36.050 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-27T11:06:36.050 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-27T11:06:36.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.081+0000 7f15a4400640 1 -- 172.21.15.97:0/1264207388 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f15840a36b0 msgr2=0x7f15840a5b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:06:36.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.081+0000 7f15a4400640 1 --2- 172.21.15.97:0/1264207388 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f15840a36b0 0x7f15840a5b70 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f1594098850 tx=0x7f159400cf50 comp rx=0 tx=0).stop 2024-03-27T11:06:36.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.081+0000 7f15a4400640 1 -- 172.21.15.97:0/1264207388 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f159c109a40 msgr2=0x7f159c19ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:06:36.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.081+0000 7f15a4400640 1 --2- 172.21.15.97:0/1264207388 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f159c109a40 0x7f159c19ce50 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f159400eaa0 tx=0x7f159400ef60 comp rx=0 tx=0).stop 2024-03-27T11:06:36.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.081+0000 7f15a4400640 1 -- 172.21.15.97:0/1264207388 shutdown_connections 2024-03-27T11:06:36.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.081+0000 7f15a4400640 1 --2- 172.21.15.97:0/1264207388 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f15840a36b0 0x7f15840a5b70 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:36.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.081+0000 7f15a4400640 1 --2- 172.21.15.97:0/1264207388 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f159c109a40 0x7f159c19ce50 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:36.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.081+0000 7f15a4400640 1 --2- 172.21.15.97:0/1264207388 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f159c075b90 0x7f159c195de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:36.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.081+0000 7f15a4400640 1 --2- 172.21.15.97:0/1264207388 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f159c0745b0 0x7f159c1958a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:36.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.081+0000 7f15a4400640 1 -- 172.21.15.97:0/1264207388 >> 172.21.15.97:0/1264207388 conn(0x7f159c0fd110 msgr2=0x7f159c101530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:06:36.053 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.082+0000 7f15a4400640 1 -- 172.21.15.97:0/1264207388 shutdown_connections 2024-03-27T11:06:36.053 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:36.082+0000 7f15a4400640 1 -- 172.21.15.97:0/1264207388 wait complete. 2024-03-27T11:06:37.638 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:06:37 smithi097 ceph-mon[31859]: pgmap v740: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 46 MiB/s wr, 784 op/s 2024-03-27T11:06:37.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:06:37 smithi177 ceph-mon[35294]: pgmap v740: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 46 MiB/s wr, 784 op/s 2024-03-27T11:06:37.764 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:06:37 smithi114 ceph-mon[35338]: pgmap v740: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 46 MiB/s wr, 784 op/s 2024-03-27T11:06:39.638 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:06:39 smithi097 ceph-mon[31859]: pgmap v741: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 49 MiB/s wr, 826 op/s 2024-03-27T11:06:39.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:06:39 smithi177 ceph-mon[35294]: pgmap v741: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 49 MiB/s wr, 826 op/s 2024-03-27T11:06:39.764 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:06:39 smithi114 ceph-mon[35338]: pgmap v741: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 49 MiB/s wr, 826 op/s 2024-03-27T11:06:41.638 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:06:41 smithi097 ceph-mon[31859]: pgmap v742: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 53 MiB/s wr, 898 op/s 2024-03-27T11:06:41.673 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:06:41 smithi177 ceph-mon[35294]: pgmap v742: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 53 MiB/s wr, 898 op/s 2024-03-27T11:06:41.764 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:06:41 smithi114 ceph-mon[35338]: pgmap v742: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 53 MiB/s wr, 898 op/s 2024-03-27T11:06:43.638 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:06:43 smithi097 ceph-mon[31859]: pgmap v743: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 47 MiB/s wr, 803 op/s 2024-03-27T11:06:43.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:06:43 smithi177 ceph-mon[35294]: pgmap v743: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 47 MiB/s wr, 803 op/s 2024-03-27T11:06:43.764 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:06:43 smithi114 ceph-mon[35338]: pgmap v743: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 47 MiB/s wr, 803 op/s 2024-03-27T11:06:45.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:06:45 smithi177 ceph-mon[35294]: pgmap v744: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 48 MiB/s wr, 824 op/s 2024-03-27T11:06:45.764 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:06:45 smithi114 ceph-mon[35338]: pgmap v744: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 48 MiB/s wr, 824 op/s 2024-03-27T11:06:45.889 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:06:45 smithi097 ceph-mon[31859]: pgmap v744: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 48 MiB/s wr, 824 op/s 2024-03-27T11:06:47.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:06:47 smithi177 ceph-mon[35294]: pgmap v745: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 47 MiB/s wr, 799 op/s 2024-03-27T11:06:47.764 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:06:47 smithi114 ceph-mon[35338]: pgmap v745: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 47 MiB/s wr, 799 op/s 2024-03-27T11:06:47.888 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:06:47 smithi097 ceph-mon[31859]: pgmap v745: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 47 MiB/s wr, 799 op/s 2024-03-27T11:06:49.272 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.302+0000 7f63e8e00640 1 -- 172.21.15.97:0/1125383481 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f63e4077750 msgr2=0x7f63e4077bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:06:49.272 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.302+0000 7f63e8e00640 1 --2- 172.21.15.97:0/1125383481 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f63e4077750 0x7f63e4077bb0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f63cc009900 tx=0x7f63cc02bc40 comp rx=0 tx=0).stop 2024-03-27T11:06:49.272 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.302+0000 7f63e8e00640 1 -- 172.21.15.97:0/1125383481 shutdown_connections 2024-03-27T11:06:49.272 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.302+0000 7f63e8e00640 1 --2- 172.21.15.97:0/1125383481 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63e40780f0 0x7f63e4078990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:49.273 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.302+0000 7f63e8e00640 1 --2- 172.21.15.97:0/1125383481 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f63e4077750 0x7f63e4077bb0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:49.273 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.302+0000 7f63e8e00640 1 --2- 172.21.15.97:0/1125383481 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f63e4076510 0x7f63e4076930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:49.273 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.302+0000 7f63e8e00640 1 -- 172.21.15.97:0/1125383481 >> 172.21.15.97:0/1125383481 conn(0x7f63e40ff310 msgr2=0x7f63e4101750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:06:49.273 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.302+0000 7f63e8e00640 1 -- 172.21.15.97:0/1125383481 shutdown_connections 2024-03-27T11:06:49.273 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.302+0000 7f63e8e00640 1 -- 172.21.15.97:0/1125383481 wait complete. 2024-03-27T11:06:49.273 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.304+0000 7f63e8e00640 1 Processor -- start 2024-03-27T11:06:49.273 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.304+0000 7f63e8e00640 1 -- start start 2024-03-27T11:06:49.275 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.304+0000 7f63e8e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63e4076510 0x7f63e419e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:06:49.275 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.304+0000 7f63e8e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f63e4077750 0x7f63e419e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:06:49.276 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.304+0000 7f63e8e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f63e40780f0 0x7f63e41a58c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:06:49.276 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.304+0000 7f63e8e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f63e4113ce0 con 0x7f63e4076510 2024-03-27T11:06:49.276 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.304+0000 7f63e8e00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f63e4113b60 con 0x7f63e4077750 2024-03-27T11:06:49.276 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.304+0000 7f63e8e00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f63e4113e60 con 0x7f63e40780f0 2024-03-27T11:06:49.276 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.305+0000 7f63dbe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f63e4077750 0x7f63e419e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:06:49.276 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.305+0000 7f63dbe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f63e4077750 0x7f63e419e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:33218/0 (socket says 172.21.15.97:33218) 2024-03-27T11:06:49.276 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.305+0000 7f63dbe00640 1 -- 172.21.15.97:0/2620662603 learned_addr learned my addr 172.21.15.97:0/2620662603 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:06:49.276 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.305+0000 7f63e1e00640 1 --2- 172.21.15.97:0/2620662603 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63e4076510 0x7f63e419e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:06:49.276 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.305+0000 7f63e2800640 1 --2- 172.21.15.97:0/2620662603 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f63e40780f0 0x7f63e41a58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:06:49.276 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.305+0000 7f63dbe00640 1 -- 172.21.15.97:0/2620662603 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f63e40780f0 msgr2=0x7f63e41a58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:06:49.276 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.305+0000 7f63dbe00640 1 --2- 172.21.15.97:0/2620662603 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f63e40780f0 0x7f63e41a58c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:49.276 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.305+0000 7f63dbe00640 1 -- 172.21.15.97:0/2620662603 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63e4076510 msgr2=0x7f63e419e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:06:49.277 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.305+0000 7f63dbe00640 1 --2- 172.21.15.97:0/2620662603 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63e4076510 0x7f63e419e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:49.277 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.305+0000 7f63dbe00640 1 -- 172.21.15.97:0/2620662603 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63e41a5f30 con 0x7f63e4077750 2024-03-27T11:06:49.277 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.305+0000 7f63e2800640 1 --2- 172.21.15.97:0/2620662603 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f63e40780f0 0x7f63e41a58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T11:06:49.277 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.305+0000 7f63e1e00640 1 --2- 172.21.15.97:0/2620662603 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63e4076510 0x7f63e419e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T11:06:49.277 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.306+0000 7f63dbe00640 1 --2- 172.21.15.97:0/2620662603 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f63e4077750 0x7f63e419e850 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f63cc02c850 tx=0x7f63cc0040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:06:49.277 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.306+0000 7f63daa00640 1 -- 172.21.15.97:0/2620662603 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63cc043020 con 0x7f63e4077750 2024-03-27T11:06:49.277 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.306+0000 7f63daa00640 1 -- 172.21.15.97:0/2620662603 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f63cc0044c0 con 0x7f63e4077750 2024-03-27T11:06:49.278 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.306+0000 7f63daa00640 1 -- 172.21.15.97:0/2620662603 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63cc003c30 con 0x7f63e4077750 2024-03-27T11:06:49.278 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.306+0000 7f63e8e00640 1 -- 172.21.15.97:0/2620662603 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63e41a61c0 con 0x7f63e4077750 2024-03-27T11:06:49.278 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.307+0000 7f63e8e00640 1 -- 172.21.15.97:0/2620662603 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63e41a6690 con 0x7f63e4077750 2024-03-27T11:06:49.281 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.311+0000 7f63daa00640 1 -- 172.21.15.97:0/2620662603 <== mon.2 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f63cc046020 con 0x7f63e4077750 2024-03-27T11:06:49.282 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.312+0000 7f63ebb8b740 1 Processor -- start 2024-03-27T11:06:49.282 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.312+0000 7f63ebb8b740 1 -- start start 2024-03-27T11:06:49.283 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.312+0000 7f63ebb8b740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c3d6285980 0x55c3d6285da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:06:49.283 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.313+0000 7f63ebb8b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3d6286370 0x55c3d628ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:06:49.283 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.313+0000 7f63ebb8b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c3d628b4e0 0x55c3d628d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:06:49.283 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.313+0000 7f63ebb8b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55c3d61ead10 con 0x55c3d628b4e0 2024-03-27T11:06:49.283 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.313+0000 7f63ebb8b740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55c3d6193260 con 0x55c3d6286370 2024-03-27T11:06:49.283 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.313+0000 7f63ebb8b740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55c3d61e6cc0 con 0x55c3d6285980 2024-03-27T11:06:49.283 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.313+0000 7f63e1e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3d6286370 0x55c3d628ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:06:49.283 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.313+0000 7f63e1e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3d6286370 0x55c3d628ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:33228/0 (socket says 172.21.15.97:33228) 2024-03-27T11:06:49.284 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.313+0000 7f63e1e00640 1 -- 172.21.15.97:0/3850163187 learned_addr learned my addr 172.21.15.97:0/3850163187 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:06:49.284 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.313+0000 7f63e1e00640 1 -- 172.21.15.97:0/3850163187 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c3d6285980 msgr2=0x55c3d6285da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:06:49.284 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.313+0000 7f63e1e00640 1 --2- 172.21.15.97:0/3850163187 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c3d6285980 0x55c3d6285da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:49.284 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.313+0000 7f63e1e00640 1 -- 172.21.15.97:0/3850163187 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c3d628b4e0 msgr2=0x55c3d628d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:06:49.284 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.313+0000 7f63e1e00640 1 --2- 172.21.15.97:0/3850163187 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c3d628b4e0 0x55c3d628d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:49.285 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.313+0000 7f63e1e00640 1 -- 172.21.15.97:0/3850163187 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3d618aaa0 con 0x55c3d6286370 2024-03-27T11:06:49.285 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.314+0000 7f63e1e00640 1 --2- 172.21.15.97:0/3850163187 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3d6286370 0x55c3d628ae70 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f63d4009d60 tx=0x7f63d40117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebeacbf85842e376 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:06:49.285 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.314+0000 7f63d3400640 1 -- 172.21.15.97:0/3850163187 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d4012650 con 0x55c3d6286370 2024-03-27T11:06:49.285 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.314+0000 7f63d3400640 1 -- 172.21.15.97:0/3850163187 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f63d4009e50 con 0x55c3d6286370 2024-03-27T11:06:49.285 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.314+0000 7f63d3400640 1 -- 172.21.15.97:0/3850163187 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d400fc00 con 0x55c3d6286370 2024-03-27T11:06:49.285 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.315+0000 7f63daa00640 1 --2- 172.21.15.97:0/2620662603 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f63c80a3780 0x7f63c80a5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:06:49.286 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.316+0000 7f63ebb8b740 1 -- 172.21.15.97:0/3850163187 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3d6286370 msgr2=0x55c3d628ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:06:49.286 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.316+0000 7f63ebb8b740 1 --2- 172.21.15.97:0/3850163187 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3d6286370 0x55c3d628ae70 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f63d4009d60 tx=0x7f63d40117b0 comp rx=0 tx=0).stop 2024-03-27T11:06:49.286 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.316+0000 7f63daa00640 1 -- 172.21.15.97:0/2620662603 <== mon.2 v2:172.21.15.114:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f63cc0f1fe0 con 0x7f63e4077750 2024-03-27T11:06:49.286 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.317+0000 7f63e2800640 1 --2- 172.21.15.97:0/2620662603 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f63c80a3780 0x7f63c80a5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:06:49.287 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.317+0000 7f63ebb8b740 1 -- 172.21.15.97:0/3850163187 shutdown_connections 2024-03-27T11:06:49.287 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.317+0000 7f63ebb8b740 1 --2- 172.21.15.97:0/3850163187 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c3d628b4e0 0x55c3d628d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:49.287 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.317+0000 7f63ebb8b740 1 --2- 172.21.15.97:0/3850163187 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3d6286370 0x55c3d628ae70 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:49.287 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.317+0000 7f63ebb8b740 1 --2- 172.21.15.97:0/3850163187 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c3d6285980 0x55c3d6285da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:49.287 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.317+0000 7f63ebb8b740 1 -- 172.21.15.97:0/3850163187 >> 172.21.15.97:0/3850163187 conn(0x55c3d6280ef0 msgr2=0x55c3d62812f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:06:49.288 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.318+0000 7f63ebb8b740 1 -- 172.21.15.97:0/3850163187 shutdown_connections 2024-03-27T11:06:49.288 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.318+0000 7f63e2800640 1 --2- 172.21.15.97:0/2620662603 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f63c80a3780 0x7f63c80a5c40 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x55c3d6286bd0 tx=0x7f63dc0094f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:06:49.288 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.318+0000 7f63ebb8b740 1 -- 172.21.15.97:0/3850163187 wait complete. 2024-03-27T11:06:49.288 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.319+0000 7f63ebb8b740 1 -- start start 2024-03-27T11:06:49.289 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.319+0000 7f63ebb8b740 1 Processor -- start 2024-03-27T11:06:49.290 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.320+0000 7f63ebb8b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3d6285980 0x55c3d628fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:06:49.291 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.320+0000 7f63ebb8b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c3d6286370 0x55c3d6290280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:06:49.291 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.320+0000 7f63ebb8b740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c3d628b4e0 0x55c3d62972c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:06:49.291 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.320+0000 7f63e1e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3d6285980 0x55c3d628fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:06:49.291 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.320+0000 7f63e1e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3d6285980 0x55c3d628fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:33232/0 (socket says 172.21.15.97:33232) 2024-03-27T11:06:49.291 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.321+0000 7f63e1e00640 1 -- 172.21.15.97:0/225340668 learned_addr learned my addr 172.21.15.97:0/225340668 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:06:49.292 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.321+0000 7f63ebb8b740 1 -- 172.21.15.97:0/225340668 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55c3d628e490 con 0x55c3d6286370 2024-03-27T11:06:49.292 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.321+0000 7f63ebb8b740 1 -- 172.21.15.97:0/225340668 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55c3d61a6630 con 0x55c3d6285980 2024-03-27T11:06:49.292 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.321+0000 7f63ebb8b740 1 -- 172.21.15.97:0/225340668 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55c3d628e610 con 0x55c3d628b4e0 2024-03-27T11:06:49.292 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.321+0000 7f63ebb8b740 10 client.0 _ll_register_callbacks cb 0x55c3d61abc50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T11:06:49.292 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.321+0000 7f63dbe00640 1 --2- 172.21.15.97:0/225340668 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c3d6286370 0x55c3d6290280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:06:49.292 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.321+0000 7f63e2800640 1 --2- 172.21.15.97:0/225340668 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c3d628b4e0 0x55c3d62972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:06:49.292 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.322+0000 7f63e2800640 1 -- 172.21.15.97:0/225340668 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3d6285980 msgr2=0x55c3d628fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:06:49.292 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.322+0000 7f63e2800640 1 --2- 172.21.15.97:0/225340668 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3d6285980 0x55c3d628fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:49.292 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.322+0000 7f63e2800640 1 -- 172.21.15.97:0/225340668 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c3d6286370 msgr2=0x55c3d6290280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:06:49.292 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.322+0000 7f63e2800640 1 --2- 172.21.15.97:0/225340668 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c3d6286370 0x55c3d6290280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:49.292 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.322+0000 7f63e2800640 1 -- 172.21.15.97:0/225340668 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3d61b25f0 con 0x55c3d628b4e0 2024-03-27T11:06:49.293 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.322+0000 7f63e2800640 1 --2- 172.21.15.97:0/225340668 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c3d628b4e0 0x55c3d62972c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f63dc005fd0 tx=0x7f63dc0107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f730159c0ef8e2c8 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:06:49.294 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.322+0000 7f63c7e00640 10 client.0 ms_handle_connect on v2:172.21.15.177:3300/0 2024-03-27T11:06:49.294 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.323+0000 7f63c7e00640 1 -- 172.21.15.97:0/225340668 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63dc022020 con 0x55c3d628b4e0 2024-03-27T11:06:49.294 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.323+0000 7f63ebb8b740 1 -- 172.21.15.97:0/225340668 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63d4009e50 con 0x55c3d628b4e0 2024-03-27T11:06:49.294 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.323+0000 7f63c7e00640 1 -- 172.21.15.97:0/225340668 <== mon.1 v2:172.21.15.177:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f63dc011c50 con 0x55c3d628b4e0 2024-03-27T11:06:49.294 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.323+0000 7f63c7e00640 1 -- 172.21.15.97:0/225340668 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f63dc00ee90 con 0x55c3d628b4e0 2024-03-27T11:06:49.294 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.323+0000 7f63ebb8b740 10 client.26278 fetch_fsmap learned FSMap version 34 2024-03-27T11:06:49.294 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.323+0000 7f63ebb8b740 1 -- 172.21.15.97:0/225340668 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55c3d5eff210 con 0x55c3d628b4e0 2024-03-27T11:06:49.294 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.324+0000 7f63c7e00640 1 -- 172.21.15.97:0/225340668 <== mon.1 v2:172.21.15.177:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f63d4009e50 con 0x55c3d628b4e0 2024-03-27T11:06:49.296 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.326+0000 7f63c7e00640 1 -- 172.21.15.97:0/225340668 <== mon.1 v2:172.21.15.177:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f63dc00f040 con 0x55c3d628b4e0 2024-03-27T11:06:49.296 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.326+0000 7f63ebb8b740 10 client.26278 fetch_fsmap finished waiting for FSMap version 34 2024-03-27T11:06:49.296 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.326+0000 7f63ebb8b740 10 client.26278 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T11:06:49.296 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.326+0000 7f63ebb8b740 20 client.26278 populate_metadata read hostname 'smithi097' 2024-03-27T11:06:49.296 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.326+0000 7f63ebb8b740 1 --2- 172.21.15.97:0/225340668 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55c3d629bb80 0x55c3d629df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:06:49.296 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.326+0000 7f63ebb8b740 4 client.26278 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T11:06:49.296 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.326+0000 7f63ebb8b740 1 -- 172.21.15.97:0/225340668 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c3d61f8250 con 0x55c3d629bb80 2024-03-27T11:06:49.297 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.327+0000 7f63e1e00640 1 --2- 172.21.15.97:0/225340668 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55c3d629bb80 0x55c3d629df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:06:49.302 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.331+0000 7f63e1e00640 1 --2- 172.21.15.97:0/225340668 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55c3d629bb80 0x55c3d629df70 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84c1a949799ddf77 server_cookie=9a919fa7b2cd88a8 in_seq=0 out_seq=0 2024-03-27T11:06:49.302 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.331+0000 7f63c7e00640 10 client.26278 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T11:06:49.303 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.332+0000 7f63c7e00640 1 -- 172.21.15.97:0/225340668 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55c3d61f8250 con 0x55c3d629bb80 2024-03-27T11:06:49.303 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.333+0000 7f63c7e00640 10 client.26278 handle_command_reply: tid=0 2024-03-27T11:06:49.304 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.333+0000 7f63ebb8b740 1 client.26278 shutdown 2024-03-27T11:06:49.304 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.334+0000 7f63ebb8b740 1 -- 172.21.15.97:0/225340668 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c3d628b4e0 msgr2=0x55c3d62972c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:06:49.304 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.334+0000 7f63ebb8b740 1 --2- 172.21.15.97:0/225340668 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c3d628b4e0 0x55c3d62972c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f63dc005fd0 tx=0x7f63dc0107a0 comp rx=0 tx=0).stop 2024-03-27T11:06:49.304 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.334+0000 7f63ebb8b740 1 -- 172.21.15.97:0/225340668 shutdown_connections 2024-03-27T11:06:49.304 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.334+0000 7f63ebb8b740 1 --2- 172.21.15.97:0/225340668 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55c3d629bb80 0x55c3d629df70 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:49.304 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.334+0000 7f63ebb8b740 1 --2- 172.21.15.97:0/225340668 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c3d628b4e0 0x55c3d62972c0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:49.304 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.334+0000 7f63ebb8b740 1 --2- 172.21.15.97:0/225340668 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c3d6286370 0x55c3d6290280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:49.305 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.334+0000 7f63ebb8b740 1 --2- 172.21.15.97:0/225340668 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3d6285980 0x55c3d628fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:49.305 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.334+0000 7f63ebb8b740 1 -- 172.21.15.97:0/225340668 >> 172.21.15.97:0/225340668 conn(0x55c3d6280ef0 msgr2=0x55c3d62800f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:06:49.305 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.334+0000 7f63c7e00640 0 client.26278 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T11:06:49.305 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.334+0000 7f63ebb8b740 1 -- 172.21.15.97:0/225340668 shutdown_connections 2024-03-27T11:06:49.305 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.335+0000 7f63ebb8b740 1 -- 172.21.15.97:0/225340668 wait complete. 2024-03-27T11:06:49.306 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.335+0000 7f63ebb8b740 20 client.26278 trim_cache size 0 max 16384 2024-03-27T11:06:49.337 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.365+0000 7f63ebb8b740 1 Processor -- start 2024-03-27T11:06:49.337 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.365+0000 7f63ebb8b740 1 -- start start 2024-03-27T11:06:49.337 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.366+0000 7f63ebb8b740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c3d6285980 0x55c3d627f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:06:49.337 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.366+0000 7f63ebb8b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c3d6286370 0x55c3d62971e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:06:49.337 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.366+0000 7f63ebb8b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3d628b4e0 0x55c3d627fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:06:49.338 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.366+0000 7f63ebb8b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f63e41af470 con 0x55c3d6286370 2024-03-27T11:06:49.338 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.366+0000 7f63ebb8b740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f63e41b0fb0 con 0x55c3d628b4e0 2024-03-27T11:06:49.338 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.366+0000 7f63ebb8b740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f63e41afb40 con 0x55c3d6285980 2024-03-27T11:06:49.338 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.366+0000 7f63e2800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3d628b4e0 0x55c3d627fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:06:49.338 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.366+0000 7f63e2800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3d628b4e0 0x55c3d627fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:33244/0 (socket says 172.21.15.97:33244) 2024-03-27T11:06:49.338 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.366+0000 7f63e2800640 1 -- 172.21.15.97:0/2678633918 learned_addr learned my addr 172.21.15.97:0/2678633918 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:06:49.338 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.367+0000 7f63e2800640 1 -- 172.21.15.97:0/2678633918 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c3d6285980 msgr2=0x55c3d627f860 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-27T11:06:49.338 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.367+0000 7f63e2800640 1 --2- 172.21.15.97:0/2678633918 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c3d6285980 0x55c3d627f860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:49.338 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.367+0000 7f63e2800640 1 -- 172.21.15.97:0/2678633918 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c3d6286370 msgr2=0x55c3d62971e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:06:49.339 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.368+0000 7f63dbe00640 1 --2- 172.21.15.97:0/2678633918 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c3d6286370 0x55c3d62971e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:06:49.339 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.368+0000 7f63e2800640 1 --2- 172.21.15.97:0/2678633918 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c3d6286370 0x55c3d62971e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:49.339 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.368+0000 7f63e2800640 1 -- 172.21.15.97:0/2678633918 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3d5fcb620 con 0x55c3d628b4e0 2024-03-27T11:06:49.339 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.368+0000 7f63dbe00640 1 --2- 172.21.15.97:0/2678633918 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c3d6286370 0x55c3d62971e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T11:06:49.339 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.369+0000 7f63e2800640 1 --2- 172.21.15.97:0/2678633918 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3d628b4e0 0x55c3d627fda0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f63dc005be0 tx=0x7f63dc011c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49a1faf0ac3ced25 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:06:49.341 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.370+0000 7f63d2a00640 1 -- 172.21.15.97:0/2678633918 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63dc036070 con 0x55c3d628b4e0 2024-03-27T11:06:49.341 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.370+0000 7f63d2a00640 1 -- 172.21.15.97:0/2678633918 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f63dc017030 con 0x55c3d628b4e0 2024-03-27T11:06:49.341 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.370+0000 7f63d2a00640 1 -- 172.21.15.97:0/2678633918 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63dc005040 con 0x55c3d628b4e0 2024-03-27T11:06:49.341 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.371+0000 7f63ebb8b740 1 -- 172.21.15.97:0/2678633918 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3d628b4e0 msgr2=0x55c3d627fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:06:49.341 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.371+0000 7f63ebb8b740 1 --2- 172.21.15.97:0/2678633918 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3d628b4e0 0x55c3d627fda0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f63dc005be0 tx=0x7f63dc011c50 comp rx=0 tx=0).stop 2024-03-27T11:06:49.341 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.371+0000 7f63ebb8b740 1 -- 172.21.15.97:0/2678633918 shutdown_connections 2024-03-27T11:06:49.341 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.371+0000 7f63ebb8b740 1 --2- 172.21.15.97:0/2678633918 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3d628b4e0 0x55c3d627fda0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:49.341 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.371+0000 7f63ebb8b740 1 --2- 172.21.15.97:0/2678633918 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c3d6286370 0x55c3d62971e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:49.341 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.371+0000 7f63ebb8b740 1 --2- 172.21.15.97:0/2678633918 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c3d6285980 0x55c3d627f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:49.342 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.371+0000 7f63ebb8b740 1 -- 172.21.15.97:0/2678633918 >> 172.21.15.97:0/2678633918 conn(0x55c3d6280ef0 msgr2=0x55c3d62876d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:06:49.342 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.371+0000 7f63ebb8b740 1 -- 172.21.15.97:0/2678633918 shutdown_connections 2024-03-27T11:06:49.342 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.371+0000 7f63ebb8b740 1 -- 172.21.15.97:0/2678633918 wait complete. 2024-03-27T11:06:49.342 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.372+0000 7f63ebb8b740 1 -- start start 2024-03-27T11:06:49.344 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.372+0000 7f63ebb8b740 1 Processor -- start 2024-03-27T11:06:49.344 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.373+0000 7f63ebb8b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c3d6285980 0x55c3d6292930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:06:49.344 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.373+0000 7f63ebb8b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3d6286370 0x55c3d6292e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:06:49.344 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.373+0000 7f63ebb8b740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c3d628b4e0 0x55c3d6295e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:06:49.344 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.373+0000 7f63ebb8b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f63e41b08e0 con 0x55c3d6285980 2024-03-27T11:06:49.344 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.373+0000 7f63ebb8b740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f63e41b0210 con 0x55c3d6286370 2024-03-27T11:06:49.344 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.373+0000 7f63ebb8b740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f63e41ad450 con 0x55c3d628b4e0 2024-03-27T11:06:49.344 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.373+0000 7f63ebb8b740 10 client.0 _ll_register_callbacks cb 0x7f63dc009b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T11:06:49.344 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.373+0000 7f63e2800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c3d628b4e0 0x55c3d6295e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:06:49.344 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.373+0000 7f63e2800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c3d628b4e0 0x55c3d6295e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.97:53116/0 (socket says 172.21.15.97:53116) 2024-03-27T11:06:49.345 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.373+0000 7f63e2800640 1 -- 172.21.15.97:0/732883934 learned_addr learned my addr 172.21.15.97:0/732883934 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:06:49.345 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.373+0000 7f63dbe00640 1 --2- 172.21.15.97:0/732883934 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3d6286370 0x55c3d6292e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:06:49.345 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.374+0000 7f63e2800640 1 -- 172.21.15.97:0/732883934 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3d6286370 msgr2=0x55c3d6292e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:06:49.345 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.374+0000 7f63e2800640 1 --2- 172.21.15.97:0/732883934 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3d6286370 0x55c3d6292e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:49.345 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.374+0000 7f63e2800640 1 -- 172.21.15.97:0/732883934 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c3d6285980 msgr2=0x55c3d6292930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:06:49.345 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.374+0000 7f63e2800640 1 --2- 172.21.15.97:0/732883934 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c3d6285980 0x55c3d6292930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:49.345 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.374+0000 7f63e2800640 1 -- 172.21.15.97:0/732883934 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3d6296480 con 0x55c3d628b4e0 2024-03-27T11:06:49.345 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.374+0000 7f63dbe00640 1 --2- 172.21.15.97:0/732883934 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3d6286370 0x55c3d6292e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T11:06:49.347 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.374+0000 7f63e2800640 1 --2- 172.21.15.97:0/732883934 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c3d628b4e0 0x55c3d6295e90 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f63dc009820 tx=0x7f63dc01ebc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6472993dad8c738e server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:06:49.347 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.374+0000 7f63c7e00640 10 client.0 ms_handle_connect on v2:172.21.15.177:3300/0 2024-03-27T11:06:49.347 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.375+0000 7f63c7e00640 1 -- 172.21.15.97:0/732883934 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63dc061070 con 0x55c3d628b4e0 2024-03-27T11:06:49.347 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.375+0000 7f63ebb8b740 1 -- 172.21.15.97:0/732883934 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63dc017030 con 0x55c3d628b4e0 2024-03-27T11:06:49.347 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.375+0000 7f63c7e00640 1 -- 172.21.15.97:0/732883934 <== mon.1 v2:172.21.15.177:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f63dc0049e0 con 0x55c3d628b4e0 2024-03-27T11:06:49.347 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.376+0000 7f63c7e00640 1 -- 172.21.15.97:0/732883934 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f63dc06d050 con 0x55c3d628b4e0 2024-03-27T11:06:49.347 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.376+0000 7f63ebb8b740 10 client.26281 fetch_fsmap learned FSMap version 34 2024-03-27T11:06:49.347 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.376+0000 7f63ebb8b740 1 -- 172.21.15.97:0/732883934 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55c3d6312e90 con 0x55c3d628b4e0 2024-03-27T11:06:49.347 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.376+0000 7f63c7e00640 1 -- 172.21.15.97:0/732883934 <== mon.1 v2:172.21.15.177:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f63dc017030 con 0x55c3d628b4e0 2024-03-27T11:06:49.347 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.377+0000 7f63c7e00640 1 -- 172.21.15.97:0/732883934 <== mon.1 v2:172.21.15.177:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f63dc05dbd0 con 0x55c3d628b4e0 2024-03-27T11:06:49.347 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.377+0000 7f63ebb8b740 10 client.26281 fetch_fsmap finished waiting for FSMap version 34 2024-03-27T11:06:49.348 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.377+0000 7f63ebb8b740 10 client.26281 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T11:06:49.348 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.377+0000 7f63ebb8b740 20 client.26281 populate_metadata read hostname 'smithi097' 2024-03-27T11:06:49.348 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.377+0000 7f63ebb8b740 1 --2- 172.21.15.97:0/732883934 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55c3d6313810 0x55c3d6315c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:06:49.348 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.377+0000 7f63ebb8b740 4 client.26281 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-27T11:06:49.348 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.377+0000 7f63ebb8b740 1 -- 172.21.15.97:0/732883934 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c3d61f25a0 con 0x55c3d6313810 2024-03-27T11:06:49.349 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.377+0000 7f63e1e00640 1 --2- 172.21.15.97:0/732883934 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55c3d6313810 0x55c3d6315c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:06:49.349 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.378+0000 7f63e1e00640 1 --2- 172.21.15.97:0/732883934 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55c3d6313810 0x55c3d6315c30 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3042fd63bb07de10 server_cookie=51b170cd6a05600 in_seq=0 out_seq=0 2024-03-27T11:06:49.349 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.378+0000 7f63c7e00640 10 client.26281 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T11:06:49.353 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.384+0000 7f63c7e00640 1 -- 172.21.15.97:0/732883934 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c3d61f25a0 con 0x55c3d6313810 2024-03-27T11:06:49.354 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.384+0000 7f63c7e00640 10 client.26281 handle_command_reply: tid=0 2024-03-27T11:06:49.354 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.384+0000 7f63ebb8b740 1 client.26281 shutdown 2024-03-27T11:06:49.355 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.385+0000 7f63ebb8b740 1 -- 172.21.15.97:0/732883934 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c3d628b4e0 msgr2=0x55c3d6295e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:06:49.355 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.385+0000 7f63ebb8b740 1 --2- 172.21.15.97:0/732883934 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c3d628b4e0 0x55c3d6295e90 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f63dc009820 tx=0x7f63dc01ebc0 comp rx=0 tx=0).stop 2024-03-27T11:06:49.356 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.385+0000 7f63ebb8b740 1 -- 172.21.15.97:0/732883934 shutdown_connections 2024-03-27T11:06:49.356 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.385+0000 7f63ebb8b740 1 --2- 172.21.15.97:0/732883934 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55c3d6313810 0x55c3d6315c30 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:49.356 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.385+0000 7f63c7e00640 0 client.26281 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T11:06:49.356 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.385+0000 7f63ebb8b740 1 --2- 172.21.15.97:0/732883934 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c3d628b4e0 0x55c3d6295e90 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:49.356 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.385+0000 7f63ebb8b740 1 --2- 172.21.15.97:0/732883934 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3d6286370 0x55c3d6292e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:49.356 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.385+0000 7f63ebb8b740 1 --2- 172.21.15.97:0/732883934 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c3d6285980 0x55c3d6292930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:49.356 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.385+0000 7f63ebb8b740 1 -- 172.21.15.97:0/732883934 >> 172.21.15.97:0/732883934 conn(0x55c3d6280ef0 msgr2=0x55c3d6284150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:06:49.356 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.386+0000 7f63ebb8b740 1 -- 172.21.15.97:0/732883934 shutdown_connections 2024-03-27T11:06:49.356 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.386+0000 7f63ebb8b740 1 -- 172.21.15.97:0/732883934 wait complete. 2024-03-27T11:06:49.356 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.386+0000 7f63ebb8b740 20 client.26281 trim_cache size 0 max 16384 2024-03-27T11:06:49.357 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-27T11:06:49.357 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-27T11:06:49.357 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-27T11:06:49.357 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-27T11:06:49.358 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.387+0000 7f63e8e00640 1 -- 172.21.15.97:0/2620662603 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f63c80a3780 msgr2=0x7f63c80a5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:06:49.358 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.387+0000 7f63e8e00640 1 --2- 172.21.15.97:0/2620662603 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f63c80a3780 0x7f63c80a5c40 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x55c3d6286bd0 tx=0x7f63dc0094f0 comp rx=0 tx=0).stop 2024-03-27T11:06:49.358 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.387+0000 7f63e8e00640 1 -- 172.21.15.97:0/2620662603 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f63e4077750 msgr2=0x7f63e419e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:06:49.358 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.387+0000 7f63e8e00640 1 --2- 172.21.15.97:0/2620662603 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f63e4077750 0x7f63e419e850 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f63cc02c850 tx=0x7f63cc0040c0 comp rx=0 tx=0).stop 2024-03-27T11:06:49.358 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.387+0000 7f63e8e00640 1 -- 172.21.15.97:0/2620662603 shutdown_connections 2024-03-27T11:06:49.358 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.387+0000 7f63e8e00640 1 --2- 172.21.15.97:0/2620662603 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f63e40780f0 0x7f63e41a58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:49.358 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.387+0000 7f63e8e00640 1 --2- 172.21.15.97:0/2620662603 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f63c80a3780 0x7f63c80a5c40 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:49.358 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.387+0000 7f63e8e00640 1 --2- 172.21.15.97:0/2620662603 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f63e4077750 0x7f63e419e850 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:49.358 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.387+0000 7f63e8e00640 1 --2- 172.21.15.97:0/2620662603 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63e4076510 0x7f63e419e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:06:49.358 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.387+0000 7f63e8e00640 1 -- 172.21.15.97:0/2620662603 >> 172.21.15.97:0/2620662603 conn(0x7f63e40ff310 msgr2=0x7f63e4100e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:06:49.358 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.387+0000 7f63e8e00640 1 -- 172.21.15.97:0/2620662603 shutdown_connections 2024-03-27T11:06:49.359 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:06:49.388+0000 7f63e8e00640 1 -- 172.21.15.97:0/2620662603 wait complete. 2024-03-27T11:06:49.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:06:49 smithi177 ceph-mon[35294]: pgmap v746: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 49 MiB/s wr, 836 op/s 2024-03-27T11:06:49.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:06:49 smithi114 ceph-mon[35338]: pgmap v746: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 49 MiB/s wr, 836 op/s 2024-03-27T11:06:49.888 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:06:49 smithi097 ceph-mon[31859]: pgmap v746: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 49 MiB/s wr, 836 op/s 2024-03-27T11:06:50.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:06:50 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T11:06:50.765 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:06:50 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T11:06:50.888 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:06:50 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T11:06:51.924 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:06:51 smithi177 ceph-mon[35294]: pgmap v747: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 53 MiB/s wr, 902 op/s 2024-03-27T11:06:51.924 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:06:51 smithi177 ceph-mon[35294]: from='mgr.14152 ' entity='mgr.x' 2024-03-27T11:06:51.924 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:06:51 smithi177 ceph-mon[35294]: from='mgr.14152 ' entity='mgr.x' 2024-03-27T11:06:51.924 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:06:51 smithi177 ceph-mon[35294]: from='mgr.14152 ' entity='mgr.x' 2024-03-27T11:06:51.924 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:06:51 smithi177 ceph-mon[35294]: from='mgr.14152 ' entity='mgr.x' 2024-03-27T11:06:52.014 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:06:51 smithi114 ceph-mon[35338]: pgmap v747: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 53 MiB/s wr, 902 op/s 2024-03-27T11:06:52.014 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:06:51 smithi114 ceph-mon[35338]: from='mgr.14152 ' entity='mgr.x' 2024-03-27T11:06:52.014 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:06:51 smithi114 ceph-mon[35338]: from='mgr.14152 ' entity='mgr.x' 2024-03-27T11:06:52.014 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:06:51 smithi114 ceph-mon[35338]: from='mgr.14152 ' entity='mgr.x' 2024-03-27T11:06:52.014 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:06:51 smithi114 ceph-mon[35338]: from='mgr.14152 ' entity='mgr.x' 2024-03-27T11:06:52.139 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:06:51 smithi097 ceph-mon[31859]: pgmap v747: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 53 MiB/s wr, 902 op/s 2024-03-27T11:06:52.139 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:06:51 smithi097 ceph-mon[31859]: from='mgr.14152 ' entity='mgr.x' 2024-03-27T11:06:52.139 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:06:51 smithi097 ceph-mon[31859]: from='mgr.14152 ' entity='mgr.x' 2024-03-27T11:06:52.139 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:06:51 smithi097 ceph-mon[31859]: from='mgr.14152 ' entity='mgr.x' 2024-03-27T11:06:52.139 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:06:51 smithi097 ceph-mon[31859]: from='mgr.14152 ' entity='mgr.x' 2024-03-27T11:06:53.889 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:06:53 smithi097 ceph-mon[31859]: pgmap v748: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 47 MiB/s wr, 803 op/s 2024-03-27T11:06:53.923 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:06:53 smithi177 ceph-mon[35294]: pgmap v748: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 47 MiB/s wr, 803 op/s 2024-03-27T11:06:54.015 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:06:53 smithi114 ceph-mon[35338]: pgmap v748: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 47 MiB/s wr, 803 op/s 2024-03-27T11:06:55.264 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:06:55 smithi114 ceph-mon[35338]: from='mgr.14152 ' entity='mgr.x' 2024-03-27T11:06:55.265 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:06:55 smithi114 ceph-mon[35338]: from='mgr.14152 ' entity='mgr.x' 2024-03-27T11:06:55.265 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:06:55 smithi114 ceph-mon[35338]: from='mgr.14152 ' entity='mgr.x' 2024-03-27T11:06:55.265 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:06:55 smithi114 ceph-mon[35338]: from='mgr.14152 ' entity='mgr.x' 2024-03-27T11:06:55.388 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:06:54 smithi097 ceph-mon[31859]: from='mgr.14152 ' entity='mgr.x' 2024-03-27T11:06:55.388 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:06:54 smithi097 ceph-mon[31859]: from='mgr.14152 ' entity='mgr.x' 2024-03-27T11:06:55.388 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:06:54 smithi097 ceph-mon[31859]: from='mgr.14152 ' entity='mgr.x' 2024-03-27T11:06:55.389 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:06:54 smithi097 ceph-mon[31859]: from='mgr.14152 ' entity='mgr.x' 2024-03-27T11:06:55.424 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:06:55 smithi177 ceph-mon[35294]: from='mgr.14152 ' entity='mgr.x' 2024-03-27T11:06:55.424 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:06:55 smithi177 ceph-mon[35294]: from='mgr.14152 ' entity='mgr.x' 2024-03-27T11:06:55.424 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:06:55 smithi177 ceph-mon[35294]: from='mgr.14152 ' entity='mgr.x' 2024-03-27T11:06:55.424 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:06:55 smithi177 ceph-mon[35294]: from='mgr.14152 ' entity='mgr.x' 2024-03-27T11:06:56.174 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:06:56 smithi177 ceph-mon[35294]: pgmap v749: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 47 MiB/s wr, 807 op/s 2024-03-27T11:06:56.174 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:06:56 smithi177 ceph-mon[35294]: from='mgr.14152 ' entity='mgr.x' 2024-03-27T11:06:56.174 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:06:56 smithi177 ceph-mon[35294]: from='mgr.14152 ' entity='mgr.x' 2024-03-27T11:06:56.264 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:06:56 smithi114 ceph-mon[35338]: pgmap v749: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 47 MiB/s wr, 807 op/s 2024-03-27T11:06:56.264 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:06:56 smithi114 ceph-mon[35338]: from='mgr.14152 ' entity='mgr.x' 2024-03-27T11:06:56.264 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:06:56 smithi114 ceph-mon[35338]: from='mgr.14152 ' entity='mgr.x' 2024-03-27T11:06:56.389 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:06:56 smithi097 ceph-mon[31859]: pgmap v749: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 47 MiB/s wr, 807 op/s 2024-03-27T11:06:56.389 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:06:56 smithi097 ceph-mon[31859]: from='mgr.14152 ' entity='mgr.x' 2024-03-27T11:06:56.389 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:06:56 smithi097 ceph-mon[31859]: from='mgr.14152 ' entity='mgr.x' 2024-03-27T11:06:58.265 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:06:58 smithi114 ceph-mon[35338]: pgmap v750: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 45 MiB/s wr, 769 op/s 2024-03-27T11:06:58.265 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:06:58 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T11:06:58.265 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:06:58 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T11:06:58.265 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:06:58 smithi114 ceph-mon[35338]: from='mgr.14152 ' entity='mgr.x' 2024-03-27T11:06:58.389 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:06:58 smithi097 ceph-mon[31859]: pgmap v750: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 45 MiB/s wr, 769 op/s 2024-03-27T11:06:58.389 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:06:58 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T11:06:58.390 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:06:58 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T11:06:58.390 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:06:58 smithi097 ceph-mon[31859]: from='mgr.14152 ' entity='mgr.x' 2024-03-27T11:06:58.424 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:06:58 smithi177 ceph-mon[35294]: pgmap v750: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 45 MiB/s wr, 769 op/s 2024-03-27T11:06:58.424 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:06:58 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T11:06:58.424 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:06:58 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T11:06:58.424 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:06:58 smithi177 ceph-mon[35294]: from='mgr.14152 ' entity='mgr.x' 2024-03-27T11:06:59.264 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:06:59 smithi114 ceph-mon[35338]: pgmap v751: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 46 MiB/s wr, 793 op/s 2024-03-27T11:06:59.389 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:06:59 smithi097 ceph-mon[31859]: pgmap v751: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 46 MiB/s wr, 793 op/s 2024-03-27T11:06:59.424 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:06:59 smithi177 ceph-mon[35294]: pgmap v751: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 46 MiB/s wr, 793 op/s 2024-03-27T11:07:01.638 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:07:01 smithi097 ceph-mon[31859]: pgmap v752: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 49 MiB/s wr, 843 op/s 2024-03-27T11:07:01.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:07:01 smithi177 ceph-mon[35294]: pgmap v752: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 49 MiB/s wr, 843 op/s 2024-03-27T11:07:01.764 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:07:01 smithi114 ceph-mon[35338]: pgmap v752: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 49 MiB/s wr, 843 op/s 2024-03-27T11:07:02.593 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.623+0000 7f6a9dc00640 1 -- 172.21.15.97:0/1202052213 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6a98077750 msgr2=0x7f6a98077bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:07:02.593 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.623+0000 7f6a9dc00640 1 --2- 172.21.15.97:0/1202052213 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6a98077750 0x7f6a98077bb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6a84009860 tx=0x7f6a8402bb70 comp rx=0 tx=0).stop 2024-03-27T11:07:02.593 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.623+0000 7f6a9dc00640 1 -- 172.21.15.97:0/1202052213 shutdown_connections 2024-03-27T11:07:02.593 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.623+0000 7f6a9dc00640 1 --2- 172.21.15.97:0/1202052213 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a980780f0 0x7f6a98078990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:02.594 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.623+0000 7f6a9dc00640 1 --2- 172.21.15.97:0/1202052213 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6a98077750 0x7f6a98077bb0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:02.594 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.623+0000 7f6a9dc00640 1 --2- 172.21.15.97:0/1202052213 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6a98076510 0x7f6a98076930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:02.594 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.623+0000 7f6a9dc00640 1 -- 172.21.15.97:0/1202052213 >> 172.21.15.97:0/1202052213 conn(0x7f6a980ff160 msgr2=0x7f6a98101580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:07:02.594 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.623+0000 7f6a9dc00640 1 -- 172.21.15.97:0/1202052213 shutdown_connections 2024-03-27T11:07:02.594 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.623+0000 7f6a9dc00640 1 -- 172.21.15.97:0/1202052213 wait complete. 2024-03-27T11:07:02.594 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.624+0000 7f6a9dc00640 1 Processor -- start 2024-03-27T11:07:02.594 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.624+0000 7f6a9dc00640 1 -- start start 2024-03-27T11:07:02.595 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.625+0000 7f6a9dc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a98076510 0x7f6a9819e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:07:02.595 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.625+0000 7f6a9dc00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6a98077750 0x7f6a9819e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:07:02.595 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.625+0000 7f6a9dc00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6a980780f0 0x7f6a981a56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:07:02.595 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.625+0000 7f6a9dc00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6a98113ad0 con 0x7f6a98076510 2024-03-27T11:07:02.595 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.625+0000 7f6a9dc00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f6a98113950 con 0x7f6a980780f0 2024-03-27T11:07:02.595 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.625+0000 7f6a9dc00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6a98113c50 con 0x7f6a98077750 2024-03-27T11:07:02.596 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.625+0000 7f6a96000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6a98077750 0x7f6a9819e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:07:02.596 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.625+0000 7f6a96000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6a98077750 0x7f6a9819e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.97:38356/0 (socket says 172.21.15.97:38356) 2024-03-27T11:07:02.596 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.625+0000 7f6a96000640 1 -- 172.21.15.97:0/259079296 learned_addr learned my addr 172.21.15.97:0/259079296 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:07:02.596 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.625+0000 7f6a97e00640 1 --2- 172.21.15.97:0/259079296 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6a980780f0 0x7f6a981a56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:07:02.596 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.626+0000 7f6a96a00640 1 --2- 172.21.15.97:0/259079296 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a98076510 0x7f6a9819e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:07:02.596 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.626+0000 7f6a97e00640 1 -- 172.21.15.97:0/259079296 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6a98077750 msgr2=0x7f6a9819e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:07:02.596 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.626+0000 7f6a97e00640 1 --2- 172.21.15.97:0/259079296 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6a98077750 0x7f6a9819e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:02.597 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.626+0000 7f6a97e00640 1 -- 172.21.15.97:0/259079296 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a98076510 msgr2=0x7f6a9819e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:07:02.597 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.626+0000 7f6a97e00640 1 --2- 172.21.15.97:0/259079296 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a98076510 0x7f6a9819e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:02.597 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.626+0000 7f6a97e00640 1 -- 172.21.15.97:0/259079296 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a981a5d60 con 0x7f6a980780f0 2024-03-27T11:07:02.597 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.626+0000 7f6a96a00640 1 --2- 172.21.15.97:0/259079296 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a98076510 0x7f6a9819e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T11:07:02.597 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.627+0000 7f6a97e00640 1 --2- 172.21.15.97:0/259079296 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6a980780f0 0x7f6a981a56f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6a90001440 tx=0x7f6a9000ed80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:07:02.597 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.627+0000 7f6a8f400640 1 -- 172.21.15.97:0/259079296 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a9000cd10 con 0x7f6a980780f0 2024-03-27T11:07:02.598 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.627+0000 7f6a8f400640 1 -- 172.21.15.97:0/259079296 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6a90013070 con 0x7f6a980780f0 2024-03-27T11:07:02.598 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.627+0000 7f6a8f400640 1 -- 172.21.15.97:0/259079296 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a9000f670 con 0x7f6a980780f0 2024-03-27T11:07:02.598 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.627+0000 7f6a9dc00640 1 -- 172.21.15.97:0/259079296 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a981a5ff0 con 0x7f6a980780f0 2024-03-27T11:07:02.598 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.627+0000 7f6a9dc00640 1 -- 172.21.15.97:0/259079296 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a981a6510 con 0x7f6a980780f0 2024-03-27T11:07:02.602 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.633+0000 7f6a8f400640 1 -- 172.21.15.97:0/259079296 <== mon.2 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f6a90015030 con 0x7f6a980780f0 2024-03-27T11:07:02.604 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.633+0000 7f6aa07eb740 1 Processor -- start 2024-03-27T11:07:02.604 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.634+0000 7f6aa07eb740 1 -- start start 2024-03-27T11:07:02.604 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.634+0000 7f6aa07eb740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556955669b70 0x556955669f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:07:02.604 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.634+0000 7f6aa07eb740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55695566a560 0x55695566f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:07:02.605 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.634+0000 7f6aa07eb740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55695566f6d0 0x556955671b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:07:02.606 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.635+0000 7f6a8f400640 1 --2- 172.21.15.97:0/259079296 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f6a7c0a3730 0x7f6a7c0a5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:07:02.606 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.635+0000 7f6aa07eb740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5569555bdd10 con 0x55695566a560 2024-03-27T11:07:02.606 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.635+0000 7f6aa07eb740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x556955566260 con 0x55695566f6d0 2024-03-27T11:07:02.606 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.635+0000 7f6aa07eb740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5569555b9cc0 con 0x556955669b70 2024-03-27T11:07:02.606 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.635+0000 7f6a8f400640 1 -- 172.21.15.97:0/259079296 <== mon.2 v2:172.21.15.114:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f6a900cf5e0 con 0x7f6a980780f0 2024-03-27T11:07:02.606 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.636+0000 7f6a96000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556955669b70 0x556955669f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:07:02.606 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.636+0000 7f6a96000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55695566f6d0 0x556955671b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:07:02.606 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.636+0000 7f6a96000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556955669b70 0x556955669f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.97:38372/0 (socket says 172.21.15.97:38372) 2024-03-27T11:07:02.606 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.636+0000 7f6a96000640 1 -- 172.21.15.97:0/392543971 learned_addr learned my addr 172.21.15.97:0/392543971 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:07:02.607 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.636+0000 7f6a97e00640 1 --2- 172.21.15.97:0/259079296 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f6a7c0a3730 0x7f6a7c0a5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:07:02.607 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.636+0000 7f6a96000640 1 -- 172.21.15.97:0/392543971 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55695566f6d0 msgr2=0x556955671b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:07:02.607 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.636+0000 7f6a96000640 1 --2- 172.21.15.97:0/392543971 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55695566f6d0 0x556955671b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:02.607 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.637+0000 7f6a96000640 1 -- 172.21.15.97:0/392543971 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55695566a560 msgr2=0x55695566f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:07:02.607 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.637+0000 7f6a96000640 1 --2- 172.21.15.97:0/392543971 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55695566a560 0x55695566f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:02.607 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.637+0000 7f6a96000640 1 -- 172.21.15.97:0/392543971 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55695555daa0 con 0x556955669b70 2024-03-27T11:07:02.607 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.637+0000 7f6a96000640 1 --2- 172.21.15.97:0/392543971 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556955669b70 0x556955669f90 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6a840057c0 tx=0x7f6a84034980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41c0adcd9aa3aff2 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:07:02.607 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.637+0000 7f6a97e00640 1 --2- 172.21.15.97:0/259079296 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f6a7c0a3730 0x7f6a7c0a5bf0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f6a90098490 tx=0x7f6a90000b60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:07:02.608 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.637+0000 7f6a83e00640 1 -- 172.21.15.97:0/392543971 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a84039040 con 0x556955669b70 2024-03-27T11:07:02.608 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.637+0000 7f6a83e00640 1 -- 172.21.15.97:0/392543971 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6a840054d0 con 0x556955669b70 2024-03-27T11:07:02.608 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.637+0000 7f6a83e00640 1 -- 172.21.15.97:0/392543971 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a8402c7e0 con 0x556955669b70 2024-03-27T11:07:02.608 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.638+0000 7f6aa07eb740 1 -- 172.21.15.97:0/392543971 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556955669b70 msgr2=0x556955669f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:07:02.608 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.638+0000 7f6aa07eb740 1 --2- 172.21.15.97:0/392543971 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556955669b70 0x556955669f90 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6a840057c0 tx=0x7f6a84034980 comp rx=0 tx=0).stop 2024-03-27T11:07:02.608 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.638+0000 7f6aa07eb740 1 -- 172.21.15.97:0/392543971 shutdown_connections 2024-03-27T11:07:02.608 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.638+0000 7f6aa07eb740 1 --2- 172.21.15.97:0/392543971 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55695566f6d0 0x556955671b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:02.609 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.638+0000 7f6aa07eb740 1 --2- 172.21.15.97:0/392543971 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55695566a560 0x55695566f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:02.609 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.638+0000 7f6aa07eb740 1 --2- 172.21.15.97:0/392543971 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556955669b70 0x556955669f90 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:02.609 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.638+0000 7f6aa07eb740 1 -- 172.21.15.97:0/392543971 >> 172.21.15.97:0/392543971 conn(0x5569556540d0 msgr2=0x5569556544d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:07:02.609 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.638+0000 7f6aa07eb740 1 -- 172.21.15.97:0/392543971 shutdown_connections 2024-03-27T11:07:02.609 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.638+0000 7f6aa07eb740 1 -- 172.21.15.97:0/392543971 wait complete. 2024-03-27T11:07:02.609 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.639+0000 7f6aa07eb740 1 -- start start 2024-03-27T11:07:02.609 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.640+0000 7f6aa07eb740 1 Processor -- start 2024-03-27T11:07:02.610 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.640+0000 7f6aa07eb740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556955669b70 0x556955673f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:07:02.610 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.640+0000 7f6aa07eb740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55695566a560 0x5569556744b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:07:02.610 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.640+0000 7f6aa07eb740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55695566f6d0 0x55695567b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:07:02.610 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.640+0000 7f6aa07eb740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x556955672620 con 0x556955669b70 2024-03-27T11:07:02.611 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.640+0000 7f6aa07eb740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x556955579630 con 0x55695566a560 2024-03-27T11:07:02.611 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.640+0000 7f6aa07eb740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5569556727a0 con 0x55695566f6d0 2024-03-27T11:07:02.611 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.640+0000 7f6aa07eb740 10 client.0 _ll_register_callbacks cb 0x55695557ec50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T11:07:02.612 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.641+0000 7f6a96a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55695566a560 0x5569556744b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:07:02.612 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.641+0000 7f6a96000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556955669b70 0x556955673f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:07:02.612 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.641+0000 7f6a96000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556955669b70 0x556955673f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38400/0 (socket says 172.21.15.97:38400) 2024-03-27T11:07:02.612 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.641+0000 7f6a96a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55695566a560 0x5569556744b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:59942/0 (socket says 172.21.15.97:59942) 2024-03-27T11:07:02.612 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.641+0000 7f6a96000640 1 -- 172.21.15.97:0/894119457 learned_addr learned my addr 172.21.15.97:0/894119457 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:07:02.612 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.641+0000 7f6a96000640 1 --2- 172.21.15.97:0/894119457 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55695566f6d0 0x55695567b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:07:02.612 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.642+0000 7f6a96a00640 1 -- 172.21.15.97:0/894119457 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55695566f6d0 msgr2=0x55695567b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:07:02.612 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.642+0000 7f6a96a00640 1 --2- 172.21.15.97:0/894119457 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55695566f6d0 0x55695567b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:02.612 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.642+0000 7f6a96a00640 1 -- 172.21.15.97:0/894119457 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556955669b70 msgr2=0x556955673f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:07:02.612 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.642+0000 7f6a96a00640 1 --2- 172.21.15.97:0/894119457 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556955669b70 0x556955673f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:02.612 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.642+0000 7f6a96a00640 1 -- 172.21.15.97:0/894119457 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569555855f0 con 0x55695566a560 2024-03-27T11:07:02.612 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.642+0000 7f6a96000640 1 --2- 172.21.15.97:0/894119457 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556955669b70 0x556955673f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T11:07:02.613 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.642+0000 7f6a96a00640 1 --2- 172.21.15.97:0/894119457 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55695566a560 0x5569556744b0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x55695566a960 tx=0x7f6a88009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1b2bb43d380b11f server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:07:02.614 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.642+0000 7f6a80c00640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-03-27T11:07:02.614 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.643+0000 7f6a80c00640 1 -- 172.21.15.97:0/894119457 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a88016070 con 0x55695566a560 2024-03-27T11:07:02.614 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.643+0000 7f6a80c00640 1 -- 172.21.15.97:0/894119457 <== mon.2 v2:172.21.15.114:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f6a88011880 con 0x55695566a560 2024-03-27T11:07:02.614 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.643+0000 7f6aa07eb740 1 -- 172.21.15.97:0/894119457 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a840054d0 con 0x55695566a560 2024-03-27T11:07:02.614 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.644+0000 7f6a80c00640 1 -- 172.21.15.97:0/894119457 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f6a88020050 con 0x55695566a560 2024-03-27T11:07:02.614 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.644+0000 7f6aa07eb740 10 client.36575 fetch_fsmap learned FSMap version 34 2024-03-27T11:07:02.614 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.644+0000 7f6aa07eb740 1 -- 172.21.15.97:0/894119457 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5569552d2210 con 0x55695566a560 2024-03-27T11:07:02.615 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.644+0000 7f6a80c00640 1 -- 172.21.15.97:0/894119457 <== mon.2 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6a840054d0 con 0x55695566a560 2024-03-27T11:07:02.615 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.644+0000 7f6a80c00640 1 -- 172.21.15.97:0/894119457 <== mon.2 v2:172.21.15.114:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f6a8800f040 con 0x55695566a560 2024-03-27T11:07:02.615 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.645+0000 7f6aa07eb740 10 client.36575 fetch_fsmap finished waiting for FSMap version 34 2024-03-27T11:07:02.615 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.645+0000 7f6aa07eb740 10 client.36575 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T11:07:02.615 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.645+0000 7f6aa07eb740 20 client.36575 populate_metadata read hostname 'smithi097' 2024-03-27T11:07:02.615 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.645+0000 7f6aa07eb740 1 --2- 172.21.15.97:0/894119457 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55695567fee0 0x5569556822f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:07:02.615 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.645+0000 7f6aa07eb740 4 client.36575 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T11:07:02.616 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.645+0000 7f6aa07eb740 1 -- 172.21.15.97:0/894119457 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5569555cb250 con 0x55695567fee0 2024-03-27T11:07:02.616 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.645+0000 7f6a96000640 1 --2- 172.21.15.97:0/894119457 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55695567fee0 0x5569556822f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:07:02.616 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.646+0000 7f6a96000640 1 --2- 172.21.15.97:0/894119457 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55695567fee0 0x5569556822f0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85cd2888290da41a server_cookie=2599eaeda1b61b05 in_seq=0 out_seq=0 2024-03-27T11:07:02.616 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.646+0000 7f6a80c00640 10 client.36575 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T11:07:02.617 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.647+0000 7f6a80c00640 1 -- 172.21.15.97:0/894119457 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5569555cb250 con 0x55695567fee0 2024-03-27T11:07:02.617 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.647+0000 7f6a80c00640 10 client.36575 handle_command_reply: tid=0 2024-03-27T11:07:02.617 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.647+0000 7f6aa07eb740 1 client.36575 shutdown 2024-03-27T11:07:02.618 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.647+0000 7f6aa07eb740 1 -- 172.21.15.97:0/894119457 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55695566a560 msgr2=0x5569556744b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:07:02.618 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.647+0000 7f6aa07eb740 1 --2- 172.21.15.97:0/894119457 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55695566a560 0x5569556744b0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x55695566a960 tx=0x7f6a88009f90 comp rx=0 tx=0).stop 2024-03-27T11:07:02.618 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.648+0000 7f6aa07eb740 1 -- 172.21.15.97:0/894119457 shutdown_connections 2024-03-27T11:07:02.618 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.648+0000 7f6aa07eb740 1 --2- 172.21.15.97:0/894119457 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55695567fee0 0x5569556822f0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:02.618 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.648+0000 7f6aa07eb740 1 --2- 172.21.15.97:0/894119457 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55695566f6d0 0x55695567b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:02.618 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.648+0000 7f6aa07eb740 1 --2- 172.21.15.97:0/894119457 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55695566a560 0x5569556744b0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:02.618 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.648+0000 7f6aa07eb740 1 --2- 172.21.15.97:0/894119457 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556955669b70 0x556955673f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:02.618 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.648+0000 7f6aa07eb740 1 -- 172.21.15.97:0/894119457 >> 172.21.15.97:0/894119457 conn(0x5569556540d0 msgr2=0x556955653310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:07:02.618 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.648+0000 7f6a80c00640 0 client.36575 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T11:07:02.618 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.648+0000 7f6aa07eb740 1 -- 172.21.15.97:0/894119457 shutdown_connections 2024-03-27T11:07:02.618 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.648+0000 7f6aa07eb740 1 -- 172.21.15.97:0/894119457 wait complete. 2024-03-27T11:07:02.618 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.648+0000 7f6aa07eb740 20 client.36575 trim_cache size 0 max 16384 2024-03-27T11:07:02.655 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.685+0000 7f6aa07eb740 1 Processor -- start 2024-03-27T11:07:02.656 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.685+0000 7f6aa07eb740 1 -- start start 2024-03-27T11:07:02.656 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.686+0000 7f6aa07eb740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556955669b70 0x55695566b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:07:02.656 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.686+0000 7f6aa07eb740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55695566f6d0 0x55695566be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:07:02.656 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.686+0000 7f6aa07eb740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556955674680 0x55695567b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:07:02.656 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.686+0000 7f6aa07eb740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6a7800e420 con 0x556955669b70 2024-03-27T11:07:02.657 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.686+0000 7f6aa07eb740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f6a7800ff60 con 0x556955674680 2024-03-27T11:07:02.657 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.686+0000 7f6a96a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55695566f6d0 0x55695566be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:07:02.657 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.686+0000 7f6aa07eb740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6a7800eaf0 con 0x55695566f6d0 2024-03-27T11:07:02.657 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.686+0000 7f6a96a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55695566f6d0 0x55695566be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.97:38388/0 (socket says 172.21.15.97:38388) 2024-03-27T11:07:02.657 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.687+0000 7f6a96a00640 1 -- 172.21.15.97:0/1785160070 learned_addr learned my addr 172.21.15.97:0/1785160070 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:07:02.657 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.687+0000 7f6a96a00640 1 -- 172.21.15.97:0/1785160070 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556955674680 msgr2=0x55695567b4b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-27T11:07:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.687+0000 7f6a96a00640 1 --2- 172.21.15.97:0/1785160070 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556955674680 0x55695567b4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.687+0000 7f6a96a00640 1 -- 172.21.15.97:0/1785160070 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556955669b70 msgr2=0x55695566b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:07:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.687+0000 7f6a96000640 1 --2- 172.21.15.97:0/1785160070 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556955669b70 0x55695566b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:07:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.688+0000 7f6a96a00640 1 --2- 172.21.15.97:0/1785160070 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556955669b70 0x55695566b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.688+0000 7f6a96a00640 1 -- 172.21.15.97:0/1785160070 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55695539e620 con 0x55695566f6d0 2024-03-27T11:07:02.659 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.689+0000 7f6a96000640 1 --2- 172.21.15.97:0/1785160070 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556955669b70 0x55695566b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T11:07:02.659 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.689+0000 7f6a96a00640 1 --2- 172.21.15.97:0/1785160070 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55695566f6d0 0x55695566be60 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6a8801bcb0 tx=0x7f6a880105e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71b63d4fa00f245d server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:07:02.659 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.689+0000 7f6a83400640 1 -- 172.21.15.97:0/1785160070 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a88016070 con 0x55695566f6d0 2024-03-27T11:07:02.659 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.689+0000 7f6a83400640 1 -- 172.21.15.97:0/1785160070 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6a88005520 con 0x55695566f6d0 2024-03-27T11:07:02.659 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.689+0000 7f6a83400640 1 -- 172.21.15.97:0/1785160070 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a8801b840 con 0x55695566f6d0 2024-03-27T11:07:02.660 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.690+0000 7f6aa07eb740 1 -- 172.21.15.97:0/1785160070 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55695566f6d0 msgr2=0x55695566be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:07:02.660 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.690+0000 7f6aa07eb740 1 --2- 172.21.15.97:0/1785160070 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55695566f6d0 0x55695566be60 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6a8801bcb0 tx=0x7f6a880105e0 comp rx=0 tx=0).stop 2024-03-27T11:07:02.660 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.690+0000 7f6aa07eb740 1 -- 172.21.15.97:0/1785160070 shutdown_connections 2024-03-27T11:07:02.660 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.690+0000 7f6aa07eb740 1 --2- 172.21.15.97:0/1785160070 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556955674680 0x55695567b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:02.660 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.690+0000 7f6aa07eb740 1 --2- 172.21.15.97:0/1785160070 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55695566f6d0 0x55695566be60 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:02.660 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.690+0000 7f6aa07eb740 1 --2- 172.21.15.97:0/1785160070 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556955669b70 0x55695566b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:02.660 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.690+0000 7f6aa07eb740 1 -- 172.21.15.97:0/1785160070 >> 172.21.15.97:0/1785160070 conn(0x5569556540d0 msgr2=0x556955673f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:07:02.661 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.691+0000 7f6aa07eb740 1 -- 172.21.15.97:0/1785160070 shutdown_connections 2024-03-27T11:07:02.661 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.691+0000 7f6aa07eb740 1 -- 172.21.15.97:0/1785160070 wait complete. 2024-03-27T11:07:02.661 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.691+0000 7f6aa07eb740 1 -- start start 2024-03-27T11:07:02.661 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.692+0000 7f6aa07eb740 1 Processor -- start 2024-03-27T11:07:02.663 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.692+0000 7f6aa07eb740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556955669b70 0x556955678f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:07:02.663 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.693+0000 7f6aa07eb740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55695566f6d0 0x5569556794c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:07:02.663 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.693+0000 7f6aa07eb740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556955674680 0x55695567a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:07:02.663 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.693+0000 7f6aa07eb740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6a7800f890 con 0x556955674680 2024-03-27T11:07:02.663 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.693+0000 7f6aa07eb740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f6a7800f1c0 con 0x556955669b70 2024-03-27T11:07:02.663 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.693+0000 7f6aa07eb740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6a7800c400 con 0x55695566f6d0 2024-03-27T11:07:02.663 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.693+0000 7f6aa07eb740 10 client.0 _ll_register_callbacks cb 0x7f6a880092a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T11:07:02.663 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.693+0000 7f6a96a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55695566f6d0 0x5569556794c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:07:02.664 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.693+0000 7f6a96000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556955674680 0x55695567a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:07:02.664 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.693+0000 7f6a96000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556955674680 0x55695567a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38426/0 (socket says 172.21.15.97:38426) 2024-03-27T11:07:02.664 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.693+0000 7f6a96000640 1 -- 172.21.15.97:0/3425756344 learned_addr learned my addr 172.21.15.97:0/3425756344 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:07:02.665 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.694+0000 7f6a96000640 1 --2- 172.21.15.97:0/3425756344 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556955669b70 0x556955678f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:07:02.665 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.694+0000 7f6a96a00640 1 -- 172.21.15.97:0/3425756344 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556955669b70 msgr2=0x556955678f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:07:02.666 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.694+0000 7f6a96a00640 1 --2- 172.21.15.97:0/3425756344 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556955669b70 0x556955678f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:02.666 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.694+0000 7f6a96a00640 1 -- 172.21.15.97:0/3425756344 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556955674680 msgr2=0x55695567a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:07:02.666 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.694+0000 7f6a96a00640 1 --2- 172.21.15.97:0/3425756344 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556955674680 0x55695567a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:02.666 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.694+0000 7f6a96a00640 1 -- 172.21.15.97:0/3425756344 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55695567aac0 con 0x55695566f6d0 2024-03-27T11:07:02.666 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.694+0000 7f6a96000640 1 --2- 172.21.15.97:0/3425756344 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556955674680 0x55695567a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T11:07:02.666 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.695+0000 7f6a96a00640 1 --2- 172.21.15.97:0/3425756344 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55695566f6d0 0x5569556794c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6a88004630 tx=0x7f6a8801bba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=769b060b9cea6a6 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:07:02.666 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.695+0000 7f6a80c00640 10 client.0 ms_handle_connect on v2:172.21.15.177:3300/0 2024-03-27T11:07:02.666 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.695+0000 7f6a80c00640 1 -- 172.21.15.97:0/3425756344 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a88032070 con 0x55695566f6d0 2024-03-27T11:07:02.666 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.695+0000 7f6a80c00640 1 -- 172.21.15.97:0/3425756344 <== mon.1 v2:172.21.15.177:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f6a88011a90 con 0x55695566f6d0 2024-03-27T11:07:02.666 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.695+0000 7f6aa07eb740 1 -- 172.21.15.97:0/3425756344 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a88005520 con 0x55695566f6d0 2024-03-27T11:07:02.666 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.696+0000 7f6a80c00640 1 -- 172.21.15.97:0/3425756344 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f6a8803e050 con 0x55695566f6d0 2024-03-27T11:07:02.666 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.696+0000 7f6aa07eb740 10 client.26299 fetch_fsmap learned FSMap version 34 2024-03-27T11:07:02.667 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.696+0000 7f6aa07eb740 1 -- 172.21.15.97:0/3425756344 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5569556e64e0 con 0x55695566f6d0 2024-03-27T11:07:02.667 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.697+0000 7f6a80c00640 1 -- 172.21.15.97:0/3425756344 <== mon.1 v2:172.21.15.177:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6a88005520 con 0x55695566f6d0 2024-03-27T11:07:02.668 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.697+0000 7f6a80c00640 1 -- 172.21.15.97:0/3425756344 <== mon.1 v2:172.21.15.177:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f6a88025440 con 0x55695566f6d0 2024-03-27T11:07:02.668 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.697+0000 7f6aa07eb740 10 client.26299 fetch_fsmap finished waiting for FSMap version 34 2024-03-27T11:07:02.668 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.697+0000 7f6aa07eb740 10 client.26299 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T11:07:02.668 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.697+0000 7f6aa07eb740 20 client.26299 populate_metadata read hostname 'smithi097' 2024-03-27T11:07:02.668 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.697+0000 7f6aa07eb740 1 --2- 172.21.15.97:0/3425756344 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5569556e6e60 0x5569556e9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:07:02.668 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.697+0000 7f6aa07eb740 4 client.26299 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-27T11:07:02.668 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.697+0000 7f6aa07eb740 1 -- 172.21.15.97:0/3425756344 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5569555c55a0 con 0x5569556e6e60 2024-03-27T11:07:02.668 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.698+0000 7f6a96000640 1 --2- 172.21.15.97:0/3425756344 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5569556e6e60 0x5569556e9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:07:02.669 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.698+0000 7f6a96000640 1 --2- 172.21.15.97:0/3425756344 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5569556e6e60 0x5569556e9280 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2f776a1dd3ff598 server_cookie=4637472f956930da in_seq=0 out_seq=0 2024-03-27T11:07:02.669 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.698+0000 7f6a80c00640 10 client.26299 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T11:07:02.674 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.704+0000 7f6a80c00640 1 -- 172.21.15.97:0/3425756344 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5569555c55a0 con 0x5569556e6e60 2024-03-27T11:07:02.674 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.704+0000 7f6a80c00640 10 client.26299 handle_command_reply: tid=0 2024-03-27T11:07:02.674 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.705+0000 7f6aa07eb740 1 client.26299 shutdown 2024-03-27T11:07:02.676 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.705+0000 7f6aa07eb740 1 -- 172.21.15.97:0/3425756344 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55695566f6d0 msgr2=0x5569556794c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:07:02.676 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.705+0000 7f6aa07eb740 1 --2- 172.21.15.97:0/3425756344 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55695566f6d0 0x5569556794c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6a88004630 tx=0x7f6a8801bba0 comp rx=0 tx=0).stop 2024-03-27T11:07:02.676 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.706+0000 7f6aa07eb740 1 -- 172.21.15.97:0/3425756344 shutdown_connections 2024-03-27T11:07:02.676 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.706+0000 7f6aa07eb740 1 --2- 172.21.15.97:0/3425756344 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x5569556e6e60 0x5569556e9280 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:02.677 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.706+0000 7f6aa07eb740 1 --2- 172.21.15.97:0/3425756344 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556955674680 0x55695567a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:02.677 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.706+0000 7f6aa07eb740 1 --2- 172.21.15.97:0/3425756344 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55695566f6d0 0x5569556794c0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:02.677 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.706+0000 7f6aa07eb740 1 --2- 172.21.15.97:0/3425756344 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556955669b70 0x556955678f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:02.677 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.706+0000 7f6aa07eb740 1 -- 172.21.15.97:0/3425756344 >> 172.21.15.97:0/3425756344 conn(0x5569556540d0 msgr2=0x55695566b920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:07:02.677 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.706+0000 7f6a80c00640 0 client.26299 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T11:07:02.677 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.706+0000 7f6aa07eb740 1 -- 172.21.15.97:0/3425756344 shutdown_connections 2024-03-27T11:07:02.677 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.707+0000 7f6aa07eb740 1 -- 172.21.15.97:0/3425756344 wait complete. 2024-03-27T11:07:02.677 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.708+0000 7f6aa07eb740 20 client.26299 trim_cache size 0 max 16384 2024-03-27T11:07:02.678 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-27T11:07:02.678 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-27T11:07:02.678 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-27T11:07:02.678 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-27T11:07:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.709+0000 7f6a9dc00640 1 -- 172.21.15.97:0/259079296 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f6a7c0a3730 msgr2=0x7f6a7c0a5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:07:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.709+0000 7f6a9dc00640 1 --2- 172.21.15.97:0/259079296 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f6a7c0a3730 0x7f6a7c0a5bf0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f6a90098490 tx=0x7f6a90000b60 comp rx=0 tx=0).stop 2024-03-27T11:07:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.709+0000 7f6a9dc00640 1 -- 172.21.15.97:0/259079296 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6a980780f0 msgr2=0x7f6a981a56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:07:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.709+0000 7f6a9dc00640 1 --2- 172.21.15.97:0/259079296 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6a980780f0 0x7f6a981a56f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6a90001440 tx=0x7f6a9000ed80 comp rx=0 tx=0).stop 2024-03-27T11:07:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.709+0000 7f6a9dc00640 1 -- 172.21.15.97:0/259079296 shutdown_connections 2024-03-27T11:07:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.709+0000 7f6a9dc00640 1 --2- 172.21.15.97:0/259079296 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f6a7c0a3730 0x7f6a7c0a5bf0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.709+0000 7f6a9dc00640 1 --2- 172.21.15.97:0/259079296 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6a980780f0 0x7f6a981a56f0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.709+0000 7f6a9dc00640 1 --2- 172.21.15.97:0/259079296 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6a98077750 0x7f6a9819e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.709+0000 7f6a9dc00640 1 --2- 172.21.15.97:0/259079296 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a98076510 0x7f6a9819e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.709+0000 7f6a9dc00640 1 -- 172.21.15.97:0/259079296 >> 172.21.15.97:0/259079296 conn(0x7f6a980ff160 msgr2=0x7f6a98100c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:07:02.680 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.709+0000 7f6a9dc00640 1 -- 172.21.15.97:0/259079296 shutdown_connections 2024-03-27T11:07:02.680 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:02.709+0000 7f6a9dc00640 1 -- 172.21.15.97:0/259079296 wait complete. 2024-03-27T11:07:03.638 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:07:03 smithi097 ceph-mon[31859]: pgmap v753: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 43 MiB/s wr, 747 op/s 2024-03-27T11:07:03.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:07:03 smithi177 ceph-mon[35294]: pgmap v753: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 43 MiB/s wr, 747 op/s 2024-03-27T11:07:03.764 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:07:03 smithi114 ceph-mon[35338]: pgmap v753: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 43 MiB/s wr, 747 op/s 2024-03-27T11:07:05.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:07:05 smithi177 ceph-mon[35294]: pgmap v754: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 45 MiB/s wr, 773 op/s 2024-03-27T11:07:05.764 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:07:05 smithi114 ceph-mon[35338]: pgmap v754: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 45 MiB/s wr, 773 op/s 2024-03-27T11:07:05.888 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:07:05 smithi097 ceph-mon[31859]: pgmap v754: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 45 MiB/s wr, 773 op/s 2024-03-27T11:07:07.671 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:07:07 smithi097 ceph-mon[31859]: pgmap v755: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 44 MiB/s wr, 755 op/s 2024-03-27T11:07:07.673 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:07:07 smithi177 ceph-mon[35294]: pgmap v755: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 44 MiB/s wr, 755 op/s 2024-03-27T11:07:07.764 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:07:07 smithi114 ceph-mon[35338]: pgmap v755: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 44 MiB/s wr, 755 op/s 2024-03-27T11:07:09.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:07:09 smithi177 ceph-mon[35294]: pgmap v756: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 46 MiB/s wr, 791 op/s 2024-03-27T11:07:09.764 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:07:09 smithi114 ceph-mon[35338]: pgmap v756: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 46 MiB/s wr, 791 op/s 2024-03-27T11:07:09.888 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:07:09 smithi097 ceph-mon[31859]: pgmap v756: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 46 MiB/s wr, 791 op/s 2024-03-27T11:07:11.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:07:11 smithi177 ceph-mon[35294]: pgmap v757: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 49 MiB/s wr, 858 op/s 2024-03-27T11:07:11.764 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:07:11 smithi114 ceph-mon[35338]: pgmap v757: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 49 MiB/s wr, 858 op/s 2024-03-27T11:07:11.888 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:07:11 smithi097 ceph-mon[31859]: pgmap v757: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 49 MiB/s wr, 858 op/s 2024-03-27T11:07:13.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:07:13 smithi177 ceph-mon[35294]: pgmap v758: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 45 MiB/s wr, 784 op/s 2024-03-27T11:07:13.764 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:07:13 smithi114 ceph-mon[35338]: pgmap v758: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 45 MiB/s wr, 784 op/s 2024-03-27T11:07:13.888 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:07:13 smithi097 ceph-mon[31859]: pgmap v758: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 45 MiB/s wr, 784 op/s 2024-03-27T11:07:15.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:07:15 smithi177 ceph-mon[35294]: pgmap v759: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 46 MiB/s wr, 801 op/s 2024-03-27T11:07:15.764 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:07:15 smithi114 ceph-mon[35338]: pgmap v759: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 46 MiB/s wr, 801 op/s 2024-03-27T11:07:15.888 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:07:15 smithi097 ceph-mon[31859]: pgmap v759: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 46 MiB/s wr, 801 op/s 2024-03-27T11:07:15.937 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.968+0000 7f927b000640 1 -- 172.21.15.97:0/2923917472 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9274101700 msgr2=0x7f9274101b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:07:15.937 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.968+0000 7f927b000640 1 --2- 172.21.15.97:0/2923917472 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9274101700 0x7f9274101b80 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f926c009ca0 tx=0x7f926c02c100 comp rx=0 tx=0).stop 2024-03-27T11:07:15.937 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.968+0000 7f927b000640 1 -- 172.21.15.97:0/2923917472 shutdown_connections 2024-03-27T11:07:15.937 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.968+0000 7f927b000640 1 --2- 172.21.15.97:0/2923917472 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9274101700 0x7f9274101b80 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:15.937 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.968+0000 7f927b000640 1 --2- 172.21.15.97:0/2923917472 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9274100d60 0x7f92741011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:15.937 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.968+0000 7f927b000640 1 --2- 172.21.15.97:0/2923917472 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9274100400 0x7f9274100820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:15.937 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.968+0000 7f927b000640 1 -- 172.21.15.97:0/2923917472 >> 172.21.15.97:0/2923917472 conn(0x7f92740fc000 msgr2=0x7f92740fe440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:07:15.938 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.968+0000 7f927b000640 1 -- 172.21.15.97:0/2923917472 shutdown_connections 2024-03-27T11:07:15.938 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.969+0000 7f927b000640 1 -- 172.21.15.97:0/2923917472 wait complete. 2024-03-27T11:07:15.938 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.969+0000 7f927b000640 1 Processor -- start 2024-03-27T11:07:15.938 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.970+0000 7f927b000640 1 -- start start 2024-03-27T11:07:15.939 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.970+0000 7f927b000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9274100400 0x7f9274104d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:07:15.939 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.970+0000 7f927b000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9274100d60 0x7f92741052c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:07:15.939 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.970+0000 7f927b000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9274101700 0x7f927410c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:07:15.939 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.970+0000 7f927b000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f927410f890 con 0x7f9274100d60 2024-03-27T11:07:15.939 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.970+0000 7f927b000640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f927410f710 con 0x7f9274101700 2024-03-27T11:07:15.939 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.970+0000 7f927b000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f927410fa10 con 0x7f9274100400 2024-03-27T11:07:15.940 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.970+0000 7f9273e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9274100400 0x7f9274104d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:07:15.940 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.971+0000 7f9273e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9274100400 0x7f9274104d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.97:37572/0 (socket says 172.21.15.97:37572) 2024-03-27T11:07:15.940 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.971+0000 7f9273e00640 1 -- 172.21.15.97:0/505584403 learned_addr learned my addr 172.21.15.97:0/505584403 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:07:15.940 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.971+0000 7f9279000640 1 --2- 172.21.15.97:0/505584403 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9274101700 0x7f927410c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:07:15.940 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.971+0000 7f9273400640 1 --2- 172.21.15.97:0/505584403 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9274100d60 0x7f92741052c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:07:15.940 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.971+0000 7f9273e00640 1 -- 172.21.15.97:0/505584403 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9274101700 msgr2=0x7f927410c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:07:15.941 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.971+0000 7f9273e00640 1 --2- 172.21.15.97:0/505584403 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9274101700 0x7f927410c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:15.941 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.971+0000 7f9273e00640 1 -- 172.21.15.97:0/505584403 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9274100d60 msgr2=0x7f92741052c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:07:15.941 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.971+0000 7f9273e00640 1 --2- 172.21.15.97:0/505584403 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9274100d60 0x7f92741052c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:15.941 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.971+0000 7f9273e00640 1 -- 172.21.15.97:0/505584403 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92741ae790 con 0x7f9274100400 2024-03-27T11:07:15.941 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.972+0000 7f9273e00640 1 --2- 172.21.15.97:0/505584403 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9274100400 0x7f9274104d80 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f926400d950 tx=0x7f926400de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:07:15.941 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.973+0000 7f9270c00640 1 -- 172.21.15.97:0/505584403 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9264017070 con 0x7f9274100400 2024-03-27T11:07:15.942 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.973+0000 7f9270c00640 1 -- 172.21.15.97:0/505584403 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9264009c30 con 0x7f9274100400 2024-03-27T11:07:15.942 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.973+0000 7f9270c00640 1 -- 172.21.15.97:0/505584403 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f926400f620 con 0x7f9274100400 2024-03-27T11:07:15.942 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.973+0000 7f927b000640 1 -- 172.21.15.97:0/505584403 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92741aea20 con 0x7f9274100400 2024-03-27T11:07:15.942 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.973+0000 7f927b000640 1 -- 172.21.15.97:0/505584403 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92741aeef0 con 0x7f9274100400 2024-03-27T11:07:15.945 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.976+0000 7f927de1c740 1 Processor -- start 2024-03-27T11:07:15.945 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.977+0000 7f927de1c740 1 -- start start 2024-03-27T11:07:15.946 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.977+0000 7f927de1c740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ae98e53980 0x55ae98e53da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:07:15.946 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.977+0000 7f927de1c740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ae98e54370 0x55ae98e58e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:07:15.946 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.977+0000 7f927de1c740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ae98e594e0 0x55ae98e5b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:07:15.946 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.977+0000 7f927de1c740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ae98db8d10 con 0x55ae98e53980 2024-03-27T11:07:15.946 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.977+0000 7f927de1c740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55ae98d61260 con 0x55ae98e594e0 2024-03-27T11:07:15.946 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.977+0000 7f927de1c740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55ae98db4cc0 con 0x55ae98e54370 2024-03-27T11:07:15.948 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.978+0000 7f9273400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ae98e594e0 0x55ae98e5b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:07:15.948 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.978+0000 7f9273400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ae98e594e0 0x55ae98e5b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:37728/0 (socket says 172.21.15.97:37728) 2024-03-27T11:07:15.948 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.978+0000 7f9273400640 1 -- 172.21.15.97:0/968420899 learned_addr learned my addr 172.21.15.97:0/968420899 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:07:15.948 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.978+0000 7f9273400640 1 -- 172.21.15.97:0/968420899 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ae98e54370 msgr2=0x55ae98e58e70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-27T11:07:15.949 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.978+0000 7f9273400640 1 --2- 172.21.15.97:0/968420899 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ae98e54370 0x55ae98e58e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:15.949 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.978+0000 7f9273400640 1 -- 172.21.15.97:0/968420899 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ae98e53980 msgr2=0x55ae98e53da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:07:15.949 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.978+0000 7f9270c00640 1 -- 172.21.15.97:0/505584403 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f92640109a0 con 0x7f9274100400 2024-03-27T11:07:15.949 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.978+0000 7f9279000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ae98e53980 0x55ae98e53da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:07:15.949 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.979+0000 7f9273400640 1 --2- 172.21.15.97:0/968420899 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ae98e53980 0x55ae98e53da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:15.949 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.979+0000 7f9273400640 1 -- 172.21.15.97:0/968420899 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae98d58aa0 con 0x55ae98e594e0 2024-03-27T11:07:15.950 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.980+0000 7f9279000640 1 --2- 172.21.15.97:0/968420899 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ae98e53980 0x55ae98e53da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T11:07:15.950 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.980+0000 7f9273400640 1 --2- 172.21.15.97:0/968420899 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ae98e594e0 0x55ae98e5b970 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9260009c90 tx=0x7f9260011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef1a94b937cd6fdf server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:07:15.950 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.980+0000 7f9270c00640 1 --2- 172.21.15.97:0/505584403 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f92580a3730 0x7f92580a5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:07:15.950 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.981+0000 7f9269600640 1 -- 172.21.15.97:0/968420899 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f926001e070 con 0x55ae98e594e0 2024-03-27T11:07:15.950 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.981+0000 7f9269600640 1 -- 172.21.15.97:0/968420899 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9260019ae0 con 0x55ae98e594e0 2024-03-27T11:07:15.950 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.981+0000 7f9269600640 1 -- 172.21.15.97:0/968420899 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f926000ea40 con 0x55ae98e594e0 2024-03-27T11:07:15.950 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.981+0000 7f9270c00640 1 -- 172.21.15.97:0/505584403 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f92640cf870 con 0x7f9274100400 2024-03-27T11:07:15.950 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.981+0000 7f9279000640 1 --2- 172.21.15.97:0/505584403 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f92580a3730 0x7f92580a5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:07:15.951 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.982+0000 7f927de1c740 1 -- 172.21.15.97:0/968420899 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ae98e594e0 msgr2=0x55ae98e5b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:07:15.951 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.982+0000 7f927de1c740 1 --2- 172.21.15.97:0/968420899 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ae98e594e0 0x55ae98e5b970 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9260009c90 tx=0x7f9260011810 comp rx=0 tx=0).stop 2024-03-27T11:07:15.951 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.982+0000 7f927de1c740 1 -- 172.21.15.97:0/968420899 shutdown_connections 2024-03-27T11:07:15.951 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.982+0000 7f927de1c740 1 --2- 172.21.15.97:0/968420899 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ae98e594e0 0x55ae98e5b970 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:15.951 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.982+0000 7f927de1c740 1 --2- 172.21.15.97:0/968420899 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ae98e54370 0x55ae98e58e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:15.951 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.982+0000 7f927de1c740 1 --2- 172.21.15.97:0/968420899 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ae98e53980 0x55ae98e53da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:15.951 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.982+0000 7f927de1c740 1 -- 172.21.15.97:0/968420899 >> 172.21.15.97:0/968420899 conn(0x55ae98e4eef0 msgr2=0x55ae98e4f2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:07:15.951 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.982+0000 7f9279000640 1 --2- 172.21.15.97:0/505584403 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f92580a3730 0x7f92580a5bf0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f926c02c710 tx=0x7f926c02e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:07:15.952 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.982+0000 7f927de1c740 1 -- 172.21.15.97:0/968420899 shutdown_connections 2024-03-27T11:07:15.953 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.985+0000 7f927de1c740 1 -- 172.21.15.97:0/968420899 wait complete. 2024-03-27T11:07:15.954 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.986+0000 7f927de1c740 1 -- start start 2024-03-27T11:07:15.955 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.986+0000 7f927de1c740 1 Processor -- start 2024-03-27T11:07:15.955 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.987+0000 7f927de1c740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ae98e53980 0x55ae98e5dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:07:15.955 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.987+0000 7f927de1c740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ae98e54370 0x55ae98e5e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:07:15.956 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.987+0000 7f927de1c740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ae98e594e0 0x55ae98e65290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:07:15.956 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.987+0000 7f927de1c740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ae98e5c490 con 0x55ae98e594e0 2024-03-27T11:07:15.956 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.987+0000 7f927de1c740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55ae98d74630 con 0x55ae98e53980 2024-03-27T11:07:15.956 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.987+0000 7f927de1c740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55ae98e5c610 con 0x55ae98e54370 2024-03-27T11:07:15.956 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.987+0000 7f9273e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ae98e53980 0x55ae98e5dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:07:15.956 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.987+0000 7f9273e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ae98e53980 0x55ae98e5dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:37742/0 (socket says 172.21.15.97:37742) 2024-03-27T11:07:15.956 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.987+0000 7f9273e00640 1 -- 172.21.15.97:0/209680715 learned_addr learned my addr 172.21.15.97:0/209680715 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:07:15.957 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.987+0000 7f927de1c740 10 client.0 _ll_register_callbacks cb 0x55ae98d79c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T11:07:15.957 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.988+0000 7f9279000640 1 --2- 172.21.15.97:0/209680715 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ae98e594e0 0x55ae98e65290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:07:15.957 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.988+0000 7f9273e00640 1 -- 172.21.15.97:0/209680715 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ae98e54370 msgr2=0x55ae98e5e250 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-27T11:07:15.957 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.988+0000 7f9273e00640 1 --2- 172.21.15.97:0/209680715 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ae98e54370 0x55ae98e5e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:15.957 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.988+0000 7f9273e00640 1 -- 172.21.15.97:0/209680715 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ae98e594e0 msgr2=0x55ae98e65290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:07:15.958 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.988+0000 7f9273e00640 1 --2- 172.21.15.97:0/209680715 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ae98e594e0 0x55ae98e65290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:15.958 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.988+0000 7f9273e00640 1 -- 172.21.15.97:0/209680715 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae98d805f0 con 0x55ae98e53980 2024-03-27T11:07:15.958 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.988+0000 7f9279000640 1 --2- 172.21.15.97:0/209680715 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ae98e594e0 0x55ae98e65290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T11:07:15.958 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.989+0000 7f9273e00640 1 --2- 172.21.15.97:0/209680715 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ae98e53980 0x55ae98e5dd10 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f92640a4490 tx=0x7f926409b760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a560bc0dce33fb3a server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:07:15.959 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.989+0000 7f925e000640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-03-27T11:07:15.959 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.989+0000 7f925e000640 1 -- 172.21.15.97:0/209680715 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9264020040 con 0x55ae98e53980 2024-03-27T11:07:15.959 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.989+0000 7f927de1c740 1 -- 172.21.15.97:0/209680715 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9260019ae0 con 0x55ae98e53980 2024-03-27T11:07:15.959 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.989+0000 7f925e000640 1 -- 172.21.15.97:0/209680715 <== mon.2 v2:172.21.15.114:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f9264098470 con 0x55ae98e53980 2024-03-27T11:07:15.959 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.989+0000 7f925e000640 1 -- 172.21.15.97:0/209680715 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f92640a0100 con 0x55ae98e53980 2024-03-27T11:07:15.959 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.989+0000 7f927de1c740 10 client.36590 fetch_fsmap learned FSMap version 34 2024-03-27T11:07:15.959 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.989+0000 7f927de1c740 1 -- 172.21.15.97:0/209680715 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55ae98acd210 con 0x55ae98e53980 2024-03-27T11:07:15.959 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.990+0000 7f925e000640 1 -- 172.21.15.97:0/209680715 <== mon.2 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9260019ae0 con 0x55ae98e53980 2024-03-27T11:07:15.959 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.990+0000 7f925e000640 1 -- 172.21.15.97:0/209680715 <== mon.2 v2:172.21.15.114:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f9264010460 con 0x55ae98e53980 2024-03-27T11:07:15.960 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.991+0000 7f927de1c740 10 client.36590 fetch_fsmap finished waiting for FSMap version 34 2024-03-27T11:07:15.960 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.991+0000 7f927de1c740 10 client.36590 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T11:07:15.960 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.991+0000 7f927de1c740 20 client.36590 populate_metadata read hostname 'smithi097' 2024-03-27T11:07:15.960 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.991+0000 7f927de1c740 1 --2- 172.21.15.97:0/209680715 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55ae98e69aa0 0x55ae98e6be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:07:15.960 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.991+0000 7f927de1c740 4 client.36590 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T11:07:15.960 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.991+0000 7f927de1c740 1 -- 172.21.15.97:0/209680715 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ae98dc6250 con 0x55ae98e69aa0 2024-03-27T11:07:15.961 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.992+0000 7f9273400640 1 --2- 172.21.15.97:0/209680715 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55ae98e69aa0 0x55ae98e6be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:07:15.963 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.994+0000 7f9273400640 1 --2- 172.21.15.97:0/209680715 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55ae98e69aa0 0x55ae98e6be90 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=878388077aa37f37 server_cookie=122152334ab271a3 in_seq=0 out_seq=0 2024-03-27T11:07:15.963 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.995+0000 7f925e000640 10 client.36590 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T11:07:15.963 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.995+0000 7f925e000640 1 -- 172.21.15.97:0/209680715 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55ae98dc6250 con 0x55ae98e69aa0 2024-03-27T11:07:15.964 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.995+0000 7f925e000640 10 client.36590 handle_command_reply: tid=0 2024-03-27T11:07:15.965 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.996+0000 7f927de1c740 1 client.36590 shutdown 2024-03-27T11:07:15.965 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.996+0000 7f927de1c740 1 -- 172.21.15.97:0/209680715 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ae98e53980 msgr2=0x55ae98e5dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:07:15.965 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.996+0000 7f927de1c740 1 --2- 172.21.15.97:0/209680715 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ae98e53980 0x55ae98e5dd10 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f92640a4490 tx=0x7f926409b760 comp rx=0 tx=0).stop 2024-03-27T11:07:15.965 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.997+0000 7f927de1c740 1 -- 172.21.15.97:0/209680715 shutdown_connections 2024-03-27T11:07:15.965 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.997+0000 7f927de1c740 1 --2- 172.21.15.97:0/209680715 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55ae98e69aa0 0x55ae98e6be90 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:15.966 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.997+0000 7f927de1c740 1 --2- 172.21.15.97:0/209680715 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ae98e594e0 0x55ae98e65290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:15.966 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.997+0000 7f927de1c740 1 --2- 172.21.15.97:0/209680715 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ae98e54370 0x55ae98e5e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:15.966 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.997+0000 7f925e000640 0 client.36590 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T11:07:15.966 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.997+0000 7f927de1c740 1 --2- 172.21.15.97:0/209680715 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ae98e53980 0x55ae98e5dd10 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:15.966 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.997+0000 7f927de1c740 1 -- 172.21.15.97:0/209680715 >> 172.21.15.97:0/209680715 conn(0x55ae98e4eef0 msgr2=0x55ae98e4e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:07:15.966 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.997+0000 7f927de1c740 1 -- 172.21.15.97:0/209680715 shutdown_connections 2024-03-27T11:07:15.967 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.998+0000 7f927de1c740 1 -- 172.21.15.97:0/209680715 wait complete. 2024-03-27T11:07:15.967 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:15.998+0000 7f927de1c740 20 client.36590 trim_cache size 0 max 16384 2024-03-27T11:07:15.997 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:16.028+0000 7f927de1c740 1 Processor -- start 2024-03-27T11:07:15.997 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:16.028+0000 7f927de1c740 1 -- start start 2024-03-27T11:07:15.998 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:16.028+0000 7f927de1c740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ae98e53980 0x55ae98e5e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:07:15.998 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:16.028+0000 7f927de1c740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ae98e54370 0x55ae98e65250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:07:15.998 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:16.028+0000 7f927de1c740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ae98e594e0 0x55ae98e70770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:07:15.998 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:16.029+0000 7f927de1c740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f92741b7e20 con 0x55ae98e54370 2024-03-27T11:07:15.998 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:16.029+0000 7f927de1c740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f92741b9960 con 0x55ae98e53980 2024-03-27T11:07:15.998 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:16.029+0000 7f927de1c740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f92741b84f0 con 0x55ae98e594e0 2024-03-27T11:07:15.998 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:16.029+0000 7f9273e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ae98e53980 0x55ae98e5e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:07:15.998 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:16.029+0000 7f9273e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ae98e53980 0x55ae98e5e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:37750/0 (socket says 172.21.15.97:37750) 2024-03-27T11:07:15.998 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:16.029+0000 7f9273e00640 1 -- 172.21.15.97:0/3563978287 learned_addr learned my addr 172.21.15.97:0/3563978287 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:07:15.999 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:16.029+0000 7f9273400640 1 --2- 172.21.15.97:0/3563978287 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ae98e54370 0x55ae98e65250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:07:15.999 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:16.030+0000 7f9273e00640 1 -- 172.21.15.97:0/3563978287 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ae98e594e0 msgr2=0x55ae98e70770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:07:15.999 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:16.030+0000 7f9273e00640 1 --2- 172.21.15.97:0/3563978287 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ae98e594e0 0x55ae98e70770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:16.000 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:16.030+0000 7f9273e00640 1 -- 172.21.15.97:0/3563978287 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ae98e54370 msgr2=0x55ae98e65250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:07:16.000 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:16.030+0000 7f9273e00640 1 --2- 172.21.15.97:0/3563978287 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ae98e54370 0x55ae98e65250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:16.000 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:16.030+0000 7f9273e00640 1 -- 172.21.15.97:0/3563978287 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae98b99620 con 0x55ae98e53980 2024-03-27T11:07:16.000 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:16.030+0000 7f9273e00640 1 --2- 172.21.15.97:0/3563978287 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ae98e53980 0x55ae98e5e030 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f92640cd3d0 tx=0x7f926409a310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4983a1309b89bb86 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:07:16.000 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:16.030+0000 7f9273400640 1 --2- 172.21.15.97:0/3563978287 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ae98e54370 0x55ae98e65250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T11:07:16.000 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:16.031+0000 7f9268c00640 1 -- 172.21.15.97:0/3563978287 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9264020030 con 0x55ae98e53980 2024-03-27T11:07:16.001 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:16.032+0000 7f9268c00640 1 -- 172.21.15.97:0/3563978287 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f926409af90 con 0x55ae98e53980 2024-03-27T11:07:16.001 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:16.032+0000 7f9268c00640 1 -- 172.21.15.97:0/3563978287 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9264097db0 con 0x55ae98e53980 2024-03-27T11:07:16.005 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:16.033+0000 7f927de1c740 1 -- 172.21.15.97:0/3563978287 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ae98e53980 msgr2=0x55ae98e5e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:07:16.005 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:16.033+0000 7f927de1c740 1 --2- 172.21.15.97:0/3563978287 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ae98e53980 0x55ae98e5e030 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f92640cd3d0 tx=0x7f926409a310 comp rx=0 tx=0).stop 2024-03-27T11:07:16.005 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:16.036+0000 7f927de1c740 1 -- 172.21.15.97:0/3563978287 shutdown_connections 2024-03-27T11:07:16.005 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:16.036+0000 7f927de1c740 1 --2- 172.21.15.97:0/3563978287 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ae98e594e0 0x55ae98e70770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:16.005 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:16.036+0000 7f927de1c740 1 --2- 172.21.15.97:0/3563978287 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ae98e54370 0x55ae98e65250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:16.005 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:16.036+0000 7f927de1c740 1 --2- 172.21.15.97:0/3563978287 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ae98e53980 0x55ae98e5e030 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:16.005 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:16.036+0000 7f927de1c740 1 -- 172.21.15.97:0/3563978287 >> 172.21.15.97:0/3563978287 conn(0x55ae98e4eef0 msgr2=0x55ae98e5d920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:07:16.005 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:16.036+0000 7f927de1c740 1 -- 172.21.15.97:0/3563978287 shutdown_connections 2024-03-27T11:07:16.007 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:16.037+0000 7f927de1c740 1 -- 172.21.15.97:0/3563978287 wait complete. 2024-03-27T11:07:16.007 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:16.037+0000 7f927de1c740 1 -- start start 2024-03-27T11:07:16.007 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:16.037+0000 7f927de1c740 1 Processor -- start 2024-03-27T11:07:16.009 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:16.041+0000 7f927de1c740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ae98e53980 0x55ae98e62e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:07:16.011 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:16.041+0000 7f927de1c740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ae98e54370 0x55ae98e633a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:07:16.011 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:16.041+0000 7f9273e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ae98e53980 0x55ae98e62e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:07:16.011 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:16.042+0000 7f9273e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ae98e53980 0x55ae98e62e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:37766/0 (socket says 172.21.15.97:37766) 2024-03-27T11:07:16.011 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:16.042+0000 7f9273e00640 1 -- 172.21.15.97:0/3076430348 learned_addr learned my addr 172.21.15.97:0/3076430348 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:07:16.012 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:16.042+0000 7f927de1c740 1 --2- 172.21.15.97:0/3076430348 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ae98e594e0 0x55ae98e643b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:07:16.012 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:16.043+0000 7f927de1c740 1 -- 172.21.15.97:0/3076430348 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f92741b9290 con 0x55ae98e594e0 2024-03-27T11:07:16.012 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:16.043+0000 7f927de1c740 1 -- 172.21.15.97:0/3076430348 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f92741b8bc0 con 0x55ae98e53980 2024-03-27T11:07:16.012 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:16.043+0000 7f927de1c740 1 -- 172.21.15.97:0/3076430348 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f92741b5e00 con 0x55ae98e54370 2024-03-27T11:07:16.012 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:16.043+0000 7f9273400640 1 --2- 172.21.15.97:0/3076430348 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ae98e54370 0x55ae98e633a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:07:16.012 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:16.043+0000 7f927de1c740 10 client.0 _ll_register_callbacks cb 0x7f926400df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T11:07:16.012 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:16.044+0000 7f9273400640 1 -- 172.21.15.97:0/3076430348 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ae98e53980 msgr2=0x55ae98e62e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:07:16.013 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:16.044+0000 7f9273400640 1 --2- 172.21.15.97:0/3076430348 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ae98e53980 0x55ae98e62e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:16.013 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:16.044+0000 7f9273400640 1 -- 172.21.15.97:0/3076430348 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ae98e594e0 msgr2=0x55ae98e643b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:07:16.013 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:16.044+0000 7f9279000640 1 --2- 172.21.15.97:0/3076430348 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ae98e594e0 0x55ae98e643b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:07:16.013 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:16.044+0000 7f9273400640 1 --2- 172.21.15.97:0/3076430348 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ae98e594e0 0x55ae98e643b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:16.013 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:16.044+0000 7f9273400640 1 -- 172.21.15.97:0/3076430348 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae98e649a0 con 0x55ae98e54370 2024-03-27T11:07:16.013 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:16.044+0000 7f9279000640 1 --2- 172.21.15.97:0/3076430348 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ae98e594e0 0x55ae98e643b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T11:07:16.014 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:16.045+0000 7f9273400640 1 --2- 172.21.15.97:0/3076430348 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ae98e54370 0x55ae98e633a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f926000a010 tx=0x7f9260009f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6289ae21efe92780 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:07:16.014 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:16.045+0000 7f925e000640 10 client.0 ms_handle_connect on v2:172.21.15.177:3300/0 2024-03-27T11:07:16.015 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:16.045+0000 7f925e000640 1 -- 172.21.15.97:0/3076430348 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f926001e040 con 0x55ae98e54370 2024-03-27T11:07:16.015 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:16.046+0000 7f925e000640 1 -- 172.21.15.97:0/3076430348 <== mon.1 v2:172.21.15.177:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f9260029490 con 0x55ae98e54370 2024-03-27T11:07:16.015 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:16.046+0000 7f927de1c740 1 -- 172.21.15.97:0/3076430348 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f926409af90 con 0x55ae98e54370 2024-03-27T11:07:16.015 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:16.046+0000 7f925e000640 1 -- 172.21.15.97:0/3076430348 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f9260029690 con 0x55ae98e54370 2024-03-27T11:07:16.015 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:16.046+0000 7f927de1c740 10 client.26308 fetch_fsmap learned FSMap version 34 2024-03-27T11:07:16.015 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:16.046+0000 7f927de1c740 1 -- 172.21.15.97:0/3076430348 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55ae98ed0280 con 0x55ae98e54370 2024-03-27T11:07:16.015 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:16.047+0000 7f925e000640 1 -- 172.21.15.97:0/3076430348 <== mon.1 v2:172.21.15.177:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f926409af90 con 0x55ae98e54370 2024-03-27T11:07:16.017 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:16.048+0000 7f925e000640 1 -- 172.21.15.97:0/3076430348 <== mon.1 v2:172.21.15.177:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f9260019740 con 0x55ae98e54370 2024-03-27T11:07:16.017 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:16.048+0000 7f927de1c740 10 client.26308 fetch_fsmap finished waiting for FSMap version 34 2024-03-27T11:07:16.017 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:16.048+0000 7f927de1c740 10 client.26308 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T11:07:16.017 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:16.048+0000 7f927de1c740 20 client.26308 populate_metadata read hostname 'smithi097' 2024-03-27T11:07:16.017 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:16.048+0000 7f927de1c740 1 --2- 172.21.15.97:0/3076430348 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55ae98ed09f0 0x55ae98ed2de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:07:16.017 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:16.048+0000 7f927de1c740 4 client.26308 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-27T11:07:16.017 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:16.048+0000 7f927de1c740 1 -- 172.21.15.97:0/3076430348 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ae98dc05a0 con 0x55ae98ed09f0 2024-03-27T11:07:16.017 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:16.048+0000 7f9273e00640 1 --2- 172.21.15.97:0/3076430348 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55ae98ed09f0 0x55ae98ed2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:07:16.022 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:16.053+0000 7f9273e00640 1 --2- 172.21.15.97:0/3076430348 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55ae98ed09f0 0x55ae98ed2de0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac59321f76715995 server_cookie=52589fd7a76ad5db in_seq=0 out_seq=0 2024-03-27T11:07:16.022 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:16.053+0000 7f925e000640 10 client.26308 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T11:07:16.047 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:16.077+0000 7f925e000640 1 -- 172.21.15.97:0/3076430348 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ae98dc05a0 con 0x55ae98ed09f0 2024-03-27T11:07:16.047 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:16.077+0000 7f925e000640 10 client.26308 handle_command_reply: tid=0 2024-03-27T11:07:16.047 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:16.077+0000 7f927de1c740 1 client.26308 shutdown 2024-03-27T11:07:16.047 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:16.078+0000 7f927de1c740 1 -- 172.21.15.97:0/3076430348 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ae98e54370 msgr2=0x55ae98e633a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:07:16.047 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:16.078+0000 7f927de1c740 1 --2- 172.21.15.97:0/3076430348 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ae98e54370 0x55ae98e633a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f926000a010 tx=0x7f9260009f70 comp rx=0 tx=0).stop 2024-03-27T11:07:16.047 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:16.078+0000 7f927de1c740 1 -- 172.21.15.97:0/3076430348 shutdown_connections 2024-03-27T11:07:16.047 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:16.078+0000 7f927de1c740 1 --2- 172.21.15.97:0/3076430348 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55ae98ed09f0 0x55ae98ed2de0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:16.047 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:16.078+0000 7f927de1c740 1 --2- 172.21.15.97:0/3076430348 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ae98e594e0 0x55ae98e643b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:16.047 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:16.078+0000 7f927de1c740 1 --2- 172.21.15.97:0/3076430348 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ae98e54370 0x55ae98e633a0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:16.078+0000 7f925e000640 0 client.26308 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T11:07:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:16.078+0000 7f927de1c740 1 --2- 172.21.15.97:0/3076430348 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ae98e53980 0x55ae98e62e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:16.078+0000 7f927de1c740 1 -- 172.21.15.97:0/3076430348 >> 172.21.15.97:0/3076430348 conn(0x55ae98e4eef0 msgr2=0x55ae98e4d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:07:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:16.079+0000 7f927de1c740 1 -- 172.21.15.97:0/3076430348 shutdown_connections 2024-03-27T11:07:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:16.079+0000 7f927de1c740 1 -- 172.21.15.97:0/3076430348 wait complete. 2024-03-27T11:07:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:16.079+0000 7f927de1c740 20 client.26308 trim_cache size 0 max 16384 2024-03-27T11:07:16.049 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-27T11:07:16.049 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-27T11:07:16.049 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-27T11:07:16.049 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-27T11:07:16.049 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:16.080+0000 7f927b000640 1 -- 172.21.15.97:0/505584403 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f92580a3730 msgr2=0x7f92580a5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:07:16.050 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:16.080+0000 7f927b000640 1 --2- 172.21.15.97:0/505584403 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f92580a3730 0x7f92580a5bf0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f926c02c710 tx=0x7f926c02e040 comp rx=0 tx=0).stop 2024-03-27T11:07:16.050 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:16.080+0000 7f927b000640 1 -- 172.21.15.97:0/505584403 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9274100400 msgr2=0x7f9274104d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:07:16.050 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:16.080+0000 7f927b000640 1 --2- 172.21.15.97:0/505584403 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9274100400 0x7f9274104d80 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f926400d950 tx=0x7f926400de10 comp rx=0 tx=0).stop 2024-03-27T11:07:16.050 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:16.080+0000 7f927b000640 1 -- 172.21.15.97:0/505584403 shutdown_connections 2024-03-27T11:07:16.050 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:16.080+0000 7f927b000640 1 --2- 172.21.15.97:0/505584403 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f92580a3730 0x7f92580a5bf0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:16.050 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:16.080+0000 7f927b000640 1 --2- 172.21.15.97:0/505584403 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9274101700 0x7f927410c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:16.050 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:16.080+0000 7f927b000640 1 --2- 172.21.15.97:0/505584403 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9274100d60 0x7f92741052c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:16.050 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:16.080+0000 7f927b000640 1 --2- 172.21.15.97:0/505584403 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9274100400 0x7f9274104d80 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:16.050 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:16.081+0000 7f927b000640 1 -- 172.21.15.97:0/505584403 >> 172.21.15.97:0/505584403 conn(0x7f92740fc000 msgr2=0x7f92740fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:07:16.050 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:16.081+0000 7f927b000640 1 -- 172.21.15.97:0/505584403 shutdown_connections 2024-03-27T11:07:16.050 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:16.081+0000 7f927b000640 1 -- 172.21.15.97:0/505584403 wait complete. 2024-03-27T11:07:17.764 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:07:17 smithi114 ceph-mon[35338]: pgmap v760: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 44 MiB/s wr, 768 op/s 2024-03-27T11:07:17.851 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:07:17 smithi097 ceph-mon[31859]: pgmap v760: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 44 MiB/s wr, 768 op/s 2024-03-27T11:07:17.923 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:07:17 smithi177 ceph-mon[35294]: pgmap v760: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 44 MiB/s wr, 768 op/s 2024-03-27T11:07:19.764 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:07:19 smithi114 ceph-mon[35338]: pgmap v761: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 47 MiB/s wr, 810 op/s 2024-03-27T11:07:19.888 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:07:19 smithi097 ceph-mon[31859]: pgmap v761: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 47 MiB/s wr, 810 op/s 2024-03-27T11:07:19.923 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:07:19 smithi177 ceph-mon[35294]: pgmap v761: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 47 MiB/s wr, 810 op/s 2024-03-27T11:07:21.764 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:07:21 smithi114 ceph-mon[35338]: pgmap v762: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 50 MiB/s wr, 882 op/s 2024-03-27T11:07:21.888 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:07:21 smithi097 ceph-mon[31859]: pgmap v762: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 50 MiB/s wr, 882 op/s 2024-03-27T11:07:21.924 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:07:21 smithi177 ceph-mon[35294]: pgmap v762: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 50 MiB/s wr, 882 op/s 2024-03-27T11:07:23.764 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:07:23 smithi114 ceph-mon[35338]: pgmap v763: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 46 MiB/s wr, 805 op/s 2024-03-27T11:07:23.899 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:07:23 smithi097 ceph-mon[31859]: pgmap v763: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 46 MiB/s wr, 805 op/s 2024-03-27T11:07:23.923 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:07:23 smithi177 ceph-mon[35294]: pgmap v763: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 46 MiB/s wr, 805 op/s 2024-03-27T11:07:25.726 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:07:25 smithi177 ceph-mon[35294]: pgmap v764: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 47 MiB/s wr, 822 op/s 2024-03-27T11:07:25.764 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:07:25 smithi114 ceph-mon[35338]: pgmap v764: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 47 MiB/s wr, 822 op/s 2024-03-27T11:07:25.888 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:07:25 smithi097 ceph-mon[31859]: pgmap v764: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 47 MiB/s wr, 822 op/s 2024-03-27T11:07:27.764 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:07:27 smithi114 ceph-mon[35338]: pgmap v765: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 45 MiB/s wr, 799 op/s 2024-03-27T11:07:27.888 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:07:27 smithi097 ceph-mon[31859]: pgmap v765: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 45 MiB/s wr, 799 op/s 2024-03-27T11:07:27.923 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:07:27 smithi177 ceph-mon[35294]: pgmap v765: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 45 MiB/s wr, 799 op/s 2024-03-27T11:07:29.414 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.446+0000 7f9294a00640 1 --2- 172.21.15.97:0/3851301117 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9290075b90 0x7f9290073fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T11:07:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.447+0000 7f9297400640 1 -- 172.21.15.97:0/3851301117 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f929010e170 msgr2=0x7f9290110560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:07:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.447+0000 7f9297400640 1 --2- 172.21.15.97:0/3851301117 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f929010e170 0x7f9290110560 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9288009be0 tx=0x7f928802bf10 comp rx=0 tx=0).stop 2024-03-27T11:07:29.416 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.448+0000 7f9297400640 1 -- 172.21.15.97:0/3851301117 shutdown_connections 2024-03-27T11:07:29.416 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.448+0000 7f9297400640 1 --2- 172.21.15.97:0/3851301117 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f929010e170 0x7f9290110560 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:29.416 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.448+0000 7f9297400640 1 --2- 172.21.15.97:0/3851301117 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f92900745b0 0x7f9290074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:29.416 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.448+0000 7f9297400640 1 --2- 172.21.15.97:0/3851301117 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9290075b90 0x7f9290073fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:29.417 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.448+0000 7f9297400640 1 -- 172.21.15.97:0/3851301117 >> 172.21.15.97:0/3851301117 conn(0x7f92900fd130 msgr2=0x7f92900ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:07:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.448+0000 7f9297400640 1 -- 172.21.15.97:0/3851301117 shutdown_connections 2024-03-27T11:07:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.448+0000 7f9297400640 1 -- 172.21.15.97:0/3851301117 wait complete. 2024-03-27T11:07:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.449+0000 7f9297400640 1 Processor -- start 2024-03-27T11:07:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.450+0000 7f9297400640 1 -- start start 2024-03-27T11:07:29.419 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.451+0000 7f9297400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f92900745b0 0x7f92900703e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:07:29.419 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.451+0000 7f9297400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9290075b90 0x7f9290070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:07:29.419 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.451+0000 7f9297400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f929010e170 0x7f92901ae440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:07:29.420 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.451+0000 7f9297400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9290113d90 con 0x7f929010e170 2024-03-27T11:07:29.420 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.451+0000 7f9297400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9290113c10 con 0x7f9290075b90 2024-03-27T11:07:29.420 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.451+0000 7f9297400640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f9290113f10 con 0x7f92900745b0 2024-03-27T11:07:29.420 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.451+0000 7f9294a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f92900745b0 0x7f92900703e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:07:29.420 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.451+0000 7f9294a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f92900745b0 0x7f92900703e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.97:45214/0 (socket says 172.21.15.97:45214) 2024-03-27T11:07:29.421 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.451+0000 7f9294a00640 1 -- 172.21.15.97:0/769776404 learned_addr learned my addr 172.21.15.97:0/769776404 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:07:29.421 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.451+0000 7f928fe00640 1 --2- 172.21.15.97:0/769776404 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9290075b90 0x7f9290070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:07:29.421 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.451+0000 7f9295400640 1 --2- 172.21.15.97:0/769776404 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f929010e170 0x7f92901ae440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:07:29.422 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.452+0000 7f9294a00640 1 -- 172.21.15.97:0/769776404 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9290075b90 msgr2=0x7f9290070920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:07:29.422 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.452+0000 7f9294a00640 1 --2- 172.21.15.97:0/769776404 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9290075b90 0x7f9290070920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:29.422 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.452+0000 7f9294a00640 1 -- 172.21.15.97:0/769776404 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f929010e170 msgr2=0x7f92901ae440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:07:29.422 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.452+0000 7f9294a00640 1 --2- 172.21.15.97:0/769776404 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f929010e170 0x7f92901ae440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:29.422 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.452+0000 7f9294a00640 1 -- 172.21.15.97:0/769776404 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9290073aa0 con 0x7f92900745b0 2024-03-27T11:07:29.422 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.452+0000 7f9295400640 1 --2- 172.21.15.97:0/769776404 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f929010e170 0x7f92901ae440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T11:07:29.422 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.453+0000 7f9294a00640 1 --2- 172.21.15.97:0/769776404 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f92900745b0 0x7f92900703e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f928400d950 tx=0x7f928400dd50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:07:29.422 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.453+0000 7f928d600640 1 -- 172.21.15.97:0/769776404 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9284017030 con 0x7f92900745b0 2024-03-27T11:07:29.422 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.453+0000 7f928d600640 1 -- 172.21.15.97:0/769776404 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9284004020 con 0x7f92900745b0 2024-03-27T11:07:29.423 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.453+0000 7f928d600640 1 -- 172.21.15.97:0/769776404 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9284005570 con 0x7f92900745b0 2024-03-27T11:07:29.423 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.453+0000 7f9297400640 1 -- 172.21.15.97:0/769776404 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92901ae9e0 con 0x7f92900745b0 2024-03-27T11:07:29.423 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.453+0000 7f9297400640 1 -- 172.21.15.97:0/769776404 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92901aeeb0 con 0x7f92900745b0 2024-03-27T11:07:29.426 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.457+0000 7f928d600640 1 -- 172.21.15.97:0/769776404 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f92840041d0 con 0x7f92900745b0 2024-03-27T11:07:29.427 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.459+0000 7f9299ff8740 1 Processor -- start 2024-03-27T11:07:29.428 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.459+0000 7f928d600640 1 --2- 172.21.15.97:0/769776404 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f92780a3730 0x7f92780a5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:07:29.428 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.459+0000 7f9299ff8740 1 -- start start 2024-03-27T11:07:29.428 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.459+0000 7f928d600640 1 -- 172.21.15.97:0/769776404 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f92840cf5b0 con 0x7f92900745b0 2024-03-27T11:07:29.428 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.459+0000 7f9299ff8740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556a485df980 0x556a485dfda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:07:29.429 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.459+0000 7f9299ff8740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556a485e0370 0x556a485e4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:07:29.429 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.459+0000 7f9299ff8740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556a485e54e0 0x556a485e7970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:07:29.430 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.459+0000 7f9299ff8740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x556a48544d10 con 0x556a485e0370 2024-03-27T11:07:29.430 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.459+0000 7f9299ff8740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x556a484ed260 con 0x556a485e54e0 2024-03-27T11:07:29.430 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.459+0000 7f9299ff8740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x556a48540cc0 con 0x556a485df980 2024-03-27T11:07:29.430 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.460+0000 7f9294a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556a485df980 0x556a485dfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:07:29.430 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.460+0000 7f9294a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556a485df980 0x556a485dfda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.97:45222/0 (socket says 172.21.15.97:45222) 2024-03-27T11:07:29.430 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.460+0000 7f9294a00640 1 -- 172.21.15.97:0/1765838299 learned_addr learned my addr 172.21.15.97:0/1765838299 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:07:29.430 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.460+0000 7f928fe00640 1 --2- 172.21.15.97:0/1765838299 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556a485e0370 0x556a485e4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:07:29.430 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.460+0000 7f9295400640 1 --2- 172.21.15.97:0/1765838299 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556a485e54e0 0x556a485e7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:07:29.430 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.460+0000 7f9295400640 1 --2- 172.21.15.97:0/769776404 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f92780a3730 0x7f92780a5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:07:29.430 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.460+0000 7f9294a00640 1 -- 172.21.15.97:0/1765838299 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556a485e54e0 msgr2=0x556a485e7970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:07:29.430 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.460+0000 7f9294a00640 1 --2- 172.21.15.97:0/1765838299 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556a485e54e0 0x556a485e7970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:29.430 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.460+0000 7f9294a00640 1 -- 172.21.15.97:0/1765838299 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556a485e0370 msgr2=0x556a485e4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:07:29.430 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.460+0000 7f9294a00640 1 --2- 172.21.15.97:0/1765838299 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556a485e0370 0x556a485e4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:29.431 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.460+0000 7f9294a00640 1 -- 172.21.15.97:0/1765838299 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a484e4aa0 con 0x556a485df980 2024-03-27T11:07:29.431 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.460+0000 7f9294a00640 1 --2- 172.21.15.97:0/1765838299 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556a485df980 0x556a485dfda0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9284098460 tx=0x7f928409b3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b521c941a5c3786b server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:07:29.431 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.461+0000 7f9281c00640 1 -- 172.21.15.97:0/1765838299 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f928409fe40 con 0x556a485df980 2024-03-27T11:07:29.431 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.461+0000 7f9281c00640 1 -- 172.21.15.97:0/1765838299 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f92840a4700 con 0x556a485df980 2024-03-27T11:07:29.431 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.461+0000 7f9281c00640 1 -- 172.21.15.97:0/1765838299 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92840977e0 con 0x556a485df980 2024-03-27T11:07:29.431 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.461+0000 7f9299ff8740 1 -- 172.21.15.97:0/1765838299 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556a485df980 msgr2=0x556a485dfda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:07:29.432 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.461+0000 7f9299ff8740 1 --2- 172.21.15.97:0/1765838299 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556a485df980 0x556a485dfda0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9284098460 tx=0x7f928409b3f0 comp rx=0 tx=0).stop 2024-03-27T11:07:29.432 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.462+0000 7f9299ff8740 1 -- 172.21.15.97:0/1765838299 shutdown_connections 2024-03-27T11:07:29.432 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.462+0000 7f9299ff8740 1 --2- 172.21.15.97:0/1765838299 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556a485e54e0 0x556a485e7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:29.432 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.462+0000 7f9299ff8740 1 --2- 172.21.15.97:0/1765838299 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556a485e0370 0x556a485e4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:29.432 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.462+0000 7f9299ff8740 1 --2- 172.21.15.97:0/1765838299 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556a485df980 0x556a485dfda0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:29.432 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.463+0000 7f9299ff8740 1 -- 172.21.15.97:0/1765838299 >> 172.21.15.97:0/1765838299 conn(0x556a485daef0 msgr2=0x556a485db2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:07:29.432 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.463+0000 7f9299ff8740 1 -- 172.21.15.97:0/1765838299 shutdown_connections 2024-03-27T11:07:29.432 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.463+0000 7f9299ff8740 1 -- 172.21.15.97:0/1765838299 wait complete. 2024-03-27T11:07:29.433 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.463+0000 7f9299ff8740 1 -- start start 2024-03-27T11:07:29.433 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.463+0000 7f9299ff8740 1 Processor -- start 2024-03-27T11:07:29.433 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.464+0000 7f9299ff8740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556a485df980 0x556a485e9d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:07:29.433 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.464+0000 7f9299ff8740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556a485e0370 0x556a485ea280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:07:29.433 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.464+0000 7f9299ff8740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556a485e54e0 0x556a485f1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:07:29.434 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.464+0000 7f9299ff8740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x556a485e8430 con 0x556a485df980 2024-03-27T11:07:29.434 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.464+0000 7f9299ff8740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x556a48500630 con 0x556a485e0370 2024-03-27T11:07:29.434 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.464+0000 7f9299ff8740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x556a485e85b0 con 0x556a485e54e0 2024-03-27T11:07:29.434 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.464+0000 7f9299ff8740 10 client.0 _ll_register_callbacks cb 0x556a48505c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T11:07:29.434 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.464+0000 7f928fe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556a485e0370 0x556a485ea280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:07:29.434 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.464+0000 7f9295400640 1 --2- 172.21.15.97:0/769776404 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f92780a3730 0x7f92780a5bf0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f9288034ef0 tx=0x7f92880333e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:07:29.434 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.464+0000 7f928fe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556a485e0370 0x556a485ea280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:47448/0 (socket says 172.21.15.97:47448) 2024-03-27T11:07:29.434 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.464+0000 7f9294a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556a485df980 0x556a485e9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:07:29.434 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.464+0000 7f928fe00640 1 -- 172.21.15.97:0/701352027 learned_addr learned my addr 172.21.15.97:0/701352027 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:07:29.434 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.465+0000 7f9295400640 1 --2- 172.21.15.97:0/701352027 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556a485e54e0 0x556a485f1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:07:29.435 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.465+0000 7f928fe00640 1 -- 172.21.15.97:0/701352027 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556a485e54e0 msgr2=0x556a485f1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:07:29.435 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.465+0000 7f928fe00640 1 --2- 172.21.15.97:0/701352027 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556a485e54e0 0x556a485f1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:29.436 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.465+0000 7f928fe00640 1 -- 172.21.15.97:0/701352027 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556a485df980 msgr2=0x556a485e9d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:07:29.436 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.465+0000 7f928fe00640 1 --2- 172.21.15.97:0/701352027 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556a485df980 0x556a485e9d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:29.436 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.465+0000 7f928fe00640 1 -- 172.21.15.97:0/701352027 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a4850c5f0 con 0x556a485e0370 2024-03-27T11:07:29.436 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.465+0000 7f9294a00640 1 --2- 172.21.15.97:0/701352027 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556a485df980 0x556a485e9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T11:07:29.436 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.465+0000 7f928fe00640 1 --2- 172.21.15.97:0/701352027 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556a485e0370 0x556a485ea280 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f927c0126d0 tx=0x7f927c012b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d460be22e5464974 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:07:29.436 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.465+0000 7f9276000640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-03-27T11:07:29.436 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.466+0000 7f9276000640 1 -- 172.21.15.97:0/701352027 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f927c004050 con 0x556a485e0370 2024-03-27T11:07:29.436 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.466+0000 7f9276000640 1 -- 172.21.15.97:0/701352027 <== mon.2 v2:172.21.15.114:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f927c0041f0 con 0x556a485e0370 2024-03-27T11:07:29.436 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.466+0000 7f9299ff8740 1 -- 172.21.15.97:0/701352027 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92840a4700 con 0x556a485e0370 2024-03-27T11:07:29.436 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.466+0000 7f9276000640 1 -- 172.21.15.97:0/701352027 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f927c0043f0 con 0x556a485e0370 2024-03-27T11:07:29.436 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.466+0000 7f9299ff8740 10 client.36605 fetch_fsmap learned FSMap version 34 2024-03-27T11:07:29.436 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.466+0000 7f9299ff8740 1 -- 172.21.15.97:0/701352027 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x556a48259210 con 0x556a485e0370 2024-03-27T11:07:29.436 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.467+0000 7f9276000640 1 -- 172.21.15.97:0/701352027 <== mon.2 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f92840a4700 con 0x556a485e0370 2024-03-27T11:07:29.437 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.467+0000 7f9276000640 1 -- 172.21.15.97:0/701352027 <== mon.2 v2:172.21.15.114:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f927c003b00 con 0x556a485e0370 2024-03-27T11:07:29.437 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.467+0000 7f9299ff8740 10 client.36605 fetch_fsmap finished waiting for FSMap version 34 2024-03-27T11:07:29.437 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.467+0000 7f9299ff8740 10 client.36605 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T11:07:29.437 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.467+0000 7f9299ff8740 20 client.36605 populate_metadata read hostname 'smithi097' 2024-03-27T11:07:29.437 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.467+0000 7f9299ff8740 1 --2- 172.21.15.97:0/701352027 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x556a485f5bd0 0x556a485f7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:07:29.437 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.467+0000 7f9299ff8740 4 client.36605 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T11:07:29.437 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.467+0000 7f9299ff8740 1 -- 172.21.15.97:0/701352027 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556a48552250 con 0x556a485f5bd0 2024-03-27T11:07:29.443 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.468+0000 7f9294a00640 1 --2- 172.21.15.97:0/701352027 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x556a485f5bd0 0x556a485f7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:07:29.444 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.475+0000 7f9294a00640 1 --2- 172.21.15.97:0/701352027 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x556a485f5bd0 0x556a485f7fc0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6bb74405eafb6d6 server_cookie=ab38c3230a460df0 in_seq=0 out_seq=0 2024-03-27T11:07:29.444 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.475+0000 7f9276000640 10 client.36605 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T11:07:29.445 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.476+0000 7f9276000640 1 -- 172.21.15.97:0/701352027 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x556a48552250 con 0x556a485f5bd0 2024-03-27T11:07:29.445 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.476+0000 7f9276000640 10 client.36605 handle_command_reply: tid=0 2024-03-27T11:07:29.445 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.476+0000 7f9299ff8740 1 client.36605 shutdown 2024-03-27T11:07:29.447 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.478+0000 7f9299ff8740 1 -- 172.21.15.97:0/701352027 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556a485e0370 msgr2=0x556a485ea280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:07:29.447 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.478+0000 7f9299ff8740 1 --2- 172.21.15.97:0/701352027 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556a485e0370 0x556a485ea280 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f927c0126d0 tx=0x7f927c012b90 comp rx=0 tx=0).stop 2024-03-27T11:07:29.448 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.478+0000 7f9299ff8740 1 -- 172.21.15.97:0/701352027 shutdown_connections 2024-03-27T11:07:29.448 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.478+0000 7f9299ff8740 1 --2- 172.21.15.97:0/701352027 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x556a485f5bd0 0x556a485f7fc0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:29.448 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.478+0000 7f9299ff8740 1 --2- 172.21.15.97:0/701352027 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556a485e54e0 0x556a485f1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:29.448 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.478+0000 7f9299ff8740 1 --2- 172.21.15.97:0/701352027 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556a485e0370 0x556a485ea280 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:29.448 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.478+0000 7f9299ff8740 1 --2- 172.21.15.97:0/701352027 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556a485df980 0x556a485e9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:29.448 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.478+0000 7f9299ff8740 1 -- 172.21.15.97:0/701352027 >> 172.21.15.97:0/701352027 conn(0x556a485daef0 msgr2=0x556a485da0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:07:29.448 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.478+0000 7f9276000640 0 client.36605 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T11:07:29.448 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.479+0000 7f9299ff8740 1 -- 172.21.15.97:0/701352027 shutdown_connections 2024-03-27T11:07:29.448 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.479+0000 7f9299ff8740 1 -- 172.21.15.97:0/701352027 wait complete. 2024-03-27T11:07:29.448 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.479+0000 7f9299ff8740 20 client.36605 trim_cache size 0 max 16384 2024-03-27T11:07:29.491 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.522+0000 7f9299ff8740 1 Processor -- start 2024-03-27T11:07:29.492 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.523+0000 7f9299ff8740 1 -- start start 2024-03-27T11:07:29.494 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.525+0000 7f9299ff8740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556a485df980 0x556a485d9860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:07:29.494 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.525+0000 7f9299ff8740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556a485e0370 0x556a485f1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:07:29.494 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.525+0000 7f9299ff8740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556a485e54e0 0x556a485d9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:07:29.494 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.525+0000 7f9299ff8740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f92901b7d20 con 0x556a485e54e0 2024-03-27T11:07:29.494 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.525+0000 7f9299ff8740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f92901b9860 con 0x556a485df980 2024-03-27T11:07:29.494 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.525+0000 7f9299ff8740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f92901b83f0 con 0x556a485e0370 2024-03-27T11:07:29.495 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.526+0000 7f928fe00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556a485e0370 0x556a485f1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:07:29.495 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.526+0000 7f9294a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556a485df980 0x556a485d9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:07:29.495 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.526+0000 7f9294a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556a485df980 0x556a485d9860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:47450/0 (socket says 172.21.15.97:47450) 2024-03-27T11:07:29.495 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.526+0000 7f9294a00640 1 -- 172.21.15.97:0/3913282154 learned_addr learned my addr 172.21.15.97:0/3913282154 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:07:29.495 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.527+0000 7f9295400640 1 --2- 172.21.15.97:0/3913282154 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556a485e54e0 0x556a485d9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:07:29.496 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.527+0000 7f9294a00640 1 -- 172.21.15.97:0/3913282154 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556a485e0370 msgr2=0x556a485f1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:07:29.496 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.527+0000 7f9294a00640 1 --2- 172.21.15.97:0/3913282154 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556a485e0370 0x556a485f1230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:29.496 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.527+0000 7f9294a00640 1 -- 172.21.15.97:0/3913282154 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556a485e54e0 msgr2=0x556a485d9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:07:29.496 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.527+0000 7f9294a00640 1 --2- 172.21.15.97:0/3913282154 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556a485e54e0 0x556a485d9da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:29.496 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.527+0000 7f9294a00640 1 -- 172.21.15.97:0/3913282154 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a48325620 con 0x556a485df980 2024-03-27T11:07:29.497 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.528+0000 7f9294a00640 1 --2- 172.21.15.97:0/3913282154 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556a485df980 0x556a485d9860 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9284098460 tx=0x7f928400df90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4973dc7d8589e5e server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:07:29.497 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.528+0000 7f9281200640 1 -- 172.21.15.97:0/3913282154 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92840a4700 con 0x556a485df980 2024-03-27T11:07:29.498 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.528+0000 7f9281200640 1 -- 172.21.15.97:0/3913282154 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f928409aee0 con 0x556a485df980 2024-03-27T11:07:29.498 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.529+0000 7f9299ff8740 1 -- 172.21.15.97:0/3913282154 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556a485df980 msgr2=0x556a485d9860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:07:29.498 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.529+0000 7f9299ff8740 1 --2- 172.21.15.97:0/3913282154 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556a485df980 0x556a485d9860 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9284098460 tx=0x7f928400df90 comp rx=0 tx=0).stop 2024-03-27T11:07:29.499 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.530+0000 7f9299ff8740 1 -- 172.21.15.97:0/3913282154 shutdown_connections 2024-03-27T11:07:29.499 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.530+0000 7f9299ff8740 1 --2- 172.21.15.97:0/3913282154 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556a485e54e0 0x556a485d9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:29.499 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.530+0000 7f9299ff8740 1 --2- 172.21.15.97:0/3913282154 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556a485e0370 0x556a485f1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:29.499 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.530+0000 7f9299ff8740 1 --2- 172.21.15.97:0/3913282154 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556a485df980 0x556a485d9860 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:29.499 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.530+0000 7f9299ff8740 1 -- 172.21.15.97:0/3913282154 >> 172.21.15.97:0/3913282154 conn(0x556a485daef0 msgr2=0x556a485e1730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:07:29.500 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.530+0000 7f9299ff8740 1 -- 172.21.15.97:0/3913282154 shutdown_connections 2024-03-27T11:07:29.500 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.530+0000 7f9299ff8740 1 -- 172.21.15.97:0/3913282154 wait complete. 2024-03-27T11:07:29.500 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.531+0000 7f9299ff8740 1 -- start start 2024-03-27T11:07:29.500 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.531+0000 7f9299ff8740 1 Processor -- start 2024-03-27T11:07:29.501 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.532+0000 7f9299ff8740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556a485df980 0x556a485ee3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:07:29.502 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.532+0000 7f9299ff8740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556a485e0370 0x556a485ee8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:07:29.502 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.532+0000 7f9299ff8740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556a485e54e0 0x556a485ef880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:07:29.502 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.532+0000 7f9299ff8740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f92901b9190 con 0x556a485e54e0 2024-03-27T11:07:29.502 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.532+0000 7f9299ff8740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f92901b8ac0 con 0x556a485df980 2024-03-27T11:07:29.502 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.532+0000 7f9299ff8740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f92901b5d00 con 0x556a485e0370 2024-03-27T11:07:29.502 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.532+0000 7f9299ff8740 10 client.0 _ll_register_callbacks cb 0x7f927c009d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T11:07:29.502 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.532+0000 7f9294a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556a485df980 0x556a485ee3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:07:29.502 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.532+0000 7f9294a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556a485df980 0x556a485ee3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:47456/0 (socket says 172.21.15.97:47456) 2024-03-27T11:07:29.502 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.532+0000 7f9294a00640 1 -- 172.21.15.97:0/1882585443 learned_addr learned my addr 172.21.15.97:0/1882585443 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:07:29.502 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.533+0000 7f9295400640 1 --2- 172.21.15.97:0/1882585443 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556a485e54e0 0x556a485ef880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:07:29.502 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.533+0000 7f928fe00640 1 --2- 172.21.15.97:0/1882585443 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556a485e0370 0x556a485ee8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:07:29.503 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.534+0000 7f9295400640 1 -- 172.21.15.97:0/1882585443 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556a485e0370 msgr2=0x556a485ee8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:07:29.503 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.534+0000 7f9295400640 1 --2- 172.21.15.97:0/1882585443 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556a485e0370 0x556a485ee8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:29.503 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.534+0000 7f9295400640 1 -- 172.21.15.97:0/1882585443 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556a485df980 msgr2=0x556a485ee3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:07:29.503 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.534+0000 7f9295400640 1 --2- 172.21.15.97:0/1882585443 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556a485df980 0x556a485ee3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:29.503 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.534+0000 7f9295400640 1 -- 172.21.15.97:0/1882585443 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a485efe70 con 0x556a485e54e0 2024-03-27T11:07:29.503 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.534+0000 7f9294a00640 1 --2- 172.21.15.97:0/1882585443 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556a485df980 0x556a485ee3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T11:07:29.503 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.534+0000 7f928fe00640 1 --2- 172.21.15.97:0/1882585443 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556a485e0370 0x556a485ee8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T11:07:29.504 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.535+0000 7f9295400640 1 --2- 172.21.15.97:0/1882585443 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556a485e54e0 0x556a485ef880 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f9288004af0 tx=0x7f9288009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67ecd539bb93c1f7 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:07:29.504 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.535+0000 7f9276000640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T11:07:29.504 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.535+0000 7f9276000640 1 -- 172.21.15.97:0/1882585443 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9288004250 con 0x556a485e54e0 2024-03-27T11:07:29.504 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.535+0000 7f9276000640 1 -- 172.21.15.97:0/1882585443 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f92880043f0 con 0x556a485e54e0 2024-03-27T11:07:29.504 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.536+0000 7f9299ff8740 1 -- 172.21.15.97:0/1882585443 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f928409aee0 con 0x556a485e54e0 2024-03-27T11:07:29.505 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.536+0000 7f9276000640 1 -- 172.21.15.97:0/1882585443 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f9288004630 con 0x556a485e54e0 2024-03-27T11:07:29.505 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.536+0000 7f9299ff8740 10 client.16917 fetch_fsmap learned FSMap version 34 2024-03-27T11:07:29.505 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.536+0000 7f9299ff8740 1 -- 172.21.15.97:0/1882585443 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x556a485f11f0 con 0x556a485e54e0 2024-03-27T11:07:29.506 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.537+0000 7f9276000640 1 -- 172.21.15.97:0/1882585443 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f928409aee0 con 0x556a485e54e0 2024-03-27T11:07:29.506 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.537+0000 7f9276000640 1 -- 172.21.15.97:0/1882585443 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f9288041460 con 0x556a485e54e0 2024-03-27T11:07:29.506 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.537+0000 7f9299ff8740 10 client.16917 fetch_fsmap finished waiting for FSMap version 34 2024-03-27T11:07:29.506 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.537+0000 7f9299ff8740 10 client.16917 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T11:07:29.506 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.537+0000 7f9299ff8740 20 client.16917 populate_metadata read hostname 'smithi097' 2024-03-27T11:07:29.507 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.537+0000 7f9299ff8740 1 --2- 172.21.15.97:0/1882585443 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x556a4865c240 0x556a4865e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:07:29.507 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.537+0000 7f9299ff8740 4 client.16917 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-27T11:07:29.507 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.537+0000 7f9299ff8740 1 -- 172.21.15.97:0/1882585443 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556a4854c5a0 con 0x556a4865c240 2024-03-27T11:07:29.507 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.538+0000 7f9294a00640 1 --2- 172.21.15.97:0/1882585443 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x556a4865c240 0x556a4865e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:07:29.507 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.539+0000 7f9294a00640 1 --2- 172.21.15.97:0/1882585443 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x556a4865c240 0x556a4865e630 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=806be3a354ae8563 server_cookie=9803a3aa4dcff642 in_seq=0 out_seq=0 2024-03-27T11:07:29.507 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.539+0000 7f9276000640 10 client.16917 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T11:07:29.520 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.552+0000 7f9276000640 1 -- 172.21.15.97:0/1882585443 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556a4854c5a0 con 0x556a4865c240 2024-03-27T11:07:29.521 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.552+0000 7f9276000640 10 client.16917 handle_command_reply: tid=0 2024-03-27T11:07:29.521 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.552+0000 7f9299ff8740 1 client.16917 shutdown 2024-03-27T11:07:29.521 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.552+0000 7f9299ff8740 1 -- 172.21.15.97:0/1882585443 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556a485e54e0 msgr2=0x556a485ef880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:07:29.521 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.552+0000 7f9299ff8740 1 --2- 172.21.15.97:0/1882585443 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556a485e54e0 0x556a485ef880 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f9288004af0 tx=0x7f9288009500 comp rx=0 tx=0).stop 2024-03-27T11:07:29.522 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.552+0000 7f9299ff8740 1 -- 172.21.15.97:0/1882585443 shutdown_connections 2024-03-27T11:07:29.522 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.552+0000 7f9299ff8740 1 --2- 172.21.15.97:0/1882585443 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x556a4865c240 0x556a4865e630 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:29.522 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.552+0000 7f9299ff8740 1 --2- 172.21.15.97:0/1882585443 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556a485e54e0 0x556a485ef880 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:29.522 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.552+0000 7f9299ff8740 1 --2- 172.21.15.97:0/1882585443 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556a485e0370 0x556a485ee8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:29.522 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.552+0000 7f9299ff8740 1 --2- 172.21.15.97:0/1882585443 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556a485df980 0x556a485ee3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:29.522 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.552+0000 7f9299ff8740 1 -- 172.21.15.97:0/1882585443 >> 172.21.15.97:0/1882585443 conn(0x556a485daef0 msgr2=0x556a485de150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:07:29.522 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.552+0000 7f9276000640 0 client.16917 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T11:07:29.522 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.553+0000 7f9299ff8740 1 -- 172.21.15.97:0/1882585443 shutdown_connections 2024-03-27T11:07:29.522 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.553+0000 7f9299ff8740 1 -- 172.21.15.97:0/1882585443 wait complete. 2024-03-27T11:07:29.522 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.553+0000 7f9299ff8740 20 client.16917 trim_cache size 0 max 16384 2024-03-27T11:07:29.523 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-27T11:07:29.523 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-27T11:07:29.523 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-27T11:07:29.523 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-27T11:07:29.524 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.554+0000 7f9297400640 1 -- 172.21.15.97:0/769776404 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f92780a3730 msgr2=0x7f92780a5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:07:29.524 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.554+0000 7f9297400640 1 --2- 172.21.15.97:0/769776404 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f92780a3730 0x7f92780a5bf0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f9288034ef0 tx=0x7f92880333e0 comp rx=0 tx=0).stop 2024-03-27T11:07:29.524 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.554+0000 7f9297400640 1 -- 172.21.15.97:0/769776404 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f92900745b0 msgr2=0x7f92900703e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:07:29.524 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.554+0000 7f9297400640 1 --2- 172.21.15.97:0/769776404 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f92900745b0 0x7f92900703e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f928400d950 tx=0x7f928400dd50 comp rx=0 tx=0).stop 2024-03-27T11:07:29.524 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.554+0000 7f9297400640 1 -- 172.21.15.97:0/769776404 shutdown_connections 2024-03-27T11:07:29.524 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.554+0000 7f9297400640 1 --2- 172.21.15.97:0/769776404 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f929010e170 0x7f92901ae440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:29.524 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.554+0000 7f9297400640 1 --2- 172.21.15.97:0/769776404 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f92780a3730 0x7f92780a5bf0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:29.524 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.554+0000 7f9297400640 1 --2- 172.21.15.97:0/769776404 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9290075b90 0x7f9290070920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:29.524 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.554+0000 7f9297400640 1 --2- 172.21.15.97:0/769776404 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f92900745b0 0x7f92900703e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:29.524 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.554+0000 7f9297400640 1 -- 172.21.15.97:0/769776404 >> 172.21.15.97:0/769776404 conn(0x7f92900fd130 msgr2=0x7f9290101400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:07:29.524 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.554+0000 7f9297400640 1 -- 172.21.15.97:0/769776404 shutdown_connections 2024-03-27T11:07:29.524 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:29.555+0000 7f9297400640 1 -- 172.21.15.97:0/769776404 wait complete. 2024-03-27T11:07:29.764 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:07:29 smithi114 ceph-mon[35338]: pgmap v766: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 48 MiB/s wr, 841 op/s 2024-03-27T11:07:29.888 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:07:29 smithi097 ceph-mon[31859]: pgmap v766: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 48 MiB/s wr, 841 op/s 2024-03-27T11:07:29.924 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:07:29 smithi177 ceph-mon[35294]: pgmap v766: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 48 MiB/s wr, 841 op/s 2024-03-27T11:07:31.764 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:07:31 smithi114 ceph-mon[35338]: pgmap v767: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 51 MiB/s wr, 900 op/s 2024-03-27T11:07:31.888 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:07:31 smithi097 ceph-mon[31859]: pgmap v767: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 51 MiB/s wr, 900 op/s 2024-03-27T11:07:31.923 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:07:31 smithi177 ceph-mon[35294]: pgmap v767: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 51 MiB/s wr, 900 op/s 2024-03-27T11:07:33.764 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:07:33 smithi114 ceph-mon[35338]: pgmap v768: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 47 MiB/s wr, 814 op/s 2024-03-27T11:07:33.890 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:07:33 smithi097 ceph-mon[31859]: pgmap v768: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 47 MiB/s wr, 814 op/s 2024-03-27T11:07:33.924 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:07:33 smithi177 ceph-mon[35294]: pgmap v768: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 47 MiB/s wr, 814 op/s 2024-03-27T11:07:35.764 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:07:35 smithi114 ceph-mon[35338]: pgmap v769: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 47 MiB/s wr, 826 op/s 2024-03-27T11:07:35.888 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:07:35 smithi097 ceph-mon[31859]: pgmap v769: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 47 MiB/s wr, 826 op/s 2024-03-27T11:07:35.923 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:07:35 smithi177 ceph-mon[35294]: pgmap v769: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 47 MiB/s wr, 826 op/s 2024-03-27T11:07:37.764 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:07:37 smithi114 ceph-mon[35338]: pgmap v770: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 46 MiB/s wr, 800 op/s 2024-03-27T11:07:37.888 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:07:37 smithi097 ceph-mon[31859]: pgmap v770: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 46 MiB/s wr, 800 op/s 2024-03-27T11:07:37.924 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:07:37 smithi177 ceph-mon[35294]: pgmap v770: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 46 MiB/s wr, 800 op/s 2024-03-27T11:07:39.764 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:07:39 smithi114 ceph-mon[35338]: pgmap v771: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 48 MiB/s wr, 832 op/s 2024-03-27T11:07:39.888 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:07:39 smithi097 ceph-mon[31859]: pgmap v771: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 48 MiB/s wr, 832 op/s 2024-03-27T11:07:39.923 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:07:39 smithi177 ceph-mon[35294]: pgmap v771: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 48 MiB/s wr, 832 op/s 2024-03-27T11:07:41.888 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:07:41 smithi097 ceph-mon[31859]: pgmap v772: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 51 MiB/s wr, 898 op/s 2024-03-27T11:07:41.923 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:07:41 smithi177 ceph-mon[35294]: pgmap v772: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 51 MiB/s wr, 898 op/s 2024-03-27T11:07:42.014 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:07:41 smithi114 ceph-mon[35338]: pgmap v772: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 51 MiB/s wr, 898 op/s 2024-03-27T11:07:42.757 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.787+0000 7fac32200640 1 -- 172.21.15.97:0/3600754126 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac2c105720 msgr2=0x7fac2c105ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:07:42.757 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.787+0000 7fac32200640 1 --2- 172.21.15.97:0/3600754126 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac2c105720 0x7fac2c105ba0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fac24009be0 tx=0x7fac2402bef0 comp rx=0 tx=0).stop 2024-03-27T11:07:42.757 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.787+0000 7fac32200640 1 -- 172.21.15.97:0/3600754126 shutdown_connections 2024-03-27T11:07:42.757 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.787+0000 7fac32200640 1 --2- 172.21.15.97:0/3600754126 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac2c105720 0x7fac2c105ba0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:42.758 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.787+0000 7fac32200640 1 --2- 172.21.15.97:0/3600754126 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fac2c102d70 0x7fac2c1051e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:42.758 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.787+0000 7fac32200640 1 --2- 172.21.15.97:0/3600754126 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fac2c100400 0x7fac2c102830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:42.758 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.788+0000 7fac32200640 1 -- 172.21.15.97:0/3600754126 >> 172.21.15.97:0/3600754126 conn(0x7fac2c0fc000 msgr2=0x7fac2c0fe440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:07:42.760 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.788+0000 7fac32200640 1 -- 172.21.15.97:0/3600754126 shutdown_connections 2024-03-27T11:07:42.761 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.788+0000 7fac32200640 1 -- 172.21.15.97:0/3600754126 wait complete. 2024-03-27T11:07:42.761 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.790+0000 7fac32200640 1 Processor -- start 2024-03-27T11:07:42.761 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.790+0000 7fac32200640 1 -- start start 2024-03-27T11:07:42.761 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.791+0000 7fac32200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fac2c100400 0x7fac2c19e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:07:42.761 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.791+0000 7fac32200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fac2c102d70 0x7fac2c19e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:07:42.761 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.791+0000 7fac32200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac2c105720 0x7fac2c1a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:07:42.761 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.791+0000 7fac32200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fac2c113ce0 con 0x7fac2c105720 2024-03-27T11:07:42.761 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.791+0000 7fac32200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fac2c113b60 con 0x7fac2c102d70 2024-03-27T11:07:42.761 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.791+0000 7fac32200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fac2c113e60 con 0x7fac2c100400 2024-03-27T11:07:42.762 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.791+0000 7fac2aa00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fac2c102d70 0x7fac2c19e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:07:42.762 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.791+0000 7fac2b400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fac2c100400 0x7fac2c19e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:07:42.762 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.791+0000 7fac2b400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fac2c100400 0x7fac2c19e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.97:48798/0 (socket says 172.21.15.97:48798) 2024-03-27T11:07:42.763 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.791+0000 7fac2b400640 1 -- 172.21.15.97:0/498405225 learned_addr learned my addr 172.21.15.97:0/498405225 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:07:42.763 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.791+0000 7fac2aa00640 1 --2- 172.21.15.97:0/498405225 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fac2c102d70 0x7fac2c19e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:51040/0 (socket says 172.21.15.97:51040) 2024-03-27T11:07:42.763 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.792+0000 7fac2aa00640 1 -- 172.21.15.97:0/498405225 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fac2c100400 msgr2=0x7fac2c19e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:07:42.763 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.792+0000 7fac2aa00640 1 --2- 172.21.15.97:0/498405225 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fac2c100400 0x7fac2c19e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:42.763 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.792+0000 7fac2aa00640 1 -- 172.21.15.97:0/498405225 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac2c105720 msgr2=0x7fac2c1a5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:07:42.763 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.792+0000 7fac2aa00640 1 --2- 172.21.15.97:0/498405225 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac2c105720 0x7fac2c1a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:42.763 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.792+0000 7fac2aa00640 1 -- 172.21.15.97:0/498405225 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac2c1a5e80 con 0x7fac2c102d70 2024-03-27T11:07:42.763 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.792+0000 7fac2aa00640 1 --2- 172.21.15.97:0/498405225 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fac2c102d70 0x7fac2c19e7a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fac1800e790 tx=0x7fac1800ec50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:07:42.763 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.792+0000 7fac23e00640 1 -- 172.21.15.97:0/498405225 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac1800eea0 con 0x7fac2c102d70 2024-03-27T11:07:42.763 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.793+0000 7fac32200640 1 -- 172.21.15.97:0/498405225 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac2c1a6170 con 0x7fac2c102d70 2024-03-27T11:07:42.763 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.793+0000 7fac32200640 1 -- 172.21.15.97:0/498405225 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac2c1a6710 con 0x7fac2c102d70 2024-03-27T11:07:42.764 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.793+0000 7fac23e00640 1 -- 172.21.15.97:0/498405225 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fac1800ccc0 con 0x7fac2c102d70 2024-03-27T11:07:42.764 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.793+0000 7fac23e00640 1 -- 172.21.15.97:0/498405225 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac18018680 con 0x7fac2c102d70 2024-03-27T11:07:42.769 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.799+0000 7fac23e00640 1 -- 172.21.15.97:0/498405225 <== mon.2 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7fac180188e0 con 0x7fac2c102d70 2024-03-27T11:07:42.771 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.801+0000 7fac34df1740 1 Processor -- start 2024-03-27T11:07:42.771 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.802+0000 7fac34df1740 1 -- start start 2024-03-27T11:07:42.772 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.802+0000 7fac34df1740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f8c6865980 0x55f8c6865da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:07:42.772 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.802+0000 7fac34df1740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f8c6866370 0x55f8c686ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:07:42.772 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.802+0000 7fac34df1740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f8c686b4e0 0x55f8c686d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:07:42.772 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.802+0000 7fac34df1740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55f8c67cad10 con 0x55f8c6866370 2024-03-27T11:07:42.772 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.802+0000 7fac34df1740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55f8c6773260 con 0x55f8c6865980 2024-03-27T11:07:42.772 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.802+0000 7fac34df1740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55f8c67c6cc0 con 0x55f8c686b4e0 2024-03-27T11:07:42.774 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.803+0000 7fac23e00640 1 --2- 172.21.15.97:0/498405225 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fac100a3780 0x7fac100a5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:07:42.774 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.803+0000 7fac2aa00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f8c686b4e0 0x55f8c686d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:07:42.774 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.803+0000 7fac2aa00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f8c686b4e0 0x55f8c686d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.97:48812/0 (socket says 172.21.15.97:48812) 2024-03-27T11:07:42.774 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.803+0000 7fac2aa00640 1 -- 172.21.15.97:0/2207812505 learned_addr learned my addr 172.21.15.97:0/2207812505 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:07:42.774 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.803+0000 7fac2be00640 1 --2- 172.21.15.97:0/2207812505 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f8c6865980 0x55f8c6865da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:07:42.774 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.803+0000 7fac23e00640 1 -- 172.21.15.97:0/498405225 <== mon.2 v2:172.21.15.114:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fac18010630 con 0x7fac2c102d70 2024-03-27T11:07:42.774 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.803+0000 7fac2b400640 1 --2- 172.21.15.97:0/2207812505 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f8c6866370 0x55f8c686ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:07:42.774 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.803+0000 7fac2aa00640 1 -- 172.21.15.97:0/2207812505 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f8c6865980 msgr2=0x55f8c6865da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:07:42.774 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.803+0000 7fac2aa00640 1 --2- 172.21.15.97:0/2207812505 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f8c6865980 0x55f8c6865da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:42.774 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.803+0000 7fac2aa00640 1 -- 172.21.15.97:0/2207812505 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f8c6866370 msgr2=0x55f8c686ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:07:42.775 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.803+0000 7fac2aa00640 1 --2- 172.21.15.97:0/2207812505 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f8c6866370 0x55f8c686ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:42.775 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.803+0000 7fac2aa00640 1 -- 172.21.15.97:0/2207812505 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8c676aaa0 con 0x55f8c686b4e0 2024-03-27T11:07:42.775 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.804+0000 7fac2aa00640 1 --2- 172.21.15.97:0/2207812505 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f8c686b4e0 0x55f8c686d970 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fac1809d070 tx=0x7fac18013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e60eb1fd8557e9e5 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:07:42.775 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.804+0000 7fac17e00640 1 -- 172.21.15.97:0/2207812505 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac18026040 con 0x55f8c686b4e0 2024-03-27T11:07:42.775 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.804+0000 7fac17e00640 1 -- 172.21.15.97:0/2207812505 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fac1809da70 con 0x55f8c686b4e0 2024-03-27T11:07:42.775 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.804+0000 7fac17e00640 1 -- 172.21.15.97:0/2207812505 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac18018bd0 con 0x55f8c686b4e0 2024-03-27T11:07:42.775 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.805+0000 7fac2be00640 1 --2- 172.21.15.97:0/498405225 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fac100a3780 0x7fac100a5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:07:42.775 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.805+0000 7fac34df1740 1 -- 172.21.15.97:0/2207812505 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f8c686b4e0 msgr2=0x55f8c686d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:07:42.776 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.805+0000 7fac34df1740 1 --2- 172.21.15.97:0/2207812505 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f8c686b4e0 0x55f8c686d970 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fac1809d070 tx=0x7fac18013000 comp rx=0 tx=0).stop 2024-03-27T11:07:42.776 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.806+0000 7fac34df1740 1 -- 172.21.15.97:0/2207812505 shutdown_connections 2024-03-27T11:07:42.776 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.806+0000 7fac34df1740 1 --2- 172.21.15.97:0/2207812505 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f8c686b4e0 0x55f8c686d970 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:42.776 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.806+0000 7fac34df1740 1 --2- 172.21.15.97:0/2207812505 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f8c6866370 0x55f8c686ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:42.776 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.806+0000 7fac34df1740 1 --2- 172.21.15.97:0/2207812505 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f8c6865980 0x55f8c6865da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:42.776 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.806+0000 7fac34df1740 1 -- 172.21.15.97:0/2207812505 >> 172.21.15.97:0/2207812505 conn(0x55f8c6860ef0 msgr2=0x55f8c68612f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:07:42.776 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.807+0000 7fac2be00640 1 --2- 172.21.15.97:0/498405225 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fac100a3780 0x7fac100a5c40 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x55f8c6866bd0 tx=0x7fac24005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:07:42.777 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.807+0000 7fac34df1740 1 -- 172.21.15.97:0/2207812505 shutdown_connections 2024-03-27T11:07:42.777 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.807+0000 7fac34df1740 1 -- 172.21.15.97:0/2207812505 wait complete. 2024-03-27T11:07:42.777 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.807+0000 7fac34df1740 1 -- start start 2024-03-27T11:07:42.777 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.807+0000 7fac34df1740 1 Processor -- start 2024-03-27T11:07:42.778 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.808+0000 7fac34df1740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f8c6865980 0x55f8c686fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:07:42.778 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.808+0000 7fac34df1740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f8c6866370 0x55f8c6870250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:07:42.778 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.808+0000 7fac34df1740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f8c686b4e0 0x55f8c6877290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:07:42.778 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.808+0000 7fac34df1740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55f8c686e430 con 0x55f8c686b4e0 2024-03-27T11:07:42.778 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.808+0000 7fac34df1740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55f8c6786630 con 0x55f8c6866370 2024-03-27T11:07:42.778 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.808+0000 7fac34df1740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55f8c686e5b0 con 0x55f8c6865980 2024-03-27T11:07:42.778 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.808+0000 7fac34df1740 10 client.0 _ll_register_callbacks cb 0x55f8c678bc50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T11:07:42.779 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.808+0000 7fac2b400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f8c6865980 0x55f8c686fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:07:42.779 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.808+0000 7fac2b400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f8c6865980 0x55f8c686fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.97:48826/0 (socket says 172.21.15.97:48826) 2024-03-27T11:07:42.779 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.808+0000 7fac2b400640 1 -- 172.21.15.97:0/4123023524 learned_addr learned my addr 172.21.15.97:0/4123023524 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:07:42.779 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.809+0000 7fac2be00640 1 --2- 172.21.15.97:0/4123023524 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f8c686b4e0 0x55f8c6877290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:07:42.779 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.809+0000 7fac2aa00640 1 --2- 172.21.15.97:0/4123023524 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f8c6866370 0x55f8c6870250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:07:42.779 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.809+0000 7fac2aa00640 1 -- 172.21.15.97:0/4123023524 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f8c6865980 msgr2=0x55f8c686fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:07:42.779 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.809+0000 7fac2aa00640 1 --2- 172.21.15.97:0/4123023524 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f8c6865980 0x55f8c686fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:42.779 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.809+0000 7fac2aa00640 1 -- 172.21.15.97:0/4123023524 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f8c686b4e0 msgr2=0x55f8c6877290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:07:42.780 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.809+0000 7fac2aa00640 1 --2- 172.21.15.97:0/4123023524 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f8c686b4e0 0x55f8c6877290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:42.780 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.810+0000 7fac2aa00640 1 -- 172.21.15.97:0/4123023524 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8c67925f0 con 0x55f8c6866370 2024-03-27T11:07:42.780 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.810+0000 7fac2be00640 1 --2- 172.21.15.97:0/4123023524 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f8c686b4e0 0x55f8c6877290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T11:07:42.780 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.810+0000 7fac2aa00640 1 --2- 172.21.15.97:0/4123023524 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f8c6866370 0x55f8c6870250 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fac18010bf0 tx=0x7fac18099c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f183292a6dfe6fa7 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:07:42.780 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.810+0000 7fac14c00640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-03-27T11:07:42.781 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.811+0000 7fac14c00640 1 -- 172.21.15.97:0/4123023524 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac1809c9d0 con 0x55f8c6866370 2024-03-27T11:07:42.781 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.811+0000 7fac14c00640 1 -- 172.21.15.97:0/4123023524 <== mon.2 v2:172.21.15.114:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fac180990c0 con 0x55f8c6866370 2024-03-27T11:07:42.781 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.811+0000 7fac34df1740 1 -- 172.21.15.97:0/4123023524 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac1809da70 con 0x55f8c6866370 2024-03-27T11:07:42.781 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.812+0000 7fac14c00640 1 -- 172.21.15.97:0/4123023524 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fac18099320 con 0x55f8c6866370 2024-03-27T11:07:42.781 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.812+0000 7fac34df1740 10 client.36620 fetch_fsmap learned FSMap version 34 2024-03-27T11:07:42.781 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.812+0000 7fac34df1740 1 -- 172.21.15.97:0/4123023524 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55f8c64df210 con 0x55f8c6866370 2024-03-27T11:07:42.783 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.813+0000 7fac14c00640 1 -- 172.21.15.97:0/4123023524 <== mon.2 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fac1809da70 con 0x55f8c6866370 2024-03-27T11:07:42.783 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.813+0000 7fac14c00640 1 -- 172.21.15.97:0/4123023524 <== mon.2 v2:172.21.15.114:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7fac180a1230 con 0x55f8c6866370 2024-03-27T11:07:42.785 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.815+0000 7fac34df1740 10 client.36620 fetch_fsmap finished waiting for FSMap version 34 2024-03-27T11:07:42.785 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.815+0000 7fac34df1740 10 client.36620 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T11:07:42.786 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.815+0000 7fac34df1740 20 client.36620 populate_metadata read hostname 'smithi097' 2024-03-27T11:07:42.786 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.815+0000 7fac34df1740 1 --2- 172.21.15.97:0/4123023524 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55f8c687b780 0x55f8c687db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:07:42.786 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.815+0000 7fac34df1740 4 client.36620 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T11:07:42.786 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.815+0000 7fac34df1740 1 -- 172.21.15.97:0/4123023524 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f8c67d8250 con 0x55f8c687b780 2024-03-27T11:07:42.786 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.816+0000 7fac2b400640 1 --2- 172.21.15.97:0/4123023524 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55f8c687b780 0x55f8c687db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:07:42.786 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.817+0000 7fac2b400640 1 --2- 172.21.15.97:0/4123023524 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55f8c687b780 0x55f8c687db70 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c928cebc0799552c server_cookie=29257aa412980ce9 in_seq=0 out_seq=0 2024-03-27T11:07:42.787 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.817+0000 7fac14c00640 10 client.36620 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T11:07:42.787 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.818+0000 7fac14c00640 1 -- 172.21.15.97:0/4123023524 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55f8c67d8250 con 0x55f8c687b780 2024-03-27T11:07:42.788 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.818+0000 7fac14c00640 10 client.36620 handle_command_reply: tid=0 2024-03-27T11:07:42.788 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.818+0000 7fac34df1740 1 client.36620 shutdown 2024-03-27T11:07:42.788 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.818+0000 7fac34df1740 1 -- 172.21.15.97:0/4123023524 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f8c6866370 msgr2=0x55f8c6870250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:07:42.788 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.818+0000 7fac34df1740 1 --2- 172.21.15.97:0/4123023524 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f8c6866370 0x55f8c6870250 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fac18010bf0 tx=0x7fac18099c80 comp rx=0 tx=0).stop 2024-03-27T11:07:42.789 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.820+0000 7fac34df1740 1 -- 172.21.15.97:0/4123023524 shutdown_connections 2024-03-27T11:07:42.790 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.820+0000 7fac34df1740 1 --2- 172.21.15.97:0/4123023524 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55f8c687b780 0x55f8c687db70 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:42.790 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.820+0000 7fac34df1740 1 --2- 172.21.15.97:0/4123023524 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f8c686b4e0 0x55f8c6877290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:42.790 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.820+0000 7fac34df1740 1 --2- 172.21.15.97:0/4123023524 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f8c6866370 0x55f8c6870250 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:42.790 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.820+0000 7fac34df1740 1 --2- 172.21.15.97:0/4123023524 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f8c6865980 0x55f8c686fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:42.790 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.820+0000 7fac34df1740 1 -- 172.21.15.97:0/4123023524 >> 172.21.15.97:0/4123023524 conn(0x55f8c6860ef0 msgr2=0x55f8c68600f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:07:42.790 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.820+0000 7fac14c00640 0 client.36620 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T11:07:42.791 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.820+0000 7fac34df1740 1 -- 172.21.15.97:0/4123023524 shutdown_connections 2024-03-27T11:07:42.791 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.820+0000 7fac34df1740 1 -- 172.21.15.97:0/4123023524 wait complete. 2024-03-27T11:07:42.791 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.821+0000 7fac34df1740 20 client.36620 trim_cache size 0 max 16384 2024-03-27T11:07:42.820 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.851+0000 7fac34df1740 1 Processor -- start 2024-03-27T11:07:42.820 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.851+0000 7fac34df1740 1 -- start start 2024-03-27T11:07:42.821 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.851+0000 7fac34df1740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f8c6865980 0x55f8c686f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:07:42.821 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.851+0000 7fac34df1740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f8c6866370 0x55f8c6877250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:07:42.821 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.851+0000 7fac34df1740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f8c686b4e0 0x55f8c6864150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:07:42.821 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.851+0000 7fac34df1740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fac0c00e550 con 0x55f8c686b4e0 2024-03-27T11:07:42.821 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.851+0000 7fac34df1740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fac0c010090 con 0x55f8c6866370 2024-03-27T11:07:42.821 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.851+0000 7fac34df1740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fac0c00ec20 con 0x55f8c6865980 2024-03-27T11:07:42.822 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.852+0000 7fac2aa00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f8c6866370 0x55f8c6877250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:07:42.822 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.852+0000 7fac2aa00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f8c6866370 0x55f8c6877250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:51074/0 (socket says 172.21.15.97:51074) 2024-03-27T11:07:42.823 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.852+0000 7fac2aa00640 1 -- 172.21.15.97:0/2153722835 learned_addr learned my addr 172.21.15.97:0/2153722835 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:07:42.823 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.852+0000 7fac2b400640 1 --2- 172.21.15.97:0/2153722835 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f8c6865980 0x55f8c686f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:07:42.823 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.852+0000 7fac2aa00640 1 -- 172.21.15.97:0/2153722835 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f8c6865980 msgr2=0x55f8c686f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:07:42.823 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.852+0000 7fac2aa00640 1 --2- 172.21.15.97:0/2153722835 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f8c6865980 0x55f8c686f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:42.823 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.852+0000 7fac2aa00640 1 -- 172.21.15.97:0/2153722835 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f8c686b4e0 msgr2=0x55f8c6864150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:07:42.823 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.852+0000 7fac2aa00640 1 --2- 172.21.15.97:0/2153722835 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f8c686b4e0 0x55f8c6864150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:42.823 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.852+0000 7fac2aa00640 1 -- 172.21.15.97:0/2153722835 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8c65ab620 con 0x55f8c6866370 2024-03-27T11:07:42.823 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.852+0000 7fac2b400640 1 --2- 172.21.15.97:0/2153722835 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f8c6865980 0x55f8c686f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T11:07:42.823 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.852+0000 7fac2aa00640 1 --2- 172.21.15.97:0/2153722835 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f8c6866370 0x55f8c6877250 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fac1809b620 tx=0x7fac1809ccd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8cad74d36d5052 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:07:42.824 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.853+0000 7fac17400640 1 -- 172.21.15.97:0/2153722835 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac1809ceb0 con 0x55f8c6866370 2024-03-27T11:07:42.824 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.853+0000 7fac17400640 1 -- 172.21.15.97:0/2153722835 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fac180bd4a0 con 0x55f8c6866370 2024-03-27T11:07:42.824 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.853+0000 7fac17400640 1 -- 172.21.15.97:0/2153722835 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac180bd720 con 0x55f8c6866370 2024-03-27T11:07:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.854+0000 7fac34df1740 1 -- 172.21.15.97:0/2153722835 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f8c6866370 msgr2=0x55f8c6877250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:07:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.854+0000 7fac34df1740 1 --2- 172.21.15.97:0/2153722835 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f8c6866370 0x55f8c6877250 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fac1809b620 tx=0x7fac1809ccd0 comp rx=0 tx=0).stop 2024-03-27T11:07:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.854+0000 7fac34df1740 1 -- 172.21.15.97:0/2153722835 shutdown_connections 2024-03-27T11:07:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.854+0000 7fac34df1740 1 --2- 172.21.15.97:0/2153722835 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f8c686b4e0 0x55f8c6864150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.854+0000 7fac34df1740 1 --2- 172.21.15.97:0/2153722835 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f8c6866370 0x55f8c6877250 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.854+0000 7fac34df1740 1 --2- 172.21.15.97:0/2153722835 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f8c6865980 0x55f8c686f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.854+0000 7fac34df1740 1 -- 172.21.15.97:0/2153722835 >> 172.21.15.97:0/2153722835 conn(0x55f8c6860ef0 msgr2=0x55f8c686efe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:07:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.854+0000 7fac34df1740 1 -- 172.21.15.97:0/2153722835 shutdown_connections 2024-03-27T11:07:42.826 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.854+0000 7fac34df1740 1 -- 172.21.15.97:0/2153722835 wait complete. 2024-03-27T11:07:42.826 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.855+0000 7fac34df1740 1 -- start start 2024-03-27T11:07:42.826 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.855+0000 7fac34df1740 1 Processor -- start 2024-03-27T11:07:42.826 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.855+0000 7fac34df1740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f8c6865980 0x55f8c6874190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:07:42.826 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.855+0000 7fac34df1740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f8c6866370 0x55f8c68746d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:07:42.826 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.855+0000 7fac34df1740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f8c686b4e0 0x55f8c68df8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:07:42.826 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.855+0000 7fac34df1740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fac0c00f9c0 con 0x55f8c686b4e0 2024-03-27T11:07:42.826 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.855+0000 7fac34df1740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fac0c00f2f0 con 0x55f8c6865980 2024-03-27T11:07:42.826 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.855+0000 7fac34df1740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fac0c00c530 con 0x55f8c6866370 2024-03-27T11:07:42.826 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.855+0000 7fac34df1740 10 client.0 _ll_register_callbacks cb 0x7fac1800ce70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T11:07:42.827 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.856+0000 7fac2b400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f8c6865980 0x55f8c6874190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:07:42.827 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.856+0000 7fac2aa00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f8c6866370 0x55f8c68746d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:07:42.828 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.856+0000 7fac2b400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f8c6865980 0x55f8c6874190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:51084/0 (socket says 172.21.15.97:51084) 2024-03-27T11:07:42.828 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.856+0000 7fac2b400640 1 -- 172.21.15.97:0/2941490244 learned_addr learned my addr 172.21.15.97:0/2941490244 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:07:42.828 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.856+0000 7fac2be00640 1 --2- 172.21.15.97:0/2941490244 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f8c686b4e0 0x55f8c68df8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:07:42.828 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.856+0000 7fac2aa00640 1 -- 172.21.15.97:0/2941490244 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f8c6865980 msgr2=0x55f8c6874190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:07:42.828 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.856+0000 7fac2aa00640 1 --2- 172.21.15.97:0/2941490244 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f8c6865980 0x55f8c6874190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:42.828 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.856+0000 7fac2aa00640 1 -- 172.21.15.97:0/2941490244 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f8c686b4e0 msgr2=0x55f8c68df8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:07:42.828 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.856+0000 7fac2aa00640 1 --2- 172.21.15.97:0/2941490244 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f8c686b4e0 0x55f8c68df8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:42.828 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.856+0000 7fac2aa00640 1 -- 172.21.15.97:0/2941490244 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8c68dfe20 con 0x55f8c6866370 2024-03-27T11:07:42.828 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.856+0000 7fac2be00640 1 --2- 172.21.15.97:0/2941490244 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f8c686b4e0 0x55f8c68df8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T11:07:42.828 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.857+0000 7fac2aa00640 1 --2- 172.21.15.97:0/2941490244 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f8c6866370 0x55f8c68746d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fac1809b620 tx=0x7fac18010ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bee5f70ddbe8650d server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:07:42.828 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.857+0000 7fac14c00640 10 client.0 ms_handle_connect on v2:172.21.15.177:3300/0 2024-03-27T11:07:42.828 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.857+0000 7fac14c00640 1 -- 172.21.15.97:0/2941490244 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac1809ceb0 con 0x55f8c6866370 2024-03-27T11:07:42.828 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.857+0000 7fac34df1740 1 -- 172.21.15.97:0/2941490244 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac180bd4a0 con 0x55f8c6866370 2024-03-27T11:07:42.829 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.857+0000 7fac14c00640 1 -- 172.21.15.97:0/2941490244 <== mon.1 v2:172.21.15.177:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fac1809c9d0 con 0x55f8c6866370 2024-03-27T11:07:42.829 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.857+0000 7fac14c00640 1 -- 172.21.15.97:0/2941490244 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fac180bd6d0 con 0x55f8c6866370 2024-03-27T11:07:42.829 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.858+0000 7fac34df1740 10 client.26341 fetch_fsmap learned FSMap version 34 2024-03-27T11:07:42.829 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.858+0000 7fac34df1740 1 -- 172.21.15.97:0/2941490244 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55f8c68dfe20 con 0x55f8c6866370 2024-03-27T11:07:42.829 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.858+0000 7fac14c00640 1 -- 172.21.15.97:0/2941490244 <== mon.1 v2:172.21.15.177:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fac180bd4a0 con 0x55f8c6866370 2024-03-27T11:07:42.831 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.858+0000 7fac14c00640 1 -- 172.21.15.97:0/2941490244 <== mon.1 v2:172.21.15.177:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7fac180a9050 con 0x55f8c6866370 2024-03-27T11:07:42.831 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.858+0000 7fac34df1740 10 client.26341 fetch_fsmap finished waiting for FSMap version 34 2024-03-27T11:07:42.831 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.858+0000 7fac34df1740 10 client.26341 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T11:07:42.831 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.858+0000 7fac34df1740 20 client.26341 populate_metadata read hostname 'smithi097' 2024-03-27T11:07:42.831 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.858+0000 7fac34df1740 1 --2- 172.21.15.97:0/2941490244 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55f8c68e1530 0x55f8c68e3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:07:42.831 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.859+0000 7fac34df1740 4 client.26341 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-27T11:07:42.831 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.859+0000 7fac34df1740 1 -- 172.21.15.97:0/2941490244 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f8c67d25a0 con 0x55f8c68e1530 2024-03-27T11:07:42.831 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.861+0000 7fac2b400640 1 --2- 172.21.15.97:0/2941490244 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55f8c68e1530 0x55f8c68e3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:07:42.832 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.862+0000 7fac14c00640 10 client.26341 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T11:07:42.832 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.862+0000 7fac2b400640 1 --2- 172.21.15.97:0/2941490244 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55f8c68e1530 0x55f8c68e3950 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=488393bdcaa3fe41 server_cookie=7a35f8faf2042f28 in_seq=0 out_seq=0 2024-03-27T11:07:42.840 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.870+0000 7fac14c00640 1 -- 172.21.15.97:0/2941490244 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f8c67d25a0 con 0x55f8c68e1530 2024-03-27T11:07:42.840 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.870+0000 7fac14c00640 10 client.26341 handle_command_reply: tid=0 2024-03-27T11:07:42.840 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.870+0000 7fac34df1740 1 client.26341 shutdown 2024-03-27T11:07:42.840 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.871+0000 7fac34df1740 1 -- 172.21.15.97:0/2941490244 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f8c6866370 msgr2=0x55f8c68746d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:07:42.840 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.871+0000 7fac34df1740 1 --2- 172.21.15.97:0/2941490244 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f8c6866370 0x55f8c68746d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fac1809b620 tx=0x7fac18010ce0 comp rx=0 tx=0).stop 2024-03-27T11:07:42.841 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.871+0000 7fac34df1740 1 -- 172.21.15.97:0/2941490244 shutdown_connections 2024-03-27T11:07:42.841 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.871+0000 7fac34df1740 1 --2- 172.21.15.97:0/2941490244 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55f8c68e1530 0x55f8c68e3950 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:42.841 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.871+0000 7fac34df1740 1 --2- 172.21.15.97:0/2941490244 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f8c686b4e0 0x55f8c68df8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:42.841 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.871+0000 7fac34df1740 1 --2- 172.21.15.97:0/2941490244 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f8c6866370 0x55f8c68746d0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:42.842 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.871+0000 7fac34df1740 1 --2- 172.21.15.97:0/2941490244 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f8c6865980 0x55f8c6874190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:42.842 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.871+0000 7fac14c00640 0 client.26341 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T11:07:42.842 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.871+0000 7fac34df1740 1 -- 172.21.15.97:0/2941490244 >> 172.21.15.97:0/2941490244 conn(0x55f8c6860ef0 msgr2=0x55f8c6864150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:07:42.842 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.871+0000 7fac34df1740 1 -- 172.21.15.97:0/2941490244 shutdown_connections 2024-03-27T11:07:42.842 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.871+0000 7fac34df1740 1 -- 172.21.15.97:0/2941490244 wait complete. 2024-03-27T11:07:42.842 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.872+0000 7fac34df1740 20 client.26341 trim_cache size 0 max 16384 2024-03-27T11:07:42.842 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-27T11:07:42.842 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-27T11:07:42.842 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-27T11:07:42.843 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-27T11:07:42.843 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.873+0000 7fac32200640 1 -- 172.21.15.97:0/498405225 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fac100a3780 msgr2=0x7fac100a5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:07:42.843 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.873+0000 7fac32200640 1 --2- 172.21.15.97:0/498405225 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fac100a3780 0x7fac100a5c40 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x55f8c6866bd0 tx=0x7fac24005fd0 comp rx=0 tx=0).stop 2024-03-27T11:07:42.843 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.873+0000 7fac32200640 1 -- 172.21.15.97:0/498405225 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fac2c102d70 msgr2=0x7fac2c19e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:07:42.843 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.873+0000 7fac32200640 1 --2- 172.21.15.97:0/498405225 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fac2c102d70 0x7fac2c19e7a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fac1800e790 tx=0x7fac1800ec50 comp rx=0 tx=0).stop 2024-03-27T11:07:42.843 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.873+0000 7fac32200640 1 -- 172.21.15.97:0/498405225 shutdown_connections 2024-03-27T11:07:42.844 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.873+0000 7fac32200640 1 --2- 172.21.15.97:0/498405225 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac2c105720 0x7fac2c1a5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:42.844 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.873+0000 7fac32200640 1 --2- 172.21.15.97:0/498405225 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fac100a3780 0x7fac100a5c40 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:42.844 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.873+0000 7fac32200640 1 --2- 172.21.15.97:0/498405225 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fac2c102d70 0x7fac2c19e7a0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:42.844 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.873+0000 7fac32200640 1 --2- 172.21.15.97:0/498405225 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fac2c100400 0x7fac2c19e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:42.844 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.873+0000 7fac32200640 1 -- 172.21.15.97:0/498405225 >> 172.21.15.97:0/498405225 conn(0x7fac2c0fc000 msgr2=0x7fac2c100d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:07:42.844 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.873+0000 7fac32200640 1 -- 172.21.15.97:0/498405225 shutdown_connections 2024-03-27T11:07:42.844 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:42.873+0000 7fac32200640 1 -- 172.21.15.97:0/498405225 wait complete. 2024-03-27T11:07:43.889 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:07:43 smithi097 ceph-mon[31859]: pgmap v773: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 46 MiB/s wr, 812 op/s 2024-03-27T11:07:43.923 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:07:43 smithi177 ceph-mon[35294]: pgmap v773: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 46 MiB/s wr, 812 op/s 2024-03-27T11:07:44.014 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:07:43 smithi114 ceph-mon[35338]: pgmap v773: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 46 MiB/s wr, 812 op/s 2024-03-27T11:07:45.888 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:07:45 smithi097 ceph-mon[31859]: pgmap v774: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 47 MiB/s wr, 824 op/s 2024-03-27T11:07:45.923 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:07:45 smithi177 ceph-mon[35294]: pgmap v774: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 47 MiB/s wr, 824 op/s 2024-03-27T11:07:46.013 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:07:45 smithi114 ceph-mon[35338]: pgmap v774: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 47 MiB/s wr, 824 op/s 2024-03-27T11:07:47.888 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:07:47 smithi097 ceph-mon[31859]: pgmap v775: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 45 MiB/s wr, 797 op/s 2024-03-27T11:07:47.923 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:07:47 smithi177 ceph-mon[35294]: pgmap v775: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 45 MiB/s wr, 797 op/s 2024-03-27T11:07:48.014 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:07:47 smithi114 ceph-mon[35338]: pgmap v775: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 45 MiB/s wr, 797 op/s 2024-03-27T11:07:49.888 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:07:49 smithi097 ceph-mon[31859]: pgmap v776: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 47 MiB/s wr, 828 op/s 2024-03-27T11:07:49.923 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:07:49 smithi177 ceph-mon[35294]: pgmap v776: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 47 MiB/s wr, 828 op/s 2024-03-27T11:07:50.014 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:07:49 smithi114 ceph-mon[35338]: pgmap v776: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 47 MiB/s wr, 828 op/s 2024-03-27T11:07:51.888 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:07:51 smithi097 ceph-mon[31859]: pgmap v777: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 50 MiB/s wr, 890 op/s 2024-03-27T11:07:51.923 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:07:51 smithi177 ceph-mon[35294]: pgmap v777: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 50 MiB/s wr, 890 op/s 2024-03-27T11:07:52.014 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:07:51 smithi114 ceph-mon[35338]: pgmap v777: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 50 MiB/s wr, 890 op/s 2024-03-27T11:07:53.888 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:07:53 smithi097 ceph-mon[31859]: pgmap v778: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 45 MiB/s wr, 800 op/s 2024-03-27T11:07:53.924 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:07:53 smithi177 ceph-mon[35294]: pgmap v778: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 45 MiB/s wr, 800 op/s 2024-03-27T11:07:54.013 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:07:53 smithi114 ceph-mon[35338]: pgmap v778: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 45 MiB/s wr, 800 op/s 2024-03-27T11:07:55.888 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:07:55 smithi097 ceph-mon[31859]: pgmap v779: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 46 MiB/s wr, 815 op/s 2024-03-27T11:07:55.923 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:07:55 smithi177 ceph-mon[35294]: pgmap v779: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 46 MiB/s wr, 815 op/s 2024-03-27T11:07:56.014 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:07:55 smithi114 ceph-mon[35338]: pgmap v779: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 46 MiB/s wr, 815 op/s 2024-03-27T11:07:56.098 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.128+0000 7f69d5800640 1 -- 172.21.15.97:0/1213213468 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f69d0102d70 msgr2=0x7f69d01031f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:07:56.098 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.128+0000 7f69d5800640 1 --2- 172.21.15.97:0/1213213468 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f69d0102d70 0x7f69d01031f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f69bc009860 tx=0x7f69bc02bb70 comp rx=0 tx=0).stop 2024-03-27T11:07:56.098 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.128+0000 7f69d5800640 1 -- 172.21.15.97:0/1213213468 shutdown_connections 2024-03-27T11:07:56.098 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.128+0000 7f69d5800640 1 --2- 172.21.15.97:0/1213213468 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69d0103730 0x7f69d0109fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:56.098 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.128+0000 7f69d5800640 1 --2- 172.21.15.97:0/1213213468 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f69d0102d70 0x7f69d01031f0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:56.098 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.128+0000 7f69d5800640 1 --2- 172.21.15.97:0/1213213468 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f69d0101b80 0x7f69d0101fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:56.098 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.128+0000 7f69d5800640 1 -- 172.21.15.97:0/1213213468 >> 172.21.15.97:0/1213213468 conn(0x7f69d00fd110 msgr2=0x7f69d00ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:07:56.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.129+0000 7f69d5800640 1 -- 172.21.15.97:0/1213213468 shutdown_connections 2024-03-27T11:07:56.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.129+0000 7f69d5800640 1 -- 172.21.15.97:0/1213213468 wait complete. 2024-03-27T11:07:56.100 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.130+0000 7f69d5800640 1 Processor -- start 2024-03-27T11:07:56.100 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.131+0000 7f69d5800640 1 -- start start 2024-03-27T11:07:56.101 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.131+0000 7f69d5800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f69d0101b80 0x7f69d0199f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:07:56.101 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.131+0000 7f69d5800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69d0102d70 0x7f69d019a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:07:56.101 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.131+0000 7f69d5800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f69d0103730 0x7f69d01a1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:07:56.101 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.131+0000 7f69d5800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f69d010d700 con 0x7f69d0102d70 2024-03-27T11:07:56.102 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.131+0000 7f69d5800640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f69d010d580 con 0x7f69d0103730 2024-03-27T11:07:56.102 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.131+0000 7f69d5800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f69d010d880 con 0x7f69d0101b80 2024-03-27T11:07:56.102 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.132+0000 7f69ce000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69d0102d70 0x7f69d019a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:07:56.102 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.132+0000 7f69cea00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f69d0101b80 0x7f69d0199f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:07:56.102 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.132+0000 7f69cea00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f69d0101b80 0x7f69d0199f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.97:48916/0 (socket says 172.21.15.97:48916) 2024-03-27T11:07:56.102 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.132+0000 7f69cea00640 1 -- 172.21.15.97:0/3867659126 learned_addr learned my addr 172.21.15.97:0/3867659126 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:07:56.102 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.132+0000 7f69cf400640 1 --2- 172.21.15.97:0/3867659126 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f69d0103730 0x7f69d01a1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:07:56.103 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.132+0000 7f69cea00640 1 -- 172.21.15.97:0/3867659126 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f69d0103730 msgr2=0x7f69d01a1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:07:56.103 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.132+0000 7f69cea00640 1 --2- 172.21.15.97:0/3867659126 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f69d0103730 0x7f69d01a1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:56.103 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.132+0000 7f69cea00640 1 -- 172.21.15.97:0/3867659126 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69d0102d70 msgr2=0x7f69d019a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:07:56.103 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.132+0000 7f69cea00640 1 --2- 172.21.15.97:0/3867659126 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69d0102d70 0x7f69d019a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:56.103 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.132+0000 7f69cea00640 1 -- 172.21.15.97:0/3867659126 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69d01a1bb0 con 0x7f69d0101b80 2024-03-27T11:07:56.103 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.133+0000 7f69cea00640 1 --2- 172.21.15.97:0/3867659126 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f69d0101b80 0x7f69d0199f90 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f69c000d750 tx=0x7f69c000db50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:07:56.103 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.133+0000 7f69c7400640 1 -- 172.21.15.97:0/3867659126 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69c0016070 con 0x7f69d0101b80 2024-03-27T11:07:56.104 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.133+0000 7f69c7400640 1 -- 172.21.15.97:0/3867659126 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f69c0009cd0 con 0x7f69d0101b80 2024-03-27T11:07:56.104 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.133+0000 7f69c7400640 1 -- 172.21.15.97:0/3867659126 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69c0003c90 con 0x7f69d0101b80 2024-03-27T11:07:56.104 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.133+0000 7f69d5800640 1 -- 172.21.15.97:0/3867659126 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69d01a1ea0 con 0x7f69d0101b80 2024-03-27T11:07:56.104 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.133+0000 7f69d5800640 1 -- 172.21.15.97:0/3867659126 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69d01a2300 con 0x7f69d0101b80 2024-03-27T11:07:56.111 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.139+0000 7f69c7400640 1 -- 172.21.15.97:0/3867659126 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f69c0019070 con 0x7f69d0101b80 2024-03-27T11:07:56.111 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.141+0000 7f69d862c740 1 Processor -- start 2024-03-27T11:07:56.111 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.141+0000 7f69d862c740 1 -- start start 2024-03-27T11:07:56.112 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.142+0000 7f69d862c740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f4d9604190 0x55f4d96045b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:07:56.112 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.142+0000 7f69d862c740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f4d9604b80 0x55f4d9609680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:07:56.112 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.142+0000 7f69d862c740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f4d9609cf0 0x55f4d960c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:07:56.112 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.142+0000 7f69d862c740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55f4d9560d10 con 0x55f4d9604190 2024-03-27T11:07:56.112 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.142+0000 7f69d862c740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55f4d9509260 con 0x55f4d9604b80 2024-03-27T11:07:56.112 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.142+0000 7f69d862c740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55f4d955ccc0 con 0x55f4d9609cf0 2024-03-27T11:07:56.115 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.142+0000 7f69cea00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f4d9604b80 0x55f4d9609680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:07:56.115 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.142+0000 7f69cea00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f4d9604b80 0x55f4d9609680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:57950/0 (socket says 172.21.15.97:57950) 2024-03-27T11:07:56.115 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.142+0000 7f69cea00640 1 -- 172.21.15.97:0/3695023160 learned_addr learned my addr 172.21.15.97:0/3695023160 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:07:56.115 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.143+0000 7f69cf400640 1 --2- 172.21.15.97:0/3695023160 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f4d9604190 0x55f4d96045b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:07:56.115 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.143+0000 7f69ce000640 1 --2- 172.21.15.97:0/3695023160 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f4d9609cf0 0x55f4d960c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:07:56.116 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.143+0000 7f69c7400640 1 --2- 172.21.15.97:0/3867659126 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f69b40a3780 0x7f69b40a5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:07:56.116 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.143+0000 7f69c7400640 1 -- 172.21.15.97:0/3867659126 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f69c00cf630 con 0x7f69d0101b80 2024-03-27T11:07:56.116 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.143+0000 7f69ce000640 1 -- 172.21.15.97:0/3695023160 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f4d9604b80 msgr2=0x55f4d9609680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:07:56.116 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.143+0000 7f69ce000640 1 --2- 172.21.15.97:0/3695023160 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f4d9604b80 0x55f4d9609680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:56.116 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.143+0000 7f69ce000640 1 -- 172.21.15.97:0/3695023160 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f4d9604190 msgr2=0x55f4d96045b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:07:56.116 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.144+0000 7f69ce000640 1 --2- 172.21.15.97:0/3695023160 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f4d9604190 0x55f4d96045b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:56.116 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.144+0000 7f69ce000640 1 -- 172.21.15.97:0/3695023160 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4d9500aa0 con 0x55f4d9609cf0 2024-03-27T11:07:56.116 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.144+0000 7f69cf400640 1 --2- 172.21.15.97:0/3695023160 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f4d9604190 0x55f4d96045b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T11:07:56.116 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.144+0000 7f69cf400640 1 --2- 172.21.15.97:0/3867659126 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f69b40a3780 0x7f69b40a5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:07:56.116 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.145+0000 7f69ce000640 1 --2- 172.21.15.97:0/3695023160 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f4d9609cf0 0x55f4d960c180 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f69bc009830 tx=0x7f69bc0095a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=473d200cdf12f8c8 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:07:56.116 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.145+0000 7f69bbe00640 1 -- 172.21.15.97:0/3695023160 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69bc053070 con 0x55f4d9609cf0 2024-03-27T11:07:56.116 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.145+0000 7f69bbe00640 1 -- 172.21.15.97:0/3695023160 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f69bc034680 con 0x55f4d9609cf0 2024-03-27T11:07:56.117 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.146+0000 7f69d862c740 1 -- 172.21.15.97:0/3695023160 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f4d9609cf0 msgr2=0x55f4d960c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:07:56.117 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.146+0000 7f69d862c740 1 --2- 172.21.15.97:0/3695023160 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f4d9609cf0 0x55f4d960c180 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f69bc009830 tx=0x7f69bc0095a0 comp rx=0 tx=0).stop 2024-03-27T11:07:56.117 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.146+0000 7f69d862c740 1 -- 172.21.15.97:0/3695023160 shutdown_connections 2024-03-27T11:07:56.117 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.146+0000 7f69d862c740 1 --2- 172.21.15.97:0/3695023160 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f4d9609cf0 0x55f4d960c180 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:56.118 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.146+0000 7f69d862c740 1 --2- 172.21.15.97:0/3695023160 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f4d9604b80 0x55f4d9609680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:56.118 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.146+0000 7f69d862c740 1 --2- 172.21.15.97:0/3695023160 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f4d9604190 0x55f4d96045b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:56.118 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.146+0000 7f69d862c740 1 -- 172.21.15.97:0/3695023160 >> 172.21.15.97:0/3695023160 conn(0x55f4d95ff700 msgr2=0x55f4d95ffb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:07:56.118 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.147+0000 7f69cf400640 1 --2- 172.21.15.97:0/3867659126 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f69b40a3780 0x7f69b40a5c40 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x55f4d96053e0 tx=0x7f69c8004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:07:56.118 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.147+0000 7f69d862c740 1 -- 172.21.15.97:0/3695023160 shutdown_connections 2024-03-27T11:07:56.118 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.147+0000 7f69d862c740 1 -- 172.21.15.97:0/3695023160 wait complete. 2024-03-27T11:07:56.118 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.148+0000 7f69d862c740 1 -- start start 2024-03-27T11:07:56.118 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.148+0000 7f69d862c740 1 Processor -- start 2024-03-27T11:07:56.119 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.149+0000 7f69d862c740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f4d9604190 0x55f4d95f83b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:07:56.119 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.149+0000 7f69d862c740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f4d9604b80 0x55f4d95f88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:07:56.119 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.149+0000 7f69d862c740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f4d9609cf0 0x55f4d95fd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:07:56.119 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.149+0000 7f69d862c740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55f4d95f6b90 con 0x55f4d9604b80 2024-03-27T11:07:56.119 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.149+0000 7f69d862c740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55f4d951c630 con 0x55f4d9604190 2024-03-27T11:07:56.119 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.149+0000 7f69d862c740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55f4d95f6d10 con 0x55f4d9609cf0 2024-03-27T11:07:56.120 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.149+0000 7f69d862c740 10 client.0 _ll_register_callbacks cb 0x55f4d9521c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T11:07:56.120 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.149+0000 7f69ce000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f4d9604b80 0x55f4d95f88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:07:56.120 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.149+0000 7f69cf400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f4d9609cf0 0x55f4d95fd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:07:56.120 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.149+0000 7f69cf400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f4d9609cf0 0x55f4d95fd920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.97:48934/0 (socket says 172.21.15.97:48934) 2024-03-27T11:07:56.120 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.149+0000 7f69cf400640 1 -- 172.21.15.97:0/2186677955 learned_addr learned my addr 172.21.15.97:0/2186677955 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:07:56.120 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.149+0000 7f69cea00640 1 --2- 172.21.15.97:0/2186677955 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f4d9604190 0x55f4d95f83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:07:56.120 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.150+0000 7f69ce000640 1 -- 172.21.15.97:0/2186677955 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f4d9609cf0 msgr2=0x55f4d95fd920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:07:56.120 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.150+0000 7f69ce000640 1 --2- 172.21.15.97:0/2186677955 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f4d9609cf0 0x55f4d95fd920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:56.120 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.150+0000 7f69ce000640 1 -- 172.21.15.97:0/2186677955 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f4d9604190 msgr2=0x55f4d95f83b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:07:56.120 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.150+0000 7f69ce000640 1 --2- 172.21.15.97:0/2186677955 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f4d9604190 0x55f4d95f83b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:56.120 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.150+0000 7f69ce000640 1 -- 172.21.15.97:0/2186677955 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4d95285f0 con 0x55f4d9604b80 2024-03-27T11:07:56.121 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.150+0000 7f69ce000640 1 --2- 172.21.15.97:0/2186677955 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f4d9604b80 0x55f4d95f88f0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f69bc037040 tx=0x7f69bc02c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1afd93f79004c961 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:07:56.121 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.150+0000 7f69b8c00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T11:07:56.121 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.151+0000 7f69b8c00640 1 -- 172.21.15.97:0/2186677955 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69bc053040 con 0x55f4d9604b80 2024-03-27T11:07:56.121 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.151+0000 7f69b8c00640 1 -- 172.21.15.97:0/2186677955 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f69bc034c80 con 0x55f4d9604b80 2024-03-27T11:07:56.121 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.151+0000 7f69d862c740 1 -- 172.21.15.97:0/2186677955 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69bc034680 con 0x55f4d9604b80 2024-03-27T11:07:56.121 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.152+0000 7f69b8c00640 1 -- 172.21.15.97:0/2186677955 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f69bc004030 con 0x55f4d9604b80 2024-03-27T11:07:56.122 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.152+0000 7f69d862c740 10 client.16935 fetch_fsmap learned FSMap version 34 2024-03-27T11:07:56.122 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.152+0000 7f69d862c740 1 -- 172.21.15.97:0/2186677955 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55f4d9275210 con 0x55f4d9604b80 2024-03-27T11:07:56.123 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.153+0000 7f69b8c00640 1 -- 172.21.15.97:0/2186677955 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f69bc034680 con 0x55f4d9604b80 2024-03-27T11:07:56.124 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.154+0000 7f69b8c00640 1 -- 172.21.15.97:0/2186677955 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f69bc058450 con 0x55f4d9604b80 2024-03-27T11:07:56.124 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.154+0000 7f69d862c740 10 client.16935 fetch_fsmap finished waiting for FSMap version 34 2024-03-27T11:07:56.124 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.154+0000 7f69d862c740 10 client.16935 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T11:07:56.124 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.154+0000 7f69d862c740 20 client.16935 populate_metadata read hostname 'smithi097' 2024-03-27T11:07:56.124 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.154+0000 7f69d862c740 1 --2- 172.21.15.97:0/2186677955 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55f4d9611670 0x55f4d9613a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:07:56.125 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.154+0000 7f69d862c740 4 client.16935 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T11:07:56.125 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.154+0000 7f69d862c740 1 -- 172.21.15.97:0/2186677955 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f4d956e250 con 0x55f4d9611670 2024-03-27T11:07:56.125 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.155+0000 7f69cea00640 1 --2- 172.21.15.97:0/2186677955 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55f4d9611670 0x55f4d9613a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:07:56.125 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.156+0000 7f69cea00640 1 --2- 172.21.15.97:0/2186677955 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55f4d9611670 0x55f4d9613a60 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c037fa6eb5584519 server_cookie=4498b9cf7b3c3171 in_seq=0 out_seq=0 2024-03-27T11:07:56.125 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.156+0000 7f69b8c00640 10 client.16935 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T11:07:56.127 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.157+0000 7f69b8c00640 1 -- 172.21.15.97:0/2186677955 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55f4d956e250 con 0x55f4d9611670 2024-03-27T11:07:56.127 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.157+0000 7f69b8c00640 10 client.16935 handle_command_reply: tid=0 2024-03-27T11:07:56.127 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.158+0000 7f69d862c740 1 client.16935 shutdown 2024-03-27T11:07:56.128 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.158+0000 7f69d862c740 1 -- 172.21.15.97:0/2186677955 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f4d9604b80 msgr2=0x55f4d95f88f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:07:56.128 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.158+0000 7f69d862c740 1 --2- 172.21.15.97:0/2186677955 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f4d9604b80 0x55f4d95f88f0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f69bc037040 tx=0x7f69bc02c530 comp rx=0 tx=0).stop 2024-03-27T11:07:56.128 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.158+0000 7f69d862c740 1 -- 172.21.15.97:0/2186677955 shutdown_connections 2024-03-27T11:07:56.128 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.158+0000 7f69d862c740 1 --2- 172.21.15.97:0/2186677955 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55f4d9611670 0x55f4d9613a60 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:56.128 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.158+0000 7f69d862c740 1 --2- 172.21.15.97:0/2186677955 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f4d9609cf0 0x55f4d95fd920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:56.128 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.158+0000 7f69d862c740 1 --2- 172.21.15.97:0/2186677955 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f4d9604b80 0x55f4d95f88f0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:56.128 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.158+0000 7f69b8c00640 0 client.16935 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T11:07:56.128 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.158+0000 7f69d862c740 1 --2- 172.21.15.97:0/2186677955 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f4d9604190 0x55f4d95f83b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:56.128 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.158+0000 7f69d862c740 1 -- 172.21.15.97:0/2186677955 >> 172.21.15.97:0/2186677955 conn(0x55f4d95ff700 msgr2=0x55f4d95f60f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:07:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.159+0000 7f69d862c740 1 -- 172.21.15.97:0/2186677955 shutdown_connections 2024-03-27T11:07:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.159+0000 7f69d862c740 1 -- 172.21.15.97:0/2186677955 wait complete. 2024-03-27T11:07:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.159+0000 7f69d862c740 20 client.16935 trim_cache size 0 max 16384 2024-03-27T11:07:56.157 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.187+0000 7f69d862c740 1 Processor -- start 2024-03-27T11:07:56.157 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.187+0000 7f69d862c740 1 -- start start 2024-03-27T11:07:56.157 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.188+0000 7f69d862c740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f4d9604190 0x55f4d95f7100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:07:56.157 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.188+0000 7f69d862c740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f4d9604b80 0x55f4d9602960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:07:56.157 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.188+0000 7f69d862c740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f4d9609cf0 0x55f4d9602ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:07:56.157 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.188+0000 7f69d862c740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f69d01a6e20 con 0x55f4d9604b80 2024-03-27T11:07:56.158 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.188+0000 7f69d862c740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f69d01a8960 con 0x55f4d9604190 2024-03-27T11:07:56.158 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.188+0000 7f69d862c740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f69d01a74f0 con 0x55f4d9609cf0 2024-03-27T11:07:56.158 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.188+0000 7f69cea00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f4d9604190 0x55f4d95f7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:07:56.158 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.188+0000 7f69cea00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f4d9604190 0x55f4d95f7100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:57968/0 (socket says 172.21.15.97:57968) 2024-03-27T11:07:56.158 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.188+0000 7f69cea00640 1 -- 172.21.15.97:0/361847021 learned_addr learned my addr 172.21.15.97:0/361847021 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:07:56.158 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.188+0000 7f69ce000640 1 --2- 172.21.15.97:0/361847021 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f4d9604b80 0x55f4d9602960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:07:56.158 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.189+0000 7f69cf400640 1 --2- 172.21.15.97:0/361847021 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f4d9609cf0 0x55f4d9602ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:07:56.159 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.189+0000 7f69cea00640 1 -- 172.21.15.97:0/361847021 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f4d9609cf0 msgr2=0x55f4d9602ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:07:56.159 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.189+0000 7f69cea00640 1 --2- 172.21.15.97:0/361847021 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f4d9609cf0 0x55f4d9602ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:56.159 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.189+0000 7f69cea00640 1 -- 172.21.15.97:0/361847021 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f4d9604b80 msgr2=0x55f4d9602960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:07:56.159 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.189+0000 7f69cea00640 1 --2- 172.21.15.97:0/361847021 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f4d9604b80 0x55f4d9602960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:56.159 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.189+0000 7f69cea00640 1 -- 172.21.15.97:0/361847021 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4d9341620 con 0x55f4d9604190 2024-03-27T11:07:56.160 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.190+0000 7f69cea00640 1 --2- 172.21.15.97:0/361847021 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f4d9604190 0x55f4d95f7100 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f69c00cd190 tx=0x7f69c00cd010 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81cd8f2cb5b1790d server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:07:56.160 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.191+0000 7f69bb400640 1 -- 172.21.15.97:0/361847021 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69c0020040 con 0x55f4d9604190 2024-03-27T11:07:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.191+0000 7f69bb400640 1 -- 172.21.15.97:0/361847021 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f69c009c210 con 0x55f4d9604190 2024-03-27T11:07:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.191+0000 7f69bb400640 1 -- 172.21.15.97:0/361847021 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69c0099e10 con 0x55f4d9604190 2024-03-27T11:07:56.162 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.192+0000 7f69d862c740 1 -- 172.21.15.97:0/361847021 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f4d9604190 msgr2=0x55f4d95f7100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:07:56.162 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.192+0000 7f69d862c740 1 --2- 172.21.15.97:0/361847021 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f4d9604190 0x55f4d95f7100 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f69c00cd190 tx=0x7f69c00cd010 comp rx=0 tx=0).stop 2024-03-27T11:07:56.162 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.192+0000 7f69d862c740 1 -- 172.21.15.97:0/361847021 shutdown_connections 2024-03-27T11:07:56.162 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.192+0000 7f69d862c740 1 --2- 172.21.15.97:0/361847021 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f4d9609cf0 0x55f4d9602ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:56.162 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.192+0000 7f69d862c740 1 --2- 172.21.15.97:0/361847021 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f4d9604b80 0x55f4d9602960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:56.162 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.192+0000 7f69d862c740 1 --2- 172.21.15.97:0/361847021 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f4d9604190 0x55f4d95f7100 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:56.162 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.192+0000 7f69d862c740 1 -- 172.21.15.97:0/361847021 >> 172.21.15.97:0/361847021 conn(0x55f4d95ff700 msgr2=0x55f4d9605ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:07:56.163 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.193+0000 7f69d862c740 1 -- 172.21.15.97:0/361847021 shutdown_connections 2024-03-27T11:07:56.163 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.193+0000 7f69d862c740 1 -- 172.21.15.97:0/361847021 wait complete. 2024-03-27T11:07:56.163 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.193+0000 7f69d862c740 1 -- start start 2024-03-27T11:07:56.163 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.194+0000 7f69d862c740 1 Processor -- start 2024-03-27T11:07:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.195+0000 7f69d862c740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f4d9604190 0x55f4d9622920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:07:56.166 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.196+0000 7f69d862c740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f4d9604b80 0x55f4d9622e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:07:56.167 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.197+0000 7f69d862c740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f4d9609cf0 0x55f4d9623e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:07:56.167 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.197+0000 7f69d862c740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f69d01a8290 con 0x55f4d9604b80 2024-03-27T11:07:56.167 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.197+0000 7f69d862c740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f69d01a7bc0 con 0x55f4d9609cf0 2024-03-27T11:07:56.167 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.197+0000 7f69d862c740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f69d01a4e40 con 0x55f4d9604190 2024-03-27T11:07:56.167 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.197+0000 7f69cf400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f4d9609cf0 0x55f4d9623e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:07:56.167 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.197+0000 7f69cf400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f4d9609cf0 0x55f4d9623e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:57982/0 (socket says 172.21.15.97:57982) 2024-03-27T11:07:56.167 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.197+0000 7f69cf400640 1 -- 172.21.15.97:0/1285871762 learned_addr learned my addr 172.21.15.97:0/1285871762 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:07:56.168 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.197+0000 7f69d862c740 10 client.0 _ll_register_callbacks cb 0x7f69c8009900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T11:07:56.168 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.198+0000 7f69cf400640 1 -- 172.21.15.97:0/1285871762 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f4d9604190 msgr2=0x55f4d9622920 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-27T11:07:56.168 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.198+0000 7f69cf400640 1 --2- 172.21.15.97:0/1285871762 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f4d9604190 0x55f4d9622920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:56.168 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.198+0000 7f69cf400640 1 -- 172.21.15.97:0/1285871762 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f4d9604b80 msgr2=0x55f4d9622e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:07:56.168 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.198+0000 7f69cf400640 1 --2- 172.21.15.97:0/1285871762 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f4d9604b80 0x55f4d9622e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:56.168 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.198+0000 7f69cf400640 1 -- 172.21.15.97:0/1285871762 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4d9624460 con 0x55f4d9609cf0 2024-03-27T11:07:56.168 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.198+0000 7f69d862c740 1 -- 172.21.15.97:0/1285871762 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69c009c210 con 0x55f4d9609cf0 2024-03-27T11:07:56.168 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.198+0000 7f69cf400640 1 --2- 172.21.15.97:0/1285871762 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f4d9609cf0 0x55f4d9623e70 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f69c800fd30 tx=0x7f69c8018000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b25841d0b733bd8b server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:07:56.168 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.198+0000 7f69b8c00640 10 client.36644 ms_handle_connect on v2:172.21.15.114:3300/0 2024-03-27T11:07:56.169 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.199+0000 7f69b8c00640 1 -- 172.21.15.97:0/1285871762 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69c8004b20 con 0x55f4d9609cf0 2024-03-27T11:07:56.169 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.199+0000 7f69b8c00640 1 -- 172.21.15.97:0/1285871762 <== mon.2 v2:172.21.15.114:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f69c8004cc0 con 0x55f4d9609cf0 2024-03-27T11:07:56.169 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.199+0000 7f69b8c00640 1 -- 172.21.15.97:0/1285871762 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f69c8005020 con 0x55f4d9609cf0 2024-03-27T11:07:56.170 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.200+0000 7f69d862c740 10 client.36644 fetch_fsmap learned FSMap version 34 2024-03-27T11:07:56.170 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.200+0000 7f69d862c740 1 -- 172.21.15.97:0/1285871762 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({fsmap=34}) -- 0x55f4d96779c0 con 0x55f4d9609cf0 2024-03-27T11:07:56.171 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.201+0000 7f69b8c00640 1 -- 172.21.15.97:0/1285871762 <== mon.2 v2:172.21.15.114:3300/0 4 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f69c80051c0 con 0x55f4d9609cf0 2024-03-27T11:07:56.172 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.202+0000 7f69d862c740 10 client.36644 fetch_fsmap finished waiting for FSMap version 34 2024-03-27T11:07:56.172 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.202+0000 7f69d862c740 10 client.36644 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T11:07:56.172 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.202+0000 7f69d862c740 20 client.36644 populate_metadata read hostname 'smithi097' 2024-03-27T11:07:56.172 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.202+0000 7f69d862c740 1 --2- 172.21.15.97:0/1285871762 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55f4d9678240 0x55f4d967a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:07:56.172 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.202+0000 7f69d862c740 4 client.36644 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-27T11:07:56.172 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.202+0000 7f69d862c740 1 -- 172.21.15.97:0/1285871762 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f4d95685a0 con 0x55f4d9678240 2024-03-27T11:07:56.172 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.202+0000 7f69cea00640 1 --2- 172.21.15.97:0/1285871762 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55f4d9678240 0x55f4d967a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:07:56.173 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.203+0000 7f69b8c00640 10 client.36644 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T11:07:56.173 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.203+0000 7f69cea00640 1 --2- 172.21.15.97:0/1285871762 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55f4d9678240 0x55f4d967a6a0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20d70e3e8a74be8d server_cookie=9ae5bee9571d166d in_seq=0 out_seq=0 2024-03-27T11:07:56.185 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.215+0000 7f69b8c00640 1 -- 172.21.15.97:0/1285871762 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f4d95685a0 con 0x55f4d9678240 2024-03-27T11:07:56.185 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.215+0000 7f69b8c00640 10 client.36644 handle_command_reply: tid=0 2024-03-27T11:07:56.185 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.215+0000 7f69d862c740 1 client.36644 shutdown 2024-03-27T11:07:56.186 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.216+0000 7f69d862c740 1 -- 172.21.15.97:0/1285871762 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f4d9609cf0 msgr2=0x55f4d9623e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:07:56.186 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.216+0000 7f69d862c740 1 --2- 172.21.15.97:0/1285871762 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f4d9609cf0 0x55f4d9623e70 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f69c800fd30 tx=0x7f69c8018000 comp rx=0 tx=0).stop 2024-03-27T11:07:56.186 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.216+0000 7f69d862c740 1 -- 172.21.15.97:0/1285871762 shutdown_connections 2024-03-27T11:07:56.186 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.216+0000 7f69d862c740 1 --2- 172.21.15.97:0/1285871762 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x55f4d9678240 0x55f4d967a6a0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:56.186 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.216+0000 7f69d862c740 1 --2- 172.21.15.97:0/1285871762 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f4d9609cf0 0x55f4d9623e70 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:56.186 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.216+0000 7f69d862c740 1 --2- 172.21.15.97:0/1285871762 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f4d9604b80 0x55f4d9622e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:56.186 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.216+0000 7f69d862c740 1 --2- 172.21.15.97:0/1285871762 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f4d9604190 0x55f4d9622920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:56.186 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.216+0000 7f69d862c740 1 -- 172.21.15.97:0/1285871762 >> 172.21.15.97:0/1285871762 conn(0x55f4d95ff700 msgr2=0x55f4d9602960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:07:56.186 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.216+0000 7f69b8c00640 0 client.36644 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T11:07:56.187 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.217+0000 7f69d862c740 1 -- 172.21.15.97:0/1285871762 shutdown_connections 2024-03-27T11:07:56.187 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.217+0000 7f69d862c740 1 -- 172.21.15.97:0/1285871762 wait complete. 2024-03-27T11:07:56.187 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.218+0000 7f69d862c740 20 client.36644 trim_cache size 0 max 16384 2024-03-27T11:07:56.187 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-27T11:07:56.187 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-27T11:07:56.187 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-27T11:07:56.188 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-27T11:07:56.188 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.218+0000 7f69d5800640 1 -- 172.21.15.97:0/3867659126 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f69b40a3780 msgr2=0x7f69b40a5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:07:56.189 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.218+0000 7f69d5800640 1 --2- 172.21.15.97:0/3867659126 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f69b40a3780 0x7f69b40a5c40 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x55f4d96053e0 tx=0x7f69c8004a40 comp rx=0 tx=0).stop 2024-03-27T11:07:56.189 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.219+0000 7f69d5800640 1 -- 172.21.15.97:0/3867659126 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f69d0101b80 msgr2=0x7f69d0199f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:07:56.189 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.219+0000 7f69d5800640 1 --2- 172.21.15.97:0/3867659126 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f69d0101b80 0x7f69d0199f90 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f69c000d750 tx=0x7f69c000db50 comp rx=0 tx=0).stop 2024-03-27T11:07:56.189 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.219+0000 7f69d5800640 1 -- 172.21.15.97:0/3867659126 shutdown_connections 2024-03-27T11:07:56.189 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.219+0000 7f69d5800640 1 --2- 172.21.15.97:0/3867659126 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f69b40a3780 0x7f69b40a5c40 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:56.189 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.219+0000 7f69d5800640 1 --2- 172.21.15.97:0/3867659126 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f69d0103730 0x7f69d01a1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:56.189 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.219+0000 7f69d5800640 1 --2- 172.21.15.97:0/3867659126 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69d0102d70 0x7f69d019a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:56.189 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.219+0000 7f69d5800640 1 --2- 172.21.15.97:0/3867659126 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f69d0101b80 0x7f69d0199f90 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:07:56.189 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.219+0000 7f69d5800640 1 -- 172.21.15.97:0/3867659126 >> 172.21.15.97:0/3867659126 conn(0x7f69d00fd110 msgr2=0x7f69d0073d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:07:56.189 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.219+0000 7f69d5800640 1 -- 172.21.15.97:0/3867659126 shutdown_connections 2024-03-27T11:07:56.189 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:07:56.219+0000 7f69d5800640 1 -- 172.21.15.97:0/3867659126 wait complete. 2024-03-27T11:07:57.887 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:07:57 smithi097 ceph-mon[31859]: pgmap v780: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 44 MiB/s wr, 790 op/s 2024-03-27T11:07:57.923 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:07:57 smithi177 ceph-mon[35294]: pgmap v780: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 44 MiB/s wr, 790 op/s 2024-03-27T11:07:58.014 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:07:57 smithi114 ceph-mon[35338]: pgmap v780: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 44 MiB/s wr, 790 op/s 2024-03-27T11:07:58.894 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:07:58 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T11:07:58.923 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:07:58 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T11:07:59.016 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:07:58 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T11:07:59.888 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:07:59 smithi097 ceph-mon[31859]: pgmap v781: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 47 MiB/s wr, 831 op/s 2024-03-27T11:07:59.923 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:07:59 smithi177 ceph-mon[35294]: pgmap v781: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 47 MiB/s wr, 831 op/s 2024-03-27T11:08:00.014 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:07:59 smithi114 ceph-mon[35338]: pgmap v781: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 47 MiB/s wr, 831 op/s 2024-03-27T11:08:00.888 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:08:00 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T11:08:00.888 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:08:00 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T11:08:00.888 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:08:00 smithi097 ceph-mon[31859]: from='mgr.14152 ' entity='mgr.x' 2024-03-27T11:08:00.923 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:08:00 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T11:08:00.923 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:08:00 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T11:08:00.923 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:08:00 smithi177 ceph-mon[35294]: from='mgr.14152 ' entity='mgr.x' 2024-03-27T11:08:01.013 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:08:00 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T11:08:01.013 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:08:00 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T11:08:01.014 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:08:00 smithi114 ceph-mon[35338]: from='mgr.14152 ' entity='mgr.x' 2024-03-27T11:08:02.263 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:08:02 smithi114 ceph-mon[35338]: pgmap v782: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 50 MiB/s wr, 895 op/s 2024-03-27T11:08:02.388 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:08:02 smithi097 ceph-mon[31859]: pgmap v782: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 50 MiB/s wr, 895 op/s 2024-03-27T11:08:02.423 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:08:02 smithi177 ceph-mon[35294]: pgmap v782: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 50 MiB/s wr, 895 op/s 2024-03-27T11:08:04.264 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:08:04 smithi114 ceph-mon[35338]: pgmap v783: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 45 MiB/s wr, 803 op/s 2024-03-27T11:08:04.387 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:08:04 smithi097 ceph-mon[31859]: pgmap v783: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 45 MiB/s wr, 803 op/s 2024-03-27T11:08:04.423 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:08:04 smithi177 ceph-mon[35294]: pgmap v783: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 45 MiB/s wr, 803 op/s 2024-03-27T11:08:05.356 INFO:tasks.workunit.client.0.smithi097.stdout:stopping iogen 2024-03-27T11:08:05.357 INFO:tasks.workunit.client.0.smithi097.stderr:+ echo 'stopping iogen' 2024-03-27T11:08:05.357 INFO:tasks.workunit.client.0.smithi097.stderr:+ ./iogen -k 2024-03-27T11:08:05.361 INFO:tasks.workunit.client.0.smithi097.stdout:OK 2024-03-27T11:08:05.361 INFO:tasks.workunit.client.0.smithi097.stderr:+ echo OK 2024-03-27T11:08:05.389 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:08:05 smithi097 ceph-mon[31859]: pgmap v784: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 45 MiB/s wr, 803 op/s 2024-03-27T11:08:05.423 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:08:05 smithi177 ceph-mon[35294]: pgmap v784: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 45 MiB/s wr, 803 op/s 2024-03-27T11:08:05.513 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:08:05 smithi114 ceph-mon[35338]: pgmap v784: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 45 MiB/s wr, 803 op/s 2024-03-27T11:08:07.508 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-27T11:08:07.508 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-27T11:08:07.673 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:08:07 smithi177 ceph-mon[35294]: pgmap v785: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 43 MiB/s wr, 777 op/s 2024-03-27T11:08:07.692 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:08:07 smithi097 ceph-mon[31859]: pgmap v785: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 43 MiB/s wr, 777 op/s 2024-03-27T11:08:07.764 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:08:07 smithi114 ceph-mon[35338]: pgmap v785: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 43 MiB/s wr, 777 op/s 2024-03-27T11:08:09.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:08:09 smithi177 ceph-mon[35294]: pgmap v786: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 45 MiB/s wr, 812 op/s 2024-03-27T11:08:09.714 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.744+0000 7f0d47200640 1 -- 172.21.15.97:0/3231758533 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d48103570 msgr2=0x7f0d48109e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:08:09.714 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.744+0000 7f0d47200640 1 --2- 172.21.15.97:0/3231758533 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d48103570 0x7f0d48109e00 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f0d40009ca0 tx=0x7f0d4002c0f0 comp rx=0 tx=0).stop 2024-03-27T11:08:09.714 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.744+0000 7f0d47200640 1 -- 172.21.15.97:0/3231758533 shutdown_connections 2024-03-27T11:08:09.714 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.744+0000 7f0d47200640 1 --2- 172.21.15.97:0/3231758533 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d48103570 0x7f0d48109e00 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:08:09.715 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.744+0000 7f0d47200640 1 --2- 172.21.15.97:0/3231758533 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0d48102bb0 0x7f0d48103030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:08:09.715 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.744+0000 7f0d47200640 1 --2- 172.21.15.97:0/3231758533 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0d481019c0 0x7f0d48101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:08:09.715 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.744+0000 7f0d47200640 1 -- 172.21.15.97:0/3231758533 >> 172.21.15.97:0/3231758533 conn(0x7f0d480fcf30 msgr2=0x7f0d480ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:08:09.715 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.744+0000 7f0d47200640 1 -- 172.21.15.97:0/3231758533 shutdown_connections 2024-03-27T11:08:09.715 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.745+0000 7f0d47200640 1 -- 172.21.15.97:0/3231758533 wait complete. 2024-03-27T11:08:09.717 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.745+0000 7f0d47200640 1 Processor -- start 2024-03-27T11:08:09.717 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.746+0000 7f0d47200640 1 -- start start 2024-03-27T11:08:09.717 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.746+0000 7f0d47200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0d481019c0 0x7f0d48199da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:08:09.717 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.746+0000 7f0d47200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0d48102bb0 0x7f0d4819a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:08:09.717 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.746+0000 7f0d47200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d48103570 0x7f0d481a1310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:08:09.717 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.746+0000 7f0d47200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0d4810d4f0 con 0x7f0d48103570 2024-03-27T11:08:09.717 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.746+0000 7f0d47200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f0d4810d370 con 0x7f0d481019c0 2024-03-27T11:08:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.746+0000 7f0d47200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f0d4810d670 con 0x7f0d48102bb0 2024-03-27T11:08:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.746+0000 7f0d46800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d48103570 0x7f0d481a1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:08:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.746+0000 7f0d46800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d48103570 0x7f0d481a1310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54656/0 (socket says 172.21.15.97:54656) 2024-03-27T11:08:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.746+0000 7f0d46800640 1 -- 172.21.15.97:0/323998158 learned_addr learned my addr 172.21.15.97:0/323998158 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:08:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.746+0000 7f0d45e00640 1 --2- 172.21.15.97:0/323998158 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0d481019c0 0x7f0d48199da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:08:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.747+0000 7f0d3d400640 1 --2- 172.21.15.97:0/323998158 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0d48102bb0 0x7f0d4819a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:08:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.747+0000 7f0d46800640 1 -- 172.21.15.97:0/323998158 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0d48102bb0 msgr2=0x7f0d4819a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:08:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.747+0000 7f0d46800640 1 --2- 172.21.15.97:0/323998158 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0d48102bb0 0x7f0d4819a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:08:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.747+0000 7f0d46800640 1 -- 172.21.15.97:0/323998158 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0d481019c0 msgr2=0x7f0d48199da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:08:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.747+0000 7f0d46800640 1 --2- 172.21.15.97:0/323998158 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0d481019c0 0x7f0d48199da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:08:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.747+0000 7f0d46800640 1 -- 172.21.15.97:0/323998158 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d481a19b0 con 0x7f0d48103570 2024-03-27T11:08:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.747+0000 7f0d46800640 1 --2- 172.21.15.97:0/323998158 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d48103570 0x7f0d481a1310 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f0d40032380 tx=0x7f0d40032820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:08:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.747+0000 7f0d3f400640 1 -- 172.21.15.97:0/323998158 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d40041020 con 0x7f0d48103570 2024-03-27T11:08:09.719 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.747+0000 7f0d3f400640 1 -- 172.21.15.97:0/323998158 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0d40003e20 con 0x7f0d48103570 2024-03-27T11:08:09.719 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.747+0000 7f0d3f400640 1 -- 172.21.15.97:0/323998158 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d40039040 con 0x7f0d48103570 2024-03-27T11:08:09.719 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.747+0000 7f0d47200640 1 -- 172.21.15.97:0/323998158 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d481a1c40 con 0x7f0d48103570 2024-03-27T11:08:09.719 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.748+0000 7f0d47200640 1 -- 172.21.15.97:0/323998158 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d481a2180 con 0x7f0d48103570 2024-03-27T11:08:09.721 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.751+0000 7f0d3f400640 1 -- 172.21.15.97:0/323998158 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f0d40044020 con 0x7f0d48103570 2024-03-27T11:08:09.722 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.752+0000 7f0d4f9d1740 1 Processor -- start 2024-03-27T11:08:09.722 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.752+0000 7f0d4f9d1740 1 -- start start 2024-03-27T11:08:09.722 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.752+0000 7f0d3f400640 1 --2- 172.21.15.97:0/323998158 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f0d2c0a3680 0x7f0d2c0a5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:08:09.724 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.753+0000 7f0d4f9d1740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56200f9bbb60 0x56200f9bbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:08:09.724 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.753+0000 7f0d4f9d1740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56200f9bc550 0x56200f9c1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:08:09.724 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.753+0000 7f0d4f9d1740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56200f9c16c0 0x56200f9c3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:08:09.724 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.753+0000 7f0d3f400640 1 -- 172.21.15.97:0/323998158 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f0d400f1f40 con 0x7f0d48103570 2024-03-27T11:08:09.724 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.753+0000 7f0d4f9d1740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56200f920d10 con 0x56200f9bc550 2024-03-27T11:08:09.724 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.753+0000 7f0d4f9d1740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x56200f8c9260 con 0x56200f9bbb60 2024-03-27T11:08:09.724 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.753+0000 7f0d4f9d1740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x56200f91ccc0 con 0x56200f9c16c0 2024-03-27T11:08:09.724 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.753+0000 7f0d3d400640 1 --2- 172.21.15.97:0/323998158 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f0d2c0a3680 0x7f0d2c0a5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:08:09.724 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.753+0000 7f0d45e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56200f9bbb60 0x56200f9bbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:08:09.724 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.753+0000 7f0d3d400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56200f9bc550 0x56200f9c1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:08:09.724 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.753+0000 7f0d45e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56200f9bbb60 0x56200f9bbf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:60860/0 (socket says 172.21.15.97:60860) 2024-03-27T11:08:09.724 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.753+0000 7f0d45e00640 1 -- 172.21.15.97:0/3946315364 learned_addr learned my addr 172.21.15.97:0/3946315364 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:08:09.725 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.753+0000 7f0d46800640 1 --2- 172.21.15.97:0/3946315364 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56200f9c16c0 0x56200f9c3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:08:09.725 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.754+0000 7f0d45e00640 1 -- 172.21.15.97:0/3946315364 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56200f9c16c0 msgr2=0x56200f9c3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:08:09.725 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.754+0000 7f0d45e00640 1 --2- 172.21.15.97:0/3946315364 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56200f9c16c0 0x56200f9c3b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:08:09.725 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.754+0000 7f0d45e00640 1 -- 172.21.15.97:0/3946315364 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56200f9bc550 msgr2=0x56200f9c1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:08:09.725 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.754+0000 7f0d45e00640 1 --2- 172.21.15.97:0/3946315364 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56200f9bc550 0x56200f9c1050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:08:09.725 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.754+0000 7f0d45e00640 1 -- 172.21.15.97:0/3946315364 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56200f8c0aa0 con 0x56200f9bbb60 2024-03-27T11:08:09.725 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.754+0000 7f0d3d400640 1 --2- 172.21.15.97:0/3946315364 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56200f9bc550 0x56200f9c1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T11:08:09.725 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.754+0000 7f0d3d400640 1 --2- 172.21.15.97:0/323998158 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f0d2c0a3680 0x7f0d2c0a5b40 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f0d300058c0 tx=0x7f0d3000ba80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:08:09.725 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.754+0000 7f0d45e00640 1 --2- 172.21.15.97:0/3946315364 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56200f9bbb60 0x56200f9bbf80 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0d38009c60 tx=0x7f0d38011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2a8f97b7c85043d server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:08:09.728 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.755+0000 7f0d37000640 1 -- 172.21.15.97:0/3946315364 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d38009d50 con 0x56200f9bbb60 2024-03-27T11:08:09.728 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.755+0000 7f0d37000640 1 -- 172.21.15.97:0/3946315364 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0d380035f0 con 0x56200f9bbb60 2024-03-27T11:08:09.728 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.755+0000 7f0d37000640 1 -- 172.21.15.97:0/3946315364 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d38019650 con 0x56200f9bbb60 2024-03-27T11:08:09.728 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.756+0000 7f0d4f9d1740 1 -- 172.21.15.97:0/3946315364 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56200f9bbb60 msgr2=0x56200f9bbf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:08:09.728 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.756+0000 7f0d4f9d1740 1 --2- 172.21.15.97:0/3946315364 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56200f9bbb60 0x56200f9bbf80 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0d38009c60 tx=0x7f0d38011810 comp rx=0 tx=0).stop 2024-03-27T11:08:09.728 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.756+0000 7f0d4f9d1740 1 -- 172.21.15.97:0/3946315364 shutdown_connections 2024-03-27T11:08:09.728 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.756+0000 7f0d4f9d1740 1 --2- 172.21.15.97:0/3946315364 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56200f9c16c0 0x56200f9c3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:08:09.728 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.756+0000 7f0d4f9d1740 1 --2- 172.21.15.97:0/3946315364 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56200f9bc550 0x56200f9c1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:08:09.728 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.756+0000 7f0d4f9d1740 1 --2- 172.21.15.97:0/3946315364 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56200f9bbb60 0x56200f9bbf80 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:08:09.729 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.756+0000 7f0d4f9d1740 1 -- 172.21.15.97:0/3946315364 >> 172.21.15.97:0/3946315364 conn(0x56200f9b70d0 msgr2=0x56200f9b74d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:08:09.729 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.756+0000 7f0d4f9d1740 1 -- 172.21.15.97:0/3946315364 shutdown_connections 2024-03-27T11:08:09.729 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.756+0000 7f0d4f9d1740 1 -- 172.21.15.97:0/3946315364 wait complete. 2024-03-27T11:08:09.729 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.756+0000 7f0d4f9d1740 1 -- start start 2024-03-27T11:08:09.729 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.757+0000 7f0d4f9d1740 1 Processor -- start 2024-03-27T11:08:09.729 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.757+0000 7f0d4f9d1740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56200f9bbb60 0x56200f9c5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:08:09.729 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.757+0000 7f0d4f9d1740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56200f9bc550 0x56200f9c63e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:08:09.729 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.757+0000 7f0d4f9d1740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56200f9c16c0 0x56200f9cd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:08:09.729 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.757+0000 7f0d4f9d1740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56200f9c4670 con 0x56200f9c16c0 2024-03-27T11:08:09.729 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.757+0000 7f0d4f9d1740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x56200f8dc630 con 0x56200f9bbb60 2024-03-27T11:08:09.729 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.757+0000 7f0d4f9d1740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x56200f9c47f0 con 0x56200f9bc550 2024-03-27T11:08:09.729 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.757+0000 7f0d4f9d1740 10 client.0 _ll_register_callbacks cb 0x56200f8e1c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T11:08:09.729 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.757+0000 7f0d46800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56200f9c16c0 0x56200f9cd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:08:09.729 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.758+0000 7f0d45e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56200f9bbb60 0x56200f9c5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:08:09.730 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.758+0000 7f0d46800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56200f9c16c0 0x56200f9cd3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54676/0 (socket says 172.21.15.97:54676) 2024-03-27T11:08:09.730 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.758+0000 7f0d46800640 1 -- 172.21.15.97:0/3663839750 learned_addr learned my addr 172.21.15.97:0/3663839750 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:08:09.730 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.758+0000 7f0d3d400640 1 --2- 172.21.15.97:0/3663839750 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56200f9bc550 0x56200f9c63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:08:09.730 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.758+0000 7f0d46800640 1 -- 172.21.15.97:0/3663839750 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56200f9bc550 msgr2=0x56200f9c63e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:08:09.730 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.758+0000 7f0d46800640 1 --2- 172.21.15.97:0/3663839750 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56200f9bc550 0x56200f9c63e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:08:09.730 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.758+0000 7f0d46800640 1 -- 172.21.15.97:0/3663839750 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56200f9bbb60 msgr2=0x56200f9c5ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:08:09.730 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.758+0000 7f0d46800640 1 --2- 172.21.15.97:0/3663839750 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56200f9bbb60 0x56200f9c5ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:08:09.730 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.758+0000 7f0d46800640 1 -- 172.21.15.97:0/3663839750 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56200f8e85f0 con 0x56200f9c16c0 2024-03-27T11:08:09.730 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.758+0000 7f0d3d400640 1 --2- 172.21.15.97:0/3663839750 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56200f9bc550 0x56200f9c63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T11:08:09.730 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.758+0000 7f0d46800640 1 --2- 172.21.15.97:0/3663839750 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56200f9c16c0 0x56200f9cd3b0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f0d40034fd0 tx=0x7f0d400bdfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e948a5f89bf1e4c2 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:08:09.730 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.758+0000 7f0d2b400640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T11:08:09.730 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.759+0000 7f0d2b400640 1 -- 172.21.15.97:0/3663839750 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d400d7060 con 0x56200f9c16c0 2024-03-27T11:08:09.732 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.759+0000 7f0d2b400640 1 -- 172.21.15.97:0/3663839750 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f0d400c6ba0 con 0x56200f9c16c0 2024-03-27T11:08:09.733 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.759+0000 7f0d4f9d1740 1 -- 172.21.15.97:0/3663839750 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d380035f0 con 0x56200f9c16c0 2024-03-27T11:08:09.733 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.759+0000 7f0d2b400640 1 -- 172.21.15.97:0/3663839750 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f0d400dd170 con 0x56200f9c16c0 2024-03-27T11:08:09.733 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.759+0000 7f0d4f9d1740 10 client.16950 fetch_fsmap learned FSMap version 35 2024-03-27T11:08:09.733 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.759+0000 7f0d4f9d1740 1 -- 172.21.15.97:0/3663839750 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56200f635210 con 0x56200f9c16c0 2024-03-27T11:08:09.733 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.759+0000 7f0d2b400640 1 -- 172.21.15.97:0/3663839750 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0d380035f0 con 0x56200f9c16c0 2024-03-27T11:08:09.733 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.759+0000 7f0d2b400640 1 -- 172.21.15.97:0/3663839750 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 35) ==== 8358+0+0 (secure 0 0 0) 0x7f0d4003c030 con 0x56200f9c16c0 2024-03-27T11:08:09.733 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.759+0000 7f0d4f9d1740 10 client.16950 fetch_fsmap finished waiting for FSMap version 35 2024-03-27T11:08:09.733 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.759+0000 7f0d4f9d1740 10 client.16950 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T11:08:09.733 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.759+0000 7f0d4f9d1740 20 client.16950 populate_metadata read hostname 'smithi097' 2024-03-27T11:08:09.733 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.759+0000 7f0d4f9d1740 1 --2- 172.21.15.97:0/3663839750 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x56200f9d1c30 0x56200f9d4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:08:09.733 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.760+0000 7f0d4f9d1740 4 client.16950 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T11:08:09.733 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.760+0000 7f0d4f9d1740 1 -- 172.21.15.97:0/3663839750 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56200f92e250 con 0x56200f9d1c30 2024-03-27T11:08:09.733 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.760+0000 7f0d45e00640 1 --2- 172.21.15.97:0/3663839750 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x56200f9d1c30 0x56200f9d4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:08:09.733 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.761+0000 7f0d2b400640 10 client.16950 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T11:08:09.734 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.761+0000 7f0d45e00640 1 --2- 172.21.15.97:0/3663839750 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x56200f9d1c30 0x56200f9d4090 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7142b1c9de6c44f server_cookie=b97ba7b46492a320 in_seq=0 out_seq=0 2024-03-27T11:08:09.734 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.762+0000 7f0d2b400640 1 -- 172.21.15.97:0/3663839750 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x56200f92e250 con 0x56200f9d1c30 2024-03-27T11:08:09.734 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.762+0000 7f0d2b400640 10 client.16950 handle_command_reply: tid=0 2024-03-27T11:08:09.734 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.762+0000 7f0d4f9d1740 1 client.16950 shutdown 2024-03-27T11:08:09.734 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.762+0000 7f0d4f9d1740 1 -- 172.21.15.97:0/3663839750 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56200f9c16c0 msgr2=0x56200f9cd3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:08:09.734 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.762+0000 7f0d4f9d1740 1 --2- 172.21.15.97:0/3663839750 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56200f9c16c0 0x56200f9cd3b0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f0d40034fd0 tx=0x7f0d400bdfc0 comp rx=0 tx=0).stop 2024-03-27T11:08:09.734 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.762+0000 7f0d4f9d1740 1 -- 172.21.15.97:0/3663839750 shutdown_connections 2024-03-27T11:08:09.734 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.762+0000 7f0d4f9d1740 1 --2- 172.21.15.97:0/3663839750 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x56200f9d1c30 0x56200f9d4090 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:08:09.734 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.762+0000 7f0d4f9d1740 1 --2- 172.21.15.97:0/3663839750 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56200f9c16c0 0x56200f9cd3b0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:08:09.734 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.762+0000 7f0d4f9d1740 1 --2- 172.21.15.97:0/3663839750 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56200f9bc550 0x56200f9c63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:08:09.734 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.762+0000 7f0d4f9d1740 1 --2- 172.21.15.97:0/3663839750 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56200f9bbb60 0x56200f9c5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:08:09.734 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.762+0000 7f0d4f9d1740 1 -- 172.21.15.97:0/3663839750 >> 172.21.15.97:0/3663839750 conn(0x56200f9b70d0 msgr2=0x56200f9b6310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:08:09.734 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.762+0000 7f0d2b400640 0 client.16950 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T11:08:09.734 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.763+0000 7f0d4f9d1740 1 -- 172.21.15.97:0/3663839750 shutdown_connections 2024-03-27T11:08:09.735 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.763+0000 7f0d4f9d1740 1 -- 172.21.15.97:0/3663839750 wait complete. 2024-03-27T11:08:09.735 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.763+0000 7f0d4f9d1740 20 client.16950 trim_cache size 0 max 16384 2024-03-27T11:08:09.748 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.778+0000 7f0d4f9d1740 1 Processor -- start 2024-03-27T11:08:09.748 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.778+0000 7f0d4f9d1740 1 -- start start 2024-03-27T11:08:09.748 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.779+0000 7f0d4f9d1740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56200f9bbb60 0x56200f9bd870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:08:09.748 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.779+0000 7f0d4f9d1740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56200f9bc550 0x56200f9cd370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:08:09.749 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.779+0000 7f0d4f9d1740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56200f9c16c0 0x56200f9bddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:08:09.749 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.779+0000 7f0d4f9d1740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0d481aad40 con 0x56200f9bbb60 2024-03-27T11:08:09.749 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.779+0000 7f0d4f9d1740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f0d481ac8b0 con 0x56200f9c16c0 2024-03-27T11:08:09.749 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.779+0000 7f0d4f9d1740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f0d481ab410 con 0x56200f9bc550 2024-03-27T11:08:09.751 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.779+0000 7f0d45e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56200f9bbb60 0x56200f9bd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:08:09.751 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.779+0000 7f0d45e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56200f9bbb60 0x56200f9bd870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54684/0 (socket says 172.21.15.97:54684) 2024-03-27T11:08:09.751 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.779+0000 7f0d45e00640 1 -- 172.21.15.97:0/3798887729 learned_addr learned my addr 172.21.15.97:0/3798887729 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:08:09.751 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.779+0000 7f0d46800640 1 --2- 172.21.15.97:0/3798887729 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56200f9c16c0 0x56200f9bddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:08:09.751 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.779+0000 7f0d3d400640 1 --2- 172.21.15.97:0/3798887729 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56200f9bc550 0x56200f9cd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:08:09.751 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.781+0000 7f0d45e00640 1 -- 172.21.15.97:0/3798887729 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56200f9bc550 msgr2=0x56200f9cd370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:08:09.751 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.781+0000 7f0d45e00640 1 --2- 172.21.15.97:0/3798887729 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56200f9bc550 0x56200f9cd370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:08:09.751 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.781+0000 7f0d45e00640 1 -- 172.21.15.97:0/3798887729 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56200f9c16c0 msgr2=0x56200f9bddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:08:09.751 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.781+0000 7f0d45e00640 1 --2- 172.21.15.97:0/3798887729 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56200f9c16c0 0x56200f9bddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:08:09.751 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.781+0000 7f0d45e00640 1 -- 172.21.15.97:0/3798887729 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56200f9c1410 con 0x56200f9bbb60 2024-03-27T11:08:09.752 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.781+0000 7f0d3d400640 1 --2- 172.21.15.97:0/3798887729 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56200f9bc550 0x56200f9cd370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T11:08:09.752 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.781+0000 7f0d45e00640 1 --2- 172.21.15.97:0/3798887729 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56200f9bbb60 0x56200f9bd870 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f0d38009710 tx=0x7f0d38005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c05f2d8c4fdac89 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:08:09.752 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.781+0000 7f0d36600640 1 -- 172.21.15.97:0/3798887729 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d38009780 con 0x56200f9bbb60 2024-03-27T11:08:09.752 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.781+0000 7f0d36600640 1 -- 172.21.15.97:0/3798887729 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0d380054f0 con 0x56200f9bbb60 2024-03-27T11:08:09.752 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.781+0000 7f0d36600640 1 -- 172.21.15.97:0/3798887729 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d38026460 con 0x56200f9bbb60 2024-03-27T11:08:09.752 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.781+0000 7f0d4f9d1740 1 -- 172.21.15.97:0/3798887729 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56200f9bbb60 msgr2=0x56200f9bd870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:08:09.752 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.781+0000 7f0d4f9d1740 1 --2- 172.21.15.97:0/3798887729 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56200f9bbb60 0x56200f9bd870 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f0d38009710 tx=0x7f0d38005d30 comp rx=0 tx=0).stop 2024-03-27T11:08:09.752 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.781+0000 7f0d4f9d1740 1 -- 172.21.15.97:0/3798887729 shutdown_connections 2024-03-27T11:08:09.752 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.781+0000 7f0d4f9d1740 1 --2- 172.21.15.97:0/3798887729 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56200f9c16c0 0x56200f9bddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:08:09.752 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.781+0000 7f0d4f9d1740 1 --2- 172.21.15.97:0/3798887729 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56200f9bc550 0x56200f9cd370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:08:09.752 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.781+0000 7f0d4f9d1740 1 --2- 172.21.15.97:0/3798887729 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56200f9bbb60 0x56200f9bd870 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:08:09.752 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.781+0000 7f0d4f9d1740 1 -- 172.21.15.97:0/3798887729 >> 172.21.15.97:0/3798887729 conn(0x56200f9b70d0 msgr2=0x56200f9c5ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:08:09.752 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.781+0000 7f0d4f9d1740 1 -- 172.21.15.97:0/3798887729 shutdown_connections 2024-03-27T11:08:09.752 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.781+0000 7f0d4f9d1740 1 -- 172.21.15.97:0/3798887729 wait complete. 2024-03-27T11:08:09.756 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.782+0000 7f0d4f9d1740 1 -- start start 2024-03-27T11:08:09.756 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.782+0000 7f0d4f9d1740 1 Processor -- start 2024-03-27T11:08:09.756 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.783+0000 7f0d4f9d1740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56200f9bbb60 0x56200f9ca6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:08:09.756 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.783+0000 7f0d4f9d1740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56200f9bc550 0x56200f9cac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:08:09.756 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.783+0000 7f0d4f9d1740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56200f9c16c0 0x56200f9cbb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:08:09.756 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.783+0000 7f0d4f9d1740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0d481ac1e0 con 0x56200f9bc550 2024-03-27T11:08:09.756 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.783+0000 7f0d4f9d1740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f0d481abae0 con 0x56200f9bbb60 2024-03-27T11:08:09.756 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.783+0000 7f0d4f9d1740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f0d481a8d20 con 0x56200f9c16c0 2024-03-27T11:08:09.756 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.783+0000 7f0d4f9d1740 10 client.0 _ll_register_callbacks cb 0x7f0d40009900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T11:08:09.756 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.783+0000 7f0d3d400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56200f9bc550 0x56200f9cac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:08:09.756 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.783+0000 7f0d3d400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56200f9bc550 0x56200f9cac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54692/0 (socket says 172.21.15.97:54692) 2024-03-27T11:08:09.756 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.783+0000 7f0d45e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56200f9bbb60 0x56200f9ca6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:08:09.757 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.783+0000 7f0d3d400640 1 -- 172.21.15.97:0/2977570273 learned_addr learned my addr 172.21.15.97:0/2977570273 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:08:09.757 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.783+0000 7f0d46800640 1 --2- 172.21.15.97:0/2977570273 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56200f9c16c0 0x56200f9cbb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:08:09.757 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.784+0000 7f0d3d400640 1 -- 172.21.15.97:0/2977570273 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56200f9c16c0 msgr2=0x56200f9cbb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:08:09.757 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.784+0000 7f0d3d400640 1 --2- 172.21.15.97:0/2977570273 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56200f9c16c0 0x56200f9cbb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:08:09.757 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.784+0000 7f0d3d400640 1 -- 172.21.15.97:0/2977570273 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56200f9bbb60 msgr2=0x56200f9ca6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T11:08:09.757 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.784+0000 7f0d3d400640 1 --2- 172.21.15.97:0/2977570273 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56200f9bbb60 0x56200f9ca6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:08:09.757 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.784+0000 7f0d3d400640 1 -- 172.21.15.97:0/2977570273 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56200f701620 con 0x56200f9bc550 2024-03-27T11:08:09.757 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.784+0000 7f0d3d400640 1 --2- 172.21.15.97:0/2977570273 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56200f9bc550 0x56200f9cac00 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f0d30010c80 tx=0x7f0d30005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69ba1afecb9dbdc2 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:08:09.757 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.784+0000 7f0d2b400640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-27T11:08:09.757 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.784+0000 7f0d2b400640 1 -- 172.21.15.97:0/2977570273 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d30013e50 con 0x56200f9bc550 2024-03-27T11:08:09.757 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.784+0000 7f0d2b400640 1 -- 172.21.15.97:0/2977570273 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f0d30005020 con 0x56200f9bc550 2024-03-27T11:08:09.757 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.784+0000 7f0d4f9d1740 1 -- 172.21.15.97:0/2977570273 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d380054f0 con 0x56200f9bc550 2024-03-27T11:08:09.757 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.785+0000 7f0d2b400640 1 -- 172.21.15.97:0/2977570273 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f0d30005220 con 0x56200f9bc550 2024-03-27T11:08:09.757 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.785+0000 7f0d4f9d1740 10 client.16956 fetch_fsmap learned FSMap version 35 2024-03-27T11:08:09.758 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.785+0000 7f0d4f9d1740 1 -- 172.21.15.97:0/2977570273 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56200f9cd320 con 0x56200f9bc550 2024-03-27T11:08:09.758 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.785+0000 7f0d2b400640 1 -- 172.21.15.97:0/2977570273 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0d380054f0 con 0x56200f9bc550 2024-03-27T11:08:09.758 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.785+0000 7f0d2b400640 1 -- 172.21.15.97:0/2977570273 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 35) ==== 8358+0+0 (secure 0 0 0) 0x7f0d30021440 con 0x56200f9bc550 2024-03-27T11:08:09.758 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.785+0000 7f0d4f9d1740 10 client.16956 fetch_fsmap finished waiting for FSMap version 35 2024-03-27T11:08:09.758 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.785+0000 7f0d4f9d1740 10 client.16956 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T11:08:09.758 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.785+0000 7f0d4f9d1740 20 client.16956 populate_metadata read hostname 'smithi097' 2024-03-27T11:08:09.758 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.785+0000 7f0d4f9d1740 1 --2- 172.21.15.97:0/2977570273 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x56200fa383b0 0x56200fa3a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:08:09.758 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.785+0000 7f0d4f9d1740 4 client.16956 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-27T11:08:09.758 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.785+0000 7f0d4f9d1740 1 -- 172.21.15.97:0/2977570273 --> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56200f9285a0 con 0x56200fa383b0 2024-03-27T11:08:09.758 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.786+0000 7f0d45e00640 1 --2- 172.21.15.97:0/2977570273 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x56200fa383b0 0x56200fa3a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:08:09.759 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.787+0000 7f0d2b400640 10 client.16956 ms_handle_connect on v2:172.21.15.114:6838/3937193784 2024-03-27T11:08:09.759 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.787+0000 7f0d45e00640 1 --2- 172.21.15.97:0/2977570273 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x56200fa383b0 0x56200fa3a7d0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee2de8ae575c61a server_cookie=d4d24b86a173132b in_seq=0 out_seq=0 2024-03-27T11:08:09.764 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:08:09 smithi114 ceph-mon[35338]: pgmap v786: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 45 MiB/s wr, 812 op/s 2024-03-27T11:08:09.797 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.827+0000 7f0d2b400640 1 -- 172.21.15.97:0/2977570273 <== mds.0 v2:172.21.15.114:6838/3937193784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56200f9285a0 con 0x56200fa383b0 2024-03-27T11:08:09.797 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.827+0000 7f0d2b400640 10 client.16956 handle_command_reply: tid=0 2024-03-27T11:08:09.797 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.827+0000 7f0d4f9d1740 1 client.16956 shutdown 2024-03-27T11:08:09.798 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.828+0000 7f0d4f9d1740 1 -- 172.21.15.97:0/2977570273 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56200f9bc550 msgr2=0x56200f9cac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:08:09.798 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.828+0000 7f0d4f9d1740 1 --2- 172.21.15.97:0/2977570273 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56200f9bc550 0x56200f9cac00 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f0d30010c80 tx=0x7f0d30005e40 comp rx=0 tx=0).stop 2024-03-27T11:08:09.798 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.828+0000 7f0d4f9d1740 1 -- 172.21.15.97:0/2977570273 shutdown_connections 2024-03-27T11:08:09.798 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.828+0000 7f0d4f9d1740 1 --2- 172.21.15.97:0/2977570273 >> [v2:172.21.15.114:6838/3937193784,v1:172.21.15.114:6839/3937193784] conn(0x56200fa383b0 0x56200fa3a7d0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:08:09.798 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.828+0000 7f0d4f9d1740 1 --2- 172.21.15.97:0/2977570273 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56200f9c16c0 0x56200f9cbb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:08:09.798 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.828+0000 7f0d4f9d1740 1 --2- 172.21.15.97:0/2977570273 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56200f9bc550 0x56200f9cac00 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:08:09.798 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.828+0000 7f0d4f9d1740 1 --2- 172.21.15.97:0/2977570273 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56200f9bbb60 0x56200f9ca6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:08:09.798 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.828+0000 7f0d2b400640 0 client.16956 ms_handle_reset on v2:172.21.15.114:6838/3937193784 2024-03-27T11:08:09.798 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.828+0000 7f0d4f9d1740 1 -- 172.21.15.97:0/2977570273 >> 172.21.15.97:0/2977570273 conn(0x56200f9b70d0 msgr2=0x56200f9c5ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:08:09.798 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.828+0000 7f0d4f9d1740 1 -- 172.21.15.97:0/2977570273 shutdown_connections 2024-03-27T11:08:09.798 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.828+0000 7f0d4f9d1740 1 -- 172.21.15.97:0/2977570273 wait complete. 2024-03-27T11:08:09.798 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.828+0000 7f0d4f9d1740 20 client.16956 trim_cache size 0 max 16384 2024-03-27T11:08:09.799 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-03-27T11:08:09.799 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-03-27T11:08:09.799 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-03-27T11:08:09.799 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-03-27T11:08:09.800 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.829+0000 7f0d47200640 1 -- 172.21.15.97:0/323998158 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f0d2c0a3680 msgr2=0x7f0d2c0a5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:08:09.800 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.829+0000 7f0d47200640 1 --2- 172.21.15.97:0/323998158 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f0d2c0a3680 0x7f0d2c0a5b40 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f0d300058c0 tx=0x7f0d3000ba80 comp rx=0 tx=0).stop 2024-03-27T11:08:09.800 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.829+0000 7f0d47200640 1 -- 172.21.15.97:0/323998158 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d48103570 msgr2=0x7f0d481a1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:08:09.800 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.829+0000 7f0d47200640 1 --2- 172.21.15.97:0/323998158 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d48103570 0x7f0d481a1310 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f0d40032380 tx=0x7f0d40032820 comp rx=0 tx=0).stop 2024-03-27T11:08:09.800 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.829+0000 7f0d47200640 1 -- 172.21.15.97:0/323998158 shutdown_connections 2024-03-27T11:08:09.800 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.829+0000 7f0d47200640 1 --2- 172.21.15.97:0/323998158 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d48103570 0x7f0d481a1310 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:08:09.800 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.829+0000 7f0d47200640 1 --2- 172.21.15.97:0/323998158 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0d48102bb0 0x7f0d4819a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:08:09.801 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.829+0000 7f0d47200640 1 --2- 172.21.15.97:0/323998158 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f0d2c0a3680 0x7f0d2c0a5b40 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:08:09.801 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.829+0000 7f0d47200640 1 --2- 172.21.15.97:0/323998158 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0d481019c0 0x7f0d48199da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:08:09.801 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.829+0000 7f0d47200640 1 -- 172.21.15.97:0/323998158 >> 172.21.15.97:0/323998158 conn(0x7f0d480fcf30 msgr2=0x7f0d48108350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:08:09.801 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.829+0000 7f0d47200640 1 -- 172.21.15.97:0/323998158 shutdown_connections 2024-03-27T11:08:09.801 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-03-27T11:08:09.829+0000 7f0d47200640 1 -- 172.21.15.97:0/323998158 wait complete. 2024-03-27T11:08:09.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:08:09 smithi097 ceph-mon[31859]: pgmap v786: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 45 MiB/s wr, 812 op/s 2024-03-27T11:08:09.818 INFO:tasks.workunit:Stopping ['suites/iogen.sh'] on client.0... 2024-03-27T11:08:09.819 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-03-27T11:08:10.764 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:08:10 smithi114 ceph-mon[35338]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T11:08:10.887 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:08:10 smithi097 ceph-mon[31859]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T11:08:10.891 DEBUG:teuthology.parallel:result is None 2024-03-27T11:08:10.891 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-03-27T11:08:10.923 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:08:10 smithi177 ceph-mon[35294]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T11:08:11.111 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-03-27T11:08:11.111 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-03-27T11:08:11.122 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi097.front.sepia.ceph.com 2024-03-27T11:08:11.144 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-03-27T11:08:11.153 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-03-27T11:08:11.153 ERROR:teuthology.run_tasks:Manager failed: fwd_scrub Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_155268c4e432a12433aa833f174f9fe3b1016ae0/qa/tasks/fwd_scrub.py", line 164, in task stop_all_fwd_scrubbers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_155268c4e432a12433aa833f174f9fe3b1016ae0/qa/tasks/fwd_scrub.py", line 99, in stop_all_fwd_scrubbers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: rank damage found: {'backtrace'} 2024-03-27T11:08:11.154 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-03-27T11:08:11.163 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-27T11:08:11.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:11.422+0000 7f34b5400640 1 -- 172.21.15.97:0/2610917030 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34b0104d50 msgr2=0x7f34b0107140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:08:11.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:11.422+0000 7f34b5400640 1 --2- 172.21.15.97:0/2610917030 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34b0104d50 0x7f34b0107140 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f349c009900 tx=0x7f349c02bc30 comp rx=0 tx=0).stop 2024-03-27T11:08:11.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:11.422+0000 7f34b5400640 1 -- 172.21.15.97:0/2610917030 shutdown_connections 2024-03-27T11:08:11.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:11.422+0000 7f34b5400640 1 --2- 172.21.15.97:0/2610917030 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f34b0107680 0x7f34b0109a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:08:11.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:11.422+0000 7f34b5400640 1 --2- 172.21.15.97:0/2610917030 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34b0104d50 0x7f34b0107140 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:08:11.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:11.422+0000 7f34b5400640 1 --2- 172.21.15.97:0/2610917030 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f34b0102420 0x7f34b0104810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:08:11.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:11.422+0000 7f34b5400640 1 -- 172.21.15.97:0/2610917030 >> 172.21.15.97:0/2610917030 conn(0x7f34b00fbdd0 msgr2=0x7f34b00fe1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:08:11.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:11.422+0000 7f34b5400640 1 -- 172.21.15.97:0/2610917030 shutdown_connections 2024-03-27T11:08:11.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:11.422+0000 7f34b5400640 1 -- 172.21.15.97:0/2610917030 wait complete. 2024-03-27T11:08:11.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:11.423+0000 7f34b5400640 1 Processor -- start 2024-03-27T11:08:11.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:11.423+0000 7f34b5400640 1 -- start start 2024-03-27T11:08:11.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:11.424+0000 7f34b5400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f34b0102420 0x7f34b0101d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:08:11.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:11.424+0000 7f34b5400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f34b0104d50 0x7f34b01003d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:08:11.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:11.424+0000 7f34b5400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34b0107680 0x7f34b0100910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:08:11.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:11.424+0000 7f34b5400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f34b010d4c0 con 0x7f34b0107680 2024-03-27T11:08:11.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:11.424+0000 7f34b5400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f34b010d340 con 0x7f34b0104d50 2024-03-27T11:08:11.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:11.424+0000 7f34b5400640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f34b010d640 con 0x7f34b0102420 2024-03-27T11:08:11.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:11.424+0000 7f34b4a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34b0107680 0x7f34b0100910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:08:11.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:11.424+0000 7f34b4a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34b0107680 0x7f34b0100910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54720/0 (socket says 172.21.15.97:54720) 2024-03-27T11:08:11.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:11.424+0000 7f34b4a00640 1 -- 172.21.15.97:0/1310792602 learned_addr learned my addr 172.21.15.97:0/1310792602 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:08:11.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:11.424+0000 7f34af400640 1 --2- 172.21.15.97:0/1310792602 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f34b0104d50 0x7f34b01003d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:08:11.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:11.424+0000 7f34afe00640 1 --2- 172.21.15.97:0/1310792602 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f34b0102420 0x7f34b0101d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:08:11.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:11.425+0000 7f34b4a00640 1 -- 172.21.15.97:0/1310792602 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f34b0102420 msgr2=0x7f34b0101d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:08:11.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:11.425+0000 7f34b4a00640 1 --2- 172.21.15.97:0/1310792602 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f34b0102420 0x7f34b0101d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:08:11.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:11.425+0000 7f34b4a00640 1 -- 172.21.15.97:0/1310792602 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f34b0104d50 msgr2=0x7f34b01003d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:08:11.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:11.425+0000 7f34b4a00640 1 --2- 172.21.15.97:0/1310792602 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f34b0104d50 0x7f34b01003d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:08:11.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:11.425+0000 7f34b4a00640 1 -- 172.21.15.97:0/1310792602 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34b01011a0 con 0x7f34b0107680 2024-03-27T11:08:11.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:11.425+0000 7f34af400640 1 --2- 172.21.15.97:0/1310792602 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f34b0104d50 0x7f34b01003d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T11:08:11.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:11.425+0000 7f34b4a00640 1 --2- 172.21.15.97:0/1310792602 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34b0107680 0x7f34b0100910 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f34a800eaa0 tx=0x7f34a800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:08:11.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:11.425+0000 7f34acc00640 1 -- 172.21.15.97:0/1310792602 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a800cdb0 con 0x7f34b0107680 2024-03-27T11:08:11.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:11.425+0000 7f34b5400640 1 -- 172.21.15.97:0/1310792602 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34b01a1c90 con 0x7f34b0107680 2024-03-27T11:08:11.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:11.425+0000 7f34acc00640 1 -- 172.21.15.97:0/1310792602 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f34a8005020 con 0x7f34b0107680 2024-03-27T11:08:11.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:11.425+0000 7f34acc00640 1 -- 172.21.15.97:0/1310792602 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a80116d0 con 0x7f34b0107680 2024-03-27T11:08:11.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:11.425+0000 7f34b5400640 1 -- 172.21.15.97:0/1310792602 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34b01a21b0 con 0x7f34b0107680 2024-03-27T11:08:11.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:11.427+0000 7f34b5400640 1 -- 172.21.15.97:0/1310792602 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34b0068cb0 con 0x7f34b0107680 2024-03-27T11:08:11.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:11.429+0000 7f34acc00640 1 -- 172.21.15.97:0/1310792602 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f34a80051d0 con 0x7f34b0107680 2024-03-27T11:08:11.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:11.431+0000 7f34acc00640 1 --2- 172.21.15.97:0/1310792602 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f34940a3610 0x7f34940a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:08:11.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:11.431+0000 7f34acc00640 1 -- 172.21.15.97:0/1310792602 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f34a80cffb0 con 0x7f34b0107680 2024-03-27T11:08:11.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:11.431+0000 7f34afe00640 1 --2- 172.21.15.97:0/1310792602 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f34940a3610 0x7f34940a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:08:11.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:11.432+0000 7f34afe00640 1 --2- 172.21.15.97:0/1310792602 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f34940a3610 0x7f34940a5ad0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f34a0005d00 tx=0x7f34a0005c50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:08:11.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:11.435+0000 7f34acc00640 1 -- 172.21.15.97:0/1310792602 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f34a8016030 con 0x7f34b0107680 2024-03-27T11:08:11.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:11.653+0000 7f34b5400640 1 -- 172.21.15.97:0/1310792602 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f34b0062780 con 0x7f34b0107680 2024-03-27T11:08:11.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:11.658+0000 7f34acc00640 1 -- 172.21.15.97:0/1310792602 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+22644 (secure 0 0 0) 0x7f34a809cb30 con 0x7f34b0107680 2024-03-27T11:08:11.627 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T11:08:11.627 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":92,"fsid":"adca88c4-ec26-11ee-95d0-87774f69a715","created":"2024-03-27T10:43:24.680694+0000","modified":"2024-03-27T10:54:41.310812+0000","last_up_change":"2024-03-27T10:53:00.552513+0000","last_in_change":"2024-03-27T10:52:36.302986+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-27T10:47:54.052810+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-27T10:54:02.845060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-27T10:54:03.474151+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b634e5f1-6118-4e2f-95e9-830db390f8d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1746812080},{"type":"v1","addr":"172.21.15.97:6803","nonce":1746812080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1746812080},{"type":"v1","addr":"172.21.15.97:6805","nonce":1746812080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":1746812080},{"type":"v1","addr":"172.21.15.97:6809","nonce":1746812080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1746812080},{"type":"v1","addr":"172.21.15.97:6807","nonce":1746812080}]},"public_addr":"172.21.15.97:6803/1746812080","cluster_addr":"172.21.15.97:6805/1746812080","heartbeat_back_addr":"172.21.15.97:6809/1746812080","heartbeat_front_addr":"172.21.15.97:6807/1746812080","state":["exists","up"]},{"osd":1,"uuid":"441e2770-3284-4388-b0de-c56eb8a373d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":3858821264},{"type":"v1","addr":"172.21.15.114:6801","nonce":3858821264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":3858821264},{"type":"v1","addr":"172.21.15.114:6803","nonce":3858821264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":3858821264},{"type":"v1","addr":"172.21.15.114:6807","nonce":3858821264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":3858821264},{"type":"v1","addr":"172.21.15.114:6805","nonce":3858821264}]},"public_addr":"172.21.15.114:6801/3858821264","cluster_addr":"172.21.15.114:6803/3858821264","heartbeat_back_addr":"172.21.15.114:6807/3858821264","heartbeat_front_addr":"172.21.15.114:6805/3858821264","state":["exists","up"]},{"osd":2,"uuid":"00108864-1aa6-48d6-8398-e8eb79fda123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":124240658},{"type":"v1","addr":"172.21.15.177:6801","nonce":124240658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":124240658},{"type":"v1","addr":"172.21.15.177:6803","nonce":124240658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":124240658},{"type":"v1","addr":"172.21.15.177:6807","nonce":124240658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":124240658},{"type":"v1","addr":"172.21.15.177:6805","nonce":124240658}]},"public_addr":"172.21.15.177:6801/124240658","cluster_addr":"172.21.15.177:6803/124240658","heartbeat_back_addr":"172.21.15.177:6807/124240658","heartbeat_front_addr":"172.21.15.177:6805/124240658","state":["exists","up"]},{"osd":3,"uuid":"1860b690-e2dc-4977-88d7-d1aacf7022ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2478179311},{"type":"v1","addr":"172.21.15.97:6811","nonce":2478179311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2478179311},{"type":"v1","addr":"172.21.15.97:6813","nonce":2478179311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":2478179311},{"type":"v1","addr":"172.21.15.97:6817","nonce":2478179311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2478179311},{"type":"v1","addr":"172.21.15.97:6815","nonce":2478179311}]},"public_addr":"172.21.15.97:6811/2478179311","cluster_addr":"172.21.15.97:6813/2478179311","heartbeat_back_addr":"172.21.15.97:6817/2478179311","heartbeat_front_addr":"172.21.15.97:6815/2478179311","state":["exists","up"]},{"osd":4,"uuid":"9245a3a4-7d49-4fdb-9d55-112e43177050","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":75755945},{"type":"v1","addr":"172.21.15.114:6809","nonce":75755945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":75755945},{"type":"v1","addr":"172.21.15.114:6811","nonce":75755945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":75755945},{"type":"v1","addr":"172.21.15.114:6815","nonce":75755945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":75755945},{"type":"v1","addr":"172.21.15.114:6813","nonce":75755945}]},"public_addr":"172.21.15.114:6809/75755945","cluster_addr":"172.21.15.114:6811/75755945","heartbeat_back_addr":"172.21.15.114:6815/75755945","heartbeat_front_addr":"172.21.15.114:6813/75755945","state":["exists","up"]},{"osd":5,"uuid":"51922f99-e8d3-461b-8615-e9bfdcbf5ed3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":1918493790},{"type":"v1","addr":"172.21.15.177:6809","nonce":1918493790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":1918493790},{"type":"v1","addr":"172.21.15.177:6811","nonce":1918493790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":1918493790},{"type":"v1","addr":"172.21.15.177:6815","nonce":1918493790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":1918493790},{"type":"v1","addr":"172.21.15.177:6813","nonce":1918493790}]},"public_addr":"172.21.15.177:6809/1918493790","cluster_addr":"172.21.15.177:6811/1918493790","heartbeat_back_addr":"172.21.15.177:6815/1918493790","heartbeat_front_addr":"172.21.15.177:6813/1918493790","state":["exists","up"]},{"osd":6,"uuid":"06dffb32-05eb-43b3-852e-915c5cc7f25a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2892389563},{"type":"v1","addr":"172.21.15.97:6819","nonce":2892389563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2892389563},{"type":"v1","addr":"172.21.15.97:6821","nonce":2892389563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2892389563},{"type":"v1","addr":"172.21.15.97:6825","nonce":2892389563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2892389563},{"type":"v1","addr":"172.21.15.97:6823","nonce":2892389563}]},"public_addr":"172.21.15.97:6819/2892389563","cluster_addr":"172.21.15.97:6821/2892389563","heartbeat_back_addr":"172.21.15.97:6825/2892389563","heartbeat_front_addr":"172.21.15.97:6823/2892389563","state":["exists","up"]},{"osd":7,"uuid":"e68e3519-9c1b-43c5-8e06-5385248b19df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":2445491153},{"type":"v1","addr":"172.21.15.114:6817","nonce":2445491153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":2445491153},{"type":"v1","addr":"172.21.15.114:6819","nonce":2445491153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2445491153},{"type":"v1","addr":"172.21.15.114:6823","nonce":2445491153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":2445491153},{"type":"v1","addr":"172.21.15.114:6821","nonce":2445491153}]},"public_addr":"172.21.15.114:6817/2445491153","cluster_addr":"172.21.15.114:6819/2445491153","heartbeat_back_addr":"172.21.15.114:6823/2445491153","heartbeat_front_addr":"172.21.15.114:6821/2445491153","state":["exists","up"]},{"osd":8,"uuid":"31ac592e-1349-49ee-9af9-8ad85d682978","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":2051134191},{"type":"v1","addr":"172.21.15.177:6817","nonce":2051134191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":2051134191},{"type":"v1","addr":"172.21.15.177:6819","nonce":2051134191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":2051134191},{"type":"v1","addr":"172.21.15.177:6823","nonce":2051134191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":2051134191},{"type":"v1","addr":"172.21.15.177:6821","nonce":2051134191}]},"public_addr":"172.21.15.177:6817/2051134191","cluster_addr":"172.21.15.177:6819/2051134191","heartbeat_back_addr":"172.21.15.177:6823/2051134191","heartbeat_front_addr":"172.21.15.177:6821/2051134191","state":["exists","up"]},{"osd":9,"uuid":"0171086a-dd41-4eb4-a389-16f0c46a64db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":1578881271},{"type":"v1","addr":"172.21.15.97:6827","nonce":1578881271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1578881271},{"type":"v1","addr":"172.21.15.97:6829","nonce":1578881271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1578881271},{"type":"v1","addr":"172.21.15.97:6833","nonce":1578881271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1578881271},{"type":"v1","addr":"172.21.15.97:6831","nonce":1578881271}]},"public_addr":"172.21.15.97:6827/1578881271","cluster_addr":"172.21.15.97:6829/1578881271","heartbeat_back_addr":"172.21.15.97:6833/1578881271","heartbeat_front_addr":"172.21.15.97:6831/1578881271","state":["exists","up"]},{"osd":10,"uuid":"64754e31-b124-4601-ad98-2f70b1dd5359","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":2893111461},{"type":"v1","addr":"172.21.15.114:6825","nonce":2893111461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":2893111461},{"type":"v1","addr":"172.21.15.114:6827","nonce":2893111461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2893111461},{"type":"v1","addr":"172.21.15.114:6831","nonce":2893111461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":2893111461},{"type":"v1","addr":"172.21.15.114:6829","nonce":2893111461}]},"public_addr":"172.21.15.114:6825/2893111461","cluster_addr":"172.21.15.114:6827/2893111461","heartbeat_back_addr":"172.21.15.114:6831/2893111461","heartbeat_front_addr":"172.21.15.114:6829/2893111461","state":["exists","up"]},{"osd":11,"uuid":"9ffc752c-adf7-4037-89d3-535787d7475d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":85,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":2333030391},{"type":"v1","addr":"172.21.15.177:6825","nonce":2333030391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":2333030391},{"type":"v1","addr":"172.21.15.177:6827","nonce":2333030391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":2333030391},{"type":"v1","addr":"172.21.15.177:6831","nonce":2333030391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":2333030391},{"type":"v1","addr":"172.21.15.177:6829","nonce":2333030391}]},"public_addr":"172.21.15.177:6825/2333030391","cluster_addr":"172.21.15.177:6827/2333030391","heartbeat_back_addr":"172.21.15.177:6831/2333030391","heartbeat_front_addr":"172.21.15.177:6829/2333030391","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:46:45.143519+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:47:18.656947+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:47:51.316345+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:48:24.575512+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:48:59.113555+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:49:32.103865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:50:06.615539+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:50:41.578341+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:51:15.254011+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:51:49.099165+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:52:24.544821+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:52:58.838484+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.28","mappings":[{"from":3,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/359646":"2024-03-28T10:44:39.889550+0000","172.21.15.97:6801/98682155":"2024-03-28T10:44:06.019075+0000","172.21.15.97:6801/1685921853":"2024-03-28T10:44:39.889550+0000","172.21.15.97:6800/98682155":"2024-03-28T10:44:06.019075+0000","172.21.15.97:0/107690680":"2024-03-28T10:44:06.019075+0000","172.21.15.97:6800/1685921853":"2024-03-28T10:44:39.889550+0000","172.21.15.97:0/437043413":"2024-03-28T10:44:39.889550+0000","172.21.15.97:0/2319706992":"2024-03-28T10:44:39.889550+0000","172.21.15.97:0/3911532654":"2024-03-28T10:44:06.019075+0000","172.21.15.97:0/3347321845":"2024-03-28T10:44:06.019075+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-27T11:08:11.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:11.661+0000 7f34b5400640 1 -- 172.21.15.97:0/1310792602 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f34940a3610 msgr2=0x7f34940a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:08:11.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:11.661+0000 7f34b5400640 1 --2- 172.21.15.97:0/1310792602 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f34940a3610 0x7f34940a5ad0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f34a0005d00 tx=0x7f34a0005c50 comp rx=0 tx=0).stop 2024-03-27T11:08:11.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:11.661+0000 7f34b5400640 1 -- 172.21.15.97:0/1310792602 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34b0107680 msgr2=0x7f34b0100910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:08:11.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:11.661+0000 7f34b5400640 1 --2- 172.21.15.97:0/1310792602 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34b0107680 0x7f34b0100910 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f34a800eaa0 tx=0x7f34a800ef60 comp rx=0 tx=0).stop 2024-03-27T11:08:11.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:11.661+0000 7f34b5400640 1 -- 172.21.15.97:0/1310792602 shutdown_connections 2024-03-27T11:08:11.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:11.661+0000 7f34b5400640 1 --2- 172.21.15.97:0/1310792602 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34b0107680 0x7f34b0100910 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:08:11.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:11.661+0000 7f34b5400640 1 --2- 172.21.15.97:0/1310792602 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f34940a3610 0x7f34940a5ad0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:08:11.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:11.661+0000 7f34b5400640 1 --2- 172.21.15.97:0/1310792602 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f34b0104d50 0x7f34b01003d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:08:11.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:11.661+0000 7f34b5400640 1 --2- 172.21.15.97:0/1310792602 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f34b0102420 0x7f34b0101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:08:11.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:11.661+0000 7f34b5400640 1 -- 172.21.15.97:0/1310792602 >> 172.21.15.97:0/1310792602 conn(0x7f34b00fbdd0 msgr2=0x7f34b0102cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:08:11.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:11.661+0000 7f34b5400640 1 -- 172.21.15.97:0/1310792602 shutdown_connections 2024-03-27T11:08:11.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:11.661+0000 7f34b5400640 1 -- 172.21.15.97:0/1310792602 wait complete. 2024-03-27T11:08:11.648 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-27T10:47:54.052810+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-27T10:54:02.845060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-27T10:54:03.474151+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-27T11:08:11.648 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-27T11:08:11.711 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:08:11 smithi097 ceph-mon[31859]: pgmap v787: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 45 MiB/s wr, 815 op/s 2024-03-27T11:08:11.763 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:08:11 smithi114 ceph-mon[35338]: pgmap v787: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 45 MiB/s wr, 815 op/s 2024-03-27T11:08:11.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:11.876+0000 7facab200640 1 -- 172.21.15.97:0/3015289133 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7facac1035e0 msgr2=0x7facac109e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:08:11.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:11.876+0000 7facab200640 1 --2- 172.21.15.97:0/3015289133 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7facac1035e0 0x7facac109e70 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7faca4009be0 tx=0x7faca402bef0 comp rx=0 tx=0).stop 2024-03-27T11:08:11.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:11.876+0000 7facab200640 1 -- 172.21.15.97:0/3015289133 shutdown_connections 2024-03-27T11:08:11.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:11.876+0000 7facab200640 1 --2- 172.21.15.97:0/3015289133 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7facac1035e0 0x7facac109e70 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:08:11.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:11.876+0000 7facab200640 1 --2- 172.21.15.97:0/3015289133 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7facac102c20 0x7facac1030a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:08:11.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:11.876+0000 7facab200640 1 --2- 172.21.15.97:0/3015289133 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7facac101a30 0x7facac101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:08:11.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:11.876+0000 7facab200640 1 -- 172.21.15.97:0/3015289133 >> 172.21.15.97:0/3015289133 conn(0x7facac0fcfc0 msgr2=0x7facac0ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:08:11.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:11.876+0000 7facab200640 1 -- 172.21.15.97:0/3015289133 shutdown_connections 2024-03-27T11:08:11.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:11.877+0000 7facab200640 1 -- 172.21.15.97:0/3015289133 wait complete. 2024-03-27T11:08:11.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:11.877+0000 7facab200640 1 Processor -- start 2024-03-27T11:08:11.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:11.878+0000 7facab200640 1 -- start start 2024-03-27T11:08:11.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:11.878+0000 7facab200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7facac101a30 0x7facac199de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:08:11.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:11.878+0000 7facab200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7facac102c20 0x7facac19a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:08:11.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:11.878+0000 7facab200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7facac1035e0 0x7facac1a1390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:08:11.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:11.878+0000 7facab200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7facac10d5b0 con 0x7facac101a30 2024-03-27T11:08:11.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:11.878+0000 7facab200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7facac10d430 con 0x7facac1035e0 2024-03-27T11:08:11.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:11.878+0000 7facab200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7facac10d730 con 0x7facac102c20 2024-03-27T11:08:11.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:11.878+0000 7faca9e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7facac101a30 0x7facac199de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:08:11.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:11.878+0000 7faca9e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7facac101a30 0x7facac199de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54732/0 (socket says 172.21.15.97:54732) 2024-03-27T11:08:11.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:11.878+0000 7faca9e00640 1 -- 172.21.15.97:0/3831934455 learned_addr learned my addr 172.21.15.97:0/3831934455 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:08:11.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:11.878+0000 7facaa800640 1 --2- 172.21.15.97:0/3831934455 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7facac1035e0 0x7facac1a1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:08:11.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:11.878+0000 7faca9400640 1 --2- 172.21.15.97:0/3831934455 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7facac102c20 0x7facac19a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:08:11.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:11.879+0000 7faca9e00640 1 -- 172.21.15.97:0/3831934455 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7facac102c20 msgr2=0x7facac19a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:08:11.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:11.879+0000 7faca9e00640 1 --2- 172.21.15.97:0/3831934455 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7facac102c20 0x7facac19a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:08:11.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:11.879+0000 7faca9e00640 1 -- 172.21.15.97:0/3831934455 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7facac1035e0 msgr2=0x7facac1a1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:08:11.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:11.879+0000 7faca9e00640 1 --2- 172.21.15.97:0/3831934455 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7facac1035e0 0x7facac1a1390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:08:11.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:11.879+0000 7faca9e00640 1 -- 172.21.15.97:0/3831934455 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facac1a1a00 con 0x7facac101a30 2024-03-27T11:08:11.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:11.879+0000 7faca9e00640 1 --2- 172.21.15.97:0/3831934455 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7facac101a30 0x7facac199de0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fac9c00e9c0 tx=0x7fac9c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:08:11.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:11.879+0000 7faca2a00640 1 -- 172.21.15.97:0/3831934455 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac9c00cd80 con 0x7facac101a30 2024-03-27T11:08:11.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:11.879+0000 7faca2a00640 1 -- 172.21.15.97:0/3831934455 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fac9c0035f0 con 0x7facac101a30 2024-03-27T11:08:11.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:11.879+0000 7facab200640 1 -- 172.21.15.97:0/3831934455 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facac1a1cf0 con 0x7facac101a30 2024-03-27T11:08:11.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:11.879+0000 7faca2a00640 1 -- 172.21.15.97:0/3831934455 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac9c010690 con 0x7facac101a30 2024-03-27T11:08:11.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:11.879+0000 7facab200640 1 -- 172.21.15.97:0/3831934455 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facac1a2190 con 0x7facac101a30 2024-03-27T11:08:11.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:11.880+0000 7facab200640 1 -- 172.21.15.97:0/3831934455 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facac1a23c0 con 0x7facac101a30 2024-03-27T11:08:11.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:11.883+0000 7faca2a00640 1 -- 172.21.15.97:0/3831934455 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7fac9c0037a0 con 0x7facac101a30 2024-03-27T11:08:11.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:11.884+0000 7faca2a00640 1 --2- 172.21.15.97:0/3831934455 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fac900a3610 0x7fac900a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:08:11.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:11.884+0000 7faca2a00640 1 -- 172.21.15.97:0/3831934455 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fac9c0d0340 con 0x7facac101a30 2024-03-27T11:08:11.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:11.884+0000 7faca9400640 1 --2- 172.21.15.97:0/3831934455 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fac900a3610 0x7fac900a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:08:11.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:11.885+0000 7faca9400640 1 --2- 172.21.15.97:0/3831934455 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fac900a3610 0x7fac900a5ad0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7facac19b2f0 tx=0x7fac98009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:08:11.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:11.889+0000 7faca2a00640 1 -- 172.21.15.97:0/3831934455 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7fac9c09cec0 con 0x7facac101a30 2024-03-27T11:08:11.923 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:08:11 smithi177 ceph-mon[35294]: pgmap v787: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 45 MiB/s wr, 815 op/s 2024-03-27T11:08:12.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.088+0000 7facab200640 1 -- 172.21.15.97:0/3831934455 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7facac1a2590 con 0x7facac101a30 2024-03-27T11:08:12.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.089+0000 7faca2a00640 1 -- 172.21.15.97:0/3831934455 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) ==== 93+0+10 (secure 0 0 0) 0x7fac9c09c7c0 con 0x7facac101a30 2024-03-27T11:08:12.058 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-03-27T11:08:12.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.092+0000 7facab200640 1 -- 172.21.15.97:0/3831934455 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fac900a3610 msgr2=0x7fac900a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:08:12.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.092+0000 7facab200640 1 --2- 172.21.15.97:0/3831934455 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fac900a3610 0x7fac900a5ad0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7facac19b2f0 tx=0x7fac98009260 comp rx=0 tx=0).stop 2024-03-27T11:08:12.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.092+0000 7facab200640 1 -- 172.21.15.97:0/3831934455 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7facac101a30 msgr2=0x7facac199de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:08:12.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.092+0000 7facab200640 1 --2- 172.21.15.97:0/3831934455 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7facac101a30 0x7facac199de0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fac9c00e9c0 tx=0x7fac9c00ee80 comp rx=0 tx=0).stop 2024-03-27T11:08:12.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.092+0000 7facab200640 1 -- 172.21.15.97:0/3831934455 shutdown_connections 2024-03-27T11:08:12.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.092+0000 7facab200640 1 --2- 172.21.15.97:0/3831934455 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fac900a3610 0x7fac900a5ad0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:08:12.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.092+0000 7facab200640 1 --2- 172.21.15.97:0/3831934455 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7facac1035e0 0x7facac1a1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:08:12.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.092+0000 7facab200640 1 --2- 172.21.15.97:0/3831934455 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7facac102c20 0x7facac19a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:08:12.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.092+0000 7facab200640 1 --2- 172.21.15.97:0/3831934455 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7facac101a30 0x7facac199de0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:08:12.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.092+0000 7facab200640 1 -- 172.21.15.97:0/3831934455 >> 172.21.15.97:0/3831934455 conn(0x7facac0fcfc0 msgr2=0x7facac073d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:08:12.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.092+0000 7facab200640 1 -- 172.21.15.97:0/3831934455 shutdown_connections 2024-03-27T11:08:12.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.092+0000 7facab200640 1 -- 172.21.15.97:0/3831934455 wait complete. 2024-03-27T11:08:12.082 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-27T11:08:12.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.302+0000 7fe5d1800640 1 -- 172.21.15.97:0/3969865903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5cc101b60 msgr2=0x7fe5cc101f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:08:12.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.302+0000 7fe5d1800640 1 --2- 172.21.15.97:0/3969865903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5cc101b60 0x7fe5cc101f80 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc009900 tx=0x7fe5bc02bc50 comp rx=0 tx=0).stop 2024-03-27T11:08:12.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.302+0000 7fe5d1800640 1 -- 172.21.15.97:0/3969865903 shutdown_connections 2024-03-27T11:08:12.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.302+0000 7fe5d1800640 1 --2- 172.21.15.97:0/3969865903 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe5cc103710 0x7fe5cc109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:08:12.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.302+0000 7fe5d1800640 1 --2- 172.21.15.97:0/3969865903 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe5cc102d50 0x7fe5cc1031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:08:12.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.302+0000 7fe5d1800640 1 --2- 172.21.15.97:0/3969865903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5cc101b60 0x7fe5cc101f80 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:08:12.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.302+0000 7fe5d1800640 1 -- 172.21.15.97:0/3969865903 >> 172.21.15.97:0/3969865903 conn(0x7fe5cc0fd0f0 msgr2=0x7fe5cc0ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:08:12.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.302+0000 7fe5d1800640 1 -- 172.21.15.97:0/3969865903 shutdown_connections 2024-03-27T11:08:12.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.302+0000 7fe5d1800640 1 -- 172.21.15.97:0/3969865903 wait complete. 2024-03-27T11:08:12.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.304+0000 7fe5d1800640 1 Processor -- start 2024-03-27T11:08:12.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.304+0000 7fe5d1800640 1 -- start start 2024-03-27T11:08:12.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.304+0000 7fe5d1800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5cc101b60 0x7fe5cc075900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:08:12.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.304+0000 7fe5d1800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe5cc102d50 0x7fe5cc073f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:08:12.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.304+0000 7fe5d1800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe5cc103710 0x7fe5cc074480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:08:12.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.304+0000 7fe5d1800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe5cc10d6b0 con 0x7fe5cc101b60 2024-03-27T11:08:12.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.304+0000 7fe5d1800640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe5cc10d530 con 0x7fe5cc102d50 2024-03-27T11:08:12.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.304+0000 7fe5d1800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe5cc10d830 con 0x7fe5cc103710 2024-03-27T11:08:12.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.305+0000 7fe5caa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5cc101b60 0x7fe5cc075900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:08:12.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.305+0000 7fe5caa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5cc101b60 0x7fe5cc075900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54756/0 (socket says 172.21.15.97:54756) 2024-03-27T11:08:12.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.305+0000 7fe5caa00640 1 -- 172.21.15.97:0/2949164412 learned_addr learned my addr 172.21.15.97:0/2949164412 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:08:12.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.305+0000 7fe5cb400640 1 --2- 172.21.15.97:0/2949164412 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe5cc103710 0x7fe5cc074480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:08:12.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.305+0000 7fe5ca000640 1 --2- 172.21.15.97:0/2949164412 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe5cc102d50 0x7fe5cc073f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:08:12.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.305+0000 7fe5caa00640 1 -- 172.21.15.97:0/2949164412 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe5cc103710 msgr2=0x7fe5cc074480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:08:12.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.305+0000 7fe5caa00640 1 --2- 172.21.15.97:0/2949164412 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe5cc103710 0x7fe5cc074480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:08:12.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.305+0000 7fe5caa00640 1 -- 172.21.15.97:0/2949164412 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe5cc102d50 msgr2=0x7fe5cc073f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:08:12.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.305+0000 7fe5caa00640 1 --2- 172.21.15.97:0/2949164412 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe5cc102d50 0x7fe5cc073f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:08:12.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.305+0000 7fe5caa00640 1 -- 172.21.15.97:0/2949164412 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5cc074d10 con 0x7fe5cc101b60 2024-03-27T11:08:12.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.306+0000 7fe5caa00640 1 --2- 172.21.15.97:0/2949164412 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5cc101b60 0x7fe5cc075900 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc000c00 tx=0x7fe5bc005e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:08:12.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.306+0000 7fe5c3400640 1 -- 172.21.15.97:0/2949164412 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5bc043020 con 0x7fe5cc101b60 2024-03-27T11:08:12.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.306+0000 7fe5c3400640 1 -- 172.21.15.97:0/2949164412 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe5bc0043d0 con 0x7fe5cc101b60 2024-03-27T11:08:12.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.306+0000 7fe5d1800640 1 -- 172.21.15.97:0/2949164412 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5cc1a1f00 con 0x7fe5cc101b60 2024-03-27T11:08:12.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.306+0000 7fe5c3400640 1 -- 172.21.15.97:0/2949164412 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5bc0039e0 con 0x7fe5cc101b60 2024-03-27T11:08:12.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.306+0000 7fe5d1800640 1 -- 172.21.15.97:0/2949164412 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5cc1a2310 con 0x7fe5cc101b60 2024-03-27T11:08:12.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.307+0000 7fe5d1800640 1 -- 172.21.15.97:0/2949164412 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5cc108ff0 con 0x7fe5cc101b60 2024-03-27T11:08:12.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.311+0000 7fe5c3400640 1 -- 172.21.15.97:0/2949164412 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7fe5bc046020 con 0x7fe5cc101b60 2024-03-27T11:08:12.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.312+0000 7fe5c3400640 1 --2- 172.21.15.97:0/2949164412 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fe5b00a3610 0x7fe5b00a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:08:12.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.312+0000 7fe5c3400640 1 -- 172.21.15.97:0/2949164412 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fe5bc0f21e0 con 0x7fe5cc101b60 2024-03-27T11:08:12.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.312+0000 7fe5ca000640 1 --2- 172.21.15.97:0/2949164412 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fe5b00a3610 0x7fe5b00a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:08:12.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.313+0000 7fe5ca000640 1 --2- 172.21.15.97:0/2949164412 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fe5b00a3610 0x7fe5b00a5ad0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fe5b80095f0 tx=0x7fe5b8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:08:12.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.314+0000 7fe5c3400640 1 -- 172.21.15.97:0/2949164412 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7fe5bc04d020 con 0x7fe5cc101b60 2024-03-27T11:08:12.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.525+0000 7fe5d1800640 1 -- 172.21.15.97:0/2949164412 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe5cc02d5c0 con 0x7fe5cc101b60 2024-03-27T11:08:12.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.525+0000 7fe5c3400640 1 -- 172.21.15.97:0/2949164412 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) ==== 104+0+11 (secure 0 0 0) 0x7fe5bc0bed60 con 0x7fe5cc101b60 2024-03-27T11:08:12.494 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-03-27T11:08:12.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.528+0000 7fe5d1800640 1 -- 172.21.15.97:0/2949164412 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fe5b00a3610 msgr2=0x7fe5b00a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:08:12.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.528+0000 7fe5d1800640 1 --2- 172.21.15.97:0/2949164412 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fe5b00a3610 0x7fe5b00a5ad0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fe5b80095f0 tx=0x7fe5b8005f90 comp rx=0 tx=0).stop 2024-03-27T11:08:12.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.528+0000 7fe5d1800640 1 -- 172.21.15.97:0/2949164412 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5cc101b60 msgr2=0x7fe5cc075900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:08:12.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.528+0000 7fe5d1800640 1 --2- 172.21.15.97:0/2949164412 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5cc101b60 0x7fe5cc075900 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc000c00 tx=0x7fe5bc005e60 comp rx=0 tx=0).stop 2024-03-27T11:08:12.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.528+0000 7fe5d1800640 1 -- 172.21.15.97:0/2949164412 shutdown_connections 2024-03-27T11:08:12.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.528+0000 7fe5d1800640 1 --2- 172.21.15.97:0/2949164412 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe5cc103710 0x7fe5cc074480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:08:12.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.528+0000 7fe5d1800640 1 --2- 172.21.15.97:0/2949164412 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fe5b00a3610 0x7fe5b00a5ad0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:08:12.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.528+0000 7fe5d1800640 1 --2- 172.21.15.97:0/2949164412 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe5cc102d50 0x7fe5cc073f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:08:12.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.528+0000 7fe5d1800640 1 --2- 172.21.15.97:0/2949164412 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5cc101b60 0x7fe5cc075900 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:08:12.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.528+0000 7fe5d1800640 1 -- 172.21.15.97:0/2949164412 >> 172.21.15.97:0/2949164412 conn(0x7fe5cc0fd0f0 msgr2=0x7fe5cc1061c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:08:12.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.528+0000 7fe5d1800640 1 -- 172.21.15.97:0/2949164412 shutdown_connections 2024-03-27T11:08:12.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.528+0000 7fe5d1800640 1 -- 172.21.15.97:0/2949164412 wait complete. 2024-03-27T11:08:12.514 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-27T11:08:12.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.749+0000 7f35b7800640 1 -- 172.21.15.97:0/1530333261 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35b0107840 msgr2=0x7f35b0109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:08:12.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.749+0000 7f35b7800640 1 --2- 172.21.15.97:0/1530333261 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35b0107840 0x7f35b0109c90 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f35a8009900 tx=0x7f35a802bc30 comp rx=0 tx=0).stop 2024-03-27T11:08:12.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.749+0000 7f35b7800640 1 -- 172.21.15.97:0/1530333261 shutdown_connections 2024-03-27T11:08:12.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.749+0000 7f35b7800640 1 --2- 172.21.15.97:0/1530333261 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35b0107840 0x7f35b0109c90 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:08:12.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.749+0000 7f35b7800640 1 --2- 172.21.15.97:0/1530333261 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f35b0104f10 0x7f35b0107300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:08:12.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.749+0000 7f35b7800640 1 --2- 172.21.15.97:0/1530333261 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f35b01025e0 0x7f35b01049d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:08:12.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.749+0000 7f35b7800640 1 -- 172.21.15.97:0/1530333261 >> 172.21.15.97:0/1530333261 conn(0x7f35b00fbff0 msgr2=0x7f35b00fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:08:12.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.749+0000 7f35b7800640 1 -- 172.21.15.97:0/1530333261 shutdown_connections 2024-03-27T11:08:12.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.749+0000 7f35b7800640 1 -- 172.21.15.97:0/1530333261 wait complete. 2024-03-27T11:08:12.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.750+0000 7f35b7800640 1 Processor -- start 2024-03-27T11:08:12.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.750+0000 7f35b7800640 1 -- start start 2024-03-27T11:08:12.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.750+0000 7f35b7800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f35b01025e0 0x7f35b0197d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:08:12.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.750+0000 7f35b7800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f35b0104f10 0x7f35b01982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:08:12.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.750+0000 7f35b7800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35b0107840 0x7f35b019f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:08:12.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.750+0000 7f35b7800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f35b010d700 con 0x7f35b0107840 2024-03-27T11:08:12.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.750+0000 7f35b7800640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f35b010d580 con 0x7f35b0104f10 2024-03-27T11:08:12.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.750+0000 7f35b7800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f35b010d880 con 0x7f35b01025e0 2024-03-27T11:08:12.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.751+0000 7f35b5800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35b0107840 0x7f35b019f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:08:12.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.751+0000 7f35b4e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f35b01025e0 0x7f35b0197d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:08:12.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.751+0000 7f35b5800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35b0107840 0x7f35b019f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54772/0 (socket says 172.21.15.97:54772) 2024-03-27T11:08:12.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.751+0000 7f35afe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f35b0104f10 0x7f35b01982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:08:12.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.751+0000 7f35b5800640 1 -- 172.21.15.97:0/451076676 learned_addr learned my addr 172.21.15.97:0/451076676 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:08:12.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.751+0000 7f35b5800640 1 -- 172.21.15.97:0/451076676 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f35b01025e0 msgr2=0x7f35b0197d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:08:12.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.751+0000 7f35b5800640 1 --2- 172.21.15.97:0/451076676 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f35b01025e0 0x7f35b0197d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:08:12.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.751+0000 7f35b5800640 1 -- 172.21.15.97:0/451076676 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f35b0104f10 msgr2=0x7f35b01982b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:08:12.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.751+0000 7f35b5800640 1 --2- 172.21.15.97:0/451076676 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f35b0104f10 0x7f35b01982b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:08:12.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.751+0000 7f35b5800640 1 -- 172.21.15.97:0/451076676 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35b019f990 con 0x7f35b0107840 2024-03-27T11:08:12.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.751+0000 7f35afe00640 1 --2- 172.21.15.97:0/451076676 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f35b0104f10 0x7f35b01982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T11:08:12.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.752+0000 7f35b5800640 1 --2- 172.21.15.97:0/451076676 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35b0107840 0x7f35b019f320 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f35a8036040 tx=0x7f35a8005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:08:12.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.752+0000 7f35ad600640 1 -- 172.21.15.97:0/451076676 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35a8043020 con 0x7f35b0107840 2024-03-27T11:08:12.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.752+0000 7f35ad600640 1 -- 172.21.15.97:0/451076676 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f35a8004410 con 0x7f35b0107840 2024-03-27T11:08:12.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.752+0000 7f35b7800640 1 -- 172.21.15.97:0/451076676 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35b019fc20 con 0x7f35b0107840 2024-03-27T11:08:12.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.752+0000 7f35ad600640 1 -- 172.21.15.97:0/451076676 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35a8003b80 con 0x7f35b0107840 2024-03-27T11:08:12.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.752+0000 7f35b7800640 1 -- 172.21.15.97:0/451076676 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35b01a01c0 con 0x7f35b0107840 2024-03-27T11:08:12.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.753+0000 7f35b7800640 1 -- 172.21.15.97:0/451076676 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35b00fdae0 con 0x7f35b0107840 2024-03-27T11:08:12.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.755+0000 7f35ad600640 1 -- 172.21.15.97:0/451076676 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f35a8046020 con 0x7f35b0107840 2024-03-27T11:08:12.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.757+0000 7f35ad600640 1 --2- 172.21.15.97:0/451076676 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f35940a3690 0x7f35940a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:08:12.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.757+0000 7f35ad600640 1 -- 172.21.15.97:0/451076676 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f35a80f1eb0 con 0x7f35b0107840 2024-03-27T11:08:12.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.757+0000 7f35b4e00640 1 --2- 172.21.15.97:0/451076676 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f35940a3690 0x7f35940a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:08:12.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.758+0000 7f35b4e00640 1 --2- 172.21.15.97:0/451076676 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f35940a3690 0x7f35940a5b50 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f35a0005fd0 tx=0x7f35a0009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:08:12.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.761+0000 7f35ad600640 1 -- 172.21.15.97:0/451076676 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f35a804d020 con 0x7f35b0107840 2024-03-27T11:08:12.763 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:08:12 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/1310792602' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T11:08:12.763 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:08:12 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/3831934455' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-27T11:08:12.887 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:08:12 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/1310792602' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T11:08:12.887 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:08:12 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/3831934455' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-27T11:08:12.924 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:08:12 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/1310792602' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T11:08:12.924 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:08:12 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/3831934455' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-27T11:08:12.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.965+0000 7f35b7800640 1 -- 172.21.15.97:0/451076676 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f35b0062750 con 0x7f35b0107840 2024-03-27T11:08:12.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.966+0000 7f35ad600640 1 -- 172.21.15.97:0/451076676 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) ==== 100+0+11 (secure 0 0 0) 0x7f35a80efe20 con 0x7f35b0107840 2024-03-27T11:08:12.935 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-03-27T11:08:12.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.969+0000 7f35b7800640 1 -- 172.21.15.97:0/451076676 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f35940a3690 msgr2=0x7f35940a5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:08:12.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.969+0000 7f35b7800640 1 --2- 172.21.15.97:0/451076676 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f35940a3690 0x7f35940a5b50 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f35a0005fd0 tx=0x7f35a0009500 comp rx=0 tx=0).stop 2024-03-27T11:08:12.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.969+0000 7f35b7800640 1 -- 172.21.15.97:0/451076676 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35b0107840 msgr2=0x7f35b019f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:08:12.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.969+0000 7f35b7800640 1 --2- 172.21.15.97:0/451076676 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35b0107840 0x7f35b019f320 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f35a8036040 tx=0x7f35a8005dd0 comp rx=0 tx=0).stop 2024-03-27T11:08:12.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.969+0000 7f35b7800640 1 -- 172.21.15.97:0/451076676 shutdown_connections 2024-03-27T11:08:12.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.969+0000 7f35b7800640 1 --2- 172.21.15.97:0/451076676 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35b0107840 0x7f35b019f320 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:08:12.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.969+0000 7f35b7800640 1 --2- 172.21.15.97:0/451076676 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f35940a3690 0x7f35940a5b50 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:08:12.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.969+0000 7f35b7800640 1 --2- 172.21.15.97:0/451076676 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f35b0104f10 0x7f35b01982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:08:12.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.969+0000 7f35b7800640 1 --2- 172.21.15.97:0/451076676 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f35b01025e0 0x7f35b0197d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:08:12.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.969+0000 7f35b7800640 1 -- 172.21.15.97:0/451076676 >> 172.21.15.97:0/451076676 conn(0x7f35b00fbff0 msgr2=0x7f35b0102f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:08:12.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.969+0000 7f35b7800640 1 -- 172.21.15.97:0/451076676 shutdown_connections 2024-03-27T11:08:12.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:12.969+0000 7f35b7800640 1 -- 172.21.15.97:0/451076676 wait complete. 2024-03-27T11:08:12.956 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-03-27T11:08:13.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:13.179+0000 7fa58ec00640 1 -- 172.21.15.97:0/3197339936 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa588101b60 msgr2=0x7fa588101f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:08:13.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:13.179+0000 7fa58ec00640 1 --2- 172.21.15.97:0/3197339936 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa588101b60 0x7fa588101f80 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fa578009900 tx=0x7fa57802bc30 comp rx=0 tx=0).stop 2024-03-27T11:08:13.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:13.179+0000 7fa58ec00640 1 -- 172.21.15.97:0/3197339936 shutdown_connections 2024-03-27T11:08:13.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:13.179+0000 7fa58ec00640 1 --2- 172.21.15.97:0/3197339936 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa588103710 0x7fa588109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:08:13.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:13.179+0000 7fa58ec00640 1 --2- 172.21.15.97:0/3197339936 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa588102d50 0x7fa5881031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:08:13.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:13.179+0000 7fa58ec00640 1 --2- 172.21.15.97:0/3197339936 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa588101b60 0x7fa588101f80 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:08:13.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:13.179+0000 7fa58ec00640 1 -- 172.21.15.97:0/3197339936 >> 172.21.15.97:0/3197339936 conn(0x7fa5880fd130 msgr2=0x7fa5880ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:08:13.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:13.179+0000 7fa58ec00640 1 -- 172.21.15.97:0/3197339936 shutdown_connections 2024-03-27T11:08:13.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:13.179+0000 7fa58ec00640 1 -- 172.21.15.97:0/3197339936 wait complete. 2024-03-27T11:08:13.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:13.180+0000 7fa58ec00640 1 Processor -- start 2024-03-27T11:08:13.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:13.180+0000 7fa58ec00640 1 -- start start 2024-03-27T11:08:13.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:13.180+0000 7fa58ec00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa588101b60 0x7fa58819a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:08:13.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:13.180+0000 7fa58ec00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa588102d50 0x7fa58819a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:08:13.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:13.180+0000 7fa58ec00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa588103710 0x7fa5881a15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:08:13.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:13.180+0000 7fa58ec00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa58810d6d0 con 0x7fa588101b60 2024-03-27T11:08:13.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:13.180+0000 7fa58ec00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa58810d550 con 0x7fa588103710 2024-03-27T11:08:13.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:13.180+0000 7fa58ec00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa58810d850 con 0x7fa588102d50 2024-03-27T11:08:13.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:13.181+0000 7fa587e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa588101b60 0x7fa58819a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:08:13.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:13.181+0000 7fa587e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa588101b60 0x7fa58819a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54800/0 (socket says 172.21.15.97:54800) 2024-03-27T11:08:13.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:13.181+0000 7fa587e00640 1 -- 172.21.15.97:0/1996842846 learned_addr learned my addr 172.21.15.97:0/1996842846 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:08:13.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:13.181+0000 7fa57fe00640 1 --2- 172.21.15.97:0/1996842846 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa588102d50 0x7fa58819a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:08:13.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:13.181+0000 7fa58cc00640 1 --2- 172.21.15.97:0/1996842846 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa588103710 0x7fa5881a15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:08:13.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:13.181+0000 7fa587e00640 1 -- 172.21.15.97:0/1996842846 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa588102d50 msgr2=0x7fa58819a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:08:13.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:13.181+0000 7fa587e00640 1 --2- 172.21.15.97:0/1996842846 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa588102d50 0x7fa58819a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:08:13.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:13.181+0000 7fa587e00640 1 -- 172.21.15.97:0/1996842846 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa588103710 msgr2=0x7fa5881a15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:08:13.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:13.181+0000 7fa587e00640 1 --2- 172.21.15.97:0/1996842846 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa588103710 0x7fa5881a15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:08:13.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:13.181+0000 7fa587e00640 1 -- 172.21.15.97:0/1996842846 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5881a1c50 con 0x7fa588101b60 2024-03-27T11:08:13.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:13.181+0000 7fa587e00640 1 --2- 172.21.15.97:0/1996842846 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa588101b60 0x7fa58819a030 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fa578000c00 tx=0x7fa5780040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:08:13.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:13.182+0000 7fa585600640 1 -- 172.21.15.97:0/1996842846 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa578043020 con 0x7fa588101b60 2024-03-27T11:08:13.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:13.182+0000 7fa585600640 1 -- 172.21.15.97:0/1996842846 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa5780044c0 con 0x7fa588101b60 2024-03-27T11:08:13.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:13.182+0000 7fa585600640 1 -- 172.21.15.97:0/1996842846 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa578003b40 con 0x7fa588101b60 2024-03-27T11:08:13.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:13.182+0000 7fa58ec00640 1 -- 172.21.15.97:0/1996842846 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5881a1ee0 con 0x7fa588101b60 2024-03-27T11:08:13.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:13.182+0000 7fa58ec00640 1 -- 172.21.15.97:0/1996842846 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5881a22f0 con 0x7fa588101b60 2024-03-27T11:08:13.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:13.183+0000 7fa58ec00640 1 -- 172.21.15.97:0/1996842846 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5881a24f0 con 0x7fa588101b60 2024-03-27T11:08:13.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:13.185+0000 7fa585600640 1 -- 172.21.15.97:0/1996842846 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7fa578046020 con 0x7fa588101b60 2024-03-27T11:08:13.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:13.187+0000 7fa585600640 1 --2- 172.21.15.97:0/1996842846 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fa56c0a3690 0x7fa56c0a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:08:13.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:13.187+0000 7fa585600640 1 -- 172.21.15.97:0/1996842846 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fa5780f1f10 con 0x7fa588101b60 2024-03-27T11:08:13.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:13.188+0000 7fa57fe00640 1 --2- 172.21.15.97:0/1996842846 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fa56c0a3690 0x7fa56c0a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:08:13.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:13.188+0000 7fa57fe00640 1 --2- 172.21.15.97:0/1996842846 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fa56c0a3690 0x7fa56c0a5b50 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fa58819b540 tx=0x7fa57000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:08:13.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:13.192+0000 7fa585600640 1 -- 172.21.15.97:0/1996842846 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7fa57804d020 con 0x7fa588101b60 2024-03-27T11:08:13.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:13.469+0000 7fa58ec00640 1 -- 172.21.15.97:0/1996842846 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7fa5881a2da0 con 0x7fa588101b60 2024-03-27T11:08:13.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:13.502+0000 7fa585600640 1 -- 172.21.15.97:0/1996842846 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v18) ==== 81+0+743807 (secure 0 0 0) 0x7fa5780bea90 con 0x7fa588101b60 2024-03-27T11:08:13.482 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T11:08:13.482 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-03-27T11:08:13.482 INFO:teuthology.orchestra.run.smithi097.stdout: "epoch": 18, 2024-03-27T11:08:13.482 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.482 INFO:teuthology.orchestra.run.smithi097.stdout: "active_gid": 14152, 2024-03-27T11:08:13.482 INFO:teuthology.orchestra.run.smithi097.stdout: "active_name": "x", 2024-03-27T11:08:13.483 INFO:teuthology.orchestra.run.smithi097.stdout: "active_addrs": { 2024-03-27T11:08:13.483 INFO:teuthology.orchestra.run.smithi097.stdout: "addrvec": [ 2024-03-27T11:08:13.483 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:13.483 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v2", 2024-03-27T11:08:13.483 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.97:6800", 2024-03-27T11:08:13.483 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 3955463538 2024-03-27T11:08:13.483 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.483 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:13.483 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2024-03-27T11:08:13.483 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.97:6801", 2024-03-27T11:08:13.483 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 3955463538 2024-03-27T11:08:13.483 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:13.483 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-03-27T11:08:13.483 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.484 INFO:teuthology.orchestra.run.smithi097.stdout: "active_addr": "172.21.15.97:6801/3955463538", 2024-03-27T11:08:13.484 INFO:teuthology.orchestra.run.smithi097.stdout: "active_change": "2024-03-27T10:44:39.889650+0000", 2024-03-27T11:08:13.484 INFO:teuthology.orchestra.run.smithi097.stdout: "active_mgr_features": 4540701547738038271, 2024-03-27T11:08:13.484 INFO:teuthology.orchestra.run.smithi097.stdout: "available": true, 2024-03-27T11:08:13.484 INFO:teuthology.orchestra.run.smithi097.stdout: "standbys": [ 2024-03-27T11:08:13.484 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:13.484 INFO:teuthology.orchestra.run.smithi097.stdout: "gid": 24109, 2024-03-27T11:08:13.484 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "z", 2024-03-27T11:08:13.484 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr_features": 4540701547738038271, 2024-03-27T11:08:13.484 INFO:teuthology.orchestra.run.smithi097.stdout: "available_modules": [ 2024-03-27T11:08:13.484 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:13.484 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "alerts", 2024-03-27T11:08:13.484 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-27T11:08:13.485 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-27T11:08:13.485 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-27T11:08:13.485 INFO:teuthology.orchestra.run.smithi097.stdout: "interval": { 2024-03-27T11:08:13.485 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "interval", 2024-03-27T11:08:13.485 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-27T11:08:13.485 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.485 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.485 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "60", 2024-03-27T11:08:13.485 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.485 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.485 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.485 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "How frequently to reexamine health status", 2024-03-27T11:08:13.485 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.486 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.486 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.486 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.486 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-27T11:08:13.486 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-27T11:08:13.486 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.486 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.486 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.486 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:13.486 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.486 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.486 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:13.487 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:13.487 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:13.487 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:13.487 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:13.487 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:13.487 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:13.487 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:13.487 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.487 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.487 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.487 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.487 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.487 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-27T11:08:13.487 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-27T11:08:13.488 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.488 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.488 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.488 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.488 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.488 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.488 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.488 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.488 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.488 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.488 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.488 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.488 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-27T11:08:13.488 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-27T11:08:13.489 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.489 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.489 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.489 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-27T11:08:13.489 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.489 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.489 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:13.489 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:13.489 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:13.489 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:13.489 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:13.489 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:13.489 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:13.489 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:13.490 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.490 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.490 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.490 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.490 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.490 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-27T11:08:13.490 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-27T11:08:13.490 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.490 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.490 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.490 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.490 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.490 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.490 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.491 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.491 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.491 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.491 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.491 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.491 INFO:teuthology.orchestra.run.smithi097.stdout: "smtp_destination": { 2024-03-27T11:08:13.491 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "smtp_destination", 2024-03-27T11:08:13.491 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.491 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.491 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.491 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:13.491 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.491 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.491 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.491 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Email address to send alerts to", 2024-03-27T11:08:13.492 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.492 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.492 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.492 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.492 INFO:teuthology.orchestra.run.smithi097.stdout: "smtp_from_name": { 2024-03-27T11:08:13.492 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "smtp_from_name", 2024-03-27T11:08:13.492 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.492 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.492 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.492 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "Ceph", 2024-03-27T11:08:13.492 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.492 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.492 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.492 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Email From: name", 2024-03-27T11:08:13.492 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.493 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.493 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.493 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.493 INFO:teuthology.orchestra.run.smithi097.stdout: "smtp_host": { 2024-03-27T11:08:13.493 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "smtp_host", 2024-03-27T11:08:13.493 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.493 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.493 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.493 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:13.493 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.493 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.493 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.493 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "SMTP server", 2024-03-27T11:08:13.494 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.494 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.494 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.494 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.494 INFO:teuthology.orchestra.run.smithi097.stdout: "smtp_password": { 2024-03-27T11:08:13.494 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "smtp_password", 2024-03-27T11:08:13.494 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.494 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.494 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.494 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:13.494 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.494 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.494 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.495 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Password to authenticate with", 2024-03-27T11:08:13.495 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.495 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.495 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.495 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.495 INFO:teuthology.orchestra.run.smithi097.stdout: "smtp_port": { 2024-03-27T11:08:13.495 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "smtp_port", 2024-03-27T11:08:13.495 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:13.495 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.495 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.495 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "465", 2024-03-27T11:08:13.495 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.495 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.495 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.495 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "SMTP port", 2024-03-27T11:08:13.496 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.496 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.496 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.496 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.496 INFO:teuthology.orchestra.run.smithi097.stdout: "smtp_sender": { 2024-03-27T11:08:13.496 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "smtp_sender", 2024-03-27T11:08:13.496 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.496 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.496 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.496 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:13.496 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.496 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.496 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.496 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "SMTP envelope sender", 2024-03-27T11:08:13.497 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.497 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.497 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.497 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.497 INFO:teuthology.orchestra.run.smithi097.stdout: "smtp_ssl": { 2024-03-27T11:08:13.497 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "smtp_ssl", 2024-03-27T11:08:13.497 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.497 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.497 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.497 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:13.497 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.497 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.497 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.497 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-27T11:08:13.498 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.498 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.498 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.498 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.498 INFO:teuthology.orchestra.run.smithi097.stdout: "smtp_user": { 2024-03-27T11:08:13.498 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "smtp_user", 2024-03-27T11:08:13.498 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.498 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.498 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.498 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:13.498 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.498 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.499 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.499 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "User to authenticate as", 2024-03-27T11:08:13.499 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.499 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.499 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.499 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:13.499 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:13.499 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.499 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:13.499 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "balancer", 2024-03-27T11:08:13.499 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-27T11:08:13.499 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-27T11:08:13.499 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-27T11:08:13.500 INFO:teuthology.orchestra.run.smithi097.stdout: "active": { 2024-03-27T11:08:13.500 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "active", 2024-03-27T11:08:13.500 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.500 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.500 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.500 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:13.500 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.500 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.500 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.500 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "automatically balance PGs across cluster", 2024-03-27T11:08:13.500 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.500 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.500 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.501 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.501 INFO:teuthology.orchestra.run.smithi097.stdout: "begin_time": { 2024-03-27T11:08:13.501 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "begin_time", 2024-03-27T11:08:13.501 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.501 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.501 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.501 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "0000", 2024-03-27T11:08:13.501 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.501 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.501 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.501 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "beginning time of day to automatically balance", 2024-03-27T11:08:13.501 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-27T11:08:13.501 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.501 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.501 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.502 INFO:teuthology.orchestra.run.smithi097.stdout: "begin_weekday": { 2024-03-27T11:08:13.502 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "begin_weekday", 2024-03-27T11:08:13.502 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "uint", 2024-03-27T11:08:13.502 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.502 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.502 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "0", 2024-03-27T11:08:13.502 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "0", 2024-03-27T11:08:13.502 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "6", 2024-03-27T11:08:13.502 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.502 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-27T11:08:13.502 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-27T11:08:13.502 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.502 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.503 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.503 INFO:teuthology.orchestra.run.smithi097.stdout: "crush_compat_max_iterations": { 2024-03-27T11:08:13.503 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "crush_compat_max_iterations", 2024-03-27T11:08:13.503 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "uint", 2024-03-27T11:08:13.503 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.503 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.503 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "25", 2024-03-27T11:08:13.503 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "1", 2024-03-27T11:08:13.503 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "250", 2024-03-27T11:08:13.503 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.503 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-27T11:08:13.503 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.503 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.503 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.503 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.504 INFO:teuthology.orchestra.run.smithi097.stdout: "crush_compat_metrics": { 2024-03-27T11:08:13.504 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "crush_compat_metrics", 2024-03-27T11:08:13.504 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.504 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.504 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.504 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "pgs,objects,bytes", 2024-03-27T11:08:13.504 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.504 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.504 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.504 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-27T11:08:13.504 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-27T11:08:13.504 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.504 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.505 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.505 INFO:teuthology.orchestra.run.smithi097.stdout: "crush_compat_step": { 2024-03-27T11:08:13.505 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "crush_compat_step", 2024-03-27T11:08:13.505 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "float", 2024-03-27T11:08:13.505 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.505 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.505 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "0.5", 2024-03-27T11:08:13.505 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "0.001", 2024-03-27T11:08:13.505 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "0.999", 2024-03-27T11:08:13.505 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.505 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "aggressiveness of optimization", 2024-03-27T11:08:13.505 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-27T11:08:13.505 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.505 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.505 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.506 INFO:teuthology.orchestra.run.smithi097.stdout: "end_time": { 2024-03-27T11:08:13.506 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "end_time", 2024-03-27T11:08:13.506 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.506 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.506 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.506 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "2359", 2024-03-27T11:08:13.506 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.506 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.506 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.506 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "ending time of day to automatically balance", 2024-03-27T11:08:13.506 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-27T11:08:13.506 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.506 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.506 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.507 INFO:teuthology.orchestra.run.smithi097.stdout: "end_weekday": { 2024-03-27T11:08:13.507 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "end_weekday", 2024-03-27T11:08:13.507 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "uint", 2024-03-27T11:08:13.507 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.507 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.507 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "0", 2024-03-27T11:08:13.507 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "0", 2024-03-27T11:08:13.507 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "6", 2024-03-27T11:08:13.507 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.507 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-27T11:08:13.507 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-27T11:08:13.507 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.508 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.508 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.508 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-27T11:08:13.508 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-27T11:08:13.508 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.508 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.508 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.508 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:13.508 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.508 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.508 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:13.508 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:13.508 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:13.508 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:13.509 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:13.509 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:13.509 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:13.509 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:13.509 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.509 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.509 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.509 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.509 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.509 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-27T11:08:13.509 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-27T11:08:13.509 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.509 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.509 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.510 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.510 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.510 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.510 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.510 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.510 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.510 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.510 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.510 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.510 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-27T11:08:13.510 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-27T11:08:13.510 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.510 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.511 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.511 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-27T11:08:13.511 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.511 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.511 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:13.511 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:13.511 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:13.511 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:13.511 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:13.511 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:13.511 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:13.511 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:13.511 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.511 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.511 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.512 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.512 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.512 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-27T11:08:13.512 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-27T11:08:13.512 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.512 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.512 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.512 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.512 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.512 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.512 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.512 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.512 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.512 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.512 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.513 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.513 INFO:teuthology.orchestra.run.smithi097.stdout: "min_score": { 2024-03-27T11:08:13.513 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "min_score", 2024-03-27T11:08:13.513 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "float", 2024-03-27T11:08:13.513 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.513 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.513 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "0", 2024-03-27T11:08:13.513 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.513 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.513 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.513 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-27T11:08:13.513 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.513 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.514 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.514 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.514 INFO:teuthology.orchestra.run.smithi097.stdout: "mode": { 2024-03-27T11:08:13.514 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "mode", 2024-03-27T11:08:13.514 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.514 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.514 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.514 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "upmap", 2024-03-27T11:08:13.514 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.514 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.514 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:13.514 INFO:teuthology.orchestra.run.smithi097.stdout: "crush-compat", 2024-03-27T11:08:13.514 INFO:teuthology.orchestra.run.smithi097.stdout: "none", 2024-03-27T11:08:13.514 INFO:teuthology.orchestra.run.smithi097.stdout: "read", 2024-03-27T11:08:13.515 INFO:teuthology.orchestra.run.smithi097.stdout: "upmap", 2024-03-27T11:08:13.515 INFO:teuthology.orchestra.run.smithi097.stdout: "upmap-read" 2024-03-27T11:08:13.515 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:13.515 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Balancer mode", 2024-03-27T11:08:13.515 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.515 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.515 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.515 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.515 INFO:teuthology.orchestra.run.smithi097.stdout: "pool_ids": { 2024-03-27T11:08:13.515 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "pool_ids", 2024-03-27T11:08:13.515 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.515 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.515 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.515 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:13.515 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.516 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.516 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.516 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-27T11:08:13.516 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.516 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.516 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.516 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.516 INFO:teuthology.orchestra.run.smithi097.stdout: "sleep_interval": { 2024-03-27T11:08:13.516 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "sleep_interval", 2024-03-27T11:08:13.516 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-27T11:08:13.516 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.516 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.516 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "60", 2024-03-27T11:08:13.516 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.517 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.517 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.517 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-27T11:08:13.517 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.517 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.517 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.517 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.517 INFO:teuthology.orchestra.run.smithi097.stdout: "upmap_max_deviation": { 2024-03-27T11:08:13.517 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "upmap_max_deviation", 2024-03-27T11:08:13.517 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:13.517 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.517 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.517 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "5", 2024-03-27T11:08:13.517 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "1", 2024-03-27T11:08:13.517 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.518 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.518 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-27T11:08:13.518 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-27T11:08:13.518 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.518 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.518 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.518 INFO:teuthology.orchestra.run.smithi097.stdout: "upmap_max_optimizations": { 2024-03-27T11:08:13.518 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "upmap_max_optimizations", 2024-03-27T11:08:13.518 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "uint", 2024-03-27T11:08:13.518 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.518 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.518 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10", 2024-03-27T11:08:13.518 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.518 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.519 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.519 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-27T11:08:13.519 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.519 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.519 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.519 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:13.519 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:13.519 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.519 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:13.519 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "cephadm", 2024-03-27T11:08:13.519 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-27T11:08:13.519 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-27T11:08:13.519 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-27T11:08:13.519 INFO:teuthology.orchestra.run.smithi097.stdout: "agent_down_multiplier": { 2024-03-27T11:08:13.519 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "agent_down_multiplier", 2024-03-27T11:08:13.520 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "float", 2024-03-27T11:08:13.520 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.520 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.520 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "3.0", 2024-03-27T11:08:13.520 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.520 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.520 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.520 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-27T11:08:13.520 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.520 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.520 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.520 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.520 INFO:teuthology.orchestra.run.smithi097.stdout: "agent_refresh_rate": { 2024-03-27T11:08:13.520 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "agent_refresh_rate", 2024-03-27T11:08:13.520 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-27T11:08:13.521 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.521 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.521 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "20", 2024-03-27T11:08:13.521 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.521 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.521 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.521 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-27T11:08:13.521 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.521 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.521 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.521 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.521 INFO:teuthology.orchestra.run.smithi097.stdout: "agent_starting_port": { 2024-03-27T11:08:13.521 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "agent_starting_port", 2024-03-27T11:08:13.521 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:13.521 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.521 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.522 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "4721", 2024-03-27T11:08:13.522 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.522 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.522 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.522 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-27T11:08:13.522 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.522 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.522 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.522 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.522 INFO:teuthology.orchestra.run.smithi097.stdout: "allow_ptrace": { 2024-03-27T11:08:13.522 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "allow_ptrace", 2024-03-27T11:08:13.522 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.522 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.523 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.523 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.523 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.523 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.523 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.523 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-27T11:08:13.523 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-27T11:08:13.523 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.523 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.523 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.523 INFO:teuthology.orchestra.run.smithi097.stdout: "autotune_interval": { 2024-03-27T11:08:13.523 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "autotune_interval", 2024-03-27T11:08:13.523 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-27T11:08:13.523 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.524 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.524 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "600", 2024-03-27T11:08:13.524 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.524 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.524 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.524 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-27T11:08:13.524 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.524 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.524 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.524 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.524 INFO:teuthology.orchestra.run.smithi097.stdout: "autotune_memory_target_ratio": { 2024-03-27T11:08:13.524 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "autotune_memory_target_ratio", 2024-03-27T11:08:13.524 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "float", 2024-03-27T11:08:13.524 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.525 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.525 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "0.7", 2024-03-27T11:08:13.525 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.525 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.525 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.525 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-27T11:08:13.525 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.525 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.525 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.525 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.525 INFO:teuthology.orchestra.run.smithi097.stdout: "cephadm_log_destination": { 2024-03-27T11:08:13.525 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "cephadm_log_destination", 2024-03-27T11:08:13.525 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.525 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.525 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.526 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:13.526 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.526 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.526 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:13.526 INFO:teuthology.orchestra.run.smithi097.stdout: "file", 2024-03-27T11:08:13.526 INFO:teuthology.orchestra.run.smithi097.stdout: "file,syslog", 2024-03-27T11:08:13.526 INFO:teuthology.orchestra.run.smithi097.stdout: "syslog" 2024-03-27T11:08:13.526 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:13.526 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-27T11:08:13.526 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.526 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.526 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.526 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.526 INFO:teuthology.orchestra.run.smithi097.stdout: "cgroups_split": { 2024-03-27T11:08:13.527 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "cgroups_split", 2024-03-27T11:08:13.527 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.527 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.527 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.527 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:13.527 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.527 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.527 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.527 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-27T11:08:13.527 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.527 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.527 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.527 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.527 INFO:teuthology.orchestra.run.smithi097.stdout: "config_checks_enabled": { 2024-03-27T11:08:13.527 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "config_checks_enabled", 2024-03-27T11:08:13.528 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.528 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.528 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.528 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.528 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.528 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.528 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.528 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-27T11:08:13.528 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.528 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.528 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.528 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.528 INFO:teuthology.orchestra.run.smithi097.stdout: "config_dashboard": { 2024-03-27T11:08:13.528 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "config_dashboard", 2024-03-27T11:08:13.528 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.529 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.529 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.529 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:13.529 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.529 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.529 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.529 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-27T11:08:13.529 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.529 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.529 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.529 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.529 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_alertmanager": { 2024-03-27T11:08:13.529 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_alertmanager", 2024-03-27T11:08:13.529 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.529 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.529 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.530 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-27T11:08:13.530 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.530 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.530 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.530 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Prometheus container image", 2024-03-27T11:08:13.530 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.530 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.530 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.530 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.530 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_base": { 2024-03-27T11:08:13.530 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_base", 2024-03-27T11:08:13.530 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.530 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.531 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.531 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-27T11:08:13.531 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.531 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.531 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.531 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Container image name, without the tag", 2024-03-27T11:08:13.531 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.531 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.531 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.531 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.531 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_elasticsearch": { 2024-03-27T11:08:13.531 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_elasticsearch", 2024-03-27T11:08:13.531 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.531 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.531 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.532 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-27T11:08:13.532 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.532 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.532 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.532 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "elasticsearch container image", 2024-03-27T11:08:13.532 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.532 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.532 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.532 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.532 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_grafana": { 2024-03-27T11:08:13.532 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_grafana", 2024-03-27T11:08:13.532 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.532 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.532 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.532 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-03-27T11:08:13.533 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.533 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.533 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.533 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Prometheus container image", 2024-03-27T11:08:13.533 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.533 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.533 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.533 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.533 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_haproxy": { 2024-03-27T11:08:13.533 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_haproxy", 2024-03-27T11:08:13.533 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.533 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.533 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.533 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-27T11:08:13.534 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.534 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.534 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.534 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "HAproxy container image", 2024-03-27T11:08:13.534 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.534 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.534 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.534 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.534 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_jaeger_agent": { 2024-03-27T11:08:13.534 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_jaeger_agent", 2024-03-27T11:08:13.534 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.534 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.534 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.535 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-27T11:08:13.535 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.535 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.535 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.535 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Jaeger agent container image", 2024-03-27T11:08:13.535 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.535 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.535 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.535 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.535 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_jaeger_collector": { 2024-03-27T11:08:13.535 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_jaeger_collector", 2024-03-27T11:08:13.535 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.535 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.535 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.535 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-27T11:08:13.536 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.536 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.536 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.536 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Jaeger collector container image", 2024-03-27T11:08:13.536 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.536 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.536 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.536 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.536 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_jaeger_query": { 2024-03-27T11:08:13.536 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_jaeger_query", 2024-03-27T11:08:13.536 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.536 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.536 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.536 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-27T11:08:13.536 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.537 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.537 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.537 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Jaeger query container image", 2024-03-27T11:08:13.537 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.537 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.537 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.537 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.537 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_keepalived": { 2024-03-27T11:08:13.537 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_keepalived", 2024-03-27T11:08:13.537 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.537 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.537 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.537 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-27T11:08:13.537 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.537 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.538 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.538 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Keepalived container image", 2024-03-27T11:08:13.538 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.538 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.538 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.538 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.538 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_loki": { 2024-03-27T11:08:13.538 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_loki", 2024-03-27T11:08:13.538 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.538 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.538 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.538 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-27T11:08:13.538 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.538 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.539 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.539 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Loki container image", 2024-03-27T11:08:13.539 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.539 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.539 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.539 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.539 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_node_exporter": { 2024-03-27T11:08:13.539 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_node_exporter", 2024-03-27T11:08:13.539 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.539 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.539 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.539 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-27T11:08:13.539 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.539 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.540 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.540 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Prometheus container image", 2024-03-27T11:08:13.540 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.540 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.540 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.540 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.540 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_nvmeof": { 2024-03-27T11:08:13.540 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_nvmeof", 2024-03-27T11:08:13.540 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.540 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.540 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.540 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-03-27T11:08:13.540 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.540 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.540 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.540 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Nvme-of container image", 2024-03-27T11:08:13.541 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.541 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.541 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.541 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.541 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_prometheus": { 2024-03-27T11:08:13.541 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_prometheus", 2024-03-27T11:08:13.541 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.541 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.541 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.541 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-27T11:08:13.541 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.541 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.541 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.541 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Prometheus container image", 2024-03-27T11:08:13.541 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.542 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.542 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.542 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.542 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_promtail": { 2024-03-27T11:08:13.542 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_promtail", 2024-03-27T11:08:13.542 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.542 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.542 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.542 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-27T11:08:13.542 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.542 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.542 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.542 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Promtail container image", 2024-03-27T11:08:13.542 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.542 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.543 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.543 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.543 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_samba": { 2024-03-27T11:08:13.543 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_samba", 2024-03-27T11:08:13.543 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.543 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.543 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.543 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-03-27T11:08:13.543 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.543 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.543 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.543 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Samba/SMB container image", 2024-03-27T11:08:13.543 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.543 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.543 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.544 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.544 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_snmp_gateway": { 2024-03-27T11:08:13.544 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_snmp_gateway", 2024-03-27T11:08:13.544 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.544 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.544 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.544 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-27T11:08:13.544 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.544 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.544 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.544 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "SNMP Gateway container image", 2024-03-27T11:08:13.544 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.544 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.544 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.544 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.545 INFO:teuthology.orchestra.run.smithi097.stdout: "container_init": { 2024-03-27T11:08:13.545 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_init", 2024-03-27T11:08:13.545 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.545 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.545 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.545 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:13.545 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.545 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.545 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.545 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Run podman/docker with `--init`", 2024-03-27T11:08:13.545 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.545 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.545 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.546 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.546 INFO:teuthology.orchestra.run.smithi097.stdout: "daemon_cache_timeout": { 2024-03-27T11:08:13.546 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "daemon_cache_timeout", 2024-03-27T11:08:13.546 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-27T11:08:13.546 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.546 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.546 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "600", 2024-03-27T11:08:13.546 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.546 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.546 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.546 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-27T11:08:13.546 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.546 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.546 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.546 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.547 INFO:teuthology.orchestra.run.smithi097.stdout: "default_cephadm_command_timeout": { 2024-03-27T11:08:13.547 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "default_cephadm_command_timeout", 2024-03-27T11:08:13.547 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:13.547 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.547 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.547 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "900", 2024-03-27T11:08:13.547 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.547 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.547 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.547 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-27T11:08:13.547 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.547 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.547 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.547 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.548 INFO:teuthology.orchestra.run.smithi097.stdout: "default_registry": { 2024-03-27T11:08:13.548 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "default_registry", 2024-03-27T11:08:13.548 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.548 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.548 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.548 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "docker.io", 2024-03-27T11:08:13.548 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.548 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.548 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.548 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-27T11:08:13.548 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.548 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.548 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.548 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.548 INFO:teuthology.orchestra.run.smithi097.stdout: "device_cache_timeout": { 2024-03-27T11:08:13.549 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "device_cache_timeout", 2024-03-27T11:08:13.549 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-27T11:08:13.549 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.549 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.549 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "1800", 2024-03-27T11:08:13.549 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.549 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.549 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.549 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "seconds to cache device inventory", 2024-03-27T11:08:13.549 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.549 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.549 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.549 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.549 INFO:teuthology.orchestra.run.smithi097.stdout: "device_enhanced_scan": { 2024-03-27T11:08:13.549 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "device_enhanced_scan", 2024-03-27T11:08:13.550 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.550 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.550 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.550 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.550 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.550 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.550 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.550 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-27T11:08:13.550 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.550 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.550 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.550 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.550 INFO:teuthology.orchestra.run.smithi097.stdout: "facts_cache_timeout": { 2024-03-27T11:08:13.550 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "facts_cache_timeout", 2024-03-27T11:08:13.551 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-27T11:08:13.551 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.551 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.551 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "60", 2024-03-27T11:08:13.551 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.551 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.551 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.551 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "seconds to cache host facts data", 2024-03-27T11:08:13.551 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.551 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.551 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.551 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.551 INFO:teuthology.orchestra.run.smithi097.stdout: "grafana_dashboards_path": { 2024-03-27T11:08:13.551 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "grafana_dashboards_path", 2024-03-27T11:08:13.552 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.552 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.552 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.552 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-03-27T11:08:13.552 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.552 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.552 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.552 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-03-27T11:08:13.552 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.552 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.552 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.552 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.552 INFO:teuthology.orchestra.run.smithi097.stdout: "host_check_interval": { 2024-03-27T11:08:13.552 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "host_check_interval", 2024-03-27T11:08:13.552 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-27T11:08:13.552 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.553 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.553 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "600", 2024-03-27T11:08:13.553 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.553 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.553 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.553 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "how frequently to perform a host check", 2024-03-27T11:08:13.553 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.553 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.553 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.553 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.553 INFO:teuthology.orchestra.run.smithi097.stdout: "hw_monitoring": { 2024-03-27T11:08:13.553 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "hw_monitoring", 2024-03-27T11:08:13.553 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.553 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.553 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.554 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.554 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.554 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.554 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.554 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-27T11:08:13.554 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.554 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.554 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.554 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.554 INFO:teuthology.orchestra.run.smithi097.stdout: "inventory_list_all": { 2024-03-27T11:08:13.554 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "inventory_list_all", 2024-03-27T11:08:13.554 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.554 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.554 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.554 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.555 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.555 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.555 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.555 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-27T11:08:13.555 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.555 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.555 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.555 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.555 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-27T11:08:13.555 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-27T11:08:13.555 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.555 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.555 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.555 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:13.555 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.556 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.556 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:13.556 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:13.556 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:13.556 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:13.556 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:13.556 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:13.556 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:13.556 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:13.556 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.556 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.556 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.556 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.556 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.556 INFO:teuthology.orchestra.run.smithi097.stdout: "log_refresh_metadata": { 2024-03-27T11:08:13.557 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_refresh_metadata", 2024-03-27T11:08:13.557 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.557 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.557 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.557 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.557 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.557 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.557 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.557 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-27T11:08:13.557 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.557 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.557 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.557 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.557 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-27T11:08:13.558 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-27T11:08:13.558 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.558 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.558 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.558 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:13.558 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.558 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.558 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.558 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-27T11:08:13.558 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.558 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.558 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.558 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.559 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-27T11:08:13.559 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-27T11:08:13.559 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.559 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.559 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.559 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-27T11:08:13.559 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.559 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.559 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:13.559 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:13.559 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:13.559 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:13.559 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:13.559 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:13.560 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:13.560 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:13.560 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.560 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.560 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.560 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.560 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.560 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-27T11:08:13.560 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-27T11:08:13.560 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.560 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.560 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.560 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.561 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.561 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.561 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.561 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.561 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.561 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.561 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.561 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.561 INFO:teuthology.orchestra.run.smithi097.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-27T11:08:13.561 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-27T11:08:13.561 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.561 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.561 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.561 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.561 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.562 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.562 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.562 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-27T11:08:13.562 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.562 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.562 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.562 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.562 INFO:teuthology.orchestra.run.smithi097.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-27T11:08:13.562 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-27T11:08:13.562 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.562 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.562 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.562 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "*", 2024-03-27T11:08:13.562 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.563 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.563 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.563 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-27T11:08:13.563 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.563 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.563 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.563 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.563 INFO:teuthology.orchestra.run.smithi097.stdout: "max_count_per_host": { 2024-03-27T11:08:13.563 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "max_count_per_host", 2024-03-27T11:08:13.563 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:13.563 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.563 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.563 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10", 2024-03-27T11:08:13.563 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.564 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.564 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.564 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "max number of daemons per service per host", 2024-03-27T11:08:13.564 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.564 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.564 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.564 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.564 INFO:teuthology.orchestra.run.smithi097.stdout: "max_osd_draining_count": { 2024-03-27T11:08:13.564 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "max_osd_draining_count", 2024-03-27T11:08:13.564 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:13.564 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.564 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.564 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10", 2024-03-27T11:08:13.564 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.564 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.565 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.565 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-27T11:08:13.565 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.565 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.565 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.565 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.565 INFO:teuthology.orchestra.run.smithi097.stdout: "migration_current": { 2024-03-27T11:08:13.565 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "migration_current", 2024-03-27T11:08:13.565 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:13.565 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.565 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.565 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:13.565 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.565 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.565 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.566 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "internal - do not modify", 2024-03-27T11:08:13.566 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.566 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.566 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.566 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.566 INFO:teuthology.orchestra.run.smithi097.stdout: "mode": { 2024-03-27T11:08:13.566 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "mode", 2024-03-27T11:08:13.566 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.566 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.566 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.566 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "root", 2024-03-27T11:08:13.566 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.566 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.567 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:13.567 INFO:teuthology.orchestra.run.smithi097.stdout: "cephadm-package", 2024-03-27T11:08:13.567 INFO:teuthology.orchestra.run.smithi097.stdout: "root" 2024-03-27T11:08:13.567 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:13.567 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "mode for remote execution of cephadm", 2024-03-27T11:08:13.567 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.567 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.567 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.567 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.567 INFO:teuthology.orchestra.run.smithi097.stdout: "oob_default_addr": { 2024-03-27T11:08:13.567 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "oob_default_addr", 2024-03-27T11:08:13.567 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.568 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.568 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.568 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "169.254.1.1", 2024-03-27T11:08:13.568 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.568 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.568 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.568 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-27T11:08:13.568 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.568 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.568 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.568 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.568 INFO:teuthology.orchestra.run.smithi097.stdout: "prometheus_alerts_path": { 2024-03-27T11:08:13.568 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "prometheus_alerts_path", 2024-03-27T11:08:13.569 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.569 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.569 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.569 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-27T11:08:13.569 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.569 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.569 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.569 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-27T11:08:13.569 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.569 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.569 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.569 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.569 INFO:teuthology.orchestra.run.smithi097.stdout: "registry_insecure": { 2024-03-27T11:08:13.569 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "registry_insecure", 2024-03-27T11:08:13.570 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.570 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.570 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.570 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.570 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.570 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.570 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.570 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-27T11:08:13.570 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.570 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.570 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.570 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.570 INFO:teuthology.orchestra.run.smithi097.stdout: "registry_password": { 2024-03-27T11:08:13.571 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "registry_password", 2024-03-27T11:08:13.571 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.571 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.571 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.571 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:13.571 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.571 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.571 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.571 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-27T11:08:13.571 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.571 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.572 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.572 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.572 INFO:teuthology.orchestra.run.smithi097.stdout: "registry_url": { 2024-03-27T11:08:13.572 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "registry_url", 2024-03-27T11:08:13.572 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.572 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.572 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.572 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:13.572 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.572 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.572 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.572 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-27T11:08:13.572 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.573 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.573 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.573 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.573 INFO:teuthology.orchestra.run.smithi097.stdout: "registry_username": { 2024-03-27T11:08:13.573 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "registry_username", 2024-03-27T11:08:13.573 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.573 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.573 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.573 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:13.573 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.573 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.573 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.573 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-27T11:08:13.574 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.574 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.574 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.574 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.574 INFO:teuthology.orchestra.run.smithi097.stdout: "secure_monitoring_stack": { 2024-03-27T11:08:13.574 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "secure_monitoring_stack", 2024-03-27T11:08:13.574 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.574 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.574 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.574 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.574 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.574 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.575 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.575 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-27T11:08:13.575 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.575 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.575 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.575 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.575 INFO:teuthology.orchestra.run.smithi097.stdout: "service_discovery_port": { 2024-03-27T11:08:13.575 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "service_discovery_port", 2024-03-27T11:08:13.575 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:13.575 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.575 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.575 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "8765", 2024-03-27T11:08:13.575 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.575 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.576 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.576 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "cephadm service discovery port", 2024-03-27T11:08:13.576 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.576 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.576 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.576 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.576 INFO:teuthology.orchestra.run.smithi097.stdout: "ssh_config_file": { 2024-03-27T11:08:13.576 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ssh_config_file", 2024-03-27T11:08:13.576 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.576 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.576 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.576 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:13.576 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.577 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.577 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.577 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-27T11:08:13.577 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.577 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.577 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.577 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.577 INFO:teuthology.orchestra.run.smithi097.stdout: "use_agent": { 2024-03-27T11:08:13.577 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "use_agent", 2024-03-27T11:08:13.577 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.577 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.577 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.577 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.577 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.578 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.578 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.578 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-27T11:08:13.578 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.578 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.578 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.578 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.578 INFO:teuthology.orchestra.run.smithi097.stdout: "use_repo_digest": { 2024-03-27T11:08:13.578 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "use_repo_digest", 2024-03-27T11:08:13.578 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.578 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.578 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.578 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:13.579 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.579 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.579 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.579 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-27T11:08:13.579 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.579 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.579 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.579 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.579 INFO:teuthology.orchestra.run.smithi097.stdout: "warn_on_failed_host_check": { 2024-03-27T11:08:13.579 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "warn_on_failed_host_check", 2024-03-27T11:08:13.579 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.579 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.579 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.580 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:13.580 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.580 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.580 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.580 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "raise a health warning if the host check fails", 2024-03-27T11:08:13.580 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.580 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.580 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.580 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.580 INFO:teuthology.orchestra.run.smithi097.stdout: "warn_on_stray_daemons": { 2024-03-27T11:08:13.580 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "warn_on_stray_daemons", 2024-03-27T11:08:13.580 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.580 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.580 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.581 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:13.581 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.581 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.581 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.581 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-27T11:08:13.581 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.581 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.581 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.581 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.581 INFO:teuthology.orchestra.run.smithi097.stdout: "warn_on_stray_hosts": { 2024-03-27T11:08:13.581 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "warn_on_stray_hosts", 2024-03-27T11:08:13.581 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.581 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.582 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.582 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:13.582 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.582 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.582 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.582 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-27T11:08:13.582 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.582 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.582 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.582 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:13.582 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:13.582 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.582 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:13.582 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "crash", 2024-03-27T11:08:13.583 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-27T11:08:13.583 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-27T11:08:13.583 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-27T11:08:13.583 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-27T11:08:13.583 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-27T11:08:13.583 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.583 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.583 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.583 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:13.583 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.583 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.583 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:13.584 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:13.584 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:13.584 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:13.584 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:13.584 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:13.584 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:13.584 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:13.584 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.584 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.584 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.584 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.584 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.584 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-27T11:08:13.585 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-27T11:08:13.585 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.585 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.585 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.585 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.585 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.585 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.585 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.585 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.585 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.585 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.585 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.585 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.585 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-27T11:08:13.585 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-27T11:08:13.586 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.586 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.586 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.586 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-27T11:08:13.586 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.586 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.586 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:13.586 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:13.586 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:13.586 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:13.586 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:13.586 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:13.586 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:13.587 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:13.587 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.587 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.587 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.587 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.587 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.587 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-27T11:08:13.587 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-27T11:08:13.587 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.587 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.587 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.587 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.587 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.588 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.588 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.588 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.588 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.588 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.588 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.588 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.588 INFO:teuthology.orchestra.run.smithi097.stdout: "retain_interval": { 2024-03-27T11:08:13.588 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "retain_interval", 2024-03-27T11:08:13.588 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-27T11:08:13.588 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.588 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.588 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "31536000", 2024-03-27T11:08:13.588 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.589 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.589 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.589 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-27T11:08:13.589 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.589 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.589 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.589 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.589 INFO:teuthology.orchestra.run.smithi097.stdout: "warn_recent_interval": { 2024-03-27T11:08:13.589 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "warn_recent_interval", 2024-03-27T11:08:13.589 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-27T11:08:13.589 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.589 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.589 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "1209600", 2024-03-27T11:08:13.589 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.589 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.590 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.590 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-27T11:08:13.590 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.590 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.590 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.590 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:13.590 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:13.590 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.590 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:13.590 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "dashboard", 2024-03-27T11:08:13.590 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-27T11:08:13.590 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-27T11:08:13.590 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-27T11:08:13.591 INFO:teuthology.orchestra.run.smithi097.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-27T11:08:13.591 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-27T11:08:13.591 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:13.591 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.591 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.591 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10", 2024-03-27T11:08:13.591 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.591 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.591 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.591 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.591 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.591 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.591 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.591 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.591 INFO:teuthology.orchestra.run.smithi097.stdout: "ALERTMANAGER_API_HOST": { 2024-03-27T11:08:13.591 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-27T11:08:13.592 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.592 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.592 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.592 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:13.592 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.592 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.592 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.592 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.592 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.592 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.592 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.592 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.592 INFO:teuthology.orchestra.run.smithi097.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-27T11:08:13.592 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-27T11:08:13.592 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.593 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.593 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.593 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:13.593 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.593 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.593 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.593 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.593 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.593 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.593 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.593 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.593 INFO:teuthology.orchestra.run.smithi097.stdout: "AUDIT_API_ENABLED": { 2024-03-27T11:08:13.593 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "AUDIT_API_ENABLED", 2024-03-27T11:08:13.593 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.593 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.594 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.594 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.594 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.594 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.594 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.594 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.594 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.594 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.594 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.594 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.594 INFO:teuthology.orchestra.run.smithi097.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-27T11:08:13.594 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-27T11:08:13.594 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.595 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.595 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.595 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:13.595 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.595 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.595 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.595 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.595 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.595 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.595 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.595 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.595 INFO:teuthology.orchestra.run.smithi097.stdout: "ENABLE_BROWSABLE_API": { 2024-03-27T11:08:13.595 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-27T11:08:13.595 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.595 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.596 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.596 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:13.596 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.596 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.596 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.596 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.596 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.596 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.596 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.596 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.596 INFO:teuthology.orchestra.run.smithi097.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-27T11:08:13.596 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-27T11:08:13.596 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.596 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.597 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.597 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:13.597 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.597 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.597 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.597 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.597 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.597 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.597 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.597 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.597 INFO:teuthology.orchestra.run.smithi097.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-27T11:08:13.597 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-27T11:08:13.597 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.597 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.598 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.598 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:13.598 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.598 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.598 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.598 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.598 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.598 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.598 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.598 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.598 INFO:teuthology.orchestra.run.smithi097.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-27T11:08:13.598 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-27T11:08:13.598 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.598 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.599 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.599 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:13.599 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.599 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.599 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.599 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.599 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.599 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.599 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.599 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.599 INFO:teuthology.orchestra.run.smithi097.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-27T11:08:13.599 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-27T11:08:13.599 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.599 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.600 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.600 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:13.600 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.600 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.600 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.600 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.600 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.600 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.600 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.600 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.600 INFO:teuthology.orchestra.run.smithi097.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-27T11:08:13.600 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-27T11:08:13.600 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.600 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.600 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.601 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:13.601 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.601 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.601 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.601 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.601 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.601 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.601 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.601 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.601 INFO:teuthology.orchestra.run.smithi097.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-27T11:08:13.601 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-27T11:08:13.601 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.601 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.601 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.602 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:13.602 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.602 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.602 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.602 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.602 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.602 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.602 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.602 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.602 INFO:teuthology.orchestra.run.smithi097.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-27T11:08:13.602 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-27T11:08:13.602 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.602 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.603 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.603 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:13.603 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.603 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.603 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.603 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.603 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.603 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.603 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.603 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.603 INFO:teuthology.orchestra.run.smithi097.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-27T11:08:13.603 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-27T11:08:13.603 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.603 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.603 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.603 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:13.604 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.604 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.604 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.604 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.604 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.604 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.604 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.604 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.604 INFO:teuthology.orchestra.run.smithi097.stdout: "GRAFANA_API_PASSWORD": { 2024-03-27T11:08:13.604 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-27T11:08:13.604 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.604 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.604 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.604 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "admin", 2024-03-27T11:08:13.605 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.605 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.605 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.605 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.605 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.605 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.605 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.605 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.605 INFO:teuthology.orchestra.run.smithi097.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-27T11:08:13.605 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-27T11:08:13.605 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.605 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.605 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.605 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:13.606 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.606 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.606 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.606 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.606 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.606 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.606 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.606 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.606 INFO:teuthology.orchestra.run.smithi097.stdout: "GRAFANA_API_URL": { 2024-03-27T11:08:13.606 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "GRAFANA_API_URL", 2024-03-27T11:08:13.606 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.606 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.606 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.606 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:13.607 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.607 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.607 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.607 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.607 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.607 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.607 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.607 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.607 INFO:teuthology.orchestra.run.smithi097.stdout: "GRAFANA_API_USERNAME": { 2024-03-27T11:08:13.607 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-27T11:08:13.607 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.607 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.607 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.607 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "admin", 2024-03-27T11:08:13.607 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.608 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.608 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.608 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.608 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.608 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.608 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.608 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.608 INFO:teuthology.orchestra.run.smithi097.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-27T11:08:13.608 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-27T11:08:13.608 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.608 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.608 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.608 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:13.608 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.609 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.609 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.609 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.609 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.609 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.609 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.609 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.609 INFO:teuthology.orchestra.run.smithi097.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-27T11:08:13.609 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-27T11:08:13.609 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.609 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.609 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.610 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.610 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.610 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.610 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.610 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.610 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.610 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.610 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.610 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.610 INFO:teuthology.orchestra.run.smithi097.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-27T11:08:13.610 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-27T11:08:13.610 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.610 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.611 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.611 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:13.611 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.611 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.611 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.611 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.611 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.611 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.611 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.611 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.611 INFO:teuthology.orchestra.run.smithi097.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-27T11:08:13.611 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-27T11:08:13.611 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.611 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.611 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.612 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:13.612 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.612 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.612 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.612 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.612 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.612 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.612 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.612 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.612 INFO:teuthology.orchestra.run.smithi097.stdout: "MULTICLUSTER_CONFIG": { 2024-03-27T11:08:13.612 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "MULTICLUSTER_CONFIG", 2024-03-27T11:08:13.612 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.612 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.613 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.613 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "{}", 2024-03-27T11:08:13.613 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.613 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.613 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.613 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.613 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.613 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.613 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.613 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.613 INFO:teuthology.orchestra.run.smithi097.stdout: "PROMETHEUS_API_HOST": { 2024-03-27T11:08:13.613 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-27T11:08:13.613 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.614 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.614 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.614 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:13.614 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.614 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.614 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.614 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.614 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.614 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.614 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.614 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.614 INFO:teuthology.orchestra.run.smithi097.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-27T11:08:13.614 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-27T11:08:13.614 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.615 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.615 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.615 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:13.615 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.615 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.615 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.615 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.615 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.615 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.615 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.615 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.615 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-27T11:08:13.615 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-27T11:08:13.616 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.616 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.616 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.616 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.616 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.616 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.616 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.616 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.616 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.616 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.616 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.616 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.616 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-27T11:08:13.617 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-27T11:08:13.617 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.617 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.617 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.617 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.617 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.617 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.617 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.617 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.617 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.617 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.617 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.617 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.617 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-27T11:08:13.618 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-27T11:08:13.618 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.618 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.618 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.618 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:13.618 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.618 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.618 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.618 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.618 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.618 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.618 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.618 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.618 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-27T11:08:13.619 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-27T11:08:13.619 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.619 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.619 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.619 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:13.619 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.619 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.619 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.619 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.619 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.619 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.619 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.619 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.619 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-27T11:08:13.620 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-27T11:08:13.620 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.620 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.620 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.620 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.620 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.620 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.620 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.620 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.620 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.620 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.620 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.621 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.621 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-27T11:08:13.621 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-27T11:08:13.621 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.621 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.621 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.621 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.621 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.621 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.621 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.621 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.621 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.621 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.621 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.621 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.622 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-27T11:08:13.622 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-27T11:08:13.622 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.622 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.622 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.622 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.622 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.622 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.622 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.622 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.622 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.622 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.622 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.623 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.623 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_ENABLED": { 2024-03-27T11:08:13.623 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-27T11:08:13.623 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.623 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.623 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.623 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:13.623 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.623 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.623 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.623 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.623 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.623 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.624 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.624 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.624 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-27T11:08:13.624 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-27T11:08:13.624 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.624 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.624 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.624 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-27T11:08:13.624 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.624 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.624 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.624 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.624 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.624 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.625 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.625 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.625 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-27T11:08:13.625 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-27T11:08:13.625 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:13.625 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.625 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.625 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10", 2024-03-27T11:08:13.625 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.625 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.625 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.625 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.625 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.625 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.626 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.626 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.626 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-27T11:08:13.626 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-27T11:08:13.626 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:13.626 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.626 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.626 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "8", 2024-03-27T11:08:13.626 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.626 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.626 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.626 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.626 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.627 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.627 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.627 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.627 INFO:teuthology.orchestra.run.smithi097.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-27T11:08:13.627 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-27T11:08:13.627 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:13.627 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.627 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.627 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "45", 2024-03-27T11:08:13.627 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.627 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.627 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.627 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.627 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.628 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.628 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.628 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.628 INFO:teuthology.orchestra.run.smithi097.stdout: "RGW_API_ACCESS_KEY": { 2024-03-27T11:08:13.628 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-27T11:08:13.628 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.628 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.628 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.628 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:13.628 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.628 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.628 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.628 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.628 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.629 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.629 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.629 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.629 INFO:teuthology.orchestra.run.smithi097.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-27T11:08:13.629 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-27T11:08:13.629 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.629 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.629 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.629 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "admin", 2024-03-27T11:08:13.629 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.629 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.629 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.629 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.629 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.630 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.630 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.630 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.630 INFO:teuthology.orchestra.run.smithi097.stdout: "RGW_API_SECRET_KEY": { 2024-03-27T11:08:13.630 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-27T11:08:13.630 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.630 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.630 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.630 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:13.630 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.630 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.630 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.630 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.630 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.631 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.631 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.631 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.631 INFO:teuthology.orchestra.run.smithi097.stdout: "RGW_API_SSL_VERIFY": { 2024-03-27T11:08:13.631 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-27T11:08:13.631 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.631 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.631 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.631 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:13.631 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.631 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.631 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.631 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.632 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.632 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.632 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.632 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.632 INFO:teuthology.orchestra.run.smithi097.stdout: "UNSAFE_TLS_v1_2": { 2024-03-27T11:08:13.632 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "UNSAFE_TLS_v1_2", 2024-03-27T11:08:13.632 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.632 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.632 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.632 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.632 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.632 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.632 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.632 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.633 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.633 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.633 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.633 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.633 INFO:teuthology.orchestra.run.smithi097.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-27T11:08:13.633 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-27T11:08:13.633 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:13.633 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.633 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.633 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "0", 2024-03-27T11:08:13.633 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.633 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.633 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.633 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.634 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.634 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.634 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.634 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.634 INFO:teuthology.orchestra.run.smithi097.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-27T11:08:13.634 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-27T11:08:13.634 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:13.634 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.634 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.634 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10", 2024-03-27T11:08:13.634 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.634 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.634 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.634 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.635 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.635 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.635 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.635 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.635 INFO:teuthology.orchestra.run.smithi097.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-27T11:08:13.635 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-27T11:08:13.635 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:13.635 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.635 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.635 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "5", 2024-03-27T11:08:13.635 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.635 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.635 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.635 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.635 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.636 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.636 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.636 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.636 INFO:teuthology.orchestra.run.smithi097.stdout: "cross_origin_url": { 2024-03-27T11:08:13.636 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "cross_origin_url", 2024-03-27T11:08:13.636 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.636 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.636 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.636 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:13.636 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.636 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.636 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.637 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.637 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.637 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.637 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.637 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.637 INFO:teuthology.orchestra.run.smithi097.stdout: "crt_file": { 2024-03-27T11:08:13.637 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "crt_file", 2024-03-27T11:08:13.637 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.637 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.637 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.637 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:13.637 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.637 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.637 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.638 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.638 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.638 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.638 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.638 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.638 INFO:teuthology.orchestra.run.smithi097.stdout: "debug": { 2024-03-27T11:08:13.638 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "debug", 2024-03-27T11:08:13.638 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.638 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.638 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.638 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.638 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.638 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.639 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.639 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Enable/disable debug options", 2024-03-27T11:08:13.639 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.639 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.639 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.639 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.639 INFO:teuthology.orchestra.run.smithi097.stdout: "jwt_token_ttl": { 2024-03-27T11:08:13.639 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "jwt_token_ttl", 2024-03-27T11:08:13.639 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:13.639 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.639 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.639 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "28800", 2024-03-27T11:08:13.639 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.639 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.640 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.640 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.640 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.640 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.640 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.640 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.640 INFO:teuthology.orchestra.run.smithi097.stdout: "key_file": { 2024-03-27T11:08:13.640 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "key_file", 2024-03-27T11:08:13.640 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.640 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.640 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.640 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:13.640 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.640 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.641 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.641 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.641 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.641 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.641 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.641 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.641 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-27T11:08:13.641 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-27T11:08:13.641 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.641 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.641 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.641 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:13.641 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.641 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.642 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:13.642 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:13.642 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:13.642 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:13.642 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:13.642 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:13.642 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:13.642 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:13.642 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.642 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.642 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.642 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.642 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.643 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-27T11:08:13.643 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-27T11:08:13.643 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.643 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.643 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.643 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.643 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.643 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.643 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.643 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.643 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.643 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.643 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.643 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.644 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-27T11:08:13.644 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-27T11:08:13.644 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.644 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.644 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.644 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-27T11:08:13.644 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.644 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.644 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:13.644 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:13.644 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:13.644 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:13.644 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:13.644 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:13.645 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:13.645 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:13.645 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.645 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.645 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.645 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.645 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.645 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-27T11:08:13.645 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-27T11:08:13.645 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.645 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.645 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.645 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.645 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.646 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.646 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.646 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.646 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.646 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.646 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.646 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.646 INFO:teuthology.orchestra.run.smithi097.stdout: "motd": { 2024-03-27T11:08:13.646 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "motd", 2024-03-27T11:08:13.646 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.646 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.646 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.646 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:13.647 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.647 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.647 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.647 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "The message of the day", 2024-03-27T11:08:13.647 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.647 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.647 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.647 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.647 INFO:teuthology.orchestra.run.smithi097.stdout: "redirect_resolve_ip_addr": { 2024-03-27T11:08:13.647 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "redirect_resolve_ip_addr", 2024-03-27T11:08:13.647 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.647 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.647 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.647 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.648 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.648 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.648 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.648 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.648 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.648 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.648 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.648 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.648 INFO:teuthology.orchestra.run.smithi097.stdout: "server_addr": { 2024-03-27T11:08:13.648 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "server_addr", 2024-03-27T11:08:13.648 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.648 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.648 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.648 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "::", 2024-03-27T11:08:13.649 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.649 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.649 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.649 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.649 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.649 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.649 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.649 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.649 INFO:teuthology.orchestra.run.smithi097.stdout: "server_port": { 2024-03-27T11:08:13.649 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "server_port", 2024-03-27T11:08:13.649 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:13.649 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.649 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.649 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "8080", 2024-03-27T11:08:13.650 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.650 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.650 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.650 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.650 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.650 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.650 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.650 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.650 INFO:teuthology.orchestra.run.smithi097.stdout: "ssl": { 2024-03-27T11:08:13.650 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ssl", 2024-03-27T11:08:13.650 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.650 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.651 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.651 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:13.651 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.651 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.651 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.651 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.651 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.651 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.651 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.651 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.651 INFO:teuthology.orchestra.run.smithi097.stdout: "ssl_server_port": { 2024-03-27T11:08:13.651 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ssl_server_port", 2024-03-27T11:08:13.651 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:13.652 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.652 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.652 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "8443", 2024-03-27T11:08:13.652 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.652 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.652 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.652 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.652 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.652 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.652 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.652 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.652 INFO:teuthology.orchestra.run.smithi097.stdout: "standby_behaviour": { 2024-03-27T11:08:13.652 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "standby_behaviour", 2024-03-27T11:08:13.652 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.653 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.653 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.653 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "redirect", 2024-03-27T11:08:13.653 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.653 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.653 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:13.653 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:13.653 INFO:teuthology.orchestra.run.smithi097.stdout: "redirect" 2024-03-27T11:08:13.653 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:13.653 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.653 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.653 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.653 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.654 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.654 INFO:teuthology.orchestra.run.smithi097.stdout: "standby_error_status_code": { 2024-03-27T11:08:13.654 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "standby_error_status_code", 2024-03-27T11:08:13.654 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:13.654 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.654 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.654 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "500", 2024-03-27T11:08:13.654 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "400", 2024-03-27T11:08:13.654 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "599", 2024-03-27T11:08:13.654 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.654 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.654 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.654 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.655 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.655 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.655 INFO:teuthology.orchestra.run.smithi097.stdout: "url_prefix": { 2024-03-27T11:08:13.655 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "url_prefix", 2024-03-27T11:08:13.655 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.655 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.655 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.655 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:13.655 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.655 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.655 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.655 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.655 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.655 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.656 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.656 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:13.656 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:13.656 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.656 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:13.656 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "devicehealth", 2024-03-27T11:08:13.656 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-27T11:08:13.656 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-27T11:08:13.656 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-27T11:08:13.656 INFO:teuthology.orchestra.run.smithi097.stdout: "enable_monitoring": { 2024-03-27T11:08:13.656 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "enable_monitoring", 2024-03-27T11:08:13.656 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.656 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.657 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.657 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:13.657 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.657 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.657 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.657 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "monitor device health metrics", 2024-03-27T11:08:13.657 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.657 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.657 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.657 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.657 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-27T11:08:13.657 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-27T11:08:13.657 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.657 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.658 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.658 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:13.658 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.658 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.658 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:13.658 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:13.658 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:13.658 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:13.658 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:13.658 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:13.658 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:13.658 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:13.658 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.658 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.659 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.659 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.659 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.659 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-27T11:08:13.659 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-27T11:08:13.659 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.659 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.659 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.659 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.659 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.659 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.659 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.659 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.660 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.660 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.660 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.660 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.660 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-27T11:08:13.660 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-27T11:08:13.660 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.660 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.660 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.660 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-27T11:08:13.660 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.660 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.660 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:13.661 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:13.661 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:13.661 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:13.661 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:13.661 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:13.661 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:13.661 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:13.661 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.661 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.661 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.661 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.661 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.661 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-27T11:08:13.661 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-27T11:08:13.661 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.662 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.662 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.662 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.662 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.662 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.662 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.662 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.662 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.662 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.662 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.662 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.662 INFO:teuthology.orchestra.run.smithi097.stdout: "mark_out_threshold": { 2024-03-27T11:08:13.662 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "mark_out_threshold", 2024-03-27T11:08:13.662 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-27T11:08:13.663 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.663 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.663 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "2419200", 2024-03-27T11:08:13.663 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.663 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.663 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.663 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-27T11:08:13.663 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.663 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.663 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.663 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.663 INFO:teuthology.orchestra.run.smithi097.stdout: "pool_name": { 2024-03-27T11:08:13.663 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "pool_name", 2024-03-27T11:08:13.663 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.664 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.664 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.664 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "device_health_metrics", 2024-03-27T11:08:13.664 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.664 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.664 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.664 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-27T11:08:13.664 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.664 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.664 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.664 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.664 INFO:teuthology.orchestra.run.smithi097.stdout: "retention_period": { 2024-03-27T11:08:13.664 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "retention_period", 2024-03-27T11:08:13.665 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-27T11:08:13.665 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.665 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.665 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "15552000", 2024-03-27T11:08:13.665 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.665 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.665 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.665 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "how long to retain device health metrics", 2024-03-27T11:08:13.665 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.665 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.665 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.665 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.665 INFO:teuthology.orchestra.run.smithi097.stdout: "scrape_frequency": { 2024-03-27T11:08:13.665 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "scrape_frequency", 2024-03-27T11:08:13.665 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-27T11:08:13.666 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.666 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.666 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "86400", 2024-03-27T11:08:13.666 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.666 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.666 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.666 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-27T11:08:13.666 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.666 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.666 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.666 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.666 INFO:teuthology.orchestra.run.smithi097.stdout: "self_heal": { 2024-03-27T11:08:13.666 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "self_heal", 2024-03-27T11:08:13.667 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.667 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.667 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.667 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:13.667 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.667 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.667 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.667 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-27T11:08:13.667 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.667 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.667 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.667 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.667 INFO:teuthology.orchestra.run.smithi097.stdout: "sleep_interval": { 2024-03-27T11:08:13.667 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "sleep_interval", 2024-03-27T11:08:13.668 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-27T11:08:13.668 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.668 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.668 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "600", 2024-03-27T11:08:13.668 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.668 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.668 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.668 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "how frequently to wake up and check device health", 2024-03-27T11:08:13.668 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.668 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.668 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.668 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.668 INFO:teuthology.orchestra.run.smithi097.stdout: "warn_threshold": { 2024-03-27T11:08:13.668 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "warn_threshold", 2024-03-27T11:08:13.668 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-27T11:08:13.669 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.669 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.669 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "7257600", 2024-03-27T11:08:13.669 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.669 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.669 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.669 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-27T11:08:13.669 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.669 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.669 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.669 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:13.669 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:13.669 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.669 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:13.670 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "diskprediction_local", 2024-03-27T11:08:13.670 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-27T11:08:13.670 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-27T11:08:13.670 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-27T11:08:13.670 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-27T11:08:13.670 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-27T11:08:13.670 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.670 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.670 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.670 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:13.670 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.670 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.670 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:13.671 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:13.671 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:13.671 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:13.671 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:13.671 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:13.671 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:13.671 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:13.671 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.671 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.671 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.671 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.671 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.671 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-27T11:08:13.671 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-27T11:08:13.671 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.672 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.672 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.672 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.672 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.672 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.672 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.672 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.672 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.672 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.672 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.672 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.672 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-27T11:08:13.672 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-27T11:08:13.672 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.672 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.673 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.673 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-27T11:08:13.673 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.673 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.673 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:13.673 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:13.673 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:13.673 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:13.673 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:13.673 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:13.673 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:13.673 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:13.673 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.673 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.674 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.674 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.674 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.674 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-27T11:08:13.674 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-27T11:08:13.674 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.674 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.674 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.674 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.674 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.674 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.674 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.674 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.675 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.675 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.675 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.675 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.675 INFO:teuthology.orchestra.run.smithi097.stdout: "predict_interval": { 2024-03-27T11:08:13.675 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "predict_interval", 2024-03-27T11:08:13.675 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.675 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.675 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.675 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "86400", 2024-03-27T11:08:13.675 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.675 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.675 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.675 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.676 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.676 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.676 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.676 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.676 INFO:teuthology.orchestra.run.smithi097.stdout: "predictor_model": { 2024-03-27T11:08:13.676 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "predictor_model", 2024-03-27T11:08:13.676 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.676 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.676 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.676 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "prophetstor", 2024-03-27T11:08:13.676 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.676 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.676 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.676 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.676 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.676 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.677 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.677 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.677 INFO:teuthology.orchestra.run.smithi097.stdout: "sleep_interval": { 2024-03-27T11:08:13.677 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "sleep_interval", 2024-03-27T11:08:13.677 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.677 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.677 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.677 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "600", 2024-03-27T11:08:13.677 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.677 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.677 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.677 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.677 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.677 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.678 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.678 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:13.678 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:13.678 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.678 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:13.678 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "influx", 2024-03-27T11:08:13.678 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": false, 2024-03-27T11:08:13.678 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "influxdb python module not found", 2024-03-27T11:08:13.678 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-27T11:08:13.678 INFO:teuthology.orchestra.run.smithi097.stdout: "batch_size": { 2024-03-27T11:08:13.678 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "batch_size", 2024-03-27T11:08:13.678 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:13.679 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.679 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.679 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "5000", 2024-03-27T11:08:13.679 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.679 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.679 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.679 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-27T11:08:13.679 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.679 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.679 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.679 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.679 INFO:teuthology.orchestra.run.smithi097.stdout: "database": { 2024-03-27T11:08:13.679 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "database", 2024-03-27T11:08:13.679 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.680 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.680 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.680 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "ceph", 2024-03-27T11:08:13.680 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.680 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.680 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.680 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-27T11:08:13.680 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.680 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.680 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.680 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.680 INFO:teuthology.orchestra.run.smithi097.stdout: "hostname": { 2024-03-27T11:08:13.680 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "hostname", 2024-03-27T11:08:13.680 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.681 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.681 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.681 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:13.681 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.681 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.681 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.681 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "InfluxDB server hostname", 2024-03-27T11:08:13.681 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.681 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.681 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.681 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.681 INFO:teuthology.orchestra.run.smithi097.stdout: "interval": { 2024-03-27T11:08:13.681 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "interval", 2024-03-27T11:08:13.681 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-27T11:08:13.682 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.682 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.682 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "30", 2024-03-27T11:08:13.682 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "5", 2024-03-27T11:08:13.682 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.682 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.682 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-27T11:08:13.682 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.682 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.682 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.682 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.682 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-27T11:08:13.683 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-27T11:08:13.683 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.683 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.683 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.683 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:13.683 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.683 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.683 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:13.683 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:13.683 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:13.683 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:13.683 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:13.683 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:13.683 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:13.684 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:13.684 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.684 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.684 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.684 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.684 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.684 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-27T11:08:13.684 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-27T11:08:13.684 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.684 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.684 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.684 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.684 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.684 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.685 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.685 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.685 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.685 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.685 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.685 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.685 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-27T11:08:13.685 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-27T11:08:13.685 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.685 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.685 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.685 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-27T11:08:13.685 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.685 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.686 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:13.686 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:13.686 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:13.686 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:13.686 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:13.686 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:13.686 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:13.686 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:13.686 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.686 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.686 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.686 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.686 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.687 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-27T11:08:13.687 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-27T11:08:13.687 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.687 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.687 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.687 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.687 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.687 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.687 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.687 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.687 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.687 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.687 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.687 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.688 INFO:teuthology.orchestra.run.smithi097.stdout: "password": { 2024-03-27T11:08:13.688 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "password", 2024-03-27T11:08:13.688 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.688 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.688 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.688 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:13.688 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.688 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.688 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.688 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "password of InfluxDB server user", 2024-03-27T11:08:13.688 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.688 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.688 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.688 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.689 INFO:teuthology.orchestra.run.smithi097.stdout: "port": { 2024-03-27T11:08:13.689 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "port", 2024-03-27T11:08:13.689 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:13.689 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.689 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.689 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "8086", 2024-03-27T11:08:13.689 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.689 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.689 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.689 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "InfluxDB server port", 2024-03-27T11:08:13.689 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.689 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.689 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.689 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.689 INFO:teuthology.orchestra.run.smithi097.stdout: "ssl": { 2024-03-27T11:08:13.689 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ssl", 2024-03-27T11:08:13.690 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.690 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.690 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.690 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "false", 2024-03-27T11:08:13.690 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.690 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.690 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.690 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-27T11:08:13.690 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.690 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.690 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.690 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.690 INFO:teuthology.orchestra.run.smithi097.stdout: "threads": { 2024-03-27T11:08:13.690 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "threads", 2024-03-27T11:08:13.691 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:13.691 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.691 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.691 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "5", 2024-03-27T11:08:13.691 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "1", 2024-03-27T11:08:13.691 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "32", 2024-03-27T11:08:13.691 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.691 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-27T11:08:13.691 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.691 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.691 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.691 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.691 INFO:teuthology.orchestra.run.smithi097.stdout: "username": { 2024-03-27T11:08:13.692 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "username", 2024-03-27T11:08:13.692 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.692 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.692 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.692 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:13.692 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.692 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.692 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.692 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "username of InfluxDB server user", 2024-03-27T11:08:13.692 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.692 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.692 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.692 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.692 INFO:teuthology.orchestra.run.smithi097.stdout: "verify_ssl": { 2024-03-27T11:08:13.692 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "verify_ssl", 2024-03-27T11:08:13.693 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.693 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.693 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.693 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "true", 2024-03-27T11:08:13.693 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.693 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.693 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.693 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-27T11:08:13.693 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.693 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.693 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.693 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:13.693 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:13.693 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.693 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:13.694 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "insights", 2024-03-27T11:08:13.694 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-27T11:08:13.694 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-27T11:08:13.694 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-27T11:08:13.694 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-27T11:08:13.694 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-27T11:08:13.694 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.694 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.694 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.694 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:13.694 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.694 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.694 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:13.695 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:13.695 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:13.695 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:13.695 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:13.695 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:13.695 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:13.695 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:13.695 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.695 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.695 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.695 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.695 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.695 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-27T11:08:13.695 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-27T11:08:13.696 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.696 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.696 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.696 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.696 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.696 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.696 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.696 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.696 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.696 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.696 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.696 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.696 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-27T11:08:13.696 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-27T11:08:13.697 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.697 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.697 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.697 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-27T11:08:13.697 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.697 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.697 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:13.697 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:13.697 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:13.697 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:13.697 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:13.697 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:13.697 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:13.697 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:13.697 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.698 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.698 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.698 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.698 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.698 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-27T11:08:13.698 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-27T11:08:13.698 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.698 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.698 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.698 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.698 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.698 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.698 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.698 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.699 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.699 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.699 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.699 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:13.699 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:13.699 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.699 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:13.699 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "iostat", 2024-03-27T11:08:13.699 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-27T11:08:13.699 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-27T11:08:13.699 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-27T11:08:13.699 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-27T11:08:13.699 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-27T11:08:13.699 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.699 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.700 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.700 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:13.700 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.700 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.700 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:13.700 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:13.700 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:13.700 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:13.700 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:13.700 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:13.700 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:13.700 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:13.700 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.700 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.700 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.701 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.701 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.701 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-27T11:08:13.701 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-27T11:08:13.701 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.701 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.701 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.701 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.701 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.701 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.701 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.701 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.701 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.701 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.701 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.702 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.702 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-27T11:08:13.702 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-27T11:08:13.702 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.702 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.702 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.702 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-27T11:08:13.702 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.702 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.702 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:13.702 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:13.702 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:13.702 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:13.703 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:13.703 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:13.703 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:13.703 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:13.703 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.703 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.703 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.703 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.703 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.703 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-27T11:08:13.703 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-27T11:08:13.703 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.703 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.703 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.703 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.703 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.704 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.704 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.704 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.704 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.704 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.704 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.704 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:13.704 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:13.704 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.704 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:13.704 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "k8sevents", 2024-03-27T11:08:13.704 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-27T11:08:13.704 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-27T11:08:13.705 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-27T11:08:13.705 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_event_retention_days": { 2024-03-27T11:08:13.705 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ceph_event_retention_days", 2024-03-27T11:08:13.705 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:13.705 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.705 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.705 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "7", 2024-03-27T11:08:13.705 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.705 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.705 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.705 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-27T11:08:13.705 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.705 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.705 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.706 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.706 INFO:teuthology.orchestra.run.smithi097.stdout: "config_check_secs": { 2024-03-27T11:08:13.706 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "config_check_secs", 2024-03-27T11:08:13.706 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:13.706 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.706 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.706 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10", 2024-03-27T11:08:13.706 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "10", 2024-03-27T11:08:13.706 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.706 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.706 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-27T11:08:13.706 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.706 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.707 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.707 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.707 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-27T11:08:13.707 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-27T11:08:13.707 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.707 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.707 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.707 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:13.707 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.707 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.707 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:13.707 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:13.707 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:13.707 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:13.707 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:13.708 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:13.708 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:13.708 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:13.708 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.708 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.708 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.708 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.708 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.708 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-27T11:08:13.708 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-27T11:08:13.708 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.708 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.708 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.708 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.708 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.709 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.709 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.709 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.709 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.709 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.709 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.709 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.709 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-27T11:08:13.709 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-27T11:08:13.709 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.709 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.709 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.709 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-27T11:08:13.710 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.710 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.710 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:13.710 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:13.710 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:13.710 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:13.710 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:13.710 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:13.710 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:13.710 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:13.710 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.710 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.710 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.710 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.711 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.711 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-27T11:08:13.711 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-27T11:08:13.711 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.711 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.711 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.711 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.711 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.711 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.711 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.711 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.711 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.711 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.711 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.712 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:13.712 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:13.712 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.712 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:13.712 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "localpool", 2024-03-27T11:08:13.712 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-27T11:08:13.712 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-27T11:08:13.712 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-27T11:08:13.712 INFO:teuthology.orchestra.run.smithi097.stdout: "failure_domain": { 2024-03-27T11:08:13.712 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "failure_domain", 2024-03-27T11:08:13.712 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.712 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.712 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.712 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "host", 2024-03-27T11:08:13.712 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.713 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.713 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.713 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "failure domain for any created local pool", 2024-03-27T11:08:13.713 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-27T11:08:13.713 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.713 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.713 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.713 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-27T11:08:13.713 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-27T11:08:13.713 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.713 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.713 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.713 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:13.713 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.714 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.714 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:13.714 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:13.714 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:13.714 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:13.714 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:13.714 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:13.714 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:13.714 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:13.714 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.714 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.714 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.714 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.714 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.715 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-27T11:08:13.715 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-27T11:08:13.715 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.715 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.715 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.715 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.715 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.715 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.715 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.715 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.715 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.715 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.715 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.715 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.715 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-27T11:08:13.716 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-27T11:08:13.716 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.716 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.716 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.716 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-27T11:08:13.716 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.716 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.716 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:13.716 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:13.716 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:13.716 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:13.716 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:13.716 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:13.716 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:13.716 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:13.716 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.717 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.717 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.717 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.717 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.717 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-27T11:08:13.717 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-27T11:08:13.717 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.717 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.717 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.717 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.717 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.717 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.717 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.717 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.718 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.718 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.718 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.718 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.718 INFO:teuthology.orchestra.run.smithi097.stdout: "min_size": { 2024-03-27T11:08:13.718 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "min_size", 2024-03-27T11:08:13.718 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:13.718 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.718 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.718 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:13.718 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.718 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.718 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.719 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "default min_size for any created local pool", 2024-03-27T11:08:13.719 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-27T11:08:13.719 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.719 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.719 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.719 INFO:teuthology.orchestra.run.smithi097.stdout: "num_rep": { 2024-03-27T11:08:13.719 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "num_rep", 2024-03-27T11:08:13.719 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:13.719 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.719 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.719 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "3", 2024-03-27T11:08:13.719 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.719 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.719 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.719 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "default replica count for any created local pool", 2024-03-27T11:08:13.720 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.720 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.720 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.720 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.720 INFO:teuthology.orchestra.run.smithi097.stdout: "pg_num": { 2024-03-27T11:08:13.720 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "pg_num", 2024-03-27T11:08:13.720 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:13.720 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.720 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.720 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "128", 2024-03-27T11:08:13.720 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.720 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.720 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.720 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "default pg_num for any created local pool", 2024-03-27T11:08:13.720 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.721 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.721 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.721 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.721 INFO:teuthology.orchestra.run.smithi097.stdout: "prefix": { 2024-03-27T11:08:13.721 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "prefix", 2024-03-27T11:08:13.721 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.721 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.721 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.721 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:13.721 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.721 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.721 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.721 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "name prefix for any created local pool", 2024-03-27T11:08:13.721 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.722 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.722 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.722 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.722 INFO:teuthology.orchestra.run.smithi097.stdout: "subtree": { 2024-03-27T11:08:13.722 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "subtree", 2024-03-27T11:08:13.722 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.722 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.722 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.722 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "rack", 2024-03-27T11:08:13.722 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.722 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.722 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.722 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-27T11:08:13.722 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-27T11:08:13.723 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.723 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.723 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:13.723 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:13.723 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.723 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:13.723 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "mds_autoscaler", 2024-03-27T11:08:13.723 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-27T11:08:13.723 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-27T11:08:13.723 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-27T11:08:13.723 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-27T11:08:13.723 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-27T11:08:13.723 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.723 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.723 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.723 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:13.724 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.724 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.724 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:13.724 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:13.724 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:13.724 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:13.724 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:13.724 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:13.724 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:13.724 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:13.724 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.724 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.724 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.724 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.725 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.725 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-27T11:08:13.725 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-27T11:08:13.725 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.725 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.725 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.725 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.725 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.725 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.725 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.725 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.725 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.725 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.725 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.726 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.726 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-27T11:08:13.726 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-27T11:08:13.726 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.726 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.726 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.726 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-27T11:08:13.726 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.726 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.726 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:13.726 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:13.726 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:13.726 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:13.726 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:13.727 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:13.727 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:13.727 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:13.727 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.727 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.727 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.727 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.727 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.727 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-27T11:08:13.727 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-27T11:08:13.727 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.727 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.727 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.727 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.727 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.728 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.728 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.728 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.728 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.728 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.728 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.728 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:13.728 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:13.728 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.728 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:13.728 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "mirroring", 2024-03-27T11:08:13.728 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-27T11:08:13.728 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-27T11:08:13.728 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-27T11:08:13.728 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-27T11:08:13.729 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-27T11:08:13.729 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.729 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.729 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.729 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:13.729 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.729 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.729 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:13.729 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:13.729 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:13.729 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:13.729 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:13.729 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:13.729 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:13.729 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:13.729 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.730 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.730 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.730 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.730 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.730 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-27T11:08:13.730 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-27T11:08:13.730 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.730 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.730 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.730 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.730 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.730 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.730 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.730 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.731 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.731 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.731 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.731 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.731 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-27T11:08:13.731 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-27T11:08:13.731 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.731 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.731 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.731 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-27T11:08:13.731 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.731 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.731 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:13.731 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:13.732 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:13.732 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:13.732 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:13.732 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:13.732 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:13.732 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:13.732 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.732 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.732 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.732 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.732 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.732 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-27T11:08:13.732 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-27T11:08:13.733 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.733 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.733 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.733 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.733 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.733 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.733 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.733 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.733 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.733 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.733 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.733 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:13.733 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:13.733 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.734 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:13.734 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "nfs", 2024-03-27T11:08:13.734 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-27T11:08:13.734 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-27T11:08:13.734 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-27T11:08:13.734 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-27T11:08:13.734 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-27T11:08:13.734 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.734 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.734 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.734 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:13.734 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.734 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.734 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:13.735 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:13.735 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:13.735 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:13.735 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:13.735 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:13.735 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:13.735 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:13.735 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.735 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.735 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.735 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.735 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.735 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-27T11:08:13.735 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-27T11:08:13.736 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.736 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.736 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.736 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.736 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.736 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.736 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.736 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.736 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.736 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.736 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.736 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.736 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-27T11:08:13.737 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-27T11:08:13.737 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.737 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.737 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.737 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-27T11:08:13.737 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.737 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.737 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:13.737 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:13.737 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:13.737 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:13.737 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:13.737 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:13.737 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:13.737 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:13.738 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.738 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.738 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.738 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.738 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.738 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-27T11:08:13.738 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-27T11:08:13.738 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.738 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.738 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.738 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.738 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.738 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.738 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.738 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.739 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.739 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.739 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.739 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:13.739 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:13.739 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.739 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:13.739 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "orchestrator", 2024-03-27T11:08:13.739 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-27T11:08:13.739 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-27T11:08:13.739 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-27T11:08:13.739 INFO:teuthology.orchestra.run.smithi097.stdout: "fail_fs": { 2024-03-27T11:08:13.739 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "fail_fs", 2024-03-27T11:08:13.739 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.740 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.740 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.740 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.740 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.740 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.740 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.740 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-27T11:08:13.740 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.740 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.740 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.740 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.740 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-27T11:08:13.740 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-27T11:08:13.740 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.741 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.741 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.741 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:13.741 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.741 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.741 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:13.741 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:13.741 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:13.741 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:13.741 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:13.741 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:13.741 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:13.741 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:13.741 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.741 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.742 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.742 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.742 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.742 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-27T11:08:13.742 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-27T11:08:13.742 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.742 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.742 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.742 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.742 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.742 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.742 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.742 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.742 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.743 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.743 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.743 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.743 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-27T11:08:13.743 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-27T11:08:13.743 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.743 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.743 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.743 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-27T11:08:13.743 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.743 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.743 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:13.743 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:13.743 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:13.744 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:13.744 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:13.744 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:13.744 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:13.744 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:13.744 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.744 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.744 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.744 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.744 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.744 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-27T11:08:13.744 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-27T11:08:13.744 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.744 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.745 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.745 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.745 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.745 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.745 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.745 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.745 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.745 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.745 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.745 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.745 INFO:teuthology.orchestra.run.smithi097.stdout: "orchestrator": { 2024-03-27T11:08:13.745 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "orchestrator", 2024-03-27T11:08:13.746 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.746 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.746 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.746 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:13.746 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.746 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.746 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:13.746 INFO:teuthology.orchestra.run.smithi097.stdout: "cephadm", 2024-03-27T11:08:13.746 INFO:teuthology.orchestra.run.smithi097.stdout: "rook", 2024-03-27T11:08:13.746 INFO:teuthology.orchestra.run.smithi097.stdout: "test_orchestrator" 2024-03-27T11:08:13.746 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:13.746 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Orchestrator backend", 2024-03-27T11:08:13.746 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.747 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.747 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.747 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:13.747 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:13.747 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.747 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:13.747 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "osd_perf_query", 2024-03-27T11:08:13.747 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-27T11:08:13.747 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-27T11:08:13.747 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-27T11:08:13.747 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-27T11:08:13.747 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-27T11:08:13.747 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.747 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.747 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.747 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:13.748 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.748 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.748 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:13.748 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:13.748 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:13.748 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:13.748 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:13.748 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:13.748 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:13.748 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:13.748 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.748 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.748 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.748 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.748 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.749 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-27T11:08:13.749 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-27T11:08:13.749 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.749 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.749 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.749 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.749 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.749 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.749 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.749 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.749 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.749 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.749 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.749 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.749 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-27T11:08:13.750 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-27T11:08:13.750 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.750 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.750 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.750 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-27T11:08:13.750 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.750 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.750 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:13.750 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:13.750 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:13.750 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:13.750 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:13.751 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:13.751 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:13.751 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:13.751 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.751 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.751 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.751 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.751 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.751 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-27T11:08:13.751 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-27T11:08:13.751 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.751 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.752 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.752 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.752 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.752 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.752 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.752 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.752 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.752 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.752 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.752 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:13.752 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:13.752 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.752 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:13.752 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "osd_support", 2024-03-27T11:08:13.753 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-27T11:08:13.753 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-27T11:08:13.753 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-27T11:08:13.753 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-27T11:08:13.753 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-27T11:08:13.753 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.753 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.753 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.753 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:13.753 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.753 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.753 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:13.753 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:13.753 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:13.753 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:13.753 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:13.754 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:13.754 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:13.754 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:13.754 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.754 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.754 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.754 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.754 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.754 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-27T11:08:13.754 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-27T11:08:13.754 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.754 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.754 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.754 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.755 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.755 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.755 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.755 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.755 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.755 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.755 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.755 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.755 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-27T11:08:13.755 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-27T11:08:13.755 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.755 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.755 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.755 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-27T11:08:13.756 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.756 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.756 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:13.756 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:13.756 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:13.756 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:13.756 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:13.756 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:13.756 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:13.756 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:13.756 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.756 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.756 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.756 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.757 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.757 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-27T11:08:13.757 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-27T11:08:13.757 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.757 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.757 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.757 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.757 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.757 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.757 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.757 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.757 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.757 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.757 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.757 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:13.758 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:13.758 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.758 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:13.758 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "pg_autoscaler", 2024-03-27T11:08:13.758 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-27T11:08:13.758 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-27T11:08:13.758 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-27T11:08:13.758 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-27T11:08:13.758 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-27T11:08:13.758 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.758 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.758 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.758 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:13.758 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.759 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.759 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:13.759 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:13.759 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:13.759 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:13.759 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:13.759 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:13.759 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:13.759 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:13.759 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.759 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.759 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.759 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.760 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.760 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-27T11:08:13.760 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-27T11:08:13.760 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.760 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.760 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.760 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.760 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.760 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.760 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.760 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.760 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.760 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.761 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.761 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.761 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-27T11:08:13.761 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-27T11:08:13.761 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.761 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.761 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.761 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-27T11:08:13.761 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.761 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.761 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:13.761 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:13.761 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:13.762 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:13.762 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:13.762 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:13.762 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:13.762 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:13.762 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.762 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.762 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.762 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.762 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.762 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-27T11:08:13.762 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-27T11:08:13.762 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.762 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.763 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.763 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.763 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.763 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.763 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.763 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.763 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.763 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.763 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.763 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.763 INFO:teuthology.orchestra.run.smithi097.stdout: "sleep_interval": { 2024-03-27T11:08:13.763 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "sleep_interval", 2024-03-27T11:08:13.763 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-27T11:08:13.763 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.764 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.764 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "60", 2024-03-27T11:08:13.764 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.764 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.764 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.764 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.764 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.764 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.764 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.764 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.764 INFO:teuthology.orchestra.run.smithi097.stdout: "threshold": { 2024-03-27T11:08:13.764 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "threshold", 2024-03-27T11:08:13.764 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "float", 2024-03-27T11:08:13.764 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.764 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.765 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "3.0", 2024-03-27T11:08:13.765 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "1.0", 2024-03-27T11:08:13.765 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.765 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.765 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "scaling threshold", 2024-03-27T11:08:13.765 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-27T11:08:13.765 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.765 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.765 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:13.765 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:13.765 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.765 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:13.765 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "progress", 2024-03-27T11:08:13.766 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-27T11:08:13.766 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-27T11:08:13.766 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-27T11:08:13.766 INFO:teuthology.orchestra.run.smithi097.stdout: "allow_pg_recovery_event": { 2024-03-27T11:08:13.766 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "allow_pg_recovery_event", 2024-03-27T11:08:13.766 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.766 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.766 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.766 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.766 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.766 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.766 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.766 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-27T11:08:13.766 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.767 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.767 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.767 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.767 INFO:teuthology.orchestra.run.smithi097.stdout: "enabled": { 2024-03-27T11:08:13.767 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "enabled", 2024-03-27T11:08:13.767 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.767 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.767 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.767 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:13.767 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.767 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.767 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.767 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.768 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.768 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.768 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.768 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.768 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-27T11:08:13.768 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-27T11:08:13.768 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.768 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.768 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.768 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:13.768 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.768 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.768 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:13.768 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:13.768 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:13.769 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:13.769 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:13.769 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:13.769 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:13.769 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:13.769 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.769 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.769 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.769 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.769 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.769 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-27T11:08:13.769 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-27T11:08:13.769 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.769 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.770 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.770 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.770 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.770 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.770 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.770 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.770 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.770 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.770 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.770 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.770 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-27T11:08:13.770 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-27T11:08:13.770 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.770 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.771 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.771 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-27T11:08:13.771 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.771 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.771 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:13.771 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:13.771 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:13.771 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:13.771 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:13.771 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:13.771 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:13.771 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:13.771 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.771 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.771 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.771 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.772 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.772 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-27T11:08:13.772 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-27T11:08:13.772 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.772 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.772 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.772 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.772 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.772 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.772 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.772 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.772 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.772 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.772 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.772 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.773 INFO:teuthology.orchestra.run.smithi097.stdout: "max_completed_events": { 2024-03-27T11:08:13.773 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "max_completed_events", 2024-03-27T11:08:13.773 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:13.773 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.773 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.773 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "50", 2024-03-27T11:08:13.773 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.773 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.773 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.773 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "number of past completed events to remember", 2024-03-27T11:08:13.773 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.773 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.773 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.773 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.774 INFO:teuthology.orchestra.run.smithi097.stdout: "sleep_interval": { 2024-03-27T11:08:13.774 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "sleep_interval", 2024-03-27T11:08:13.774 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-27T11:08:13.774 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.774 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.774 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "5", 2024-03-27T11:08:13.774 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.774 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.774 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.774 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "how long the module is going to sleep", 2024-03-27T11:08:13.774 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.774 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.774 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.774 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:13.775 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:13.775 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.775 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:13.775 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "prometheus", 2024-03-27T11:08:13.775 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-27T11:08:13.775 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-27T11:08:13.775 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-27T11:08:13.775 INFO:teuthology.orchestra.run.smithi097.stdout: "cache": { 2024-03-27T11:08:13.775 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "cache", 2024-03-27T11:08:13.775 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.775 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.775 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.775 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:13.775 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.776 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.776 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.776 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.776 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.776 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.776 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.776 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.776 INFO:teuthology.orchestra.run.smithi097.stdout: "exclude_perf_counters": { 2024-03-27T11:08:13.776 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "exclude_perf_counters", 2024-03-27T11:08:13.776 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.776 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.776 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.776 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:13.776 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.776 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.777 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.777 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-27T11:08:13.777 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-27T11:08:13.777 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.777 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.777 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.777 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-27T11:08:13.777 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-27T11:08:13.777 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.777 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.777 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.777 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:13.777 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.777 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.778 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:13.778 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:13.778 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:13.778 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:13.778 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:13.778 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:13.778 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:13.778 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:13.778 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.778 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.778 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.778 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.778 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.779 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-27T11:08:13.779 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-27T11:08:13.779 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.779 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.779 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.779 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.779 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.779 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.779 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.779 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.779 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.779 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.779 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.779 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.780 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-27T11:08:13.780 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-27T11:08:13.780 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.780 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.780 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.780 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-27T11:08:13.780 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.780 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.780 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:13.780 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:13.780 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:13.780 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:13.780 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:13.780 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:13.780 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:13.780 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:13.781 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.781 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.781 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.781 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.781 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.781 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-27T11:08:13.781 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-27T11:08:13.781 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.781 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.781 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.781 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.781 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.781 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.781 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.782 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.782 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.782 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.782 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.782 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.782 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_stats_pools": { 2024-03-27T11:08:13.782 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rbd_stats_pools", 2024-03-27T11:08:13.782 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.782 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.782 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.782 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:13.782 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.782 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.782 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.783 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.783 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.783 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.783 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.783 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.783 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-27T11:08:13.783 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-27T11:08:13.783 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:13.783 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.783 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.783 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "300", 2024-03-27T11:08:13.783 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.783 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.783 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.783 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.783 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.784 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.784 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.784 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.784 INFO:teuthology.orchestra.run.smithi097.stdout: "scrape_interval": { 2024-03-27T11:08:13.784 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "scrape_interval", 2024-03-27T11:08:13.784 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "float", 2024-03-27T11:08:13.784 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.784 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.784 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "15.0", 2024-03-27T11:08:13.784 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.784 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.784 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.784 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.784 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.785 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.785 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.785 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.785 INFO:teuthology.orchestra.run.smithi097.stdout: "server_addr": { 2024-03-27T11:08:13.785 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "server_addr", 2024-03-27T11:08:13.785 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.785 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.785 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.785 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "::", 2024-03-27T11:08:13.785 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.785 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.785 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.785 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-27T11:08:13.785 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.786 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.786 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.786 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.786 INFO:teuthology.orchestra.run.smithi097.stdout: "server_port": { 2024-03-27T11:08:13.786 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "server_port", 2024-03-27T11:08:13.786 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:13.786 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.786 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.786 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "9283", 2024-03-27T11:08:13.786 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.786 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.786 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.786 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-27T11:08:13.786 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.787 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.787 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.787 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.787 INFO:teuthology.orchestra.run.smithi097.stdout: "stale_cache_strategy": { 2024-03-27T11:08:13.787 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "stale_cache_strategy", 2024-03-27T11:08:13.787 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.787 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.787 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.787 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "log", 2024-03-27T11:08:13.787 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.787 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.787 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.787 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.787 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.788 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.788 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.788 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.788 INFO:teuthology.orchestra.run.smithi097.stdout: "standby_behaviour": { 2024-03-27T11:08:13.788 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "standby_behaviour", 2024-03-27T11:08:13.788 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.788 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.788 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.788 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "default", 2024-03-27T11:08:13.788 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.788 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.788 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:13.788 INFO:teuthology.orchestra.run.smithi097.stdout: "default", 2024-03-27T11:08:13.789 INFO:teuthology.orchestra.run.smithi097.stdout: "error" 2024-03-27T11:08:13.789 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:13.789 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.789 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.789 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.789 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.789 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.789 INFO:teuthology.orchestra.run.smithi097.stdout: "standby_error_status_code": { 2024-03-27T11:08:13.789 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "standby_error_status_code", 2024-03-27T11:08:13.789 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:13.789 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.789 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.789 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "500", 2024-03-27T11:08:13.789 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "400", 2024-03-27T11:08:13.789 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "599", 2024-03-27T11:08:13.790 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.790 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.790 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.790 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.790 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.790 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:13.790 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:13.790 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.790 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:13.790 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rbd_support", 2024-03-27T11:08:13.790 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-27T11:08:13.790 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-27T11:08:13.790 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-27T11:08:13.790 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-27T11:08:13.791 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-27T11:08:13.791 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.791 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.791 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.791 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:13.791 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.791 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.791 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:13.791 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:13.791 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:13.791 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:13.791 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:13.791 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:13.791 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:13.792 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:13.792 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.792 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.792 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.792 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.792 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.792 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-27T11:08:13.792 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-27T11:08:13.792 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.792 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.792 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.792 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.792 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.792 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.792 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.793 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.793 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.793 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.793 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.793 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.793 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-27T11:08:13.793 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-27T11:08:13.793 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.793 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.793 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.793 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-27T11:08:13.793 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.793 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.793 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:13.793 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:13.793 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:13.794 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:13.794 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:13.794 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:13.794 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:13.794 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:13.794 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.794 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.794 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.794 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.794 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.794 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-27T11:08:13.794 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-27T11:08:13.794 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.794 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.795 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.795 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.795 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.795 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.795 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.795 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.795 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.795 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.795 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.795 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.795 INFO:teuthology.orchestra.run.smithi097.stdout: "max_concurrent_snap_create": { 2024-03-27T11:08:13.795 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "max_concurrent_snap_create", 2024-03-27T11:08:13.795 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:13.795 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.795 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.796 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10", 2024-03-27T11:08:13.796 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.796 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.796 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.796 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.796 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.796 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.796 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.796 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.796 INFO:teuthology.orchestra.run.smithi097.stdout: "mirror_snapshot_schedule": { 2024-03-27T11:08:13.796 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "mirror_snapshot_schedule", 2024-03-27T11:08:13.796 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.796 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.796 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.797 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:13.797 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.797 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.797 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.797 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.797 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.797 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.797 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.797 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.797 INFO:teuthology.orchestra.run.smithi097.stdout: "trash_purge_schedule": { 2024-03-27T11:08:13.797 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "trash_purge_schedule", 2024-03-27T11:08:13.797 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.797 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.797 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.797 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:13.798 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.798 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.798 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.798 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.798 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.798 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.798 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.798 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:13.798 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:13.798 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.798 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:13.798 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "restful", 2024-03-27T11:08:13.799 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-27T11:08:13.799 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-27T11:08:13.799 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-27T11:08:13.799 INFO:teuthology.orchestra.run.smithi097.stdout: "enable_auth": { 2024-03-27T11:08:13.799 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "enable_auth", 2024-03-27T11:08:13.799 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.799 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.799 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.799 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:13.799 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.799 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.799 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.799 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.800 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.800 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.800 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.800 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.800 INFO:teuthology.orchestra.run.smithi097.stdout: "key_file": { 2024-03-27T11:08:13.800 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "key_file", 2024-03-27T11:08:13.800 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.800 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.800 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.800 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:13.800 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.800 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.800 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.800 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.800 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.801 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.801 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.801 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.801 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-27T11:08:13.801 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-27T11:08:13.801 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.801 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.801 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.801 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:13.801 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.801 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.801 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:13.801 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:13.801 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:13.802 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:13.802 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:13.802 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:13.802 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:13.802 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:13.802 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.802 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.802 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.802 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.802 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.802 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-27T11:08:13.802 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-27T11:08:13.802 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.803 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.803 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.803 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.803 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.803 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.803 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.803 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.803 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.803 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.803 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.803 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.803 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-27T11:08:13.803 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-27T11:08:13.803 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.803 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.804 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.804 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-27T11:08:13.804 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.804 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.804 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:13.804 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:13.804 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:13.804 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:13.804 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:13.804 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:13.804 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:13.804 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:13.804 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.804 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.804 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.804 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.805 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.805 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-27T11:08:13.805 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-27T11:08:13.805 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.805 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.805 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.805 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.805 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.805 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.805 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.805 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.805 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.805 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.805 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.805 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.806 INFO:teuthology.orchestra.run.smithi097.stdout: "server_addr": { 2024-03-27T11:08:13.806 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "server_addr", 2024-03-27T11:08:13.806 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.806 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.806 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.806 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:13.806 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.806 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.806 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.806 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.806 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.806 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.806 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.807 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.807 INFO:teuthology.orchestra.run.smithi097.stdout: "server_port": { 2024-03-27T11:08:13.807 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "server_port", 2024-03-27T11:08:13.807 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.807 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.807 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.807 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:13.807 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.807 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.807 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.807 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.807 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.807 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.807 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.807 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:13.808 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:13.808 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.808 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:13.808 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rgw", 2024-03-27T11:08:13.808 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-27T11:08:13.808 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-27T11:08:13.808 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-27T11:08:13.808 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-27T11:08:13.808 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-27T11:08:13.808 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.808 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.808 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.808 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:13.808 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.809 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.809 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:13.809 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:13.809 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:13.809 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:13.809 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:13.809 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:13.809 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:13.809 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:13.809 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.809 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.809 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.809 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.809 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.810 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-27T11:08:13.810 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-27T11:08:13.810 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.810 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.810 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.810 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.810 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.810 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.810 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.810 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.810 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.810 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.810 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.810 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.811 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-27T11:08:13.811 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-27T11:08:13.811 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.811 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.811 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.811 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-27T11:08:13.811 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.811 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.811 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:13.811 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:13.811 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:13.811 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:13.811 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:13.811 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:13.811 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:13.812 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:13.812 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.812 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.812 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.812 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.812 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.812 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-27T11:08:13.812 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-27T11:08:13.812 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.812 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.812 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.812 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.812 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.812 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.813 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.813 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.813 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.813 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.813 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.813 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:13.813 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:13.813 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.813 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:13.813 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rook", 2024-03-27T11:08:13.813 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-27T11:08:13.813 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-27T11:08:13.813 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-27T11:08:13.813 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-27T11:08:13.813 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-27T11:08:13.814 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.814 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.814 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.814 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:13.814 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.814 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.814 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:13.814 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:13.814 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:13.814 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:13.814 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:13.814 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:13.815 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:13.815 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:13.815 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.815 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.815 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.815 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.815 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.815 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-27T11:08:13.815 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-27T11:08:13.815 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.815 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.815 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.815 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.815 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.816 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.816 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.816 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.816 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.816 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.816 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.816 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.816 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-27T11:08:13.816 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-27T11:08:13.816 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.816 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.816 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.816 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-27T11:08:13.816 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.817 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.817 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:13.817 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:13.817 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:13.817 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:13.817 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:13.817 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:13.817 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:13.817 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:13.817 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.817 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.817 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.817 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.817 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.818 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-27T11:08:13.818 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-27T11:08:13.818 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.818 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.818 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.818 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.818 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.818 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.818 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.818 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.818 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.818 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.819 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.819 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.819 INFO:teuthology.orchestra.run.smithi097.stdout: "storage_class": { 2024-03-27T11:08:13.819 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "storage_class", 2024-03-27T11:08:13.819 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.819 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.819 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.819 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "local", 2024-03-27T11:08:13.819 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.819 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.819 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.819 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-27T11:08:13.819 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.819 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.819 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.820 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:13.820 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:13.820 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.820 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:13.820 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "selftest", 2024-03-27T11:08:13.820 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-27T11:08:13.820 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-27T11:08:13.820 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-27T11:08:13.820 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-27T11:08:13.820 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-27T11:08:13.820 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.820 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.820 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.820 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:13.820 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.821 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.821 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:13.821 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:13.821 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:13.821 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:13.821 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:13.821 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:13.821 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:13.821 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:13.821 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.821 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.821 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.821 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.821 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.822 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-27T11:08:13.822 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-27T11:08:13.822 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.822 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.822 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.822 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.822 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.822 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.822 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.822 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.822 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.822 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.822 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.823 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.823 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-27T11:08:13.823 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-27T11:08:13.823 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.823 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.823 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.823 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-27T11:08:13.823 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.823 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.823 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:13.823 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:13.823 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:13.823 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:13.823 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:13.824 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:13.824 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:13.824 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:13.824 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.824 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.824 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.824 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.824 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.824 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-27T11:08:13.824 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-27T11:08:13.824 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.824 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.824 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.824 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.825 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.825 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.825 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.825 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.825 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.825 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.825 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.825 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.825 INFO:teuthology.orchestra.run.smithi097.stdout: "roption1": { 2024-03-27T11:08:13.825 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "roption1", 2024-03-27T11:08:13.825 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.825 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.825 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.825 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:13.825 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.826 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.826 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.826 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.826 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.826 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.826 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.826 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.826 INFO:teuthology.orchestra.run.smithi097.stdout: "roption2": { 2024-03-27T11:08:13.826 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "roption2", 2024-03-27T11:08:13.826 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.826 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.826 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.827 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "xyz", 2024-03-27T11:08:13.827 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.827 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.827 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.827 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.827 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.827 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.827 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.827 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.827 INFO:teuthology.orchestra.run.smithi097.stdout: "rwoption1": { 2024-03-27T11:08:13.827 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rwoption1", 2024-03-27T11:08:13.827 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.827 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.827 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.828 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:13.828 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.828 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.828 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.828 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.828 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.828 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.828 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.828 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.828 INFO:teuthology.orchestra.run.smithi097.stdout: "rwoption2": { 2024-03-27T11:08:13.828 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rwoption2", 2024-03-27T11:08:13.828 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:13.828 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.829 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.829 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:13.829 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.829 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.829 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.829 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.829 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.829 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.829 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.829 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.829 INFO:teuthology.orchestra.run.smithi097.stdout: "rwoption3": { 2024-03-27T11:08:13.829 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rwoption3", 2024-03-27T11:08:13.829 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "float", 2024-03-27T11:08:13.829 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.829 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.830 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:13.830 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.830 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.830 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.830 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.830 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.830 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.830 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.830 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.830 INFO:teuthology.orchestra.run.smithi097.stdout: "rwoption4": { 2024-03-27T11:08:13.830 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rwoption4", 2024-03-27T11:08:13.830 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.830 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.831 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.831 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:13.831 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.831 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.831 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.831 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.831 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.831 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.831 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.831 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.831 INFO:teuthology.orchestra.run.smithi097.stdout: "rwoption5": { 2024-03-27T11:08:13.831 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rwoption5", 2024-03-27T11:08:13.831 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.831 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.832 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.832 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:13.832 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.832 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.832 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.832 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.832 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.832 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.832 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.832 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.832 INFO:teuthology.orchestra.run.smithi097.stdout: "rwoption6": { 2024-03-27T11:08:13.832 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rwoption6", 2024-03-27T11:08:13.832 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.833 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.833 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.833 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:13.833 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.833 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.833 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.833 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.833 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.833 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.833 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.833 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.833 INFO:teuthology.orchestra.run.smithi097.stdout: "rwoption7": { 2024-03-27T11:08:13.833 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rwoption7", 2024-03-27T11:08:13.833 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:13.834 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.834 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.834 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:13.834 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "1", 2024-03-27T11:08:13.834 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "42", 2024-03-27T11:08:13.834 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.834 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.834 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.834 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.834 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.834 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.834 INFO:teuthology.orchestra.run.smithi097.stdout: "testkey": { 2024-03-27T11:08:13.834 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "testkey", 2024-03-27T11:08:13.835 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.835 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.835 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.835 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:13.835 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.835 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.835 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.835 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.835 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.835 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.835 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.835 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.835 INFO:teuthology.orchestra.run.smithi097.stdout: "testlkey": { 2024-03-27T11:08:13.835 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "testlkey", 2024-03-27T11:08:13.836 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.836 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.836 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.836 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:13.836 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.836 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.836 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.836 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.836 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.836 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.836 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.836 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.836 INFO:teuthology.orchestra.run.smithi097.stdout: "testnewline": { 2024-03-27T11:08:13.836 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "testnewline", 2024-03-27T11:08:13.837 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.837 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.837 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.837 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:13.837 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.837 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.837 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.837 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.837 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.837 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.837 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.837 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:13.837 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:13.837 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.838 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:13.838 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "snap_schedule", 2024-03-27T11:08:13.838 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-27T11:08:13.838 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-27T11:08:13.838 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-27T11:08:13.838 INFO:teuthology.orchestra.run.smithi097.stdout: "allow_m_granularity": { 2024-03-27T11:08:13.838 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "allow_m_granularity", 2024-03-27T11:08:13.838 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.838 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.838 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.838 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.838 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.838 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.839 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.839 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "allow minute scheduled snapshots", 2024-03-27T11:08:13.839 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.839 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.839 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.839 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.839 INFO:teuthology.orchestra.run.smithi097.stdout: "dump_on_update": { 2024-03-27T11:08:13.839 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "dump_on_update", 2024-03-27T11:08:13.839 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.839 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.839 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.839 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.839 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.839 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.840 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.840 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "dump database to debug log on update", 2024-03-27T11:08:13.840 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.840 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.840 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.840 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.840 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-27T11:08:13.840 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-27T11:08:13.840 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.840 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.840 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.840 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:13.840 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.840 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.841 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:13.841 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:13.841 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:13.841 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:13.841 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:13.841 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:13.841 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:13.841 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:13.841 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.841 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.841 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.841 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.841 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.841 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-27T11:08:13.842 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-27T11:08:13.842 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.842 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.842 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.842 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.842 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.842 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.842 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.842 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.842 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.842 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.842 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.842 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.842 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-27T11:08:13.843 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-27T11:08:13.843 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.843 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.843 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.843 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-27T11:08:13.843 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.843 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.843 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:13.843 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:13.843 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:13.843 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:13.843 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:13.843 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:13.843 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:13.844 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:13.844 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.844 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.844 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.844 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.844 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.844 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-27T11:08:13.844 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-27T11:08:13.844 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.844 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.844 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.844 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.844 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.844 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.845 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.845 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.845 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.845 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.845 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.845 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:13.845 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:13.845 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.845 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:13.845 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "stats", 2024-03-27T11:08:13.845 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-27T11:08:13.845 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-27T11:08:13.845 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-27T11:08:13.845 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-27T11:08:13.846 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-27T11:08:13.846 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.846 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.846 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.846 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:13.846 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.846 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.846 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:13.846 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:13.846 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:13.846 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:13.846 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:13.846 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:13.847 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:13.847 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:13.847 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.847 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.847 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.847 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.847 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.847 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-27T11:08:13.847 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-27T11:08:13.847 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.847 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.847 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.847 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.847 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.847 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.848 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.848 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.848 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.848 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.848 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.848 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.848 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-27T11:08:13.848 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-27T11:08:13.848 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.848 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.848 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.848 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-27T11:08:13.848 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.848 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.848 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:13.849 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:13.849 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:13.849 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:13.849 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:13.849 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:13.849 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:13.849 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:13.849 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.849 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.849 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.849 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.849 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.849 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-27T11:08:13.849 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-27T11:08:13.850 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.850 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.850 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.850 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.850 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.850 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.850 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.850 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.850 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.850 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.850 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.850 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:13.850 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:13.851 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.851 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:13.851 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "status", 2024-03-27T11:08:13.851 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-27T11:08:13.851 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-27T11:08:13.851 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-27T11:08:13.851 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-27T11:08:13.851 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-27T11:08:13.851 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.851 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.851 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.851 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:13.851 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.851 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.852 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:13.852 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:13.852 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:13.852 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:13.852 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:13.852 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:13.852 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:13.852 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:13.852 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.852 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.852 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.852 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.852 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.852 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-27T11:08:13.852 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-27T11:08:13.853 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.853 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.853 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.853 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.853 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.853 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.853 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.853 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.853 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.853 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.853 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.853 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.853 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-27T11:08:13.853 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-27T11:08:13.853 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.854 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.854 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.854 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-27T11:08:13.854 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.854 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.854 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:13.854 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:13.854 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:13.854 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:13.854 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:13.854 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:13.854 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:13.855 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:13.855 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.855 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.855 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.855 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.855 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.855 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-27T11:08:13.855 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-27T11:08:13.855 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.855 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.855 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.855 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.855 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.855 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.856 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.856 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.856 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.856 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.856 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.856 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:13.856 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:13.856 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.856 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:13.856 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "telegraf", 2024-03-27T11:08:13.856 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-27T11:08:13.856 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-27T11:08:13.856 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-27T11:08:13.856 INFO:teuthology.orchestra.run.smithi097.stdout: "address": { 2024-03-27T11:08:13.856 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "address", 2024-03-27T11:08:13.857 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.857 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.857 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.857 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-27T11:08:13.857 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.857 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.857 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.857 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.857 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.857 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.857 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.857 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.857 INFO:teuthology.orchestra.run.smithi097.stdout: "interval": { 2024-03-27T11:08:13.857 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "interval", 2024-03-27T11:08:13.858 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-27T11:08:13.858 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.858 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.858 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "15", 2024-03-27T11:08:13.858 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.858 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.858 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.858 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.858 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.858 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.858 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.858 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.858 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-27T11:08:13.858 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-27T11:08:13.859 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.859 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.859 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.859 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:13.859 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.859 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.859 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:13.859 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:13.859 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:13.859 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:13.859 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:13.859 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:13.859 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:13.859 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:13.860 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.860 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.860 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.860 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.860 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.860 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-27T11:08:13.860 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-27T11:08:13.860 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.860 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.860 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.860 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.860 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.860 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.861 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.861 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.861 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.861 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.861 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.861 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.861 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-27T11:08:13.861 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-27T11:08:13.861 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.861 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.861 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.861 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-27T11:08:13.861 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.862 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.862 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:13.862 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:13.862 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:13.862 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:13.862 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:13.862 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:13.862 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:13.862 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:13.862 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.862 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.862 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.862 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.863 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.863 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-27T11:08:13.863 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-27T11:08:13.863 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.863 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.863 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.863 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.863 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.863 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.863 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.863 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.863 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.863 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.863 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.863 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:13.864 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:13.864 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.864 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:13.864 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "telemetry", 2024-03-27T11:08:13.864 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-27T11:08:13.864 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-27T11:08:13.864 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-27T11:08:13.864 INFO:teuthology.orchestra.run.smithi097.stdout: "channel_basic": { 2024-03-27T11:08:13.864 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "channel_basic", 2024-03-27T11:08:13.864 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.864 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.864 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.864 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:13.864 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.864 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.864 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.865 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-27T11:08:13.865 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.865 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.865 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.865 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.865 INFO:teuthology.orchestra.run.smithi097.stdout: "channel_crash": { 2024-03-27T11:08:13.865 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "channel_crash", 2024-03-27T11:08:13.865 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.865 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.865 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.865 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:13.865 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.865 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.865 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.866 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-27T11:08:13.866 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.866 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.866 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.866 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.866 INFO:teuthology.orchestra.run.smithi097.stdout: "channel_device": { 2024-03-27T11:08:13.866 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "channel_device", 2024-03-27T11:08:13.866 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.866 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.866 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.866 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:13.866 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.866 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.867 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.867 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-27T11:08:13.867 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.867 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.867 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.867 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.867 INFO:teuthology.orchestra.run.smithi097.stdout: "channel_ident": { 2024-03-27T11:08:13.867 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "channel_ident", 2024-03-27T11:08:13.867 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.867 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.867 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.867 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.867 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.867 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.867 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.868 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-27T11:08:13.868 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.868 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.868 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.868 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.868 INFO:teuthology.orchestra.run.smithi097.stdout: "channel_perf": { 2024-03-27T11:08:13.868 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "channel_perf", 2024-03-27T11:08:13.868 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.868 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.868 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.868 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.868 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.868 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.868 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.869 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-27T11:08:13.869 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.869 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.869 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.869 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.869 INFO:teuthology.orchestra.run.smithi097.stdout: "contact": { 2024-03-27T11:08:13.869 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "contact", 2024-03-27T11:08:13.869 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.869 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.869 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.869 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:13.869 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.869 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.869 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.870 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.870 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.870 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.870 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.870 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.870 INFO:teuthology.orchestra.run.smithi097.stdout: "description": { 2024-03-27T11:08:13.870 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "description", 2024-03-27T11:08:13.870 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.870 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.870 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.870 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:13.870 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.870 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.871 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.871 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.871 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.871 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.871 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.871 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.871 INFO:teuthology.orchestra.run.smithi097.stdout: "device_url": { 2024-03-27T11:08:13.871 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "device_url", 2024-03-27T11:08:13.871 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.871 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.871 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.871 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-27T11:08:13.871 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.871 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.872 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.872 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.872 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.872 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.872 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.872 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.872 INFO:teuthology.orchestra.run.smithi097.stdout: "enabled": { 2024-03-27T11:08:13.872 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "enabled", 2024-03-27T11:08:13.872 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.872 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.872 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.872 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.872 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.872 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.872 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.873 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.873 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.873 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.873 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.873 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.873 INFO:teuthology.orchestra.run.smithi097.stdout: "interval": { 2024-03-27T11:08:13.873 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "interval", 2024-03-27T11:08:13.873 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:13.873 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.873 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.873 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "24", 2024-03-27T11:08:13.873 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "8", 2024-03-27T11:08:13.873 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.873 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.873 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.874 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.874 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.874 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.874 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.874 INFO:teuthology.orchestra.run.smithi097.stdout: "last_opt_revision": { 2024-03-27T11:08:13.874 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "last_opt_revision", 2024-03-27T11:08:13.874 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:13.874 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.874 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.874 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "1", 2024-03-27T11:08:13.874 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.874 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.874 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.875 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.875 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.875 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.875 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.875 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.875 INFO:teuthology.orchestra.run.smithi097.stdout: "leaderboard": { 2024-03-27T11:08:13.875 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "leaderboard", 2024-03-27T11:08:13.875 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.875 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.875 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.875 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.875 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.875 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.875 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.876 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.876 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.876 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.876 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.876 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.876 INFO:teuthology.orchestra.run.smithi097.stdout: "leaderboard_description": { 2024-03-27T11:08:13.876 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "leaderboard_description", 2024-03-27T11:08:13.876 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.876 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.876 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.876 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:13.876 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.876 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.876 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.876 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.877 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.877 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.877 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.877 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.877 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-27T11:08:13.877 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-27T11:08:13.877 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.877 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.877 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.877 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:13.877 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.877 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.877 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:13.877 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:13.877 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:13.878 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:13.878 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:13.878 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:13.878 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:13.878 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:13.878 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.878 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.878 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.878 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.878 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.878 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-27T11:08:13.878 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-27T11:08:13.878 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.879 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.879 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.879 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.879 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.879 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.879 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.879 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.879 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.879 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.879 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.879 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.879 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-27T11:08:13.879 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-27T11:08:13.879 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.879 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.880 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.880 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-27T11:08:13.880 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.880 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.880 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:13.880 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:13.880 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:13.880 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:13.880 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:13.880 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:13.880 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:13.880 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:13.880 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.880 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.880 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.880 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.881 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.881 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-27T11:08:13.881 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-27T11:08:13.881 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.881 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.881 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.881 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.881 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.881 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.881 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.881 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.881 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.881 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.881 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.882 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.882 INFO:teuthology.orchestra.run.smithi097.stdout: "organization": { 2024-03-27T11:08:13.882 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "organization", 2024-03-27T11:08:13.882 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.882 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.882 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.882 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:13.882 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.882 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.882 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.882 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.882 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.882 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.882 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.883 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.883 INFO:teuthology.orchestra.run.smithi097.stdout: "proxy": { 2024-03-27T11:08:13.883 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "proxy", 2024-03-27T11:08:13.883 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.883 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.883 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.883 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:13.883 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.883 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.883 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.883 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.883 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.883 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.883 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.883 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.884 INFO:teuthology.orchestra.run.smithi097.stdout: "url": { 2024-03-27T11:08:13.884 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "url", 2024-03-27T11:08:13.884 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.884 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.884 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.884 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-27T11:08:13.884 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.884 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.884 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.884 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.884 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.884 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.884 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.884 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:13.884 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:13.885 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.885 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:13.885 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "test_orchestrator", 2024-03-27T11:08:13.885 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-27T11:08:13.885 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-27T11:08:13.885 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-27T11:08:13.885 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-27T11:08:13.885 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-27T11:08:13.885 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.885 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.885 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.885 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:13.885 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.885 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.885 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:13.886 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:13.886 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:13.886 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:13.886 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:13.886 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:13.886 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:13.886 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:13.886 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.886 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.886 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.886 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.886 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.886 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-27T11:08:13.887 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-27T11:08:13.887 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.887 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.887 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.887 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.887 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.887 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.887 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.887 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.887 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.887 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.887 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.887 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.887 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-27T11:08:13.888 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-27T11:08:13.888 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.888 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.888 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.888 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-27T11:08:13.888 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.888 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.888 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:13.888 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:13.888 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:13.888 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:13.888 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:13.888 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:13.888 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:13.889 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:13.889 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.889 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.889 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.889 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.889 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.889 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-27T11:08:13.889 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-27T11:08:13.889 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.889 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.889 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.889 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.889 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.889 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.889 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.890 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.890 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.890 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.890 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.890 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:13.890 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:13.890 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.890 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:13.890 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "volumes", 2024-03-27T11:08:13.890 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-27T11:08:13.890 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-27T11:08:13.890 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-27T11:08:13.890 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-27T11:08:13.891 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-27T11:08:13.891 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.891 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.891 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.891 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:13.891 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.891 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.891 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:13.891 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:13.891 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:13.891 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:13.891 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:13.891 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:13.891 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:13.892 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:13.892 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.892 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.892 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.892 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.892 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.892 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-27T11:08:13.892 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-27T11:08:13.892 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.892 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.892 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.892 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.892 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.892 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.893 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.893 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.893 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.893 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.893 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.893 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.893 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-27T11:08:13.893 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-27T11:08:13.893 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.893 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.893 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.893 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-27T11:08:13.893 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.893 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.893 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:13.893 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:13.894 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:13.894 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:13.894 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:13.894 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:13.894 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:13.894 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:13.894 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.894 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.894 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.894 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.894 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.894 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-27T11:08:13.894 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-27T11:08:13.894 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.894 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.895 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.895 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.895 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.895 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.895 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.895 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.895 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.895 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.895 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.895 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.895 INFO:teuthology.orchestra.run.smithi097.stdout: "max_concurrent_clones": { 2024-03-27T11:08:13.895 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "max_concurrent_clones", 2024-03-27T11:08:13.895 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:13.895 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.896 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.896 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "4", 2024-03-27T11:08:13.896 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.896 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.896 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.896 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-27T11:08:13.896 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.896 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.896 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.896 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.896 INFO:teuthology.orchestra.run.smithi097.stdout: "periodic_async_work": { 2024-03-27T11:08:13.896 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "periodic_async_work", 2024-03-27T11:08:13.896 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.896 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.896 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.897 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.897 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.897 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.897 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.897 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Periodically check for async work", 2024-03-27T11:08:13.897 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.897 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.897 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.897 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.897 INFO:teuthology.orchestra.run.smithi097.stdout: "snapshot_clone_delay": { 2024-03-27T11:08:13.897 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "snapshot_clone_delay", 2024-03-27T11:08:13.897 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:13.897 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.897 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.898 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "0", 2024-03-27T11:08:13.898 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.898 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.898 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.898 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-27T11:08:13.898 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.898 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.898 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.898 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.898 INFO:teuthology.orchestra.run.smithi097.stdout: "snapshot_clone_no_wait": { 2024-03-27T11:08:13.898 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "snapshot_clone_no_wait", 2024-03-27T11:08:13.898 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.898 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.898 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.899 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:13.899 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.899 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.899 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.899 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-27T11:08:13.899 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.899 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.899 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.899 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:13.899 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:13.899 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.899 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:13.899 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "zabbix", 2024-03-27T11:08:13.899 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-27T11:08:13.899 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-27T11:08:13.900 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-27T11:08:13.900 INFO:teuthology.orchestra.run.smithi097.stdout: "discovery_interval": { 2024-03-27T11:08:13.900 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "discovery_interval", 2024-03-27T11:08:13.900 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "uint", 2024-03-27T11:08:13.900 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.900 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.900 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "100", 2024-03-27T11:08:13.900 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.900 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.900 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.900 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.900 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.900 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.900 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.900 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.901 INFO:teuthology.orchestra.run.smithi097.stdout: "identifier": { 2024-03-27T11:08:13.901 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "identifier", 2024-03-27T11:08:13.901 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.901 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.901 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.901 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:13.901 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.901 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.901 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.901 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.901 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.901 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.901 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.901 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.901 INFO:teuthology.orchestra.run.smithi097.stdout: "interval": { 2024-03-27T11:08:13.902 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "interval", 2024-03-27T11:08:13.902 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-27T11:08:13.902 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.902 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.902 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "60", 2024-03-27T11:08:13.902 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.902 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.902 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.902 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.902 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.902 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.902 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.902 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.902 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-27T11:08:13.902 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-27T11:08:13.903 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.903 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.903 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.903 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:13.903 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.903 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.903 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:13.903 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:13.903 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:13.903 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:13.903 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:13.903 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:13.903 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:13.903 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:13.903 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.904 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.904 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.904 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.904 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.904 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-27T11:08:13.904 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-27T11:08:13.904 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.904 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.904 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.904 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.904 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.904 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.904 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.904 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.904 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.905 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.905 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.905 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.905 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-27T11:08:13.905 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-27T11:08:13.905 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.905 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.905 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.905 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-27T11:08:13.905 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.905 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.905 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:13.905 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:13.905 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:13.905 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:13.906 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:13.906 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:13.906 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:13.906 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:13.906 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.906 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.906 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.906 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.906 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.906 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-27T11:08:13.906 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-27T11:08:13.906 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.906 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.906 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.907 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.907 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.907 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.907 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.907 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.907 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.907 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.907 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.907 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.907 INFO:teuthology.orchestra.run.smithi097.stdout: "zabbix_host": { 2024-03-27T11:08:13.907 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "zabbix_host", 2024-03-27T11:08:13.907 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.907 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.907 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.907 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:13.907 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.908 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.908 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.908 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.908 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.908 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.908 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.908 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.908 INFO:teuthology.orchestra.run.smithi097.stdout: "zabbix_port": { 2024-03-27T11:08:13.908 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "zabbix_port", 2024-03-27T11:08:13.908 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:13.908 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.908 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.908 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10051", 2024-03-27T11:08:13.908 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.909 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.909 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.909 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.909 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.909 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.909 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.909 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.909 INFO:teuthology.orchestra.run.smithi097.stdout: "zabbix_sender": { 2024-03-27T11:08:13.909 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "zabbix_sender", 2024-03-27T11:08:13.909 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.909 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.909 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.909 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-27T11:08:13.909 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.909 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.910 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.910 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.910 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.910 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.910 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.910 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:13.910 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:13.910 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:13.910 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-03-27T11:08:13.910 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.910 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:13.910 INFO:teuthology.orchestra.run.smithi097.stdout: "gid": 34109, 2024-03-27T11:08:13.910 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "y", 2024-03-27T11:08:13.910 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr_features": 4540701547738038271, 2024-03-27T11:08:13.911 INFO:teuthology.orchestra.run.smithi097.stdout: "available_modules": [ 2024-03-27T11:08:13.911 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:13.911 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "alerts", 2024-03-27T11:08:13.911 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-27T11:08:13.911 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-27T11:08:13.911 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-27T11:08:13.911 INFO:teuthology.orchestra.run.smithi097.stdout: "interval": { 2024-03-27T11:08:13.911 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "interval", 2024-03-27T11:08:13.911 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-27T11:08:13.911 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.911 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.911 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "60", 2024-03-27T11:08:13.911 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.911 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.911 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.912 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "How frequently to reexamine health status", 2024-03-27T11:08:13.912 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.912 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.912 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.912 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.912 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-27T11:08:13.912 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-27T11:08:13.912 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.912 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.912 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.912 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:13.912 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.912 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.912 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:13.913 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:13.913 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:13.913 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:13.913 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:13.913 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:13.913 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:13.913 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:13.913 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.913 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.913 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.913 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.913 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.913 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-27T11:08:13.913 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-27T11:08:13.913 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.914 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.914 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.914 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.914 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.914 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.914 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.914 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.914 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.914 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.914 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.914 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.914 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-27T11:08:13.914 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-27T11:08:13.914 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.914 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.915 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.915 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-27T11:08:13.915 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.915 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.915 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:13.915 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:13.915 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:13.915 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:13.915 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:13.915 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:13.915 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:13.915 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:13.915 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.915 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.916 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.916 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.916 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.916 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-27T11:08:13.916 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-27T11:08:13.916 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.916 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.916 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.916 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.916 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.916 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.916 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.916 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.916 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.916 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.916 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.917 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.917 INFO:teuthology.orchestra.run.smithi097.stdout: "smtp_destination": { 2024-03-27T11:08:13.917 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "smtp_destination", 2024-03-27T11:08:13.917 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.917 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.917 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.917 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:13.917 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.917 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.917 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.917 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Email address to send alerts to", 2024-03-27T11:08:13.917 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.917 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.917 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.917 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.918 INFO:teuthology.orchestra.run.smithi097.stdout: "smtp_from_name": { 2024-03-27T11:08:13.918 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "smtp_from_name", 2024-03-27T11:08:13.918 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.918 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.918 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.918 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "Ceph", 2024-03-27T11:08:13.918 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.918 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.918 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.918 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Email From: name", 2024-03-27T11:08:13.918 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.918 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.918 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.918 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.918 INFO:teuthology.orchestra.run.smithi097.stdout: "smtp_host": { 2024-03-27T11:08:13.919 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "smtp_host", 2024-03-27T11:08:13.919 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.919 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.919 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.919 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:13.919 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.919 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.919 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.919 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "SMTP server", 2024-03-27T11:08:13.919 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.919 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.919 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.919 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.919 INFO:teuthology.orchestra.run.smithi097.stdout: "smtp_password": { 2024-03-27T11:08:13.920 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "smtp_password", 2024-03-27T11:08:13.920 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.920 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.920 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.920 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:13.920 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.920 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.920 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.920 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Password to authenticate with", 2024-03-27T11:08:13.920 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.920 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.920 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.920 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.920 INFO:teuthology.orchestra.run.smithi097.stdout: "smtp_port": { 2024-03-27T11:08:13.920 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "smtp_port", 2024-03-27T11:08:13.920 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:13.921 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.921 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.921 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "465", 2024-03-27T11:08:13.921 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.921 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.921 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.921 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "SMTP port", 2024-03-27T11:08:13.921 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.921 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.921 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.921 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.921 INFO:teuthology.orchestra.run.smithi097.stdout: "smtp_sender": { 2024-03-27T11:08:13.921 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "smtp_sender", 2024-03-27T11:08:13.921 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.921 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.922 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.922 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:13.922 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.922 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.922 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.922 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "SMTP envelope sender", 2024-03-27T11:08:13.922 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.922 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.922 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.922 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.922 INFO:teuthology.orchestra.run.smithi097.stdout: "smtp_ssl": { 2024-03-27T11:08:13.922 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "smtp_ssl", 2024-03-27T11:08:13.922 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.922 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.923 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.923 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:13.923 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.923 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.923 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.923 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-27T11:08:13.923 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.923 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.923 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.923 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.923 INFO:teuthology.orchestra.run.smithi097.stdout: "smtp_user": { 2024-03-27T11:08:13.923 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "smtp_user", 2024-03-27T11:08:13.923 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.923 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.924 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.924 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:13.924 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.924 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.924 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.924 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "User to authenticate as", 2024-03-27T11:08:13.924 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.924 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.924 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.924 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:13.924 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:13.924 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.924 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:13.924 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "balancer", 2024-03-27T11:08:13.924 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-27T11:08:13.925 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-27T11:08:13.925 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-27T11:08:13.925 INFO:teuthology.orchestra.run.smithi097.stdout: "active": { 2024-03-27T11:08:13.925 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "active", 2024-03-27T11:08:13.925 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.925 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.925 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.925 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:13.925 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.925 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.925 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.925 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "automatically balance PGs across cluster", 2024-03-27T11:08:13.925 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.925 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.926 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.926 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.926 INFO:teuthology.orchestra.run.smithi097.stdout: "begin_time": { 2024-03-27T11:08:13.926 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "begin_time", 2024-03-27T11:08:13.926 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.926 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.926 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.926 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "0000", 2024-03-27T11:08:13.926 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.926 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.926 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.926 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "beginning time of day to automatically balance", 2024-03-27T11:08:13.926 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-27T11:08:13.927 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.927 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.927 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.927 INFO:teuthology.orchestra.run.smithi097.stdout: "begin_weekday": { 2024-03-27T11:08:13.927 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "begin_weekday", 2024-03-27T11:08:13.927 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "uint", 2024-03-27T11:08:13.927 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.927 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.927 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "0", 2024-03-27T11:08:13.927 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "0", 2024-03-27T11:08:13.927 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "6", 2024-03-27T11:08:13.927 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.927 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-27T11:08:13.927 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-27T11:08:13.927 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.928 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.928 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.928 INFO:teuthology.orchestra.run.smithi097.stdout: "crush_compat_max_iterations": { 2024-03-27T11:08:13.928 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "crush_compat_max_iterations", 2024-03-27T11:08:13.928 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "uint", 2024-03-27T11:08:13.928 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.928 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.928 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "25", 2024-03-27T11:08:13.928 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "1", 2024-03-27T11:08:13.928 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "250", 2024-03-27T11:08:13.928 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.928 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-27T11:08:13.928 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.928 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.929 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.929 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.929 INFO:teuthology.orchestra.run.smithi097.stdout: "crush_compat_metrics": { 2024-03-27T11:08:13.929 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "crush_compat_metrics", 2024-03-27T11:08:13.929 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.929 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.929 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.929 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "pgs,objects,bytes", 2024-03-27T11:08:13.929 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.929 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.929 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.929 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-27T11:08:13.929 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-27T11:08:13.929 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.929 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.930 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.930 INFO:teuthology.orchestra.run.smithi097.stdout: "crush_compat_step": { 2024-03-27T11:08:13.930 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "crush_compat_step", 2024-03-27T11:08:13.930 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "float", 2024-03-27T11:08:13.930 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.930 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.930 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "0.5", 2024-03-27T11:08:13.930 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "0.001", 2024-03-27T11:08:13.930 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "0.999", 2024-03-27T11:08:13.930 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.930 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "aggressiveness of optimization", 2024-03-27T11:08:13.930 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-27T11:08:13.930 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.930 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.930 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.931 INFO:teuthology.orchestra.run.smithi097.stdout: "end_time": { 2024-03-27T11:08:13.931 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "end_time", 2024-03-27T11:08:13.931 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.931 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.931 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.931 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "2359", 2024-03-27T11:08:13.931 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.931 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.931 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.931 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "ending time of day to automatically balance", 2024-03-27T11:08:13.931 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-27T11:08:13.931 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.931 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.931 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.931 INFO:teuthology.orchestra.run.smithi097.stdout: "end_weekday": { 2024-03-27T11:08:13.931 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "end_weekday", 2024-03-27T11:08:13.932 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "uint", 2024-03-27T11:08:13.932 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.932 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.932 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "0", 2024-03-27T11:08:13.932 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "0", 2024-03-27T11:08:13.932 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "6", 2024-03-27T11:08:13.932 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.932 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-27T11:08:13.932 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-27T11:08:13.932 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.932 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.932 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.932 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-27T11:08:13.933 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-27T11:08:13.933 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.933 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.933 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.933 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:13.933 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.933 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.933 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:13.933 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:13.933 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:13.933 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:13.933 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:13.933 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:13.933 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:13.934 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:13.934 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.934 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.934 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.934 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.934 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.934 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-27T11:08:13.934 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-27T11:08:13.934 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.934 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.934 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.934 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.934 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.934 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.935 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.935 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.935 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.935 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.935 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.935 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.935 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-27T11:08:13.935 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-27T11:08:13.935 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.935 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.935 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.935 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-27T11:08:13.935 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.935 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.936 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:13.936 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:13.936 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:13.936 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:13.936 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:13.936 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:13.936 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:13.936 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:13.936 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.936 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.936 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.936 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.936 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.936 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-27T11:08:13.937 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-27T11:08:13.937 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.937 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.937 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.937 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.937 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.937 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.937 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.937 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.937 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.937 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.937 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.937 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.937 INFO:teuthology.orchestra.run.smithi097.stdout: "min_score": { 2024-03-27T11:08:13.937 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "min_score", 2024-03-27T11:08:13.938 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "float", 2024-03-27T11:08:13.938 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.938 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.938 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "0", 2024-03-27T11:08:13.938 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.938 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.938 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.938 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-27T11:08:13.938 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.938 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.938 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.938 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.938 INFO:teuthology.orchestra.run.smithi097.stdout: "mode": { 2024-03-27T11:08:13.939 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "mode", 2024-03-27T11:08:13.939 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.939 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.939 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.939 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "upmap", 2024-03-27T11:08:13.939 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.939 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.939 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:13.939 INFO:teuthology.orchestra.run.smithi097.stdout: "crush-compat", 2024-03-27T11:08:13.939 INFO:teuthology.orchestra.run.smithi097.stdout: "none", 2024-03-27T11:08:13.939 INFO:teuthology.orchestra.run.smithi097.stdout: "read", 2024-03-27T11:08:13.939 INFO:teuthology.orchestra.run.smithi097.stdout: "upmap", 2024-03-27T11:08:13.939 INFO:teuthology.orchestra.run.smithi097.stdout: "upmap-read" 2024-03-27T11:08:13.940 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:13.940 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Balancer mode", 2024-03-27T11:08:13.940 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.940 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.940 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.940 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.940 INFO:teuthology.orchestra.run.smithi097.stdout: "pool_ids": { 2024-03-27T11:08:13.940 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "pool_ids", 2024-03-27T11:08:13.940 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.940 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.940 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.940 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:13.940 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.941 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.941 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.941 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-27T11:08:13.941 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.941 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.941 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.941 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.941 INFO:teuthology.orchestra.run.smithi097.stdout: "sleep_interval": { 2024-03-27T11:08:13.941 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "sleep_interval", 2024-03-27T11:08:13.941 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-27T11:08:13.941 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.941 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.941 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "60", 2024-03-27T11:08:13.941 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.941 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.942 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.942 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-27T11:08:13.942 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.942 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.942 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.942 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.942 INFO:teuthology.orchestra.run.smithi097.stdout: "upmap_max_deviation": { 2024-03-27T11:08:13.942 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "upmap_max_deviation", 2024-03-27T11:08:13.942 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:13.942 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.942 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.942 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "5", 2024-03-27T11:08:13.942 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "1", 2024-03-27T11:08:13.942 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.942 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.943 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-27T11:08:13.943 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-27T11:08:13.943 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.943 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.943 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.943 INFO:teuthology.orchestra.run.smithi097.stdout: "upmap_max_optimizations": { 2024-03-27T11:08:13.943 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "upmap_max_optimizations", 2024-03-27T11:08:13.943 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "uint", 2024-03-27T11:08:13.943 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.943 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.943 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10", 2024-03-27T11:08:13.943 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.943 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.943 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.944 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-27T11:08:13.944 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.944 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.944 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.944 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:13.944 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:13.944 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.944 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:13.944 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "cephadm", 2024-03-27T11:08:13.944 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-27T11:08:13.944 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-27T11:08:13.944 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-27T11:08:13.944 INFO:teuthology.orchestra.run.smithi097.stdout: "agent_down_multiplier": { 2024-03-27T11:08:13.944 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "agent_down_multiplier", 2024-03-27T11:08:13.944 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "float", 2024-03-27T11:08:13.945 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.945 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.945 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "3.0", 2024-03-27T11:08:13.945 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.945 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.945 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.945 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-27T11:08:13.945 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.945 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.945 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.945 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.945 INFO:teuthology.orchestra.run.smithi097.stdout: "agent_refresh_rate": { 2024-03-27T11:08:13.945 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "agent_refresh_rate", 2024-03-27T11:08:13.945 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-27T11:08:13.946 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.946 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.946 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "20", 2024-03-27T11:08:13.946 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.946 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.946 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.946 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-27T11:08:13.946 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.946 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.946 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.946 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.946 INFO:teuthology.orchestra.run.smithi097.stdout: "agent_starting_port": { 2024-03-27T11:08:13.946 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "agent_starting_port", 2024-03-27T11:08:13.946 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:13.947 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.947 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.947 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "4721", 2024-03-27T11:08:13.947 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.947 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.947 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.947 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-27T11:08:13.947 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.947 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.947 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.947 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.947 INFO:teuthology.orchestra.run.smithi097.stdout: "allow_ptrace": { 2024-03-27T11:08:13.947 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "allow_ptrace", 2024-03-27T11:08:13.947 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.948 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.948 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.948 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.948 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.948 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.948 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.948 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-27T11:08:13.948 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-27T11:08:13.948 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.948 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.948 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.948 INFO:teuthology.orchestra.run.smithi097.stdout: "autotune_interval": { 2024-03-27T11:08:13.948 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "autotune_interval", 2024-03-27T11:08:13.948 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-27T11:08:13.949 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.949 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.949 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "600", 2024-03-27T11:08:13.949 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.949 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.949 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.949 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-27T11:08:13.949 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.949 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.949 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.949 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.949 INFO:teuthology.orchestra.run.smithi097.stdout: "autotune_memory_target_ratio": { 2024-03-27T11:08:13.949 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "autotune_memory_target_ratio", 2024-03-27T11:08:13.949 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "float", 2024-03-27T11:08:13.949 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.950 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.950 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "0.7", 2024-03-27T11:08:13.950 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.950 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.950 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.950 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-27T11:08:13.950 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.950 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.950 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.950 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.950 INFO:teuthology.orchestra.run.smithi097.stdout: "cephadm_log_destination": { 2024-03-27T11:08:13.950 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "cephadm_log_destination", 2024-03-27T11:08:13.950 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.950 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.951 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.951 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:13.951 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.951 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.951 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:13.951 INFO:teuthology.orchestra.run.smithi097.stdout: "file", 2024-03-27T11:08:13.951 INFO:teuthology.orchestra.run.smithi097.stdout: "file,syslog", 2024-03-27T11:08:13.951 INFO:teuthology.orchestra.run.smithi097.stdout: "syslog" 2024-03-27T11:08:13.951 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:13.951 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-27T11:08:13.951 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.951 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.951 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.952 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.952 INFO:teuthology.orchestra.run.smithi097.stdout: "cgroups_split": { 2024-03-27T11:08:13.952 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "cgroups_split", 2024-03-27T11:08:13.952 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.952 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.952 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.952 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:13.952 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.952 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.952 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.952 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-27T11:08:13.952 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.952 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.952 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.953 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.953 INFO:teuthology.orchestra.run.smithi097.stdout: "config_checks_enabled": { 2024-03-27T11:08:13.953 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "config_checks_enabled", 2024-03-27T11:08:13.953 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.953 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.953 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.953 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.953 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.953 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.953 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.953 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-27T11:08:13.953 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.953 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.953 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.953 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.954 INFO:teuthology.orchestra.run.smithi097.stdout: "config_dashboard": { 2024-03-27T11:08:13.954 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "config_dashboard", 2024-03-27T11:08:13.954 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.954 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.954 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.954 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:13.954 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.954 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.954 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.954 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-27T11:08:13.954 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.954 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.954 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.954 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.955 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_alertmanager": { 2024-03-27T11:08:13.955 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_alertmanager", 2024-03-27T11:08:13.955 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.955 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.955 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.955 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-27T11:08:13.955 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.955 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.955 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.955 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Prometheus container image", 2024-03-27T11:08:13.955 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.955 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.955 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.955 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.955 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_base": { 2024-03-27T11:08:13.956 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_base", 2024-03-27T11:08:13.956 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.956 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.956 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.956 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-27T11:08:13.956 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.956 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.956 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.956 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Container image name, without the tag", 2024-03-27T11:08:13.956 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.956 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.956 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.956 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.956 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_elasticsearch": { 2024-03-27T11:08:13.957 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_elasticsearch", 2024-03-27T11:08:13.957 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.957 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.957 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.957 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-27T11:08:13.957 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.957 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.957 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.957 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "elasticsearch container image", 2024-03-27T11:08:13.957 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.957 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.957 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.957 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.957 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_grafana": { 2024-03-27T11:08:13.958 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_grafana", 2024-03-27T11:08:13.958 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.958 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.958 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.958 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-03-27T11:08:13.958 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.958 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.958 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.958 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Prometheus container image", 2024-03-27T11:08:13.958 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.958 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.958 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.958 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.959 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_haproxy": { 2024-03-27T11:08:13.959 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_haproxy", 2024-03-27T11:08:13.959 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.959 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.959 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.959 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-27T11:08:13.959 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.959 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.959 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.959 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "HAproxy container image", 2024-03-27T11:08:13.959 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.959 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.959 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.960 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.960 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_jaeger_agent": { 2024-03-27T11:08:13.960 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_jaeger_agent", 2024-03-27T11:08:13.960 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.960 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.960 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.960 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-27T11:08:13.960 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.960 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.960 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.960 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Jaeger agent container image", 2024-03-27T11:08:13.960 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.960 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.961 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.961 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.961 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_jaeger_collector": { 2024-03-27T11:08:13.961 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_jaeger_collector", 2024-03-27T11:08:13.961 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.961 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.961 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.961 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-27T11:08:13.961 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.961 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.961 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.961 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Jaeger collector container image", 2024-03-27T11:08:13.961 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.961 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.962 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.962 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.962 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_jaeger_query": { 2024-03-27T11:08:13.962 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_jaeger_query", 2024-03-27T11:08:13.962 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.962 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.962 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.962 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-27T11:08:13.962 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.962 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.962 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.963 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Jaeger query container image", 2024-03-27T11:08:13.963 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.963 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.963 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.963 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.963 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_keepalived": { 2024-03-27T11:08:13.963 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_keepalived", 2024-03-27T11:08:13.963 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.963 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.963 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.963 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-27T11:08:13.963 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.963 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.963 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.964 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Keepalived container image", 2024-03-27T11:08:13.964 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.964 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.964 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.964 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.964 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_loki": { 2024-03-27T11:08:13.964 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_loki", 2024-03-27T11:08:13.964 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.964 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.964 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.964 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-27T11:08:13.964 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.964 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.964 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.965 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Loki container image", 2024-03-27T11:08:13.965 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.965 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.965 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.965 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.965 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_node_exporter": { 2024-03-27T11:08:13.965 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_node_exporter", 2024-03-27T11:08:13.965 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.965 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.965 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.965 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-27T11:08:13.965 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.965 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.965 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.966 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Prometheus container image", 2024-03-27T11:08:13.966 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.966 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.966 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.966 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.966 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_nvmeof": { 2024-03-27T11:08:13.966 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_nvmeof", 2024-03-27T11:08:13.966 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.966 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.966 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.966 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-03-27T11:08:13.966 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.966 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.966 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.967 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Nvme-of container image", 2024-03-27T11:08:13.967 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.967 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.967 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.967 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.967 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_prometheus": { 2024-03-27T11:08:13.967 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_prometheus", 2024-03-27T11:08:13.967 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.967 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.967 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.967 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-27T11:08:13.967 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.967 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.967 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.967 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Prometheus container image", 2024-03-27T11:08:13.968 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.968 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.968 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.968 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.968 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_promtail": { 2024-03-27T11:08:13.968 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_promtail", 2024-03-27T11:08:13.968 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.968 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.968 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.968 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-27T11:08:13.968 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.968 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.968 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.968 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Promtail container image", 2024-03-27T11:08:13.969 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.969 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.969 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.969 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.969 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_samba": { 2024-03-27T11:08:13.969 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_samba", 2024-03-27T11:08:13.969 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.969 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.969 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.969 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-03-27T11:08:13.969 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.969 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.969 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.969 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Samba/SMB container image", 2024-03-27T11:08:13.970 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.970 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.970 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.970 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.970 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_snmp_gateway": { 2024-03-27T11:08:13.970 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_snmp_gateway", 2024-03-27T11:08:13.970 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.970 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.970 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.970 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-27T11:08:13.970 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.970 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.970 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.971 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "SNMP Gateway container image", 2024-03-27T11:08:13.971 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.971 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.971 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.971 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.971 INFO:teuthology.orchestra.run.smithi097.stdout: "container_init": { 2024-03-27T11:08:13.971 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_init", 2024-03-27T11:08:13.971 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.971 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.971 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.971 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:13.971 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.971 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.971 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.972 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Run podman/docker with `--init`", 2024-03-27T11:08:13.972 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.972 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.972 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.972 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.972 INFO:teuthology.orchestra.run.smithi097.stdout: "daemon_cache_timeout": { 2024-03-27T11:08:13.972 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "daemon_cache_timeout", 2024-03-27T11:08:13.972 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-27T11:08:13.972 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.972 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.972 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "600", 2024-03-27T11:08:13.972 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.972 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.972 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.973 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-27T11:08:13.973 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.973 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.973 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.973 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.973 INFO:teuthology.orchestra.run.smithi097.stdout: "default_cephadm_command_timeout": { 2024-03-27T11:08:13.973 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "default_cephadm_command_timeout", 2024-03-27T11:08:13.973 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:13.973 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.973 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.973 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "900", 2024-03-27T11:08:13.973 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.973 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.973 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.974 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-27T11:08:13.974 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.974 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.974 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.974 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.974 INFO:teuthology.orchestra.run.smithi097.stdout: "default_registry": { 2024-03-27T11:08:13.974 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "default_registry", 2024-03-27T11:08:13.974 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.974 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.974 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.974 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "docker.io", 2024-03-27T11:08:13.974 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.974 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.975 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.975 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-27T11:08:13.975 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.975 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.975 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.975 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.975 INFO:teuthology.orchestra.run.smithi097.stdout: "device_cache_timeout": { 2024-03-27T11:08:13.975 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "device_cache_timeout", 2024-03-27T11:08:13.975 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-27T11:08:13.975 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.975 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.975 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "1800", 2024-03-27T11:08:13.975 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.975 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.976 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.976 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "seconds to cache device inventory", 2024-03-27T11:08:13.976 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.976 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.976 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.976 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.976 INFO:teuthology.orchestra.run.smithi097.stdout: "device_enhanced_scan": { 2024-03-27T11:08:13.976 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "device_enhanced_scan", 2024-03-27T11:08:13.976 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.976 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.976 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.976 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.976 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.976 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.977 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.977 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-27T11:08:13.977 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.977 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.977 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.977 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.977 INFO:teuthology.orchestra.run.smithi097.stdout: "facts_cache_timeout": { 2024-03-27T11:08:13.977 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "facts_cache_timeout", 2024-03-27T11:08:13.977 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-27T11:08:13.977 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.977 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.977 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "60", 2024-03-27T11:08:13.977 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.977 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.978 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.978 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "seconds to cache host facts data", 2024-03-27T11:08:13.978 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.978 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.978 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.978 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.978 INFO:teuthology.orchestra.run.smithi097.stdout: "grafana_dashboards_path": { 2024-03-27T11:08:13.978 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "grafana_dashboards_path", 2024-03-27T11:08:13.978 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.978 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.978 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.978 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-03-27T11:08:13.978 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.979 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.979 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.979 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-03-27T11:08:13.979 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.979 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.979 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.979 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.979 INFO:teuthology.orchestra.run.smithi097.stdout: "host_check_interval": { 2024-03-27T11:08:13.979 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "host_check_interval", 2024-03-27T11:08:13.979 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-27T11:08:13.979 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.979 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.979 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "600", 2024-03-27T11:08:13.979 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.980 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.980 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.980 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "how frequently to perform a host check", 2024-03-27T11:08:13.980 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.980 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.980 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.980 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.980 INFO:teuthology.orchestra.run.smithi097.stdout: "hw_monitoring": { 2024-03-27T11:08:13.980 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "hw_monitoring", 2024-03-27T11:08:13.980 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.980 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.980 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.980 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.980 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.980 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.981 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.981 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-27T11:08:13.981 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.981 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.981 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.981 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.981 INFO:teuthology.orchestra.run.smithi097.stdout: "inventory_list_all": { 2024-03-27T11:08:13.981 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "inventory_list_all", 2024-03-27T11:08:13.981 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.981 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.981 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.981 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.981 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.981 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.982 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.982 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-27T11:08:13.982 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.982 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.982 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.982 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.982 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-27T11:08:13.982 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-27T11:08:13.982 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.982 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.982 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.982 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:13.982 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.982 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.983 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:13.983 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:13.983 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:13.983 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:13.983 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:13.983 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:13.983 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:13.983 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:13.983 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.983 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.983 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.983 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.983 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.984 INFO:teuthology.orchestra.run.smithi097.stdout: "log_refresh_metadata": { 2024-03-27T11:08:13.984 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_refresh_metadata", 2024-03-27T11:08:13.984 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.984 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.984 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.984 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.984 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.984 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.984 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.984 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-27T11:08:13.984 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.984 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.984 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.984 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.985 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-27T11:08:13.985 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-27T11:08:13.985 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.985 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.985 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.985 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:13.985 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.985 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.985 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.985 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-27T11:08:13.985 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.985 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.985 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.985 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.985 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-27T11:08:13.986 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-27T11:08:13.986 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.986 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.986 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.986 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-27T11:08:13.986 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.986 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.986 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:13.986 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:13.986 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:13.986 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:13.986 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:13.986 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:13.987 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:13.987 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:13.987 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.987 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.987 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.987 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.987 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.987 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-27T11:08:13.987 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-27T11:08:13.987 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.987 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.987 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:13.987 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.988 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.988 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.988 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.988 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:13.988 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.988 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.988 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.988 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.988 INFO:teuthology.orchestra.run.smithi097.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-27T11:08:13.988 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-27T11:08:13.988 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.988 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.988 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.988 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.989 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.989 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.989 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.989 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-27T11:08:13.989 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.989 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.989 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.989 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.989 INFO:teuthology.orchestra.run.smithi097.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-27T11:08:13.989 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-27T11:08:13.989 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.989 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.989 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.989 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "*", 2024-03-27T11:08:13.990 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.990 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.990 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.990 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-27T11:08:13.990 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.990 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.990 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.990 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.990 INFO:teuthology.orchestra.run.smithi097.stdout: "max_count_per_host": { 2024-03-27T11:08:13.990 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "max_count_per_host", 2024-03-27T11:08:13.990 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:13.990 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.990 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.990 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10", 2024-03-27T11:08:13.991 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.991 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.991 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.991 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "max number of daemons per service per host", 2024-03-27T11:08:13.991 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.991 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.991 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.991 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.991 INFO:teuthology.orchestra.run.smithi097.stdout: "max_osd_draining_count": { 2024-03-27T11:08:13.991 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "max_osd_draining_count", 2024-03-27T11:08:13.991 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:13.991 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.991 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.991 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10", 2024-03-27T11:08:13.991 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.992 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.992 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.992 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-27T11:08:13.992 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.992 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.992 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.992 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.992 INFO:teuthology.orchestra.run.smithi097.stdout: "migration_current": { 2024-03-27T11:08:13.992 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "migration_current", 2024-03-27T11:08:13.992 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:13.992 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.992 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.992 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:13.992 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.993 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.993 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.993 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "internal - do not modify", 2024-03-27T11:08:13.993 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.993 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.993 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.993 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.993 INFO:teuthology.orchestra.run.smithi097.stdout: "mode": { 2024-03-27T11:08:13.993 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "mode", 2024-03-27T11:08:13.993 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.993 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.993 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.993 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "root", 2024-03-27T11:08:13.993 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.993 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.994 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:13.994 INFO:teuthology.orchestra.run.smithi097.stdout: "cephadm-package", 2024-03-27T11:08:13.994 INFO:teuthology.orchestra.run.smithi097.stdout: "root" 2024-03-27T11:08:13.994 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:13.994 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "mode for remote execution of cephadm", 2024-03-27T11:08:13.994 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.994 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.994 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.994 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.994 INFO:teuthology.orchestra.run.smithi097.stdout: "oob_default_addr": { 2024-03-27T11:08:13.994 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "oob_default_addr", 2024-03-27T11:08:13.994 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.994 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.995 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.995 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "169.254.1.1", 2024-03-27T11:08:13.995 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.995 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.995 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.995 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-27T11:08:13.995 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.995 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.995 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.995 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.995 INFO:teuthology.orchestra.run.smithi097.stdout: "prometheus_alerts_path": { 2024-03-27T11:08:13.995 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "prometheus_alerts_path", 2024-03-27T11:08:13.995 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.995 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.996 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.996 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-27T11:08:13.996 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.996 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.996 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.996 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-27T11:08:13.996 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.996 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.996 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.996 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.996 INFO:teuthology.orchestra.run.smithi097.stdout: "registry_insecure": { 2024-03-27T11:08:13.996 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "registry_insecure", 2024-03-27T11:08:13.996 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:13.996 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.996 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.997 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:13.997 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.997 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.997 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.997 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-27T11:08:13.997 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.997 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.997 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.997 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.997 INFO:teuthology.orchestra.run.smithi097.stdout: "registry_password": { 2024-03-27T11:08:13.997 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "registry_password", 2024-03-27T11:08:13.997 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.997 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.997 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.998 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:13.998 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.998 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.998 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.998 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-27T11:08:13.998 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.998 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.998 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.998 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.998 INFO:teuthology.orchestra.run.smithi097.stdout: "registry_url": { 2024-03-27T11:08:13.998 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "registry_url", 2024-03-27T11:08:13.998 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.998 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.998 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:13.999 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:13.999 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:13.999 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:13.999 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:13.999 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-27T11:08:13.999 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:13.999 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:13.999 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:13.999 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:13.999 INFO:teuthology.orchestra.run.smithi097.stdout: "registry_username": { 2024-03-27T11:08:13.999 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "registry_username", 2024-03-27T11:08:13.999 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:13.999 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:13.999 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.000 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.000 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.000 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.000 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.000 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-27T11:08:14.000 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.000 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.000 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.000 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.000 INFO:teuthology.orchestra.run.smithi097.stdout: "secure_monitoring_stack": { 2024-03-27T11:08:14.000 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "secure_monitoring_stack", 2024-03-27T11:08:14.000 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.000 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.000 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.001 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.001 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.001 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.001 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.001 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-27T11:08:14.001 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.001 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.001 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.001 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.001 INFO:teuthology.orchestra.run.smithi097.stdout: "service_discovery_port": { 2024-03-27T11:08:14.001 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "service_discovery_port", 2024-03-27T11:08:14.001 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:14.001 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.001 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.002 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "8765", 2024-03-27T11:08:14.002 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.002 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.002 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.002 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "cephadm service discovery port", 2024-03-27T11:08:14.002 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.002 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.002 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.002 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.002 INFO:teuthology.orchestra.run.smithi097.stdout: "ssh_config_file": { 2024-03-27T11:08:14.002 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ssh_config_file", 2024-03-27T11:08:14.002 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.002 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.002 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.003 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.003 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.003 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.003 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.005 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-27T11:08:14.005 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.005 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.005 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.005 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.006 INFO:teuthology.orchestra.run.smithi097.stdout: "use_agent": { 2024-03-27T11:08:14.006 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "use_agent", 2024-03-27T11:08:14.006 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.006 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.006 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.006 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.006 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.006 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.006 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.006 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-27T11:08:14.006 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.006 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.006 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.007 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.007 INFO:teuthology.orchestra.run.smithi097.stdout: "use_repo_digest": { 2024-03-27T11:08:14.007 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "use_repo_digest", 2024-03-27T11:08:14.007 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.007 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.007 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.007 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:14.007 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.007 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.007 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.007 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-27T11:08:14.007 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.007 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.008 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.008 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.008 INFO:teuthology.orchestra.run.smithi097.stdout: "warn_on_failed_host_check": { 2024-03-27T11:08:14.008 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "warn_on_failed_host_check", 2024-03-27T11:08:14.008 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.008 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.008 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.008 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:14.008 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.008 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.008 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.008 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "raise a health warning if the host check fails", 2024-03-27T11:08:14.008 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.008 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.009 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.009 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.009 INFO:teuthology.orchestra.run.smithi097.stdout: "warn_on_stray_daemons": { 2024-03-27T11:08:14.009 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "warn_on_stray_daemons", 2024-03-27T11:08:14.009 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.009 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.009 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.009 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:14.009 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.009 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.009 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.009 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-27T11:08:14.009 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.009 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.010 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.010 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.010 INFO:teuthology.orchestra.run.smithi097.stdout: "warn_on_stray_hosts": { 2024-03-27T11:08:14.010 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "warn_on_stray_hosts", 2024-03-27T11:08:14.010 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.010 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.010 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.010 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:14.010 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.010 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.010 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.010 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-27T11:08:14.011 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.011 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.011 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.011 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.011 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.011 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.011 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:14.011 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "crash", 2024-03-27T11:08:14.011 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-27T11:08:14.011 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-27T11:08:14.011 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-27T11:08:14.011 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-27T11:08:14.011 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-27T11:08:14.011 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.012 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.012 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.012 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.012 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.012 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.012 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.012 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.012 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.012 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.012 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.012 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.012 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.012 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.012 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.013 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.013 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.013 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.013 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.013 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-27T11:08:14.013 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-27T11:08:14.013 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.013 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.013 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.013 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.013 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.013 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.013 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.013 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.014 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.014 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.014 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.014 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.014 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-27T11:08:14.014 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-27T11:08:14.014 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.014 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.014 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.014 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-27T11:08:14.014 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.014 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.014 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.015 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.015 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.015 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.015 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.015 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.015 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.015 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.015 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.015 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.015 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.015 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.015 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.015 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-27T11:08:14.016 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-27T11:08:14.016 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.016 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.016 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.016 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.016 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.016 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.016 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.016 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.016 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.016 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.016 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.016 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.016 INFO:teuthology.orchestra.run.smithi097.stdout: "retain_interval": { 2024-03-27T11:08:14.017 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "retain_interval", 2024-03-27T11:08:14.017 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-27T11:08:14.017 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.017 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.017 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "31536000", 2024-03-27T11:08:14.017 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.017 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.017 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.017 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-27T11:08:14.017 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.017 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.017 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.017 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.017 INFO:teuthology.orchestra.run.smithi097.stdout: "warn_recent_interval": { 2024-03-27T11:08:14.018 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "warn_recent_interval", 2024-03-27T11:08:14.018 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-27T11:08:14.018 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.018 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.018 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "1209600", 2024-03-27T11:08:14.018 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.018 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.018 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.018 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-27T11:08:14.018 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.018 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.018 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.018 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.019 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.019 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.019 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:14.019 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "dashboard", 2024-03-27T11:08:14.019 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-27T11:08:14.019 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-27T11:08:14.019 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-27T11:08:14.019 INFO:teuthology.orchestra.run.smithi097.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-27T11:08:14.019 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-27T11:08:14.019 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:14.019 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.019 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.019 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10", 2024-03-27T11:08:14.019 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.020 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.020 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.020 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.020 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.020 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.020 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.020 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.020 INFO:teuthology.orchestra.run.smithi097.stdout: "ALERTMANAGER_API_HOST": { 2024-03-27T11:08:14.020 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-27T11:08:14.020 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.020 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.020 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.020 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.020 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.021 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.021 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.021 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.021 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.021 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.021 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.021 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.021 INFO:teuthology.orchestra.run.smithi097.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-27T11:08:14.021 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-27T11:08:14.021 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.021 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.021 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.021 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:14.021 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.022 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.022 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.022 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.022 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.022 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.022 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.022 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.022 INFO:teuthology.orchestra.run.smithi097.stdout: "AUDIT_API_ENABLED": { 2024-03-27T11:08:14.022 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "AUDIT_API_ENABLED", 2024-03-27T11:08:14.022 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.022 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.022 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.022 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.022 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.023 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.023 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.023 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.023 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.023 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.023 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.023 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.023 INFO:teuthology.orchestra.run.smithi097.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-27T11:08:14.023 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-27T11:08:14.023 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.023 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.023 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.023 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:14.023 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.023 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.024 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.024 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.024 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.024 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.024 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.024 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.024 INFO:teuthology.orchestra.run.smithi097.stdout: "ENABLE_BROWSABLE_API": { 2024-03-27T11:08:14.024 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-27T11:08:14.024 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.024 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.024 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.024 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:14.024 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.025 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.025 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.025 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.025 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.025 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.025 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.025 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.025 INFO:teuthology.orchestra.run.smithi097.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-27T11:08:14.025 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-27T11:08:14.025 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.025 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.025 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.025 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:14.026 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.026 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.026 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.026 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.026 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.026 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.026 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.026 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.026 INFO:teuthology.orchestra.run.smithi097.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-27T11:08:14.026 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-27T11:08:14.026 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.026 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.026 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.027 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:14.027 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.027 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.027 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.027 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.027 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.027 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.027 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.027 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.027 INFO:teuthology.orchestra.run.smithi097.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-27T11:08:14.027 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-27T11:08:14.027 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.027 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.028 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.028 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:14.028 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.028 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.028 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.028 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.028 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.028 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.028 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.028 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.028 INFO:teuthology.orchestra.run.smithi097.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-27T11:08:14.028 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-27T11:08:14.028 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.029 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.029 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.029 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:14.029 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.029 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.029 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.029 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.029 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.029 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.029 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.029 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.029 INFO:teuthology.orchestra.run.smithi097.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-27T11:08:14.029 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-27T11:08:14.029 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.029 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.030 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.030 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:14.030 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.030 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.030 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.030 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.030 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.030 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.030 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.030 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.030 INFO:teuthology.orchestra.run.smithi097.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-27T11:08:14.030 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-27T11:08:14.030 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.030 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.031 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.031 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:14.031 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.031 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.031 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.031 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.031 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.031 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.031 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.031 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.031 INFO:teuthology.orchestra.run.smithi097.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-27T11:08:14.031 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-27T11:08:14.031 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.031 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.032 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.032 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:14.032 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.032 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.032 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.032 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.032 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.032 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.032 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.032 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.032 INFO:teuthology.orchestra.run.smithi097.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-27T11:08:14.032 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-27T11:08:14.032 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.032 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.033 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.033 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.033 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.033 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.033 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.033 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.033 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.033 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.033 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.033 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.033 INFO:teuthology.orchestra.run.smithi097.stdout: "GRAFANA_API_PASSWORD": { 2024-03-27T11:08:14.033 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-27T11:08:14.033 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.033 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.034 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.034 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "admin", 2024-03-27T11:08:14.034 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.034 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.034 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.034 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.034 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.034 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.034 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.034 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.034 INFO:teuthology.orchestra.run.smithi097.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-27T11:08:14.034 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-27T11:08:14.034 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.035 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.035 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.035 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:14.035 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.035 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.035 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.035 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.035 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.035 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.035 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.035 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.035 INFO:teuthology.orchestra.run.smithi097.stdout: "GRAFANA_API_URL": { 2024-03-27T11:08:14.035 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "GRAFANA_API_URL", 2024-03-27T11:08:14.035 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.036 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.036 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.036 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.036 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.036 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.036 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.036 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.036 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.036 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.036 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.036 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.036 INFO:teuthology.orchestra.run.smithi097.stdout: "GRAFANA_API_USERNAME": { 2024-03-27T11:08:14.036 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-27T11:08:14.036 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.037 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.037 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.037 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "admin", 2024-03-27T11:08:14.037 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.037 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.037 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.037 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.037 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.037 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.037 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.037 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.037 INFO:teuthology.orchestra.run.smithi097.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-27T11:08:14.037 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-27T11:08:14.038 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.038 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.038 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.038 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.038 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.038 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.038 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.038 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.038 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.038 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.038 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.038 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.038 INFO:teuthology.orchestra.run.smithi097.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-27T11:08:14.038 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-27T11:08:14.039 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.039 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.039 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.039 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.039 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.039 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.039 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.039 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.039 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.039 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.039 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.039 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.039 INFO:teuthology.orchestra.run.smithi097.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-27T11:08:14.039 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-27T11:08:14.040 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.040 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.040 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.040 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:14.040 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.040 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.040 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.040 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.040 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.040 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.040 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.040 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.040 INFO:teuthology.orchestra.run.smithi097.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-27T11:08:14.040 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-27T11:08:14.041 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.041 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.041 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.041 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.041 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.041 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.041 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.041 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.041 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.041 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.041 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.041 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.041 INFO:teuthology.orchestra.run.smithi097.stdout: "MULTICLUSTER_CONFIG": { 2024-03-27T11:08:14.041 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "MULTICLUSTER_CONFIG", 2024-03-27T11:08:14.042 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.042 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.042 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.042 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "{}", 2024-03-27T11:08:14.042 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.042 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.042 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.042 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.042 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.042 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.042 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.042 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.042 INFO:teuthology.orchestra.run.smithi097.stdout: "PROMETHEUS_API_HOST": { 2024-03-27T11:08:14.043 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-27T11:08:14.043 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.043 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.043 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.043 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.043 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.043 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.043 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.043 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.043 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.043 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.043 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.043 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.043 INFO:teuthology.orchestra.run.smithi097.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-27T11:08:14.044 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-27T11:08:14.044 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.044 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.044 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.044 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:14.044 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.044 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.044 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.044 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.044 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.044 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.044 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.044 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.044 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-27T11:08:14.044 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-27T11:08:14.045 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.045 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.045 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.045 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.045 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.045 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.045 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.045 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.045 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.045 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.045 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.045 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.045 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-27T11:08:14.046 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-27T11:08:14.046 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.046 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.046 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.046 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.046 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.046 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.046 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.046 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.046 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.046 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.046 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.046 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.046 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-27T11:08:14.047 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-27T11:08:14.047 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.047 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.047 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.047 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:14.047 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.047 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.047 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.047 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.047 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.047 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.047 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.047 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.047 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-27T11:08:14.048 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-27T11:08:14.048 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.048 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.048 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.048 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:14.048 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.048 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.048 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.048 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.048 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.048 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.048 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.048 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.048 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-27T11:08:14.048 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-27T11:08:14.049 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.049 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.049 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.049 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.049 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.049 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.049 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.049 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.049 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.049 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.049 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.049 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.049 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-27T11:08:14.049 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-27T11:08:14.049 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.050 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.050 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.050 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.050 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.050 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.050 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.050 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.050 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.050 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.050 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.050 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.050 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-27T11:08:14.051 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-27T11:08:14.051 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.051 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.051 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.051 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.051 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.051 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.051 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.051 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.051 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.051 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.051 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.051 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.052 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_ENABLED": { 2024-03-27T11:08:14.052 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-27T11:08:14.052 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.052 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.052 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.052 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:14.052 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.052 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.052 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.052 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.052 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.052 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.052 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.052 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.053 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-27T11:08:14.053 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-27T11:08:14.053 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.053 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.053 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.053 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-27T11:08:14.053 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.053 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.053 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.053 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.053 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.053 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.053 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.053 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.054 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-27T11:08:14.054 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-27T11:08:14.054 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:14.054 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.054 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.054 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10", 2024-03-27T11:08:14.054 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.054 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.054 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.054 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.054 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.054 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.054 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.055 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.055 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-27T11:08:14.055 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-27T11:08:14.055 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:14.055 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.055 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.055 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "8", 2024-03-27T11:08:14.055 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.055 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.055 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.055 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.055 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.055 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.055 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.056 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.056 INFO:teuthology.orchestra.run.smithi097.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-27T11:08:14.056 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-27T11:08:14.056 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:14.056 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.056 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.056 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "45", 2024-03-27T11:08:14.056 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.056 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.056 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.056 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.056 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.056 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.057 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.057 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.057 INFO:teuthology.orchestra.run.smithi097.stdout: "RGW_API_ACCESS_KEY": { 2024-03-27T11:08:14.057 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-27T11:08:14.057 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.057 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.057 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.057 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.057 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.057 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.057 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.057 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.057 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.057 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.058 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.058 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.058 INFO:teuthology.orchestra.run.smithi097.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-27T11:08:14.058 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-27T11:08:14.058 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.058 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.058 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.058 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "admin", 2024-03-27T11:08:14.058 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.058 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.058 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.058 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.058 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.059 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.059 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.059 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.059 INFO:teuthology.orchestra.run.smithi097.stdout: "RGW_API_SECRET_KEY": { 2024-03-27T11:08:14.059 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-27T11:08:14.059 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.059 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.059 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.059 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.059 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.059 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.059 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.059 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.059 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.060 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.060 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.060 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.060 INFO:teuthology.orchestra.run.smithi097.stdout: "RGW_API_SSL_VERIFY": { 2024-03-27T11:08:14.060 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-27T11:08:14.060 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.060 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.060 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.060 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:14.060 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.060 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.060 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.060 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.060 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.061 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.061 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.061 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.061 INFO:teuthology.orchestra.run.smithi097.stdout: "UNSAFE_TLS_v1_2": { 2024-03-27T11:08:14.061 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "UNSAFE_TLS_v1_2", 2024-03-27T11:08:14.061 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.061 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.061 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.061 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.061 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.061 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.061 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.061 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.061 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.062 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.062 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.062 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.062 INFO:teuthology.orchestra.run.smithi097.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-27T11:08:14.062 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-27T11:08:14.062 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:14.062 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.062 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.062 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "0", 2024-03-27T11:08:14.062 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.062 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.062 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.062 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.062 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.063 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.063 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.063 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.063 INFO:teuthology.orchestra.run.smithi097.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-27T11:08:14.063 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-27T11:08:14.063 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:14.063 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.063 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.063 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10", 2024-03-27T11:08:14.063 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.063 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.063 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.063 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.063 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.064 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.064 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.064 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.064 INFO:teuthology.orchestra.run.smithi097.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-27T11:08:14.064 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-27T11:08:14.064 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:14.064 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.064 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.064 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "5", 2024-03-27T11:08:14.064 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.064 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.064 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.064 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.065 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.065 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.065 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.065 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.065 INFO:teuthology.orchestra.run.smithi097.stdout: "cross_origin_url": { 2024-03-27T11:08:14.065 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "cross_origin_url", 2024-03-27T11:08:14.065 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.065 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.065 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.065 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.065 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.065 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.065 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.065 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.065 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.066 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.066 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.066 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.066 INFO:teuthology.orchestra.run.smithi097.stdout: "crt_file": { 2024-03-27T11:08:14.066 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "crt_file", 2024-03-27T11:08:14.066 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.066 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.066 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.066 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.066 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.066 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.066 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.066 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.067 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.067 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.067 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.067 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.067 INFO:teuthology.orchestra.run.smithi097.stdout: "debug": { 2024-03-27T11:08:14.067 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "debug", 2024-03-27T11:08:14.067 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.067 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.067 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.067 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.067 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.067 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.067 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.067 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Enable/disable debug options", 2024-03-27T11:08:14.068 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.068 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.068 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.068 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.068 INFO:teuthology.orchestra.run.smithi097.stdout: "jwt_token_ttl": { 2024-03-27T11:08:14.068 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "jwt_token_ttl", 2024-03-27T11:08:14.068 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:14.068 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.068 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.068 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "28800", 2024-03-27T11:08:14.068 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.068 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.068 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.068 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.069 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.069 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.069 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.069 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.069 INFO:teuthology.orchestra.run.smithi097.stdout: "key_file": { 2024-03-27T11:08:14.069 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "key_file", 2024-03-27T11:08:14.069 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.069 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.069 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.069 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.069 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.069 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.069 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.069 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.070 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.070 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.070 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.070 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.070 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-27T11:08:14.070 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-27T11:08:14.070 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.070 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.070 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.070 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.070 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.070 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.070 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.070 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.071 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.071 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.071 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.071 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.071 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.071 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.071 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.071 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.071 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.071 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.071 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.071 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-27T11:08:14.071 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-27T11:08:14.071 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.071 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.072 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.072 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.072 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.072 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.072 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.072 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.072 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.072 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.072 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.072 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.072 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-27T11:08:14.072 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-27T11:08:14.072 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.072 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.072 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.073 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-27T11:08:14.073 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.073 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.073 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.073 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.073 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.073 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.073 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.073 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.073 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.073 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.073 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.073 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.073 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.074 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.074 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.074 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-27T11:08:14.074 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-27T11:08:14.074 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.074 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.074 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.074 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.074 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.074 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.074 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.074 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.074 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.074 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.075 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.075 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.075 INFO:teuthology.orchestra.run.smithi097.stdout: "motd": { 2024-03-27T11:08:14.075 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "motd", 2024-03-27T11:08:14.075 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.075 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.075 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.075 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.075 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.075 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.075 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.075 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "The message of the day", 2024-03-27T11:08:14.075 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.075 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.076 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.076 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.076 INFO:teuthology.orchestra.run.smithi097.stdout: "redirect_resolve_ip_addr": { 2024-03-27T11:08:14.076 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "redirect_resolve_ip_addr", 2024-03-27T11:08:14.076 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.076 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.076 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.076 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.076 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.076 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.076 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.076 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.076 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.076 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.076 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.076 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.077 INFO:teuthology.orchestra.run.smithi097.stdout: "server_addr": { 2024-03-27T11:08:14.077 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "server_addr", 2024-03-27T11:08:14.077 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.077 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.077 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.077 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "::", 2024-03-27T11:08:14.077 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.077 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.077 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.077 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.077 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.077 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.077 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.077 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.077 INFO:teuthology.orchestra.run.smithi097.stdout: "server_port": { 2024-03-27T11:08:14.078 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "server_port", 2024-03-27T11:08:14.078 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:14.078 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.078 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.078 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "8080", 2024-03-27T11:08:14.078 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.078 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.078 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.078 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.078 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.078 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.078 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.078 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.079 INFO:teuthology.orchestra.run.smithi097.stdout: "ssl": { 2024-03-27T11:08:14.079 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ssl", 2024-03-27T11:08:14.079 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.079 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.079 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.079 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:14.079 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.079 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.079 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.079 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.079 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.079 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.079 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.079 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.080 INFO:teuthology.orchestra.run.smithi097.stdout: "ssl_server_port": { 2024-03-27T11:08:14.080 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ssl_server_port", 2024-03-27T11:08:14.080 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:14.080 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.080 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.080 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "8443", 2024-03-27T11:08:14.080 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.080 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.080 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.080 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.080 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.080 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.080 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.080 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.080 INFO:teuthology.orchestra.run.smithi097.stdout: "standby_behaviour": { 2024-03-27T11:08:14.081 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "standby_behaviour", 2024-03-27T11:08:14.081 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.081 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.081 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.081 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "redirect", 2024-03-27T11:08:14.081 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.081 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.081 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.081 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.081 INFO:teuthology.orchestra.run.smithi097.stdout: "redirect" 2024-03-27T11:08:14.081 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.081 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.081 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.081 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.082 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.082 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.082 INFO:teuthology.orchestra.run.smithi097.stdout: "standby_error_status_code": { 2024-03-27T11:08:14.082 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "standby_error_status_code", 2024-03-27T11:08:14.082 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:14.082 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.082 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.082 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "500", 2024-03-27T11:08:14.082 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "400", 2024-03-27T11:08:14.082 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "599", 2024-03-27T11:08:14.082 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.082 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.082 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.083 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.083 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.083 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.083 INFO:teuthology.orchestra.run.smithi097.stdout: "url_prefix": { 2024-03-27T11:08:14.083 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "url_prefix", 2024-03-27T11:08:14.083 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.083 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.083 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.083 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.083 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.083 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.083 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.083 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.083 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.083 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.084 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.084 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.084 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.084 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.084 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:14.084 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "devicehealth", 2024-03-27T11:08:14.084 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-27T11:08:14.084 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-27T11:08:14.084 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-27T11:08:14.084 INFO:teuthology.orchestra.run.smithi097.stdout: "enable_monitoring": { 2024-03-27T11:08:14.084 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "enable_monitoring", 2024-03-27T11:08:14.084 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.084 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.084 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.084 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:14.085 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.085 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.085 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.085 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "monitor device health metrics", 2024-03-27T11:08:14.085 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.085 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.085 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.085 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.085 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-27T11:08:14.085 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-27T11:08:14.085 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.085 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.085 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.085 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.085 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.085 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.086 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.086 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.086 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.086 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.086 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.086 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.086 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.086 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.086 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.086 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.086 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.086 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.086 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.087 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-27T11:08:14.087 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-27T11:08:14.087 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.087 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.087 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.087 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.087 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.087 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.087 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.087 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.087 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.087 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.088 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.088 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.088 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-27T11:08:14.088 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-27T11:08:14.088 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.088 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.088 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.088 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-27T11:08:14.088 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.088 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.088 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.089 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.089 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.089 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.089 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.089 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.089 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.089 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.089 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.089 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.089 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.089 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.089 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.089 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-27T11:08:14.089 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-27T11:08:14.090 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.090 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.090 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.090 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.090 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.090 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.090 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.090 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.090 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.090 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.090 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.090 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.090 INFO:teuthology.orchestra.run.smithi097.stdout: "mark_out_threshold": { 2024-03-27T11:08:14.090 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "mark_out_threshold", 2024-03-27T11:08:14.091 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-27T11:08:14.091 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.091 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.091 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "2419200", 2024-03-27T11:08:14.091 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.091 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.091 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.091 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-27T11:08:14.091 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.091 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.091 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.091 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.091 INFO:teuthology.orchestra.run.smithi097.stdout: "pool_name": { 2024-03-27T11:08:14.091 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "pool_name", 2024-03-27T11:08:14.092 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.092 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.092 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.092 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "device_health_metrics", 2024-03-27T11:08:14.092 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.092 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.092 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.092 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-27T11:08:14.092 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.092 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.092 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.092 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.092 INFO:teuthology.orchestra.run.smithi097.stdout: "retention_period": { 2024-03-27T11:08:14.092 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "retention_period", 2024-03-27T11:08:14.093 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-27T11:08:14.093 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.093 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.093 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "15552000", 2024-03-27T11:08:14.093 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.093 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.093 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.093 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "how long to retain device health metrics", 2024-03-27T11:08:14.093 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.093 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.093 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.093 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.093 INFO:teuthology.orchestra.run.smithi097.stdout: "scrape_frequency": { 2024-03-27T11:08:14.093 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "scrape_frequency", 2024-03-27T11:08:14.093 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-27T11:08:14.094 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.094 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.094 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "86400", 2024-03-27T11:08:14.094 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.094 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.094 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.094 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-27T11:08:14.094 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.094 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.094 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.094 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.094 INFO:teuthology.orchestra.run.smithi097.stdout: "self_heal": { 2024-03-27T11:08:14.094 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "self_heal", 2024-03-27T11:08:14.094 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.095 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.095 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.095 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:14.095 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.095 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.095 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.095 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-27T11:08:14.095 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.095 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.095 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.095 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.095 INFO:teuthology.orchestra.run.smithi097.stdout: "sleep_interval": { 2024-03-27T11:08:14.095 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "sleep_interval", 2024-03-27T11:08:14.095 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-27T11:08:14.096 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.096 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.096 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "600", 2024-03-27T11:08:14.096 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.096 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.096 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.096 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "how frequently to wake up and check device health", 2024-03-27T11:08:14.096 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.096 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.096 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.096 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.096 INFO:teuthology.orchestra.run.smithi097.stdout: "warn_threshold": { 2024-03-27T11:08:14.096 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "warn_threshold", 2024-03-27T11:08:14.096 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-27T11:08:14.096 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.097 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.097 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "7257600", 2024-03-27T11:08:14.097 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.097 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.097 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.097 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-27T11:08:14.097 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.097 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.097 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.097 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.097 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.097 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.097 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:14.098 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "diskprediction_local", 2024-03-27T11:08:14.098 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-27T11:08:14.098 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-27T11:08:14.098 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-27T11:08:14.098 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-27T11:08:14.098 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-27T11:08:14.098 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.098 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.098 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.098 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.098 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.098 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.098 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.098 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.099 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.099 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.099 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.099 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.099 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.099 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.099 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.099 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.099 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.099 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.099 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.099 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-27T11:08:14.099 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-27T11:08:14.099 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.099 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.099 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.100 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.100 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.100 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.100 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.100 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.100 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.100 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.100 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.100 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.100 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-27T11:08:14.100 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-27T11:08:14.100 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.100 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.100 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.101 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-27T11:08:14.101 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.101 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.101 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.101 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.101 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.101 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.101 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.101 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.101 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.101 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.101 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.101 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.101 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.101 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.102 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.102 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-27T11:08:14.102 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-27T11:08:14.102 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.102 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.102 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.102 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.102 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.102 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.102 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.102 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.102 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.102 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.102 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.103 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.103 INFO:teuthology.orchestra.run.smithi097.stdout: "predict_interval": { 2024-03-27T11:08:14.103 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "predict_interval", 2024-03-27T11:08:14.103 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.103 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.103 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.103 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "86400", 2024-03-27T11:08:14.103 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.103 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.103 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.103 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.103 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.103 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.103 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.103 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.104 INFO:teuthology.orchestra.run.smithi097.stdout: "predictor_model": { 2024-03-27T11:08:14.104 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "predictor_model", 2024-03-27T11:08:14.104 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.104 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.104 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.104 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "prophetstor", 2024-03-27T11:08:14.104 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.104 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.104 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.104 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.104 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.104 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.104 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.104 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.105 INFO:teuthology.orchestra.run.smithi097.stdout: "sleep_interval": { 2024-03-27T11:08:14.105 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "sleep_interval", 2024-03-27T11:08:14.105 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.105 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.105 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.105 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "600", 2024-03-27T11:08:14.105 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.105 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.105 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.105 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.105 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.105 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.105 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.105 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.105 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.106 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.106 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:14.106 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "influx", 2024-03-27T11:08:14.106 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": false, 2024-03-27T11:08:14.106 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "influxdb python module not found", 2024-03-27T11:08:14.106 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-27T11:08:14.106 INFO:teuthology.orchestra.run.smithi097.stdout: "batch_size": { 2024-03-27T11:08:14.106 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "batch_size", 2024-03-27T11:08:14.106 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:14.106 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.106 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.106 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "5000", 2024-03-27T11:08:14.106 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.106 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.106 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.107 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-27T11:08:14.107 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.107 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.107 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.107 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.107 INFO:teuthology.orchestra.run.smithi097.stdout: "database": { 2024-03-27T11:08:14.107 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "database", 2024-03-27T11:08:14.107 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.107 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.107 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.107 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "ceph", 2024-03-27T11:08:14.107 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.107 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.107 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.108 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-27T11:08:14.108 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.108 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.108 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.108 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.108 INFO:teuthology.orchestra.run.smithi097.stdout: "hostname": { 2024-03-27T11:08:14.108 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "hostname", 2024-03-27T11:08:14.108 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.108 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.108 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.108 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.108 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.108 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.109 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.109 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "InfluxDB server hostname", 2024-03-27T11:08:14.109 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.109 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.109 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.109 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.109 INFO:teuthology.orchestra.run.smithi097.stdout: "interval": { 2024-03-27T11:08:14.109 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "interval", 2024-03-27T11:08:14.109 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-27T11:08:14.109 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.109 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.109 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "30", 2024-03-27T11:08:14.109 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "5", 2024-03-27T11:08:14.109 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.109 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.110 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-27T11:08:14.110 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.110 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.110 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.110 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.110 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-27T11:08:14.110 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-27T11:08:14.110 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.110 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.110 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.110 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.110 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.111 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.111 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.111 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.111 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.111 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.111 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.111 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.111 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.111 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.111 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.111 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.111 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.111 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.111 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.111 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-27T11:08:14.112 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-27T11:08:14.112 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.112 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.112 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.112 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.112 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.112 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.112 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.112 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.112 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.112 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.112 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.112 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.112 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-27T11:08:14.112 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-27T11:08:14.113 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.113 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.113 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.113 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-27T11:08:14.113 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.113 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.113 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.113 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.113 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.113 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.113 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.113 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.113 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.113 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.113 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.113 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.114 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.114 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.114 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.114 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-27T11:08:14.114 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-27T11:08:14.114 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.114 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.114 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.114 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.114 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.114 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.114 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.114 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.114 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.114 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.115 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.115 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.115 INFO:teuthology.orchestra.run.smithi097.stdout: "password": { 2024-03-27T11:08:14.115 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "password", 2024-03-27T11:08:14.115 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.115 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.115 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.115 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.115 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.115 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.115 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.115 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "password of InfluxDB server user", 2024-03-27T11:08:14.115 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.115 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.115 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.116 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.116 INFO:teuthology.orchestra.run.smithi097.stdout: "port": { 2024-03-27T11:08:14.116 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "port", 2024-03-27T11:08:14.116 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:14.116 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.116 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.116 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "8086", 2024-03-27T11:08:14.116 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.116 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.116 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.116 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "InfluxDB server port", 2024-03-27T11:08:14.116 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.116 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.116 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.116 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.116 INFO:teuthology.orchestra.run.smithi097.stdout: "ssl": { 2024-03-27T11:08:14.117 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ssl", 2024-03-27T11:08:14.117 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.117 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.117 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.117 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "false", 2024-03-27T11:08:14.117 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.117 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.117 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.117 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-27T11:08:14.117 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.117 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.117 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.117 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.117 INFO:teuthology.orchestra.run.smithi097.stdout: "threads": { 2024-03-27T11:08:14.117 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "threads", 2024-03-27T11:08:14.118 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:14.118 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.118 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.118 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "5", 2024-03-27T11:08:14.118 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "1", 2024-03-27T11:08:14.118 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "32", 2024-03-27T11:08:14.118 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.118 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-27T11:08:14.118 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.118 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.118 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.118 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.118 INFO:teuthology.orchestra.run.smithi097.stdout: "username": { 2024-03-27T11:08:14.118 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "username", 2024-03-27T11:08:14.119 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.119 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.119 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.119 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.119 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.119 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.119 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.119 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "username of InfluxDB server user", 2024-03-27T11:08:14.119 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.119 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.119 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.119 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.119 INFO:teuthology.orchestra.run.smithi097.stdout: "verify_ssl": { 2024-03-27T11:08:14.119 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "verify_ssl", 2024-03-27T11:08:14.119 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.120 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.120 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.120 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "true", 2024-03-27T11:08:14.120 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.120 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.120 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.120 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-27T11:08:14.120 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.120 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.120 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.120 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.120 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.120 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.120 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:14.120 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "insights", 2024-03-27T11:08:14.121 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-27T11:08:14.121 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-27T11:08:14.121 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-27T11:08:14.121 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-27T11:08:14.121 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-27T11:08:14.121 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.121 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.121 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.121 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.121 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.121 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.121 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.121 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.121 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.121 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.122 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.122 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.122 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.122 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.122 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.122 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.122 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.122 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.122 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.122 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-27T11:08:14.122 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-27T11:08:14.122 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.122 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.122 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.122 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.123 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.123 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.123 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.123 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.123 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.123 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.123 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.123 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.123 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-27T11:08:14.123 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-27T11:08:14.123 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.123 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.123 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.123 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-27T11:08:14.123 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.124 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.124 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.124 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.124 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.124 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.124 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.124 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.124 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.124 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.124 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.124 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.124 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.124 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.124 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.124 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-27T11:08:14.125 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-27T11:08:14.125 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.125 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.125 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.125 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.125 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.125 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.125 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.125 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.125 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.125 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.125 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.125 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.125 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.126 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.126 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:14.126 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "iostat", 2024-03-27T11:08:14.126 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-27T11:08:14.126 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-27T11:08:14.126 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-27T11:08:14.126 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-27T11:08:14.126 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-27T11:08:14.126 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.126 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.126 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.126 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.126 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.126 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.127 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.127 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.127 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.127 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.127 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.127 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.127 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.127 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.127 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.127 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.127 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.127 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.127 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.127 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-27T11:08:14.127 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-27T11:08:14.128 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.128 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.128 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.128 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.128 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.128 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.128 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.128 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.128 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.128 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.128 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.128 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.128 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-27T11:08:14.128 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-27T11:08:14.128 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.129 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.129 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.129 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-27T11:08:14.129 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.129 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.129 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.129 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.129 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.129 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.129 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.129 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.129 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.129 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.129 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.129 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.130 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.130 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.130 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.130 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-27T11:08:14.130 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-27T11:08:14.130 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.130 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.130 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.130 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.130 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.130 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.130 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.131 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.131 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.131 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.131 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.131 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.131 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.131 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.131 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:14.131 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "k8sevents", 2024-03-27T11:08:14.131 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-27T11:08:14.131 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-27T11:08:14.131 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-27T11:08:14.131 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_event_retention_days": { 2024-03-27T11:08:14.132 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ceph_event_retention_days", 2024-03-27T11:08:14.132 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:14.132 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.132 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.132 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "7", 2024-03-27T11:08:14.132 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.132 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.132 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.132 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-27T11:08:14.132 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.132 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.132 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.132 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.132 INFO:teuthology.orchestra.run.smithi097.stdout: "config_check_secs": { 2024-03-27T11:08:14.133 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "config_check_secs", 2024-03-27T11:08:14.133 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:14.133 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.133 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.133 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10", 2024-03-27T11:08:14.133 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "10", 2024-03-27T11:08:14.133 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.133 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.133 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-27T11:08:14.133 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.133 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.133 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.133 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.134 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-27T11:08:14.134 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-27T11:08:14.134 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.134 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.134 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.134 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.134 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.134 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.134 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.134 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.134 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.134 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.135 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.135 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.135 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.135 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.135 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.135 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.135 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.135 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.135 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.135 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-27T11:08:14.135 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-27T11:08:14.135 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.135 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.136 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.136 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.136 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.136 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.136 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.136 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.136 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.136 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.136 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.136 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.136 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-27T11:08:14.136 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-27T11:08:14.136 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.137 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.137 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.137 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-27T11:08:14.137 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.137 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.137 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.137 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.137 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.137 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.137 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.137 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.137 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.137 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.138 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.138 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.138 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.138 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.138 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.138 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-27T11:08:14.138 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-27T11:08:14.138 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.138 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.138 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.138 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.138 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.138 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.139 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.139 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.139 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.139 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.139 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.139 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.139 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.139 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.139 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:14.139 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "localpool", 2024-03-27T11:08:14.139 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-27T11:08:14.139 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-27T11:08:14.139 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-27T11:08:14.139 INFO:teuthology.orchestra.run.smithi097.stdout: "failure_domain": { 2024-03-27T11:08:14.140 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "failure_domain", 2024-03-27T11:08:14.140 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.140 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.140 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.140 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "host", 2024-03-27T11:08:14.140 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.140 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.140 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.140 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "failure domain for any created local pool", 2024-03-27T11:08:14.140 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-27T11:08:14.140 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.140 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.140 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.140 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-27T11:08:14.140 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-27T11:08:14.141 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.141 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.141 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.141 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.141 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.141 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.141 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.141 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.141 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.141 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.141 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.141 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.141 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.141 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.141 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.142 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.142 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.142 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.142 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.142 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-27T11:08:14.142 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-27T11:08:14.142 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.142 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.142 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.142 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.142 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.142 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.142 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.142 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.142 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.143 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.143 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.143 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.143 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-27T11:08:14.143 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-27T11:08:14.143 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.143 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.143 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.143 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-27T11:08:14.143 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.143 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.143 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.143 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.143 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.143 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.143 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.144 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.144 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.144 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.144 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.144 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.144 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.144 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.144 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.144 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-27T11:08:14.144 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-27T11:08:14.144 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.145 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.145 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.145 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.145 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.145 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.145 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.145 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.145 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.145 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.145 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.145 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.145 INFO:teuthology.orchestra.run.smithi097.stdout: "min_size": { 2024-03-27T11:08:14.146 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "min_size", 2024-03-27T11:08:14.146 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:14.146 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.146 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.146 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.146 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.146 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.146 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.146 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "default min_size for any created local pool", 2024-03-27T11:08:14.146 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-27T11:08:14.146 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.146 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.146 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.146 INFO:teuthology.orchestra.run.smithi097.stdout: "num_rep": { 2024-03-27T11:08:14.147 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "num_rep", 2024-03-27T11:08:14.147 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:14.147 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.147 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.147 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "3", 2024-03-27T11:08:14.147 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.147 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.147 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.147 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "default replica count for any created local pool", 2024-03-27T11:08:14.147 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.147 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.147 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.147 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.147 INFO:teuthology.orchestra.run.smithi097.stdout: "pg_num": { 2024-03-27T11:08:14.148 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "pg_num", 2024-03-27T11:08:14.148 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:14.148 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.148 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.148 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "128", 2024-03-27T11:08:14.148 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.148 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.148 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.148 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "default pg_num for any created local pool", 2024-03-27T11:08:14.148 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.148 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.148 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.148 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.148 INFO:teuthology.orchestra.run.smithi097.stdout: "prefix": { 2024-03-27T11:08:14.148 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "prefix", 2024-03-27T11:08:14.149 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.149 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.149 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.149 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.149 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.149 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.149 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.149 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "name prefix for any created local pool", 2024-03-27T11:08:14.149 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.149 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.149 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.149 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.149 INFO:teuthology.orchestra.run.smithi097.stdout: "subtree": { 2024-03-27T11:08:14.149 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "subtree", 2024-03-27T11:08:14.150 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.150 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.150 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.150 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "rack", 2024-03-27T11:08:14.150 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.150 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.150 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.150 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-27T11:08:14.150 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-27T11:08:14.150 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.150 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.150 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.150 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.150 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.151 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:14.151 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "mds_autoscaler", 2024-03-27T11:08:14.151 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-27T11:08:14.151 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-27T11:08:14.151 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-27T11:08:14.151 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-27T11:08:14.151 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-27T11:08:14.151 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.151 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.151 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.151 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.151 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.151 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.151 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.152 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.152 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.152 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.152 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.152 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.152 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.152 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.152 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.152 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.152 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.152 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.152 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.152 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-27T11:08:14.152 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-27T11:08:14.152 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.153 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.153 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.153 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.153 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.153 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.153 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.153 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.153 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.153 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.153 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.153 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.153 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-27T11:08:14.153 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-27T11:08:14.153 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.153 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.154 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.154 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-27T11:08:14.154 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.154 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.154 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.154 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.154 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.154 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.154 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.154 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.154 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.154 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.154 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.154 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.155 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.155 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.155 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.155 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-27T11:08:14.155 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-27T11:08:14.155 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.155 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.155 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.155 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.155 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.155 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.155 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.155 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.155 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.155 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.156 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.156 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.156 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.156 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.156 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:14.156 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "mirroring", 2024-03-27T11:08:14.156 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-27T11:08:14.156 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-27T11:08:14.156 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-27T11:08:14.156 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-27T11:08:14.156 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-27T11:08:14.156 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.156 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.156 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.157 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.157 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.157 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.157 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.157 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.157 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.157 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.157 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.157 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.157 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.157 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.157 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.157 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.157 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.157 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.158 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.158 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-27T11:08:14.158 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-27T11:08:14.158 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.158 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.158 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.158 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.158 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.158 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.158 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.158 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.158 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.158 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.158 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.159 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.159 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-27T11:08:14.159 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-27T11:08:14.159 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.159 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.159 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.159 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-27T11:08:14.159 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.159 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.159 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.159 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.159 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.159 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.159 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.159 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.160 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.160 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.160 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.160 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.160 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.160 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.160 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.160 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-27T11:08:14.160 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-27T11:08:14.160 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.160 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.160 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.160 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.160 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.161 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.161 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.161 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.161 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.161 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.161 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.161 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.161 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.161 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.161 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:14.161 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "nfs", 2024-03-27T11:08:14.161 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-27T11:08:14.161 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-27T11:08:14.161 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-27T11:08:14.161 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-27T11:08:14.162 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-27T11:08:14.162 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.162 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.162 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.162 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.162 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.162 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.162 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.162 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.162 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.162 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.162 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.162 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.163 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.163 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.163 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.163 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.163 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.163 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.163 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.163 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-27T11:08:14.163 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-27T11:08:14.163 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.163 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.163 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.163 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.163 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.164 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.164 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.164 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.164 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.164 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.164 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.164 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.164 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-27T11:08:14.164 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-27T11:08:14.164 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.164 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.164 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.164 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-27T11:08:14.164 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.165 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.165 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.165 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.165 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.165 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.165 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.165 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.165 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.165 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.165 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.165 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.165 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.165 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.165 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.165 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-27T11:08:14.166 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-27T11:08:14.166 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.166 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.166 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.166 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.166 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.166 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.166 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.166 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.166 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.166 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.166 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.166 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.166 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.167 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.167 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:14.167 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "orchestrator", 2024-03-27T11:08:14.167 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-27T11:08:14.167 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-27T11:08:14.167 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-27T11:08:14.167 INFO:teuthology.orchestra.run.smithi097.stdout: "fail_fs": { 2024-03-27T11:08:14.167 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "fail_fs", 2024-03-27T11:08:14.167 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.167 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.167 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.167 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.167 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.167 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.167 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.168 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-27T11:08:14.168 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.168 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.168 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.168 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.168 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-27T11:08:14.168 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-27T11:08:14.168 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.168 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.168 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.168 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.168 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.168 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.168 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.169 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.169 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.169 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.169 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.169 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.169 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.169 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.169 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.169 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.169 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.169 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.169 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.169 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-27T11:08:14.169 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-27T11:08:14.170 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.170 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.170 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.170 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.170 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.170 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.170 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.170 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.170 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.170 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.170 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.170 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.170 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-27T11:08:14.171 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-27T11:08:14.171 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.171 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.171 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.171 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-27T11:08:14.171 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.171 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.171 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.171 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.171 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.171 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.171 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.171 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.171 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.172 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.172 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.172 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.172 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.172 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.172 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.172 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-27T11:08:14.172 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-27T11:08:14.172 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.172 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.172 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.172 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.172 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.172 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.172 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.173 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.173 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.173 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.173 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.173 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.173 INFO:teuthology.orchestra.run.smithi097.stdout: "orchestrator": { 2024-03-27T11:08:14.173 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "orchestrator", 2024-03-27T11:08:14.173 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.173 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.173 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.173 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.173 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.173 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.173 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.173 INFO:teuthology.orchestra.run.smithi097.stdout: "cephadm", 2024-03-27T11:08:14.174 INFO:teuthology.orchestra.run.smithi097.stdout: "rook", 2024-03-27T11:08:14.174 INFO:teuthology.orchestra.run.smithi097.stdout: "test_orchestrator" 2024-03-27T11:08:14.174 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.174 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Orchestrator backend", 2024-03-27T11:08:14.174 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.174 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.174 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.174 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.174 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.174 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.174 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:14.174 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "osd_perf_query", 2024-03-27T11:08:14.174 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-27T11:08:14.174 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-27T11:08:14.175 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-27T11:08:14.175 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-27T11:08:14.175 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-27T11:08:14.175 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.175 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.175 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.175 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.175 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.175 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.175 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.175 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.175 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.175 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.175 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.176 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.176 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.176 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.176 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.176 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.176 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.176 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.176 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.176 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-27T11:08:14.176 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-27T11:08:14.176 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.176 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.176 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.176 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.176 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.177 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.177 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.177 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.177 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.177 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.177 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.177 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.177 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-27T11:08:14.177 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-27T11:08:14.177 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.177 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.177 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.177 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-27T11:08:14.177 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.178 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.178 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.178 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.178 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.178 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.178 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.178 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.178 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.178 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.178 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.178 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.178 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.178 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.178 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.179 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-27T11:08:14.179 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-27T11:08:14.179 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.179 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.179 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.179 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.179 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.179 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.179 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.179 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.179 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.179 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.179 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.179 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.180 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.180 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.180 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:14.180 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "osd_support", 2024-03-27T11:08:14.180 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-27T11:08:14.180 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-27T11:08:14.180 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-27T11:08:14.180 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-27T11:08:14.180 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-27T11:08:14.180 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.180 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.180 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.180 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.180 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.181 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.181 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.181 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.181 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.181 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.181 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.181 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.181 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.181 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.181 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.181 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.181 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.181 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.181 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.181 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-27T11:08:14.182 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-27T11:08:14.182 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.182 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.182 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.182 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.182 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.182 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.182 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.182 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.182 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.182 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.182 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.183 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.183 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-27T11:08:14.183 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-27T11:08:14.183 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.183 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.183 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.183 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-27T11:08:14.183 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.183 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.183 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.183 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.183 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.183 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.183 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.183 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.183 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.184 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.184 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.184 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.184 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.184 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.184 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.184 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-27T11:08:14.184 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-27T11:08:14.184 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.184 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.184 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.184 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.184 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.184 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.184 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.185 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.185 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.185 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.185 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.185 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.185 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.185 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.185 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:14.185 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "pg_autoscaler", 2024-03-27T11:08:14.185 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-27T11:08:14.185 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-27T11:08:14.185 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-27T11:08:14.185 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-27T11:08:14.185 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-27T11:08:14.185 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.186 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.186 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.186 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.186 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.186 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.186 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.186 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.186 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.186 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.186 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.186 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.186 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.187 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.187 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.187 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.187 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.187 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.187 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.187 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-27T11:08:14.187 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-27T11:08:14.187 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.187 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.187 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.187 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.187 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.188 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.188 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.188 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.188 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.188 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.188 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.188 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.188 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-27T11:08:14.188 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-27T11:08:14.188 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.188 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.188 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.188 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-27T11:08:14.189 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.189 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.189 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.189 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.189 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.189 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.189 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.189 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.189 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.189 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.189 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.189 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.189 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.189 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.189 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.190 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-27T11:08:14.190 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-27T11:08:14.190 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.190 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.190 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.190 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.190 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.190 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.190 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.190 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.190 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.190 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.190 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.191 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.191 INFO:teuthology.orchestra.run.smithi097.stdout: "sleep_interval": { 2024-03-27T11:08:14.191 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "sleep_interval", 2024-03-27T11:08:14.191 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-27T11:08:14.191 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.191 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.191 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "60", 2024-03-27T11:08:14.191 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.191 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.191 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.191 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.191 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.191 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.191 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.192 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.192 INFO:teuthology.orchestra.run.smithi097.stdout: "threshold": { 2024-03-27T11:08:14.192 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "threshold", 2024-03-27T11:08:14.192 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "float", 2024-03-27T11:08:14.192 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.192 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.192 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "3.0", 2024-03-27T11:08:14.192 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "1.0", 2024-03-27T11:08:14.192 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.192 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.192 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "scaling threshold", 2024-03-27T11:08:14.192 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-27T11:08:14.192 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.192 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.192 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.192 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.193 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.193 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:14.193 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "progress", 2024-03-27T11:08:14.193 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-27T11:08:14.193 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-27T11:08:14.193 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-27T11:08:14.193 INFO:teuthology.orchestra.run.smithi097.stdout: "allow_pg_recovery_event": { 2024-03-27T11:08:14.193 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "allow_pg_recovery_event", 2024-03-27T11:08:14.193 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.193 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.193 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.193 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.193 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.193 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.194 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.194 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-27T11:08:14.194 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.194 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.194 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.194 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.194 INFO:teuthology.orchestra.run.smithi097.stdout: "enabled": { 2024-03-27T11:08:14.194 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "enabled", 2024-03-27T11:08:14.194 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.194 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.194 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.194 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:14.194 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.194 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.194 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.195 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.195 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.195 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.195 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.195 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.195 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-27T11:08:14.195 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-27T11:08:14.195 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.195 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.195 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.195 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.195 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.195 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.195 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.195 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.195 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.196 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.196 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.196 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.196 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.196 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.196 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.196 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.196 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.196 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.196 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.196 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-27T11:08:14.196 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-27T11:08:14.196 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.196 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.196 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.197 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.197 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.197 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.197 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.197 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.197 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.197 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.197 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.197 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.197 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-27T11:08:14.197 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-27T11:08:14.197 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.197 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.197 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.197 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-27T11:08:14.197 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.198 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.198 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.198 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.198 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.198 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.198 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.198 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.198 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.198 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.198 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.198 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.198 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.198 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.199 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.199 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-27T11:08:14.199 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-27T11:08:14.199 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.199 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.199 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.199 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.199 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.199 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.199 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.199 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.199 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.199 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.199 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.199 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.200 INFO:teuthology.orchestra.run.smithi097.stdout: "max_completed_events": { 2024-03-27T11:08:14.200 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "max_completed_events", 2024-03-27T11:08:14.200 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:14.200 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.200 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.200 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "50", 2024-03-27T11:08:14.200 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.200 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.200 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.200 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "number of past completed events to remember", 2024-03-27T11:08:14.200 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.200 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.200 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.200 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.200 INFO:teuthology.orchestra.run.smithi097.stdout: "sleep_interval": { 2024-03-27T11:08:14.201 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "sleep_interval", 2024-03-27T11:08:14.201 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-27T11:08:14.201 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.201 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.201 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "5", 2024-03-27T11:08:14.201 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.201 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.201 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.201 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "how long the module is going to sleep", 2024-03-27T11:08:14.201 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.201 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.201 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.201 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.201 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.201 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.202 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:14.202 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "prometheus", 2024-03-27T11:08:14.202 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-27T11:08:14.202 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-27T11:08:14.202 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-27T11:08:14.202 INFO:teuthology.orchestra.run.smithi097.stdout: "cache": { 2024-03-27T11:08:14.202 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "cache", 2024-03-27T11:08:14.202 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.202 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.202 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.202 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:14.202 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.202 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.202 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.203 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.203 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.203 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.203 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.203 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.203 INFO:teuthology.orchestra.run.smithi097.stdout: "exclude_perf_counters": { 2024-03-27T11:08:14.203 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "exclude_perf_counters", 2024-03-27T11:08:14.203 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.203 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.203 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.203 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:14.203 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.203 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.203 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.203 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-27T11:08:14.204 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-27T11:08:14.204 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.204 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.204 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.204 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-27T11:08:14.204 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-27T11:08:14.204 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.204 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.204 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.204 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.204 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.204 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.204 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.204 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.205 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.205 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.205 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.205 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.205 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.205 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.205 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.205 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.205 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.205 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.205 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.205 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-27T11:08:14.205 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-27T11:08:14.205 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.205 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.205 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.206 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.206 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.206 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.206 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.206 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.206 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.206 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.206 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.206 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.206 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-27T11:08:14.206 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-27T11:08:14.206 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.206 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.206 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.207 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-27T11:08:14.207 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.207 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.207 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.207 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.207 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.209 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.209 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.209 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.209 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.209 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.210 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.210 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.210 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.210 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.210 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.210 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-27T11:08:14.210 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-27T11:08:14.210 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.210 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.210 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.210 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.210 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.210 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.211 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.211 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.211 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.211 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.211 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.211 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.211 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_stats_pools": { 2024-03-27T11:08:14.211 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rbd_stats_pools", 2024-03-27T11:08:14.211 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.211 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.211 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.211 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.211 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.211 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.211 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.212 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.212 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.212 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.212 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.212 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.212 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-27T11:08:14.212 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-27T11:08:14.212 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:14.212 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.212 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.212 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "300", 2024-03-27T11:08:14.212 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.212 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.212 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.212 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.213 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.213 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.213 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.213 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.213 INFO:teuthology.orchestra.run.smithi097.stdout: "scrape_interval": { 2024-03-27T11:08:14.213 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "scrape_interval", 2024-03-27T11:08:14.213 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "float", 2024-03-27T11:08:14.213 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.213 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.213 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "15.0", 2024-03-27T11:08:14.213 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.213 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.213 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.213 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.214 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.214 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.214 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.214 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.214 INFO:teuthology.orchestra.run.smithi097.stdout: "server_addr": { 2024-03-27T11:08:14.214 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "server_addr", 2024-03-27T11:08:14.214 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.214 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.214 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.214 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "::", 2024-03-27T11:08:14.214 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.214 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.214 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.214 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-27T11:08:14.214 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.215 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.215 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.215 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.215 INFO:teuthology.orchestra.run.smithi097.stdout: "server_port": { 2024-03-27T11:08:14.215 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "server_port", 2024-03-27T11:08:14.215 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:14.215 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.215 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.215 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "9283", 2024-03-27T11:08:14.215 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.215 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.215 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.215 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-27T11:08:14.215 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.216 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.216 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.216 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.216 INFO:teuthology.orchestra.run.smithi097.stdout: "stale_cache_strategy": { 2024-03-27T11:08:14.216 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "stale_cache_strategy", 2024-03-27T11:08:14.216 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.216 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.216 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.216 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "log", 2024-03-27T11:08:14.216 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.216 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.216 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.216 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.216 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.216 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.217 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.217 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.217 INFO:teuthology.orchestra.run.smithi097.stdout: "standby_behaviour": { 2024-03-27T11:08:14.217 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "standby_behaviour", 2024-03-27T11:08:14.217 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.217 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.217 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.217 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "default", 2024-03-27T11:08:14.217 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.217 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.217 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.217 INFO:teuthology.orchestra.run.smithi097.stdout: "default", 2024-03-27T11:08:14.217 INFO:teuthology.orchestra.run.smithi097.stdout: "error" 2024-03-27T11:08:14.217 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.217 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.218 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.218 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.218 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.218 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.218 INFO:teuthology.orchestra.run.smithi097.stdout: "standby_error_status_code": { 2024-03-27T11:08:14.218 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "standby_error_status_code", 2024-03-27T11:08:14.218 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:14.218 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.218 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.218 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "500", 2024-03-27T11:08:14.218 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "400", 2024-03-27T11:08:14.218 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "599", 2024-03-27T11:08:14.218 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.218 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.218 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.219 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.219 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.219 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.219 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.219 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.219 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:14.219 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rbd_support", 2024-03-27T11:08:14.219 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-27T11:08:14.219 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-27T11:08:14.219 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-27T11:08:14.219 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-27T11:08:14.219 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-27T11:08:14.219 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.219 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.219 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.220 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.220 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.220 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.220 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.220 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.220 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.220 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.220 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.220 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.220 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.220 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.220 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.220 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.220 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.220 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.221 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.221 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-27T11:08:14.221 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-27T11:08:14.221 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.221 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.221 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.221 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.221 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.221 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.221 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.221 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.221 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.221 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.221 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.221 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.222 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-27T11:08:14.222 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-27T11:08:14.222 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.222 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.222 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.222 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-27T11:08:14.222 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.222 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.222 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.222 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.222 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.222 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.222 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.222 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.222 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.223 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.223 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.223 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.223 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.223 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.223 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.223 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-27T11:08:14.223 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-27T11:08:14.223 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.223 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.223 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.223 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.223 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.223 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.224 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.224 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.224 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.224 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.224 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.224 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.224 INFO:teuthology.orchestra.run.smithi097.stdout: "max_concurrent_snap_create": { 2024-03-27T11:08:14.224 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "max_concurrent_snap_create", 2024-03-27T11:08:14.224 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:14.224 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.224 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.224 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10", 2024-03-27T11:08:14.224 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.224 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.224 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.224 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.225 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.225 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.225 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.225 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.225 INFO:teuthology.orchestra.run.smithi097.stdout: "mirror_snapshot_schedule": { 2024-03-27T11:08:14.225 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "mirror_snapshot_schedule", 2024-03-27T11:08:14.225 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.225 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.225 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.225 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.225 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.225 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.225 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.225 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.225 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.226 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.226 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.226 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.226 INFO:teuthology.orchestra.run.smithi097.stdout: "trash_purge_schedule": { 2024-03-27T11:08:14.226 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "trash_purge_schedule", 2024-03-27T11:08:14.226 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.226 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.226 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.226 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.226 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.226 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.226 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.226 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.227 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.227 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.227 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.227 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.227 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.227 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.227 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:14.227 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "restful", 2024-03-27T11:08:14.227 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-27T11:08:14.227 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-27T11:08:14.227 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-27T11:08:14.227 INFO:teuthology.orchestra.run.smithi097.stdout: "enable_auth": { 2024-03-27T11:08:14.227 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "enable_auth", 2024-03-27T11:08:14.227 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.227 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.228 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.228 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:14.228 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.228 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.228 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.228 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.228 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.228 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.228 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.228 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.228 INFO:teuthology.orchestra.run.smithi097.stdout: "key_file": { 2024-03-27T11:08:14.228 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "key_file", 2024-03-27T11:08:14.228 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.228 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.228 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.229 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.229 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.229 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.229 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.229 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.229 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.229 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.229 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.229 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.229 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-27T11:08:14.229 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-27T11:08:14.229 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.229 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.229 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.230 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.230 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.230 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.230 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.230 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.230 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.230 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.230 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.230 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.230 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.230 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.230 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.230 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.231 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.231 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.231 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.231 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-27T11:08:14.231 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-27T11:08:14.231 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.231 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.231 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.231 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.231 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.231 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.231 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.231 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.231 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.231 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.232 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.232 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.232 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-27T11:08:14.232 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-27T11:08:14.232 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.232 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.232 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.232 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-27T11:08:14.232 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.232 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.232 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.232 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.232 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.233 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.233 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.233 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.233 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.233 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.233 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.233 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.233 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.233 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.233 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.233 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-27T11:08:14.233 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-27T11:08:14.233 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.233 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.233 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.234 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.234 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.234 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.234 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.234 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.234 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.234 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.234 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.234 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.234 INFO:teuthology.orchestra.run.smithi097.stdout: "server_addr": { 2024-03-27T11:08:14.234 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "server_addr", 2024-03-27T11:08:14.234 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.234 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.235 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.235 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.235 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.235 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.235 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.235 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.235 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.235 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.235 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.235 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.235 INFO:teuthology.orchestra.run.smithi097.stdout: "server_port": { 2024-03-27T11:08:14.235 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "server_port", 2024-03-27T11:08:14.235 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.236 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.236 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.236 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.236 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.236 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.236 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.236 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.236 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.236 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.236 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.236 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.236 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.236 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.236 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:14.236 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rgw", 2024-03-27T11:08:14.237 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-27T11:08:14.237 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-27T11:08:14.237 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-27T11:08:14.237 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-27T11:08:14.237 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-27T11:08:14.237 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.237 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.237 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.237 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.237 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.237 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.237 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.237 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.237 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.238 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.238 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.238 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.238 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.238 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.238 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.238 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.238 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.238 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.238 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.238 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-27T11:08:14.238 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-27T11:08:14.238 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.239 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.239 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.239 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.239 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.239 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.239 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.239 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.239 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.239 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.239 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.239 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.239 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-27T11:08:14.239 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-27T11:08:14.239 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.240 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.240 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.240 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-27T11:08:14.240 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.240 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.240 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.240 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.240 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.240 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.240 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.240 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.240 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.240 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.241 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.241 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.241 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.241 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.241 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.241 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-27T11:08:14.241 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-27T11:08:14.241 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.241 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.241 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.241 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.241 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.241 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.241 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.242 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.242 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.242 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.242 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.242 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.242 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.242 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.242 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:14.242 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rook", 2024-03-27T11:08:14.242 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-27T11:08:14.242 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-27T11:08:14.242 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-27T11:08:14.242 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-27T11:08:14.243 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-27T11:08:14.243 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.243 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.243 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.243 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.243 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.243 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.243 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.243 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.243 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.243 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.243 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.243 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.243 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.244 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.244 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.244 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.244 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.244 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.244 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.244 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-27T11:08:14.244 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-27T11:08:14.244 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.244 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.244 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.244 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.244 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.244 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.244 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.245 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.245 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.245 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.245 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.245 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.245 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-27T11:08:14.245 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-27T11:08:14.245 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.245 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.245 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.245 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-27T11:08:14.245 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.245 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.245 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.246 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.246 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.246 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.246 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.246 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.246 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.246 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.246 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.246 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.246 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.246 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.246 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.246 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-27T11:08:14.246 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-27T11:08:14.247 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.247 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.247 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.247 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.247 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.247 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.247 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.247 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.247 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.247 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.247 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.247 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.247 INFO:teuthology.orchestra.run.smithi097.stdout: "storage_class": { 2024-03-27T11:08:14.247 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "storage_class", 2024-03-27T11:08:14.248 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.248 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.248 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.248 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "local", 2024-03-27T11:08:14.248 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.248 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.248 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.248 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-27T11:08:14.248 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.248 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.248 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.248 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.248 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.248 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.248 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:14.249 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "selftest", 2024-03-27T11:08:14.249 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-27T11:08:14.249 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-27T11:08:14.249 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-27T11:08:14.249 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-27T11:08:14.249 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-27T11:08:14.249 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.249 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.249 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.249 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.249 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.249 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.249 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.249 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.249 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.250 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.250 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.250 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.250 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.250 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.250 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.250 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.250 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.250 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.250 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.250 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-27T11:08:14.250 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-27T11:08:14.250 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.250 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.251 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.251 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.251 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.251 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.251 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.251 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.251 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.251 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.251 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.251 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.251 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-27T11:08:14.251 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-27T11:08:14.251 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.251 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.252 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.252 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-27T11:08:14.252 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.252 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.252 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.252 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.252 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.252 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.252 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.252 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.252 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.252 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.252 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.252 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.253 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.253 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.253 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.253 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-27T11:08:14.253 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-27T11:08:14.253 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.253 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.253 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.253 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.253 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.253 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.253 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.253 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.253 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.253 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.253 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.254 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.254 INFO:teuthology.orchestra.run.smithi097.stdout: "roption1": { 2024-03-27T11:08:14.254 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "roption1", 2024-03-27T11:08:14.254 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.254 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.254 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.254 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.254 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.254 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.254 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.254 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.254 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.254 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.255 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.255 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.255 INFO:teuthology.orchestra.run.smithi097.stdout: "roption2": { 2024-03-27T11:08:14.255 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "roption2", 2024-03-27T11:08:14.255 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.255 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.255 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.255 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "xyz", 2024-03-27T11:08:14.255 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.255 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.255 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.255 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.255 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.255 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.255 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.256 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.256 INFO:teuthology.orchestra.run.smithi097.stdout: "rwoption1": { 2024-03-27T11:08:14.256 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rwoption1", 2024-03-27T11:08:14.256 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.256 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.256 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.256 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.256 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.256 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.256 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.256 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.256 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.256 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.256 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.256 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.257 INFO:teuthology.orchestra.run.smithi097.stdout: "rwoption2": { 2024-03-27T11:08:14.257 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rwoption2", 2024-03-27T11:08:14.257 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:14.257 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.257 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.257 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.257 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.257 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.257 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.257 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.257 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.257 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.257 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.257 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.257 INFO:teuthology.orchestra.run.smithi097.stdout: "rwoption3": { 2024-03-27T11:08:14.258 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rwoption3", 2024-03-27T11:08:14.258 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "float", 2024-03-27T11:08:14.258 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.258 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.258 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.258 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.258 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.258 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.258 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.258 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.258 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.258 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.258 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.259 INFO:teuthology.orchestra.run.smithi097.stdout: "rwoption4": { 2024-03-27T11:08:14.259 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rwoption4", 2024-03-27T11:08:14.259 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.259 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.259 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.259 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.259 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.259 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.259 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.259 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.259 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.259 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.259 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.259 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.259 INFO:teuthology.orchestra.run.smithi097.stdout: "rwoption5": { 2024-03-27T11:08:14.260 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rwoption5", 2024-03-27T11:08:14.260 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.260 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.260 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.260 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.260 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.260 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.260 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.260 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.260 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.260 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.260 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.260 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.260 INFO:teuthology.orchestra.run.smithi097.stdout: "rwoption6": { 2024-03-27T11:08:14.260 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rwoption6", 2024-03-27T11:08:14.261 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.261 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.261 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.261 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:14.261 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.261 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.261 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.261 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.261 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.261 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.261 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.261 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.261 INFO:teuthology.orchestra.run.smithi097.stdout: "rwoption7": { 2024-03-27T11:08:14.261 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rwoption7", 2024-03-27T11:08:14.261 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:14.262 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.262 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.262 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.262 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "1", 2024-03-27T11:08:14.262 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "42", 2024-03-27T11:08:14.262 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.262 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.262 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.262 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.262 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.262 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.262 INFO:teuthology.orchestra.run.smithi097.stdout: "testkey": { 2024-03-27T11:08:14.262 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "testkey", 2024-03-27T11:08:14.263 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.263 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.263 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.263 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.263 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.263 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.263 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.263 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.263 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.263 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.263 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.263 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.263 INFO:teuthology.orchestra.run.smithi097.stdout: "testlkey": { 2024-03-27T11:08:14.264 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "testlkey", 2024-03-27T11:08:14.264 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.264 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.264 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.264 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.264 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.264 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.264 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.264 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.264 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.264 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.264 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.264 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.264 INFO:teuthology.orchestra.run.smithi097.stdout: "testnewline": { 2024-03-27T11:08:14.264 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "testnewline", 2024-03-27T11:08:14.265 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.265 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.265 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.265 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.265 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.265 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.265 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.265 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.265 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.265 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.265 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.265 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.265 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.265 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.265 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:14.265 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "snap_schedule", 2024-03-27T11:08:14.266 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-27T11:08:14.266 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-27T11:08:14.266 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-27T11:08:14.266 INFO:teuthology.orchestra.run.smithi097.stdout: "allow_m_granularity": { 2024-03-27T11:08:14.266 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "allow_m_granularity", 2024-03-27T11:08:14.266 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.266 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.266 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.266 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.266 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.266 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.266 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.266 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "allow minute scheduled snapshots", 2024-03-27T11:08:14.266 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.267 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.267 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.267 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.267 INFO:teuthology.orchestra.run.smithi097.stdout: "dump_on_update": { 2024-03-27T11:08:14.267 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "dump_on_update", 2024-03-27T11:08:14.267 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.267 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.267 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.267 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.267 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.267 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.267 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.267 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "dump database to debug log on update", 2024-03-27T11:08:14.267 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.267 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.268 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.268 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.268 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-27T11:08:14.268 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-27T11:08:14.268 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.268 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.268 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.268 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.268 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.268 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.268 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.268 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.268 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.268 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.269 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.269 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.269 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.269 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.269 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.269 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.269 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.269 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.269 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.269 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-27T11:08:14.269 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-27T11:08:14.269 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.269 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.269 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.270 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.270 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.270 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.270 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.270 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.270 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.270 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.270 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.270 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.270 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-27T11:08:14.270 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-27T11:08:14.270 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.270 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.271 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.271 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-27T11:08:14.271 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.271 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.271 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.271 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.271 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.271 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.271 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.271 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.271 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.271 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.271 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.271 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.272 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.272 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.272 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.272 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-27T11:08:14.272 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-27T11:08:14.272 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.272 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.272 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.272 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.272 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.272 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.272 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.272 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.272 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.272 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.273 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.273 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.273 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.273 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.273 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:14.273 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "stats", 2024-03-27T11:08:14.273 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-27T11:08:14.273 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-27T11:08:14.273 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-27T11:08:14.273 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-27T11:08:14.273 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-27T11:08:14.273 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.273 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.273 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.273 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.274 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.274 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.274 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.274 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.274 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.274 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.274 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.274 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.274 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.274 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.274 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.274 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.274 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.275 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.275 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.275 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-27T11:08:14.275 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-27T11:08:14.275 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.275 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.275 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.275 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.275 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.275 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.275 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.275 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.275 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.275 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.275 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.276 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.276 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-27T11:08:14.276 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-27T11:08:14.276 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.276 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.276 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.276 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-27T11:08:14.276 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.276 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.276 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.276 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.276 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.276 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.276 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.276 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.277 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.277 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.277 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.277 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.277 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.277 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.277 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.277 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-27T11:08:14.277 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-27T11:08:14.277 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.277 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.277 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.277 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.277 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.277 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.278 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.278 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.278 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.278 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.278 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.278 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.278 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.278 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.278 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:14.278 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "status", 2024-03-27T11:08:14.278 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-27T11:08:14.278 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-27T11:08:14.279 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-27T11:08:14.279 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-27T11:08:14.279 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-27T11:08:14.279 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.279 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.279 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.279 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.279 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.279 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.279 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.279 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.279 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.279 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.280 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.280 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.280 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.280 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.280 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.280 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.280 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.280 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.280 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.280 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-27T11:08:14.280 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-27T11:08:14.280 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.280 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.280 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.280 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.281 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.281 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.281 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.281 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.281 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.281 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.281 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.281 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.281 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-27T11:08:14.281 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-27T11:08:14.281 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.281 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.281 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.281 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-27T11:08:14.282 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.282 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.282 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.282 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.282 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.282 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.282 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.282 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.282 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.282 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.282 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.282 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.282 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.283 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.283 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.283 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-27T11:08:14.283 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-27T11:08:14.283 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.283 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.283 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.283 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.283 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.283 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.283 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.283 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.283 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.283 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.284 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.284 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.284 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.284 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.284 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:14.284 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "telegraf", 2024-03-27T11:08:14.284 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-27T11:08:14.284 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-27T11:08:14.284 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-27T11:08:14.284 INFO:teuthology.orchestra.run.smithi097.stdout: "address": { 2024-03-27T11:08:14.284 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "address", 2024-03-27T11:08:14.284 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.284 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.285 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.285 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-27T11:08:14.285 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.285 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.285 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.285 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.285 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.285 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.285 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.285 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.285 INFO:teuthology.orchestra.run.smithi097.stdout: "interval": { 2024-03-27T11:08:14.285 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "interval", 2024-03-27T11:08:14.285 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-27T11:08:14.285 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.286 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.286 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "15", 2024-03-27T11:08:14.286 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.286 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.286 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.286 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.286 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.286 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.286 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.286 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.286 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-27T11:08:14.286 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-27T11:08:14.287 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.287 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.287 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.287 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.287 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.287 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.287 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.287 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.287 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.287 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.287 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.287 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.287 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.287 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.288 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.288 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.288 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.288 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.288 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.288 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-27T11:08:14.288 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-27T11:08:14.288 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.288 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.288 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.288 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.288 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.288 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.289 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.289 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.289 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.289 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.289 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.289 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.289 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-27T11:08:14.289 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-27T11:08:14.289 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.289 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.289 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.289 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-27T11:08:14.289 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.289 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.290 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.290 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.290 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.290 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.290 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.290 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.290 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.290 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.290 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.290 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.290 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.290 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.290 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.290 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-27T11:08:14.291 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-27T11:08:14.291 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.291 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.291 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.291 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.291 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.291 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.291 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.291 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.291 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.291 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.291 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.291 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.291 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.291 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.292 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:14.292 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "telemetry", 2024-03-27T11:08:14.292 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-27T11:08:14.292 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-27T11:08:14.292 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-27T11:08:14.292 INFO:teuthology.orchestra.run.smithi097.stdout: "channel_basic": { 2024-03-27T11:08:14.292 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "channel_basic", 2024-03-27T11:08:14.292 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.292 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.292 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.292 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:14.292 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.292 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.292 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.293 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-27T11:08:14.293 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.293 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.293 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.293 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.293 INFO:teuthology.orchestra.run.smithi097.stdout: "channel_crash": { 2024-03-27T11:08:14.293 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "channel_crash", 2024-03-27T11:08:14.293 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.293 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.293 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.293 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:14.293 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.293 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.293 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.294 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-27T11:08:14.294 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.294 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.294 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.294 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.294 INFO:teuthology.orchestra.run.smithi097.stdout: "channel_device": { 2024-03-27T11:08:14.294 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "channel_device", 2024-03-27T11:08:14.294 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.294 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.294 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.294 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:14.294 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.294 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.294 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.295 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-27T11:08:14.295 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.295 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.295 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.295 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.295 INFO:teuthology.orchestra.run.smithi097.stdout: "channel_ident": { 2024-03-27T11:08:14.295 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "channel_ident", 2024-03-27T11:08:14.295 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.295 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.295 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.295 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.295 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.295 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.295 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.295 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-27T11:08:14.296 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.296 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.296 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.296 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.296 INFO:teuthology.orchestra.run.smithi097.stdout: "channel_perf": { 2024-03-27T11:08:14.296 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "channel_perf", 2024-03-27T11:08:14.296 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.296 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.296 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.296 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.296 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.296 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.296 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.296 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-27T11:08:14.296 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.297 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.297 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.297 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.297 INFO:teuthology.orchestra.run.smithi097.stdout: "contact": { 2024-03-27T11:08:14.297 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "contact", 2024-03-27T11:08:14.297 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.297 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.297 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.297 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.297 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.297 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.297 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.297 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.297 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.297 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.298 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.298 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.298 INFO:teuthology.orchestra.run.smithi097.stdout: "description": { 2024-03-27T11:08:14.298 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "description", 2024-03-27T11:08:14.298 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.298 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.298 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.298 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.298 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.298 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.298 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.298 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.298 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.299 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.299 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.299 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.299 INFO:teuthology.orchestra.run.smithi097.stdout: "device_url": { 2024-03-27T11:08:14.299 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "device_url", 2024-03-27T11:08:14.299 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.299 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.299 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.299 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-27T11:08:14.299 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.299 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.299 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.299 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.299 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.300 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.300 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.300 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.300 INFO:teuthology.orchestra.run.smithi097.stdout: "enabled": { 2024-03-27T11:08:14.300 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "enabled", 2024-03-27T11:08:14.300 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.300 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.300 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.300 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.300 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.300 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.300 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.300 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.300 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.300 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.301 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.301 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.301 INFO:teuthology.orchestra.run.smithi097.stdout: "interval": { 2024-03-27T11:08:14.301 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "interval", 2024-03-27T11:08:14.301 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:14.301 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.301 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.301 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "24", 2024-03-27T11:08:14.301 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "8", 2024-03-27T11:08:14.301 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.301 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.301 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.301 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.301 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.301 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.302 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.302 INFO:teuthology.orchestra.run.smithi097.stdout: "last_opt_revision": { 2024-03-27T11:08:14.302 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "last_opt_revision", 2024-03-27T11:08:14.302 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:14.302 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.302 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.302 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "1", 2024-03-27T11:08:14.302 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.302 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.302 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.302 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.302 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.302 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.302 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.303 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.303 INFO:teuthology.orchestra.run.smithi097.stdout: "leaderboard": { 2024-03-27T11:08:14.303 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "leaderboard", 2024-03-27T11:08:14.303 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.303 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.303 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.303 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.303 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.303 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.303 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.303 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.303 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.303 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.303 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.303 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.304 INFO:teuthology.orchestra.run.smithi097.stdout: "leaderboard_description": { 2024-03-27T11:08:14.304 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "leaderboard_description", 2024-03-27T11:08:14.304 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.304 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.304 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.304 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.304 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.304 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.304 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.304 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.304 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.304 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.304 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.304 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.305 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-27T11:08:14.305 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-27T11:08:14.305 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.305 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.305 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.305 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.305 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.305 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.305 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.305 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.305 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.305 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.305 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.305 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.306 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.306 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.306 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.306 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.306 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.306 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.306 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.306 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-27T11:08:14.306 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-27T11:08:14.306 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.306 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.306 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.306 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.306 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.307 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.307 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.307 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.307 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.307 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.307 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.307 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.307 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-27T11:08:14.307 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-27T11:08:14.307 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.307 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.307 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.307 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-27T11:08:14.307 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.307 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.307 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.308 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.308 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.308 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.308 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.308 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.308 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.308 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.308 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.308 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.308 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.308 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.308 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.308 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-27T11:08:14.308 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-27T11:08:14.309 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.309 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.309 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.309 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.309 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.309 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.309 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.309 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.309 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.309 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.309 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.309 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.309 INFO:teuthology.orchestra.run.smithi097.stdout: "organization": { 2024-03-27T11:08:14.309 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "organization", 2024-03-27T11:08:14.309 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.310 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.310 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.310 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.310 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.310 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.310 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.310 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.310 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.310 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.310 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.310 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.310 INFO:teuthology.orchestra.run.smithi097.stdout: "proxy": { 2024-03-27T11:08:14.311 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "proxy", 2024-03-27T11:08:14.311 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.311 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.311 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.311 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.311 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.311 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.311 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.311 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.311 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.311 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.311 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.311 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.311 INFO:teuthology.orchestra.run.smithi097.stdout: "url": { 2024-03-27T11:08:14.312 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "url", 2024-03-27T11:08:14.312 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.312 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.312 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.312 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-27T11:08:14.312 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.312 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.312 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.312 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.312 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.312 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.312 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.312 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.313 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.313 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.313 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:14.313 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "test_orchestrator", 2024-03-27T11:08:14.313 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-27T11:08:14.313 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-27T11:08:14.313 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-27T11:08:14.313 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-27T11:08:14.313 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-27T11:08:14.313 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.313 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.313 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.313 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.313 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.314 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.314 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.314 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.314 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.314 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.314 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.314 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.314 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.314 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.314 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.314 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.314 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.314 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.314 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.315 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-27T11:08:14.315 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-27T11:08:14.315 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.315 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.315 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.315 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.315 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.315 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.315 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.315 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.315 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.315 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.315 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.315 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.315 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-27T11:08:14.315 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-27T11:08:14.316 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.316 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.316 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.316 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-27T11:08:14.316 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.316 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.316 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.316 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.316 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.316 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.316 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.316 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.316 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.316 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.316 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.317 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.317 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.317 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.317 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.317 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-27T11:08:14.317 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-27T11:08:14.317 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.317 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.317 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.317 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.317 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.317 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.317 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.317 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.318 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.318 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.318 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.318 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.318 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.318 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.318 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:14.318 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "volumes", 2024-03-27T11:08:14.318 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-27T11:08:14.318 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-27T11:08:14.318 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-27T11:08:14.318 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-27T11:08:14.318 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-27T11:08:14.318 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.319 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.319 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.319 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.319 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.319 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.319 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.319 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.319 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.319 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.319 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.319 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.319 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.319 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.319 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.320 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.320 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.320 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.320 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.320 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-27T11:08:14.320 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-27T11:08:14.320 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.320 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.320 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.320 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.320 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.320 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.320 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.320 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.321 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.321 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.321 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.321 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.321 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-27T11:08:14.321 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-27T11:08:14.321 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.321 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.321 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.321 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-27T11:08:14.321 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.321 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.321 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.321 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.322 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.322 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.322 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.322 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.322 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.322 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.322 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.322 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.322 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.322 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.322 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.322 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-27T11:08:14.322 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-27T11:08:14.323 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.323 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.323 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.323 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.323 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.323 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.323 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.323 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.323 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.323 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.323 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.323 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.323 INFO:teuthology.orchestra.run.smithi097.stdout: "max_concurrent_clones": { 2024-03-27T11:08:14.324 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "max_concurrent_clones", 2024-03-27T11:08:14.324 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:14.324 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.324 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.324 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "4", 2024-03-27T11:08:14.324 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.324 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.324 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.324 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-27T11:08:14.324 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.324 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.324 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.324 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.325 INFO:teuthology.orchestra.run.smithi097.stdout: "periodic_async_work": { 2024-03-27T11:08:14.325 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "periodic_async_work", 2024-03-27T11:08:14.325 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.325 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.325 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.325 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.325 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.325 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.325 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.325 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Periodically check for async work", 2024-03-27T11:08:14.325 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.325 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.326 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.326 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.326 INFO:teuthology.orchestra.run.smithi097.stdout: "snapshot_clone_delay": { 2024-03-27T11:08:14.326 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "snapshot_clone_delay", 2024-03-27T11:08:14.326 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:14.326 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.326 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.326 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "0", 2024-03-27T11:08:14.326 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.326 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.326 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.326 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-27T11:08:14.326 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.327 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.327 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.327 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.327 INFO:teuthology.orchestra.run.smithi097.stdout: "snapshot_clone_no_wait": { 2024-03-27T11:08:14.327 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "snapshot_clone_no_wait", 2024-03-27T11:08:14.327 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.327 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.327 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.327 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:14.327 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.327 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.327 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.327 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-27T11:08:14.328 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.328 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.328 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.328 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.328 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.328 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.328 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:14.328 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "zabbix", 2024-03-27T11:08:14.328 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-27T11:08:14.328 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-27T11:08:14.328 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-27T11:08:14.328 INFO:teuthology.orchestra.run.smithi097.stdout: "discovery_interval": { 2024-03-27T11:08:14.328 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "discovery_interval", 2024-03-27T11:08:14.328 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "uint", 2024-03-27T11:08:14.329 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.329 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.329 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "100", 2024-03-27T11:08:14.329 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.329 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.329 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.329 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.329 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.329 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.329 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.329 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.329 INFO:teuthology.orchestra.run.smithi097.stdout: "identifier": { 2024-03-27T11:08:14.329 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "identifier", 2024-03-27T11:08:14.330 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.330 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.330 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.330 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.330 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.330 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.330 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.330 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.330 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.330 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.330 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.330 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.330 INFO:teuthology.orchestra.run.smithi097.stdout: "interval": { 2024-03-27T11:08:14.331 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "interval", 2024-03-27T11:08:14.331 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-27T11:08:14.331 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.331 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.331 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "60", 2024-03-27T11:08:14.331 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.331 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.331 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.331 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.331 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.331 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.331 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.331 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.331 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-27T11:08:14.331 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-27T11:08:14.332 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.332 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.332 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.332 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.332 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.332 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.332 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.332 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.332 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.332 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.332 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.332 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.332 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.332 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.333 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.333 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.333 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.333 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.333 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.333 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-27T11:08:14.333 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-27T11:08:14.333 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.333 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.333 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.333 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.333 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.333 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.334 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.334 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.334 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.334 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.334 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.334 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.334 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-27T11:08:14.334 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-27T11:08:14.334 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.334 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.334 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.334 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-27T11:08:14.334 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.335 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.335 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.335 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.335 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.335 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.335 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.335 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.335 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.335 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.335 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.335 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.335 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.335 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.335 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.336 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-27T11:08:14.336 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-27T11:08:14.336 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.336 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.336 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.336 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.336 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.336 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.336 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.336 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.336 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.336 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.336 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.337 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.337 INFO:teuthology.orchestra.run.smithi097.stdout: "zabbix_host": { 2024-03-27T11:08:14.337 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "zabbix_host", 2024-03-27T11:08:14.337 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.337 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.337 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.337 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.337 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.337 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.337 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.337 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.337 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.337 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.337 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.338 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.338 INFO:teuthology.orchestra.run.smithi097.stdout: "zabbix_port": { 2024-03-27T11:08:14.338 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "zabbix_port", 2024-03-27T11:08:14.338 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:14.338 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.338 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.338 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10051", 2024-03-27T11:08:14.338 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.338 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.338 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.338 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.338 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.338 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.339 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.339 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.339 INFO:teuthology.orchestra.run.smithi097.stdout: "zabbix_sender": { 2024-03-27T11:08:14.339 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "zabbix_sender", 2024-03-27T11:08:14.339 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.339 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.339 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.339 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-27T11:08:14.339 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.339 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.339 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.339 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.339 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.339 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.340 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.340 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.340 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.340 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.340 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-03-27T11:08:14.340 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.340 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.340 INFO:teuthology.orchestra.run.smithi097.stdout: "modules": [ 2024-03-27T11:08:14.340 INFO:teuthology.orchestra.run.smithi097.stdout: "cephadm", 2024-03-27T11:08:14.340 INFO:teuthology.orchestra.run.smithi097.stdout: "dashboard", 2024-03-27T11:08:14.340 INFO:teuthology.orchestra.run.smithi097.stdout: "iostat", 2024-03-27T11:08:14.340 INFO:teuthology.orchestra.run.smithi097.stdout: "nfs", 2024-03-27T11:08:14.340 INFO:teuthology.orchestra.run.smithi097.stdout: "restful" 2024-03-27T11:08:14.340 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.341 INFO:teuthology.orchestra.run.smithi097.stdout: "available_modules": [ 2024-03-27T11:08:14.341 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:14.341 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "alerts", 2024-03-27T11:08:14.341 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-27T11:08:14.341 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-27T11:08:14.341 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-27T11:08:14.341 INFO:teuthology.orchestra.run.smithi097.stdout: "interval": { 2024-03-27T11:08:14.341 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "interval", 2024-03-27T11:08:14.341 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-27T11:08:14.341 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.341 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.341 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "60", 2024-03-27T11:08:14.341 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.341 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.342 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.342 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "How frequently to reexamine health status", 2024-03-27T11:08:14.342 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.342 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.342 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.342 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.342 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-27T11:08:14.342 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-27T11:08:14.342 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.342 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.342 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.342 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.342 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.343 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.343 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.343 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.343 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.343 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.343 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.343 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.343 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.343 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.343 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.343 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.343 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.343 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.344 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.344 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-27T11:08:14.344 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-27T11:08:14.344 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.344 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.344 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.344 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.344 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.344 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.344 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.344 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.344 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.344 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.344 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.345 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.345 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-27T11:08:14.345 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-27T11:08:14.345 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.345 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.345 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.345 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-27T11:08:14.345 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.345 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.345 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.345 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.345 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.345 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.345 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.346 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.346 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.346 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.346 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.346 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.346 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.346 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.346 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.346 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-27T11:08:14.346 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-27T11:08:14.346 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.346 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.346 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.346 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.347 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.347 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.347 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.347 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.347 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.347 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.347 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.347 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.347 INFO:teuthology.orchestra.run.smithi097.stdout: "smtp_destination": { 2024-03-27T11:08:14.347 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "smtp_destination", 2024-03-27T11:08:14.347 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.347 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.347 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.348 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.348 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.348 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.348 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.348 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Email address to send alerts to", 2024-03-27T11:08:14.348 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.348 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.348 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.348 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.348 INFO:teuthology.orchestra.run.smithi097.stdout: "smtp_from_name": { 2024-03-27T11:08:14.348 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "smtp_from_name", 2024-03-27T11:08:14.348 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.348 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.349 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.349 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "Ceph", 2024-03-27T11:08:14.349 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.349 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.349 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.349 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Email From: name", 2024-03-27T11:08:14.349 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.349 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.349 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.349 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.349 INFO:teuthology.orchestra.run.smithi097.stdout: "smtp_host": { 2024-03-27T11:08:14.349 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "smtp_host", 2024-03-27T11:08:14.349 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.349 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.349 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.350 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.350 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.350 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.350 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.350 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "SMTP server", 2024-03-27T11:08:14.350 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.350 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.350 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.350 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.350 INFO:teuthology.orchestra.run.smithi097.stdout: "smtp_password": { 2024-03-27T11:08:14.350 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "smtp_password", 2024-03-27T11:08:14.350 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.350 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.351 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.351 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.351 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.351 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.351 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.351 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Password to authenticate with", 2024-03-27T11:08:14.351 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.351 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.351 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.351 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.351 INFO:teuthology.orchestra.run.smithi097.stdout: "smtp_port": { 2024-03-27T11:08:14.351 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "smtp_port", 2024-03-27T11:08:14.351 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:14.351 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.351 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.352 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "465", 2024-03-27T11:08:14.352 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.352 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.352 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.352 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "SMTP port", 2024-03-27T11:08:14.352 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.352 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.352 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.352 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.352 INFO:teuthology.orchestra.run.smithi097.stdout: "smtp_sender": { 2024-03-27T11:08:14.352 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "smtp_sender", 2024-03-27T11:08:14.352 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.352 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.352 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.353 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.353 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.353 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.353 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.353 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "SMTP envelope sender", 2024-03-27T11:08:14.353 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.353 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.353 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.353 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.353 INFO:teuthology.orchestra.run.smithi097.stdout: "smtp_ssl": { 2024-03-27T11:08:14.353 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "smtp_ssl", 2024-03-27T11:08:14.353 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.353 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.353 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.354 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:14.354 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.354 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.354 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.354 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-27T11:08:14.354 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.354 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.354 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.354 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.354 INFO:teuthology.orchestra.run.smithi097.stdout: "smtp_user": { 2024-03-27T11:08:14.354 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "smtp_user", 2024-03-27T11:08:14.354 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.354 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.355 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.355 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.355 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.355 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.355 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.355 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "User to authenticate as", 2024-03-27T11:08:14.355 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.355 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.355 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.355 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.355 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.355 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.355 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:14.355 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "balancer", 2024-03-27T11:08:14.356 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-27T11:08:14.356 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-27T11:08:14.356 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-27T11:08:14.356 INFO:teuthology.orchestra.run.smithi097.stdout: "active": { 2024-03-27T11:08:14.356 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "active", 2024-03-27T11:08:14.356 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.356 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.356 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.356 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:14.356 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.356 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.356 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.356 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "automatically balance PGs across cluster", 2024-03-27T11:08:14.356 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.356 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.357 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.357 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.357 INFO:teuthology.orchestra.run.smithi097.stdout: "begin_time": { 2024-03-27T11:08:14.357 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "begin_time", 2024-03-27T11:08:14.357 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.357 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.357 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.357 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "0000", 2024-03-27T11:08:14.357 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.357 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.357 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.357 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "beginning time of day to automatically balance", 2024-03-27T11:08:14.357 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-27T11:08:14.357 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.357 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.357 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.358 INFO:teuthology.orchestra.run.smithi097.stdout: "begin_weekday": { 2024-03-27T11:08:14.358 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "begin_weekday", 2024-03-27T11:08:14.358 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "uint", 2024-03-27T11:08:14.358 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.358 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.358 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "0", 2024-03-27T11:08:14.358 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "0", 2024-03-27T11:08:14.358 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "6", 2024-03-27T11:08:14.358 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.358 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-27T11:08:14.358 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-27T11:08:14.358 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.358 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.359 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.359 INFO:teuthology.orchestra.run.smithi097.stdout: "crush_compat_max_iterations": { 2024-03-27T11:08:14.359 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "crush_compat_max_iterations", 2024-03-27T11:08:14.359 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "uint", 2024-03-27T11:08:14.359 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.359 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.359 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "25", 2024-03-27T11:08:14.359 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "1", 2024-03-27T11:08:14.359 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "250", 2024-03-27T11:08:14.359 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.359 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-27T11:08:14.359 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.359 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.359 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.359 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.360 INFO:teuthology.orchestra.run.smithi097.stdout: "crush_compat_metrics": { 2024-03-27T11:08:14.360 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "crush_compat_metrics", 2024-03-27T11:08:14.360 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.360 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.360 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.360 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "pgs,objects,bytes", 2024-03-27T11:08:14.360 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.360 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.360 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.360 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-27T11:08:14.360 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-27T11:08:14.360 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.360 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.361 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.361 INFO:teuthology.orchestra.run.smithi097.stdout: "crush_compat_step": { 2024-03-27T11:08:14.361 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "crush_compat_step", 2024-03-27T11:08:14.361 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "float", 2024-03-27T11:08:14.361 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.361 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.361 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "0.5", 2024-03-27T11:08:14.361 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "0.001", 2024-03-27T11:08:14.361 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "0.999", 2024-03-27T11:08:14.361 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.361 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "aggressiveness of optimization", 2024-03-27T11:08:14.361 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-27T11:08:14.361 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.361 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.361 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.362 INFO:teuthology.orchestra.run.smithi097.stdout: "end_time": { 2024-03-27T11:08:14.362 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "end_time", 2024-03-27T11:08:14.362 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.362 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.362 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.362 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "2359", 2024-03-27T11:08:14.362 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.362 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.362 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.362 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "ending time of day to automatically balance", 2024-03-27T11:08:14.362 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-27T11:08:14.362 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.363 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.363 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.363 INFO:teuthology.orchestra.run.smithi097.stdout: "end_weekday": { 2024-03-27T11:08:14.363 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "end_weekday", 2024-03-27T11:08:14.363 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "uint", 2024-03-27T11:08:14.363 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.363 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.363 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "0", 2024-03-27T11:08:14.363 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "0", 2024-03-27T11:08:14.363 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "6", 2024-03-27T11:08:14.363 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.363 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-27T11:08:14.363 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-27T11:08:14.363 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.364 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.364 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.364 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-27T11:08:14.364 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-27T11:08:14.364 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.364 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.364 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.364 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.364 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.364 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.364 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.364 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.364 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.364 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.365 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.365 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.365 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.365 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.365 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.365 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.365 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.365 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.365 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.365 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-27T11:08:14.365 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-27T11:08:14.365 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.365 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.365 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.365 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.366 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.366 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.366 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.366 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.366 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.366 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.366 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.366 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.366 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-27T11:08:14.366 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-27T11:08:14.366 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.366 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.366 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.366 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-27T11:08:14.366 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.367 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.367 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.367 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.367 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.367 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.367 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.367 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.367 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.367 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.367 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.367 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.367 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.367 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.367 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.367 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-27T11:08:14.367 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-27T11:08:14.368 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.368 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.368 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.368 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.368 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.368 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.368 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.368 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.368 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.368 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.368 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.368 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.368 INFO:teuthology.orchestra.run.smithi097.stdout: "min_score": { 2024-03-27T11:08:14.368 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "min_score", 2024-03-27T11:08:14.369 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "float", 2024-03-27T11:08:14.369 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.369 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.369 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "0", 2024-03-27T11:08:14.369 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.369 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.369 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.369 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-27T11:08:14.369 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.369 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.369 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.369 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.369 INFO:teuthology.orchestra.run.smithi097.stdout: "mode": { 2024-03-27T11:08:14.369 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "mode", 2024-03-27T11:08:14.369 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.370 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.370 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.370 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "upmap", 2024-03-27T11:08:14.370 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.370 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.370 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.370 INFO:teuthology.orchestra.run.smithi097.stdout: "crush-compat", 2024-03-27T11:08:14.370 INFO:teuthology.orchestra.run.smithi097.stdout: "none", 2024-03-27T11:08:14.370 INFO:teuthology.orchestra.run.smithi097.stdout: "read", 2024-03-27T11:08:14.370 INFO:teuthology.orchestra.run.smithi097.stdout: "upmap", 2024-03-27T11:08:14.370 INFO:teuthology.orchestra.run.smithi097.stdout: "upmap-read" 2024-03-27T11:08:14.370 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.371 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Balancer mode", 2024-03-27T11:08:14.371 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.371 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.371 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.371 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.371 INFO:teuthology.orchestra.run.smithi097.stdout: "pool_ids": { 2024-03-27T11:08:14.371 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "pool_ids", 2024-03-27T11:08:14.371 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.371 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.371 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.371 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.371 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.371 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.371 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.372 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-27T11:08:14.372 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.372 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.372 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.372 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.372 INFO:teuthology.orchestra.run.smithi097.stdout: "sleep_interval": { 2024-03-27T11:08:14.372 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "sleep_interval", 2024-03-27T11:08:14.372 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-27T11:08:14.372 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.372 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.372 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "60", 2024-03-27T11:08:14.372 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.372 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.372 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.373 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-27T11:08:14.373 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.373 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.373 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.373 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.373 INFO:teuthology.orchestra.run.smithi097.stdout: "upmap_max_deviation": { 2024-03-27T11:08:14.373 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "upmap_max_deviation", 2024-03-27T11:08:14.373 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:14.373 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.373 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.373 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "5", 2024-03-27T11:08:14.373 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "1", 2024-03-27T11:08:14.373 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.373 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.374 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-27T11:08:14.374 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-27T11:08:14.374 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.374 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.374 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.374 INFO:teuthology.orchestra.run.smithi097.stdout: "upmap_max_optimizations": { 2024-03-27T11:08:14.374 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "upmap_max_optimizations", 2024-03-27T11:08:14.374 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "uint", 2024-03-27T11:08:14.374 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.374 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.374 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10", 2024-03-27T11:08:14.374 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.374 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.375 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.375 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-27T11:08:14.375 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.375 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.375 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.375 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.375 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.375 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.375 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:14.375 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "cephadm", 2024-03-27T11:08:14.375 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-27T11:08:14.375 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-27T11:08:14.375 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-27T11:08:14.375 INFO:teuthology.orchestra.run.smithi097.stdout: "agent_down_multiplier": { 2024-03-27T11:08:14.376 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "agent_down_multiplier", 2024-03-27T11:08:14.376 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "float", 2024-03-27T11:08:14.376 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.376 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.376 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "3.0", 2024-03-27T11:08:14.376 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.376 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.376 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.376 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-27T11:08:14.376 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.376 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.376 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.376 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.376 INFO:teuthology.orchestra.run.smithi097.stdout: "agent_refresh_rate": { 2024-03-27T11:08:14.377 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "agent_refresh_rate", 2024-03-27T11:08:14.377 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-27T11:08:14.377 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.377 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.377 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "20", 2024-03-27T11:08:14.377 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.377 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.377 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.377 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-27T11:08:14.377 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.377 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.377 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.377 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.377 INFO:teuthology.orchestra.run.smithi097.stdout: "agent_starting_port": { 2024-03-27T11:08:14.378 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "agent_starting_port", 2024-03-27T11:08:14.378 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:14.378 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.378 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.378 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "4721", 2024-03-27T11:08:14.378 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.378 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.378 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.378 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-27T11:08:14.378 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.378 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.378 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.378 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.378 INFO:teuthology.orchestra.run.smithi097.stdout: "allow_ptrace": { 2024-03-27T11:08:14.378 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "allow_ptrace", 2024-03-27T11:08:14.379 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.379 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.379 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.379 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.379 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.379 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.379 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.379 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-27T11:08:14.379 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-27T11:08:14.379 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.379 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.379 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.379 INFO:teuthology.orchestra.run.smithi097.stdout: "autotune_interval": { 2024-03-27T11:08:14.379 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "autotune_interval", 2024-03-27T11:08:14.379 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-27T11:08:14.380 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.380 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.380 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "600", 2024-03-27T11:08:14.380 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.380 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.380 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.380 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-27T11:08:14.380 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.380 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.380 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.380 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.380 INFO:teuthology.orchestra.run.smithi097.stdout: "autotune_memory_target_ratio": { 2024-03-27T11:08:14.380 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "autotune_memory_target_ratio", 2024-03-27T11:08:14.381 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "float", 2024-03-27T11:08:14.381 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.381 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.381 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "0.7", 2024-03-27T11:08:14.381 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.381 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.381 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.381 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-27T11:08:14.381 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.381 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.381 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.381 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.381 INFO:teuthology.orchestra.run.smithi097.stdout: "cephadm_log_destination": { 2024-03-27T11:08:14.381 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "cephadm_log_destination", 2024-03-27T11:08:14.381 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.382 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.382 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.382 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.382 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.382 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.382 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.382 INFO:teuthology.orchestra.run.smithi097.stdout: "file", 2024-03-27T11:08:14.382 INFO:teuthology.orchestra.run.smithi097.stdout: "file,syslog", 2024-03-27T11:08:14.382 INFO:teuthology.orchestra.run.smithi097.stdout: "syslog" 2024-03-27T11:08:14.382 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.382 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-27T11:08:14.382 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.382 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.382 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.383 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.383 INFO:teuthology.orchestra.run.smithi097.stdout: "cgroups_split": { 2024-03-27T11:08:14.383 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "cgroups_split", 2024-03-27T11:08:14.383 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.383 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.383 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.383 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:14.383 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.383 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.383 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.383 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-27T11:08:14.383 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.383 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.383 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.384 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.384 INFO:teuthology.orchestra.run.smithi097.stdout: "config_checks_enabled": { 2024-03-27T11:08:14.384 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "config_checks_enabled", 2024-03-27T11:08:14.384 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.384 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.384 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.384 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.384 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.384 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.384 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.384 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-27T11:08:14.384 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.384 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.384 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.385 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.385 INFO:teuthology.orchestra.run.smithi097.stdout: "config_dashboard": { 2024-03-27T11:08:14.385 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "config_dashboard", 2024-03-27T11:08:14.385 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.385 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.385 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.385 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:14.385 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.385 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.385 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.385 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-27T11:08:14.385 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.385 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.385 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.385 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.386 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_alertmanager": { 2024-03-27T11:08:14.386 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_alertmanager", 2024-03-27T11:08:14.386 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.386 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.386 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.386 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-27T11:08:14.386 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.386 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.386 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.386 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Prometheus container image", 2024-03-27T11:08:14.386 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.386 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.386 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.387 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.387 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_base": { 2024-03-27T11:08:14.387 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_base", 2024-03-27T11:08:14.387 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.387 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.387 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.387 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-27T11:08:14.387 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.387 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.387 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.387 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Container image name, without the tag", 2024-03-27T11:08:14.387 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.387 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.387 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.388 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.388 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_elasticsearch": { 2024-03-27T11:08:14.388 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_elasticsearch", 2024-03-27T11:08:14.388 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.388 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.388 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.388 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-27T11:08:14.388 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.388 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.388 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.388 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "elasticsearch container image", 2024-03-27T11:08:14.388 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.388 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.388 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.389 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.389 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_grafana": { 2024-03-27T11:08:14.389 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_grafana", 2024-03-27T11:08:14.389 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.389 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.389 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.389 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-03-27T11:08:14.389 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.389 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.389 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.389 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Prometheus container image", 2024-03-27T11:08:14.389 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.389 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.390 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.390 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.390 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_haproxy": { 2024-03-27T11:08:14.390 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_haproxy", 2024-03-27T11:08:14.390 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.390 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.390 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.390 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-27T11:08:14.390 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.390 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.390 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.390 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "HAproxy container image", 2024-03-27T11:08:14.390 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.390 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.391 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.391 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.391 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_jaeger_agent": { 2024-03-27T11:08:14.391 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_jaeger_agent", 2024-03-27T11:08:14.391 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.391 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.391 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.391 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-27T11:08:14.391 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.391 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.391 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.391 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Jaeger agent container image", 2024-03-27T11:08:14.391 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.391 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.392 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.392 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.392 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_jaeger_collector": { 2024-03-27T11:08:14.392 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_jaeger_collector", 2024-03-27T11:08:14.392 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.392 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.392 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.392 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-27T11:08:14.392 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.392 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.392 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.392 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Jaeger collector container image", 2024-03-27T11:08:14.392 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.392 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.393 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.393 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.393 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_jaeger_query": { 2024-03-27T11:08:14.393 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_jaeger_query", 2024-03-27T11:08:14.393 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.393 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.393 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.393 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-27T11:08:14.393 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.393 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.393 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.393 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Jaeger query container image", 2024-03-27T11:08:14.393 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.393 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.393 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.394 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.394 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_keepalived": { 2024-03-27T11:08:14.394 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_keepalived", 2024-03-27T11:08:14.394 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.394 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.394 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.394 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-27T11:08:14.394 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.394 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.394 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.394 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Keepalived container image", 2024-03-27T11:08:14.394 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.394 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.394 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.395 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.395 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_loki": { 2024-03-27T11:08:14.395 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_loki", 2024-03-27T11:08:14.395 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.395 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.395 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.395 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-27T11:08:14.395 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.395 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.395 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.395 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Loki container image", 2024-03-27T11:08:14.395 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.395 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.395 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.395 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.396 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_node_exporter": { 2024-03-27T11:08:14.396 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_node_exporter", 2024-03-27T11:08:14.396 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.396 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.396 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.396 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-27T11:08:14.396 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.396 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.396 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.396 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Prometheus container image", 2024-03-27T11:08:14.396 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.396 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.396 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.396 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.396 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_nvmeof": { 2024-03-27T11:08:14.397 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_nvmeof", 2024-03-27T11:08:14.397 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.397 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.397 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.397 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-03-27T11:08:14.397 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.397 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.397 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.397 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Nvme-of container image", 2024-03-27T11:08:14.397 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.397 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.397 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.397 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.398 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_prometheus": { 2024-03-27T11:08:14.398 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_prometheus", 2024-03-27T11:08:14.398 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.398 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.398 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.398 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-27T11:08:14.398 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.398 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.398 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.398 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Prometheus container image", 2024-03-27T11:08:14.398 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.398 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.399 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.399 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.399 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_promtail": { 2024-03-27T11:08:14.399 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_promtail", 2024-03-27T11:08:14.399 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.399 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.399 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.399 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-27T11:08:14.399 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.399 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.399 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.400 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Promtail container image", 2024-03-27T11:08:14.400 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.400 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.400 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.400 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.400 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_samba": { 2024-03-27T11:08:14.400 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_samba", 2024-03-27T11:08:14.400 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.400 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.400 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.400 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-03-27T11:08:14.400 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.400 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.401 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.401 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Samba/SMB container image", 2024-03-27T11:08:14.401 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.401 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.401 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.401 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.401 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_snmp_gateway": { 2024-03-27T11:08:14.401 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_snmp_gateway", 2024-03-27T11:08:14.401 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.401 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.401 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.401 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-27T11:08:14.402 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.402 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.402 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.402 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "SNMP Gateway container image", 2024-03-27T11:08:14.402 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.402 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.402 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.402 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.402 INFO:teuthology.orchestra.run.smithi097.stdout: "container_init": { 2024-03-27T11:08:14.402 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_init", 2024-03-27T11:08:14.402 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.402 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.403 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.403 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:14.403 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.403 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.403 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.403 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Run podman/docker with `--init`", 2024-03-27T11:08:14.403 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.403 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.403 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.403 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.403 INFO:teuthology.orchestra.run.smithi097.stdout: "daemon_cache_timeout": { 2024-03-27T11:08:14.403 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "daemon_cache_timeout", 2024-03-27T11:08:14.403 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-27T11:08:14.404 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.404 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.404 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "600", 2024-03-27T11:08:14.404 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.404 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.404 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.404 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-27T11:08:14.404 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.404 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.404 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.404 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.404 INFO:teuthology.orchestra.run.smithi097.stdout: "default_cephadm_command_timeout": { 2024-03-27T11:08:14.404 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "default_cephadm_command_timeout", 2024-03-27T11:08:14.405 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:14.405 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.405 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.405 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "900", 2024-03-27T11:08:14.405 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.405 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.405 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.405 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-27T11:08:14.405 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.405 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.405 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.405 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.406 INFO:teuthology.orchestra.run.smithi097.stdout: "default_registry": { 2024-03-27T11:08:14.406 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "default_registry", 2024-03-27T11:08:14.406 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.406 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.406 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.406 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "docker.io", 2024-03-27T11:08:14.406 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.406 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.406 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.406 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-27T11:08:14.406 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.406 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.406 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.407 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.407 INFO:teuthology.orchestra.run.smithi097.stdout: "device_cache_timeout": { 2024-03-27T11:08:14.407 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "device_cache_timeout", 2024-03-27T11:08:14.407 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-27T11:08:14.407 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.407 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.407 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "1800", 2024-03-27T11:08:14.407 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.407 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.407 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.407 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "seconds to cache device inventory", 2024-03-27T11:08:14.407 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.407 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.408 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.408 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.408 INFO:teuthology.orchestra.run.smithi097.stdout: "device_enhanced_scan": { 2024-03-27T11:08:14.408 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "device_enhanced_scan", 2024-03-27T11:08:14.408 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.408 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.408 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.408 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.408 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.408 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.408 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.408 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-27T11:08:14.408 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.408 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.409 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.409 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.409 INFO:teuthology.orchestra.run.smithi097.stdout: "facts_cache_timeout": { 2024-03-27T11:08:14.409 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "facts_cache_timeout", 2024-03-27T11:08:14.409 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-27T11:08:14.409 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.409 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.409 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "60", 2024-03-27T11:08:14.409 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.409 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.409 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.409 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "seconds to cache host facts data", 2024-03-27T11:08:14.409 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.409 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.410 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.410 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.410 INFO:teuthology.orchestra.run.smithi097.stdout: "grafana_dashboards_path": { 2024-03-27T11:08:14.410 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "grafana_dashboards_path", 2024-03-27T11:08:14.410 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.410 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.410 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.410 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-03-27T11:08:14.410 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.410 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.410 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.410 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-03-27T11:08:14.411 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.411 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.411 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.411 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.411 INFO:teuthology.orchestra.run.smithi097.stdout: "host_check_interval": { 2024-03-27T11:08:14.411 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "host_check_interval", 2024-03-27T11:08:14.411 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-27T11:08:14.411 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.411 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.411 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "600", 2024-03-27T11:08:14.411 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.411 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.411 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.411 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "how frequently to perform a host check", 2024-03-27T11:08:14.412 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.412 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.412 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.412 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.412 INFO:teuthology.orchestra.run.smithi097.stdout: "hw_monitoring": { 2024-03-27T11:08:14.412 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "hw_monitoring", 2024-03-27T11:08:14.412 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.412 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.412 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.412 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.412 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.412 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.412 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.412 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-27T11:08:14.412 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.413 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.413 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.413 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.413 INFO:teuthology.orchestra.run.smithi097.stdout: "inventory_list_all": { 2024-03-27T11:08:14.413 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "inventory_list_all", 2024-03-27T11:08:14.413 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.413 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.413 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.413 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.413 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.413 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.413 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.414 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-27T11:08:14.414 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.414 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.414 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.414 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.414 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-27T11:08:14.414 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-27T11:08:14.414 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.414 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.414 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.414 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.414 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.414 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.415 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.415 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.415 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.415 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.415 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.415 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.415 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.415 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.415 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.415 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.415 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.415 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.415 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.415 INFO:teuthology.orchestra.run.smithi097.stdout: "log_refresh_metadata": { 2024-03-27T11:08:14.416 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_refresh_metadata", 2024-03-27T11:08:14.416 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.416 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.416 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.416 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.416 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.416 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.416 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.416 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-27T11:08:14.416 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.416 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.416 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.416 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.416 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-27T11:08:14.416 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-27T11:08:14.417 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.417 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.417 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.417 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:14.417 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.417 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.417 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.417 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-27T11:08:14.417 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.417 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.417 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.417 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.417 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-27T11:08:14.417 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-27T11:08:14.418 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.418 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.418 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.418 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-27T11:08:14.418 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.418 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.418 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.418 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.418 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.418 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.418 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.418 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.418 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.419 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.419 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.419 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.419 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.419 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.419 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.419 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-27T11:08:14.419 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-27T11:08:14.419 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.419 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.419 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.419 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.419 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.419 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.419 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.420 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.420 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.420 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.420 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.420 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.420 INFO:teuthology.orchestra.run.smithi097.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-27T11:08:14.420 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-27T11:08:14.420 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.420 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.420 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.420 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.420 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.420 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.420 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.420 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-27T11:08:14.421 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.421 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.421 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.421 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.421 INFO:teuthology.orchestra.run.smithi097.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-27T11:08:14.421 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-27T11:08:14.421 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.421 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.421 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.421 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "*", 2024-03-27T11:08:14.421 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.421 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.421 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.421 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-27T11:08:14.421 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.422 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.422 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.422 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.422 INFO:teuthology.orchestra.run.smithi097.stdout: "max_count_per_host": { 2024-03-27T11:08:14.422 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "max_count_per_host", 2024-03-27T11:08:14.422 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:14.422 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.422 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.422 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10", 2024-03-27T11:08:14.422 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.422 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.422 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.422 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "max number of daemons per service per host", 2024-03-27T11:08:14.422 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.423 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.423 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.423 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.423 INFO:teuthology.orchestra.run.smithi097.stdout: "max_osd_draining_count": { 2024-03-27T11:08:14.423 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "max_osd_draining_count", 2024-03-27T11:08:14.423 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:14.423 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.423 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.423 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10", 2024-03-27T11:08:14.423 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.423 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.423 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.423 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-27T11:08:14.423 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.423 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.424 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.424 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.424 INFO:teuthology.orchestra.run.smithi097.stdout: "migration_current": { 2024-03-27T11:08:14.424 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "migration_current", 2024-03-27T11:08:14.424 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:14.424 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.424 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.424 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.424 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.424 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.424 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.424 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "internal - do not modify", 2024-03-27T11:08:14.424 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.424 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.424 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.425 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.425 INFO:teuthology.orchestra.run.smithi097.stdout: "mode": { 2024-03-27T11:08:14.425 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "mode", 2024-03-27T11:08:14.425 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.425 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.425 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.425 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "root", 2024-03-27T11:08:14.425 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.425 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.425 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.425 INFO:teuthology.orchestra.run.smithi097.stdout: "cephadm-package", 2024-03-27T11:08:14.425 INFO:teuthology.orchestra.run.smithi097.stdout: "root" 2024-03-27T11:08:14.425 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.425 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "mode for remote execution of cephadm", 2024-03-27T11:08:14.425 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.426 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.426 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.426 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.426 INFO:teuthology.orchestra.run.smithi097.stdout: "oob_default_addr": { 2024-03-27T11:08:14.426 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "oob_default_addr", 2024-03-27T11:08:14.426 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.426 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.426 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.426 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "169.254.1.1", 2024-03-27T11:08:14.426 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.426 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.426 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.426 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-27T11:08:14.426 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.426 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.427 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.427 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.427 INFO:teuthology.orchestra.run.smithi097.stdout: "prometheus_alerts_path": { 2024-03-27T11:08:14.427 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "prometheus_alerts_path", 2024-03-27T11:08:14.427 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.427 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.427 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.427 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-27T11:08:14.427 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.427 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.427 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.427 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-27T11:08:14.427 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.427 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.427 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.428 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.428 INFO:teuthology.orchestra.run.smithi097.stdout: "registry_insecure": { 2024-03-27T11:08:14.428 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "registry_insecure", 2024-03-27T11:08:14.428 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.428 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.428 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.428 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.428 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.428 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.428 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.428 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-27T11:08:14.428 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.428 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.428 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.428 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.429 INFO:teuthology.orchestra.run.smithi097.stdout: "registry_password": { 2024-03-27T11:08:14.429 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "registry_password", 2024-03-27T11:08:14.429 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.429 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.429 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.429 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.429 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.429 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.429 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.429 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-27T11:08:14.429 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.429 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.429 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.429 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.429 INFO:teuthology.orchestra.run.smithi097.stdout: "registry_url": { 2024-03-27T11:08:14.430 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "registry_url", 2024-03-27T11:08:14.430 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.430 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.430 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.430 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.430 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.430 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.430 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.430 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-27T11:08:14.430 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.430 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.430 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.430 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.430 INFO:teuthology.orchestra.run.smithi097.stdout: "registry_username": { 2024-03-27T11:08:14.431 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "registry_username", 2024-03-27T11:08:14.431 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.431 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.431 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.431 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.431 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.431 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.431 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.431 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-27T11:08:14.431 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.431 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.431 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.431 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.431 INFO:teuthology.orchestra.run.smithi097.stdout: "secure_monitoring_stack": { 2024-03-27T11:08:14.431 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "secure_monitoring_stack", 2024-03-27T11:08:14.431 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.432 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.432 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.432 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.432 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.432 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.432 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.432 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-27T11:08:14.432 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.432 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.432 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.432 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.432 INFO:teuthology.orchestra.run.smithi097.stdout: "service_discovery_port": { 2024-03-27T11:08:14.432 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "service_discovery_port", 2024-03-27T11:08:14.432 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:14.433 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.433 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.433 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "8765", 2024-03-27T11:08:14.433 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.433 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.433 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.433 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "cephadm service discovery port", 2024-03-27T11:08:14.433 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.433 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.433 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.433 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.433 INFO:teuthology.orchestra.run.smithi097.stdout: "ssh_config_file": { 2024-03-27T11:08:14.433 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ssh_config_file", 2024-03-27T11:08:14.433 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.433 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.434 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.434 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.434 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.434 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.434 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.434 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-27T11:08:14.434 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.434 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.434 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.434 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.434 INFO:teuthology.orchestra.run.smithi097.stdout: "use_agent": { 2024-03-27T11:08:14.434 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "use_agent", 2024-03-27T11:08:14.434 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.434 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.434 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.435 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.435 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.435 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.435 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.435 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-27T11:08:14.435 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.435 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.435 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.435 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.435 INFO:teuthology.orchestra.run.smithi097.stdout: "use_repo_digest": { 2024-03-27T11:08:14.435 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "use_repo_digest", 2024-03-27T11:08:14.435 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.435 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.435 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.436 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:14.436 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.436 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.436 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.436 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-27T11:08:14.436 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.436 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.436 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.436 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.436 INFO:teuthology.orchestra.run.smithi097.stdout: "warn_on_failed_host_check": { 2024-03-27T11:08:14.436 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "warn_on_failed_host_check", 2024-03-27T11:08:14.436 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.436 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.436 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.437 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:14.437 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.437 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.437 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.437 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "raise a health warning if the host check fails", 2024-03-27T11:08:14.437 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.437 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.437 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.437 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.437 INFO:teuthology.orchestra.run.smithi097.stdout: "warn_on_stray_daemons": { 2024-03-27T11:08:14.437 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "warn_on_stray_daemons", 2024-03-27T11:08:14.437 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.437 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.437 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.437 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:14.438 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.438 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.438 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.438 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-27T11:08:14.438 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.438 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.438 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.438 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.438 INFO:teuthology.orchestra.run.smithi097.stdout: "warn_on_stray_hosts": { 2024-03-27T11:08:14.438 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "warn_on_stray_hosts", 2024-03-27T11:08:14.438 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.438 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.438 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.438 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:14.439 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.439 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.439 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.439 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-27T11:08:14.439 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.439 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.439 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.439 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.439 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.439 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.439 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:14.439 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "crash", 2024-03-27T11:08:14.439 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-27T11:08:14.439 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-27T11:08:14.439 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-27T11:08:14.440 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-27T11:08:14.440 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-27T11:08:14.440 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.440 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.440 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.440 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.440 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.440 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.440 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.440 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.440 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.440 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.440 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.440 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.440 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.441 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.441 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.441 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.441 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.441 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.441 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.441 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-27T11:08:14.441 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-27T11:08:14.441 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.441 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.441 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.441 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.441 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.441 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.441 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.442 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.442 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.442 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.442 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.442 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.442 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-27T11:08:14.442 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-27T11:08:14.442 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.442 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.442 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.442 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-27T11:08:14.442 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.442 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.443 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.443 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.443 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.443 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.443 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.443 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.443 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.443 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.443 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.443 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.443 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.443 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.443 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.443 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-27T11:08:14.443 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-27T11:08:14.444 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.444 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.444 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.444 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.444 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.444 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.444 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.444 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.444 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.444 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.444 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.444 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.445 INFO:teuthology.orchestra.run.smithi097.stdout: "retain_interval": { 2024-03-27T11:08:14.445 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "retain_interval", 2024-03-27T11:08:14.445 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-27T11:08:14.445 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.445 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.445 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "31536000", 2024-03-27T11:08:14.445 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.445 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.445 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.445 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-27T11:08:14.445 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.445 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.445 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.445 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.446 INFO:teuthology.orchestra.run.smithi097.stdout: "warn_recent_interval": { 2024-03-27T11:08:14.446 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "warn_recent_interval", 2024-03-27T11:08:14.446 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-27T11:08:14.446 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.446 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.446 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "1209600", 2024-03-27T11:08:14.446 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.446 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.446 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.446 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-27T11:08:14.446 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.446 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.446 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.447 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.447 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.447 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.447 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:14.447 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "dashboard", 2024-03-27T11:08:14.447 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-27T11:08:14.447 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-27T11:08:14.447 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-27T11:08:14.447 INFO:teuthology.orchestra.run.smithi097.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-27T11:08:14.447 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-27T11:08:14.447 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:14.447 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.447 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.447 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10", 2024-03-27T11:08:14.447 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.448 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.448 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.448 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.448 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.448 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.448 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.448 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.448 INFO:teuthology.orchestra.run.smithi097.stdout: "ALERTMANAGER_API_HOST": { 2024-03-27T11:08:14.448 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-27T11:08:14.448 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.448 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.448 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.448 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.448 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.449 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.449 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.449 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.449 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.449 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.449 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.449 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.449 INFO:teuthology.orchestra.run.smithi097.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-27T11:08:14.449 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-27T11:08:14.449 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.449 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.449 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.449 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:14.449 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.450 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.450 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.450 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.450 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.450 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.450 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.450 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.450 INFO:teuthology.orchestra.run.smithi097.stdout: "AUDIT_API_ENABLED": { 2024-03-27T11:08:14.450 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "AUDIT_API_ENABLED", 2024-03-27T11:08:14.450 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.450 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.450 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.450 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.450 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.451 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.451 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.451 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.451 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.451 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.451 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.451 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.451 INFO:teuthology.orchestra.run.smithi097.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-27T11:08:14.451 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-27T11:08:14.451 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.451 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.451 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.451 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:14.451 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.451 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.452 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.452 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.452 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.452 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.452 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.452 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.452 INFO:teuthology.orchestra.run.smithi097.stdout: "ENABLE_BROWSABLE_API": { 2024-03-27T11:08:14.452 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-27T11:08:14.452 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.452 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.452 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.452 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:14.452 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.452 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.453 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.453 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.453 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.453 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.453 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.453 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.453 INFO:teuthology.orchestra.run.smithi097.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-27T11:08:14.453 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-27T11:08:14.453 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.453 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.453 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.453 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:14.453 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.454 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.454 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.454 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.454 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.454 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.454 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.454 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.454 INFO:teuthology.orchestra.run.smithi097.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-27T11:08:14.454 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-27T11:08:14.454 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.454 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.454 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.454 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:14.454 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.455 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.455 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.455 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.455 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.455 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.455 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.455 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.455 INFO:teuthology.orchestra.run.smithi097.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-27T11:08:14.455 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-27T11:08:14.455 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.455 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.455 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.455 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:14.455 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.455 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.455 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.456 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.456 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.456 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.456 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.456 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.456 INFO:teuthology.orchestra.run.smithi097.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-27T11:08:14.456 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-27T11:08:14.456 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.456 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.456 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.456 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:14.456 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.456 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.456 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.456 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.457 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.457 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.457 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.457 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.457 INFO:teuthology.orchestra.run.smithi097.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-27T11:08:14.457 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-27T11:08:14.457 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.457 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.457 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.457 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:14.457 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.457 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.457 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.457 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.457 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.457 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.458 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.458 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.458 INFO:teuthology.orchestra.run.smithi097.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-27T11:08:14.458 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-27T11:08:14.458 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.458 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.458 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.458 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:14.458 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.458 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.458 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.458 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.458 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.458 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.459 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.459 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.459 INFO:teuthology.orchestra.run.smithi097.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-27T11:08:14.459 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-27T11:08:14.459 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.459 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.459 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.459 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:14.459 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.459 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.459 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.459 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.459 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.459 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.459 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.460 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.460 INFO:teuthology.orchestra.run.smithi097.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-27T11:08:14.460 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-27T11:08:14.460 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.460 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.460 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.460 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.460 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.460 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.460 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.460 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.460 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.460 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.461 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.461 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.461 INFO:teuthology.orchestra.run.smithi097.stdout: "GRAFANA_API_PASSWORD": { 2024-03-27T11:08:14.461 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-27T11:08:14.461 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.461 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.461 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.461 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "admin", 2024-03-27T11:08:14.461 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.461 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.461 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.461 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.461 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.462 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.462 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.462 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.462 INFO:teuthology.orchestra.run.smithi097.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-27T11:08:14.462 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-27T11:08:14.462 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.462 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.462 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.462 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:14.462 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.462 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.462 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.462 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.462 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.463 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.463 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.463 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.463 INFO:teuthology.orchestra.run.smithi097.stdout: "GRAFANA_API_URL": { 2024-03-27T11:08:14.463 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "GRAFANA_API_URL", 2024-03-27T11:08:14.463 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.463 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.463 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.463 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.463 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.463 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.463 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.463 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.463 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.464 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.464 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.464 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.464 INFO:teuthology.orchestra.run.smithi097.stdout: "GRAFANA_API_USERNAME": { 2024-03-27T11:08:14.464 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-27T11:08:14.464 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.464 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.464 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.464 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "admin", 2024-03-27T11:08:14.464 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.464 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.464 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.464 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.464 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.464 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.465 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.465 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.465 INFO:teuthology.orchestra.run.smithi097.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-27T11:08:14.465 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-27T11:08:14.465 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.465 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.465 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.465 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.465 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.465 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.465 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.465 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.465 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.466 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.466 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.466 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.466 INFO:teuthology.orchestra.run.smithi097.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-27T11:08:14.466 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-27T11:08:14.466 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.466 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.466 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.466 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.466 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.466 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.466 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.466 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.466 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.467 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.467 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.467 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.467 INFO:teuthology.orchestra.run.smithi097.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-27T11:08:14.467 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-27T11:08:14.467 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.467 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.467 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.467 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:14.467 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.467 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.467 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.467 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.467 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.467 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.468 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.468 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.468 INFO:teuthology.orchestra.run.smithi097.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-27T11:08:14.468 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-27T11:08:14.468 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.468 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.468 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.468 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.468 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.468 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.468 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.468 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.468 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.468 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.468 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.468 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.469 INFO:teuthology.orchestra.run.smithi097.stdout: "MULTICLUSTER_CONFIG": { 2024-03-27T11:08:14.469 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "MULTICLUSTER_CONFIG", 2024-03-27T11:08:14.469 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.469 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.469 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.469 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "{}", 2024-03-27T11:08:14.469 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.469 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.469 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.469 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.469 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.469 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.469 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.469 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.469 INFO:teuthology.orchestra.run.smithi097.stdout: "PROMETHEUS_API_HOST": { 2024-03-27T11:08:14.470 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-27T11:08:14.470 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.470 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.470 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.470 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.470 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.470 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.470 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.470 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.470 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.470 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.470 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.470 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.470 INFO:teuthology.orchestra.run.smithi097.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-27T11:08:14.471 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-27T11:08:14.471 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.471 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.471 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.471 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:14.471 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.471 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.471 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.471 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.471 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.471 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.471 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.471 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.472 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-27T11:08:14.472 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-27T11:08:14.472 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.472 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.472 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.472 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.472 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.472 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.473 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.473 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.473 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.473 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.473 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.473 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.473 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-27T11:08:14.473 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-27T11:08:14.474 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.474 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.474 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.474 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.474 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.474 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.474 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.474 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.474 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.474 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.474 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.474 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.475 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-27T11:08:14.475 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-27T11:08:14.475 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.475 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.475 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.475 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:14.475 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.475 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.475 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.475 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.475 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.475 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.475 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.476 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.476 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-27T11:08:14.476 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-27T11:08:14.476 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.476 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.476 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.476 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:14.476 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.476 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.476 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.476 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.476 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.476 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.476 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.477 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.477 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-27T11:08:14.477 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-27T11:08:14.477 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.477 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.477 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.477 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.477 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.477 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.477 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.477 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.477 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.477 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.477 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.478 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.478 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-27T11:08:14.478 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-27T11:08:14.478 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.478 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.478 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.478 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.478 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.478 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.478 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.478 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.478 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.478 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.478 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.479 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.479 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-27T11:08:14.479 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-27T11:08:14.479 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.479 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.479 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.479 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.479 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.479 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.479 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.479 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.479 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.479 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.479 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.479 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.480 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_ENABLED": { 2024-03-27T11:08:14.480 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-27T11:08:14.480 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.480 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.480 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.480 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:14.480 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.480 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.480 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.480 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.480 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.480 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.480 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.480 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.481 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-27T11:08:14.481 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-27T11:08:14.481 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.481 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.481 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.481 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-27T11:08:14.481 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.481 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.481 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.481 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.481 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.481 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.481 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.481 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.482 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-27T11:08:14.482 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-27T11:08:14.482 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:14.482 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.482 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.482 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10", 2024-03-27T11:08:14.482 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.482 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.482 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.482 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.482 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.482 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.483 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.483 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.483 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-27T11:08:14.483 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-27T11:08:14.483 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:14.483 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.483 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.483 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "8", 2024-03-27T11:08:14.483 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.483 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.484 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.484 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.484 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.484 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.484 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.484 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.484 INFO:teuthology.orchestra.run.smithi097.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-27T11:08:14.484 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-27T11:08:14.484 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:14.484 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.484 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.484 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "45", 2024-03-27T11:08:14.484 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.485 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.485 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.485 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.485 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.485 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.485 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.485 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.485 INFO:teuthology.orchestra.run.smithi097.stdout: "RGW_API_ACCESS_KEY": { 2024-03-27T11:08:14.485 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-27T11:08:14.485 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.485 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.485 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.485 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.485 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.486 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.486 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.486 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.486 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.486 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.486 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.486 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.486 INFO:teuthology.orchestra.run.smithi097.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-27T11:08:14.486 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-27T11:08:14.486 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.486 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.486 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.487 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "admin", 2024-03-27T11:08:14.487 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.487 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.487 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.487 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.487 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.487 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.487 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.487 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.487 INFO:teuthology.orchestra.run.smithi097.stdout: "RGW_API_SECRET_KEY": { 2024-03-27T11:08:14.487 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-27T11:08:14.487 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.487 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.488 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.488 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.488 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.488 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.488 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.488 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.488 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.488 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.488 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.488 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.488 INFO:teuthology.orchestra.run.smithi097.stdout: "RGW_API_SSL_VERIFY": { 2024-03-27T11:08:14.488 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-27T11:08:14.488 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.488 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.489 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.489 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:14.489 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.489 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.489 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.489 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.489 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.489 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.489 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.489 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.489 INFO:teuthology.orchestra.run.smithi097.stdout: "UNSAFE_TLS_v1_2": { 2024-03-27T11:08:14.489 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "UNSAFE_TLS_v1_2", 2024-03-27T11:08:14.489 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.489 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.490 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.490 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.490 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.490 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.490 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.490 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.490 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.490 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.490 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.490 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.490 INFO:teuthology.orchestra.run.smithi097.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-27T11:08:14.490 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-27T11:08:14.490 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:14.491 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.491 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.491 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "0", 2024-03-27T11:08:14.491 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.491 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.491 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.491 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.491 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.491 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.491 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.491 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.491 INFO:teuthology.orchestra.run.smithi097.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-27T11:08:14.491 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-27T11:08:14.491 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:14.491 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.491 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.492 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10", 2024-03-27T11:08:14.492 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.492 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.492 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.492 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.492 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.492 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.492 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.492 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.492 INFO:teuthology.orchestra.run.smithi097.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-27T11:08:14.492 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-27T11:08:14.492 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:14.492 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.493 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.493 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "5", 2024-03-27T11:08:14.493 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.493 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.493 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.493 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.493 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.493 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.493 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.493 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.493 INFO:teuthology.orchestra.run.smithi097.stdout: "cross_origin_url": { 2024-03-27T11:08:14.493 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "cross_origin_url", 2024-03-27T11:08:14.493 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.494 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.494 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.494 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.494 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.494 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.494 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.494 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.494 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.494 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.494 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.494 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.494 INFO:teuthology.orchestra.run.smithi097.stdout: "crt_file": { 2024-03-27T11:08:14.494 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "crt_file", 2024-03-27T11:08:14.495 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.495 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.495 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.495 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.495 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.495 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.495 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.495 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.495 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.495 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.495 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.495 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.495 INFO:teuthology.orchestra.run.smithi097.stdout: "debug": { 2024-03-27T11:08:14.495 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "debug", 2024-03-27T11:08:14.496 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.496 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.496 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.496 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.496 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.496 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.496 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.496 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Enable/disable debug options", 2024-03-27T11:08:14.496 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.496 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.496 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.496 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.496 INFO:teuthology.orchestra.run.smithi097.stdout: "jwt_token_ttl": { 2024-03-27T11:08:14.497 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "jwt_token_ttl", 2024-03-27T11:08:14.497 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:14.497 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.497 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.497 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "28800", 2024-03-27T11:08:14.497 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.497 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.497 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.497 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.497 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.497 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.497 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.497 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.497 INFO:teuthology.orchestra.run.smithi097.stdout: "key_file": { 2024-03-27T11:08:14.497 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "key_file", 2024-03-27T11:08:14.498 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.498 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.498 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.498 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.498 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.498 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.498 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.498 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.498 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.498 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.498 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.498 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.498 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-27T11:08:14.498 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-27T11:08:14.499 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.499 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.499 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.499 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.499 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.499 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.499 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.499 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.499 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.499 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.499 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.499 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.499 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.499 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.499 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.500 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.500 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.500 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.500 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.500 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-27T11:08:14.500 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-27T11:08:14.500 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.500 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.500 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.500 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.500 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.500 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.500 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.500 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.500 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.501 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.501 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.501 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.501 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-27T11:08:14.501 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-27T11:08:14.501 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.501 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.501 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.501 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-27T11:08:14.501 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.501 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.501 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.501 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.501 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.501 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.502 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.502 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.502 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.502 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.502 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.502 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.502 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.502 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.502 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.502 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-27T11:08:14.502 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-27T11:08:14.502 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.502 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.502 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.502 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.503 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.503 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.503 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.503 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.503 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.503 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.503 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.503 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.503 INFO:teuthology.orchestra.run.smithi097.stdout: "motd": { 2024-03-27T11:08:14.503 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "motd", 2024-03-27T11:08:14.503 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.503 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.503 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.504 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.504 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.504 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.504 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.504 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "The message of the day", 2024-03-27T11:08:14.504 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.504 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.504 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.504 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.504 INFO:teuthology.orchestra.run.smithi097.stdout: "redirect_resolve_ip_addr": { 2024-03-27T11:08:14.504 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "redirect_resolve_ip_addr", 2024-03-27T11:08:14.504 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.504 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.505 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.505 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.505 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.505 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.505 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.505 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.505 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.505 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.505 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.505 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.505 INFO:teuthology.orchestra.run.smithi097.stdout: "server_addr": { 2024-03-27T11:08:14.505 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "server_addr", 2024-03-27T11:08:14.505 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.505 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.506 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.506 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "::", 2024-03-27T11:08:14.506 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.506 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.506 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.506 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.506 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.506 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.506 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.506 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.506 INFO:teuthology.orchestra.run.smithi097.stdout: "server_port": { 2024-03-27T11:08:14.506 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "server_port", 2024-03-27T11:08:14.506 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:14.506 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.507 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.507 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "8080", 2024-03-27T11:08:14.507 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.507 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.507 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.507 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.507 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.507 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.507 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.507 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.507 INFO:teuthology.orchestra.run.smithi097.stdout: "ssl": { 2024-03-27T11:08:14.507 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ssl", 2024-03-27T11:08:14.507 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.507 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.507 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.508 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:14.508 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.508 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.508 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.508 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.508 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.508 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.508 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.508 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.508 INFO:teuthology.orchestra.run.smithi097.stdout: "ssl_server_port": { 2024-03-27T11:08:14.508 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ssl_server_port", 2024-03-27T11:08:14.508 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:14.508 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.508 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.509 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "8443", 2024-03-27T11:08:14.509 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.509 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.509 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.509 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.509 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.509 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.509 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.509 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.509 INFO:teuthology.orchestra.run.smithi097.stdout: "standby_behaviour": { 2024-03-27T11:08:14.509 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "standby_behaviour", 2024-03-27T11:08:14.509 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.509 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.509 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.509 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "redirect", 2024-03-27T11:08:14.510 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.510 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.510 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.510 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.510 INFO:teuthology.orchestra.run.smithi097.stdout: "redirect" 2024-03-27T11:08:14.510 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.510 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.510 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.510 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.510 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.510 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.510 INFO:teuthology.orchestra.run.smithi097.stdout: "standby_error_status_code": { 2024-03-27T11:08:14.510 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "standby_error_status_code", 2024-03-27T11:08:14.510 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:14.510 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.511 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.511 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "500", 2024-03-27T11:08:14.511 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "400", 2024-03-27T11:08:14.511 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "599", 2024-03-27T11:08:14.511 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.511 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.511 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.511 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.511 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.511 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.511 INFO:teuthology.orchestra.run.smithi097.stdout: "url_prefix": { 2024-03-27T11:08:14.511 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "url_prefix", 2024-03-27T11:08:14.511 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.511 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.512 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.512 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.512 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.512 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.512 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.512 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.512 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.512 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.512 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.512 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.512 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.512 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.512 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:14.512 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "devicehealth", 2024-03-27T11:08:14.512 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-27T11:08:14.513 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-27T11:08:14.513 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-27T11:08:14.513 INFO:teuthology.orchestra.run.smithi097.stdout: "enable_monitoring": { 2024-03-27T11:08:14.513 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "enable_monitoring", 2024-03-27T11:08:14.513 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.513 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.513 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.513 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:14.513 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.513 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.513 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.513 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "monitor device health metrics", 2024-03-27T11:08:14.513 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.513 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.514 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.514 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.514 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-27T11:08:14.514 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-27T11:08:14.514 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.514 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.514 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.514 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.514 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.514 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.514 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.514 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.514 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.515 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.515 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.515 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.515 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.515 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.515 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.515 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.515 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.515 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.515 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.515 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-27T11:08:14.515 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-27T11:08:14.515 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.515 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.516 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.516 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.516 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.516 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.516 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.516 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.516 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.516 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.516 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.516 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.516 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-27T11:08:14.516 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-27T11:08:14.516 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.516 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.516 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.517 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-27T11:08:14.517 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.517 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.517 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.517 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.517 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.517 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.517 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.517 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.517 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.517 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.517 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.517 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.517 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.517 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.518 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.518 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-27T11:08:14.518 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-27T11:08:14.518 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.518 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.518 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.518 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.518 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.518 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.518 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.518 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.518 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.518 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.518 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.518 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.519 INFO:teuthology.orchestra.run.smithi097.stdout: "mark_out_threshold": { 2024-03-27T11:08:14.519 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "mark_out_threshold", 2024-03-27T11:08:14.519 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-27T11:08:14.519 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.519 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.519 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "2419200", 2024-03-27T11:08:14.519 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.519 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.519 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.519 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-27T11:08:14.519 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.519 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.519 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.519 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.519 INFO:teuthology.orchestra.run.smithi097.stdout: "pool_name": { 2024-03-27T11:08:14.520 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "pool_name", 2024-03-27T11:08:14.520 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.520 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.520 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.520 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "device_health_metrics", 2024-03-27T11:08:14.520 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.520 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.520 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.520 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-27T11:08:14.520 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.520 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.520 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.520 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.520 INFO:teuthology.orchestra.run.smithi097.stdout: "retention_period": { 2024-03-27T11:08:14.520 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "retention_period", 2024-03-27T11:08:14.521 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-27T11:08:14.521 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.521 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.521 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "15552000", 2024-03-27T11:08:14.521 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.521 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.521 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.521 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "how long to retain device health metrics", 2024-03-27T11:08:14.521 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.521 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.521 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.521 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.521 INFO:teuthology.orchestra.run.smithi097.stdout: "scrape_frequency": { 2024-03-27T11:08:14.521 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "scrape_frequency", 2024-03-27T11:08:14.521 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-27T11:08:14.521 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.522 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.522 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "86400", 2024-03-27T11:08:14.522 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.522 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.522 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.522 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-27T11:08:14.522 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.522 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.522 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.522 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.522 INFO:teuthology.orchestra.run.smithi097.stdout: "self_heal": { 2024-03-27T11:08:14.522 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "self_heal", 2024-03-27T11:08:14.522 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.522 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.523 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.523 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:14.523 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.523 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.523 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.523 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-27T11:08:14.523 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.523 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.523 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.523 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.523 INFO:teuthology.orchestra.run.smithi097.stdout: "sleep_interval": { 2024-03-27T11:08:14.523 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "sleep_interval", 2024-03-27T11:08:14.523 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-27T11:08:14.523 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.523 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.524 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "600", 2024-03-27T11:08:14.524 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.524 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.524 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.524 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "how frequently to wake up and check device health", 2024-03-27T11:08:14.524 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.524 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.524 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.524 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.524 INFO:teuthology.orchestra.run.smithi097.stdout: "warn_threshold": { 2024-03-27T11:08:14.525 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "warn_threshold", 2024-03-27T11:08:14.525 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-27T11:08:14.525 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.525 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.525 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "7257600", 2024-03-27T11:08:14.525 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.525 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.525 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.525 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-27T11:08:14.525 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.525 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.525 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.525 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.525 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.526 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.526 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:14.526 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "diskprediction_local", 2024-03-27T11:08:14.526 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-27T11:08:14.526 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-27T11:08:14.526 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-27T11:08:14.526 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-27T11:08:14.526 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-27T11:08:14.526 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.526 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.526 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.526 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.526 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.526 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.527 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.527 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.527 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.527 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.527 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.527 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.527 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.527 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.527 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.527 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.527 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.527 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.527 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.527 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-27T11:08:14.527 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-27T11:08:14.528 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.528 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.528 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.528 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.528 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.528 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.528 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.528 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.528 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.528 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.528 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.528 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.528 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-27T11:08:14.529 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-27T11:08:14.529 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.529 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.529 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.529 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-27T11:08:14.529 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.529 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.529 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.529 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.529 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.529 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.529 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.529 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.529 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.529 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.530 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.530 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.530 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.530 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.530 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.530 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-27T11:08:14.530 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-27T11:08:14.530 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.530 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.530 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.530 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.530 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.530 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.530 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.531 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.531 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.531 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.531 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.531 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.531 INFO:teuthology.orchestra.run.smithi097.stdout: "predict_interval": { 2024-03-27T11:08:14.531 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "predict_interval", 2024-03-27T11:08:14.531 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.531 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.531 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.531 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "86400", 2024-03-27T11:08:14.531 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.531 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.531 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.532 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.532 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.532 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.532 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.532 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.532 INFO:teuthology.orchestra.run.smithi097.stdout: "predictor_model": { 2024-03-27T11:08:14.532 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "predictor_model", 2024-03-27T11:08:14.532 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.532 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.532 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.532 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "prophetstor", 2024-03-27T11:08:14.532 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.532 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.532 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.533 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.533 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.533 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.533 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.533 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.533 INFO:teuthology.orchestra.run.smithi097.stdout: "sleep_interval": { 2024-03-27T11:08:14.533 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "sleep_interval", 2024-03-27T11:08:14.533 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.533 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.533 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.533 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "600", 2024-03-27T11:08:14.533 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.533 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.533 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.533 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.534 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.534 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.534 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.534 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.534 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.534 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.534 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:14.534 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "influx", 2024-03-27T11:08:14.534 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": false, 2024-03-27T11:08:14.534 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "influxdb python module not found", 2024-03-27T11:08:14.534 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-27T11:08:14.534 INFO:teuthology.orchestra.run.smithi097.stdout: "batch_size": { 2024-03-27T11:08:14.534 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "batch_size", 2024-03-27T11:08:14.534 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:14.535 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.535 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.535 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "5000", 2024-03-27T11:08:14.535 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.535 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.535 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.535 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-27T11:08:14.535 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.535 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.535 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.535 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.535 INFO:teuthology.orchestra.run.smithi097.stdout: "database": { 2024-03-27T11:08:14.535 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "database", 2024-03-27T11:08:14.535 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.535 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.536 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.536 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "ceph", 2024-03-27T11:08:14.536 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.536 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.536 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.536 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-27T11:08:14.536 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.536 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.536 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.536 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.536 INFO:teuthology.orchestra.run.smithi097.stdout: "hostname": { 2024-03-27T11:08:14.536 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "hostname", 2024-03-27T11:08:14.536 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.536 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.536 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.536 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.537 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.537 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.537 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.537 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "InfluxDB server hostname", 2024-03-27T11:08:14.537 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.537 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.537 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.537 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.537 INFO:teuthology.orchestra.run.smithi097.stdout: "interval": { 2024-03-27T11:08:14.537 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "interval", 2024-03-27T11:08:14.537 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-27T11:08:14.537 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.537 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.538 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "30", 2024-03-27T11:08:14.538 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "5", 2024-03-27T11:08:14.538 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.538 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.538 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-27T11:08:14.538 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.538 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.538 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.538 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.538 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-27T11:08:14.538 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-27T11:08:14.538 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.538 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.538 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.539 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.539 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.539 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.539 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.539 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.539 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.539 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.539 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.539 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.539 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.539 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.539 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.539 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.539 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.539 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.539 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.540 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-27T11:08:14.540 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-27T11:08:14.540 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.540 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.540 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.540 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.540 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.540 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.540 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.540 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.540 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.540 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.540 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.540 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.541 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-27T11:08:14.541 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-27T11:08:14.541 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.541 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.541 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.541 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-27T11:08:14.541 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.541 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.541 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.541 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.541 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.541 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.541 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.541 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.542 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.542 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.542 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.542 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.542 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.542 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.542 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.542 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-27T11:08:14.542 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-27T11:08:14.542 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.542 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.542 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.542 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.543 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.543 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.543 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.543 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.543 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.543 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.543 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.543 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.543 INFO:teuthology.orchestra.run.smithi097.stdout: "password": { 2024-03-27T11:08:14.543 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "password", 2024-03-27T11:08:14.543 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.543 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.543 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.544 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.544 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.544 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.544 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.544 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "password of InfluxDB server user", 2024-03-27T11:08:14.544 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.544 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.544 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.544 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.544 INFO:teuthology.orchestra.run.smithi097.stdout: "port": { 2024-03-27T11:08:14.544 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "port", 2024-03-27T11:08:14.544 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:14.544 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.544 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.545 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "8086", 2024-03-27T11:08:14.545 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.545 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.545 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.545 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "InfluxDB server port", 2024-03-27T11:08:14.545 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.545 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.545 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.545 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.545 INFO:teuthology.orchestra.run.smithi097.stdout: "ssl": { 2024-03-27T11:08:14.545 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ssl", 2024-03-27T11:08:14.545 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.545 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.546 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.546 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "false", 2024-03-27T11:08:14.546 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.546 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.546 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.546 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-27T11:08:14.546 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.546 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.546 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.546 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.546 INFO:teuthology.orchestra.run.smithi097.stdout: "threads": { 2024-03-27T11:08:14.546 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "threads", 2024-03-27T11:08:14.546 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:14.546 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.547 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.547 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "5", 2024-03-27T11:08:14.547 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "1", 2024-03-27T11:08:14.547 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "32", 2024-03-27T11:08:14.547 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.547 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-27T11:08:14.547 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.547 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.547 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.547 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.547 INFO:teuthology.orchestra.run.smithi097.stdout: "username": { 2024-03-27T11:08:14.547 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "username", 2024-03-27T11:08:14.547 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.547 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.547 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.548 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.548 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.548 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.548 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.548 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "username of InfluxDB server user", 2024-03-27T11:08:14.548 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.548 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.548 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.548 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.548 INFO:teuthology.orchestra.run.smithi097.stdout: "verify_ssl": { 2024-03-27T11:08:14.548 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "verify_ssl", 2024-03-27T11:08:14.548 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.548 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.548 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.548 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "true", 2024-03-27T11:08:14.549 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.549 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.549 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.549 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-27T11:08:14.549 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.549 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.549 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.549 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.549 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.549 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.549 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:14.549 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "insights", 2024-03-27T11:08:14.549 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-27T11:08:14.549 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-27T11:08:14.549 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-27T11:08:14.550 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-27T11:08:14.550 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-27T11:08:14.550 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.550 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.550 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.550 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.550 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.550 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.550 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.550 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.550 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.550 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.550 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.551 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.551 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.551 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.551 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.551 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.551 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.551 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.551 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.551 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-27T11:08:14.551 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-27T11:08:14.551 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.551 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.551 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.551 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.552 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.552 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.552 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.552 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.552 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.552 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.552 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.552 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.552 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-27T11:08:14.552 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-27T11:08:14.552 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.552 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.552 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.552 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-27T11:08:14.553 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.553 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.553 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.553 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.553 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.553 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.553 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.553 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.553 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.553 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.553 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.553 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.553 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.553 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.554 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.554 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-27T11:08:14.554 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-27T11:08:14.554 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.554 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.554 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.554 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.554 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.554 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.554 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.554 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.554 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.554 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.554 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.555 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.555 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.555 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.555 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:14.555 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "iostat", 2024-03-27T11:08:14.555 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-27T11:08:14.555 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-27T11:08:14.555 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-27T11:08:14.555 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-27T11:08:14.555 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-27T11:08:14.555 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.555 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.555 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.555 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.556 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.556 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.556 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.556 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.556 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.556 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.556 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.556 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.556 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.556 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.556 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.556 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.556 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.556 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.557 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.557 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-27T11:08:14.557 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-27T11:08:14.557 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.557 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.557 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.557 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.557 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.557 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.557 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.557 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.557 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.557 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.557 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.557 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.558 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-27T11:08:14.558 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-27T11:08:14.558 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.558 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.558 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.558 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-27T11:08:14.558 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.558 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.558 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.558 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.558 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.558 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.558 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.558 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.559 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.559 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.559 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.559 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.559 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.559 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.559 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.559 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-27T11:08:14.559 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-27T11:08:14.559 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.559 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.559 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.559 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.559 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.560 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.560 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.560 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.560 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.560 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.560 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.560 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.560 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.560 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.560 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:14.560 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "k8sevents", 2024-03-27T11:08:14.560 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-27T11:08:14.560 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-27T11:08:14.560 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-27T11:08:14.561 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_event_retention_days": { 2024-03-27T11:08:14.561 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ceph_event_retention_days", 2024-03-27T11:08:14.561 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:14.561 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.561 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.561 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "7", 2024-03-27T11:08:14.561 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.561 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.561 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.561 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-27T11:08:14.561 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.561 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.561 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.561 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.561 INFO:teuthology.orchestra.run.smithi097.stdout: "config_check_secs": { 2024-03-27T11:08:14.562 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "config_check_secs", 2024-03-27T11:08:14.562 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:14.562 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.562 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.562 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10", 2024-03-27T11:08:14.562 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "10", 2024-03-27T11:08:14.562 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.562 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.562 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-27T11:08:14.562 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.562 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.562 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.562 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.563 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-27T11:08:14.563 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-27T11:08:14.563 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.563 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.563 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.563 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.563 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.563 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.563 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.563 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.563 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.563 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.563 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.563 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.563 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.564 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.564 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.564 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.564 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.564 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.564 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.564 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-27T11:08:14.564 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-27T11:08:14.564 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.564 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.564 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.564 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.564 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.565 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.565 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.565 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.565 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.565 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.565 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.565 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.565 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-27T11:08:14.565 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-27T11:08:14.565 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.565 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.565 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.565 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-27T11:08:14.565 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.566 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.566 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.566 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.566 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.566 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.566 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.566 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.566 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.566 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.566 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.566 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.566 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.567 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.567 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.567 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-27T11:08:14.567 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-27T11:08:14.567 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.567 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.567 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.567 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.567 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.567 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.567 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.567 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.567 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.567 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.568 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.568 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.568 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.568 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.568 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:14.568 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "localpool", 2024-03-27T11:08:14.568 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-27T11:08:14.568 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-27T11:08:14.568 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-27T11:08:14.568 INFO:teuthology.orchestra.run.smithi097.stdout: "failure_domain": { 2024-03-27T11:08:14.568 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "failure_domain", 2024-03-27T11:08:14.568 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.568 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.568 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.569 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "host", 2024-03-27T11:08:14.569 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.569 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.569 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.569 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "failure domain for any created local pool", 2024-03-27T11:08:14.569 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-27T11:08:14.569 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.569 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.569 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.569 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-27T11:08:14.569 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-27T11:08:14.569 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.569 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.569 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.569 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.570 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.570 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.570 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.570 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.570 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.570 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.570 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.570 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.570 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.570 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.570 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.570 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.570 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.570 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.571 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.571 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-27T11:08:14.571 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-27T11:08:14.571 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.571 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.571 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.571 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.571 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.571 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.571 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.571 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.571 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.571 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.571 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.572 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.572 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-27T11:08:14.572 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-27T11:08:14.572 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.572 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.572 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.572 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-27T11:08:14.572 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.572 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.572 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.572 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.572 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.572 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.572 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.573 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.573 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.573 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.573 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.573 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.573 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.573 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.573 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.573 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-27T11:08:14.573 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-27T11:08:14.573 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.573 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.573 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.573 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.573 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.574 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.574 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.574 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.574 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.574 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.574 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.574 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.574 INFO:teuthology.orchestra.run.smithi097.stdout: "min_size": { 2024-03-27T11:08:14.574 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "min_size", 2024-03-27T11:08:14.574 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:14.574 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.574 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.574 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.575 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.575 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.575 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.575 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "default min_size for any created local pool", 2024-03-27T11:08:14.575 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-27T11:08:14.575 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.575 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.575 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.575 INFO:teuthology.orchestra.run.smithi097.stdout: "num_rep": { 2024-03-27T11:08:14.575 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "num_rep", 2024-03-27T11:08:14.575 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:14.575 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.575 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.576 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "3", 2024-03-27T11:08:14.576 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.576 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.576 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.576 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "default replica count for any created local pool", 2024-03-27T11:08:14.576 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.576 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.576 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.576 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.576 INFO:teuthology.orchestra.run.smithi097.stdout: "pg_num": { 2024-03-27T11:08:14.576 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "pg_num", 2024-03-27T11:08:14.576 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:14.576 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.576 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.577 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "128", 2024-03-27T11:08:14.577 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.577 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.577 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.577 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "default pg_num for any created local pool", 2024-03-27T11:08:14.577 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.577 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.577 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.577 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.577 INFO:teuthology.orchestra.run.smithi097.stdout: "prefix": { 2024-03-27T11:08:14.577 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "prefix", 2024-03-27T11:08:14.577 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.577 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.577 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.577 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.578 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.578 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.578 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.578 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "name prefix for any created local pool", 2024-03-27T11:08:14.578 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.578 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.578 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.578 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.578 INFO:teuthology.orchestra.run.smithi097.stdout: "subtree": { 2024-03-27T11:08:14.578 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "subtree", 2024-03-27T11:08:14.578 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.578 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.578 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.579 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "rack", 2024-03-27T11:08:14.579 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.579 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.579 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.579 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-27T11:08:14.579 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-27T11:08:14.579 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.579 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.579 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.579 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.579 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.579 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:14.579 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "mds_autoscaler", 2024-03-27T11:08:14.579 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-27T11:08:14.580 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-27T11:08:14.580 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-27T11:08:14.580 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-27T11:08:14.580 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-27T11:08:14.580 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.580 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.580 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.580 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.580 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.580 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.580 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.581 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.581 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.581 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.581 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.581 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.581 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.581 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.581 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.581 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.581 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.581 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.581 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.582 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-27T11:08:14.582 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-27T11:08:14.582 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.582 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.582 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.582 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.582 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.582 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.582 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.582 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.582 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.582 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.583 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.583 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.583 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-27T11:08:14.583 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-27T11:08:14.583 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.583 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.583 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.583 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-27T11:08:14.583 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.583 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.583 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.583 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.583 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.584 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.584 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.584 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.584 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.584 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.584 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.584 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.584 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.584 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.585 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.585 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-27T11:08:14.585 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-27T11:08:14.585 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.585 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.585 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.585 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.585 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.585 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.585 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.585 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.585 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.585 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.585 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.586 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.586 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.586 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.586 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:14.586 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "mirroring", 2024-03-27T11:08:14.586 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-27T11:08:14.586 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-27T11:08:14.586 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-27T11:08:14.586 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-27T11:08:14.586 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-27T11:08:14.586 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.587 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.587 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.587 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.587 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.587 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.587 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.587 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.587 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.587 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.587 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.587 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.587 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.587 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.587 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.588 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.588 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.588 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.588 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.588 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-27T11:08:14.588 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-27T11:08:14.588 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.588 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.588 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.588 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.588 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.588 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.588 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.589 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.589 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.589 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.589 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.589 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.589 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-27T11:08:14.589 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-27T11:08:14.589 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.589 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.589 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.589 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-27T11:08:14.589 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.589 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.589 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.590 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.590 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.590 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.590 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.590 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.590 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.590 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.590 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.590 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.590 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.590 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.590 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.590 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-27T11:08:14.590 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-27T11:08:14.591 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.591 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.591 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.591 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.591 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.591 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.591 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.591 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.591 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.591 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.591 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.591 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.591 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.591 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.591 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:14.592 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "nfs", 2024-03-27T11:08:14.592 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-27T11:08:14.592 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-27T11:08:14.592 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-27T11:08:14.592 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-27T11:08:14.592 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-27T11:08:14.592 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.592 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.592 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.592 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.592 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.592 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.592 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.592 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.593 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.593 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.593 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.593 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.593 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.593 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.593 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.593 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.593 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.593 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.593 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.593 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-27T11:08:14.593 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-27T11:08:14.594 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.594 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.594 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.594 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.594 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.594 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.594 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.594 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.594 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.594 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.594 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.594 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.594 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-27T11:08:14.595 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-27T11:08:14.595 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.595 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.595 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.595 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-27T11:08:14.595 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.595 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.595 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.595 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.595 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.595 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.595 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.595 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.595 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.595 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.596 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.596 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.596 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.596 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.596 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.596 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-27T11:08:14.596 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-27T11:08:14.596 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.596 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.596 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.596 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.596 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.596 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.596 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.597 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.597 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.597 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.597 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.597 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.597 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.597 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.597 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:14.597 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "orchestrator", 2024-03-27T11:08:14.597 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-27T11:08:14.597 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-27T11:08:14.597 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-27T11:08:14.597 INFO:teuthology.orchestra.run.smithi097.stdout: "fail_fs": { 2024-03-27T11:08:14.597 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "fail_fs", 2024-03-27T11:08:14.597 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.598 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.598 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.598 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.598 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.598 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.598 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.598 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-27T11:08:14.598 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.598 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.598 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.598 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.598 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-27T11:08:14.598 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-27T11:08:14.598 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.598 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.599 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.599 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.599 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.599 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.599 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.599 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.599 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.599 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.599 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.599 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.599 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.599 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.599 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.599 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.599 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.600 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.600 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.600 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-27T11:08:14.600 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-27T11:08:14.600 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.600 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.600 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.600 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.600 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.600 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.600 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.600 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.600 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.600 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.600 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.601 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.601 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-27T11:08:14.601 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-27T11:08:14.601 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.601 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.601 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.601 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-27T11:08:14.601 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.601 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.601 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.601 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.601 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.601 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.601 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.601 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.601 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.602 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.602 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.602 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.602 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.602 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.602 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.602 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-27T11:08:14.602 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-27T11:08:14.602 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.602 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.602 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.602 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.602 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.602 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.602 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.603 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.603 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.603 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.603 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.603 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.603 INFO:teuthology.orchestra.run.smithi097.stdout: "orchestrator": { 2024-03-27T11:08:14.603 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "orchestrator", 2024-03-27T11:08:14.603 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.603 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.603 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.603 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.603 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.603 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.603 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.604 INFO:teuthology.orchestra.run.smithi097.stdout: "cephadm", 2024-03-27T11:08:14.604 INFO:teuthology.orchestra.run.smithi097.stdout: "rook", 2024-03-27T11:08:14.604 INFO:teuthology.orchestra.run.smithi097.stdout: "test_orchestrator" 2024-03-27T11:08:14.604 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.604 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Orchestrator backend", 2024-03-27T11:08:14.604 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.604 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.604 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.604 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.604 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.604 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.604 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:14.604 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "osd_perf_query", 2024-03-27T11:08:14.605 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-27T11:08:14.605 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-27T11:08:14.605 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-27T11:08:14.605 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-27T11:08:14.605 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-27T11:08:14.605 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.605 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.605 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.605 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.605 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.605 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.605 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.605 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.605 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.606 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.606 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.606 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.606 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.606 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.606 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.606 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.606 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.606 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.606 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.606 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-27T11:08:14.606 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-27T11:08:14.606 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.607 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.607 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.607 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.607 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.607 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.607 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.607 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.607 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.607 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.607 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.607 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.607 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-27T11:08:14.607 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-27T11:08:14.607 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.608 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.608 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.608 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-27T11:08:14.608 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.608 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.608 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.608 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.608 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.608 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.608 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.608 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.608 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.608 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.608 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.608 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.609 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.609 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.609 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.609 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-27T11:08:14.609 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-27T11:08:14.609 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.609 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.609 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.609 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.609 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.609 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.609 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.609 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.609 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.610 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.610 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.610 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.610 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.610 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.610 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:14.610 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "osd_support", 2024-03-27T11:08:14.610 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-27T11:08:14.610 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-27T11:08:14.610 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-27T11:08:14.610 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-27T11:08:14.610 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-27T11:08:14.610 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.610 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.610 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.611 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.611 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.611 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.611 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.611 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.611 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.611 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.611 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.611 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.611 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.611 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.611 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.611 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.611 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.612 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.612 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.612 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-27T11:08:14.612 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-27T11:08:14.612 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.612 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.612 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.612 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.612 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.612 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.612 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.612 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.612 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.612 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.613 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.613 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.613 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-27T11:08:14.613 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-27T11:08:14.613 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.613 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.613 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.613 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-27T11:08:14.613 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.613 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.613 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.613 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.613 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.614 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.614 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.614 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.614 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.614 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.614 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.614 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.614 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.614 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.614 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.614 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-27T11:08:14.614 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-27T11:08:14.614 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.615 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.615 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.615 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.615 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.615 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.615 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.615 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.615 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.615 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.615 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.615 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.615 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.615 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.616 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:14.616 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "pg_autoscaler", 2024-03-27T11:08:14.616 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-27T11:08:14.616 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-27T11:08:14.616 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-27T11:08:14.616 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-27T11:08:14.616 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-27T11:08:14.616 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.616 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.616 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.616 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.616 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.616 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.617 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.617 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.617 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.617 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.617 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.617 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.617 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.617 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.617 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.617 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.617 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.617 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.617 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.618 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-27T11:08:14.618 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-27T11:08:14.618 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.618 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.618 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.618 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.618 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.618 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.618 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.618 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.618 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.618 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.618 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.618 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.619 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-27T11:08:14.619 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-27T11:08:14.619 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.619 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.619 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.619 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-27T11:08:14.619 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.619 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.619 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.619 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.619 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.619 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.619 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.619 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.620 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.620 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.620 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.620 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.620 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.620 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.620 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.620 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-27T11:08:14.620 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-27T11:08:14.620 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.620 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.620 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.620 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.620 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.620 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.621 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.621 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.621 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.621 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.621 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.621 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.621 INFO:teuthology.orchestra.run.smithi097.stdout: "sleep_interval": { 2024-03-27T11:08:14.621 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "sleep_interval", 2024-03-27T11:08:14.621 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-27T11:08:14.621 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.621 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.621 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "60", 2024-03-27T11:08:14.621 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.621 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.621 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.622 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.622 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.622 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.622 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.622 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.622 INFO:teuthology.orchestra.run.smithi097.stdout: "threshold": { 2024-03-27T11:08:14.622 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "threshold", 2024-03-27T11:08:14.622 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "float", 2024-03-27T11:08:14.622 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.622 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.622 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "3.0", 2024-03-27T11:08:14.622 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "1.0", 2024-03-27T11:08:14.622 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.622 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.623 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "scaling threshold", 2024-03-27T11:08:14.623 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-27T11:08:14.623 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.623 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.623 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.623 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.623 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.623 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:14.623 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "progress", 2024-03-27T11:08:14.623 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-27T11:08:14.623 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-27T11:08:14.623 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-27T11:08:14.623 INFO:teuthology.orchestra.run.smithi097.stdout: "allow_pg_recovery_event": { 2024-03-27T11:08:14.624 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "allow_pg_recovery_event", 2024-03-27T11:08:14.624 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.624 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.624 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.624 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.624 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.624 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.624 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.624 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-27T11:08:14.624 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.624 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.624 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.624 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.624 INFO:teuthology.orchestra.run.smithi097.stdout: "enabled": { 2024-03-27T11:08:14.625 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "enabled", 2024-03-27T11:08:14.625 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.625 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.625 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.625 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:14.625 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.625 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.625 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.625 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.625 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.625 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.625 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.625 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.625 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-27T11:08:14.625 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-27T11:08:14.626 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.626 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.626 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.626 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.626 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.626 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.626 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.626 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.626 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.626 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.626 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.626 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.626 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.627 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.627 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.627 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.627 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.627 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.627 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.627 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-27T11:08:14.627 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-27T11:08:14.627 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.627 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.627 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.627 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.627 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.627 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.628 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.628 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.628 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.628 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.628 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.628 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.628 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-27T11:08:14.628 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-27T11:08:14.628 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.628 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.628 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.628 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-27T11:08:14.628 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.628 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.629 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.629 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.629 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.629 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.629 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.629 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.629 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.629 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.629 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.629 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.629 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.629 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.629 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.629 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-27T11:08:14.629 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-27T11:08:14.630 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.630 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.630 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.630 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.630 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.630 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.630 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.630 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.630 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.630 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.630 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.630 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.630 INFO:teuthology.orchestra.run.smithi097.stdout: "max_completed_events": { 2024-03-27T11:08:14.630 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "max_completed_events", 2024-03-27T11:08:14.630 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:14.631 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.631 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.631 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "50", 2024-03-27T11:08:14.631 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.631 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.631 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.631 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "number of past completed events to remember", 2024-03-27T11:08:14.631 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.631 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.631 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.631 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.631 INFO:teuthology.orchestra.run.smithi097.stdout: "sleep_interval": { 2024-03-27T11:08:14.631 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "sleep_interval", 2024-03-27T11:08:14.631 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-27T11:08:14.632 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.632 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.632 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "5", 2024-03-27T11:08:14.632 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.632 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.632 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.632 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "how long the module is going to sleep", 2024-03-27T11:08:14.632 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.632 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.632 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.632 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.632 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.632 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.633 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:14.633 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "prometheus", 2024-03-27T11:08:14.633 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-27T11:08:14.633 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-27T11:08:14.633 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-27T11:08:14.633 INFO:teuthology.orchestra.run.smithi097.stdout: "cache": { 2024-03-27T11:08:14.633 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "cache", 2024-03-27T11:08:14.633 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.633 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.633 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.633 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:14.633 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.633 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.633 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.634 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.634 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.634 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.634 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.634 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.634 INFO:teuthology.orchestra.run.smithi097.stdout: "exclude_perf_counters": { 2024-03-27T11:08:14.634 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "exclude_perf_counters", 2024-03-27T11:08:14.634 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.634 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.634 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.634 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:14.634 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.634 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.635 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.635 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-27T11:08:14.635 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-27T11:08:14.635 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.635 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.635 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.635 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-27T11:08:14.635 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-27T11:08:14.635 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.635 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.635 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.635 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.635 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.635 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.636 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.636 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.636 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.636 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.636 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.636 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.636 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.636 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.636 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.636 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.636 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.636 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.636 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.636 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-27T11:08:14.637 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-27T11:08:14.637 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.637 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.637 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.637 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.637 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.637 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.637 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.637 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.637 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.637 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.637 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.637 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.638 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-27T11:08:14.638 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-27T11:08:14.638 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.638 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.638 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.638 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-27T11:08:14.638 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.638 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.638 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.638 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.638 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.638 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.638 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.639 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.639 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.639 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.639 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.639 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.639 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.639 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.639 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.639 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-27T11:08:14.639 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-27T11:08:14.639 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.639 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.639 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.640 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.640 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.640 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.640 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.640 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.640 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.640 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.640 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.640 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.640 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_stats_pools": { 2024-03-27T11:08:14.640 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rbd_stats_pools", 2024-03-27T11:08:14.640 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.640 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.641 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.641 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.641 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.641 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.641 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.641 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.641 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.641 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.641 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.641 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.641 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-27T11:08:14.641 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-27T11:08:14.641 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:14.642 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.642 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.642 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "300", 2024-03-27T11:08:14.642 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.642 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.642 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.642 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.642 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.642 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.642 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.642 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.642 INFO:teuthology.orchestra.run.smithi097.stdout: "scrape_interval": { 2024-03-27T11:08:14.642 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "scrape_interval", 2024-03-27T11:08:14.643 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "float", 2024-03-27T11:08:14.643 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.643 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.643 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "15.0", 2024-03-27T11:08:14.643 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.643 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.643 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.643 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.643 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.643 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.643 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.643 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.643 INFO:teuthology.orchestra.run.smithi097.stdout: "server_addr": { 2024-03-27T11:08:14.643 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "server_addr", 2024-03-27T11:08:14.643 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.644 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.644 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.644 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "::", 2024-03-27T11:08:14.644 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.644 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.644 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.644 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-27T11:08:14.644 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.644 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.644 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.644 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.644 INFO:teuthology.orchestra.run.smithi097.stdout: "server_port": { 2024-03-27T11:08:14.644 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "server_port", 2024-03-27T11:08:14.644 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:14.645 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.645 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.645 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "9283", 2024-03-27T11:08:14.645 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.645 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.645 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.645 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-27T11:08:14.645 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.645 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.645 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.645 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.645 INFO:teuthology.orchestra.run.smithi097.stdout: "stale_cache_strategy": { 2024-03-27T11:08:14.645 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "stale_cache_strategy", 2024-03-27T11:08:14.645 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.646 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.646 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.646 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "log", 2024-03-27T11:08:14.646 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.646 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.646 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.646 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.646 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.646 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.646 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.646 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.646 INFO:teuthology.orchestra.run.smithi097.stdout: "standby_behaviour": { 2024-03-27T11:08:14.646 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "standby_behaviour", 2024-03-27T11:08:14.647 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.647 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.647 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.647 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "default", 2024-03-27T11:08:14.647 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.647 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.647 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.647 INFO:teuthology.orchestra.run.smithi097.stdout: "default", 2024-03-27T11:08:14.647 INFO:teuthology.orchestra.run.smithi097.stdout: "error" 2024-03-27T11:08:14.647 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.647 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.647 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.647 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.647 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.648 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.648 INFO:teuthology.orchestra.run.smithi097.stdout: "standby_error_status_code": { 2024-03-27T11:08:14.648 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "standby_error_status_code", 2024-03-27T11:08:14.648 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:14.648 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.648 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.648 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "500", 2024-03-27T11:08:14.648 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "400", 2024-03-27T11:08:14.648 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "599", 2024-03-27T11:08:14.648 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.648 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.648 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.648 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.648 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.649 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.649 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.649 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.649 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:14.649 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rbd_support", 2024-03-27T11:08:14.649 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-27T11:08:14.649 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-27T11:08:14.649 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-27T11:08:14.649 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-27T11:08:14.649 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-27T11:08:14.649 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.649 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.649 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.649 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.650 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.650 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.650 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.650 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.650 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.650 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.650 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.650 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.650 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.650 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.650 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.650 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.651 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.651 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.651 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.651 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-27T11:08:14.651 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-27T11:08:14.651 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.651 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.651 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.651 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.651 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.651 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.651 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.651 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.651 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.652 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.652 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.652 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.652 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-27T11:08:14.652 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-27T11:08:14.652 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.652 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.652 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.652 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-27T11:08:14.652 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.652 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.652 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.652 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.652 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.653 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.653 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.653 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.653 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.653 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.653 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.653 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.653 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.653 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.653 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.653 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-27T11:08:14.653 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-27T11:08:14.654 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.654 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.654 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.654 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.654 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.654 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.654 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.654 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.654 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.654 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.654 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.654 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.654 INFO:teuthology.orchestra.run.smithi097.stdout: "max_concurrent_snap_create": { 2024-03-27T11:08:14.655 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "max_concurrent_snap_create", 2024-03-27T11:08:14.655 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:14.655 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.655 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.655 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10", 2024-03-27T11:08:14.655 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.655 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.655 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.655 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.655 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.655 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.655 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.655 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.656 INFO:teuthology.orchestra.run.smithi097.stdout: "mirror_snapshot_schedule": { 2024-03-27T11:08:14.656 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "mirror_snapshot_schedule", 2024-03-27T11:08:14.656 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.656 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.656 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.656 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.656 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.656 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.656 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.656 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.656 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.656 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.656 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.656 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.656 INFO:teuthology.orchestra.run.smithi097.stdout: "trash_purge_schedule": { 2024-03-27T11:08:14.657 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "trash_purge_schedule", 2024-03-27T11:08:14.657 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.657 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.657 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.657 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.657 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.657 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.657 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.657 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.657 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.657 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.657 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.657 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.657 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.658 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.658 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:14.658 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "restful", 2024-03-27T11:08:14.658 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-27T11:08:14.658 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-27T11:08:14.658 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-27T11:08:14.658 INFO:teuthology.orchestra.run.smithi097.stdout: "enable_auth": { 2024-03-27T11:08:14.658 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "enable_auth", 2024-03-27T11:08:14.658 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.658 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.658 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.658 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:14.658 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.658 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.659 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.659 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.659 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.659 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.659 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.659 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.659 INFO:teuthology.orchestra.run.smithi097.stdout: "key_file": { 2024-03-27T11:08:14.659 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "key_file", 2024-03-27T11:08:14.659 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.659 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.659 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.659 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.659 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.659 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.660 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.660 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.660 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.660 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.660 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.660 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.660 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-27T11:08:14.660 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-27T11:08:14.660 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.660 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.660 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.660 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.660 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.660 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.660 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.661 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.661 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.661 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.661 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.661 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.661 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.661 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.661 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.661 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.661 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.661 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.661 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.662 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-27T11:08:14.662 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-27T11:08:14.662 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.662 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.662 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.662 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.662 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.662 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.662 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.662 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.662 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.662 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.662 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.663 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.663 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-27T11:08:14.663 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-27T11:08:14.663 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.663 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.663 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.663 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-27T11:08:14.663 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.663 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.663 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.663 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.663 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.663 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.664 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.664 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.664 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.664 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.664 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.664 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.664 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.664 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.664 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.664 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-27T11:08:14.664 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-27T11:08:14.664 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.664 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.664 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.665 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.665 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.665 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.665 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.665 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.665 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.665 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.665 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.665 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.665 INFO:teuthology.orchestra.run.smithi097.stdout: "server_addr": { 2024-03-27T11:08:14.665 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "server_addr", 2024-03-27T11:08:14.665 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.665 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.666 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.666 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.666 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.666 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.666 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.666 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.666 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.666 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.666 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.666 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.666 INFO:teuthology.orchestra.run.smithi097.stdout: "server_port": { 2024-03-27T11:08:14.666 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "server_port", 2024-03-27T11:08:14.666 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.667 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.667 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.667 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.667 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.667 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.667 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.667 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.667 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.667 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.667 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.667 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.667 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.667 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.667 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:14.667 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rgw", 2024-03-27T11:08:14.668 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-27T11:08:14.668 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-27T11:08:14.668 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-27T11:08:14.668 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-27T11:08:14.668 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-27T11:08:14.668 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.668 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.668 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.668 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.668 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.668 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.668 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.668 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.668 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.668 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.669 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.669 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.669 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.669 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.669 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.669 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.669 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.669 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.669 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.669 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-27T11:08:14.669 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-27T11:08:14.669 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.669 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.669 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.670 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.670 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.670 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.670 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.670 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.670 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.670 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.670 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.670 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.670 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-27T11:08:14.670 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-27T11:08:14.670 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.670 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.671 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.671 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-27T11:08:14.671 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.671 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.671 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.671 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.671 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.671 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.671 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.671 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.671 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.671 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.672 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.672 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.672 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.672 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.672 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.672 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-27T11:08:14.672 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-27T11:08:14.672 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.672 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.672 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.672 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.672 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.672 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.672 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.672 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.673 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.673 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.673 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.673 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.673 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.673 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.673 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:14.673 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rook", 2024-03-27T11:08:14.673 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-27T11:08:14.673 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-27T11:08:14.673 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-27T11:08:14.673 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-27T11:08:14.673 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-27T11:08:14.673 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.674 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.674 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.674 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.674 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.674 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.674 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.674 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.674 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.674 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.674 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.674 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.674 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.674 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.675 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.675 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.675 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.675 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.675 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.675 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-27T11:08:14.675 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-27T11:08:14.675 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.675 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.675 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.675 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.675 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.675 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.675 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.676 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.676 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.676 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.676 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.676 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.676 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-27T11:08:14.676 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-27T11:08:14.676 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.676 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.676 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.676 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-27T11:08:14.676 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.676 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.676 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.677 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.677 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.677 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.677 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.677 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.677 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.677 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.677 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.677 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.677 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.677 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.677 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.677 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-27T11:08:14.677 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-27T11:08:14.677 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.678 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.678 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.678 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.678 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.678 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.678 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.678 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.678 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.678 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.678 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.678 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.678 INFO:teuthology.orchestra.run.smithi097.stdout: "storage_class": { 2024-03-27T11:08:14.678 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "storage_class", 2024-03-27T11:08:14.679 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.679 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.679 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.679 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "local", 2024-03-27T11:08:14.679 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.679 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.679 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.679 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-27T11:08:14.679 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.679 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.679 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.679 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.679 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.679 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.680 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:14.680 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "selftest", 2024-03-27T11:08:14.680 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-27T11:08:14.680 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-27T11:08:14.680 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-27T11:08:14.680 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-27T11:08:14.680 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-27T11:08:14.680 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.680 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.680 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.680 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.680 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.680 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.680 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.680 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.681 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.681 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.681 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.681 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.681 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.681 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.681 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.681 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.681 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.681 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.681 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.681 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-27T11:08:14.681 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-27T11:08:14.681 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.681 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.682 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.682 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.682 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.682 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.682 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.682 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.682 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.682 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.682 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.682 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.682 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-27T11:08:14.682 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-27T11:08:14.682 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.682 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.683 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.683 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-27T11:08:14.683 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.683 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.683 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.683 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.683 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.683 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.683 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.683 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.683 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.683 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.683 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.684 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.684 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.684 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.684 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.684 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-27T11:08:14.684 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-27T11:08:14.684 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.684 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.684 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.684 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.684 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.684 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.684 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.685 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.685 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.685 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.685 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.685 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.685 INFO:teuthology.orchestra.run.smithi097.stdout: "roption1": { 2024-03-27T11:08:14.685 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "roption1", 2024-03-27T11:08:14.685 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.685 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.685 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.685 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.685 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.685 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.685 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.686 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.686 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.686 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.686 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.686 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.686 INFO:teuthology.orchestra.run.smithi097.stdout: "roption2": { 2024-03-27T11:08:14.686 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "roption2", 2024-03-27T11:08:14.686 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.686 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.686 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.686 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "xyz", 2024-03-27T11:08:14.686 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.686 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.687 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.687 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.687 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.687 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.687 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.687 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.687 INFO:teuthology.orchestra.run.smithi097.stdout: "rwoption1": { 2024-03-27T11:08:14.687 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rwoption1", 2024-03-27T11:08:14.687 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.687 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.687 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.687 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.687 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.687 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.687 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.688 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.688 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.688 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.688 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.688 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.688 INFO:teuthology.orchestra.run.smithi097.stdout: "rwoption2": { 2024-03-27T11:08:14.688 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rwoption2", 2024-03-27T11:08:14.688 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:14.688 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.688 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.688 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.688 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.688 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.688 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.688 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.689 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.689 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.689 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.689 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.689 INFO:teuthology.orchestra.run.smithi097.stdout: "rwoption3": { 2024-03-27T11:08:14.689 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rwoption3", 2024-03-27T11:08:14.689 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "float", 2024-03-27T11:08:14.689 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.689 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.689 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.689 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.689 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.689 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.690 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.690 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.690 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.690 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.690 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.690 INFO:teuthology.orchestra.run.smithi097.stdout: "rwoption4": { 2024-03-27T11:08:14.690 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rwoption4", 2024-03-27T11:08:14.690 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.690 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.690 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.690 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.690 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.690 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.690 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.691 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.691 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.691 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.691 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.691 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.691 INFO:teuthology.orchestra.run.smithi097.stdout: "rwoption5": { 2024-03-27T11:08:14.691 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rwoption5", 2024-03-27T11:08:14.691 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.691 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.691 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.691 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.691 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.691 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.691 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.691 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.692 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.692 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.692 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.692 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.692 INFO:teuthology.orchestra.run.smithi097.stdout: "rwoption6": { 2024-03-27T11:08:14.692 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rwoption6", 2024-03-27T11:08:14.692 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.692 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.692 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.692 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:14.692 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.692 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.692 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.692 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.692 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.693 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.693 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.693 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.693 INFO:teuthology.orchestra.run.smithi097.stdout: "rwoption7": { 2024-03-27T11:08:14.693 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rwoption7", 2024-03-27T11:08:14.693 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:14.693 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.693 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.693 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.693 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "1", 2024-03-27T11:08:14.693 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "42", 2024-03-27T11:08:14.693 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.693 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.694 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.694 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.694 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.694 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.694 INFO:teuthology.orchestra.run.smithi097.stdout: "testkey": { 2024-03-27T11:08:14.694 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "testkey", 2024-03-27T11:08:14.694 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.694 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.694 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.694 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.694 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.694 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.694 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.694 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.695 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.695 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.695 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.695 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.695 INFO:teuthology.orchestra.run.smithi097.stdout: "testlkey": { 2024-03-27T11:08:14.695 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "testlkey", 2024-03-27T11:08:14.695 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.695 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.695 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.695 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.695 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.695 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.695 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.695 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.695 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.696 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.696 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.696 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.696 INFO:teuthology.orchestra.run.smithi097.stdout: "testnewline": { 2024-03-27T11:08:14.696 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "testnewline", 2024-03-27T11:08:14.696 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.696 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.696 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.696 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.696 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.696 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.696 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.696 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.696 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.696 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.697 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.697 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.697 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.697 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.697 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:14.697 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "snap_schedule", 2024-03-27T11:08:14.697 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-27T11:08:14.697 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-27T11:08:14.697 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-27T11:08:14.697 INFO:teuthology.orchestra.run.smithi097.stdout: "allow_m_granularity": { 2024-03-27T11:08:14.697 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "allow_m_granularity", 2024-03-27T11:08:14.697 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.697 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.697 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.697 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.698 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.698 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.698 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.698 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "allow minute scheduled snapshots", 2024-03-27T11:08:14.698 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.698 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.698 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.698 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.698 INFO:teuthology.orchestra.run.smithi097.stdout: "dump_on_update": { 2024-03-27T11:08:14.698 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "dump_on_update", 2024-03-27T11:08:14.698 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.698 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.698 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.698 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.699 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.699 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.699 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.699 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "dump database to debug log on update", 2024-03-27T11:08:14.699 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.699 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.699 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.699 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.699 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-27T11:08:14.699 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-27T11:08:14.699 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.699 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.699 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.699 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.699 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.700 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.700 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.700 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.700 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.700 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.700 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.700 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.700 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.700 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.700 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.700 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.700 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.700 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.700 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.700 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-27T11:08:14.701 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-27T11:08:14.701 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.701 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.701 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.701 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.701 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.701 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.701 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.701 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.701 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.701 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.701 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.701 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.701 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-27T11:08:14.702 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-27T11:08:14.702 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.702 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.702 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.702 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-27T11:08:14.702 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.702 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.702 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.702 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.702 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.702 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.702 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.702 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.702 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.702 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.703 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.703 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.703 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.703 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.703 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.703 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-27T11:08:14.703 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-27T11:08:14.703 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.703 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.703 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.703 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.703 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.703 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.703 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.703 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.704 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.704 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.704 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.704 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.704 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.704 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.704 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:14.704 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "stats", 2024-03-27T11:08:14.704 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-27T11:08:14.704 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-27T11:08:14.704 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-27T11:08:14.704 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-27T11:08:14.704 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-27T11:08:14.704 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.704 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.705 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.705 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.705 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.705 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.705 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.705 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.705 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.705 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.705 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.705 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.705 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.705 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.705 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.705 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.706 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.706 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.706 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.706 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-27T11:08:14.706 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-27T11:08:14.706 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.706 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.706 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.706 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.706 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.706 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.706 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.706 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.707 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.707 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.707 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.707 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.707 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-27T11:08:14.707 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-27T11:08:14.707 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.707 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.707 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.707 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-27T11:08:14.707 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.707 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.707 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.707 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.707 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.707 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.708 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.708 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.708 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.708 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.708 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.708 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.708 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.708 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.708 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.708 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-27T11:08:14.708 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-27T11:08:14.708 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.708 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.708 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.708 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.709 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.709 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.709 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.709 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.709 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.709 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.709 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.709 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.709 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.709 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.709 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:14.709 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "status", 2024-03-27T11:08:14.709 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-27T11:08:14.709 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-27T11:08:14.709 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-27T11:08:14.710 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-27T11:08:14.710 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-27T11:08:14.710 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.710 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.710 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.710 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.710 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.710 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.710 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.710 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.710 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.710 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.710 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.711 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.711 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.711 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.711 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.711 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.711 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.711 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.711 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.711 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-27T11:08:14.711 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-27T11:08:14.711 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.711 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.711 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.711 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.712 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.712 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.712 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.712 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.712 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.712 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.712 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.712 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.712 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-27T11:08:14.712 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-27T11:08:14.712 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.712 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.712 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.713 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-27T11:08:14.713 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.713 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.713 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.713 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.713 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.713 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.713 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.713 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.713 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.713 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.713 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.713 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.713 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.714 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.714 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.714 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-27T11:08:14.714 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-27T11:08:14.714 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.714 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.714 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.714 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.714 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.714 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.714 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.714 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.714 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.714 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.715 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.715 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.715 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.715 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.715 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:14.715 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "telegraf", 2024-03-27T11:08:14.715 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-27T11:08:14.715 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-27T11:08:14.715 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-27T11:08:14.715 INFO:teuthology.orchestra.run.smithi097.stdout: "address": { 2024-03-27T11:08:14.715 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "address", 2024-03-27T11:08:14.715 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.715 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.715 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.716 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-27T11:08:14.716 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.716 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.716 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.716 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.716 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.716 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.716 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.716 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.716 INFO:teuthology.orchestra.run.smithi097.stdout: "interval": { 2024-03-27T11:08:14.716 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "interval", 2024-03-27T11:08:14.716 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-27T11:08:14.716 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.716 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.716 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "15", 2024-03-27T11:08:14.717 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.717 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.717 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.717 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.717 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.717 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.717 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.717 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.717 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-27T11:08:14.717 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-27T11:08:14.717 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.717 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.717 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.717 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.717 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.718 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.718 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.718 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.718 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.718 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.718 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.718 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.718 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.718 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.718 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.718 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.718 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.718 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.719 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.719 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-27T11:08:14.719 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-27T11:08:14.719 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.719 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.719 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.719 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.719 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.719 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.719 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.719 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.719 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.719 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.720 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.720 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.720 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-27T11:08:14.720 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-27T11:08:14.720 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.720 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.720 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.720 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-27T11:08:14.720 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.720 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.720 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.720 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.720 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.720 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.721 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.721 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.721 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.721 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.721 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.721 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.721 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.721 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.721 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.721 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-27T11:08:14.721 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-27T11:08:14.721 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.721 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.721 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.722 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.722 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.722 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.722 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.722 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.722 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.722 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.722 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.722 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.722 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.722 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.722 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:14.722 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "telemetry", 2024-03-27T11:08:14.722 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-27T11:08:14.723 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-27T11:08:14.723 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-27T11:08:14.723 INFO:teuthology.orchestra.run.smithi097.stdout: "channel_basic": { 2024-03-27T11:08:14.723 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "channel_basic", 2024-03-27T11:08:14.723 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.723 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.723 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.723 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:14.723 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.723 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.723 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.723 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-27T11:08:14.723 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.724 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.724 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.724 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.724 INFO:teuthology.orchestra.run.smithi097.stdout: "channel_crash": { 2024-03-27T11:08:14.724 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "channel_crash", 2024-03-27T11:08:14.724 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.724 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.724 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.724 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:14.724 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.724 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.724 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.724 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-27T11:08:14.724 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.725 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.725 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.725 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.725 INFO:teuthology.orchestra.run.smithi097.stdout: "channel_device": { 2024-03-27T11:08:14.725 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "channel_device", 2024-03-27T11:08:14.725 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.725 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.725 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.725 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:14.725 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.725 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.725 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.725 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-27T11:08:14.725 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.726 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.726 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.726 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.726 INFO:teuthology.orchestra.run.smithi097.stdout: "channel_ident": { 2024-03-27T11:08:14.726 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "channel_ident", 2024-03-27T11:08:14.726 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.726 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.726 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.726 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.726 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.726 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.726 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.726 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-27T11:08:14.726 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.727 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.727 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.727 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.727 INFO:teuthology.orchestra.run.smithi097.stdout: "channel_perf": { 2024-03-27T11:08:14.727 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "channel_perf", 2024-03-27T11:08:14.727 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.727 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.727 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.727 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.727 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.727 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.727 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.727 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-27T11:08:14.727 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.727 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.728 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.728 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.728 INFO:teuthology.orchestra.run.smithi097.stdout: "contact": { 2024-03-27T11:08:14.728 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "contact", 2024-03-27T11:08:14.728 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.728 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.728 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.728 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.728 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.728 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.728 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.728 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.728 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.728 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.729 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.729 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.729 INFO:teuthology.orchestra.run.smithi097.stdout: "description": { 2024-03-27T11:08:14.729 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "description", 2024-03-27T11:08:14.729 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.729 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.729 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.729 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.729 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.729 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.729 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.729 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.729 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.730 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.730 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.730 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.730 INFO:teuthology.orchestra.run.smithi097.stdout: "device_url": { 2024-03-27T11:08:14.730 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "device_url", 2024-03-27T11:08:14.730 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.730 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.730 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.730 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-27T11:08:14.730 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.730 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.730 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.730 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.731 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.731 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.731 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.731 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.731 INFO:teuthology.orchestra.run.smithi097.stdout: "enabled": { 2024-03-27T11:08:14.731 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "enabled", 2024-03-27T11:08:14.731 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.731 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.731 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.731 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.731 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.731 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.731 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.732 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.732 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.732 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.732 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.732 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.732 INFO:teuthology.orchestra.run.smithi097.stdout: "interval": { 2024-03-27T11:08:14.732 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "interval", 2024-03-27T11:08:14.732 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:14.732 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.732 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.732 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "24", 2024-03-27T11:08:14.732 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "8", 2024-03-27T11:08:14.732 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.732 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.733 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.733 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.733 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.733 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.733 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.733 INFO:teuthology.orchestra.run.smithi097.stdout: "last_opt_revision": { 2024-03-27T11:08:14.733 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "last_opt_revision", 2024-03-27T11:08:14.733 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:14.733 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.733 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.733 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "1", 2024-03-27T11:08:14.733 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.733 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.733 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.734 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.734 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.734 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.734 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.734 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.734 INFO:teuthology.orchestra.run.smithi097.stdout: "leaderboard": { 2024-03-27T11:08:14.734 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "leaderboard", 2024-03-27T11:08:14.734 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.734 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.734 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.734 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.734 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.735 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.735 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.735 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.735 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.735 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.735 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.735 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.735 INFO:teuthology.orchestra.run.smithi097.stdout: "leaderboard_description": { 2024-03-27T11:08:14.735 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "leaderboard_description", 2024-03-27T11:08:14.735 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.735 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.735 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.735 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.735 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.736 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.736 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.736 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.736 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.736 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.736 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.736 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.736 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-27T11:08:14.736 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-27T11:08:14.736 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.736 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.736 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.736 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.736 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.736 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.737 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.737 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.737 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.737 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.737 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.737 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.737 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.737 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.737 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.737 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.737 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.737 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.737 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.738 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-27T11:08:14.738 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-27T11:08:14.738 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.738 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.738 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.738 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.738 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.738 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.738 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.738 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.738 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.738 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.738 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.738 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.739 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-27T11:08:14.739 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-27T11:08:14.739 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.739 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.739 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.739 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-27T11:08:14.739 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.739 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.739 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.739 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.739 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.739 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.739 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.739 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.739 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.739 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.740 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.740 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.740 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.740 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.740 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.740 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-27T11:08:14.740 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-27T11:08:14.740 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.740 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.740 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.740 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.740 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.740 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.740 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.741 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.741 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.741 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.741 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.741 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.741 INFO:teuthology.orchestra.run.smithi097.stdout: "organization": { 2024-03-27T11:08:14.741 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "organization", 2024-03-27T11:08:14.741 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.741 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.741 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.741 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.741 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.742 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.742 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.742 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.742 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.742 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.742 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.742 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.742 INFO:teuthology.orchestra.run.smithi097.stdout: "proxy": { 2024-03-27T11:08:14.742 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "proxy", 2024-03-27T11:08:14.742 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.742 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.742 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.743 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.743 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.743 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.743 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.743 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.743 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.743 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.743 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.743 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.743 INFO:teuthology.orchestra.run.smithi097.stdout: "url": { 2024-03-27T11:08:14.743 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "url", 2024-03-27T11:08:14.743 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.743 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.743 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.743 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-27T11:08:14.744 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.744 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.744 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.744 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.744 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.744 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.744 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.744 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.744 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.744 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.744 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:14.744 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "test_orchestrator", 2024-03-27T11:08:14.744 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-27T11:08:14.744 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-27T11:08:14.744 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-27T11:08:14.744 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-27T11:08:14.745 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-27T11:08:14.745 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.745 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.745 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.745 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.745 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.745 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.745 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.745 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.745 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.745 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.745 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.745 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.745 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.745 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.746 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.746 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.746 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.746 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.746 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.746 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-27T11:08:14.746 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-27T11:08:14.746 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.746 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.746 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.746 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.746 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.746 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.747 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.747 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.747 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.747 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.747 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.747 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.747 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-27T11:08:14.747 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-27T11:08:14.747 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.747 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.747 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.747 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-27T11:08:14.747 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.747 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.748 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.748 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.748 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.748 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.748 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.748 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.748 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.748 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.748 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.748 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.748 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.748 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.748 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.748 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-27T11:08:14.748 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-27T11:08:14.749 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.749 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.749 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.749 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.749 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.749 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.749 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.749 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.749 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.749 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.749 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.749 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.749 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.749 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.749 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:14.749 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "volumes", 2024-03-27T11:08:14.750 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-27T11:08:14.750 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-27T11:08:14.750 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-27T11:08:14.750 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-27T11:08:14.750 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-27T11:08:14.750 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.750 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.750 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.750 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.750 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.750 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.750 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.750 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.751 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.751 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.751 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.751 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.751 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.751 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.751 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.751 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.751 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.751 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.751 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.751 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-27T11:08:14.751 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-27T11:08:14.752 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.752 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.752 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.752 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.752 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.752 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.752 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.752 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.752 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.752 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.752 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.752 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.752 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-27T11:08:14.752 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-27T11:08:14.752 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.753 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.753 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.753 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-27T11:08:14.753 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.753 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.753 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.753 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.753 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.753 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.753 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.753 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.753 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.753 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.753 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.753 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.754 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.754 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.754 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.754 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-27T11:08:14.754 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-27T11:08:14.754 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.754 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.754 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.754 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.754 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.754 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.754 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.755 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.755 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.755 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.755 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.755 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.755 INFO:teuthology.orchestra.run.smithi097.stdout: "max_concurrent_clones": { 2024-03-27T11:08:14.755 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "max_concurrent_clones", 2024-03-27T11:08:14.755 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:14.755 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.755 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.755 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "4", 2024-03-27T11:08:14.755 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.755 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.755 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.755 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-27T11:08:14.755 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.756 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.756 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.756 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.756 INFO:teuthology.orchestra.run.smithi097.stdout: "periodic_async_work": { 2024-03-27T11:08:14.756 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "periodic_async_work", 2024-03-27T11:08:14.756 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.756 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.756 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.756 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.756 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.756 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.756 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.756 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Periodically check for async work", 2024-03-27T11:08:14.756 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.756 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.757 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.757 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.757 INFO:teuthology.orchestra.run.smithi097.stdout: "snapshot_clone_delay": { 2024-03-27T11:08:14.757 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "snapshot_clone_delay", 2024-03-27T11:08:14.757 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:14.757 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.757 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.757 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "0", 2024-03-27T11:08:14.757 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.757 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.757 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.757 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-27T11:08:14.757 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.757 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.757 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.758 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.758 INFO:teuthology.orchestra.run.smithi097.stdout: "snapshot_clone_no_wait": { 2024-03-27T11:08:14.758 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "snapshot_clone_no_wait", 2024-03-27T11:08:14.758 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.758 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.758 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.758 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-03-27T11:08:14.758 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.758 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.758 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.758 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-27T11:08:14.758 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.758 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.758 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.759 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.759 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.759 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.759 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:14.759 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "zabbix", 2024-03-27T11:08:14.759 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-03-27T11:08:14.759 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-03-27T11:08:14.759 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-03-27T11:08:14.759 INFO:teuthology.orchestra.run.smithi097.stdout: "discovery_interval": { 2024-03-27T11:08:14.759 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "discovery_interval", 2024-03-27T11:08:14.759 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "uint", 2024-03-27T11:08:14.759 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.759 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.759 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "100", 2024-03-27T11:08:14.759 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.760 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.760 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.760 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.760 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.760 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.760 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.760 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.760 INFO:teuthology.orchestra.run.smithi097.stdout: "identifier": { 2024-03-27T11:08:14.760 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "identifier", 2024-03-27T11:08:14.760 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.760 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.760 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.760 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.760 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.760 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.760 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.761 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.761 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.761 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.761 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.761 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.761 INFO:teuthology.orchestra.run.smithi097.stdout: "interval": { 2024-03-27T11:08:14.761 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "interval", 2024-03-27T11:08:14.761 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-03-27T11:08:14.761 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.761 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.761 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "60", 2024-03-27T11:08:14.761 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.761 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.761 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.762 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.762 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.762 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.762 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.762 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.762 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-03-27T11:08:14.762 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-03-27T11:08:14.762 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.762 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.762 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.762 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.762 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.763 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.763 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.763 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.763 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.763 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.763 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.763 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.763 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.763 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.763 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.763 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.763 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.763 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.764 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.764 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-03-27T11:08:14.764 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-03-27T11:08:14.764 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.764 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.764 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.764 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.764 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.764 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.764 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.764 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.764 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.764 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.764 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.765 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.765 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-03-27T11:08:14.765 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-03-27T11:08:14.765 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.765 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.765 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.765 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-03-27T11:08:14.765 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.765 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.765 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-03-27T11:08:14.765 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-03-27T11:08:14.765 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-03-27T11:08:14.765 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-03-27T11:08:14.766 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-03-27T11:08:14.766 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-03-27T11:08:14.766 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-03-27T11:08:14.766 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.766 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.766 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.766 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.766 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.766 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.766 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-03-27T11:08:14.766 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-03-27T11:08:14.766 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-03-27T11:08:14.766 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.766 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-03-27T11:08:14.766 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-03-27T11:08:14.767 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.767 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.767 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.767 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.767 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.767 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.767 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.767 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.767 INFO:teuthology.orchestra.run.smithi097.stdout: "zabbix_host": { 2024-03-27T11:08:14.767 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "zabbix_host", 2024-03-27T11:08:14.767 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.767 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.767 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.767 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-03-27T11:08:14.767 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.768 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.768 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.768 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.768 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.768 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.768 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.768 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.768 INFO:teuthology.orchestra.run.smithi097.stdout: "zabbix_port": { 2024-03-27T11:08:14.768 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "zabbix_port", 2024-03-27T11:08:14.768 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-03-27T11:08:14.768 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.768 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.768 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10051", 2024-03-27T11:08:14.768 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.768 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.769 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.769 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.769 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.769 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.769 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.769 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.769 INFO:teuthology.orchestra.run.smithi097.stdout: "zabbix_sender": { 2024-03-27T11:08:14.769 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "zabbix_sender", 2024-03-27T11:08:14.769 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-03-27T11:08:14.769 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-03-27T11:08:14.769 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-03-27T11:08:14.769 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-27T11:08:14.769 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-03-27T11:08:14.769 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-03-27T11:08:14.769 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-03-27T11:08:14.769 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-03-27T11:08:14.770 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-03-27T11:08:14.770 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-03-27T11:08:14.770 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-03-27T11:08:14.770 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.770 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.770 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.770 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.770 INFO:teuthology.orchestra.run.smithi097.stdout: "services": { 2024-03-27T11:08:14.770 INFO:teuthology.orchestra.run.smithi097.stdout: "dashboard": "https://172.21.15.97:8443/" 2024-03-27T11:08:14.770 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.770 INFO:teuthology.orchestra.run.smithi097.stdout: "always_on_modules": { 2024-03-27T11:08:14.770 INFO:teuthology.orchestra.run.smithi097.stdout: "octopus": [ 2024-03-27T11:08:14.770 INFO:teuthology.orchestra.run.smithi097.stdout: "balancer", 2024-03-27T11:08:14.770 INFO:teuthology.orchestra.run.smithi097.stdout: "crash", 2024-03-27T11:08:14.771 INFO:teuthology.orchestra.run.smithi097.stdout: "devicehealth", 2024-03-27T11:08:14.771 INFO:teuthology.orchestra.run.smithi097.stdout: "orchestrator", 2024-03-27T11:08:14.771 INFO:teuthology.orchestra.run.smithi097.stdout: "pg_autoscaler", 2024-03-27T11:08:14.771 INFO:teuthology.orchestra.run.smithi097.stdout: "progress", 2024-03-27T11:08:14.771 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_support", 2024-03-27T11:08:14.771 INFO:teuthology.orchestra.run.smithi097.stdout: "status", 2024-03-27T11:08:14.771 INFO:teuthology.orchestra.run.smithi097.stdout: "telemetry", 2024-03-27T11:08:14.771 INFO:teuthology.orchestra.run.smithi097.stdout: "volumes" 2024-03-27T11:08:14.771 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.771 INFO:teuthology.orchestra.run.smithi097.stdout: "pacific": [ 2024-03-27T11:08:14.771 INFO:teuthology.orchestra.run.smithi097.stdout: "balancer", 2024-03-27T11:08:14.771 INFO:teuthology.orchestra.run.smithi097.stdout: "crash", 2024-03-27T11:08:14.771 INFO:teuthology.orchestra.run.smithi097.stdout: "devicehealth", 2024-03-27T11:08:14.771 INFO:teuthology.orchestra.run.smithi097.stdout: "orchestrator", 2024-03-27T11:08:14.772 INFO:teuthology.orchestra.run.smithi097.stdout: "pg_autoscaler", 2024-03-27T11:08:14.772 INFO:teuthology.orchestra.run.smithi097.stdout: "progress", 2024-03-27T11:08:14.772 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_support", 2024-03-27T11:08:14.772 INFO:teuthology.orchestra.run.smithi097.stdout: "status", 2024-03-27T11:08:14.772 INFO:teuthology.orchestra.run.smithi097.stdout: "telemetry", 2024-03-27T11:08:14.772 INFO:teuthology.orchestra.run.smithi097.stdout: "volumes" 2024-03-27T11:08:14.772 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.772 INFO:teuthology.orchestra.run.smithi097.stdout: "quincy": [ 2024-03-27T11:08:14.772 INFO:teuthology.orchestra.run.smithi097.stdout: "balancer", 2024-03-27T11:08:14.772 INFO:teuthology.orchestra.run.smithi097.stdout: "crash", 2024-03-27T11:08:14.772 INFO:teuthology.orchestra.run.smithi097.stdout: "devicehealth", 2024-03-27T11:08:14.772 INFO:teuthology.orchestra.run.smithi097.stdout: "orchestrator", 2024-03-27T11:08:14.772 INFO:teuthology.orchestra.run.smithi097.stdout: "pg_autoscaler", 2024-03-27T11:08:14.773 INFO:teuthology.orchestra.run.smithi097.stdout: "progress", 2024-03-27T11:08:14.773 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_support", 2024-03-27T11:08:14.773 INFO:teuthology.orchestra.run.smithi097.stdout: "status", 2024-03-27T11:08:14.773 INFO:teuthology.orchestra.run.smithi097.stdout: "telemetry", 2024-03-27T11:08:14.773 INFO:teuthology.orchestra.run.smithi097.stdout: "volumes" 2024-03-27T11:08:14.773 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.773 INFO:teuthology.orchestra.run.smithi097.stdout: "reef": [ 2024-03-27T11:08:14.773 INFO:teuthology.orchestra.run.smithi097.stdout: "balancer", 2024-03-27T11:08:14.773 INFO:teuthology.orchestra.run.smithi097.stdout: "crash", 2024-03-27T11:08:14.773 INFO:teuthology.orchestra.run.smithi097.stdout: "devicehealth", 2024-03-27T11:08:14.773 INFO:teuthology.orchestra.run.smithi097.stdout: "orchestrator", 2024-03-27T11:08:14.773 INFO:teuthology.orchestra.run.smithi097.stdout: "pg_autoscaler", 2024-03-27T11:08:14.773 INFO:teuthology.orchestra.run.smithi097.stdout: "progress", 2024-03-27T11:08:14.774 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_support", 2024-03-27T11:08:14.774 INFO:teuthology.orchestra.run.smithi097.stdout: "status", 2024-03-27T11:08:14.774 INFO:teuthology.orchestra.run.smithi097.stdout: "telemetry", 2024-03-27T11:08:14.774 INFO:teuthology.orchestra.run.smithi097.stdout: "volumes" 2024-03-27T11:08:14.774 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-03-27T11:08:14.774 INFO:teuthology.orchestra.run.smithi097.stdout: "squid": [ 2024-03-27T11:08:14.774 INFO:teuthology.orchestra.run.smithi097.stdout: "balancer", 2024-03-27T11:08:14.774 INFO:teuthology.orchestra.run.smithi097.stdout: "crash", 2024-03-27T11:08:14.774 INFO:teuthology.orchestra.run.smithi097.stdout: "devicehealth", 2024-03-27T11:08:14.774 INFO:teuthology.orchestra.run.smithi097.stdout: "orchestrator", 2024-03-27T11:08:14.774 INFO:teuthology.orchestra.run.smithi097.stdout: "pg_autoscaler", 2024-03-27T11:08:14.774 INFO:teuthology.orchestra.run.smithi097.stdout: "progress", 2024-03-27T11:08:14.774 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_support", 2024-03-27T11:08:14.774 INFO:teuthology.orchestra.run.smithi097.stdout: "status", 2024-03-27T11:08:14.775 INFO:teuthology.orchestra.run.smithi097.stdout: "telemetry", 2024-03-27T11:08:14.775 INFO:teuthology.orchestra.run.smithi097.stdout: "volumes" 2024-03-27T11:08:14.775 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-03-27T11:08:14.775 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.775 INFO:teuthology.orchestra.run.smithi097.stdout: "last_failure_osd_epoch": 3, 2024-03-27T11:08:14.775 INFO:teuthology.orchestra.run.smithi097.stdout: "active_clients": [ 2024-03-27T11:08:14.775 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:14.775 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "devicehealth", 2024-03-27T11:08:14.775 INFO:teuthology.orchestra.run.smithi097.stdout: "addrvec": [ 2024-03-27T11:08:14.775 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:14.775 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v2", 2024-03-27T11:08:14.775 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.97:0", 2024-03-27T11:08:14.775 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 1887548681 2024-03-27T11:08:14.775 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.776 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-03-27T11:08:14.776 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.776 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:14.776 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "libcephsqlite", 2024-03-27T11:08:14.776 INFO:teuthology.orchestra.run.smithi097.stdout: "addrvec": [ 2024-03-27T11:08:14.776 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:14.776 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v2", 2024-03-27T11:08:14.776 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.97:0", 2024-03-27T11:08:14.776 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 2584280197 2024-03-27T11:08:14.776 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.776 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-03-27T11:08:14.776 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.776 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:14.776 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rbd_support", 2024-03-27T11:08:14.776 INFO:teuthology.orchestra.run.smithi097.stdout: "addrvec": [ 2024-03-27T11:08:14.776 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:14.777 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v2", 2024-03-27T11:08:14.777 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.97:0", 2024-03-27T11:08:14.777 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 3268554894 2024-03-27T11:08:14.777 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.777 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-03-27T11:08:14.777 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:14.777 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:14.777 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "volumes", 2024-03-27T11:08:14.777 INFO:teuthology.orchestra.run.smithi097.stdout: "addrvec": [ 2024-03-27T11:08:14.777 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-27T11:08:14.777 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v2", 2024-03-27T11:08:14.777 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.97:0", 2024-03-27T11:08:14.777 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 1075669214 2024-03-27T11:08:14.777 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.777 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-03-27T11:08:14.778 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:14.778 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-03-27T11:08:14.778 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-03-27T11:08:14.779 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:08:13 smithi177 ceph-mon[35294]: pgmap v788: 97 pgs: 97 active+clean; 9.8 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 36 MiB/s wr, 645 op/s 2024-03-27T11:08:14.779 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:08:13 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/2949164412' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-27T11:08:14.779 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:08:13 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/451076676' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-27T11:08:14.780 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:08:13 smithi114 ceph-mon[35338]: pgmap v788: 97 pgs: 97 active+clean; 9.8 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 36 MiB/s wr, 645 op/s 2024-03-27T11:08:14.780 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:08:13 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/2949164412' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-27T11:08:14.780 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:08:13 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/451076676' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-27T11:08:14.780 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:08:14 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/1996842846' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-27T11:08:14.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:13.512+0000 7fa58ec00640 1 -- 172.21.15.97:0/1996842846 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fa56c0a3690 msgr2=0x7fa56c0a5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:08:14.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:13.512+0000 7fa58ec00640 1 --2- 172.21.15.97:0/1996842846 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fa56c0a3690 0x7fa56c0a5b50 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fa58819b540 tx=0x7fa57000b390 comp rx=0 tx=0).stop 2024-03-27T11:08:14.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:13.512+0000 7fa58ec00640 1 -- 172.21.15.97:0/1996842846 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa588101b60 msgr2=0x7fa58819a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:08:14.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:13.512+0000 7fa58ec00640 1 --2- 172.21.15.97:0/1996842846 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa588101b60 0x7fa58819a030 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fa578000c00 tx=0x7fa5780040c0 comp rx=0 tx=0).stop 2024-03-27T11:08:14.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:13.513+0000 7fa58ec00640 1 -- 172.21.15.97:0/1996842846 shutdown_connections 2024-03-27T11:08:14.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:13.513+0000 7fa58ec00640 1 --2- 172.21.15.97:0/1996842846 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fa56c0a3690 0x7fa56c0a5b50 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:08:14.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:13.513+0000 7fa58ec00640 1 --2- 172.21.15.97:0/1996842846 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa588103710 0x7fa5881a15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:08:14.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:13.513+0000 7fa58ec00640 1 --2- 172.21.15.97:0/1996842846 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa588102d50 0x7fa58819a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:08:14.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:13.513+0000 7fa58ec00640 1 --2- 172.21.15.97:0/1996842846 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa588101b60 0x7fa58819a030 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:08:14.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:13.513+0000 7fa58ec00640 1 -- 172.21.15.97:0/1996842846 >> 172.21.15.97:0/1996842846 conn(0x7fa5880fd130 msgr2=0x7fa588073d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:08:14.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:13.513+0000 7fa58ec00640 1 -- 172.21.15.97:0/1996842846 shutdown_connections 2024-03-27T11:08:14.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:13.513+0000 7fa58ec00640 1 -- 172.21.15.97:0/1996842846 wait complete. 2024-03-27T11:08:14.787 DEBUG:tasks.check_counter:Getting stats from a 2024-03-27T11:08:14.788 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell --fsid adca88c4-ec26-11ee-95d0-87774f69a715 -- ceph daemon mds.a perf dump 2024-03-27T11:08:14.788 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:08:13 smithi097 ceph-mon[31859]: pgmap v788: 97 pgs: 97 active+clean; 9.8 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 36 MiB/s wr, 645 op/s 2024-03-27T11:08:14.788 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:08:13 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/2949164412' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-27T11:08:14.788 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:08:13 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/451076676' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-27T11:08:14.859 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:08:14 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/1996842846' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-27T11:08:14.923 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:08:14 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/1996842846' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-27T11:08:15.235 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T11:08:15.638 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:08:15 smithi097 ceph-mon[31859]: pgmap v789: 97 pgs: 97 active+clean; 9.4 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 30 MiB/s wr, 556 op/s 2024-03-27T11:08:15.763 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:08:15 smithi114 ceph-mon[35338]: pgmap v789: 97 pgs: 97 active+clean; 9.4 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 30 MiB/s wr, 556 op/s 2024-03-27T11:08:15.924 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:08:15 smithi177 ceph-mon[35294]: pgmap v789: 97 pgs: 97 active+clean; 9.4 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 30 MiB/s wr, 556 op/s 2024-03-27T11:08:17.653 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-03-27T11:08:17.654 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-0": { 2024-03-27T11:08:17.654 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 1093, 2024-03-27T11:08:17.654 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 1082, 2024-03-27T11:08:17.654 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 837920, 2024-03-27T11:08:17.654 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 411720, 2024-03-27T11:08:17.654 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 5, 2024-03-27T11:08:17.654 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 1, 2024-03-27T11:08:17.654 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.198241141, 2024-03-27T11:08:17.654 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.073291485, 2024-03-27T11:08:17.654 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.060113533, 2024-03-27T11:08:17.654 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.051170274, 2024-03-27T11:08:17.655 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-03-27T11:08:17.655 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 1081, 2024-03-27T11:08:17.655 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.111336310, 2024-03-27T11:08:17.655 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000102993 2024-03-27T11:08:17.655 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:17.655 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-03-27T11:08:17.655 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-27T11:08:17.655 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-27T11:08:17.655 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-27T11:08:17.655 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:17.655 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 554336, 2024-03-27T11:08:17.655 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 143328 2024-03-27T11:08:17.655 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:17.655 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-1": { 2024-03-27T11:08:17.656 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 714, 2024-03-27T11:08:17.656 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 889, 2024-03-27T11:08:17.656 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 176210, 2024-03-27T11:08:17.656 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 617842, 2024-03-27T11:08:17.656 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 5, 2024-03-27T11:08:17.656 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 4, 2024-03-27T11:08:17.656 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.131561782, 2024-03-27T11:08:17.656 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.051464762, 2024-03-27T11:08:17.656 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.024447850, 2024-03-27T11:08:17.656 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.041641407, 2024-03-27T11:08:17.656 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-03-27T11:08:17.656 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 887, 2024-03-27T11:08:17.656 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.103178030, 2024-03-27T11:08:17.657 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000116322 2024-03-27T11:08:17.657 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:17.657 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-03-27T11:08:17.657 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-27T11:08:17.657 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-27T11:08:17.657 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-27T11:08:17.657 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:17.657 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-27T11:08:17.657 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 391040 2024-03-27T11:08:17.657 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:17.657 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-2": { 2024-03-27T11:08:17.657 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 852, 2024-03-27T11:08:17.657 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 852, 2024-03-27T11:08:17.657 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 226242, 2024-03-27T11:08:17.658 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 270113, 2024-03-27T11:08:17.658 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 4, 2024-03-27T11:08:17.658 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 3, 2024-03-27T11:08:17.658 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.116366678, 2024-03-27T11:08:17.658 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.036581237, 2024-03-27T11:08:17.658 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.028334864, 2024-03-27T11:08:17.658 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.035449208, 2024-03-27T11:08:17.658 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-03-27T11:08:17.658 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 851, 2024-03-27T11:08:17.658 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.092909162, 2024-03-27T11:08:17.658 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000109176 2024-03-27T11:08:17.658 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:17.658 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-03-27T11:08:17.659 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-27T11:08:17.659 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-27T11:08:17.659 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-27T11:08:17.659 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:17.659 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-03-27T11:08:17.659 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 448 2024-03-27T11:08:17.659 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:17.659 INFO:teuthology.orchestra.run.smithi097.stdout: "cct": { 2024-03-27T11:08:17.659 INFO:teuthology.orchestra.run.smithi097.stdout: "total_workers": 1, 2024-03-27T11:08:17.659 INFO:teuthology.orchestra.run.smithi097.stdout: "unhealthy_workers": 0 2024-03-27T11:08:17.659 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:17.659 INFO:teuthology.orchestra.run.smithi097.stdout: "finisher-MDSRank": { 2024-03-27T11:08:17.659 INFO:teuthology.orchestra.run.smithi097.stdout: "queue_len": 0, 2024-03-27T11:08:17.660 INFO:teuthology.orchestra.run.smithi097.stdout: "complete_latency": { 2024-03-27T11:08:17.660 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 3922, 2024-03-27T11:08:17.660 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.167716281, 2024-03-27T11:08:17.660 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000042762 2024-03-27T11:08:17.660 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:17.660 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:17.660 INFO:teuthology.orchestra.run.smithi097.stdout: "finisher-PurgeQueue": { 2024-03-27T11:08:17.660 INFO:teuthology.orchestra.run.smithi097.stdout: "queue_len": 0, 2024-03-27T11:08:17.660 INFO:teuthology.orchestra.run.smithi097.stdout: "complete_latency": { 2024-03-27T11:08:17.660 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-27T11:08:17.660 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-27T11:08:17.660 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-27T11:08:17.660 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:17.661 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:17.661 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": { 2024-03-27T11:08:17.661 INFO:teuthology.orchestra.run.smithi097.stdout: "request": 0, 2024-03-27T11:08:17.661 INFO:teuthology.orchestra.run.smithi097.stdout: "reply": 0, 2024-03-27T11:08:17.661 INFO:teuthology.orchestra.run.smithi097.stdout: "reply_latency": { 2024-03-27T11:08:17.661 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-27T11:08:17.661 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-27T11:08:17.661 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-27T11:08:17.661 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:17.661 INFO:teuthology.orchestra.run.smithi097.stdout: "slow_reply": 0, 2024-03-27T11:08:17.661 INFO:teuthology.orchestra.run.smithi097.stdout: "forward": 0, 2024-03-27T11:08:17.661 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_fetch_complete": 0, 2024-03-27T11:08:17.661 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_fetch_keys": 0, 2024-03-27T11:08:17.661 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_commit": 0, 2024-03-27T11:08:17.661 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_split": 0, 2024-03-27T11:08:17.662 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_merge": 0, 2024-03-27T11:08:17.662 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes": 0, 2024-03-27T11:08:17.662 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_top": 0, 2024-03-27T11:08:17.662 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_bottom": 0, 2024-03-27T11:08:17.662 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_pin_tail": 0, 2024-03-27T11:08:17.662 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_pinned": 0, 2024-03-27T11:08:17.662 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_expired": 0, 2024-03-27T11:08:17.662 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_with_caps": 0, 2024-03-27T11:08:17.662 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": 0, 2024-03-27T11:08:17.662 INFO:teuthology.orchestra.run.smithi097.stdout: "subtrees": 1, 2024-03-27T11:08:17.662 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse": 0, 2024-03-27T11:08:17.662 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_hit": 0, 2024-03-27T11:08:17.662 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_forward": 0, 2024-03-27T11:08:17.663 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_discover": 0, 2024-03-27T11:08:17.663 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_dir_fetch": 0, 2024-03-27T11:08:17.663 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_remote_ino": 0, 2024-03-27T11:08:17.663 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_lock": 0, 2024-03-27T11:08:17.663 INFO:teuthology.orchestra.run.smithi097.stdout: "load_cent": 0, 2024-03-27T11:08:17.663 INFO:teuthology.orchestra.run.smithi097.stdout: "q": 0, 2024-03-27T11:08:17.663 INFO:teuthology.orchestra.run.smithi097.stdout: "exported": 0, 2024-03-27T11:08:17.663 INFO:teuthology.orchestra.run.smithi097.stdout: "exported_inodes": 0, 2024-03-27T11:08:17.663 INFO:teuthology.orchestra.run.smithi097.stdout: "imported": 0, 2024-03-27T11:08:17.663 INFO:teuthology.orchestra.run.smithi097.stdout: "imported_inodes": 0, 2024-03-27T11:08:17.663 INFO:teuthology.orchestra.run.smithi097.stdout: "openino_dir_fetch": 0, 2024-03-27T11:08:17.663 INFO:teuthology.orchestra.run.smithi097.stdout: "openino_backtrace_fetch": 0, 2024-03-27T11:08:17.663 INFO:teuthology.orchestra.run.smithi097.stdout: "openino_peer_discover": 0, 2024-03-27T11:08:17.663 INFO:teuthology.orchestra.run.smithi097.stdout: "root_rfiles": 0, 2024-03-27T11:08:17.664 INFO:teuthology.orchestra.run.smithi097.stdout: "root_rbytes": 0, 2024-03-27T11:08:17.664 INFO:teuthology.orchestra.run.smithi097.stdout: "root_rsnaps": 0, 2024-03-27T11:08:17.664 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_backtrace_fetch": 0, 2024-03-27T11:08:17.664 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_set_tag": 0, 2024-03-27T11:08:17.664 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_backtrace_repaired": 0, 2024-03-27T11:08:17.664 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_inotable_repaired": 0, 2024-03-27T11:08:17.664 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_dir_inodes": 0, 2024-03-27T11:08:17.664 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_dir_base_inodes": 0, 2024-03-27T11:08:17.664 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_dirfrag_rstats": 0, 2024-03-27T11:08:17.664 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_file_inodes": 0, 2024-03-27T11:08:17.664 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_inode_file_caps": 0, 2024-03-27T11:08:17.664 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_revoke": 0, 2024-03-27T11:08:17.664 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_grant": 0, 2024-03-27T11:08:17.664 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_trunc": 0, 2024-03-27T11:08:17.665 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-27T11:08:17.665 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-27T11:08:17.665 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_caps": 0, 2024-03-27T11:08:17.665 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_caps_dirty": 0, 2024-03-27T11:08:17.665 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_cap_release": 0, 2024-03-27T11:08:17.665 INFO:teuthology.orchestra.run.smithi097.stdout: "process_request_cap_release": 0 2024-03-27T11:08:17.665 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:17.665 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_cache": { 2024-03-27T11:08:17.665 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_update": 0, 2024-03-27T11:08:17.665 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_update_receipt": 0, 2024-03-27T11:08:17.665 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_try_discover": 0, 2024-03-27T11:08:17.665 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_send_discover": 0, 2024-03-27T11:08:17.665 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_handle_discover": 0, 2024-03-27T11:08:17.665 INFO:teuthology.orchestra.run.smithi097.stdout: "num_strays": 0, 2024-03-27T11:08:17.666 INFO:teuthology.orchestra.run.smithi097.stdout: "num_strays_delayed": 0, 2024-03-27T11:08:17.666 INFO:teuthology.orchestra.run.smithi097.stdout: "num_strays_enqueuing": 0, 2024-03-27T11:08:17.666 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_created": 0, 2024-03-27T11:08:17.666 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_enqueued": 0, 2024-03-27T11:08:17.666 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_reintegrated": 0, 2024-03-27T11:08:17.666 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_migrated": 0, 2024-03-27T11:08:17.666 INFO:teuthology.orchestra.run.smithi097.stdout: "num_recovering_processing": 0, 2024-03-27T11:08:17.666 INFO:teuthology.orchestra.run.smithi097.stdout: "num_recovering_enqueued": 0, 2024-03-27T11:08:17.666 INFO:teuthology.orchestra.run.smithi097.stdout: "num_recovering_prioritized": 0, 2024-03-27T11:08:17.666 INFO:teuthology.orchestra.run.smithi097.stdout: "recovery_started": 0, 2024-03-27T11:08:17.666 INFO:teuthology.orchestra.run.smithi097.stdout: "recovery_completed": 0, 2024-03-27T11:08:17.666 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_quiesce_path": 0, 2024-03-27T11:08:17.666 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_quiesce_inode": 0, 2024-03-27T11:08:17.667 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_enqueue_scrub": 0, 2024-03-27T11:08:17.667 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_exportdir": 0, 2024-03-27T11:08:17.667 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_flush": 0, 2024-03-27T11:08:17.667 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_fragmentdir": 0, 2024-03-27T11:08:17.667 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_fragstats": 0, 2024-03-27T11:08:17.667 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_inodestats": 0 2024-03-27T11:08:17.667 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:17.667 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_log": { 2024-03-27T11:08:17.667 INFO:teuthology.orchestra.run.smithi097.stdout: "evlrg": 0, 2024-03-27T11:08:17.667 INFO:teuthology.orchestra.run.smithi097.stdout: "evadd": 0, 2024-03-27T11:08:17.667 INFO:teuthology.orchestra.run.smithi097.stdout: "evex": 0, 2024-03-27T11:08:17.667 INFO:teuthology.orchestra.run.smithi097.stdout: "evtrm": 0, 2024-03-27T11:08:17.667 INFO:teuthology.orchestra.run.smithi097.stdout: "ev": 3, 2024-03-27T11:08:17.667 INFO:teuthology.orchestra.run.smithi097.stdout: "evexg": 0, 2024-03-27T11:08:17.667 INFO:teuthology.orchestra.run.smithi097.stdout: "evexd": 0, 2024-03-27T11:08:17.668 INFO:teuthology.orchestra.run.smithi097.stdout: "segadd": 0, 2024-03-27T11:08:17.668 INFO:teuthology.orchestra.run.smithi097.stdout: "segex": 0, 2024-03-27T11:08:17.668 INFO:teuthology.orchestra.run.smithi097.stdout: "segtrm": 0, 2024-03-27T11:08:17.668 INFO:teuthology.orchestra.run.smithi097.stdout: "seg": 2, 2024-03-27T11:08:17.668 INFO:teuthology.orchestra.run.smithi097.stdout: "segmjr": 2, 2024-03-27T11:08:17.668 INFO:teuthology.orchestra.run.smithi097.stdout: "segexg": 0, 2024-03-27T11:08:17.668 INFO:teuthology.orchestra.run.smithi097.stdout: "segexd": 0, 2024-03-27T11:08:17.668 INFO:teuthology.orchestra.run.smithi097.stdout: "expos": 4194304, 2024-03-27T11:08:17.668 INFO:teuthology.orchestra.run.smithi097.stdout: "wrpos": 4195385, 2024-03-27T11:08:17.668 INFO:teuthology.orchestra.run.smithi097.stdout: "rdpos": 4194892, 2024-03-27T11:08:17.668 INFO:teuthology.orchestra.run.smithi097.stdout: "jlat": { 2024-03-27T11:08:17.668 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-27T11:08:17.668 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-27T11:08:17.668 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-27T11:08:17.669 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:17.669 INFO:teuthology.orchestra.run.smithi097.stdout: "replayed": 3 2024-03-27T11:08:17.669 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:17.669 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_mem": { 2024-03-27T11:08:17.669 INFO:teuthology.orchestra.run.smithi097.stdout: "ino": 3, 2024-03-27T11:08:17.669 INFO:teuthology.orchestra.run.smithi097.stdout: "ino+": 3, 2024-03-27T11:08:17.669 INFO:teuthology.orchestra.run.smithi097.stdout: "ino-": 0, 2024-03-27T11:08:17.669 INFO:teuthology.orchestra.run.smithi097.stdout: "dir": 1, 2024-03-27T11:08:17.669 INFO:teuthology.orchestra.run.smithi097.stdout: "dir+": 1, 2024-03-27T11:08:17.669 INFO:teuthology.orchestra.run.smithi097.stdout: "dir-": 0, 2024-03-27T11:08:17.669 INFO:teuthology.orchestra.run.smithi097.stdout: "dn": 0, 2024-03-27T11:08:17.669 INFO:teuthology.orchestra.run.smithi097.stdout: "dn+": 0, 2024-03-27T11:08:17.669 INFO:teuthology.orchestra.run.smithi097.stdout: "dn-": 0, 2024-03-27T11:08:17.669 INFO:teuthology.orchestra.run.smithi097.stdout: "cap": 0, 2024-03-27T11:08:17.669 INFO:teuthology.orchestra.run.smithi097.stdout: "cap+": 0, 2024-03-27T11:08:17.670 INFO:teuthology.orchestra.run.smithi097.stdout: "cap-": 0, 2024-03-27T11:08:17.670 INFO:teuthology.orchestra.run.smithi097.stdout: "rss": 48896, 2024-03-27T11:08:17.670 INFO:teuthology.orchestra.run.smithi097.stdout: "heap": 215416 2024-03-27T11:08:17.670 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:17.670 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_server": { 2024-03-27T11:08:17.670 INFO:teuthology.orchestra.run.smithi097.stdout: "dispatch_client_request": 0, 2024-03-27T11:08:17.670 INFO:teuthology.orchestra.run.smithi097.stdout: "dispatch_server_request": 0, 2024-03-27T11:08:17.670 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_request": 0, 2024-03-27T11:08:17.670 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_session": 0, 2024-03-27T11:08:17.670 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_peer_request": 0, 2024-03-27T11:08:17.670 INFO:teuthology.orchestra.run.smithi097.stdout: "req_create_latency": { 2024-03-27T11:08:17.670 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-27T11:08:17.670 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-27T11:08:17.671 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-27T11:08:17.671 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:17.671 INFO:teuthology.orchestra.run.smithi097.stdout: "req_getattr_latency": { 2024-03-27T11:08:17.671 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-27T11:08:17.671 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-27T11:08:17.671 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-27T11:08:17.671 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:17.671 INFO:teuthology.orchestra.run.smithi097.stdout: "req_getfilelock_latency": { 2024-03-27T11:08:17.671 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-27T11:08:17.671 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-27T11:08:17.671 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-27T11:08:17.671 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:17.671 INFO:teuthology.orchestra.run.smithi097.stdout: "req_link_latency": { 2024-03-27T11:08:17.671 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-27T11:08:17.672 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-27T11:08:17.672 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-27T11:08:17.672 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:17.672 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookup_latency": { 2024-03-27T11:08:17.672 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-27T11:08:17.672 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-27T11:08:17.672 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-27T11:08:17.672 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:17.672 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookuphash_latency": { 2024-03-27T11:08:17.672 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-27T11:08:17.672 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-27T11:08:17.672 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-27T11:08:17.672 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:17.672 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupino_latency": { 2024-03-27T11:08:17.673 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-27T11:08:17.673 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-27T11:08:17.673 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-27T11:08:17.673 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:17.673 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupname_latency": { 2024-03-27T11:08:17.673 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-27T11:08:17.673 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-27T11:08:17.673 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-27T11:08:17.673 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:17.673 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupparent_latency": { 2024-03-27T11:08:17.673 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-27T11:08:17.673 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-27T11:08:17.673 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-27T11:08:17.673 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:17.673 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupsnap_latency": { 2024-03-27T11:08:17.674 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-27T11:08:17.674 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-27T11:08:17.674 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-27T11:08:17.674 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:17.674 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lssnap_latency": { 2024-03-27T11:08:17.674 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-27T11:08:17.674 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-27T11:08:17.674 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-27T11:08:17.674 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:17.674 INFO:teuthology.orchestra.run.smithi097.stdout: "req_mkdir_latency": { 2024-03-27T11:08:17.674 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-27T11:08:17.675 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-27T11:08:17.675 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-27T11:08:17.675 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:17.675 INFO:teuthology.orchestra.run.smithi097.stdout: "req_mknod_latency": { 2024-03-27T11:08:17.675 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-27T11:08:17.675 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-27T11:08:17.676 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-27T11:08:17.676 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:17.676 INFO:teuthology.orchestra.run.smithi097.stdout: "req_mksnap_latency": { 2024-03-27T11:08:17.676 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-27T11:08:17.676 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-27T11:08:17.676 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-27T11:08:17.676 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:17.676 INFO:teuthology.orchestra.run.smithi097.stdout: "req_open_latency": { 2024-03-27T11:08:17.676 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-27T11:08:17.676 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-27T11:08:17.676 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-27T11:08:17.676 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:17.676 INFO:teuthology.orchestra.run.smithi097.stdout: "req_readdir_latency": { 2024-03-27T11:08:17.676 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-27T11:08:17.676 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-27T11:08:17.677 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-27T11:08:17.677 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:17.677 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rename_latency": { 2024-03-27T11:08:17.677 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-27T11:08:17.677 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-27T11:08:17.677 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-27T11:08:17.677 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:17.677 INFO:teuthology.orchestra.run.smithi097.stdout: "req_renamesnap_latency": { 2024-03-27T11:08:17.677 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-27T11:08:17.677 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-27T11:08:17.677 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-27T11:08:17.677 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:17.677 INFO:teuthology.orchestra.run.smithi097.stdout: "req_snapdiff_latency": { 2024-03-27T11:08:17.677 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-27T11:08:17.678 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-27T11:08:17.678 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-27T11:08:17.678 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:17.678 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rmdir_latency": { 2024-03-27T11:08:17.678 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-27T11:08:17.678 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-27T11:08:17.678 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-27T11:08:17.678 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:17.678 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rmsnap_latency": { 2024-03-27T11:08:17.678 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-27T11:08:17.678 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-27T11:08:17.678 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-27T11:08:17.678 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:17.679 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rmxattr_latency": { 2024-03-27T11:08:17.679 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-27T11:08:17.679 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-27T11:08:17.679 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-27T11:08:17.679 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:17.679 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setattr_latency": { 2024-03-27T11:08:17.679 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-27T11:08:17.679 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-27T11:08:17.679 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-27T11:08:17.679 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:17.679 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setdirlayout_latency": { 2024-03-27T11:08:17.679 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-27T11:08:17.679 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-27T11:08:17.679 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-27T11:08:17.680 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:17.680 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setfilelock_latency": { 2024-03-27T11:08:17.680 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-27T11:08:17.680 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-27T11:08:17.680 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-27T11:08:17.680 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:17.680 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setlayout_latency": { 2024-03-27T11:08:17.680 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-27T11:08:17.680 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-27T11:08:17.680 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-27T11:08:17.680 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:17.680 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setxattr_latency": { 2024-03-27T11:08:17.680 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-27T11:08:17.681 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-27T11:08:17.681 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-27T11:08:17.681 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:17.681 INFO:teuthology.orchestra.run.smithi097.stdout: "req_symlink_latency": { 2024-03-27T11:08:17.681 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-27T11:08:17.681 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-27T11:08:17.681 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-27T11:08:17.681 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:17.681 INFO:teuthology.orchestra.run.smithi097.stdout: "req_unlink_latency": { 2024-03-27T11:08:17.681 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-27T11:08:17.681 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-27T11:08:17.681 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-27T11:08:17.681 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:17.681 INFO:teuthology.orchestra.run.smithi097.stdout: "cap_revoke_eviction": 0, 2024-03-27T11:08:17.682 INFO:teuthology.orchestra.run.smithi097.stdout: "cap_acquisition_throttle": 0, 2024-03-27T11:08:17.682 INFO:teuthology.orchestra.run.smithi097.stdout: "req_getvxattr_latency": { 2024-03-27T11:08:17.682 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-27T11:08:17.682 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-27T11:08:17.682 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-27T11:08:17.682 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:17.682 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:17.682 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_sessions": { 2024-03-27T11:08:17.682 INFO:teuthology.orchestra.run.smithi097.stdout: "session_count": 1, 2024-03-27T11:08:17.682 INFO:teuthology.orchestra.run.smithi097.stdout: "session_add": 1, 2024-03-27T11:08:17.682 INFO:teuthology.orchestra.run.smithi097.stdout: "session_remove": 0, 2024-03-27T11:08:17.682 INFO:teuthology.orchestra.run.smithi097.stdout: "sessions_open": 1, 2024-03-27T11:08:17.683 INFO:teuthology.orchestra.run.smithi097.stdout: "sessions_stale": 0, 2024-03-27T11:08:17.683 INFO:teuthology.orchestra.run.smithi097.stdout: "total_load": 0, 2024-03-27T11:08:17.683 INFO:teuthology.orchestra.run.smithi097.stdout: "average_load": 0, 2024-03-27T11:08:17.683 INFO:teuthology.orchestra.run.smithi097.stdout: "avg_session_uptime": 2129, 2024-03-27T11:08:17.683 INFO:teuthology.orchestra.run.smithi097.stdout: "mdthresh_evicted": 0 2024-03-27T11:08:17.683 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:17.683 INFO:teuthology.orchestra.run.smithi097.stdout: "mempool": { 2024-03-27T11:08:17.683 INFO:teuthology.orchestra.run.smithi097.stdout: "bloom_filter_bytes": 0, 2024-03-27T11:08:17.683 INFO:teuthology.orchestra.run.smithi097.stdout: "bloom_filter_items": 0, 2024-03-27T11:08:17.683 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_alloc_bytes": 0, 2024-03-27T11:08:17.683 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_alloc_items": 0, 2024-03-27T11:08:17.683 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_data_bytes": 0, 2024-03-27T11:08:17.683 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_data_items": 0, 2024-03-27T11:08:17.683 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-27T11:08:17.684 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_onode_items": 0, 2024-03-27T11:08:17.684 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-27T11:08:17.684 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_meta_items": 0, 2024-03-27T11:08:17.684 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_other_bytes": 0, 2024-03-27T11:08:17.684 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_other_items": 0, 2024-03-27T11:08:17.684 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-27T11:08:17.684 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_buffer_items": 0, 2024-03-27T11:08:17.684 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_extent_bytes": 0, 2024-03-27T11:08:17.684 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_extent_items": 0, 2024-03-27T11:08:17.684 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_blob_bytes": 0, 2024-03-27T11:08:17.684 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_blob_items": 0, 2024-03-27T11:08:17.684 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-27T11:08:17.684 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_shared_blob_items": 0, 2024-03-27T11:08:17.684 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-27T11:08:17.685 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_inline_bl_items": 0, 2024-03-27T11:08:17.685 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_bytes": 0, 2024-03-27T11:08:17.685 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_items": 0, 2024-03-27T11:08:17.685 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_txc_bytes": 0, 2024-03-27T11:08:17.685 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_txc_items": 0, 2024-03-27T11:08:17.685 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-27T11:08:17.685 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_deferred_items": 0, 2024-03-27T11:08:17.685 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_bytes": 0, 2024-03-27T11:08:17.685 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_items": 0, 2024-03-27T11:08:17.685 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_bytes": 0, 2024-03-27T11:08:17.685 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_items": 0, 2024-03-27T11:08:17.685 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_reader_bytes": 0, 2024-03-27T11:08:17.685 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_reader_items": 0, 2024-03-27T11:08:17.685 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_writer_bytes": 0, 2024-03-27T11:08:17.686 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_writer_items": 0, 2024-03-27T11:08:17.686 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_anon_bytes": 189423, 2024-03-27T11:08:17.686 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_anon_items": 51, 2024-03-27T11:08:17.686 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_meta_bytes": 0, 2024-03-27T11:08:17.686 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_meta_items": 0, 2024-03-27T11:08:17.686 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_bytes": 0, 2024-03-27T11:08:17.686 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_items": 0, 2024-03-27T11:08:17.686 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mapbl_bytes": 0, 2024-03-27T11:08:17.686 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mapbl_items": 0, 2024-03-27T11:08:17.686 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pglog_bytes": 0, 2024-03-27T11:08:17.686 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pglog_items": 0, 2024-03-27T11:08:17.686 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_bytes": 7528, 2024-03-27T11:08:17.686 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_items": 158, 2024-03-27T11:08:17.686 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_mapping_bytes": 0, 2024-03-27T11:08:17.687 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_mapping_items": 0, 2024-03-27T11:08:17.687 INFO:teuthology.orchestra.run.smithi097.stdout: "pgmap_bytes": 0, 2024-03-27T11:08:17.687 INFO:teuthology.orchestra.run.smithi097.stdout: "pgmap_items": 0, 2024-03-27T11:08:17.687 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_co_bytes": 8464, 2024-03-27T11:08:17.687 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_co_items": 13, 2024-03-27T11:08:17.687 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_1_bytes": 0, 2024-03-27T11:08:17.687 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_1_items": 0, 2024-03-27T11:08:17.687 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_2_bytes": 0, 2024-03-27T11:08:17.687 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_2_items": 0 2024-03-27T11:08:17.687 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:17.687 INFO:teuthology.orchestra.run.smithi097.stdout: "objecter": { 2024-03-27T11:08:17.687 INFO:teuthology.orchestra.run.smithi097.stdout: "op_active": 0, 2024-03-27T11:08:17.687 INFO:teuthology.orchestra.run.smithi097.stdout: "op_laggy": 0, 2024-03-27T11:08:17.688 INFO:teuthology.orchestra.run.smithi097.stdout: "op_send": 2405, 2024-03-27T11:08:17.688 INFO:teuthology.orchestra.run.smithi097.stdout: "op_send_bytes": 21, 2024-03-27T11:08:17.688 INFO:teuthology.orchestra.run.smithi097.stdout: "op_resend": 3, 2024-03-27T11:08:17.688 INFO:teuthology.orchestra.run.smithi097.stdout: "op_reply": 2405, 2024-03-27T11:08:17.688 INFO:teuthology.orchestra.run.smithi097.stdout: "op_latency": { 2024-03-27T11:08:17.688 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 2405, 2024-03-27T11:08:17.688 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 2.091964453, 2024-03-27T11:08:17.688 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000869839 2024-03-27T11:08:17.688 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:17.688 INFO:teuthology.orchestra.run.smithi097.stdout: "op_inflight": 0, 2024-03-27T11:08:17.688 INFO:teuthology.orchestra.run.smithi097.stdout: "oplen_avg": { 2024-03-27T11:08:17.688 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 2405, 2024-03-27T11:08:17.688 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 2406 2024-03-27T11:08:17.688 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:17.688 INFO:teuthology.orchestra.run.smithi097.stdout: "op": 2405, 2024-03-27T11:08:17.689 INFO:teuthology.orchestra.run.smithi097.stdout: "op_r": 2405, 2024-03-27T11:08:17.689 INFO:teuthology.orchestra.run.smithi097.stdout: "op_w": 0, 2024-03-27T11:08:17.689 INFO:teuthology.orchestra.run.smithi097.stdout: "op_rmw": 0, 2024-03-27T11:08:17.689 INFO:teuthology.orchestra.run.smithi097.stdout: "op_pg": 0, 2024-03-27T11:08:17.689 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_stat": 1554, 2024-03-27T11:08:17.689 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_create": 0, 2024-03-27T11:08:17.689 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_read": 849, 2024-03-27T11:08:17.689 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_write": 0, 2024-03-27T11:08:17.689 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_writefull": 0, 2024-03-27T11:08:17.689 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_writesame": 0, 2024-03-27T11:08:17.689 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_append": 0, 2024-03-27T11:08:17.689 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_zero": 0, 2024-03-27T11:08:17.689 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_truncate": 0, 2024-03-27T11:08:17.689 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_delete": 0, 2024-03-27T11:08:17.690 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_mapext": 0, 2024-03-27T11:08:17.690 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_sparse_read": 0, 2024-03-27T11:08:17.690 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_clonerange": 0, 2024-03-27T11:08:17.690 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_getxattr": 1, 2024-03-27T11:08:17.690 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_setxattr": 0, 2024-03-27T11:08:17.690 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_cmpxattr": 0, 2024-03-27T11:08:17.690 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_rmxattr": 0, 2024-03-27T11:08:17.690 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_resetxattrs": 0, 2024-03-27T11:08:17.690 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_call": 0, 2024-03-27T11:08:17.690 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_watch": 0, 2024-03-27T11:08:17.690 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_notify": 0, 2024-03-27T11:08:17.690 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_src_cmpxattr": 0, 2024-03-27T11:08:17.690 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_pgls": 0, 2024-03-27T11:08:17.690 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_pgls_filter": 0, 2024-03-27T11:08:17.691 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_other": 0, 2024-03-27T11:08:17.691 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_active": 0, 2024-03-27T11:08:17.691 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_send": 0, 2024-03-27T11:08:17.691 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_resend": 0, 2024-03-27T11:08:17.691 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_ping": 0, 2024-03-27T11:08:17.691 INFO:teuthology.orchestra.run.smithi097.stdout: "poolop_active": 0, 2024-03-27T11:08:17.691 INFO:teuthology.orchestra.run.smithi097.stdout: "poolop_send": 0, 2024-03-27T11:08:17.691 INFO:teuthology.orchestra.run.smithi097.stdout: "poolop_resend": 0, 2024-03-27T11:08:17.691 INFO:teuthology.orchestra.run.smithi097.stdout: "poolstat_active": 0, 2024-03-27T11:08:17.691 INFO:teuthology.orchestra.run.smithi097.stdout: "poolstat_send": 0, 2024-03-27T11:08:17.691 INFO:teuthology.orchestra.run.smithi097.stdout: "poolstat_resend": 0, 2024-03-27T11:08:17.691 INFO:teuthology.orchestra.run.smithi097.stdout: "statfs_active": 0, 2024-03-27T11:08:17.691 INFO:teuthology.orchestra.run.smithi097.stdout: "statfs_send": 0, 2024-03-27T11:08:17.691 INFO:teuthology.orchestra.run.smithi097.stdout: "statfs_resend": 0, 2024-03-27T11:08:17.692 INFO:teuthology.orchestra.run.smithi097.stdout: "command_active": 0, 2024-03-27T11:08:17.692 INFO:teuthology.orchestra.run.smithi097.stdout: "command_send": 0, 2024-03-27T11:08:17.692 INFO:teuthology.orchestra.run.smithi097.stdout: "command_resend": 0, 2024-03-27T11:08:17.692 INFO:teuthology.orchestra.run.smithi097.stdout: "map_epoch": 92, 2024-03-27T11:08:17.692 INFO:teuthology.orchestra.run.smithi097.stdout: "map_full": 0, 2024-03-27T11:08:17.692 INFO:teuthology.orchestra.run.smithi097.stdout: "map_inc": 2, 2024-03-27T11:08:17.692 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_sessions": 5, 2024-03-27T11:08:17.692 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_session_open": 5, 2024-03-27T11:08:17.692 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_session_close": 0, 2024-03-27T11:08:17.692 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_laggy": 0, 2024-03-27T11:08:17.692 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_wr": 0, 2024-03-27T11:08:17.692 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_rd": 2, 2024-03-27T11:08:17.692 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_del": 0 2024-03-27T11:08:17.692 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:17.692 INFO:teuthology.orchestra.run.smithi097.stdout: "oft": { 2024-03-27T11:08:17.693 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_objs": 0, 2024-03-27T11:08:17.693 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_kv_pairs": 0, 2024-03-27T11:08:17.693 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_updates": 0, 2024-03-27T11:08:17.693 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_removes": 0 2024-03-27T11:08:17.693 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:17.693 INFO:teuthology.orchestra.run.smithi097.stdout: "osd-slow-ops": { 2024-03-27T11:08:17.693 INFO:teuthology.orchestra.run.smithi097.stdout: "slow_ops_count": 0 2024-03-27T11:08:17.693 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:17.693 INFO:teuthology.orchestra.run.smithi097.stdout: "purge_queue": { 2024-03-27T11:08:17.693 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing_ops": 0, 2024-03-27T11:08:17.693 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing_ops_high_water": 0, 2024-03-27T11:08:17.693 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing": 0, 2024-03-27T11:08:17.693 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing_high_water": 0, 2024-03-27T11:08:17.693 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executed_ops": 0, 2024-03-27T11:08:17.693 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executed": 0, 2024-03-27T11:08:17.694 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_item_in_journal": 0 2024-03-27T11:08:17.694 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:17.694 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-27T11:08:17.694 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-03-27T11:08:17.694 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 104857600, 2024-03-27T11:08:17.694 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-03-27T11:08:17.694 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 2655, 2024-03-27T11:08:17.694 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 985628, 2024-03-27T11:08:17.694 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-03-27T11:08:17.694 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 2655, 2024-03-27T11:08:17.694 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2024-03-27T11:08:17.694 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2024-03-27T11:08:17.694 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 2655, 2024-03-27T11:08:17.694 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 985628, 2024-03-27T11:08:17.695 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-03-27T11:08:17.695 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-27T11:08:17.695 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-27T11:08:17.695 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-27T11:08:17.695 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:17.695 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:17.695 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-objecter_bytes": { 2024-03-27T11:08:17.695 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-03-27T11:08:17.695 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 104857600, 2024-03-27T11:08:17.695 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-03-27T11:08:17.695 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 0, 2024-03-27T11:08:17.695 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 0, 2024-03-27T11:08:17.695 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-03-27T11:08:17.695 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 0, 2024-03-27T11:08:17.696 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 2405, 2024-03-27T11:08:17.696 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 1086, 2024-03-27T11:08:17.696 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 2405, 2024-03-27T11:08:17.696 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 1086, 2024-03-27T11:08:17.696 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-03-27T11:08:17.696 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-27T11:08:17.696 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-27T11:08:17.696 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-27T11:08:17.696 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:17.696 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:17.696 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-objecter_ops": { 2024-03-27T11:08:17.696 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-03-27T11:08:17.696 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 1024, 2024-03-27T11:08:17.696 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-03-27T11:08:17.696 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 0, 2024-03-27T11:08:17.696 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 0, 2024-03-27T11:08:17.697 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-03-27T11:08:17.697 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 0, 2024-03-27T11:08:17.697 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 2405, 2024-03-27T11:08:17.697 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 2405, 2024-03-27T11:08:17.697 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 2405, 2024-03-27T11:08:17.697 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 2405, 2024-03-27T11:08:17.697 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-03-27T11:08:17.697 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-27T11:08:17.697 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-27T11:08:17.697 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-27T11:08:17.697 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:17.697 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:17.697 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-write_buf_throttle": { 2024-03-27T11:08:17.697 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-03-27T11:08:17.697 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 3758096384, 2024-03-27T11:08:17.698 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-03-27T11:08:17.698 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 0, 2024-03-27T11:08:17.698 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 0, 2024-03-27T11:08:17.698 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-03-27T11:08:17.698 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 0, 2024-03-27T11:08:17.698 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2024-03-27T11:08:17.698 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2024-03-27T11:08:17.698 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 0, 2024-03-27T11:08:17.698 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 0, 2024-03-27T11:08:17.698 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-03-27T11:08:17.698 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-27T11:08:17.698 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-27T11:08:17.698 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-27T11:08:17.698 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:17.699 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:17.699 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-write_buf_throttle-0x55c948482780": { 2024-03-27T11:08:17.699 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-03-27T11:08:17.699 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 3758096384, 2024-03-27T11:08:17.699 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-03-27T11:08:17.699 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 0, 2024-03-27T11:08:17.699 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 0, 2024-03-27T11:08:17.699 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-03-27T11:08:17.699 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 0, 2024-03-27T11:08:17.699 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2024-03-27T11:08:17.699 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2024-03-27T11:08:17.699 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 0, 2024-03-27T11:08:17.699 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 0, 2024-03-27T11:08:17.699 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-03-27T11:08:17.700 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-27T11:08:17.700 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-27T11:08:17.700 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-27T11:08:17.700 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:17.700 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:17.700 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-03-27T11:08:17.763 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:08:17 smithi114 ceph-mon[35338]: pgmap v790: 97 pgs: 97 active+clean; 9.0 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 23 MiB/s wr, 444 op/s 2024-03-27T11:08:17.888 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:08:17 smithi097 ceph-mon[31859]: pgmap v790: 97 pgs: 97 active+clean; 9.0 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 23 MiB/s wr, 444 op/s 2024-03-27T11:08:17.923 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:08:17 smithi177 ceph-mon[35294]: pgmap v790: 97 pgs: 97 active+clean; 9.0 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 23 MiB/s wr, 444 op/s 2024-03-27T11:08:18.715 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-03-27T11:08:18.715 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-03-27T11:08:18.716 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update=0 2024-03-27T11:08:18.716 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update_receipt=0 2024-03-27T11:08:18.716 DEBUG:tasks.check_counter:Getting stats from d 2024-03-27T11:08:18.716 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell --fsid adca88c4-ec26-11ee-95d0-87774f69a715 -- ceph daemon mds.d perf dump 2024-03-27T11:08:19.172 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T11:08:19.570 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:08:19 smithi097 ceph-mon[31859]: pgmap v791: 97 pgs: 97 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 17 MiB/s wr, 357 op/s 2024-03-27T11:08:19.763 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:08:19 smithi114 ceph-mon[35338]: pgmap v791: 97 pgs: 97 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 17 MiB/s wr, 357 op/s 2024-03-27T11:08:19.923 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:08:19 smithi177 ceph-mon[35294]: pgmap v791: 97 pgs: 97 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 17 MiB/s wr, 357 op/s 2024-03-27T11:08:21.407 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-03-27T11:08:21.407 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-0": { 2024-03-27T11:08:21.407 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 246, 2024-03-27T11:08:21.407 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 241, 2024-03-27T11:08:21.407 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 489120, 2024-03-27T11:08:21.407 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 147712, 2024-03-27T11:08:21.407 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 4, 2024-03-27T11:08:21.407 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 0, 2024-03-27T11:08:21.407 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.072794774, 2024-03-27T11:08:21.408 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.031610436, 2024-03-27T11:08:21.408 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.033371630, 2024-03-27T11:08:21.408 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.008242331, 2024-03-27T11:08:21.408 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-03-27T11:08:21.408 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 241, 2024-03-27T11:08:21.408 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.030234572, 2024-03-27T11:08:21.408 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000125454 2024-03-27T11:08:21.408 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:21.408 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-03-27T11:08:21.408 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-27T11:08:21.408 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-27T11:08:21.408 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-27T11:08:21.408 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:21.408 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 489120, 2024-03-27T11:08:21.408 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 147712 2024-03-27T11:08:21.409 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:21.409 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-1": { 2024-03-27T11:08:21.409 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 1, 2024-03-27T11:08:21.409 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 182, 2024-03-27T11:08:21.409 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 160, 2024-03-27T11:08:21.409 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 109712, 2024-03-27T11:08:21.409 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 3, 2024-03-27T11:08:21.409 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 2, 2024-03-27T11:08:21.409 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.025789275, 2024-03-27T11:08:21.409 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.020047647, 2024-03-27T11:08:21.409 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.003217572, 2024-03-27T11:08:21.409 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-27T11:08:21.409 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-03-27T11:08:21.409 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 180, 2024-03-27T11:08:21.410 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.019261728, 2024-03-27T11:08:21.410 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000107009 2024-03-27T11:08:21.410 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:21.410 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-03-27T11:08:21.410 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-27T11:08:21.410 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-27T11:08:21.410 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-27T11:08:21.410 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:21.410 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-27T11:08:21.410 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 109712 2024-03-27T11:08:21.410 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:21.410 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-2": { 2024-03-27T11:08:21.411 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 0, 2024-03-27T11:08:21.411 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 2, 2024-03-27T11:08:21.411 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 0, 2024-03-27T11:08:21.411 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 0, 2024-03-27T11:08:21.411 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 2, 2024-03-27T11:08:21.411 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 1, 2024-03-27T11:08:21.411 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.000954145, 2024-03-27T11:08:21.411 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.000000000, 2024-03-27T11:08:21.411 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.000775318, 2024-03-27T11:08:21.411 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-27T11:08:21.411 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-03-27T11:08:21.411 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-27T11:08:21.411 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-27T11:08:21.411 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-27T11:08:21.411 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:21.412 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-03-27T11:08:21.412 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-27T11:08:21.412 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-27T11:08:21.412 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-27T11:08:21.412 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:21.412 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-27T11:08:21.412 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 0 2024-03-27T11:08:21.412 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:21.412 INFO:teuthology.orchestra.run.smithi097.stdout: "cct": { 2024-03-27T11:08:21.412 INFO:teuthology.orchestra.run.smithi097.stdout: "total_workers": 0, 2024-03-27T11:08:21.412 INFO:teuthology.orchestra.run.smithi097.stdout: "unhealthy_workers": 0 2024-03-27T11:08:21.412 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:21.412 INFO:teuthology.orchestra.run.smithi097.stdout: "mempool": { 2024-03-27T11:08:21.412 INFO:teuthology.orchestra.run.smithi097.stdout: "bloom_filter_bytes": 0, 2024-03-27T11:08:21.412 INFO:teuthology.orchestra.run.smithi097.stdout: "bloom_filter_items": 0, 2024-03-27T11:08:21.413 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_alloc_bytes": 0, 2024-03-27T11:08:21.413 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_alloc_items": 0, 2024-03-27T11:08:21.413 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_data_bytes": 0, 2024-03-27T11:08:21.413 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_data_items": 0, 2024-03-27T11:08:21.413 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-27T11:08:21.413 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_onode_items": 0, 2024-03-27T11:08:21.413 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-27T11:08:21.413 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_meta_items": 0, 2024-03-27T11:08:21.413 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_other_bytes": 0, 2024-03-27T11:08:21.413 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_other_items": 0, 2024-03-27T11:08:21.413 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-27T11:08:21.413 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_buffer_items": 0, 2024-03-27T11:08:21.413 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_extent_bytes": 0, 2024-03-27T11:08:21.414 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_extent_items": 0, 2024-03-27T11:08:21.414 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_blob_bytes": 0, 2024-03-27T11:08:21.414 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_blob_items": 0, 2024-03-27T11:08:21.414 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-27T11:08:21.414 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_shared_blob_items": 0, 2024-03-27T11:08:21.414 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-27T11:08:21.414 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_inline_bl_items": 0, 2024-03-27T11:08:21.414 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_bytes": 0, 2024-03-27T11:08:21.414 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_items": 0, 2024-03-27T11:08:21.414 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_txc_bytes": 0, 2024-03-27T11:08:21.414 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_txc_items": 0, 2024-03-27T11:08:21.414 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-27T11:08:21.414 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_deferred_items": 0, 2024-03-27T11:08:21.415 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_bytes": 0, 2024-03-27T11:08:21.415 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_items": 0, 2024-03-27T11:08:21.415 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_bytes": 0, 2024-03-27T11:08:21.415 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_items": 0, 2024-03-27T11:08:21.415 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_reader_bytes": 0, 2024-03-27T11:08:21.415 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_reader_items": 0, 2024-03-27T11:08:21.415 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_writer_bytes": 0, 2024-03-27T11:08:21.415 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_writer_items": 0, 2024-03-27T11:08:21.415 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_anon_bytes": 149001, 2024-03-27T11:08:21.415 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_anon_items": 31, 2024-03-27T11:08:21.415 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_meta_bytes": 0, 2024-03-27T11:08:21.415 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_meta_items": 0, 2024-03-27T11:08:21.415 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_bytes": 0, 2024-03-27T11:08:21.415 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_items": 0, 2024-03-27T11:08:21.415 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mapbl_bytes": 0, 2024-03-27T11:08:21.415 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mapbl_items": 0, 2024-03-27T11:08:21.416 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pglog_bytes": 0, 2024-03-27T11:08:21.416 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pglog_items": 0, 2024-03-27T11:08:21.416 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_bytes": 0, 2024-03-27T11:08:21.416 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_items": 0, 2024-03-27T11:08:21.416 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_mapping_bytes": 0, 2024-03-27T11:08:21.416 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_mapping_items": 0, 2024-03-27T11:08:21.416 INFO:teuthology.orchestra.run.smithi097.stdout: "pgmap_bytes": 0, 2024-03-27T11:08:21.416 INFO:teuthology.orchestra.run.smithi097.stdout: "pgmap_items": 0, 2024-03-27T11:08:21.416 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_co_bytes": 928, 2024-03-27T11:08:21.416 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_co_items": 2, 2024-03-27T11:08:21.416 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_1_bytes": 0, 2024-03-27T11:08:21.416 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_1_items": 0, 2024-03-27T11:08:21.416 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_2_bytes": 0, 2024-03-27T11:08:21.416 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_2_items": 0 2024-03-27T11:08:21.416 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:21.416 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-27T11:08:21.417 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-03-27T11:08:21.417 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 104857600, 2024-03-27T11:08:21.417 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-03-27T11:08:21.417 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 243, 2024-03-27T11:08:21.417 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 451894, 2024-03-27T11:08:21.417 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-03-27T11:08:21.417 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 243, 2024-03-27T11:08:21.417 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2024-03-27T11:08:21.417 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2024-03-27T11:08:21.417 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 243, 2024-03-27T11:08:21.417 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 451894, 2024-03-27T11:08:21.417 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-03-27T11:08:21.417 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-27T11:08:21.417 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-27T11:08:21.417 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-27T11:08:21.417 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:21.418 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:21.418 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-03-27T11:08:21.763 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:08:21 smithi114 ceph-mon[35338]: pgmap v792: 97 pgs: 97 active+clean; 7.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 9.3 MiB/s wr, 244 op/s 2024-03-27T11:08:21.828 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:08:21 smithi097 ceph-mon[31859]: pgmap v792: 97 pgs: 97 active+clean; 7.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 9.3 MiB/s wr, 244 op/s 2024-03-27T11:08:21.923 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:08:21 smithi177 ceph-mon[35294]: pgmap v792: 97 pgs: 97 active+clean; 7.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 9.3 MiB/s wr, 244 op/s 2024-03-27T11:08:22.241 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-03-27T11:08:22.241 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-03-27T11:08:22.241 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-03-27T11:08:22.241 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-03-27T11:08:22.241 DEBUG:tasks.check_counter:Getting stats from g 2024-03-27T11:08:22.242 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell --fsid adca88c4-ec26-11ee-95d0-87774f69a715 -- ceph daemon mds.g perf dump 2024-03-27T11:08:22.711 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T11:08:23.772 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:08:23 smithi097 ceph-mon[31859]: pgmap v793: 97 pgs: 97 active+clean; 6.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 1.3 MiB/s wr, 108 op/s 2024-03-27T11:08:23.923 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:08:23 smithi177 ceph-mon[35294]: pgmap v793: 97 pgs: 97 active+clean; 6.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 1.3 MiB/s wr, 108 op/s 2024-03-27T11:08:24.013 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:08:23 smithi114 ceph-mon[35338]: pgmap v793: 97 pgs: 97 active+clean; 6.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 1.3 MiB/s wr, 108 op/s 2024-03-27T11:08:25.026 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-03-27T11:08:25.027 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-0": { 2024-03-27T11:08:25.027 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 1103, 2024-03-27T11:08:25.027 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 1092, 2024-03-27T11:08:25.027 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 841088, 2024-03-27T11:08:25.027 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 415866, 2024-03-27T11:08:25.027 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 5, 2024-03-27T11:08:25.027 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 1, 2024-03-27T11:08:25.027 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.240094344, 2024-03-27T11:08:25.027 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.111836148, 2024-03-27T11:08:25.027 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.060826098, 2024-03-27T11:08:25.027 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.052703942, 2024-03-27T11:08:25.027 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-03-27T11:08:25.028 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 1091, 2024-03-27T11:08:25.028 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.128039874, 2024-03-27T11:08:25.028 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000117360 2024-03-27T11:08:25.028 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:25.028 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-03-27T11:08:25.028 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-27T11:08:25.028 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-27T11:08:25.028 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-27T11:08:25.028 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:25.028 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 555152, 2024-03-27T11:08:25.028 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 145248 2024-03-27T11:08:25.028 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:25.028 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-1": { 2024-03-27T11:08:25.028 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 1576, 2024-03-27T11:08:25.028 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 1751, 2024-03-27T11:08:25.029 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 404379, 2024-03-27T11:08:25.029 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 894707, 2024-03-27T11:08:25.029 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 5, 2024-03-27T11:08:25.029 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 4, 2024-03-27T11:08:25.029 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.207677708, 2024-03-27T11:08:25.029 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.076231915, 2024-03-27T11:08:25.029 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.049070610, 2024-03-27T11:08:25.029 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.067004848, 2024-03-27T11:08:25.029 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-03-27T11:08:25.029 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 1750, 2024-03-27T11:08:25.029 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.169269881, 2024-03-27T11:08:25.029 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000096725 2024-03-27T11:08:25.029 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:25.030 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-03-27T11:08:25.030 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-27T11:08:25.030 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-27T11:08:25.030 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-27T11:08:25.030 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:25.030 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-03-27T11:08:25.030 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 395072 2024-03-27T11:08:25.030 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:25.030 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-2": { 2024-03-27T11:08:25.030 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 4, 2024-03-27T11:08:25.030 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 6, 2024-03-27T11:08:25.030 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 1629, 2024-03-27T11:08:25.030 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 1284, 2024-03-27T11:08:25.031 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 5, 2024-03-27T11:08:25.031 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 4, 2024-03-27T11:08:25.031 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.003270463, 2024-03-27T11:08:25.031 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.000145226, 2024-03-27T11:08:25.031 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.002859484, 2024-03-27T11:08:25.031 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.000101453, 2024-03-27T11:08:25.031 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-03-27T11:08:25.031 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 4, 2024-03-27T11:08:25.031 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.005081231, 2024-03-27T11:08:25.031 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.001270307 2024-03-27T11:08:25.031 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:25.031 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-03-27T11:08:25.031 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-27T11:08:25.032 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-27T11:08:25.032 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-27T11:08:25.032 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:25.032 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-27T11:08:25.032 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 0 2024-03-27T11:08:25.032 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:25.032 INFO:teuthology.orchestra.run.smithi097.stdout: "cct": { 2024-03-27T11:08:25.032 INFO:teuthology.orchestra.run.smithi097.stdout: "total_workers": 1, 2024-03-27T11:08:25.032 INFO:teuthology.orchestra.run.smithi097.stdout: "unhealthy_workers": 0 2024-03-27T11:08:25.032 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:25.032 INFO:teuthology.orchestra.run.smithi097.stdout: "finisher-MDSRank": { 2024-03-27T11:08:25.032 INFO:teuthology.orchestra.run.smithi097.stdout: "queue_len": 0, 2024-03-27T11:08:25.032 INFO:teuthology.orchestra.run.smithi097.stdout: "complete_latency": { 2024-03-27T11:08:25.032 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 3955, 2024-03-27T11:08:25.033 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.150880682, 2024-03-27T11:08:25.033 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000038149 2024-03-27T11:08:25.033 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:25.033 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:25.033 INFO:teuthology.orchestra.run.smithi097.stdout: "finisher-PurgeQueue": { 2024-03-27T11:08:25.033 INFO:teuthology.orchestra.run.smithi097.stdout: "queue_len": 0, 2024-03-27T11:08:25.033 INFO:teuthology.orchestra.run.smithi097.stdout: "complete_latency": { 2024-03-27T11:08:25.033 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-27T11:08:25.033 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-27T11:08:25.033 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-27T11:08:25.033 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:25.033 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:25.033 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": { 2024-03-27T11:08:25.033 INFO:teuthology.orchestra.run.smithi097.stdout: "request": 0, 2024-03-27T11:08:25.034 INFO:teuthology.orchestra.run.smithi097.stdout: "reply": 0, 2024-03-27T11:08:25.034 INFO:teuthology.orchestra.run.smithi097.stdout: "reply_latency": { 2024-03-27T11:08:25.034 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-27T11:08:25.034 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-27T11:08:25.034 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-27T11:08:25.034 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:25.034 INFO:teuthology.orchestra.run.smithi097.stdout: "slow_reply": 0, 2024-03-27T11:08:25.034 INFO:teuthology.orchestra.run.smithi097.stdout: "forward": 0, 2024-03-27T11:08:25.034 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_fetch_complete": 0, 2024-03-27T11:08:25.034 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_fetch_keys": 0, 2024-03-27T11:08:25.034 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_commit": 0, 2024-03-27T11:08:25.034 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_split": 0, 2024-03-27T11:08:25.034 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_merge": 0, 2024-03-27T11:08:25.035 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes": 0, 2024-03-27T11:08:25.035 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_top": 0, 2024-03-27T11:08:25.035 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_bottom": 0, 2024-03-27T11:08:25.035 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_pin_tail": 0, 2024-03-27T11:08:25.035 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_pinned": 0, 2024-03-27T11:08:25.035 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_expired": 0, 2024-03-27T11:08:25.035 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_with_caps": 0, 2024-03-27T11:08:25.035 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": 0, 2024-03-27T11:08:25.035 INFO:teuthology.orchestra.run.smithi097.stdout: "subtrees": 1, 2024-03-27T11:08:25.035 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse": 0, 2024-03-27T11:08:25.035 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_hit": 0, 2024-03-27T11:08:25.035 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_forward": 0, 2024-03-27T11:08:25.035 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_discover": 0, 2024-03-27T11:08:25.035 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_dir_fetch": 0, 2024-03-27T11:08:25.035 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_remote_ino": 0, 2024-03-27T11:08:25.036 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_lock": 0, 2024-03-27T11:08:25.036 INFO:teuthology.orchestra.run.smithi097.stdout: "load_cent": 0, 2024-03-27T11:08:25.036 INFO:teuthology.orchestra.run.smithi097.stdout: "q": 0, 2024-03-27T11:08:25.036 INFO:teuthology.orchestra.run.smithi097.stdout: "exported": 0, 2024-03-27T11:08:25.036 INFO:teuthology.orchestra.run.smithi097.stdout: "exported_inodes": 0, 2024-03-27T11:08:25.036 INFO:teuthology.orchestra.run.smithi097.stdout: "imported": 0, 2024-03-27T11:08:25.036 INFO:teuthology.orchestra.run.smithi097.stdout: "imported_inodes": 0, 2024-03-27T11:08:25.036 INFO:teuthology.orchestra.run.smithi097.stdout: "openino_dir_fetch": 0, 2024-03-27T11:08:25.036 INFO:teuthology.orchestra.run.smithi097.stdout: "openino_backtrace_fetch": 0, 2024-03-27T11:08:25.036 INFO:teuthology.orchestra.run.smithi097.stdout: "openino_peer_discover": 0, 2024-03-27T11:08:25.036 INFO:teuthology.orchestra.run.smithi097.stdout: "root_rfiles": 0, 2024-03-27T11:08:25.036 INFO:teuthology.orchestra.run.smithi097.stdout: "root_rbytes": 0, 2024-03-27T11:08:25.036 INFO:teuthology.orchestra.run.smithi097.stdout: "root_rsnaps": 0, 2024-03-27T11:08:25.036 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_backtrace_fetch": 0, 2024-03-27T11:08:25.037 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_set_tag": 0, 2024-03-27T11:08:25.037 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_backtrace_repaired": 0, 2024-03-27T11:08:25.037 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_inotable_repaired": 0, 2024-03-27T11:08:25.037 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_dir_inodes": 0, 2024-03-27T11:08:25.037 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_dir_base_inodes": 0, 2024-03-27T11:08:25.037 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_dirfrag_rstats": 0, 2024-03-27T11:08:25.037 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_file_inodes": 0, 2024-03-27T11:08:25.037 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_inode_file_caps": 0, 2024-03-27T11:08:25.037 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_revoke": 0, 2024-03-27T11:08:25.037 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_grant": 0, 2024-03-27T11:08:25.037 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_trunc": 0, 2024-03-27T11:08:25.037 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-27T11:08:25.037 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-27T11:08:25.037 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_caps": 0, 2024-03-27T11:08:25.037 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_caps_dirty": 0, 2024-03-27T11:08:25.038 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_cap_release": 0, 2024-03-27T11:08:25.038 INFO:teuthology.orchestra.run.smithi097.stdout: "process_request_cap_release": 0 2024-03-27T11:08:25.038 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:25.038 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_cache": { 2024-03-27T11:08:25.038 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_update": 0, 2024-03-27T11:08:25.038 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_update_receipt": 0, 2024-03-27T11:08:25.038 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_try_discover": 0, 2024-03-27T11:08:25.038 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_send_discover": 0, 2024-03-27T11:08:25.038 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_handle_discover": 0, 2024-03-27T11:08:25.038 INFO:teuthology.orchestra.run.smithi097.stdout: "num_strays": 0, 2024-03-27T11:08:25.038 INFO:teuthology.orchestra.run.smithi097.stdout: "num_strays_delayed": 0, 2024-03-27T11:08:25.038 INFO:teuthology.orchestra.run.smithi097.stdout: "num_strays_enqueuing": 0, 2024-03-27T11:08:25.038 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_created": 0, 2024-03-27T11:08:25.038 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_enqueued": 0, 2024-03-27T11:08:25.038 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_reintegrated": 0, 2024-03-27T11:08:25.039 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_migrated": 0, 2024-03-27T11:08:25.039 INFO:teuthology.orchestra.run.smithi097.stdout: "num_recovering_processing": 0, 2024-03-27T11:08:25.039 INFO:teuthology.orchestra.run.smithi097.stdout: "num_recovering_enqueued": 0, 2024-03-27T11:08:25.039 INFO:teuthology.orchestra.run.smithi097.stdout: "num_recovering_prioritized": 0, 2024-03-27T11:08:25.039 INFO:teuthology.orchestra.run.smithi097.stdout: "recovery_started": 0, 2024-03-27T11:08:25.039 INFO:teuthology.orchestra.run.smithi097.stdout: "recovery_completed": 0, 2024-03-27T11:08:25.039 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_quiesce_path": 0, 2024-03-27T11:08:25.039 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_quiesce_inode": 0, 2024-03-27T11:08:25.039 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_enqueue_scrub": 0, 2024-03-27T11:08:25.039 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_exportdir": 0, 2024-03-27T11:08:25.039 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_flush": 0, 2024-03-27T11:08:25.039 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_fragmentdir": 0, 2024-03-27T11:08:25.039 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_fragstats": 0, 2024-03-27T11:08:25.039 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_inodestats": 0 2024-03-27T11:08:25.039 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:25.039 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_log": { 2024-03-27T11:08:25.040 INFO:teuthology.orchestra.run.smithi097.stdout: "evlrg": 0, 2024-03-27T11:08:25.040 INFO:teuthology.orchestra.run.smithi097.stdout: "evadd": 0, 2024-03-27T11:08:25.040 INFO:teuthology.orchestra.run.smithi097.stdout: "evex": 0, 2024-03-27T11:08:25.040 INFO:teuthology.orchestra.run.smithi097.stdout: "evtrm": 0, 2024-03-27T11:08:25.040 INFO:teuthology.orchestra.run.smithi097.stdout: "ev": 3, 2024-03-27T11:08:25.040 INFO:teuthology.orchestra.run.smithi097.stdout: "evexg": 0, 2024-03-27T11:08:25.040 INFO:teuthology.orchestra.run.smithi097.stdout: "evexd": 0, 2024-03-27T11:08:25.040 INFO:teuthology.orchestra.run.smithi097.stdout: "segadd": 0, 2024-03-27T11:08:25.040 INFO:teuthology.orchestra.run.smithi097.stdout: "segex": 0, 2024-03-27T11:08:25.040 INFO:teuthology.orchestra.run.smithi097.stdout: "segtrm": 0, 2024-03-27T11:08:25.040 INFO:teuthology.orchestra.run.smithi097.stdout: "seg": 2, 2024-03-27T11:08:25.040 INFO:teuthology.orchestra.run.smithi097.stdout: "segmjr": 2, 2024-03-27T11:08:25.040 INFO:teuthology.orchestra.run.smithi097.stdout: "segexg": 0, 2024-03-27T11:08:25.040 INFO:teuthology.orchestra.run.smithi097.stdout: "segexd": 0, 2024-03-27T11:08:25.041 INFO:teuthology.orchestra.run.smithi097.stdout: "expos": 4194304, 2024-03-27T11:08:25.041 INFO:teuthology.orchestra.run.smithi097.stdout: "wrpos": 4195385, 2024-03-27T11:08:25.041 INFO:teuthology.orchestra.run.smithi097.stdout: "rdpos": 4194892, 2024-03-27T11:08:25.041 INFO:teuthology.orchestra.run.smithi097.stdout: "jlat": { 2024-03-27T11:08:25.041 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-27T11:08:25.041 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-27T11:08:25.041 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-27T11:08:25.041 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:25.041 INFO:teuthology.orchestra.run.smithi097.stdout: "replayed": 3 2024-03-27T11:08:25.041 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:25.041 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_mem": { 2024-03-27T11:08:25.041 INFO:teuthology.orchestra.run.smithi097.stdout: "ino": 3, 2024-03-27T11:08:25.041 INFO:teuthology.orchestra.run.smithi097.stdout: "ino+": 3, 2024-03-27T11:08:25.041 INFO:teuthology.orchestra.run.smithi097.stdout: "ino-": 0, 2024-03-27T11:08:25.042 INFO:teuthology.orchestra.run.smithi097.stdout: "dir": 1, 2024-03-27T11:08:25.042 INFO:teuthology.orchestra.run.smithi097.stdout: "dir+": 1, 2024-03-27T11:08:25.042 INFO:teuthology.orchestra.run.smithi097.stdout: "dir-": 0, 2024-03-27T11:08:25.042 INFO:teuthology.orchestra.run.smithi097.stdout: "dn": 0, 2024-03-27T11:08:25.042 INFO:teuthology.orchestra.run.smithi097.stdout: "dn+": 0, 2024-03-27T11:08:25.042 INFO:teuthology.orchestra.run.smithi097.stdout: "dn-": 0, 2024-03-27T11:08:25.042 INFO:teuthology.orchestra.run.smithi097.stdout: "cap": 0, 2024-03-27T11:08:25.042 INFO:teuthology.orchestra.run.smithi097.stdout: "cap+": 0, 2024-03-27T11:08:25.042 INFO:teuthology.orchestra.run.smithi097.stdout: "cap-": 0, 2024-03-27T11:08:25.042 INFO:teuthology.orchestra.run.smithi097.stdout: "rss": 49152, 2024-03-27T11:08:25.042 INFO:teuthology.orchestra.run.smithi097.stdout: "heap": 215416 2024-03-27T11:08:25.042 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:25.042 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_server": { 2024-03-27T11:08:25.043 INFO:teuthology.orchestra.run.smithi097.stdout: "dispatch_client_request": 0, 2024-03-27T11:08:25.043 INFO:teuthology.orchestra.run.smithi097.stdout: "dispatch_server_request": 0, 2024-03-27T11:08:25.043 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_request": 0, 2024-03-27T11:08:25.043 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_session": 0, 2024-03-27T11:08:25.043 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_peer_request": 0, 2024-03-27T11:08:25.043 INFO:teuthology.orchestra.run.smithi097.stdout: "req_create_latency": { 2024-03-27T11:08:25.043 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-27T11:08:25.043 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-27T11:08:25.043 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-27T11:08:25.043 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:25.043 INFO:teuthology.orchestra.run.smithi097.stdout: "req_getattr_latency": { 2024-03-27T11:08:25.043 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-27T11:08:25.043 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-27T11:08:25.043 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-27T11:08:25.044 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:25.044 INFO:teuthology.orchestra.run.smithi097.stdout: "req_getfilelock_latency": { 2024-03-27T11:08:25.044 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-27T11:08:25.044 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-27T11:08:25.044 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-27T11:08:25.044 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:25.044 INFO:teuthology.orchestra.run.smithi097.stdout: "req_link_latency": { 2024-03-27T11:08:25.044 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-27T11:08:25.044 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-27T11:08:25.044 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-27T11:08:25.044 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:25.044 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookup_latency": { 2024-03-27T11:08:25.044 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-27T11:08:25.045 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-27T11:08:25.045 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-27T11:08:25.045 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:25.045 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookuphash_latency": { 2024-03-27T11:08:25.045 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-27T11:08:25.045 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-27T11:08:25.045 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-27T11:08:25.045 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:25.045 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupino_latency": { 2024-03-27T11:08:25.045 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-27T11:08:25.045 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-27T11:08:25.045 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-27T11:08:25.045 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:25.045 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupname_latency": { 2024-03-27T11:08:25.045 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-27T11:08:25.045 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-27T11:08:25.046 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-27T11:08:25.046 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:25.046 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupparent_latency": { 2024-03-27T11:08:25.046 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-27T11:08:25.046 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-27T11:08:25.046 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-27T11:08:25.046 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:25.046 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupsnap_latency": { 2024-03-27T11:08:25.046 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-27T11:08:25.046 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-27T11:08:25.046 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-27T11:08:25.046 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:25.046 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lssnap_latency": { 2024-03-27T11:08:25.046 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-27T11:08:25.047 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-27T11:08:25.047 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-27T11:08:25.047 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:25.047 INFO:teuthology.orchestra.run.smithi097.stdout: "req_mkdir_latency": { 2024-03-27T11:08:25.047 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-27T11:08:25.047 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-27T11:08:25.048 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-27T11:08:25.048 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:25.048 INFO:teuthology.orchestra.run.smithi097.stdout: "req_mknod_latency": { 2024-03-27T11:08:25.048 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-27T11:08:25.048 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-27T11:08:25.048 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-27T11:08:25.048 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:25.048 INFO:teuthology.orchestra.run.smithi097.stdout: "req_mksnap_latency": { 2024-03-27T11:08:25.048 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-27T11:08:25.048 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-27T11:08:25.048 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-27T11:08:25.048 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:25.048 INFO:teuthology.orchestra.run.smithi097.stdout: "req_open_latency": { 2024-03-27T11:08:25.049 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-27T11:08:25.049 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-27T11:08:25.049 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-27T11:08:25.049 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:25.049 INFO:teuthology.orchestra.run.smithi097.stdout: "req_readdir_latency": { 2024-03-27T11:08:25.049 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-27T11:08:25.049 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-27T11:08:25.049 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-27T11:08:25.049 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:25.049 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rename_latency": { 2024-03-27T11:08:25.049 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-27T11:08:25.049 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-27T11:08:25.049 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-27T11:08:25.049 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:25.050 INFO:teuthology.orchestra.run.smithi097.stdout: "req_renamesnap_latency": { 2024-03-27T11:08:25.050 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-27T11:08:25.050 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-27T11:08:25.050 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-27T11:08:25.050 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:25.050 INFO:teuthology.orchestra.run.smithi097.stdout: "req_snapdiff_latency": { 2024-03-27T11:08:25.050 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-27T11:08:25.050 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-27T11:08:25.050 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-27T11:08:25.050 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:25.050 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rmdir_latency": { 2024-03-27T11:08:25.050 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-27T11:08:25.051 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-27T11:08:25.051 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-27T11:08:25.051 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:25.051 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rmsnap_latency": { 2024-03-27T11:08:25.051 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-27T11:08:25.051 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-27T11:08:25.051 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-27T11:08:25.051 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:25.051 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rmxattr_latency": { 2024-03-27T11:08:25.051 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-27T11:08:25.051 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-27T11:08:25.051 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-27T11:08:25.051 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:25.052 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setattr_latency": { 2024-03-27T11:08:25.052 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-27T11:08:25.052 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-27T11:08:25.052 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-27T11:08:25.052 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:25.052 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setdirlayout_latency": { 2024-03-27T11:08:25.052 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-27T11:08:25.052 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-27T11:08:25.052 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-27T11:08:25.052 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:25.052 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setfilelock_latency": { 2024-03-27T11:08:25.052 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-27T11:08:25.052 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-27T11:08:25.052 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-27T11:08:25.053 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:25.053 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setlayout_latency": { 2024-03-27T11:08:25.053 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-27T11:08:25.053 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-27T11:08:25.053 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-27T11:08:25.053 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:25.053 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setxattr_latency": { 2024-03-27T11:08:25.053 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-27T11:08:25.053 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-27T11:08:25.053 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-27T11:08:25.053 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:25.053 INFO:teuthology.orchestra.run.smithi097.stdout: "req_symlink_latency": { 2024-03-27T11:08:25.053 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-27T11:08:25.054 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-27T11:08:25.054 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-27T11:08:25.054 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:25.054 INFO:teuthology.orchestra.run.smithi097.stdout: "req_unlink_latency": { 2024-03-27T11:08:25.054 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-27T11:08:25.054 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-27T11:08:25.054 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-27T11:08:25.054 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:25.054 INFO:teuthology.orchestra.run.smithi097.stdout: "cap_revoke_eviction": 0, 2024-03-27T11:08:25.054 INFO:teuthology.orchestra.run.smithi097.stdout: "cap_acquisition_throttle": 0, 2024-03-27T11:08:25.054 INFO:teuthology.orchestra.run.smithi097.stdout: "req_getvxattr_latency": { 2024-03-27T11:08:25.054 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-27T11:08:25.054 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-27T11:08:25.055 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-27T11:08:25.055 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:25.055 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:25.055 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_sessions": { 2024-03-27T11:08:25.055 INFO:teuthology.orchestra.run.smithi097.stdout: "session_count": 1, 2024-03-27T11:08:25.055 INFO:teuthology.orchestra.run.smithi097.stdout: "session_add": 1, 2024-03-27T11:08:25.055 INFO:teuthology.orchestra.run.smithi097.stdout: "session_remove": 0, 2024-03-27T11:08:25.055 INFO:teuthology.orchestra.run.smithi097.stdout: "sessions_open": 1, 2024-03-27T11:08:25.055 INFO:teuthology.orchestra.run.smithi097.stdout: "sessions_stale": 0, 2024-03-27T11:08:25.055 INFO:teuthology.orchestra.run.smithi097.stdout: "total_load": 0, 2024-03-27T11:08:25.055 INFO:teuthology.orchestra.run.smithi097.stdout: "average_load": 0, 2024-03-27T11:08:25.055 INFO:teuthology.orchestra.run.smithi097.stdout: "avg_session_uptime": 2135, 2024-03-27T11:08:25.055 INFO:teuthology.orchestra.run.smithi097.stdout: "mdthresh_evicted": 0 2024-03-27T11:08:25.056 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:25.056 INFO:teuthology.orchestra.run.smithi097.stdout: "mempool": { 2024-03-27T11:08:25.056 INFO:teuthology.orchestra.run.smithi097.stdout: "bloom_filter_bytes": 0, 2024-03-27T11:08:25.056 INFO:teuthology.orchestra.run.smithi097.stdout: "bloom_filter_items": 0, 2024-03-27T11:08:25.056 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_alloc_bytes": 0, 2024-03-27T11:08:25.056 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_alloc_items": 0, 2024-03-27T11:08:25.056 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_data_bytes": 0, 2024-03-27T11:08:25.056 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_data_items": 0, 2024-03-27T11:08:25.056 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-27T11:08:25.056 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_onode_items": 0, 2024-03-27T11:08:25.056 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-27T11:08:25.056 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_meta_items": 0, 2024-03-27T11:08:25.056 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_other_bytes": 0, 2024-03-27T11:08:25.056 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_other_items": 0, 2024-03-27T11:08:25.057 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-27T11:08:25.057 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_buffer_items": 0, 2024-03-27T11:08:25.057 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_extent_bytes": 0, 2024-03-27T11:08:25.057 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_extent_items": 0, 2024-03-27T11:08:25.057 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_blob_bytes": 0, 2024-03-27T11:08:25.057 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_blob_items": 0, 2024-03-27T11:08:25.057 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-27T11:08:25.057 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_shared_blob_items": 0, 2024-03-27T11:08:25.057 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-27T11:08:25.057 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_inline_bl_items": 0, 2024-03-27T11:08:25.057 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_bytes": 0, 2024-03-27T11:08:25.057 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_items": 0, 2024-03-27T11:08:25.057 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_txc_bytes": 0, 2024-03-27T11:08:25.057 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_txc_items": 0, 2024-03-27T11:08:25.058 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-27T11:08:25.058 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_deferred_items": 0, 2024-03-27T11:08:25.058 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_bytes": 0, 2024-03-27T11:08:25.058 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_items": 0, 2024-03-27T11:08:25.058 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_bytes": 0, 2024-03-27T11:08:25.058 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_items": 0, 2024-03-27T11:08:25.058 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_reader_bytes": 0, 2024-03-27T11:08:25.058 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_reader_items": 0, 2024-03-27T11:08:25.058 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_writer_bytes": 0, 2024-03-27T11:08:25.058 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_writer_items": 0, 2024-03-27T11:08:25.058 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_anon_bytes": 197543, 2024-03-27T11:08:25.058 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_anon_items": 56, 2024-03-27T11:08:25.058 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_meta_bytes": 0, 2024-03-27T11:08:25.058 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_meta_items": 0, 2024-03-27T11:08:25.059 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_bytes": 0, 2024-03-27T11:08:25.059 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_items": 0, 2024-03-27T11:08:25.059 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mapbl_bytes": 0, 2024-03-27T11:08:25.059 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mapbl_items": 0, 2024-03-27T11:08:25.059 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pglog_bytes": 0, 2024-03-27T11:08:25.059 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pglog_items": 0, 2024-03-27T11:08:25.059 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_bytes": 7528, 2024-03-27T11:08:25.059 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_items": 158, 2024-03-27T11:08:25.059 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_mapping_bytes": 0, 2024-03-27T11:08:25.059 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_mapping_items": 0, 2024-03-27T11:08:25.059 INFO:teuthology.orchestra.run.smithi097.stdout: "pgmap_bytes": 0, 2024-03-27T11:08:25.059 INFO:teuthology.orchestra.run.smithi097.stdout: "pgmap_items": 0, 2024-03-27T11:08:25.059 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_co_bytes": 8464, 2024-03-27T11:08:25.059 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_co_items": 13, 2024-03-27T11:08:25.059 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_1_bytes": 0, 2024-03-27T11:08:25.060 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_1_items": 0, 2024-03-27T11:08:25.060 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_2_bytes": 0, 2024-03-27T11:08:25.060 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_2_items": 0 2024-03-27T11:08:25.060 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:25.060 INFO:teuthology.orchestra.run.smithi097.stdout: "objecter": { 2024-03-27T11:08:25.060 INFO:teuthology.orchestra.run.smithi097.stdout: "op_active": 0, 2024-03-27T11:08:25.060 INFO:teuthology.orchestra.run.smithi097.stdout: "op_laggy": 0, 2024-03-27T11:08:25.060 INFO:teuthology.orchestra.run.smithi097.stdout: "op_send": 2426, 2024-03-27T11:08:25.060 INFO:teuthology.orchestra.run.smithi097.stdout: "op_send_bytes": 21, 2024-03-27T11:08:25.060 INFO:teuthology.orchestra.run.smithi097.stdout: "op_resend": 3, 2024-03-27T11:08:25.060 INFO:teuthology.orchestra.run.smithi097.stdout: "op_reply": 2426, 2024-03-27T11:08:25.060 INFO:teuthology.orchestra.run.smithi097.stdout: "op_latency": { 2024-03-27T11:08:25.060 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 2426, 2024-03-27T11:08:25.060 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 2.295961045, 2024-03-27T11:08:25.061 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000946397 2024-03-27T11:08:25.061 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:25.061 INFO:teuthology.orchestra.run.smithi097.stdout: "op_inflight": 0, 2024-03-27T11:08:25.061 INFO:teuthology.orchestra.run.smithi097.stdout: "oplen_avg": { 2024-03-27T11:08:25.061 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 2426, 2024-03-27T11:08:25.061 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 2427 2024-03-27T11:08:25.061 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:25.061 INFO:teuthology.orchestra.run.smithi097.stdout: "op": 2426, 2024-03-27T11:08:25.061 INFO:teuthology.orchestra.run.smithi097.stdout: "op_r": 2426, 2024-03-27T11:08:25.061 INFO:teuthology.orchestra.run.smithi097.stdout: "op_w": 0, 2024-03-27T11:08:25.061 INFO:teuthology.orchestra.run.smithi097.stdout: "op_rmw": 0, 2024-03-27T11:08:25.061 INFO:teuthology.orchestra.run.smithi097.stdout: "op_pg": 0, 2024-03-27T11:08:25.061 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_stat": 1568, 2024-03-27T11:08:25.061 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_create": 0, 2024-03-27T11:08:25.062 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_read": 856, 2024-03-27T11:08:25.062 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_write": 0, 2024-03-27T11:08:25.062 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_writefull": 0, 2024-03-27T11:08:25.062 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_writesame": 0, 2024-03-27T11:08:25.062 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_append": 0, 2024-03-27T11:08:25.062 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_zero": 0, 2024-03-27T11:08:25.062 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_truncate": 0, 2024-03-27T11:08:25.062 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_delete": 0, 2024-03-27T11:08:25.062 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_mapext": 0, 2024-03-27T11:08:25.062 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_sparse_read": 0, 2024-03-27T11:08:25.062 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_clonerange": 0, 2024-03-27T11:08:25.062 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_getxattr": 1, 2024-03-27T11:08:25.062 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_setxattr": 0, 2024-03-27T11:08:25.063 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_cmpxattr": 0, 2024-03-27T11:08:25.063 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_rmxattr": 0, 2024-03-27T11:08:25.063 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_resetxattrs": 0, 2024-03-27T11:08:25.063 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_call": 0, 2024-03-27T11:08:25.063 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_watch": 0, 2024-03-27T11:08:25.063 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_notify": 0, 2024-03-27T11:08:25.063 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_src_cmpxattr": 0, 2024-03-27T11:08:25.063 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_pgls": 0, 2024-03-27T11:08:25.063 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_pgls_filter": 0, 2024-03-27T11:08:25.063 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_other": 0, 2024-03-27T11:08:25.063 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_active": 0, 2024-03-27T11:08:25.063 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_send": 0, 2024-03-27T11:08:25.063 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_resend": 0, 2024-03-27T11:08:25.063 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_ping": 0, 2024-03-27T11:08:25.063 INFO:teuthology.orchestra.run.smithi097.stdout: "poolop_active": 0, 2024-03-27T11:08:25.064 INFO:teuthology.orchestra.run.smithi097.stdout: "poolop_send": 0, 2024-03-27T11:08:25.064 INFO:teuthology.orchestra.run.smithi097.stdout: "poolop_resend": 0, 2024-03-27T11:08:25.064 INFO:teuthology.orchestra.run.smithi097.stdout: "poolstat_active": 0, 2024-03-27T11:08:25.064 INFO:teuthology.orchestra.run.smithi097.stdout: "poolstat_send": 0, 2024-03-27T11:08:25.064 INFO:teuthology.orchestra.run.smithi097.stdout: "poolstat_resend": 0, 2024-03-27T11:08:25.064 INFO:teuthology.orchestra.run.smithi097.stdout: "statfs_active": 0, 2024-03-27T11:08:25.064 INFO:teuthology.orchestra.run.smithi097.stdout: "statfs_send": 0, 2024-03-27T11:08:25.064 INFO:teuthology.orchestra.run.smithi097.stdout: "statfs_resend": 0, 2024-03-27T11:08:25.064 INFO:teuthology.orchestra.run.smithi097.stdout: "command_active": 0, 2024-03-27T11:08:25.064 INFO:teuthology.orchestra.run.smithi097.stdout: "command_send": 0, 2024-03-27T11:08:25.064 INFO:teuthology.orchestra.run.smithi097.stdout: "command_resend": 0, 2024-03-27T11:08:25.064 INFO:teuthology.orchestra.run.smithi097.stdout: "map_epoch": 92, 2024-03-27T11:08:25.064 INFO:teuthology.orchestra.run.smithi097.stdout: "map_full": 0, 2024-03-27T11:08:25.064 INFO:teuthology.orchestra.run.smithi097.stdout: "map_inc": 2, 2024-03-27T11:08:25.064 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_sessions": 6, 2024-03-27T11:08:25.065 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_session_open": 6, 2024-03-27T11:08:25.065 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_session_close": 0, 2024-03-27T11:08:25.065 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_laggy": 0, 2024-03-27T11:08:25.065 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_wr": 0, 2024-03-27T11:08:25.065 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_rd": 2, 2024-03-27T11:08:25.065 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_del": 0 2024-03-27T11:08:25.065 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:25.065 INFO:teuthology.orchestra.run.smithi097.stdout: "oft": { 2024-03-27T11:08:25.065 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_objs": 0, 2024-03-27T11:08:25.065 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_kv_pairs": 0, 2024-03-27T11:08:25.065 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_updates": 0, 2024-03-27T11:08:25.065 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_removes": 0 2024-03-27T11:08:25.065 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:25.065 INFO:teuthology.orchestra.run.smithi097.stdout: "osd-slow-ops": { 2024-03-27T11:08:25.065 INFO:teuthology.orchestra.run.smithi097.stdout: "slow_ops_count": 0 2024-03-27T11:08:25.066 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:25.066 INFO:teuthology.orchestra.run.smithi097.stdout: "purge_queue": { 2024-03-27T11:08:25.066 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing_ops": 0, 2024-03-27T11:08:25.066 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing_ops_high_water": 0, 2024-03-27T11:08:25.066 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing": 0, 2024-03-27T11:08:25.066 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing_high_water": 0, 2024-03-27T11:08:25.066 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executed_ops": 0, 2024-03-27T11:08:25.066 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executed": 0, 2024-03-27T11:08:25.066 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_item_in_journal": 0 2024-03-27T11:08:25.066 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:25.066 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-27T11:08:25.066 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-03-27T11:08:25.066 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 104857600, 2024-03-27T11:08:25.067 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-03-27T11:08:25.067 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 2679, 2024-03-27T11:08:25.067 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 990036, 2024-03-27T11:08:25.067 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-03-27T11:08:25.067 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 2679, 2024-03-27T11:08:25.067 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2024-03-27T11:08:25.067 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2024-03-27T11:08:25.067 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 2679, 2024-03-27T11:08:25.067 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 990036, 2024-03-27T11:08:25.067 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-03-27T11:08:25.067 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-27T11:08:25.067 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-27T11:08:25.067 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-27T11:08:25.067 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:25.068 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:25.068 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-objecter_bytes": { 2024-03-27T11:08:25.068 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-03-27T11:08:25.068 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 104857600, 2024-03-27T11:08:25.068 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-03-27T11:08:25.068 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 0, 2024-03-27T11:08:25.068 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 0, 2024-03-27T11:08:25.068 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-03-27T11:08:25.068 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 0, 2024-03-27T11:08:25.068 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 2426, 2024-03-27T11:08:25.068 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 1086, 2024-03-27T11:08:25.068 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 2426, 2024-03-27T11:08:25.068 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 1086, 2024-03-27T11:08:25.068 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-03-27T11:08:25.068 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-27T11:08:25.069 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-27T11:08:25.069 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-27T11:08:25.069 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:25.069 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:25.069 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-objecter_ops": { 2024-03-27T11:08:25.069 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-03-27T11:08:25.069 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 1024, 2024-03-27T11:08:25.069 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-03-27T11:08:25.069 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 0, 2024-03-27T11:08:25.069 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 0, 2024-03-27T11:08:25.069 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-03-27T11:08:25.069 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 0, 2024-03-27T11:08:25.069 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 2426, 2024-03-27T11:08:25.069 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 2426, 2024-03-27T11:08:25.070 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 2426, 2024-03-27T11:08:25.070 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 2426, 2024-03-27T11:08:25.070 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-03-27T11:08:25.070 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-27T11:08:25.070 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-27T11:08:25.070 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-27T11:08:25.070 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:25.070 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:25.070 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-write_buf_throttle": { 2024-03-27T11:08:25.070 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-03-27T11:08:25.070 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 3758096384, 2024-03-27T11:08:25.070 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-03-27T11:08:25.070 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 0, 2024-03-27T11:08:25.070 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 0, 2024-03-27T11:08:25.071 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-03-27T11:08:25.071 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 0, 2024-03-27T11:08:25.071 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2024-03-27T11:08:25.071 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2024-03-27T11:08:25.071 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 0, 2024-03-27T11:08:25.071 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 0, 2024-03-27T11:08:25.071 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-03-27T11:08:25.071 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-27T11:08:25.071 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-27T11:08:25.071 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-27T11:08:25.071 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:25.071 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:25.071 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-write_buf_throttle-0x55d0eb2ae780": { 2024-03-27T11:08:25.071 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-03-27T11:08:25.071 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 3758096384, 2024-03-27T11:08:25.071 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-03-27T11:08:25.072 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 0, 2024-03-27T11:08:25.072 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 0, 2024-03-27T11:08:25.072 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-03-27T11:08:25.072 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 0, 2024-03-27T11:08:25.072 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2024-03-27T11:08:25.072 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2024-03-27T11:08:25.072 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 0, 2024-03-27T11:08:25.072 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 0, 2024-03-27T11:08:25.072 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-03-27T11:08:25.072 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-27T11:08:25.072 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-27T11:08:25.072 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-27T11:08:25.072 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:25.072 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:25.072 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-03-27T11:08:25.888 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:08:25 smithi097 ceph-mon[31859]: pgmap v794: 97 pgs: 97 active+clean; 6.3 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 34 KiB/s wr, 94 op/s 2024-03-27T11:08:25.923 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:08:25 smithi177 ceph-mon[35294]: pgmap v794: 97 pgs: 97 active+clean; 6.3 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 34 KiB/s wr, 94 op/s 2024-03-27T11:08:26.007 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-03-27T11:08:26.008 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-03-27T11:08:26.008 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update=0 2024-03-27T11:08:26.008 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update_receipt=0 2024-03-27T11:08:26.008 DEBUG:tasks.check_counter:Getting stats from j 2024-03-27T11:08:26.008 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell --fsid adca88c4-ec26-11ee-95d0-87774f69a715 -- ceph daemon mds.j perf dump 2024-03-27T11:08:26.013 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:08:25 smithi114 ceph-mon[35338]: pgmap v794: 97 pgs: 97 active+clean; 6.3 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 34 KiB/s wr, 94 op/s 2024-03-27T11:08:26.455 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T11:08:27.825 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:08:27 smithi097 ceph-mon[31859]: pgmap v795: 97 pgs: 97 active+clean; 5.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 3.2 KiB/s wr, 88 op/s 2024-03-27T11:08:27.923 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:08:27 smithi177 ceph-mon[35294]: pgmap v795: 97 pgs: 97 active+clean; 5.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 3.2 KiB/s wr, 88 op/s 2024-03-27T11:08:28.013 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:08:27 smithi114 ceph-mon[35338]: pgmap v795: 97 pgs: 97 active+clean; 5.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 3.2 KiB/s wr, 88 op/s 2024-03-27T11:08:28.663 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-03-27T11:08:28.663 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-0": { 2024-03-27T11:08:28.663 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 0, 2024-03-27T11:08:28.663 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 2, 2024-03-27T11:08:28.663 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 0, 2024-03-27T11:08:28.663 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 0, 2024-03-27T11:08:28.663 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 4, 2024-03-27T11:08:28.663 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 0, 2024-03-27T11:08:28.663 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.003327422, 2024-03-27T11:08:28.663 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.000000000, 2024-03-27T11:08:28.664 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.000474679, 2024-03-27T11:08:28.664 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-27T11:08:28.664 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-03-27T11:08:28.664 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-27T11:08:28.664 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-27T11:08:28.664 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-27T11:08:28.664 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:28.664 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-03-27T11:08:28.664 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-27T11:08:28.664 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-27T11:08:28.664 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-27T11:08:28.664 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:28.664 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-27T11:08:28.664 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 0 2024-03-27T11:08:28.664 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:28.665 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-1": { 2024-03-27T11:08:28.665 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 250, 2024-03-27T11:08:28.665 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 245, 2024-03-27T11:08:28.665 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 490832, 2024-03-27T11:08:28.665 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 149968, 2024-03-27T11:08:28.665 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 2, 2024-03-27T11:08:28.665 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 1, 2024-03-27T11:08:28.665 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.073177234, 2024-03-27T11:08:28.665 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.030385161, 2024-03-27T11:08:28.665 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.038693826, 2024-03-27T11:08:28.665 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.009447330, 2024-03-27T11:08:28.665 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-03-27T11:08:28.665 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 245, 2024-03-27T11:08:28.665 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.030390654, 2024-03-27T11:08:28.666 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000124043 2024-03-27T11:08:28.666 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:28.666 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-03-27T11:08:28.666 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-27T11:08:28.666 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-27T11:08:28.666 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-27T11:08:28.666 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:28.666 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 490832, 2024-03-27T11:08:28.666 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 149968 2024-03-27T11:08:28.666 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:28.666 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-2": { 2024-03-27T11:08:28.666 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 2, 2024-03-27T11:08:28.667 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 185, 2024-03-27T11:08:28.667 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 320, 2024-03-27T11:08:28.667 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 188272, 2024-03-27T11:08:28.667 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 4, 2024-03-27T11:08:28.667 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 2, 2024-03-27T11:08:28.667 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.025888418, 2024-03-27T11:08:28.667 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.020652104, 2024-03-27T11:08:28.667 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.002513293, 2024-03-27T11:08:28.667 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-27T11:08:28.667 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-03-27T11:08:28.667 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 183, 2024-03-27T11:08:28.667 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.019716678, 2024-03-27T11:08:28.667 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000107741 2024-03-27T11:08:28.667 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:28.668 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-03-27T11:08:28.668 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-27T11:08:28.668 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-27T11:08:28.668 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-27T11:08:28.668 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:28.668 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-27T11:08:28.668 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 188272 2024-03-27T11:08:28.668 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:28.668 INFO:teuthology.orchestra.run.smithi097.stdout: "cct": { 2024-03-27T11:08:28.668 INFO:teuthology.orchestra.run.smithi097.stdout: "total_workers": 0, 2024-03-27T11:08:28.668 INFO:teuthology.orchestra.run.smithi097.stdout: "unhealthy_workers": 0 2024-03-27T11:08:28.668 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:28.668 INFO:teuthology.orchestra.run.smithi097.stdout: "mempool": { 2024-03-27T11:08:28.668 INFO:teuthology.orchestra.run.smithi097.stdout: "bloom_filter_bytes": 0, 2024-03-27T11:08:28.669 INFO:teuthology.orchestra.run.smithi097.stdout: "bloom_filter_items": 0, 2024-03-27T11:08:28.669 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_alloc_bytes": 0, 2024-03-27T11:08:28.669 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_alloc_items": 0, 2024-03-27T11:08:28.669 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_data_bytes": 0, 2024-03-27T11:08:28.669 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_data_items": 0, 2024-03-27T11:08:28.669 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-27T11:08:28.669 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_onode_items": 0, 2024-03-27T11:08:28.669 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-27T11:08:28.669 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_meta_items": 0, 2024-03-27T11:08:28.669 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_other_bytes": 0, 2024-03-27T11:08:28.669 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_other_items": 0, 2024-03-27T11:08:28.669 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-27T11:08:28.669 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_buffer_items": 0, 2024-03-27T11:08:28.669 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_extent_bytes": 0, 2024-03-27T11:08:28.669 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_extent_items": 0, 2024-03-27T11:08:28.670 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_blob_bytes": 0, 2024-03-27T11:08:28.670 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_blob_items": 0, 2024-03-27T11:08:28.670 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-27T11:08:28.670 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_shared_blob_items": 0, 2024-03-27T11:08:28.670 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-27T11:08:28.670 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_inline_bl_items": 0, 2024-03-27T11:08:28.670 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_bytes": 0, 2024-03-27T11:08:28.670 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_items": 0, 2024-03-27T11:08:28.670 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_txc_bytes": 0, 2024-03-27T11:08:28.670 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_txc_items": 0, 2024-03-27T11:08:28.670 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-27T11:08:28.670 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_deferred_items": 0, 2024-03-27T11:08:28.670 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_bytes": 0, 2024-03-27T11:08:28.671 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_items": 0, 2024-03-27T11:08:28.671 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_bytes": 0, 2024-03-27T11:08:28.671 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_items": 0, 2024-03-27T11:08:28.671 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_reader_bytes": 0, 2024-03-27T11:08:28.671 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_reader_items": 0, 2024-03-27T11:08:28.671 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_writer_bytes": 0, 2024-03-27T11:08:28.671 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_writer_items": 0, 2024-03-27T11:08:28.671 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_anon_bytes": 148967, 2024-03-27T11:08:28.671 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_anon_items": 29, 2024-03-27T11:08:28.671 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_meta_bytes": 0, 2024-03-27T11:08:28.671 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_meta_items": 0, 2024-03-27T11:08:28.671 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_bytes": 0, 2024-03-27T11:08:28.671 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_items": 0, 2024-03-27T11:08:28.671 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mapbl_bytes": 0, 2024-03-27T11:08:28.672 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mapbl_items": 0, 2024-03-27T11:08:28.672 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pglog_bytes": 0, 2024-03-27T11:08:28.672 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pglog_items": 0, 2024-03-27T11:08:28.672 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_bytes": 0, 2024-03-27T11:08:28.672 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_items": 0, 2024-03-27T11:08:28.672 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_mapping_bytes": 0, 2024-03-27T11:08:28.672 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_mapping_items": 0, 2024-03-27T11:08:28.672 INFO:teuthology.orchestra.run.smithi097.stdout: "pgmap_bytes": 0, 2024-03-27T11:08:28.672 INFO:teuthology.orchestra.run.smithi097.stdout: "pgmap_items": 0, 2024-03-27T11:08:28.672 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_co_bytes": 928, 2024-03-27T11:08:28.672 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_co_items": 2, 2024-03-27T11:08:28.672 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_1_bytes": 0, 2024-03-27T11:08:28.672 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_1_items": 0, 2024-03-27T11:08:28.672 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_2_bytes": 0, 2024-03-27T11:08:28.672 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_2_items": 0 2024-03-27T11:08:28.673 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-27T11:08:28.673 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-27T11:08:28.673 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-03-27T11:08:28.673 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 104857600, 2024-03-27T11:08:28.673 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-03-27T11:08:28.673 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 248, 2024-03-27T11:08:28.673 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 453078, 2024-03-27T11:08:28.673 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-03-27T11:08:28.673 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 248, 2024-03-27T11:08:28.673 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2024-03-27T11:08:28.673 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2024-03-27T11:08:28.673 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 248, 2024-03-27T11:08:28.673 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 453078, 2024-03-27T11:08:28.673 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-03-27T11:08:28.674 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-03-27T11:08:28.674 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-03-27T11:08:28.674 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-03-27T11:08:28.674 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:28.674 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-27T11:08:28.674 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-03-27T11:08:29.520 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-03-27T11:08:29.520 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-03-27T11:08:29.520 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-03-27T11:08:29.520 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-03-27T11:08:29.520 DEBUG:tasks.check_counter:Getting stats from b 2024-03-27T11:08:29.521 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell --fsid adca88c4-ec26-11ee-95d0-87774f69a715 -- ceph daemon mds.b perf dump 2024-03-27T11:08:29.887 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:08:29 smithi097 ceph-mon[31859]: pgmap v796: 97 pgs: 97 active+clean; 5.3 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 2.3 KiB/s wr, 91 op/s 2024-03-27T11:08:29.923 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:08:29 smithi177 ceph-mon[35294]: pgmap v796: 97 pgs: 97 active+clean; 5.3 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 2.3 KiB/s wr, 91 op/s 2024-03-27T11:08:30.013 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:08:29 smithi114 ceph-mon[35338]: pgmap v796: 97 pgs: 97 active+clean; 5.3 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 2.3 KiB/s wr, 91 op/s 2024-03-27T11:08:30.023 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/mon.b/config 2024-03-27T11:08:31.871 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:08:31 smithi114 ceph-mon[35338]: pgmap v797: 97 pgs: 97 active+clean; 4.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 2.2 KiB/s wr, 99 op/s 2024-03-27T11:08:31.887 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:08:31 smithi097 ceph-mon[31859]: pgmap v797: 97 pgs: 97 active+clean; 4.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 2.2 KiB/s wr, 99 op/s 2024-03-27T11:08:31.923 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:08:31 smithi177 ceph-mon[35294]: pgmap v797: 97 pgs: 97 active+clean; 4.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 2.2 KiB/s wr, 99 op/s 2024-03-27T11:08:32.532 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-03-27T11:08:32.532 INFO:teuthology.orchestra.run.smithi114.stdout: "AsyncMessenger::Worker-0": { 2024-03-27T11:08:32.532 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_messages": 2982, 2024-03-27T11:08:32.532 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages": 2221, 2024-03-27T11:08:32.532 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_bytes": 1041173, 2024-03-27T11:08:32.532 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_bytes": 2016512, 2024-03-27T11:08:32.532 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_created_connections": 18, 2024-03-27T11:08:32.533 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_active_connections": 4, 2024-03-27T11:08:32.533 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_total_time": 0.426583580, 2024-03-27T11:08:32.533 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_send_time": 0.128998039, 2024-03-27T11:08:32.533 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_recv_time": 0.148373979, 2024-03-27T11:08:32.533 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_fast_dispatch_time": 0.116234933, 2024-03-27T11:08:32.533 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages_queue_lat": { 2024-03-27T11:08:32.533 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 2215, 2024-03-27T11:08:32.533 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.548727911, 2024-03-27T11:08:32.533 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000247732 2024-03-27T11:08:32.533 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:32.533 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_handle_ack_lat": { 2024-03-27T11:08:32.533 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 2698, 2024-03-27T11:08:32.533 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.001423128, 2024-03-27T11:08:32.534 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000527 2024-03-27T11:08:32.534 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:32.534 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_encrypted_bytes": 57696, 2024-03-27T11:08:32.534 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_encrypted_bytes": 29824 2024-03-27T11:08:32.534 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:32.534 INFO:teuthology.orchestra.run.smithi114.stdout: "AsyncMessenger::Worker-1": { 2024-03-27T11:08:32.534 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_messages": 1967, 2024-03-27T11:08:32.534 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages": 2006, 2024-03-27T11:08:32.534 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_bytes": 651276, 2024-03-27T11:08:32.534 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_bytes": 1130643, 2024-03-27T11:08:32.534 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_created_connections": 9, 2024-03-27T11:08:32.534 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_active_connections": 6, 2024-03-27T11:08:32.534 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_total_time": 0.277370492, 2024-03-27T11:08:32.534 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_send_time": 0.123324449, 2024-03-27T11:08:32.535 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_recv_time": 0.083466690, 2024-03-27T11:08:32.535 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_fast_dispatch_time": 0.060536665, 2024-03-27T11:08:32.535 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages_queue_lat": { 2024-03-27T11:08:32.535 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 2008, 2024-03-27T11:08:32.535 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.715128049, 2024-03-27T11:08:32.535 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000356139 2024-03-27T11:08:32.535 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:32.535 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_handle_ack_lat": { 2024-03-27T11:08:32.535 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 1151, 2024-03-27T11:08:32.535 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.001420186, 2024-03-27T11:08:32.535 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000001233 2024-03-27T11:08:32.535 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:32.535 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-27T11:08:32.535 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_encrypted_bytes": 369008 2024-03-27T11:08:32.535 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:32.536 INFO:teuthology.orchestra.run.smithi114.stdout: "AsyncMessenger::Worker-2": { 2024-03-27T11:08:32.536 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_messages": 2385, 2024-03-27T11:08:32.536 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages": 2094, 2024-03-27T11:08:32.536 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_bytes": 1373958, 2024-03-27T11:08:32.536 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_bytes": 2442791, 2024-03-27T11:08:32.536 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_created_connections": 149, 2024-03-27T11:08:32.536 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_active_connections": 8, 2024-03-27T11:08:32.536 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_total_time": 0.459951060, 2024-03-27T11:08:32.536 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_send_time": 0.168097650, 2024-03-27T11:08:32.536 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_recv_time": 0.218182143, 2024-03-27T11:08:32.536 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_fast_dispatch_time": 0.070117810, 2024-03-27T11:08:32.536 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages_queue_lat": { 2024-03-27T11:08:32.536 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 2094, 2024-03-27T11:08:32.536 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.422616841, 2024-03-27T11:08:32.537 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000201822 2024-03-27T11:08:32.537 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:32.537 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_handle_ack_lat": { 2024-03-27T11:08:32.537 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 2478, 2024-03-27T11:08:32.537 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000654085, 2024-03-27T11:08:32.537 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000263 2024-03-27T11:08:32.537 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:32.537 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_encrypted_bytes": 602320, 2024-03-27T11:08:32.537 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_encrypted_bytes": 156800 2024-03-27T11:08:32.537 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:32.537 INFO:teuthology.orchestra.run.smithi114.stdout: "cct": { 2024-03-27T11:08:32.537 INFO:teuthology.orchestra.run.smithi114.stdout: "total_workers": 1, 2024-03-27T11:08:32.538 INFO:teuthology.orchestra.run.smithi114.stdout: "unhealthy_workers": 0 2024-03-27T11:08:32.538 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:32.538 INFO:teuthology.orchestra.run.smithi114.stdout: "finisher-MDSRank": { 2024-03-27T11:08:32.538 INFO:teuthology.orchestra.run.smithi114.stdout: "queue_len": 0, 2024-03-27T11:08:32.538 INFO:teuthology.orchestra.run.smithi114.stdout: "complete_latency": { 2024-03-27T11:08:32.538 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 1050, 2024-03-27T11:08:32.538 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.361263369, 2024-03-27T11:08:32.538 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000344060 2024-03-27T11:08:32.538 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-03-27T11:08:32.538 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:32.538 INFO:teuthology.orchestra.run.smithi114.stdout: "finisher-PurgeQueue": { 2024-03-27T11:08:32.538 INFO:teuthology.orchestra.run.smithi114.stdout: "queue_len": 0, 2024-03-27T11:08:32.538 INFO:teuthology.orchestra.run.smithi114.stdout: "complete_latency": { 2024-03-27T11:08:32.539 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 1835, 2024-03-27T11:08:32.539 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.086968898, 2024-03-27T11:08:32.539 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000047394 2024-03-27T11:08:32.539 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-03-27T11:08:32.539 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:32.539 INFO:teuthology.orchestra.run.smithi114.stdout: "mds": { 2024-03-27T11:08:32.539 INFO:teuthology.orchestra.run.smithi114.stdout: "request": 126, 2024-03-27T11:08:32.539 INFO:teuthology.orchestra.run.smithi114.stdout: "reply": 124, 2024-03-27T11:08:32.539 INFO:teuthology.orchestra.run.smithi114.stdout: "reply_latency": { 2024-03-27T11:08:32.539 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 124, 2024-03-27T11:08:32.539 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.496668374, 2024-03-27T11:08:32.539 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.004005390 2024-03-27T11:08:32.539 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:32.539 INFO:teuthology.orchestra.run.smithi114.stdout: "slow_reply": 0, 2024-03-27T11:08:32.540 INFO:teuthology.orchestra.run.smithi114.stdout: "forward": 2, 2024-03-27T11:08:32.540 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_fetch_complete": 0, 2024-03-27T11:08:32.540 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_fetch_keys": 0, 2024-03-27T11:08:32.540 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_commit": 177, 2024-03-27T11:08:32.540 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_split": 0, 2024-03-27T11:08:32.540 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_merge": 0, 2024-03-27T11:08:32.540 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes": 40, 2024-03-27T11:08:32.540 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes_top": 4, 2024-03-27T11:08:32.540 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes_bottom": 36, 2024-03-27T11:08:32.540 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes_pin_tail": 0, 2024-03-27T11:08:32.540 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes_pinned": 33, 2024-03-27T11:08:32.540 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes_expired": 2, 2024-03-27T11:08:32.540 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes_with_caps": 2, 2024-03-27T11:08:32.540 INFO:teuthology.orchestra.run.smithi114.stdout: "caps": 2, 2024-03-27T11:08:32.540 INFO:teuthology.orchestra.run.smithi114.stdout: "subtrees": 2, 2024-03-27T11:08:32.541 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse": 194, 2024-03-27T11:08:32.541 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse_hit": 176, 2024-03-27T11:08:32.541 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse_forward": 0, 2024-03-27T11:08:32.541 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse_discover": 0, 2024-03-27T11:08:32.541 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse_dir_fetch": 0, 2024-03-27T11:08:32.541 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse_remote_ino": 0, 2024-03-27T11:08:32.541 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse_lock": 0, 2024-03-27T11:08:32.541 INFO:teuthology.orchestra.run.smithi114.stdout: "load_cent": 1757, 2024-03-27T11:08:32.541 INFO:teuthology.orchestra.run.smithi114.stdout: "q": 1, 2024-03-27T11:08:32.541 INFO:teuthology.orchestra.run.smithi114.stdout: "exported": 1, 2024-03-27T11:08:32.541 INFO:teuthology.orchestra.run.smithi114.stdout: "exported_inodes": 2, 2024-03-27T11:08:32.541 INFO:teuthology.orchestra.run.smithi114.stdout: "imported": 1, 2024-03-27T11:08:32.541 INFO:teuthology.orchestra.run.smithi114.stdout: "imported_inodes": 0, 2024-03-27T11:08:32.541 INFO:teuthology.orchestra.run.smithi114.stdout: "openino_dir_fetch": 0, 2024-03-27T11:08:32.542 INFO:teuthology.orchestra.run.smithi114.stdout: "openino_backtrace_fetch": 79, 2024-03-27T11:08:32.542 INFO:teuthology.orchestra.run.smithi114.stdout: "openino_peer_discover": 0, 2024-03-27T11:08:32.542 INFO:teuthology.orchestra.run.smithi114.stdout: "root_rfiles": 4, 2024-03-27T11:08:32.542 INFO:teuthology.orchestra.run.smithi114.stdout: "root_rbytes": 234, 2024-03-27T11:08:32.542 INFO:teuthology.orchestra.run.smithi114.stdout: "root_rsnaps": 0, 2024-03-27T11:08:32.542 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_backtrace_fetch": 79, 2024-03-27T11:08:32.542 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_set_tag": 0, 2024-03-27T11:08:32.542 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_backtrace_repaired": 0, 2024-03-27T11:08:32.542 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_inotable_repaired": 0, 2024-03-27T11:08:32.542 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_dir_inodes": 43, 2024-03-27T11:08:32.542 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_dir_base_inodes": 5, 2024-03-27T11:08:32.542 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_dirfrag_rstats": 38, 2024-03-27T11:08:32.542 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_file_inodes": 36, 2024-03-27T11:08:32.543 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_inode_file_caps": 0, 2024-03-27T11:08:32.543 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph_cap_op_revoke": 43, 2024-03-27T11:08:32.543 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph_cap_op_grant": 132, 2024-03-27T11:08:32.543 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph_cap_op_trunc": 0, 2024-03-27T11:08:32.543 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-27T11:08:32.543 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-27T11:08:32.543 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_client_caps": 179, 2024-03-27T11:08:32.543 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_client_caps_dirty": 143, 2024-03-27T11:08:32.543 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_client_cap_release": 5, 2024-03-27T11:08:32.543 INFO:teuthology.orchestra.run.smithi114.stdout: "process_request_cap_release": 79 2024-03-27T11:08:32.543 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:32.543 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_cache": { 2024-03-27T11:08:32.543 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_update": 16, 2024-03-27T11:08:32.544 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_update_receipt": 0, 2024-03-27T11:08:32.544 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_try_discover": 0, 2024-03-27T11:08:32.544 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_send_discover": 0, 2024-03-27T11:08:32.544 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_handle_discover": 39, 2024-03-27T11:08:32.544 INFO:teuthology.orchestra.run.smithi114.stdout: "num_strays": 0, 2024-03-27T11:08:32.544 INFO:teuthology.orchestra.run.smithi114.stdout: "num_strays_delayed": 0, 2024-03-27T11:08:32.544 INFO:teuthology.orchestra.run.smithi114.stdout: "num_strays_enqueuing": 0, 2024-03-27T11:08:32.544 INFO:teuthology.orchestra.run.smithi114.stdout: "strays_created": 20, 2024-03-27T11:08:32.544 INFO:teuthology.orchestra.run.smithi114.stdout: "strays_enqueued": 20, 2024-03-27T11:08:32.544 INFO:teuthology.orchestra.run.smithi114.stdout: "strays_reintegrated": 0, 2024-03-27T11:08:32.544 INFO:teuthology.orchestra.run.smithi114.stdout: "strays_migrated": 0, 2024-03-27T11:08:32.544 INFO:teuthology.orchestra.run.smithi114.stdout: "num_recovering_processing": 0, 2024-03-27T11:08:32.544 INFO:teuthology.orchestra.run.smithi114.stdout: "num_recovering_enqueued": 0, 2024-03-27T11:08:32.545 INFO:teuthology.orchestra.run.smithi114.stdout: "num_recovering_prioritized": 0, 2024-03-27T11:08:32.545 INFO:teuthology.orchestra.run.smithi114.stdout: "recovery_started": 0, 2024-03-27T11:08:32.545 INFO:teuthology.orchestra.run.smithi114.stdout: "recovery_completed": 0, 2024-03-27T11:08:32.545 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_quiesce_path": 0, 2024-03-27T11:08:32.545 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_quiesce_inode": 0, 2024-03-27T11:08:32.545 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_enqueue_scrub": 5, 2024-03-27T11:08:32.545 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_exportdir": 3, 2024-03-27T11:08:32.545 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_flush": 0, 2024-03-27T11:08:32.545 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_fragmentdir": 0, 2024-03-27T11:08:32.545 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_fragstats": 0, 2024-03-27T11:08:32.545 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_inodestats": 0 2024-03-27T11:08:32.545 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:32.545 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_log": { 2024-03-27T11:08:32.545 INFO:teuthology.orchestra.run.smithi114.stdout: "evlrg": 0, 2024-03-27T11:08:32.546 INFO:teuthology.orchestra.run.smithi114.stdout: "evadd": 327, 2024-03-27T11:08:32.546 INFO:teuthology.orchestra.run.smithi114.stdout: "evex": 297, 2024-03-27T11:08:32.546 INFO:teuthology.orchestra.run.smithi114.stdout: "evtrm": 297, 2024-03-27T11:08:32.546 INFO:teuthology.orchestra.run.smithi114.stdout: "ev": 30, 2024-03-27T11:08:32.546 INFO:teuthology.orchestra.run.smithi114.stdout: "evexg": 0, 2024-03-27T11:08:32.546 INFO:teuthology.orchestra.run.smithi114.stdout: "evexd": 32, 2024-03-27T11:08:32.546 INFO:teuthology.orchestra.run.smithi114.stdout: "segadd": 54, 2024-03-27T11:08:32.546 INFO:teuthology.orchestra.run.smithi114.stdout: "segex": 53, 2024-03-27T11:08:32.546 INFO:teuthology.orchestra.run.smithi114.stdout: "segtrm": 53, 2024-03-27T11:08:32.546 INFO:teuthology.orchestra.run.smithi114.stdout: "seg": 1, 2024-03-27T11:08:32.546 INFO:teuthology.orchestra.run.smithi114.stdout: "segmjr": 1, 2024-03-27T11:08:32.546 INFO:teuthology.orchestra.run.smithi114.stdout: "segexg": 0, 2024-03-27T11:08:32.546 INFO:teuthology.orchestra.run.smithi114.stdout: "segexd": 1, 2024-03-27T11:08:32.546 INFO:teuthology.orchestra.run.smithi114.stdout: "expos": 5252281, 2024-03-27T11:08:32.546 INFO:teuthology.orchestra.run.smithi114.stdout: "wrpos": 5296319, 2024-03-27T11:08:32.547 INFO:teuthology.orchestra.run.smithi114.stdout: "rdpos": 0, 2024-03-27T11:08:32.547 INFO:teuthology.orchestra.run.smithi114.stdout: "jlat": { 2024-03-27T11:08:32.547 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 202, 2024-03-27T11:08:32.547 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 1.778973287, 2024-03-27T11:08:32.547 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.008806798 2024-03-27T11:08:32.547 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:32.547 INFO:teuthology.orchestra.run.smithi114.stdout: "replayed": 0 2024-03-27T11:08:32.547 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:32.547 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_mem": { 2024-03-27T11:08:32.547 INFO:teuthology.orchestra.run.smithi114.stdout: "ino": 23, 2024-03-27T11:08:32.547 INFO:teuthology.orchestra.run.smithi114.stdout: "ino+": 50, 2024-03-27T11:08:32.547 INFO:teuthology.orchestra.run.smithi114.stdout: "ino-": 27, 2024-03-27T11:08:32.547 INFO:teuthology.orchestra.run.smithi114.stdout: "dir": 18, 2024-03-27T11:08:32.547 INFO:teuthology.orchestra.run.smithi114.stdout: "dir+": 22, 2024-03-27T11:08:32.548 INFO:teuthology.orchestra.run.smithi114.stdout: "dir-": 4, 2024-03-27T11:08:32.548 INFO:teuthology.orchestra.run.smithi114.stdout: "dn": 40, 2024-03-27T11:08:32.548 INFO:teuthology.orchestra.run.smithi114.stdout: "dn+": 64, 2024-03-27T11:08:32.548 INFO:teuthology.orchestra.run.smithi114.stdout: "dn-": 24, 2024-03-27T11:08:32.548 INFO:teuthology.orchestra.run.smithi114.stdout: "cap": 2, 2024-03-27T11:08:32.548 INFO:teuthology.orchestra.run.smithi114.stdout: "cap+": 35, 2024-03-27T11:08:32.548 INFO:teuthology.orchestra.run.smithi114.stdout: "cap-": 33, 2024-03-27T11:08:32.548 INFO:teuthology.orchestra.run.smithi114.stdout: "rss": 55296, 2024-03-27T11:08:32.548 INFO:teuthology.orchestra.run.smithi114.stdout: "heap": 231800 2024-03-27T11:08:32.548 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:32.548 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_server": { 2024-03-27T11:08:32.548 INFO:teuthology.orchestra.run.smithi114.stdout: "dispatch_client_request": 175, 2024-03-27T11:08:32.548 INFO:teuthology.orchestra.run.smithi114.stdout: "dispatch_server_request": 4, 2024-03-27T11:08:32.548 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_client_request": 126, 2024-03-27T11:08:32.548 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_client_session": 17, 2024-03-27T11:08:32.549 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_peer_request": 16, 2024-03-27T11:08:32.549 INFO:teuthology.orchestra.run.smithi114.stdout: "req_create_latency": { 2024-03-27T11:08:32.549 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 22, 2024-03-27T11:08:32.549 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.050157476, 2024-03-27T11:08:32.549 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.002279885 2024-03-27T11:08:32.549 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:32.549 INFO:teuthology.orchestra.run.smithi114.stdout: "req_getattr_latency": { 2024-03-27T11:08:32.549 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 29, 2024-03-27T11:08:32.549 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.011614603, 2024-03-27T11:08:32.549 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000400503 2024-03-27T11:08:32.549 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:32.549 INFO:teuthology.orchestra.run.smithi114.stdout: "req_getfilelock_latency": { 2024-03-27T11:08:32.549 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:32.549 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:32.549 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:32.550 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:32.550 INFO:teuthology.orchestra.run.smithi114.stdout: "req_link_latency": { 2024-03-27T11:08:32.550 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:32.550 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:32.550 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:32.550 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:32.550 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lookup_latency": { 2024-03-27T11:08:32.550 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 29, 2024-03-27T11:08:32.550 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.088901613, 2024-03-27T11:08:32.550 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.003065572 2024-03-27T11:08:32.550 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:32.550 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lookuphash_latency": { 2024-03-27T11:08:32.550 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:32.550 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:32.550 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:32.551 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:32.551 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lookupino_latency": { 2024-03-27T11:08:32.551 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:32.551 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:32.551 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:32.551 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:32.551 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lookupname_latency": { 2024-03-27T11:08:32.552 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:32.552 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:32.552 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:32.552 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:32.552 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lookupparent_latency": { 2024-03-27T11:08:32.552 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:32.552 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:32.552 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:32.552 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:32.552 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lookupsnap_latency": { 2024-03-27T11:08:32.552 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:32.552 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:32.552 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:32.552 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:32.553 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lssnap_latency": { 2024-03-27T11:08:32.553 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:32.553 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:32.553 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:32.553 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:32.553 INFO:teuthology.orchestra.run.smithi114.stdout: "req_mkdir_latency": { 2024-03-27T11:08:32.554 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 10, 2024-03-27T11:08:32.554 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.028837902, 2024-03-27T11:08:32.554 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.002883790 2024-03-27T11:08:32.554 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:32.554 INFO:teuthology.orchestra.run.smithi114.stdout: "req_mknod_latency": { 2024-03-27T11:08:32.554 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:32.554 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:32.554 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:32.554 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:32.554 INFO:teuthology.orchestra.run.smithi114.stdout: "req_mksnap_latency": { 2024-03-27T11:08:32.554 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:32.554 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:32.554 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:32.554 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:32.555 INFO:teuthology.orchestra.run.smithi114.stdout: "req_open_latency": { 2024-03-27T11:08:32.555 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 1, 2024-03-27T11:08:32.555 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000828329, 2024-03-27T11:08:32.555 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000828329 2024-03-27T11:08:32.555 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:32.555 INFO:teuthology.orchestra.run.smithi114.stdout: "req_readdir_latency": { 2024-03-27T11:08:32.555 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 3, 2024-03-27T11:08:32.555 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.021701211, 2024-03-27T11:08:32.555 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.007233737 2024-03-27T11:08:32.555 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:32.555 INFO:teuthology.orchestra.run.smithi114.stdout: "req_rename_latency": { 2024-03-27T11:08:32.555 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 2, 2024-03-27T11:08:32.556 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.007562245, 2024-03-27T11:08:32.556 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.003781122 2024-03-27T11:08:32.556 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:32.556 INFO:teuthology.orchestra.run.smithi114.stdout: "req_renamesnap_latency": { 2024-03-27T11:08:32.556 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:32.556 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:32.556 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:32.556 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:32.556 INFO:teuthology.orchestra.run.smithi114.stdout: "req_snapdiff_latency": { 2024-03-27T11:08:32.556 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:32.556 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:32.556 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:32.556 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:32.556 INFO:teuthology.orchestra.run.smithi114.stdout: "req_rmdir_latency": { 2024-03-27T11:08:32.557 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 4, 2024-03-27T11:08:32.557 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.190400172, 2024-03-27T11:08:32.557 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.047600043 2024-03-27T11:08:32.557 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:32.557 INFO:teuthology.orchestra.run.smithi114.stdout: "req_rmsnap_latency": { 2024-03-27T11:08:32.557 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:32.557 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:32.557 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:32.557 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:32.557 INFO:teuthology.orchestra.run.smithi114.stdout: "req_rmxattr_latency": { 2024-03-27T11:08:32.557 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:32.557 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:32.557 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:32.557 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:32.558 INFO:teuthology.orchestra.run.smithi114.stdout: "req_setattr_latency": { 2024-03-27T11:08:32.558 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 1, 2024-03-27T11:08:32.558 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.003647948, 2024-03-27T11:08:32.558 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.003647948 2024-03-27T11:08:32.558 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:32.558 INFO:teuthology.orchestra.run.smithi114.stdout: "req_setdirlayout_latency": { 2024-03-27T11:08:32.558 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:32.558 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:32.558 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:32.558 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:32.558 INFO:teuthology.orchestra.run.smithi114.stdout: "req_setfilelock_latency": { 2024-03-27T11:08:32.558 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:32.559 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:32.559 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:32.559 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:32.559 INFO:teuthology.orchestra.run.smithi114.stdout: "req_setlayout_latency": { 2024-03-27T11:08:32.559 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:32.559 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:32.559 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:32.559 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:32.559 INFO:teuthology.orchestra.run.smithi114.stdout: "req_setxattr_latency": { 2024-03-27T11:08:32.559 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 7, 2024-03-27T11:08:32.559 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.033582302, 2024-03-27T11:08:32.559 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.004797471 2024-03-27T11:08:32.559 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:32.559 INFO:teuthology.orchestra.run.smithi114.stdout: "req_symlink_latency": { 2024-03-27T11:08:32.559 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:32.560 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:32.560 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:32.560 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:32.560 INFO:teuthology.orchestra.run.smithi114.stdout: "req_unlink_latency": { 2024-03-27T11:08:32.560 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 16, 2024-03-27T11:08:32.560 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.059434573, 2024-03-27T11:08:32.560 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.003714660 2024-03-27T11:08:32.560 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:32.560 INFO:teuthology.orchestra.run.smithi114.stdout: "cap_revoke_eviction": 0, 2024-03-27T11:08:32.560 INFO:teuthology.orchestra.run.smithi114.stdout: "cap_acquisition_throttle": 0, 2024-03-27T11:08:32.560 INFO:teuthology.orchestra.run.smithi114.stdout: "req_getvxattr_latency": { 2024-03-27T11:08:32.560 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:32.561 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:32.561 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:32.561 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-03-27T11:08:32.561 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:32.561 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_sessions": { 2024-03-27T11:08:32.561 INFO:teuthology.orchestra.run.smithi114.stdout: "session_count": 1, 2024-03-27T11:08:32.561 INFO:teuthology.orchestra.run.smithi114.stdout: "session_add": 2, 2024-03-27T11:08:32.561 INFO:teuthology.orchestra.run.smithi114.stdout: "session_remove": 1, 2024-03-27T11:08:32.561 INFO:teuthology.orchestra.run.smithi114.stdout: "sessions_open": 1, 2024-03-27T11:08:32.561 INFO:teuthology.orchestra.run.smithi114.stdout: "sessions_stale": 0, 2024-03-27T11:08:32.561 INFO:teuthology.orchestra.run.smithi114.stdout: "total_load": 39, 2024-03-27T11:08:32.561 INFO:teuthology.orchestra.run.smithi114.stdout: "average_load": 39, 2024-03-27T11:08:32.561 INFO:teuthology.orchestra.run.smithi114.stdout: "avg_session_uptime": 723, 2024-03-27T11:08:32.561 INFO:teuthology.orchestra.run.smithi114.stdout: "mdthresh_evicted": 0 2024-03-27T11:08:32.562 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:32.562 INFO:teuthology.orchestra.run.smithi114.stdout: "mempool": { 2024-03-27T11:08:32.562 INFO:teuthology.orchestra.run.smithi114.stdout: "bloom_filter_bytes": 0, 2024-03-27T11:08:32.562 INFO:teuthology.orchestra.run.smithi114.stdout: "bloom_filter_items": 0, 2024-03-27T11:08:32.562 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_alloc_bytes": 0, 2024-03-27T11:08:32.562 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_alloc_items": 0, 2024-03-27T11:08:32.562 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_data_bytes": 0, 2024-03-27T11:08:32.562 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_data_items": 0, 2024-03-27T11:08:32.562 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-27T11:08:32.562 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_onode_items": 0, 2024-03-27T11:08:32.562 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-27T11:08:32.562 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_meta_items": 0, 2024-03-27T11:08:32.562 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_other_bytes": 0, 2024-03-27T11:08:32.563 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_other_items": 0, 2024-03-27T11:08:32.563 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-27T11:08:32.563 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_buffer_items": 0, 2024-03-27T11:08:32.563 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_extent_bytes": 0, 2024-03-27T11:08:32.563 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_extent_items": 0, 2024-03-27T11:08:32.563 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_blob_bytes": 0, 2024-03-27T11:08:32.563 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_blob_items": 0, 2024-03-27T11:08:32.563 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-27T11:08:32.563 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_shared_blob_items": 0, 2024-03-27T11:08:32.563 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-27T11:08:32.563 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_inline_bl_items": 0, 2024-03-27T11:08:32.563 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_fsck_bytes": 0, 2024-03-27T11:08:32.563 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_fsck_items": 0, 2024-03-27T11:08:32.564 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_txc_bytes": 0, 2024-03-27T11:08:32.564 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_txc_items": 0, 2024-03-27T11:08:32.564 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-27T11:08:32.564 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_writing_deferred_items": 0, 2024-03-27T11:08:32.564 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_writing_bytes": 0, 2024-03-27T11:08:32.564 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_writing_items": 0, 2024-03-27T11:08:32.564 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_bytes": 0, 2024-03-27T11:08:32.564 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_items": 0, 2024-03-27T11:08:32.564 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_file_reader_bytes": 0, 2024-03-27T11:08:32.564 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_file_reader_items": 0, 2024-03-27T11:08:32.564 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_file_writer_bytes": 0, 2024-03-27T11:08:32.564 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_file_writer_items": 0, 2024-03-27T11:08:32.564 INFO:teuthology.orchestra.run.smithi114.stdout: "buffer_anon_bytes": 309846, 2024-03-27T11:08:32.564 INFO:teuthology.orchestra.run.smithi114.stdout: "buffer_anon_items": 127, 2024-03-27T11:08:32.565 INFO:teuthology.orchestra.run.smithi114.stdout: "buffer_meta_bytes": 176, 2024-03-27T11:08:32.565 INFO:teuthology.orchestra.run.smithi114.stdout: "buffer_meta_items": 2, 2024-03-27T11:08:32.565 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_bytes": 0, 2024-03-27T11:08:32.565 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_items": 0, 2024-03-27T11:08:32.565 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_mapbl_bytes": 0, 2024-03-27T11:08:32.565 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_mapbl_items": 0, 2024-03-27T11:08:32.565 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_pglog_bytes": 0, 2024-03-27T11:08:32.565 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_pglog_items": 0, 2024-03-27T11:08:32.565 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_bytes": 7528, 2024-03-27T11:08:32.565 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_items": 158, 2024-03-27T11:08:32.565 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_mapping_bytes": 0, 2024-03-27T11:08:32.565 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_mapping_items": 0, 2024-03-27T11:08:32.565 INFO:teuthology.orchestra.run.smithi114.stdout: "pgmap_bytes": 0, 2024-03-27T11:08:32.565 INFO:teuthology.orchestra.run.smithi114.stdout: "pgmap_items": 0, 2024-03-27T11:08:32.566 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_co_bytes": 109314, 2024-03-27T11:08:32.566 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_co_items": 852, 2024-03-27T11:08:32.566 INFO:teuthology.orchestra.run.smithi114.stdout: "unittest_1_bytes": 0, 2024-03-27T11:08:32.566 INFO:teuthology.orchestra.run.smithi114.stdout: "unittest_1_items": 0, 2024-03-27T11:08:32.566 INFO:teuthology.orchestra.run.smithi114.stdout: "unittest_2_bytes": 0, 2024-03-27T11:08:32.566 INFO:teuthology.orchestra.run.smithi114.stdout: "unittest_2_items": 0 2024-03-27T11:08:32.566 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:32.566 INFO:teuthology.orchestra.run.smithi114.stdout: "objecter": { 2024-03-27T11:08:32.566 INFO:teuthology.orchestra.run.smithi114.stdout: "op_active": 10, 2024-03-27T11:08:32.566 INFO:teuthology.orchestra.run.smithi114.stdout: "op_laggy": 0, 2024-03-27T11:08:32.566 INFO:teuthology.orchestra.run.smithi114.stdout: "op_send": 3099, 2024-03-27T11:08:32.566 INFO:teuthology.orchestra.run.smithi114.stdout: "op_send_bytes": 1350455, 2024-03-27T11:08:32.566 INFO:teuthology.orchestra.run.smithi114.stdout: "op_resend": 31, 2024-03-27T11:08:32.566 INFO:teuthology.orchestra.run.smithi114.stdout: "op_reply": 3089, 2024-03-27T11:08:32.567 INFO:teuthology.orchestra.run.smithi114.stdout: "op_latency": { 2024-03-27T11:08:32.567 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 3089, 2024-03-27T11:08:32.567 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 252.652612985, 2024-03-27T11:08:32.567 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.081791069 2024-03-27T11:08:32.567 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:32.567 INFO:teuthology.orchestra.run.smithi114.stdout: "op_inflight": 9, 2024-03-27T11:08:32.567 INFO:teuthology.orchestra.run.smithi114.stdout: "oplen_avg": { 2024-03-27T11:08:32.567 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 3099, 2024-03-27T11:08:32.567 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 3573 2024-03-27T11:08:32.567 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:32.567 INFO:teuthology.orchestra.run.smithi114.stdout: "op": 3099, 2024-03-27T11:08:32.567 INFO:teuthology.orchestra.run.smithi114.stdout: "op_r": 92, 2024-03-27T11:08:32.567 INFO:teuthology.orchestra.run.smithi114.stdout: "op_w": 3007, 2024-03-27T11:08:32.568 INFO:teuthology.orchestra.run.smithi114.stdout: "op_rmw": 0, 2024-03-27T11:08:32.568 INFO:teuthology.orchestra.run.smithi114.stdout: "op_pg": 0, 2024-03-27T11:08:32.568 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_stat": 155, 2024-03-27T11:08:32.568 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_create": 62, 2024-03-27T11:08:32.568 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_read": 8, 2024-03-27T11:08:32.568 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_write": 207, 2024-03-27T11:08:32.568 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_writefull": 141, 2024-03-27T11:08:32.568 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_writesame": 0, 2024-03-27T11:08:32.568 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_append": 0, 2024-03-27T11:08:32.568 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_zero": 0, 2024-03-27T11:08:32.568 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_truncate": 0, 2024-03-27T11:08:32.568 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_delete": 2327, 2024-03-27T11:08:32.568 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_mapext": 0, 2024-03-27T11:08:32.569 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_sparse_read": 0, 2024-03-27T11:08:32.569 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_clonerange": 0, 2024-03-27T11:08:32.569 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_getxattr": 84, 2024-03-27T11:08:32.569 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_setxattr": 124, 2024-03-27T11:08:32.569 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_cmpxattr": 0, 2024-03-27T11:08:32.569 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_rmxattr": 0, 2024-03-27T11:08:32.569 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_resetxattrs": 0, 2024-03-27T11:08:32.569 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_call": 0, 2024-03-27T11:08:32.569 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_watch": 0, 2024-03-27T11:08:32.569 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_notify": 0, 2024-03-27T11:08:32.569 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_src_cmpxattr": 0, 2024-03-27T11:08:32.569 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_pgls": 0, 2024-03-27T11:08:32.569 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_pgls_filter": 0, 2024-03-27T11:08:32.569 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_other": 0, 2024-03-27T11:08:32.570 INFO:teuthology.orchestra.run.smithi114.stdout: "linger_active": 0, 2024-03-27T11:08:32.570 INFO:teuthology.orchestra.run.smithi114.stdout: "linger_send": 0, 2024-03-27T11:08:32.570 INFO:teuthology.orchestra.run.smithi114.stdout: "linger_resend": 0, 2024-03-27T11:08:32.570 INFO:teuthology.orchestra.run.smithi114.stdout: "linger_ping": 0, 2024-03-27T11:08:32.570 INFO:teuthology.orchestra.run.smithi114.stdout: "poolop_active": 0, 2024-03-27T11:08:32.570 INFO:teuthology.orchestra.run.smithi114.stdout: "poolop_send": 0, 2024-03-27T11:08:32.570 INFO:teuthology.orchestra.run.smithi114.stdout: "poolop_resend": 0, 2024-03-27T11:08:32.570 INFO:teuthology.orchestra.run.smithi114.stdout: "poolstat_active": 0, 2024-03-27T11:08:32.570 INFO:teuthology.orchestra.run.smithi114.stdout: "poolstat_send": 0, 2024-03-27T11:08:32.570 INFO:teuthology.orchestra.run.smithi114.stdout: "poolstat_resend": 0, 2024-03-27T11:08:32.570 INFO:teuthology.orchestra.run.smithi114.stdout: "statfs_active": 0, 2024-03-27T11:08:32.570 INFO:teuthology.orchestra.run.smithi114.stdout: "statfs_send": 0, 2024-03-27T11:08:32.570 INFO:teuthology.orchestra.run.smithi114.stdout: "statfs_resend": 0, 2024-03-27T11:08:32.571 INFO:teuthology.orchestra.run.smithi114.stdout: "command_active": 0, 2024-03-27T11:08:32.571 INFO:teuthology.orchestra.run.smithi114.stdout: "command_send": 0, 2024-03-27T11:08:32.571 INFO:teuthology.orchestra.run.smithi114.stdout: "command_resend": 0, 2024-03-27T11:08:32.571 INFO:teuthology.orchestra.run.smithi114.stdout: "map_epoch": 92, 2024-03-27T11:08:32.571 INFO:teuthology.orchestra.run.smithi114.stdout: "map_full": 0, 2024-03-27T11:08:32.571 INFO:teuthology.orchestra.run.smithi114.stdout: "map_inc": 3, 2024-03-27T11:08:32.571 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_sessions": 12, 2024-03-27T11:08:32.571 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_session_open": 12, 2024-03-27T11:08:32.571 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_session_close": 0, 2024-03-27T11:08:32.571 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_laggy": 0, 2024-03-27T11:08:32.571 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_wr": 442, 2024-03-27T11:08:32.571 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_rd": 0, 2024-03-27T11:08:32.571 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_del": 23 2024-03-27T11:08:32.571 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:32.572 INFO:teuthology.orchestra.run.smithi114.stdout: "oft": { 2024-03-27T11:08:32.572 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_total_objs": 1, 2024-03-27T11:08:32.572 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_total_kv_pairs": 0, 2024-03-27T11:08:32.572 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_total_updates": 0, 2024-03-27T11:08:32.572 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_total_removes": 0 2024-03-27T11:08:32.572 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:32.572 INFO:teuthology.orchestra.run.smithi114.stdout: "osd-slow-ops": { 2024-03-27T11:08:32.572 INFO:teuthology.orchestra.run.smithi114.stdout: "slow_ops_count": 0 2024-03-27T11:08:32.572 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:32.572 INFO:teuthology.orchestra.run.smithi114.stdout: "purge_queue": { 2024-03-27T11:08:32.572 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_executing_ops": 513, 2024-03-27T11:08:32.572 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_executing_ops_high_water": 518, 2024-03-27T11:08:32.572 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_executing": 1, 2024-03-27T11:08:32.572 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_executing_high_water": 6, 2024-03-27T11:08:32.572 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_executed_ops": 1546, 2024-03-27T11:08:32.573 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_executed": 10, 2024-03-27T11:08:32.573 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_item_in_journal": 10 2024-03-27T11:08:32.573 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:32.573 INFO:teuthology.orchestra.run.smithi114.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-27T11:08:32.573 INFO:teuthology.orchestra.run.smithi114.stdout: "val": 0, 2024-03-27T11:08:32.573 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 104857600, 2024-03-27T11:08:32.573 INFO:teuthology.orchestra.run.smithi114.stdout: "get_started": 0, 2024-03-27T11:08:32.573 INFO:teuthology.orchestra.run.smithi114.stdout: "get": 7330, 2024-03-27T11:08:32.573 INFO:teuthology.orchestra.run.smithi114.stdout: "get_sum": 2382111, 2024-03-27T11:08:32.573 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_fail": 0, 2024-03-27T11:08:32.573 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_success": 7330, 2024-03-27T11:08:32.573 INFO:teuthology.orchestra.run.smithi114.stdout: "take": 0, 2024-03-27T11:08:32.573 INFO:teuthology.orchestra.run.smithi114.stdout: "take_sum": 0, 2024-03-27T11:08:32.573 INFO:teuthology.orchestra.run.smithi114.stdout: "put": 7330, 2024-03-27T11:08:32.573 INFO:teuthology.orchestra.run.smithi114.stdout: "put_sum": 2382111, 2024-03-27T11:08:32.574 INFO:teuthology.orchestra.run.smithi114.stdout: "wait": { 2024-03-27T11:08:32.574 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:32.574 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:32.574 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:32.574 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-03-27T11:08:32.574 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:32.574 INFO:teuthology.orchestra.run.smithi114.stdout: "throttle-objecter_bytes": { 2024-03-27T11:08:32.574 INFO:teuthology.orchestra.run.smithi114.stdout: "val": 0, 2024-03-27T11:08:32.574 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 104857600, 2024-03-27T11:08:32.574 INFO:teuthology.orchestra.run.smithi114.stdout: "get_started": 0, 2024-03-27T11:08:32.574 INFO:teuthology.orchestra.run.smithi114.stdout: "get": 0, 2024-03-27T11:08:32.574 INFO:teuthology.orchestra.run.smithi114.stdout: "get_sum": 0, 2024-03-27T11:08:32.574 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_fail": 0, 2024-03-27T11:08:32.574 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_success": 0, 2024-03-27T11:08:32.575 INFO:teuthology.orchestra.run.smithi114.stdout: "take": 3099, 2024-03-27T11:08:32.575 INFO:teuthology.orchestra.run.smithi114.stdout: "take_sum": 1352475, 2024-03-27T11:08:32.575 INFO:teuthology.orchestra.run.smithi114.stdout: "put": 3089, 2024-03-27T11:08:32.575 INFO:teuthology.orchestra.run.smithi114.stdout: "put_sum": 1352475, 2024-03-27T11:08:32.575 INFO:teuthology.orchestra.run.smithi114.stdout: "wait": { 2024-03-27T11:08:32.575 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:32.575 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:32.575 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:32.575 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-03-27T11:08:32.575 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:32.575 INFO:teuthology.orchestra.run.smithi114.stdout: "throttle-objecter_ops": { 2024-03-27T11:08:32.575 INFO:teuthology.orchestra.run.smithi114.stdout: "val": 10, 2024-03-27T11:08:32.575 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 1024, 2024-03-27T11:08:32.575 INFO:teuthology.orchestra.run.smithi114.stdout: "get_started": 0, 2024-03-27T11:08:32.576 INFO:teuthology.orchestra.run.smithi114.stdout: "get": 0, 2024-03-27T11:08:32.576 INFO:teuthology.orchestra.run.smithi114.stdout: "get_sum": 0, 2024-03-27T11:08:32.576 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_fail": 0, 2024-03-27T11:08:32.576 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_success": 0, 2024-03-27T11:08:32.576 INFO:teuthology.orchestra.run.smithi114.stdout: "take": 3099, 2024-03-27T11:08:32.576 INFO:teuthology.orchestra.run.smithi114.stdout: "take_sum": 3099, 2024-03-27T11:08:32.576 INFO:teuthology.orchestra.run.smithi114.stdout: "put": 3089, 2024-03-27T11:08:32.576 INFO:teuthology.orchestra.run.smithi114.stdout: "put_sum": 3089, 2024-03-27T11:08:32.576 INFO:teuthology.orchestra.run.smithi114.stdout: "wait": { 2024-03-27T11:08:32.576 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:32.576 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:32.576 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:32.576 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-03-27T11:08:32.576 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:32.576 INFO:teuthology.orchestra.run.smithi114.stdout: "throttle-write_buf_throttle": { 2024-03-27T11:08:32.576 INFO:teuthology.orchestra.run.smithi114.stdout: "val": 0, 2024-03-27T11:08:32.577 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 3758096384, 2024-03-27T11:08:32.577 INFO:teuthology.orchestra.run.smithi114.stdout: "get_started": 0, 2024-03-27T11:08:32.577 INFO:teuthology.orchestra.run.smithi114.stdout: "get": 20, 2024-03-27T11:08:32.577 INFO:teuthology.orchestra.run.smithi114.stdout: "get_sum": 2020, 2024-03-27T11:08:32.577 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_fail": 0, 2024-03-27T11:08:32.577 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_success": 20, 2024-03-27T11:08:32.577 INFO:teuthology.orchestra.run.smithi114.stdout: "take": 0, 2024-03-27T11:08:32.577 INFO:teuthology.orchestra.run.smithi114.stdout: "take_sum": 0, 2024-03-27T11:08:32.577 INFO:teuthology.orchestra.run.smithi114.stdout: "put": 5, 2024-03-27T11:08:32.577 INFO:teuthology.orchestra.run.smithi114.stdout: "put_sum": 2020, 2024-03-27T11:08:32.577 INFO:teuthology.orchestra.run.smithi114.stdout: "wait": { 2024-03-27T11:08:32.577 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:32.577 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:32.577 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:32.577 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-03-27T11:08:32.578 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:32.578 INFO:teuthology.orchestra.run.smithi114.stdout: "throttle-write_buf_throttle-0x565055b68500": { 2024-03-27T11:08:32.578 INFO:teuthology.orchestra.run.smithi114.stdout: "val": 0, 2024-03-27T11:08:32.578 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 3758096384, 2024-03-27T11:08:32.578 INFO:teuthology.orchestra.run.smithi114.stdout: "get_started": 0, 2024-03-27T11:08:32.578 INFO:teuthology.orchestra.run.smithi114.stdout: "get": 327, 2024-03-27T11:08:32.578 INFO:teuthology.orchestra.run.smithi114.stdout: "get_sum": 1102015, 2024-03-27T11:08:32.578 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_fail": 0, 2024-03-27T11:08:32.578 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_success": 327, 2024-03-27T11:08:32.578 INFO:teuthology.orchestra.run.smithi114.stdout: "take": 0, 2024-03-27T11:08:32.578 INFO:teuthology.orchestra.run.smithi114.stdout: "take_sum": 0, 2024-03-27T11:08:32.578 INFO:teuthology.orchestra.run.smithi114.stdout: "put": 202, 2024-03-27T11:08:32.578 INFO:teuthology.orchestra.run.smithi114.stdout: "put_sum": 1102015, 2024-03-27T11:08:32.579 INFO:teuthology.orchestra.run.smithi114.stdout: "wait": { 2024-03-27T11:08:32.579 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:32.579 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:32.579 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:32.579 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-03-27T11:08:32.579 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-03-27T11:08:32.579 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-03-27T11:08:33.411 INFO:tasks.check_counter:Daemon mds.b mds.exported=1 2024-03-27T11:08:33.412 INFO:tasks.check_counter:Daemon mds.b mds.imported=1 2024-03-27T11:08:33.412 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=16 2024-03-27T11:08:33.412 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=0 2024-03-27T11:08:33.412 DEBUG:tasks.check_counter:Getting stats from e 2024-03-27T11:08:33.412 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell --fsid adca88c4-ec26-11ee-95d0-87774f69a715 -- ceph daemon mds.e perf dump 2024-03-27T11:08:33.763 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:08:33 smithi114 ceph-mon[35338]: pgmap v798: 97 pgs: 97 active+clean; 3.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 86 op/s 2024-03-27T11:08:33.888 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:08:33 smithi097 ceph-mon[31859]: pgmap v798: 97 pgs: 97 active+clean; 3.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 86 op/s 2024-03-27T11:08:33.923 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:08:33 smithi177 ceph-mon[35294]: pgmap v798: 97 pgs: 97 active+clean; 3.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 86 op/s 2024-03-27T11:08:33.966 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/mon.b/config 2024-03-27T11:08:35.809 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:08:35 smithi114 ceph-mon[35338]: pgmap v799: 97 pgs: 97 active+clean; 3.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 682 B/s wr, 90 op/s 2024-03-27T11:08:35.887 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:08:35 smithi097 ceph-mon[31859]: pgmap v799: 97 pgs: 97 active+clean; 3.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 682 B/s wr, 90 op/s 2024-03-27T11:08:35.923 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:08:35 smithi177 ceph-mon[35294]: pgmap v799: 97 pgs: 97 active+clean; 3.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 682 B/s wr, 90 op/s 2024-03-27T11:08:36.565 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-03-27T11:08:36.565 INFO:teuthology.orchestra.run.smithi114.stdout: "AsyncMessenger::Worker-0": { 2024-03-27T11:08:36.565 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_messages": 56, 2024-03-27T11:08:36.565 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages": 57, 2024-03-27T11:08:36.565 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_bytes": 13791, 2024-03-27T11:08:36.566 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_bytes": 234163, 2024-03-27T11:08:36.566 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_created_connections": 27, 2024-03-27T11:08:36.566 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_active_connections": 2, 2024-03-27T11:08:36.566 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_total_time": 0.032189989, 2024-03-27T11:08:36.566 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_send_time": 0.003144638, 2024-03-27T11:08:36.566 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_recv_time": 0.021213694, 2024-03-27T11:08:36.566 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_fast_dispatch_time": 0.000687531, 2024-03-27T11:08:36.566 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages_queue_lat": { 2024-03-27T11:08:36.566 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 56, 2024-03-27T11:08:36.566 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.082231352, 2024-03-27T11:08:36.566 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.001468417 2024-03-27T11:08:36.566 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:36.567 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_handle_ack_lat": { 2024-03-27T11:08:36.567 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 45, 2024-03-27T11:08:36.567 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000010857, 2024-03-27T11:08:36.567 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000241 2024-03-27T11:08:36.567 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:36.567 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_encrypted_bytes": 4080, 2024-03-27T11:08:36.567 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_encrypted_bytes": 58592 2024-03-27T11:08:36.567 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:36.567 INFO:teuthology.orchestra.run.smithi114.stdout: "AsyncMessenger::Worker-1": { 2024-03-27T11:08:36.567 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_messages": 142, 2024-03-27T11:08:36.567 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages": 326, 2024-03-27T11:08:36.567 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_bytes": 130638, 2024-03-27T11:08:36.568 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_bytes": 510292, 2024-03-27T11:08:36.568 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_created_connections": 8, 2024-03-27T11:08:36.568 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_active_connections": 5, 2024-03-27T11:08:36.568 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_total_time": 0.039858645, 2024-03-27T11:08:36.568 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_send_time": 0.020464830, 2024-03-27T11:08:36.568 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_recv_time": 0.017581165, 2024-03-27T11:08:36.568 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_fast_dispatch_time": 0.000475976, 2024-03-27T11:08:36.568 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages_queue_lat": { 2024-03-27T11:08:36.568 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 324, 2024-03-27T11:08:36.568 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.127423318, 2024-03-27T11:08:36.568 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000393281 2024-03-27T11:08:36.568 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:36.568 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_handle_ack_lat": { 2024-03-27T11:08:36.569 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 217, 2024-03-27T11:08:36.569 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000066623, 2024-03-27T11:08:36.569 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000307 2024-03-27T11:08:36.569 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:36.569 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-27T11:08:36.569 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_encrypted_bytes": 365248 2024-03-27T11:08:36.569 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:36.569 INFO:teuthology.orchestra.run.smithi114.stdout: "AsyncMessenger::Worker-2": { 2024-03-27T11:08:36.569 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_messages": 892, 2024-03-27T11:08:36.569 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages": 1013, 2024-03-27T11:08:36.569 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_bytes": 849716, 2024-03-27T11:08:36.569 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_bytes": 525013, 2024-03-27T11:08:36.569 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_created_connections": 7, 2024-03-27T11:08:36.569 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_active_connections": 4, 2024-03-27T11:08:36.570 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_total_time": 0.199233218, 2024-03-27T11:08:36.570 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_send_time": 0.104998138, 2024-03-27T11:08:36.570 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_recv_time": 0.090862946, 2024-03-27T11:08:36.570 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_fast_dispatch_time": 0.013057486, 2024-03-27T11:08:36.570 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages_queue_lat": { 2024-03-27T11:08:36.570 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 1012, 2024-03-27T11:08:36.570 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.120702330, 2024-03-27T11:08:36.570 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000119271 2024-03-27T11:08:36.570 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:36.570 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_handle_ack_lat": { 2024-03-27T11:08:36.570 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 1265, 2024-03-27T11:08:36.570 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000332784, 2024-03-27T11:08:36.570 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000263 2024-03-27T11:08:36.571 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:36.571 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_encrypted_bytes": 591008, 2024-03-27T11:08:36.571 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_encrypted_bytes": 155504 2024-03-27T11:08:36.571 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:36.571 INFO:teuthology.orchestra.run.smithi114.stdout: "cct": { 2024-03-27T11:08:36.571 INFO:teuthology.orchestra.run.smithi114.stdout: "total_workers": 1, 2024-03-27T11:08:36.571 INFO:teuthology.orchestra.run.smithi114.stdout: "unhealthy_workers": 0 2024-03-27T11:08:36.571 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:36.571 INFO:teuthology.orchestra.run.smithi114.stdout: "finisher-MDSRank": { 2024-03-27T11:08:36.571 INFO:teuthology.orchestra.run.smithi114.stdout: "queue_len": 0, 2024-03-27T11:08:36.571 INFO:teuthology.orchestra.run.smithi114.stdout: "complete_latency": { 2024-03-27T11:08:36.571 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 54, 2024-03-27T11:08:36.571 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.002546524, 2024-03-27T11:08:36.571 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000047157 2024-03-27T11:08:36.572 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-03-27T11:08:36.572 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:36.572 INFO:teuthology.orchestra.run.smithi114.stdout: "finisher-PurgeQueue": { 2024-03-27T11:08:36.572 INFO:teuthology.orchestra.run.smithi114.stdout: "queue_len": 0, 2024-03-27T11:08:36.572 INFO:teuthology.orchestra.run.smithi114.stdout: "complete_latency": { 2024-03-27T11:08:36.572 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 2, 2024-03-27T11:08:36.572 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000008436, 2024-03-27T11:08:36.572 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000004218 2024-03-27T11:08:36.572 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-03-27T11:08:36.572 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:36.572 INFO:teuthology.orchestra.run.smithi114.stdout: "mds": { 2024-03-27T11:08:36.572 INFO:teuthology.orchestra.run.smithi114.stdout: "request": 2, 2024-03-27T11:08:36.572 INFO:teuthology.orchestra.run.smithi114.stdout: "reply": 1, 2024-03-27T11:08:36.572 INFO:teuthology.orchestra.run.smithi114.stdout: "reply_latency": { 2024-03-27T11:08:36.572 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 1, 2024-03-27T11:08:36.573 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000549542, 2024-03-27T11:08:36.573 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000549542 2024-03-27T11:08:36.573 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:36.573 INFO:teuthology.orchestra.run.smithi114.stdout: "slow_reply": 0, 2024-03-27T11:08:36.573 INFO:teuthology.orchestra.run.smithi114.stdout: "forward": 1, 2024-03-27T11:08:36.573 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_fetch_complete": 0, 2024-03-27T11:08:36.573 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_fetch_keys": 0, 2024-03-27T11:08:36.573 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_commit": 11, 2024-03-27T11:08:36.573 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_split": 0, 2024-03-27T11:08:36.573 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_merge": 0, 2024-03-27T11:08:36.573 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes": 17, 2024-03-27T11:08:36.573 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes_top": 1, 2024-03-27T11:08:36.574 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes_bottom": 16, 2024-03-27T11:08:36.574 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes_pin_tail": 0, 2024-03-27T11:08:36.574 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes_pinned": 13, 2024-03-27T11:08:36.574 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes_expired": 8, 2024-03-27T11:08:36.574 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes_with_caps": 1, 2024-03-27T11:08:36.574 INFO:teuthology.orchestra.run.smithi114.stdout: "caps": 1, 2024-03-27T11:08:36.574 INFO:teuthology.orchestra.run.smithi114.stdout: "subtrees": 3, 2024-03-27T11:08:36.574 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse": 13, 2024-03-27T11:08:36.574 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse_hit": 8, 2024-03-27T11:08:36.574 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse_forward": 1, 2024-03-27T11:08:36.574 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse_discover": 4, 2024-03-27T11:08:36.574 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse_dir_fetch": 0, 2024-03-27T11:08:36.575 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse_remote_ino": 0, 2024-03-27T11:08:36.575 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse_lock": 0, 2024-03-27T11:08:36.575 INFO:teuthology.orchestra.run.smithi114.stdout: "load_cent": 0, 2024-03-27T11:08:36.575 INFO:teuthology.orchestra.run.smithi114.stdout: "q": 0, 2024-03-27T11:08:36.575 INFO:teuthology.orchestra.run.smithi114.stdout: "exported": 0, 2024-03-27T11:08:36.575 INFO:teuthology.orchestra.run.smithi114.stdout: "exported_inodes": 0, 2024-03-27T11:08:36.575 INFO:teuthology.orchestra.run.smithi114.stdout: "imported": 0, 2024-03-27T11:08:36.575 INFO:teuthology.orchestra.run.smithi114.stdout: "imported_inodes": 0, 2024-03-27T11:08:36.575 INFO:teuthology.orchestra.run.smithi114.stdout: "openino_dir_fetch": 0, 2024-03-27T11:08:36.575 INFO:teuthology.orchestra.run.smithi114.stdout: "openino_backtrace_fetch": 0, 2024-03-27T11:08:36.575 INFO:teuthology.orchestra.run.smithi114.stdout: "openino_peer_discover": 0, 2024-03-27T11:08:36.575 INFO:teuthology.orchestra.run.smithi114.stdout: "root_rfiles": 22, 2024-03-27T11:08:36.575 INFO:teuthology.orchestra.run.smithi114.stdout: "root_rbytes": 1460871314, 2024-03-27T11:08:36.575 INFO:teuthology.orchestra.run.smithi114.stdout: "root_rsnaps": 0, 2024-03-27T11:08:36.575 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_backtrace_fetch": 0, 2024-03-27T11:08:36.576 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_set_tag": 0, 2024-03-27T11:08:36.576 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_backtrace_repaired": 0, 2024-03-27T11:08:36.576 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_inotable_repaired": 0, 2024-03-27T11:08:36.576 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_dir_inodes": 0, 2024-03-27T11:08:36.576 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_dir_base_inodes": 0, 2024-03-27T11:08:36.576 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_dirfrag_rstats": 0, 2024-03-27T11:08:36.576 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_file_inodes": 0, 2024-03-27T11:08:36.576 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_inode_file_caps": 0, 2024-03-27T11:08:36.576 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph_cap_op_revoke": 0, 2024-03-27T11:08:36.576 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph_cap_op_grant": 0, 2024-03-27T11:08:36.576 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph_cap_op_trunc": 0, 2024-03-27T11:08:36.576 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-27T11:08:36.576 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-27T11:08:36.576 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_client_caps": 0, 2024-03-27T11:08:36.576 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_client_caps_dirty": 0, 2024-03-27T11:08:36.577 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_client_cap_release": 0, 2024-03-27T11:08:36.577 INFO:teuthology.orchestra.run.smithi114.stdout: "process_request_cap_release": 0 2024-03-27T11:08:36.577 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:36.577 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_cache": { 2024-03-27T11:08:36.577 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_update": 0, 2024-03-27T11:08:36.577 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_update_receipt": 16, 2024-03-27T11:08:36.577 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_try_discover": 10, 2024-03-27T11:08:36.577 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_send_discover": 12, 2024-03-27T11:08:36.577 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_handle_discover": 0, 2024-03-27T11:08:36.577 INFO:teuthology.orchestra.run.smithi114.stdout: "num_strays": 0, 2024-03-27T11:08:36.577 INFO:teuthology.orchestra.run.smithi114.stdout: "num_strays_delayed": 0, 2024-03-27T11:08:36.577 INFO:teuthology.orchestra.run.smithi114.stdout: "num_strays_enqueuing": 0, 2024-03-27T11:08:36.577 INFO:teuthology.orchestra.run.smithi114.stdout: "strays_created": 0, 2024-03-27T11:08:36.577 INFO:teuthology.orchestra.run.smithi114.stdout: "strays_enqueued": 0, 2024-03-27T11:08:36.578 INFO:teuthology.orchestra.run.smithi114.stdout: "strays_reintegrated": 0, 2024-03-27T11:08:36.578 INFO:teuthology.orchestra.run.smithi114.stdout: "strays_migrated": 0, 2024-03-27T11:08:36.578 INFO:teuthology.orchestra.run.smithi114.stdout: "num_recovering_processing": 0, 2024-03-27T11:08:36.578 INFO:teuthology.orchestra.run.smithi114.stdout: "num_recovering_enqueued": 0, 2024-03-27T11:08:36.578 INFO:teuthology.orchestra.run.smithi114.stdout: "num_recovering_prioritized": 0, 2024-03-27T11:08:36.578 INFO:teuthology.orchestra.run.smithi114.stdout: "recovery_started": 0, 2024-03-27T11:08:36.578 INFO:teuthology.orchestra.run.smithi114.stdout: "recovery_completed": 0, 2024-03-27T11:08:36.578 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_quiesce_path": 0, 2024-03-27T11:08:36.578 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_quiesce_inode": 0, 2024-03-27T11:08:36.578 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_enqueue_scrub": 0, 2024-03-27T11:08:36.578 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_exportdir": 0, 2024-03-27T11:08:36.578 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_flush": 0, 2024-03-27T11:08:36.578 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_fragmentdir": 0, 2024-03-27T11:08:36.579 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_fragstats": 0, 2024-03-27T11:08:36.579 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_inodestats": 0 2024-03-27T11:08:36.579 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:36.579 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_log": { 2024-03-27T11:08:36.579 INFO:teuthology.orchestra.run.smithi114.stdout: "evlrg": 0, 2024-03-27T11:08:36.579 INFO:teuthology.orchestra.run.smithi114.stdout: "evadd": 5, 2024-03-27T11:08:36.579 INFO:teuthology.orchestra.run.smithi114.stdout: "evex": 0, 2024-03-27T11:08:36.579 INFO:teuthology.orchestra.run.smithi114.stdout: "evtrm": 0, 2024-03-27T11:08:36.579 INFO:teuthology.orchestra.run.smithi114.stdout: "ev": 5, 2024-03-27T11:08:36.579 INFO:teuthology.orchestra.run.smithi114.stdout: "evexg": 0, 2024-03-27T11:08:36.579 INFO:teuthology.orchestra.run.smithi114.stdout: "evexd": 0, 2024-03-27T11:08:36.579 INFO:teuthology.orchestra.run.smithi114.stdout: "segadd": 2, 2024-03-27T11:08:36.579 INFO:teuthology.orchestra.run.smithi114.stdout: "segex": 0, 2024-03-27T11:08:36.580 INFO:teuthology.orchestra.run.smithi114.stdout: "segtrm": 0, 2024-03-27T11:08:36.580 INFO:teuthology.orchestra.run.smithi114.stdout: "seg": 2, 2024-03-27T11:08:36.580 INFO:teuthology.orchestra.run.smithi114.stdout: "segmjr": 2, 2024-03-27T11:08:36.580 INFO:teuthology.orchestra.run.smithi114.stdout: "segexg": 0, 2024-03-27T11:08:36.580 INFO:teuthology.orchestra.run.smithi114.stdout: "segexd": 0, 2024-03-27T11:08:36.580 INFO:teuthology.orchestra.run.smithi114.stdout: "expos": 4194304, 2024-03-27T11:08:36.580 INFO:teuthology.orchestra.run.smithi114.stdout: "wrpos": 4195917, 2024-03-27T11:08:36.580 INFO:teuthology.orchestra.run.smithi114.stdout: "rdpos": 0, 2024-03-27T11:08:36.580 INFO:teuthology.orchestra.run.smithi114.stdout: "jlat": { 2024-03-27T11:08:36.580 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 4, 2024-03-27T11:08:36.580 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.012836369, 2024-03-27T11:08:36.580 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.003209092 2024-03-27T11:08:36.580 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:36.580 INFO:teuthology.orchestra.run.smithi114.stdout: "replayed": 0 2024-03-27T11:08:36.580 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:36.581 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_mem": { 2024-03-27T11:08:36.581 INFO:teuthology.orchestra.run.smithi114.stdout: "ino": 21, 2024-03-27T11:08:36.581 INFO:teuthology.orchestra.run.smithi114.stdout: "ino+": 25, 2024-03-27T11:08:36.581 INFO:teuthology.orchestra.run.smithi114.stdout: "ino-": 4, 2024-03-27T11:08:36.581 INFO:teuthology.orchestra.run.smithi114.stdout: "dir": 20, 2024-03-27T11:08:36.581 INFO:teuthology.orchestra.run.smithi114.stdout: "dir+": 26, 2024-03-27T11:08:36.581 INFO:teuthology.orchestra.run.smithi114.stdout: "dir-": 6, 2024-03-27T11:08:36.581 INFO:teuthology.orchestra.run.smithi114.stdout: "dn": 17, 2024-03-27T11:08:36.581 INFO:teuthology.orchestra.run.smithi114.stdout: "dn+": 25, 2024-03-27T11:08:36.581 INFO:teuthology.orchestra.run.smithi114.stdout: "dn-": 8, 2024-03-27T11:08:36.581 INFO:teuthology.orchestra.run.smithi114.stdout: "cap": 1, 2024-03-27T11:08:36.581 INFO:teuthology.orchestra.run.smithi114.stdout: "cap+": 1, 2024-03-27T11:08:36.581 INFO:teuthology.orchestra.run.smithi114.stdout: "cap-": 0, 2024-03-27T11:08:36.581 INFO:teuthology.orchestra.run.smithi114.stdout: "rss": 51712, 2024-03-27T11:08:36.581 INFO:teuthology.orchestra.run.smithi114.stdout: "heap": 223608 2024-03-27T11:08:36.582 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:36.582 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_server": { 2024-03-27T11:08:36.582 INFO:teuthology.orchestra.run.smithi114.stdout: "dispatch_client_request": 2, 2024-03-27T11:08:36.582 INFO:teuthology.orchestra.run.smithi114.stdout: "dispatch_server_request": 1, 2024-03-27T11:08:36.582 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_client_request": 2, 2024-03-27T11:08:36.582 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_client_session": 14, 2024-03-27T11:08:36.582 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_peer_request": 2, 2024-03-27T11:08:36.582 INFO:teuthology.orchestra.run.smithi114.stdout: "req_create_latency": { 2024-03-27T11:08:36.582 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:36.582 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:36.582 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:36.582 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:36.582 INFO:teuthology.orchestra.run.smithi114.stdout: "req_getattr_latency": { 2024-03-27T11:08:36.583 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 1, 2024-03-27T11:08:36.583 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000549542, 2024-03-27T11:08:36.583 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000549542 2024-03-27T11:08:36.583 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:36.583 INFO:teuthology.orchestra.run.smithi114.stdout: "req_getfilelock_latency": { 2024-03-27T11:08:36.583 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:36.583 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:36.583 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:36.583 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:36.583 INFO:teuthology.orchestra.run.smithi114.stdout: "req_link_latency": { 2024-03-27T11:08:36.583 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:36.583 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:36.583 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:36.583 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:36.584 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lookup_latency": { 2024-03-27T11:08:36.584 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:36.584 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:36.584 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:36.584 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:36.584 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lookuphash_latency": { 2024-03-27T11:08:36.584 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:36.584 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:36.584 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:36.584 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:36.584 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lookupino_latency": { 2024-03-27T11:08:36.584 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:36.584 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:36.584 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:36.585 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:36.585 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lookupname_latency": { 2024-03-27T11:08:36.585 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:36.585 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:36.585 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:36.585 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:36.585 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lookupparent_latency": { 2024-03-27T11:08:36.585 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:36.585 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:36.585 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:36.585 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:36.585 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lookupsnap_latency": { 2024-03-27T11:08:36.585 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:36.586 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:36.586 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:36.586 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:36.586 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lssnap_latency": { 2024-03-27T11:08:36.586 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:36.586 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:36.586 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:36.586 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:36.586 INFO:teuthology.orchestra.run.smithi114.stdout: "req_mkdir_latency": { 2024-03-27T11:08:36.587 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:36.587 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:36.587 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:36.587 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:36.587 INFO:teuthology.orchestra.run.smithi114.stdout: "req_mknod_latency": { 2024-03-27T11:08:36.587 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:36.587 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:36.587 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:36.587 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:36.588 INFO:teuthology.orchestra.run.smithi114.stdout: "req_mksnap_latency": { 2024-03-27T11:08:36.588 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:36.588 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:36.588 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:36.588 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:36.588 INFO:teuthology.orchestra.run.smithi114.stdout: "req_open_latency": { 2024-03-27T11:08:36.588 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:36.588 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:36.588 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:36.588 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:36.588 INFO:teuthology.orchestra.run.smithi114.stdout: "req_readdir_latency": { 2024-03-27T11:08:36.588 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:36.588 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:36.589 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:36.589 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:36.589 INFO:teuthology.orchestra.run.smithi114.stdout: "req_rename_latency": { 2024-03-27T11:08:36.589 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:36.589 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:36.589 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:36.589 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:36.589 INFO:teuthology.orchestra.run.smithi114.stdout: "req_renamesnap_latency": { 2024-03-27T11:08:36.589 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:36.589 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:36.589 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:36.589 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:36.589 INFO:teuthology.orchestra.run.smithi114.stdout: "req_snapdiff_latency": { 2024-03-27T11:08:36.590 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:36.590 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:36.590 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:36.590 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:36.590 INFO:teuthology.orchestra.run.smithi114.stdout: "req_rmdir_latency": { 2024-03-27T11:08:36.590 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:36.590 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:36.590 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:36.590 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:36.590 INFO:teuthology.orchestra.run.smithi114.stdout: "req_rmsnap_latency": { 2024-03-27T11:08:36.590 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:36.590 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:36.591 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:36.591 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:36.591 INFO:teuthology.orchestra.run.smithi114.stdout: "req_rmxattr_latency": { 2024-03-27T11:08:36.591 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:36.591 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:36.591 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:36.591 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:36.591 INFO:teuthology.orchestra.run.smithi114.stdout: "req_setattr_latency": { 2024-03-27T11:08:36.591 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:36.591 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:36.591 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:36.591 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:36.591 INFO:teuthology.orchestra.run.smithi114.stdout: "req_setdirlayout_latency": { 2024-03-27T11:08:36.592 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:36.592 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:36.592 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:36.592 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:36.592 INFO:teuthology.orchestra.run.smithi114.stdout: "req_setfilelock_latency": { 2024-03-27T11:08:36.592 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:36.592 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:36.592 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:36.592 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:36.592 INFO:teuthology.orchestra.run.smithi114.stdout: "req_setlayout_latency": { 2024-03-27T11:08:36.592 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:36.592 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:36.592 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:36.593 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:36.593 INFO:teuthology.orchestra.run.smithi114.stdout: "req_setxattr_latency": { 2024-03-27T11:08:36.593 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:36.593 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:36.593 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:36.593 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:36.593 INFO:teuthology.orchestra.run.smithi114.stdout: "req_symlink_latency": { 2024-03-27T11:08:36.593 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:36.593 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:36.593 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:36.593 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:36.593 INFO:teuthology.orchestra.run.smithi114.stdout: "req_unlink_latency": { 2024-03-27T11:08:36.593 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:36.594 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:36.594 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:36.594 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:36.594 INFO:teuthology.orchestra.run.smithi114.stdout: "cap_revoke_eviction": 0, 2024-03-27T11:08:36.594 INFO:teuthology.orchestra.run.smithi114.stdout: "cap_acquisition_throttle": 0, 2024-03-27T11:08:36.594 INFO:teuthology.orchestra.run.smithi114.stdout: "req_getvxattr_latency": { 2024-03-27T11:08:36.594 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:36.594 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:36.594 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:36.594 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-03-27T11:08:36.594 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:36.594 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_sessions": { 2024-03-27T11:08:36.595 INFO:teuthology.orchestra.run.smithi114.stdout: "session_count": 1, 2024-03-27T11:08:36.595 INFO:teuthology.orchestra.run.smithi114.stdout: "session_add": 2, 2024-03-27T11:08:36.595 INFO:teuthology.orchestra.run.smithi114.stdout: "session_remove": 1, 2024-03-27T11:08:36.595 INFO:teuthology.orchestra.run.smithi114.stdout: "sessions_open": 1, 2024-03-27T11:08:36.595 INFO:teuthology.orchestra.run.smithi114.stdout: "sessions_stale": 0, 2024-03-27T11:08:36.595 INFO:teuthology.orchestra.run.smithi114.stdout: "total_load": 1, 2024-03-27T11:08:36.595 INFO:teuthology.orchestra.run.smithi114.stdout: "average_load": 1, 2024-03-27T11:08:36.595 INFO:teuthology.orchestra.run.smithi114.stdout: "avg_session_uptime": 726, 2024-03-27T11:08:36.595 INFO:teuthology.orchestra.run.smithi114.stdout: "mdthresh_evicted": 0 2024-03-27T11:08:36.595 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:36.595 INFO:teuthology.orchestra.run.smithi114.stdout: "mempool": { 2024-03-27T11:08:36.595 INFO:teuthology.orchestra.run.smithi114.stdout: "bloom_filter_bytes": 0, 2024-03-27T11:08:36.595 INFO:teuthology.orchestra.run.smithi114.stdout: "bloom_filter_items": 0, 2024-03-27T11:08:36.596 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_alloc_bytes": 0, 2024-03-27T11:08:36.596 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_alloc_items": 0, 2024-03-27T11:08:36.596 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_data_bytes": 0, 2024-03-27T11:08:36.596 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_data_items": 0, 2024-03-27T11:08:36.596 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-27T11:08:36.596 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_onode_items": 0, 2024-03-27T11:08:36.596 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-27T11:08:36.596 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_meta_items": 0, 2024-03-27T11:08:36.596 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_other_bytes": 0, 2024-03-27T11:08:36.596 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_other_items": 0, 2024-03-27T11:08:36.596 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-27T11:08:36.596 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_buffer_items": 0, 2024-03-27T11:08:36.596 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_extent_bytes": 0, 2024-03-27T11:08:36.596 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_extent_items": 0, 2024-03-27T11:08:36.597 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_blob_bytes": 0, 2024-03-27T11:08:36.597 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_blob_items": 0, 2024-03-27T11:08:36.597 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-27T11:08:36.597 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_shared_blob_items": 0, 2024-03-27T11:08:36.597 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-27T11:08:36.597 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_inline_bl_items": 0, 2024-03-27T11:08:36.597 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_fsck_bytes": 0, 2024-03-27T11:08:36.597 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_fsck_items": 0, 2024-03-27T11:08:36.597 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_txc_bytes": 0, 2024-03-27T11:08:36.597 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_txc_items": 0, 2024-03-27T11:08:36.597 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-27T11:08:36.597 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_writing_deferred_items": 0, 2024-03-27T11:08:36.597 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_writing_bytes": 0, 2024-03-27T11:08:36.597 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_writing_items": 0, 2024-03-27T11:08:36.597 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_bytes": 0, 2024-03-27T11:08:36.598 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_items": 0, 2024-03-27T11:08:36.598 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_file_reader_bytes": 0, 2024-03-27T11:08:36.598 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_file_reader_items": 0, 2024-03-27T11:08:36.598 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_file_writer_bytes": 0, 2024-03-27T11:08:36.598 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_file_writer_items": 0, 2024-03-27T11:08:36.598 INFO:teuthology.orchestra.run.smithi114.stdout: "buffer_anon_bytes": 266422, 2024-03-27T11:08:36.598 INFO:teuthology.orchestra.run.smithi114.stdout: "buffer_anon_items": 88, 2024-03-27T11:08:36.598 INFO:teuthology.orchestra.run.smithi114.stdout: "buffer_meta_bytes": 0, 2024-03-27T11:08:36.598 INFO:teuthology.orchestra.run.smithi114.stdout: "buffer_meta_items": 0, 2024-03-27T11:08:36.598 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_bytes": 0, 2024-03-27T11:08:36.598 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_items": 0, 2024-03-27T11:08:36.598 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_mapbl_bytes": 0, 2024-03-27T11:08:36.598 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_mapbl_items": 0, 2024-03-27T11:08:36.599 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_pglog_bytes": 0, 2024-03-27T11:08:36.599 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_pglog_items": 0, 2024-03-27T11:08:36.599 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_bytes": 7528, 2024-03-27T11:08:36.599 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_items": 158, 2024-03-27T11:08:36.599 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_mapping_bytes": 0, 2024-03-27T11:08:36.599 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_mapping_items": 0, 2024-03-27T11:08:36.599 INFO:teuthology.orchestra.run.smithi114.stdout: "pgmap_bytes": 0, 2024-03-27T11:08:36.599 INFO:teuthology.orchestra.run.smithi114.stdout: "pgmap_items": 0, 2024-03-27T11:08:36.599 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_co_bytes": 85141, 2024-03-27T11:08:36.599 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_co_items": 336, 2024-03-27T11:08:36.599 INFO:teuthology.orchestra.run.smithi114.stdout: "unittest_1_bytes": 0, 2024-03-27T11:08:36.599 INFO:teuthology.orchestra.run.smithi114.stdout: "unittest_1_items": 0, 2024-03-27T11:08:36.599 INFO:teuthology.orchestra.run.smithi114.stdout: "unittest_2_bytes": 0, 2024-03-27T11:08:36.600 INFO:teuthology.orchestra.run.smithi114.stdout: "unittest_2_items": 0 2024-03-27T11:08:36.600 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:36.600 INFO:teuthology.orchestra.run.smithi114.stdout: "objecter": { 2024-03-27T11:08:36.600 INFO:teuthology.orchestra.run.smithi114.stdout: "op_active": 0, 2024-03-27T11:08:36.600 INFO:teuthology.orchestra.run.smithi114.stdout: "op_laggy": 0, 2024-03-27T11:08:36.600 INFO:teuthology.orchestra.run.smithi114.stdout: "op_send": 41, 2024-03-27T11:08:36.600 INFO:teuthology.orchestra.run.smithi114.stdout: "op_send_bytes": 12009, 2024-03-27T11:08:36.600 INFO:teuthology.orchestra.run.smithi114.stdout: "op_resend": 33, 2024-03-27T11:08:36.600 INFO:teuthology.orchestra.run.smithi114.stdout: "op_reply": 41, 2024-03-27T11:08:36.600 INFO:teuthology.orchestra.run.smithi114.stdout: "op_latency": { 2024-03-27T11:08:36.600 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 41, 2024-03-27T11:08:36.600 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.501989014, 2024-03-27T11:08:36.600 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.012243634 2024-03-27T11:08:36.600 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:36.600 INFO:teuthology.orchestra.run.smithi114.stdout: "op_inflight": 0, 2024-03-27T11:08:36.601 INFO:teuthology.orchestra.run.smithi114.stdout: "oplen_avg": { 2024-03-27T11:08:36.601 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 41, 2024-03-27T11:08:36.601 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 64 2024-03-27T11:08:36.601 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:36.601 INFO:teuthology.orchestra.run.smithi114.stdout: "op": 41, 2024-03-27T11:08:36.601 INFO:teuthology.orchestra.run.smithi114.stdout: "op_r": 0, 2024-03-27T11:08:36.601 INFO:teuthology.orchestra.run.smithi114.stdout: "op_w": 41, 2024-03-27T11:08:36.601 INFO:teuthology.orchestra.run.smithi114.stdout: "op_rmw": 0, 2024-03-27T11:08:36.601 INFO:teuthology.orchestra.run.smithi114.stdout: "op_pg": 0, 2024-03-27T11:08:36.601 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_stat": 0, 2024-03-27T11:08:36.601 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_create": 10, 2024-03-27T11:08:36.601 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_read": 0, 2024-03-27T11:08:36.601 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_write": 4, 2024-03-27T11:08:36.601 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_writefull": 7, 2024-03-27T11:08:36.601 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_writesame": 0, 2024-03-27T11:08:36.602 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_append": 0, 2024-03-27T11:08:36.602 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_zero": 0, 2024-03-27T11:08:36.602 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_truncate": 0, 2024-03-27T11:08:36.602 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_delete": 6, 2024-03-27T11:08:36.602 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_mapext": 0, 2024-03-27T11:08:36.602 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_sparse_read": 0, 2024-03-27T11:08:36.602 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_clonerange": 0, 2024-03-27T11:08:36.602 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_getxattr": 0, 2024-03-27T11:08:36.602 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_setxattr": 20, 2024-03-27T11:08:36.602 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_cmpxattr": 0, 2024-03-27T11:08:36.602 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_rmxattr": 0, 2024-03-27T11:08:36.602 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_resetxattrs": 0, 2024-03-27T11:08:36.603 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_call": 0, 2024-03-27T11:08:36.603 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_watch": 0, 2024-03-27T11:08:36.603 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_notify": 0, 2024-03-27T11:08:36.603 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_src_cmpxattr": 0, 2024-03-27T11:08:36.603 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_pgls": 0, 2024-03-27T11:08:36.603 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_pgls_filter": 0, 2024-03-27T11:08:36.603 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_other": 0, 2024-03-27T11:08:36.603 INFO:teuthology.orchestra.run.smithi114.stdout: "linger_active": 0, 2024-03-27T11:08:36.603 INFO:teuthology.orchestra.run.smithi114.stdout: "linger_send": 0, 2024-03-27T11:08:36.603 INFO:teuthology.orchestra.run.smithi114.stdout: "linger_resend": 0, 2024-03-27T11:08:36.603 INFO:teuthology.orchestra.run.smithi114.stdout: "linger_ping": 0, 2024-03-27T11:08:36.603 INFO:teuthology.orchestra.run.smithi114.stdout: "poolop_active": 0, 2024-03-27T11:08:36.603 INFO:teuthology.orchestra.run.smithi114.stdout: "poolop_send": 0, 2024-03-27T11:08:36.603 INFO:teuthology.orchestra.run.smithi114.stdout: "poolop_resend": 0, 2024-03-27T11:08:36.604 INFO:teuthology.orchestra.run.smithi114.stdout: "poolstat_active": 0, 2024-03-27T11:08:36.604 INFO:teuthology.orchestra.run.smithi114.stdout: "poolstat_send": 0, 2024-03-27T11:08:36.604 INFO:teuthology.orchestra.run.smithi114.stdout: "poolstat_resend": 0, 2024-03-27T11:08:36.604 INFO:teuthology.orchestra.run.smithi114.stdout: "statfs_active": 0, 2024-03-27T11:08:36.604 INFO:teuthology.orchestra.run.smithi114.stdout: "statfs_send": 0, 2024-03-27T11:08:36.604 INFO:teuthology.orchestra.run.smithi114.stdout: "statfs_resend": 0, 2024-03-27T11:08:36.604 INFO:teuthology.orchestra.run.smithi114.stdout: "command_active": 0, 2024-03-27T11:08:36.604 INFO:teuthology.orchestra.run.smithi114.stdout: "command_send": 0, 2024-03-27T11:08:36.604 INFO:teuthology.orchestra.run.smithi114.stdout: "command_resend": 0, 2024-03-27T11:08:36.604 INFO:teuthology.orchestra.run.smithi114.stdout: "map_epoch": 92, 2024-03-27T11:08:36.604 INFO:teuthology.orchestra.run.smithi114.stdout: "map_full": 0, 2024-03-27T11:08:36.604 INFO:teuthology.orchestra.run.smithi114.stdout: "map_inc": 2, 2024-03-27T11:08:36.604 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_sessions": 8, 2024-03-27T11:08:36.605 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_session_open": 8, 2024-03-27T11:08:36.605 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_session_close": 0, 2024-03-27T11:08:36.605 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_laggy": 0, 2024-03-27T11:08:36.605 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_wr": 16, 2024-03-27T11:08:36.605 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_rd": 0, 2024-03-27T11:08:36.605 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_del": 1 2024-03-27T11:08:36.605 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:36.605 INFO:teuthology.orchestra.run.smithi114.stdout: "oft": { 2024-03-27T11:08:36.605 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_total_objs": 1, 2024-03-27T11:08:36.605 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_total_kv_pairs": 0, 2024-03-27T11:08:36.605 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_total_updates": 0, 2024-03-27T11:08:36.605 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_total_removes": 0 2024-03-27T11:08:36.605 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:36.605 INFO:teuthology.orchestra.run.smithi114.stdout: "osd-slow-ops": { 2024-03-27T11:08:36.605 INFO:teuthology.orchestra.run.smithi114.stdout: "slow_ops_count": 0 2024-03-27T11:08:36.606 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:36.606 INFO:teuthology.orchestra.run.smithi114.stdout: "purge_queue": { 2024-03-27T11:08:36.606 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_executing_ops": 0, 2024-03-27T11:08:36.606 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_executing_ops_high_water": 0, 2024-03-27T11:08:36.606 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_executing": 0, 2024-03-27T11:08:36.606 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_executing_high_water": 0, 2024-03-27T11:08:36.606 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_executed_ops": 0, 2024-03-27T11:08:36.606 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_executed": 0, 2024-03-27T11:08:36.606 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_item_in_journal": 0 2024-03-27T11:08:36.606 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:36.606 INFO:teuthology.orchestra.run.smithi114.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-27T11:08:36.606 INFO:teuthology.orchestra.run.smithi114.stdout: "val": 0, 2024-03-27T11:08:36.606 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 104857600, 2024-03-27T11:08:36.606 INFO:teuthology.orchestra.run.smithi114.stdout: "get_started": 0, 2024-03-27T11:08:36.607 INFO:teuthology.orchestra.run.smithi114.stdout: "get": 1086, 2024-03-27T11:08:36.607 INFO:teuthology.orchestra.run.smithi114.stdout: "get_sum": 878126, 2024-03-27T11:08:36.607 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_fail": 0, 2024-03-27T11:08:36.607 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_success": 1086, 2024-03-27T11:08:36.607 INFO:teuthology.orchestra.run.smithi114.stdout: "take": 0, 2024-03-27T11:08:36.607 INFO:teuthology.orchestra.run.smithi114.stdout: "take_sum": 0, 2024-03-27T11:08:36.607 INFO:teuthology.orchestra.run.smithi114.stdout: "put": 1086, 2024-03-27T11:08:36.607 INFO:teuthology.orchestra.run.smithi114.stdout: "put_sum": 878126, 2024-03-27T11:08:36.607 INFO:teuthology.orchestra.run.smithi114.stdout: "wait": { 2024-03-27T11:08:36.607 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:36.607 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:36.607 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:36.607 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-03-27T11:08:36.607 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:36.607 INFO:teuthology.orchestra.run.smithi114.stdout: "throttle-objecter_bytes": { 2024-03-27T11:08:36.608 INFO:teuthology.orchestra.run.smithi114.stdout: "val": 0, 2024-03-27T11:08:36.608 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 104857600, 2024-03-27T11:08:36.608 INFO:teuthology.orchestra.run.smithi114.stdout: "get_started": 0, 2024-03-27T11:08:36.608 INFO:teuthology.orchestra.run.smithi114.stdout: "get": 0, 2024-03-27T11:08:36.608 INFO:teuthology.orchestra.run.smithi114.stdout: "get_sum": 0, 2024-03-27T11:08:36.608 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_fail": 0, 2024-03-27T11:08:36.608 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_success": 0, 2024-03-27T11:08:36.608 INFO:teuthology.orchestra.run.smithi114.stdout: "take": 41, 2024-03-27T11:08:36.608 INFO:teuthology.orchestra.run.smithi114.stdout: "take_sum": 12009, 2024-03-27T11:08:36.608 INFO:teuthology.orchestra.run.smithi114.stdout: "put": 41, 2024-03-27T11:08:36.608 INFO:teuthology.orchestra.run.smithi114.stdout: "put_sum": 12009, 2024-03-27T11:08:36.608 INFO:teuthology.orchestra.run.smithi114.stdout: "wait": { 2024-03-27T11:08:36.608 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:36.608 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:36.609 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:36.609 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-03-27T11:08:36.609 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:36.609 INFO:teuthology.orchestra.run.smithi114.stdout: "throttle-objecter_ops": { 2024-03-27T11:08:36.609 INFO:teuthology.orchestra.run.smithi114.stdout: "val": 0, 2024-03-27T11:08:36.609 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 1024, 2024-03-27T11:08:36.609 INFO:teuthology.orchestra.run.smithi114.stdout: "get_started": 0, 2024-03-27T11:08:36.609 INFO:teuthology.orchestra.run.smithi114.stdout: "get": 0, 2024-03-27T11:08:36.609 INFO:teuthology.orchestra.run.smithi114.stdout: "get_sum": 0, 2024-03-27T11:08:36.609 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_fail": 0, 2024-03-27T11:08:36.609 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_success": 0, 2024-03-27T11:08:36.609 INFO:teuthology.orchestra.run.smithi114.stdout: "take": 41, 2024-03-27T11:08:36.609 INFO:teuthology.orchestra.run.smithi114.stdout: "take_sum": 41, 2024-03-27T11:08:36.609 INFO:teuthology.orchestra.run.smithi114.stdout: "put": 41, 2024-03-27T11:08:36.610 INFO:teuthology.orchestra.run.smithi114.stdout: "put_sum": 41, 2024-03-27T11:08:36.610 INFO:teuthology.orchestra.run.smithi114.stdout: "wait": { 2024-03-27T11:08:36.610 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:36.610 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:36.610 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:36.610 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-03-27T11:08:36.610 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:36.610 INFO:teuthology.orchestra.run.smithi114.stdout: "throttle-write_buf_throttle": { 2024-03-27T11:08:36.610 INFO:teuthology.orchestra.run.smithi114.stdout: "val": 0, 2024-03-27T11:08:36.610 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 3758096384, 2024-03-27T11:08:36.610 INFO:teuthology.orchestra.run.smithi114.stdout: "get_started": 0, 2024-03-27T11:08:36.610 INFO:teuthology.orchestra.run.smithi114.stdout: "get": 0, 2024-03-27T11:08:36.610 INFO:teuthology.orchestra.run.smithi114.stdout: "get_sum": 0, 2024-03-27T11:08:36.611 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_fail": 0, 2024-03-27T11:08:36.611 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_success": 0, 2024-03-27T11:08:36.611 INFO:teuthology.orchestra.run.smithi114.stdout: "take": 0, 2024-03-27T11:08:36.611 INFO:teuthology.orchestra.run.smithi114.stdout: "take_sum": 0, 2024-03-27T11:08:36.611 INFO:teuthology.orchestra.run.smithi114.stdout: "put": 0, 2024-03-27T11:08:36.611 INFO:teuthology.orchestra.run.smithi114.stdout: "put_sum": 0, 2024-03-27T11:08:36.611 INFO:teuthology.orchestra.run.smithi114.stdout: "wait": { 2024-03-27T11:08:36.611 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:36.611 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:36.611 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:36.611 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-03-27T11:08:36.611 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:36.611 INFO:teuthology.orchestra.run.smithi114.stdout: "throttle-write_buf_throttle-0x55e5776fc500": { 2024-03-27T11:08:36.611 INFO:teuthology.orchestra.run.smithi114.stdout: "val": 0, 2024-03-27T11:08:36.611 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 3758096384, 2024-03-27T11:08:36.612 INFO:teuthology.orchestra.run.smithi114.stdout: "get_started": 0, 2024-03-27T11:08:36.612 INFO:teuthology.orchestra.run.smithi114.stdout: "get": 5, 2024-03-27T11:08:36.612 INFO:teuthology.orchestra.run.smithi114.stdout: "get_sum": 1613, 2024-03-27T11:08:36.612 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_fail": 0, 2024-03-27T11:08:36.612 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_success": 5, 2024-03-27T11:08:36.612 INFO:teuthology.orchestra.run.smithi114.stdout: "take": 0, 2024-03-27T11:08:36.612 INFO:teuthology.orchestra.run.smithi114.stdout: "take_sum": 0, 2024-03-27T11:08:36.612 INFO:teuthology.orchestra.run.smithi114.stdout: "put": 4, 2024-03-27T11:08:36.612 INFO:teuthology.orchestra.run.smithi114.stdout: "put_sum": 1613, 2024-03-27T11:08:36.612 INFO:teuthology.orchestra.run.smithi114.stdout: "wait": { 2024-03-27T11:08:36.612 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:36.612 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:36.612 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:36.612 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-03-27T11:08:36.612 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-03-27T11:08:36.613 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-03-27T11:08:37.498 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-03-27T11:08:37.498 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-03-27T11:08:37.498 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-03-27T11:08:37.498 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=16 2024-03-27T11:08:37.498 DEBUG:tasks.check_counter:Getting stats from h 2024-03-27T11:08:37.499 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell --fsid adca88c4-ec26-11ee-95d0-87774f69a715 -- ceph daemon mds.h perf dump 2024-03-27T11:08:37.763 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:08:37 smithi114 ceph-mon[35338]: pgmap v800: 97 pgs: 97 active+clean; 2.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 86 op/s 2024-03-27T11:08:37.888 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:08:37 smithi097 ceph-mon[31859]: pgmap v800: 97 pgs: 97 active+clean; 2.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 86 op/s 2024-03-27T11:08:37.923 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:08:37 smithi177 ceph-mon[35294]: pgmap v800: 97 pgs: 97 active+clean; 2.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 86 op/s 2024-03-27T11:08:37.995 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/mon.b/config 2024-03-27T11:08:39.763 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:08:39 smithi114 ceph-mon[35338]: pgmap v801: 97 pgs: 97 active+clean; 2.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 89 op/s 2024-03-27T11:08:39.888 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:08:39 smithi097 ceph-mon[31859]: pgmap v801: 97 pgs: 97 active+clean; 2.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 89 op/s 2024-03-27T11:08:39.923 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:08:39 smithi177 ceph-mon[35294]: pgmap v801: 97 pgs: 97 active+clean; 2.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 89 op/s 2024-03-27T11:08:40.321 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-03-27T11:08:40.321 INFO:teuthology.orchestra.run.smithi114.stdout: "AsyncMessenger::Worker-0": { 2024-03-27T11:08:40.321 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_messages": 143, 2024-03-27T11:08:40.321 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages": 152, 2024-03-27T11:08:40.321 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_bytes": 87550, 2024-03-27T11:08:40.322 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_bytes": 275288, 2024-03-27T11:08:40.322 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_created_connections": 26, 2024-03-27T11:08:40.322 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_active_connections": 6, 2024-03-27T11:08:40.322 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_total_time": 0.041711569, 2024-03-27T11:08:40.322 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_send_time": 0.007391658, 2024-03-27T11:08:40.322 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_recv_time": 0.025492375, 2024-03-27T11:08:40.322 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_fast_dispatch_time": 0.001134747, 2024-03-27T11:08:40.322 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages_queue_lat": { 2024-03-27T11:08:40.322 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 146, 2024-03-27T11:08:40.322 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.051603810, 2024-03-27T11:08:40.322 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000353450 2024-03-27T11:08:40.322 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:40.322 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_handle_ack_lat": { 2024-03-27T11:08:40.323 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 180, 2024-03-27T11:08:40.323 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000038976, 2024-03-27T11:08:40.323 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000216 2024-03-27T11:08:40.323 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:40.323 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_encrypted_bytes": 1936, 2024-03-27T11:08:40.323 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_encrypted_bytes": 1584 2024-03-27T11:08:40.323 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:40.323 INFO:teuthology.orchestra.run.smithi114.stdout: "AsyncMessenger::Worker-1": { 2024-03-27T11:08:40.323 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_messages": 629, 2024-03-27T11:08:40.323 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages": 949, 2024-03-27T11:08:40.323 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_bytes": 275666, 2024-03-27T11:08:40.323 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_bytes": 751120, 2024-03-27T11:08:40.324 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_created_connections": 9, 2024-03-27T11:08:40.324 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_active_connections": 5, 2024-03-27T11:08:40.324 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_total_time": 0.158505148, 2024-03-27T11:08:40.324 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_send_time": 0.088699276, 2024-03-27T11:08:40.324 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_recv_time": 0.061168297, 2024-03-27T11:08:40.324 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_fast_dispatch_time": 0.005488107, 2024-03-27T11:08:40.324 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages_queue_lat": { 2024-03-27T11:08:40.324 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 951, 2024-03-27T11:08:40.324 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.130989063, 2024-03-27T11:08:40.324 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000137738 2024-03-27T11:08:40.324 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:40.324 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_handle_ack_lat": { 2024-03-27T11:08:40.324 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 1319, 2024-03-27T11:08:40.324 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000605757, 2024-03-27T11:08:40.325 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000459 2024-03-27T11:08:40.325 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:40.325 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-27T11:08:40.325 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_encrypted_bytes": 370160 2024-03-27T11:08:40.325 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:40.325 INFO:teuthology.orchestra.run.smithi114.stdout: "AsyncMessenger::Worker-2": { 2024-03-27T11:08:40.325 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_messages": 387, 2024-03-27T11:08:40.325 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages": 365, 2024-03-27T11:08:40.325 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_bytes": 672445, 2024-03-27T11:08:40.325 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_bytes": 241271, 2024-03-27T11:08:40.325 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_created_connections": 8, 2024-03-27T11:08:40.325 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_active_connections": 6, 2024-03-27T11:08:40.325 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_total_time": 0.080892495, 2024-03-27T11:08:40.326 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_send_time": 0.032758902, 2024-03-27T11:08:40.326 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_recv_time": 0.043461948, 2024-03-27T11:08:40.326 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_fast_dispatch_time": 0.008275069, 2024-03-27T11:08:40.326 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages_queue_lat": { 2024-03-27T11:08:40.326 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 365, 2024-03-27T11:08:40.326 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.064093150, 2024-03-27T11:08:40.326 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000175597 2024-03-27T11:08:40.326 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:40.327 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_handle_ack_lat": { 2024-03-27T11:08:40.327 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 108, 2024-03-27T11:08:40.327 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000023046, 2024-03-27T11:08:40.327 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000213 2024-03-27T11:08:40.327 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:40.327 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_encrypted_bytes": 596688, 2024-03-27T11:08:40.327 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_encrypted_bytes": 163184 2024-03-27T11:08:40.327 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:40.327 INFO:teuthology.orchestra.run.smithi114.stdout: "cct": { 2024-03-27T11:08:40.327 INFO:teuthology.orchestra.run.smithi114.stdout: "total_workers": 1, 2024-03-27T11:08:40.327 INFO:teuthology.orchestra.run.smithi114.stdout: "unhealthy_workers": 0 2024-03-27T11:08:40.328 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:40.328 INFO:teuthology.orchestra.run.smithi114.stdout: "finisher-MDSRank": { 2024-03-27T11:08:40.328 INFO:teuthology.orchestra.run.smithi114.stdout: "queue_len": 0, 2024-03-27T11:08:40.328 INFO:teuthology.orchestra.run.smithi114.stdout: "complete_latency": { 2024-03-27T11:08:40.328 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 79, 2024-03-27T11:08:40.328 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.008412254, 2024-03-27T11:08:40.328 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000106484 2024-03-27T11:08:40.328 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-03-27T11:08:40.328 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:40.328 INFO:teuthology.orchestra.run.smithi114.stdout: "finisher-PurgeQueue": { 2024-03-27T11:08:40.328 INFO:teuthology.orchestra.run.smithi114.stdout: "queue_len": 0, 2024-03-27T11:08:40.329 INFO:teuthology.orchestra.run.smithi114.stdout: "complete_latency": { 2024-03-27T11:08:40.329 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 17, 2024-03-27T11:08:40.329 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000536285, 2024-03-27T11:08:40.329 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000031546 2024-03-27T11:08:40.329 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-03-27T11:08:40.329 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:40.329 INFO:teuthology.orchestra.run.smithi114.stdout: "mds": { 2024-03-27T11:08:40.329 INFO:teuthology.orchestra.run.smithi114.stdout: "request": 9, 2024-03-27T11:08:40.329 INFO:teuthology.orchestra.run.smithi114.stdout: "reply": 7, 2024-03-27T11:08:40.329 INFO:teuthology.orchestra.run.smithi114.stdout: "reply_latency": { 2024-03-27T11:08:40.329 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 7, 2024-03-27T11:08:40.330 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.013632731, 2024-03-27T11:08:40.330 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.001947533 2024-03-27T11:08:40.330 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:40.330 INFO:teuthology.orchestra.run.smithi114.stdout: "slow_reply": 0, 2024-03-27T11:08:40.330 INFO:teuthology.orchestra.run.smithi114.stdout: "forward": 2, 2024-03-27T11:08:40.330 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_fetch_complete": 0, 2024-03-27T11:08:40.330 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_fetch_keys": 0, 2024-03-27T11:08:40.330 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_commit": 11, 2024-03-27T11:08:40.330 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_split": 0, 2024-03-27T11:08:40.330 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_merge": 0, 2024-03-27T11:08:40.331 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes": 17, 2024-03-27T11:08:40.331 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes_top": 1, 2024-03-27T11:08:40.331 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes_bottom": 16, 2024-03-27T11:08:40.331 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes_pin_tail": 0, 2024-03-27T11:08:40.331 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes_pinned": 13, 2024-03-27T11:08:40.331 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes_expired": 8, 2024-03-27T11:08:40.331 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes_with_caps": 2, 2024-03-27T11:08:40.331 INFO:teuthology.orchestra.run.smithi114.stdout: "caps": 2, 2024-03-27T11:08:40.331 INFO:teuthology.orchestra.run.smithi114.stdout: "subtrees": 3, 2024-03-27T11:08:40.331 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse": 26, 2024-03-27T11:08:40.332 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse_hit": 15, 2024-03-27T11:08:40.332 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse_forward": 0, 2024-03-27T11:08:40.332 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse_discover": 5, 2024-03-27T11:08:40.332 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse_dir_fetch": 0, 2024-03-27T11:08:40.332 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse_remote_ino": 0, 2024-03-27T11:08:40.332 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse_lock": 0, 2024-03-27T11:08:40.332 INFO:teuthology.orchestra.run.smithi114.stdout: "load_cent": 0, 2024-03-27T11:08:40.332 INFO:teuthology.orchestra.run.smithi114.stdout: "q": 0, 2024-03-27T11:08:40.332 INFO:teuthology.orchestra.run.smithi114.stdout: "exported": 1, 2024-03-27T11:08:40.332 INFO:teuthology.orchestra.run.smithi114.stdout: "exported_inodes": 0, 2024-03-27T11:08:40.332 INFO:teuthology.orchestra.run.smithi114.stdout: "imported": 1, 2024-03-27T11:08:40.332 INFO:teuthology.orchestra.run.smithi114.stdout: "imported_inodes": 2, 2024-03-27T11:08:40.333 INFO:teuthology.orchestra.run.smithi114.stdout: "openino_dir_fetch": 0, 2024-03-27T11:08:40.333 INFO:teuthology.orchestra.run.smithi114.stdout: "openino_backtrace_fetch": 2, 2024-03-27T11:08:40.333 INFO:teuthology.orchestra.run.smithi114.stdout: "openino_peer_discover": 0, 2024-03-27T11:08:40.333 INFO:teuthology.orchestra.run.smithi114.stdout: "root_rfiles": 22, 2024-03-27T11:08:40.333 INFO:teuthology.orchestra.run.smithi114.stdout: "root_rbytes": 1460871314, 2024-03-27T11:08:40.333 INFO:teuthology.orchestra.run.smithi114.stdout: "root_rsnaps": 0, 2024-03-27T11:08:40.333 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_backtrace_fetch": 2, 2024-03-27T11:08:40.333 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_set_tag": 0, 2024-03-27T11:08:40.333 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_backtrace_repaired": 0, 2024-03-27T11:08:40.333 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_inotable_repaired": 0, 2024-03-27T11:08:40.333 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_dir_inodes": 0, 2024-03-27T11:08:40.334 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_dir_base_inodes": 0, 2024-03-27T11:08:40.334 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_dirfrag_rstats": 0, 2024-03-27T11:08:40.334 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_file_inodes": 2, 2024-03-27T11:08:40.334 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_inode_file_caps": 0, 2024-03-27T11:08:40.334 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph_cap_op_revoke": 1, 2024-03-27T11:08:40.334 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph_cap_op_grant": 3, 2024-03-27T11:08:40.334 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph_cap_op_trunc": 0, 2024-03-27T11:08:40.334 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-27T11:08:40.334 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-27T11:08:40.334 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_client_caps": 1, 2024-03-27T11:08:40.334 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_client_caps_dirty": 0, 2024-03-27T11:08:40.335 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_client_cap_release": 2, 2024-03-27T11:08:40.335 INFO:teuthology.orchestra.run.smithi114.stdout: "process_request_cap_release": 6 2024-03-27T11:08:40.335 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:40.335 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_cache": { 2024-03-27T11:08:40.335 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_update": 3, 2024-03-27T11:08:40.335 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_update_receipt": 12, 2024-03-27T11:08:40.335 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_try_discover": 8, 2024-03-27T11:08:40.335 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_send_discover": 10, 2024-03-27T11:08:40.335 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_handle_discover": 3, 2024-03-27T11:08:40.335 INFO:teuthology.orchestra.run.smithi114.stdout: "num_strays": 0, 2024-03-27T11:08:40.335 INFO:teuthology.orchestra.run.smithi114.stdout: "num_strays_delayed": 0, 2024-03-27T11:08:40.335 INFO:teuthology.orchestra.run.smithi114.stdout: "num_strays_enqueuing": 0, 2024-03-27T11:08:40.335 INFO:teuthology.orchestra.run.smithi114.stdout: "strays_created": 2, 2024-03-27T11:08:40.336 INFO:teuthology.orchestra.run.smithi114.stdout: "strays_enqueued": 2, 2024-03-27T11:08:40.336 INFO:teuthology.orchestra.run.smithi114.stdout: "strays_reintegrated": 0, 2024-03-27T11:08:40.336 INFO:teuthology.orchestra.run.smithi114.stdout: "strays_migrated": 0, 2024-03-27T11:08:40.336 INFO:teuthology.orchestra.run.smithi114.stdout: "num_recovering_processing": 0, 2024-03-27T11:08:40.336 INFO:teuthology.orchestra.run.smithi114.stdout: "num_recovering_enqueued": 0, 2024-03-27T11:08:40.336 INFO:teuthology.orchestra.run.smithi114.stdout: "num_recovering_prioritized": 0, 2024-03-27T11:08:40.336 INFO:teuthology.orchestra.run.smithi114.stdout: "recovery_started": 0, 2024-03-27T11:08:40.336 INFO:teuthology.orchestra.run.smithi114.stdout: "recovery_completed": 0, 2024-03-27T11:08:40.336 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_quiesce_path": 0, 2024-03-27T11:08:40.336 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_quiesce_inode": 0, 2024-03-27T11:08:40.336 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_enqueue_scrub": 0, 2024-03-27T11:08:40.336 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_exportdir": 1, 2024-03-27T11:08:40.336 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_flush": 0, 2024-03-27T11:08:40.337 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_fragmentdir": 0, 2024-03-27T11:08:40.337 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_fragstats": 0, 2024-03-27T11:08:40.337 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_inodestats": 0 2024-03-27T11:08:40.337 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:40.337 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_log": { 2024-03-27T11:08:40.337 INFO:teuthology.orchestra.run.smithi114.stdout: "evlrg": 0, 2024-03-27T11:08:40.337 INFO:teuthology.orchestra.run.smithi114.stdout: "evadd": 16, 2024-03-27T11:08:40.337 INFO:teuthology.orchestra.run.smithi114.stdout: "evex": 0, 2024-03-27T11:08:40.337 INFO:teuthology.orchestra.run.smithi114.stdout: "evtrm": 0, 2024-03-27T11:08:40.337 INFO:teuthology.orchestra.run.smithi114.stdout: "ev": 16, 2024-03-27T11:08:40.337 INFO:teuthology.orchestra.run.smithi114.stdout: "evexg": 0, 2024-03-27T11:08:40.338 INFO:teuthology.orchestra.run.smithi114.stdout: "evexd": 0, 2024-03-27T11:08:40.338 INFO:teuthology.orchestra.run.smithi114.stdout: "segadd": 2, 2024-03-27T11:08:40.338 INFO:teuthology.orchestra.run.smithi114.stdout: "segex": 0, 2024-03-27T11:08:40.338 INFO:teuthology.orchestra.run.smithi114.stdout: "segtrm": 0, 2024-03-27T11:08:40.338 INFO:teuthology.orchestra.run.smithi114.stdout: "seg": 2, 2024-03-27T11:08:40.338 INFO:teuthology.orchestra.run.smithi114.stdout: "segmjr": 2, 2024-03-27T11:08:40.338 INFO:teuthology.orchestra.run.smithi114.stdout: "segexg": 0, 2024-03-27T11:08:40.338 INFO:teuthology.orchestra.run.smithi114.stdout: "segexd": 0, 2024-03-27T11:08:40.338 INFO:teuthology.orchestra.run.smithi114.stdout: "expos": 4194304, 2024-03-27T11:08:40.338 INFO:teuthology.orchestra.run.smithi114.stdout: "wrpos": 4217513, 2024-03-27T11:08:40.338 INFO:teuthology.orchestra.run.smithi114.stdout: "rdpos": 0, 2024-03-27T11:08:40.338 INFO:teuthology.orchestra.run.smithi114.stdout: "jlat": { 2024-03-27T11:08:40.339 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 12, 2024-03-27T11:08:40.339 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.402999480, 2024-03-27T11:08:40.339 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.033583290 2024-03-27T11:08:40.339 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:40.339 INFO:teuthology.orchestra.run.smithi114.stdout: "replayed": 0 2024-03-27T11:08:40.339 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:40.339 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_mem": { 2024-03-27T11:08:40.339 INFO:teuthology.orchestra.run.smithi114.stdout: "ino": 21, 2024-03-27T11:08:40.339 INFO:teuthology.orchestra.run.smithi114.stdout: "ino+": 27, 2024-03-27T11:08:40.339 INFO:teuthology.orchestra.run.smithi114.stdout: "ino-": 6, 2024-03-27T11:08:40.339 INFO:teuthology.orchestra.run.smithi114.stdout: "dir": 20, 2024-03-27T11:08:40.340 INFO:teuthology.orchestra.run.smithi114.stdout: "dir+": 25, 2024-03-27T11:08:40.340 INFO:teuthology.orchestra.run.smithi114.stdout: "dir-": 5, 2024-03-27T11:08:40.340 INFO:teuthology.orchestra.run.smithi114.stdout: "dn": 17, 2024-03-27T11:08:40.340 INFO:teuthology.orchestra.run.smithi114.stdout: "dn+": 29, 2024-03-27T11:08:40.340 INFO:teuthology.orchestra.run.smithi114.stdout: "dn-": 12, 2024-03-27T11:08:40.340 INFO:teuthology.orchestra.run.smithi114.stdout: "cap": 2, 2024-03-27T11:08:40.340 INFO:teuthology.orchestra.run.smithi114.stdout: "cap+": 6, 2024-03-27T11:08:40.340 INFO:teuthology.orchestra.run.smithi114.stdout: "cap-": 4, 2024-03-27T11:08:40.340 INFO:teuthology.orchestra.run.smithi114.stdout: "rss": 51584, 2024-03-27T11:08:40.340 INFO:teuthology.orchestra.run.smithi114.stdout: "heap": 223608 2024-03-27T11:08:40.340 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:40.341 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_server": { 2024-03-27T11:08:40.341 INFO:teuthology.orchestra.run.smithi114.stdout: "dispatch_client_request": 17, 2024-03-27T11:08:40.341 INFO:teuthology.orchestra.run.smithi114.stdout: "dispatch_server_request": 1, 2024-03-27T11:08:40.341 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_client_request": 9, 2024-03-27T11:08:40.341 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_client_session": 14, 2024-03-27T11:08:40.341 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_peer_request": 6, 2024-03-27T11:08:40.341 INFO:teuthology.orchestra.run.smithi114.stdout: "req_create_latency": { 2024-03-27T11:08:40.341 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:40.341 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:40.341 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:40.341 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:40.341 INFO:teuthology.orchestra.run.smithi114.stdout: "req_getattr_latency": { 2024-03-27T11:08:40.341 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 3, 2024-03-27T11:08:40.342 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.003400627, 2024-03-27T11:08:40.342 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.001133542 2024-03-27T11:08:40.342 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:40.342 INFO:teuthology.orchestra.run.smithi114.stdout: "req_getfilelock_latency": { 2024-03-27T11:08:40.342 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:40.342 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:40.342 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:40.342 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:40.342 INFO:teuthology.orchestra.run.smithi114.stdout: "req_link_latency": { 2024-03-27T11:08:40.342 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:40.342 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:40.343 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:40.343 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:40.343 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lookup_latency": { 2024-03-27T11:08:40.343 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 1, 2024-03-27T11:08:40.343 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000466043, 2024-03-27T11:08:40.343 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000466043 2024-03-27T11:08:40.343 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:40.343 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lookuphash_latency": { 2024-03-27T11:08:40.343 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:40.343 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:40.343 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:40.343 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:40.344 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lookupino_latency": { 2024-03-27T11:08:40.344 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:40.344 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:40.344 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:40.344 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:40.344 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lookupname_latency": { 2024-03-27T11:08:40.344 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:40.344 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:40.344 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:40.344 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:40.344 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lookupparent_latency": { 2024-03-27T11:08:40.344 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:40.344 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:40.345 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:40.345 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:40.345 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lookupsnap_latency": { 2024-03-27T11:08:40.345 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:40.345 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:40.345 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:40.345 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:40.345 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lssnap_latency": { 2024-03-27T11:08:40.345 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:40.345 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:40.345 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:40.345 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:40.345 INFO:teuthology.orchestra.run.smithi114.stdout: "req_mkdir_latency": { 2024-03-27T11:08:40.346 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:40.346 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:40.347 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:40.347 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:40.347 INFO:teuthology.orchestra.run.smithi114.stdout: "req_mknod_latency": { 2024-03-27T11:08:40.347 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:40.347 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:40.347 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:40.347 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:40.347 INFO:teuthology.orchestra.run.smithi114.stdout: "req_mksnap_latency": { 2024-03-27T11:08:40.347 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:40.347 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:40.347 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:40.347 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:40.348 INFO:teuthology.orchestra.run.smithi114.stdout: "req_open_latency": { 2024-03-27T11:08:40.348 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:40.348 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:40.348 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:40.348 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:40.348 INFO:teuthology.orchestra.run.smithi114.stdout: "req_readdir_latency": { 2024-03-27T11:08:40.348 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 1, 2024-03-27T11:08:40.348 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000895132, 2024-03-27T11:08:40.348 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000895132 2024-03-27T11:08:40.348 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:40.348 INFO:teuthology.orchestra.run.smithi114.stdout: "req_rename_latency": { 2024-03-27T11:08:40.348 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:40.348 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:40.349 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:40.349 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:40.349 INFO:teuthology.orchestra.run.smithi114.stdout: "req_renamesnap_latency": { 2024-03-27T11:08:40.349 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:40.349 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:40.349 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:40.349 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:40.349 INFO:teuthology.orchestra.run.smithi114.stdout: "req_snapdiff_latency": { 2024-03-27T11:08:40.349 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:40.349 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:40.349 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:40.349 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:40.349 INFO:teuthology.orchestra.run.smithi114.stdout: "req_rmdir_latency": { 2024-03-27T11:08:40.350 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:40.350 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:40.350 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:40.350 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:40.350 INFO:teuthology.orchestra.run.smithi114.stdout: "req_rmsnap_latency": { 2024-03-27T11:08:40.350 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:40.350 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:40.350 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:40.350 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:40.350 INFO:teuthology.orchestra.run.smithi114.stdout: "req_rmxattr_latency": { 2024-03-27T11:08:40.350 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:40.350 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:40.351 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:40.351 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:40.351 INFO:teuthology.orchestra.run.smithi114.stdout: "req_setattr_latency": { 2024-03-27T11:08:40.351 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:40.351 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:40.351 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:40.351 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:40.351 INFO:teuthology.orchestra.run.smithi114.stdout: "req_setdirlayout_latency": { 2024-03-27T11:08:40.351 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:40.351 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:40.351 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:40.351 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:40.352 INFO:teuthology.orchestra.run.smithi114.stdout: "req_setfilelock_latency": { 2024-03-27T11:08:40.352 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:40.352 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:40.352 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:40.352 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:40.352 INFO:teuthology.orchestra.run.smithi114.stdout: "req_setlayout_latency": { 2024-03-27T11:08:40.352 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:40.352 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:40.352 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:40.352 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:40.352 INFO:teuthology.orchestra.run.smithi114.stdout: "req_setxattr_latency": { 2024-03-27T11:08:40.352 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:40.352 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:40.352 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:40.353 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:40.353 INFO:teuthology.orchestra.run.smithi114.stdout: "req_symlink_latency": { 2024-03-27T11:08:40.353 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:40.353 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:40.353 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:40.353 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:40.353 INFO:teuthology.orchestra.run.smithi114.stdout: "req_unlink_latency": { 2024-03-27T11:08:40.353 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 2, 2024-03-27T11:08:40.353 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.008870929, 2024-03-27T11:08:40.353 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.004435464 2024-03-27T11:08:40.353 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:40.353 INFO:teuthology.orchestra.run.smithi114.stdout: "cap_revoke_eviction": 0, 2024-03-27T11:08:40.354 INFO:teuthology.orchestra.run.smithi114.stdout: "cap_acquisition_throttle": 0, 2024-03-27T11:08:40.354 INFO:teuthology.orchestra.run.smithi114.stdout: "req_getvxattr_latency": { 2024-03-27T11:08:40.354 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:40.354 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:40.354 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:40.354 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-03-27T11:08:40.354 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:40.354 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_sessions": { 2024-03-27T11:08:40.354 INFO:teuthology.orchestra.run.smithi114.stdout: "session_count": 1, 2024-03-27T11:08:40.354 INFO:teuthology.orchestra.run.smithi114.stdout: "session_add": 2, 2024-03-27T11:08:40.355 INFO:teuthology.orchestra.run.smithi114.stdout: "session_remove": 1, 2024-03-27T11:08:40.355 INFO:teuthology.orchestra.run.smithi114.stdout: "sessions_open": 1, 2024-03-27T11:08:40.355 INFO:teuthology.orchestra.run.smithi114.stdout: "sessions_stale": 0, 2024-03-27T11:08:40.355 INFO:teuthology.orchestra.run.smithi114.stdout: "total_load": 4, 2024-03-27T11:08:40.355 INFO:teuthology.orchestra.run.smithi114.stdout: "average_load": 4, 2024-03-27T11:08:40.355 INFO:teuthology.orchestra.run.smithi114.stdout: "avg_session_uptime": 733, 2024-03-27T11:08:40.355 INFO:teuthology.orchestra.run.smithi114.stdout: "mdthresh_evicted": 0 2024-03-27T11:08:40.355 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:40.355 INFO:teuthology.orchestra.run.smithi114.stdout: "mempool": { 2024-03-27T11:08:40.355 INFO:teuthology.orchestra.run.smithi114.stdout: "bloom_filter_bytes": 0, 2024-03-27T11:08:40.355 INFO:teuthology.orchestra.run.smithi114.stdout: "bloom_filter_items": 0, 2024-03-27T11:08:40.355 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_alloc_bytes": 0, 2024-03-27T11:08:40.356 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_alloc_items": 0, 2024-03-27T11:08:40.356 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_data_bytes": 0, 2024-03-27T11:08:40.356 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_data_items": 0, 2024-03-27T11:08:40.356 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-27T11:08:40.356 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_onode_items": 0, 2024-03-27T11:08:40.356 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-27T11:08:40.356 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_meta_items": 0, 2024-03-27T11:08:40.356 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_other_bytes": 0, 2024-03-27T11:08:40.356 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_other_items": 0, 2024-03-27T11:08:40.356 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-27T11:08:40.356 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_buffer_items": 0, 2024-03-27T11:08:40.356 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_extent_bytes": 0, 2024-03-27T11:08:40.357 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_extent_items": 0, 2024-03-27T11:08:40.357 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_blob_bytes": 0, 2024-03-27T11:08:40.357 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_blob_items": 0, 2024-03-27T11:08:40.357 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-27T11:08:40.357 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_shared_blob_items": 0, 2024-03-27T11:08:40.357 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-27T11:08:40.357 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_inline_bl_items": 0, 2024-03-27T11:08:40.357 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_fsck_bytes": 0, 2024-03-27T11:08:40.357 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_fsck_items": 0, 2024-03-27T11:08:40.357 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_txc_bytes": 0, 2024-03-27T11:08:40.357 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_txc_items": 0, 2024-03-27T11:08:40.357 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-27T11:08:40.358 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_writing_deferred_items": 0, 2024-03-27T11:08:40.358 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_writing_bytes": 0, 2024-03-27T11:08:40.358 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_writing_items": 0, 2024-03-27T11:08:40.358 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_bytes": 0, 2024-03-27T11:08:40.358 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_items": 0, 2024-03-27T11:08:40.358 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_file_reader_bytes": 0, 2024-03-27T11:08:40.358 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_file_reader_items": 0, 2024-03-27T11:08:40.358 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_file_writer_bytes": 0, 2024-03-27T11:08:40.358 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_file_writer_items": 0, 2024-03-27T11:08:40.358 INFO:teuthology.orchestra.run.smithi114.stdout: "buffer_anon_bytes": 267131, 2024-03-27T11:08:40.358 INFO:teuthology.orchestra.run.smithi114.stdout: "buffer_anon_items": 90, 2024-03-27T11:08:40.359 INFO:teuthology.orchestra.run.smithi114.stdout: "buffer_meta_bytes": 0, 2024-03-27T11:08:40.359 INFO:teuthology.orchestra.run.smithi114.stdout: "buffer_meta_items": 0, 2024-03-27T11:08:40.359 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_bytes": 0, 2024-03-27T11:08:40.359 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_items": 0, 2024-03-27T11:08:40.359 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_mapbl_bytes": 0, 2024-03-27T11:08:40.359 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_mapbl_items": 0, 2024-03-27T11:08:40.359 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_pglog_bytes": 0, 2024-03-27T11:08:40.359 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_pglog_items": 0, 2024-03-27T11:08:40.359 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_bytes": 7528, 2024-03-27T11:08:40.359 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_items": 158, 2024-03-27T11:08:40.359 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_mapping_bytes": 0, 2024-03-27T11:08:40.359 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_mapping_items": 0, 2024-03-27T11:08:40.359 INFO:teuthology.orchestra.run.smithi114.stdout: "pgmap_bytes": 0, 2024-03-27T11:08:40.359 INFO:teuthology.orchestra.run.smithi114.stdout: "pgmap_items": 0, 2024-03-27T11:08:40.360 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_co_bytes": 85653, 2024-03-27T11:08:40.360 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_co_items": 356, 2024-03-27T11:08:40.360 INFO:teuthology.orchestra.run.smithi114.stdout: "unittest_1_bytes": 0, 2024-03-27T11:08:40.360 INFO:teuthology.orchestra.run.smithi114.stdout: "unittest_1_items": 0, 2024-03-27T11:08:40.360 INFO:teuthology.orchestra.run.smithi114.stdout: "unittest_2_bytes": 0, 2024-03-27T11:08:40.360 INFO:teuthology.orchestra.run.smithi114.stdout: "unittest_2_items": 0 2024-03-27T11:08:40.360 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:40.360 INFO:teuthology.orchestra.run.smithi114.stdout: "objecter": { 2024-03-27T11:08:40.360 INFO:teuthology.orchestra.run.smithi114.stdout: "op_active": 0, 2024-03-27T11:08:40.360 INFO:teuthology.orchestra.run.smithi114.stdout: "op_laggy": 0, 2024-03-27T11:08:40.360 INFO:teuthology.orchestra.run.smithi114.stdout: "op_send": 68, 2024-03-27T11:08:40.360 INFO:teuthology.orchestra.run.smithi114.stdout: "op_send_bytes": 35168, 2024-03-27T11:08:40.360 INFO:teuthology.orchestra.run.smithi114.stdout: "op_resend": 32, 2024-03-27T11:08:40.361 INFO:teuthology.orchestra.run.smithi114.stdout: "op_reply": 68, 2024-03-27T11:08:40.361 INFO:teuthology.orchestra.run.smithi114.stdout: "op_latency": { 2024-03-27T11:08:40.361 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 68, 2024-03-27T11:08:40.361 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.971979012, 2024-03-27T11:08:40.361 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.014293809 2024-03-27T11:08:40.361 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:40.361 INFO:teuthology.orchestra.run.smithi114.stdout: "op_inflight": 0, 2024-03-27T11:08:40.361 INFO:teuthology.orchestra.run.smithi114.stdout: "oplen_avg": { 2024-03-27T11:08:40.361 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 68, 2024-03-27T11:08:40.361 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 95 2024-03-27T11:08:40.361 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:40.361 INFO:teuthology.orchestra.run.smithi114.stdout: "op": 68, 2024-03-27T11:08:40.362 INFO:teuthology.orchestra.run.smithi114.stdout: "op_r": 3, 2024-03-27T11:08:40.362 INFO:teuthology.orchestra.run.smithi114.stdout: "op_w": 65, 2024-03-27T11:08:40.362 INFO:teuthology.orchestra.run.smithi114.stdout: "op_rmw": 0, 2024-03-27T11:08:40.362 INFO:teuthology.orchestra.run.smithi114.stdout: "op_pg": 0, 2024-03-27T11:08:40.362 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_stat": 0, 2024-03-27T11:08:40.362 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_create": 10, 2024-03-27T11:08:40.362 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_read": 1, 2024-03-27T11:08:40.362 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_write": 13, 2024-03-27T11:08:40.362 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_writefull": 11, 2024-03-27T11:08:40.362 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_writesame": 0, 2024-03-27T11:08:40.362 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_append": 0, 2024-03-27T11:08:40.362 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_zero": 0, 2024-03-27T11:08:40.363 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_truncate": 0, 2024-03-27T11:08:40.363 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_delete": 14, 2024-03-27T11:08:40.363 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_mapext": 0, 2024-03-27T11:08:40.363 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_sparse_read": 0, 2024-03-27T11:08:40.363 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_clonerange": 0, 2024-03-27T11:08:40.363 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_getxattr": 2, 2024-03-27T11:08:40.363 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_setxattr": 20, 2024-03-27T11:08:40.363 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_cmpxattr": 0, 2024-03-27T11:08:40.363 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_rmxattr": 0, 2024-03-27T11:08:40.363 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_resetxattrs": 0, 2024-03-27T11:08:40.363 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_call": 0, 2024-03-27T11:08:40.363 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_watch": 0, 2024-03-27T11:08:40.364 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_notify": 0, 2024-03-27T11:08:40.364 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_src_cmpxattr": 0, 2024-03-27T11:08:40.364 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_pgls": 0, 2024-03-27T11:08:40.364 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_pgls_filter": 0, 2024-03-27T11:08:40.364 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_other": 0, 2024-03-27T11:08:40.364 INFO:teuthology.orchestra.run.smithi114.stdout: "linger_active": 0, 2024-03-27T11:08:40.364 INFO:teuthology.orchestra.run.smithi114.stdout: "linger_send": 0, 2024-03-27T11:08:40.364 INFO:teuthology.orchestra.run.smithi114.stdout: "linger_resend": 0, 2024-03-27T11:08:40.364 INFO:teuthology.orchestra.run.smithi114.stdout: "linger_ping": 0, 2024-03-27T11:08:40.364 INFO:teuthology.orchestra.run.smithi114.stdout: "poolop_active": 0, 2024-03-27T11:08:40.364 INFO:teuthology.orchestra.run.smithi114.stdout: "poolop_send": 0, 2024-03-27T11:08:40.364 INFO:teuthology.orchestra.run.smithi114.stdout: "poolop_resend": 0, 2024-03-27T11:08:40.365 INFO:teuthology.orchestra.run.smithi114.stdout: "poolstat_active": 0, 2024-03-27T11:08:40.365 INFO:teuthology.orchestra.run.smithi114.stdout: "poolstat_send": 0, 2024-03-27T11:08:40.365 INFO:teuthology.orchestra.run.smithi114.stdout: "poolstat_resend": 0, 2024-03-27T11:08:40.365 INFO:teuthology.orchestra.run.smithi114.stdout: "statfs_active": 0, 2024-03-27T11:08:40.365 INFO:teuthology.orchestra.run.smithi114.stdout: "statfs_send": 0, 2024-03-27T11:08:40.365 INFO:teuthology.orchestra.run.smithi114.stdout: "statfs_resend": 0, 2024-03-27T11:08:40.365 INFO:teuthology.orchestra.run.smithi114.stdout: "command_active": 0, 2024-03-27T11:08:40.365 INFO:teuthology.orchestra.run.smithi114.stdout: "command_send": 0, 2024-03-27T11:08:40.365 INFO:teuthology.orchestra.run.smithi114.stdout: "command_resend": 0, 2024-03-27T11:08:40.365 INFO:teuthology.orchestra.run.smithi114.stdout: "map_epoch": 92, 2024-03-27T11:08:40.365 INFO:teuthology.orchestra.run.smithi114.stdout: "map_full": 0, 2024-03-27T11:08:40.365 INFO:teuthology.orchestra.run.smithi114.stdout: "map_inc": 2, 2024-03-27T11:08:40.366 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_sessions": 10, 2024-03-27T11:08:40.366 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_session_open": 10, 2024-03-27T11:08:40.366 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_session_close": 0, 2024-03-27T11:08:40.366 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_laggy": 0, 2024-03-27T11:08:40.366 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_wr": 21, 2024-03-27T11:08:40.366 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_rd": 0, 2024-03-27T11:08:40.366 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_del": 3 2024-03-27T11:08:40.366 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:40.366 INFO:teuthology.orchestra.run.smithi114.stdout: "oft": { 2024-03-27T11:08:40.366 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_total_objs": 1, 2024-03-27T11:08:40.366 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_total_kv_pairs": 0, 2024-03-27T11:08:40.366 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_total_updates": 0, 2024-03-27T11:08:40.366 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_total_removes": 0 2024-03-27T11:08:40.367 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:40.367 INFO:teuthology.orchestra.run.smithi114.stdout: "osd-slow-ops": { 2024-03-27T11:08:40.367 INFO:teuthology.orchestra.run.smithi114.stdout: "slow_ops_count": 0 2024-03-27T11:08:40.367 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:40.367 INFO:teuthology.orchestra.run.smithi114.stdout: "purge_queue": { 2024-03-27T11:08:40.367 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_executing_ops": 0, 2024-03-27T11:08:40.367 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_executing_ops_high_water": 2, 2024-03-27T11:08:40.367 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_executing": 0, 2024-03-27T11:08:40.367 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_executing_high_water": 2, 2024-03-27T11:08:40.367 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_executed_ops": 2, 2024-03-27T11:08:40.367 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_executed": 2, 2024-03-27T11:08:40.367 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_item_in_journal": 0 2024-03-27T11:08:40.367 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:40.367 INFO:teuthology.orchestra.run.smithi114.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-27T11:08:40.368 INFO:teuthology.orchestra.run.smithi114.stdout: "val": 0, 2024-03-27T11:08:40.368 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 104857600, 2024-03-27T11:08:40.368 INFO:teuthology.orchestra.run.smithi114.stdout: "get_started": 0, 2024-03-27T11:08:40.368 INFO:teuthology.orchestra.run.smithi114.stdout: "get": 1155, 2024-03-27T11:08:40.368 INFO:teuthology.orchestra.run.smithi114.stdout: "get_sum": 913358, 2024-03-27T11:08:40.368 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_fail": 0, 2024-03-27T11:08:40.368 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_success": 1155, 2024-03-27T11:08:40.368 INFO:teuthology.orchestra.run.smithi114.stdout: "take": 0, 2024-03-27T11:08:40.368 INFO:teuthology.orchestra.run.smithi114.stdout: "take_sum": 0, 2024-03-27T11:08:40.368 INFO:teuthology.orchestra.run.smithi114.stdout: "put": 1155, 2024-03-27T11:08:40.368 INFO:teuthology.orchestra.run.smithi114.stdout: "put_sum": 913358, 2024-03-27T11:08:40.369 INFO:teuthology.orchestra.run.smithi114.stdout: "wait": { 2024-03-27T11:08:40.369 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:40.369 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:40.369 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:40.369 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-03-27T11:08:40.369 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:40.369 INFO:teuthology.orchestra.run.smithi114.stdout: "throttle-objecter_bytes": { 2024-03-27T11:08:40.369 INFO:teuthology.orchestra.run.smithi114.stdout: "val": 0, 2024-03-27T11:08:40.369 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 104857600, 2024-03-27T11:08:40.369 INFO:teuthology.orchestra.run.smithi114.stdout: "get_started": 0, 2024-03-27T11:08:40.369 INFO:teuthology.orchestra.run.smithi114.stdout: "get": 0, 2024-03-27T11:08:40.369 INFO:teuthology.orchestra.run.smithi114.stdout: "get_sum": 0, 2024-03-27T11:08:40.369 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_fail": 0, 2024-03-27T11:08:40.369 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_success": 0, 2024-03-27T11:08:40.370 INFO:teuthology.orchestra.run.smithi114.stdout: "take": 68, 2024-03-27T11:08:40.370 INFO:teuthology.orchestra.run.smithi114.stdout: "take_sum": 35370, 2024-03-27T11:08:40.370 INFO:teuthology.orchestra.run.smithi114.stdout: "put": 68, 2024-03-27T11:08:40.370 INFO:teuthology.orchestra.run.smithi114.stdout: "put_sum": 35370, 2024-03-27T11:08:40.370 INFO:teuthology.orchestra.run.smithi114.stdout: "wait": { 2024-03-27T11:08:40.370 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:40.370 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:40.370 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:40.370 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-03-27T11:08:40.370 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:40.370 INFO:teuthology.orchestra.run.smithi114.stdout: "throttle-objecter_ops": { 2024-03-27T11:08:40.370 INFO:teuthology.orchestra.run.smithi114.stdout: "val": 0, 2024-03-27T11:08:40.371 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 1024, 2024-03-27T11:08:40.371 INFO:teuthology.orchestra.run.smithi114.stdout: "get_started": 0, 2024-03-27T11:08:40.371 INFO:teuthology.orchestra.run.smithi114.stdout: "get": 0, 2024-03-27T11:08:40.371 INFO:teuthology.orchestra.run.smithi114.stdout: "get_sum": 0, 2024-03-27T11:08:40.371 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_fail": 0, 2024-03-27T11:08:40.371 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_success": 0, 2024-03-27T11:08:40.371 INFO:teuthology.orchestra.run.smithi114.stdout: "take": 68, 2024-03-27T11:08:40.371 INFO:teuthology.orchestra.run.smithi114.stdout: "take_sum": 68, 2024-03-27T11:08:40.371 INFO:teuthology.orchestra.run.smithi114.stdout: "put": 68, 2024-03-27T11:08:40.371 INFO:teuthology.orchestra.run.smithi114.stdout: "put_sum": 68, 2024-03-27T11:08:40.371 INFO:teuthology.orchestra.run.smithi114.stdout: "wait": { 2024-03-27T11:08:40.371 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:40.371 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:40.372 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:40.372 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-03-27T11:08:40.372 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:40.372 INFO:teuthology.orchestra.run.smithi114.stdout: "throttle-write_buf_throttle": { 2024-03-27T11:08:40.372 INFO:teuthology.orchestra.run.smithi114.stdout: "val": 0, 2024-03-27T11:08:40.372 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 3758096384, 2024-03-27T11:08:40.372 INFO:teuthology.orchestra.run.smithi114.stdout: "get_started": 0, 2024-03-27T11:08:40.372 INFO:teuthology.orchestra.run.smithi114.stdout: "get": 2, 2024-03-27T11:08:40.372 INFO:teuthology.orchestra.run.smithi114.stdout: "get_sum": 202, 2024-03-27T11:08:40.372 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_fail": 0, 2024-03-27T11:08:40.372 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_success": 2, 2024-03-27T11:08:40.372 INFO:teuthology.orchestra.run.smithi114.stdout: "take": 0, 2024-03-27T11:08:40.372 INFO:teuthology.orchestra.run.smithi114.stdout: "take_sum": 0, 2024-03-27T11:08:40.373 INFO:teuthology.orchestra.run.smithi114.stdout: "put": 1, 2024-03-27T11:08:40.373 INFO:teuthology.orchestra.run.smithi114.stdout: "put_sum": 202, 2024-03-27T11:08:40.373 INFO:teuthology.orchestra.run.smithi114.stdout: "wait": { 2024-03-27T11:08:40.373 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:40.373 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:40.373 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:40.373 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-03-27T11:08:40.373 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:40.373 INFO:teuthology.orchestra.run.smithi114.stdout: "throttle-write_buf_throttle-0x563b8a0e2500": { 2024-03-27T11:08:40.373 INFO:teuthology.orchestra.run.smithi114.stdout: "val": 0, 2024-03-27T11:08:40.373 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 3758096384, 2024-03-27T11:08:40.373 INFO:teuthology.orchestra.run.smithi114.stdout: "get_started": 0, 2024-03-27T11:08:40.374 INFO:teuthology.orchestra.run.smithi114.stdout: "get": 16, 2024-03-27T11:08:40.374 INFO:teuthology.orchestra.run.smithi114.stdout: "get_sum": 23209, 2024-03-27T11:08:40.374 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_fail": 0, 2024-03-27T11:08:40.374 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_success": 16, 2024-03-27T11:08:40.374 INFO:teuthology.orchestra.run.smithi114.stdout: "take": 0, 2024-03-27T11:08:40.374 INFO:teuthology.orchestra.run.smithi114.stdout: "take_sum": 0, 2024-03-27T11:08:40.374 INFO:teuthology.orchestra.run.smithi114.stdout: "put": 12, 2024-03-27T11:08:40.374 INFO:teuthology.orchestra.run.smithi114.stdout: "put_sum": 23209, 2024-03-27T11:08:40.374 INFO:teuthology.orchestra.run.smithi114.stdout: "wait": { 2024-03-27T11:08:40.374 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:40.374 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:40.375 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:40.375 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-03-27T11:08:40.375 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-03-27T11:08:40.375 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-03-27T11:08:41.428 INFO:tasks.check_counter:Daemon mds.h mds.exported=1 2024-03-27T11:08:41.428 INFO:tasks.check_counter:Daemon mds.h mds.imported=1 2024-03-27T11:08:41.428 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=3 2024-03-27T11:08:41.429 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=12 2024-03-27T11:08:41.429 DEBUG:tasks.check_counter:Getting stats from k 2024-03-27T11:08:41.429 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell --fsid adca88c4-ec26-11ee-95d0-87774f69a715 -- ceph daemon mds.k perf dump 2024-03-27T11:08:41.888 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:08:41 smithi097 ceph-mon[31859]: pgmap v802: 97 pgs: 97 active+clean; 1.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 85 B/s wr, 99 op/s 2024-03-27T11:08:41.912 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/mon.b/config 2024-03-27T11:08:41.923 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:08:41 smithi177 ceph-mon[35294]: pgmap v802: 97 pgs: 97 active+clean; 1.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 85 B/s wr, 99 op/s 2024-03-27T11:08:42.013 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:08:41 smithi114 ceph-mon[35338]: pgmap v802: 97 pgs: 97 active+clean; 1.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 85 B/s wr, 99 op/s 2024-03-27T11:08:43.888 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:08:43 smithi097 ceph-mon[31859]: pgmap v803: 97 pgs: 97 active+clean; 654 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 87 op/s 2024-03-27T11:08:43.923 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:08:43 smithi177 ceph-mon[35294]: pgmap v803: 97 pgs: 97 active+clean; 654 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 87 op/s 2024-03-27T11:08:43.931 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:08:43 smithi114 ceph-mon[35338]: pgmap v803: 97 pgs: 97 active+clean; 654 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 87 op/s 2024-03-27T11:08:44.156 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-03-27T11:08:44.157 INFO:teuthology.orchestra.run.smithi114.stdout: "AsyncMessenger::Worker-0": { 2024-03-27T11:08:44.157 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_messages": 1851, 2024-03-27T11:08:44.157 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages": 1851, 2024-03-27T11:08:44.157 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_bytes": 1652528, 2024-03-27T11:08:44.157 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_bytes": 587804, 2024-03-27T11:08:44.157 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_created_connections": 6, 2024-03-27T11:08:44.157 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_active_connections": 2, 2024-03-27T11:08:44.157 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_total_time": 0.273429958, 2024-03-27T11:08:44.157 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_send_time": 0.115198735, 2024-03-27T11:08:44.157 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_recv_time": 0.063161048, 2024-03-27T11:08:44.157 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_fast_dispatch_time": 0.070670441, 2024-03-27T11:08:44.157 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages_queue_lat": { 2024-03-27T11:08:44.158 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 1850, 2024-03-27T11:08:44.158 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.115741941, 2024-03-27T11:08:44.158 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000062563 2024-03-27T11:08:44.158 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:44.158 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_handle_ack_lat": { 2024-03-27T11:08:44.158 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:44.158 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:44.158 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:44.158 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:44.158 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-03-27T11:08:44.158 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_encrypted_bytes": 448 2024-03-27T11:08:44.158 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:44.159 INFO:teuthology.orchestra.run.smithi114.stdout: "AsyncMessenger::Worker-1": { 2024-03-27T11:08:44.159 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_messages": 6, 2024-03-27T11:08:44.159 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages": 193, 2024-03-27T11:08:44.159 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_bytes": 1976, 2024-03-27T11:08:44.159 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_bytes": 427930, 2024-03-27T11:08:44.159 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_created_connections": 6, 2024-03-27T11:08:44.159 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_active_connections": 4, 2024-03-27T11:08:44.159 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_total_time": 0.019796473, 2024-03-27T11:08:44.159 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_send_time": 0.012184057, 2024-03-27T11:08:44.159 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_recv_time": 0.004746189, 2024-03-27T11:08:44.159 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_fast_dispatch_time": 0.000117590, 2024-03-27T11:08:44.159 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages_queue_lat": { 2024-03-27T11:08:44.159 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 191, 2024-03-27T11:08:44.159 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.022826986, 2024-03-27T11:08:44.160 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000119513 2024-03-27T11:08:44.160 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:44.160 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_handle_ack_lat": { 2024-03-27T11:08:44.160 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:44.160 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:44.160 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:44.160 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:44.160 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-27T11:08:44.160 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_encrypted_bytes": 426592 2024-03-27T11:08:44.160 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:44.160 INFO:teuthology.orchestra.run.smithi114.stdout: "AsyncMessenger::Worker-2": { 2024-03-27T11:08:44.160 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_messages": 1125, 2024-03-27T11:08:44.160 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages": 1109, 2024-03-27T11:08:44.161 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_bytes": 795019, 2024-03-27T11:08:44.161 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_bytes": 424997, 2024-03-27T11:08:44.161 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_created_connections": 3, 2024-03-27T11:08:44.161 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_active_connections": 2, 2024-03-27T11:08:44.161 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_total_time": 0.168182563, 2024-03-27T11:08:44.161 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_send_time": 0.059640457, 2024-03-27T11:08:44.161 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_recv_time": 0.055003311, 2024-03-27T11:08:44.161 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_fast_dispatch_time": 0.046924805, 2024-03-27T11:08:44.161 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages_queue_lat": { 2024-03-27T11:08:44.161 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 1108, 2024-03-27T11:08:44.161 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.088427729, 2024-03-27T11:08:44.161 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000079808 2024-03-27T11:08:44.161 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:44.161 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_handle_ack_lat": { 2024-03-27T11:08:44.162 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:44.162 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:44.162 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:44.162 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:44.162 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_encrypted_bytes": 584864, 2024-03-27T11:08:44.162 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_encrypted_bytes": 153424 2024-03-27T11:08:44.162 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:44.162 INFO:teuthology.orchestra.run.smithi114.stdout: "cct": { 2024-03-27T11:08:44.162 INFO:teuthology.orchestra.run.smithi114.stdout: "total_workers": 1, 2024-03-27T11:08:44.162 INFO:teuthology.orchestra.run.smithi114.stdout: "unhealthy_workers": 0 2024-03-27T11:08:44.162 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:44.162 INFO:teuthology.orchestra.run.smithi114.stdout: "finisher-MDSRank": { 2024-03-27T11:08:44.162 INFO:teuthology.orchestra.run.smithi114.stdout: "queue_len": 0, 2024-03-27T11:08:44.163 INFO:teuthology.orchestra.run.smithi114.stdout: "complete_latency": { 2024-03-27T11:08:44.163 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 4444, 2024-03-27T11:08:44.163 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.161587556, 2024-03-27T11:08:44.163 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000036360 2024-03-27T11:08:44.163 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-03-27T11:08:44.163 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:44.163 INFO:teuthology.orchestra.run.smithi114.stdout: "finisher-PurgeQueue": { 2024-03-27T11:08:44.163 INFO:teuthology.orchestra.run.smithi114.stdout: "queue_len": 0, 2024-03-27T11:08:44.163 INFO:teuthology.orchestra.run.smithi114.stdout: "complete_latency": { 2024-03-27T11:08:44.163 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:44.163 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:44.163 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:44.164 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-03-27T11:08:44.164 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:44.164 INFO:teuthology.orchestra.run.smithi114.stdout: "mds": { 2024-03-27T11:08:44.164 INFO:teuthology.orchestra.run.smithi114.stdout: "request": 0, 2024-03-27T11:08:44.164 INFO:teuthology.orchestra.run.smithi114.stdout: "reply": 0, 2024-03-27T11:08:44.164 INFO:teuthology.orchestra.run.smithi114.stdout: "reply_latency": { 2024-03-27T11:08:44.164 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:44.164 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:44.164 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:44.164 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:44.164 INFO:teuthology.orchestra.run.smithi114.stdout: "slow_reply": 0, 2024-03-27T11:08:44.164 INFO:teuthology.orchestra.run.smithi114.stdout: "forward": 0, 2024-03-27T11:08:44.164 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_fetch_complete": 0, 2024-03-27T11:08:44.164 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_fetch_keys": 0, 2024-03-27T11:08:44.165 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_commit": 0, 2024-03-27T11:08:44.165 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_split": 0, 2024-03-27T11:08:44.165 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_merge": 0, 2024-03-27T11:08:44.165 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes": 35, 2024-03-27T11:08:44.165 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes_top": 4, 2024-03-27T11:08:44.165 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes_bottom": 31, 2024-03-27T11:08:44.165 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes_pin_tail": 0, 2024-03-27T11:08:44.165 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes_pinned": 28, 2024-03-27T11:08:44.165 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes_expired": 0, 2024-03-27T11:08:44.165 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes_with_caps": 0, 2024-03-27T11:08:44.165 INFO:teuthology.orchestra.run.smithi114.stdout: "caps": 0, 2024-03-27T11:08:44.165 INFO:teuthology.orchestra.run.smithi114.stdout: "subtrees": 2, 2024-03-27T11:08:44.165 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse": 0, 2024-03-27T11:08:44.166 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse_hit": 0, 2024-03-27T11:08:44.166 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse_forward": 0, 2024-03-27T11:08:44.166 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse_discover": 0, 2024-03-27T11:08:44.166 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse_dir_fetch": 0, 2024-03-27T11:08:44.166 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse_remote_ino": 0, 2024-03-27T11:08:44.166 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse_lock": 0, 2024-03-27T11:08:44.166 INFO:teuthology.orchestra.run.smithi114.stdout: "load_cent": 0, 2024-03-27T11:08:44.166 INFO:teuthology.orchestra.run.smithi114.stdout: "q": 0, 2024-03-27T11:08:44.166 INFO:teuthology.orchestra.run.smithi114.stdout: "exported": 0, 2024-03-27T11:08:44.166 INFO:teuthology.orchestra.run.smithi114.stdout: "exported_inodes": 0, 2024-03-27T11:08:44.166 INFO:teuthology.orchestra.run.smithi114.stdout: "imported": 0, 2024-03-27T11:08:44.166 INFO:teuthology.orchestra.run.smithi114.stdout: "imported_inodes": 0, 2024-03-27T11:08:44.167 INFO:teuthology.orchestra.run.smithi114.stdout: "openino_dir_fetch": 0, 2024-03-27T11:08:44.167 INFO:teuthology.orchestra.run.smithi114.stdout: "openino_backtrace_fetch": 0, 2024-03-27T11:08:44.167 INFO:teuthology.orchestra.run.smithi114.stdout: "openino_peer_discover": 0, 2024-03-27T11:08:44.167 INFO:teuthology.orchestra.run.smithi114.stdout: "root_rfiles": 4, 2024-03-27T11:08:44.167 INFO:teuthology.orchestra.run.smithi114.stdout: "root_rbytes": 234, 2024-03-27T11:08:44.167 INFO:teuthology.orchestra.run.smithi114.stdout: "root_rsnaps": 0, 2024-03-27T11:08:44.167 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_backtrace_fetch": 0, 2024-03-27T11:08:44.167 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_set_tag": 0, 2024-03-27T11:08:44.167 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_backtrace_repaired": 0, 2024-03-27T11:08:44.167 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_inotable_repaired": 0, 2024-03-27T11:08:44.167 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_dir_inodes": 0, 2024-03-27T11:08:44.167 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_dir_base_inodes": 0, 2024-03-27T11:08:44.167 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_dirfrag_rstats": 0, 2024-03-27T11:08:44.167 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_file_inodes": 0, 2024-03-27T11:08:44.168 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_inode_file_caps": 0, 2024-03-27T11:08:44.168 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph_cap_op_revoke": 0, 2024-03-27T11:08:44.168 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph_cap_op_grant": 0, 2024-03-27T11:08:44.168 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph_cap_op_trunc": 0, 2024-03-27T11:08:44.168 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-27T11:08:44.168 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-27T11:08:44.168 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_client_caps": 0, 2024-03-27T11:08:44.168 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_client_caps_dirty": 0, 2024-03-27T11:08:44.168 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_client_cap_release": 0, 2024-03-27T11:08:44.168 INFO:teuthology.orchestra.run.smithi114.stdout: "process_request_cap_release": 0 2024-03-27T11:08:44.168 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:44.168 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_cache": { 2024-03-27T11:08:44.168 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_update": 0, 2024-03-27T11:08:44.168 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_update_receipt": 0, 2024-03-27T11:08:44.169 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_try_discover": 0, 2024-03-27T11:08:44.169 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_send_discover": 0, 2024-03-27T11:08:44.169 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_handle_discover": 0, 2024-03-27T11:08:44.169 INFO:teuthology.orchestra.run.smithi114.stdout: "num_strays": 0, 2024-03-27T11:08:44.169 INFO:teuthology.orchestra.run.smithi114.stdout: "num_strays_delayed": 0, 2024-03-27T11:08:44.169 INFO:teuthology.orchestra.run.smithi114.stdout: "num_strays_enqueuing": 0, 2024-03-27T11:08:44.169 INFO:teuthology.orchestra.run.smithi114.stdout: "strays_created": 0, 2024-03-27T11:08:44.169 INFO:teuthology.orchestra.run.smithi114.stdout: "strays_enqueued": 0, 2024-03-27T11:08:44.169 INFO:teuthology.orchestra.run.smithi114.stdout: "strays_reintegrated": 0, 2024-03-27T11:08:44.169 INFO:teuthology.orchestra.run.smithi114.stdout: "strays_migrated": 0, 2024-03-27T11:08:44.169 INFO:teuthology.orchestra.run.smithi114.stdout: "num_recovering_processing": 0, 2024-03-27T11:08:44.169 INFO:teuthology.orchestra.run.smithi114.stdout: "num_recovering_enqueued": 0, 2024-03-27T11:08:44.169 INFO:teuthology.orchestra.run.smithi114.stdout: "num_recovering_prioritized": 0, 2024-03-27T11:08:44.169 INFO:teuthology.orchestra.run.smithi114.stdout: "recovery_started": 0, 2024-03-27T11:08:44.170 INFO:teuthology.orchestra.run.smithi114.stdout: "recovery_completed": 0, 2024-03-27T11:08:44.170 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_quiesce_path": 0, 2024-03-27T11:08:44.170 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_quiesce_inode": 0, 2024-03-27T11:08:44.170 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_enqueue_scrub": 0, 2024-03-27T11:08:44.170 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_exportdir": 0, 2024-03-27T11:08:44.170 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_flush": 0, 2024-03-27T11:08:44.170 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_fragmentdir": 0, 2024-03-27T11:08:44.170 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_fragstats": 0, 2024-03-27T11:08:44.170 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_inodestats": 0 2024-03-27T11:08:44.170 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:44.170 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_log": { 2024-03-27T11:08:44.170 INFO:teuthology.orchestra.run.smithi114.stdout: "evlrg": 0, 2024-03-27T11:08:44.170 INFO:teuthology.orchestra.run.smithi114.stdout: "evadd": 0, 2024-03-27T11:08:44.170 INFO:teuthology.orchestra.run.smithi114.stdout: "evex": 0, 2024-03-27T11:08:44.171 INFO:teuthology.orchestra.run.smithi114.stdout: "evtrm": 0, 2024-03-27T11:08:44.171 INFO:teuthology.orchestra.run.smithi114.stdout: "ev": 327, 2024-03-27T11:08:44.171 INFO:teuthology.orchestra.run.smithi114.stdout: "evexg": 0, 2024-03-27T11:08:44.171 INFO:teuthology.orchestra.run.smithi114.stdout: "evexd": 0, 2024-03-27T11:08:44.171 INFO:teuthology.orchestra.run.smithi114.stdout: "segadd": 0, 2024-03-27T11:08:44.171 INFO:teuthology.orchestra.run.smithi114.stdout: "segex": 0, 2024-03-27T11:08:44.171 INFO:teuthology.orchestra.run.smithi114.stdout: "segtrm": 0, 2024-03-27T11:08:44.171 INFO:teuthology.orchestra.run.smithi114.stdout: "seg": 2, 2024-03-27T11:08:44.171 INFO:teuthology.orchestra.run.smithi114.stdout: "segmjr": 54, 2024-03-27T11:08:44.171 INFO:teuthology.orchestra.run.smithi114.stdout: "segexg": 0, 2024-03-27T11:08:44.171 INFO:teuthology.orchestra.run.smithi114.stdout: "segexd": 0, 2024-03-27T11:08:44.171 INFO:teuthology.orchestra.run.smithi114.stdout: "expos": 5252281, 2024-03-27T11:08:44.171 INFO:teuthology.orchestra.run.smithi114.stdout: "wrpos": 5296319, 2024-03-27T11:08:44.171 INFO:teuthology.orchestra.run.smithi114.stdout: "rdpos": 5294668, 2024-03-27T11:08:44.172 INFO:teuthology.orchestra.run.smithi114.stdout: "jlat": { 2024-03-27T11:08:44.172 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:44.172 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:44.172 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:44.172 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:44.172 INFO:teuthology.orchestra.run.smithi114.stdout: "replayed": 327 2024-03-27T11:08:44.172 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:44.172 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_mem": { 2024-03-27T11:08:44.172 INFO:teuthology.orchestra.run.smithi114.stdout: "ino": 15, 2024-03-27T11:08:44.172 INFO:teuthology.orchestra.run.smithi114.stdout: "ino+": 37, 2024-03-27T11:08:44.172 INFO:teuthology.orchestra.run.smithi114.stdout: "ino-": 22, 2024-03-27T11:08:44.173 INFO:teuthology.orchestra.run.smithi114.stdout: "dir": 10, 2024-03-27T11:08:44.173 INFO:teuthology.orchestra.run.smithi114.stdout: "dir+": 14, 2024-03-27T11:08:44.173 INFO:teuthology.orchestra.run.smithi114.stdout: "dir-": 4, 2024-03-27T11:08:44.173 INFO:teuthology.orchestra.run.smithi114.stdout: "dn": 35, 2024-03-27T11:08:44.173 INFO:teuthology.orchestra.run.smithi114.stdout: "dn+": 56, 2024-03-27T11:08:44.173 INFO:teuthology.orchestra.run.smithi114.stdout: "dn-": 21, 2024-03-27T11:08:44.173 INFO:teuthology.orchestra.run.smithi114.stdout: "cap": 0, 2024-03-27T11:08:44.173 INFO:teuthology.orchestra.run.smithi114.stdout: "cap+": 0, 2024-03-27T11:08:44.173 INFO:teuthology.orchestra.run.smithi114.stdout: "cap-": 0, 2024-03-27T11:08:44.173 INFO:teuthology.orchestra.run.smithi114.stdout: "rss": 49660, 2024-03-27T11:08:44.173 INFO:teuthology.orchestra.run.smithi114.stdout: "heap": 215416 2024-03-27T11:08:44.173 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:44.173 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_server": { 2024-03-27T11:08:44.174 INFO:teuthology.orchestra.run.smithi114.stdout: "dispatch_client_request": 0, 2024-03-27T11:08:44.174 INFO:teuthology.orchestra.run.smithi114.stdout: "dispatch_server_request": 0, 2024-03-27T11:08:44.174 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_client_request": 0, 2024-03-27T11:08:44.174 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_client_session": 0, 2024-03-27T11:08:44.174 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_peer_request": 0, 2024-03-27T11:08:44.174 INFO:teuthology.orchestra.run.smithi114.stdout: "req_create_latency": { 2024-03-27T11:08:44.174 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:44.174 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:44.174 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:44.174 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:44.174 INFO:teuthology.orchestra.run.smithi114.stdout: "req_getattr_latency": { 2024-03-27T11:08:44.174 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:44.174 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:44.175 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:44.175 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:44.175 INFO:teuthology.orchestra.run.smithi114.stdout: "req_getfilelock_latency": { 2024-03-27T11:08:44.175 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:44.175 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:44.175 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:44.175 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:44.175 INFO:teuthology.orchestra.run.smithi114.stdout: "req_link_latency": { 2024-03-27T11:08:44.175 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:44.175 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:44.175 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:44.175 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:44.175 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lookup_latency": { 2024-03-27T11:08:44.175 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:44.176 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:44.176 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:44.176 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:44.176 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lookuphash_latency": { 2024-03-27T11:08:44.176 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:44.176 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:44.176 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:44.176 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:44.176 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lookupino_latency": { 2024-03-27T11:08:44.176 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:44.176 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:44.176 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:44.176 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:44.177 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lookupname_latency": { 2024-03-27T11:08:44.177 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:44.177 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:44.177 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:44.177 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:44.177 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lookupparent_latency": { 2024-03-27T11:08:44.177 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:44.177 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:44.177 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:44.177 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:44.177 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lookupsnap_latency": { 2024-03-27T11:08:44.177 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:44.177 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:44.177 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:44.177 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:44.178 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lssnap_latency": { 2024-03-27T11:08:44.178 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:44.178 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:44.178 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:44.178 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:44.178 INFO:teuthology.orchestra.run.smithi114.stdout: "req_mkdir_latency": { 2024-03-27T11:08:44.179 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:44.179 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:44.179 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:44.179 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:44.179 INFO:teuthology.orchestra.run.smithi114.stdout: "req_mknod_latency": { 2024-03-27T11:08:44.179 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:44.179 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:44.179 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:44.179 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:44.179 INFO:teuthology.orchestra.run.smithi114.stdout: "req_mksnap_latency": { 2024-03-27T11:08:44.180 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:44.180 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:44.180 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:44.180 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:44.180 INFO:teuthology.orchestra.run.smithi114.stdout: "req_open_latency": { 2024-03-27T11:08:44.180 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:44.180 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:44.180 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:44.180 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:44.180 INFO:teuthology.orchestra.run.smithi114.stdout: "req_readdir_latency": { 2024-03-27T11:08:44.180 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:44.180 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:44.180 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:44.180 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:44.181 INFO:teuthology.orchestra.run.smithi114.stdout: "req_rename_latency": { 2024-03-27T11:08:44.181 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:44.181 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:44.181 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:44.181 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:44.181 INFO:teuthology.orchestra.run.smithi114.stdout: "req_renamesnap_latency": { 2024-03-27T11:08:44.181 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:44.181 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:44.181 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:44.181 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:44.181 INFO:teuthology.orchestra.run.smithi114.stdout: "req_snapdiff_latency": { 2024-03-27T11:08:44.181 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:44.181 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:44.182 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:44.182 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:44.182 INFO:teuthology.orchestra.run.smithi114.stdout: "req_rmdir_latency": { 2024-03-27T11:08:44.182 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:44.182 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:44.182 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:44.182 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:44.182 INFO:teuthology.orchestra.run.smithi114.stdout: "req_rmsnap_latency": { 2024-03-27T11:08:44.182 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:44.182 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:44.182 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:44.182 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:44.183 INFO:teuthology.orchestra.run.smithi114.stdout: "req_rmxattr_latency": { 2024-03-27T11:08:44.183 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:44.183 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:44.183 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:44.183 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:44.183 INFO:teuthology.orchestra.run.smithi114.stdout: "req_setattr_latency": { 2024-03-27T11:08:44.183 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:44.183 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:44.183 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:44.183 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:44.183 INFO:teuthology.orchestra.run.smithi114.stdout: "req_setdirlayout_latency": { 2024-03-27T11:08:44.183 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:44.183 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:44.183 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:44.184 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:44.184 INFO:teuthology.orchestra.run.smithi114.stdout: "req_setfilelock_latency": { 2024-03-27T11:08:44.184 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:44.184 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:44.184 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:44.184 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:44.184 INFO:teuthology.orchestra.run.smithi114.stdout: "req_setlayout_latency": { 2024-03-27T11:08:44.184 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:44.184 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:44.184 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:44.184 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:44.184 INFO:teuthology.orchestra.run.smithi114.stdout: "req_setxattr_latency": { 2024-03-27T11:08:44.184 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:44.185 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:44.185 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:44.185 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:44.185 INFO:teuthology.orchestra.run.smithi114.stdout: "req_symlink_latency": { 2024-03-27T11:08:44.185 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:44.185 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:44.185 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:44.185 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:44.185 INFO:teuthology.orchestra.run.smithi114.stdout: "req_unlink_latency": { 2024-03-27T11:08:44.185 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:44.185 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:44.185 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:44.185 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:44.185 INFO:teuthology.orchestra.run.smithi114.stdout: "cap_revoke_eviction": 0, 2024-03-27T11:08:44.186 INFO:teuthology.orchestra.run.smithi114.stdout: "cap_acquisition_throttle": 0, 2024-03-27T11:08:44.186 INFO:teuthology.orchestra.run.smithi114.stdout: "req_getvxattr_latency": { 2024-03-27T11:08:44.186 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:44.186 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:44.186 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:44.186 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-03-27T11:08:44.186 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:44.186 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_sessions": { 2024-03-27T11:08:44.186 INFO:teuthology.orchestra.run.smithi114.stdout: "session_count": 1, 2024-03-27T11:08:44.186 INFO:teuthology.orchestra.run.smithi114.stdout: "session_add": 2, 2024-03-27T11:08:44.186 INFO:teuthology.orchestra.run.smithi114.stdout: "session_remove": 1, 2024-03-27T11:08:44.186 INFO:teuthology.orchestra.run.smithi114.stdout: "sessions_open": 1, 2024-03-27T11:08:44.186 INFO:teuthology.orchestra.run.smithi114.stdout: "sessions_stale": 0, 2024-03-27T11:08:44.187 INFO:teuthology.orchestra.run.smithi114.stdout: "total_load": 0, 2024-03-27T11:08:44.187 INFO:teuthology.orchestra.run.smithi114.stdout: "average_load": 0, 2024-03-27T11:08:44.187 INFO:teuthology.orchestra.run.smithi114.stdout: "avg_session_uptime": 2178, 2024-03-27T11:08:44.187 INFO:teuthology.orchestra.run.smithi114.stdout: "mdthresh_evicted": 0 2024-03-27T11:08:44.187 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:44.187 INFO:teuthology.orchestra.run.smithi114.stdout: "mempool": { 2024-03-27T11:08:44.187 INFO:teuthology.orchestra.run.smithi114.stdout: "bloom_filter_bytes": 0, 2024-03-27T11:08:44.187 INFO:teuthology.orchestra.run.smithi114.stdout: "bloom_filter_items": 0, 2024-03-27T11:08:44.187 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_alloc_bytes": 0, 2024-03-27T11:08:44.187 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_alloc_items": 0, 2024-03-27T11:08:44.187 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_data_bytes": 0, 2024-03-27T11:08:44.187 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_data_items": 0, 2024-03-27T11:08:44.187 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-27T11:08:44.188 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_onode_items": 0, 2024-03-27T11:08:44.188 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-27T11:08:44.188 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_meta_items": 0, 2024-03-27T11:08:44.188 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_other_bytes": 0, 2024-03-27T11:08:44.188 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_other_items": 0, 2024-03-27T11:08:44.188 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-27T11:08:44.188 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_buffer_items": 0, 2024-03-27T11:08:44.188 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_extent_bytes": 0, 2024-03-27T11:08:44.188 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_extent_items": 0, 2024-03-27T11:08:44.188 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_blob_bytes": 0, 2024-03-27T11:08:44.188 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_blob_items": 0, 2024-03-27T11:08:44.188 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-27T11:08:44.188 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_shared_blob_items": 0, 2024-03-27T11:08:44.188 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-27T11:08:44.189 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_inline_bl_items": 0, 2024-03-27T11:08:44.189 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_fsck_bytes": 0, 2024-03-27T11:08:44.189 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_fsck_items": 0, 2024-03-27T11:08:44.189 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_txc_bytes": 0, 2024-03-27T11:08:44.189 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_txc_items": 0, 2024-03-27T11:08:44.189 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-27T11:08:44.189 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_writing_deferred_items": 0, 2024-03-27T11:08:44.189 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_writing_bytes": 0, 2024-03-27T11:08:44.189 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_writing_items": 0, 2024-03-27T11:08:44.189 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_bytes": 0, 2024-03-27T11:08:44.189 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_items": 0, 2024-03-27T11:08:44.189 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_file_reader_bytes": 0, 2024-03-27T11:08:44.189 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_file_reader_items": 0, 2024-03-27T11:08:44.189 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_file_writer_bytes": 0, 2024-03-27T11:08:44.190 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_file_writer_items": 0, 2024-03-27T11:08:44.190 INFO:teuthology.orchestra.run.smithi114.stdout: "buffer_anon_bytes": 189367, 2024-03-27T11:08:44.190 INFO:teuthology.orchestra.run.smithi114.stdout: "buffer_anon_items": 49, 2024-03-27T11:08:44.190 INFO:teuthology.orchestra.run.smithi114.stdout: "buffer_meta_bytes": 0, 2024-03-27T11:08:44.190 INFO:teuthology.orchestra.run.smithi114.stdout: "buffer_meta_items": 0, 2024-03-27T11:08:44.190 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_bytes": 0, 2024-03-27T11:08:44.190 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_items": 0, 2024-03-27T11:08:44.190 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_mapbl_bytes": 0, 2024-03-27T11:08:44.190 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_mapbl_items": 0, 2024-03-27T11:08:44.190 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_pglog_bytes": 0, 2024-03-27T11:08:44.190 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_pglog_items": 0, 2024-03-27T11:08:44.190 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_bytes": 7528, 2024-03-27T11:08:44.190 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_items": 158, 2024-03-27T11:08:44.191 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_mapping_bytes": 0, 2024-03-27T11:08:44.191 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_mapping_items": 0, 2024-03-27T11:08:44.191 INFO:teuthology.orchestra.run.smithi114.stdout: "pgmap_bytes": 0, 2024-03-27T11:08:44.191 INFO:teuthology.orchestra.run.smithi114.stdout: "pgmap_items": 0, 2024-03-27T11:08:44.191 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_co_bytes": 70754, 2024-03-27T11:08:44.191 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_co_items": 340, 2024-03-27T11:08:44.191 INFO:teuthology.orchestra.run.smithi114.stdout: "unittest_1_bytes": 0, 2024-03-27T11:08:44.191 INFO:teuthology.orchestra.run.smithi114.stdout: "unittest_1_items": 0, 2024-03-27T11:08:44.191 INFO:teuthology.orchestra.run.smithi114.stdout: "unittest_2_bytes": 0, 2024-03-27T11:08:44.191 INFO:teuthology.orchestra.run.smithi114.stdout: "unittest_2_items": 0 2024-03-27T11:08:44.191 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:44.191 INFO:teuthology.orchestra.run.smithi114.stdout: "objecter": { 2024-03-27T11:08:44.191 INFO:teuthology.orchestra.run.smithi114.stdout: "op_active": 0, 2024-03-27T11:08:44.191 INFO:teuthology.orchestra.run.smithi114.stdout: "op_laggy": 0, 2024-03-27T11:08:44.192 INFO:teuthology.orchestra.run.smithi114.stdout: "op_send": 2705, 2024-03-27T11:08:44.192 INFO:teuthology.orchestra.run.smithi114.stdout: "op_send_bytes": 26, 2024-03-27T11:08:44.192 INFO:teuthology.orchestra.run.smithi114.stdout: "op_resend": 4, 2024-03-27T11:08:44.192 INFO:teuthology.orchestra.run.smithi114.stdout: "op_reply": 2705, 2024-03-27T11:08:44.192 INFO:teuthology.orchestra.run.smithi114.stdout: "op_latency": { 2024-03-27T11:08:44.192 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 2705, 2024-03-27T11:08:44.192 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 2.193952509, 2024-03-27T11:08:44.192 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000811073 2024-03-27T11:08:44.192 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:44.192 INFO:teuthology.orchestra.run.smithi114.stdout: "op_inflight": 0, 2024-03-27T11:08:44.192 INFO:teuthology.orchestra.run.smithi114.stdout: "oplen_avg": { 2024-03-27T11:08:44.192 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 2705, 2024-03-27T11:08:44.192 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 2706 2024-03-27T11:08:44.192 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:44.193 INFO:teuthology.orchestra.run.smithi114.stdout: "op": 2705, 2024-03-27T11:08:44.193 INFO:teuthology.orchestra.run.smithi114.stdout: "op_r": 2705, 2024-03-27T11:08:44.193 INFO:teuthology.orchestra.run.smithi114.stdout: "op_w": 0, 2024-03-27T11:08:44.193 INFO:teuthology.orchestra.run.smithi114.stdout: "op_rmw": 0, 2024-03-27T11:08:44.193 INFO:teuthology.orchestra.run.smithi114.stdout: "op_pg": 0, 2024-03-27T11:08:44.193 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_stat": 1726, 2024-03-27T11:08:44.193 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_create": 0, 2024-03-27T11:08:44.193 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_read": 976, 2024-03-27T11:08:44.193 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_write": 0, 2024-03-27T11:08:44.194 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_writefull": 0, 2024-03-27T11:08:44.194 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_writesame": 0, 2024-03-27T11:08:44.194 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_append": 0, 2024-03-27T11:08:44.194 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_zero": 0, 2024-03-27T11:08:44.194 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_truncate": 0, 2024-03-27T11:08:44.194 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_delete": 0, 2024-03-27T11:08:44.194 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_mapext": 0, 2024-03-27T11:08:44.194 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_sparse_read": 0, 2024-03-27T11:08:44.194 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_clonerange": 0, 2024-03-27T11:08:44.194 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_getxattr": 2, 2024-03-27T11:08:44.194 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_setxattr": 0, 2024-03-27T11:08:44.194 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_cmpxattr": 0, 2024-03-27T11:08:44.194 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_rmxattr": 0, 2024-03-27T11:08:44.194 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_resetxattrs": 0, 2024-03-27T11:08:44.195 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_call": 0, 2024-03-27T11:08:44.195 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_watch": 0, 2024-03-27T11:08:44.195 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_notify": 0, 2024-03-27T11:08:44.195 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_src_cmpxattr": 0, 2024-03-27T11:08:44.195 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_pgls": 0, 2024-03-27T11:08:44.195 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_pgls_filter": 0, 2024-03-27T11:08:44.195 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_other": 0, 2024-03-27T11:08:44.195 INFO:teuthology.orchestra.run.smithi114.stdout: "linger_active": 0, 2024-03-27T11:08:44.195 INFO:teuthology.orchestra.run.smithi114.stdout: "linger_send": 0, 2024-03-27T11:08:44.195 INFO:teuthology.orchestra.run.smithi114.stdout: "linger_resend": 0, 2024-03-27T11:08:44.195 INFO:teuthology.orchestra.run.smithi114.stdout: "linger_ping": 0, 2024-03-27T11:08:44.195 INFO:teuthology.orchestra.run.smithi114.stdout: "poolop_active": 0, 2024-03-27T11:08:44.195 INFO:teuthology.orchestra.run.smithi114.stdout: "poolop_send": 0, 2024-03-27T11:08:44.196 INFO:teuthology.orchestra.run.smithi114.stdout: "poolop_resend": 0, 2024-03-27T11:08:44.196 INFO:teuthology.orchestra.run.smithi114.stdout: "poolstat_active": 0, 2024-03-27T11:08:44.196 INFO:teuthology.orchestra.run.smithi114.stdout: "poolstat_send": 0, 2024-03-27T11:08:44.196 INFO:teuthology.orchestra.run.smithi114.stdout: "poolstat_resend": 0, 2024-03-27T11:08:44.196 INFO:teuthology.orchestra.run.smithi114.stdout: "statfs_active": 0, 2024-03-27T11:08:44.196 INFO:teuthology.orchestra.run.smithi114.stdout: "statfs_send": 0, 2024-03-27T11:08:44.196 INFO:teuthology.orchestra.run.smithi114.stdout: "statfs_resend": 0, 2024-03-27T11:08:44.196 INFO:teuthology.orchestra.run.smithi114.stdout: "command_active": 0, 2024-03-27T11:08:44.196 INFO:teuthology.orchestra.run.smithi114.stdout: "command_send": 0, 2024-03-27T11:08:44.196 INFO:teuthology.orchestra.run.smithi114.stdout: "command_resend": 0, 2024-03-27T11:08:44.196 INFO:teuthology.orchestra.run.smithi114.stdout: "map_epoch": 92, 2024-03-27T11:08:44.196 INFO:teuthology.orchestra.run.smithi114.stdout: "map_full": 0, 2024-03-27T11:08:44.196 INFO:teuthology.orchestra.run.smithi114.stdout: "map_inc": 2, 2024-03-27T11:08:44.196 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_sessions": 5, 2024-03-27T11:08:44.196 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_session_open": 5, 2024-03-27T11:08:44.197 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_session_close": 0, 2024-03-27T11:08:44.197 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_laggy": 0, 2024-03-27T11:08:44.197 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_wr": 0, 2024-03-27T11:08:44.197 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_rd": 2, 2024-03-27T11:08:44.197 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_del": 0 2024-03-27T11:08:44.197 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:44.197 INFO:teuthology.orchestra.run.smithi114.stdout: "oft": { 2024-03-27T11:08:44.197 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_total_objs": 0, 2024-03-27T11:08:44.197 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_total_kv_pairs": 0, 2024-03-27T11:08:44.197 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_total_updates": 0, 2024-03-27T11:08:44.197 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_total_removes": 0 2024-03-27T11:08:44.197 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:44.197 INFO:teuthology.orchestra.run.smithi114.stdout: "osd-slow-ops": { 2024-03-27T11:08:44.197 INFO:teuthology.orchestra.run.smithi114.stdout: "slow_ops_count": 0 2024-03-27T11:08:44.198 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:44.198 INFO:teuthology.orchestra.run.smithi114.stdout: "purge_queue": { 2024-03-27T11:08:44.198 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_executing_ops": 0, 2024-03-27T11:08:44.198 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_executing_ops_high_water": 0, 2024-03-27T11:08:44.198 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_executing": 0, 2024-03-27T11:08:44.198 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_executing_high_water": 0, 2024-03-27T11:08:44.198 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_executed_ops": 0, 2024-03-27T11:08:44.198 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_executed": 0, 2024-03-27T11:08:44.198 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_item_in_journal": 0 2024-03-27T11:08:44.198 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:44.198 INFO:teuthology.orchestra.run.smithi114.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-27T11:08:44.198 INFO:teuthology.orchestra.run.smithi114.stdout: "val": 0, 2024-03-27T11:08:44.198 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 104857600, 2024-03-27T11:08:44.199 INFO:teuthology.orchestra.run.smithi114.stdout: "get_started": 0, 2024-03-27T11:08:44.199 INFO:teuthology.orchestra.run.smithi114.stdout: "get": 2978, 2024-03-27T11:08:44.199 INFO:teuthology.orchestra.run.smithi114.stdout: "get_sum": 2164566, 2024-03-27T11:08:44.199 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_fail": 0, 2024-03-27T11:08:44.199 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_success": 2978, 2024-03-27T11:08:44.199 INFO:teuthology.orchestra.run.smithi114.stdout: "take": 0, 2024-03-27T11:08:44.199 INFO:teuthology.orchestra.run.smithi114.stdout: "take_sum": 0, 2024-03-27T11:08:44.199 INFO:teuthology.orchestra.run.smithi114.stdout: "put": 2978, 2024-03-27T11:08:44.199 INFO:teuthology.orchestra.run.smithi114.stdout: "put_sum": 2164566, 2024-03-27T11:08:44.199 INFO:teuthology.orchestra.run.smithi114.stdout: "wait": { 2024-03-27T11:08:44.199 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:44.199 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:44.199 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:44.199 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-03-27T11:08:44.200 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:44.200 INFO:teuthology.orchestra.run.smithi114.stdout: "throttle-objecter_bytes": { 2024-03-27T11:08:44.200 INFO:teuthology.orchestra.run.smithi114.stdout: "val": 0, 2024-03-27T11:08:44.200 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 104857600, 2024-03-27T11:08:44.200 INFO:teuthology.orchestra.run.smithi114.stdout: "get_started": 0, 2024-03-27T11:08:44.200 INFO:teuthology.orchestra.run.smithi114.stdout: "get": 0, 2024-03-27T11:08:44.200 INFO:teuthology.orchestra.run.smithi114.stdout: "get_sum": 0, 2024-03-27T11:08:44.200 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_fail": 0, 2024-03-27T11:08:44.200 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_success": 0, 2024-03-27T11:08:44.200 INFO:teuthology.orchestra.run.smithi114.stdout: "take": 2705, 2024-03-27T11:08:44.200 INFO:teuthology.orchestra.run.smithi114.stdout: "take_sum": 1102025, 2024-03-27T11:08:44.200 INFO:teuthology.orchestra.run.smithi114.stdout: "put": 2705, 2024-03-27T11:08:44.200 INFO:teuthology.orchestra.run.smithi114.stdout: "put_sum": 1102025, 2024-03-27T11:08:44.200 INFO:teuthology.orchestra.run.smithi114.stdout: "wait": { 2024-03-27T11:08:44.201 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:44.201 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:44.201 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:44.201 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-03-27T11:08:44.201 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:44.201 INFO:teuthology.orchestra.run.smithi114.stdout: "throttle-objecter_ops": { 2024-03-27T11:08:44.201 INFO:teuthology.orchestra.run.smithi114.stdout: "val": 0, 2024-03-27T11:08:44.201 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 1024, 2024-03-27T11:08:44.201 INFO:teuthology.orchestra.run.smithi114.stdout: "get_started": 0, 2024-03-27T11:08:44.201 INFO:teuthology.orchestra.run.smithi114.stdout: "get": 0, 2024-03-27T11:08:44.201 INFO:teuthology.orchestra.run.smithi114.stdout: "get_sum": 0, 2024-03-27T11:08:44.201 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_fail": 0, 2024-03-27T11:08:44.201 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_success": 0, 2024-03-27T11:08:44.201 INFO:teuthology.orchestra.run.smithi114.stdout: "take": 2705, 2024-03-27T11:08:44.202 INFO:teuthology.orchestra.run.smithi114.stdout: "take_sum": 2705, 2024-03-27T11:08:44.202 INFO:teuthology.orchestra.run.smithi114.stdout: "put": 2705, 2024-03-27T11:08:44.202 INFO:teuthology.orchestra.run.smithi114.stdout: "put_sum": 2705, 2024-03-27T11:08:44.202 INFO:teuthology.orchestra.run.smithi114.stdout: "wait": { 2024-03-27T11:08:44.202 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:44.202 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:44.202 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:44.202 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-03-27T11:08:44.202 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:44.202 INFO:teuthology.orchestra.run.smithi114.stdout: "throttle-write_buf_throttle": { 2024-03-27T11:08:44.202 INFO:teuthology.orchestra.run.smithi114.stdout: "val": 0, 2024-03-27T11:08:44.202 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 3758096384, 2024-03-27T11:08:44.202 INFO:teuthology.orchestra.run.smithi114.stdout: "get_started": 0, 2024-03-27T11:08:44.203 INFO:teuthology.orchestra.run.smithi114.stdout: "get": 0, 2024-03-27T11:08:44.203 INFO:teuthology.orchestra.run.smithi114.stdout: "get_sum": 0, 2024-03-27T11:08:44.203 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_fail": 0, 2024-03-27T11:08:44.203 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_success": 0, 2024-03-27T11:08:44.203 INFO:teuthology.orchestra.run.smithi114.stdout: "take": 0, 2024-03-27T11:08:44.203 INFO:teuthology.orchestra.run.smithi114.stdout: "take_sum": 0, 2024-03-27T11:08:44.203 INFO:teuthology.orchestra.run.smithi114.stdout: "put": 0, 2024-03-27T11:08:44.203 INFO:teuthology.orchestra.run.smithi114.stdout: "put_sum": 0, 2024-03-27T11:08:44.203 INFO:teuthology.orchestra.run.smithi114.stdout: "wait": { 2024-03-27T11:08:44.203 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:44.203 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:44.203 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:44.203 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-03-27T11:08:44.203 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-03-27T11:08:44.204 INFO:teuthology.orchestra.run.smithi114.stdout: "throttle-write_buf_throttle-0x55c339e98780": { 2024-03-27T11:08:44.204 INFO:teuthology.orchestra.run.smithi114.stdout: "val": 0, 2024-03-27T11:08:44.204 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 3758096384, 2024-03-27T11:08:44.204 INFO:teuthology.orchestra.run.smithi114.stdout: "get_started": 0, 2024-03-27T11:08:44.204 INFO:teuthology.orchestra.run.smithi114.stdout: "get": 0, 2024-03-27T11:08:44.204 INFO:teuthology.orchestra.run.smithi114.stdout: "get_sum": 0, 2024-03-27T11:08:44.204 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_fail": 0, 2024-03-27T11:08:44.204 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_success": 0, 2024-03-27T11:08:44.204 INFO:teuthology.orchestra.run.smithi114.stdout: "take": 0, 2024-03-27T11:08:44.204 INFO:teuthology.orchestra.run.smithi114.stdout: "take_sum": 0, 2024-03-27T11:08:44.204 INFO:teuthology.orchestra.run.smithi114.stdout: "put": 0, 2024-03-27T11:08:44.204 INFO:teuthology.orchestra.run.smithi114.stdout: "put_sum": 0, 2024-03-27T11:08:44.204 INFO:teuthology.orchestra.run.smithi114.stdout: "wait": { 2024-03-27T11:08:44.204 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-03-27T11:08:44.205 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-03-27T11:08:44.205 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-03-27T11:08:44.205 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-03-27T11:08:44.205 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-03-27T11:08:44.205 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-03-27T11:08:45.025 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-03-27T11:08:45.025 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-03-27T11:08:45.025 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update=0 2024-03-27T11:08:45.025 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update_receipt=0 2024-03-27T11:08:45.026 DEBUG:tasks.check_counter:Getting stats from c 2024-03-27T11:08:45.026 DEBUG:teuthology.orchestra.run.smithi177:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell --fsid adca88c4-ec26-11ee-95d0-87774f69a715 -- ceph daemon mds.c perf dump 2024-03-27T11:08:45.574 INFO:teuthology.orchestra.run.smithi177.stderr:Inferring config /var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/mon.c/config 2024-03-27T11:08:45.851 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:08:45 smithi177 ceph-mon[35294]: pgmap v804: 97 pgs: 97 active+clean; 274 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 85 B/s wr, 86 op/s 2024-03-27T11:08:45.887 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:08:45 smithi097 ceph-mon[31859]: pgmap v804: 97 pgs: 97 active+clean; 274 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 85 B/s wr, 86 op/s 2024-03-27T11:08:46.014 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:08:45 smithi114 ceph-mon[35338]: pgmap v804: 97 pgs: 97 active+clean; 274 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 85 B/s wr, 86 op/s 2024-03-27T11:08:47.644 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-03-27T11:08:47.645 INFO:teuthology.orchestra.run.smithi177.stdout: "AsyncMessenger::Worker-0": { 2024-03-27T11:08:47.645 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_messages": 883, 2024-03-27T11:08:47.645 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages": 885, 2024-03-27T11:08:47.645 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_bytes": 233534, 2024-03-27T11:08:47.645 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_bytes": 280858, 2024-03-27T11:08:47.645 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_created_connections": 6, 2024-03-27T11:08:47.645 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_active_connections": 2, 2024-03-27T11:08:47.645 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_total_time": 0.116941067, 2024-03-27T11:08:47.645 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_send_time": 0.038285805, 2024-03-27T11:08:47.645 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_recv_time": 0.029170337, 2024-03-27T11:08:47.645 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_fast_dispatch_time": 0.033961860, 2024-03-27T11:08:47.646 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages_queue_lat": { 2024-03-27T11:08:47.646 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 883, 2024-03-27T11:08:47.646 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.059240689, 2024-03-27T11:08:47.646 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000067090 2024-03-27T11:08:47.646 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:47.646 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_handle_ack_lat": { 2024-03-27T11:08:47.646 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:47.646 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:47.646 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:47.646 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:47.646 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-27T11:08:47.646 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_encrypted_bytes": 0 2024-03-27T11:08:47.646 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:47.647 INFO:teuthology.orchestra.run.smithi177.stdout: "AsyncMessenger::Worker-1": { 2024-03-27T11:08:47.647 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_messages": 789, 2024-03-27T11:08:47.647 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages": 973, 2024-03-27T11:08:47.647 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_bytes": 194030, 2024-03-27T11:08:47.647 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_bytes": 622653, 2024-03-27T11:08:47.647 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_created_connections": 5, 2024-03-27T11:08:47.647 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_active_connections": 4, 2024-03-27T11:08:47.647 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_total_time": 0.118462799, 2024-03-27T11:08:47.647 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_send_time": 0.044135728, 2024-03-27T11:08:47.647 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_recv_time": 0.026580466, 2024-03-27T11:08:47.647 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_fast_dispatch_time": 0.035354330, 2024-03-27T11:08:47.647 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages_queue_lat": { 2024-03-27T11:08:47.647 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 971, 2024-03-27T11:08:47.647 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.061521790, 2024-03-27T11:08:47.648 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000063359 2024-03-27T11:08:47.648 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:47.648 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_handle_ack_lat": { 2024-03-27T11:08:47.648 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:47.648 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:47.648 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:47.648 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:47.648 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-27T11:08:47.648 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_encrypted_bytes": 372064 2024-03-27T11:08:47.648 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:47.648 INFO:teuthology.orchestra.run.smithi177.stdout: "AsyncMessenger::Worker-2": { 2024-03-27T11:08:47.648 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_messages": 1146, 2024-03-27T11:08:47.648 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages": 1128, 2024-03-27T11:08:47.649 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_bytes": 880944, 2024-03-27T11:08:47.649 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_bytes": 429720, 2024-03-27T11:08:47.649 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_created_connections": 3, 2024-03-27T11:08:47.649 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_active_connections": 2, 2024-03-27T11:08:47.649 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_total_time": 0.192962263, 2024-03-27T11:08:47.649 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_send_time": 0.070479776, 2024-03-27T11:08:47.649 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_recv_time": 0.066202780, 2024-03-27T11:08:47.649 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_fast_dispatch_time": 0.046880505, 2024-03-27T11:08:47.649 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages_queue_lat": { 2024-03-27T11:08:47.649 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 1128, 2024-03-27T11:08:47.649 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.093896382, 2024-03-27T11:08:47.649 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000083241 2024-03-27T11:08:47.649 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:47.650 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_handle_ack_lat": { 2024-03-27T11:08:47.650 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:47.650 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:47.650 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:47.650 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:47.650 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_encrypted_bytes": 586608, 2024-03-27T11:08:47.650 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_encrypted_bytes": 151152 2024-03-27T11:08:47.650 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:47.650 INFO:teuthology.orchestra.run.smithi177.stdout: "cct": { 2024-03-27T11:08:47.650 INFO:teuthology.orchestra.run.smithi177.stdout: "total_workers": 1, 2024-03-27T11:08:47.650 INFO:teuthology.orchestra.run.smithi177.stdout: "unhealthy_workers": 0 2024-03-27T11:08:47.650 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:47.650 INFO:teuthology.orchestra.run.smithi177.stdout: "finisher-MDSRank": { 2024-03-27T11:08:47.651 INFO:teuthology.orchestra.run.smithi177.stdout: "queue_len": 0, 2024-03-27T11:08:47.651 INFO:teuthology.orchestra.run.smithi177.stdout: "complete_latency": { 2024-03-27T11:08:47.651 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 4230, 2024-03-27T11:08:47.651 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.161224069, 2024-03-27T11:08:47.651 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000038114 2024-03-27T11:08:47.651 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-03-27T11:08:47.651 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:47.651 INFO:teuthology.orchestra.run.smithi177.stdout: "finisher-PurgeQueue": { 2024-03-27T11:08:47.651 INFO:teuthology.orchestra.run.smithi177.stdout: "queue_len": 0, 2024-03-27T11:08:47.651 INFO:teuthology.orchestra.run.smithi177.stdout: "complete_latency": { 2024-03-27T11:08:47.651 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:47.651 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:47.651 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:47.652 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-03-27T11:08:47.652 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:47.652 INFO:teuthology.orchestra.run.smithi177.stdout: "mds": { 2024-03-27T11:08:47.652 INFO:teuthology.orchestra.run.smithi177.stdout: "request": 0, 2024-03-27T11:08:47.652 INFO:teuthology.orchestra.run.smithi177.stdout: "reply": 0, 2024-03-27T11:08:47.652 INFO:teuthology.orchestra.run.smithi177.stdout: "reply_latency": { 2024-03-27T11:08:47.652 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:47.652 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:47.652 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:47.652 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:47.652 INFO:teuthology.orchestra.run.smithi177.stdout: "slow_reply": 0, 2024-03-27T11:08:47.652 INFO:teuthology.orchestra.run.smithi177.stdout: "forward": 0, 2024-03-27T11:08:47.652 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_fetch_complete": 0, 2024-03-27T11:08:47.652 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_fetch_keys": 0, 2024-03-27T11:08:47.653 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_commit": 0, 2024-03-27T11:08:47.653 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_split": 0, 2024-03-27T11:08:47.653 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_merge": 0, 2024-03-27T11:08:47.653 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes": 0, 2024-03-27T11:08:47.653 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes_top": 0, 2024-03-27T11:08:47.653 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes_bottom": 0, 2024-03-27T11:08:47.653 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes_pin_tail": 0, 2024-03-27T11:08:47.653 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes_pinned": 0, 2024-03-27T11:08:47.653 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes_expired": 0, 2024-03-27T11:08:47.653 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes_with_caps": 0, 2024-03-27T11:08:47.653 INFO:teuthology.orchestra.run.smithi177.stdout: "caps": 0, 2024-03-27T11:08:47.653 INFO:teuthology.orchestra.run.smithi177.stdout: "subtrees": 1, 2024-03-27T11:08:47.653 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse": 0, 2024-03-27T11:08:47.654 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse_hit": 0, 2024-03-27T11:08:47.654 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse_forward": 0, 2024-03-27T11:08:47.654 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse_discover": 0, 2024-03-27T11:08:47.654 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse_dir_fetch": 0, 2024-03-27T11:08:47.654 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse_remote_ino": 0, 2024-03-27T11:08:47.654 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse_lock": 0, 2024-03-27T11:08:47.654 INFO:teuthology.orchestra.run.smithi177.stdout: "load_cent": 0, 2024-03-27T11:08:47.654 INFO:teuthology.orchestra.run.smithi177.stdout: "q": 0, 2024-03-27T11:08:47.654 INFO:teuthology.orchestra.run.smithi177.stdout: "exported": 0, 2024-03-27T11:08:47.654 INFO:teuthology.orchestra.run.smithi177.stdout: "exported_inodes": 0, 2024-03-27T11:08:47.654 INFO:teuthology.orchestra.run.smithi177.stdout: "imported": 0, 2024-03-27T11:08:47.654 INFO:teuthology.orchestra.run.smithi177.stdout: "imported_inodes": 0, 2024-03-27T11:08:47.654 INFO:teuthology.orchestra.run.smithi177.stdout: "openino_dir_fetch": 0, 2024-03-27T11:08:47.654 INFO:teuthology.orchestra.run.smithi177.stdout: "openino_backtrace_fetch": 0, 2024-03-27T11:08:47.655 INFO:teuthology.orchestra.run.smithi177.stdout: "openino_peer_discover": 0, 2024-03-27T11:08:47.655 INFO:teuthology.orchestra.run.smithi177.stdout: "root_rfiles": 0, 2024-03-27T11:08:47.655 INFO:teuthology.orchestra.run.smithi177.stdout: "root_rbytes": 0, 2024-03-27T11:08:47.655 INFO:teuthology.orchestra.run.smithi177.stdout: "root_rsnaps": 0, 2024-03-27T11:08:47.655 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_backtrace_fetch": 0, 2024-03-27T11:08:47.655 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_set_tag": 0, 2024-03-27T11:08:47.655 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_backtrace_repaired": 0, 2024-03-27T11:08:47.655 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_inotable_repaired": 0, 2024-03-27T11:08:47.655 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_dir_inodes": 0, 2024-03-27T11:08:47.655 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_dir_base_inodes": 0, 2024-03-27T11:08:47.655 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_dirfrag_rstats": 0, 2024-03-27T11:08:47.655 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_file_inodes": 0, 2024-03-27T11:08:47.655 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_inode_file_caps": 0, 2024-03-27T11:08:47.656 INFO:teuthology.orchestra.run.smithi177.stdout: "ceph_cap_op_revoke": 0, 2024-03-27T11:08:47.656 INFO:teuthology.orchestra.run.smithi177.stdout: "ceph_cap_op_grant": 0, 2024-03-27T11:08:47.656 INFO:teuthology.orchestra.run.smithi177.stdout: "ceph_cap_op_trunc": 0, 2024-03-27T11:08:47.656 INFO:teuthology.orchestra.run.smithi177.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-27T11:08:47.656 INFO:teuthology.orchestra.run.smithi177.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-27T11:08:47.656 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_client_caps": 0, 2024-03-27T11:08:47.656 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_client_caps_dirty": 0, 2024-03-27T11:08:47.656 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_client_cap_release": 0, 2024-03-27T11:08:47.656 INFO:teuthology.orchestra.run.smithi177.stdout: "process_request_cap_release": 0 2024-03-27T11:08:47.656 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:47.656 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_cache": { 2024-03-27T11:08:47.656 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_update": 0, 2024-03-27T11:08:47.656 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_update_receipt": 0, 2024-03-27T11:08:47.656 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_try_discover": 0, 2024-03-27T11:08:47.657 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_send_discover": 0, 2024-03-27T11:08:47.657 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_handle_discover": 0, 2024-03-27T11:08:47.657 INFO:teuthology.orchestra.run.smithi177.stdout: "num_strays": 0, 2024-03-27T11:08:47.657 INFO:teuthology.orchestra.run.smithi177.stdout: "num_strays_delayed": 0, 2024-03-27T11:08:47.657 INFO:teuthology.orchestra.run.smithi177.stdout: "num_strays_enqueuing": 0, 2024-03-27T11:08:47.657 INFO:teuthology.orchestra.run.smithi177.stdout: "strays_created": 0, 2024-03-27T11:08:47.657 INFO:teuthology.orchestra.run.smithi177.stdout: "strays_enqueued": 0, 2024-03-27T11:08:47.657 INFO:teuthology.orchestra.run.smithi177.stdout: "strays_reintegrated": 0, 2024-03-27T11:08:47.657 INFO:teuthology.orchestra.run.smithi177.stdout: "strays_migrated": 0, 2024-03-27T11:08:47.657 INFO:teuthology.orchestra.run.smithi177.stdout: "num_recovering_processing": 0, 2024-03-27T11:08:47.657 INFO:teuthology.orchestra.run.smithi177.stdout: "num_recovering_enqueued": 0, 2024-03-27T11:08:47.657 INFO:teuthology.orchestra.run.smithi177.stdout: "num_recovering_prioritized": 0, 2024-03-27T11:08:47.657 INFO:teuthology.orchestra.run.smithi177.stdout: "recovery_started": 0, 2024-03-27T11:08:47.657 INFO:teuthology.orchestra.run.smithi177.stdout: "recovery_completed": 0, 2024-03-27T11:08:47.658 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_quiesce_path": 0, 2024-03-27T11:08:47.658 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_quiesce_inode": 0, 2024-03-27T11:08:47.658 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_enqueue_scrub": 0, 2024-03-27T11:08:47.658 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_exportdir": 0, 2024-03-27T11:08:47.658 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_flush": 0, 2024-03-27T11:08:47.658 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_fragmentdir": 0, 2024-03-27T11:08:47.658 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_fragstats": 0, 2024-03-27T11:08:47.658 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_inodestats": 0 2024-03-27T11:08:47.658 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:47.658 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_log": { 2024-03-27T11:08:47.658 INFO:teuthology.orchestra.run.smithi177.stdout: "evlrg": 0, 2024-03-27T11:08:47.658 INFO:teuthology.orchestra.run.smithi177.stdout: "evadd": 0, 2024-03-27T11:08:47.658 INFO:teuthology.orchestra.run.smithi177.stdout: "evex": 0, 2024-03-27T11:08:47.659 INFO:teuthology.orchestra.run.smithi177.stdout: "evtrm": 0, 2024-03-27T11:08:47.659 INFO:teuthology.orchestra.run.smithi177.stdout: "ev": 5, 2024-03-27T11:08:47.659 INFO:teuthology.orchestra.run.smithi177.stdout: "evexg": 0, 2024-03-27T11:08:47.659 INFO:teuthology.orchestra.run.smithi177.stdout: "evexd": 0, 2024-03-27T11:08:47.659 INFO:teuthology.orchestra.run.smithi177.stdout: "segadd": 0, 2024-03-27T11:08:47.659 INFO:teuthology.orchestra.run.smithi177.stdout: "segex": 0, 2024-03-27T11:08:47.659 INFO:teuthology.orchestra.run.smithi177.stdout: "segtrm": 0, 2024-03-27T11:08:47.659 INFO:teuthology.orchestra.run.smithi177.stdout: "seg": 2, 2024-03-27T11:08:47.659 INFO:teuthology.orchestra.run.smithi177.stdout: "segmjr": 2, 2024-03-27T11:08:47.659 INFO:teuthology.orchestra.run.smithi177.stdout: "segexg": 0, 2024-03-27T11:08:47.659 INFO:teuthology.orchestra.run.smithi177.stdout: "segexd": 0, 2024-03-27T11:08:47.659 INFO:teuthology.orchestra.run.smithi177.stdout: "expos": 4194304, 2024-03-27T11:08:47.659 INFO:teuthology.orchestra.run.smithi177.stdout: "wrpos": 4195917, 2024-03-27T11:08:47.659 INFO:teuthology.orchestra.run.smithi177.stdout: "rdpos": 4195424, 2024-03-27T11:08:47.660 INFO:teuthology.orchestra.run.smithi177.stdout: "jlat": { 2024-03-27T11:08:47.660 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:47.660 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:47.660 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:47.660 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:47.660 INFO:teuthology.orchestra.run.smithi177.stdout: "replayed": 5 2024-03-27T11:08:47.660 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:47.660 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_mem": { 2024-03-27T11:08:47.660 INFO:teuthology.orchestra.run.smithi177.stdout: "ino": 3, 2024-03-27T11:08:47.660 INFO:teuthology.orchestra.run.smithi177.stdout: "ino+": 3, 2024-03-27T11:08:47.660 INFO:teuthology.orchestra.run.smithi177.stdout: "ino-": 0, 2024-03-27T11:08:47.660 INFO:teuthology.orchestra.run.smithi177.stdout: "dir": 1, 2024-03-27T11:08:47.660 INFO:teuthology.orchestra.run.smithi177.stdout: "dir+": 1, 2024-03-27T11:08:47.660 INFO:teuthology.orchestra.run.smithi177.stdout: "dir-": 0, 2024-03-27T11:08:47.661 INFO:teuthology.orchestra.run.smithi177.stdout: "dn": 0, 2024-03-27T11:08:47.661 INFO:teuthology.orchestra.run.smithi177.stdout: "dn+": 0, 2024-03-27T11:08:47.661 INFO:teuthology.orchestra.run.smithi177.stdout: "dn-": 0, 2024-03-27T11:08:47.661 INFO:teuthology.orchestra.run.smithi177.stdout: "cap": 0, 2024-03-27T11:08:47.661 INFO:teuthology.orchestra.run.smithi177.stdout: "cap+": 0, 2024-03-27T11:08:47.661 INFO:teuthology.orchestra.run.smithi177.stdout: "cap-": 0, 2024-03-27T11:08:47.661 INFO:teuthology.orchestra.run.smithi177.stdout: "rss": 48896, 2024-03-27T11:08:47.661 INFO:teuthology.orchestra.run.smithi177.stdout: "heap": 215416 2024-03-27T11:08:47.661 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:47.661 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_server": { 2024-03-27T11:08:47.661 INFO:teuthology.orchestra.run.smithi177.stdout: "dispatch_client_request": 0, 2024-03-27T11:08:47.661 INFO:teuthology.orchestra.run.smithi177.stdout: "dispatch_server_request": 0, 2024-03-27T11:08:47.661 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_client_request": 0, 2024-03-27T11:08:47.661 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_client_session": 0, 2024-03-27T11:08:47.662 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_peer_request": 0, 2024-03-27T11:08:47.662 INFO:teuthology.orchestra.run.smithi177.stdout: "req_create_latency": { 2024-03-27T11:08:47.662 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:47.662 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:47.662 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:47.662 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:47.662 INFO:teuthology.orchestra.run.smithi177.stdout: "req_getattr_latency": { 2024-03-27T11:08:47.662 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:47.662 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:47.662 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:47.662 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:47.662 INFO:teuthology.orchestra.run.smithi177.stdout: "req_getfilelock_latency": { 2024-03-27T11:08:47.662 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:47.662 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:47.663 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:47.663 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:47.663 INFO:teuthology.orchestra.run.smithi177.stdout: "req_link_latency": { 2024-03-27T11:08:47.663 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:47.663 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:47.663 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:47.663 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:47.663 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lookup_latency": { 2024-03-27T11:08:47.663 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:47.663 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:47.663 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:47.663 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:47.663 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lookuphash_latency": { 2024-03-27T11:08:47.663 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:47.664 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:47.664 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:47.664 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:47.664 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lookupino_latency": { 2024-03-27T11:08:47.664 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:47.664 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:47.664 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:47.664 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:47.664 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lookupname_latency": { 2024-03-27T11:08:47.664 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:47.664 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:47.664 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:47.664 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:47.664 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lookupparent_latency": { 2024-03-27T11:08:47.665 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:47.665 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:47.665 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:47.665 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:47.665 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lookupsnap_latency": { 2024-03-27T11:08:47.665 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:47.665 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:47.665 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:47.665 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:47.665 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lssnap_latency": { 2024-03-27T11:08:47.665 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:47.665 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:47.665 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:47.665 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:47.666 INFO:teuthology.orchestra.run.smithi177.stdout: "req_mkdir_latency": { 2024-03-27T11:08:47.666 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:47.666 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:47.666 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:47.667 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:47.667 INFO:teuthology.orchestra.run.smithi177.stdout: "req_mknod_latency": { 2024-03-27T11:08:47.667 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:47.667 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:47.667 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:47.667 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:47.667 INFO:teuthology.orchestra.run.smithi177.stdout: "req_mksnap_latency": { 2024-03-27T11:08:47.667 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:47.667 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:47.667 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:47.667 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:47.667 INFO:teuthology.orchestra.run.smithi177.stdout: "req_open_latency": { 2024-03-27T11:08:47.668 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:47.668 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:47.668 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:47.668 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:47.668 INFO:teuthology.orchestra.run.smithi177.stdout: "req_readdir_latency": { 2024-03-27T11:08:47.668 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:47.668 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:47.668 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:47.668 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:47.668 INFO:teuthology.orchestra.run.smithi177.stdout: "req_rename_latency": { 2024-03-27T11:08:47.668 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:47.668 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:47.668 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:47.668 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:47.669 INFO:teuthology.orchestra.run.smithi177.stdout: "req_renamesnap_latency": { 2024-03-27T11:08:47.669 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:47.669 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:47.669 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:47.669 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:47.669 INFO:teuthology.orchestra.run.smithi177.stdout: "req_snapdiff_latency": { 2024-03-27T11:08:47.669 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:47.669 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:47.669 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:47.669 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:47.669 INFO:teuthology.orchestra.run.smithi177.stdout: "req_rmdir_latency": { 2024-03-27T11:08:47.669 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:47.669 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:47.670 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:47.670 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:47.670 INFO:teuthology.orchestra.run.smithi177.stdout: "req_rmsnap_latency": { 2024-03-27T11:08:47.670 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:47.670 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:47.670 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:47.670 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:47.670 INFO:teuthology.orchestra.run.smithi177.stdout: "req_rmxattr_latency": { 2024-03-27T11:08:47.670 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:47.670 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:47.670 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:47.670 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:47.670 INFO:teuthology.orchestra.run.smithi177.stdout: "req_setattr_latency": { 2024-03-27T11:08:47.671 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:47.671 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:47.671 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:47.671 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:47.671 INFO:teuthology.orchestra.run.smithi177.stdout: "req_setdirlayout_latency": { 2024-03-27T11:08:47.671 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:47.671 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:47.671 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:47.671 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:47.671 INFO:teuthology.orchestra.run.smithi177.stdout: "req_setfilelock_latency": { 2024-03-27T11:08:47.671 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:47.671 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:47.671 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:47.672 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:47.672 INFO:teuthology.orchestra.run.smithi177.stdout: "req_setlayout_latency": { 2024-03-27T11:08:47.672 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:47.672 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:47.672 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:47.672 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:47.672 INFO:teuthology.orchestra.run.smithi177.stdout: "req_setxattr_latency": { 2024-03-27T11:08:47.672 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:47.672 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:47.672 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:47.672 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:47.672 INFO:teuthology.orchestra.run.smithi177.stdout: "req_symlink_latency": { 2024-03-27T11:08:47.672 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:47.672 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:47.673 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:47.673 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:47.673 INFO:teuthology.orchestra.run.smithi177.stdout: "req_unlink_latency": { 2024-03-27T11:08:47.673 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:47.673 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:47.673 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:47.673 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:47.673 INFO:teuthology.orchestra.run.smithi177.stdout: "cap_revoke_eviction": 0, 2024-03-27T11:08:47.673 INFO:teuthology.orchestra.run.smithi177.stdout: "cap_acquisition_throttle": 0, 2024-03-27T11:08:47.673 INFO:teuthology.orchestra.run.smithi177.stdout: "req_getvxattr_latency": { 2024-03-27T11:08:47.673 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:47.673 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:47.673 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:47.673 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-03-27T11:08:47.674 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:47.674 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_sessions": { 2024-03-27T11:08:47.674 INFO:teuthology.orchestra.run.smithi177.stdout: "session_count": 1, 2024-03-27T11:08:47.674 INFO:teuthology.orchestra.run.smithi177.stdout: "session_add": 2, 2024-03-27T11:08:47.674 INFO:teuthology.orchestra.run.smithi177.stdout: "session_remove": 1, 2024-03-27T11:08:47.674 INFO:teuthology.orchestra.run.smithi177.stdout: "sessions_open": 1, 2024-03-27T11:08:47.674 INFO:teuthology.orchestra.run.smithi177.stdout: "sessions_stale": 0, 2024-03-27T11:08:47.674 INFO:teuthology.orchestra.run.smithi177.stdout: "total_load": 0, 2024-03-27T11:08:47.674 INFO:teuthology.orchestra.run.smithi177.stdout: "average_load": 0, 2024-03-27T11:08:47.674 INFO:teuthology.orchestra.run.smithi177.stdout: "avg_session_uptime": 2159, 2024-03-27T11:08:47.674 INFO:teuthology.orchestra.run.smithi177.stdout: "mdthresh_evicted": 0 2024-03-27T11:08:47.674 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:47.674 INFO:teuthology.orchestra.run.smithi177.stdout: "mempool": { 2024-03-27T11:08:47.674 INFO:teuthology.orchestra.run.smithi177.stdout: "bloom_filter_bytes": 0, 2024-03-27T11:08:47.675 INFO:teuthology.orchestra.run.smithi177.stdout: "bloom_filter_items": 0, 2024-03-27T11:08:47.675 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_alloc_bytes": 0, 2024-03-27T11:08:47.675 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_alloc_items": 0, 2024-03-27T11:08:47.675 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_data_bytes": 0, 2024-03-27T11:08:47.675 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_data_items": 0, 2024-03-27T11:08:47.675 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-27T11:08:47.675 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_onode_items": 0, 2024-03-27T11:08:47.675 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-27T11:08:47.675 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_meta_items": 0, 2024-03-27T11:08:47.675 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_other_bytes": 0, 2024-03-27T11:08:47.675 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_other_items": 0, 2024-03-27T11:08:47.675 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-27T11:08:47.675 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_buffer_items": 0, 2024-03-27T11:08:47.676 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_extent_bytes": 0, 2024-03-27T11:08:47.676 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_extent_items": 0, 2024-03-27T11:08:47.676 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_blob_bytes": 0, 2024-03-27T11:08:47.676 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_blob_items": 0, 2024-03-27T11:08:47.676 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-27T11:08:47.676 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_shared_blob_items": 0, 2024-03-27T11:08:47.676 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-27T11:08:47.676 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_inline_bl_items": 0, 2024-03-27T11:08:47.676 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_fsck_bytes": 0, 2024-03-27T11:08:47.676 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_fsck_items": 0, 2024-03-27T11:08:47.676 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_txc_bytes": 0, 2024-03-27T11:08:47.676 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_txc_items": 0, 2024-03-27T11:08:47.676 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-27T11:08:47.676 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_writing_deferred_items": 0, 2024-03-27T11:08:47.677 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_writing_bytes": 0, 2024-03-27T11:08:47.677 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_writing_items": 0, 2024-03-27T11:08:47.677 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_bytes": 0, 2024-03-27T11:08:47.677 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_items": 0, 2024-03-27T11:08:47.677 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_file_reader_bytes": 0, 2024-03-27T11:08:47.677 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_file_reader_items": 0, 2024-03-27T11:08:47.677 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_file_writer_bytes": 0, 2024-03-27T11:08:47.677 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_file_writer_items": 0, 2024-03-27T11:08:47.677 INFO:teuthology.orchestra.run.smithi177.stdout: "buffer_anon_bytes": 189463, 2024-03-27T11:08:47.677 INFO:teuthology.orchestra.run.smithi177.stdout: "buffer_anon_items": 52, 2024-03-27T11:08:47.677 INFO:teuthology.orchestra.run.smithi177.stdout: "buffer_meta_bytes": 0, 2024-03-27T11:08:47.677 INFO:teuthology.orchestra.run.smithi177.stdout: "buffer_meta_items": 0, 2024-03-27T11:08:47.677 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_bytes": 0, 2024-03-27T11:08:47.677 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_items": 0, 2024-03-27T11:08:47.678 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_mapbl_bytes": 0, 2024-03-27T11:08:47.678 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_mapbl_items": 0, 2024-03-27T11:08:47.678 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_pglog_bytes": 0, 2024-03-27T11:08:47.678 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_pglog_items": 0, 2024-03-27T11:08:47.678 INFO:teuthology.orchestra.run.smithi177.stdout: "osdmap_bytes": 7528, 2024-03-27T11:08:47.678 INFO:teuthology.orchestra.run.smithi177.stdout: "osdmap_items": 158, 2024-03-27T11:08:47.678 INFO:teuthology.orchestra.run.smithi177.stdout: "osdmap_mapping_bytes": 0, 2024-03-27T11:08:47.678 INFO:teuthology.orchestra.run.smithi177.stdout: "osdmap_mapping_items": 0, 2024-03-27T11:08:47.678 INFO:teuthology.orchestra.run.smithi177.stdout: "pgmap_bytes": 0, 2024-03-27T11:08:47.678 INFO:teuthology.orchestra.run.smithi177.stdout: "pgmap_items": 0, 2024-03-27T11:08:47.678 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_co_bytes": 8464, 2024-03-27T11:08:47.678 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_co_items": 13, 2024-03-27T11:08:47.678 INFO:teuthology.orchestra.run.smithi177.stdout: "unittest_1_bytes": 0, 2024-03-27T11:08:47.679 INFO:teuthology.orchestra.run.smithi177.stdout: "unittest_1_items": 0, 2024-03-27T11:08:47.679 INFO:teuthology.orchestra.run.smithi177.stdout: "unittest_2_bytes": 0, 2024-03-27T11:08:47.679 INFO:teuthology.orchestra.run.smithi177.stdout: "unittest_2_items": 0 2024-03-27T11:08:47.679 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:47.679 INFO:teuthology.orchestra.run.smithi177.stdout: "objecter": { 2024-03-27T11:08:47.679 INFO:teuthology.orchestra.run.smithi177.stdout: "op_active": 0, 2024-03-27T11:08:47.679 INFO:teuthology.orchestra.run.smithi177.stdout: "op_laggy": 0, 2024-03-27T11:08:47.679 INFO:teuthology.orchestra.run.smithi177.stdout: "op_send": 2547, 2024-03-27T11:08:47.679 INFO:teuthology.orchestra.run.smithi177.stdout: "op_send_bytes": 21, 2024-03-27T11:08:47.679 INFO:teuthology.orchestra.run.smithi177.stdout: "op_resend": 3, 2024-03-27T11:08:47.679 INFO:teuthology.orchestra.run.smithi177.stdout: "op_reply": 2547, 2024-03-27T11:08:47.679 INFO:teuthology.orchestra.run.smithi177.stdout: "op_latency": { 2024-03-27T11:08:47.679 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 2547, 2024-03-27T11:08:47.679 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 2.540930886, 2024-03-27T11:08:47.680 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000997617 2024-03-27T11:08:47.680 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:47.680 INFO:teuthology.orchestra.run.smithi177.stdout: "op_inflight": 0, 2024-03-27T11:08:47.680 INFO:teuthology.orchestra.run.smithi177.stdout: "oplen_avg": { 2024-03-27T11:08:47.680 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 2547, 2024-03-27T11:08:47.680 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 2548 2024-03-27T11:08:47.680 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:47.680 INFO:teuthology.orchestra.run.smithi177.stdout: "op": 2547, 2024-03-27T11:08:47.680 INFO:teuthology.orchestra.run.smithi177.stdout: "op_r": 2547, 2024-03-27T11:08:47.680 INFO:teuthology.orchestra.run.smithi177.stdout: "op_w": 0, 2024-03-27T11:08:47.680 INFO:teuthology.orchestra.run.smithi177.stdout: "op_rmw": 0, 2024-03-27T11:08:47.680 INFO:teuthology.orchestra.run.smithi177.stdout: "op_pg": 0, 2024-03-27T11:08:47.680 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_stat": 1662, 2024-03-27T11:08:47.680 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_create": 0, 2024-03-27T11:08:47.681 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_read": 883, 2024-03-27T11:08:47.681 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_write": 0, 2024-03-27T11:08:47.681 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_writefull": 0, 2024-03-27T11:08:47.681 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_writesame": 0, 2024-03-27T11:08:47.681 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_append": 0, 2024-03-27T11:08:47.681 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_zero": 0, 2024-03-27T11:08:47.681 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_truncate": 0, 2024-03-27T11:08:47.681 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_delete": 0, 2024-03-27T11:08:47.681 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_mapext": 0, 2024-03-27T11:08:47.681 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_sparse_read": 0, 2024-03-27T11:08:47.681 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_clonerange": 0, 2024-03-27T11:08:47.681 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_getxattr": 1, 2024-03-27T11:08:47.681 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_setxattr": 0, 2024-03-27T11:08:47.681 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_cmpxattr": 0, 2024-03-27T11:08:47.682 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_rmxattr": 0, 2024-03-27T11:08:47.682 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_resetxattrs": 0, 2024-03-27T11:08:47.682 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_call": 0, 2024-03-27T11:08:47.682 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_watch": 0, 2024-03-27T11:08:47.682 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_notify": 0, 2024-03-27T11:08:47.682 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_src_cmpxattr": 0, 2024-03-27T11:08:47.682 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_pgls": 0, 2024-03-27T11:08:47.682 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_pgls_filter": 0, 2024-03-27T11:08:47.682 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_other": 0, 2024-03-27T11:08:47.682 INFO:teuthology.orchestra.run.smithi177.stdout: "linger_active": 0, 2024-03-27T11:08:47.682 INFO:teuthology.orchestra.run.smithi177.stdout: "linger_send": 0, 2024-03-27T11:08:47.682 INFO:teuthology.orchestra.run.smithi177.stdout: "linger_resend": 0, 2024-03-27T11:08:47.682 INFO:teuthology.orchestra.run.smithi177.stdout: "linger_ping": 0, 2024-03-27T11:08:47.682 INFO:teuthology.orchestra.run.smithi177.stdout: "poolop_active": 0, 2024-03-27T11:08:47.683 INFO:teuthology.orchestra.run.smithi177.stdout: "poolop_send": 0, 2024-03-27T11:08:47.683 INFO:teuthology.orchestra.run.smithi177.stdout: "poolop_resend": 0, 2024-03-27T11:08:47.683 INFO:teuthology.orchestra.run.smithi177.stdout: "poolstat_active": 0, 2024-03-27T11:08:47.683 INFO:teuthology.orchestra.run.smithi177.stdout: "poolstat_send": 0, 2024-03-27T11:08:47.683 INFO:teuthology.orchestra.run.smithi177.stdout: "poolstat_resend": 0, 2024-03-27T11:08:47.683 INFO:teuthology.orchestra.run.smithi177.stdout: "statfs_active": 0, 2024-03-27T11:08:47.683 INFO:teuthology.orchestra.run.smithi177.stdout: "statfs_send": 0, 2024-03-27T11:08:47.683 INFO:teuthology.orchestra.run.smithi177.stdout: "statfs_resend": 0, 2024-03-27T11:08:47.683 INFO:teuthology.orchestra.run.smithi177.stdout: "command_active": 0, 2024-03-27T11:08:47.683 INFO:teuthology.orchestra.run.smithi177.stdout: "command_send": 0, 2024-03-27T11:08:47.683 INFO:teuthology.orchestra.run.smithi177.stdout: "command_resend": 0, 2024-03-27T11:08:47.683 INFO:teuthology.orchestra.run.smithi177.stdout: "map_epoch": 92, 2024-03-27T11:08:47.683 INFO:teuthology.orchestra.run.smithi177.stdout: "map_full": 0, 2024-03-27T11:08:47.683 INFO:teuthology.orchestra.run.smithi177.stdout: "map_inc": 2, 2024-03-27T11:08:47.684 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_sessions": 5, 2024-03-27T11:08:47.684 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_session_open": 5, 2024-03-27T11:08:47.684 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_session_close": 0, 2024-03-27T11:08:47.684 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_laggy": 0, 2024-03-27T11:08:47.684 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_wr": 0, 2024-03-27T11:08:47.684 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_rd": 2, 2024-03-27T11:08:47.684 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_del": 0 2024-03-27T11:08:47.684 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:47.684 INFO:teuthology.orchestra.run.smithi177.stdout: "oft": { 2024-03-27T11:08:47.684 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_total_objs": 0, 2024-03-27T11:08:47.684 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_total_kv_pairs": 0, 2024-03-27T11:08:47.684 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_total_updates": 0, 2024-03-27T11:08:47.684 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_total_removes": 0 2024-03-27T11:08:47.684 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:47.685 INFO:teuthology.orchestra.run.smithi177.stdout: "osd-slow-ops": { 2024-03-27T11:08:47.685 INFO:teuthology.orchestra.run.smithi177.stdout: "slow_ops_count": 0 2024-03-27T11:08:47.685 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:47.685 INFO:teuthology.orchestra.run.smithi177.stdout: "purge_queue": { 2024-03-27T11:08:47.685 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_executing_ops": 0, 2024-03-27T11:08:47.685 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_executing_ops_high_water": 0, 2024-03-27T11:08:47.685 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_executing": 0, 2024-03-27T11:08:47.685 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_executing_high_water": 0, 2024-03-27T11:08:47.685 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_executed_ops": 0, 2024-03-27T11:08:47.685 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_executed": 0, 2024-03-27T11:08:47.685 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_item_in_journal": 0 2024-03-27T11:08:47.685 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:47.685 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-27T11:08:47.686 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2024-03-27T11:08:47.686 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 104857600, 2024-03-27T11:08:47.686 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2024-03-27T11:08:47.686 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 2814, 2024-03-27T11:08:47.686 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 1038608, 2024-03-27T11:08:47.686 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2024-03-27T11:08:47.686 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 2814, 2024-03-27T11:08:47.686 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 0, 2024-03-27T11:08:47.686 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 0, 2024-03-27T11:08:47.686 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 2814, 2024-03-27T11:08:47.687 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 1038608, 2024-03-27T11:08:47.687 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2024-03-27T11:08:47.687 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:47.687 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:47.687 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:47.687 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-03-27T11:08:47.687 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:47.687 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-objecter_bytes": { 2024-03-27T11:08:47.687 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2024-03-27T11:08:47.687 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 104857600, 2024-03-27T11:08:47.687 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2024-03-27T11:08:47.687 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 0, 2024-03-27T11:08:47.687 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 0, 2024-03-27T11:08:47.687 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2024-03-27T11:08:47.688 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 0, 2024-03-27T11:08:47.688 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 2547, 2024-03-27T11:08:47.688 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 1618, 2024-03-27T11:08:47.688 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 2547, 2024-03-27T11:08:47.688 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 1618, 2024-03-27T11:08:47.688 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2024-03-27T11:08:47.688 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:47.688 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:47.688 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:47.688 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-03-27T11:08:47.688 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:47.688 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-objecter_ops": { 2024-03-27T11:08:47.688 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2024-03-27T11:08:47.689 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 1024, 2024-03-27T11:08:47.689 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2024-03-27T11:08:47.689 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 0, 2024-03-27T11:08:47.689 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 0, 2024-03-27T11:08:47.689 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2024-03-27T11:08:47.689 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 0, 2024-03-27T11:08:47.689 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 2547, 2024-03-27T11:08:47.689 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 2547, 2024-03-27T11:08:47.689 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 2547, 2024-03-27T11:08:47.689 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 2547, 2024-03-27T11:08:47.689 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2024-03-27T11:08:47.689 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:47.689 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:47.689 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:47.690 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-03-27T11:08:47.690 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:47.690 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-write_buf_throttle": { 2024-03-27T11:08:47.690 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2024-03-27T11:08:47.690 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 3758096384, 2024-03-27T11:08:47.690 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2024-03-27T11:08:47.690 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 0, 2024-03-27T11:08:47.690 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 0, 2024-03-27T11:08:47.690 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2024-03-27T11:08:47.690 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 0, 2024-03-27T11:08:47.690 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 0, 2024-03-27T11:08:47.690 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 0, 2024-03-27T11:08:47.690 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 0, 2024-03-27T11:08:47.691 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 0, 2024-03-27T11:08:47.691 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2024-03-27T11:08:47.691 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:47.691 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:47.691 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:47.691 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-03-27T11:08:47.691 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:47.691 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-write_buf_throttle-0x564f5228c780": { 2024-03-27T11:08:47.691 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2024-03-27T11:08:47.691 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 3758096384, 2024-03-27T11:08:47.691 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2024-03-27T11:08:47.691 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 0, 2024-03-27T11:08:47.691 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 0, 2024-03-27T11:08:47.691 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2024-03-27T11:08:47.692 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 0, 2024-03-27T11:08:47.692 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 0, 2024-03-27T11:08:47.692 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 0, 2024-03-27T11:08:47.692 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 0, 2024-03-27T11:08:47.692 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 0, 2024-03-27T11:08:47.692 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2024-03-27T11:08:47.692 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:47.692 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:47.692 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:47.692 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-03-27T11:08:47.692 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-03-27T11:08:47.692 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-03-27T11:08:47.856 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:08:47 smithi097 ceph-mon[31859]: pgmap v805: 97 pgs: 97 active+clean; 46 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 170 B/s wr, 78 op/s 2024-03-27T11:08:47.923 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:08:47 smithi177 ceph-mon[35294]: pgmap v805: 97 pgs: 97 active+clean; 46 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 170 B/s wr, 78 op/s 2024-03-27T11:08:48.014 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:08:47 smithi114 ceph-mon[35338]: pgmap v805: 97 pgs: 97 active+clean; 46 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 170 B/s wr, 78 op/s 2024-03-27T11:08:48.619 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-03-27T11:08:48.619 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-03-27T11:08:48.620 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2024-03-27T11:08:48.620 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=0 2024-03-27T11:08:48.620 DEBUG:tasks.check_counter:Getting stats from f 2024-03-27T11:08:48.620 DEBUG:teuthology.orchestra.run.smithi177:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell --fsid adca88c4-ec26-11ee-95d0-87774f69a715 -- ceph daemon mds.f perf dump 2024-03-27T11:08:49.072 INFO:teuthology.orchestra.run.smithi177.stderr:Inferring config /var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/mon.c/config 2024-03-27T11:08:49.644 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:08:49 smithi177 ceph-mon[35294]: pgmap v806: 97 pgs: 97 active+clean; 1.6 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 170 B/s wr, 68 op/s 2024-03-27T11:08:49.888 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:08:49 smithi097 ceph-mon[31859]: pgmap v806: 97 pgs: 97 active+clean; 1.6 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 170 B/s wr, 68 op/s 2024-03-27T11:08:50.014 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:08:49 smithi114 ceph-mon[35338]: pgmap v806: 97 pgs: 97 active+clean; 1.6 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 170 B/s wr, 68 op/s 2024-03-27T11:08:51.135 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-03-27T11:08:51.135 INFO:teuthology.orchestra.run.smithi177.stdout: "AsyncMessenger::Worker-0": { 2024-03-27T11:08:51.135 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_messages": 610, 2024-03-27T11:08:51.135 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages": 745, 2024-03-27T11:08:51.135 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_bytes": 260675, 2024-03-27T11:08:51.135 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_bytes": 360546, 2024-03-27T11:08:51.135 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_created_connections": 10, 2024-03-27T11:08:51.135 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_active_connections": 1, 2024-03-27T11:08:51.135 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_total_time": 0.126621968, 2024-03-27T11:08:51.135 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_send_time": 0.049931135, 2024-03-27T11:08:51.135 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_recv_time": 0.064966329, 2024-03-27T11:08:51.136 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_fast_dispatch_time": 0.006216788, 2024-03-27T11:08:51.136 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages_queue_lat": { 2024-03-27T11:08:51.136 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 750, 2024-03-27T11:08:51.136 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.109773615, 2024-03-27T11:08:51.136 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000146364 2024-03-27T11:08:51.136 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:51.136 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_handle_ack_lat": { 2024-03-27T11:08:51.136 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 1265, 2024-03-27T11:08:51.136 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000443994, 2024-03-27T11:08:51.136 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000350 2024-03-27T11:08:51.136 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:51.136 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-03-27T11:08:51.137 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_encrypted_bytes": 448 2024-03-27T11:08:51.137 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:51.137 INFO:teuthology.orchestra.run.smithi177.stdout: "AsyncMessenger::Worker-1": { 2024-03-27T11:08:51.137 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_messages": 385, 2024-03-27T11:08:51.137 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages": 374, 2024-03-27T11:08:51.137 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_bytes": 653603, 2024-03-27T11:08:51.137 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_bytes": 348997, 2024-03-27T11:08:51.137 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_created_connections": 17, 2024-03-27T11:08:51.137 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_active_connections": 6, 2024-03-27T11:08:51.137 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_total_time": 0.092719300, 2024-03-27T11:08:51.137 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_send_time": 0.035328578, 2024-03-27T11:08:51.137 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_recv_time": 0.053296108, 2024-03-27T11:08:51.137 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_fast_dispatch_time": 0.009377812, 2024-03-27T11:08:51.138 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages_queue_lat": { 2024-03-27T11:08:51.138 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 371, 2024-03-27T11:08:51.138 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.062810097, 2024-03-27T11:08:51.138 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000169299 2024-03-27T11:08:51.138 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:51.138 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_handle_ack_lat": { 2024-03-27T11:08:51.138 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 123, 2024-03-27T11:08:51.138 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000063660, 2024-03-27T11:08:51.138 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000517 2024-03-27T11:08:51.138 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:51.138 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_encrypted_bytes": 584880, 2024-03-27T11:08:51.138 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_encrypted_bytes": 157424 2024-03-27T11:08:51.139 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:51.139 INFO:teuthology.orchestra.run.smithi177.stdout: "AsyncMessenger::Worker-2": { 2024-03-27T11:08:51.139 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_messages": 86, 2024-03-27T11:08:51.139 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages": 275, 2024-03-27T11:08:51.139 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_bytes": 67361, 2024-03-27T11:08:51.139 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_bytes": 547739, 2024-03-27T11:08:51.139 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_created_connections": 14, 2024-03-27T11:08:51.139 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_active_connections": 6, 2024-03-27T11:08:51.139 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_total_time": 0.039382743, 2024-03-27T11:08:51.139 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_send_time": 0.019422911, 2024-03-27T11:08:51.139 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_recv_time": 0.017248445, 2024-03-27T11:08:51.139 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_fast_dispatch_time": 0.000202154, 2024-03-27T11:08:51.139 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages_queue_lat": { 2024-03-27T11:08:51.139 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 269, 2024-03-27T11:08:51.140 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.044908811, 2024-03-27T11:08:51.140 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000166947 2024-03-27T11:08:51.140 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:51.140 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_handle_ack_lat": { 2024-03-27T11:08:51.140 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 125, 2024-03-27T11:08:51.140 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000029434, 2024-03-27T11:08:51.140 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000235 2024-03-27T11:08:51.140 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:51.140 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-27T11:08:51.140 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_encrypted_bytes": 427856 2024-03-27T11:08:51.140 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:51.140 INFO:teuthology.orchestra.run.smithi177.stdout: "cct": { 2024-03-27T11:08:51.140 INFO:teuthology.orchestra.run.smithi177.stdout: "total_workers": 1, 2024-03-27T11:08:51.141 INFO:teuthology.orchestra.run.smithi177.stdout: "unhealthy_workers": 0 2024-03-27T11:08:51.141 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:51.141 INFO:teuthology.orchestra.run.smithi177.stdout: "finisher-MDSRank": { 2024-03-27T11:08:51.141 INFO:teuthology.orchestra.run.smithi177.stdout: "queue_len": 0, 2024-03-27T11:08:51.141 INFO:teuthology.orchestra.run.smithi177.stdout: "complete_latency": { 2024-03-27T11:08:51.141 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 39, 2024-03-27T11:08:51.141 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.002880377, 2024-03-27T11:08:51.141 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000073855 2024-03-27T11:08:51.141 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-03-27T11:08:51.141 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:51.141 INFO:teuthology.orchestra.run.smithi177.stdout: "finisher-PurgeQueue": { 2024-03-27T11:08:51.141 INFO:teuthology.orchestra.run.smithi177.stdout: "queue_len": 0, 2024-03-27T11:08:51.142 INFO:teuthology.orchestra.run.smithi177.stdout: "complete_latency": { 2024-03-27T11:08:51.142 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 2, 2024-03-27T11:08:51.142 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000010666, 2024-03-27T11:08:51.142 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000005333 2024-03-27T11:08:51.142 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-03-27T11:08:51.142 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:51.142 INFO:teuthology.orchestra.run.smithi177.stdout: "mds": { 2024-03-27T11:08:51.142 INFO:teuthology.orchestra.run.smithi177.stdout: "request": 1, 2024-03-27T11:08:51.142 INFO:teuthology.orchestra.run.smithi177.stdout: "reply": 0, 2024-03-27T11:08:51.142 INFO:teuthology.orchestra.run.smithi177.stdout: "reply_latency": { 2024-03-27T11:08:51.142 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:51.143 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:51.143 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:51.143 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:51.143 INFO:teuthology.orchestra.run.smithi177.stdout: "slow_reply": 0, 2024-03-27T11:08:51.143 INFO:teuthology.orchestra.run.smithi177.stdout: "forward": 1, 2024-03-27T11:08:51.143 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_fetch_complete": 0, 2024-03-27T11:08:51.143 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_fetch_keys": 0, 2024-03-27T11:08:51.143 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_commit": 11, 2024-03-27T11:08:51.143 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_split": 0, 2024-03-27T11:08:51.144 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_merge": 0, 2024-03-27T11:08:51.144 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes": 17, 2024-03-27T11:08:51.144 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes_top": 1, 2024-03-27T11:08:51.144 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes_bottom": 16, 2024-03-27T11:08:51.144 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes_pin_tail": 0, 2024-03-27T11:08:51.144 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes_pinned": 13, 2024-03-27T11:08:51.144 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes_expired": 8, 2024-03-27T11:08:51.144 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes_with_caps": 0, 2024-03-27T11:08:51.144 INFO:teuthology.orchestra.run.smithi177.stdout: "caps": 0, 2024-03-27T11:08:51.144 INFO:teuthology.orchestra.run.smithi177.stdout: "subtrees": 3, 2024-03-27T11:08:51.144 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse": 11, 2024-03-27T11:08:51.144 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse_hit": 6, 2024-03-27T11:08:51.144 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse_forward": 1, 2024-03-27T11:08:51.144 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse_discover": 4, 2024-03-27T11:08:51.145 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse_dir_fetch": 0, 2024-03-27T11:08:51.145 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse_remote_ino": 0, 2024-03-27T11:08:51.145 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse_lock": 0, 2024-03-27T11:08:51.145 INFO:teuthology.orchestra.run.smithi177.stdout: "load_cent": 1000, 2024-03-27T11:08:51.145 INFO:teuthology.orchestra.run.smithi177.stdout: "q": 1, 2024-03-27T11:08:51.145 INFO:teuthology.orchestra.run.smithi177.stdout: "exported": 0, 2024-03-27T11:08:51.145 INFO:teuthology.orchestra.run.smithi177.stdout: "exported_inodes": 0, 2024-03-27T11:08:51.145 INFO:teuthology.orchestra.run.smithi177.stdout: "imported": 0, 2024-03-27T11:08:51.145 INFO:teuthology.orchestra.run.smithi177.stdout: "imported_inodes": 0, 2024-03-27T11:08:51.145 INFO:teuthology.orchestra.run.smithi177.stdout: "openino_dir_fetch": 0, 2024-03-27T11:08:51.145 INFO:teuthology.orchestra.run.smithi177.stdout: "openino_backtrace_fetch": 0, 2024-03-27T11:08:51.145 INFO:teuthology.orchestra.run.smithi177.stdout: "openino_peer_discover": 0, 2024-03-27T11:08:51.145 INFO:teuthology.orchestra.run.smithi177.stdout: "root_rfiles": 22, 2024-03-27T11:08:51.146 INFO:teuthology.orchestra.run.smithi177.stdout: "root_rbytes": 1460871314, 2024-03-27T11:08:51.146 INFO:teuthology.orchestra.run.smithi177.stdout: "root_rsnaps": 0, 2024-03-27T11:08:51.146 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_backtrace_fetch": 0, 2024-03-27T11:08:51.146 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_set_tag": 0, 2024-03-27T11:08:51.146 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_backtrace_repaired": 0, 2024-03-27T11:08:51.146 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_inotable_repaired": 0, 2024-03-27T11:08:51.146 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_dir_inodes": 0, 2024-03-27T11:08:51.146 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_dir_base_inodes": 0, 2024-03-27T11:08:51.146 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_dirfrag_rstats": 0, 2024-03-27T11:08:51.146 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_file_inodes": 0, 2024-03-27T11:08:51.146 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_inode_file_caps": 0, 2024-03-27T11:08:51.146 INFO:teuthology.orchestra.run.smithi177.stdout: "ceph_cap_op_revoke": 0, 2024-03-27T11:08:51.146 INFO:teuthology.orchestra.run.smithi177.stdout: "ceph_cap_op_grant": 0, 2024-03-27T11:08:51.147 INFO:teuthology.orchestra.run.smithi177.stdout: "ceph_cap_op_trunc": 0, 2024-03-27T11:08:51.147 INFO:teuthology.orchestra.run.smithi177.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-27T11:08:51.147 INFO:teuthology.orchestra.run.smithi177.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-27T11:08:51.147 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_client_caps": 0, 2024-03-27T11:08:51.147 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_client_caps_dirty": 0, 2024-03-27T11:08:51.147 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_client_cap_release": 0, 2024-03-27T11:08:51.147 INFO:teuthology.orchestra.run.smithi177.stdout: "process_request_cap_release": 0 2024-03-27T11:08:51.147 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:51.147 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_cache": { 2024-03-27T11:08:51.147 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_update": 0, 2024-03-27T11:08:51.147 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_update_receipt": 14, 2024-03-27T11:08:51.147 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_try_discover": 9, 2024-03-27T11:08:51.147 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_send_discover": 11, 2024-03-27T11:08:51.147 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_handle_discover": 0, 2024-03-27T11:08:51.148 INFO:teuthology.orchestra.run.smithi177.stdout: "num_strays": 0, 2024-03-27T11:08:51.148 INFO:teuthology.orchestra.run.smithi177.stdout: "num_strays_delayed": 0, 2024-03-27T11:08:51.148 INFO:teuthology.orchestra.run.smithi177.stdout: "num_strays_enqueuing": 0, 2024-03-27T11:08:51.148 INFO:teuthology.orchestra.run.smithi177.stdout: "strays_created": 0, 2024-03-27T11:08:51.148 INFO:teuthology.orchestra.run.smithi177.stdout: "strays_enqueued": 0, 2024-03-27T11:08:51.148 INFO:teuthology.orchestra.run.smithi177.stdout: "strays_reintegrated": 0, 2024-03-27T11:08:51.148 INFO:teuthology.orchestra.run.smithi177.stdout: "strays_migrated": 0, 2024-03-27T11:08:51.148 INFO:teuthology.orchestra.run.smithi177.stdout: "num_recovering_processing": 0, 2024-03-27T11:08:51.148 INFO:teuthology.orchestra.run.smithi177.stdout: "num_recovering_enqueued": 0, 2024-03-27T11:08:51.148 INFO:teuthology.orchestra.run.smithi177.stdout: "num_recovering_prioritized": 0, 2024-03-27T11:08:51.148 INFO:teuthology.orchestra.run.smithi177.stdout: "recovery_started": 0, 2024-03-27T11:08:51.148 INFO:teuthology.orchestra.run.smithi177.stdout: "recovery_completed": 0, 2024-03-27T11:08:51.148 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_quiesce_path": 0, 2024-03-27T11:08:51.148 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_quiesce_inode": 0, 2024-03-27T11:08:51.149 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_enqueue_scrub": 0, 2024-03-27T11:08:51.149 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_exportdir": 0, 2024-03-27T11:08:51.149 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_flush": 0, 2024-03-27T11:08:51.149 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_fragmentdir": 0, 2024-03-27T11:08:51.149 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_fragstats": 0, 2024-03-27T11:08:51.149 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_inodestats": 0 2024-03-27T11:08:51.149 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:51.149 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_log": { 2024-03-27T11:08:51.149 INFO:teuthology.orchestra.run.smithi177.stdout: "evlrg": 0, 2024-03-27T11:08:51.149 INFO:teuthology.orchestra.run.smithi177.stdout: "evadd": 3, 2024-03-27T11:08:51.149 INFO:teuthology.orchestra.run.smithi177.stdout: "evex": 0, 2024-03-27T11:08:51.149 INFO:teuthology.orchestra.run.smithi177.stdout: "evtrm": 0, 2024-03-27T11:08:51.149 INFO:teuthology.orchestra.run.smithi177.stdout: "ev": 3, 2024-03-27T11:08:51.149 INFO:teuthology.orchestra.run.smithi177.stdout: "evexg": 0, 2024-03-27T11:08:51.150 INFO:teuthology.orchestra.run.smithi177.stdout: "evexd": 0, 2024-03-27T11:08:51.150 INFO:teuthology.orchestra.run.smithi177.stdout: "segadd": 2, 2024-03-27T11:08:51.150 INFO:teuthology.orchestra.run.smithi177.stdout: "segex": 0, 2024-03-27T11:08:51.150 INFO:teuthology.orchestra.run.smithi177.stdout: "segtrm": 0, 2024-03-27T11:08:51.150 INFO:teuthology.orchestra.run.smithi177.stdout: "seg": 2, 2024-03-27T11:08:51.150 INFO:teuthology.orchestra.run.smithi177.stdout: "segmjr": 2, 2024-03-27T11:08:51.150 INFO:teuthology.orchestra.run.smithi177.stdout: "segexg": 0, 2024-03-27T11:08:51.150 INFO:teuthology.orchestra.run.smithi177.stdout: "segexd": 0, 2024-03-27T11:08:51.150 INFO:teuthology.orchestra.run.smithi177.stdout: "expos": 4194304, 2024-03-27T11:08:51.150 INFO:teuthology.orchestra.run.smithi177.stdout: "wrpos": 4195385, 2024-03-27T11:08:51.150 INFO:teuthology.orchestra.run.smithi177.stdout: "rdpos": 0, 2024-03-27T11:08:51.150 INFO:teuthology.orchestra.run.smithi177.stdout: "jlat": { 2024-03-27T11:08:51.150 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 2, 2024-03-27T11:08:51.151 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.005539029, 2024-03-27T11:08:51.151 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.002769514 2024-03-27T11:08:51.151 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:51.151 INFO:teuthology.orchestra.run.smithi177.stdout: "replayed": 0 2024-03-27T11:08:51.151 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:51.151 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_mem": { 2024-03-27T11:08:51.151 INFO:teuthology.orchestra.run.smithi177.stdout: "ino": 21, 2024-03-27T11:08:51.151 INFO:teuthology.orchestra.run.smithi177.stdout: "ino+": 25, 2024-03-27T11:08:51.151 INFO:teuthology.orchestra.run.smithi177.stdout: "ino-": 4, 2024-03-27T11:08:51.151 INFO:teuthology.orchestra.run.smithi177.stdout: "dir": 20, 2024-03-27T11:08:51.151 INFO:teuthology.orchestra.run.smithi177.stdout: "dir+": 26, 2024-03-27T11:08:51.151 INFO:teuthology.orchestra.run.smithi177.stdout: "dir-": 6, 2024-03-27T11:08:51.151 INFO:teuthology.orchestra.run.smithi177.stdout: "dn": 17, 2024-03-27T11:08:51.151 INFO:teuthology.orchestra.run.smithi177.stdout: "dn+": 25, 2024-03-27T11:08:51.152 INFO:teuthology.orchestra.run.smithi177.stdout: "dn-": 8, 2024-03-27T11:08:51.152 INFO:teuthology.orchestra.run.smithi177.stdout: "cap": 0, 2024-03-27T11:08:51.152 INFO:teuthology.orchestra.run.smithi177.stdout: "cap+": 0, 2024-03-27T11:08:51.152 INFO:teuthology.orchestra.run.smithi177.stdout: "cap-": 0, 2024-03-27T11:08:51.152 INFO:teuthology.orchestra.run.smithi177.stdout: "rss": 51456, 2024-03-27T11:08:51.152 INFO:teuthology.orchestra.run.smithi177.stdout: "heap": 223608 2024-03-27T11:08:51.152 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:51.152 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_server": { 2024-03-27T11:08:51.152 INFO:teuthology.orchestra.run.smithi177.stdout: "dispatch_client_request": 1, 2024-03-27T11:08:51.152 INFO:teuthology.orchestra.run.smithi177.stdout: "dispatch_server_request": 1, 2024-03-27T11:08:51.152 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_client_request": 1, 2024-03-27T11:08:51.152 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_client_session": 10, 2024-03-27T11:08:51.152 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_peer_request": 2, 2024-03-27T11:08:51.152 INFO:teuthology.orchestra.run.smithi177.stdout: "req_create_latency": { 2024-03-27T11:08:51.153 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:51.153 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:51.153 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:51.153 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:51.153 INFO:teuthology.orchestra.run.smithi177.stdout: "req_getattr_latency": { 2024-03-27T11:08:51.153 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:51.153 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:51.153 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:51.153 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:51.153 INFO:teuthology.orchestra.run.smithi177.stdout: "req_getfilelock_latency": { 2024-03-27T11:08:51.153 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:51.153 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:51.153 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:51.154 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:51.154 INFO:teuthology.orchestra.run.smithi177.stdout: "req_link_latency": { 2024-03-27T11:08:51.154 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:51.154 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:51.154 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:51.154 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:51.154 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lookup_latency": { 2024-03-27T11:08:51.154 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:51.154 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:51.154 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:51.154 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:51.154 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lookuphash_latency": { 2024-03-27T11:08:51.154 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:51.155 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:51.155 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:51.155 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:51.155 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lookupino_latency": { 2024-03-27T11:08:51.155 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:51.155 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:51.155 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:51.155 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:51.155 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lookupname_latency": { 2024-03-27T11:08:51.155 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:51.155 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:51.155 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:51.155 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:51.155 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lookupparent_latency": { 2024-03-27T11:08:51.155 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:51.156 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:51.156 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:51.156 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:51.156 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lookupsnap_latency": { 2024-03-27T11:08:51.156 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:51.156 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:51.156 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:51.156 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:51.156 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lssnap_latency": { 2024-03-27T11:08:51.156 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:51.156 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:51.156 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:51.156 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:51.156 INFO:teuthology.orchestra.run.smithi177.stdout: "req_mkdir_latency": { 2024-03-27T11:08:51.157 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:51.157 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:51.157 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:51.157 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:51.157 INFO:teuthology.orchestra.run.smithi177.stdout: "req_mknod_latency": { 2024-03-27T11:08:51.157 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:51.158 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:51.158 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:51.158 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:51.158 INFO:teuthology.orchestra.run.smithi177.stdout: "req_mksnap_latency": { 2024-03-27T11:08:51.158 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:51.158 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:51.158 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:51.158 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:51.158 INFO:teuthology.orchestra.run.smithi177.stdout: "req_open_latency": { 2024-03-27T11:08:51.158 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:51.158 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:51.158 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:51.158 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:51.158 INFO:teuthology.orchestra.run.smithi177.stdout: "req_readdir_latency": { 2024-03-27T11:08:51.159 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:51.159 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:51.159 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:51.159 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:51.159 INFO:teuthology.orchestra.run.smithi177.stdout: "req_rename_latency": { 2024-03-27T11:08:51.159 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:51.159 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:51.159 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:51.159 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:51.159 INFO:teuthology.orchestra.run.smithi177.stdout: "req_renamesnap_latency": { 2024-03-27T11:08:51.159 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:51.159 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:51.160 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:51.160 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:51.160 INFO:teuthology.orchestra.run.smithi177.stdout: "req_snapdiff_latency": { 2024-03-27T11:08:51.160 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:51.160 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:51.160 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:51.160 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:51.160 INFO:teuthology.orchestra.run.smithi177.stdout: "req_rmdir_latency": { 2024-03-27T11:08:51.160 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:51.160 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:51.160 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:51.160 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:51.160 INFO:teuthology.orchestra.run.smithi177.stdout: "req_rmsnap_latency": { 2024-03-27T11:08:51.160 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:51.161 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:51.161 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:51.161 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:51.161 INFO:teuthology.orchestra.run.smithi177.stdout: "req_rmxattr_latency": { 2024-03-27T11:08:51.161 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:51.161 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:51.161 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:51.161 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:51.161 INFO:teuthology.orchestra.run.smithi177.stdout: "req_setattr_latency": { 2024-03-27T11:08:51.161 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:51.161 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:51.161 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:51.161 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:51.161 INFO:teuthology.orchestra.run.smithi177.stdout: "req_setdirlayout_latency": { 2024-03-27T11:08:51.162 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:51.162 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:51.162 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:51.162 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:51.162 INFO:teuthology.orchestra.run.smithi177.stdout: "req_setfilelock_latency": { 2024-03-27T11:08:51.162 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:51.162 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:51.162 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:51.162 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:51.162 INFO:teuthology.orchestra.run.smithi177.stdout: "req_setlayout_latency": { 2024-03-27T11:08:51.162 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:51.162 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:51.163 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:51.163 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:51.163 INFO:teuthology.orchestra.run.smithi177.stdout: "req_setxattr_latency": { 2024-03-27T11:08:51.163 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:51.163 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:51.163 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:51.163 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:51.163 INFO:teuthology.orchestra.run.smithi177.stdout: "req_symlink_latency": { 2024-03-27T11:08:51.163 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:51.163 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:51.163 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:51.163 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:51.163 INFO:teuthology.orchestra.run.smithi177.stdout: "req_unlink_latency": { 2024-03-27T11:08:51.164 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:51.164 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:51.164 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:51.164 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:51.164 INFO:teuthology.orchestra.run.smithi177.stdout: "cap_revoke_eviction": 0, 2024-03-27T11:08:51.164 INFO:teuthology.orchestra.run.smithi177.stdout: "cap_acquisition_throttle": 0, 2024-03-27T11:08:51.164 INFO:teuthology.orchestra.run.smithi177.stdout: "req_getvxattr_latency": { 2024-03-27T11:08:51.164 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:51.164 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:51.164 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:51.164 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-03-27T11:08:51.164 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:51.164 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_sessions": { 2024-03-27T11:08:51.165 INFO:teuthology.orchestra.run.smithi177.stdout: "session_count": 1, 2024-03-27T11:08:51.165 INFO:teuthology.orchestra.run.smithi177.stdout: "session_add": 1, 2024-03-27T11:08:51.165 INFO:teuthology.orchestra.run.smithi177.stdout: "session_remove": 0, 2024-03-27T11:08:51.165 INFO:teuthology.orchestra.run.smithi177.stdout: "sessions_open": 1, 2024-03-27T11:08:51.165 INFO:teuthology.orchestra.run.smithi177.stdout: "sessions_stale": 0, 2024-03-27T11:08:51.165 INFO:teuthology.orchestra.run.smithi177.stdout: "total_load": 0, 2024-03-27T11:08:51.165 INFO:teuthology.orchestra.run.smithi177.stdout: "average_load": 0, 2024-03-27T11:08:51.165 INFO:teuthology.orchestra.run.smithi177.stdout: "avg_session_uptime": 740, 2024-03-27T11:08:51.165 INFO:teuthology.orchestra.run.smithi177.stdout: "mdthresh_evicted": 0 2024-03-27T11:08:51.165 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:51.165 INFO:teuthology.orchestra.run.smithi177.stdout: "mempool": { 2024-03-27T11:08:51.165 INFO:teuthology.orchestra.run.smithi177.stdout: "bloom_filter_bytes": 0, 2024-03-27T11:08:51.165 INFO:teuthology.orchestra.run.smithi177.stdout: "bloom_filter_items": 0, 2024-03-27T11:08:51.165 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_alloc_bytes": 0, 2024-03-27T11:08:51.166 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_alloc_items": 0, 2024-03-27T11:08:51.166 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_data_bytes": 0, 2024-03-27T11:08:51.166 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_data_items": 0, 2024-03-27T11:08:51.166 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-27T11:08:51.166 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_onode_items": 0, 2024-03-27T11:08:51.166 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-27T11:08:51.166 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_meta_items": 0, 2024-03-27T11:08:51.166 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_other_bytes": 0, 2024-03-27T11:08:51.166 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_other_items": 0, 2024-03-27T11:08:51.166 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-27T11:08:51.166 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_buffer_items": 0, 2024-03-27T11:08:51.166 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_extent_bytes": 0, 2024-03-27T11:08:51.167 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_extent_items": 0, 2024-03-27T11:08:51.167 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_blob_bytes": 0, 2024-03-27T11:08:51.167 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_blob_items": 0, 2024-03-27T11:08:51.167 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-27T11:08:51.167 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_shared_blob_items": 0, 2024-03-27T11:08:51.167 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-27T11:08:51.167 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_inline_bl_items": 0, 2024-03-27T11:08:51.167 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_fsck_bytes": 0, 2024-03-27T11:08:51.167 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_fsck_items": 0, 2024-03-27T11:08:51.167 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_txc_bytes": 0, 2024-03-27T11:08:51.167 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_txc_items": 0, 2024-03-27T11:08:51.167 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-27T11:08:51.167 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_writing_deferred_items": 0, 2024-03-27T11:08:51.167 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_writing_bytes": 0, 2024-03-27T11:08:51.167 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_writing_items": 0, 2024-03-27T11:08:51.168 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_bytes": 0, 2024-03-27T11:08:51.168 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_items": 0, 2024-03-27T11:08:51.168 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_file_reader_bytes": 0, 2024-03-27T11:08:51.168 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_file_reader_items": 0, 2024-03-27T11:08:51.168 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_file_writer_bytes": 0, 2024-03-27T11:08:51.168 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_file_writer_items": 0, 2024-03-27T11:08:51.168 INFO:teuthology.orchestra.run.smithi177.stdout: "buffer_anon_bytes": 233919, 2024-03-27T11:08:51.168 INFO:teuthology.orchestra.run.smithi177.stdout: "buffer_anon_items": 73, 2024-03-27T11:08:51.168 INFO:teuthology.orchestra.run.smithi177.stdout: "buffer_meta_bytes": 0, 2024-03-27T11:08:51.168 INFO:teuthology.orchestra.run.smithi177.stdout: "buffer_meta_items": 0, 2024-03-27T11:08:51.168 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_bytes": 0, 2024-03-27T11:08:51.168 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_items": 0, 2024-03-27T11:08:51.168 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_mapbl_bytes": 0, 2024-03-27T11:08:51.168 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_mapbl_items": 0, 2024-03-27T11:08:51.168 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_pglog_bytes": 0, 2024-03-27T11:08:51.169 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_pglog_items": 0, 2024-03-27T11:08:51.169 INFO:teuthology.orchestra.run.smithi177.stdout: "osdmap_bytes": 7528, 2024-03-27T11:08:51.169 INFO:teuthology.orchestra.run.smithi177.stdout: "osdmap_items": 158, 2024-03-27T11:08:51.169 INFO:teuthology.orchestra.run.smithi177.stdout: "osdmap_mapping_bytes": 0, 2024-03-27T11:08:51.169 INFO:teuthology.orchestra.run.smithi177.stdout: "osdmap_mapping_items": 0, 2024-03-27T11:08:51.169 INFO:teuthology.orchestra.run.smithi177.stdout: "pgmap_bytes": 0, 2024-03-27T11:08:51.169 INFO:teuthology.orchestra.run.smithi177.stdout: "pgmap_items": 0, 2024-03-27T11:08:51.169 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_co_bytes": 84765, 2024-03-27T11:08:51.169 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_co_items": 333, 2024-03-27T11:08:51.169 INFO:teuthology.orchestra.run.smithi177.stdout: "unittest_1_bytes": 0, 2024-03-27T11:08:51.169 INFO:teuthology.orchestra.run.smithi177.stdout: "unittest_1_items": 0, 2024-03-27T11:08:51.169 INFO:teuthology.orchestra.run.smithi177.stdout: "unittest_2_bytes": 0, 2024-03-27T11:08:51.169 INFO:teuthology.orchestra.run.smithi177.stdout: "unittest_2_items": 0 2024-03-27T11:08:51.169 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:51.170 INFO:teuthology.orchestra.run.smithi177.stdout: "objecter": { 2024-03-27T11:08:51.170 INFO:teuthology.orchestra.run.smithi177.stdout: "op_active": 0, 2024-03-27T11:08:51.170 INFO:teuthology.orchestra.run.smithi177.stdout: "op_laggy": 0, 2024-03-27T11:08:51.170 INFO:teuthology.orchestra.run.smithi177.stdout: "op_send": 37, 2024-03-27T11:08:51.170 INFO:teuthology.orchestra.run.smithi177.stdout: "op_send_bytes": 11293, 2024-03-27T11:08:51.170 INFO:teuthology.orchestra.run.smithi177.stdout: "op_resend": 33, 2024-03-27T11:08:51.170 INFO:teuthology.orchestra.run.smithi177.stdout: "op_reply": 37, 2024-03-27T11:08:51.170 INFO:teuthology.orchestra.run.smithi177.stdout: "op_latency": { 2024-03-27T11:08:51.170 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 37, 2024-03-27T11:08:51.170 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.405988631, 2024-03-27T11:08:51.170 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.010972665 2024-03-27T11:08:51.170 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:51.170 INFO:teuthology.orchestra.run.smithi177.stdout: "op_inflight": 0, 2024-03-27T11:08:51.171 INFO:teuthology.orchestra.run.smithi177.stdout: "oplen_avg": { 2024-03-27T11:08:51.171 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 37, 2024-03-27T11:08:51.171 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 59 2024-03-27T11:08:51.171 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:51.171 INFO:teuthology.orchestra.run.smithi177.stdout: "op": 37, 2024-03-27T11:08:51.171 INFO:teuthology.orchestra.run.smithi177.stdout: "op_r": 0, 2024-03-27T11:08:51.171 INFO:teuthology.orchestra.run.smithi177.stdout: "op_w": 37, 2024-03-27T11:08:51.171 INFO:teuthology.orchestra.run.smithi177.stdout: "op_rmw": 0, 2024-03-27T11:08:51.171 INFO:teuthology.orchestra.run.smithi177.stdout: "op_pg": 0, 2024-03-27T11:08:51.171 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_stat": 0, 2024-03-27T11:08:51.171 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_create": 10, 2024-03-27T11:08:51.171 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_read": 0, 2024-03-27T11:08:51.171 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_write": 2, 2024-03-27T11:08:51.171 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_writefull": 6, 2024-03-27T11:08:51.172 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_writesame": 0, 2024-03-27T11:08:51.172 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_append": 0, 2024-03-27T11:08:51.172 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_zero": 0, 2024-03-27T11:08:51.172 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_truncate": 0, 2024-03-27T11:08:51.172 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_delete": 6, 2024-03-27T11:08:51.172 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_mapext": 0, 2024-03-27T11:08:51.172 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_sparse_read": 0, 2024-03-27T11:08:51.172 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_clonerange": 0, 2024-03-27T11:08:51.172 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_getxattr": 0, 2024-03-27T11:08:51.172 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_setxattr": 20, 2024-03-27T11:08:51.172 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_cmpxattr": 0, 2024-03-27T11:08:51.172 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_rmxattr": 0, 2024-03-27T11:08:51.172 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_resetxattrs": 0, 2024-03-27T11:08:51.173 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_call": 0, 2024-03-27T11:08:51.173 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_watch": 0, 2024-03-27T11:08:51.173 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_notify": 0, 2024-03-27T11:08:51.173 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_src_cmpxattr": 0, 2024-03-27T11:08:51.173 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_pgls": 0, 2024-03-27T11:08:51.173 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_pgls_filter": 0, 2024-03-27T11:08:51.173 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_other": 0, 2024-03-27T11:08:51.173 INFO:teuthology.orchestra.run.smithi177.stdout: "linger_active": 0, 2024-03-27T11:08:51.173 INFO:teuthology.orchestra.run.smithi177.stdout: "linger_send": 0, 2024-03-27T11:08:51.173 INFO:teuthology.orchestra.run.smithi177.stdout: "linger_resend": 0, 2024-03-27T11:08:51.173 INFO:teuthology.orchestra.run.smithi177.stdout: "linger_ping": 0, 2024-03-27T11:08:51.173 INFO:teuthology.orchestra.run.smithi177.stdout: "poolop_active": 0, 2024-03-27T11:08:51.173 INFO:teuthology.orchestra.run.smithi177.stdout: "poolop_send": 0, 2024-03-27T11:08:51.174 INFO:teuthology.orchestra.run.smithi177.stdout: "poolop_resend": 0, 2024-03-27T11:08:51.174 INFO:teuthology.orchestra.run.smithi177.stdout: "poolstat_active": 0, 2024-03-27T11:08:51.174 INFO:teuthology.orchestra.run.smithi177.stdout: "poolstat_send": 0, 2024-03-27T11:08:51.174 INFO:teuthology.orchestra.run.smithi177.stdout: "poolstat_resend": 0, 2024-03-27T11:08:51.174 INFO:teuthology.orchestra.run.smithi177.stdout: "statfs_active": 0, 2024-03-27T11:08:51.174 INFO:teuthology.orchestra.run.smithi177.stdout: "statfs_send": 0, 2024-03-27T11:08:51.174 INFO:teuthology.orchestra.run.smithi177.stdout: "statfs_resend": 0, 2024-03-27T11:08:51.174 INFO:teuthology.orchestra.run.smithi177.stdout: "command_active": 0, 2024-03-27T11:08:51.174 INFO:teuthology.orchestra.run.smithi177.stdout: "command_send": 0, 2024-03-27T11:08:51.174 INFO:teuthology.orchestra.run.smithi177.stdout: "command_resend": 0, 2024-03-27T11:08:51.174 INFO:teuthology.orchestra.run.smithi177.stdout: "map_epoch": 92, 2024-03-27T11:08:51.174 INFO:teuthology.orchestra.run.smithi177.stdout: "map_full": 0, 2024-03-27T11:08:51.174 INFO:teuthology.orchestra.run.smithi177.stdout: "map_inc": 2, 2024-03-27T11:08:51.175 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_sessions": 8, 2024-03-27T11:08:51.175 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_session_open": 8, 2024-03-27T11:08:51.175 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_session_close": 0, 2024-03-27T11:08:51.175 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_laggy": 0, 2024-03-27T11:08:51.175 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_wr": 14, 2024-03-27T11:08:51.175 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_rd": 0, 2024-03-27T11:08:51.175 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_del": 1 2024-03-27T11:08:51.175 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:51.175 INFO:teuthology.orchestra.run.smithi177.stdout: "oft": { 2024-03-27T11:08:51.175 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_total_objs": 1, 2024-03-27T11:08:51.175 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_total_kv_pairs": 0, 2024-03-27T11:08:51.175 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_total_updates": 0, 2024-03-27T11:08:51.175 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_total_removes": 0 2024-03-27T11:08:51.175 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:51.176 INFO:teuthology.orchestra.run.smithi177.stdout: "osd-slow-ops": { 2024-03-27T11:08:51.176 INFO:teuthology.orchestra.run.smithi177.stdout: "slow_ops_count": 0 2024-03-27T11:08:51.176 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:51.176 INFO:teuthology.orchestra.run.smithi177.stdout: "purge_queue": { 2024-03-27T11:08:51.176 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_executing_ops": 0, 2024-03-27T11:08:51.176 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_executing_ops_high_water": 0, 2024-03-27T11:08:51.176 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_executing": 0, 2024-03-27T11:08:51.176 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_executing_high_water": 0, 2024-03-27T11:08:51.176 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_executed_ops": 0, 2024-03-27T11:08:51.176 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_executed": 0, 2024-03-27T11:08:51.176 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_item_in_journal": 0 2024-03-27T11:08:51.176 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:51.176 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-27T11:08:51.176 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2024-03-27T11:08:51.177 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 104857600, 2024-03-27T11:08:51.177 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2024-03-27T11:08:51.177 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 1077, 2024-03-27T11:08:51.177 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 866543, 2024-03-27T11:08:51.177 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2024-03-27T11:08:51.177 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 1077, 2024-03-27T11:08:51.177 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 0, 2024-03-27T11:08:51.177 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 0, 2024-03-27T11:08:51.177 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 1077, 2024-03-27T11:08:51.177 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 866543, 2024-03-27T11:08:51.177 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2024-03-27T11:08:51.177 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:51.177 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:51.177 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:51.177 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-03-27T11:08:51.178 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:51.178 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-objecter_bytes": { 2024-03-27T11:08:51.178 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2024-03-27T11:08:51.178 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 104857600, 2024-03-27T11:08:51.178 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2024-03-27T11:08:51.178 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 0, 2024-03-27T11:08:51.178 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 0, 2024-03-27T11:08:51.178 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2024-03-27T11:08:51.178 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 0, 2024-03-27T11:08:51.178 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 37, 2024-03-27T11:08:51.178 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 11293, 2024-03-27T11:08:51.178 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 37, 2024-03-27T11:08:51.178 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 11293, 2024-03-27T11:08:51.178 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2024-03-27T11:08:51.179 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:51.179 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:51.179 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:51.179 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-03-27T11:08:51.179 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:51.179 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-objecter_ops": { 2024-03-27T11:08:51.179 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2024-03-27T11:08:51.179 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 1024, 2024-03-27T11:08:51.179 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2024-03-27T11:08:51.179 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 0, 2024-03-27T11:08:51.179 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 0, 2024-03-27T11:08:51.179 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2024-03-27T11:08:51.179 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 0, 2024-03-27T11:08:51.179 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 37, 2024-03-27T11:08:51.180 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 37, 2024-03-27T11:08:51.180 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 37, 2024-03-27T11:08:51.180 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 37, 2024-03-27T11:08:51.180 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2024-03-27T11:08:51.180 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:51.180 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:51.180 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:51.180 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-03-27T11:08:51.180 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:51.180 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-write_buf_throttle": { 2024-03-27T11:08:51.180 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2024-03-27T11:08:51.180 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 3758096384, 2024-03-27T11:08:51.180 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2024-03-27T11:08:51.180 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 0, 2024-03-27T11:08:51.180 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 0, 2024-03-27T11:08:51.180 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2024-03-27T11:08:51.181 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 0, 2024-03-27T11:08:51.181 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 0, 2024-03-27T11:08:51.181 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 0, 2024-03-27T11:08:51.181 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 0, 2024-03-27T11:08:51.181 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 0, 2024-03-27T11:08:51.181 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2024-03-27T11:08:51.181 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:51.181 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:51.181 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:51.181 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-03-27T11:08:51.181 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:51.181 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-write_buf_throttle-0x55bba4f22500": { 2024-03-27T11:08:51.181 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2024-03-27T11:08:51.181 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 3758096384, 2024-03-27T11:08:51.181 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2024-03-27T11:08:51.182 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 3, 2024-03-27T11:08:51.182 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 1081, 2024-03-27T11:08:51.182 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2024-03-27T11:08:51.182 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 3, 2024-03-27T11:08:51.182 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 0, 2024-03-27T11:08:51.182 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 0, 2024-03-27T11:08:51.182 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 2, 2024-03-27T11:08:51.182 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 1081, 2024-03-27T11:08:51.182 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2024-03-27T11:08:51.182 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:51.182 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:51.182 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:51.182 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-03-27T11:08:51.183 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-03-27T11:08:51.183 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-03-27T11:08:51.888 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:08:51 smithi097 ceph-mon[31859]: pgmap v807: 97 pgs: 97 active+clean; 1.6 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 170 B/s wr, 57 op/s 2024-03-27T11:08:51.915 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:08:51 smithi177 ceph-mon[35294]: pgmap v807: 97 pgs: 97 active+clean; 1.6 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 170 B/s wr, 57 op/s 2024-03-27T11:08:51.962 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-03-27T11:08:51.962 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-03-27T11:08:51.962 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-03-27T11:08:51.962 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=14 2024-03-27T11:08:51.962 DEBUG:tasks.check_counter:Getting stats from i 2024-03-27T11:08:51.962 DEBUG:teuthology.orchestra.run.smithi177:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell --fsid adca88c4-ec26-11ee-95d0-87774f69a715 -- ceph daemon mds.i perf dump 2024-03-27T11:08:52.013 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:08:51 smithi114 ceph-mon[35338]: pgmap v807: 97 pgs: 97 active+clean; 1.6 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 170 B/s wr, 57 op/s 2024-03-27T11:08:52.439 INFO:teuthology.orchestra.run.smithi177.stderr:Inferring config /var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/mon.c/config 2024-03-27T11:08:53.673 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:08:53 smithi177 ceph-mon[35294]: pgmap v808: 97 pgs: 97 active+clean; 1.6 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 33 op/s 2024-03-27T11:08:53.888 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:08:53 smithi097 ceph-mon[31859]: pgmap v808: 97 pgs: 97 active+clean; 1.6 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 33 op/s 2024-03-27T11:08:54.014 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:08:53 smithi114 ceph-mon[35338]: pgmap v808: 97 pgs: 97 active+clean; 1.6 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 33 op/s 2024-03-27T11:08:54.787 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-03-27T11:08:54.788 INFO:teuthology.orchestra.run.smithi177.stdout: "AsyncMessenger::Worker-0": { 2024-03-27T11:08:54.788 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_messages": 2574, 2024-03-27T11:08:54.788 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages": 2574, 2024-03-27T11:08:54.788 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_bytes": 752642, 2024-03-27T11:08:54.788 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_bytes": 817715, 2024-03-27T11:08:54.788 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_created_connections": 6, 2024-03-27T11:08:54.788 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_active_connections": 2, 2024-03-27T11:08:54.788 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_total_time": 0.341238733, 2024-03-27T11:08:54.788 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_send_time": 0.115540669, 2024-03-27T11:08:54.788 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_recv_time": 0.093306990, 2024-03-27T11:08:54.789 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_fast_dispatch_time": 0.105507653, 2024-03-27T11:08:54.789 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages_queue_lat": { 2024-03-27T11:08:54.789 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 2573, 2024-03-27T11:08:54.789 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.153395342, 2024-03-27T11:08:54.789 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000059617 2024-03-27T11:08:54.789 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:54.789 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_handle_ack_lat": { 2024-03-27T11:08:54.789 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:54.789 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:54.789 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:54.789 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:54.789 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-03-27T11:08:54.789 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_encrypted_bytes": 448 2024-03-27T11:08:54.790 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:54.790 INFO:teuthology.orchestra.run.smithi177.stdout: "AsyncMessenger::Worker-1": { 2024-03-27T11:08:54.790 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_messages": 4, 2024-03-27T11:08:54.790 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages": 188, 2024-03-27T11:08:54.790 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_bytes": 876, 2024-03-27T11:08:54.790 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_bytes": 429994, 2024-03-27T11:08:54.790 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_created_connections": 5, 2024-03-27T11:08:54.790 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_active_connections": 3, 2024-03-27T11:08:54.790 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_total_time": 0.021817195, 2024-03-27T11:08:54.790 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_send_time": 0.013681626, 2024-03-27T11:08:54.790 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_recv_time": 0.004092353, 2024-03-27T11:08:54.790 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_fast_dispatch_time": 0.001032143, 2024-03-27T11:08:54.791 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages_queue_lat": { 2024-03-27T11:08:54.791 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 186, 2024-03-27T11:08:54.791 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.016261420, 2024-03-27T11:08:54.791 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000087426 2024-03-27T11:08:54.791 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:54.791 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_handle_ack_lat": { 2024-03-27T11:08:54.791 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:54.791 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:54.791 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:54.791 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:54.791 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-27T11:08:54.791 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_encrypted_bytes": 429296 2024-03-27T11:08:54.791 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:54.791 INFO:teuthology.orchestra.run.smithi177.stdout: "AsyncMessenger::Worker-2": { 2024-03-27T11:08:54.792 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_messages": 267, 2024-03-27T11:08:54.792 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages": 251, 2024-03-27T11:08:54.792 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_bytes": 583884, 2024-03-27T11:08:54.792 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_bytes": 151022, 2024-03-27T11:08:54.792 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_created_connections": 3, 2024-03-27T11:08:54.792 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_active_connections": 2, 2024-03-27T11:08:54.792 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_total_time": 0.069825687, 2024-03-27T11:08:54.792 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_send_time": 0.029478264, 2024-03-27T11:08:54.792 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_recv_time": 0.036428174, 2024-03-27T11:08:54.792 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_fast_dispatch_time": 0.008501930, 2024-03-27T11:08:54.792 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages_queue_lat": { 2024-03-27T11:08:54.792 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 250, 2024-03-27T11:08:54.792 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.019087653, 2024-03-27T11:08:54.793 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000076350 2024-03-27T11:08:54.793 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:54.793 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_handle_ack_lat": { 2024-03-27T11:08:54.793 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:54.793 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:54.793 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:54.793 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:54.793 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_encrypted_bytes": 583616, 2024-03-27T11:08:54.793 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_encrypted_bytes": 150704 2024-03-27T11:08:54.793 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:54.793 INFO:teuthology.orchestra.run.smithi177.stdout: "cct": { 2024-03-27T11:08:54.793 INFO:teuthology.orchestra.run.smithi177.stdout: "total_workers": 1, 2024-03-27T11:08:54.793 INFO:teuthology.orchestra.run.smithi177.stdout: "unhealthy_workers": 0 2024-03-27T11:08:54.793 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:54.794 INFO:teuthology.orchestra.run.smithi177.stdout: "finisher-MDSRank": { 2024-03-27T11:08:54.794 INFO:teuthology.orchestra.run.smithi177.stdout: "queue_len": 0, 2024-03-27T11:08:54.794 INFO:teuthology.orchestra.run.smithi177.stdout: "complete_latency": { 2024-03-27T11:08:54.794 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 4312, 2024-03-27T11:08:54.794 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.149616031, 2024-03-27T11:08:54.794 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000034697 2024-03-27T11:08:54.794 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-03-27T11:08:54.794 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:54.794 INFO:teuthology.orchestra.run.smithi177.stdout: "finisher-PurgeQueue": { 2024-03-27T11:08:54.794 INFO:teuthology.orchestra.run.smithi177.stdout: "queue_len": 0, 2024-03-27T11:08:54.794 INFO:teuthology.orchestra.run.smithi177.stdout: "complete_latency": { 2024-03-27T11:08:54.794 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:54.794 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:54.795 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:54.795 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-03-27T11:08:54.795 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:54.795 INFO:teuthology.orchestra.run.smithi177.stdout: "mds": { 2024-03-27T11:08:54.795 INFO:teuthology.orchestra.run.smithi177.stdout: "request": 0, 2024-03-27T11:08:54.795 INFO:teuthology.orchestra.run.smithi177.stdout: "reply": 0, 2024-03-27T11:08:54.795 INFO:teuthology.orchestra.run.smithi177.stdout: "reply_latency": { 2024-03-27T11:08:54.795 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:54.795 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:54.795 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:54.795 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:54.795 INFO:teuthology.orchestra.run.smithi177.stdout: "slow_reply": 0, 2024-03-27T11:08:54.795 INFO:teuthology.orchestra.run.smithi177.stdout: "forward": 0, 2024-03-27T11:08:54.795 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_fetch_complete": 0, 2024-03-27T11:08:54.796 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_fetch_keys": 0, 2024-03-27T11:08:54.796 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_commit": 0, 2024-03-27T11:08:54.796 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_split": 0, 2024-03-27T11:08:54.796 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_merge": 0, 2024-03-27T11:08:54.796 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes": 3, 2024-03-27T11:08:54.796 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes_top": 0, 2024-03-27T11:08:54.796 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes_bottom": 3, 2024-03-27T11:08:54.796 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes_pin_tail": 0, 2024-03-27T11:08:54.796 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes_pinned": 3, 2024-03-27T11:08:54.796 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes_expired": 0, 2024-03-27T11:08:54.796 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes_with_caps": 0, 2024-03-27T11:08:54.796 INFO:teuthology.orchestra.run.smithi177.stdout: "caps": 0, 2024-03-27T11:08:54.796 INFO:teuthology.orchestra.run.smithi177.stdout: "subtrees": 1, 2024-03-27T11:08:54.796 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse": 0, 2024-03-27T11:08:54.797 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse_hit": 0, 2024-03-27T11:08:54.797 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse_forward": 0, 2024-03-27T11:08:54.797 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse_discover": 0, 2024-03-27T11:08:54.797 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse_dir_fetch": 0, 2024-03-27T11:08:54.797 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse_remote_ino": 0, 2024-03-27T11:08:54.797 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse_lock": 0, 2024-03-27T11:08:54.797 INFO:teuthology.orchestra.run.smithi177.stdout: "load_cent": 0, 2024-03-27T11:08:54.797 INFO:teuthology.orchestra.run.smithi177.stdout: "q": 0, 2024-03-27T11:08:54.797 INFO:teuthology.orchestra.run.smithi177.stdout: "exported": 0, 2024-03-27T11:08:54.797 INFO:teuthology.orchestra.run.smithi177.stdout: "exported_inodes": 0, 2024-03-27T11:08:54.797 INFO:teuthology.orchestra.run.smithi177.stdout: "imported": 0, 2024-03-27T11:08:54.797 INFO:teuthology.orchestra.run.smithi177.stdout: "imported_inodes": 0, 2024-03-27T11:08:54.797 INFO:teuthology.orchestra.run.smithi177.stdout: "openino_dir_fetch": 0, 2024-03-27T11:08:54.797 INFO:teuthology.orchestra.run.smithi177.stdout: "openino_backtrace_fetch": 0, 2024-03-27T11:08:54.798 INFO:teuthology.orchestra.run.smithi177.stdout: "openino_peer_discover": 0, 2024-03-27T11:08:54.798 INFO:teuthology.orchestra.run.smithi177.stdout: "root_rfiles": 0, 2024-03-27T11:08:54.798 INFO:teuthology.orchestra.run.smithi177.stdout: "root_rbytes": 0, 2024-03-27T11:08:54.798 INFO:teuthology.orchestra.run.smithi177.stdout: "root_rsnaps": 0, 2024-03-27T11:08:54.798 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_backtrace_fetch": 0, 2024-03-27T11:08:54.798 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_set_tag": 0, 2024-03-27T11:08:54.798 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_backtrace_repaired": 0, 2024-03-27T11:08:54.798 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_inotable_repaired": 0, 2024-03-27T11:08:54.798 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_dir_inodes": 0, 2024-03-27T11:08:54.798 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_dir_base_inodes": 0, 2024-03-27T11:08:54.798 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_dirfrag_rstats": 0, 2024-03-27T11:08:54.798 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_file_inodes": 0, 2024-03-27T11:08:54.798 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_inode_file_caps": 0, 2024-03-27T11:08:54.799 INFO:teuthology.orchestra.run.smithi177.stdout: "ceph_cap_op_revoke": 0, 2024-03-27T11:08:54.799 INFO:teuthology.orchestra.run.smithi177.stdout: "ceph_cap_op_grant": 0, 2024-03-27T11:08:54.799 INFO:teuthology.orchestra.run.smithi177.stdout: "ceph_cap_op_trunc": 0, 2024-03-27T11:08:54.799 INFO:teuthology.orchestra.run.smithi177.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-27T11:08:54.799 INFO:teuthology.orchestra.run.smithi177.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-27T11:08:54.799 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_client_caps": 0, 2024-03-27T11:08:54.799 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_client_caps_dirty": 0, 2024-03-27T11:08:54.799 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_client_cap_release": 0, 2024-03-27T11:08:54.799 INFO:teuthology.orchestra.run.smithi177.stdout: "process_request_cap_release": 0 2024-03-27T11:08:54.799 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:54.799 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_cache": { 2024-03-27T11:08:54.799 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_update": 0, 2024-03-27T11:08:54.799 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_update_receipt": 0, 2024-03-27T11:08:54.799 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_try_discover": 0, 2024-03-27T11:08:54.799 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_send_discover": 0, 2024-03-27T11:08:54.800 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_handle_discover": 0, 2024-03-27T11:08:54.800 INFO:teuthology.orchestra.run.smithi177.stdout: "num_strays": 0, 2024-03-27T11:08:54.800 INFO:teuthology.orchestra.run.smithi177.stdout: "num_strays_delayed": 0, 2024-03-27T11:08:54.800 INFO:teuthology.orchestra.run.smithi177.stdout: "num_strays_enqueuing": 0, 2024-03-27T11:08:54.800 INFO:teuthology.orchestra.run.smithi177.stdout: "strays_created": 0, 2024-03-27T11:08:54.800 INFO:teuthology.orchestra.run.smithi177.stdout: "strays_enqueued": 0, 2024-03-27T11:08:54.800 INFO:teuthology.orchestra.run.smithi177.stdout: "strays_reintegrated": 0, 2024-03-27T11:08:54.800 INFO:teuthology.orchestra.run.smithi177.stdout: "strays_migrated": 0, 2024-03-27T11:08:54.800 INFO:teuthology.orchestra.run.smithi177.stdout: "num_recovering_processing": 0, 2024-03-27T11:08:54.800 INFO:teuthology.orchestra.run.smithi177.stdout: "num_recovering_enqueued": 0, 2024-03-27T11:08:54.800 INFO:teuthology.orchestra.run.smithi177.stdout: "num_recovering_prioritized": 0, 2024-03-27T11:08:54.800 INFO:teuthology.orchestra.run.smithi177.stdout: "recovery_started": 0, 2024-03-27T11:08:54.800 INFO:teuthology.orchestra.run.smithi177.stdout: "recovery_completed": 0, 2024-03-27T11:08:54.800 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_quiesce_path": 0, 2024-03-27T11:08:54.801 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_quiesce_inode": 0, 2024-03-27T11:08:54.801 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_enqueue_scrub": 0, 2024-03-27T11:08:54.801 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_exportdir": 0, 2024-03-27T11:08:54.801 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_flush": 0, 2024-03-27T11:08:54.801 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_fragmentdir": 0, 2024-03-27T11:08:54.801 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_fragstats": 0, 2024-03-27T11:08:54.801 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_inodestats": 0 2024-03-27T11:08:54.801 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:54.801 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_log": { 2024-03-27T11:08:54.801 INFO:teuthology.orchestra.run.smithi177.stdout: "evlrg": 0, 2024-03-27T11:08:54.801 INFO:teuthology.orchestra.run.smithi177.stdout: "evadd": 0, 2024-03-27T11:08:54.801 INFO:teuthology.orchestra.run.smithi177.stdout: "evex": 0, 2024-03-27T11:08:54.801 INFO:teuthology.orchestra.run.smithi177.stdout: "evtrm": 0, 2024-03-27T11:08:54.802 INFO:teuthology.orchestra.run.smithi177.stdout: "ev": 16, 2024-03-27T11:08:54.802 INFO:teuthology.orchestra.run.smithi177.stdout: "evexg": 0, 2024-03-27T11:08:54.802 INFO:teuthology.orchestra.run.smithi177.stdout: "evexd": 0, 2024-03-27T11:08:54.802 INFO:teuthology.orchestra.run.smithi177.stdout: "segadd": 0, 2024-03-27T11:08:54.802 INFO:teuthology.orchestra.run.smithi177.stdout: "segex": 0, 2024-03-27T11:08:54.802 INFO:teuthology.orchestra.run.smithi177.stdout: "segtrm": 0, 2024-03-27T11:08:54.802 INFO:teuthology.orchestra.run.smithi177.stdout: "seg": 2, 2024-03-27T11:08:54.802 INFO:teuthology.orchestra.run.smithi177.stdout: "segmjr": 2, 2024-03-27T11:08:54.802 INFO:teuthology.orchestra.run.smithi177.stdout: "segexg": 0, 2024-03-27T11:08:54.802 INFO:teuthology.orchestra.run.smithi177.stdout: "segexd": 0, 2024-03-27T11:08:54.802 INFO:teuthology.orchestra.run.smithi177.stdout: "expos": 4194304, 2024-03-27T11:08:54.802 INFO:teuthology.orchestra.run.smithi177.stdout: "wrpos": 4217513, 2024-03-27T11:08:54.802 INFO:teuthology.orchestra.run.smithi177.stdout: "rdpos": 4215862, 2024-03-27T11:08:54.802 INFO:teuthology.orchestra.run.smithi177.stdout: "jlat": { 2024-03-27T11:08:54.803 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:54.803 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:54.803 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:54.803 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:54.803 INFO:teuthology.orchestra.run.smithi177.stdout: "replayed": 16 2024-03-27T11:08:54.803 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:54.803 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_mem": { 2024-03-27T11:08:54.803 INFO:teuthology.orchestra.run.smithi177.stdout: "ino": 4, 2024-03-27T11:08:54.803 INFO:teuthology.orchestra.run.smithi177.stdout: "ino+": 16, 2024-03-27T11:08:54.803 INFO:teuthology.orchestra.run.smithi177.stdout: "ino-": 12, 2024-03-27T11:08:54.803 INFO:teuthology.orchestra.run.smithi177.stdout: "dir": 2, 2024-03-27T11:08:54.803 INFO:teuthology.orchestra.run.smithi177.stdout: "dir+": 13, 2024-03-27T11:08:54.803 INFO:teuthology.orchestra.run.smithi177.stdout: "dir-": 11, 2024-03-27T11:08:54.804 INFO:teuthology.orchestra.run.smithi177.stdout: "dn": 3, 2024-03-27T11:08:54.804 INFO:teuthology.orchestra.run.smithi177.stdout: "dn+": 15, 2024-03-27T11:08:54.804 INFO:teuthology.orchestra.run.smithi177.stdout: "dn-": 12, 2024-03-27T11:08:54.804 INFO:teuthology.orchestra.run.smithi177.stdout: "cap": 0, 2024-03-27T11:08:54.804 INFO:teuthology.orchestra.run.smithi177.stdout: "cap+": 0, 2024-03-27T11:08:54.804 INFO:teuthology.orchestra.run.smithi177.stdout: "cap-": 0, 2024-03-27T11:08:54.804 INFO:teuthology.orchestra.run.smithi177.stdout: "rss": 49024, 2024-03-27T11:08:54.804 INFO:teuthology.orchestra.run.smithi177.stdout: "heap": 215416 2024-03-27T11:08:54.804 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:54.804 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_server": { 2024-03-27T11:08:54.804 INFO:teuthology.orchestra.run.smithi177.stdout: "dispatch_client_request": 0, 2024-03-27T11:08:54.804 INFO:teuthology.orchestra.run.smithi177.stdout: "dispatch_server_request": 0, 2024-03-27T11:08:54.804 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_client_request": 0, 2024-03-27T11:08:54.804 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_client_session": 0, 2024-03-27T11:08:54.805 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_peer_request": 0, 2024-03-27T11:08:54.805 INFO:teuthology.orchestra.run.smithi177.stdout: "req_create_latency": { 2024-03-27T11:08:54.805 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:54.805 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:54.805 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:54.805 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:54.805 INFO:teuthology.orchestra.run.smithi177.stdout: "req_getattr_latency": { 2024-03-27T11:08:54.805 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:54.805 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:54.805 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:54.805 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:54.805 INFO:teuthology.orchestra.run.smithi177.stdout: "req_getfilelock_latency": { 2024-03-27T11:08:54.805 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:54.805 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:54.806 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:54.806 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:54.806 INFO:teuthology.orchestra.run.smithi177.stdout: "req_link_latency": { 2024-03-27T11:08:54.806 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:54.806 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:54.806 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:54.806 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:54.806 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lookup_latency": { 2024-03-27T11:08:54.806 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:54.806 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:54.806 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:54.806 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:54.806 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lookuphash_latency": { 2024-03-27T11:08:54.806 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:54.807 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:54.807 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:54.807 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:54.807 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lookupino_latency": { 2024-03-27T11:08:54.807 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:54.807 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:54.807 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:54.807 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:54.807 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lookupname_latency": { 2024-03-27T11:08:54.807 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:54.807 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:54.807 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:54.807 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:54.807 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lookupparent_latency": { 2024-03-27T11:08:54.808 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:54.808 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:54.808 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:54.808 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:54.808 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lookupsnap_latency": { 2024-03-27T11:08:54.808 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:54.808 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:54.808 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:54.808 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:54.808 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lssnap_latency": { 2024-03-27T11:08:54.808 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:54.808 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:54.808 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:54.808 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:54.808 INFO:teuthology.orchestra.run.smithi177.stdout: "req_mkdir_latency": { 2024-03-27T11:08:54.809 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:54.809 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:54.810 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:54.810 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:54.810 INFO:teuthology.orchestra.run.smithi177.stdout: "req_mknod_latency": { 2024-03-27T11:08:54.810 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:54.810 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:54.810 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:54.810 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:54.810 INFO:teuthology.orchestra.run.smithi177.stdout: "req_mksnap_latency": { 2024-03-27T11:08:54.810 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:54.810 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:54.810 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:54.810 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:54.810 INFO:teuthology.orchestra.run.smithi177.stdout: "req_open_latency": { 2024-03-27T11:08:54.811 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:54.811 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:54.811 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:54.811 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:54.811 INFO:teuthology.orchestra.run.smithi177.stdout: "req_readdir_latency": { 2024-03-27T11:08:54.811 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:54.811 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:54.811 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:54.811 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:54.811 INFO:teuthology.orchestra.run.smithi177.stdout: "req_rename_latency": { 2024-03-27T11:08:54.811 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:54.811 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:54.811 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:54.811 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:54.812 INFO:teuthology.orchestra.run.smithi177.stdout: "req_renamesnap_latency": { 2024-03-27T11:08:54.812 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:54.812 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:54.812 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:54.812 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:54.812 INFO:teuthology.orchestra.run.smithi177.stdout: "req_snapdiff_latency": { 2024-03-27T11:08:54.812 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:54.812 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:54.812 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:54.812 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:54.812 INFO:teuthology.orchestra.run.smithi177.stdout: "req_rmdir_latency": { 2024-03-27T11:08:54.812 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:54.812 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:54.813 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:54.813 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:54.813 INFO:teuthology.orchestra.run.smithi177.stdout: "req_rmsnap_latency": { 2024-03-27T11:08:54.813 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:54.813 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:54.813 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:54.813 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:54.813 INFO:teuthology.orchestra.run.smithi177.stdout: "req_rmxattr_latency": { 2024-03-27T11:08:54.813 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:54.813 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:54.813 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:54.813 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:54.813 INFO:teuthology.orchestra.run.smithi177.stdout: "req_setattr_latency": { 2024-03-27T11:08:54.813 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:54.814 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:54.814 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:54.814 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:54.814 INFO:teuthology.orchestra.run.smithi177.stdout: "req_setdirlayout_latency": { 2024-03-27T11:08:54.814 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:54.814 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:54.814 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:54.814 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:54.814 INFO:teuthology.orchestra.run.smithi177.stdout: "req_setfilelock_latency": { 2024-03-27T11:08:54.814 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:54.814 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:54.814 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:54.814 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:54.814 INFO:teuthology.orchestra.run.smithi177.stdout: "req_setlayout_latency": { 2024-03-27T11:08:54.815 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:54.815 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:54.815 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:54.815 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:54.815 INFO:teuthology.orchestra.run.smithi177.stdout: "req_setxattr_latency": { 2024-03-27T11:08:54.815 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:54.815 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:54.815 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:54.815 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:54.815 INFO:teuthology.orchestra.run.smithi177.stdout: "req_symlink_latency": { 2024-03-27T11:08:54.815 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:54.815 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:54.815 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:54.815 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:54.816 INFO:teuthology.orchestra.run.smithi177.stdout: "req_unlink_latency": { 2024-03-27T11:08:54.816 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:54.816 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:54.816 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:54.816 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:54.816 INFO:teuthology.orchestra.run.smithi177.stdout: "cap_revoke_eviction": 0, 2024-03-27T11:08:54.816 INFO:teuthology.orchestra.run.smithi177.stdout: "cap_acquisition_throttle": 0, 2024-03-27T11:08:54.816 INFO:teuthology.orchestra.run.smithi177.stdout: "req_getvxattr_latency": { 2024-03-27T11:08:54.816 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:54.816 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:54.816 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:54.816 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-03-27T11:08:54.816 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:54.816 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_sessions": { 2024-03-27T11:08:54.817 INFO:teuthology.orchestra.run.smithi177.stdout: "session_count": 1, 2024-03-27T11:08:54.817 INFO:teuthology.orchestra.run.smithi177.stdout: "session_add": 2, 2024-03-27T11:08:54.817 INFO:teuthology.orchestra.run.smithi177.stdout: "session_remove": 1, 2024-03-27T11:08:54.817 INFO:teuthology.orchestra.run.smithi177.stdout: "sessions_open": 1, 2024-03-27T11:08:54.817 INFO:teuthology.orchestra.run.smithi177.stdout: "sessions_stale": 0, 2024-03-27T11:08:54.817 INFO:teuthology.orchestra.run.smithi177.stdout: "total_load": 0, 2024-03-27T11:08:54.817 INFO:teuthology.orchestra.run.smithi177.stdout: "average_load": 0, 2024-03-27T11:08:54.817 INFO:teuthology.orchestra.run.smithi177.stdout: "avg_session_uptime": 2167, 2024-03-27T11:08:54.817 INFO:teuthology.orchestra.run.smithi177.stdout: "mdthresh_evicted": 0 2024-03-27T11:08:54.817 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:54.817 INFO:teuthology.orchestra.run.smithi177.stdout: "mempool": { 2024-03-27T11:08:54.817 INFO:teuthology.orchestra.run.smithi177.stdout: "bloom_filter_bytes": 0, 2024-03-27T11:08:54.817 INFO:teuthology.orchestra.run.smithi177.stdout: "bloom_filter_items": 0, 2024-03-27T11:08:54.818 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_alloc_bytes": 0, 2024-03-27T11:08:54.818 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_alloc_items": 0, 2024-03-27T11:08:54.818 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_data_bytes": 0, 2024-03-27T11:08:54.818 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_data_items": 0, 2024-03-27T11:08:54.818 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-27T11:08:54.818 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_onode_items": 0, 2024-03-27T11:08:54.818 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-27T11:08:54.818 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_meta_items": 0, 2024-03-27T11:08:54.818 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_other_bytes": 0, 2024-03-27T11:08:54.818 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_other_items": 0, 2024-03-27T11:08:54.818 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-27T11:08:54.818 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_buffer_items": 0, 2024-03-27T11:08:54.818 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_extent_bytes": 0, 2024-03-27T11:08:54.819 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_extent_items": 0, 2024-03-27T11:08:54.819 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_blob_bytes": 0, 2024-03-27T11:08:54.819 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_blob_items": 0, 2024-03-27T11:08:54.819 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-27T11:08:54.819 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_shared_blob_items": 0, 2024-03-27T11:08:54.819 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-27T11:08:54.819 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_inline_bl_items": 0, 2024-03-27T11:08:54.819 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_fsck_bytes": 0, 2024-03-27T11:08:54.819 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_fsck_items": 0, 2024-03-27T11:08:54.819 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_txc_bytes": 0, 2024-03-27T11:08:54.819 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_txc_items": 0, 2024-03-27T11:08:54.819 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-27T11:08:54.819 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_writing_deferred_items": 0, 2024-03-27T11:08:54.820 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_writing_bytes": 0, 2024-03-27T11:08:54.820 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_writing_items": 0, 2024-03-27T11:08:54.820 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_bytes": 0, 2024-03-27T11:08:54.820 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_items": 0, 2024-03-27T11:08:54.820 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_file_reader_bytes": 0, 2024-03-27T11:08:54.820 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_file_reader_items": 0, 2024-03-27T11:08:54.820 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_file_writer_bytes": 0, 2024-03-27T11:08:54.820 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_file_writer_items": 0, 2024-03-27T11:08:54.820 INFO:teuthology.orchestra.run.smithi177.stdout: "buffer_anon_bytes": 181287, 2024-03-27T11:08:54.820 INFO:teuthology.orchestra.run.smithi177.stdout: "buffer_anon_items": 45, 2024-03-27T11:08:54.820 INFO:teuthology.orchestra.run.smithi177.stdout: "buffer_meta_bytes": 0, 2024-03-27T11:08:54.820 INFO:teuthology.orchestra.run.smithi177.stdout: "buffer_meta_items": 0, 2024-03-27T11:08:54.820 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_bytes": 0, 2024-03-27T11:08:54.820 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_items": 0, 2024-03-27T11:08:54.821 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_mapbl_bytes": 0, 2024-03-27T11:08:54.821 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_mapbl_items": 0, 2024-03-27T11:08:54.821 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_pglog_bytes": 0, 2024-03-27T11:08:54.821 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_pglog_items": 0, 2024-03-27T11:08:54.821 INFO:teuthology.orchestra.run.smithi177.stdout: "osdmap_bytes": 7528, 2024-03-27T11:08:54.821 INFO:teuthology.orchestra.run.smithi177.stdout: "osdmap_items": 158, 2024-03-27T11:08:54.821 INFO:teuthology.orchestra.run.smithi177.stdout: "osdmap_mapping_bytes": 0, 2024-03-27T11:08:54.821 INFO:teuthology.orchestra.run.smithi177.stdout: "osdmap_mapping_items": 0, 2024-03-27T11:08:54.821 INFO:teuthology.orchestra.run.smithi177.stdout: "pgmap_bytes": 0, 2024-03-27T11:08:54.821 INFO:teuthology.orchestra.run.smithi177.stdout: "pgmap_items": 0, 2024-03-27T11:08:54.821 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_co_bytes": 14096, 2024-03-27T11:08:54.821 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_co_items": 40, 2024-03-27T11:08:54.821 INFO:teuthology.orchestra.run.smithi177.stdout: "unittest_1_bytes": 0, 2024-03-27T11:08:54.821 INFO:teuthology.orchestra.run.smithi177.stdout: "unittest_1_items": 0, 2024-03-27T11:08:54.821 INFO:teuthology.orchestra.run.smithi177.stdout: "unittest_2_bytes": 0, 2024-03-27T11:08:54.822 INFO:teuthology.orchestra.run.smithi177.stdout: "unittest_2_items": 0 2024-03-27T11:08:54.822 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:54.822 INFO:teuthology.orchestra.run.smithi177.stdout: "objecter": { 2024-03-27T11:08:54.822 INFO:teuthology.orchestra.run.smithi177.stdout: "op_active": 0, 2024-03-27T11:08:54.822 INFO:teuthology.orchestra.run.smithi177.stdout: "op_laggy": 0, 2024-03-27T11:08:54.822 INFO:teuthology.orchestra.run.smithi177.stdout: "op_send": 2573, 2024-03-27T11:08:54.822 INFO:teuthology.orchestra.run.smithi177.stdout: "op_send_bytes": 21, 2024-03-27T11:08:54.822 INFO:teuthology.orchestra.run.smithi177.stdout: "op_resend": 3, 2024-03-27T11:08:54.822 INFO:teuthology.orchestra.run.smithi177.stdout: "op_reply": 2573, 2024-03-27T11:08:54.822 INFO:teuthology.orchestra.run.smithi177.stdout: "op_latency": { 2024-03-27T11:08:54.822 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 2573, 2024-03-27T11:08:54.822 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 2.162940710, 2024-03-27T11:08:54.822 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000840629 2024-03-27T11:08:54.823 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:54.823 INFO:teuthology.orchestra.run.smithi177.stdout: "op_inflight": 0, 2024-03-27T11:08:54.823 INFO:teuthology.orchestra.run.smithi177.stdout: "oplen_avg": { 2024-03-27T11:08:54.823 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 2573, 2024-03-27T11:08:54.823 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 2574 2024-03-27T11:08:54.823 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:54.823 INFO:teuthology.orchestra.run.smithi177.stdout: "op": 2573, 2024-03-27T11:08:54.823 INFO:teuthology.orchestra.run.smithi177.stdout: "op_r": 2573, 2024-03-27T11:08:54.823 INFO:teuthology.orchestra.run.smithi177.stdout: "op_w": 0, 2024-03-27T11:08:54.823 INFO:teuthology.orchestra.run.smithi177.stdout: "op_rmw": 0, 2024-03-27T11:08:54.823 INFO:teuthology.orchestra.run.smithi177.stdout: "op_pg": 0, 2024-03-27T11:08:54.823 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_stat": 1676, 2024-03-27T11:08:54.823 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_create": 0, 2024-03-27T11:08:54.823 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_read": 895, 2024-03-27T11:08:54.824 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_write": 0, 2024-03-27T11:08:54.824 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_writefull": 0, 2024-03-27T11:08:54.824 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_writesame": 0, 2024-03-27T11:08:54.824 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_append": 0, 2024-03-27T11:08:54.824 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_zero": 0, 2024-03-27T11:08:54.824 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_truncate": 0, 2024-03-27T11:08:54.824 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_delete": 0, 2024-03-27T11:08:54.824 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_mapext": 0, 2024-03-27T11:08:54.824 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_sparse_read": 0, 2024-03-27T11:08:54.824 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_clonerange": 0, 2024-03-27T11:08:54.824 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_getxattr": 1, 2024-03-27T11:08:54.824 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_setxattr": 0, 2024-03-27T11:08:54.824 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_cmpxattr": 0, 2024-03-27T11:08:54.824 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_rmxattr": 0, 2024-03-27T11:08:54.825 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_resetxattrs": 0, 2024-03-27T11:08:54.825 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_call": 0, 2024-03-27T11:08:54.825 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_watch": 0, 2024-03-27T11:08:54.825 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_notify": 0, 2024-03-27T11:08:54.825 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_src_cmpxattr": 0, 2024-03-27T11:08:54.825 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_pgls": 0, 2024-03-27T11:08:54.825 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_pgls_filter": 0, 2024-03-27T11:08:54.825 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_other": 0, 2024-03-27T11:08:54.825 INFO:teuthology.orchestra.run.smithi177.stdout: "linger_active": 0, 2024-03-27T11:08:54.825 INFO:teuthology.orchestra.run.smithi177.stdout: "linger_send": 0, 2024-03-27T11:08:54.825 INFO:teuthology.orchestra.run.smithi177.stdout: "linger_resend": 0, 2024-03-27T11:08:54.825 INFO:teuthology.orchestra.run.smithi177.stdout: "linger_ping": 0, 2024-03-27T11:08:54.825 INFO:teuthology.orchestra.run.smithi177.stdout: "poolop_active": 0, 2024-03-27T11:08:54.825 INFO:teuthology.orchestra.run.smithi177.stdout: "poolop_send": 0, 2024-03-27T11:08:54.826 INFO:teuthology.orchestra.run.smithi177.stdout: "poolop_resend": 0, 2024-03-27T11:08:54.826 INFO:teuthology.orchestra.run.smithi177.stdout: "poolstat_active": 0, 2024-03-27T11:08:54.826 INFO:teuthology.orchestra.run.smithi177.stdout: "poolstat_send": 0, 2024-03-27T11:08:54.826 INFO:teuthology.orchestra.run.smithi177.stdout: "poolstat_resend": 0, 2024-03-27T11:08:54.826 INFO:teuthology.orchestra.run.smithi177.stdout: "statfs_active": 0, 2024-03-27T11:08:54.826 INFO:teuthology.orchestra.run.smithi177.stdout: "statfs_send": 0, 2024-03-27T11:08:54.826 INFO:teuthology.orchestra.run.smithi177.stdout: "statfs_resend": 0, 2024-03-27T11:08:54.826 INFO:teuthology.orchestra.run.smithi177.stdout: "command_active": 0, 2024-03-27T11:08:54.826 INFO:teuthology.orchestra.run.smithi177.stdout: "command_send": 0, 2024-03-27T11:08:54.826 INFO:teuthology.orchestra.run.smithi177.stdout: "command_resend": 0, 2024-03-27T11:08:54.826 INFO:teuthology.orchestra.run.smithi177.stdout: "map_epoch": 92, 2024-03-27T11:08:54.826 INFO:teuthology.orchestra.run.smithi177.stdout: "map_full": 0, 2024-03-27T11:08:54.826 INFO:teuthology.orchestra.run.smithi177.stdout: "map_inc": 2, 2024-03-27T11:08:54.827 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_sessions": 4, 2024-03-27T11:08:54.827 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_session_open": 4, 2024-03-27T11:08:54.827 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_session_close": 0, 2024-03-27T11:08:54.827 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_laggy": 0, 2024-03-27T11:08:54.827 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_wr": 0, 2024-03-27T11:08:54.827 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_rd": 2, 2024-03-27T11:08:54.827 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_del": 0 2024-03-27T11:08:54.827 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:54.827 INFO:teuthology.orchestra.run.smithi177.stdout: "oft": { 2024-03-27T11:08:54.827 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_total_objs": 0, 2024-03-27T11:08:54.827 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_total_kv_pairs": 0, 2024-03-27T11:08:54.827 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_total_updates": 0, 2024-03-27T11:08:54.827 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_total_removes": 0 2024-03-27T11:08:54.827 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:54.828 INFO:teuthology.orchestra.run.smithi177.stdout: "osd-slow-ops": { 2024-03-27T11:08:54.828 INFO:teuthology.orchestra.run.smithi177.stdout: "slow_ops_count": 0 2024-03-27T11:08:54.828 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:54.828 INFO:teuthology.orchestra.run.smithi177.stdout: "purge_queue": { 2024-03-27T11:08:54.828 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_executing_ops": 0, 2024-03-27T11:08:54.828 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_executing_ops_high_water": 0, 2024-03-27T11:08:54.828 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_executing": 0, 2024-03-27T11:08:54.828 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_executing_high_water": 0, 2024-03-27T11:08:54.828 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_executed_ops": 0, 2024-03-27T11:08:54.828 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_executed": 0, 2024-03-27T11:08:54.828 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_item_in_journal": 0 2024-03-27T11:08:54.828 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:54.828 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-27T11:08:54.828 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2024-03-27T11:08:54.829 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 104857600, 2024-03-27T11:08:54.829 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2024-03-27T11:08:54.829 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 2841, 2024-03-27T11:08:54.829 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 1065029, 2024-03-27T11:08:54.829 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2024-03-27T11:08:54.829 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 2841, 2024-03-27T11:08:54.829 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 0, 2024-03-27T11:08:54.829 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 0, 2024-03-27T11:08:54.829 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 2841, 2024-03-27T11:08:54.829 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 1065029, 2024-03-27T11:08:54.829 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2024-03-27T11:08:54.829 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:54.829 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:54.829 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:54.830 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-03-27T11:08:54.830 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:54.830 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-objecter_bytes": { 2024-03-27T11:08:54.830 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2024-03-27T11:08:54.830 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 104857600, 2024-03-27T11:08:54.830 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2024-03-27T11:08:54.830 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 0, 2024-03-27T11:08:54.830 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 0, 2024-03-27T11:08:54.830 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2024-03-27T11:08:54.830 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 0, 2024-03-27T11:08:54.830 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 2573, 2024-03-27T11:08:54.830 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 23214, 2024-03-27T11:08:54.830 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 2573, 2024-03-27T11:08:54.830 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 23214, 2024-03-27T11:08:54.831 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2024-03-27T11:08:54.831 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:54.831 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:54.831 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:54.831 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-03-27T11:08:54.831 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:54.831 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-objecter_ops": { 2024-03-27T11:08:54.831 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2024-03-27T11:08:54.831 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 1024, 2024-03-27T11:08:54.831 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2024-03-27T11:08:54.831 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 0, 2024-03-27T11:08:54.831 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 0, 2024-03-27T11:08:54.831 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2024-03-27T11:08:54.831 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 0, 2024-03-27T11:08:54.832 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 2573, 2024-03-27T11:08:54.832 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 2573, 2024-03-27T11:08:54.832 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 2573, 2024-03-27T11:08:54.832 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 2573, 2024-03-27T11:08:54.832 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2024-03-27T11:08:54.832 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:54.832 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:54.832 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:54.832 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-03-27T11:08:54.833 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:54.833 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-write_buf_throttle": { 2024-03-27T11:08:54.833 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2024-03-27T11:08:54.833 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 3758096384, 2024-03-27T11:08:54.833 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2024-03-27T11:08:54.833 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 0, 2024-03-27T11:08:54.833 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 0, 2024-03-27T11:08:54.833 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2024-03-27T11:08:54.833 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 0, 2024-03-27T11:08:54.833 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 0, 2024-03-27T11:08:54.833 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 0, 2024-03-27T11:08:54.833 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 0, 2024-03-27T11:08:54.833 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 0, 2024-03-27T11:08:54.833 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2024-03-27T11:08:54.833 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:54.834 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:54.834 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:54.834 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-03-27T11:08:54.834 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:54.834 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-write_buf_throttle-0x561d8b11a780": { 2024-03-27T11:08:54.834 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2024-03-27T11:08:54.834 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 3758096384, 2024-03-27T11:08:54.834 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2024-03-27T11:08:54.834 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 0, 2024-03-27T11:08:54.834 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 0, 2024-03-27T11:08:54.834 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2024-03-27T11:08:54.834 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 0, 2024-03-27T11:08:54.834 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 0, 2024-03-27T11:08:54.835 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 0, 2024-03-27T11:08:54.835 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 0, 2024-03-27T11:08:54.835 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 0, 2024-03-27T11:08:54.835 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2024-03-27T11:08:54.835 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:54.835 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:54.835 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:54.835 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-03-27T11:08:54.835 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-03-27T11:08:54.835 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-03-27T11:08:55.655 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-03-27T11:08:55.656 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-03-27T11:08:55.656 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=0 2024-03-27T11:08:55.656 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=0 2024-03-27T11:08:55.656 DEBUG:tasks.check_counter:Getting stats from l 2024-03-27T11:08:55.656 DEBUG:teuthology.orchestra.run.smithi177:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell --fsid adca88c4-ec26-11ee-95d0-87774f69a715 -- ceph daemon mds.l perf dump 2024-03-27T11:08:55.887 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:08:55 smithi097 ceph-mon[31859]: pgmap v809: 97 pgs: 97 active+clean; 1.6 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 85 B/s wr, 25 op/s 2024-03-27T11:08:55.923 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:08:55 smithi177 ceph-mon[35294]: pgmap v809: 97 pgs: 97 active+clean; 1.6 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 85 B/s wr, 25 op/s 2024-03-27T11:08:56.013 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:08:55 smithi114 ceph-mon[35338]: pgmap v809: 97 pgs: 97 active+clean; 1.6 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 85 B/s wr, 25 op/s 2024-03-27T11:08:56.128 INFO:teuthology.orchestra.run.smithi177.stderr:Inferring config /var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/mon.c/config 2024-03-27T11:08:57.887 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:08:57 smithi097 ceph-mon[31859]: pgmap v810: 97 pgs: 97 active+clean; 1.6 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 15 op/s 2024-03-27T11:08:57.923 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:08:57 smithi177 ceph-mon[35294]: pgmap v810: 97 pgs: 97 active+clean; 1.6 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 15 op/s 2024-03-27T11:08:58.014 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:08:57 smithi114 ceph-mon[35338]: pgmap v810: 97 pgs: 97 active+clean; 1.6 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 15 op/s 2024-03-27T11:08:58.211 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-03-27T11:08:58.211 INFO:teuthology.orchestra.run.smithi177.stdout: "AsyncMessenger::Worker-0": { 2024-03-27T11:08:58.212 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_messages": 531, 2024-03-27T11:08:58.212 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages": 685, 2024-03-27T11:08:58.212 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_bytes": 184543, 2024-03-27T11:08:58.212 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_bytes": 405207, 2024-03-27T11:08:58.212 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_created_connections": 20, 2024-03-27T11:08:58.212 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_active_connections": 3, 2024-03-27T11:08:58.212 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_total_time": 0.124634947, 2024-03-27T11:08:58.212 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_send_time": 0.046953557, 2024-03-27T11:08:58.212 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_recv_time": 0.065872616, 2024-03-27T11:08:58.212 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_fast_dispatch_time": 0.005954046, 2024-03-27T11:08:58.212 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages_queue_lat": { 2024-03-27T11:08:58.212 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 681, 2024-03-27T11:08:58.213 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.077865789, 2024-03-27T11:08:58.213 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000114340 2024-03-27T11:08:58.213 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:58.213 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_handle_ack_lat": { 2024-03-27T11:08:58.213 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 1143, 2024-03-27T11:08:58.213 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000366780, 2024-03-27T11:08:58.213 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000320 2024-03-27T11:08:58.213 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:58.213 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-27T11:08:58.213 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_encrypted_bytes": 0 2024-03-27T11:08:58.214 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:58.214 INFO:teuthology.orchestra.run.smithi177.stdout: "AsyncMessenger::Worker-1": { 2024-03-27T11:08:58.214 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_messages": 451, 2024-03-27T11:08:58.214 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages": 436, 2024-03-27T11:08:58.214 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_bytes": 707225, 2024-03-27T11:08:58.214 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_bytes": 345258, 2024-03-27T11:08:58.214 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_created_connections": 12, 2024-03-27T11:08:58.214 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_active_connections": 6, 2024-03-27T11:08:58.214 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_total_time": 0.096169825, 2024-03-27T11:08:58.214 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_send_time": 0.039554645, 2024-03-27T11:08:58.214 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_recv_time": 0.053717394, 2024-03-27T11:08:58.215 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_fast_dispatch_time": 0.009571196, 2024-03-27T11:08:58.215 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages_queue_lat": { 2024-03-27T11:08:58.215 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 437, 2024-03-27T11:08:58.215 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.073665698, 2024-03-27T11:08:58.215 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000168571 2024-03-27T11:08:58.215 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:58.215 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_handle_ack_lat": { 2024-03-27T11:08:58.215 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 241, 2024-03-27T11:08:58.215 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000056178, 2024-03-27T11:08:58.215 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000233 2024-03-27T11:08:58.215 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:58.215 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_encrypted_bytes": 576608, 2024-03-27T11:08:58.216 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_encrypted_bytes": 161264 2024-03-27T11:08:58.216 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:58.216 INFO:teuthology.orchestra.run.smithi177.stdout: "AsyncMessenger::Worker-2": { 2024-03-27T11:08:58.216 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_messages": 86, 2024-03-27T11:08:58.216 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages": 277, 2024-03-27T11:08:58.216 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_bytes": 71380, 2024-03-27T11:08:58.216 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_bytes": 510034, 2024-03-27T11:08:58.216 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_created_connections": 9, 2024-03-27T11:08:58.216 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_active_connections": 4, 2024-03-27T11:08:58.216 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_total_time": 0.033061107, 2024-03-27T11:08:58.216 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_send_time": 0.017405808, 2024-03-27T11:08:58.217 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_recv_time": 0.013233293, 2024-03-27T11:08:58.217 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_fast_dispatch_time": 0.000580828, 2024-03-27T11:08:58.217 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages_queue_lat": { 2024-03-27T11:08:58.217 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 276, 2024-03-27T11:08:58.217 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.071219207, 2024-03-27T11:08:58.217 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000258040 2024-03-27T11:08:58.217 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:58.217 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_handle_ack_lat": { 2024-03-27T11:08:58.217 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 114, 2024-03-27T11:08:58.217 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000050333, 2024-03-27T11:08:58.217 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000441 2024-03-27T11:08:58.217 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:58.218 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-03-27T11:08:58.218 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_encrypted_bytes": 432592 2024-03-27T11:08:58.218 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:58.218 INFO:teuthology.orchestra.run.smithi177.stdout: "cct": { 2024-03-27T11:08:58.218 INFO:teuthology.orchestra.run.smithi177.stdout: "total_workers": 1, 2024-03-27T11:08:58.218 INFO:teuthology.orchestra.run.smithi177.stdout: "unhealthy_workers": 0 2024-03-27T11:08:58.218 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:58.218 INFO:teuthology.orchestra.run.smithi177.stdout: "finisher-MDSRank": { 2024-03-27T11:08:58.218 INFO:teuthology.orchestra.run.smithi177.stdout: "queue_len": 0, 2024-03-27T11:08:58.218 INFO:teuthology.orchestra.run.smithi177.stdout: "complete_latency": { 2024-03-27T11:08:58.218 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 40, 2024-03-27T11:08:58.218 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.002279753, 2024-03-27T11:08:58.219 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000056993 2024-03-27T11:08:58.219 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-03-27T11:08:58.219 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:58.219 INFO:teuthology.orchestra.run.smithi177.stdout: "finisher-PurgeQueue": { 2024-03-27T11:08:58.219 INFO:teuthology.orchestra.run.smithi177.stdout: "queue_len": 0, 2024-03-27T11:08:58.219 INFO:teuthology.orchestra.run.smithi177.stdout: "complete_latency": { 2024-03-27T11:08:58.219 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 2, 2024-03-27T11:08:58.219 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000008948, 2024-03-27T11:08:58.219 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000004474 2024-03-27T11:08:58.219 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-03-27T11:08:58.219 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:58.219 INFO:teuthology.orchestra.run.smithi177.stdout: "mds": { 2024-03-27T11:08:58.220 INFO:teuthology.orchestra.run.smithi177.stdout: "request": 0, 2024-03-27T11:08:58.220 INFO:teuthology.orchestra.run.smithi177.stdout: "reply": 0, 2024-03-27T11:08:58.220 INFO:teuthology.orchestra.run.smithi177.stdout: "reply_latency": { 2024-03-27T11:08:58.220 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:58.220 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:58.220 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:58.220 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:58.220 INFO:teuthology.orchestra.run.smithi177.stdout: "slow_reply": 0, 2024-03-27T11:08:58.220 INFO:teuthology.orchestra.run.smithi177.stdout: "forward": 0, 2024-03-27T11:08:58.220 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_fetch_complete": 0, 2024-03-27T11:08:58.220 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_fetch_keys": 0, 2024-03-27T11:08:58.220 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_commit": 11, 2024-03-27T11:08:58.221 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_split": 0, 2024-03-27T11:08:58.221 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_merge": 0, 2024-03-27T11:08:58.221 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes": 16, 2024-03-27T11:08:58.221 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes_top": 0, 2024-03-27T11:08:58.221 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes_bottom": 16, 2024-03-27T11:08:58.221 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes_pin_tail": 0, 2024-03-27T11:08:58.221 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes_pinned": 13, 2024-03-27T11:08:58.221 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes_expired": 8, 2024-03-27T11:08:58.221 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes_with_caps": 0, 2024-03-27T11:08:58.221 INFO:teuthology.orchestra.run.smithi177.stdout: "caps": 0, 2024-03-27T11:08:58.221 INFO:teuthology.orchestra.run.smithi177.stdout: "subtrees": 3, 2024-03-27T11:08:58.221 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse": 8, 2024-03-27T11:08:58.222 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse_hit": 5, 2024-03-27T11:08:58.222 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse_forward": 0, 2024-03-27T11:08:58.222 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse_discover": 3, 2024-03-27T11:08:58.222 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse_dir_fetch": 0, 2024-03-27T11:08:58.222 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse_remote_ino": 0, 2024-03-27T11:08:58.222 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse_lock": 0, 2024-03-27T11:08:58.222 INFO:teuthology.orchestra.run.smithi177.stdout: "load_cent": 0, 2024-03-27T11:08:58.222 INFO:teuthology.orchestra.run.smithi177.stdout: "q": 0, 2024-03-27T11:08:58.222 INFO:teuthology.orchestra.run.smithi177.stdout: "exported": 0, 2024-03-27T11:08:58.222 INFO:teuthology.orchestra.run.smithi177.stdout: "exported_inodes": 0, 2024-03-27T11:08:58.223 INFO:teuthology.orchestra.run.smithi177.stdout: "imported": 0, 2024-03-27T11:08:58.223 INFO:teuthology.orchestra.run.smithi177.stdout: "imported_inodes": 0, 2024-03-27T11:08:58.223 INFO:teuthology.orchestra.run.smithi177.stdout: "openino_dir_fetch": 0, 2024-03-27T11:08:58.223 INFO:teuthology.orchestra.run.smithi177.stdout: "openino_backtrace_fetch": 0, 2024-03-27T11:08:58.223 INFO:teuthology.orchestra.run.smithi177.stdout: "openino_peer_discover": 0, 2024-03-27T11:08:58.223 INFO:teuthology.orchestra.run.smithi177.stdout: "root_rfiles": 22, 2024-03-27T11:08:58.223 INFO:teuthology.orchestra.run.smithi177.stdout: "root_rbytes": 1460871314, 2024-03-27T11:08:58.223 INFO:teuthology.orchestra.run.smithi177.stdout: "root_rsnaps": 0, 2024-03-27T11:08:58.223 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_backtrace_fetch": 0, 2024-03-27T11:08:58.223 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_set_tag": 0, 2024-03-27T11:08:58.223 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_backtrace_repaired": 0, 2024-03-27T11:08:58.223 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_inotable_repaired": 0, 2024-03-27T11:08:58.224 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_dir_inodes": 0, 2024-03-27T11:08:58.224 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_dir_base_inodes": 0, 2024-03-27T11:08:58.224 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_dirfrag_rstats": 0, 2024-03-27T11:08:58.224 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_file_inodes": 0, 2024-03-27T11:08:58.224 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_inode_file_caps": 0, 2024-03-27T11:08:58.224 INFO:teuthology.orchestra.run.smithi177.stdout: "ceph_cap_op_revoke": 0, 2024-03-27T11:08:58.224 INFO:teuthology.orchestra.run.smithi177.stdout: "ceph_cap_op_grant": 0, 2024-03-27T11:08:58.224 INFO:teuthology.orchestra.run.smithi177.stdout: "ceph_cap_op_trunc": 0, 2024-03-27T11:08:58.224 INFO:teuthology.orchestra.run.smithi177.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-27T11:08:58.224 INFO:teuthology.orchestra.run.smithi177.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-27T11:08:58.224 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_client_caps": 0, 2024-03-27T11:08:58.224 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_client_caps_dirty": 0, 2024-03-27T11:08:58.225 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_client_cap_release": 0, 2024-03-27T11:08:58.225 INFO:teuthology.orchestra.run.smithi177.stdout: "process_request_cap_release": 0 2024-03-27T11:08:58.225 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:58.225 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_cache": { 2024-03-27T11:08:58.225 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_update": 0, 2024-03-27T11:08:58.225 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_update_receipt": 11, 2024-03-27T11:08:58.225 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_try_discover": 7, 2024-03-27T11:08:58.225 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_send_discover": 9, 2024-03-27T11:08:58.225 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_handle_discover": 0, 2024-03-27T11:08:58.225 INFO:teuthology.orchestra.run.smithi177.stdout: "num_strays": 0, 2024-03-27T11:08:58.225 INFO:teuthology.orchestra.run.smithi177.stdout: "num_strays_delayed": 0, 2024-03-27T11:08:58.225 INFO:teuthology.orchestra.run.smithi177.stdout: "num_strays_enqueuing": 0, 2024-03-27T11:08:58.226 INFO:teuthology.orchestra.run.smithi177.stdout: "strays_created": 0, 2024-03-27T11:08:58.226 INFO:teuthology.orchestra.run.smithi177.stdout: "strays_enqueued": 0, 2024-03-27T11:08:58.226 INFO:teuthology.orchestra.run.smithi177.stdout: "strays_reintegrated": 0, 2024-03-27T11:08:58.226 INFO:teuthology.orchestra.run.smithi177.stdout: "strays_migrated": 0, 2024-03-27T11:08:58.226 INFO:teuthology.orchestra.run.smithi177.stdout: "num_recovering_processing": 0, 2024-03-27T11:08:58.226 INFO:teuthology.orchestra.run.smithi177.stdout: "num_recovering_enqueued": 0, 2024-03-27T11:08:58.226 INFO:teuthology.orchestra.run.smithi177.stdout: "num_recovering_prioritized": 0, 2024-03-27T11:08:58.226 INFO:teuthology.orchestra.run.smithi177.stdout: "recovery_started": 0, 2024-03-27T11:08:58.226 INFO:teuthology.orchestra.run.smithi177.stdout: "recovery_completed": 0, 2024-03-27T11:08:58.226 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_quiesce_path": 0, 2024-03-27T11:08:58.226 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_quiesce_inode": 0, 2024-03-27T11:08:58.226 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_enqueue_scrub": 0, 2024-03-27T11:08:58.227 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_exportdir": 0, 2024-03-27T11:08:58.227 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_flush": 0, 2024-03-27T11:08:58.227 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_fragmentdir": 0, 2024-03-27T11:08:58.227 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_fragstats": 0, 2024-03-27T11:08:58.227 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_inodestats": 0 2024-03-27T11:08:58.227 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:58.227 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_log": { 2024-03-27T11:08:58.227 INFO:teuthology.orchestra.run.smithi177.stdout: "evlrg": 0, 2024-03-27T11:08:58.227 INFO:teuthology.orchestra.run.smithi177.stdout: "evadd": 3, 2024-03-27T11:08:58.227 INFO:teuthology.orchestra.run.smithi177.stdout: "evex": 0, 2024-03-27T11:08:58.227 INFO:teuthology.orchestra.run.smithi177.stdout: "evtrm": 0, 2024-03-27T11:08:58.227 INFO:teuthology.orchestra.run.smithi177.stdout: "ev": 3, 2024-03-27T11:08:58.228 INFO:teuthology.orchestra.run.smithi177.stdout: "evexg": 0, 2024-03-27T11:08:58.228 INFO:teuthology.orchestra.run.smithi177.stdout: "evexd": 0, 2024-03-27T11:08:58.228 INFO:teuthology.orchestra.run.smithi177.stdout: "segadd": 2, 2024-03-27T11:08:58.228 INFO:teuthology.orchestra.run.smithi177.stdout: "segex": 0, 2024-03-27T11:08:58.228 INFO:teuthology.orchestra.run.smithi177.stdout: "segtrm": 0, 2024-03-27T11:08:58.228 INFO:teuthology.orchestra.run.smithi177.stdout: "seg": 2, 2024-03-27T11:08:58.228 INFO:teuthology.orchestra.run.smithi177.stdout: "segmjr": 2, 2024-03-27T11:08:58.228 INFO:teuthology.orchestra.run.smithi177.stdout: "segexg": 0, 2024-03-27T11:08:58.228 INFO:teuthology.orchestra.run.smithi177.stdout: "segexd": 0, 2024-03-27T11:08:58.228 INFO:teuthology.orchestra.run.smithi177.stdout: "expos": 4194304, 2024-03-27T11:08:58.228 INFO:teuthology.orchestra.run.smithi177.stdout: "wrpos": 4195385, 2024-03-27T11:08:58.228 INFO:teuthology.orchestra.run.smithi177.stdout: "rdpos": 0, 2024-03-27T11:08:58.229 INFO:teuthology.orchestra.run.smithi177.stdout: "jlat": { 2024-03-27T11:08:58.229 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 2, 2024-03-27T11:08:58.229 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.004331164, 2024-03-27T11:08:58.229 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.002165582 2024-03-27T11:08:58.229 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:58.229 INFO:teuthology.orchestra.run.smithi177.stdout: "replayed": 0 2024-03-27T11:08:58.229 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:58.229 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_mem": { 2024-03-27T11:08:58.229 INFO:teuthology.orchestra.run.smithi177.stdout: "ino": 20, 2024-03-27T11:08:58.229 INFO:teuthology.orchestra.run.smithi177.stdout: "ino+": 24, 2024-03-27T11:08:58.229 INFO:teuthology.orchestra.run.smithi177.stdout: "ino-": 4, 2024-03-27T11:08:58.229 INFO:teuthology.orchestra.run.smithi177.stdout: "dir": 19, 2024-03-27T11:08:58.230 INFO:teuthology.orchestra.run.smithi177.stdout: "dir+": 25, 2024-03-27T11:08:58.230 INFO:teuthology.orchestra.run.smithi177.stdout: "dir-": 6, 2024-03-27T11:08:58.230 INFO:teuthology.orchestra.run.smithi177.stdout: "dn": 16, 2024-03-27T11:08:58.230 INFO:teuthology.orchestra.run.smithi177.stdout: "dn+": 24, 2024-03-27T11:08:58.230 INFO:teuthology.orchestra.run.smithi177.stdout: "dn-": 8, 2024-03-27T11:08:58.230 INFO:teuthology.orchestra.run.smithi177.stdout: "cap": 0, 2024-03-27T11:08:58.230 INFO:teuthology.orchestra.run.smithi177.stdout: "cap+": 0, 2024-03-27T11:08:58.230 INFO:teuthology.orchestra.run.smithi177.stdout: "cap-": 0, 2024-03-27T11:08:58.230 INFO:teuthology.orchestra.run.smithi177.stdout: "rss": 51012, 2024-03-27T11:08:58.230 INFO:teuthology.orchestra.run.smithi177.stdout: "heap": 223608 2024-03-27T11:08:58.230 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:58.230 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_server": { 2024-03-27T11:08:58.231 INFO:teuthology.orchestra.run.smithi177.stdout: "dispatch_client_request": 0, 2024-03-27T11:08:58.231 INFO:teuthology.orchestra.run.smithi177.stdout: "dispatch_server_request": 1, 2024-03-27T11:08:58.231 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_client_request": 0, 2024-03-27T11:08:58.231 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_client_session": 10, 2024-03-27T11:08:58.231 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_peer_request": 2, 2024-03-27T11:08:58.231 INFO:teuthology.orchestra.run.smithi177.stdout: "req_create_latency": { 2024-03-27T11:08:58.231 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:58.231 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:58.231 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:58.231 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:58.231 INFO:teuthology.orchestra.run.smithi177.stdout: "req_getattr_latency": { 2024-03-27T11:08:58.231 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:58.232 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:58.232 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:58.232 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:58.232 INFO:teuthology.orchestra.run.smithi177.stdout: "req_getfilelock_latency": { 2024-03-27T11:08:58.232 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:58.232 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:58.232 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:58.232 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:58.232 INFO:teuthology.orchestra.run.smithi177.stdout: "req_link_latency": { 2024-03-27T11:08:58.232 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:58.233 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:58.233 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:58.233 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:58.233 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lookup_latency": { 2024-03-27T11:08:58.233 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:58.233 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:58.233 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:58.233 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:58.233 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lookuphash_latency": { 2024-03-27T11:08:58.233 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:58.233 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:58.233 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:58.234 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:58.234 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lookupino_latency": { 2024-03-27T11:08:58.234 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:58.234 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:58.234 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:58.234 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:58.234 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lookupname_latency": { 2024-03-27T11:08:58.234 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:58.234 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:58.234 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:58.234 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:58.235 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lookupparent_latency": { 2024-03-27T11:08:58.235 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:58.235 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:58.235 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:58.235 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:58.235 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lookupsnap_latency": { 2024-03-27T11:08:58.235 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:58.235 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:58.235 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:58.235 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:58.235 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lssnap_latency": { 2024-03-27T11:08:58.235 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:58.235 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:58.236 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:58.236 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:58.236 INFO:teuthology.orchestra.run.smithi177.stdout: "req_mkdir_latency": { 2024-03-27T11:08:58.236 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:58.237 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:58.237 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:58.237 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:58.237 INFO:teuthology.orchestra.run.smithi177.stdout: "req_mknod_latency": { 2024-03-27T11:08:58.237 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:58.237 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:58.237 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:58.237 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:58.237 INFO:teuthology.orchestra.run.smithi177.stdout: "req_mksnap_latency": { 2024-03-27T11:08:58.237 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:58.237 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:58.237 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:58.238 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:58.238 INFO:teuthology.orchestra.run.smithi177.stdout: "req_open_latency": { 2024-03-27T11:08:58.238 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:58.238 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:58.238 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:58.238 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:58.238 INFO:teuthology.orchestra.run.smithi177.stdout: "req_readdir_latency": { 2024-03-27T11:08:58.238 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:58.238 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:58.238 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:58.238 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:58.239 INFO:teuthology.orchestra.run.smithi177.stdout: "req_rename_latency": { 2024-03-27T11:08:58.239 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:58.239 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:58.239 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:58.239 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:58.239 INFO:teuthology.orchestra.run.smithi177.stdout: "req_renamesnap_latency": { 2024-03-27T11:08:58.239 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:58.239 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:58.239 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:58.239 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:58.239 INFO:teuthology.orchestra.run.smithi177.stdout: "req_snapdiff_latency": { 2024-03-27T11:08:58.239 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:58.240 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:58.240 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:58.240 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:58.240 INFO:teuthology.orchestra.run.smithi177.stdout: "req_rmdir_latency": { 2024-03-27T11:08:58.240 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:58.240 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:58.240 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:58.240 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:58.240 INFO:teuthology.orchestra.run.smithi177.stdout: "req_rmsnap_latency": { 2024-03-27T11:08:58.240 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:58.240 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:58.240 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:58.241 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:58.241 INFO:teuthology.orchestra.run.smithi177.stdout: "req_rmxattr_latency": { 2024-03-27T11:08:58.241 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:58.241 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:58.241 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:58.241 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:58.241 INFO:teuthology.orchestra.run.smithi177.stdout: "req_setattr_latency": { 2024-03-27T11:08:58.241 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:58.241 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:58.241 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:58.242 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:58.242 INFO:teuthology.orchestra.run.smithi177.stdout: "req_setdirlayout_latency": { 2024-03-27T11:08:58.242 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:58.242 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:58.242 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:58.242 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:58.242 INFO:teuthology.orchestra.run.smithi177.stdout: "req_setfilelock_latency": { 2024-03-27T11:08:58.242 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:58.242 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:58.242 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:58.242 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:58.243 INFO:teuthology.orchestra.run.smithi177.stdout: "req_setlayout_latency": { 2024-03-27T11:08:58.243 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:58.243 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:58.243 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:58.243 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:58.243 INFO:teuthology.orchestra.run.smithi177.stdout: "req_setxattr_latency": { 2024-03-27T11:08:58.243 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:58.243 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:58.243 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:58.243 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:58.243 INFO:teuthology.orchestra.run.smithi177.stdout: "req_symlink_latency": { 2024-03-27T11:08:58.243 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:58.244 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:58.244 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:58.244 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:58.244 INFO:teuthology.orchestra.run.smithi177.stdout: "req_unlink_latency": { 2024-03-27T11:08:58.244 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:58.244 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:58.244 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:58.244 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:58.244 INFO:teuthology.orchestra.run.smithi177.stdout: "cap_revoke_eviction": 0, 2024-03-27T11:08:58.244 INFO:teuthology.orchestra.run.smithi177.stdout: "cap_acquisition_throttle": 0, 2024-03-27T11:08:58.244 INFO:teuthology.orchestra.run.smithi177.stdout: "req_getvxattr_latency": { 2024-03-27T11:08:58.244 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:58.245 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:58.245 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:58.245 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-03-27T11:08:58.245 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:58.245 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_sessions": { 2024-03-27T11:08:58.245 INFO:teuthology.orchestra.run.smithi177.stdout: "session_count": 1, 2024-03-27T11:08:58.245 INFO:teuthology.orchestra.run.smithi177.stdout: "session_add": 1, 2024-03-27T11:08:58.245 INFO:teuthology.orchestra.run.smithi177.stdout: "session_remove": 0, 2024-03-27T11:08:58.245 INFO:teuthology.orchestra.run.smithi177.stdout: "sessions_open": 1, 2024-03-27T11:08:58.245 INFO:teuthology.orchestra.run.smithi177.stdout: "sessions_stale": 0, 2024-03-27T11:08:58.246 INFO:teuthology.orchestra.run.smithi177.stdout: "total_load": 0, 2024-03-27T11:08:58.246 INFO:teuthology.orchestra.run.smithi177.stdout: "average_load": 0, 2024-03-27T11:08:58.246 INFO:teuthology.orchestra.run.smithi177.stdout: "avg_session_uptime": 750, 2024-03-27T11:08:58.246 INFO:teuthology.orchestra.run.smithi177.stdout: "mdthresh_evicted": 0 2024-03-27T11:08:58.246 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:58.246 INFO:teuthology.orchestra.run.smithi177.stdout: "mempool": { 2024-03-27T11:08:58.246 INFO:teuthology.orchestra.run.smithi177.stdout: "bloom_filter_bytes": 0, 2024-03-27T11:08:58.246 INFO:teuthology.orchestra.run.smithi177.stdout: "bloom_filter_items": 0, 2024-03-27T11:08:58.246 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_alloc_bytes": 0, 2024-03-27T11:08:58.246 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_alloc_items": 0, 2024-03-27T11:08:58.246 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_data_bytes": 0, 2024-03-27T11:08:58.246 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_data_items": 0, 2024-03-27T11:08:58.247 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-27T11:08:58.247 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_onode_items": 0, 2024-03-27T11:08:58.247 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-27T11:08:58.247 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_meta_items": 0, 2024-03-27T11:08:58.247 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_other_bytes": 0, 2024-03-27T11:08:58.247 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_other_items": 0, 2024-03-27T11:08:58.247 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-27T11:08:58.247 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_buffer_items": 0, 2024-03-27T11:08:58.247 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_extent_bytes": 0, 2024-03-27T11:08:58.247 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_extent_items": 0, 2024-03-27T11:08:58.247 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_blob_bytes": 0, 2024-03-27T11:08:58.248 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_blob_items": 0, 2024-03-27T11:08:58.248 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-27T11:08:58.248 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_shared_blob_items": 0, 2024-03-27T11:08:58.248 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-27T11:08:58.248 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_inline_bl_items": 0, 2024-03-27T11:08:58.248 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_fsck_bytes": 0, 2024-03-27T11:08:58.248 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_fsck_items": 0, 2024-03-27T11:08:58.248 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_txc_bytes": 0, 2024-03-27T11:08:58.248 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_txc_items": 0, 2024-03-27T11:08:58.248 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-27T11:08:58.249 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_writing_deferred_items": 0, 2024-03-27T11:08:58.249 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_writing_bytes": 0, 2024-03-27T11:08:58.249 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_writing_items": 0, 2024-03-27T11:08:58.249 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_bytes": 0, 2024-03-27T11:08:58.249 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_items": 0, 2024-03-27T11:08:58.249 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_file_reader_bytes": 0, 2024-03-27T11:08:58.249 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_file_reader_items": 0, 2024-03-27T11:08:58.249 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_file_writer_bytes": 0, 2024-03-27T11:08:58.249 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_file_writer_items": 0, 2024-03-27T11:08:58.249 INFO:teuthology.orchestra.run.smithi177.stdout: "buffer_anon_bytes": 259663, 2024-03-27T11:08:58.250 INFO:teuthology.orchestra.run.smithi177.stdout: "buffer_anon_items": 87, 2024-03-27T11:08:58.250 INFO:teuthology.orchestra.run.smithi177.stdout: "buffer_meta_bytes": 0, 2024-03-27T11:08:58.250 INFO:teuthology.orchestra.run.smithi177.stdout: "buffer_meta_items": 0, 2024-03-27T11:08:58.250 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_bytes": 0, 2024-03-27T11:08:58.250 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_items": 0, 2024-03-27T11:08:58.250 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_mapbl_bytes": 0, 2024-03-27T11:08:58.250 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_mapbl_items": 0, 2024-03-27T11:08:58.250 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_pglog_bytes": 0, 2024-03-27T11:08:58.250 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_pglog_items": 0, 2024-03-27T11:08:58.250 INFO:teuthology.orchestra.run.smithi177.stdout: "osdmap_bytes": 7528, 2024-03-27T11:08:58.250 INFO:teuthology.orchestra.run.smithi177.stdout: "osdmap_items": 158, 2024-03-27T11:08:58.251 INFO:teuthology.orchestra.run.smithi177.stdout: "osdmap_mapping_bytes": 0, 2024-03-27T11:08:58.251 INFO:teuthology.orchestra.run.smithi177.stdout: "osdmap_mapping_items": 0, 2024-03-27T11:08:58.251 INFO:teuthology.orchestra.run.smithi177.stdout: "pgmap_bytes": 0, 2024-03-27T11:08:58.251 INFO:teuthology.orchestra.run.smithi177.stdout: "pgmap_items": 0, 2024-03-27T11:08:58.251 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_co_bytes": 80557, 2024-03-27T11:08:58.251 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_co_items": 316, 2024-03-27T11:08:58.251 INFO:teuthology.orchestra.run.smithi177.stdout: "unittest_1_bytes": 0, 2024-03-27T11:08:58.251 INFO:teuthology.orchestra.run.smithi177.stdout: "unittest_1_items": 0, 2024-03-27T11:08:58.251 INFO:teuthology.orchestra.run.smithi177.stdout: "unittest_2_bytes": 0, 2024-03-27T11:08:58.251 INFO:teuthology.orchestra.run.smithi177.stdout: "unittest_2_items": 0 2024-03-27T11:08:58.251 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:58.251 INFO:teuthology.orchestra.run.smithi177.stdout: "objecter": { 2024-03-27T11:08:58.251 INFO:teuthology.orchestra.run.smithi177.stdout: "op_active": 0, 2024-03-27T11:08:58.252 INFO:teuthology.orchestra.run.smithi177.stdout: "op_laggy": 0, 2024-03-27T11:08:58.252 INFO:teuthology.orchestra.run.smithi177.stdout: "op_send": 37, 2024-03-27T11:08:58.252 INFO:teuthology.orchestra.run.smithi177.stdout: "op_send_bytes": 11293, 2024-03-27T11:08:58.252 INFO:teuthology.orchestra.run.smithi177.stdout: "op_resend": 33, 2024-03-27T11:08:58.252 INFO:teuthology.orchestra.run.smithi177.stdout: "op_reply": 37, 2024-03-27T11:08:58.252 INFO:teuthology.orchestra.run.smithi177.stdout: "op_latency": { 2024-03-27T11:08:58.252 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 37, 2024-03-27T11:08:58.252 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.404988639, 2024-03-27T11:08:58.253 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.010945638 2024-03-27T11:08:58.253 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:58.253 INFO:teuthology.orchestra.run.smithi177.stdout: "op_inflight": 0, 2024-03-27T11:08:58.253 INFO:teuthology.orchestra.run.smithi177.stdout: "oplen_avg": { 2024-03-27T11:08:58.253 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 37, 2024-03-27T11:08:58.253 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 59 2024-03-27T11:08:58.253 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:58.253 INFO:teuthology.orchestra.run.smithi177.stdout: "op": 37, 2024-03-27T11:08:58.253 INFO:teuthology.orchestra.run.smithi177.stdout: "op_r": 0, 2024-03-27T11:08:58.253 INFO:teuthology.orchestra.run.smithi177.stdout: "op_w": 37, 2024-03-27T11:08:58.253 INFO:teuthology.orchestra.run.smithi177.stdout: "op_rmw": 0, 2024-03-27T11:08:58.254 INFO:teuthology.orchestra.run.smithi177.stdout: "op_pg": 0, 2024-03-27T11:08:58.254 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_stat": 0, 2024-03-27T11:08:58.254 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_create": 10, 2024-03-27T11:08:58.254 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_read": 0, 2024-03-27T11:08:58.254 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_write": 2, 2024-03-27T11:08:58.254 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_writefull": 6, 2024-03-27T11:08:58.254 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_writesame": 0, 2024-03-27T11:08:58.254 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_append": 0, 2024-03-27T11:08:58.254 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_zero": 0, 2024-03-27T11:08:58.254 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_truncate": 0, 2024-03-27T11:08:58.254 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_delete": 6, 2024-03-27T11:08:58.254 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_mapext": 0, 2024-03-27T11:08:58.255 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_sparse_read": 0, 2024-03-27T11:08:58.255 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_clonerange": 0, 2024-03-27T11:08:58.255 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_getxattr": 0, 2024-03-27T11:08:58.255 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_setxattr": 20, 2024-03-27T11:08:58.255 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_cmpxattr": 0, 2024-03-27T11:08:58.255 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_rmxattr": 0, 2024-03-27T11:08:58.255 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_resetxattrs": 0, 2024-03-27T11:08:58.255 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_call": 0, 2024-03-27T11:08:58.255 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_watch": 0, 2024-03-27T11:08:58.255 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_notify": 0, 2024-03-27T11:08:58.255 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_src_cmpxattr": 0, 2024-03-27T11:08:58.255 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_pgls": 0, 2024-03-27T11:08:58.256 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_pgls_filter": 0, 2024-03-27T11:08:58.256 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_other": 0, 2024-03-27T11:08:58.256 INFO:teuthology.orchestra.run.smithi177.stdout: "linger_active": 0, 2024-03-27T11:08:58.256 INFO:teuthology.orchestra.run.smithi177.stdout: "linger_send": 0, 2024-03-27T11:08:58.256 INFO:teuthology.orchestra.run.smithi177.stdout: "linger_resend": 0, 2024-03-27T11:08:58.256 INFO:teuthology.orchestra.run.smithi177.stdout: "linger_ping": 0, 2024-03-27T11:08:58.256 INFO:teuthology.orchestra.run.smithi177.stdout: "poolop_active": 0, 2024-03-27T11:08:58.256 INFO:teuthology.orchestra.run.smithi177.stdout: "poolop_send": 0, 2024-03-27T11:08:58.256 INFO:teuthology.orchestra.run.smithi177.stdout: "poolop_resend": 0, 2024-03-27T11:08:58.256 INFO:teuthology.orchestra.run.smithi177.stdout: "poolstat_active": 0, 2024-03-27T11:08:58.256 INFO:teuthology.orchestra.run.smithi177.stdout: "poolstat_send": 0, 2024-03-27T11:08:58.256 INFO:teuthology.orchestra.run.smithi177.stdout: "poolstat_resend": 0, 2024-03-27T11:08:58.257 INFO:teuthology.orchestra.run.smithi177.stdout: "statfs_active": 0, 2024-03-27T11:08:58.257 INFO:teuthology.orchestra.run.smithi177.stdout: "statfs_send": 0, 2024-03-27T11:08:58.257 INFO:teuthology.orchestra.run.smithi177.stdout: "statfs_resend": 0, 2024-03-27T11:08:58.257 INFO:teuthology.orchestra.run.smithi177.stdout: "command_active": 0, 2024-03-27T11:08:58.257 INFO:teuthology.orchestra.run.smithi177.stdout: "command_send": 0, 2024-03-27T11:08:58.257 INFO:teuthology.orchestra.run.smithi177.stdout: "command_resend": 0, 2024-03-27T11:08:58.257 INFO:teuthology.orchestra.run.smithi177.stdout: "map_epoch": 92, 2024-03-27T11:08:58.257 INFO:teuthology.orchestra.run.smithi177.stdout: "map_full": 0, 2024-03-27T11:08:58.257 INFO:teuthology.orchestra.run.smithi177.stdout: "map_inc": 2, 2024-03-27T11:08:58.257 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_sessions": 9, 2024-03-27T11:08:58.257 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_session_open": 9, 2024-03-27T11:08:58.257 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_session_close": 0, 2024-03-27T11:08:58.258 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_laggy": 0, 2024-03-27T11:08:58.258 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_wr": 14, 2024-03-27T11:08:58.258 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_rd": 0, 2024-03-27T11:08:58.258 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_del": 1 2024-03-27T11:08:58.258 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:58.258 INFO:teuthology.orchestra.run.smithi177.stdout: "oft": { 2024-03-27T11:08:58.258 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_total_objs": 1, 2024-03-27T11:08:58.258 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_total_kv_pairs": 0, 2024-03-27T11:08:58.258 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_total_updates": 0, 2024-03-27T11:08:58.258 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_total_removes": 0 2024-03-27T11:08:58.258 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:58.258 INFO:teuthology.orchestra.run.smithi177.stdout: "osd-slow-ops": { 2024-03-27T11:08:58.259 INFO:teuthology.orchestra.run.smithi177.stdout: "slow_ops_count": 0 2024-03-27T11:08:58.259 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:58.259 INFO:teuthology.orchestra.run.smithi177.stdout: "purge_queue": { 2024-03-27T11:08:58.259 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_executing_ops": 0, 2024-03-27T11:08:58.259 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_executing_ops_high_water": 0, 2024-03-27T11:08:58.259 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_executing": 0, 2024-03-27T11:08:58.259 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_executing_high_water": 0, 2024-03-27T11:08:58.259 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_executed_ops": 0, 2024-03-27T11:08:58.259 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_executed": 0, 2024-03-27T11:08:58.259 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_item_in_journal": 0 2024-03-27T11:08:58.259 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:58.259 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-27T11:08:58.260 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2024-03-27T11:08:58.260 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 104857600, 2024-03-27T11:08:58.260 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2024-03-27T11:08:58.260 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 1064, 2024-03-27T11:08:58.260 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 848998, 2024-03-27T11:08:58.260 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2024-03-27T11:08:58.260 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 1064, 2024-03-27T11:08:58.260 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 0, 2024-03-27T11:08:58.260 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 0, 2024-03-27T11:08:58.260 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 1064, 2024-03-27T11:08:58.260 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 848998, 2024-03-27T11:08:58.260 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2024-03-27T11:08:58.261 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:58.261 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:58.261 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:58.261 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-03-27T11:08:58.261 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:58.261 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-objecter_bytes": { 2024-03-27T11:08:58.261 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2024-03-27T11:08:58.261 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 104857600, 2024-03-27T11:08:58.261 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2024-03-27T11:08:58.261 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 0, 2024-03-27T11:08:58.261 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 0, 2024-03-27T11:08:58.261 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2024-03-27T11:08:58.261 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 0, 2024-03-27T11:08:58.262 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 37, 2024-03-27T11:08:58.262 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 11293, 2024-03-27T11:08:58.262 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 37, 2024-03-27T11:08:58.262 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 11293, 2024-03-27T11:08:58.262 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2024-03-27T11:08:58.262 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:58.262 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:58.262 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:58.262 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-03-27T11:08:58.262 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:58.262 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-objecter_ops": { 2024-03-27T11:08:58.262 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2024-03-27T11:08:58.263 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 1024, 2024-03-27T11:08:58.263 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2024-03-27T11:08:58.263 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 0, 2024-03-27T11:08:58.263 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 0, 2024-03-27T11:08:58.263 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2024-03-27T11:08:58.263 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 0, 2024-03-27T11:08:58.263 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 37, 2024-03-27T11:08:58.263 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 37, 2024-03-27T11:08:58.263 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 37, 2024-03-27T11:08:58.263 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 37, 2024-03-27T11:08:58.263 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2024-03-27T11:08:58.263 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:58.264 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:58.264 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:58.264 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-03-27T11:08:58.264 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:58.264 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-write_buf_throttle": { 2024-03-27T11:08:58.264 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2024-03-27T11:08:58.264 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 3758096384, 2024-03-27T11:08:58.264 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2024-03-27T11:08:58.264 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 0, 2024-03-27T11:08:58.264 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 0, 2024-03-27T11:08:58.264 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2024-03-27T11:08:58.264 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 0, 2024-03-27T11:08:58.264 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 0, 2024-03-27T11:08:58.265 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 0, 2024-03-27T11:08:58.265 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 0, 2024-03-27T11:08:58.265 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 0, 2024-03-27T11:08:58.265 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2024-03-27T11:08:58.265 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:58.265 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:58.265 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:58.265 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-03-27T11:08:58.265 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-03-27T11:08:58.265 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-write_buf_throttle-0x5561f0b58500": { 2024-03-27T11:08:58.265 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2024-03-27T11:08:58.265 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 3758096384, 2024-03-27T11:08:58.266 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2024-03-27T11:08:58.266 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 3, 2024-03-27T11:08:58.266 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 1081, 2024-03-27T11:08:58.266 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2024-03-27T11:08:58.266 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 3, 2024-03-27T11:08:58.266 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 0, 2024-03-27T11:08:58.266 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 0, 2024-03-27T11:08:58.266 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 2, 2024-03-27T11:08:58.266 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 1081, 2024-03-27T11:08:58.266 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2024-03-27T11:08:58.266 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-03-27T11:08:58.266 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-03-27T11:08:58.267 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-03-27T11:08:58.267 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-03-27T11:08:58.267 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-03-27T11:08:58.267 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-03-27T11:08:59.002 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-03-27T11:08:59.002 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-03-27T11:08:59.002 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update=0 2024-03-27T11:08:59.002 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update_receipt=11 2024-03-27T11:08:59.002 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-03-27T11:08:59.013 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-03-27T11:08:59.013 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-27T11:08:59.014 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-27T11:08:59.088 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-27T11:08:59.088 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-27T11:08:59.142 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-27T11:08:59.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:59.393+0000 7f709f000640 1 -- 172.21.15.97:0/4289290237 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7098104f80 msgr2=0x7f7098105400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:08:59.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:59.393+0000 7f709f000640 1 --2- 172.21.15.97:0/4289290237 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7098104f80 0x7f7098105400 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f7080009900 tx=0x7f708002bc30 comp rx=0 tx=0).stop 2024-03-27T11:08:59.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:59.393+0000 7f709f000640 1 -- 172.21.15.97:0/4289290237 shutdown_connections 2024-03-27T11:08:59.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:59.393+0000 7f709f000640 1 --2- 172.21.15.97:0/4289290237 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7098105940 0x7f709810c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:08:59.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:59.393+0000 7f709f000640 1 --2- 172.21.15.97:0/4289290237 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7098104f80 0x7f7098105400 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:08:59.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:59.393+0000 7f709f000640 1 --2- 172.21.15.97:0/4289290237 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7098103d90 0x7f70981041b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:08:59.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:59.393+0000 7f709f000640 1 -- 172.21.15.97:0/4289290237 >> 172.21.15.97:0/4289290237 conn(0x7f70980ff300 msgr2=0x7f7098101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:08:59.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:59.393+0000 7f709f000640 1 -- 172.21.15.97:0/4289290237 shutdown_connections 2024-03-27T11:08:59.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:59.393+0000 7f709f000640 1 -- 172.21.15.97:0/4289290237 wait complete. 2024-03-27T11:08:59.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:59.394+0000 7f709f000640 1 Processor -- start 2024-03-27T11:08:59.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:59.394+0000 7f709f000640 1 -- start start 2024-03-27T11:08:59.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:59.395+0000 7f709f000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7098103d90 0x7f7098199f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:08:59.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:59.395+0000 7f709f000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7098104f80 0x7f709819a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:08:59.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:59.395+0000 7f709f000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7098105940 0x7f70981a14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:08:59.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:59.395+0000 7f709f000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f709810f8c0 con 0x7f7098103d90 2024-03-27T11:08:59.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:59.395+0000 7f709f000640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f709810f740 con 0x7f7098104f80 2024-03-27T11:08:59.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:59.395+0000 7f709f000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f709810fa40 con 0x7f7098105940 2024-03-27T11:08:59.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:59.395+0000 7f7097e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7098103d90 0x7f7098199f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:08:59.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:59.395+0000 7f7097e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7098103d90 0x7f7098199f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40330/0 (socket says 172.21.15.97:40330) 2024-03-27T11:08:59.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:59.395+0000 7f7097e00640 1 -- 172.21.15.97:0/2587152774 learned_addr learned my addr 172.21.15.97:0/2587152774 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:08:59.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:59.395+0000 7f708fe00640 1 --2- 172.21.15.97:0/2587152774 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7098104f80 0x7f709819a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:08:59.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:59.395+0000 7f709d000640 1 --2- 172.21.15.97:0/2587152774 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7098105940 0x7f70981a14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:08:59.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:59.395+0000 7f7097e00640 1 -- 172.21.15.97:0/2587152774 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7098105940 msgr2=0x7f70981a14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:08:59.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:59.395+0000 7f7097e00640 1 --2- 172.21.15.97:0/2587152774 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7098105940 0x7f70981a14f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:08:59.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:59.395+0000 7f7097e00640 1 -- 172.21.15.97:0/2587152774 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7098104f80 msgr2=0x7f709819a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:08:59.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:59.395+0000 7f7097e00640 1 --2- 172.21.15.97:0/2587152774 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7098104f80 0x7f709819a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:08:59.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:59.395+0000 7f7097e00640 1 -- 172.21.15.97:0/2587152774 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70981a1b60 con 0x7f7098103d90 2024-03-27T11:08:59.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:59.396+0000 7f7097e00640 1 --2- 172.21.15.97:0/2587152774 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7098103d90 0x7f7098199f40 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f708800d770 tx=0x7f708800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:08:59.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:59.396+0000 7f7095600640 1 -- 172.21.15.97:0/2587152774 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7088017070 con 0x7f7098103d90 2024-03-27T11:08:59.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:59.396+0000 7f709f000640 1 -- 172.21.15.97:0/2587152774 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70981a1e50 con 0x7f7098103d90 2024-03-27T11:08:59.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:59.396+0000 7f7095600640 1 -- 172.21.15.97:0/2587152774 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7088009c30 con 0x7f7098103d90 2024-03-27T11:08:59.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:59.396+0000 7f7095600640 1 -- 172.21.15.97:0/2587152774 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7088003d00 con 0x7f7098103d90 2024-03-27T11:08:59.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:59.396+0000 7f709f000640 1 -- 172.21.15.97:0/2587152774 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70981a22c0 con 0x7f7098103d90 2024-03-27T11:08:59.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:59.397+0000 7f709f000640 1 -- 172.21.15.97:0/2587152774 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f709810b130 con 0x7f7098103d90 2024-03-27T11:08:59.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:59.399+0000 7f7095600640 1 -- 172.21.15.97:0/2587152774 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f7088003ea0 con 0x7f7098103d90 2024-03-27T11:08:59.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:59.400+0000 7f7095600640 1 --2- 172.21.15.97:0/2587152774 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f70780a3690 0x7f70780a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:08:59.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:59.401+0000 7f7095600640 1 -- 172.21.15.97:0/2587152774 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f70880cfad0 con 0x7f7098103d90 2024-03-27T11:08:59.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:59.401+0000 7f708fe00640 1 --2- 172.21.15.97:0/2587152774 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f70780a3690 0x7f70780a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:08:59.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:59.402+0000 7f708fe00640 1 --2- 172.21.15.97:0/2587152774 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f70780a3690 0x7f70780a5b50 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f709819b450 tx=0x7f7080005be0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:08:59.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:59.405+0000 7f7095600640 1 -- 172.21.15.97:0/2587152774 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f708809c650 con 0x7f7098103d90 2024-03-27T11:08:59.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:59.606+0000 7f709f000640 1 -- 172.21.15.97:0/2587152774 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7098062820 con 0x7f7098103d90 2024-03-27T11:08:59.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:59.608+0000 7f7095600640 1 -- 172.21.15.97:0/2587152774 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+22644 (secure 0 0 0) 0x7f7088009de0 con 0x7f7098103d90 2024-03-27T11:08:59.578 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T11:08:59.578 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":92,"fsid":"adca88c4-ec26-11ee-95d0-87774f69a715","created":"2024-03-27T10:43:24.680694+0000","modified":"2024-03-27T10:54:41.310812+0000","last_up_change":"2024-03-27T10:53:00.552513+0000","last_in_change":"2024-03-27T10:52:36.302986+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-27T10:47:54.052810+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-27T10:54:02.845060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-27T10:54:03.474151+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b634e5f1-6118-4e2f-95e9-830db390f8d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1746812080},{"type":"v1","addr":"172.21.15.97:6803","nonce":1746812080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1746812080},{"type":"v1","addr":"172.21.15.97:6805","nonce":1746812080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":1746812080},{"type":"v1","addr":"172.21.15.97:6809","nonce":1746812080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1746812080},{"type":"v1","addr":"172.21.15.97:6807","nonce":1746812080}]},"public_addr":"172.21.15.97:6803/1746812080","cluster_addr":"172.21.15.97:6805/1746812080","heartbeat_back_addr":"172.21.15.97:6809/1746812080","heartbeat_front_addr":"172.21.15.97:6807/1746812080","state":["exists","up"]},{"osd":1,"uuid":"441e2770-3284-4388-b0de-c56eb8a373d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":3858821264},{"type":"v1","addr":"172.21.15.114:6801","nonce":3858821264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":3858821264},{"type":"v1","addr":"172.21.15.114:6803","nonce":3858821264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":3858821264},{"type":"v1","addr":"172.21.15.114:6807","nonce":3858821264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":3858821264},{"type":"v1","addr":"172.21.15.114:6805","nonce":3858821264}]},"public_addr":"172.21.15.114:6801/3858821264","cluster_addr":"172.21.15.114:6803/3858821264","heartbeat_back_addr":"172.21.15.114:6807/3858821264","heartbeat_front_addr":"172.21.15.114:6805/3858821264","state":["exists","up"]},{"osd":2,"uuid":"00108864-1aa6-48d6-8398-e8eb79fda123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":124240658},{"type":"v1","addr":"172.21.15.177:6801","nonce":124240658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":124240658},{"type":"v1","addr":"172.21.15.177:6803","nonce":124240658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":124240658},{"type":"v1","addr":"172.21.15.177:6807","nonce":124240658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":124240658},{"type":"v1","addr":"172.21.15.177:6805","nonce":124240658}]},"public_addr":"172.21.15.177:6801/124240658","cluster_addr":"172.21.15.177:6803/124240658","heartbeat_back_addr":"172.21.15.177:6807/124240658","heartbeat_front_addr":"172.21.15.177:6805/124240658","state":["exists","up"]},{"osd":3,"uuid":"1860b690-e2dc-4977-88d7-d1aacf7022ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2478179311},{"type":"v1","addr":"172.21.15.97:6811","nonce":2478179311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2478179311},{"type":"v1","addr":"172.21.15.97:6813","nonce":2478179311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":2478179311},{"type":"v1","addr":"172.21.15.97:6817","nonce":2478179311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2478179311},{"type":"v1","addr":"172.21.15.97:6815","nonce":2478179311}]},"public_addr":"172.21.15.97:6811/2478179311","cluster_addr":"172.21.15.97:6813/2478179311","heartbeat_back_addr":"172.21.15.97:6817/2478179311","heartbeat_front_addr":"172.21.15.97:6815/2478179311","state":["exists","up"]},{"osd":4,"uuid":"9245a3a4-7d49-4fdb-9d55-112e43177050","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":75755945},{"type":"v1","addr":"172.21.15.114:6809","nonce":75755945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":75755945},{"type":"v1","addr":"172.21.15.114:6811","nonce":75755945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":75755945},{"type":"v1","addr":"172.21.15.114:6815","nonce":75755945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":75755945},{"type":"v1","addr":"172.21.15.114:6813","nonce":75755945}]},"public_addr":"172.21.15.114:6809/75755945","cluster_addr":"172.21.15.114:6811/75755945","heartbeat_back_addr":"172.21.15.114:6815/75755945","heartbeat_front_addr":"172.21.15.114:6813/75755945","state":["exists","up"]},{"osd":5,"uuid":"51922f99-e8d3-461b-8615-e9bfdcbf5ed3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":1918493790},{"type":"v1","addr":"172.21.15.177:6809","nonce":1918493790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":1918493790},{"type":"v1","addr":"172.21.15.177:6811","nonce":1918493790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":1918493790},{"type":"v1","addr":"172.21.15.177:6815","nonce":1918493790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":1918493790},{"type":"v1","addr":"172.21.15.177:6813","nonce":1918493790}]},"public_addr":"172.21.15.177:6809/1918493790","cluster_addr":"172.21.15.177:6811/1918493790","heartbeat_back_addr":"172.21.15.177:6815/1918493790","heartbeat_front_addr":"172.21.15.177:6813/1918493790","state":["exists","up"]},{"osd":6,"uuid":"06dffb32-05eb-43b3-852e-915c5cc7f25a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2892389563},{"type":"v1","addr":"172.21.15.97:6819","nonce":2892389563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2892389563},{"type":"v1","addr":"172.21.15.97:6821","nonce":2892389563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2892389563},{"type":"v1","addr":"172.21.15.97:6825","nonce":2892389563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2892389563},{"type":"v1","addr":"172.21.15.97:6823","nonce":2892389563}]},"public_addr":"172.21.15.97:6819/2892389563","cluster_addr":"172.21.15.97:6821/2892389563","heartbeat_back_addr":"172.21.15.97:6825/2892389563","heartbeat_front_addr":"172.21.15.97:6823/2892389563","state":["exists","up"]},{"osd":7,"uuid":"e68e3519-9c1b-43c5-8e06-5385248b19df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":2445491153},{"type":"v1","addr":"172.21.15.114:6817","nonce":2445491153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":2445491153},{"type":"v1","addr":"172.21.15.114:6819","nonce":2445491153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2445491153},{"type":"v1","addr":"172.21.15.114:6823","nonce":2445491153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":2445491153},{"type":"v1","addr":"172.21.15.114:6821","nonce":2445491153}]},"public_addr":"172.21.15.114:6817/2445491153","cluster_addr":"172.21.15.114:6819/2445491153","heartbeat_back_addr":"172.21.15.114:6823/2445491153","heartbeat_front_addr":"172.21.15.114:6821/2445491153","state":["exists","up"]},{"osd":8,"uuid":"31ac592e-1349-49ee-9af9-8ad85d682978","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":2051134191},{"type":"v1","addr":"172.21.15.177:6817","nonce":2051134191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":2051134191},{"type":"v1","addr":"172.21.15.177:6819","nonce":2051134191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":2051134191},{"type":"v1","addr":"172.21.15.177:6823","nonce":2051134191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":2051134191},{"type":"v1","addr":"172.21.15.177:6821","nonce":2051134191}]},"public_addr":"172.21.15.177:6817/2051134191","cluster_addr":"172.21.15.177:6819/2051134191","heartbeat_back_addr":"172.21.15.177:6823/2051134191","heartbeat_front_addr":"172.21.15.177:6821/2051134191","state":["exists","up"]},{"osd":9,"uuid":"0171086a-dd41-4eb4-a389-16f0c46a64db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":1578881271},{"type":"v1","addr":"172.21.15.97:6827","nonce":1578881271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1578881271},{"type":"v1","addr":"172.21.15.97:6829","nonce":1578881271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1578881271},{"type":"v1","addr":"172.21.15.97:6833","nonce":1578881271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1578881271},{"type":"v1","addr":"172.21.15.97:6831","nonce":1578881271}]},"public_addr":"172.21.15.97:6827/1578881271","cluster_addr":"172.21.15.97:6829/1578881271","heartbeat_back_addr":"172.21.15.97:6833/1578881271","heartbeat_front_addr":"172.21.15.97:6831/1578881271","state":["exists","up"]},{"osd":10,"uuid":"64754e31-b124-4601-ad98-2f70b1dd5359","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":2893111461},{"type":"v1","addr":"172.21.15.114:6825","nonce":2893111461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":2893111461},{"type":"v1","addr":"172.21.15.114:6827","nonce":2893111461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2893111461},{"type":"v1","addr":"172.21.15.114:6831","nonce":2893111461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":2893111461},{"type":"v1","addr":"172.21.15.114:6829","nonce":2893111461}]},"public_addr":"172.21.15.114:6825/2893111461","cluster_addr":"172.21.15.114:6827/2893111461","heartbeat_back_addr":"172.21.15.114:6831/2893111461","heartbeat_front_addr":"172.21.15.114:6829/2893111461","state":["exists","up"]},{"osd":11,"uuid":"9ffc752c-adf7-4037-89d3-535787d7475d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":85,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":2333030391},{"type":"v1","addr":"172.21.15.177:6825","nonce":2333030391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":2333030391},{"type":"v1","addr":"172.21.15.177:6827","nonce":2333030391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":2333030391},{"type":"v1","addr":"172.21.15.177:6831","nonce":2333030391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":2333030391},{"type":"v1","addr":"172.21.15.177:6829","nonce":2333030391}]},"public_addr":"172.21.15.177:6825/2333030391","cluster_addr":"172.21.15.177:6827/2333030391","heartbeat_back_addr":"172.21.15.177:6831/2333030391","heartbeat_front_addr":"172.21.15.177:6829/2333030391","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:46:45.143519+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:47:18.656947+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:47:51.316345+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:48:24.575512+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:48:59.113555+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:49:32.103865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:50:06.615539+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:50:41.578341+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:51:15.254011+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:51:49.099165+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:52:24.544821+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:52:58.838484+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.28","mappings":[{"from":3,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/359646":"2024-03-28T10:44:39.889550+0000","172.21.15.97:6801/98682155":"2024-03-28T10:44:06.019075+0000","172.21.15.97:6801/1685921853":"2024-03-28T10:44:39.889550+0000","172.21.15.97:6800/98682155":"2024-03-28T10:44:06.019075+0000","172.21.15.97:0/107690680":"2024-03-28T10:44:06.019075+0000","172.21.15.97:6800/1685921853":"2024-03-28T10:44:39.889550+0000","172.21.15.97:0/437043413":"2024-03-28T10:44:39.889550+0000","172.21.15.97:0/2319706992":"2024-03-28T10:44:39.889550+0000","172.21.15.97:0/3911532654":"2024-03-28T10:44:06.019075+0000","172.21.15.97:0/3347321845":"2024-03-28T10:44:06.019075+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-27T11:08:59.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:59.610+0000 7f709f000640 1 -- 172.21.15.97:0/2587152774 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f70780a3690 msgr2=0x7f70780a5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:08:59.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:59.610+0000 7f709f000640 1 --2- 172.21.15.97:0/2587152774 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f70780a3690 0x7f70780a5b50 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f709819b450 tx=0x7f7080005be0 comp rx=0 tx=0).stop 2024-03-27T11:08:59.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:59.610+0000 7f709f000640 1 -- 172.21.15.97:0/2587152774 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7098103d90 msgr2=0x7f7098199f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:08:59.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:59.611+0000 7f709f000640 1 --2- 172.21.15.97:0/2587152774 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7098103d90 0x7f7098199f40 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f708800d770 tx=0x7f708800dc30 comp rx=0 tx=0).stop 2024-03-27T11:08:59.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:59.611+0000 7f709f000640 1 -- 172.21.15.97:0/2587152774 shutdown_connections 2024-03-27T11:08:59.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:59.611+0000 7f709f000640 1 --2- 172.21.15.97:0/2587152774 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7098105940 0x7f70981a14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:08:59.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:59.611+0000 7f709f000640 1 --2- 172.21.15.97:0/2587152774 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f70780a3690 0x7f70780a5b50 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:08:59.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:59.611+0000 7f709f000640 1 --2- 172.21.15.97:0/2587152774 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7098104f80 0x7f709819a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:08:59.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:59.611+0000 7f709f000640 1 --2- 172.21.15.97:0/2587152774 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7098103d90 0x7f7098199f40 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:08:59.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:59.611+0000 7f709f000640 1 -- 172.21.15.97:0/2587152774 >> 172.21.15.97:0/2587152774 conn(0x7f70980ff300 msgr2=0x7f7098108300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:08:59.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:59.611+0000 7f709f000640 1 -- 172.21.15.97:0/2587152774 shutdown_connections 2024-03-27T11:08:59.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:59.611+0000 7f709f000640 1 -- 172.21.15.97:0/2587152774 wait complete. 2024-03-27T11:08:59.600 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-27T10:47:54.052810+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-27T10:54:02.845060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-27T10:54:03.474151+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-27T11:08:59.601 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-27T11:08:59.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:59.836+0000 7fae35000640 1 -- 172.21.15.97:0/2707854487 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae30100440 msgr2=0x7fae30100860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:08:59.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:59.836+0000 7fae35000640 1 --2- 172.21.15.97:0/2707854487 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae30100440 0x7fae30100860 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fae20009900 tx=0x7fae2002bcd0 comp rx=0 tx=0).stop 2024-03-27T11:08:59.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:59.837+0000 7fae35000640 1 -- 172.21.15.97:0/2707854487 shutdown_connections 2024-03-27T11:08:59.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:59.837+0000 7fae35000640 1 --2- 172.21.15.97:0/2707854487 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fae30101740 0x7fae30101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:08:59.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:59.837+0000 7fae35000640 1 --2- 172.21.15.97:0/2707854487 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fae30100da0 0x7fae30101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:08:59.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:59.837+0000 7fae35000640 1 --2- 172.21.15.97:0/2707854487 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae30100440 0x7fae30100860 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:08:59.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:59.837+0000 7fae35000640 1 -- 172.21.15.97:0/2707854487 >> 172.21.15.97:0/2707854487 conn(0x7fae300fbfe0 msgr2=0x7fae300fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:08:59.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:59.837+0000 7fae35000640 1 -- 172.21.15.97:0/2707854487 shutdown_connections 2024-03-27T11:08:59.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:59.837+0000 7fae35000640 1 -- 172.21.15.97:0/2707854487 wait complete. 2024-03-27T11:08:59.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:59.838+0000 7fae35000640 1 Processor -- start 2024-03-27T11:08:59.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:59.838+0000 7fae35000640 1 -- start start 2024-03-27T11:08:59.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:59.838+0000 7fae35000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fae30100440 0x7fae30199fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:08:59.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:59.838+0000 7fae35000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae30100da0 0x7fae3019a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:08:59.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:59.838+0000 7fae35000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fae30101740 0x7fae301a1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:08:59.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:59.838+0000 7fae35000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fae3010f880 con 0x7fae30100da0 2024-03-27T11:08:59.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:59.838+0000 7fae35000640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fae3010f700 con 0x7fae30101740 2024-03-27T11:08:59.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:59.838+0000 7fae35000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fae3010fa00 con 0x7fae30100440 2024-03-27T11:08:59.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:59.838+0000 7fae2d400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae30100da0 0x7fae3019a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:08:59.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:59.838+0000 7fae2d400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae30100da0 0x7fae3019a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40362/0 (socket says 172.21.15.97:40362) 2024-03-27T11:08:59.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:59.838+0000 7fae2d400640 1 -- 172.21.15.97:0/1279236196 learned_addr learned my addr 172.21.15.97:0/1279236196 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:08:59.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:59.839+0000 7fae2e800640 1 --2- 172.21.15.97:0/1279236196 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fae30101740 0x7fae301a1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:08:59.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:59.839+0000 7fae2de00640 1 --2- 172.21.15.97:0/1279236196 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fae30100440 0x7fae30199fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:08:59.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:59.839+0000 7fae2d400640 1 -- 172.21.15.97:0/1279236196 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fae30100440 msgr2=0x7fae30199fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:08:59.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:59.839+0000 7fae2d400640 1 --2- 172.21.15.97:0/1279236196 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fae30100440 0x7fae30199fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:08:59.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:59.839+0000 7fae2d400640 1 -- 172.21.15.97:0/1279236196 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fae30101740 msgr2=0x7fae301a1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:08:59.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:59.839+0000 7fae2d400640 1 --2- 172.21.15.97:0/1279236196 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fae30101740 0x7fae301a1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:08:59.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:59.839+0000 7fae2d400640 1 -- 172.21.15.97:0/1279236196 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae301a1c00 con 0x7fae30100da0 2024-03-27T11:08:59.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:59.839+0000 7fae2e800640 1 --2- 172.21.15.97:0/1279236196 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fae30101740 0x7fae301a1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T11:08:59.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:59.839+0000 7fae2de00640 1 --2- 172.21.15.97:0/1279236196 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fae30100440 0x7fae30199fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T11:08:59.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:59.839+0000 7fae2d400640 1 --2- 172.21.15.97:0/1279236196 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae30100da0 0x7fae3019a520 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fae1c00d770 tx=0x7fae1c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:08:59.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:59.839+0000 7fae26a00640 1 -- 172.21.15.97:0/1279236196 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae1c017070 con 0x7fae30100da0 2024-03-27T11:08:59.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:59.839+0000 7fae26a00640 1 -- 172.21.15.97:0/1279236196 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fae1c0054d0 con 0x7fae30100da0 2024-03-27T11:08:59.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:59.839+0000 7fae35000640 1 -- 172.21.15.97:0/1279236196 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae301a1ef0 con 0x7fae30100da0 2024-03-27T11:08:59.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:59.840+0000 7fae26a00640 1 -- 172.21.15.97:0/1279236196 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae1c004cf0 con 0x7fae30100da0 2024-03-27T11:08:59.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:59.840+0000 7fae35000640 1 -- 172.21.15.97:0/1279236196 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae301a2360 con 0x7fae30100da0 2024-03-27T11:08:59.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:59.841+0000 7fae35000640 1 -- 172.21.15.97:0/1279236196 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae3010b470 con 0x7fae30100da0 2024-03-27T11:08:59.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:59.843+0000 7fae26a00640 1 -- 172.21.15.97:0/1279236196 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7fae1c004e90 con 0x7fae30100da0 2024-03-27T11:08:59.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:59.844+0000 7fae26a00640 1 --2- 172.21.15.97:0/1279236196 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fae140a3610 0x7fae140a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:08:59.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:59.844+0000 7fae26a00640 1 -- 172.21.15.97:0/1279236196 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fae1c0cf860 con 0x7fae30100da0 2024-03-27T11:08:59.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:59.844+0000 7fae2de00640 1 --2- 172.21.15.97:0/1279236196 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fae140a3610 0x7fae140a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:08:59.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:59.845+0000 7fae2de00640 1 --2- 172.21.15.97:0/1279236196 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fae140a3610 0x7fae140a5ad0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fae20006010 tx=0x7fae2002c880 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:08:59.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:08:59.848+0000 7fae26a00640 1 -- 172.21.15.97:0/1279236196 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7fae1c09c3e0 con 0x7fae30100da0 2024-03-27T11:08:59.887 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:08:59 smithi097 ceph-mon[31859]: pgmap v811: 97 pgs: 97 active+clean; 1.6 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-27T11:08:59.888 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:08:59 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/2587152774' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T11:08:59.923 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:08:59 smithi177 ceph-mon[35294]: pgmap v811: 97 pgs: 97 active+clean; 1.6 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-27T11:08:59.924 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:08:59 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/2587152774' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T11:09:00.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.037+0000 7fae35000640 1 -- 172.21.15.97:0/1279236196 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fae300627f0 con 0x7fae30100da0 2024-03-27T11:09:00.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.038+0000 7fae26a00640 1 -- 172.21.15.97:0/1279236196 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) ==== 93+0+10 (secure 0 0 0) 0x7fae1c09bce0 con 0x7fae30100da0 2024-03-27T11:09:00.007 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-03-27T11:09:00.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.040+0000 7fae35000640 1 -- 172.21.15.97:0/1279236196 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fae140a3610 msgr2=0x7fae140a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:09:00.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.040+0000 7fae35000640 1 --2- 172.21.15.97:0/1279236196 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fae140a3610 0x7fae140a5ad0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fae20006010 tx=0x7fae2002c880 comp rx=0 tx=0).stop 2024-03-27T11:09:00.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.040+0000 7fae35000640 1 -- 172.21.15.97:0/1279236196 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae30100da0 msgr2=0x7fae3019a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:09:00.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.040+0000 7fae35000640 1 --2- 172.21.15.97:0/1279236196 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae30100da0 0x7fae3019a520 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fae1c00d770 tx=0x7fae1c00dc30 comp rx=0 tx=0).stop 2024-03-27T11:09:00.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.040+0000 7fae35000640 1 -- 172.21.15.97:0/1279236196 shutdown_connections 2024-03-27T11:09:00.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.040+0000 7fae35000640 1 --2- 172.21.15.97:0/1279236196 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fae140a3610 0x7fae140a5ad0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:09:00.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.040+0000 7fae35000640 1 --2- 172.21.15.97:0/1279236196 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fae30101740 0x7fae301a1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:09:00.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.040+0000 7fae35000640 1 --2- 172.21.15.97:0/1279236196 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae30100da0 0x7fae3019a520 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:09:00.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.040+0000 7fae35000640 1 --2- 172.21.15.97:0/1279236196 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fae30100440 0x7fae30199fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:09:00.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.040+0000 7fae35000640 1 -- 172.21.15.97:0/1279236196 >> 172.21.15.97:0/1279236196 conn(0x7fae300fbfe0 msgr2=0x7fae300fd7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:09:00.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.040+0000 7fae35000640 1 -- 172.21.15.97:0/1279236196 shutdown_connections 2024-03-27T11:09:00.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.040+0000 7fae35000640 1 -- 172.21.15.97:0/1279236196 wait complete. 2024-03-27T11:09:00.013 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:08:59 smithi114 ceph-mon[35338]: pgmap v811: 97 pgs: 97 active+clean; 1.6 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-27T11:09:00.013 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:08:59 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/2587152774' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T11:09:00.028 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-27T11:09:00.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.269+0000 7f1b87c00640 1 -- 172.21.15.97:0/1889322955 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b80103d90 msgr2=0x7f1b801041b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:09:00.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.269+0000 7f1b87c00640 1 --2- 172.21.15.97:0/1889322955 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b80103d90 0x7f1b801041b0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f1b70009ca0 tx=0x7f1b7002c0f0 comp rx=0 tx=0).stop 2024-03-27T11:09:00.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.269+0000 7f1b87c00640 1 -- 172.21.15.97:0/1889322955 shutdown_connections 2024-03-27T11:09:00.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.269+0000 7f1b87c00640 1 --2- 172.21.15.97:0/1889322955 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1b80105940 0x7f1b8010c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:09:00.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.269+0000 7f1b87c00640 1 --2- 172.21.15.97:0/1889322955 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1b80104f80 0x7f1b80105400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:09:00.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.269+0000 7f1b87c00640 1 --2- 172.21.15.97:0/1889322955 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b80103d90 0x7f1b801041b0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:09:00.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.269+0000 7f1b87c00640 1 -- 172.21.15.97:0/1889322955 >> 172.21.15.97:0/1889322955 conn(0x7f1b800ff300 msgr2=0x7f1b80101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:09:00.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.269+0000 7f1b87c00640 1 -- 172.21.15.97:0/1889322955 shutdown_connections 2024-03-27T11:09:00.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.269+0000 7f1b87c00640 1 -- 172.21.15.97:0/1889322955 wait complete. 2024-03-27T11:09:00.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.270+0000 7f1b87c00640 1 Processor -- start 2024-03-27T11:09:00.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.270+0000 7f1b87c00640 1 -- start start 2024-03-27T11:09:00.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.270+0000 7f1b87c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1b80103d90 0x7f1b8019a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:09:00.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.270+0000 7f1b87c00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1b80104f80 0x7f1b8019a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:09:00.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.270+0000 7f1b87c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b80105940 0x7f1b801a15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:09:00.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.270+0000 7f1b87c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1b8010f8c0 con 0x7f1b80105940 2024-03-27T11:09:00.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.270+0000 7f1b87c00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f1b8010f740 con 0x7f1b80103d90 2024-03-27T11:09:00.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.270+0000 7f1b87c00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1b8010fa40 con 0x7f1b80104f80 2024-03-27T11:09:00.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.270+0000 7f1b85c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b80105940 0x7f1b801a15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:09:00.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.270+0000 7f1b85c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b80105940 0x7f1b801a15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40382/0 (socket says 172.21.15.97:40382) 2024-03-27T11:09:00.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.270+0000 7f1b85c00640 1 -- 172.21.15.97:0/1833972529 learned_addr learned my addr 172.21.15.97:0/1833972529 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:09:00.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.271+0000 7f1b77e00640 1 --2- 172.21.15.97:0/1833972529 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1b80104f80 0x7f1b8019a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:09:00.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.271+0000 7f1b85200640 1 --2- 172.21.15.97:0/1833972529 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1b80103d90 0x7f1b8019a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:09:00.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.271+0000 7f1b85c00640 1 -- 172.21.15.97:0/1833972529 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1b80104f80 msgr2=0x7f1b8019a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:09:00.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.271+0000 7f1b85c00640 1 --2- 172.21.15.97:0/1833972529 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1b80104f80 0x7f1b8019a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:09:00.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.271+0000 7f1b85c00640 1 -- 172.21.15.97:0/1833972529 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1b80103d90 msgr2=0x7f1b8019a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:09:00.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.271+0000 7f1b85c00640 1 --2- 172.21.15.97:0/1833972529 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1b80103d90 0x7f1b8019a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:09:00.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.271+0000 7f1b85c00640 1 -- 172.21.15.97:0/1833972529 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b801a1c20 con 0x7f1b80105940 2024-03-27T11:09:00.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.271+0000 7f1b85c00640 1 --2- 172.21.15.97:0/1833972529 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b80105940 0x7f1b801a15b0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f1b7800e970 tx=0x7f1b7800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:09:00.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.271+0000 7f1b7e000640 1 -- 172.21.15.97:0/1833972529 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b7800ccc0 con 0x7f1b80105940 2024-03-27T11:09:00.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.271+0000 7f1b7e000640 1 -- 172.21.15.97:0/1833972529 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1b7800ce60 con 0x7f1b80105940 2024-03-27T11:09:00.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.271+0000 7f1b87c00640 1 -- 172.21.15.97:0/1833972529 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b801a1f10 con 0x7f1b80105940 2024-03-27T11:09:00.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.271+0000 7f1b7e000640 1 -- 172.21.15.97:0/1833972529 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b78010430 con 0x7f1b80105940 2024-03-27T11:09:00.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.272+0000 7f1b87c00640 1 -- 172.21.15.97:0/1833972529 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b801a23d0 con 0x7f1b80105940 2024-03-27T11:09:00.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.273+0000 7f1b87c00640 1 -- 172.21.15.97:0/1833972529 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b8010b1d0 con 0x7f1b80105940 2024-03-27T11:09:00.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.275+0000 7f1b7e000640 1 -- 172.21.15.97:0/1833972529 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f1b78004f70 con 0x7f1b80105940 2024-03-27T11:09:00.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.276+0000 7f1b7e000640 1 --2- 172.21.15.97:0/1833972529 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f1b640a3610 0x7f1b640a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:09:00.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.276+0000 7f1b7e000640 1 -- 172.21.15.97:0/1833972529 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f1b780d04a0 con 0x7f1b80105940 2024-03-27T11:09:00.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.276+0000 7f1b85200640 1 --2- 172.21.15.97:0/1833972529 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f1b640a3610 0x7f1b640a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:09:00.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.277+0000 7f1b85200640 1 --2- 172.21.15.97:0/1833972529 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f1b640a3610 0x7f1b640a5ad0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f1b7002c630 tx=0x7f1b70000f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:09:00.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.278+0000 7f1b7e000640 1 -- 172.21.15.97:0/1833972529 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f1b78053cb0 con 0x7f1b80105940 2024-03-27T11:09:00.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.480+0000 7f1b87c00640 1 -- 172.21.15.97:0/1833972529 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1b8002d5c0 con 0x7f1b80105940 2024-03-27T11:09:00.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.480+0000 7f1b7e000640 1 -- 172.21.15.97:0/1833972529 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) ==== 104+0+11 (secure 0 0 0) 0x7f1b7809d020 con 0x7f1b80105940 2024-03-27T11:09:00.451 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-03-27T11:09:00.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.483+0000 7f1b87c00640 1 -- 172.21.15.97:0/1833972529 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f1b640a3610 msgr2=0x7f1b640a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:09:00.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.483+0000 7f1b87c00640 1 --2- 172.21.15.97:0/1833972529 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f1b640a3610 0x7f1b640a5ad0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f1b7002c630 tx=0x7f1b70000f40 comp rx=0 tx=0).stop 2024-03-27T11:09:00.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.483+0000 7f1b87c00640 1 -- 172.21.15.97:0/1833972529 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b80105940 msgr2=0x7f1b801a15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:09:00.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.483+0000 7f1b87c00640 1 --2- 172.21.15.97:0/1833972529 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b80105940 0x7f1b801a15b0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f1b7800e970 tx=0x7f1b7800ed70 comp rx=0 tx=0).stop 2024-03-27T11:09:00.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.483+0000 7f1b87c00640 1 -- 172.21.15.97:0/1833972529 shutdown_connections 2024-03-27T11:09:00.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.483+0000 7f1b87c00640 1 --2- 172.21.15.97:0/1833972529 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b80105940 0x7f1b801a15b0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:09:00.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.483+0000 7f1b87c00640 1 --2- 172.21.15.97:0/1833972529 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1b80104f80 0x7f1b8019a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:09:00.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.483+0000 7f1b87c00640 1 --2- 172.21.15.97:0/1833972529 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f1b640a3610 0x7f1b640a5ad0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:09:00.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.483+0000 7f1b87c00640 1 --2- 172.21.15.97:0/1833972529 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1b80103d90 0x7f1b8019a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:09:00.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.483+0000 7f1b87c00640 1 -- 172.21.15.97:0/1833972529 >> 172.21.15.97:0/1833972529 conn(0x7f1b800ff300 msgr2=0x7f1b801083a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:09:00.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.483+0000 7f1b87c00640 1 -- 172.21.15.97:0/1833972529 shutdown_connections 2024-03-27T11:09:00.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.483+0000 7f1b87c00640 1 -- 172.21.15.97:0/1833972529 wait complete. 2024-03-27T11:09:00.471 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-27T11:09:00.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.696+0000 7f457dc00640 1 -- 172.21.15.97:0/2659918919 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4578075b90 msgr2=0x7f4578073fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:09:00.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.696+0000 7f457dc00640 1 --2- 172.21.15.97:0/2659918919 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4578075b90 0x7f4578073fe0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f4568009900 tx=0x7f456802bc30 comp rx=0 tx=0).stop 2024-03-27T11:09:00.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.696+0000 7f457dc00640 1 -- 172.21.15.97:0/2659918919 shutdown_connections 2024-03-27T11:09:00.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.696+0000 7f457dc00640 1 --2- 172.21.15.97:0/2659918919 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4578109910 0x7f457810bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:09:00.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.696+0000 7f457dc00640 1 --2- 172.21.15.97:0/2659918919 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f45780745b0 0x7f4578074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:09:00.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.696+0000 7f457dc00640 1 --2- 172.21.15.97:0/2659918919 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4578075b90 0x7f4578073fe0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:09:00.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.696+0000 7f457dc00640 1 -- 172.21.15.97:0/2659918919 >> 172.21.15.97:0/2659918919 conn(0x7f45780fcfc0 msgr2=0x7f45780ff400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:09:00.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.697+0000 7f457dc00640 1 -- 172.21.15.97:0/2659918919 shutdown_connections 2024-03-27T11:09:00.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.697+0000 7f457dc00640 1 -- 172.21.15.97:0/2659918919 wait complete. 2024-03-27T11:09:00.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.698+0000 7f457dc00640 1 Processor -- start 2024-03-27T11:09:00.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.698+0000 7f457dc00640 1 -- start start 2024-03-27T11:09:00.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.698+0000 7f457dc00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f45780745b0 0x7f4578199eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:09:00.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.698+0000 7f457dc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4578075b90 0x7f457819a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:09:00.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.698+0000 7f457dc00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4578109910 0x7f45781a1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:09:00.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.698+0000 7f457dc00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f457810f790 con 0x7f4578075b90 2024-03-27T11:09:00.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.698+0000 7f457dc00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f457810f610 con 0x7f45780745b0 2024-03-27T11:09:00.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.698+0000 7f457dc00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f457810f910 con 0x7f4578109910 2024-03-27T11:09:00.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.698+0000 7f456fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4578075b90 0x7f457819a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:09:00.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.698+0000 7f4577e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f45780745b0 0x7f4578199eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:09:00.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.698+0000 7f456fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4578075b90 0x7f457819a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40414/0 (socket says 172.21.15.97:40414) 2024-03-27T11:09:00.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.698+0000 7f4577e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f45780745b0 0x7f4578199eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:37944/0 (socket says 172.21.15.97:37944) 2024-03-27T11:09:00.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.698+0000 7f456fe00640 1 -- 172.21.15.97:0/3252196992 learned_addr learned my addr 172.21.15.97:0/3252196992 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:09:00.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.698+0000 7f457d200640 1 --2- 172.21.15.97:0/3252196992 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4578109910 0x7f45781a1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:09:00.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.699+0000 7f456fe00640 1 -- 172.21.15.97:0/3252196992 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4578109910 msgr2=0x7f45781a1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:09:00.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.699+0000 7f456fe00640 1 --2- 172.21.15.97:0/3252196992 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4578109910 0x7f45781a1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:09:00.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.699+0000 7f456fe00640 1 -- 172.21.15.97:0/3252196992 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f45780745b0 msgr2=0x7f4578199eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:09:00.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.699+0000 7f456fe00640 1 --2- 172.21.15.97:0/3252196992 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f45780745b0 0x7f4578199eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:09:00.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.699+0000 7f456fe00640 1 -- 172.21.15.97:0/3252196992 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45781a1a80 con 0x7f4578075b90 2024-03-27T11:09:00.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.699+0000 7f4577e00640 1 --2- 172.21.15.97:0/3252196992 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f45780745b0 0x7f4578199eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T11:09:00.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.699+0000 7f456fe00640 1 --2- 172.21.15.97:0/3252196992 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4578075b90 0x7f457819a3f0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f456000d950 tx=0x7f456000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:09:00.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.699+0000 7f4575600640 1 -- 172.21.15.97:0/3252196992 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4560017030 con 0x7f4578075b90 2024-03-27T11:09:00.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.699+0000 7f4575600640 1 -- 172.21.15.97:0/3252196992 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4560004020 con 0x7f4578075b90 2024-03-27T11:09:00.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.699+0000 7f457dc00640 1 -- 172.21.15.97:0/3252196992 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45781a1d70 con 0x7f4578075b90 2024-03-27T11:09:00.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.699+0000 7f4575600640 1 -- 172.21.15.97:0/3252196992 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45600047a0 con 0x7f4578075b90 2024-03-27T11:09:00.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.700+0000 7f457dc00640 1 -- 172.21.15.97:0/3252196992 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45781a2260 con 0x7f4578075b90 2024-03-27T11:09:00.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.701+0000 7f457dc00640 1 -- 172.21.15.97:0/3252196992 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4578108640 con 0x7f4578075b90 2024-03-27T11:09:00.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.703+0000 7f4575600640 1 -- 172.21.15.97:0/3252196992 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f45600041d0 con 0x7f4578075b90 2024-03-27T11:09:00.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.704+0000 7f4575600640 1 --2- 172.21.15.97:0/3252196992 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f455c0a3610 0x7f455c0a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:09:00.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.705+0000 7f4575600640 1 -- 172.21.15.97:0/3252196992 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f45600cf780 con 0x7f4578075b90 2024-03-27T11:09:00.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.705+0000 7f4577e00640 1 --2- 172.21.15.97:0/3252196992 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f455c0a3610 0x7f455c0a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:09:00.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.705+0000 7f4577e00640 1 --2- 172.21.15.97:0/3252196992 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f455c0a3610 0x7f455c0a5ad0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f4568000c00 tx=0x7f456802c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:09:00.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.714+0000 7f4575600640 1 -- 172.21.15.97:0/3252196992 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f456009c300 con 0x7f4578075b90 2024-03-27T11:09:00.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.912+0000 7f457dc00640 1 -- 172.21.15.97:0/3252196992 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f4578062750 con 0x7f4578075b90 2024-03-27T11:09:00.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.913+0000 7f4575600640 1 -- 172.21.15.97:0/3252196992 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) ==== 100+0+11 (secure 0 0 0) 0x7f456009bc00 con 0x7f4578075b90 2024-03-27T11:09:00.883 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-03-27T11:09:00.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.916+0000 7f457dc00640 1 -- 172.21.15.97:0/3252196992 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f455c0a3610 msgr2=0x7f455c0a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:09:00.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.916+0000 7f457dc00640 1 --2- 172.21.15.97:0/3252196992 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f455c0a3610 0x7f455c0a5ad0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f4568000c00 tx=0x7f456802c7e0 comp rx=0 tx=0).stop 2024-03-27T11:09:00.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.916+0000 7f457dc00640 1 -- 172.21.15.97:0/3252196992 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4578075b90 msgr2=0x7f457819a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:09:00.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.916+0000 7f457dc00640 1 --2- 172.21.15.97:0/3252196992 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4578075b90 0x7f457819a3f0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f456000d950 tx=0x7f456000de10 comp rx=0 tx=0).stop 2024-03-27T11:09:00.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.916+0000 7f457dc00640 1 -- 172.21.15.97:0/3252196992 shutdown_connections 2024-03-27T11:09:00.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.916+0000 7f457dc00640 1 --2- 172.21.15.97:0/3252196992 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4578109910 0x7f45781a1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:09:00.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.916+0000 7f457dc00640 1 --2- 172.21.15.97:0/3252196992 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4578075b90 0x7f457819a3f0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:09:00.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.916+0000 7f457dc00640 1 --2- 172.21.15.97:0/3252196992 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f455c0a3610 0x7f455c0a5ad0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:09:00.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.916+0000 7f457dc00640 1 --2- 172.21.15.97:0/3252196992 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f45780745b0 0x7f4578199eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:09:00.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.916+0000 7f457dc00640 1 -- 172.21.15.97:0/3252196992 >> 172.21.15.97:0/3252196992 conn(0x7f45780fcfc0 msgr2=0x7f4578105810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:09:00.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.916+0000 7f457dc00640 1 -- 172.21.15.97:0/3252196992 shutdown_connections 2024-03-27T11:09:00.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:00.916+0000 7f457dc00640 1 -- 172.21.15.97:0/3252196992 wait complete. 2024-03-27T11:09:00.887 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:09:00 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/1279236196' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-27T11:09:00.887 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:09:00 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/1833972529' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-27T11:09:00.888 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:09:00 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T11:09:00.903 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-27T11:09:00.923 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:09:00 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/1279236196' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-27T11:09:00.923 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:09:00 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/1833972529' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-27T11:09:00.923 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:09:00 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T11:09:01.013 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:09:00 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/1279236196' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-27T11:09:01.014 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:09:00 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/1833972529' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-27T11:09:01.014 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:09:00 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T11:09:01.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.126+0000 7f2116c00640 1 -- 172.21.15.97:0/1400528728 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2110105940 msgr2=0x7f211010c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:09:01.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.126+0000 7f2116c00640 1 --2- 172.21.15.97:0/1400528728 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2110105940 0x7f211010c1d0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f2108009ca0 tx=0x7f210802c100 comp rx=0 tx=0).stop 2024-03-27T11:09:01.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.126+0000 7f2116c00640 1 -- 172.21.15.97:0/1400528728 shutdown_connections 2024-03-27T11:09:01.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.126+0000 7f2116c00640 1 --2- 172.21.15.97:0/1400528728 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2110105940 0x7f211010c1d0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:09:01.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.126+0000 7f2116c00640 1 --2- 172.21.15.97:0/1400528728 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2110104f80 0x7f2110105400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:09:01.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.126+0000 7f2116c00640 1 --2- 172.21.15.97:0/1400528728 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2110103d90 0x7f21101041b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:09:01.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.126+0000 7f2116c00640 1 -- 172.21.15.97:0/1400528728 >> 172.21.15.97:0/1400528728 conn(0x7f21100ff300 msgr2=0x7f2110101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:09:01.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.126+0000 7f2116c00640 1 -- 172.21.15.97:0/1400528728 shutdown_connections 2024-03-27T11:09:01.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.126+0000 7f2116c00640 1 -- 172.21.15.97:0/1400528728 wait complete. 2024-03-27T11:09:01.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.127+0000 7f2116c00640 1 Processor -- start 2024-03-27T11:09:01.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.127+0000 7f2116c00640 1 -- start start 2024-03-27T11:09:01.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.127+0000 7f2116c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2110103d90 0x7f2110199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:09:01.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.127+0000 7f2116c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2110104f80 0x7f211019a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:09:01.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.127+0000 7f2116c00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2110105940 0x7f21101a1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:09:01.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.127+0000 7f2116c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f211010f8c0 con 0x7f2110103d90 2024-03-27T11:09:01.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.127+0000 7f2116c00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f211010f740 con 0x7f2110104f80 2024-03-27T11:09:01.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.127+0000 7f2116c00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f211010fa40 con 0x7f2110105940 2024-03-27T11:09:01.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.128+0000 7f210fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2110103d90 0x7f2110199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:09:01.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.128+0000 7f210fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2110103d90 0x7f2110199f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40436/0 (socket says 172.21.15.97:40436) 2024-03-27T11:09:01.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.128+0000 7f210fe00640 1 -- 172.21.15.97:0/2641851836 learned_addr learned my addr 172.21.15.97:0/2641851836 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:09:01.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.128+0000 7f2114c00640 1 --2- 172.21.15.97:0/2641851836 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2110105940 0x7f21101a1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:09:01.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.128+0000 7f210f400640 1 --2- 172.21.15.97:0/2641851836 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2110104f80 0x7f211019a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:09:01.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.128+0000 7f210fe00640 1 -- 172.21.15.97:0/2641851836 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2110105940 msgr2=0x7f21101a1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:09:01.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.128+0000 7f210fe00640 1 --2- 172.21.15.97:0/2641851836 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2110105940 0x7f21101a1500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:09:01.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.128+0000 7f210fe00640 1 -- 172.21.15.97:0/2641851836 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2110104f80 msgr2=0x7f211019a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:09:01.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.128+0000 7f210fe00640 1 --2- 172.21.15.97:0/2641851836 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2110104f80 0x7f211019a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:09:01.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.128+0000 7f210fe00640 1 -- 172.21.15.97:0/2641851836 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21101a1b70 con 0x7f2110103d90 2024-03-27T11:09:01.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.128+0000 7f210fe00640 1 --2- 172.21.15.97:0/2641851836 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2110103d90 0x7f2110199f50 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f210000e9c0 tx=0x7f210000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:09:01.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.129+0000 7f210cc00640 1 -- 172.21.15.97:0/2641851836 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f210000cd80 con 0x7f2110103d90 2024-03-27T11:09:01.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.129+0000 7f210cc00640 1 -- 172.21.15.97:0/2641851836 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f21000035f0 con 0x7f2110103d90 2024-03-27T11:09:01.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.129+0000 7f2116c00640 1 -- 172.21.15.97:0/2641851836 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21101a1e60 con 0x7f2110103d90 2024-03-27T11:09:01.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.129+0000 7f210cc00640 1 -- 172.21.15.97:0/2641851836 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2100010690 con 0x7f2110103d90 2024-03-27T11:09:01.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.129+0000 7f2116c00640 1 -- 172.21.15.97:0/2641851836 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21101a2320 con 0x7f2110103d90 2024-03-27T11:09:01.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.130+0000 7f2116c00640 1 -- 172.21.15.97:0/2641851836 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f211010b150 con 0x7f2110103d90 2024-03-27T11:09:01.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.132+0000 7f210cc00640 1 -- 172.21.15.97:0/2641851836 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f21000037a0 con 0x7f2110103d90 2024-03-27T11:09:01.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.134+0000 7f210cc00640 1 --2- 172.21.15.97:0/2641851836 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f20f40a3690 0x7f20f40a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:09:01.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.134+0000 7f210cc00640 1 -- 172.21.15.97:0/2641851836 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f21000d00f0 con 0x7f2110103d90 2024-03-27T11:09:01.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.134+0000 7f210f400640 1 --2- 172.21.15.97:0/2641851836 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f20f40a3690 0x7f20f40a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:09:01.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.135+0000 7f210f400640 1 --2- 172.21.15.97:0/2641851836 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f20f40a3690 0x7f20f40a5b50 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f211019b460 tx=0x7f20fc00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:09:01.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.136+0000 7f210cc00640 1 -- 172.21.15.97:0/2641851836 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f210009ccd0 con 0x7f2110103d90 2024-03-27T11:09:01.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.346+0000 7f2116c00640 1 -- 172.21.15.97:0/2641851836 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f211002d370 con 0x7f2110103d90 2024-03-27T11:09:01.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.346+0000 7f210cc00640 1 -- 172.21.15.97:0/2641851836 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v92) ==== 81+0+582 (secure 0 0 0) 0x7f210009c5d0 con 0x7f2110103d90 2024-03-27T11:09:01.316 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:0/359646 2024-03-28T10:44:39.889550+0000 2024-03-27T11:09:01.316 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6801/98682155 2024-03-28T10:44:06.019075+0000 2024-03-27T11:09:01.316 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6801/1685921853 2024-03-28T10:44:39.889550+0000 2024-03-27T11:09:01.316 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6800/98682155 2024-03-28T10:44:06.019075+0000 2024-03-27T11:09:01.316 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:0/107690680 2024-03-28T10:44:06.019075+0000 2024-03-27T11:09:01.316 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6800/1685921853 2024-03-28T10:44:39.889550+0000 2024-03-27T11:09:01.316 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:0/437043413 2024-03-28T10:44:39.889550+0000 2024-03-27T11:09:01.317 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:0/2319706992 2024-03-28T10:44:39.889550+0000 2024-03-27T11:09:01.317 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:0/3911532654 2024-03-28T10:44:06.019075+0000 2024-03-27T11:09:01.317 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:0/3347321845 2024-03-28T10:44:06.019075+0000 2024-03-27T11:09:01.317 INFO:teuthology.orchestra.run.smithi097.stderr:listed 10 entries 2024-03-27T11:09:01.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.349+0000 7f2116c00640 1 -- 172.21.15.97:0/2641851836 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f20f40a3690 msgr2=0x7f20f40a5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:09:01.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.349+0000 7f2116c00640 1 --2- 172.21.15.97:0/2641851836 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f20f40a3690 0x7f20f40a5b50 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f211019b460 tx=0x7f20fc00b390 comp rx=0 tx=0).stop 2024-03-27T11:09:01.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.349+0000 7f2116c00640 1 -- 172.21.15.97:0/2641851836 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2110103d90 msgr2=0x7f2110199f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:09:01.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.349+0000 7f2116c00640 1 --2- 172.21.15.97:0/2641851836 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2110103d90 0x7f2110199f50 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f210000e9c0 tx=0x7f210000ee80 comp rx=0 tx=0).stop 2024-03-27T11:09:01.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.349+0000 7f2116c00640 1 -- 172.21.15.97:0/2641851836 shutdown_connections 2024-03-27T11:09:01.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.349+0000 7f2116c00640 1 --2- 172.21.15.97:0/2641851836 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2110105940 0x7f21101a1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:09:01.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.349+0000 7f2116c00640 1 --2- 172.21.15.97:0/2641851836 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f20f40a3690 0x7f20f40a5b50 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:09:01.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.349+0000 7f2116c00640 1 --2- 172.21.15.97:0/2641851836 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2110104f80 0x7f211019a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:09:01.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.349+0000 7f2116c00640 1 --2- 172.21.15.97:0/2641851836 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2110103d90 0x7f2110199f50 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:09:01.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.349+0000 7f2116c00640 1 -- 172.21.15.97:0/2641851836 >> 172.21.15.97:0/2641851836 conn(0x7f21100ff300 msgr2=0x7f2110108320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:09:01.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.349+0000 7f2116c00640 1 -- 172.21.15.97:0/2641851836 shutdown_connections 2024-03-27T11:09:01.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.349+0000 7f2116c00640 1 -- 172.21.15.97:0/2641851836 wait complete. 2024-03-27T11:09:01.336 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-27T11:09:01.336 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-27T11:09:01.376 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-27T11:09:01.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.652+0000 7fed1c600640 1 -- 172.21.15.97:0/1616012443 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed140780f0 msgr2=0x7fed14078990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:09:01.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.652+0000 7fed1c600640 1 --2- 172.21.15.97:0/1616012443 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed140780f0 0x7fed14078990 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fed0c009ca0 tx=0x7fed0c02c100 comp rx=0 tx=0).stop 2024-03-27T11:09:01.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.653+0000 7fed1c600640 1 -- 172.21.15.97:0/1616012443 shutdown_connections 2024-03-27T11:09:01.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.653+0000 7fed1c600640 1 --2- 172.21.15.97:0/1616012443 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed140780f0 0x7fed14078990 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:09:01.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.653+0000 7fed1c600640 1 --2- 172.21.15.97:0/1616012443 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fed14077750 0x7fed14077bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:09:01.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.653+0000 7fed1c600640 1 --2- 172.21.15.97:0/1616012443 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fed14076510 0x7fed14076930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:09:01.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.653+0000 7fed1c600640 1 -- 172.21.15.97:0/1616012443 >> 172.21.15.97:0/1616012443 conn(0x7fed140ff300 msgr2=0x7fed14101720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:09:01.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.653+0000 7fed1c600640 1 -- 172.21.15.97:0/1616012443 shutdown_connections 2024-03-27T11:09:01.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.653+0000 7fed1c600640 1 -- 172.21.15.97:0/1616012443 wait complete. 2024-03-27T11:09:01.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.654+0000 7fed1c600640 1 Processor -- start 2024-03-27T11:09:01.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.654+0000 7fed1c600640 1 -- start start 2024-03-27T11:09:01.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.654+0000 7fed1c600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fed14076510 0x7fed1419e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:09:01.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.654+0000 7fed1c600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed14077750 0x7fed1419e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:09:01.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.654+0000 7fed1c600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fed140780f0 0x7fed141a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:09:01.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.654+0000 7fed1c600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fed14113cb0 con 0x7fed14077750 2024-03-27T11:09:01.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.654+0000 7fed1c600640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fed14113b30 con 0x7fed14076510 2024-03-27T11:09:01.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.654+0000 7fed1c600640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fed14113e30 con 0x7fed140780f0 2024-03-27T11:09:01.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.655+0000 7fed19200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed14077750 0x7fed1419e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:09:01.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.655+0000 7fed19c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fed14076510 0x7fed1419e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:09:01.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.655+0000 7fed19200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed14077750 0x7fed1419e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40454/0 (socket says 172.21.15.97:40454) 2024-03-27T11:09:01.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.655+0000 7fed19c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fed14076510 0x7fed1419e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.97:37982/0 (socket says 172.21.15.97:37982) 2024-03-27T11:09:01.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.655+0000 7fed19200640 1 -- 172.21.15.97:0/2379722293 learned_addr learned my addr 172.21.15.97:0/2379722293 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:09:01.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.655+0000 7fed1a600640 1 --2- 172.21.15.97:0/2379722293 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fed140780f0 0x7fed141a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:09:01.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.655+0000 7fed19200640 1 -- 172.21.15.97:0/2379722293 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fed140780f0 msgr2=0x7fed141a58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:09:01.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.655+0000 7fed19200640 1 --2- 172.21.15.97:0/2379722293 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fed140780f0 0x7fed141a58f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:09:01.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.655+0000 7fed19200640 1 -- 172.21.15.97:0/2379722293 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fed14076510 msgr2=0x7fed1419e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:09:01.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.655+0000 7fed19200640 1 --2- 172.21.15.97:0/2379722293 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fed14076510 0x7fed1419e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:09:01.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.655+0000 7fed19200640 1 -- 172.21.15.97:0/2379722293 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed141a5f60 con 0x7fed14077750 2024-03-27T11:09:01.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.655+0000 7fed1a600640 1 --2- 172.21.15.97:0/2379722293 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fed140780f0 0x7fed141a58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T11:09:01.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.656+0000 7fed19200640 1 --2- 172.21.15.97:0/2379722293 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed14077750 0x7fed1419e880 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fed0000e9f0 tx=0x7fed0000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:09:01.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.656+0000 7fed12000640 1 -- 172.21.15.97:0/2379722293 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed0000cd80 con 0x7fed14077750 2024-03-27T11:09:01.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.656+0000 7fed12000640 1 -- 172.21.15.97:0/2379722293 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fed000108e0 con 0x7fed14077750 2024-03-27T11:09:01.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.656+0000 7fed12000640 1 -- 172.21.15.97:0/2379722293 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed00018920 con 0x7fed14077750 2024-03-27T11:09:01.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.656+0000 7fed1c600640 1 -- 172.21.15.97:0/2379722293 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed141a6250 con 0x7fed14077750 2024-03-27T11:09:01.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.656+0000 7fed1c600640 1 -- 172.21.15.97:0/2379722293 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed141a6750 con 0x7fed14077750 2024-03-27T11:09:01.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.657+0000 7fed1c600640 1 -- 172.21.15.97:0/2379722293 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed14110860 con 0x7fed14077750 2024-03-27T11:09:01.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.660+0000 7fed12000640 1 -- 172.21.15.97:0/2379722293 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7fed00010a90 con 0x7fed14077750 2024-03-27T11:09:01.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.661+0000 7fed12000640 1 --2- 172.21.15.97:0/2379722293 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fecfc0a3610 0x7fecfc0a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:09:01.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.661+0000 7fed12000640 1 -- 172.21.15.97:0/2379722293 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fed000d00c0 con 0x7fed14077750 2024-03-27T11:09:01.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.661+0000 7fed19c00640 1 --2- 172.21.15.97:0/2379722293 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fecfc0a3610 0x7fecfc0a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:09:01.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.662+0000 7fed19c00640 1 --2- 172.21.15.97:0/2379722293 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fecfc0a3610 0x7fecfc0a5ad0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fed08009bd0 tx=0x7fed08005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:09:01.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.665+0000 7fed12000640 1 -- 172.21.15.97:0/2379722293 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7fed0009cca0 con 0x7fed14077750 2024-03-27T11:09:01.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.863+0000 7fed1c600640 1 -- 172.21.15.97:0/2379722293 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fed14062780 con 0x7fed14077750 2024-03-27T11:09:01.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.866+0000 7fed12000640 1 -- 172.21.15.97:0/2379722293 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+22644 (secure 0 0 0) 0x7fed0009c5a0 con 0x7fed14077750 2024-03-27T11:09:01.836 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-27T11:09:01.836 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":92,"fsid":"adca88c4-ec26-11ee-95d0-87774f69a715","created":"2024-03-27T10:43:24.680694+0000","modified":"2024-03-27T10:54:41.310812+0000","last_up_change":"2024-03-27T10:53:00.552513+0000","last_in_change":"2024-03-27T10:52:36.302986+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-27T10:47:54.052810+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-27T10:54:02.845060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-27T10:54:03.474151+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b634e5f1-6118-4e2f-95e9-830db390f8d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1746812080},{"type":"v1","addr":"172.21.15.97:6803","nonce":1746812080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1746812080},{"type":"v1","addr":"172.21.15.97:6805","nonce":1746812080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":1746812080},{"type":"v1","addr":"172.21.15.97:6809","nonce":1746812080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1746812080},{"type":"v1","addr":"172.21.15.97:6807","nonce":1746812080}]},"public_addr":"172.21.15.97:6803/1746812080","cluster_addr":"172.21.15.97:6805/1746812080","heartbeat_back_addr":"172.21.15.97:6809/1746812080","heartbeat_front_addr":"172.21.15.97:6807/1746812080","state":["exists","up"]},{"osd":1,"uuid":"441e2770-3284-4388-b0de-c56eb8a373d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":3858821264},{"type":"v1","addr":"172.21.15.114:6801","nonce":3858821264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":3858821264},{"type":"v1","addr":"172.21.15.114:6803","nonce":3858821264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":3858821264},{"type":"v1","addr":"172.21.15.114:6807","nonce":3858821264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":3858821264},{"type":"v1","addr":"172.21.15.114:6805","nonce":3858821264}]},"public_addr":"172.21.15.114:6801/3858821264","cluster_addr":"172.21.15.114:6803/3858821264","heartbeat_back_addr":"172.21.15.114:6807/3858821264","heartbeat_front_addr":"172.21.15.114:6805/3858821264","state":["exists","up"]},{"osd":2,"uuid":"00108864-1aa6-48d6-8398-e8eb79fda123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":124240658},{"type":"v1","addr":"172.21.15.177:6801","nonce":124240658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":124240658},{"type":"v1","addr":"172.21.15.177:6803","nonce":124240658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":124240658},{"type":"v1","addr":"172.21.15.177:6807","nonce":124240658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":124240658},{"type":"v1","addr":"172.21.15.177:6805","nonce":124240658}]},"public_addr":"172.21.15.177:6801/124240658","cluster_addr":"172.21.15.177:6803/124240658","heartbeat_back_addr":"172.21.15.177:6807/124240658","heartbeat_front_addr":"172.21.15.177:6805/124240658","state":["exists","up"]},{"osd":3,"uuid":"1860b690-e2dc-4977-88d7-d1aacf7022ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2478179311},{"type":"v1","addr":"172.21.15.97:6811","nonce":2478179311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2478179311},{"type":"v1","addr":"172.21.15.97:6813","nonce":2478179311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":2478179311},{"type":"v1","addr":"172.21.15.97:6817","nonce":2478179311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2478179311},{"type":"v1","addr":"172.21.15.97:6815","nonce":2478179311}]},"public_addr":"172.21.15.97:6811/2478179311","cluster_addr":"172.21.15.97:6813/2478179311","heartbeat_back_addr":"172.21.15.97:6817/2478179311","heartbeat_front_addr":"172.21.15.97:6815/2478179311","state":["exists","up"]},{"osd":4,"uuid":"9245a3a4-7d49-4fdb-9d55-112e43177050","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":75755945},{"type":"v1","addr":"172.21.15.114:6809","nonce":75755945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":75755945},{"type":"v1","addr":"172.21.15.114:6811","nonce":75755945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":75755945},{"type":"v1","addr":"172.21.15.114:6815","nonce":75755945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":75755945},{"type":"v1","addr":"172.21.15.114:6813","nonce":75755945}]},"public_addr":"172.21.15.114:6809/75755945","cluster_addr":"172.21.15.114:6811/75755945","heartbeat_back_addr":"172.21.15.114:6815/75755945","heartbeat_front_addr":"172.21.15.114:6813/75755945","state":["exists","up"]},{"osd":5,"uuid":"51922f99-e8d3-461b-8615-e9bfdcbf5ed3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":1918493790},{"type":"v1","addr":"172.21.15.177:6809","nonce":1918493790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":1918493790},{"type":"v1","addr":"172.21.15.177:6811","nonce":1918493790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":1918493790},{"type":"v1","addr":"172.21.15.177:6815","nonce":1918493790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":1918493790},{"type":"v1","addr":"172.21.15.177:6813","nonce":1918493790}]},"public_addr":"172.21.15.177:6809/1918493790","cluster_addr":"172.21.15.177:6811/1918493790","heartbeat_back_addr":"172.21.15.177:6815/1918493790","heartbeat_front_addr":"172.21.15.177:6813/1918493790","state":["exists","up"]},{"osd":6,"uuid":"06dffb32-05eb-43b3-852e-915c5cc7f25a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2892389563},{"type":"v1","addr":"172.21.15.97:6819","nonce":2892389563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2892389563},{"type":"v1","addr":"172.21.15.97:6821","nonce":2892389563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2892389563},{"type":"v1","addr":"172.21.15.97:6825","nonce":2892389563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2892389563},{"type":"v1","addr":"172.21.15.97:6823","nonce":2892389563}]},"public_addr":"172.21.15.97:6819/2892389563","cluster_addr":"172.21.15.97:6821/2892389563","heartbeat_back_addr":"172.21.15.97:6825/2892389563","heartbeat_front_addr":"172.21.15.97:6823/2892389563","state":["exists","up"]},{"osd":7,"uuid":"e68e3519-9c1b-43c5-8e06-5385248b19df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":2445491153},{"type":"v1","addr":"172.21.15.114:6817","nonce":2445491153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":2445491153},{"type":"v1","addr":"172.21.15.114:6819","nonce":2445491153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2445491153},{"type":"v1","addr":"172.21.15.114:6823","nonce":2445491153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":2445491153},{"type":"v1","addr":"172.21.15.114:6821","nonce":2445491153}]},"public_addr":"172.21.15.114:6817/2445491153","cluster_addr":"172.21.15.114:6819/2445491153","heartbeat_back_addr":"172.21.15.114:6823/2445491153","heartbeat_front_addr":"172.21.15.114:6821/2445491153","state":["exists","up"]},{"osd":8,"uuid":"31ac592e-1349-49ee-9af9-8ad85d682978","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":2051134191},{"type":"v1","addr":"172.21.15.177:6817","nonce":2051134191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":2051134191},{"type":"v1","addr":"172.21.15.177:6819","nonce":2051134191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":2051134191},{"type":"v1","addr":"172.21.15.177:6823","nonce":2051134191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":2051134191},{"type":"v1","addr":"172.21.15.177:6821","nonce":2051134191}]},"public_addr":"172.21.15.177:6817/2051134191","cluster_addr":"172.21.15.177:6819/2051134191","heartbeat_back_addr":"172.21.15.177:6823/2051134191","heartbeat_front_addr":"172.21.15.177:6821/2051134191","state":["exists","up"]},{"osd":9,"uuid":"0171086a-dd41-4eb4-a389-16f0c46a64db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":1578881271},{"type":"v1","addr":"172.21.15.97:6827","nonce":1578881271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1578881271},{"type":"v1","addr":"172.21.15.97:6829","nonce":1578881271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1578881271},{"type":"v1","addr":"172.21.15.97:6833","nonce":1578881271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1578881271},{"type":"v1","addr":"172.21.15.97:6831","nonce":1578881271}]},"public_addr":"172.21.15.97:6827/1578881271","cluster_addr":"172.21.15.97:6829/1578881271","heartbeat_back_addr":"172.21.15.97:6833/1578881271","heartbeat_front_addr":"172.21.15.97:6831/1578881271","state":["exists","up"]},{"osd":10,"uuid":"64754e31-b124-4601-ad98-2f70b1dd5359","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":2893111461},{"type":"v1","addr":"172.21.15.114:6825","nonce":2893111461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":2893111461},{"type":"v1","addr":"172.21.15.114:6827","nonce":2893111461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2893111461},{"type":"v1","addr":"172.21.15.114:6831","nonce":2893111461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":2893111461},{"type":"v1","addr":"172.21.15.114:6829","nonce":2893111461}]},"public_addr":"172.21.15.114:6825/2893111461","cluster_addr":"172.21.15.114:6827/2893111461","heartbeat_back_addr":"172.21.15.114:6831/2893111461","heartbeat_front_addr":"172.21.15.114:6829/2893111461","state":["exists","up"]},{"osd":11,"uuid":"9ffc752c-adf7-4037-89d3-535787d7475d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":85,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":2333030391},{"type":"v1","addr":"172.21.15.177:6825","nonce":2333030391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":2333030391},{"type":"v1","addr":"172.21.15.177:6827","nonce":2333030391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":2333030391},{"type":"v1","addr":"172.21.15.177:6831","nonce":2333030391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":2333030391},{"type":"v1","addr":"172.21.15.177:6829","nonce":2333030391}]},"public_addr":"172.21.15.177:6825/2333030391","cluster_addr":"172.21.15.177:6827/2333030391","heartbeat_back_addr":"172.21.15.177:6831/2333030391","heartbeat_front_addr":"172.21.15.177:6829/2333030391","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:46:45.143519+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:47:18.656947+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:47:51.316345+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:48:24.575512+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:48:59.113555+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:49:32.103865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:50:06.615539+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:50:41.578341+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:51:15.254011+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:51:49.099165+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:52:24.544821+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T10:52:58.838484+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.28","mappings":[{"from":3,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/359646":"2024-03-28T10:44:39.889550+0000","172.21.15.97:6801/98682155":"2024-03-28T10:44:06.019075+0000","172.21.15.97:6801/1685921853":"2024-03-28T10:44:39.889550+0000","172.21.15.97:6800/98682155":"2024-03-28T10:44:06.019075+0000","172.21.15.97:0/107690680":"2024-03-28T10:44:06.019075+0000","172.21.15.97:6800/1685921853":"2024-03-28T10:44:39.889550+0000","172.21.15.97:0/437043413":"2024-03-28T10:44:39.889550+0000","172.21.15.97:0/2319706992":"2024-03-28T10:44:39.889550+0000","172.21.15.97:0/3911532654":"2024-03-28T10:44:06.019075+0000","172.21.15.97:0/3347321845":"2024-03-28T10:44:06.019075+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-27T11:09:01.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.868+0000 7fed1c600640 1 -- 172.21.15.97:0/2379722293 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fecfc0a3610 msgr2=0x7fecfc0a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:09:01.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.868+0000 7fed1c600640 1 --2- 172.21.15.97:0/2379722293 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fecfc0a3610 0x7fecfc0a5ad0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fed08009bd0 tx=0x7fed08005f90 comp rx=0 tx=0).stop 2024-03-27T11:09:01.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.868+0000 7fed1c600640 1 -- 172.21.15.97:0/2379722293 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed14077750 msgr2=0x7fed1419e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:09:01.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.868+0000 7fed1c600640 1 --2- 172.21.15.97:0/2379722293 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed14077750 0x7fed1419e880 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fed0000e9f0 tx=0x7fed0000eeb0 comp rx=0 tx=0).stop 2024-03-27T11:09:01.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.869+0000 7fed1c600640 1 -- 172.21.15.97:0/2379722293 shutdown_connections 2024-03-27T11:09:01.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.869+0000 7fed1c600640 1 --2- 172.21.15.97:0/2379722293 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fed140780f0 0x7fed141a58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:09:01.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.869+0000 7fed1c600640 1 --2- 172.21.15.97:0/2379722293 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed14077750 0x7fed1419e880 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:09:01.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.869+0000 7fed1c600640 1 --2- 172.21.15.97:0/2379722293 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fecfc0a3610 0x7fecfc0a5ad0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:09:01.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.869+0000 7fed1c600640 1 --2- 172.21.15.97:0/2379722293 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fed14076510 0x7fed1419e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:09:01.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.869+0000 7fed1c600640 1 -- 172.21.15.97:0/2379722293 >> 172.21.15.97:0/2379722293 conn(0x7fed140ff300 msgr2=0x7fed141009f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:09:01.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.869+0000 7fed1c600640 1 -- 172.21.15.97:0/2379722293 shutdown_connections 2024-03-27T11:09:01.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:01.869+0000 7fed1c600640 1 -- 172.21.15.97:0/2379722293 wait complete. 2024-03-27T11:09:01.857 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-27T10:47:54.052810+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-27T10:54:02.845060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-27T10:54:03.474151+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-27T11:09:01.858 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-27T11:09:01.887 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:09:01 smithi097 ceph-mon[31859]: pgmap v812: 97 pgs: 97 active+clean; 1.6 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-27T11:09:01.887 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:09:01 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/3252196992' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-27T11:09:01.887 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:09:01 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/2641851836' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-27T11:09:01.923 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:09:01 smithi177 ceph-mon[35294]: pgmap v812: 97 pgs: 97 active+clean; 1.6 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-27T11:09:01.924 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:09:01 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/3252196992' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-27T11:09:01.924 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:09:01 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/2641851836' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-27T11:09:02.013 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:09:01 smithi114 ceph-mon[35338]: pgmap v812: 97 pgs: 97 active+clean; 1.6 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-27T11:09:02.014 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:09:01 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/3252196992' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-27T11:09:02.014 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:09:01 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/2641851836' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-27T11:09:02.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.103+0000 7fc501000640 1 -- 172.21.15.97:0/1662118481 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4fc1015c0 msgr2=0x7fc4fc101a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:09:02.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.103+0000 7fc501000640 1 --2- 172.21.15.97:0/1662118481 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4fc1015c0 0x7fc4fc101a40 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fc4f4009900 tx=0x7fc4f402bc30 comp rx=0 tx=0).stop 2024-03-27T11:09:02.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.103+0000 7fc501000640 1 -- 172.21.15.97:0/1662118481 shutdown_connections 2024-03-27T11:09:02.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.103+0000 7fc501000640 1 --2- 172.21.15.97:0/1662118481 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4fc1015c0 0x7fc4fc101a40 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:09:02.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.103+0000 7fc501000640 1 --2- 172.21.15.97:0/1662118481 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc4fc100c20 0x7fc4fc101080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:09:02.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.103+0000 7fc501000640 1 --2- 172.21.15.97:0/1662118481 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc4fc1002c0 0x7fc4fc1006e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:09:02.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.103+0000 7fc501000640 1 -- 172.21.15.97:0/1662118481 >> 172.21.15.97:0/1662118481 conn(0x7fc4fc0fbec0 msgr2=0x7fc4fc0fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:09:02.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.103+0000 7fc501000640 1 -- 172.21.15.97:0/1662118481 shutdown_connections 2024-03-27T11:09:02.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.103+0000 7fc501000640 1 -- 172.21.15.97:0/1662118481 wait complete. 2024-03-27T11:09:02.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.104+0000 7fc501000640 1 Processor -- start 2024-03-27T11:09:02.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.104+0000 7fc501000640 1 -- start start 2024-03-27T11:09:02.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.104+0000 7fc501000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc4fc1002c0 0x7fc4fc199dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:09:02.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.105+0000 7fc501000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc4fc100c20 0x7fc4fc19a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:09:02.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.105+0000 7fc501000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4fc1015c0 0x7fc4fc1a1370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:09:02.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.105+0000 7fc501000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc4fc10f7a0 con 0x7fc4fc1015c0 2024-03-27T11:09:02.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.105+0000 7fc501000640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc4fc10f620 con 0x7fc4fc100c20 2024-03-27T11:09:02.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.105+0000 7fc501000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc4fc10f920 con 0x7fc4fc1002c0 2024-03-27T11:09:02.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.105+0000 7fc4fbe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4fc1015c0 0x7fc4fc1a1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:09:02.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.105+0000 7fc4fbe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4fc1015c0 0x7fc4fc1a1370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40474/0 (socket says 172.21.15.97:40474) 2024-03-27T11:09:02.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.105+0000 7fc4fbe00640 1 -- 172.21.15.97:0/205102407 learned_addr learned my addr 172.21.15.97:0/205102407 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:09:02.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.105+0000 7fc4faa00640 1 --2- 172.21.15.97:0/205102407 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc4fc100c20 0x7fc4fc19a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:09:02.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.105+0000 7fc4fb400640 1 --2- 172.21.15.97:0/205102407 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc4fc1002c0 0x7fc4fc199dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:09:02.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.105+0000 7fc4fbe00640 1 -- 172.21.15.97:0/205102407 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc4fc1002c0 msgr2=0x7fc4fc199dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:09:02.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.105+0000 7fc4fbe00640 1 --2- 172.21.15.97:0/205102407 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc4fc1002c0 0x7fc4fc199dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:09:02.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.105+0000 7fc4fbe00640 1 -- 172.21.15.97:0/205102407 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc4fc100c20 msgr2=0x7fc4fc19a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:09:02.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.105+0000 7fc4fbe00640 1 --2- 172.21.15.97:0/205102407 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc4fc100c20 0x7fc4fc19a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:09:02.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.105+0000 7fc4fbe00640 1 -- 172.21.15.97:0/205102407 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4fc1a19e0 con 0x7fc4fc1015c0 2024-03-27T11:09:02.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.106+0000 7fc4fbe00640 1 --2- 172.21.15.97:0/205102407 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4fc1015c0 0x7fc4fc1a1370 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fc4f4036040 tx=0x7fc4f402cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:09:02.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.106+0000 7fc4f3e00640 1 -- 172.21.15.97:0/205102407 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f4043020 con 0x7fc4fc1015c0 2024-03-27T11:09:02.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.106+0000 7fc4f3e00640 1 -- 172.21.15.97:0/205102407 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc4f40044c0 con 0x7fc4fc1015c0 2024-03-27T11:09:02.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.106+0000 7fc4f3e00640 1 -- 172.21.15.97:0/205102407 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f4003bf0 con 0x7fc4fc1015c0 2024-03-27T11:09:02.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.106+0000 7fc501000640 1 -- 172.21.15.97:0/205102407 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4fc1a1c70 con 0x7fc4fc1015c0 2024-03-27T11:09:02.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.106+0000 7fc501000640 1 -- 172.21.15.97:0/205102407 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4fc1a2170 con 0x7fc4fc1015c0 2024-03-27T11:09:02.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.107+0000 7fc501000640 1 -- 172.21.15.97:0/205102407 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4fc10b160 con 0x7fc4fc1015c0 2024-03-27T11:09:02.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.109+0000 7fc4f3e00640 1 -- 172.21.15.97:0/205102407 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7fc4f4046020 con 0x7fc4fc1015c0 2024-03-27T11:09:02.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.110+0000 7fc4f3e00640 1 --2- 172.21.15.97:0/205102407 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fc4e00a3690 0x7fc4e00a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:09:02.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.111+0000 7fc4f3e00640 1 -- 172.21.15.97:0/205102407 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7fc4f40f1fd0 con 0x7fc4fc1015c0 2024-03-27T11:09:02.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.111+0000 7fc4fb400640 1 --2- 172.21.15.97:0/205102407 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fc4e00a3690 0x7fc4e00a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:09:02.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.111+0000 7fc4fb400640 1 --2- 172.21.15.97:0/205102407 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fc4e00a3690 0x7fc4e00a5b50 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fc4ec005fd0 tx=0x7fc4ec009520 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:09:02.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.115+0000 7fc4f3e00640 1 -- 172.21.15.97:0/205102407 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7fc4f404d020 con 0x7fc4fc1015c0 2024-03-27T11:09:02.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.314+0000 7fc501000640 1 -- 172.21.15.97:0/205102407 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc4fc02d590 con 0x7fc4fc1015c0 2024-03-27T11:09:02.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.315+0000 7fc4f3e00640 1 -- 172.21.15.97:0/205102407 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) ==== 93+0+10 (secure 0 0 0) 0x7fc4f40beb50 con 0x7fc4fc1015c0 2024-03-27T11:09:02.285 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-03-27T11:09:02.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.318+0000 7fc501000640 1 -- 172.21.15.97:0/205102407 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fc4e00a3690 msgr2=0x7fc4e00a5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:09:02.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.318+0000 7fc501000640 1 --2- 172.21.15.97:0/205102407 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fc4e00a3690 0x7fc4e00a5b50 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fc4ec005fd0 tx=0x7fc4ec009520 comp rx=0 tx=0).stop 2024-03-27T11:09:02.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.318+0000 7fc501000640 1 -- 172.21.15.97:0/205102407 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4fc1015c0 msgr2=0x7fc4fc1a1370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:09:02.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.318+0000 7fc501000640 1 --2- 172.21.15.97:0/205102407 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4fc1015c0 0x7fc4fc1a1370 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fc4f4036040 tx=0x7fc4f402cdd0 comp rx=0 tx=0).stop 2024-03-27T11:09:02.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.318+0000 7fc501000640 1 -- 172.21.15.97:0/205102407 shutdown_connections 2024-03-27T11:09:02.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.318+0000 7fc501000640 1 --2- 172.21.15.97:0/205102407 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4fc1015c0 0x7fc4fc1a1370 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:09:02.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.318+0000 7fc501000640 1 --2- 172.21.15.97:0/205102407 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7fc4e00a3690 0x7fc4e00a5b50 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:09:02.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.318+0000 7fc501000640 1 --2- 172.21.15.97:0/205102407 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc4fc100c20 0x7fc4fc19a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:09:02.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.318+0000 7fc501000640 1 --2- 172.21.15.97:0/205102407 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc4fc1002c0 0x7fc4fc199dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:09:02.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.318+0000 7fc501000640 1 -- 172.21.15.97:0/205102407 >> 172.21.15.97:0/205102407 conn(0x7fc4fc0fbec0 msgr2=0x7fc4fc0fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:09:02.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.318+0000 7fc501000640 1 -- 172.21.15.97:0/205102407 shutdown_connections 2024-03-27T11:09:02.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.318+0000 7fc501000640 1 -- 172.21.15.97:0/205102407 wait complete. 2024-03-27T11:09:02.308 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-27T11:09:02.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.604+0000 7f3541e00640 1 -- 172.21.15.97:0/1999791184 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f353c104f80 msgr2=0x7f353c105400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:09:02.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.604+0000 7f3541e00640 1 --2- 172.21.15.97:0/1999791184 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f353c104f80 0x7f353c105400 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f3528009900 tx=0x7f352802bc30 comp rx=0 tx=0).stop 2024-03-27T11:09:02.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.604+0000 7f3541e00640 1 -- 172.21.15.97:0/1999791184 shutdown_connections 2024-03-27T11:09:02.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.604+0000 7f3541e00640 1 --2- 172.21.15.97:0/1999791184 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f353c105940 0x7f353c10c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:09:02.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.604+0000 7f3541e00640 1 --2- 172.21.15.97:0/1999791184 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f353c104f80 0x7f353c105400 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:09:02.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.604+0000 7f3541e00640 1 --2- 172.21.15.97:0/1999791184 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f353c103d90 0x7f353c1041b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:09:02.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.604+0000 7f3541e00640 1 -- 172.21.15.97:0/1999791184 >> 172.21.15.97:0/1999791184 conn(0x7f353c0ff300 msgr2=0x7f353c101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:09:02.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.604+0000 7f3541e00640 1 -- 172.21.15.97:0/1999791184 shutdown_connections 2024-03-27T11:09:02.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.604+0000 7f3541e00640 1 -- 172.21.15.97:0/1999791184 wait complete. 2024-03-27T11:09:02.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.605+0000 7f3541e00640 1 Processor -- start 2024-03-27T11:09:02.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.606+0000 7f3541e00640 1 -- start start 2024-03-27T11:09:02.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.606+0000 7f3541e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f353c103d90 0x7f353c199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:09:02.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.606+0000 7f3541e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f353c104f80 0x7f353c19a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:09:02.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.606+0000 7f3541e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f353c105940 0x7f353c1a1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:09:02.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.606+0000 7f3541e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f353c10f8f0 con 0x7f353c103d90 2024-03-27T11:09:02.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.606+0000 7f3541e00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f353c10f770 con 0x7f353c105940 2024-03-27T11:09:02.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.606+0000 7f3541e00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f353c10fa70 con 0x7f353c104f80 2024-03-27T11:09:02.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.606+0000 7f353aa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f353c103d90 0x7f353c199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:09:02.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.606+0000 7f353b400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f353c105940 0x7f353c1a1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:09:02.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.606+0000 7f353aa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f353c103d90 0x7f353c199fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40492/0 (socket says 172.21.15.97:40492) 2024-03-27T11:09:02.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.606+0000 7f353aa00640 1 -- 172.21.15.97:0/1379309453 learned_addr learned my addr 172.21.15.97:0/1379309453 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:09:02.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.606+0000 7f353a000640 1 --2- 172.21.15.97:0/1379309453 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f353c104f80 0x7f353c19a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:09:02.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.607+0000 7f353aa00640 1 -- 172.21.15.97:0/1379309453 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f353c104f80 msgr2=0x7f353c19a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:09:02.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.607+0000 7f353aa00640 1 --2- 172.21.15.97:0/1379309453 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f353c104f80 0x7f353c19a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:09:02.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.607+0000 7f353aa00640 1 -- 172.21.15.97:0/1379309453 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f353c105940 msgr2=0x7f353c1a1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:09:02.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.607+0000 7f353aa00640 1 --2- 172.21.15.97:0/1379309453 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f353c105940 0x7f353c1a1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:09:02.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.607+0000 7f353aa00640 1 -- 172.21.15.97:0/1379309453 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f353c1a1bf0 con 0x7f353c103d90 2024-03-27T11:09:02.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.607+0000 7f353aa00640 1 --2- 172.21.15.97:0/1379309453 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f353c103d90 0x7f353c199fd0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f352c00d770 tx=0x7f352c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:09:02.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.607+0000 7f3540a00640 1 -- 172.21.15.97:0/1379309453 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f352c017070 con 0x7f353c103d90 2024-03-27T11:09:02.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.608+0000 7f3540a00640 1 -- 172.21.15.97:0/1379309453 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f352c009c30 con 0x7f353c103d90 2024-03-27T11:09:02.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.608+0000 7f3540a00640 1 -- 172.21.15.97:0/1379309453 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f352c003d00 con 0x7f353c103d90 2024-03-27T11:09:02.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.608+0000 7f3541e00640 1 -- 172.21.15.97:0/1379309453 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f353c1a1ee0 con 0x7f353c103d90 2024-03-27T11:09:02.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.608+0000 7f3541e00640 1 -- 172.21.15.97:0/1379309453 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f353c1a2340 con 0x7f353c103d90 2024-03-27T11:09:02.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.609+0000 7f3541e00640 1 -- 172.21.15.97:0/1379309453 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f353c10b1d0 con 0x7f353c103d90 2024-03-27T11:09:02.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.611+0000 7f3540a00640 1 -- 172.21.15.97:0/1379309453 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f352c003ea0 con 0x7f353c103d90 2024-03-27T11:09:02.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.612+0000 7f3540a00640 1 --2- 172.21.15.97:0/1379309453 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f35200a3610 0x7f35200a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:09:02.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.612+0000 7f3540a00640 1 -- 172.21.15.97:0/1379309453 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7f352c0cf9b0 con 0x7f353c103d90 2024-03-27T11:09:02.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.612+0000 7f353a000640 1 --2- 172.21.15.97:0/1379309453 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f35200a3610 0x7f35200a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:09:02.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.613+0000 7f353a000640 1 --2- 172.21.15.97:0/1379309453 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f35200a3610 0x7f35200a5ad0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f3528000c00 tx=0x7f352802c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:09:02.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.616+0000 7f3540a00640 1 -- 172.21.15.97:0/1379309453 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f352c009de0 con 0x7f353c103d90 2024-03-27T11:09:02.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.818+0000 7f3541e00640 1 -- 172.21.15.97:0/1379309453 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f353c062850 con 0x7f353c103d90 2024-03-27T11:09:02.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.819+0000 7f3540a00640 1 -- 172.21.15.97:0/1379309453 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) ==== 104+0+11 (secure 0 0 0) 0x7f352c09bf40 con 0x7f353c103d90 2024-03-27T11:09:02.789 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-03-27T11:09:02.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.822+0000 7f3541e00640 1 -- 172.21.15.97:0/1379309453 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f35200a3610 msgr2=0x7f35200a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:09:02.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.822+0000 7f3541e00640 1 --2- 172.21.15.97:0/1379309453 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f35200a3610 0x7f35200a5ad0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f3528000c00 tx=0x7f352802c7e0 comp rx=0 tx=0).stop 2024-03-27T11:09:02.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.822+0000 7f3541e00640 1 -- 172.21.15.97:0/1379309453 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f353c103d90 msgr2=0x7f353c199fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:09:02.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.822+0000 7f3541e00640 1 --2- 172.21.15.97:0/1379309453 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f353c103d90 0x7f353c199fd0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f352c00d770 tx=0x7f352c00dc30 comp rx=0 tx=0).stop 2024-03-27T11:09:02.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.822+0000 7f3541e00640 1 -- 172.21.15.97:0/1379309453 shutdown_connections 2024-03-27T11:09:02.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.822+0000 7f3541e00640 1 --2- 172.21.15.97:0/1379309453 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7f35200a3610 0x7f35200a5ad0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:09:02.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.822+0000 7f3541e00640 1 --2- 172.21.15.97:0/1379309453 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f353c105940 0x7f353c1a1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:09:02.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.822+0000 7f3541e00640 1 --2- 172.21.15.97:0/1379309453 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f353c104f80 0x7f353c19a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:09:02.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.822+0000 7f3541e00640 1 --2- 172.21.15.97:0/1379309453 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f353c103d90 0x7f353c199fd0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:09:02.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.822+0000 7f3541e00640 1 -- 172.21.15.97:0/1379309453 >> 172.21.15.97:0/1379309453 conn(0x7f353c0ff300 msgr2=0x7f353c1083a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:09:02.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.822+0000 7f3541e00640 1 -- 172.21.15.97:0/1379309453 shutdown_connections 2024-03-27T11:09:02.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:02.822+0000 7f3541e00640 1 -- 172.21.15.97:0/1379309453 wait complete. 2024-03-27T11:09:02.810 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-27T11:09:02.891 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:09:02 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T11:09:02.891 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:09:02 smithi097 ceph-mon[31859]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T11:09:02.891 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:09:02 smithi097 ceph-mon[31859]: from='mgr.14152 ' entity='mgr.x' 2024-03-27T11:09:02.891 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:09:02 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/2379722293' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T11:09:02.891 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:09:02 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/205102407' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-27T11:09:02.923 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:09:02 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T11:09:02.924 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:09:02 smithi177 ceph-mon[35294]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T11:09:02.924 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:09:02 smithi177 ceph-mon[35294]: from='mgr.14152 ' entity='mgr.x' 2024-03-27T11:09:02.924 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:09:02 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/2379722293' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T11:09:02.924 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:09:02 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/205102407' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-27T11:09:03.013 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:09:02 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T11:09:03.014 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:09:02 smithi114 ceph-mon[35338]: from='mgr.14152 172.21.15.97:0/212100474' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T11:09:03.014 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:09:02 smithi114 ceph-mon[35338]: from='mgr.14152 ' entity='mgr.x' 2024-03-27T11:09:03.014 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:09:02 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/2379722293' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T11:09:03.014 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:09:02 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/205102407' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-27T11:09:03.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.057+0000 7ff31e200640 1 -- 172.21.15.97:0/2351528288 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff318100da0 msgr2=0x7ff318101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:09:03.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.057+0000 7ff31e200640 1 --2- 172.21.15.97:0/2351528288 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff318100da0 0x7ff318101200 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7ff300009900 tx=0x7ff30002bcd0 comp rx=0 tx=0).stop 2024-03-27T11:09:03.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.057+0000 7ff31e200640 1 -- 172.21.15.97:0/2351528288 shutdown_connections 2024-03-27T11:09:03.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.057+0000 7ff31e200640 1 --2- 172.21.15.97:0/2351528288 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff318101740 0x7ff318101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:09:03.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.057+0000 7ff31e200640 1 --2- 172.21.15.97:0/2351528288 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff318100da0 0x7ff318101200 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:09:03.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.057+0000 7ff31e200640 1 --2- 172.21.15.97:0/2351528288 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff318100440 0x7ff318100860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:09:03.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.057+0000 7ff31e200640 1 -- 172.21.15.97:0/2351528288 >> 172.21.15.97:0/2351528288 conn(0x7ff3180fbfe0 msgr2=0x7ff3180fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:09:03.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.057+0000 7ff31e200640 1 -- 172.21.15.97:0/2351528288 shutdown_connections 2024-03-27T11:09:03.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.057+0000 7ff31e200640 1 -- 172.21.15.97:0/2351528288 wait complete. 2024-03-27T11:09:03.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.058+0000 7ff31e200640 1 Processor -- start 2024-03-27T11:09:03.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.058+0000 7ff31e200640 1 -- start start 2024-03-27T11:09:03.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.058+0000 7ff31e200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff318100440 0x7ff318199fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:09:03.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.058+0000 7ff31e200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff318100da0 0x7ff31819a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:09:03.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.058+0000 7ff31e200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff318101740 0x7ff3181a1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:09:03.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.058+0000 7ff31e200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff31810f8d0 con 0x7ff318100da0 2024-03-27T11:09:03.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.058+0000 7ff31e200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff31810f750 con 0x7ff318101740 2024-03-27T11:09:03.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.058+0000 7ff31e200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff31810fa50 con 0x7ff318100440 2024-03-27T11:09:03.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.059+0000 7ff30ea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff318100da0 0x7ff31819a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:09:03.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.059+0000 7ff30ea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff318100da0 0x7ff31819a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40510/0 (socket says 172.21.15.97:40510) 2024-03-27T11:09:03.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.059+0000 7ff30ea00640 1 -- 172.21.15.97:0/892789775 learned_addr learned my addr 172.21.15.97:0/892789775 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:09:03.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.059+0000 7ff317e00640 1 --2- 172.21.15.97:0/892789775 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff318101740 0x7ff3181a1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:09:03.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.059+0000 7ff317400640 1 --2- 172.21.15.97:0/892789775 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff318100440 0x7ff318199fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:09:03.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.059+0000 7ff30ea00640 1 -- 172.21.15.97:0/892789775 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff318100440 msgr2=0x7ff318199fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:09:03.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.059+0000 7ff30ea00640 1 --2- 172.21.15.97:0/892789775 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff318100440 0x7ff318199fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:09:03.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.059+0000 7ff30ea00640 1 -- 172.21.15.97:0/892789775 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff318101740 msgr2=0x7ff3181a1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:09:03.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.059+0000 7ff30ea00640 1 --2- 172.21.15.97:0/892789775 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff318101740 0x7ff3181a1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:09:03.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.059+0000 7ff30ea00640 1 -- 172.21.15.97:0/892789775 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3181a1c00 con 0x7ff318100da0 2024-03-27T11:09:03.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.059+0000 7ff317400640 1 --2- 172.21.15.97:0/892789775 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff318100440 0x7ff318199fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T11:09:03.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.059+0000 7ff317e00640 1 --2- 172.21.15.97:0/892789775 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff318101740 0x7ff3181a1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T11:09:03.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.059+0000 7ff30ea00640 1 --2- 172.21.15.97:0/892789775 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff318100da0 0x7ff31819a520 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7ff300000c00 tx=0x7ff30002d990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:09:03.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.060+0000 7ff314c00640 1 -- 172.21.15.97:0/892789775 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff300043020 con 0x7ff318100da0 2024-03-27T11:09:03.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.060+0000 7ff314c00640 1 -- 172.21.15.97:0/892789775 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff30002dd90 con 0x7ff318100da0 2024-03-27T11:09:03.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.060+0000 7ff31e200640 1 -- 172.21.15.97:0/892789775 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3181a1e90 con 0x7ff318100da0 2024-03-27T11:09:03.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.060+0000 7ff314c00640 1 -- 172.21.15.97:0/892789775 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff300003c20 con 0x7ff318100da0 2024-03-27T11:09:03.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.060+0000 7ff31e200640 1 -- 172.21.15.97:0/892789775 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3181a23a0 con 0x7ff318100da0 2024-03-27T11:09:03.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.061+0000 7ff31e200640 1 -- 172.21.15.97:0/892789775 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff31810b3d0 con 0x7ff318100da0 2024-03-27T11:09:03.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.063+0000 7ff314c00640 1 -- 172.21.15.97:0/892789775 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7ff300046020 con 0x7ff318100da0 2024-03-27T11:09:03.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.064+0000 7ff314c00640 1 --2- 172.21.15.97:0/892789775 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7ff2fc0a3690 0x7ff2fc0a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:09:03.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.064+0000 7ff314c00640 1 -- 172.21.15.97:0/892789775 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7ff3000f1fe0 con 0x7ff318100da0 2024-03-27T11:09:03.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.065+0000 7ff317400640 1 --2- 172.21.15.97:0/892789775 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7ff2fc0a3690 0x7ff2fc0a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:09:03.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.065+0000 7ff317400640 1 --2- 172.21.15.97:0/892789775 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7ff2fc0a3690 0x7ff2fc0a5b50 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7ff308009b40 tx=0x7ff308005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:09:03.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.069+0000 7ff314c00640 1 -- 172.21.15.97:0/892789775 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7ff30004d020 con 0x7ff318100da0 2024-03-27T11:09:03.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.269+0000 7ff31e200640 1 -- 172.21.15.97:0/892789775 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff3180627f0 con 0x7ff318100da0 2024-03-27T11:09:03.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.270+0000 7ff314c00640 1 -- 172.21.15.97:0/892789775 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) ==== 100+0+11 (secure 0 0 0) 0x7ff3000beb60 con 0x7ff318100da0 2024-03-27T11:09:03.240 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-03-27T11:09:03.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.273+0000 7ff31e200640 1 -- 172.21.15.97:0/892789775 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7ff2fc0a3690 msgr2=0x7ff2fc0a5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:09:03.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.273+0000 7ff31e200640 1 --2- 172.21.15.97:0/892789775 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7ff2fc0a3690 0x7ff2fc0a5b50 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7ff308009b40 tx=0x7ff308005e50 comp rx=0 tx=0).stop 2024-03-27T11:09:03.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.273+0000 7ff31e200640 1 -- 172.21.15.97:0/892789775 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff318100da0 msgr2=0x7ff31819a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:09:03.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.273+0000 7ff31e200640 1 --2- 172.21.15.97:0/892789775 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff318100da0 0x7ff31819a520 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7ff300000c00 tx=0x7ff30002d990 comp rx=0 tx=0).stop 2024-03-27T11:09:03.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.273+0000 7ff31e200640 1 -- 172.21.15.97:0/892789775 shutdown_connections 2024-03-27T11:09:03.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.273+0000 7ff31e200640 1 --2- 172.21.15.97:0/892789775 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7ff2fc0a3690 0x7ff2fc0a5b50 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:09:03.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.273+0000 7ff31e200640 1 --2- 172.21.15.97:0/892789775 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff318101740 0x7ff3181a1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:09:03.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.273+0000 7ff31e200640 1 --2- 172.21.15.97:0/892789775 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff318100da0 0x7ff31819a520 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:09:03.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.273+0000 7ff31e200640 1 --2- 172.21.15.97:0/892789775 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff318100440 0x7ff318199fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:09:03.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.273+0000 7ff31e200640 1 -- 172.21.15.97:0/892789775 >> 172.21.15.97:0/892789775 conn(0x7ff3180fbfe0 msgr2=0x7ff3180fd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:09:03.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.273+0000 7ff31e200640 1 -- 172.21.15.97:0/892789775 shutdown_connections 2024-03-27T11:09:03.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.273+0000 7ff31e200640 1 -- 172.21.15.97:0/892789775 wait complete. 2024-03-27T11:09:03.262 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-27T11:09:03.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.513+0000 7ff785800640 1 -- 172.21.15.97:0/2208140459 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff780076510 msgr2=0x7ff780076930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:09:03.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.513+0000 7ff785800640 1 --2- 172.21.15.97:0/2208140459 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff780076510 0x7ff780076930 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7ff770009900 tx=0x7ff77002bc30 comp rx=0 tx=0).stop 2024-03-27T11:09:03.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.513+0000 7ff785800640 1 -- 172.21.15.97:0/2208140459 shutdown_connections 2024-03-27T11:09:03.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.513+0000 7ff785800640 1 --2- 172.21.15.97:0/2208140459 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff7800780f0 0x7ff780078990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:09:03.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.513+0000 7ff785800640 1 --2- 172.21.15.97:0/2208140459 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff780077750 0x7ff780077bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:09:03.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.513+0000 7ff785800640 1 --2- 172.21.15.97:0/2208140459 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff780076510 0x7ff780076930 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:09:03.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.513+0000 7ff785800640 1 -- 172.21.15.97:0/2208140459 >> 172.21.15.97:0/2208140459 conn(0x7ff7800ff340 msgr2=0x7ff780101760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:09:03.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.513+0000 7ff785800640 1 -- 172.21.15.97:0/2208140459 shutdown_connections 2024-03-27T11:09:03.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.513+0000 7ff785800640 1 -- 172.21.15.97:0/2208140459 wait complete. 2024-03-27T11:09:03.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.514+0000 7ff785800640 1 Processor -- start 2024-03-27T11:09:03.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.514+0000 7ff785800640 1 -- start start 2024-03-27T11:09:03.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.514+0000 7ff785800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff780076510 0x7ff78019e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:09:03.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.514+0000 7ff785800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff780077750 0x7ff78019e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:09:03.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.514+0000 7ff785800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7800780f0 0x7ff7801a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:09:03.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.514+0000 7ff785800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff780113d10 con 0x7ff780076510 2024-03-27T11:09:03.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.514+0000 7ff785800640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff780113b90 con 0x7ff780077750 2024-03-27T11:09:03.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.514+0000 7ff785800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff780113e90 con 0x7ff7800780f0 2024-03-27T11:09:03.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.514+0000 7ff77ea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff780076510 0x7ff78019e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:09:03.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.515+0000 7ff77ea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff780076510 0x7ff78019e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40520/0 (socket says 172.21.15.97:40520) 2024-03-27T11:09:03.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.515+0000 7ff77ea00640 1 -- 172.21.15.97:0/946936412 learned_addr learned my addr 172.21.15.97:0/946936412 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-27T11:09:03.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.515+0000 7ff776000640 1 --2- 172.21.15.97:0/946936412 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff780077750 0x7ff78019e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:09:03.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.515+0000 7ff77f400640 1 --2- 172.21.15.97:0/946936412 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7800780f0 0x7ff7801a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:09:03.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.515+0000 7ff77ea00640 1 -- 172.21.15.97:0/946936412 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7800780f0 msgr2=0x7ff7801a59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:09:03.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.515+0000 7ff77ea00640 1 --2- 172.21.15.97:0/946936412 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7800780f0 0x7ff7801a59a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:09:03.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.515+0000 7ff77ea00640 1 -- 172.21.15.97:0/946936412 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff780077750 msgr2=0x7ff78019e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:09:03.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.515+0000 7ff77ea00640 1 --2- 172.21.15.97:0/946936412 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff780077750 0x7ff78019e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:09:03.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.515+0000 7ff77ea00640 1 -- 172.21.15.97:0/946936412 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7801a6010 con 0x7ff780076510 2024-03-27T11:09:03.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.515+0000 7ff77ea00640 1 --2- 172.21.15.97:0/946936412 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff780076510 0x7ff78019e3f0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7ff770000c00 tx=0x7ff7700042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:09:03.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.515+0000 7ff777e00640 1 -- 172.21.15.97:0/946936412 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff770043020 con 0x7ff780076510 2024-03-27T11:09:03.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.516+0000 7ff777e00640 1 -- 172.21.15.97:0/946936412 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff770004aa0 con 0x7ff780076510 2024-03-27T11:09:03.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.516+0000 7ff785800640 1 -- 172.21.15.97:0/946936412 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7801a62a0 con 0x7ff780076510 2024-03-27T11:09:03.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.516+0000 7ff777e00640 1 -- 172.21.15.97:0/946936412 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff770003d00 con 0x7ff780076510 2024-03-27T11:09:03.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.516+0000 7ff785800640 1 -- 172.21.15.97:0/946936412 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7801a6840 con 0x7ff780076510 2024-03-27T11:09:03.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.517+0000 7ff785800640 1 -- 172.21.15.97:0/946936412 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff780110700 con 0x7ff780076510 2024-03-27T11:09:03.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.519+0000 7ff777e00640 1 -- 172.21.15.97:0/946936412 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7ff770046020 con 0x7ff780076510 2024-03-27T11:09:03.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.520+0000 7ff777e00640 1 --2- 172.21.15.97:0/946936412 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7ff7640a3690 0x7ff7640a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T11:09:03.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.520+0000 7ff777e00640 1 -- 172.21.15.97:0/946936412 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8211+0+0 (secure 0 0 0) 0x7ff7700f2100 con 0x7ff780076510 2024-03-27T11:09:03.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.520+0000 7ff776000640 1 --2- 172.21.15.97:0/946936412 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7ff7640a3690 0x7ff7640a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T11:09:03.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.521+0000 7ff776000640 1 --2- 172.21.15.97:0/946936412 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7ff7640a3690 0x7ff7640a5b50 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ff78019f900 tx=0x7ff768005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T11:09:03.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.524+0000 7ff777e00640 1 -- 172.21.15.97:0/946936412 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7ff77004d020 con 0x7ff780076510 2024-03-27T11:09:03.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.736+0000 7ff785800640 1 -- 172.21.15.97:0/946936412 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff78002d370 con 0x7ff780076510 2024-03-27T11:09:03.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.737+0000 7ff777e00640 1 -- 172.21.15.97:0/946936412 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v92) ==== 81+0+582 (secure 0 0 0) 0x7ff7700bece0 con 0x7ff780076510 2024-03-27T11:09:03.707 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:0/359646 2024-03-28T10:44:39.889550+0000 2024-03-27T11:09:03.707 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6801/98682155 2024-03-28T10:44:06.019075+0000 2024-03-27T11:09:03.707 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6801/1685921853 2024-03-28T10:44:39.889550+0000 2024-03-27T11:09:03.707 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6800/98682155 2024-03-28T10:44:06.019075+0000 2024-03-27T11:09:03.707 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:0/107690680 2024-03-28T10:44:06.019075+0000 2024-03-27T11:09:03.707 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6800/1685921853 2024-03-28T10:44:39.889550+0000 2024-03-27T11:09:03.707 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:0/437043413 2024-03-28T10:44:39.889550+0000 2024-03-27T11:09:03.707 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:0/2319706992 2024-03-28T10:44:39.889550+0000 2024-03-27T11:09:03.707 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:0/3911532654 2024-03-28T10:44:06.019075+0000 2024-03-27T11:09:03.707 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:0/3347321845 2024-03-28T10:44:06.019075+0000 2024-03-27T11:09:03.708 INFO:teuthology.orchestra.run.smithi097.stderr:listed 10 entries 2024-03-27T11:09:03.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.739+0000 7ff785800640 1 -- 172.21.15.97:0/946936412 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7ff7640a3690 msgr2=0x7ff7640a5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:09:03.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.739+0000 7ff785800640 1 --2- 172.21.15.97:0/946936412 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7ff7640a3690 0x7ff7640a5b50 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ff78019f900 tx=0x7ff768005f90 comp rx=0 tx=0).stop 2024-03-27T11:09:03.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.739+0000 7ff785800640 1 -- 172.21.15.97:0/946936412 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff780076510 msgr2=0x7ff78019e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T11:09:03.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.739+0000 7ff785800640 1 --2- 172.21.15.97:0/946936412 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff780076510 0x7ff78019e3f0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7ff770000c00 tx=0x7ff7700042f0 comp rx=0 tx=0).stop 2024-03-27T11:09:03.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.740+0000 7ff785800640 1 -- 172.21.15.97:0/946936412 shutdown_connections 2024-03-27T11:09:03.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.740+0000 7ff785800640 1 --2- 172.21.15.97:0/946936412 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7800780f0 0x7ff7801a59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:09:03.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.740+0000 7ff785800640 1 --2- 172.21.15.97:0/946936412 >> [v2:172.21.15.97:6800/3955463538,v1:172.21.15.97:6801/3955463538] conn(0x7ff7640a3690 0x7ff7640a5b50 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:09:03.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.740+0000 7ff785800640 1 --2- 172.21.15.97:0/946936412 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff780077750 0x7ff78019e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:09:03.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.740+0000 7ff785800640 1 --2- 172.21.15.97:0/946936412 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff780076510 0x7ff78019e3f0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T11:09:03.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.740+0000 7ff785800640 1 -- 172.21.15.97:0/946936412 >> 172.21.15.97:0/946936412 conn(0x7ff7800ff340 msgr2=0x7ff780100e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T11:09:03.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.740+0000 7ff785800640 1 -- 172.21.15.97:0/946936412 shutdown_connections 2024-03-27T11:09:03.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-27T11:09:03.740+0000 7ff785800640 1 -- 172.21.15.97:0/946936412 wait complete. 2024-03-27T11:09:03.728 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi097.front.sepia.ceph.com... 2024-03-27T11:09:03.729 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-27T11:09:03.729 DEBUG:teuthology.orchestra.run.smithi097:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-03-27T11:09:03.842 INFO:teuthology.orchestra.run:waiting for 300 2024-03-27T11:09:03.845 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:ceph-fuse[84404]: fuse finished with error 0 and tester_r 0 2024-03-27T11:09:04.014 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:09:03 smithi114 ceph-mon[35338]: pgmap v813: 97 pgs: 97 active+clean; 1.6 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T11:09:04.014 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:09:03 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/1379309453' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-27T11:09:04.014 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:09:03 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/892789775' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-27T11:09:04.014 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:09:03 smithi114 ceph-mon[35338]: from='client.? 172.21.15.97:0/946936412' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-27T11:09:04.138 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:09:03 smithi097 ceph-mon[31859]: pgmap v813: 97 pgs: 97 active+clean; 1.6 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T11:09:04.138 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:09:03 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/1379309453' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-27T11:09:04.138 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:09:03 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/892789775' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-27T11:09:04.138 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:09:03 smithi097 ceph-mon[31859]: from='client.? 172.21.15.97:0/946936412' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-27T11:09:04.173 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:09:03 smithi177 ceph-mon[35294]: pgmap v813: 97 pgs: 97 active+clean; 1.6 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T11:09:04.173 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:09:03 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/1379309453' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-27T11:09:04.174 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:09:03 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/892789775' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-27T11:09:04.174 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:09:03 smithi177 ceph-mon[35294]: from='client.? 172.21.15.97:0/946936412' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-27T11:09:06.014 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:09:05 smithi114 ceph-mon[35338]: pgmap v814: 97 pgs: 97 active+clean; 1.6 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-27T11:09:06.137 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:09:05 smithi097 ceph-mon[31859]: pgmap v814: 97 pgs: 97 active+clean; 1.6 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-27T11:09:06.173 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:09:05 smithi177 ceph-mon[35294]: pgmap v814: 97 pgs: 97 active+clean; 1.6 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-27T11:09:08.014 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:09:07 smithi114 ceph-mon[35338]: pgmap v815: 97 pgs: 97 active+clean; 1.6 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-27T11:09:08.138 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:09:07 smithi097 ceph-mon[31859]: pgmap v815: 97 pgs: 97 active+clean; 1.6 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-27T11:09:08.173 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:09:07 smithi177 ceph-mon[35294]: pgmap v815: 97 pgs: 97 active+clean; 1.6 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-27T11:09:09.944 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2024-03-27T11:09:09.944 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-27T11:09:09.944 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-27T11:09:10.013 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:09:09 smithi114 ceph-mon[35338]: pgmap v816: 97 pgs: 97 active+clean; 1.6 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-03-27T11:09:10.017 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-27T11:09:10.017 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2024-03-27T11:09:10.117 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-27T11:09:10.118 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-27T11:09:10.118 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-27T11:09:10.137 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:09:09 smithi097 ceph-mon[31859]: pgmap v816: 97 pgs: 97 active+clean; 1.6 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-03-27T11:09:10.173 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:09:09 smithi177 ceph-mon[35294]: pgmap v816: 97 pgs: 97 active+clean; 1.6 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-03-27T11:09:10.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-27T11:09:10.222 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link delete ceph-brx 2024-03-27T11:09:10.386 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-03-27T11:09:10.396 INFO:tasks.cephadm:Teardown begin 2024-03-27T11:09:10.396 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_155268c4e432a12433aa833f174f9fe3b1016ae0/qa/tasks/cephadm.py", line 2230, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_155268c4e432a12433aa833f174f9fe3b1016ae0/qa/tasks/fwd_scrub.py", line 164, in task stop_all_fwd_scrubbers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_155268c4e432a12433aa833f174f9fe3b1016ae0/qa/tasks/fwd_scrub.py", line 99, in stop_all_fwd_scrubbers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: rank damage found: {'backtrace'} 2024-03-27T11:09:10.398 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-27T11:09:10.502 DEBUG:teuthology.orchestra.run.smithi114:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-27T11:09:10.601 DEBUG:teuthology.orchestra.run.smithi177:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-27T11:09:10.697 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-03-27T11:09:10.698 DEBUG:teuthology.orchestra.run.smithi097:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-27T11:09:10.751 DEBUG:teuthology.orchestra.run.smithi114:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-27T11:09:10.801 DEBUG:teuthology.orchestra.run.smithi177:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-27T11:09:10.857 INFO:tasks.cephadm:Stopping all daemons... 2024-03-27T11:09:10.857 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-03-27T11:09:10.857 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-adca88c4-ec26-11ee-95d0-87774f69a715@mon.a 2024-03-27T11:09:11.388 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:09:11 smithi097 systemd[1]: Stopping Ceph mon.a for adca88c4-ec26-11ee-95d0-87774f69a715... 2024-03-27T11:09:11.888 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:09:11 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mon-a[31855]: 2024-03-27T11:09:11.532+0000 7fd0ee200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-27T11:09:11.888 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:09:11 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mon-a[31855]: 2024-03-27T11:09:11.532+0000 7fd0ee200700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-03-27T11:09:11.889 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:09:11 smithi097 podman[96721]: 2024-03-27 11:09:11.595647217 +0000 UTC m=+0.262438921 container died 08bedf620c2406d1f45ed195cb2a75fd28bfa2ca98b7b8f0115c3ba5f467131d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mon-a, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240326.131558-155268c, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-03-27T11:09:12.256 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:09:12 smithi097 podman[96721]: 2024-03-27 11:09:12.076754706 +0000 UTC m=+0.743546439 container cleanup 08bedf620c2406d1f45ed195cb2a75fd28bfa2ca98b7b8f0115c3ba5f467131d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mon-a, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, RELEASE=wip-rishabh-testing-20240326.131558-155268c, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=) 2024-03-27T11:09:12.256 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:09:12 smithi097 bash[96721]: ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mon-a 2024-03-27T11:09:12.638 INFO:journalctl@ceph.mon.a.smithi097.stdout:Mar 27 11:09:12 smithi097 podman[96735]: 2024-03-27 11:09:12.286087539 +0000 UTC m=+0.680254422 container remove 08bedf620c2406d1f45ed195cb2a75fd28bfa2ca98b7b8f0115c3ba5f467131d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mon-a, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, ceph=True, maintainer=Guillaume Abrioux ) 2024-03-27T11:09:12.839 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-adca88c4-ec26-11ee-95d0-87774f69a715@mon.a.service' 2024-03-27T11:09:12.958 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-27T11:09:12.959 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-03-27T11:09:12.959 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-03-27T11:09:12.959 DEBUG:teuthology.orchestra.run.smithi114:> sudo systemctl stop ceph-adca88c4-ec26-11ee-95d0-87774f69a715@mon.b 2024-03-27T11:09:13.512 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:09:13 smithi114 systemd[1]: Stopping Ceph mon.b for adca88c4-ec26-11ee-95d0-87774f69a715... 2024-03-27T11:09:13.763 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:09:13 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mon-b[35334]: 2024-03-27T11:09:13.542+0000 7f6609400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-27T11:09:13.764 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:09:13 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mon-b[35334]: 2024-03-27T11:09:13.542+0000 7f6609400700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-03-27T11:09:13.764 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:09:13 smithi114 podman[70334]: 2024-03-27 11:09:13.757174341 +0000 UTC m=+0.368593657 container died be7a212c5f9df21ebb1eeae7ac16109ac796a86c64eba1e50b8ce424ae0d0033 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mon-b, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.build-date=20240326, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-03-27T11:09:14.418 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:09:14 smithi114 podman[70334]: 2024-03-27 11:09:14.229299453 +0000 UTC m=+0.840718774 container cleanup be7a212c5f9df21ebb1eeae7ac16109ac796a86c64eba1e50b8ce424ae0d0033 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mon-b, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD) 2024-03-27T11:09:14.418 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:09:14 smithi114 bash[70334]: ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mon-b 2024-03-27T11:09:14.763 INFO:journalctl@ceph.mon.b.smithi114.stdout:Mar 27 11:09:14 smithi114 podman[70349]: 2024-03-27 11:09:14.449243728 +0000 UTC m=+0.670480505 container remove be7a212c5f9df21ebb1eeae7ac16109ac796a86c64eba1e50b8ce424ae0d0033 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mon-b, CEPH_POINT_RELEASE=, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-03-27T11:09:14.850 DEBUG:teuthology.orchestra.run.smithi114:> sudo pkill -f 'journalctl -f -n 0 -u ceph-adca88c4-ec26-11ee-95d0-87774f69a715@mon.b.service' 2024-03-27T11:09:14.975 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-27T11:09:14.975 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-03-27T11:09:14.975 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-03-27T11:09:14.975 DEBUG:teuthology.orchestra.run.smithi177:> sudo systemctl stop ceph-adca88c4-ec26-11ee-95d0-87774f69a715@mon.c 2024-03-27T11:09:15.424 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:09:15 smithi177 systemd[1]: Stopping Ceph mon.c for adca88c4-ec26-11ee-95d0-87774f69a715... 2024-03-27T11:09:15.880 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:09:15 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mon-c[35268]: 2024-03-27T11:09:15.539+0000 7fdd4ba00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-27T11:09:15.880 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:09:15 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mon-c[35268]: 2024-03-27T11:09:15.539+0000 7fdd4ba00700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-03-27T11:09:15.880 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:09:15 smithi177 podman[71244]: 2024-03-27 11:09:15.602103911 +0000 UTC m=+0.228337280 container died 181e3ed3cffed8391460cf6cc9fd5d696fb08e63795ac414596c49fb46791dca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mon-c, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-03-27T11:09:16.173 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:09:16 smithi177 podman[71244]: 2024-03-27 11:09:16.005955105 +0000 UTC m=+0.632188477 container cleanup 181e3ed3cffed8391460cf6cc9fd5d696fb08e63795ac414596c49fb46791dca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mon-c, GIT_CLEAN=True, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True) 2024-03-27T11:09:16.173 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:09:16 smithi177 bash[71244]: ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mon-c 2024-03-27T11:09:16.570 INFO:journalctl@ceph.mon.c.smithi177.stdout:Mar 27 11:09:16 smithi177 podman[71258]: 2024-03-27 11:09:16.34967372 +0000 UTC m=+0.728479142 container remove 181e3ed3cffed8391460cf6cc9fd5d696fb08e63795ac414596c49fb46791dca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mon-c, org.label-schema.license=GPLv2, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.build-date=20240326, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-03-27T11:09:16.595 DEBUG:teuthology.orchestra.run.smithi177:> sudo pkill -f 'journalctl -f -n 0 -u ceph-adca88c4-ec26-11ee-95d0-87774f69a715@mon.c.service' 2024-03-27T11:09:16.747 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-27T11:09:16.748 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-03-27T11:09:16.748 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-03-27T11:09:16.748 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-adca88c4-ec26-11ee-95d0-87774f69a715@mgr.x 2024-03-27T11:09:17.138 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 11:09:16 smithi097 systemd[1]: Stopping Ceph mgr.x for adca88c4-ec26-11ee-95d0-87774f69a715... 2024-03-27T11:09:17.638 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 11:09:17 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: 2024-03-27T11:09:17.337+0000 7fb33c200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-27T11:09:17.638 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 11:09:17 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x[32086]: 2024-03-27T11:09:17.337+0000 7fb33c200700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-03-27T11:09:17.639 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 11:09:17 smithi097 podman[96842]: 2024-03-27 11:09:17.429814165 +0000 UTC m=+0.226636970 container died 662f8153b4104b0eb187f8544269161a090eebbf78819b65b358ee399dcb4086 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, GIT_CLEAN=True, ceph=True, RELEASE=wip-rishabh-testing-20240326.131558-155268c, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-03-27T11:09:18.138 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 11:09:17 smithi097 podman[96842]: 2024-03-27 11:09:17.931193858 +0000 UTC m=+0.728016603 container cleanup 662f8153b4104b0eb187f8544269161a090eebbf78819b65b358ee399dcb4086 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240326.131558-155268c, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-03-27T11:09:18.138 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 11:09:17 smithi097 bash[96842]: ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x 2024-03-27T11:09:18.638 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Mar 27 11:09:18 smithi097 podman[96856]: 2024-03-27 11:09:18.188094997 +0000 UTC m=+0.737899389 container remove 662f8153b4104b0eb187f8544269161a090eebbf78819b65b358ee399dcb4086 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-x, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, RELEASE=wip-rishabh-testing-20240326.131558-155268c, ceph=True, org.label-schema.license=GPLv2) 2024-03-27T11:09:18.698 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-adca88c4-ec26-11ee-95d0-87774f69a715@mgr.x.service' 2024-03-27T11:09:18.815 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-27T11:09:18.815 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-03-27T11:09:18.816 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-03-27T11:09:18.816 DEBUG:teuthology.orchestra.run.smithi114:> sudo systemctl stop ceph-adca88c4-ec26-11ee-95d0-87774f69a715@mgr.y 2024-03-27T11:09:19.263 INFO:journalctl@ceph.mgr.y.smithi114.stdout:Mar 27 11:09:18 smithi114 systemd[1]: Stopping Ceph mgr.y for adca88c4-ec26-11ee-95d0-87774f69a715... 2024-03-27T11:09:19.763 INFO:journalctl@ceph.mgr.y.smithi114.stdout:Mar 27 11:09:19 smithi114 podman[70452]: 2024-03-27 11:09:19.537916441 +0000 UTC m=+0.270269146 container died 72bfb2b6dd9bd36c98d50ba3ef8f8ffd9e75cf29306fc8a107727e738ea9bc72 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-y, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.schema-version=1.0) 2024-03-27T11:09:20.231 INFO:journalctl@ceph.mgr.y.smithi114.stdout:Mar 27 11:09:19 smithi114 podman[70452]: 2024-03-27 11:09:19.986134855 +0000 UTC m=+0.718487564 container cleanup 72bfb2b6dd9bd36c98d50ba3ef8f8ffd9e75cf29306fc8a107727e738ea9bc72 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-y, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, RELEASE=wip-rishabh-testing-20240326.131558-155268c, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-27T11:09:20.231 INFO:journalctl@ceph.mgr.y.smithi114.stdout:Mar 27 11:09:19 smithi114 bash[70452]: ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-y 2024-03-27T11:09:20.513 INFO:journalctl@ceph.mgr.y.smithi114.stdout:Mar 27 11:09:20 smithi114 podman[70466]: 2024-03-27 11:09:20.262484329 +0000 UTC m=+0.706411897 container remove 72bfb2b6dd9bd36c98d50ba3ef8f8ffd9e75cf29306fc8a107727e738ea9bc72 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-y, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326) 2024-03-27T11:09:20.514 INFO:journalctl@ceph.mgr.y.smithi114.stdout:Mar 27 11:09:20 smithi114 systemd[1]: ceph-adca88c4-ec26-11ee-95d0-87774f69a715@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-03-27T11:09:20.655 DEBUG:teuthology.orchestra.run.smithi114:> sudo pkill -f 'journalctl -f -n 0 -u ceph-adca88c4-ec26-11ee-95d0-87774f69a715@mgr.y.service' 2024-03-27T11:09:20.756 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-27T11:09:20.756 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-03-27T11:09:20.757 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-03-27T11:09:20.757 DEBUG:teuthology.orchestra.run.smithi177:> sudo systemctl stop ceph-adca88c4-ec26-11ee-95d0-87774f69a715@mgr.z 2024-03-27T11:09:21.173 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Mar 27 11:09:20 smithi177 systemd[1]: Stopping Ceph mgr.z for adca88c4-ec26-11ee-95d0-87774f69a715... 2024-03-27T11:09:21.674 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Mar 27 11:09:21 smithi177 podman[71374]: 2024-03-27 11:09:21.440065464 +0000 UTC m=+0.219344131 container died f9408119d0ecefc054f81700b1359a244e85efc9105597f499358c382cec6714 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-z, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=) 2024-03-27T11:09:22.172 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Mar 27 11:09:21 smithi177 podman[71374]: 2024-03-27 11:09:21.986132864 +0000 UTC m=+0.765411499 container cleanup f9408119d0ecefc054f81700b1359a244e85efc9105597f499358c382cec6714 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-z, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e) 2024-03-27T11:09:22.172 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Mar 27 11:09:21 smithi177 bash[71374]: ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-z 2024-03-27T11:09:22.423 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Mar 27 11:09:22 smithi177 podman[71388]: 2024-03-27 11:09:22.203872625 +0000 UTC m=+0.754433633 container remove f9408119d0ecefc054f81700b1359a244e85efc9105597f499358c382cec6714 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mgr-z, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-27T11:09:22.423 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Mar 27 11:09:22 smithi177 systemd[1]: ceph-adca88c4-ec26-11ee-95d0-87774f69a715@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-03-27T11:09:22.628 DEBUG:teuthology.orchestra.run.smithi177:> sudo pkill -f 'journalctl -f -n 0 -u ceph-adca88c4-ec26-11ee-95d0-87774f69a715@mgr.z.service' 2024-03-27T11:09:22.723 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-27T11:09:22.723 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-03-27T11:09:22.723 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-03-27T11:09:22.723 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-adca88c4-ec26-11ee-95d0-87774f69a715@osd.0 2024-03-27T11:09:23.137 INFO:journalctl@ceph.osd.0.smithi097.stdout:Mar 27 11:09:22 smithi097 systemd[1]: Stopping Ceph osd.0 for adca88c4-ec26-11ee-95d0-87774f69a715... 2024-03-27T11:09:23.638 INFO:journalctl@ceph.osd.0.smithi097.stdout:Mar 27 11:09:23 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-0[46880]: 2024-03-27T11:09:23.316+0000 7f96ce200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-27T11:09:23.639 INFO:journalctl@ceph.osd.0.smithi097.stdout:Mar 27 11:09:23 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-0[46880]: 2024-03-27T11:09:23.316+0000 7f96ce200700 -1 osd.0 92 *** Got signal Terminated *** 2024-03-27T11:09:23.639 INFO:journalctl@ceph.osd.0.smithi097.stdout:Mar 27 11:09:23 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-0[46880]: 2024-03-27T11:09:23.316+0000 7f96ce200700 -1 osd.0 92 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-27T11:09:29.194 INFO:journalctl@ceph.osd.0.smithi097.stdout:Mar 27 11:09:28 smithi097 podman[96964]: 2024-03-27 11:09:28.944676484 +0000 UTC m=+5.763115351 container died 50851d00ea4c08b42bce8a07c1440a8143fe09292a9b070d60358ad5fc345a99 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-27T11:09:29.638 INFO:journalctl@ceph.osd.0.smithi097.stdout:Mar 27 11:09:29 smithi097 podman[96964]: 2024-03-27 11:09:29.360080207 +0000 UTC m=+6.178519068 container cleanup 50851d00ea4c08b42bce8a07c1440a8143fe09292a9b070d60358ad5fc345a99 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-0, GIT_CLEAN=True, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-03-27T11:09:29.638 INFO:journalctl@ceph.osd.0.smithi097.stdout:Mar 27 11:09:29 smithi097 bash[96964]: ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-0 2024-03-27T11:09:30.137 INFO:journalctl@ceph.osd.0.smithi097.stdout:Mar 27 11:09:29 smithi097 podman[96983]: 2024-03-27 11:09:29.694962775 +0000 UTC m=+0.736900670 container remove 50851d00ea4c08b42bce8a07c1440a8143fe09292a9b070d60358ad5fc345a99 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-0, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240326.131558-155268c, ceph=True) 2024-03-27T11:09:30.800 INFO:journalctl@ceph.osd.0.smithi097.stdout:Mar 27 11:09:30 smithi097 podman[97073]: 2024-03-27 11:09:30.453989162 +0000 UTC m=+0.036887154 image pull b379bbd8da6b5d0fa9a3523941efac213b6815e1cbf05c6507954f0a3fe7a801 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772 2024-03-27T11:09:30.800 INFO:journalctl@ceph.osd.0.smithi097.stdout:Mar 27 11:09:30 smithi097 podman[97073]: 2024-03-27 11:09:30.562631584 +0000 UTC m=+0.145529601 container create a038986af138b9ae3b5bd7969fe064b936a45a6bb18db44cd726648cd2695695 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-0-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-03-27T11:09:31.138 INFO:journalctl@ceph.osd.0.smithi097.stdout:Mar 27 11:09:30 smithi097 podman[97073]: 2024-03-27 11:09:30.83123047 +0000 UTC m=+0.414128530 container init a038986af138b9ae3b5bd7969fe064b936a45a6bb18db44cd726648cd2695695 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-03-27T11:09:31.138 INFO:journalctl@ceph.osd.0.smithi097.stdout:Mar 27 11:09:30 smithi097 podman[97073]: 2024-03-27 11:09:30.84203472 +0000 UTC m=+0.424932741 container start a038986af138b9ae3b5bd7969fe064b936a45a6bb18db44cd726648cd2695695 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-rishabh-testing-20240326.131558-155268c) 2024-03-27T11:09:31.138 INFO:journalctl@ceph.osd.0.smithi097.stdout:Mar 27 11:09:30 smithi097 podman[97073]: 2024-03-27 11:09:30.998510792 +0000 UTC m=+0.581408856 container attach a038986af138b9ae3b5bd7969fe064b936a45a6bb18db44cd726648cd2695695 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-0-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.vendor=CentOS) 2024-03-27T11:09:32.092 INFO:journalctl@ceph.osd.0.smithi097.stdout:Mar 27 11:09:31 smithi097 podman[97073]: 2024-03-27 11:09:31.735515765 +0000 UTC m=+1.318413816 container died a038986af138b9ae3b5bd7969fe064b936a45a6bb18db44cd726648cd2695695 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-0-deactivate, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240326.131558-155268c, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e) 2024-03-27T11:09:32.495 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-adca88c4-ec26-11ee-95d0-87774f69a715@osd.0.service' 2024-03-27T11:09:32.611 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-27T11:09:32.612 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-03-27T11:09:32.612 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-03-27T11:09:32.612 DEBUG:teuthology.orchestra.run.smithi114:> sudo systemctl stop ceph-adca88c4-ec26-11ee-95d0-87774f69a715@osd.1 2024-03-27T11:09:33.013 INFO:journalctl@ceph.osd.1.smithi114.stdout:Mar 27 11:09:32 smithi114 systemd[1]: Stopping Ceph osd.1 for adca88c4-ec26-11ee-95d0-87774f69a715... 2024-03-27T11:09:33.514 INFO:journalctl@ceph.osd.1.smithi114.stdout:Mar 27 11:09:33 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-1[39933]: 2024-03-27T11:09:33.152+0000 7f73ca600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-27T11:09:33.514 INFO:journalctl@ceph.osd.1.smithi114.stdout:Mar 27 11:09:33 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-1[39933]: 2024-03-27T11:09:33.152+0000 7f73ca600700 -1 osd.1 92 *** Got signal Terminated *** 2024-03-27T11:09:33.514 INFO:journalctl@ceph.osd.1.smithi114.stdout:Mar 27 11:09:33 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-1[39933]: 2024-03-27T11:09:33.152+0000 7f73ca600700 -1 osd.1 92 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-27T11:09:39.014 INFO:journalctl@ceph.osd.1.smithi114.stdout:Mar 27 11:09:38 smithi114 podman[70575]: 2024-03-27 11:09:38.774213155 +0000 UTC m=+5.747604598 container died 079e646c3b07c4f50bf0a50351465e52212081ad232329eabd7e8766147282d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-1, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240326.131558-155268c) 2024-03-27T11:09:39.513 INFO:journalctl@ceph.osd.1.smithi114.stdout:Mar 27 11:09:39 smithi114 podman[70575]: 2024-03-27 11:09:39.250910631 +0000 UTC m=+6.224302100 container cleanup 079e646c3b07c4f50bf0a50351465e52212081ad232329eabd7e8766147282d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-1, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-03-27T11:09:39.513 INFO:journalctl@ceph.osd.1.smithi114.stdout:Mar 27 11:09:39 smithi114 bash[70575]: ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-1 2024-03-27T11:09:40.013 INFO:journalctl@ceph.osd.1.smithi114.stdout:Mar 27 11:09:39 smithi114 podman[70590]: 2024-03-27 11:09:39.651309139 +0000 UTC m=+0.862261452 container remove 079e646c3b07c4f50bf0a50351465e52212081ad232329eabd7e8766147282d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-1, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240326.131558-155268c, io.buildah.version=1.33.5) 2024-03-27T11:09:40.468 INFO:journalctl@ceph.osd.1.smithi114.stdout:Mar 27 11:09:40 smithi114 podman[70676]: 2024-03-27 11:09:40.080840794 +0000 UTC m=+0.047992983 image pull b379bbd8da6b5d0fa9a3523941efac213b6815e1cbf05c6507954f0a3fe7a801 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772 2024-03-27T11:09:40.468 INFO:journalctl@ceph.osd.1.smithi114.stdout:Mar 27 11:09:40 smithi114 podman[70676]: 2024-03-27 11:09:40.209734496 +0000 UTC m=+0.176886681 container create 2599802fb7023c313fba691b51a74376fdabc816ab7438ce83cb3cd4c9ca7241 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-1-deactivate, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, RELEASE=wip-rishabh-testing-20240326.131558-155268c) 2024-03-27T11:09:40.468 INFO:journalctl@ceph.osd.1.smithi114.stdout:Mar 27 11:09:40 smithi114 podman[70676]: 2024-03-27 11:09:40.498731979 +0000 UTC m=+0.465884261 container init 2599802fb7023c313fba691b51a74376fdabc816ab7438ce83cb3cd4c9ca7241 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-1-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-27T11:09:40.764 INFO:journalctl@ceph.osd.1.smithi114.stdout:Mar 27 11:09:40 smithi114 podman[70676]: 2024-03-27 11:09:40.508812937 +0000 UTC m=+0.475965123 container start 2599802fb7023c313fba691b51a74376fdabc816ab7438ce83cb3cd4c9ca7241 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-1-deactivate, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.vendor=CentOS) 2024-03-27T11:09:40.764 INFO:journalctl@ceph.osd.1.smithi114.stdout:Mar 27 11:09:40 smithi114 podman[70676]: 2024-03-27 11:09:40.612919911 +0000 UTC m=+0.580072117 container attach 2599802fb7023c313fba691b51a74376fdabc816ab7438ce83cb3cd4c9ca7241 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-1-deactivate, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e) 2024-03-27T11:09:41.263 INFO:journalctl@ceph.osd.1.smithi114.stdout:Mar 27 11:09:40 smithi114 podman[70676]: 2024-03-27 11:09:40.973950861 +0000 UTC m=+0.941103041 container died 2599802fb7023c313fba691b51a74376fdabc816ab7438ce83cb3cd4c9ca7241 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-1-deactivate, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e) 2024-03-27T11:09:41.791 DEBUG:teuthology.orchestra.run.smithi114:> sudo pkill -f 'journalctl -f -n 0 -u ceph-adca88c4-ec26-11ee-95d0-87774f69a715@osd.1.service' 2024-03-27T11:09:41.898 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-27T11:09:41.899 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-03-27T11:09:41.899 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-03-27T11:09:41.899 DEBUG:teuthology.orchestra.run.smithi177:> sudo systemctl stop ceph-adca88c4-ec26-11ee-95d0-87774f69a715@osd.2 2024-03-27T11:09:42.423 INFO:journalctl@ceph.osd.2.smithi177.stdout:Mar 27 11:09:42 smithi177 systemd[1]: Stopping Ceph osd.2 for adca88c4-ec26-11ee-95d0-87774f69a715... 2024-03-27T11:09:42.924 INFO:journalctl@ceph.osd.2.smithi177.stdout:Mar 27 11:09:42 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-2[40422]: 2024-03-27T11:09:42.494+0000 7f78b4e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-27T11:09:42.924 INFO:journalctl@ceph.osd.2.smithi177.stdout:Mar 27 11:09:42 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-2[40422]: 2024-03-27T11:09:42.494+0000 7f78b4e00700 -1 osd.2 92 *** Got signal Terminated *** 2024-03-27T11:09:42.924 INFO:journalctl@ceph.osd.2.smithi177.stdout:Mar 27 11:09:42 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-2[40422]: 2024-03-27T11:09:42.494+0000 7f78b4e00700 -1 osd.2 92 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-27T11:09:48.505 INFO:journalctl@ceph.osd.2.smithi177.stdout:Mar 27 11:09:48 smithi177 podman[71496]: 2024-03-27 11:09:48.249981394 +0000 UTC m=+5.874782411 container died 17f47ec3825545ebda174b99e6bb04d51417d58a825247e81935158a5f7a5724 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e) 2024-03-27T11:09:48.883 INFO:journalctl@ceph.osd.2.smithi177.stdout:Mar 27 11:09:48 smithi177 podman[71496]: 2024-03-27 11:09:48.663882341 +0000 UTC m=+6.288683377 container cleanup 17f47ec3825545ebda174b99e6bb04d51417d58a825247e81935158a5f7a5724 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.schema-version=1.0) 2024-03-27T11:09:48.883 INFO:journalctl@ceph.osd.2.smithi177.stdout:Mar 27 11:09:48 smithi177 bash[71496]: ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-2 2024-03-27T11:09:49.173 INFO:journalctl@ceph.osd.2.smithi177.stdout:Mar 27 11:09:48 smithi177 podman[71512]: 2024-03-27 11:09:48.915040283 +0000 UTC m=+0.660171471 container remove 17f47ec3825545ebda174b99e6bb04d51417d58a825247e81935158a5f7a5724 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-2, org.label-schema.build-date=20240326, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-03-27T11:09:49.815 INFO:journalctl@ceph.osd.2.smithi177.stdout:Mar 27 11:09:49 smithi177 podman[71595]: 2024-03-27 11:09:49.484675668 +0000 UTC m=+0.031477509 image pull b379bbd8da6b5d0fa9a3523941efac213b6815e1cbf05c6507954f0a3fe7a801 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772 2024-03-27T11:09:49.815 INFO:journalctl@ceph.osd.2.smithi177.stdout:Mar 27 11:09:49 smithi177 podman[71595]: 2024-03-27 11:09:49.607600498 +0000 UTC m=+0.154402372 container create 54bb37aaf01cef8200f2860350e3fff8873d34a8a1baf0106496fa6829814df7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-2-deactivate, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326) 2024-03-27T11:09:49.888 INFO:journalctl@ceph.osd.3.smithi097.stdout:Mar 27 11:09:49 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-3[53399]: 2024-03-27T11:09:49.430+0000 7f9d46c00700 -1 osd.3 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:25.247533+0000 front 2024-03-27T11:09:25.246502+0000 (oldest deadline 2024-03-27T11:09:49.346631+0000) 2024-03-27T11:09:50.120 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:09:49 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:09:49.822+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.380254+0000 front 2024-03-27T11:09:23.380098+0000 (oldest deadline 2024-03-27T11:09:49.280089+0000) 2024-03-27T11:09:50.174 INFO:journalctl@ceph.osd.2.smithi177.stdout:Mar 27 11:09:49 smithi177 podman[71595]: 2024-03-27 11:09:49.845647049 +0000 UTC m=+0.392448953 container init 54bb37aaf01cef8200f2860350e3fff8873d34a8a1baf0106496fa6829814df7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-2-deactivate, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-03-27T11:09:50.174 INFO:journalctl@ceph.osd.2.smithi177.stdout:Mar 27 11:09:49 smithi177 podman[71595]: 2024-03-27 11:09:49.857641311 +0000 UTC m=+0.404443150 container start 54bb37aaf01cef8200f2860350e3fff8873d34a8a1baf0106496fa6829814df7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-2-deactivate, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-27T11:09:50.174 INFO:journalctl@ceph.osd.2.smithi177.stdout:Mar 27 11:09:49 smithi177 podman[71595]: 2024-03-27 11:09:49.935120652 +0000 UTC m=+0.481922510 container attach 54bb37aaf01cef8200f2860350e3fff8873d34a8a1baf0106496fa6829814df7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-2-deactivate, ceph=True, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240326.131558-155268c, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-03-27T11:09:50.339 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:09:49 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:09:49.925+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.654238+0000 front 2024-03-27T11:09:23.654318+0000 (oldest deadline 2024-03-27T11:09:48.954440+0000) 2024-03-27T11:09:50.626 INFO:journalctl@ceph.osd.2.smithi177.stdout:Mar 27 11:09:50 smithi177 podman[71595]: 2024-03-27 11:09:50.39384495 +0000 UTC m=+0.940646810 container died 54bb37aaf01cef8200f2860350e3fff8873d34a8a1baf0106496fa6829814df7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-2-deactivate, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-03-27T11:09:50.637 INFO:journalctl@ceph.osd.3.smithi097.stdout:Mar 27 11:09:50 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-3[53399]: 2024-03-27T11:09:50.460+0000 7f9d46c00700 -1 osd.3 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:25.247533+0000 front 2024-03-27T11:09:25.246502+0000 (oldest deadline 2024-03-27T11:09:49.346631+0000) 2024-03-27T11:09:50.638 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 27 11:09:50 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-6[60366]: 2024-03-27T11:09:50.369+0000 7f8ead400700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:27.412885+0000 front 2024-03-27T11:09:27.412671+0000 (oldest deadline 2024-03-27T11:09:49.712654+0000) 2024-03-27T11:09:50.958 DEBUG:teuthology.orchestra.run.smithi177:> sudo pkill -f 'journalctl -f -n 0 -u ceph-adca88c4-ec26-11ee-95d0-87774f69a715@osd.2.service' 2024-03-27T11:09:51.061 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-27T11:09:51.061 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-03-27T11:09:51.062 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-03-27T11:09:51.062 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-adca88c4-ec26-11ee-95d0-87774f69a715@osd.3 2024-03-27T11:09:51.137 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:09:50 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:09:50.886+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.654238+0000 front 2024-03-27T11:09:23.654318+0000 (oldest deadline 2024-03-27T11:09:48.954440+0000) 2024-03-27T11:09:51.263 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:09:50 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:09:50.861+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.380254+0000 front 2024-03-27T11:09:23.380098+0000 (oldest deadline 2024-03-27T11:09:49.280089+0000) 2024-03-27T11:09:51.372 INFO:journalctl@ceph.osd.5.smithi177.stdout:Mar 27 11:09:50 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-5[47487]: 2024-03-27T11:09:50.980+0000 7fe117000700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:25.457341+0000 front 2024-03-27T11:09:25.457314+0000 (oldest deadline 2024-03-27T11:09:50.157224+0000) 2024-03-27T11:09:51.452 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 27 11:09:51 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-6[60366]: 2024-03-27T11:09:51.372+0000 7f8ead400700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:27.412885+0000 front 2024-03-27T11:09:27.412671+0000 (oldest deadline 2024-03-27T11:09:49.712654+0000) 2024-03-27T11:09:51.452 INFO:journalctl@ceph.osd.3.smithi097.stdout:Mar 27 11:09:51 smithi097 systemd[1]: Stopping Ceph osd.3 for adca88c4-ec26-11ee-95d0-87774f69a715... 2024-03-27T11:09:51.673 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:09:51 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:09:51.402+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.236297+0000 front 2024-03-27T11:09:28.236728+0000 (oldest deadline 2024-03-27T11:09:51.136525+0000) 2024-03-27T11:09:51.887 INFO:journalctl@ceph.osd.3.smithi097.stdout:Mar 27 11:09:51 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-3[53399]: 2024-03-27T11:09:51.482+0000 7f9d46c00700 -1 osd.3 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:25.247533+0000 front 2024-03-27T11:09:25.246502+0000 (oldest deadline 2024-03-27T11:09:49.346631+0000) 2024-03-27T11:09:51.887 INFO:journalctl@ceph.osd.3.smithi097.stdout:Mar 27 11:09:51 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-3[53399]: 2024-03-27T11:09:51.599+0000 7f9d50400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-27T11:09:51.887 INFO:journalctl@ceph.osd.3.smithi097.stdout:Mar 27 11:09:51 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-3[53399]: 2024-03-27T11:09:51.599+0000 7f9d50400700 -1 osd.3 92 *** Got signal Terminated *** 2024-03-27T11:09:51.888 INFO:journalctl@ceph.osd.3.smithi097.stdout:Mar 27 11:09:51 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-3[53399]: 2024-03-27T11:09:51.599+0000 7f9d50400700 -1 osd.3 92 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-27T11:09:52.209 INFO:journalctl@ceph.osd.5.smithi177.stdout:Mar 27 11:09:51 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-5[47487]: 2024-03-27T11:09:51.960+0000 7fe117000700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:25.457341+0000 front 2024-03-27T11:09:25.457314+0000 (oldest deadline 2024-03-27T11:09:50.157224+0000) 2024-03-27T11:09:52.263 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:09:51 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:09:51.840+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.380254+0000 front 2024-03-27T11:09:23.380098+0000 (oldest deadline 2024-03-27T11:09:49.280089+0000) 2024-03-27T11:09:52.311 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:09:51 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:09:51.929+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.654238+0000 front 2024-03-27T11:09:23.654318+0000 (oldest deadline 2024-03-27T11:09:48.954440+0000) 2024-03-27T11:09:52.637 INFO:journalctl@ceph.osd.3.smithi097.stdout:Mar 27 11:09:52 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-3[53399]: 2024-03-27T11:09:52.440+0000 7f9d46c00700 -1 osd.3 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:25.247533+0000 front 2024-03-27T11:09:25.246502+0000 (oldest deadline 2024-03-27T11:09:49.346631+0000) 2024-03-27T11:09:52.638 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 27 11:09:52 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-6[60366]: 2024-03-27T11:09:52.341+0000 7f8ead400700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:27.412885+0000 front 2024-03-27T11:09:27.412671+0000 (oldest deadline 2024-03-27T11:09:49.712654+0000) 2024-03-27T11:09:52.673 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:09:52 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:09:52.409+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.236297+0000 front 2024-03-27T11:09:28.236728+0000 (oldest deadline 2024-03-27T11:09:51.136525+0000) 2024-03-27T11:09:53.263 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:09:52 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:09:52.799+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.380254+0000 front 2024-03-27T11:09:23.380098+0000 (oldest deadline 2024-03-27T11:09:49.280089+0000) 2024-03-27T11:09:53.334 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:09:52 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:09:52.969+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.654238+0000 front 2024-03-27T11:09:23.654318+0000 (oldest deadline 2024-03-27T11:09:48.954440+0000) 2024-03-27T11:09:53.356 INFO:journalctl@ceph.osd.5.smithi177.stdout:Mar 27 11:09:52 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-5[47487]: 2024-03-27T11:09:52.974+0000 7fe117000700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:25.457341+0000 front 2024-03-27T11:09:25.457314+0000 (oldest deadline 2024-03-27T11:09:50.157224+0000) 2024-03-27T11:09:53.637 INFO:journalctl@ceph.osd.3.smithi097.stdout:Mar 27 11:09:53 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-3[53399]: 2024-03-27T11:09:53.420+0000 7f9d46c00700 -1 osd.3 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:25.247533+0000 front 2024-03-27T11:09:25.246502+0000 (oldest deadline 2024-03-27T11:09:49.346631+0000) 2024-03-27T11:09:53.638 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 27 11:09:53 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-6[60366]: 2024-03-27T11:09:53.365+0000 7f8ead400700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:27.412885+0000 front 2024-03-27T11:09:27.412671+0000 (oldest deadline 2024-03-27T11:09:49.712654+0000) 2024-03-27T11:09:53.673 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:09:53 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:09:53.387+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.236297+0000 front 2024-03-27T11:09:28.236728+0000 (oldest deadline 2024-03-27T11:09:51.136525+0000) 2024-03-27T11:09:54.173 INFO:journalctl@ceph.osd.5.smithi177.stdout:Mar 27 11:09:53 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-5[47487]: 2024-03-27T11:09:53.927+0000 7fe117000700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:25.457341+0000 front 2024-03-27T11:09:25.457314+0000 (oldest deadline 2024-03-27T11:09:50.157224+0000) 2024-03-27T11:09:54.263 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:09:53 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:09:53.840+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.380254+0000 front 2024-03-27T11:09:23.380098+0000 (oldest deadline 2024-03-27T11:09:49.280089+0000) 2024-03-27T11:09:54.303 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:09:53 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:09:53.931+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.654238+0000 front 2024-03-27T11:09:23.654318+0000 (oldest deadline 2024-03-27T11:09:48.954440+0000) 2024-03-27T11:09:54.637 INFO:journalctl@ceph.osd.3.smithi097.stdout:Mar 27 11:09:54 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-3[53399]: 2024-03-27T11:09:54.431+0000 7f9d46c00700 -1 osd.3 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:25.247533+0000 front 2024-03-27T11:09:25.246502+0000 (oldest deadline 2024-03-27T11:09:49.346631+0000) 2024-03-27T11:09:54.638 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 27 11:09:54 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-6[60366]: 2024-03-27T11:09:54.334+0000 7f8ead400700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:27.412885+0000 front 2024-03-27T11:09:27.412671+0000 (oldest deadline 2024-03-27T11:09:49.712654+0000) 2024-03-27T11:09:54.673 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:09:54 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:09:54.407+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.236297+0000 front 2024-03-27T11:09:28.236728+0000 (oldest deadline 2024-03-27T11:09:51.136525+0000) 2024-03-27T11:09:54.673 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:09:54 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:09:54.253+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.140407+0000 front 2024-03-27T11:09:28.140164+0000 (oldest deadline 2024-03-27T11:09:54.040233+0000) 2024-03-27T11:09:55.173 INFO:journalctl@ceph.osd.5.smithi177.stdout:Mar 27 11:09:54 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-5[47487]: 2024-03-27T11:09:54.914+0000 7fe117000700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:25.457341+0000 front 2024-03-27T11:09:25.457314+0000 (oldest deadline 2024-03-27T11:09:50.157224+0000) 2024-03-27T11:09:55.263 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:09:54 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:09:54.818+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.380254+0000 front 2024-03-27T11:09:23.380098+0000 (oldest deadline 2024-03-27T11:09:49.280089+0000) 2024-03-27T11:09:55.296 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:09:54 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:09:54.943+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.654238+0000 front 2024-03-27T11:09:23.654318+0000 (oldest deadline 2024-03-27T11:09:48.954440+0000) 2024-03-27T11:09:55.637 INFO:journalctl@ceph.osd.3.smithi097.stdout:Mar 27 11:09:55 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-3[53399]: 2024-03-27T11:09:55.434+0000 7f9d46c00700 -1 osd.3 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:25.247533+0000 front 2024-03-27T11:09:25.246502+0000 (oldest deadline 2024-03-27T11:09:49.346631+0000) 2024-03-27T11:09:55.638 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 27 11:09:55 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-6[60366]: 2024-03-27T11:09:55.326+0000 7f8ead400700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:27.412885+0000 front 2024-03-27T11:09:27.412671+0000 (oldest deadline 2024-03-27T11:09:49.712654+0000) 2024-03-27T11:09:55.673 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:09:55 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:09:55.274+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.140407+0000 front 2024-03-27T11:09:28.140164+0000 (oldest deadline 2024-03-27T11:09:54.040233+0000) 2024-03-27T11:09:55.673 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:09:55 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:09:55.391+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.236297+0000 front 2024-03-27T11:09:28.236728+0000 (oldest deadline 2024-03-27T11:09:51.136525+0000) 2024-03-27T11:09:56.173 INFO:journalctl@ceph.osd.5.smithi177.stdout:Mar 27 11:09:55 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-5[47487]: 2024-03-27T11:09:55.930+0000 7fe117000700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:25.457341+0000 front 2024-03-27T11:09:25.457314+0000 (oldest deadline 2024-03-27T11:09:50.157224+0000) 2024-03-27T11:09:56.263 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:09:55 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:09:55.806+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.380254+0000 front 2024-03-27T11:09:23.380098+0000 (oldest deadline 2024-03-27T11:09:49.280089+0000) 2024-03-27T11:09:56.319 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:09:55 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:09:55.920+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.654238+0000 front 2024-03-27T11:09:23.654318+0000 (oldest deadline 2024-03-27T11:09:48.954440+0000) 2024-03-27T11:09:56.637 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 27 11:09:56 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-6[60366]: 2024-03-27T11:09:56.349+0000 7f8ead400700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:27.412885+0000 front 2024-03-27T11:09:27.412671+0000 (oldest deadline 2024-03-27T11:09:49.712654+0000) 2024-03-27T11:09:56.638 INFO:journalctl@ceph.osd.3.smithi097.stdout:Mar 27 11:09:56 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-3[53399]: 2024-03-27T11:09:56.452+0000 7f9d46c00700 -1 osd.3 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:25.247533+0000 front 2024-03-27T11:09:25.246502+0000 (oldest deadline 2024-03-27T11:09:49.346631+0000) 2024-03-27T11:09:56.673 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:09:56 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:09:56.294+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.140407+0000 front 2024-03-27T11:09:28.140164+0000 (oldest deadline 2024-03-27T11:09:54.040233+0000) 2024-03-27T11:09:56.673 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:09:56 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:09:56.428+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.236297+0000 front 2024-03-27T11:09:28.236728+0000 (oldest deadline 2024-03-27T11:09:51.136525+0000) 2024-03-27T11:09:57.227 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:09:56 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:09:56.926+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.654238+0000 front 2024-03-27T11:09:23.654318+0000 (oldest deadline 2024-03-27T11:09:48.954440+0000) 2024-03-27T11:09:57.263 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:09:56 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:09:56.816+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.380254+0000 front 2024-03-27T11:09:23.380098+0000 (oldest deadline 2024-03-27T11:09:49.280089+0000) 2024-03-27T11:09:57.291 INFO:journalctl@ceph.osd.5.smithi177.stdout:Mar 27 11:09:56 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-5[47487]: 2024-03-27T11:09:56.962+0000 7fe117000700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:25.457341+0000 front 2024-03-27T11:09:25.457314+0000 (oldest deadline 2024-03-27T11:09:50.157224+0000) 2024-03-27T11:09:57.562 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 27 11:09:57 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-6[60366]: 2024-03-27T11:09:57.353+0000 7f8ead400700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:27.412885+0000 front 2024-03-27T11:09:27.412671+0000 (oldest deadline 2024-03-27T11:09:49.712654+0000) 2024-03-27T11:09:57.562 INFO:journalctl@ceph.osd.3.smithi097.stdout:Mar 27 11:09:57 smithi097 podman[97182]: 2024-03-27 11:09:57.259307551 +0000 UTC m=+5.788066147 container died c5a89c0f5607b43ba3ff268548b499e1e5e2babf67e1d946904eb66af44b3230 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-3, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.schema-version=1.0) 2024-03-27T11:09:57.673 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:09:57 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:09:57.321+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.140407+0000 front 2024-03-27T11:09:28.140164+0000 (oldest deadline 2024-03-27T11:09:54.040233+0000) 2024-03-27T11:09:57.673 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:09:57 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:09:57.405+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.236297+0000 front 2024-03-27T11:09:28.236728+0000 (oldest deadline 2024-03-27T11:09:51.136525+0000) 2024-03-27T11:09:57.888 INFO:journalctl@ceph.osd.3.smithi097.stdout:Mar 27 11:09:57 smithi097 podman[97182]: 2024-03-27 11:09:57.712857479 +0000 UTC m=+6.241616085 container cleanup c5a89c0f5607b43ba3ff268548b499e1e5e2babf67e1d946904eb66af44b3230 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-3, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240326.131558-155268c, io.buildah.version=1.33.5) 2024-03-27T11:09:57.888 INFO:journalctl@ceph.osd.3.smithi097.stdout:Mar 27 11:09:57 smithi097 bash[97182]: ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-3 2024-03-27T11:09:58.263 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:09:57 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:09:57.794+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.380254+0000 front 2024-03-27T11:09:23.380098+0000 (oldest deadline 2024-03-27T11:09:49.280089+0000) 2024-03-27T11:09:58.282 INFO:journalctl@ceph.osd.5.smithi177.stdout:Mar 27 11:09:57 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-5[47487]: 2024-03-27T11:09:57.986+0000 7fe117000700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:25.457341+0000 front 2024-03-27T11:09:25.457314+0000 (oldest deadline 2024-03-27T11:09:50.157224+0000) 2024-03-27T11:09:58.328 INFO:journalctl@ceph.osd.3.smithi097.stdout:Mar 27 11:09:57 smithi097 podman[97209]: 2024-03-27 11:09:57.971671131 +0000 UTC m=+0.698478881 container remove c5a89c0f5607b43ba3ff268548b499e1e5e2babf67e1d946904eb66af44b3230 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-3, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, GIT_CLEAN=True) 2024-03-27T11:09:58.328 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:09:57 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:09:57.945+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.654238+0000 front 2024-03-27T11:09:23.654318+0000 (oldest deadline 2024-03-27T11:09:48.954440+0000) 2024-03-27T11:09:58.637 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 27 11:09:58 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-6[60366]: 2024-03-27T11:09:58.358+0000 7f8ead400700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:27.412885+0000 front 2024-03-27T11:09:27.412671+0000 (oldest deadline 2024-03-27T11:09:49.712654+0000) 2024-03-27T11:09:58.673 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:09:58 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:09:58.364+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.236297+0000 front 2024-03-27T11:09:28.236728+0000 (oldest deadline 2024-03-27T11:09:51.136525+0000) 2024-03-27T11:09:58.673 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:09:58 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:09:58.312+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.140407+0000 front 2024-03-27T11:09:28.140164+0000 (oldest deadline 2024-03-27T11:09:54.040233+0000) 2024-03-27T11:09:58.998 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:09:58 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:09:58.990+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.654238+0000 front 2024-03-27T11:09:23.654318+0000 (oldest deadline 2024-03-27T11:09:48.954440+0000) 2024-03-27T11:09:58.998 INFO:journalctl@ceph.osd.3.smithi097.stdout:Mar 27 11:09:58 smithi097 podman[97297]: 2024-03-27 11:09:58.773214819 +0000 UTC m=+0.129532959 container create 081425ec41dc543d6d2b55c366b635b36cb001807a5965e0620a1f768fb189d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-3-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.build-date=20240326, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-03-27T11:09:58.998 INFO:journalctl@ceph.osd.3.smithi097.stdout:Mar 27 11:09:58 smithi097 podman[97297]: 2024-03-27 11:09:58.681519291 +0000 UTC m=+0.037837418 image pull b379bbd8da6b5d0fa9a3523941efac213b6815e1cbf05c6507954f0a3fe7a801 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772 2024-03-27T11:09:58.998 INFO:journalctl@ceph.osd.3.smithi097.stdout:Mar 27 11:09:59 smithi097 podman[97297]: 2024-03-27 11:09:59.018438627 +0000 UTC m=+0.374756771 container init 081425ec41dc543d6d2b55c366b635b36cb001807a5965e0620a1f768fb189d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240326.131558-155268c, ceph=True, GIT_CLEAN=True, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-03-27T11:09:58.998 INFO:journalctl@ceph.osd.3.smithi097.stdout:Mar 27 11:09:59 smithi097 podman[97297]: 2024-03-27 11:09:59.029626769 +0000 UTC m=+0.385944916 container start 081425ec41dc543d6d2b55c366b635b36cb001807a5965e0620a1f768fb189d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-3-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240326.131558-155268c, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-27T11:09:59.263 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:09:58 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:09:58.827+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.380254+0000 front 2024-03-27T11:09:23.380098+0000 (oldest deadline 2024-03-27T11:09:49.280089+0000) 2024-03-27T11:09:59.291 INFO:journalctl@ceph.osd.3.smithi097.stdout:Mar 27 11:09:59 smithi097 podman[97297]: 2024-03-27 11:09:59.10085755 +0000 UTC m=+0.457175734 container attach 081425ec41dc543d6d2b55c366b635b36cb001807a5965e0620a1f768fb189d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-3-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240326.131558-155268c, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True) 2024-03-27T11:09:59.292 INFO:journalctl@ceph.osd.5.smithi177.stdout:Mar 27 11:09:58 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-5[47487]: 2024-03-27T11:09:58.972+0000 7fe117000700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:25.457341+0000 front 2024-03-27T11:09:25.457314+0000 (oldest deadline 2024-03-27T11:09:50.157224+0000) 2024-03-27T11:09:59.638 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 27 11:09:59 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-6[60366]: 2024-03-27T11:09:59.321+0000 7f8ead400700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:27.412885+0000 front 2024-03-27T11:09:27.412671+0000 (oldest deadline 2024-03-27T11:09:49.712654+0000) 2024-03-27T11:09:59.638 INFO:journalctl@ceph.osd.3.smithi097.stdout:Mar 27 11:09:59 smithi097 podman[97297]: 2024-03-27 11:09:59.463021654 +0000 UTC m=+0.819339803 container died 081425ec41dc543d6d2b55c366b635b36cb001807a5965e0620a1f768fb189d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-3-deactivate, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.schema-version=1.0, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-03-27T11:09:59.673 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:09:59 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:09:59.322+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.236297+0000 front 2024-03-27T11:09:28.236728+0000 (oldest deadline 2024-03-27T11:09:51.136525+0000) 2024-03-27T11:09:59.673 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:09:59 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:09:59.329+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.140407+0000 front 2024-03-27T11:09:28.140164+0000 (oldest deadline 2024-03-27T11:09:54.040233+0000) 2024-03-27T11:09:59.763 INFO:journalctl@ceph.osd.4.smithi114.stdout:Mar 27 11:09:59 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-4[46437]: 2024-03-27T11:09:59.370+0000 7f5ddae00700 -1 osd.4 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:34.024051+0000 front 2024-03-27T11:09:34.024118+0000 (oldest deadline 2024-03-27T11:09:59.324207+0000) 2024-03-27T11:10:00.097 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:00 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:00.033+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.654238+0000 front 2024-03-27T11:09:23.654318+0000 (oldest deadline 2024-03-27T11:09:48.954440+0000) 2024-03-27T11:10:00.152 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-adca88c4-ec26-11ee-95d0-87774f69a715@osd.3.service' 2024-03-27T11:10:00.173 INFO:journalctl@ceph.osd.5.smithi177.stdout:Mar 27 11:09:59 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-5[47487]: 2024-03-27T11:09:59.933+0000 7fe117000700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:25.457341+0000 front 2024-03-27T11:09:25.457314+0000 (oldest deadline 2024-03-27T11:09:50.157224+0000) 2024-03-27T11:10:00.263 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:09:59 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:09:59.877+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.380254+0000 front 2024-03-27T11:09:23.380098+0000 (oldest deadline 2024-03-27T11:09:49.280089+0000) 2024-03-27T11:10:00.268 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-27T11:10:00.268 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-03-27T11:10:00.269 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-03-27T11:10:00.269 DEBUG:teuthology.orchestra.run.smithi114:> sudo systemctl stop ceph-adca88c4-ec26-11ee-95d0-87774f69a715@osd.4 2024-03-27T11:10:00.387 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 27 11:10:00 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-6[60366]: 2024-03-27T11:10:00.359+0000 7f8ead400700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:27.412885+0000 front 2024-03-27T11:09:27.412671+0000 (oldest deadline 2024-03-27T11:09:49.712654+0000) 2024-03-27T11:10:00.388 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 27 11:10:00 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-6[60366]: 2024-03-27T11:10:00.359+0000 7f8ead400700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:35.416884+0000 front 2024-03-27T11:09:35.416826+0000 (oldest deadline 2024-03-27T11:09:59.516984+0000) 2024-03-27T11:10:00.673 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:00 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:00.301+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.236297+0000 front 2024-03-27T11:09:28.236728+0000 (oldest deadline 2024-03-27T11:09:51.136525+0000) 2024-03-27T11:10:00.674 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:00 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:00.337+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.140407+0000 front 2024-03-27T11:09:28.140164+0000 (oldest deadline 2024-03-27T11:09:54.040233+0000) 2024-03-27T11:10:00.763 INFO:journalctl@ceph.osd.4.smithi114.stdout:Mar 27 11:10:00 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-4[46437]: 2024-03-27T11:10:00.360+0000 7f5ddae00700 -1 osd.4 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:34.024051+0000 front 2024-03-27T11:09:34.024118+0000 (oldest deadline 2024-03-27T11:09:59.324207+0000) 2024-03-27T11:10:00.763 INFO:journalctl@ceph.osd.4.smithi114.stdout:Mar 27 11:10:00 smithi114 systemd[1]: Stopping Ceph osd.4 for adca88c4-ec26-11ee-95d0-87774f69a715... 2024-03-27T11:10:01.173 INFO:journalctl@ceph.osd.5.smithi177.stdout:Mar 27 11:10:00 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-5[47487]: 2024-03-27T11:10:00.922+0000 7fe117000700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:25.457341+0000 front 2024-03-27T11:09:25.457314+0000 (oldest deadline 2024-03-27T11:09:50.157224+0000) 2024-03-27T11:10:01.264 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:00 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:00.854+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.380254+0000 front 2024-03-27T11:09:23.380098+0000 (oldest deadline 2024-03-27T11:09:49.280089+0000) 2024-03-27T11:10:01.264 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:00 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:00.854+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:34.582052+0000 front 2024-03-27T11:09:34.582101+0000 (oldest deadline 2024-03-27T11:10:00.482194+0000) 2024-03-27T11:10:01.264 INFO:journalctl@ceph.osd.4.smithi114.stdout:Mar 27 11:10:00 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-4[46437]: 2024-03-27T11:10:00.843+0000 7f5de4600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-27T11:10:01.264 INFO:journalctl@ceph.osd.4.smithi114.stdout:Mar 27 11:10:00 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-4[46437]: 2024-03-27T11:10:00.843+0000 7f5de4600700 -1 osd.4 92 *** Got signal Terminated *** 2024-03-27T11:10:01.264 INFO:journalctl@ceph.osd.4.smithi114.stdout:Mar 27 11:10:00 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-4[46437]: 2024-03-27T11:10:00.843+0000 7f5de4600700 -1 osd.4 92 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-27T11:10:01.296 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:01 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:01.029+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.654238+0000 front 2024-03-27T11:09:23.654318+0000 (oldest deadline 2024-03-27T11:09:48.954440+0000) 2024-03-27T11:10:01.637 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 27 11:10:01 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-6[60366]: 2024-03-27T11:10:01.327+0000 7f8ead400700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:27.412885+0000 front 2024-03-27T11:09:27.412671+0000 (oldest deadline 2024-03-27T11:09:49.712654+0000) 2024-03-27T11:10:01.637 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 27 11:10:01 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-6[60366]: 2024-03-27T11:10:01.327+0000 7f8ead400700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:35.416884+0000 front 2024-03-27T11:09:35.416826+0000 (oldest deadline 2024-03-27T11:09:59.516984+0000) 2024-03-27T11:10:01.673 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:01 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:01.277+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.236297+0000 front 2024-03-27T11:09:28.236728+0000 (oldest deadline 2024-03-27T11:09:51.136525+0000) 2024-03-27T11:10:01.673 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:01 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:01.313+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.140407+0000 front 2024-03-27T11:09:28.140164+0000 (oldest deadline 2024-03-27T11:09:54.040233+0000) 2024-03-27T11:10:01.763 INFO:journalctl@ceph.osd.4.smithi114.stdout:Mar 27 11:10:01 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-4[46437]: 2024-03-27T11:10:01.313+0000 7f5ddae00700 -1 osd.4 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:34.024051+0000 front 2024-03-27T11:09:34.024118+0000 (oldest deadline 2024-03-27T11:09:59.324207+0000) 2024-03-27T11:10:01.764 INFO:journalctl@ceph.osd.7.smithi114.stdout:Mar 27 11:10:01 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-7[53256]: 2024-03-27T11:10:01.295+0000 7fd5e5800700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.916460+0000 front 2024-03-27T11:09:36.916564+0000 (oldest deadline 2024-03-27T11:10:01.016630+0000) 2024-03-27T11:10:02.173 INFO:journalctl@ceph.osd.5.smithi177.stdout:Mar 27 11:10:01 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-5[47487]: 2024-03-27T11:10:01.892+0000 7fe117000700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:25.457341+0000 front 2024-03-27T11:09:25.457314+0000 (oldest deadline 2024-03-27T11:09:50.157224+0000) 2024-03-27T11:10:02.220 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:01 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:01.818+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.380254+0000 front 2024-03-27T11:09:23.380098+0000 (oldest deadline 2024-03-27T11:09:49.280089+0000) 2024-03-27T11:10:02.220 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:01 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:01.818+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:34.582052+0000 front 2024-03-27T11:09:34.582101+0000 (oldest deadline 2024-03-27T11:10:00.482194+0000) 2024-03-27T11:10:02.387 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 27 11:10:02 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-6[60366]: 2024-03-27T11:10:02.294+0000 7f8ead400700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:27.412885+0000 front 2024-03-27T11:09:27.412671+0000 (oldest deadline 2024-03-27T11:09:49.712654+0000) 2024-03-27T11:10:02.387 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 27 11:10:02 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-6[60366]: 2024-03-27T11:10:02.294+0000 7f8ead400700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:35.416884+0000 front 2024-03-27T11:09:35.416826+0000 (oldest deadline 2024-03-27T11:09:59.516984+0000) 2024-03-27T11:10:02.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:02 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:02.063+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.654238+0000 front 2024-03-27T11:09:23.654318+0000 (oldest deadline 2024-03-27T11:09:48.954440+0000) 2024-03-27T11:10:02.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:02 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:02.063+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:35.957512+0000 front 2024-03-27T11:09:35.957437+0000 (oldest deadline 2024-03-27T11:10:01.857676+0000) 2024-03-27T11:10:02.513 INFO:journalctl@ceph.osd.7.smithi114.stdout:Mar 27 11:10:02 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-7[53256]: 2024-03-27T11:10:02.251+0000 7fd5e5800700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.916460+0000 front 2024-03-27T11:09:36.916564+0000 (oldest deadline 2024-03-27T11:10:01.016630+0000) 2024-03-27T11:10:02.514 INFO:journalctl@ceph.osd.4.smithi114.stdout:Mar 27 11:10:02 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-4[46437]: 2024-03-27T11:10:02.356+0000 7f5ddae00700 -1 osd.4 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:34.024051+0000 front 2024-03-27T11:09:34.024118+0000 (oldest deadline 2024-03-27T11:09:59.324207+0000) 2024-03-27T11:10:02.673 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:02 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:02.324+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.140407+0000 front 2024-03-27T11:09:28.140164+0000 (oldest deadline 2024-03-27T11:09:54.040233+0000) 2024-03-27T11:10:02.673 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:02 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:02.324+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.844416+0000 front 2024-03-27T11:09:36.843999+0000 (oldest deadline 2024-03-27T11:10:01.544084+0000) 2024-03-27T11:10:02.674 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:02 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:02.314+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.236297+0000 front 2024-03-27T11:09:28.236728+0000 (oldest deadline 2024-03-27T11:09:51.136525+0000) 2024-03-27T11:10:02.674 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:02 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:02.314+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.340023+0000 front 2024-03-27T11:09:36.340351+0000 (oldest deadline 2024-03-27T11:10:02.239659+0000) 2024-03-27T11:10:03.171 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:02 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:02.806+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.380254+0000 front 2024-03-27T11:09:23.380098+0000 (oldest deadline 2024-03-27T11:09:49.280089+0000) 2024-03-27T11:10:03.171 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:02 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:02.806+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:34.582052+0000 front 2024-03-27T11:09:34.582101+0000 (oldest deadline 2024-03-27T11:10:00.482194+0000) 2024-03-27T11:10:03.173 INFO:journalctl@ceph.osd.5.smithi177.stdout:Mar 27 11:10:02 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-5[47487]: 2024-03-27T11:10:02.913+0000 7fe117000700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:25.457341+0000 front 2024-03-27T11:09:25.457314+0000 (oldest deadline 2024-03-27T11:09:50.157224+0000) 2024-03-27T11:10:03.173 INFO:journalctl@ceph.osd.5.smithi177.stdout:Mar 27 11:10:02 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-5[47487]: 2024-03-27T11:10:02.913+0000 7fe117000700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.560634+0000 front 2024-03-27T11:09:36.561253+0000 (oldest deadline 2024-03-27T11:10:02.460286+0000) 2024-03-27T11:10:03.387 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 27 11:10:03 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-6[60366]: 2024-03-27T11:10:03.293+0000 7f8ead400700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:27.412885+0000 front 2024-03-27T11:09:27.412671+0000 (oldest deadline 2024-03-27T11:09:49.712654+0000) 2024-03-27T11:10:03.387 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 27 11:10:03 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-6[60366]: 2024-03-27T11:10:03.293+0000 7f8ead400700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:35.416884+0000 front 2024-03-27T11:09:35.416826+0000 (oldest deadline 2024-03-27T11:09:59.516984+0000) 2024-03-27T11:10:03.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:03 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:03.045+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.654238+0000 front 2024-03-27T11:09:23.654318+0000 (oldest deadline 2024-03-27T11:09:48.954440+0000) 2024-03-27T11:10:03.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:03 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:03.045+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:35.957512+0000 front 2024-03-27T11:09:35.957437+0000 (oldest deadline 2024-03-27T11:10:01.857676+0000) 2024-03-27T11:10:03.513 INFO:journalctl@ceph.osd.4.smithi114.stdout:Mar 27 11:10:03 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-4[46437]: 2024-03-27T11:10:03.365+0000 7f5ddae00700 -1 osd.4 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:34.024051+0000 front 2024-03-27T11:09:34.024118+0000 (oldest deadline 2024-03-27T11:09:59.324207+0000) 2024-03-27T11:10:03.513 INFO:journalctl@ceph.osd.7.smithi114.stdout:Mar 27 11:10:03 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-7[53256]: 2024-03-27T11:10:03.202+0000 7fd5e5800700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.916460+0000 front 2024-03-27T11:09:36.916564+0000 (oldest deadline 2024-03-27T11:10:01.016630+0000) 2024-03-27T11:10:03.673 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:03 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:03.374+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.140407+0000 front 2024-03-27T11:09:28.140164+0000 (oldest deadline 2024-03-27T11:09:54.040233+0000) 2024-03-27T11:10:03.673 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:03 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:03.374+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.844416+0000 front 2024-03-27T11:09:36.843999+0000 (oldest deadline 2024-03-27T11:10:01.544084+0000) 2024-03-27T11:10:03.674 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:03 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:03.357+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.236297+0000 front 2024-03-27T11:09:28.236728+0000 (oldest deadline 2024-03-27T11:09:51.136525+0000) 2024-03-27T11:10:03.674 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:03 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:03.357+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.340023+0000 front 2024-03-27T11:09:36.340351+0000 (oldest deadline 2024-03-27T11:10:02.239659+0000) 2024-03-27T11:10:04.173 INFO:journalctl@ceph.osd.5.smithi177.stdout:Mar 27 11:10:03 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-5[47487]: 2024-03-27T11:10:03.901+0000 7fe117000700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:25.457341+0000 front 2024-03-27T11:09:25.457314+0000 (oldest deadline 2024-03-27T11:09:50.157224+0000) 2024-03-27T11:10:04.173 INFO:journalctl@ceph.osd.5.smithi177.stdout:Mar 27 11:10:03 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-5[47487]: 2024-03-27T11:10:03.902+0000 7fe117000700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.560634+0000 front 2024-03-27T11:09:36.561253+0000 (oldest deadline 2024-03-27T11:10:02.460286+0000) 2024-03-27T11:10:04.189 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:03 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:03.834+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.380254+0000 front 2024-03-27T11:09:23.380098+0000 (oldest deadline 2024-03-27T11:09:49.280089+0000) 2024-03-27T11:10:04.190 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:03 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:03.834+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:34.582052+0000 front 2024-03-27T11:09:34.582101+0000 (oldest deadline 2024-03-27T11:10:00.482194+0000) 2024-03-27T11:10:04.387 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 27 11:10:04 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-6[60366]: 2024-03-27T11:10:04.267+0000 7f8ead400700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:27.412885+0000 front 2024-03-27T11:09:27.412671+0000 (oldest deadline 2024-03-27T11:09:49.712654+0000) 2024-03-27T11:10:04.387 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 27 11:10:04 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-6[60366]: 2024-03-27T11:10:04.267+0000 7f8ead400700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:35.416884+0000 front 2024-03-27T11:09:35.416826+0000 (oldest deadline 2024-03-27T11:09:59.516984+0000) 2024-03-27T11:10:04.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:04 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:04.023+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.654238+0000 front 2024-03-27T11:09:23.654318+0000 (oldest deadline 2024-03-27T11:09:48.954440+0000) 2024-03-27T11:10:04.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:04 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:04.023+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:35.957512+0000 front 2024-03-27T11:09:35.957437+0000 (oldest deadline 2024-03-27T11:10:01.857676+0000) 2024-03-27T11:10:04.513 INFO:journalctl@ceph.osd.4.smithi114.stdout:Mar 27 11:10:04 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-4[46437]: 2024-03-27T11:10:04.352+0000 7f5ddae00700 -1 osd.4 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:34.024051+0000 front 2024-03-27T11:09:34.024118+0000 (oldest deadline 2024-03-27T11:09:59.324207+0000) 2024-03-27T11:10:04.513 INFO:journalctl@ceph.osd.7.smithi114.stdout:Mar 27 11:10:04 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-7[53256]: 2024-03-27T11:10:04.221+0000 7fd5e5800700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.916460+0000 front 2024-03-27T11:09:36.916564+0000 (oldest deadline 2024-03-27T11:10:01.016630+0000) 2024-03-27T11:10:04.673 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:04 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:04.343+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.236297+0000 front 2024-03-27T11:09:28.236728+0000 (oldest deadline 2024-03-27T11:09:51.136525+0000) 2024-03-27T11:10:04.673 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:04 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:04.343+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.340023+0000 front 2024-03-27T11:09:36.340351+0000 (oldest deadline 2024-03-27T11:10:02.239659+0000) 2024-03-27T11:10:04.673 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:04 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:04.345+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.140407+0000 front 2024-03-27T11:09:28.140164+0000 (oldest deadline 2024-03-27T11:09:54.040233+0000) 2024-03-27T11:10:04.673 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:04 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:04.345+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.844416+0000 front 2024-03-27T11:09:36.843999+0000 (oldest deadline 2024-03-27T11:10:01.544084+0000) 2024-03-27T11:10:05.173 INFO:journalctl@ceph.osd.5.smithi177.stdout:Mar 27 11:10:04 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-5[47487]: 2024-03-27T11:10:04.885+0000 7fe117000700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:25.457341+0000 front 2024-03-27T11:09:25.457314+0000 (oldest deadline 2024-03-27T11:09:50.157224+0000) 2024-03-27T11:10:05.173 INFO:journalctl@ceph.osd.5.smithi177.stdout:Mar 27 11:10:04 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-5[47487]: 2024-03-27T11:10:04.885+0000 7fe117000700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.560634+0000 front 2024-03-27T11:09:36.561253+0000 (oldest deadline 2024-03-27T11:10:02.460286+0000) 2024-03-27T11:10:05.198 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:04 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:04.828+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.380254+0000 front 2024-03-27T11:09:23.380098+0000 (oldest deadline 2024-03-27T11:09:49.280089+0000) 2024-03-27T11:10:05.198 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:04 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:04.828+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:34.582052+0000 front 2024-03-27T11:09:34.582101+0000 (oldest deadline 2024-03-27T11:10:00.482194+0000) 2024-03-27T11:10:05.387 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 27 11:10:05 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-6[60366]: 2024-03-27T11:10:05.250+0000 7f8ead400700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:27.412885+0000 front 2024-03-27T11:09:27.412671+0000 (oldest deadline 2024-03-27T11:09:49.712654+0000) 2024-03-27T11:10:05.387 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 27 11:10:05 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-6[60366]: 2024-03-27T11:10:05.250+0000 7f8ead400700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:35.416884+0000 front 2024-03-27T11:09:35.416826+0000 (oldest deadline 2024-03-27T11:09:59.516984+0000) 2024-03-27T11:10:05.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:05 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:05.036+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.654238+0000 front 2024-03-27T11:09:23.654318+0000 (oldest deadline 2024-03-27T11:09:48.954440+0000) 2024-03-27T11:10:05.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:05 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:05.036+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:35.957512+0000 front 2024-03-27T11:09:35.957437+0000 (oldest deadline 2024-03-27T11:10:01.857676+0000) 2024-03-27T11:10:05.513 INFO:journalctl@ceph.osd.7.smithi114.stdout:Mar 27 11:10:05 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-7[53256]: 2024-03-27T11:10:05.229+0000 7fd5e5800700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.916460+0000 front 2024-03-27T11:09:36.916564+0000 (oldest deadline 2024-03-27T11:10:01.016630+0000) 2024-03-27T11:10:05.513 INFO:journalctl@ceph.osd.4.smithi114.stdout:Mar 27 11:10:05 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-4[46437]: 2024-03-27T11:10:05.314+0000 7f5ddae00700 -1 osd.4 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:34.024051+0000 front 2024-03-27T11:09:34.024118+0000 (oldest deadline 2024-03-27T11:09:59.324207+0000) 2024-03-27T11:10:05.673 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:05 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:05.382+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.140407+0000 front 2024-03-27T11:09:28.140164+0000 (oldest deadline 2024-03-27T11:09:54.040233+0000) 2024-03-27T11:10:05.673 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:05 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:05.382+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.844416+0000 front 2024-03-27T11:09:36.843999+0000 (oldest deadline 2024-03-27T11:10:01.544084+0000) 2024-03-27T11:10:05.674 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:05 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:05.302+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.236297+0000 front 2024-03-27T11:09:28.236728+0000 (oldest deadline 2024-03-27T11:09:51.136525+0000) 2024-03-27T11:10:05.674 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:05 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:05.302+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.340023+0000 front 2024-03-27T11:09:36.340351+0000 (oldest deadline 2024-03-27T11:10:02.239659+0000) 2024-03-27T11:10:06.013 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:05 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:05.791+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.380254+0000 front 2024-03-27T11:09:23.380098+0000 (oldest deadline 2024-03-27T11:09:49.280089+0000) 2024-03-27T11:10:06.013 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:05 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:05.791+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:34.582052+0000 front 2024-03-27T11:09:34.582101+0000 (oldest deadline 2024-03-27T11:10:00.482194+0000) 2024-03-27T11:10:06.173 INFO:journalctl@ceph.osd.5.smithi177.stdout:Mar 27 11:10:05 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-5[47487]: 2024-03-27T11:10:05.915+0000 7fe117000700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:25.457341+0000 front 2024-03-27T11:09:25.457314+0000 (oldest deadline 2024-03-27T11:09:50.157224+0000) 2024-03-27T11:10:06.173 INFO:journalctl@ceph.osd.5.smithi177.stdout:Mar 27 11:10:05 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-5[47487]: 2024-03-27T11:10:05.915+0000 7fe117000700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.560634+0000 front 2024-03-27T11:09:36.561253+0000 (oldest deadline 2024-03-27T11:10:02.460286+0000) 2024-03-27T11:10:06.387 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 27 11:10:06 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-6[60366]: 2024-03-27T11:10:06.250+0000 7f8ead400700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:27.412885+0000 front 2024-03-27T11:09:27.412671+0000 (oldest deadline 2024-03-27T11:09:49.712654+0000) 2024-03-27T11:10:06.387 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 27 11:10:06 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-6[60366]: 2024-03-27T11:10:06.250+0000 7f8ead400700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:35.416884+0000 front 2024-03-27T11:09:35.416826+0000 (oldest deadline 2024-03-27T11:09:59.516984+0000) 2024-03-27T11:10:06.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:06 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:06.081+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.654238+0000 front 2024-03-27T11:09:23.654318+0000 (oldest deadline 2024-03-27T11:09:48.954440+0000) 2024-03-27T11:10:06.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:06 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:06.081+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:35.957512+0000 front 2024-03-27T11:09:35.957437+0000 (oldest deadline 2024-03-27T11:10:01.857676+0000) 2024-03-27T11:10:06.513 INFO:journalctl@ceph.osd.7.smithi114.stdout:Mar 27 11:10:06 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-7[53256]: 2024-03-27T11:10:06.215+0000 7fd5e5800700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.916460+0000 front 2024-03-27T11:09:36.916564+0000 (oldest deadline 2024-03-27T11:10:01.016630+0000) 2024-03-27T11:10:06.673 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:06 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:06.266+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.236297+0000 front 2024-03-27T11:09:28.236728+0000 (oldest deadline 2024-03-27T11:09:51.136525+0000) 2024-03-27T11:10:06.673 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:06 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:06.266+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.340023+0000 front 2024-03-27T11:09:36.340351+0000 (oldest deadline 2024-03-27T11:10:02.239659+0000) 2024-03-27T11:10:06.673 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:06 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:06.339+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.140407+0000 front 2024-03-27T11:09:28.140164+0000 (oldest deadline 2024-03-27T11:09:54.040233+0000) 2024-03-27T11:10:06.673 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:06 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:06.339+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.844416+0000 front 2024-03-27T11:09:36.843999+0000 (oldest deadline 2024-03-27T11:10:01.544084+0000) 2024-03-27T11:10:06.925 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:06 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:06.765+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.380254+0000 front 2024-03-27T11:09:23.380098+0000 (oldest deadline 2024-03-27T11:09:49.280089+0000) 2024-03-27T11:10:06.925 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:06 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:06.765+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:34.582052+0000 front 2024-03-27T11:09:34.582101+0000 (oldest deadline 2024-03-27T11:10:00.482194+0000) 2024-03-27T11:10:06.925 INFO:journalctl@ceph.osd.4.smithi114.stdout:Mar 27 11:10:06 smithi114 podman[70779]: 2024-03-27 11:10:06.660808099 +0000 UTC m=+5.978556072 container died 3e575c8ed6a58353edc85e9a408dc6acc8477b13f2be0b8b0395597501dcfdb1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-4, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240326.131558-155268c, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-03-27T11:10:07.173 INFO:journalctl@ceph.osd.5.smithi177.stdout:Mar 27 11:10:06 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-5[47487]: 2024-03-27T11:10:06.869+0000 7fe117000700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:25.457341+0000 front 2024-03-27T11:09:25.457314+0000 (oldest deadline 2024-03-27T11:09:50.157224+0000) 2024-03-27T11:10:07.173 INFO:journalctl@ceph.osd.5.smithi177.stdout:Mar 27 11:10:06 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-5[47487]: 2024-03-27T11:10:06.869+0000 7fe117000700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.560634+0000 front 2024-03-27T11:09:36.561253+0000 (oldest deadline 2024-03-27T11:10:02.460286+0000) 2024-03-27T11:10:07.216 INFO:journalctl@ceph.osd.4.smithi114.stdout:Mar 27 11:10:07 smithi114 podman[70779]: 2024-03-27 11:10:07.128056926 +0000 UTC m=+6.445804929 container cleanup 3e575c8ed6a58353edc85e9a408dc6acc8477b13f2be0b8b0395597501dcfdb1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-4, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, RELEASE=wip-rishabh-testing-20240326.131558-155268c, io.buildah.version=1.33.5) 2024-03-27T11:10:07.216 INFO:journalctl@ceph.osd.4.smithi114.stdout:Mar 27 11:10:07 smithi114 bash[70779]: ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-4 2024-03-27T11:10:07.387 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 27 11:10:07 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-6[60366]: 2024-03-27T11:10:07.209+0000 7f8ead400700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:27.412885+0000 front 2024-03-27T11:09:27.412671+0000 (oldest deadline 2024-03-27T11:09:49.712654+0000) 2024-03-27T11:10:07.387 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 27 11:10:07 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-6[60366]: 2024-03-27T11:10:07.209+0000 7f8ead400700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:35.416884+0000 front 2024-03-27T11:09:35.416826+0000 (oldest deadline 2024-03-27T11:09:59.516984+0000) 2024-03-27T11:10:07.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:07 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:07.073+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.654238+0000 front 2024-03-27T11:09:23.654318+0000 (oldest deadline 2024-03-27T11:09:48.954440+0000) 2024-03-27T11:10:07.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:07 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:07.073+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:35.957512+0000 front 2024-03-27T11:09:35.957437+0000 (oldest deadline 2024-03-27T11:10:01.857676+0000) 2024-03-27T11:10:07.513 INFO:journalctl@ceph.osd.4.smithi114.stdout:Mar 27 11:10:07 smithi114 podman[70798]: 2024-03-27 11:10:07.41261301 +0000 UTC m=+0.738785832 container remove 3e575c8ed6a58353edc85e9a408dc6acc8477b13f2be0b8b0395597501dcfdb1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-4, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.vendor=CentOS, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, ceph=True) 2024-03-27T11:10:07.514 INFO:journalctl@ceph.osd.7.smithi114.stdout:Mar 27 11:10:07 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-7[53256]: 2024-03-27T11:10:07.247+0000 7fd5e5800700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.916460+0000 front 2024-03-27T11:09:36.916564+0000 (oldest deadline 2024-03-27T11:10:01.016630+0000) 2024-03-27T11:10:07.673 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:07 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:07.218+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.236297+0000 front 2024-03-27T11:09:28.236728+0000 (oldest deadline 2024-03-27T11:09:51.136525+0000) 2024-03-27T11:10:07.673 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:07 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:07.218+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.340023+0000 front 2024-03-27T11:09:36.340351+0000 (oldest deadline 2024-03-27T11:10:02.239659+0000) 2024-03-27T11:10:07.673 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:07 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:07.307+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.140407+0000 front 2024-03-27T11:09:28.140164+0000 (oldest deadline 2024-03-27T11:09:54.040233+0000) 2024-03-27T11:10:07.674 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:07 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:07.307+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.844416+0000 front 2024-03-27T11:09:36.843999+0000 (oldest deadline 2024-03-27T11:10:01.544084+0000) 2024-03-27T11:10:08.013 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:07 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:07.776+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.380254+0000 front 2024-03-27T11:09:23.380098+0000 (oldest deadline 2024-03-27T11:09:49.280089+0000) 2024-03-27T11:10:08.013 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:07 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:07.776+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:34.582052+0000 front 2024-03-27T11:09:34.582101+0000 (oldest deadline 2024-03-27T11:10:00.482194+0000) 2024-03-27T11:10:08.173 INFO:journalctl@ceph.osd.5.smithi177.stdout:Mar 27 11:10:07 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-5[47487]: 2024-03-27T11:10:07.917+0000 7fe117000700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:25.457341+0000 front 2024-03-27T11:09:25.457314+0000 (oldest deadline 2024-03-27T11:09:50.157224+0000) 2024-03-27T11:10:08.173 INFO:journalctl@ceph.osd.5.smithi177.stdout:Mar 27 11:10:07 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-5[47487]: 2024-03-27T11:10:07.917+0000 7fe117000700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.560634+0000 front 2024-03-27T11:09:36.561253+0000 (oldest deadline 2024-03-27T11:10:02.460286+0000) 2024-03-27T11:10:08.377 INFO:journalctl@ceph.osd.7.smithi114.stdout:Mar 27 11:10:08 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-7[53256]: 2024-03-27T11:10:08.254+0000 7fd5e5800700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.916460+0000 front 2024-03-27T11:09:36.916564+0000 (oldest deadline 2024-03-27T11:10:01.016630+0000) 2024-03-27T11:10:08.377 INFO:journalctl@ceph.osd.4.smithi114.stdout:Mar 27 11:10:08 smithi114 podman[70885]: 2024-03-27 11:10:08.004820941 +0000 UTC m=+0.037377781 image pull b379bbd8da6b5d0fa9a3523941efac213b6815e1cbf05c6507954f0a3fe7a801 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772 2024-03-27T11:10:08.378 INFO:journalctl@ceph.osd.4.smithi114.stdout:Mar 27 11:10:08 smithi114 podman[70885]: 2024-03-27 11:10:08.151682814 +0000 UTC m=+0.184239686 container create ce6ac66a4a25b2422865debd336b4f55d052725361c74d5513b9b3e9b04fa36f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-03-27T11:10:08.378 INFO:journalctl@ceph.osd.4.smithi114.stdout:Mar 27 11:10:08 smithi114 podman[70885]: 2024-03-27 11:10:08.408908695 +0000 UTC m=+0.441465547 container init ce6ac66a4a25b2422865debd336b4f55d052725361c74d5513b9b3e9b04fa36f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-4-deactivate, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.build-date=20240326) 2024-03-27T11:10:08.387 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 27 11:10:08 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-6[60366]: 2024-03-27T11:10:08.225+0000 7f8ead400700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:27.412885+0000 front 2024-03-27T11:09:27.412671+0000 (oldest deadline 2024-03-27T11:09:49.712654+0000) 2024-03-27T11:10:08.387 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 27 11:10:08 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-6[60366]: 2024-03-27T11:10:08.225+0000 7f8ead400700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:35.416884+0000 front 2024-03-27T11:09:35.416826+0000 (oldest deadline 2024-03-27T11:09:59.516984+0000) 2024-03-27T11:10:08.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:08 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:08.107+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.654238+0000 front 2024-03-27T11:09:23.654318+0000 (oldest deadline 2024-03-27T11:09:48.954440+0000) 2024-03-27T11:10:08.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:08 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:08.107+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:35.957512+0000 front 2024-03-27T11:09:35.957437+0000 (oldest deadline 2024-03-27T11:10:01.857676+0000) 2024-03-27T11:10:08.673 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:08 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:08.214+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.236297+0000 front 2024-03-27T11:09:28.236728+0000 (oldest deadline 2024-03-27T11:09:51.136525+0000) 2024-03-27T11:10:08.673 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:08 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:08.214+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.340023+0000 front 2024-03-27T11:09:36.340351+0000 (oldest deadline 2024-03-27T11:10:02.239659+0000) 2024-03-27T11:10:08.673 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:08 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:08.295+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.140407+0000 front 2024-03-27T11:09:28.140164+0000 (oldest deadline 2024-03-27T11:09:54.040233+0000) 2024-03-27T11:10:08.674 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:08 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:08.295+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.844416+0000 front 2024-03-27T11:09:36.843999+0000 (oldest deadline 2024-03-27T11:10:01.544084+0000) 2024-03-27T11:10:08.748 INFO:journalctl@ceph.osd.4.smithi114.stdout:Mar 27 11:10:08 smithi114 podman[70885]: 2024-03-27 11:10:08.422112778 +0000 UTC m=+0.454669620 container start ce6ac66a4a25b2422865debd336b4f55d052725361c74d5513b9b3e9b04fa36f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-4-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240326.131558-155268c, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e) 2024-03-27T11:10:08.748 INFO:journalctl@ceph.osd.4.smithi114.stdout:Mar 27 11:10:08 smithi114 podman[70885]: 2024-03-27 11:10:08.54333379 +0000 UTC m=+0.575890632 container attach ce6ac66a4a25b2422865debd336b4f55d052725361c74d5513b9b3e9b04fa36f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-4-deactivate, ceph=True, RELEASE=wip-rishabh-testing-20240326.131558-155268c, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-27T11:10:09.013 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:08 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:08.780+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.380254+0000 front 2024-03-27T11:09:23.380098+0000 (oldest deadline 2024-03-27T11:09:49.280089+0000) 2024-03-27T11:10:09.013 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:08 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:08.780+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:34.582052+0000 front 2024-03-27T11:09:34.582101+0000 (oldest deadline 2024-03-27T11:10:00.482194+0000) 2024-03-27T11:10:09.014 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:08 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:08.780+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:45.784185+0000 front 2024-03-27T11:09:45.784030+0000 (oldest deadline 2024-03-27T11:10:08.683959+0000) 2024-03-27T11:10:09.014 INFO:journalctl@ceph.osd.4.smithi114.stdout:Mar 27 11:10:08 smithi114 podman[70885]: 2024-03-27 11:10:08.895665175 +0000 UTC m=+0.928222020 container died ce6ac66a4a25b2422865debd336b4f55d052725361c74d5513b9b3e9b04fa36f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-4-deactivate, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-27T11:10:09.197 INFO:journalctl@ceph.osd.5.smithi177.stdout:Mar 27 11:10:08 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-5[47487]: 2024-03-27T11:10:08.960+0000 7fe117000700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:25.457341+0000 front 2024-03-27T11:09:25.457314+0000 (oldest deadline 2024-03-27T11:09:50.157224+0000) 2024-03-27T11:10:09.198 INFO:journalctl@ceph.osd.5.smithi177.stdout:Mar 27 11:10:08 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-5[47487]: 2024-03-27T11:10:08.960+0000 7fe117000700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.560634+0000 front 2024-03-27T11:09:36.561253+0000 (oldest deadline 2024-03-27T11:10:02.460286+0000) 2024-03-27T11:10:09.387 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 27 11:10:09 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-6[60366]: 2024-03-27T11:10:09.208+0000 7f8ead400700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:27.412885+0000 front 2024-03-27T11:09:27.412671+0000 (oldest deadline 2024-03-27T11:09:49.712654+0000) 2024-03-27T11:10:09.387 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 27 11:10:09 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-6[60366]: 2024-03-27T11:10:09.208+0000 7f8ead400700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:35.416884+0000 front 2024-03-27T11:09:35.416826+0000 (oldest deadline 2024-03-27T11:09:59.516984+0000) 2024-03-27T11:10:09.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:09 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:09.107+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.654238+0000 front 2024-03-27T11:09:23.654318+0000 (oldest deadline 2024-03-27T11:09:48.954440+0000) 2024-03-27T11:10:09.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:09 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:09.107+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:35.957512+0000 front 2024-03-27T11:09:35.957437+0000 (oldest deadline 2024-03-27T11:10:01.857676+0000) 2024-03-27T11:10:09.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:09 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:09.107+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:44.160107+0000 front 2024-03-27T11:09:44.160318+0000 (oldest deadline 2024-03-27T11:10:08.260147+0000) 2024-03-27T11:10:09.449 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:09 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:09.229+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.236297+0000 front 2024-03-27T11:09:28.236728+0000 (oldest deadline 2024-03-27T11:09:51.136525+0000) 2024-03-27T11:10:09.449 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:09 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:09.229+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.340023+0000 front 2024-03-27T11:09:36.340351+0000 (oldest deadline 2024-03-27T11:10:02.239659+0000) 2024-03-27T11:10:09.449 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:09 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:09.298+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.140407+0000 front 2024-03-27T11:09:28.140164+0000 (oldest deadline 2024-03-27T11:09:54.040233+0000) 2024-03-27T11:10:09.449 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:09 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:09.298+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.844416+0000 front 2024-03-27T11:09:36.843999+0000 (oldest deadline 2024-03-27T11:10:01.544084+0000) 2024-03-27T11:10:09.513 INFO:journalctl@ceph.osd.7.smithi114.stdout:Mar 27 11:10:09 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-7[53256]: 2024-03-27T11:10:09.253+0000 7fd5e5800700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.916460+0000 front 2024-03-27T11:09:36.916564+0000 (oldest deadline 2024-03-27T11:10:01.016630+0000) 2024-03-27T11:10:09.513 INFO:journalctl@ceph.osd.7.smithi114.stdout:Mar 27 11:10:09 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-7[53256]: 2024-03-27T11:10:09.253+0000 7fd5e5800700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:43.918347+0000 front 2024-03-27T11:09:43.918473+0000 (oldest deadline 2024-03-27T11:10:08.618602+0000) 2024-03-27T11:10:09.864 DEBUG:teuthology.orchestra.run.smithi114:> sudo pkill -f 'journalctl -f -n 0 -u ceph-adca88c4-ec26-11ee-95d0-87774f69a715@osd.4.service' 2024-03-27T11:10:09.959 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-27T11:10:09.959 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-03-27T11:10:09.959 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-03-27T11:10:09.959 DEBUG:teuthology.orchestra.run.smithi177:> sudo systemctl stop ceph-adca88c4-ec26-11ee-95d0-87774f69a715@osd.5 2024-03-27T11:10:10.156 INFO:journalctl@ceph.osd.5.smithi177.stdout:Mar 27 11:10:09 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-5[47487]: 2024-03-27T11:10:09.922+0000 7fe117000700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:25.457341+0000 front 2024-03-27T11:09:25.457314+0000 (oldest deadline 2024-03-27T11:09:50.157224+0000) 2024-03-27T11:10:10.156 INFO:journalctl@ceph.osd.5.smithi177.stdout:Mar 27 11:10:09 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-5[47487]: 2024-03-27T11:10:09.922+0000 7fe117000700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.560634+0000 front 2024-03-27T11:09:36.561253+0000 (oldest deadline 2024-03-27T11:10:02.460286+0000) 2024-03-27T11:10:10.156 INFO:journalctl@ceph.osd.5.smithi177.stdout:Mar 27 11:10:10 smithi177 systemd[1]: Stopping Ceph osd.5 for adca88c4-ec26-11ee-95d0-87774f69a715... 2024-03-27T11:10:10.193 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:09 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:09.820+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.380254+0000 front 2024-03-27T11:09:23.380098+0000 (oldest deadline 2024-03-27T11:09:49.280089+0000) 2024-03-27T11:10:10.193 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:09 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:09.820+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:34.582052+0000 front 2024-03-27T11:09:34.582101+0000 (oldest deadline 2024-03-27T11:10:00.482194+0000) 2024-03-27T11:10:10.193 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:09 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:09.820+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:45.784185+0000 front 2024-03-27T11:09:45.784030+0000 (oldest deadline 2024-03-27T11:10:08.683959+0000) 2024-03-27T11:10:10.387 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 27 11:10:10 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-6[60366]: 2024-03-27T11:10:10.169+0000 7f8ead400700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:27.412885+0000 front 2024-03-27T11:09:27.412671+0000 (oldest deadline 2024-03-27T11:09:49.712654+0000) 2024-03-27T11:10:10.388 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 27 11:10:10 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-6[60366]: 2024-03-27T11:10:10.169+0000 7f8ead400700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:35.416884+0000 front 2024-03-27T11:09:35.416826+0000 (oldest deadline 2024-03-27T11:09:59.516984+0000) 2024-03-27T11:10:10.388 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 27 11:10:10 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-6[60366]: 2024-03-27T11:10:10.169+0000 7f8ead400700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:47.119756+0000 front 2024-03-27T11:09:47.120188+0000 (oldest deadline 2024-03-27T11:10:10.019663+0000) 2024-03-27T11:10:10.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:10 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:10.103+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.654238+0000 front 2024-03-27T11:09:23.654318+0000 (oldest deadline 2024-03-27T11:09:48.954440+0000) 2024-03-27T11:10:10.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:10 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:10.103+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:35.957512+0000 front 2024-03-27T11:09:35.957437+0000 (oldest deadline 2024-03-27T11:10:01.857676+0000) 2024-03-27T11:10:10.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:10 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:10.103+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:44.160107+0000 front 2024-03-27T11:09:44.160318+0000 (oldest deadline 2024-03-27T11:10:08.260147+0000) 2024-03-27T11:10:10.423 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:10 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:10.186+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.236297+0000 front 2024-03-27T11:09:28.236728+0000 (oldest deadline 2024-03-27T11:09:51.136525+0000) 2024-03-27T11:10:10.423 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:10 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:10.186+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.340023+0000 front 2024-03-27T11:09:36.340351+0000 (oldest deadline 2024-03-27T11:10:02.239659+0000) 2024-03-27T11:10:10.423 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:10 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:10.327+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.140407+0000 front 2024-03-27T11:09:28.140164+0000 (oldest deadline 2024-03-27T11:09:54.040233+0000) 2024-03-27T11:10:10.423 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:10 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:10.327+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.844416+0000 front 2024-03-27T11:09:36.843999+0000 (oldest deadline 2024-03-27T11:10:01.544084+0000) 2024-03-27T11:10:10.423 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:10 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:10.327+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:47.348157+0000 front 2024-03-27T11:09:47.348480+0000 (oldest deadline 2024-03-27T11:10:09.647966+0000) 2024-03-27T11:10:10.513 INFO:journalctl@ceph.osd.7.smithi114.stdout:Mar 27 11:10:10 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-7[53256]: 2024-03-27T11:10:10.224+0000 7fd5e5800700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.916460+0000 front 2024-03-27T11:09:36.916564+0000 (oldest deadline 2024-03-27T11:10:01.016630+0000) 2024-03-27T11:10:10.513 INFO:journalctl@ceph.osd.7.smithi114.stdout:Mar 27 11:10:10 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-7[53256]: 2024-03-27T11:10:10.224+0000 7fd5e5800700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:43.918347+0000 front 2024-03-27T11:09:43.918473+0000 (oldest deadline 2024-03-27T11:10:08.618602+0000) 2024-03-27T11:10:10.916 INFO:journalctl@ceph.osd.5.smithi177.stdout:Mar 27 11:10:10 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-5[47487]: 2024-03-27T11:10:10.579+0000 7fe120800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-27T11:10:10.917 INFO:journalctl@ceph.osd.5.smithi177.stdout:Mar 27 11:10:10 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-5[47487]: 2024-03-27T11:10:10.579+0000 7fe120800700 -1 osd.5 92 *** Got signal Terminated *** 2024-03-27T11:10:10.917 INFO:journalctl@ceph.osd.5.smithi177.stdout:Mar 27 11:10:10 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-5[47487]: 2024-03-27T11:10:10.579+0000 7fe120800700 -1 osd.5 92 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-27T11:10:11.159 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:10 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:10.837+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.380254+0000 front 2024-03-27T11:09:23.380098+0000 (oldest deadline 2024-03-27T11:09:49.280089+0000) 2024-03-27T11:10:11.160 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:10 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:10.837+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:34.582052+0000 front 2024-03-27T11:09:34.582101+0000 (oldest deadline 2024-03-27T11:10:00.482194+0000) 2024-03-27T11:10:11.160 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:10 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:10.837+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:45.784185+0000 front 2024-03-27T11:09:45.784030+0000 (oldest deadline 2024-03-27T11:10:08.683959+0000) 2024-03-27T11:10:11.160 INFO:journalctl@ceph.osd.7.smithi114.stdout:Mar 27 11:10:11 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-7[53256]: 2024-03-27T11:10:11.190+0000 7fd5e5800700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.916460+0000 front 2024-03-27T11:09:36.916564+0000 (oldest deadline 2024-03-27T11:10:01.016630+0000) 2024-03-27T11:10:11.160 INFO:journalctl@ceph.osd.7.smithi114.stdout:Mar 27 11:10:11 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-7[53256]: 2024-03-27T11:10:11.190+0000 7fd5e5800700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:43.918347+0000 front 2024-03-27T11:09:43.918473+0000 (oldest deadline 2024-03-27T11:10:08.618602+0000) 2024-03-27T11:10:11.173 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:11 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:11.187+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.236297+0000 front 2024-03-27T11:09:28.236728+0000 (oldest deadline 2024-03-27T11:09:51.136525+0000) 2024-03-27T11:10:11.173 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:11 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:11.187+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.340023+0000 front 2024-03-27T11:09:36.340351+0000 (oldest deadline 2024-03-27T11:10:02.239659+0000) 2024-03-27T11:10:11.173 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:11 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:11.187+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:46.143084+0000 front 2024-03-27T11:09:46.143228+0000 (oldest deadline 2024-03-27T11:10:10.843246+0000) 2024-03-27T11:10:11.173 INFO:journalctl@ceph.osd.5.smithi177.stdout:Mar 27 11:10:10 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-5[47487]: 2024-03-27T11:10:10.948+0000 7fe117000700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:25.457341+0000 front 2024-03-27T11:09:25.457314+0000 (oldest deadline 2024-03-27T11:09:50.157224+0000) 2024-03-27T11:10:11.174 INFO:journalctl@ceph.osd.5.smithi177.stdout:Mar 27 11:10:10 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-5[47487]: 2024-03-27T11:10:10.948+0000 7fe117000700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.560634+0000 front 2024-03-27T11:09:36.561253+0000 (oldest deadline 2024-03-27T11:10:02.460286+0000) 2024-03-27T11:10:11.174 INFO:journalctl@ceph.osd.5.smithi177.stdout:Mar 27 11:10:10 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-5[47487]: 2024-03-27T11:10:10.948+0000 7fe117000700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:42.460953+0000 front 2024-03-27T11:09:42.461038+0000 (oldest deadline 2024-03-27T11:10:10.062141+0000) 2024-03-27T11:10:11.387 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 27 11:10:11 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-6[60366]: 2024-03-27T11:10:11.148+0000 7f8ead400700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:27.412885+0000 front 2024-03-27T11:09:27.412671+0000 (oldest deadline 2024-03-27T11:09:49.712654+0000) 2024-03-27T11:10:11.388 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 27 11:10:11 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-6[60366]: 2024-03-27T11:10:11.148+0000 7f8ead400700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:35.416884+0000 front 2024-03-27T11:09:35.416826+0000 (oldest deadline 2024-03-27T11:09:59.516984+0000) 2024-03-27T11:10:11.388 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 27 11:10:11 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-6[60366]: 2024-03-27T11:10:11.148+0000 7f8ead400700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:47.119756+0000 front 2024-03-27T11:09:47.120188+0000 (oldest deadline 2024-03-27T11:10:10.019663+0000) 2024-03-27T11:10:11.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:11 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:11.088+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.654238+0000 front 2024-03-27T11:09:23.654318+0000 (oldest deadline 2024-03-27T11:09:48.954440+0000) 2024-03-27T11:10:11.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:11 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:11.088+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:35.957512+0000 front 2024-03-27T11:09:35.957437+0000 (oldest deadline 2024-03-27T11:10:01.857676+0000) 2024-03-27T11:10:11.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:11 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:11.088+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:44.160107+0000 front 2024-03-27T11:09:44.160318+0000 (oldest deadline 2024-03-27T11:10:08.260147+0000) 2024-03-27T11:10:11.673 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:11 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:11.317+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.140407+0000 front 2024-03-27T11:09:28.140164+0000 (oldest deadline 2024-03-27T11:09:54.040233+0000) 2024-03-27T11:10:11.673 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:11 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:11.317+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.844416+0000 front 2024-03-27T11:09:36.843999+0000 (oldest deadline 2024-03-27T11:10:01.544084+0000) 2024-03-27T11:10:11.674 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:11 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:11.317+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:47.348157+0000 front 2024-03-27T11:09:47.348480+0000 (oldest deadline 2024-03-27T11:10:09.647966+0000) 2024-03-27T11:10:12.165 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:11 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:11.872+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.380254+0000 front 2024-03-27T11:09:23.380098+0000 (oldest deadline 2024-03-27T11:09:49.280089+0000) 2024-03-27T11:10:12.165 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:11 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:11.872+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:34.582052+0000 front 2024-03-27T11:09:34.582101+0000 (oldest deadline 2024-03-27T11:10:00.482194+0000) 2024-03-27T11:10:12.165 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:11 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:11.872+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:45.784185+0000 front 2024-03-27T11:09:45.784030+0000 (oldest deadline 2024-03-27T11:10:08.683959+0000) 2024-03-27T11:10:12.263 INFO:journalctl@ceph.osd.5.smithi177.stdout:Mar 27 11:10:11 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-5[47487]: 2024-03-27T11:10:11.958+0000 7fe117000700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:25.457341+0000 front 2024-03-27T11:09:25.457314+0000 (oldest deadline 2024-03-27T11:09:50.157224+0000) 2024-03-27T11:10:12.263 INFO:journalctl@ceph.osd.5.smithi177.stdout:Mar 27 11:10:11 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-5[47487]: 2024-03-27T11:10:11.958+0000 7fe117000700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.560634+0000 front 2024-03-27T11:09:36.561253+0000 (oldest deadline 2024-03-27T11:10:02.460286+0000) 2024-03-27T11:10:12.263 INFO:journalctl@ceph.osd.5.smithi177.stdout:Mar 27 11:10:11 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-5[47487]: 2024-03-27T11:10:11.958+0000 7fe117000700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:42.460953+0000 front 2024-03-27T11:09:42.461038+0000 (oldest deadline 2024-03-27T11:10:10.062141+0000) 2024-03-27T11:10:12.263 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:12 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:12.150+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.236297+0000 front 2024-03-27T11:09:28.236728+0000 (oldest deadline 2024-03-27T11:09:51.136525+0000) 2024-03-27T11:10:12.263 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:12 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:12.150+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.340023+0000 front 2024-03-27T11:09:36.340351+0000 (oldest deadline 2024-03-27T11:10:02.239659+0000) 2024-03-27T11:10:12.263 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:12 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:12.150+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:46.143084+0000 front 2024-03-27T11:09:46.143228+0000 (oldest deadline 2024-03-27T11:10:10.843246+0000) 2024-03-27T11:10:12.387 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 27 11:10:12 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-6[60366]: 2024-03-27T11:10:12.156+0000 7f8ead400700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:27.412885+0000 front 2024-03-27T11:09:27.412671+0000 (oldest deadline 2024-03-27T11:09:49.712654+0000) 2024-03-27T11:10:12.388 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 27 11:10:12 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-6[60366]: 2024-03-27T11:10:12.156+0000 7f8ead400700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:35.416884+0000 front 2024-03-27T11:09:35.416826+0000 (oldest deadline 2024-03-27T11:09:59.516984+0000) 2024-03-27T11:10:12.388 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 27 11:10:12 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-6[60366]: 2024-03-27T11:10:12.156+0000 7f8ead400700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:47.119756+0000 front 2024-03-27T11:09:47.120188+0000 (oldest deadline 2024-03-27T11:10:10.019663+0000) 2024-03-27T11:10:12.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:12 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:12.114+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.654238+0000 front 2024-03-27T11:09:23.654318+0000 (oldest deadline 2024-03-27T11:09:48.954440+0000) 2024-03-27T11:10:12.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:12 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:12.114+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:35.957512+0000 front 2024-03-27T11:09:35.957437+0000 (oldest deadline 2024-03-27T11:10:01.857676+0000) 2024-03-27T11:10:12.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:12 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:12.114+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:44.160107+0000 front 2024-03-27T11:09:44.160318+0000 (oldest deadline 2024-03-27T11:10:08.260147+0000) 2024-03-27T11:10:12.513 INFO:journalctl@ceph.osd.7.smithi114.stdout:Mar 27 11:10:12 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-7[53256]: 2024-03-27T11:10:12.195+0000 7fd5e5800700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.916460+0000 front 2024-03-27T11:09:36.916564+0000 (oldest deadline 2024-03-27T11:10:01.016630+0000) 2024-03-27T11:10:12.513 INFO:journalctl@ceph.osd.7.smithi114.stdout:Mar 27 11:10:12 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-7[53256]: 2024-03-27T11:10:12.196+0000 7fd5e5800700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:43.918347+0000 front 2024-03-27T11:09:43.918473+0000 (oldest deadline 2024-03-27T11:10:08.618602+0000) 2024-03-27T11:10:12.673 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:12 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:12.293+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.140407+0000 front 2024-03-27T11:09:28.140164+0000 (oldest deadline 2024-03-27T11:09:54.040233+0000) 2024-03-27T11:10:12.673 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:12 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:12.293+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.844416+0000 front 2024-03-27T11:09:36.843999+0000 (oldest deadline 2024-03-27T11:10:01.544084+0000) 2024-03-27T11:10:12.673 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:12 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:12.293+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:47.348157+0000 front 2024-03-27T11:09:47.348480+0000 (oldest deadline 2024-03-27T11:10:09.647966+0000) 2024-03-27T11:10:13.175 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:12 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:12.847+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.380254+0000 front 2024-03-27T11:09:23.380098+0000 (oldest deadline 2024-03-27T11:09:49.280089+0000) 2024-03-27T11:10:13.175 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:12 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:12.847+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:34.582052+0000 front 2024-03-27T11:09:34.582101+0000 (oldest deadline 2024-03-27T11:10:00.482194+0000) 2024-03-27T11:10:13.175 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:12 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:12.847+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:45.784185+0000 front 2024-03-27T11:09:45.784030+0000 (oldest deadline 2024-03-27T11:10:08.683959+0000) 2024-03-27T11:10:13.232 INFO:journalctl@ceph.osd.5.smithi177.stdout:Mar 27 11:10:12 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-5[47487]: 2024-03-27T11:10:12.966+0000 7fe117000700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:25.457341+0000 front 2024-03-27T11:09:25.457314+0000 (oldest deadline 2024-03-27T11:09:50.157224+0000) 2024-03-27T11:10:13.232 INFO:journalctl@ceph.osd.5.smithi177.stdout:Mar 27 11:10:12 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-5[47487]: 2024-03-27T11:10:12.966+0000 7fe117000700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.560634+0000 front 2024-03-27T11:09:36.561253+0000 (oldest deadline 2024-03-27T11:10:02.460286+0000) 2024-03-27T11:10:13.232 INFO:journalctl@ceph.osd.5.smithi177.stdout:Mar 27 11:10:12 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-5[47487]: 2024-03-27T11:10:12.966+0000 7fe117000700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:42.460953+0000 front 2024-03-27T11:09:42.461038+0000 (oldest deadline 2024-03-27T11:10:10.062141+0000) 2024-03-27T11:10:13.232 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:13 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:13.189+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.236297+0000 front 2024-03-27T11:09:28.236728+0000 (oldest deadline 2024-03-27T11:09:51.136525+0000) 2024-03-27T11:10:13.233 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:13 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:13.189+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.340023+0000 front 2024-03-27T11:09:36.340351+0000 (oldest deadline 2024-03-27T11:10:02.239659+0000) 2024-03-27T11:10:13.233 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:13 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:13.189+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:46.143084+0000 front 2024-03-27T11:09:46.143228+0000 (oldest deadline 2024-03-27T11:10:10.843246+0000) 2024-03-27T11:10:13.387 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 27 11:10:13 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-6[60366]: 2024-03-27T11:10:13.158+0000 7f8ead400700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:27.412885+0000 front 2024-03-27T11:09:27.412671+0000 (oldest deadline 2024-03-27T11:09:49.712654+0000) 2024-03-27T11:10:13.388 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 27 11:10:13 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-6[60366]: 2024-03-27T11:10:13.158+0000 7f8ead400700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:35.416884+0000 front 2024-03-27T11:09:35.416826+0000 (oldest deadline 2024-03-27T11:09:59.516984+0000) 2024-03-27T11:10:13.388 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 27 11:10:13 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-6[60366]: 2024-03-27T11:10:13.158+0000 7f8ead400700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:47.119756+0000 front 2024-03-27T11:09:47.120188+0000 (oldest deadline 2024-03-27T11:10:10.019663+0000) 2024-03-27T11:10:13.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:13 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:13.093+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.654238+0000 front 2024-03-27T11:09:23.654318+0000 (oldest deadline 2024-03-27T11:09:48.954440+0000) 2024-03-27T11:10:13.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:13 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:13.093+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:35.957512+0000 front 2024-03-27T11:09:35.957437+0000 (oldest deadline 2024-03-27T11:10:01.857676+0000) 2024-03-27T11:10:13.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:13 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:13.093+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:44.160107+0000 front 2024-03-27T11:09:44.160318+0000 (oldest deadline 2024-03-27T11:10:08.260147+0000) 2024-03-27T11:10:13.513 INFO:journalctl@ceph.osd.7.smithi114.stdout:Mar 27 11:10:13 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-7[53256]: 2024-03-27T11:10:13.206+0000 7fd5e5800700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.916460+0000 front 2024-03-27T11:09:36.916564+0000 (oldest deadline 2024-03-27T11:10:01.016630+0000) 2024-03-27T11:10:13.513 INFO:journalctl@ceph.osd.7.smithi114.stdout:Mar 27 11:10:13 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-7[53256]: 2024-03-27T11:10:13.206+0000 7fd5e5800700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:43.918347+0000 front 2024-03-27T11:09:43.918473+0000 (oldest deadline 2024-03-27T11:10:08.618602+0000) 2024-03-27T11:10:13.673 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:13 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:13.263+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.140407+0000 front 2024-03-27T11:09:28.140164+0000 (oldest deadline 2024-03-27T11:09:54.040233+0000) 2024-03-27T11:10:13.673 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:13 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:13.263+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.844416+0000 front 2024-03-27T11:09:36.843999+0000 (oldest deadline 2024-03-27T11:10:01.544084+0000) 2024-03-27T11:10:13.673 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:13 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:13.263+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:47.348157+0000 front 2024-03-27T11:09:47.348480+0000 (oldest deadline 2024-03-27T11:10:09.647966+0000) 2024-03-27T11:10:14.176 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:13 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:13.831+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.380254+0000 front 2024-03-27T11:09:23.380098+0000 (oldest deadline 2024-03-27T11:09:49.280089+0000) 2024-03-27T11:10:14.176 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:13 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:13.831+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:34.582052+0000 front 2024-03-27T11:09:34.582101+0000 (oldest deadline 2024-03-27T11:10:00.482194+0000) 2024-03-27T11:10:14.176 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:13 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:13.831+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:45.784185+0000 front 2024-03-27T11:09:45.784030+0000 (oldest deadline 2024-03-27T11:10:08.683959+0000) 2024-03-27T11:10:14.255 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:14 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:14.197+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.236297+0000 front 2024-03-27T11:09:28.236728+0000 (oldest deadline 2024-03-27T11:09:51.136525+0000) 2024-03-27T11:10:14.255 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:14 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:14.197+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.340023+0000 front 2024-03-27T11:09:36.340351+0000 (oldest deadline 2024-03-27T11:10:02.239659+0000) 2024-03-27T11:10:14.255 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:14 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:14.197+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:46.143084+0000 front 2024-03-27T11:09:46.143228+0000 (oldest deadline 2024-03-27T11:10:10.843246+0000) 2024-03-27T11:10:14.255 INFO:journalctl@ceph.osd.5.smithi177.stdout:Mar 27 11:10:14 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-5[47487]: 2024-03-27T11:10:14.015+0000 7fe117000700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:25.457341+0000 front 2024-03-27T11:09:25.457314+0000 (oldest deadline 2024-03-27T11:09:50.157224+0000) 2024-03-27T11:10:14.255 INFO:journalctl@ceph.osd.5.smithi177.stdout:Mar 27 11:10:14 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-5[47487]: 2024-03-27T11:10:14.015+0000 7fe117000700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.560634+0000 front 2024-03-27T11:09:36.561253+0000 (oldest deadline 2024-03-27T11:10:02.460286+0000) 2024-03-27T11:10:14.255 INFO:journalctl@ceph.osd.5.smithi177.stdout:Mar 27 11:10:14 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-5[47487]: 2024-03-27T11:10:14.015+0000 7fe117000700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:42.460953+0000 front 2024-03-27T11:09:42.461038+0000 (oldest deadline 2024-03-27T11:10:10.062141+0000) 2024-03-27T11:10:14.387 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 27 11:10:14 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-6[60366]: 2024-03-27T11:10:14.178+0000 7f8ead400700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:27.412885+0000 front 2024-03-27T11:09:27.412671+0000 (oldest deadline 2024-03-27T11:09:49.712654+0000) 2024-03-27T11:10:14.387 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 27 11:10:14 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-6[60366]: 2024-03-27T11:10:14.178+0000 7f8ead400700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:35.416884+0000 front 2024-03-27T11:09:35.416826+0000 (oldest deadline 2024-03-27T11:09:59.516984+0000) 2024-03-27T11:10:14.387 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 27 11:10:14 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-6[60366]: 2024-03-27T11:10:14.178+0000 7f8ead400700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:47.119756+0000 front 2024-03-27T11:09:47.120188+0000 (oldest deadline 2024-03-27T11:10:10.019663+0000) 2024-03-27T11:10:14.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:14 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:14.049+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.654238+0000 front 2024-03-27T11:09:23.654318+0000 (oldest deadline 2024-03-27T11:09:48.954440+0000) 2024-03-27T11:10:14.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:14 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:14.049+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:35.957512+0000 front 2024-03-27T11:09:35.957437+0000 (oldest deadline 2024-03-27T11:10:01.857676+0000) 2024-03-27T11:10:14.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:14 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:14.049+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:44.160107+0000 front 2024-03-27T11:09:44.160318+0000 (oldest deadline 2024-03-27T11:10:08.260147+0000) 2024-03-27T11:10:14.513 INFO:journalctl@ceph.osd.7.smithi114.stdout:Mar 27 11:10:14 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-7[53256]: 2024-03-27T11:10:14.207+0000 7fd5e5800700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.916460+0000 front 2024-03-27T11:09:36.916564+0000 (oldest deadline 2024-03-27T11:10:01.016630+0000) 2024-03-27T11:10:14.513 INFO:journalctl@ceph.osd.7.smithi114.stdout:Mar 27 11:10:14 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-7[53256]: 2024-03-27T11:10:14.207+0000 7fd5e5800700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:43.918347+0000 front 2024-03-27T11:09:43.918473+0000 (oldest deadline 2024-03-27T11:10:08.618602+0000) 2024-03-27T11:10:14.673 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:14 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:14.286+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.140407+0000 front 2024-03-27T11:09:28.140164+0000 (oldest deadline 2024-03-27T11:09:54.040233+0000) 2024-03-27T11:10:14.673 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:14 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:14.286+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.844416+0000 front 2024-03-27T11:09:36.843999+0000 (oldest deadline 2024-03-27T11:10:01.544084+0000) 2024-03-27T11:10:14.673 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:14 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:14.286+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:47.348157+0000 front 2024-03-27T11:09:47.348480+0000 (oldest deadline 2024-03-27T11:10:09.647966+0000) 2024-03-27T11:10:15.226 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:14 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:14.841+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.380254+0000 front 2024-03-27T11:09:23.380098+0000 (oldest deadline 2024-03-27T11:09:49.280089+0000) 2024-03-27T11:10:15.226 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:14 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:14.841+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:34.582052+0000 front 2024-03-27T11:09:34.582101+0000 (oldest deadline 2024-03-27T11:10:00.482194+0000) 2024-03-27T11:10:15.227 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:14 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:14.841+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:45.784185+0000 front 2024-03-27T11:09:45.784030+0000 (oldest deadline 2024-03-27T11:10:08.683959+0000) 2024-03-27T11:10:15.285 INFO:journalctl@ceph.osd.5.smithi177.stdout:Mar 27 11:10:15 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-5[47487]: 2024-03-27T11:10:15.001+0000 7fe117000700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:25.457341+0000 front 2024-03-27T11:09:25.457314+0000 (oldest deadline 2024-03-27T11:09:50.157224+0000) 2024-03-27T11:10:15.285 INFO:journalctl@ceph.osd.5.smithi177.stdout:Mar 27 11:10:15 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-5[47487]: 2024-03-27T11:10:15.002+0000 7fe117000700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.560634+0000 front 2024-03-27T11:09:36.561253+0000 (oldest deadline 2024-03-27T11:10:02.460286+0000) 2024-03-27T11:10:15.286 INFO:journalctl@ceph.osd.5.smithi177.stdout:Mar 27 11:10:15 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-5[47487]: 2024-03-27T11:10:15.002+0000 7fe117000700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:42.460953+0000 front 2024-03-27T11:09:42.461038+0000 (oldest deadline 2024-03-27T11:10:10.062141+0000) 2024-03-27T11:10:15.286 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:15 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:15.202+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.236297+0000 front 2024-03-27T11:09:28.236728+0000 (oldest deadline 2024-03-27T11:09:51.136525+0000) 2024-03-27T11:10:15.286 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:15 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:15.202+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.340023+0000 front 2024-03-27T11:09:36.340351+0000 (oldest deadline 2024-03-27T11:10:02.239659+0000) 2024-03-27T11:10:15.286 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:15 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:15.202+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:46.143084+0000 front 2024-03-27T11:09:46.143228+0000 (oldest deadline 2024-03-27T11:10:10.843246+0000) 2024-03-27T11:10:15.387 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 27 11:10:15 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-6[60366]: 2024-03-27T11:10:15.182+0000 7f8ead400700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:27.412885+0000 front 2024-03-27T11:09:27.412671+0000 (oldest deadline 2024-03-27T11:09:49.712654+0000) 2024-03-27T11:10:15.387 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 27 11:10:15 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-6[60366]: 2024-03-27T11:10:15.182+0000 7f8ead400700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:35.416884+0000 front 2024-03-27T11:09:35.416826+0000 (oldest deadline 2024-03-27T11:09:59.516984+0000) 2024-03-27T11:10:15.387 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 27 11:10:15 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-6[60366]: 2024-03-27T11:10:15.182+0000 7f8ead400700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:47.119756+0000 front 2024-03-27T11:09:47.120188+0000 (oldest deadline 2024-03-27T11:10:10.019663+0000) 2024-03-27T11:10:15.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:15 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:15.059+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.654238+0000 front 2024-03-27T11:09:23.654318+0000 (oldest deadline 2024-03-27T11:09:48.954440+0000) 2024-03-27T11:10:15.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:15 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:15.059+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:35.957512+0000 front 2024-03-27T11:09:35.957437+0000 (oldest deadline 2024-03-27T11:10:01.857676+0000) 2024-03-27T11:10:15.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:15 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:15.059+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:44.160107+0000 front 2024-03-27T11:09:44.160318+0000 (oldest deadline 2024-03-27T11:10:08.260147+0000) 2024-03-27T11:10:15.513 INFO:journalctl@ceph.osd.7.smithi114.stdout:Mar 27 11:10:15 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-7[53256]: 2024-03-27T11:10:15.257+0000 7fd5e5800700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.916460+0000 front 2024-03-27T11:09:36.916564+0000 (oldest deadline 2024-03-27T11:10:01.016630+0000) 2024-03-27T11:10:15.513 INFO:journalctl@ceph.osd.7.smithi114.stdout:Mar 27 11:10:15 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-7[53256]: 2024-03-27T11:10:15.257+0000 7fd5e5800700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:43.918347+0000 front 2024-03-27T11:09:43.918473+0000 (oldest deadline 2024-03-27T11:10:08.618602+0000) 2024-03-27T11:10:15.673 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:15 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:15.316+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.140407+0000 front 2024-03-27T11:09:28.140164+0000 (oldest deadline 2024-03-27T11:09:54.040233+0000) 2024-03-27T11:10:15.674 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:15 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:15.316+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.844416+0000 front 2024-03-27T11:09:36.843999+0000 (oldest deadline 2024-03-27T11:10:01.544084+0000) 2024-03-27T11:10:15.674 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:15 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:15.316+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:47.348157+0000 front 2024-03-27T11:09:47.348480+0000 (oldest deadline 2024-03-27T11:10:09.647966+0000) 2024-03-27T11:10:16.261 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:15 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:15.808+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.380254+0000 front 2024-03-27T11:09:23.380098+0000 (oldest deadline 2024-03-27T11:09:49.280089+0000) 2024-03-27T11:10:16.261 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:15 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:15.808+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:34.582052+0000 front 2024-03-27T11:09:34.582101+0000 (oldest deadline 2024-03-27T11:10:00.482194+0000) 2024-03-27T11:10:16.261 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:15 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:15.808+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:45.784185+0000 front 2024-03-27T11:09:45.784030+0000 (oldest deadline 2024-03-27T11:10:08.683959+0000) 2024-03-27T11:10:16.387 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 27 11:10:16 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-6[60366]: 2024-03-27T11:10:16.199+0000 7f8ead400700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:27.412885+0000 front 2024-03-27T11:09:27.412671+0000 (oldest deadline 2024-03-27T11:09:49.712654+0000) 2024-03-27T11:10:16.387 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 27 11:10:16 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-6[60366]: 2024-03-27T11:10:16.199+0000 7f8ead400700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:35.416884+0000 front 2024-03-27T11:09:35.416826+0000 (oldest deadline 2024-03-27T11:09:59.516984+0000) 2024-03-27T11:10:16.388 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 27 11:10:16 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-6[60366]: 2024-03-27T11:10:16.199+0000 7f8ead400700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:47.119756+0000 front 2024-03-27T11:09:47.120188+0000 (oldest deadline 2024-03-27T11:10:10.019663+0000) 2024-03-27T11:10:16.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:16 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:16.030+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.654238+0000 front 2024-03-27T11:09:23.654318+0000 (oldest deadline 2024-03-27T11:09:48.954440+0000) 2024-03-27T11:10:16.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:16 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:16.030+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:35.957512+0000 front 2024-03-27T11:09:35.957437+0000 (oldest deadline 2024-03-27T11:10:01.857676+0000) 2024-03-27T11:10:16.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:16 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:16.030+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:44.160107+0000 front 2024-03-27T11:09:44.160318+0000 (oldest deadline 2024-03-27T11:10:08.260147+0000) 2024-03-27T11:10:16.513 INFO:journalctl@ceph.osd.7.smithi114.stdout:Mar 27 11:10:16 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-7[53256]: 2024-03-27T11:10:16.293+0000 7fd5e5800700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.916460+0000 front 2024-03-27T11:09:36.916564+0000 (oldest deadline 2024-03-27T11:10:01.016630+0000) 2024-03-27T11:10:16.513 INFO:journalctl@ceph.osd.7.smithi114.stdout:Mar 27 11:10:16 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-7[53256]: 2024-03-27T11:10:16.293+0000 7fd5e5800700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:43.918347+0000 front 2024-03-27T11:09:43.918473+0000 (oldest deadline 2024-03-27T11:10:08.618602+0000) 2024-03-27T11:10:16.561 INFO:journalctl@ceph.osd.5.smithi177.stdout:Mar 27 11:10:16 smithi177 podman[71709]: 2024-03-27 11:10:16.294898187 +0000 UTC m=+5.852888107 container died b7b79355c0dc147faa998e47ccaaa03920c11987c0b890c3e5fb7d44b0382e28 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240326.131558-155268c, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-03-27T11:10:16.561 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:16 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:16.216+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.236297+0000 front 2024-03-27T11:09:28.236728+0000 (oldest deadline 2024-03-27T11:09:51.136525+0000) 2024-03-27T11:10:16.562 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:16 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:16.216+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.340023+0000 front 2024-03-27T11:09:36.340351+0000 (oldest deadline 2024-03-27T11:10:02.239659+0000) 2024-03-27T11:10:16.562 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:16 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:16.216+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:46.143084+0000 front 2024-03-27T11:09:46.143228+0000 (oldest deadline 2024-03-27T11:10:10.843246+0000) 2024-03-27T11:10:16.562 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:16 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:16.338+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.140407+0000 front 2024-03-27T11:09:28.140164+0000 (oldest deadline 2024-03-27T11:09:54.040233+0000) 2024-03-27T11:10:16.562 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:16 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:16.338+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.844416+0000 front 2024-03-27T11:09:36.843999+0000 (oldest deadline 2024-03-27T11:10:01.544084+0000) 2024-03-27T11:10:16.562 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:16 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:16.338+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:47.348157+0000 front 2024-03-27T11:09:47.348480+0000 (oldest deadline 2024-03-27T11:10:09.647966+0000) 2024-03-27T11:10:16.923 INFO:journalctl@ceph.osd.5.smithi177.stdout:Mar 27 11:10:16 smithi177 podman[71709]: 2024-03-27 11:10:16.706973459 +0000 UTC m=+6.264963432 container cleanup b7b79355c0dc147faa998e47ccaaa03920c11987c0b890c3e5fb7d44b0382e28 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, GIT_CLEAN=True, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, RELEASE=wip-rishabh-testing-20240326.131558-155268c, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-03-27T11:10:16.923 INFO:journalctl@ceph.osd.5.smithi177.stdout:Mar 27 11:10:16 smithi177 bash[71709]: ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-5 2024-03-27T11:10:17.013 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:16 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:16.777+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.380254+0000 front 2024-03-27T11:09:23.380098+0000 (oldest deadline 2024-03-27T11:09:49.280089+0000) 2024-03-27T11:10:17.013 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:16 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:16.777+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:34.582052+0000 front 2024-03-27T11:09:34.582101+0000 (oldest deadline 2024-03-27T11:10:00.482194+0000) 2024-03-27T11:10:17.014 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:16 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:16.777+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:45.784185+0000 front 2024-03-27T11:09:45.784030+0000 (oldest deadline 2024-03-27T11:10:08.683959+0000) 2024-03-27T11:10:17.274 INFO:journalctl@ceph.osd.5.smithi177.stdout:Mar 27 11:10:17 smithi177 podman[71728]: 2024-03-27 11:10:17.007595257 +0000 UTC m=+0.703288700 container remove b7b79355c0dc147faa998e47ccaaa03920c11987c0b890c3e5fb7d44b0382e28 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-5, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.build-date=20240326, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0) 2024-03-27T11:10:17.274 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:17 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:17.175+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.236297+0000 front 2024-03-27T11:09:28.236728+0000 (oldest deadline 2024-03-27T11:09:51.136525+0000) 2024-03-27T11:10:17.274 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:17 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:17.175+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.340023+0000 front 2024-03-27T11:09:36.340351+0000 (oldest deadline 2024-03-27T11:10:02.239659+0000) 2024-03-27T11:10:17.274 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:17 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:17.175+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:46.143084+0000 front 2024-03-27T11:09:46.143228+0000 (oldest deadline 2024-03-27T11:10:10.843246+0000) 2024-03-27T11:10:17.275 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:17 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:17.305+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.140407+0000 front 2024-03-27T11:09:28.140164+0000 (oldest deadline 2024-03-27T11:09:54.040233+0000) 2024-03-27T11:10:17.275 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:17 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:17.305+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.844416+0000 front 2024-03-27T11:09:36.843999+0000 (oldest deadline 2024-03-27T11:10:01.544084+0000) 2024-03-27T11:10:17.275 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:17 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:17.305+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:47.348157+0000 front 2024-03-27T11:09:47.348480+0000 (oldest deadline 2024-03-27T11:10:09.647966+0000) 2024-03-27T11:10:17.387 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 27 11:10:17 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-6[60366]: 2024-03-27T11:10:17.188+0000 7f8ead400700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:27.412885+0000 front 2024-03-27T11:09:27.412671+0000 (oldest deadline 2024-03-27T11:09:49.712654+0000) 2024-03-27T11:10:17.387 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 27 11:10:17 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-6[60366]: 2024-03-27T11:10:17.188+0000 7f8ead400700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:35.416884+0000 front 2024-03-27T11:09:35.416826+0000 (oldest deadline 2024-03-27T11:09:59.516984+0000) 2024-03-27T11:10:17.387 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 27 11:10:17 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-6[60366]: 2024-03-27T11:10:17.188+0000 7f8ead400700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:47.119756+0000 front 2024-03-27T11:09:47.120188+0000 (oldest deadline 2024-03-27T11:10:10.019663+0000) 2024-03-27T11:10:17.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:16 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:16.997+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.654238+0000 front 2024-03-27T11:09:23.654318+0000 (oldest deadline 2024-03-27T11:09:48.954440+0000) 2024-03-27T11:10:17.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:16 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:16.997+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:35.957512+0000 front 2024-03-27T11:09:35.957437+0000 (oldest deadline 2024-03-27T11:10:01.857676+0000) 2024-03-27T11:10:17.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:16 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:16.997+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:44.160107+0000 front 2024-03-27T11:09:44.160318+0000 (oldest deadline 2024-03-27T11:10:08.260147+0000) 2024-03-27T11:10:17.763 INFO:journalctl@ceph.osd.7.smithi114.stdout:Mar 27 11:10:17 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-7[53256]: 2024-03-27T11:10:17.339+0000 7fd5e5800700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.916460+0000 front 2024-03-27T11:09:36.916564+0000 (oldest deadline 2024-03-27T11:10:01.016630+0000) 2024-03-27T11:10:17.763 INFO:journalctl@ceph.osd.7.smithi114.stdout:Mar 27 11:10:17 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-7[53256]: 2024-03-27T11:10:17.339+0000 7fd5e5800700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:43.918347+0000 front 2024-03-27T11:09:43.918473+0000 (oldest deadline 2024-03-27T11:10:08.618602+0000) 2024-03-27T11:10:17.923 INFO:journalctl@ceph.osd.5.smithi177.stdout:Mar 27 11:10:17 smithi177 podman[71815]: 2024-03-27 11:10:17.598947029 +0000 UTC m=+0.033927053 image pull b379bbd8da6b5d0fa9a3523941efac213b6815e1cbf05c6507954f0a3fe7a801 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772 2024-03-27T11:10:17.924 INFO:journalctl@ceph.osd.5.smithi177.stdout:Mar 27 11:10:17 smithi177 podman[71815]: 2024-03-27 11:10:17.708468835 +0000 UTC m=+0.143448849 container create cc864a02dc04a66af25c1dd06d3519b82eb51e5480b0f22b6cb4d8c21a2325b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-5-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, GIT_CLEAN=True, ceph=True, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, RELEASE=wip-rishabh-testing-20240326.131558-155268c, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-03-27T11:10:18.263 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:17 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:17.808+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.380254+0000 front 2024-03-27T11:09:23.380098+0000 (oldest deadline 2024-03-27T11:09:49.280089+0000) 2024-03-27T11:10:18.264 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:17 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:17.808+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:34.582052+0000 front 2024-03-27T11:09:34.582101+0000 (oldest deadline 2024-03-27T11:10:00.482194+0000) 2024-03-27T11:10:18.264 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:17 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:17.808+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:45.784185+0000 front 2024-03-27T11:09:45.784030+0000 (oldest deadline 2024-03-27T11:10:08.683959+0000) 2024-03-27T11:10:18.264 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:17 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:17.808+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:55.086370+0000 front 2024-03-27T11:09:55.086460+0000 (oldest deadline 2024-03-27T11:10:17.386639+0000) 2024-03-27T11:10:18.295 INFO:journalctl@ceph.osd.5.smithi177.stdout:Mar 27 11:10:17 smithi177 podman[71815]: 2024-03-27 11:10:17.986700109 +0000 UTC m=+0.421680128 container init cc864a02dc04a66af25c1dd06d3519b82eb51e5480b0f22b6cb4d8c21a2325b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-5-deactivate, RELEASE=wip-rishabh-testing-20240326.131558-155268c, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.build-date=20240326, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS) 2024-03-27T11:10:18.296 INFO:journalctl@ceph.osd.5.smithi177.stdout:Mar 27 11:10:17 smithi177 podman[71815]: 2024-03-27 11:10:17.997500973 +0000 UTC m=+0.432481008 container start cc864a02dc04a66af25c1dd06d3519b82eb51e5480b0f22b6cb4d8c21a2325b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-5-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, GIT_CLEAN=True) 2024-03-27T11:10:18.296 INFO:journalctl@ceph.osd.5.smithi177.stdout:Mar 27 11:10:18 smithi177 podman[71815]: 2024-03-27 11:10:18.077436337 +0000 UTC m=+0.512416367 container attach cc864a02dc04a66af25c1dd06d3519b82eb51e5480b0f22b6cb4d8c21a2325b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-5-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-03-27T11:10:18.296 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:18 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:18.156+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.236297+0000 front 2024-03-27T11:09:28.236728+0000 (oldest deadline 2024-03-27T11:09:51.136525+0000) 2024-03-27T11:10:18.296 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:18 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:18.156+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.340023+0000 front 2024-03-27T11:09:36.340351+0000 (oldest deadline 2024-03-27T11:10:02.239659+0000) 2024-03-27T11:10:18.296 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:18 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:18.156+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:46.143084+0000 front 2024-03-27T11:09:46.143228+0000 (oldest deadline 2024-03-27T11:10:10.843246+0000) 2024-03-27T11:10:18.387 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 27 11:10:18 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-6[60366]: 2024-03-27T11:10:18.164+0000 7f8ead400700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:27.412885+0000 front 2024-03-27T11:09:27.412671+0000 (oldest deadline 2024-03-27T11:09:49.712654+0000) 2024-03-27T11:10:18.387 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 27 11:10:18 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-6[60366]: 2024-03-27T11:10:18.164+0000 7f8ead400700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:35.416884+0000 front 2024-03-27T11:09:35.416826+0000 (oldest deadline 2024-03-27T11:09:59.516984+0000) 2024-03-27T11:10:18.388 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 27 11:10:18 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-6[60366]: 2024-03-27T11:10:18.164+0000 7f8ead400700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:47.119756+0000 front 2024-03-27T11:09:47.120188+0000 (oldest deadline 2024-03-27T11:10:10.019663+0000) 2024-03-27T11:10:18.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:17 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:17.962+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.654238+0000 front 2024-03-27T11:09:23.654318+0000 (oldest deadline 2024-03-27T11:09:48.954440+0000) 2024-03-27T11:10:18.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:17 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:17.962+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:35.957512+0000 front 2024-03-27T11:09:35.957437+0000 (oldest deadline 2024-03-27T11:10:01.857676+0000) 2024-03-27T11:10:18.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:17 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:17.962+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:44.160107+0000 front 2024-03-27T11:09:44.160318+0000 (oldest deadline 2024-03-27T11:10:08.260147+0000) 2024-03-27T11:10:18.673 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:18 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:18.326+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.140407+0000 front 2024-03-27T11:09:28.140164+0000 (oldest deadline 2024-03-27T11:09:54.040233+0000) 2024-03-27T11:10:18.673 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:18 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:18.327+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.844416+0000 front 2024-03-27T11:09:36.843999+0000 (oldest deadline 2024-03-27T11:10:01.544084+0000) 2024-03-27T11:10:18.674 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:18 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:18.327+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:47.348157+0000 front 2024-03-27T11:09:47.348480+0000 (oldest deadline 2024-03-27T11:10:09.647966+0000) 2024-03-27T11:10:18.674 INFO:journalctl@ceph.osd.5.smithi177.stdout:Mar 27 11:10:18 smithi177 podman[71815]: 2024-03-27 11:10:18.47327684 +0000 UTC m=+0.908256914 container died cc864a02dc04a66af25c1dd06d3519b82eb51e5480b0f22b6cb4d8c21a2325b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-5-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240326.131558-155268c, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=) 2024-03-27T11:10:18.757 INFO:journalctl@ceph.osd.7.smithi114.stdout:Mar 27 11:10:18 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-7[53256]: 2024-03-27T11:10:18.385+0000 7fd5e5800700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.916460+0000 front 2024-03-27T11:09:36.916564+0000 (oldest deadline 2024-03-27T11:10:01.016630+0000) 2024-03-27T11:10:18.757 INFO:journalctl@ceph.osd.7.smithi114.stdout:Mar 27 11:10:18 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-7[53256]: 2024-03-27T11:10:18.385+0000 7fd5e5800700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:43.918347+0000 front 2024-03-27T11:09:43.918473+0000 (oldest deadline 2024-03-27T11:10:08.618602+0000) 2024-03-27T11:10:19.013 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:18 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:18.788+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.380254+0000 front 2024-03-27T11:09:23.380098+0000 (oldest deadline 2024-03-27T11:09:49.280089+0000) 2024-03-27T11:10:19.395 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:18 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:18.788+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:34.582052+0000 front 2024-03-27T11:09:34.582101+0000 (oldest deadline 2024-03-27T11:10:00.482194+0000) 2024-03-27T11:10:19.395 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:18 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:18.788+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:45.784185+0000 front 2024-03-27T11:09:45.784030+0000 (oldest deadline 2024-03-27T11:10:08.683959+0000) 2024-03-27T11:10:19.395 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:18 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:18.788+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:55.086370+0000 front 2024-03-27T11:09:55.086460+0000 (oldest deadline 2024-03-27T11:10:17.386639+0000) 2024-03-27T11:10:19.396 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 27 11:10:19 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-6[60366]: 2024-03-27T11:10:19.134+0000 7f8ead400700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:27.412885+0000 front 2024-03-27T11:09:27.412671+0000 (oldest deadline 2024-03-27T11:09:49.712654+0000) 2024-03-27T11:10:19.396 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 27 11:10:19 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-6[60366]: 2024-03-27T11:10:19.134+0000 7f8ead400700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:35.416884+0000 front 2024-03-27T11:09:35.416826+0000 (oldest deadline 2024-03-27T11:09:59.516984+0000) 2024-03-27T11:10:19.396 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 27 11:10:19 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-6[60366]: 2024-03-27T11:10:19.134+0000 7f8ead400700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:47.119756+0000 front 2024-03-27T11:09:47.120188+0000 (oldest deadline 2024-03-27T11:10:10.019663+0000) 2024-03-27T11:10:19.396 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 27 11:10:19 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-6[60366]: 2024-03-27T11:10:19.134+0000 7f8ead400700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:55.921855+0000 front 2024-03-27T11:09:55.922085+0000 (oldest deadline 2024-03-27T11:10:18.821583+0000) 2024-03-27T11:10:19.397 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:18 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:18.919+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.654238+0000 front 2024-03-27T11:09:23.654318+0000 (oldest deadline 2024-03-27T11:09:48.954440+0000) 2024-03-27T11:10:19.397 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:18 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:18.919+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:35.957512+0000 front 2024-03-27T11:09:35.957437+0000 (oldest deadline 2024-03-27T11:10:01.857676+0000) 2024-03-27T11:10:19.397 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:18 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:18.919+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:44.160107+0000 front 2024-03-27T11:09:44.160318+0000 (oldest deadline 2024-03-27T11:10:08.260147+0000) 2024-03-27T11:10:19.397 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:18 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:18.919+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.463226+0000 front 2024-03-27T11:09:56.463040+0000 (oldest deadline 2024-03-27T11:10:18.763354+0000) 2024-03-27T11:10:19.398 DEBUG:teuthology.orchestra.run.smithi177:> sudo pkill -f 'journalctl -f -n 0 -u ceph-adca88c4-ec26-11ee-95d0-87774f69a715@osd.5.service' 2024-03-27T11:10:19.423 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:19 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:19.155+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.236297+0000 front 2024-03-27T11:09:28.236728+0000 (oldest deadline 2024-03-27T11:09:51.136525+0000) 2024-03-27T11:10:19.423 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:19 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:19.155+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.340023+0000 front 2024-03-27T11:09:36.340351+0000 (oldest deadline 2024-03-27T11:10:02.239659+0000) 2024-03-27T11:10:19.423 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:19 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:19.155+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:46.143084+0000 front 2024-03-27T11:09:46.143228+0000 (oldest deadline 2024-03-27T11:10:10.843246+0000) 2024-03-27T11:10:19.424 INFO:journalctl@ceph.osd.5.smithi177.stdout:Mar 27 11:10:19 smithi177 podman[71840]: 2024-03-27 11:10:19.139301158 +0000 UTC m=+0.649331045 container remove cc864a02dc04a66af25c1dd06d3519b82eb51e5480b0f22b6cb4d8c21a2325b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-5-deactivate, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.schema-version=1.0, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240326) 2024-03-27T11:10:19.424 INFO:journalctl@ceph.osd.5.smithi177.stdout:Mar 27 11:10:19 smithi177 systemd[1]: ceph-adca88c4-ec26-11ee-95d0-87774f69a715@osd.5.service: Deactivated successfully. 2024-03-27T11:10:19.424 INFO:journalctl@ceph.osd.5.smithi177.stdout:Mar 27 11:10:19 smithi177 systemd[1]: Stopped Ceph osd.5 for adca88c4-ec26-11ee-95d0-87774f69a715. 2024-03-27T11:10:19.424 INFO:journalctl@ceph.osd.5.smithi177.stdout:Mar 27 11:10:19 smithi177 systemd[1]: ceph-adca88c4-ec26-11ee-95d0-87774f69a715@osd.5.service: Consumed 7min 18.826s CPU time. 2024-03-27T11:10:19.424 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:19 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:19.376+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.140407+0000 front 2024-03-27T11:09:28.140164+0000 (oldest deadline 2024-03-27T11:09:54.040233+0000) 2024-03-27T11:10:19.424 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:19 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:19.376+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.844416+0000 front 2024-03-27T11:09:36.843999+0000 (oldest deadline 2024-03-27T11:10:01.544084+0000) 2024-03-27T11:10:19.424 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:19 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:19.376+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:47.348157+0000 front 2024-03-27T11:09:47.348480+0000 (oldest deadline 2024-03-27T11:10:09.647966+0000) 2024-03-27T11:10:19.503 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-27T11:10:19.503 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-03-27T11:10:19.504 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-03-27T11:10:19.504 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-adca88c4-ec26-11ee-95d0-87774f69a715@osd.6 2024-03-27T11:10:19.762 INFO:journalctl@ceph.osd.7.smithi114.stdout:Mar 27 11:10:19 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-7[53256]: 2024-03-27T11:10:19.402+0000 7fd5e5800700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.916460+0000 front 2024-03-27T11:09:36.916564+0000 (oldest deadline 2024-03-27T11:10:01.016630+0000) 2024-03-27T11:10:19.762 INFO:journalctl@ceph.osd.7.smithi114.stdout:Mar 27 11:10:19 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-7[53256]: 2024-03-27T11:10:19.402+0000 7fd5e5800700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:43.918347+0000 front 2024-03-27T11:09:43.918473+0000 (oldest deadline 2024-03-27T11:10:08.618602+0000) 2024-03-27T11:10:19.849 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 27 11:10:19 smithi097 systemd[1]: Stopping Ceph osd.6 for adca88c4-ec26-11ee-95d0-87774f69a715... 2024-03-27T11:10:20.013 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:19 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:19.793+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.380254+0000 front 2024-03-27T11:09:23.380098+0000 (oldest deadline 2024-03-27T11:09:49.280089+0000) 2024-03-27T11:10:20.013 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:19 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:19.793+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:34.582052+0000 front 2024-03-27T11:09:34.582101+0000 (oldest deadline 2024-03-27T11:10:00.482194+0000) 2024-03-27T11:10:20.013 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:19 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:19.793+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:45.784185+0000 front 2024-03-27T11:09:45.784030+0000 (oldest deadline 2024-03-27T11:10:08.683959+0000) 2024-03-27T11:10:20.013 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:19 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:19.793+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:55.086370+0000 front 2024-03-27T11:09:55.086460+0000 (oldest deadline 2024-03-27T11:10:17.386639+0000) 2024-03-27T11:10:20.105 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:19 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:19.880+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.654238+0000 front 2024-03-27T11:09:23.654318+0000 (oldest deadline 2024-03-27T11:09:48.954440+0000) 2024-03-27T11:10:20.105 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:19 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:19.880+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:35.957512+0000 front 2024-03-27T11:09:35.957437+0000 (oldest deadline 2024-03-27T11:10:01.857676+0000) 2024-03-27T11:10:20.105 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:19 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:19.880+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:44.160107+0000 front 2024-03-27T11:09:44.160318+0000 (oldest deadline 2024-03-27T11:10:08.260147+0000) 2024-03-27T11:10:20.105 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:19 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:19.880+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.463226+0000 front 2024-03-27T11:09:56.463040+0000 (oldest deadline 2024-03-27T11:10:18.763354+0000) 2024-03-27T11:10:20.105 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 27 11:10:20 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-6[60366]: 2024-03-27T11:10:20.087+0000 7f8eb1c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-27T11:10:20.105 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 27 11:10:20 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-6[60366]: 2024-03-27T11:10:20.087+0000 7f8eb1c00700 -1 osd.6 92 *** Got signal Terminated *** 2024-03-27T11:10:20.106 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 27 11:10:20 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-6[60366]: 2024-03-27T11:10:20.087+0000 7f8eb1c00700 -1 osd.6 92 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-27T11:10:20.106 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 27 11:10:20 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-6[60366]: 2024-03-27T11:10:20.136+0000 7f8ead400700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:27.412885+0000 front 2024-03-27T11:09:27.412671+0000 (oldest deadline 2024-03-27T11:09:49.712654+0000) 2024-03-27T11:10:20.106 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 27 11:10:20 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-6[60366]: 2024-03-27T11:10:20.136+0000 7f8ead400700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:35.416884+0000 front 2024-03-27T11:09:35.416826+0000 (oldest deadline 2024-03-27T11:09:59.516984+0000) 2024-03-27T11:10:20.106 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 27 11:10:20 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-6[60366]: 2024-03-27T11:10:20.136+0000 7f8ead400700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:47.119756+0000 front 2024-03-27T11:09:47.120188+0000 (oldest deadline 2024-03-27T11:10:10.019663+0000) 2024-03-27T11:10:20.369 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:20 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:20.133+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.236297+0000 front 2024-03-27T11:09:28.236728+0000 (oldest deadline 2024-03-27T11:09:51.136525+0000) 2024-03-27T11:10:20.369 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:20 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:20.133+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.340023+0000 front 2024-03-27T11:09:36.340351+0000 (oldest deadline 2024-03-27T11:10:02.239659+0000) 2024-03-27T11:10:20.369 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:20 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:20.133+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:46.143084+0000 front 2024-03-27T11:09:46.143228+0000 (oldest deadline 2024-03-27T11:10:10.843246+0000) 2024-03-27T11:10:20.387 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 27 11:10:20 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-6[60366]: 2024-03-27T11:10:20.136+0000 7f8ead400700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:55.921855+0000 front 2024-03-27T11:09:55.922085+0000 (oldest deadline 2024-03-27T11:10:18.821583+0000) 2024-03-27T11:10:20.673 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:20 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:20.400+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.140407+0000 front 2024-03-27T11:09:28.140164+0000 (oldest deadline 2024-03-27T11:09:54.040233+0000) 2024-03-27T11:10:20.673 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:20 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:20.400+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.844416+0000 front 2024-03-27T11:09:36.843999+0000 (oldest deadline 2024-03-27T11:10:01.544084+0000) 2024-03-27T11:10:20.673 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:20 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:20.400+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:47.348157+0000 front 2024-03-27T11:09:47.348480+0000 (oldest deadline 2024-03-27T11:10:09.647966+0000) 2024-03-27T11:10:20.763 INFO:journalctl@ceph.osd.7.smithi114.stdout:Mar 27 11:10:20 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-7[53256]: 2024-03-27T11:10:20.400+0000 7fd5e5800700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.916460+0000 front 2024-03-27T11:09:36.916564+0000 (oldest deadline 2024-03-27T11:10:01.016630+0000) 2024-03-27T11:10:20.763 INFO:journalctl@ceph.osd.7.smithi114.stdout:Mar 27 11:10:20 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-7[53256]: 2024-03-27T11:10:20.400+0000 7fd5e5800700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:43.918347+0000 front 2024-03-27T11:09:43.918473+0000 (oldest deadline 2024-03-27T11:10:08.618602+0000) 2024-03-27T11:10:20.763 INFO:journalctl@ceph.osd.7.smithi114.stdout:Mar 27 11:10:20 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-7[53256]: 2024-03-27T11:10:20.400+0000 7fd5e5800700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:55.522718+0000 front 2024-03-27T11:09:55.522838+0000 (oldest deadline 2024-03-27T11:10:20.222876+0000) 2024-03-27T11:10:21.014 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:20 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:20.794+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.380254+0000 front 2024-03-27T11:09:23.380098+0000 (oldest deadline 2024-03-27T11:09:49.280089+0000) 2024-03-27T11:10:21.014 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:20 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:20.794+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:34.582052+0000 front 2024-03-27T11:09:34.582101+0000 (oldest deadline 2024-03-27T11:10:00.482194+0000) 2024-03-27T11:10:21.014 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:20 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:20.794+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:45.784185+0000 front 2024-03-27T11:09:45.784030+0000 (oldest deadline 2024-03-27T11:10:08.683959+0000) 2024-03-27T11:10:21.014 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:20 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:20.794+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:55.086370+0000 front 2024-03-27T11:09:55.086460+0000 (oldest deadline 2024-03-27T11:10:17.386639+0000) 2024-03-27T11:10:21.137 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 27 11:10:21 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-6[60366]: 2024-03-27T11:10:21.095+0000 7f8ead400700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:27.412885+0000 front 2024-03-27T11:09:27.412671+0000 (oldest deadline 2024-03-27T11:09:49.712654+0000) 2024-03-27T11:10:21.138 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 27 11:10:21 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-6[60366]: 2024-03-27T11:10:21.095+0000 7f8ead400700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:35.416884+0000 front 2024-03-27T11:09:35.416826+0000 (oldest deadline 2024-03-27T11:09:59.516984+0000) 2024-03-27T11:10:21.138 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 27 11:10:21 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-6[60366]: 2024-03-27T11:10:21.095+0000 7f8ead400700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:47.119756+0000 front 2024-03-27T11:09:47.120188+0000 (oldest deadline 2024-03-27T11:10:10.019663+0000) 2024-03-27T11:10:21.138 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 27 11:10:21 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-6[60366]: 2024-03-27T11:10:21.095+0000 7f8ead400700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:55.921855+0000 front 2024-03-27T11:09:55.922085+0000 (oldest deadline 2024-03-27T11:10:18.821583+0000) 2024-03-27T11:10:21.138 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:20 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:20.916+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.654238+0000 front 2024-03-27T11:09:23.654318+0000 (oldest deadline 2024-03-27T11:09:48.954440+0000) 2024-03-27T11:10:21.138 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:20 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:20.916+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:35.957512+0000 front 2024-03-27T11:09:35.957437+0000 (oldest deadline 2024-03-27T11:10:01.857676+0000) 2024-03-27T11:10:21.138 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:20 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:20.916+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:44.160107+0000 front 2024-03-27T11:09:44.160318+0000 (oldest deadline 2024-03-27T11:10:08.260147+0000) 2024-03-27T11:10:21.138 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:20 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:20.916+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.463226+0000 front 2024-03-27T11:09:56.463040+0000 (oldest deadline 2024-03-27T11:10:18.763354+0000) 2024-03-27T11:10:21.174 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:21 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:21.091+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.236297+0000 front 2024-03-27T11:09:28.236728+0000 (oldest deadline 2024-03-27T11:09:51.136525+0000) 2024-03-27T11:10:21.174 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:21 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:21.091+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.340023+0000 front 2024-03-27T11:09:36.340351+0000 (oldest deadline 2024-03-27T11:10:02.239659+0000) 2024-03-27T11:10:21.174 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:21 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:21.091+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:46.143084+0000 front 2024-03-27T11:09:46.143228+0000 (oldest deadline 2024-03-27T11:10:10.843246+0000) 2024-03-27T11:10:21.174 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:21 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:21.091+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.046124+0000 front 2024-03-27T11:09:56.046017+0000 (oldest deadline 2024-03-27T11:10:20.746204+0000) 2024-03-27T11:10:21.673 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:21 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:21.408+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.140407+0000 front 2024-03-27T11:09:28.140164+0000 (oldest deadline 2024-03-27T11:09:54.040233+0000) 2024-03-27T11:10:21.673 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:21 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:21.408+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.844416+0000 front 2024-03-27T11:09:36.843999+0000 (oldest deadline 2024-03-27T11:10:01.544084+0000) 2024-03-27T11:10:21.673 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:21 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:21.408+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:47.348157+0000 front 2024-03-27T11:09:47.348480+0000 (oldest deadline 2024-03-27T11:10:09.647966+0000) 2024-03-27T11:10:21.673 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:21 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:21.408+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.051047+0000 front 2024-03-27T11:09:56.050946+0000 (oldest deadline 2024-03-27T11:10:20.750821+0000) 2024-03-27T11:10:21.763 INFO:journalctl@ceph.osd.7.smithi114.stdout:Mar 27 11:10:21 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-7[53256]: 2024-03-27T11:10:21.450+0000 7fd5e5800700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.916460+0000 front 2024-03-27T11:09:36.916564+0000 (oldest deadline 2024-03-27T11:10:01.016630+0000) 2024-03-27T11:10:21.763 INFO:journalctl@ceph.osd.7.smithi114.stdout:Mar 27 11:10:21 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-7[53256]: 2024-03-27T11:10:21.450+0000 7fd5e5800700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:43.918347+0000 front 2024-03-27T11:09:43.918473+0000 (oldest deadline 2024-03-27T11:10:08.618602+0000) 2024-03-27T11:10:21.763 INFO:journalctl@ceph.osd.7.smithi114.stdout:Mar 27 11:10:21 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-7[53256]: 2024-03-27T11:10:21.450+0000 7fd5e5800700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:55.522718+0000 front 2024-03-27T11:09:55.522838+0000 (oldest deadline 2024-03-27T11:10:20.222876+0000) 2024-03-27T11:10:22.137 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 27 11:10:22 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-6[60366]: 2024-03-27T11:10:22.094+0000 7f8ead400700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:27.412885+0000 front 2024-03-27T11:09:27.412671+0000 (oldest deadline 2024-03-27T11:09:49.712654+0000) 2024-03-27T11:10:22.138 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 27 11:10:22 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-6[60366]: 2024-03-27T11:10:22.094+0000 7f8ead400700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:35.416884+0000 front 2024-03-27T11:09:35.416826+0000 (oldest deadline 2024-03-27T11:09:59.516984+0000) 2024-03-27T11:10:22.138 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 27 11:10:22 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-6[60366]: 2024-03-27T11:10:22.094+0000 7f8ead400700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:47.119756+0000 front 2024-03-27T11:09:47.120188+0000 (oldest deadline 2024-03-27T11:10:10.019663+0000) 2024-03-27T11:10:22.138 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 27 11:10:22 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-6[60366]: 2024-03-27T11:10:22.094+0000 7f8ead400700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:55.921855+0000 front 2024-03-27T11:09:55.922085+0000 (oldest deadline 2024-03-27T11:10:18.821583+0000) 2024-03-27T11:10:22.138 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:21 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:21.868+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.654238+0000 front 2024-03-27T11:09:23.654318+0000 (oldest deadline 2024-03-27T11:09:48.954440+0000) 2024-03-27T11:10:22.138 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:21 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:21.868+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:35.957512+0000 front 2024-03-27T11:09:35.957437+0000 (oldest deadline 2024-03-27T11:10:01.857676+0000) 2024-03-27T11:10:22.138 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:21 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:21.868+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:44.160107+0000 front 2024-03-27T11:09:44.160318+0000 (oldest deadline 2024-03-27T11:10:08.260147+0000) 2024-03-27T11:10:22.138 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:21 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:21.868+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.463226+0000 front 2024-03-27T11:09:56.463040+0000 (oldest deadline 2024-03-27T11:10:18.763354+0000) 2024-03-27T11:10:22.263 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:21 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:21.840+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.380254+0000 front 2024-03-27T11:09:23.380098+0000 (oldest deadline 2024-03-27T11:09:49.280089+0000) 2024-03-27T11:10:22.263 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:21 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:21.840+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:34.582052+0000 front 2024-03-27T11:09:34.582101+0000 (oldest deadline 2024-03-27T11:10:00.482194+0000) 2024-03-27T11:10:22.263 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:21 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:21.840+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:45.784185+0000 front 2024-03-27T11:09:45.784030+0000 (oldest deadline 2024-03-27T11:10:08.683959+0000) 2024-03-27T11:10:22.263 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:21 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:21.840+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:55.086370+0000 front 2024-03-27T11:09:55.086460+0000 (oldest deadline 2024-03-27T11:10:17.386639+0000) 2024-03-27T11:10:22.401 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:22 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:22.142+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.236297+0000 front 2024-03-27T11:09:28.236728+0000 (oldest deadline 2024-03-27T11:09:51.136525+0000) 2024-03-27T11:10:22.401 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:22 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:22.142+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.340023+0000 front 2024-03-27T11:09:36.340351+0000 (oldest deadline 2024-03-27T11:10:02.239659+0000) 2024-03-27T11:10:22.401 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:22 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:22.142+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:46.143084+0000 front 2024-03-27T11:09:46.143228+0000 (oldest deadline 2024-03-27T11:10:10.843246+0000) 2024-03-27T11:10:22.401 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:22 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:22.142+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.046124+0000 front 2024-03-27T11:09:56.046017+0000 (oldest deadline 2024-03-27T11:10:20.746204+0000) 2024-03-27T11:10:22.673 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:22 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:22.432+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.140407+0000 front 2024-03-27T11:09:28.140164+0000 (oldest deadline 2024-03-27T11:09:54.040233+0000) 2024-03-27T11:10:22.673 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:22 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:22.432+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.844416+0000 front 2024-03-27T11:09:36.843999+0000 (oldest deadline 2024-03-27T11:10:01.544084+0000) 2024-03-27T11:10:22.673 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:22 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:22.432+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:47.348157+0000 front 2024-03-27T11:09:47.348480+0000 (oldest deadline 2024-03-27T11:10:09.647966+0000) 2024-03-27T11:10:22.673 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:22 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:22.432+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.051047+0000 front 2024-03-27T11:09:56.050946+0000 (oldest deadline 2024-03-27T11:10:20.750821+0000) 2024-03-27T11:10:22.763 INFO:journalctl@ceph.osd.7.smithi114.stdout:Mar 27 11:10:22 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-7[53256]: 2024-03-27T11:10:22.412+0000 7fd5e5800700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.916460+0000 front 2024-03-27T11:09:36.916564+0000 (oldest deadline 2024-03-27T11:10:01.016630+0000) 2024-03-27T11:10:22.763 INFO:journalctl@ceph.osd.7.smithi114.stdout:Mar 27 11:10:22 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-7[53256]: 2024-03-27T11:10:22.412+0000 7fd5e5800700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:43.918347+0000 front 2024-03-27T11:09:43.918473+0000 (oldest deadline 2024-03-27T11:10:08.618602+0000) 2024-03-27T11:10:22.763 INFO:journalctl@ceph.osd.7.smithi114.stdout:Mar 27 11:10:22 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-7[53256]: 2024-03-27T11:10:22.412+0000 7fd5e5800700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:55.522718+0000 front 2024-03-27T11:09:55.522838+0000 (oldest deadline 2024-03-27T11:10:20.222876+0000) 2024-03-27T11:10:23.137 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 27 11:10:23 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-6[60366]: 2024-03-27T11:10:23.100+0000 7f8ead400700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:27.412885+0000 front 2024-03-27T11:09:27.412671+0000 (oldest deadline 2024-03-27T11:09:49.712654+0000) 2024-03-27T11:10:23.138 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 27 11:10:23 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-6[60366]: 2024-03-27T11:10:23.100+0000 7f8ead400700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:35.416884+0000 front 2024-03-27T11:09:35.416826+0000 (oldest deadline 2024-03-27T11:09:59.516984+0000) 2024-03-27T11:10:23.138 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 27 11:10:23 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-6[60366]: 2024-03-27T11:10:23.100+0000 7f8ead400700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:47.119756+0000 front 2024-03-27T11:09:47.120188+0000 (oldest deadline 2024-03-27T11:10:10.019663+0000) 2024-03-27T11:10:23.138 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 27 11:10:23 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-6[60366]: 2024-03-27T11:10:23.100+0000 7f8ead400700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:55.921855+0000 front 2024-03-27T11:09:55.922085+0000 (oldest deadline 2024-03-27T11:10:18.821583+0000) 2024-03-27T11:10:23.138 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:22 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:22.864+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.654238+0000 front 2024-03-27T11:09:23.654318+0000 (oldest deadline 2024-03-27T11:09:48.954440+0000) 2024-03-27T11:10:23.138 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:22 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:22.864+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:35.957512+0000 front 2024-03-27T11:09:35.957437+0000 (oldest deadline 2024-03-27T11:10:01.857676+0000) 2024-03-27T11:10:23.138 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:22 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:22.864+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:44.160107+0000 front 2024-03-27T11:09:44.160318+0000 (oldest deadline 2024-03-27T11:10:08.260147+0000) 2024-03-27T11:10:23.138 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:22 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:22.864+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.463226+0000 front 2024-03-27T11:09:56.463040+0000 (oldest deadline 2024-03-27T11:10:18.763354+0000) 2024-03-27T11:10:23.263 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:22 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:22.823+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.380254+0000 front 2024-03-27T11:09:23.380098+0000 (oldest deadline 2024-03-27T11:09:49.280089+0000) 2024-03-27T11:10:23.264 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:22 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:22.823+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:34.582052+0000 front 2024-03-27T11:09:34.582101+0000 (oldest deadline 2024-03-27T11:10:00.482194+0000) 2024-03-27T11:10:23.264 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:22 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:22.823+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:45.784185+0000 front 2024-03-27T11:09:45.784030+0000 (oldest deadline 2024-03-27T11:10:08.683959+0000) 2024-03-27T11:10:23.264 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:22 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:22.823+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:55.086370+0000 front 2024-03-27T11:09:55.086460+0000 (oldest deadline 2024-03-27T11:10:17.386639+0000) 2024-03-27T11:10:23.419 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:23 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:23.126+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.236297+0000 front 2024-03-27T11:09:28.236728+0000 (oldest deadline 2024-03-27T11:09:51.136525+0000) 2024-03-27T11:10:23.419 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:23 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:23.126+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.340023+0000 front 2024-03-27T11:09:36.340351+0000 (oldest deadline 2024-03-27T11:10:02.239659+0000) 2024-03-27T11:10:23.419 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:23 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:23.127+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:46.143084+0000 front 2024-03-27T11:09:46.143228+0000 (oldest deadline 2024-03-27T11:10:10.843246+0000) 2024-03-27T11:10:23.420 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:23 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:23.127+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.046124+0000 front 2024-03-27T11:09:56.046017+0000 (oldest deadline 2024-03-27T11:10:20.746204+0000) 2024-03-27T11:10:23.673 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:23 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:23.449+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.140407+0000 front 2024-03-27T11:09:28.140164+0000 (oldest deadline 2024-03-27T11:09:54.040233+0000) 2024-03-27T11:10:23.673 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:23 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:23.449+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.844416+0000 front 2024-03-27T11:09:36.843999+0000 (oldest deadline 2024-03-27T11:10:01.544084+0000) 2024-03-27T11:10:23.673 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:23 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:23.449+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:47.348157+0000 front 2024-03-27T11:09:47.348480+0000 (oldest deadline 2024-03-27T11:10:09.647966+0000) 2024-03-27T11:10:23.673 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:23 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:23.449+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.051047+0000 front 2024-03-27T11:09:56.050946+0000 (oldest deadline 2024-03-27T11:10:20.750821+0000) 2024-03-27T11:10:23.763 INFO:journalctl@ceph.osd.7.smithi114.stdout:Mar 27 11:10:23 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-7[53256]: 2024-03-27T11:10:23.410+0000 7fd5e5800700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.916460+0000 front 2024-03-27T11:09:36.916564+0000 (oldest deadline 2024-03-27T11:10:01.016630+0000) 2024-03-27T11:10:23.763 INFO:journalctl@ceph.osd.7.smithi114.stdout:Mar 27 11:10:23 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-7[53256]: 2024-03-27T11:10:23.410+0000 7fd5e5800700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:43.918347+0000 front 2024-03-27T11:09:43.918473+0000 (oldest deadline 2024-03-27T11:10:08.618602+0000) 2024-03-27T11:10:23.763 INFO:journalctl@ceph.osd.7.smithi114.stdout:Mar 27 11:10:23 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-7[53256]: 2024-03-27T11:10:23.410+0000 7fd5e5800700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:55.522718+0000 front 2024-03-27T11:09:55.522838+0000 (oldest deadline 2024-03-27T11:10:20.222876+0000) 2024-03-27T11:10:24.137 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 27 11:10:24 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-6[60366]: 2024-03-27T11:10:24.119+0000 7f8ead400700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:27.412885+0000 front 2024-03-27T11:09:27.412671+0000 (oldest deadline 2024-03-27T11:09:49.712654+0000) 2024-03-27T11:10:24.138 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 27 11:10:24 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-6[60366]: 2024-03-27T11:10:24.119+0000 7f8ead400700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:35.416884+0000 front 2024-03-27T11:09:35.416826+0000 (oldest deadline 2024-03-27T11:09:59.516984+0000) 2024-03-27T11:10:24.138 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 27 11:10:24 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-6[60366]: 2024-03-27T11:10:24.119+0000 7f8ead400700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:47.119756+0000 front 2024-03-27T11:09:47.120188+0000 (oldest deadline 2024-03-27T11:10:10.019663+0000) 2024-03-27T11:10:24.138 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 27 11:10:24 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-6[60366]: 2024-03-27T11:10:24.119+0000 7f8ead400700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:55.921855+0000 front 2024-03-27T11:09:55.922085+0000 (oldest deadline 2024-03-27T11:10:18.821583+0000) 2024-03-27T11:10:24.138 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:23 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:23.912+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.654238+0000 front 2024-03-27T11:09:23.654318+0000 (oldest deadline 2024-03-27T11:09:48.954440+0000) 2024-03-27T11:10:24.138 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:23 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:23.912+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:35.957512+0000 front 2024-03-27T11:09:35.957437+0000 (oldest deadline 2024-03-27T11:10:01.857676+0000) 2024-03-27T11:10:24.138 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:23 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:23.912+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:44.160107+0000 front 2024-03-27T11:09:44.160318+0000 (oldest deadline 2024-03-27T11:10:08.260147+0000) 2024-03-27T11:10:24.138 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:23 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:23.912+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.463226+0000 front 2024-03-27T11:09:56.463040+0000 (oldest deadline 2024-03-27T11:10:18.763354+0000) 2024-03-27T11:10:24.263 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:23 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:23.812+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.380254+0000 front 2024-03-27T11:09:23.380098+0000 (oldest deadline 2024-03-27T11:09:49.280089+0000) 2024-03-27T11:10:24.263 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:23 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:23.812+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:34.582052+0000 front 2024-03-27T11:09:34.582101+0000 (oldest deadline 2024-03-27T11:10:00.482194+0000) 2024-03-27T11:10:24.263 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:23 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:23.812+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:45.784185+0000 front 2024-03-27T11:09:45.784030+0000 (oldest deadline 2024-03-27T11:10:08.683959+0000) 2024-03-27T11:10:24.264 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:23 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:23.812+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:55.086370+0000 front 2024-03-27T11:09:55.086460+0000 (oldest deadline 2024-03-27T11:10:17.386639+0000) 2024-03-27T11:10:24.423 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:24 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:24.128+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.236297+0000 front 2024-03-27T11:09:28.236728+0000 (oldest deadline 2024-03-27T11:09:51.136525+0000) 2024-03-27T11:10:24.423 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:24 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:24.128+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.340023+0000 front 2024-03-27T11:09:36.340351+0000 (oldest deadline 2024-03-27T11:10:02.239659+0000) 2024-03-27T11:10:24.423 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:24 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:24.128+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:46.143084+0000 front 2024-03-27T11:09:46.143228+0000 (oldest deadline 2024-03-27T11:10:10.843246+0000) 2024-03-27T11:10:24.423 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:24 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:24.128+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.046124+0000 front 2024-03-27T11:09:56.046017+0000 (oldest deadline 2024-03-27T11:10:20.746204+0000) 2024-03-27T11:10:24.763 INFO:journalctl@ceph.osd.7.smithi114.stdout:Mar 27 11:10:24 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-7[53256]: 2024-03-27T11:10:24.393+0000 7fd5e5800700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.916460+0000 front 2024-03-27T11:09:36.916564+0000 (oldest deadline 2024-03-27T11:10:01.016630+0000) 2024-03-27T11:10:24.763 INFO:journalctl@ceph.osd.7.smithi114.stdout:Mar 27 11:10:24 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-7[53256]: 2024-03-27T11:10:24.393+0000 7fd5e5800700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:43.918347+0000 front 2024-03-27T11:09:43.918473+0000 (oldest deadline 2024-03-27T11:10:08.618602+0000) 2024-03-27T11:10:24.763 INFO:journalctl@ceph.osd.7.smithi114.stdout:Mar 27 11:10:24 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-7[53256]: 2024-03-27T11:10:24.393+0000 7fd5e5800700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:55.522718+0000 front 2024-03-27T11:09:55.522838+0000 (oldest deadline 2024-03-27T11:10:20.222876+0000) 2024-03-27T11:10:24.923 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:24 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:24.493+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.140407+0000 front 2024-03-27T11:09:28.140164+0000 (oldest deadline 2024-03-27T11:09:54.040233+0000) 2024-03-27T11:10:24.923 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:24 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:24.493+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.844416+0000 front 2024-03-27T11:09:36.843999+0000 (oldest deadline 2024-03-27T11:10:01.544084+0000) 2024-03-27T11:10:24.923 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:24 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:24.493+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:47.348157+0000 front 2024-03-27T11:09:47.348480+0000 (oldest deadline 2024-03-27T11:10:09.647966+0000) 2024-03-27T11:10:24.923 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:24 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:24.493+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.051047+0000 front 2024-03-27T11:09:56.050946+0000 (oldest deadline 2024-03-27T11:10:20.750821+0000) 2024-03-27T11:10:25.138 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 27 11:10:25 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-6[60366]: 2024-03-27T11:10:25.072+0000 7f8ead400700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:27.412885+0000 front 2024-03-27T11:09:27.412671+0000 (oldest deadline 2024-03-27T11:09:49.712654+0000) 2024-03-27T11:10:25.138 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 27 11:10:25 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-6[60366]: 2024-03-27T11:10:25.072+0000 7f8ead400700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:35.416884+0000 front 2024-03-27T11:09:35.416826+0000 (oldest deadline 2024-03-27T11:09:59.516984+0000) 2024-03-27T11:10:25.138 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 27 11:10:25 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-6[60366]: 2024-03-27T11:10:25.072+0000 7f8ead400700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:47.119756+0000 front 2024-03-27T11:09:47.120188+0000 (oldest deadline 2024-03-27T11:10:10.019663+0000) 2024-03-27T11:10:25.138 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 27 11:10:25 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-6[60366]: 2024-03-27T11:10:25.072+0000 7f8ead400700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:55.921855+0000 front 2024-03-27T11:09:55.922085+0000 (oldest deadline 2024-03-27T11:10:18.821583+0000) 2024-03-27T11:10:25.138 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:24 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:24.864+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.654238+0000 front 2024-03-27T11:09:23.654318+0000 (oldest deadline 2024-03-27T11:09:48.954440+0000) 2024-03-27T11:10:25.138 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:24 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:24.864+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:35.957512+0000 front 2024-03-27T11:09:35.957437+0000 (oldest deadline 2024-03-27T11:10:01.857676+0000) 2024-03-27T11:10:25.138 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:24 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:24.864+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:44.160107+0000 front 2024-03-27T11:09:44.160318+0000 (oldest deadline 2024-03-27T11:10:08.260147+0000) 2024-03-27T11:10:25.138 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:24 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:24.864+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.463226+0000 front 2024-03-27T11:09:56.463040+0000 (oldest deadline 2024-03-27T11:10:18.763354+0000) 2024-03-27T11:10:25.263 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:24 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:24.811+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.380254+0000 front 2024-03-27T11:09:23.380098+0000 (oldest deadline 2024-03-27T11:09:49.280089+0000) 2024-03-27T11:10:25.263 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:24 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:24.811+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:34.582052+0000 front 2024-03-27T11:09:34.582101+0000 (oldest deadline 2024-03-27T11:10:00.482194+0000) 2024-03-27T11:10:25.263 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:24 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:24.811+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:45.784185+0000 front 2024-03-27T11:09:45.784030+0000 (oldest deadline 2024-03-27T11:10:08.683959+0000) 2024-03-27T11:10:25.264 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:24 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:24.811+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:55.086370+0000 front 2024-03-27T11:09:55.086460+0000 (oldest deadline 2024-03-27T11:10:17.386639+0000) 2024-03-27T11:10:25.423 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:25 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:25.080+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.236297+0000 front 2024-03-27T11:09:28.236728+0000 (oldest deadline 2024-03-27T11:09:51.136525+0000) 2024-03-27T11:10:25.423 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:25 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:25.080+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.340023+0000 front 2024-03-27T11:09:36.340351+0000 (oldest deadline 2024-03-27T11:10:02.239659+0000) 2024-03-27T11:10:25.423 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:25 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:25.080+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:46.143084+0000 front 2024-03-27T11:09:46.143228+0000 (oldest deadline 2024-03-27T11:10:10.843246+0000) 2024-03-27T11:10:25.423 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:25 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:25.080+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.046124+0000 front 2024-03-27T11:09:56.046017+0000 (oldest deadline 2024-03-27T11:10:20.746204+0000) 2024-03-27T11:10:25.759 INFO:journalctl@ceph.osd.7.smithi114.stdout:Mar 27 11:10:25 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-7[53256]: 2024-03-27T11:10:25.349+0000 7fd5e5800700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.916460+0000 front 2024-03-27T11:09:36.916564+0000 (oldest deadline 2024-03-27T11:10:01.016630+0000) 2024-03-27T11:10:25.759 INFO:journalctl@ceph.osd.7.smithi114.stdout:Mar 27 11:10:25 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-7[53256]: 2024-03-27T11:10:25.349+0000 7fd5e5800700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:43.918347+0000 front 2024-03-27T11:09:43.918473+0000 (oldest deadline 2024-03-27T11:10:08.618602+0000) 2024-03-27T11:10:25.759 INFO:journalctl@ceph.osd.7.smithi114.stdout:Mar 27 11:10:25 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-7[53256]: 2024-03-27T11:10:25.349+0000 7fd5e5800700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:55.522718+0000 front 2024-03-27T11:09:55.522838+0000 (oldest deadline 2024-03-27T11:10:20.222876+0000) 2024-03-27T11:10:25.923 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:25 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:25.477+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.140407+0000 front 2024-03-27T11:09:28.140164+0000 (oldest deadline 2024-03-27T11:09:54.040233+0000) 2024-03-27T11:10:25.923 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:25 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:25.477+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.844416+0000 front 2024-03-27T11:09:36.843999+0000 (oldest deadline 2024-03-27T11:10:01.544084+0000) 2024-03-27T11:10:25.923 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:25 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:25.477+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:47.348157+0000 front 2024-03-27T11:09:47.348480+0000 (oldest deadline 2024-03-27T11:10:09.647966+0000) 2024-03-27T11:10:25.923 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:25 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:25.477+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.051047+0000 front 2024-03-27T11:09:56.050946+0000 (oldest deadline 2024-03-27T11:10:20.750821+0000) 2024-03-27T11:10:26.013 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:25 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:25.789+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.380254+0000 front 2024-03-27T11:09:23.380098+0000 (oldest deadline 2024-03-27T11:09:49.280089+0000) 2024-03-27T11:10:26.013 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:25 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:25.789+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:34.582052+0000 front 2024-03-27T11:09:34.582101+0000 (oldest deadline 2024-03-27T11:10:00.482194+0000) 2024-03-27T11:10:26.013 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:25 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:25.789+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:45.784185+0000 front 2024-03-27T11:09:45.784030+0000 (oldest deadline 2024-03-27T11:10:08.683959+0000) 2024-03-27T11:10:26.013 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:25 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:25.789+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:55.086370+0000 front 2024-03-27T11:09:55.086460+0000 (oldest deadline 2024-03-27T11:10:17.386639+0000) 2024-03-27T11:10:26.106 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 27 11:10:25 smithi097 podman[97406]: 2024-03-27 11:10:25.789040251 +0000 UTC m=+5.854642130 container died 864b71757d0d35f73bc5d08edce014dfd17c9ae3f2150d3f4cfcce8c80615898 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-6, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240326.131558-155268c, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-27T11:10:26.106 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:25 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:25.832+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.654238+0000 front 2024-03-27T11:09:23.654318+0000 (oldest deadline 2024-03-27T11:09:48.954440+0000) 2024-03-27T11:10:26.106 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:25 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:25.832+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:35.957512+0000 front 2024-03-27T11:09:35.957437+0000 (oldest deadline 2024-03-27T11:10:01.857676+0000) 2024-03-27T11:10:26.106 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:25 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:25.832+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:44.160107+0000 front 2024-03-27T11:09:44.160318+0000 (oldest deadline 2024-03-27T11:10:08.260147+0000) 2024-03-27T11:10:26.106 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:25 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:25.832+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.463226+0000 front 2024-03-27T11:09:56.463040+0000 (oldest deadline 2024-03-27T11:10:18.763354+0000) 2024-03-27T11:10:26.387 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 27 11:10:26 smithi097 podman[97406]: 2024-03-27 11:10:26.273500423 +0000 UTC m=+6.339102331 container cleanup 864b71757d0d35f73bc5d08edce014dfd17c9ae3f2150d3f4cfcce8c80615898 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-6, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=) 2024-03-27T11:10:26.387 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 27 11:10:26 smithi097 bash[97406]: ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-6 2024-03-27T11:10:26.412 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:26 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:26.040+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.236297+0000 front 2024-03-27T11:09:28.236728+0000 (oldest deadline 2024-03-27T11:09:51.136525+0000) 2024-03-27T11:10:26.412 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:26 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:26.040+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.340023+0000 front 2024-03-27T11:09:36.340351+0000 (oldest deadline 2024-03-27T11:10:02.239659+0000) 2024-03-27T11:10:26.412 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:26 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:26.040+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:46.143084+0000 front 2024-03-27T11:09:46.143228+0000 (oldest deadline 2024-03-27T11:10:10.843246+0000) 2024-03-27T11:10:26.412 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:26 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:26.040+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.046124+0000 front 2024-03-27T11:09:56.046017+0000 (oldest deadline 2024-03-27T11:10:20.746204+0000) 2024-03-27T11:10:26.673 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:26 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:26.442+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.140407+0000 front 2024-03-27T11:09:28.140164+0000 (oldest deadline 2024-03-27T11:09:54.040233+0000) 2024-03-27T11:10:26.673 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:26 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:26.442+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.844416+0000 front 2024-03-27T11:09:36.843999+0000 (oldest deadline 2024-03-27T11:10:01.544084+0000) 2024-03-27T11:10:26.673 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:26 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:26.442+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:47.348157+0000 front 2024-03-27T11:09:47.348480+0000 (oldest deadline 2024-03-27T11:10:09.647966+0000) 2024-03-27T11:10:26.673 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:26 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:26.442+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.051047+0000 front 2024-03-27T11:09:56.050946+0000 (oldest deadline 2024-03-27T11:10:20.750821+0000) 2024-03-27T11:10:26.727 INFO:journalctl@ceph.osd.7.smithi114.stdout:Mar 27 11:10:26 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-7[53256]: 2024-03-27T11:10:26.321+0000 7fd5e5800700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.916460+0000 front 2024-03-27T11:09:36.916564+0000 (oldest deadline 2024-03-27T11:10:01.016630+0000) 2024-03-27T11:10:26.727 INFO:journalctl@ceph.osd.7.smithi114.stdout:Mar 27 11:10:26 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-7[53256]: 2024-03-27T11:10:26.321+0000 7fd5e5800700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:43.918347+0000 front 2024-03-27T11:09:43.918473+0000 (oldest deadline 2024-03-27T11:10:08.618602+0000) 2024-03-27T11:10:26.727 INFO:journalctl@ceph.osd.7.smithi114.stdout:Mar 27 11:10:26 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-7[53256]: 2024-03-27T11:10:26.321+0000 7fd5e5800700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:55.522718+0000 front 2024-03-27T11:09:55.522838+0000 (oldest deadline 2024-03-27T11:10:20.222876+0000) 2024-03-27T11:10:26.842 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 27 11:10:26 smithi097 podman[97436]: 2024-03-27 11:10:26.541303078 +0000 UTC m=+0.738653562 container remove 864b71757d0d35f73bc5d08edce014dfd17c9ae3f2150d3f4cfcce8c80615898 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-6, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240326) 2024-03-27T11:10:27.013 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:26 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:26.757+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.380254+0000 front 2024-03-27T11:09:23.380098+0000 (oldest deadline 2024-03-27T11:09:49.280089+0000) 2024-03-27T11:10:27.014 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:26 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:26.757+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:34.582052+0000 front 2024-03-27T11:09:34.582101+0000 (oldest deadline 2024-03-27T11:10:00.482194+0000) 2024-03-27T11:10:27.014 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:26 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:26.757+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:45.784185+0000 front 2024-03-27T11:09:45.784030+0000 (oldest deadline 2024-03-27T11:10:08.683959+0000) 2024-03-27T11:10:27.014 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:26 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:26.757+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:55.086370+0000 front 2024-03-27T11:09:55.086460+0000 (oldest deadline 2024-03-27T11:10:17.386639+0000) 2024-03-27T11:10:27.137 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:26 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:26.873+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.654238+0000 front 2024-03-27T11:09:23.654318+0000 (oldest deadline 2024-03-27T11:09:48.954440+0000) 2024-03-27T11:10:27.137 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:26 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:26.873+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:35.957512+0000 front 2024-03-27T11:09:35.957437+0000 (oldest deadline 2024-03-27T11:10:01.857676+0000) 2024-03-27T11:10:27.137 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:26 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:26.873+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:44.160107+0000 front 2024-03-27T11:09:44.160318+0000 (oldest deadline 2024-03-27T11:10:08.260147+0000) 2024-03-27T11:10:27.137 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:26 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:26.873+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.463226+0000 front 2024-03-27T11:09:56.463040+0000 (oldest deadline 2024-03-27T11:10:18.763354+0000) 2024-03-27T11:10:27.412 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:27 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:27.066+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.236297+0000 front 2024-03-27T11:09:28.236728+0000 (oldest deadline 2024-03-27T11:09:51.136525+0000) 2024-03-27T11:10:27.412 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:27 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:27.066+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.340023+0000 front 2024-03-27T11:09:36.340351+0000 (oldest deadline 2024-03-27T11:10:02.239659+0000) 2024-03-27T11:10:27.412 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:27 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:27.066+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:46.143084+0000 front 2024-03-27T11:09:46.143228+0000 (oldest deadline 2024-03-27T11:10:10.843246+0000) 2024-03-27T11:10:27.413 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:27 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:27.066+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.046124+0000 front 2024-03-27T11:09:56.046017+0000 (oldest deadline 2024-03-27T11:10:20.746204+0000) 2024-03-27T11:10:27.589 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 27 11:10:27 smithi097 podman[97526]: 2024-03-27 11:10:27.204297075 +0000 UTC m=+0.036471380 image pull b379bbd8da6b5d0fa9a3523941efac213b6815e1cbf05c6507954f0a3fe7a801 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772 2024-03-27T11:10:27.589 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 27 11:10:27 smithi097 podman[97526]: 2024-03-27 11:10:27.325480302 +0000 UTC m=+0.157654628 container create 75f1472308be336d78183b1ef39fd1cab6e2fd400ea7f21ca1643053a985fa59 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-6-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-03-27T11:10:27.589 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 27 11:10:27 smithi097 podman[97526]: 2024-03-27 11:10:27.62024243 +0000 UTC m=+0.452416766 container init 75f1472308be336d78183b1ef39fd1cab6e2fd400ea7f21ca1643053a985fa59 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-6-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-03-27T11:10:27.673 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:27 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:27.442+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.140407+0000 front 2024-03-27T11:09:28.140164+0000 (oldest deadline 2024-03-27T11:09:54.040233+0000) 2024-03-27T11:10:27.673 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:27 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:27.442+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.844416+0000 front 2024-03-27T11:09:36.843999+0000 (oldest deadline 2024-03-27T11:10:01.544084+0000) 2024-03-27T11:10:27.673 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:27 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:27.442+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:47.348157+0000 front 2024-03-27T11:09:47.348480+0000 (oldest deadline 2024-03-27T11:10:09.647966+0000) 2024-03-27T11:10:27.673 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:27 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:27.442+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.051047+0000 front 2024-03-27T11:09:56.050946+0000 (oldest deadline 2024-03-27T11:10:20.750821+0000) 2024-03-27T11:10:27.722 INFO:journalctl@ceph.osd.7.smithi114.stdout:Mar 27 11:10:27 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-7[53256]: 2024-03-27T11:10:27.352+0000 7fd5e5800700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.916460+0000 front 2024-03-27T11:09:36.916564+0000 (oldest deadline 2024-03-27T11:10:01.016630+0000) 2024-03-27T11:10:27.723 INFO:journalctl@ceph.osd.7.smithi114.stdout:Mar 27 11:10:27 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-7[53256]: 2024-03-27T11:10:27.352+0000 7fd5e5800700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:43.918347+0000 front 2024-03-27T11:09:43.918473+0000 (oldest deadline 2024-03-27T11:10:08.618602+0000) 2024-03-27T11:10:27.723 INFO:journalctl@ceph.osd.7.smithi114.stdout:Mar 27 11:10:27 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-7[53256]: 2024-03-27T11:10:27.352+0000 7fd5e5800700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:55.522718+0000 front 2024-03-27T11:09:55.522838+0000 (oldest deadline 2024-03-27T11:10:20.222876+0000) 2024-03-27T11:10:27.723 INFO:journalctl@ceph.osd.7.smithi114.stdout:Mar 27 11:10:27 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-7[53256]: 2024-03-27T11:10:27.352+0000 7fd5e5800700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:04.924837+0000 front 2024-03-27T11:10:04.924883+0000 (oldest deadline 2024-03-27T11:10:27.224800+0000) 2024-03-27T11:10:27.887 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 27 11:10:27 smithi097 podman[97526]: 2024-03-27 11:10:27.630689907 +0000 UTC m=+0.462864239 container start 75f1472308be336d78183b1ef39fd1cab6e2fd400ea7f21ca1643053a985fa59 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-6-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_BRANCH=HEAD, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-27T11:10:27.887 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 27 11:10:27 smithi097 podman[97526]: 2024-03-27 11:10:27.719562795 +0000 UTC m=+0.551737175 container attach 75f1472308be336d78183b1ef39fd1cab6e2fd400ea7f21ca1643053a985fa59 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-6-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-03-27T11:10:27.888 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:27 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:27.845+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.654238+0000 front 2024-03-27T11:09:23.654318+0000 (oldest deadline 2024-03-27T11:09:48.954440+0000) 2024-03-27T11:10:27.888 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:27 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:27.845+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:35.957512+0000 front 2024-03-27T11:09:35.957437+0000 (oldest deadline 2024-03-27T11:10:01.857676+0000) 2024-03-27T11:10:27.888 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:27 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:27.845+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:44.160107+0000 front 2024-03-27T11:09:44.160318+0000 (oldest deadline 2024-03-27T11:10:08.260147+0000) 2024-03-27T11:10:27.888 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:27 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:27.845+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.463226+0000 front 2024-03-27T11:09:56.463040+0000 (oldest deadline 2024-03-27T11:10:18.763354+0000) 2024-03-27T11:10:28.013 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:27 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:27.753+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.380254+0000 front 2024-03-27T11:09:23.380098+0000 (oldest deadline 2024-03-27T11:09:49.280089+0000) 2024-03-27T11:10:28.013 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:27 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:27.753+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:34.582052+0000 front 2024-03-27T11:09:34.582101+0000 (oldest deadline 2024-03-27T11:10:00.482194+0000) 2024-03-27T11:10:28.013 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:27 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:27.753+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:45.784185+0000 front 2024-03-27T11:09:45.784030+0000 (oldest deadline 2024-03-27T11:10:08.683959+0000) 2024-03-27T11:10:28.013 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:27 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:27.753+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:55.086370+0000 front 2024-03-27T11:09:55.086460+0000 (oldest deadline 2024-03-27T11:10:17.386639+0000) 2024-03-27T11:10:28.387 INFO:journalctl@ceph.osd.6.smithi097.stdout:Mar 27 11:10:28 smithi097 podman[97526]: 2024-03-27 11:10:28.064886964 +0000 UTC m=+0.897061330 container died 75f1472308be336d78183b1ef39fd1cab6e2fd400ea7f21ca1643053a985fa59 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-6-deactivate, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-03-27T11:10:28.423 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:28 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:28.100+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.236297+0000 front 2024-03-27T11:09:28.236728+0000 (oldest deadline 2024-03-27T11:09:51.136525+0000) 2024-03-27T11:10:28.423 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:28 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:28.100+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.340023+0000 front 2024-03-27T11:09:36.340351+0000 (oldest deadline 2024-03-27T11:10:02.239659+0000) 2024-03-27T11:10:28.423 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:28 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:28.100+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:46.143084+0000 front 2024-03-27T11:09:46.143228+0000 (oldest deadline 2024-03-27T11:10:10.843246+0000) 2024-03-27T11:10:28.423 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:28 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:28.100+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.046124+0000 front 2024-03-27T11:09:56.046017+0000 (oldest deadline 2024-03-27T11:10:20.746204+0000) 2024-03-27T11:10:28.752 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-adca88c4-ec26-11ee-95d0-87774f69a715@osd.6.service' 2024-03-27T11:10:28.763 INFO:journalctl@ceph.osd.7.smithi114.stdout:Mar 27 11:10:28 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-7[53256]: 2024-03-27T11:10:28.327+0000 7fd5e5800700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.916460+0000 front 2024-03-27T11:09:36.916564+0000 (oldest deadline 2024-03-27T11:10:01.016630+0000) 2024-03-27T11:10:28.763 INFO:journalctl@ceph.osd.7.smithi114.stdout:Mar 27 11:10:28 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-7[53256]: 2024-03-27T11:10:28.327+0000 7fd5e5800700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:43.918347+0000 front 2024-03-27T11:09:43.918473+0000 (oldest deadline 2024-03-27T11:10:08.618602+0000) 2024-03-27T11:10:28.763 INFO:journalctl@ceph.osd.7.smithi114.stdout:Mar 27 11:10:28 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-7[53256]: 2024-03-27T11:10:28.327+0000 7fd5e5800700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:55.522718+0000 front 2024-03-27T11:09:55.522838+0000 (oldest deadline 2024-03-27T11:10:20.222876+0000) 2024-03-27T11:10:28.763 INFO:journalctl@ceph.osd.7.smithi114.stdout:Mar 27 11:10:28 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-7[53256]: 2024-03-27T11:10:28.327+0000 7fd5e5800700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:04.924837+0000 front 2024-03-27T11:10:04.924883+0000 (oldest deadline 2024-03-27T11:10:27.224800+0000) 2024-03-27T11:10:28.862 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-27T11:10:28.862 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-03-27T11:10:28.862 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-03-27T11:10:28.862 DEBUG:teuthology.orchestra.run.smithi114:> sudo systemctl stop ceph-adca88c4-ec26-11ee-95d0-87774f69a715@osd.7 2024-03-27T11:10:28.923 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:28 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:28.488+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.140407+0000 front 2024-03-27T11:09:28.140164+0000 (oldest deadline 2024-03-27T11:09:54.040233+0000) 2024-03-27T11:10:28.923 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:28 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:28.488+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.844416+0000 front 2024-03-27T11:09:36.843999+0000 (oldest deadline 2024-03-27T11:10:01.544084+0000) 2024-03-27T11:10:28.923 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:28 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:28.488+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:47.348157+0000 front 2024-03-27T11:09:47.348480+0000 (oldest deadline 2024-03-27T11:10:09.647966+0000) 2024-03-27T11:10:28.923 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:28 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:28.488+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.051047+0000 front 2024-03-27T11:09:56.050946+0000 (oldest deadline 2024-03-27T11:10:20.750821+0000) 2024-03-27T11:10:29.013 INFO:journalctl@ceph.osd.7.smithi114.stdout:Mar 27 11:10:29 smithi114 systemd[1]: Stopping Ceph osd.7 for adca88c4-ec26-11ee-95d0-87774f69a715... 2024-03-27T11:10:29.013 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:28 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:28.793+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.380254+0000 front 2024-03-27T11:09:23.380098+0000 (oldest deadline 2024-03-27T11:09:49.280089+0000) 2024-03-27T11:10:29.013 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:28 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:28.793+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:34.582052+0000 front 2024-03-27T11:09:34.582101+0000 (oldest deadline 2024-03-27T11:10:00.482194+0000) 2024-03-27T11:10:29.014 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:28 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:28.793+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:45.784185+0000 front 2024-03-27T11:09:45.784030+0000 (oldest deadline 2024-03-27T11:10:08.683959+0000) 2024-03-27T11:10:29.014 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:28 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:28.793+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:55.086370+0000 front 2024-03-27T11:09:55.086460+0000 (oldest deadline 2024-03-27T11:10:17.386639+0000) 2024-03-27T11:10:29.014 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:28 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:28.793+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:03.189572+0000 front 2024-03-27T11:10:03.189451+0000 (oldest deadline 2024-03-27T11:10:28.390279+0000) 2024-03-27T11:10:29.137 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:28 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:28.878+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.654238+0000 front 2024-03-27T11:09:23.654318+0000 (oldest deadline 2024-03-27T11:09:48.954440+0000) 2024-03-27T11:10:29.137 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:28 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:28.878+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:35.957512+0000 front 2024-03-27T11:09:35.957437+0000 (oldest deadline 2024-03-27T11:10:01.857676+0000) 2024-03-27T11:10:29.137 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:28 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:28.878+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:44.160107+0000 front 2024-03-27T11:09:44.160318+0000 (oldest deadline 2024-03-27T11:10:08.260147+0000) 2024-03-27T11:10:29.137 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:28 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:28.878+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.463226+0000 front 2024-03-27T11:09:56.463040+0000 (oldest deadline 2024-03-27T11:10:18.763354+0000) 2024-03-27T11:10:29.138 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:28 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:28.878+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:05.767047+0000 front 2024-03-27T11:10:05.766994+0000 (oldest deadline 2024-03-27T11:10:28.067070+0000) 2024-03-27T11:10:29.423 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:29 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:29.117+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.236297+0000 front 2024-03-27T11:09:28.236728+0000 (oldest deadline 2024-03-27T11:09:51.136525+0000) 2024-03-27T11:10:29.424 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:29 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:29.117+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.340023+0000 front 2024-03-27T11:09:36.340351+0000 (oldest deadline 2024-03-27T11:10:02.239659+0000) 2024-03-27T11:10:29.424 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:29 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:29.117+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:46.143084+0000 front 2024-03-27T11:09:46.143228+0000 (oldest deadline 2024-03-27T11:10:10.843246+0000) 2024-03-27T11:10:29.424 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:29 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:29.117+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.046124+0000 front 2024-03-27T11:09:56.046017+0000 (oldest deadline 2024-03-27T11:10:20.746204+0000) 2024-03-27T11:10:29.424 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:29 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:29.117+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:00.747126+0000 front 2024-03-27T11:10:00.747259+0000 (oldest deadline 2024-03-27T11:10:28.348502+0000) 2024-03-27T11:10:29.513 INFO:journalctl@ceph.osd.7.smithi114.stdout:Mar 27 11:10:29 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-7[53256]: 2024-03-27T11:10:29.292+0000 7fd5e5800700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.916460+0000 front 2024-03-27T11:09:36.916564+0000 (oldest deadline 2024-03-27T11:10:01.016630+0000) 2024-03-27T11:10:29.513 INFO:journalctl@ceph.osd.7.smithi114.stdout:Mar 27 11:10:29 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-7[53256]: 2024-03-27T11:10:29.292+0000 7fd5e5800700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:43.918347+0000 front 2024-03-27T11:09:43.918473+0000 (oldest deadline 2024-03-27T11:10:08.618602+0000) 2024-03-27T11:10:29.514 INFO:journalctl@ceph.osd.7.smithi114.stdout:Mar 27 11:10:29 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-7[53256]: 2024-03-27T11:10:29.292+0000 7fd5e5800700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:55.522718+0000 front 2024-03-27T11:09:55.522838+0000 (oldest deadline 2024-03-27T11:10:20.222876+0000) 2024-03-27T11:10:29.514 INFO:journalctl@ceph.osd.7.smithi114.stdout:Mar 27 11:10:29 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-7[53256]: 2024-03-27T11:10:29.292+0000 7fd5e5800700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:04.924837+0000 front 2024-03-27T11:10:04.924883+0000 (oldest deadline 2024-03-27T11:10:27.224800+0000) 2024-03-27T11:10:29.514 INFO:journalctl@ceph.osd.7.smithi114.stdout:Mar 27 11:10:29 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-7[53256]: 2024-03-27T11:10:29.456+0000 7fd5ea000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-27T11:10:29.514 INFO:journalctl@ceph.osd.7.smithi114.stdout:Mar 27 11:10:29 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-7[53256]: 2024-03-27T11:10:29.456+0000 7fd5ea000700 -1 osd.7 92 *** Got signal Terminated *** 2024-03-27T11:10:29.514 INFO:journalctl@ceph.osd.7.smithi114.stdout:Mar 27 11:10:29 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-7[53256]: 2024-03-27T11:10:29.456+0000 7fd5ea000700 -1 osd.7 92 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-27T11:10:29.923 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:29 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:29.530+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.140407+0000 front 2024-03-27T11:09:28.140164+0000 (oldest deadline 2024-03-27T11:09:54.040233+0000) 2024-03-27T11:10:29.923 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:29 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:29.530+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.844416+0000 front 2024-03-27T11:09:36.843999+0000 (oldest deadline 2024-03-27T11:10:01.544084+0000) 2024-03-27T11:10:29.923 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:29 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:29.530+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:47.348157+0000 front 2024-03-27T11:09:47.348480+0000 (oldest deadline 2024-03-27T11:10:09.647966+0000) 2024-03-27T11:10:29.924 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:29 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:29.530+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.051047+0000 front 2024-03-27T11:09:56.050946+0000 (oldest deadline 2024-03-27T11:10:20.750821+0000) 2024-03-27T11:10:30.137 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:29 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:29.886+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.654238+0000 front 2024-03-27T11:09:23.654318+0000 (oldest deadline 2024-03-27T11:09:48.954440+0000) 2024-03-27T11:10:30.137 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:29 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:29.886+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:35.957512+0000 front 2024-03-27T11:09:35.957437+0000 (oldest deadline 2024-03-27T11:10:01.857676+0000) 2024-03-27T11:10:30.137 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:29 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:29.886+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:44.160107+0000 front 2024-03-27T11:09:44.160318+0000 (oldest deadline 2024-03-27T11:10:08.260147+0000) 2024-03-27T11:10:30.138 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:29 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:29.886+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.463226+0000 front 2024-03-27T11:09:56.463040+0000 (oldest deadline 2024-03-27T11:10:18.763354+0000) 2024-03-27T11:10:30.138 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:29 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:29.886+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:05.767047+0000 front 2024-03-27T11:10:05.766994+0000 (oldest deadline 2024-03-27T11:10:28.067070+0000) 2024-03-27T11:10:30.232 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:29 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:29.833+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.380254+0000 front 2024-03-27T11:09:23.380098+0000 (oldest deadline 2024-03-27T11:09:49.280089+0000) 2024-03-27T11:10:30.232 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:29 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:29.833+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:34.582052+0000 front 2024-03-27T11:09:34.582101+0000 (oldest deadline 2024-03-27T11:10:00.482194+0000) 2024-03-27T11:10:30.232 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:29 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:29.833+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:45.784185+0000 front 2024-03-27T11:09:45.784030+0000 (oldest deadline 2024-03-27T11:10:08.683959+0000) 2024-03-27T11:10:30.232 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:29 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:29.833+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:55.086370+0000 front 2024-03-27T11:09:55.086460+0000 (oldest deadline 2024-03-27T11:10:17.386639+0000) 2024-03-27T11:10:30.232 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:29 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:29.833+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:03.189572+0000 front 2024-03-27T11:10:03.189451+0000 (oldest deadline 2024-03-27T11:10:28.390279+0000) 2024-03-27T11:10:30.423 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:30 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:30.139+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.236297+0000 front 2024-03-27T11:09:28.236728+0000 (oldest deadline 2024-03-27T11:09:51.136525+0000) 2024-03-27T11:10:30.423 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:30 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:30.139+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.340023+0000 front 2024-03-27T11:09:36.340351+0000 (oldest deadline 2024-03-27T11:10:02.239659+0000) 2024-03-27T11:10:30.423 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:30 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:30.139+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:46.143084+0000 front 2024-03-27T11:09:46.143228+0000 (oldest deadline 2024-03-27T11:10:10.843246+0000) 2024-03-27T11:10:30.423 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:30 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:30.139+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.046124+0000 front 2024-03-27T11:09:56.046017+0000 (oldest deadline 2024-03-27T11:10:20.746204+0000) 2024-03-27T11:10:30.424 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:30 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:30.139+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:00.747126+0000 front 2024-03-27T11:10:00.747259+0000 (oldest deadline 2024-03-27T11:10:28.348502+0000) 2024-03-27T11:10:30.513 INFO:journalctl@ceph.osd.7.smithi114.stdout:Mar 27 11:10:30 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-7[53256]: 2024-03-27T11:10:30.262+0000 7fd5e5800700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.916460+0000 front 2024-03-27T11:09:36.916564+0000 (oldest deadline 2024-03-27T11:10:01.016630+0000) 2024-03-27T11:10:30.513 INFO:journalctl@ceph.osd.7.smithi114.stdout:Mar 27 11:10:30 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-7[53256]: 2024-03-27T11:10:30.262+0000 7fd5e5800700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:43.918347+0000 front 2024-03-27T11:09:43.918473+0000 (oldest deadline 2024-03-27T11:10:08.618602+0000) 2024-03-27T11:10:30.513 INFO:journalctl@ceph.osd.7.smithi114.stdout:Mar 27 11:10:30 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-7[53256]: 2024-03-27T11:10:30.262+0000 7fd5e5800700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:55.522718+0000 front 2024-03-27T11:09:55.522838+0000 (oldest deadline 2024-03-27T11:10:20.222876+0000) 2024-03-27T11:10:30.513 INFO:journalctl@ceph.osd.7.smithi114.stdout:Mar 27 11:10:30 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-7[53256]: 2024-03-27T11:10:30.262+0000 7fd5e5800700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:04.924837+0000 front 2024-03-27T11:10:04.924883+0000 (oldest deadline 2024-03-27T11:10:27.224800+0000) 2024-03-27T11:10:30.923 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:30 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:30.521+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.140407+0000 front 2024-03-27T11:09:28.140164+0000 (oldest deadline 2024-03-27T11:09:54.040233+0000) 2024-03-27T11:10:30.924 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:30 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:30.521+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.844416+0000 front 2024-03-27T11:09:36.843999+0000 (oldest deadline 2024-03-27T11:10:01.544084+0000) 2024-03-27T11:10:30.924 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:30 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:30.521+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:47.348157+0000 front 2024-03-27T11:09:47.348480+0000 (oldest deadline 2024-03-27T11:10:09.647966+0000) 2024-03-27T11:10:30.924 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:30 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:30.521+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.051047+0000 front 2024-03-27T11:09:56.050946+0000 (oldest deadline 2024-03-27T11:10:20.750821+0000) 2024-03-27T11:10:30.924 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:30 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:30.521+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:01.852819+0000 front 2024-03-27T11:10:01.852917+0000 (oldest deadline 2024-03-27T11:10:29.955550+0000) 2024-03-27T11:10:31.137 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:30 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:30.897+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.654238+0000 front 2024-03-27T11:09:23.654318+0000 (oldest deadline 2024-03-27T11:09:48.954440+0000) 2024-03-27T11:10:31.137 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:30 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:30.897+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:35.957512+0000 front 2024-03-27T11:09:35.957437+0000 (oldest deadline 2024-03-27T11:10:01.857676+0000) 2024-03-27T11:10:31.137 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:30 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:30.897+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:44.160107+0000 front 2024-03-27T11:09:44.160318+0000 (oldest deadline 2024-03-27T11:10:08.260147+0000) 2024-03-27T11:10:31.138 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:30 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:30.897+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.463226+0000 front 2024-03-27T11:09:56.463040+0000 (oldest deadline 2024-03-27T11:10:18.763354+0000) 2024-03-27T11:10:31.138 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:30 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:30.897+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:05.767047+0000 front 2024-03-27T11:10:05.766994+0000 (oldest deadline 2024-03-27T11:10:28.067070+0000) 2024-03-27T11:10:31.263 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:30 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:30.860+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.380254+0000 front 2024-03-27T11:09:23.380098+0000 (oldest deadline 2024-03-27T11:09:49.280089+0000) 2024-03-27T11:10:31.264 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:30 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:30.860+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:34.582052+0000 front 2024-03-27T11:09:34.582101+0000 (oldest deadline 2024-03-27T11:10:00.482194+0000) 2024-03-27T11:10:31.264 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:30 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:30.860+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:45.784185+0000 front 2024-03-27T11:09:45.784030+0000 (oldest deadline 2024-03-27T11:10:08.683959+0000) 2024-03-27T11:10:31.264 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:30 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:30.860+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:55.086370+0000 front 2024-03-27T11:09:55.086460+0000 (oldest deadline 2024-03-27T11:10:17.386639+0000) 2024-03-27T11:10:31.264 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:30 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:30.860+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:03.189572+0000 front 2024-03-27T11:10:03.189451+0000 (oldest deadline 2024-03-27T11:10:28.390279+0000) 2024-03-27T11:10:31.424 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:31 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:31.131+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.236297+0000 front 2024-03-27T11:09:28.236728+0000 (oldest deadline 2024-03-27T11:09:51.136525+0000) 2024-03-27T11:10:31.424 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:31 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:31.131+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.340023+0000 front 2024-03-27T11:09:36.340351+0000 (oldest deadline 2024-03-27T11:10:02.239659+0000) 2024-03-27T11:10:31.424 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:31 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:31.131+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:46.143084+0000 front 2024-03-27T11:09:46.143228+0000 (oldest deadline 2024-03-27T11:10:10.843246+0000) 2024-03-27T11:10:31.424 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:31 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:31.131+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.046124+0000 front 2024-03-27T11:09:56.046017+0000 (oldest deadline 2024-03-27T11:10:20.746204+0000) 2024-03-27T11:10:31.424 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:31 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:31.131+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:00.747126+0000 front 2024-03-27T11:10:00.747259+0000 (oldest deadline 2024-03-27T11:10:28.348502+0000) 2024-03-27T11:10:31.763 INFO:journalctl@ceph.osd.7.smithi114.stdout:Mar 27 11:10:31 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-7[53256]: 2024-03-27T11:10:31.303+0000 7fd5e5800700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.916460+0000 front 2024-03-27T11:09:36.916564+0000 (oldest deadline 2024-03-27T11:10:01.016630+0000) 2024-03-27T11:10:31.763 INFO:journalctl@ceph.osd.7.smithi114.stdout:Mar 27 11:10:31 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-7[53256]: 2024-03-27T11:10:31.303+0000 7fd5e5800700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:43.918347+0000 front 2024-03-27T11:09:43.918473+0000 (oldest deadline 2024-03-27T11:10:08.618602+0000) 2024-03-27T11:10:31.763 INFO:journalctl@ceph.osd.7.smithi114.stdout:Mar 27 11:10:31 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-7[53256]: 2024-03-27T11:10:31.303+0000 7fd5e5800700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:55.522718+0000 front 2024-03-27T11:09:55.522838+0000 (oldest deadline 2024-03-27T11:10:20.222876+0000) 2024-03-27T11:10:31.764 INFO:journalctl@ceph.osd.7.smithi114.stdout:Mar 27 11:10:31 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-7[53256]: 2024-03-27T11:10:31.303+0000 7fd5e5800700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:04.924837+0000 front 2024-03-27T11:10:04.924883+0000 (oldest deadline 2024-03-27T11:10:27.224800+0000) 2024-03-27T11:10:31.923 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:31 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:31.494+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.140407+0000 front 2024-03-27T11:09:28.140164+0000 (oldest deadline 2024-03-27T11:09:54.040233+0000) 2024-03-27T11:10:31.923 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:31 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:31.494+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.844416+0000 front 2024-03-27T11:09:36.843999+0000 (oldest deadline 2024-03-27T11:10:01.544084+0000) 2024-03-27T11:10:31.923 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:31 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:31.494+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:47.348157+0000 front 2024-03-27T11:09:47.348480+0000 (oldest deadline 2024-03-27T11:10:09.647966+0000) 2024-03-27T11:10:31.924 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:31 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:31.494+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.051047+0000 front 2024-03-27T11:09:56.050946+0000 (oldest deadline 2024-03-27T11:10:20.750821+0000) 2024-03-27T11:10:31.924 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:31 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:31.494+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:01.852819+0000 front 2024-03-27T11:10:01.852917+0000 (oldest deadline 2024-03-27T11:10:29.955550+0000) 2024-03-27T11:10:32.137 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:31 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:31.886+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.654238+0000 front 2024-03-27T11:09:23.654318+0000 (oldest deadline 2024-03-27T11:09:48.954440+0000) 2024-03-27T11:10:32.138 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:31 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:31.886+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:35.957512+0000 front 2024-03-27T11:09:35.957437+0000 (oldest deadline 2024-03-27T11:10:01.857676+0000) 2024-03-27T11:10:32.138 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:31 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:31.886+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:44.160107+0000 front 2024-03-27T11:09:44.160318+0000 (oldest deadline 2024-03-27T11:10:08.260147+0000) 2024-03-27T11:10:32.138 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:31 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:31.886+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.463226+0000 front 2024-03-27T11:09:56.463040+0000 (oldest deadline 2024-03-27T11:10:18.763354+0000) 2024-03-27T11:10:32.138 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:31 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:31.886+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:05.767047+0000 front 2024-03-27T11:10:05.766994+0000 (oldest deadline 2024-03-27T11:10:28.067070+0000) 2024-03-27T11:10:32.263 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:31 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:31.836+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.380254+0000 front 2024-03-27T11:09:23.380098+0000 (oldest deadline 2024-03-27T11:09:49.280089+0000) 2024-03-27T11:10:32.263 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:31 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:31.836+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:34.582052+0000 front 2024-03-27T11:09:34.582101+0000 (oldest deadline 2024-03-27T11:10:00.482194+0000) 2024-03-27T11:10:32.263 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:31 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:31.836+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:45.784185+0000 front 2024-03-27T11:09:45.784030+0000 (oldest deadline 2024-03-27T11:10:08.683959+0000) 2024-03-27T11:10:32.264 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:31 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:31.836+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:55.086370+0000 front 2024-03-27T11:09:55.086460+0000 (oldest deadline 2024-03-27T11:10:17.386639+0000) 2024-03-27T11:10:32.264 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:31 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:31.836+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:03.189572+0000 front 2024-03-27T11:10:03.189451+0000 (oldest deadline 2024-03-27T11:10:28.390279+0000) 2024-03-27T11:10:32.423 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:32 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:32.122+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.236297+0000 front 2024-03-27T11:09:28.236728+0000 (oldest deadline 2024-03-27T11:09:51.136525+0000) 2024-03-27T11:10:32.423 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:32 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:32.122+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.340023+0000 front 2024-03-27T11:09:36.340351+0000 (oldest deadline 2024-03-27T11:10:02.239659+0000) 2024-03-27T11:10:32.423 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:32 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:32.122+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:46.143084+0000 front 2024-03-27T11:09:46.143228+0000 (oldest deadline 2024-03-27T11:10:10.843246+0000) 2024-03-27T11:10:32.424 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:32 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:32.122+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.046124+0000 front 2024-03-27T11:09:56.046017+0000 (oldest deadline 2024-03-27T11:10:20.746204+0000) 2024-03-27T11:10:32.424 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:32 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:32.122+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:00.747126+0000 front 2024-03-27T11:10:00.747259+0000 (oldest deadline 2024-03-27T11:10:28.348502+0000) 2024-03-27T11:10:32.674 INFO:journalctl@ceph.osd.7.smithi114.stdout:Mar 27 11:10:32 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-7[53256]: 2024-03-27T11:10:32.297+0000 7fd5e5800700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.916460+0000 front 2024-03-27T11:09:36.916564+0000 (oldest deadline 2024-03-27T11:10:01.016630+0000) 2024-03-27T11:10:32.674 INFO:journalctl@ceph.osd.7.smithi114.stdout:Mar 27 11:10:32 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-7[53256]: 2024-03-27T11:10:32.297+0000 7fd5e5800700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:43.918347+0000 front 2024-03-27T11:09:43.918473+0000 (oldest deadline 2024-03-27T11:10:08.618602+0000) 2024-03-27T11:10:32.674 INFO:journalctl@ceph.osd.7.smithi114.stdout:Mar 27 11:10:32 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-7[53256]: 2024-03-27T11:10:32.297+0000 7fd5e5800700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:55.522718+0000 front 2024-03-27T11:09:55.522838+0000 (oldest deadline 2024-03-27T11:10:20.222876+0000) 2024-03-27T11:10:32.674 INFO:journalctl@ceph.osd.7.smithi114.stdout:Mar 27 11:10:32 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-7[53256]: 2024-03-27T11:10:32.297+0000 7fd5e5800700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:04.924837+0000 front 2024-03-27T11:10:04.924883+0000 (oldest deadline 2024-03-27T11:10:27.224800+0000) 2024-03-27T11:10:32.923 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:32 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:32.497+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.140407+0000 front 2024-03-27T11:09:28.140164+0000 (oldest deadline 2024-03-27T11:09:54.040233+0000) 2024-03-27T11:10:32.923 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:32 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:32.497+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.844416+0000 front 2024-03-27T11:09:36.843999+0000 (oldest deadline 2024-03-27T11:10:01.544084+0000) 2024-03-27T11:10:32.923 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:32 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:32.497+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:47.348157+0000 front 2024-03-27T11:09:47.348480+0000 (oldest deadline 2024-03-27T11:10:09.647966+0000) 2024-03-27T11:10:32.923 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:32 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:32.497+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.051047+0000 front 2024-03-27T11:09:56.050946+0000 (oldest deadline 2024-03-27T11:10:20.750821+0000) 2024-03-27T11:10:32.924 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:32 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:32.497+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:01.852819+0000 front 2024-03-27T11:10:01.852917+0000 (oldest deadline 2024-03-27T11:10:29.955550+0000) 2024-03-27T11:10:33.013 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:32 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:32.820+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.380254+0000 front 2024-03-27T11:09:23.380098+0000 (oldest deadline 2024-03-27T11:09:49.280089+0000) 2024-03-27T11:10:33.013 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:32 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:32.820+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:34.582052+0000 front 2024-03-27T11:09:34.582101+0000 (oldest deadline 2024-03-27T11:10:00.482194+0000) 2024-03-27T11:10:33.013 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:32 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:32.820+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:45.784185+0000 front 2024-03-27T11:09:45.784030+0000 (oldest deadline 2024-03-27T11:10:08.683959+0000) 2024-03-27T11:10:33.014 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:32 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:32.820+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:55.086370+0000 front 2024-03-27T11:09:55.086460+0000 (oldest deadline 2024-03-27T11:10:17.386639+0000) 2024-03-27T11:10:33.014 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:32 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:32.820+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:03.189572+0000 front 2024-03-27T11:10:03.189451+0000 (oldest deadline 2024-03-27T11:10:28.390279+0000) 2024-03-27T11:10:33.137 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:32 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:32.892+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.654238+0000 front 2024-03-27T11:09:23.654318+0000 (oldest deadline 2024-03-27T11:09:48.954440+0000) 2024-03-27T11:10:33.138 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:32 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:32.892+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:35.957512+0000 front 2024-03-27T11:09:35.957437+0000 (oldest deadline 2024-03-27T11:10:01.857676+0000) 2024-03-27T11:10:33.138 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:32 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:32.892+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:44.160107+0000 front 2024-03-27T11:09:44.160318+0000 (oldest deadline 2024-03-27T11:10:08.260147+0000) 2024-03-27T11:10:33.138 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:32 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:32.892+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.463226+0000 front 2024-03-27T11:09:56.463040+0000 (oldest deadline 2024-03-27T11:10:18.763354+0000) 2024-03-27T11:10:33.138 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:32 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:32.892+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:05.767047+0000 front 2024-03-27T11:10:05.766994+0000 (oldest deadline 2024-03-27T11:10:28.067070+0000) 2024-03-27T11:10:33.423 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:33 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:33.095+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.236297+0000 front 2024-03-27T11:09:28.236728+0000 (oldest deadline 2024-03-27T11:09:51.136525+0000) 2024-03-27T11:10:33.424 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:33 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:33.095+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.340023+0000 front 2024-03-27T11:09:36.340351+0000 (oldest deadline 2024-03-27T11:10:02.239659+0000) 2024-03-27T11:10:33.424 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:33 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:33.095+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:46.143084+0000 front 2024-03-27T11:09:46.143228+0000 (oldest deadline 2024-03-27T11:10:10.843246+0000) 2024-03-27T11:10:33.424 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:33 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:33.095+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.046124+0000 front 2024-03-27T11:09:56.046017+0000 (oldest deadline 2024-03-27T11:10:20.746204+0000) 2024-03-27T11:10:33.424 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:33 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:33.095+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:00.747126+0000 front 2024-03-27T11:10:00.747259+0000 (oldest deadline 2024-03-27T11:10:28.348502+0000) 2024-03-27T11:10:33.763 INFO:journalctl@ceph.osd.7.smithi114.stdout:Mar 27 11:10:33 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-7[53256]: 2024-03-27T11:10:33.341+0000 7fd5e5800700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.916460+0000 front 2024-03-27T11:09:36.916564+0000 (oldest deadline 2024-03-27T11:10:01.016630+0000) 2024-03-27T11:10:33.764 INFO:journalctl@ceph.osd.7.smithi114.stdout:Mar 27 11:10:33 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-7[53256]: 2024-03-27T11:10:33.341+0000 7fd5e5800700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:43.918347+0000 front 2024-03-27T11:09:43.918473+0000 (oldest deadline 2024-03-27T11:10:08.618602+0000) 2024-03-27T11:10:33.764 INFO:journalctl@ceph.osd.7.smithi114.stdout:Mar 27 11:10:33 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-7[53256]: 2024-03-27T11:10:33.341+0000 7fd5e5800700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:55.522718+0000 front 2024-03-27T11:09:55.522838+0000 (oldest deadline 2024-03-27T11:10:20.222876+0000) 2024-03-27T11:10:33.764 INFO:journalctl@ceph.osd.7.smithi114.stdout:Mar 27 11:10:33 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-7[53256]: 2024-03-27T11:10:33.341+0000 7fd5e5800700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:04.924837+0000 front 2024-03-27T11:10:04.924883+0000 (oldest deadline 2024-03-27T11:10:27.224800+0000) 2024-03-27T11:10:33.923 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:33 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:33.517+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.140407+0000 front 2024-03-27T11:09:28.140164+0000 (oldest deadline 2024-03-27T11:09:54.040233+0000) 2024-03-27T11:10:33.923 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:33 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:33.517+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.844416+0000 front 2024-03-27T11:09:36.843999+0000 (oldest deadline 2024-03-27T11:10:01.544084+0000) 2024-03-27T11:10:33.923 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:33 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:33.517+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:47.348157+0000 front 2024-03-27T11:09:47.348480+0000 (oldest deadline 2024-03-27T11:10:09.647966+0000) 2024-03-27T11:10:33.923 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:33 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:33.517+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.051047+0000 front 2024-03-27T11:09:56.050946+0000 (oldest deadline 2024-03-27T11:10:20.750821+0000) 2024-03-27T11:10:33.923 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:33 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:33.517+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:01.852819+0000 front 2024-03-27T11:10:01.852917+0000 (oldest deadline 2024-03-27T11:10:29.955550+0000) 2024-03-27T11:10:34.263 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:33 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:33.861+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.380254+0000 front 2024-03-27T11:09:23.380098+0000 (oldest deadline 2024-03-27T11:09:49.280089+0000) 2024-03-27T11:10:34.263 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:33 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:33.861+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:34.582052+0000 front 2024-03-27T11:09:34.582101+0000 (oldest deadline 2024-03-27T11:10:00.482194+0000) 2024-03-27T11:10:34.263 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:33 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:33.861+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:45.784185+0000 front 2024-03-27T11:09:45.784030+0000 (oldest deadline 2024-03-27T11:10:08.683959+0000) 2024-03-27T11:10:34.264 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:33 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:33.861+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:55.086370+0000 front 2024-03-27T11:09:55.086460+0000 (oldest deadline 2024-03-27T11:10:17.386639+0000) 2024-03-27T11:10:34.264 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:33 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:33.861+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:03.189572+0000 front 2024-03-27T11:10:03.189451+0000 (oldest deadline 2024-03-27T11:10:28.390279+0000) 2024-03-27T11:10:34.387 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:33 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:33.922+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.654238+0000 front 2024-03-27T11:09:23.654318+0000 (oldest deadline 2024-03-27T11:09:48.954440+0000) 2024-03-27T11:10:34.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:33 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:33.922+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:35.957512+0000 front 2024-03-27T11:09:35.957437+0000 (oldest deadline 2024-03-27T11:10:01.857676+0000) 2024-03-27T11:10:34.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:33 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:33.922+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:44.160107+0000 front 2024-03-27T11:09:44.160318+0000 (oldest deadline 2024-03-27T11:10:08.260147+0000) 2024-03-27T11:10:34.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:33 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:33.922+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.463226+0000 front 2024-03-27T11:09:56.463040+0000 (oldest deadline 2024-03-27T11:10:18.763354+0000) 2024-03-27T11:10:34.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:33 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:33.922+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:05.767047+0000 front 2024-03-27T11:10:05.766994+0000 (oldest deadline 2024-03-27T11:10:28.067070+0000) 2024-03-27T11:10:34.423 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:34 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:34.078+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.236297+0000 front 2024-03-27T11:09:28.236728+0000 (oldest deadline 2024-03-27T11:09:51.136525+0000) 2024-03-27T11:10:34.423 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:34 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:34.078+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.340023+0000 front 2024-03-27T11:09:36.340351+0000 (oldest deadline 2024-03-27T11:10:02.239659+0000) 2024-03-27T11:10:34.424 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:34 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:34.078+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:46.143084+0000 front 2024-03-27T11:09:46.143228+0000 (oldest deadline 2024-03-27T11:10:10.843246+0000) 2024-03-27T11:10:34.424 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:34 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:34.078+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.046124+0000 front 2024-03-27T11:09:56.046017+0000 (oldest deadline 2024-03-27T11:10:20.746204+0000) 2024-03-27T11:10:34.424 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:34 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:34.078+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:00.747126+0000 front 2024-03-27T11:10:00.747259+0000 (oldest deadline 2024-03-27T11:10:28.348502+0000) 2024-03-27T11:10:34.766 INFO:journalctl@ceph.osd.7.smithi114.stdout:Mar 27 11:10:34 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-7[53256]: 2024-03-27T11:10:34.386+0000 7fd5e5800700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.916460+0000 front 2024-03-27T11:09:36.916564+0000 (oldest deadline 2024-03-27T11:10:01.016630+0000) 2024-03-27T11:10:34.767 INFO:journalctl@ceph.osd.7.smithi114.stdout:Mar 27 11:10:34 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-7[53256]: 2024-03-27T11:10:34.386+0000 7fd5e5800700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:43.918347+0000 front 2024-03-27T11:09:43.918473+0000 (oldest deadline 2024-03-27T11:10:08.618602+0000) 2024-03-27T11:10:34.767 INFO:journalctl@ceph.osd.7.smithi114.stdout:Mar 27 11:10:34 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-7[53256]: 2024-03-27T11:10:34.386+0000 7fd5e5800700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:55.522718+0000 front 2024-03-27T11:09:55.522838+0000 (oldest deadline 2024-03-27T11:10:20.222876+0000) 2024-03-27T11:10:34.767 INFO:journalctl@ceph.osd.7.smithi114.stdout:Mar 27 11:10:34 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-7[53256]: 2024-03-27T11:10:34.386+0000 7fd5e5800700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:04.924837+0000 front 2024-03-27T11:10:04.924883+0000 (oldest deadline 2024-03-27T11:10:27.224800+0000) 2024-03-27T11:10:34.923 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:34 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:34.546+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.140407+0000 front 2024-03-27T11:09:28.140164+0000 (oldest deadline 2024-03-27T11:09:54.040233+0000) 2024-03-27T11:10:34.923 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:34 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:34.546+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.844416+0000 front 2024-03-27T11:09:36.843999+0000 (oldest deadline 2024-03-27T11:10:01.544084+0000) 2024-03-27T11:10:34.923 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:34 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:34.546+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:47.348157+0000 front 2024-03-27T11:09:47.348480+0000 (oldest deadline 2024-03-27T11:10:09.647966+0000) 2024-03-27T11:10:34.923 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:34 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:34.546+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.051047+0000 front 2024-03-27T11:09:56.050946+0000 (oldest deadline 2024-03-27T11:10:20.750821+0000) 2024-03-27T11:10:34.923 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:34 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:34.546+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:01.852819+0000 front 2024-03-27T11:10:01.852917+0000 (oldest deadline 2024-03-27T11:10:29.955550+0000) 2024-03-27T11:10:35.227 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:34 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:34.821+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.380254+0000 front 2024-03-27T11:09:23.380098+0000 (oldest deadline 2024-03-27T11:09:49.280089+0000) 2024-03-27T11:10:35.227 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:34 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:34.821+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:34.582052+0000 front 2024-03-27T11:09:34.582101+0000 (oldest deadline 2024-03-27T11:10:00.482194+0000) 2024-03-27T11:10:35.227 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:34 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:34.821+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:45.784185+0000 front 2024-03-27T11:09:45.784030+0000 (oldest deadline 2024-03-27T11:10:08.683959+0000) 2024-03-27T11:10:35.227 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:34 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:34.821+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:55.086370+0000 front 2024-03-27T11:09:55.086460+0000 (oldest deadline 2024-03-27T11:10:17.386639+0000) 2024-03-27T11:10:35.227 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:34 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:34.821+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:03.189572+0000 front 2024-03-27T11:10:03.189451+0000 (oldest deadline 2024-03-27T11:10:28.390279+0000) 2024-03-27T11:10:35.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:34 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:34.922+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.654238+0000 front 2024-03-27T11:09:23.654318+0000 (oldest deadline 2024-03-27T11:09:48.954440+0000) 2024-03-27T11:10:35.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:34 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:34.922+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:35.957512+0000 front 2024-03-27T11:09:35.957437+0000 (oldest deadline 2024-03-27T11:10:01.857676+0000) 2024-03-27T11:10:35.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:34 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:34.922+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:44.160107+0000 front 2024-03-27T11:09:44.160318+0000 (oldest deadline 2024-03-27T11:10:08.260147+0000) 2024-03-27T11:10:35.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:34 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:34.922+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.463226+0000 front 2024-03-27T11:09:56.463040+0000 (oldest deadline 2024-03-27T11:10:18.763354+0000) 2024-03-27T11:10:35.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:34 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:34.922+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:05.767047+0000 front 2024-03-27T11:10:05.766994+0000 (oldest deadline 2024-03-27T11:10:28.067070+0000) 2024-03-27T11:10:35.423 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:35 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:35.117+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.236297+0000 front 2024-03-27T11:09:28.236728+0000 (oldest deadline 2024-03-27T11:09:51.136525+0000) 2024-03-27T11:10:35.423 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:35 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:35.117+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.340023+0000 front 2024-03-27T11:09:36.340351+0000 (oldest deadline 2024-03-27T11:10:02.239659+0000) 2024-03-27T11:10:35.424 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:35 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:35.117+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:46.143084+0000 front 2024-03-27T11:09:46.143228+0000 (oldest deadline 2024-03-27T11:10:10.843246+0000) 2024-03-27T11:10:35.424 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:35 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:35.117+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.046124+0000 front 2024-03-27T11:09:56.046017+0000 (oldest deadline 2024-03-27T11:10:20.746204+0000) 2024-03-27T11:10:35.424 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:35 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:35.117+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:00.747126+0000 front 2024-03-27T11:10:00.747259+0000 (oldest deadline 2024-03-27T11:10:28.348502+0000) 2024-03-27T11:10:35.513 INFO:journalctl@ceph.osd.7.smithi114.stdout:Mar 27 11:10:35 smithi114 podman[71002]: 2024-03-27 11:10:35.259388195 +0000 UTC m=+5.949809374 container died c6da1bc97668485b1825e72534b6b16bfdb11dc95bbc6451dbc7595600f61f96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-7, RELEASE=wip-rishabh-testing-20240326.131558-155268c, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-27T11:10:35.923 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:35 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:35.519+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.140407+0000 front 2024-03-27T11:09:28.140164+0000 (oldest deadline 2024-03-27T11:09:54.040233+0000) 2024-03-27T11:10:35.923 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:35 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:35.519+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.844416+0000 front 2024-03-27T11:09:36.843999+0000 (oldest deadline 2024-03-27T11:10:01.544084+0000) 2024-03-27T11:10:35.923 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:35 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:35.519+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:47.348157+0000 front 2024-03-27T11:09:47.348480+0000 (oldest deadline 2024-03-27T11:10:09.647966+0000) 2024-03-27T11:10:35.923 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:35 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:35.519+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.051047+0000 front 2024-03-27T11:09:56.050946+0000 (oldest deadline 2024-03-27T11:10:20.750821+0000) 2024-03-27T11:10:35.923 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:35 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:35.519+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:01.852819+0000 front 2024-03-27T11:10:01.852917+0000 (oldest deadline 2024-03-27T11:10:29.955550+0000) 2024-03-27T11:10:35.944 INFO:journalctl@ceph.osd.7.smithi114.stdout:Mar 27 11:10:35 smithi114 podman[71002]: 2024-03-27 11:10:35.749278319 +0000 UTC m=+6.439699474 container cleanup c6da1bc97668485b1825e72534b6b16bfdb11dc95bbc6451dbc7595600f61f96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-7, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e) 2024-03-27T11:10:35.944 INFO:journalctl@ceph.osd.7.smithi114.stdout:Mar 27 11:10:35 smithi114 bash[71002]: ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-7 2024-03-27T11:10:35.944 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:35 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:35.806+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.380254+0000 front 2024-03-27T11:09:23.380098+0000 (oldest deadline 2024-03-27T11:09:49.280089+0000) 2024-03-27T11:10:35.945 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:35 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:35.806+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:34.582052+0000 front 2024-03-27T11:09:34.582101+0000 (oldest deadline 2024-03-27T11:10:00.482194+0000) 2024-03-27T11:10:35.945 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:35 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:35.806+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:45.784185+0000 front 2024-03-27T11:09:45.784030+0000 (oldest deadline 2024-03-27T11:10:08.683959+0000) 2024-03-27T11:10:35.945 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:35 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:35.806+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:55.086370+0000 front 2024-03-27T11:09:55.086460+0000 (oldest deadline 2024-03-27T11:10:17.386639+0000) 2024-03-27T11:10:35.945 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:35 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:35.806+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:03.189572+0000 front 2024-03-27T11:10:03.189451+0000 (oldest deadline 2024-03-27T11:10:28.390279+0000) 2024-03-27T11:10:36.137 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:35 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:35.876+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.654238+0000 front 2024-03-27T11:09:23.654318+0000 (oldest deadline 2024-03-27T11:09:48.954440+0000) 2024-03-27T11:10:36.137 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:35 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:35.877+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:35.957512+0000 front 2024-03-27T11:09:35.957437+0000 (oldest deadline 2024-03-27T11:10:01.857676+0000) 2024-03-27T11:10:36.138 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:35 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:35.877+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:44.160107+0000 front 2024-03-27T11:09:44.160318+0000 (oldest deadline 2024-03-27T11:10:08.260147+0000) 2024-03-27T11:10:36.138 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:35 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:35.877+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.463226+0000 front 2024-03-27T11:09:56.463040+0000 (oldest deadline 2024-03-27T11:10:18.763354+0000) 2024-03-27T11:10:36.138 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:35 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:35.877+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:05.767047+0000 front 2024-03-27T11:10:05.766994+0000 (oldest deadline 2024-03-27T11:10:28.067070+0000) 2024-03-27T11:10:36.263 INFO:journalctl@ceph.osd.7.smithi114.stdout:Mar 27 11:10:35 smithi114 podman[71018]: 2024-03-27 11:10:35.975818932 +0000 UTC m=+0.692945097 container remove c6da1bc97668485b1825e72534b6b16bfdb11dc95bbc6451dbc7595600f61f96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-7, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240326.131558-155268c, ceph=True, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326) 2024-03-27T11:10:36.423 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:36 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:36.144+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.236297+0000 front 2024-03-27T11:09:28.236728+0000 (oldest deadline 2024-03-27T11:09:51.136525+0000) 2024-03-27T11:10:36.424 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:36 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:36.144+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.340023+0000 front 2024-03-27T11:09:36.340351+0000 (oldest deadline 2024-03-27T11:10:02.239659+0000) 2024-03-27T11:10:36.424 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:36 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:36.144+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:46.143084+0000 front 2024-03-27T11:09:46.143228+0000 (oldest deadline 2024-03-27T11:10:10.843246+0000) 2024-03-27T11:10:36.424 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:36 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:36.144+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.046124+0000 front 2024-03-27T11:09:56.046017+0000 (oldest deadline 2024-03-27T11:10:20.746204+0000) 2024-03-27T11:10:36.424 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:36 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:36.144+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:00.747126+0000 front 2024-03-27T11:10:00.747259+0000 (oldest deadline 2024-03-27T11:10:28.348502+0000) 2024-03-27T11:10:36.923 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:36 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:36.500+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.140407+0000 front 2024-03-27T11:09:28.140164+0000 (oldest deadline 2024-03-27T11:09:54.040233+0000) 2024-03-27T11:10:36.924 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:36 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:36.500+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.844416+0000 front 2024-03-27T11:09:36.843999+0000 (oldest deadline 2024-03-27T11:10:01.544084+0000) 2024-03-27T11:10:36.924 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:36 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:36.500+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:47.348157+0000 front 2024-03-27T11:09:47.348480+0000 (oldest deadline 2024-03-27T11:10:09.647966+0000) 2024-03-27T11:10:36.924 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:36 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:36.500+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.051047+0000 front 2024-03-27T11:09:56.050946+0000 (oldest deadline 2024-03-27T11:10:20.750821+0000) 2024-03-27T11:10:36.924 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:36 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:36.500+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:01.852819+0000 front 2024-03-27T11:10:01.852917+0000 (oldest deadline 2024-03-27T11:10:29.955550+0000) 2024-03-27T11:10:36.982 INFO:journalctl@ceph.osd.7.smithi114.stdout:Mar 27 11:10:36 smithi114 podman[71103]: 2024-03-27 11:10:36.612484288 +0000 UTC m=+0.034368097 image pull b379bbd8da6b5d0fa9a3523941efac213b6815e1cbf05c6507954f0a3fe7a801 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772 2024-03-27T11:10:36.982 INFO:journalctl@ceph.osd.7.smithi114.stdout:Mar 27 11:10:36 smithi114 podman[71103]: 2024-03-27 11:10:36.743358492 +0000 UTC m=+0.165242306 container create a28f4680e411727af1770af717e2128968a9fd09cb2fbbc558b5fb93f41ccea4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-7-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240326.131558-155268c) 2024-03-27T11:10:36.982 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:36 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:36.826+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.380254+0000 front 2024-03-27T11:09:23.380098+0000 (oldest deadline 2024-03-27T11:09:49.280089+0000) 2024-03-27T11:10:36.983 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:36 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:36.826+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:34.582052+0000 front 2024-03-27T11:09:34.582101+0000 (oldest deadline 2024-03-27T11:10:00.482194+0000) 2024-03-27T11:10:36.983 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:36 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:36.826+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:45.784185+0000 front 2024-03-27T11:09:45.784030+0000 (oldest deadline 2024-03-27T11:10:08.683959+0000) 2024-03-27T11:10:36.983 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:36 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:36.826+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:55.086370+0000 front 2024-03-27T11:09:55.086460+0000 (oldest deadline 2024-03-27T11:10:17.386639+0000) 2024-03-27T11:10:36.983 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:36 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:36.826+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:03.189572+0000 front 2024-03-27T11:10:03.189451+0000 (oldest deadline 2024-03-27T11:10:28.390279+0000) 2024-03-27T11:10:37.137 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:36 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:36.911+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.654238+0000 front 2024-03-27T11:09:23.654318+0000 (oldest deadline 2024-03-27T11:09:48.954440+0000) 2024-03-27T11:10:37.138 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:36 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:36.911+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:35.957512+0000 front 2024-03-27T11:09:35.957437+0000 (oldest deadline 2024-03-27T11:10:01.857676+0000) 2024-03-27T11:10:37.138 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:36 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:36.911+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:44.160107+0000 front 2024-03-27T11:09:44.160318+0000 (oldest deadline 2024-03-27T11:10:08.260147+0000) 2024-03-27T11:10:37.138 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:36 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:36.911+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.463226+0000 front 2024-03-27T11:09:56.463040+0000 (oldest deadline 2024-03-27T11:10:18.763354+0000) 2024-03-27T11:10:37.138 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:36 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:36.911+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:05.767047+0000 front 2024-03-27T11:10:05.766994+0000 (oldest deadline 2024-03-27T11:10:28.067070+0000) 2024-03-27T11:10:37.264 INFO:journalctl@ceph.osd.7.smithi114.stdout:Mar 27 11:10:37 smithi114 podman[71103]: 2024-03-27 11:10:37.013405579 +0000 UTC m=+0.435289389 container init a28f4680e411727af1770af717e2128968a9fd09cb2fbbc558b5fb93f41ccea4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-7-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240326.131558-155268c, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-27T11:10:37.264 INFO:journalctl@ceph.osd.7.smithi114.stdout:Mar 27 11:10:37 smithi114 podman[71103]: 2024-03-27 11:10:37.024501585 +0000 UTC m=+0.446385388 container start a28f4680e411727af1770af717e2128968a9fd09cb2fbbc558b5fb93f41ccea4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-7-deactivate, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-rishabh-testing-20240326.131558-155268c, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-03-27T11:10:37.264 INFO:journalctl@ceph.osd.7.smithi114.stdout:Mar 27 11:10:37 smithi114 podman[71103]: 2024-03-27 11:10:37.120933564 +0000 UTC m=+0.542817444 container attach a28f4680e411727af1770af717e2128968a9fd09cb2fbbc558b5fb93f41ccea4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-7-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240326.131558-155268c) 2024-03-27T11:10:37.423 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:37 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:37.164+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.236297+0000 front 2024-03-27T11:09:28.236728+0000 (oldest deadline 2024-03-27T11:09:51.136525+0000) 2024-03-27T11:10:37.423 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:37 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:37.164+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.340023+0000 front 2024-03-27T11:09:36.340351+0000 (oldest deadline 2024-03-27T11:10:02.239659+0000) 2024-03-27T11:10:37.423 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:37 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:37.164+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:46.143084+0000 front 2024-03-27T11:09:46.143228+0000 (oldest deadline 2024-03-27T11:10:10.843246+0000) 2024-03-27T11:10:37.423 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:37 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:37.164+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.046124+0000 front 2024-03-27T11:09:56.046017+0000 (oldest deadline 2024-03-27T11:10:20.746204+0000) 2024-03-27T11:10:37.423 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:37 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:37.164+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:00.747126+0000 front 2024-03-27T11:10:00.747259+0000 (oldest deadline 2024-03-27T11:10:28.348502+0000) 2024-03-27T11:10:37.424 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:37 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:37.164+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-03-27T11:10:14.250568+0000 front 2024-03-27T11:10:14.250492+0000 (oldest deadline 2024-03-27T11:10:37.150604+0000) 2024-03-27T11:10:37.766 INFO:journalctl@ceph.osd.7.smithi114.stdout:Mar 27 11:10:37 smithi114 podman[71103]: 2024-03-27 11:10:37.494819284 +0000 UTC m=+0.916703098 container died a28f4680e411727af1770af717e2128968a9fd09cb2fbbc558b5fb93f41ccea4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-7-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.vendor=CentOS) 2024-03-27T11:10:37.923 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:37 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:37.544+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.140407+0000 front 2024-03-27T11:09:28.140164+0000 (oldest deadline 2024-03-27T11:09:54.040233+0000) 2024-03-27T11:10:37.924 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:37 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:37.544+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.844416+0000 front 2024-03-27T11:09:36.843999+0000 (oldest deadline 2024-03-27T11:10:01.544084+0000) 2024-03-27T11:10:37.924 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:37 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:37.544+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:47.348157+0000 front 2024-03-27T11:09:47.348480+0000 (oldest deadline 2024-03-27T11:10:09.647966+0000) 2024-03-27T11:10:37.924 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:37 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:37.544+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.051047+0000 front 2024-03-27T11:09:56.050946+0000 (oldest deadline 2024-03-27T11:10:20.750821+0000) 2024-03-27T11:10:37.924 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:37 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:37.544+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:01.852819+0000 front 2024-03-27T11:10:01.852917+0000 (oldest deadline 2024-03-27T11:10:29.955550+0000) 2024-03-27T11:10:38.137 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:37 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:37.906+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.654238+0000 front 2024-03-27T11:09:23.654318+0000 (oldest deadline 2024-03-27T11:09:48.954440+0000) 2024-03-27T11:10:38.138 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:37 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:37.906+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:35.957512+0000 front 2024-03-27T11:09:35.957437+0000 (oldest deadline 2024-03-27T11:10:01.857676+0000) 2024-03-27T11:10:38.138 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:37 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:37.906+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:44.160107+0000 front 2024-03-27T11:09:44.160318+0000 (oldest deadline 2024-03-27T11:10:08.260147+0000) 2024-03-27T11:10:38.138 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:37 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:37.906+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.463226+0000 front 2024-03-27T11:09:56.463040+0000 (oldest deadline 2024-03-27T11:10:18.763354+0000) 2024-03-27T11:10:38.138 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:37 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:37.906+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:05.767047+0000 front 2024-03-27T11:10:05.766994+0000 (oldest deadline 2024-03-27T11:10:28.067070+0000) 2024-03-27T11:10:38.138 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:37 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:37.906+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-03-27T11:10:13.171475+0000 front 2024-03-27T11:10:13.171056+0000 (oldest deadline 2024-03-27T11:10:37.171547+0000) 2024-03-27T11:10:38.201 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:37 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:37.868+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.380254+0000 front 2024-03-27T11:09:23.380098+0000 (oldest deadline 2024-03-27T11:09:49.280089+0000) 2024-03-27T11:10:38.201 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:37 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:37.868+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:34.582052+0000 front 2024-03-27T11:09:34.582101+0000 (oldest deadline 2024-03-27T11:10:00.482194+0000) 2024-03-27T11:10:38.201 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:37 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:37.868+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:45.784185+0000 front 2024-03-27T11:09:45.784030+0000 (oldest deadline 2024-03-27T11:10:08.683959+0000) 2024-03-27T11:10:38.201 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:37 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:37.868+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:55.086370+0000 front 2024-03-27T11:09:55.086460+0000 (oldest deadline 2024-03-27T11:10:17.386639+0000) 2024-03-27T11:10:38.201 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:37 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:37.868+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:03.189572+0000 front 2024-03-27T11:10:03.189451+0000 (oldest deadline 2024-03-27T11:10:28.390279+0000) 2024-03-27T11:10:38.234 DEBUG:teuthology.orchestra.run.smithi114:> sudo pkill -f 'journalctl -f -n 0 -u ceph-adca88c4-ec26-11ee-95d0-87774f69a715@osd.7.service' 2024-03-27T11:10:38.329 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-27T11:10:38.329 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-03-27T11:10:38.329 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-03-27T11:10:38.329 DEBUG:teuthology.orchestra.run.smithi177:> sudo systemctl stop ceph-adca88c4-ec26-11ee-95d0-87774f69a715@osd.8 2024-03-27T11:10:38.403 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:38 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:38.128+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.236297+0000 front 2024-03-27T11:09:28.236728+0000 (oldest deadline 2024-03-27T11:09:51.136525+0000) 2024-03-27T11:10:38.403 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:38 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:38.128+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.340023+0000 front 2024-03-27T11:09:36.340351+0000 (oldest deadline 2024-03-27T11:10:02.239659+0000) 2024-03-27T11:10:38.403 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:38 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:38.128+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:46.143084+0000 front 2024-03-27T11:09:46.143228+0000 (oldest deadline 2024-03-27T11:10:10.843246+0000) 2024-03-27T11:10:38.403 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:38 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:38.128+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.046124+0000 front 2024-03-27T11:09:56.046017+0000 (oldest deadline 2024-03-27T11:10:20.746204+0000) 2024-03-27T11:10:38.403 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:38 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:38.128+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:00.747126+0000 front 2024-03-27T11:10:00.747259+0000 (oldest deadline 2024-03-27T11:10:28.348502+0000) 2024-03-27T11:10:38.403 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:38 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:38.128+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-03-27T11:10:14.250568+0000 front 2024-03-27T11:10:14.250492+0000 (oldest deadline 2024-03-27T11:10:37.150604+0000) 2024-03-27T11:10:38.673 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:38 smithi177 systemd[1]: Stopping Ceph osd.8 for adca88c4-ec26-11ee-95d0-87774f69a715... 2024-03-27T11:10:38.673 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:38 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:38.593+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.140407+0000 front 2024-03-27T11:09:28.140164+0000 (oldest deadline 2024-03-27T11:09:54.040233+0000) 2024-03-27T11:10:38.673 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:38 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:38.593+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.844416+0000 front 2024-03-27T11:09:36.843999+0000 (oldest deadline 2024-03-27T11:10:01.544084+0000) 2024-03-27T11:10:38.673 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:38 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:38.593+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:47.348157+0000 front 2024-03-27T11:09:47.348480+0000 (oldest deadline 2024-03-27T11:10:09.647966+0000) 2024-03-27T11:10:38.673 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:38 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:38.593+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.051047+0000 front 2024-03-27T11:09:56.050946+0000 (oldest deadline 2024-03-27T11:10:20.750821+0000) 2024-03-27T11:10:38.673 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:38 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:38.593+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:01.852819+0000 front 2024-03-27T11:10:01.852917+0000 (oldest deadline 2024-03-27T11:10:29.955550+0000) 2024-03-27T11:10:39.083 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:38 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:38.837+0000 7f06aac00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-27T11:10:39.083 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:38 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:38.837+0000 7f06aac00700 -1 osd.8 92 *** Got signal Terminated *** 2024-03-27T11:10:39.083 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:38 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:38.837+0000 7f06aac00700 -1 osd.8 92 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-27T11:10:39.263 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:38 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:38.874+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.380254+0000 front 2024-03-27T11:09:23.380098+0000 (oldest deadline 2024-03-27T11:09:49.280089+0000) 2024-03-27T11:10:39.263 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:38 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:38.874+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:34.582052+0000 front 2024-03-27T11:09:34.582101+0000 (oldest deadline 2024-03-27T11:10:00.482194+0000) 2024-03-27T11:10:39.263 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:38 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:38.874+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:45.784185+0000 front 2024-03-27T11:09:45.784030+0000 (oldest deadline 2024-03-27T11:10:08.683959+0000) 2024-03-27T11:10:39.264 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:38 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:38.874+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:55.086370+0000 front 2024-03-27T11:09:55.086460+0000 (oldest deadline 2024-03-27T11:10:17.386639+0000) 2024-03-27T11:10:39.264 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:38 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:38.874+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:03.189572+0000 front 2024-03-27T11:10:03.189451+0000 (oldest deadline 2024-03-27T11:10:28.390279+0000) 2024-03-27T11:10:39.387 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:38 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:38.949+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.654238+0000 front 2024-03-27T11:09:23.654318+0000 (oldest deadline 2024-03-27T11:09:48.954440+0000) 2024-03-27T11:10:39.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:38 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:38.949+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:35.957512+0000 front 2024-03-27T11:09:35.957437+0000 (oldest deadline 2024-03-27T11:10:01.857676+0000) 2024-03-27T11:10:39.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:38 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:38.949+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:44.160107+0000 front 2024-03-27T11:09:44.160318+0000 (oldest deadline 2024-03-27T11:10:08.260147+0000) 2024-03-27T11:10:39.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:38 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:38.949+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.463226+0000 front 2024-03-27T11:09:56.463040+0000 (oldest deadline 2024-03-27T11:10:18.763354+0000) 2024-03-27T11:10:39.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:38 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:38.949+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:05.767047+0000 front 2024-03-27T11:10:05.766994+0000 (oldest deadline 2024-03-27T11:10:28.067070+0000) 2024-03-27T11:10:39.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:38 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:38.949+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-03-27T11:10:13.171475+0000 front 2024-03-27T11:10:13.171056+0000 (oldest deadline 2024-03-27T11:10:37.171547+0000) 2024-03-27T11:10:39.423 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:39 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:39.114+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.236297+0000 front 2024-03-27T11:09:28.236728+0000 (oldest deadline 2024-03-27T11:09:51.136525+0000) 2024-03-27T11:10:39.423 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:39 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:39.114+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.340023+0000 front 2024-03-27T11:09:36.340351+0000 (oldest deadline 2024-03-27T11:10:02.239659+0000) 2024-03-27T11:10:39.423 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:39 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:39.114+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:46.143084+0000 front 2024-03-27T11:09:46.143228+0000 (oldest deadline 2024-03-27T11:10:10.843246+0000) 2024-03-27T11:10:39.424 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:39 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:39.114+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.046124+0000 front 2024-03-27T11:09:56.046017+0000 (oldest deadline 2024-03-27T11:10:20.746204+0000) 2024-03-27T11:10:39.424 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:39 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:39.114+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:00.747126+0000 front 2024-03-27T11:10:00.747259+0000 (oldest deadline 2024-03-27T11:10:28.348502+0000) 2024-03-27T11:10:39.424 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:39 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:39.114+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-03-27T11:10:14.250568+0000 front 2024-03-27T11:10:14.250492+0000 (oldest deadline 2024-03-27T11:10:37.150604+0000) 2024-03-27T11:10:39.923 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:39 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:39.588+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.140407+0000 front 2024-03-27T11:09:28.140164+0000 (oldest deadline 2024-03-27T11:09:54.040233+0000) 2024-03-27T11:10:39.924 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:39 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:39.588+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.844416+0000 front 2024-03-27T11:09:36.843999+0000 (oldest deadline 2024-03-27T11:10:01.544084+0000) 2024-03-27T11:10:39.924 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:39 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:39.588+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:47.348157+0000 front 2024-03-27T11:09:47.348480+0000 (oldest deadline 2024-03-27T11:10:09.647966+0000) 2024-03-27T11:10:39.924 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:39 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:39.588+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.051047+0000 front 2024-03-27T11:09:56.050946+0000 (oldest deadline 2024-03-27T11:10:20.750821+0000) 2024-03-27T11:10:39.924 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:39 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:39.588+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:01.852819+0000 front 2024-03-27T11:10:01.852917+0000 (oldest deadline 2024-03-27T11:10:29.955550+0000) 2024-03-27T11:10:40.013 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:39 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:39.859+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.380254+0000 front 2024-03-27T11:09:23.380098+0000 (oldest deadline 2024-03-27T11:09:49.280089+0000) 2024-03-27T11:10:40.013 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:39 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:39.859+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:34.582052+0000 front 2024-03-27T11:09:34.582101+0000 (oldest deadline 2024-03-27T11:10:00.482194+0000) 2024-03-27T11:10:40.014 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:39 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:39.859+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:45.784185+0000 front 2024-03-27T11:09:45.784030+0000 (oldest deadline 2024-03-27T11:10:08.683959+0000) 2024-03-27T11:10:40.014 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:39 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:39.859+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:55.086370+0000 front 2024-03-27T11:09:55.086460+0000 (oldest deadline 2024-03-27T11:10:17.386639+0000) 2024-03-27T11:10:40.014 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:39 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:39.859+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:03.189572+0000 front 2024-03-27T11:10:03.189451+0000 (oldest deadline 2024-03-27T11:10:28.390279+0000) 2024-03-27T11:10:40.014 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:39 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:39.859+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-03-27T11:10:14.792988+0000 front 2024-03-27T11:10:14.793114+0000 (oldest deadline 2024-03-27T11:10:38.892796+0000) 2024-03-27T11:10:40.387 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:39 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:39.963+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.654238+0000 front 2024-03-27T11:09:23.654318+0000 (oldest deadline 2024-03-27T11:09:48.954440+0000) 2024-03-27T11:10:40.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:39 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:39.963+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:35.957512+0000 front 2024-03-27T11:09:35.957437+0000 (oldest deadline 2024-03-27T11:10:01.857676+0000) 2024-03-27T11:10:40.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:39 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:39.963+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:44.160107+0000 front 2024-03-27T11:09:44.160318+0000 (oldest deadline 2024-03-27T11:10:08.260147+0000) 2024-03-27T11:10:40.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:39 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:39.963+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.463226+0000 front 2024-03-27T11:09:56.463040+0000 (oldest deadline 2024-03-27T11:10:18.763354+0000) 2024-03-27T11:10:40.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:39 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:39.963+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:05.767047+0000 front 2024-03-27T11:10:05.766994+0000 (oldest deadline 2024-03-27T11:10:28.067070+0000) 2024-03-27T11:10:40.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:39 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:39.963+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-03-27T11:10:13.171475+0000 front 2024-03-27T11:10:13.171056+0000 (oldest deadline 2024-03-27T11:10:37.171547+0000) 2024-03-27T11:10:40.423 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:40 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:40.081+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.236297+0000 front 2024-03-27T11:09:28.236728+0000 (oldest deadline 2024-03-27T11:09:51.136525+0000) 2024-03-27T11:10:40.423 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:40 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:40.082+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.340023+0000 front 2024-03-27T11:09:36.340351+0000 (oldest deadline 2024-03-27T11:10:02.239659+0000) 2024-03-27T11:10:40.423 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:40 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:40.082+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:46.143084+0000 front 2024-03-27T11:09:46.143228+0000 (oldest deadline 2024-03-27T11:10:10.843246+0000) 2024-03-27T11:10:40.423 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:40 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:40.082+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.046124+0000 front 2024-03-27T11:09:56.046017+0000 (oldest deadline 2024-03-27T11:10:20.746204+0000) 2024-03-27T11:10:40.424 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:40 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:40.082+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:00.747126+0000 front 2024-03-27T11:10:00.747259+0000 (oldest deadline 2024-03-27T11:10:28.348502+0000) 2024-03-27T11:10:40.424 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:40 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:40.082+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-03-27T11:10:14.250568+0000 front 2024-03-27T11:10:14.250492+0000 (oldest deadline 2024-03-27T11:10:37.150604+0000) 2024-03-27T11:10:40.923 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:40 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:40.582+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.140407+0000 front 2024-03-27T11:09:28.140164+0000 (oldest deadline 2024-03-27T11:09:54.040233+0000) 2024-03-27T11:10:40.923 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:40 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:40.582+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.844416+0000 front 2024-03-27T11:09:36.843999+0000 (oldest deadline 2024-03-27T11:10:01.544084+0000) 2024-03-27T11:10:40.924 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:40 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:40.582+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:47.348157+0000 front 2024-03-27T11:09:47.348480+0000 (oldest deadline 2024-03-27T11:10:09.647966+0000) 2024-03-27T11:10:40.924 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:40 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:40.582+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.051047+0000 front 2024-03-27T11:09:56.050946+0000 (oldest deadline 2024-03-27T11:10:20.750821+0000) 2024-03-27T11:10:40.924 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:40 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:40.582+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:01.852819+0000 front 2024-03-27T11:10:01.852917+0000 (oldest deadline 2024-03-27T11:10:29.955550+0000) 2024-03-27T11:10:40.924 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:40 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:40.582+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-03-27T11:10:09.956841+0000 front 2024-03-27T11:10:09.956653+0000 (oldest deadline 2024-03-27T11:10:39.957986+0000) 2024-03-27T11:10:41.118 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:40 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:40.838+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.380254+0000 front 2024-03-27T11:09:23.380098+0000 (oldest deadline 2024-03-27T11:09:49.280089+0000) 2024-03-27T11:10:41.118 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:40 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:40.838+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:34.582052+0000 front 2024-03-27T11:09:34.582101+0000 (oldest deadline 2024-03-27T11:10:00.482194+0000) 2024-03-27T11:10:41.118 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:40 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:40.838+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:45.784185+0000 front 2024-03-27T11:09:45.784030+0000 (oldest deadline 2024-03-27T11:10:08.683959+0000) 2024-03-27T11:10:41.118 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:40 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:40.838+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:55.086370+0000 front 2024-03-27T11:09:55.086460+0000 (oldest deadline 2024-03-27T11:10:17.386639+0000) 2024-03-27T11:10:41.118 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:40 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:40.838+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:03.189572+0000 front 2024-03-27T11:10:03.189451+0000 (oldest deadline 2024-03-27T11:10:28.390279+0000) 2024-03-27T11:10:41.118 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:40 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:40.838+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-03-27T11:10:14.792988+0000 front 2024-03-27T11:10:14.793114+0000 (oldest deadline 2024-03-27T11:10:38.892796+0000) 2024-03-27T11:10:41.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:40 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:40.946+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.654238+0000 front 2024-03-27T11:09:23.654318+0000 (oldest deadline 2024-03-27T11:09:48.954440+0000) 2024-03-27T11:10:41.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:40 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:40.946+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:35.957512+0000 front 2024-03-27T11:09:35.957437+0000 (oldest deadline 2024-03-27T11:10:01.857676+0000) 2024-03-27T11:10:41.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:40 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:40.946+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:44.160107+0000 front 2024-03-27T11:09:44.160318+0000 (oldest deadline 2024-03-27T11:10:08.260147+0000) 2024-03-27T11:10:41.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:40 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:40.946+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.463226+0000 front 2024-03-27T11:09:56.463040+0000 (oldest deadline 2024-03-27T11:10:18.763354+0000) 2024-03-27T11:10:41.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:40 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:40.946+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:05.767047+0000 front 2024-03-27T11:10:05.766994+0000 (oldest deadline 2024-03-27T11:10:28.067070+0000) 2024-03-27T11:10:41.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:40 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:40.946+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-03-27T11:10:13.171475+0000 front 2024-03-27T11:10:13.171056+0000 (oldest deadline 2024-03-27T11:10:37.171547+0000) 2024-03-27T11:10:41.423 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:41 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:41.077+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.236297+0000 front 2024-03-27T11:09:28.236728+0000 (oldest deadline 2024-03-27T11:09:51.136525+0000) 2024-03-27T11:10:41.423 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:41 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:41.077+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.340023+0000 front 2024-03-27T11:09:36.340351+0000 (oldest deadline 2024-03-27T11:10:02.239659+0000) 2024-03-27T11:10:41.423 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:41 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:41.077+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:46.143084+0000 front 2024-03-27T11:09:46.143228+0000 (oldest deadline 2024-03-27T11:10:10.843246+0000) 2024-03-27T11:10:41.424 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:41 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:41.077+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.046124+0000 front 2024-03-27T11:09:56.046017+0000 (oldest deadline 2024-03-27T11:10:20.746204+0000) 2024-03-27T11:10:41.424 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:41 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:41.077+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:00.747126+0000 front 2024-03-27T11:10:00.747259+0000 (oldest deadline 2024-03-27T11:10:28.348502+0000) 2024-03-27T11:10:41.424 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:41 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:41.077+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-03-27T11:10:14.250568+0000 front 2024-03-27T11:10:14.250492+0000 (oldest deadline 2024-03-27T11:10:37.150604+0000) 2024-03-27T11:10:41.923 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:41 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:41.606+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.140407+0000 front 2024-03-27T11:09:28.140164+0000 (oldest deadline 2024-03-27T11:09:54.040233+0000) 2024-03-27T11:10:41.923 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:41 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:41.606+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.844416+0000 front 2024-03-27T11:09:36.843999+0000 (oldest deadline 2024-03-27T11:10:01.544084+0000) 2024-03-27T11:10:41.923 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:41 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:41.606+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:47.348157+0000 front 2024-03-27T11:09:47.348480+0000 (oldest deadline 2024-03-27T11:10:09.647966+0000) 2024-03-27T11:10:41.924 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:41 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:41.606+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.051047+0000 front 2024-03-27T11:09:56.050946+0000 (oldest deadline 2024-03-27T11:10:20.750821+0000) 2024-03-27T11:10:41.924 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:41 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:41.606+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:01.852819+0000 front 2024-03-27T11:10:01.852917+0000 (oldest deadline 2024-03-27T11:10:29.955550+0000) 2024-03-27T11:10:41.924 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:41 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:41.606+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-03-27T11:10:09.956841+0000 front 2024-03-27T11:10:09.956653+0000 (oldest deadline 2024-03-27T11:10:39.957986+0000) 2024-03-27T11:10:42.264 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:41 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:41.852+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.380254+0000 front 2024-03-27T11:09:23.380098+0000 (oldest deadline 2024-03-27T11:09:49.280089+0000) 2024-03-27T11:10:42.264 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:41 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:41.852+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:34.582052+0000 front 2024-03-27T11:09:34.582101+0000 (oldest deadline 2024-03-27T11:10:00.482194+0000) 2024-03-27T11:10:42.264 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:41 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:41.852+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:45.784185+0000 front 2024-03-27T11:09:45.784030+0000 (oldest deadline 2024-03-27T11:10:08.683959+0000) 2024-03-27T11:10:42.264 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:41 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:41.852+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:55.086370+0000 front 2024-03-27T11:09:55.086460+0000 (oldest deadline 2024-03-27T11:10:17.386639+0000) 2024-03-27T11:10:42.264 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:41 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:41.852+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:03.189572+0000 front 2024-03-27T11:10:03.189451+0000 (oldest deadline 2024-03-27T11:10:28.390279+0000) 2024-03-27T11:10:42.264 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:41 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:41.852+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-03-27T11:10:14.792988+0000 front 2024-03-27T11:10:14.793114+0000 (oldest deadline 2024-03-27T11:10:38.892796+0000) 2024-03-27T11:10:42.387 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:41 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:41.972+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.654238+0000 front 2024-03-27T11:09:23.654318+0000 (oldest deadline 2024-03-27T11:09:48.954440+0000) 2024-03-27T11:10:42.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:41 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:41.972+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:35.957512+0000 front 2024-03-27T11:09:35.957437+0000 (oldest deadline 2024-03-27T11:10:01.857676+0000) 2024-03-27T11:10:42.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:41 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:41.972+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:44.160107+0000 front 2024-03-27T11:09:44.160318+0000 (oldest deadline 2024-03-27T11:10:08.260147+0000) 2024-03-27T11:10:42.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:41 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:41.973+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.463226+0000 front 2024-03-27T11:09:56.463040+0000 (oldest deadline 2024-03-27T11:10:18.763354+0000) 2024-03-27T11:10:42.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:41 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:41.973+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:05.767047+0000 front 2024-03-27T11:10:05.766994+0000 (oldest deadline 2024-03-27T11:10:28.067070+0000) 2024-03-27T11:10:42.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:41 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:41.973+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-03-27T11:10:13.171475+0000 front 2024-03-27T11:10:13.171056+0000 (oldest deadline 2024-03-27T11:10:37.171547+0000) 2024-03-27T11:10:42.423 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:42 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:42.080+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.236297+0000 front 2024-03-27T11:09:28.236728+0000 (oldest deadline 2024-03-27T11:09:51.136525+0000) 2024-03-27T11:10:42.423 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:42 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:42.080+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.340023+0000 front 2024-03-27T11:09:36.340351+0000 (oldest deadline 2024-03-27T11:10:02.239659+0000) 2024-03-27T11:10:42.423 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:42 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:42.080+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:46.143084+0000 front 2024-03-27T11:09:46.143228+0000 (oldest deadline 2024-03-27T11:10:10.843246+0000) 2024-03-27T11:10:42.423 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:42 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:42.080+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.046124+0000 front 2024-03-27T11:09:56.046017+0000 (oldest deadline 2024-03-27T11:10:20.746204+0000) 2024-03-27T11:10:42.423 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:42 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:42.080+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:00.747126+0000 front 2024-03-27T11:10:00.747259+0000 (oldest deadline 2024-03-27T11:10:28.348502+0000) 2024-03-27T11:10:42.424 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:42 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:42.080+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-03-27T11:10:14.250568+0000 front 2024-03-27T11:10:14.250492+0000 (oldest deadline 2024-03-27T11:10:37.150604+0000) 2024-03-27T11:10:42.923 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:42 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:42.585+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.140407+0000 front 2024-03-27T11:09:28.140164+0000 (oldest deadline 2024-03-27T11:09:54.040233+0000) 2024-03-27T11:10:42.923 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:42 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:42.585+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.844416+0000 front 2024-03-27T11:09:36.843999+0000 (oldest deadline 2024-03-27T11:10:01.544084+0000) 2024-03-27T11:10:42.923 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:42 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:42.585+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:47.348157+0000 front 2024-03-27T11:09:47.348480+0000 (oldest deadline 2024-03-27T11:10:09.647966+0000) 2024-03-27T11:10:42.923 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:42 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:42.585+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.051047+0000 front 2024-03-27T11:09:56.050946+0000 (oldest deadline 2024-03-27T11:10:20.750821+0000) 2024-03-27T11:10:42.923 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:42 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:42.585+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:01.852819+0000 front 2024-03-27T11:10:01.852917+0000 (oldest deadline 2024-03-27T11:10:29.955550+0000) 2024-03-27T11:10:42.924 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:42 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:42.585+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-03-27T11:10:09.956841+0000 front 2024-03-27T11:10:09.956653+0000 (oldest deadline 2024-03-27T11:10:39.957986+0000) 2024-03-27T11:10:43.263 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:42 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:42.850+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.380254+0000 front 2024-03-27T11:09:23.380098+0000 (oldest deadline 2024-03-27T11:09:49.280089+0000) 2024-03-27T11:10:43.264 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:42 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:42.850+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:34.582052+0000 front 2024-03-27T11:09:34.582101+0000 (oldest deadline 2024-03-27T11:10:00.482194+0000) 2024-03-27T11:10:43.264 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:42 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:42.850+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:45.784185+0000 front 2024-03-27T11:09:45.784030+0000 (oldest deadline 2024-03-27T11:10:08.683959+0000) 2024-03-27T11:10:43.264 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:42 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:42.850+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:55.086370+0000 front 2024-03-27T11:09:55.086460+0000 (oldest deadline 2024-03-27T11:10:17.386639+0000) 2024-03-27T11:10:43.264 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:42 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:42.850+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:03.189572+0000 front 2024-03-27T11:10:03.189451+0000 (oldest deadline 2024-03-27T11:10:28.390279+0000) 2024-03-27T11:10:43.264 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:42 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:42.850+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-03-27T11:10:14.792988+0000 front 2024-03-27T11:10:14.793114+0000 (oldest deadline 2024-03-27T11:10:38.892796+0000) 2024-03-27T11:10:43.387 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:42 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:42.973+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.654238+0000 front 2024-03-27T11:09:23.654318+0000 (oldest deadline 2024-03-27T11:09:48.954440+0000) 2024-03-27T11:10:43.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:42 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:42.973+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:35.957512+0000 front 2024-03-27T11:09:35.957437+0000 (oldest deadline 2024-03-27T11:10:01.857676+0000) 2024-03-27T11:10:43.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:42 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:42.973+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:44.160107+0000 front 2024-03-27T11:09:44.160318+0000 (oldest deadline 2024-03-27T11:10:08.260147+0000) 2024-03-27T11:10:43.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:42 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:42.973+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.463226+0000 front 2024-03-27T11:09:56.463040+0000 (oldest deadline 2024-03-27T11:10:18.763354+0000) 2024-03-27T11:10:43.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:42 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:42.973+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:05.767047+0000 front 2024-03-27T11:10:05.766994+0000 (oldest deadline 2024-03-27T11:10:28.067070+0000) 2024-03-27T11:10:43.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:42 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:42.973+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-03-27T11:10:13.171475+0000 front 2024-03-27T11:10:13.171056+0000 (oldest deadline 2024-03-27T11:10:37.171547+0000) 2024-03-27T11:10:43.423 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:43 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:43.068+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.236297+0000 front 2024-03-27T11:09:28.236728+0000 (oldest deadline 2024-03-27T11:09:51.136525+0000) 2024-03-27T11:10:43.423 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:43 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:43.068+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.340023+0000 front 2024-03-27T11:09:36.340351+0000 (oldest deadline 2024-03-27T11:10:02.239659+0000) 2024-03-27T11:10:43.423 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:43 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:43.068+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:46.143084+0000 front 2024-03-27T11:09:46.143228+0000 (oldest deadline 2024-03-27T11:10:10.843246+0000) 2024-03-27T11:10:43.423 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:43 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:43.068+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.046124+0000 front 2024-03-27T11:09:56.046017+0000 (oldest deadline 2024-03-27T11:10:20.746204+0000) 2024-03-27T11:10:43.423 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:43 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:43.068+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:00.747126+0000 front 2024-03-27T11:10:00.747259+0000 (oldest deadline 2024-03-27T11:10:28.348502+0000) 2024-03-27T11:10:43.424 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:43 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:43.068+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-03-27T11:10:14.250568+0000 front 2024-03-27T11:10:14.250492+0000 (oldest deadline 2024-03-27T11:10:37.150604+0000) 2024-03-27T11:10:43.923 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:43 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:43.589+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.140407+0000 front 2024-03-27T11:09:28.140164+0000 (oldest deadline 2024-03-27T11:09:54.040233+0000) 2024-03-27T11:10:43.923 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:43 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:43.589+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.844416+0000 front 2024-03-27T11:09:36.843999+0000 (oldest deadline 2024-03-27T11:10:01.544084+0000) 2024-03-27T11:10:43.923 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:43 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:43.589+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:47.348157+0000 front 2024-03-27T11:09:47.348480+0000 (oldest deadline 2024-03-27T11:10:09.647966+0000) 2024-03-27T11:10:43.924 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:43 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:43.589+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.051047+0000 front 2024-03-27T11:09:56.050946+0000 (oldest deadline 2024-03-27T11:10:20.750821+0000) 2024-03-27T11:10:43.924 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:43 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:43.589+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:01.852819+0000 front 2024-03-27T11:10:01.852917+0000 (oldest deadline 2024-03-27T11:10:29.955550+0000) 2024-03-27T11:10:43.924 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:43 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8[54176]: 2024-03-27T11:10:43.589+0000 7f06a1400700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-03-27T11:10:09.956841+0000 front 2024-03-27T11:10:09.956653+0000 (oldest deadline 2024-03-27T11:10:39.957986+0000) 2024-03-27T11:10:44.263 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:43 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:43.895+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.380254+0000 front 2024-03-27T11:09:23.380098+0000 (oldest deadline 2024-03-27T11:09:49.280089+0000) 2024-03-27T11:10:44.263 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:43 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:43.895+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:34.582052+0000 front 2024-03-27T11:09:34.582101+0000 (oldest deadline 2024-03-27T11:10:00.482194+0000) 2024-03-27T11:10:44.263 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:43 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:43.895+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:45.784185+0000 front 2024-03-27T11:09:45.784030+0000 (oldest deadline 2024-03-27T11:10:08.683959+0000) 2024-03-27T11:10:44.263 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:43 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:43.895+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:55.086370+0000 front 2024-03-27T11:09:55.086460+0000 (oldest deadline 2024-03-27T11:10:17.386639+0000) 2024-03-27T11:10:44.263 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:43 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:43.895+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:03.189572+0000 front 2024-03-27T11:10:03.189451+0000 (oldest deadline 2024-03-27T11:10:28.390279+0000) 2024-03-27T11:10:44.263 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:43 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:43.895+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-03-27T11:10:14.792988+0000 front 2024-03-27T11:10:14.793114+0000 (oldest deadline 2024-03-27T11:10:38.892796+0000) 2024-03-27T11:10:44.387 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:44 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:44.003+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.654238+0000 front 2024-03-27T11:09:23.654318+0000 (oldest deadline 2024-03-27T11:09:48.954440+0000) 2024-03-27T11:10:44.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:44 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:44.003+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:35.957512+0000 front 2024-03-27T11:09:35.957437+0000 (oldest deadline 2024-03-27T11:10:01.857676+0000) 2024-03-27T11:10:44.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:44 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:44.003+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:44.160107+0000 front 2024-03-27T11:09:44.160318+0000 (oldest deadline 2024-03-27T11:10:08.260147+0000) 2024-03-27T11:10:44.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:44 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:44.003+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.463226+0000 front 2024-03-27T11:09:56.463040+0000 (oldest deadline 2024-03-27T11:10:18.763354+0000) 2024-03-27T11:10:44.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:44 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:44.003+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:05.767047+0000 front 2024-03-27T11:10:05.766994+0000 (oldest deadline 2024-03-27T11:10:28.067070+0000) 2024-03-27T11:10:44.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:44 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:44.003+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-03-27T11:10:13.171475+0000 front 2024-03-27T11:10:13.171056+0000 (oldest deadline 2024-03-27T11:10:37.171547+0000) 2024-03-27T11:10:44.415 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:44 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:44.056+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.236297+0000 front 2024-03-27T11:09:28.236728+0000 (oldest deadline 2024-03-27T11:09:51.136525+0000) 2024-03-27T11:10:44.415 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:44 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:44.056+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.340023+0000 front 2024-03-27T11:09:36.340351+0000 (oldest deadline 2024-03-27T11:10:02.239659+0000) 2024-03-27T11:10:44.415 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:44 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:44.056+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:46.143084+0000 front 2024-03-27T11:09:46.143228+0000 (oldest deadline 2024-03-27T11:10:10.843246+0000) 2024-03-27T11:10:44.415 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:44 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:44.056+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.046124+0000 front 2024-03-27T11:09:56.046017+0000 (oldest deadline 2024-03-27T11:10:20.746204+0000) 2024-03-27T11:10:44.415 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:44 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:44.056+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:00.747126+0000 front 2024-03-27T11:10:00.747259+0000 (oldest deadline 2024-03-27T11:10:28.348502+0000) 2024-03-27T11:10:44.415 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:44 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:44.056+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-03-27T11:10:14.250568+0000 front 2024-03-27T11:10:14.250492+0000 (oldest deadline 2024-03-27T11:10:37.150604+0000) 2024-03-27T11:10:44.673 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:44 smithi177 podman[71931]: 2024-03-27 11:10:44.44682475 +0000 UTC m=+5.726252874 container died 80d92c6617869447f8363c75554d3d54bfa25f29febe1c84bbc8a4f591c68b93 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326) 2024-03-27T11:10:45.018 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:44 smithi177 podman[71931]: 2024-03-27 11:10:44.874172092 +0000 UTC m=+6.153600245 container cleanup 80d92c6617869447f8363c75554d3d54bfa25f29febe1c84bbc8a4f591c68b93 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8, RELEASE=wip-rishabh-testing-20240326.131558-155268c, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS) 2024-03-27T11:10:45.018 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:44 smithi177 bash[71931]: ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8 2024-03-27T11:10:45.263 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:44 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:44.937+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.380254+0000 front 2024-03-27T11:09:23.380098+0000 (oldest deadline 2024-03-27T11:09:49.280089+0000) 2024-03-27T11:10:45.264 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:44 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:44.937+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:34.582052+0000 front 2024-03-27T11:09:34.582101+0000 (oldest deadline 2024-03-27T11:10:00.482194+0000) 2024-03-27T11:10:45.264 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:44 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:44.937+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:45.784185+0000 front 2024-03-27T11:09:45.784030+0000 (oldest deadline 2024-03-27T11:10:08.683959+0000) 2024-03-27T11:10:45.264 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:44 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:44.937+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:55.086370+0000 front 2024-03-27T11:09:55.086460+0000 (oldest deadline 2024-03-27T11:10:17.386639+0000) 2024-03-27T11:10:45.264 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:44 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:44.937+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:03.189572+0000 front 2024-03-27T11:10:03.189451+0000 (oldest deadline 2024-03-27T11:10:28.390279+0000) 2024-03-27T11:10:45.264 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:44 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:44.937+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-03-27T11:10:14.792988+0000 front 2024-03-27T11:10:14.793114+0000 (oldest deadline 2024-03-27T11:10:38.892796+0000) 2024-03-27T11:10:45.387 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:44 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:44.982+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.654238+0000 front 2024-03-27T11:09:23.654318+0000 (oldest deadline 2024-03-27T11:09:48.954440+0000) 2024-03-27T11:10:45.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:44 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:44.982+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:35.957512+0000 front 2024-03-27T11:09:35.957437+0000 (oldest deadline 2024-03-27T11:10:01.857676+0000) 2024-03-27T11:10:45.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:44 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:44.982+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:44.160107+0000 front 2024-03-27T11:09:44.160318+0000 (oldest deadline 2024-03-27T11:10:08.260147+0000) 2024-03-27T11:10:45.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:44 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:44.982+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.463226+0000 front 2024-03-27T11:09:56.463040+0000 (oldest deadline 2024-03-27T11:10:18.763354+0000) 2024-03-27T11:10:45.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:44 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:44.982+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:05.767047+0000 front 2024-03-27T11:10:05.766994+0000 (oldest deadline 2024-03-27T11:10:28.067070+0000) 2024-03-27T11:10:45.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:44 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:44.982+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-03-27T11:10:13.171475+0000 front 2024-03-27T11:10:13.171056+0000 (oldest deadline 2024-03-27T11:10:37.171547+0000) 2024-03-27T11:10:45.423 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:45 smithi177 podman[71949]: 2024-03-27 11:10:45.15871772 +0000 UTC m=+0.696099658 container remove 80d92c6617869447f8363c75554d3d54bfa25f29febe1c84bbc8a4f591c68b93 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240326.131558-155268c, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-27T11:10:45.423 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:45 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:45.049+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.236297+0000 front 2024-03-27T11:09:28.236728+0000 (oldest deadline 2024-03-27T11:09:51.136525+0000) 2024-03-27T11:10:45.423 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:45 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:45.049+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.340023+0000 front 2024-03-27T11:09:36.340351+0000 (oldest deadline 2024-03-27T11:10:02.239659+0000) 2024-03-27T11:10:45.423 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:45 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:45.049+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:46.143084+0000 front 2024-03-27T11:09:46.143228+0000 (oldest deadline 2024-03-27T11:10:10.843246+0000) 2024-03-27T11:10:45.423 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:45 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:45.049+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.046124+0000 front 2024-03-27T11:09:56.046017+0000 (oldest deadline 2024-03-27T11:10:20.746204+0000) 2024-03-27T11:10:45.424 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:45 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:45.049+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:00.747126+0000 front 2024-03-27T11:10:00.747259+0000 (oldest deadline 2024-03-27T11:10:28.348502+0000) 2024-03-27T11:10:45.424 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:45 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:45.049+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-03-27T11:10:14.250568+0000 front 2024-03-27T11:10:14.250492+0000 (oldest deadline 2024-03-27T11:10:37.150604+0000) 2024-03-27T11:10:46.173 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:45 smithi177 podman[72038]: 2024-03-27 11:10:45.834805849 +0000 UTC m=+0.033518098 image pull b379bbd8da6b5d0fa9a3523941efac213b6815e1cbf05c6507954f0a3fe7a801 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772 2024-03-27T11:10:46.173 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:45 smithi177 podman[72038]: 2024-03-27 11:10:45.934626178 +0000 UTC m=+0.133338457 container create c05743acac6e2d6d478b9b942c3bdc3d3862f2602d04ceae9e0ecd69fcf634fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8-deactivate, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-03-27T11:10:46.173 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:46 smithi177 podman[72038]: 2024-03-27 11:10:46.130307724 +0000 UTC m=+0.329020013 container init c05743acac6e2d6d478b9b942c3bdc3d3862f2602d04ceae9e0ecd69fcf634fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240326, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-03-27T11:10:46.173 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:46 smithi177 podman[72038]: 2024-03-27 11:10:46.143767552 +0000 UTC m=+0.342479820 container start c05743acac6e2d6d478b9b942c3bdc3d3862f2602d04ceae9e0ecd69fcf634fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.schema-version=1.0, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e) 2024-03-27T11:10:46.174 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:46 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:46.098+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.236297+0000 front 2024-03-27T11:09:28.236728+0000 (oldest deadline 2024-03-27T11:09:51.136525+0000) 2024-03-27T11:10:46.174 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:46 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:46.099+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.340023+0000 front 2024-03-27T11:09:36.340351+0000 (oldest deadline 2024-03-27T11:10:02.239659+0000) 2024-03-27T11:10:46.174 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:46 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:46.099+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:46.143084+0000 front 2024-03-27T11:09:46.143228+0000 (oldest deadline 2024-03-27T11:10:10.843246+0000) 2024-03-27T11:10:46.174 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:46 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:46.099+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.046124+0000 front 2024-03-27T11:09:56.046017+0000 (oldest deadline 2024-03-27T11:10:20.746204+0000) 2024-03-27T11:10:46.174 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:46 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:46.099+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:00.747126+0000 front 2024-03-27T11:10:00.747259+0000 (oldest deadline 2024-03-27T11:10:28.348502+0000) 2024-03-27T11:10:46.174 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:46 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:46.099+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-03-27T11:10:14.250568+0000 front 2024-03-27T11:10:14.250492+0000 (oldest deadline 2024-03-27T11:10:37.150604+0000) 2024-03-27T11:10:46.263 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:45 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:45.897+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.380254+0000 front 2024-03-27T11:09:23.380098+0000 (oldest deadline 2024-03-27T11:09:49.280089+0000) 2024-03-27T11:10:46.263 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:45 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:45.897+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:34.582052+0000 front 2024-03-27T11:09:34.582101+0000 (oldest deadline 2024-03-27T11:10:00.482194+0000) 2024-03-27T11:10:46.264 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:45 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:45.897+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:45.784185+0000 front 2024-03-27T11:09:45.784030+0000 (oldest deadline 2024-03-27T11:10:08.683959+0000) 2024-03-27T11:10:46.264 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:45 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:45.897+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:55.086370+0000 front 2024-03-27T11:09:55.086460+0000 (oldest deadline 2024-03-27T11:10:17.386639+0000) 2024-03-27T11:10:46.264 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:45 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:45.897+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:03.189572+0000 front 2024-03-27T11:10:03.189451+0000 (oldest deadline 2024-03-27T11:10:28.390279+0000) 2024-03-27T11:10:46.264 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:45 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:45.897+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-03-27T11:10:14.792988+0000 front 2024-03-27T11:10:14.793114+0000 (oldest deadline 2024-03-27T11:10:38.892796+0000) 2024-03-27T11:10:46.387 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:45 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:45.986+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.654238+0000 front 2024-03-27T11:09:23.654318+0000 (oldest deadline 2024-03-27T11:09:48.954440+0000) 2024-03-27T11:10:46.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:45 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:45.986+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:35.957512+0000 front 2024-03-27T11:09:35.957437+0000 (oldest deadline 2024-03-27T11:10:01.857676+0000) 2024-03-27T11:10:46.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:45 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:45.986+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:44.160107+0000 front 2024-03-27T11:09:44.160318+0000 (oldest deadline 2024-03-27T11:10:08.260147+0000) 2024-03-27T11:10:46.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:45 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:45.986+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.463226+0000 front 2024-03-27T11:09:56.463040+0000 (oldest deadline 2024-03-27T11:10:18.763354+0000) 2024-03-27T11:10:46.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:45 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:45.986+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:05.767047+0000 front 2024-03-27T11:10:05.766994+0000 (oldest deadline 2024-03-27T11:10:28.067070+0000) 2024-03-27T11:10:46.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:45 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:45.986+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-03-27T11:10:13.171475+0000 front 2024-03-27T11:10:13.171056+0000 (oldest deadline 2024-03-27T11:10:37.171547+0000) 2024-03-27T11:10:46.534 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:46 smithi177 podman[72038]: 2024-03-27 11:10:46.220200882 +0000 UTC m=+0.418913145 container attach c05743acac6e2d6d478b9b942c3bdc3d3862f2602d04ceae9e0ecd69fcf634fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240326, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-27T11:10:46.534 INFO:journalctl@ceph.osd.8.smithi177.stdout:Mar 27 11:10:46 smithi177 podman[72038]: 2024-03-27 11:10:46.566110102 +0000 UTC m=+0.764822359 container died c05743acac6e2d6d478b9b942c3bdc3d3862f2602d04ceae9e0ecd69fcf634fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-8-deactivate, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-03-27T11:10:47.173 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:47 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:47.104+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.236297+0000 front 2024-03-27T11:09:28.236728+0000 (oldest deadline 2024-03-27T11:09:51.136525+0000) 2024-03-27T11:10:47.173 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:47 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:47.105+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.340023+0000 front 2024-03-27T11:09:36.340351+0000 (oldest deadline 2024-03-27T11:10:02.239659+0000) 2024-03-27T11:10:47.173 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:47 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:47.105+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:46.143084+0000 front 2024-03-27T11:09:46.143228+0000 (oldest deadline 2024-03-27T11:10:10.843246+0000) 2024-03-27T11:10:47.173 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:47 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:47.105+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.046124+0000 front 2024-03-27T11:09:56.046017+0000 (oldest deadline 2024-03-27T11:10:20.746204+0000) 2024-03-27T11:10:47.174 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:47 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:47.105+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:00.747126+0000 front 2024-03-27T11:10:00.747259+0000 (oldest deadline 2024-03-27T11:10:28.348502+0000) 2024-03-27T11:10:47.174 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:47 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:47.105+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-03-27T11:10:14.250568+0000 front 2024-03-27T11:10:14.250492+0000 (oldest deadline 2024-03-27T11:10:37.150604+0000) 2024-03-27T11:10:47.236 DEBUG:teuthology.orchestra.run.smithi177:> sudo pkill -f 'journalctl -f -n 0 -u ceph-adca88c4-ec26-11ee-95d0-87774f69a715@osd.8.service' 2024-03-27T11:10:47.263 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:46 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:46.938+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.380254+0000 front 2024-03-27T11:09:23.380098+0000 (oldest deadline 2024-03-27T11:09:49.280089+0000) 2024-03-27T11:10:47.263 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:46 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:46.938+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:34.582052+0000 front 2024-03-27T11:09:34.582101+0000 (oldest deadline 2024-03-27T11:10:00.482194+0000) 2024-03-27T11:10:47.263 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:46 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:46.938+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:45.784185+0000 front 2024-03-27T11:09:45.784030+0000 (oldest deadline 2024-03-27T11:10:08.683959+0000) 2024-03-27T11:10:47.263 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:46 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:46.938+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:55.086370+0000 front 2024-03-27T11:09:55.086460+0000 (oldest deadline 2024-03-27T11:10:17.386639+0000) 2024-03-27T11:10:47.263 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:46 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:46.938+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:03.189572+0000 front 2024-03-27T11:10:03.189451+0000 (oldest deadline 2024-03-27T11:10:28.390279+0000) 2024-03-27T11:10:47.264 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:46 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:46.938+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-03-27T11:10:14.792988+0000 front 2024-03-27T11:10:14.793114+0000 (oldest deadline 2024-03-27T11:10:38.892796+0000) 2024-03-27T11:10:47.352 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-27T11:10:47.352 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-03-27T11:10:47.352 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-03-27T11:10:47.353 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-adca88c4-ec26-11ee-95d0-87774f69a715@osd.9 2024-03-27T11:10:47.387 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:46 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:46.966+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.654238+0000 front 2024-03-27T11:09:23.654318+0000 (oldest deadline 2024-03-27T11:09:48.954440+0000) 2024-03-27T11:10:47.387 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:46 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:46.966+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:35.957512+0000 front 2024-03-27T11:09:35.957437+0000 (oldest deadline 2024-03-27T11:10:01.857676+0000) 2024-03-27T11:10:47.387 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:46 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:46.966+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:44.160107+0000 front 2024-03-27T11:09:44.160318+0000 (oldest deadline 2024-03-27T11:10:08.260147+0000) 2024-03-27T11:10:47.387 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:46 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:46.966+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.463226+0000 front 2024-03-27T11:09:56.463040+0000 (oldest deadline 2024-03-27T11:10:18.763354+0000) 2024-03-27T11:10:47.387 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:46 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:46.966+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:05.767047+0000 front 2024-03-27T11:10:05.766994+0000 (oldest deadline 2024-03-27T11:10:28.067070+0000) 2024-03-27T11:10:47.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:46 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:46.966+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-03-27T11:10:13.171475+0000 front 2024-03-27T11:10:13.171056+0000 (oldest deadline 2024-03-27T11:10:37.171547+0000) 2024-03-27T11:10:47.887 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:47 smithi097 systemd[1]: Stopping Ceph osd.9 for adca88c4-ec26-11ee-95d0-87774f69a715... 2024-03-27T11:10:48.263 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:47 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:47.900+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.380254+0000 front 2024-03-27T11:09:23.380098+0000 (oldest deadline 2024-03-27T11:09:49.280089+0000) 2024-03-27T11:10:48.263 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:47 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:47.900+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:34.582052+0000 front 2024-03-27T11:09:34.582101+0000 (oldest deadline 2024-03-27T11:10:00.482194+0000) 2024-03-27T11:10:48.263 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:47 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:47.900+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:45.784185+0000 front 2024-03-27T11:09:45.784030+0000 (oldest deadline 2024-03-27T11:10:08.683959+0000) 2024-03-27T11:10:48.263 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:47 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:47.900+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:55.086370+0000 front 2024-03-27T11:09:55.086460+0000 (oldest deadline 2024-03-27T11:10:17.386639+0000) 2024-03-27T11:10:48.264 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:47 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:47.900+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:03.189572+0000 front 2024-03-27T11:10:03.189451+0000 (oldest deadline 2024-03-27T11:10:28.390279+0000) 2024-03-27T11:10:48.264 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:47 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:47.900+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-03-27T11:10:14.792988+0000 front 2024-03-27T11:10:14.793114+0000 (oldest deadline 2024-03-27T11:10:38.892796+0000) 2024-03-27T11:10:48.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:48 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:48.007+0000 7fddcba00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-27T11:10:48.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:48 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:48.007+0000 7fddcba00700 -1 osd.9 92 *** Got signal Terminated *** 2024-03-27T11:10:48.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:48 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:48.007+0000 7fddcba00700 -1 osd.9 92 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-27T11:10:48.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:48 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:48.010+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.654238+0000 front 2024-03-27T11:09:23.654318+0000 (oldest deadline 2024-03-27T11:09:48.954440+0000) 2024-03-27T11:10:48.389 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:48 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:48.010+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:35.957512+0000 front 2024-03-27T11:09:35.957437+0000 (oldest deadline 2024-03-27T11:10:01.857676+0000) 2024-03-27T11:10:48.389 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:48 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:48.010+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:44.160107+0000 front 2024-03-27T11:09:44.160318+0000 (oldest deadline 2024-03-27T11:10:08.260147+0000) 2024-03-27T11:10:48.389 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:48 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:48.010+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.463226+0000 front 2024-03-27T11:09:56.463040+0000 (oldest deadline 2024-03-27T11:10:18.763354+0000) 2024-03-27T11:10:48.389 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:48 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:48.010+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:05.767047+0000 front 2024-03-27T11:10:05.766994+0000 (oldest deadline 2024-03-27T11:10:28.067070+0000) 2024-03-27T11:10:48.389 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:48 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:48.010+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-03-27T11:10:13.171475+0000 front 2024-03-27T11:10:13.171056+0000 (oldest deadline 2024-03-27T11:10:37.171547+0000) 2024-03-27T11:10:48.424 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:48 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:48.061+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.236297+0000 front 2024-03-27T11:09:28.236728+0000 (oldest deadline 2024-03-27T11:09:51.136525+0000) 2024-03-27T11:10:48.424 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:48 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:48.061+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.340023+0000 front 2024-03-27T11:09:36.340351+0000 (oldest deadline 2024-03-27T11:10:02.239659+0000) 2024-03-27T11:10:48.424 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:48 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:48.061+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:46.143084+0000 front 2024-03-27T11:09:46.143228+0000 (oldest deadline 2024-03-27T11:10:10.843246+0000) 2024-03-27T11:10:48.424 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:48 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:48.061+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.046124+0000 front 2024-03-27T11:09:56.046017+0000 (oldest deadline 2024-03-27T11:10:20.746204+0000) 2024-03-27T11:10:48.424 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:48 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:48.061+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:00.747126+0000 front 2024-03-27T11:10:00.747259+0000 (oldest deadline 2024-03-27T11:10:28.348502+0000) 2024-03-27T11:10:48.424 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:48 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:48.061+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-03-27T11:10:14.250568+0000 front 2024-03-27T11:10:14.250492+0000 (oldest deadline 2024-03-27T11:10:37.150604+0000) 2024-03-27T11:10:49.263 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:48 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:48.864+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.380254+0000 front 2024-03-27T11:09:23.380098+0000 (oldest deadline 2024-03-27T11:09:49.280089+0000) 2024-03-27T11:10:49.263 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:48 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:48.864+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:34.582052+0000 front 2024-03-27T11:09:34.582101+0000 (oldest deadline 2024-03-27T11:10:00.482194+0000) 2024-03-27T11:10:49.264 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:48 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:48.864+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:45.784185+0000 front 2024-03-27T11:09:45.784030+0000 (oldest deadline 2024-03-27T11:10:08.683959+0000) 2024-03-27T11:10:49.264 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:48 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:48.864+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:55.086370+0000 front 2024-03-27T11:09:55.086460+0000 (oldest deadline 2024-03-27T11:10:17.386639+0000) 2024-03-27T11:10:49.264 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:48 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:48.864+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:03.189572+0000 front 2024-03-27T11:10:03.189451+0000 (oldest deadline 2024-03-27T11:10:28.390279+0000) 2024-03-27T11:10:49.264 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:48 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:48.864+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-03-27T11:10:14.792988+0000 front 2024-03-27T11:10:14.793114+0000 (oldest deadline 2024-03-27T11:10:38.892796+0000) 2024-03-27T11:10:49.264 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:48 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:48.864+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6822 osd.6 since back 2024-03-27T11:10:24.695220+0000 front 2024-03-27T11:10:24.695490+0000 (oldest deadline 2024-03-27T11:10:48.597165+0000) 2024-03-27T11:10:49.293 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:49 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:49.023+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.654238+0000 front 2024-03-27T11:09:23.654318+0000 (oldest deadline 2024-03-27T11:09:48.954440+0000) 2024-03-27T11:10:49.293 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:49 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:49.023+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:35.957512+0000 front 2024-03-27T11:09:35.957437+0000 (oldest deadline 2024-03-27T11:10:01.857676+0000) 2024-03-27T11:10:49.293 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:49 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:49.023+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:44.160107+0000 front 2024-03-27T11:09:44.160318+0000 (oldest deadline 2024-03-27T11:10:08.260147+0000) 2024-03-27T11:10:49.293 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:49 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:49.023+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.463226+0000 front 2024-03-27T11:09:56.463040+0000 (oldest deadline 2024-03-27T11:10:18.763354+0000) 2024-03-27T11:10:49.293 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:49 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:49.023+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:05.767047+0000 front 2024-03-27T11:10:05.766994+0000 (oldest deadline 2024-03-27T11:10:28.067070+0000) 2024-03-27T11:10:49.293 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:49 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:49.023+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-03-27T11:10:13.171475+0000 front 2024-03-27T11:10:13.171056+0000 (oldest deadline 2024-03-27T11:10:37.171547+0000) 2024-03-27T11:10:49.293 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:49 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:49.023+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.97:6822 osd.6 since back 2024-03-27T11:10:24.675573+0000 front 2024-03-27T11:10:24.675618+0000 (oldest deadline 2024-03-27T11:10:48.175737+0000) 2024-03-27T11:10:49.423 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:49 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:49.032+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.236297+0000 front 2024-03-27T11:09:28.236728+0000 (oldest deadline 2024-03-27T11:09:51.136525+0000) 2024-03-27T11:10:49.423 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:49 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:49.032+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.340023+0000 front 2024-03-27T11:09:36.340351+0000 (oldest deadline 2024-03-27T11:10:02.239659+0000) 2024-03-27T11:10:49.423 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:49 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:49.032+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:46.143084+0000 front 2024-03-27T11:09:46.143228+0000 (oldest deadline 2024-03-27T11:10:10.843246+0000) 2024-03-27T11:10:49.423 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:49 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:49.032+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.046124+0000 front 2024-03-27T11:09:56.046017+0000 (oldest deadline 2024-03-27T11:10:20.746204+0000) 2024-03-27T11:10:49.424 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:49 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:49.032+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:00.747126+0000 front 2024-03-27T11:10:00.747259+0000 (oldest deadline 2024-03-27T11:10:28.348502+0000) 2024-03-27T11:10:49.424 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:49 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:49.032+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-03-27T11:10:14.250568+0000 front 2024-03-27T11:10:14.250492+0000 (oldest deadline 2024-03-27T11:10:37.150604+0000) 2024-03-27T11:10:50.165 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:49 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:49.852+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.380254+0000 front 2024-03-27T11:09:23.380098+0000 (oldest deadline 2024-03-27T11:09:49.280089+0000) 2024-03-27T11:10:50.165 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:49 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:49.852+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:34.582052+0000 front 2024-03-27T11:09:34.582101+0000 (oldest deadline 2024-03-27T11:10:00.482194+0000) 2024-03-27T11:10:50.165 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:49 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:49.852+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:45.784185+0000 front 2024-03-27T11:09:45.784030+0000 (oldest deadline 2024-03-27T11:10:08.683959+0000) 2024-03-27T11:10:50.165 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:49 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:49.852+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:55.086370+0000 front 2024-03-27T11:09:55.086460+0000 (oldest deadline 2024-03-27T11:10:17.386639+0000) 2024-03-27T11:10:50.166 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:49 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:49.852+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:03.189572+0000 front 2024-03-27T11:10:03.189451+0000 (oldest deadline 2024-03-27T11:10:28.390279+0000) 2024-03-27T11:10:50.166 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:49 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:49.852+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-03-27T11:10:14.792988+0000 front 2024-03-27T11:10:14.793114+0000 (oldest deadline 2024-03-27T11:10:38.892796+0000) 2024-03-27T11:10:50.166 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:49 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:49.852+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6822 osd.6 since back 2024-03-27T11:10:24.695220+0000 front 2024-03-27T11:10:24.695490+0000 (oldest deadline 2024-03-27T11:10:48.597165+0000) 2024-03-27T11:10:50.387 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:49 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:49.973+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.654238+0000 front 2024-03-27T11:09:23.654318+0000 (oldest deadline 2024-03-27T11:09:48.954440+0000) 2024-03-27T11:10:50.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:49 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:49.973+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:35.957512+0000 front 2024-03-27T11:09:35.957437+0000 (oldest deadline 2024-03-27T11:10:01.857676+0000) 2024-03-27T11:10:50.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:49 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:49.973+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:44.160107+0000 front 2024-03-27T11:09:44.160318+0000 (oldest deadline 2024-03-27T11:10:08.260147+0000) 2024-03-27T11:10:50.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:49 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:49.973+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.463226+0000 front 2024-03-27T11:09:56.463040+0000 (oldest deadline 2024-03-27T11:10:18.763354+0000) 2024-03-27T11:10:50.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:49 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:49.973+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:05.767047+0000 front 2024-03-27T11:10:05.766994+0000 (oldest deadline 2024-03-27T11:10:28.067070+0000) 2024-03-27T11:10:50.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:49 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:49.973+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-03-27T11:10:13.171475+0000 front 2024-03-27T11:10:13.171056+0000 (oldest deadline 2024-03-27T11:10:37.171547+0000) 2024-03-27T11:10:50.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:49 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:49.973+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.97:6822 osd.6 since back 2024-03-27T11:10:24.675573+0000 front 2024-03-27T11:10:24.675618+0000 (oldest deadline 2024-03-27T11:10:48.175737+0000) 2024-03-27T11:10:50.423 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:49 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:49.983+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.236297+0000 front 2024-03-27T11:09:28.236728+0000 (oldest deadline 2024-03-27T11:09:51.136525+0000) 2024-03-27T11:10:50.513 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:49 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:49.984+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.340023+0000 front 2024-03-27T11:09:36.340351+0000 (oldest deadline 2024-03-27T11:10:02.239659+0000) 2024-03-27T11:10:50.513 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:49 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:49.984+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:46.143084+0000 front 2024-03-27T11:09:46.143228+0000 (oldest deadline 2024-03-27T11:10:10.843246+0000) 2024-03-27T11:10:50.513 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:49 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:49.984+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.046124+0000 front 2024-03-27T11:09:56.046017+0000 (oldest deadline 2024-03-27T11:10:20.746204+0000) 2024-03-27T11:10:50.513 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:49 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:49.984+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:00.747126+0000 front 2024-03-27T11:10:00.747259+0000 (oldest deadline 2024-03-27T11:10:28.348502+0000) 2024-03-27T11:10:50.513 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:49 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:49.984+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-03-27T11:10:14.250568+0000 front 2024-03-27T11:10:14.250492+0000 (oldest deadline 2024-03-27T11:10:37.150604+0000) 2024-03-27T11:10:51.239 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:50 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:50.842+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.380254+0000 front 2024-03-27T11:09:23.380098+0000 (oldest deadline 2024-03-27T11:09:49.280089+0000) 2024-03-27T11:10:51.239 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:50 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:50.842+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:34.582052+0000 front 2024-03-27T11:09:34.582101+0000 (oldest deadline 2024-03-27T11:10:00.482194+0000) 2024-03-27T11:10:51.239 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:50 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:50.842+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:45.784185+0000 front 2024-03-27T11:09:45.784030+0000 (oldest deadline 2024-03-27T11:10:08.683959+0000) 2024-03-27T11:10:51.239 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:50 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:50.842+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:55.086370+0000 front 2024-03-27T11:09:55.086460+0000 (oldest deadline 2024-03-27T11:10:17.386639+0000) 2024-03-27T11:10:51.240 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:50 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:50.842+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:03.189572+0000 front 2024-03-27T11:10:03.189451+0000 (oldest deadline 2024-03-27T11:10:28.390279+0000) 2024-03-27T11:10:51.240 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:50 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:50.842+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-03-27T11:10:14.792988+0000 front 2024-03-27T11:10:14.793114+0000 (oldest deadline 2024-03-27T11:10:38.892796+0000) 2024-03-27T11:10:51.240 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:50 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:50.842+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6822 osd.6 since back 2024-03-27T11:10:24.695220+0000 front 2024-03-27T11:10:24.695490+0000 (oldest deadline 2024-03-27T11:10:48.597165+0000) 2024-03-27T11:10:51.387 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:50 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:50.962+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.654238+0000 front 2024-03-27T11:09:23.654318+0000 (oldest deadline 2024-03-27T11:09:48.954440+0000) 2024-03-27T11:10:51.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:50 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:50.962+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:35.957512+0000 front 2024-03-27T11:09:35.957437+0000 (oldest deadline 2024-03-27T11:10:01.857676+0000) 2024-03-27T11:10:51.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:50 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:50.962+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:44.160107+0000 front 2024-03-27T11:09:44.160318+0000 (oldest deadline 2024-03-27T11:10:08.260147+0000) 2024-03-27T11:10:51.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:50 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:50.962+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.463226+0000 front 2024-03-27T11:09:56.463040+0000 (oldest deadline 2024-03-27T11:10:18.763354+0000) 2024-03-27T11:10:51.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:50 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:50.962+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:05.767047+0000 front 2024-03-27T11:10:05.766994+0000 (oldest deadline 2024-03-27T11:10:28.067070+0000) 2024-03-27T11:10:51.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:50 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:50.962+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-03-27T11:10:13.171475+0000 front 2024-03-27T11:10:13.171056+0000 (oldest deadline 2024-03-27T11:10:37.171547+0000) 2024-03-27T11:10:51.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:50 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:50.962+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.97:6822 osd.6 since back 2024-03-27T11:10:24.675573+0000 front 2024-03-27T11:10:24.675618+0000 (oldest deadline 2024-03-27T11:10:48.175737+0000) 2024-03-27T11:10:51.423 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:50 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:50.992+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.236297+0000 front 2024-03-27T11:09:28.236728+0000 (oldest deadline 2024-03-27T11:09:51.136525+0000) 2024-03-27T11:10:51.423 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:50 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:50.992+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.340023+0000 front 2024-03-27T11:09:36.340351+0000 (oldest deadline 2024-03-27T11:10:02.239659+0000) 2024-03-27T11:10:51.423 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:50 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:50.992+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:46.143084+0000 front 2024-03-27T11:09:46.143228+0000 (oldest deadline 2024-03-27T11:10:10.843246+0000) 2024-03-27T11:10:51.423 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:50 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:50.992+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.046124+0000 front 2024-03-27T11:09:56.046017+0000 (oldest deadline 2024-03-27T11:10:20.746204+0000) 2024-03-27T11:10:51.423 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:50 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:50.992+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:00.747126+0000 front 2024-03-27T11:10:00.747259+0000 (oldest deadline 2024-03-27T11:10:28.348502+0000) 2024-03-27T11:10:51.424 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:50 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:50.992+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-03-27T11:10:14.250568+0000 front 2024-03-27T11:10:14.250492+0000 (oldest deadline 2024-03-27T11:10:37.150604+0000) 2024-03-27T11:10:52.264 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:51 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:51.806+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.380254+0000 front 2024-03-27T11:09:23.380098+0000 (oldest deadline 2024-03-27T11:09:49.280089+0000) 2024-03-27T11:10:52.264 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:51 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:51.806+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:34.582052+0000 front 2024-03-27T11:09:34.582101+0000 (oldest deadline 2024-03-27T11:10:00.482194+0000) 2024-03-27T11:10:52.264 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:51 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:51.806+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:45.784185+0000 front 2024-03-27T11:09:45.784030+0000 (oldest deadline 2024-03-27T11:10:08.683959+0000) 2024-03-27T11:10:52.264 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:51 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:51.806+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:55.086370+0000 front 2024-03-27T11:09:55.086460+0000 (oldest deadline 2024-03-27T11:10:17.386639+0000) 2024-03-27T11:10:52.264 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:51 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:51.806+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:03.189572+0000 front 2024-03-27T11:10:03.189451+0000 (oldest deadline 2024-03-27T11:10:28.390279+0000) 2024-03-27T11:10:52.264 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:51 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:51.806+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-03-27T11:10:14.792988+0000 front 2024-03-27T11:10:14.793114+0000 (oldest deadline 2024-03-27T11:10:38.892796+0000) 2024-03-27T11:10:52.264 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:51 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:51.806+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6822 osd.6 since back 2024-03-27T11:10:24.695220+0000 front 2024-03-27T11:10:24.695490+0000 (oldest deadline 2024-03-27T11:10:48.597165+0000) 2024-03-27T11:10:52.387 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:51 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:51.973+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.654238+0000 front 2024-03-27T11:09:23.654318+0000 (oldest deadline 2024-03-27T11:09:48.954440+0000) 2024-03-27T11:10:52.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:51 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:51.973+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:35.957512+0000 front 2024-03-27T11:09:35.957437+0000 (oldest deadline 2024-03-27T11:10:01.857676+0000) 2024-03-27T11:10:52.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:51 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:51.973+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:44.160107+0000 front 2024-03-27T11:09:44.160318+0000 (oldest deadline 2024-03-27T11:10:08.260147+0000) 2024-03-27T11:10:52.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:51 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:51.973+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.463226+0000 front 2024-03-27T11:09:56.463040+0000 (oldest deadline 2024-03-27T11:10:18.763354+0000) 2024-03-27T11:10:52.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:51 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:51.973+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:05.767047+0000 front 2024-03-27T11:10:05.766994+0000 (oldest deadline 2024-03-27T11:10:28.067070+0000) 2024-03-27T11:10:52.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:51 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:51.973+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-03-27T11:10:13.171475+0000 front 2024-03-27T11:10:13.171056+0000 (oldest deadline 2024-03-27T11:10:37.171547+0000) 2024-03-27T11:10:52.388 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:51 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9[66985]: 2024-03-27T11:10:51.973+0000 7fddc2200700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.97:6822 osd.6 since back 2024-03-27T11:10:24.675573+0000 front 2024-03-27T11:10:24.675618+0000 (oldest deadline 2024-03-27T11:10:48.175737+0000) 2024-03-27T11:10:52.423 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:51 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:51.977+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.236297+0000 front 2024-03-27T11:09:28.236728+0000 (oldest deadline 2024-03-27T11:09:51.136525+0000) 2024-03-27T11:10:52.424 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:51 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:51.977+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.340023+0000 front 2024-03-27T11:09:36.340351+0000 (oldest deadline 2024-03-27T11:10:02.239659+0000) 2024-03-27T11:10:52.424 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:51 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:51.977+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:46.143084+0000 front 2024-03-27T11:09:46.143228+0000 (oldest deadline 2024-03-27T11:10:10.843246+0000) 2024-03-27T11:10:52.424 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:51 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:51.977+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.046124+0000 front 2024-03-27T11:09:56.046017+0000 (oldest deadline 2024-03-27T11:10:20.746204+0000) 2024-03-27T11:10:52.424 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:51 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:51.977+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:00.747126+0000 front 2024-03-27T11:10:00.747259+0000 (oldest deadline 2024-03-27T11:10:28.348502+0000) 2024-03-27T11:10:52.424 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:51 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:51.978+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-03-27T11:10:14.250568+0000 front 2024-03-27T11:10:14.250492+0000 (oldest deadline 2024-03-27T11:10:37.150604+0000) 2024-03-27T11:10:53.263 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:52 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:52.831+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.380254+0000 front 2024-03-27T11:09:23.380098+0000 (oldest deadline 2024-03-27T11:09:49.280089+0000) 2024-03-27T11:10:53.264 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:52 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:52.831+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:34.582052+0000 front 2024-03-27T11:09:34.582101+0000 (oldest deadline 2024-03-27T11:10:00.482194+0000) 2024-03-27T11:10:53.264 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:52 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:52.831+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:45.784185+0000 front 2024-03-27T11:09:45.784030+0000 (oldest deadline 2024-03-27T11:10:08.683959+0000) 2024-03-27T11:10:53.264 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:52 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:52.831+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:55.086370+0000 front 2024-03-27T11:09:55.086460+0000 (oldest deadline 2024-03-27T11:10:17.386639+0000) 2024-03-27T11:10:53.264 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:52 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:52.831+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:03.189572+0000 front 2024-03-27T11:10:03.189451+0000 (oldest deadline 2024-03-27T11:10:28.390279+0000) 2024-03-27T11:10:53.264 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:52 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:52.831+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-03-27T11:10:14.792988+0000 front 2024-03-27T11:10:14.793114+0000 (oldest deadline 2024-03-27T11:10:38.892796+0000) 2024-03-27T11:10:53.264 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:52 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:52.831+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6822 osd.6 since back 2024-03-27T11:10:24.695220+0000 front 2024-03-27T11:10:24.695490+0000 (oldest deadline 2024-03-27T11:10:48.597165+0000) 2024-03-27T11:10:53.423 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:53 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:53.012+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.236297+0000 front 2024-03-27T11:09:28.236728+0000 (oldest deadline 2024-03-27T11:09:51.136525+0000) 2024-03-27T11:10:53.424 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:53 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:53.012+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.340023+0000 front 2024-03-27T11:09:36.340351+0000 (oldest deadline 2024-03-27T11:10:02.239659+0000) 2024-03-27T11:10:53.424 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:53 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:53.012+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:46.143084+0000 front 2024-03-27T11:09:46.143228+0000 (oldest deadline 2024-03-27T11:10:10.843246+0000) 2024-03-27T11:10:53.424 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:53 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:53.012+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.046124+0000 front 2024-03-27T11:09:56.046017+0000 (oldest deadline 2024-03-27T11:10:20.746204+0000) 2024-03-27T11:10:53.424 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:53 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:53.012+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:00.747126+0000 front 2024-03-27T11:10:00.747259+0000 (oldest deadline 2024-03-27T11:10:28.348502+0000) 2024-03-27T11:10:53.424 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:53 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:53.012+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-03-27T11:10:14.250568+0000 front 2024-03-27T11:10:14.250492+0000 (oldest deadline 2024-03-27T11:10:37.150604+0000) 2024-03-27T11:10:53.773 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:53 smithi097 podman[97634]: 2024-03-27 11:10:53.545880873 +0000 UTC m=+5.702406222 container died ddeecfa0134c2247b657cf28e6c2866ae1be798965ec212ec4495b9219fa5aa3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240326.131558-155268c) 2024-03-27T11:10:54.013 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:53 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:53.786+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.380254+0000 front 2024-03-27T11:09:23.380098+0000 (oldest deadline 2024-03-27T11:09:49.280089+0000) 2024-03-27T11:10:54.013 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:53 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:53.786+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:34.582052+0000 front 2024-03-27T11:09:34.582101+0000 (oldest deadline 2024-03-27T11:10:00.482194+0000) 2024-03-27T11:10:54.014 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:53 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:53.786+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:45.784185+0000 front 2024-03-27T11:09:45.784030+0000 (oldest deadline 2024-03-27T11:10:08.683959+0000) 2024-03-27T11:10:54.014 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:53 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:53.786+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:55.086370+0000 front 2024-03-27T11:09:55.086460+0000 (oldest deadline 2024-03-27T11:10:17.386639+0000) 2024-03-27T11:10:54.014 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:53 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:53.786+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:03.189572+0000 front 2024-03-27T11:10:03.189451+0000 (oldest deadline 2024-03-27T11:10:28.390279+0000) 2024-03-27T11:10:54.014 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:53 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:53.786+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-03-27T11:10:14.792988+0000 front 2024-03-27T11:10:14.793114+0000 (oldest deadline 2024-03-27T11:10:38.892796+0000) 2024-03-27T11:10:54.014 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:53 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:53.786+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6822 osd.6 since back 2024-03-27T11:10:24.695220+0000 front 2024-03-27T11:10:24.695490+0000 (oldest deadline 2024-03-27T11:10:48.597165+0000) 2024-03-27T11:10:54.111 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:53 smithi097 podman[97634]: 2024-03-27 11:10:53.95856634 +0000 UTC m=+6.115091725 container cleanup ddeecfa0134c2247b657cf28e6c2866ae1be798965ec212ec4495b9219fa5aa3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e) 2024-03-27T11:10:54.111 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:53 smithi097 bash[97634]: ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9 2024-03-27T11:10:54.387 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:54 smithi097 podman[97653]: 2024-03-27 11:10:54.142743802 +0000 UTC m=+0.583699426 container remove ddeecfa0134c2247b657cf28e6c2866ae1be798965ec212ec4495b9219fa5aa3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240326.131558-155268c) 2024-03-27T11:10:54.423 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:54 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:54.035+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.236297+0000 front 2024-03-27T11:09:28.236728+0000 (oldest deadline 2024-03-27T11:09:51.136525+0000) 2024-03-27T11:10:54.423 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:54 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:54.035+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.340023+0000 front 2024-03-27T11:09:36.340351+0000 (oldest deadline 2024-03-27T11:10:02.239659+0000) 2024-03-27T11:10:54.423 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:54 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:54.035+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:46.143084+0000 front 2024-03-27T11:09:46.143228+0000 (oldest deadline 2024-03-27T11:10:10.843246+0000) 2024-03-27T11:10:54.423 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:54 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:54.035+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.046124+0000 front 2024-03-27T11:09:56.046017+0000 (oldest deadline 2024-03-27T11:10:20.746204+0000) 2024-03-27T11:10:54.424 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:54 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:54.035+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:00.747126+0000 front 2024-03-27T11:10:00.747259+0000 (oldest deadline 2024-03-27T11:10:28.348502+0000) 2024-03-27T11:10:54.424 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:54 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:54.035+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-03-27T11:10:14.250568+0000 front 2024-03-27T11:10:14.250492+0000 (oldest deadline 2024-03-27T11:10:37.150604+0000) 2024-03-27T11:10:55.014 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:54 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:54.778+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.380254+0000 front 2024-03-27T11:09:23.380098+0000 (oldest deadline 2024-03-27T11:09:49.280089+0000) 2024-03-27T11:10:55.014 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:54 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:54.778+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:34.582052+0000 front 2024-03-27T11:09:34.582101+0000 (oldest deadline 2024-03-27T11:10:00.482194+0000) 2024-03-27T11:10:55.014 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:54 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:54.778+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:45.784185+0000 front 2024-03-27T11:09:45.784030+0000 (oldest deadline 2024-03-27T11:10:08.683959+0000) 2024-03-27T11:10:55.014 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:54 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:54.778+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:55.086370+0000 front 2024-03-27T11:09:55.086460+0000 (oldest deadline 2024-03-27T11:10:17.386639+0000) 2024-03-27T11:10:55.014 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:54 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:54.778+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:03.189572+0000 front 2024-03-27T11:10:03.189451+0000 (oldest deadline 2024-03-27T11:10:28.390279+0000) 2024-03-27T11:10:55.014 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:54 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:54.778+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-03-27T11:10:14.792988+0000 front 2024-03-27T11:10:14.793114+0000 (oldest deadline 2024-03-27T11:10:38.892796+0000) 2024-03-27T11:10:55.014 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:54 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:54.778+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6822 osd.6 since back 2024-03-27T11:10:24.695220+0000 front 2024-03-27T11:10:24.695490+0000 (oldest deadline 2024-03-27T11:10:48.597165+0000) 2024-03-27T11:10:55.184 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:54 smithi097 podman[97740]: 2024-03-27 11:10:54.856243721 +0000 UTC m=+0.035696110 image pull b379bbd8da6b5d0fa9a3523941efac213b6815e1cbf05c6507954f0a3fe7a801 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772 2024-03-27T11:10:55.184 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:54 smithi097 podman[97740]: 2024-03-27 11:10:54.968981222 +0000 UTC m=+0.148433648 container create 4c2ab08cf6595d255208bce89177891dca5de07e9298e1d110d5c9e08265fcac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.schema-version=1.0, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326) 2024-03-27T11:10:55.185 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:55 smithi097 podman[97740]: 2024-03-27 11:10:55.216068584 +0000 UTC m=+0.395521043 container init 4c2ab08cf6595d255208bce89177891dca5de07e9298e1d110d5c9e08265fcac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, RELEASE=wip-rishabh-testing-20240326.131558-155268c, CEPH_POINT_RELEASE=) 2024-03-27T11:10:55.423 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:55 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:55.007+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.236297+0000 front 2024-03-27T11:09:28.236728+0000 (oldest deadline 2024-03-27T11:09:51.136525+0000) 2024-03-27T11:10:55.423 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:55 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:55.007+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.340023+0000 front 2024-03-27T11:09:36.340351+0000 (oldest deadline 2024-03-27T11:10:02.239659+0000) 2024-03-27T11:10:55.423 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:55 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:55.007+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:46.143084+0000 front 2024-03-27T11:09:46.143228+0000 (oldest deadline 2024-03-27T11:10:10.843246+0000) 2024-03-27T11:10:55.423 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:55 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:55.007+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.046124+0000 front 2024-03-27T11:09:56.046017+0000 (oldest deadline 2024-03-27T11:10:20.746204+0000) 2024-03-27T11:10:55.423 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:55 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:55.007+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:00.747126+0000 front 2024-03-27T11:10:00.747259+0000 (oldest deadline 2024-03-27T11:10:28.348502+0000) 2024-03-27T11:10:55.424 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:55 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:55.007+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-03-27T11:10:14.250568+0000 front 2024-03-27T11:10:14.250492+0000 (oldest deadline 2024-03-27T11:10:37.150604+0000) 2024-03-27T11:10:55.628 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:55 smithi097 podman[97740]: 2024-03-27 11:10:55.226838886 +0000 UTC m=+0.406291282 container start 4c2ab08cf6595d255208bce89177891dca5de07e9298e1d110d5c9e08265fcac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9-deactivate, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, GIT_CLEAN=True) 2024-03-27T11:10:55.628 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:55 smithi097 podman[97740]: 2024-03-27 11:10:55.296243044 +0000 UTC m=+0.475695443 container attach 4c2ab08cf6595d255208bce89177891dca5de07e9298e1d110d5c9e08265fcac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-27T11:10:55.887 INFO:journalctl@ceph.osd.9.smithi097.stdout:Mar 27 11:10:55 smithi097 podman[97740]: 2024-03-27 11:10:55.659877089 +0000 UTC m=+0.839329525 container died 4c2ab08cf6595d255208bce89177891dca5de07e9298e1d110d5c9e08265fcac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-9-deactivate, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux ) 2024-03-27T11:10:56.263 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:55 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:55.824+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.380254+0000 front 2024-03-27T11:09:23.380098+0000 (oldest deadline 2024-03-27T11:09:49.280089+0000) 2024-03-27T11:10:56.263 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:55 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:55.824+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:34.582052+0000 front 2024-03-27T11:09:34.582101+0000 (oldest deadline 2024-03-27T11:10:00.482194+0000) 2024-03-27T11:10:56.264 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:55 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:55.824+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:45.784185+0000 front 2024-03-27T11:09:45.784030+0000 (oldest deadline 2024-03-27T11:10:08.683959+0000) 2024-03-27T11:10:56.264 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:55 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:55.824+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:55.086370+0000 front 2024-03-27T11:09:55.086460+0000 (oldest deadline 2024-03-27T11:10:17.386639+0000) 2024-03-27T11:10:56.264 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:55 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:55.824+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:03.189572+0000 front 2024-03-27T11:10:03.189451+0000 (oldest deadline 2024-03-27T11:10:28.390279+0000) 2024-03-27T11:10:56.264 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:55 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:55.824+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-03-27T11:10:14.792988+0000 front 2024-03-27T11:10:14.793114+0000 (oldest deadline 2024-03-27T11:10:38.892796+0000) 2024-03-27T11:10:56.264 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:55 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:55.824+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6822 osd.6 since back 2024-03-27T11:10:24.695220+0000 front 2024-03-27T11:10:24.695490+0000 (oldest deadline 2024-03-27T11:10:48.597165+0000) 2024-03-27T11:10:56.279 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-adca88c4-ec26-11ee-95d0-87774f69a715@osd.9.service' 2024-03-27T11:10:56.423 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:55 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:55.981+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.236297+0000 front 2024-03-27T11:09:28.236728+0000 (oldest deadline 2024-03-27T11:09:51.136525+0000) 2024-03-27T11:10:56.423 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:55 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:55.981+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.340023+0000 front 2024-03-27T11:09:36.340351+0000 (oldest deadline 2024-03-27T11:10:02.239659+0000) 2024-03-27T11:10:56.423 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:55 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:55.981+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:46.143084+0000 front 2024-03-27T11:09:46.143228+0000 (oldest deadline 2024-03-27T11:10:10.843246+0000) 2024-03-27T11:10:56.423 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:55 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:55.981+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.046124+0000 front 2024-03-27T11:09:56.046017+0000 (oldest deadline 2024-03-27T11:10:20.746204+0000) 2024-03-27T11:10:56.423 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:55 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:55.982+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:00.747126+0000 front 2024-03-27T11:10:00.747259+0000 (oldest deadline 2024-03-27T11:10:28.348502+0000) 2024-03-27T11:10:56.424 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:55 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:55.982+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-03-27T11:10:14.250568+0000 front 2024-03-27T11:10:14.250492+0000 (oldest deadline 2024-03-27T11:10:37.150604+0000) 2024-03-27T11:10:56.443 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-27T11:10:56.443 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-03-27T11:10:56.443 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-03-27T11:10:56.443 DEBUG:teuthology.orchestra.run.smithi114:> sudo systemctl stop ceph-adca88c4-ec26-11ee-95d0-87774f69a715@osd.10 2024-03-27T11:10:57.014 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:56 smithi114 systemd[1]: Stopping Ceph osd.10 for adca88c4-ec26-11ee-95d0-87774f69a715... 2024-03-27T11:10:57.014 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:56 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:56.783+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.380254+0000 front 2024-03-27T11:09:23.380098+0000 (oldest deadline 2024-03-27T11:09:49.280089+0000) 2024-03-27T11:10:57.014 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:56 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:56.783+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:34.582052+0000 front 2024-03-27T11:09:34.582101+0000 (oldest deadline 2024-03-27T11:10:00.482194+0000) 2024-03-27T11:10:57.014 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:56 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:56.783+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:45.784185+0000 front 2024-03-27T11:09:45.784030+0000 (oldest deadline 2024-03-27T11:10:08.683959+0000) 2024-03-27T11:10:57.014 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:56 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:56.783+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:55.086370+0000 front 2024-03-27T11:09:55.086460+0000 (oldest deadline 2024-03-27T11:10:17.386639+0000) 2024-03-27T11:10:57.014 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:56 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:56.783+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:03.189572+0000 front 2024-03-27T11:10:03.189451+0000 (oldest deadline 2024-03-27T11:10:28.390279+0000) 2024-03-27T11:10:57.014 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:56 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:56.783+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-03-27T11:10:14.792988+0000 front 2024-03-27T11:10:14.793114+0000 (oldest deadline 2024-03-27T11:10:38.892796+0000) 2024-03-27T11:10:57.014 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:56 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:56.783+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6822 osd.6 since back 2024-03-27T11:10:24.695220+0000 front 2024-03-27T11:10:24.695490+0000 (oldest deadline 2024-03-27T11:10:48.597165+0000) 2024-03-27T11:10:57.173 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:56 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:56.933+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.236297+0000 front 2024-03-27T11:09:28.236728+0000 (oldest deadline 2024-03-27T11:09:51.136525+0000) 2024-03-27T11:10:57.173 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:56 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:56.933+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.340023+0000 front 2024-03-27T11:09:36.340351+0000 (oldest deadline 2024-03-27T11:10:02.239659+0000) 2024-03-27T11:10:57.173 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:56 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:56.933+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:46.143084+0000 front 2024-03-27T11:09:46.143228+0000 (oldest deadline 2024-03-27T11:10:10.843246+0000) 2024-03-27T11:10:57.173 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:56 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:56.933+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.046124+0000 front 2024-03-27T11:09:56.046017+0000 (oldest deadline 2024-03-27T11:10:20.746204+0000) 2024-03-27T11:10:57.174 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:56 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:56.934+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:00.747126+0000 front 2024-03-27T11:10:00.747259+0000 (oldest deadline 2024-03-27T11:10:28.348502+0000) 2024-03-27T11:10:57.174 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:56 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:56.934+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-03-27T11:10:14.250568+0000 front 2024-03-27T11:10:14.250492+0000 (oldest deadline 2024-03-27T11:10:37.150604+0000) 2024-03-27T11:10:57.513 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:57 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:57.057+0000 7f831f600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-27T11:10:57.513 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:57 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:57.057+0000 7f831f600700 -1 osd.10 92 *** Got signal Terminated *** 2024-03-27T11:10:57.513 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:57 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:57.057+0000 7f831f600700 -1 osd.10 92 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-27T11:10:58.013 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:57 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:57.792+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.380254+0000 front 2024-03-27T11:09:23.380098+0000 (oldest deadline 2024-03-27T11:09:49.280089+0000) 2024-03-27T11:10:58.013 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:57 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:57.792+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:34.582052+0000 front 2024-03-27T11:09:34.582101+0000 (oldest deadline 2024-03-27T11:10:00.482194+0000) 2024-03-27T11:10:58.014 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:57 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:57.792+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:45.784185+0000 front 2024-03-27T11:09:45.784030+0000 (oldest deadline 2024-03-27T11:10:08.683959+0000) 2024-03-27T11:10:58.014 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:57 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:57.792+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:55.086370+0000 front 2024-03-27T11:09:55.086460+0000 (oldest deadline 2024-03-27T11:10:17.386639+0000) 2024-03-27T11:10:58.014 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:57 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:57.792+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:03.189572+0000 front 2024-03-27T11:10:03.189451+0000 (oldest deadline 2024-03-27T11:10:28.390279+0000) 2024-03-27T11:10:58.014 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:57 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:57.792+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-03-27T11:10:14.792988+0000 front 2024-03-27T11:10:14.793114+0000 (oldest deadline 2024-03-27T11:10:38.892796+0000) 2024-03-27T11:10:58.014 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:57 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:57.792+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6822 osd.6 since back 2024-03-27T11:10:24.695220+0000 front 2024-03-27T11:10:24.695490+0000 (oldest deadline 2024-03-27T11:10:48.597165+0000) 2024-03-27T11:10:58.173 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:57 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:57.901+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.236297+0000 front 2024-03-27T11:09:28.236728+0000 (oldest deadline 2024-03-27T11:09:51.136525+0000) 2024-03-27T11:10:58.173 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:57 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:57.901+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.340023+0000 front 2024-03-27T11:09:36.340351+0000 (oldest deadline 2024-03-27T11:10:02.239659+0000) 2024-03-27T11:10:58.173 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:57 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:57.901+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:46.143084+0000 front 2024-03-27T11:09:46.143228+0000 (oldest deadline 2024-03-27T11:10:10.843246+0000) 2024-03-27T11:10:58.173 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:57 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:57.901+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.046124+0000 front 2024-03-27T11:09:56.046017+0000 (oldest deadline 2024-03-27T11:10:20.746204+0000) 2024-03-27T11:10:58.173 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:57 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:57.901+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:00.747126+0000 front 2024-03-27T11:10:00.747259+0000 (oldest deadline 2024-03-27T11:10:28.348502+0000) 2024-03-27T11:10:58.173 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:57 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:57.901+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-03-27T11:10:14.250568+0000 front 2024-03-27T11:10:14.250492+0000 (oldest deadline 2024-03-27T11:10:37.150604+0000) 2024-03-27T11:10:59.173 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:58 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:58.935+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.236297+0000 front 2024-03-27T11:09:28.236728+0000 (oldest deadline 2024-03-27T11:09:51.136525+0000) 2024-03-27T11:10:59.173 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:58 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:58.935+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.340023+0000 front 2024-03-27T11:09:36.340351+0000 (oldest deadline 2024-03-27T11:10:02.239659+0000) 2024-03-27T11:10:59.173 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:58 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:58.935+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:46.143084+0000 front 2024-03-27T11:09:46.143228+0000 (oldest deadline 2024-03-27T11:10:10.843246+0000) 2024-03-27T11:10:59.173 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:58 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:58.935+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.046124+0000 front 2024-03-27T11:09:56.046017+0000 (oldest deadline 2024-03-27T11:10:20.746204+0000) 2024-03-27T11:10:59.173 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:58 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:58.935+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:00.747126+0000 front 2024-03-27T11:10:00.747259+0000 (oldest deadline 2024-03-27T11:10:28.348502+0000) 2024-03-27T11:10:59.173 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:58 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:58.935+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-03-27T11:10:14.250568+0000 front 2024-03-27T11:10:14.250492+0000 (oldest deadline 2024-03-27T11:10:37.150604+0000) 2024-03-27T11:10:59.264 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:58 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:58.838+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.380254+0000 front 2024-03-27T11:09:23.380098+0000 (oldest deadline 2024-03-27T11:09:49.280089+0000) 2024-03-27T11:10:59.264 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:58 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:58.838+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:34.582052+0000 front 2024-03-27T11:09:34.582101+0000 (oldest deadline 2024-03-27T11:10:00.482194+0000) 2024-03-27T11:10:59.264 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:58 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:58.838+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:45.784185+0000 front 2024-03-27T11:09:45.784030+0000 (oldest deadline 2024-03-27T11:10:08.683959+0000) 2024-03-27T11:10:59.264 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:58 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:58.838+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:55.086370+0000 front 2024-03-27T11:09:55.086460+0000 (oldest deadline 2024-03-27T11:10:17.386639+0000) 2024-03-27T11:10:59.264 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:58 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:58.838+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:03.189572+0000 front 2024-03-27T11:10:03.189451+0000 (oldest deadline 2024-03-27T11:10:28.390279+0000) 2024-03-27T11:10:59.264 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:58 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:58.838+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-03-27T11:10:14.792988+0000 front 2024-03-27T11:10:14.793114+0000 (oldest deadline 2024-03-27T11:10:38.892796+0000) 2024-03-27T11:10:59.264 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:58 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:58.838+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6822 osd.6 since back 2024-03-27T11:10:24.695220+0000 front 2024-03-27T11:10:24.695490+0000 (oldest deadline 2024-03-27T11:10:48.597165+0000) 2024-03-27T11:11:00.173 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:59 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:59.938+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.236297+0000 front 2024-03-27T11:09:28.236728+0000 (oldest deadline 2024-03-27T11:09:51.136525+0000) 2024-03-27T11:11:00.173 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:59 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:59.938+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.340023+0000 front 2024-03-27T11:09:36.340351+0000 (oldest deadline 2024-03-27T11:10:02.239659+0000) 2024-03-27T11:11:00.173 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:59 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:59.938+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:46.143084+0000 front 2024-03-27T11:09:46.143228+0000 (oldest deadline 2024-03-27T11:10:10.843246+0000) 2024-03-27T11:11:00.173 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:59 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:59.938+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.046124+0000 front 2024-03-27T11:09:56.046017+0000 (oldest deadline 2024-03-27T11:10:20.746204+0000) 2024-03-27T11:11:00.173 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:59 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:59.938+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:00.747126+0000 front 2024-03-27T11:10:00.747259+0000 (oldest deadline 2024-03-27T11:10:28.348502+0000) 2024-03-27T11:11:00.173 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:59 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:59.938+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-03-27T11:10:14.250568+0000 front 2024-03-27T11:10:14.250492+0000 (oldest deadline 2024-03-27T11:10:37.150604+0000) 2024-03-27T11:11:00.173 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:10:59 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:10:59.938+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6820 osd.7 since back 2024-03-27T11:10:33.956937+0000 front 2024-03-27T11:10:33.957085+0000 (oldest deadline 2024-03-27T11:10:59.856975+0000) 2024-03-27T11:11:00.264 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:59 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:59.823+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.380254+0000 front 2024-03-27T11:09:23.380098+0000 (oldest deadline 2024-03-27T11:09:49.280089+0000) 2024-03-27T11:11:00.264 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:59 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:59.823+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:34.582052+0000 front 2024-03-27T11:09:34.582101+0000 (oldest deadline 2024-03-27T11:10:00.482194+0000) 2024-03-27T11:11:00.264 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:59 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:59.823+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:45.784185+0000 front 2024-03-27T11:09:45.784030+0000 (oldest deadline 2024-03-27T11:10:08.683959+0000) 2024-03-27T11:11:00.264 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:59 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:59.823+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:55.086370+0000 front 2024-03-27T11:09:55.086460+0000 (oldest deadline 2024-03-27T11:10:17.386639+0000) 2024-03-27T11:11:00.264 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:59 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:59.823+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:03.189572+0000 front 2024-03-27T11:10:03.189451+0000 (oldest deadline 2024-03-27T11:10:28.390279+0000) 2024-03-27T11:11:00.264 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:59 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:59.823+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-03-27T11:10:14.792988+0000 front 2024-03-27T11:10:14.793114+0000 (oldest deadline 2024-03-27T11:10:38.892796+0000) 2024-03-27T11:11:00.264 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:10:59 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:10:59.823+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6822 osd.6 since back 2024-03-27T11:10:24.695220+0000 front 2024-03-27T11:10:24.695490+0000 (oldest deadline 2024-03-27T11:10:48.597165+0000) 2024-03-27T11:11:01.263 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:11:00 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:11:00.849+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.380254+0000 front 2024-03-27T11:09:23.380098+0000 (oldest deadline 2024-03-27T11:09:49.280089+0000) 2024-03-27T11:11:01.263 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:11:00 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:11:00.849+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:34.582052+0000 front 2024-03-27T11:09:34.582101+0000 (oldest deadline 2024-03-27T11:10:00.482194+0000) 2024-03-27T11:11:01.263 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:11:00 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:11:00.849+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:45.784185+0000 front 2024-03-27T11:09:45.784030+0000 (oldest deadline 2024-03-27T11:10:08.683959+0000) 2024-03-27T11:11:01.263 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:11:00 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:11:00.849+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:55.086370+0000 front 2024-03-27T11:09:55.086460+0000 (oldest deadline 2024-03-27T11:10:17.386639+0000) 2024-03-27T11:11:01.264 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:11:00 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:11:00.849+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:03.189572+0000 front 2024-03-27T11:10:03.189451+0000 (oldest deadline 2024-03-27T11:10:28.390279+0000) 2024-03-27T11:11:01.264 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:11:00 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:11:00.849+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-03-27T11:10:14.792988+0000 front 2024-03-27T11:10:14.793114+0000 (oldest deadline 2024-03-27T11:10:38.892796+0000) 2024-03-27T11:11:01.264 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:11:00 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:11:00.849+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6822 osd.6 since back 2024-03-27T11:10:24.695220+0000 front 2024-03-27T11:10:24.695490+0000 (oldest deadline 2024-03-27T11:10:48.597165+0000) 2024-03-27T11:11:01.423 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:11:00 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:11:00.967+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.236297+0000 front 2024-03-27T11:09:28.236728+0000 (oldest deadline 2024-03-27T11:09:51.136525+0000) 2024-03-27T11:11:01.424 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:11:00 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:11:00.967+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.340023+0000 front 2024-03-27T11:09:36.340351+0000 (oldest deadline 2024-03-27T11:10:02.239659+0000) 2024-03-27T11:11:01.424 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:11:00 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:11:00.967+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:46.143084+0000 front 2024-03-27T11:09:46.143228+0000 (oldest deadline 2024-03-27T11:10:10.843246+0000) 2024-03-27T11:11:01.424 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:11:00 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:11:00.967+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.046124+0000 front 2024-03-27T11:09:56.046017+0000 (oldest deadline 2024-03-27T11:10:20.746204+0000) 2024-03-27T11:11:01.424 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:11:00 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:11:00.967+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:00.747126+0000 front 2024-03-27T11:10:00.747259+0000 (oldest deadline 2024-03-27T11:10:28.348502+0000) 2024-03-27T11:11:01.424 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:11:00 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:11:00.967+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-03-27T11:10:14.250568+0000 front 2024-03-27T11:10:14.250492+0000 (oldest deadline 2024-03-27T11:10:37.150604+0000) 2024-03-27T11:11:01.424 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:11:00 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:11:00.968+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6820 osd.7 since back 2024-03-27T11:10:33.956937+0000 front 2024-03-27T11:10:33.957085+0000 (oldest deadline 2024-03-27T11:10:59.856975+0000) 2024-03-27T11:11:02.173 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:11:01 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:11:01.928+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.236297+0000 front 2024-03-27T11:09:28.236728+0000 (oldest deadline 2024-03-27T11:09:51.136525+0000) 2024-03-27T11:11:02.173 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:11:01 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:11:01.928+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.340023+0000 front 2024-03-27T11:09:36.340351+0000 (oldest deadline 2024-03-27T11:10:02.239659+0000) 2024-03-27T11:11:02.174 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:11:01 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:11:01.928+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:46.143084+0000 front 2024-03-27T11:09:46.143228+0000 (oldest deadline 2024-03-27T11:10:10.843246+0000) 2024-03-27T11:11:02.174 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:11:01 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:11:01.928+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.046124+0000 front 2024-03-27T11:09:56.046017+0000 (oldest deadline 2024-03-27T11:10:20.746204+0000) 2024-03-27T11:11:02.174 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:11:01 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:11:01.928+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:00.747126+0000 front 2024-03-27T11:10:00.747259+0000 (oldest deadline 2024-03-27T11:10:28.348502+0000) 2024-03-27T11:11:02.174 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:11:01 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:11:01.928+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-03-27T11:10:14.250568+0000 front 2024-03-27T11:10:14.250492+0000 (oldest deadline 2024-03-27T11:10:37.150604+0000) 2024-03-27T11:11:02.174 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:11:01 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:11:01.928+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6820 osd.7 since back 2024-03-27T11:10:33.956937+0000 front 2024-03-27T11:10:33.957085+0000 (oldest deadline 2024-03-27T11:10:59.856975+0000) 2024-03-27T11:11:02.263 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:11:01 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:11:01.848+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:23.380254+0000 front 2024-03-27T11:09:23.380098+0000 (oldest deadline 2024-03-27T11:09:49.280089+0000) 2024-03-27T11:11:02.263 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:11:01 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:11:01.848+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:34.582052+0000 front 2024-03-27T11:09:34.582101+0000 (oldest deadline 2024-03-27T11:10:00.482194+0000) 2024-03-27T11:11:02.264 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:11:01 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:11:01.848+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:45.784185+0000 front 2024-03-27T11:09:45.784030+0000 (oldest deadline 2024-03-27T11:10:08.683959+0000) 2024-03-27T11:11:02.264 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:11:01 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:11:01.848+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:55.086370+0000 front 2024-03-27T11:09:55.086460+0000 (oldest deadline 2024-03-27T11:10:17.386639+0000) 2024-03-27T11:11:02.264 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:11:01 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:11:01.848+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:03.189572+0000 front 2024-03-27T11:10:03.189451+0000 (oldest deadline 2024-03-27T11:10:28.390279+0000) 2024-03-27T11:11:02.264 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:11:01 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:11:01.848+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-03-27T11:10:14.792988+0000 front 2024-03-27T11:10:14.793114+0000 (oldest deadline 2024-03-27T11:10:38.892796+0000) 2024-03-27T11:11:02.264 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:11:01 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10[60259]: 2024-03-27T11:11:01.848+0000 7f8315e00700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.97:6822 osd.6 since back 2024-03-27T11:10:24.695220+0000 front 2024-03-27T11:10:24.695490+0000 (oldest deadline 2024-03-27T11:10:48.597165+0000) 2024-03-27T11:11:03.013 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:11:02 smithi114 podman[71220]: 2024-03-27 11:11:02.710601297 +0000 UTC m=+5.830350244 container died 2eeb802829458d4190bd6ed0c707a5114e13375332937588ef0ce17e8f492da4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.schema-version=1.0, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-03-27T11:11:03.173 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:11:02 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:11:02.942+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.236297+0000 front 2024-03-27T11:09:28.236728+0000 (oldest deadline 2024-03-27T11:09:51.136525+0000) 2024-03-27T11:11:03.174 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:11:02 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:11:02.943+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.340023+0000 front 2024-03-27T11:09:36.340351+0000 (oldest deadline 2024-03-27T11:10:02.239659+0000) 2024-03-27T11:11:03.174 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:11:02 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:11:02.943+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:46.143084+0000 front 2024-03-27T11:09:46.143228+0000 (oldest deadline 2024-03-27T11:10:10.843246+0000) 2024-03-27T11:11:03.174 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:11:02 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:11:02.943+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.046124+0000 front 2024-03-27T11:09:56.046017+0000 (oldest deadline 2024-03-27T11:10:20.746204+0000) 2024-03-27T11:11:03.174 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:11:02 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:11:02.943+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:00.747126+0000 front 2024-03-27T11:10:00.747259+0000 (oldest deadline 2024-03-27T11:10:28.348502+0000) 2024-03-27T11:11:03.174 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:11:02 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:11:02.943+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-03-27T11:10:14.250568+0000 front 2024-03-27T11:10:14.250492+0000 (oldest deadline 2024-03-27T11:10:37.150604+0000) 2024-03-27T11:11:03.174 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:11:02 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:11:02.943+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6820 osd.7 since back 2024-03-27T11:10:33.956937+0000 front 2024-03-27T11:10:33.957085+0000 (oldest deadline 2024-03-27T11:10:59.856975+0000) 2024-03-27T11:11:03.388 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:11:03 smithi114 podman[71220]: 2024-03-27 11:11:03.177640409 +0000 UTC m=+6.297389393 container cleanup 2eeb802829458d4190bd6ed0c707a5114e13375332937588ef0ce17e8f492da4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10, org.label-schema.vendor=CentOS, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-03-27T11:11:03.388 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:11:03 smithi114 bash[71220]: ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10 2024-03-27T11:11:03.763 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:11:03 smithi114 podman[71237]: 2024-03-27 11:11:03.420117909 +0000 UTC m=+0.696945036 container remove 2eeb802829458d4190bd6ed0c707a5114e13375332937588ef0ce17e8f492da4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-03-27T11:11:04.173 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:11:03 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:11:03.910+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.236297+0000 front 2024-03-27T11:09:28.236728+0000 (oldest deadline 2024-03-27T11:09:51.136525+0000) 2024-03-27T11:11:04.173 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:11:03 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:11:03.910+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.340023+0000 front 2024-03-27T11:09:36.340351+0000 (oldest deadline 2024-03-27T11:10:02.239659+0000) 2024-03-27T11:11:04.173 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:11:03 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:11:03.910+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:46.143084+0000 front 2024-03-27T11:09:46.143228+0000 (oldest deadline 2024-03-27T11:10:10.843246+0000) 2024-03-27T11:11:04.173 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:11:03 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:11:03.910+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.046124+0000 front 2024-03-27T11:09:56.046017+0000 (oldest deadline 2024-03-27T11:10:20.746204+0000) 2024-03-27T11:11:04.174 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:11:03 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:11:03.910+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:00.747126+0000 front 2024-03-27T11:10:00.747259+0000 (oldest deadline 2024-03-27T11:10:28.348502+0000) 2024-03-27T11:11:04.174 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:11:03 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:11:03.910+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-03-27T11:10:14.250568+0000 front 2024-03-27T11:10:14.250492+0000 (oldest deadline 2024-03-27T11:10:37.150604+0000) 2024-03-27T11:11:04.174 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:11:03 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:11:03.910+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6820 osd.7 since back 2024-03-27T11:10:33.956937+0000 front 2024-03-27T11:10:33.957085+0000 (oldest deadline 2024-03-27T11:10:59.856975+0000) 2024-03-27T11:11:04.434 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:11:04 smithi114 podman[71320]: 2024-03-27 11:11:04.09445467 +0000 UTC m=+0.049665488 image pull b379bbd8da6b5d0fa9a3523941efac213b6815e1cbf05c6507954f0a3fe7a801 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772 2024-03-27T11:11:04.434 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:11:04 smithi114 podman[71320]: 2024-03-27 11:11:04.212647282 +0000 UTC m=+0.167858080 container create f033b89d346d518005fa2fea0d6098244b1b250d911669c912a78ecac6a9a787 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, GIT_BRANCH=HEAD, ceph=True) 2024-03-27T11:11:04.434 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:11:04 smithi114 podman[71320]: 2024-03-27 11:11:04.466307511 +0000 UTC m=+0.421518355 container init f033b89d346d518005fa2fea0d6098244b1b250d911669c912a78ecac6a9a787 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240326.131558-155268c) 2024-03-27T11:11:04.763 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:11:04 smithi114 podman[71320]: 2024-03-27 11:11:04.478503052 +0000 UTC m=+0.433713862 container start f033b89d346d518005fa2fea0d6098244b1b250d911669c912a78ecac6a9a787 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True) 2024-03-27T11:11:04.763 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:11:04 smithi114 podman[71320]: 2024-03-27 11:11:04.566421407 +0000 UTC m=+0.521632225 container attach f033b89d346d518005fa2fea0d6098244b1b250d911669c912a78ecac6a9a787 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10-deactivate, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-03-27T11:11:05.173 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:11:04 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:11:04.935+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.236297+0000 front 2024-03-27T11:09:28.236728+0000 (oldest deadline 2024-03-27T11:09:51.136525+0000) 2024-03-27T11:11:05.173 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:11:04 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:11:04.936+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.340023+0000 front 2024-03-27T11:09:36.340351+0000 (oldest deadline 2024-03-27T11:10:02.239659+0000) 2024-03-27T11:11:05.173 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:11:04 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:11:04.936+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:46.143084+0000 front 2024-03-27T11:09:46.143228+0000 (oldest deadline 2024-03-27T11:10:10.843246+0000) 2024-03-27T11:11:05.173 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:11:04 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:11:04.936+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.046124+0000 front 2024-03-27T11:09:56.046017+0000 (oldest deadline 2024-03-27T11:10:20.746204+0000) 2024-03-27T11:11:05.173 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:11:04 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:11:04.936+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:00.747126+0000 front 2024-03-27T11:10:00.747259+0000 (oldest deadline 2024-03-27T11:10:28.348502+0000) 2024-03-27T11:11:05.173 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:11:04 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:11:04.936+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-03-27T11:10:14.250568+0000 front 2024-03-27T11:10:14.250492+0000 (oldest deadline 2024-03-27T11:10:37.150604+0000) 2024-03-27T11:11:05.174 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:11:04 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:11:04.936+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6820 osd.7 since back 2024-03-27T11:10:33.956937+0000 front 2024-03-27T11:10:33.957085+0000 (oldest deadline 2024-03-27T11:10:59.856975+0000) 2024-03-27T11:11:05.263 INFO:journalctl@ceph.osd.10.smithi114.stdout:Mar 27 11:11:04 smithi114 podman[71320]: 2024-03-27 11:11:04.988546447 +0000 UTC m=+0.943757269 container died f033b89d346d518005fa2fea0d6098244b1b250d911669c912a78ecac6a9a787 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-10-deactivate, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-03-27T11:11:05.782 DEBUG:teuthology.orchestra.run.smithi114:> sudo pkill -f 'journalctl -f -n 0 -u ceph-adca88c4-ec26-11ee-95d0-87774f69a715@osd.10.service' 2024-03-27T11:11:05.877 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-27T11:11:05.877 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-03-27T11:11:05.877 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-03-27T11:11:05.877 DEBUG:teuthology.orchestra.run.smithi177:> sudo systemctl stop ceph-adca88c4-ec26-11ee-95d0-87774f69a715@osd.11 2024-03-27T11:11:06.174 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:11:05 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:11:05.931+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.236297+0000 front 2024-03-27T11:09:28.236728+0000 (oldest deadline 2024-03-27T11:09:51.136525+0000) 2024-03-27T11:11:06.174 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:11:05 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:11:05.931+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.340023+0000 front 2024-03-27T11:09:36.340351+0000 (oldest deadline 2024-03-27T11:10:02.239659+0000) 2024-03-27T11:11:06.174 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:11:05 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:11:05.931+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:46.143084+0000 front 2024-03-27T11:09:46.143228+0000 (oldest deadline 2024-03-27T11:10:10.843246+0000) 2024-03-27T11:11:06.174 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:11:05 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:11:05.931+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.046124+0000 front 2024-03-27T11:09:56.046017+0000 (oldest deadline 2024-03-27T11:10:20.746204+0000) 2024-03-27T11:11:06.174 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:11:05 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:11:05.931+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:00.747126+0000 front 2024-03-27T11:10:00.747259+0000 (oldest deadline 2024-03-27T11:10:28.348502+0000) 2024-03-27T11:11:06.174 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:11:05 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:11:05.931+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-03-27T11:10:14.250568+0000 front 2024-03-27T11:10:14.250492+0000 (oldest deadline 2024-03-27T11:10:37.150604+0000) 2024-03-27T11:11:06.174 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:11:05 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:11:05.931+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6820 osd.7 since back 2024-03-27T11:10:33.956937+0000 front 2024-03-27T11:10:33.957085+0000 (oldest deadline 2024-03-27T11:10:59.856975+0000) 2024-03-27T11:11:06.174 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:11:05 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:11:05.931+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.177:6820 osd.8 since back 2024-03-27T11:10:39.857816+0000 front 2024-03-27T11:10:39.857840+0000 (oldest deadline 2024-03-27T11:11:05.157632+0000) 2024-03-27T11:11:06.175 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:11:06 smithi177 systemd[1]: Stopping Ceph osd.11 for adca88c4-ec26-11ee-95d0-87774f69a715... 2024-03-27T11:11:06.673 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:11:06 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:11:06.407+0000 7f5a5c000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-27T11:11:06.674 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:11:06 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:11:06.407+0000 7f5a5c000700 -1 osd.11 92 *** Got signal Terminated *** 2024-03-27T11:11:06.674 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:11:06 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:11:06.407+0000 7f5a5c000700 -1 osd.11 92 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-27T11:11:07.173 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:11:06 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:11:06.907+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.236297+0000 front 2024-03-27T11:09:28.236728+0000 (oldest deadline 2024-03-27T11:09:51.136525+0000) 2024-03-27T11:11:07.174 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:11:06 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:11:06.907+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.340023+0000 front 2024-03-27T11:09:36.340351+0000 (oldest deadline 2024-03-27T11:10:02.239659+0000) 2024-03-27T11:11:07.174 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:11:06 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:11:06.907+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:46.143084+0000 front 2024-03-27T11:09:46.143228+0000 (oldest deadline 2024-03-27T11:10:10.843246+0000) 2024-03-27T11:11:07.174 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:11:06 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:11:06.907+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.046124+0000 front 2024-03-27T11:09:56.046017+0000 (oldest deadline 2024-03-27T11:10:20.746204+0000) 2024-03-27T11:11:07.174 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:11:06 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:11:06.907+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:00.747126+0000 front 2024-03-27T11:10:00.747259+0000 (oldest deadline 2024-03-27T11:10:28.348502+0000) 2024-03-27T11:11:07.174 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:11:06 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:11:06.907+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-03-27T11:10:14.250568+0000 front 2024-03-27T11:10:14.250492+0000 (oldest deadline 2024-03-27T11:10:37.150604+0000) 2024-03-27T11:11:07.174 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:11:06 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:11:06.907+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6820 osd.7 since back 2024-03-27T11:10:33.956937+0000 front 2024-03-27T11:10:33.957085+0000 (oldest deadline 2024-03-27T11:10:59.856975+0000) 2024-03-27T11:11:07.174 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:11:06 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:11:06.907+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.177:6820 osd.8 since back 2024-03-27T11:10:39.857816+0000 front 2024-03-27T11:10:39.857840+0000 (oldest deadline 2024-03-27T11:11:05.157632+0000) 2024-03-27T11:11:08.173 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:11:07 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:11:07.892+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.236297+0000 front 2024-03-27T11:09:28.236728+0000 (oldest deadline 2024-03-27T11:09:51.136525+0000) 2024-03-27T11:11:08.174 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:11:07 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:11:07.892+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.340023+0000 front 2024-03-27T11:09:36.340351+0000 (oldest deadline 2024-03-27T11:10:02.239659+0000) 2024-03-27T11:11:08.174 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:11:07 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:11:07.892+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:46.143084+0000 front 2024-03-27T11:09:46.143228+0000 (oldest deadline 2024-03-27T11:10:10.843246+0000) 2024-03-27T11:11:08.174 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:11:07 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:11:07.892+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.046124+0000 front 2024-03-27T11:09:56.046017+0000 (oldest deadline 2024-03-27T11:10:20.746204+0000) 2024-03-27T11:11:08.174 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:11:07 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:11:07.892+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:00.747126+0000 front 2024-03-27T11:10:00.747259+0000 (oldest deadline 2024-03-27T11:10:28.348502+0000) 2024-03-27T11:11:08.174 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:11:07 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:11:07.892+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-03-27T11:10:14.250568+0000 front 2024-03-27T11:10:14.250492+0000 (oldest deadline 2024-03-27T11:10:37.150604+0000) 2024-03-27T11:11:08.174 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:11:07 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:11:07.892+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6820 osd.7 since back 2024-03-27T11:10:33.956937+0000 front 2024-03-27T11:10:33.957085+0000 (oldest deadline 2024-03-27T11:10:59.856975+0000) 2024-03-27T11:11:08.174 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:11:07 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:11:07.892+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.177:6820 osd.8 since back 2024-03-27T11:10:39.857816+0000 front 2024-03-27T11:10:39.857840+0000 (oldest deadline 2024-03-27T11:11:05.157632+0000) 2024-03-27T11:11:09.174 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:11:08 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:11:08.900+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.236297+0000 front 2024-03-27T11:09:28.236728+0000 (oldest deadline 2024-03-27T11:09:51.136525+0000) 2024-03-27T11:11:09.174 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:11:08 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:11:08.900+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.340023+0000 front 2024-03-27T11:09:36.340351+0000 (oldest deadline 2024-03-27T11:10:02.239659+0000) 2024-03-27T11:11:09.174 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:11:08 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:11:08.900+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:46.143084+0000 front 2024-03-27T11:09:46.143228+0000 (oldest deadline 2024-03-27T11:10:10.843246+0000) 2024-03-27T11:11:09.174 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:11:08 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:11:08.900+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.046124+0000 front 2024-03-27T11:09:56.046017+0000 (oldest deadline 2024-03-27T11:10:20.746204+0000) 2024-03-27T11:11:09.174 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:11:08 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:11:08.900+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:00.747126+0000 front 2024-03-27T11:10:00.747259+0000 (oldest deadline 2024-03-27T11:10:28.348502+0000) 2024-03-27T11:11:09.174 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:11:08 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:11:08.900+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-03-27T11:10:14.250568+0000 front 2024-03-27T11:10:14.250492+0000 (oldest deadline 2024-03-27T11:10:37.150604+0000) 2024-03-27T11:11:09.174 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:11:08 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:11:08.900+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6820 osd.7 since back 2024-03-27T11:10:33.956937+0000 front 2024-03-27T11:10:33.957085+0000 (oldest deadline 2024-03-27T11:10:59.856975+0000) 2024-03-27T11:11:09.174 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:11:08 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:11:08.900+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.177:6820 osd.8 since back 2024-03-27T11:10:39.857816+0000 front 2024-03-27T11:10:39.857840+0000 (oldest deadline 2024-03-27T11:11:05.157632+0000) 2024-03-27T11:11:10.123 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:11:09 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:11:09.858+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.236297+0000 front 2024-03-27T11:09:28.236728+0000 (oldest deadline 2024-03-27T11:09:51.136525+0000) 2024-03-27T11:11:10.123 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:11:09 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:11:09.858+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.340023+0000 front 2024-03-27T11:09:36.340351+0000 (oldest deadline 2024-03-27T11:10:02.239659+0000) 2024-03-27T11:11:10.123 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:11:09 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:11:09.858+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:46.143084+0000 front 2024-03-27T11:09:46.143228+0000 (oldest deadline 2024-03-27T11:10:10.843246+0000) 2024-03-27T11:11:10.123 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:11:09 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:11:09.858+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.046124+0000 front 2024-03-27T11:09:56.046017+0000 (oldest deadline 2024-03-27T11:10:20.746204+0000) 2024-03-27T11:11:10.123 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:11:09 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:11:09.858+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:00.747126+0000 front 2024-03-27T11:10:00.747259+0000 (oldest deadline 2024-03-27T11:10:28.348502+0000) 2024-03-27T11:11:10.123 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:11:09 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:11:09.858+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-03-27T11:10:14.250568+0000 front 2024-03-27T11:10:14.250492+0000 (oldest deadline 2024-03-27T11:10:37.150604+0000) 2024-03-27T11:11:10.124 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:11:09 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:11:09.858+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6820 osd.7 since back 2024-03-27T11:10:33.956937+0000 front 2024-03-27T11:10:33.957085+0000 (oldest deadline 2024-03-27T11:10:59.856975+0000) 2024-03-27T11:11:10.124 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:11:09 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:11:09.858+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.177:6820 osd.8 since back 2024-03-27T11:10:39.857816+0000 front 2024-03-27T11:10:39.857840+0000 (oldest deadline 2024-03-27T11:11:05.157632+0000) 2024-03-27T11:11:11.173 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:11:10 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:11:10.857+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-03-27T11:09:28.236297+0000 front 2024-03-27T11:09:28.236728+0000 (oldest deadline 2024-03-27T11:09:51.136525+0000) 2024-03-27T11:11:11.174 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:11:10 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:11:10.857+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-03-27T11:09:36.340023+0000 front 2024-03-27T11:09:36.340351+0000 (oldest deadline 2024-03-27T11:10:02.239659+0000) 2024-03-27T11:11:11.174 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:11:10 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:11:10.857+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-03-27T11:09:46.143084+0000 front 2024-03-27T11:09:46.143228+0000 (oldest deadline 2024-03-27T11:10:10.843246+0000) 2024-03-27T11:11:11.174 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:11:10 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:11:10.857+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-03-27T11:09:56.046124+0000 front 2024-03-27T11:09:56.046017+0000 (oldest deadline 2024-03-27T11:10:20.746204+0000) 2024-03-27T11:11:11.174 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:11:10 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:11:10.857+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-03-27T11:10:00.747126+0000 front 2024-03-27T11:10:00.747259+0000 (oldest deadline 2024-03-27T11:10:28.348502+0000) 2024-03-27T11:11:11.174 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:11:10 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:11:10.857+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-03-27T11:10:14.250568+0000 front 2024-03-27T11:10:14.250492+0000 (oldest deadline 2024-03-27T11:10:37.150604+0000) 2024-03-27T11:11:11.174 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:11:10 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:11:10.857+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.114:6820 osd.7 since back 2024-03-27T11:10:33.956937+0000 front 2024-03-27T11:10:33.957085+0000 (oldest deadline 2024-03-27T11:10:59.856975+0000) 2024-03-27T11:11:11.174 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:11:10 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11[61121]: 2024-03-27T11:11:10.857+0000 7f5a57800700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.177:6820 osd.8 since back 2024-03-27T11:10:39.857816+0000 front 2024-03-27T11:10:39.857840+0000 (oldest deadline 2024-03-27T11:11:05.157632+0000) 2024-03-27T11:11:12.403 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:11:12 smithi177 podman[72153]: 2024-03-27 11:11:12.134445513 +0000 UTC m=+5.847197420 container died 40b79e22b3c8efa8f8f06ac87544e674b8574b530fb51fb1fa1218bfb52e9134 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11, ceph=True, RELEASE=wip-rishabh-testing-20240326.131558-155268c, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e) 2024-03-27T11:11:12.672 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:11:12 smithi177 podman[72153]: 2024-03-27 11:11:12.582804152 +0000 UTC m=+6.295556035 container cleanup 40b79e22b3c8efa8f8f06ac87544e674b8574b530fb51fb1fa1218bfb52e9134 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.build-date=20240326, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-03-27T11:11:12.673 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:11:12 smithi177 bash[72153]: ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11 2024-03-27T11:11:13.173 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:11:12 smithi177 podman[72171]: 2024-03-27 11:11:12.800640953 +0000 UTC m=+0.656063795 container remove 40b79e22b3c8efa8f8f06ac87544e674b8574b530fb51fb1fa1218bfb52e9134 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-03-27T11:11:13.822 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:11:13 smithi177 podman[72260]: 2024-03-27 11:11:13.498777179 +0000 UTC m=+0.033030751 image pull b379bbd8da6b5d0fa9a3523941efac213b6815e1cbf05c6507954f0a3fe7a801 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772 2024-03-27T11:11:13.822 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:11:13 smithi177 podman[72260]: 2024-03-27 11:11:13.601554187 +0000 UTC m=+0.135807796 container create ea7c79f7835eca9315a4f70b2596eb528f81c6f3ff3c8639eacf6fdad6ed739c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11-deactivate, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240326.131558-155268c, ceph=True, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-27T11:11:14.173 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:11:13 smithi177 podman[72260]: 2024-03-27 11:11:13.854146278 +0000 UTC m=+0.388399862 container init ea7c79f7835eca9315a4f70b2596eb528f81c6f3ff3c8639eacf6fdad6ed739c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11-deactivate, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=) 2024-03-27T11:11:14.173 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:11:13 smithi177 podman[72260]: 2024-03-27 11:11:13.862497175 +0000 UTC m=+0.396750756 container start ea7c79f7835eca9315a4f70b2596eb528f81c6f3ff3c8639eacf6fdad6ed739c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240326.131558-155268c, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240326) 2024-03-27T11:11:14.173 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:11:13 smithi177 podman[72260]: 2024-03-27 11:11:13.945323271 +0000 UTC m=+0.479576860 container attach ea7c79f7835eca9315a4f70b2596eb528f81c6f3ff3c8639eacf6fdad6ed739c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11-deactivate, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-03-27T11:11:14.629 INFO:journalctl@ceph.osd.11.smithi177.stdout:Mar 27 11:11:14 smithi177 podman[72260]: 2024-03-27 11:11:14.336912561 +0000 UTC m=+0.871166143 container died ea7c79f7835eca9315a4f70b2596eb528f81c6f3ff3c8639eacf6fdad6ed739c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-osd-11-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_CLEAN=True) 2024-03-27T11:11:15.001 DEBUG:teuthology.orchestra.run.smithi177:> sudo pkill -f 'journalctl -f -n 0 -u ceph-adca88c4-ec26-11ee-95d0-87774f69a715@osd.11.service' 2024-03-27T11:11:15.103 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-27T11:11:15.104 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-03-27T11:11:15.104 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-03-27T11:11:15.104 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-adca88c4-ec26-11ee-95d0-87774f69a715@mds.a 2024-03-27T11:11:15.637 INFO:journalctl@ceph.mds.a.smithi097.stdout:Mar 27 11:11:15 smithi097 systemd[1]: Stopping Ceph mds.a for adca88c4-ec26-11ee-95d0-87774f69a715... 2024-03-27T11:11:16.137 INFO:journalctl@ceph.mds.a.smithi097.stdout:Mar 27 11:11:15 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-a[73845]: 2024-03-27T11:11:15.707+0000 7f4ce2800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-27T11:11:16.137 INFO:journalctl@ceph.mds.a.smithi097.stdout:Mar 27 11:11:15 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-a[73845]: 2024-03-27T11:11:15.707+0000 7f4ce2800700 -1 mds.a *** got signal Terminated *** 2024-03-27T11:11:20.225 INFO:journalctl@ceph.mds.a.smithi097.stdout:Mar 27 11:11:19 smithi097 podman[97844]: 2024-03-27 11:11:19.929872059 +0000 UTC m=+4.371239020 container died c886b2875232f40ad088dbdd0f95feefc2590f10073fbcd9ae6dd9afb670f3f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-a, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True) 2024-03-27T11:11:20.595 INFO:journalctl@ceph.mds.a.smithi097.stdout:Mar 27 11:11:20 smithi097 podman[97844]: 2024-03-27 11:11:20.366934654 +0000 UTC m=+4.808301639 container cleanup c886b2875232f40ad088dbdd0f95feefc2590f10073fbcd9ae6dd9afb670f3f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-a, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, RELEASE=wip-rishabh-testing-20240326.131558-155268c, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-27T11:11:20.595 INFO:journalctl@ceph.mds.a.smithi097.stdout:Mar 27 11:11:20 smithi097 bash[97844]: ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-a 2024-03-27T11:11:20.887 INFO:journalctl@ceph.mds.a.smithi097.stdout:Mar 27 11:11:20 smithi097 podman[97858]: 2024-03-27 11:11:20.626263477 +0000 UTC m=+0.680050807 container remove c886b2875232f40ad088dbdd0f95feefc2590f10073fbcd9ae6dd9afb670f3f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-a, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240326.131558-155268c, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-03-27T11:11:21.118 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-adca88c4-ec26-11ee-95d0-87774f69a715@mds.a.service' 2024-03-27T11:11:21.242 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-27T11:11:21.242 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-03-27T11:11:21.243 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-03-27T11:11:21.243 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-adca88c4-ec26-11ee-95d0-87774f69a715@mds.d 2024-03-27T11:11:21.637 INFO:journalctl@ceph.mds.d.smithi097.stdout:Mar 27 11:11:21 smithi097 systemd[1]: Stopping Ceph mds.d for adca88c4-ec26-11ee-95d0-87774f69a715... 2024-03-27T11:11:22.137 INFO:journalctl@ceph.mds.d.smithi097.stdout:Mar 27 11:11:21 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-d[72808]: 2024-03-27T11:11:21.843+0000 7fd06de00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-27T11:11:22.137 INFO:journalctl@ceph.mds.d.smithi097.stdout:Mar 27 11:11:21 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-d[72808]: 2024-03-27T11:11:21.843+0000 7fd06de00700 -1 mds.d *** got signal Terminated *** 2024-03-27T11:11:24.836 INFO:journalctl@ceph.mds.d.smithi097.stdout:Mar 27 11:11:24 smithi097 podman[97963]: 2024-03-27 11:11:24.574963345 +0000 UTC m=+2.850286523 container died 693e794650a1cd93a6351af8dbd720ef93b5ff1ff097a10e2aeaf5d74c1138ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-d, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240326.131558-155268c) 2024-03-27T11:11:25.137 INFO:journalctl@ceph.mds.d.smithi097.stdout:Mar 27 11:11:24 smithi097 podman[97963]: 2024-03-27 11:11:24.979056043 +0000 UTC m=+3.254379248 container cleanup 693e794650a1cd93a6351af8dbd720ef93b5ff1ff097a10e2aeaf5d74c1138ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-d, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-03-27T11:11:25.137 INFO:journalctl@ceph.mds.d.smithi097.stdout:Mar 27 11:11:24 smithi097 bash[97963]: ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-d 2024-03-27T11:11:25.564 INFO:journalctl@ceph.mds.d.smithi097.stdout:Mar 27 11:11:25 smithi097 podman[97989]: 2024-03-27 11:11:25.330168824 +0000 UTC m=+0.737163019 container remove 693e794650a1cd93a6351af8dbd720ef93b5ff1ff097a10e2aeaf5d74c1138ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-d, RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, ceph=True) 2024-03-27T11:11:25.581 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-adca88c4-ec26-11ee-95d0-87774f69a715@mds.d.service' 2024-03-27T11:11:25.700 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-27T11:11:25.700 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-03-27T11:11:25.700 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-03-27T11:11:25.701 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-adca88c4-ec26-11ee-95d0-87774f69a715@mds.g 2024-03-27T11:11:26.137 INFO:journalctl@ceph.mds.g.smithi097.stdout:Mar 27 11:11:25 smithi097 systemd[1]: Stopping Ceph mds.g for adca88c4-ec26-11ee-95d0-87774f69a715... 2024-03-27T11:11:26.637 INFO:journalctl@ceph.mds.g.smithi097.stdout:Mar 27 11:11:26 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-g[73493]: 2024-03-27T11:11:26.364+0000 7f156e200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-27T11:11:26.637 INFO:journalctl@ceph.mds.g.smithi097.stdout:Mar 27 11:11:26 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-g[73493]: 2024-03-27T11:11:26.364+0000 7f156e200700 -1 mds.g *** got signal Terminated *** 2024-03-27T11:11:28.366 INFO:journalctl@ceph.mds.g.smithi097.stdout:Mar 27 11:11:28 smithi097 podman[98094]: 2024-03-27 11:11:28.11507112 +0000 UTC m=+1.900955277 container died d58bb8d449717d12e61a7c321da2487155a9c921960565fb8bc0078c03f97745 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-g, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-03-27T11:11:28.637 INFO:journalctl@ceph.mds.g.smithi097.stdout:Mar 27 11:11:28 smithi097 podman[98094]: 2024-03-27 11:11:28.516134261 +0000 UTC m=+2.302018445 container cleanup d58bb8d449717d12e61a7c321da2487155a9c921960565fb8bc0078c03f97745 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-g, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240326.131558-155268c) 2024-03-27T11:11:28.637 INFO:journalctl@ceph.mds.g.smithi097.stdout:Mar 27 11:11:28 smithi097 bash[98094]: ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-g 2024-03-27T11:11:29.137 INFO:journalctl@ceph.mds.g.smithi097.stdout:Mar 27 11:11:28 smithi097 podman[98109]: 2024-03-27 11:11:28.750635332 +0000 UTC m=+0.624569997 container remove d58bb8d449717d12e61a7c321da2487155a9c921960565fb8bc0078c03f97745 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-g, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240326.131558-155268c, maintainer=Guillaume Abrioux , GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.build-date=20240326, ceph=True) 2024-03-27T11:11:29.196 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-adca88c4-ec26-11ee-95d0-87774f69a715@mds.g.service' 2024-03-27T11:11:29.313 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-27T11:11:29.313 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-03-27T11:11:29.313 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-03-27T11:11:29.313 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-adca88c4-ec26-11ee-95d0-87774f69a715@mds.j 2024-03-27T11:11:29.637 INFO:journalctl@ceph.mds.j.smithi097.stdout:Mar 27 11:11:29 smithi097 systemd[1]: Stopping Ceph mds.j for adca88c4-ec26-11ee-95d0-87774f69a715... 2024-03-27T11:11:30.137 INFO:journalctl@ceph.mds.j.smithi097.stdout:Mar 27 11:11:29 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-j[72431]: 2024-03-27T11:11:29.909+0000 7fbf0b400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-27T11:11:30.137 INFO:journalctl@ceph.mds.j.smithi097.stdout:Mar 27 11:11:29 smithi097 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-j[72431]: 2024-03-27T11:11:29.909+0000 7fbf0b400700 -1 mds.j *** got signal Terminated *** 2024-03-27T11:11:32.887 INFO:journalctl@ceph.mds.j.smithi097.stdout:Mar 27 11:11:32 smithi097 podman[98215]: 2024-03-27 11:11:32.622335256 +0000 UTC m=+2.852155592 container died c8fdfbf183e03bb5e20a05450e8ba6458851e39713b58307de8143280bae40b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-j, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240326.131558-155268c, ceph=True) 2024-03-27T11:11:33.180 INFO:journalctl@ceph.mds.j.smithi097.stdout:Mar 27 11:11:33 smithi097 podman[98215]: 2024-03-27 11:11:33.036458803 +0000 UTC m=+3.266279139 container cleanup c8fdfbf183e03bb5e20a05450e8ba6458851e39713b58307de8143280bae40b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240326.131558-155268c, ceph=True, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-03-27T11:11:33.180 INFO:journalctl@ceph.mds.j.smithi097.stdout:Mar 27 11:11:33 smithi097 bash[98215]: ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-j 2024-03-27T11:11:33.637 INFO:journalctl@ceph.mds.j.smithi097.stdout:Mar 27 11:11:33 smithi097 podman[98229]: 2024-03-27 11:11:33.212297285 +0000 UTC m=+0.576908187 container remove c8fdfbf183e03bb5e20a05450e8ba6458851e39713b58307de8143280bae40b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240326.131558-155268c, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, GIT_CLEAN=True) 2024-03-27T11:11:33.724 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-adca88c4-ec26-11ee-95d0-87774f69a715@mds.j.service' 2024-03-27T11:11:33.840 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-27T11:11:33.840 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-03-27T11:11:33.840 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-03-27T11:11:33.840 DEBUG:teuthology.orchestra.run.smithi114:> sudo systemctl stop ceph-adca88c4-ec26-11ee-95d0-87774f69a715@mds.b 2024-03-27T11:11:34.263 INFO:journalctl@ceph.mds.b.smithi114.stdout:Mar 27 11:11:33 smithi114 systemd[1]: Stopping Ceph mds.b for adca88c4-ec26-11ee-95d0-87774f69a715... 2024-03-27T11:11:34.726 INFO:journalctl@ceph.mds.b.smithi114.stdout:Mar 27 11:11:34 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-b[65857]: 2024-03-27T11:11:34.403+0000 7f039e000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-27T11:11:34.726 INFO:journalctl@ceph.mds.b.smithi114.stdout:Mar 27 11:11:34 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-b[65857]: 2024-03-27T11:11:34.403+0000 7f039e000700 -1 mds.b *** got signal Terminated *** 2024-03-27T11:11:41.697 INFO:journalctl@ceph.mds.b.smithi114.stdout:Mar 27 11:11:41 smithi114 podman[71418]: 2024-03-27 11:11:41.370805782 +0000 UTC m=+7.144380227 container died 0a34b4897eb0a1695933352cc841dc014f933be063a9a01ae49eda30160e36ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-b, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326) 2024-03-27T11:11:42.013 INFO:journalctl@ceph.mds.b.smithi114.stdout:Mar 27 11:11:41 smithi114 podman[71418]: 2024-03-27 11:11:41.86200964 +0000 UTC m=+7.635584028 container cleanup 0a34b4897eb0a1695933352cc841dc014f933be063a9a01ae49eda30160e36ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-b, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, ceph=True, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240326.131558-155268c, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-03-27T11:11:42.013 INFO:journalctl@ceph.mds.b.smithi114.stdout:Mar 27 11:11:41 smithi114 bash[71418]: ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-b 2024-03-27T11:11:42.513 INFO:journalctl@ceph.mds.b.smithi114.stdout:Mar 27 11:11:42 smithi114 podman[71441]: 2024-03-27 11:11:42.155415489 +0000 UTC m=+0.762799878 container remove 0a34b4897eb0a1695933352cc841dc014f933be063a9a01ae49eda30160e36ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-b, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True) 2024-03-27T11:11:42.571 DEBUG:teuthology.orchestra.run.smithi114:> sudo pkill -f 'journalctl -f -n 0 -u ceph-adca88c4-ec26-11ee-95d0-87774f69a715@mds.b.service' 2024-03-27T11:11:42.679 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-27T11:11:42.680 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-03-27T11:11:42.680 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-03-27T11:11:42.680 DEBUG:teuthology.orchestra.run.smithi114:> sudo systemctl stop ceph-adca88c4-ec26-11ee-95d0-87774f69a715@mds.e 2024-03-27T11:11:43.249 INFO:journalctl@ceph.mds.e.smithi114.stdout:Mar 27 11:11:42 smithi114 systemd[1]: Stopping Ceph mds.e for adca88c4-ec26-11ee-95d0-87774f69a715... 2024-03-27T11:11:43.513 INFO:journalctl@ceph.mds.e.smithi114.stdout:Mar 27 11:11:43 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-e[65242]: 2024-03-27T11:11:43.280+0000 7f416e000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-27T11:11:43.513 INFO:journalctl@ceph.mds.e.smithi114.stdout:Mar 27 11:11:43 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-e[65242]: 2024-03-27T11:11:43.280+0000 7f416e000700 -1 mds.e *** got signal Terminated *** 2024-03-27T11:11:46.763 INFO:journalctl@ceph.mds.e.smithi114.stdout:Mar 27 11:11:46 smithi114 podman[71544]: 2024-03-27 11:11:46.489137672 +0000 UTC m=+3.363346014 container died 427c9c4785157f00a0cfc142b9766f8948dde380c8db8dd00c52e104d44d7163 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-e, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-03-27T11:11:47.085 INFO:journalctl@ceph.mds.e.smithi114.stdout:Mar 27 11:11:46 smithi114 podman[71544]: 2024-03-27 11:11:46.941344811 +0000 UTC m=+3.815553168 container cleanup 427c9c4785157f00a0cfc142b9766f8948dde380c8db8dd00c52e104d44d7163 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-e, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_BRANCH=HEAD) 2024-03-27T11:11:47.085 INFO:journalctl@ceph.mds.e.smithi114.stdout:Mar 27 11:11:46 smithi114 bash[71544]: ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-e 2024-03-27T11:11:47.513 INFO:journalctl@ceph.mds.e.smithi114.stdout:Mar 27 11:11:47 smithi114 podman[71558]: 2024-03-27 11:11:47.117387876 +0000 UTC m=+0.616264008 container remove 427c9c4785157f00a0cfc142b9766f8948dde380c8db8dd00c52e104d44d7163 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-e, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-03-27T11:11:47.574 DEBUG:teuthology.orchestra.run.smithi114:> sudo pkill -f 'journalctl -f -n 0 -u ceph-adca88c4-ec26-11ee-95d0-87774f69a715@mds.e.service' 2024-03-27T11:11:47.680 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-27T11:11:47.680 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-03-27T11:11:47.681 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-03-27T11:11:47.681 DEBUG:teuthology.orchestra.run.smithi114:> sudo systemctl stop ceph-adca88c4-ec26-11ee-95d0-87774f69a715@mds.h 2024-03-27T11:11:48.012 INFO:journalctl@ceph.mds.h.smithi114.stdout:Mar 27 11:11:47 smithi114 systemd[1]: Stopping Ceph mds.h for adca88c4-ec26-11ee-95d0-87774f69a715... 2024-03-27T11:11:48.513 INFO:journalctl@ceph.mds.h.smithi114.stdout:Mar 27 11:11:48 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-h[65555]: 2024-03-27T11:11:48.234+0000 7fd7f9800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-27T11:11:48.513 INFO:journalctl@ceph.mds.h.smithi114.stdout:Mar 27 11:11:48 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-h[65555]: 2024-03-27T11:11:48.234+0000 7fd7f9800700 -1 mds.h *** got signal Terminated *** 2024-03-27T11:11:54.076 INFO:journalctl@ceph.mds.h.smithi114.stdout:Mar 27 11:11:53 smithi114 podman[71662]: 2024-03-27 11:11:53.820602879 +0000 UTC m=+5.708943555 container died 21762b79bb89e8819bfc30cec93becabe29481c20677e6f5ca410dc2e47ad7d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-h, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-03-27T11:11:54.513 INFO:journalctl@ceph.mds.h.smithi114.stdout:Mar 27 11:11:54 smithi114 podman[71662]: 2024-03-27 11:11:54.304190332 +0000 UTC m=+6.192531002 container cleanup 21762b79bb89e8819bfc30cec93becabe29481c20677e6f5ca410dc2e47ad7d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-h, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.build-date=20240326) 2024-03-27T11:11:54.513 INFO:journalctl@ceph.mds.h.smithi114.stdout:Mar 27 11:11:54 smithi114 bash[71662]: ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-h 2024-03-27T11:11:54.899 INFO:journalctl@ceph.mds.h.smithi114.stdout:Mar 27 11:11:54 smithi114 podman[71676]: 2024-03-27 11:11:54.663748746 +0000 UTC m=+0.828347557 container remove 21762b79bb89e8819bfc30cec93becabe29481c20677e6f5ca410dc2e47ad7d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-h, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240326.131558-155268c, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240326) 2024-03-27T11:11:54.915 DEBUG:teuthology.orchestra.run.smithi114:> sudo pkill -f 'journalctl -f -n 0 -u ceph-adca88c4-ec26-11ee-95d0-87774f69a715@mds.h.service' 2024-03-27T11:11:55.026 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-27T11:11:55.026 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-03-27T11:11:55.026 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-03-27T11:11:55.026 DEBUG:teuthology.orchestra.run.smithi114:> sudo systemctl stop ceph-adca88c4-ec26-11ee-95d0-87774f69a715@mds.k 2024-03-27T11:11:55.512 INFO:journalctl@ceph.mds.k.smithi114.stdout:Mar 27 11:11:55 smithi114 systemd[1]: Stopping Ceph mds.k for adca88c4-ec26-11ee-95d0-87774f69a715... 2024-03-27T11:11:56.013 INFO:journalctl@ceph.mds.k.smithi114.stdout:Mar 27 11:11:55 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-k[64945]: 2024-03-27T11:11:55.721+0000 7fc474c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-27T11:11:56.013 INFO:journalctl@ceph.mds.k.smithi114.stdout:Mar 27 11:11:55 smithi114 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-k[64945]: 2024-03-27T11:11:55.721+0000 7fc474c00700 -1 mds.k *** got signal Terminated *** 2024-03-27T11:12:00.188 INFO:journalctl@ceph.mds.k.smithi114.stdout:Mar 27 11:11:59 smithi114 podman[71782]: 2024-03-27 11:11:59.888690705 +0000 UTC m=+4.338798913 container died 012040f065d4fea01dd5ced86d8fae509821420a41664023970e25053584d551 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-k, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240326.131558-155268c, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e) 2024-03-27T11:12:00.513 INFO:journalctl@ceph.mds.k.smithi114.stdout:Mar 27 11:12:00 smithi114 podman[71782]: 2024-03-27 11:12:00.332897966 +0000 UTC m=+4.783006128 container cleanup 012040f065d4fea01dd5ced86d8fae509821420a41664023970e25053584d551 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-k, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240326.131558-155268c) 2024-03-27T11:12:00.513 INFO:journalctl@ceph.mds.k.smithi114.stdout:Mar 27 11:12:00 smithi114 bash[71782]: ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-k 2024-03-27T11:12:00.983 INFO:journalctl@ceph.mds.k.smithi114.stdout:Mar 27 11:12:00 smithi114 podman[71796]: 2024-03-27 11:12:00.634184811 +0000 UTC m=+0.722567764 container remove 012040f065d4fea01dd5ced86d8fae509821420a41664023970e25053584d551 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-k, RELEASE=wip-rishabh-testing-20240326.131558-155268c, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-27T11:12:01.009 DEBUG:teuthology.orchestra.run.smithi114:> sudo pkill -f 'journalctl -f -n 0 -u ceph-adca88c4-ec26-11ee-95d0-87774f69a715@mds.k.service' 2024-03-27T11:12:01.118 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-27T11:12:01.119 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-03-27T11:12:01.119 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-03-27T11:12:01.119 DEBUG:teuthology.orchestra.run.smithi177:> sudo systemctl stop ceph-adca88c4-ec26-11ee-95d0-87774f69a715@mds.c 2024-03-27T11:12:01.558 INFO:journalctl@ceph.mds.c.smithi177.stdout:Mar 27 11:12:01 smithi177 systemd[1]: Stopping Ceph mds.c for adca88c4-ec26-11ee-95d0-87774f69a715... 2024-03-27T11:12:01.922 INFO:journalctl@ceph.mds.c.smithi177.stdout:Mar 27 11:12:01 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-c[66456]: 2024-03-27T11:12:01.742+0000 7fd2d8a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-27T11:12:01.923 INFO:journalctl@ceph.mds.c.smithi177.stdout:Mar 27 11:12:01 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-c[66456]: 2024-03-27T11:12:01.742+0000 7fd2d8a00700 -1 mds.c *** got signal Terminated *** 2024-03-27T11:12:04.422 INFO:journalctl@ceph.mds.c.smithi177.stdout:Mar 27 11:12:04 smithi177 podman[72374]: 2024-03-27 11:12:04.237804622 +0000 UTC m=+2.632898934 container died 5ffb343bec61f7e87208fb4fd64bc226bdbe48924ae95e1c4efdf4729ce9fc7c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-c, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-27T11:12:04.850 INFO:journalctl@ceph.mds.c.smithi177.stdout:Mar 27 11:12:04 smithi177 podman[72374]: 2024-03-27 11:12:04.705620755 +0000 UTC m=+3.100715076 container cleanup 5ffb343bec61f7e87208fb4fd64bc226bdbe48924ae95e1c4efdf4729ce9fc7c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240326.131558-155268c, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-03-27T11:12:04.850 INFO:journalctl@ceph.mds.c.smithi177.stdout:Mar 27 11:12:04 smithi177 bash[72374]: ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-c 2024-03-27T11:12:05.172 INFO:journalctl@ceph.mds.c.smithi177.stdout:Mar 27 11:12:04 smithi177 podman[72399]: 2024-03-27 11:12:04.88185953 +0000 UTC m=+0.627084849 container remove 5ffb343bec61f7e87208fb4fd64bc226bdbe48924ae95e1c4efdf4729ce9fc7c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-c, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, ceph=True, org.label-schema.build-date=20240326) 2024-03-27T11:12:05.290 DEBUG:teuthology.orchestra.run.smithi177:> sudo pkill -f 'journalctl -f -n 0 -u ceph-adca88c4-ec26-11ee-95d0-87774f69a715@mds.c.service' 2024-03-27T11:12:05.388 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-27T11:12:05.388 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-03-27T11:12:05.388 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-03-27T11:12:05.389 DEBUG:teuthology.orchestra.run.smithi177:> sudo systemctl stop ceph-adca88c4-ec26-11ee-95d0-87774f69a715@mds.f 2024-03-27T11:12:05.922 INFO:journalctl@ceph.mds.f.smithi177.stdout:Mar 27 11:12:05 smithi177 systemd[1]: Stopping Ceph mds.f for adca88c4-ec26-11ee-95d0-87774f69a715... 2024-03-27T11:12:06.422 INFO:journalctl@ceph.mds.f.smithi177.stdout:Mar 27 11:12:05 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-f[66157]: 2024-03-27T11:12:05.957+0000 7fb6a7200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-27T11:12:06.423 INFO:journalctl@ceph.mds.f.smithi177.stdout:Mar 27 11:12:05 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-f[66157]: 2024-03-27T11:12:05.957+0000 7fb6a7200700 -1 mds.f *** got signal Terminated *** 2024-03-27T11:12:11.575 INFO:journalctl@ceph.mds.f.smithi177.stdout:Mar 27 11:12:11 smithi177 podman[72502]: 2024-03-27 11:12:11.332248528 +0000 UTC m=+5.506252796 container died 4a238616523cba3fbe16e747258fb61c033ec87300db453cea07faf41a5f7132 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-f, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, RELEASE=wip-rishabh-testing-20240326.131558-155268c, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-03-27T11:12:11.855 INFO:journalctl@ceph.mds.f.smithi177.stdout:Mar 27 11:12:11 smithi177 podman[72502]: 2024-03-27 11:12:11.719101159 +0000 UTC m=+5.893105480 container cleanup 4a238616523cba3fbe16e747258fb61c033ec87300db453cea07faf41a5f7132 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-f, ceph=True, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-03-27T11:12:11.855 INFO:journalctl@ceph.mds.f.smithi177.stdout:Mar 27 11:12:11 smithi177 bash[72502]: ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-f 2024-03-27T11:12:12.172 INFO:journalctl@ceph.mds.f.smithi177.stdout:Mar 27 11:12:11 smithi177 podman[72515]: 2024-03-27 11:12:11.887047403 +0000 UTC m=+0.535250078 container remove 4a238616523cba3fbe16e747258fb61c033ec87300db453cea07faf41a5f7132 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-f, RELEASE=wip-rishabh-testing-20240326.131558-155268c, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-27T11:12:12.408 DEBUG:teuthology.orchestra.run.smithi177:> sudo pkill -f 'journalctl -f -n 0 -u ceph-adca88c4-ec26-11ee-95d0-87774f69a715@mds.f.service' 2024-03-27T11:12:12.516 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-27T11:12:12.517 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-03-27T11:12:12.517 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-03-27T11:12:12.517 DEBUG:teuthology.orchestra.run.smithi177:> sudo systemctl stop ceph-adca88c4-ec26-11ee-95d0-87774f69a715@mds.i 2024-03-27T11:12:12.672 INFO:journalctl@ceph.mds.i.smithi177.stdout:Mar 27 11:12:12 smithi177 systemd[1]: Stopping Ceph mds.i for adca88c4-ec26-11ee-95d0-87774f69a715... 2024-03-27T11:12:13.422 INFO:journalctl@ceph.mds.i.smithi177.stdout:Mar 27 11:12:13 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-i[66757]: 2024-03-27T11:12:13.086+0000 7f467a000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-27T11:12:13.423 INFO:journalctl@ceph.mds.i.smithi177.stdout:Mar 27 11:12:13 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-i[66757]: 2024-03-27T11:12:13.086+0000 7f467a000700 -1 mds.i *** got signal Terminated *** 2024-03-27T11:12:16.569 INFO:journalctl@ceph.mds.i.smithi177.stdout:Mar 27 11:12:16 smithi177 podman[72622]: 2024-03-27 11:12:16.290295273 +0000 UTC m=+3.351740602 container died 2e7f049a1279bada5ae2740f8b74503a5da9c6e28459983ebd986c2543ecaf41 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-i, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-03-27T11:12:16.922 INFO:journalctl@ceph.mds.i.smithi177.stdout:Mar 27 11:12:16 smithi177 podman[72622]: 2024-03-27 11:12:16.714008438 +0000 UTC m=+3.775453769 container cleanup 2e7f049a1279bada5ae2740f8b74503a5da9c6e28459983ebd986c2543ecaf41 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-i, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.license=GPLv2) 2024-03-27T11:12:16.923 INFO:journalctl@ceph.mds.i.smithi177.stdout:Mar 27 11:12:16 smithi177 bash[72622]: ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-i 2024-03-27T11:12:17.372 INFO:journalctl@ceph.mds.i.smithi177.stdout:Mar 27 11:12:16 smithi177 podman[72635]: 2024-03-27 11:12:16.989819545 +0000 UTC m=+0.683072969 container remove 2e7f049a1279bada5ae2740f8b74503a5da9c6e28459983ebd986c2543ecaf41 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-i, ceph=True, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-03-27T11:12:17.408 DEBUG:teuthology.orchestra.run.smithi177:> sudo pkill -f 'journalctl -f -n 0 -u ceph-adca88c4-ec26-11ee-95d0-87774f69a715@mds.i.service' 2024-03-27T11:12:17.519 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-27T11:12:17.520 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-03-27T11:12:17.520 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-03-27T11:12:17.520 DEBUG:teuthology.orchestra.run.smithi177:> sudo systemctl stop ceph-adca88c4-ec26-11ee-95d0-87774f69a715@mds.l 2024-03-27T11:12:17.922 INFO:journalctl@ceph.mds.l.smithi177.stdout:Mar 27 11:12:17 smithi177 systemd[1]: Stopping Ceph mds.l for adca88c4-ec26-11ee-95d0-87774f69a715... 2024-03-27T11:12:18.422 INFO:journalctl@ceph.mds.l.smithi177.stdout:Mar 27 11:12:18 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-l[65836]: 2024-03-27T11:12:18.073+0000 7fd702000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-27T11:12:18.423 INFO:journalctl@ceph.mds.l.smithi177.stdout:Mar 27 11:12:18 smithi177 ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-l[65836]: 2024-03-27T11:12:18.073+0000 7fd702000700 -1 mds.l *** got signal Terminated *** 2024-03-27T11:12:20.733 INFO:journalctl@ceph.mds.l.smithi177.stdout:Mar 27 11:12:20 smithi177 podman[72741]: 2024-03-27 11:12:20.485836348 +0000 UTC m=+2.536607797 container died 5ebaf095548b340079f8207835a4684bf862e553ad22da4c434b12885197431d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-l, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-03-27T11:12:21.027 INFO:journalctl@ceph.mds.l.smithi177.stdout:Mar 27 11:12:20 smithi177 podman[72741]: 2024-03-27 11:12:20.875008934 +0000 UTC m=+2.925780399 container cleanup 5ebaf095548b340079f8207835a4684bf862e553ad22da4c434b12885197431d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-l, io.buildah.version=1.33.5, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-03-27T11:12:21.027 INFO:journalctl@ceph.mds.l.smithi177.stdout:Mar 27 11:12:20 smithi177 bash[72741]: ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-l 2024-03-27T11:12:21.422 INFO:journalctl@ceph.mds.l.smithi177.stdout:Mar 27 11:12:21 smithi177 podman[72753]: 2024-03-27 11:12:21.058959932 +0000 UTC m=+0.560757178 container remove 5ebaf095548b340079f8207835a4684bf862e553ad22da4c434b12885197431d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-adca88c4-ec26-11ee-95d0-87774f69a715-mds-l, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-03-27T11:12:21.504 DEBUG:teuthology.orchestra.run.smithi177:> sudo pkill -f 'journalctl -f -n 0 -u ceph-adca88c4-ec26-11ee-95d0-87774f69a715@mds.l.service' 2024-03-27T11:12:21.610 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-27T11:12:21.610 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-03-27T11:12:21.611 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid adca88c4-ec26-11ee-95d0-87774f69a715 --force --keep-logs 2024-03-27T11:12:21.933 INFO:teuthology.orchestra.run.smithi097.stdout:Deleting cluster with fsid: adca88c4-ec26-11ee-95d0-87774f69a715 2024-03-27T11:12:26.659 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid adca88c4-ec26-11ee-95d0-87774f69a715 --force --keep-logs 2024-03-27T11:12:26.979 INFO:teuthology.orchestra.run.smithi114.stdout:Deleting cluster with fsid: adca88c4-ec26-11ee-95d0-87774f69a715 2024-03-27T11:12:31.939 DEBUG:teuthology.orchestra.run.smithi177:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid adca88c4-ec26-11ee-95d0-87774f69a715 --force --keep-logs 2024-03-27T11:12:32.260 INFO:teuthology.orchestra.run.smithi177.stdout:Deleting cluster with fsid: adca88c4-ec26-11ee-95d0-87774f69a715 2024-03-27T11:12:36.989 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-27T11:12:37.092 DEBUG:teuthology.orchestra.run.smithi114:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-27T11:12:37.191 DEBUG:teuthology.orchestra.run.smithi177:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-27T11:12:37.295 INFO:tasks.cephadm:Archiving crash dumps... 2024-03-27T11:12:37.297 DEBUG:teuthology.misc:Transferring archived files from smithi097:/var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/crash to /home/teuthworker/archive/rishabh-2024-03-27_05:27:11-fs-wip-rishabh-testing-20240326.131558-testing-default-smithi/7625781/remote/smithi097/crash 2024-03-27T11:12:37.297 DEBUG:teuthology.orchestra.run.smithi097:> sudo tar c -f - -C /var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/crash -- . 2024-03-27T11:12:37.439 INFO:teuthology.orchestra.run.smithi097.stderr:tar: /var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/crash: Cannot open: No such file or directory 2024-03-27T11:12:37.439 INFO:teuthology.orchestra.run.smithi097.stderr:tar: Error is not recoverable: exiting now 2024-03-27T11:12:37.445 DEBUG:teuthology.misc:Transferring archived files from smithi114:/var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/crash to /home/teuthworker/archive/rishabh-2024-03-27_05:27:11-fs-wip-rishabh-testing-20240326.131558-testing-default-smithi/7625781/remote/smithi114/crash 2024-03-27T11:12:37.446 DEBUG:teuthology.orchestra.run.smithi114:> sudo tar c -f - -C /var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/crash -- . 2024-03-27T11:12:37.561 INFO:teuthology.orchestra.run.smithi114.stderr:tar: /var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/crash: Cannot open: No such file or directory 2024-03-27T11:12:37.561 INFO:teuthology.orchestra.run.smithi114.stderr:tar: Error is not recoverable: exiting now 2024-03-27T11:12:37.567 DEBUG:teuthology.misc:Transferring archived files from smithi177:/var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/crash to /home/teuthworker/archive/rishabh-2024-03-27_05:27:11-fs-wip-rishabh-testing-20240326.131558-testing-default-smithi/7625781/remote/smithi177/crash 2024-03-27T11:12:37.568 DEBUG:teuthology.orchestra.run.smithi177:> sudo tar c -f - -C /var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/crash -- . 2024-03-27T11:12:37.673 INFO:teuthology.orchestra.run.smithi177.stderr:tar: /var/lib/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/crash: Cannot open: No such file or directory 2024-03-27T11:12:37.673 INFO:teuthology.orchestra.run.smithi177.stderr:tar: Error is not recoverable: exiting now 2024-03-27T11:12:37.677 INFO:tasks.cephadm:Checking cluster log for badness... 2024-03-27T11:12:37.678 DEBUG:teuthology.orchestra.run.smithi097:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-03-27T11:12:37.845 INFO:teuthology.orchestra.run.smithi097.stdout:2024-03-27T10:58:22.037772+0000 mds.h (mds.1) 2 : cluster [WRN] Scrub error on inode 0x10000000204 (/volumes/qa/sv_0/22bc04cd-8106-4d11-83f1-2aead5019d07/client.0/tmp/iogen_3.1p0/src/iogen.c) see mds.h log and `damage ls` output for details 2024-03-27T11:12:37.846 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-03-27T11:12:37.846 INFO:tasks.cephadm:Compressing logs... 2024-03-27T11:12:37.846 DEBUG:teuthology.orchestra.run.smithi097:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-27T11:12:37.852 DEBUG:teuthology.orchestra.run.smithi114:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-27T11:12:37.859 DEBUG:teuthology.orchestra.run.smithi177:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-27T11:12:37.945 INFO:teuthology.orchestra.run.smithi097.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-03-27T11:12:37.946 INFO:teuthology.orchestra.run.smithi114.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-03-27T11:12:37.946 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-27T11:12:37.947 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph-osd.10.log 2024-03-27T11:12:37.947 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-27T11:12:37.948 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph.audit.log 2024-03-27T11:12:37.948 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.log 2024-03-27T11:12:37.949 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph-mds.b.log 2024-03-27T11:12:37.949 INFO:teuthology.orchestra.run.smithi114.stderr: 92.0% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-27T11:12:37.950 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph-osd.6.log 2024-03-27T11:12:37.950 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph-osd.4.log 2024-03-27T11:12:37.951 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.0.log: gzip -5 --verbose -- /var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph-mon.a.log 2024-03-27T11:12:37.952 INFO:teuthology.orchestra.run.smithi177.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-27T11:12:37.953 INFO:teuthology.orchestra.run.smithi177.stderr:: No such file or directory 2024-03-27T11:12:37.954 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph.audit.log 2024-03-27T11:12:37.954 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph-mds.g.log 2024-03-27T11:12:37.954 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph-osd.1.log 2024-03-27T11:12:37.954 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph-mds.k.log 2024-03-27T11:12:37.954 INFO:teuthology.orchestra.run.smithi114.stderr: 90.4% -- replaced with /var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph.audit.log.gz 2024-03-27T11:12:37.954 INFO:teuthology.orchestra.run.smithi114.stderr:gzip/var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph-osd.1.log: -5 --verbose -- /var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph-mds.h.log 2024-03-27T11:12:37.956 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph-mds.j.log 2024-03-27T11:12:37.956 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph-osd.3.log 2024-03-27T11:12:37.957 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph-mgr.x.log 2024-03-27T11:12:37.957 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph-mds.l.log 2024-03-27T11:12:37.957 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph.audit.log 2024-03-27T11:12:37.957 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph-mds.i.log 2024-03-27T11:12:37.957 INFO:teuthology.orchestra.run.smithi177.stderr: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-27T11:12:37.957 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph-osd.11.log 2024-03-27T11:12:37.957 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph-mgr.z.log 2024-03-27T11:12:37.959 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph-osd.7.log 2024-03-27T11:12:37.959 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph-mds.e.log 2024-03-27T11:12:37.960 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph-osd.0.log 2024-03-27T11:12:37.960 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph.cephadm.log 2024-03-27T11:12:37.960 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph.log 2024-03-27T11:12:37.960 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph-volume.log 2024-03-27T11:12:37.961 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph.log: 94.5% -- replaced with /var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph-mds.j.log.gz 2024-03-27T11:12:37.961 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph-osd.9.log 2024-03-27T11:12:37.964 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph-osd.7.log: /var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph-mgr.y.log 2024-03-27T11:12:37.964 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph-mds.d.log 2024-03-27T11:12:37.964 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph-mds.a.log 2024-03-27T11:12:37.964 INFO:teuthology.orchestra.run.smithi097.stderr: 87.2% -- replaced with /var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph.log.gz 2024-03-27T11:12:37.964 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph-osd.8.log 2024-03-27T11:12:37.965 INFO:teuthology.orchestra.run.smithi177.stderr: 90.4% -- replaced with /var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph.audit.log.gz 2024-03-27T11:12:37.965 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph-osd.5.log 2024-03-27T11:12:37.965 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph-mds.f.log 2024-03-27T11:12:37.965 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph.cephadm.log 2024-03-27T11:12:37.966 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph.cephadm.log 2024-03-27T11:12:37.967 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph-mds.f.log: /var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph.log 2024-03-27T11:12:37.967 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph-mds.d.log: /var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph-mds.a.log: 94.6% -- replaced with /var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph-mds.d.log.gz 2024-03-27T11:12:37.968 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph.log 2024-03-27T11:12:37.969 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph.log: 82.3% -- replaced with /var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph.cephadm.log.gz 2024-03-27T11:12:37.970 INFO:teuthology.orchestra.run.smithi177.stderr: 93.1% -- replaced with /var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph-mgr.z.log.gz 2024-03-27T11:12:37.970 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph-osd.2.log 2024-03-27T11:12:37.971 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph-mds.c.log 2024-03-27T11:12:37.972 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph.cephadm.log: 82.4% -- replaced with /var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph.cephadm.log.gz 2024-03-27T11:12:37.972 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph-volume.log 2024-03-27T11:12:37.976 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph-mds.c.log: /var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph-volume.log 2024-03-27T11:12:37.977 INFO:teuthology.orchestra.run.smithi177.stderr: 87.2% -- replaced with /var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph.log.gz 2024-03-27T11:12:37.979 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph.log: /var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph-mon.b.log 2024-03-27T11:12:37.980 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph-mon.c.log 2024-03-27T11:12:37.981 INFO:teuthology.orchestra.run.smithi114.stderr: 87.2% -- replaced with /var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph.log.gz 2024-03-27T11:12:37.986 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph-mon.b.log: 93.0% -- replaced with /var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph-mgr.y.log.gz 2024-03-27T11:12:38.011 INFO:teuthology.orchestra.run.smithi114.stderr: 90.8% -- replaced with /var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph-volume.log.gz 2024-03-27T11:12:38.015 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph-mon.c.log: 90.8% -- replaced with /var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph-volume.log.gz 2024-03-27T11:12:38.020 INFO:teuthology.orchestra.run.smithi177.stderr: 93.7% -- replaced with /var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph-mds.l.log.gz 2024-03-27T11:12:38.020 INFO:teuthology.orchestra.run.smithi097.stderr: 90.2% -- replaced with /var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph.audit.log.gz 2024-03-27T11:12:38.029 INFO:teuthology.orchestra.run.smithi177.stderr: 93.7% -- replaced with /var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph-mds.f.log.gz 2024-03-27T11:12:38.029 INFO:teuthology.orchestra.run.smithi177.stderr: 94.8% -- replaced with /var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph-mds.i.log.gz 2024-03-27T11:12:38.035 INFO:teuthology.orchestra.run.smithi114.stderr: 93.5% -- replaced with /var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph-mds.h.log.gz 2024-03-27T11:12:38.039 INFO:teuthology.orchestra.run.smithi114.stderr: 93.6% -- replaced with /var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph-mds.e.log.gz 2024-03-27T11:12:38.041 INFO:teuthology.orchestra.run.smithi097.stderr: 94.9% -- replaced with /var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph-mds.a.log.gz 2024-03-27T11:12:38.064 INFO:teuthology.orchestra.run.smithi177.stderr: 95.0% -- replaced with /var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph-mds.c.log.gz 2024-03-27T11:12:38.065 INFO:teuthology.orchestra.run.smithi114.stderr: 93.8% -- replaced with /var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph-mds.k.log.gz 2024-03-27T11:12:38.162 INFO:teuthology.orchestra.run.smithi097.stderr: 94.9% -- replaced with /var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph-mds.g.log.gz 2024-03-27T11:12:38.393 INFO:teuthology.orchestra.run.smithi114.stderr: 92.8% -- replaced with /var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph-mds.b.log.gz 2024-03-27T11:12:38.486 INFO:teuthology.orchestra.run.smithi177.stderr: 91.8% -- replaced with /var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph-mon.c.log.gz 2024-03-27T11:12:38.494 INFO:teuthology.orchestra.run.smithi114.stderr: 91.8% -- replaced with /var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph-mon.b.log.gz 2024-03-27T11:12:38.570 INFO:teuthology.orchestra.run.smithi097.stderr: 93.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-27T11:12:38.590 INFO:teuthology.orchestra.run.smithi097.stderr: 82.7% -- replaced with /var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph.cephadm.log.gz 2024-03-27T11:12:38.751 INFO:teuthology.orchestra.run.smithi097.stderr: 90.2% -- replaced with /var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph-mgr.x.log.gz 2024-03-27T11:12:38.790 INFO:teuthology.orchestra.run.smithi097.stderr: 90.8% -- replaced with /var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph-volume.log.gz 2024-03-27T11:12:41.945 INFO:teuthology.orchestra.run.smithi097.stderr: 90.3% -- replaced with /var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph-mon.a.log.gz 2024-03-27T11:12:49.254 INFO:teuthology.orchestra.run.smithi114.stderr: 93.6% -- replaced with /var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph-osd.4.log.gz 2024-03-27T11:12:50.169 INFO:teuthology.orchestra.run.smithi177.stderr: 93.4% -- replaced with /var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph-osd.8.log.gz 2024-03-27T11:12:50.941 INFO:teuthology.orchestra.run.smithi177.stderr: 93.4% -- replaced with /var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph-osd.2.log.gz 2024-03-27T11:12:53.033 INFO:teuthology.orchestra.run.smithi114.stderr: 93.4% -- replaced with /var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph-osd.10.log.gz 2024-03-27T11:12:53.107 INFO:teuthology.orchestra.run.smithi177.stderr: 93.4% -- replaced with /var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph-osd.5.log.gz 2024-03-27T11:12:55.560 INFO:teuthology.orchestra.run.smithi114.stderr: 93.5% -- replaced with /var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph-osd.1.log.gz 2024-03-27T11:12:55.868 INFO:teuthology.orchestra.run.smithi177.stderr: 93.6% -- replaced with /var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph-osd.11.log.gz 2024-03-27T11:12:55.872 INFO:teuthology.orchestra.run.smithi177.stderr: 2024-03-27T11:12:55.872 INFO:teuthology.orchestra.run.smithi177.stderr:real 0m17.959s 2024-03-27T11:12:55.872 INFO:teuthology.orchestra.run.smithi177.stderr:user 0m54.228s 2024-03-27T11:12:55.872 INFO:teuthology.orchestra.run.smithi177.stderr:sys 0m4.734s 2024-03-27T11:12:55.924 INFO:teuthology.orchestra.run.smithi114.stderr: 93.6% -- replaced with /var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph-osd.7.log.gz 2024-03-27T11:12:55.929 INFO:teuthology.orchestra.run.smithi114.stderr: 2024-03-27T11:12:55.929 INFO:teuthology.orchestra.run.smithi114.stderr:real 0m18.024s 2024-03-27T11:12:55.929 INFO:teuthology.orchestra.run.smithi114.stderr:user 0m58.383s 2024-03-27T11:12:55.929 INFO:teuthology.orchestra.run.smithi114.stderr:sys 0m4.851s 2024-03-27T11:13:27.241 INFO:teuthology.orchestra.run.smithi097.stderr: 93.4% -- replaced with /var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph-osd.9.log.gz 2024-03-27T11:13:45.418 INFO:teuthology.orchestra.run.smithi097.stderr: 93.4% -- replaced with /var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph-osd.3.log.gz 2024-03-27T11:13:49.032 INFO:teuthology.orchestra.run.smithi097.stderr: 93.5% -- replaced with /var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph-osd.0.log.gz 2024-03-27T11:13:49.312 INFO:teuthology.orchestra.run.smithi097.stderr: 93.4% -- replaced with /var/log/ceph/adca88c4-ec26-11ee-95d0-87774f69a715/ceph-osd.6.log.gz 2024-03-27T11:15:37.122 INFO:teuthology.orchestra.run.smithi097.stderr: 94.2% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2024-03-27T11:15:37.125 INFO:teuthology.orchestra.run.smithi097.stderr: 2024-03-27T11:15:37.125 INFO:teuthology.orchestra.run.smithi097.stderr:real 2m59.234s 2024-03-27T11:15:37.125 INFO:teuthology.orchestra.run.smithi097.stderr:user 3m33.937s 2024-03-27T11:15:37.125 INFO:teuthology.orchestra.run.smithi097.stderr:sys 0m25.339s 2024-03-27T11:15:37.127 INFO:tasks.cephadm:Archiving logs... 2024-03-27T11:15:37.128 DEBUG:teuthology.misc:Transferring archived files from smithi097:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-03-27_05:27:11-fs-wip-rishabh-testing-20240326.131558-testing-default-smithi/7625781/remote/smithi097/log 2024-03-27T11:15:37.129 DEBUG:teuthology.orchestra.run.smithi097:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-27T11:16:02.888 DEBUG:teuthology.misc:Transferring archived files from smithi114:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-03-27_05:27:11-fs-wip-rishabh-testing-20240326.131558-testing-default-smithi/7625781/remote/smithi114/log 2024-03-27T11:16:02.890 DEBUG:teuthology.orchestra.run.smithi114:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-27T11:16:13.130 DEBUG:teuthology.misc:Transferring archived files from smithi177:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-03-27_05:27:11-fs-wip-rishabh-testing-20240326.131558-testing-default-smithi/7625781/remote/smithi177/log 2024-03-27T11:16:13.132 DEBUG:teuthology.orchestra.run.smithi177:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-27T11:16:21.607 INFO:tasks.cephadm:Removing cluster... 2024-03-27T11:16:21.608 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid adca88c4-ec26-11ee-95d0-87774f69a715 --force 2024-03-27T11:16:21.936 INFO:teuthology.orchestra.run.smithi097.stdout:Deleting cluster with fsid: adca88c4-ec26-11ee-95d0-87774f69a715 2024-03-27T11:16:22.733 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid adca88c4-ec26-11ee-95d0-87774f69a715 --force 2024-03-27T11:16:23.056 INFO:teuthology.orchestra.run.smithi114.stdout:Deleting cluster with fsid: adca88c4-ec26-11ee-95d0-87774f69a715 2024-03-27T11:16:23.832 DEBUG:teuthology.orchestra.run.smithi177:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid adca88c4-ec26-11ee-95d0-87774f69a715 --force 2024-03-27T11:16:24.162 INFO:teuthology.orchestra.run.smithi177.stdout:Deleting cluster with fsid: adca88c4-ec26-11ee-95d0-87774f69a715 2024-03-27T11:16:24.948 INFO:tasks.cephadm:Removing cephadm ... 2024-03-27T11:16:24.949 DEBUG:teuthology.orchestra.run.smithi097:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-27T11:16:25.005 DEBUG:teuthology.orchestra.run.smithi114:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-27T11:16:25.061 DEBUG:teuthology.orchestra.run.smithi177:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-27T11:16:25.116 INFO:tasks.cephadm:Teardown complete 2024-03-27T11:16:25.116 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-03-27T11:16:25.130 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/task/install/__init__.py", line 626, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_155268c4e432a12433aa833f174f9fe3b1016ae0/qa/tasks/fwd_scrub.py", line 164, in task stop_all_fwd_scrubbers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_155268c4e432a12433aa833f174f9fe3b1016ae0/qa/tasks/fwd_scrub.py", line 99, in stop_all_fwd_scrubbers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: rank damage found: {'backtrace'} 2024-03-27T11:16:25.131 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-03-27T11:16:25.131 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-27T11:16:25.138 DEBUG:teuthology.orchestra.run.smithi114:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-27T11:16:25.146 DEBUG:teuthology.orchestra.run.smithi177:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-27T11:16:25.310 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-27T11:16:25.321 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-27T11:16:25.321 DEBUG:teuthology.orchestra.run.smithi097:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-27T11:16:25.329 DEBUG:teuthology.orchestra.run.smithi114:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-27T11:16:25.337 DEBUG:teuthology.orchestra.run.smithi177:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-27T11:16:25.372 INFO:teuthology.orchestra.run.smithi177.stderr:bash: line 1: ntpq: command not found 2024-03-27T11:16:25.374 INFO:teuthology.orchestra.run.smithi097.stderr:bash: line 1: ntpq: command not found 2024-03-27T11:16:25.378 INFO:teuthology.orchestra.run.smithi177.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-27T11:16:25.379 INFO:teuthology.orchestra.run.smithi177.stdout:=============================================================================== 2024-03-27T11:16:25.379 INFO:teuthology.orchestra.run.smithi177.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 52 +569us[ +569us] +/- 91ms 2024-03-27T11:16:25.379 INFO:teuthology.orchestra.run.smithi177.stdout:^* hv02.front.sepia.ceph.com 3 6 377 53 -433us[ -369us] +/- 57ms 2024-03-27T11:16:25.379 INFO:teuthology.orchestra.run.smithi177.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 52 +259us[ +259us] +/- 74ms 2024-03-27T11:16:25.379 INFO:teuthology.orchestra.run.smithi177.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-27T11:16:25.381 INFO:teuthology.orchestra.run.smithi114.stderr:bash: line 1: ntpq: command not found 2024-03-27T11:16:25.384 INFO:teuthology.orchestra.run.smithi097.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-27T11:16:25.384 INFO:teuthology.orchestra.run.smithi097.stdout:=============================================================================== 2024-03-27T11:16:25.384 INFO:teuthology.orchestra.run.smithi097.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 54 +533us[ +533us] +/- 91ms 2024-03-27T11:16:25.384 INFO:teuthology.orchestra.run.smithi097.stdout:^* hv02.front.sepia.ceph.com 3 6 377 58 -447us[ -411us] +/- 57ms 2024-03-27T11:16:25.384 INFO:teuthology.orchestra.run.smithi097.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 48 +241us[ +241us] +/- 74ms 2024-03-27T11:16:25.384 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-27T11:16:25.392 INFO:teuthology.orchestra.run.smithi114.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-27T11:16:25.392 INFO:teuthology.orchestra.run.smithi114.stdout:=============================================================================== 2024-03-27T11:16:25.392 INFO:teuthology.orchestra.run.smithi114.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 54 +539us[ +563us] +/- 91ms 2024-03-27T11:16:25.392 INFO:teuthology.orchestra.run.smithi114.stdout:^* hv02.front.sepia.ceph.com 3 6 377 54 -475us[ -450us] +/- 57ms 2024-03-27T11:16:25.392 INFO:teuthology.orchestra.run.smithi114.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 53 +215us[ +215us] +/- 74ms 2024-03-27T11:16:25.392 INFO:teuthology.orchestra.run.smithi114.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-27T11:16:25.393 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-27T11:16:25.403 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-27T11:16:25.404 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-27T11:16:25.416 DEBUG:teuthology.orchestra.run.smithi097:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-27T11:16:25.679 DEBUG:teuthology.orchestra.run.smithi114:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-27T11:16:25.898 DEBUG:teuthology.orchestra.run.smithi177:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-27T11:16:26.099 DEBUG:teuthology.orchestra.run.smithi097:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-27T11:16:26.202 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-27T11:16:26.203 DEBUG:teuthology.orchestra.run.smithi114:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-27T11:16:26.302 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-27T11:16:26.303 DEBUG:teuthology.orchestra.run.smithi177:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-27T11:16:26.406 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-27T11:16:26.407 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-27T11:16:26.418 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-27T11:16:26.432 INFO:teuthology.task.internal:Duration was 2588.209278 seconds 2024-03-27T11:16:26.432 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-27T11:16:26.447 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-27T11:16:26.447 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-27T11:16:26.455 DEBUG:teuthology.orchestra.run.smithi114:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-27T11:16:26.463 DEBUG:teuthology.orchestra.run.smithi177:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-27T11:16:26.621 INFO:teuthology.orchestra.run.smithi114.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-27T11:16:26.624 INFO:teuthology.orchestra.run.smithi177.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-27T11:16:26.649 INFO:teuthology.orchestra.run.smithi097.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-27T11:16:27.225 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-27T11:16:27.225 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi097.front.sepia.ceph.com 2024-03-27T11:16:27.225 DEBUG:teuthology.orchestra.run.smithi097:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-27T11:16:27.283 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi114.front.sepia.ceph.com 2024-03-27T11:16:27.286 DEBUG:teuthology.orchestra.run.smithi114:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-27T11:16:27.353 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi177.front.sepia.ceph.com 2024-03-27T11:16:27.353 DEBUG:teuthology.orchestra.run.smithi177:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-27T11:16:27.427 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-27T11:16:27.427 DEBUG:teuthology.orchestra.run.smithi097:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-27T11:16:27.432 DEBUG:teuthology.orchestra.run.smithi114:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-27T11:16:27.438 DEBUG:teuthology.orchestra.run.smithi177:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-27T11:16:27.522 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-27T11:16:27.523 DEBUG:teuthology.orchestra.run.smithi097:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-27T11:16:27.527 DEBUG:teuthology.orchestra.run.smithi114:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-27T11:16:27.531 DEBUG:teuthology.orchestra.run.smithi177:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-27T11:16:28.030 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-27T11:16:28.041 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-27T11:16:28.042 DEBUG:teuthology.orchestra.run.smithi097:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-27T11:16:28.155 DEBUG:teuthology.orchestra.run.smithi114:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-27T11:16:28.246 DEBUG:teuthology.orchestra.run.smithi177:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-27T11:16:28.346 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-27T11:16:28.357 DEBUG:teuthology.orchestra.run.smithi097:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-27T11:16:28.365 DEBUG:teuthology.orchestra.run.smithi114:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-27T11:16:28.373 DEBUG:teuthology.orchestra.run.smithi177:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-27T11:16:28.476 INFO:teuthology.orchestra.run.smithi114.stdout:kernel.core_pattern = core 2024-03-27T11:16:28.480 INFO:teuthology.orchestra.run.smithi097.stdout:kernel.core_pattern = core 2024-03-27T11:16:28.483 INFO:teuthology.orchestra.run.smithi177.stdout:kernel.core_pattern = core 2024-03-27T11:16:28.559 DEBUG:teuthology.orchestra.run.smithi097:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-27T11:16:28.611 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-27T11:16:28.611 DEBUG:teuthology.orchestra.run.smithi114:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-27T11:16:28.664 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-27T11:16:28.665 DEBUG:teuthology.orchestra.run.smithi177:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-27T11:16:28.719 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-27T11:16:28.720 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-27T11:16:28.732 INFO:teuthology.task.internal:Transferring archived files... 2024-03-27T11:16:28.733 DEBUG:teuthology.misc:Transferring archived files from smithi097:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-03-27_05:27:11-fs-wip-rishabh-testing-20240326.131558-testing-default-smithi/7625781/remote/smithi097 2024-03-27T11:16:28.734 DEBUG:teuthology.orchestra.run.smithi097:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-27T11:16:28.850 DEBUG:teuthology.misc:Transferring archived files from smithi114:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-03-27_05:27:11-fs-wip-rishabh-testing-20240326.131558-testing-default-smithi/7625781/remote/smithi114 2024-03-27T11:16:28.851 DEBUG:teuthology.orchestra.run.smithi114:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-27T11:16:28.961 DEBUG:teuthology.misc:Transferring archived files from smithi177:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-03-27_05:27:11-fs-wip-rishabh-testing-20240326.131558-testing-default-smithi/7625781/remote/smithi177 2024-03-27T11:16:28.961 DEBUG:teuthology.orchestra.run.smithi177:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-27T11:16:29.071 INFO:teuthology.task.internal:Removing archive directory... 2024-03-27T11:16:29.071 DEBUG:teuthology.orchestra.run.smithi097:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-27T11:16:29.079 DEBUG:teuthology.orchestra.run.smithi114:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-27T11:16:29.087 DEBUG:teuthology.orchestra.run.smithi177:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-27T11:16:29.165 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-27T11:16:29.179 INFO:teuthology.task.internal:Not uploading archives. 2024-03-27T11:16:29.179 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-27T11:16:29.193 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-27T11:16:29.193 DEBUG:teuthology.orchestra.run.smithi097:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-27T11:16:29.201 DEBUG:teuthology.orchestra.run.smithi114:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-27T11:16:29.209 DEBUG:teuthology.orchestra.run.smithi177:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-27T11:16:29.251 INFO:teuthology.orchestra.run.smithi097.stdout: 395555 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 27 11:16 /home/ubuntu/cephtest 2024-03-27T11:16:29.253 INFO:teuthology.orchestra.run.smithi114.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 27 11:16 /home/ubuntu/cephtest 2024-03-27T11:16:29.267 INFO:teuthology.orchestra.run.smithi177.stdout: 393638 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 27 11:16 /home/ubuntu/cephtest 2024-03-27T11:16:29.270 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-27T11:16:29.282 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-27T11:16:29.297 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: RuntimeError: error during scrub thrashing: rank damage found: {'backtrace'} 2024-03-27T11:16:29.297 INFO:teuthology.nuke:Checking targets against current locks 2024-03-27T11:16:29.323 DEBUG:teuthology.nuke:shortname: smithi097 2024-03-27T11:16:29.324 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-27T11:16:29.351 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi097.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-03-27_05:27:11-fs-wip-rishabh-testing-20240326.131558-testing-default-smithi/7625781', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-27 10:24:05.053643', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNlLz08IArfvIjetErgvQ+5YKaLSe8NPzl4LVgM6pSqZOod3t3W3hh5h6dQ+in5EMm6YNDwtMDLymdBFBgPG6uA='} 2024-03-27T11:16:29.355 DEBUG:teuthology.nuke:shortname: smithi114 2024-03-27T11:16:29.355 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-27T11:16:29.384 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi114.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-03-27_05:27:11-fs-wip-rishabh-testing-20240326.131558-testing-default-smithi/7625781', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-27 10:24:05.050943', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKCZcHiv9DXJnWWjv/t9fR+w9U+PfTdXPwrECaQtupN73Qxfc/JcEfWcTUNP8swunI9TpqdmVDfG7D3A2/EP31M='} 2024-03-27T11:16:29.385 DEBUG:teuthology.nuke:shortname: smithi177 2024-03-27T11:16:29.385 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-27T11:16:29.411 INFO:teuthology.orchestra.console.smithi097:Power off 2024-03-27T11:16:29.412 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-27T11:16:29.425 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi177.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-03-27_05:27:11-fs-wip-rishabh-testing-20240326.131558-testing-default-smithi/7625781', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-27 10:24:05.052479', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJwXtJQVOIeDN2PKjI9hI9Oa4+xqJ7yG+AvSuY1u9Xl/rYTP4QQUptvjx9OC2DqSRqOojIU4egKICDfXbjLAC6A='} 2024-03-27T11:16:29.437 DEBUG:teuthology.orchestra.console.smithi097:power off output: Chassis Power Control: Down/Off 2024-03-27T11:16:29.438 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-27T11:16:29.464 INFO:teuthology.orchestra.console.smithi114:Power off 2024-03-27T11:16:29.465 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-27T11:16:29.474 DEBUG:teuthology.orchestra.console.smithi097:check power output: Chassis Power is on 2024-03-27T11:16:29.486 DEBUG:teuthology.orchestra.console.smithi114:power off output: Chassis Power Control: Down/Off 2024-03-27T11:16:29.486 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-27T11:16:29.500 INFO:teuthology.orchestra.console.smithi177:Power off 2024-03-27T11:16:29.500 DEBUG:teuthology.orchestra.console.smithi177:pexpect command: ipmitool -H smithi177.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-27T11:16:29.509 DEBUG:teuthology.orchestra.console.smithi114:check power output: Chassis Power is on 2024-03-27T11:16:29.524 DEBUG:teuthology.orchestra.console.smithi177:power off output: Chassis Power Control: Down/Off 2024-03-27T11:16:29.524 DEBUG:teuthology.orchestra.console.smithi177:pexpect command: ipmitool -H smithi177.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-27T11:16:29.549 DEBUG:teuthology.orchestra.console.smithi177:check power output: Chassis Power is on 2024-03-27T11:16:33.475 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-27T11:16:33.511 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-27T11:16:33.550 DEBUG:teuthology.orchestra.console.smithi177:pexpect command: ipmitool -H smithi177.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-27T11:16:33.586 DEBUG:teuthology.orchestra.console.smithi097:check power output: Chassis Power is on 2024-03-27T11:16:33.623 DEBUG:teuthology.orchestra.console.smithi114:check power output: Chassis Power is on 2024-03-27T11:16:33.660 DEBUG:teuthology.orchestra.console.smithi177:check power output: Chassis Power is on 2024-03-27T11:16:37.589 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-27T11:16:37.624 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-27T11:16:37.661 DEBUG:teuthology.orchestra.console.smithi177:pexpect command: ipmitool -H smithi177.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-27T11:16:37.700 DEBUG:teuthology.orchestra.console.smithi097:check power output: Chassis Power is on 2024-03-27T11:16:37.735 DEBUG:teuthology.orchestra.console.smithi114:check power output: Chassis Power is on 2024-03-27T11:16:37.772 DEBUG:teuthology.orchestra.console.smithi177:check power output: Chassis Power is off 2024-03-27T11:16:37.872 INFO:teuthology.orchestra.console.smithi177:Power off completed 2024-03-27T11:16:41.702 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-27T11:16:41.736 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-27T11:16:41.815 DEBUG:teuthology.orchestra.console.smithi097:check power output: Chassis Power is off 2024-03-27T11:16:41.847 DEBUG:teuthology.orchestra.console.smithi114:check power output: Chassis Power is off 2024-03-27T11:16:41.916 INFO:teuthology.orchestra.console.smithi097:Power off completed 2024-03-27T11:16:41.949 INFO:teuthology.orchestra.console.smithi114:Power off completed 2024-03-27T11:16:42.054 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/suites/iogen}} duration: 2588.209278345108 failure_reason: 'error during scrub thrashing: rank damage found: {''backtrace''}' flavor: default mon.a-kernel-sha1: 1df0c7e8a2354c42d738e5454ba83ea7332deac2 mon.b-kernel-sha1: 1df0c7e8a2354c42d738e5454ba83ea7332deac2 mon.c-kernel-sha1: 1df0c7e8a2354c42d738e5454ba83ea7332deac2 owner: scheduled_rishabh@teuthology status: fail success: false 2024-03-27T11:16:42.055 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-27T11:16:42.139 INFO:teuthology.run:FAIL